[ { "path": ".dockerignore", "content": ".go/\n.go.std/\nsite/\n" }, { "path": ".github/ISSUE_TEMPLATE/bug_report.md", "content": "---\nname: Bug report\nabout: Tell us about a problem you are experiencing\n\n---\n\n**What steps did you take and what happened:**\n\n\n\n**What did you expect to happen:**\n\n**The following information will help us better understand what's going on**:\n\n_If you are using velero v1.7.0+:_ \nPlease use `velero debug --backup --restore ` to generate the support bundle, and attach to this issue, more options please refer to `velero debug --help` \n\n_If you are using earlier versions:_ \nPlease provide the output of the following commands (Pasting long output into a [GitHub gist](https://gist.github.com) or other pastebin is fine.)\n- `kubectl logs deployment/velero -n velero`\n- `velero backup describe ` or `kubectl get backup/ -n velero -o yaml`\n- `velero backup logs `\n- `velero restore describe ` or `kubectl get restore/ -n velero -o yaml`\n- `velero restore logs `\n\n\n**Anything else you would like to add:**\n\n\n\n**Environment:**\n\n- Velero version (use `velero version`): \n- Velero features (use `velero client config get features`): \n- Kubernetes version (use `kubectl version`):\n- Kubernetes installer & version:\n- Cloud provider or hardware configuration:\n- OS (e.g. from `/etc/os-release`):\n\n\n**Vote on this issue!**\n\nThis is an invitation to the Velero community to vote on issues, you can see the project's [top voted issues listed here](https://github.com/vmware-tanzu/velero/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc). \nUse the \"reaction smiley face\" up to the right of this comment to vote.\n\n- :+1: for \"I would like to see this bug fixed as soon as possible\"\n- :-1: for \"There are more important bugs to focus on right now\"\n" }, { "path": ".github/ISSUE_TEMPLATE/config.yml", "content": "blank_issues_enabled: false\ncontact_links:\n - name: Velero Q&A\n url: https://github.com/vmware-tanzu/velero/discussions/categories/community-support-q-a \n about: Have questions about Velero? Please ask them here.\n" }, { "path": ".github/ISSUE_TEMPLATE/feature-enhancement-request.md", "content": "---\nname: Feature enhancement request\nabout: Suggest an idea for this project\n\n---\n\n**Describe the problem/challenge you have**\n\n\n\n**Describe the solution you'd like**\n\n\n\n**Anything else you would like to add:**\n\n\n\n**Environment:**\n\n- Velero version (use `velero version`):\n- Kubernetes version (use `kubectl version`):\n- Kubernetes installer & version:\n- Cloud provider or hardware configuration:\n- OS (e.g. from `/etc/os-release`):\n\n**Vote on this issue!**\n\nThis is an invitation to the Velero community to vote on issues, you can see the project's [top voted issues listed here](https://github.com/vmware-tanzu/velero/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc). \nUse the \"reaction smiley face\" up to the right of this comment to vote.\n\n- :+1: for \"The project would be better with this feature added\"\n- :-1: for \"This feature will not enhance the project in a meaningful way\"\n" }, { "path": ".github/auto-assignees.yml", "content": "---\n# This assigns a PR to its author\naddAssignees: author\n\nreviewers:\n # The default reviewers\n defaults:\n - maintainers\n\n groups:\n maintainers:\n - sseago\n - reasonerjt\n - ywk253100\n - blackpiglet\n - shubham-pampattiwar\n - Lyndon-Li\n - anshulahuja98\n - kaovilai\n\n tech-writer:\n - sseago\n - reasonerjt\n - ywk253100\n - Lyndon-Li\n\nfiles:\n 'site/**':\n - tech-writer\n '**/*.md':\n - tech-writer\n # Technical design requests are \".md\" files but should\n # be reviewed by maintainers\n '/design/**':\n - maintainers\n\noptions:\n ignore_draft: true\n ignored_keywords:\n - WIP\n - wip\n - DO NOT MERGE\n enable_group_assignment: true\n number_of_reviewers: 2\n" }, { "path": ".github/dependabot.yml", "content": "version: 2\nupdates:\n # Dependencies listed in .github/workflows\n - package-ecosystem: \"github-actions\"\n directory: \"/\"\n schedule:\n interval: \"weekly\"\n labels:\n - \"Dependencies\"\n - \"github_actions\"\n - \"kind/changelog-not-required\"\n # Dependencies listed in go.mod\n - package-ecosystem: \"gomod\"\n directory: \"/\" # Location of package manifests\n schedule:\n interval: \"weekly\"\n labels:\n - \"kind/changelog-not-required\"\n ignore:\n - dependency-name: \"*\"\n update-types: [\"version-update:semver-major\", \"version-update:semver-minor\", \"version-update:semver-patch\"]\n" }, { "path": ".github/labeler.yml", "content": "# This file is used by Auto Label PRs action.\n# Works with https://github.com/actions/labeler/\n# Below this line, the keys are labels to be applied, and the values are the file globs to match against.\n# Anything in the `design` directory gets the `Design` label.\nArea/Design:\n - changed-files:\n - any-glob-to-any-file: design/*\n# Anything that has plugin infra will be labeled.\n# Individual plugins don't necessarily live here, though\nArea/Plugins:\n - changed-files:\n - any-glob-to-any-file: pkg/plugins/**/*\nDependencies:\n - changed-files:\n - any-glob-to-any-file: go.mod\nDocumentation:\n - changed-files:\n - any-glob-to-any-file: site/content/docs/**/*\n# Anything in the site directory gets the website label *EXCEPT* docs\nWebsite:\n - all:\n - changed-files:\n - any-glob-to-any-file: site/**/*\n - all-globs-to-all-files: '!site/content/docs/**/*'\nhas-changelog:\n - changed-files:\n - any-glob-to-any-file: changelogs/**\nhas-e2e-2tests:\n - changed-files:\n - any-glob-to-any-file: test/e2e/**/*\nhas-unit-tests:\n - changed-files:\n - any-glob-to-any-file: pkg/**/*_test.go\n" }, { "path": ".github/labels.yaml", "content": "# This file is used by [prow github action](https://github.com/jpmcb/prow-github-actions/) in .github/workflows/prow-action.yml.\n# This file only has values for kind and area commands.\narea:\n - CLI\n - CSI\n - Cloud/AWS\n - Cloud/Azure\n - Cloud/DigitalOcean\n - Cloud/GCP\n - Cloud/vSphere\n - Design\n - Documentation\n - Filters\n - Plugins\n - Process\n - Storage/Minio\n - Storage/Cinder\n - WindowsSupport\n - datamover\n - fs-backup\n - fs-backup/deletion\n - fs-backup/file-selectable\n - fs-uploader\n - kopia-integration\n - migration\n - multi-tenancy\n - progress-monitoring\n - resilience\n - schedule\n - storage/IBM-ObjectStorage\n - upgrade\n - volume-snapshot-dm\nkind:\n - changelog-not-required\n - question\n - refactor\n - requirement\n - release-note\n - release-blocker\n - spike\n - tech-debt\n - usage-error\n - voting\n" }, { "path": ".github/pull_request_template.md", "content": "Thank you for contributing to Velero!\n\n# Please add a summary of your change\n\n# Does your change fix a particular issue?\n\nFixes #(issue)\n\n# Please indicate you've done the following:\n\n- [ ] [Accepted the DCO](https://velero.io/docs/v1.5/code-standards/#dco-sign-off). Commits without the DCO will delay acceptance.\n- [ ] [Created a changelog file (`make new-changelog`)](https://velero.io/docs/main/code-standards/#adding-a-changelog) or comment `/kind changelog-not-required` on this PR.\n- [ ] Updated the corresponding documentation in `site/content/docs/main`.\n" }, { "path": ".github/workflows/auto_assign_prs.yml", "content": "---\nname: \"Auto Assign Author\"\n\n# pull_request_target means that this will run on pull requests, but in\n# the context of the base repo. This should mean PRs from forks are supported.\non:\n pull_request_target:\n types: [opened, reopened, ready_for_review]\n\njobs:\n # Automatically assigns reviewers and owner\n add-reviews:\n runs-on: ubuntu-latest\n steps:\n - name: Set the author of a PR as the assignee\n uses: kentaro-m/auto-assign-action@v2.0.0\n with:\n configuration-path: \".github/auto-assignees.yml\"\n repo-token: \"${{ secrets.GITHUB_TOKEN }}\"\n" }, { "path": ".github/workflows/auto_label_prs.yml", "content": "name: \"Auto Label PRs\"\n# pull_request_target means that this will run on pull requests, but in the context of the base repo.\n# This should mean PRs from forks are supported. \n# Because it includes the `synchronize` parameter, any push of a new commit to the HEAD ref of a pull request\n# will trigger this process.\n\non:\n pull_request_target:\n types: [opened, reopened, synchronize, ready_for_review]\n\njobs:\n # Automatically labels PRs based on file globs in the change.\n triage:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/labeler@v5\n with:\n repo-token: \"${{ secrets.GITHUB_TOKEN }}\"\n configuration-path: .github/labeler.yml\n" }, { "path": ".github/workflows/auto_request_review.yml", "content": "---\nname: \"Auto Request Review\"\n\non:\n pull_request_target:\n types: [opened, ready_for_review, reopened]\n\njobs:\n auto-request-review:\n name: Auto Request Review\n runs-on: ubuntu-latest\n steps:\n - name: Request a PR review based on files types/paths, and/or groups the author belongs to\n uses: necojackarc/auto-request-review@v0.13.0\n with:\n token: ${{ secrets.GITHUB_TOKEN }}\n config: .github/auto-assignees.yml\n" }, { "path": ".github/workflows/e2e-test-kind.yaml", "content": "name: \"Run the E2E test on kind\"\non:\n push:\n pull_request:\n # Do not run when the change only includes these directories.\n paths-ignore:\n - \"site/**\"\n - \"design/**\"\n - \"**/*.md\"\njobs:\n get-go-version:\n uses: ./.github/workflows/get-go-version.yaml\n with:\n ref: ${{ github.event.pull_request.base.ref }}\n\n # Build the Velero CLI and image once for all Kubernetes versions, and cache it so the fan-out workers can get it.\n build:\n runs-on: ubuntu-latest\n needs: get-go-version\n outputs:\n minio-dockerfile-sha: ${{ steps.minio-version.outputs.dockerfile_sha }}\n steps:\n - name: Check out the code\n uses: actions/checkout@v6\n \n - name: Set up Go version\n uses: actions/setup-go@v6\n with:\n go-version: ${{ needs.get-go-version.outputs.version }}\n\n # Look for a CLI that's made for this PR\n - name: Fetch built CLI\n id: cli-cache\n uses: actions/cache@v4\n with:\n path: ./_output/bin/linux/amd64/velero\n # The cache key a combination of the current PR number and the commit SHA\n key: velero-cli-${{ github.event.pull_request.number }}-${{ github.sha }}\n - name: Fetch built image\n id: image-cache\n uses: actions/cache@v4\n with:\n path: ./velero.tar\n # The cache key a combination of the current PR number and the commit SHA\n key: velero-image-${{ github.event.pull_request.number }}-${{ github.sha }}\n # If no binaries were built for this PR, build it now.\n - name: Build Velero CLI\n if: steps.cli-cache.outputs.cache-hit != 'true'\n run: |\n make local\n # If no image were built for this PR, build it now.\n - name: Build Velero Image\n if: steps.image-cache.outputs.cache-hit != 'true'\n run: |\n IMAGE=velero VERSION=pr-test BUILD_OUTPUT_TYPE=docker make container\n docker save velero:pr-test-linux-amd64 -o ./velero.tar\n # Check and build MinIO image once for all e2e tests\n - name: Check Bitnami MinIO Dockerfile version\n id: minio-version\n run: |\n DOCKERFILE_SHA=$(curl -s https://api.github.com/repos/bitnami/containers/commits?path=bitnami/minio/2025/debian-12/Dockerfile\\&per_page=1 | jq -r '.[0].sha')\n echo \"dockerfile_sha=${DOCKERFILE_SHA}\" >> $GITHUB_OUTPUT\n - name: Cache MinIO Image\n uses: actions/cache@v4\n id: minio-cache\n with:\n path: ./minio-image.tar\n key: minio-bitnami-${{ steps.minio-version.outputs.dockerfile_sha }}\n - name: Build MinIO Image from Bitnami Dockerfile\n if: steps.minio-cache.outputs.cache-hit != 'true'\n run: |\n echo \"Building MinIO image from Bitnami Dockerfile...\"\n git clone --depth 1 https://github.com/bitnami/containers.git /tmp/bitnami-containers\n cd /tmp/bitnami-containers/bitnami/minio/2025/debian-12\n docker build -t bitnami/minio:local .\n docker save bitnami/minio:local > ${{ github.workspace }}/minio-image.tar\n # Create json of k8s versions to test\n # from guide: https://stackoverflow.com/a/65094398/4590470\n setup-test-matrix:\n runs-on: ubuntu-latest\n env:\n GH_TOKEN: ${{ github.token }}\n outputs:\n matrix: ${{ steps.set-matrix.outputs.matrix }}\n steps:\n - name: Set k8s versions\n id: set-matrix\n # everything excluding older tags. limits needs to be high enough to cover all latest versions\n # and test labels\n # grep -E \"v[1-9]\\.(2[5-9]|[3-9][0-9])\" filters for v1.25 to v9.99\n # and removes older patches of the same minor version\n # awk -F. '{if(!a[$1\".\"$2]++)print $1\".\"$2\".\"$NF}'\n run: |\n echo \"matrix={\\\n \\\"k8s\\\":$(wget -q -O - \"https://hub.docker.com/v2/namespaces/kindest/repositories/node/tags?page_size=50\" | grep -o '\"name\": *\"[^\"]*' | grep -o '[^\"]*$' | grep -v -E \"alpha|beta\" | grep -E \"v[1-9]\\.(2[5-9]|[3-9][0-9])\" | awk -F. '{if(!a[$1\".\"$2]++)print $1\".\"$2\".\"$NF}' | sort -r | sed s/v//g | jq -R -c -s 'split(\"\\n\")[:-1]'),\\\n \\\"labels\\\":[\\\n \\\"Basic && (ClusterResource || NodePort || StorageClass)\\\", \\\n \\\"ResourceFiltering && !Restic\\\", \\\n \\\"ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources\\\", \\\n \\\"(NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)\\\"\\\n ]}\" >> $GITHUB_OUTPUT\n\n # Run E2E test against all Kubernetes versions on kind\n run-e2e-test:\n needs:\n - build\n - setup-test-matrix\n - get-go-version\n runs-on: ubuntu-latest\n strategy:\n matrix: ${{fromJson(needs.setup-test-matrix.outputs.matrix)}}\n fail-fast: false\n steps:\n - name: Check out the code\n uses: actions/checkout@v6\n\n - name: Set up Go version\n uses: actions/setup-go@v6\n with:\n go-version: ${{ needs.get-go-version.outputs.version }}\n\n # Fetch the pre-built MinIO image from the build job\n - name: Fetch built MinIO Image\n uses: actions/cache@v4\n id: minio-cache\n with:\n path: ./minio-image.tar\n key: minio-bitnami-${{ needs.build.outputs.minio-dockerfile-sha }}\n - name: Load MinIO Image\n run: |\n echo \"Loading MinIO image...\"\n docker load < ./minio-image.tar\n - name: Install MinIO\n run: |\n docker run -d --rm -p 9000:9000 -e \"MINIO_ROOT_USER=minio\" -e \"MINIO_ROOT_PASSWORD=minio123\" -e \"MINIO_DEFAULT_BUCKETS=bucket,additional-bucket\" bitnami/minio:local\n - uses: engineerd/setup-kind@v0.6.2\n with:\n skipClusterLogsExport: true\n version: \"v0.27.0\"\n image: \"kindest/node:v${{ matrix.k8s }}\"\n - name: Fetch built CLI\n id: cli-cache\n uses: actions/cache@v4\n with:\n path: ./_output/bin/linux/amd64/velero\n key: velero-cli-${{ github.event.pull_request.number }}-${{ github.sha }}\n - name: Fetch built Image\n id: image-cache\n uses: actions/cache@v4\n with:\n path: ./velero.tar\n key: velero-image-${{ github.event.pull_request.number }}-${{ github.sha }}\n - name: Load Velero Image\n run:\n kind load image-archive velero.tar\n - name: Run E2E test\n run: |\n cat << EOF > /tmp/credential\n [default]\n aws_access_key_id=minio\n aws_secret_access_key=minio123\n EOF\n\n # Match kubectl version to k8s server version\n curl -LO https://dl.k8s.io/release/v${{ matrix.k8s }}/bin/linux/amd64/kubectl\n sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl\n\n git clone https://github.com/vmware-tanzu-experiments/distributed-data-generator.git -b main /tmp/kibishii\n\n GOPATH=~/go \\\n CLOUD_PROVIDER=kind \\\n OBJECT_STORE_PROVIDER=aws \\\n BSL_CONFIG=region=minio,s3ForcePathStyle=\"true\",s3Url=http://$(hostname -i):9000 \\\n CREDS_FILE=/tmp/credential \\\n BSL_BUCKET=bucket \\\n ADDITIONAL_OBJECT_STORE_PROVIDER=aws \\\n ADDITIONAL_BSL_CONFIG=region=minio,s3ForcePathStyle=\"true\",s3Url=http://$(hostname -i):9000 \\\n ADDITIONAL_CREDS_FILE=/tmp/credential \\\n ADDITIONAL_BSL_BUCKET=additional-bucket \\\n VELERO_IMAGE=velero:pr-test-linux-amd64 \\\n PLUGINS=velero/velero-plugin-for-aws:latest \\\n GINKGO_LABELS=\"${{ matrix.labels }}\" \\\n KIBISHII_DIRECTORY=/tmp/kibishii/kubernetes/yaml/ \\\n make -C test/ run-e2e\n timeout-minutes: 30\n - name: Upload debug bundle\n if: ${{ failure() }}\n uses: actions/upload-artifact@v5\n with:\n name: DebugBundle-k8s-${{ matrix.k8s }}-job-${{ strategy.job-index }}\n path: /home/runner/work/velero/velero/test/e2e/debug-bundle*\n" }, { "path": ".github/workflows/get-go-version.yaml", "content": "on:\r\n workflow_call:\r\n inputs:\r\n ref:\r\n description: \"The target branch's ref\"\r\n required: true\r\n type: string\r\n outputs:\r\n version: \r\n description: \"The expected Go version\"\r\n value: ${{ jobs.extract.outputs.version }}\r\n\r\njobs:\r\n extract:\r\n runs-on: ubuntu-latest\r\n outputs:\r\n version: ${{ steps.pick-version.outputs.version }}\r\n steps:\r\n - name: Check out the code\r\n uses: actions/checkout@v6\r\n\r\n - id: pick-version\r\n run: |\r\n if [ \"${{ inputs.ref }}\" == \"main\" ]; then\r\n version=$(grep '^go ' go.mod | awk '{print $2}' | cut -d. -f1-2)\r\n else\r\n goDirectiveVersion=$(grep '^go ' go.mod | awk '{print $2}')\r\n toolChainVersion=$(grep '^toolchain ' go.mod | awk '{print $2}')\r\n version=$(printf \"%s\\n%s\\n\" \"$goDirectiveVersion\" \"$toolChainVersion\" | sort -V | tail -n1)\r\n fi\r\n\r\n echo \"version=$version\"\r\n echo \"version=$version\" >> $GITHUB_OUTPUT\r\n" }, { "path": ".github/workflows/nightly-trivy-scan.yml", "content": "name: Trivy Nightly Scan\non:\n schedule:\n - cron: '0 2 * * *' # run at 2 AM UTC\n\njobs:\n nightly-scan:\n name: Trivy nightly scan\n runs-on: ubuntu-latest\n strategy:\n fail-fast: false\n matrix:\n # maintain the versions of Velero those need security scan\n versions: [main]\n # list of images that need scan\n images: [velero, velero-plugin-for-aws, velero-plugin-for-gcp, velero-plugin-for-microsoft-azure]\n permissions:\n security-events: write # for github/codeql-action/upload-sarif to upload SARIF results\n\n steps:\n - name: Checkout code\n uses: actions/checkout@v6\n\n - name: Run Trivy vulnerability scanner\n uses: aquasecurity/trivy-action@master\n with:\n image-ref: 'docker.io/velero/${{ matrix.images }}:${{ matrix.versions }}'\n severity: 'CRITICAL,HIGH,MEDIUM'\n format: 'template'\n template: '@/contrib/sarif.tpl'\n output: 'trivy-results.sarif'\n\n - name: Upload Trivy scan results to GitHub Security tab\n uses: github/codeql-action/upload-sarif@v3\n with:\n sarif_file: 'trivy-results.sarif'" }, { "path": ".github/workflows/pr-changelog-check.yml", "content": "name: Pull Request Changelog Check\n# by setting `on: [pull_request]`, that means action will be trigger when PR is opened, synchronize, reopened.\n# Add labeled and unlabeled events too.\non:\n pull_request:\n types: [opened, synchronize, reopened, labeled, unlabeled]\njobs:\n\n build:\n name: Run Changelog Check\n runs-on: ubuntu-latest\n steps:\n\n - name: Check out the code\n uses: actions/checkout@v6\n\n - name: Changelog check\n if: ${{ !(contains(github.event.pull_request.labels.*.name, 'kind/changelog-not-required') || contains(github.event.pull_request.labels.*.name, 'Design') || contains(github.event.pull_request.labels.*.name, 'Website') || contains(github.event.pull_request.labels.*.name, 'Documentation'))}}\n run: ./hack/changelog-check.sh\n" }, { "path": ".github/workflows/pr-ci-check.yml", "content": "name: Pull Request CI Check\non: [pull_request]\njobs:\n get-go-version:\n uses: ./.github/workflows/get-go-version.yaml\n with:\n ref: ${{ github.event.pull_request.base.ref }}\n\n build:\n name: Run CI\n needs: get-go-version\n runs-on: ubuntu-latest\n strategy:\n fail-fast: false\n steps:\n - name: Check out the code\n uses: actions/checkout@v6\n\n - name: Set up Go version\n uses: actions/setup-go@v6\n with:\n go-version: ${{ needs.get-go-version.outputs.version }} \n\n - name: Make ci\n run: make ci\n - name: Upload test coverage\n uses: codecov/codecov-action@v5\n with:\n token: ${{ secrets.CODECOV_TOKEN }}\n files: coverage.out\n verbose: true\n fail_ci_if_error: true\n" }, { "path": ".github/workflows/pr-codespell.yml", "content": "name: Pull Request Codespell Check\non: [pull_request]\njobs:\n\n codespell:\n name: Run Codespell\n runs-on: ubuntu-latest\n steps:\n\n - name: Check out the code\n uses: actions/checkout@v6\n\n - name: Codespell\n uses: codespell-project/actions-codespell@master\n with:\n # ignore the config/.../crd.go file as it's generated binary data that is edited elsewhere.\n skip: .git,*.png,*.jpg,*.woff,*.ttf,*.gif,*.ico,./config/crd/v1beta1/crds/crds.go,./config/crd/v1/crds/crds.go,./config/crd/v2alpha1/crds/crds.go,./go.sum,./LICENSE\n ignore_words_list: iam,aks,ist,bridget,ue,shouldnot,atleast,notin,sme,optin,sie\n check_filenames: true\n check_hidden: true\n\n - name: Velero.io word list check\n shell: bash {0}\n run: |\n IGNORE_COMMENT=\"Velero.io word list : ignore\"\n FILES_TO_CHECK=$(find . -type f \\\n ! -path \"./.git/*\" \\\n ! -path \"./site/content/docs/v*\" \\\n ! -path \"./changelogs/CHANGELOG-*\" \\\n ! -path \"./.github/workflows/pr-codespell.yml\" \\\n ! -path \"./site/static/fonts/Metropolis/Open Font License.md\" \\\n ! -regex '.*\\.\\(png\\|jpg\\|woff\\|ttf\\|gif\\|ico\\|svg\\)'\n )\n function check_word_in_files() {\n local word=$1\n\n xargs grep -Iinr \"$word\" <<< \"$FILES_TO_CHECK\" | \\\n grep -v \"$IGNORE_COMMENT\" | \\\n grep -i --color=always \"$word\" && \\\n EXIT_STATUS=1\n }\n function check_word_case_sensitive_in_files() {\n local word=$1\n\n xargs grep -Inr \"$word\" <<< \"$FILES_TO_CHECK\" | \\\n grep -v \"$IGNORE_COMMENT\" | \\\n grep --color=always \"$word\" && \\\n EXIT_STATUS=1\n }\n EXIT_STATUS=0\n check_word_case_sensitive_in_files ' kubernetes '\n check_word_in_files 'on-premise\\b'\n check_word_in_files 'back-up'\n check_word_in_files 'plug-in'\n check_word_in_files 'whitelist'\n check_word_in_files 'blacklist'\n exit $EXIT_STATUS\n" }, { "path": ".github/workflows/pr-containers.yml", "content": "name: build Velero containers on Dockerfile change\n\non:\n pull_request:\n branches:\n - 'main'\n - 'release-**'\n paths:\n - 'Dockerfile'\n\njobs:\n build:\n name: Build\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v6\n name: Checkout\n\n - name: Set up QEMU\n id: qemu\n uses: docker/setup-qemu-action@v3\n with:\n platforms: all\n\n - name: Set up Docker Buildx\n id: buildx\n uses: docker/setup-buildx-action@v3\n with:\n version: latest\n\n # Although this action also calls docker-push.sh, it is not triggered\n # by push, so BRANCH and TAG are empty by default. docker-push.sh will\n # only build Velero image without pushing.\n - name: Make Velero container without pushing to registry.\n if: github.repository == 'vmware-tanzu/velero'\n run: |\n ./hack/docker-push.sh" }, { "path": ".github/workflows/pr-goreleaser.yml", "content": "name: Verify goreleaser change\n\non:\n pull_request:\n branches:\n - 'main'\n - 'release-**'\n paths:\n - '.goreleaser.yml'\n - 'hack/release-tools/goreleaser.sh'\n\njobs:\n build:\n name: Build\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v6\n name: Checkout\n\n - name: Verify .goreleaser.yml and try a dryrun release.\n if: github.repository == 'vmware-tanzu/velero'\n run: |\n CHANGELOG=$(ls changelogs | sort -V -r | head -n 1)\n GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }} \\\n REGISTRY=velero \\\n RELEASE_NOTES_FILE=changelogs/$CHANGELOG \\\n PUBLISH=false \\\n make release\n\n" }, { "path": ".github/workflows/pr-linter-check.yml", "content": "name: Pull Request Linter Check\non:\n pull_request:\n # Do not run when the change only includes these directories.\n paths-ignore:\n - \"site/**\"\n - \"design/**\"\n - \"**/*.md\"\njobs:\n get-go-version:\n uses: ./.github/workflows/get-go-version.yaml\n with:\n ref: ${{ github.event.pull_request.base.ref }}\n\n build:\n name: Run Linter Check\n runs-on: ubuntu-latest\n needs: get-go-version\n steps:\n - name: Check out the code\n uses: actions/checkout@v6\n\n - name: Set up Go version\n uses: actions/setup-go@v6\n with:\n go-version: ${{ needs.get-go-version.outputs.version }}\n\n - name: Linter check\n uses: golangci/golangci-lint-action@v9\n with:\n version: v2.5.0\n args: --verbose\n" }, { "path": ".github/workflows/prow-action.yml", "content": "# Adds support for prow-like commands\n# Uses .github/labels.yaml to define areas and kinds\nname: \"Prow github actions\"\non:\n issue_comment:\n types: [created]\n\njobs:\n execute:\n runs-on: ubuntu-latest\n steps:\n - uses: jpmcb/prow-github-actions@v1.1.3\n with:\n # TODO: before allowing the /lgtm command, see if we can block merging if changelog labels are missing.\n prow-commands: |\n /approve\n /area\n /assign\n /cc\n /close\n /hold\n /kind\n /milestone\n /retitle\n /remove\n /reopen\n /uncc\n /unassign\n github-token: \"${{ secrets.GITHUB_TOKEN }}\"\n" }, { "path": ".github/workflows/push-builder.yml", "content": "name: build-image\n\non:\n push:\n branches: [ main ]\n paths:\n - 'hack/build-image/Dockerfile'\n\njobs:\n build:\n name: Build\n runs-on: ubuntu-latest\n steps:\n\n - uses: actions/checkout@v6\n with:\n # The default value is \"1\" which fetches only a single commit. If we merge PR without squash or rebase,\n # there are at least two commits: the first one is the merge commit and the second one is the real commit\n # contains the changes.\n # As we use the Dockerfile's commit ID as the tag of the build-image, fetching only 1 commit causes the merge\n # commit ID to be the tag.\n # While when running make commands locally, as the local git repository usually contains all commits, the Dockerfile's\n # commit ID is the second one. This is mismatch with the images in Dockerhub\n fetch-depth: 2\n\n - name: Build\n run: make build-image\n\n # Only try to publish the container image from the root repo; forks don't have permission to do so and will always get failures.\n - name: Publish container image\n if: github.repository == 'vmware-tanzu/velero'\n run: |\n docker login -u ${{ secrets.DOCKER_USER }} -p ${{ secrets.DOCKER_PASSWORD }}\n \n make push-build-image\n" }, { "path": ".github/workflows/push.yml", "content": "name: Main CI\n\non:\n push:\n branches:\n - 'main'\n - 'release-**'\n tags:\n - '*'\n\njobs:\n get-go-version:\n uses: ./.github/workflows/get-go-version.yaml\n with:\n ref: ${{ github.ref_name }}\n\n build:\n name: Build\n runs-on: ubuntu-latest\n needs: get-go-version\n steps:\n - name: Check out the code\n uses: actions/checkout@v6\n\n - name: Set up Go version\n uses: actions/setup-go@v6\n with:\n go-version: ${{ needs.get-go-version.outputs.version }}\n\n - name: Set up QEMU\n id: qemu\n uses: docker/setup-qemu-action@v3\n with:\n platforms: all\n - name: Set up Docker Buildx\n id: buildx\n uses: docker/setup-buildx-action@v3\n with:\n version: latest\n - name: Build\n run: |\n make local\n # Clean go cache to ease the build environment storage pressure.\n go clean -modcache -cache\n - name: Test\n run: make test\n - name: Upload test coverage\n uses: codecov/codecov-action@v5\n with:\n token: ${{ secrets.CODECOV_TOKEN }}\n files: coverage.out\n verbose: true\n # Only try to publish the container image from the root repo; forks don't have permission to do so and will always get failures.\n - name: Publish container image\n if: github.repository == 'vmware-tanzu/velero'\n run: |\n sudo swapoff -a\n sudo rm -f /mnt/swapfile\n docker system prune -a --force\n \n # Build and push Velero image to docker registry\n docker login -u ${{ secrets.DOCKER_USER }} -p ${{ secrets.DOCKER_PASSWORD }}\n ./hack/docker-push.sh\n" }, { "path": ".github/workflows/rebase.yml", "content": "on: \n issue_comment:\n types: [created]\nname: Automatic Rebase\njobs:\n rebase:\n name: Rebase\n if: github.event.issue.pull_request != '' && contains(github.event.comment.body, '/rebase')\n runs-on: ubuntu-latest\n steps:\n - name: Checkout the latest code\n uses: actions/checkout@v6\n with:\n fetch-depth: 0\n - name: Automatic Rebase\n uses: cirrus-actions/rebase@1.8\n env:\n GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n" }, { "path": ".github/workflows/stale-issues.yml", "content": "name: \"Close stale issues and PRs\"\non:\n schedule:\n - cron: \"30 1 * * *\" # Every day at 1:30 UTC\n\njobs:\n stale:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/stale@v10.1.1\n with:\n repo-token: ${{ secrets.GITHUB_TOKEN }}\n stale-issue-message: \"This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days. If a Velero team member has requested log or more information, please provide the output of the shared commands.\"\n close-issue-message: \"This issue was closed because it has been stalled for 14 days with no activity.\"\n days-before-issue-stale: 60\n days-before-issue-close: 14\n stale-issue-label: staled\n # Disable stale PRs for now; they can remain open.\n days-before-pr-stale: -1\n days-before-pr-close: -1\n # Only issues made after Feb 09 2021.\n start-date: \"2021-09-02T00:00:00\"\n exempt-issue-labels: \"Epic,Area/CLI,Area/Cloud/AWS,Area/Cloud/Azure,Area/Cloud/GCP,Area/Cloud/vSphere,Area/CSI,Area/Design,Area/Documentation,Area/Plugins,Bug,Enhancement/User,kind/requirement,kind/refactor,kind/tech-debt,limitation,Needs investigation,Needs triage,Needs Product,P0 - Hair on fire,P1 - Important,P2 - Long-term important,P3 - Wouldn't it be nice if...,Product Requirements,Restic - GA,Restic,release-blocker,Security,backlog\"\n" }, { "path": ".gitignore", "content": "# Compiled Object files, Static and Dynamic libs (Shared Objects)\n*.o\n*.a\n*.so\n\n# Folders\n_obj\n_test\n_output\n\n# Architecture specific extensions/prefixes\n*.[568vq]\n[568vq].out\n\n*.cgo1.go\n*.cgo2.c\n_cgo_defun.c\n_cgo_gotypes.go\n_cgo_export.*\n\n_testmain.go\n\n*.exe\n*.test\n*.prof\n\n/velero\n.idea/\n\n.container-*\n.vimrc\n.go\n.DS_Store\n.push-*\n.vscode\n*.diff\n\n# Hugo compiled data\nsite/public\nsite/resources\nsite/.hugo_build.lock\n\n.vs\n\n# these are files for local Tilt development:\n_tiltbuild\ntilt-resources/tilt-settings.json\ntilt-resources/velero_v1_backupstoragelocation.yaml\ntilt-resources/deployment.yaml\ntilt-resources/node-agent.yaml\ntilt-resources/cloud\n\n# test generated files\ntest/e2e/report.xml\ncoverage.out\n__debug_bin*\ndebug.test*\n\n# make lint cache\n.cache/\n\n# Go telemetry directory created when container sets HOME to working directory\n# This happens because Makefile uses 'docker run -w /github.com/vmware-tanzu/velero'\n# and Go's os.UserConfigDir() falls back to $HOME/.config when XDG_CONFIG_HOME is unset\n.config/\n" }, { "path": ".golangci.yaml", "content": "# This file contains all available configuration options\n# with their default values.\n\n# options for analysis running\nrun:\n # default concurrency is a available CPU number\n concurrency: 4\n\n # timeout for analysis, e.g. 30s, 5m, default is 0\n timeout: 20m\n\n # exit code when at least one issue was found, default is 1\n issues-exit-code: 1\n\n # by default isn't set. If set we pass it to \"go list -mod={option}\". From \"go help modules\":\n # If invoked with -mod=readonly, the go command is disallowed from the implicit\n # automatic updating of go.mod described above. Instead, it fails when any changes\n # to go.mod are needed. This setting is most useful to check that go.mod does\n # not need updates, such as in a continuous integration and testing system.\n # If invoked with -mod=vendor, the go command assumes that the vendor\n # directory holds the correct copies of dependencies and ignores\n # the dependency descriptions in go.mod.\n # modules-download-mode: readonly|release|vendor\n modules-download-mode: readonly\n\n # Allow multiple parallel golangci-lint instances running.\n # If false (default) - golangci-lint acquires file lock on start.\n allow-parallel-runners: false\n\n# output configuration options\noutput:\n formats:\n text:\n path: stdout\n\n # print lines of code with issue, default is true\n print-issued-lines: true\n\n # print linter name in the end of issue text, default is true\n print-linter-name: true\n\n # Show statistics per linter. \n show-stats: false\n\nlinters:\n # all available settings of specific linters\n settings:\n depguard:\n rules:\n main:\n deny:\n # specify an error message to output when a denylisted package is used\n - pkg: github.com/sirupsen/logrus\n desc: \"logging is allowed only by logutils.Log\"\n\n dogsled:\n # checks assignments with too many blank identifiers; default is 2\n max-blank-identifiers: 2\n\n dupl:\n # tokens count to trigger issue, 150 by default\n threshold: 100\n\n errcheck:\n # report about not checking of errors in type assertions: `a := b.(MyStruct)`;\n # default is false: such cases aren't reported by default.\n check-type-assertions: false\n\n # report about assignment of errors to blank identifier: `num, _ := strconv.Atoi(numStr)`;\n # default is false: such cases aren't reported by default.\n check-blank: false\n\n\n exhaustive:\n # indicates that switch statements are to be considered exhaustive if a\n # 'default' case is present, even if all enum members aren't listed in the\n # switch\n default-signifies-exhaustive: false\n\n funlen:\n lines: 60\n statements: 40\n\n gocognit:\n # minimal code complexity to report, 30 by default (but we recommend 10-20)\n min-complexity: 10\n\n nestif:\n # minimal complexity of if statements to report, 5 by default\n min-complexity: 4\n\n goconst:\n # minimal length of string constant, 3 by default\n min-len: 3\n # minimal occurrences count to trigger, 3 by default\n min-occurrences: 5\n\n gocritic:\n # Which checks should be enabled; can't be combined with 'disabled-checks';\n # See https://go-critic.github.io/overview#checks-overview\n # To check which checks are enabled run `GL_DEBUG=gocritic golangci-lint run`\n # By default list of stable checks is used.\n settings: # settings passed to gocritic\n captLocal: # must be valid enabled check name\n paramsOnly: true\n\n gocyclo:\n # minimal code complexity to report, 30 by default (but we recommend 10-20)\n min-complexity: 10\n\n godot:\n # check all top-level comments, not only declarations\n check-all: false\n\n godox:\n # report any comments starting with keywords, this is useful for TODO or FIXME comments that\n # might be left in the code accidentally and should be resolved before merging\n keywords: # default keywords are TODO, BUG, and FIXME, these can be overwritten by this setting\n - NOTE\n - OPTIMIZE # marks code that should be optimized before merging\n - HACK # marks hack-arounds that should be removed before merging\n\n gosec:\n excludes:\n - G115\n\n govet:\n # enable or disable analyzers by name\n enable:\n - atomicalign\n enable-all: false\n disable:\n - shadow\n disable-all: false\n \n importas:\n alias:\n - alias: appsv1api\n pkg: k8s.io/api/apps/v1\n - alias: corev1api\n pkg: k8s.io/api/core/v1\n - alias: rbacv1\n pkg: k8s.io/api/rbac/v1\n - alias: apierrors\n pkg: k8s.io/apimachinery/pkg/api/errors\n - alias: apiextv1\n pkg: k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1\n - alias: metav1\n pkg: k8s.io/apimachinery/pkg/apis/meta/v1\n - alias: storagev1api\n pkg: k8s.io/api/storage/v1\n - alias: batchv1api\n pkg: k8s.io/api/batch/v1\n\n lll:\n # max line length, lines longer will be reported. Default is 120.\n # '\\t' is counted as 1 character by default, and can be changed with the tab-width option\n line-length: 120\n # tab width in spaces. Default to 1.\n tab-width: 1\n\n misspell:\n # Correct spellings using locale preferences for US or UK.\n # Default is to use a neutral variety of English.\n # Setting locale to US will correct the British spelling of 'colour' to 'color'.\n locale: US\n ignore-rules:\n - someword\n\n nakedret:\n # make an issue if func has more lines of code than this setting and it has naked returns; default is 30\n max-func-lines: 30\n\n prealloc:\n # XXX: we don't recommend using this linter before doing performance profiling.\n # For most programs usage of prealloc will be a premature optimization.\n\n # Report preallocation suggestions only on simple loops that have no returns/breaks/continues/gotos in them.\n # True by default.\n simple: true\n range-loops: true # Report preallocation suggestions on range loops, true by default\n for-loops: false # Report preallocation suggestions on for loops, false by default\n\n nolintlint:\n # Enable to ensure that nolint directives are all used. Default is true.\n allow-unused: false\n # Exclude following linters from requiring an explanation. Default is [].\n allow-no-explanation: []\n # Enable to require an explanation of nonzero length after each nolint directive. Default is false.\n require-explanation: true\n # Enable to require nolint directives to mention the specific linter being suppressed. Default is false.\n require-specific: true\n\n perfsprint:\n strconcat: false\n sprintf1: false\n errorf: false\n int-conversion: true\n\n revive:\n rules:\n - name: blank-imports\n disabled: true\n - name: context-as-argument\n disabled: true\n - name: context-keys-type\n - name: dot-imports\n disabled: true\n - name: early-return\n disabled: true\n arguments:\n - \"preserveScope\"\n - name: empty-block\n disabled: true\n - name: error-naming\n disabled: true\n - name: error-return\n disabled: true\n - name: error-strings\n disabled: true\n - name: errorf\n disabled: true\n - name: increment-decrement\n - name: indent-error-flow\n disabled: true\n - name: range\n - name: receiver-naming\n disabled: true\n - name: redefines-builtin-id\n disabled: true\n - name: superfluous-else\n disabled: true\n arguments:\n - \"preserveScope\"\n - name: time-naming\n - name: unexported-return\n disabled: true\n - name: unnecessary-stmt\n - name: unreachable-code\n - name: unused-parameter\n disabled: true\n - name: use-any\n - name: var-declaration\n - name: var-naming\n disabled: true\n\n rowserrcheck:\n packages:\n - github.com/jmoiron/sqlx\n\n staticcheck:\n checks:\n - all\n - -QF1001 # FIXME\n - -QF1003 # FIXME\n - -QF1004 # FIXME\n - -QF1007 # FIXME\n - -QF1008 # FIXME\n - -QF1009 # FIXME\n - -QF1012 # FIXME\n\n testifylint:\n # TODO: enable them all\n disable:\n - float-compare\n - go-require\n enable-all: true\n\n testpackage:\n # regexp pattern to skip files\n skip-regexp: (export|internal)_test\\.go\n unparam:\n # Inspect exported functions, default is false. Set to true if no external program/library imports your code.\n # XXX: if you enable this setting, unparam will report a lot of false-positives in text editors:\n # if it's called for subdir of a project it can't find external interfaces. All text editor integrations\n # with golangci-lint call it on a directory with the changed file.\n check-exported: false\n\n usetesting:\n os-setenv: false\n\n whitespace:\n multi-if: false # Enforces newlines (or comments) after every multi-line if statement\n multi-func: false # Enforces newlines (or comments) after every multi-line function signature\n\n wsl:\n # If true append is only allowed to be cuddled if appending value is\n # matching variables, fields or types on line above. Default is true.\n strict-append: true\n # Allow calls and assignments to be cuddled as long as the lines have any\n # matching variables, fields or types. Default is true.\n allow-assign-and-call: true\n # Allow multiline assignments to be cuddled. Default is true.\n allow-multiline-assign: true\n # Allow declarations (var) to be cuddled.\n allow-cuddle-declarations: false\n # Allow trailing comments in ending of blocks\n allow-trailing-comment: false\n # Force newlines in end of case at this limit (0 = never).\n force-case-trailing-whitespace: 0\n # Force cuddling of err checks with err var assignment\n force-err-cuddling: false\n # Allow leading comments to be separated with empty lines\n allow-separated-leading-comment: false\n\n default: none\n enable:\n - asasalint\n - asciicheck\n - bidichk\n - bodyclose\n - copyloopvar\n - dogsled\n - dupword\n - durationcheck\n - errcheck\n - errchkjson\n - exptostd\n - ginkgolinter\n - goconst\n - goheader\n - goprintffuncname\n - gosec\n - govet\n - importas\n - ineffassign\n - misspell\n - nakedret\n - nilerr\n - noctx\n - nolintlint\n - nosprintfhostport\n - perfsprint\n - revive\n - staticcheck\n - testifylint\n - thelper\n - unconvert\n - unparam\n - unused\n - usestdlibvars\n - usetesting\n - whitespace\n\n exclusions:\n # which dirs to skip: issues from them won't be reported;\n # can use regexp here: generated.*, regexp is applied on full path;\n # default value is empty list, but default dirs are skipped independently\n # from this option's value (see skip-dirs-use-default).\n # \"/\" will be replaced by current OS file path separator to properly work\n # on Windows.\n paths:\n - pkg/plugin/generated/*\n - third_party\n\n rules:\n - linters:\n - staticcheck\n text: \"DefaultVolumesToRestic\" # No need to report deprecate for DefaultVolumesToRestic.\n - path: \".*_test.go$\"\n linters:\n - errcheck\n - goconst\n - gosec\n - govet\n - staticcheck\n - unparam\n - unused\n - path: test/\n linters:\n - errcheck\n - goconst\n - gosec\n - nilerr\n - staticcheck\n - unparam\n - unused\n - path: \".*data_upload_controller_test.go$\"\n linters:\n - dupword\n text: \"type\"\n - path: \".*config_test.go$\"\n linters:\n - dupword\n text: \"bucket\"\n\n generated: lax\n presets:\n - comments\n - common-false-positives\n - legacy\n - std-error-handling\n\nissues:\n # Maximum issues count per one linter. Set to 0 to disable. Default is 50.\n max-issues-per-linter: 0\n\n # Maximum count of issues with the same text. Set to 0 to disable. Default is 3.\n max-same-issues: 0\n\n # make issues output unique by line, default is true\n uniq-by-line: true\n\n# This file contains all available configuration options\n# with their default values.\nformatters:\n enable:\n - gofmt\n - goimports\n\n exclusions:\n generated: lax\n paths:\n - pkg/plugin/generated/*\n - third_party\n\n settings:\n gofmt:\n # simplify code: gofmt with `-s` option, true by default\n simplify: true\n goimports:\n local-prefixes:\n - github.com/vmware-tanzu/velero\n\nseverity:\n default: error\n\n # Default value is empty list.\n # When a list of severity rules are provided, severity information will be added to lint\n # issues. Severity rules have the same filtering capability as exclude rules except you \n # are allowed to specify one matcher per severity rule.\n # Only affects out formats that support setting severity information.\n rules:\n - linters:\n - dupl\n severity: info\n\nversion: \"2\"\n" }, { "path": ".goreleaser.yml", "content": "# Copyright 2018 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\ndist: _output\nbuilds:\n - main: ./cmd/velero/velero.go\n env:\n - CGO_ENABLED=0\n goos:\n - linux\n - darwin\n - windows\n goarch:\n - amd64\n - arm\n - arm64\n - ppc64le\n - s390x\n ignore:\n # don't build arm for darwin and arm/arm64 for windows\n - goos: darwin\n goarch: arm\n - goos: darwin\n goarch: ppc64le\n - goos: darwin\n goarch: s390x\n - goos: windows\n goarch: arm\n - goos: windows\n goarch: arm64\n - goos: windows\n goarch: ppc64le\n - goos: windows\n goarch: s390x\n ldflags:\n - -X \"github.com/vmware-tanzu/velero/pkg/buildinfo.Version={{ .Tag }}\" -X \"github.com/vmware-tanzu/velero/pkg/buildinfo.GitSHA={{ .FullCommit }}\" -X \"github.com/vmware-tanzu/velero/pkg/buildinfo.GitTreeState={{ .Env.GIT_TREE_STATE }}\" -X \"github.com/vmware-tanzu/velero/pkg/buildinfo.ImageRegistry={{ .Env.REGISTRY }}\"\narchives:\n - name_template: \"{{ .ProjectName }}-{{ .Tag }}-{{ .Os }}-{{ .Arch }}\"\n wrap_in_directory: true\n files:\n - LICENSE\n - examples/**/*\nchecksum:\n name_template: 'CHECKSUM'\nrelease:\n github:\n owner: vmware-tanzu\n name: velero\n draft: true\n prerelease: auto\n\ngit:\n # What should be used to sort tags when gathering the current and previous\n # tags if there are more than one tag in the same commit.\n #\n # Default: `-version:refname`\n tag_sort: -version:creatordate\n" }, { "path": "ADOPTERS.md", "content": "# Velero Adopters\n\nIf you're using Velero and want to add your organization to this list, \n[follow these directions][1]!\n\n\"pitsdatarecovery.net\"\n\"bitgo.com\"     \n\"nirmata.com\"     \n\"kyma-project.io\"     \n\"redhat.com\"     \n\"dellemc.com\"     \n\"bugsnag.com\"     \n\"okteto.com\"     \n\"banzaicloud.com\"     \n\"sighup.io\"     \n\"mayadata.io\"     \n\"replicated.com\"\n\"cloudcasa.io\"\n\"azure.com\"\n\"broadcom.com\"\n## Success Stories\n\nBelow is a list of adopters of Velero in **production environments** that have\npublicly shared the details of how they use it.\n\n**[BitGo][20]** \nBitGo uses Velero backup and restore capabilities to seamlessly provision and scale fullnode statefulsets on the fly as well as having it serve an integral piece for our Kubernetes disaster-recovery story.\n\n**[Bugsnag][30]** \nWe use Velero for managing backups of an internal instance of our on-premise clustered solution. We also recommend our users of [on-premise Bugsnag installations](https://www.bugsnag.com/on-premise) use Velero for [managing their own backups](https://docs.bugsnag.com/on-premise/clustered/backup-restore/). \n\n**[Banzai Cloud][60]** \n[Banzai Cloud Pipeline][61] is a Kubernetes-based microservices platform that integrates services needed for Day-1 and Day-2 operations along with first-class support both for on-prem and hybrid multi-cloud deployments. We use Velero to periodically [backup and restore these clusters in case of disasters][62].\n\n## Solutions built with Velero\n\nBelow is a list of solutions where Velero is being used as a component.\n\n**[Nirmata][10]** \nWe have integrated our [solution with Velero][11] to provide our customers with out of box backup/DR.\n\n**[Kyma][40]** \nKyma [integrates with Velero][41] to effortlessly back up and restore Kyma clusters with all its resources. Velero capabilities allow Kyma users to define and run manual and scheduled backups in order to successfully handle a disaster-recovery scenario.\n\n**[Red Hat][50]** \nRed Hat has developed 2 operators for the OpenShift platform:\n- [Migration Toolkit for Containers][51] (Crane): This operator uses [Velero and Restic][52] to drive the migration of applications between OpenShift clusters.\n- [OADP (OpenShift API for Data Protection) Operator][53]: This operator sets up and installs Velero on the OpenShift platform, allowing users to backup and restore applications.\n\n**[Dell EMC][70]** \nFor Kubernetes environments, [PowerProtect Data Manager][71] leverages the Container Storage Interface (CSI) framework to take snapshots to back up the persistent data or the data that the application creates e.g. databases. [Dell EMC leverages Velero][72] to backup the namespace configuration files (also known as Namespace meta data) for enterprise grade data protection.\n\n**[SIGHUP][80]** \nSIGHUP integrates Velero in its [Fury Kubernetes Distribution][81] providing predefined schedules and configurations to ensure an optimized disaster recovery experience.\n[Fury Kubernetes Disaster Recovery Module][82] is ready to be deployed into any Kubernetes cluster running anywhere.\n\n**[MayaData][90]** \nMayaData is a large user of Velero as well as a contributor. MayaData offers a Data Agility platform called [OpenEBS Director][91], that helps customers confidently and easily manage stateful workloads in Kubernetes. Velero is one of the core software building block of the OpenEBS Director's [DMaaS or data migration as a service offering][92] used to enable data protection strategies.\n\n**[Okteto][93]** \nOkteto integrates Velero in [Okteto Cloud][94] and [Okteto Enterprise][95] to periodically backup and restore our clusters for disaster recovery. Velero is also a core software building block to provide namespace cloning capabilities, a feature that allows our users cloning staging environments into their personal development namespace for providing production-like development environments.\n\n**[Replicated][100]**
\nReplicated uses the Velero open source project to enable snapshots in [KOTS][101] to backup Kubernetes manifests & persistent volumes. In addition to the default functionality that Velero provides, [KOTS][101] provides a detailed interface in the [Admin Console][102] that can be used to manage the storage destination and schedule, and to perform and monitor the backup and restore process.
\n\n**[CloudCasa][103]**
\n[Catalogic Software][104] integrates Velero with [CloudCasa][103] - A Smart Home in the Cloud for Backups. CloudCasa is a full-featured, scalable, cloud-native solution providing Kubernetes data protection, disaster recovery, and migration as a service. An option to manage existing Velero instances and an enterprise self-hosted option are also available.
\n\n**[Microsoft Azure][105]**
\n[Azure Backup for AKS][106] is an Azure native, Kubernetes aware, Enterprise ready backup for containerized applications deployed on Azure Kubernetes Service (AKS). AKS Backup utilizes Velero to perform backup and restore operations to protect stateful applications in AKS clusters.
\n\n**[Broadcom][107]**
\n[VMware Cloud Foundation][108] (VCF) offers built-in [vSphere Kubernetes Service][109] (VKS), a Kubernetes runtime that includes a CNCF certified Kubernetes distribution, to deploy and manage containerized workloads. VCF empowers platform engineers with native [Kubernetes multi-cluster management][110] capability for managing Kubernetes (K8s) infrastructure at scale. VCF utilizes Velero for Kubernetes data protection enabling platform engineers to back up and restore containerized workloads manifests & persistent volumes, helping to increase the resiliency of stateful applications in VKS cluster.\n\n## Adding your organization to the list of Velero Adopters\n\nIf you are using Velero and would like to be included in the list of `Velero Adopters`, add an SVG version of your logo to the `site/static/img/adopters` directory in this repo and submit a [pull request][3] with your change. Name the image file something that reflects your company (e.g., if your company is called Acme, name the image acme.png). See this for an example [PR][4].\n\n### Adding a logo to velero.io\n\nIf you would like to add your logo to a future `Adopters of Velero` section on [velero.io][2], follow the steps above to add your organization to the list of Velero Adopters. Our community will follow up and publish it to the [velero.io][2] website.\n\n[1]: #adding-a-logo-to-veleroio\n[2]: https://velero.io\n[3]: https://github.com/vmware-tanzu/velero/pulls\n[4]: https://github.com/vmware-tanzu/velero/pull/2242\n\n[10]: https://www.nirmata.com/2019/08/14/kubernetes-disaster-recovery-using-velero-and-nirmata/\n[11]: https://nirmata.com\n\n[20]: https://bitgo.com\n\n[30]: https://bugsnag.com\n\n[40]: https://kyma-project.io\n[41]: https://kyma-project.io/docs/components/backup/#overview-overview\n\n[50]: https://redhat.com\n[51]: https://github.com/fusor/mig-operator\n[52]: https://github.com/fusor/mig-operator/blob/master/docs/usage/2.md\n[53]: https://github.com/openshift/oadp-operator\n\n[60]: https://banzaicloud.com\n[61]: https://banzaicloud.com/products/pipeline/\n[62]: https://banzaicloud.com/blog/vault-backup-velero/\n\n[70]: https://dellemc.com\n[71]: https://dellemc.com/dataprotection\n[72]: https://www.dellemc.com/resources/en-us/asset/briefs-handouts/solutions/h18141-dellemc-dpd-kubernetes.pdf\n\n[80]: https://sighup.io\n[81]: https://github.com/sighupio/fury-distribution\n[82]: https://github.com/sighupio/fury-kubernetes-dr\n\n[90]: https://mayadata.io\n[91]: https://director.mayadata.io/\n[92]: https://help.mayadata.io/hc/en-us/articles/360033401591-DMaaS\n\n[93]: https://okteto.com\n[94]: https://cloud.okteto.com\n[95]: https://okteto.com/enterprise/\n\n[100]: https://www.replicated.com\n[101]: https://kots.io\n[102]: https://kots.io/kotsadm/snapshots/overview/\n\n[103]: https://cloudcasa.io/\n[104]: https://www.catalogicsoftware.com/\n\n[105]: https://azure.microsoft.com/\n[106]: https://learn.microsoft.com/azure/backup/backup-overview\n\n[107]: https://www.broadcom.com/\n[108]: https://www.vmware.com/products/cloud-infrastructure/vmware-cloud-foundation\n[109]: https://www.vmware.com/products/cloud-infrastructure/vsphere-kubernetes-service\n[110]: https://blogs.vmware.com/cloud-foundation/2025/09/29/empowering-platform-engineers-with-native-kubernetes-multi-cluster-management-in-vmware-cloud-foundation/" }, { "path": "CHANGELOG.md", "content": "## Current release:\n * [CHANGELOG-1.15.md][25]\n\n## Older releases:\n * [CHANGELOG-1.14.md][24]\n * [CHANGELOG-1.13.md][23]\n * [CHANGELOG-1.12.md][22]\n * [CHANGELOG-1.11.md][21]\n * [CHANGELOG-1.10.md][20]\n * [CHANGELOG-1.9.md][19]\n * [CHANGELOG-1.8.md][18]\n * [CHANGELOG-1.7.md][17]\n * [CHANGELOG-1.6.md][16]\n * [CHANGELOG-1.5.md][15]\n * [CHANGELOG-1.4.md][14]\n * [CHANGELOG-1.3.md][13]\n * [CHANGELOG-1.2.md][12]\n * [CHANGELOG-1.1.md][11]\n * [CHANGELOG-1.0.md][10]\n * [CHANGELOG-0.11.md][9]\n * [CHANGELOG-0.10.md][8]\n * [CHANGELOG-0.9.md][7]\n * [CHANGELOG-0.8.md][6]\n * [CHANGELOG-0.7.md][5]\n * [CHANGELOG-0.6.md][4]\n * [CHANGELOG-0.5.md][3]\n * [CHANGELOG-0.4.md][2]\n * [CHANGELOG-0.3.md][1]\n\n\n[25]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.15.md\n[24]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.14.md\n[23]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.13.md\n[22]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.12.md\n[21]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.11.md\n[20]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.10.md\n[19]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.9.md\n[18]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.8.md\n[17]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.7.md\n[16]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.6.md\n[15]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.5.md\n[14]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.4.md\n[13]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.3.md\n[12]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.2.md\n[11]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.1.md\n[10]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.0.md\n[9]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-0.11.md\n[8]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-0.10.md\n[7]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-0.9.md\n[6]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-0.8.md\n[5]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-0.7.md\n[4]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-0.6.md\n[3]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-0.5.md\n[2]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-0.4.md\n[1]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-0.3.md\n[0]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/unreleased\n" }, { "path": "CODE_OF_CONDUCT.md", "content": "# Contributor Covenant Code of Conduct\n\n## Our Pledge\n\nWe as members, contributors, and leaders pledge to make participation in the Velero project and our\ncommunity a harassment-free experience for everyone, regardless of age, body\nsize, visible or invisible disability, ethnicity, sex characteristics, gender\nidentity and expression, level of experience, education, socioeconomic status,\nnationality, personal appearance, race, religion, or sexual identity\nand orientation.\n\nWe pledge to act and interact in ways that contribute to an open, welcoming,\ndiverse, inclusive, and healthy community.\n\n## Our Standards\n\nExamples of behavior that contributes to a positive environment for our\ncommunity include:\n\n* Demonstrating empathy and kindness toward other people\n* Being respectful of differing opinions, viewpoints, and experiences\n* Giving and gracefully accepting constructive feedback\n* Accepting responsibility and apologizing to those affected by our mistakes,\n and learning from the experience\n* Focusing on what is best not just for us as individuals, but for the\n overall community\n\nExamples of unacceptable behavior include:\n\n* The use of sexualized language or imagery, and sexual attention or\n advances of any kind\n* Trolling, insulting or derogatory comments, and personal or political attacks\n* Public or private harassment\n* Publishing others' private information, such as a physical or email\n address, without their explicit permission\n* Other conduct which could reasonably be considered inappropriate in a\n professional setting\n\n## Enforcement Responsibilities\n\nCommunity leaders are responsible for clarifying and enforcing our standards of\nacceptable behavior and will take appropriate and fair corrective action in\nresponse to any behavior that they deem inappropriate, threatening, offensive,\nor harmful.\n\nCommunity leaders have the right and responsibility to remove, edit, or reject\ncomments, commits, code, wiki edits, issues, and other contributions that are\nnot aligned to this Code of Conduct, and will communicate reasons for moderation\ndecisions when appropriate.\n\n## Scope\n\nThis Code of Conduct applies within all community spaces, and also applies when\nan individual is officially representing the community in public spaces.\nExamples of representing our community include using an official e-mail address,\nposting via an official social media account, or acting as an appointed\nrepresentative at an online or offline event.\n\n## Enforcement\n\nInstances of abusive, harassing, or otherwise unacceptable behavior may be\nreported to the community leaders responsible for enforcement at oss-coc@vmware.com.\nAll complaints will be reviewed and investigated promptly and fairly.\n\nAll community leaders are obligated to respect the privacy and security of the\nreporter of any incident.\n\n## Enforcement Guidelines\n\nCommunity leaders will follow these Community Impact Guidelines in determining\nthe consequences for any action they deem in violation of this Code of Conduct:\n\n### 1. Correction\n\n**Community Impact**: Use of inappropriate language or other behavior deemed\nunprofessional or unwelcome in the community.\n\n**Consequence**: A private, written warning from community leaders, providing\nclarity around the nature of the violation and an explanation of why the\nbehavior was inappropriate. A public apology may be requested.\n\n### 2. Warning\n\n**Community Impact**: A violation through a single incident or series\nof actions.\n\n**Consequence**: A warning with consequences for continued behavior. No\ninteraction with the people involved, including unsolicited interaction with\nthose enforcing the Code of Conduct, for a specified period of time. This\nincludes avoiding interactions in community spaces as well as external channels\nlike social media. Violating these terms may lead to a temporary or\npermanent ban.\n\n### 3. Temporary Ban\n\n**Community Impact**: A serious violation of community standards, including\nsustained inappropriate behavior.\n\n**Consequence**: A temporary ban from any sort of interaction or public\ncommunication with the community for a specified period of time. No public or\nprivate interaction with the people involved, including unsolicited interaction\nwith those enforcing the Code of Conduct, is allowed during this period.\nViolating these terms may lead to a permanent ban.\n\n### 4. Permanent Ban\n\n**Community Impact**: Demonstrating a pattern of violation of community\nstandards, including sustained inappropriate behavior, harassment of an\nindividual, or aggression toward or disparagement of classes of individuals.\n\n**Consequence**: A permanent ban from any sort of public interaction within\nthe community.\n\n## Attribution\n\nThis Code of Conduct is adapted from the [Contributor Covenant][homepage],\nversion 2.0, available at\nhttps://www.contributor-covenant.org/version/2/0/code_of_conduct.html.\n\nCommunity Impact Guidelines were inspired by [Mozilla's code of conduct\nenforcement ladder](https://github.com/mozilla/diversity).\n\n[homepage]: https://www.contributor-covenant.org\n\nFor answers to common questions about this code of conduct, see the FAQ at\nhttps://www.contributor-covenant.org/faq. Translations are available at\nhttps://www.contributor-covenant.org/translations." }, { "path": "CONTRIBUTING.md", "content": "# Contributing\n\nAuthors are expected to follow some guidelines when submitting PRs. Please see [our documentation](https://velero.io/docs/main/code-standards/) for details.\n" }, { "path": "Dockerfile", "content": "# Copyright 2020 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# Velero binary build section\nFROM --platform=$BUILDPLATFORM golang:1.25-bookworm AS velero-builder\n\nARG GOPROXY\nARG BIN\nARG PKG\nARG VERSION\nARG REGISTRY\nARG GIT_SHA\nARG GIT_TREE_STATE\nARG TARGETOS\nARG TARGETARCH\nARG TARGETVARIANT\n\nENV CGO_ENABLED=0 \\\n GO111MODULE=on \\\n GOPROXY=${GOPROXY} \\\n GOOS=${TARGETOS} \\\n GOARCH=${TARGETARCH} \\\n GOARM=${TARGETVARIANT} \\\n LDFLAGS=\"-X ${PKG}/pkg/buildinfo.Version=${VERSION} -X ${PKG}/pkg/buildinfo.GitSHA=${GIT_SHA} -X ${PKG}/pkg/buildinfo.GitTreeState=${GIT_TREE_STATE} -X ${PKG}/pkg/buildinfo.ImageRegistry=${REGISTRY}\"\n\nWORKDIR /go/src/github.com/vmware-tanzu/velero\n\nCOPY . /go/src/github.com/vmware-tanzu/velero\n\nRUN mkdir -p /output/usr/bin && \\\n export GOARM=$( echo \"${GOARM}\" | cut -c2-) && \\\n go build -o /output/${BIN} \\\n -ldflags \"${LDFLAGS}\" ${PKG}/cmd/${BIN} && \\\n go build -o /output/velero-restore-helper \\\n -ldflags \"${LDFLAGS}\" ${PKG}/cmd/velero-restore-helper && \\\n go build -o /output/velero-helper \\\n -ldflags \"${LDFLAGS}\" ${PKG}/cmd/velero-helper && \\\n go clean -modcache -cache\n\n# Restic binary build section\nFROM --platform=$BUILDPLATFORM golang:1.25-bookworm AS restic-builder\n\nARG GOPROXY\nARG BIN\nARG TARGETOS\nARG TARGETARCH\nARG TARGETVARIANT\nARG RESTIC_VERSION\n\nENV CGO_ENABLED=0 \\\n GO111MODULE=on \\\n GOPROXY=${GOPROXY} \\\n GOOS=${TARGETOS} \\\n GOARCH=${TARGETARCH} \\\n GOARM=${TARGETVARIANT}\n\nCOPY . /go/src/github.com/vmware-tanzu/velero\n\nRUN mkdir -p /output/usr/bin && \\\n export GOARM=$(echo \"${GOARM}\" | cut -c2-) && \\\n /go/src/github.com/vmware-tanzu/velero/hack/build-restic.sh && \\\n go clean -modcache -cache\n\n# Velero image packing section\nFROM paketobuildpacks/run-jammy-tiny:latest\n\nLABEL maintainer=\"Xun Jiang \"\n\nCOPY --from=velero-builder /output /\n\nCOPY --from=restic-builder /output /\n\nUSER cnb:cnb\n\n" }, { "path": "Dockerfile-Windows", "content": "# Copyright the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nARG OS_VERSION=1809\n\n# Velero binary build section\nFROM --platform=$BUILDPLATFORM golang:1.25-bookworm AS velero-builder\n\nARG GOPROXY\nARG BIN\nARG PKG\nARG VERSION\nARG REGISTRY\nARG GIT_SHA\nARG GIT_TREE_STATE\nARG TARGETOS\nARG TARGETARCH\nARG TARGETVARIANT\n\nENV CGO_ENABLED=0 \\\n GO111MODULE=on \\\n GOPROXY=${GOPROXY} \\\n GOOS=${TARGETOS} \\\n GOARCH=${TARGETARCH} \\\n GOARM=${TARGETVARIANT} \\\n LDFLAGS=\"-X ${PKG}/pkg/buildinfo.Version=${VERSION} -X ${PKG}/pkg/buildinfo.GitSHA=${GIT_SHA} -X ${PKG}/pkg/buildinfo.GitTreeState=${GIT_TREE_STATE} -X ${PKG}/pkg/buildinfo.ImageRegistry=${REGISTRY}\"\n\nWORKDIR /go/src/github.com/vmware-tanzu/velero\n\nCOPY . /go/src/github.com/vmware-tanzu/velero\n\nRUN mkdir -p /output/usr/bin && \\\n export GOARM=$( echo \"${GOARM}\" | cut -c2-) && \\\n go build -o /output/${BIN}.exe \\\n -ldflags \"${LDFLAGS}\" ${PKG}/cmd/${BIN} && \\\n go build -o /output/velero-restore-helper.exe \\\n -ldflags \"${LDFLAGS}\" ${PKG}/cmd/velero-restore-helper && \\ \n go build -o /output/velero-helper.exe \\\n -ldflags \"${LDFLAGS}\" ${PKG}/cmd/velero-helper && \\\n go clean -modcache -cache\n\n# Velero image packing section\nFROM mcr.microsoft.com/windows/nanoserver:${OS_VERSION}\nCOPY --from=velero-builder /output /\n\nUSER ContainerUser" }, { "path": "GOVERNANCE.md", "content": "# Velero Governance\n\nThis document defines the project governance for Velero.\n\n## Overview\n\n**Velero**, an open source project, is committed to building an open, inclusive, productive and self-governing open source community focused on building a high quality tool that enables users to safely backup and restore, perform disaster recovery, and migrate Kubernetes cluster resources and persistent volumes. The community is governed by this document with the goal of defining how community should work together to achieve this goal.\n\n## Code Repositories\n\nThe following code repositories are governed by Velero community and maintained under the `vmware-tanzu\\Velero` organization.\n\n* **[Velero](https://github.com/vmware-tanzu/velero):** Main Velero codebase\n* **[Helm Chart](https://github.com/vmware-tanzu/helm-charts/tree/main/charts/velero):** The Helm chart for the Velero server component\n* **[Velero CSI Plugin](https://github.com/vmware-tanzu/velero-plugin-for-csi):** This repository contains Velero plugins for snapshotting CSI backed PVCs using the CSI beta snapshot APIs\n* **[Velero Plugin for vSphere](https://github.com/vmware-tanzu/velero-plugin-for-vsphere):** This repository contains the Velero Plugin for vSphere. This plugin is a volume snapshotter plugin that provides crash-consistent snapshots of vSphere block volumes and backup of volume data into S3 compatible storage.\n* **[Velero Plugin for AWS](https://github.com/vmware-tanzu/velero-plugin-for-aws):** This repository contains the plugins to support running Velero on AWS, including the object store plugin and the volume snapshotter plugin\n* **[Velero Plugin for GCP](https://github.com/vmware-tanzu/velero-plugin-for-gcp):** This repository contains the plugins to support running Velero on GCP, including the object store plugin and the volume snapshotter plugin\n* **[Velero Plugin for Azure](https://github.com/vmware-tanzu/velero-plugin-for-microsoft-azure):** This repository contains the plugins to support running Velero on Azure, including the object store plugin and the volume snapshotter plugin\n* **[Velero Plugin Example](https://github.com/vmware-tanzu/velero-plugin-example):** This repository contains example plugins for Velero\n\n\n## Community Roles\n\n* **Users:** Members that engage with the Velero community via any medium (Slack, GitHub, mailing lists, etc.).\n* **Contributors:** Regular contributions to projects (documentation, code reviews, responding to issues, participation in proposal discussions, contributing code, etc.). \n* **Maintainers**: The Velero project leaders. They are responsible for the overall health and direction of the project; final reviewers of PRs and responsible for releases. Some Maintainers are responsible for one or more components within a project, acting as technical leads for that component. Maintainers are expected to contribute code and documentation, review PRs including ensuring quality of code, triage issues, proactively fix bugs, and perform maintenance tasks for these components.\n\n### Maintainers\n\nNew maintainers must be nominated by an existing maintainer and must be elected by a supermajority of existing maintainers. Likewise, maintainers can be removed by a supermajority of the existing maintainers or can resign by notifying one of the maintainers.\n\n### Supermajority\n\nA supermajority is defined as two-thirds of members in the group.\nA supermajority of [Maintainers](#maintainers) is required for certain\ndecisions as outlined above. A supermajority vote is equivalent to the number of votes in favor being at least twice the number of votes against. For example, if you have 5 maintainers, a supermajority vote is 4 votes. Voting on decisions can happen on the mailing list, GitHub, Slack, email, or via a voting service, when appropriate. Maintainers can either vote \"agree, yes, +1\", \"disagree, no, -1\", or \"abstain\". A vote passes when supermajority is met. An abstain vote equals not voting at all.\n\n### Decision Making\n\nIdeally, all project decisions are resolved by consensus. If impossible, any\nmaintainer may call a vote. Unless otherwise specified in this document, any\nvote will be decided by a supermajority of maintainers.\n\nVotes by maintainers belonging to the same company\nwill count as one vote; e.g., 4 maintainers employed by fictional company **Valerium** will\nonly have **one** combined vote. If voting members from a given company do not\nagree, the company's vote is determined by a supermajority of voters from that\ncompany. If no supermajority is achieved, the company is considered to have\nabstained.\n\n## Proposal Process\n\nOne of the most important aspects in any open source community is the concept\nof proposals. Large changes to the codebase and / or new features should be\npreceded by a proposal in our community repo. This process allows for all\nmembers of the community to weigh in on the concept (including the technical\ndetails), share their comments and ideas, and offer to help. It also ensures\nthat members are not duplicating work or inadvertently stepping on toes by\nmaking large conflicting changes.\n\nThe project roadmap is defined by accepted proposals.\n\nProposals should cover the high-level objectives, use cases, and technical\nrecommendations on how to implement. In general, the community member(s)\ninterested in implementing the proposal should be either deeply engaged in the\nproposal process or be an author of the proposal.\n\nThe proposal should be documented as a separated markdown file pushed to the root of the \n`design` folder in the [Velero](https://github.com/vmware-tanzu/velero/tree/main/design)\nrepository via PR. The name of the file should follow the name pattern `_design.md`, e.g:\n`restore-hooks-design.md`.\n\nUse the [Proposal Template](https://github.com/vmware-tanzu/velero/blob/main/design/_template.md) as a starting point.\n\n### Proposal Lifecycle\n\nThe proposal PR can follow the GitHub lifecycle of the PR to indicate its status:\n\n* **Open**: Proposal is created and under review and discussion.\n* **Merged**: Proposal has been reviewed and is accepted (either by consensus or through a vote).\n* **Closed**: Proposal has been reviewed and was rejected (either by consensus or through a vote).\n\n## Lazy Consensus\n\nTo maintain velocity in a project as busy as Velero, the concept of [Lazy\nConsensus](http://en.osswiki.info/concepts/lazy_consensus) is practiced. Ideas\nand / or proposals should be shared by maintainers via\nGitHub with the appropriate maintainer groups (e.g.,\n`@vmware-tanzu/velero-maintainers`) tagged. Out of respect for other contributors,\nmajor changes should also be accompanied by a ping on Slack or a note on the\nVelero mailing list as appropriate. Author(s) of proposal, Pull Requests,\nissues, etc. will give a time period of no less than five (5) working days for\ncomment and remain cognizant of popular observed world holidays.\n\nOther maintainers may chime in and request additional time for review, but\nshould remain cognizant of blocking progress and abstain from delaying\nprogress unless absolutely needed. The expectation is that blocking progress\nis accompanied by a guarantee to review and respond to the relevant action(s)\n(proposals, PRs, issues, etc.) in short order.\n\nLazy Consensus is practiced for all projects in the `Velero` org, including\nthe main project repository and the additional repositories.\n\nLazy consensus does _not_ apply to the process of:\n\n* Removal of maintainers from Velero\n\n## Deprecation Policy\n\n### Deprecation Process\n\nAny contributor may introduce a request to deprecate a feature or an option of a feature by opening a feature request issue in the vmware-tanzu/velero GitHub project. The issue should describe why the feature is no longer needed or has become detrimental to Velero, as well as whether and how it has been superseded. The submitter should give as much detail as possible.\n\nOnce the issue is filed, a one-month discussion period begins. Discussions take place within the issue itself as well as in the community meetings. The person who opens the issue, or a maintainer, should add the date and time marking the end of the discussion period in a comment on the issue as soon as possible after it is opened. A decision on the issue needs to be made within this one-month period.\n\nThe feature will be deprecated by a supermajority vote of 50% plus one of the project maintainers at the time of the vote tallying, which is 72 hours after the end of the community meeting that is the end of the comment period. (Maintainers are permitted to vote in advance of the deadline, but should hold their votes until as close as possible to hear all possible discussion.) Votes will be tallied in comments on the issue. \n\nNon-maintainers may add non-binding votes in comments to the issue as well; these are opinions to be taken into consideration by maintainers, but they do not count as votes. \n\nIf the vote passes, the deprecation window takes effect in the subsequent release, and the removal follows the schedule. \n\n### Schedule\nIf depreciation proposal passes by supermajority votes, the feature is deprecated in the next minor release and the feature can be removed completely after two minor version or equivalent major version e.g., if feature gets deprecated in Nth minor version, then feature can be removed after N+2 minor version or its equivalent if the major version number changes.\n\n### Deprecation Window\n\nThe deprecation window is the period from the release in which the deprecation takes effect through the release in which the feature is removed. During this period, only critical security vulnerabilities and catastrophic bugs should be fixed.\n\n**Note:** If a backup relies on a deprecated feature, then backups made with the last Velero release before this feature is removed must still be restorable in version `n+2`. For instance, something like restic feature support, that might mean that restic is removed from the list of supported uploader types in version `n` but the underlying implementation required to restore from a restic backup won't be removed until release `n+2`.\n\n## Updating Governance\n\nAll substantive changes in Governance require a supermajority agreement by all maintainers.\n" }, { "path": "LICENSE", "content": "Apache License\nVersion 2.0, January 2004\nhttp://www.apache.org/licenses/\n\nTERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n1. Definitions.\n\n\"License\" shall mean the terms and conditions for use, reproduction,\nand distribution as defined by Sections 1 through 9 of this document.\n\n\"Licensor\" shall mean the copyright owner or entity authorized by\nthe copyright owner that is granting the License.\n\n\"Legal Entity\" shall mean the union of the acting entity and all\nother entities that control, are controlled by, or are under common\ncontrol with that entity. For the purposes of this definition,\n\"control\" means (i) the power, direct or indirect, to cause the\ndirection or management of such entity, whether by contract or\notherwise, or (ii) ownership of fifty percent (50%) or more of the\noutstanding shares, or (iii) beneficial ownership of such entity.\n\n\"You\" (or \"Your\") shall mean an individual or Legal Entity\nexercising permissions granted by this License.\n\n\"Source\" form shall mean the preferred form for making modifications,\nincluding but not limited to software source code, documentation\nsource, and configuration files.\n\n\"Object\" form shall mean any form resulting from mechanical\ntransformation or translation of a Source form, including but\nnot limited to compiled object code, generated documentation,\nand conversions to other media types.\n\n\"Work\" shall mean the work of authorship, whether in Source or\nObject form, made available under the License, as indicated by a\ncopyright notice that is included in or attached to the work\n(an example is provided in the Appendix below).\n\n\"Derivative Works\" shall mean any work, whether in Source or Object\nform, that is based on (or derived from) the Work and for which the\neditorial revisions, annotations, elaborations, or other modifications\nrepresent, as a whole, an original work of authorship. For the purposes\nof this License, Derivative Works shall not include works that remain\nseparable from, or merely link (or bind by name) to the interfaces of,\nthe Work and Derivative Works thereof.\n\n\"Contribution\" shall mean any work of authorship, including\nthe original version of the Work and any modifications or additions\nto that Work or Derivative Works thereof, that is intentionally\nsubmitted to Licensor for inclusion in the Work by the copyright owner\nor by an individual or Legal Entity authorized to submit on behalf of\nthe copyright owner. For the purposes of this definition, \"submitted\"\nmeans any form of electronic, verbal, or written communication sent\nto the Licensor or its representatives, including but not limited to\ncommunication on electronic mailing lists, source code control systems,\nand issue tracking systems that are managed by, or on behalf of, the\nLicensor for the purpose of discussing and improving the Work, but\nexcluding communication that is conspicuously marked or otherwise\ndesignated in writing by the copyright owner as \"Not a Contribution.\"\n\n\"Contributor\" shall mean Licensor and any individual or Legal Entity\non behalf of whom a Contribution has been received by Licensor and\nsubsequently incorporated within the Work.\n\n2. Grant of Copyright License. Subject to the terms and conditions of\nthis License, each Contributor hereby grants to You a perpetual,\nworldwide, non-exclusive, no-charge, royalty-free, irrevocable\ncopyright license to reproduce, prepare Derivative Works of,\npublicly display, publicly perform, sublicense, and distribute the\nWork and such Derivative Works in Source or Object form.\n\n3. Grant of Patent License. Subject to the terms and conditions of\nthis License, each Contributor hereby grants to You a perpetual,\nworldwide, non-exclusive, no-charge, royalty-free, irrevocable\n(except as stated in this section) patent license to make, have made,\nuse, offer to sell, sell, import, and otherwise transfer the Work,\nwhere such license applies only to those patent claims licensable\nby such Contributor that are necessarily infringed by their\nContribution(s) alone or by combination of their Contribution(s)\nwith the Work to which such Contribution(s) was submitted. If You\ninstitute patent litigation against any entity (including a\ncross-claim or counterclaim in a lawsuit) alleging that the Work\nor a Contribution incorporated within the Work constitutes direct\nor contributory patent infringement, then any patent licenses\ngranted to You under this License for that Work shall terminate\nas of the date such litigation is filed.\n\n4. Redistribution. You may reproduce and distribute copies of the\nWork or Derivative Works thereof in any medium, with or without\nmodifications, and in Source or Object form, provided that You\nmeet the following conditions:\n\n(a) You must give any other recipients of the Work or\nDerivative Works a copy of this License; and\n\n(b) You must cause any modified files to carry prominent notices\nstating that You changed the files; and\n\n(c) You must retain, in the Source form of any Derivative Works\nthat You distribute, all copyright, patent, trademark, and\nattribution notices from the Source form of the Work,\nexcluding those notices that do not pertain to any part of\nthe Derivative Works; and\n\n(d) If the Work includes a \"NOTICE\" text file as part of its\ndistribution, then any Derivative Works that You distribute must\ninclude a readable copy of the attribution notices contained\nwithin such NOTICE file, excluding those notices that do not\npertain to any part of the Derivative Works, in at least one\nof the following places: within a NOTICE text file distributed\nas part of the Derivative Works; within the Source form or\ndocumentation, if provided along with the Derivative Works; or,\nwithin a display generated by the Derivative Works, if and\nwherever such third-party notices normally appear. The contents\nof the NOTICE file are for informational purposes only and\ndo not modify the License. You may add Your own attribution\nnotices within Derivative Works that You distribute, alongside\nor as an addendum to the NOTICE text from the Work, provided\nthat such additional attribution notices cannot be construed\nas modifying the License.\n\nYou may add Your own copyright statement to Your modifications and\nmay provide additional or different license terms and conditions\nfor use, reproduction, or distribution of Your modifications, or\nfor any such Derivative Works as a whole, provided Your use,\nreproduction, and distribution of the Work otherwise complies with\nthe conditions stated in this License.\n\n5. Submission of Contributions. Unless You explicitly state otherwise,\nany Contribution intentionally submitted for inclusion in the Work\nby You to the Licensor shall be under the terms and conditions of\nthis License, without any additional terms or conditions.\nNotwithstanding the above, nothing herein shall supersede or modify\nthe terms of any separate license agreement you may have executed\nwith Licensor regarding such Contributions.\n\n6. Trademarks. This License does not grant permission to use the trade\nnames, trademarks, service marks, or product names of the Licensor,\nexcept as required for reasonable and customary use in describing the\norigin of the Work and reproducing the content of the NOTICE file.\n\n7. Disclaimer of Warranty. Unless required by applicable law or\nagreed to in writing, Licensor provides the Work (and each\nContributor provides its Contributions) on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\nimplied, including, without limitation, any warranties or conditions\nof TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\nPARTICULAR PURPOSE. You are solely responsible for determining the\nappropriateness of using or redistributing the Work and assume any\nrisks associated with Your exercise of permissions under this License.\n\n8. Limitation of Liability. In no event and under no legal theory,\nwhether in tort (including negligence), contract, or otherwise,\nunless required by applicable law (such as deliberate and grossly\nnegligent acts) or agreed to in writing, shall any Contributor be\nliable to You for damages, including any direct, indirect, special,\nincidental, or consequential damages of any character arising as a\nresult of this License or out of the use or inability to use the\nWork (including but not limited to damages for loss of goodwill,\nwork stoppage, computer failure or malfunction, or any and all\nother commercial damages or losses), even if such Contributor\nhas been advised of the possibility of such damages.\n\n9. Accepting Warranty or Additional Liability. While redistributing\nthe Work or Derivative Works thereof, You may choose to offer,\nand charge a fee for, acceptance of support, warranty, indemnity,\nor other liability obligations and/or rights consistent with this\nLicense. However, in accepting such obligations, You may act only\non Your own behalf and on Your sole responsibility, not on behalf\nof any other Contributor, and only if You agree to indemnify,\ndefend, and hold each Contributor harmless for any liability\nincurred by, or claims asserted against, such Contributor by reason\nof your accepting any such warranty or additional liability.\n\nEND OF TERMS AND CONDITIONS\n\nAPPENDIX: How to apply the Apache License to your work.\n\nTo apply the Apache License to your work, attach the following\nboilerplate notice, with the fields enclosed by brackets \"{}\"\nreplaced with your own identifying information. (Don't include\nthe brackets!) The text should be enclosed in the appropriate\ncomment syntax for the file format. We also recommend that a\nfile or class name and description of purpose be included on the\nsame \"printed page\" as the copyright notice for easier\nidentification within third-party archives.\n\nCopyright {yyyy} {name of copyright owner}\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n" }, { "path": "MAINTAINERS.md", "content": "# Velero Maintainers\n\n[GOVERNANCE.md](https://github.com/vmware-tanzu/velero/blob/main/GOVERNANCE.md) describes governance guidelines and maintainer responsibilities.\n\n## Maintainers\n\n| Maintainer | GitHub ID | Affiliation |\n|---------------------|---------------------------------------------------------------|--------------------------------------------------|\n| Scott Seago | [sseago](https://github.com/sseago) | [OpenShift](https://github.com/openshift) |\n| Daniel Jiang | [reasonerjt](https://github.com/reasonerjt) | Broadcom |\n| Wenkai Yin | [ywk253100](https://github.com/ywk253100) | Broadcom |\n| Xun Jiang | [blackpiglet](https://github.com/blackpiglet) | Broadcom |\n| Shubham Pampattiwar | [shubham-pampattiwar](https://github.com/shubham-pampattiwar) | [OpenShift](https://github.com/openshift) |\n| Yonghui Li | [Lyndon-Li](https://github.com/Lyndon-Li) | Broadcom |\n| Anshul Ahuja | [anshulahuja98](https://github.com/anshulahuja98) | [Microsoft Azure](https://www.github.com/azure/) |\n| Tiger Kaovilai | [kaovilai](https://github.com/kaovilai) | [OpenShift](https://github.com/openshift) |\n\n## Emeritus Maintainers\n* Adnan Abdulhussein ([prydonius](https://github.com/prydonius))\n* Andy Goldstein ([ncdc](https://github.com/ncdc))\n* Steve Kriss ([skriss](https://github.com/skriss))\n* Carlos Panato ([cpanato](https://github.com/cpanato))\n* Nolan Brubaker ([nrb](https://github.com/nrb))\n* Ashish Amarnath ([ashish-amarnath](https://github.com/ashish-amarnath))\n* Carlisia Thompson ([carlisia](https://github.com/carlisia))\n* Bridget McErlean ([zubron](https://github.com/zubron))\n* JenTing Hsiao ([jenting](https://github.com/jenting))\n* Dave Smith-Uchida ([dsu-igeek](https://github.com/dsu-igeek))\n* Ming Qiu ([qiuming-best](https://github.com/qiuming-best))\n" }, { "path": "Makefile", "content": "# Copyright 2016 The Kubernetes Authors.\n#\n# Modifications Copyright 2020 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# The binary to build (just the basename).\nBIN ?= velero\n\n# This repo's root import path (under GOPATH).\nPKG := github.com/vmware-tanzu/velero\n\n# Where to push the docker image.\nREGISTRY ?= velero\n# In order to push images to an insecure registry, follow the two steps:\n# 1. Set \"INSECURE_REGISTRY=true\" \n# 2. Provide your own buildx builder instance by setting \"BUILDX_INSTANCE=your-own-builder-instance\"\n# The builder can be created with the following command:\n# cat << EOF > buildkitd.toml\n# [registry.\"insecure-registry-ip:port\"]\n# http = true\n# insecure = true\n# EOF\n# docker buildx create --name=velero-builder --driver=docker-container --bootstrap --use --config ./buildkitd.toml\n# Refer to https://github.com/docker/buildx/issues/1370#issuecomment-1288516840 for more details\nINSECURE_REGISTRY ?= false\n\n# Image name\nIMAGE ?= $(REGISTRY)/$(BIN)\n\n# We allow the Dockerfile to be configurable to enable the use of custom Dockerfiles\n# that pull base images from different registries.\nVELERO_DOCKERFILE ?= Dockerfile\nVELERO_DOCKERFILE_WINDOWS ?= Dockerfile-Windows\nBUILDER_IMAGE_DOCKERFILE ?= hack/build-image/Dockerfile\n\n# Calculate the realpath of the build-image Dockerfile as we `cd` into the hack/build\n# directory before this Dockerfile is used and any relative path will not be valid.\nBUILDER_IMAGE_DOCKERFILE_REALPATH := $(shell realpath $(BUILDER_IMAGE_DOCKERFILE))\n\n# Build image handling. We push a build image for every changed version of\n# /hack/build-image/Dockerfile. We tag the dockerfile with the short commit hash\n# of the commit that changed it. When determining if there is a build image in\n# the registry to use we look for one that matches the current \"commit\" for the\n# Dockerfile else we make one.\n# In the case where the Dockerfile for the build image has been overridden using\n# the BUILDER_IMAGE_DOCKERFILE variable, we always force a build.\n\nifneq \"$(origin BUILDER_IMAGE_DOCKERFILE)\" \"file\"\n\tBUILDER_IMAGE_TAG := \"custom\"\nelse\n\tBUILDER_IMAGE_TAG := $(shell git log -1 --pretty=%h $(BUILDER_IMAGE_DOCKERFILE))\nendif\n\nBUILDER_IMAGE := $(REGISTRY)/build-image:$(BUILDER_IMAGE_TAG)\nBUILDER_IMAGE_CACHED := $(shell docker images -q ${BUILDER_IMAGE} 2>/dev/null )\n\nHUGO_IMAGE := ghcr.io/gohugoio/hugo\n\n# Which architecture to build - see $(ALL_ARCH) for options.\n# if the 'local' rule is being run, detect the ARCH from 'go env'\n# if it wasn't specified by the caller.\nlocal : ARCH ?= $(shell go env GOOS)-$(shell go env GOARCH)\nARCH ?= linux-amd64\n\nVERSION ?= main\n\nTAG_LATEST ?= false\n\nifeq ($(TAG_LATEST), true)\n\tIMAGE_TAGS ?= $(IMAGE):$(VERSION) $(IMAGE):latest\nelse\n\tIMAGE_TAGS ?= $(IMAGE):$(VERSION)\nendif\n\n# check buildx is enabled only if docker is in path\n# macOS/Windows docker cli without Docker Desktop license: https://github.com/abiosoft/colima\n# To add buildx to docker cli: https://github.com/abiosoft/colima/discussions/273#discussioncomment-2684502\nifeq ($(shell which docker 2>/dev/null 1>&2 && docker buildx inspect 2>/dev/null | awk '/Status/ { print $$2 }'), running)\n\tBUILDX_ENABLED ?= true\n# if emulated docker cli from podman, assume enabled\n# emulated docker cli from podman: https://podman-desktop.io/docs/migrating-from-docker/emulating-docker-cli-with-podman\n# podman known issues:\n# - on remote podman, such as on macOS,\n# --output issue: https://github.com/containers/podman/issues/15922\nelse ifeq ($(shell which docker 2>/dev/null 1>&2 && cat $(shell which docker) | grep -c \"exec podman\"), 1)\n\tBUILDX_ENABLED ?= true\nelse\n\tBUILDX_ENABLED ?= false\nendif\n\ndefine BUILDX_ERROR\nbuildx not enabled, refusing to run this recipe\nsee: https://velero.io/docs/main/build-from-source/#making-images-and-updating-velero for more info\nendef\n# comma cannot be escaped and can only be used in Make function arguments by putting into variable\ncomma=,\n# The version of restic binary to be downloaded\nRESTIC_VERSION ?= 0.15.0\n\nCLI_PLATFORMS ?= linux-amd64 linux-arm linux-arm64 darwin-amd64 darwin-arm64 windows-amd64 linux-ppc64le linux-s390x\nBUILD_OUTPUT_TYPE ?= docker\nBUILD_OS ?= linux\nBUILD_ARCH ?= amd64\nBUILD_WINDOWS_VERSION ?= ltsc2022\n\nifeq ($(BUILD_OUTPUT_TYPE), docker)\n\tALL_OS = linux\n\tALL_ARCH.linux = $(word 2, $(subst -, ,$(shell go env GOOS)-$(shell go env GOARCH)))\nelse\n\tALL_OS = $(subst $(comma), ,$(BUILD_OS))\n\tALL_ARCH.linux = $(subst $(comma), ,$(BUILD_ARCH))\nendif\n\nALL_ARCH.windows = $(if $(filter windows,$(ALL_OS)),amd64,)\nALL_OSVERSIONS.windows = $(if $(filter windows,$(ALL_OS)),$(BUILD_WINDOWS_VERSION),)\nALL_OS_ARCH.linux = $(foreach os, $(filter linux,$(ALL_OS)), $(foreach arch, ${ALL_ARCH.linux}, ${os}-$(arch)))\nALL_OS_ARCH.windows = $(foreach os, $(filter windows,$(ALL_OS)), $(foreach arch, $(ALL_ARCH.windows), $(foreach osversion, ${ALL_OSVERSIONS.windows}, ${os}-${osversion}-${arch})))\nALL_OS_ARCH = $(ALL_OS_ARCH.linux)$(ALL_OS_ARCH.windows)\n\nALL_IMAGE_TAGS = $(IMAGE_TAGS)\n\n# set git sha and tree state\nGIT_SHA = $(shell git rev-parse HEAD)\nifneq ($(shell git status --porcelain 2> /dev/null),)\n\tGIT_TREE_STATE ?= dirty\nelse\n\tGIT_TREE_STATE ?= clean\nendif\n\n###\n### These variables should not need tweaking.\n###\n\nplatform_temp = $(subst -, ,$(ARCH))\nGOOS = $(word 1, $(platform_temp))\nGOARCH = $(word 2, $(platform_temp))\nGOPROXY ?= https://proxy.golang.org\nGOBIN=$$(pwd)/.go/bin\n\n# If you want to build all binaries, see the 'all-build' rule.\n# If you want to build all containers, see the 'all-containers' rule.\nall:\n\t@$(MAKE) build\n\nbuild-%:\n\t@$(MAKE) --no-print-directory ARCH=$* build\n\nall-build: $(addprefix build-, $(CLI_PLATFORMS))\n\nall-containers:\n\t@$(MAKE) --no-print-directory container\n\nlocal: build-dirs\n# Add DEBUG=1 to enable debug locally\n\tGOOS=$(GOOS) \\\n\tGOARCH=$(GOARCH) \\\n\tGOBIN=$(GOBIN) \\\n\tVERSION=$(VERSION) \\\n\tREGISTRY=$(REGISTRY) \\\n\tPKG=$(PKG) \\\n\tBIN=$(BIN) \\\n\tGIT_SHA=$(GIT_SHA) \\\n\tGIT_TREE_STATE=$(GIT_TREE_STATE) \\\n\tOUTPUT_DIR=$$(pwd)/_output/bin/$(GOOS)/$(GOARCH) \\\n\t./hack/build.sh\n\nbuild: _output/bin/$(GOOS)/$(GOARCH)/$(BIN)\n\n_output/bin/$(GOOS)/$(GOARCH)/$(BIN): build-dirs\n\t@echo \"building: $@\"\n\t$(MAKE) shell CMD=\"-c '\\\n\t\tGOOS=$(GOOS) \\\n\t\tGOARCH=$(GOARCH) \\\n\t\tGOBIN=$(GOBIN) \\\n\t\tVERSION=$(VERSION) \\\n\t\tREGISTRY=$(REGISTRY) \\\n\t\tPKG=$(PKG) \\\n\t\tBIN=$(BIN) \\\n\t\tGIT_SHA=$(GIT_SHA) \\\n\t\tGIT_TREE_STATE=$(GIT_TREE_STATE) \\\n\t\tOUTPUT_DIR=/output/$(GOOS)/$(GOARCH) \\\n\t\t./hack/build.sh'\"\n\nTTY := $(shell tty -s && echo \"-t\")\n\n# Example: make shell CMD=\"date > datefile\"\nshell: build-dirs build-env\n\t@# bind-mount the Velero root dir in at /github.com/vmware-tanzu/velero\n\t@# because the Kubernetes code-generator tools require the project to\n\t@# exist in a directory hierarchy ending like this (but *NOT* necessarily\n\t@# under $GOPATH).\n\t@docker run \\\n\t\t-e GOFLAGS \\\n\t\t-e GOPROXY \\\n\t\t-i $(TTY) \\\n\t\t--rm \\\n\t\t-u $$(id -u):$$(id -g) \\\n\t\t-v \"$$(pwd):/github.com/vmware-tanzu/velero:delegated\" \\\n\t\t-v \"$$(pwd)/_output/bin:/output:delegated\" \\\n\t\t-v \"$$(pwd)/.go/pkg:/go/pkg:delegated\" \\\n\t\t-v \"$$(pwd)/.go/std:/go/std:delegated\" \\\n\t\t-v \"$$(pwd)/.go/std/$(GOOS)/$(GOARCH):/usr/local/go/pkg/$(GOOS)_$(GOARCH)_static:delegated\" \\\n\t\t-v \"$$(pwd)/.go/go-build:/.cache/go-build:delegated\" \\\n\t\t-v \"$$(pwd)/.go/golangci-lint:/.cache/golangci-lint:delegated\" \\\n\t\t-w /github.com/vmware-tanzu/velero \\\n\t\t$(BUILDER_IMAGE) \\\n\t\t/bin/sh $(CMD)\n\ncontainer:\nifneq ($(BUILDX_ENABLED), true)\n\t$(error $(BUILDX_ERROR))\nendif\n\nifeq ($(BUILDX_INSTANCE),)\n\t@echo creating a buildx instance\n\t-docker buildx rm velero-builder || true\n\t@docker buildx create --use --name=velero-builder\nelse\n\t@echo using a specified buildx instance $(BUILDX_INSTANCE)\n\t@docker buildx use $(BUILDX_INSTANCE)\nendif\n\n\t@mkdir -p _output\n\n\t@for osarch in $(ALL_OS_ARCH); do \\\n\t\t$(MAKE) container-$${osarch}; \\\n\tdone\n\nifeq ($(BUILD_OUTPUT_TYPE), registry)\n\t@for tag in $(ALL_IMAGE_TAGS); do \\\n\t\tIMAGE_TAG=$${tag} $(MAKE) push-manifest; \\\n\tdone\nendif\n\ncontainer-linux-%:\n\t@BUILDX_ARCH=$* $(MAKE) container-linux\n\ncontainer-linux:\n\t@echo \"building container: $(IMAGE):$(VERSION)-linux-$(BUILDX_ARCH)\"\n\n\t@docker buildx build --pull \\\n\t--output=\"type=$(BUILD_OUTPUT_TYPE)$(if $(findstring tar, $(BUILD_OUTPUT_TYPE)),$(comma)dest=_output/$(BIN)-$(VERSION)-linux-$(BUILDX_ARCH).tar,)\" \\\n\t--platform=\"linux/$(BUILDX_ARCH)\" \\\n\t$(addprefix -t , $(addsuffix \"-linux-$(BUILDX_ARCH)\",$(ALL_IMAGE_TAGS))) \\\n\t--build-arg=GOPROXY=$(GOPROXY) \\\n\t--build-arg=PKG=$(PKG) \\\n\t--build-arg=BIN=$(BIN) \\\n\t--build-arg=VERSION=$(VERSION) \\\n\t--build-arg=GIT_SHA=$(GIT_SHA) \\\n\t--build-arg=GIT_TREE_STATE=$(GIT_TREE_STATE) \\\n\t--build-arg=REGISTRY=$(REGISTRY) \\\n\t--build-arg=RESTIC_VERSION=$(RESTIC_VERSION) \\\n\t--provenance=false \\\n\t--sbom=false \\\n\t-f $(VELERO_DOCKERFILE) .\n\n\t@echo \"built container: $(IMAGE):$(VERSION)-linux-$(BUILDX_ARCH)\"\n\ncontainer-windows-%:\n\t@BUILDX_OSVERSION=$(firstword $(subst -, ,$*)) BUILDX_ARCH=$(lastword $(subst -, ,$*)) $(MAKE) container-windows\n\ncontainer-windows:\n\t@echo \"building container: $(IMAGE):$(VERSION)-windows-$(BUILDX_OSVERSION)-$(BUILDX_ARCH)\"\n\n\t@docker buildx build --pull \\\n\t--output=\"type=$(BUILD_OUTPUT_TYPE)$(if $(findstring tar, $(BUILD_OUTPUT_TYPE)),$(comma)dest=_output/$(BIN)-$(VERSION)-windows-$(BUILDX_OSVERSION)-$(BUILDX_ARCH).tar,)\" \\\n\t--platform=\"windows/$(BUILDX_ARCH)\" \\\n\t$(addprefix -t , $(addsuffix \"-windows-$(BUILDX_OSVERSION)-$(BUILDX_ARCH)\",$(ALL_IMAGE_TAGS))) \\\n\t--build-arg=GOPROXY=$(GOPROXY) \\\n\t--build-arg=PKG=$(PKG) \\\n\t--build-arg=BIN=$(BIN) \\\n\t--build-arg=VERSION=$(VERSION) \\\n\t--build-arg=OS_VERSION=$(BUILDX_OSVERSION) \\\n\t--build-arg=GIT_SHA=$(GIT_SHA) \\\n --build-arg=GIT_TREE_STATE=$(GIT_TREE_STATE) \\\n\t--build-arg=REGISTRY=$(REGISTRY) \\\n\t--provenance=false \\\n\t--sbom=false \\\n\t-f $(VELERO_DOCKERFILE_WINDOWS) .\n\n\t@echo \"built container: $(IMAGE):$(VERSION)-windows-$(BUILDX_OSVERSION)-$(BUILDX_ARCH)\"\n\npush-manifest:\n\t@echo \"building manifest: $(IMAGE_TAG) for $(foreach osarch, $(ALL_OS_ARCH), $(IMAGE_TAG)-${osarch})\"\n\t@docker manifest create --amend --insecure=$(INSECURE_REGISTRY) $(IMAGE_TAG) $(foreach osarch, $(ALL_OS_ARCH), $(IMAGE_TAG)-${osarch})\n\n\t@set -x; \\\n\tfor arch in $(ALL_ARCH.windows); do \\\n\t\tfor osversion in $(ALL_OSVERSIONS.windows); do \\\n\t\t\tBASEIMAGE=mcr.microsoft.com/windows/nanoserver:$${osversion}; \\\n\t\t\tfull_version=`docker manifest inspect --insecure=$(INSECURE_REGISTRY) $${BASEIMAGE} | jq -r '.manifests[0].platform[\"os.version\"]'`; \\\n\t\t\tdocker manifest annotate --os windows --arch $${arch} --os-version $${full_version} $(IMAGE_TAG) $(IMAGE_TAG)-windows-$${osversion}-$${arch}; \\\n\t\tdone; \\\n\tdone\n\n\t@echo \"pushing manifest $(IMAGE_TAG)\"\n\t@docker manifest push --purge --insecure=$(INSECURE_REGISTRY) $(IMAGE_TAG)\n\n\t@echo \"pushed manifest $(IMAGE_TAG):\"\n\t@docker manifest inspect --insecure=$(INSECURE_REGISTRY) $(IMAGE_TAG)\n\nSKIP_TESTS ?=\ntest: build-dirs\nifneq ($(SKIP_TESTS), 1)\n\t@$(MAKE) shell CMD=\"-c 'hack/test.sh $(WHAT)'\"\nendif\n\ntest-local: build-dirs\nifneq ($(SKIP_TESTS), 1)\n\thack/test.sh $(WHAT)\nendif\n\nverify:\nifneq ($(SKIP_TESTS), 1)\n\t@$(MAKE) shell CMD=\"-c 'hack/verify-all.sh'\"\nendif\n\nlint:\nifneq ($(SKIP_TESTS), 1)\n\t@$(MAKE) shell CMD=\"-c 'hack/lint.sh'\"\nendif\n\nlocal-lint:\nifneq ($(SKIP_TESTS), 1)\n\t@hack/lint.sh\nendif\n\nupdate:\n\t@$(MAKE) shell CMD=\"-c 'hack/update-all.sh'\"\n\n# update-crd is for development purpose only, it is faster than update, so is a shortcut when you want to generate CRD changes only\nupdate-crd:\n\t@$(MAKE) shell CMD=\"-c 'hack/update-3generated-crd-code.sh'\"\t\n\nbuild-dirs:\n\t@mkdir -p _output/bin/$(GOOS)/$(GOARCH)\n\t@mkdir -p .go/src/$(PKG) .go/pkg .go/bin .go/std/$(GOOS)/$(GOARCH) .go/go-build .go/golangci-lint\n\nbuild-env:\n\t@# if we have overridden the value for the build-image Dockerfile,\n\t@# force a build using that Dockerfile\n\t@# if we detect changes in dockerfile force a new build-image\n\t@# else if we dont have a cached image make one\n\t@# finally use the cached image\nifneq \"$(origin BUILDER_IMAGE_DOCKERFILE)\" \"file\"\n\t@echo \"Dockerfile for builder image has been overridden to $(BUILDER_IMAGE_DOCKERFILE)\"\n\t@echo \"Preparing a new builder-image\"\n\t$(MAKE) build-image\nelse ifneq ($(shell git diff --quiet HEAD -- $(BUILDER_IMAGE_DOCKERFILE); echo $$?), 0)\n\t@echo \"Local changes detected in $(BUILDER_IMAGE_DOCKERFILE)\"\n\t@echo \"Preparing a new builder-image\"\n\t$(MAKE) build-image\nelse ifneq ($(BUILDER_IMAGE_CACHED),)\n\t@echo \"Using Cached Image: $(BUILDER_IMAGE)\"\nelse\n\t@echo \"Trying to pull build-image: $(BUILDER_IMAGE)\"\n\tdocker pull -q $(BUILDER_IMAGE) || $(MAKE) build-image\nendif\n\nbuild-image:\n\t@# When we build a new image we just untag the old one.\n\t@# This makes sure we don't leave the orphaned image behind.\n\t$(eval old_id=$(shell docker image inspect --format '{{ .ID }}' ${BUILDER_IMAGE} 2>/dev/null))\nifeq ($(BUILDX_ENABLED), true)\n\t@cd hack/build-image && docker buildx build --build-arg=GOPROXY=$(GOPROXY) --output=type=docker --pull -t $(BUILDER_IMAGE) -f $(BUILDER_IMAGE_DOCKERFILE_REALPATH) .\nelse\n\t@cd hack/build-image && docker build --build-arg=GOPROXY=$(GOPROXY) --pull -t $(BUILDER_IMAGE) -f $(BUILDER_IMAGE_DOCKERFILE_REALPATH) .\nendif\n\t$(eval new_id=$(shell docker image inspect --format '{{ .ID }}' ${BUILDER_IMAGE} 2>/dev/null))\n\t@if [ \"$(old_id)\" != \"\" ] && [ \"$(old_id)\" != \"$(new_id)\" ]; then \\\n\t\tdocker rmi -f $$id || true; \\\n\tfi\n\npush-build-image:\n\t@# this target will push the build-image it assumes you already have docker\n\t@# credentials needed to accomplish this.\n\t@# Pushing will be skipped if a custom Dockerfile was used to build the image.\nifneq \"$(origin BUILDER_IMAGE_DOCKERFILE)\" \"file\"\n\t@echo \"Dockerfile for builder image has been overridden\"\n\t@echo \"Skipping push of custom image\"\nelse\n\tdocker push $(BUILDER_IMAGE)\nendif\n\nbuild-image-hugo:\n\tcd site && docker build --pull -t $(HUGO_IMAGE) .\n\nclean:\n# if we have a cached image then use it to run go clean --modcache\n# this test checks if we there is an image id in the BUILDER_IMAGE_CACHED variable.\nifneq ($(strip $(BUILDER_IMAGE_CACHED)),)\n\t$(MAKE) shell CMD=\"-c 'go clean --modcache'\"\n\tdocker rmi -f $(BUILDER_IMAGE) || true\nendif\n\trm -rf .go _output\n\tdocker rmi $(HUGO_IMAGE)\n\n\n.PHONY: modules\nmodules:\n\tgo mod tidy\n\n\n.PHONY: verify-modules\nverify-modules: modules\n\t@if !(git diff --quiet HEAD -- go.sum go.mod); then \\\n\t\techo \"go module files are out of date, please commit the changes to go.mod and go.sum\"; exit 1; \\\n\tfi\n\n\nci: verify-modules verify all test\n\n\nchangelog:\n\thack/release-tools/changelog.sh\n\n# release builds a GitHub release using goreleaser within the build container.\n#\n# To dry-run the release, which will build the binaries/artifacts locally but\n# will *not* create a GitHub release:\n#\t\tGITHUB_TOKEN=an-invalid-token-so-you-dont-accidentally-push-release \\\n#\t\tRELEASE_NOTES_FILE=changelogs/CHANGELOG-1.2.md \\\n#\t\tPUBLISH=false \\\n#\t\tmake release\n#\n# To run the release, which will publish a *DRAFT* GitHub release in github.com/vmware-tanzu/velero\n# (you still need to review/publish the GitHub release manually):\n#\t\tGITHUB_TOKEN=your-github-token \\\n#\t\tRELEASE_NOTES_FILE=changelogs/CHANGELOG-1.2.md \\\n#\t\tPUBLISH=true \\\n#\t\tmake release\nrelease:\n\t$(MAKE) shell CMD=\"-c '\\\n\t\tGITHUB_TOKEN=$(GITHUB_TOKEN) \\\n\t\tRELEASE_NOTES_FILE=$(RELEASE_NOTES_FILE) \\\n\t\tPUBLISH=$(PUBLISH) \\\n\t\tREGISTRY=$(REGISTRY) \\\n\t\t./hack/release-tools/goreleaser.sh'\"\n\nserve-docs: build-image-hugo\n\tdocker run \\\n\t--rm \\\n\t-v \"$$(pwd)/site:/project\" \\\n\t-it -p 1313:1313 \\\n\t$(HUGO_IMAGE) \\\n\tserver --bind=0.0.0.0 --enableGitInfo=false\n# gen-docs generates a new versioned docs directory under site/content/docs.\n# Please read the documentation in the script for instructions on how to use it.\ngen-docs:\n\t@hack/release-tools/gen-docs.sh\n\n.PHONY: test-e2e\ntest-e2e: local\n\t$(MAKE) -e VERSION=$(VERSION) -C test/ run-e2e\n\n.PHONY: test-perf\ntest-perf: local\n\t$(MAKE) -e VERSION=$(VERSION) -C test/ run-perf\n\ngo-generate:\n\tgo generate ./pkg/...\n\n# requires an authenticated gh cli\n# gh: https://cli.github.com/\n# First create a PR\n# gh pr create --title 'Title name' --body 'PR body'\n# by default uses PR title as changelog body but can be overwritten like so\n# make new-changelog CHANGELOG_BODY=\"Changes you have made\"\nnew-changelog: GH_LOGIN ?= $(shell gh pr view --json author --jq .author.login 2> /dev/null)\nnew-changelog: GH_PR_NUMBER ?= $(shell gh pr view --json number --jq .number 2> /dev/null)\nnew-changelog: CHANGELOG_BODY ?= '$(shell gh pr view --json title --jq .title)'\nnew-changelog:\n\t@if [ \"$(GH_LOGIN)\" = \"\" ]; then \\\n\t\techo \"branch does not have PR or cli not logged in, try 'gh auth login' or 'gh pr create'\"; \\\n\t\texit 1; \\\n\tfi\n\t@mkdir -p ./changelogs/unreleased/ && \\\n\techo $(CHANGELOG_BODY) > ./changelogs/unreleased/$(GH_PR_NUMBER)-$(GH_LOGIN) && \\\n\techo \\\"$(CHANGELOG_BODY)\\\" added to \"./changelogs/unreleased/$(GH_PR_NUMBER)-$(GH_LOGIN)\"" }, { "path": "OWNERS", "content": "# This file is used by the [PROW action](https://github.com/jpmcb/prow-github-actions) to approve and merge PRs.\r\n# The file's format follows the [OWNERS SPEC](https://www.kubernetes.dev/docs/guide/owners/#owners-spec).\r\n\r\n# List of usernames who may use /lgtm\r\nreviewers:\r\n- @Lyndon-Li\r\n- @anshulahuja98\r\n- @blackpiglet\r\n- @qiuming-best\r\n- @reasonerjt\r\n- @shubham-pampattiwar\r\n- @sseago\r\n- @ywk253100\r\n\r\n# List of usernames who may use /approve\r\napprovers:\r\n- @Lyndon-Li\r\n- @anshulahuja98\r\n- @blackpiglet\r\n- @qiuming-best\r\n- @reasonerjt\r\n- @shubham-pampattiwar\r\n- @sseago\r\n- @ywk253100\r\n" }, { "path": "README.md", "content": "![100]\n\n[![Build Status][1]][2] [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/3811/badge)](https://bestpractices.coreinfrastructure.org/projects/3811)\n![GitHub release (latest SemVer)](https://img.shields.io/github/v/release/vmware-tanzu/velero)\n\n## Overview\n\nVelero (formerly Heptio Ark) gives you tools to back up and restore your Kubernetes cluster resources and persistent volumes. You can run Velero with a public cloud platform or on-premises. \n\nVelero lets you:\n\n* Take backups of your cluster and restore in case of loss.\n* Migrate cluster resources to other clusters.\n* Replicate your production cluster to development and testing clusters.\n\nVelero consists of:\n\n* A server that runs on your cluster\n* A command-line client that runs locally\n\n## Documentation\n\n[The documentation][29] provides a getting started guide and information about building from source, architecture, extending Velero and more.\n\nPlease use the version selector at the top of the site to ensure you are using the appropriate documentation for your version of Velero.\n\n## Troubleshooting\n\nIf you encounter issues, review the [troubleshooting docs][30], [file an issue][4], or talk to us on the [#velero channel][25] on the Kubernetes Slack server.\n\n## Contributing\n\nIf you are ready to jump in and test, add code, or help with documentation, follow the instructions on our [Start contributing][31] documentation for guidance on how to setup Velero for development.\n\n## Changelog\n\nSee [the list of releases][6] to find out about feature changes.\n\n### Velero compatibility matrix\n\nThe following is a list of the supported Kubernetes versions for each Velero version.\n\n| Velero version | Expected Kubernetes version compatibility | Tested on Kubernetes version |\n|----------------|-------------------------------------------|-------------------------------------|\n| 1.18 | 1.18-latest | 1.33.7, 1.34.1, and 1.35.0 |\n| 1.17 | 1.18-latest | 1.31.7, 1.32.3, 1.33.1, and 1.34.0 |\n| 1.16 | 1.18-latest | 1.31.4, 1.32.3, and 1.33.0 |\n| 1.15 | 1.18-latest | 1.28.8, 1.29.8, 1.30.4 and 1.31.1 |\n| 1.14 | 1.18-latest | 1.27.9, 1.28.9, and 1.29.4 |\n\nVelero supports IPv4, IPv6, and dual stack environments. Support for this was tested against Velero v1.8.\n\nThe Velero maintainers are continuously working to expand testing coverage, but are not able to test every combination of Velero and supported Kubernetes versions for each Velero release. The table above is meant to track the current testing coverage and the expected supported Kubernetes versions for each Velero version.\n\nIf you are interested in using a different version of Kubernetes with a given Velero version, we'd recommend that you perform testing before installing or upgrading your environment. For full information around capabilities within a release, also see the Velero [release notes](https://github.com/vmware-tanzu/velero/releases) or Kubernetes [release notes](https://github.com/kubernetes/kubernetes/tree/master/CHANGELOG). See the Velero [support page](https://velero.io/docs/latest/support-process/) for information about supported versions of Velero.\n\nFor each release, Velero maintainers run the test to ensure the upgrade path from n-2 minor release. For example, before the release of v1.10.x, the test will verify that the backup created by v1.9.x and v1.8.x can be restored using the build to be tagged as v1.10.x.\n\n[1]: https://github.com/vmware-tanzu/velero/workflows/Main%20CI/badge.svg\n[2]: https://github.com/vmware-tanzu/velero/actions?query=workflow%3A\"Main+CI\"\n[4]: https://github.com/vmware-tanzu/velero/issues\n[6]: https://github.com/vmware-tanzu/velero/releases\n[9]: https://kubernetes.io/docs/setup/\n[10]: https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-with-homebrew-on-macos\n[11]: https://kubernetes.io/docs/tasks/tools/install-kubectl/#tabset-1\n[12]: https://github.com/kubernetes/kubernetes/blob/master/cluster/addons/dns/README.md\n[14]: https://github.com/kubernetes/kubernetes\n[24]: https://groups.google.com/forum/#!forum/projectvelero\n[25]: https://kubernetes.slack.com/messages/velero\n[29]: https://velero.io/docs/\n[30]: https://velero.io/docs/troubleshooting\n[31]: https://velero.io/docs/start-contributing\n[100]: https://velero.io/docs/main/img/velero.png\n" }, { "path": "ROADMAP.md", "content": "# Please go to the [Velero Wiki](https://github.com/vmware-tanzu/velero/wiki/) to see our latest roadmap, archived roadmaps and roadmap guidance." }, { "path": "SECURITY.md", "content": "# Security Release Process\n\nVelero is an open source tool with a growing community devoted to safe backup and restore, disaster recovery, and data migration of Kubernetes resources and persistent volumes. The community has adopted this security disclosure and response policy to ensure we responsibly handle critical issues.\n\n\n## Supported Versions\n\nThe Velero project maintains the following [governance document](https://github.com/vmware-tanzu/velero/blob/main/GOVERNANCE.md), [release document](https://github.com/vmware-tanzu/velero/blob/f42c63af1b9af445e38f78a7256b1c48ef79c10e/site/docs/main/release-instructions.md), and [support document](https://velero.io/docs/main/support-process/). Please refer to these for release and related details. Only the most recent version of Velero is supported. Each [release](https://github.com/vmware-tanzu/velero/releases) includes information about upgrading to the latest version.\n\n\n## Reporting a Vulnerability - Private Disclosure Process\n\nSecurity is of the highest importance and all security vulnerabilities or suspected security vulnerabilities should be reported to Velero privately, to minimize attacks against current users of Velero before they are fixed. Vulnerabilities will be investigated and patched on the next patch (or minor) release as soon as possible. This information could be kept entirely internal to the project. \n\nIf you know of a publicly disclosed security vulnerability for Velero, please **IMMEDIATELY** contact the Security Team (velero-security.pdl@broadcom.com).\n\n \n\n**IMPORTANT: Do not file public issues on GitHub for security vulnerabilities**\n\nTo report a vulnerability or a security-related issue, please contact the email address with the details of the vulnerability. The email will be fielded by the Security Team and then shared with the Velero maintainers who have committer and release permissions. Emails will be addressed within 3 business days, including a detailed plan to investigate the issue and any potential workarounds to perform in the meantime. Do not report non-security-impacting bugs through this channel. Use [GitHub issues](https://github.com/vmware-tanzu/velero/issues/new/choose) instead.\n\n\n## Proposed Email Content\n\nProvide a descriptive subject line and in the body of the email include the following information:\n\n\n\n* Basic identity information, such as your name and your affiliation or company.\n* Detailed steps to reproduce the vulnerability (POC scripts, screenshots, and logs are all helpful to us).\n* Description of the effects of the vulnerability on Velero and the related hardware and software configurations, so that the Security Team can reproduce it.\n* How the vulnerability affects Velero usage and an estimation of the attack surface, if there is one.\n* List other projects or dependencies that were used in conjunction with Velero to produce the vulnerability.\n\n \n\n\n## When to report a vulnerability\n\n\n\n* When you think Velero has a potential security vulnerability.\n* When you suspect a potential vulnerability but you are unsure that it impacts Velero.\n* When you know of or suspect a potential vulnerability on another project that is used by Velero.\n\n \n\n\n## Patch, Release, and Disclosure\n\nThe Security Team will respond to vulnerability reports as follows:\n\n \n\n\n\n1. The Security Team will investigate the vulnerability and determine its effects and criticality.\n2. If the issue is not deemed to be a vulnerability, the Security Team will follow up with a detailed reason for rejection.\n3. The Security Team will initiate a conversation with the reporter within 3 business days.\n4. If a vulnerability is acknowledged and the timeline for a fix is determined, the Security Team will work on a plan to communicate with the appropriate community, including identifying mitigating steps that affected users can take to protect themselves until the fix is rolled out.\n5. The Security Team will also create a [CVSS](https://www.first.org/cvss/specification-document) using the [CVSS Calculator](https://www.first.org/cvss/calculator/3.0). The Security Team makes the final call on the calculated CVSS; it is better to move quickly than making the CVSS perfect. Issues may also be reported to [Mitre](https://cve.mitre.org/) using this [scoring calculator](https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator). The CVE will initially be set to private.\n6. The Security Team will work on fixing the vulnerability and perform internal testing before preparing to roll out the fix.\n7. The Security Team will provide early disclosure of the vulnerability by emailing the [Velero Distributors](https://groups.google.com/u/1/g/projectvelero-distributors) mailing list. Distributors can initially plan for the vulnerability patch ahead of the fix, and later can test the fix and provide feedback to the Velero team. See the section **Early Disclosure to Velero Distributors List** for details about how to join this mailing list. \n8. A public disclosure date is negotiated by the SecurityTeam, the bug submitter, and the distributors list. We prefer to fully disclose the bug as soon as possible once a user mitigation or patch is available. It is reasonable to delay disclosure when the bug or the fix is not yet fully understood, the solution is not well-tested, or for distributor coordination. The timeframe for disclosure is from immediate (especially if it’s already publicly known) to a few weeks. For a critical vulnerability with a straightforward mitigation, we expect the report date for the public disclosure date to be on the order of 14 business days. The Security Team holds the final say when setting a public disclosure date.\n9. Once the fix is confirmed, the Security Team will patch the vulnerability in the next patch or minor release, and backport a patch release into all earlier supported releases. Upon release of the patched version of Velero, we will follow the **Public Disclosure Process**.\n\n\n## Public Disclosure Process\n\nThe Security Team publishes a [public advisory](https://github.com/vmware-tanzu/velero/security/advisories) to the Velero community via GitHub. In most cases, additional communication via Slack, Twitter, mailing lists, blog and other channels will assist in educating Velero users and rolling out the patched release to affected users. \n\nThe Security Team will also publish any mitigating steps users can take until the fix can be applied to their Velero instances. Velero distributors will handle creating and publishing their own security advisories.\n\n \n\n\n## Mailing lists\n\n\n\n* Use velero-security.pdl@broadcom.com to report security concerns to the Security Team, who uses the list to privately discuss security issues and fixes prior to disclosure.\n* Join the [Velero Distributors](https://groups.google.com/u/1/g/projectvelero-distributors) mailing list for early private information and vulnerability disclosure. Early disclosure may include mitigating steps and additional information on security patch releases. See below for information on how Velero distributors or vendors can apply to join this list.\n\n\n## Early Disclosure to Velero Distributors List\n\nThe private list is intended to be used primarily to provide actionable information to multiple distributor projects at once. This list is not intended to inform individuals about security issues. \n\n\n## Membership Criteria\n\nTo be eligible to join the [Velero Distributors](https://groups.google.com/u/1/g/projectvelero-distributors) mailing list, you should:\n\n\n\n1. Be an active distributor of Velero.\n2. Have a user base that is not limited to your own organization.\n3. Have a publicly verifiable track record up to the present day of fixing security issues.\n4. Not be a downstream or rebuild of another distributor.\n5. Be a participant and active contributor in the Velero community.\n6. Accept the Embargo Policy that is outlined below. \n7. Have someone who is already on the list vouch for the person requesting membership on behalf of your distribution.\n\n**The terms and conditions of the Embargo Policy apply to all members of this mailing list. A request for membership represents your acceptance to the terms and conditions of the Embargo Policy.**\n\n\n## Embargo Policy\n\nThe information that members receive on the Velero Distributors mailing list must not be made public, shared, or even hinted at anywhere beyond those who need to know within your specific team, unless you receive explicit approval to do so from the Security Team. This remains true until the public disclosure date/time agreed upon by the list. Members of the list and others cannot use the information for any reason other than to get the issue fixed for your respective distribution's users.\n\nBefore you share any information from the list with members of your team who are required to fix the issue, these team members must agree to the same terms, and only be provided with information on a need-to-know basis.\n\nIn the unfortunate event that you share information beyond what is permitted by this policy, you must urgently inform the Security Team (velero-security.pdl@broadcom.com) of exactly what information was leaked and to whom. If you continue to leak information and break the policy outlined here, you will be permanently removed from the list.\n\n \n\n\n## Requesting to Join\n\nSend new membership requests to projectvelero-distributors@googlegroups.com. In the body of your request please specify how you qualify for membership and fulfill each criterion listed in the Membership Criteria section above.\n\n\n## Confidentiality, integrity and availability\n\nWe consider vulnerabilities leading to the compromise of data confidentiality, elevation of privilege, or integrity to be our highest priority concerns. Availability, in particular in areas relating to DoS and resource exhaustion, is also a serious security concern. The Security Team takes all vulnerabilities, potential vulnerabilities, and suspected vulnerabilities seriously and will investigate them in an urgent and expeditious manner.\n\nNote that we do not currently consider the default settings for Velero to be secure-by-default. It is necessary for operators to explicitly configure settings, role based access control, and other resource related features in Velero to provide a hardened Velero environment. We will not act on any security disclosure that relates to a lack of safe defaults. Over time, we will work towards improved safe-by-default configuration, taking into account backwards compatibility.\n" }, { "path": "SUPPORT.md", "content": "# Velero Support\n\nThanks for trying out Velero! We welcome all feedback, find all the ways to connect with us on our Community page:\n\n- [Velero Community](https://velero.io/community/)\n\nYou can find details on the Velero maintainers' support process [here](https://velero.io/docs/main/support-process/).\n" }, { "path": "Tiltfile", "content": "# -*- mode: Python -*-\n\nk8s_yaml([\n 'config/crd/v1/bases/velero.io_backups.yaml',\n 'config/crd/v1/bases/velero.io_backupstoragelocations.yaml',\n 'config/crd/v1/bases/velero.io_deletebackuprequests.yaml',\n 'config/crd/v1/bases/velero.io_downloadrequests.yaml',\n 'config/crd/v1/bases/velero.io_podvolumebackups.yaml',\n 'config/crd/v1/bases/velero.io_podvolumerestores.yaml',\n 'config/crd/v1/bases/velero.io_backuprepositories.yaml',\n 'config/crd/v1/bases/velero.io_restores.yaml',\n 'config/crd/v1/bases/velero.io_schedules.yaml',\n 'config/crd/v1/bases/velero.io_serverstatusrequests.yaml',\n 'config/crd/v1/bases/velero.io_volumesnapshotlocations.yaml',\n 'config/crd/v2alpha1/bases/velero.io_datauploads.yaml',\n 'config/crd/v2alpha1/bases/velero.io_datadownloads.yaml', \n])\n\n# default values\nsettings = {\n \"default_registry\": \"docker.io/velero\",\n \"use_node_agent\": False,\n \"enable_debug\": False,\n \"debug_continue_on_start\": True, # Continue the velero process by default when in debug mode\n \"create_backup_locations\": False,\n \"setup-minio\": False,\n}\n\n# global settings\nsettings.update(read_json(\n \"tilt-resources/tilt-settings.json\",\n default = {},\n))\n\nk8s_yaml(kustomize('tilt-resources'))\nk8s_yaml('tilt-resources/deployment.yaml')\nif settings.get(\"enable_debug\"):\n k8s_resource('velero', port_forwards = '2345')\n # TODO: Need to figure out how to apply port forwards for all node-agent pods\nif settings.get(\"use_node_agent\"):\n k8s_yaml('tilt-resources/node-agent.yaml')\nif settings.get(\"create_backup_locations\"):\n k8s_yaml('tilt-resources/velero_v1_backupstoragelocation.yaml')\nif settings.get(\"setup-minio\"):\n k8s_yaml('examples/minio/00-minio-deployment.yaml', allow_duplicates = True)\n\n# By default, Tilt automatically allows Minikube, Docker for Desktop, Microk8s, Red Hat CodeReady Containers, Kind, K3D, and Krucible.\nallow_k8s_contexts(settings.get(\"allowed_contexts\"))\ndefault_registry(settings.get(\"default_registry\"))\nlocal_goos = str(local(\"go env GOOS\", quiet = True, echo_off = True)).strip()\ngit_sha = str(local(\"git rev-parse HEAD\", quiet = True, echo_off = True)).strip()\n\ntilt_helper_dockerfile_header = \"\"\"\n# Tilt image\nFROM golang:1.25 as tilt-helper\n\n# Support live reloading with Tilt\nRUN wget --output-document /restart.sh --quiet https://raw.githubusercontent.com/windmilleng/rerun-process-wrapper/master/restart.sh && \\\n wget --output-document /start.sh --quiet https://raw.githubusercontent.com/windmilleng/rerun-process-wrapper/master/start.sh && \\\n chmod +x /start.sh && chmod +x /restart.sh\n\"\"\"\n\nadditional_docker_helper_commands = \"\"\"\n# Install delve to allow debugging\nRUN go install github.com/go-delve/delve/cmd/dlv@latest\n\nRUN wget -qO- https://dl.k8s.io/v1.25.2/kubernetes-client-linux-amd64.tar.gz | tar xvz\nRUN wget -qO- https://get.docker.com | sh\n\"\"\"\n\nadditional_docker_build_commands = \"\"\"\nCOPY --from=tilt-helper /go/bin/dlv /usr/bin/dlv\nCOPY --from=tilt-helper /usr/bin/docker /usr/bin/docker\nCOPY --from=tilt-helper /go/kubernetes/client/bin/kubectl /usr/bin/kubectl\n\"\"\"\n\n##############################\n# Setup Velero\n##############################\n\ndef get_debug_flag():\n \"\"\"\n Returns the flag to enable debug building of Velero if debug\n mode is enabled.\n \"\"\"\n\n if settings.get('enable_debug'):\n return \"DEBUG=1\"\n return \"\"\n\n\n# Set up a local_resource build of the Velero binary. The binary is written to _tiltbuild/velero.\nlocal_resource(\n \"velero_server_binary\",\n cmd = 'cd ' + '.' + ';mkdir -p _tiltbuild;PKG=. BIN=velero GOOS=linux GOARCH=amd64 GIT_SHA=' + git_sha + ' VERSION=main GIT_TREE_STATE=dirty OUTPUT_DIR=_tiltbuild ' + get_debug_flag() + ' REGISTRY=' + settings.get(\"default_registry\") + ' ./hack/build.sh',\n deps = [\"cmd\", \"internal\", \"pkg\"],\n ignore = [\"pkg/cmd\"],\n)\n\nlocal_resource(\n \"velero_local_binary\",\n cmd = 'cd ' + '.' + ';mkdir -p _tiltbuild/local;PKG=. BIN=velero GOOS=' + local_goos + ' GOARCH=amd64 GIT_SHA=' + git_sha + ' VERSION=main GIT_TREE_STATE=dirty OUTPUT_DIR=_tiltbuild/local ' + get_debug_flag() + ' REGISTRY=' + settings.get(\"default_registry\") + ' ./hack/build.sh',\n deps = [\"internal\", \"pkg/cmd\"],\n)\n\nlocal_resource(\n \"restic_binary\",\n cmd = 'cd ' + '.' + ';mkdir -p _tiltbuild/restic; BIN=velero GOOS=linux GOARCH=amd64 GOARM=\"\" RESTIC_VERSION=0.13.1 OUTPUT_DIR=_tiltbuild/restic ./hack/build-restic.sh',\n)\n\n# Note: we need a distro with a bash shell to exec into the Velero container\ntilt_dockerfile_header = \"\"\"\nFROM ubuntu:22.04 as tilt\n\nRUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -qq -y ca-certificates tzdata && rm -rf /var/lib/apt/lists/*\n\nWORKDIR /\nCOPY --from=tilt-helper /start.sh .\nCOPY --from=tilt-helper /restart.sh .\nCOPY velero .\nCOPY restic/restic /usr/bin/restic\n\"\"\"\n\ndockerfile_contents = \"\\n\".join([\n tilt_helper_dockerfile_header,\n additional_docker_helper_commands,\n tilt_dockerfile_header,\n additional_docker_build_commands,\n])\n\ndef get_velero_entrypoint():\n \"\"\"\n Returns the entrypoint for the Velero container image.\n \"\"\"\n\n entrypoint = [\"sh\", \"/start.sh\"]\n\n if settings.get(\"enable_debug\"):\n # If debug mode is enabled, start the velero process using Delve\n entrypoint.extend(\n [\"dlv\", \"--listen=:2345\", \"--headless=true\", \"--api-version=2\", \"--accept-multiclient\", \"exec\"])\n\n # Set whether or not to continue the debugged process on start\n # See https://github.com/go-delve/delve/blob/master/Documentation/usage/dlv_exec.md\n if settings.get(\"debug_continue_on_start\"):\n entrypoint.append(\"--continue\")\n\n entrypoint.append(\"--\")\n\n entrypoint.append(\"/velero\")\n\n return entrypoint\n\n\n# Set up an image build for Velero. The live update configuration syncs the output from the local_resource\n# build into the container.\ndocker_build(\n ref = \"velero/velero\",\n context = \"_tiltbuild\",\n dockerfile_contents = dockerfile_contents,\n target = \"tilt\",\n entrypoint = get_velero_entrypoint(),\n live_update = [\n sync(\"./_tiltbuild/velero\", \"/velero\"),\n run(\"sh /restart.sh\"),\n ])\n\n\n##############################\n# Setup plugins\n##############################\n\ndef load_provider_tiltfiles():\n all_providers = settings.get(\"providers\", {})\n enable_providers = settings.get(\"enable_providers\", [])\n providers = []\n\n ## Load settings only for providers to enable\n for name in enable_providers:\n repo = all_providers.get(name)\n if not repo:\n print(\"Enabled provider '{}' does not exist in list of supported providers\".format(name))\n continue\n file = repo + \"/tilt-provider.json\"\n if not os.path.exists(file):\n print(\"Provider settings not found for \\\"{}\\\". Please ensure this plugin repository has a tilt-provider.json file included.\".format(name))\n continue\n provider_details = read_json(file, default = {})\n if type(provider_details) == \"dict\":\n provider_details[\"name\"] = name\n if \"context\" in provider_details:\n provider_details[\"context\"] = os.path.join(repo, \"/\", provider_details[\"context\"])\n else:\n provider_details[\"context\"] = repo\n if \"go_main\" not in provider_details:\n provider_details[\"go_main\"] = \"main.go\"\n providers.append(provider_details)\n\n return providers\n\n\n# Enable each provider\ndef enable_providers(providers):\n if not providers:\n print(\"No providers to enable.\")\n return\n for p in providers:\n enable_provider(p)\n\n\n# Configures a provider by doing the following:\n#\n# 1. Enables a local_resource go build of the provider's local binary\n# 2. Configures a docker build for the provider, with live updating of the local binary\ndef enable_provider(provider):\n name = provider.get(\"name\")\n plugin_name = provider.get(\"plugin_name\")\n\n # Note: we need a distro with a shell to do a copy of the plugin binary\n tilt_dockerfile_header = \"\"\"\n FROM ubuntu:22.04 as tilt\n WORKDIR /\n COPY --from=tilt-helper /start.sh .\n COPY --from=tilt-helper /restart.sh .\n COPY \"\"\" + plugin_name + \"\"\" .\n \"\"\"\n\n dockerfile_contents = \"\\n\".join([\n tilt_helper_dockerfile_header,\n additional_docker_helper_commands,\n tilt_dockerfile_header,\n additional_docker_build_commands,\n ])\n\n context = provider.get(\"context\")\n go_main = provider.get(\"go_main\", \"main.go\")\n\n live_reload_deps = []\n for d in provider.get(\"live_reload_deps\", []):\n live_reload_deps.append(os.path.join(context, \"/\", d))\n\n # Set up a local_resource build of the plugin binary. The main.go path must be provided via go_main option. The binary is written to _tiltbuild/.\n local_resource(\n name + \"_plugin\",\n cmd = 'cd ' + context + ';mkdir -p _tiltbuild;PKG=' + context + ' BIN=' + go_main + ' GOOS=linux GOARCH=amd64 OUTPUT_DIR=_tiltbuild ./hack/build.sh',\n deps = live_reload_deps,\n )\n\n # Set up an image build for the plugin. The live update configuration syncs the output from the local_resource\n # build into the init container, and that restarts the Velero container.\n docker_build(\n ref = provider.get(\"image\"),\n context = os.path.join(context, \"/_tiltbuild/\"),\n dockerfile_contents = dockerfile_contents,\n target = \"tilt\",\n entrypoint = [\"/bin/bash\", \"-c\", \"cp /\" + plugin_name + \" /target/.\"],\n live_update = [\n sync(os.path.join(context, \"/_tiltbuild/\", plugin_name), os.path.join(\"/\", plugin_name))\n ]\n )\n\n\n##############################\n# Start\n#############################\n\nenable_providers(load_provider_tiltfiles())\n" }, { "path": "assets/README.md", "content": "# Velero Assets\n\nThis folder contains logo images for Velero in gray (for light backgrounds) and white (for dark backgrounds like black t-shirts or dark mode!) – horizontal and stacked… in .eps and .svg.\n\n## Some general guidelines for usage\n\n• Don’t alter the logos/graphics: resize, reformat, recolor. Keep them intact.\n\n• Don’t separate the word mark (Velero) from the icon) – we are still building a strong name and identity – and the logo by itself doesn’t have any strong recognition or association with as yet: so best practice keep the two together. Nike kept its name with the swoosh for quite some time before the swoosh became iconic.\n\n• Don’t append the name to another brand – let it stand alone!" }, { "path": "changelogs/CHANGELOG-0.10.md", "content": " - [v0.10.2](#v0102)\n - [v0.10.1](#v0101)\n - [v0.10.0](#v0100)\n\n## v0.10.2\n#### 2019-02-28\n\n### Download\n- https://github.com/heptio/ark/releases/tag/v0.10.2\n\n### Changes\n * upgrade restic to v0.9.4 & replace --hostname flag with --host (#1156, @skriss)\n * use 'restic stats' instead of 'restic check' to determine if repo exists (#1171, @skriss)\n * Fix concurrency bug in code ensuring restic repository exists (#1235, @skriss)\n\n## v0.10.1\n#### 2019-01-10\n\n### Download\n- https://github.com/heptio/ark/releases/tag/v0.10.1\n\n### Changes\n * Fix minio setup job command (#1118, @acbramley)\n * Add debugging-install link in doc get-started.md (#1131, @hex108)\n * `ark version`: show full git SHA & combine git tree state indicator with git SHA line (#1124, @skriss)\n * Delete spec.priority in pod restore action (#879, @mwieczorek)\n * Allow to use AWS Signature v1 for creating signed AWS urls (#811, @bashofmann)\n * add multizone/regional support to gcp (#765, @wwitzel3)\n * Fixed the newline output when deleting a schedule. (#1120, @jwhitcraft)\n * Remove obsolete make targets and rename 'make goreleaser' to 'make release' (#1114, @skriss)\n * Update to go 1.11 (#1069, @gliptak)\n * Update CHANGELOGs (#1063, @wwitzel3)\n * Initialize empty schedule metrics on server init (#1054, @cbeneke)\n * Added brew reference (#1051, @omerlh)\n * Remove default token from all service accounts (#1048, @ncdc)\n * Add pprof support to the Ark server (#234, @ncdc)\n\n## v0.10.0\n#### 2018-11-15\n\n### Download\n- https://github.com/heptio/ark/releases/tag/v0.10.0\n\n### Highlights\n- We've introduced two new custom resource definitions, `BackupStorageLocation` and `VolumeSnapshotLocation`, that replace the `Config` CRD from\nprevious versions. As part of this, you may now configure more than one possible location for where backups and snapshots are stored, and when you\ncreate a `Backup` you can select the location where you'd like that particular backup to be stored. See the [Locations documentation][2] for an overview\nof this feature.\n- Ark's plugin system has been significantly refactored to improve robustness and ease of development. Plugin processes are now automatically restarted\nif they unexpectedly terminate. Additionally, plugin binaries can now contain more than one plugin implementation (e.g. and object store *and* a block store,\nor many backup item actions).\n- The sync process, which ensures that Backup custom resources exist for each backup in object storage, has been revamped to run much more frequently (once\nper minute rather than once per hour), to use significantly fewer cloud provider API calls, and to not generate spurious Kubernetes API errors.\n- Ark can now be configured to store all data under a prefix within an object storage bucket. This means that you no longer need a separate bucket per Ark\ninstance; you can now have all of your clusters' Ark backups go into a single bucket, with each cluster having its own prefix/subdirectory\nwithin that bucket.\n- Restic backup data is now automatically stored within the same bucket/prefix as the rest of the Ark data. A separate bucket is no longer required (or allowed).\n- Ark resources (backups, restores, schedules) can now be bulk-deleted through the `ark` CLI, using the `--all` or `--selector` flags, or by specifying \nmultiple resource names as arguments to the `delete` commands.\n- The `ark` CLI now supports waiting for backups and restores to complete with the `--wait` flag for `ark backup create` and `ark restore create`\n- Restores can be created directly from the most recent backup for a schedule, using `ark restore create --from-schedule SCHEDULE_NAME`\n\n### Breaking Changes\n\nHeptio Ark v0.10 contains a number of breaking changes. Upgrading will require some additional steps beyond just updating your client binary and your\ncontainer image tag. We've provided a [detailed set of instructions][1] to help you with the upgrade process. **Please read and follow these instructions \ncarefully to ensure a successful upgrade!**\n\n- The `Config` CRD has been replaced by `BackupStorageLocation` and `VolumeSnapshotLocation` CRDs. \n- The interface for external plugins (object/block stores, backup/restore item actions) has changed. If you have authored any custom plugins, they'll \nneed to be updated for v0.10.\n - The [`ObjectStore.ListCommonPrefixes`](https://github.com/vmware-tanzu/velero/blob/main/pkg/cloudprovider/object_store.go#L50) signature has changed to add a `prefix` parameter.\n - Registering plugins has changed. Create a new plugin server with the `NewServer` function, and register plugins with the appropriate functions. See the [`Server`](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/server.go#L37) interface for details.\n- The organization of Ark data in object storage has changed. Existing data will need to be moved around to conform to the new layout.\n\n### All Changes\n- [b9de44ff](https://github.com/heptio/ark/commit/b9de44ff)\tupdate docs to reference config/ dir within release tarballs\n- [eace0255](https://github.com/heptio/ark/commit/eace0255)\tgoreleaser: update example image tags to match version being released\n- [cff02159](https://github.com/heptio/ark/commit/cff02159)\tadd rbac content, rework get-started for NodePort and publicUrl, add versioning information\n- [fa14255e](https://github.com/heptio/ark/commit/fa14255e)\tadd content for docs issue 819\n- [22959071](https://github.com/heptio/ark/commit/22959071)\tadd doc explaining locations\n- [e5556fe6](https://github.com/heptio/ark/commit/e5556fe6)\tAdded qps and burst to server's client\n- [9ae861c9](https://github.com/heptio/ark/commit/9ae861c9)\tSupport a separate URL base for pre-signed URLs\n- [698420b6](https://github.com/heptio/ark/commit/698420b6)\tUpdate storage-layout-reorg-v0.10.md\n- [6c9e1f18](https://github.com/heptio/ark/commit/6c9e1f18)\tlower some noisy logs to debug level\n- [318fd8a8](https://github.com/heptio/ark/commit/318fd8a8)\tadd troubleshooting for loadbalancer restores\n- [defb8aa8](https://github.com/heptio/ark/commit/defb8aa8)\tremove code that checks directly for a backup from restore controller\n- [7abe1156](https://github.com/heptio/ark/commit/7abe1156)\tMove clearing up of metadata before plugin's actions\n- [ec013e6f](https://github.com/heptio/ark/commit/ec013e6f)\tDocument upgrading plugins in the deployment\n- [d6162e94](https://github.com/heptio/ark/commit/d6162e94)\tfix goreleaser bugs\n- [a15df276](https://github.com/heptio/ark/commit/a15df276)\tAdd correct link and change role\n- [46bed015](https://github.com/heptio/ark/commit/46bed015)\tadd 0.10 breaking changes warning to readme in main\n- [e3a7d6a2](https://github.com/heptio/ark/commit/e3a7d6a2)\tadd content for issue 994\n- [400911e9](https://github.com/heptio/ark/commit/400911e9)\taddress docs issue #978\n- [b818cc27](https://github.com/heptio/ark/commit/b818cc27)\tdon't require a default provider VSL if there's only 1\n- [90638086](https://github.com/heptio/ark/commit/90638086)\tv0.10 changelog\n- [6e2166c4](https://github.com/heptio/ark/commit/6e2166c4)\tadd docs page on versions and upgrading\n- [18b434cb](https://github.com/heptio/ark/commit/18b434cb)\tgoreleaser scripts for building/creating a release on a workstation\n- [bb65d67a](https://github.com/heptio/ark/commit/bb65d67a)\tupdate restic prerequisite with min k8s version\n- [b5a2ccd5](https://github.com/heptio/ark/commit/b5a2ccd5)\tSilence git detached HEAD advice in build container\n- [67749141](https://github.com/heptio/ark/commit/67749141)\tinstructions for upgrading to v0.10\n- [516422c2](https://github.com/heptio/ark/commit/516422c2)\tsync controller: fill in missing .spec.storageLocation\n- \t[195e6aaf](https://github.com/heptio/ark/commit/195e6aaf)\tfix bug preventing PV snapshots from v0.10 backups from restoring\n- \t[bca58516](https://github.com/heptio/ark/commit/bca58516)\tRun 'make update' to update formatting\n- \t[573ce7d0](https://github.com/heptio/ark/commit/573ce7d0)\tUpdate formatting script\n- \t[90d9be59](https://github.com/heptio/ark/commit/90d9be59)\tsupport restoring/deleting legacy backups with .status.volumeBackups\n- \t[ef194972](https://github.com/heptio/ark/commit/ef194972)\trename variables #967\n- \t[6d4e702c](https://github.com/heptio/ark/commit/6d4e702c)\tfix broken link\n- \t[596eea1b](https://github.com/heptio/ark/commit/596eea1b)\trestore storageclasses before pvs and pvcs\n- \t[f014cab1](https://github.com/heptio/ark/commit/f014cab1)\tbackup describer: show snapshot summary by default, details optionally\n- \t[8acc66d0](https://github.com/heptio/ark/commit/8acc66d0)\tremove pvProviderExists param from NewRestoreController\n- \t[57ce590f](https://github.com/heptio/ark/commit/57ce590f)\tcreate a struct for multiple return of same type in restore_contoroller #967\n- \t[028fafb6](https://github.com/heptio/ark/commit/028fafb6)\tCorrected grammatical error\n- \t[db856aff](https://github.com/heptio/ark/commit/db856aff)\tSpecify return arguments\n- \t[9952dfb0](https://github.com/heptio/ark/commit/9952dfb0)\tAddress #424: Add CRDs to list of prioritized resources\n- \t[cf2c2714](https://github.com/heptio/ark/commit/cf2c2714)\tfix bugs in GetBackupVolumeSnapshots and add test\n- \t[ec124673](https://github.com/heptio/ark/commit/ec124673)\tremove all references to Config from docs/examples\n- \t[c36131a0](https://github.com/heptio/ark/commit/c36131a0)\tremove Config-related code\n- \t[406b50a7](https://github.com/heptio/ark/commit/406b50a7)\tupdate restore process using snapshot locations\n- \t[268080ad](https://github.com/heptio/ark/commit/268080ad)\tavoid panics if can't get block store during deletion\n- \t[4a03370f](https://github.com/heptio/ark/commit/4a03370f)\tupdate backup deletion controller for snapshot locations\n- \t[38c72b8c](https://github.com/heptio/ark/commit/38c72b8c)\tinclude snapshot locations in created schedule's backup spec\n- \t[0ec2de55](https://github.com/heptio/ark/commit/0ec2de55)\tazure: update blockstore to allow storing snaps in different resource group\n- \t[35bb533c](https://github.com/heptio/ark/commit/35bb533c)\tclose gzip writer before uploading volumesnapshots file\n- \t[da9ed38c](https://github.com/heptio/ark/commit/da9ed38c)\tstore volume snapshot info as JSON in backup storage\n- \t[e24248e0](https://github.com/heptio/ark/commit/e24248e0)\tadd --volume-snapshot-locations flag to ark backup create\n- \t[df07b7dc](https://github.com/heptio/ark/commit/df07b7dc)\tupdate backup code to work with volume snapshot locations\n- \t[4af89fa8](https://github.com/heptio/ark/commit/4af89fa8)\tadd unit test for getDefaultVolumeSnapshotLocations\n- \t[02f50b9c](https://github.com/heptio/ark/commit/02f50b9c)\tadd default-volume-snapshot-locations to server cmd\n- \t[1aa712d2](https://github.com/heptio/ark/commit/1aa712d2)\tDefault and validate VolumeSnapshotLocations\n- \t[bbf76985](https://github.com/heptio/ark/commit/bbf76985)\tadd create CLI command for snapshot locations\n- \t[aeb221ea](https://github.com/heptio/ark/commit/aeb221ea)\tAdd printer for snapshot locations\n- \t[ffc612ac](https://github.com/heptio/ark/commit/ffc612ac)\tAdd volume snapshot CLI get command\n- \t[f20342aa](https://github.com/heptio/ark/commit/f20342aa)\tAdd VolumeLocation and Snapshot.\n- \t[7172db8a](https://github.com/heptio/ark/commit/7172db8a)\tupgrade to restic v0.9.3\n- \t[99adc4fa](https://github.com/heptio/ark/commit/99adc4fa)\tRemove broken references to docs that are not existing\n- \t[474efde6](https://github.com/heptio/ark/commit/474efde6)\tFixed relative link for image\n- \t[41735154](https://github.com/heptio/ark/commit/41735154)\tdon't require a default backup storage location to exist\n- \t[0612c5de](https://github.com/heptio/ark/commit/0612c5de)\ttemplatize error message in DeleteOptions\n- \t[66bcbc05](https://github.com/heptio/ark/commit/66bcbc05)\tadd support for bulk deletion to ark schedule delete\n- \t[3af43b49](https://github.com/heptio/ark/commit/3af43b49)\tadd azure-specific code to support multi-location restic\n- \t[d009163b](https://github.com/heptio/ark/commit/d009163b)\tupdate restic to support multiple backup storage locations\n- \t[f4c99c77](https://github.com/heptio/ark/commit/f4c99c77)\tChange link for the support matrix\n- \t[91e45d56](https://github.com/heptio/ark/commit/91e45d56)\tFix broken storage providers link\n- \t[ed0eb865](https://github.com/heptio/ark/commit/ed0eb865)\tfix backup storage location example YAMLs\n- \t[eb709b8f](https://github.com/heptio/ark/commit/eb709b8f)\tonly sync a backup location if it's changed since last sync\n- \t[af3af1b5](https://github.com/heptio/ark/commit/af3af1b5)\tclarify Azure resource group usage in docs\n- \t[9fdf8513](https://github.com/heptio/ark/commit/9fdf8513)\tMinor code cleanup\n- \t[2073e15a](https://github.com/heptio/ark/commit/2073e15a)\tFix formatting for live site\n- \t[0fc3e8d8](https://github.com/heptio/ark/commit/0fc3e8d8)\tadd documentation on running Ark on-premises\n- \t[e46e89cb](https://github.com/heptio/ark/commit/e46e89cb)\thave restic share main Ark bucket\n- \t[42b54586](https://github.com/heptio/ark/commit/42b54586)\trefactor to make valid dirs part of an object store layout\n- \t[8bc7e4f6](https://github.com/heptio/ark/commit/8bc7e4f6)\tstore backups & restores in backups/, restores/ subdirs in obj storage\n- \t[e3232b7e](https://github.com/heptio/ark/commit/e3232b7e)\tadd support for bulk deletion to ark restore delete\n- \t[17be71e1](https://github.com/heptio/ark/commit/17be71e1)\tremove deps used for docs gen\n- \t[20635106](https://github.com/heptio/ark/commit/20635106)\tremove script for generating docs\n- \t[6fd9ea9d](https://github.com/heptio/ark/commit/6fd9ea9d)\tremove cli reference docs and related scripts\n- \t[4833607a](https://github.com/heptio/ark/commit/4833607a)\tFix infinite sleep in fsfreeze container\n- \t[7668bfd4](https://github.com/heptio/ark/commit/7668bfd4)\tAdd links for Portworx plugin support\n- \t[468006e6](https://github.com/heptio/ark/commit/468006e6)\tFix Portworx name in doc\n- \t[e6b44539](https://github.com/heptio/ark/commit/e6b44539)\tMake fsfreeze image building consistent\n- \t[fcd27a13](https://github.com/heptio/ark/commit/fcd27a13)\tget a new metadata accessor after calling backup item actions\n- \t[ffef86e3](https://github.com/heptio/ark/commit/ffef86e3)\tAdding support for the AWS_CLUSTER_NAME env variable allowing to claim volumes ownership\n- \t[cda3dff8](https://github.com/heptio/ark/commit/cda3dff8)\tDocument single binary plugins\n- \t[f049e078](https://github.com/heptio/ark/commit/f049e078)\tRemove ROADMAP.md, update ZenHub link to Ark board\n- \t[94617b30](https://github.com/heptio/ark/commit/94617b30)\tconvert all controllers to use genericController, logContext -> log\n- \t[779cb428](https://github.com/heptio/ark/commit/779cb428)\tDocument SignatureDoesNotMatch error and triaging\n- \t[7d8813a9](https://github.com/heptio/ark/commit/7d8813a9)\tmove ObjectStore mock into pkg/cloudprovider/mocks\n- \t[f0edf733](https://github.com/heptio/ark/commit/f0edf733)\tadd a BackupStore to pkg/persistence that supports prefixes\n- \t[af64069d](https://github.com/heptio/ark/commit/af64069d)\tcreate pkg/persistence and move relevant code from pkg/cloudprovider into it\n- \t[29d75d72](https://github.com/heptio/ark/commit/29d75d72)\tmove object and block store interfaces to their own files\n- \t[211aa7b7](https://github.com/heptio/ark/commit/211aa7b7)\tSet schedule labels to subsequent backups\n- \t[d34994cb](https://github.com/heptio/ark/commit/d34994cb)\tset azure restic env vars based on default backup location's config\n- \t[a50367f1](https://github.com/heptio/ark/commit/a50367f1)\tRegenerate CLI docs\n- \t[7bc27bbb](https://github.com/heptio/ark/commit/7bc27bbb)\tPin cobra version\n- \t[e94277ac](https://github.com/heptio/ark/commit/e94277ac)\tUpdate pflag version\n- \t[df69b274](https://github.com/heptio/ark/commit/df69b274)\tazure: update documentation and examples\n- \t[cb321db2](https://github.com/heptio/ark/commit/cb321db2)\tazure: refactor to not use helpers/ pkg, validate all env/config inputs\n- \t[9d7ea748](https://github.com/heptio/ark/commit/9d7ea748)\tazure: support different RGs/storage accounts per backup location\n- \t[cd4e9f53](https://github.com/heptio/ark/commit/cd4e9f53)\tazure: fix for breaking change in blob.GetSASURI\n- \t[a440029c](https://github.com/heptio/ark/commit/a440029c)\tbump Azure SDK version and include storage mgmt package\n- \t[b31e25bf](https://github.com/heptio/ark/commit/b31e25bf)\tserver: remove unused code, replace deprecated func\n- \t[729d7339](https://github.com/heptio/ark/commit/729d7339)\tcontrollers: take a newPluginManager func in constructors\n- \t[6445dbf1](https://github.com/heptio/ark/commit/6445dbf1)\tUpdate examples and docs for backup locations\n- \t[133dc185](https://github.com/heptio/ark/commit/133dc185)\tbackup sync: process the default location first\n- \t[7a1e6d16](https://github.com/heptio/ark/commit/7a1e6d16)\tgeneric controller: allow controllers with only a resync func\n- \t[6f7bfe54](https://github.com/heptio/ark/commit/6f7bfe54)\tremove Config CRD's BackupStorageProvider & other obsolete code\n- \t[bd4d97b9](https://github.com/heptio/ark/commit/bd4d97b9)\tmove server's defaultBackupLocation into config struct\n- \t[0e94fa37](https://github.com/heptio/ark/commit/0e94fa37)\tupdate sync controller for backup locations\n- \t[2750aa71](https://github.com/heptio/ark/commit/2750aa71)\tUse backup storage location during restore\n- \t[20f89fbc](https://github.com/heptio/ark/commit/20f89fbc)\tuse the default backup storage location for restic\n- \t[833a6307](https://github.com/heptio/ark/commit/833a6307)\tAdd storage location to backup get/describe\n- \t[cf7c8587](https://github.com/heptio/ark/commit/cf7c8587)\tdownload request: fix setting of log level for plugin manager\n- \t[3234124a](https://github.com/heptio/ark/commit/3234124a)\tbackup deletion: fix setting of log level in plugin manager\n- \t[74043ab4](https://github.com/heptio/ark/commit/74043ab4)\tdownload request controller: fix bug in determining expiration\n- \t[7007f198](https://github.com/heptio/ark/commit/7007f198)\trefactor download request controller test and add test cases\n- \t[8f534615](https://github.com/heptio/ark/commit/8f534615)\tdownload request controller: use backup location for object store\n- \t[bab08ed1](https://github.com/heptio/ark/commit/bab08ed1)\tbackup deletion controller: use backup location for object store\n- \t[c6f488f7](https://github.com/heptio/ark/commit/c6f488f7)\tUse backup location in the backup controller\n- \t[06b5af44](https://github.com/heptio/ark/commit/06b5af44)\tadd create and get CLI commands for backup locations\n- \t[adbcd370](https://github.com/heptio/ark/commit/adbcd370)\tadd --default-backup-storage-location flag to server cmd\n- \t[2a34772e](https://github.com/heptio/ark/commit/2a34772e)\tAdd --storage-location argument to create commands\n- \t[56f16170](https://github.com/heptio/ark/commit/56f16170)\tCorrect metadata for BackupStorageLocationList\n- \t[345c3c39](https://github.com/heptio/ark/commit/345c3c39)\tGenerate clients for BackupStorageLocation\n- \t[a25eb032](https://github.com/heptio/ark/commit/a25eb032)\tAdd BackupStorageLocation API type\n- \t[575c4ddc](https://github.com/heptio/ark/commit/575c4ddc)\tapply annotations on single line, no restore mode\n- \t[030ea6c0](https://github.com/heptio/ark/commit/030ea6c0)\tminor word updates and command wrapping\n- \t[d32f8dbb](https://github.com/heptio/ark/commit/d32f8dbb)\tUpdate hooks/fsfreeze example\n- \t[342a1c64](https://github.com/heptio/ark/commit/342a1c64)\tadd an ark bug command\n- \t[9c11ba90](https://github.com/heptio/ark/commit/9c11ba90)\tAdd DigitalOcean to S3-compatible backup providers\n- \t[ea50ebf2](https://github.com/heptio/ark/commit/ea50ebf2)\tFix map merging logic\n- \t[9508e4a2](https://github.com/heptio/ark/commit/9508e4a2)\tSwitch Config CRD elements to server flags\n- \t[0c3ac67b](https://github.com/heptio/ark/commit/0c3ac67b)\tstart using a namespaced label on restored objects, deprecate old label\n- \t[6e53aa03](https://github.com/heptio/ark/commit/6e53aa03)\tBring back 'make local'\n- \t[5acccaa7](https://github.com/heptio/ark/commit/5acccaa7)\tadd bulk deletion support to ark backup delete\n- \t[3aa241a7](https://github.com/heptio/ark/commit/3aa241a7)\tPreserve node ports during restore when annotations hold specification.\n- \t[c5f5862c](https://github.com/heptio/ark/commit/c5f5862c)\tAdd --wait support to ark backup create\n- \t[eb6f742b](https://github.com/heptio/ark/commit/eb6f742b)\tDocument CRD not found errors\n- \t[fb4d507c](https://github.com/heptio/ark/commit/fb4d507c)\tExtend doc about synchronization\n- \t[e7bb5926](https://github.com/heptio/ark/commit/e7bb5926)\tAdd --wait support to `ark restore create`\n- \t[8ce513ac](https://github.com/heptio/ark/commit/8ce513ac)\tOnly delete unused backup if they are complete\n- \t[1c26fbde](https://github.com/heptio/ark/commit/1c26fbde)\tremove SnapshotService, replace with direct BlockStore usage\n- \t[13051218](https://github.com/heptio/ark/commit/13051218)\tRefactor plugin management\n- \t[74dbf387](https://github.com/heptio/ark/commit/74dbf387)\tAdd restore failed phase and metrics\n- \t[8789ae5c](https://github.com/heptio/ark/commit/8789ae5c)\tupdate testify to latest released version\n- \t[fe9d61a9](https://github.com/heptio/ark/commit/fe9d61a9)\tAdd schedule command info to quickstart\n- \t[ca5656c2](https://github.com/heptio/ark/commit/ca5656c2)\tfix bug preventing backup item action item updates from saving\n- \t[d2e629f5](https://github.com/heptio/ark/commit/d2e629f5)\tDelete backups from etcd if they're not in storage\n- \t[625ba481](https://github.com/heptio/ark/commit/625ba481)\tFix ZenHub link on Readme.md\n- \t[dcae6eb0](https://github.com/heptio/ark/commit/dcae6eb0)\tUpdate gcp-config.md\n- \t[06d6665a](https://github.com/heptio/ark/commit/06d6665a)\tcheck s3URL scheme upon AWS ObjectStore Init()\n- \t[cc359f6e](https://github.com/heptio/ark/commit/cc359f6e)\tAdd contributor docs for our ZenHub usage\n- \t[f6204562](https://github.com/heptio/ark/commit/f6204562)\tcleanup service account action log statement\n- \t[450fa72f](https://github.com/heptio/ark/commit/450fa72f)\tInitialize schedule Prometheus metrics to have them created beforehand (see https://prometheus.io/docs/practices/instrumentation/#avoid-missing-metrics)\n- \t[39c4267a](https://github.com/heptio/ark/commit/39c4267a)\tClarify that object storage should per-cluster\n- \t[78cbdf95](https://github.com/heptio/ark/commit/78cbdf95)\tdelete old deletion requests for backup when processing a new one\n- \t[85a61b8e](https://github.com/heptio/ark/commit/85a61b8e)\treturn nil error if 404 encountered when deleting snapshots\n- \t[a2a7dbda](https://github.com/heptio/ark/commit/a2a7dbda)\tfix tagging latest by using make's ifeq\n- \t[b4a52e45](https://github.com/heptio/ark/commit/b4a52e45)\tAdd commands for context to the bug template\n- \t[3efe6770](https://github.com/heptio/ark/commit/3efe6770)\tUpdate Ark library code to work with Kubernetes 1.11\n- \t[7e8c8c69](https://github.com/heptio/ark/commit/7e8c8c69)\tAdd some basic troubleshooting commands\n- \t[d1955120](https://github.com/heptio/ark/commit/d1955120)\trequire namespace for backups/etc. to exist at server startup\n- \t[683f7afc](https://github.com/heptio/ark/commit/683f7afc)\tswitch to using .status.startTimestamp for sorting backups\n- \t[b71a37db](https://github.com/heptio/ark/commit/b71a37db)\tRecord backup completion time before uploading\n- \t[217084cd](https://github.com/heptio/ark/commit/217084cd)\tAdd example ark version command to issue templates\n- \t[040788bb](https://github.com/heptio/ark/commit/040788bb)\tAdd minor improvements and aws exampledelimitMateCR\n- \t[5b89f7b6](https://github.com/heptio/ark/commit/5b89f7b6)\tSkip backup sync if it already exists in k8s\n- \t[c6050845](https://github.com/heptio/ark/commit/c6050845)\trestore controller: switch to 'c' for receiver name\n- \t[706ae07d](https://github.com/heptio/ark/commit/706ae07d)\tenable a schedule to be provided as the source for a restore\n- \t[aea68414](https://github.com/heptio/ark/commit/aea68414)\tfix up Slack link in troubleshooting on main branch\n- \t[bb8e2e91](https://github.com/heptio/ark/commit/bb8e2e91)\tDocument how to run the Ark server locally\n- \t[dc84e591](https://github.com/heptio/ark/commit/dc84e591)\tRemove outdated namespace deletion content\n- \t[23abbc9a](https://github.com/heptio/ark/commit/23abbc9a)\tfix paths\n- \t[f0426538](https://github.com/heptio/ark/commit/f0426538)\tuse posix-compliant conditional for checking TAG_LATEST\n- \t[cf336d80](https://github.com/heptio/ark/commit/cf336d80)\tAdded new templates\n- \t[795dc262](https://github.com/heptio/ark/commit/795dc262)\treplace pkg/restore's osFileSystem with pkg/util/filesystem's\n- \t[eabef085](https://github.com/heptio/ark/commit/eabef085)\tUpdate generated Ark code based on the 1.11 k8s.io/code-generator script\n- \t[f5eac0b4](https://github.com/heptio/ark/commit/f5eac0b4)\tUpdate vendored library code for Kubernetes 1.11\n\n[1]: https://heptio.github.io/velero/v0.10.0/upgrading-to-v0.10\n[2]: locations.md\n" }, { "path": "changelogs/CHANGELOG-0.11.md", "content": "## v0.11.1\n#### 2019-05-17\n\n### Download\n- https://github.com/heptio/velero/releases/tag/v0.11.1\n\n### Highlights\n* Added the `velero migrate-backups` command to migrate legacy Ark backup metadata to the current Velero format in object storage. This command needs to be run in preparation for upgrading to v1.0, **if** you have backups that were originally created prior to v0.11 (i.e. when the project was named Ark).\n\n## v0.11.0\n#### 2019-02-28\n\n### Download\n- https://github.com/heptio/velero/releases/tag/v0.11.0\n\n### Highlights\n* Heptio Ark is now Velero! This release is the first one to use the new name. For details on the changes and how to migrate to v0.11, see the [migration instructions][1]. **Please follow the instructions to ensure a successful upgrade to v0.11.**\n* Restic has been upgraded to v0.9.4, which brings significantly faster restores thanks to a new multi-threaded restorer.\n* Velero now waits for terminating namespaces and persistent volumes to delete before attempting to restore them, rather than trying and failing to restore them while they're being deleted.\n\n### All Changes\n* Fix concurrency bug in code ensuring restic repository exists (#1235, @skriss)\n* Wait for PVs and namespaces to delete before attempting to restore them. (#826, @nrb)\n* Set the zones for GCP regional disks on restore. This requires the `compute.zones.get` permission on the GCP serviceaccount in order to work correctly. (#1200, @nrb)\n* Renamed Heptio Ark to Velero. Changed internal imports, environment variables, and binary name. (#1184, @nrb)\n* use 'restic stats' instead of 'restic check' to determine if repo exists (#1171, @skriss)\n* upgrade restic to v0.9.4 & replace --hostname flag with --host (#1156, @skriss)\n* Clarify restore log when object unchanged (#1153, @daved)\n* Add backup-version file in backup tarball. (#1117, @wwitzel3)\n* add ServerStatusRequest CRD and show server version in `ark version` output (#1116, @skriss)\n\n[1]: https://heptio.github.io/velero/v0.11.0/migrating-to-velero\n" }, { "path": "changelogs/CHANGELOG-0.3.md", "content": "- [v0.3.3](#v033)\n- [v0.3.2](#v032)\n- [v0.3.1](#v031)\n- [v0.3.0](#v030)\n\n## v0.3.3\n#### 2017-08-10\n### Download\n - https://github.com/heptio/ark/tree/v0.3.3\n\n### Bug Fixes\n * Treat the first field in a schedule's cron expression as minutes, not seconds\n\n\n## v0.3.2\n#### 2017-08-07\n### Download\n - https://github.com/heptio/ark/tree/v0.3.2\n\n### New Features\n * Add client-go auth provider plugins for Azure, GCP, OIDC\n\n\n## v0.3.1\n#### 2017-08-03\n### Download\n - https://github.com/heptio/ark/tree/v0.3.1\n\n### Bug Fixes\n * Fix Makefile VERSION\n\n\n## v0.3.0\n#### 2017-08-03\n### Download\n - https://github.com/heptio/ark/tree/v0.3.0\n\n### All New Features\n * Initial Release\n" }, { "path": "changelogs/CHANGELOG-0.4.md", "content": "- [v0.4.0](#v040)\n\n## v0.4.0\n#### 2017-09-14\n### Download\n - https://github.com/heptio/ark/tree/v0.4.0\n\n### Breaking changes\n * Snapshotting and restoring volumes is now enabled by default\n * The --namespaces flag for 'ark restore create' has been replaced by --include-namespaces and\n --exclude-namespaces\n\n### New features\n * Support for S3 SSE with KMS\n * Cloud provider configurations are validated at startup\n * The persistentVolumeProvider is now optional\n * Restore objects are garbage collected\n * Each backup now has an associated log file, viewable via 'ark backup logs'\n * Each restore now has an associated log file, viewable via 'ark restore logs'\n * Add --include-resources/--exclude-resources for restores\n\n### Bug fixes\n * Only save/use iops for io1 volumes on AWS\n * When restoring, try to retrieve the Backup directly from object storage if it's not found\n * When syncing Backups from object storage to Kubernetes, don't return at the first error\n encountered\n * More closely match how kubectl performs kubeconfig resolution\n * Increase default Azure API request timeout to 2 minutes\n * Update Azure diskURI to match diskName\n" }, { "path": "changelogs/CHANGELOG-0.5.md", "content": "- [v0.5.1](#v051)\n- [v0.5.0](#v050)\n\n## v0.5.1\n#### 2017-11-06\n### Download\n - https://github.com/heptio/ark/tree/v0.5.1\n\n### Bug fixes\n * If a Service is headless, retain ClusterIP = None when backing up and restoring.\n * Use the specified --label-selector when listing backups, schedules, and restores.\n * Restore namespace mapping functionality that was accidentally broken in 0.5.0.\n * Always include namespaces in the backup, regardless of the --include-cluster-resources setting.\n\n\n## v0.5.0\n#### 2017-10-26\n### Download\n - https://github.com/heptio/ark/tree/v0.5.0\n\n### Breaking changes\n * The backup tar file format has changed. Backups created using previous versions of Ark cannot be restored using v0.5.0.\n * When backing up one or more specific namespaces, cluster-scoped resources are no longer backed up by default, with the exception of PVs that are used within the target namespace(s). Cluster-scoped resources can still be included by explicitly specifying `--include-cluster-resources`.\n\n### New features\n * Add customized user-agent string for Ark CLI\n * Switch from glog to logrus\n * Exclude nodes from restoration\n * Add a FAQ\n * Record PV availability zone and use it when restoring volumes from snapshots\n * Back up the PV associated with a PVC\n * Add `--include-cluster-resources` flag to `ark backup create`\n * Add `--include-cluster-resources` flag to `ark restore create`\n * Properly support resource restore priorities across cluster-scoped and namespace-scoped resources\n * Support `ark create ...` and `ark get ...`\n * Make ark run as cluster-admin\n * Add pod exec backup hooks\n * Support cross-compilation & upgrade to go 1.9\n \n### Bug fixes\n * Make config change detection more robust\n" }, { "path": "changelogs/CHANGELOG-0.6.md", "content": "- [v0.6.0](#v060)\n\n## v0.6.0\n#### 2017-11-30\n### Download\n - https://github.com/heptio/ark/tree/v0.6.0\n\n### Highlights\n * **Plugins** - We now support user-defined plugins that can extend Ark functionality to meet your custom backup/restore needs without needing to be compiled into the core binary. We support pluggable block and object stores as well as per-item backup and restore actions that can execute arbitrary logic, including modifying the items being backed up or restored. For more information see the [documentation](docs/plugins.md), which includes a reference to a fully-functional sample plugin repository. (#174 #188 #206 #213 #215 #217 #223 #226)\n * **Describers** - The Ark CLI now includes `describe` commands for `backups`, `restores`, and `schedules` that provide human-friendly representations of the relevant API objects.\n\n### Breaking Changes\n * The config object format has changed. In order to upgrade to v0.6.0, the config object will have to be updated to match the new format. See the [examples](examples) and [documentation](docs/config-definition.md) for more information.\n * The restore object format has changed. The `warnings` and `errors` fields are now ints containing the counts, while full warnings and errors are now stored in the object store instead of etcd. Restore objects created prior to v.0.6.0 should be deleted, or a new bucket used, and the old restore objects deleted from Kubernetes (`kubectl -n heptio-ark delete restore --all`).\n\n### All New Features\n * Add `ark plugin add` and `ark plugin remove` commands #217, @skriss\n * Add plugin support for block/object stores, backup/restore item actions #174 #188 #206 #213 #215 #223 #226, @skriss @ncdc\n * Improve Azure deployment instructions #216, @ncdc\n * Change default TTL for backups to 30 days #204, @nrb\n * Improve logging for backups and restores #199, @ncdc\n * Add `ark backup describe`, `ark schedule describe` #196, @ncdc\n * Add `ark restore describe` and move restore warnings/errors to object storage #173 #201 #202, @ncdc\n * Upgrade to client-go v5.0.1, kubernetes v1.8.2 #157, @ncdc\n * Add Travis CI support #165 #166, @ncdc\n\n### Bug Fixes\n * Fix log location hook prefix stripping #222, @ncdc\n * When running `ark backup download`, remove file if there's an error #154, @ncdc\n * Update documentation for AWS KMS Key alias support #163, @lli-hiya\n * Remove clock from `volume_snapshot_action` #137, @athampy\n" }, { "path": "changelogs/CHANGELOG-0.7.md", "content": "- [v0.7.1](#v071)\n- [v0.7.0](#v070)\n\n## v0.7.1\n#### 2018-02-22\n### Download\n - https://github.com/heptio/ark/releases/tag/v0.7.1\n\n### Bug Fixes:\n * Run the Ark server in its own namespace, separate from backups/schedules/restores/config (#322, @ncdc)\n\n\n## v0.7.0\n#### 2018-02-15\n### Download\n - https://github.com/heptio/ark/releases/tag/v0.7.0\n\n### New Features:\n * Run the Ark server in any namespace (#272, @ncdc)\n * Add ability to delete backups and their associated data (#252, @skriss)\n * Support both pre and post backup hooks (#243, @ncdc)\n\n### Bug Fixes / Other Changes:\n * Switch from Update() to Patch() when updating Ark resources (#241, @skriss)\n * Don't fail the backup if a PVC is not bound to a PV (#256, @skriss)\n * Restore serviceaccounts prior to workload controllers (#258, @ncdc)\n * Stop removing annotations from PVs when restoring them (#263, @skriss)\n * Update GCP client libraries (#249, @skriss)\n * Clarify backup and restore creation messages (#270, @nrb)\n * Update S3 bucket creation docs for us-east-1 (#285, @lypht)\n" }, { "path": "changelogs/CHANGELOG-0.8.md", "content": "- [v0.8.3](#v083)\n- [v0.8.2](#v082)\n- [v0.8.1](#v081)\n- [v0.8.0](#v080)\n\n## v0.8.3\n#### 2018-06-29\n### Download\n - https://github.com/heptio/ark/releases/tag/v0.8.3\n\n### Bug Fixes:\n * Don't restore backup and restore resources to avoid possible data corruption (#622, @ncdc)\n\n\n## v0.8.2\n#### 2018-06-01\n### Download\n - https://github.com/heptio/ark/releases/tag/v0.8.2\n\n### Bug Fixes:\n * Don't crash when a persistent volume claim is missing spec.volumeName (#520, @ncdc)\n\n\n## v0.8.1\n#### 2018-04-23\n### Download\n - https://github.com/heptio/ark/releases/tag/v0.8.1\n\n### Bug Fixes:\n * Azure: allow pre-v0.8.0 backups with disk snapshots to be restored and deleted (#446 #449, @skriss)\n\n\n## v0.8.0\n#### 2018-04-19\n### Download\n - https://github.com/heptio/ark/releases/tag/v0.8.0\n\n### Highlights:\n * Backup deletion has been completely revamped to make it simpler and less error-prone. As a user, you still use the `ark backup delete` command to request deletion of a backup and its associated cloud\n resources; behind the scenes, we've switched to using a new `DeleteBackupRequest` Custom Resource and associated controller for processing deletion requests.\n * We've reduced the number of required fields in the Ark config. For Azure, `location` is no longer required, and for GCP, `project` is not needed.\n * Ark now copies tags from volumes to snapshots during backup, and from snapshots to new volumes during restore. \n\n### Breaking Changes:\n * Ark has moved back to a single namespace (`heptio-ark` by default) as part of #383.\n\n### All New Features:\n * Add global `--kubecontext` flag to Ark CLI (#296, @blakebarnett)\n * Azure: support cross-resource group restores of volumes (#356 #378, @skriss)\n * AWS/Azure/GCP: copy tags from volumes to snapshots, and from snapshots to volumes (#341, @skriss)\n * Replace finalizer for backup deletion with `DeleteBackupRequest` custom resource & controller (#383 #431, @ncdc @nrb)\n * Don't log warnings during restore if an identical object already exists in the cluster (#405, @nrb)\n * Add bash & zsh completion support (#384, @containscafeine)\n \n### Bug Fixes / Other Changes:\n * Error from the Ark CLI if attempting to restore a non-existent backup (#302, @ncdc)\n * Enable running the Ark server locally for development purposes (#334, @ncdc)\n * Add examples to `ark schedule create` documentation (#331, @lypht)\n * GCP: Remove `project` requirement from Ark config (#345, @skriss)\n * Add `--from-backup` flag to `ark restore create` and allow custom restore names (#342 #409, @skriss)\n * Azure: remove `location` requirement from Ark config (#344, @skriss)\n * Add documentation/examples for storing backups in IBM Cloud Object Storage (#321, @roytman)\n * Reduce verbosity of hooks logging (#362, @skriss)\n * AWS: Add minimal IAM policy to documentation (#363 #419, @hopkinsth)\n * Don't restore events (#374, @sanketjpatel)\n * Azure: reduce API polling interval from 60s to 5s (#359, @skriss)\n * Switch from hostPath to emptyDir volume type for minio example (#386, @containscafeine)\n * Add limit ranges as a prioritized resource for restores (#392, @containscafeine)\n * AWS: Add documentation on using Ark with kube2iam (#402, @domderen)\n * Azure: add node selector so Ark pod is scheduled on a linux node (#415, @ffd2subroutine)\n * Error from the Ark CLI if attempting to get logs for a non-existent restore (#391, @containscafeine)\n * GCP: Add minimal IAM policy to documentation (#429, @skriss @jody-frankowski)\n\n### Upgrading from v0.7.1:\n Ark v0.7.1 moved the Ark server deployment into a separate namespace, `heptio-ark-server`. As of v0.8.0 we've\n returned to a single namespace, `heptio-ark`, for all Ark-related resources. If you're currently running v0.7.1,\n here are the steps you can take to upgrade:\n\n1. Execute the steps from the **Credentials and configuration** section for your cloud:\n * [AWS](https://heptio.github.io/velero/v0.8.0/aws-config#credentials-and-configuration)\n * [Azure](https://heptio.github.io/velero/v0.8.0/azure-config#credentials-and-configuration)\n * [GCP](https://heptio.github.io/velero/v0.8.0/gcp-config#credentials-and-configuration)\n\n When you get to the secret creation step, if you don't have your `credentials-ark` file handy, \n you can copy the existing secret from your `heptio-ark-server` namespace into the `heptio-ark` namespace:\n ```bash\n kubectl get secret/cloud-credentials -n heptio-ark-server --export -o json | \\\n jq '.metadata.namespace=\"heptio-ark\"' | \\\n kubectl apply -f -\n ```\n\n2. You can now safely delete the `heptio-ark-server` namespace:\n ```bash\n kubectl delete namespace heptio-ark-server\n ```\n\n3. Execute the commands from the **Start the server** section for your cloud:\n * [AWS](https://heptio.github.io/velero/v0.8.0/aws-config#start-the-server)\n * [Azure](https://heptio.github.io/velero/v0.8.0/azure-config#start-the-server)\n * [GCP](https://heptio.github.io/velero/v0.8.0/gcp-config#start-the-server)\n" }, { "path": "changelogs/CHANGELOG-0.9.md", "content": " - [v0.9.11](#v0911)\n - [v0.9.10](#v0910)\n - [v0.9.9](#v099)\n - [v0.9.8](#v098)\n - [v0.9.7](#v097)\n - [v0.9.6](#v096)\n - [v0.9.5](#v095)\n - [v0.9.4](#v094)\n - [v0.9.3](#v093)\n - [v0.9.2](#v092)\n - [v0.9.1](#v091)\n - [v0.9.0](#v090)\n\n## v0.9.11\n#### 2018-11-08\n### Download\n - https://github.com/heptio/ark/releases/tag/v0.9.11\n\n### Bug Fixes\n * Fix bug preventing PV snapshots from being restored (#1040, @ncdc)\n\n\n## v0.9.10\n#### 2018-11-01\n### Download\n - https://github.com/heptio/ark/releases/tag/v0.9.10\n\n### Bug Fixes\n * restore storageclasses before pvs and pvcs (#594, @shubheksha)\n * AWS: Ensure that the order returned by ListObjects is consistent (#999, @bashofmann)\n * Add CRDs to list of prioritized resources (#424, @domenicrosati)\n * Verify PV doesn't exist before creating new volume (#609, @nrb)\n * Update README.md - Grammar mistake corrected (#1018, @midhunbiju)\n\n\n## v0.9.9\n#### 2018-10-24\n### Download\n - https://github.com/heptio/ark/releases/tag/v0.9.9\n\n### Bug Fixes\n * Check if initContainers key exists before attempting to remove volume mounts. (#927, @skriss)\n\n\n## v0.9.8\n#### 2018-10-18\n### Download\n - https://github.com/heptio/ark/releases/tag/v0.9.8\n\n### Bug Fixes\n * Discard service account token volume mounts from init containers on restore (#910, @james-powis)\n * Support --include-cluster-resources flag when creating schedule (#942, @captjt)\n * Remove logic to get a GCP project (#926, @shubheksha)\n * Only try to back up PVCs linked PV if the PVC's phase is Bound (#920, @skriss)\n * Claim ownership of new AWS volumes on Kubernetes cluster being restored into (#801, @ljakimczuk)\n * Remove timeout check when taking snapshots (#928, @carlisia)\n\n\n## v0.9.7\n#### 2018-10-04\n### Download\n - https://github.com/heptio/ark/releases/tag/v0.9.7\n\n### Bug Fixes\n * Preserve explicitly-specified node ports during restore (#712, @timoreimann)\n * Enable restoring resources with ownerReference set (#837, @mwieczorek)\n * Fix error when restoring ExternalName services (#869, @shubheksha)\n * remove restore log helper for accurate line numbers (#891, @skriss)\n * Display backup StartTimestamp in `ark backup get` output (#894, @marctc)\n * Fix restic restores when using namespace mappings (#900, @skriss)\n\n\n## v0.9.6\n#### 2018-09-21\n### Download\n - https://github.com/heptio/ark/releases/tag/v0.9.6\n\n### Bug Fixes\n * Discard service account tokens from non-default service accounts on restore (#843, @james-powis)\n * Update Docker images to use `alpine:3.8` (#852, @nrb)\n\n\n## v0.9.5\n#### 2018-09-17\n### Download\n - https://github.com/heptio/ark/releases/tag/v0.9.5\n\n### Bug Fixes\n * Fix issue causing restic restores not to work (#834, @skriss)\n\n\n## v0.9.4\n#### 20180-09-05\n### Download\n - https://github.com/heptio/ark/releases/tag/v0.9.4\n\n### Bug Fixes\n * Terminate plugin clients to resolve memory leaks (#797, @skriss)\n * Fix nil map errors when merging annotations (#812, @nrb)\n\n\n## v0.9.3\n#### 2018-08-10\n### Download\n - https://github.com/heptio/ark/releases/tag/v0.9.3\n### Bug Fixes\n * Initialize Prometheus metrics when creating a new schedule (#689, @lemaral)\n\n\n## v0.9.2\n#### 2018-07-26\n### Download\n - https://github.com/heptio/ark/releases/tag/v0.9.2) - 2018-07-26\n\n### Bug Fixes:\n * Fix issue where modifications made by backup item actions were not being saved to backup tarball (#704, @skriss)\n\n\n## v0.9.1\n#### 2018-07-23\n### Download\n - https://github.com/heptio/ark/releases/tag/v0.9.1\n\n### Bug Fixes:\n * Require namespace for Ark's CRDs to already exist at server startup (#676, @skriss)\n * Require all Ark CRDs to exist at server startup (#683, @skriss)\n * Fix `latest` tagging in Makefile (#690, @skriss)\n * Make Ark compatible with clusters that don't have the `rbac.authorization.k8s.io/v1` API group (#682, @nrb)\n * Don't consider missing snapshots an error during backup deletion, limit backup deletion requests per backup to 1 (#687, @skriss)\n\n\n## v0.9.0\n#### 2018-07-06\n### Download\n - https://github.com/heptio/ark/releases/tag/v0.9.0\n\n### Highlights:\n * Ark now has support for backing up and restoring Kubernetes volumes using a free open-source backup tool called [restic](https://github.com/restic/restic).\n This provides users an out-of-the-box solution for backing up and restoring almost any type of Kubernetes volume, whether or not it has snapshot support\n integrated with Ark. For more information, see the [documentation](https://github.com/vmware-tanzu/velero/blob/main/docs/restic.md).\n * Support for Prometheus metrics has been added! View total number of backup attempts (including success or failure), total backup size in bytes, and backup\n durations. More metrics coming in future releases!\n\n### All New Features:\n * Add restic support (#508 #532 #533 #534 #535 #537 #540 #541 #545 #546 #547 #548 #555 #557 #561 #563 #569 #570 #571 #606 #608 #610 #621 #631 #636, @skriss)\n * Add prometheus metrics (#531 #551 #564, @ashish-amarnath @nrb)\n * When backing up a service account, include cluster roles/cluster role bindings that reference it (#470, @skriss)\n * When restoring service accounts, copy secrets/image pull secrets into the target cluster even if the service account already exists (#403, @nrb)\n\n### Bug Fixes / Other Changes:\n * Upgrade to Kubernetes 1.10 dependencies (#417, @skriss)\n * Upgrade to go 1.10 and alpine 3.7 (#456, @skriss)\n * Display no excluded resources/namespaces as `` rather than `*` (#453, @nrb)\n * Skip completed jobs and pods when restoring (#463, @nrb)\n * Set namespace correctly when syncing backups from object storage (#472, @skriss)\n * When building on macOS, bind-mount volumes with delegated config (#478, @skriss)\n * Add replica sets and daemonsets to cohabiting resources so they're not backed up twice (#482 #485, @skriss)\n * Shut down the Ark server gracefully on SIGINT/SIGTERM (#483, @skriss)\n * Only back up resources that support GET and DELETE in addition to LIST and CREATE (#486, @nrb)\n * Show a better error message when trying to get an incomplete restore's logs (#496, @nrb)\n * Stop processing when setting a backup deletion request's phase to `Deleting` fails (#500, @nrb)\n * Add library code to install Ark's server components (#437 #506, @marpaia)\n * Properly handle errors when backing up additional items (#512, @carlpett)\n * Run post hooks even if backup actions fail (#514, @carlpett)\n * GCP: fail backup if upload to object storage fails (#510, @nrb)\n * AWS: don't require `region` as part of backup storage provider config (#455, @skriss)\n * Ignore terminating resources while doing a backup (#526, @yastij)\n * Log to stdout instead of stderr (#553, @ncdc)\n * Move sample minio deployment's config to an emptyDir (#566, @runyontr)\n * Add `omitempty` tag to optional API fields (@580, @nikhita)\n * Don't restore PVs with a reclaim policy of `Delete` and no snapshot (#613, @ncdc)\n * Don't restore mirror pods (#619, @ncdc)\n\n### Docs Contributors:\n * @gianrubio\n * @castrojo\n * @dhananjaysathe\n * @c-knowles\n * @mattkelly\n * @ae-v\n * @hamidzr\n" }, { "path": "changelogs/CHANGELOG-1.0.md", "content": "## v1.0.0\n#### 2019-05-20\n\n### Highlights\n- We've added a new command, `velero install`, to make it easier to get up and running with Velero. This CLI command replaces the static YAML installation files that were previously part of release tarballs. See the updated [install instructions][3] for more information.\n- We've made a number of improvements to the plugin framework:\n - we've reorganized the relevant packages to minimize the import surface for plugin authors\n - all plugins are now wrapped in panic handlers that will report information on panics back to Velero\n - Velero's `--log-level` flag is now passed to plugin implementations\n - Errors logged within plugins are now annotated with the file/line of where the error occurred\n - Restore item actions can now optionally return a list of additional related items that should be restored\n - Restore item actions can now indicate that an item *should not* be restored\n- For Azure installation, the `cloud-credentials` secret can now be created from a file containing a list of environment variables. Note that `velero install` always uses this method of providing credentials for Azure. For more details, see [Run on Azure][0].\n- We've added a new phase, `PartiallyFailed`, for both backups and restores. This new phase is used for backups/restores that successfully process some but not all of their items.\n- We removed all legacy Ark references, including API types, prometheus metrics, restic & hook annotations, etc.\n- The restic integration remains a **beta feature**. Please continue to try it out and provide feedback, and we'll be working over the next couple of releases to bring it to GA.\n\n### Breaking Changes\n\n#### API\n* All legacy Ark data types and pre-1.0 compatibility code has been removed. Users should migrate any backups created pre-v0.11.0 with the `velero migrate-backups` command, available in [v0.11.1][2].\n\n#### Image\n* The base container image has been switched to `ubuntu:bionic`\n\n#### Labels/Annotations/Metrics\n* The \"ark\" annotations for specifying hooks are no longer supported, and have been replaced with \"velero\"-based equivalents.\n* The \"ark\" annotation for specifying restic backups is no longer supported, and has been replaced with a \"velero\"-based equivalent.\n* The \"ark\" prometheus metrics no longer exist, and have been replaced with \"velero\"-based equivalents.\n\n#### Plugin Development\n* `BlockStore` plugins are now named `VolumeSnapshotter` plugins\n* Plugin APIs have moved to reduce the import surface:\n * Plugin gRPC servers live in `github.com/heptio/velero/pkg/plugin/framework`\n * Plugin interface types live in `github.com/heptio/velero/pkg/plugin/velero`\n* RestoreItemAction interface now takes the original item from the backup as a parameter\n* RestoreItemAction plugins can now return additional items to restore\n* RestoreItemAction plugins can now skip restoring an item\n* Plugins may now send stack traces with errors to the Velero server, so that the errors may be put into the server log\n* Plugins must now be \"namespaced,\" using `example.domain.com/plugin-name` format\n * For external ObjectStore and VolumeSnapshotter plugins. this name will also be the provider name in BackupStorageLoction and VolumeSnapshotLocation objects\n* `--log-level` flag is now passed to all plugins\n\n#### Validation\n* Configs for Azure, AWS, and GCP are now checked for invalid or extra keys, and the server is halted if any are found\n\n### Download\n- https://github.com/heptio/velero/releases/tag/v1.0.0\n\n### Container Image\n`gcr.io/heptio-images/velero:v1.0.0`\n\n### Documentation\nhttps://velero.io/docs/v1.0.0/\n\n### Upgrading\nTo upgrade from a previous version of Velero, see our [upgrade instructions][1].\n\n### All Changes\n* Change base images to ubuntu:bionic (#1488, @skriss)\n* Expose the timestamp of the last successful backup in a gauge (#1448, @fabito)\n* check backup existence before download (#1447, @fabito)\n* Use `latest` image tag if no version information is provided at build time (#1439, @nrb)\n* switch from `restic stats` to `restic snapshots` for checking restic repository existence (#1416, @skriss)\n* GCP: add optional 'project' config to volume snapshot location for if snapshots are in a different project than the IAM account (#1405, @skriss)\n* Disallow bucket names starting with '-' (#1407, @nrb)\n* Shorten label values when they're longer than 63 characters (#1392, @anshulc)\n* Fail backup if it already exists in object storage. (#1390, @ncdc,carlisia)\n* Add PartiallyFailed phase for backups, log + continue on errors during backup process (#1386, @skriss)\n* Remove deprecated \"hooks\" for backups (they've been replaced by \"pre hooks\") (#1384, @skriss)\n* Restic repo ensurer: return error if new repository does not become ready within a minute, and fix channel closing/deletion (#1367, @skriss)\n* Support non-namespaced names for built-in plugins (#1366, @nrb)\n* Change container base images to debian:stretch-slim and upgrade to go 1.12 (#1365, @skriss)\n* Azure: allow credentials to be provided in a .env file (path specified by $AZURE_CREDENTIALS_FILE), formatted like (#1364, @skriss):\n ```\n AZURE_TENANT_ID=${AZURE_TENANT_ID}\n AZURE_SUBSCRIPTION_ID=${AZURE_SUBSCRIPTION_ID}\n AZURE_CLIENT_ID=${AZURE_CLIENT_ID}\n AZURE_CLIENT_SECRET=${AZURE_CLIENT_SECRET}\n AZURE_RESOURCE_GROUP=${AZURE_RESOURCE_GROUP} \n ```\n* Instantiate the plugin manager with the per-restore logger so plugin logs are captured in the per-restore log (#1358, @skriss)\n* Add gauge metrics for number of existing backups and restores (#1353, @fabito)\n* Set default TTL for backups (#1352, @vorar)\n* Validate that there can't be any duplicate plugin name, and that the name format is `example.io/name`. (#1339, @carlisia)\n* AWS/Azure/GCP: fail fast if unsupported keys are provided in BackupStorageLocation/VolumeSnapshotLocation config (#1338, @skriss)\n* `velero backup logs` & `velero restore logs`: show helpful error message if backup/restore does not exist or is not finished processing (#1337, @skriss)\n* Add support for allowing a RestoreItemAction to skip item restore. (#1336, @sseago)\n* Improve error message around invalid S3 URLs, and gracefully handle trailing backslashes. (#1331, @skriss)\n* Set backup's start timestamp before patching it to InProgress so start times display in `velero backup get` while in progress (#1330, @skriss)\n* Added ability to dynamically disable controllers (#1326, @amanw)\n* Remove deprecated code in preparation for v1.0 release (#1323, @skriss):\n - remove ark.heptio.com API group\n - remove support for reading ark-backup.json files from object storage\n - remove Ark field from RestoreResult type\n - remove support for \"hook.backup.ark.heptio.com/...\" annotations for specifying hooks\n - remove support for $HOME/.config/ark/ client config directory\n - remove support for restoring Azure snapshots using short snapshot ID formats in backup metadata\n - stop applying \"velero-restore\" label to restored resources and remove it from the API pkg\n - remove code that strips the \"gc.ark.heptio.com\" finalizer from backups\n - remove support for \"backup.ark.heptio.com/...\" annotations for requesting restic backups\n - remove \"ark\"-prefixed prometheus metrics\n - remove VolumeBackups field and related code from Backup's status\n* Rename BlockStore plugin to VolumeSnapshotter (#1321, @skriss)\n* Bump plugin ProtocolVersion to version 2 (#1319, @carlisia)\n* Remove Warning field from restore item action output (#1318, @skriss)\n* Fix for #1312, use describe to determine if AWS EBS snapshot is encrypted and explicitly pass that value in EC2 CreateVolume call. (#1316, @mstump)\n* Allow restic restore helper image name to be optionally specified via ConfigMap (#1311, @skriss)\n* Compile only once to lower the initialization cost for regexp.MustCompile. (#1306, @pei0804)\n* Enable restore item actions to return additional related items to be restored; have pods return PVCs and PVCs return PVs (#1304, @skriss)\n* Log error locations from plugin logger, and don't overwrite them in the client logger if they exist already (#1301, @skriss)\n* Send stack traces from plugin errors to Velero via gRPC so error location info can be logged (#1300, @skriss)\n* Azure: restore volumes in the original region's zone (#1298, @sylr)\n* Check for and exclude hostPath-based persistent volumes from restic backup (#1297, @skriss)\n* Make resticrepositories non-restorable resources (#1296, @skriss)\n* Gracefully handle failed API groups from the discovery API (#1293, @fabito)\n* Add `velero install` command for basic use cases. (#1287, @nrb)\n* Collect 3 new metrics: backup_deletion_{attempt|failure|success}_total (#1280, @fabito)\n* Pass --log-level flag to internal/external plugins, matching Velero server's log level (#1278, @skriss)\n* AWS EBS Volume IDs now contain AZ (#1274, @tsturzl)\n* Add panic handlers to all server-side plugin methods (#1270, @skriss)\n* Move all the interfaces and associated types necessary to implement all of the Velero plugins to under the new package `velero`. (#1264, @carlisia)\n* Update `velero restore` to not open every single file open during extraction of the data (#1261, @asaf)\n* Remove restore code that waits for a PV to become Available (#1254, @skriss)\n* Improve `describe` output\n* Move Phase to right under Metadata(name/namespace/label/annotations)\n* Move Validation errors: section right after Phase: section and only show it if the item has a phase of FailedValidation\n* For restores move Warnings and Errors under Validation errors. Leave their display as is. (#1248, @DheerajSShetty)\n* Don't remove storage class from a persistent volume when restoring it (#1246, @skriss)\n* Need to defer closing the the ReadCloser in ObjectStoreGRPCServer.GetObject (#1236, @DheerajSShetty)\n* Update Kubernetes dependencies to match v1.12, and update Azure SDK to v19.0.0 (GA) (#1231, @skriss)\n* Remove pkg/util/collections/map_utils.go, replace with structured API types and apimachinery's unstructured helpers (#1146, @skriss)\n* Add original resource (from backup) to restore item action interface (#1123, @mwieczorek)\n\n\n[0]: https://velero.io/docs/v1.0.0/azure-config\n[1]: https://velero.io/docs/v1.0.0/upgrade-to-1.0\n[2]: https://github.com/heptio/velero/releases/tag/v0.11.1\n[3]: https://velero.io/docs/v1.0.0/install-overview\n" }, { "path": "changelogs/CHANGELOG-1.1.md", "content": "## v1.1.0\n#### 2019-08-22\n\n### Download\n- https://github.com/heptio/velero/releases/tag/v1.1.0\n\n### Container Image\n`gcr.io/heptio-images/velero:v1.1.0`\n\n### Documentation\nhttps://velero.io/docs/v1.1.0/\n\n### Upgrading\n\n**If you are running Velero in a non-default namespace**, i.e. any namespace other than `velero`, manual intervention is required when upgrading to v1.1. See [upgrading to v1.1](https://velero.io/docs/v1.1.0/upgrade-to-1.1/) for details.\n\n### Highlights\n\n#### Improved Restic Support\n\nA big focus of our work this cycle was continuing to improve support for restic. To that end, we’ve fixed the following bugs:\n\n\n- Prior to version 1.1, restic backups could be delayed or failed due to long-lived locks on the repository. Now, Velero removes stale locks from restic repositories every 5 minutes, ensuring they do not interrupt normal operations. \n- Previously, the PodVolumeBackup custom resources that represented a restic backup within a cluster were not synchronized between clusters, making it unclear what restic volumes were available to restore into a new cluster. In version 1.1, these resources are synced into clusters, so they are more visible to you when you are trying to restore volumes. \n- Originally, Velero would not validate the host path in which volumes were mounted on a given node. If a node did not expose the filesystem correctly, you wouldn’t know about it until a backup failed. Now, Velero’s restic server will validate that the directory structure is correct on startup, providing earlier feedback when it’s not. \n- Velero’s restic support is intended to work on a broad range of volume types. With the general release of the [Container Storage Interface API](https://kubernetes.io/blog/2019/01/15/container-storage-interface-ga/), Velero can now use restic to back up CSI volumes. \n\nAlong with our bug fixes, we’ve provided an easier way to move restic backups between storage providers. Different providers often have different StorageClasses, requiring user intervention to make restores successfully complete.\n\nTo make cross-provider moves simpler, we’ve introduced a StorageClass remapping plug-in. It allows you to automatically translate one StorageClass on PersistentVolumeClaims and PersistentVolumes to another. You can read more about it in our [documentation](https://velero.io/docs/v1.1.0/restore-reference/#changing-pv-pvc-storage-classes).\n\n#### Quality-of-Life Improvements\n\nWe’ve also made several other enhancements to Velero that should benefit all users.\n\nUsers sometimes ask about recommendations for Velero’s resource allocation within their cluster. To help with this concern, we’ve added default resource requirements to the Velero Deployment and restic init containers, along with configurable requests and limits for the restic DaemonSet. All these values can be adjusted if your environment requires it.\n\nWe’ve also taken some time to improve Velero for the future by updating the Deployment and DaemonSet to use the apps/v1 API group, which will be the [default in Kubernetes 1.16](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.16.md#action-required-3). This change means that `velero install` and the `velero plugin` commands will require Kubernetes 1.9 or later to work. Existing Velero installs will continue to work without needing changes, however.\n\nIn order to help you better understand what resources have been backed up, we’ve added a list of resources in the `velero backup describe --details` command. This change makes it easier to inspect a backup without having to download and extract it.\n\nIn the same vein, we’ve added the ability to put custom tags on cloud-provider snapshots. This approach should provide a better way to keep track of the resources being created in your cloud account. To add a label to a snapshot at backup time, use the `--labels` argument in the `velero backup create` command.\n\nOur final change for increasing visibility into your Velero installation is the `velero plugin get` command. This command will report all the plug-ins within the Velero deployment..\n\nVelero has previously used a restore-only flag on the server to control whether a cluster could write backups to object storage. With Velero 1.1, we’ve now moved the restore-only behavior into read-only BackupStorageLocations. This move means that the Velero server can use a BackupStorageLocation as a source to restore from, but not for backups, while still retaining the ability to back up to other configured locations. In the future, the `--restore-only` flag will be removed in favor of configuring read-only BackupStorageLocations.\n\n#### Community Contributions\n\nWe appreciate all community contributions, whether they be pull requests, bug reports, feature requests, or just questions. With this release, we wanted to draw attention to a few contributions in particular:\n\nFor users of node-based IAM authentication systems such as kube2iam, `velero install` now supports the `--pod-annotations` argument for applying necessary annotations at install time. This support should make `velero install` more flexible for scenarios that do not use Secrets for access to their cloud buckets and volumes. You can read more about how to use this new argument in our [AWS documentation](https://velero.io/docs/v1.1.0/aws-config/#alternative-setup-permissions-using-kube2iam). Huge thanks to [Traci Kamp](https://github.com/tlkamp) for this contribution.\n\nStructured logging is important for any application, and Velero is no different. Starting with version 1.1, the Velero server can now output its logs in a JSON format, allowing easier parsing and ingestion. Thank you to [Donovan Carthew](https://github.com/carthewd) for this feature.\n\nAWS supports multiple profiles for accessing object storage, but in the past Velero only used the default. With v.1.1, you can set the `profile` key on yourBackupStorageLocation to specify an alternate profile. If no profile is set, the default one is used, making this change backward compatible. Thanks [Pranav Gaikwad](https://github.com/pranavgaikwad) for this change.\n\nFinally, thanks to testing by [Dylan Murray](https://github.com/dymurray) and [Scott Seago](https://github.com/sseago), an issue with running Velero in non-default namespaces was found in our beta version for this release. If you’re running Velero in a namespace other than `velero`, please follow the [upgrade instructions](https://velero.io/docs/v1.1.0/upgrade-to-1.1/).\n\n### All Changes\n * Add the prefix to BSL config map so that object stores can use it when initializing (#1767, @betta1)\n * Use `VELERO_NAMESPACE` to determine what namespace Velero server is running in. For any v1.0 installations using a different namespace, the `VELERO_NAMESPACE` environment variable will need to be set to the correct namespace. (#1748, @nrb)\n * support setting CPU/memory requests with unbounded limits using velero install (#1745, @prydonius)\n * sort output of resource list in `velero backup describe --details` (#1741, @prydonius)\n * adds the ability to define custom tags to be added to snapshots by specifying custom labels on the Backup CR with the velero backup create --labels flag (#1729, @prydonius)\n * Restore restic volumes from PodVolumeBackups CRs (#1723, @carlisia)\n * properly restore PVs backed up with restic and a reclaim policy of \"Retain\" (#1713, @skriss)\n * Make `--secret-file` flag on `velero install` optional, add `--no-secret` flag for explicit confirmation (#1699, @nrb)\n * Add low cpu/memory limits to the restic init container. This allows for restoration into namespaces with quotas defined. (#1677, @nrb)\n * Adds configurable CPU/memory requests and limits to the restic DaemonSet generated by velero install. (#1710, @prydonius)\n * remove any stale locks from restic repositories every 5m (#1708, @skriss)\n * error if backup storage location's Bucket field also contains a prefix, and gracefully handle leading/trailing slashes on Bucket and Prefix fields. (#1694, @skriss)\n * enhancement: allow option to choose JSON log output (#1654, @carthewd)\n * Adds configurable CPU/memory requests and limits to the Velero Deployment generated by velero install. (#1678, @prydonius)\n * Store restic PodVolumeBackups in obj storage & use that as source of truth like regular backups. (#1577, @carlisia)\n * Update Velero Deployment to use apps/v1 API group. `velero install` and `velero plugin add/remove` commands will now require Kubernetes 1.9+ (#1673, @nrb)\n * Respect the --kubecontext and --kubeconfig arguments for `velero install`. (#1656, @nrb)\n * add plugin for updating PV & PVC storage classes on restore based on a config map (#1621, @skriss)\n * Add restic support for CSI volumes (#1615, @nrb)\n * bug fix: Fixed namespace usage with cli command 'version' (#1630, @jwmatthews)\n * enhancement: allow users to specify additional Velero/Restic pod annotations on the command line with the pod-annotations flag. (#1626, @tlkamp)\n * adds validation for pod volumes hostPath mount on restic server startup (#1616, @prydonius)\n * enable support for ppc64le architecture (#1605, @prajyot)\n * bug fix: only restore additional items returned from restore item actions if they match the restore's namespace/resource selectors (#1612, @skriss)\n * add startTimestamp and completionTimestamp to PodVolumeBackup and PodVolumeRestore status fields (#1609, @prydonius)\n * bug fix: respect namespace selector when determining which restore item actions to run (#1607, @skriss)\n * ensure correct backup item actions run with namespace selector (#1601, @prydonius)\n * allows excluding resources from backups with the `velero.io/exclude-from-backup=true` label (#1588, @prydonius)\n * ensures backup item action modifications to an item's namespace/name are saved in the file path in the tarball (#1587, @prydonius)\n * Hides `velero server` and `velero restic server` commands from the list of available commands as these are not intended for use by the velero CLI user. (#1561, @prydonius)\n * remove dependency on glog, update to klog (#1559, @skriss)\n * move issue-template-gen from docs/ to hack/ (#1558, @skriss)\n * fix panic when processing DeleteBackupRequest objects without labels (#1556, @prydonius)\n * support for multiple AWS profiles (#1548, @pranavgaikwad)\n * Add CLI command to list (get) all Velero plugins (#1535, @carlisia)\n * Added author as a tag on blog post. Should fix 404 error when trying to follow link as specified in issue #1522. (#1522, @coonsd)\n * Allow individual backup storage locations to be read-only (#1517, @skriss)\n * Stop returning an error when a restic volume is empty since it is a valid scenario. (#1480, @carlisia)\n * add ability to use wildcard in includes/excludes (#1428, @guilhem)\n" }, { "path": "changelogs/CHANGELOG-1.10.md", "content": " ## v1.10.0\n### 2022-11-23\n\n### Download\nhttps://github.com/vmware-tanzu/velero/releases/tag/v1.10.0\n\n### Container Image\n`velero/velero:v1.10.0`\n\n### Documentation\nhttps://velero.io/docs/v1.10/\n\n### Upgrading\nhttps://velero.io/docs/v1.10/upgrade-to-1.10/\n\n### Highlights\n\n#### Unified Repository and Kopia integration\nIn this release, we introduced the Unified Repository architecture to build a data path where data movers and the backup repository are decoupled and a unified backup repository could serve various data movement activities.\n\nIn this release, we also deeply integrate Velero with Kopia, specifically, Kopia's uploader modules are isolated as a generic file system uploader; Kopia's repository modules are encapsulated as the unified backup repository.\n\nFor more information, refer to the [design document](https://github.com/vmware-tanzu/velero/blob/v1.10.0/design/unified-repo-and-kopia-integration/unified-repo-and-kopia-integration.md).\n\n#### File system backup refactor\nVelero's file system backup (a.k.s. pod volume backup or formerly restic backup) is refactored as the first user of the Unified Repository architecture. Specifically, we added a new path, the Kopia path, besides the existing Restic path. While Restic path is still available and set as default, you can opt in Kopia path by specifying the `uploader-type` parameter at installation time. Meanwhile, you are free to restore from existing backups under either path, Velero dynamically switches to the correct path to process the restore.\n\nBecause of the new path, we renamed some modules and parameters, refer to the Break Changes section for more details.\n\nFor more information, visit the [file system backup document](https://velero.io/docs/v1.10/file-system-backup/) and [v1.10 upgrade guide document](https://velero.io/docs/v1.10/upgrade-to-1.10/).\n\nMeanwhile, we've created a performance guide for both Restic path and Kopia path, which helps you to choose between the two paths and provides you the best practice to configure them under different scenarios. Please note that the results in the guide are based on our testing environments, you may get different results when testing in your own ones. For more information, visit the [performance guide document](https://velero.io/docs/v1.10/performance-guidance/).\n\n#### Plugin versioning V1 refactor\nIn this release, Velero moves plugins BackupItemAction, RestoreItemAction and VolumeSnapshotterAction to version v1, this allows future plugin changes that do not support backward compatibility, so is a preparation for various complex tasks, for example, data movement tasks.\nFor more information, refer to the [plugin versioning design document](https://github.com/vmware-tanzu/velero/blob/v1.10.0/design/plugin-versioning.md).\n\n#### Refactor the controllers using Kubebuilder v3\nIn this release we continued our code modernization work, rewriting some controllers using Kubebuilder v3. This work is ongoing and we will continue to make progress in future releases.\n\n#### Add credentials to volume snapshot locations\nIn this release, we enabled dedicate credentials options to volume snapshot locations so that you can specify credentials per volume snapshot location as same as backup storage location.\n\nFor more information, please visit the [locations document](https://velero.io/docs/v1.10/locations/).\n\n#### CSI snapshot enhancements\nIn this release we added several changes to enhance the robustness of CSI snapshot procedures, for example, some protection code for error handling, and a mechanism to skip exclusion checks so that CSI snapshot works with various backup resource filters.\n\n#### Backup schedule pause/unpause\nIn this release, Velero supports to pause/unpause a backup schedule during or after its creation. Specifically:\n\nAt creation time, you can specify `–paused` flag to `velero schedule create` command, if so, you will create a paused schedule that will not run until it is unpaused\nAfter creation, you can run `velero schedule pause` or `velero schedule unpause` command to pause/unpause a schedule\n\n#### Runtime and dependencies\nIn order to fix CVEs, we changed Velero's runtime and dependencies as follows:\n\nBump go runtime to v1.18.8\nBump some core dependent libraries to newer versions\nCompile Restic (v0.13.1) with go 1.18.8 instead of packaging the official binary\n\n\n#### Breaking changes\nDue to file system backup refactor, below modules and parameters name have been changed in this release:\n\n`restic` daemonset is renamed to `node-agent`\n`resticRepository` CR is renamed to `backupRepository`\n`velero restic repo` command is renamed to `velero repo`\n`velero-restic-credentials` secret is renamed to `velero-repo-credentials`\n`default-volumes-to-restic` parameter is renamed to `default-volumes-to-fs-backup`\n`restic-timeout` parameter is renamed to `fs-backup-timeout`\n`default-restic-prune-frequency` parameter is renamed to `default-repo-maintain-frequency`\n\n#### Upgrade\nDue to the major changes of file system backup, the old upgrade steps are not suitable any more. For the new upgrade steps, visit [v1.10 upgrade guide document](https://velero.io/docs/v1.10/upgrade-to-1.10/).\n\n#### Limitations/Known issues\nIn this release, Kopia backup repository (so the Kopia path of file system backup) doesn't support self signed certificate for S3 compatible storage. To track this problem, refer to this [Velero issue](https://github.com/vmware-tanzu/velero/issues/5123) or [Kopia issue](https://github.com/kopia/kopia/issues/1443). \n\nDue to the code change in Velero, there will be some code change required in vSphere plugin, without which the functionality may be impacted. Therefore, if you are using vSphere plugin in your workflow, please hold the upgrade until the issue [#485](https://github.com/vmware-tanzu/velero-plugin-for-vsphere/issues/485) is fixed in vSphere plugin.\n\n### All changes\n \n * Restore ClusterBootstrap before Cluster otherwise a new default ClusterBootstrap object is create for the cluster (#5616, @ywk253100)\n * Add compile restic binary for CVE fix (#5574, @qiuming-best)\n * Fix controller problematic log output (#5572, @qiuming-best)\n * Enhance the restore priorities list to support specifying the low prioritized resources that need to be restored in the last (#5535, @ywk253100)\n * fix restic backup progress error (#5534, @qiuming-best)\n * fix restic backup failure with self-signed certification backend storage (#5526, @qiuming-best)\n * Add credential store in backup deletion controller to support VSL credential. (#5521, @blackpiglet)\n * Fix issue 5505: the pod volume backups/restores except the first one fail under the kopia path if \"AZURE_CLOUD_NAME\" is specified (#5512, @Lyndon-Li)\n * After Pod Volume Backup/Restore refactor, remove all the unreasonable appearance of \"restic\" word from documents (#5499, @Lyndon-Li)\n * Refactor Pod Volume Backup/Restore doc to match the new behavior (#5484, @Lyndon-Li)\n * Remove redundancy code block left by #5388. (#5483, @blackpiglet)\n * Issue fix 5477: create the common way to support S3 compatible object storages that work for both Restic and Kopia; Keep the resticRepoPrefix parameter for compatibility (#5478, @Lyndon-Li)\n * Update the k8s.io dependencies to 0.24.0.\nThis also required an update to github.com/bombsimon/logrusr/v3.\nRemoved the `WithClusterName` method\nas it is a \"legacy field that was\nalways cleared by the system and never used\" as per upstream k8s\nhttps://github.com/kubernetes/apimachinery/blob/release-1.24/pkg/apis/meta/v1/types.go#L257-L259 (#5471, @kcboyle)\n * Add v1.10 velero upgrade doc (#5468, @qiuming-best)\n * Upgrade velero docker image to use go 1.18 and upgrade golangci-lint to 1.45.0 (#5459, @Lyndon-Li)\n * Add VolumeSnapshot client back. (#5449, @blackpiglet)\n * Change subcommand `velero restic repo` to `velero repo` (#5446, @allenxu404)\n * Remove irrational \"Restic\" names in Velero code after the PVBR refactor (#5444, @Lyndon-Li)\n * moved RIA execute input/output structs back to velero package (#5441, @sseago)\n * Rename Velero pod volume restore init helper from \"velero-restic-restore-helper\" to \"velero-restore-helper\" (#5432, @Lyndon-Li)\n * Skip the exclusion check for additional resources returned by BIA (#5429, @reasonerjt)\n * Change B/R describe CLI to support Kopia (#5412, @allenxu404)\n * Add nil check before execution of csi snapshot delete (#5401, @shubham-pampattiwar)\n * update velero using klog to version v2.9.0 (#5396, @blackpiglet)\n * Fix Test_prepareBackupRequest_BackupStorageLocation UT failure. (#5394, @blackpiglet)\n * Rename Velero daemonset from \"restic\" to \"node-agent\" (#5390, @Lyndon-Li)\n * Add some corner cases checking for CSI snapshot in backup controller. (#5388, @blackpiglet)\n * Fix issue 5386: Velero providers a full URL as the S3Url while the underlying minio client only accept the host part of the URL as the endpoint and the schema should be specified separately. (#5387, @Lyndon-Li)\n * Fix restore error with flag namespace-mappings (#5377, @qiuming-best)\n * Pod Volume Backup/Restore Refactor: Rename parameters in CRDs and commands to remove \"Restic\" word (#5370, @Lyndon-Li)\n * Added backupController's UT to test the prepareBackupRequest() method BackupStorageLocation processing logic (#5362, @niulechuan)\n * Fix a repoEnsurer problem introduced by the refactor - The repoEnsurer didn't check \"\" state of BackupRepository, as a result, the function GetBackupRepository always returns without an error even though the ensreReady is specified. (#5359, @Lyndon-Li)\n * Add E2E test for schedule backup (#5355, @danfengliu)\n * Add useOwnerReferencesInBackup field doc for schedule. (#5353, @cleverhu)\n * Clarify the help message for the default value of parameter --snapshot-volumes, when it's not set. (#5350, @blackpiglet)\n * Fix restore cmd extraflag overwrite bug (#5347, @qiuming-best)\n * Resolve gopkg.in/yaml.v3 vulnerabilities by upgrading gopkg.in/yaml.v3 to v3.0.1 (#5344, @kaovilai)\n * Increase ensure restic repository timeout to 5m (#5335, @shubham-pampattiwar)\n * Add opt-in and opt-out PersistentVolume backup to E2E tests (#5331, @danfengliu)\n * Cancel downloadRequest when timeout without downloadURL (#5329, @kaovilai)\n * Fix PVB finds wrong parent snapshot (#5322, @qiuming-best)\n * Fix issue 4874 and 4752: check the daemonset pod is running in the node where the workload pod resides before running the PVB for the pod (#5319, @Lyndon-Li)\n * plugin versioning v1 refactor for VolumeSnapshotter (#5318, @sseago)\n * Change the status of restore to completed from partially failed when restore empty backup (#5314, @allenxu404)\n * RestoreItemAction v1 refactoring for plugin api versioning (#5312, @sseago)\n * Refactor the repoEnsurer code to use controller runtime client and wrap some common BackupRepository operations to share with other modules (#5308, @Lyndon-Li)\n * Remove snapshot related lister, informer and client from backup controller. (#5299, @jxun)\n * Remove github.com/apex/log logger. (#5297, @blackpiglet)\n * change CSISnapshotTimeout from pointer to normal variables. (#5294, @cleverhu)\n * Optimize code for restore exists resources. (#5293, @cleverhu)\n * Add more detailed comments for labels columns. (#5291, @cleverhu)\n * Add backup status checking in schedule controller. (#5283, @blackpiglet)\n * Add changes for problems/enhancements found during smoking test for Kopia pod volume backup/restore (#5282, @Lyndon-Li)\n * Support pause/unpause schedules (#5279, @ywk253100)\n * plugin/clientmgmt refactoring for BackupItemAction v1 (#5271, @sseago)\n * Don't move velero v1 plugins to new proto dir (#5263, @sseago)\n * Fill gaps for Kopia path of PVBR: integrate Repo Manager with Unified Repo; pass UploaderType to PVBR backupper and restorer; pass RepositoryType to BackupRepository controller and Repo Ensurer (#5259, @Lyndon-Li)\n * Add csiSnapshotTimeout for describe backup (#5252, @cleverhu)\n * equip gc controller with configurable frequency (#5248, @allenxu404)\n * Fix nil pointer panic when restoring StatefulSets (#5247, @divolgin)\n * Controller refactor code modifications. (#5241, @jxun)\n * Fix edge cases for already exists resources (#5239, @shubham-pampattiwar)\n * Check for empty ns list before checking nslist[0] (#5236, @sseago)\n * Remove reference to non-existent doc (#5234, @reasonerjt)\n * Add changes for Kopia Integration: Kopia Lib - method implementation. Add changes to write Kopia Repository logs to Velero log (#5233, @Lyndon-Li)\n * Add changes for Kopia Integration: Kopia Lib - initialize Kopia repo (#5231, @Lyndon-Li)\n * Uploader Implementation: Kopia backup and restore (#5221, @qiuming-best)\n * Migrate backup sync controller from code-generator to kubebuilder. (#5218, @jxun)\n * check vsc null pointer (#5217, @lilongfeng0902)\n * Refactor GCController with kubebuilder (#5215, @allenxu404)\n * Uploader Implementation: Restic backup and restore (#5214, @qiuming-best)\n * Add parameter \"uploader-type\" to velero server (#5212, @reasonerjt)\n * Add annotation \"pv.kubernetes.io/migrated-to\" for CSI checking. (#5181, @jxun)\n * Add changes for Kopia Integration: Unified Repository Provider - method implementation (#5179, @Lyndon-Li)\n * Treat namespaces with exclude label as excludedNamespaces\nRelated issue: #2413 (#5178, @allenxu404)\n * Reduce CRD size. (#5174, @jxun)\n * Fix restic backups to multiple backup storage locations bug (#5172, @qiuming-best)\n * Add changes for Kopia Integration: Unified Repository Provider - Repo Password (#5167, @Lyndon-Li)\n * Skip registering \"crd-remap-version\" plugin when feature flag \"EnableAPIGroupVersions\" is set (#5165, @reasonerjt)\n * Kopia uploader integration on shim progress uploader module (#5163, @qiuming-best)\n * Add labeled and unlabeled events for PR changelog check action. (#5157, @jxun)\n * VolumeSnapshotLocation refactor with kubebuilder. (#5148, @jxun)\n * Delay CA file deletion in PVB controller. (#5145, @jxun)\n * This commit splits the pkg/restic package into several packages to support Kopia integration works (#5143, @ywk253100)\n * Kopia Integration: Add the Unified Repository Interface definition. Kopia Integration: Add the changes for Unified Repository storage config. Related Issues; #5076, #5080 (#5142, @Lyndon-Li)\n * Update the CRD for kopia integration (#5135, @reasonerjt)\n * Let \"make shell xxx\" respect GOPROXY (#5128, @reasonerjt)\n * Modify BackupStoreGetter to avoid BSL spec changes (#5122, @sseago)\n * Dump stack trace when the plugin server handles panic (#5110, @reasonerjt)\n * Make CSI snapshot creation timeout configurable. (#5104, @jxun)\n * Fix bsl validation bug: the BSL is validated continually and doesn't respect the validation period configured (#5101, @ywk253100)\n * Exclude \"csinodes.storage.k8s.io\" and \"volumeattachments.storage.k8s.io\" from restore by default. (#5064, @jxun)\n * Move 'velero.io/exclude-from-backup' label string to const (#5053, @niulechuan)\n * Modify Github actions. (#5052, @jxun)\n * Fix typo in doc, in https://velero.io/docs/main/restore-reference/ \"Restore order\" section, \"Mamespace\" should be \"Namespace\". (#5051, @niulechuan)\n * Delete opened issues triage action. (#5041, @jxun)\n * When spec.RestoreStatus is empty, don't restore status (#5008, @sseago)\n * Added DownloadTargetKindCSIBackupVolumeSnapshots for retrieving the signed URL to download only the ``-csi-volumesnapshots.json.gz and DownloadTargetKindCSIBackupVolumeSnapshotContents to download only ``-csi-volumesnapshotcontents.json.gz in the DownloadRequest CR structure. These files are already present in the backup layout. (#4980, @anshulahuja98)\n * Refactor BackupItemAction proto and related code to backupitemaction/v1 package. This is part of implementation of the plugin version design https://github.com/vmware-tanzu/velero/blob/main/design/plugin-versioning.md (#4943, @phuongatemc)\n * Unified Repository Design (#4926, @Lyndon-Li)\n * Add credentials to volume snapshot locations (#4864, @sseago)" }, { "path": "changelogs/CHANGELOG-1.11.md", "content": "## v1.11\n### 2023-04-07\n\n### Download\nhttps://github.com/vmware-tanzu/velero/releases/tag/v1.11.0\n\n### Container Image\n`velero/velero:v1.11.0`\n\n### Documentation\nhttps://velero.io/docs/v1.11/\n\n### Upgrading\nhttps://velero.io/docs/v1.11/upgrade-to-1.11/\n\n### Highlights\n\n#### BackupItemAction v2\nThis feature implements the BackupItemAction v2. BIA v2 has two new methods: Progress() and Cancel() and modifies the Execute() return value.\n\nThe API change is needed to facilitate long-running BackupItemAction plugin actions that may not be complete when the Execute() method returns. This will allow long-running BackupItemAction plugin actions to continue in the background while the Velero moves to the following plugin or the next item.\n\n#### RestoreItemAction v2\nThis feature implemented the RestoreItemAction v2. RIA v2 has three new methods: Progress(), Cancel(), and AreAdditionalItemsReady(), and it modifies RestoreItemActionExecuteOutput() structure in the RIA return value.\n\nThe Progress() and Cancel() methods are needed to facilitate long-running RestoreItemAction plugin actions that may not be complete when the Execute() method returns. This will allow long-running RestoreItemAction plugin actions to continue in the background while the Velero moves to the following plugin or the next item. The AreAdditionalItemsReady() method is needed to allow plugins to tell Velero to wait until the returned additional items have been restored and are ready for use in the cluster before restoring the current item.\n\n#### Plugin Progress Monitoring\nThis is intended as a replacement for the previously-approved Upload Progress Monitoring design ([Upload Progress Monitoring](https://github.com/vmware-tanzu/velero/blob/main/design/upload-progress.md)) to expand the supported use cases beyond snapshot upload to include what was previously called Async Backup/Restore Item Actions.\n\n#### Flexible resource policy that can filter volumes to skip in the backup\nThis feature provides a flexible policy to filter volumes in the backup without requiring patching any labels or annotations to the pods or volumes. This policy is configured as k8s ConfigMap and maintained by the users themselves, and it can be extended to more scenarios in the future. By now, the policy rules out volumes from backup depending on the CSI driver, NFS setting, volume size, and StorageClass setting. Please refer to [policy API design](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/handle-backup-of-volumes-by-resources-filters.md#api-design) for the policy's ConifgMap format. It is not guaranteed to work on unofficial third-party plugins as it may not follow the existing backup workflow code logic of Velero.\n\n#### Resource Filters that can distinguish cluster scope and namespace scope resources\nThis feature adds four new resource filters for backup. The new filters are separated into cluster scope and namespace scope. Before this feature, Velero could not filter cluster scope resources precisely. This feature provides the ability and refactors existing resource filter parameters.\n\n#### Add a parameter for setting the Velero server connection with the k8s API server's timeout\nIn Velero, some code pieces need to communicate with the k8s API server. Before v1.11, these code pieces used hard-code timeout settings. This feature adds a resource-timeout parameter in the velero server binary to make it configurable.\n\n#### Add resource list in the output of the restore describe command\nBefore this feature, Velero restore didn't have a restored resources list as the Velero backup. It's not convenient for users to learn what is restored. This feature adds the resources list and the handling result of the resources (including created, updated, failed, and skipped).\n\n#### Refactor controllers with controller-runtime\nIn v1.11, Backup Controller and Restore controller are refactored with controller-runtime. Till v1.11, all Velero controllers use the controller-runtime framework.\n\n#### Runtime and dependencies\nTo fix CVEs and keep pace with Golang, Velero made changes as follows:\n* Bump Golang runtime to v1.19.8.\n* Bump several dependent libraries to new versions.\n* Compile Restic (v0.15.0) with Golang v1.19.8 instead of packaging the official binary.\n\n\n### Breaking changes\n* The Velero CSI plugin now determines whether to restore Volume's data from snapshots on the restore's restorePVs setting. Before v1.11, the CSI plugin doesn't check the restorePVs parameter setting. \n\n\n### Limitations/Known issues\n* The Flexible resource policy that can filter volumes to skip in the backup is not guaranteed to work on unofficial third-party plugins because the plugins may not follow the existing backup workflow code logic of Velero. The ConfigMap used as the policy is supposed to be maintained by users.\n\n\n### All Changes\n* Modify new scope resource filters name. (#6089, @blackpiglet)\n* Make Velero not exits when EnableCSI is on and CSI snapshot not installed (#6062, @blackpiglet)\n* Restore Services before Clusters (#6057, @ywk253100)\n* Fixed backup deletion bug related to async operations (#6041, @sseago)\n* Update Golang version to v1.19 for branch main. (#6039, @blackpiglet)\n* Fix issue #5972, don't assume errorField as error type when dealing with logger.WithError (#6028, @Lyndon-Li)\n* distinguish between New and InProgress operations (#6012, @sseago)\n* Modify golangci.yaml file. Resolve found lint issues. (#6008, @blackpiglet)\n* Remove Reference of itemsnapshotter (#5997, @reasonerjt)\n* minor fixes for backup_operations_controller (#5996, @sseago)\n* RIAv2 async operations controller work (#5993, @sseago)\n* Follow-on fixes for BIAv2 controller work (#5971, @sseago)\n* Refactor backup controller based on the controller-runtime framework. (#5969, @qiuming-best)\n* Fix client wait problem after async operation change, velero backup/restore --wait should check a full list of the terminal status (#5964, @Lyndon-Li)\n* Fix issue #5935, refactor the logics for backup/restore persistent log, so as to remove the contest to gzip writer (#5956, @Lyndon-Li)\n* Switch the base image to distroless/base-nossl-debian11 to reduce the CVE triage efforts (#5939, @ywk253100)\n* Wait for additional items to be ready before restoring current item (#5933, @sseago)\n* Add configurable server setting for default timeouts (#5926, @eemcmullan)\n* Add warning/error result to cmd `velero backup describe` (#5916, @allenxu404)\n* Fix Dependabot alerts. Use 1.18 and 1.19 golang instead of patch image in dockerfile. Add release-1.10 and release-1.9 in Trivy daily scan. (#5911, @blackpiglet)\n* Update client-go to v0.25.6 (#5907, @kaovilai)\n* Limit the concurrent number for backup's VolumeSnapshot operation. (#5900, @blackpiglet)\n* Fix goreleaser issue for resolving tags and updated it's version. (#5899, @anshulahuja98)\n* This is to fix issue 5881, enhance the PVB tracker in two modes, Track and Taken (#5894, @Lyndon-Li)\n* Add labels for velero installed namespace to support PSA. (#5873, @blackpiglet)\n* Add restored resource list in the restore describe command (#5867, @ywk253100)\n* Add a json output to cmd velero backup describe (#5865, @allenxu404)\n* Make restore controller adopting the controller-runtime framework. (#5864, @blackpiglet)\n* Replace k8s.io/apimachinery/pkg/util/clock with k8s.io/utils/clock (#5859, @hezhizhen)\n* Restore finalizer and managedFields of metadata during the restoration (#5853, @ywk253100)\n* BIAv2 async operations controller work (#5849, @sseago)\n* Add secret restore item action to handle service account token secret (#5843, @ywk253100)\n* Add new resource filters can separate cluster and namespace scope resources. (#5838, @blackpiglet)\n* Correct PVB/PVR Failed Phase patching during startup (#5828, @kaovilai)\n* bump up golang net to fix CVE-2022-41721 (#5812, @Lyndon-Li)\n* Update CRD descriptions for SnapshotVolumes and restorePVs (#5807, @shubham-pampattiwar)\n* Add mapped selected-node existence check (#5806, @blackpiglet)\n* Add option \"--service-account-name\" to install cmd (#5802, @reasonerjt)\n* Enable staticcheck linter. (#5788, @blackpiglet)\n* Set Kopia IgnoreUnknownTypes in ErrorHandlingPolicy to True for ignoring backup unknown file type (#5786, @qiuming-best)\n* Bump up Restic version to 0.15.0 (#5784, @qiuming-best)\n* Add File system backup related metrics to Grafana dashboard\n - Add metrics backup_warning_total for record of total warnings\n - Add metrics backup_last_status for record of last status of the backup (#5779, @allenxu404)\n* Design for Handling backup of volumes by resources filters (#5773, @qiuming-best)\n* Add PR container build action, which will not push image. Add GOARM parameter. (#5771, @blackpiglet)\n* Fix issue 5458, track pod volume backup until the CR is submitted in case it is skipped half way (#5769, @Lyndon-Li)\n* Fix issue 5226, invalidate the related backup repositories whenever the backup storage info change in BSL (#5768, @Lyndon-Li)\n* Add Restic builder in Dockerfile, and keep the used built Golang image version in accordance with upstream Restic. (#5764, @blackpiglet)\n* Fix issue 5043, after the restore pod is scheduled, check if the node-agent pod is running in the same node. (#5760, @Lyndon-Li)\n* Remove restore controller's redundant client. (#5759, @blackpiglet)\n* Define itemoperations.json format and update DownloadRequest API (#5752, @sseago)\n* Add Trivy nightly scan. (#5740, @jxun)\n* Fix issue 5696, check if the repo is still openable before running the prune and forget operation, if not, try to reconnect the repo (#5715, @Lyndon-Li)\n* Fix error with Restic backup empty volumes (#5713, @qiuming-best)\n* new backup and restore phases to support async plugin operations:\n - WaitingForPluginOperations\n - WaitingForPluginOperationsPartiallyFailed (#5710, @sseago)\n* Prevent nil panic on exec restore hooks (#5675, @dymurray)\n* Fix CVEs scanned by trivy (#5653, @qiuming-best)\n* Publish backupresults json to enhance error info during backups. (#5576, @anshulahuja98)\n* RestoreItemAction v2 API implementation (#5569, @sseago)\n* add new RestoreItemAction of \"velero.io/change-image-name\" to handle the issue mentioned at #5519 (#5540, @wenterjoy)\n* BackupItemAction v2 API implementation (#5442, @sseago)\n* Proposal to separate resource filter into cluster scope and namespace scope (#5333, @blackpiglet)\n" }, { "path": "changelogs/CHANGELOG-1.12.md", "content": "## v1.12\n### 2023-08-18\n\n### Download\nhttps://github.com/vmware-tanzu/velero/releases/tag/v1.12.0\n\n### Container Image\n`velero/velero:v1.12.0`\n\n### Documentation\nhttps://velero.io/docs/v1.12/\n\n### Upgrading\nhttps://velero.io/docs/v1.12/upgrade-to-1.12/\n\n### Highlights\n\n#### CSI Snapshot Data Movement\nCSI Snapshot Data Movement refers to back up CSI snapshot data from the volatile and limited production environment into durable, heterogeneous, and scalable backup storage in a consistent manner; and restore the data to volumes in the original or alternative environment.\n\nCSI Snapshot Data Movement is useful in below scenarios:\n\n* For on-premises users, the storage usually doesn't support durable snapshots, so it is impossible/less efficient/cost ineffective to keep volume snapshots by the storage This feature helps to move the snapshot data to a storage with lower cost and larger scale for long time preservation.\n* For public cloud users, this feature helps users to fulfill the multiple cloud strategy. It allows users to back up volume snapshots from one cloud provider and preserve or restore the data to another cloud provider. Then users will be free to flow their business data across cloud providers based on Velero backup and restore\n\nCSI Snapshot Data Movement is built according to the Volume Snapshot Data Movement design ([Volume Snapshot Data Movement](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/unified-repo-and-kopia-integration/unified-repo-and-kopia-integration.md)). More details can be found in the design.\n\n#### Resource Modifiers\nIn many use cases, customers often need to substitute specific values in Kubernetes resources during the restoration process like changing the namespace, changing the storage class, etc. \n\nTo address this need, Resource Modifiers (also known as JSON Substitutions) offer a generic solution in the restore workflow. It allows the user to define filters for specific resources and then specify a JSON patch (operator, path, value) to apply to the resource. This feature simplifies the process of making substitutions without requiring the implementation of a new RestoreItemAction plugin. More details can be found in Volume Snapshot Resource Modifiers design ([Resource Modifiers](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/json-substitution-action-design.md)).\n\n#### Multiple VolumeSnapshotClasses\nPrior to version 1.12, the Velero CSI plugin would choose the VolumeSnapshotClass in the cluster based on matching driver names and the presence of the \"velero.io/csi-volumesnapshot-class\" label. However, this approach proved inadequate for many user scenarios.\n\nWith the introduction of version 1.12, Velero now offers support for multiple VolumeSnapshotClasses in the CSI Plugin, enabling users to select a specific class for a particular backup. More details can be found in Multiple VolumeSnapshotClasses design ([Multiple VolumeSnapshotClasses](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/multiple-csi-volumesnapshotclass-support.md)).\n\n#### Restore Finalizer\nBefore v1.12, the restore controller would only delete restore resources but wouldn’t delete restore data from the backup storage location when the command `velero restore delete` was executed. The only chance Velero deletes restores data from the backup storage location is when the associated backup is deleted.\n\nIn this version, Velero introduces a finalizer that ensures the cleanup of all associated data for restores when running the command `velero restore delete`.\n\n#### Runtime and dependencies\nTo fix CVEs and keep pace with Golang, Velero made changes as follows:\n* Bump Golang runtime to v1.20.7.\n* Bump several dependent libraries to new versions.\n* Bump Kopia to v0.13.\n\n\n### Breaking changes\n* Prior to v1.12, the parameter `uploader-type` for Velero installation had a default value of \"restic\". However, starting from this version, the default value has been changed to \"kopia\". This means that Velero will now use Kopia as the default path for file system backup.\n* The ways of setting CSI snapshot time have changed in v1.12. First, the sync waiting time for creating a snapshot handle in the CSI plugin is changed from the fixed 10 minutes into backup.Spec.CSISnapshotTimeout. The second, the async waiting time for VolumeSnapshot and VolumeSnapshotContent's status turning into `ReadyToUse` in operation uses the operation's timeout. The default value is 4 hours.\n* As from [Velero helm chart v4.0.0](https://github.com/vmware-tanzu/helm-charts/releases/tag/velero-4.0.0), it supports multiple BSL and VSL, and the BSL and VSL have changed from the map into a slice, and[ this breaking change](https://github.com/vmware-tanzu/helm-charts/pull/413) is not backward compatible. So it would be best to change the BSL and VSL configuration into slices before the Upgrade.\n\n\n### Limitations/Known issues\n* The Azure plugin supports Azure AD Workload identity way, but it only works for Velero native snapshots. It cannot support filesystem backup and snapshot data mover scenarios. \n\n\n### All Changes\n* Fixes #6498. Get resource client again after restore actions in case resource's gv is changed. This is an improvement of pr #6499, to support group changes. A group change usually happens in a restore plugin which is used for resource conversion: convert a resource from a not supported gv to a supported gv (#6634, @27149chen)\n* Add API support for volMode block, only error for now. (#6608, @shawn-hurley)\n* Fix how the AWS credentials are obtained from configuration (#6598, @aws_creds)\n* Add performance E2E test (#6569, @qiuming-best)\n* Non default s3 credential profiles work on Unified Repository Provider (kopia) (#6558, @kaovilai)\n* Fix issue #6571, fix the problem for restore item operation to set the errors correctly so that they can be recorded by Velero restore and then reflect the correct status for Velero restore. (#6594, @Lyndon-Li)\n* Fix issue 6575, flush the repo after delete the snapshot, otherwise, the changes(deleting repo snapshot) cannot be committed to the repo. (#6587, @Lyndon-Li)\n* Delete moved snapshots when the backup is deleted (#6547, @reasonerjt)\n* check if restore crd exist before operating restores (#6544, @allenxu404)\n* Remove PVC's selector in backup's PVC action. (#6481, @blackpiglet)\n* Delete the expired deletebackuprequests that are stuck in \"InProgress\" (#6476, @reasonerjt)\n* Fix issue #6534, reset PVB CR's StorageLocation to the latest one during backup sync as same as the backup CR. Also fix similar problem with DataUploadResult for data mover restore. (#6533, @Lyndon-Li)\n* Fix issue #6519. Restrict the client manager of node-agent server to include only Velero resources from the server's namespace, otherwise, the controllers will try to reconcile CRs from all the installed Velero namespaces. (#6523, @Lyndon-Li)\n* Track the skipped PVC and print the summary in backup log (#6496, @reasonerjt)\n* Add restore finalizer to clean up external resources (#6479, @allenxu404)\n* fix: Typos and add more spell checking rules to CI (#6415, @mateusoliveira43)\n* Add missing CompletionTimestamp and metrics when restore moved into terminal phase in restoreOperationsReconciler (#6397, @Nutrymaco)\n* Add support for resource Modifications in the restore flow. Also known as JSON Substitutions. (#6452, @anshulahuja98)\n* Remove dependency of the legacy client code from pkg/cmd directory part 2 (#6497, @blackpiglet)\n* Add data upload and download metrics (#6493, @allenxu404)\n* Fix issue 6490, If a backup/restore has multiple async operations and one operation fails while others are still in-progress, when all the operations finish, the backup/restore will be set as Completed falsely (#6491, @Lyndon-Li)\n* Velero Plugins no longer need kopia indirect dependency in their go.mod (#6484, @kaovilai)\n* Remove dependency of the legacy client code from pkg/cmd directory (#6469, @blackpiglet)\n* Add support for OpenStack CSI drivers topology keys (#6464, @openstack-csi-topology-keys)\n* Add exit code log and possible memory shortage warning log for Restic command failure. (#6459, @blackpiglet)\n* Modify DownloadRequest controller logic (#6433, @blackpiglet)\n* Add data download controller for data mover (#6436, @qiuming-best)\n* Fix hook filter display issue for backup describer (#6434, @allenxu404)\n* Retrieve DataUpload into backup result ConfigMap during volume snapshot restore. (#6410, @blackpiglet)\n* Design to add support for Multiple VolumeSnapshotClasses in CSI Plugin. (#5774, @anshulahuja98)\n* Clarify the deletion frequency for gc controller (#6414, @allenxu404)\n* Add unit tests for pkg/archive (#6396, @allenxu404)\n* Add UT for pkg/discovery (#6394, @qiuming-best)\n* Add UT for pkg/util (#6368, @Lyndon-Li)\n* Add the code for data mover restore expose (#6357, @Lyndon-Li)\n* Restore Endpoints before Services (#6315, @ywk253100)\n* Add warning message for volume snapshotter in data mover case. (#6377, @blackpiglet)\n* Add unit test for pkg/uploader (#6374, @qiuming-best)\n* Change kopia as the default path of PVB (#6370, @Lyndon-Li)\n* Do not persist VolumeSnapshot and VolumeSnapshotContent for snapshot DataMover case. (#6366, @blackpiglet)\n* Add data mover related options in CLI (#6365, @ywk253100)\n* Add dataupload controller (#6337, @qiuming-best)\n* Add UT cases for pkg/podvolume (#6336, @Lyndon-Li)\n* Remove Wait VolumeSnapshot to ReadyToUse logic. (#6327, @blackpiglet)\n* Enhance the code because of #6297, the return value of GetBucketRegion is not recorded, as a result, when it fails, we have no way to get the cause (#6326, @Lyndon-Li)\n* Skip updating status when CRDs are restored (#6325, @reasonerjt)\n* Include namespaces needed by namespaced-scope resources in backup. (#6320, @blackpiglet)\n* Update metrics when backup failed with validation error (#6318, @ywk253100)\n* Add the code for data mover backup expose (#6308, @Lyndon-Li)\n* Fix a PVR issue for generic data path -- the namespace remap was not honored, and enhance the code for better error handling (#6303, @Lyndon-Li)\n* Add default values for defaultItemOperationTimeout and itemOperationSyncFrequency in velero CLI (#6298, @shubham-pampattiwar)\n* Add UT cases for pkg/repository (#6296, @Lyndon-Li)\n* Fix issue #5875. Since Kopia has supported IAM, Velero should not require static credentials all the time (#6283, @Lyndon-Li)\n* Fixed a bug where status.progress is not getting updated for backups. (#6276, @kkothule)\n* Add code change for async generic data path that is used by both PVB/PVR and data mover (#6226, @Lyndon-Li)\n* Add data mover CRD under v2alpha1, include DataUpload CRD and DataDownload CRD (#6176, @Lyndon-Li)\n* Remove any dataSource or dataSourceRef fields from PVCs in PVC BIA for cases of\nprior PVC restores with CSI (#6111, @eemcmullan)\n* Add the design for Volume Snapshot Data Movement (#5968, @Lyndon-Li)\n* Fix issue #5123, Kopia repository supports self-cert CA for S3 compatible storage. (#6268, @Lyndon-Li)\n* Bump up Kopia to v0.13 (#6248, @Lyndon-Li)\n* log volumes to backup to help debug why `IsPodRunning` is called. (#6232, @kaovilai)\n* Enable errcheck linter and resolve found issues (#6208, @blackpiglet)\n* Enable more linters, and remove mal-functioned milestoned issue action. (#6194, @blackpiglet)\n* Enable stylecheck linter and resolve found issues. (#6185, @blackpiglet)\n* Fix issue #6182. If pod is not running, don't treat it as an error, let it go and leave a warning. (#6184, @Lyndon-Li)\n* Enable staticcheck and resolve found issues (#6183, @blackpiglet)\n* Enable linter revive and resolve found errors: part 2 (#6177, @blackpiglet)\n* Enable linter revive and resolve found errors: part 1 (#6173, @blackpiglet)\n* Fix usestdlibvars and whitespace linters issues. (#6162, @blackpiglet)\n* Update Golang to v1.20 for main. (#6158, @blackpiglet)\n* Make GetPluginConfig accessible from other packages. (#6151, @tkaovila)\n* Ignore not found error during patching managedFields (#6136, @ywk253100)\n* Fix the goreleaser issues and add a new goreleaser action (#6109, @blackpiglet)\n" }, { "path": "changelogs/CHANGELOG-1.13.md", "content": "## v1.13\n### 2024-01-10\n\n### Download\nhttps://github.com/vmware-tanzu/velero/releases/tag/v1.13.0\n\n### Container Image\n`velero/velero:v1.13.0`\n\n### Documentation\nhttps://velero.io/docs/v1.13/\n\n### Upgrading\nhttps://velero.io/docs/v1.13/upgrade-to-1.13/\n\n### Highlights\n\n#### Resource Modifier Enhancement\nVelero introduced the Resource Modifiers in v1.12.0. This feature allows users to specify a ConfigMap with a set of rules to modify the resources during restoration. However, only the JSON Patch is supported when creating the rules, and JSON Patch has some limitations, which cannot cover all use cases. In v1.13.0, Velero adds new support for JSON Merge Patch and Strategic Merge Patch, which provide more power and flexibility and allow users to use the same ConfigMap to apply patches on the resources. More design details can be found in [Support JSON Merge Patch and Strategic Merge Patch in Resource Modifiers](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/merge-patch-and-strategic-in-resource-modifier.md) design. For instructions on how to use the feature, please refer to the [Resource Modifiers](https://velero.io/docs/v1.13/restore-resource-modifiers/) doc.\n\n#### Node-Agent Concurrency\nVelero data movement activities from fs-backups and CSI snapshot data movements run in Velero node-agent, so may be hosted by every node in the cluster and consume resources (i.e. CPU, memory, network bandwidth) from there. With v1.13, users are allowed to configure how many data movement activities (a.k.a, loads) run in each node globally or by node, so that users can better leverage the performance of Velero data movement activities and the resource consumption in the cluster. For more information, check the [Node-Agent Concurrency](https://velero.io/docs/v1.13/node-agent-concurrency/) document. \n\n#### Parallel Files Upload Options\nVelero now supports configurable options for parallel files upload when using Kopia uploader to do fs-backups or CSI snapshot data movements which makes speed up backup possible.\nFor more information, please check [Here](https://velero.io/docs/v1.13/backup-reference/#parallel-files-upload).\n\n#### Write Sparse Files Options\nIf using fs-restore or CSI snapshot data movements, it’s supported to write sparse files during restore. For more information, please check [Here](https://velero.io/docs/v1.13/restore-reference/#write-sparse-files).\n\n#### Backup Describe\nIn v1.13, the Backup Volume section is added to the velero backup describe command output. The backup Volumes section describes information for all the volumes included in the backup of various backup types, i.e. native snapshot, fs-backup, CSI snapshot, and CSI snapshot data movement. Particularly, the velero backup description now supports showing the information of CSI snapshot data movements, which is not supported in v1.12.\n\nAdditionally, backup describe command will not check EnableCSI feature gate from client side, so if a backup has volumes with CSI snapshot or CSI snapshot data movement, backup describe command always shows the corresponding information in its output.\n\n#### Backup's new VolumeInfo metadata\nCreate a new metadata file in the backup repository's backup name sub-directory to store the backup-including PVC and PV information. The information includes the backing-up method of the PVC and PV data, snapshot information, and status. The VolumeInfo metadata file determines how the PV resource should be restored. The Velero downstream software can also use this metadata file to get a summary of the backup's volume data information.\n\n#### Enhancement for CSI Snapshot Data Movements when Velero Pod Restart\nWhen performing backup and restore operations, enhancements have been implemented for Velero server pods or node agents to ensure that the current backup or restore process is not stuck or interrupted after restart due to certain exceptional circumstances.\n\n#### New status fields added to show hook execution details\nHook execution status is now included in the backup/restore CR status and displayed in the backup/restore describe command output. Specifically, it will show the number of hooks which attempted to execute under the HooksAttempted field and the number of hooks which failed to execute under the HooksFailed field. \n\n#### AWS SDK Bump Up\nBump up AWS SDK for Go to version 2, which offers significant performance improvements in CPU and memory utilization over version 1.\n\n#### Azure AD/Workload Identity Support\nAzure AD/Workload Identity is the recommended approach to do the authentication with Azure services/AKS, Velero has introduced support for Azure AD/Workload Identity on the Velero Azure plugin side in previous releases, and in v1.13.0 Velero adds new support for Kopia operations(file system backup/data mover/etc.) with Azure AD/Workload Identity.\n\n#### Runtime and dependencies\nTo fix CVEs and keep pace with Golang, Velero made changes as follows:\n* Bump Golang runtime to v1.21.6.\n* Bump several dependent libraries to new versions.\n* Bump Kopia to v0.15.0.\n\n\n### Breaking changes\n* Backup describe command: due to the backup describe output enhancement, some existing information (i.e. the output for native snapshot, CSI snapshot, and fs-backup) has been moved to the Backup Volumes section with some format changes.\n* API type changes: changes the field [DataMoverConfig](https://github.com/vmware-tanzu/velero/blob/v1.13.0/pkg/apis/velero/v2alpha1/data_upload_types.go#L54) in DataUploadSpec from `*map[string][string]`` to `map[string]string`\n* Velero install command: due to the issue [#7264](https://github.com/vmware-tanzu/velero/issues/7264), v1.13.0 introduces a break change that make the informer cache enabled by default to keep the actual behavior consistent with the helper message(the informer cache is disabled by default before the change).\n\n\n### Limitations/Known issues\n* The backup's VolumeInfo metadata doesn't have the information updated in the async operations. This function could be supported in v1.14 release. \n\n### Note\n* Velero introduces the informer cache which is enabled by default. The informer cache improves the restore performance but may cause higher memory consumption. Increase the memory limit of the Velero pod or disable the informer cache by specifying the `--disable-informer-cache` option when installing Velero if you get the OOM error.\n\n### Deprecation announcement\n* The generated k8s clients, informers, and listers are deprecated in the Velero v1.13 release. They are put in the Velero repository's pkg/generated directory. According to the n+2 supporting policy, the deprecated are kept for two more releases. The pkg/generated directory should be deleted in the v1.15 release.\n* After the backup VolumeInfo metadata file is added to the backup, Velero decides how to restore the PV resource according to the VolumeInfo content. To support the backup generated by the older version of Velero, the old logic is also kept. The support for the backup without the VolumeInfo metadata file will be kept for two releases. The support logic will be deleted in the v1.15 release.\n\n### All Changes\n * Make \"disable-informer-cache\" option false(enabled) by default to keep it consistent with the help message (#7294, @ywk253100)\n * Fix issue #6928, remove snapshot deletion timeout for PVB (#7282, @Lyndon-Li)\n * Do not set \"targetNamespace\" to namespace items (#7274, @reasonerjt)\n * Fix issue #7244. By the end of the upload, check the outstanding incomplete snapshots and delete them by calling ApplyRetentionPolicy (#7245, @Lyndon-Li)\n * Adjust the newline output of resource list in restore describer (#7238, @allenxu404)\n * Remove the redundant newline in backup describe output (#7229, @allenxu404)\n * Fix issue #7189, data mover generic restore - don't assume the first volume as the restore volume (#7201, @Lyndon-Li)\n * Update CSIVolumeSnapshotsCompleted in backup's status and the metric\nduring backup finalize stage according to async operations content. (#7184, @blackpiglet)\n * Refactor DownloadRequest Stream function (#7175, @blackpiglet)\n * Add `--skip-immediately` flag to schedule commands; `--schedule-skip-immediately` server and install (#7169, @kaovilai)\n * Add node-agent concurrency doc and change the config name from dataPathConcurrency to loadCocurrency (#7161, @Lyndon-Li)\n * Enhance hooks tracker by adding a returned error to record function (#7153, @allenxu404)\n * Track the skipped PV when SnapshotVolumes set as false (#7152, @reasonerjt)\n * Add more linters part 2. (#7151, @blackpiglet)\n * Fix issue #7135, check pod status before checking node-agent pod status (#7150, @Lyndon-Li)\n * Treat namespace as a regular restorable item (#7143, @reasonerjt)\n * Allow sparse option for Kopia & Restic restore (#7141, @qiuming-best)\n * Use VolumeInfo to help restore the PV. (#7138, @blackpiglet)\n * Node agent restart enhancement (#7130, @qiuming-best)\n * Fix issue #6695, add describe for data mover backups (#7125, @Lyndon-Li)\n * Add hooks status to backup/restore CR (#7117, @allenxu404)\n * Include plugin name in the error message by operations (#7115, @reasonerjt)\n * Fix issue #7068, due to a behavior of CSI external snapshotter, manipulations of VS and VSC may not be handled in the same order inside external snapshotter as the API is called. So add a protection finalizer to ensure the order (#7102, @Lyndon-Li)\n * Generate VolumeInfo for backup. (#7100, @blackpiglet)\n * Fix issue #7094, fallback to full backup if previous snapshot is not found (#7096, @Lyndon-Li)\n * Fix issue #7068, due to an behavior of CSI external snapshotter, manipulations of VS and VSC may not be handled in the same order inside external snapshotter as the API is called. So add a protection finalizer to ensure the order (#7095, @Lyndon-Li)\n * Skip syncing the backup which doesn't contain backup metadata (#7081, @ywk253100)\n * Fix issue #6693, partially fail restore if CSI snapshot is involved but CSI feature is not ready, i.e., CSI feature gate is not enabled or CSI plugin is not installed. (#7077, @Lyndon-Li)\n * Truncate the credential file to avoid the change of secret content messing it up (#7072, @ywk253100)\n * Add VolumeInfo metadata structures. (#7070, @blackpiglet)\n * improve discoveryHelper.Refresh() in restore (#7069, @27149chen)\n * Add DataUpload Result and CSI VolumeSnapshot check for restore PV. (#7061, @blackpiglet)\n * Add the implementation for design #6950, configurable data path concurrency (#7059, @Lyndon-Li)\n * Make data mover fail early (#7052, @qiuming-best)\n * Remove dependency of generated client part 3. (#7051, @blackpiglet)\n * Update Backup.Status.CSIVolumeSnapshotsCompleted during finalize (#7046, @kaovilai)\n * Remove the Velero generated client. (#7041, @blackpiglet)\n * Fix issue #7027, data mover backup exposer should not assume the first volume as the backup volume in backup pod (#7038, @Lyndon-Li)\n * Read information from the credential specified by BSL (#7034, @ywk253100)\n * Fix #6857. Added check for matching Owner References when synchronizing backups, removing references that are not found/have mismatched uid. (#7032, @deefdragon)\n * Add description markers for dataupload and datadownload CRDs (#7028, @shubham-pampattiwar)\n * Add HealthCheckNodePort deletion logic for Service restore. (#7026, @blackpiglet)\n * Fix inconsistent behavior of Backup and Restore hook execution (#7022, @allenxu404)\n * Fix #6964. Don't use csiSnapshotTimeout (10 min) for waiting snapshot to readyToUse for data mover, so as to make the behavior complied with CSI snapshot backup (#7011, @Lyndon-Li)\n * restore: Use warning when Create IsAlreadyExist and Get error (#7004, @kaovilai)\n * Bump kopia to 0.15.0 (#7001, @Lyndon-Li)\n * Make Kopia file parallelism configurable (#7000, @qiuming-best)\n * Fix unified repository (kopia) s3 credentials profile selection (#6995, @kaovilai)\n * Fix #6988, always get region from BSL if it is not empty (#6990, @Lyndon-Li)\n * Limit PVC block mode logic to non-Windows platform. (#6989, @blackpiglet)\n * It is a valid case that the Status.RestoreSize field in VolumeSnapshot is not set, if so, get the volume size from the source PVC to create the backup PVC (#6976, @Lyndon-Li)\n * Check whether the action is a CSI action and whether CSI feature is enabled, before executing the action. (#6968, @blackpiglet)\n * Add the PV backup information design document. (#6962, @blackpiglet)\n * Change controller-runtime List option from MatchingFields to ListOptions (#6958, @blackpiglet)\n * Add the design for node-agent concurrency (#6950, @Lyndon-Li)\n * Import auth provider plugins (#6947, @0x113)\n * Fix #6668, add a limitation for file system restore parallelism with other types of restores (CSI snapshot restore, CSI snapshot movement restore) (#6946, @Lyndon-Li)\n * Add MSI Support for Azure plugin. (#6938, @yanggangtony)\n * Partially fix #6734, guide Kubernetes' scheduler to spread backup pods evenly across nodes as much as possible, so that data mover backup could achieve better parallelism (#6926, @Lyndon-Li)\n * Bump up aws sdk to aws-sdk-go-v2 (#6923, @reasonerjt)\n * Optional check if targeted container is ready before executing a hook (#6918, @Ripolin)\n * Support JSON Merge Patch and Strategic Merge Patch in Resource Modifiers (#6917, @27149chen)\n * Fix issue 6913: Velero Built-in Datamover: Backup stucks in phase WaitingForPluginOperations when Node Agent pod gets restarted (#6914, @shubham-pampattiwar)\n * Set ParallelUploadAboveSize as MaxInt64 and flush repo after setting up policy so that policy is retrieved correctly by TreeForSource (#6885, @Lyndon-Li)\n * Replace the base image with paketobuildpacks image (#6883, @ywk253100)\n * Fix issue #6859, move plugin depending podvolume functions to util pkg, so as to remove the dependencies to unnecessary repository packages like kopia, azure, etc. (#6875, @Lyndon-Li)\n * Fix #6861. Only Restic path requires repoIdentifier, so for non-restic path, set the repoIdentifier fields as empty in PVB and PVR and also remove the RepoIdentifier column in the get output of PVBs and PVRs (#6872, @Lyndon-Li)\n * Add volume types filter in resource policies (#6863, @qiuming-best)\n * change the metrics backup_attempt_total default value to 1. (#6838, @yanggangtony)\n * Bump kopia to v0.14 (#6833, @Lyndon-Li)\n * Retry failed create when using generateName (#6830, @sseago)\n * Fix issue #6786, always delete VSC regardless of the deletion policy (#6827, @Lyndon-Li)\n * Proposal to support JSON Merge Patch and Strategic Merge Patch in Resource Modifiers (#6797, @27149chen)\n * Fix the node-agent missing metrics-address defines. (#6784, @yanggangtony)\n * Fix default BSL setting not work (#6771, @qiuming-best)\n * Update restore controller logic for restore deletion (#6770, @ywk253100)\n * Fix #6752: add namespace exclude check. (#6760, @blackpiglet)\n * Fix issue #6753, remove the check for read-only BSL in restore async operation controller since Velero cannot fully support read-only mode BSL in restore at present (#6757, @Lyndon-Li)\n * Fix issue #6647, add the --default-snapshot-move-data parameter to Velero install, so that users don't need to specify --snapshot-move-data per backup when they want to move snapshot data for all backups (#6751, @Lyndon-Li)\n * Use old(origin) namespace in resource modifier conditions in case namespace may change during restore (#6724, @27149chen)\n * Perf improvements for existing resource restore (#6723, @sseago)\n * Remove schedule-related metrics on schedule delete (#6715, @nilesh-akhade)\n * Kubernetes 1.27 new job label batch.kubernetes.io/controller-uid are deleted during restore per https://github.com/kubernetes/kubernetes/pull/114930 (#6712, @kaovilai)\n * This pr made some improvements in Resource Modifiers: 1. add label selector 2. change the field name from groupKind to groupResource (#6704, @27149chen)\n * Make Kopia support Azure AD (#6686, @ywk253100)\n * Add support for block volumes with Kopia (#6680, @dzaninovic)\n * Delete PartiallyFailed orphaned backups as well as Completed ones (#6649, @sseago)\n * Add CSI snapshot data movement doc (#6637, @Lyndon-Li)\n * Fixes #6636, skip subresource in resource discovery (#6635, @27149chen)\n * Add `orLabelSelectors` for backup, restore commands (#6475, @nilesh-akhade)\n * fix run preHook and postHook on completed pods (#5211, @cleverhu)" }, { "path": "changelogs/CHANGELOG-1.14.md", "content": "## v1.14\n\n### Download\nhttps://github.com/vmware-tanzu/velero/releases/tag/v1.14.0\n\n### Container Image\n`velero/velero:v1.14.0`\n\n### Documentation\nhttps://velero.io/docs/v1.14/\n\n### Upgrading\nhttps://velero.io/docs/v1.14/upgrade-to-1.14/\n\n### Highlights\n\n#### The maintenance work for kopia/restic backup repositories is run in jobs\nSince velero started using kopia as the approach for filesystem-level backup/restore, we've noticed an issue when velero connects to the kopia backup repositories and performs maintenance, it sometimes consumes excessive memory that can cause the velero pod to get OOM Killed. To mitigate this issue, the maintenance work will be moved out of velero pod to a separate kubernetes job, and the user will be able to specify the resource request in \"velero install\".\n#### Volume Policies are extended to support more actions to handle volumes\nIn an earlier release, a flexible volume policy was introduced to skip certain volumes from a backup. In v1.14 we've made enhancement to this policy to allow the user to set how the volumes should be backed up. The user will be able to set \"fs-backup\" or \"snapshot\" as value of “action\" in the policy and velero will backup the volumes accordingly. This enhancement allows the user to achieve a fine-grained control like \"opt-in/out\" without having to update the target workload. For more details please refer to https://velero.io/docs/v1.14/resource-filtering/#supported-volumepolicy-actions\n#### Node Selection for Data Movement Backup\nIn velero the data movement flow relies on datamover pods, and these pods may take substantial resources and keep running for a long time. In v1.14, the user will be able to create a configmap to define the eligible nodes on which the datamover pods are launched. For more details refer to https://velero.io/docs/v1.14/data-movement-backup-node-selection/ \n#### VolumeInfo metadata for restored volumes\nIn v1.13, we introduced volumeinfo metadata for backup to help velero CLI and downstream adopter understand how velero handles each volume during backup. In v1.14, similar metadata will be persisted for each restore. velero CLI is also updated to bring more info in the output of \"velero restore describe\".\n#### \"Finalizing\" phase is introduced to restores\nThe \"Finalizing\" phase is added to the state transition flow to restore, which helps us fix several issues: The labels added to PVs will be restored after the data in the PV is restored via volumesnapshotter. The post restore hook will be executed after datamovement is finished.\n#### Certificate-based authentication support for Azure\nBesides the service principal with secret(password)-based authentication, Velero introduces the new support for service principal with certificate-based authentication in v1.14.0. This approach enables you to adopt a phishing resistant authentication by using conditional access policies, which better protects Azure resources and is the recommended way by Azure.\n\n### Runtime and dependencies\n* Golang runtime: v1.22.2\n* kopia: v0.17.0\n\n### Limitations/Known issues\n* For the external BackupItemAction plugins that take snapshots for PVs, such as vsphere plugin. If the plugin checks the value of the field \"snapshotVolumes\" in the backup spec as a criteria for snapshot, the settings in the volume policy will not take effect. For example, if the \"snapshotVolumes\" is set to False in the backup spec, but a volume meets the condition in the volume policy for \"snapshot\" action, because the plugin will not check the settings in the volume policy, the plugin will not take snapshot for the volume. For more details please refer to #7818\n\n### Breaking changes\n* CSI plugin has been merged into velero repo in v1.14 release. It will be installed by default as an internal plugin, and should not be installed via \"–plugins \" parameter in \"velero install\" command.\n* The default resource requests and limitations for node agent are removed in v1.14, to make the node agent pods have the QoS class of \"BestEffort\", more details please refer to #7391\n* There's a change in namespace filtering behavior during backup: In v1.14, when the includedNamespaces/excludedNamespaces fields are not set and the labelSelector/OrLabelSelectors are set in the backup spec, the backup will only include the namespaces which contain the resources that match the label selectors, while in previous releases all namespaces will be included in the backup with such settings. More details refer to #7105\n* Patching the PV in the \"Finalizing\" state may cause the restore to be in \"PartiallyFailed\" state when the PV is blocked in \"Pending\" state, while in the previous release the restore may end up being in \"Complete\" state. For more details refer to #7866\n\n### All Changes\n* Fix backup log to show error string, not index (#7805, @piny940)\n* Modify the volume helper logic. (#7794, @blackpiglet)\n* Add documentation for extension of volume policy feature (#7779, @shubham-pampattiwar)\n* Surface errors when waiting for backupRepository and timeout occurs (#7762, @kaovilai)\n* Add existingResourcePolicy restore CR validation to controller (#7757, @kaovilai)\n* Fix condition matching in resource modifier when there are multiple rules (#7715, @27149chen)\n* Bump up the version of KinD and k8s in github actions (#7702, @reasonerjt)\n* Implementation for Extending VolumePolicies to support more actions (#7664, @shubham-pampattiwar)\n* Migrate from `github.com/Azure/azure-storage-blob-go` to `github.com/Azure/azure-sdk-for-go/sdk/storage/azblob` (#7598, @mmorel-35)\n* When Included/ExcludedNamespaces are omitted, and LabelSelector or OrLabelSelector is used, namespaces without selected items are excluded from backup. (#7697, @blackpiglet)\n* Display CSI snapshot restores in restore describe (#7687, @reasonerjt)\n* Use specific credential rather than the credential chain for Azure (#7680, @ywk253100)\n* Modify hook docs for clarity on displaying hook execution results (#7679, @allenxu404)\n* Wait for results of restore exec hook executions in Finalizing phase instead of InProgress phase (#7619, @allenxu404)\n* migrating to `sdk/resourcemanager/**/arm**` from `services/**/mgmt/**` (#7596, @mmorel-35)\n* Bump up to go1.22 (#7666, @reasonerjt)\n* Fix issue #7648. Adjust the exposing logic to avoid exposing failure and snapshot leak when expose fails (#7662, @Lyndon-Li)\n* Track and persist restore volume info (#7630, @reasonerjt)\n* Check the existence of the namespaces provided in the \"--include-namespaces\" option (#7569, @ywk253100)\n* Add the finalization phase to the restore workflow (#7377, @allenxu404)\n* Upgrade the version of go plugin related libs/tools (#7373, @ywk253100)\n* Check resource Group Version and Kind is available in cluster before attempting restore to prevent being stuck. (#7322, @kaovilai)\n* Merge CSI plugin code into Velero. (#7609, @blackpiglet)\n* Fix issue #7391, remove the default constraint for node-agent pods (#7488, @Lyndon-Li)\n* Fix DataDownload fails during restore for empty PVC workload (#7521, @qiuming-best)\n* Add repository maintenance job (#7451, @qiuming-best)\n* Check whether the VolumeSnapshot's source PVC is nil before using it.\n Skip populate VolumeInfo for data-moved PV when CSI is not enabled. (#7515, @blackpiglet)\n* Fix issue #7308, change the data path requeue time to 5 second for data mover backup/restore, PVB and PVR. (#7458, @Lyndon-Li)\n* Patch newly dynamically provisioned PV with volume info to restore custom setting of PV (#7504, @allenxu404)\n* Adjust the logic for the backup_last_status metrics to stop incorrectly incrementing over time (#7445, @allenxu404)\n* dependabot: support github-actions updates (#7594, @mmorel-35)\n* Include the design for adding the finalization phase to the restore workflow (#7317, @allenxu404)\n* Fix issue #7211. Enable advanced feature capability and add support to concatenate objects for unified repo. (#7452, @Lyndon-Li)\n* Add design to introduce restore volume info (#7610, @reasonerjt)\n* Increase the k8s client QPS/burst to avoid throttling request errors (#7311, @ywk253100)\n* Support update the backup VolumeInfos by the Async ops result. (#7554, @blackpiglet)\n* FS backup create PodVolumeBackup when the backup excluded PVC,\n so I added logic to skip PVC volume type when PVC is not included in the backup resources to be backed up. (#7472, @sbahar619)\n* Respect and use `credentialsFile` specified in BSL.spec.config when IRSA is configured over Velero Pod Environment credentials (#7374, @reasonerjt)\n* Move the native snapshot definition code into internal directory (#7544, @blackpiglet)\n* Fix issue #7036. Add the implementation of node selection for data mover backups (#7437, @Lyndon-Li)\n* Fix issue #7535, add the MustHave resource check during item collection and item filter for restore (#7585, @Lyndon-Li)\n* build(deps): bump json-patch to v5.8.0 (#7584, @mmorel-35)\n* Add confirm flag to velero plugin add (#7566, @kaovilai)\n* do not skip unknown gvr at the beginning and get new gr when kind is changed (#7523, @27149chen)\n* Fix snapshot leak for backup (#7558, @qiuming-best)\n* For issue #7036, add the document for data mover node selection (#7640, @Lyndon-Li)\n* Add design for Extending VolumePolicies to support more actions (#6956, @shubham-pampattiwar)\n* BackupRepositories associated with a BSL are invalidated when BSL is (re-)created. (#7380, @kaovilai)\n* Improve the concurrency for PVBs in different pods (#7571, @ywk253100)\n* Bump up Kopia to v0.16.0 and open kopia repo with no index change (#7559, @Lyndon-Li)\n* Bump up the versions of several Kubernetes-related libs (#7489, @ywk253100)\n* Make parallel restore configurable (#7512, @qiuming-best)\n* Support certificate-based authentication for Azure (#7549, @ywk253100)\n* Fix issue #7281, batch delete snapshots in the same repo (#7438, @Lyndon-Li)\n* Add CRD name to error message when it is not ready to use (#7295, @josemarevalo)\n* Add the design for node selection for data mover backup (#7383, @Lyndon-Li)\n* Bump up aws-sdk to latest version to leverage Pod Identity credentials. (#7307, @guikcd)\n* Fix issue #7246. Document the behavior for repo snapshot deletion (#7622, @Lyndon-Li)\n* Fix issue #7583, set backupName optional for Restore CRD (#7617, @Lyndon-Li)\n\n" }, { "path": "changelogs/CHANGELOG-1.15.md", "content": "## v1.15\n\n### Download\nhttps://github.com/vmware-tanzu/velero/releases/tag/v1.15.0\n\n### Container Image\n`velero/velero:v1.15.0`\n\n### Documentation\nhttps://velero.io/docs/v1.15/\n\n### Upgrading\nhttps://velero.io/docs/v1.15/upgrade-to-1.15/\n\n### Highlights\n#### Data mover micro service\nData transfer activities for CSI Snapshot Data Movement are moved from node-agent pods to dedicate backupPods or restorePods. This brings many benefits such as: \n- This avoids to access volume data through host path, while host path access is privileged and may involve security escalations, which are concerned by users.\n- This enables users to to control resource (i.e., cpu, memory) allocations in a granular manner, e.g., control them per backup/restore of a volume.\n- This enhances the resilience, crash of one data movement activity won't affect others.\n- This prevents unnecessary full backup because of host path changes after workload pods restart.\n- For more information, check the design https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/vgdp-micro-service/vgdp-micro-service.md.\n\n#### Item Block concepts and ItemBlockAction (IBA) plugin\nItem Block concepts are introduced for resource backups to help to achieve multiple thread backups. Specifically, correlated resources are categorized in the same item block and item blocks could be processed concurrently in multiple threads. \nItemBlockAction plugin is introduced to help Velero to categorize resources into item blocks. At present, Velero provides built-in IBAs for pods and PVCs and Velero also supports customized IBAs for any resources. \nIn v1.15, Velero doesn't support multiple thread process of item blocks though item block concepts and IBA plugins are fully supported. The multiple thread support will be delivered in future releases. \nFor more information, check the design https://github.com/vmware-tanzu/velero/blob/main/design/backup-performance-improvements.md. \n\n#### Node selection for repository maintenance job\nRepository maintenance are resource consuming tasks, Velero now allows you to configure the nodes to run repository maintenance jobs, so that you can run repository maintenance jobs in idle nodes or avoid them to run in nodes hosting critical workloads. \nTo support the configuration, a new repository maintenance configuration configMap is introduced. \nFor more information, check the document https://velero.io/docs/v1.15/repository-maintenance/. \n\n#### Backup PVC read-only configuration\nIn 1.15, Velero allows you to configure the data mover backupPods to read-only mount the backupPVCs. In this way, the data mover expose process could be significantly accelerated for some storages (i.e., ceph). \nTo support the configuration, a new backup PVC configuration configMap is introduced. \nFor more information, check the document https://velero.io/docs/v1.15/data-movement-backup-pvc-configuration/. \n\n#### Backup PVC storage class configuration\nIn 1.15, Velero allows you to configure the storageclass used by the data mover backupPods. In this way, the provision of backupPVCs don't need to adhere to the same pattern as workload PVCs, e.g., for a backupPVC, it only needs one replica, whereas, the a workload PVC may have multiple replicas. \nTo support the configuration, the same backup PVC configuration configMap is used. \nFor more information, check the document https://velero.io/docs/v1.15/data-movement-backup-pvc-configuration/. \n\n#### Backup repository data cache configuration\nThe backup repository may need to cache data on the client side during various repository operations, i.e., read, write, maintenance, etc. The cache consumes the root file system space of the pod where the repository access happens. \nIn 1.15, Velero allows you to configure the total size of the cache per repository. In this way, if your pod doesn't have enough space in its root file system, the pod won't be evicted due to running out of ephemeral storage. \nTo support the configuration, a new backup repository configuration configMap is introduced. \nFor more information, check the document https://velero.io/docs/v1.15/backup-repository-configuration/. \n\n#### Performance improvements\nIn 1.15, several performance related issues/enhancements are included, which makes significant performance improvements in specific scenarios: \n- There was a memory leak of Velero server after plugin calls, now it is fixed, see issue https://github.com/vmware-tanzu/velero/issues/7925\n- The `client-burst/client-qps` parameters are automatically inherited to plugins, so that you can use the same velero server parameters to accelerate the plugin executions when large number of API server calls happen, see issue https://github.com/vmware-tanzu/velero/issues/7806\n- Maintenance of Kopia repository takes huge memory in scenarios that huge number of files have been backed up, Velero 1.15 has included the Kopia upstream enhancement to fix the problem, see issue https://github.com/vmware-tanzu/velero/issues/7510\n\n### Runtime and dependencies\nGolang runtime: v1.22.8 \nkopia: v0.17.0\n\n### Limitations/Known issues\n#### Read-only backup PVC may not work on SELinux environments\nDue to an issue of Kubernetes upstream, if a volume is mounted as read-only in SELinux environments, the read privilege is not granted to any user, as a result, the data mover backup will fail. On the other hand, the backupPVC must be mounted as read-only in order to accelerate the data mover expose process. \nTherefore, a user option is added in the same backup PVC configuration configMap, once the option is enabled, the backupPod container will run as a super privileged container and disable SELinux access control. If you have concern in this super privileged container or you have configured [pod security admissions](https://kubernetes.io/docs/concepts/security/pod-security-admission/) and don't allow super privileged containers, you will not be able to use this read-only backupPVC feature and lose the benefit to accelerate the data mover expose process. \n\n### Breaking changes\n#### Deprecation of Restic\nRestic path for fs-backup is in deprecation process starting from 1.15. According to [Velero deprecation policy](https://github.com/vmware-tanzu/velero/blob/v1.15/GOVERNANCE.md#deprecation-policy), for 1.15, if Restic path is used the backup/restore of fs-backup still creates and succeeds, but you will see warnings in below scenarios: \n- When `--uploader-type=restic` is used in Velero installation\n- When Restic path is used to create backup/restore of fs-backup\n\n#### node-agent configuration name is configurable\nPreviously, a fixed name is searched for node-agent configuration configMap. Now in 1.15, Velero allows you to customize the name of the configMap, on the other hand, the name must be specified by node-agent server parameter `node-agent-configmap`. \n\n#### Repository maintenance job configurations in Velero server parameter are moved to repository maintenance job configuration configMap\nIn 1.15, below Velero server parameters for repository maintenance jobs are moved to the repository maintenance job configuration configMap. While for back compatibility reason, the same Velero sever parameters are preserved as is. But the configMap is recommended and the same values in the configMap take preference if they exist in both places: \n```\n--keep-latest-maintenance-jobs\n--maintenance-job-cpu-request\n--maintenance-job-mem-request\n--maintenance-job-cpu-limit\n--maintenance-job-mem-limit\n```\n\n#### Changing PVC selected-node feature is deprecated\nIn 1.15, the [Changing PVC selected-node feature](https://velero.io/docs/v1.15/restore-reference/#changing-pvc-selected-node) enters deprecation process and will be removed in future releases according to [Velero deprecation policy](https://github.com/vmware-tanzu/velero/blob/v1.15/GOVERNANCE.md#deprecation-policy). Usage of this feature for any purpose is not recommended. \n\n### All Changes\n * add no-relabeling option to backupPVC configmap (#8288, @sseago)\n * only set spec.volumes readonly if PVC is readonly for datamover (#8284, @sseago)\n * Add labels to maintenance job pods (#8256, @shubham-pampattiwar)\n * Add the Carvel package related resources to the restore priority list (#8228, @ywk253100)\n * Reduces indirect imports for plugin/framework importers (#8208, @kaovilai)\n * Add controller name to periodical_enqueue_source. The logger parameter now includes an additional field with the value of reflect.TypeOf(objList).String() and another field with the value of controllerName. (#8198, @kaovilai)\n * Update Openshift SCC docs link (#8170, @shubham-pampattiwar)\n * Partially fix issue #8138, add doc for node-agent memory preserve (#8167, @Lyndon-Li)\n * Pass Velero server command args to the plugins (#8166, @ywk253100)\n * Fix issue #8155, Merge Kopia upstream commits for critical issue fixes and performance improvements (#8158, @Lyndon-Li)\n * Implement the Repo maintenance Job configuration. (#8145, @blackpiglet)\n * Add document for data mover micro service (#8144, @Lyndon-Li)\n * Fix issue #8134, allow to config resource request/limit for data mover micro service pods (#8143, @Lyndon-Li)\n * Apply backupPVCConfig to backupPod volume spec (#8141, @shubham-pampattiwar)\n * Add resource modifier for velero restore describe CLI (#8139, @blackpiglet)\n * Fix issue #7620, add doc for backup repo config (#8131, @Lyndon-Li)\n * Modify E2E and perf test report generated directory (#8129, @blackpiglet)\n * Add docs for backup pvc config support (#8119, @shubham-pampattiwar)\n * Delete generated k8s client and informer. (#8114, @blackpiglet)\n * Add support for backup PVC configuration (#8109, @shubham-pampattiwar)\n * ItemBlock model and phase 1 (single-thread) workflow changes (#8102, @sseago)\n * Fix issue #8032, make node-agent configMap name configurable (#8097, @Lyndon-Li)\n * Fix issue #8072, add the warning messages for restic deprecation (#8096, @Lyndon-Li)\n * Fix issue #7620, add backup repository configuration implementation and support cacheLimit configuration for Kopia repo (#8093, @Lyndon-Li)\n * Patch dbr's status when error happens (#8086, @reasonerjt)\n * According to design #7576, after node-agent restarts, if a DU/DD is in InProgress status, re-capture the data mover ms pod and continue the execution (#8085, @Lyndon-Li)\n * Updates to IBM COS documentation to match current version (#8082, @gjanders)\n * Data mover micro service DUCR/DDCR controller refactor according to design #7576 (#8074, @Lyndon-Li)\n * add retries with timeout to existing patch calls that moves a backup/restore from InProgress/Finalizing to a final status phase. (#8068, @kaovilai)\n * Data mover micro service restore according to design #7576 (#8061, @Lyndon-Li)\n * Internal ItemBlockAction plugins (#8054, @sseago)\n * Data mover micro service backup according to design #7576 (#8046, @Lyndon-Li)\n * Avoid wrapping failed PVB status with empty message. (#8028, @mrnold)\n * Created new ItemBlockAction (IBA) plugin type (#8026, @sseago)\n * Make PVPatchMaximumDuration timeout configurable (#8021, @shubham-pampattiwar)\n * Reuse existing plugin manager for get/put volume info (#8012, @sseago)\n * Data mover ms watcher according to design #7576 (#7999, @Lyndon-Li)\n * New data path for data mover ms according to design #7576 (#7988, @Lyndon-Li)\n * For issue #7700 and #7747, add the design for backup PVC configurations (#7982, @Lyndon-Li)\n * Only get VolumeSnapshotClass when DataUpload exists. (#7974, @blackpiglet)\n * Fix issue #7972, sync the backupPVC deletion in expose clean up (#7973, @Lyndon-Li)\n * Expose the VolumeHelper to third-party plugins. (#7969, @blackpiglet)\n * Check whether the volume's source is PVC before fetching its PV. (#7967, @blackpiglet)\n * Check whether the namespaces specified in namespace filter exist. (#7965, @blackpiglet)\n * Add design for backup repository configurations for issue #7620, #7301 (#7963, @Lyndon-Li)\n * New data path for data mover ms according to design #7576 (#7955, @Lyndon-Li)\n * Skip PV patch step in Restoe workflow for WaitForFirstConsumer VolumeBindingMode Pending state PVCs (#7953, @shubham-pampattiwar)\n * Fix issue #7904, add the deprecation and limitation clarification for change PVC selected-node feature (#7948, @Lyndon-Li)\n * Expose the VolumeHelper to third-party plugins. (#7944, @blackpiglet)\n * Don't consider unschedulable pods unrecoverable (#7899, @sseago)\n * Upgrade to robfig/cron/v3 to support time zone specification. (#7793, @kaovilai)\n * Add the result in the backup's VolumeInfo. (#7775, @blackpiglet)\n * Migrate from github.com/golang/protobuf to google.golang.org/protobuf (#7593, @mmorel-35)\n * Add the design for data mover micro service (#7576, @Lyndon-Li)\n * Descriptive restore error when restoring into a terminating namespace. (#7424, @kaovilai)\n * Ignore missing path error in conditional match (#7410, @seanblong)\n * Propose a deprecation process for velero (#5532, @shubham-pampattiwar)\n" }, { "path": "changelogs/CHANGELOG-1.16.md", "content": "## v1.16\n\n### Download\nhttps://github.com/vmware-tanzu/velero/releases/tag/v1.16.0\n\n### Container Image\n`velero/velero:v1.16.0`\n\n### Documentation\nhttps://velero.io/docs/v1.16/\n\n### Upgrading\nhttps://velero.io/docs/v1.16/upgrade-to-1.16/\n\n### Highlights\n#### Windows cluster support\nIn v1.16, Velero supports to run in Windows clusters and backup/restore Windows workloads, either stateful or stateless:\n * Hybrid build and all-in-one image: the build process is enhanced to build an all-in-one image for hybrid CPU architecture and hybrid platform. For more information, check the design https://github.com/vmware-tanzu/velero/blob/main/design/multiple-arch-build-with-windows.md\n * Deployment in Windows clusters: Velero node-agent, data mover pods and maintenance jobs now support to run in both linux and Windows nodes\n * Data mover backup/restore Windows workloads: Velero built-in data mover supports Windows workloads throughout its full cycle, i.e., discovery, backup, restore, pre/post hook, etc. It automatically identifies Windows workloads and schedules data mover pods to the right group of nodes\n\nCheck the epic issue https://github.com/vmware-tanzu/velero/issues/8289 for more information. \n\n#### Parallel Item Block backup\nv1.16 now supports to back up item blocks in parallel. Specifically, during backup, correlated resources are grouped in item blocks and Velero backup engine creates a thread pool to back up the item blocks in parallel. This significantly improves the backup throughput, especially when there are large scale of resources. \nPre/post hooks also belongs to item blocks, so will also run in parallel along with the item blocks. \nUsers are allowed to configure the parallelism through the `--item-block-worker-count` Velero server parameter. If not configured, the default parallelism is 1. \n\nFor more information, check issue https://github.com/vmware-tanzu/velero/issues/8334. \n\n#### Data mover restore enhancement in scalability\nIn previous releases, for each volume of WaitForFirstConsumer mode, data mover restore is only allowed to happen in the node that the volume is attached. This severely degrades the parallelism and the balance of node resource(CPU, memory, network bandwidth) consumption for data mover restore (https://github.com/vmware-tanzu/velero/issues/8044). \n\nIn v1.16, users are allowed to configure data mover restores running and spreading evenly across all nodes in the cluster. The configuration is done through a new flag `ignoreDelayBinding` in node-agent configuration (https://github.com/vmware-tanzu/velero/issues/8242). \n\n#### Data mover enhancements in observability \nIn 1.16, some observability enhancements are added:\n * Output various statuses of intermediate objects for failures of data mover backup/restore (https://github.com/vmware-tanzu/velero/issues/8267)\n * Output the errors when Velero fails to delete intermediate objects during clean up (https://github.com/vmware-tanzu/velero/issues/8125)\n\nThe outputs are in the same node-agent log and enabled automatically. \n\n#### CSI snapshot backup/restore enhancement in usability\nIn previous releases, a unnecessary VolumeSnapshotContent object is retained for each backup and synced to other clusters sharing the same backup storage location. And during restore, the retained VolumeSnapshotContent is also restored unnecessarily. \n\nIn 1.16, the retained VolumeSnapshotContent is removed from the backup, so no unnecessary CSI objects are synced or restored. \n\nFor more information, check issue https://github.com/vmware-tanzu/velero/issues/8725. \n\n#### Backup Repository Maintenance enhancement in resiliency and observability\nIn v1.16, some enhancements of backup repository maintenance are added to improve the observability and resiliency:\n * A new backup repository maintenance history section, called `RecentMaintenance`, is added to the BackupRepository CR. Specifically, for each BackupRepository, including start/completion time, completion status and error message. (https://github.com/vmware-tanzu/velero/issues/7810)\n * Running maintenance jobs are now recaptured after Velero server restarts. (https://github.com/vmware-tanzu/velero/issues/7753)\n * The maintenance job will not be launched for readOnly BackupStorageLocation. (https://github.com/vmware-tanzu/velero/issues/8238)\n * The backup repository will not try to initialize a new repository for readOnly BackupStorageLocation. (https://github.com/vmware-tanzu/velero/issues/8091)\n * Users now are allowed to configure the intervals of an effective maintenance in the way of `normalGC`, `fastGC` and `eagerGC`, through the `fullMaintenanceInterval` parameter in backupRepository configuration. (https://github.com/vmware-tanzu/velero/issues/8364)\n\n#### Volume Policy enhancement of filtering volumes by PVC labels\nIn v1.16, Volume Policy is extended to support filtering volumes by PVC labels. (https://github.com/vmware-tanzu/velero/issues/8256). \n\n#### Resource Status restore per object\nIn v1.16, users are allowed to define whether to restore resource status per object through an annotation `velero.io/restore-status` set on the object. (https://github.com/vmware-tanzu/velero/issues/8204). \n\n#### Velero Restore Helper binary is merged into Velero image \nIn v1.16, Velero banaries, i.e., velero, velero-helper and velero-restore-helper, are all included into the single Velero image. (https://github.com/vmware-tanzu/velero/issues/8484). \n\n### Runtime and dependencies\nGolang runtime: 1.23.7 \nkopia: 0.19.0\n\n### Limitations/Known issues\n#### Limitations of Windows support\n * fs-backup is not supported for Windows workloads and so fs-backup runs only in linux nodes for linux workloads\n * Backup/restore of NTFS extended attributes/advanced features are not supported, i.e., Security Descriptors, System/Hidden/ReadOnly attributes, Creation Time, NTFS Streams, etc.\n\n### All Changes\n * Add third party annotation support for maintenance job, so that the declared third party annotations could be added to the maintenance job pods (#8812, @Lyndon-Li)\n * Fix issue #8803, use deterministic name to create backupRepository (#8808, @Lyndon-Li)\n * Refactor restoreItem and related functions to differentiate the backup resource name and the restore target resource name. (#8797, @blackpiglet)\n * ensure that PV is removed before VS is deleted (#8777, @ix-rzi)\n * host_pods should not be mandatory to node-agent (#8774, @mpryc)\n * Log doesn't show pv name, but displays %!s(MISSING) instead (#8771, @hu-keyu)\n * Fix issue #8754, add third party annotation support for data mover (#8770, @Lyndon-Li)\n * Add docs for volume policy with labels as a criteria (#8759, @shubham-pampattiwar)\n * Move pvc annotation removal from CSI RIA to regular PVC RIA (#8755, @sseago)\n * Add doc for maintenance history (#8747, @Lyndon-Li)\n * Fix issue #8733, add doc for restorePVC (#8737, @Lyndon-Li)\n * Fix issue #8426, add doc for Windows support (#8736, @Lyndon-Li)\n * Fix issue #8475, refactor build-from-source doc for hybrid image build (#8729, @Lyndon-Li)\n * Return directly if no pod volme backup are tracked (#8728, @ywk253100)\n * Fix issue #8706, for immediate volumes, there is no selected-node annotation on PVC, so deduce the attached node from VolumeAttachment CRs (#8715, @Lyndon-Li)\n * Add labels as a criteria for volume policy (#8713, @shubham-pampattiwar)\n * Copy SecurityContext from Containers[0] if present for PVR (#8712, @sseago)\n * Support pushing images to an insecure registry (#8703, @ywk253100)\n * Modify golangci configuration to make it work. (#8695, @blackpiglet)\n * Run backup post hooks inside ItemBlock synchronously (#8694, @ywk253100)\n * Add docs for object level status restore (#8693, @shubham-pampattiwar)\n * Clean artifacts generated during CSI B/R. (#8684, @blackpiglet)\n * Don't run maintenance on the ReadOnly BackupRepositories. (#8681, @blackpiglet)\n * Fix #8657: WaitGroup panic issue (#8679, @ywk253100)\n * Fixes issue #8214, validate `--from-schedule` flag in create backup command to prevent empty or whitespace-only values. (#8665, @aj-2000)\n * Implement parallel ItemBlock processing via backup_controller goroutines (#8659, @sseago)\n * Clean up leaked CSI snapshot for incomplete backup (#8637, @raesonerjt)\n * Handle update conflict when restoring the status (#8630, @ywk253100)\n * Fix issue #8419, support repo maintenance job to run on Windows nodes (#8626, @Lyndon-Li)\n * Always create DataUpload configmap in restore namespace (#8621, @sseago)\n * Fix issue #8091, avoid to create new repo when BSL is readonly (#8615, @Lyndon-Li)\n * Fix issue #8242, distribute dd evenly across nodes (#8611, @Lyndon-Li)\n * Fix issue #8497, update du/dd progress on completion (#8608, @Lyndon-Li)\n * Fix issue #8418, add Windows toleration to data mover pods (#8606, @Lyndon-Li)\n * Check the PVB status via podvolume Backupper rather than calling API server to avoid API server issue (#8603, @ywk253100)\n * Fix issue #8067, add tmp folder (/tmp for linux, C:\\Windows\\Temp for Windows) as an alternative of udmrepo's config file location (#8602, @Lyndon-Li)\n * Data mover restore for Windows (#8594, @Lyndon-Li)\n * Skip patching the PV in finalization for failed operation (#8591, @reasonerjt)\n * Fix issue #8579, set event burst to block event broadcaster from filtering events (#8590, @Lyndon-Li)\n * Configurable Kopia Maintenance Interval. backup-repository-configmap adds an option for configurable`fullMaintenanceInterval` where fastGC (12 hours), and eagerGC (6 hours) allowing for faster removal of deleted velero backups from kopia repo. (#8581, @kaovilai)\n * Fix issue #7753, recall repo maintenance history on Velero server restart (#8580, @Lyndon-Li)\n * Clear validation errors when schedule is valid (#8575, @ywk253100)\n * Merge restore helper image into Velero server image (#8574, @ywk253100)\n * Don't include excluded items in ItemBlocks (#8572, @sseago)\n * fs uploader and block uploader support Windows nodes (#8569, @Lyndon-Li)\n * Fix issue #8418, support data mover backup for Windows nodes (#8555, @Lyndon-Li)\n * Fix issue #8044, allow users to ignore delay binding the restorePVC of data mover when it is in WaitForFirstConsumer mode (#8550, @Lyndon-Li)\n * Fix issue #8539, validate uploader types when o.CRDsOnly is set to false only since CRD installation doesn't rely on uploader types (#8538, @Lyndon-Li)\n * Fix issue #7810, add maintenance history for backupRepository CRs (#8532, @Lyndon-Li)\n * Make fs-backup work on linux nodes with the new Velero deployment and disable fs-backup if the source/target pod is running in non-linux node (#8424) (#8518, @Lyndon-Li)\n * Fix issue: backup schedule pause/unpause doesn't work (#8512, @ywk253100)\n * Fix backup post hook issue #8159 (caused by #7571): always execute backup post hooks after PVBs are handled (#8509, @ywk253100)\n * Fix issue #8267, enhance the error message when expose fails (#8508, @Lyndon-Li)\n * Fix issue #8416, #8417, deploy Velero server and node-agent in linux/Windows hybrid env (#8504, @Lyndon-Li)\n * Design to add label selector as a criteria for volume policy (#8503, @shubham-pampattiwar)\n * Related to issue #8485, move the acceptedByNode and acceptedTimestamp to Status of DU/DD CRD (#8498, @Lyndon-Li)\n * Add SecurityContext to restore-helper (#8491, @reasonerjt)\n * Fix issue #8433, add third party labels to data mover pods when the same labels exist in node-agent pods (#8487, @Lyndon-Li)\n * Fix issue #8485, add an accepted time so as to count the prepare timeout (#8486, @Lyndon-Li)\n * Fix issue #8125, log diagnostic info for data mover exposers when expose timeout (#8482, @Lyndon-Li)\n * Fix issue #8415, implement multi-arch build and Windows build (#8476, @Lyndon-Li)\n * Pin kopia to 0.18.2 (#8472, @Lyndon-Li)\n * Add nil check for updating DataUpload VolumeInfo in finalizing phase (#8471, @blackpiglet)\n * Allowing Object-Level Resource Status Restore (#8464, @shubham-pampattiwar)\n * For issue #8429. Add the design for multi-arch build and windows build (#8459, @Lyndon-Li)\n * Upgrade go.mod k8s.io/ go.mod to v0.31.3 and implemented proper logger configuration for both client-go and controller-runtime libraries. This change ensures that logging format and level settings are properly applied throughout the codebase. The update improves logging consistency and control across the Velero system. (#8450, @kaovilai)\n * Add Design for Allowing Object-Level Resource Status Restore (#8403, @shubham-pampattiwar)\n * Fix issue #8391, check ErrCancelled from suffix of data mover pod's termination message (#8396, @Lyndon-Li)\n * Fix issue #8394, don't call closeDataPath in VGDP callbacks, otherwise, the VGDP cleanup will hang (#8395, @Lyndon-Li)\n * Adding support in velero Resource Policies for filtering PVs based on additional VolumeAttributes properties under CSI PVs (#8383, @mayankagg9722)\n * Add --item-block-worker-count flag to velero install and server (#8380, @sseago)\n * Make BackedUpItems thread safe (#8366, @sseago)\n * Include --annotations flag in backup and restore create commands (#8354, @alromeros)\n * Use aggregated discovery API to discovery API groups and resources (#8353, @ywk253100)\n * Copy \"envFrom\" from Velero server when creating maintenance jobs (#8343, @evhan)\n * Set hinting region to use for GetBucketRegion() in pkg/repository/config/aws.go (#8297, @kaovilai)\n * Bump up version of client-go and controller-runtime (#8275, @ywk253100)\n * fix(pkg/repository/maintenance): don't panic when there's no container statuses (#8271, @mcluseau)\n * Add Backup warning for inclusion of NS managed by ArgoCD (#8257, @shubham-pampattiwar)\n * Added tracking for deleted namespace status check in restore flow. (#8233, @sangitaray2021)" }, { "path": "changelogs/CHANGELOG-1.17.md", "content": "## v1.17\n\n### Download\nhttps://github.com/vmware-tanzu/velero/releases/tag/v1.17.0\n\n### Container Image\n`velero/velero:v1.17.0`\n\n### Documentation\nhttps://velero.io/docs/v1.17/\n\n### Upgrading\nhttps://velero.io/docs/v1.17/upgrade-to-1.17/\n\n### Highlights\n#### Modernized fs-backup\nIn v1.17, Velero fs-backup is modernized to the micro-service architecture, which brings below benefits: \n- Many features that were absent to fs-backup are now available, i.e., load concurrency control, cancel, resume on restart, etc.\n- fs-backup is more robust, the running backup/restore could survive from node-agent restart; and the resource allocation is in a more granular manner, the failure of one backup/restore won't impact others. \n- The resource usage of node-agent is steady, especially, the node-agent pods won't request huge memory and hold it for a long time. \n\nCheck design https://github.com/vmware-tanzu/velero/blob/main/design/vgdp-micro-service-for-fs-backup/vgdp-micro-service-for-fs-backup.md for more details. \n\n#### fs-backup support Windows cluster\nIn v1.17, Velero fs-backup supports to backup/restore Windows workloads. By leveraging the new micro-service architecture for fs-backup, data mover pods could run in Windows nodes and backup/restore Windows volumes. Together with CSI snapshot data movement for Windows which is delivered in 1.16, Velero now supports Windows workload backup/restore in full scenarios. \nCheck design https://github.com/vmware-tanzu/velero/blob/main/design/vgdp-micro-service-for-fs-backup/vgdp-micro-service-for-fs-backup.md for more details. \n\n#### Volume group snapshot support\nIn v1.17, Velero supports [volume group snapshots](https://kubernetes.io/blog/2024/12/18/kubernetes-1-32-volume-group-snapshot-beta/) which is a beta feature in Kubernetes upstream, for both CSI snapshot backup and CSI snapshot data movement. This allows a snapshot to be taken from multiple volumes at the same point-in-time to achieve write order consistency, which is helpful to achieve better data consistency when multiple volumes being backed up are correlated. \nCheck the document https://velero.io/docs/main/volume-group-snapshots/ for more details. \n\n#### Priority class support\nIn v1.17, [Kubernetes priority class](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/#priorityclass) is supported for all modules across Velero. Specifically, users are allowed to configure priority class to Velero server, node-agent, data mover pods, backup repository maintenance jobs separately. \nCheck design https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/priority-class-name-support_design.md for more details. \n\n#### Scalability and Resiliency improvements of data movers\n##### Reduce excessive number of data mover pods in Pending state\nIn v1.17, Velero allows users to set a `PrepareQueueLength` in the node-agent configuration, data mover pods and volumes out of this number won't be created until data path quota is available, so that excessive number cluster resources won't be taken unnecessarily, which is particularly helpful for large scale environments. This improvement applies to all kinds of data movements, including fs-backup and CSI snapshot data movement. \nCheck design https://github.com/vmware-tanzu/velero/blob/main/design/node-agent-load-soothing.md for more details. \n\n##### Enhancement on node-agent restart handling for data movements\nIn v1.17, data movements in all phases could survive from node-agent restart and resume themselves; when a data movement gets orphaned in special cases, e.g., cluster node absent, it could also be canceled appropriately after the restart. This improvement applies to all kinds of data movements, including fs-backup and CSI snapshot data movement. \nCheck issue https://github.com/vmware-tanzu/velero/issues/8534 for more details. \n\n##### CSI snapshot data movement restore node-selection and node-selection by storage class\nIn v1.17, CSI snapshot data movement restore acquires the same node-selection capability as backup, that is, users could specify which nodes can/cannot run data mover pods for both backup and restore now. And users are also allowed to configure the node-selection per storage class, which is particularly helpful to the environments where a storage class are not usable by all cluster nodes. \nCheck issue https://github.com/vmware-tanzu/velero/issues/8186 and https://github.com/vmware-tanzu/velero/issues/8223 for more details. \n\n#### Include/exclude policy support for resource policy\nIn v1.17, Velero resource policy supports `includeExcludePolicy` besides the existing `volumePolicy`. This allows users to set include/exclude filters for resources in a resource policy configmap, so that these filters are reusable among multiple backups. \nCheck the document https://velero.io/docs/main/resource-filtering/#creating-resource-policies:~:text=resources%3D%22*%22-,Resource%20policies,-Velero%20provides%20resource for more details. \n\n### Runtime and dependencies\nGolang runtime: 1.24.6 \nkopia: 0.21.1 \n\n### Limitations/Known issues\n\n### Breaking changes\n#### Deprecation of Restic\nAccording to [Velero deprecation policy](https://github.com/vmware-tanzu/velero/blob/main/GOVERNANCE.md#deprecation-policy), backup of fs-backup under Restic path is removed in v1.17, so `--uploader-type=restic` is not a valid installation configuration anymore. This means you cannot create a backup under Restic path, but you can still restore from the previous backups under Restic path until v1.19. \n\n#### Repository maintenance job configurations are removed from Velero server parameter\nSince the repository maintenance job configurations are moved to repository maintenance job configMap, in v1.17 below Velero sever parameters are removed:\n- --keep-latest-maintenance-jobs\n- --maintenance-job-cpu-request\n- --maintenance-job-mem-request\n- --maintenance-job-cpu-limit\n- --maintenance-job-mem-limit\n\n### All Changes\n * Add ConfigMap parameters validation for install CLI and server start. (#9200, @blackpiglet)\n * Add priorityclasses to high priority restore list (#9175, @kaovilai)\n * Introduced context-based logger for backend implementations (Azure, GCS, S3, and Filesystem) (#9168, @priyansh17)\n * Fix issue #9140, add os=windows:NoSchedule toleration for Windows pods (#9165, @Lyndon-Li)\n * Remove the repository maintenance job parameters from velero server. (#9147, @blackpiglet)\n * Add include/exclude policy to resources policy (#9145, @reasonerjt)\n * Add ConfigMap support for keepLatestMaintenanceJobs with CLI parameter fallback (#9135, @shubham-pampattiwar)\n * Fix the dd and du's node affinity issue. (#9130, @blackpiglet)\n * Remove the WaitUntilVSCHandleIsReady from vs BIA. (#9124, @blackpiglet)\n * Add comprehensive Volume Group Snapshots documentation with workflow diagrams and examples (#9123, @shubham-pampattiwar)\n * Fix issue #9065, add doc for node-agent prepare queue length (#9118, @Lyndon-Li)\n * Fix issue #9095, update restore doc for PVC selected-node (#9117, @Lyndon-Li)\n * Update CSI Snapshot Data Movement doc for issue #8534, #8185 (#9113, @Lyndon-Li)\n * Fix issue #8986, refactor fs-backup doc after VGDP Micro Service for fs-backup (#9112, @Lyndon-Li)\n * Return error if timeout when checking server version (#9111, @ywk253100)\n * Update \"Default Volumes to Fs Backup\" to \"File System Backup (Default)\" (#9105, @shubham-pampattiwar)\n * Fix issue #9077, don't block backup deletion on list VS error (#9100, @Lyndon-Li)\n * Bump up Kopia to v0.21.1 (#9098, @Lyndon-Li)\n * Add imagePullSecrets inheritance for VGDP pod and maintenance job. (#9096, @blackpiglet)\n * Avoid checking the VS and VSC status in the backup finalizing phase. (#9092, @blackpiglet)\n * Fix issue #9053, Always remove selected-node annotation during PVC restore when no node mapping exists. Breaking change: Previously, the annotation was preserved if the node existed. (#9076, @Lyndon-Li)\n * Enable parameterized kubelet mount path during node-agent installation (#9074, @longxiucai)\n * Fix issue #8857, support third party tolerations for data mover pods (#9072, @Lyndon-Li)\n * Fix issue #8813, remove restic from the valid uploader type (#9069, @Lyndon-Li)\n * Fix issue #8185, allow users to disable pod volume host path mount for node-agent (#9068, @Lyndon-Li)\n * Fix #8344, add the design for a mechanism to soothe creation of data mover pods for DataUpload, DataDownload, PodVolumeBackup and PodVolumeRestore (#9067, @Lyndon-Li)\n * Fix #8344, add a mechanism to soothe creation of data mover pods for DataUpload, DataDownload, PodVolumeBackup and PodVolumeRestore (#9064, @Lyndon-Li)\n * Add Gauge metric for BSL availability (#9059, @reasonerjt)\n * Fix missing defaultVolumesToFsBackup flag output in Velero describe backup cmd (#9056, @shubham-pampattiwar)\n * Allow for proper tracking of multiple hooks per container (#9048, @sseago)\n * Make the backup repository controller doesn't invalidate the BSL on restart (#9046, @blackpiglet)\n * Removed username/password credential handling from newConfigCredential as azidentity.UsernamePasswordCredentialOptions is reported as deprecated. (#9041, @priyansh17)\n * Remove dependency with VolumeSnapshotClass in DataUpload. (#9040, @blackpiglet)\n * Fix issue #8961, cancel PVB/PVR on Velero server restart (#9031, @Lyndon-Li)\n * Fix issue #8962, resume PVB/PVR during node-agent restarts (#9030, @Lyndon-Li)\n * Bump kopia v0.20.1 (#9027, @Lyndon-Li)\n * Fix issue #8965, support PVB/PVR's cancel state in the backup/restore (#9026, @Lyndon-Li)\n * Fix Issue 8816 When specifying LabelSelector on restore, related items such as PVC and VolumeSnapshot are not included (#9024, @amastbau)\n * Fix issue #8963, add legacy PVR controller for Restic path (#9022, @Lyndon-Li)\n * Fix issue #8964, add Windows support for VGDP MS for fs-backup (#9021, @Lyndon-Li)\n * Accommodate VGS workflows in PVC CSI plugin (#9019, @shubham-pampattiwar)\n * Fix issue #8958, add VGDP MS PVB controller (#9015, @Lyndon-Li)\n * Fix issue #8959, add VGDP MS PVR controller (#9014, @Lyndon-Li)\n * Fix issue #8988, add data path for VGDP ms PVR (#9005, @Lyndon-Li)\n * Fix issue #8988, add data path for VGDP ms pvb (#8998, @Lyndon-Li)\n * Skip VS and VSC not created by backup. (#8990, @blackpiglet)\n * Make ResticIdentifier optional for kopia BackupRepositories (#8987, @kaovilai)\n * Fix issue #8960, implement PodVolume exposer for PVB/PVR (#8985, @Lyndon-Li)\n * fix: update mc command in minio-deployment example (#8982, @vishal-chdhry)\n * Fix issue #8957, add design for VGDP MS for fs-backup (#8979, @Lyndon-Li)\n * Add BSL status check for backup/restore operations. (#8976, @blackpiglet)\n * Mark BackupRepository not ready when BSL changed (#8975, @ywk253100)\n * Add support for [distributed snapshotting](https://github.com/kubernetes-csi/external-snapshotter/tree/4cedb3f45790ac593ebfa3324c490abedf739477?tab=readme-ov-file#distributed-snapshotting) (#8969, @flx5)\n * Fix issue #8534, refactor dm controllers to tolerate cancel request in more cases, e.g., node restart, node drain (#8952, @Lyndon-Li)\n * The backup and restore VGDP affinity enhancement implementation. (#8949, @blackpiglet)\n * Remove CSI VS and VSC metadata from backup. (#8946, @blackpiglet)\n * Extend PVCAction itemblock plugin to support grouping PVCs under VGS label key (#8944, @shubham-pampattiwar)\n * Copy security context from origin pod (#8943, @farodin91)\n * Add support for configuring VGS label key (#8938, @shubham-pampattiwar)\n * Add VolumeSnapshotContent into the RIA and the mustHave resource list. (#8924, @blackpiglet)\n * Mounted cloud credentials should not be world-readable (#8919, @sseago)\n * Warn for not found error in patching managed fields (#8902, @sseago)\n * Fix issue 8878, relief node os deduction error checks (#8891, @Lyndon-Li)\n * Skip namespace in terminating state in backup resource collection. (#8890, @blackpiglet)\n * Implement PriorityClass Support (#8883, @kaovilai)\n * Fix Velero adding restore-wait init container when not needed. (#8880, @kaovilai)\n * Pass the logger in kopia related operations. (#8875, @hu-keyu)\n * Inherit the dnsPolicy and dnsConfig from the node agent pod. This is done so that the kopia task uses the same configuration. (#8845, @flx5)\n * Add design for VolumeGroupSnapshot support (#8778, @shubham-pampattiwar)\n * Inherit k8s default volumeSnapshotClass. (#8719, @hu-keyu)\n * CLI automatically discovers and uses cacert from BSL for download requests (#8557, @kaovilai)\n * This PR aims to add s390x support to Velero binary. (#7505, @pandurangkhandeparker)" }, { "path": "changelogs/CHANGELOG-1.18.md", "content": "## v1.18\n\n### Download\nhttps://github.com/vmware-tanzu/velero/releases/tag/v1.18.0\n\n### Container Image\n`velero/velero:v1.18.0`\n\n### Documentation\nhttps://velero.io/docs/v1.18/\n\n### Upgrading\nhttps://velero.io/docs/v1.18/upgrade-to-1.18/\n\n### Highlights\n#### Concurrent backup\nIn v1.18, Velero is capable to process multiple backups concurrently. This is a significant usability improvement, especially for multiple tenants or multiple users case, backups submitted from different users could run their backups simultaneously without interfering with each other. \n\nCheck design https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/concurrent-backup-processing.md for more details.\n\n#### Cache volume for data movers\nIn v1.18, Velero allows users to configure cache volumes for data mover pods during restore for CSI snapshot data movement and fs-backup. This brings below benefits:\n- Solve the problem that data mover pods fail to when pod's ephemeral disk is limited\n- Solve the problem that multiple data mover pods fail to run concurrently in one node when the node's ephemeral disk is limited\n- Working together with backup repository's cache limit configuration, cache volume with appropriate size helps to improve the restore throughput\n\nCheck design https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/backup-repo-cache-volume.md for more details.\n\n#### Incremental size for data movers\nIn v1.18, Velero allows users to observe the incremental size of data movers backups for CSI snapshot data movement and fs-backup, so that users could visually see the data reduction due to incremental backup.\n\n#### Wildcard support for namespaces\nIn v1.18, Velero allows to use Glob regular expressions for namespace filters during backup and restore, so that users could filter namespaces in a batch manner.\n\n#### VolumePolicy for PVC phase\nIn v1.18, Velero VolumePolicy supports actions by PVC phase, which help users to do special operations for PVCs with a specific phase, e.g., skip PVCs in Pending/Lost status from the backup. \n\n#### Scalability and Resiliency improvements\n##### Prevent Velero server OOM Kill for large backup repositories\nIn v1.18, some backup repository operations are delay executed out of Velero server, so Velero server won't be OOM Killed.\n\n#### Performance improvement for VolumePolicy\nIn v1.18, VolumePolicy is enhanced for large number of pods/PVCs so that the performance is significantly improved.\n\n#### Events for data mover pod diagnostic\nIn v1.18, events are recorded into data mover pod diagnostic, which allows user to see more information for troubleshooting when the data mover pod fails.\n\n### Runtime and dependencies\nGolang runtime: 1.25.7 \nkopia: 0.22.3\n\n### Limitations/Known issues\n\n### Breaking changes\n#### Deprecation of PVC selected node feature\nAccording to [Velero deprecation policy](https://github.com/vmware-tanzu/velero/blob/main/GOVERNANCE.md#deprecation-policy), PVC selected node feature is deprecated in v1.18. Velero could appropriately handle PVC's selected-node annotation, so users don't need to do anything particularly.\n\n### All Changes\n* Remove backup from running list when backup fails validation (#9498, @sseago)\n* Maintenance Job only uses the first element of the LoadAffinity array (#9494, @blackpiglet)\n* Fix issue #9478, add diagnose info on expose peek fails (#9481, @Lyndon-Li)\n* Add Role, RoleBinding, ClusterRole, and ClusterRoleBinding in restore sequence. (#9474, @blackpiglet)\n* Add maintenance job and data mover pod's labels and annotations setting. (#9452, @blackpiglet)\n* Fix plugin init container names exceeding DNS-1123 limit (#9445, @mpryc)\n* Add PVC-to-Pod cache to improve volume policy performance (#9441, @shubham-pampattiwar)\n* Remove VolumeSnapshotClass from CSI B/R process. (#9431, @blackpiglet)\n* Use hookIndex for recording multiple restore exec hooks. (#9366, @blackpiglet)\n* Sanitize Azure HTTP responses in BSL status messages (#9321, @shubham-pampattiwar)\n* Remove labels associated with previous backups (#9206, @Joeavaikath)\n* Add VolumePolicy support for PVC Phase conditions to allow skipping Pending PVCs (#9166, @claude)\n* feat: Enhance BackupStorageLocation with Secret-based CA certificate support (#9141, @kaovilai)\n* Add `--apply` flag to `install` command, allowing usage of Kubernetes apply to make changes to existing installs (#9132, @mjnagel)\n* Fix issue #9194, add doc for GOMAXPROCS behavior change (#9420, @Lyndon-Li)\n* Apply volume policies to VolumeGroupSnapshot PVC filtering (#9419, @shubham-pampattiwar)\n* Fix issue #9276, add doc for cache volume support (#9418, @Lyndon-Li)\n* Add Prometheus metrics for maintenance jobs (#9414, @shubham-pampattiwar)\n* Fix issue #9400, connect repo first time after creation so that init params could be written (#9407, @Lyndon-Li)\n* Cache volume for PVR (#9397, @Lyndon-Li)\n* Cache volume support for DataDownload (#9391, @Lyndon-Li)\n* don't copy securitycontext from first container if configmap found (#9389, @sseago)\n* Refactor repo provider interface for static configuration (#9379, @Lyndon-Li)\n* Fix issue #9365, prevent fake completion notification due to multiple update of single PVR (#9375, @Lyndon-Li)\n* Add cache volume configuration (#9370, @Lyndon-Li)\n* Track actual resource names for GenerateName in restore status (#9368, @shubham-pampattiwar)\n* Fix managed fields patch for resources using GenerateName (#9367, @shubham-pampattiwar)\n* Support cache volume for generic restore exposer and pod volume exposer (#9362, @Lyndon-Li)\n* Add incrementalSize to DU/PVB for reporting new/changed size (#9357, @sseago)\n* Add snapshotSize for DataDownload, PodVolumeRestore (#9354, @Lyndon-Li)\n* Add cache dir configuration for udmrepo (#9353, @Lyndon-Li)\n* Fix the Job build error when BackupReposiotry name longer than 63. (#9350, @blackpiglet)\n* Add cache configuration to VGDP (#9342, @Lyndon-Li)\n* Fix issue #9332, add bytesDone for cache files (#9333, @Lyndon-Li)\n* Fix typos in documentation (#9329, @T4iFooN-IX)\n* Concurrent backup processing (#9307, @sseago)\n* VerifyJSONConfigs verify every elements in Data. (#9302, @blackpiglet)\n* Fix issue #9267, add events to data mover prepare diagnostic (#9296, @Lyndon-Li)\n* Add option for privileged fs-backup pod (#9295, @sseago)\n* Fix issue #9193, don't connect repo in repo controller (#9291, @Lyndon-Li)\n* Implement concurrency control for cache of native VolumeSnapshotter plugin. (#9281, @0xLeo258)\n* Fix issue #7904, remove the code and doc for PVC node selection (#9269, @Lyndon-Li)\n* Fix schedule controller to prevent backup queue accumulation during extended blocking scenarios by properly handling empty backup phases (#9264, @shubham-pampattiwar)\n* Fix repository maintenance jobs to inherit allowlisted tolerations from Velero deployment (#9256, @shubham-pampattiwar)\n* Implement wildcard namespace pattern expansion for backup namespace includes/excludes. This change adds support for wildcard patterns (*, ?, [abc], {a,b,c}) in namespace includes and excludes during backup operations (#9255, @Joeavaikath)\n* Protect VolumeSnapshot field from race condition during multi-thread backup (#9248, @0xLeo258)\n* Update AzureAD Microsoft Authentication Library to v1.5.0 (#9244, @priyansh17)\n* Get pod list once per namespace in pvc IBA (#9226, @sseago)\n* Fix issue #7725, add design for backup repo cache configuration (#9148, @Lyndon-Li)\n* Fix issue #9229, don't attach backupPVC to the source node (#9233, @Lyndon-Li)\n* feat: Permit specifying annotations for the BackupPVC (#9173, @clementnuss)" }, { "path": "changelogs/CHANGELOG-1.2.md", "content": "## v1.2.0\n#### 2019-11-07\n\n### Download\nhttps://github.com/vmware-tanzu/velero/releases/tag/v1.2.0\n\n### Container Image\n`velero/velero:v1.2.0`\n\nPlease note that as of this release we are no longer publishing new container images to `gcr.io/heptio-images`. The existing ones will remain there for the foreseeable future.\n\n### Documentation\nhttps://velero.io/docs/v1.2.0/\n\n### Upgrading\nhttps://velero.io/docs/v1.2.0/upgrade-to-1.2/\n\n### Highlights\n## Moving Cloud Provider Plugins Out of Tree\n\nVelero has had built-in support for AWS, Microsoft Azure, and Google Cloud Platform (GCP) since day 1. When Velero moved to a plugin architecture for object store providers and volume snapshotters in version 0.6, the code for these three providers was converted to use the plugin interface provided by this new architecture, but the cloud provider code still remained inside the Velero codebase. This put the AWS, Azure, and GCP plugins in a different position compared with other providers’ plugins, since they automatically shipped with the Velero binary and could include documentation in-tree.\n\nWith version 1.2, we’ve extracted the AWS, Azure, and GCP plugins into their own repositories, one per provider. We now also publish one plugin image per provider. This change brings these providers to parity with other providers’ plugin implementations, reduces the size of the core Velero binary by not requiring each provider’s SDK to be included, and opens the door for the plugins to be maintained and released independently of core Velero.\n\n## Restic Integration Improvements\n\nWe’ve continued to work on improving Velero’s restic integration. With this release, we’ve made the following enhancements:\n\n- Restic backup and restore progress is now captured during execution and visible to the user through the `velero backup/restore describe --details` command. The details are updated every 10 seconds. This provides a new level of visibility into restic operations for users.\n- Restic backups of persistent volume claims (PVCs) now remain incremental across the rescheduling of a pod. Previously, if the pod using a PVC was rescheduled, the next restic backup would require a full rescan of the volume’s contents. This improvement potentially makes such backups significantly faster.\n- Read-write-many volumes are no longer backed up once for every pod using the volume, but instead just once per Velero backup. This improvement speeds up backups and prevents potential restore issues due to multiple copies of the backup being processed simultaneously.\n\n\n## Clone PVs When Cloning a Namespace\n\nBefore version 1.2, you could clone a Kubernetes namespace by backing it up and then restoring it to a different namespace in the same cluster by using the `--namespace-mappings` flag with the `velero restore create` command. However, in this scenario, Velero was unable to clone persistent volumes used by the namespace, leading to errors for users.\n\nIn version 1.2, Velero automatically detects when you are trying to clone an existing namespace, and clones the persistent volumes used by the namespace as well. This doesn’t require the user to specify any additional flags for the `velero restore create` command. This change lets you fully achieve your goal of cloning namespaces using persistent storage within a cluster.\n\n## Improved Server-Side Encryption Support\n\nTo help you secure your important backup data, we’ve added support for more forms of server-side encryption of backup data on both AWS and GCP. Specifically:\n- On AWS, Velero now supports Amazon S3-managed encryption keys (SSE-S3), which uses AES256 encryption, by specifying `serverSideEncryption: AES256` in a backup storage location’s config.\n- On GCP, Velero now supports using a specific Cloud KMS key for server-side encryption by specifying `kmsKeyName: ` in a backup storage location’s config.\n\n## CRD Structural Schema\n\nIn Kubernetes 1.16, custom resource definitions (CRDs) reached general availability. Structural schemas are required for CRDs created in the `apiextensions.k8s.io/v1` API group. Velero now defines a structural schema for each of its CRDs and automatically applies it the user runs the `velero install` command. The structural schemas enable the user to get quicker feedback when their backup, restore, or schedule request is invalid, so they can immediately remediate their request.\n\n### All Changes\n * Ensure object store plugin processes are cleaned up after restore and after BSL validation during server start up (#2041, @betta1)\n * bug fix: don't try to restore pod volume backups that don't have a snapshot ID (#2031, @skriss)\n * Restore Documentation: Updated Restore Documentation with Clarification implications of removing restore object. (#1957, @nainav)\n * add `--allow-partially-failed` flag to `velero restore create` for use with `--from-schedule` to allow partially-failed backups to be restored (#1994, @skriss)\n * Allow backup storage locations to specify backup sync period or toggle off sync (#1936, @betta1)\n * Remove cloud provider code (#1985, @carlisia)\n * Restore action for cluster/namespace role bindings (#1974, @alexander-demichev)\n * Add `--no-default-backup-location` flag to `velero install` (#1931, @Frank51)\n * If includeClusterResources is nil/auto, pull in necessary CRDs in backupResource (#1831, @sseago)\n * Azure: add support for Azure China/German clouds (#1938, @andyzhangx)\n * Add a new required `--plugins` flag for `velero install` command. `--plugins` takes a list of container images to add as initcontainers. (#1930, @nrb)\n * restic: only backup read-write-many PVCs at most once, even if they're annotated for backup from multiple pods. (#1896, @skriss)\n * Azure: add support for cross-subscription backups (#1895, @boxcee)\n * adds `insecureSkipTLSVerify` server config for AWS storage and `--insecure-skip-tls-verify` flag on client for self-signed certs (#1793, @s12chung)\n * Add check to update resource field during backupItem (#1904, @spiffcs)\n * Add `LD_LIBRARY_PATH` (=/plugins) to the env variables of velero deployment. (#1893, @lintongj)\n * backup sync controller: stop using `metadata/revision` file, do a full diff of bucket contents vs. cluster contents each sync interval (#1892, @skriss)\n * bug fix: during restore, check item's original namespace, not the remapped one, for inclusion/exclusion (#1909, @skriss)\n * adds structural schema to Velero CRDs created on Velero install, enabling validation of Velero API fields (#1898, @prydonius)\n * GCP: add support for specifying a Cloud KMS key name to use for encrypting backups in a storage location. (#1879, @skriss)\n * AWS: add support for SSE-S3 AES256 encryption via `serverSideEncryption` config field in BackupStorageLocation (#1869, @skriss)\n * change default `restic prune` interval to 7 days, add `velero server/install` flags for specifying an alternate default value. (#1864, @skriss)\n * velero install: if `--use-restic` and `--wait` are specified, wait up to a minute for restic daemonset to be ready (#1859, @skriss)\n * report restore progress in PodVolumeRestores and expose progress in the velero restore describe --details command (#1854, @prydonius)\n * Jekyll Site updates - modifies documentation to use a wider layout; adds better markdown table formatting (#1848, @ccbayer)\n * fix excluding additional items with the velero.io/exclude-from-backup=true label (#1843, @prydonius)\n * report backup progress in PodVolumeBackups and expose progress in the velero backup describe --details command. Also upgrades restic to v0.9.5 (#1821, @prydonius)\n * Add `--features` argument to all velero commands to provide feature flags that can control enablement of pre-release features. (#1798, @nrb)\n * when backing up PVCs with restic, specify `--parent` flag to prevent full volume rescans after pod reschedules (#1807, @skriss)\n * remove 'restic check' calls from before/after 'restic prune' since they're redundant (#1794, @skriss)\n * fix error formatting due interpreting % as printf formatted strings (#1781, @s12chung)\n * when using `velero restore create --namespace-mappings ...` to create a second copy of a namespace in a cluster, create copies of the PVs used (#1779, @skriss)\n * adds --from-schedule flag to the `velero create backup` command to create a Backup from an existing Schedule (#1734, @prydonius)\n" }, { "path": "changelogs/CHANGELOG-1.3.md", "content": "## v1.3.2\n### 2020-04-03\n\n### Download\nhttps://github.com/vmware-tanzu/velero/releases/tag/v1.3.2\n\n### Container Image\n`velero/velero:v1.3.2`\n\n### Documentation\nhttps://velero.io/docs/v1.3.2/\n\n### Upgrading\nhttps://velero.io/docs/v1.3.2/upgrade-to-1.3/\n\n### All Changes\n* Allow `plugins/` as a valid top-level directory within backup storage locations. This directory is a place for plugin authors to store arbitrary data as needed. It is recommended to create an additional subdirectory under `plugins/` specifically for your plugin, e.g. `plugins/my-plugin-data/`. (#2350, @skriss)\n* bug fix: don't panic in `velero restic repo get` when last maintenance time is `nil` (#2315, @skriss)\n\n## v1.3.1\n### 2020-03-10\n\n### Download\nhttps://github.com/vmware-tanzu/velero/releases/tag/v1.3.1\n\n### Container Image\n`velero/velero:v1.3.1`\n\n### Documentation\nhttps://velero.io/docs/v1.3.1/\n\n### Upgrading\nhttps://velero.io/docs/v1.3.1/upgrade-to-1.3/\n\n### Highlights\n\nFixed a bug that caused failures when backing up CustomResourceDefinitions with whole numbers in numeric fields.\n\n### All Changes\n * Fix CRD backup failures when fields contained a whole number. (#2322, @nrb)\n\n\n## v1.3.0\n#### 2020-03-02\n\n### Download\nhttps://github.com/vmware-tanzu/velero/releases/tag/v1.3.0\n\n### Container Image\n`velero/velero:v1.3.0`\n\n### Documentation\nhttps://velero.io/docs/v1.3.0/\n\n### Upgrading\nhttps://velero.io/docs/v1.3.0/upgrade-to-1.3/\n\n### Highlights\n\n#### Custom Resource Definition Backup and Restore Improvements\n\nThis release includes a number of related bug fixes and improvements to how Velero backs up and restores custom resource definitions (CRDs) and instances of those CRDs.\n\nWe found and fixed three issues around restoring CRDs that were originally created via the `v1beta1` CRD API. The first issue affected CRDs that had the `PreserveUnknownFields` field set to `true`. These CRDs could not be restored into 1.16+ Kubernetes clusters, because the `v1` CRD API does not allow this field to be set to `true`. We added code to the restore process to check for this scenario, to set the `PreserveUnknownFields` field to `false`, and to instead set `x-kubernetes-preserve-unknown-fields` to `true` in the OpenAPIv3 structural schema, per Kubernetes guidance. For more information on this, see the [Kubernetes documentation](https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#pruning-versus-preserving-unknown-fields). The second issue affected CRDs without structural schemas. These CRDs need to be backed up/restored through the `v1beta1` API, since all CRDs created through the `v1` API must have structural schemas. We added code to detect these CRDs and always back them up/restore them through the `v1beta1` API. Finally, related to the previous issue, we found that our restore code was unable to handle backups with multiple API versions for a given resource type, and we’ve remediated this as well.\n\nWe also improved the CRD restore process to enable users to properly restore CRDs and instances of those CRDs in a single restore operation. Previously, users found that they needed to run two separate restores: one to restore the CRD(s), and another to restore instances of the CRD(s). This was due to two deficiencies in the Velero code. First, Velero did not wait for a CRD to be fully accepted by the Kubernetes API server and ready for serving before moving on; and second, Velero did not refresh its cached list of available APIs in the target cluster after restoring CRDs, so it was not aware that it could restore instances of those CRDs.\n\nWe fixed both of these issues by (1) adding code to wait for CRDs to be “ready” after restore before moving on, and (2) refreshing the cached list of APIs after restoring CRDs, so any instances of newly-restored CRDs could subsequently be restored.\n\nWith all of these fixes and improvements in place, we hope that the CRD backup and restore experience is now seamless across all supported versions of Kubernetes.\n\n\n#### Multi-Arch Docker Images\n\nThanks to community members [@Prajyot-Parab](https://github.com/Prajyot-Parab) and [@shaneutt](https://github.com/shaneutt), Velero now provides multi-arch container images by using Docker manifest lists. We are currently publishing images for `linux/amd64`, `linux/arm64`, `linux/arm`, and `linux/ppc64le` in [our Docker repository](https://hub.docker.com/r/velero/velero/tags?page=1&name=v1.3&ordering=last_updated).\n\nUsers don’t need to change anything other than updating their version tag - the v1.3 image is `velero/velero:v1.3.0`, and Docker will automatically pull the proper architecture for the host.\n\nFor more information on manifest lists, see [Docker’s documentation](https://docs.docker.com/registry/spec/manifest-v2-2/). \n\n\n#### Bug Fixes, Usability Enhancements, and More\n\nWe fixed a large number of bugs and made some smaller usability improvements in this release. Here are a few highlights:\n\n- Support private registries with custom ports for the restic restore helper image ([PR #1999](https://github.com/vmware-tanzu/velero/pull/1999), [@cognoz](https://github.com/cognoz))\n- Use AWS profile from BackupStorageLocation when invoking restic ([PR #2096](https://github.com/vmware-tanzu/velero/pull/2096), [@dinesh](https://github.com/dinesh))\n- Allow restores from schedules in other clusters ([PR #2218](https://github.com/vmware-tanzu/velero/pull/2218), [@cpanato](https://github.com/cpanato))\n- Fix memory leak & race condition in restore code ([PR #2201](https://github.com/vmware-tanzu/velero/pull/2201), [@skriss](https://github.com/skriss))\n\n\n### All Changes\n * Corrected the selfLink for Backup CR in site/docs/main/output-file-format.md (#2292, @RushinthJohn)\n * Back up schema-less CustomResourceDefinitions as v1beta1, even if they are retrieved via the v1 endpoint. (#2264, @nrb)\n * Bug fix: restic backup volume snapshot to the second location failed (#2244, @jenting)\n * Added support of using PV name from volumesnapshotter('SetVolumeID') in case of PV renaming during the restore (#2216, @mynktl)\n * Replaced deprecated helm repo url at all it appearance at docs. (#2209, @markrity)\n * added support for arm and arm64 images (#2227, @shaneutt)\n * when restoring from a schedule, validate by checking for backup(s) labeled with the schedule name rather than existence of the schedule itself, to allow for restoring from deleted schedules and schedules in other clusters (#2218, @cpanato)\n * bug fix: back up server-preferred version of CRDs rather than always the `v1beta1` version (#2230, @skriss)\n * Wait for CustomResourceDefinitions to be ready before restoring CustomResources. Also refresh the resource list from the Kubernetes API server after restoring CRDs in order to properly restore CRs. (#1937, @nrb)\n * When restoring a v1 CRD with PreserveUnknownFields = True, make sure that the preservation behavior is maintained by copying the flag into the Open API V3 schema, but update the flag so as to allow the Kubernetes API server to accept the CRD without error. (#2197, @nrb)\n * Enable pruning unknown CRD fields (#2187, @jenting)\n * bump restic to 0.9.6 to fix some issues with non AWS standard regions (#2210, @Sh4d1)\n * bug fix: fix race condition resulting in restores sometimes succeeding despite restic restore failures (#2201, @skriss)\n * Bug fix: Check for nil LastMaintenanceTime in ResticRepository dueForMaintenance (#2200, @sseago)\n * repopulate backup_last_successful_timestamp metrics for each schedule after server restart (#2196, @skriss)\n * added support for ppc64le images and manifest lists (#1768, @prajyot)\n * bug fix: only prioritize restoring `replicasets.apps`, not `replicasets.extensions` (#2157, @skriss)\n * bug fix: restore both `replicasets.apps` *and* `replicasets.extensions` before `deployments` (#2120, @skriss)\n * bug fix: don't restore cluster-scoped resources when restoring specific namespaces and IncludeClusterResources is nil (#2118, @skriss)\n * Enabling Velero to switch credentials (`AWS_PROFILE`) if multiple s3-compatible backupLocations are present (#2096, @dinesh)\n * bug fix: deep-copy backup's labels when constructing snapshot tags, so the PV name isn't added as a label to the backup (#2075, @skriss)\n * remove the `fsfreeze-pause` image being published from this repo; replace it with `ubuntu:bionic` in the nginx example app (#2068, @skriss)\n * add support for a private registry with a custom port in a restic-helper image (#1999, @cognoz)\n * return better error message to user when cluster config can't be found via `--kubeconfig`, `$KUBECONFIG`, or in-cluster config (#2057, @skriss)\n" }, { "path": "changelogs/CHANGELOG-1.4.md", "content": "## v1.4.2\n### 2020-07-13\n\n### Download\nhttps://github.com/vmware-tanzu/velero/releases/tag/v1.4.2\n\n### Container Image\n`velero/velero:v1.4.2`\n\n### Documentation\nhttps://velero.io/docs/v1.4/\n\n### Upgrading\nhttps://velero.io/docs/v1.4/upgrade-to-1.4/\n\n### All Changes\n * log a warning instead of erroring if an additional item returned from a plugin can't be found in the Kubernetes API (#2595, @skriss)\n * Adjust restic default time out to 4 hours and base pod resource requests to 500m CPU/512Mi memory. (#2696, @nrb)\n * capture version of the CRD prior before invoking the remap_crd_version backup item action (#2683, @ashish-amarnath)\n\n\n## v1.4.1\n\nThis tag was created in code, but has no associated docker image due to misconfigured building infrastructure. v1.4.2 fixes this.\n\n## v1.4.0\n### 2020-05-26\n\n### Download\nhttps://github.com/vmware-tanzu/velero/releases/tag/v1.4.0\n\n### Container Image\n`velero/velero:v1.4.0`\n\n### Documentation\nhttps://velero.io/docs/v1.4/\n\n### Upgrading\nhttps://velero.io/docs/v1.4/upgrade-to-1.4/\n\n### Highlights\n\n * Added beta-level CSI support!\n * Added custom CA certificate support\n * Backup progress reporting\n * Changed backup tarball format to support all versions of a given resource\n\n### All Changes\n * increment restic volumesnapshot count after successful pvb create (#2542, @ashish-amarnath)\n * Add details of CSI volumesnapshotcontents associated with a backup to `velero backup describe` when the `EnableCSI` feature flag is given on the velero client. (#2448, @nrb)\n * Allow users the option to retrieve all versions of a given resource (instead of just the preferred version) from the API server with the `EnableAPIGroupVersions` feature flag. (#2373, @brito-rafa)\n * Changed backup tarball format to store all versions of a given resource, updated backup tarball format to 1.1.0. (#2373, @brito-rafa)\n * allow feature flags to be passed from install CLI (#2503, @ashish-amarnath)\n * sync backups' CSI API objects into the cluster as part of the backup sync controller (#2496, @ashish-amarnath)\n * bug fix: in error location logging hook, if the item logged under the `error` key doesn't implement the `error` interface, don't return an error since this is a valid scenario (#2487, @skriss)\n * bug fix: in CRD restore plugin, don't use runtime.DefaultUnstructuredConverter.FromUnstructured(...) to avoid conversion issues when float64 fields contain int values (#2484, @skriss)\n * during backup deletion also delete CSI volumesnapshotcontents that were created as a part of the backup but the associated volumesnapshot object does not exist (#2480, @ashish-amarnath)\n * If plugins don't support the `--features` flag, don't pass it to them. Also, update the standard plugin server to ignore unknown flags. (#2479, @skriss)\n * At backup time, if a CustomResourceDefinition appears to have been created via the v1beta1 endpoint, retrieve it from the v1beta1 endpoint instead of simply changing the APIVersion. (#2478, @nrb)\n * update container base images from ubuntu:bionic to ubuntu:focal (#2471, @skriss)\n * bug fix: when a resource includes/excludes list contains unresolvable items, don't remove them from the list, so that the list doesn't inadvertently end up matching *all* resources. (#2462, @skriss)\n * Azure: add support for getting storage account key for restic directly from an environment variable (#2455, @jaygridley)\n * Support to skip VSL validation for the backup having SnapshotVolumes set to false or created with `--snapshot-volumes=false` (#2450, @mynktl)\n * report backup progress (number of items backed up so far out of an estimated total number of items) during backup in the logs and as status fields on the Backup custom resource (#2440, @skriss)\n * bug fix: populate namespace in logs for backup errors (#2438, @skriss)\n * during backup deletion also delete CSI volumesnapshots that were created as a part of the backup (#2411, @ashish-amarnath)\n * bump Kubernetes module dependencies to v0.17.4 to get fix for https://github.com/kubernetes/kubernetes/issues/86149 (#2407, @skriss)\n * bug fix: save PodVolumeBackup manifests to object storage even if the volume was empty, so that on restore, the PV is dynamically reprovisioned if applicable (#2390, @skriss)\n * Adding new restoreItemAction for PVC to update the selected-node annotation (#2377, @mynktl)\n * Added a --cacert flag to the install command to provide the CA bundle to use when verifying TLS connections to object storage (#2368, @mansam)\n * Added a `--cacert` flag to the velero client describe, download, and logs commands to allow passing a path to a certificate to use when verifying TLS connections to object storage. Also added a corresponding client config option called `cacert` which takes a path to a certificate bundle to use as a default when `--cacert` is not specified. (#2364, @mansam)\n * support setting a custom CA certificate on a BSL to use when verifying TLS connections (#2353, @mansam)\n * adding annotations on backup CRD for k8s major, minor and git versions (#2346, @brito-rafa)\n * When the EnableCSI feature flag is provided, upload CSI VolumeSnapshots and VolumeSnapshotContents to object storage as gzipped JSON. (#2323, @nrb)\n * add CSI snapshot API types into default restore priorities (#2318, @ashish-amarnath)\n * refactoring: wait for all informer caches to sync before running controllers (#2299, @skriss)\n * refactor restore code to lazily resolve resources via discovery and eliminate second restore loop for instances of restored CRDs (#2248, @skriss)\n * upgrade to go 1.14 and migrate from `dep` to go modules (#2214, @skriss)\n * clarify the wording for restore describe for namespaces included\n" }, { "path": "changelogs/CHANGELOG-1.5.md", "content": "## v1.5.1\n### 2020-09-16\n\n### Download\nhttps://github.com/vmware-tanzu/velero/releases/tag/v1.5.1\n\n### Container Image\n`velero/velero:v1.5.1`\n\n### Documentation\nhttps://velero.io/docs/v1.5/\n\n### Upgrading\nhttps://velero.io/docs/v1.5/upgrade-to-1.5/\n\n### Highlights\n\n * Auto Volume Backup Using Restic with `--default-volumes-to-restic` flag\n * DeleteItemAction plugins\n * Code modernization\n * Restore Hooks: InitContianer Restore Hooks and Exec Restore Hooks\n\n### All Changes\n\n * 🏃‍♂️ add shortnames for CRDs (#2911, @ashish-amarnath)\n * Use format version instead of version on `velero backup describe` since version has been deprecated (#2901, @jenting)\n * fix EnableAPIGroupersions output log format (#2882, @jenting)\n * Convert ServerStatusRequest controller to kubebuilder (#2838, @carlisia)\n * rename the PV if VolumeSnapshotter has modified the PV name (#2835, @pawanpraka1)\n * Implement post-restore exec hooks in pod containers (#2804, @areed)\n * Check for errors on restic backup command (#2863, @dymurray)\n * 🐛 fix passing LDFLAGS across build stages (#2853, @ashish-amarnath)\n * Feature: Invoke DeleteItemAction plugins based on backup contents when a backup is deleted. (#2815, @nrb)\n * When JSON logging format is enabled, place error message at \"error.message\" instead of \"error\" for compatibility with Elasticsearch/ELK and the Elastic Common Schema (#2830, @bgagnon)\n * discovery Helper support get GroupVersionResource and an APIResource from GroupVersionKind (#2764, @runzexia)\n * Migrate site from Jekyll to Hugo (#2720, @tbatard)\n * Add the DeleteItemAction plugin type (#2808, @nrb)\n * 🐛 Manually patch the generated yaml for restore CRD as a hacky workaround (#2814, @ashish-amarnath)\n * Setup crd validation github action on k8s versions (#2805, @ashish-amarnath)\n * 🐛 Supply command to run restic-wait init container (#2802, @ashish-amarnath)\n * Make init and exec restore hooks as optional in restore hookSpec (#2793, @ashish-amarnath)\n * Implement restore hooks injecting init containers into pod spec (#2787, @ashish-amarnath)\n * Pass default-volumes-to-restic flag from create schedule to backup (#2776, @ashish-amarnath)\n * Enhance Backup to support backing up resources in specific orders and add --ordered-resources option to support this feature. (#2724, @phuong)\n * Fix inconsistent type for the \"resource\" structured logging field (#2796, @bgagnon)\n * Add the ability to set the allowPrivilegeEscalation flag in the securityContext for the Restic restore helper. (#2792, @doughepi)\n * Add cacert flag for velero backup-location create (#2778, @jenting)\n * Exclude volumes mounting secrets and configmaps from defaulting volume backups to restic (#2762, @ashish-amarnath)\n * Add types to implement restore hooks (#2761, @ashish-amarnath)\n * Add wait group and error channel for restore hooks to restore context. (#2755, @areed)\n * Refactor image builds to use buildx for multi arch image building (#2754, @robreus)\n * Add annotation key constants for restore hooks (#2750, @ashish-amarnath)\n * Adds Start and CompletionTimestamp to RestoreStatus\nDisplays the Timestamps when issued a print or describe (#2748, @thejasbabu)\n * Move pkg/backup/item_hook_handlers.go to internal/hook (#2734, @nrb)\n * add metrics for restic back up operation (#2719, @ashish-amarnath)\n * StorageGrid compatibility by removing explicit gzip accept header setting (#2712, @fvsqr)\n * restic: add support for setting SecurityContext (runAsUser, runAsGroup) for restore (#2621, @jaygridley)\n * Add backupValidationFailureTotal to metrics (#2714, @kathpeony)\n * bump Kubernetes module dependencies to v0.18.4 to fix https://github.com/vmware-tanzu/velero/issues/2540 by adding code compatibility with kubernetes v1.18 (#2651, @laverya)\n * Add a BSL controller to handle validation + update BSL status phase (validation removed from the server and no longer blocks when there's any invalid BSL) (#2674, @carlisia)\n * updated acceptable values on cron schedule from 0-7 to 0-6 (#2676, @dthrasher)\n * Improve velero download doc (#2660, @carlisia)\n * Update basic-install and release-instructions documentation for Windows Chocolatey package (#2638, @adamrushuk)\n * move CSI plugin out of prototype into beta (#2636, @ashish-amarnath)\n * Add a new supported provider for an object storage plugin for Storj (#2635, @jessicagreben)\n * Update basic-install.md documentation: Add windows cli installation option via chocolatey (#2629, @adamrushuk)\n * Documentation: Update Jekyll to 4.1.0. Switch from redcarpet to kramdown for Markdown renderer (#2625, @tbatard)\n * improve builder image handling so that we don't rebuild each `make shell` (#2620, @mauilion)\n * first check if there are pending changed on the build-image dockerfile if so build it.\n * then check if there is an image in the registry if so pull it.\n * then build an image cause we don't have a cached image. (this handles the backward compat case.)\n * fix make clean to clear go mod cache before removing dirs (for containerized builds)\n * Add linter checks to Makefile (#2615, @tbatard)\n * add a CI check for a changelog file (#2613, @ashish-amarnath)\n * implement option to back up all volumes by default with restic (#2611, @ashish-amarnath)\n * When a timeout string can't be parsed, log the error as a warning instead of silently consuming the error. (#2610, @nrb)\n * Azure: support using `aad-pod-identity` auth when using restic (#2602, @skriss)\n * log a warning instead of erroring if an additional item returned from a plugin can't be found in the Kubernetes API (#2595, @skriss)\n * when creating new backup from schedule from cli, allow backup name to be automatically generated (#2569, @cblecker)\n * Convert manifests + BSL api client to kubebuilder (#2561, @carlisia)\n * backup/restore: reinstantiate backup store just before uploading artifacts to ensure credentials are up-to-date (#2550, @skriss)\n" }, { "path": "changelogs/CHANGELOG-1.6.md", "content": "## v1.6.0\n### 2021-04-12\n\n### Download\nhttps://github.com/vmware-tanzu/velero/releases/tag/v1.6.0\n\n### Container Image\n`velero/velero:v1.6.0`\n\n### Documentation\nhttps://velero.io/docs/v1.6/\n\n### Upgrading\nhttps://velero.io/docs/v1.6/upgrade-to-1.6/\n\n### Highlights\n\n * Support for per-BSL credentials\n * Progress reporting for restores\n * Restore API Groups by priority level\n * Restic v0.12.0 upgrade\n * End-to-end testing \n * CLI usability improvements\n\n### All Changes\n\n * Add support for restic to use per-BSL credentials. Velero will now serialize the secret referenced by the `Credential` field in the BSL and use this path when setting provider specific environment variables for restic commands. (#3489, @zubron)\n * Upgrade restic from v0.9.6 to v0.12.0. (#3528, @ashish-amarnath)\n * Progress reporting added for Velero Restores (#3125, @pranavgaikwad)\n * Add uninstall option for velero cli (#3399, @vadasambar)\n * Add support for per-BSL credentials. Velero will now serialize the secret referenced by the `Credential` field in the BSL and pass this path through to Object Storage plugins via the `config` map using the `credentialsFile` key. (#3442, @zubron)\n * Fixed a bug where restic volumes would not be restored when using a namespace mapping. (#3475, @zubron)\n * Restore API group version by priority. Increase timeout to 3 minutes in DeploymentIsReady(...) function in the install package (#3133, @codegold79)\n * Add field and cli flag to associate a credential with a BSL on BSL create|set. (#3190, @carlisia)\n * Add colored output to `describe schedule/backup/restore` commands (#3275, @mike1808)\n * Add CAPI Cluster and ClusterResourceSets to default restore priorities so that the capi-controller-manager does not panic on restores. (#3446, @nrb)\n * Use label to select Velero deployment in plugin cmd (#3447, @codegold79)\n * feat: support setting BackupStorageLocation CA certificate via `velero backup-location set --cacert` (#3167, @jenting)\n * Add restic initContainer length check in pod volume restore to prevent restic plugin container disappear in runtime (#3198, @shellwedance)\n * Bump versions of external snapshotter and others in order to make `go get` to succeed (#3202, @georgettica)\n * Support fish shell completion (#3231, @jenting)\n * Change the logging level of PV deletion timeout from Debug to Warn (#3316, @MadhavJivrajani)\n * Set the BSL created at install time as the \"default\" (#3172, @carlisia)\n * Capitalize all help messages (#3209, @jenting)\n * Increased default Velero pod memory limit to 512Mi (#3234, @dsmithuchida)\n * Fixed an issue where the deletion of a backup would fail if the backup tarball couldn't be downloaded from object storage. Now the tarball is only downloaded if there are associated DeleteItemAction plugins and if downloading the tarball fails, the plugins are skipped. (#2993, @zubron)\n * feat: add delete sub-command for BSL (#3073, @jenting)\n * 🐛 BSLs with validation disabled should be validated at least once (#3084, @ashish-amarnath)\n * feat: support configures BackupStorageLocation custom resources to indicate which one is the default (#3092, @jenting)\n * Added \"--preserve-nodeports\" flag to preserve original nodePorts when restoring. (#3095, @yusufgungor)\n * Owner reference in backup when created from schedule (#3127, @matheusjuvelino)\n * issue: add flag to the schedule cmd to configure the `useOwnerReferencesInBackup` option #3176 (#3182, @matheusjuvelino)\n * cli: allow creating multiple instances of Velero across two different namespaces (#2886, @alaypatel07)\n * Feature: It is possible to change the timezone of the container by specifying in the manifest.. env: [TZ: Zone/Country], or in the Helm Chart.. configuration: {extraEnvVars: [TZ: 'Zone/Country']} (#2944, @mickkael)\n * Fix issue where bare `velero` command returned an error code. (#2947, @nrb)\n * Restore CRD Resource name to fix CRD wait functionality. (#2949, @sseago)\n * Fixed 'velero.io/change-pvc-node-selector' plugin to fetch configmap using label key \"velero.io/change-pvc-node-selector\" (#2970, @mynktl)\n * Compile with Go 1.15 (#2974, @gliptak)\n * Fix BSL controller to avoid invoking init() on all BSLs regardless of ValidationFrequency (#2992, @betta1)\n * Ensure that bound PVCs and PVs remain bound on restore. (#3007, @nrb)\n * Allows the restic-wait container to exist in any order in the pod being restored. Prints a warning message in the case where the restic-wait container isn't the first container in the list of initialization containers. (#3011, @doughepi)\n * Add warning to velero version cmd if the client and server versions mismatch. (#3024, @cvhariharan)\n * 🐛 Use namespace and name to match PVB to Pod restore (#3051, @ashish-amarnath)\n * Fixed various typos across codebase (#3057, @invidian)\n * 🐛 ItemAction plugins for unresolvable types should not be run for all types (#3059, @ashish-amarnath)\n * Basic end-to-end tests, generate data/backup/remove/restore/verify. Uses distributed data generator (#3060, @dsu-igeek)\n * Added GitHub Workflow running Codespell for spell checking (#3064, @invidian)\n * Pass annotations from schedule to backup it creates the same way it is done for labels. Add WithannotationsMap function to builder to be able to pass map instead of key/val list (#3067, @funkycode)\n * Add instructions to clone repository for examples in docs (#3074, @MadhavJivrajani)\n * 🏃‍♂️ update setup-kind github actions CI (#3085, @ashish-amarnath)\n * Modify wrong function name to correct one. (#3106, @shellwedance)\n" }, { "path": "changelogs/CHANGELOG-1.7.md", "content": "## v1.7.0\n### 2021-09-07\n\n### Download\nhttps://github.com/vmware-tanzu/velero/releases/tag/v1.7.0\n\n### Container Image\n`velero/velero:v1.7.0`\n\n### Documentation\nhttps://velero.io/docs/v1.7/\n\n### Upgrading\nhttps://velero.io/docs/v1.7/upgrade-to-1.7/\n\n### Highlights\n\n#### Distroless images\n\nThe Velero container images now use [distroless base images](https://github.com/GoogleContainerTools/distroless).\nUsing distroless images as the base ensures that only the packages and programs necessary for running Velero are included.\nUnrelated libraries and OS packages, that often contain security vulnerabilities, are now excluded.\nThis change reduces the size of both the server and restic restore helper image by approximately 62MB.\n\nAs the [distroless](https://github.com/GoogleContainerTools/distroless) images do not contain a shell, it will no longer be possible to exec into Velero containers using these images.\n\n#### New \"debug\" command\n\nThis release introduces the new `velero debug` command.\nThis command collects information about a Velero installation, such as pod logs and resources managed by Velero, in a tarball which can be provided to the Velero maintainer team to help diagnose issues.\n\n### All changes\n\n * Distinguish between different unnamed node ports when preserving (#4026, @sseago)\n * Validate namespace in Velero backup create command (#4057, @codegold79)\n * Empty the \"ClusterIPs\" along with \"ClusterIP\" when \"ClusterIP\" isn't \"None\" (#4101, @ywk253100)\n * Add a RestoreItemAction plugin (`velero.io/apiservice`) which skips the restore of any `APIService` which is managed by Kubernetes. These are identified using the `kube-aggregator.kubernetes.io/automanaged` label. (#4028, @zubron)\n * Change the base image to distroless (#4055, @ywk253100)\n * Updated the version of velero/velero-plugin-for-aws version from v1.2.0 to v1.2.1 (#4064, @kahirokunn)\n * Skip the backup and restore of DownwardAPI volumes when using restic. (#4076, @zubron)\n * Bump up Go to 1.16 (#3990, @reasonerjt)\n * Fix restic error when volume is emptyDir and Pod not running (#3993, @mahaupt)\n * Select the velero deployment with both label and container name (#3996, @ywk253100)\n * Wait for the namespace to be deleted before removing the CRDs during uninstall. This deprecates the `--wait` flag of the `uninstall` command (#4007, @ywk253100)\n * Use the cluster preferred CRD API version when polling for Velero CRD readiness. (#4015, @zubron)\n * Implement velero debug (#4022, @reasonerjt)\n * Skip the restore of volumes that originally came from a projected volume when using restic. (#3877, @zubron)\n * Run the E2E test with kind(provision various versions of k8s cluster) and MinIO on Github Action (#3912, @ywk253100)\n * Fix -install-velero flag for e2e tests (#3919, @jaidevmane)\n * Upgrade Velero ClusterRoleBinding to use v1 API (#3926, @jenting)\n * enable e2e tests to choose crd apiVersion (#3941, @sseago)\n * Fixing multipleNamespaceTest bug - Missing expect statement in test (#3983, @jaidevmane)\n * Add --client-page-size flag to server to allow chunking Kubernetes API LIST calls across multiple requests on large clusters (#3823, @dharmab)\n * Fix CR restore regression introduced in 1.6 restore progress. (#3845, @sseago)\n * Use region specified in the BackupStorageLocation spec when getting restic repo identifier. Originally fixed by @jala-dx in #3617. (#3857, @zubron)\n * skip backuping projected volume when using restic (#3866, @alaypatel07)\n * Install Kubernetes preferred CRDs API version (v1beta1/v1). (#3614, @jenting)\n * Add Label to BackupSpec so that labels can explicitly be provided to Schedule.Spec.Template.Metadata.Labels which will be reflected on the backups created. (#3641, @arush-sal)\n * Add PVC UID label to PodVolumeRestore (#3792, @sseago)\n * Support pulling plugin images by digest (#3803, @2uasimojo)\n * Added BackupPhaseUploading and BackupPhaseUploadingPartialFailure backup phases as part of Upload Progress Monitoring. (#3805, @dsmithuchida)\n\n Uploading (new)\n The \"Uploading\" phase signifies that the main part of the backup, including \n snapshotting has completed successfully and uploading is continuing. In \n the event of an error during uploading, the phase will change to \n UploadingPartialFailure. On success, the phase changes to Completed. The \n backup cannot be restored from when it is in the Uploading state.\n\n UploadingPartialFailure (new)\n The \"UploadingPartialFailure\" phase signifies that the main part of the backup,\n including snapshotting has completed, but there were partial failures either \n during the main part or during the uploading. The backup cannot be restored \n from when it is in the UploadingPartialFailure state.\n * 🐛 Fix plugin name derivation from image name (#3711, @ashish-amarnath)\n * ✨ ⚠️ Remove CSI volumesnapshot artifact deletion\n\nThis change requires https://github.com/vmware-tanzu/velero-plugin-for-csi/pull/86 for Velero to continue\ndeleting of CSI volumesnapshots when the corresponding backups are deleted. (#3734, @ashish-amarnath)\n * use unstructured to marshal selective fields for service restore action (#3789, @alaypatel07)\n" }, { "path": "changelogs/CHANGELOG-1.8.md", "content": "## v1.8.0\n### 2022-01-14\n\n### Download\nhttps://github.com/vmware-tanzu/velero/releases/tag/v1.8.0\n\n### Container Image\n`velero/velero:v1.8.0`\n\n### Documentation\nhttps://velero.io/docs/v1.8\n\n### Upgrading\nhttps://velero.io/docs/v1.8/upgrade-to-1.8/\n\n### Highlights\n\n#### Velero plugins now support handling volumes created by the CSI drivers of cloud providers\nVersions 1.4 of the Velero plugins for AWS, Azure and GCP now support snapshotting and restoring the persistent volumes provisioned by CSI driver via the APIs of the cloud providers. With this enhancement, users can backup and restore the persistent volumes on these cloud providers without using the Velero CSI plugin. The CSI plugin will remain beta and the feature flag `EnableCSI` will be disabled by default.\n\nFor the version of the plugins and the CSI drivers they support respectively please see the table:\n\n| Plugin | Version | CSI Driver |\n| --- | ----------- | ---------- |\n| velero-plugin-for-aws | v1.4.0 | ebs.csi.aws.com |\n| velero-plugin-for-microsoft-azure | v1.4.0 | disk.csi.azure.com |\n| velero-plugin-for-gcp | v1.4.0 | pd.csi.storage.gke.io |\n\n#### IPv6 dual stack support\nWe've verified the functionality of Velero on IPv6 dual stack by successfully running the E2E test on IPv6 dual stack environment.\n#### Refactor the controllers using Kubebuilder v3\nIn this release we continued our code modernization work, rewriting some controllers using Kubebuilder v3. This work is ongoing and we will continue to make progress in future releases.\n#### Enhancements to E2E test cases\nMore test cases have been added to the E2E test suite to improve the release health.\n#### Respect the cron setting of scheduled backup\nThe creation time is now taken into account to calculate the next run for scheduled backup.\n\n#### Deleting BSLs also cleans up related resources\n\nWhen a Backup Storage Location (BSL) is deleted, backup and Restic repository resources will also be deleted.\n\n#### Breaking changes\n\nStarting in v1.8, Velero will only support Kubernetes v1 CRD meaning that Velero v1.8+ will only run on Kubernetes v1.16+. Before upgrading, make sure you are running a supported Kubernetes version. For more information, see our [compatibility matrix](https://github.com/vmware-tanzu/velero#velero-compatibility-matrix).\n\n#### Upload Progress Monitoring and Item Snapshotter\nItem Snapshotter plugin API was merged. This will support both Upload Progress\nmonitoring and the planned Data Mover. Upload Progress monitoring PRs are\nin progress for 1.9.\n\n### All changes\n\n* E2E test on ssr object with controller namespace mix-ups (#4521, @mqiu)\n* Check whether the volume is provisioned by CSI driver or not by the annotation as well (#4513, @ywk253100)\n* Initialize the labels field of `velero backup-location create` option to avoid #4484 (#4491, @ywk253100)\n* Fix e2e 2500 namespaces scale test timeout problem (#4480, @mqiu)\n* Add backup deletion e2e test (#4401, @danfengliu)\n* Return the error when getting backup store in backup deletion controller (#4465, @reasonerjt)\n* Ignore the provided port is already allocated error when restoring the LoadBalancer service (#4462, @ywk253100)\n* Revert #4423 migrate backup sync controller to kubebuilder. (#4457, @jxun)\n* Add rbac and annotation test cases (#4455, @mqiu)\n* remove --crds-version in velero install command. (#4446, @jxun)\n* Upgrade e2e test vsphere plugin (#4440, @mqiu)\n* Fix e2e test failures for the inappropriate optimize of velero install (#4438, @mqiu)\n* Limit backup namespaces on test resource filtering cases (#4437, @mqiu)\n* Bump up Go to 1.17 (#4431, @reasonerjt)\n* Added ``-itemsnapshots.json.gz to the backup format. This file exists\n when item snapshots are taken and contains an array of volume.Itemsnapshots\n containing the information about the snapshots. This will not be used unless\n upload progress monitoring and item snapshots are enabled and an ItemSnapshot\n plugin is used to take snapshots.\n\nAlso added DownloadTargetKindBackupItemSnapshots for retrieving the signed URL to download only the ``-itemsnapshots.json.gz part of a backup for use by\n`velero backup describe`. (#4429, @dsmithuchida)\n* Migrate backup sync controller from code-generator to kubebuilder. (#4423, @jxun)\n* Added UploadProgressFeature flag to enable Upload Progress Monitoring and Item\n Snapshotters. (#4416, @dsmithuchida)\n* Added BackupWithResolvers and RestoreWithResolvers calls. Will eventually replace Backup and Restore methods.\n Adds ItemSnapshotters to Backup and Restore workflows. (#4410, @dsu)\n* Build for darwin-arm64 (#4409, @epk)\n* Add resource filtering test cases (#4404, @mqiu)\n* Fix the issue that the backup cannot be deleted after the application uninstalled (#4398, @ywk253100)\n* Add restoreactionitem plugin to handle admission webhook configurations (#4397, @reasonerjt)\n* Keep the annotation \"pv.kubernetes.io/provisioned-by\" when restoring PVs (#4391, @ywk253100)\n* Adjust structure of e2e test codes (#4386, @mqiu)\n* feat: migrate velero controller from kubebuilder v2 to v3\n From Velero v1.8, apiextesions.k8s.io/v1beta1 is no longer supported,\n which means only CRD of apiextensions.k8s.io/v1 is supported,\n and the supported Kubernetes version is updated to v1.16 and later. (#4382, @jxun)\n* Delete backups and Restic repos associated with deleted BSL(s) (#4377, @codegold79)\n* Add the key for GKE zone for AZ collection (#4376, @reasonerjt)\n* Fix statefulsets volumeClaimTemplates storageClassName when use Changing PV/PVC Storage Classes (#4375, @Box-Cube)\n* Fix snapshot e2e test issue of jsonpath (#4372, @danfengliu)\n* Modify the timestamp in the name of a backup generated from schedule to use UTC. (#4353, @jxun)\n* Read Availability zone from nodeAffinity requirements (#4350, @reasonerjt)\n* Use factory.Namespace() to replace hardcoded velero namespace (#4346, @half-life666)\n* Return the error if velero failed to detect S3 region for restic repo (#4343, @reasonerjt)\n* Add init log option for velero controller-runtime manager. (#4341, @jxun)\n* Ignore the `provided port is already allocated` error when restoring the `NodePort` service (#4336, @ywk253100)\n* Fixed an issue with the `backup-location create` command where the BSL Credential field would be set to an invalid empty SecretKeySelector when no credential details were provided. (#4322, @zubron)\n* fix buggy pager func (#4306, @alaypatel07)\n* Don't create a backup immediately after creating a schedule (#4281, @ywk253100)\n* Fix CVE-2020-29652 and CVE-2020-26160 (#4274, @ywk253100)\n* Refine tag-release.sh to align with change in release process (#4185, @reasonerjt)\n* Fix plugins incompatible issue in upgrade test (#4141, @danfengliu)\n* Verify group before treating resource as cohabiting (#4126, @sseago)\n* Added ItemSnapshotter plugin definition and plugin framework - addresses #3533.\n Part of the Upload Progress enhancement (#3533) (#4077, @dsmithuchida)\n* Add upgrade test in E2E test (#4058, @danfengliu)\n* Handle namespace mapping for PVs without snapshots on restore (#3708, @sseago)\n" }, { "path": "changelogs/CHANGELOG-1.9.md", "content": "## v1.9.0\n### 2022-06-13\n\n### Download\nhttps://github.com/vmware-tanzu/velero/releases/tag/v1.9.0\n\n### Container Image\n`velero/velero:v1.9.0`\n\n### Documentation\nhttps://velero.io/docs/v1.9/\n\n### Upgrading\nhttps://velero.io/docs/v1.9/upgrade-to-1.9/\n\n### Highlights\n\n#### Improvement to the CSI plugin\n- Bump up to the CSI volume snapshot v1 API\n- No VolumeSnapshot will be left in the source namespace of the workload\n- Report metrics for CSI snapshots\n\nMore improvements please refer to [CSI plugin improvement](https://github.com/vmware-tanzu/velero/issues?q=is%3Aissue+label%3A%22CSI+plugin+-+GA+-+phase1%22+is%3Aclosed)\n\nWith these improvements we'll provide official support for CSI snapshots on AKS/EKS clusters. (with CSI plugin v0.3.0)\n\n#### Refactor the controllers using Kubebuilder v3\nIn this release we continued our code modernization work, rewriting some controllers using Kubebuilder v3. This work is ongoing and we will continue to make progress in future releases.\n\n#### Optionally restore status on selected resources\nOptions are added to the CLI and Restore spec to control the group of resources whose status will be restored.\n\n#### ExistingResourcePolicy in the restore API\nUsers can choose to overwrite or patch the existing resources during restore by setting this policy.\n\n#### Upgrade integrated Restic version and add skip TLS validation in Restic command\nUpgrade integrated Restic version, which will resolve some of the CVEs, and support skip TLS validation in Restic backup/restore.\n\n#### Breaking changes\nWith bumping up the API to v1 in CSI plugin, the v0.3.0 CSI plugin will only work for Kubernetes v1.20+\n\n### All changes\n\n * restic: add full support for setting SecurityContext for restore init container from configMap. (#4084, @MatthieuFin)\n * Add metrics backup_items_total and backup_items_errors (#4296, @tobiasgiese)\n * Convert PodVolumebackup controller to the Kubebuilder framework (#4436, @fgold)\n * Skip not mounted volumes when backing up (#4497, @dkeven)\n * Update doc for v1.8 (#4517, @reasonerjt)\n * Fix bug to make the restic prune frequency configurable (#4518, @ywk253100)\n * Add E2E test of backups sync from BSL (#4545, @mqiu)\n * Fix: OrderedResources in Schedules (#4550, @dbrekau)\n * Skip volumes of non-running pods when backing up (#4584, @bynare)\n * E2E SSR test add retry mechanism and logs (#4591, @mqiu)\n * Add pushing image to GCR in github workflow to facilitate some environments that have rate limitation to docker hub, e.g. vSphere. (#4623, @jxun)\n * Add existingResourcePolicy to Restore API (#4628, @shubham-pampattiwar)\n * Fix E2E backup namespaces test (#4634, @qiuming-best)\n * Update image used by E2E test to gcr.io (#4639, @jxun)\n * Add multiple label selector support to Velero Backup and Restore APIs (#4650, @shubham-pampattiwar)\n * Convert Pod Volume Restore resource/controller to the Kubebuilder framework (#4655, @ywk253100)\n * Update --use-owner-references-in-backup description in velero command line. (#4660, @jxun)\n * Avoid overwritten hook's exec.container parameter when running pod command executor. (#4661, @jxun)\n * Support regional pv for GKE (#4680, @jxun)\n * Bypass the remap CRD version plugin when v1beta1 CRD is not supported (#4686, @reasonerjt)\n * Add GINKGO_SKIP to support skip specific case in e2e test. (#4692, @jxun)\n * Add --pod-labels flag to velero install (#4694, @j4m3s-s)\n * Enable coverage in test.sh and upload to codecov (#4704, @reasonerjt)\n * Mark the BSL as \"Unavailable\" when gets any error and add a new field \"Message\" to the status to record the error message (#4719, @ywk253100)\n * Support multiple skip option for E2E test (#4725, @jxun)\n * Add PriorityClass to the AdditionalItems of Backup's PodAction and Restore's PodAction plugin to backup and restore PriorityClass if it is used by a Pod. (#4740, @phuongatemc)\n * Insert all restore errors and warnings into restore log. (#4743, @sseago)\n * Refactor schedule controller with kubebuilder (#4748, @ywk253100)\n * Garbage collector now adds labels to backups that failed to delete for BSLNotFound, BSLCannotGet, BSLReadOnly reasons. (#4757, @kaovilai)\n * Skip podvolumerestore creation when restore excludes pv/pvc (#4769, @half-life666)\n * Add parameter for e2e test to support modify kibishii install path. (#4778, @jxun)\n * Ensure the restore hook applied to new namespace based on the mapping (#4779, @reasonerjt)\n * Add ability to restore status on selected resources (#4785, @RafaeLeal)\n * Do not take snapshot for PV to avoid duplicated snapshotting, when CSI feature is enabled. (#4797, @jxun)\n * Bump up to v1 API for CSI snapshot (#4800, @reasonerjt)\n * fix: delete empty backups (#4817, @yuvalman)\n * Add CSI VolumeSnapshot related metrics. (#4818, @jxun)\n * Fix default-backup-ttl not work (#4831, @qiuming-best)\n * Make the vsc created by backup sync controller deletable (#4832, @reasonerjt)\n * Make in-progress backup/restore as failed when doing the reconcile to avoid hanging in in-progress status (#4833, @ywk253100)\n * Use controller-gen to generate the deep copy methods for objects (#4838, @ywk253100)\n * Update integrated Restic version and add insecureSkipTLSVerify for Restic CLI. (#4839, @jxun)\n * Modify CSI VolumeSnapshot metric related code. (#4854, @jxun)\n * Refactor backup deletion controller based on kubebuilder (#4855, @reasonerjt)\n * Remove VolumeSnapshots created during backup when CSI feature is enabled. (#4858, @jxun)\n * Convert Restic Repository resource/controller to the Kubebuilder framework (#4859, @qiuming-best)\n * Add ClusterClasses to the restore priority list (#4866, @reasonerjt)\n * Cleanup the .velero folder after restic done (#4872, @big-appled)\n * Delete orphan CSI snapshots in backup sync controller (#4887, @reasonerjt)\n * Make waiting VolumeSnapshot to ready process parallel. (#4889, @jxun)\n * continue rather than return for non-matching restore action label (#4890, @sseago)\n * Make in-progress PVB/PVR as failed when restic controller restarts to avoid hanging backup/restore (#4893, @ywk253100)\n * Refactor BSL controller with periodical enqueue source (#4894, @jxun)\n * Make garbage collection for expired backups configurable (#4897, @ywk253100)\n * Bump up the version of distroless to base-debian11 (#4898, @ywk253100)\n * Add schedule ordered resources E2E test (#4913, @qiuming-best)\n * Make velero completion zsh command output can be used by `source` command. (#4914, @jxun)\n * Enhance the map flag to support parsing input value contains entry delimiters (#4920, @ywk253100)\n * Fix E2E test [Backups][Deletion][Restic] on GCP. (#4968, @jxun)\n * Disable status as sub resource in CRDs (#4972, @ywk253100)\n * Add more information for failing to get path or snapshot in restic backup and restore. (#4988, @jxun)\n" }, { "path": "changelogs/unreleased/9502-Joeavaikath", "content": "Support all glob wildcard characters in namespace validation\n" }, { "path": "changelogs/unreleased/9508-kaovilai", "content": "Fix VolumePolicy PVC phase condition filter for unbound PVCs (#9507)\n" }, { "path": "changelogs/unreleased/9532-Lyndon-Li", "content": "Fix issue #9343, include PV topology to data mover pod affinities" }, { "path": "changelogs/unreleased/9533-Lyndon-Li‎‎", "content": "Fix issue #9496, support customized host os" }, { "path": "changelogs/unreleased/9547-blackpiglet", "content": "If BIA return updateObj with SkipFromBackupAnnotation, treat it as skip the resource from backup." }, { "path": "changelogs/unreleased/9554-testsabirweb", "content": "Issue #9544: Add test coverage for S3 bucket name in MRAP ARN notation and fix bucket validation to accept ARN format" }, { "path": "changelogs/unreleased/9560-Lyndon-Li‎‎", "content": "Fix issue #9475, use node-selector instead of nodName for generic restore" }, { "path": "changelogs/unreleased/9561-Lyndon-Li‎‎", "content": "Fix issue #9460, flush buffer before data mover completes" }, { "path": "changelogs/unreleased/9570-H-M-Quang-Ngo", "content": "Add schedule_expected_interval_seconds metric for dynamic backup alerting thresholds (#9559)\n" }, { "path": "changelogs/unreleased/9574-blackpiglet", "content": "Add ephemeral storage limit and request support for data mover and maintenance job" }, { "path": "changelogs/unreleased/9581-shubham-pampattiwar", "content": "Fix DBR stuck when CSI snapshot no longer exists in cloud provider\n" }, { "path": "cmd/velero/velero.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage main\n\nimport (\n\t\"os\"\n\t\"path/filepath\"\n\n\t\"k8s.io/klog/v2\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/velero\"\n)\n\nfunc main() {\n\tdefer klog.Flush()\n\n\tbaseName := filepath.Base(os.Args[0])\n\n\terr := velero.NewCommand(baseName).Execute()\n\tcmd.CheckError(err)\n}\n" }, { "path": "cmd/velero-helper/velero-helper.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage main\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"time\"\n)\n\nconst (\n\t// workingModePause indicates it is for general purpose to hold the pod under running state\n\tworkingModePause = \"pause\"\n)\n\nfunc main() {\n\tif len(os.Args) < 2 {\n\t\tfmt.Fprintln(os.Stderr, \"ERROR: at least one argument must be provided, the working mode\")\n\t\tos.Exit(1)\n\t}\n\n\tswitch os.Args[1] {\n\tcase workingModePause:\n\t\ttime.Sleep(time.Duration(1<<63 - 1))\n\tdefault:\n\t\tfmt.Fprintln(os.Stderr, \"ERROR: wrong working mode provided\")\n\t\tos.Exit(1)\n\t}\n}\n" }, { "path": "cmd/velero-restore-helper/velero-restore-helper.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage main\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"time\"\n)\n\nfunc main() {\n\tif len(os.Args) != 2 {\n\t\tfmt.Fprintln(os.Stderr, \"ERROR: exactly one argument must be provided, the restore's UID\")\n\t\tos.Exit(1)\n\t}\n\n\tticker := time.NewTicker(time.Second)\n\tdefer ticker.Stop()\n\n\tfor {\n\t\t<-ticker.C\n\t\tif done() {\n\t\t\tfmt.Println(\"All restic restores are done\")\n\t\t\terr := removeFolder()\n\t\t\tif err != nil {\n\t\t\t\tfmt.Println(err)\n\t\t\t} else {\n\t\t\t\tfmt.Println(\"Done cleanup .velero folder\")\n\t\t\t}\n\t\t\treturn\n\t\t}\n\t}\n}\n\n// done returns true if for each directory under /restores, a file exists\n// within the .velero/ subdirectory whose name is equal to os.Args[1], or\n// false otherwise\nfunc done() bool {\n\tchildren, err := os.ReadDir(\"/restores\")\n\tif err != nil {\n\t\tfmt.Fprintf(os.Stderr, \"ERROR reading /restores directory: %s\\n\", err)\n\t\treturn false\n\t}\n\n\tfor _, child := range children {\n\t\tif !child.IsDir() {\n\t\t\tfmt.Printf(\"%s is not a directory, skipping.\\n\", child.Name())\n\t\t\tcontinue\n\t\t}\n\n\t\tdoneFile := filepath.Join(\"/restores\", child.Name(), \".velero\", os.Args[1])\n\n\t\tif _, err := os.Stat(doneFile); os.IsNotExist(err) {\n\t\t\tfmt.Printf(\"The filesystem restore done file %s is not found yet. Retry later.\\n\", doneFile)\n\t\t\treturn false\n\t\t} else if err != nil {\n\t\t\tfmt.Fprintf(os.Stderr, \"ERROR looking filesystem restore done file %s: %s\\n\", doneFile, err)\n\t\t\treturn false\n\t\t}\n\n\t\tfmt.Printf(\"Found the done file %s\\n\", doneFile)\n\t}\n\n\treturn true\n}\n\n// remove .velero folder\nfunc removeFolder() error {\n\tchildren, err := os.ReadDir(\"/restores\")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfor _, child := range children {\n\t\tif !child.IsDir() {\n\t\t\tfmt.Printf(\"%s is not a directory, skipping.\\n\", child.Name())\n\t\t\tcontinue\n\t\t}\n\n\t\tdonePath := filepath.Join(\"/restores\", child.Name(), \".velero\")\n\n\t\terr = os.RemoveAll(donePath)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tfmt.Printf(\"Deleted %s\", donePath)\n\t}\n\n\treturn nil\n}\n" }, { "path": "config/crd/v1/bases/velero.io_backuprepositories.yaml", "content": "---\napiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: v0.16.5\n name: backuprepositories.velero.io\nspec:\n group: velero.io\n names:\n kind: BackupRepository\n listKind: BackupRepositoryList\n plural: backuprepositories\n singular: backuprepository\n scope: Namespaced\n versions:\n - additionalPrinterColumns:\n - jsonPath: .metadata.creationTimestamp\n name: Age\n type: date\n - jsonPath: .spec.repositoryType\n name: Repository Type\n type: string\n name: v1\n schema:\n openAPIV3Schema:\n properties:\n apiVersion:\n description: |-\n APIVersion defines the versioned schema of this representation of an object.\n Servers should convert recognized schemas to the latest internal value, and\n may reject unrecognized values.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\n type: string\n kind:\n description: |-\n Kind is a string value representing the REST resource this object represents.\n Servers may infer this from the endpoint the client submits requests to.\n Cannot be updated.\n In CamelCase.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\n type: string\n metadata:\n type: object\n spec:\n description: BackupRepositorySpec is the specification for a BackupRepository.\n properties:\n backupStorageLocation:\n description: |-\n BackupStorageLocation is the name of the BackupStorageLocation\n that should contain this repository.\n type: string\n maintenanceFrequency:\n description: MaintenanceFrequency is how often maintenance should\n be run.\n type: string\n repositoryConfig:\n additionalProperties:\n type: string\n description: RepositoryConfig is for repository-specific configuration\n fields.\n nullable: true\n type: object\n repositoryType:\n description: RepositoryType indicates the type of the backend repository\n enum:\n - kopia\n - restic\n - \"\"\n type: string\n resticIdentifier:\n description: |-\n ResticIdentifier is the full restic-compatible string for identifying\n this repository. This field is only used when RepositoryType is \"restic\".\n type: string\n volumeNamespace:\n description: |-\n VolumeNamespace is the namespace this backup repository contains\n pod volume backups for.\n type: string\n required:\n - backupStorageLocation\n - maintenanceFrequency\n - volumeNamespace\n type: object\n status:\n description: BackupRepositoryStatus is the current status of a BackupRepository.\n properties:\n lastMaintenanceTime:\n description: LastMaintenanceTime is the last time repo maintenance\n succeeded.\n format: date-time\n nullable: true\n type: string\n message:\n description: Message is a message about the current status of the\n BackupRepository.\n type: string\n phase:\n description: Phase is the current state of the BackupRepository.\n enum:\n - New\n - Ready\n - NotReady\n type: string\n recentMaintenance:\n description: RecentMaintenance is status of the recent repo maintenance.\n items:\n properties:\n completeTimestamp:\n description: CompleteTimestamp is the completion time of the\n repo maintenance.\n format: date-time\n nullable: true\n type: string\n message:\n description: Message is a message about the current status of\n the repo maintenance.\n type: string\n result:\n description: Result is the result of the repo maintenance.\n enum:\n - Succeeded\n - Failed\n type: string\n startTimestamp:\n description: StartTimestamp is the start time of the repo maintenance.\n format: date-time\n nullable: true\n type: string\n type: object\n type: array\n type: object\n type: object\n served: true\n storage: true\n subresources: {}\n" }, { "path": "config/crd/v1/bases/velero.io_backups.yaml", "content": "---\napiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: v0.16.5\n name: backups.velero.io\nspec:\n group: velero.io\n names:\n kind: Backup\n listKind: BackupList\n plural: backups\n singular: backup\n scope: Namespaced\n versions:\n - name: v1\n schema:\n openAPIV3Schema:\n description: |-\n Backup is a Velero resource that represents the capture of Kubernetes\n cluster state at a point in time (API objects and associated volume state).\n properties:\n apiVersion:\n description: |-\n APIVersion defines the versioned schema of this representation of an object.\n Servers should convert recognized schemas to the latest internal value, and\n may reject unrecognized values.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\n type: string\n kind:\n description: |-\n Kind is a string value representing the REST resource this object represents.\n Servers may infer this from the endpoint the client submits requests to.\n Cannot be updated.\n In CamelCase.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\n type: string\n metadata:\n type: object\n spec:\n description: BackupSpec defines the specification for a Velero backup.\n properties:\n csiSnapshotTimeout:\n description: |-\n CSISnapshotTimeout specifies the time used to wait for CSI VolumeSnapshot status turns to\n ReadyToUse during creation, before returning error as timeout.\n The default value is 10 minute.\n type: string\n datamover:\n description: |-\n DataMover specifies the data mover to be used by the backup.\n If DataMover is \"\" or \"velero\", the built-in data mover will be used.\n type: string\n defaultVolumesToFsBackup:\n description: |-\n DefaultVolumesToFsBackup specifies whether pod volume file system backup should be used\n for all volumes by default.\n nullable: true\n type: boolean\n defaultVolumesToRestic:\n description: |-\n DefaultVolumesToRestic specifies whether restic should be used to take a\n backup of all pod volumes by default.\n\n Deprecated: this field is no longer used and will be removed entirely in future. Use DefaultVolumesToFsBackup instead.\n nullable: true\n type: boolean\n excludedClusterScopedResources:\n description: |-\n ExcludedClusterScopedResources is a slice of cluster-scoped\n resource type names to exclude from the backup.\n If set to \"*\", all cluster-scoped resource types are excluded.\n The default value is empty.\n items:\n type: string\n nullable: true\n type: array\n excludedNamespaceScopedResources:\n description: |-\n ExcludedNamespaceScopedResources is a slice of namespace-scoped\n resource type names to exclude from the backup.\n If set to \"*\", all namespace-scoped resource types are excluded.\n The default value is empty.\n items:\n type: string\n nullable: true\n type: array\n excludedNamespaces:\n description: |-\n ExcludedNamespaces contains a list of namespaces that are not\n included in the backup.\n items:\n type: string\n nullable: true\n type: array\n excludedResources:\n description: |-\n ExcludedResources is a slice of resource names that are not\n included in the backup.\n items:\n type: string\n nullable: true\n type: array\n hooks:\n description: Hooks represent custom behaviors that should be executed\n at different phases of the backup.\n properties:\n resources:\n description: Resources are hooks that should be executed when\n backing up individual instances of a resource.\n items:\n description: |-\n BackupResourceHookSpec defines one or more BackupResourceHooks that should be executed based on\n the rules defined for namespaces, resources, and label selector.\n properties:\n excludedNamespaces:\n description: ExcludedNamespaces specifies the namespaces\n to which this hook spec does not apply.\n items:\n type: string\n nullable: true\n type: array\n excludedResources:\n description: ExcludedResources specifies the resources to\n which this hook spec does not apply.\n items:\n type: string\n nullable: true\n type: array\n includedNamespaces:\n description: |-\n IncludedNamespaces specifies the namespaces to which this hook spec applies. If empty, it applies\n to all namespaces.\n items:\n type: string\n nullable: true\n type: array\n includedResources:\n description: |-\n IncludedResources specifies the resources to which this hook spec applies. If empty, it applies\n to all resources.\n items:\n type: string\n nullable: true\n type: array\n labelSelector:\n description: LabelSelector, if specified, filters the resources\n to which this hook spec applies.\n nullable: true\n properties:\n matchExpressions:\n description: matchExpressions is a list of label selector\n requirements. The requirements are ANDed.\n items:\n description: |-\n A label selector requirement is a selector that contains values, a key, and an operator that\n relates the key and values.\n properties:\n key:\n description: key is the label key that the selector\n applies to.\n type: string\n operator:\n description: |-\n operator represents a key's relationship to a set of values.\n Valid operators are In, NotIn, Exists and DoesNotExist.\n type: string\n values:\n description: |-\n values is an array of string values. If the operator is In or NotIn,\n the values array must be non-empty. If the operator is Exists or DoesNotExist,\n the values array must be empty. This array is replaced during a strategic\n merge patch.\n items:\n type: string\n type: array\n x-kubernetes-list-type: atomic\n required:\n - key\n - operator\n type: object\n type: array\n x-kubernetes-list-type: atomic\n matchLabels:\n additionalProperties:\n type: string\n description: |-\n matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels\n map is equivalent to an element of matchExpressions, whose key field is \"key\", the\n operator is \"In\", and the values array contains only \"value\". The requirements are ANDed.\n type: object\n type: object\n x-kubernetes-map-type: atomic\n name:\n description: Name is the name of this hook.\n type: string\n post:\n description: |-\n PostHooks is a list of BackupResourceHooks to execute after storing the item in the backup.\n These are executed after all \"additional items\" from item actions are processed.\n items:\n description: BackupResourceHook defines a hook for a resource.\n properties:\n exec:\n description: Exec defines an exec hook.\n properties:\n command:\n description: Command is the command and arguments\n to execute.\n items:\n type: string\n minItems: 1\n type: array\n container:\n description: |-\n Container is the container in the pod where the command should be executed. If not specified,\n the pod's first container is used.\n type: string\n onError:\n description: OnError specifies how Velero should\n behave if it encounters an error executing this\n hook.\n enum:\n - Continue\n - Fail\n type: string\n timeout:\n description: |-\n Timeout defines the maximum amount of time Velero should wait for the hook to complete before\n considering the execution a failure.\n type: string\n required:\n - command\n type: object\n required:\n - exec\n type: object\n type: array\n pre:\n description: |-\n PreHooks is a list of BackupResourceHooks to execute prior to storing the item in the backup.\n These are executed before any \"additional items\" from item actions are processed.\n items:\n description: BackupResourceHook defines a hook for a resource.\n properties:\n exec:\n description: Exec defines an exec hook.\n properties:\n command:\n description: Command is the command and arguments\n to execute.\n items:\n type: string\n minItems: 1\n type: array\n container:\n description: |-\n Container is the container in the pod where the command should be executed. If not specified,\n the pod's first container is used.\n type: string\n onError:\n description: OnError specifies how Velero should\n behave if it encounters an error executing this\n hook.\n enum:\n - Continue\n - Fail\n type: string\n timeout:\n description: |-\n Timeout defines the maximum amount of time Velero should wait for the hook to complete before\n considering the execution a failure.\n type: string\n required:\n - command\n type: object\n required:\n - exec\n type: object\n type: array\n required:\n - name\n type: object\n nullable: true\n type: array\n type: object\n includeClusterResources:\n description: |-\n IncludeClusterResources specifies whether cluster-scoped resources\n should be included for consideration in the backup.\n nullable: true\n type: boolean\n includedClusterScopedResources:\n description: |-\n IncludedClusterScopedResources is a slice of cluster-scoped\n resource type names to include in the backup.\n If set to \"*\", all cluster-scoped resource types are included.\n The default value is empty, which means only related\n cluster-scoped resources are included.\n items:\n type: string\n nullable: true\n type: array\n includedNamespaceScopedResources:\n description: |-\n IncludedNamespaceScopedResources is a slice of namespace-scoped\n resource type names to include in the backup.\n The default value is \"*\".\n items:\n type: string\n nullable: true\n type: array\n includedNamespaces:\n description: |-\n IncludedNamespaces is a slice of namespace names to include objects\n from. If empty, all namespaces are included.\n items:\n type: string\n nullable: true\n type: array\n includedResources:\n description: |-\n IncludedResources is a slice of resource names to include\n in the backup. If empty, all resources are included.\n items:\n type: string\n nullable: true\n type: array\n itemOperationTimeout:\n description: |-\n ItemOperationTimeout specifies the time used to wait for asynchronous BackupItemAction operations\n The default value is 4 hour.\n type: string\n labelSelector:\n description: |-\n LabelSelector is a metav1.LabelSelector to filter with\n when adding individual objects to the backup. If empty\n or nil, all objects are included. Optional.\n nullable: true\n properties:\n matchExpressions:\n description: matchExpressions is a list of label selector requirements.\n The requirements are ANDed.\n items:\n description: |-\n A label selector requirement is a selector that contains values, a key, and an operator that\n relates the key and values.\n properties:\n key:\n description: key is the label key that the selector applies\n to.\n type: string\n operator:\n description: |-\n operator represents a key's relationship to a set of values.\n Valid operators are In, NotIn, Exists and DoesNotExist.\n type: string\n values:\n description: |-\n values is an array of string values. If the operator is In or NotIn,\n the values array must be non-empty. If the operator is Exists or DoesNotExist,\n the values array must be empty. This array is replaced during a strategic\n merge patch.\n items:\n type: string\n type: array\n x-kubernetes-list-type: atomic\n required:\n - key\n - operator\n type: object\n type: array\n x-kubernetes-list-type: atomic\n matchLabels:\n additionalProperties:\n type: string\n description: |-\n matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels\n map is equivalent to an element of matchExpressions, whose key field is \"key\", the\n operator is \"In\", and the values array contains only \"value\". The requirements are ANDed.\n type: object\n type: object\n x-kubernetes-map-type: atomic\n metadata:\n properties:\n labels:\n additionalProperties:\n type: string\n type: object\n type: object\n orLabelSelectors:\n description: |-\n OrLabelSelectors is list of metav1.LabelSelector to filter with\n when adding individual objects to the backup. If multiple provided\n they will be joined by the OR operator. LabelSelector as well as\n OrLabelSelectors cannot co-exist in backup request, only one of them\n can be used.\n items:\n description: |-\n A label selector is a label query over a set of resources. The result of matchLabels and\n matchExpressions are ANDed. An empty label selector matches all objects. A null\n label selector matches no objects.\n properties:\n matchExpressions:\n description: matchExpressions is a list of label selector requirements.\n The requirements are ANDed.\n items:\n description: |-\n A label selector requirement is a selector that contains values, a key, and an operator that\n relates the key and values.\n properties:\n key:\n description: key is the label key that the selector applies\n to.\n type: string\n operator:\n description: |-\n operator represents a key's relationship to a set of values.\n Valid operators are In, NotIn, Exists and DoesNotExist.\n type: string\n values:\n description: |-\n values is an array of string values. If the operator is In or NotIn,\n the values array must be non-empty. If the operator is Exists or DoesNotExist,\n the values array must be empty. This array is replaced during a strategic\n merge patch.\n items:\n type: string\n type: array\n x-kubernetes-list-type: atomic\n required:\n - key\n - operator\n type: object\n type: array\n x-kubernetes-list-type: atomic\n matchLabels:\n additionalProperties:\n type: string\n description: |-\n matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels\n map is equivalent to an element of matchExpressions, whose key field is \"key\", the\n operator is \"In\", and the values array contains only \"value\". The requirements are ANDed.\n type: object\n type: object\n x-kubernetes-map-type: atomic\n nullable: true\n type: array\n orderedResources:\n additionalProperties:\n type: string\n description: |-\n OrderedResources specifies the backup order of resources of specific Kind.\n The map key is the resource name and value is a list of object names separated by commas.\n Each resource name has format \"namespace/objectname\". For cluster resources, simply use \"objectname\".\n nullable: true\n type: object\n resourcePolicy:\n description: ResourcePolicy specifies the referenced resource policies\n that backup should follow\n properties:\n apiGroup:\n description: |-\n APIGroup is the group for the resource being referenced.\n If APIGroup is not specified, the specified Kind must be in the core API group.\n For any other third-party types, APIGroup is required.\n type: string\n kind:\n description: Kind is the type of resource being referenced\n type: string\n name:\n description: Name is the name of resource being referenced\n type: string\n required:\n - kind\n - name\n type: object\n x-kubernetes-map-type: atomic\n snapshotMoveData:\n description: SnapshotMoveData specifies whether snapshot data should\n be moved\n nullable: true\n type: boolean\n snapshotVolumes:\n description: |-\n SnapshotVolumes specifies whether to take snapshots\n of any PV's referenced in the set of objects included\n in the Backup.\n nullable: true\n type: boolean\n storageLocation:\n description: StorageLocation is a string containing the name of a\n BackupStorageLocation where the backup should be stored.\n type: string\n ttl:\n description: |-\n TTL is a time.Duration-parseable string describing how long\n the Backup should be retained for.\n type: string\n uploaderConfig:\n description: UploaderConfig specifies the configuration for the uploader.\n nullable: true\n properties:\n parallelFilesUpload:\n description: ParallelFilesUpload is the number of files parallel\n uploads to perform when using the uploader.\n type: integer\n type: object\n volumeGroupSnapshotLabelKey:\n description: VolumeGroupSnapshotLabelKey specifies the label key to\n group PVCs under a VGS.\n type: string\n volumeSnapshotLocations:\n description: VolumeSnapshotLocations is a list containing names of\n VolumeSnapshotLocations associated with this backup.\n items:\n type: string\n type: array\n type: object\n status:\n description: BackupStatus captures the current status of a Velero backup.\n properties:\n backupItemOperationsAttempted:\n description: |-\n BackupItemOperationsAttempted is the total number of attempted\n async BackupItemAction operations for this backup.\n type: integer\n backupItemOperationsCompleted:\n description: |-\n BackupItemOperationsCompleted is the total number of successfully completed\n async BackupItemAction operations for this backup.\n type: integer\n backupItemOperationsFailed:\n description: |-\n BackupItemOperationsFailed is the total number of async\n BackupItemAction operations for this backup which ended with an error.\n type: integer\n completionTimestamp:\n description: |-\n CompletionTimestamp records the time a backup was completed.\n Completion time is recorded even on failed backups.\n Completion time is recorded before uploading the backup object.\n The server's time is used for CompletionTimestamps\n format: date-time\n nullable: true\n type: string\n csiVolumeSnapshotsAttempted:\n description: |-\n CSIVolumeSnapshotsAttempted is the total number of attempted\n CSI VolumeSnapshots for this backup.\n type: integer\n csiVolumeSnapshotsCompleted:\n description: |-\n CSIVolumeSnapshotsCompleted is the total number of successfully\n completed CSI VolumeSnapshots for this backup.\n type: integer\n errors:\n description: |-\n Errors is a count of all error messages that were generated during\n execution of the backup. The actual errors are in the backup's log\n file in object storage.\n type: integer\n expiration:\n description: Expiration is when this Backup is eligible for garbage-collection.\n format: date-time\n nullable: true\n type: string\n failureReason:\n description: FailureReason is an error that caused the entire backup\n to fail.\n type: string\n formatVersion:\n description: FormatVersion is the backup format version, including\n major, minor, and patch version.\n type: string\n hookStatus:\n description: HookStatus contains information about the status of the\n hooks.\n nullable: true\n properties:\n hooksAttempted:\n description: |-\n HooksAttempted is the total number of attempted hooks\n Specifically, HooksAttempted represents the number of hooks that failed to execute\n and the number of hooks that executed successfully.\n type: integer\n hooksFailed:\n description: HooksFailed is the total number of hooks which ended\n with an error\n type: integer\n type: object\n phase:\n description: Phase is the current state of the Backup.\n enum:\n - New\n - Queued\n - ReadyToStart\n - FailedValidation\n - InProgress\n - WaitingForPluginOperations\n - WaitingForPluginOperationsPartiallyFailed\n - Finalizing\n - FinalizingPartiallyFailed\n - Completed\n - PartiallyFailed\n - Failed\n - Deleting\n type: string\n progress:\n description: |-\n Progress contains information about the backup's execution progress. Note\n that this information is best-effort only -- if Velero fails to update it\n during a backup for any reason, it may be inaccurate/stale.\n nullable: true\n properties:\n itemsBackedUp:\n description: |-\n ItemsBackedUp is the number of items that have actually been written to the\n backup tarball so far.\n type: integer\n totalItems:\n description: |-\n TotalItems is the total number of items to be backed up. This number may change\n throughout the execution of the backup due to plugins that return additional related\n items to back up, the velero.io/exclude-from-backup label, and various other\n filters that happen as items are processed.\n type: integer\n type: object\n queuePosition:\n description: |-\n QueuePosition is the position of the backup in the queue.\n Only relevant when Phase is \"Queued\"\n type: integer\n startTimestamp:\n description: |-\n StartTimestamp records the time a backup was started.\n Separate from CreationTimestamp, since that value changes\n on restores.\n The server's time is used for StartTimestamps\n format: date-time\n nullable: true\n type: string\n validationErrors:\n description: |-\n ValidationErrors is a slice of all validation errors (if\n applicable).\n items:\n type: string\n nullable: true\n type: array\n version:\n description: |-\n Version is the backup format major version.\n Deprecated: Please see FormatVersion\n type: integer\n volumeSnapshotsAttempted:\n description: |-\n VolumeSnapshotsAttempted is the total number of attempted\n volume snapshots for this backup.\n type: integer\n volumeSnapshotsCompleted:\n description: |-\n VolumeSnapshotsCompleted is the total number of successfully\n completed volume snapshots for this backup.\n type: integer\n warnings:\n description: |-\n Warnings is a count of all warning messages that were generated during\n execution of the backup. The actual warnings are in the backup's log\n file in object storage.\n type: integer\n type: object\n type: object\n served: true\n storage: true\n" }, { "path": "config/crd/v1/bases/velero.io_backupstoragelocations.yaml", "content": "---\napiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: v0.16.5\n name: backupstoragelocations.velero.io\nspec:\n group: velero.io\n names:\n kind: BackupStorageLocation\n listKind: BackupStorageLocationList\n plural: backupstoragelocations\n shortNames:\n - bsl\n singular: backupstoragelocation\n scope: Namespaced\n versions:\n - additionalPrinterColumns:\n - description: Backup Storage Location status such as Available/Unavailable\n jsonPath: .status.phase\n name: Phase\n type: string\n - description: LastValidationTime is the last time the backup store location was\n validated\n jsonPath: .status.lastValidationTime\n name: Last Validated\n type: date\n - jsonPath: .metadata.creationTimestamp\n name: Age\n type: date\n - description: Default backup storage location\n jsonPath: .spec.default\n name: Default\n type: boolean\n name: v1\n schema:\n openAPIV3Schema:\n description: BackupStorageLocation is a location where Velero stores backup\n objects\n properties:\n apiVersion:\n description: |-\n APIVersion defines the versioned schema of this representation of an object.\n Servers should convert recognized schemas to the latest internal value, and\n may reject unrecognized values.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\n type: string\n kind:\n description: |-\n Kind is a string value representing the REST resource this object represents.\n Servers may infer this from the endpoint the client submits requests to.\n Cannot be updated.\n In CamelCase.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\n type: string\n metadata:\n type: object\n spec:\n description: BackupStorageLocationSpec defines the desired state of a\n Velero BackupStorageLocation\n properties:\n accessMode:\n description: AccessMode defines the permissions for the backup storage\n location.\n enum:\n - ReadOnly\n - ReadWrite\n type: string\n backupSyncPeriod:\n description: BackupSyncPeriod defines how frequently to sync backup\n API objects from object storage. A value of 0 disables sync.\n nullable: true\n type: string\n config:\n additionalProperties:\n type: string\n description: Config is for provider-specific configuration fields.\n type: object\n credential:\n description: Credential contains the credential information intended\n to be used with this location\n properties:\n key:\n description: The key of the secret to select from. Must be a\n valid secret key.\n type: string\n name:\n default: \"\"\n description: |-\n Name of the referent.\n This field is effectively required, but due to backwards compatibility is\n allowed to be empty. Instances of this type with an empty value here are\n almost certainly wrong.\n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\n type: string\n optional:\n description: Specify whether the Secret or its key must be defined\n type: boolean\n required:\n - key\n type: object\n x-kubernetes-map-type: atomic\n default:\n description: Default indicates this location is the default backup\n storage location.\n type: boolean\n objectStorage:\n description: ObjectStorageLocation specifies the settings necessary\n to connect to a provider's object storage.\n properties:\n bucket:\n description: Bucket is the bucket to use for object storage.\n type: string\n caCert:\n description: |-\n CACert defines a CA bundle to use when verifying TLS connections to the provider.\n Deprecated: Use CACertRef instead.\n format: byte\n type: string\n caCertRef:\n description: |-\n CACertRef is a reference to a Secret containing the CA certificate bundle to use\n when verifying TLS connections to the provider. The Secret must be in the same\n namespace as the BackupStorageLocation.\n properties:\n key:\n description: The key of the secret to select from. Must be\n a valid secret key.\n type: string\n name:\n default: \"\"\n description: |-\n Name of the referent.\n This field is effectively required, but due to backwards compatibility is\n allowed to be empty. Instances of this type with an empty value here are\n almost certainly wrong.\n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\n type: string\n optional:\n description: Specify whether the Secret or its key must be\n defined\n type: boolean\n required:\n - key\n type: object\n x-kubernetes-map-type: atomic\n prefix:\n description: Prefix is the path inside a bucket to use for Velero\n storage. Optional.\n type: string\n required:\n - bucket\n type: object\n provider:\n description: Provider is the provider of the backup storage.\n type: string\n validationFrequency:\n description: ValidationFrequency defines how frequently to validate\n the corresponding object storage. A value of 0 disables validation.\n nullable: true\n type: string\n required:\n - objectStorage\n - provider\n type: object\n status:\n description: BackupStorageLocationStatus defines the observed state of\n BackupStorageLocation\n properties:\n accessMode:\n description: |-\n AccessMode is an unused field.\n\n Deprecated: there is now an AccessMode field on the Spec and this field\n will be removed entirely as of v2.0.\n enum:\n - ReadOnly\n - ReadWrite\n type: string\n lastSyncedRevision:\n description: |-\n LastSyncedRevision is the value of the `metadata/revision` file in the backup\n storage location the last time the BSL's contents were synced into the cluster.\n\n Deprecated: this field is no longer updated or used for detecting changes to\n the location's contents and will be removed entirely in v2.0.\n type: string\n lastSyncedTime:\n description: |-\n LastSyncedTime is the last time the contents of the location were synced into\n the cluster.\n format: date-time\n nullable: true\n type: string\n lastValidationTime:\n description: |-\n LastValidationTime is the last time the backup store location was validated\n the cluster.\n format: date-time\n nullable: true\n type: string\n message:\n description: Message is a message about the backup storage location's\n status.\n type: string\n phase:\n description: Phase is the current state of the BackupStorageLocation.\n enum:\n - Available\n - Unavailable\n type: string\n type: object\n type: object\n served: true\n storage: true\n subresources: {}\n" }, { "path": "config/crd/v1/bases/velero.io_deletebackuprequests.yaml", "content": "---\napiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: v0.16.5\n name: deletebackuprequests.velero.io\nspec:\n group: velero.io\n names:\n kind: DeleteBackupRequest\n listKind: DeleteBackupRequestList\n plural: deletebackuprequests\n singular: deletebackuprequest\n scope: Namespaced\n versions:\n - additionalPrinterColumns:\n - description: The name of the backup to be deleted\n jsonPath: .spec.backupName\n name: BackupName\n type: string\n - description: The status of the deletion request\n jsonPath: .status.phase\n name: Status\n type: string\n name: v1\n schema:\n openAPIV3Schema:\n description: DeleteBackupRequest is a request to delete one or more backups.\n properties:\n apiVersion:\n description: |-\n APIVersion defines the versioned schema of this representation of an object.\n Servers should convert recognized schemas to the latest internal value, and\n may reject unrecognized values.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\n type: string\n kind:\n description: |-\n Kind is a string value representing the REST resource this object represents.\n Servers may infer this from the endpoint the client submits requests to.\n Cannot be updated.\n In CamelCase.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\n type: string\n metadata:\n type: object\n spec:\n description: DeleteBackupRequestSpec is the specification for which backups\n to delete.\n properties:\n backupName:\n type: string\n required:\n - backupName\n type: object\n status:\n description: DeleteBackupRequestStatus is the current status of a DeleteBackupRequest.\n properties:\n errors:\n description: Errors contains any errors that were encountered during\n the deletion process.\n items:\n type: string\n nullable: true\n type: array\n phase:\n description: Phase is the current state of the DeleteBackupRequest.\n enum:\n - New\n - InProgress\n - Processed\n type: string\n type: object\n type: object\n served: true\n storage: true\n subresources: {}\n" }, { "path": "config/crd/v1/bases/velero.io_downloadrequests.yaml", "content": "---\napiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: v0.16.5\n name: downloadrequests.velero.io\nspec:\n group: velero.io\n names:\n kind: DownloadRequest\n listKind: DownloadRequestList\n plural: downloadrequests\n singular: downloadrequest\n scope: Namespaced\n versions:\n - name: v1\n schema:\n openAPIV3Schema:\n description: |-\n DownloadRequest is a request to download an artifact from backup object storage, such as a backup\n log file.\n properties:\n apiVersion:\n description: |-\n APIVersion defines the versioned schema of this representation of an object.\n Servers should convert recognized schemas to the latest internal value, and\n may reject unrecognized values.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\n type: string\n kind:\n description: |-\n Kind is a string value representing the REST resource this object represents.\n Servers may infer this from the endpoint the client submits requests to.\n Cannot be updated.\n In CamelCase.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\n type: string\n metadata:\n type: object\n spec:\n description: DownloadRequestSpec is the specification for a download request.\n properties:\n target:\n description: Target is what to download (e.g. logs for a backup).\n properties:\n kind:\n description: Kind is the type of file to download.\n enum:\n - BackupLog\n - BackupContents\n - BackupVolumeSnapshots\n - BackupItemOperations\n - BackupResourceList\n - BackupResults\n - RestoreLog\n - RestoreResults\n - RestoreResourceList\n - RestoreItemOperations\n - CSIBackupVolumeSnapshots\n - CSIBackupVolumeSnapshotContents\n - BackupVolumeInfos\n - RestoreVolumeInfo\n type: string\n name:\n description: Name is the name of the Kubernetes resource with\n which the file is associated.\n type: string\n required:\n - kind\n - name\n type: object\n required:\n - target\n type: object\n status:\n description: DownloadRequestStatus is the current status of a DownloadRequest.\n properties:\n downloadURL:\n description: DownloadURL contains the pre-signed URL for the target\n file.\n type: string\n expiration:\n description: Expiration is when this DownloadRequest expires and can\n be deleted by the system.\n format: date-time\n nullable: true\n type: string\n phase:\n description: Phase is the current state of the DownloadRequest.\n enum:\n - New\n - Processed\n type: string\n type: object\n type: object\n served: true\n storage: true\n" }, { "path": "config/crd/v1/bases/velero.io_podvolumebackups.yaml", "content": "---\napiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: v0.16.5\n name: podvolumebackups.velero.io\nspec:\n group: velero.io\n names:\n kind: PodVolumeBackup\n listKind: PodVolumeBackupList\n plural: podvolumebackups\n singular: podvolumebackup\n scope: Namespaced\n versions:\n - additionalPrinterColumns:\n - description: PodVolumeBackup status such as New/InProgress\n jsonPath: .status.phase\n name: Status\n type: string\n - description: Time duration since this PodVolumeBackup was started\n jsonPath: .status.startTimestamp\n name: Started\n type: date\n - description: Completed bytes\n format: int64\n jsonPath: .status.progress.bytesDone\n name: Bytes Done\n type: integer\n - description: Total bytes\n format: int64\n jsonPath: .status.progress.totalBytes\n name: Total Bytes\n type: integer\n - description: Incremental bytes\n format: int64\n jsonPath: .status.incrementalBytes\n name: Incremental Bytes\n priority: 10\n type: integer\n - description: Name of the Backup Storage Location where this backup should be\n stored\n jsonPath: .spec.backupStorageLocation\n name: Storage Location\n type: string\n - description: Time duration since this PodVolumeBackup was created\n jsonPath: .metadata.creationTimestamp\n name: Age\n type: date\n - description: Name of the node where the PodVolumeBackup is processed\n jsonPath: .status.node\n name: Node\n type: string\n - description: The type of the uploader to handle data transfer\n jsonPath: .spec.uploaderType\n name: Uploader\n type: string\n name: v1\n schema:\n openAPIV3Schema:\n properties:\n apiVersion:\n description: |-\n APIVersion defines the versioned schema of this representation of an object.\n Servers should convert recognized schemas to the latest internal value, and\n may reject unrecognized values.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\n type: string\n kind:\n description: |-\n Kind is a string value representing the REST resource this object represents.\n Servers may infer this from the endpoint the client submits requests to.\n Cannot be updated.\n In CamelCase.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\n type: string\n metadata:\n type: object\n spec:\n description: PodVolumeBackupSpec is the specification for a PodVolumeBackup.\n properties:\n backupStorageLocation:\n description: |-\n BackupStorageLocation is the name of the backup storage location\n where the backup repository is stored.\n type: string\n cancel:\n description: |-\n Cancel indicates request to cancel the ongoing PodVolumeBackup. It can be set\n when the PodVolumeBackup is in InProgress phase\n type: boolean\n node:\n description: Node is the name of the node that the Pod is running\n on.\n type: string\n pod:\n description: Pod is a reference to the pod containing the volume to\n be backed up.\n properties:\n apiVersion:\n description: API version of the referent.\n type: string\n fieldPath:\n description: |-\n If referring to a piece of an object instead of an entire object, this string\n should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2].\n For example, if the object reference is to a container within a pod, this would take on a value like:\n \"spec.containers{name}\" (where \"name\" refers to the name of the container that triggered\n the event) or if no container name is specified \"spec.containers[2]\" (container with\n index 2 in this pod). This syntax is chosen only to have some well-defined way of\n referencing a part of an object.\n type: string\n kind:\n description: |-\n Kind of the referent.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\n type: string\n name:\n description: |-\n Name of the referent.\n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\n type: string\n namespace:\n description: |-\n Namespace of the referent.\n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/\n type: string\n resourceVersion:\n description: |-\n Specific resourceVersion to which this reference is made, if any.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency\n type: string\n uid:\n description: |-\n UID of the referent.\n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids\n type: string\n type: object\n x-kubernetes-map-type: atomic\n repoIdentifier:\n description: RepoIdentifier is the backup repository identifier.\n type: string\n tags:\n additionalProperties:\n type: string\n description: |-\n Tags are a map of key-value pairs that should be applied to the\n volume backup as tags.\n type: object\n uploaderSettings:\n additionalProperties:\n type: string\n description: |-\n UploaderSettings are a map of key-value pairs that should be applied to the\n uploader configuration.\n nullable: true\n type: object\n uploaderType:\n description: UploaderType is the type of the uploader to handle the\n data transfer.\n enum:\n - kopia\n - restic\n - \"\"\n type: string\n volume:\n description: |-\n Volume is the name of the volume within the Pod to be backed\n up.\n type: string\n required:\n - backupStorageLocation\n - node\n - pod\n - repoIdentifier\n - volume\n type: object\n status:\n description: PodVolumeBackupStatus is the current status of a PodVolumeBackup.\n properties:\n acceptedTimestamp:\n description: |-\n AcceptedTimestamp records the time the pod volume backup is to be prepared.\n The server's time is used for AcceptedTimestamp\n format: date-time\n nullable: true\n type: string\n completionTimestamp:\n description: |-\n CompletionTimestamp records the time a backup was completed.\n Completion time is recorded even on failed backups.\n Completion time is recorded before uploading the backup object.\n The server's time is used for CompletionTimestamps\n format: date-time\n nullable: true\n type: string\n incrementalBytes:\n description: IncrementalBytes holds the number of bytes new or changed\n since the last backup\n format: int64\n type: integer\n message:\n description: Message is a message about the pod volume backup's status.\n type: string\n path:\n description: Path is the full path within the controller pod being\n backed up.\n type: string\n phase:\n description: Phase is the current state of the PodVolumeBackup.\n enum:\n - New\n - Accepted\n - Prepared\n - InProgress\n - Canceling\n - Canceled\n - Completed\n - Failed\n type: string\n progress:\n description: |-\n Progress holds the total number of bytes of the volume and the current\n number of backed up bytes. This can be used to display progress information\n about the backup operation.\n properties:\n bytesDone:\n format: int64\n type: integer\n totalBytes:\n format: int64\n type: integer\n type: object\n snapshotID:\n description: SnapshotID is the identifier for the snapshot of the\n pod volume.\n type: string\n startTimestamp:\n description: |-\n StartTimestamp records the time a backup was started.\n Separate from CreationTimestamp, since that value changes\n on restores.\n The server's time is used for StartTimestamps\n format: date-time\n nullable: true\n type: string\n type: object\n type: object\n served: true\n storage: true\n subresources: {}\n" }, { "path": "config/crd/v1/bases/velero.io_podvolumerestores.yaml", "content": "---\napiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: v0.16.5\n name: podvolumerestores.velero.io\nspec:\n group: velero.io\n names:\n kind: PodVolumeRestore\n listKind: PodVolumeRestoreList\n plural: podvolumerestores\n singular: podvolumerestore\n scope: Namespaced\n versions:\n - additionalPrinterColumns:\n - description: PodVolumeRestore status such as New/InProgress\n jsonPath: .status.phase\n name: Status\n type: string\n - description: Time duration since this PodVolumeRestore was started\n jsonPath: .status.startTimestamp\n name: Started\n type: date\n - description: Completed bytes\n format: int64\n jsonPath: .status.progress.bytesDone\n name: Bytes Done\n type: integer\n - description: Total bytes\n format: int64\n jsonPath: .status.progress.totalBytes\n name: Total Bytes\n type: integer\n - description: Name of the Backup Storage Location where the backup data is stored\n jsonPath: .spec.backupStorageLocation\n name: Storage Location\n type: string\n - description: Time duration since this PodVolumeRestore was created\n jsonPath: .metadata.creationTimestamp\n name: Age\n type: date\n - description: Name of the node where the PodVolumeRestore is processed\n jsonPath: .status.node\n name: Node\n type: string\n - description: The type of the uploader to handle data transfer\n jsonPath: .spec.uploaderType\n name: Uploader Type\n type: string\n name: v1\n schema:\n openAPIV3Schema:\n properties:\n apiVersion:\n description: |-\n APIVersion defines the versioned schema of this representation of an object.\n Servers should convert recognized schemas to the latest internal value, and\n may reject unrecognized values.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\n type: string\n kind:\n description: |-\n Kind is a string value representing the REST resource this object represents.\n Servers may infer this from the endpoint the client submits requests to.\n Cannot be updated.\n In CamelCase.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\n type: string\n metadata:\n type: object\n spec:\n description: PodVolumeRestoreSpec is the specification for a PodVolumeRestore.\n properties:\n backupStorageLocation:\n description: |-\n BackupStorageLocation is the name of the backup storage location\n where the backup repository is stored.\n type: string\n cancel:\n description: |-\n Cancel indicates request to cancel the ongoing PodVolumeRestore. It can be set\n when the PodVolumeRestore is in InProgress phase\n type: boolean\n pod:\n description: Pod is a reference to the pod containing the volume to\n be restored.\n properties:\n apiVersion:\n description: API version of the referent.\n type: string\n fieldPath:\n description: |-\n If referring to a piece of an object instead of an entire object, this string\n should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2].\n For example, if the object reference is to a container within a pod, this would take on a value like:\n \"spec.containers{name}\" (where \"name\" refers to the name of the container that triggered\n the event) or if no container name is specified \"spec.containers[2]\" (container with\n index 2 in this pod). This syntax is chosen only to have some well-defined way of\n referencing a part of an object.\n type: string\n kind:\n description: |-\n Kind of the referent.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\n type: string\n name:\n description: |-\n Name of the referent.\n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\n type: string\n namespace:\n description: |-\n Namespace of the referent.\n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/\n type: string\n resourceVersion:\n description: |-\n Specific resourceVersion to which this reference is made, if any.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency\n type: string\n uid:\n description: |-\n UID of the referent.\n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids\n type: string\n type: object\n x-kubernetes-map-type: atomic\n repoIdentifier:\n description: RepoIdentifier is the backup repository identifier.\n type: string\n snapshotID:\n description: SnapshotID is the ID of the volume snapshot to be restored.\n type: string\n snapshotSize:\n description: SnapshotSize is the logical size in Bytes of the snapshot.\n format: int64\n type: integer\n sourceNamespace:\n description: SourceNamespace is the original namespace for namaspace\n mapping.\n type: string\n uploaderSettings:\n additionalProperties:\n type: string\n description: |-\n UploaderSettings are a map of key-value pairs that should be applied to the\n uploader configuration.\n nullable: true\n type: object\n uploaderType:\n description: UploaderType is the type of the uploader to handle the\n data transfer.\n enum:\n - kopia\n - restic\n - \"\"\n type: string\n volume:\n description: Volume is the name of the volume within the Pod to be\n restored.\n type: string\n required:\n - backupStorageLocation\n - pod\n - repoIdentifier\n - snapshotID\n - sourceNamespace\n - volume\n type: object\n status:\n description: PodVolumeRestoreStatus is the current status of a PodVolumeRestore.\n properties:\n acceptedTimestamp:\n description: |-\n AcceptedTimestamp records the time the pod volume restore is to be prepared.\n The server's time is used for AcceptedTimestamp\n format: date-time\n nullable: true\n type: string\n completionTimestamp:\n description: |-\n CompletionTimestamp records the time a restore was completed.\n Completion time is recorded even on failed restores.\n The server's time is used for CompletionTimestamps\n format: date-time\n nullable: true\n type: string\n message:\n description: Message is a message about the pod volume restore's status.\n type: string\n node:\n description: Node is name of the node where the pod volume restore\n is processed.\n type: string\n phase:\n description: Phase is the current state of the PodVolumeRestore.\n enum:\n - New\n - Accepted\n - Prepared\n - InProgress\n - Canceling\n - Canceled\n - Completed\n - Failed\n type: string\n progress:\n description: |-\n Progress holds the total number of bytes of the snapshot and the current\n number of restored bytes. This can be used to display progress information\n about the restore operation.\n properties:\n bytesDone:\n format: int64\n type: integer\n totalBytes:\n format: int64\n type: integer\n type: object\n startTimestamp:\n description: |-\n StartTimestamp records the time a restore was started.\n The server's time is used for StartTimestamps\n format: date-time\n nullable: true\n type: string\n type: object\n type: object\n served: true\n storage: true\n subresources: {}\n" }, { "path": "config/crd/v1/bases/velero.io_restores.yaml", "content": "---\napiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: v0.16.5\n name: restores.velero.io\nspec:\n group: velero.io\n names:\n kind: Restore\n listKind: RestoreList\n plural: restores\n singular: restore\n scope: Namespaced\n versions:\n - name: v1\n schema:\n openAPIV3Schema:\n description: |-\n Restore is a Velero resource that represents the application of\n resources from a Velero backup to a target Kubernetes cluster.\n properties:\n apiVersion:\n description: |-\n APIVersion defines the versioned schema of this representation of an object.\n Servers should convert recognized schemas to the latest internal value, and\n may reject unrecognized values.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\n type: string\n kind:\n description: |-\n Kind is a string value representing the REST resource this object represents.\n Servers may infer this from the endpoint the client submits requests to.\n Cannot be updated.\n In CamelCase.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\n type: string\n metadata:\n type: object\n spec:\n description: RestoreSpec defines the specification for a Velero restore.\n properties:\n backupName:\n description: |-\n BackupName is the unique name of the Velero backup to restore\n from.\n type: string\n excludedNamespaces:\n description: |-\n ExcludedNamespaces contains a list of namespaces that are not\n included in the restore.\n items:\n type: string\n nullable: true\n type: array\n excludedResources:\n description: |-\n ExcludedResources is a slice of resource names that are not\n included in the restore.\n items:\n type: string\n nullable: true\n type: array\n existingResourcePolicy:\n description: ExistingResourcePolicy specifies the restore behavior\n for the Kubernetes resource to be restored\n nullable: true\n type: string\n hooks:\n description: Hooks represent custom behaviors that should be executed\n during or post restore.\n properties:\n resources:\n items:\n description: |-\n RestoreResourceHookSpec defines one or more RestoreResrouceHooks that should be executed based on\n the rules defined for namespaces, resources, and label selector.\n properties:\n excludedNamespaces:\n description: ExcludedNamespaces specifies the namespaces\n to which this hook spec does not apply.\n items:\n type: string\n nullable: true\n type: array\n excludedResources:\n description: ExcludedResources specifies the resources to\n which this hook spec does not apply.\n items:\n type: string\n nullable: true\n type: array\n includedNamespaces:\n description: |-\n IncludedNamespaces specifies the namespaces to which this hook spec applies. If empty, it applies\n to all namespaces.\n items:\n type: string\n nullable: true\n type: array\n includedResources:\n description: |-\n IncludedResources specifies the resources to which this hook spec applies. If empty, it applies\n to all resources.\n items:\n type: string\n nullable: true\n type: array\n labelSelector:\n description: LabelSelector, if specified, filters the resources\n to which this hook spec applies.\n nullable: true\n properties:\n matchExpressions:\n description: matchExpressions is a list of label selector\n requirements. The requirements are ANDed.\n items:\n description: |-\n A label selector requirement is a selector that contains values, a key, and an operator that\n relates the key and values.\n properties:\n key:\n description: key is the label key that the selector\n applies to.\n type: string\n operator:\n description: |-\n operator represents a key's relationship to a set of values.\n Valid operators are In, NotIn, Exists and DoesNotExist.\n type: string\n values:\n description: |-\n values is an array of string values. If the operator is In or NotIn,\n the values array must be non-empty. If the operator is Exists or DoesNotExist,\n the values array must be empty. This array is replaced during a strategic\n merge patch.\n items:\n type: string\n type: array\n x-kubernetes-list-type: atomic\n required:\n - key\n - operator\n type: object\n type: array\n x-kubernetes-list-type: atomic\n matchLabels:\n additionalProperties:\n type: string\n description: |-\n matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels\n map is equivalent to an element of matchExpressions, whose key field is \"key\", the\n operator is \"In\", and the values array contains only \"value\". The requirements are ANDed.\n type: object\n type: object\n x-kubernetes-map-type: atomic\n name:\n description: Name is the name of this hook.\n type: string\n postHooks:\n description: PostHooks is a list of RestoreResourceHooks\n to execute during and after restoring a resource.\n items:\n description: RestoreResourceHook defines a restore hook\n for a resource.\n properties:\n exec:\n description: Exec defines an exec restore hook.\n properties:\n command:\n description: Command is the command and arguments\n to execute from within a container after a pod\n has been restored.\n items:\n type: string\n minItems: 1\n type: array\n container:\n description: |-\n Container is the container in the pod where the command should be executed. If not specified,\n the pod's first container is used.\n type: string\n execTimeout:\n description: |-\n ExecTimeout defines the maximum amount of time Velero should wait for the hook to complete before\n considering the execution a failure.\n type: string\n onError:\n description: OnError specifies how Velero should\n behave if it encounters an error executing this\n hook.\n enum:\n - Continue\n - Fail\n type: string\n waitForReady:\n description: WaitForReady ensures command will\n be launched when container is Ready instead\n of Running.\n nullable: true\n type: boolean\n waitTimeout:\n description: |-\n WaitTimeout defines the maximum amount of time Velero should wait for the container to be Ready\n before attempting to run the command.\n type: string\n required:\n - command\n type: object\n init:\n description: Init defines an init restore hook.\n properties:\n initContainers:\n description: InitContainers is list of init containers\n to be added to a pod during its restore.\n items:\n type: object\n x-kubernetes-preserve-unknown-fields: true\n type: array\n x-kubernetes-preserve-unknown-fields: true\n timeout:\n description: Timeout defines the maximum amount\n of time Velero should wait for the initContainers\n to complete.\n type: string\n type: object\n type: object\n type: array\n required:\n - name\n type: object\n type: array\n type: object\n includeClusterResources:\n description: |-\n IncludeClusterResources specifies whether cluster-scoped resources\n should be included for consideration in the restore. If null, defaults\n to true.\n nullable: true\n type: boolean\n includedNamespaces:\n description: |-\n IncludedNamespaces is a slice of namespace names to include objects\n from. If empty, all namespaces are included.\n items:\n type: string\n nullable: true\n type: array\n includedResources:\n description: |-\n IncludedResources is a slice of resource names to include\n in the restore. If empty, all resources in the backup are included.\n items:\n type: string\n nullable: true\n type: array\n itemOperationTimeout:\n description: |-\n ItemOperationTimeout specifies the time used to wait for RestoreItemAction operations\n The default value is 4 hour.\n type: string\n labelSelector:\n description: |-\n LabelSelector is a metav1.LabelSelector to filter with\n when restoring individual objects from the backup. If empty\n or nil, all objects are included. Optional.\n nullable: true\n properties:\n matchExpressions:\n description: matchExpressions is a list of label selector requirements.\n The requirements are ANDed.\n items:\n description: |-\n A label selector requirement is a selector that contains values, a key, and an operator that\n relates the key and values.\n properties:\n key:\n description: key is the label key that the selector applies\n to.\n type: string\n operator:\n description: |-\n operator represents a key's relationship to a set of values.\n Valid operators are In, NotIn, Exists and DoesNotExist.\n type: string\n values:\n description: |-\n values is an array of string values. If the operator is In or NotIn,\n the values array must be non-empty. If the operator is Exists or DoesNotExist,\n the values array must be empty. This array is replaced during a strategic\n merge patch.\n items:\n type: string\n type: array\n x-kubernetes-list-type: atomic\n required:\n - key\n - operator\n type: object\n type: array\n x-kubernetes-list-type: atomic\n matchLabels:\n additionalProperties:\n type: string\n description: |-\n matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels\n map is equivalent to an element of matchExpressions, whose key field is \"key\", the\n operator is \"In\", and the values array contains only \"value\". The requirements are ANDed.\n type: object\n type: object\n x-kubernetes-map-type: atomic\n namespaceMapping:\n additionalProperties:\n type: string\n description: |-\n NamespaceMapping is a map of source namespace names\n to target namespace names to restore into. Any source\n namespaces not included in the map will be restored into\n namespaces of the same name.\n type: object\n orLabelSelectors:\n description: |-\n OrLabelSelectors is list of metav1.LabelSelector to filter with\n when restoring individual objects from the backup. If multiple provided\n they will be joined by the OR operator. LabelSelector as well as\n OrLabelSelectors cannot co-exist in restore request, only one of them\n can be used\n items:\n description: |-\n A label selector is a label query over a set of resources. The result of matchLabels and\n matchExpressions are ANDed. An empty label selector matches all objects. A null\n label selector matches no objects.\n properties:\n matchExpressions:\n description: matchExpressions is a list of label selector requirements.\n The requirements are ANDed.\n items:\n description: |-\n A label selector requirement is a selector that contains values, a key, and an operator that\n relates the key and values.\n properties:\n key:\n description: key is the label key that the selector applies\n to.\n type: string\n operator:\n description: |-\n operator represents a key's relationship to a set of values.\n Valid operators are In, NotIn, Exists and DoesNotExist.\n type: string\n values:\n description: |-\n values is an array of string values. If the operator is In or NotIn,\n the values array must be non-empty. If the operator is Exists or DoesNotExist,\n the values array must be empty. This array is replaced during a strategic\n merge patch.\n items:\n type: string\n type: array\n x-kubernetes-list-type: atomic\n required:\n - key\n - operator\n type: object\n type: array\n x-kubernetes-list-type: atomic\n matchLabels:\n additionalProperties:\n type: string\n description: |-\n matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels\n map is equivalent to an element of matchExpressions, whose key field is \"key\", the\n operator is \"In\", and the values array contains only \"value\". The requirements are ANDed.\n type: object\n type: object\n x-kubernetes-map-type: atomic\n nullable: true\n type: array\n preserveNodePorts:\n description: PreserveNodePorts specifies whether to restore old nodePorts\n from backup.\n nullable: true\n type: boolean\n resourceModifier:\n description: ResourceModifier specifies the reference to JSON resource\n patches that should be applied to resources before restoration.\n nullable: true\n properties:\n apiGroup:\n description: |-\n APIGroup is the group for the resource being referenced.\n If APIGroup is not specified, the specified Kind must be in the core API group.\n For any other third-party types, APIGroup is required.\n type: string\n kind:\n description: Kind is the type of resource being referenced\n type: string\n name:\n description: Name is the name of resource being referenced\n type: string\n required:\n - kind\n - name\n type: object\n x-kubernetes-map-type: atomic\n restorePVs:\n description: |-\n RestorePVs specifies whether to restore all included\n PVs from snapshot\n nullable: true\n type: boolean\n restoreStatus:\n description: |-\n RestoreStatus specifies which resources we should restore the status\n field. If nil, no objects are included. Optional.\n nullable: true\n properties:\n excludedResources:\n description: ExcludedResources specifies the resources to which\n will not restore the status.\n items:\n type: string\n nullable: true\n type: array\n includedResources:\n description: |-\n IncludedResources specifies the resources to which will restore the status.\n If empty, it applies to all resources.\n items:\n type: string\n nullable: true\n type: array\n type: object\n scheduleName:\n description: |-\n ScheduleName is the unique name of the Velero schedule to restore\n from. If specified, and BackupName is empty, Velero will restore\n from the most recent successful backup created from this schedule.\n type: string\n uploaderConfig:\n description: UploaderConfig specifies the configuration for the restore.\n nullable: true\n properties:\n parallelFilesDownload:\n description: ParallelFilesDownload is the concurrency number setting\n for restore.\n type: integer\n writeSparseFiles:\n description: WriteSparseFiles is a flag to indicate whether write\n files sparsely or not.\n nullable: true\n type: boolean\n type: object\n type: object\n status:\n description: RestoreStatus captures the current status of a Velero restore\n properties:\n completionTimestamp:\n description: |-\n CompletionTimestamp records the time the restore operation was completed.\n Completion time is recorded even on failed restore.\n The server's time is used for StartTimestamps\n format: date-time\n nullable: true\n type: string\n errors:\n description: |-\n Errors is a count of all error messages that were generated during\n execution of the restore. The actual errors are stored in object storage.\n type: integer\n failureReason:\n description: FailureReason is an error that caused the entire restore\n to fail.\n type: string\n hookStatus:\n description: HookStatus contains information about the status of the\n hooks.\n nullable: true\n properties:\n hooksAttempted:\n description: |-\n HooksAttempted is the total number of attempted hooks\n Specifically, HooksAttempted represents the number of hooks that failed to execute\n and the number of hooks that executed successfully.\n type: integer\n hooksFailed:\n description: HooksFailed is the total number of hooks which ended\n with an error\n type: integer\n type: object\n phase:\n description: Phase is the current state of the Restore\n enum:\n - New\n - FailedValidation\n - InProgress\n - WaitingForPluginOperations\n - WaitingForPluginOperationsPartiallyFailed\n - Completed\n - PartiallyFailed\n - Failed\n - Finalizing\n - FinalizingPartiallyFailed\n type: string\n progress:\n description: |-\n Progress contains information about the restore's execution progress. Note\n that this information is best-effort only -- if Velero fails to update it\n during a restore for any reason, it may be inaccurate/stale.\n nullable: true\n properties:\n itemsRestored:\n description: ItemsRestored is the number of items that have actually\n been restored so far\n type: integer\n totalItems:\n description: |-\n TotalItems is the total number of items to be restored. This number may change\n throughout the execution of the restore due to plugins that return additional related\n items to restore\n type: integer\n type: object\n restoreItemOperationsAttempted:\n description: |-\n RestoreItemOperationsAttempted is the total number of attempted\n async RestoreItemAction operations for this restore.\n type: integer\n restoreItemOperationsCompleted:\n description: |-\n RestoreItemOperationsCompleted is the total number of successfully completed\n async RestoreItemAction operations for this restore.\n type: integer\n restoreItemOperationsFailed:\n description: |-\n RestoreItemOperationsFailed is the total number of async\n RestoreItemAction operations for this restore which ended with an error.\n type: integer\n startTimestamp:\n description: |-\n StartTimestamp records the time the restore operation was started.\n The server's time is used for StartTimestamps\n format: date-time\n nullable: true\n type: string\n validationErrors:\n description: |-\n ValidationErrors is a slice of all validation errors (if\n applicable)\n items:\n type: string\n nullable: true\n type: array\n warnings:\n description: |-\n Warnings is a count of all warning messages that were generated during\n execution of the restore. The actual warnings are stored in object storage.\n type: integer\n type: object\n type: object\n served: true\n storage: true\n" }, { "path": "config/crd/v1/bases/velero.io_schedules.yaml", "content": "---\napiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: v0.16.5\n name: schedules.velero.io\nspec:\n group: velero.io\n names:\n kind: Schedule\n listKind: ScheduleList\n plural: schedules\n singular: schedule\n scope: Namespaced\n versions:\n - additionalPrinterColumns:\n - description: Status of the schedule\n jsonPath: .status.phase\n name: Status\n type: string\n - description: A Cron expression defining when to run the Backup\n jsonPath: .spec.schedule\n name: Schedule\n type: string\n - description: The last time a Backup was run for this schedule\n jsonPath: .status.lastBackup\n name: LastBackup\n type: date\n - jsonPath: .metadata.creationTimestamp\n name: Age\n type: date\n - jsonPath: .spec.paused\n name: Paused\n type: boolean\n name: v1\n schema:\n openAPIV3Schema:\n description: |-\n Schedule is a Velero resource that represents a pre-scheduled or\n periodic Backup that should be run.\n properties:\n apiVersion:\n description: |-\n APIVersion defines the versioned schema of this representation of an object.\n Servers should convert recognized schemas to the latest internal value, and\n may reject unrecognized values.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\n type: string\n kind:\n description: |-\n Kind is a string value representing the REST resource this object represents.\n Servers may infer this from the endpoint the client submits requests to.\n Cannot be updated.\n In CamelCase.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\n type: string\n metadata:\n type: object\n spec:\n description: ScheduleSpec defines the specification for a Velero schedule\n properties:\n paused:\n description: Paused specifies whether the schedule is paused or not\n type: boolean\n schedule:\n description: |-\n Schedule is a Cron expression defining when to run\n the Backup.\n type: string\n skipImmediately:\n description: |-\n SkipImmediately specifies whether to skip backup if schedule is due immediately from `schedule.status.lastBackup` timestamp when schedule is unpaused or if schedule is new.\n If true, backup will be skipped immediately when schedule is unpaused if it is due based on .Status.LastBackupTimestamp or schedule is new, and will run at next schedule time.\n If false, backup will not be skipped immediately when schedule is unpaused, but will run at next schedule time.\n If empty, will follow server configuration (default: false).\n type: boolean\n template:\n description: |-\n Template is the definition of the Backup to be run\n on the provided schedule\n properties:\n csiSnapshotTimeout:\n description: |-\n CSISnapshotTimeout specifies the time used to wait for CSI VolumeSnapshot status turns to\n ReadyToUse during creation, before returning error as timeout.\n The default value is 10 minute.\n type: string\n datamover:\n description: |-\n DataMover specifies the data mover to be used by the backup.\n If DataMover is \"\" or \"velero\", the built-in data mover will be used.\n type: string\n defaultVolumesToFsBackup:\n description: |-\n DefaultVolumesToFsBackup specifies whether pod volume file system backup should be used\n for all volumes by default.\n nullable: true\n type: boolean\n defaultVolumesToRestic:\n description: |-\n DefaultVolumesToRestic specifies whether restic should be used to take a\n backup of all pod volumes by default.\n\n Deprecated: this field is no longer used and will be removed entirely in future. Use DefaultVolumesToFsBackup instead.\n nullable: true\n type: boolean\n excludedClusterScopedResources:\n description: |-\n ExcludedClusterScopedResources is a slice of cluster-scoped\n resource type names to exclude from the backup.\n If set to \"*\", all cluster-scoped resource types are excluded.\n The default value is empty.\n items:\n type: string\n nullable: true\n type: array\n excludedNamespaceScopedResources:\n description: |-\n ExcludedNamespaceScopedResources is a slice of namespace-scoped\n resource type names to exclude from the backup.\n If set to \"*\", all namespace-scoped resource types are excluded.\n The default value is empty.\n items:\n type: string\n nullable: true\n type: array\n excludedNamespaces:\n description: |-\n ExcludedNamespaces contains a list of namespaces that are not\n included in the backup.\n items:\n type: string\n nullable: true\n type: array\n excludedResources:\n description: |-\n ExcludedResources is a slice of resource names that are not\n included in the backup.\n items:\n type: string\n nullable: true\n type: array\n hooks:\n description: Hooks represent custom behaviors that should be executed\n at different phases of the backup.\n properties:\n resources:\n description: Resources are hooks that should be executed when\n backing up individual instances of a resource.\n items:\n description: |-\n BackupResourceHookSpec defines one or more BackupResourceHooks that should be executed based on\n the rules defined for namespaces, resources, and label selector.\n properties:\n excludedNamespaces:\n description: ExcludedNamespaces specifies the namespaces\n to which this hook spec does not apply.\n items:\n type: string\n nullable: true\n type: array\n excludedResources:\n description: ExcludedResources specifies the resources\n to which this hook spec does not apply.\n items:\n type: string\n nullable: true\n type: array\n includedNamespaces:\n description: |-\n IncludedNamespaces specifies the namespaces to which this hook spec applies. If empty, it applies\n to all namespaces.\n items:\n type: string\n nullable: true\n type: array\n includedResources:\n description: |-\n IncludedResources specifies the resources to which this hook spec applies. If empty, it applies\n to all resources.\n items:\n type: string\n nullable: true\n type: array\n labelSelector:\n description: LabelSelector, if specified, filters the\n resources to which this hook spec applies.\n nullable: true\n properties:\n matchExpressions:\n description: matchExpressions is a list of label\n selector requirements. The requirements are ANDed.\n items:\n description: |-\n A label selector requirement is a selector that contains values, a key, and an operator that\n relates the key and values.\n properties:\n key:\n description: key is the label key that the\n selector applies to.\n type: string\n operator:\n description: |-\n operator represents a key's relationship to a set of values.\n Valid operators are In, NotIn, Exists and DoesNotExist.\n type: string\n values:\n description: |-\n values is an array of string values. If the operator is In or NotIn,\n the values array must be non-empty. If the operator is Exists or DoesNotExist,\n the values array must be empty. This array is replaced during a strategic\n merge patch.\n items:\n type: string\n type: array\n x-kubernetes-list-type: atomic\n required:\n - key\n - operator\n type: object\n type: array\n x-kubernetes-list-type: atomic\n matchLabels:\n additionalProperties:\n type: string\n description: |-\n matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels\n map is equivalent to an element of matchExpressions, whose key field is \"key\", the\n operator is \"In\", and the values array contains only \"value\". The requirements are ANDed.\n type: object\n type: object\n x-kubernetes-map-type: atomic\n name:\n description: Name is the name of this hook.\n type: string\n post:\n description: |-\n PostHooks is a list of BackupResourceHooks to execute after storing the item in the backup.\n These are executed after all \"additional items\" from item actions are processed.\n items:\n description: BackupResourceHook defines a hook for\n a resource.\n properties:\n exec:\n description: Exec defines an exec hook.\n properties:\n command:\n description: Command is the command and arguments\n to execute.\n items:\n type: string\n minItems: 1\n type: array\n container:\n description: |-\n Container is the container in the pod where the command should be executed. If not specified,\n the pod's first container is used.\n type: string\n onError:\n description: OnError specifies how Velero\n should behave if it encounters an error\n executing this hook.\n enum:\n - Continue\n - Fail\n type: string\n timeout:\n description: |-\n Timeout defines the maximum amount of time Velero should wait for the hook to complete before\n considering the execution a failure.\n type: string\n required:\n - command\n type: object\n required:\n - exec\n type: object\n type: array\n pre:\n description: |-\n PreHooks is a list of BackupResourceHooks to execute prior to storing the item in the backup.\n These are executed before any \"additional items\" from item actions are processed.\n items:\n description: BackupResourceHook defines a hook for\n a resource.\n properties:\n exec:\n description: Exec defines an exec hook.\n properties:\n command:\n description: Command is the command and arguments\n to execute.\n items:\n type: string\n minItems: 1\n type: array\n container:\n description: |-\n Container is the container in the pod where the command should be executed. If not specified,\n the pod's first container is used.\n type: string\n onError:\n description: OnError specifies how Velero\n should behave if it encounters an error\n executing this hook.\n enum:\n - Continue\n - Fail\n type: string\n timeout:\n description: |-\n Timeout defines the maximum amount of time Velero should wait for the hook to complete before\n considering the execution a failure.\n type: string\n required:\n - command\n type: object\n required:\n - exec\n type: object\n type: array\n required:\n - name\n type: object\n nullable: true\n type: array\n type: object\n includeClusterResources:\n description: |-\n IncludeClusterResources specifies whether cluster-scoped resources\n should be included for consideration in the backup.\n nullable: true\n type: boolean\n includedClusterScopedResources:\n description: |-\n IncludedClusterScopedResources is a slice of cluster-scoped\n resource type names to include in the backup.\n If set to \"*\", all cluster-scoped resource types are included.\n The default value is empty, which means only related\n cluster-scoped resources are included.\n items:\n type: string\n nullable: true\n type: array\n includedNamespaceScopedResources:\n description: |-\n IncludedNamespaceScopedResources is a slice of namespace-scoped\n resource type names to include in the backup.\n The default value is \"*\".\n items:\n type: string\n nullable: true\n type: array\n includedNamespaces:\n description: |-\n IncludedNamespaces is a slice of namespace names to include objects\n from. If empty, all namespaces are included.\n items:\n type: string\n nullable: true\n type: array\n includedResources:\n description: |-\n IncludedResources is a slice of resource names to include\n in the backup. If empty, all resources are included.\n items:\n type: string\n nullable: true\n type: array\n itemOperationTimeout:\n description: |-\n ItemOperationTimeout specifies the time used to wait for asynchronous BackupItemAction operations\n The default value is 4 hour.\n type: string\n labelSelector:\n description: |-\n LabelSelector is a metav1.LabelSelector to filter with\n when adding individual objects to the backup. If empty\n or nil, all objects are included. Optional.\n nullable: true\n properties:\n matchExpressions:\n description: matchExpressions is a list of label selector\n requirements. The requirements are ANDed.\n items:\n description: |-\n A label selector requirement is a selector that contains values, a key, and an operator that\n relates the key and values.\n properties:\n key:\n description: key is the label key that the selector\n applies to.\n type: string\n operator:\n description: |-\n operator represents a key's relationship to a set of values.\n Valid operators are In, NotIn, Exists and DoesNotExist.\n type: string\n values:\n description: |-\n values is an array of string values. If the operator is In or NotIn,\n the values array must be non-empty. If the operator is Exists or DoesNotExist,\n the values array must be empty. This array is replaced during a strategic\n merge patch.\n items:\n type: string\n type: array\n x-kubernetes-list-type: atomic\n required:\n - key\n - operator\n type: object\n type: array\n x-kubernetes-list-type: atomic\n matchLabels:\n additionalProperties:\n type: string\n description: |-\n matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels\n map is equivalent to an element of matchExpressions, whose key field is \"key\", the\n operator is \"In\", and the values array contains only \"value\". The requirements are ANDed.\n type: object\n type: object\n x-kubernetes-map-type: atomic\n metadata:\n properties:\n labels:\n additionalProperties:\n type: string\n type: object\n type: object\n orLabelSelectors:\n description: |-\n OrLabelSelectors is list of metav1.LabelSelector to filter with\n when adding individual objects to the backup. If multiple provided\n they will be joined by the OR operator. LabelSelector as well as\n OrLabelSelectors cannot co-exist in backup request, only one of them\n can be used.\n items:\n description: |-\n A label selector is a label query over a set of resources. The result of matchLabels and\n matchExpressions are ANDed. An empty label selector matches all objects. A null\n label selector matches no objects.\n properties:\n matchExpressions:\n description: matchExpressions is a list of label selector\n requirements. The requirements are ANDed.\n items:\n description: |-\n A label selector requirement is a selector that contains values, a key, and an operator that\n relates the key and values.\n properties:\n key:\n description: key is the label key that the selector\n applies to.\n type: string\n operator:\n description: |-\n operator represents a key's relationship to a set of values.\n Valid operators are In, NotIn, Exists and DoesNotExist.\n type: string\n values:\n description: |-\n values is an array of string values. If the operator is In or NotIn,\n the values array must be non-empty. If the operator is Exists or DoesNotExist,\n the values array must be empty. This array is replaced during a strategic\n merge patch.\n items:\n type: string\n type: array\n x-kubernetes-list-type: atomic\n required:\n - key\n - operator\n type: object\n type: array\n x-kubernetes-list-type: atomic\n matchLabels:\n additionalProperties:\n type: string\n description: |-\n matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels\n map is equivalent to an element of matchExpressions, whose key field is \"key\", the\n operator is \"In\", and the values array contains only \"value\". The requirements are ANDed.\n type: object\n type: object\n x-kubernetes-map-type: atomic\n nullable: true\n type: array\n orderedResources:\n additionalProperties:\n type: string\n description: |-\n OrderedResources specifies the backup order of resources of specific Kind.\n The map key is the resource name and value is a list of object names separated by commas.\n Each resource name has format \"namespace/objectname\". For cluster resources, simply use \"objectname\".\n nullable: true\n type: object\n resourcePolicy:\n description: ResourcePolicy specifies the referenced resource\n policies that backup should follow\n properties:\n apiGroup:\n description: |-\n APIGroup is the group for the resource being referenced.\n If APIGroup is not specified, the specified Kind must be in the core API group.\n For any other third-party types, APIGroup is required.\n type: string\n kind:\n description: Kind is the type of resource being referenced\n type: string\n name:\n description: Name is the name of resource being referenced\n type: string\n required:\n - kind\n - name\n type: object\n x-kubernetes-map-type: atomic\n snapshotMoveData:\n description: SnapshotMoveData specifies whether snapshot data\n should be moved\n nullable: true\n type: boolean\n snapshotVolumes:\n description: |-\n SnapshotVolumes specifies whether to take snapshots\n of any PV's referenced in the set of objects included\n in the Backup.\n nullable: true\n type: boolean\n storageLocation:\n description: StorageLocation is a string containing the name of\n a BackupStorageLocation where the backup should be stored.\n type: string\n ttl:\n description: |-\n TTL is a time.Duration-parseable string describing how long\n the Backup should be retained for.\n type: string\n uploaderConfig:\n description: UploaderConfig specifies the configuration for the\n uploader.\n nullable: true\n properties:\n parallelFilesUpload:\n description: ParallelFilesUpload is the number of files parallel\n uploads to perform when using the uploader.\n type: integer\n type: object\n volumeGroupSnapshotLabelKey:\n description: VolumeGroupSnapshotLabelKey specifies the label key\n to group PVCs under a VGS.\n type: string\n volumeSnapshotLocations:\n description: VolumeSnapshotLocations is a list containing names\n of VolumeSnapshotLocations associated with this backup.\n items:\n type: string\n type: array\n type: object\n useOwnerReferencesInBackup:\n description: |-\n UseOwnerReferencesBackup specifies whether to use\n OwnerReferences on backups created by this Schedule.\n nullable: true\n type: boolean\n required:\n - schedule\n - template\n type: object\n status:\n description: ScheduleStatus captures the current state of a Velero schedule\n properties:\n lastBackup:\n description: |-\n LastBackup is the last time a Backup was run for this\n Schedule schedule\n format: date-time\n nullable: true\n type: string\n lastSkipped:\n description: LastSkipped is the last time a Schedule was skipped\n format: date-time\n nullable: true\n type: string\n phase:\n description: Phase is the current phase of the Schedule\n enum:\n - New\n - Enabled\n - FailedValidation\n type: string\n validationErrors:\n description: |-\n ValidationErrors is a slice of all validation errors (if\n applicable)\n items:\n type: string\n type: array\n type: object\n type: object\n served: true\n storage: true\n subresources: {}\n" }, { "path": "config/crd/v1/bases/velero.io_serverstatusrequests.yaml", "content": "---\napiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: v0.16.5\n name: serverstatusrequests.velero.io\nspec:\n group: velero.io\n names:\n kind: ServerStatusRequest\n listKind: ServerStatusRequestList\n plural: serverstatusrequests\n shortNames:\n - ssr\n singular: serverstatusrequest\n scope: Namespaced\n versions:\n - name: v1\n schema:\n openAPIV3Schema:\n description: |-\n ServerStatusRequest is a request to access current status information about\n the Velero server.\n properties:\n apiVersion:\n description: |-\n APIVersion defines the versioned schema of this representation of an object.\n Servers should convert recognized schemas to the latest internal value, and\n may reject unrecognized values.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\n type: string\n kind:\n description: |-\n Kind is a string value representing the REST resource this object represents.\n Servers may infer this from the endpoint the client submits requests to.\n Cannot be updated.\n In CamelCase.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\n type: string\n metadata:\n type: object\n spec:\n description: ServerStatusRequestSpec is the specification for a ServerStatusRequest.\n type: object\n status:\n description: ServerStatusRequestStatus is the current status of a ServerStatusRequest.\n properties:\n phase:\n description: Phase is the current lifecycle phase of the ServerStatusRequest.\n enum:\n - New\n - Processed\n type: string\n plugins:\n description: Plugins list information about the plugins running on\n the Velero server\n items:\n description: PluginInfo contains attributes of a Velero plugin\n properties:\n kind:\n type: string\n name:\n type: string\n required:\n - kind\n - name\n type: object\n nullable: true\n type: array\n processedTimestamp:\n description: |-\n ProcessedTimestamp is when the ServerStatusRequest was processed\n by the ServerStatusRequestController.\n format: date-time\n nullable: true\n type: string\n serverVersion:\n description: ServerVersion is the Velero server version.\n type: string\n type: object\n type: object\n served: true\n storage: true\n" }, { "path": "config/crd/v1/bases/velero.io_volumesnapshotlocations.yaml", "content": "---\napiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: v0.16.5\n name: volumesnapshotlocations.velero.io\nspec:\n group: velero.io\n names:\n kind: VolumeSnapshotLocation\n listKind: VolumeSnapshotLocationList\n plural: volumesnapshotlocations\n shortNames:\n - vsl\n singular: volumesnapshotlocation\n scope: Namespaced\n versions:\n - name: v1\n schema:\n openAPIV3Schema:\n description: VolumeSnapshotLocation is a location where Velero stores volume\n snapshots.\n properties:\n apiVersion:\n description: |-\n APIVersion defines the versioned schema of this representation of an object.\n Servers should convert recognized schemas to the latest internal value, and\n may reject unrecognized values.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\n type: string\n kind:\n description: |-\n Kind is a string value representing the REST resource this object represents.\n Servers may infer this from the endpoint the client submits requests to.\n Cannot be updated.\n In CamelCase.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\n type: string\n metadata:\n type: object\n spec:\n description: VolumeSnapshotLocationSpec defines the specification for\n a Velero VolumeSnapshotLocation.\n properties:\n config:\n additionalProperties:\n type: string\n description: Config is for provider-specific configuration fields.\n type: object\n credential:\n description: Credential contains the credential information intended\n to be used with this location\n properties:\n key:\n description: The key of the secret to select from. Must be a\n valid secret key.\n type: string\n name:\n default: \"\"\n description: |-\n Name of the referent.\n This field is effectively required, but due to backwards compatibility is\n allowed to be empty. Instances of this type with an empty value here are\n almost certainly wrong.\n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\n type: string\n optional:\n description: Specify whether the Secret or its key must be defined\n type: boolean\n required:\n - key\n type: object\n x-kubernetes-map-type: atomic\n provider:\n description: Provider is the provider of the volume storage.\n type: string\n required:\n - provider\n type: object\n status:\n description: VolumeSnapshotLocationStatus describes the current status\n of a Velero VolumeSnapshotLocation.\n properties:\n phase:\n description: VolumeSnapshotLocationPhase is the lifecycle phase of\n a Velero VolumeSnapshotLocation.\n enum:\n - Available\n - Unavailable\n type: string\n type: object\n type: object\n served: true\n storage: true\n" }, { "path": "config/crd/v1/crds/crds.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\n// Code generated by crds_generate.go; DO NOT EDIT.\n\npackage crds\n\nimport (\n\t\"bytes\"\n\t\"compress/gzip\"\n\t\"io\"\n\n\tapiextinstall \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/install\"\n\tapiextv1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1\"\n\t\"k8s.io/client-go/kubernetes/scheme\"\n)\n\nvar rawCRDs = [][]byte{\n\t[]byte(\"\\x1f\\x8b\\b\\x00\\x00\\x00\\x00\\x00\\x00\\xff\\xccXK\\x8f۶\\x13\\xbf\\xfbS\\f\\xf6\\x7f\\xfd\\xcbnP\\xb4(|K\\xdc\\x06\\b\\x9a\\x04\\v{\\x91;-\\x8dlf)\\x92%\\x87N\\xdd\\xc7w/\\x86\\x94lY\\xa2\\xad\\xf5\\x1e\\x8a\\xf2&r\\u07bfy\\xd9EQ̄\\x95_\\xd0yi\\xf4\\x12\\x84\\x95\\xf8;\\xa1\\xe6/?\\x7f\\xfe\\xc9ϥY\\x1c\\xde̞\\xa5\\xae\\x96\\xb0\\n\\x9eL\\xb3Fo\\x82+\\xf1g\\xac\\xa5\\x96$\\x8d\\x9e5H\\xa2\\x12$\\x963\\x00\\xa1\\xb5!\\xc1מ?\\x01J\\xa3\\xc9\\x19\\xa5\\xd0\\x15;\\xd4\\xf3\\xe7\\xb0\\xc5m\\x90\\xaaB\\x17\\x85w\\xaa\\x0f\\xdf\\xcd\\xdf\\xfc8\\xffa\\x06\\xa0E\\x83K؊\\xf29X\\x87\\xd6xI\\xc6I\\xf4\\xf3\\x03*tf.\\xcd\\xcc[,Y\\xfaΙ`\\x97p~Hܭ\\xe6d\\xf5\\xbb(h\\xdd\\t:\\xc6'%=\\xfd\\x9a}\\xfe(=E\\x12\\xab\\x82\\x13*gH|\\xf6R\\xef\\x82\\x12nD\\xc0\\n|i,.\\xe13\\xdbbE\\x89\\xd5\\f\\xa0\\xf54\\xdaV\\x80\\xa8\\xaa\\x18;\\xa1\\x1e\\x9dԄneTh\\xba\\x98\\x15\\xf0\\xd5\\x1b\\xfd(h\\xbf\\x84y\\x17\\xddy\\xe90\\x06\\xf6I6\\xe8I46\\xd2v\\x01{\\xbb\\xc3\\xf6\\x9b\\x8e\\xac\\xbc\\x12\\x84ca\\x1c\\xb9\\xf9\\xd9֧\\xa3\\xc5\\v)\\xe7@@\\xef-I\\xf4\\xe4\\xa4\\xde\\xcd\\xceć7)\\x14\\xe5\\x1e\\x1b\\xb1li\\x8dE\\xfd\\xf6\\xf1×\\xef7\\x17\\xd7\\x00\\xd6\\x19\\x8b\\x8ed\\aO:\\xbd\\xf4\\xeb\\xdd\\x02T\\xe8K'-\\xc5\\xe4\\xf8\\xab\\xb8x\\x03`\\x05\\x89\\v*\\xceC\\xf4@{\\xecb\\x8cUk\\x13\\x98\\x1ah/=8\\xb4\\x0e=ꔙ|-4\\x98\\xedW,i>\\x10\\xbdA\\xc7b\\xc0\\xefMP\\x15\\xa7\\xef\\x01\\x1d\\x81\\xc3\\xd2\\xec\\xb4\\xfc\\xe3$\\xdb\\x03\\x99\\xa8T\\tBO\\x10Q\\xd4B\\xc1A\\xa8\\x80\\xff\\a\\xa1\\xab\\x81\\xe4F\\x1c\\xc1!넠{\\xf2\\\"\\x83\\x1f\\xda\\xf1\\xc98\\x04\\xa9k\\xb3\\x84=\\x91\\xf5\\xcb\\xc5b'\\xa9+\\xca\\xd24MВ\\x8e\\x8bX_r\\x1b\\xc88\\xbf\\xa8\\xf0\\x80j\\xe1\\xe5\\xae\\x10\\xae\\xdcK\\u0092\\x82Å\\xb0\\xb2\\x88\\x8e\\xe8X\\x98\\xf3\\xa6\\xfa\\x9fk\\xcb\\xd8_\\xa8\\x1d\\x01\\x9dN\\xac\\xa4;\\xe0\\xe1\\xd2\\x02\\xe9A\\xb4\\xa2\\x92\\x8bg\\x14\\xf8\\x8aC\\xb7\\xfee\\xf3\\x04\\x9d%\\t\\xa9\\x04ʙt\\x14\\x97\\x0e\\x1f\\x8e\\xa6\\xd45\\xba\\xc4W;\\xd3D\\x99\\xa8+k\\xa4\\xa6\\xf8Q*\\x89\\x9a\\xc0\\x87m#\\x89\\xd3ව\\x9e\\x18\\xba\\xa1\\xd8Ul\\\\\\xb0E\\b\\x96K\\xa7\\x1a\\x12|а\\x12\\r\\xaa\\x95\\xf0\\xf8/cŨ\\xf8\\x82Ax\\x11Z\\xfdv<$N\\xe1\\xed=t\\xad\\xf4\\n\\xb4\\xc3\\xf6\\xb8\\xb1X2\\xb2\\x1c\\\\f\\x95\\xb5,SM\\xd5Ɓ\\x18\\xd1_F*\\xdf\\x02\\xf8\\xa4&\\xba!\\xe3\\xc4\\x0e?\\x9a$sH4\\x95v|\\xde\\xe5\\x04u\\x16s\\xdbJ=\\x01\\xf3\\x84\\x19\\x81\\xb4\\x17\\xd4k\\x06$\\xa4>\\xf5\\x94\\xac\\x937\\x90\\x89\\xe8\\b\\xee\\x14Z\\xe8\\x12\\xdf\\xc7|\\xd4\\xe5q\\xc2\\xd1O\\x19\\x16vio\\xbe\\x81\\xa9\\tu_hkkƓ-\\x82\\v\\xfa.c\\xcf>\\xae\\x8c\\xae\\xe5nlh\\x7f\\x90]\\x03wB\\xc9\\xc0\\xdb\\xf5@'{\\xca\\xc9u\\xb6\\xa5\\xe82\\x8f\\x01\\xa9\\xe5.\\xb8k\\xe0\\xd5\\x12U5j!\\x00:(%\\xb6\\n\\x97@.\\xe0\\x95\\x88\\x8cj\\xe52\\\"<\\x1f'\\x80[_\\x10\\x83\\xd4\\x15WK;\\xacXI\\x97\\x8c\\x9c\\xfe\\xa8+p\\x97kJ\\xff\\xa0\\x0e\\xcdX]\\x01\\xcf\\xc6J\\x91\\xb9w\\xe8I\\x96\\x99\\x87\\x87\\x87\\xfb2\\x80\\xc5|\\xa8\\xb8\\x1d\\xd5\\x12\\xddkjr=\\x90ѕc\\x1d\\x94j\\x15\\x14\\xa5i\\xac \\xb9U\\xd8\\xcd\\f\\xc6\\\\&\\x9ec.i`T\\x86\\xf0\\x14'\\x01c\\xce*\\x8cVG\\b\\x1e+\\xf8\\xb6G=\\x02\\xc3\\xc3C\\xd2\\xfdpWI\\x1cxQ\\xc3\\xd3j\\xf7\\x9ax|\\xb9\\x14\\xd1\\xefN\\xe9\\\":\\x96ZbϿ\\xae\\xfd\\xf8\\x8cHk\\xaaֲ\\x96/\\xd6\\xcc\\x1d\\x8eq_\\x91\\x0e\\ac\\xbe\\xc87\\xe6\\x01M\\xae\\xa5\\rH\\x06Q{\\xd1d\\\"A\\xc1\\xdf3\\x9b\\\"C\\x17\\xcd28\\x17g\\x7f\\xba\\xe5\\x95\\xef\\xd5\\xd3I\\tO\\xbd&\\xcc\\v\\xf8\\x04\\xee\\x1f\\xc7\\x1c\\x9da,\\f\\x88/\\x18\\xda~\\xf02\\xb8\\xfaP\\x96\\x88\\xd5x\\x1d\\x01Ʒ\\x11\\x94\\x16\\xfd\\x82彮\\xcb\\xe5\\x87\\x14z/vSN~JTi\\xd3kY@lM\\xa0+\\b\\xd0>\\xe7\\xe3mT&,\\xb5{\\xe1\\xa7\\xec|d\\x9a\\\\^\\f\\x96\\x81[&\\\\k\\xbf\\x9f\\xf1[\\xe6v\\x8d\\xa2\\x1a\\xb7\\xf0\\x02>\\x1b\\xca?\\xdd\\xec\\xc0%\\xea~2M\\x0e\\x9d\\x01={~\\x81A+r\\x94\\x7fc\\xaf%a\\x93\\x1d\\xe7\\xd7k%\\x1dn\\xe7\\n\\tO\\xbfU\\xf3d\\x03\\xd3WC\\xae\\x13h\\xe9\\x81W\\xb9X9Ws\\xa9\\vٔc\\xe9L\\x97P:\\x13\\x85\\x94\\xce\\xcd\\r\\an\\x15U&\\x12\\xf7\\x96\\xd6\\xd5P$\\xb8_\\x16\\x8eI\\x0f\\x1c\\xfa\\xa0\\xe8E\\x0e\\xac#i\\x87_b<\\xa7\\xdf\\xcb\\xec\\xc9\\xd7\\\\:\\x05l\\xba\\xd6x\\x95⽐\\xea\\xea\\U000e4cde\\x84\\xa3\\xfb\\xf2ws\\xc1r\\xfa\\x9dķ\\xfd\\xbc\\xfdO\\xe6獝\\xb7{\\x14Ή\\xe3\\xf4\\xe8\\x1e]z\\xfe\\xc9^\\xf5\\x8c\\xf3i\\x9d\\xe8߄\\xed\\xe9\\x1f\\x89%\\xfc\\xf9\\xf7\\xec\\x9f\\x00\\x00\\x00\\xff\\xff\\x18\\xd9g\\x90\\x9b\\x14\\x00\\x00\"),\n\t[]byte(\"\\x1f\\x8b\\b\\x00\\x00\\x00\\x00\\x00\\x00\\xff\\xec=]s\\xdb8\\x92\\xef\\xf9\\x15(\\xdd\\xc3\\xecnI\\xf6\\xa6\\ue8ee\\xfc\\x96q\\x92\\x1d\\xd5\\xcc$\\xde\\xd8\\xe3}\\x86Ȗ\\x841\\bp\\x00P\\xb6\\xf6\\xee\\xfe\\xfb\\x15\\x1a\\x00?DP\\x04eٓ\\xdd\\r_\\x12\\x8b`\\x03\\xfd\\xddh4\\x80\\xc5b\\xf1\\x86\\x96\\xec\\x1e\\x94fR\\\\\\x11Z2x2 \\xec_\\xfa\\xe2\\xe1\\xbf\\xf5\\x05\\x93\\x97\\xbb\\xb7o\\x1e\\x98ȯ\\xc8u\\xa5\\x8d,\\xbe\\x80\\x96\\x95\\xca\\xe0=\\xac\\x99`\\x86I\\xf1\\xa6\\x00Csj\\xe8\\xd5\\x1bB\\xa8\\x10\\xd2P\\xfb\\xb3\\xb6\\x7f\\x12\\x92Ia\\x94\\xe4\\x1c\\xd4b\\x03\\xe2\\xe2\\xa1Z\\xc1\\xaab<\\a\\x85\\xc0C\\u05fb?_\\xbc\\xfd\\xaf\\x8b\\xff|C\\x88\\xa0\\x05\\\\\\x91\\x15\\xcd\\x1e\\xaaR_쀃\\x92\\x17L\\xbe\\xd1%d\\x16\\xe4Fɪ\\xbc\\\"\\xcd\\v\\xf7\\x89\\xef\\xce\\r\\xf5{\\xfc\\x1a\\x7f\\xe0L\\x9b\\x1f[?\\xfeĴ\\xc1\\x17%\\xaf\\x14\\xe5uO\\xf8\\x9bfbSq\\xaa¯o\\bљ,\\xe1\\x8a|\\xb2]\\x944\\x83\\xfc\\r!~\\xd4\\xd8\\xe5\\xc2\\x0fx\\xf7\\xd6AȶPP7\\x16Bd\\t\\xe2\\xdd\\xcd\\xf2\\xfe\\xdfo;?\\x13\\x92\\x83\\xce\\x14+\\r\\xe2\\xfe\\xbf\\x8b\\xfaw\\xe2GI\\x98&\\x94\\xdc#\\x8eDy\\x92\\x13\\xb3\\xa5\\x86((\\x15h\\x10F\\x13\\xb3\\x05\\x92\\xd1\\xd2T\\n\\x88\\\\\\x93\\x1f\\xab\\x15(\\x01\\x06t\\v^\\xc6+m@\\x11m\\xa8\\x01B\\r\\xa1\\xa4\\x94L\\x18\\xc2\\x041\\xac\\x00\\xf2\\x87w7K\\\"W\\xbfBf4\\xa1\\\"'Tk\\x991j ';ɫ\\x02ܷ\\x7f\\xbc\\xa8\\xa1\\x96J\\x96\\xa0\\f\\vDwOK\\x92Z\\xbf\\x1e\\xc3\\xd5>\\x96<\\xee+\\x92[\\x91\\x02\\x87\\x96'1䞢\\x16?\\xb3e\\xbaA\\x1f\\x85\\xcc\\xfeL\\x85\\x1f\\xfe\\xc5\\x01\\xe8[P\\x16\\f\\xd1[Y\\xf1\\xdcJ\\xe2\\x0e\\x94%`&7\\x82\\xfd\\xbd\\x86\\xad\\x89\\x91\\xd8)\\xa7\\x06\\xb4\\xa5\\x8c\\x01%(';\\xca+\\x98[\\xa2\\x1c@.\\xe8\\x9e(\\xb0}\\x92J\\xb4\\xe0\\xe1\\a\\xfap\\x1c?K\\x05\\x84\\x89\\xb5\\xbc\\\"[cJ}uy\\xb9a&\\xe8W&\\x8b\\xa2\\x12\\xcc\\xec/QUت2R\\xe9\\xcb\\x1cv\\xc0/5\\xdb,\\xa8ʶ\\xcc@f\\xd9|IK\\xb6@D\\x04\\xea\\xd8E\\x91\\xff[\\x10\\x0f\\xdd\\xe9\\xd6\\xec\\xad\\xd8j\\xa3\\x98ش^\\xa0~L`\\x8fU\\x1d'\\x8c\\x0e\\x94C\\xb1\\xe1\\x82\\xfdɒ\\xeeˇۻ\\xb6\\xa02\\xed\\x99Ғ\\xd7!\\xfeXj2\\xb1\\x06\\xe5\\xbe[+Y L\\x10\\xb9\\x13U\\x94s\\xce@\\x18\\xa2\\xabU\\xc1\\x8c\\x15\\x83\\xdf*\\xd0V\\a\\xe4!\\xd8k\\xb4Ad\\x05\\xa4*s+Ƈ\\r\\x96\\x82\\\\\\xd3\\x02\\xf85\\xd5\\xf0ʼ\\xb2\\\\\\xd1\\v˄$n\\xb5-\\xebacG\\xde\\u058b` \\aX\\xeb\\f\\xcbm\\tYG\\xd1\\xecWl\\xcd2\\xa7Nk\\xa9\\x1a\\xbb\\xe3l`\\x97BqշO\\xa6٭\\xa0\\xa5\\xdeJs\\xc7\\n\\x90\\x959l1&kȼ\\xdb\\xe5\\x01\\x940B?^\\xb4Y\\x95\\x86\\xdc*\\xed#e\\x06\\xc7|}\\xbb$\\xf7h\\xac\\xc2\\xd7h\\xb4*ML\\xa5\\x84\\x95\\x92H__\\x80\\xe6\\xfb;\\xf9\\x8b\\x06\\x92W(ܙ\\x02\\xa4Ü\\xac`m%A\\x81\\xfd\\u07be\\x02\\xa5,m4\\x0e@V=cc\\x9f\\xbb-X\\xdaҊ\\x1b\\xaf'L\\x93\\xb7\\x7f&\\x05\\x13\\x95\\xe9\\x89\\xda בR\\xd4\\xd0B\\xee@\\x9dB\\xc4\\xf7\\xd4П\\xed\\xc7\\a\\xb4\\xb3@\\tB\\xb5\\xc4[y:\\xae\\xf6\\xf82\\xc6m\\xf7,\\xd7-\\x88L\\x93ٌHEf\\xce\\x03\\xcf\\xe6\\xee\\xeb\\x8aq\\xb3`\\xa2\\xdd\\xc7#\\xe3<\\xf42\\ryGC\\xc7P}'?j'\\xbc'\\xd1b\\x00V\\x8b4\\x8f[0[P\\xa4\\x94\\xb5\\xc7[3\\x0eD﵁\\xc2\\x13&x\\x11\\x8fO\\xa4'\\xd4\\x1d\\xce=\\bm\\xe9\\xea\\x11\\xe9#/*\\xce\\xe9\\x8a\\xc3\\x151\\xaa\\x82\\x01ڬ\\xa4\\xe4@\\xc5\\bq\\xbe\\x806,;\\ai\\x1c\\xa4\\ba\\x94\\x7fѡ\\x00:M\\xfa\\x00\\x84F@{\\x9aY\\xef\\xccy\\x8b\\xb0]\\xaaD\\xc7T*Ȭվ\\xf2ހ\\x01G\\x0f$$\\xe1Rl@\\xb9\\xdem\\xa4\\x12\\x04L\\x81\\x15\\xb8\\x9cXC\\xab\\x80[oB֕\\xb5\\xc1\\x17\\xc4j\\xf7\\xa0\\f0\\xa1\\rЈp>\\x83?\\xf0\\x94\\xf1*\\x87\\xfc\\xda\\x05^\\xb76~\\xccC\\xd4ܳ\\x9a)|\\xfap\\x14\\xa2\\xf7Μe\\x18\\x04\\xfaxo\\x81qkLL\\x1b'\\xbd/\\xc1\\x85Ζ\\x95~؍\\xf7=j\\x0f4\\x18\\xfb\\xd1\\xecO\\xb39r\\xb8\\xdbk\\xb7\\x0fM\\xa8\\x82\\x9a,\\xc9v\\x13\\x8a\\xd2\\xec\\xfb\\xad\\x99\\x81\\\"Bţ\\xf6$\\x91\\x9fT)\\xba\\x1f\\xe0f\\x1d\\xff\\x9f\\x91\\x9fC0\\x0f8*B\\xb3W\\xe6\\xe9a\\xbf\\xff\\xcc\\\\=\\x0f\\x1f5\\xcev)\\x13\\x96\\x7fv\\xe2\\xd9a\\x9fv\\xf37K6!M\\x04\\x1e\\x13\\x0e\\x1eN͎p\\xebw\\\"\\xd6Yd~H\\xc8k\\xd9\\xf2\\xc2\\xfb\\x0fI\\xa9\\xad\\x94\\x0fc\\xd4\\xf9\\xc1\\xb6i&E$ì\\nY\\xc1\\x96\\xee\\x98T\\x1e\\xf5\\xc6\\xd5\\xc2\\x13d\\x95\\x89j=5$g\\xeb5(\\v\\xa7\\xdcR\\r\\xdaM\\x93\\x87\\t2\\x1c\\xbe\\x93\\x96\\x19\\x89\\xbe<\\xc0\\xa3a\\xa4e\\x13b>4t\\x1bG\\x1cz\\xc9\\xf0\\u0601\\xda\\xf0\\x1a\\x9dq\\xcev,\\xaf(G\\xbfLE\\xe6\\xf0\\xa1\\xf5\\xb8bV\\xe6\\b\\x93{c\\x8eJ\\xa6{\\\\@\\x10\\x90\\xb2L\\xea̔\\xa4\\x00\\x1b\\xf3\\x16vN\\xd0o:\\x8c\\xf9\\x8a\\xdaXE\\x0eaO\\x90Y\\xaa\\xe2\\xa0}W9\\x86\\x91\\x8d͘7L\\xc1D\\x04\\xe1t\\x05\\x9ch\\xe0\\x90\\x19\\xa9\\xe2\\x14\\x19\\xe3\\xb3{R\\x8c\\xe0\\x00!#\\x96\\xaf;\\xd3h\\x108\\x02\\x92\\xe0\\x14n˲\\xad\\v\\xf5\\xac\\x10!\\x1c\\x92K\\xb0\\x01\\x9f!\\xb4,y\\xc4]4\\xcfQ\\xe6\\xfbN\\x8e\\xe9z\\xf3\\x8ch\\xfd!\\xbc\\x98\\xfe7O\\x82\\xcdl\\x9e(i\\x1b\\xfd\\xeaR\\xb6\\x16\\x87\\xf8\\x9c\\xb6y\\xfe9\\t\\x1b,\\xff\\tB{D\\xfb\\tf\\x85\\x92ezPn-U\\x19\\xe8\\v\\x1bNa\\xa43'̄_\\xc74\\xa1\\x13s\\xf5\\x92e\\x1d\\\"|ݼ\\x99.\\xf4\\x89\\xacIщ\\x17bL\\xdd\\xc5? _\\xd0e\\xdcz\\x8f\\x91̓\\x9f\\xda_\\xcd\\t[\\xd7D\\xcf\\xe7d\\u0378\\x01u@\\xfd\\x93L}\\xe0\\xcc9\\x88\\x91\\xe2\\xf5\\b\\xa6\\xefM\\xb6\\xfd\\xf0dC0ݬT%\\xd2\\xe5\\xf0c\\x17Ȇh\\xbf\\xeb\\x9eG\\xe0\\x12Lc3\\x05\\x05\\xa6\\xc7q\\xc6\\xd4\\xfe\\x05C\\xabw\\x9f\\xde\\xc7\\xe7W\\xed'A\\xf2z\\x88\\x8c(\\x9d{\\xde\\x1d`\\xd4\\x1e\\x9f\\x0f\\xe1\\xc3\\x1b\\x8c\\x81\\xea\\t\\x90[\\n\\x99\\x13J\\x1e`\\xefB\\x17*\\x88\\xe5\\x0f\\r\\x8d\\x13\\xbaW\\x80k2(g\\x0f\\xb0G0\\xf1E\\x96\\xfe\\x93*\\r\\xeey\\x80}J\\xb3\\x03\\x1a\\xda11\\xed\\x17\\x8f,\\x9d\\xec\\x0fH\\b̭\\xa7\\x8a\\x81{\\xbc*D\\x964\\xe2O\\xa2-\\tO\\xa0\\xfd\\th&\\x89J\\xbb\\x8f\\xf6*%J\\xc0w\\xda\\xf1\\xd2j̖\\x95hV1\\xe3 \\xd7\\xc9\\fu\\xcf=\\xe5,\\xaf;r:\\xb2\\x14s\\xf2I\\x1a\\xfbχ'\\xa6\\xfdB\\xe6{\\t\\xfa\\x934\\xf8ˋP\\xd4\\r\\xfc%\\xe9\\xe9z@E\\x13\\xce\\xca[\\x82\\xb5\\x97\\xe2\\x9cO\\xb3\\xd2VӞi\\xb2\\x14v\\xba\\xe2H\\x92\\xd8\\x15\\xae\\xba\\xba\\xee\\\\GE\\xa5q\\x15MH\\xb1pi\\x9bXO\\x9e\\xdeRu\\xc8\\xfd\\xecN}\\x87w\\xd6Y\\xb87n\\xed\\x97\\xd3\\f\\xf2\\xb0\\\\\\x83\\x8b\\x92\\xd4\\xc0\\x86e\\x89\\xfd\\x15\\xa06@Jk\\xc2\\xd3$\\\"Ѱzl\\xa6\\x89O\\x9a\\xf7n?O\\x8b\\x87z\\x8d\\x7fa]\\xce\\xc2C0\\xb2H\\xa0\\x81\\xb7\\xdd\\xf98>\\v\\xab\\xb3\\t\\xad\\x82$\\x8c6\\x1dX\\xb3\\x1cn\\x9aB\\x94g\\x90\\x03\\xbd8\\x868\\xa3ܥy\\x8eu.\\x94\\xdfL\\xf0(\\x13da\\xaaih\\x8dݹ\\xe0\\x82\\xe2R\\xcb\\xffXO\\x8b\\xda\\xf4\\x7f\\xa4\\xa4L\\xe9\\v\\xf2\\x0eKZ8t\\xde\\xf9\\xa4Y\\vLB\\x97X\\x92b\\xe5gG\\xb9\\xf5\\xfdր\\v\\x02\\xdcE\\x02r\\u074b\\x8b\\xe6\\xe4q+\\xb5s\\xdb\\xf5\\\"\\xce\\xec\\x01\\xf6n\\xc5p\\xb4˶\\x91\\x99-\\xc5\\xcc\\xc5\\x10=\\x83Q\\a\\x1cR\\xf0=\\x99\\xe1\\xbb\\xd9sB\\xa9DIMl\\xd6\\x11т\\x96i\\x12\\x8a%E\\xa9\\x81\\xba\\x9d\\xb0\\x86 \\xc4~X\\x97\\xca\\xd8 \\xfb\\x18\\xb6I\\\"ZJ\\x1dY\\xc8\\x1f\\x18ʈ\\xf0\\xdeHm\\\\\\xbe\\xac\\x133G\\x13j2$\\xd1\\b]\\xbb\\xfa%\\xa9B\\xb1\\x895\\xcac\\xa9\\xdf\\xf6s\\xb7\\x05\\r~\\xbd\\xc2'\\xe6\\x1cP;\\xb3\\x9b5\\xfa\\xed\\xac\\xfḓ\\x97`'4È\\x05\\xbf-\\x95\\xcc@Gײ\\x9b'\\xc1_D\\xaa2ڸ\\xd79G\\xeafI\\xae$\\xe3x\\n4<\\xe9!\\xaf%\\xc4\\xc4\\xf9\\u0087\\xa7VB\\xd4\\xea\\xbe\\xfd{LƦ\\x8e\\x8b`\\xc9`Q\\xd0\\xc32\\xa5\\xa4!^\\xbb/\\x836x@n\\xf2\\xa16\\x15Z\\x82T_^\\v\\xe0\\xd7\\x10(\\x14L,\\xb1\\x03\\xf2\\xf6\\x05\\x02\\voCc\\xc5&\\xb1\\xe7\\xb4P\\xf6:t\\xd2p\\xa7\\xfe\\xc1\\xa9r)q\\xa9@A\\x87y\\xfd\\xac:ơB\\x9aVBbB\\xb8Y\\xca\\xfc;M\\xd6Li\\xd3\\x1e\\x82\\x1e(S\\x89\\x82\\x998\\xf1\\x12\\x1f\\x94:i\\xde\\xf5\\xd9}\\xd9Jwm\\xe5c(\\xcfr\\x84I\\xc4\\x1cח\\x80\\xb05a\\x86\\x80\\xc8d%0\\x81c\\xf5\\x18\\xbbp\\xc4u\\x16\\x96\\xa5*I\\x9a\\xf6\\xdb\\aDU\\xa4\\x11`\\x81\\x92\\xc2\\xc4\\xd1LO\\xbb\\xf9G\\xca\\xf8K\\xb0\\xcd\\fU\\xb1Ş\\xd3t\\\"\\x94\\xb8\\xb5\\v\\xf2\\n\\xfaĊ\\xaa \\xb4\\xb0\\x1f\\xc0\\xb0\\x8c\\x0f\\xa1\\xdd+\\xc5\\xc8E\\xc5\\r+9.\\xa4\\xeeX\\x1eM6\\x98-\\xec\\xeb\\x034~\\x95\\xb8\\xf5ԟ\\x04\\xf3\\xf9K-\\xb5\\x17\\a\\x91>\\xd5\\xe4\\x118'4\\xa6W=\\xcc3w\\x12S&\\x17`\\xfd\\x91\\xd5N\\x7f0\\x88?\\xbei\\xee\\xc4\\x1dwעW+b)&*\\x86O\\x91\\x19t\\x1c)\\xf6\\xa6\\x17\\xc1\\xba8\\x1c\\x7f\\xfb\\xad\\x02\\xb5'x\\x8eM\\x1d\\xe74\\x9b\\xc0\\xbcbj;\\x11\\v\\xa6\\u009b\\xad\\xa1\\xfcy/\\xe8oT\\x99\\xbc\\x13\\xce\\xeb\\x1e\\x8e\\a\\xbf\\xb16\\xa2\\x99\\xd4X\\xc3g\\xe7+\\xd1>\\x06>\\x17\\xb2\\xfe:\\xf2\\xd9X\\x80\\x9c\\xba[\\xeae\\xa78\\xd3'9\\xa3QEz\\xe4\\xf7;\\xed\\x82:e\\xf7SZ\\x01\\xc0\\xe8n\\xa7\\x97\\x9a\\xf2\\x8cMz\\x92㼴\\xddL\\xd3\\x16\\v_p\\xf7\\xd2K\\xecZJ\\xa4T\\xca.\\xa5itz\\x85]I\\xaf\\xba\\x1b\\xe9\\xb5v!%\\xef>J*qI^\\x05N-Q9q;\\xcd\\xf8\\x1a\\xef\\xf1\\xddD\\t\\xbb\\x88\\x12V\\x7fǑ<\\x01\\xbd\\x84]B\\xd3v\\a%\\xf0,U\\x15_q\\x17\\xd0+\\xee\\xfey\\xed]?#\\x925\\xf2z\\xda\\ue793\\x97,\\xa4\\xcaA\\x1d]\\xf6I\\x95£\\xf2\\x972\\xb7\\xe9\\x0e\\xe4`\\xbd#\\x9c\\xfag[u\\xe2et\\x0f\\xfe\\xa0Q\\x97G\\xb3\\xa2\\xe4{;C!\\xb3\\xf6\\a\\xa7I@T\\xdaBo7\\x92\\xb3,\\x12\\xbbE\\xcffr\\x8d{\\x87e\\xe0\\x89QY\\xbbd\\xa0\\xb4\\r\\xe3\\xa1\\x1b\\x86y\\xdd#0גs\\xf98q\\xeeOK\\xf6\\x17<\\xb9\\xfb\\x19١w7K\\x84\\x11\\xc4\\x03\\x8f\\x02\\xaf\\x8b\\xb3jlV`\\xddr\\x83\\xe7\\x90\\xee/\\xd7\\x1d\\x88\\xdd:\\xc7\\xf6Ḑ\\xbbs\\x90CX\\xe0Mg&\\xadu\\xb9Y\\xbaq\\f\\xf5be\\x86\\x8a=\\x91XQc\\xb6L勒*\\xb3w\\x85\\x1a\\xf3\\xce\\x18\\x82/=\\x96\\xdd\\x19\\xf4\\x1e\\xfd\\xb3\\x9d\\xa3\\xe4\\rG:\\xe3\\n\\xe5\\xbe\\xec.\\xfa\\x1e\\xd2\\xee\\x94q\\f\\xef^\\x1cݷx\\xc6q\\f\\x87%\\v\\xa4T\\xe4\\xe7h\\xe5\\xd7ٲfڟL\\xfc\\xb3\\xdc\\xc1\\xfbh\\xf6\\xacC\\x9eۃ\\xe6\\x91\\xf2\\xac\\x00\\xd1\\x1d\\xba;X\\xa5\\xba\\x02<\\x90\\xb7\\xff\\xea\\x19\\xf5V\\xa1k\\x7f\\xa6\\xea)\\x89\\xb2\\xdb.\\x88\\b~\\xe1\\x84\\xd9\\xd0Y\\xcc>\\xe1\\x01\\xf0{rs\\x8fs\\xb4ڴy\\x15\\xf5s\\xb4\\x90*\\v\\x8b\\xc1\\x118\\xfe\\x83\\xef\\xcf_\\x9a\\xa6\\x8dTt\\x03?Iw\\xc6\\xf6\\x18ۻ\\xad;g\\xaf\\xfb\\xa8'ԏ\\x06\\xa5\\x89\\x1d\\xc0\\xebO\\xfb>\\x00\\xd6\\xd4|\\xf7\\x0e5\\xb6\\xa3\\x9cxL\\xb31\\xfc\\x14\\xbe\\xdf\\xdd\\xfd\\xe4\\xb02\\xac\\x80\\x8b\\xf7\\x95+w\\xb06Q\\x83%q\\xc0\\xd6AZ\\xd9\\xffn\\xe5#\\x1e\\xfe\\x1b\\xcfc\\x86;\\x13\\x1ad\\x14`\\xb19\\x96 NB\\xa9*\\xb9\\xa49\\xa8k)\\xd6l3\\x82\\xdd/\\x9d\\xc6\\an6\\xc3\\x1f=r\\xb5\\x8f\\n\\xf0\\xcf\\\\\\x83`c\\x1e\\u0381\\x7fd\\x1c\\xb4\\x1bV\\x82\\x01\\xbe\\xe9\\x7fU\\xdb\\xe3\\xaaX\\xb9\\x18nm_\\xd6\\x1d\\f\\xf88\\x87\\x16\\xa6\\xa2KP6\\x8arI\\xebJ\\aY\\x1dF\\xbc\\xe1\\b\\x13\\x066П\\x05\\x1e\\xb1\\xc0\\xeeTit\\x9f\\xc1\\x9c\\xe0\\\\\\xe6\\xc7X~\\xab\\x83\\xfc\\xfd\\xf0\\x97\\a\\x9cl\\xa5\\xbcb'\\xee\\xb9 \\xe4\\xe6\\xfeZ\\x93J\\xe4\\x98.\\xbe\\xff\\xcb\\xed$\\xa9\\xdbuN\\xae\\x0f\\xda:fT\\xef\\xe3_\\xb5\\x82㖽pѱ\\\\G\\x10\\x18\\x82Ӻ\\a\\xe4\\x91\\x19\\x7fp\\xd7yOZ\\x1d\\x9a\\xf2\\f\\xddp\\x80G\\xfa\\x8f\\xdfq\\xe0N\\xfe\\xf77\\xa3xu\\xac\\x14\\x1e\\x93\\xeao\\x05\\xc0cEO\\xba\\xe6`U\\x17l\\xd5\\xc5_\\xfa\\x9d1P\\x94&\\x16k\\x8c\\x9b\\xc3\\xef\\x8f\\x01\\xac\\xe34i(oi%\\r\\rb\\x91\\xb6ދ\\xecXa\\x99\\xb7FG\\xb8yL\\x1fc\\x04\\xb8\\xf6\\xfb!\\xceF\\x80\\x1a\\xe0\\x10\\x01t\\x95e\\xa0\\xf5\\xba\\xe2|_o\\xc7\\xf8J\\xa8\\xf1\\x912~>R8h\\x83\\x82`\\xd1;\\ni\\x14a_\\xee\\r\\\"\\x0f\\x9a\\x1e\\xb6*M#\\x85炯\\x86Ԇ\\x16']\\xd8p\\xdd\\a\\x83W\\xf6\\xa8\\xbcUTI\\xeb\\xb1Sݰ?\\xe6\\\\\\x1ap\\xeeK\\x9cdYh\\x90\\x13\\u0601 \\xd6;;\\x12\\x87;\\xa7&B\\xf1;\\\\\\x9d\\x87\\v\\xfe.\\xa4B\\xa2\\x17\\x13\\x11\\x9f\\xed\\xd0x\\x01\\xcew\\xba\\x86\\x89\\xb5\\xa2x\\x9fI\\x9f\\b\\xfd\\xe0\\xd7e+\\xael\\xf4\\x0f\\v\\v\\u2d285j\\x9b3ͺ~\\xe1yF\\xee\\xfav9\\x04\\xee\\x14\\x13\\u05ff\\xee\\xe5\\x99j\\xdcG\\xf7Y&\\xad\\x8f\\xee$\\x83\\x16\\x81X\\xcb\\xf8\\xf9qGU?\\xedPw\\xfc\\xd2\\x05\\x1cY\\xd8CG9\\xf7\\x1b\\x1d\\vКn\\xc2i\\xee\\x8fv\\xea\\xb1\\x01\\x01.=\\xe7\\x16O\\\"@\\x9b]qݳ̝\\xca\\xd0\\xccT\\xd4w\\x10\\n|[\\xad\\xbeӄ\\xcb\\x18T\\xbcЅ\\x85\\x9b\\xc2\\u009cl\\\"\\xa1\\x9eJ\\xa6R\\xe6p\\x1fꆖ6\\x18\\t#w\\x9a\\xbb݀\\xb3\\r\\xb3s\\x1d˹\\rU+\\xba\\x81E&9\\a\\xb4\\xd6\\xfdq\\xbd\\xa4\\xae\\xfb\\xbd\\x87_\\x80\\xeaQ\\xd4>\\xb6\\xdb\\xfa\\x15@\\xc7m\\xb7\\xf0M]\\xb9;\\xde\\xdee\\x98\\x82\\xe6\\\"\\xbdހ$v<)PvT\\x88\\xde2\\xd7\\x1fi\\xbbm\\xd0:o\\x96}\\x9e\\xd7_27\\xf7y\\x81\\xb8<\\x16\\xf4W\\xa9\\xe6\\xa4`\\xc2\\xfeCE\\xee\\x16\\xf0\\xc2Ǔƿ\\x95\\xf2\\xe16\\x12\\xc4\\xf6\\x06\\xffCݰY\\xea`\\xc2\\r\\x1b7\\x8c\\xaed\\xe5W\\xdf\\xeb\\x806\\xbe\\xac\\x82'\\xf3\\x9fy\\xba\\x890\\x8f\\xf8\\x83\\x1e:\\x83\\x19\\xdd\\x1f:\\x90F]\\x81\\xeby\\x00\\xd6m\\xb8Ɍ\\xf3\\xfd\\xfc\\x10\\xf2\\xc1\\xad\\x89\\r\\xec\\xd6\\xcd\\x05>\\fh\\xce#\\x18\\xe8(\\xacHE\\x81\\xd4\\a_\\xb4\\r\\xfa)\\xb3^O\\xe6\\xa1`\\xb2G\\xe3\\x1f\\x9a\\xd6Ctt\\xc3l\\x85{\\x03\\bv\\x82\\xc0\\xf3N\\xd8\\xf1\\x9a\\x8a\\x11\\u1ff1m\\xea\\xb3\\vZ\\x13\\xb7P%6\\x98\\xa5\\x8b\\xef}_\\x90O\\xd0_\\xaeX\\x90\\xbfVPEh\\xb0\\b\\x17\\xc3\\xdd\\x1a\\xaa\\xfa)_\\xb7\\r\\x1er\\xac\\xe8@m\\x8c4Y\\x8a\\x1b%7\\nt_X\\x17\\xe4o\\x94\\x19&6\\x1f\\xa5\\xba\\xe1Ն\\x89\\xcf\\xc3[~\\x8e5\\xbe\\xa1\\xca0+\\xecn<\\xb1\\x812A9\\xfb{̮\\xb5_\\x8e\\x03\\xba\\x1e\\x9c`-H\\xc20\\x86^\\xbc\\a\\x1b\\xe3\\x0e\\xe6\\x05\\xa2&\\xb4\\xf4t=%^\\t<\\x19\\xb3\\xa9u,\\xd1\\xc4\\\"\\xa1\\xdb\\v\\xf2IF\\r\\x83/\\x87b]\\x986$\\x03m\\x16\\xb0^Ke\\xdcj\\xf5bA\\xd8:$\\x1f\\xac\\xcd\\xc1\\xbc\\x99\\xbb\\xab\\x92\\xb0\\xd82s]hҸ/Lz+\\xf4\\xc2x\\x94}A\\xf7ne\\x8afYe#\\xacKm(\\x8f\\x048\\xcf2\\xfc\\x98\\xe5\\xb1\\xca\\a\\xf9/\\xcfZ\\xc9[\\xb6\\x01\\xf5\\x93\\x8e؏#)\\x1e\\xa6\\xe1\\xa2>nQ\\x04A\\x1e\\x153\\xc6\\xc6T\\xf2H)\\x81'\\x95\\xb1\\xb1\\x15\\xe7D[R\\x9f\\x94}$Ό.\\x87Kr\\xd2P\\xbe\\xab\\xa1\\f\\x99g\\x8f5\\xde̸B\\xda\\x10\\x1b\\xf7b\\xf5\\x91oeٜm\\xa9\\xd8\\f\\x9eP\\xb0U\\xb2\\xdal\\x83$\\x0f\\x04\\xd3$\\xaf\\x00\\x93\\xb5hRt\\xb8X\\xd8TJ\\xb4J\\t\\x8el\\xfb&A\\x18p\\xb84{ U9\\xf7\\x17\\xf7\\xfa{\\x99/\\xfd\\x1d(\\x8b\\xb5\\x92\\xc5\\xc2\\xf7\\x8b\\xb9Թ_\\xc9WL\\xda\\xc8\\xc5l\\xa3T'.j\\xf7\\xd7\\f\\xa0$\\x94%\\bB\\xb5\\xef9ᤨ\\x93\\xdd\\xd4o\\xd65\\xdcH\\xcd\\x12\\xa2\\xfd(\\xc7\\xff\\xda\\x06\\x10\\x18^\\x86\\xbf\\xbb\\xcc\\xf03\\x18\\xec3\\x86\\xc7g\\xbf\\x05\\x1fvT\\x187\\x9d\\xa8]\\xe4\\xcc9\\xb1٤\\x89\\x8c\\xb6\\x8e\\xedYI\\x9a\\xdb\\x0e\\x84\\x91\\xfc\\fv\\x17gѭ/\\xd7p\\a\\x81]\\xfb\\xebWk\\xc0s\\xa2\\x99\\b\\x17_\\xbb\\xd2\\x0f'\\xfdѕ@\\x81\\x17UJ\\x15\\xaf\\xc6<\\x9ep\\xe9\\\"\\xf4\\xba\\xb9\\x96]\\x1dI|8y*~\\x7f\\x00\\xe3`S7\\xdeKZ7\\t\\xd3\\xe7?\\xb0\\xd8z\\x00\\x96\\xf1f\\x16\\x95?\\xfe\\ue6f5wIS\\xbd8E\\x8e\\xcd\\xfcpR7<\\x85\\xeb\\xdeCz\\xc3\\xc1j\\x9b\\x06\\xe8N*'\\xe9\\xdc\\xee\\x8cٴs\\xa6\\xd2\\xc2\\x15\\xef\\xe7\\xc9%\\xedΘD{\\xb1\\f\\xdayQ~\\xa4xA\\xf4IZ\\xfb7\\xffm$\\x85\\xe6\\xc1\\x9e;\\x89\\xd6ʡ\\x85\\x81\\xbfj\\x16-\\xeas{?\\xa2\\x9d\\xce[\\xd6\\xc2\\xf7\\xe4\\x7f\\xf9\\xff\\x00\\x00\\x00\\xff\\xff<\\x82OF\\xb8\\x82\\x00\\x00\"),\n\t[]byte(\"\\x1f\\x8b\\b\\x00\\x00\\x00\\x00\\x00\\x00\\xff\\xccZK\\x93۸\\x11\\xbe\\xebWt\\xed\\x1e\\xf6\\xb2\\x94줒J\\xe96\\x96\\x93*W\\xc6\\xf1\\xd4hvr]\\bhJ\\xb0@\\x80\\x01@\\xc9\\xca㿧\\x1a\\x0f\\x89\\xe2C\\x0f;q\\u008b-\\x12h\\xf4\\xf3\\xeb\\x0f\\xc0\\x14E1a\\xb5|E\\xeb\\xa4\\xd1s`\\xb5\\xc4/\\x1e5\\xfdr\\xd3\\xed\\x1f\\xdcT\\x9a\\xd9\\xee\\xedd+\\xb5\\x98âq\\xdeT\\xcf\\xe8Lc9\\xbe\\xc7Rj\\xe9\\xa5ѓ\\n=\\x13̳\\xf9\\x04\\x80im<\\xa3\\u05ce~\\x02p\\xa3\\xbd5J\\xa1-֨\\xa7\\xdbf\\x85\\xabF*\\x816\\b\\xcfK\\xef\\xdeL\\xdf\\xfe~\\xfa\\xbb\\t\\x80f\\x15\\xcea\\xc5\\xf8\\xb6\\xa9\\x9d7\\x96\\xadQ\\x19\\x1eENw\\xa8К\\xa94\\x13W#\\xa7\\x15\\xd6\\xd64\\xf5\\x1cN\\x1f\\xa2\\x84\\xb4z\\xd4\\xfc]\\x10\\xb6\\x8c\\xc2\\x1e\\x93\\xb0\\xf0]I\\xe7\\xff<>\\xe6Q:\\x1f\\xc6ժ\\xb1L\\x8d\\xa9\\x15\\x86\\xb8\\x8d\\xb1\\xfe/\\xa7\\xa5\\vX9\\x15\\xbfH\\xbdn\\x14\\xb3#\\xd3'\\x00\\x8e\\x9b\\x1a\\xe7\\x10f\\u05cc\\xa3\\x98\\x00$\\xd7\\x04i\\x050!\\x82\\xb3\\x99z\\xb2R{\\xb4\\v\\xa3\\x9aJ\\x1f\\xd7\\x12踕\\xb5\\x0fΌ\\xb6@2\\x06\\xb25\\xe0<\\xf3\\x8d\\x03\\xd7\\xf0\\r0\\a\\x0f;&\\x15[)\\x9c\\xfd\\xa2Y\\xfe\\x7f\\x90\\a\\xf0\\xd9\\x19\\xfd\\xc4\\xfcf\\x0e\\xd38kZo\\x98\\xcb_c\\x8c\\x9eZo\\xfc\\x81\\fp\\xdeJ\\xbd\\x1eR\\xe9\\x919\\xffʔ\\x14A\\x93\\x17Y!H\\a~\\x83\\xa0\\x98\\xf3\\xe0\\xe9\\x05\\xfd\\x8a\\x1e\\x02r\\x11B\\xf6\\x10\\xec\\x99K\\xeb\\x00좔\\xe0\\xa3aMUo\\xad3\\xb5I\\x15x\\xedH\\x89\\xfaӛ\\xa4}Kl\\xce\\xef)\\xb7x\\x14\\xe9<\\xab\\xea3\\xb9\\x0fk\\x1c\\x13v\\xe6\\x8a\\xf7X\\xb2F\\xf9\\xb6\\xa9\\x14%\\xd5\\xce\\xcbs\\xb3j\\xe4S\\x11g\\x9d\\xad\\xf8\\xfe\\xec]\\\\ue\\x8cB\\x16\\xa5\\xc4Q\\xbb\\xb71\\v\\xf9\\x06+6O\\x83M\\x8d\\xfa\\xe1\\xe9\\xc3\\xebo\\x97g\\xafa(\\x91:EA\\x81c\\xad\\xd8l\\xd0\\\"\\xbc\\x86\\xfa\\x8bqsɴ\\xa3L\\x00\\xb3\\xfa\\x8cܟ\\x82X[S\\xa3\\xf52\\x17K|ZX\\xd4z\\xdb\\xd1\\xe9\\x9f\\xc5\\xd97\\x002#\\xce\\x02A\\xa0\\x841\\xafR\\xfd\\xa0H\\x96\\x83)\\xc1o\\xa4\\x03\\x8b\\xb5E\\x87:\\xc2\\x14\\xbdf:)8\\xed\\x88^\\xa2%1Tۍ\\x12\\x84e;\\xb4\\x1e,r\\xb3\\xd6\\xf2\\xefG\\xd9\\x0e\\xbcI\\xc9\\xec\\xd1y\\b\\x15\\xaa\\x99\\xa2dm\\xf0g`Zt$W\\xec\\x00\\x16iMhtK^\\x98\\xe0\\xbaz|\\xa4j\\x90\\xba4s\\xd8x_\\xbb\\xf9l\\xb6\\x96>#47U\\xd5h\\xe9\\x0f\\xb3\\x00\\xb6r\\xd5xc\\xddL\\xe0\\x0e\\xd5\\xcc\\xc9u\\xc1,\\xdfH\\x8f\\xdc7\\x16g\\xac\\x96E0DGH\\xadď6a\\xba;[\\xb6W\\xd2\\xf1\\t\\x90zGx\\b^c\\xcaDQ\\xd1\\xc4S\\x14\\xe8\\x15\\xb9\\xee\\xf9\\x8f\\xcb\\x17Ț\\xc4HŠ\\x9c\\x86\\xf6\\xfc\\x92\\xe3Cޔ\\xbaD\\x1b\\xe7\\x95\\xd6TA&jQ\\x1b\\xa9}\\xf8\\xc1\\x95D\\xed\\xc15\\xabJzJ\\x83\\xbf5\\xe8<\\x85\\xae+v\\x11\\xba\\x18\\xac\\x10\\x9a:\\x80Dw\\xc0\\a\\r\\vV\\xa1Z0\\x87\\xdf9V\\x14\\x15WP\\x10n\\x8aV\\xbb7w\\aG\\xf7\\xb6>\\xe4\\x9e:\\x12\\xdaA4X\\xd6\\xc8\\xcf\\xeaN\\xa0\\x93\\x96*\\xc33\\x8f\\xa1\\xba:\\x0eJP1ޔ\\xf33\\f\\x12\\xf40\\xceѹ\\x8fF`\\xf7KG\\xe5\\x87\\xe3\\xc03\\x1dk\\xb4\\x95t\\xa1\\xbdBil\\xb7\\xf3\\xb0#\\x92\\xb7\\x9f\\x8cx݀\\x03\\xa0n\\xaa\\xbe\\\"\\x05<#\\x13\\x9f\\xb4:\\x8c|\\xfa\\xab\\x95\\xbe\\xbf\\xd0H \\xe9\\x89*.\\x0f\\x9a?\\xa1\\x95F\\\\1\\xfe]g\\xf8\\xd1\\x05\\x1b\\xb3\\x872\\xe4\\xbf\\xf6\\xea@\\xd8\\xe5\\x0e\\x9a\\xf7Q;?\\x0fO\\x1f2\\x82\\xc7\\xdaJ\\x85\\x99|5\\x85\\x87TԦ\\x847 \\xa4#\\\"\\xe1\\x82о\\xb3t\\xa3\\x02ј\\x83\\xb7\\xcd]\\xe6s\\xa3K\\xb9\\xee\\x1b\\xdd\\xe6Fc\\x19sEt\\xc7s\\x8b\\xb0\\x12\\xa1\\x16eGm\\xcdN\\n\\xb4\\x05Շ,%O\\x9a46v\\x90R\\xa2\\x12=l\\x1a\\xad\\xb2`\\x8aEAE\\xcdԕ\\x18.\\x8e\\x03\\x03\\x93fR\\xc7\\f>\\t\\bXc\\xabԚ\\xb5G-\\xb0\\xdbm\\x826&\\x00\\x9aC\\x01{\\xe97\\x11)\\xd5P\\xdd\\xc1\\xc5ڣg\\x8b\\x87\\xa1\\xd7\\x1d\\xdd_6H#c\\xe3Ep\\xc8-\\xfa\\x90m\\xa8(}(\\x95\\xa6\\x00\\x1f\\x1b\\x17\\xb0\\xb6\\x8b\\x13\\xf9\\t\\x84/\\xcf\\xde\\xe2\\xa1\\xefh\\xb8\\x16\\xdcD\\x85FT\\x0e$j\\x0e?\\xfcpݤ^w\\xcb\\x0fQ\\xf7l\\xa8\\xc5\\x12-\\xea\\x1e\\x9b\\xc8\\xcfK\\xe8Q\\x944\\x94aX\\x96Ƚܡ:\\x84\\x9eD\\xe0\\xf93\\xac\\x1a\\x0f\\xa2\\xc1\\x105Ʒ{f\\x85\\x03n\\xaa\\x9ay\\xb9\\x92J\\xfa\\x03H7\\\"\\x9f)e\\xf6(Rı\\xaa\\xfda\\n\\x1f\\xb4\\xf3LstG\\x1eD\\x1e\\x8b\\xa9\\xc0t\\x1c\\x95\\xaa8\\x10:f\\x8700\\x8a\\xaf\\x8c\\xf3\\xc0\\xd1R:\\xaa\\x03\\xec\\xad\\xd1\\xeb1c\\a\\xda!\\xed\\x01\\xadF\\x8f\\xa1#\\n\\xc3\\x1d5C\\x8e\\xb5w3\\xb3C\\xbb\\x93\\xb8\\x9f\\xed\\x8d\\xddJ\\xbd.H\\xc1\\\"\\x81\\xcf,\\xec\\xecf?\\x86\\x7f\\xbe&\\vL\\x1dq\\xe2\\x86\\xe4]\\x86Z?\\x10\\xbd\\xf5\\x1b\\x8c-b\\x19s\\xd0X \\x02A\\xa9]\\xa5܍\\xc8:TvC\\xbc\\xbc\\xfd\\xe4\\x90\\x0f\\xf5\\x8f-\\xf6[\\xc7\\x05P\\x01\\xf8R\\x9c|[T\\xac.\\xe2h\\xe6M%\\xf9\\xa4km\\xcc\\xfb\\xcb\\xf8\\x937+R\\vɉܞ\\xe3F\\xdeĉ\\xb3=̀\\x1b\\xba\\xbb\\x9c1\\xb4\\x1cvS47q\\x85+\\x1a\\x7fj\\x8f=m}#t\\xa7\\xfe\\xef\\xd0\\x13\\xeft\\xa0\\x91\\xf8\\x01\\xb3}?\\a\\xc0\\xe4FkB*o\\x80\\x1d\\xdb\\xc0O\\xae\\xdb\\xff\\xeeD\\xcfU÷8\\xe0\\xf8\\x9e)\\xef\\xc2\\xc0\\xec\\xe38\\x8dti\\x1c\\x86\\xc6tM\\r\\xb8^\\x11\\x9c-\\xd0ޢ\\xcb(\\xf2-\\x1eH\\u0091[0X<\\xc0\\xaa\\xd1BaVu\\xbfAM\\xdb1Y\\x1e\\x88\\xec\\xbf<.\\xb3c\\x03\\x01K[\\xa7\\xec\\xde1 yO\\xbb\\x00JA1\\x87_\\x1c\\xa6u\\x9f\\xb1\\x04\\xa9\\x9dG\\xd6#\\xe9\\xf1\\x89\\xbdq\\x0e\\xab\\xc3\\x00\\u05fa\\xd9A\\xcfX~\\xbb\\x8f\\x82\\xae\\xe4\\xa1\\xd4 8\\xc6\\xc4J\\xb0\\x92\\xfa{\\xde\\x0f-\\x1e\\x02\\xc4\\x12\\xdf \\\"}\\xe6ґe\\xeetth\\xd0i\\xf1\\fdR\\xc7\\x02a\\xd5\\xd8\\\":\\x1fR\\x01\\x8by9H䇃q\\xb9.\\xe0\\x12\\xb3\\xe89\\xfb>v1*\\x13\\x80\\xdd\\xc80\\xe0z\\xb2\\xc0E\\xa6\\x017\\xb0\\x8d\\x9e\\x99\\xa39\\x05w\\xb2\\x0e\\xf8\\x0e\\xcc\\x03\\xfe\\xfb\\xec\\x03\\xeef \\xf0\\xddY\\bܖ)\\x97\\xd9\\b|\\x13#\\xb9\\xe0\\x8bK\\\\\\x05\\xae\\xf2\\x15\\xb8\\xc8Y`\\x94\\xb7\\xc05\\xee\\x02w\\xf2\\x17\\bx\\x82\\xa5\\xfcr\\x032?\\x85\\x81\\xb9\\x93\\xd6\\xcco\\xa8kH\\x81\\xc0\\x06\\xfaj<\\xa1\\x18q\\xd0q\\xd3\\xfb)\\x85\\xef+\\xfa\\xee%\\xd2\\x17չ\\x87\\xf7e@\\xbfB\\x8c\\x9eҰ\\xa3\\x17\\xf2\\xef\\x04 \\xe7\\a c\\x04mТ\\xdd\\xf1\\xb4\\xfdO\\xf1X\\x81\\x0f\\xa0\\xf8\\x992\\xaf\\xfd\\x19\\x17\\x8e'\\xf2\\x99\\xff\\x10K\\xa3Ͱ\\xb1\\x16]m\\xb4\\xa0\\xb6w\\xdb\\xe1\\xc4I\\xe5\\xff\\xdc\\x11\\xc5pX\\x8bs\\xfa\\xda\\xf9\\x96\\xa3p\\xd3\\xf9\\\\\\xb8߸\\xfb\\x84.\\xde\\xfa\\xb4Ͽ\\xccʡݵ\\x0e\\xe9:6~\\x97\\xb3\\xb9\\xc1\\xce\\xd6:\\xb0#\\xaa\\xa4\\xa1\\xd1\\xe1\\xc8\\\"4\\xad\\xe9d`F\\x9b\\x17\\xfa\\xd0<\\xa4\\x03m\\xf64\\xb9%-v=\\x13\\xe9M8\\xb4dZ\\xa4\\xe3b\\xfa4 y/\\x95\\xa2\\x1ef\\xb12\\xe4,\\xd4^Zj\\x96,\\xb4\\xb1\\xddo\\xa6o\\xfewg\\x81\\x8a9\\xbf\\xc6Qq\\xff\\x99\\xa6\\x00[\\x99\\xc6\\x0f\\xf4\\xfeV\\xc2\\x0f\\xd6t\\xb8e\\xbfG\\xc7\\xf0\\xb7\\x03\\xd7\\xe8\\t\\x8d\\xc9\\x11፵\\xe1\\xb2._\\\"ݱ\\xd1\\x1cC\\xe0\\x87Ο8\\xb4\\xbf\\xf5\\xff\\x00\\xe2\\x06\\xbb\\x06\\xbbt\\xefe촭\\xb8&'\\xb7\\xdf4\\xab\\xe3\\x15\\xec\\x1c\\xfe\\xf1\\xafɿ\\x03\\x00\\x00\\xff\\xff%\\xff\\\\)\\x99#\\x00\\x00\"),\n\t[]byte(\"\\x1f\\x8b\\b\\x00\\x00\\x00\\x00\\x00\\x00\\xff\\xbcVMo\\x1b7\\x10\\xbd\\xebW\\f\\xd0kwU\\xa3hQ\\xec\\xadqr0\\xda\\x06\\x82\\x1d\\xe4N\\x91#-c.\\xc9\\xce\\f\\xe5\\xba\\x1f\\xff\\xbd \\xb9+K\\xab\\x95\\x93\\\\\\xb27\\x91Ù\\xc7\\xf7f\\x1e\\xd54\\xcdJE\\xfb\\x11\\x89m\\xf0\\x1d\\xa8h\\xf1/A\\x9f\\x7fq\\xfb\\xf8\\v\\xb76\\xac\\x0f7\\xabG\\xebM\\a\\xb7\\x89%\\f\\xf7\\xc8!\\x91Ʒ\\xb8\\xb3ފ\\r~5\\xa0(\\xa3Du+\\x00\\xe5}\\x10\\x95\\x979\\xff\\x04\\xd0\\xc1\\v\\x05琚=\\xfa\\xf61mq\\x9b\\xac3H%\\xf9T\\xfa\\xf0C{\\xf3s\\xfb\\xd3\\n\\xc0\\xab\\x01;0\\xe8Pp\\xab\\xf4c\\x8a\\x84\\x7f&d\\xe1\\xf6\\x80\\x0e)\\xb46\\xac8\\xa2\\xce\\xf9\\xf7\\x14R\\xec\\xe0e\\xa3\\x9e\\x1fkW\\xdcoK\\xaa7%\\xd5}MUv\\x9de\\xf9\\xedZ\\xc4\\xefv\\x8c\\x8a.\\x91rˀJ\\x00[\\xbfON\\xd1b\\xc8\\n\\x80u\\x88\\xd8\\xc1\\xfb\\f+*\\x8df\\x050^\\xbb\\xc0l@\\x19S\\x88TnC\\xd6\\v\\xd2mpi\\x98\\bl\\xc0 k\\xb2Q\\nQ\\x1fz,W\\x84\\xb0\\x03\\xe9\\x11j9\\x90\\x00[\\x1c\\x11\\x98r\\x0e\\xe0\\x13\\a\\xbfQ\\xd2w\\xd0f\\xbe\\xda\\x1a\\x9a\\x81\\x8c\\x01\\x95\\xea7\\xf3ey\\u0380Y\\xc8\\xfa\\xfd5\\b,J\\x12O J]\\x1b<\\xd0\\t\\xbf\\xe7\\x00J|\\x1b{\\xc5\\xe7\\xd5\\x1f\\xcaƵ\\xca5\\xe6pS\\x99\\xd6=\\x0e\\xaa\\x1bcCD\\xff\\xeb\\xe6\\xee\\xe3\\x8f\\x0fg\\xcbp\\x8euAZ\\xb0\\fjB\\x9a\\x89\\xab\\xacA\\xf0\\b\\x81`\\b4\\xb1\\xca\\xed1i\\xa4\\x10\\x91\\xc4N\\xadU\\xbf\\x93\\xe19Y\\x9dA\\xf8\\xb79\\xdb\\x03Ȩ\\xeb)0y\\x8a\\x90\\v\\x89cS\\xa0\\x19/Zɵ\\f\\x84\\x91\\x90\\xd1\\u05f9\\xca\\xcb\\xcaC\\xd8~B-\\xed,\\xf5\\x03RN\\x03܇\\xe4L\\x1e\\xbe\\x03\\x92\\x00\\xa1\\x0e{o\\xff>\\xe6\\xe6|\\xef\\\\\\xd4))\\x94\\xe4\\xb6\\xf3\\xca\\xc1A\\xb9\\x84߃\\xf2f\\x96yP\\xcf@\\x98kB\\xf2'\\xf9\\xca\\x01\\x9e\\xe3\\xf8#\\x93h\\xfd.tЋD\\xee\\xd6뽕\\xc9Rt\\x18\\x86\\xe4\\xad<\\xaf\\x8b;\\xd8m\\x92@\\xbc6x@\\xb7f\\xbbo\\x14\\xe9\\xde\\njI\\x84k\\x15mS.⋭\\xb4\\x83\\xf9\\x8eF\\x13Ⳳ\\x17\\xddS\\xbf\\xe2\\x02_!O\\xf6\\x84\\xda#5U\\xbd\\xe2\\x8b\\ny)Sw\\xff\\xee\\xe1\\x03LH\\xaaRU\\x94\\x97\\xd0\\v^&}2\\x9b\\xd6\\xef\\x90\\xea\\xb9\\x1d\\x85\\xa1\\xe4Dob\\xb0^\\xca\\x0f\\xed,z\\x01N\\xdb\\xc1\\nO\\x1d\\x9b\\xa5\\x9b\\xa7\\xbd-\\xb6\\x9b\\x1d E\\xa3\\x04\\xcd<\\xe0\\xceí\\x1a\\xd0\\xdd*\\xc6o\\xacUV\\x85\\x9b,\\xc2\\x17\\xa9u\\xfa\\x98̃+\\xbd'\\x1b\\xd33pEڅ\\xe1\\x7f\\x88\\xa8\\xb3\\xb8\\x99\\xdf|\\xda\\ueb2ec\\xb5\\v\\x04O\\xbd\\xd5\\xfd4\\xfc3\\x9a\\x8eFq\\xce߲1\\xe4\\xef\\xc5n\\xe7;W/\\x0fEdK8k\\xd8\\x06.\\xbc\\xfbu^\\x8a\\xa9~%3\\xd5\\xd1Gnt\\\"*\\xcdw\\xf4y\\xb5t\\xe8K\\xb9@\\xa2@\\x17\\xab3P\\xefJP\\xf9Ǡ\\xacgP\\xfey<\\b\\xd2+\\x81'\\xa4\\r\\x97\\x95\\x1ax\\x8fO\\v\\xabw~CaO\\xc8\\xf3\\x96ϛ\\x9b\\xca\\x1e\\xce߃WXZlʋE\\xceVhNXd\\t\\xa4\\xf6\\xa7\\xbcr\\xda\\x1e\\x9d\\xbe\\x83\\x7f\\xfe[\\xfd\\x1f\\x00\\x00\\xff\\xff\\xbeM\\x1a\\xea\\xb1\\n\\x00\\x00\"),\n\t[]byte(\"\\x1f\\x8b\\b\\x00\\x00\\x00\\x00\\x00\\x00\\xff\\xbcWMo\\xe36\\x10\\xbd\\xfbW\\f\\xd0K\\v\\xac\\xe4\\x06E\\x8b·\\xd6\\xd9C\\xb0\\xe96\\x88\\xb7\\xb9S\\xd4HbC\\x91,9t6E\\x7f|1\\xa4\\xe4\\x0fYv\\x9c\\xcb\\xea\\xe6\\xe1p\\xf8\\xe6\\xcd\\xcc#]\\x14\\xc5B8\\xf5\\x84>(kV \\x9c¯\\x84\\x86\\x7f\\x85\\xf2\\xf9\\xd7P*\\xbb\\xdc\\xde,\\x9e\\x95\\xa9W\\xb0\\x8e\\x81l\\xff\\x88\\xc1F/\\xf1\\x16\\x1be\\x14)k\\x16=\\x92\\xa8\\x05\\x89\\xd5\\x02@\\x18cI\\xb09\\xf0O\\x00i\\ry\\xab5\\xfa\\xa2ES>\\xc7\\n\\xab\\xa8t\\x8d>\\x05\\x1f\\x8f\\xde\\xfeX\\xde\\xfcR\\xfe\\xbc\\x000\\xa2\\xc7\\x15\\xd4\\xf6\\xc5h+j\\x8f\\xffD\\f\\x14\\xca-j\\xf4\\xb6Tv\\x11\\x1cJ\\x8e\\xddz\\x1b\\xdd\\n\\xf6\\vy\\xefpn\\xc6|;\\x84y\\xccaҊV\\x81>ͭޫ\\xc1\\xc3\\xe9\\xe8\\x85>\\x05\\x91\\x16\\x832m\\xd4\\u009f,/\\x00\\x82\\xb4\\x0eW\\xf0\\x99a8!\\xb1^\\x00\\f)&XŐ\\xdd\\xf6&\\x87\\x92\\x1d\\xf6\\\"\\xe3\\x05\\xb0\\x0e\\xcdo\\x0fwO?m\\x8e\\xcc\\x005\\x06镣D\\xd4\\x7f\\xc5\\xce\\x0e\\xd3\\x04@\\x05\\x100\\xc0\\x01\\xb2;\\x84 \\f\\bO\\xaa\\x11\\x92\\xa0\\xf1\\xb6\\x87J\\xc8\\xe7\\xe8\\xc0V\\x7f\\xa3$\\bd\\xbdh\\xf1\\x03\\x84(;\\x10\\x1c%;\\x1c\\x9c\\xa5m\\v\\x8d\\xd2X\\xeel\\xce[\\x87\\x9e\\xd4Hy\\xfe\\x0e\\x1a\\xea\\xc0z)\\v\\xfe8\\xf1\\xbc\\vj\\xee,\\f@\\x1d\\x8e\\xe4a=p\\x05\\xb6\\x01\\xeaT\\x00\\x8f\\xcec@\\x93{\\x8d\\xcd\\xc2\\fٔ\\x93\\xd0\\x1b\\xf4\\x1c\\x06Bg\\xa3\\xae\\xb9!\\xb7\\xe8\\t\\x1aE\\xaf\\xcb41\\xaa\\x8ad}XָE\\xbd\\f\\xaa-\\x84\\x97\\x9d\\\"\\x94\\x14=.\\x85SEJĤQ+\\xfb\\xfa;?\\ff8:\\x96^\\xb9!\\x03yeڃ\\x854\\x1d\\xef(\\x0f\\xcfK\\xee\\xae\\x1c*\\xa7\\xb8\\xaf\\x02\\x9b\\x98\\xbaǏ\\x9b/0\\\"ɕ\\x1aZl\\xe7z\\xc2\\xcbX\\x1ffS\\x99\\x06}ޗڔc\\xa2\\xa9\\x9dU\\x86\\xd2\\x0f\\xa9\\x15\\x1a\\x82\\x10\\xab^Q\\x18{\\x9dK7\\r\\xbbNR\\x04\\x15Bt\\xb5 \\xac\\xa7\\x0ew\\x06֢G\\xbd\\x16\\x01\\xbfq\\xad\\xb8*\\xa1\\xe0\\\"\\\\U\\xadC\\x81\\x9d:gz\\x0f\\x16Fy&j^\\x01\\x128\\xe1[\\xa4\\xa9u\\x82\\xe5Kr\\xe2\\xe3_:q,X\\xdfcٖ\\xac9a\\x00\\x92\\xf5\\xe8\\x87i\\xa1.a\\x80\\xd9F\\x9fE2\\xf67\\xd3\\xc0\\xbc\\xb2\\xa0\\xb0\\xd8\\x1db:=\\x9a?4\\xb1\\x9f?\\xa0\\x80\\xdf\\x13\\xe6{\\xdb^\\\\_[C<\\x17\\x17\\x9d\\x9e\\xac\\x8e=n\\x8cp\\xa1\\xb3o\\xf8\\xde\\x11\\xf6\\x7f:\\xf4\\xf9\\x1a\\xbe\\xe8:\\xde滫\\xef\\x82c\\xd4g\\xcf}D\\xbeA\\xf0|\\xa6\\x83\\xc3UQ\\xae\\xc04x^\\x95\\xe8zs\\xf7\\x1e\\nϸ\\xbf\\xa3Hw\\xa6\\xb1o\\xa4\\xb8w\\x9c\\xf5;#\\x03\\xe3\\x97\\xde\\x10o\\xf74\\xbfBƞ\\xe6-\\xf9\\xeeD\\xf8\\x14+\\xf4\\x06\\t\\xc3^\\xa9_\\x14u\\xb3\\x11\\x01^:%\\xbb\\xb41\\r\\x04_\\x02!X\\xa9\\xe6$\\xf5\\n\\xf8\\xac#\\xca\\xe3\\xccP\\x16iXg\\xcc\\f\\xfe\\xc4|F\\xfd\\xce\\x1dP\\f\\x8at\\x95\\x82\\x92\\xa0\\x18ޡ\\xa1\\xc9\\x7f\\xa4ZF\\xef\\xd3\\x15\\x95\\xad\\xfc2\\x99n\\xb8VDG\\xe5\\xf9\\xeb\\xf1\\xfe\\r%\\xbd\\xdd{\\xa6\\x17\\xb7P&\\xa3q\\x1e\\x8b\\xa0Z~A\\xf1\\x1akiҸS2\\xf2w\\xfc\\xc2;&j\\xb6\\xa2\\xf8թ<\\x80o@\\xfc\\xb8ŝ\\x8f&\\xdf\\xf3\\xd37l\\n\\x88\\x81\\x9f[ \\x85\\x99\\xc1X!Ԩ\\x91\\xb0\\x86\\xea5\\xdf\\\\\\xaf\\x81\\xb0?\\xc5\\xddX\\xdf\\vZ\\x01\\xdf\\xff\\x05\\xa9\\x9962QkQi\\\\\\x01\\xf9x\\xae\\xcbf\\x13w\\x9d\\b3cx\\x94\\xf3\\x03\\xfb\\xcc5\\xc6n\\x18/v\\x06\\x9c\\xbd_\\n\\xf8\\x8c/3\\xd6\\ao%\\x86\\x80\\xa7ct6\\x93\\xd9!81\\x06~\\xa4\\xd5\\a,\\r\\x7f\\x19\\x06\\xcb\\xff\\x01\\x00\\x00\\xff\\xffx\\xae@\\xbaJ\\x0e\\x00\\x00\"),\n\t[]byte(\"\\x1f\\x8b\\b\\x00\\x00\\x00\\x00\\x00\\x00\\xff\\xc4:Ks\\x1b7\\xd2w\\xfd\\x8a.吤\\xca$\\xe3|ߦ\\xb6x\\xb3\\xe5͖v\\x13\\xafʔ}I\\xe5\\xd0\\x1c49\\x88f\\x00,\\x80\\x11\\xcd\\xcd\\xe6\\xbfo5\\x80\\xe1\\xbc@R\\xa2\\x93\\x18\\x17\\x89x4\\xfa\\xfd\\xc2\\xccf\\xb3+4\\xf2\\x03Y'\\xb5Z\\x02\\x1aI\\x1f=)\\xfe\\xe5\\xe6\\x0f\\x7fus\\xa9\\x17\\x8f/\\xaf\\x1e\\xa4\\x12K\\xb8i\\x9c\\xd7\\xf5;r\\xba\\xb1\\x05\\xbd\\xa1\\x8dT\\xd2K\\xad\\xaej\\xf2(\\xd0\\xe3\\xf2\\n\\x00\\x95\\xd2\\x1ey\\xda\\xf1O\\x80B+ouU\\x91\\x9dmI\\xcd\\x1f\\x9a5\\xad\\x1bY\\t\\xb2\\x01x{\\xf5\\xe37\\xf3\\x97\\xdf\\xcd\\xffr\\x05\\xa0\\xb0\\xa6%\\x18-\\x1eu\\xd5Դ\\xc6\\xe2\\xa11n\\xfeH\\x15Y=\\x97\\xfa\\xca\\x19*\\x18\\xf6\\xd6\\xea\\xc6,\\xa1[\\x88gӽ\\x11\\xe7;->\\x040\\xaf\\x03\\x98\\xb0RI\\xe7\\xff\\x99[\\xfdA:\\x1fv\\x98\\xaa\\xb1XM\\x91\\b\\x8bN\\xaamS\\xa1\\x9d,_\\x01\\xb8B\\x1bZ\\xc2[F\\xc3`A\\xe2\\n \\x91\\x18К\\x01\\n\\x11\\x98\\x86՝\\x95ʓ\\xbda\\b-\\xb3f \\xc8\\x15V\\x1a\\x1f\\x982\\xc2\\x0f\\x9cG\\xdf8pMQ\\x02:xK\\xbbŭ\\xba\\xb3zk\\xc9E\\xe4\\x00~qZݡ/\\x970\\x8f\\xdb\\xe7\\xa6DGi52w\\x15\\x16Ҕ\\xdf3\\xca\\xce[\\xa9\\xb69$\\xeeeM \\x1a\\x1b\\x84\\xca\\xd4\\x17\\x04\\xbe\\x94n\\x82\\xdd\\x0e\\x1dch} ;\\x8fKXg\\x88\\xcecm\\xc6H\\xf5\\x8eF\\xac\\x04z\\xca\\xe1t\\xa3kS\\x91'\\x01뽧\\x96\\x92\\x8d\\xb65\\xfa%H\\xe5\\xbf\\xfb\\xff\\xe3\\xecH\\xfc\\x9a\\x87\\xa3o\\xb4\\x1a\\xf2\\xe65\\xcfBo:b²ڒ\\xcd2H{\\xac>\\x05\\x11\\xcf\\x00^\\xf7\\xceGL\\\"\\xdc\\xfe\\xfcYTnUa\\xa9&u\\x19B\\xb2;=Ŧ\\x0f\\xba\\xbfj\\xac\\xd4V\\xfa\\xfd\\x12^~\\xf3T4\\xd9>@o\\xc0\\x97\\x04IyV^[\\xdc\\x12\\xfc\\xa0\\x8b\\xa8h\\xbb\\x92lR\\xb4u\\xd2\\xfeR7\\x95\\x80u+\\x18\\x00\\xe7\\xb5\\xcd*\\x9b\\xa1b\\x1eO%\\xb8-ؑ\\xc6\\r\\xef\\xfc#\\f\\xa2\\xb0\\x84Y\\x83h\\x9d\\xe6<\\xec\\x90Z\\xe5\\xad\\xe2Ֆ\\x9ed\\x11}\\x96*-\\xe8\\xc0?\\x9a\\xa0%\\x1d\\x18\\xab\\vr\\ue1212\\x8c\\x01\\\"o\\xbb\\x89\\xb3\\f*)\\xeci\\xf1iL\\xa5Q\\x90\\x05\\xaf\\xa1D%*b2\\x10\\xbcE\\xe56IE\\xa6\\x02l\\x8f\\xdd\\xef\\xcd\\x10\\x95\\xf7i\\xe1\\x18:q\\xd7\\xe3\\xcb讋\\x92j\\\\\\xa6\\xbdڐzuw\\xfb\\xe1\\xffV\\x83iVcm\\xc8zن\\x8f8z\\xc1\\xb17\\vCr\\xff;\\x1b\\xac\\x01\\xf0\\x05\\xf1\\x14\\b\\x8e\\x92\\xe4\\x02\\x1bR \\x91p\\x8a\\xec\\x91\\x0e,\\x19K\\x8eM+h\\x94\\xde\\x00*\\xd0\\xeb_\\xa8\\xf0\\xf3\\x11\\xe8\\x15Y\\x06\\xd3\\xdaB\\xa1\\xd5#Y\\x0f\\x96\\n\\xbdU\\xf2?\\a؎y͗V\\xe8\\xc9\\xf9`\\x8cVa\\x05\\x8fX5\\xf4\\x02P\\x89\\x11\\xe4\\x1a\\xf7`\\x89\\xef\\x84F\\xf5\\xe0\\x85\\x03n\\x8cǏ\\xda\\x12H\\xb5\\xd1K(\\xbd7n\\xb9Xl\\xa5oS\\x86B\\xd7u\\xa3\\xa4\\xdf/B\\xf4\\x97\\xeb\\xc6k\\xeb\\x16\\x82\\x1e\\xa9Z8\\xb9\\x9d\\xa1-J\\xe9\\xa9\\xf0\\x8d\\xa5\\x05\\x1a9\\v\\x84\\xa8\\x906\\xcck\\xf1\\x85MI\\x86\\x1b\\\\;\\x11t\\x1c!\\xd2?C<\\x1c\\xfb\\xd9\\b0\\x81\\x8a$vR\\xe0)fݻ\\xbf\\xad\\xee\\xa1\\xc5$J*\\n\\xa5\\xdb:\\xe1K+\\x1f\\xe6\\xa6T\\x1b\\xd6y>\\xb7\\xb1\\xba\\x0e0I\\t\\xa3\\xa5\\xf2\\xe1GQIR\\x1e\\\\\\xb3\\xae\\xa5g5\\xf8wCγ\\xe8\\xc6`oBZ\\x05k\\xb6%\\xf6\\x00b\\xbc\\xe1V\\xc1\\r\\xd6Tݠ\\xa3?YV,\\x157c!\\x89wg\\xf9\\xc3c#\\xa9\\x12!s8\\x7fwVsy\\xdcn\\\"\\x12!\\\"x\\r\\bFRA\\x83h\\fR9O(\\xd2$;AKi\\xedE\\xf4\\xf4G\\x91\\xe4\\xd1Em\\x96\\t G\\x1e)\\xe0\\x1f\\xab\\x7f\\xbd]\\xfc]G:\\x00\\vN\\xcdB\\xad\\x17\\xf2\\xed\\x17\\x87zO\\x90\\x93\\x96\\x04Wo4\\xafQ\\xc9\\r9?O\\xd0Ⱥ\\x9f\\xbe\\xfd9\\xcf?\\x80\\xef\\xb5\\x05\\xfa\\x88\\\\5\\xbd\\x00\\x19y~\\bf\\xad\\xdaH\\x17\\t?@\\x84\\x9d\\xf4e@\\xd4h\\x91\\b\\xdc\\x05\\x12<>\\xb0%G\\x12\\x1a\\x82J>d\\xec'\\x8e\\xeb\\x90\\xcduh\\xfe\\xca\\xd6\\xf3\\xdb5|\\x15\\x9d\\xd75\\xff\\xbc\\x8eh\\x1cҖ\\xbe\\x81u\\xe8D+\\xb3r\\xbb\\xa5.\\xef\\x9f(\\v\\x87Y\\x0eP_\\x83\\xb6L\\xab\\xd2=\\x10\\x010\\xcb)\\xc6\\a\\x12\\x13\\xf4~\\xfa\\xf6\\xe7k\\xf8jȃ#WI%\\xe8#|\\xcb\\xde'\\xf0\\xc6h\\xf1\\xf5\\x1c\\xee\\x83\\x1e\\xec\\x95Ǐ|SQjG\\n\\xb4\\xaa\\xf61\\x01~$p\\xba&\\xd8QU\\xcdb\\x82(`\\x87{Л#\\xf7\\xb4\\\"b\\xd5D0h\\xfd\\xc9$1\\xf1\\xe1\\xb4\\xd1L\\xb3\\xa6v<\\xcd^B\\x16\\xf5$\\xeb\\xfdl\\x19\\xc8\\x139\\x11ʅO\\xe0D\\xbf\\xf4\\xba\\x80\\x13\\x0f͚\\xac\\\"O\\x81\\x19B\\x17\\x8e\\xf9P\\x90\\xf1n\\xa1\\x1f\\xc9>J\\xda-v\\xda>H\\xb5\\x9d\\xb12\\u03a2\\xd4\\xdd\\\"t\\xbb\\x16_\\x84?\\x97\\x12\\x1e\\xdaT\\x9fJ}\\x00\\xf2\\xf9X\\xc0\\xb7\\xbb\\xc5%\\x1ch\\xb3\\xfb\\xa7Ǯ\\xa3|X\\xa5\\x84s\\f\\x93m~Wʢlk\\xbd\\x9e\\xb7\\xadQDw\\x8cj\\xff\\x99l\\x87\\xf9\\xdcX\\xc6h?K\\xad\\xda\\x19*\\xc1\\xff;\\xe9<\\xcf_\\xc2\\xd8F~\\x92sy\\x7f\\xfb\\xe6sZT#/\\xf1$Gj\\x988>\\xce:\\xacf5\\x9aY܍^ײ\\x18\\xed\\xe6\\x1c\\xfeV\\xb0\\x906\\x92\\xec\\x99\\xf4\\xef\\xdd`s\\x9b\\xa0f\\xaa\\x81Þg\\xe5\\x9f\\x1e\\xb7\\x99\\x84\\xaf\\xdf\\xc5>\\x95\\x16\\x9e\\xe4\\xd7yU\\xb8ǭ\\x03\\xb4\\x04\\b5\\x1aֈ\\a\\xda\\xcfb\\xc6aPr\\xba\\xc0\\x19\\xc1\\xa11\\bhL\\xc51=f\\x11\\x19\\x88)\\xffM\\xecA\\x17\\xe8;Ɛ\\xac(ۮԊ\\xbc\\x97\\xea32\\xe7\\xfd\\b\\x91ߗQ\\x87\\x9e]\\xa1\\xd5FnS\\xb7s\\xca)\\xd5T\\x15\\xae+Z\\x82\\xb7ͱ\\x9a\\xeb$#\\xefy\\xcbi\\xfa\\xdf\\xf7\\xb6\\xb6\\x1a~\\xa6\\xc1\\x98\\xa7j\\xd0v\\x9c\\x12C\\xaa\\xa9\\xa7\\xa8\\xcc\\xe0A\\x1b\\x89\\x99yK\\xceO\\xac\\x97\\x17\\xae\\xaf\\x9fccQ)/)\\xb9c\\x19\\x9c\\xabJ\\x93\\xa2\\xa7\\x04\\xbe\\xadL\\xbd\\ueabc\\xacП\\xe1\\x1b\\xb8\\xba\\xe7rd\\x88\\xf7,\\xdf.\\x19\\xed\\xe9u\\x97\\xdb)\\xa3\\xc5hf\\xe8\\x06G\\x8b\\x91\\xbe'\\xf5\\x90BC\\xfb\\x19]\\xa4\\xf8Ȗx\\x1a\\x83\\xa3o\\x9f\\xde8\\xed\\xbe\\xb4\\x8fą\\x9d\\xf1$\\x0e\\x8d\\xfeK$\\xfej\\f$\\xf4~\\xadHF!k:\\x94\\xfeC_\\x17\\x8b\\xbb5\\x81\\xb1d0\\xdb\\x15\\x82йw\\xa1\\x85\\xf9\\xa5\\x8b\\xc0\\xa4\\x83Ƒ\\b\\x1d\\xb4\\xc9\\xdd\\x13\\b\\xed;\\x93@O3>\\x7f\\x99\\xbf\\xc87\\xa6\\xe2\\x9b_\\xff\\xa5\\xe4\\xa2.\\xd5\\x14̔\\x85\\xd8r-<ᴏ\\x8d9\\x8eu\\xe0\\x0e\\xfc\\x8a\\xd0H\\x84*\\x94\\x8b\\xe4\\rʊ\\x04\\xb4/\\xd9τ\\xb2\\xa6\\r\\xa78\\xd1ǵ}\\x9c\\x84\\xde\\xf1\\xfa\\xef\\xb4$3L\\x98&<\\x7f\\xa40\\xc7O\\x8dg$y;\\xda\\x0e\\xa5\\xae\\x92\\xbcTS\\xafɲa\\x86\\aOP\\xb4㺿(Qm\\xb3N\\xae}\\xb0#\\xa8\\xd0yXw\\x1f\\x06\\xe4\\x88\\uffd8\\x8e)\\xeb\\xbfpv\\xa3&\\xe7p{Ν\\xff\\x18w\\xc5\\xce]:\\x02\\xb8֍\\xcf\\xdb\\xef\\x97.\\xb9\\xa0\\xe7u\\x0f\\xb3M\\xb1\\xa1\\xf7C_\\xb6\\xcen\\xd3TU8ӏ\\x1b\\xdd\\a\\x1c\\x01\\xab5\\xe53\\xfe\\x13\\xad\\xc3S\\b\\x96\\xe8α\\xea\\x8e\\xf7\\xe4\\xfc\\xf1!؝t\\xc8p\\\"\\xb0\\xbf\\xa5]f\\xb6\\xf5s\\x99\\xa5\\xbb\\xe4<3K\\x93/1\\xfa\\x8b\\xb17\\x9e\\xe3\\\\\\xbb\\x96\\x85y\\xf8\\xce!\\xb3\\xf6}\\xf0*\\xcfbv\\xc2\\xef\\x12\\xb7y\\xe8\\xadw\\x96\\x17>[\\x98\\xd8\\xdf0\\xff@%\\xfab\\xcb5!\\xba\\xf3\\xad\\x06EH\\xa9\\x91\\x96\\x9e\\x04\\x82\\xeb\\xf2\\x1a\\x84t\\xa6\\xc2\\xfd\\x81\\x96P\\xfa\\xb1\\xa9\\xe6\\xdfG:\\x8bj=\\xa6\\xa1c\\xa9\\xec\\xe9\\x0e\\xf7\\xe1k\\x91|]{\\xda_\\xc0\\x19\\x9f\\x11\\xd6\\xf5qg\\xf8{\\xdcp\\\"\\x15w\\n\\x8d+\\xb5\\xbf}sF5V\\x87\\x8d\\xad=vee\\b,\\xe1\\xe9-mJ\\xaa\\x90A\\xb5\\xf3n\\xcfr\\x16Ï\\x87.\\xd1\\xe2\\xd5\\x00\\u0099\\xb8\\x9f\\xbee\\xcaE\\xd7\\x15{\\x01v@\\xe1a\\xf7f\\xfc\\x05NjC\\x90A\\x9f\\x1a\\xe41\\x1e\\xe5\\xba\\nZ\\x85:B\\xdb\\xe9+;\\x9c\\r\\xe4C\\x82\\xfe\\xcc\\x18\\x9eU\\xa7\\xc9d\\xc0\\\\\\xf4`\\xa77\\xcd\\xfeL\\xb3><\\xf7/\\xe1\\xd7߮\\xfe\\x17\\x00\\x00\\xff\\xfff=C\\x19\\x96(\\x00\\x00\"),\n\t[]byte(\"\\x1f\\x8b\\b\\x00\\x00\\x00\\x00\\x00\\x00\\xff\\xc4Z͒\\x1b\\xb7\\x11\\xbe\\xefSt\\xad\\x0f\\xb6\\xab4d\\xa4$\\xae\\x14o\\xd2*Nmbo\\xb6DI\\x17\\x97\\x0f\\xe0\\xa09\\x03s\\x06\\x80\\x01\\f\\xb9\\xb4\\xe3wO5\\x80\\x19\\xce\\x0fH.\\xa9\\x925\\x17i\\xf1\\xd3\\xf8\\xf0u\\xa3\\xbb\\xd1`\\x96e7L\\x8b\\x8fh\\xacPr\\x01L\\v|r(\\xe9/;\\xdb\\xfc\\xc3΄\\x9ao_\\xdel\\x84\\xe4\\v\\xb8k\\xacS\\xf5;\\xb4\\xaa19\\xbeŵ\\x90\\xc2\\t%ojt\\x8c3\\xc7\\x167\\x00LJ\\xe5\\x185[\\xfa\\x13 W\\xd2\\x19UUh\\xb2\\x02\\xe5lӬpՈ\\x8a\\xa3\\xf1\\xc2ۥ\\xb7\\x7f\\x99\\xbd\\xfcn\\xf6\\xf7\\x1b\\x00\\xc9j\\\\\\x80V|\\xab\\xaa\\xa6F\\x83\\xd6)\\x83v\\xb6\\xc5\\n\\x8d\\x9a\\tuc5\\xe6$\\xbc0\\xaa\\xd1\\v8t\\x84\\xc9q\\xe1\\x00\\xfaQ\\xf1\\x8f^λ \\xc7wUº\\xff$\\xbb\\x7f\\x10\\xd6\\xf9!\\xbaj\\f\\xab\\x128|\\xaf\\x15\\xb2h*f\\xa6\\xfd7\\x006W\\x1a\\x17\\xf0@P4ˑ\\xdf\\x00\\xc4}zh\\x190\\xce=s\\xacz4B:4w$\\xa2e,\\x03\\x8e67B;\\xcf\\xcc\\x18\\\"X\\xc7\\\\c\\xc16y\\t\\xcc\\xc2\\x03\\xee\\xe6\\xf7\\xf2Ѩ\\u00a0\\r\\xf0\\x00~\\xb1J>2W.`\\x16\\x86\\xcft\\xc9,\\xc6\\xde@\\xf1\\xd2w\\xc4&\\xb7'\\xcc\\xd6\\x19!\\x8b\\x14\\x8a\\xf7\\xa2F\\xe0\\x8d\\xf1\\xaa\\xa5\\xfd\\xe7\\b\\xae\\x14v\\no\\xc7,A4\\xceo<\\r\\xc6\\xf7\\x93H\\xebX\\xadǨzS\\x03,\\xce\\x1c\\xa6@ݩZW\\xe8\\x90\\xc3j\\xef\\xb0\\xdd\\xcaZ\\x99\\x9a\\xb9\\x05\\b\\xe9\\xbe\\xfb\\xdbq>\\\"a3?\\xf5\\xad\\x92Cr\\xdeP+\\xf4\\x9a\\x03\\x12\\xd2V\\x81&ɐr\\xac\\xfa\\x14 \\x8e\\x04\\xbc\\xe9\\xcd\\x0fH\\x82\\xdc~\\xfbY(dz\\xa0\\xd6\\xe0J\\x847,\\xdf4\\x1a\\x96N\\x19V \\xfc\\xa0\\xf2\\xa0\\xc2]\\x89\\x06\\xfd\\x88U\\x18A'\\x18\\x04\\xe9N\\x99\\xa4\\xea4\\xe6\\xb306\\nke\\x8d\\xf47\\\\\\xe8\\xb3\\xd8Wn\\x90%\\xed\\xabuE3?B(\\x996\\xb2\\xd7\\x05>\\xcb\\xc0\\xfaDJű\\xc7\\xda\\x04\\x97\\xb0\\xa0\\x8d\\xca\\xd1\\xda\\x13\\x86OB\\x06H\\x1e\\x0e\\rg)*яi\\x015\\xbaR\\x8c\\xa3\\x01\\xa7\\xa0d\\x92W\\x18t\\xe8\\f\\x93v\\x1d-c\\xaa\\xc2v\\xda\\xfb\\xbd\\x1eB\\xf9\\xd0\\xca\\xeb\\xf5L0\\x85\\xa1ۗ\\xc1\\r\\xe6%\\xd6l\\x11\\xc7*\\x8d\\xf2\\xf5\\xe3\\xfdǿ.\\a\\xcd@\\xb4h4N\\xb4\\x9e9|\\xbd\\xc0\\xd3k\\x85\\xe1\\x9e\\xff\\x97\\r\\xfa\\x00h\\x810\\v8E \\xb4\\x9e\\x8b\\xe8_\\x91GL\\x81#a\\xc1\\xa06hQ\\x86\\x98D\\xcdL\\x82Z\\xfd\\x82\\xb9\\x9b\\x8dD/ѐ\\x18\\xb0\\xa5j*N\\x81k\\x8bƁ\\xc1\\\\\\x15R\\xfc\\xd6ɶD8-Z1\\x87\\xd6\\xf9\\x83h$\\xab`˪\\x06_\\x00\\x93|$\\xb9f{0HkB#{\\xf2\\xfc\\x04;\\xc6\\xf1\\xa3\\xb7&\\xb9V\\v(\\x9d\\xd3v1\\x9f\\x17µ\\xe18Wu\\xddH\\xe1\\xf6s\\x1fYŪq\\xca\\xd89\\xc7-Vs+\\x8a\\x8c\\x99\\xbc\\x14\\x0es\\xd7\\x18\\x9c3-2\\xbf\\x11\\xe9C\\xf2\\xac\\xe6_\\x99\\x18\\xc0\\xed`ى\\xa2\\xc3\\xe7\\x83\\xe8\\x05ꡨJ'\\x81EQa\\x8b\\a-P\\x13Q\\xf7\\xee\\x9f\\xcb\\xf7\\xd0\\\"\\t\\x9a\\nJ9\\f\\x9d\\xf0\\xd2\\xea\\x87\\xd8\\x14rM\\x86O\\xf3\\xd6F\\xd5^&J\\xae\\x95\\x90\\xce\\xff\\x91W\\x02\\xa5\\x03۬j\\xe1\\xc8\\f~m\\xd0:R\\xddX\\xec\\x9dOY`E\\a\\x8a\\xfc\\x00\\x1f\\x0f\\xb8\\x97p\\xc7j\\xac\\xee\\x98\\xc5?YW\\xa4\\x15\\x9b\\x91\\x12\\x9e\\xa5\\xad~\\\"6\\x1e\\x1c\\xe8\\xedu\\xb4Y\\xd4\\x11Վ\\xfd\\xdbRcN\\x9a%ri\\xaaX\\x8b\\x18I\\xd6\\xca\\x00\\x9b\\x8c\\x1f2\\x95v\\x01\\xf4%#\\xcax\\xd09\\xb3\\xa3\\xefMJP\\x8bX\\xf6\\x1cy\\x8cw6\\x06\\xaaj\\x18\\xa8\\xfa\\xdf$F\\x1a\\xd4\\xca\\n\\xa7\\xcc\\xfe\\x10)\\xc7&qT;\\xf4\\xe5L\\xe6X]\\xb3\\xbd;?\\x13\\x84\\xe4\\xc4;v&M\\xce(H\\xf5@\\x95,\\x14\\x1d\\xb2\\x89:\\xe0\\xde\\xd18\\xb2s\\x8b.\\xbdYy4\\xb2\\t\\t\\x87\\x1c\\x13\\xfa\\xb9\\xe4x\\xdb+\\xa5*dc6\\xb5\\xe2g6\\xfd\\xa8\\xa2\\xe30\\xb8F\\x83>\\xfe\\a7\\xab\\x95wƎ\\tٺ\\x8f\\x90r\\x83S\\x89}\\xac\\xc8\\xdd\\x1cS\\xcdq;\\x84\\x13!)\\t\\xf8\\xf5\\xe3}\\x1bvZˊ\\xd0'\\x91\\xa5\\xcfO\\xd2,\\xe8[\\v\\xac\\xb8\\x0f\\xd4\\xe7\\xd7NZ\\b}\\xf7\\xeb\\x00\\xc2\\xfb^\\xa7\\x80\\x81\\x16\\x98\\xe3 \\ue050\\xd6!㱑܍\\xc1\\xd8\\xf7\\\"\\xf8ԣ \\xe9;\\xc4GR\\t0\\xf2\\xf1\\x82ÿ\\x97\\xff}\\x98\\xffK\\x85}\\x00\\xcb)\\x13\\xf2w\\x15\\xacQ\\xba\\x17\\xdd}\\x85\\xa3\\x15\\x069\\xdd>pV3)\\xd6h\\xdd,JCc\\x7fz\\xf5s\\x9a?\\x80\\xef\\x95\\x01|b\\x94\\xf4\\xbf\\x00\\x118\\xef\\xc2Fk5\\u0086\\x8dw\\x12a'\\\\\\xe9\\x81j\\xc5\\xe3\\x06w~\\v\\x8em\\xe8Ą-4\\b\\x95\\xd8`\\x9a}\\x80[\\x9f<\\x1d`\\xfeN.\\xe5\\x8f[\\xf8&8\\x89[\\xfa\\xf36\\xc0\\xe8\\x12\\x84\\xbe\\xd79\\xc0q%s\\xe0\\x8c(\\n<$\\xda\\x13c\\xa1\\x80F\\xa1\\xe0[P\\x86\\xf6*UO\\x84\\x17Lz\\n\\x8e\\x18\\xf9\\x04\\xdeO\\xaf~\\xbe\\x85o\\x86\\x1c\\x1cYJH\\x8eO\\xf0\\x8aθ\\xe7F+\\xfe\\xed\\f\\xde{;\\xd8KǞh\\xa5\\xbcT\\x16%(Y\\xedC\\xbe\\xb9E\\xb0\\xaaF\\xd8aUe!\\x15\\xe3\\xb0c{P\\xeb#\\xeb\\xb4*\\\"\\xd3d\\xa0\\x99q'ӱ\\xc8\\xc3\\xe9C3\\xcdO\\xda\\xefy\\xe7\\xc5\\xe7+\\xcf:\\xbd_,\\xd6?\\x93\\t\\x9f\\x98\\x7f\\x02\\x13\\xfd\\xab\\xce\\x15Ll\\x9a\\x15\\x1a\\x89\\x0e=\\x19\\\\\\xe5\\x96x\\xc8Q;;W[4[\\x81\\xbb\\xf9N\\x99\\x8d\\x90EFƘ\\x05\\xad۹/\\xd9̿\\xf2\\xff\\\\\\xbbq_g\\xf9\\xd4\\xdd{!_\\x8e\\x02Z\\xddίa\\xa0ͣ\\x9f\\x1f\\xbb\\x8e\\U000b0319\\xddX&\\x9d\\xf9])\\xf2\\xb2\\xbdU\\xf5\\xbcm\\xcdxp\\xc7L\\xee\\xbf\\xd0\\xd9!\\x9e\\x1bC\\x88\\xf6Y,8fLr\\xfa\\xbf\\x15\\xd6Q\\xfb5\\xc46ⓜˇ\\xfb\\xb7_\\xf2D5\\xe2\\x1aOr\\xe4\\xb6\\x10\\xbe\\xa7\\xec\\x80*\\xab\\x99\\xce\\xc2h\\xe6T-\\xf2\\xd1hʕ\\xef9)i-М\\xc9\\xfe\\xde\\r\\x06\\xb7Y{\\\"\\xeb\\xee\\xc6\\\\\\x94v[ɴ-\\x95\\xbb\\x7f{\\x06Dz\\x1b\\xd8b8\\xe80&\\x9d\\xad,:\\x12's\\xcdg\\xe0Y\\x8a\\xdf\\x12n+\\x89\\x88\\x86\\xb6\\x98*U\\x88\\x9cU`}\\x9b\\x8c\\xc5\\xca\\b\\xb3\\x95=\\x05\\x94\\xaaG\\x8e\\xe1\\xf6\\xab\\x8a=\\xbc\\xde\\x17<\\x1c\\xf7\\xb4C\\xc8\\xc3\\xd1-jeD!$\\xab\\x0e\\x1e\\xdb_\\x1d%\\xab\\x99\\xff+a\\xab5\\xd3Z\\xc8\\xe2\\\"n\\xdb\\xfa\\xd6\\x12\\x9d\\x13\\xb2H$\\xfa\\xfd\\xf2\\xfb\\xa9\\xeb\\xc0\\xc9sr\\xde\\x05|\\x18\\x01\\x01f\\x10\\x18\\xed\\x89T\\xb5\\xc1}\\x16\\xb2N\\xcd\\x04\\xa5\\x8c\\x94\\x15\\xc6\\xd4z\\x85\\xc0\\xb4\\xae(\\xaf\\v\\x99d\\xca7\\xb5պ\\\\ɵ(b\\xe5tʔl\\xaa\\x8a\\xad*\\\\\\x803ͱK[\\xf2\\xb8\\xf7\\v\\x85g4\\xfe\\xa17\\xb4U\\xf7\\x99RezW\\x83\\x02\\xe6t3(\\x9bz\\n%\\x83\\x8d҂%\\xda\\xe9pN\\x1c\\x13u\\xdc\\xde^bR\\xe1\\xe4\\x9f\\xe1 ܙS\\x05\\x87\\xe88\\xe25$^\\xb1\\x83\\xfbHG\\xf3K\\x1d\\x8a\\xc1_\\x1b\\xbaS\\r\\x11f\\xe9\\xda\\xcah\\x8cV\\xfcfLZ\\xdf\\x17\\x8f:\\x0f\\x9et\\xdc1<\\xf4\\xa3\\xde@\\xc1\\xb3\\xcaR\\xbeP~Ia*<\\x87E\\xdeC\\x1a\\xe0\\xdaG2\\xba`\\\\]\\x9a\\xa2;\\xacvȻ7\\x84k\\xea6\\xaf\\xc7B|A\\xd9\\xf0xHD\\x8d]\\x91#ډ9\\x94]B\\x88\\xd1\\x065KZ\\x04\\xf8G\\x01\\xeb\\v\\xa3_\\xdb MXh,r\\xef['\\x8b\\x1f\\x8d\\t\\x9c9\\xcch\\xfeu\\x0e$]\\xec\\n\\xcfs\\xfdW\\x98\\xab*_S1S\\x0eYG\\x9b\\x7f\\x1fj\\x1f\\x06S\\x94\\x1d\\xe4u\\x84\\x05q\\xc8\\xfd\\x95\\x1b\\x94\\x845\\x13\\x15r\\xe8\\x1e\\x9f/f>\\x01z\\x9a\\x8c}N\\xf2k\\xb4\\x96\\x15\\xe7\\x9c֏aT\\xa8\\xbc\\xc5)\\xc0V\\xaaqG\\xac\\xf2k\\x1b\\x8f\\xd6E1Y*~\\x0eɃ\\xe2\\x1e\\x86<\\xfe\\xe46E\\x93PK\\xff\\x19\\xee\\\"\\x8c\\xbe\\xa8y\\xaeHIcR\\xae\\xa6\\x83|\\xda\\xd7\\xc0\\x89\\x18\\xf6\\x80\\xbbDk{\\x82\\x13]\\x8f\\xd1-$\\xba&\\xbf\\a\\xe8w\\x86Jr*\\xa7i\\xfb\\x922\\xbb\\xc7\\xf6D\\xdf\\xf7\\xfe\\xb8\\\\\\xc4v\\xc4w\\x8dC\\xe8\\xeaХ\\xaaZ\\x1f\\xe0\\x1f\\xc9eS\\xafА*V\\xa9\\x8c\\x18\\x98\\xe4}ͥ\\x8a\\t\\x9d\\x846\\f\\aQ\\xb1\\x1e\\x16\\v\\xe8\\xfe\\x94;\\x05\\\\X]\\xb1}\\xb7\\x19\\x7f\\x83\\xa3#\\x9d~N8\\x9c\\xab\\xd6WQ\\xe49\\x92\\xb7\\x9d\\xaeTw?ZH\\xdfOOg\\xfap&\\xdb\\xf7\\xfdݏ\\x11>\\xcf\\n'\\xf2\\xce\\xe1\\x8fC\\xae1\\x90\\xe5@¹`\\x11\\x7f\\xacr\\xb9\\x8f\\x1f.\\xf3g\\xba\\xf7${\\x93F\\x8f\\x9c\\xf7d\\xc7'\\xaf~K\\xb3\\xeaރ\\x17\\xf0\\xfb\\x1f7\\xff\\x0f\\x00\\x00\\xff\\xff;\\xa8N\\xc3\\x13&\\x00\\x00\"),\n\t[]byte(\"\\x1f\\x8b\\b\\x00\\x00\\x00\\x00\\x00\\x00\\xff\\xdc=[s\\xdb8w\\xef\\xf9\\x15\\x98\\xf4a\\xdb\\x19\\xcbi\\xa6\\x97\\xe9\\xf8\\xcd\\xf5:\\x8d\\xfb}\\xebx\\xec4\\xfb\\f\\x91G\\\">\\x83\\x00\\x17\\x00\\xa5h\\xdb\\xfe\\xf7\\x0e\\x0e.$%\\x90\\x84d˛-^2\\xa6\\x80\\x03\\xe0\\xdc\\xcf\\xc1\\x01\\xb2X,\\xdeц}\\x03\\xa5\\x99\\x14W\\x846\\f\\xbe\\x1b\\x10\\xf6/}\\xf9\\xfco\\xfa\\x92\\xc9\\x0f\\x9b\\x8f\\uf799(\\xaf\\xc8M\\xab\\x8d\\xac\\x1fA\\xcbV\\x15\\xf03\\xac\\x98`\\x86I\\xf1\\xae\\x06CKj\\xe8\\xd5;B\\xa8\\x10\\xd2P\\xfbY\\xdb?\\t)\\xa40Jr\\x0ej\\xb1\\x06q\\xf9\\xdc.a\\xd92^\\x82B\\xe0a\\xea\\xcd?^~\\xfc\\xd7\\xcb\\x7fyG\\x88\\xa05\\\\\\x11\\x05\\xdaH\\x05\\xfar\\x03\\x1c\\x94\\xbcd\\xf2\\x9dn\\xa0\\xb00\\xd7J\\xb6\\xcd\\x15\\xe9~pc\\xfc|n\\xad\\x8fn8~\\xe1L\\x9b\\xbf\\xf4\\xbf\\xfe\\x95i\\x83\\xbf4\\xbcU\\x94w\\x93\\xe1G\\xcdĺ\\xe5T\\xc5\\xcf\\xef\\bхl\\xe0\\x8a\\xdc\\xdbi\\x1aZ@\\xf9\\x8e\\x10\\xbft\\x9cv\\xe1W\\xbd\\xf9\\xe8@\\x14\\x15\\xd4ԭ\\x87\\x10ـ\\xb8~\\xb8\\xfb\\xf6OO\\x83τ\\x94\\xa0\\v\\xc5\\x1a\\x83\\b\\xf8\\x9fE\\xfcN\\xc2B\\tӄ\\x92o\\xb8Q\\xbb\\x1aD<1\\x155DA\\xa3@\\x830\\x9a\\x98\\n\\bm\\x1a\\xce\\n\\xc4;\\x91\\xab\\x1e\\xa40J\\x93\\x95\\x92u\\amI\\x8b\\xe7\\xb6!F\\x12J\\fUk0\\xe4/\\xed\\x12\\x94\\x00\\x03\\x9a\\x14\\xbc\\xd5\\x06\\xd4e\\x04\\xd4(ـ2,`ٵ\\x1e\\xef\\xf4\\xbeNm\\xcc6\\x8b\\v7\\x8a\\x94\\x96\\x89\\xc0m\\xc1\\xe3\\x13J\\x8f>\\\"W\\xc4TLw[\\r\\xdb#T\\x10\\xb9\\xfc\\x1b\\x14\\xe6r\\x0f\\xf4\\x13(\\v\\x86\\xe8J\\xb6\\xbc\\xb4\\xbc\\xb7\\x01e\\x91Uȵ`\\xbfG\\xd8\\xdan\\xdcNʩ\\x01m\\b\\x13\\x06\\x94\\xa0\\x9cl(o\\xe1\\x82PQ\\xeeA\\xae\\xe9\\x8e(\\xb0s\\x92V\\xf4\\xe0\\xe1\\x00\\xbd\\xbf\\x8e_\\x90xb%\\xafHeL\\xa3\\xaf>|X3\\x13$\\xaa\\x90u\\xdd\\nfv\\x1fP8ز5R\\xe9\\x0f%l\\x80\\x7f\\xd0l\\xbd\\xa0\\xaa\\xa8\\x98\\x81´\\n>І-p#\\x02\\xa5\\xea\\xb2.\\xff.\\x12u0\\xad\\xd9Y\\x1e\\xd5F1\\xb1\\xee\\xfd\\x80\\x02q\\x04y\\xac\\xa88\\xc6s\\xa0\\xdc\\x16;*\\xd8O\\x16u\\x8f\\xb7O_\\xfbLɴ'J\\x8f7\\xc7\\xe8c\\xb1\\xc9\\xc4\\n\\x94\\x1b\\x87\\xacia\\x82(\\x1bɄ\\xc1?\\n\\xce@\\x18\\xa2\\xdbe͌e\\x83\\xdfZЖ\\xdf\\xe5>\\xd8\\x1b\\xd4:d\\t\\xa4mJj\\xa0\\xdc\\xefp'\\xc8\\r\\xad\\x81\\xdfP\\roL+K\\x15\\xbd\\xb0DȢV_\\x97\\xeewv\\xe8\\xed\\xfd\\x104\\xe2\\bi\\xbd\\x16yj\\xa0\\x18H\\x9a\\x1d\\xc6VA]\\xac\\xa4\\x1a(\\x19;d\\x88\\xa3\\xb4\\xf0\\xdb洈U\\x8b\\xfb\\xbf\\xccq\\x99m\\xff\\x1eG[~\\xb3+k\\x05\\xfb\\xad\\x05T\\xa6N\\xfc\\xe1P_\\xa9\\x9ej\\x1f6\\xcbF\\xfb\\xd4\\x1dE\\xb4m\\xf0\\xbd\\xe0m\\te\\xd4\\xeb\\a\\x1b\\xcc\\xd9\\xc6\\xed\\x01\\x144z\\x94\\t+D\\xd6\\xfaؽ\\x88\\xeeWT\\xe0T\\x01\\x11\\xd2$\\xe01\\xe1\\xe0\\x11&\\x10\\x03I\\x9a`G\\x03ubœ[&D\\xb4\\x9c\\xd3%\\x87+bT{\\x88F7\\x96*Ew#\\xd8\\n\\x1e\\xc0\\x8b\\x90\\x15\\x81xU\\xc3Y\\x81$\\x8f\\n\\x05\\xf1\\xf5\\xe7E\\x15\\xd3VQ\\x86]>HΊ\\xdd\\f\\xben\\x93\\x83\\x82\\xb4z\\xd9\\xf5;$K\\xa8\\xe8\\x86I\\x95\\x12\\x03\\xa9\\xb0kϞwjZZ-\\xe9\\x81\\xec۸\\xcc\\r'\\x91UI\\xf9<\\xc7\\x10\\x9fm\\x9f\\xce:\\x90\\x02\\x1dʸ\\x15Omo\\xbb\\x97@\\xe0;\\x14\\xadI,\\x93\\x90\\xb2E\\xd3$\\x15i\\xa46\\xe3t\\x1fW]\\xa4\\xef\\x1c\\xa5~\\x9c`\\x9a\\x83\\x9d%Y\\xdd5\\xaf\\x84\\x03Q-\\x0e\\x06\\nY\\n\\xb0ۨ-Q\\xbb\\xbeJ\\xb6\\xae\\xef(RȒj(\\x89\\x14\\xa33#\\xbb\\xb4\\x1c\\xb4\\x9f\\xabD\\xce\\xe8\\xf4\\xd0E\\xb7\\x7f\\xf4x\\b\\xa7K\\xe0D\\x03\\x87\\xc2Hu\\x88\\xcc\\x1c\\x94\\xba\\x96\\xa3XGP\\x99ЦC\\t\\xe860\\x01\\x92XN\\xdfV\\xac\\xa8\\x9c\\x87a\\xd9\\x13\\xe1\\x90R\\x82\\xb6\\xda\\x04]\\xe6\\xdd\\xd8&\\xc9\\x1c\\xf9\\xfd$Sڣk3b\\xb5\\x0f/\\xa5Q\\xba\\x96\\xa1\\x86\\xbb\\x96Dm\\xa7{\\x0ft\\x8b\\xffn\\xe4\\xe4\\xb6\\xff\\x7f\\\"6\\x18\\x93\\x13\\x98vB\\xfe\\t\\xba\\x9f\\xd9<=ʷ\\x18ၾ$w+\\x02ucv\\x17\\x84\\x99\\xf0uN\\x12(\\xe7\\xbd9\\xfeĴ9\\x9e\\xe93I\\x93#\\x13g\\\"L\\x9c\\xe2OH\\x174\\x19O\\xdebd\\xd3\\xe4\\xaf\\xfdQ\\x17\\x84\\xad\\\"\\xd2\\xcb\\v\\xb2b܀\\xda\\xc3\\xfeI\\xaa>P\\xe65\\x90\\x91c\\xf5\\b\\xe6\\tLQ\\xdd~\\xb7.\\x8e\\xee\\x92`\\x99x\\xd9\\x1f\\xec|\\xe3\\x10A\\f\\xcd\\xf3\\f\\\\\\x82\\xf12SPc\\x1cN\\xbe\\\"6\\xbb/\\xe8T_\\xdf\\xff|\\x18+\\xef\\xb7\\f\\xce;\\xd8Ȍйv\\xbd\\xb7\\xa3\\xfe\\xfa|T\\x10~A\\x1f(\\x06U.\\xe7rA(y\\x86\\x9ds]\\xa8 \\x96>4tΘ^\\x01&\\x7f\\x90Ϟa\\x87`\\xd2ٜÖ\\xcb\\r\\xae=C\\xc2\\xf5O\\xb5\\x01\\x0e\\xed\\x9a|X\\xec\\xf0d? \\\"0\\x86\\xcfe\\x03\\u05fc($r'閩KB\\v\\xb8?a\\x9bY\\xacҟ\\xa3\\x9f\\xfaD\\x0e\\xf8I;ZZ\\x89\\xa9\\x98\\xcfij@\\x99\\xc9%\\xa8k\\xdf(ge\\x9c\\xc8\\xc9ȝ\\xb8 \\xf7\\xd2\\xd8\\x7f0@\\xd3\\xc8(?K\\xd0\\xf7\\xd2\\xe0\\x97\\xb3`\\xd4-\\xfc\\x9c\\xf8t3\\xa0\\xa0\\t\\xa7\\xe5-\\xc2\\xfa9?g\\xd3,\\xb7E\\xdc3M\\ue10dW\\x1cJ2\\xa7\\xc2\\xf4\\xae\\x9b\\xceMT\\xb7\\x1a\\xd3uB\\x8a\\x05\\xda\\xcc\\xe4L\\x1e\\xdfR\\r\\xd0\\xfd\\xe2I\\xfd\\x84_\\xad\\xb1p\\xbf\\xb8$3\\xa7\\x05\\x94!\\xb2\\xc4\\xec'5\\xb0fE\\xe6|5\\xa85\\x90ƪ\\xf0<\\x8e\\xc8T\\xac~7DZO\\x9e\\xf5\\xee\\xb7\\xef\\x8b\\xe7\\x98/XX\\x93\\xb3\\xf0\\x10\\x8c\\xac3p\\xe0uw9\\xbf\\x9f\\x85\\x95ٌ^\\x81\\x13f\\xbb\\x8e$Gǻ\\xe6 \\xe5\\x05\\xe8@+\\x8e.\\xce,uiY\\xe2\\x11\\x1a\\xe5\\x0fGX\\x94#x\\xe1X\\xd5\\xd0[\\xbb3\\xc15m\\xacZ\\xf8okiQ\\x9a\\xfe\\x974\\x94)}I\\xae\\xf1\\xa4\\x8c\\xc3\\xe07\\x9f\\x87\\xeb\\x81ɘ\\xb2\\xb1SY\\xfe\\xd9Pnm\\xbfU\\xe0\\x82\\x00w\\x9e\\x80\\\\\\x1d\\xf8E\\x17d[I\\xed\\xcc\\xf6\\x8a\\x01\\xc7\\xf3\\x8a\\xf7ϰ{\\x7fa\\xa7\\x9f\\x9d\\xb2\\xafd\\xde߉\\xf7·8P\\x18\\xd1ᐂ\\xef\\xc8{\\xfc\\xed\\xfdK\\\\\\xa9LN\\xcd\\xec6`њ6y\\x1c*\\x92\\xc9\\xfa\\xae\\r8\\xa6\\x9f\\x9b\\xef\\x92\\xf2\\xdeɞ\\xdam\\x16\\x8b6R\\x9b\\xcf\\xe9\\xbc\\xe1\\xc8z\\x1e\\u0088\\xa1g\\x9cȱ\\xcdF\\f>\\x8f\\x16\\xf5\\xbdu\\\"W\\x06\\x94\\xcf%:\\x1b\\x10\\xe2\\x8f\\x17Ff\\xa9S\\x99\\xfebc2\\x90\\xc6\\xfc\\xaeE\\xf0\\f7\\xb9\\x83\\x9b\\x9c%\\x1e\\xe3\\xb0Z\\xbc\\x1c\\xe9\\xed\\xdf~\\xef\\xe53\\xad\\xe4ڿ\\xfb\\x1bym\\x87\\xba\\x90uM\\xf7O5\\xb3\\x96z\\xe3F\\x06\\x9e\\xf6\\x80\\x1c\\xf5պEyε\\xc8\\x1d\\x0f\\xe1\\xf9喙\\x8a\\tB\\x83\\xda\\x00\\xe5\\x19\\x8a\\x92F\\xa6rةVQM\\x96\\x00\\\"\\xa6\\xe8\\x7f\\x04W\\xa2f\\xe2\\x0e' \\x1f\\xcf\\xe0zDt\\x9d\\xd3ٽ\\x894\\x89\\x94\\x8f\\x1f\\x9c\\xc9jdI\\xb6\\x15(\\x180\\xc6a\\xde\\x1d=U!M/eq\\x84C\\xda\\xc8\\xf2'MVLi\\xd3_\\x82&\\xadΥ\\xf5\\x91\\xe4\\xb3\\xeb\\xfe\\xcaj\\x90\\xad9'\\x82o\\xbbi\\x06g\\xcd5\\xfd\\xce\\xea\\xb6&\\xb4\\x96\\xad3\\xe6\\x86\\xd5\\xf1TףwK\\x99\\x89\\xc7V\\x98\\xbf1Ғ\\xa0\\xe1`\\x80,a\\x95>\\xefM\\xb5B\\n\\xcdJP\\xa1J\\xc1\\x91\\x8dI+\\x98+\\xcax\\x9b:%J\\xb5c#`q\\xab\\xd4I\\x01\\xf0\\x177\\xb2\\x97w\\xac\\xe4v\\x88\\xa0̽\\xe3A\\x1a\\x10\\xb6\\\"\\xcc\\x10\\x10\\x85\\xc58(\\xa7\\x92q\\n\\x8f\\fD\\r\\xcb\\xd5sy\\n\\xdc6\\x10m\\x9d\\x87\\x80\\x05\\n$\\x13\\x93)\\xb7~\\xf7O\\x94\\xf1s\\x90\\xcdr\\xde'\\xa9\\x1e\\x81\\x96\\xa7\\xe4h~\\xed\\r' t\\xab\\xf0\\xf0\\xdf\\xe9\\x8e-\\xe3yk\\xb6\\x94#\\x9c\\xb6\\xa2\\xa8\\x00\\x95\\x90\\x18\\xea\\x06\\a\\x9e\\tm\\x80\\xe6\\xf2\\x82\\xf5\\x8aZ!\\x98X\\xe7\\xd1.;\\x11\\xda5\\x87\\ua954\\x1c\\xe8\\xf8)d\\xd7,\\xae\\xdf@\\x13\\xfd\\xdaM\\xf3BM\\xd4\\x11\\xc1\\x1d\\x9b#\\x1d\\xb2)j\\x95\\x16\\xa1\\xc6@\\xdd8\\x91\\x93D\\xb5\\xa2o]Π\\x88\\x8e\\t\\xc3\\xfd*^3\\xbef\\x82e\\xd0v@\\xd7;\\xc1L\\xdfy\\xb4 \\xce\\xea<\\xda\\t\\xa2;pJ\\x86\\xedn\\x00\\xc0\\nh\\x88Cp\\xed\\x91k\\x8ep$\\x97@hYB\\xe9r\\x97\\xd6\\x15\\xf1a\\x89+|\\x1b)nH\\xee\\xeexO0\\x8b\\xb2\\xa1\\r\\x82N\\xccê\\r,Z\\xf1,\\xe4V,0\\x18\\xd7G\\xeb\\x90\\x13\\xb3T/\\x9dޜ\\xac\\x8c\\xe6\\xf5K\\xbe\\x9a\\x9e\\xd3BC~\\xcd\\xe7\\xa9\\xe0?\\x9dA\\xcbd\\xf3\\xcdQ\\t\\x8f).\\x98\\xd3k\\xae\\x00{\\xe4\\xc7\\xd9UL\\xcd?1\\xd8\\x1fJ߸b\\xe9\\x17\\x95\\xc5ݥA\\xf5\\x9c\\xc2m\\x05\\xa6\\x02\\x15J\\xb3\\x17X\\x92^N\\x9e\\x90v\\xc1K\\xac\\x93\\xb3L\\x15\\\\dW\\xfe\\xb9W9\\x87\\xd1M\\xcb\\xf9\\x85\\xe5m\\xda\\xf2d8l$\\x8a\\xd8!geՏ\\xa5=\\x86\\x9c\\xea\\x8bl<\\xf6+-\\x86\\xf5\\x85\\xb1\\n\\\"\\x14\\x18\\xca0\\xb3\\xa7qj\\xbfXX\\xda;\\xdf\\x1f\\x96S`\\xfe/,\\xff\\x0f/=̨\\x94\\xc8Gcn\\x95fDb\\x02V\\x82\\xc1zh\\xec\\xea+|?_\\xe8\\xfbc\\xe1\\xd4@\\xfd\\xa5\\xf1\\x123\\xea\\xc2f\\xa05\\x01g\\xaf\\xde\\x04\\xadA\\xab\\x9d+\\x10\\xed\\x80\\xcf\\x19\\xda\\xf1ׅ\\xbb\\x05\\x11\\xc0\\xa4\\xf8\\xf5k\\x05A|}\\xf5>\\xd3\\xe4\\x9fI%\\xdbDU\\xdf\\x04\\xcaf\\xaa;\\xe67<(\\xf4\\xf0\\a\\n`\\xe8\\xe6\\xe3\\xe5\\xf0\\x17#}\\xd9\\af\\xd1\\x12\\x800(\\xea2\\xb3L\\x94l\\xc3ʖ\\xf2 \\xb5\\xdd\\x1d\\x02\\xc7@\\x1d\\x9f%\\xa0IE\\x04\\xe3\\x8e\\x01\\xc3\\xf8\\x01Ñ/\\x8d;\\x969Z\\xc5M\\xfb\\xa2y\\xd5!'ׄ\\fk>F\\xac\\xe1\\xb1\\xc7\\x17\\xafR\\x05\\xfb\\x87\\xd4z\\x1c_\\xe1\\x91\\x13I\\xccTs\\x9cPÑY,\\xf6\\xe2\\xf3\\x96\\x9c*\\x8dcb\\xee\\xb3Ud\\xbc~\\x1dF\\x16~\\xe6k.\\x8e\\xc1\\xce\\xd9\\xeb+ް\\xaa\\xe2mj)2+(^\\xaf\\x142/\\xfa<\\xa9\\x14`>`\\x19\\xaf\\x82\\x98\\xad}xQ@sҖfk\\x1a\\x8e\\xa9d\\x98\\xa5N\\x9e\\x98\\xbdY\\xad\\u009bU(\\xbcm]\\xc2$\\x17M\\xfexL\\xe5A\\x8c\\x93~\\xa1M\\xc3\\xc4\\xfa\\x90)rYg\\x92m\\xe6Y\\xe6~o!\\x03\\x9e\\xe9\\x873]t8\\x12\\xfa\\xba\\xeb҉H2\\xa4-\\x990\\xf2\\x92\\\\\\x8b\\x9d\\x87\\x9b\\x80\\xd3\\v\\x1f\\x854\\a\\x17\\xd9첶\\x8c\\xf3\\xfem-\\x04;\\r\\xcaߙԴv\\xab\\x1a\\xf3\\xf6\\x93t\\x95j\\xe0\\x94\\x9f\\x148~ك\\xd1ώ\\xbe\\xa5\\xe7_\\xb7ܰ\\x86\\x83\\xf5\\xe86\\xacL\\xde!3\\x15\\xec\\\"\\x92\\xff&\\xf1\\x86\\xd4r\\x87\\x90\\xbe]\\xf4\\x9eQ\\xec\\x9eq\\x926\\xbf\\xc9\\x13\\xb6\\x97Q\\xcc~\\\\\\x11{\\x06\\xcdrE\\xf1\\r\\x8b\\xd5߰H\\xfd\\xad\\x8b\\xd3g8k\\xe6\\xe7\\xe3\\x8a\\xd0O>\\x81\\tG\\xfd\\xf7\\xb2\\x84\\a\\xa9\\xcc\\\\p\\xf2\\xb0\\xdf?q\\x92\\xda\\v\\xd8$/\\x89\\b]\\x13\\xbb\\xc4\\x10Ç\\x17\\xa7m*}\\xe8\\x19\\xdc\\xe9_di\\xd76w\\xc6\\xf2\\xb8\\xd7\\xfd\\xe0\\xae\\xf2\\n\\x14\\b\\xf7\\xcc\\xc7\\x7f>}\\xb9\\x8f\\xf0S>\\xaf\\xf7\\x8c\\xf7\\x9e\\x97p\\x1eL\\xe9\\x91\\xe3\\x8f\\xe6|1\\x93\\xc3\\x16\\xfa\\x00\\xaf|.B\\x1b\\xf6\\x1f\\xf8\\xaa\\xdb\\v\\xd2A\\xd7\\x0fw\\b#\\xf8i\\xf8L\\\\\\xac\\xa2\\x88'\\x96K\\xb0\\x16+\\xa2jT,\\xeeV\\x03\\x88Ê\\xdf\\xfe3JP\\xba'\\xb3\\x82\\xc5d\\xa1\\xc6\\xcb\\n\\xdeÝ[\\xc7\\xd8,\\x9f\\xac\\xd3(vD:\\x8e\\xac\\x98*\\x17\\rUf\\x87l\\xa3/\\x06k\\bff*\\x9d3\\xaaX\\x0f\\x9f\\x01K\\xa27\\xbc\\xfe\\x85g\\x91\\xbbfxڻ\\x8f\\xbbS\\xd61~\\xffd\\xf6\\xe6\\xc9+\\xaec\\xdcb/\\x10S\\x89\\xcf\\xc9\\x02\\x93WK\\x93yM\\xf4\\xf0\\xed\\xa4\\xb4\\xcbc\\x1c=\\xad\\xe7l\\x14\\x1dRM\\t0v<\\xaa:-h\\xa3\\xabēK/\\xd3u\\xf8\\x1a\\x99\\xa1\\xa6}\\xc9&\\x1d\\x80\\xc1>YQ\\xf5\\xb4\\xd5\\x16\\x82>\\v\\xdbFi\\xc5a)\\xddnm\\xb3\\xab{a\\xfc\\xa2\\x97)x\\x9b#\\xe1\\xcc\\xe7\\\\N~\\xc8šgD\\xfd`\\xf6˪\\xb6CL\\x9dp\\x18<\\xeb\\xdae\\x14\\x19O;\\xb1\\x99π\\xe4\\x19\\x8c\\x13\\x9e\\xfe@|\\xe5\\xe2\\x8a$_\\x04\\xc9|\\xf5\\xe3\\x0fE\\xf4\\x84V\\xd3E\\x05e\\xcb\\xe1\\xd47\\xff\\x9ez\\xe3\\xe7_\\xfd\\v\\xb3e\\xbc\\xfbg\\x91\\xdd3\\xd0\\xd6g\\x1e\\xbe/\\xe8)\\xe1!\\xf7)9\\xe6\\xf0ap\\xe0\\x9e\\x17+\\xdcK\\x94E\\x01Z\\xafZ\\x1e\\xaa\\x94\\n\\x05\\xd4@\\x19\\xba3\\x1dW|T\\x9dM\\xdbpIKP7R\\xacX\\xe2\\x84d\\x80\\xd6\\xff\\x1at\\xde\\xe3\\xd9\\x02?\\xb6\\xaa{\\xdaq\\xf2Y\\xbc\\x17i\\xae\\x86*\\xca9\\xf0O\\x8c\\x83\\xfeYn\\x85]W\\x86@>\\xa4\\xc6\\xf5\\xeee\\x15\\xad\\xb2f}GD[/\\xad\\x93\\vƌ\\a\\x8b+\\xa9\\xa6+\\xa4\\x1dޙ0\\xb0\\x86T|\\xbdU\\xcc\\xc0SC\\x95\\x06\\\\Q\\xc6\\x0e~\\xdd\\x1b\\xe2\\xa2\\xcf\\x15\\xa7kW\\nW\\xb2\\x82\\x1a\\x88\\x06\\x18g\\x18[>\\x8e\\xd7\\b\\x8b\\xef\\xb02I\\x8e$\\xbd\\xb2\\x85z\\xecJƨX\\x8f=/\\x9a0\\xd5\\xc9\\aF\\x9dE.hc\\xf0\\x02\\f\\xd2\\x11\\x89h<\\f|\\xb4w\\xef\\x8d\\xd1\\x01\\xd8qN\\xf3e̾`N\\x1bZ'\\xa2\\x84y\\xbdss\\b\\x06\\x9f\\x05Ve\\xaf\\xee\\xae\\xff\\xc0b,\\xb0#[\\xaac1u\\xd2\\xf7\\xee`;0\\xe8\\xaa[\\xd0P\\x12\\u0600 V\\x14)\\xe3PNq\\xeaWL$\\xab\\r\\xa8\\x9ft\\x84\\x83\\x95\\x80\\x96ş\\fU&.\\xfdЏYIUSsEJj`aG\\x9f溥\\x9fIU\\xea\\xc4\\xe3@\\xbc\\xd9\\xe6ţ\\b\\xd7n\\xac\\xf5s\\xf7\\xd1jК\\xaeC\\x10\\xba\\x05\\x05d\\r\\xc2\\xe2=\\xe6\\x16\\x93\\x1eS\\xb8\\xd2\\xe7\\x8dE,-\\xb5(\\xa4\\x85i\\xa9\\x9f\\xc0\\xb9p\\xf1\\xf44\\xbcO\\x8cQ\\xeczTE\\xa7U\\x85\\xbf<\\xf8\\bT\\xef?w}\\x80\\x8bO\\xfd\\xbe>I\\xecv\\xec\\xceF\\xa8+\\xf0\\xc4\\a\\x8f\\r\\x8b\\x91uJ\\xa6\\x8dę\\x8f2'\\x95\\x94\\xcfYn\\xf6\\xe7رK'1\\xe1X\\t\\xafL.ekz~\\x8eGxb\\x99\\xf8\\xfc\\xe7+\\xdb\\x17\\x84y\\xed.P\\x8d\\xe5V\\xf3<\\xbd\\xcf\\x03H1\\xbc\\x95\\x86\\xf2`d,_\\xc6\\x0e\\xd5\\xc4\\x03\\x02O\\xe1\\xf1d\\xcew\\x17\\xfb\\x90\\xf7^e\\xef`W\\xddS\\x9e^\\x13t\\xd7\\xc7\\xc7Ҫ>\\xeb\\x97\\x04\\x12_\\x01\\xed|\\x92\\xb17\\x17\\xe7\\xec\\x1fB\\xfd\\x84\\x8b\\xca\\xc0\\xf1\\xe7\\xae\\xf7\\x18\\x1e\\xdd2\\x9d\\xc3\\f\\\"\\x1di\\x12\\f>L\\x15%ㄥOx\\xa9ME\\xf5\\x9c{\\xfa`\\xfbD\\xb7\\xa3g\\xae\\xa2\\x13\\xfa8\\\"\\x95\\xe9{\\xae\\vr\\x0f\\xdb\\xc4W\\x87,<\\xfdB\\xa9Jt\\xb9\\x13\\x0fJ\\xae\\x15\\xe8C\\xa6[\\xe0}F&֟\\xa4z\\xe0횉/\\xe3\\x95\\xdfS\\x9d\\x1f\\xa82\\xcc2\\xad[Ob\\xecM\\xb0q\\x89\\xdf\\xe6G\\x8f\\xff\\xc0\\x04\\xe5\\xec\\xf7\\x94.\\xef\\xff87Ä\\xbek<\\xf2N\\xb1P\\x01\\xf1s\\n\\xd0k\\xe8\\x9ft\\xcf\\xfc\\x84y/ɽL\\x8a\\xb1? fC\\xa0L\\x93%h\\xb3\\x80\\xd5J*\\xe3\\xf2\\xf7\\x8b\\x05a\\xab\\xe0 Y\\r\\x81q\\xa2{͞\\xb0T\\xe2=\\x1e\\xbd\\x05\\x87e\\xe5S\\x89\\n\\xad\\x0e\\x86\\x9c5ݹ\\x8c$-\\n\\x1b\\x13\\xc0\\amh*6y\\x91\\x9e\\xc6P\\xd5\\xcbJ\\x8e\\n\\xb9\\xeb\\xf7\\x8f9\\xbe\\xa8>\\x10\\x9cC\\x1d^gw\\x06\\x9d\\x8f\\x9di\\r^\\xcb \\xdab\\xef\\x14eB\\x9c\\x1a\\xbb\\x1b\\x0f\\xbb\\xf3L\\xcd\\xd7\\beL=\\xfa\\xfd\\r\\x1e\\xe2\\xf6\\a\\xac\\xbe\\x93%[QQ\\xb1\\x1e\\xbd\\xd0V)ٮ\\xab\\xc0\\x9bc\\x0e\\x11)[\\x8c\\x9c\\x1bT\\x05:\\xfc\\xc7!\\xa6U\\xa2wh\\xe7k,ƴt\\\\\\uee0f\\xf2\\x02E\\xad\\xba\\x8b-\\x9d\\xaa\\x9a\\xb0\\xf9\\xd9Y\\xc2\\x11\\x88\\xb3\\xb6?\\x01\\x91\\xea\\x9d(&\\xaf\\xe0\\xf8@\\x9bM\\xdc՝\\xc2P\\x12\\tQ\\x1b\\xbf\\x1a\\x12\\\"\\xc41$\\xf4}\\x89.\\xe2\\xf9a02棜\\x88\\x8ei'\\x06\\xb78\\rj~\\xd3}'h\\xe8\\xee\\x1c\\x87\\x0e=\\b\\xfeNJ\\xbb\\r \\x1c\\x13\\xf9\\xe2\\xdc\\xe9\\xb8\\xf7ǍX7\\xd1ۺ=9v\\xfd\\xb6\\ac\\xef\\n\\xa4\\x8db\\xbbiB\\xbc\\xf9\\xf7l\\x95\\x92\\x17\\xf7\\xbf3-9\\xfc\\xc3\\xc1\\xafo|\\x95qK\\x95`b}\\x12F~\\xf5c\\x13\\xf1\\xbc\\a{Έ>\\xac\\xfc\\xd5b\\xfa\\xa4Y:\\xf8\\x88\\f^\\xf6\\xf0\\xecg\\xf2_\\xfe/\\x00\\x00\\xff\\xffP\\a\\xb5\\x16Cm\\x00\\x00\"),\n\t[]byte(\"\\x1f\\x8b\\b\\x00\\x00\\x00\\x00\\x00\\x00\\xff\\xec=]s\\x1c)\\x92\\xef\\xfa\\x15\\x84\\xeea?B\\xdd^\\xc7}ą\\xde|\\xb2gO\\xb1\\x1e[ai\\xf4\\xbctU\\xb6\\x9aQ\\x15\\xd4\\x00\\xd5r\\xdf\\xde\\xfe\\xf7\\x8dL\\xa0\\xbe\\xba\\xe8\\xa2Z-ygǼت\\x86$\\xc9L\\xf2\\x03\\x12X,\\x16g\\xbc\\x12\\xf7\\xa0\\x8dP\\xf2\\x92\\xf1J\\xc0W\\v\\x12\\xff2\\xcb\\xc7\\xff6K\\xa1\\xdelߞ=\\n\\x99_\\xb2\\xab\\xdaXU~\\x01\\xa3j\\x9d\\xc1{X\\v)\\xacP\\xf2\\xac\\x04\\xcbsn\\xf9\\xe5\\x19c\\\\Je9~6\\xf8'c\\x99\\x92V\\xab\\xa2\\x00\\xbdx\\x00\\xb9|\\xacW\\xb0\\xaaE\\x91\\x83&\\xe0\\xa1\\xebퟖo\\xffk\\xf9\\x9fg\\x8cI^\\xc2%3\\xd9\\x06\\xf2\\xba\\x00\\xb3\\xdcB\\x01Z-\\x85:3\\x15d\\b\\xf4A\\xab\\xba\\xbad\\xed\\x0f\\xae\\x91\\xef\\xd0!{\\xeb\\xdbӧB\\x18\\xfb\\x97\\xde\\xe7\\x8f\\xc2X\\xfa\\xa9*j͋N\\x7f\\xf4\\xd5\\b\\xf9P\\x17\\\\\\xb7\\xdf\\xcf\\x183\\x99\\xaa\\xe0\\x92}®*\\x9eA~Ƙǟ\\xba^0\\x9e\\xe7D\\x11^\\xdch!-\\xe8+U\\xd4e\\xa0Ă\\xe5`2-*K#\\xbe\\xb5\\xdcֆ\\xa95\\xb3\\x1b\\xe8\\xf6\\x83\\xe5g\\xa3\\xe4\\r\\xb7\\x9bK\\xb64ToYm\\xb8\\t\\xbf:\\x129\\x00\\xfe\\x93\\xdd!n\\xc6j!\\x1f\\xc6z{Ǯ\\xb4\\x92\\f\\xbeV\\x1a\\f\\xa2\\xccrb\\xa0|`O\\x1b\\x90\\xcc*\\xa6kI\\xa8\\xfc\\x0f\\xcf\\x1e\\xebj\\x04\\x91\\n\\xb2\\xe5\\x00O\\x8fI\\xff\\xe3\\x14.w\\x1b`\\x057\\x96YQ\\x02\\xe3\\xbeC\\xf6\\xc4\\r\\xe1\\xb0V\\x9aٍ0\\xd34A =l\\x1d:\\x1f\\x87\\x9f\\x1dB9\\xb7\\xe0\\xd1\\xe9\\x80\\n»\\xcc4\\x90\\xdcމ\\x12\\x8c\\xe5e\\x1f\\xe6\\xbb\\aH\\x00F$\\xaaxmH8\\xda\\xd67\\xddO\\x0e\\xc0J\\xa9\\x02\\xb8\\x80vX4\\xb6\\nu%\\xa0\\x80\\xe6\\f\\xddN\\x8d\\x16FH\\xb6\\xae\\xd1#]2\\xd4\\x12Q\\x19\\x11\\xd2X\\xe0\\x11a>\\x01\\xef\\xe0kV\\xd49\\xe4WEm,\\xe8\\xdbLU\\x90\\x87E\\xa6Q͜\\xca\\xc3\\x0f\\a!\\xfb\\xf8\\xa5\\x10\\x19 \\x1f2WiA\\x8b<1\\xd1nC\\x99]\\x05n\\xcd\\tY\\xed\\x87\\xd0\\xc6(\\x93\\xbaŀņ\\xe7\\x7f<\\xbf \\t\\xe8\\xf7\\xde\\xef\\xc70\\xae\\xa1!\\xd3,\\xddL\\x16\\x7f\\xbc\\x85\\xb0PF\\xa8;\\xa9\\xa3f\\xf0\\x9dk\\xcdw\\a\\xb8\\xde,\\xa6\\xbd\\x00\\xdfc\\xb0\\a\\x9c\\x97\\xa1\\xda7\\xe2\\xfd\\xb0\\xff\\xdf\\\"\\xf7O\\xcboC\\x8b\\xce\\\\H\\xe4s!\\x8c\\xed\\xb1ٸU,$\\xebX\\b\\xe9\\t$\\x1dLT\\x93S\\\\\\xfd'!\\xe6I\\xe7Nl\\xb24\\xb2\\xe9'\\xc0\\xbf\\x14%7J=\\xa6P\\xef\\x7f\\xb1^\\xbb\\x84\\xc52\\xda\\x18a+\\xd8\\xf0\\xadP\\xda\\f\\x97I\\xe1+d\\xb5\\x8dj\\x16nY.\\xd6k\\xd0\\b\\x8b\\x96\\xf9\\x9b]\\x81C\\xc4:\\x1c\\xbe\\xb0\\x8eʊV\\x18\\x8c\\xabe:\\xb2\\x94\\xa8\\x11\\x1b\\n\\x05\\xa8Q\\xa8\\xce\\xc1\\xc1Ђ\\x1c\\x88\\\\lE^\\xf3\\x82|\\t.37>\\xde\\xe0\\x17\\xd3j\\x13\\x02\\xb1\\x87\\x7fT\\xaa]q\\x0eM\\x18$2\\xb1\\xb7\\xea\\xa5$\\xa0\\x8f_bl\\xb4_5N\\x89\\xb0\\x94p\\xb0od\\xa6\\xae\\v0\\xbe\\xbb\\x9c\\xdc\\xe4V']\\xb4\\xccrk\\f\\x05_A\\xc1\\f\\x14\\x90Y\\xa5\\xe3\\x14J\\x91\\x03WR\\x95n\\x84\\xb8#Z\\xb6\\x1fm\\xb5\\x83\\x99\\x00\\xcb(\\xc4݈l\\xe3\\xdcW\\x144\\x82\\xc5r\\x05\\x86VExU\\x15\\x11\\xd3ՖI\\xe1\\xf0\\x9dM鍶$h\\x90!ܘ.iK\\xa2~n\\xcb(\\xd9۹٧\\xfa\\xf8:\\xff(\\xbe\\xbf%\\xa2\\a\\xabs\\xa4\\xb0Oh\\x12F\\xfb\\x05\\xc9\\xf3!Jz\\xa4\\xb8\\x00\\xb3\\xec\\xac\\xce\\t\\x1b\\xbe\\xa60\\xb4\\xe7?\\xeem\\xa5\\xec\\x11\\xe5\\xd7Ż\\xe3&\\xcc\\f\\xd6MΩ\\x97e\\\\\\xd3Ϳ\\b\\xdf\\xc8d\\xddz\\x8b5\\x8bg\\x1f\\xbb-/hW\\xc03$\\xbf`kQX \\xa7j\\nQ6\\x83s\\xa7$P\\xaa\\x05f\\xb4Il\\xb3͇f\\xef(\\xa1ŀVC\\x00\\xceA\\x0fQ\\x0e\\xf1 \\x01$k\\\\\\v\\xda4\\x15\\x1aJڌ\\xa5H\\xb2\\xfb\\x85\\\\\\xc1w\\x9f\\xde\\xc7c\\xcfnI\\x94ԽA%LZW\\xde\\r\\x1c\\xa3.\\xae>T\\t\\xbf\\x90\\xbf\\xd6\\x04\\x82n\\x13\\xfe\\x82q\\xf6\\b;\\xe7bqɐo)\\x8b\\xff|\\xf8*\\fv,s\\xf6^\\x81\\xf9\\xa4,}yQ*\\xbbA\\xbc\\x06\\x8d]O4A\\xa5\\xb3$H\\xc4n\\U00088ce5(\\xa8\\r?\\x84a\\xd7\\x12C2G\\xa2\\x19\\xddQ\\xae\\x90\\xeb\\xd2uVֆ\\xb6Z\\xa5\\x92\\v\\xb7,6֛\\xe7\\x81\\xd2=\\x16\\x9c\\xa4c\\xdf\\xe9\\x1d\\x1a#\\xf7\\x8b\\xcbZ*x\\x06yآ\\xa3t\\x1an\\xe1Ad3\\xfa,A?\\x00\\xab\\xd0,\\xa4K\\xcb\\fE\\xedG6_\\xbc\\xd2=\\x87n\\xf9\\xbax\\xacW\\xa0%X0\\v4k\\v\\x0fŪ2\\x91.\\xde&\\x8c䜌\\x95\\x05\\xce\\xf5ĚAZ\\x92\\xaaG2r\\x0eWO%\\xd63\\xc9D^\\x04\\xb9]IR\\xd0Ml\\x9dg\\xbdf\\xca\\xcd1*\\xa63\\x16\\xe7\\x02\\x94\\x9c\\xb6\\xd6\\xfe\\x86\\x96\\x9ef\\xe3\\xdfYŅ6K\\xf6\\x8e2{\\v\\xe8\\xfd\\xe6\\x17&;`\\x12\\xbb\\xadh\\x95\\xfd\\x97Zly\\x81\\xfe\\a\\x1a\\bɠpވZ\\xef\\xf9j\\x17\\xeci\\xa3\\x8cs\\x1b\\x9aM\\xbb\\xf3Gع\\x1d\\xe5\\xa4n\\xbb\\n\\xeb\\xfcZ\\x9e;_fO\\xf14\\x8e\\x8f\\x92Ŏ\\x9d\\xd3o\\xe7\\xcfu\\xeffH\\xf4\\x8c\\xaa=Q.y\\x95.ɔ7;'\\xd0\\xc0`=8DظI \\xc5\\x00a\\x8a\\x02ɢ\\\\)\\x13I\\x16\\x89\\xa0\\x95 \\xe87\\xcaX\\xb7\\x0e\\xd9\\xf3\\xf7G\\x17*UX\\x9cd|mA3c\\x95\\x0e)\\x99\\xa8\\xf8S\\x96\\xe2\\xbb\\xe5n\\x03\\x06\\xfc>\\x94_\\xf4t\\x801\\x8a=ou\\x83\\xb3*\\xe7n/\\x8c:\\xe2\\x19yOԶ\\xd2*\\x03\\x13͋hK\\xa2m\\xeaQp\\x9f\\x0eͺ.w\\xd1\\xdf:Ik\\xa7,J\\x872ϑG\\xd2\\x1d\\x11\\x19}\\xf8\\xdaY\\xa2F\\xed\\x82\\x7f\\xa7H\\xeb182:\\xafQ\\x96|\\x98\\x0e\\x9c\\x8c\\xee\\x95k\\x1d\\xe6\\x98\\a\\xe6\\xc2-\\xfdP\\x93Ι\\xe3u4\\xa2\\xfc\\xcf\\xe6ڔB^SG\\xec\\xed\\v\\xbaC^\\x8b\\xc7ң\\xc6\\xca\\xf1N\\xfaU\\xe8\\xac\\xe5^\\xf3\\xc1\\xe7\\xd4)\\xda\\xf8\\xd1\\xd0c\\xee\\xfe\\x9e\\by\\xd7R\\xd9\\xce2\\xceL'\\xbaR\\xf9\\xef\\f[\\vml\\x17\\rs \\xb1j\\x14\\xd4\\x11\\xa1\\xa7\\xfc\\xa0\\xf5ё\\xe7g\\u05fa\\xb3\\xa0\\xb8QO>qzN\\xbc\\x1dH\\xba\\xe1[\\xf0\\x99\\xab 3UKZ\\nC=\\x80\\xdd̀\\xe8X\\xe3\\xac@\\xa2\\xbd\\xeb4\\x96u\\x99N\\x90\\x05I\\x92\\x90\\x93\\xebf\\xdd&?p\\x91\\xb6nŎc\\xab=\\x94\\xc39V\\x8e\\x9fG!\\xc1\\xb3\\x9bN_\\U000af8acK\\xc6K\\xe4!\\xb9\\x1d\\xa2\\x84&\\xa3ޱ\\xbbI\\xfb\\xc4\\x16d\\xb4\\xac\\xc2YV\\x15`\\xc1\\xa7m\\xce\\xc0#S҈\\x1c\\x1a\\xd3\\xefE@I\\xc6ٚ\\x8b\\xa2\\xd63\\xb4\\xeal\\x92\\xcf\\r¼69}d\\x95\\x8eȂH\\x94\\xb8\\xce>\\xc3\\v\\x9e\\xd6\\xf8\\x95\\x9e\\xe7Ǧ8\\x8c\\x1a\\xe6\\xfb\\x8b\\x95\\x16\\xca\\x1d\\x068\\xbd\\xcb\\xe8ӎ\\xb9\\xdc}\\xf7\\x19\\xbf\\xfb\\x8c\\xdf}\\xc69\\x1d}\\xf7\\x19'\\xcaw\\x9f\\xf1\\xbb\\xcfx\\xb8|\\xf7\\x19S\\xcaw\\x9fq&\\\"\\xdf\\xcagL\\xc1pAk\\x9c\\a*$a\\x95\\x98\\n1\\x85\\xf6D_>\\xe9ǟ\\xd58I.\\xf3\\xf58ȑC<\\x91\\xe3\\x171\\xaf\\xa35^Mr3\\xce\\xc00w\\xdc)\\xca\\x04\\x87\\xf9\\x04\\xa7g\\x02\\x02\\xa7?=s}\\x10\\xf2\\tO\\xcf\\xf8!\\xa4E\\x18G\\x9d\\x9d\\tD\\x9a\\x7fz\\xe2\\xc2'\\x11\\x95\\xc0\\xc3V\\x8aK\\xff\\x88\\x8d1&I\\tx|\\xe3\\xe4\\xf7\\xbd\\x8c\\xc9\\x17\\x90\\xa5W9\\x913K\\x9eFY\\x7f\\xfe\\xc7\\xf3_\\a\\x8bN˔(\\x1b\\xf6i\\xeb\\xd4xL?b,\\xdfM\\x8d\\xecg\\xa9\\xfez\\xa6\\xc2Ie?\\xf5DMC\\xe4\\b\\xbc\\xbeX\\x0f\\xa8\\xfck\\xd27\\x16\\xcaϕ\\xb7\\x96'8a\\x7f=\\x02/\\xe9\\x8c=7;\\x99m\\xb4\\x92\\xaa6~M\\ba\\xbd\\xcbܽ\\x03\\x01dL\\xd8G5\\xc8\\x7f\\xb0\\x8d\\xaa#\\xa76&H\\x9b\\x90E\\x9bF\\x90^R\\xadO\\x8c\\x00˷o\\x97\\xfd_\\xac\\xf2)\\xb6\\xecI\\xd8M\\x04\\x18\\xddG\\xc1\\xf3\\x1c\\xe3\\x82\\u0381\\x1e\\xaf\\a\\xc2UIC\\xa1\\x8c\\x00S\\x9aIQ8\\x89\\r\\x10z\\xf2\\xca>Wnu\\xf0h\\xbfiz\\r+=\\x11wn\\xfam\\x93-9\\xed\\xbe?#\\xe9\\xf6\\xa4G\\xa3\\xbeYZ\\xedqɴ\\xa9+\\x94\\t\\x89\\xb3\\xe9\\xe9\\xb2)lu%=I69BNM\\x88\\x9d\\xbb\\x02\\xf1\\xa2ɯ/\\x93\\xf2\\x9aL\\xb3\\xb4\\xf4ֹ\\x14{\\x95T\\xd6WN`}\\xbd\\xb4\\xd5\\x19ɪ\\xa7?\\xf5\\x92\\xbe\\x96~tveڲ\\xcc\\xe1\\x84Ӥ4Ӥ\\xa5\\x9b\\x94\\x01\\x1f5Ԥ\\xf4ѹI\\xa3I\\x9cL\\x9f\\xae\\xaf\\x9a\\x16\\xfa\\xaaɠ\\xaf\\x9f\\x02:)m\\x93\\x15\\xe6&y\\x8e_r\\x18ʴ\\x03P|\\v\\xe1|.\\x99\\x94\\xee\\xb9\\xe6ϊ;?\\x0f`\\xa1\\xb0\\x047\\xf5\\x15〲.\\xac\\xa8\\x8a\\xf6>\\xb6X\\xc0\\xb9\\x81]sY\\xd1ϊ\\x8e\\xc8\\xfb\\x9b\\xba>\\x7fi$~9\\x88j\\xb8aOP\\x14\\x8c\\xc7\\xe6\\xe6\\x1e\\x152w\\x0fh\\xa6\\x16\\x80\\xb6\\x11g\\xb9\\xbf\\x8c\\xc9_\\x1ez\\xe1\\xa6\\v\\xdd\\x06@\\x16\\xb6\\x8c-\\xf5qy\\xf8\\xa6\\xaf\\x83\\x06,U\\x8f\\xedy\\xe6.ޠo\\xbfԠw\\x8c\\xee\\x1dk|\\xb3\\xf6P\\xa9\\x9f\\xe8\\x06\\x03Ӡ~\\xbc:<\\xb4g\\xb2\\x17\\xe0\\xb4ꁽ\\x93\\xce#\\x18\\xe2DmP\\xef\\xb4\\x01\\x1d*U\\x8cӢ\\xfdD@H\\xd5@\\x884Mq\\xfe眲|\\x89\\xf0\\xee\\x14\\x01^\\x92\\a4\\xcf{\\xfd\\x86\\xa7'\\x8f=5\\x99\\x9e\\x8c\\x92tJ\\xf2%½9\\x01\\xdf,\\x7f5\\xfd\\x14\\xe4\\xfc\\x8d\\xe7\\x17>\\xf5\\xf8R\\xa7\\x1dgP/\\xf5t\\xe3|ڽ\\xd2i\\xc6W?\\xc5\\xf8\\x9a\\xa7\\x17g\\x9dZLNϚ\\x95q0'\\xb5\\xea\\x19\\xc7\\xed\\xd2r\\t\\xa6O!&\\x9e>L\\xcc4H\\x1b\\xfc\\x91\\xc3N<]8\\xffTa\\\"\\x7f\\xe7L\\xe9W>=\\xf8ʧ\\x06\\xbf\\xc5i\\xc1\\x04\\tL\\xa82\\xffT\\u0cf7\\xa4\\x94\\xceAOn\\xfb͑\\xdaIyM\\x8d\\xe5\\xfa\\x88\\r\\xf6\\xb5\\xc2m\\xb2X\\xab\\x17\\x03\\x90Y\\xf2\\x17\\xf9ӣ\\r\\x87\\xb6\\xc1Q2;\\x1eQo_\\xb2u\\xd7\\xfa\\x0e\\xb1\\x7f\\xcd\\xc1m]\\x1a\\xa88\\x1a\\x00\\n\\xdc(5+\\xea*|\\xe0\\xd9f\\xd0Æ\\x1b\\xb6V\\xba䖝7\\x9b\\xc5o\\\\\\a\\xf8\\xf7\\xf9\\x92\\xb1\\x1fT\\x93\\xabӽ/͈\\xb2*v\\x18\\x89\\xb1\\xf3n\\x83\\xe7IIT:C\\xcf7\\xaa\\x10Y\\xc4\\xe7\\x1c\\xbdW\\xcf5ػl\\x88n\\xfe\\xcb:\\xd9\\\"\\xb1\\xc0\\a\\x9b\\x8bp\\xebb\\xffJfw\\x9f\\xfb\\x91k%\\xbc\\x12\\x7f\\xa6'\\x95N\\xb0\\xea\\xf6\\xee\\xe6\\x9a`\\x051\\xa2\\xb7\\x9a\\x9a\\x04ņ\\xe5+@\\x97\\xa1\\x1d\\xfb!}r\\xbd\\xeeA\\xed\\xe7\\bw\\x1f\\xab\\x80ܽL\\x12\\xdc\\x16\\xaf\\x9a3\\x85Z\\xeb\\xe6\\xda\\xe1r\\xa8'\\x94/.wL\\xf9\\xa7'\\x84\\xce\\x17\\x15\\xd7v璉.zx\\x04\\xbb>\\xb5jv\\xd0Z\\xed\\xbf\\xbc\\xd2-=\\xb2\\x87GWh'{W\\xf5\\x93\\a\\x86\\xf4|\\x0eN\\x87OUO\\x9e\\xa7~\\x01\\x9c\\x0e\\xbbP\\v\\xa2b\\xe4\\xa7h\\x06\\xe4\\xc9W,\\x8d\\xbf\\xa1\\xffG\\xb5\\x85\\xf7ѕ\\xcb\\xfe\\xeb+\\x83&#\\xa9\\x89\\x01*]2\\x1f\\xa1`\\x9b\\x8fHw|?O\\xed\\xc5s\\r\\x03*\\xfe\\x8e\\xf0\\xe7,N\\xde\\xf6A\\x8d?HB7\\xa8\\x87Nc^\\x15=\\xf5\\xb4c7\\xf7\\x14\\xb76\\xaa\\xd4O}\\x1f\\xb7\\x86\\xe5ɐ`\\x10\\x81%\\xe4\\xc17ZNEF\\xab4\\x7f\\x80\\x8fʽ\\xad\\x93\\\"&\\xfd\\x16\\xbd\\x97\\x97\\xbc\\xe7\\x16\\xf2\\xb5\\xfd$\\x8c)z?\\xb6!\\xc0\\xf6|\\xc6\\xdeE\\xff\\x88\\xed\\x91O\\x19X[\\xba\\x91ғ&\\xef\\xfd\\xeb$\\xa8\\x8f\\r \\v\\x02\\x05\\x1c\\xb4\\x15\\xfew\\xa3\\x9e\\xe8\\x02\\xfc\\xf8\\x1asx@\\xa4\\xf3\\x86\\x19\\xd0A\\x11J\\xe1=j\\x98uU(\\x9e\\x83\\xbe\\xa2GT\\x12F\\xfcS\\xaf\\xc1\\xc0\\x1d\\xe8?\\xc5\\xe2\\xedfd<\\xa1\\xe7\\x17̒A\\x8f\\xae(\\xa0\\xf8A\\x14`\\x1c≦\\xe1f\\xbfec)\\xear\\xe5<\\xd55\\xfe\\xd8tr\\xc02\\xbb\\xa1\\xd2\\x06C\\x05\\x1a\\xfdD\\xb7\\x15Q\\x9b \\xf9\\x87\\x89\\xc1\\x1a>\\ni\\xe1\\x01\\xc6c\\xe8\\t\\x9b\\xe0\\xdeh \\a (0\\x8a\\xf8\\xfe\\x12[y\\xec\\x11\\xe4>\\xdez \\x03\\xcdbdL\\x8e\\x95w\\xabn\\xee\\xaf\\f\\xabeN\\x1b\\x00\\xf7\\x7f\\xbe=J~\\xb7\\xbd\\xf7e\\x82NHQ\\xef\\xf7\\xe3-;!BG;\\x91O\\x1fW\\xe21X\\xdc\\x18\\x95\\t\\x8a*\\x9e\\x84\\xf5\\xd79\\xbe\\xdc\\x1d\\xe2\\x87\\x02\\xc4\\x03\\xd2Q\\x1b\\xf8\\xfc$A\\x7f\\t\\x16\\xc8\\\\\\xcbػ-\\xd3\\xda\\xef\\xa7=h\\xd1\\xf7Z\\xac¾G`\\f\\x000\\x15\\xf6\\xb9\\x8c{\\t(l\\xaf\\t\\xd3H\\x1c\\xc4>\\x01\\xdcyG\\xc8ik\\x85\\xb4\\xe3\\x9c!n\\x9bVt\\xd8tDCN\\x8b\\xed\\xfd\\x00\\xc6 \\x93\\x9d\\x1e}j\\xaa\\xb8Ӧ\\x86\\xfd^\\x8cy\\xa3\\xb4c\\x96\\xe1@\\xff\\xb0\\xf7kT\\x83\\x1f\\xd4\\xde1\\xcd=\\xaaF\\xf6>\\xd2CxyGr\\xbc\\x97\\xde\\xfdR\\xaf\\xda\\a\\x15\\xd8\\xdf\\xfe~\\xf6\\x8f\\x00\\x00\\x00\\xff\\xff)\\x00\\x87w>{\\x00\\x00\"),\n\t[]byte(\"\\x1f\\x8b\\b\\x00\\x00\\x00\\x00\\x00\\x00\\xff\\xbcV\\xcfo+5\\x10\\xbe\\xe7\\xaf\\x18\\x89+\\xbb\\xa1B \\x94\\x1b*\\x1c*\\xe0\\xa9j\\x9ezw\\xbc\\x93d\\xa8\\xd7^f\\xc6)A\\xfc\\xf1\\xc8\\xf6n\\x9b\\xee:\\xb4\\x8f\\x03\\xbe\\xad\\xed\\xf9\\xe6\\x9bo~x\\x9b\\xa6Y\\x99\\x81\\x1e\\x91\\x85\\x82߀\\x19\\b\\xffT\\xf4\\xe9Kڧ\\x1f\\xa4\\xa5\\xb0>ݬ\\x9e\\xc8w\\x1b\\xb8\\x8d\\xa2\\xa1\\x7f@\\t\\x91-\\xfe\\x84{\\xf2\\xa4\\x14\\xfc\\xaaG5\\x9dQ\\xb3Y\\x01\\x18\\uf0da\\xb4-\\xe9\\x13\\xc0\\x06\\xaf\\x1c\\x9cCn\\x0e\\xe8ۧ\\xb8\\xc3]$\\xd7!g\\xf0\\xc9\\xf5\\xe9\\x9b\\xf6\\xe6\\xfb\\xf6\\xbb\\x15\\x807=n@\\x90ә\\x1a\\x8d\\xc2\\xf8GDQiO\\xe8\\x90CKa%\\x03ڄ\\x7f\\xe0\\x10\\x87\\r\\xbc\\x1e\\x14\\xfb\\xd1w\\xe1\\xbd\\xcdP\\xdb\\f\\xf5P\\xa0\\xf2\\xa9#\\xd1_\\xae\\xdd\\xf8\\x95\\xc6[\\x83\\x8bl\\\\\\x9dP\\xbe \\xc7\\xc0\\xfa\\xe9\\xd5i\\x03\\\"\\\\N\\xc8\\x1f\\xa23\\\\5^\\x01\\x88\\r\\x03n \\xdb\\x0e\\xc6b\\xb7\\x02\\x18\\x05\\xc9Xͨ\\xc5\\xe9\\xa6\\xc0\\xd9#\\xf6\\xa68\\x01\\b\\x03\\xfa\\x1f\\xef\\xef\\x1e\\xbfݾ\\xd9\\x06\\xe8P,ӠYֿ\\x9b\\x97}\\xa8\\x85\\t$``\\xa4\\x04\\x1a\\xc0X\\x8b\\\"`#3z\\x85B\\x19\\xc8\\xef\\x03\\xf79\\xad`v!\\xea\\x05\\xaa\\x1e\\x11\\x1e\\xb3\\xfec\\x98\\xed\\xcb\\xe1\\xc0a@V\\x9a\\xa4)\\xeb\\xa2\\xe2.v\\xff\\x8dxZ)\\xd6b\\x05]*=\\x94\\xecy\\xd4\\v\\xbbQ\\x1e\\b{\\xd0#\\t0\\x0e\\x8c\\x82\\xbe\\x14c\\xda6\\x1e\\xc2\\xeew\\xb4\\xdaΠ\\x8b.\\x922\\x19]\\x97*\\xf6\\x84\\xac\\xc0h\\xc3\\xc1\\xd3_/ؒ\\x04JN\\x9dѬ\\x9dWdo\\x1c\\x9c\\x8c\\x8b\\xf85\\x18\\xdf͐{s\\x06\\xc6\\xe4\\x13\\xa2\\xbf\\xc0\\xcb\\x062\\xe7\\xf1[`\\xccRo\\xe0\\xa8:\\xc8f\\xbd>\\x90N}hC\\xdfGOz^疢]\\xd4\\xc0\\xb2\\xee\\xf0\\x84n-th\\f\\xdb#)Z\\x8d\\x8ck3P\\x93\\x03\\xf1\\xb9\\x17۾\\xfb\\x8a\\xc7Ε7n\\xf5\\x9cjP\\x94\\xc9\\x1f.\\x0er\\xeb|AzR#\\x95b*P%\\xc4\\xd7,\\xa4\\xad$\\xdd\\xc3\\xcf\\xdb\\xcf01)\\x99*Iy\\xbd\\xba\\xd0e\\xcaOR\\x93\\xfc\\x1e\\xb9\\xd8\\xed9\\xf4\\x19\\x13}7\\x04\\xf2\\x9a?\\xac\\xa3\\\\\\xb8qד\\xcaT\\xda)us\\xd8\\xdb<\\xab`\\x87\\x10\\x87\\xce(v\\xf3\\vw\\x1enM\\x8f\\xee\\xd6\\b\\xfeϹJY\\x91&%\\xe1Cٺ\\x9c\\xc0\\xf3\\xcbEދ\\x83iv^ImeJl\\a\\xb4)\\xb9I\\xdfdM{\\xb2\\xa5\\xad\\xf6\\x81\\xc1\\xd4L\\xda\\x0f1\\xc9\\x16_\\xc8e\\x9cH\\x85\\xcdlN\\xa5.\\x7f\\x9fM},哣\\x11\\x9co\\xce8ݧ;s\\xff\\x8e\\xf6h\\xcf\\xd6a\\x81(S\\bߧ\\x92\\x16\\xfa\\xd8/}6\\xf0\\t\\x9f+\\xbb\\xf7\\x1c҄\\xc6\\xf9\\xa8\\xb9Z\\x1bP\\x1e\\xb1\\x03\\xf9E\\xb8\\xf3\\xc8ʭ\\xfc0.G~\\x0eh\\x04\\x02\\x8eާ\\x96\\x0e~\\x01Yy\\x11\\x16wH\\xb1\\xaf\\xb0\\xa9\\xf2\\xb9\\xf3\\xfb\\x90\\xff\\\"Lrl\\xb4\\xb4\\x13\\x8e\\xc9\\x1e\\xfd\\x14^\\x15\\xc0\\xeb\\xb9.k9\\xe7>$hY\\xf9y\\xfeo\\xc6i.\\x11c\\xd5w\\x93YU\\x0f\\x92ǚ\\xe2\\xf5\\xfe\\x1aYF\\xe7\\xcc\\xce\\xe1\\x06\\x94\\xe3Һ\\xd8\\x1afs\\x9eW\\xcdTj\\x9f\\xa9GQ\\xd3\\x0f\\xef\\x14\\xd0\\xe2UH\\xeb~\\x81\\x92\\x9a\\xe7\\xf9\\x88\\xfeZ\\x8b\\xc0\\xb3\\x91W\\xe7\\x15\\xc8\\xdd\\xf9\\x9a\\xe9\\xed\\xcb\\xdf\\xe6\\xb2\\xcfJ=o \\xcd\\xfaF\\xa9\\\"䇔\\xaa\\xa6\\xb4\\xd4y\\xf5\\xb7f\\xa1\\xd2\\xf6\\xf2\\xee4H\\xde\\xf4\\xcb\\xf4W\\xb3\\x8c\\xe1*\\x85j\\x05,63|w\\x11\\x9eh`s\\x98\\x02\\xfe'\\x00\\x00\\xff\\xff\\xef\\xf8\\xa6>\\x10\\f\\x00\\x00\"),\n\t[]byte(\"\\x1f\\x8b\\b\\x00\\x00\\x00\\x00\\x00\\x00\\xff\\xbcVM\\x8f\\xdb6\\x10\\xbd\\xfbW\\f\\x92kd7(Z\\x14\\xbe\\x05\\xdb\\x1e\\x82&\\xc5\\\"N\\xf7N\\x93#{j\\x8ad\\x87C9.\\xfa\\xe3\\v\\x92Ү-\\xcb\\xc9nQT\\x17\\xc3\\xe4\\xf0q>\\u07bca\\xd34\\v\\x15\\xe8\\x019\\x92wkP\\x81\\xf0\\x8b\\xa0\\xcb\\xff\\xe2\\xf2\\xf0S\\\\\\x92_\\xf5o\\x17\\arf\\rw)\\x8a\\xef>a\\xf4\\x895\\xfe\\x8c-9\\x12\\xf2nѡ(\\xa3D\\xad\\x17\\x00\\xca9/*/\\xc7\\xfc\\x17@{'\\xec\\xadEnv薇\\xb4\\xc5m\\\"k\\x90\\v\\xf8xu\\xff\\xdd\\xf2\\xed\\x8f\\xcb\\x1f\\x16\\x00Nu\\xb8\\x86\\xde\\xdb\\xd4at*Ľ\\x17\\xebu\\xc5\\\\\\xf6h\\x91\\xfd\\x92\\xfc\\\"\\x06\\xd4\\xf9\\x8a\\x1d\\xfb\\x14\\xd6\\xf0\\xb4Q!\\x86\\xeb\\xab\\xeb\\x0f\\x05m3\\xa0}\\x18Њ\\x81\\xa5(\\xbf~\\xc5\\xe8\\x03E)\\x86\\xc1&V\\xf6\\xa6g\\xc5&\\xee=\\xcboO\\xb77\\xd0G[w\\xc8\\xed\\x92U|\\xeb\\xfc\\x02 j\\x1fp\\r\\xe5xP\\x1a\\xcd\\x02`\\xc8O\\x81k\\xc6Լ\\xad\\x88z\\x8f\\x9d\\xaa\\xf7\\x00\\xf8\\x80\\xee\\xdd\\xfd\\xfb\\x87\\xef7\\x17\\xcb\\x00\\x06\\xa3f\\nR\\xb2<\\x1f\\\"P\\x04\\x05\\xa3'p\\xdc##<\\x94|B\\x14\\xcf\\x18\\a\\xa7\\x1fA\\x01F\\xff\\xe3\\xf2q1\\xb0\\x0f\\xc8Bc\\xf0\\xf5;\\xe3\\xd7\\xd9\\xeaį\\xbf\\x9b\\x8b=\\x80\\x1cJ=\\x05&\\x13\\r#\\xc8\\x1e\\xc7t\\xa0\\x19\\xa2\\a߂\\xec)\\x02c`\\x8c\\xe8*\\xf5\\xf2\\xb2r\\xe0\\xb7\\x7f\\xa0\\x96\\xe5\\x04z\\x83\\x9car\\xad\\x925\\x99\\x9f=\\xb2\\x00\\xa3\\xf6;G\\x7f=bG\\x10_.\\xb5J0\\n\\x90\\x13d\\xa7,\\xf4\\xca&|\\x03ʙ\\tr\\xa7N\\xc0\\x98\\xef\\x84\\xe4\\xce\\xf0ʁ8\\xf5\\xe3\\xa3g\\x04r\\xad_\\xc3^$\\xc4\\xf5j\\xb5#\\x19\\xbbN\\xfb\\xaeK\\x8e\\xe4\\xb4*\\rD\\xdb$\\x9e\\xe3\\xca`\\x8fv\\x15i\\xd7(\\xd6{\\x12Ԓ\\x18W*PS\\x02q\\xb5K:\\xf3\\x9a\\x87>\\x8d\\x17\\xd7\\xca)S,\\n\\x93\\u06ddm\\x94.yAyr\\xc3T\\xd6T\\xa8\\x1a\\xe2S\\x15\\xf2RNݧ_6\\x9fa\\xf4\\xa4V\\xaa\\x16\\xe5\\xc9\\xf4*/c}r6ɵ\\xc8\\xf5\\\\˾+\\x98\\xe8L\\xf0\\xe4\\xa4\\xfcі\\xd0\\tĴ\\xedH2\\r\\xfeL\\x18%\\x97n\\n{W\\x94\\t\\xb6\\b)\\x18%h\\xa6\\x06\\xef\\x1dܩ\\x0e흊\\xf8?\\xd7*W%6\\xb9\\bϪֹ\\xdeN\\x8dkz\\xcf\\x1bu\\x90\\xc9\\x1b\\xa5\\x9dW\\x84M@}\\xd1x\\x19\\x85Z\\x1a\\x14\\xa2\\xf5i\\x8b\\x15\\x10|;ý\\x17\\xb9\\x9c?t\\xa9\\x9b#\\xe2\\xbb^\\x91U[{-\\t\\r\\xfc\\xee\\xd4\\xcdݛş\\xad\\xe7\\xd5b̏=\\xb3\\x06\\xe1T\\xb1\\a\\x96\\r+\\xff\\x04\\x00\\x00\\xff\\xffNy\\xc1Q\\xa1\\x0e\\x00\\x00\"),\n}\n\nvar CRDs = crds()\n\nfunc crds() []*apiextv1.CustomResourceDefinition {\n\tapiextinstall.Install(scheme.Scheme)\n\tdecode := scheme.Codecs.UniversalDeserializer().Decode\n\tvar objs []*apiextv1.CustomResourceDefinition\n\tfor _, crd := range rawCRDs {\n\t\tgzr, err := gzip.NewReader(bytes.NewReader(crd))\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\tbytes, err := io.ReadAll(gzr)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\tgzr.Close()\n\n\t\tobj, _, err := decode(bytes, nil, nil)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\tobjs = append(objs, obj.(*apiextv1.CustomResourceDefinition))\n\t}\n\treturn objs\n}\n" }, { "path": "config/crd/v1/crds/doc.go", "content": "// Package crds embeds the controller-tools generated CRD manifests\npackage crds\n\n//go:generate go run ../../../../hack/crd-gen/v1/main.go\n" }, { "path": "config/crd/v2alpha1/bases/velero.io_datadownloads.yaml", "content": "---\napiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: v0.16.5\n name: datadownloads.velero.io\nspec:\n group: velero.io\n names:\n kind: DataDownload\n listKind: DataDownloadList\n plural: datadownloads\n singular: datadownload\n scope: Namespaced\n versions:\n - additionalPrinterColumns:\n - description: DataDownload status such as New/InProgress\n jsonPath: .status.phase\n name: Status\n type: string\n - description: Time duration since this DataDownload was started\n jsonPath: .status.startTimestamp\n name: Started\n type: date\n - description: Completed bytes\n format: int64\n jsonPath: .status.progress.bytesDone\n name: Bytes Done\n type: integer\n - description: Total bytes\n format: int64\n jsonPath: .status.progress.totalBytes\n name: Total Bytes\n type: integer\n - description: Name of the Backup Storage Location where the backup data is stored\n jsonPath: .spec.backupStorageLocation\n name: Storage Location\n type: string\n - description: Time duration since this DataDownload was created\n jsonPath: .metadata.creationTimestamp\n name: Age\n type: date\n - description: Name of the node where the DataDownload is processed\n jsonPath: .status.node\n name: Node\n type: string\n name: v2alpha1\n schema:\n openAPIV3Schema:\n description: DataDownload acts as the protocol between data mover plugins\n and data mover controller for the datamover restore operation\n properties:\n apiVersion:\n description: |-\n APIVersion defines the versioned schema of this representation of an object.\n Servers should convert recognized schemas to the latest internal value, and\n may reject unrecognized values.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\n type: string\n kind:\n description: |-\n Kind is a string value representing the REST resource this object represents.\n Servers may infer this from the endpoint the client submits requests to.\n Cannot be updated.\n In CamelCase.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\n type: string\n metadata:\n type: object\n spec:\n description: DataDownloadSpec is the specification for a DataDownload.\n properties:\n backupStorageLocation:\n description: |-\n BackupStorageLocation is the name of the backup storage location\n where the backup repository is stored.\n type: string\n cancel:\n description: |-\n Cancel indicates request to cancel the ongoing DataDownload. It can be set\n when the DataDownload is in InProgress phase\n type: boolean\n dataMoverConfig:\n additionalProperties:\n type: string\n description: DataMoverConfig is for data-mover-specific configuration\n fields.\n type: object\n datamover:\n description: |-\n DataMover specifies the data mover to be used by the backup.\n If DataMover is \"\" or \"velero\", the built-in data mover will be used.\n type: string\n nodeOS:\n description: NodeOS is OS of the node where the DataDownload is processed.\n enum:\n - auto\n - linux\n - windows\n type: string\n operationTimeout:\n description: |-\n OperationTimeout specifies the time used to wait internal operations,\n before returning error as timeout.\n type: string\n snapshotID:\n description: SnapshotID is the ID of the Velero backup snapshot to\n be restored from.\n type: string\n snapshotSize:\n description: SnapshotSize is the logical size in Bytes of the snapshot.\n format: int64\n type: integer\n sourceNamespace:\n description: |-\n SourceNamespace is the original namespace where the volume is backed up from.\n It may be different from SourcePVC's namespace if namespace is remapped during restore.\n type: string\n targetVolume:\n description: TargetVolume is the information of the target PVC and\n PV.\n properties:\n namespace:\n description: Namespace is the target namespace\n type: string\n pv:\n description: PV is the name of the target PV that is created by\n Velero restore\n type: string\n pvc:\n description: PVC is the name of the target PVC that is created\n by Velero restore\n type: string\n required:\n - namespace\n - pv\n - pvc\n type: object\n required:\n - backupStorageLocation\n - operationTimeout\n - snapshotID\n - sourceNamespace\n - targetVolume\n type: object\n status:\n description: DataDownloadStatus is the current status of a DataDownload.\n properties:\n acceptedByNode:\n description: Node is name of the node where the DataUpload is prepared.\n type: string\n acceptedTimestamp:\n description: |-\n AcceptedTimestamp records the time the DataUpload is to be prepared.\n The server's time is used for AcceptedTimestamp\n format: date-time\n nullable: true\n type: string\n completionTimestamp:\n description: |-\n CompletionTimestamp records the time a restore was completed.\n Completion time is recorded even on failed restores.\n The server's time is used for CompletionTimestamps\n format: date-time\n nullable: true\n type: string\n message:\n description: Message is a message about the DataDownload's status.\n type: string\n node:\n description: Node is name of the node where the DataDownload is processed.\n type: string\n phase:\n description: Phase is the current state of the DataDownload.\n enum:\n - New\n - Accepted\n - Prepared\n - InProgress\n - Canceling\n - Canceled\n - Completed\n - Failed\n type: string\n progress:\n description: |-\n Progress holds the total number of bytes of the snapshot and the current\n number of restored bytes. This can be used to display progress information\n about the restore operation.\n properties:\n bytesDone:\n format: int64\n type: integer\n totalBytes:\n format: int64\n type: integer\n type: object\n startTimestamp:\n description: |-\n StartTimestamp records the time a restore was started.\n The server's time is used for StartTimestamps\n format: date-time\n nullable: true\n type: string\n type: object\n type: object\n served: true\n storage: true\n subresources: {}\n" }, { "path": "config/crd/v2alpha1/bases/velero.io_datauploads.yaml", "content": "---\napiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: v0.16.5\n name: datauploads.velero.io\nspec:\n group: velero.io\n names:\n kind: DataUpload\n listKind: DataUploadList\n plural: datauploads\n singular: dataupload\n scope: Namespaced\n versions:\n - additionalPrinterColumns:\n - description: DataUpload status such as New/InProgress\n jsonPath: .status.phase\n name: Status\n type: string\n - description: Time duration since this DataUpload was started\n jsonPath: .status.startTimestamp\n name: Started\n type: date\n - description: Completed bytes\n format: int64\n jsonPath: .status.progress.bytesDone\n name: Bytes Done\n type: integer\n - description: Total bytes\n format: int64\n jsonPath: .status.progress.totalBytes\n name: Total Bytes\n type: integer\n - description: Incremental bytes\n format: int64\n jsonPath: .status.incrementalBytes\n name: Incremental Bytes\n priority: 10\n type: integer\n - description: Name of the Backup Storage Location where this backup should be\n stored\n jsonPath: .spec.backupStorageLocation\n name: Storage Location\n type: string\n - description: Time duration since this DataUpload was created\n jsonPath: .metadata.creationTimestamp\n name: Age\n type: date\n - description: Name of the node where the DataUpload is processed\n jsonPath: .status.node\n name: Node\n type: string\n name: v2alpha1\n schema:\n openAPIV3Schema:\n description: DataUpload acts as the protocol between data mover plugins and\n data mover controller for the datamover backup operation\n properties:\n apiVersion:\n description: |-\n APIVersion defines the versioned schema of this representation of an object.\n Servers should convert recognized schemas to the latest internal value, and\n may reject unrecognized values.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\n type: string\n kind:\n description: |-\n Kind is a string value representing the REST resource this object represents.\n Servers may infer this from the endpoint the client submits requests to.\n Cannot be updated.\n In CamelCase.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\n type: string\n metadata:\n type: object\n spec:\n description: DataUploadSpec is the specification for a DataUpload.\n properties:\n backupStorageLocation:\n description: |-\n BackupStorageLocation is the name of the backup storage location\n where the backup repository is stored.\n type: string\n cancel:\n description: |-\n Cancel indicates request to cancel the ongoing DataUpload. It can be set\n when the DataUpload is in InProgress phase\n type: boolean\n csiSnapshot:\n description: If SnapshotType is CSI, CSISnapshot provides the information\n of the CSI snapshot.\n nullable: true\n properties:\n driver:\n description: Driver is the driver used by the VolumeSnapshotContent\n type: string\n snapshotClass:\n description: SnapshotClass is the name of the snapshot class that\n the volume snapshot is created with\n type: string\n storageClass:\n description: StorageClass is the name of the storage class of\n the PVC that the volume snapshot is created from\n type: string\n volumeSnapshot:\n description: VolumeSnapshot is the name of the volume snapshot\n to be backed up\n type: string\n required:\n - storageClass\n - volumeSnapshot\n type: object\n dataMoverConfig:\n additionalProperties:\n type: string\n description: DataMoverConfig is for data-mover-specific configuration\n fields.\n nullable: true\n type: object\n datamover:\n description: |-\n DataMover specifies the data mover to be used by the backup.\n If DataMover is \"\" or \"velero\", the built-in data mover will be used.\n type: string\n operationTimeout:\n description: |-\n OperationTimeout specifies the time used to wait internal operations,\n before returning error as timeout.\n type: string\n snapshotType:\n description: SnapshotType is the type of the snapshot to be backed\n up.\n type: string\n sourceNamespace:\n description: |-\n SourceNamespace is the original namespace where the volume is backed up from.\n It is the same namespace for SourcePVC and CSI namespaced objects.\n type: string\n sourcePVC:\n description: SourcePVC is the name of the PVC which the snapshot is\n taken for.\n type: string\n required:\n - backupStorageLocation\n - operationTimeout\n - snapshotType\n - sourceNamespace\n - sourcePVC\n type: object\n status:\n description: DataUploadStatus is the current status of a DataUpload.\n properties:\n acceptedByNode:\n description: AcceptedByNode is name of the node where the DataUpload\n is prepared.\n type: string\n acceptedTimestamp:\n description: |-\n AcceptedTimestamp records the time the DataUpload is to be prepared.\n The server's time is used for AcceptedTimestamp\n format: date-time\n nullable: true\n type: string\n completionTimestamp:\n description: |-\n CompletionTimestamp records the time a backup was completed.\n Completion time is recorded even on failed backups.\n Completion time is recorded before uploading the backup object.\n The server's time is used for CompletionTimestamps\n format: date-time\n nullable: true\n type: string\n dataMoverResult:\n additionalProperties:\n type: string\n description: DataMoverResult stores data-mover-specific information\n as a result of the DataUpload.\n nullable: true\n type: object\n incrementalBytes:\n description: IncrementalBytes holds the number of bytes new or changed\n since the last backup\n format: int64\n type: integer\n message:\n description: Message is a message about the DataUpload's status.\n type: string\n node:\n description: Node is name of the node where the DataUpload is processed.\n type: string\n nodeOS:\n description: NodeOS is OS of the node where the DataUpload is processed.\n enum:\n - auto\n - linux\n - windows\n type: string\n path:\n description: Path is the full path of the snapshot volume being backed\n up.\n type: string\n phase:\n description: Phase is the current state of the DataUpload.\n enum:\n - New\n - Accepted\n - Prepared\n - InProgress\n - Canceling\n - Canceled\n - Completed\n - Failed\n type: string\n progress:\n description: |-\n Progress holds the total number of bytes of the volume and the current\n number of backed up bytes. This can be used to display progress information\n about the backup operation.\n properties:\n bytesDone:\n format: int64\n type: integer\n totalBytes:\n format: int64\n type: integer\n type: object\n snapshotID:\n description: SnapshotID is the identifier for the snapshot in the\n backup repository.\n type: string\n startTimestamp:\n description: |-\n StartTimestamp records the time a backup was started.\n Separate from CreationTimestamp, since that value changes\n on restores.\n The server's time is used for StartTimestamps\n format: date-time\n nullable: true\n type: string\n type: object\n type: object\n served: true\n storage: true\n subresources: {}\n" }, { "path": "config/crd/v2alpha1/crds/crds.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\n// Code generated by crds_generate.go; DO NOT EDIT.\n\npackage crds\n\nimport (\n\t\"bytes\"\n\t\"compress/gzip\"\n\t\"io\"\n\n\tapiextinstall \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/install\"\n\tapiextv1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1\"\n\t\"k8s.io/client-go/kubernetes/scheme\"\n)\n\nvar rawCRDs = [][]byte{\n\t[]byte(\"\\x1f\\x8b\\b\\x00\\x00\\x00\\x00\\x00\\x00\\xff\\xbcYK\\x93\\xe3\\xb8\\r\\xbe\\xf7\\xaf@M\\x0es\\x19\\xbb3yl\\xa5|\\x9bq'U]\\xd9\\xe9q\\xad;}\\xa7$X\\xe6\\x0eE2|\\xd8\\xebM\\xf2\\xdfS %\\x99\\x92\\xe8\\xe7>|3\\t\\x82\\x1f\\x01\\x10\\xf8@\\xcdf\\xb3\\a\\xa6\\xf9\\x1b\\x1a˕\\\\\\x00\\xd3\\x1c\\x7fr(韝\\x7f\\xfb\\x9b\\x9ds\\xf5\\xb8\\xfb\\xf8\\xf0\\x8d\\xcbj\\x01Ko\\x9dj~@\\xab\\xbc)\\xf1\\t7\\\\rǕ|hб\\x8a9\\xb6x\\x00`R*\\xc7h\\xd8\\xd2_\\x80RIg\\x94\\x10hf5\\xca\\xf97_`Ṩ\\xd0\\x04\\xe5\\xddֻ?\\xce?~7\\xff\\xeb\\x03\\x80d\\r.\\x80\\xf4Uj/\\x85b\\x95\\x9d\\xefP\\xa0Qs\\xae\\x1e\\xacƒ\\x14\\xd7Fy\\xbd\\x80\\xe3D\\\\\\xd8n\\x1a\\x01?1ǞZ\\x1daXp\\xeb\\xfe9\\x99\\xfa\\x9e[\\x17\\xa6\\xb5\\xf0\\x86\\x89\\xd1\\xdea\\xc6rY{\\xc1\\xccp\\xee\\x01\\xc0\\x96J\\xe3\\x02^hk\\xcdJ\\xa4\\xb1\\xf6L\\x01\\xca\\fXU\\x05+1\\xb12\\\\:4K%|\\xd3Yg\\x06\\x15\\xda\\xd2p\\xed\\x82\\x15RX`\\x1dsނ\\xf5\\xe5\\x16\\x98\\x85\\x17\\xdc?>˕Q\\xb5A\\x1ba\\x01\\xfch\\x95\\\\1\\xb7]\\xc0<\\x8a\\xcf\\xf5\\x96Ylg\\xa3)\\xd7a\\xa2\\x1dr\\a\\xc2k\\x9d\\xe1\\xb2\\xce!x\\xe5\\rB\\xe5Mp!\\x9d\\xbbDp[n\\x87\\xd0\\xf6\\xcc\\x12<\\xe3\\xb0:\\t$̓:\\xebX\\xa3Lj\\x92\\xa5\\x11R\\xc5\\x1c\\xe6\\x00-U\\xa3\\x05:\\xac\\xa088쎱Q\\xa6an\\x01\\\\\\xba\\xef\\xfer\\xda\\x16\\xad\\xb1\\xe6a铒C\\xc3|\\xa6QH\\x86#\\x12\\xf2R\\x8d&k\\x1d\\xe5\\x98\\xf8%@\\x1c)\\xf8\\x9c\\xac\\x8fH\\xa2\\xdet\\xfc\\\"\\x14\\n9P\\x1bp[\\x84Ϭ\\xfc\\xe65\\xac\\x9d2\\xacF\\xf8^\\x95\\xd1}\\xfb-\\x1a\\f\\x12E\\x94\\xa0\\xe8\\x05N\\xbeS&\\xeb:\\x8d\\xe5<ʶ\\xca:]#\\xff\\r7\\xfa\\xd5c\\xab4Ȳ\\xb1ե\\x9ay\\x90\\xe0J\\xe6\\x03\\xecS\\x8dW\\x05WjD\\xa9*L,6\\xc0\\xc4-h\\xa3J\\xb4\\xf6L\\xc0\\x93\\x82\\x01\\x8a\\x97\\xe3\\xc0\\xc44Qb\\xf7'&\\xf4\\x96}\\x8cI\\xa6\\xdcb\\xc3\\x16\\xed\\n\\xa5Q~Z=\\xbf\\xfdy=\\x18\\x863\\t\\x83\\x95\\xceR\\xa6 \\xf8\\xda(\\xa7J%\\xa0@\\xb7G\\x94\\xd1\\xf5\\x8dڡ\\xa1\\x9a\\x8d\\x93\\x06C\\xf4\\x10@\\x93z\\x1fhO\\x8d\\xc6\\xf1.\\v\\xb7\\xba\\x8f\\x05&\\x19\\x1d\\x9d㿳\\xc1\\x1c\\x00\\x1d=\\xae\\x82\\x8a*\\r\\xc6c\\xb5\\xb9\\x15\\xab\\xd6Z\\xd1y܂AmТ\\x8c\\xb5\\x87\\x86\\x99\\x04U\\xfc\\x88\\xa5\\x9b\\x8fT\\xafѐ\\x1a\\xb0[\\xe5EE\\x87ݡq`\\xb0T\\xb5\\xe4?\\xf7\\xba-8\\x156\\x15̡u\\xe12\\x1a\\xc9\\x04\\xec\\x98\\xf0\\xf8\\x81\\x8c6\\xd2ܰ\\x03\\x18\\xa4=\\xc1\\xcbD_X`\\xc78\\xbe\\x90\\x15\\xb9ܨ\\x05l\\x9d\\xd3v\\xf1\\xf8Xsו\\xddR5\\x8d\\x97\\xdc\\x1d\\x1e\\x837x\\xe1\\x9d2\\xf6\\xb1\\xc2\\x1d\\x8aG\\xcb\\xeb\\x193\\xe5\\x96;,\\x9d7\\xf8\\xc84\\x9f\\x85\\x83\\xc8Pz\\xe7M\\xf5\\a\\xd3\\x16j;\\xd8v\\x12\\x88\\xf1\\x17\\n\\xe6\\r\\xee\\xa1*J\\xb7\\x82\\xb5\\xaa\\xe2\\x11\\x8f^\\xa0!2\\xdd\\x0f\\x7f_\\xbfB\\x87$z*:\\xe5(:\\xb1K\\xe7\\x1f\\xb2&\\x97\\x1b4q\\xddƨ&\\xe8DYiť\\v\\x7fJ\\xc1Q:\\xb0\\xbeh\\xb8\\xa30\\xf8\\xb7G\\xeb\\xc8uc\\xb5\\xcb@M\\xa0@\\xf0\\x9a\\xf2A5\\x16x\\x96\\xb0d\\r\\x8a%\\xb3\\xf8;\\xfb\\x8a\\xbcbg䄫\\xbc\\x95\\x12\\xae\\xb1p4o2\\xd11\\xa6\\x13\\xaeM3\\xc8ZcI^%\\xc3\\xd22\\xbe\\xe1m%\\xa14\\xc0\\x06\\xb2C\\v\\xe5\\xaf>\\xfd\\xb2\\xd5d,t)\\xdc\\xe8\\xf79\\xa7\\xa8C+\\x93D\\xde\\xd6:\\xdb\\x16)1,R\\xe9oR\\x1f\\rje\\xb9S\\xe6p\\xac\\x92\\xe3P8\\xe9\\x15\\xfa\\x95L\\x96(\\xee9\\xde2\\xac\\x04.+\\xb29\\xf6\\xa1LI(j\\r@\\x95\\xac\\x15]\\xae\\x81+\\xe0ّ\\fŶE\\x97?\\xa8\\xccV5.\\xe1\\xc8)!\\xe5\\x8e\\xe3\\xe3\\x16J\\tdc+R\\x14~\\xa1\\xb2\\xb0Tr\\xc3\\xeb\\xe9\\xc1S\\xfa{*D.\\xd84\\x13\\xb0ɖt\\n\\x8aNB2\\v\\x15jօ.\\xa5\\xf6\\r\\xaf\\xbd9\\xe5\\xff\\rGQM\\xf2\\xcfɛ\\xd4\\x1d8\\xecr\\x8f\\x8f{\\xe8\\xdd\\xedj\\xabZRz\\x9d\\n\\x19\\xca\\x06\\xbe\\x9b\\x84\\xe6\\x14$\\xc0\\xf3&\\xd1\\xc8-\\xbc{\\a\\xca\\xc0\\xbb\\xd8\\x13\\xbd\\xfb\\x10W{.܌\\x0f\\xea\\xff\\x9e\\v\\xd1\\xedrSt\\x13\\xc3\\xf9\\xba\\xbep\\xf2\\x97 Dx\\xbe\\xaeo\\xe5VS4(}3\\xddp\\x06\\xcc;\\x95\\x19\\x16\\\\\\xfa\\x9f2\\xe3{.+\\xb5\\xb7\\xb7\\x1c\\xb6\\xe77D1\\x95w\\xf78\\xfc\\xebH\\xc7\\xc8\\xef\\x8e\\bq\\xf0\\xb5S\\xb0g<\\xe1\\x18\\xfd\\xee\\xf6CFo\\x81\\x1b*H\\x06\\x9d7\\x92\\xd2\\x01\\x1aC\\x19\\xda\\x06\\x95\\xcaO8\\xcfٓZɴ\\xdd*\\xf7\\xfct\\xe1\\x8c\\xeb^\\xb0˻\\xcfO\\x9d\\x8b\\xdfB\\xd4\\xf5ɷ\\x95\\x84\\x8c\\x97\\b~\\xc7\\\"\\xabP\\xd6\\xefB\\xbb\\xe6?\\xe3\\x95xI\\xb4C,T\\xcdK&\\xc0\\x861\\xd96\\x81\\xed!:\\xddS@\\xb9>o\\f7\\xed\\xd6\\x12\\xbc\\x81\\xfb\\xf4/\\x04\\xf7\\x84\\xd1z\\xa8\\xa2;\\x8a2\\xbc\\xe6\\x14,\\xb2\\x9f9ޱ\\x9d\\x12\\xbe\\t\\xa2\\xe4\\x12\\xac\\xc0\\xeb\\x13\\xb6\\x06*\\x1fD\\xb6\\n\\x84\\x8ao6h\\x88Q\\x05\\xba\\x157^\\xbd-\\xdf\\xdbd\\x13\\xbeI\\xffP\\xa5j\\x98\\xd6XQoG\\xc1\\xd8\\xfa\\xf6&\\xaf:fjto\\x01\\xf4\\x05\\x13\\xbd&\\xa2\\x9d)\\x88\\x9a\\x91\\x83Z\\xee\\x1f.W\\x10\\x83\\xd5\\xdb2\\xc3\\xd4\\xe9\\xb7z\\x9b\\\"<\\xcdc\\xa0m\\xdaN8q\\x82r\\xe2\\xad\\x16O\\xaf#\\xab\\xe2l\\x19\\x04л+v^\\xbd\\xe5XQo\\x0ep[\\xe6H\\xa2m\\xb2\\xa18duBw\\xa5[wއ\\xb7\\xbc\\n\\xf0\\xf2,\\xe2\\xe5\\x18\\xf2\\t\\xbc\\xc5\\xe1\\x17C&\\xd2\\xc5\\rV\\xb9\\x92s\\xdas3л\\xec`y=\\xb5\\xc8\\xef<\\xcb\\xf3\\xe7\\x91̸T\\x8d\\xa6\\x8f\\xf9}<1\\xcc+\\xa3\\xd9\\xf4J^\\xd5h\\x84g\\x90k[\\x8d\\xf8\\xb8ٺ\\xbd\\xf4&$\\x9d\\xf6ɓ\\xba\\xf7\\xbb\\x9a\\rV\\x96\\xa8\\x1dV\\x9f\\x0f\\xc4B\\xae *\\x04@\\x9e\\x7f\\x04\\xfa\\x97>\\xd2\\x14\\xd4\\xec֎\\xa0\\x83\\xd4?T\\xddS\\x00>\\x8d\\x95\\x84\\xd7\\nS%4b\\n7R\\xc9Ӡ\\x01^\\xa9\\xe4\\x85n\\xfb}d\\x0e\\xb4,\\xf0\\x11bԓMO\\x16Ej\\xa7g\\xb4~\\\"!\\xbd\\x10\\xac\\x10\\xb8\\x00g\\xfc\\xa9\\xd6\\\"\\xdfI\\xc5w\\xdf\\xf4\\x89﮶j\\xaafj;\\xd6?j\\x85\\xc7\\xc7\\xee\\xc59g\\xb2\\xa3\\xbe\\xde`Q\\x1dV\\x80;\\x94@\\xcd2\\xe3\\x02\\xabNg\\xa6\\xbf\\xb8d\\xf9\\f\\xe8)u\\xfd-\\x8dߠ\\xb5\\xac\\xbet\\x81\\xbeD\\xa9\\xf8\\x0e\\xd4.\\x01V\\x10\\xcf\\x1d\\xb3\\xfc\\xf7\\xb6\\xbd\\xdb7\\xf7\\x1b\\xbf\\xce%\\xbe\\xb2\\xdb8\\x83%\\xf4\\xc6\\x17\\xc0\\xacH&\\x97\\xd3zh\\xa7\\x93\\x1a\\x9civ^p\\x9f\\x19\\xed\\xeegfj\\xd5^\\xfa\\xcc\\xd4\\xe4\\x13R:\\x19\\x1f!r\\x85\\xb1\\x9b\\xcb\\xea\\xec\\xbf\\xd1d\\xe6\\xfe\\x11.\\xc3M\\x96n\\xf1\\xdds\\xdd\\xfb\\xa7\\x8c\\xad\\x12\\xdd\\r\\x0f\\xdfV\\xa4o\\n4\\xe4\\x86\\\"G\\xf8\\xc3\\vx\\xe2\\xb5\\x1c\\xf9\\xeb5\\xf4\\xbdKP5\\x87\\xd7-Q\\x93\\xf8\\xfe\\xd2us\\x15\\xb7Z\\xb0C\\x7f\\x98\\x94\\xa1f\\x94\\x1fo\\xcd\\xe4y\\xfdV\\x92\\xda\\x7f\\xeb\\xca3\\xaf\\xf3\\x8d\\f\\\\hf\\xc2|\\xff\\r\\xeb\\xb7\\xd9\\xe1\\xcc\\xeb\\xcb\\xf0\\x9b\\xe2]\\xad\\xd4@åR\\xd0~\\xe3\\xbc=\\x83\\x0f\\xb7\\xf9=\\x93w\\xd6z\\x93\\xc1\\x80\\xbcJt\\xb7\\xaf\\xa5\\xe9\\x88/\\xfaO\\b\\v\\xf8\\xcf\\xff\\x1e\\xfe\\x1f\\x00\\x00\\xff\\xff73Hq. \\x00\\x00\"),\n\t[]byte(\"\\x1f\\x8b\\b\\x00\\x00\\x00\\x00\\x00\\x00\\xff\\xbcZIs\\xe3\\xb8\\x15\\xbe\\xfbW\\xbc\\xea\\x1c\\xe6Ғ\\xa7\\xb3L\\xa5tk\\xcbI\\x95*3nW\\xcb\\xf1\\x1d\\\"\\x9fD\\x8cA\\x80\\xc1\\\"\\x8d\\xb3\\xfc\\xf7\\xd4\\x03\\b\\n$!Q\\xd2\\xf4\\f\\x0f]-,\\x0fo\\xc3\\xf7\\x16x6\\x9bݱ\\x86\\xbf\\xa26\\\\\\xc9\\x05\\xb0\\x86\\xe3/\\x16%\\xfd2\\U000f7fda9W\\xf7\\xfbOwo\\\\\\x96\\vX:cU\\xfd\\x15\\x8dr\\xba\\xc0G\\xdcr\\xc9-W\\xf2\\xaeF\\xcbJf\\xd9\\xe2\\x0e\\x80I\\xa9,\\xa3aC?\\x01\\n%\\xadVB\\xa0\\x9e\\xedP\\xce\\xdf\\xdc\\x067\\x8e\\x8b\\x12\\xb5'\\x1e\\x8f\\xde\\x7f?\\xff\\xf4\\xc3\\xfc/w\\x00\\x92ո\\x00\\xa2\\xe7\\x1a\\xa1Xi\\xe6{\\x14\\xa8՜\\xab;\\xd3`AdwZ\\xb9f\\x01lj\\xb0\\xad=2\\xb0\\xfb\\xc8,\\xfb\\xa7\\xa7\\xe0\\a\\x057\\xf6\\x1f\\x83\\x89\\x1f\\xb9\\xb1~\\xb2\\x11N3\\xd1;Տ\\x1b.wN0\\x9d\\xce\\xdc\\x01\\x98B5\\xb8\\x80':\\xb2a\\x05\\xd2X+\\x89ga\\x06\\xac,\\xbdn\\x98x\\xd6\\\\Z\\xd4K%\\\\\\x1du2\\x83\\x12M\\xa1yc\\xbd\\xecG\\x86\\xc0Xf\\x9d\\x01\\xe3\\x8a\\n\\x98\\x81'<ܯ\\xe4\\xb3V;\\x8d&\\xb0\\x04\\xf0\\xb3Q\\xf2\\x99\\xd9j\\x01\\xf3\\xb0|\\xdeT\\xcc`;\\x1bԷ\\xf6\\x13\\xed\\x90}'n\\x8d\\xd5\\\\\\xeer\\xe7\\xbf\\xf0\\x1a\\xa1tڛ\\x8dd.\\x10l\\xc5M\\xca\\u0601\\x19bN[,O\\xb2\\xe1牘\\xb1\\xacn\\x86\\xfc$[\\x03C%\\xb3\\x98cg\\xa9\\xeaF\\xa0\\xc5\\x126\\xef\\x16\\xa3\\x10[\\xa5kf\\x17\\xc0\\xa5\\xfd\\xe1ϧ5Ѫj\\xee\\xb7>*\\xd9W\\xcb\\x03\\x8dB2\\x1c8!\\v\\xedPgu\\xa3,\\x13\\xbf\\x86\\x11K\\x04\\x1e\\x92\\xfd\\x81\\x93@7\\x1d\\x9fde%\\v\\x8d5\\xca\\xdb\\x18\\xe2\\xc7\\xddcnR\\xd2\\xe9l\\xa3\\xb9\\xd2ܾ/\\xe0\\xd3\\xf7\\x97\\xb2I\\xb7\\x02\\xd4\\x16l\\x85\\xf0\\xc0\\x8a7\\xd7\\xc0\\xda*\\xcdv\\b?\\xaa\\\"\\xf8ءB\\xdd\\xfa\\xd8&,1\\x95r\\xa2\\x84M4\\f\\x80\\xb1Jg\\x9d\\xad\\xc1b\\x1ev\\xb5t#ف\\xc7\\xf5\\xcf\\xfc\\xc6w\\xa1\\xd0Ȳw!\\x82\\xe1ܯ\\xe0J\\xe6/\\xc4\\xe7\\x1d^t\\x19RmJUb\\xa7:L9\\xe2\\x06\\x1a\\xad\\n4\\xe6\\xcc\\xf5\\xa4\\xed=\\x1e\\x9e\\x8e\\x03#\\xb5\\x84\\x15\\xfb?2\\xd1T\\xecS\\x00â\\u009a-\\xda\\x1d\\xaaA\\xf9\\xf9y\\xf5\\xfa\\xa7uo\\x18NB\\x1b+\\xac!L#\\xd6\\x1b\\xad\\xac*\\x94\\x80\\r\\xda\\x03\\xa2\\xf4\\xf0\\n\\xb5ڣ&,\\xdeqi\\x80ɲ\\xa3\\t\\xe9\\x82cD!\\xd7\\xf7\\xf4h6L\\xb6\\xee\\xa4\\x1aԩ\\xd9ɕi\\xcc\\xf2\\x18$\\u0097D\\xbfdt \\xc4\\x7fg\\xbd9\\x00\\x92;삒\\xc2 \\x06\\xa9\\xda\\x10\\x80e\\xab\\xaa`7n@c\\xa3\\xd1\\xd0\\xf5\\xf2^\\xa5\\xb6\\xc0$\\xa8\\xcd\\xcfX\\xd8\\xf9\\x80\\xf4\\x1a5\\x91\\x89\\xf7\\xa1Pr\\x8fڂ\\xc6B\\xed$\\xffwGۀU\\xfeP\\xc1,\\x1a\\xeb/\\xa4\\x96L\\xc0\\x9e\\t\\x87\\x1f\\aڣ\\xaff\\uf811\\xce\\x04'\\x13z~\\x83\\x19\\xf2\\xf1\\x93\\xd2\\b\\\\n\\xd5\\x02*k\\x1b\\xb3\\xb8\\xbf\\xdfq\\x1bs\\x82Bյ\\x93ܾ\\xdf{c\\xf0\\x8d\\xb3J\\x9b\\xfb\\x12\\xf7(\\xee\\r\\xdf͘.*n\\xb1\\xb0N\\xe3=k\\xf8\\xcc\\v\\\"}^0\\xaf\\xcb?\\xe86\\x8b0\\xbdcG^\\x18>\\x1fϯ0\\x0f\\x85y\\xba\\x12\\xac%\\x15D\\xa4M.\\xb7\\xa8þ\\xadV\\xb5\\xa7\\x89\\xb2l\\x14\\x97\\xd6\\xff(\\x04Gi\\xc1\\xb8M\\xcd-\\xb9\\xc1\\xbf\\x1c\\x1aK\\xa6\\x1b\\x92]\\xfa\\xbc\\t6\\b\\xae!((\\x87\\vV\\x12\\x96\\xacF\\xb1d\\x06\\x7fg[\\x91Ǔ\\x8cp\\x91\\xb5\\xd2lp\\xb88\\xa87\\x99\\x88\\t\\xdd\\t\\xd3\\x1e\\xe1c\\xdd`A6%\\xb5\\xd2&\\xbe\\xe5m,!\\f`\\xc9ʾv\\xf2מ\\xbel\\b\\x19.\\x9ar5\\xfa\\x1er\\x84\\\"\\xaf2\\xc1\\xef\\x18\\xea\\xda\\xc8$\\xfa\\x91)\\xfd\\x8e \\xdf\\xee\\xd1\\xd8(í\\xd2\\xefD8\\x84ơ\\x1b\\x9c\\xb4\\b}\\x05\\x93\\x05\\x8a[\\xc4[\\xfa\\x9d\\xc0eI\\x1a\\xc7\\u038d\\t\\x80\\x02UϨ\\x92;E\\x17+1\\x04\\xac,\\xad \\xaf6h\\xf3b\\xcaL(\\xe3\\x12\\x8eI/\\xa4\\xc9\\xedPԍR\\x02\\xd9P\\x83\\x85\\xe1k\\xc9\\x1aS);!\\xf0j\\vq\\xe5\\xcb{\\x83t\\xf8r\\xbd\\xfaH\\xff\\xc4q\\xf2\\xa0=/[\\x88\\xa7[F\\xd9V\\xdel\\xad\\x9d\\x97\\xeb\\x15\\x98v\\xfb\\xd8H\\xd2\\t\\xc16\\x02\\x17`\\xb5\\x1b\\vv\\xdaa=\\xf7\\x9a\\xefQ\\xe7f\\x867\\xc7/\\x8c^\\x18\\xb6\\x813>\\xa9\\xf6C\\xafT\\x90`\\x94r\\xa9\\xa4E\\x99\\xb3\\xd1Y\\xaf\\xa2/J\\xba\\x14\\xccdy\\x1ep\\xb6N\\xd7\\xe7\\xaeI$\\b\\x85_a+\\x96\\xe7\\vB\\xd0\\xf5r\\x1c7\\xf1.7\\x83\\x03\\xb7\\xd5M\\x12\\x85\\vz\\xb1@\\xc9\\xf2\\xac<\\xed}\\x0f\\xe2\\xa8\\xed\\x19a\\x9e_\\x97^\\xde)\\xc9(\\xdc\\xdc\\\"پg\\xf4\\vd\\xeb{IN\\xba\\x01\\x97\\xa7\\x84S\\x84\\x02\\x04fX\\x82k\\xae\\xe7\\x9d@\\x87k,\\xc7<\\xcfz\\xf6\\xcaL\\xf7\\x85>\\x81$\\xa3\\xc8\\x04m\\xd2\\xf9\\x13\\xa5\\x95K%\\xb7|7>;-\\xf3\\xcf]۳\\xa2\\x8d\\\"^r$i\\x9c\\x02\\x1cq2\\xf3\\x19\\xee,F?\\xca\\r\\xb7|\\xe7\\xf4)4\\xdar\\x14\\xe5(\\x81\\x99\\x04\\xa0\\t}x&n\\x89#\\x9dd1~\\xb7\\x90\\x9ad\\xf6\\xc1KR\\x94\\n\\xe1o,\\x03\\x10t\\x1f)r\\x03\\x1f>\\x80\\xd2\\xf0!\\xb4\\x84>|\\f\\xbb\\x1d\\x17v\\xc6{\\xe5Ł\\v\\x11O\\xb9*\\x82v%\\x05\\x15t\\xcaM\\x85\\x96\\xac\\x0e\\xbe\\fh\\fTa\\xa9\\xf8\\xf4\\xe2[\\x05\\aƓ\\xb4\\xbe;\\xdd|\\xcc\\xd0\\xdd\\xe0\\x96r@\\x8d\\xd6iIQ\\x18\\xb5\\xa6\\xb4\\xc8x\\x92\\xcae\\xc2\\xd0\\x19IM\\x12\\x12'\\xa4\\x1cFO/\\x05\\xfd\\x7f\\x88\\xe5)\\x00d\\x04\\xc8\\xd9\\xf8\\x1c\\x87>e\\xef\\xfao\\xb7\\x98b\\xdd'\\x11\\x99W\\x9a\\xef8)\\\\v3\\xc7d\\xacź\\xb6k\\xe1\\x91\\xccCq\\xd6?;\\xb44\\x84\\x96Grt\\x9d\\xc3\\xe1\\x84\\xf6L\\x96>_\\xe8\\xe6\\xcb\\xf6\\xeae.\\xee\\xa4B\\x9e_\\x97S\\xf6\\xea\\x0e\\xce@9\\r\\x1f*^T}\\xd3\\xf11\\xa8\\x02X\\xf6\\x86>\\xf7\\xbe\\x82\\xcd<\\x86\\xcf\\xf2\\x99\\xf8`\\xcd\\xf0\\xf6\\r\\xa6S\\x97\\x1dN\\xf5\\r\\x9d\\x9d}~]^T\\xad\\xf8F\\xcae\\xf5Jh\\xe4\\xb6Z.\\x9c־\\x12\\f\\xa3j{S\\xc5\\u008a\\x02\\x1b\\x8b\\xe5\\xc3\\xfb\\x93*\\xa7\\x9c\\xfeso11\\\"/i%eL\\xed\\x9bKذkK\\x8e\\xc8n\\xd7\\x00\\xbb\\xe5\\x9a~\\x1e\\x12\\xf1\\xad\\x10]&\\x809. \\x02\\u061cf\\x1a\\xe0\\x85\\x1cܗ\\xf2\\xdf\\x05\\x8c\\xa4m\\x1ey\\xe9z\\x8e\\x0e\\x1dQ\\x88=W\\xaa\\xd5g\\xb4\\xff\\xb6(\\x9b/\\xd5B\\xff;m\\x1d\\xdeT\\xb7\\x8dɌu\\xc7b\\x81\\xe9{\\x9a\\xb1\\xf1\\x9e\\xd3ؑ\\\\\\xa7\\xaf@\\rK\\xc0=J\\xa0R\\x9cqA\\xb1ۓ\\xcc\\x00\\xd8y*m\\x10\\v\\xaf,\\xb1G\\x13\\xfby\\xd9fٴ%3J\\x18\\xa3\\xd9oi\\xcc.\\x85\\xfc\\x8aƉL\\xd2\\xf0\\x1b\\xa6\\x90\\xe1\\xc8\\xd0-0\\xd9\\x14\\xf2|9\\xcb\\f0ЁH\\x8b\\x1b\\xa7@\\xebb%e\\xf3\\xca\\xe1\\xdb\\xc4T\\xd5>X\\x0e\\x95\\x12\\xadSKWoP\\x13\\xb7\\xfe\\x85\\x04$\\x1e(-,*&w\\xd9\\xc4#v\\xf8\\x11\\x043\\xb6u\\xb7\\x93\\x1e\\x92>\\xb1\\f%K\\x9fD\\x8e_\\x8dư\\xdd\\x14X\\xff\\x14V\\x85\\xa6e\\xbb\\x05؆2ľֿ3m\\f\\xb9\\n\\x89\\xe5t\\xb8\\xb8*H\\xf4\\xde\\x1b\\xae\\xe6\\xe4\\xcb\\xfa\\x02^\\xbe\\xac\\xe9\\x90/\\xeb_\\xcb\\vJW\\xe7jF\\xe6\\xac\\xca\\f\\v.\\xdd/\\x99\\xf1\\x03\\x97\\xa5:\\x8c\\xa1㌨\\r\\xb3Մ\\xa0\\xcf\\xccV1E\\xd8:!\\xfc\\x9eQ\\xea\\xdcf\\x9d\\x1b$L\\xfcV\\x19\\xb4\\xef\\xaaM\\xb1Gkr)\\f^\\x02\\a\\xa74\\xff\\x84\\x87\\xcch\\f\\xb9\\x99\\xa9\\xe76\\x8eg\\xa6Fo\\xe3\\xe9dh\\\\\\xe6\\xe02\\xceeiv\\xcfϙ\\xb9\\xbf\\xfb\\x00w\\x95\\x9e[\\xfen\\x89\\xe0]\\v\\xf4\\x88o\\xfe5y\\x84r\\xfdV\\f\\x95\\x14\\x89\\xc52\\x84\\x93\\xfd]\\x1d\\xe3)\\xcd\\xe1\\xa5\\xe2&6mc%Zr\\xd3\\b\\xf6\\xde\\xc92\\x156:\\xdc\\x1a>ƍ\\x9d\\xe4|\\xb7\\xb3{\\xc4\\xcfw\\xaaΣ2L \\xb3\\x9fW\\xa7Cη8\\xe1L̋\\xd7{\\xf5xa\\x89\\xbdz\\x8cW\\x91\\x97(-\\xdf\\xf2\\xe4\\x01\\xf4X\\xac\\xf9\\x86zN\\x97Ç\\x84\\xeb\\xea\\xcbޟv\\xdcTo\\xf7(Ld\\xa2\\xed_\\x9a\\xe4\\xf2\\xbd5\\x81\\x01A\\x90\\x7fr[\\x0e\\x1f\\xd9?v\\x11\\x9d\\xd9\\xf6\\xdd/\\x04\\xff\\\\\\x11\\xab$\\xa57>=\\xba>\\xb5\\xec\\v\\xf4{f\\x95Y\\xaf\\x1a\\rz\\xce˄v\\xdb&MGܦ{\\x88]\\xc0\\x7f\\xfew\\xf7\\xff\\x00\\x00\\x00\\xff\\xff\\x12=\\xc7\\xe9\\x11&\\x00\\x00\"),\n}\n\nvar CRDs = crds()\n\nfunc crds() []*apiextv1.CustomResourceDefinition {\n\tapiextinstall.Install(scheme.Scheme)\n\tdecode := scheme.Codecs.UniversalDeserializer().Decode\n\tvar objs []*apiextv1.CustomResourceDefinition\n\tfor _, crd := range rawCRDs {\n\t\tgzr, err := gzip.NewReader(bytes.NewReader(crd))\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\tbytes, err := io.ReadAll(gzr)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\tgzr.Close()\n\n\t\tobj, _, err := decode(bytes, nil, nil)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\tobjs = append(objs, obj.(*apiextv1.CustomResourceDefinition))\n\t}\n\treturn objs\n}\n" }, { "path": "config/crd/v2alpha1/crds/doc.go", "content": "// Package crds embeds the controller-tools generated CRD manifests\npackage crds\n\n//go:generate go run ../../../../hack/crd-gen/v1/main.go\n" }, { "path": "config/rbac/role.yaml", "content": "---\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRole\nmetadata:\n name: velero-perms\nrules:\n- apiGroups:\n - \"\"\n resources:\n - persistentvolumerclaims\n - persistentvolumes\n - pods\n verbs:\n - get\n- apiGroups:\n - velero.io\n resources:\n - backuprepositories\n - backups\n - backupstoragelocations\n - datadownloads\n - datauploads\n - deletebackuprequests\n - downloadrequests\n - podvolumebackups\n - podvolumerestores\n - restores\n - schedules\n - serverstatusrequests\n - volumesnapshotlocations\n verbs:\n - create\n - delete\n - get\n - list\n - patch\n - update\n - watch\n- apiGroups:\n - velero.io\n resources:\n - backuprepositories/status\n - backups/status\n - backupstoragelocations/status\n - datadownloads/status\n - datauploads/status\n - deletebackuprequests/status\n - downloadrequests/status\n - podvolumebackups/status\n - podvolumerestores/status\n - restores/status\n - schedules/status\n - serverstatusrequests/status\n verbs:\n - get\n - patch\n - update\n" }, { "path": "design/2082-bsl-delete-associated-resources_design.md", "content": "# Delete Backup and Restic Repo Resources when BSL is Deleted\n\n## Abstract\n\nIssue #2082 requested that with the command `velero backup-location delete ` (implemented in Velero 1.6 with #3073), the following will be deleted:\n\n- associated Velero backups (to be clear, these are custom Kubernetes resources called \"backups\" that are stored in the API server)\n- associated Restic repositories (custom Kubernetes resources called \"resticrepositories\")\n\nThis design doc explains how the request will be implemented.\n\n## Background\n\nWhen a BSL resource is deleted from its Velero namespace, the associated custom Kubernetes resources, backups and Restic repositories, can no longer be used.\nIt makes sense to clean those resources up when a BSL is deleted.\n\n## Goals\n\nUpdate the `velero backup-location delete ` command to delete associated backup and Restic repository resources in the same Velero namespace.\n\n## Non Goals\n\n[It was suggested](https://github.com/vmware-tanzu/velero/issues/2082#issuecomment-827951311) to fix bug #2697 alongside this issue.\nHowever, I think that should be fixed separately because although it is similar (restore objects are not being deleted), it is also quite different.\nOne is adding a command feature update (this issue) and the other is a bug fix and each affect different parts of the code base.\n\n## High-Level Design\n\nUpdate the `velero backup-location delete ` command to do the following:\n\n- find in the same Velero namespace from which the BSL was deleted the associated backup resources and Restic repositories, called \"backups.velero.io\" and \"resticrepositories.velero.io\" respectively\n- delete the resources found\n\nThe above logic will be added to [where BSLs are deleted](https://github.com/vmware-tanzu/velero/blob/main/pkg/cmd/cli/backuplocation/delete.go).\n\n## Alternative Considered\n\nI had considered deleting the backup files (the ones in json format and tarballs) in the BSL itself.\nHowever, a standard use case is to back up a cluster and then restore into a new cluster.\nDeleting the backup storage location in either location is not expected to remove all of the backups in the backup storage location and should not be done.\n" }, { "path": "design/CLI/PoC/base/CRDs.yaml", "content": "---\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: (unknown)\n labels:\n component: velero\n name: backups.velero.io\nspec:\n group: velero.io\n names:\n kind: Backup\n listKind: BackupList\n plural: backups\n singular: backup\n scope: \"\"\n validation:\n openAPIV3Schema:\n description: Backup is a Velero resource that represents the capture of Kubernetes\n cluster state at a point in time (API objects and associated volume state).\n properties:\n apiVersion:\n description: 'APIVersion defines the versioned schema of this representation\n of an object. Servers should convert recognized schemas to the latest\n internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources'\n type: string\n kind:\n description: 'Kind is a string value representing the REST resource this\n object represents. Servers may infer this from the endpoint the client\n submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'\n type: string\n metadata:\n type: object\n spec:\n description: BackupSpec defines the specification for a Velero backup.\n properties:\n excludedNamespaces:\n description: ExcludedNamespaces contains a list of namespaces that\n are not included in the backup.\n items:\n type: string\n nullable: true\n type: array\n excludedResources:\n description: ExcludedResources is a slice of resource names that are\n not included in the backup.\n items:\n type: string\n nullable: true\n type: array\n hooks:\n description: Hooks represent custom behaviors that should be executed\n at different phases of the backup.\n properties:\n resources:\n description: Resources are hooks that should be executed when\n backing up individual instances of a resource.\n items:\n description: BackupResourceHookSpec defines one or more BackupResourceHooks\n that should be executed based on the rules defined for namespaces,\n resources, and label selector.\n properties:\n excludedNamespaces:\n description: ExcludedNamespaces specifies the namespaces\n to which this hook spec does not apply.\n items:\n type: string\n nullable: true\n type: array\n excludedResources:\n description: ExcludedResources specifies the resources to\n which this hook spec does not apply.\n items:\n type: string\n nullable: true\n type: array\n includedNamespaces:\n description: IncludedNamespaces specifies the namespaces\n to which this hook spec applies. If empty, it applies\n to all namespaces.\n items:\n type: string\n nullable: true\n type: array\n includedResources:\n description: IncludedResources specifies the resources to\n which this hook spec applies. If empty, it applies to\n all resources.\n items:\n type: string\n nullable: true\n type: array\n labelSelector:\n description: LabelSelector, if specified, filters the resources\n to which this hook spec applies.\n nullable: true\n properties:\n matchExpressions:\n description: matchExpressions is a list of label selector\n requirements. The requirements are ANDed.\n items:\n description: A label selector requirement is a selector\n that contains values, a key, and an operator that\n relates the key and values.\n properties:\n key:\n description: key is the label key that the selector\n applies to.\n type: string\n operator:\n description: operator represents a key's relationship\n to a set of values. Valid operators are In,\n NotIn, Exists and DoesNotExist.\n type: string\n values:\n description: values is an array of string values.\n If the operator is In or NotIn, the values array\n must be non-empty. If the operator is Exists\n or DoesNotExist, the values array must be empty.\n This array is replaced during a strategic merge\n patch.\n items:\n type: string\n type: array\n required:\n - key\n - operator\n type: object\n type: array\n matchLabels:\n additionalProperties:\n type: string\n description: matchLabels is a map of {key,value} pairs.\n A single {key,value} in the matchLabels map is equivalent\n to an element of matchExpressions, whose key field\n is \"key\", the operator is \"In\", and the values array\n contains only \"value\". The requirements are ANDed.\n type: object\n type: object\n name:\n description: Name is the name of this hook.\n type: string\n post:\n description: PostHooks is a list of BackupResourceHooks\n to execute after storing the item in the backup. These\n are executed after all \"additional items\" from item actions\n are processed.\n items:\n description: BackupResourceHook defines a hook for a resource.\n properties:\n exec:\n description: Exec defines an exec hook.\n properties:\n command:\n description: Command is the command and arguments\n to execute.\n items:\n type: string\n minItems: 1\n type: array\n container:\n description: Container is the container in the\n pod where the command should be executed. If\n not specified, the pod's first container is\n used.\n type: string\n onError:\n description: OnError specifies how Velero should\n behave if it encounters an error executing this\n hook.\n items:\n enum:\n - Continue\n - Fail\n type: string\n timeout:\n description: Timeout defines the maximum amount\n of time Velero should wait for the hook to complete\n before considering the execution a failure.\n type: string\n required:\n - command\n type: object\n required:\n - exec\n type: object\n type: array\n pre:\n description: PreHooks is a list of BackupResourceHooks to\n execute prior to storing the item in the backup. These\n are executed before any \"additional items\" from item actions\n are processed.\n items:\n description: BackupResourceHook defines a hook for a resource.\n properties:\n exec:\n description: Exec defines an exec hook.\n properties:\n command:\n description: Command is the command and arguments\n to execute.\n items:\n type: string\n minItems: 1\n type: array\n container:\n description: Container is the container in the\n pod where the command should be executed. If\n not specified, the pod's first container is\n used.\n type: string\n onError:\n description: OnError specifies how Velero should\n behave if it encounters an error executing this\n hook.\n items:\n enum:\n - Continue\n - Fail\n type: string\n timeout:\n description: Timeout defines the maximum amount\n of time Velero should wait for the hook to complete\n before considering the execution a failure.\n type: string\n required:\n - command\n type: object\n required:\n - exec\n type: object\n type: array\n required:\n - name\n type: object\n nullable: true\n type: array\n type: object\n includeClusterResources:\n description: IncludeClusterResources specifies whether cluster-scoped\n resources should be included for consideration in the backup.\n nullable: true\n type: boolean\n includedNamespaces:\n description: IncludedNamespaces is a slice of namespace names to include\n objects from. If empty, all namespaces are included.\n items:\n type: string\n nullable: true\n type: array\n includedResources:\n description: IncludedResources is a slice of resource names to include\n in the backup. If empty, all resources are included.\n items:\n type: string\n nullable: true\n type: array\n labelSelector:\n description: LabelSelector is a metav1.LabelSelector to filter with\n when adding individual objects to the backup. If empty or nil, all\n objects are included. Optional.\n nullable: true\n properties:\n matchExpressions:\n description: matchExpressions is a list of label selector requirements.\n The requirements are ANDed.\n items:\n description: A label selector requirement is a selector that\n contains values, a key, and an operator that relates the key\n and values.\n properties:\n key:\n description: key is the label key that the selector applies\n to.\n type: string\n operator:\n description: operator represents a key's relationship to\n a set of values. Valid operators are In, NotIn, Exists\n and DoesNotExist.\n type: string\n values:\n description: values is an array of string values. If the\n operator is In or NotIn, the values array must be non-empty.\n If the operator is Exists or DoesNotExist, the values\n array must be empty. This array is replaced during a strategic\n merge patch.\n items:\n type: string\n type: array\n required:\n - key\n - operator\n type: object\n type: array\n matchLabels:\n additionalProperties:\n type: string\n description: matchLabels is a map of {key,value} pairs. A single\n {key,value} in the matchLabels map is equivalent to an element\n of matchExpressions, whose key field is \"key\", the operator\n is \"In\", and the values array contains only \"value\". The requirements\n are ANDed.\n type: object\n type: object\n snapshotVolumes:\n description: SnapshotVolumes specifies whether to take cloud snapshots\n of any PV's referenced in the set of objects included in the Backup.\n nullable: true\n type: boolean\n storageLocation:\n description: StorageLocation is a string containing the name of a\n BackupStorageLocation where the backup should be stored.\n type: string\n ttl:\n description: TTL is a time.Duration-parseable string describing how\n long the Backup should be retained for.\n type: string\n volumeSnapshotLocations:\n description: VolumeSnapshotLocations is a list containing names of\n VolumeSnapshotLocations associated with this backup.\n items:\n type: string\n type: array\n type: object\n status:\n description: BackupStatus captures the current status of a Velero backup.\n properties:\n completionTimestamp:\n description: CompletionTimestamp records the time a backup was completed. Completion time is recorded even on failed backups. Completion time is recorded before uploading the backup object. The server's time is used for CompletionTimestamps\n format: date-time\n nullable: true\n type: string\n errors:\n description: Errors is a count of all error messages that were generated during execution of the backup. The actual errors are in the backup's log file in object storage.\n type: integer\n expiration:\n description: Expiration is when this Backup is eligible for garbage-collection.\n format: date-time\n nullable: true\n type: string\n phase:\n description: Phase is the current state of the Backup.\n items:\n enum:\n - New\n - FailedValidation\n - InProgress\n - Completed\n - PartiallyFailed\n - Failed\n - Deleting\n type: string\n startTimestamp:\n description: StartTimestamp records the time a backup was started.\n Separate from CreationTimestamp, since that value changes on restores.\n The server's time is used for StartTimestamps\n format: date-time\n nullable: true\n type: string\n validationErrors:\n description: ValidationErrors is a slice of all validation errors\n (if applicable).\n items:\n type: string\n nullable: true\n type: array\n version:\n description: Version is the backup format version.\n type: integer\n volumeSnapshotsAttempted:\n description: VolumeSnapshotsAttempted is the total number of attempted\n volume snapshots for this backup.\n type: integer\n volumeSnapshotsCompleted:\n description: VolumeSnapshotsCompleted is the total number of successfully\n completed volume snapshots for this backup.\n type: integer\n warnings:\n description: Warnings is a count of all warning messages that were\n generated during execution of the backup. The actual warnings are\n in the backup's log file in object storage.\n type: integer\n type: object\n type: object\n version: v1beta1\n versions:\n - name: v1beta1\n served: true\n storage: true\n---\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: (unknown)\n labels:\n component: velero\n name: deletebackuprequests.velero.io\nspec:\n group: velero.io\n names:\n kind: DeleteBackupRequest\n listKind: DeleteBackupRequestList\n plural: deletebackuprequests\n singular: deletebackuprequest\n scope: \"\"\n validation:\n openAPIV3Schema:\n description: DeleteBackupRequest is a request to delete one or more backups.\n properties:\n apiVersion:\n description: 'APIVersion defines the versioned schema of this representation\n of an object. Servers should convert recognized schemas to the latest\n internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources'\n type: string\n kind:\n description: 'Kind is a string value representing the REST resource this\n object represents. Servers may infer this from the endpoint the client\n submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'\n type: string\n metadata:\n type: object\n spec:\n description: DeleteBackupRequestSpec is the specification for which backups\n to delete.\n properties:\n backupName:\n type: string\n required:\n - backupName\n type: object\n status:\n description: DeleteBackupRequestStatus is the current status of a DeleteBackupRequest.\n properties:\n errors:\n description: Errors contains any errors that were encountered during\n the deletion process.\n items:\n type: string\n nullable: true\n type: array\n phase:\n description: Phase is the current state of the DeleteBackupRequest.\n items:\n enum:\n - New\n - InProgress\n - Processed\n type: string\n type: object\n type: object\n version: v1beta1\n versions:\n - name: v1beta1\n served: true\n storage: true\n---\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: (unknown)\n labels:\n component: velero\n name: downloadrequests.velero.io\nspec:\n group: velero.io\n names:\n kind: DownloadRequest\n listKind: DownloadRequestList\n plural: downloadrequests\n singular: downloadrequest\n scope: \"\"\n validation:\n openAPIV3Schema:\n description: DownloadRequest is a request to download an artifact from backup\n object storage, such as a backup log file.\n properties:\n apiVersion:\n description: 'APIVersion defines the versioned schema of this representation\n of an object. Servers should convert recognized schemas to the latest\n internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources'\n type: string\n kind:\n description: 'Kind is a string value representing the REST resource this\n object represents. Servers may infer this from the endpoint the client\n submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'\n type: string\n metadata:\n type: object\n spec:\n description: DownloadRequestSpec is the specification for a download request.\n properties:\n target:\n description: Target is what to download (e.g. logs for a backup).\n properties:\n kind:\n description: Kind is the type of file to download.\n items:\n enum:\n - BackupLog\n - BackupContents\n - BackupVolumeSnapshot\n - BackupResourceList\n - RestoreLog\n - RestoreResults\n - CSIBackupVolumeSnapshots\n - CSIBackupVolumeSnapshotContents\n type: string\n name:\n description: Name is the name of the Kubernetes resource with\n which the file is associated.\n type: string\n required:\n - kind\n - name\n type: object\n required:\n - target\n type: object\n status:\n description: DownloadRequestStatus is the current status of a DownloadRequest.\n properties:\n downloadURL:\n description: DownloadURL contains the pre-signed URL for the target\n file.\n type: string\n expiration:\n description: Expiration is when this DownloadRequest expires and can\n be deleted by the system.\n format: date-time\n nullable: true\n type: string\n phase:\n description: Phase is the current state of the DownloadRequest.\n items:\n enum:\n - New\n - Processed\n type: string\n type: object\n type: object\n version: v1beta1\n versions:\n - name: v1beta1\n served: true\n storage: true\n---\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: (unknown)\n labels:\n component: velero\n name: restores.velero.io\nspec:\n group: velero.io\n names:\n kind: Restore\n listKind: RestoreList\n plural: restores\n singular: restore\n scope: \"\"\n validation:\n openAPIV3Schema:\n description: Restore is a Velero resource that represents the application\n of resources from a Velero backup to a target Kubernetes cluster.\n properties:\n apiVersion:\n description: 'APIVersion defines the versioned schema of this representation\n of an object. Servers should convert recognized schemas to the latest\n internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources'\n type: string\n kind:\n description: 'Kind is a string value representing the REST resource this\n object represents. Servers may infer this from the endpoint the client\n submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'\n type: string\n metadata:\n type: object\n spec:\n description: RestoreSpec defines the specification for a Velero restore.\n properties:\n backupName:\n description: BackupName is the unique name of the Velero backup to\n restore from.\n type: string\n excludedNamespaces:\n description: ExcludedNamespaces contains a list of namespaces that\n are not included in the restore.\n items:\n type: string\n nullable: true\n type: array\n excludedResources:\n description: ExcludedResources is a slice of resource names that are\n not included in the restore.\n items:\n type: string\n nullable: true\n type: array\n includeClusterResources:\n description: IncludeClusterResources specifies whether cluster-scoped\n resources should be included for consideration in the restore. If\n null, defaults to true.\n nullable: true\n type: boolean\n includedNamespaces:\n description: IncludedNamespaces is a slice of namespace names to include\n objects from. If empty, all namespaces are included.\n items:\n type: string\n nullable: true\n type: array\n includedResources:\n description: IncludedResources is a slice of resource names to include\n in the restore. If empty, all resources in the backup are included.\n items:\n type: string\n nullable: true\n type: array\n labelSelector:\n description: LabelSelector is a metav1.LabelSelector to filter with\n when restoring individual objects from the backup. If empty or nil,\n all objects are included. Optional.\n nullable: true\n properties:\n matchExpressions:\n description: matchExpressions is a list of label selector requirements.\n The requirements are ANDed.\n items:\n description: A label selector requirement is a selector that\n contains values, a key, and an operator that relates the key\n and values.\n properties:\n key:\n description: key is the label key that the selector applies\n to.\n type: string\n operator:\n description: operator represents a key's relationship to\n a set of values. Valid operators are In, NotIn, Exists\n and DoesNotExist.\n type: string\n values:\n description: values is an array of string values. If the\n operator is In or NotIn, the values array must be non-empty.\n If the operator is Exists or DoesNotExist, the values\n array must be empty. This array is replaced during a strategic\n merge patch.\n items:\n type: string\n type: array\n required:\n - key\n - operator\n type: object\n type: array\n matchLabels:\n additionalProperties:\n type: string\n description: matchLabels is a map of {key,value} pairs. A single\n {key,value} in the matchLabels map is equivalent to an element\n of matchExpressions, whose key field is \"key\", the operator\n is \"In\", and the values array contains only \"value\". The requirements\n are ANDed.\n type: object\n type: object\n namespaceMapping:\n additionalProperties:\n type: string\n description: NamespaceMapping is a map of source namespace names to\n target namespace names to restore into. Any source namespaces not\n included in the map will be restored into namespaces of the same\n name.\n type: object\n restorePVs:\n description: RestorePVs specifies whether to restore all included\n PVs from snapshot (via the cloudprovider).\n nullable: true\n type: boolean\n preserveNodePorts:\n description: PreserveNodePorts specifies whether to restore old nodePorts from backup.\n nullable: true\n type: boolean\n scheduleName:\n description: ScheduleName is the unique name of the Velero schedule\n to restore from. If specified, and BackupName is empty, Velero will\n restore from the most recent successful backup created from this\n schedule.\n type: string\n required:\n - backupName\n type: object\n status:\n description: RestoreStatus captures the current status of a Velero restore\n properties:\n errors:\n description: Errors is a count of all error messages that were generated\n during execution of the restore. The actual errors are stored in\n object storage.\n type: integer\n failureReason:\n description: FailureReason is an error that caused the entire restore\n to fail.\n type: string\n phase:\n description: Phase is the current state of the Restore\n items:\n enum:\n - New\n - FailedValidation\n - InProgress\n - Completed\n - PartiallyFailed\n - Failed\n type: string\n validationErrors:\n description: ValidationErrors is a slice of all validation errors\n (if applicable)\n items:\n type: string\n nullable: true\n type: array\n warnings:\n description: Warnings is a count of all warning messages that were\n generated during execution of the restore. The actual warnings are\n stored in object storage.\n type: integer\n type: object\n type: object\n version: v1beta1\n versions:\n - name: v1beta1\n served: true\n storage: true\n---\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: (unknown)\n labels:\n component: velero\n name: schedules.velero.io\nspec:\n group: velero.io\n names:\n kind: Schedule\n listKind: ScheduleList\n plural: schedules\n singular: schedule\n scope: \"\"\n validation:\n openAPIV3Schema:\n description: Schedule is a Velero resource that represents a pre-scheduled\n or periodic Backup that should be run.\n properties:\n apiVersion:\n description: 'APIVersion defines the versioned schema of this representation\n of an object. Servers should convert recognized schemas to the latest\n internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources'\n type: string\n kind:\n description: 'Kind is a string value representing the REST resource this\n object represents. Servers may infer this from the endpoint the client\n submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'\n type: string\n metadata:\n type: object\n spec:\n description: ScheduleSpec defines the specification for a Velero schedule\n properties:\n schedule:\n description: Schedule is a Cron expression defining when to run the\n Backup.\n type: string\n template:\n description: Template is the definition of the Backup to be run on\n the provided schedule\n properties:\n excludedNamespaces:\n description: ExcludedNamespaces contains a list of namespaces\n that are not included in the backup.\n items:\n type: string\n nullable: true\n type: array\n excludedResources:\n description: ExcludedResources is a slice of resource names that\n are not included in the backup.\n items:\n type: string\n nullable: true\n type: array\n hooks:\n description: Hooks represent custom behaviors that should be executed\n at different phases of the backup.\n properties:\n resources:\n description: Resources are hooks that should be executed when\n backing up individual instances of a resource.\n items:\n description: BackupResourceHookSpec defines one or more\n BackupResourceHooks that should be executed based on the\n rules defined for namespaces, resources, and label selector.\n properties:\n excludedNamespaces:\n description: ExcludedNamespaces specifies the namespaces\n to which this hook spec does not apply.\n items:\n type: string\n nullable: true\n type: array\n excludedResources:\n description: ExcludedResources specifies the resources\n to which this hook spec does not apply.\n items:\n type: string\n nullable: true\n type: array\n includedNamespaces:\n description: IncludedNamespaces specifies the namespaces\n to which this hook spec applies. If empty, it applies\n to all namespaces.\n items:\n type: string\n nullable: true\n type: array\n includedResources:\n description: IncludedResources specifies the resources\n to which this hook spec applies. If empty, it applies\n to all resources.\n items:\n type: string\n nullable: true\n type: array\n labelSelector:\n description: LabelSelector, if specified, filters the\n resources to which this hook spec applies.\n nullable: true\n properties:\n matchExpressions:\n description: matchExpressions is a list of label\n selector requirements. The requirements are ANDed.\n items:\n description: A label selector requirement is a\n selector that contains values, a key, and an\n operator that relates the key and values.\n properties:\n key:\n description: key is the label key that the\n selector applies to.\n type: string\n operator:\n description: operator represents a key's relationship\n to a set of values. Valid operators are\n In, NotIn, Exists and DoesNotExist.\n type: string\n values:\n description: values is an array of string\n values. If the operator is In or NotIn,\n the values array must be non-empty. If the\n operator is Exists or DoesNotExist, the\n values array must be empty. This array is\n replaced during a strategic merge patch.\n items:\n type: string\n type: array\n required:\n - key\n - operator\n type: object\n type: array\n matchLabels:\n additionalProperties:\n type: string\n description: matchLabels is a map of {key,value}\n pairs. A single {key,value} in the matchLabels\n map is equivalent to an element of matchExpressions,\n whose key field is \"key\", the operator is \"In\",\n and the values array contains only \"value\". The\n requirements are ANDed.\n type: object\n type: object\n name:\n description: Name is the name of this hook.\n type: string\n post:\n description: PostHooks is a list of BackupResourceHooks\n to execute after storing the item in the backup. These\n are executed after all \"additional items\" from item\n actions are processed.\n items:\n description: BackupResourceHook defines a hook for\n a resource.\n properties:\n exec:\n description: Exec defines an exec hook.\n properties:\n command:\n description: Command is the command and arguments\n to execute.\n items:\n type: string\n minItems: 1\n type: array\n container:\n description: Container is the container in\n the pod where the command should be executed.\n If not specified, the pod's first container\n is used.\n type: string\n onError:\n description: OnError specifies how Velero\n should behave if it encounters an error\n executing this hook.\n items:\n enum:\n - Continue\n - Fail\n type: string\n timeout:\n description: Timeout defines the maximum amount\n of time Velero should wait for the hook\n to complete before considering the execution\n a failure.\n type: string\n required:\n - command\n type: object\n required:\n - exec\n type: object\n type: array\n pre:\n description: PreHooks is a list of BackupResourceHooks\n to execute prior to storing the item in the backup.\n These are executed before any \"additional items\" from\n item actions are processed.\n items:\n description: BackupResourceHook defines a hook for\n a resource.\n properties:\n exec:\n description: Exec defines an exec hook.\n properties:\n command:\n description: Command is the command and arguments\n to execute.\n items:\n type: string\n minItems: 1\n type: array\n container:\n description: Container is the container in\n the pod where the command should be executed.\n If not specified, the pod's first container\n is used.\n type: string\n onError:\n description: OnError specifies how Velero\n should behave if it encounters an error\n executing this hook.\n items:\n enum:\n - Continue\n - Fail\n type: string\n timeout:\n description: Timeout defines the maximum amount\n of time Velero should wait for the hook\n to complete before considering the execution\n a failure.\n type: string\n required:\n - command\n type: object\n required:\n - exec\n type: object\n type: array\n required:\n - name\n type: object\n nullable: true\n type: array\n type: object\n includeClusterResources:\n description: IncludeClusterResources specifies whether cluster-scoped\n resources should be included for consideration in the backup.\n nullable: true\n type: boolean\n includedNamespaces:\n description: IncludedNamespaces is a slice of namespace names\n to include objects from. If empty, all namespaces are included.\n items:\n type: string\n nullable: true\n type: array\n includedResources:\n description: IncludedResources is a slice of resource names to\n include in the backup. If empty, all resources are included.\n items:\n type: string\n nullable: true\n type: array\n labelSelector:\n description: LabelSelector is a metav1.LabelSelector to filter\n with when adding individual objects to the backup. If empty\n or nil, all objects are included. Optional.\n nullable: true\n properties:\n matchExpressions:\n description: matchExpressions is a list of label selector\n requirements. The requirements are ANDed.\n items:\n description: A label selector requirement is a selector\n that contains values, a key, and an operator that relates\n the key and values.\n properties:\n key:\n description: key is the label key that the selector\n applies to.\n type: string\n operator:\n description: operator represents a key's relationship\n to a set of values. Valid operators are In, NotIn,\n Exists and DoesNotExist.\n type: string\n values:\n description: values is an array of string values. If\n the operator is In or NotIn, the values array must\n be non-empty. If the operator is Exists or DoesNotExist,\n the values array must be empty. This array is replaced\n during a strategic merge patch.\n items:\n type: string\n type: array\n required:\n - key\n - operator\n type: object\n type: array\n matchLabels:\n additionalProperties:\n type: string\n description: matchLabels is a map of {key,value} pairs. A\n single {key,value} in the matchLabels map is equivalent\n to an element of matchExpressions, whose key field is \"key\",\n the operator is \"In\", and the values array contains only\n \"value\". The requirements are ANDed.\n type: object\n type: object\n snapshotVolumes:\n description: SnapshotVolumes specifies whether to take cloud snapshots\n of any PV's referenced in the set of objects included in the\n Backup.\n nullable: true\n type: boolean\n storageLocation:\n description: StorageLocation is a string containing the name of\n a BackupStorageLocation where the backup should be stored.\n type: string\n ttl:\n description: TTL is a time.Duration-parseable string describing\n how long the Backup should be retained for.\n type: string\n volumeSnapshotLocations:\n description: VolumeSnapshotLocations is a list containing names\n of VolumeSnapshotLocations associated with this backup.\n items:\n type: string\n type: array\n type: object\n required:\n - schedule\n - template\n type: object\n status:\n description: ScheduleStatus captures the current state of a Velero schedule\n properties:\n lastBackup:\n description: LastBackup is the last time a Backup was run for this Schedule schedule \n format: date-time \n nullable: true \n type: string \n phase: \n description: Phase is the current phase of the Schedule \n items: \n enum: \n - New \n - Enabled \n - FailedValidation \n type: string \n validationErrors: \n description: ValidationErrors is a slice of all validation errors (if applicable)\n items:\n type: string\n type: array\n type: object\n type: object\n version: v1beta1\n versions:\n - name: v1beta1\n served: true\n storage: true \n---\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: (unknown)\n labels:\n component: velero\n name: serverstatusrequests.velero.io\nspec:\n group: velero.io\n names:\n kind: ServerStatusRequest\n listKind: ServerStatusRequestList\n plural: serverstatusrequests\n singular: serverstatusrequest\n scope: \"\"\n validation:\n openAPIV3Schema:\n description: ServerStatusRequest is a request to access current status information\n about the Velero server.\n properties:\n apiVersion:\n description: 'APIVersion defines the versioned schema of this representation\n of an object. Servers should convert recognized schemas to the latest\n internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources'\n type: string\n kind:\n description: 'Kind is a string value representing the REST resource this\n object represents. Servers may infer this from the endpoint the client\n submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'\n type: string\n metadata:\n type: object\n spec:\n description: ServerStatusRequestSpec is the specification for a ServerStatusRequest.\n type: object\n status:\n description: ServerStatusRequestStatus is the current status of a ServerStatusRequest.\n properties:\n phase:\n description: Phase is the current lifecycle phase of the ServerStatusRequest.\n items:\n enum:\n - New\n - Processed\n type: string\n plugins:\n description: Plugins list information about the plugins running on\n the Velero server\n items:\n description: PluginInfo contains attributes of a Velero plugin\n properties:\n kind:\n type: string\n name:\n type: string\n required:\n - kind\n - name\n type: object\n nullable: true\n type: array\n processedTimestamp:\n description: ProcessedTimestamp is when the ServerStatusRequest was\n processed by the ServerStatusRequestController.\n format: date-time \n nullable: true \n type: string \n serverVersion: \n description: ServerVersion is the Velero server version. \n type: string\n type: object\n type: object\n version: v1beta1\n versions:\n - name: v1beta1\n served: true\n storage: true " }, { "path": "design/CLI/PoC/base/backupstoragelocations.yaml", "content": "---\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: (unknown)\n labels:\n component: velero\n name: backupstoragelocations.velero.io\nspec:\n group: velero.io\n names:\n kind: BackupStorageLocation\n listKind: BackupStorageLocationList\n plural: backupstoragelocations\n singular: backupstoragelocation\n scope: \"\"\n validation:\n openAPIV3Schema:\n description: BackupStorageLocation is a location where Velero stores backup\n objects.\n properties:\n apiVersion:\n description: 'APIVersion defines the versioned schema of this representation\n of an object. Servers should convert recognized schemas to the latest\n internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources'\n type: string\n kind:\n description: 'Kind is a string value representing the REST resource this\n object represents. Servers may infer this from the endpoint the client\n submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'\n type: string\n metadata:\n type: object\n spec:\n description: BackupStorageLocationSpec defines the specification for a\n Velero BackupStorageLocation.\n properties:\n accessMode:\n description: AccessMode defines the permissions for the backup storage\n location.\n enum:\n - ReadOnly\n - ReadWrite\n type: string\n backupSyncPeriod:\n description: BackupSyncPeriod defines how frequently to sync backup\n API objects from object storage. A value of 0 disables sync.\n nullable: true\n type: string\n config:\n additionalProperties:\n type: string\n description: Config is for provider-specific configuration fields.\n type: object\n objectStorage:\n description: ObjectStorageLocation specifies the settings necessary\n to connect to a provider's object storage.\n properties:\n bucket:\n description: Bucket is the bucket to use for object storage.\n type: string\n prefix:\n description: Prefix is the path inside a bucket to use for Velero\n storage. Optional.\n type: string\n required:\n - bucket\n type: object\n provider:\n description: Provider is the provider of the backup storage.\n type: string\n required:\n - objectStorage\n - provider\n type: object\n status:\n description: BackupStorageLocationStatus describes the current status\n of a Velero BackupStorageLocation.\n properties:\n accessMode:\n description: \"AccessMode is an unused field. \\n Deprecated: there\n is now an AccessMode field on the Spec and this field will be removed\n entirely as of v2.0.\"\n enum:\n - ReadOnly\n - ReadWrite\n type: string\n lastSyncedRevision:\n description: \"LastSyncedRevision is the value of the `metadata/revision`\n file in the backup storage location the last time the BSL's contents\n were synced into the cluster. \\n Deprecated: this field is no longer\n updated or used for detecting changes to the location's contents\n and will be removed entirely in v2.0.\"\n type: string\n lastSyncedTime:\n description: LastSyncedTime is the last time the contents of the location\n were synced into the cluster.\n format: date-time\n nullable: true\n type: string\n phase:\n description: Phase is the current state of the BackupStorageLocation.\n enum:\n - Available\n - Unavailable\n type: string\n type: object\n type: object\n version: v1\n versions:\n - name: v1\n served: true\n storage: true\n---\napiVersion: velero.io/v1\nkind: BackupStorageLocation\nmetadata:\n creationTimestamp: null\n labels:\n component: velero\n name: default\n namespace: velero\nspec:\n config:\n region: minio\n s3ForcePathStyle: \"true\"\n s3Url: http://minio.velero.svc:9000\n objectStorage:\n bucket: velero\n provider: aws\n" }, { "path": "design/CLI/PoC/base/deployment.yaml", "content": "---\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n labels:\n component: velero\n name: velero\n namespace: velero\nspec:\n selector:\n matchLabels:\n deploy: velero\n strategy: {}\n template:\n metadata:\n annotations:\n prometheus.io/path: /metrics\n prometheus.io/port: \"8085\"\n prometheus.io/scrape: \"true\"\n labels:\n component: velero\n deploy: velero\n spec:\n containers:\n - args:\n - server \n command:\n - /velero\n env:\n - name: VELERO_SCRATCH_DIR\n value: /scratch\n - name: VELERO_NAMESPACE\n valueFrom:\n fieldRef:\n fieldPath: metadata.namespace\n - name: LD_LIBRARY_PATH\n value: /plugins\n name: velero\n image: velero/velero:latest\n imagePullPolicy: Always\n ports:\n - containerPort: 8085\n name: metrics\n resources:\n limits:\n cpu: \"1\"\n memory: 256Mi\n requests:\n cpu: 500m\n memory: 128Mi\n volumeMounts:\n - mountPath: /scratch\n name: scratch\n restartPolicy: Always\n serviceAccountName: velero\n volumes:\n - emptyDir: {}\n name: scratch\n---\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRoleBinding\nmetadata:\n labels:\n component: velero\n name: velero\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: cluster-admin\nsubjects:\n - kind: ServiceAccount\n name: velero\n namespace: velero\n---\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n labels:\n component: velero\n name: velero\n namespace: velero\n---\napiVersion: v1\nkind: Namespace\nmetadata:\n labels:\n component: velero\n name: velero\nspec: {}" }, { "path": "design/CLI/PoC/base/kustomization.yaml", "content": "apiVersion: kustomize.config.k8s.io/v1beta1\nkind: Kustomization\n\nresources:\n - deployment.yaml\n - CRDs.yaml\n - backupstoragelocations.yaml\n - volumesnapshotlocations.yaml # including so the velero server can run\n - resticrepository.yaml # including so the velero server can runl\n - podvolumes.yaml # including so the velero server can runl\n - minio.yaml\n\n" }, { "path": "design/CLI/PoC/base/minio.yaml", "content": "# Copyright 2017 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n---\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n namespace: velero\n name: minio\n labels:\n component: minio\nspec:\n strategy:\n type: Recreate\n selector:\n matchLabels:\n component: minio\n template:\n metadata:\n labels:\n component: minio\n spec:\n volumes:\n - name: storage\n emptyDir: {}\n - name: config\n emptyDir: {}\n containers:\n - name: minio\n image: minio/minio:latest\n imagePullPolicy: IfNotPresent\n args:\n - server\n - /storage\n - --config-dir=/config\n env:\n - name: MINIO_ACCESS_KEY\n value: \"minio\"\n - name: MINIO_SECRET_KEY\n value: \"minio123\"\n ports:\n - containerPort: 9000\n volumeMounts:\n - name: storage\n mountPath: \"/storage\"\n - name: config\n mountPath: \"/config\"\n\n---\napiVersion: v1\nkind: Service\nmetadata:\n namespace: velero\n name: minio\n labels:\n component: minio\nspec:\n # ClusterIP is recommended for production environments.\n # Change to NodePort if needed per documentation,\n # but only if you run Minio in a test/trial environment, for example with Minikube.\n type: ClusterIP\n ports:\n - port: 9000\n targetPort: 9000\n protocol: TCP\n selector:\n component: minio\n\n---\napiVersion: batch/v1\nkind: Job\nmetadata:\n namespace: velero\n name: minio-setup\n labels:\n component: minio\nspec:\n template:\n metadata:\n name: minio-setup\n spec:\n restartPolicy: OnFailure\n volumes:\n - name: config\n emptyDir: {}\n containers:\n - name: mc\n image: minio/mc:latest\n imagePullPolicy: IfNotPresent\n command:\n - /bin/sh\n - -c\n - \"mc --config-dir=/config config host add velero http://minio:9000 minio minio123 && mc --config-dir=/config mb -p velero/velero\"\n volumeMounts:\n - name: config\n mountPath: \"/config\"\n" }, { "path": "design/CLI/PoC/base/podvolumes.yaml", "content": "---\n apiVersion: apiextensions.k8s.io/v1beta1\n kind: CustomResourceDefinition\n metadata:\n annotations:\n controller-gen.kubebuilder.io/version: (unknown)\n creationTimestamp: null\n labels:\n component: velero\n name: podvolumebackups.velero.io\n spec:\n group: velero.io\n names:\n kind: PodVolumeBackup\n listKind: PodVolumeBackupList\n plural: podvolumebackups\n singular: podvolumebackup\n scope: \"\"\n validation:\n openAPIV3Schema:\n properties:\n apiVersion:\n description: 'APIVersion defines the versioned schema of this representation\n of an object. Servers should convert recognized schemas to the latest\n internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources'\n type: string\n kind:\n description: 'Kind is a string value representing the REST resource this\n object represents. Servers may infer this from the endpoint the client\n submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'\n type: string\n metadata:\n type: object\n spec:\n description: PodVolumeBackupSpec is the specification for a PodVolumeBackup.\n properties:\n backupStorageLocation:\n description: BackupStorageLocation is the name of the backup storage\n location where the restic repository is stored.\n type: string\n node:\n description: Node is the name of the node that the Pod is running\n on.\n type: string\n pod:\n description: Pod is a reference to the pod containing the volume to\n be backed up.\n properties:\n apiVersion:\n description: API version of the referent.\n type: string\n fieldPath:\n description: 'If referring to a piece of an object instead of\n an entire object, this string should contain a valid JSON/Go\n field access statement, such as desiredState.manifest.containers[2].\n For example, if the object reference is to a container within\n a pod, this would take on a value like: \"spec.containers{name}\"\n (where \"name\" refers to the name of the container that triggered\n the event) or if no container name is specified \"spec.containers[2]\"\n (container with index 2 in this pod). This syntax is chosen\n only to have some well-defined way of referencing a part of\n an object. TODO: this design is not final and this field is\n subject to change in the future.'\n type: string\n kind:\n description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'\n type: string\n name:\n description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'\n type: string\n namespace:\n description: 'Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'\n type: string\n resourceVersion:\n description: 'Specific resourceVersion to which this reference\n is made, if any. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency'\n type: string\n uid:\n description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'\n type: string\n type: object\n repoIdentifier:\n description: RepoIdentifier is the restic repository identifier.\n type: string\n tags:\n additionalProperties:\n type: string\n description: Tags are a map of key-value pairs that should be applied\n to the volume backup as tags.\n type: object\n volume:\n description: Volume is the name of the volume within the Pod to be\n backed up.\n type: string\n required:\n - backupStorageLocation\n - node\n - pod\n - repoIdentifier\n - volume\n type: object\n status:\n description: PodVolumeBackupStatus is the current status of a PodVolumeBackup.\n properties:\n completionTimestamp:\n description: CompletionTimestamp records the time a backup was completed.\n Completion time is recorded even on failed backups. Completion time\n is recorded before uploading the backup object. The server's time\n is used for CompletionTimestamps\n format: date-time\n nullable: true\n type: string\n message:\n description: Message is a message about the pod volume backup's status.\n type: string\n path:\n description: Path is the full path within the controller pod being\n backed up.\n type: string\n phase:\n description: Phase is the current state of the PodVolumeBackup.\n enum:\n - New\n - InProgress\n - Completed\n - Failed\n type: string\n progress:\n description: Progress holds the total number of bytes of the volume\n and the current number of backed up bytes. This can be used to display\n progress information about the backup operation.\n properties:\n bytesDone:\n format: int64\n type: integer\n totalBytes:\n format: int64\n type: integer\n type: object\n snapshotID:\n description: SnapshotID is the identifier for the snapshot of the\n pod volume.\n type: string\n startTimestamp:\n description: StartTimestamp records the time a backup was started.\n Separate from CreationTimestamp, since that value changes on restores.\n The server's time is used for StartTimestamps\n format: date-time\n nullable: true\n type: string\n type: object\n type: object\n version: v1\n versions:\n - name: v1\n served: true\n storage: true\n---\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: (unknown)\n creationTimestamp: null\n labels:\n component: velero\n name: podvolumerestores.velero.io\nspec:\n group: velero.io\n names:\n kind: PodVolumeRestore\n listKind: PodVolumeRestoreList\n plural: podvolumerestores\n singular: podvolumerestore\n scope: \"\"\n validation:\n openAPIV3Schema:\n properties:\n apiVersion:\n description: 'APIVersion defines the versioned schema of this representation\n of an object. Servers should convert recognized schemas to the latest\n internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources'\n type: string\n kind:\n description: 'Kind is a string value representing the REST resource this\n object represents. Servers may infer this from the endpoint the client\n submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'\n type: string\n metadata:\n type: object\n spec:\n description: PodVolumeRestoreSpec is the specification for a PodVolumeRestore.\n properties:\n backupStorageLocation:\n description: BackupStorageLocation is the name of the backup storage\n location where the restic repository is stored.\n type: string\n pod:\n description: Pod is a reference to the pod containing the volume to\n be restored.\n properties:\n apiVersion:\n description: API version of the referent.\n type: string\n fieldPath:\n description: 'If referring to a piece of an object instead of\n an entire object, this string should contain a valid JSON/Go\n field access statement, such as desiredState.manifest.containers[2].\n For example, if the object reference is to a container within\n a pod, this would take on a value like: \"spec.containers{name}\"\n (where \"name\" refers to the name of the container that triggered\n the event) or if no container name is specified \"spec.containers[2]\"\n (container with index 2 in this pod). This syntax is chosen\n only to have some well-defined way of referencing a part of\n an object. TODO: this design is not final and this field is\n subject to change in the future.'\n type: string\n kind:\n description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'\n type: string\n name:\n description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'\n type: string\n namespace:\n description: 'Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'\n type: string\n resourceVersion:\n description: 'Specific resourceVersion to which this reference\n is made, if any. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency'\n type: string\n uid:\n description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'\n type: string\n type: object\n repoIdentifier:\n description: RepoIdentifier is the restic repository identifier.\n type: string\n snapshotID:\n description: SnapshotID is the ID of the volume snapshot to be restored.\n type: string\n volume:\n description: Volume is the name of the volume within the Pod to be\n restored.\n type: string\n required:\n - backupStorageLocation\n - pod\n - repoIdentifier\n - snapshotID\n - volume\n type: object\n status:\n description: PodVolumeRestoreStatus is the current status of a PodVolumeRestore.\n properties:\n completionTimestamp:\n description: CompletionTimestamp records the time a restore was completed.\n Completion time is recorded even on failed restores. The server's\n time is used for CompletionTimestamps\n format: date-time\n nullable: true\n type: string\n message:\n description: Message is a message about the pod volume restore's status.\n type: string\n phase:\n description: Phase is the current state of the PodVolumeRestore.\n enum:\n - New\n - InProgress\n - Completed\n - Failed\n type: string\n progress:\n description: Progress holds the total number of bytes of the snapshot\n and the current number of restored bytes. This can be used to display\n progress information about the restore operation.\n properties:\n bytesDone:\n format: int64\n type: integer\n totalBytes:\n format: int64\n type: integer\n type: object\n startTimestamp:\n description: StartTimestamp records the time a restore was started.\n The server's time is used for StartTimestamps\n format: date-time\n nullable: true\n type: string\n type: object\n type: object\n version: v1\n versions:\n - name: v1\n served: true\n storage: true" }, { "path": "design/CLI/PoC/base/resticrepository.yaml", "content": "---\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: (unknown)\n creationTimestamp: null\n labels:\n component: velero\n name: resticrepositories.velero.io\nspec:\n group: velero.io\n names:\n kind: ResticRepository\n listKind: ResticRepositoryList\n plural: resticrepositories\n singular: resticrepository\n scope: \"\"\n validation:\n openAPIV3Schema:\n properties:\n apiVersion:\n description: 'APIVersion defines the versioned schema of this representation\n of an object. Servers should convert recognized schemas to the latest\n internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources'\n type: string\n kind:\n description: 'Kind is a string value representing the REST resource this\n object represents. Servers may infer this from the endpoint the client\n submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'\n type: string\n metadata:\n type: object\n spec:\n description: ResticRepositorySpec is the specification for a ResticRepository.\n properties:\n backupStorageLocation:\n description: BackupStorageLocation is the name of the BackupStorageLocation\n that should contain this repository.\n type: string\n maintenanceFrequency:\n description: MaintenanceFrequency is how often maintenance should\n be run.\n type: string\n resticIdentifier:\n description: ResticIdentifier is the full restic-compatible string\n for identifying this repository.\n type: string\n volumeNamespace:\n description: VolumeNamespace is the namespace this restic repository\n contains pod volume backups for.\n type: string\n required:\n - backupStorageLocation\n - maintenanceFrequency\n - resticIdentifier\n - volumeNamespace\n type: object\n status:\n description: ResticRepositoryStatus is the current status of a ResticRepository.\n properties:\n lastMaintenanceTime:\n description: LastMaintenanceTime is the last time maintenance was\n run.\n format: date-time\n nullable: true\n type: string\n message:\n description: Message is a message about the current status of the\n ResticRepository.\n type: string\n phase:\n description: Phase is the current state of the ResticRepository.\n enum:\n - New\n - Ready\n - NotReady\n type: string\n type: object\n type: object\n version: v1\n versions:\n - name: v1\n served: true\n storage: true" }, { "path": "design/CLI/PoC/base/volumesnapshotlocations.yaml", "content": "---\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: (unknown)\n labels:\n component: velero\n name: volumesnapshotlocations.velero.io\nspec:\n group: velero.io\n names:\n kind: VolumeSnapshotLocation\n listKind: VolumeSnapshotLocationList\n plural: volumesnapshotlocations\n singular: volumesnapshotlocation\n scope: \"\"\n validation:\n openAPIV3Schema:\n description: VolumeSnapshotLocation is a location where Velero stores volume\n snapshots.\n properties:\n apiVersion:\n description: 'APIVersion defines the versioned schema of this representation\n of an object. Servers should convert recognized schemas to the latest\n internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources'\n type: string\n kind:\n description: 'Kind is a string value representing the REST resource this\n object represents. Servers may infer this from the endpoint the client\n submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'\n type: string\n metadata:\n type: object\n spec:\n description: VolumeSnapshotLocationSpec defines the specification for\n a Velero VolumeSnapshotLocation.\n properties:\n config:\n additionalProperties:\n type: string\n description: Config is for provider-specific configuration fields.\n type: object\n provider:\n description: Provider is the provider of the volume storage.\n type: string\n required:\n - provider\n type: object\n status:\n description: VolumeSnapshotLocationStatus describes the current status\n of a Velero VolumeSnapshotLocation.\n properties:\n phase:\n description: VolumeSnapshotLocationPhase is the lifecycle phase of\n a Velero VolumeSnapshotLocation.\n enum:\n - Available\n - Unavailable\n type: string\n type: object\n type: object\n version: v1\n versions:\n - name: v1\n served: true\n storage: true\n---\napiVersion: velero.io/v1\nkind: VolumeSnapshotLocation\nmetadata:\n creationTimestamp: null\n labels:\n component: velero\n name: default\n namespace: velero\nspec:\n config:\n region: us-east-2\n provider: aws" }, { "path": "design/CLI/PoC/overlays/plugins/aws-plugin.yaml", "content": "---\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n name: velero\nspec:\n selector:\n matchLabels:\n deploy: velero\n template:\n metadata:\n labels:\n component: velero\n deploy: velero\n spec:\n containers:\n - args:\n - server\n name: velero\n env:\n - name: AWS_SHARED_CREDENTIALS_FILE\n value: /credentials/cloud\n volumeMounts:\n - mountPath: /plugins\n name: plugins\n - mountPath: /credentials\n name: cloud-credential-aws\n initContainers:\n - image: velero/velero-plugin-for-aws:v1.0.1\n imagePullPolicy: Always\n name: velero-plugin-for-aws\n volumeMounts:\n - mountPath: /target\n name: plugins\n volumes:\n - emptyDir: {}\n name: plugins\n - name: cloud-credential-aws\n secret:\n secretName: cloud-credential-aws\n" }, { "path": "design/CLI/PoC/overlays/plugins/azure-plugin.yaml", "content": "---\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n name: velero\nspec:\n selector:\n matchLabels:\n deploy: velero\n template:\n metadata:\n labels:\n component: velero\n deploy: velero\n spec:\n containers:\n - args:\n - server\n name: velero\n env:\n - name: AZURE_SHARED_CREDENTIALS_FILE\n value: /credentials/cloud\n volumeMounts:\n - mountPath: /plugins\n name: plugins\n - mountPath: /credentials\n name: cloud-credential-azure\n initContainers:\n - image: velero/velero-plugin-for-microsoft-azure:v1.0.1\n imagePullPolicy: Always\n name: velero-plugin-for-microsoft-azure\n volumeMounts:\n - mountPath: /target\n name: plugins\n volumes:\n - emptyDir: {}\n name: plugins\n - name: cloud-credential-azure\n secret:\n secretName: cloud-credential-azure\n" }, { "path": "design/CLI/PoC/overlays/plugins/cloud", "content": "[default]\naws_access_key_id = minio\naws_secret_access_key = minio123" }, { "path": "design/CLI/PoC/overlays/plugins/kustomization.yaml", "content": "apiVersion: kustomize.config.k8s.io/v1beta1\nkind: Kustomization\n\nbases:\n - ../../base\n\npatchesStrategicMerge:\n - aws-plugin.yaml # this patches the Velero deployment\n # - azure-plugin.yaml # this patches the Velero deployment\n\ngeneratorOptions:\n disableNameSuffixHash: true\n labels: \n component: velero\n\nsecretGenerator:\n- name: cloud-credentials\n files:\n - \"cloud\"\n\n\n\n\n\n" }, { "path": "design/CLI/PoC/overlays/plugins/node-agent.yaml", "content": "---\napiVersion: apps/v1\nkind: DaemonSet\nmetadata:\n creationTimestamp: null\n labels:\n component: velero\n name: node-agent\n namespace: velero\nspec:\n selector:\n matchLabels:\n name: node-agent\n template:\n metadata:\n creationTimestamp: null\n labels:\n component: velero\n name: node-agent\n spec:\n containers:\n - args:\n - node-agent\n - server\n command:\n - /velero\n env:\n - name: NODE_NAME\n valueFrom:\n fieldRef:\n fieldPath: spec.nodeName\n - name: VELERO_NAMESPACE\n valueFrom:\n fieldRef:\n fieldPath: metadata.namespace\n - name: VELERO_SCRATCH_DIR\n value: /scratch\n - name: GOOGLE_APPLICATION_CREDENTIALS\n value: /credentials/cloud\n - name: AWS_SHARED_CREDENTIALS_FILE\n value: /credentials/cloud\n - name: AZURE_CREDENTIALS_FILE\n value: /credentials/cloud\n image: velero/velero:latest\n imagePullPolicy: Always\n name: node-agent\n resources: {}\n volumeMounts:\n - mountPath: /host_pods\n mountPropagation: HostToContainer\n name: host-pods\n - mountPath: /var/lib/kubelet/plugins\n mountPropagation: HostToContainer\n name: host-plugins\n - mountPath: /scratch\n name: scratch\n - mountPath: /credentials\n name: cloud-credentials\n securityContext:\n runAsUser: 0\n serviceAccountName: velero\n volumes:\n - hostPath:\n path: /var/lib/kubelet/pods\n name: host-pods\n - hostPath:\n path: /var/lib/kubelet/plugins\n name: host-plugins\n - emptyDir: {}\n name: scratch\n - name: cloud-credentials\n secret:\n secretName: cloud-credentials\n updateStrategy: {}\n" }, { "path": "design/Implemented/Extend-VolumePolicies-to-support-more-actions.md", "content": "# Extend VolumePolicies to support more actions\n\n## Abstract\n\nCurrently, the [VolumePolicies feature](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/handle-backup-of-volumes-by-resources-filters.md) which can be used to filter/handle volumes during backup only supports the skip action on matching conditions. Users need more actions to be supported.\n\n## Background\n\nThe `VolumePolicies` feature was introduced in Velero 1.11 as a flexible way to handle volumes. The main agenda of\nintroducing the VolumePolicies feature was to improve the overall user experience when performing backup operations\nfor volume resources, the feature enables users to group volumes according the `conditions` (criteria) specified and\nalso lets you specify the `action` that velero needs to take for these grouped volumes during the backup operation.\nThe limitation being that currently `VolumePolicies` only supports `skip` as an action, We want to extend the `action`\nfunctionality to support more usable options like `fs-backup` (File system backup) and `snapshot` (VolumeSnapshots).\n\n## Goals\n- Extending the VolumePolicies to support more actions like `fs-backup` (File system backup) and `snapshot` (VolumeSnapshots).\n- Improve user experience when backing up Volumes via Velero\n\n## Non-Goals\n- No changes to existing approaches to opt-in/opt-out annotations for volumes\n- No changes to existing `VolumePolicies` functionalities\n- No additions or implementations to support more granular actions like `snapshot-csi` and `snapshot-datamover`. These actions can be implemented as a future enhancement\n\n\n## Use-cases/Scenarios\n\n**Use-case 1:**\n- A user wants to use `snapshot` (volumesnapshots) backup option for all the csi supported volumes and `fs-backup` for the rest of the volumes.\n- Currently, velero supports this use-case but the user experience is not that great.\n- The user will have to individually annotate the volume mounting pod with the annotation \"backup.velero.io/backup-volumes\" for `fs-backup`\n- This becomes cumbersome at scale.\n- Using `VolumePolicies`, the user can just specify 2 simple `VolumePolicies` like for csi supported volumes as `snapshot` action and rest can be backed up`fs-backup` action:\n```yaml\nversion: v1\nvolumePolicies:\n - conditions:\n storageClass:\n - gp2\n action:\n type: snapshot\n - conditions: {}\n action:\n type: fs-backup\n```\n\n**Use-case 2:**\n- A user wants to use `fs-backup` for nfs volumes pertaining to a particular server\n- In such a scenario the user can just specify a `VolumePolicy` like:\n```yaml\nversion: v1\nvolumePolicies:\n- conditions:\n nfs:\n server: 192.168.200.90\n action:\n type: fs-backup\n```\n## High-Level Design\n- When the VolumePolicy action is set as `fs-backup` the backup workflow modifications would be:\n - We call [backupItem() -> backupItemInternal()](https://github.com/vmware-tanzu/velero/blob/main/pkg/backup/item_backupper.go#L95) on all the items that are to be backed up\n - Here when we encounter [Pod as an item ](https://github.com/vmware-tanzu/velero/blob/main/pkg/backup/item_backupper.go#L195)\n - We will have to modify the backup workflow to account for the `fs-backup` VolumePolicy action\n\n\n- When the VolumePolicy action is set as `snapshot` the backup workflow modifications would be:\n - Once again, We call [backupItem() -> backupItemInternal()](https://github.com/vmware-tanzu/velero/blob/main/pkg/backup/item_backupper.go#L95) on all the items that are to be backed up\n - Here when we encounter [Persistent Volume as an item](https://github.com/vmware-tanzu/velero/blob/d4128542590470b204a642ee43311921c11db880/pkg/backup/item_backupper.go#L253)\n - And we call the [takePVSnapshot func](https://github.com/vmware-tanzu/velero/blob/d4128542590470b204a642ee43311921c11db880/pkg/backup/item_backupper.go#L508)\n - We need to modify the takePVSnapshot function to account for the `snapshot` VolumePolicy action.\n - In case of csi snapshots for PVC objects, these snapshot actions are taken by the velero-plugin-for-csi, we need to modify the [executeActions()](https://github.com/vmware-tanzu/velero/blob/512fe0dabdcb3bbf1ca68a9089056ae549663bcf/pkg/backup/item_backupper.go#L232) function to account for the `snapshot` VolumePolicy action.\n\n**Note:** `Snapshot` action can either be a native snapshot or a csi snapshot, as is the case with the current flow where velero itself makes the decision based on the backup CR.\n\n## Detailed Design\n- Update VolumePolicy action type validation to account for `fs-backup` and `snapshot` as valid VolumePolicy actions.\n- Modifications needed for `fs-backup` action:\n - Now based on the specification of volume policy on backup request we will decide whether to go via legacy pod annotations approach or the newer volume policy based fs-backup action approach.\n - If there is a presence of volume policy(fs-backup/snapshot) on the backup request that matches as an action for a volume we use the newer volume policy approach to get the list of the volumes for `fs-backup` action\n - Else continue with the annotation based legacy approach workflow.\n\n- Modifications needed for `snapshot` action:\n - In the [takePVSnapshot function](https://github.com/vmware-tanzu/velero/blob/d4128542590470b204a642ee43311921c11db880/pkg/backup/item_backupper.go#L508) we will check the PV fits the volume policy criteria and see if the associated action is `snapshot`\n - If it is not snapshot then we skip the further workflow and avoid taking the snapshot of the PV\n - Similarly, For csi snapshot of PVC object, we need to do similar changes in [executeAction() function](https://github.com/vmware-tanzu/velero/blob/512fe0dabdcb3bbf1ca68a9089056ae549663bcf/pkg/backup/item_backupper.go#L348). we will check the PVC fits the volume policy criteria and see if the associated action is `snapshot` via csi plugin\n - If it is not snapshot then we skip the csi BIA execute action and avoid taking the snapshot of the PVC by not invoking the csi plugin action for the PVC\n\n**Note:**\n- When we are using the `VolumePolicy` approach for backing up the volumes then the volume policy criteria and action need to be specific and explicit, there is no default behavior, if a volume matches `fs-backup` action then `fs-backup` method will be used for that volume and similarly if the volume matches the criteria for `snapshot` action then the snapshot workflow will be used for the volume backup.\n- Another thing to note is the workflow proposed in this design uses the legacy `opt-in/opt-out` approach as a fallback option. For instance, the user specifies a VolumePolicy but for a particular volume included in the backup there are no actions(fs-backup/snapshot) matching in the volume policy for that volume, in such a scenario the legacy approach will be used for backing up the particular volume.\n- The relation between the `VolumePolicy` and the backup's legacy parameter `SnapshotVolumes`: \n - The `VolumePolicy`'s `snapshot` action matching for volume has higher priority. When there is a `snapshot` action matching for the selected volume, it will be backed by the snapshot way, no matter of the `backup.Spec.SnapshotVolumes` setting.\n - If there is no `snapshot` action matching the selected volume in the `VolumePolicy`, then the volume will be backed up by `snapshot` way, if the `backup.Spec.SnapshotVolumes` is not set to false.\n- The relation between the `VolumePolicy` and the backup's legacy filesystem `opt-in/opt-out` approach:\n - The `VolumePolicy`'s `fs-backup` action matching for volume has higher priority. When there is a `fs-backup` action matching for the selected volume, it will be backed by the fs-backup way, no matter of the `backup.Spec.DefaultVolumesToFsBackup` setting and the pod's `opt-in/opt-out` annotation setting.\n - If there is no `fs-backup` action matching the selected volume in the `VolumePolicy`, then the volume will be backed up by the legacy `opt-in/opt-out` way.\n\n## Implementation\n\n- The implementation should be included in velero 1.14\n\n- We will introduce a `VolumeHelper` interface. It will consist of two methods:\n```go\ntype VolumeHelper interface {\n\tShouldPerformSnapshot(obj runtime.Unstructured, groupResource schema.GroupResource) (bool, error)\n\tShouldPerformFSBackup(volume corev1api.Volume, pod corev1api.Pod) (bool, error)\n}\n```\n- The `VolumeHelperImpl` struct will implement the `VolumeHelper` interface and will consist of the functions that we will use through the backup workflow to accommodate volume policies for PVs and PVCs.\n```go\ntype volumeHelperImpl struct {\n\tvolumePolicy *resourcepolicies.Policies\n\tsnapshotVolumes *bool\n\tlogger logrus.FieldLogger\n\tclient crclient.Client\n\tdefaultVolumesToFSBackup bool\n\tbackupExcludePVC bool\n}\n\n```\n\n- We will create an instance of the structure `volumeHelperImpl` in `item_backupper.go`\n```go\n\titemBackupper := &itemBackupper{\n\t\t...\n\t\tvolumeHelperImpl: volumehelper.NewVolumeHelperImpl(\n\t\t\tresourcePolicy,\n\t\t\tbackupRequest.Spec.SnapshotVolumes,\n\t\t\tlog,\n\t\t\tkb.kbClient,\n\t\t\tboolptr.IsSetToTrue(backupRequest.Spec.DefaultVolumesToFsBackup),\n\t\t\t!backupRequest.ResourceIncludesExcludes.ShouldInclude(kuberesource.PersistentVolumeClaims.String()),\n\t\t),\n\t}\n```\n\n\n#### FS-Backup\n- Regarding `fs-backup` action to decide whether to use legacy annotation based approach or volume policy based approach:\n - We will use the `vh.ShouldPerformFSBackup()` function from the `volumehelper` package\n - Functions involved in processing `fs-backup` volume policy action will somewhat look like:\n\n```go\nfunc (v volumeHelperImpl) ShouldPerformFSBackup(volume corev1api.Volume, pod corev1api.Pod) (bool, error) {\n\tif !v.shouldIncludeVolumeInBackup(volume) {\n\t\tv.logger.Debugf(\"skip fs-backup action for pod %s's volume %s, due to not pass volume check.\", pod.Namespace+\"/\"+pod.Name, volume.Name)\n\t\treturn false, nil\n\t}\n\n\tif v.volumePolicy != nil {\n\t\tpvc, err := kubeutil.GetPVCForPodVolume(&volume, &pod, v.client)\n\t\tif err != nil {\n\t\t\tv.logger.WithError(err).Errorf(\"fail to get PVC for pod %s\", pod.Namespace+\"/\"+pod.Name)\n\t\t\treturn false, err\n\t\t}\n\t\tpv, err := kubeutil.GetPVForPVC(pvc, v.client)\n\t\tif err != nil {\n\t\t\tv.logger.WithError(err).Errorf(\"fail to get PV for PVC %s\", pvc.Namespace+\"/\"+pvc.Name)\n\t\t\treturn false, err\n\t\t}\n\n\t\taction, err := v.volumePolicy.GetMatchAction(pv)\n\t\tif err != nil {\n\t\t\tv.logger.WithError(err).Errorf(\"fail to get VolumePolicy match action for PV %s\", pv.Name)\n\t\t\treturn false, err\n\t\t}\n\n\t\tif action != nil {\n\t\t\tif action.Type == resourcepolicies.FSBackup {\n\t\t\t\tv.logger.Infof(\"Perform fs-backup action for volume %s of pod %s due to volume policy match\",\n\t\t\t\t\tvolume.Name, pod.Namespace+\"/\"+pod.Name)\n\t\t\t\treturn true, nil\n\t\t\t} else {\n\t\t\t\tv.logger.Infof(\"Skip fs-backup action for volume %s for pod %s because the action type is %s\",\n\t\t\t\t\tvolume.Name, pod.Namespace+\"/\"+pod.Name, action.Type)\n\t\t\t\treturn false, nil\n\t\t\t}\n\t\t}\n\t}\n\n\tif v.shouldPerformFSBackupLegacy(volume, pod) {\n\t\tv.logger.Infof(\"Perform fs-backup action for volume %s of pod %s due to opt-in/out way\",\n\t\t\tvolume.Name, pod.Namespace+\"/\"+pod.Name)\n\t\treturn true, nil\n\t} else {\n\t\tv.logger.Infof(\"Skip fs-backup action for volume %s of pod %s due to opt-in/out way\",\n\t\t\tvolume.Name, pod.Namespace+\"/\"+pod.Name)\n\t\treturn false, nil\n\t}\n}\n```\n\n- The main function from the above will be called when we encounter Pods during the backup workflow:\n```go\n\t\t\tfor _, volume := range pod.Spec.Volumes {\n\t\t\t\tshouldDoFSBackup, err := ib.volumeHelperImpl.ShouldPerformFSBackup(volume, *pod)\n\t\t\t\tif err != nil {\n\t\t\t\t\tbackupErrs = append(backupErrs, errors.WithStack(err))\n\t\t\t\t}\n\t\t\t\t...\n\t\t\t}\n```\n\n#### Snapshot (PV)\n\n- Making sure that `snapshot` action is skipped for PVs that do not fit the volume policy criteria, for this we will use the `vh.ShouldPerformSnapshot` from the `VolumeHelperImpl(vh)` receiver.\n```go\nfunc (v *volumeHelperImpl) ShouldPerformSnapshot(obj runtime.Unstructured, groupResource schema.GroupResource) (bool, error) {\n\t// check if volume policy exists and also check if the object(pv/pvc) fits a volume policy criteria and see if the associated action is snapshot\n\t// if it is not snapshot then skip the code path for snapshotting the PV/PVC\n\tpvc := new(corev1api.PersistentVolumeClaim)\n\tpv := new(corev1api.PersistentVolume)\n\tvar err error\n\n\tif groupResource == kuberesource.PersistentVolumeClaims {\n\t\tif err = runtime.DefaultUnstructuredConverter.FromUnstructured(obj.UnstructuredContent(), &pvc); err != nil {\n\t\t\treturn false, err\n\t\t}\n\n\t\tpv, err = kubeutil.GetPVForPVC(pvc, v.client)\n\t\tif err != nil {\n\t\t\treturn false, err\n\t\t}\n\t}\n\n\tif groupResource == kuberesource.PersistentVolumes {\n\t\tif err = runtime.DefaultUnstructuredConverter.FromUnstructured(obj.UnstructuredContent(), &pv); err != nil {\n\t\t\treturn false, err\n\t\t}\n\t}\n\n\tif v.volumePolicy != nil {\n\t\taction, err := v.volumePolicy.GetMatchAction(pv)\n\t\tif err != nil {\n\t\t\treturn false, err\n\t\t}\n\n\t\t// If there is a match action, and the action type is snapshot, return true,\n\t\t// or the action type is not snapshot, then return false.\n\t\t// If there is no match action, go on to the next check.\n\t\tif action != nil {\n\t\t\tif action.Type == resourcepolicies.Snapshot {\n\t\t\t\tv.logger.Infof(fmt.Sprintf(\"performing snapshot action for pv %s\", pv.Name))\n\t\t\t\treturn true, nil\n\t\t\t} else {\n\t\t\t\tv.logger.Infof(\"Skip snapshot action for pv %s as the action type is %s\", pv.Name, action.Type)\n\t\t\t\treturn false, nil\n\t\t\t}\n\t\t}\n\t}\n\n\t\t// If this PV is claimed, see if we've already taken a (pod volume backup)\n\t// snapshot of the contents of this PV. If so, don't take a snapshot.\n\tif pv.Spec.ClaimRef != nil {\n\t\tpods, err := podvolumeutil.GetPodsUsingPVC(\n\t\t\tpv.Spec.ClaimRef.Namespace,\n\t\t\tpv.Spec.ClaimRef.Name,\n\t\t\tv.client,\n\t\t)\n\t\tif err != nil {\n\t\t\tv.logger.WithError(err).Errorf(\"fail to get pod for PV %s\", pv.Name)\n\t\t\treturn false, err\n\t\t}\n\n\t\tfor _, pod := range pods {\n\t\t\tfor _, vol := range pod.Spec.Volumes {\n\t\t\t\tif vol.PersistentVolumeClaim != nil &&\n\t\t\t\t\tvol.PersistentVolumeClaim.ClaimName == pv.Spec.ClaimRef.Name &&\n\t\t\t\t\tv.shouldPerformFSBackupLegacy(vol, pod) {\n\t\t\t\t\tv.logger.Infof(\"Skipping snapshot of pv %s because it is backed up with PodVolumeBackup.\", pv.Name)\n\t\t\t\t\treturn false, nil\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\tif !boolptr.IsSetToFalse(v.snapshotVolumes) {\n\t\t// If the backup.Spec.SnapshotVolumes is not set, or set to true, then should take the snapshot.\n\t\tv.logger.Infof(\"performing snapshot action for pv %s as the snapshotVolumes is not set to false\", pv.Name)\n\t\treturn true, nil\n\t}\n\n\tv.logger.Infof(fmt.Sprintf(\"skipping snapshot action for pv %s possibly due to no volume policy setting or snapshotVolumes is false\", pv.Name))\n\treturn false, nil\n}\n```\n\n- The function `ShouldPerformSnapshot` will be used as follows in `takePVSnapshot` function of the backup workflow:\n```go\n\tsnapshotVolume, err := ib.volumeHelperImpl.ShouldPerformSnapshot(obj, kuberesource.PersistentVolumes)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif !snapshotVolume {\n\t\tlog.Info(fmt.Sprintf(\"skipping volume snapshot for PV %s as it does not fit the volume policy criteria specified by the user for snapshot action\", pv.Name))\n\t\tib.trackSkippedPV(obj, kuberesource.PersistentVolumes, volumeSnapshotApproach, \"does not satisfy the criteria for volume policy based snapshot action\", log)\n\t\treturn nil\n\t}\n```\n\n#### Snapshot (PVC)\n\n- Making sure that `snapshot` action is skipped for PVCs that do not fit the volume policy criteria, for this we will again use the `vh.ShouldPerformSnapshot` from the `VolumeHelperImpl(vh)` receiver.\n- We will pass the `VolumeHelperImpl(vh)` instance in `executeActions` method so that it is available to use.\n```go\n\n```\n- The above function will be used as follows in the `executeActions` function of backup workflow.\n- Considering the vSphere plugin doesn't support the VolumePolicy yet, don't use the VolumePolicy for vSphere plugin by now.\n```go\n\t\tif groupResource == kuberesource.PersistentVolumeClaims {\n\t\t\tif actionName == csiBIAPluginName {\n\t\t\t\tsnapshotVolume, err := ib.volumeHelperImpl.ShouldPerformSnapshot(obj, kuberesource.PersistentVolumeClaims)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn nil, itemFiles, errors.WithStack(err)\n\t\t\t\t}\n\n\t\t\t\tif !snapshotVolume {\n\t\t\t\t\tlog.Info(fmt.Sprintf(\"skipping csi volume snapshot for PVC %s as it does not fit the volume policy criteria specified by the user for snapshot action\", namespace+\"/\"+name))\n\t\t\t\t\tib.trackSkippedPV(obj, kuberesource.PersistentVolumeClaims, volumeSnapshotApproach, \"does not satisfy the criteria for volume policy based snapshot action\", log)\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t}\n\t\t}\n```\n\n\n## Future Implementation\nIt makes sense to add more specific actions in the future, once we deprecate the legacy opt-in/opt-out approach to keep things simple. Another point of note is, csi related action can be\neasier to implement once we decide to merge the csi plugin into main velero code flow.\nIn the future, we envision the following actions that can be implemented:\n- `snapshot-native`: only use volume snapshotter (native cloud provider snapshots), do nothing if not present/not compatible\n- `snapshot-csi`: only use csi-plugin, don't use volume snapshotter(native cloud provider snapshots), don't use datamover even if snapshotMoveData is true\n- `snapshot-datamover`: only use csi with datamover, don't use volume snapshotter (native cloud provider snapshots), use datamover even if snapshotMoveData is false\n\n**Note:** The above actions are just suggestions for future scope, we may not use/implement them as is. We could definitely merge these suggested actions as `Snapshot` actions and use volume policy parameters and criteria to segregate them instead of making the user explicitly supply the action names to such granular levels.\n\n## Related to Design\n\n[Handle backup of volumes by resources filters](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/handle-backup-of-volumes-by-resources-filters.md)\n\n## Alternatives Considered\nSame as the earlier design as this is an extension of the original VolumePolicies design" }, { "path": "design/Implemented/apply-flag.md", "content": "# Apply flag for install command\n\n## Abstract\nAdd an `--apply` flag to the install command that enables applying existing resources rather than creating them. This can be useful as part of the upgrade process for existing installations.\n\n## Background\nThe current Velero install command creates resources but doesn't provide a direct way to apply updates to an existing installation.\nUsers attempting to run the install command on an existing installation receive \"already exists\" messages.\nUpgrade steps for existing installs typically involve a three (or more) step process to apply updated CRDs (using `--dry-run` and piping to `kubectl apply`) and then updating/setting images on the Velero deployment and node-agent.\n\n## Goals\n- Provide a simple flag to enable applying resources on an existing Velero installation.\n- Use server-side apply to update existing resources rather than attempting to create them.\n- Maintain consistency with the regular install flow.\n\n## Non Goals\n- Implement special logic for specific version-to-version upgrades (i.e. resource deletion, etc).\n- Add complex upgrade validation or pre/post-upgrade hooks.\n- Provide rollback capabilities.\n\n## High-Level Design\nThe `--apply` flag will be added to the Velero install command.\nWhen this flag is set, the installation process will use server-side apply to update existing resources instead of using create on new resources.\nThis flag can be used as _part_ of the upgrade process, but will not always fully handle an upgrade.\n\n## Detailed Design\nThe implementation adds a new boolean flag `--apply` to the install command.\nThis flag will be passed through to the underlying install functions where the resource creation logic resides.\n\nWhen the flag is set to true:\n- The `createOrApplyResource` function will use server-side apply with field manager \"velero-cli\" and `force=true` to update resources.\n- Resources will be applied in the same order as they would be created during installation.\n- Custom Resource Definitions will still be processed first, and the system will wait for them to be established before continuing.\n\nThe server-side apply approach with `force=true` ensures that resources are updated even if there are conflicts with the last applied state.\nThis provides a best-effort mechanism to apply resources that follows the same flow as installation but updates resources instead of creating them.\n\nNo special handling is added for specific versions or resource structures, making this a general-purpose mechanism for applying resources.\n\n## Alternatives Considered\n1. Creating a separate `upgrade` command that would duplicate much of the install command logic.\n - Rejected due to code duplication and maintenance overhead.\n\n2. Implementing version-specific upgrade logic to handle breaking changes between versions.\n - Rejected as overly complex and difficult to maintain across multiple version paths.\n - This could be considered again in the future, but is not in the scope of the current design.\n\n3. Adding automatic detection of existing resources and switching to apply mode.\n - Rejected as it could lead to unexpected behavior and confusion if users unintentionally apply changes to existing resources.\n\n## Security Considerations\nThe apply flag maintains the same security profile as the install command.\nNo additional permissions are required beyond what is needed for resource creation.\nThe use of `force=true` with server-side apply could potentially override manual changes made to resources, but this is a necessary trade-off to ensure apply is successful.\n\n## Compatibility\nThis enhancement is compatible with all existing Velero installations as it is a new opt-in flag.\nIt does not change any resource formats or API contracts.\nThe apply process is best-effort and does not guarantee compatibility between arbitrary versions of Velero.\nUsers should still consult release notes for any breaking changes that may require manual intervention.\nThis flag could be adopted by the helm chart, specifically for CRD updates, to simplify the CRD update job.\n\n## Implementation\nThe implementation involves:\n1. Adding support for `Apply` to the existing Kubernetes client code.\n1. Adding the `--apply` flag to the install command options.\n1. Changing `createResource` to `createOrApplyResource` and updating it to use server-side apply when the `apply` boolean is set.\n\nThe implementation is straightforward and follows existing code patterns.\nNo migration of state or special handling of specific resources is required.\n" }, { "path": "design/Implemented/backup-performance-improvements.md", "content": "# Velero Backup performance Improvements and VolumeGroupSnapshot enablement\n\nThere are two different goals here, linked by a single primary missing feature in the Velero backup workflow.\nThe first goal is to enhance backup performance by allowing the primary backup controller to run in multiple threads, enabling Velero to back up multiple items at the same time for a given backup.\nThe second goal is to enable Velero to eventually support VolumeGroupSnapshots.\nFor both of these goals, Velero needs a way to determine which items should be backed up together.\n\nThis design proposal will include two development phases:\n- Phase 1 will refactor the backup workflow to identify blocks of related items that should be backed up together, and then coordinate backup hooks among items in the block.\n- Phase 2 will add multiple worker threads for backing up item blocks, so instead of backing up each block as it identified, the velero backup workflow will instead add the block to a channel and one of the workers will pick it up.\n- Actual support for VolumeGroupSnapshots is out-of-scope here and will be handled in a future design proposal, but the item block refactor introduced in Phase 1 is a primary building block for this future proposal.\n\n## Background\nCurrently, during backup processing, the main Velero backup controller runs in a single thread, completely finishing the primary backup processing for one resource before moving on to the next one.\nWe can improve the overall backup performance by backing up multiple items for a backup at the same time, but before we can do this we must first identify resources that need to be backed up together.\nGenerally speaking, resources that need to be backed up together are resources with interdependencies -- pods with their PVCs, PVCs with their PVs, groups of pods that form a single application, CRs, pods, and other resources that belong to the same operator, etc.\nAs part of this initial refactoring, once these \"Item Blocks\" are identified, an additional change will be to move pod hook processing up to the ItemBlock level.\nIf there are multiple pods in the ItemBlock, pre-hooks for all pods will be run before backing up the items, followed by post-hooks for all pods.\nThis change to hook processing is another prerequisite for future VolumeGroupSnapshot support, since supporting this will require backing up the pods and volumes together for any volumes which belong to the same group.\nOnce we are backing up items by block, the next step will be to create multiple worker threads to process and back up ItemBlocks, so that we can back up multiple ItemBlocks at the same time.\n\nIn looking at the different kinds of large backups that Velero must deal with, two obvious scenarios come to mind:\n1. Backups with a relatively small number of large volumes\n2. Backups with a large number of relatively small volumes.\n\nIn case 1, the majority of the time spent on the backup is in the asynchronous phases -- CSI snapshot creation actions after the snaphandle exists, and DataUpload processing. In that case, parallel item processing will likely have a minimal impact on overall backup completion time.\n\nIn case 2, the majority of time spent on the backup will likely be during the synchronous actions. Especially as regards CSI snapshot creation, the waiting for the VSC snaphandle to exist will result in significant passage of time with thousands of volumes. This is the sort of use case which will benefit the most from parallel item processing.\n\n## Goals\n- Identify groups of related items to back up together (ItemBlocks).\n- Manage backup hooks at the ItemBlock level rather than per-item.\n- Using worker threads, back up ItemBlocks at the same time.\n\n## Non Goals\n- Support VolumeGroupSnapshots: this is a future feature, although certain prerequisites for this enhancement are included in this proposal.\n- Process multiple backups in parallel: this is a future feature, although certain prerequisites for this enhancement are included in this proposal.\n- Refactoring plugin infrastructure to avoid RPC calls for internal plugins.\n- Restore performance improvements: this is potentially a future feature\n\n## High-Level Design\n\n### ItemBlock concept\n\nThe updated design is based on a new struct/type called `ItemBlock`.\nEssentially, an `ItemBlock` is a group of items that must be backed up together in order to guarantee backup integrity.\nWhen we eventually split item backup across multiple worker threads, `ItemBlocks` will be kept together as the basic unit of backup.\nTo facilitate this, a new plugin type, `ItemBlockAction` will allow relationships between items to be identified by velero -- any resources that must be backed up with other resources will need IBA plugins defined for them.\nExamples of `ItemBlocks` include:\n1. A pod, its mounted PVCs, and the bound PVs for those PVCs.\n2. A VolumeGroup (related PVCs and PVs) along with any pods mounting these volumes.\n3. For a ReadWriteMany PVC, the PVC, its bound PV, and all pods mounting this PVC.\n\n### Phase 1: ItemBlock processing\n- A new plugin type, `ItemBlockAction`, will be created\n- `ItemBlockAction` will contain the API method `GetRelatedItems`, which will be needed for determining which items to group together into `ItemBlocks`.\n- When processing the list of items returned from the item collector, instead of simply calling `BackupItem` on each in turn, we will use the `GetRelatedItems` API call to determine other items to include with the current item in an ItemBlock. Repeat recursively on each item returned.\n- Don't include an item in more than one ItemBlock -- if the next item from the item collector is already in a block, skip it.\n- Once ItemBlock is determined, call new func `BackupItemBlock` instead of `BackupItem`.\n- New func `BackupItemBlock` will call pre hooks for any pods in the block, then back up the items in the block (`BackupItem` will no longer run hooks directly), then call post hooks for any pods in the block.\n- The finalize phase will not be affected by the ItemBlock design, since this is just updating resources after async operations are completed on the items and there is no need to run these updates in parallel.\n\n### Phase 2: Process ItemBlocks for a single backup in multiple threads\n- Concurrent `BackupItemBlock` operations will be executed by worker threads invoked by the backup controller, which will communicate with the backup controller operation via a shared channel.\n- The ItemBlock processing loop implemented in Phase 1 will be modified to send each newly-created ItemBlock to the shared channel rather than calling `BackupItemBlock` inline.\n- Users will be able to configure the number of workers available for concurrent `BackupItemBlock` operations.\n- Access to the BackedUpItems map must be synchronized\n\n## Detailed Design\n\n### Phase 1: ItemBlock processing\n\n#### New ItemBlockAction plugin type\n\nIn order for Velero to identify groups of items to back up together in an ItemBlock, we need a way to identify items which need to be backed up along with the current item. While the current `Execute` BackupItemAction method does return a list of additional items which are required by the current item, we need to know this *before* we start the item backup. To support this, we need a new plugin type, `ItemBlockAction` (IBA) with an API method, `GetRelatedItems` which Velero will call on each item as it processes. The expectation is that the registered IBA plugins will return the same items as returned as additional items by the BIA `Execute` method, with the exception that items which are not created until calling `Execute` should not be returned here, as they don't exist yet.\n\n#### Proto definition (compiled into golang by protoc)\n\nThe ItemBlockAction plugin type is defined as follows:\n```\nservice ItemBlockAction {\n rpc AppliesTo(ItemBlockActionAppliesToRequest) returns (ItemBlockActionAppliesToResponse);\n rpc GetRelatedItems(ItemBlockActionGetRelatedItemsRequest) returns (ItemBlockActionGetRelatedItemsResponse);\n}\n\nmessage ItemBlockActionAppliesToRequest {\n string plugin = 1;\n}\n\nmessage ItemBlockActionAppliesToResponse {\n ResourceSelector ResourceSelector = 1;\n}\n\nmessage ItemBlockActionGetRelatedItemsRequest {\n string plugin = 1;\n bytes item = 2;\n bytes backup = 3;\n}\n\nmessage ItemBlockActionGetRelatedItemsResponse {\n repeated generated.ResourceIdentifier relatedItems = 1;\n}\n```\n\nA new PluginKind, `ItemBlockAction`, will be created, and the backup process will be modified to use this plugin kind.\n\nFor any BIA plugins which return additional items from `Execute()` that need to be backed up at the same time or sequentially in the same worker thread as the current items should add a new IBA plugin to return these same items (minus any which won't exist before BIA `Execute()` is called).\nThis mainly applies to plugins that operate on pods which reference resources which must be backed up along with the pod and are potentially affected by pod hooks or for plugins which connect multiple pods whose volumes should be backed up at the same time.\n\n### Changes to processing item list from the Item Collector\n\n#### New structs BackupItemBlock, ItemBlock, and ItemBlockItem\n```go\npackage backup\n\ntype BackupItemBlock struct {\n itemblock.ItemBlock\n // This is a reference to the shared itemBackupper for the backup\n itemBackupper *itemBackupper\n}\n\npackage itemblock\n\ntype ItemBlock struct {\n Log logrus.FieldLogger\n Items []ItemBlockItem\n}\n\ntype ItemBlockItem struct {\n Gr schema.GroupResource\n Item *unstructured.Unstructured\n PreferredGVR schema.GroupVersionResource\n}\n```\n\n#### Current workflow\nIn the `BackupWithResolvers` func, the current Velero implementation iterates over the list of items for backup returned by the Item Collector. For each item, Velero loads the item from the file created by the Item Collector, we call `backupItem`, update the GR map if successful, remove the (temporary) file containing item metadata, and update progress for the backup.\n\n#### Modifications to the loop over ItemCollector results\nThe `kubernetesResource` struct used by the item collector will be modified to add an `orderedResource` bool which will be set true for all of the resources moved to the beginning for each GroupResource as a result of being ordered resources.\nIn addition, an `inItemBlock` bool is added to the struct which will be set to true later when processing the list when each item is added to an ItemBlock.\nWhile the item collector already puts ordered resources first for each GR, there is no indication in the list which of these initial items are from the ordered resources list and which are the remaining (unordered) items.\nVelero needs to know which resources are ordered because when we process them later, the ordered resources for each GroupResource must be processed sequentially in a single ItemBlock.\n\nThe current workflow within each iteration of the ItemCollector.items loop will replaced with the following:\n- (note that some of the below should be pulled out into a helper func to facilitate recursive call to it for items returned from `GetRelatedItems`.)\n- Before loop iteration, create a pointer to a `BackupItemBlock` which will represent the current ItemBlock being processed.\n- If `item` has `inItemBlock==true`, continue. This one has already been processed.\n- If current `itemBlock` is nil, create it.\n- Add `item` to `itemBlock`.\n- Load item from ItemCollector file. Close/remove file after loading (on error return or not, possibly with similar anonymous func to current impl)\n- If other versions of the same item exist (via EnableAPIGroupVersions), add these to the `itemBlock` as well (and load from ItemCollector file)\n- Get matching IBA plugins for item, call `GetRelatedItems` for each. For each item returned, get full item content from ItemCollector (if present in item list, pulling from file, removing file when done) or from cluster (if not present in item list), add item to the current block, add item to `itemsInBlock` map, and then recursively apply current step to each (i.e. call IBA method, add to block, etc.)\n- If current item and next item are both ordered items for the same GR, then continue to next item, adding to current `itemBlock`.\n- Once full ItemBlock list is generated, call `backupItemBlock(block ItemBlock)\n- Add `backupItemBlock` return values to `backedUpGroupResources` map\n\n\n#### New func `backupItemBlock`\n\nMethod signature for new func `backupItemBlock` is as follows:\n```go\nfunc (kb *kubernetesBackupper) backupItemBlock(block BackupItemBlock) []schema.GroupResource\n```\nThe return value is a slice of GRs for resources which were backed up. Velero tracks these to determine which CRDs need to be included in the backup. Note that we need to make sure we include in this not only those resources that were backed up directly, but also those backed up indirectly via additional items BIA execute returns.\n\nIn order to handle backup hooks, this func will first take the input item list (`block.items`) and get a list of included pods, filtered to include only those not yet backed up (using `block.itemBackupper.backupRequest.BackedUpItems`). Iterate over this list and execute pre hooks (pulled out of `itemBackupper.backupItemInternal`) for each item.\nNow iterate over the full list (`block.items`) and call `backupItem` for each. After the first, the later items should already have been backed up, but calling a second time is harmless, since the first thing Velero does is check the `BackedUpItems` map, exiting if item is already backed up). We still need this call in case there's a plugin which returns something in `GetAdditionalItems` but forgets to return it in the `Execute` additional items return value. If we don't do this, we could end up missing items.\n\nAfter backing up the items in the block, we now execute post hooks using the same filtered item list we used for pre hooks, again taking the logic from `itemBackupper.backupItemInternal`).\n\n#### `itemBackupper.backupItemInternal` cleanup\n\nAfter implementing backup hooks in `backupItemBlock`, hook processing should be removed from `itemBackupper.backupItemInternal`.\n\n### Phase 2: Process ItemBlocks for a single backup in multiple threads\n\n#### New input field for number of ItemBlock workers\n\nThe velero installer and server CLIs will get a new input field `itemBlockWorkerCount`, which will be passed along to the `backupReconciler`.\nThe `backupReconciler` struct will also have this new field added. \n\n#### Worker pool for item block processing\n\nA new type, `ItemBlockWorker` will be added which will manage a pool of worker goroutines which will process item blocks, a shared input channel for passing blocks to workers, and a WaitGroup to shut down cleanly when the reconciler exits.\n```go\ntype ItemBlockWorkerPool struct {\n itemBlockChannel chan ItemBlockInput\n wg *sync.WaitGroup\n logger logrus.FieldLogger\n}\n\ntype ItemBlockInput struct {\n itemBlock *BackupItemBlock\n returnChan chan ItemBlockReturn\n}\n\ntype ItemBlockReturn struct {\n itemBlock *BackupItemBlock\n resources []schema.GroupResource\n err error\n}\n\nfunc (*p ItemBlockWorkerPool) getInputChannel() chan ItemBlockInput\nfunc StartItemBlockWorkerPool(context context.Context, workers int, logger logrus.FieldLogger) ItemBlockWorkerPool\nfunc processItemBlockWorker(context context.Context, itemBlockChannel chan ItemBlockInput, logger logrus.FieldLogger, wg *sync.WaitGroup)\n```\n\nThe worker pool will be started by calling `StartItemBlockWorkerPool` in `NewBackupReconciler()`, passing in the worker count and reconciler context.\n`backupreconciler.prepareBackupRequest` will also add the input channel to the `backupRequest` so that it will be available during backup processing.\nThe func `StartItemBlockWorkerPool` will create the `ItemBlockWorkerPool` with a shared buffered input channel (fixed buffer size) and start `workers` gororoutines which will each call `processItemBlockWorker`.\nThe `processItemBlockWorker` func (run by the worker goroutines) will read from `itemBlockChannel`, call `BackupItemBlock` on the retrieved `ItemBlock`, and then send the return value to the retrieved `returnChan`, and then process the next block.\n\n#### Modify ItemBlock processing loop to send ItemBlocks to the worker pool rather than backing them up directly\n\nThe ItemBlock processing loop implemented in Phase 1 will be modified to send each newly-created ItemBlock to the shared channel rather than calling `BackupItemBlock` inline, using a WaitGroup to manage in-process items. A separate goroutine will be created to process returns for this backup. After completion of the ItemBlock processing loop, velero will use the WaitGroup to wait for all ItemBlock processing to complete before moving forward.\n\nA simplified example of what this response goroutine might look like:\n```go\n // omitting cancel handling, context, etc\n ret := make(chan ItemBlockReturn)\n wg := &sync.WaitGroup{}\n // Handle returns\n go func() {\n for {\n select {\n case response := <-ret: // process each BackupItemBlock response\n func() {\n defer wg.Done()\n responses = append(responses, response)\n }()\n case <-ctx.Done():\n return\n }\n }\n }()\n // Simplified illustration, looping over and assumed already-determined ItemBlock list\n for _, itemBlock := range itemBlocks {\n wg.Add(1)\n inputChan <- ItemBlockInput{itemBlock: itemBlock, returnChan: ret}\n }\n done := make(chan struct{})\n go func() {\n defer close(done)\n wg.Wait()\n }()\n // Wait for all the ItemBlocks to be processed\n select {\n case <-done:\n logger.Info(\"done processing ItemBlocks\")\n }\n // responses from BackupItemBlock calls are in responses\n```\n\nWhen processing the responses, the main thing is to set `backedUpGroupResources[item.groupResource]=true` for each GR returned, which will give the same result as the current implementation calling items one-by-one and setting that field as needed.\n\nThe ItemBlock processing loop described above will be split into two separate iterations. For the first iteration, velero will only process those items at the beginning of the loop identified as `orderedResources` -- when the groups generated from these resources are passed to the worker channel, velero will wait for the response before moving on to the next ItemBlock.\nThis is to ensure that the ordered resources are processed in the required order. Once the last ordered resource is processed, the remaining ItemBlocks will be processed and sent to the worker channel without waiting for a response, in order to allow these ItemBlocks to be processed in parallel.\nThe reason we must execute `ItemBlocks` with ordered resources first (and one at a time) is that this is a list of resources identified by the user as resources which must be backed up first, and in a particular order.\n\n#### Synchronize access to the BackedUpItems map\n\nVelero uses a map of BackedUpItems to track which items have already been backed up. This prevents velero from attempting to back up an item more than once, as well as guarding against creating infinite loops due to circular dependencies in the additional items returns. Since velero will now be accessing this map from the parallel goroutines, access to the map must be synchronized with mutexes.\n\n### Backup Finalize phase\n\nThe finalize phase will not be affected by the ItemBlock design, since this is just updating resources after async operations are completed on the items and there is no need to run these updates in parallel.\n\n## Alternatives considered\n\n### BackpuItemAction v3 API\n\nInstead of adding a new `ItemBlockAction` plugin type, we could add a `GetAdditionalItems` method to BackupItemAction.\nThis was rejected because the new plugin type provides a cleaner interface, and keeps the function of grouping related items separate from the function of modifying item content for the backup.\n\n### Per-backup worker pool\n\nThe current design makes use of a permanent worker pool, started at backup controller startup time. With this design, when we follow on with running multiple backups in parallel, the same set of workers will take ItemBlock inputs from more than one backup. Another approach that was initially considered was a temporary worker pool, created while processing a backup, and deleted upon backup completion. \n\n#### User-visible API differences between the two approaches\n\nThe main user-visible difference here is in the configuration API. For the permanent worker approach, the worker count represents the total worker count for all backups. The concurrent backup count represents the number of backups running at the same time. At any given time, though, the maximum number of worker threads backing up items concurrently is equal to the worker count. If worker count is 15 and the concurrent backup count is 3, then there will be, at most, 15 items being processed at the same time, split among up to three running backups.\n\nFor the per-backup worker approach, the worker count represents the worker count for each backup. The concurrent backup count, as before, represents the number of backups running at the same time. If worker count is 15 and the concurrent backup count is 3, then there will be, at most, 45 items being processed at the same time, up to 15 for each of up to three running backups.\n#### Comparison of the two approaches\n\n- Permanent worker pool advantages:\n - This is the more commonly-followed Kubernetes pattern. It's generally better to follow standard practices, unless there are genuine reasons for the use case to go in a different way.\n - It's easier for users to understand the maximum number of concurrent items processed, which will have performance impact and impact on the resource requirements for the Velero pod. Users will not have to multiply the config numbers in their heads when working out how many total workers are present.\n - It will give us more flexibility for future enhancements around concurrent backups. One possible use case: backup priority. Maybe a user wants scheduled backups to have a lower priority than user-generated backups, since a user is sitting there waiting for completion -- a shared worker pool could react to the priority by taking ItemBlocks for the higher priority backup first, which would allow a large lower-priority backup's items to be preempted by a higher-priority backup's items without needing to explicitly stop the main controller flow for that backup.\n- Per-backup worker pool advantages:\n - Lower memory consumption than permanent worker pool, but the total memory used by a worker blocked on input will be pretty low, so if we're talking only 10-20 workers, the impact will be minimal.\n\n## Compatibility\n\n### Example IBA implementation for BIA plugins which return additional items\n\nIncluded below is an example of what might be required for a BIA plugin which returns additional items.\nThe code is taken from the internal velero `pod_action.go` which identifies the items required for a given pod.\n\nIn this particular case, the only function of pod_action is to return additional items, so we can really just convert this plugin to an IBA plugin. If there were other actions, such as modifying the pod content on backup, then we would still need the pod action, and the related items vs. content manipulation functions would need to be separated.\n\n```go\n// PodAction implements ItemBlockAction.\ntype PodAction struct {\n\tlog logrus.FieldLogger\n}\n\n// NewPodAction creates a new ItemAction for pods.\nfunc NewPodAction(logger logrus.FieldLogger) *PodAction {\n\treturn &PodAction{log: logger}\n}\n\n// AppliesTo returns a ResourceSelector that applies only to pods.\nfunc (a *PodAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"pods\"},\n\t}, nil\n}\n\n// GetRelatedItems scans the pod's spec.volumes for persistentVolumeClaim volumes and returns a\n// ResourceIdentifier list containing references to all of the persistentVolumeClaim volumes used by\n// the pod. This ensures that when a pod is backed up, all referenced PVCs are backed up too.\nfunc (a *PodAction) GetRelatedItems(item runtime.Unstructured, backup *v1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, error) {\n\tpod := new(corev1api.Pod)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(item.UnstructuredContent(), pod); err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tvar relatedItems []velero.ResourceIdentifier\n\tif pod.Spec.PriorityClassName != \"\" {\n\t\ta.log.Infof(\"Adding priorityclass %s to relatedItems\", pod.Spec.PriorityClassName)\n\t\trelatedItems = append(relatedItems, velero.ResourceIdentifier{\n\t\t\tGroupResource: kuberesource.PriorityClasses,\n\t\t\tName: pod.Spec.PriorityClassName,\n\t\t})\n\t}\n\n\tif len(pod.Spec.Volumes) == 0 {\n\t\ta.log.Info(\"pod has no volumes\")\n\t\treturn relatedItems, nil\n\t}\n\n\tfor _, volume := range pod.Spec.Volumes {\n\t\tif volume.PersistentVolumeClaim != nil && volume.PersistentVolumeClaim.ClaimName != \"\" {\n\t\t\ta.log.Infof(\"Adding pvc %s to relatedItems\", volume.PersistentVolumeClaim.ClaimName)\n\n\t\t\trelatedItems = append(relatedItems, velero.ResourceIdentifier{\n\t\t\t\tGroupResource: kuberesource.PersistentVolumeClaims,\n\t\t\t\tNamespace: pod.Namespace,\n\t\t\t\tName: volume.PersistentVolumeClaim.ClaimName,\n\t\t\t})\n\t\t}\n\t}\n\n\treturn relatedItems, nil\n}\n\n// API call\nfunc (a *PodAction) Name() string {\n\treturn \"PodAction\"\n}\n\n```\n\n\n## Implementation\nPhase 1 and Phase 2 could be implemented within the same Velero release cycle, but they need not be.\nPhase 1 is expected to be implemented in Velero 1.15.\nPhase 2 is expected to be implemented in Velero 1.16.\n" }, { "path": "design/Implemented/backup-pvc-config.md", "content": "# Backup PVC Configuration Design\n\n## Glossary & Abbreviation\n\n**Velero Generic Data Path (VGDP)**: VGDP is the collective modules that is introduced in [Unified Repository design][1]. Velero uses these modules to finish data transfer for various purposes (i.e., PodVolume backup/restore, Volume Snapshot Data Movement). VGDP modules include uploaders and the backup repository. \n\n**Exposer**: Exposer is a module that is introduced in [Volume Snapshot Data Movement Design][2]. Velero uses this module to expose the volume snapshots to Velero node-agent pods or node-agent associated pods so as to complete the data movement from the snapshots. \n\n**backupPVC**: The intermediate PVC created by the exposer for VGDP to access data from, see [Volume Snapshot Data Movement Design][2] for more details. \n\n**backupPod**: The pod consumes the backupPVC so that VGDP could access data from the backupPVC, see [Volume Snapshot Data Movement Design][2] for more details. \n\n**sourcePVC**: The PVC to be backed up, see [Volume Snapshot Data Movement Design][2] for more details. \n\n## Background\n\nAs elaberated in [Volume Snapshot Data Movement Design][2], a backupPVC may be created by the Exposer and the VGDP reads data from the backupPVC. \nIn some scenarios, users may need to configure some advanced settings of the backupPVC so that the data movement could work in best performance in their environments. Specifically: \n- For some storage providers, when creating a read-only volume from a snapshot, it is very fast; whereas, if a writable volume is created from the snapshot, they need to clone the entire disk data, which is time consuming. If the backupPVC's `accessModes` is set as `ReadOnlyMany`, the volume driver is able to tell the storage to create a read-only volume, which may dramatically shorten the snapshot expose time. On the other hand, `ReadOnlyMany` is not supported by all volumes. Therefore, users should be allowed to configure the `accessModes` for the backupPVC. \n- Some storage providers create one or more replicas when creating a volume, the number of replicas is defined in the storage class. However, it doesn't make any sense to keep replicas when an intermediate volume used by the backup. Therefore, users should be allowed to configure another storage class specifically used by the backupPVC. \n\n## Goals\n\n- Create a mechanism for users to specify various configurations for backupPVC \n\n## Non-Goals\n\n## Solution\n\nWe will use the ConfigMap specified by `velero node-agent` CLI's parameter `--node-agent-configmap` to host the backupPVC configurations.\nThis configMap is not created by Velero, users should create it manually on demand. The configMap should be in the same namespace where Velero is installed. If multiple Velero instances are installed in different namespaces, there should be one configMap in each namespace which applies to node-agent in that namespace only. \nNode-agent server checks these configurations at startup time and use it to initiate the related Exposer modules. Therefore, users could edit this configMap any time, but in order to make the changes effective, node-agent server needs to be restarted. \nInside the ConfigMap we will add one new kind of configuration as the data in the configMap, the name is ```backupPVC```. \nUsers may want to set different backupPVC configurations for different volumes, therefore, we define the configurations as a map and allow users to specific configurations by storage class. Specifically, the key of the map element is the storage class name used by the sourcePVC and the value is the set of configurations for the backupPVC created for the sourcePVC. \n\nThe data structure is as below:\n```go\ntype Configs struct {\n\t// LoadConcurrency is the config for data path load concurrency per node.\n\tLoadConcurrency *LoadConcurrency `json:\"loadConcurrency,omitempty\"`\n\n\t// LoadAffinity is the config for data path load affinity.\n\tLoadAffinity []*LoadAffinity `json:\"loadAffinity,omitempty\"`\n\n\t// BackupPVC is the config for backupPVC of snapshot data movement.\n\tBackupPVC map[string]BackupPVC `json:\"backupPVC,omitempty\"`\n}\n\ntype BackupPVC struct {\n\t// StorageClass is the name of storage class to be used by the backupPVC.\n\tStorageClass string `json:\"storageClass,omitempty\"`\n\n\t// ReadOnly sets the backupPVC's access mode as read only.\n\tReadOnly bool `json:\"readOnly,omitempty\"`\n}\n``` \n\n### Sample\nA sample of the ConfigMap is as below:\n```json\n{\n \"backupPVC\": {\n \"storage-class-1\": {\n \"storageClass\": \"snapshot-storage-class\",\n \"readOnly\": true\n },\n \"storage-class-2\": {\n \"storageClass\": \"snapshot-storage-class\"\n },\n \"storage-class-3\": {\n \"readOnly\": true\n } \n }\n}\n```\n\nTo create the configMap, users need to save something like the above sample to a json file and then run below command:\n```\nkubectl create cm -n velero --from-file=\n``` \n\n### Implementation\nThe `backupPVC` is passed to the exposer and the exposer sets the related specification and create the backupPVC. \nIf `backupPVC.storageClass` doesn't exist or set as empty, the sourcePVC's storage class will be used. \nIf `backupPVC.readOnly` is set to true, `ReadOnlyMany` will be the only value set to the backupPVC's `accessModes`, otherwise, `ReadWriteOnce` is used. \n\nOnce `backupPVC.storageClass` is set, users must make sure that the specified storage class exists in the cluster and can be used the the backupPVC, otherwise, the corresponding DataUpload CR will stay in `Accepted` phase until the prepare timeout (by default 30min). \nOnce `backupPVC.readOnly` is set to true, users must make sure that the storage supports to create a `ReadOnlyMany` PVC from a snapshot, otherwise, the corresponding DataUpload CR will stay in `Accepted` phase until the prepare timeout (by default 30min). \n\nOnce above problems happen, the DataUpload CR is cancelled after prepare timeout and the backupPVC and backupPod will be deleted, so there is no way to tell the cause is one of the above problems or others. \nTo help the troubleshooting, we can add some diagnostic mechanism to discover the status of the backupPod before deleting it as a result of the prepare timeout. \n\n[1]: unified-repo-and-kopia-integration/unified-repo-and-kopia-integration.md\n[2]: volume-snapshot-data-movement/volume-snapshot-data-movement.md" }, { "path": "design/Implemented/backup-repo-cache-volume.md", "content": "# Backup Repository Cache Volume Design\n\n## Glossary & Abbreviation\n\n**Backup Storage**: The storage to store the backup data. Check [Unified Repository design][1] for details. \n**Backup Repository**: Backup repository is layered between BR data movers and Backup Storage to provide BR related features that is introduced in [Unified Repository design][1]. \n**Velero Generic Data Path (VGDP)**: VGDP is the collective of modules that is introduced in [Unified Repository design][1]. Velero uses these modules to finish data transfer for various purposes (i.e., PodVolume backup/restore, Volume Snapshot Data Movement). VGDP modules include uploaders and the backup repository. \n**Data Mover Pods**: Intermediate pods which hold VGDP and complete the data transfer. See [VGDP Micro Service for Volume Snapshot Data Movement][2] and [VGDP Micro Service For fs-backup][3] for details. \n**Repository Maintenance Pods**: Pods for [Repository Maintenance Jobs][4], which holds VGDP to run repository maintenance. \n\n## Background\n\nAccording to the [Unified Repository design][1] Velero uses selectable backup repositories for various backup/restore methods, i.e., fs-backup, volume snapshot data movement, etc. Some backup repositories may need to cache data on the client side for various repository operation, so as to accelerate the execution. \nIn the existing [Backup Repository Configuration][5], we allow users to configure the cache data size (`cacheLimitMB`). However, the cache data is still stored in the root file system of data mover pods/repository maintenance pods, so stored in the root file system of the node. This is not good enough, reasons: \n- In many distributions, the node's system disk size is predefined, non configurable and limit, e.g., the system disk size may be 20G or less\n- Velero supports concurrent data movements in each node. The cache in each of the concurrent data mover pods could quickly run out of the system disk and cause problems like pod eviction, failure of pod creation, degradation of Kubernetes QoS, etc. \n\nWe need to allow users to prepare a dedicated location, e.g., a dedictated volume, for the cache. \nNot all backup repositories or not all backup repository operations require cache, we need to define the details when and how the cache is used. \n\n## Goals\n\n- Create a mechanism for users to configure cache volumes for various pods running VGDP\n- Design the workflow to assign the cache volume pod path to backup repositories\n- Describe when and how the cache volume is used \n\n## Non-Goals\n\n- The solution is based on [Unified Repository design][1], [VGDP Micro Service for Volume Snapshot Data Movement][2] and [VGDP Micro Service For fs-backup][3], legacy data paths are not supported. E.g., when a pod volume restore (PVR) runs with legacy Restic path, if any data is cached, the cache still resides in the root file system. \n\n## Solution\n\n### Cache Data\n\nVarying on backup repositoires, cache data may include payload data or repository metadata, e.g., indexes to the payload data chunks. \n\nPayload data is highly related to the backup data, and normally take the majority of the repository data as well as the cache data.\n\nRepository metadata is related to the backup repository's chunking algorithm, data chunk mapping method, etc, and so the size is not proportional to the backup data size. \nOn the other hand for some backup repository, in extreme cases, the repository metadata may be significantly large. E.g., Kopia's indexes are per chunks, if there are huge number of small files in the repository, Kopia's index data may be in the same level of or even larger than the payload data. \nHowever, in the cases that repository metadata data become the majority, other bottlenecks may emerge and concurrency of data movers may be significantly constrained, so the requirement to cache volumes may go away. \n\nTherefore, for now we only consider the cache volume requirement for payload data, and leave the consideration for metadata as a future enhancement. \n\n### Scenarios\n\nBackup repository cache varies on backup repositories and backup repository operation during VGDP runs. Below are the scenarios when VGDP runs:\n- Data Upload for Backup: this is the process to upload/write the backup data into the backup repository, e.g., DataUpload or PodVolumeBackup. The pieces of data is almost directly written to the repository, sometimes with a small group staying shortly in the local place. That is to say, there should not be large scale data cached for this scenario, so we don't prepare dedicated cache for this scenario.\n- Repository Maintenance: Repository maintenance most often visits the backup repository's metadata and sometimes it needs to visit the file system directories from the backed up data. On the other hand, it is not practical to run concurrent maintenance jobs in one node. So the cache data is neither large nor affect the root file system too much. Therefore, we don't need to prepare dedicated cache for this scenario.\n- Data Download for Restore: this is the process to download/read the backup data from the backup repository during restore, e.g., DataDownload or PodVolumeRestore. For backup repositories for which data are stored in remote backup storages (e.g., Kopia repository stores data in remote object stores), large scale of data are cached locally to accerlerate the restore. Therefore, we need dedicate cache volumes for this scenario. \n- Backup Deletion: During this scenario, backup repository is connected, metadata is enumerated to find the repository snapshot representing the backup data. That is to say, only metadata is cached if any. Therefore, dedicated cache volumes are not required in this scenario.\n\nThe above analyses are based on the common behavior of backup repositories and they are not considering the case that backup repository metadata takes majority or siginficant proportion of the cache data. \nAs a conclusion of the analyses, we will create dedicated cache volumes for restore scenarios. \nFor other scenarios, we can add them regarded to the future changes/requirements. The mechanism to expose and connect the cache volumes should work for all scenarios. E.g., if we need to consider the backup repository metadata case, we may need cache volumes for backup and repository maintenance as well, then we can just reuse the same cache volume provision and connection mechanism to backup and repository maintenance scenarios. \n\n### Cache Data and Lifecycle\n\nIf available, one cache volume is dedicately assigned to one data mover pod. That is, the cached data is destroyed when the data mover pod completes. Then the backup repository instance also closes. \nCache data are fully managed by the specific backup repository. So the backup repository may also have its own way to GC the cache data. \nThat is to say, cache data GC may be launched by the backup repository instance during the running of the data mover pod; then the left data are automatically destroyed when the data mover pod and the cache PVC are destroyed (cache PVC's `reclaimPolicy` is always `Deleted`, so once the cache PVC is destroyed, the volume will also be destroyed). So no specially logics are needed for cache data GC. \n\n### Data Size\n\nCache volumes take storage space and cluster resources (PVC, PV), therefore, cache volumes should be created only when necessary and the volumes should be with reasonable size based on the cache data size: \n- It is not a good bargain to have cache volumes for small backups, small backups will use resident cache location (the cache location in the root file system)\n- The cache data size has a limit, the existing `cacheLimitMB` is used for this purpose. E.g., it could be set as 1024 for a 1TB backup, which means 1GB of data is cached and the old cache data exceeding this size will be cleared. Therefore, it is meaningless to set the cache volume size much larger than `cacheLimitMB`\n\n### Cache Volume Size\n\nThe cache volume size is calculated from below factors (for Restore scenarios): \n- **Limit**: The limit of the cache data, that is represented by `cacheLimitMB`, the default value is 5GB\n- **backupSize**: The size of the backup as a reference to evaluate whether to create a cache volume. It doesn't mean the backup data really decides the cache data all the time, it is just a reference to evaluate the scale of the backup, small scale backups may need small cache data. Sometimes, backupSize is not irrelevant to the size of cache data, in this case, ResidentThreshold should not be set, Limit will be used directly. It is unlikely that backupSize is unavailable, but once that happens, ResidentThreshold is ignored, Limit will be used directly. \n- **ResidentThreshold**: The minimum backup size that a cache volume is created\n- **InflationPercentage**: Considering the overhead of the file system and the possible delay of the cache cleanup, there should be an inflation for the final volume size vs. the logical size, otherwise, the cache volume may be overrun. This inflation percentage is hardcoded, e.g., 20%. \n\nA formula is as below: \n```\ncacheVolumeSize = ((backupSize != 0 ? (backupSize > residentThreshold ? limit : 0) : limit) * (100 + inflationPercentage)) / 100\n```\nFinally, the `cacheVolumeSize` will be rounded up to GiB considering the UX friendliness, storage friendliness and management friendliness. \n\n### PVC/PV\n\nThe PVC for a cache volume is created in Velero namespace and a storage class is required for the cache PVC. The PVC's accessMode is `ReadWriteOnce` and volumeMode is `FileSystem`, so the storage class provided should support this specification. Otherwise, if the storageclass doesn't support either of the specifications, the data mover pod may be hang in `Pending` state until a timeout setting with the data movement (e.g. `prepareTimeout`) and the data movement will finally fail. \nIt is not expected that the cache volume is retained after data mover pod is deleted, so the `reclaimPolicy` for the storageclass must be `Delete`. \n\nTo detect the problems in the storageclass and fail earlier, a validation is applied to the storageclass and once the validation fails, the cache configuration will be ignored, so the data mover pod will be created without a cache volume. \n\n### Cache Volume Configurations\n\nBelow configurations are introduced:\n- **residentThresholdMB**: the minimum data size(in MB) to be processed (if available) that a cache volume is created\n- **cacheStorageClass**: the name of the storage class to provision the cache PVC\n\nNot like `cacheLimitMB` which is set to and affect the backup repository, the above two configurations are actually data mover configurations of how to create cache volumes to data mover pods; and the two configurations don't need to be per backup repository. So we add them to the node-agent Configuration. \n\n### Sample\n\nBelow are some examples of the node-agent configMap with the configurations:\n\nSample-1: \n```json\n{\n \"cacheVolume\": {\n \"storageClass\": \"sc-1\",\n \"residentThresholdMB\": 1024 \n }\n}\n```\n\nSample-2: \n```json\n{\n \"cacheVolume\": {\n \"storageClass\": \"sc-1\", \n }\n}\n```\n\nSample-3: \n```json\n{\n \"cacheVolume\": {\n \"residentThresholdMB\": 1024 \n }\n}\n```\n\n**sample-1**: This is a valid configuration. Restores with backup data size larger than 1G will be assigned a cache volume using storage class `sc-1`. \n**sample-2**: This is a valid configuration. Data mover pods are always assigned a cache volume using storage class `sc-1`. \n**sample-3**: This is not a valid configuration because the storage class is absent. Velero gives up creating a cache volume. \n\nTo create the configMap, users need to save something like the above sample to a json file and then run below command:\n```\nkubectl create cm -n velero --from-file=\n```\n\nThe cache volume configurations will be visited by node-agent server, so they also need to specify the `--node-agent-configmap` to the `velero node-agent` parameters. \n\n## Detailed Design\n\n### Backup and Restore\n\nThe restore needs to know the backup size so as to calculate the cache volume size, some new fields are added to the DataDownload and PodVolumeRestore CRDs. \n\n`snapshotSize` field is also added to DataDownload and PodVolumeRestore's `spec`:\n```yaml\n spec:\n snapshotID:\n description: SnapshotID is the ID of the Velero backup snapshot to\n be restored from.\n type: string\n snapshotSize:\n description: SnapshotSize is the logical size of the snapshot.\n format: int64\n type: integer\n```\n\n`snapshotSize` represents the total size of the backup; during restore, the value is transferred from DataUpload/PodVolumeBackup's `Status.Progress.TotalBytes` to DataDownload/PodVolumeRestore. \n\nIt is unlikely that `Status.Progress.TotalBytes` from DataUpload/PodVolumeBackup is unavailable, but once it happens, according to the above formula, `residentThresholdMB` is ignored, cache volume size is calculated directly from cache limit for the corresponding backup repository. \n\n### Exposer\n\nCache volume configurations are retrieved by node-agent and passed through DataDownload/PodVolumeRestore to GenericRestore exposer/PodVolume exposer. \nThe exposers are responsible to calculate cache volume size, create cache PVCs and mount them to the restorePods. \nIf the calculated cache volume size is 0, or any of the critical parameters is missing (e.g., cache volume storage class), the exposers ignore the cache volume configuration and continue with creating restorePods without cache volumes, so no impact to the result of the restore. \n\nExposers mount the cache volume to a predefined directory and pass the directory to the data mover pods through the `cache-volume-path` parameter. \n\nBelow data structure is added to the exposers' expose parameters: \n\n```go\ntype GenericRestoreExposeParam struct {\n\t// RestoreSize specifies the data size for the volume to be restored\n\tRestoreSize int64\n\n\t// CacheVolume specifies the info for cache volumes\n\tCacheVolume *CacheVolumeInfo\n}\n\ntype PodVolumeExposeParam struct {\n\t// RestoreSize specifies the data size for the volume to be restored\n\tRestoreSize int64\n\n\t// CacheVolume specifies the info for cache volumes\n\tCacheVolume *repocache.CacheConfigs\n}\n\ntype CacheConfigs struct {\n\t// StorageClass specifies the storage class for cache volumes\n\tStorageClass string\n\n\t// Limit specifies the maximum size of the cache data\n\tLimit int64\n\n\t// ResidentThreshold specifies the minimum size of the cache data to create a cache volume\n\tResidentThreshold int64\n}\n```\n\n### Data Mover Pods\n\nData mover pods retrieve the cache volume directory from `cache-volume-path` parameter and pass it to Unified Repository. \nIf the directory is empty, Unified Repository uses the resident location for data cache, that is, the root file system. \n\n### Kopia Repository\n\nKopia repository supports cache directory configuration for both metadata and data. The existing `SetupConnectOptions` is modified to customize the `CacheDirectory`: \n\n```go\nfunc SetupConnectOptions(ctx context.Context, repoOptions udmrepo.RepoOptions) repo.ConnectOptions {\n ...\n\n\treturn repo.ConnectOptions{\n\t\tCachingOptions: content.CachingOptions{\n\t\t\tCacheDirectory: cacheDir,\n\t\t\t...\n\t\t},\n\t\t...\n\t}\n}\n``` \n\n\n[1]: Implemented/unified-repo-and-kopia-integration/unified-repo-and-kopia-integration.md\n[2]: Implemented/vgdp-micro-service/vgdp-micro-service.md\n[3]: Implemented/vgdp-micro-service-for-fs-backup/vgdp-micro-service-for-fs-backup.md\n[4]: Implemented/repo_maintenance_job_config.md\n[5]: Implemented/backup-repo-config.md" }, { "path": "design/Implemented/backup-repo-config.md", "content": "# Backup Repository Configuration Design\n\n## Glossary & Abbreviation\n\n**Backup Storage**: The storage to store the backup data. Check [Unified Repository design][1] for details. \n**Backup Repository**: Backup repository is layered between BR data movers and Backup Storage to provide BR related features that is introduced in [Unified Repository design][1]. \n\n## Background\n\nAccording to the [Unified Repository design][1] Velero uses selectable backup repositories for various backup/restore methods, i.e., fs-backup, volume snapshot data movement, etc. To achieve the best performance, backup repositories may need to be configured according to the running environments. \nFor example, if there are sufficient CPU and memory resources in the environment, users may enable compression feature provided by the backup repository, so as to achieve the best backup throughput. \nAs another example, if the local disk space is not sufficient, users may want to constraint the backup repository's cache size, so as to prevent the repository from running out of the disk space. \nTherefore, it is worthy to allow users to configure some essential parameters of the backup repsoitories, and the configuration may vary from backup repositories. \n\n## Goals\n\n- Create a mechanism for users to specify configurations for backup repositories \n\n## Non-Goals\n\n## Solution\n\n### BackupRepository CRD\n\nAfter a backup repository is initialized, a BackupRepository CR is created to represent the instance of the backup repository. The BackupRepository's spec is a core parameter used by Unified Repo modules when interactive with the backup repsoitory. Therefore, we can add the configurations into the BackupRepository CR called ```repositoryConfig```. \nThe configurations may be different varying from backup repositories, therefore, we will not define each of the configurations explicitly. Instead, we add a map in the BackupRepository's spec to take any configuration to be set to the backup repository. \n\nDuring various operations to the backup repository, the Unified Repo modules will retrieve from the map for the specific configuration that is required at that time. So even though it is specified, a configuration may not be visited/hornored if the operations don't require it for the specific backup repository, this won't bring any issue. When and how a configuration is hornored is decided by the configuration itself and should be clarified in the configuration's specification. \n\nBelow is the new BackupRepository's spec after adding the configuration map: \n```yaml\n spec:\n description: BackupRepositorySpec is the specification for a BackupRepository.\n properties:\n backupStorageLocation:\n description: |-\n BackupStorageLocation is the name of the BackupStorageLocation\n that should contain this repository.\n type: string\n maintenanceFrequency:\n description: MaintenanceFrequency is how often maintenance should\n be run.\n type: string\n repositoryConfig:\n additionalProperties:\n type: string\n description: RepositoryConfig contains configurations for the specific\n repository.\n type: object\n repositoryType:\n description: RepositoryType indicates the type of the backend repository\n enum:\n - kopia\n - restic\n - \"\"\n type: string\n resticIdentifier:\n description: |-\n ResticIdentifier is the full restic-compatible string for identifying\n this repository.\n type: string\n volumeNamespace:\n description: |-\n VolumeNamespace is the namespace this backup repository contains\n pod volume backups for.\n type: string\n required:\n - backupStorageLocation\n - maintenanceFrequency\n - resticIdentifier\n - volumeNamespace\n type: object\n``` \n\n### BackupRepository configMap\n\nThe BackupRepository CR is not created explicitly by a Velero CLI, but created as part of the backup/restore/maintenance operation if the CR doesn't exist. As a result, users don't have any way to specify the configurations before the BackupRepository CR is created. \nTherefore, a BackupRepository configMap is introduced as a template of the configurations to be applied to the backup repository CR. \nWhen the backup repository CR is created by the BackupRepository controller, the configurations in the configMap are copied to the ```repositoryConfig``` field. \nFor an existing BackupRepository CR, the configMap is never visited, if users want to modify the configuration value, they should directly edit the BackupRepository CR. \n\nThe BackupRepository configMap is created by users in velero installation namespace. The configMap name must be specified in the velero server parameter ```--backup-repository-configmap```, otherwise, it won't effect. \nIf the configMap name is specified but the configMap doesn't exist by the time of a backup repository is created, the configMap name is ignored. \nFor any reason, if the configMap doesn't effect, nothing is specified to the backup repository CR, so the Unified Repo modules use the hard-coded values to configure the backup repository. \n\nThe BackupRepository configMap supports backup repository type specific configurations, even though users can only specify one configMap. \nSo in the configMap struct, multiple entries are supported, indexed by the backup repository type. During the backup repository creation, the configMap is searched by the repository type. \n\n### Configurations\n\nWith the above mechanisms, any kind of configuration could be added. Here list the configurations defined at present: \n```cacheLimitMB```: specifies the size limit(in MB) for the local data cache. The more data is cached locally, the less data may be downloaded from the backup storage, so the better performance may be achieved. Practically, users can specify any size that is smaller than the free space so that the disk space won't run out. This parameter is for each repository connection, that is, users could change it before connecting to the repository. If a backup repository doesn't use local cache, this parameter will be ignored. For Kopia repository, this parameter is supported. \n```enableCompression```: specifies to enable/disable compression for a backup repsotiory. Most of the backup repositories support the data compression feature, if it is not supported by a backup repository, this parameter is ignored. Most of the backup repositories support to dynamically enable/disable compression, so this parameter is defined to be used whenever creating a write connection to the backup repository, if the dynamically changing is not supported, this parameter will be hornored only when initializing the backup repository. For Kopia repository, this parameter is supported and can be dynamically modified. \n\n### Sample\nBelow is an example of the BackupRepository configMap with the configurations: \n```yaml\napiVersion: v1\nkind: ConfigMap\nmetadata:\n name: \n namespace: velero\ndata:\n : |\n {\n \"cacheLimitMB\": 2048,\n \"enableCompression\": true \n }\n : |\n {\n \"cacheLimitMB\": 1,\n \"enableCompression\": false \n } \n```\n\nTo create the configMap, users need to save something like the above sample to a file and then run below commands: \n```\nkubectl apply -f \n``` \n\n\n\n[1]: unified-repo-and-kopia-integration/unified-repo-and-kopia-integration.md" }, { "path": "design/Implemented/backup-resource-list.md", "content": "# Expose list of backed up resources in backup details\n\nStatus: Accepted\n\nTo increase the visibility of what a backup might contain, this document proposes storing metadata about backed up resources in object storage and adding a new section to the detailed backup description output to list them.\n\n## Goals\n\n- Include a list of backed up resources as metadata in the bucket\n- Enable users to get a view of what resources are included in a backup using the Velero CLI\n\n## Non Goals\n\n- Expose the full manifests of the backed up resources\n\n## Background\n\nAs reported in [#396](https://github.com/heptio/velero/issues/396), the information reported in a `velero backup describe --details` command is fairly limited, and does not easily describe what resources a backup contains.\nIn order to see what a backup might contain, a user would have to download the backup tarball and extract it.\nThis makes it difficult to keep track of different backups in a cluster.\n\n## High-Level Design\n\nAfter performing a backup, a new file will be created that contains the list of the resources that have been included in the backup.\nThis file will be persisted in object storage alongside the backup contents and existing metadata.\n\nA section will be added to the output of `velero backup describe --details` command to view this metadata.\n\n## Detailed Design\n\n### Metadata file\n\nThis metadata will be in JSON (or YAML) format so that it can be easily inspected from the bucket outside of Velero tooling, and will contain the API resource and group, namespaces and names of the resources:\n\n```\napps/v1/Deployment:\n- default/database\n- default/wordpress\nv1/Service:\n- default/database\n- default/wordpress\nv1/Secret:\n- default/database-root-password\n- default/database-user-password\nv1/ConfigMap:\n- default/database\nv1/PersistentVolume:\n- my-pv\n```\n\nThe filename for this metadata will be `-resource-list.json.gz`.\nThe top-level key is the string form of the `schema.GroupResource` type that we currently keep track of in the backup controller code path.\n\n### Changes in Backup controller\n\nThe Backupper currently initialises a map to track the `backedUpItems` (https://github.com/heptio/velero/blob/1594bdc8d0132f548e18ffcc1db8c4cd2b042726/pkg/backup/backup.go#L269), this is passed down through GroupBackupper, ResourceBackupper and ItemBackupper where ItemBackupper records each backed up item.\nThis property will be moved to the [Backup request struct](https://github.com/heptio/velero/blob/16910a6215cbd8f0bde385dba9879629ebcbcc28/pkg/backup/request.go#L11), allowing the BackupController to access it after a successful backup.\n\n`backedUpItems` currently uses the `schema.GroupResource` as a key for the resource.\nIn order to record the API group, version and kind for the resource, this key will be constructed from the object's `schema.GroupVersionKind` in the format `{group}/{version}/{kind}` (e.g. `apps/v1/Deployment`).\n\nThe `backedUpItems` map is kept as a flat structure internally for quick lookup.\nWhen the backup is ready to upload, `backedUpItems` will be converted to a nested structure representing the metadata file above, grouped by `schema.GroupVersionKind`.\nAfter converting to the right format, it can be passed to the `persistBackup` function to persist the file in object storage.\n\n### Changes to DownloadRequest CRD and processing\n\nA new `DownloadTargetKind` \"BackupResourceList\" will be added to the DownloadRequest CR.\n\nThe `GetDownloadURL` function in the `persistence` package will be updated to handle this new DownloadTargetKind to enable the Velero client to fetch the metadata from the bucket.\n\n### Changes to `velero backup describe --details`\n\nThis command will need to be updated to fetch the metadata from the bucket using the `Stream` method used in other commands.\nThe file will be read in memory and displayed in the output of the command.\nDepending on the format the metadata is stored in, it may need processing to print in a more human-readable format.\nIf we choose to store the metadata in YAML, it can likely be directly printed out.\n\nIf the metadata file does not exist, this is an older backup and we cannot display the list of resources that were backed up.\n\n## Open Questions\n\n## Alternatives Considered\n\n### Fetch backup contents archive and walkthrough to list contents\n\nInstead of recording new metadata about what resources have been backed up, we could simply download the backup contents archive and walkthrough it to list the contents every time `velero backup describe --details` is run.\n\nThe advantage of this approach is that we don't need to change any backup procedures as we already have this content, and we will also be able to list resources for older backups.\nAdditionally, if we wanted to expose more information about the backed up resources, we can do so without having to update what we store in the metadata.\n\nThe disadvantages are:\n- downloading the whole backup archive will be larger than just downloading a smaller file with metadata\n- reduces the metadata available in the bucket that users might want to inspect outside of Velero tooling (though this is not an explicit requirement)\n\n## Security Considerations\n" }, { "path": "design/Implemented/backup-resources-order.md", "content": "## Backup Resources Order\r\nThis document proposes a solution that allows user to specify a backup order for resources of specific resource type.\r\n\r\n## Background\r\nDuring backup process, user may need to back up resources of specific type in some specific order to ensure the resources were backup properly because these resources are related and ordering might be required to preserve the consistency for the apps to recover itself from the backup image \r\n(Ex: primary-secondary database pods in a cluster).\r\n\r\n## Goals\r\n- Enable user to specify an order of backup resources belong to specific resource type\r\n\r\n## Alternatives Considered\r\n- Use a plugin to backup an resources and all the sub resources. For example use a plugin for StatefulSet and backup pods belong to the StatefulSet in specific order. This plugin solution is not generic and requires plugin for each resource type.\r\n\r\n## High-Level Design\r\nUser will specify a map of resource type to list resource names (separate by semicolons). Each name will be in the format \"namespaceName/resourceName\" to enable ordering across namespaces. Based on this map, the resources of each resource type will be sorted by the order specified in the list of resources. If a resource instance belong to that specific type but its name is not in the order list, then it will be put behind other resources that are in the list.\r\n\r\n### Changes to BackupSpec\r\nAdd new field to BackupSpec\r\n\r\n type BackupSpec struct {\r\n ...\r\n // OrderedResources contains a list of key-value pairs that represent the order\r\n // of backup of resources that belong to specific resource type\r\n // +optional\r\n // +nullable\r\n OrderedResources map[string]string\r\n }\r\n\r\n### Changes to itemCollector\r\nFunction getResourceItems collects all items belong to a specific resource type. This function will be enhanced to check with the map to see whether the OrderedResources has specified the order for this resource type. If such order exists, then sort the items by such order being process before return.\r\n\r\n### Changes to velero CLI\r\nAdd new flag \"--ordered-resources\" to Velero backup create command which takes a string of key-values pairs which represents the map between resource type and the order of the items of such resource type. Key-value pairs are separated by semicolon, items in the value are separated by commas.\r\n\r\nExample:\r\n>velero backup create mybackup --ordered-resources \"pod=ns1/pod1,ns1/pod2;persistentvolumeclaim=n2/slavepod,ns2/primarypod\"\r\n\r\n## Open Issues\r\n- In the CLI, the design proposes to use commas to separate items of a resource type and semicolon to separate key-value pairs. This follows the convention of using commas to separate items in a list (For example: --include-namespaces ns1,ns2). However, the syntax for map in labels and annotations use commas to separate key-value pairs. So it introduces some inconsistency.\r\n- For pods that managed by Deployment or DaemonSet, this design may not work because the pods' name is randomly generated and if pods are restarted, they would have different names so the Backup operation may not consider the restarted pods in the sorting algorithm. This problem will be addressed when we enhance the design to use regular expression to specify the OrderResources instead of exact match. \r\n" }, { "path": "design/Implemented/biav2-design.md", "content": "# Design for BackupItemAction v2 API\n\n## Abstract\nThis design includes the changes to the BackupItemAction (BIA) api design as required by the [Item Action Progress Monitoring](general-progress-monitoring.md) feature.\nThe BIA v2 interface will have two new methods, and the Execute() return signature will be modified.\nIf there are any additional BIA API changes that are needed in the same Velero release cycle as this change, those can be added here as well.\n\n## Background\nThis API change is needed to facilitate long-running plugin actions that may not be complete when the Execute() method returns.\nIt is an optional feature, so plugins which don't need this feature can simply return an empty operation ID and the new methods can be no-ops.\nThis will allow long-running plugin actions to continue in the background while Velero moves on to the next plugin, the next item, etc.\n\n## Goals\n- Allow for BIA Execute() to optionally initiate a long-running operation and report on operation status.\n\n## Non Goals\n- Allowing velero control over when the long-running operation begins.\n\n\n## High-Level Design\nAs per the [Plugin Versioning](plugin-versioning.md) design, a new BIAv2 plugin `.proto` file will be created to define the GRPC interface.\nv2 go files will also be created in `plugin/clientmgmt/backupitemaction` and `plugin/framework/backupitemaction`, and a new PluginKind will be created.\nThe velero Backup process will be modified to reference v2 plugins instead of v1 plugins.\nAn adapter will be created so that any existing BIA v1 plugin can be executed as a v2 plugin when executing a backup.\n\n## Detailed Design\n\n### proto changes (compiled into golang by protoc)\n\nThe v2 BackupItemAction.proto will be like the current v1 version with the following changes:\nExecuteResponse gets a new field:\n```\nmessage ExecuteResponse {\n bytes item = 1;\n repeated generated.ResourceIdentifier additionalItems = 2;\n string operationID = 3;\n repeated generated.ResourceIdentifier itemsToUpdate = 4;\n}\n```\nThe BackupItemAction service gets two new rpc methods:\n```\nservice BackupItemAction {\n rpc AppliesTo(BackupItemActionAppliesToRequest) returns (BackupItemActionAppliesToResponse);\n rpc Execute(ExecuteRequest) returns (ExecuteResponse);\n rpc Progress(BackupItemActionProgressRequest) returns (BackupItemActionProgressResponse);\n rpc Cancel(BackupItemActionCancelRequest) returns (google.protobuf.Empty);\n}\n```\nTo support these new rpc methods, we define new request/response message types:\n```\nmessage BackupItemActionProgressRequest {\n string plugin = 1;\n string operationID = 2;\n bytes backup = 3;\n}\n\nmessage BackupItemActionProgressResponse {\n generated.OperationProgress progress = 1;\n}\n\nmessage BackupItemActionCancelRequest {\n string plugin = 1;\n string operationID = 2;\n bytes backup = 3;\n}\n\n```\nOne new shared message type will be added, as this will also be needed for v2 RestoreItemAction and VolmeSnapshotter:\n```\nmessage OperationProgress {\n bool completed = 1;\n string err = 2;\n int64 nCompleted = 3;\n int64 nTotal = 4;\n string operationUnits = 5;\n string description = 6;\n google.protobuf.Timestamp started = 7;\n google.protobuf.Timestamp updated = 8;\n}\n```\n\nIn addition to the two new rpc methods added to the BackupItemAction interface, there is also a new `Name()` method. This one is only actually used internally by Velero to get the name that the plugin was registered with, but it still must be defined in a plugin which implements BackupItemActionV2 in order to implement the interface. It doesn't really matter what it returns, though, as this particular method is not delegated to the plugin via RPC calls. The new (and modified) interface methods for `BackupItemAction` are as follows:\n```\ntype BackupItemAction interface {\n...\n\tName() string\n...\n\tExecute(item runtime.Unstructured, backup *api.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error)\n\tProgress(operationID string, backup *api.Backup) (velero.OperationProgress, error)\n\tCancel(operationID string, backup *api.Backup) error\n...\n}\n```\n\nA new PluginKind, `BackupItemActionV2`, will be created, and the backup process will be modified to use this plugin kind.\nSee [Plugin Versioning](plugin-versioning.md) for more details on implementation plans, including v1 adapters, etc.\n\n\n## Compatibility\nThe included v1 adapter will allow any existing BackupItemAction plugin to work as expected, with an empty operation ID returned from Execute() and no-op Progress() and Cancel() methods.\n\n## Implementation\nThis will be implemented during the Velero 1.11 development cycle.\n" }, { "path": "design/Implemented/bsl-certificate-support_design.md", "content": "# Design for BSL Certificate Support Enhancement\n\n## Abstract\n\nThis design document describes the enhancement of BackupStorageLocation (BSL) certificate management in Velero, introducing a Secret-based certificate reference mechanism (`caCertRef`) alongside the existing inline certificate field (`caCert`). This enhancement provides a more secure, Kubernetes-native approach to certificate management while enabling future CLI improvements for automatic certificate discovery.\n\n## Background\n\nCurrently, Velero supports TLS certificate verification for object storage providers through an inline `caCert` field in the BSL specification. While functional, this approach has several limitations:\n\n- **Security**: Certificates are stored directly in the BSL YAML, potentially exposing sensitive data\n- **Management**: Certificate rotation requires updating the BSL resource itself\n- **CLI Usability**: Users must manually specify certificates when using CLI commands\n- **Size Limitations**: Large certificate bundles can make BSL resources unwieldy\n\nIssue #9097 and PR #8557 highlight the need for improved certificate management that addresses these concerns while maintaining backward compatibility.\n\n## Goals\n\n- Provide a secure, Secret-based certificate storage mechanism\n- Maintain full backward compatibility with existing BSL configurations\n- Enable future CLI enhancements for automatic certificate discovery\n- Simplify certificate rotation and management\n- Provide clear migration path for existing users\n\n## Non-Goals\n\n- Removing support for inline certificates immediately\n- Changing the behavior of existing BSL configurations\n- Implementing client-side certificate validation\n- Supporting certificates from ConfigMaps or other resource types\n\n## High-Level Design\n\n### API Changes\n\n#### New Field: CACertRef\n\n```go\ntype ObjectStorageLocation struct {\n // Existing field (now deprecated)\n // +optional\n // +kubebuilder:deprecatedversion:warning=\"caCert is deprecated, use caCertRef instead\"\n CACert []byte `json:\"caCert,omitempty\"`\n\n // New field for Secret reference\n // +optional\n CACertRef *corev1api.SecretKeySelector `json:\"caCertRef,omitempty\"`\n}\n```\n\nThe `SecretKeySelector` follows standard Kubernetes patterns:\n```go\ntype SecretKeySelector struct {\n // Name of the Secret\n Name string `json:\"name\"`\n // Key within the Secret\n Key string `json:\"key\"`\n}\n```\n\n### Certificate Resolution Logic\n\nThe system follows a priority-based resolution:\n\n1. If `caCertRef` is specified, retrieve certificate from the referenced Secret\n2. If `caCert` is specified (and `caCertRef` is not), use the inline certificate\n3. If neither is specified, no custom CA certificate is used\n\n### Validation\n\nBSL validation ensures mutual exclusivity:\n```go\nfunc (bsl *BackupStorageLocation) Validate() error {\n if bsl.Spec.ObjectStorage != nil &&\n bsl.Spec.ObjectStorage.CACert != nil &&\n bsl.Spec.ObjectStorage.CACertRef != nil {\n return errors.New(\"cannot specify both caCert and caCertRef in objectStorage\")\n }\n return nil\n}\n```\n\n## Detailed Design\n\n### BSL Controller Changes\n\nThe BSL controller incorporates validation during reconciliation:\n\n```go\nfunc (r *backupStorageLocationReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {\n // ... existing code ...\n \n // Validate BSL configuration\n if err := location.Validate(); err != nil {\n r.logger.WithError(err).Error(\"BSL validation failed\")\n return ctrl.Result{}, err\n }\n \n // ... continue reconciliation ...\n}\n```\n\n### Repository Provider Integration\n\nAll repository providers implement consistent certificate handling:\n\n```go\nfunc configureCACert(bsl *velerov1api.BackupStorageLocation, credGetter *credentials.CredentialGetter) ([]byte, error) {\n if bsl.Spec.ObjectStorage == nil {\n return nil, nil\n }\n\n // Prefer caCertRef (new method)\n if bsl.Spec.ObjectStorage.CACertRef != nil {\n certString, err := credGetter.FromSecret.Get(bsl.Spec.ObjectStorage.CACertRef)\n if err != nil {\n return nil, errors.Wrap(err, \"error getting CA certificate from secret\")\n }\n return []byte(certString), nil\n }\n\n // Fall back to caCert (deprecated)\n if bsl.Spec.ObjectStorage.CACert != nil {\n return bsl.Spec.ObjectStorage.CACert, nil\n }\n\n return nil, nil\n}\n```\n\n### CLI Certificate Discovery Integration\n\n#### Background: PR #8557 Implementation\nPR #8557 (\"CLI automatically discovers and uses cacert from BSL\") was merged in August 2025, introducing automatic CA certificate discovery from BackupStorageLocation for Velero CLI download operations. This eliminated the need for users to manually specify the `--cacert` flag when performing operations like `backup describe`, `backup download`, `backup logs`, and `restore logs`.\n\n#### Current Implementation (Post PR #8557)\nThe CLI now automatically discovers certificates from BSL through the `pkg/cmd/util/cacert/bsl_cacert.go` module:\n\n```go\n// Current implementation only supports inline caCert\nfunc GetCACertFromBSL(ctx context.Context, client kbclient.Client, namespace, bslName string) (string, error) {\n // ... fetch BSL ...\n if bsl.Spec.ObjectStorage != nil && len(bsl.Spec.ObjectStorage.CACert) > 0 {\n return string(bsl.Spec.ObjectStorage.CACert), nil\n }\n return \"\", nil\n}\n```\n\n#### Enhancement with caCertRef Support\nThis design extends the existing CLI certificate discovery to support the new `caCertRef` field:\n\n```go\n// Enhanced implementation supporting both caCert and caCertRef\nfunc GetCACertFromBSL(ctx context.Context, client kbclient.Client, namespace, bslName string) (string, error) {\n // ... fetch BSL ...\n\n // Prefer caCertRef over inline caCert\n if bsl.Spec.ObjectStorage.CACertRef != nil {\n secret := &corev1api.Secret{}\n key := types.NamespacedName{\n Name: bsl.Spec.ObjectStorage.CACertRef.Name,\n Namespace: namespace,\n }\n if err := client.Get(ctx, key, secret); err != nil {\n return \"\", errors.Wrap(err, \"error getting certificate secret\")\n }\n\n certData, ok := secret.Data[bsl.Spec.ObjectStorage.CACertRef.Key]\n if !ok {\n return \"\", errors.Errorf(\"key %s not found in secret\",\n bsl.Spec.ObjectStorage.CACertRef.Key)\n }\n return string(certData), nil\n }\n\n // Fall back to inline caCert (deprecated)\n if bsl.Spec.ObjectStorage.CACert != nil {\n return string(bsl.Spec.ObjectStorage.CACert), nil\n }\n\n return \"\", nil\n}\n```\n\n#### Certificate Resolution Priority\n\nThe CLI follows this priority order for certificate resolution:\n\n1. **`--cacert` flag** - Manual override, highest priority\n2. **`caCertRef`** - Secret-based certificate (recommended)\n3. **`caCert`** - Inline certificate (deprecated)\n4. **System certificate pool** - Default fallback\n\n#### User Experience Improvements\n\nWith both PR #8557 and this enhancement:\n\n```bash\n# Automatic discovery - works with both caCert and caCertRef\nvelero backup describe my-backup\nvelero backup download my-backup\nvelero backup logs my-backup\nvelero restore logs my-restore\n\n# Manual override still available\nvelero backup describe my-backup --cacert /custom/ca.crt\n\n# Debug output shows certificate source\nvelero backup download my-backup --log-level=debug\n# [DEBUG] Resolved CA certificate from BSL 'default' Secret 'storage-ca-cert' key 'ca-bundle.crt'\n```\n\n#### RBAC Considerations for CLI\n\nCLI users need read access to Secrets when using `caCertRef`:\n\n```yaml\napiVersion: rbac.authorization.k8s.io/v1\nkind: Role\nmetadata:\n name: velero-cli-user\n namespace: velero\nrules:\n- apiGroups: [\"velero.io\"]\n resources: [\"backups\", \"restores\", \"backupstoragelocations\"]\n verbs: [\"get\", \"list\"]\n- apiGroups: [\"\"]\n resources: [\"secrets\"]\n verbs: [\"get\"]\n # Limited to secrets referenced by BSLs\n```\n\n### Migration Strategy\n\n#### Phase 1: Introduction (Current)\n- Add `caCertRef` field\n- Mark `caCert` as deprecated\n- Both fields supported, mutual exclusivity enforced\n\n#### Phase 2: Migration Period\n- Documentation and tools to help users migrate\n- Warning messages for `caCert` usage\n- CLI enhancements to leverage `caCertRef`\n\n#### Phase 3: Future Removal\n- Remove `caCert` field in major version update\n- Provide migration tool for automatic conversion\n\n## User Experience\n\n### Creating a BSL with Certificate Reference\n\n1. Create a Secret containing the CA certificate:\n```yaml\napiVersion: v1\nkind: Secret\nmetadata:\n name: storage-ca-cert\n namespace: velero\ntype: Opaque\ndata:\n ca-bundle.crt: \n```\n\n2. Reference the Secret in BSL:\n```yaml\napiVersion: velero.io/v1\nkind: BackupStorageLocation\nmetadata:\n name: default\n namespace: velero\nspec:\n provider: aws\n objectStorage:\n bucket: my-bucket\n caCertRef:\n name: storage-ca-cert\n key: ca-bundle.crt\n```\n\n### Certificate Rotation\n\nWith Secret-based certificates:\n```bash\n# Update the Secret with new certificate\nkubectl create secret generic storage-ca-cert \\\n --from-file=ca-bundle.crt=new-ca.crt \\\n --dry-run=client -o yaml | kubectl apply -f -\n\n# No BSL update required - changes take effect on next use\n```\n\n### CLI Usage Examples\n\n#### Immediate Benefits\n- No change required for existing workflows\n- Certificate validation errors include helpful context\n\n#### Future CLI Enhancements\n```bash\n# Automatic certificate discovery\nvelero backup download my-backup\n\n# Manual override still available\nvelero backup download my-backup --cacert /custom/ca.crt\n\n# Debug certificate resolution\nvelero backup download my-backup --log-level=debug\n# [DEBUG] Resolved CA certificate from BSL 'default' Secret 'storage-ca-cert'\n```\n\n## Security Considerations\n\n### Advantages of Secret-based Storage\n\n1. **Encryption at Rest**: Secrets are encrypted in etcd\n2. **RBAC Control**: Fine-grained access control via Kubernetes RBAC\n3. **Audit Trail**: Secret access is auditable\n4. **Separation of Concerns**: Certificates separate from configuration\n\n### Required Permissions\n\nThe Velero server requires additional RBAC permissions:\n```yaml\n- apiGroups: [\"\"]\n resources: [\"secrets\"]\n verbs: [\"get\"]\n # Scoped to secrets referenced by BSLs\n```\n\n## Compatibility\n\n### Backward Compatibility\n\n- Existing BSLs with `caCert` continue to function unchanged\n- No breaking changes to API\n- Gradual migration path\n\n### Forward Compatibility\n\n- Design allows for future enhancements:\n - Multiple certificate support\n - Certificate chain validation\n - Automatic certificate discovery from cloud providers\n\n## Implementation Phases\n\n### Phase 1: Core Implementation ✓ (Current PR)\n- API changes with new `caCertRef` field\n- Controller validation\n- Repository provider updates\n- Basic testing\n\n### Phase 2: CLI Enhancement (Future)\n- Automatic certificate discovery in CLI\n- Enhanced error messages\n- Debug logging for certificate resolution\n\n### Phase 3: Migration Tools (Future)\n- Automated migration scripts\n- Validation tools\n- Documentation updates\n\n## Testing\n\n### Unit Tests\n- BSL validation logic\n- Certificate resolution in providers\n- Controller behavior\n\n### Integration Tests\n- End-to-end backup/restore with `caCertRef`\n- Certificate rotation scenarios\n- Migration from `caCert` to `caCertRef`\n\n### Manual Testing Scenarios\n1. Create BSL with `caCertRef`\n2. Perform backup/restore operations\n3. Rotate certificate in Secret\n4. Verify continued operation\n\n## Documentation\n\n### User Documentation\n- Migration guide from `caCert` to `caCertRef`\n- Examples for common cloud providers\n- Troubleshooting guide\n\n### API Documentation\n- Updated API reference\n- Deprecation notices\n- Field descriptions\n\n## Alternatives Considered\n\n### ConfigMap-based Storage\n- Pros: Similar to Secrets, simpler API\n- Cons: Not designed for sensitive data, no encryption at rest\n- Decision: Secrets are the Kubernetes-standard for sensitive data\n\n### External Certificate Management\n- Pros: Integration with cert-manager, etc.\n- Cons: Additional complexity, dependencies\n- Decision: Keep it simple, allow users to manage certificates as needed\n\n### Immediate Removal of Inline Certificates\n- Pros: Cleaner API, forces best practices\n- Cons: Breaking change, migration burden\n- Decision: Gradual deprecation respects existing users\n\n## Conclusion\n\nThis design provides a secure, Kubernetes-native approach to certificate management in Velero while maintaining backward compatibility. It establishes the foundation for enhanced CLI functionality and improved user experience, addressing the concerns raised in issue #9097 and enabling the features proposed in PR #8557.\n\nThe phased approach ensures smooth migration for existing users while delivering immediate security benefits for new deployments." }, { "path": "design/Implemented/clean_artifacts_in_csi_flow.md", "content": "# Design to clean the artifacts generated in the CSI backup and restore workflows\n\n## Terminology\n\n* VSC: VolumeSnapshotContent\n* VS: VolumeSnapshot\n\n## Abstract\n* The design aims to delete the unnecessary VSs and VSCs generated during CSI backup and restore process. \n* The design stop creating related VSCs during backup syncing.\n\n## Background\nIn the current CSI backup and restore workflows, please notice the CSI B/R workflows means only using the CSI snapshots in the B/R, not including the CSI snapshot data movement workflows, some generated artifacts are kept after the backup or the restore process completion.\n\nSome of them are kept due to design, for example, the VolumeSnapshotContents generated during the backup are kept to make sure the backup deletion can clean the snapshots in the storage providers.\n\nSome of them are kept by accident, for example, after restore, two VolumeSnapshotContents are generated for the same VolumeSnapshot. One is from the backup content, and one is dynamically generated from the restore's VolumeSnapshot.\n\nThe design aims to clean the unnecessary artifacts, and make the CSI B/R workflow more concise and reliable.\n\n## Goals\n- Clean the redundant VSC generated during CSI backup and restore.\n- Remove the VSCs in the backup sync process.\n\n## Non Goals\n- There were some discussion about whether Velero backup should include VSs and VSCs not generated in during the backup. By far, the conclusion is not including them is a better option. Although that is a useful enhancement, that is not included this design.\n- Delete all the CSI-related metadata files in the BSL is not the aim of this design. \n\n## Detailed Design\n### Backup\nDuring backup, the main change is the backup-generated VSCs should not kept anymore.\n\nThe reasons is we don't need them to ensure the snapshots clean up during backup deletion. Please reference to the [Backup Deletion section](#backup-deletion) section for detail.\n\nAs a result, we can simplify the VS deletion logic in the backup. Before, we need to not only delete the VS, but also recreate a static VSC pointing a non-exiting VS.\n\nThe deletion code in VS BackupItemAction can be simplify to the following:\n\n``` go\n\tif backup.Status.Phase == velerov1api.BackupPhaseFinalizing ||\n\t\tbackup.Status.Phase == velerov1api.BackupPhaseFinalizingPartiallyFailed {\n\t\tp.log.\n\t\t\tWithField(\"Backup\", fmt.Sprintf(\"%s/%s\", backup.Namespace, backup.Name)).\n\t\t\tWithField(\"BackupPhase\", backup.Status.Phase).Debugf(\"Cleaning VolumeSnapshots.\")\n\n\t\tif vsc == nil {\n\t\t\tvsc = &snapshotv1api.VolumeSnapshotContent{}\n\t\t}\n\n\t\tcsi.DeleteReadyVolumeSnapshot(*vs, *vsc, p.crClient, p.log)\n\t\treturn item, nil, \"\", nil, nil\n\t}\n\n\nfunc DeleteReadyVolumeSnapshot(\n\tvs snapshotv1api.VolumeSnapshot,\n\tvsc snapshotv1api.VolumeSnapshotContent,\n\tclient crclient.Client,\n\tlogger logrus.FieldLogger,\n) {\n\tlogger.Infof(\"Deleting Volumesnapshot %s/%s\", vs.Namespace, vs.Name)\n\tif vs.Status == nil ||\n\t\tvs.Status.BoundVolumeSnapshotContentName == nil ||\n\t\tlen(*vs.Status.BoundVolumeSnapshotContentName) <= 0 {\n\t\tlogger.Errorf(\"VolumeSnapshot %s/%s is not ready. This is not expected.\",\n\t\t\tvs.Namespace, vs.Name)\n\t\treturn\n\t}\n\n\tif vs.Status != nil && vs.Status.BoundVolumeSnapshotContentName != nil {\n\t\t// Patch the DeletionPolicy of the VolumeSnapshotContent to set it to Retain.\n\t\t// This ensures that the volume snapshot in the storage provider is kept.\n\t\tif err := SetVolumeSnapshotContentDeletionPolicy(\n\t\t\tvsc.Name,\n\t\t\tclient,\n\t\t\tsnapshotv1api.VolumeSnapshotContentRetain,\n\t\t); err != nil {\n\t\t\tlogger.Warnf(\"Failed to patch DeletionPolicy of volume snapshot %s/%s\",\n\t\t\t\tvs.Namespace, vs.Name)\n\t\t\treturn\n\t\t}\n\n\t\tif err := client.Delete(context.TODO(), &vsc); err != nil {\n\t\t\tlogger.Warnf(\"Failed to delete the VSC %s: %s\", vsc.Name, err.Error())\n\t\t}\n\t}\n\tif err := client.Delete(context.TODO(), &vs); err != nil {\n\t\tlogger.Warnf(\"Failed to delete volumesnapshot %s/%s: %v\", vs.Namespace, vs.Name, err)\n\t} else {\n\t\tlogger.Infof(\"Deleted volumesnapshot with volumesnapshotContent %s/%s\",\n\t\t\tvs.Namespace, vs.Name)\n\t}\n}\n```\n\n### Restore\n\n#### Restore the VolumeSnapshotContent\nThe current behavior of VSC restoration is that the VSC from the backup is restore, and the restored VS also triggers creating a new VSC dynamically.\n\nTwo VSCs created for the same VS in one restore seems not right.\n\nSkip restore the VSC from the backup is not a viable alternative, because VSC may reference to a [snapshot create secret](https://kubernetes-csi.github.io/docs/secrets-and-credentials-volume-snapshot-class.html?highlight=snapshotter-secret-name#createdelete-volumesnapshot-secret).\n\nIf the `SkipRestore` is set true in the restore action's result, the secret returned in the additional items is ignored too.\n\nAs a result, restore the VSC from the backup, and setup the VSC and the VS's relation is a better choice.\n\nAnother consideration is the VSC name should not be the same as the backed-up VSC's, because the older version Velero's restore and backup keep the VSC after completion.\n\nThere's high possibility that the restore will fail due to the VSC already exists in the cluster.\n\nMultiple restores of the same backup will also meet the same problem.\n\nThe proposed solution is using the restore's UID and the VS's name to generate sha256 hash value as the new VSC name. Both the VS and VSC RestoreItemAction can access those UIDs, and it will avoid the conflicts issues.\n\nThe restored VS name also shares the same generated name.\n\nThe VS-referenced VSC name and the VSC's snapshot handle name are in their status.\n\nVelero restore process purges the restore resources' metadata and status before running the RestoreItemActions.\n\nAs a result, we cannot read these information in the VS and VSC RestoreItemActions.\n\nFortunately, RestoreItemAction input parameters includes the `ItemFromBackup`. The status is intact in `ItemFromBackup`.\n\n``` go\nfunc (p *volumeSnapshotRestoreItemAction) Execute(\n\tinput *velero.RestoreItemActionExecuteInput,\n) (*velero.RestoreItemActionExecuteOutput, error) {\n\tp.log.Info(\"Starting VolumeSnapshotRestoreItemAction\")\n\n\tif boolptr.IsSetToFalse(input.Restore.Spec.RestorePVs) {\n\t\tp.log.Infof(\"Restore %s/%s did not request for PVs to be restored.\",\n\t\t\tinput.Restore.Namespace, input.Restore.Name)\n\t\treturn &velero.RestoreItemActionExecuteOutput{SkipRestore: true}, nil\n\t}\n\n\tvar vs snapshotv1api.VolumeSnapshot\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(\n\t\tinput.Item.UnstructuredContent(), &vs); err != nil {\n\t\treturn &velero.RestoreItemActionExecuteOutput{},\n\t\t\terrors.Wrapf(err, \"failed to convert input.Item from unstructured\")\n\t}\n\n\tvar vsFromBackup snapshotv1api.VolumeSnapshot\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(\n\t\tinput.ItemFromBackup.UnstructuredContent(), &vsFromBackup); err != nil {\n\t\treturn &velero.RestoreItemActionExecuteOutput{},\n\t\t\terrors.Wrapf(err, \"failed to convert input.Item from unstructured\")\n\t}\n\n\t// If cross-namespace restore is configured, change the namespace\n\t// for VolumeSnapshot object to be restored\n\tnewNamespace, ok := input.Restore.Spec.NamespaceMapping[vs.GetNamespace()]\n\tif !ok {\n\t\t// Use original namespace\n\t\tnewNamespace = vs.Namespace\n\t}\n\n\tif csiutil.IsVolumeSnapshotExists(newNamespace, vs.Name, p.crClient) {\n\t\tp.log.Debugf(\"VolumeSnapshot %s already exists in the cluster. Return without change.\", vs.Namespace+\"/\"+vs.Name)\n\t\treturn &velero.RestoreItemActionExecuteOutput{UpdatedItem: input.Item}, nil\n\t}\n\n\tnewVSCName := generateSha256FromRestoreAndVsUID(string(input.Restore.UID), string(vsFromBackup.UID))\n\t// Reset Spec to convert the VolumeSnapshot from using\n\t// the dynamic VolumeSnapshotContent to the static one.\n\tresetVolumeSnapshotSpecForRestore(&vs, &newVSCName)\n\n\t// Reset VolumeSnapshot annotation. By now, only change\n\t// DeletionPolicy to Retain.\n\tresetVolumeSnapshotAnnotation(&vs)\n\n\tvsMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&vs)\n\tif err != nil {\n\t\tp.log.Errorf(\"Fail to convert VS %s to unstructured\", vs.Namespace+\"/\"+vs.Name)\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tp.log.Infof(`Returning from VolumeSnapshotRestoreItemAction with \n\t\tno additionalItems`)\n\n\treturn &velero.RestoreItemActionExecuteOutput{\n\t\tUpdatedItem: &unstructured.Unstructured{Object: vsMap},\n\t\tAdditionalItems: []velero.ResourceIdentifier{},\n\t}, nil\n}\n\n// generateSha256FromRestoreAndVsUID Use the restore UID and the VS UID to generate the new VSC name.\n// By this way, VS and VSC RIA action can get the same VSC name.\nfunc generateSha256FromRestoreAndVsUID(restoreUID string, vsUID string) string {\n\tsha256Bytes := sha256.Sum256([]byte(restoreUID + \"/\" + vsUID))\n\treturn \"vsc-\" + hex.EncodeToString(sha256Bytes[:])\n}\n```\n\n#### Restore the VolumeSnapshot\n``` go\n// Execute restores a VolumeSnapshotContent object without modification\n// returning the snapshot lister secret, if any, as additional items to restore.\nfunc (p *volumeSnapshotContentRestoreItemAction) Execute(\n\tinput *velero.RestoreItemActionExecuteInput,\n) (*velero.RestoreItemActionExecuteOutput, error) {\n\tif boolptr.IsSetToFalse(input.Restore.Spec.RestorePVs) {\n\t\tp.log.Infof(\"Restore did not request for PVs to be restored %s/%s\",\n\t\t\tinput.Restore.Namespace, input.Restore.Name)\n\t\treturn &velero.RestoreItemActionExecuteOutput{SkipRestore: true}, nil\n\t}\n\n\tp.log.Info(\"Starting VolumeSnapshotContentRestoreItemAction\")\n\n\tvar vsc snapshotv1api.VolumeSnapshotContent\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(\n\t\tinput.Item.UnstructuredContent(), &vsc); err != nil {\n\t\treturn &velero.RestoreItemActionExecuteOutput{},\n\t\t\terrors.Wrapf(err, \"failed to convert input.Item from unstructured\")\n\t}\n\n\tvar vscFromBackup snapshotv1api.VolumeSnapshotContent\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(\n\t\tinput.ItemFromBackup.UnstructuredContent(), &vscFromBackup); err != nil {\n\t\treturn &velero.RestoreItemActionExecuteOutput{},\n\t\t\terrors.Errorf(err.Error(), \"failed to convert input.ItemFromBackup from unstructured\")\n\t}\n\n\t// If cross-namespace restore is configured, change the namespace\n\t// for VolumeSnapshot object to be restored\n\tnewNamespace, ok := input.Restore.Spec.NamespaceMapping[vsc.Spec.VolumeSnapshotRef.Namespace]\n\tif ok {\n\t\t// Update the referenced VS namespace to the mapping one.\n\t\tvsc.Spec.VolumeSnapshotRef.Namespace = newNamespace\n\t}\n\n\t// Reset VSC name to align with VS.\n\tvsc.Name = generateSha256FromRestoreAndVsUID(string(input.Restore.UID), string(vscFromBackup.Spec.VolumeSnapshotRef.UID))\n\n\t// Reset the ResourceVersion and UID of referenced VolumeSnapshot.\n\tvsc.Spec.VolumeSnapshotRef.ResourceVersion = \"\"\n\tvsc.Spec.VolumeSnapshotRef.UID = \"\"\n\n\t// Set the DeletionPolicy to Retain to avoid VS deletion will not trigger snapshot deletion\n\tvsc.Spec.DeletionPolicy = snapshotv1api.VolumeSnapshotContentRetain\n\n\tif vscFromBackup.Status != nil && vscFromBackup.Status.SnapshotHandle != nil {\n\t\tvsc.Spec.Source.VolumeHandle = nil\n\t\tvsc.Spec.Source.SnapshotHandle = vscFromBackup.Status.SnapshotHandle\n\t} else {\n\t\tp.log.Errorf(\"fail to get snapshot handle from VSC %s status\", vsc.Name)\n\t\treturn nil, errors.Errorf(\"fail to get snapshot handle from VSC %s status\", vsc.Name)\n\t}\n\n\tadditionalItems := []velero.ResourceIdentifier{}\n\tif csi.IsVolumeSnapshotContentHasDeleteSecret(&vsc) {\n\t\tadditionalItems = append(additionalItems,\n\t\t\tvelero.ResourceIdentifier{\n\t\t\t\tGroupResource: schema.GroupResource{Group: \"\", Resource: \"secrets\"},\n\t\t\t\tName: vsc.Annotations[velerov1api.PrefixedSecretNameAnnotation],\n\t\t\t\tNamespace: vsc.Annotations[velerov1api.PrefixedSecretNamespaceAnnotation],\n\t\t\t},\n\t\t)\n\t}\n\n\tvscMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&vsc)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tp.log.Infof(\"Returning from VolumeSnapshotContentRestoreItemAction with %d additionalItems\",\n\t\tlen(additionalItems))\n\treturn &velero.RestoreItemActionExecuteOutput{\n\t\tUpdatedItem: &unstructured.Unstructured{Object: vscMap},\n\t\tAdditionalItems: additionalItems,\n\t}, nil\n}\n```\n\n\n### Backup Sync\ncsi-volumesnapshotclasses.json, csi-volumesnapshotcontents.json, and csi-volumesnapshots.json are CSI-related metadata files in the BSL for each backup.\n\ncsi-volumesnapshotcontents.json and csi-volumesnapshots.json are not needed anymore, but csi-volumesnapshotclasses.json is still needed.\n\nOne concrete scenario is that a backup is created in cluster-A, then the backup is synced to cluster-B, and the backup is deleted in the cluster-B. In this case, we don't have a chance to create the VS and VSC needed VolumeSnapshotClass.\n\nThe VSC deletion workflow proposed by this design needs to create the VSC first. If the VSC's referenced VolumeSnapshotClass doesn't exist in cluster, the creation of VSC will fail.\n\nAs a result, the VolumeSnapshotClass should still be synced in the backup sync process.\n\n### Backup Deletion\nTwo factors are worthy for consideration for the backup deletion change:\n* Because the VSCs generated by the backup are not synced anymore, and the VSCs generated during the backup will not be kept too. The backup deletion needs to generate a VSC, then deletes it to make sure the snapshots in the storage provider are clean too.\n* The VSs generated by the backup are already deleted in the backup process, we don't need a DeleteItemAction for the VS anymore. As a result, the `velero.io/csi-volumesnapshot-delete` plugin is unneeded.\n\nFor the VSC DeleteItemAction, we need to generate a VSC. Because we only care about the snapshot deletion, we don't need to create a VS associated with the VSC.\n\nCreate a static VSC, then point it to a pseudo VS, and reference to the snapshot handle should be enough.\n\nTo avoid the created VSC conflict with older version Velero B/R generated ones, the VSC name is set to `vsc-uuid`.\n\nThe following is an example of the implementation.\n``` go\n\tuuid, err := uuid.NewRandom()\n\tif err != nil {\n\t\tp.log.WithError(err).Errorf(\"Fail to generate the UUID to create VSC %s\", snapCont.Name)\n\t\treturn errors.Wrapf(err, \"Fail to generate the UUID to create VSC %s\", snapCont.Name)\n\t}\n\tsnapCont.Name = \"vsc-\" + uuid.String()\n\n\tsnapCont.Spec.DeletionPolicy = snapshotv1api.VolumeSnapshotContentDelete\n\n\tsnapCont.Spec.Source = snapshotv1api.VolumeSnapshotContentSource{\n\t\tSnapshotHandle: snapCont.Status.SnapshotHandle,\n\t}\n\n\tsnapCont.Spec.VolumeSnapshotRef = corev1api.ObjectReference{\n\t\tAPIVersion: snapshotv1api.SchemeGroupVersion.String(),\n\t\tKind: \"VolumeSnapshot\",\n\t\tNamespace: \"ns-\" + string(snapCont.UID),\n\t\tName: \"name-\" + string(snapCont.UID),\n\t}\n\n\tsnapCont.ResourceVersion = \"\"\n\n\tif err := p.crClient.Create(context.TODO(), &snapCont); err != nil {\n\t\treturn errors.Wrapf(err, \"fail to create VolumeSnapshotContent %s\", snapCont.Name)\n\t}\n\n\t// Read resource timeout from backup annotation, if not set, use default value.\n\ttimeout, err := time.ParseDuration(\n\t\tinput.Backup.Annotations[velerov1api.ResourceTimeoutAnnotation])\n\tif err != nil {\n\t\tp.log.Warnf(\"fail to parse resource timeout annotation %s: %s\",\n\t\t\tinput.Backup.Annotations[velerov1api.ResourceTimeoutAnnotation], err.Error())\n\t\ttimeout = 10 * time.Minute\n\t}\n\tp.log.Debugf(\"resource timeout is set to %s\", timeout.String())\n\n\tinterval := 5 * time.Second\n\n\t// Wait until VSC created and ReadyToUse is true.\n\tif err := wait.PollUntilContextTimeout(\n\t\tcontext.Background(),\n\t\tinterval,\n\t\ttimeout,\n\t\ttrue,\n\t\tfunc(ctx context.Context) (bool, error) {\n\t\t\ttmpVSC := new(snapshotv1api.VolumeSnapshotContent)\n\t\t\tif err := p.crClient.Get(ctx, crclient.ObjectKeyFromObject(&snapCont), tmpVSC); err != nil {\n\t\t\t\treturn false, errors.Wrapf(\n\t\t\t\t\terr, \"failed to get VolumeSnapshotContent %s\", snapCont.Name,\n\t\t\t\t)\n\t\t\t}\n\n\t\t\tif tmpVSC.Status != nil && boolptr.IsSetToTrue(tmpVSC.Status.ReadyToUse) {\n\t\t\t\treturn true, nil\n\t\t\t}\n\n\t\t\treturn false, nil\n\t\t},\n\t); err != nil {\n\t\treturn errors.Wrapf(err, \"fail to wait VolumeSnapshotContent %s becomes ready.\", snapCont.Name)\n\t}\n```\n\n## Security Considerations\nSecurity is not relevant to this design.\n\n## Compatibility\nIn this design, no new information is added in backup and restore. As a result, this design doesn't have any compatibility issue.\n\n## Open Issues\nPlease notice the CSI snapshot backup and restore mechanism not supporting all file-store-based volume, e.g. Azure Files, EFS or vSphere CNS File Volume. Only block-based volumes are supported.\nRefer to [this comment](https://github.com/vmware-tanzu/velero/issues/3151#issuecomment-2623507686) for more details.\n" }, { "path": "design/Implemented/cluster-scope-resource-filter.md", "content": "# Proposal to add resource filters for backup can distinguish whether resource is cluster-scoped or namespace-scoped.\n\n- [Proposal to add resource filters for backup can distinguish whether resource is cluster-scoped or namespace-scoped.](#proposal-to-add-resource-filters-for-backup-can-distinguish-whether-resource-is-cluster-scoped-or-namespace-scoped)\n\t- [Abstract](#abstract)\n\t- [Background](#background)\n\t- [Goals](#goals)\n\t- [Non Goals](#non-goals)\n\t- [High-Level Design](#high-level-design)\n\t\t- [Parameters Rules](#parameters-rules)\n\t\t- [Using scenarios:](#using-scenarios)\n\t\t\t- [no namespace-scoped resources + some cluster-scoped resources](#no-namespace-scoped-resources--some-cluster-scoped-resources)\n\t\t\t- [no namespace-scoped resources + all cluster-scoped resources](#no-namespace-scoped-resources--all-cluster-scoped-resources)\n\t\t\t- [some namespace-scoped resources + no cluster-scoped resources](#some-namespace-scoped-resources--no-cluster-scoped-resources)\n\t\t\t\t- [scenario 1](#scenario-1)\n\t\t\t\t- [scenario 2](#scenario-2)\n\t\t\t\t- [scenario 3](#scenario-3)\n\t\t\t\t- [scenario 4](#scenario-4)\n\t\t\t- [some namespace-scoped resources + only related cluster-scoped resources](#some-namespace-scoped-resources--only-related-cluster-scoped-resources)\n\t\t\t\t- [scenario 1](#scenario-1-1)\n\t\t\t\t- [scenario 2](#scenario-2-1)\n\t\t\t\t- [scenario 3](#scenario-3-1)\n\t\t\t- [some namespace-scoped resources + some additional cluster-scoped resources](#some-namespace-scoped-resources--some-additional-cluster-scoped-resources)\n\t\t\t\t- [scenario 1](#scenario-1-2)\n\t\t\t\t- [scenario 2](#scenario-2-2)\n\t\t\t\t- [scenario 3](#scenario-3-2)\n\t\t\t\t- [scenario 4](#scenario-4-1)\n\t\t\t- [some namespace-scoped resources + all cluster-scoped resources](#some-namespace-scoped-resources--all-cluster-scoped-resources)\n\t\t\t\t- [scenario 1](#scenario-1-3)\n\t\t\t\t- [scenario 2](#scenario-2-3)\n\t\t\t\t- [scenario 3](#scenario-3-3)\n\t\t\t- [all namespace-scoped resources + no cluster-scoped resources](#all-namespace-scoped-resources--no-cluster-scoped-resources)\n\t\t\t- [all namespace-scoped resources + some additional cluster-scoped resources](#all-namespace-scoped-resources--some-additional-cluster-scoped-resources)\n\t\t\t- [all namespace-scoped resources + all cluster-scoped resources](#all-namespace-scoped-resources--all-cluster-scoped-resources)\n\t\t\t- [describe command change](#describe-command-change)\n\t- [Detailed Design](#detailed-design)\n\t- [Alternatives Considered](#alternatives-considered)\n\t- [Security Considerations](#security-considerations)\n\t- [Compatibility](#compatibility)\n\t- [Implementation](#implementation)\n\t- [Open Issues](#open-issues)\n\n## Abstract\nThe current filter (IncludedResources/ExcludedResources + IncludeClusterResources flag) is not enough for some special cases, e.g. all namespace-scoped resources + some kind of cluster-scoped resource and all namespace-scoped resources + cluster-scoped resource excludes.\nPropose to add a new group of resource filtering parameters, which can distinguish cluster-scoped and namespace-scoped resources.\n\n## Background\nThere are two sets of resource filters for Velero: `IncludedNamespaces/ExcludedNamespaces` and `IncludedResources/ExcludedResources`. \n`IncludedResources` means only including the resource types specified in the parameter. Both cluster-scoped and namespace-scoped resources are handled in this parameter by now.\nThe k8s resources are separated into cluster-scoped and namespace-scoped.\nAs a result, it's hard to include all resources in one group and only including specified resource in the other group.\n\n## Goals\n- Make Velero can support more complicated namespace-scoped and cluster-scoped resources filtering scenarios in backup.\n\n## Non Goals\n- Enrich the resource filtering rules, for example, advanced PV filtering and filtering by resource names.\n\n\n## High-Level Design\nFour new parameters are added into command `velero backup create`: `--include-cluster-scoped-resources`, `--exclude-cluster-scoped-resources`, `--include-namespace-scoped-resources` and `--exclude-namespace-scoped-resources`. \n`--include-cluster-scoped-resources` and `--exclude-cluster-scoped-resources` are used to filter cluster-scoped resources included or excluded in backup per resource type.\n`--include-namespace-scoped-resources` and `--exclude-namespace-scoped-resources` are used to filter namespace-scoped resources included or excluded in backup per resource type.\nRestore and other code pieces also use resource filtering will be handled in future releases.\n\n### Parameters Rules\n\n* `--include-cluster-scoped-resources`, `--include-namespace-scoped-resources`, `--exclude-cluster-scoped-resources` and `--exclude-namespace-scoped-resources` valid value include `*` and comma separated string. Each element of the CSV string should a k8s resource name. The format should be `resource.group`, such as `storageclasses.storage.k8s.io.`.\n\n* `--include-cluster-scoped-resources`, `--include-namespace-scoped-resources`, `--exclude-cluster-scoped-resources` and `--exclude-namespace-scoped-resources` parameters are mutual exclusive with `--include-cluster-resources`, `--include-resources` and `--exclude-resources` parameters. If both sets of parameters are provisioned, validation failure should be returned.\n\n* `--include-cluster-scoped-resources` and `--exclude-cluster-scoped-resources` should only contain cluster-scoped resource type names. If namespace-scoped resource type names are included, they are ignored.\n\n* If there are conflicts between `--include-cluster-scoped-resources` and `--exclude-cluster-scoped-resources` specified resources type lists, `--exclude-cluster-scoped-resources` parameter has higher priority.\n\n* `--include-namespace-scoped-resources` and `--exclude-namespace-scoped-resources` should only contain namespace-scoped resource type names. If cluster-scoped resource type names are included, they are ignored.\n\n* If there are conflicts between `--include-namespace-scoped-resources` and `--exclude-namespace-scoped-resources` specified resources type lists, `--exclude-namespace-scoped-resources` parameter has higher priority.\n\n* If `--include-namespace-scoped-resources` is not present, it means all namespace-scoped resources are included per resource type.\n\n* If both `--include-cluster-scoped-resources` and `--exclude-cluster-scoped-resources` are not present, it means no additional cluster-scoped resource is included per resource type, just as the existing `--include-cluster-resources` parameter not setting value. Cluster-scoped resources are related to the namespace-scoped resources, which means those are returned in the namespace-scoped resources' BackupItemAction's result AdditionalItems array, are still included in backup by default. Taking backing up PVC scenario as an example, PVC is namespace-scoped, PV is cluster-scoped. PVC's BIA will include PVC related PV into backup too.\n\n### Using scenarios:\nPlease notice, if the scenario give the example of using old filtering parameters (`--include-cluster-resources`, `--include-resources` and `--exclude-resources`), that means the old parameters also work for this case. If old parameters example is not given, that means they don't work for this scenario, only new parameters (`--include-cluster-scoped-resources`, `--include-namespace-scoped-resources`, `--exclude-cluster-scoped-resources` and `--exclude-namespace-scoped-resources`) work.\n\n#### no namespace-scoped resources + some cluster-scoped resources\nThe following command means backup no namespace-scoped resources and some cluster-scoped resources.\n\n``` bash\nvelero backup create \n\t--exclude-namespace-scoped-resources=*\n\t--include-cluster-scoped-resources=storageclass\n```\n\n#### no namespace-scoped resources + all cluster-scoped resources\nThe following command means backup no namespace-scoped resources and all cluster-scoped resources.\n\n``` bash\nvelero backup create \n\t--exclude-namespace-scoped-resources=*\n\t--include-cluster-scoped-resources=*\n```\n\n#### some namespace-scoped resources + no cluster-scoped resources\n##### scenario 1\nThe following commands mean backup all resources in namespaces default and kube-system, and no cluster-scoped resources.\n\nExample of new parameters:\n``` bash\nvelero backup create \n\t--include-namespaces=default,kube-system\n\t--exclude-cluster-scoped-resources=*\n```\n\nExample of old parameters:\n``` bash\nvelero backup create \n\t--include-namespaces=default,kube-system\n\t--include-cluster-resources=false\n```\n##### scenario 2\nThe following commands mean backup PVC, Deployment, Service, Endpoint, Pod and ReplicaSet resources in all namespaces, and no cluster-scoped resources. Although PVC's related PV should be included, due to no cluster-scoped resources are included, so they are ruled out too.\n\nExample of new parameters:\n``` bash\nvelero backup create \n\t--include-namespace-scoped-resources=persistentvolumeclaim,deployment,service,endpoint,pod,replicaset\n\t--exclude-cluster-scope-resources=*\n```\n\nExample of old parameters:\n``` bash\nvelero backup create \n\t--include-resources=persistentvolumeclaim,deployment,service,endpoint,pod,replicaset\n\t--include-cluster-resources=false\n```\n##### scenario 3\nThe following commands mean backup PVC, Deployment, Service, Endpoint, Pod and ReplicaSet resources in namespace default and kube-system, and no cluster-scoped resources. Although PVC's related PV should be included, due to no cluster-scoped resources are included, so they are ruled out too.\n\nExample of new parameters:\n``` bash\nvelero backup create \n\t--include-namespaces=default,kube-system\n\t--include-namespace-scoped-resources=persistentvolumeclaim,deployment,service,endpoint,pod,replicaset\n\t--exclude-cluster-scope-resources=*\n```\n\nExample of old parameters:\n``` bash\nvelero backup create \n\t--include-namespaces=default,kube-system\n\t--include-resources=persistentvolumeclaim,deployment,service,endpoint,pod,replicaset\n\t--include-cluster-resources=false\n```\n##### scenario 4\nThe following commands mean backup all resources except Ingress type resources in all namespaces, and no cluster-scoped resources.\n\nExample of new parameters:\n``` bash\nvelero backup create \n\t--exclude-namespace-scoped-resources=ingress\n\t--exclude-cluster-scoped-resources=*\n```\n\nExample of old parameters:\n``` bash\nvelero backup create \n\t--exclude-resources=ingress\n\t--include-cluster-resources=false\n```\n\n#### some namespace-scoped resources + only related cluster-scoped resources\n##### scenario 1\nThis means backup all resources in namespaces default and kube-system, and related cluster-scoped resources.\n``` bash\nvelero backup create \n\t--include-namespaces=default,kube-system\n```\n\n##### scenario 2\nThis means backup pods and configmaps in namespaces default and kube-system, and related cluster-scoped resources.\n``` bash\nvelero backup create \n\t--include-namespaces=default,kube-system\n\t--include-namespace-scoped-resources=pods,configmaps\n```\n\n##### scenario 3\nThis means backup all resources except Ingress type resources in all namespaces, and related cluster-scoped resources.\n\nExample of new parameters:\n``` bash\nvelero backup create \n\t--exclude-namespace-scoped-resources=ingress\n```\n\nExample of old parameters:\n``` bash\nvelero backup create \n\t--exclude-resources=ingress\n```\n\n#### some namespace-scoped resources + some additional cluster-scoped resources\n##### scenario 1\nThis means backup all resources in namespace in default, kube-system, and related cluster-scoped resources, plus all StorageClass resources.\n``` bash\nvelero backup create \n\t--include-namespaces=default,kube-system\n\t--include-cluster-scoped-resources=storageclass\n```\n\n##### scenario 2\nThis means backup PVC, Deployment, Service, Endpoint, Pod and ReplicaSet resources in all namespaces, and related cluster-scoped resources, plus all StorageClass resources, and PVC related PV.\n``` bash\nvelero backup create \n\t--include-namespace-scoped-resources=persistentvolumeclaim,deployment,service,endpoint,pod,replicaset\n\t--include-cluster-scoped-resources=storageclass\n```\n\n##### scenario 3\nThis means backup PVC, Deployment, Service, Endpoint, Pod and ReplicaSet resources in default and kube-system namespaces, and related cluster-scoped resources, plus all StorageClass resources, and PVC related PV.\n``` bash\nvelero backup create \n\t--include-namespace-scoped-resources=persistentvolumeclaim,deployment,service,endpoint,pod,replicaset\n\t--include-namespaces=default,kube-system\n\t--include-cluster-scoped-resources=storageclass\n```\n\n##### scenario 4\nThis means backup PVC, Deployment, Service, Endpoint, Pod and ReplicaSet resources in default and kube-system namespaces, and related cluster-scoped resources, plus all cluster-scoped resources except StorageClass type resources.\n``` bash\nvelero backup create \n\t--include-namespace-scoped-resources=persistentvolumeclaim,deployment,service,endpoint,pod,replicaset\n\t--include-namespaces=default,kube-system\n\t--exclude-cluster-scoped-resources=storageclass\n```\n\n#### some namespace-scoped resources + all cluster-scoped resources\n##### scenario 1\nThe following commands mean backup all resources in namespace in default, kube-system, and all cluster-scoped resources.\n\nExample of new parameters:\n``` bash\nvelero backup create \n\t--include-namespaces=default,kube-system\n\t--include-cluster-scoped-resources=*\n```\n\nExample of old parameters:\n``` bash\nvelero backup create \n\t--include-namespaces=default,kube-system\n\t--include-cluster-resources=true\n```\n\n##### scenario 2\nThis means backup Deployment, Service, Endpoint, Pod and ReplicaSet resources in all namespaces, and all cluster-scoped resources.\n``` bash\nvelero backup create \n\t--include-namespace-scoped-resources=deployment,service,endpoint,pod,replicaset\n\t--include-cluster-scoped-resources=*\n```\n\n##### scenario 3\nThis means backup Deployment, Service, Endpoint, Pod and ReplicaSet resources in default and kube-system namespaces, and all cluster-scoped resources.\n``` bash\nvelero backup create \n\t--include-namespaces=default,kube-system\n\t--include-namespace-scoped-resources=deployment,service,endpoint,pod,replicaset\n\t--include-cluster-scoped-resources=*\n```\n\n#### all namespace-scoped resources + no cluster-scoped resources\nThe following commands all mean backup all namespace-scoped resources and no cluster-scoped resources.\n\nExample of new parameters:\n``` bash\nvelero backup create \n\t--exclude-cluster-scoped-resources=*\n```\n\nExample of old parameters:\n``` bash\nvelero backup create \n\t--include-cluster-resources=false\n```\n\n#### all namespace-scoped resources + some additional cluster-scoped resources\nThis command means backup all namespace-scoped resources, and related cluster-scoped resources, plus all PersistentVolume resources.\n``` bash\nvelero backup create \n\t--include-namespaces=*\n\t--include-cluster-scoped-resources=persistentvolume\n```\n\n#### all namespace-scoped resources + all cluster-scoped resources\nThe following commands have the same meaning: backup all namespace-scoped resources, and all cluster-scoped resources.\n``` bash\nvelero backup create \n\t--include-cluster-scoped-resources=*\n```\n\n``` bash\nvelero backup create \n\t--include-cluster-resources=true\n```\n\n#### describe command change\nIn `velero backup describe` command, the four new parameters should be outputted too.\n``` bash\n velero backup describe \n......\n\nNamespaces:\n Included: ns2\n Excluded: \n\nResources:\n Included cluster-scoped: StorageClass,PersistentVolume\n Excluded cluster-scoped: \n Included namespace-scoped: default\n Excluded namespace-scoped: \n......\n```\n\n**Note:** `velero restore` command doesn't support those four new parameter in Velero v1.11, but `velero schedule` supports the four new parameters through backup specification.\n\n## Detailed Design\nWith adding `IncludedNamespaceScopedResources`, `ExcludedNamespaceScopedResources`, `IncludedClusterScopedResources` and `ExcludedClusterScopedResources`, the `BackupSpec` looks like:\n``` go\ntype BackupSpec struct {\n ......\n\t// IncludedResources is a slice of resource names to include\n\t// in the backup. If empty, all resources are included.\n\t// +optional\n\t// +nullable\n\tIncludedResources []string `json:\"includedResources,omitempty\"`\n\n\t// ExcludedResources is a slice of resource names that are not\n\t// included in the backup.\n\t// +optional\n\t// +nullable\n\tExcludedResources []string `json:\"excludedResources,omitempty\"`\n\n\t// IncludeClusterResources specifies whether cluster-scoped resources\n\t// should be included for consideration in the backup.\n\t// +optional\n\t// +nullable\n\tIncludeClusterResources *bool `json:\"includeClusterResources,omitempty\"`\n\n\t// IncludedClusterScopedResources is a slice of cluster-scoped\n\t// resource type names to include in the backup.\n\t// If set to \"*\", all cluster scope resource types are included.\n\t// The default value is empty, which means only related cluster \n\t// scope resources are included.\n\t// +optional\n\t// +nullable\n\tIncludedClusterScopedResources []string `json:\"includedClusterScopedResources,omitempty\"`\n\n\t// ExcludedClusterScopedResources is a slice of cluster-scoped \n\t// resource type names to exclude from the backup.\n\t// If set to \"*\", all cluster scope resource types are excluded.\n\t// +optional\n\t// +nullable\n\tExcludedClusterScopedResources []string `json:\"excludedClusterScopedResources,omitempty\"`\n\n\t// IncludedNamespaceScopedResources is a slice of namespace-scoped\n\t// resource type names to include in the backup.\n\t// The default value is \"*\".\n\t// +optional\n\t// +nullable\n\tIncludedNamespaceScopedResources []string `json:\"includedNamespaceScopedResources,omitempty\"`\n\n\t// ExcludedNamespaceScopedResources is a slice of namespace-scoped\n\t// resource type names to exclude from the backup.\n\t// If set to \"*\", all namespace scope resource types are excluded.\n\t// +optional\n\t// +nullable\n\tExcludedNamespaceScopedResources []string `json:\"excludedNamespaceScopedResources,omitempty\"`\n ......\n}\n```\n\n## Alternatives Considered\nProposal from Jibu Data [Issue 5120](https://github.com/vmware-tanzu/velero/issues/5120#issue-1304534563)\n\n## Security Considerations\nNo security impact.\n\n## Compatibility\nThe four new parameters cannot be mixed with existing resource filter parameters: `IncludedResources`, `ExcludedResources` and `IncludeClusterResources`.\nIf the new parameters and old parameters both appears in command line, or are specified in backup spec, the command line and the backup should fail.\n\n## Implementation\nThis change should be included into Velero v1.11.\nNew parameters will coexist with `IncludedResources`, `ExcludedResources` and `IncludeClusterResources`.\nPlan to deprecate `IncludedResources`, `ExcludedResources` and `IncludeClusterResources` in future releases, but also open to the community's feedback.\n\n## Open Issues\n`LabelSelector/OrLabelSelectors` apply to namespace-scoped resources.\nIt may be reasonable to make them also working on cluster-scoped resources.\nAn issue is created to trace this topic [resource label selector not work for cluster-scoped resources](https://github.com/vmware-tanzu/velero/issues/5787)\n" }, { "path": "design/Implemented/concurrent-backup-processing.md", "content": "# Concurrent Backup Processing\n\nThis enhancement will enable Velero to process multiple backups at the same time. This is largely a usability enhancement rather than a performance enhancement, since the overall backup throughput may not be significantly improved over the current implementation, since we are already processing individual backup items in parallel. It is a significant usability improvement, though, as with the current design, a user who submits a small backup may have to wait significantly longer than expected if the backup is submitted immediately after a large backup.\n\n## Background\n\nWith the current implementation, only one backup may be `InProgress` at a time. A second backup created will not start processing until the first backup moves on to `WaitingForPluginOperations` or `Finalizing`. This is a usability concern, especially in clusters when multiple users are initiating backups. With this enhancement, we intend to allow multiple backups to be processed concurrently. This will allow backups to start processing immediately, even if a large backup was just submitted by another user. This enhancement will build on top of the prior parallel item processing feature by creating a dedicatede ItemBlock worker pool for each running backup. The pool will be created at the beginning of the backup reconcile, and the input channel will be passed to the Kubernetes backupper just like it is in the current release.\n\nThe primary challenge is to make sure that the same workload in multiple backups is not backed up concurrently. If that were to happen, we would risk data corruption, especially around the processing of pod hooks and volume backup. For this first release we will take a conservative, high-level approach to overlap detection. Two backups will not run concurrently if there is any overlap in included namespaces. For example, if a backup that includes `ns1` and `ns2` is running, then a second backup for `ns2` and `ns3` will not be started. If a backup which does not filter namespaces is running (either a whole cluster backup or a non-namespace-limited backup with a label selector) then no other backups will be started, since a backup across all namespaces overlaps with any other backup. Calculating item-level overlap for queued backups is problematic since we don't know which items are included in a backup until backup processing has begun. A future release may add ItemBlock overlap detection, where at the item block worker level, the same item will not be processed by two different workers at the same time. This works together with workload conflict detection to further detect conflicts in a more granular level for shared resources between backups. Eventually, with a more complete understanding of individual workloads (either via ItemBlocks or some higher level model), the namespace level overlap detection may be relaxed in future versions.\n\n## Goals\n- Process multiple backups concurrently\n- Detect namespace overlap to avoid conflicts\n- For queued backups (not yet runnable due to concurrency limits or overlap), indicate the queue position in status\n\n## Non Goals\n- Handling NFS PVs when more than one PV point to the same underlying NFS share\n- Handling VGDP cancellation for failed backups on restart\n- Mounting a PVC for scenarios in which /tmp is too small for the number of concurrent backups\n- Providing a mechanism to identify high priority backups which get preferential treatment in terms of ItemBlock worker availability\n- Item-level overlap detection (future feature)\n- Providing the ability to disable namespace-level overlap detection once Item-level overlap detection is in place (although this may be supported in a future version).\n\n## High-Level Design\n\n### Backup CRD changes\n\nTwo new backup phases will be added: `Queued` and `ReadyToStart`. In the Backup workflow, new backups will be moved to the Queued phase when they are added to the backup queue. When a backup is removed from the queue because it is now able to run, it will be moved to the `ReadyToStart` phase, which will allow the backup controller to start processing it.\n\nIn addition, a new Status field, `QueuePosition`, will be added to track the backup's current position in the queue.\n\n### New Controller: `backupQueueReconciler`\n\nA new reconciler will be added, `backupQueueReconciler` which will use the current `backupReconciler` logic for reconciling `New` backups but instead of running the backup, it will move the Backup to the `Queued` phase and set `QueuePosition`.\n\nIn addition, this reconciler will periodically reconcile all queued backups (on some configurable time interval) and if there is a runnable backup, remove it from the queue, update `QueuePosition` for any queued backups behind it, and update its phase to `ReadyToStart`.\n\nQueued backups will be reconciled in order based on `QueuePosition`, so the first runnable backup found will be processed. A backup is runnable if both of the following conditions are true:\n1) The total number of backups either `InProgress` or `ReadyToStart` is less than the configured number of concurrent backups.\n2) The backup has no overlap with any backups currently `InProgress` or `ReadyToStart` or with any `Queued` backups with a higher (i.e. closer to 1) queue position than this backup.\n\n### Updates to Backup controller\n\nThe current `backupReconciler` will change its reconciling rules. Instead of watching and reconciling New backups, it will reconcile `ReadyToStart` backups. In addition, it will be configured to run in parallel by setting `MaxConcurrentReconciles` based on the `concurrent-backups` server arg.\n\nThe startup (and shutdown) of the ItemBlock worker pool will be moved from reconciler startup to the backup reconcile, which will give each running backup its own dedicated worker pool. The per-backup worker pool will will use the existing `--item-block-worker-count` installer/server arg. This means that the maximum number of ItemBlock workers for the entire Velero pod will be the ItemBlock worker count multiplied by concurrentBackups. For example, if concurrentBackups is 5, and itemBlockWorkerCount is 6, then there will be, at most, 30 worker threads active, 5 dedicated to each InProgress backup, but this maximum will only be achieved when the maximum number of backups are InProgress. This also means that each InProgress backup will have a dedicated ItemBlock input channel with the same fixed buffer size.\n\n## Detailed Design\n\n### New Install/Server configuration args\n\nA new install/server arg, `concurrent-backups` will be added. This will be an int-valued field specifying the number of backups which may be processed concurrently (with phase `InProgress`). If not specified, the default value of 1 will be used.\n\n### Consideration of backup overlap and concurrent backup processing\n\nThe primary consideration for running additional backups concurrently is the configured `concurrent-backups` parameter. If the total number of `InProgress` and `ReadyToStart` backups is equal to `concurrent-backups` then any `Queued` backups will remain in the queue.\n\nThe second consideration is backup overlap. In order to prevent interaction between running backups (particularly around volume backup and pod hooks), we cannot allow two overlapping backups to run at the same time. For now, we will define overlap broadly -- requiring that two concurrent backups don't include any of the same namespaces. A backup for `ns1` can run concurrently with a backup for `ns2`, but a backup for `[ns1,ns2]` cannot run concurrently with a backup for `ns1`. One consequence of this approach is that a backup which includes all namespaces (even if further filtered by resource or label) cannot run concurrently with *any other backup*.\n\nWhen determining which queued backup to run next, velero will look for the next queued backup which has no overlap with any InProgress backup or any Queued backup ahead of it. The reason we need to consider queued as well as running backups for overlap detection is as follows.\n\nConsider the following scenario. These are the current not-completed backups (ordered from oldest to newest)\n1. backup1, includedNamespaces: [ns1, ns2], phase: InProgress\n2. backup2, includedNamespaces: [ns2, ns3, ns5], phase: Queued, QueuePosition: 1\n3. backup3, includedNamespaces: [ns4, ns3], phase: Queued, QueuePosition: 2\n4. backup4, includedNamespaces: [ns5, ns6], phase: Queued, QueuePosition: 2\n5. backup5, includedNamespaces: [ns8, ns9], phase: Queued, QueuePosition: 3\n\nAssuming `concurrent-backups` is 2, on the next reconcile, Velero will be able to start a second backup if there is one with no overlap. `backup2` cannot run, since `ns2` overlaps between it and the running `backup1`. If we only considered running overlap (and not queued overlap), then `backup3` could run now. It conflicts with the queued `backup2` on `ns3` but it does not conflict with the running backup. However, if it runs now, then when `backup1` completes, then `backup2` still can't run (since it now overlaps with running `backup3`on `ns3`), so `backup4` starts instead. Now when `backup3` completes, `backup2` still can't run (since it now conflicts with `backup4` on `ns5`). This means that even though it was the second backup created, it's the fourth to run -- providing worse time to completion than without parallel backups. If a queued backup has a large number of namespaces (a full-cluster backup for example), it would never run as long as new single-namespace backups keep being added to the queue.\n\nTo resolve this problem we consider both running backups as well as backups ahead in the queue when resolving overlap conflicts. In the above scenario, `backup2` can't run yet since it overlaps with the running backup on `ns2`. In addition, `backup3` and `backup4` also can't run yet since they overlap with queued `backup2`. Therefore, `backup5` will run now. Once `backup1` completes, `backup2` will be free to run.\n\n### Backup CRD changes\n\nNew Backup phases:\n```go\nconst (\n\t// BackupPhaseQueued means the backup has been added to the\n\t// queue by the BackupQueueReconciler.\n\tBackupPhaseQueued BackupPhase = \"Queued\"\n\n\t// BackupPhaseReadyToStart means the backup has been removed from the\n\t// queue by the BackupQueueReconciler and is ready to start.\n\tBackupPhaseReadyToStart BackupPhase = \"ReadyToStart\"\n)\n```\n\nIn addition, a new Status field, `queuePosition`, will be added to track the backup's current position in the queue.\n```go\n\t// QueuePosition is the position held by the backup in the queue.\n\t// QueuePosition=1 means this backup is the next to be considered.\n\t// Only relevant when Phase is \"Queued\"\n\t// +optional\n\tQueuePosition int `json:\"queuePosition,omitempty\"`\n```\n\n### New Controller: `backupQueueReconciler`\n\nA new reconciler will be added, `backupQueueReconciler` which will reconcile backups under these conditions:\n1) Watching Create/Update for backups in `New` (or empty) phase\n2) Watching for Backup phase transition from `InProgress` to something else to reconcile all `Queued` backups\n2) Watching for Backup phase transition from `New` (or empty) to `Queued` to reconcile all `Queued` backups\n2) Periodic reconcile of `Queued` backups to handle backups queued at server startup as well as to make sure we never have a situation where backups are queued indefinitely because of a race condition or was otherwise missed in the reconcile on prior backup completion.\n\nThe reconciler will be set up as follows -- note that New backups are reconciled on Create/Update, while Queued backups are reconciled when an InProgress backup moves on to another state or when a new backup moves to the Queued state. We also reconcile Queued backups periodically to handle the case of a Velero pod restart with Queued backups, as well as to handle possible edge cases where a queued backup doesn't get moved out of the queue at the point of backup completion or an error occurs during a prior Queued backup reconcile.\n\n```go\nfunc (c *backupOperationsReconciler) SetupWithManager(mgr ctrl.Manager) error {\n // only consider Queued backups, order by QueuePosition\n\tgp := kube.NewGenericEventPredicate(func(object client.Object) bool {\n\t\tbackup := object.(*velerov1api.Backup)\n\t\treturn (backup.Status.Phase == velerov1api.BackupPhaseQueued)\n\t})\n\ts := kube.NewPeriodicalEnqueueSource(c.logger.WithField(\"controller\", constant.ControllerBackupOperations), mgr.GetClient(), &velerov1api.BackupList{}, c.frequency, kube.PeriodicalEnqueueSourceOption{\n\t\tPredicates: []predicate.Predicate{gp},\n\t\tOrderFunc: queuePositionOrderFunc,\n\t})\n\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\tFor(&velerov1api.Backup{}, builder.WithPredicates(predicate.Funcs{\n\t\t\t\tUpdateFunc: func(ue event.UpdateEvent) bool {\n\t\t\t\t\tbackup := ue.ObjectNew.(*velerov1api.Backup)\n\t\t\t\t\treturn backup.Status.Phase == \"\" || backup.status.Phase == velerov1api.BackupPhaseNew\n\t\t\t\t},\n\t\t\t\tCreateFunc: func(event.CreateEvent) bool {\n\t\t\t\t\treturn backup.Status.Phase == \"\" || backup.status.Phase == velerov1api.BackupPhaseNew\n\t\t\t\t},\n\t\t\t\tDeleteFunc: func(de event.DeleteEvent) bool {\n\t\t\t\t\treturn false\n\t\t\t\t},\n\t\t\t\tGenericFunc: func(ge event.GenericEvent) bool {\n\t\t\t\t\treturn false\n\t\t\t\t},\n\t\t})).\n\t\tWatch(\n\t\t\t&source.Kind{Type: &velerov1api.Backup{}},\n\t\t\t&handler.EnqueueRequestsFromMapFunc{\n\t\t\t\tToRequests: handler.ToRequestsFunc(func(a handler.MapObject) []reconcile.Request {\n\t\t\t\t\tbackupList := velerov1api.BackupList{}\n\t\t\t\t\tif err := p.List(ctx, backupList); err != nil {\n\t\t\t\t\t\tp.logger.WithError(err).Error(\"error listing backups\")\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\trequests = []reconcile.request{}\n\t\t\t\t\t// filter backup list by Phase=queued\n\t\t\t\t\t// sort backup list by queuePosition\n\t\t\t\t\treturn requests\n\t\t\t\t}),\n\t\t},\n\t\tbuilder.WithPredicates(predicate.Funcs{\n\t\t\t\tUpdateFunc: func(ue event.UpdateEvent) bool {\n\t\t\t\t\toldBackup := ue.ObjectOld.(*velerov1api.Backup)\n\t\t\t\t\tnewBackup := ue.ObjectNew.(*velerov1api.Backup)\n\t\t\t\t\treturn oldBackup.Status.Phase == velerov1api.BackupPhaseInProgress &&\n\t\t\t\t\t\tnewBackup.Status.Phase != velerov1api.BackupPhaseInProgress ||\n\t\t\t\t\t\toldBackup.Status.Phase != velerov1api.BackupPhaseQueued &&\n\t\t\t\t\t\tnewBackup.Status.Phase == velerov1api.BackupPhaseQueued\n\t\t\t\t},\n\t\t\t\tCreateFunc: func(event.CreateEvent) bool {\n\t\t\t\t\treturn false\n\t\t\t\t},\n\t\t\t\tDeleteFunc: func(de event.DeleteEvent) bool {\n\t\t\t\t\treturn false\n\t\t\t\t},\n\t\t\t\tGenericFunc: func(ge event.GenericEvent) bool {\n\t\t\t\t\treturn false\n\t\t\t\t},\n\t\t}).\n\t\tWatchesRawSource(s).\n\t\tNamed(constant.ControllerBackupQueue).\n\t\tComplete(c)\n}\n```\n\nNew backups will be queued: Phase will be set to `Queued`, and `QueuePosition` will be set to a int value incremented from the highest current `QueuePosition` value among Queued backups.\n\nQueued backups will be removed from the queue if runnable:\n1) If the total number of backups either InProgress or ReadyToStart is greater than or equal to the concurrency limit, then exit without removing from the queue.\n2) If the current backup overlaps with any InProgress, ReadyToStart, or Queued backup with `QueuePosition < currentBackup.QueuePosition` then exit without removing from the queue.\n3) If we get here, the backup is runnable. To resolve a potential race condition where an InProgress backup completes between reconciling the backup with QueuePosition `n-1` and reconciling the current backup with QueuePosition `n`, we also check to see whether there are any runnable backups in the queue ahead of this one. The only time this will happen is if a backup completes immediately before reconcile starts which either frees up a concurrency slot or removes a namespace conflict. In this case, we don't want to run the current backup since the one ahead of this one in the queue (which was recently passed over before the InProgress backup completed) must run first. In this case, exit without removing from the queue.\n4) If we get here, remove the backup from the queue by setting Phase to `ReadyToStart` and `QueuePosition` to zero. Decrement the `QueuePosition` of any other Queued backups with a `QueuePosition` higher than the current backup's queue position prior to dequeuing. At this point, the backup reconciler will start the backup.\n\n`if len(inProgressBackups)+len(pendingStartBackups) >= concurrentBackups`\n\n```\n\tswitch original.Status.Phase {\n\tcase \"\", velerov1api.BackupPhaseNew:\n\t\t// enqueue backup -- set phase=Queued, set queuePosition=maxCurrentQueuePosition+1\n\t}\n\t// We should only ever get these events when added in order by the periodical enqueue source\n\t// so as long as the current backup has not conflicts ahead of it or running, we should be good to\n\t// dequeue\n\tcase \"\", velerov1api.BackupPhaseQueued:\n\t\t// list backups, filter on Queued, ReadyToStart, and InProgress\n\t\t// if number of InProgress backups + number of ReadyToStart backups >= concurrency limit, exit\n\t\t// generate list of all namespaces included in InProgress, ReadyToStart, and Queued backups with\n\t\t// queuePosition < backup.Status.QueuePosition\n\t\t// if overlap found, exit\n\t\t// check backups ahead of this one in the queue for runnability. If any are runnable, exit\n\t\t// dequeue backup: set Phase to ReadyToStart, QueuePosition to 0, and decrement QueuePosition\n\t\t// for all QueuedBackups behind this one in the queue\n\t}\n\n```\n\nThe queue controller will run as a single reconciler thread, so we will not need to deal with concurrency issues when moving backups from New to Queued or from Queued to ReadyToStart, and all of the updates to QueuePosition will be from a single thread.\n\n### Updates to Backup controller\n\nThe Reconcile logic will be updated to respond to ReadyToStart backups instead of New backups:\n\n```\n@@ -234,8 +234,8 @@ func (b *backupReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctr\n // InProgress, we still need this check so we can return nil to indicate we've finished processing\n // this key (even though it was a no-op).\n switch original.Status.Phase {\n- case \"\", velerov1api.BackupPhaseNew:\n- // only process new backups\n+ case velerov1api.BackupPhaseReadyToStart:\n+ // only process ReadyToStart backups\n default:\n b.logger.WithFields(logrus.Fields{\n \"backup\": kubeutil.NamespaceAndName(original),\n```\n\nIn addition, it will be configured to run in parallel by setting `MaxConcurrentReconciles` based on the `concurrent-backups` server arg.\n\n```\n@@ -149,6 +149,9 @@ func NewBackupReconciler(\n func (b *backupReconciler) SetupWithManager(mgr ctrl.Manager) error {\n return ctrl.NewControllerManagedBy(mgr).\n For(&velerov1api.Backup{}).\n+ WithOptions(controller.Options{\n+ MaxConcurrentReconciles: concurrentBackups,\n+ }).\n Named(constant.ControllerBackup).\n Complete(b)\n }\n```\n\nThe controller-runtime core reconciler logic already prevents the same resource from being reconciled by two different reconciler threads, so we don't need to worry about concurrency issues at the controller level.\n\nThe workerPool reference will be moved from the backupReconciler to the backupRequest, since this will now be backup-specific, and the initialization code for the worker pool will be moved from the reconciler init into the backup reconcile. This worker pool will be shut down upon exiting the Reconcile method.\n\n### Resilience to restart of velero pod\n\nThe new backup phases (`Queued` and `ReadyToStart`) will be resilient to velero pod restarts. If the velero pod crashes or is restarted, only backups in the `InProgress` phase will be failed, so there is no change to current behavior. Queued backups will retain their queue position on restart, and ReadyToStart backups will move to InProgress when reconciled.\n\n### Observability\n\n#### Logging\n\nWhen a backup is dequeued, an info log message will also include the wait time, calculated as `now - creationTimestamp`. When a backup is passed over due to overlap, an info log message will indicate which namespaces were in conflict.\n\n#### Velero CLI\n\nThe `velero backup describe` output will include the current queue position for queued backups.\n" }, { "path": "design/Implemented/csi-snapshots.md", "content": "# CSI Snapshot Support\n\nThe Container Storage Interface (CSI) [introduced an alpha snapshot API in Kubernetes v1.12][1].\nIt will reach beta support in Kubernetes v1.17, scheduled for release in December 2019.\nThis proposal documents an approach for integrating support for this snapshot API within Velero, augmenting its existing capabilities.\n\n## Goals\n\n- Enable Velero to backup and restore CSI-backed volumes using the Kubernetes CSI CustomResourceDefinition API\n\n## Non Goals\n\n- Replacing Velero's existing [VolumeSnapshotter][7] API\n- Replacing Velero's Restic support\n\n## Background\n\nVelero has had support for performing persistent volume snapshots since its inception.\nHowever, support has been limited to a handful of providers.\nThe plugin API introduced in Velero v0.7 enabled the community to expand the number of supported providers.\nIn the meantime, the Kubernetes sig-storage advanced the CSI spec to allow for a generic storage interface, opening up the possibility of moving storage code out of the core Kubernetes code base.\nThe CSI working group has also developed a generic snapshotting API that any CSI driver developer may implement, giving users the ability to snapshot volumes from a standard interface.\n\nBy supporting the CSI snapshot API, Velero can extend its support to any CSI driver, without requiring a Velero-specific plugin be written, easing the development burden on providers while also reaching more end users.\n\n## High-Level Design\n\nIn order to support CSI's snapshot API, Velero must interact with the [`VolumeSnapshot`][2] and [`VolumeSnapshotContent`][3] CRDs.\nThese act as requests to the CSI driver to perform a snapshot on the underlying provider's volume.\nThis can largely be accomplished with Velero `BackupItemAction` and `RestoreItemAction` plugins that operate on these CRDs.\n\nAdditionally, changes to the Velero server and client code are necessary to track `VolumeSnapshot`s that are associated with a given backup, similarly to how Velero tracks its own [`volume.Snapshot`][4] type.\nTracking these is important for allowing users to see what is in their backup, and provides parity for the existing `volume.Snapshot` and [`PodVolumeBackup`][5] types.\nThis is also done to retain the object store as Velero's source of truth, without having to query the Kubernetes API server for associated `VolumeSnapshot`s.\n\n`velero backup describe --details` will use the stored VolumeSnapshots to list CSI snapshots included in the backup to the user.\n\n## Detailed Design\n\n### Resource Plugins\n\nA set of [prototype][6] plugins was developed that informed this design.\n\nThe plugins will be as follows:\n\n\n#### A `BackupItemAction` for `PersistentVolumeClaim`s, named `velero.io/csi-pvc` \n\nThis plugin will act directly on PVCs, since an implementation of Velero's VolumeSnapshotter does not have enough information about the StorageClass to properly create the `VolumeSnapshot` objects.\n\nThe associated PV will be queried and checked for the presence of `PersistentVolume.Spec.PersistentVolumeSource.CSI`. (See the \"Snapshot Mechanism Selection\" section below).\nIf this field is `nil`, then the plugin will return early without taking action.\nIf the `Backup.Spec.SnapshotVolumes` value is `false`, the plugin will return early without taking action.\n\nAdditionally, to prevent creating CSI snapshots for volumes backed up by restic, the plugin will query for all pods in the `PersistentVolumeClaim`'s namespace.\nIt will then filter out the pods that have the PVC mounted, and inspect the `backup.velero.io/backup-volumes` annotation for the associated volume's name.\nIf the name is found in the list, then the plugin will return early without taking further action.\n\nCreate a `VolumeSnapshot.snapshot.storage.k8s.io` object from the PVC.\nLabel the `VolumeSnapshot` object with the [`velero.io/backup-name`][10] label for ease of lookup later.\nAlso set an ownerRef on the `VolumeSnapshot` so that cascading deletion of the Velero `Backup` will delete associated `VolumeSnapshots`.\n\nThe CSI controllers will create a `VolumeSnapshotContent.snapshot.storage.k8s.io` object associated with the `VolumeSnapshot`.\n\nAssociated `VolumeSnapshotContent` objects will be retrieved and updated with the [`velero.io/backup-name`][10] label for ease of lookup later.\n`velero.io/volume-snapshot-name` will be applied as a label to the PVC so that the `VolumeSnapshot` can be found easily for restore.\n\n`VolumeSnapshot`, `VolumeSnapshotContent`, and `VolumeSnapshotClass` objects would be returned as additional items to be backed up. GitHub issue [1566][18] represents this work.\n\nThe `VolumeSnapshotContent.Spec.VolumeSnapshotSource.SnapshotHandle` field is the link to the underlying platform's on-disk snapshot, and must be preserved for restoration.\n\nThe plugin will _not_ wait for the `VolumeSnapshot.Status.readyToUse` field to be `true` before returning.\nThis field indicates that the snapshot is ready to use for restoration, and for different vendors can indicate that the snapshot has been made durable.\nHowever, the applications can proceed as soon as `VolumeSnapshot.Status.CreationTime` is set.\nThis also maintains current Velero behavior, which allows applications to quiesce and resume quickly, with minimal interruption.\n\nAny sort of monitoring or waiting for durable snapshots, either Velero-native or CSI snapshots, are not covered by this proposal.\n\n```\nK8s object relationships inside of the backup tarball\n+-----------------------+ +-----------------------+\n| PersistentVolumeClaim +-------------->+ PersistentVolume |\n+-----------+-----------+ +-----------+-----------+\n ^ ^\n | |\n | |\n | |\n+-----------+-----------+ +-----------+-----------+\n| VolumeSnapshot +<------------->+ VolumeSnapshotContent |\n+-----------------------+ +-----------------------+\n```\n\n#### A `RestoreItemAction` for `VolumeSnapshotContent` objects, named `velero.io/csi-vsc`\n\nOn restore, `VolumeSnapshotContent` objects are cleaned so that they may be properly associated with IDs assigned by the target cluster.\n\nOnly `VolumeSnapshotContent` objects with the `velero.io/backup-name` label will be processed, using the plugin's `AppliesTo` function.\n\nThe metadata (excluding labels), `PersistentVolumeClaim.UUID`, and `VolumeSnapshotRef.UUID` fields will be cleared.\nThe reference fields are cleared because the associated objects will get new UUIDs in the cluster.\nThis also maps to the \"import\" case of [the snapshot API][1].\n\nThis means the relationship between the `VolumeSnapshot` and `VolumeSnapshotContent` is\none way until the CSI controllers rebind them.\n\n\n```\nK8s objects after the velero.io/csi-vsc plugin has run\n+-----------------------+ +-----------------------+\n| PersistentVolumeClaim +-------------->+ PersistentVolume |\n+-----------------------+ +-----------------------+\n \n \n+-----------------------+ +-----------------------+\n| VolumeSnapshot +-------------->+ VolumeSnapshotContent |\n+-----------------------+ +-----------------------+\n```\n\n#### A `RestoreItemAction` for `VolumeSnapshot` objects, named `velero.io/csi-vs`\n\n`VolumeSnapshot` objects must be prepared for importing into the target cluster by removing IDs and metadata associated with their origin cluster.\n\nOnly `VolumeSnapshot` objects with the `velero.io/backup-name` label will be processed, using the plugin's `AppliesTo` function.\n\nMetadata (excluding labels) and `Source` (that is, the pointer to the `PersistentVolumeClaim`) fields on the object will be cleared.\nThe `VolumeSnapshot.Spec.SnapshotContentName` is the link back to the `VolumeSnapshotContent` object, and thus the actual snapshot.\nThe `Source` field indicates that a new CSI snapshot operation should be performed, which isn't relevant on restore.\nThis follows the \"import\" case of [the snapshot API][1].\n\nThe `Backup` associated with the `VolumeSnapshot` will be queried, and set as an ownerRef on the `VolumeSnapshot` so that deletion can cascade.\n\n```\n+-----------------------+ +-----------------------+\n| PersistentVolumeClaim +-------------->+ PersistentVolume |\n+-----------------------+ +-----------------------+\n \n \n+-----------------------+ +-----------------------+\n| VolumeSnapshot +-------------->+ VolumeSnapshotContent |\n+-----------------------+ +-----------------------+\n```\n\n#### A `RestoreItemAction` for `PersistentVolumeClaim`s named `velero.io/csi-pvc`\n\nOn restore, `PersistentVolumeClaims` will need to be created from the snapshot, and thus will require editing before submission.\n\nOnly `PersistentVolumeClaim` objects with the `velero.io/volume-snapshot-name` label will be processed, using the plugin's `AppliesTo` function.\nMetadata (excluding labels) will be cleared, and the `velero.io/volume-snapshot-name` label will be used to find the relevant `VolumeSnapshot`.\nA reference to the `VolumeSnapshot` will be added to the `PersistentVolumeClaim.DataSource` field.\n\n```\n+-----------------------+ \n| PersistentVolumeClaim | \n+-----------------------+ \n \n+-----------------------+ +-----------------------+\n| VolumeSnapshot +-------------->+ VolumeSnapshotContent |\n+-----------------------+ +-----------------------+\n```\n\n#### VolumeSnapshotClasses\n\nNo special logic is required to restore `VolumeSnapshotClass` objects.\n\nThese plugins should be provided with Velero, as there will also be some changes to core Velero code to enable association of a `Backup` to the included `VolumeSnapshot`s.\n\n\n\n### Velero server changes\n\nAny non-plugin code changes must be behind a `EnableCSI` feature flag and the behavior will be opt-in until it's exited beta status.\nThis will allow the development to continue on the feature while it's in pre-production state, while also reducing the need for long-lived feature branches.\n\n[`persistBackup`][8] will be extended to query for all `VolumeSnapshot`s associated with the backup, and persist the list to JSON.\n\n[`BackupStore.PutBackup`][9] will receive an additional argument, `volumeSnapshots io.Reader`, that contains the JSON representation of `VolumeSnapshots`.\nThis will be written to a file named `csi-snapshots.json.gz`.\n\n[`defaultRestorePriorities`][11] should be rewritten to the following to accommodate proper association between the CSI objects and PVCs. `CustomResourceDefinition`s are moved up because they're necessary for creating the CSI CRDs. The CSI CRDs are created before `PersistentVolume`s and `PersistentVolumeClaim`s so that they may be used as data sources.\nGitHub issue [1565][17] represents this work.\n\n```go\nvar defaultRestorePriorities = []string{\n \"namespaces\",\n \"storageclasses\",\n \"customresourcedefinitions\",\n \"volumesnapshotclass.snapshot.storage.k8s.io\",\n \"volumesnapshotcontents.snapshot.storage.k8s.io\",\n \"volumesnapshots.snapshot.storage.k8s.io\",\n \"persistentvolumes\",\n \"persistentvolumeclaims\",\n \"secrets\",\n \"configmaps\",\n \"serviceaccounts\",\n \"limitranges\",\n \"pods\",\n \"replicaset\",\n}\n```\n### Restic and CSI interaction\n\nVolumes found in a `Pod`'s `backup.velero.io/backup-volumes` list will use Velero's current Restic code path.\nThis also means Velero will continue to offer Restic as an option for CSI volumes.\n\nThe `velero.io/csi-pvc` BackupItemAction plugin will inspect pods in the namespace to ensure that it does not act on PVCs already being backed up by restic.\n\nThis is preferred to modifying the PVC due to the fact that Velero's current backup process backs up PVCs and PVs mounted to pods at the same time as the pod.\n\nA drawback to this approach is that we're querying all pods in the namespace per PVC, which could be a large number.\nIn the future, the plugin interface could be improved to have some sort of context argument, so that additional data such as our existing `resticSnapshotTracker` could be passed to plugins and reduce work.\n\n### Garbage collection and deletion\n\nTo ensure that all created resources are deleted when a backup expires or is deleted, `VolumeSnapshot`s will have an `ownerRef` defined pointing to the Velero backup that created them.\n\nIn order to fully delete these objects, each `VolumeSnapshotContent`s object will need to be edited to ensure the associated provider snapshot is deleted.\nThis will be done by editing the object and setting `VolumeSnapshotContent.Spec.DeletionPolicy` to `Delete`, regardless of whether or not the default policy for the class is `Retain`.\nSee the Deletion Policies section below.\nThe edit will happen before making Kubernetes API deletion calls to ensure that the cascade works as expected.\n\nDeleting a Velero `Backup` or any associated CSI object via `kubectl` is unsupported; data will be lost or orphaned if this is done.\n\n### Other snapshots included in the backup\n\nSince `VolumeSnapshot` and `VolumeSnapshotContent` objects are contained within a Velero backup tarball, it is possible that all CRDs and on-disk provider snapshots have been deleted, yet the CRDs are still within other Velero backup tarballs.\nThus, when a Velero backup that contains these CRDs is restored, the `VolumeSnapshot` and `VolumeSnapshotContent` objects are restored into the cluster, the CSI controllers will attempt to reconcile their state, and there are two possible states when the on-disk snapshot has been deleted:\n\n 1) If the driver _does not_ support the `ListSnapshots` gRPC method, then the CSI controllers have no way of knowing how to find it, and sets the `VolumeSnapshot.Status.readyToUse` field to `true`.\n 2) If the driver _does_ support the `ListSnapshots` gRPC method, then the CSI controllers will query the state of the on-disk snapshot, see it is missing, and set `VolumeSnapshot.Status.readyToUse` and `VolumeSnapshotContent.Status.readyToUse` fields to `false`.\n\n## Velero client changes\n\nTo use CSI features, the Velero client must use the `EnableCSI` feature flag.\n\n[`DescribeBackupStatus`][13] will be extended to download the `csi-snapshots.json.gz` file for processing. GitHub Issue [1568][19] captures this work.\n\nA new `describeCSIVolumeSnapshots` function should be added to the [output][12] package that knows how to render the included `VolumeSnapshot` names referenced in the `csi-snapshots.json.gz` file.\n\n### Snapshot selection mechanism\n\nThe most accurate, reliable way to detect if a PersistentVolume is a CSI volume is to check for a non-`nil` [`PersistentVolume.Spec.PersistentVolumeSource.CSI`][16] field.\nUsing the [`volume.beta.kubernetes.io/storage-provisioner`][14] is not viable, since the usage is for any PVC that should be dynamically provisioned, and is _not_ limited to CSI implementations.\nIt was [introduced with dynamic provisioning support][15] in 2016, predating CSI.\n\nIn the `BackupItemAction` for PVCs, the associated PV will be queried and checked for the presence of `PersistentVolume.Spec.PersistentVolumeSource.CSI`.\nVolumes with any other `PersistentVolumeSource` set will use Velero's current VolumeSnapshotter plugin code path.\n\n### VolumeSnapshotLocations and VolumeSnapshotClasses\n\nVelero uses its own `VolumeSnapshotLocation` CRDs to specify configuration options for a given storage system.\nIn Velero, this often includes topology information such as regions or availability zones, as well as credential information.\n\nCSI volume snapshotting has a `VolumeSnapshotClass` CRD which also contains configuration options for a given storage system, but these options are not the same as those that Velero would use.\nSince CSI volume snapshotting is operating within the same storage system that manages the volumes already, it does not need the same topology or credential information that Velero does.\n\nAs such, when used with CSI volumes, Velero's `VolumeSnapshotLocation` CRDs are not relevant, and could be omitted.\n\nThis will create a separate path in our documentation for the time being, and should be called out explicitly.\n\n## Alternatives Considered\n\n* Implementing similar logic in a Velero VolumeSnapshotter plugin was considered.\nHowever, this is inappropriate given CSI's data model, which requires a PVC/PV's StorageClass.\nGiven the arguments to the VolumeSnapshotter interface, the plugin would have to instantiate its own client and do queries against the Kubernetes API server to get the necessary information.\n\nThis is unnecessary given the fact that the `BackupItemAction` and `RestoreItemAction` APIs can act directly on the appropriate objects.\n\nAdditionally, the VolumeSnapshotter plugins and CSI volume snapshot drivers overlap - both produce a snapshot on backup and a PersistentVolume on restore.\nThus, there's not a logical place to fit the creation of VolumeSnapshot creation in the VolumeSnapshotter interface.\n\n* Implement CSI logic directly in Velero core code.\nThe plugins could be packaged separately, but that doesn't necessarily make sense with server and client changes being made to accommodate CSI snapshot lookup.\n\n* Implementing the CSI logic entirely in external plugins.\nAs mentioned above, the necessary plugins for `PersistentVolumeClaim`, `VolumeSnapshot`, and `VolumeSnapshotContent` could be hosted out-out-of-tree from Velero.\nIn fact, much of the logic for creating the CSI objects will be driven entirely inside of the plugin implementation.\n\nHowever, Velero currently has no way for plugins to communicate that some arbitrary data should be stored in or retrieved from object storage, such as list of all `VolumeSnapshot` objects associated with a given `Backup`.\nThis is important, because to display snapshots included in a backup, whether as native snapshots or Restic backups, separate JSON-encoded lists are stored within the backup on object storage.\nSnapshots are not listed directly on the `Backup` to fit within the etcd size limitations.\nAdditionally, there are no client-side Velero plugin mechanisms, which means that the `velero describe backup --details` command would have no way of displaying the objects to the user, even if they were stored.\n\n## Deletion Policies\n\nIn order for underlying, provider-level snapshots to be retained similarly to Velero's current functionality, the `VolumeSnapshotContent.Spec.DeletionPolicy` field must be set to `Retain`.\n\nThis is most easily accomplished by setting the `VolumeSnapshotClass.DeletionPolicy` field to `Retain`, which will be inherited by all `VolumeSnapshotContent` objects associated with the `VolumeSnapshotClass`.\n\nThe current default for dynamically provisioned `VolumeSnapshotContent` objects is `Delete`, which will delete the provider-level snapshot when the `VolumeSnapshotContent` object representing it is deleted.\nAdditionally, the `Delete` policy will cascade a deletion of a `VolumeSnapshot`, removing the associated `VolumeSnapshotContent` object.\n\nIt is not currently possible to define a deletion policy on a `VolumeSnapshot` that gets passed to a `VolumeSnapshotContent` object on an individual basis.\n\n## Security Considerations\n\nThis proposal does not significantly change Velero's security implications within a cluster.\n\nIf a deployment is using solely CSI volumes, Velero will no longer need privileges to interact with volumes or snapshots, as these will be handled by the CSI driver.\nThis reduces the provider permissions footprint of Velero.\n\nVelero must still be able to access cluster-scoped resources in order to back up `VolumeSnapshotContent` objects.\nWithout these objects, the provider-level snapshots cannot be located in order to re-associate them with volumes in the event of a restore.\n\n\n\n[1]: https://kubernetes.io/blog/2018/10/09/introducing-volume-snapshot-alpha-for-kubernetes/\n[2]: https://github.com/kubernetes-csi/external-snapshotter/blob/master/client/apis/volumesnapshot/v1/types.go#L42\n[3]: https://github.com/kubernetes-csi/external-snapshotter/blob/master/client/apis/volumesnapshot/v1/types.go#L262\n[4]: https://github.com/heptio/velero/blob/main/pkg/volume/snapshot.go#L21\n[5]: https://github.com/heptio/velero/blob/main/pkg/apis/velero/v1/pod_volume_backup.go#L88\n[6]: https://github.com/heptio/velero-csi-plugin/\n[7]: https://github.com/heptio/velero/blob/main/pkg/plugin/velero/volume_snapshotter.go#L26\n[8]: https://github.com/heptio/velero/blob/main/pkg/controller/backup_controller.go#L560\n[9]: https://github.com/heptio/velero/blob/main/pkg/persistence/object_store.go#L46\n[10]: https://github.com/heptio/velero/blob/main/pkg/apis/velero/v1/labels_annotations.go#L21\n[11]: https://github.com/heptio/velero/blob/main/pkg/cmd/server/server.go#L471\n[12]: https://github.com/heptio/velero/blob/main/pkg/cmd/util/output/backup_describer.go\n[13]: https://github.com/heptio/velero/blob/main/pkg/cmd/util/output/backup_describer.go#L214\n[14]: https://github.com/kubernetes/kubernetes/blob/8ea9edbb0290e9de1e6d274e816a4002892cca6f/pkg/controller/volume/persistentvolume/util/util.go#L69\n[15]: https://github.com/kubernetes/kubernetes/pull/30285\n[16]: https://github.com/kubernetes/kubernetes/blob/master/pkg/apis/core/types.go#L237\n[17]: https://github.com/heptio/velero/issues/1565\n[18]: https://github.com/heptio/velero/issues/1566\n[19]: https://github.com/heptio/velero/issues/1568\n" }, { "path": "design/Implemented/custom-ca-support.md", "content": "# Custom CA Bundle Support for S3 Object Storage\n\nIt is desired that Velero performs SSL verification on the Object Storage\nendpoint (BackupStorageLocation), but it is not guaranteed that the Velero\ncontainer has the endpoints' CA bundle in it's system store. Velero needs to\nsupport the ability for a user to specify custom CA bundles at installation\ntime and Velero needs to support a mechanism in the BackupStorageLocation\nCustom Resource to allow a user to specify a custom CA bundle. This mechanism\nneeds to also allow Restic to access and use this custom CA bundle.\n\n## Goals\n\n- Enable Velero to be configured with a custom CA bundle at installation\n- Enable Velero support for custom CA bundles with S3 API BackupStorageLocations\n- Enable Restic to use the custom CA bundles whether it is configured at installation time or on the BackupStorageLocation\n- Enable Velero client to take a CA bundle as an argument\n\n## Non Goals\n\n- Support non-S3 providers\n\n## Background\n\nCurrently, in order for Velero to perform SSL verification of the object\nstorage endpoint the user must manually set the `AWS_CA_BUNDLE` environment\nvariable on the Velero deployment. If the user is using Restic, the user has to\neither:\n1. Add the certs to the Restic container's system store\n1. Modify Velero to pass in the certs as a CLI parameter to Restic - requiring\n a custom Velero deployment\n\n## High-Level Design\n\nThere are really 2 methods of using Velero with custom certificates:\n1. Including a custom certificate at Velero installation\n1. Specifying a custom certificate to be used with a `BackupStorageLocation`\n\n### Specifying a custom cert at installation\n\nOn the Velero deployment at install time, we can set the AWS environment variable\n`AWS_CA_BUNDLE` which will allow Velero to communicate over https with the\nproper certs when communicating with the S3 bucket. This means we will add the\nability to specify a custom CA bundle at installation time. For more\ninformation, see \"Install Command Changes\".\n\nOn the Restic daemonset, we will want to also mount this secret at a pre-defined\nlocation. In the `restic` pkg, the command to invoke restic will need to be\nupdated to pass the path to the cert file that is mounted if it is specified in\nthe config.\n\nThis is good, but doesn't allow us to specify different certs when\n`BackupStorageLocation` resources are created.\n\n### Specifying a custom cert on BSL\n\nIn order to support custom certs for object storage, Velero will add an\nadditional field to the `BackupStorageLocation`'s provider `Config` resource to\nprovide a secretRef which will contain the coordinates to a secret containing\nthe relevant cert file for object storage. \n\nIn order for Restic to be able to consume and use this cert, Velero will need\nthe ability to write the CA bundle somewhere in memory for the Restic pod to\nconsume it.\n\nTo accomplish this, we can look at the code for managing restic repository\ncredentials. The way this works today is that the key is stored in a secret in\nthe Velero namespace, and each time Velero executes a restic command, the\ncontents of the secret are read and written out to a temp file. The path to\nthis file is then passed to restic and removed afterwards. pass the path of the\ntemp file to restic, and then remove the temp file afterwards. See ref #1 and #2.\n\nThis same approach can be taken for CA bundles. The bundle can be stored in a\nsecret which is referenced on the BSL and written to a temp file prior to\ninvoking Restic.\n\n[1](https://github.com/vmware-tanzu/velero/blob/main/pkg/restic/repository_manager.go#L238-L245)\n[2](https://github.com/vmware-tanzu/velero/blob/main/pkg/restic/common.go#L168-L203)\n\n## Detailed Design\n\nThe `AWS_CA_BUNDLE` environment variable works for the Velero deployment\nbecause this environment variable is passed into the AWS SDK which is used in\nthe [plugin][1] to build up the config object. This means that a user can\nsimply define the CA bundle in the deployment as an env var. This can be\nutilized for the installation of Velero with a custom cert by simply setting\nthis env var to the contents of the CA bundle, or the env var can be mapped to\na secret which is controlled at installation time. I recommend using a secret\nas it makes the Restic integration easier as well.\n\nAt installation time, if a user has specified a custom cert then the Restic\ndaemonset should be updated to include the secret mounted at a predefined path.\nWe could optionally use the system store for all custom certs added at\ninstallation time. Restic supports using the custom certs [in addition][3] to\nthe root certs.\n\nIn the case of the BSL being created with a secret reference, then at runtime\nthe secret will need to be consumed. This secret will be read and applied to\nthe AWS `session` object. The `getSession()` function will need to be updated\nto take in the custom CA bundle so it can be passed [here][4].\n\nThe Restic controller will need to be updated to write the contents of the CA\nbundle secret out to a temporary file inside of the restic pod.The restic\n[command invocation][2] will need to be updated to include the path to the file\nas an argument to the restic server using `--cacert`. For the path when a user\ndefines a custom cert on the BSL, Velero will be responsible for updating the\ndaemonset to include the secret mounted as a volume at a predefined path.\n\nWhere we mount the secret is a fine detail, but I recommend mounting the certs\nto `/certs` to keep it in line with the other volume mount paths being used.\n\n### Install command changes\n\nThe installation flags should be updated to include the ability to pass in a\ncert file. Then the install command would do the heavy lifting of creating a\nsecret and updating the proper fields on the deployment and daemonset to mount\nthe secret at a well defined path.\n\n### Velero client changes\n\nSince the Velero client is responsible for gathering logs and information about\nthe Object Storage, this implementation should include a new flag `--cacert`\nwhich can be used when communicating with the Object Storage. Additionally, the\nuser should be able to set this in their client configuration. The command\nwould look like:\n```\n$ velero client config set cacert PATH\n```\n\n[1]: https://github.com/vmware-tanzu/velero-plugin-for-aws/blob/main/velero-plugin-for-aws/object_store.go#L135\n[2]: https://github.com/vmware-tanzu/velero/blob/main/pkg/restic/command.go#L47\n[3]: https://github.com/restic/restic/blob/main/internal/backend/http_transport.go#L81\n[4]: https://github.com/vmware-tanzu/velero-plugin-for-aws/blob/main/velero-plugin-for-aws/object_store.go#L154\n" }, { "path": "design/Implemented/delete-item-action.md", "content": "# Delete Item Action Plugins\n\n## Abstract\n\nVelero should provide a way to delete items created during a backup, with a model and interface similar to that of BackupItemAction and RestoreItemAction plugins.\nThese plugins would be invoked when a backup is deleted, and would receive items from within the backup tarball.\n\n## Background\n\nAs part of Container Storage Interface (CSI) snapshot support, Velero added a new pattern for backing up and restoring snapshots via BackupItemAction and RestoreItemAction plugins.\nWhen others have tried to use this pattern, however, they encountered issues with deleting the resources made in their own ItemAction plugins, as Velero does not expose any sort of extension at backup deletion time.\nThese plugins largely seek to delete resources that exist outside of Kubernetes.\nThis design seeks to provide the missing extension point.\n\n## Goals\n\n- Provide a DeleteItemAction API for plugins to implement\n- Update Velero backup deletion logic to invoke registered DeleteItemAction plugins.\n\n## Non Goals\n\n- Specific implementations of the DeleteItemAction API beyond test cases.\n- Rollback of DeleteItemAction execution.\n\n## High-Level Design\n\nThe DeleteItemAction plugin API will closely resemble the RestoreItemAction plugin design, in that plugins will receive the Velero `Backup` Go struct that is being deleted and a matching Kubernetes resource extracted from the backup tarball.\n\nThe Velero backup deletion process will be modified so that if there are any DeleteItemAction plugins registered, the backup tarball will be downloaded and extracted, similar to how restore logic works now.\nThen, each item in the backup tarball will be iterated over to see if a DeleteItemAction plugin matches for it.\nIf a DeleteItemAction plugin matches, the `Backup` and relevant item will be passed to the DeleteItemAction.\n\nThe DeleteItemAction plugins will be run _first_ in the backup deletion process, before deleting snapshots from storage or `Restore`s from the Kubernetes API server.\n\nDeleteItemAction plugins *cannot* rollback their actions.\nThis is because there is currently no way to recover other deleted components of a backup, such as volume/restic snapshots or other DeleteItemAction resources.\n\nDeleteItemAction plugins will be run in alphanumeric order based on their registered names.\n\n## Detailed Design\n\n### New types\n\nThe `DeleteItemAction` interface is as follows:\n\n```go\n// DeleteItemAction is an actor that performs an action based on an item in a backup that is being deleted.\ntype DeleteItemAction interface {\n\t// AppliesTo returns information about which resources this action should be invoked for.\n\t// A DeleteItemAction's Execute function will only be invoked on items that match the returned\n\t// selector. A zero-valued ResourceSelector matches all resources.\n AppliesTo() (ResourceSelector, error)\n\n\t// Execute allows the ItemAction to perform arbitrary logic with the item being deleted.\n Execute(DeleteItemActionInput) error\n}\n```\n\nThe `DeleteItemActionInput` type is defined as follows:\n\n```go\ntype DeleteItemActionInput struct {\n\t// Item is the item taken from the pristine backed up version of resource.\n\tItem runtime.Unstructured\n\t// Backup is the representation of the backup resource processed by Velero.\n\tBackup *api.Backup\n}\n```\n\nBoth `DeleteItemAction` and `DeleteItemActionInput` will be defined in `pkg/plugin/velero/delete_item_action.go`.\n\n### Generate protobuf definitions and client/servers\n\nIn `pkg/plugin/proto`, add `DeleteItemAction.proto`.\n\nProtobuf definitions will be necessary for:\n\n```protobuf\nmessage DeleteItemActionExecuteRequest {\n\t...\n}\n\nmessage DeleteItemActionExecuteResponse {\n\t...\n}\n\nmessage DeleteItemActionAppliesToRequest {\n\t...\n}\n\nmessage DeleteItemActionAppliesToResponse {\n\t...\n}\n\nservice DeleteItemAction {\n\trpc AppliesTo(DeleteItemActionAppliesToRequest) returns (DeleteItemActionAppliesToResponse)\n\trpc Execute(DeleteItemActionExecuteRequest) returns (DeleteItemActionExecuteResponse)\n}\n```\n\nOnce these are written, then a client and server implementation can be written in `pkg/plugin/framework/delete_item_action_client.go` and `pkg/plugin/framework/delete_item_action_server.go`, respectively.\nThese should be largely the same as the client and server implementations for `RestoreItemAction` and `BackupItemAction` plugins.\n\n### Restartable delete plugins\n\nSimilar to `RestoreItemAction` and `BackupItemAction` plugins, restartable processes will need to be implemented.\n\nIn `pkg/plugin/clientmgmt`, add `restartable_delete_item_action.go`, creating the following unexported type:\n\n```go\ntype restartableDeleteItemAction struct {\n\tkey kindAndName\n\tsharedPluginProcess RestartableProcess\n\tconfig map[string]string\n}\n\n// newRestartableDeleteItemAction returns a new restartableDeleteItemAction.\nfunc newRestartableDeleteItemAction(name string, sharedPluginProcess RestartableProcess) *restartableDeleteItemAction {\n\t// ...\n}\n\n// getDeleteItemAction returns the delete item action for this restartableDeleteItemAction. It does *not* restart the\n// plugin process.\nfunc (r *restartableDeleteItemAction) getDeleteItemAction() (velero.DeleteItemAction, error) {\n\t// ...\n}\n\n// getDelegate restarts the plugin process (if needed) and returns the delete item action for this restartableDeleteItemAction.\nfunc (r *restartableDeleteItemAction) getDelegate() (velero.DeleteItemAction, error) {\n\t// ...\n}\n\n// AppliesTo restarts the plugin's process if needed, then delegates the call.\nfunc (r *restartableDeleteItemAction) AppliesTo() (velero.ResourceSelector, error) {\n\t// ...\n}\n\n// Execute restarts the plugin's process if needed, then delegates the call.\nfunc (r *restartableDeleteItemAction) Execute(input *velero.DeleteItemActionInput) (error) {\n\t// ...\n}\n```\n\nThis file will be very similar in structure to \n\n### Plugin manager changes\n\nAdd the following methods to `pkg/plugin/clientmgmt/manager.go`'s `Manager` interface:\n\n```go\ntype Manager interface {\n\t...\n\t// Get DeleteItemAction returns a DeleteItemAction plugin for name.\n\tGetDeleteItemAction(name string) (DeleteItemAction, error)\n\n\t// GetDeteteItemActions returns the all DeleteItemAction plugins.\n\tGetDeleteItemActions() ([]DeleteItemAction, error)\n}\n```\n\nThe unexported `manager` type should implement both the `GetDeleteItemAction` and `GetDeleteItemActions`.\n\nBoth of these methods should have the same exception for `velero.io/`-prefixed plugins that all other types do.\n\n`GetDeleteItemAction` and `GetDeleteItemActions` will invoke the `restartableDeleteItemAction` implementations.\n\n\n### Deletion controller modifications\n\n`pkg/controller/backup_deletion_controller.go` will be updated to have plugin management invoked.\n\nIn `processRequest`, before deleting snapshots, get any registered `DeleteItemAction` plugins.\nIf there are none, proceed as normal.\nIf there are one or more, download the backup tarball from backup storage, untar it to temporary storage, and iterate through the items, matching them to the applicable plugins.\n\n## Alternatives Considered\n\nAnother proposal for higher level `DeleteItemActions` was initially included, which would require implementers to individually download the backup tarball themselves.\nWhile this may be useful long term, it is not a good fit for the current goals as each plugin would be re-implementing a lot of boilerplate.\nSee the deletion-plugins.md file for this alternative proposal in more detail.\n\nThe `VolumeSnapshotter` interface is not generic enough to meet the requirements here, as it is specifically for taking snapshots of block devices.\n\n## Security Considerations\n\nBy their nature, `DeleteItemAction` plugins will be deleting data, which would normally be a security concern.\nHowever, these will only be invoked in two situations: either when a `BackupDeleteRequest` is sent via a user with the `velero` CLI or some other management system, or when a Velero `Backup` expires by going over its TTL.\nBecause of this, the data deletion is not a concern.\n\n## Compatibility\n\nIn terms of backwards compatibility, this design should stay compatible with most Velero installations that are upgrading.\nIf not DeleteItemAction plugins are present, then the backup deletion process should proceed the same way it worked prior to their inclusion.\n\n## Implementation\n\nThe implementation dependencies are, roughly, in the order as they are described in the [Detailed Design](#detailed-design) section.\n\n## Open Issues\n" }, { "path": "design/Implemented/deletion-plugins.md", "content": "# Deletion Plugins\n\nStatus: Alternative Proposal\n\n\n## Abstract\nVelero should introduce a new type of plugin that runs when a backup is deleted.\nThese plugins will delete any external resources associated with the backup so that they will not be left orphaned.\n\n## Background\nWith the CSI plugin, Velero developers introduced a pattern of using BackupItemAction and RestoreItemAction plugins tied to PersistentVolumeClaims to create other resources to complete a backup.\nIn the CSI plugin case, Velero does clean up of these other resources, which are Kubernetes Custom Resources, within the core Velero server.\nHowever, for external plugins that wish to use this same pattern, this is not a practical solution.\nVelero's core cannot be extended for all possible Custom Resources, and not external resources that get created are Kubernetes Custom Resources.\n\nTherefore, Velero needs some mechanism that allows plugin authors who have created resources within a BackupItemAction or RestoreItemAction plugin to ensure those resources are deleted, regardless of what system those resources reside in.\n\n## Goals\n- Provide a new plugin type in Velero that is invoked when a backup is deleted.\n\n## Non Goals\n- Implementations of specific deletion plugins.\n- Rollback of deletion plugin execution.\n\n\n## High-Level Design\nVelero will provide a new plugin type that is similar to its existing plugin architecture.\nThese plugins will be referred to as `DeleteAction` plugins.\n`DeleteAction` plugins will receive the `Backup` CustomResource being deleted on execution.\n\n`DeleteAction` plugins cannot prevent deletion of an item.\nThis is because multiple `DeleteAction` plugins can be registered, and this proposal does not include rollback and undoing of a deletion action.\nThus, if multiple `DeleteAction` plugins have already run but another would request the deletion of a backup stopped, the backup that's retained would be inconsistent.\n\n`DeleteActions` will apply to `Backup`s based on a label on the `Backup` itself.\nIn order to ensure that `Backup`s don't execute `DeleteAction` plugins that are not relevant to them, `DeleteAction` plugins can register an `AppliesTo` function which will define a label selector on Velero backups.\n\n`DeleteActions` will be run in alphanumerical order by plugin name.\nThis order is somewhat arbitrary, but will be used to give authors and users a somewhat predictable order of events.\n\n## Detailed Design\nThe `DeleteAction` plugins will implement the following Go interface, defined in `pkg/plugin/velero/deletion_action.go`:\n\n```go\ntype DeleteAction struct {\n\n // AppliesTo will match the DeleteAction plugin against Velero Backups that it should operate against.\n AppliesTo()\n\n // Execute runs the custom plugin logic and may connect to external services.\n Execute(backup *api.backup) error\n}\n\n```\n\nThe following methods would be added to the `clientmgmt.Manager` interface in `pkg/pluginclientmgmt/manager.go`:\n\n```\ntype Manager interface {\n ...\n\n // GetDeleteActions returns the registered DeleteActions.\n //TODO: do we need to get these by name, or can we get them all?\n GetDeleteActions([]velero.DeleteAction, error)\n ...\n```\n\n\n## Alternatives Considered\nTODO\n\n## Security Considerations\nTODO\n\n## Compatibility\nBackwards compatibility should be straight-forward; if there are no installed `DeleteAction` plugins, then the backup deletion process will proceed as it does today.\n\n## Implementation\nTODO\n\n## Open Issues\nIn order to add a custom label to the backup, the backup must be modifiable inside of the `BackupItemActon` and `RestoreItemAction` plugins, which it currently is not. A work around for now is for the user to apply a label to the backup at creation time, but that is not ideal.\n" }, { "path": "design/Implemented/existing-resource-policy_design.md", "content": "# Add support for `ExistingResourcePolicy` to restore API\n## Abstract\nVelero currently does not support any restore policy on Kubernetes resources that are already present in-cluster. Velero skips over the restore of the resource if it already exists in the namespace/cluster irrespective of whether the resource present in the restore is the same or different from the one present on the cluster. It is desired that Velero gives the option to the user to decide whether or not the resource in backup should overwrite the one present in the cluster.\n\n## Background\nAs of Today, Velero will skip over the restoration of resources that already exist in the cluster. The current workflow followed by Velero is (Using a `service` that is backed up for example):\n- Velero tries to attempt restore of the `service` \n- Fetches the `service` from the cluster \n- If the `service` exists then:\n - Checks whether the `service` instance in the cluster is equal to the `service` instance present in backup\n - If not equal then skips the restore of the `service` and adds a restore warning (except for [ServiceAccount objects](https://github.com/vmware-tanzu/velero/blob/574baeb3c920f97b47985ec3957debdc70bcd5f8/pkg/restore/restore.go#L1246))\n - If equal then skips the restore of the `service` and mentions that the restore of resource `service` is skipped in logs\n\nIt is desired to add the functionality to specify whether or not to overwrite the instance of resource `service` in cluster with the one present in backup during the restore process.\n\nRelated issue: https://github.com/vmware-tanzu/velero/issues/4066\n\n## Goals\n- Add support for `ExistingResourcePolicy` to restore API for Kubernetes resources.\n\n## Non Goals\n- Change existing restore workflow for `ServiceAccount` objects\n- Add support for `ExistingResourcePolicy` as `recreate` for Kubernetes resources. (Future scope feature)\n\n## Unrelated Proposals (Completely different functionalities than the one proposed in the design)\n- Add support for `ExistingResourcePolicy` to restore API for Non-Kubernetes resources.\n- Add support for `ExistingResourcePolicy` to restore API for `PersistentVolume` data.\n\n### Use-cases/Scenarios\n\n### A. Production Cluster - Backup Cluster:\nLet's say you have a Backup Cluster which is identical to the Production Cluster. After some operations/usage/time the Production Cluster had changed itself, there might be new deployments, some secrets might have been updated. Now, this means that the Backup cluster will no longer be identical to the Production Cluster. In order to keep the Backup Cluster up to date/identical to the Production Cluster with respect to Kubernetes resources except PV data we would like to use Velero for scheduling new backups which would in turn help us update the Backup Cluster via Velero restore.\n\nReference: https://github.com/vmware-tanzu/velero/issues/4066#issuecomment-954320686\n\n### B. Help identify resource delta:\nHere delta resources mean the resources restored by a previous backup, but they are no longer in the latest backup. Let's follow a sequence of steps to understand this scenario:\n- Consider there are 2 clusters, Cluster A, which has 3 resources - P1, P2 and P3.\n- Create a Backup1 from Cluster A which has P1, P2 and P3.\n- Perform restore on a new Cluster B using Backup1.\n- Now, Lets say in Cluster A resource P1 gets deleted and resource P2 gets updated.\n- Create a new Backup2 with the new state of Cluster A, keep in mind Backup1 has P1, P2 and P3 while Backup2 has P2' and P3.\n- So the Delta here is (|Cluster B - Backup2|), Delete P1 and Update P2.\n- During Restore time we would want the Restore to help us identify this resource delta.\n\nReference: https://github.com/vmware-tanzu/velero/pull/4613#issuecomment-1027260446\n\n## High-Level Design\n### Approach 1: Add a new spec field `existingResourcePolicy` to the Restore API\nIn this approach we do *not* change existing velero behavior. If the resource to restore in cluster is equal to the one backed up then do nothing following current Velero behavior. For resources that already exist in the cluster that are not equal to the resource in the backup (other than Service Accounts). We add a new optional spec field `existingResourcePolicy` which can have the following values:\n1. `none`: This is the existing behavior, if Velero encounters a resource that already exists in the cluster, we simply\nskip restoration.\n2. `update`: This option would provide the following behavior.\n - Unchanged resources: Velero would update the backup/restore labels on the unchanged resources, if labels patch fails Velero adds a restore error.\n - Changed resources: Velero will first try to patch the changed resource, Now if the patch:\n - succeeds: Then the in-cluster resource gets updated with the labels as well as the resource diff\n - fails: Velero adds a restore warning and tries to just update the backup/restore labels on the resource, if the labels patch also fails then we add restore error.\n3. `recreate`: If resource already exists, then Velero will delete it and recreate the resource.\n\n*Note:* The `recreate` option is a non-goal for this enhancement proposal, but it is considered as a future scope.\nAnother thing to highlight is that Velero will not be deleting any resources in any of the policy options proposed in \nthis design but Velero will patch the resources in `update` policy option.\n\nExample:\nA. The following Restore will execute the `existingResourcePolicy` restore type `none` for the `services` and `deployments` present in the `velero-protection` namespace.\n\n```\nKind: Restore\n\n…\n\nincludeNamespaces: velero-protection\nincludeResources:\n - services\n - deployments\nexistingResourcePolicy: none\n\n```\n\nB. The following Restore will execute the `existingResourcePolicy` restore type `update` for the `secrets` and `daemonsets` present in the `gdpr-application` namespace.\n```\nKind: Restore\n\n…\nincludeNamespaces: gdpr-application\nincludeResources:\n - secrets\n - daemonsets\nexistingResourcePolicy: update\n```\n\n### Approach 2: Add a new spec field `existingResourcePolicyConfig` to the Restore API\nIn this approach we give user the ability to specify which resources are to be included for a particular kind of force update behaviour, essentially a more granular approach where in the user is able to specify a resource:behaviour mapping. It would look like:\n`existingResourcePolicyConfig`:\n- `patch:` \n - `includedResources:` [ ]string\n- `recreate:`\n - `includedResources:` [ ]string\n\n*Note:* \n- There is no `none` behaviour in this approach as that would conform to the current/default Velero restore behaviour.\n- The `recreate` option is a non-goal for this enhancement proposal, but it is considered as a future scope.\n\n\nExample:\nA. The following Restore will execute the restore type `patch` and apply the `existingResourcePolicyConfig` for `secrets` and `daemonsets` present in the `inventory-app` namespace.\n```\nKind: Restore\n…\nincludeNamespaces: inventory-app\nexistingResourcePolicyConfig:\n patch:\n includedResources\n - secrets\n - daemonsets\n\n```\n\n\n### Approach 3: Combination of Approach 1 and Approach 2\n\nNow, this approach is somewhat a combination of the aforementioned approaches. Here we propose addition of two spec fields to the Restore API - `existingResourceDefaultPolicy` and `existingResourcePolicyOverrides`. As the names suggest ,the idea being that `existingResourceDefaultPolicy` would describe the default velero behaviour for this restore and `existingResourcePolicyOverrides` would override the default policy explicitly for some resources.\n\nExample:\nA. The following Restore will execute the restore type `patch` as the `existingResourceDefaultPolicy` but will override the default policy for `secrets` using the `existingResourcePolicyOverrides` spec as `none`.\n```\nKind: Restore\n…\nincludeNamespaces: inventory-app\nexistingResourceDefaultPolicy: patch\nexistingResourcePolicyOverrides:\n none:\n includedResources\n - secrets\n\n```\n\n## Detailed Design\n### Approach 1: Add a new spec field `existingResourcePolicy` to the Restore API\nThe `existingResourcePolicy` spec field will be an `PolicyType` type field. \n\nRestore API:\n```\ntype RestoreSpec struct {\n.\n.\n.\n// ExistingResourcePolicy specifies the restore behaviour for the Kubernetes resource to be restored\n// +optional\nExistingResourcePolicy PolicyType\n\n}\n```\nPolicyType:\n```\ntype PolicyType string\nconst PolicyTypeNone PolicyType = \"none\"\nconst PolicyTypePatch PolicyType = \"update\"\n```\n\n### Approach 2: Add a new spec field `existingResourcePolicyConfig` to the Restore API\nThe `existingResourcePolicyConfig` will be a spec of type `PolicyConfiguration` which gets added to the Restore API.\n\nRestore API:\n```\ntype RestoreSpec struct {\n.\n.\n.\n// ExistingResourcePolicyConfig specifies the restore behaviour for a particular/list of Kubernetes resource(s) to be restored\n// +optional\nExistingResourcePolicyConfig []PolicyConfiguration\n\n}\n```\n\nPolicyConfiguration:\n```\ntype PolicyConfiguration struct {\n\nPolicyTypeMapping map[PolicyType]ResourceList\n\n}\n```\n\nPolicyType:\n```\ntype PolicyType string\nconst PolicyTypePatch PolicyType = \"patch\"\nconst PolicyTypeRecreate PolicyType = \"recreate\"\n```\n\nResourceList:\n```\ntype ResourceList struct {\nIncludedResources []string \n}\n```\n\n### Approach 3: Combination of Approach 1 and Approach 2\n\nRestore API:\n```\ntype RestoreSpec struct {\n.\n.\n.\n// ExistingResourceDefaultPolicy specifies the default restore behaviour for the Kubernetes resource to be restored\n// +optional\nexistingResourceDefaultPolicy PolicyType\n\n// ExistingResourcePolicyOverrides specifies the restore behaviour for a particular/list of Kubernetes resource(s) to be restored\n// +optional\nexistingResourcePolicyOverrides []PolicyConfiguration\n\n}\n```\n\nPolicyType:\n```\ntype PolicyType string\nconst PolicyTypeNone PolicyType = \"none\"\nconst PolicyTypePatch PolicyType = \"patch\"\nconst PolicyTypeRecreate PolicyType = \"recreate\"\n```\nPolicyConfiguration:\n```\ntype PolicyConfiguration struct {\n\nPolicyTypeMapping map[PolicyType]ResourceList\n\n}\n```\nResourceList:\n```\ntype ResourceList struct {\nIncludedResources []string \n}\n```\n\nThe restore workflow changes will be done [here](https://github.com/vmware-tanzu/velero/blob/b40bbda2d62af2f35d1406b9af4d387d4b396839/pkg/restore/restore.go#L1245)\n\n### CLI changes for Approach 1\nWe would introduce a new CLI flag called `existing-resource-policy` of string type. This flag would be used to accept the \npolicy from the user. The velero restore command would look somewhat like this:\n```\nvelero create restore --existing-resource-policy=update \n```\n\nHelp message `Restore Policy to be used during the restore workflow, can be - none, update`\n\nThe CLI changes will go at `pkg/cmd/cli/restore/create.go`\n\nWe would also add a validation which checks for invalid policy values provided to this flag.\n\nRestore describer will also be updated to reflect the policy `pkg/cmd/util/output/restore_describer.go`\n\n### Implementation Decision\nWe have decided to go ahead with the implementation of Approach 1 as:\n- It is easier to implement \n- It is also easier to scale and leaves room for improvement and the door open to expanding to approach 3\n- It also provides an option to preserve the existing velero restore workflow" }, { "path": "design/Implemented/feature-flags.md", "content": "# Feature Flags\n\nStatus: Accepted\n\nSome features may take a while to get fully implemented, and we don't necessarily want to have long-lived feature branches\nA simple feature flag implementation allows code to be merged into main, but not used unless a flag is set.\n\n## Goals\n\n- Allow unfinished features to be present in Velero releases, but only enabled when the associated flag is set.\n\n## Non Goals\n\n- A robust feature flag library.\n\n## Background\n\nWhen considering the [CSI integration work](https://github.com/heptio/velero/pull/1661), the timelines involved presented a problem in balancing a release and longer-running feature work.\nA simple implementation of feature flags can help protect unfinished code while allowing the rest of the changes to ship.\n\n## High-Level Design\n\nA new command line flag, `--features` will be added to the root `velero` command.\n\n`--features` will accept a comma-separated list of features, such as `--features EnableCSI,Replication`.\nEach feature listed will correspond to a key in a map in `pkg/features/features.go` defining whether a feature should be enabled.\n\nAny code implementing the feature would then import the map and look up the key's value.\n\nFor the Velero client, a `features` key can be added to the `config.json` file for more convenient client invocations.\n\n## Detailed Design\n\nA new `features` package will be introduced with these basic structs:\n\n```go\ntype FeatureFlagSet struct {\n flags map[string]bool\n}\n\ntype Flags interface {\n // Enabled reports whether or not the specified flag is found.\n Enabled(name string) bool\n\n // Enable adds the specified flags to the list of enabled flags.\n Enable(names ...string)\n\n // All returns all enabled features\n All() []string\n}\n\n// NewFeatureFlagSet initializes and populates a new FeatureFlagSet\nfunc NewFeatureFlagSet(flags ...string) FeatureFlagSet\n```\n\nWhen parsing the `--features` flag, the entire `[]string` will be passed to `NewFeatureFlagSet`.\nAdditional features can be added with the `Enable` function.\nParsed features will be printed as an `Info` level message on server start up.\n\nNo verification of features will be done in order to keep the implementation minimal.\n\nOn the client side, `--features` and the `features` key in `config.json` file will be additive, resulting in the union of both.\n\nTo disable a feature, the server must be stopped and restarted with a modified `--features` list.\nSimilarly, the client process must be stopped and restarted without features.\n\n## Alternatives Considered\nOmitted\n\n## Security Considerations\nOmitted\n" }, { "path": "design/Implemented/general-progress-monitoring.md", "content": "# Plugin Progress Monitoring\n\nThis is intended as a replacement for the previously-approved Upload Progress Monitoring design\n([Upload Progress Monitoring](upload-progress.md)) in order to expand the supported use cases beyond\nsnapshot uploads to include what was previously called Async Backup/Restore Item Actions. This\nupdated design should handle the combined set of use cases for those previously separate designs.\n\nVolume snapshotter plugin are used by Velero to take snapshots of persistent volume contents. \nDepending on the underlying storage system, those snapshots may be available to use immediately, \nthey may be uploaded to stable storage internally by the plugin or they may need to be uploaded after\nthe snapshot has been taken. We would like for Velero to continue on to the next part of the backup as quickly\nas possible but we would also like the backup to not be marked as complete until it is a usable backup. We'd also\neventually like to bring the control of upload under the control of Velero and allow the user to make decisions\nabout the ultimate destination of backup data independent of the storage system they're using.\n\nWe would also like any internal or third party Backup or Restore Item Action to have the option of\nmaking use of this same ability to run some external process without blocking the current backup or\nrestore operation. Beyond Volume Snapshotters, this is also needed for data mover operations on both\nbackup and restore, and potentially useful for other third party operations -- for example\nin-cluster registry image backup or restore could make use of this feature in a third party plugin).\n\n### Glossary\n- BIA: BackupItemAction\n- RIA: RestoreItemAction\n\n## Examples\n- AWS - AWS snapshots return quickly, but are then uploaded in the background and cannot be used until EBS moves\nthe data into S3 internally.\n\n- vSphere - The vSphere plugin takes a local snapshot and then the vSphere plugin uploads the data to S3. The local\nsnapshot is usable before the upload completes.\n\n- Restic - Does not go through the volume snapshot path. Restic backups will block Velero progress\nuntil completed. However, with the more generalized approach in the revised design, restic/kopia\nbackup and restore *could* make use of this framework if their actions are refactored as\nBackup/RestoreItemActions.\n\n- Data Movers\n - Data movers are asynchronous processes executed inside backup/restore item actions that applies to a specific Kubernetes resources. A common use case for data mover is to backup/restore PVCs whose data we want to move to some form of backup storage outside of using velero kopia/restic implementations.\n - Workflow\n - User takes velero backup of PVC A\n - BIA plugin applies to PVCs with compatible storage driver\n - BIA plugin triggers data mover\n\t - Most common use case would be for the plugin action to create a new CR which would\n trigger an external controller action\n\t - Another possible use case would be for the plugin to run its own async action in a\n goroutine, although this would be less resilient to plugin container restarts.\n - BIA plugin returns\n - Velero backup process continues\n - Main velero backup process monitors running BIA threads via gRPC to determine if process is done and healthy\n\n\n## Primary changes from the original Upload Progress Monitoring design\n\nThe most fundamental change here is that rather than proposing a new special-purpose\nSnapshotItemAction, the existing BackupItemAction plugin will be modified to accommodate an optional\nsnapshot (or other item operation) ID return. The primary reasons for this change are as follows:\n\n1. The intended scope has moved beyond snapshot processing, so it makes sense to support\nasynchronous operations in other backup or restore item actions.\n\n2. We expect to have plugin API versioning implemented in Velero 1.10, making it feasible to\nimplement changes in the existing plugin APIs now.\n\n3. We will need this feature on both backup and restore, meaning that if we took the \"new plugin\ntype\" approach, we'd need two new plugin types.\n\n4. Other than the snapshot/operation ID return, the rest of the plugin processing is identical to\nBackup/RestoreItemActions. With separate plugin types, we'd have to repeat all of that logic\n(including dealing with additional items, etc.) twice.\n\nThe other major change is that we will be applying this to both backups and restores, although the\nVolume Snapshotter use case only needs this on backup. This means that everything we're doing around\nbackup phase and workflow will also need to be done for restore.\n\nThen there are various minor changes around terminology to make things more generic. Instead of\n\"snapshotID\", we'll have \"operationID\" (which for volume snapshotters will be a snapshot\nID).\n\n## Goals\n\n- Enable monitoring of backup/restore item action operations that continue after snapshotting and other operations have completed\n- Keep non-usable backups and restores (upload/persistence has not finished, etc.) from appearing as completed\n- Make use of plugin API versioning functionality to manage changes to Backup/RestoreItemAction interfaces\n- Enable vendors to plug their own data movers into velero using BIA/RIA plugins\n\n## Non-goals\n- Today, Velero is unable to recover from an in progress backup when the velero server crashes (pod is deleted). This has an impact on running asynchronous processes, but it’s not something we intend to solve in this design.\n\n## Models\n\n### Internal configuration and management\nIn this model, movement of the snapshot to stable storage is under the control of the snapshot\nplugin. Decisions about where and when the snapshot gets moved to stable storage are not\ndirectly controlled by Velero. This is the model for the current VolumeSnapshot plugins.\n\n### Velero controlled management\nIn this model, the snapshot is moved to external storage under the control of Velero. This\nenables Velero to move data between storage systems. This also allows backup partners to use\nVelero to snapshot data and then move the data into their backup repository.\n\n## Backup and Restore phases\n\nVelero currently has backup/restore phases \"InProgress\" and \"Completed\". A backup moves to the\nCompleted phase when all of the volume snapshots have completed and the Kubernetes metadata has been\nwritten into the object store. However, the actual data movement may be happening in the background\nafter the backup has been marked \"Completed\". The backup is not actually a stable backup until the\ndata has been persisted properly. In some cases (e.g. AWS) the backup cannot be restored from until\nthe snapshots have been persisted.\n\nOnce the snapshots have been taken, however, it is possible for additional backups or restores (as\nlong as they don't use not-yet-completed backups) to be made without interference. Waiting until\nall data has been moved before starting the next backup will slow the progress of the system without\nadding any actual benefit to the user.\n\nNew backup/restore phases, \"WaitingForPluginOperations\" and\n\"WaitingForPluginOperationsPartiallyFailed\" will be introduced. When a backup or restore has\nentered one of these phases, Velero is free to start another backup/restore. The backup/restore\nwill remain in the \"WaitingForPluginOperations\" phase until all BIA/RIA operations have completed\n(for example, for a volume snapshotter, until all data has been successfully moved to persistent\nstorage). The backup/restore will not fail once it reaches this phase, although an error return\nfrom a plugin could cause a backup or restore to move to \"PartiallyFailed\". If the backup is\ndeleted (cancelled), the plugins will attempt to delete the snapshots and stop the data movement -\nthis may not be possible with all storage systems.\n\nIn addition, for backups (but not restores), there will also be two additional phases, \"Finalizing\"\nand \"FinalizingPartiallyFailed\", which will handle any steps required after plugin operations have\nall completed. Initially, this will just include adding any required resources to the backup that\nmight have changed during asynchronous operation execution, although eventually other cleanup\nactions could be added to this phase.\n\n### State progression\n\n![image](AsyncActionFSM.png)\n### New\nWhen a backup/restore request is initially created, it is in the \"New\" phase. \n\nThe next state is either \"InProgress\" or \"FailedValidation\"\n\n### FailedValidation\nIf the backup/restore request is incorrectly formed, it goes to the \"FailedValidation\" phase and\nterminates\n\n### InProgress\nWhen work on the backup/restore begins, it moves to the \"InProgress\" phase. It remains in the\n\"InProgress\" phase until all pre/post execution hooks have been executed, all snapshots have been\ntaken and the Kubernetes metadata and backup/restore info is safely written to the object store\nplugin.\n\nIn the current implementation, Restic backups will move data during the \"InProgress\" phase. In the\nfuture, it may be possible to combine a snapshot with a Restic (or equivalent) backup which would\nallow for data movement to be handled in the \"WaitingForPluginOperations\" phase,\n\nThe next phase would be \"WaitingForPluginOperations\" for backups or restores which have unfinished\nasynchronous plugin operations and no errors so far, \"WaitingForPluginOperationsPartiallyFailed\" for\nbackups or restores which have unfinished asynchronous plugin operations at least one error,\n\"Completed\" for restores with no unfinished asynchronous plugin operations and no errors,\n\"PartiallyFailed\" for restores with no unfinished asynchronous plugin operations and at least one\nerror, \"Finalizing\" for backups with no unfinished asynchronous plugin operations and no errors,\n\"FinalizingPartiallyFailed\" for backups with no unfinished asynchronous plugin operations and at\nleast one error, or \"PartiallyFailed\". Backups/restores which would have a final phase of\n\"Completed\" or \"PartiallyFailed\" may move to the \"WaitingForPluginOperations\" or\n\"WaitingForPluginOperationsPartiallyFailed\" state. A backup/restore which will be marked \"Failed\"\nwill go directly to the \"Failed\" phase. Uploads may continue in the background for snapshots that\nwere taken by a \"Failed\" backup/restore, but no progress will not be monitored or updated. If there\nare any operations in progress when a backup is moved to the \"Failed\" phase (although with the\ncurrent workflow, that shouldn't happen), Cancel() should be called on these operations. When a\n\"Failed\" backup is deleted, all snapshots will be deleted and at that point any uploads still in\nprogress should be aborted.\n\n### WaitingForPluginOperations (new)\nThe \"WaitingForPluginOperations\" phase signifies that the main part of the backup/restore, including\nsnapshotting has completed successfully and uploading and any other asynchronous BIA/RIA plugin\noperations are continuing. In the event of an error during this phase, the phase will change to\nWaitingForPluginOperationsPartiallyFailed. On success, the phase changes to\n\"Finalizing\" for backups and \"Completed\" for restores. Backups cannot be\nrestored from when they are in the WaitingForPluginOperations state.\n\n### WaitingForPluginOperationsPartiallyFailed (new)\nThe \"WaitingForPluginOperationsPartiallyFailed\" phase signifies that the main part of the\nbackup/restore, including snapshotting has completed, but there were partial failures either during\nthe main part or during any async operations, including snapshot uploads. Backups cannot be\nrestored from when they are in the WaitingForPluginOperationsPartiallyFailed state.\n\n### Finalizing (new)\nThe \"Finalizing\" phase signifies that asynchronous backup operations have all completed successfully\nand Velero is currently backing up any resources indicated by asynchronous plugins as items to back\nup after operations complete. Once this is done, the phase changes to Completed. Backups cannot be\nrestored from when they are in the Finalizing state.\n\n### FinalizingPartiallyFailed (new)\n\nThe \"FinalizingPartiallyFailed\" phase signifies that, for a backup which had errors during initial\nprocessing or asynchronous plugin operation, asynchronous backup operations have all completed and\nVelero is currently backing up any resources indicated by asynchronous plugins as items to back up\nafter operations complete. Once this is done, the phase changes to PartiallyFailed. Backups cannot\nbe restored from when they are in the FinalizingPartiallyFailed state.\n\n### Failed\nWhen a backup/restore has had fatal errors it is marked as \"Failed\" Backups in this state cannot be\nrestored from.\n\n### Completed\nThe \"Completed\" phase signifies that the backup/restore has completed, all data has been transferred\nto stable storage (or restored to the cluster) and any backup in this state is ready to be used in a\nrestore. When the Completed phase has been reached for a backup it is safe to remove any of the\nitems that were backed up.\n\n### PartiallyFailed\nThe \"PartiallyFailed\" phase signifies that the backup/restore has completed and at least part of the\nbackup/restore is usable. Restoration from a PartiallyFailed backup will not result in a complete\nrestoration but pieces may be available.\n\n## Workflow\n\nWhen a Backup or Restore Action is executed, any BackupItemAction, RestoreItemAction, or\nVolumeSnapshot plugins will return operation IDs (snapshot IDs or other plugin-specific\nidentifiers). The plugin should be able to provide status on the progress for the snapshot and\nhandle cancellation of the operation/upload if the snapshot is deleted. If the plugin is restarted,\nthe operation ID should remain valid.\n\nWhen all snapshots have been taken and Kubernetes resources have been persisted to the ObjectStorePlugin\nthe backup will either have fatal errors or will be at least partially usable.\n\nIf the backup/restore has fatal errors it will move to the \"Failed\" state and finish. If a\nbackup/restore fails, the upload or other operation will not be cancelled but it will not be\nmonitored either. For backups in any phase, all snapshots will be deleted when the backup is\ndeleted. Plugins will cancel any data movement or other operations and remove snapshots and other\nassociated resources when the VolumeSnapshotter DeleteSnapshot method or DeleteItemAction Execute\nmethod is called.\n\nVelero will poll the plugins for status on the operations when the backup/restore exits the\n\"InProgress\" phase and has no fatal errors.\n\nIf any operations are not complete, the backup/restore will move to either WaitingForPluginOperations\nor WaitingForPluginOperationsPartiallyFailed or Failed.\n\nPost-snapshot and other operations may take a long time and Velero and its plugins may be restarted\nduring this time. Once a backup/restore has moved into the WaitingForPluginOperations or\nWaitingForPluginOperationsPartiallyFailed phase, another backup/restore may be started.\n\nWhile in the WaitingForPluginOperations or WaitingForPluginOperationsPartiallyFailed phase, the\nsnapshots and item actions will be periodically polled. When all of the snapshots and item actions\nhave reported success, restores will move directly to the Completed or PartiallyFailed phase, and\nbackups will move to the Finalizing or FinalizingPartiallyFailed phase, depending on whether the\nbackup/restore was in the WaitingForPluginOperations or WaitingForPluginOperationsPartiallyFailed\nphase.\n\nWhile in the Finalizing or FinalizingPartiallyFailed phase, Velero will update the backup with any\nresources indicated by plugins that they must be added to the backup after operations are completed,\nand then the backup will move to the Completed or PartiallyFailed phase, depending on whether there\nare any backup errors.\n\nThe Backup resources will be written to object storage at the time the backup leaves the InProgress\nphase, but it will not be synced to other clusters (or usable for restores in the current cluster)\nuntil the backup has entered a final phase: Completed, Failed or PartiallyFailed. During the\nFinalizing phases, a the backup resources will be updated with any required resources related to\nasynchronous plugins.\n\n## Reconciliation of InProgress backups\n\nInProgress backups will not have a `velero-backup.json` present in the object store. During\nreconciliation, backups which do not have a `velero-backup.json` object in the object store will be\nignored.\n\n## Plugin API changes\n\n### OperationProgress struct\n\n type OperationProgress struct {\n Completed bool // True when the operation has completed, either successfully or with a failure\n Err string // Set when the operation has failed\n NCompleted, NTotal int64 // Quantity completed so far and the total quantity associated with the operation in operationUnits\n // For data mover and volume snapshotter use cases, this would be in bytes\n // On successful completion, completed and total should be the same.\n OperationUnits string // Units represented by completed and total -- for data mover and item\n\t // snapshotters, this will usually be bytes.\n Description string // Optional description of operation progress\n Started, Updated time.Time // When the upload was started and when the last update was seen. Not all\n // systems retain when the upload was begun, return Time 0 (time.Unix(0, 0))\n // if unknown.\n }\n\n### VolumeSnapshotter changes\n\nTwo new methods will be added to the VolumeSnapshotter interface:\n\n Progress(snapshotID string) (OperationProgress, error)\n Cancel(snapshotID string) (error)\n\nProgress will report the current status of a snapshot upload. This should be callable at\nany time after the snapshot has been taken. In the event a plugin is restarted, if the operationID\n(snapshot ID) continues to be valid it should be possible to retrieve the progress.\n\n`error` is set if there is an issue retrieving progress. If the snapshot is has encountered an\nerror during the upload, the error should be returned in OperationProgress and error should be nil.\n\n### BackupItemAction and RestoreItemAction plugin changes\n\nCurrently CSI snapshots and the Velero Plugin for vSphere are implemented as BackupItemAction\nplugins. While the majority of BackupItemAction plugins do not take snapshots or upload data, this\nfunctionality is useful for any longstanding plugin operation managed by an external\nprocess/controller so we will modify BackupItemAction and RestoreItemAction to optionally return an\noperationID in addition to the modified item.\n\nVelero can attempt to cancel an operation by calling the Cancel API call on the BIA/RIA. The plugin\ncan then take any appropriate action as needed. Cancel will be called for unfinished operations on\nbackup deletion, and possibly reaching timeout. Cancel is not intended to be used to delete/remove\nthe results of completed actions and will have no effect on a completed action. Cancel has no return\nvalue apart from the standard Error return, but this should only be used for unexpected\nfailures. Under normal operations, Cancel will simply return a nil error (and nothing else), whether\nor not the plugin is able to cancel the operation.\n\n_AsyncOperationsNotSupportedError_ should only be returned (by Progress) if the\nBackup/RestoreItemAction plugin should not be handling the item. If the Backup/RestoreItemAction\nplugin should handle the item but, for example, the item/snapshot ID cannot be found to report\nprogress, Progress will return an InvalidOperationIDError error rather than a populated\nOperationProgress struct. If the item action does not start an asynchronous operation, then\noperationID will be empty.\n\nThree new methods will be added to the BackupItemAction interface, and the Execute() return signature\nwill be modified:\n\n\t// Name returns the name of this BIA. Plugins which implement this interface must defined Name,\n\t// but its content is unimportant, as it won't actually be called via RPC. Velero's plugin infrastructure\n\t// will implement this directly rather than delegating to the RPC plugin in order to return the name\n\t// that the plugin was registered under. The plugins must implement the method to complete the interface.\n\tName() string\n\t// Execute allows the BackupItemAction to perform arbitrary logic with the item being backed up,\n\t// including mutating the item itself prior to backup. The item (unmodified or modified)\n\t// should be returned, along with an optional slice of ResourceIdentifiers specifying\n\t// additional related items that should be backed up now, an optional operationID for actions which\n\t// initiate asynchronous actions, and a second slice of ResourceIdentifiers specifying related items\n\t// which should be backed up after all asynchronous operations have completed. This last field will be\n\t// ignored if operationID is empty, and should not be filled in unless the resource must be updated in the\n\t// backup after async operations complete (i.e. some of the item's Kubernetes metadata will be updated\n\t// during the asynch operation which will be required during restore)\n\tExecute(item runtime.Unstructured, backup *api.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error)\n \t\n \t// Progress \n\tProgress(operationID string, backup *api.Backup) (velero.OperationProgress, error)\n \t// Cancel \n\tCancel(operationID string, backup *api.Backup) error\n\nThree new methods will be added to the RestoreItemAction interface, and the\nRestoreItemActionExecuteOutput struct will be modified:\n\n\t// Name returns the name of this RIA. Plugins which implement this interface must defined Name,\n\t// but its content is unimportant, as it won't actually be called via RPC. Velero's plugin infrastructure\n\t// will implement this directly rather than delegating to the RPC plugin in order to return the name\n\t// that the plugin was registered under. The plugins must implement the method to complete the interface.\n\tName() string\n\t// Execute allows the ItemAction to perform arbitrary logic with the item being restored,\n\t// including mutating the item itself prior to restore. The item (unmodified or modified)\n\t// should be returned, an optional OperationID, along with an optional slice of ResourceIdentifiers\n\t// specifying additional related items that should be restored, a warning (which will be\n\t// logged but will not prevent the item from being restored) or error (which will be logged\n\t// and will prevent the item from being restored) if applicable. If OperationID is specified\n \t// then velero will wait for this operation to complete before the restore is marked Completed.\n\tExecute(input *RestoreItemActionExecuteInput) (*RestoreItemActionExecuteOutput, error)\n\n \t\n \t// Progress \n\tProgress(operationID string, restore *api.Restore) (velero.OperationProgress, error)\n\n // Cancel \n\tCancel(operationID string, restore *api.Restore) error\n \n // RestoreItemActionExecuteOutput contains the output variables for the ItemAction's Execution function.\n type RestoreItemActionExecuteOutput struct {\n // UpdatedItem is the item being restored mutated by ItemAction.\n UpdatedItem runtime.Unstructured\n\n // AdditionalItems is a list of additional related items that should\n // be restored.\n AdditionalItems []ResourceIdentifier\n\n // SkipRestore tells velero to stop executing further actions\n // on this item, and skip the restore step. When this field's\n // value is true, AdditionalItems will be ignored.\n SkipRestore bool\n\n // OperationID is an identifier which indicates an ongoing asynchronous action which Velero will\n // continue to monitor after restoring this item. If left blank, then there is no ongoing operation\n OperationID string\n }\n\n## Changes in Velero backup format\n\nNo changes to the existing format are introduced by this change. As part of the backup workflow changes, a\n`-itemoperations.json.gz` file will be added that contains the items and operation IDs\n(snapshotIDs) returned by VolumeSnapshotter and BackupItemAction plugins. Also, the creation of the\n`velero-backup.json` object will not occur until the backup moves to one of the terminal phases\n(_Completed_, _PartiallyFailed_, or _Failed_). Reconciliation should ignore backups that do not\nhave a `velero-backup.json` object.\n\nThe Backup/RestoreItemAction plugin identifier as well as the ItemID and OperationID will be stored\nin the `-itemoperations.json.gz`. When checking for progress, this info will be used\nto select the appropriate Backup/RestoreItemAction plugin to query for progress. Here's an example\nof what a record for a datamover plugin might look like:\n```\n {\n \"spec\": {\n \"backupName\": \"backup-1\",\n \"backupUID\": \"f8c72709-0f73-46e1-a071-116bc4a76b07\",\n \"backupItemAction\": \"velero.io/volumesnapshotcontent-backup\",\n \"resourceIdentifier\": {\n\t \"Group\": \"snapshot.storage.k8s.io\",\n \"Resource\": \"VolumeSnapshotContent\",\n \"Namespace\": \"my-app\",\n \"Name\": \"my-volume-vsc\"\n },\n \"operationID\": \"\",\n \"itemsToUpdate\": [\n\t {\n\t \"Group\": \"velero.io\",\n \"Resource\": \"VolumeSnapshotBackup\",\n \"Namespace\": \"my-app\",\n \"Name\": \"vsb-1\"\n }\n\t ]\n\t},\n \"status\": {\n \"operationPhase\": \"Completed\",\n \"error\": \"\",\n \"nCompleted\": 12345,\n \"nTotal\": 12345,\n \"operationUnits\": \"byte\",\n \"description\": \"\",\n \"Created\": \"2022-12-14T12:00:00Z\",\n \"Started\": \"2022-12-14T12:01:00Z\",\n \"Updated\": \"2022-12-14T12:11:02Z\"\n },\n }\n```\n\nThe cluster that is creating the backup will have the Backup resource present and will be able to\nmanage the backup before the backup completes.\n\nIf the Backup resource is removed (e.g. Velero is uninstalled) before a backup completes and writes\nits `velero-backup.json` object, the other objects in the object store for the backup will be\neffectively orphaned. This can currently happen but the current window is much smaller.\n\n### `-itemoperations.json.gz`\nThe itemoperations file is similar to the existing `-itemsnapshots.json.gz` Each snapshot taken via\nBackupItemAction will have a JSON record in the file. Exact format TBD.\n\nThis file will be uploaded to object storage at the end of processing all of the items in the\nbackup, before the phase moves away from `InProgress`.\n\n## Changes to Velero restores\n\nA `-itemoperations.json.gz` file will be added that contains the items and operation\nIDs returned by RestoreItemActions. The format will be the same as the\n`-itemoperations.json.gz` generated for backups.\n\nThis file will be uploaded to object storage at the end of processing all of the items in the\nrestore, before the phase moves away from `InProgress`.\n\n## CSI snapshots\n\nFor systems such as EBS, a snapshot is not available until the storage system has transferred the\nsnapshot to stable storage. CSI snapshots expose the _readyToUse_ state that, in the case of EBS,\nindicates that the snapshot has been transferred to durable storage and is ready to be used. The\nCSI BackupItemAction.Progress method will poll that field and when completed, return completion.\n\n## vSphere plugin\n\nThe vSphere Plugin for Velero uploads snapshots to S3 in the background. This is also a\nBackupItemAction plugin, it will check the status of the Upload records for the snapshot and return\nprogress.\n\n## Backup workflow changes\n\nThe backup workflow remains the same until we get to the point where the `velero-backup.json` object\nis written. At this point, Velero will\nrun across all of the VolumeSnapshotter/BackupItemAction operations and call the _Progress_ method\non each of them.\n\nIf all backup item operations have finished (either successfully or failed), the backup will move to\none of the finalize phases.\n\nIf any of the snapshots or backup items are still being processed, the phase of the backup will be\nset to the appropriate phase (_WaitingForPluginOperations_ or\n_WaitingForPluginOperationsPartiallyFailed_), and the async backup operations controller will\nreconcile periodically and call Progress on any unfinished operations. In the event of any of the\nprogress checks return an error, the phase will move to _WaitingForPluginOperationsPartiallyFailed_.\n\nOnce all operations have completed, the backup will be moved to one of the finalize phases, and the\nbackup finalizer controller will update the the `velero-backup.json`in the object store with any\nresources necessary after asynchronous operations are complete and the backup will move to the\nappropriate terminal phase.\n\n\n## Restore workflow changes\n\nThe restore workflow remains the same until velero would currently move the backup into one of the\nterminal states. At this point, Velero will run across all of the RestoreItemAction operations and\ncall the _Progress_ method on each of them.\n\nIf all restore item operations have finished (either successfully or failed), the restore will be\ncompleted and the restore will move to the appropriate terminal phase and the restore will be\ncomplete.\n\nIf any of the restore items are still being processed, the phase of the restore will be set to the\nappropriate phase (_WaitingForPluginOperations_ or _WaitingForPluginOperationsPartiallyFailed_), and\nthe async restore operations controller will reconcile periodically and call Progress on any\nunfinished operations. In the event of any of the progress checks return an error, the phase will\nmove to _WaitingForPluginOperationsPartiallyFailed_. Once all of the operations have completed, the\nrestore will be moved to the appropriate terminal phase.\n\n## Restart workflow\n\nOn restart, the Velero server will scan all Backup/Restore resources. Any Backup/Restore resources\nwhich are in the _InProgress_ phase will be moved to the _Failed_ phase. Any Backup/Restore\nresources in the _WaitingForPluginOperations_ or _WaitingForPluginOperationsPartiallyFailed_ phase\nwill be treated as if they have been requeued and progress checked and the backup/restore will be\nrequeued or moved to a terminal phase as appropriate.\n\n## Notes on already-merged code which may need updating\n\nSince this design is modifying a previously-approved design, there is some preparation work based on\nthe earlier upload progress monitoring design that may need modification as a result of these\nupdates. Here is a list of some of these items:\n\n1. Consts for the \"Uploading\" and \"UploadingPartiallyFailed\" phases have already been defined. These\n will need to be removed when the \"WaitingForPluginOperations\" and\n \"WaitingForPluginOperationsPartiallyFailed\" phases are defined.\n - https://github.com/vmware-tanzu/velero/pull/3805\n1. Remove the ItemSnapshotter plugin APIs (and related code) since the revised design will reuse\n VolumeSnapshotter and BackupItemAction plugins.\n - https://github.com/vmware-tanzu/velero/pull/4077\n - https://github.com/vmware-tanzu/velero/pull/4417\n1. UploadProgressFeatureFlag shouldn't be needed anymore. The current design won't really need a\n feature flag here -- the new features will be added to V2 of the VolumeSnapshotter,\n BackupItemAction, and RestoreItemAction plugins, and it will only be used if there are plugins which\n return operation IDs.\n - https://github.com/vmware-tanzu/velero/pull/4416\n1. Adds -itemsnapshots.gz file to backup (when provided) -- this is still part of the\n revised design, so it should stay.\n - https://github.com/vmware-tanzu/velero/pull/4429\n1. Upload Progress Monitoring and Item Snapshotter basic support: This PR is not yet merged, so\n nothing will need to be reverted. While the implementation here will be useful in informing the\n implementation of the new design, several things have changed in the design proposal since the PR\n was written.\n - https://github.com/vmware-tanzu/velero/pull/4467\n\n# Implementation tasks\n\nVolumeSnapshotter new plugin APIs \nBackupItemAction new plugin APIs \nRestoreItemAction new plugin APIs \nNew backup phases \nNew restore phases \nDefer uploading `velero-backup.json` \nAWS EBS plugin Progress implementation \nOperation monitoring \nImplementation of `-itemoperations.json.gz` file \nImplementation of `-itemoperations.json.gz` file\nRestart logic \nChange in reconciliation logic to ignore backups/restores that have not completed \nCSI plugin BackupItemAction Progress implementation \nvSphere plugin BackupItemAction Progress implementation (vSphere plugin team) \n\n\n# Open Questions\n\n1. Do we need a Cancel operation for VolumeSnapshotter?\n - From feedback, I'm thinking we probably don't need it. The only real purpose of Cancel\n here is to tell the plugin that Velero won't be waiting anymore, so if there are any\n required custom cancellation actions, now would be a good time to perform them. For snapshot\n uploads that are already in proress, there's not really anything else to cancel.\n2. Should we actually write the backup *before* moving to the WaitingForPluginOperations or\n WaitingForPluginOperationsPartiallyFailed phase rather than waiting until all operations\n have completed? The operations in question won't affect what gets written to object storage\n for the backup, and since we've already written the list of operations we're waiting for to\n object storage, writing the backup now would make the process resilient to Velero restart if\n it happens during WaitingForPluginOperations or WaitingForPluginOperationsPartiallyFailed\n\n" }, { "path": "design/Implemented/generating-velero-crds-with-structural-schema.md", "content": "# Generating Velero CRDs with structural schema support\n\nAs the apiextensions.k8s.io API moves to GA, structural schema in Custom Resource Definitions (CRDs) will become required.\n\nThis document proposes updating the CRD generation logic as part of `velero install` to include structural schema for each Velero CRD.\n\n## Goals\n\n- Enable structural schema and validation for Velero Custom Resources.\n\n## Non Goals\n\n- Update Velero codebase to use Kubebuilder for controller/code generation.\n- Solve for keeping CRDs in the Velero Helm chart up-to-date.\n\n## Background\n\nCurrently, Velero CRDs created by the `velero install` command do not contain any structural schema.\nThe CRD is simply [generated at runtime](https://github.com/heptio/velero/blob/8b0cf3855c2b8aa631cf22e63da0955f7b1d06a8/pkg/install/crd.go#L39) using the name and plurals from the [`velerov1api.CustomResources()`](https://github.com/heptio/velero/blob/8b0cf3855c2b8aa631cf22e63da0955f7b1d06a8/pkg/apis/velero/v1/register.go#L60) info.\n\nUpdating the info returned by that method would be one way to add support for structural schema when generating the CRDs, but this would require manually describing the schema and would duplicate information from the API structs (e.g. comments describing a field).\n\nInstead, the [controller-tools](https://github.com/kubernetes-sigs/controller-tools) project from Kubebuilder provides tooling for generating CRD manifests (YAML) from the Velero API types.\nThis document proposes adding _controller-tools_ to the project to automatically generate CRDs, and use these generated CRDs as part of `velero install`.\n\n## High-Level Design\n\n_controller-tools_ works by reading the Go files that contain the API type definitions.\nIt uses a combination of the struct fields, types, tags and comments to build the OpenAPIv3 schema for the CRDs. The tooling makes some assumptions based on conventions followed in upstream Kubernetes and the ecosystem, which involves some changes to the Velero API type definitions, especially around optional fields.\n\nIn order for _controller-tools_ to read the Go files containing Velero API type definitions, the CRDs need to be generated at build time, as these files are not available at runtime (i.e. the Go files are not accessible by the compiled binary).\nThese generated CRD manifests (YAML) will then need to be available to the `pkg/install` package for it to include when installing Velero resources.\n\n## Detailed Design\n\n### Changes to Velero API type definitions\n\nAPI type definitions need to be updated to correctly identify optional and required fields for each API type.\nUpstream Kubernetes defines all optional fields using the `omitempty` tag as well as a `// +optional` annotation above the field (e.g. see [PodSpec definition](https://github.com/kubernetes/api/blob/master/core/v1/types.go#L2835-L2838)).\n_controller-tools_ will mark a field as optional if it sees either the tag or the annotation, but to keep consistent with upstream, optional fields will be updated to use both indicators (as [suggested](https://github.com/kubernetes-sigs/kubebuilder/issues/479) by the Kubebuilder project).\nAdditionally, upstream Kubernetes defines the metav1.ObjectMeta, metav1.ListMeta, Spec and Status as [optional on all types](https://github.com/kubernetes/api/blob/master/core/v1/types.go#L3517-L3531).\nSome Velero API types set the `omitempty` tag on Status, but not on other fields - these will all need to be updated to be made optional.\n\nBelow is a list of the Velero API type fields and what changes (if any) will be made.\nNote that this only includes fields used in the spec, all status fields will become optional.\n\n| Type | Field | Changes |\n|---------------------------------|-------------------------|-------------------------------------------------------------|\n| BackupSpec | IncludedNamespaces | make optional |\n| | ExcludedNamespaces | make optional |\n| | IncludedResources | make optional |\n| | ExcludedResources | make optional |\n| | LabelSelector | make optional |\n| | SnapshotVolumes | make optional |\n| | TTL | make optional |\n| | IncludeClusterResources | make optional |\n| | Hooks | make optional |\n| | StorageLocation | make optional |\n| | VolumeSnapshotLocations | make optional |\n| BackupHooks | Resources | make optional |\n| BackupResourceHookSpec | Name | none (required) |\n| | IncludedNamespaces | make optional |\n| | ExcludedNamespaces | make optional |\n| | IncludedResources | make optional |\n| | ExcludedResources | make optional |\n| | LabelSelector | make optional |\n| | PreHooks | make optional |\n| | PostHooks | make optional |\n| BackupResourceHook | Exec | none (required) |\n| ExecHook | Container | make optional |\n| | Command | required, validation: MinItems=1 |\n| | OnError | make optional |\n| | Timeout | make optional |\n| HookErrorMode | | validation: Enum |\n| BackupStorageLocationSpec | Provider | none (required) |\n| | Config | make optional |\n| | StorageType | none (required) |\n| | AccessMode | make optional |\n| StorageType | ObjectStorage | make required |\n| ObjectStorageLocation | Bucket | none (required) |\n| | Prefix | make optional |\n| BackupStorageLocationAccessMode | | validation: Enum |\n| DeleteBackupRequestSpec | BackupName | none (required) |\n| DownloadRequestSpec | Target | none (required) |\n| DownloadTarget | Kind | none (required) |\n| | Name | none (required) |\n| DownloadTargetKind | | validation: Enum |\n| PodVolumeBackupSpec | Node | none (required) |\n| | Pod | none (required) |\n| | Volume | none (required) |\n| | BackupStorageLocation | none (required) |\n| | RepoIdentifier | none (required) |\n| | Tags | make optional |\n| PodVolumeRestoreSpec | Pod | none (required) |\n| | Volume | none (required) |\n| | BackupStorageLocation | none (required) |\n| | RepoIdentifier | none (required) |\n| | SnapshotID | none (required) |\n| ResticRepositorySpec | VolumeNamespace | none (required) |\n| | BackupStorageLocation | none (required) |\n| | ResticIdentifier | none (required) |\n| | MaintenanceFrequency | none (required) |\n| RestoreSpec | BackupName | none (required) - should be set to \"\" if using ScheduleName |\n| | ScheduleName | make optional |\n| | IncludedNamespaces | make optional |\n| | ExcludedNamespaces | make optional |\n| | IncludedResources | make optional |\n| | ExcludedResources | make optional |\n| | NamespaceMapping | make optional |\n| | LabelSelector | make optional |\n| | RestorePVs | make optional |\n| | IncludeClusterResources | make optional |\n| ScheduleSpec | Template | none (required) |\n| | Schedule | none (required) |\n| VolumeSnapshotLocationSpec | Provider | none (required) |\n| | Config | make optional |\n\n### Build-time generation of CRD manifests\n\nThe build image will be updated as follows to include the _controller-tool_ tooling:\n\n\n```diff\ndiff --git a/hack/build-image/Dockerfile b/hack/build-image/Dockerfile\nindex b69a8c8a..07eac9c6 100644\n--- a/hack/build-image/Dockerfile\n+++ b/hack/build-image/Dockerfile\n@@ -21,6 +21,8 @@ RUN mkdir -p /go/src/k8s.io && \\\n git clone -b kubernetes-1.15.3 https://github.com/kubernetes/apimachinery && \\\n # vendor code-generator go modules to be compatible with pre-1.15\n cd /go/src/k8s.io/code-generator && GO111MODULE=on go mod vendor && \\\n+ go get -d sigs.k8s.io/controller-tools/cmd/controller-gen && \\\n+ cd /go/src/sigs.k8s.io/controller-tools && GO111MODULE=on go mod vendor && \\\n go get golang.org/x/tools/cmd/goimports && \\\n cd /go/src/golang.org/x/tools && \\\n git checkout 40a48ad93fbe707101afb2099b738471f70594ec && \\\n```\n\nTo tie in the CRD manifest generation with existing scripts/workflows, the `hack/update-generated-crd-code.sh` script will be updated to use _controller-tools_ to generate CRDs manifests after it generates the client code.\n\nThe generated CRD manifests will be placed in the `pkg/generated/crds/manifests` folder.\nSimilarly to client code generation, these manifests will be checked-in to the git repo.\nChecking in these manifests allows including documentation and schema changes to API types as part of code review.\n\n### Updating `velero install` to include generated CRD manifests\n\nAs described above, CRD generation using _controller-tools_ will happen at build time due to need to inspect Go files.\nTo enable the `velero install` to access the generated CRD manifests at runtime, the `pkg/generated/crds/manifests` folder will be embedded as binary data in the Velero binary (e.g. using a tool like [vfsgen](https://github.com/shurcooL/vfsgen) - see [POC branch](https://github.com/prydonius/velero/commit/4aa7413f97ce9b23e071b6054f600dd0c283351e)).\n\n`velero install` will then unmarshal the binary data as `unstructured.Unstructured` types and append them to the [resources list](https://github.com/heptio/velero/blob/8b0cf3855c2b8aa631cf22e63da0955f7b1d06a8/pkg/install/resources.go#L217) in place of the existing CRD generation.\n\n## Alternatives Considered\n\nInstead of generating and bundling CRD manifests, it could be possible to instead embed the `pkg/apis` package in the Velero binary.\nWith this, _controller-tools_ could be run at runtime during `velero install` to generate the CRD manifests.\nHowever, this would require including _controller-tools_ as a dependency in the project, which might not be desirable as it is a developer tool.\n\nAnother option, to avoid embedding static files in the binary, would be to generate the CRD manifest as one YAML file in CI and upload it as a release artifact (e.g. using GitHub releases).\n`velero install` could then download this file for the current version and use it on install.\nThe downside here is that `velero install` becomes dependent on the GitHub network, and we lose visibility on changes to the CRD manifests in the Git history.\n\n## Security Considerations\n\nn/a\n" }, { "path": "design/Implemented/handle-backup-of-volumes-by-resources-filters.md", "content": "# Handle backup of volumes by resources filters\n \n## Abstract\nCurrently, Velero doesn't have one flexible way to handle volumes.\n \nIf users want to skip the backup of volumes or only backup some volumes in different namespaces in batch, currently they need to use the opt-in and opt-out approach one by one, or use label-selector but if it has big different labels on each different related pod, which is cumbersome when they have lots of volumes to handle with. it would be convenient if Velero could provide policies to handle the backup of volumes just by `some specific volumes conditions`.\n \n## Background\nAs of Today, Velero has lots of filters to handle (backup or skip backup) resources including resources filters like `IncludedNamespaces, ExcludedNamespaces`, label selectors like `LabelSelector, OrLabelSelectors`, annotation like `backup.velero.io/must-include-additional-items` etc. But it's not enough flexible to handle volumes, we need one generic way to handle volumes.\n \n## Goals\n- Introducing flexible policies to handle volumes, and do not patch any labels or annotations to the pods or volumes.\n \n## Non-Goals\n- We only handle volumes for backup and do not support restore.\n- Currently, only handles volumes, and does not support other resources.\n- Only environment-unrelated and platform-independent general volumes attributes are supported, do not support volumes attributes related to a specific environment.\n \n## Use-cases/Scenarios\n### Skip backup volumes by some attributes\nUsers want to skip PV with the requirements:\n- option to skip all PV data\n- option to skip specified PV type (RBD, NFS)\n- option to skip specified PV size\n- option to skip specified storage-class\n \n## High-Level Design\nFirst, Velero will provide the user with one YAML file template and all supported volume policies will be in.\n\nSecond, writing your own configuration file by imitating the YAML template, it could be partial volume policies from the template.\n\nThird, create one configmap from your own configuration file, and the configmap should be in Velero install namespace.\n\nFourth, create a backup with the command `velero backup create --resource-policies-configmap $policiesConfigmap`, which will reference the current backup to your volume policies. At the same time, Velero will validate all volume policies user imported, the backup will fail if the volume policies are not supported or some items could not be parsed.\n\nFifth, the current backup CR will record the reference of volume policies configmap.\n\nSixth, Velero first filters volumes by other current supported filters, at last, it will apply the volume policies to the filtered volumes to get the final matched volume to handle.\n \n## Detailed Design\nThe volume resources policies should contain a list of policies which is the combination of conditions and related `action`, when target volumes meet the conditions, the related `action` will take effection.\n\nBelow is the API Design for the user configuration:\n\n### API Design\n```go\ntype VolumeActionType string\n\nconst Skip VolumeActionType = \"skip\"\n\n// Action defined as one action for a specific way of backup\ntype Action struct {\n\t// Type defined specific type of action, it could be 'file-system-backup', 'volume-snapshot', or 'skip' currently\n\tType VolumeActionType `yaml:\"type\"`\n\t// Parameters defined map of parameters when executing a specific action\n\t// +optional\n\t// +nullable\n\tParameters map[string]interface{} `yaml:\"parameters,omitempty\"`\n}\n\n// VolumePolicy defined policy to conditions to match Volumes and related action to handle matched Volumes\ntype VolumePolicy struct {\n\t// Conditions defined list of conditions to match Volumes\n\tConditions map[string]interface{} `yaml:\"conditions\"`\n\tAction Action `yaml:\"action\"`\n}\n\n// ResourcePolicies currently defined slice of volume policies to handle backup\ntype ResourcePolicies struct {\n Version string `yaml:\"version\"`\n\tVolumePolicies []VolumePolicy `yaml:\"volumePolicies\"`\n\t// we may support other resource policies in the future, and they could be added separately\n \t// OtherResourcePolicies: []OtherResourcePolicy\n}\n```\n\nThe policies YAML config file would look like this:\n```yaml\nversion: v1\nvolumePolicies:\n# it's a list and if the input item matches the first policy, the latters will be ignored\n# each policy consists of a list of conditions and an action\n\n# each key in the object is one condition, and one policy will apply to resources that meet ALL conditions\n- conditions:\n # capacity condition matches the volumes whose capacity falls into the range\n capacity: \"0,100Gi\"\n csi:\n driver: ebs.csi.aws.com\n fsType: ext4\n storageClass:\n - gp2\n - ebs-sc\n action:\n type: volume-snapshot\n parameters:\n # optional parameters which are custom-defined parameters when doing an action\n volume-snapshot-timeout: \"6h\"\n- conditions:\n capacity: \"0,100Gi\"\n storageClass:\n - gp2\n - ebs-sc\n action:\n type: file-system-backup\n- conditions:\n nfs:\n server: 192.168.200.90\n action:\n # type of file-system-backup could be defined a second time\n type: file-system-backup\n- conditions:\n nfs: {}\n action:\n type: skip\n- conditions:\n csi:\n driver: aws.efs.csi.driver\n action:\n type: skip\n```\n \n### Filter rules\n#### VolumePolicies\nThe whole resource policies consist of groups of volume policies.\n\nFor one specific volume policy which is a combination of one action and serval conditions. which means one action and serval conditions are the smallest unit of volume policy.\n\nVolume policies are a list and if the target volumes match the first policy, the latter will be ignored, which would reduce the complexity of matching volumes especially when there are multiple complex volumes policies.\n\n#### Action\n`Action` defined one action for a specific way of backup:\n - if choosing `Kopia` or `Restic`, the action value would be `file-system-backup`.\n - if choosing volume snapshot, the action value would be `volume-snapshot`.\n - if choosing skip backup of volume, the action value would be `skip`, and it will skip backup of volume no matter is `file-system-backup` or `volume-snapshot`.\n\nThe policies could be extended for later other ways of backup, which means it may have some other `Action` value that will be assigned in the future.\n\nBoth `file-system-backup` `volume-snapshot`, and `skip` could be partially or fully configured in the YAML file. And configuration could take effect only for the related action.\n\n#### Conditions\nThe conditions are serials of volume attributes, the matched Volumes should meet all the volume attributes in one conditions configuration.\n\n##### Supported conditions\nIn Velero 1.11, we want to support the volume attributes listed below:\n- capacity: matching volumes have the capacity that falls within this `capacity` range.\n- storageClass: matching volumes those with specified `storageClass`, such as `gp2`, `ebs-sc` in eks.\n- matching volumes that used specified volume sources.\n##### Parameters\nParameters are optional for one specific action. For example, it could be `csi-snapshot-timeout: 6h` for CSI snapshot.\n\n#### Special rule definitions:\n- One single condition in `Conditions` with a specific key and empty value, which means the value matches any value. For example, if the `conditions.nfs` is `{}`, it means if `NFS` is used as `persistentVolumeSource` in Persistent Volume will be skipped no matter what the NFS server or NFS Path is.\n \n- The size of each single filter value should limit to 256 bytes in case of an unfriendly long variable assignment.\n \n- For capacity for PV or size for Volume, the value should include the lower value and upper value concatenated by commas. And it has several combinations below:\n - \"0,5Gi\" or \"0Gi,5Gi\" which means capacity or size matches from 0 to 5Gi, including value 0 and value 5Gi\n - \",5Gi\" which is equal to \"0,5Gi\"\n - \"5Gi,\" which means capacity or size matches larger than 5Gi, including value 5Gi\n - \"5Gi\" which is not supported and will be failed in validating configuration.\n \n### Configmap Reference\nCurrently, resources policies are defined in `BackupSpec` struct, it will be more and more bloated with adding more and more filters which makes the size of `Backup` CR bigger and bigger, so we want to store the resources policies in configmap, and `Backup` CRD reference to current configmap.\n\nthe `configmap` user created would be like this:\n```yaml\napiVersion: v1\ndata:\n policies.yaml:\n ----\n version: v1\n volumePolicies:\n - conditions:\n capacity: \"0,100Gi\"\n csi:\n driver: ebs.csi.aws.com\n fsType: ext4\n storageClass:\n - gp2\n - ebs-sc\n action:\n type: volume-snapshot\n parameters:\n volume-snapshot-timeout: \"6h\"\nkind: ConfigMap\nmetadata:\n creationTimestamp: \"2023-01-16T14:08:12Z\"\n name: backup01\n namespace: velero\n resourceVersion: \"17891025\"\n uid: b73e7f76-fc9e-4e72-8e2e-79db717fe9f1\n```\n\nA new variable `resourcePolices` would be added into `BackupSpec`, it's value is assigned with the current resources policy configmap\n```yaml\napiVersion: velero.io/v1\nkind: Backup\nmetadata:\n name: backup-1\n spec:\n resourcePolices:\n refType: Configmap\n ref: backup01\n ...\n```\nThe configmap only stores those assigned values, not the whole resources policies.\n \nThe name of the configmap is `$BackupName`, and it's in Velero install namespace.\n \n#### Resource policies configmap related\nThe life cycle of resource policies configmap is managed by the user instead of Velero, which could make it more flexible and easy to maintain.\n- The resource policies configmap will remain in the cluster until the user deletes it.\n- Unlike backup, the resource policies configmap will not sync to the new cluster. So if the user wants to use one resource policies that do not sync to the new cluster, the backup will fail with resource policies not found.\n- One resource policies configmap could be used by multiple backups.\n- If the backup referenced resource policies configmap is been deleted, it won't affect the already existing backups, but if the user wants to reference the deleted configmap to create one new backup, it will fail with resource policies not found.\n\n#### Versioning\nWe want to introduce the version field in the YAML data to contain break changes. Therefore, we won't follow a semver paradigm, for example in v1.11 the data look like this:\n```yaml\nversion: v1\nvolumePolicies:\n ....\n```\nHypothetically, in v1.12 we add new fields like clusterResourcePolicies, the version will remain as v1 b/c this change is backward compatible:\n```yaml\nversion: v1\nvolumePolicies:\n ....\nclusterResourcePolicies:\n ....\n```\nSuppose in v1.13, we have to introduce a break change, at this time we will bump up the version:\n```yaml\nversion: v2\n# This is just an example, we should try to avoid break change\nvolume-policies: \n ....\n```\nWe only support one version in Velero, so it won't be recognized if backup using a former version of YAML data.\n\n#### Multiple versions supporting\nTo manage the effort for maintenance, we will only support one version of the data in Velero. Suppose that there is one break change for the YAML data in Velero v1.13, we should bump up the config version to v2, and v2 is only supported in v1.13. For the existing data with version: v1, it should migrate them when the Velero startup, this won't hurt the existing backup schedule CR as it only references the configmap. To make the migration easier, the configmap for such resource filter policies should be labeled manually before Velero startup like this, Velero will migrate the labeled configmap.\n\nWe only support migrating from the previous version to the current version in case of complexity in data format conversion, which users could regenerate configmap in the new YAML data version, and it is easier to do version control.\n\n```yaml\napiVersion: v1\nkind: ConfigMap\nmetadata:\n labels:\n# This label can be optional but if this is not set, the backup will fail after the breaking change and the user will need to update the data manually \n velero.io/resource-filter-policies: \"true\"\n name: example\n namespace: velero\ndata:\n .....\n```\n### Display of resources policies\nAs the resource policies configmap is referenced by backup CR, the policies in configmap are not so intuitive, so we need to integrate policies in configmap to the output of the command `velero backup describe`, and make it more readable.\n \n## Compatibility\nCurrently, we have these resources filters:\n- IncludedNamespaces\n- ExcludedNamespaces\n- IncludedResources\n- ExcludedResources\n- LabelSelector\n- OrLabelSelectors\n- IncludeClusterResources\n- UseVolumeSnapshots\n- velero.io/exclude-from-backup=true\n- backup.velero.io/backup-volumes-excludes\n- backup.velero.io/backup-volumes\n- backup.velero.io/must-include-additional-items\n \nSo it should be careful with the combination of volumes resources policies and the above resources filters.\n- When volume resource policies conflict with the above resource filters, we should respect the above resource filters. For example, if the user used the opt-out approach to `backup.velero.io/backup-volumes-excludes` annotation on the pod and also defined include volume in volumes resources filters configuration, we should respect the opt-out approach to skip backup of the volume.\n- If volume resource policies conflict with themselves, the first matched policy will be respect.\n\n## Implementation\nThis implementation should be included in Velero v1.11.0\n\nCurrently, in Velero v1.11.0 we only support `Action`\n `skip`, and support `file-system-backup` and `volume-snapshot` for the later version. And `Parameters` in `Action` is also not supported in v1.11.0, we will support in a later version.\n\nIn Velero 1.11, we supported Conditions and format listed below:\n - capacity\n ```yaml\n capacity: \"10Gi,100Gi\" // match volume has the size between 10Gi and 100Gi\n ```\n - storageClass\n ```yaml\n storageClass: // match volume has the storage class gp2 or ebs-sc\n - gp2\n - ebs-sc\n ```\n- volume sources (currently only support below format and attributes)\n 1. Specify the volume source name, the name could be `nfs`, `rbd`, `iscsi`, `csi` etc.\n ```yaml\n nfs : {} // match any volume has nfs volume source\n \n csi : {} // match any volume has csi volume source\n ```\n\n 2. Specify details for the related volume source (currently we only support csi driver filter and nfs server or path filter)\n ```yaml\n csi: // match volume has nfs volume source and using `aws.efs.csi.driver`\n driver: aws.efs.csi.driver \n \n nfs: // match volume has nfs volume source and using below server and path\n server: 192.168.200.90\n path: /mnt/nfs\n ```\nThe conditions also could be extended in later versions, such as we could further supporting filtering other volume source detail not only NFS and CSI.\n\n## Alternatives Considered\n### Configmap VS CRD\nHere we support the user define the YAML config file and storing the resources policies into configmap, also we could define one resource's policies CRD and store policies imported from the user-defined config file in the related CR.\n\nBut CRD is more like one kind of resource with status, Kubernetes API Server handles the lifecycle of a CR and handles it in different statuses. Compared to CRD, Configmap is more focused to store data.\n\n## Open Issues\nShould we support more than one version of filter policies configmap?" }, { "path": "design/Implemented/include-exclude-in-resource-policy.md", "content": "# Proposal to add include exclude policy to resource policy\n\nThis enhancement will allow the user to set include and exclude filters for resources in a resource policy configmap, so that\nthese filters are reusable and the user will not need to set them each time they create a backup.\n\n## Background\nAs mentioned in issue [#8610](https://github.com/vmware-tanzu/velero/issues/8610). When there's a long list of resources \nto include or exclude in a backup, it can be cumbersome to set them each time a backup is created. There's a requirement to\nset these filters in a separate data structure so that they can be reused in multiple backups.\n\n## High-Level Design\nWe may extend the data structure of resource policy to add `includeExcludePolicy`, which include the include and exclude filters \nin the BackupSpec. When the user creates a backup which references the resource policy config `velero backup create --resource-policies-configmap `,\nthe filters in \"includeExcludePolicy\" will take effect to filter the resources when velero collects the resources to backup.\n\n## Detailed Design\n\n### Data Structure\nThe map `includeExcludePolicy` contains four fields `includedClusterScopedResources`, `excludedClusterScopedResources`, \n`includedNamespaceScopedResources`,`excludedNamespaceScopedResources`. These filters work exactly as the filters defined BackupSpec with\nthe same names. An example of the policy looks like:\n```yaml\n#omitted other irrelevant fields like 'version', 'volumePolicies'\nincludeExcludePolicy:\n includedClusterScopedResources:\n - \"cr\"\n - \"crd\"\n - \"pv\"\n excludedClusterScopedResources:\n - \"volumegroupsnapshotclass\"\n - \"ingressclass\"\n includedNamespaceScopedResources:\n - \"pod\"\n - \"service\"\n - \"deployment\"\n - \"pvc\"\n excludedNamespaceScopedResources:\n - \"configmap\"\n```\nThese filters are in the form of scoped include/exclude filters, which by design will not work with the \"old\" resource filters.\nTherefore, when a Backup references a resource policy configmap which has `includeExcludePolicy`, and at the same time it has \nthe \"old\" resource filters, i.e. `includedResources`, `excludedResources`, `includeClusterResources` set in the BackupSpec, the\nBackup will fail with a validation error.\n\n### Priorities \nA user may set the include/exclude filters in Backupspec and also in the resource policy configmap. In this case, the filters \nin both the Backupspec and the resource policy configmap will take effect. When there's a conflict, the filters in the Backupspec \nwill take precedence. For example, if resource X is in the list of `includedNamespaceScopedResources` filter in the Backupspec, but \nit's also in the list of `excludedClusterScopedResources` in the resource policy configmap, then resource X will be included in the backup.\nIn this way, users can set the filters in the resource policy configmap to cover most of their use cases, and then override them \nin the Backupspec when needed.\n\n### Implementation\nIn addition to the data structure change, we will need to implement the following changes:\n1. A new function `CombineWithPolicy` will be added to the struct `ScopeIncludesExcludes`, which will combine the include/exclude filters\nin the resource policy configmap with the include/exclude filters in the Backupspec: \n```go\nfunc (ie *ScopeIncludesExcludes) CombineWithPolicy(policy resourcepolicies.IncludeExcludePolicy) {\n\tmapFunc := scopeResourceMapFunc(ie.helper)\n\tfor _, item := range policy.ExcludedNamespaceScopedResources {\n\t\tresolvedItem := mapFunc(item, true)\n\t\tif resolvedItem == \"\" {\n\t\t\tcontinue\n\t\t}\n\t\tif !ie.ShouldInclude(resolvedItem) && !ie.ShouldExclude(resolvedItem) {\n\t\t\t// The existing includeExcludes in the struct has higher priority, therefore, we should only add the item to the filter\n\t\t\t// when the struct does not include this item and this item is not yet in the excludes filter.\n\t\t\tie.namespaceScopedResourceFilter.excludes.Insert(resolvedItem)\n\t\t}\n\t\t\n\t}\n.....\n```\nThis function will be called in the `kubernetesBackupper.BackupWithResolvers` function, to make sure the combined `ScopeIncludesExcludes` \nfilter will be assigned to the `ResourceIncludesExcludes` filter of the Backup request.\n\n2. Extra validation code will be added to the function `prepareBackupRequest` of `BackupReconciler` to check if there are \"old\"\nResource filters in the BackupSpec when the Backup references a resource policy configmap which has `includeExcludePolicy`.\n\n## Alternatives Considered\nWe may put `includeExcludePolicy` in a separate configmap, but it will require adding extra field to BackupSpec to reference the configmap,\nwhich is not necessary.\n" }, { "path": "design/Implemented/json-substitution-action-design.md", "content": "# Proposal to add support for Resource Modifiers (AKA JSON Substitutions) in Restore Workflow\n\n- [Proposal to add support for Resource Modifiers (AKA JSON Substitutions) in Restore Workflow](#proposal-to-add-support-for-resource-modifiers-aka-json-substitutions-in-restore-workflow)\n - [Abstract](#abstract)\n - [Goals](#goals)\n - [Non Goals](#non-goals)\n - [User Stories](#user-stories)\n - [Scenario 1](#scenario-1)\n - [Scenario 2](#scenario-2)\n - [Detailed Design](#detailed-design)\n - [Reference in velero API](#reference-in-velero-api)\n - [ConfigMap Structure](#configmap-structure)\n - [Operations supported by the JSON Patch library:](#operations-supported-by-the-json-patch-library)\n - [Advance scenarios](#advance-scenarios)\n - [Conditional patches using test operation](#conditional-patches-using-test-operation)\n - [Alternatives Considered](#alternatives-considered)\n - [Security Considerations](#security-considerations)\n - [Compatibility](#compatibility)\n - [Implementation](#implementation)\n - [Future Enhancements](#future-enhancements)\n - [Open Issues](#open-issues)\n\n## Abstract\nCurrently velero supports substituting certain values in the K8s resources during restoration like changing the namespace, changing the storage class, etc. This proposal is to add generic support for JSON substitutions in the restore workflow. This will allow the user specify filters for particular resources and then specify a JSON patch (operator, path, value) to apply on a resource. This will allow the user to substitute any value in the K8s resource without having to write a new RestoreItemAction plugin for each kind of substitution.\n\n\n\n## Goals\n- Allow the user to specify a GroupResource, Name(optional), JSON patch for modification.\n- Allow the user to specify multiple JSON patch.\n\n## Non Goals\n- Deprecating the existing RestoreItemAction plugins for standard substitutions(like changing the namespace, changing the storage class, etc.)\n\n## User Stories\n\n### Scenario 1\n- Alice has a PVC which is encrypted using a DES(Disk Encryption Set - Azure example) mentioned in the PVC YAML through the StorageClass YAML. \n- Alice wishes to restore this snapshot to a different cluster. The new cluster does not have access to the same DES to provision disk's out of the snapshot.\n- She wishes to use a different DES for all the PVCs which use the certain DES.\n- She can use this feature to substitute the DES in all StorageClass YAMLs with the new DES without having to create a fresh storageclass, or understanding the name of the storageclass.\n\n### Scenario 2\n- Bob has multi zone cluster where nodes are spread across zones. \n- Bob has pinned certain pods to a particular zone using nodeSelector/ nodeaffinity on the pod spec.\n- In case of zone outage of the cloudprovider, Bob wishes to restore the workload to a different namespace in the same cluster, but change the zone pinning of the workload.\n- Bob can use this feature to substitute the nodeSelector/ nodeaffinity in the pod spec with the new zone pinning to quickly failover the workload to a different zone's nodes.\n\n## Detailed Design\n- The design and approach is inspired from [kubectl patch command](https://github.com/kubernetes/kubectl/blob/0a61782351a027411b8b45b1443ec3dceddef421/pkg/cmd/patch/patch.go#L102C2-L104C1)\n```bash\n# Update a container's image using a json patch with positional arrays\nkubectl patch pod valid-pod -type='json' -p='[{\"op\": \"replace\", \"path\": \"/spec/containers/0/image\", \"value\":\"new image\"}]'\n```\n- The user is expected to create a configmap with the desired Resource Modifications. Then the reference of the configmap will be provided in the RestoreSpec.\n- The core restore workflow before creating/updating a particular resource in the cluster will be checked against the filters provided and respective substitutions will be applied on it.\n\n### Reference in velero API \n> Example of Reference to configmap in RestoreSpec\n```yaml\napiVersion: velero.io/v1\nkind: Restore\nmetadata:\n name: restore-1\nspec:\n resourceModifier:\n refType: Configmap\n ref: resourcemodifierconfigmap\n```\n> Example CLI Command\n```bash\nvelero restore create --from-backup backup-1 --resource-modifier-configmap resourcemodifierconfigmap\n```\n\n### Resource Modifier ConfigMap Structure\n- User first needs to provide details on which resources the JSON Substitutions need to be applied. \n - For this the user will provide 4 inputs - Namespaces(for NS Scoped resources), GroupResource (resource.group format similar to includeResources field in velero) and Name Regex(optional).\n - If the user does not provide the Name, the JSON Substitutions will be applied to all the resources of the given Group and Kind under the given namespaces.\n\n- Further the use will specify the JSON Patch using the structure of kubectl's \"JSON Patch\" based inputs.\n- Sample data in ConfigMap\n```yaml\nversion: v1\nresourceModifierRules:\n- conditions:\n groupResource: persistentvolumeclaims\n resourceNameRegex: \"mysql.*\"\n namespaces:\n - bar\n - foo\n patches:\n - operation: replace\n path: \"/spec/storageClassName\"\n value: \"premium\"\n - operation: remove\n path: \"/metadata/labels/test\"\n```\n- The above configmap will apply the JSON Patch to all the PVCs in the namespaces bar and foo with name starting with mysql. The JSON Patch will replace the storageClassName with \"premium\" and remove the label \"test\" from the PVCs.\n- Note that the Namespace here is the original namespace of the backed up resource, not the new namespace where the resource is going to be restored.\n- The user can specify multiple JSON Patches for a particular resource. The patches will be applied in the order specified in the configmap. A subsequent patch is applied in order and if multiple patches are specified for the same path, the last patch will override the previous patches.\n- The user can specify multiple resourceModifierRules in the configmap. The rules will be applied in the order specified in the configmap. \n\n> Users need to create one configmap in Velero install namespace from a YAML file that defined resource modifiers. The creating command would be like the below:\n```bash\nkubectl create cm --from-file -n velero\n```\n\n### Operations supported by the JSON Patch library: \n- add\n- remove\n- replace\n- move\n- copy\n- test (covered below)\n\n### Advance scenarios\n#### **Conditional patches using test operation**\n The `test` operation can be used to check if a particular value is present in the resource. If the value is present, the patch will be applied. If the value is not present, the patch will not be applied. This can be used to apply a patch only if a particular value is present in the resource. For example, if the user wishes to change the storage class of a PVC only if the PVC is using a particular storage class, the user can use the following configmap.\n```yaml\nversion: v1\nresourceModifierRules:\n- conditions:\n groupResource: persistentvolumeclaims.storage.k8s.io\n resourceNameRegex: \".*\"\n namespaces:\n - bar\n - foo\n patches:\n - operation: test\n path: \"/spec/storageClassName\"\n value: \"premium\"\n - operation: replace\n path: \"/spec/storageClassName\"\n value: \"standard\"\n```\n\n## Alternatives Considered\n1. JSON Path based addressal of json fields in the resource\n - This was the initial planned approach, but there is no open source library which gives satisfactory edit functionality with support for all operators supported by the JsonPath RFC.\n - We attempted modifying the [https://kubernetes.io/docs/reference/kubectl/jsonpath/](https://kubernetes.io/docs/reference/kubectl/jsonpath/) but given the complexity of the code it did not make sense to change it since it would become a long term maintainability problem.\n1. RestoreItemAction for each kind of standard substitution\n - Not an extensible design. If a new kind of substitution is required, a new RestoreItemAction needs to be written.\n1. RIA for JSON Substitution: The approach of doing JSON Substitution through a RestoreItemAction plugin was considered. But it is likely to have performance implications as the plugin will be invoked for all the resources.\n\n## Security Considerations\nNo security impact.\n\n## Compatibility\nCompatibility with existing StorageClass mapping RestoreItemAction and similar plugins needs to be evaluated. \n\n## Implementation\n- Changes in Restore CRD. Add a new field to the RestoreSpec to reference the configmap.\n- One example of where code will be modified: https://github.com/vmware-tanzu/velero/blob/eeee4e06d209df7f08bfabda326b27aaf0054759/pkg/restore/restore.go#L1266 On the obj before Creation, we can apply the conditions to check if the resource is filtered out using given parameters. Then using JsonPatch provided, we can update the resource.\n- For Jsonpatch - https://github.com/evanphx/json-patch library is used.\n- JSON Patch RFC https://datatracker.ietf.org/doc/html/rfc6902\n\n## Future enhancements\n- Additional features such as wildcard support in path, regex match support in value, etc. can be added in future. This would involve forking the https://github.com/evanphx/json-patch library and adding the required features, since those features are not supported by the library currently and are not part of jsonpatch RFC.\n\n## Open Issues\nNA\n" }, { "path": "design/Implemented/merge-patch-and-strategic-in-resource-modifier.md", "content": "# Proposal to Support JSON Merge Patch and Strategic Merge Patch in Resource Modifiers\n\n- [Proposal to Support JSON Merge Patch and Strategic Merge Patch in Resource Modifiers](#proposal-to-support-json-merge-patch-and-strategic-merge-patch-in-resource-modifiers)\n - [Abstract](#abstract)\n - [Goals](#goals)\n - [Non Goals](#non-goals)\n - [User Stories](#user-stories)\n - [Scenario 1](#scenario-1)\n - [Scenario 2](#scenario-2)\n - [Detailed Design](#detailed-design)\n - [How to choose the right patch type](#how-to-choose-the-right-patch-type)\n - [New Field MergePatches](#new-field-mergepatches)\n - [New Field StrategicPatches](#new-field-strategicpatches)\n - [Conditional Patches in ALL Patch Types](#conditional-patches-in-all-patch-types)\n - [Wildcard Support for GroupResource](#wildcard-support-for-groupresource)\n - [Helper Command to Generate Merge Patch and Strategic Merge Patch](#helper-command-to-generate-merge-patch-and-strategic-merge-patch)\n - [Security Considerations](#security-considerations)\n - [Compatibility](#compatibility)\n - [Implementation](#implementation)\n - [Future Enhancements](#future-enhancements)\n - [Open Issues](#open-issues)\n\n## Abstract\nVelero introduced the concept of Resource Modifiers in v1.12.0. This feature allows the user to specify a configmap with a set of rules to modify the resources during restore. The user can specify the filters to select the resources and then specify the JSON Patch to apply on the resource. This feature is currently limited to the operations supported by JSON Patch RFC.\nThis proposal is to add support for JSON Merge Patch and Strategic Merge Patch in the Resource Modifiers. This will allow the user to use the same configmap to apply JSON Merge Patch and Strategic Merge Patch on the resources during restore.\n\n## Goals\n- Allow the user to specify a JSON patch, JSON Merge Patch or Strategic Merge Patch for modification.\n- Allow the user to specify multiple JSON Patch, JSON Merge Patch or Strategic Merge Patch.\n- Allow the user to specify mixed JSON Patch, JSON Merge Patch and Strategic Merge Patch in the same configmap.\n\n## Non Goals\n- Deprecating the existing RestoreItemAction plugins for standard substitutions(like changing the namespace, changing the storage class, etc.)\n\n## User Stories\n\n### Scenario 1\n- Alice has some Pods and part of them have an annotation `{\"for\": \"bar\"}`.\n- Alice wishes to restore these Pods to a different cluster without this annotation.\n- Alice can use this feature to remove this annotation during restore.\n\n### Scenario 2\n- Bob has a Pod with several containers and one container with name nginx has an image `repo1/nginx`.\n- Bob wishes to restore this Pod to a different cluster, but new cluster can not access repo1, so he pushes the image to repo2.\n- Bob can use this feature to update the image of container nginx to `repo2/nginx` during restore.\n\n## Detailed Design\n- The design and approach is inspired by kubectl patch command and [this doc](https://kubernetes.io/docs/tasks/manage-kubernetes-objects/update-api-object-kubectl-patch/).\n- New fields `MergePatches` and `StrategicPatches` will be added to the `ResourceModifierRule` struct to support all three patch types.\n- Only one of the three patch types can be specified in a single `ResourceModifierRule`.\n- Add wildcard support for `groupResource` in `conditions` struct.\n- The workflow to create Resource Modifier ConfigMap and reference it in RestoreSpec will remain the same as described in document [Resource Modifiers](https://github.com/vmware-tanzu/velero/blob/main/site/content/docs/main/restore-resource-modifiers.md).\n\n### How to choose the right patch type\n- [JSON Merge Patch](https://datatracker.ietf.org/doc/html/rfc7386) is a naively simple format, with limited usability. Probably it is a good choice if you are building something small, with very simple JSON Schema.\n- [JSON Patch](https://datatracker.ietf.org/doc/html/rfc6902) is a more complex format, but it is applicable to any JSON documents. For a comparison of JSON patch and JSON merge patch, see [JSON Patch and JSON Merge Patch](https://erosb.github.io/post/json-patch-vs-merge-patch/).\n- Strategic Merge Patch is a Kubernetes defined patch type, mainly used to process resources of type list. You can replace/merge a list, add/remove items from a list by key, change the order of items in a list, etc. Strategic merge patch is not supported for custom resources. For more details, see [this doc](https://kubernetes.io/docs/tasks/manage-kubernetes-objects/update-api-object-kubectl-patch/).\n\n### New Field MergePatches\nMergePatches is a list to specify the merge patches to be applied on the resource. The merge patches will be applied in the order specified in the configmap. A subsequent patch is applied in order and if multiple patches are specified for the same path, the last patch will override the previous patches.\n\nExample of MergePatches in ResourceModifierRule\n```yaml\nversion: v1\nresourceModifierRules:\n- conditions:\n groupResource: pods\n namespaces:\n - ns1\n mergePatches:\n - patchData: |\n {\n \"metadata\": {\n \"annotations\": {\n \"foo\": null\n }\n }\n }\n```\n- The above configmap will apply the Merge Patch to all the pods in namespace ns1 and remove the annotation `foo` from the pods.\n- Both json and yaml format are supported for the patchData.\n\n### New Field StrategicPatches\nStrategicPatches is a list to specify the strategic merge patches to be applied on the resource. The strategic merge patches will be applied in the order specified in the configmap. A subsequent patch is applied in order and if multiple patches are specified for the same path, the last patch will override the previous patches.\n\nExample of StrategicPatches in ResourceModifierRule\n```yaml\nversion: v1\nresourceModifierRules:\n- conditions:\n groupResource: pods\n resourceNameRegex: \"^my-pod$\"\n namespaces:\n - ns1\n strategicPatches:\n - patchData: |\n {\n \"spec\": {\n \"containers\": [\n {\n \"name\": \"nginx\",\n \"image\": \"repo2/nginx\"\n }\n ]\n }\n }\n```\n- The above configmap will apply the Strategic Merge Patch to the pod with name my-pod in namespace ns1 and update the image of container nginx to `repo2/nginx`.\n- Both json and yaml format are supported for the patchData.\n\n### Conditional Patches in ALL Patch Types\nSince JSON Merge Patch and Strategic Merge Patch do not support conditional patches, we will use the `test` operation of JSON Patch to support conditional patches in all patch types by adding it to `Conditions` struct in `ResourceModifierRule`.\n\nExample of test in conditions\n```yaml\nversion: v1\nresourceModifierRules:\n- conditions:\n groupResource: persistentvolumeclaims.storage.k8s.io\n matches:\n - path: \"/spec/storageClassName\"\n value: \"premium\"\n mergePatches:\n - patchData: |\n {\n \"metadata\": {\n \"annotations\": {\n \"foo\": null\n }\n }\n }\n```\n- The above configmap will apply the Merge Patch to all the PVCs in all namespaces with storageClassName premium and remove the annotation `foo` from the PVCs.\n- You can specify multiple rules in the `matches` list. The patch will be applied only if all the matches are satisfied.\n\n### Wildcard Support for GroupResource\nThe user can specify a wildcard for groupResource in the conditions' struct. This will allow the user to apply the patches for all the resources of a particular group or all resources in all groups. For example, `*.apps` will apply to all the resources in the `apps` group, `*` will apply to all the resources in all groups.\n\n### Helper Command to Generate Merge Patch and Strategic Merge Patch\nThe patchData of Strategic Merge Patch is sometimes a bit complex for user to write. We can provide a helper command to generate the patchData for Strategic Merge Patch. The command will take the original resource and the modified resource as input and generate the patchData.\nIt can also be used in JSON Merge Patch.\n\nHere is a sample code snippet to achieve this:\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\n\tcorev1 \"k8s.io/api/core/v1\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n)\n\nfunc main() {\n\tpod := &corev1.Pod{\n\t\tSpec: corev1.PodSpec{\n\t\t\tContainers: []corev1.Container{\n\t\t\t\t{\n\t\t\t\t\tName: \"web\",\n\t\t\t\t\tImage: \"nginx\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tnewPod := pod.DeepCopy()\n\tpatch := client.StrategicMergeFrom(pod)\n\tnewPod.Spec.Containers[0].Image = \"nginx1\"\n\n\tdata, _ := patch.Data(newPod)\n\tfmt.Println(string(data))\n\t// Output:\n\t// {\"spec\":{\"$setElementOrder/containers\":[{\"name\":\"web\"}],\"containers\":[{\"image\":\"nginx1\",\"name\":\"web\"}]}}\n}\n```\n\n## Security Considerations\nNo security impact.\n\n## Compatibility\nCompatible with current Resource Modifiers.\n\n## Implementation\n- Use \"github.com/evanphx/json-patch\" to support JSON Merge Patch.\n- Use \"k8s.io/apimachinery/pkg/util/strategicpatch\" to support Strategic Merge Patch.\n- Use glob to support wildcard for `groupResource` in `conditions` struct.\n- Use `test` operation of JSON Patch to calculate the `matches` in `conditions` struct.\n\n## Future enhancements\n- add a Velero subcommand to generate/validate the patchData for Strategic Merge Patch and JSON Merge Patch.\n- add jq support for more complex conditions or patches, to meet the situations that the current conditions or patches can not handle. like [this issue](https://github.com/vmware-tanzu/velero/issues/6344)\n\n## Open Issues\nN/A\n" }, { "path": "design/Implemented/move-gh-org.md", "content": "# Plan for moving the Velero GitHub repo into the VMware GitHub organization\n\nCurrently, the Velero repository sits under the Heptio GitHub organization. With the acquisition of Heptio by VMware, it is due time that this repo moves to one of the VMware GitHub organizations. This document outlines a plan to move this repo to the VMware Tanzu (https://github.com/vmware-tanzu) organization.\n\n## Goals\n\n- List all steps necessary to have this repo fully functional under the new org\n\n## Non Goals\n\n- Highlight any step necessary around setting up the new organization and its members\n\n## Action items\n\n### Todo list\n\n#### Pre move\n\n- [ ] PR: Blog post communicating the move. https://github.com/heptio/velero/issues/1841. Who: TBD.\n- [ ] PR: Find/replace in all Go, script, yaml, documentation, and website files: `github.com/heptio/velero -> github.com/vmware-tanzu/velero`. Who: a Velero developer; TBD\n- [ ] PR: Update website with the correct GH links. Who: a Velero developer; TBD\n- [ ] PR: Change deployment and grpc-push scripts with the new location path. Who: a Velero developer; TBD\n- [ ] Delete branches not to be carried over (https://github.com/heptio/velero/branches/all). Who: Any of the current repo owners; TBD\n\n#### Move\n\n- [ ] Use GH UI to transfer the repository to the VMW org; must be accepted within a day. Who: new org owner; TBD\n- [ ] Make owners of this repo owners of repo in the new org. Who: new org owner; TBD\n- [ ] Update Travis CI. Who: Any of the new repo owners; TBD\n- [ ] Add DCO for signoff check (https://probot.github.io/apps/dco/). Who: Any of the new repo owners; TBD\n\n\n#### Post move\n\n- [ ] Each individual developer should point their origin to the new location: `git remote set-url origin git@github.com:vmware-tanzu/velero.git`\n- [ ] Transfer ZenHub. Who: Any of the new repo owners; TBD\n- [ ] Update Netlify deploy settings. Any of the new repo owners; TBD\n- [ ] GH app: Netlify integration. Who: Any of the new repo owners; TBD\n- [ ] GH app: Slack integration. Who: Any of the new repo owners; TBD\n- [ ] Add webhook: travis CI. Who: Any of the new repo owners; TBD\n- [ ] Add webhook: zenhub. Who: Any of the new repo owners; TBD\n- [ ] Move all 3 native provider plugins into their own individual repo. https://github.com/heptio/velero/issues/1537. Who: @carlisia.\n- [ ] Merge PRs from the \"pre move\" section\n- [ ] Create a team for the Velero core members (https://github.com/orgs/vmware-tanzu/teams/). Who: Any of the new repo owners; TBD\n\n### Notes/How-Tos\n\n#### Transferring the GH repository\n\nAll action items needed for the repo transfer are listed in the Todo list above. For details about what gets moved and other info, this is the GH documentation: https://help.github.com/en/articles/transferring-a-repository\n\n[Pending] We will find out this week who will be the organization owner(s) who will accept this transfer in the new GH org. This organization owner will make all current owners in this repo owners in the new org Velero repo.\n\n#### Updating Travis CI\n\nSomeone with owner permission on the new repository needs to go to their Travis CI account and authorize Travis CI on the repo. Here are instructions: https://docs.travis-ci.com/user/tutorial/.\n\nAfter this, webhook notifications can be added following these instructions: https://docs.travis-ci.com/user/notifications/#configuring-webhook-notifications.\n\n#### Transferring ZenHub\n\nPre-requisite: A new Zenhub account must exist for a vmware or vmware-tanzu organization.\n\nThis page contains a pre-migration checklist for ensuring a repo migration goes well with Zenhub: https://help.zenhub.com/support/solutions/articles/43000010366-moving-a-repo-cross-organization-or-to-a-new-organization. After this, webhooks can be added by following these instructions: https://github.com/ZenHubIO/API#webhooks.\n\n#### Updating Netlify\n\nThe settings for Netlify should remain the same, except that it now needs to be installed in the new repo. The instructions on how to install Netlify on the new repo are here: https://www.netlify.com/docs/github-permissions/.\n\n#### Communication strategy\n\n[Pending] We will find out this week how this move will be communicated to the community. In particular, the Velero repository move might be tied to the move of our provider plugins into their own repos, also in the new org: https://github.com/heptio/velero/issues/1814.\n\n#### TBD\n\nMany items on the todo list must be done by a repository member with owner permission. This doesn't all need to be done by the same person obviously, but we should specify if @skriss wants to split these tasks with any other owner(s).\n\n#### Other notes\n\nMight want to exclude updating documentation prior to v1.0.0.\nGH documentation does not specify if branches on the server are also moved.\nAll links to the original repository location are automatically redirected to the new location.\n\n## Alternatives Considered\n\nAlternatives such as moving Velero to its own organization, or even not moving at all, were considered. Collectively, however, the open source leadership decided it would be best to move it so it lives alongside other VMware supported cloud native related repositories.\n\n## Security Considerations\n\n- Ensure that only the Velero core team has maintainer/owner privileges.\n" }, { "path": "design/Implemented/move-plugin-repos.md", "content": "# Plan to extract the provider plugins out of (the Velero) tree\n\nCurrently, the Velero project contains in-tree plugins for three cloud providers: AWS, Azure, and GCP. The Velero team has decided to extract each of those plugins into their own separate repository. This document details the steps necessary to create the new repositories, as well as a general design for what each plugin project will look like.\n\n## Goals\n\n- Have 3 new repositories for each cloud provider plugin currently supported by the Velero team: AWS, Azure, and GCP\n- Have the currently in-tree cloud provider plugins behave like any other plugin external to Velero\n\n## Non Goals\n\n- Extend the Velero plugin framework capability in any way\n- Create GH repositories for any plugin other then the currently 3 in-tree plugins\n- Extract out any plugin that is not a cloud provider plugin (ex: item action related plugins)\n\n## Background\n\nWith more and more providers wanting to support Velero, it gets more difficult to justify excluding those from being in-tree just as with the three original ones. At the same time, if we were to include any more plugins in-tree, it would ultimately become the responsibility of the Velero team to maintain an increasing number of plugins. This move aims to equalize the field so all plugins are treated equally. We also hope that, with time, developers interested in getting involved in the upkeep of those plugins will become active enough to be promoted to maintainers. Lastly, having the plugins live in their own individual repositories allows for iteration on them separately from the core codebase.\n\n## Action items\n\n### Todo list\n\n#### Repository creation\n\n- [ ] Use GH UI to create each repository in the new VMW org. Who: new org owner; TBD\n- [ ] Make owners of the Velero repo owners of each repo in the new org. Who: new org owner; TBD\n- [ ] Add Travis CI. Who: Any of the new repo owners; TBD\n- [ ] Add webhook: travis CI. Who: Any of the new repo owners; TBD\n- [ ] Add DCO for signoff check (https://probot.github.io/apps/dco/). Who: Any of the new repo owners; TBD\n\n#### Plugin changes\n\n- [ ] Modify Velero so it can install any of the provider plugins. https://github.com/heptio/velero/issues/1740 - Who: @nrb\n- [ ] Extract each provider plugin into their own repo. https://github.com/heptio/velero/issues/1537\n- [ ] Create deployment and gcr-push scripts with the new location path. Who: @carlisia\n- [ ] Add documentation for how to use the plugin. Who: @carlisia\n- [ ] Update Helm chart to install Velero using any of the provider plugins. https://github.com/heptio/velero/issues/1819\n- [ ] Upgrade script. https://github.com/heptio/velero/issues/1889.\n\n### Notes/How-Tos\n\n#### Creating the GH repository\n\n[Pending] The organization owner will make all current owners in the Velero repo also owners in each of the new org plugin repos.\n\n#### Setting up Travis CI\n\nSomeone with owner permission on the new repository needs to go to their Travis CI account and authorize Travis CI on the repo. Here are instructions: https://docs.travis-ci.com/user/tutorial/.\n\nAfter this, any webhook notifications can be added following these instructions: https://docs.travis-ci.com/user/notifications/#configuring-webhook-notifications.\n\n## High-Level Design\n\nEach provider plugin will be an independent project, using the Velero library to implement their specific functionalities.\n\nThe way Velero is installed will be changed to accommodate installing these plugins at deploy time, namely the Velero `install` command, as well as the Helm chart.\n\nEach plugin repository will need to have their respective images built and pushed to the same registry as the Velero images.\n\n## Detailed Design\n\n### Projects\n\nEach provider plugin will be an independent GH repository, named: `velero-plugin-aws`, `velero-plugin-azure`, and `velero-plugin-gcp`.\n\nBuild of the project will be done the same way as with Velero, using Travis.\n\nImages for all the plugins will be pushed to the same repository as the Velero image, also using Travis.\n\nReleases of each of these plugins will happen in sync with releases of Velero. This will consist of having a tag in the repo and a tagged image build with the same release version as Velero so it makes it easy to identify what versions are compatible, starting at v1.2.\n\nDocumentation for how to install and use the plugins will be augmented in the existing Plugins section of the Velero documentation.\n\nDocumentation for how to use each plugin will reside in their respective repos. The navigation on the Velero documentation will be modified for easy discovery of the docs/images for these plugins.\n\n#### Version compatibility\n\nWe will keep the major and minor release points in sync, but the plugins can have multiple minor dot something releases as long as it remains compatible with the corresponding major/minor release of Velero. Ex:\n\n| Velero | Plugin | Compatible? |\n|---|---|---|\n| v1.2 | v1.2 | ✅ |\n| v1.2 | v1.2.3 | ✅ |\n| v1.2 | v1.3 | 🚫 |\n| v1.3 | v1.2 | 🚫 |\n| v1.3 | v1.3.3 | ✅ |\n\n### Installation\n\nAs per https://github.com/heptio/velero/issues/1740, we will add a `plugins` flag to the Velero install command which will accept an array of URLs pointing to +1 images of plugins to be installed. The `velero plugin add` command should continue working as is, in specific, it should also allow the installation of any of the new 3 provider plugins. @nrb will provide specifics about how this change will be tackled, as well as what will be documented. Part of the work of adding the `plugins` flag will be removing the logic that adds `velero.io` name spacing to plugins that are added without it.\n\nThe Helm chart that allows the installation of Velero will be modified to accept the array of plugin images with an added `plugins` configuration item.\n\n### Design code changes and considerations\n\nThe naming convention to use for name spacing each plugin will be `velero.io`, since they are currently maintained by the Velero team.\n\nInstall dep\n\nQuestion: are there any places outside the plugins where we depend on the cloud-provider SDKs? can we eliminate those dependencies too? x\n\n- the `restic` package uses the `aws`. SDK to get the bucket region for the AWS object store (https://github.com/carlisia/velero/blob/32d46871ccbc6b03e415d1e3d4ad9ae2268b977b/pkg/restic/config.go#L41)\n- could not find usage of the cloud provider SDKs anywhere else.\n\nPlugins such as the pod -> pvc -> pv backupitemaction ones make sense to stay in the core repo as they provide some important logic that just happens to be implemented in a plugin.\n\n### Upgrade\n\nThe documentation for how to fresh install the out-of-tree plugin with Velero v1.2 will be specified together with the documentation for the install changes on issue https://github.com/heptio/velero/issues/1740.\n\nFor upgrades, we will provide a script that will:\n\n- change the tag on the Velero deployment yaml for both the main image and any of th three plugins installed.\n- rename existing aws, azure or gcp plugin names to have the `velero.io/` namespace preceding the name (ex: `velero.io/aws).\n\nAlternatively, we could add CLI `velero upgrade` command that would make these changes. Ex: `velero upgrade 1.3` would upgrade from `v1.2` to `v1.3`.\n\nFor upgrading:\n\n- Edit the provider field in the backupstoragelocations and volumesnapshotlocations CRDs to include the new namespace.\n\n## Alternatives Considered\n\nWe considered having the plugins all live in the same GH repository. The downside of that approach is ending up with a binary and image bigger than necessary, since they would contain the SDKs of all three providers.\n\n## Security Considerations\n\n- Ensure that only the Velero core team has maintainer/owner privileges." }, { "path": "design/Implemented/multiple-arch-build-with-windows.md", "content": "# Multi-arch Build and Windows Build Support\n\n## Background\n\nAt present, Velero images could be built for linux-amd64 and linux-arm64. We need to support other platforms, i.e., windows-amd64. \nAt present, for linux image build, we leverage Buildkit's `--platform` option to create the image manifest list in one build call. However, it is a limited way and doesn't fully support all multi-arch scenarios. Specifically, since the build is done in one call with the same parameters, it is impossbile to build images with different configurations (e.g., Windows build requires a different Dockerfile). \nAt present, Velero by default build images locally, or no image or manifest is pushed to registry. However, docker doesn't support multi-arch build locally. We need to clarify the behavior of local build. \n\n## Goals\n- Refactor the `make container` process to fully support multi-arch build\n- Add Windows build to the existing build process\n- Clarify the behavior of local build with multi-arch build capabilities\n- Don't change the pattern of the final image tag to be used by users\n\n## Non-Goals\n- There may be some workarounds to make the multi-arch image/manifest fully available locally. These workarounds will not be adopted, so local build always build single-arch images\n\n## Local Build\n\nFor local build, two values of `--output` parameter for `docker buildx build` are supported:\n- `docker`: a docker format image is built, but the image is only built for the platform (`/`) as same as the building env. E.g., when building from linux-amd64 env, a single manifest of linux-amd64 is created regardless how the input parameters are configured. \n- `tar`: one or more images are built as tarballs according to the input platform (`/`) parameters. Specifically, one tarball is generated for each platform. The build process is the same with the `Build Separate Manifests` of `Push Build` as detailed below. Merely, the `--output` parameter diffs, as `type=tar;dest=`. The tarball is generated to the `_output` folder and named with the platform info, e.g., `_output/velero-main-linux-amd64.tar`. \n\n## Push Build\n\nFor push build, the `--output` parameter for `docker buildx build` is always `registry`. And build will go according to the input parameters and create multi-arch manifest lists. \n\n### Step 1: Build Separate Manifests\n\nInstead of specifying multiple platforms (`/`) to `--platform` option, we add multiple `container-%` targets in Makefile and each target builds one platform representively. \n\nThe goal here is to build multiple manifests through the multiple targets. However, `docker buildx build` by default creates a manifest list even though there is only one element in `--platform`. Therefore, two flags `--provenance=false` and `--sbom=false` will be set additionally to force `docker buildx build` to create manifests. \n\nEach manifest has a unique tag, the OS type and arch is added to the tag, in the pattern `$(REGISTRY)/$(BIN):$(VERSION)-$(OS)-$(ARCH)`. For example, `velero/velero:main-linux-amd64`. \n\nAll the created manifests will be pushed to registry so that the all-in-one manifest list could be created. \n\n### Step 2: Create All-In-One Manifest List\n\nThe next step is to create a manifest list to include all the created manifests. This could be done by `docker manifest create` command, the tags created and pushed at Step 1 are passed to this command. \nA tag is also created for the manifest list, in the pattern `$(REGISTRY)/$(BIN):$(VERSION)`. For example, `velero/velero:main`. \n\n### Step 3: Push All-In-One Manifest List\n\nThe created manifest will be pushed to registry by command `docker manifest push`. \n\n## Input Parameters\n\nBelow are the input parameters that are configurable to meet different build purposes during Dev and release cycle:\n- BUILD_OUTPUT_TYPE: the type of output for the build, i.e., `docker`, `tar`, `registry`, while `docker` and `tar` is for local build; `registry` means push build. Default value is `docker` \n- BUILD_OS: which types of OS should be built for. Multiple values are accepted, e.g., `linux,windows`. Default value is `linux` \n- BUILD_ARCH: which types of architecture should be built for. Multiple values are accepted, e.g., `amd64,arm64`. Default value is `amd64` \n- BUILDX_INSTANCE: an existing buildx instance to be used by the build. Default value is which indicates the build to create a new buildx instance \n\n## Windows Build\n\nWindows container images vary from Windows OS versions, e.g., `ltsc2022` for Windows server 2022 and `1809` for Windows server 2019. Images for different OS versions should be built separately. \nTherefore, separate build targets are added for each OS version, like `container-windows-%`. \nFor the same reason, a new input parameter is added, `BUILD_WINDOWS_VERSION`. The default value is `ltsc2022`. Windows server 2022 is the only base image we will deliver officially, Windows server 2019 is not supported. In future, we may need to support Windows server 2025 base image. \nFor local build to tar, the Windows OS version is also added to the name of the tarball, e.g., `_output/velero-main-windows-ltsc2022-amd64.tar`. \n\nAt present, Windows container image only supports `amd64` as the architecture, so `BUILD_ARCH` is ignored for Windows. \n\nThe Windows manifests need to be annotated with os type, arch, and os version. This will be done through `docker manifest annotate` command. \n\n## Use Malti-arch Images\n\nIn order to use the images, the manifest list's tag should be provided to `velero install` command or helm, the individual manifests are covered by the manifest list. During launch time, the container engine will load the right image to the container according to the platform of the running node. \n\n## Build Samples\n\n**Local build to docker**\n```\nmake container\n```\nThe built image could be listed by `docker image ls`. \n\n**Local build for linux-amd64 and windows-amd64 to tar**\n```\nBUILD_OUTPUT_TYPE=tar BUILD_OS=linux,windows make container\n```\nUnder `_output` directory, below files are generated: \n```\nvelero-main-linux-amd64.tar\nvelero-main-windows-ltsc2022-amd64.tar\n``` \n\n**Local build for linux-amd64, linux-arm64 and windows-amd64 to tar**\n```\nBUILD_OUTPUT_TYPE=tar BUILD_OS=linux,windows BUILD_ARCH=amd64,arm64 make container\n```\nUnder `_output` directory, below files are generated: \n```\nvelero-main-linux-amd64.tar\nvelero-main-linux-arm64.tar\nvelero-main-windows-ltsc2022-amd64.tar\n```\n\n**Push build for linux-amd64 and windows-amd64** \nPrerequisite: login to registry, e.g., through `docker login` \n```\nBUILD_OUTPUT_TYPE=registry REGISTRY= BUILD_OS=linux,windows make container\n```\nNothing is available locally, in the registry 3 tags are available:\n```\nvelero/velero:main\nvelero/velero:main-windows-ltsc2022-amd64\nvelero/velero:main-linux-amd64\n```\n\n**Push build for linux-amd64, linux-arm64 and windows-amd64** \nPrerequisite: login to registry, e.g., through `docker login` \n```\nBUILD_OUTPUT_TYPE=registry REGISTRY= BUILD_OS=linux,windows BUILD_ARCH=amd64,arm64 make container\n```\nNothing is available locally, in the registry 4 tags are available:\n```\nvelero/velero:main\nvelero/velero:main-windows-ltsc2022-amd64\nvelero/velero:main-linux-amd64\nvelero/velero:main-linux-arm64\n```\n" }, { "path": "design/Implemented/multiple-csi-volumesnapshotclass-support.md", "content": "# Proposal to add support for Multiple VolumeSnapshotClasses in CSI Plugin\n\n- [Proposal to add support for Multiple VolumeSnapshotClasses in CSI Plugin](#proposal-to-add-support-for-multiple-volumesnapshotclasses-in-csi-plugin)\n\t- [Abstract](#abstract)\n\t- [Background](#background)\n\t- [Goals](#goals)\n\t- [Non Goals](#non-goals)\n - [User Stories](#user-stories)\n - [Scenario 1](#scenario-1)\n - [Scenario 2](#scenario-2)\n - [Detailed Design](#detailed-design)\n - [Plugin Inputs Contract Changes](#plugin-inputs-contract-changes)\n - [Using Plugin Inputs for CSI Plugin](#using-plugin-inputs-for-csi-plugin)\n - [Annotations overrides on PVC for CSI Plugin](#annotations-overrides-on-pvc-for-csi-plugin)\n - [Using Plugin Inputs for Other Plugins](#using-plugin-inputs-for-other-plugins)\n - [Alternatives Considered](#alternatives-considered)\n - [Security Considerations](#security-considerations)\n - [Compatibility](#compatibility)\n - [Implementation](#implementation)\n - [Open Issues](#open-issues)\n\n\n## Abstract\nCurrently the Velero CSI plugin chooses the VolumeSnapshotClass in the cluster that has the same driver name and also has the velero.io/csi-volumesnapshot-class label set on it. This global selection is not sufficient for many use cases. This proposal is to add support for multiple VolumeSnapshotClasses in CSI Plugin where the user can specify the VolumeSnapshotClass to use for a particular driver and backup.\n\n\n## Background\nThe Velero CSI plugin chooses the VolumeSnapshotClass in the cluster that has the same driver name and also has the velero.io/csi-volumesnapshot-class label set on it. This global selection is not sufficient for many use cases. For example, if a cluster has multiple VolumeSnapshotClasses for the same driver, the user may want to use a VolumeSnapshotClass that is different from the default one. The user might also have different schedules set up for backing up different parts of the cluster and might wish to use different VolumeSnapshotClasses for each of these backups. \n\n## Goals\n- Allow the user to specify the VolumeSnapshotClass to use for a particular driver and backup.\n\n## Non Goals\n- Deprecating existing VSC selection behaviour. (The current behaviour will remain the default behaviour if the user does not specify the VolumeSnapshotClass to use for a particular driver and backup.)\n\n\n## User Stories\n\n### Scenario 1\n- Consider Alice is a cluster admin and has a cluster with multiple VolumeSnapshotClasses for the same driver. Each VSC stores the snapshots taken in different ResourceGroup(Azure equivalent). \n- Alice has configured multiple scheduled backups each covering a different set of namespaces, representing different apps owned by different teams. \n- Alice wants to use a different VolumeSnapshotClass for each backup such that each snapshot goes in it's respective ResourceGroup to simply management of snapshots(COGS, RBAC etc).\n- In current velero, Alice can't achieve this as the CSI plugin will use the default VolumeSnapshotClass for the driver and all snapshots will go in the same ResourceGroup.\n- Proposed design will allow Alice to achieve this by specifying the VolumeSnapshotClass to use for a particular driver and backup/schedule.\n\n## Scenario 2\n- Bob is a cluster admin has PVCs storing different types of data. \n- Most of the PVCs are used for storing non-sensitive application data. But certain PVCs store critical financial data.\n- For such PVCs Bob wants to use a VolumeSnapshotClass with certain encryption related parameters set. \n- In current velero, Bob can't achieve this as the CSI plugin will use the default VolumeSnapshotClass for the driver and all snapshots will be taken using the same VolumeSnapshotClass.\n- Proposed design will allow Bob to achieve this by overriding the VolumeSnapshotClass to use for a particular driver and backup/schedule using annotations on those specific PVCs.\n\n\n## Detailed Design\n\n### Staged Approach: \n\n### Stage 1 Approach\n#### Through Annotations\n 1. **Support VolumeSnapshotClass selection at backup/schedule level**\n The user can annotate the backup/ schedule with driver and VolumeSnapshotClass name. The CSI plugin will use the VolumeSnapshotClass specified in the annotation. If the annotation is not present, the CSI plugin will use the default VolumeSnapshotClass for the driver.\n\n *example annotation on backup/schedule:*\n ```yaml\n apiVersion: velero.io/v1\n kind: Backup\n metadata:\n name: backup-1\n annotations:\n velero.io/csi-volumesnapshot-class_csi.cloud.disk.driver: csi-diskdriver-snapclass\n velero.io/csi-volumesnapshot-class_csi.cloud.file.driver: csi-filedriver-snapclass\n velero.io/csi-volumesnapshot-class_: csi-snapclass\n ```\n\n To query the annotations on a backup: \"velero.io/csi-volumesnapshot-class_'driver name'\" - where driver names comes from the PVC's driver.\n\n 2. **Support VolumeSnapshotClass selection at PVC level**\n The user can annotate the PVCs with driver and VolumeSnapshotClass name. The CSI plugin will use the VolumeSnapshotClass specified in the annotation. If the annotation is not present, the CSI plugin will use the default VolumeSnapshotClass for the driver. If the VolumeSnapshotClass provided is of a different driver, the CSI plugin will use the default VolumeSnapshotClass for the driver.\n\n *example annotation on PVC:*\n ```yaml\n apiVersion: v1\n kind: PersistentVolumeClaim\n metadata:\n name: pvc-1\n annotations:\n velero.io/csi-volumesnapshot-class: csi-diskdriver-snapclass\n \n ```\n \n Consider this as a override option in conjunction to part 1.\n\n**Note**: The user has to annotate the PVCs or backups with the VolumeSnapshotClass to use for each driver. This is not ideal for the user experience.\n - **Mitigation**: We can extend Velero CLI to also annotate backups/schedules with the VolumeSnapshotClass to use for each driver. This will make it easier for the user to annotate the backups/schedules. This mitigation is not for the PVCs though, since PVCs is anyways a specific use case. Similar to : \" kubectl run --image myimage --annotations=\"foo=bar\" --annotations=\"another=one\" mypod\"\n We can add support for - velero backup create my-backup --annotations \"velero.io/csi:csi.cloud.disk.driver=csi-diskdriver-snapclass\"\n\n### Stage 2 Approach\nThe above annotations route is to get started and for initial design closure/ implementation, north star is to either introduce CSI specific fields (considering that CSI might be a very core part of velero going forward) in the backup/restore CR OR leverage the pluginInputs field as being tracked in: https://github.com/vmware-tanzu/velero/pull/5981\n\nRefer section Alternatives 2. **Through generic property bag in the velero contracts**: in the design doc for more details on the pluginInputs field.\n\n\n## Alternatives Considered\n1. **Through CSI Specific Fields in Velero contracts**\n\n **Considerations**\n - Since CSI snapshotting is done through the plugin, we don't intend to bloat up the Backup Spec with CSI specific fields. \n - But considering that CSI Snapshotting is the way forward, we can debate if we should add a CSI section to the Backup Spec.\n\n\n **Approach**: Similar to VolumeSnapshotLocation param in the Backup Spec, we can add a VolumeSnapshotClass param in the Backup Spec. This will allow the user to specify the VolumeSnapshotClass to use for the backup. The CSI plugin will use the VolumeSnapshotClass specified in the Backup Spec. If the VolumeSnapshotClass is not specified, the CSI plugin will use the default VolumeSnapshotClass for the driver.\n\n *example of VolumeSnapshotClass param in the Backup Spec:*\n ```yaml\n apiVersion: velero.io/v1\n kind: Backup\n metadata:\n name: backup-1\n spec:\n csiParameters: \n volumeSnapshotClasses: \n driver: csi.cloud.disk.driver\n snapClass: csi-diskdriver-snapclass\n timeout: 10m\n ```\n\n1. **Through changes in velero contracts**\n 1. **Through configmap references.**\n Currently even the storageclass mapping plugin expects the user to create a configmap which is used globally, and fetched through labels. This behaviour has same issue as the VolumeSnapshotClass selection. We can introduce a field in the velero contracts which allow passing configmap references for each plugin. And then the plugin can honour the configmap passed in as reference. The configmap can be used to pass the VolumeSnapshotClass to use for the backup, and also other parameters to tweak. This can help in making plugins more flexible while not depending on global behaviour.\n \n\n *example of configmap reference in the velero contracts:*\n ```yaml\n apiVersion: velero.io/v1\n kind: Backup\n metadata:\n name: backup-1\n spec:\n configmapRefs:\n - name: csi-volumesnapshotclass-configmap\n - namespace: velero\n - plugin: velero.io/csi\n ```\n\n 2. **Through generic property bag in the velero contracts**: We can introduce a field in the velero contracts which allow passing a generic property bag for each plugin. And then the plugin can honour the property bag passed in.\n\n\n *example of property bag in the velero contracts:*\n ```yaml\n apiVersion: velero.io/v1\n kind: Backup\n metadata:\n name: backup-1\n spec:\n pluginInputs:\n - name: velero.io/csi\n - properties:\n - key: csi.cloud.disk.driver\n - value: csi-diskdriver-snapclass\n - key: csi.cloud.file.driver\n - value: csi-filedriver-snapclass\n ```\n\n **Note**: Both these approaches can also be used to tweak other parameters such as CSI Snapshotting Timeout/intervals. And further can be used by other plugins.\n\n\n## Security Considerations\nNo security impact.\n\n## Compatibility\nExisting behaviour of csi plugin will be retained where it fetches the VolumeSnapshotClass through the label. This will be the default behaviour if the user does not specify the VolumeSnapshotClass.\n\n## Implementation\nTBD based on closure of high level design proposals.\n\n## Open Issues\nNA\n" }, { "path": "design/Implemented/multiple-label-selectors_design.md", "content": "# Ensure support for backing up resources based on multiple labels\n## Abstract\nAs of today Velero supports filtering of resources based on single label selector per backup. It is desired that Velero\nsupport backing up of resources based on multiple labels (OR logic).\n\n**Note:** This solution is required because Kubernetes label selectors only allow AND logic of labels.\n\n## Background\nCurrently, Velero's Backup/Restore API has a spec field `LabelSelector` which helps in filtering of resources based on\na **single** label value per backup/restore request. For instance, if the user specifies the `Backup.Spec.LabelSelector` as\n`data-protection-app: true`, Velero will grab all the resources that possess this label and perform the backup\noperation on them. The `LabelSelector` field does not accept more than one labels, and thus if the user want to take\nbackup for resources consisting of a label from a set of labels (label1 OR label2 OR label3) then the user needs to\ncreate multiple backups per label rule. It would be really useful if Velero Backup API could respect a set of\nlabels (OR Rule) for a single backup request.\n\nRelated Issue: https://github.com/vmware-tanzu/velero/issues/1508\n\n## Goals\n- Enable support for backing up resources based on multiple labels (OR Logic) in a single backup config.\n- Enable support for restoring resources based on multiple labels (OR Logic) in a single restore config.\n\n## Use Case/Scenario\nLet's say as a Velero user you want to take a backup of secrets, but all these secrets do not have one single consistent\nlabel on them. We want to take backup of secrets having any one label in `app=gdpr`, `app=wpa` and `app=ccpa`. Here\nwe would have to create 3 instances of backup for each label rule. This can become cumbersome at scale.\n\n## High-Level Design\n### Addition of `OrLabelSelectors` spec to Velero Backup/Restore API\nFor Velero to back up resources if they consist of any one label from a set of labels, we would like to add a new spec\nfield `OrLabelSelectors` which would enable user to specify them. The Velero backup would somewhat look like:\n\n```\napiVersion: velero.io/v1\nkind: Backup\nmetadata:\n name: backup-101\n namespace: openshift-adp\nspec:\n includedNamespaces:\n - test\n storageLocation: velero-sample-1\n ttl: 720h0m0s\n orLabelSelectors:\n - matchLabels:\n app=gdpr\n - matchLabels:\n app=wpa\n - matchLabels:\n app=ccpa\n```\n\n**Note:** This approach will **not** be changing any current behavior related to Backup API spec `LabelSelector`. Rather we\npropose that the label in `LabelSelector` spec and labels in `OrLabelSelectors` should be treated as different Velero functionalities.\nBoth these fields will be treated as separate Velero Backup API specs. If `LabelSelector` (singular) is present then just match that label.\nAnd if `OrLabelSelectors` is present then match to any label in the set specified by the user. For backup case, if both the `LabelSelector` and `OrLabelSelectors` \nare specified (we do not anticipate this as a real world use-case) then the `OrLabelSelectors` will take precedence, `LabelSelector` will\nonly be used to filter only when `OrLabelSelectors` is not specified by the user. This helps to keep both spec behaviour independent and not confuse the users. \nThis way we preserve the existing Velero behaviour and implement the new functionality in a much cleaner way.\nFor instance, let's take a look the following cases:\n\n1. Only `LabelSelector` specified: Velero will create a backup with resources matching label `app=protect-db`\n```\napiVersion: velero.io/v1\nkind: Backup\nmetadata:\n name: backup-101\n namespace: openshift-adp\nspec:\n includedNamespaces:\n - test\n storageLocation: velero-sample-1\n ttl: 720h0m0s\n labelSelector:\n - matchLabels:\n app=gdpr\n```\n2. Only `OrLabelSelectors` specified: Velero will create a backup with resources matching any label from set `{app=gdpr, app=wpa, app=ccpa}`\n```\napiVersion: velero.io/v1\nkind: Backup\nmetadata:\n name: backup-101\n namespace: openshift-adp\nspec:\n includedNamespaces:\n - test\n storageLocation: velero-sample-1\n ttl: 720h0m0s\n orLabelSelectors:\n - matchLabels:\n app=gdpr\n - matchLabels:\n app=wpa\n - matchLabels:\n app=ccpa\n```\n\nSimilar implementation will be done for the Restore API as well.\n\n## Detailed Design\nWith the Introduction of `OrLabelSelectors` the BackupSpec and RestoreSpec will look like:\n\nBackupSpec:\n```\ntype BackupSpec struct {\n[...]\n// OrLabelSelectors is a set of []metav1.LabelSelector to filter with\n// when adding individual objects to the backup. Resources matching any one\n// label from the set of labels will be added to the backup. If empty\n// or nil, all objects are included. Optional.\n// +optional\nOrLabelSelectors []\\*metav1.LabelSelector\n[...]\n}\n```\n\nRestoreSpec:\n```\ntype RestoreSpec struct {\n[...]\n// OrLabelSelectors is a set of []metav1.LabelSelector to filter with\n// when restoring objects from the backup. Resources matching any one\n// label from the set of labels will be restored from the backup. If empty\n// or nil, all objects are included from the backup. Optional.\n// +optional\nOrLabelSelectors []\\*metav1.LabelSelector\n[...]\n}\n```\n\nThe logic to collect resources to be backed up for a particular backup will be updated in the `backup/item_collector.go`\naround [here](https://github.com/vmware-tanzu/velero/blob/574baeb3c920f97b47985ec3957debdc70bcd5f8/pkg/backup/item_collector.go#L294).\n\nAnd for filtering the resources to be restored, the changes will go [here](https://github.com/vmware-tanzu/velero/blob/d1063bda7e513150fd9ae09c3c3c8b1115cb1965/pkg/restore/restore.go#L1769)\n\n**Note:**\n- This feature will not be exposed via Velero CLI." }, { "path": "design/Implemented/node-agent-affinity.md", "content": "# Node-agent Load Affinity Design\n\n## Glossary & Abbreviation\n\n**Velero Generic Data Path (VGDP)**: VGDP is the collective modules that is introduced in [Unified Repository design][1]. Velero uses these modules to finish data transfer for various purposes (i.e., PodVolume backup/restore, Volume Snapshot Data Movement). VGDP modules include uploaders and the backup repository. \n\n**Exposer**: Exposer is a module that is introduced in [Volume Snapshot Data Movement Design][2]. Velero uses this module to expose the volume snapshots to Velero node-agent pods or node-agent associated pods so as to complete the data movement from the snapshots. \n\n## Background\n\nVelero node-agent is a daemonset hosting controllers and VGDP modules to complete the concrete work of backups/restores, i.e., PodVolume backup/restore, Volume Snapshot Data Movement backup/restore. \nSpecifically, node-agent runs DataUpload controllers to watch DataUpload CRs for Volume Snapshot Data Movement backups, so there is one controller instance in each node. One controller instance takes a DataUpload CR and then launches a VGDP instance, which initializes a uploader instance and the backup repository connection, to finish the data transfer. The VGDP instance runs inside a node-agent pod or in a pod associated to the node-agent pod in the same node. \n\nVarying from the data size, data complexity, resource availability, VGDP may take a long time and remarkable resources (CPU, memory, network bandwidth, etc.). \nTechnically, VGDP instances are able to run in any node that allows pod schedule. On the other hand, users may want to constrain the nodes where VGDP instances run for various reasons, below are some examples: \n- Prevent VGDP instances from running in specific nodes because users have more critical workloads in the nodes \n- Constrain VGDP instances to run in specific nodes because these nodes have more resources than others \n- Constrain VGDP instances to run in specific nodes because the storage allows volume/snapshot provisions in these nodes only \n\nTherefore, in order to improve the compatibility, it is worthy to configure the affinity of VGDP to nodes, especially for backups for which VGDP instances run frequently and centrally. \n\n## Goals\n\n- Define the behaviors of node affinity of VGDP instances in node-agent for volume snapshot data movement backups \n- Create a mechanism for users to specify the node affinity of VGDP instances for volume snapshot data movement backups \n\n## Non-Goals\n- It is also beneficial to support VGDP instances affinity for PodVolume backup/restore, however, it is not possible since VGDP instances for PodVolume backup/restore should always run in the node where the source/target pods are created. \n- It is also beneficial to support VGDP instances affinity for data movement restores, however, it is not possible in some cases. For example, when the `volumeBindingMode` in the StorageClass is `WaitForFirstConsumer`, the restore volume must be mounted in the node where the target pod is scheduled, so the VGDP instance must run in the same node. On the other hand, considering the fact that restores may not frequently and centrally run, we will not support data movement restores. \n- As elaborated in the [Volume Snapshot Data Movement Design][2], the Exposer may take different ways to expose snapshots, i.e., through backup pods (this is the only way supported at present). The implementation section below only considers this approach currently, if a new expose method is introduced in future, the definition of the affinity configurations and behaviors should still work, but we may need a new implementation. \n\n## Solution\n\nWe will use the ConfigMap specified by `velero node-agent` CLI's parameter `--node-agent-configmap` to host the node affinity configurations.\nThis configMap is not created by Velero, users should create it manually on demand. The configMap should be in the same namespace where Velero is installed. If multiple Velero instances are installed in different namespaces, there should be one configMap in each namespace which applies to node-agent in that namespace only. \nNode-agent server checks these configurations at startup time and use it to initiate the related VGDP modules. Therefore, users could edit this configMap any time, but in order to make the changes effective, node-agent server needs to be restarted. \nInside the ConfigMap we will add one new kind of configuration as the data in the configMap, the name is ```loadAffinity```. \nUsers may want to set different LoadAffinity configurations according to different conditions (i.e., for different storages represented by StorageClass, CSI driver, etc.), so we define ```loadAffinity``` as an array. This is for extensibility consideration, at present, we don't implement multiple configurations support, so if there are multiple configurations, we always take the first one in the array. \n\nThe data structure is as below:\n```go\ntype Configs struct {\n\t// LoadConcurrency is the config for load concurrency per node.\n\tLoadConcurrency *LoadConcurrency `json:\"loadConcurrency,omitempty\"`\n\n\t// LoadAffinity is the config for data path load affinity.\n\tLoadAffinity []*LoadAffinity `json:\"loadAffinity,omitempty\"` \n}\n\ntype LoadAffinity struct {\n // NodeSelector specifies the label selector to match nodes\n NodeSelector metav1.LabelSelector `json:\"nodeSelector\"`\n}\n```\n\n### Affinity\nAffinity configuration means allowing VGDP instances running in the nodes specified. There are two ways to define it:\n- It could be defined by `MatchLabels` of `metav1.LabelSelector`. The labels defined in `MatchLabels` means a `LabelSelectorOpIn` operation by default, so in the current context, they will be treated as affinity rules. \n- It could be defined by `MatchExpressions` of `metav1.LabelSelector`. The labels are defined in `Key` and `Values` of `MatchExpressions` and the `Operator` should be defined as `LabelSelectorOpIn` or `LabelSelectorOpExists`. \n\n### Anti-affinity\nAnti-affinity configuration means preventing VGDP instances running in the nodes specified. Below is the way to define it: \n- It could be defined by `MatchExpressions` of `metav1.LabelSelector`. The labels are defined in `Key` and `Values` of `MatchExpressions` and the `Operator` should be defined as `LabelSelectorOpNotIn` or `LabelSelectorOpDoesNotExist`. \n\n### Sample\nA sample of the ConfigMap is as below:\n```json\n{\n \"loadAffinity\": [\n {\n \"nodeSelector\": {\n \"matchLabels\": {\n \"beta.kubernetes.io/instance-type\": \"Standard_B4ms\"\n },\n \"matchExpressions\": [\n {\n \"key\": \"kubernetes.io/hostname\",\n \"values\": [\n \"node-1\",\n \"node-2\",\n \"node-3\"\n ],\n \"operator\": \"In\"\n },\n {\n \"key\": \"xxx/critial-workload\",\n \"operator\": \"DoesNotExist\"\n }\n ] \n }\n }\n ]\n}\n```\nThis sample showcases two affinity configurations:\n- matchLabels: VGDP instances will run only in nodes with label key `beta.kubernetes.io/instance-type` and value `Standard_B4ms`\n- matchExpressions: VGDP instances will run in node `node1`, `node2` and `node3` (selected by `kubernetes.io/hostname` label)\n\nThis sample showcases one anti-affinity configuration:\n- matchExpressions: VGDP instances will not run in nodes with label key `xxx/critial-workload`\n\nTo create the configMap, users need to save something like the above sample to a json file and then run below command:\n```\nkubectl create cm -n velero --from-file=\n``` \n\n### Implementation\nAs mentioned in the [Volume Snapshot Data Movement Design][2], the exposer decides where to launch the VGDP instances. At present, for volume snapshot data movement backups, the exposer creates backupPods and the VGDP instances will be initiated in the nodes where backupPods are scheduled. So the loadAffinity will be translated (from `metav1.LabelSelector` to `corev1.Affinity`) and set to the backupPods. \n\nIt is possible that node-agent pods, as a daemonset, don't run in every worker nodes, users could fulfil this by specify `nodeSelector` or `nodeAffinity` to the node-agent daemonset spec. On the other hand, at present, VGDP instances must be assigned to nodes where node-agent pods are running. Therefore, if there is any node selection for node-agent pods, users must consider this into this load affinity configuration, so as to guarantee that VGDP instances are always assigned to nodes where node-agent pods are available. This is done by users, we don't inherit any node selection configuration from node-agent daemonset as we think daemonset scheduler works differently from plain pod scheduler, simply inheriting all the configurations may cause unexpected result of backupPod schedule. \nOtherwise, if a backupPod are scheduled to a node where node-agent pod is absent, the corresponding DataUpload CR will stay in `Accepted` phase until the prepare timeout (by default 30min). \n\nAt present, as part of the expose operations, the exposer creates a volume, represented by backupPVC, from the snapshot. The backupPVC uses the same storageClass with the source volume. If the `volumeBindingMode` in the storageClass is `Immediate`, the volume is immediately allocated from the underlying storage without waiting for the backupPod. On the other hand, the loadAffinity is set to the backupPod's affinity. If the backupPod is scheduled to a node where the snapshot volume is not accessible, e.g., because of storage topologies, the backupPod won't get into Running state, concequently, the data movement won't complete. \nOnce this problem happens, the backupPod stays in `Pending` phase, and the corresponding DataUpload CR stays in `Accepted` phase until the prepare timeout (by default 30min). Below is an example of the backupPod's status when the problem happens: \n```\n status:\n conditions:\n - lastProbeTime: null\n message: '0/2 nodes are available: 1 node(s) didn''t match Pod''s node affinity/selector,\n 1 node(s) had volume node affinity conflict. preemption: 0/2 nodes are available:\n 2 Preemption is not helpful for scheduling..'\n reason: Unschedulable\n status: \"False\"\n type: PodScheduled\n phase: Pending\n``` \n\nOn the other hand, the backupPod is deleted after the prepare timeout, so there is no way to tell the cause is one of the above problems or others. \nTo help the troubleshooting, we can add some diagnostic mechanism to discover the status of the backupPod and node-agent in the same node before deleting it as a result of the prepare timeout. \n\n[1]: unified-repo-and-kopia-integration/unified-repo-and-kopia-integration.md\n[2]: volume-snapshot-data-movement/volume-snapshot-data-movement.md" }, { "path": "design/Implemented/node-agent-concurrency.md", "content": "# Node-agent Concurrency Design\n\n## Glossary & Abbreviation\n\n**Velero Generic Data Path (VGDP)**: VGDP is the collective of modules that is introduced in [Unified Repository design][1]. Velero uses these modules to finish data transfer for various purposes (i.e., PodVolume backup/restore, Volume Snapshot Data Movement). VGDP modules include uploaders and the backup repository. \n\n## Background\n\nVelero node-agent is a daemonset hosting controllers and VGDP modules to complete the concrete work of backups/restores, i.e., PodVolume backup/restore, Volume Snapshot Data Movement backup/restore. \nFor example, node-agent runs DataUpload controllers to watch DataUpload CRs for Volume Snapshot Data Movement backups, so there is one controller instance in each node. One controller instance takes a DataUpload CR and then launches a VGDP instance, which initializes a uploader instance and the backup repository connection, to finish the data transfer. The VGDP instance runs inside the node-agent pod or in a pod associated to the node-agent pod in the same node. \n\nVarying from the data size, data complexity, resource availability, VGDP may take a long time and remarkable resources (CPU, memory, network bandwidth, etc.). \nTechnically, VGDP instances are able to run in concurrent regardless of the requesters. For example, a VGDP instance for a PodVolume backup could run in parallel with another VGDP instance for a DataUpload. Then the two VGDP instances share the same resources if they are running in the same node. \n\nTherefore, in order to gain the optimized performance with the limited resources, it is worthy to configure the concurrent number of VGDP per node. When the resources are sufficient in nodes, users can set a large concurrent number, so as to reduce the backup/restore time; otherwise, the concurrency should be reduced, otherwise, the backup/restore may encounter problems, i.e., time lagging, hang or OOM kill. \n\n## Goals\n\n- Define the behaviors of concurrent VGDP instances in node-agent\n- Create a mechanism for users to specify the concurrent number of VGDP per node\n\n## Non-Goals\n- VGDP instances from different nodes always run in concurrent since in most common cases the resources are isolated. For special cases that some resources are shared across nodes, there is no support at present\n- In practice, restores run in prioritized scenarios, e.g., disaster recovery. However, the current design doesn't consider this difference, a VGDP instance for a restore is blocked if it reaches to the limit of the concurrency, even though the ones block it are for backups. If users do meet some problems here, they should consider to stop the backups first\n- Sometimes, users wants to totally block backups/restores from running in a specific node, this is out of the scope the current design. To archive this, more modules need to be considered (i.e., expoers of data movers), simply blocking the VGDP (e.g., by setting its concurrent number to 0) doesn't work. E.g., for a fs backup, VGDP instance must run in the node the source pod is running in, if we simply block from VGDP instance, the PodVolumeBackup CR is still submitted but never processed. \n\n## Solution\n\nWe introduce a ConfigMap specified by `velero node-agent` CLI's parameter `--node-agent-configmap` for users to specify the node-agent related configurations. This configMap is not created by Velero, users should create it manually on demand. The configMap should be in the same namespace where Velero is installed. If multiple Velero instances are installed in different namespaces, there should be one configMap in each namespace which applies to node-agent in that namespace only. \nNode-agent server checks these configurations at startup time and use it to initiate the related VGDP modules. Therefore, users could edit this configMap any time, but in order to make the changes effective, node-agent server needs to be restarted. \nThe ConfigMap may be used for other purpose of configuring node-agent in future, at present, there is only one kind of configuration as the data in the configMap, the name is ```loadConcurrency```. \n\nThe data structure is as below:\n```go\ntype Configs struct {\n\t// LoadConcurrency is the config for load concurrency per node.\n\tLoadConcurrency *LoadConcurrency `json:\"loadConcurrency,omitempty\"`\n}\n\ntype LoadConcurrency struct {\n // GlobalConfig specifies the concurrency number to all nodes for which per-node config is not specified\n GlobalConfig int `json:\"globalConfig,omitempty\"`\n\n // PerNodeConfig specifies the concurrency number to nodes matched by rules\n PerNodeConfig []RuledConfigs `json:\"perNodeConfig,omitempty\"`\n}\n\ntype RuledConfigs struct {\n // NodeSelector specifies the label selector to match nodes\n NodeSelector metav1.LabelSelector `json:\"nodeSelector\"`\n\n // Number specifies the number value associated to the matched nodes\n Number int `json:\"number\"`\n}\n```\n\n### Global concurrent number\nWe allow users to specify a concurrent number that will be applied to all nodes if the per-node number is not specified. This number is set through ```globalConfig```. \nThe number starts from 1 which means there is no concurrency, only one instance of VGDP is allowed. There is no roof limit. \nIf this number is not specified or not valid, a hard-coded default value will be used, the value is set to 1. \n\n### Per-node concurrent number\nWe allow users to specify different concurrent number per node, for example, users can set 3 concurrent instances in Node-1, 2 instances in Node-2 and 1 instance in Node-3. This is for below considerations:\n- The resources may be different among nodes. Then users could specify smaller concurrent number for nodes with less resources while larger number for the ones with more resources\n- Help users to isolate critical environments. Users may run some critical workloads in some specified nodes, since VGDP instances may take large resource consumption, users may want to run less number of instances in the nodes with critical workloads\n\nThe range of Per-node concurrent number is the same with Global concurrent number. \nPer-node concurrent number is preferable to Global concurrent number, so it will overwrite the Global concurrent number for that node. \n\nPer-node concurrent number is implemented through ```perNodeConfig``` field. \n\n```perNodeConfig``` is a list of ```RuledConfigs``` each item of which matches one or more nodes by label selectors and specify the concurrent number for the matched nodes. This means, the nodes are identified by labels. \n\nFor example, the ```perNodeConfig`` could have below elements:\n```\n\"nodeSelector: kubernetes.io/hostname=node1; number: 3\"\n\"nodeSelector: beta.kubernetes.io/instance-type=Standard_B4ms; number: 5\"\n```\nThe first element means the node with host name ```node1``` gets the Per-node concurrent number of 3. \nThe second element means all the nodes with label ```beta.kubernetes.io/instance-type``` of value ```Standard_B4ms``` get the Per-node concurrent number of 5. \nAt least one node is expected to have a label with the specified ```RuledConfigs``` element (rule). If no node is with this label, the Per-node rule makes no effect. \nIf one node falls into more than one rules, e.g., if node1 also has the label ```beta.kubernetes.io/instance-type=Standard_B4ms```, the smallest number (3) will be used. \n\n### Sample\nA sample of the ConfigMap is as below:\n```json\n{\n \"loadConcurrency\": {\n \"globalConfig\": 2,\n \"perNodeConfig\": [\n {\n \"nodeSelector\": {\n \"matchLabels\": {\n \"kubernetes.io/hostname\": \"node1\"\n }\n },\n \"number\": 3\n },\n {\n \"nodeSelector\": {\n \"matchLabels\": {\n \"beta.kubernetes.io/instance-type\": \"Standard_B4ms\"\n }\n },\n \"number\": 5\n }\n ]\n }\n}\n```\nTo create the configMap, users need to save something like the above sample to a json file and then run below command:\n```\nkubectl create cm -n velero --from-file=\n```\n\n### Global data path manager\nAs for the code implementation, data path manager is to maintain the total number of the running VGDP instances and ensure the limit is not excceeded. At present, there is one data path manager instance per controller, as a result, the concurrent numbers are calculated separately for each controller. This doesn't help to limit the concurrency among different requesters. \nTherefore, we need to create one global data path manager instance server-wide, and pass it to different controllers. The instance will be created at node-agent server startup. \nThe concurrent number is required to initiate a data path manager, the number comes from either Per-node concurrent number or Global concurrent number. \nBelow are some prototypes related to data path manager: \n\n```go\nfunc NewManager(cocurrentNum int) *Manager\nfunc (m *Manager) CreateFileSystemBR(jobName string, requestorType string, ctx context.Context, client client.Client, namespace string, callbacks Callbacks, log logrus.FieldLogger) (AsyncBR, error)\n```\n\n\n\n\n\n[1]: Implemented/unified-repo-and-kopia-integration/unified-repo-and-kopia-integration.md" }, { "path": "design/Implemented/node-agent-load-soothing.md", "content": "# Node-agent Load Soothing Design\n\n## Glossary & Abbreviation\n\n**Velero Generic Data Path (VGDP)**: VGDP is the collective of modules that is introduced in [Unified Repository design][1]. Velero uses these modules to finish data transfer for various purposes (i.e., PodVolume backup/restore, Volume Snapshot Data Movement). VGDP modules include uploaders and the backup repository. \n\n## Background\n\nAs mentioned in [node-agent Concurrency design][2], [CSI Snapshot Data Movement design][3], [VGDP Micro Service design][4] and [VGDP Micro Service for fs-backup design][5], all data movement activities for CSI snapshot data movement backups/restores and fs-backup respect the `loadConcurrency` settings configured in the `node-agent-configmap`. Once the number of existing loads exceeds the corresponding `loadConcurrency` setting, the loads will be throttled and some loads will be held until VGDP quotas are available. \nHowever, this throttling only happens after the data mover pod is started and gets to `running`. As a result, when there are large number of concurrent volume backups, there may be many data mover pods get created but the VGDP instances inside them are actually on hold because of the VGDP throttling. \nThis could cause below problems:\n- In some environments, there is a pod limit in each node of the cluster or a pod limit throughout the cluster, too many of the inactive data mover pods may block other pods from running\n- In some environments, the system disk for each node of the cluster is limited, while pods also occupy system disk space, etc., many of the inactive data mover pods also take unnecessary space from system disk and cause other critical pods evicted\n- For CSI snapshot data movement backup, before creation of the data mover pod, the volume snapshot has also created, this means excessive number of snapshots may also be created and live for longer time since the VGDP won't start until the quota is available. However, in some environments, large number of snapshots is not allowed or may cause degradation of the storage peroformance\n\nOn the other hand, the VGDP throttling mentioned in [node-agent Concurrency design][2] is an accurate controlling mechanism, that is, exactly the required number of data mover pods are throttled. \n\nTherefore, another mechanism is required to soothe the creation of the data mover pods and volume snapshots before the VGDP throttling. It doesn't need to accurately control these creations but should effectively reduce the excessive number of inactive data mover pods and volume snapshots. \nIt is not practical to make an accurate control as it is almost impossible to predict which group of nodes a data mover pod is scheduled to, under the consideration of many complex factors, i.e., selected node, affinity, node OS, etc. \n\n\n## Goals\n\n- Allow users to configure the expected number of loads pending on waiting for VGDP load concurrency quota\n- Create a soothing mechanism to prevent new loads from starting if the number of existing loads excceds the expected number\n\n## Non-Goals\n- Accurately controlling the loads from initiation is not a goal \n\n## Solution\n\nWe introduce a new field `prepareQueueLength` in `loadConcurrency` of `node-agent-configmap` as the allowed number of loads that are under preparing (expose). Specifically, loads are in this situation after its CR is in `Accepted` and `Prepared` phase. The `prepareQueueLength` should be a positive number, negative numbers will be ignored. \nOnce the value is set, the soothing mechanism takes effect, as the best effort, only the allowed number of CRs go into `Accepted` or `Prepared` phase, others will wait and stay as `New` state; and thereby only the allowed number of data mover pods, volume snapshots are created. \nOtherwise, node-agent works the same as the legacy behavior, CRs go to `Accepted` or `Prepared` state as soon as the controllers process them and data mover pods and volume snapshots are also created without any constraints. \nIf users want to constrain the excessive number of pending data mover pods and volume snapshots, they could set a value by considering the VGDP load concurrency; otherwise, if they don't see constrains for pods or volume snapshots in their environment, they don't need to use this feature, in parallel preparing could also be beneficial for increasing the concurrency. \n\nNode-agent server checks this configuration at startup time and use it to initiate the related VGDP modules. Therefore, users could edit this configMap any time, but in order to make the changes effective, node-agent server needs to be restarted. \n\nThe data structure is as below:\n```go\ntype LoadConcurrency struct {\n // GlobalConfig specifies the concurrency number to all nodes for which per-node config is not specified\n GlobalConfig int `json:\"globalConfig,omitempty\"`\n\n // PerNodeConfig specifies the concurrency number to nodes matched by rules\n PerNodeConfig []RuledConfigs `json:\"perNodeConfig,omitempty\"`\n\n // PrepareQueueLength specifies the max number of loads that are under expose\n\tPrepareQueueLength int `json:\"prepareQueueLength,omitempty\"` \n}\n```\n\n### Sample\nA sample of the ConfigMap is as below:\n```json\n{\n \"loadConcurrency\": {\n \"globalConfig\": 2,\n \"perNodeConfig\": [\n {\n \"nodeSelector\": {\n \"matchLabels\": {\n \"kubernetes.io/hostname\": \"node1\"\n }\n },\n \"number\": 3\n },\n {\n \"nodeSelector\": {\n \"matchLabels\": {\n \"beta.kubernetes.io/instance-type\": \"Standard_B4ms\"\n }\n },\n \"number\": 5\n }\n ],\n \"prepareQueueLength\": 2\n }\n}\n```\nTo create the configMap, users need to save something like the above sample to a json file and then run below command:\n```\nkubectl create cm -n velero --from-file=\n```\n\n## Detailed Design\nChanges apply to the DataUpload Controller, DataDownload Controller, PodVolumeBackup Controller and PodVolumeRestore Controller, as below:\n1. The soothe happens to data mover CRs (DataUpload, DataDownload, PodVolumeBackup or PodVolumeRestore) that are in `New` state\n2. Before starting processing the CR, the corresponding controller counts the existing CRs under or pending for expose in the cluster, that is a total number of existing DataUpload, DataDownload, PodVolumeBackup and PodVolumeRestore that are in either `Accepted` or `Preparing` state \n3. If the total number doesn't exceed the allowed number, the controller set the CR's phase to `Accepted`\n4. Once the total number exceeds the allowed number, the controller gives up processing the CR and have it requeued later. The delay for the requeue is 5 seconds\n\nThe count happens for all the controllers in all nodes, to prevent the checks drain out the API server, the count happens to controller client caches for those CRs. And the count result is also cached, so that the count only happens whenever necessary. Below shows how it judges the necessity:\n- When one or more CRs' phase change to `Accepted`\n- When one or more CRs' phase change from `Accepted` to one of the terminal phases\n- When one or more CRs' phase change from `Prepared` to one of the terminal phases\n- When one or more CRs' phase change from `Prepared` to `InProgress`\n\nIdeally, 2~3 in the above steps need to be synchornized among controllers in all nodes. However, this synchronization is not implemented, the consideration is as below: \n1. It is impossible to accurately synchronize the count among controllers in different nodes, because the client cache is not coherrent among nodes. \n2. It is possible to synchronize the count among controllers in the same node. However, it is too expensive to make this synchronization, because 2~3 are part of the expose workflow, the synchronization impacts the performance and stability of the existing workflow. \n3. Even without the synchronization, the soothing mechanism still works eventually -- when the controllers see all the discharged loads (expected ones and over-discharged ones), they will stop creating new loads until the quota is available again. \n4. Step 2~3 that need to be synchronized could complete very quickly. \n\nThis is why we say this mechanism is not an accurate control. Or in another word, it is possible that more loads than the number of `prepareQueueLength` are discharged if controllers make the count and expose in the overlapped time (step 2~3). \nFor example, when multiple controllers of the same type (DataUpload, DataDownload, PodVolumeBackup or PodVolumeRestore) from different nodes make the count: \n```\nmax number of waiting loads = number defined by `prepareQueueLength` + number of nodes in cluster\n```\nAs another example, when hybrid loads are running the count concurrently, e.g., mix of data mover backups, data mover restores, pod volume backups or pod volume restores, more loads may be discharged and the number depends on the number of concurrent hybrid loads. \nIn either case, because step 2~3 is short in time, it is less likely to reach the theoretically worset result. \n\n\n\n\n\n[1]: unified-repo-and-kopia-integration/unified-repo-and-kopia-integration.md\n[2]: node-agent-concurrency.md\n[3]: volume-snapshot-data-movement/volume-snapshot-data-movement.md\n[4]: vgdp-micro-service/vgdp-micro-service.md\n[5]: vgdp-micro-service-for-fs-backup/vgdp-micro-service-for-fs-backup.md" }, { "path": "design/Implemented/plugin-backup-and-restore-progress-design.md", "content": "# Progress reporting for backups and restores handled by volume snapshotters\n\nUsers face difficulty in knowing the progress of backup/restore operations of volume snapshotters. This is very similar to the issues faced by users to know progress for restic backup/restore, like, estimation of operation, operation in-progress/hung etc.\n\nEach plugin might be providing a way to know the progress, but, it need not uniform across the plugins.\n\nEven though plugins provide the way to know the progress of backup operation, this information won't be available to user during restore time on the destination cluster.\n\nSo, apart from the issues like progress, status of operation, volume snapshotters have unique problems like\n- not being uniform across plugins\n- not knowing the backup information during restore operation\n- need to be optional as few plugins may not have a way to provide the progress information\n\nThis document proposes an approach for plugins to follow to provide backup/restore progress, which can be used by users to know the progress.\n\n## Goals\n\n- Provide uniform way of visibility into backup/restore operations performed by volume snapshotters\n\n## Non Goals\n- Plugin implementation for this approach\n\n## Background\n\n(Omitted, see introduction)\n\n## High-Level Design\n\n### Progress of backup operation handled by volume snapshotter\n\nProgress will be updated by volume snapshotter in VolumePluginBackup CR which is specific to that backup operation.\n\n### Progress of restore operation handled by volume snapshotter\n\nProgress will be updated by volume snapshotter in VolumePluginRestore CR which is specific to that restore operation.\n\n## Detailed Design\n\n### Approach 1\n\nExisting `Snapshot` Go struct from `volume` package have most of the details related to backup operation performed by volumesnapshotters.\nThis struct also gets backed up to backup location. But, this struct doesn't get synced on other clusters at regular intervals.\nIt is currently synced only during restore operation, and velero CLI shows few of its contents.\n\nAt a high level, in this approach, this struct will be converted to a CR by adding new fields (related to Progress tracking) to it, and gets rid of `volume.Snapshot` struct.\n\nInstead of backing up of Go struct, proposal is: to backup CRs to backup location, and sync them into other cluster by backupSyncController running in that cluster.\n\n#### VolumePluginBackup CR\n\nThere is one addition to volume.SnapshotSpec, i.e., ProviderName to convert it to CR's spec. Below is the updated VolumePluginBackup CR's Spec:\n\n```\ntype VolumePluginBackupSpec struct {\n\t// BackupName is the name of the Velero backup this snapshot\n\t// is associated with.\n\tBackupName string `json:\"backupName\"`\n\n\t// BackupUID is the UID of the Velero backup this snapshot\n\t// is associated with.\n\tBackupUID string `json:\"backupUID\"`\n\n\t// Location is the name of the VolumeSnapshotLocation where this snapshot is stored.\n\tLocation string `json:\"location\"`\n\n\t// PersistentVolumeName is the Kubernetes name for the volume.\n\tPersistentVolumeName string `json:\"persistentVolumeName\"`\n\n\t// ProviderVolumeID is the provider's ID for the volume.\n\tProviderVolumeID string `json:\"providerVolumeID\"`\n\n\t// Provider is the Provider field given in VolumeSnapshotLocation\n\tProvider string `json:\"provider\"`\n\n\t// VolumeType is the type of the disk/volume in the cloud provider\n\t// API.\n\tVolumeType string `json:\"volumeType\"`\n\n\t// VolumeAZ is the where the volume is provisioned\n\t// in the cloud provider.\n\tVolumeAZ string `json:\"volumeAZ,omitempty\"`\n\n\t// VolumeIOPS is the optional value of provisioned IOPS for the\n\t// disk/volume in the cloud provider API.\n\tVolumeIOPS *int64 `json:\"volumeIOPS,omitempty\"`\n}\n```\n\nFew fields (except first two) are added to volume.SnapshotStatus to convert it to CR's status. Below is the updated VolumePluginBackup CR's status:\n```\ntype VolumePluginBackupStatus struct {\n\t// ProviderSnapshotID is the ID of the snapshot taken in the cloud\n\t// provider API of this volume.\n\tProviderSnapshotID string `json:\"providerSnapshotID,omitempty\"`\n\n\t// Phase is the current state of the VolumeSnapshot.\n\tPhase SnapshotPhase `json:\"phase,omitempty\"`\n\n\t// PluginSpecific are a map of key-value pairs that plugin want to provide\n\t// to user to identify plugin properties related to this backup\n\t// +optional\n\tPluginSpecific map[string]string `json:\"pluginSpecific,omitempty\"`\n\n\t// Message is a message about the volume plugin's backup's status.\n\t// +optional\n\tMessage string `json:\"message,omitempty\"`\n\n\t// StartTimestamp records the time a backup was started.\n\t// Separate from CreationTimestamp, since that value changes\n\t// on restores.\n\t// The server's time is used for StartTimestamps\n\t// +optional\n\t// +nullable\n\tStartTimestamp *metav1.Time `json:\"startTimestamp,omitempty\"`\n\n\t// CompletionTimestamp records the time a backup was completed.\n\t// Completion time is recorded even on failed backups.\n\t// Completion time is recorded before uploading the backup object.\n\t// The server's time is used for CompletionTimestamps\n\t// +optional\n\t// +nullable\n\tCompletionTimestamp *metav1.Time `json:\"completionTimestamp,omitempty\"`\n\n\t// Progress holds the total number of bytes of the volume and the current\n\t// number of backed up bytes. This can be used to display progress information\n\t// about the backup operation.\n\t// +optional\n\tProgress VolumeOperationProgress `json:\"progress,omitempty\"`\n}\n\ntype VolumeOperationProgress struct {\n\tTotalBytes int64\n\tBytesDone int64\n}\n\ntype VolumePluginBackup struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ObjectMeta `json:\"metadata,omitempty\"`\n\n\t// +optional\n\tSpec VolumePluginBackupSpec `json:\"spec,omitempty\"`\n\n\t// +optional\n\tStatus VolumePluginBackupStatus `json:\"status,omitempty\"`\n}\n```\n\nFor every backup operation of volume, Velero creates VolumePluginBackup CR before calling volumesnapshotter's CreateSnapshot API.\n\nIn order to know the CR created for the particular backup of a volume, Velero adds following labels to CR:\n- `velero.io/backup-name` with value as Backup Name, and,\n- `velero.io/pv-name` with value as volume that is undergoing backup\n\nBackup name being unique won't cause issues like duplicates in identifying the CR.\nLabels will be set with the value returned from `GetValidName` function. (https://github.com/vmware-tanzu/velero/blob/main/pkg/label/label.go#L35).\n\nIf Plugin supports showing progress of the operation it is performing, it does following:\n- finds the VolumePluginBackup CR related to this backup operation by using `tags` passed in CreateSnapshot call\n- updates the CR with the progress regularly.\n\nAfter return from `CreateSnapshot` in `takePVSnapshot`, currently Velero adds `volume.Snapshot` to `backupRequest`. Instead of this, CR will be added to `backupRequest`.\nDuring persistBackup call, this CR also will be backed up to backup location.\n\nIn backupSyncController, it checks for any VolumePluginBackup CRs that need to be synced from backup location, and syncs them to cluster if needed.\n\nVolumePluginBackup will be useful as long as backed up data is available at backup location. When the Backup is deleted either by manually or due to expiry, VolumePluginBackup also can be deleted.\n\n`processRequest` of `backupDeletionController` will perform deletion of VolumePluginBackup before volumesnapshotter's DeleteSnapshot is called.\n\n#### Backward compatibility:\n\nCurrently `volume.Snapshot` is backed up as `-volumesnapshots.json.gz` file in the backup location.\n\nAs the VolumePluginBackup CR is backed up instead of `volume.Snapshot`, to provide backward compatibility, CR will be backed as the same file i.e., `-volumesnapshots.json.gz` file in the backup location.\n\nFor backward compatibility on restore side, consider below possible cases wrt Velero version on restore side and format of json.gz file at object location:\n\n- older version of Velero, older json.gz file (backupname-volumesnapshots.json.gz)\n\n- older version of Velero, newer json.gz file\n\n- newer version of Velero, older json.gz file\n\n- newer version of Velero, newer json.gz file\n\nFirst and last should be fine.\n\nFor second case, decode in `GetBackupVolumeSnapshots` on the restore side should fill only required fields of older version and should work.\n\nFor third case, after decode, metadata.name will be empty. `GetBackupVolumeSnapshots` decodes older json.gz into the CR which goes fine.\nIt will be modified to return []VolumePluginBackupSpec, and the changes are done accordingly in its caller.\n\nIf decode fails in second case during implementation, this CR need to be backed up to different file. And, for backward compatibility, newer code should check for old file existence, and follow older code if exists. If it doesn't exists, check for newer file and follow the newer code.\n\n`backupSyncController` on restore clusters gets the `-volumesnapshots.json.gz` object from backup location and decodes it to in-memory VolumePluginBackup CR. If its `metadata.name` is populated, controller creates CR. Otherwise, it will not create the CR on the cluster. It can be even considered to create CR on the cluster.\n\n#### VolumePluginRestore CR\n\n```\n// VolumePluginRestoreSpec is the specification for a VolumePluginRestore CR.\ntype VolumePluginRestoreSpec struct {\n\t// SnapshotID is the identifier for the snapshot of the volume.\n\t// This will be used to relate with output in 'velero describe backup'\n\tSnapshotID string `json:\"snapshotID\"`\n\n\t// BackupName is the name of the Velero backup from which PV will be\n\t// created.\n\tBackupName string `json:\"backupName\"`\n\n\t// Provider is the Provider field given in VolumeSnapshotLocation\n\tProvider string `json:\"provider\"`\n\n\t// VolumeType is the type of the disk/volume in the cloud provider\n\t// API.\n\tVolumeType string `json:\"volumeType\"`\n\n\t// VolumeAZ is the where the volume is provisioned\n\t// in the cloud provider.\n\tVolumeAZ string `json:\"volumeAZ,omitempty\"`\n}\n\n// VolumePluginRestoreStatus is the current status of a VolumePluginRestore CR.\ntype VolumePluginRestoreStatus struct {\n\t// Phase is the current state of the VolumePluginRestore.\n\tPhase string `json:\"phase\"`\n\n\t// VolumeID is the PV name to which restore done\n\tVolumeID string `json:\"volumeID\"`\n\n\t// Message is a message about the volume plugin's restore's status.\n\t// +optional\n\tMessage string `json:\"message,omitempty\"`\n\n\t// StartTimestamp records the time a restore was started.\n\t// Separate from CreationTimestamp, since that value changes\n\t// on restores.\n\t// The server's time is used for StartTimestamps\n\t// +optional\n\t// +nullable\n\tStartTimestamp *metav1.Time `json:\"startTimestamp,omitempty\"`\n\n\t// CompletionTimestamp records the time a restore was completed.\n\t// Completion time is recorded even on failed restores.\n\t// The server's time is used for CompletionTimestamps\n\t// +optional\n\t// +nullable\n\tCompletionTimestamp *metav1.Time `json:\"completionTimestamp,omitempty\"`\n\n\t// Progress holds the total number of bytes of the snapshot and the current\n\t// number of restored bytes. This can be used to display progress information\n\t// about the restore operation.\n\t// +optional\n\tProgress VolumeOperationProgress `json:\"progress,omitempty\"`\n\n\t// PluginSpecific are a map of key-value pairs that plugin want to provide\n\t// to user to identify plugin properties related to this restore\n\t// +optional\n\tPluginSpecific map[string]string `json:\"pluginSpecific,omitempty\"`\n}\n\ntype VolumePluginRestore struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ObjectMeta `json:\"metadata,omitempty\"`\n\n\t// +optional\n\tSpec VolumePluginRestoreSpec `json:\"spec,omitempty\"`\n\n\t// +optional\n\tStatus VolumePluginRestoreStatus `json:\"status,omitempty\"`\n}\n```\n\nFor every restore operation, Velero creates VolumePluginRestore CR before calling volumesnapshotter's CreateVolumeFromSnapshot API.\n\nIn order to know the CR created for the particular restore of a volume, Velero adds following labels to CR:\n- `velero.io/backup-name` with value as Backup Name, and,\n- `velero.io/snapshot-id` with value as snapshot id that need to be restored\n- `velero.io/provider` with value as `Provider` in `VolumeSnapshotLocation`\n\nLabels will be set with the value returned from `GetValidName` function. (https://github.com/vmware-tanzu/velero/blob/main/pkg/label/label.go#L35).\n\nPlugin will be able to identify CR by using snapshotID that it received as parameter of CreateVolumeFromSnapshot API, and plugin's Provider name.\nIt updates the progress of restore operation regularly if plugin supports feature of showing progress.\n\nVelero deletes VolumePluginRestore CR when it handles deletion of Restore CR.\n\n### Approach 2\n\nThis approach is different to approach 1 only with respect to Backup.\n\n#### VolumePluginBackup CR\n\n```\n// VolumePluginBackupSpec is the specification for a VolumePluginBackup CR.\ntype VolumePluginBackupSpec struct {\n\t// Volume is the PV name to be backed up.\n\tVolume string `json:\"volume\"`\n\n\t// Backup name\n\tBackup string `json:\"backup\"`\n\n\t// Provider is the Provider field given in VolumeSnapshotLocation\n\tProvider string `json:\"provider\"`\n}\n\n// VolumePluginBackupStatus is the current status of a VolumePluginBackup CR.\ntype VolumePluginBackupStatus struct {\n\t// Phase is the current state of the VolumePluginBackup.\n\tPhase string `json:\"phase\"`\n\n\t// SnapshotID is the identifier for the snapshot of the volume.\n\t// This will be used to relate with output in 'velero describe backup'\n\tSnapshotID string `json:\"snapshotID\"`\n\n\t// Message is a message about the volume plugin's backup's status.\n\t// +optional\n\tMessage string `json:\"message,omitempty\"`\n\n\t// StartTimestamp records the time a backup was started.\n\t// Separate from CreationTimestamp, since that value changes\n\t// on restores.\n\t// The server's time is used for StartTimestamps\n\t// +optional\n\t// +nullable\n\tStartTimestamp *metav1.Time `json:\"startTimestamp,omitempty\"`\n\n\t// CompletionTimestamp records the time a backup was completed.\n\t// Completion time is recorded even on failed backups.\n\t// Completion time is recorded before uploading the backup object.\n\t// The server's time is used for CompletionTimestamps\n\t// +optional\n\t// +nullable\n\tCompletionTimestamp *metav1.Time `json:\"completionTimestamp,omitempty\"`\n\n\t// PluginSpecific are a map of key-value pairs that plugin want to provide\n\t// to user to identify plugin properties related to this backup\n\t// +optional\n\tPluginSpecific map[string]string `json:\"pluginSpecific,omitempty\"`\n\n\t// Progress holds the total number of bytes of the volume and the current\n\t// number of backed up bytes. This can be used to display progress information\n\t// about the backup operation.\n\t// +optional\n\tProgress VolumeOperationProgress `json:\"progress,omitempty\"`\n}\n\ntype VolumeOperationProgress struct {\n\tTotalBytes int64\n\tBytesDone int64\n}\n\ntype VolumePluginBackup struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ObjectMeta `json:\"metadata,omitempty\"`\n\n\t// +optional\n\tSpec VolumePluginBackupSpec `json:\"spec,omitempty\"`\n\n\t// +optional\n\tStatus VolumePluginBackupStatus `json:\"status,omitempty\"`\n}\n```\n\nFor every backup operation of volume, volume snapshotter creates VolumePluginBackup CR in Velero namespace.\nIt keep updating the progress of operation along with other details like Volume name, Backup Name, SnapshotID etc as mentioned in the CR.\n\nIn order to know the CR created for the particular backup of a volume, volume snapshotters adds following labels to CR:\n- `velero.io/backup-name` with value as Backup Name, and,\n- `velero.io/volume-name` with value as volume that is undergoing backup\n\nBackup name being unique won't cause issues like duplicates in identifying the CR.\n\nPlugin need to sanitize the value that can be set for above labels. Label need to be set with the value returned from `GetValidName` function. (https://github.com/vmware-tanzu/velero/blob/main/pkg/label/label.go#L35).\n\nThough no restrictions are required on the name of CR, as a general practice, volume snapshotter can name this CR with the value same as return value of CreateSnapshot.\n\nAfter return from `CreateSnapshot` in `takePVSnapshot`, if VolumePluginBackup CR exists for particular backup of the volume, velero adds this CR to `backupRequest`.\nDuring persistBackup call, this CR also will be backed up to backup location.\n\nIn backupSyncController, it checks for any VolumePluginBackup CRs that need to be synced from backup location, and syncs them to cluster if needed.\n\n`processRequest` of `backupDeletionController` will perform deletion of VolumePluginBackup before volumesnapshotter's DeleteSnapshot is called.\n\nAnother alternative is:\nDeletion of `VolumePluginBackup` CR can be delegated to plugin. Plugin can perform deletion of VolumePluginBackup using the `snapshotID` passed in volumesnapshotter's DeleteSnapshot request.\n\n### 'core' Velero client/server required changes\n\n- Creation of the VolumePluginBackup/VolumePluginRestore CRDs at installation time\n- Persistence of VolumePluginBackup CRs towards the end of the backup operation\n- As part of backup synchronization, VolumePluginBackup CRs related to the backup will be synced.\n- Deletion of VolumePluginBackup when volumeshapshotter's DeleteSnapshot is called\n- Deletion of VolumePluginRestore as part of handling deletion of Restore CR\n- In case of approach 1,\n - converting `volume.Snapshot` struct as CR and its related changes\n - creation of VolumePlugin(Backup|Restore) CRs before calling volumesnapshotter's API\n - `GetBackupVolumeSnapshots` and its callers related changes for change in return type from []volume.Snapshot to []VolumePluginBackupSpec.\n\n### Velero CLI required changes\n\nIn 'velero describe' CLI, required CRs will be fetched from API server and its contents like backupName, PVName (if changed due to label size limitation), size of PV snapshot will be shown in the output.\n\n### API Upgrade\nWhen CRs gets upgraded, velero can support older API versions also (till they get deprecated) to identify the CRs that need to be persisted to backup location.\nHowever, it can provide preference over latest supported API.\n\nIf new fields are added without changing API version, it won't cause any problem as these resources are intended to provide information, and, there is no reconciliation on these resources.\n\n### Compatibility of latest plugin with older version of Velero\nPlugin that supports this CR should handle the situation gracefully when CRDs are not installed. It can handle the errors occurred during creation/updation of the CRs.\n\n## Limitations:\n\nNon K8s native plugins will not be able to implement this as they can not create the CRs.\n\n## Open Questions\n\n## Alternatives Considered\n\n### Add another method to VolumeSnapshotter interface \nAbove proposed approach have limitation that plugin need to be K8s native in order to create, update CRs.\nInstead, a new method for 'Progress' will be added to interface. Velero server regularly polls this 'Progress' method and updates VolumePluginBackup CR on behalf of plugin.\n\nBut, this involves good amount of changes and needs a way for backward compatibility.\n\nAs volume plugins are mostly K8s native, its fine to go ahead with current limitation.\n\n### Update Backup CR\nInstead of creating new CRs, plugins can directly update the status of Backup CR. But, this deviates from current approach of having separate CRs like PodVolumeBackup/PodVolumeRestore to know operations progress.\n\n### Restricting on name rather than using labels\nInstead of using labels to identify the CR related to particular backup on a volume, restrictions can be placed on the name of VolumePluginBackup CR to be same as the value returned from CreateSnapshot.\nBut, this can cause issue when volume snapshotter just crashed without returning snapshot id to velero.\n\n### Backing up VolumePluginBackup CR to different object\nIf CR is backed up to different object other than `#backup-volumesnapshots.json.gz` in backup location, restore controller need to follow 'fall-back model'.\nIt first need to check for new kind of object, and, if it doesn't exists, follow the old model. To avoid 'fall-back' model which prone to errors, VolumePluginBackup CR is backed to same location as that of `volume.Snapshot` location.\n\n## Security Considerations\n\nCurrently everything runs under the same `velero` service account so all plugins have broad access, which would include being able to modify CRs created by another plugin.\n\n" }, { "path": "design/Implemented/plugin-versioning.md", "content": "# Plugin Versioning\n\n## Abstract\nThis proposal outlines an approach to support versioning of Velero's plugin APIs to enable changes to those APIs.\nIt will allow for backwards compatible changes to be made, such as the addition of new plugin methods, but also backwards incompatible changes such as method removal or method signature changes.\n\n\n## Background\nWhen changes are made to Velero’s plugin APIs, there is no mechanism for the Velero server to communicate the version of the API that is supported, or for plugins to communicate what version they implement.\nThis means that any modification to a plugin API is a backwards incompatible change as it requires all plugins which implement the API to update and implement the new method.\n\nThere are several components involved to use plugins within Velero.\nFrom the perspective of the core Velero codebase, all plugin kinds (e.g. `ObjectStore`, `BackupItemAction`) are defined by a single API interface and all interactions with plugins are managed by a plugin manager which provides an implementation of the plugin API interface for Velero to use.\n\nVelero communicates with plugins via gRPC.\nThe core Velero project provides a framework (using the [go-plugin project](https://github.com/hashicorp/go-plugin)) for plugin authors to use to implement their plugins which manages the creation of gRPC servers and clients.\nVelero plugins import the Velero plugin library in order to use this framework.\nWhen a change is made to a plugin API, it needs to be made to the Go interface used by the Velero codebase, and also to the rpc service definition which is compiled to form part of the framework.\nAs each plugin kind is defined by a single interface, when a plugin imports the latest version of the Velero framework, it will need to implement the new APIs in order to build and run successfully.\nIf a plugin does not use the latest version of the framework, and is used with a newer version of Velero that expects the plugin to implement those methods, this will result in a runtime error as the plugin is incompatible.\n\nWith this proposal, we aim to break this coupling and introduce plugin API versions.\n\n## Scenarios to Support\nThe following describes interactions between Velero and its plugins that will be supported with the implementation of this proposal.\nFor the purposes of this list, we will refer to existing Velero and plugin versions as `v1` and all following versions as version `n`.\n\nVelero client communicating with plugins or plugin client calling other plugins:\n\n- Version `n` client will be able to communicate with Version `n` plugin\n- Version `n` client will be able to communicate with all previous versions of the plugin (Version `n-1` back to `v1`)\n\nVelero plugins importing Velero framework:\n- `v1` plugin built against Version `n` Velero framework\n - A plugin may choose to only implement a `v1` API, but it must be able to be built using Version `n` of the Velero framework\n\n\n## Goals\n\n- Allow plugin APIs to change without requiring all plugins to implement the latest changes (even if they upgrade the version of Velero that is imported)\n- Allow plugins to choose which plugin versions they support and enable them to support multiple versions\n- Support breaking changes in the plugin APIs such as method removal or method signature changes\n- Establish a design process for modifying plugin APIs such as method addition and removal and signature changes\n- Establish a process for newer Velero clients to use older versions of a plugin API through adaptation\n\n## Non Goals\n\n- Change how plugins are managed or added\n- Allow older plugin clients to communicate with new versions of plugins\n\n## High-Level Design\n\nWith each change to a plugin API, a new version of the plugin interface and the proto service definition will be created which describes the new plugin API.\nThe plugin framework will be adapted to allow these new plugin versions to be registered.\nPlugins can opt to implement any or all versions of an API, however Velero will always attempt to use the latest version, and the plugin management will be modified to adapt earlier versions of a plugin to be compatible with the latest API where possible.\nUnder the existing plugin framework, any new plugin version will be treated as a new plugin with a new kind.\nThe plugin manager (which provides implementations of a plugin to Velero) will include an adapter layer which will manage the different versions and provide the adaptation for versions which do not implement the latest version of the plugin API.\nProviding an adaptation layer enables Velero and other plugin clients to use an older version of a plugin if it can be safely adapted.\nAs the plugins will be able to introduce backwards incompatible changes, it will _not_ be possible for older version of Velero to use plugins which only support the latest versions of the plugin APIs.\n\nAlthough adding new rpc methods to a service is considered a backwards compatible change within gRPC, due to the way the proto definitions are compiled and included in the framework used by plugins, this will require every plugin to implement the new methods.\nInstead, we are opting to treat the addition of a method to an API as one requiring versioning.\n\nThe addition of optional fields to existing structs which are used as parameters to or return values of API methods will not be considered as a change requiring versioning.\nThese kinds of changes do not modify method signatures and have been safely made in the past with no impact on existing plugins.\n\n## Detailed Design\n\nThe following areas will need to be adapted to support plugin versioning.\n\n### Plugin Interface Definitions\n\nTo provide versioned plugins, any change to a plugin interface (method addition, removal, or signature change) will require a new versioned interface to be created.\nCurrently, all plugin interface definitions reside in `pkg/plugin/velero` in a file corresponding to their plugin kind.\nThese files will be rearranged to be grouped by kind and then versioned: `pkg/plugin/velero///`.\n\nThe following are examples of how each change may be treated:\n\n#### Complete Interface Change\nIf the entire `ObjectStore` interface is being changed such that no previous methods are being included, a file would be added to `pkg/plugin/velero/objectstore/v2/` and would contain the new interface definition:\n\n```\ntype ObjectStore interface {\n\t// Only include new methods that the new API version will support\n\t\n\tNewMethod()\n\t// ...\n}\n```\n\n#### Method Addition\nIf a method is being added to the `ObjectStore` API, a file would be added to `pkg/plugin/velero/objectstore/v2/` and may contain a new API definition as follows:\n\n```\nimport \"github.com/vmware-tanzu/velero/pkg/plugin/velero/objectstore/v1\"\n\ntype ObjectStore interface {\n // Import all the methods from the previous version of the API if they are to be included as is\n v1.ObjectStore\n\n // Provide definitions of any new methods\n NewMethod()\n```\n\n#### Method Removal\nIf a method is being removed from the `ObjectStore` API, a file would be added to `pkg/plugin/velero/objectstore/v2/` and may contain a new API definition as follows:\n\n```\ntype ObjectStore interface {\n\t// Methods which are required from the previous API version must be included, for example\n\tInit(config)\n\tPutObject(bucket, key, body)\n // ...\n\n // Methods which are to be removed are not included\n```\n\n#### Method Signature modification\nIf a method signature in the `ObjectStore` API is being modified, a file would be added to `pkg/plugin/velero/objectstore/v2/` and may contain a new API definition as follows:\n\n```\ntype ObjectStore interface {\n\t// Methods which are required from the previous API version must be included, for example\n\tInit(config)\n\tPutObject(bucket, key, body)\n // ...\t\n\n // Provide new definitions for methods which are being modified\n\tList(bucket, prefix, newParameter)\n\t\n}\n```\n\n### Proto Service Definitions\n\nThe proto service definitions of the plugins will also be versioned and arranged by their plugin kind.\nCurrently, all the proto definitions reside under `pkg/plugin/proto` in a file corresponding to their plugin kind.\nThese files will be rearranged to be grouped by kind and then versioned: `pkg/plugin/proto//`,\nexcept for the current v1 plugins. Those will remain in their current package/location for backwards compatibility.\nThis will allow plugin images built with earlier versions of velero to work with the latest velero (for v1 plugins\nonly). The go_package option will be added to all proto service definitions to allow the proto compilation script\nto place the generated go code for each plugin api version in the proper go package directory.\n\nIt is not possible to import an existing proto service into a new one, so any methods will need to be duplicated across versions if they are required by the new version.\nThe message definitions can be shared however, so these could be extracted from the service definition files and placed in a file that can be shared across all versions of the service.\n\n### Plugin Framework\n\nTo allow plugins to register which versions of the API they implement, the plugin framework will need to be adapted to accept new versions.\nCurrently, the plugin manager stores a [`map[string]RestartableProcess`](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/clientmgmt/manager.go#L69), where the string key is the binary name for the plugin process (e.g. \"velero-plugin-for-aws\").\nEach `RestartableProcess` contains a [`map[kindAndName]interface{}`](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/clientmgmt/restartable_process.go#L60) which represents each of the unique plugin implementations provided by that binary.\n[`kindAndName`](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/clientmgmt/registry.go#L42) is a struct which combines the plugin kind (`ObjectStore`, `VolumeSnapshotter`) and the plugin name (\"velero.io/aws\", \"velero.io/azure\").\n\nEach plugin version registration must be unique (to allow for multiple versions to be implemented within the same plugin binary).\nThis will be achieved by adding a specific registration method for each version to the Server interface in the plugin framework.\nFor example, if adding a V2 `RestoreItemAction` plugin, the Server interface would be modified to add the `RegisterRestoreItemActionV2` method.\nThis would require [adding a new plugin Kind const](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/framework/plugin_kinds.go#L28-L46) to represent the new plugin version, e.g. `PluginKindRestoreItemActionV2`.\nIt also requires the creation of a new implementation of the go-plugin interface ([example](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/framework/object_store.go)) to support that version and use the generated gRPC code from the proto definition (including a client and server implementation).\nThe Server will also need to be adapted to recognize this new plugin Kind and to serve the new implementation.\n\nExisting plugin Kind consts and registration methods will be left unchanged and will correspond to the current version of the plugin APIs (assumed to be v1).\n\n### Plugin Manager\n\nThe plugin manager is responsible for managing the lifecycle of plugins.\nIt provides an interface which is used by Velero to retrieve an instance of a plugin kind with a specific name (e.g. `ObjectStore` with the name \"velero.io/aws\").\nThe manager contains a registry of all available plugins which is populated during the main Velero server startup.\nWhen the plugin manager is requested to provide a particular plugin, it checks the registry for that plugin kind and name.\nIf it is available in the registry, the manager retrieves a `RestartableProcess` for the plugin binary, creating it if it does not already exist.\nThat `RestartableProcess` is then used by individual restartable implementations of a plugin kind (e.g. `restartableObjectStore`, `restartableVolumeSnapshotter`).\n\nAs new plugin versions are added, the plugin manager will be modified to always retrieve the latest version of a plugin kind.\nThis is to allow the remainder of the Velero codebase to assume that it will always interact with the latest version of a plugin.\nIf the latest version of a plugin is not available, it will attempt to fall back to previous versions and use an implementation adapted to the latest version if available.\nIt will be up to the author of new plugin versions to determine whether a previous version of a plugin can be adapted to work with the interface of the new version.\n\nFor each plugin kind, a new `Restartable` struct will be introduced which will contain the plugin Kind and a function, `Get`, which will instantiate a restartable instance of that plugin kind and perform any adaptation required to make it compatible with the latest version.\nFor example, `RestartableObjectStore` or `RestartableVolumeSnapshotter`.\nFor each restartable plugin kind, a new function will be introduced which will return a slice of `Restartable` objects, sorted by version in descending order.\n\nThe manager will iterate through the list of `Restartable`s and will check the registry for the given plugin kind and name.\nIf the requested version is not found, it will skip and continue to iterate, attempting to fetch previous versions of the plugin kind.\nOnce the requested version is found, the `Get` function will be called, returning the restartable implementation of the latest version of that plugin Kind.\n\n```\ntype RestartableObjectStore struct {\n\tkind framework.PluginKind\n\t\n\t// Get returns a restartable ObjectStore for the given name and process, wrapping if necessary\n\tGet func(name string, restartableProcess RestartableProcess) v2.ObjectStore\n}\n\nfunc (m *manager) restartableObjectStores() []RestartableObjectStore {\n\treturn []RestartableObjectStore{\n\t\t{\n\t\t\tkind: framework.PluginKindObjectStoreV2,\n\t\t\tGet: newRestartableObjectStoreV2,\n\t\t},\n\t\t{\n\t\t\tkind: framework.PluginKindObjectStore,\n\t\t\tGet: func(name string, restartableProcess RestartableProcess) v2.ObjectStore {\n\t\t\t\t// Adapt the existing restartable v1 plugin to be compatible with the v2 interface\n\t\t\t\treturn newAdaptedV1ObjectStore(newRestartableObjectStore(name, restartableProcess))\n\t\t\t},\n\t\t},\n\t}\n}\n\n// GetObjectStore returns a restartableObjectStore for name.\nfunc (m *manager) GetObjectStore(name string) (v2.ObjectStore, error) {\n\tname = sanitizeName(name)\n\n\tfor _, restartableObjStore := range m.restartableObjectStores() {\n\t\trestartableProcess, err := m.getRestartableProcess(restartableObjStore.kind, name)\n\t\tif err != nil {\n\t\t\t// Check if plugin was not found\n\t\t\tif errors.Is(err, &pluginNotFoundError{}) {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\treturn nil, err\n\t\t}\n\t\treturn restartableObjStore.Get(name, restartableProcess), nil\n\t}\n\n\treturn nil, fmt.Errorf(\"unable to get valid ObjectStore for %q\", name)\n}\n```\n\nIf the previous version is not available, or can not be adapted to the latest version, it should not be included in the `restartableObjectStores` slice.\nThis will result in an error being returned as is currently the case when a plugin implementation for a particular kind and provider can not be found.\n\nThere are situations where it may be beneficial to check at the point where a plugin API call is made whether it implements a specific version of the API.\nThis is something that can be addressed with future amendments to this design, however it does not seem to be necessary at this time.\n\n#### Plugin Adaptation\n\nWhen a new plugin API version is being proposed, it will be up to the author and the maintainer team to determine whether older versions of an API can be safely adapted to the latest version.\nAn adaptation will implement the latest version of the plugin API interface but will use the methods from the version that is being adapted.\nIn cases where the methods signatures remain the same, the adaptation layer will call through to the same method in the version being adapted.\n\nExamples where an adaptation may be safe:\n- A method signature is being changed to add a new parameter but the parameter could be optional (for example, adding a context parameter). The adaptation could call through to the method provided in the previous version but omit the parameter.\n- A method signature is being changed to remove a parameter, but it is safe to pass a default value to the previous version. The adaptation could call through to the method provided in the previous version but use a default value for the parameter.\n- A new method is being added but does not impact any existing behaviour of Velero (for example, a new method which will allow Velero to [wait for additional items to be ready](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/wait-for-additional-items.md)). The adaptation would return a value which allows the existing behaviour to be performed.\n- A method is being deleted as it is no longer used. The adaptation would call through to any methods which are still included but would omit the deleted method in the adaptation.\n\nExamples where an adaptation may not be safe:\n- A new method is added which is used to provide new critical functionality in Velero. If this functionality can not be replicated using existing plugin methods in previous API versions, this should not be adapted and instead the plugin manager should return an error indicating that the plugin implementation can not be found.\n\n### Restartable Plugin Process\n\nAs new versions of plugins are added, new restartable implementations of plugins will also need to be created.\nThese are currently located within \"pkg/plugin/clientmgmt\" but will be rearranged to be grouped by kind and version like other plugin files.\n\n## Versioning Considerations\n\nIt should be noted that if changes are being made to a plugin's API, it will only be necessary to bump the API version once within a release cycle, regardless of how many changes are made within that cycle.\nThis is because the changes will only be available to consumers when they upgrade to the next minor version of the Velero library.\nNew plugin API versions will not be introduced or backported to patch releases.\n\nOnce a new minor or major version of Velero has been released however, any further changes will need to follow the process above and use a new API version.\n\n## Alternatives Considered\n\n### Relying on gRPC’s backwards compatibility when adding new methods\n\nOne approach for adapting the plugin APIs would have been to rely on the fact that adding methods to gRPC services is a backwards compatible change.\nThis approach would allow older clients to communicate with newer plugins as the existing interface would still be provided.\nThis was considered but ruled out as our current framework would require any plugin that recompiles using the latest version of the framework to adapt to the new version.\nAlso, without specific versioned interfaces, it would require checking plugin implementations at runtime for the specific methods that are supported.\n\n## Compatibility\n\nThis design doc aims to allow plugin API changes to be made in a manner that may provide some backwards compatibility.\nOlder versions of Velero will not be able to make use of new plugin versions however may continue to use previous versions of a plugin API if supported by the plugin.\n\nAll compatibility concerns are addressed earlier in the document.\n\n## Implementation\n\nThis design document primarily outlines an approach to allow future plugin API changes to be made.\nHowever, there are changes to the existing code base that will be made to allow plugin authors to more easily propose and introduce changes to these APIs.\n\n* Plugin interface definitions (currently in `pkg/plugin/velero`) will be rearranged to be grouped by kind and then versioned: `pkg/plugin/velero///`.\n* Proto definitions (currently in `pkg/plugin/proto`) will be rearranged to be grouped by kind and then versioned: `pkg/plugin/proto//`.\n * This will also require changes to the `make update` build task to correctly find the new proto location and output to the versioned directories.\n\nIt is anticipated that changes to the plugin APIs will be made as part of the 1.9 release cycle.\nTo assist with this work, an additional follow-up task to the ones listed above would be to prepare a V2 version of each of the existing plugins.\nThese new versions will not yet provide any new API methods but will provide a layout for new additions to be made\n\n## Open Issues\n" }, { "path": "design/Implemented/priority-class-name-support_design.md", "content": "# PriorityClass Support Design Proposal\n\n## Abstract\n\nThis design document outlines the implementation of priority class name support for Velero components, including the Velero server deployment, node agent daemonset, and maintenance jobs. This feature allows users to specify a priority class name for Velero components, which can be used to influence the scheduling and eviction behavior of these components.\n\n## Background\n\nKubernetes allows users to define priority classes, which can be used to influence the scheduling and eviction behavior of pods. Priority classes are defined as cluster-wide resources, and pods can reference them by name. When a pod is created, the priority admission controller uses the priority class name to populate the priority value for the pod. The scheduler then uses this priority value to determine the order in which pods are scheduled.\n\nCurrently, Velero does not provide a way for users to specify a priority class name for its components. This can be problematic in clusters where resource contention is high, as Velero components may be evicted or not scheduled in a timely manner, potentially impacting backup and restore operations.\n\n## Goals\n\n- Add support for specifying priority class names for Velero components\n- Update the Velero CLI to accept priority class name parameters for different components\n- Update the Velero deployment, node agent daemonset, maintenance jobs, and data mover pods to use the specified priority class names\n\n## Non Goals\n\n- Creating or managing priority classes\n- Automatically determining the appropriate priority class for Velero components\n\n## High-Level Design\n\nThe implementation will add new fields to the Velero options struct to store the priority class names for the server deployment and node agent daemonset. The Velero CLI will be updated to accept new flags for these components. For data mover pods and maintenance jobs, priority class names will be configured through existing ConfigMap mechanisms (`node-agent-configmap` for data movers and `repo-maintenance-job-configmap` for maintenance jobs). The Velero deployment, node agent daemonset, maintenance jobs, and data mover pods will be updated to use their respective priority class names.\n\n## Detailed Design\n\n### CLI Changes\n\nNew flags will be added to the `velero install` command to specify priority class names for different components:\n\n```go\nflags.StringVar(\n &o.ServerPriorityClassName,\n \"server-priority-class-name\",\n o.ServerPriorityClassName,\n \"Priority class name for the Velero server deployment. Optional.\",\n)\n\nflags.StringVar(\n &o.NodeAgentPriorityClassName,\n \"node-agent-priority-class-name\",\n o.NodeAgentPriorityClassName,\n \"Priority class name for the node agent daemonset. Optional.\",\n)\n```\n\nNote: Priority class names for data mover pods and maintenance jobs will be configured through their respective ConfigMaps (`--node-agent-configmap` for data movers and `--repo-maintenance-job-configmap` for maintenance jobs).\n\n### Velero Options Changes\n\nThe `VeleroOptions` struct in `pkg/install/resources.go` will be updated to include new fields for priority class names:\n\n```go\ntype VeleroOptions struct {\n // ... existing fields ...\n ServerPriorityClassName string\n NodeAgentPriorityClassName string\n}\n```\n\n### Deployment Changes\n\nThe `podTemplateConfig` struct in `pkg/install/deployment.go` will be updated to include a new field for the priority class name:\n\n```go\ntype podTemplateConfig struct {\n // ... existing fields ...\n priorityClassName string\n}\n```\n\nA new function, `WithPriorityClassName`, will be added to set this field:\n\n```go\nfunc WithPriorityClassName(priorityClassName string) podTemplateOption {\n return func(c *podTemplateConfig) {\n c.priorityClassName = priorityClassName\n }\n}\n```\n\nThe `Deployment` function will be updated to use the priority class name:\n\n```go\ndeployment := &appsv1api.Deployment{\n // ... existing fields ...\n Spec: appsv1api.DeploymentSpec{\n // ... existing fields ...\n Template: corev1api.PodTemplateSpec{\n // ... existing fields ...\n Spec: corev1api.PodSpec{\n // ... existing fields ...\n PriorityClassName: c.priorityClassName,\n },\n },\n },\n}\n```\n\n### DaemonSet Changes\n\nThe `DaemonSet` function will use the priority class name passed via the podTemplateConfig (from the CLI flag):\n\n```go\ndaemonSet := &appsv1api.DaemonSet{\n // ... existing fields ...\n Spec: appsv1api.DaemonSetSpec{\n // ... existing fields ...\n Template: corev1api.PodTemplateSpec{\n // ... existing fields ...\n Spec: corev1api.PodSpec{\n // ... existing fields ...\n PriorityClassName: c.priorityClassName,\n },\n },\n },\n}\n```\n\n### Maintenance Job Changes\n\nThe `JobConfigs` struct in `pkg/repository/maintenance/maintenance.go` will be updated to include a field for the priority class name:\n\n```go\ntype JobConfigs struct {\n // LoadAffinities is the config for repository maintenance job load affinity.\n LoadAffinities []*kube.LoadAffinity `json:\"loadAffinity,omitempty\"`\n\n // PodResources is the config for the CPU and memory resources setting.\n PodResources *kube.PodResources `json:\"podResources,omitempty\"`\n \n // PriorityClassName is the priority class name for the maintenance job pod\n // Note: This is only read from the global configuration, not per-repository\n PriorityClassName string `json:\"priorityClassName,omitempty\"`\n}\n```\n\nThe `buildJob` function will be updated to use the priority class name from the global job configuration:\n\n```go\nfunc buildJob(cli client.Client, ctx context.Context, repo *velerov1api.BackupRepository, bslName string, config *JobConfigs,\n podResources kube.PodResources, logLevel logrus.Level, logFormat *logging.FormatFlag) (*batchv1.Job, error) {\n // ... existing code ...\n \n // Use the priority class name from the global job configuration if available\n // Note: Priority class is only read from global config, not per-repository\n priorityClassName := \"\"\n if config != nil && config.PriorityClassName != \"\" {\n priorityClassName = config.PriorityClassName\n }\n \n // ... existing code ...\n \n job := &batchv1.Job{\n // ... existing fields ...\n Spec: batchv1.JobSpec{\n // ... existing fields ...\n Template: corev1api.PodTemplateSpec{\n // ... existing fields ...\n Spec: corev1api.PodSpec{\n // ... existing fields ...\n PriorityClassName: priorityClassName,\n },\n },\n },\n }\n \n // ... existing code ...\n}\n```\n\nUsers will be able to configure the priority class name for all maintenance jobs by creating the repository maintenance job ConfigMap before installation. For example:\n\n```bash\n# Create the ConfigMap before running velero install\ncat < repo-maintenance-job-config.json\n{\n \"global\": {\n podResources: {\n \"cpuRequest\": \"100m\",\n \"cpuLimit\": \"200m\",\n \"memoryRequest\": \"100Mi\",\n \"memoryLimit\": \"200Mi\"\n },\n \"loadAffinity\": [\n {\n \"nodeSelector\": {\n \"matchExpressions\": [\n {\n \"key\": \"cloud.google.com/machine-family\",\n \"operator\": \"In\",\n \"values\": [\n \"e2\"\n ]\n }\n ] \n }\n },\n {\n \"nodeSelector\": {\n \"matchExpressions\": [\n {\n \"key\": \"topology.kubernetes.io/zone\",\n \"operator\": \"In\",\n \"values\": [\n \"us-central1-a\",\n \"us-central1-b\",\n \"us-central1-c\"\n ]\n }\n ] \n }\n }\n ]\n }\n}\nEOF\n```\nThis sample showcases two affinity configurations:\n- matchLabels: maintenance job runs on nodes with label key `cloud.google.com/machine-family` and value `e2`.\n- matchLabels: maintenance job runs on nodes located in `us-central1-a`, `us-central1-b` and `us-central1-c`.\nThe nodes matching one of the two conditions are selected.\n\nTo create the configMap, users need to save something like the above sample to a json file and then run below command:\n```\nkubectl create cm repo-maintenance-job-config -n velero --from-file=repo-maintenance-job-config.json\n```\n\n### Value assigning rules\nIf the Velero BackupRepositoryController cannot find the introduced ConfigMap, the following default values are used for repository maintenance job:\n``` go\nconfig := Configs {\n // LoadAffinity is the config for data path load affinity.\n LoadAffinity: nil,\n\n // Resources is the config for the CPU and memory resources setting.\n PodResources: &kube.PodResources{\n // The repository maintenance job CPU request setting\n\t CPURequest: \"0m\",\n\n // The repository maintenance job memory request setting\n\t MemoryRequest: \"0Mi\",\n\n // The repository maintenance job CPU limit setting\n\t CPULimit: \"0m\",\n\n // The repository maintenance job memory limit setting\n\t MemoryLimit: \"0Mi\",\n },\n}\n```\n\nIf the Velero BackupRepositoryController finds the introduced ConfigMap with only `global` element, the `global` value is used.\n\nIf the Velero BackupRepositoryController finds the introduced ConfigMap with only element matches the BackupRepository, the matched element value is used.\n\n\nIf the Velero BackupRepositoryController finds the introduced ConfigMap with both `global` element and element matches the BackupRepository, the matched element defined values overwrite the `global` value, and the `global` value is still used for matched element undefined values.\n\nFor example, the ConfigMap content has two elements.\n``` json\n{\n \"global\": {\n \"loadAffinity\": [\n {\n \"nodeSelector\": {\n \"matchExpressions\": [\n {\n \"key\": \"cloud.google.com/machine-family\",\n \"operator\": \"In\",\n \"values\": [\n \"e2\"\n ]\n }\n ] \n }\n },\n ],\n \"podResources\": {\n \"cpuRequest\": \"100m\",\n \"cpuLimit\": \"200m\",\n \"memoryRequest\": \"100Mi\",\n \"memoryLimit\": \"200Mi\"\n }\n },\n \"ns1-default-kopia\": {\n \"podResources\": {\n \"memoryRequest\": \"400Mi\",\n \"memoryLimit\": \"800Mi\"\n }\n }\n}\n```\nThe config value used for BackupRepository backing up volume data in namespace `ns1`, referencing BSL `default`, and the type is `Kopia`:\n``` go\nconfig := Configs {\n // LoadAffinity is the config for data path load affinity.\n LoadAffinity: []*kube.LoadAffinity{\n {\n\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t{\n\t\t\t\t\t\tKey: \"cloud.google.com/machine-family\",\n\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\tValues: []string{\"e2\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n },\n PodResources: &kube.PodResources{\n // The repository maintenance job CPU request setting\n\t CPURequest: \"\",\n // The repository maintenance job memory request setting\n\t MemoryRequest: \"400Mi\",\n // The repository maintenance job CPU limit setting\n\t CPULimit: \"\",\n // The repository maintenance job memory limit setting\n\t MemoryLimit: \"800Mi\",\n }\n}\n```\n\n\n### Implementation\nDuring the Velero repository controller starts to maintain a repository, it will call the repository manager's `PruneRepo` function to build the maintenance Job.\nThe ConfigMap specified by `velero server` CLI parameter `--repo-maintenance-job-configmap` is get to reinitialize the repository `MaintenanceConfig` setting.\n\n``` go\n\tjobConfig, err := getMaintenanceJobConfig(\n\t\tcontext.Background(),\n\t\tm.client,\n\t\tm.log,\n\t\tm.namespace,\n\t\tm.repoMaintenanceJobConfig,\n\t\trepo,\n\t)\n\tif err != nil {\n log.Infof(\"Cannot find the ConfigMap %s with error: %s. Use default value.\",\n\t\t\tm.namespace+\"/\"+m.repoMaintenanceJobConfig,\n\t\t\terr.Error(),\n\t\t)\n\t}\n\n\tlog.Info(\"Start to maintenance repo\")\n\n\tmaintenanceJob, err := m.buildMaintenanceJob(\n\t\tjobConfig,\n\t\tparam,\n\t)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to build maintenance job\")\n\t}\n```\n\n## Alternatives Considered\nAn other option is creating each ConfigMap for a BackupRepository.\nThis is not ideal for scenario that has a lot of BackupRepositories in the cluster." }, { "path": "design/Implemented/repository-maintenance.md", "content": "# Design for repository maintenance job\n\n## Abstract\nThis design proposal aims to decouple repository maintenance from the Velero server by launching a maintenance job when needed, to mitigate the impact on the Velero server during backups.\n\n## Background\nDuring backups, Velero performs periodic maintenance on the repository. This operation may consume significant CPU and memory resources in some cases, leading to potential issues such as the Velero server being killed by OOM. This proposal addresses these challenges by separating repository maintenance from the Velero server.\n\n## Goals\n1. **Independent Repository Maintenance**: Decouple maintenance from Velero's main logic to reduce the impact on the Velero server pod.\n\n2. **Configurable Resources Usage**: Make the resources used by the maintenance job configurable.\n\n3. **No API Changes**: Retain existing APIs and workflow in the backup repository controller.\n\n## Non Goals\nWe have lots of concerns over parallel maintenance, which will increase the complexity of our design currently.\n\n - Non-blocking maintenance job: it may conflict with updating the same `backuprepositories` CR when parallel maintenance.\n\n - Maintenance job concurrency control: there is no one suitable mechanism in Kubernetes to control the concurrency of different jobs.\n\n - Parallel maintenance: Maintaining the same repo by multiple jobs at the same time would have some compatible cases that some providers may not support.\n\nUnfortunately, parallel maintenance is currently not a priority because of the concerns above, improving maintenance efficiency is not the primary focus at this stage.\n\n## High-Level Design\n1. **Add Maintenance Subcommand**: Introduce a new Velero server subcommand for repository maintenance.\n\n2. **Create Jobs by Repository Manager**: Modify the backup repository controller to create a maintenance job instead of directly calling the multiple chain calls for Kopia or Restic maintenance.\n\n3. **Update Maintenance Job Result in BackupRepository CR**: Retrieve the result of the maintenance job and update the status of the `BackupRepository` CR accordingly.\n\n4. **Add Setting for Maintenance Job**: Introduce a configuration option to set maintenance jobs, including resource limits (CPU and memory), keeping the latest N maintenance jobs for each repository.\n\n## Detailed Design\n\n### 1. Add Maintenance sub-command\n\nThe CLI command will be added to the Velero CLI, the command is designed for use in a pod of maintenance jobs. \n\nOur CLI command is designed as follows:\n```shell\n$ velero repo-maintenance --repo-name $repo-name --repo-type $repo-type --backup-storage-location $bsl\n```\n\nCompared with other CLI commands, the maintenance command is used in a pod of maintenance jobs not for user use, and the job should show the result of maintenance after finish.\n\nHere we will write the error message into one specific file which could be read by the maintenance job.\n\non the whole, we record two kinds of logs:\n\n- one is the log output of the intermediate maintenance process: this log could be retrieved via the Kubernetes API server, including the error log.\n\n- one is the result of the command which could indicate whether the execution is an error or not: the result could be redirected to a file that the maintenance job itself could read, and the file only contains the error message.\n\nwe will write the error message into the `/dev/termination-log` file if execution is failed.\n\nThe main maintenance logic would be using the repository provider to do the maintenance.\n\n```golang\nfunc checkError(err error, file *os.File) {\n\tif err != nil {\n\t\tif err != context.Canceled {\n\t\t\tif _, errWrite := file.WriteString(fmt.Sprintf(\"An error occurred: %v\", err)); errWrite != nil {\n\t\t\t\tfmt.Fprintf(os.Stderr, \"Failed to write error to termination log file: %v\\n\", errWrite)\n\t\t\t}\n\t\t\tfile.Close()\n\t\t\tos.Exit(1) // indicate the command executed failed\n\t\t}\n\t}\n}\n\nfunc (o *Options) Run(f veleroCli.Factory) {\n\tlogger := logging.DefaultLogger(o.LogLevelFlag.Parse(), o.FormatFlag.Parse())\n\tlogger.SetOutput(os.Stdout)\n\n\terrorFile, err := os.Create(\"/dev/termination-log\")\n\tif err != nil {\n\t\tfmt.Fprintf(os.Stderr, \"Failed to create termination log file: %v\\n\", err)\n\t\treturn\n\t}\n\tdefer errorFile.Close()\n\t...\n\n err = o.runRepoPrune(cli, f.Namespace(), logger)\n\tcheckError(err, errorFile)\n\t...\n}\n\nfunc (o *Options) runRepoPrune(cli client.Client, namespace string, logger logrus.FieldLogger) error {\n ...\n\tvar repoProvider provider.Provider\n\tif o.RepoType == velerov1api.BackupRepositoryTypeRestic {\n\t\trepoProvider = provider.NewResticRepositoryProvider(credentialFileStore, filesystem.NewFileSystem(), logger)\n\t} else {\n\t\trepoProvider = provider.NewUnifiedRepoProvider(\n\t\t\tcredentials.CredentialGetter{\n\t\t\t\tFromFile: credentialFileStore,\n\t\t\t\tFromSecret: credentialSecretStore,\n\t\t\t}, o.RepoType, cli, logger)\n\t}\n ...\n\n\terr = repoProvider.BoostRepoConnect(context.Background(), para)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"failed to boost repo connect\")\n\t}\n\n\terr = repoProvider.PruneRepo(context.Background(), para)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"failed to prune repo\")\n\t}\n\treturn nil\n}\n```\n\n### 2. Create Jobs by Repository Manager\nCurrently, the backup repository controller will call the repository manager to do the `PruneRepo`, and Kopia or Restic maintenance is then finally called through multiple chain calls.\n\nWe will keep using the `PruneRepo` function in the repository manager, but we cut off the multiple chain calls by creating a maintenance job.\n\nThe job definition would be like below:\n```yaml\napiVersion: v1\nitems:\n- apiVersion: batch/v1\n kind: Job\n metadata:\n # labels or affinity or topology settings would inherit from the velero deployment\n labels:\n # label the job name for later list jobs by name\n job-name: nginx-example-default-kopia-pqz6c\n name: nginx-example-default-kopia-pqz6c\n namespace: velero\n spec:\n # Not retry it again\n backoffLimit: 1\n # Only have one job one time\n completions: 1\n # Not parallel running job\n parallelism: 1\n template:\n metadata:\n labels:\n job-name: nginx-example-default-kopia-pqz6c\n name: kopia-maintenance-job\n spec:\n containers:\n # arguments for repo maintenance job\n - args:\n - repo-maintenance\n - --repo-name=nginx-example\n - --repo-type=kopia\n - --backup-storage-location=default\n\t # inherit from Velero server\n - --log-level=debug\n command:\n - /velero\n # inherit environment variables from the velero deployment\n env:\n - name: AZURE_CREDENTIALS_FILE\n value: /credentials/cloud\n # inherit image from the velero deployment\n image: velero/velero:main\n imagePullPolicy: IfNotPresent\n name: kopia-maintenance-container\n\t # resource limitation set by Velero server configuration\n\t # if not specified, it would apply best effort resources allocation strategy\n resources: {}\n # error message would be written to /dev/termination-log\n terminationMessagePath: /dev/termination-log\n terminationMessagePolicy: File\n # inherit volume mounts from the velero deployment\n volumeMounts:\n - mountPath: /credentials\n name: cloud-credentials\n dnsPolicy: ClusterFirst\n restartPolicy: Never\n schedulerName: default-scheduler\n securityContext: {}\n # inherit service account from the velero deployment\n serviceAccount: velero\n serviceAccountName: velero\n volumes:\n # inherit cloud credentials from the velero deployment\n - name: cloud-credentials\n secret:\n defaultMode: 420\n secretName: cloud-credentials\n\t# ttlSecondsAfterFinished set the job expired seconds\n\tttlSecondsAfterFinished: 86400\n status:\n\t# which contains the result after maintenance\n\tmessage: \"\"\n\tlastMaintenanceTime: \"\"\n```\n\nNow, the backup repository controller will call the repository manager to create one maintenance job and wait for the job to complete. The Kopia or Restic maintenance multiple chains are called by the job.\n\n### 3. Update the Result of the Maintenance Job into BackupRepository CR\n\nThe backup repository controller will update the result of the maintenance job into the backup repository CR.\n\nFor how to get the result of the maintenance job we could refer to [here](https://kubernetes.io/docs/tasks/debug/debug-application/determine-reason-pod-failure/#writing-and-reading-a-termination-message).\n\nAfter the maintenance job is finished, we could get the result of maintenance by getting the terminated message from the related pod:\n\n```golang\nfunc GetContainerTerminatedMessage(pod *v1.Pod) string {\n\t...\n\tfor _, containerStatus := range pod.Status.ContainerStatuses {\n\t\tif containerStatus.LastTerminationState.Terminated != nil {\n\t\t\treturn containerStatus.LastTerminationState.Terminated.Message\n\t\t}\n\t}\n\t...\n\treturn \"\"\n}\n```\nThen we could update the status of backupRepository CR with the message.\n\n### 4. Add Setting for Resource Usage of Maintenance\nAdd one configuration for setting the resource limit of maintenance jobs as below:\n```shell\n velero server --maintenance-job-cpu-request $cpu-request --maintenance-job-mem-request $mem-request --maintenance-job-cpu-limit $cpu-limit --maintenance-job-mem-limit $mem-limit\n```\nOur default value is 0, which means we don't limit the resources, and the resource allocation strategy would be [best effort](https://kubernetes.io/docs/concepts/workloads/pods/pod-qos/#besteffort).\n\n### 5. Automatic Cleanup for Finished Maintenance Jobs\nAdd configuration for clean up maintenance jobs:\n\n- keep-latest-maintenance-jobs: the number of keeping latest maintenance jobs for each repository.\n\n\t```shell\n\t\tvelero server --keep-latest-maintenance-jobs $num\n\t```\n\nWe would check and keep the latest N jobs after a new job is finished.\n```golang\nfunc deleteOldMaintenanceJobs(cli client.Client, repo string, keep int) error {\n\t// Get the maintenance job list by label\n\tjobList := &batchv1.JobList{}\n\terr := cli.List(context.TODO(), jobList, client.MatchingLabels(map[string]string{RepositoryNameLabel: repo}))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Delete old maintenance jobs\n\tif len(jobList.Items) > keep {\n\t\tsort.Slice(jobList.Items, func(i, j int) bool {\n\t\t\treturn jobList.Items[i].CreationTimestamp.Before(&jobList.Items[j].CreationTimestamp)\n\t\t})\n\t\tfor i := 0; i < len(jobList.Items)-keep; i++ {\n\t\t\terr = cli.Delete(context.TODO(), &jobList.Items[i], client.PropagationPolicy(metav1.DeletePropagationBackground))\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t}\n\n\treturn nil\n}\n```\n\n### 6 Velero Install with Maintenance Options\nAll the above maintenance options should be supported by Velero install command.\n\n### 7. Observability and Debuggability\nSome monitoring metrics are added for backup repository maintenance:\n- repo_maintenance_total\n- repo_maintenance_success_total\n- repo_maintenance_failed_total\n- repo_maintenance_duration_seconds\n\nWe will keep the latest N maintenance jobs for each repo, and users can get the log from the job. the job log level inherent from the Velero server setting.\n\nAlso, we would integrate maintenance job logs and `backuprepositories` CRs into `velero debug`.\n\nRoughly, the process is as follows:\n1. The backup repository controller will check the BackupRepository request in the queue periodically.\n\n2. If the maintenance period of the repository checked by `runMaintenanceIfDue` in `Reconcile` is due, then the backup repository controller will call the Repository manager to execute `PruneRepo`\n\n3. The `PruneRepo` of the Repository manager will create one maintenance job, the resource limitation, environment variables, service account, images, etc. would inherit from the Velero server pod. Also, one clean up TTL would be set to maintenance job.\n\n4. The maintenance job will execute the Velero maintenance command, wait for maintaining to finish and write the maintenance result into the terminationMessagePath file of the related pod.\n\n5. Kubernetes could show the result in the status of the pod by reading the termination message in the pod.\n\n6. The backup repository controller will wait for the maintenance job to finish and read the status of the maintenance job, then update the message field and phase in the status of `backuprepositories` CR accordingly.\n\n6. Clean up old maintenance jobs and keep only N latest for each repository.\n\n### 8. Codes Refinement\nOnce `backuprepositories` CR status is modified, the CR would re-queue to be reconciled, and re-execute logics in reconcile shortly not respecting the re-queue frequency configured by `repoSyncPeriod`.\nFor one abnormal scenario if the maintenance job fails, the status of `backuprepositories` CR would be updated and the CR will re-queue immediately, if the new maintenance job still fails, then it will re-queue again, making the logic of `backuprepositories` CR re-queue like a dead loop.\n\nSo we change the Predicates logic in Controller manager making it only re-queue if the Spec of `backuprepositories` CR is changed.\n\n```golang\n ctrl.NewControllerManagedBy(mgr).For(&velerov1api.BackupRepository{}, builder.WithPredicates(kube.SpecChangePredicate{}))\n```\n\nThis change would bring the behavior different from the previous, errors that occurred in the maintenance job would retry in the next reconciliation period instead of retrying immediately.\n\n## Prospects for Future Work\nFuture work may focus on improving the efficiency of Velero maintenance through non-blocking parallel modes. Potential areas for enhancement include:\n\n**Non-blocking Mode**: Explore the implementation of a non-blocking mode for parallel maintenance to enhance overall efficiency.\n\n**Concurrency Control**: Investigate mechanisms for better concurrency control of different maintenance jobs.\n\n**Provider Support for Parallel Maintenance**: Evaluate the feasibility of parallel maintenance for different providers and address any compatibility issues.\n\n**Efficiency Improvements**: Investigate strategies to optimize maintenance efficiency without compromising reliability.\n\nBy considering these areas, future iterations of Velero may benefit from enhanced parallelization and improved resource utilization during repository maintenance.\n" }, { "path": "design/Implemented/resource-status-restore.md", "content": "# Allow Object-Level Resource Status Restore in Velero\n\n## Abstract\nThis design proposes a way to enhance Velero’s restore functionality by enabling object-level resource status restoration through annotations. \nCurrently, Velero allows restoring resource statuses only at a resource type level, which lacks granularity of restoring the status of specific resources. \nBy introducing an annotation that controllers can set on individual resource objects, this design aims to improve flexibility and autonomy for users/resource-controllers, providing a more way\nto enable resource status restore.\n\n\n## Background\nVelero provides the `restoreStatus` field in the Restore API to specify resource types for status restoration. However, this feature is limited to resource types as a whole, lacking the granularity needed to restore specific objects of a resource type. Resource controllers, especially those managing custom resources with external dependencies, may need to restore status on a per-object basis based on internal logic and dependencies.\n\nThis design adds an annotation-based approach to allow controllers to specify status restoration at the object level, enabling Velero to handle status restores more flexibly.\n\n## Goals\n- Provide a mechanism to specify the restoration of a resource’s status at an object level.\n- Maintain backwards compatibility with existing functionality, allowing gradual adoption of this feature.\n- Integrate the new annotation-based objects-level status restore with Velero’s existing resource-type-level `restoreStatus` configuration.\n\n## Non-Goals\n- Alter Velero’s existing resource type-level status restoration mechanism for resources without annotations.\n\n## Use-Cases/Scenarios\n\n1. Controller managing specific Resources\n - A resource controller identifies that a specific object of a resource should have its status restored due to particular dependencies\n - The controller automatically sets the `velero.io/restore-status: true` annotation on the resource.\n - During restore, Velero restores the status of this object, while leaving other resources unaffected.\n - The status for the annotated object will be restored regardless of its inclusion/exclusion in `restoreStatus.includedResources`\n\n2. A specific object must not have its status restored even if its included in `restoreStatus.includedResources`\n - A user specifies a resource type in the `restoreStatus.includedResources` field within the Restore custom resource.\n - A particular object of that resource type is annotated with `velero.io/restore-status: false` by the user.\n - The status of the annotated object will not restored even though its included in `restoreStatus.includedResources` because annotation is `false` and it takes precedence.\n\n4. Default Behavior for objects Without the Annotation\n - Objects without the `velero.io/restore-status` annotation behave as they currently do: Velero skips their status restoration unless the resource type is specified in the `restoreStatus.includedResources` field.\n\n## High-Level Design\n\n- Object-Level Status Restore Annotation: We are introducing the `velero.io/restore-status` annotation at the resource object level to mark specific objects for status restoration.\n - `true`: Indicates that the status should be restored for this object\n - `false`: Skip restoring status for this specific object\n - Invalid or missing annotations defer to the meaning of existing resource type-level logic.\n\n- Restore logic precedence: \n - Annotations take precedence when they exist with valid values (`true` or `false`).\n - Restore spec `restoreStatus.includedResources` is only used when annotations are invalid or missing.\n\n- Velero Restore Logic Update: During a restore operation, Velero will:\n - Extend the existing restore logic to parse and prioritize annotations introduced in this design.\n - Update resource objects accordingly based on their annotation values or fallback configuration.\n\n\n## Detailed Design\n\n- Annotation for object-Level Status Restore: The `velero.io/restore-status` annotation will be set on individual resource objects by users/controllers as needed:\n```yaml\nmetadata:\n annotations:\n velero.io/restore-status: \"true\"\n```\n\n- Restore Logic Modifications: During the restore operation, the restore controller will follow these steps:\n - Parse the `restoreStatus.includedResources` spec to determine resource types eligible for status restoration.\n - For each resource object:\n - Check for the `velero.io/restore-status` annotation.\n - If the annotation value is:\n - `true`: Restore the status of the object\n - `false`: Skip restoring the status of the object\n - If the annotation is invalid or missing:\n - Default to the `restoreStatus.includedResources` configuration\n\n\n## Implementation\n\nWe are targeting the implementation of this design for Velero 1.16 release.\n\nCurrent restoreStatus logic resides here: https://github.com/vmware-tanzu/velero/blob/32a8c62920ad96c70f1465252c0197b83d5fa6b6/pkg/restore/restore.go#L1652\n\nThe modified logic would look somewhat like:\n\n```go\n// Determine whether to restore status from resource type configuration\nshouldRestoreStatus := ctx.resourceStatusIncludesExcludes != nil && ctx.resourceStatusIncludesExcludes.ShouldInclude(groupResource.String())\n\n// Check for object-level annotation\nannotations := obj.GetAnnotations()\nobjectAnnotation := annotations[\"velero.io/restore-status\"]\nannotationValid := objectAnnotation == \"true\" || objectAnnotation == \"false\"\n\n// Determine restore behavior based on annotation precedence\nshouldRestoreStatus = (annotationValid && objectAnnotation == \"true\") || (!annotationValid && shouldRestoreStatus)\n\nctx.log.Debugf(\"status field for %s: exists: %v, should restore: %v (by annotation: %v)\", newGR, statusFieldExists, shouldRestoreStatus, annotationValid)\n\nif shouldRestoreStatus && statusFieldExists {\n if err := unstructured.SetNestedField(obj.Object, objStatus, \"status\"); err != nil {\n ctx.log.Errorf(\"Could not set status field %s: %v\", kube.NamespaceAndName(obj), err)\n errs.Add(namespace, err)\n return warnings, errs, itemExists\n }\n obj.SetResourceVersion(createdObj.GetResourceVersion())\n updated, err := resourceClient.UpdateStatus(obj, metav1.UpdateOptions{})\n if err != nil {\n ctx.log.Infof(\"Status field update failed %s: %v\", kube.NamespaceAndName(obj), err)\n warnings.Add(namespace, err)\n\t} else {\n createdObj = updated\n }\n}\n```\n\n" }, { "path": "design/Implemented/restic-backup-and-restore-progress.md", "content": "# Progress reporting for restic backups and restores\n\nStatus: Accepted\n\nDuring long-running restic backups/restores, there is no visibility into what (if anything) is happening, making it hard to know if the backup/restore is making progress or hung, how long the operation might take, etc.\nWe should capture progress during restic operations and make it user-visible so that it's easier to reason about.\nThis document proposes an approach for capturing progress of backup and restore operations and exposing this information to users.\n\n## Goals\n\n- Provide basic visibility into restic operations to inform users about their progress.\n\n## Non Goals\n\n- Capturing progress for non-restic backups and restores.\n\n## Background\n\n(Omitted, see introduction)\n\n## High-Level Design\n\n### restic backup progress\n\nThe `restic backup` command provides progress reporting to stdout in JSON format, which includes the completion percentage of the backup.\nThis progress will be read on some interval and the PodVolumeBackup Custom Resource's (CR) status will be updated with this information.\n\n### restic restore progress\n\nThe `restic stats` command returns the total size of a backup.\nThis can be compared with the total size the volume periodically to calculate the completion percentage of the restore.\nThe PodVolumeRestore CR's status will be updated with this information.\n\n## Detailed Design\n\n## Changes to PodVolumeBackup and PodVolumeRestore Status type\n\nA new `Progress` field will be added to PodVolumeBackupStatus and PodVolumeRestoreStatus of type `PodVolumeOperationProgress`:\n\n```\ntype PodVolumeOperationProgress struct {\n TotalBytes int64\n BytesDone int64\n}\n```\n\n### restic backup progress\n\nrestic added support for [streaming JSON output for the `restic backup` command](https://github.com/restic/restic/pull/1944) in 0.9.5.\nOur current images ship restic 0.9.4, and so the Dockerfile will be updated to pull the new version: https://github.com/heptio/velero/blob/af4b9373fc73047f843cd4bc3648603d780c8b74/Dockerfile-velero#L21.\nWith the `--json` flag, `restic backup` outputs single lines of JSON reporting the status of the backup:\n\n```\n{\"message_type\":\"status\",\"percent_done\":0,\"total_files\":1,\"total_bytes\":21424504832}\n{\"message_type\":\"status\",\"action\":\"scan_finished\",\"item\":\"\",\"duration\":0.219241873,\"data_size\":49461329920,\"metadata_size\":0,\"total_files\":10}\n{\"message_type\":\"status\",\"percent_done\":0,\"total_files\":10,\"total_bytes\":49461329920,\"current_files\":[\"/file3\"]}\n{\"message_type\":\"status\",\"percent_done\":0.0003815984736061056,\"total_files\":10,\"total_bytes\":49461329920,\"bytes_done\":18874368,\"current_files\":[\"/file1\",\"/file3\"]}\n{\"message_type\":\"status\",\"percent_done\":0.0011765952936188255,\"total_files\":10,\"total_bytes\":49461329920,\"bytes_done\":58195968,\"current_files\":[\"/file1\",\"/file3\"]}\n{\"message_type\":\"status\",\"percent_done\":0.0019503921984312064,\"total_files\":10,\"total_bytes\":49461329920,\"bytes_done\":96468992,\"current_files\":[\"/file1\",\"/file3\"]}\n{\"message_type\":\"status\",\"percent_done\":0.0028089887640449437,\"total_files\":10,\"total_bytes\":49461329920,\"bytes_done\":138936320,\"current_files\":[\"/file1\",\"/file3\"]}\n```\n\nThe [command factory for backup](https://github.com/heptio/velero/blob/af4b9373fc73047f843cd4bc3648603d780c8b74/pkg/restic/command_factory.go#L37) will be updated to include the `--json` flag.\nThe code to run the `restic backup` command (https://github.com/heptio/velero/blob/af4b9373fc73047f843cd4bc3648603d780c8b74/pkg/controller/pod_volume_backup_controller.go#L241) will be changed to include a Goroutine that reads from the command's stdout stream.\nThe implementation of this will largely follow [@jmontleon's PoC](https://github.com/fusor/velero/pull/4/files) of this.\nThe Goroutine will periodically read the stream (every 10 seconds) and get the last printed status line, which will be converted to JSON.\nIf `bytes_done` is empty, restic has not finished scanning the volume and hasn't calculated the `total_bytes`.\nIn this case, we will not update the PodVolumeBackup and instead will wait for the next iteration.\nOnce we get a non-zero value for `bytes_done`, the `bytes_done` and `total_bytes` properties will be read and the PodVolumeBackup will be patched to update `status.Progress.BytesDone` and `status.Progress.TotalBytes` respectively.\n\nOnce the backup has completed successfully, the PodVolumeBackup will be patched to set `status.Progress.BytesDone = status.Progress.TotalBytes`.\nThis is done since the main thread may cause early termination of the Goroutine once the operation has finished, preventing a final update to the `BytesDone` property.\n\n### restic restore progress\n\nThe `restic stats --json` command provides information about the size of backups:\n\n```\n{\"total_size\":10558111744,\"total_file_count\":11}\n```\n\nBefore beginning the restore operation, we can use the output of `restic stats` to get the total size of the backup.\nThe PodVolumeRestore will be patched to set `status.Progress.TotalBytes` to the total size of the backup.\n\nThe code to run the `restic restore` command will be changed to include a Goroutine that periodically (every 10 seconds) gets the current size of the volume.\nTo get the current size of the volume, we will recursively walkthrough all files in the volume to accumulate the total size.\nThe current total size is the number of bytes transferred so far and the PodVolumeRestore will be patched to update `status.Progress.BytesDone`.\n\nOnce the restore has completed successfully, the PodVolumeRestore will be patched to set `status.Progress.BytesDone = status.Progress.TotalBytes`.\nThis is done since the main thread may cause early termination of the Goroutine once the operation has finished, preventing a final update to the `BytesDone` property.\n\n### Velero CLI changes\n\nThe output that describes detailed information about [PodVolumeBackups](https://github.com/heptio/velero/blob/559d62a2ec99f7a522924348fc4a173a0699813a/pkg/cmd/util/output/backup_describer.go#L349) and [PodVolumeRestores](https://github.com/heptio/velero/blob/559d62a2ec99f7a522924348fc4a173a0699813a/pkg/cmd/util/output/restore_describer.go#L160) will be updated to calculate and display a completion percentage from `status.Progress.TotalBytes` and `status.Progress.BytesDone` if available.\n\n## Open Questions\n\n- Can we assume that the volume we are restoring in will be empty? Can it contain other artefacts?\n - Based on discussion in this PR, we are okay making the assumption that the PVC is empty and will proceed with the above proposed approach.\n\n## Alternatives Considered\n\n### restic restore progress\n\nIf we cannot assume that the volume we are restoring into will be empty, we can instead use the output from `restic snapshot` to get the list of files in the backup.\nThis can then be used to calculate the current total size of just those files in the volume, so that we avoid considering any other files unrelated to the backup.\nThe above proposed approach is simpler than this one, as we don't need to keep track of each file in the backup, but this will be more robust if the volume could contain other files not included in the backup.\nIt's possible that certain volume types may contain hidden files that could attribute to the total size of the volume, though these should be small enough that the BytesDone calculation will only be slightly inflated.\n\nAnother option is to contribute progress reporting similar to `restic backup` for `restic restore` upstream.\nThis may take more time, but would give us a more native view on the progress of a restore.\nThere are several issues about this already in the restic repo (https://github.com/restic/restic/issues/426, https://github.com/restic/restic/issues/1154), and what looks like an abandoned attempt (https://github.com/restic/restic/pull/2003) which we may be able to pick up.\n\n## Security Considerations\n\nN/A\n" }, { "path": "design/Implemented/restore-finalizing-phase_design.md", "content": "# Design for Adding Finalization Phase in Restore Workflow\n\n## Abstract\nThis design proposes adding the finalization phase to the restore workflow. The finalization phase would be entered after all item restoration and plugin operations have been completed, similar to the way the backup process proceeds. Its purpose is to perform any wrap-up work necessary before transitioning the restore process to a terminal phase.\n\n## Background\nCurrently, the restore process enters a terminal phase once all item restoration and plugin operations have been completed. However, there are some wrap-up works that need to be performed after item restoration and plugin operations have been fully executed. There is no suitable opportunity to perform them at present.\n\nTo address this, a new finalization phase should be added to the existing restore workflow. in this phase, all plugin operations and item restoration has been fully completed, which provides a clean opportunity to perform any wrap-up work before termination, improving the overall restore process.\n\nWrap-up tasks in Velero can serve several purposes:\n- Post-restore modification - Velero can modify the restored data that was temporarily changed for some purpose but required to be changed back finally or data that was newly created but missing some information. For example, [issue6435](https://github.com/vmware-tanzu/velero/issues/6435) indicates that some custom settings(like labels, reclaim policy) on restored PVs was lost because those restored PVs was newly dynamically provisioned. Velero can address it by patching the PVs' custom settings back in the finalization phase.\n- Clean up unused data - Velero can identify and delete any data that are no longer needed after a successful restore in the finalization phase.\n- Post-restore validation - Velero can validate the state of restored data and report any errors to help users locate the issue in the finalization phase.\n\nThe uses of wrap-up tasks are not limited to these examples. Additional needs may be addressed as they develop over time.\n\n## Goals\n- Add the finalization phase and the corresponding controller to restore workflow.\n\n## Non Goals\n- Implement the specific wrap-up work.\n\n\n## High-Level Design\n- The finalization phase will be added to current restore workflow. \n- The logic for handling current phase transition in restore and restore operations controller will be modified with the introduction of the finalization phase.\n- A new restore finalizer controller will be implemented to handle the finalization phase.\n\n## Detailed Design\n\n### phase transition\nTwo new phases related to finalization will be added to restore workflow, which are `FinalizingPartiallyFailed` and `Finalizing`. The new phase transition will be similar to backup workflow, proceeding as follow: \n\n![image](restore-phases-transition.png)\n\n### restore finalizer controller\nThe new restore finalizer controller will be implemented to watch for restores in `FinalizingPartiallyFailed` and `Finalizing` phases. Any wrap-up work that needs to wait for the completion of item restoration and plugin operations will be executed by this controller, and the phase will be set to either `Completed` or `PartiallyFailed` based on the results of these works. \n\nPoints worth noting about the new restore finalizer controller:\n\nA new structure `finalizerContext` will be created to facilitate the implementation of any wrap-up tasks. It includes all the dependencies the tasks require as well as a function `execute()` to orderly implement task logic. \n```\n// finalizerContext includes all the dependencies required by wrap-up tasks\ntype finalizerContext struct {\n .......\n\trestore *velerov1api.Restore\n\tlog logrus.FieldLogger\n .......\n}\n\n// execute executes all the wrap-up tasks and return the result\nfunc (ctx *finalizerContext) execute() (results.Result, results.Result) {\n // execute task1\n .......\n\n // execute task2\n .......\n\n // the task execution logic will be expanded as new tasks are included\n .......\n}\n\n// newFinalizerContext returns a finalizerContext object, the parameters will be added as new tasks are included. \nfunc newFinalizerContext(restore *velerov1api.Restore, log logrus.FieldLogger, ...) *finalizerContext{\n return &finalizerContext{\n .......\n restore: restore,\n log: log,\n ....... \n }\n}\n```\nThe finalizer controller is responsible for collecting all dependencies and creating a `finalizerContext` object using those dependencies. It then invokes the `execute` function. \n```\nfunc (r *restoreFinalizerReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n ....... \n \n // collect all dependencies required by wrap-up tasks\n .......\n\n // create a finalizerContext object and invoke execute()\n finalizerCtx := newFinalizerContext(restore, log, ...)\n warnings, errs := finalizerCtx.execute()\n\n .......\n}\n\n```\nAfter completing all necessary tasks, the result metadata in object storage will be updated if any errors or warnings occur during the execution. This behavior breaks the feature of keeping metadata files in object storage immutable, However, we believe the tradeoff is justified because it provides users with the access to examine the error/warning details when the wrap-up tasks go wrong.\n\n```\n// UpdateResults updates the result metadata in object storage if necessary \nfunc (r *restoreFinalizerReconciler) UpdateResults(restore *api.Restore, newWarnings *results.Result, newErrs *results.Result, backupStore persistence.BackupStore) error {\n\toriginResults, err := backupStore.GetRestoreResults(restore.Name)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error getting restore results\")\n\t}\n\twarnings := originResults[\"warnings\"]\n\terrs := originResults[\"errors\"]\n\twarnings.Merge(newWarnings)\n\terrs.Merge(newErrs)\n\n\tm := map[string]results.Result{\n\t\t\"warnings\": warnings,\n\t\t\"errors\": errs,\n\t}\n\tif err := putResults(restore, m, backupStore); err != nil {\n\t\treturn errors.Wrap(err, \"error putting restore results\")\n\t}\n\n\treturn nil\n}\n```\n\n## Compatibility\nThe new finalization phases are added without modifying the existing phases in the restore workflow. Both new and ongoing restore processes will continue to eventually transition to a terminal phase from any prior phase, ensuring backward compatibility.\n\n## Implementation\nThis will be implemented during the Velero 1.14 development cycle.\n" }, { "path": "design/Implemented/restore-hooks.md", "content": "# Restore Hooks\n\nThis document proposes a solution that allows a user to specify Restore Hooks, much like Backup Hooks, that can be executed during the restore process.\n\n## Goals\n\n- Enable custom commands to be run during a restore in order to mirror the commands that are available to the backup process.\n- Provide observability into the result of commands run in restored pods.\n\n## Non Goals\n\n- Handling any application specific scenarios (postgres, mongo, etc)\n\n## Background\n\nVelero supports Backup Hooks to execute commands before and/or after a backup.\nThis enables a user to, among other things, prepare data to be backed up without having to freeze an in-use volume.\nAn example of this would be to attach an empty volume to a Postgres pod, use a backup hook to execute `pg_dump` from the data volume, and back up the volume containing the export.\nThe problem is that there's no easy or automated way to include an automated restore process.\nAfter a restore with the example configuration above, the postgres pod will be empty, but there will be a need to manually exec in and run `pg_restore`.\n\n## High-Level Design\n\nThe Restore spec will have a `spec.hooks` section matching the same section on the Backup spec except no `pre` hooks can be defined - only `post`.\nAnnotations comparable to the annotations used during backup can also be set on pods.\nFor each restored pod, the Velero server will check if there are any hooks applicable to the pod.\nIf a restored pod has any applicable hooks, Velero will wait for the container where the hook is to be executed to reach status Running.\nThe Restore log will include the results of each post-restore hook and the Restore object status will incorporate the results of hooks.\nThe Restore log will include the results of each hook and the Restore object status will incorporate the results of hooks.\n\nA new section at `spec.hooks.resources.initContainers` will allow for injecting initContainers into restored pods.\nAnnotations can be set as an alternative to defining the initContainers in the Restore object.\n\n## Detailed Design\n\nPost-restore hooks can be defined by annotation and/or by an array of resource hooks in the Restore spec.\n\nThe following annotations are supported:\n- post.hook.restore.velero.io/container\n- post.hook.restore.velero.io/command\n- post.hook.restore.velero.io/on-error\n- post.hook.restore.velero.io/exec-timeout\n- post.hook.restore.velero.io/wait-timeout\n\nInit restore hooks can be defined by annotation and/or in the new `initContainers` section in the Restore spec.\nThe initContainers schema is `pod.spec.initContainers`.\n\nThe following annotations are supported:\n- init.hook.restore.velero.io/timeout\n- init.hook.restore.velero.io/initContainers\n\nThis is an example of defining hooks in the Restore spec.\n\n```yaml\napiVersion: velero.io/v1\nkind: Restore\nspec:\n ...\n hooks:\n resources:\n -\n name: my-hook\n includedNamespaces:\n - '*'\n excludedNamespaces:\n - some-namespace\n includedResources:\n - pods\n excludedResources: []\n labelSelector:\n matchLabels:\n app: velero\n component: server\n post:\n -\n exec:\n container: postgres\n command:\n - /bin/bash\n - -c\n - rm /docker-entrypoint-initdb.d/dump.sql\n onError: Fail\n timeout: 10s\n readyTimeout: 60s\n init:\n timeout: 120s\n initContainers:\n - name: restore\n image: postgres:12\n command: [\"/bin/bash\", \"-c\", \"mv /backup/dump.sql /docker-entrypoint-initdb.d/\"]\n volumeMounts:\n - name: backup\n mountPath: /backup\n```\n\nAs with Backups, if an annotation is defined on a pod then no hooks from the Restore spec will be applied.\n\n### Implementation\n\nThe types and function in pkg/backup/item_hook_handler.go will be moved to a new package (pkg/hooks) and exported so they can be used for both backups and restores.\n\nThe post-restore hooks implementation will closely follow the design of restoring pod volumes with restic.\nThe pkg/restore.context type will have new fields `hooksWaitGroup` and `hooksErrs` comparable to `resticWaitGroup` and `resticErr`.\nThe pkg/restore.context.execute function will start a goroutine for each pod with applicable hooks and then continue with restoring other items.\nEach hooks goroutine will create a pkg/util/hooks.ItemHookHandler for each pod and send any error on the context.hooksErrs channel.\nThe ItemHookHandler already includes stdout and stderr and other metadata in the Backup log so the same logs will automatically be added to the Restore log (passed as the first argument to the ItemHookhandler.HandleHooks method.)\n\nThe pkg/restore.context.execute function will wait for the hooksWaitGroup before returning.\nAny errors received on context.hooksErrs will be added to errs.Velero.\n\nOne difference compared to the restic restore design is that any error on the context.hooksErrs channel will cancel the context of all hooks, since errors are only reported on this channel if the hook specified `onError: Fail`.\nHowever, canceling the hooks goroutines will not cancel the restic goroutines.\nIn practice the restic goroutines will complete before the hooks since the hooks do not run until a pod is ready, but it's possible a hook will be executed and fail while a different pod is still in the pod volume restore phase.\n\nFailed hooks with `onError: Continue` will appear in the Restore log but will not affect the status of the parent Restore.\nFailed hooks with `onError: Fail` will cause the parent Restore to have status Partially Failed.\n\nIf initContainers are specified for a pod, Velero will inject the containers into the beginning of the pod's initContainers list.\nIf a restic initContainer is also being injected, the restore initContainers will be injected directly after the restic initContainer.\nThe restore will use a RestoreItemAction to inject the initContainers.\nStdout and stderr of the restore initContainers will not be added to the Restore logs.\nInitContainers that fail will not affect the parent Restore's status.\n\n## Alternatives Considered\n\nWait for all restored Pods to report Ready, then execute the first hook in all applicable Pods simultaneously, then proceed to the next hook, etc.\nThat could introduce deadlock, e.g. if an API pod cannot be ready until the DB pod is restored.\n\nPut the restore hooks on the Backup spec as a third lifecycle event named `restore` along with `pre` and `post`.\nThat would be confusing since `pre` and `post` would appear in the Backup log but `restore` would only be in the Restore log.\n\nExecute restore hooks in parallel for each Pod.\nThat would not match the behavior of Backups.\n\nWait for PodStatus ready before executing the post-restore hooks in any container.\nThere are cases where the pod should not report itself ready until after the restore hook has run.\n\nInclude the logs from initContainers in the Restore log.\nUnlike exec hooks where stdout and stderr are permanently lost if not added to the Restore log, the logs of the injected initContainers are available through the K8s API with kubectl or another client.\n\n## Security Considerations\n\nStdout or stderr in the Restore log may contain sensitive information, but the same risk already exists for Backup hooks.\n" }, { "path": "design/Implemented/restore-with-EnableAPIGroupVersions-feature.md", "content": "# Restore API Group Version by Priority Level When EnableAPIGroupVersions Feature is Set\n\nStatus: Accepted\n\n## Abstract\n\nThis document proposes a solution to select an API group version to restore from the versions backed up using the feature flag EnableAPIGroupVersions.\n\n## Background\n\nIt is possible that between the time a backup has been made and a restore occurs that the target Kubernetes version has incremented more than one version. In such a case where at least a versions of Kubernetes was skipped, the preferred source cluster's API group versions for resources may no longer be supported by the target cluster. With [PR#2373](https://github.com/vmware-tanzu/velero/pull/2373), all supported API group versions were backed up if the EnableAPIGroupVersions feature flag was set for Velero. The next step (outlined by this design proposal) will be to see if any of the backed up versions are supported in the target cluster and if so, choose one to restore for each backed up resource.\n\n## Goals\n\n- Choose an API group to restore from backups given a priority system or a user-provided prioritization of versions.\n- Restore resources using the chosen API group version.\n\n## Non Goals\n\n- Allow users to restore onto a cluster that is running a Kubernetes version older than the source cluster. The changes proposed here only allow for skipping ahead to a newer Kubernetes version, but not going backward.\n- Allow restoring from backups created using Velero version 1.3 or older. This proposal will only work on backups created using Velero 1.4+.\n- Modifying the compressed backup tarball files. We don't want to risk corrupting the backups.\n- Using plugins to restore a resource when the target supports none of the source cluster's API group versions. The ability to use plugins will hopefully be something added in the future, but not at this time.\n\n## High-Level Design\n\nDuring restore, the proposal is that Velero will determine if the `APIGroupVersionsFeatureFlag` was enabled in the target cluster and `Status.FormatVersion 1.1.0` was used during backup. Only if these two conditions are met will the changes proposed here take effect.\n\nThe proposed code starts with creating three lists for each backed up resource. The three lists will be created by\n (1) reading the directory names in the backup tarball file and seeing which API group versions were backed up from the source cluster,\n (2) looking at the target cluster and determining which API group versions are supported, and\n (3) getting ConfigMaps from the target cluster in order to get user-defined prioritization of versions.\n\n The three lists will be used to create a map of chosen versions for each resource to restore. If there is a user-defined list of priority versions, the versions will be checked against the supported versions lists. The highest user-defined priority version that is/was supported by both target and source clusters will be the chosen version for that resource. If no user specified versions are supported by neither target nor source, the versions will be logged and the restore will continue with other prioritizations.\n\n Without a user-defined prioritization of versions, the following version prioritization will be followed, starting from the highest priority: target cluster preferred version, source cluster preferred version, and a common supported version. Should there be multiple common supported versions, the one that will be chosen will be based on the [Kubernetes version priorities](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning/#version-priority).\n \n Once the version to restore is chosen, the file path to the backed up resource in the tarball will be modified such that it points to the resources' chosen API group version. If no version is found in common between the source and target clusters, the chosen version will default to the source cluster's preferred version (the version being restored currently without the changes proposed here). Restore will be allowed to continue as before.\n\n## Detailed Design\n\nThere are six objectives to achieve the above stated goals:\n\n1. Determine if the APIGroupVersionsFeatureFlag is enabled and Backup Objects use Status.FormatVersion 1.1.0.\n1. List the backed up API group versions.\n1. List the API group versions supported by the target cluster.\n1. Get the user-defined version priorities.\n1. Use a priority system to determine which version to restore. The source preferred version will be the default if the priorities fail.\n1. Modify the paths to the backup files in the tarball in the resource restore process.\n\n### Objective 1: Determine if the APIGroupVersionsFeatureFlag is enabled and Backup Objects use Status.FormatVersion 1.1.0\n\nFor restore to be able to choose from multiple supported backed up versions, the feature flag must have been enabled during the restore processes. Backup objects must also have [Status.FormatVersion == \"1.1.0\"](https://github.com/vmware-tanzu/velero/blob/a1e182e723a8c5f6d4175d8db2361233a94d2502/pkg/backup/backup.go#L58).\n\nThe reason for checking for the feature flag during restore is to ensure the user would like to restore a version that might not be the source cluster preferred version. This check is done via `features.IsEnabled(velerov1api.APIGroupVersionsFeatureFlag)`.\n\nThe reason for checking `Status.FormatVersion` is to ensure the changes made by this proposed design is backward compatible. Only with Velero version 1.4 and forward was Format Version 1.1.0 used to structure the backup directories. Format Version 1.1.0 is required for the restore process proposed in this design doc to work. Before v1.4, the backed up files were in a directory structure that will not be recognized by the proposed code changes. In this case, restore should not attempt to restore from multiple versions as they will not exist.\n\nThe [`Status.FormatVersion`](https://github.com/vmware-tanzu/velero/blob/6808acd92e30848056a21faf373af03ddb8a3b71/pkg/apis/velero/v1/backup.go#L235) is stored in a `restoreContext` struct field called [`backup`](https://github.com/vmware-tanzu/velero/blob/6808acd92e30848056a21faf373af03ddb8a3b71/pkg/restore/restore.go#L229). The full chain is `ctx.backup.Status.FormatVersion`. \n\nThe above two checks can be done inside a new method on the `*restoreContext` object with the method signature `meetsAPIGVRestoreReqs() bool`. This method can remain in the `restore` package, but for organizational purposes, it can be moved to a file called `prioritize_group_version.go`.\n\n### Objective 2: List the backed up API group versions\n\nCurrently, in `pkg/restore/restore.go`, in the `execute(...)` method, around [line 363](https://github.com/vmware-tanzu/velero/blob/7a103b9eda878769018386ecae78da4e4f8dde83/pkg/restore/restore.go#L363), the resources and their backed up items are saved in a map called `backupResources`.\n\nAt this point, the feature flag and format versions can be checked (described in Objective #1). If the requirements are met, the `backedupResources` map can be sent to a method (to be created) with the signature `ctx.chooseAPIVersionsToRestore(backupResources)`. The `ctx` object has the type `*restore.Context`.\n\nThe `chooseAPIVersionsToRestore` method can remain in the `restore` package, but for organizational purposes, it can be moved to a file called `prioritize_group_version.go`.\n\nInside the `chooseAPIVersionsToRestore` method, we can take advantage of the `archive` package's `Parser` type. `ParseGroupVersions(backupDir string) (map[string]metav1.APIGroup, error)`. The `ParseGroupVersions(...)` method will loop through the `resources`, `resource.group`, and group version directories to populate a map called `sourceRGVersions`.\n\nThe `sourceRGVersions` map's keys will be strings in the format `.`, e.g. \"horizontalpodautoscalers.autoscaling\". The values will be APIGroup structs. The API Group struct can be imported from k8s.io/apimachinery/pkg/apis/meta/v1. Order the APIGroup.Versions slices using a sort function copied from `k8s.io/apimachinery/pkg/version`.\n\n```go\nsort.SliceStable(gvs, func(i, j int) bool {\n return version.CompareKubeAwareVersionStrings(gvs[i].Version, gvs[j].Version) > 0\n})\n```\n\n### Objective 3: List the API group versions supported by the target cluster\n\nStill within the `chooseAPIVersionsToRestore` method, the target cluster's resource group versions can now be obtained.\n\n```go\ntargetRGVersions := ctx.discoveryHelper.APIGroups()\n```\n\nOrder the APIGroup.Versions slices using a sort function copied from `k8s.io/apimachinery/pkg/version`.\n\n```go\nsort.SliceStable(gvs, func(i, j int) bool {\n return version.CompareKubeAwareVersionStrings(gvs[i].Version, gvs[j].Version) > 0\n})\n```\n\n### Objective 4: Get the user-defined version priorities\n\nStill within the `chooseAPIVersionsToRestore` method, the user-defined version priorities can be retrieved. These priorities are expected to be in a config map named `enableapigroupversions` in the `velero` namespace. An example config map is\n\n```yaml\napiVersion: v1\nkind: ConfigMap\nmetadata:\n name: enableapigroupversions\n  namespace: velero\ndata:\n  restoreResourcesVersionPriority: | -\n    rockbands.music.example.io=v2beta1,v2beta2\n    orchestras.music.example.io=v2,v3alpha1\n    subscriptions.operators.coreos.com=v2,v1\n```\n\nIn the config map, the resources and groups and the user-defined version priorities will be listed in the `data.restoreResourcesVersionPriority` field following the following general format: `.=[, ...]`.\n\nA map will be created to store the user-defined priority versions. The map's keys will be strings in the format `.`. The values will be APIGroup structs that will be imported from `k8s.io/apimachinery/pkg/apis/meta/v1`. Within the APIGroup structs will be versions in the order that the user provides in the config map. The PreferredVersion field in APIGroup struct will be left empty.\n\n### Objective 5: Use a priority system to determine which version to restore. The source preferred version will be the default if the priorities fail\n\nDetermining the priority will also be done in the `chooseAPIVersionsToRestore` method. Once a version is chosen, it will be stored in a new map of the form `map[string]ChosenGRVersion` where the key is the `.` and the values are of the `ChosenGroupVersion` struct type (shown below). The map will be saved to the `restore.Context` object in a field called `chosenGrpVersToRestore`.\n\n```go\ntype ChosenGroupVersion struct {\n Group string\n Version string\n Dir string\n}\n```\n\nThe first method called will be `ctx.gatherSTUVersions()` and it will gather the source cluster group resource and versions (`sgvs`), target cluster group versions (`tgvs`), and custom user resource and group versions (`ugvs`).\n\nLoop through the source cluster resource and group versions (`sgvs`). Find the versions for the group in the target cluster.\n\nAn attempt will first be made to `findSupportedUserVersion`. Loop through the resource.groups in the custom user resource and group versions (`ugvs`) map. If a version is supported by both `tgvs` and `sgvs`, that will be set as the chosen version for the corresponding resource in `ctx.chosenGrpVersToRestore`\n\nIf no three-way match can be made between the versions in `ugvs`, `tgvs`, and `sgvs`, move on to attempting to use the target cluster preferred version. Loop through the `sgvs` versions for the resource and see if any of them match the first item in the `tgvs` version list. Because the versions in `tgvs` have been ordered, the first version in the version slide will be the preferred version.\n\nIf target preferred version cannot be used, attempt to choose the source cluster preferred version. Loop through the target versions and see if any of them match the first item in the source version slice, which will be the preferred version due to Kubernetes version ordering.\n\nIf neither clusters' preferred version can be used, look through remaining versions in the target version list and see if there is a match with the remaining versions in the source versions list.\n\nIf none of the previous checks produce a chosen version, the source preferred version will be the default and the restore process will continue.\n\nHere is another way to list the priority versions described above:\n\n- **Priority 0** ((User override). Users determine restore version priority using a config map\n- **Priority 1**. Target preferred version can be used.\n- **Priority 2**. Source preferred version can be used.\n- **Priority 3**. A common supported version can be used. This means\n - target supported version == source supported version\n - if multiple support versions intersect, choose the version using the [Kubernetes’ version prioritization system](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning/#version-priority)\n\nIf there is no common supported version between target and source clusters, then the default `ChosenGRVersion` will be the source preferred version. This is the version that would have been assumed for restore before the changes proposed here.\n\nNote that adding a field to `restore.Context` will mean having to make a map for the field during instantiation.\n\nTo see example cases with version priorities, see a blog post written by Rafael Brito: https://github.com/brito-rafa/k8s-webhooks/tree/master/examples-for-projectvelero.\n\n### Objective 6: Modify the paths to the backup files in the tarball\n\nThe method doing the bulk of the restoration work is `ctx.restoreResource(...)`. Inside this method, around [line 714](https://github.com/vmware-tanzu/velero/blob/7a103b9eda878769018386ecae78da4e4f8dde83/pkg/restore/restore.go#L714) in `pkg/restore/restore.go`, the path to backup json file for the item being restored is set.\n\nAfter the groupResource is instantiated at pkg/restore/restore.go:733, and before the `for` loop that ranges through the `items`, the `ctx.chosenGRVsToRestore` map can be checked. If the groupResource exists in the map, the path saved to `resource` variable can be updated.\n\nCurrently, the item paths look something like\n\n```bash\n/var/folders/zj/vc4ln5h14djg9svz7x_t1d0r0000gq/T/620385697/resources/horizontalpodautoscalers.autoscaling/namespaces/myexample/php-apache-autoscaler.json\n```\n\nThis proposal will have the path changed to something like\n\n```bash\n/var/folders/zj/vc4ln5h14djg9svz7x_t1d0r0000gq/T/620385697/resources/horizontalpodautoscalers.autoscaling/v2beta2/namespaces/myexample/php-apache-autoscaler.json\n```\n\nThe `horizontalpodautoscalers.autoscaling` part of the path will be updated to `horizontalpodautoscalers.autoscaling/v2beta2` using\n\n```go\nversion, ok := ctx.chosenGVsToRestore[groupResource.String()]\n if ok {\n resource = filepath.Join(groupResource.String(), version.VerDir)\n }\n```\n\nThe restore can now proceed as normal.\n\n## Alternatives Considered\n\n- Look for plugins if no common supported API group version could be found between the target and source clusters. We had considered searching for plugins that could handle converting an outdated resource to a new one that is supported in the target cluster, but it is difficult, will take a lot of time, and currently will not be useful because we are not aware of such plugins. It would be better to keep the initial changes simple to see how it works out and progress to more complex solutions as demand necessitates.\n- It was considered to modify the backed up json files such that the resources API versions are supported by the target but modifying backups is discouraged for several reasons, including introducing data corruption.\n\n## Security Considerations\n\nI can't think of any additional risks in terms of Velero security here.\n\n## Compatibility\n\nI have made it such that the changes in code will only affect Velero installations that have `APIGroupVersionsFeatureFlag` enabled during restore and Format Version 1.1.0 was used during backup. If both these requirements are not met, the changes will have no affect on the restore process, making the changes here entirely backward compatible.\n\n## Implementation\n\nThis first draft of the proposal will be submitted Oct. 30, 2020. Once this proposal is approved, I can have the code and unit tests written within a week and submit a PR that fixes Issue #2551.\n\n## Open Issues\n\nAt the time of writing this design proposal, I had not seen any of @jenting's work for solving Issue #2551. He had independently covered the first two priorities I mentioned above before I was even aware of the issue. I hope to not let his efforts go to waste and welcome incorporating his ideas here to make this design proposal better.\n" }, { "path": "design/Implemented/retry-patching-configuration_design.md", "content": "# Backup Restore Status Patch Retrying Configuration\n\n## Abstract\nWhen a backup/restore completes, we want to ensure that the custom resource progresses to the correct status.\nIf a patch call fails to update status to completion, it should be retried up to a certain time limit.\n\nThis design proposes a way to configure timeout for this retry time limit.\n\n## Background\nOriginal Issue: https://github.com/vmware-tanzu/velero/issues/7207\n\nVelero was performing a restore when the API server was rolling out to a new version.\nIt had trouble connecting to the API server, but eventually, the restore was successful.\nHowever, since the API server was still in the middle of rolling out, Velero failed to update the restore CR status and gave up.\n\nAfter the connection was restored, it didn't attempt to update, causing the restore CR to be stuck at \"In progress\" indefinitely.\nThis can lead to incorrect decisions for other components that rely on the backup/restore CR status to determine completion.\n\n## Goals\n- Make timeout configurable for retry patching by reusing existing [`--resource-timeout` server flag](https://github.com/vmware-tanzu/velero/blob/d9ca14747925630664c9e4f85a682b5fc356806d/pkg/cmd/server/server.go#L245)\n\n## Non Goals\n- Create a new timeout flag\n- Refactor backup/restore workflow\n\n\n## High-Level Design\nWe will add retries with timeout to existing patch calls that moves a backup/restore from InProgress to a different status phase such as\n- FailedValidation (final)\n- Failed (final)\n- WaitingForPluginOperations\n- WaitingForPluginOperationsPartiallyFailed\n- Finalizing\n- FinalizingPartiallyFailed\n\nand from above non final phases to\n- Completed\n- PartiallyFailed\n\nOnce backup/restore is in some phase it will already be reconciled again periodically and do not need additional retry\n- WaitingForPluginOperations\n- WaitingForPluginOperationsPartiallyFailed\n\n## Detailed Design\nRelevant reconcilers will have `resourceTimeout time.Duration` added to its struct and to parameters of New[Backup|Restore]XReconciler functions.\n\npkg/cmd/server/server.go in `func (s *server) runControllers(..) error` also update the New[Backup|Restore]XCReconciler with added duration parameters using value from existing `--resource-timeout` server flag.\n\nCurrent calls to kube.PatchResource involving status patch will be replaced with kube.PatchResourceWithRetriesOnErrors added to package `kube` below.\n\nCalls where there is a ...client.Patch() will be wrapped with client.RetriesPhasePatchFuncOnErrors() added to package `client` below.\n\npkg/util/kube/client.go\n```go\n// PatchResourceWithRetries patches the original resource with the updated resource, retrying when the provided retriable function returns true.\nfunc PatchResourceWithRetries(maxDuration time.Duration, original, updated client.Object, kbClient client.Client, retriable func(error) bool) error {\n\treturn veleroPkgClient.RetryOnRetriableMaxBackOff(maxDuration, func() error { return PatchResource(original, updated, kbClient) }, retriable)\n}\n\n// PatchResourceWithRetriesOnErrors patches the original resource with the updated resource, retrying when the operation returns an error.\nfunc PatchResourceWithRetriesOnErrors(maxDuration time.Duration, original, updated client.Object, kbClient client.Client) error {\n\treturn PatchResourceWithRetries(maxDuration, original, updated, kbClient, func(err error) bool {\n\t\t// retry using DefaultBackoff to resolve connection refused error that may occur when the server is under heavy load\n\t\t// TODO: consider using a more specific error type to retry, for now, we retry on all errors\n\t\t// specific errors:\n\t\t// - connection refused: https://pkg.go.dev/syscall#:~:text=Errno(0x67)-,ECONNREFUSED,-%3D%20Errno(0x6f\n\t\treturn err != nil\n\t})\n}\n```\n\npkg/client/retry.go\n```go\n// CapBackoff provides a backoff with a set backoff cap\nfunc CapBackoff(cap time.Duration) wait.Backoff {\n\tif cap < 0 {\n\t\tcap = 0\n\t}\n\treturn wait.Backoff{\n\t\tSteps: math.MaxInt,\n\t\tDuration: 10 * time.Millisecond,\n\t\tCap: cap,\n\t\tFactor: retry.DefaultBackoff.Factor,\n\t\tJitter: retry.DefaultBackoff.Jitter,\n\t}\n}\n\n// RetryOnRetriableMaxBackOff accepts a patch function param, retrying when the provided retriable function returns true.\nfunc RetryOnRetriableMaxBackOff(maxDuration time.Duration, fn func() error, retriable func(error) bool) error {\n\treturn retry.OnError(CapBackoff(maxDuration), func(err error) bool { return retriable(err) }, fn)\n}\n\n// RetryOnErrorMaxBackOff accepts a patch function param, retrying when the error is not nil.\nfunc RetryOnErrorMaxBackOff(maxDuration time.Duration, fn func() error) error {\n\treturn RetryOnRetriableMaxBackOff(maxDuration, fn, func(err error) bool { return err != nil })\n}\n```\n\n## Alternatives Considered\n - Requeuing InProgress backups that is not known by current velero instance to still be in progress as failed (attempted in [#7863](https://github.com/vmware-tanzu/velero/pull/7863))\n - It was deemed as making backup restore flow hard to enhance for future reconciler updates such as adding cancel or adding parallel backups.\n\n## Security Considerations\nNone\n\n## Compatibility\nRetry should only trigger a restore or backup that is already in progress and not patching successfully by current instance. Prior InProgress backups/restores will not be re-processed and will remain stuck InProgress until there is another velero server (re)start.\n\n## Implementation\nThere is a past implementation in [#7845](https://github.com/vmware-tanzu/velero/pull/7845/) where implementation for this design will be based upon.\n\n" }, { "path": "design/Implemented/riav2-design.md", "content": "# Design for RestoreItemAction v2 API\n\n## Abstract\nThis design includes the changes to the RestoreItemAction (RIA) api design as required by the [Item Action Progress Monitoring](general-progress-monitoring.md) feature.\nIt also includes changes as required by the [Wait For Additional Items](wait-for-additional-items.md) feature.\nThe BIA v2 interface will have three new methods, and the RestoreItemActionExecuteOutput() struct in the return from Execute() will have three optional fields added.\nIf there are any additional RIA API changes that are needed in the same Velero release cycle as this change, those can be added here as well.\n\n## Background\nThis API change is needed to facilitate long-running plugin actions that may not be complete when the Execute() method returns.\nIt is an optional feature, so plugins which don't need this feature can simply return an empty operation ID and the new methods can be no-ops.\nThis will allow long-running plugin actions to continue in the background while Velero moves on to the next plugin, the next item, etc.\nThe other change allows Velero to wait until newly-restored AdditionalItems returned by a RIA plugin are ready before moving on to restoring the current item.\n\n## Goals\n- Allow for RIA Execute() to optionally initiate a long-running operation and report on operation status.\n- Allow for RIA to allow Velero to call back into the plugin to wait until AdditionalItems are ready before continuing with restore.\n\n## Non Goals\n- Allowing velero control over when the long-running operation begins.\n\n\n## High-Level Design\nAs per the [Plugin Versioning](plugin-versioning.md) design, a new RIAv2 plugin `.proto` file will be created to define the GRPC interface.\nv2 go files will also be created in `plugin/clientmgmt/restoreitemaction` and `plugin/framework/restoreitemaction`, and a new PluginKind will be created.\nChanges to RestoreItemActionExecuteOutput will be made to the existing struct.\nSince the new fields are optional elements of the struct, the new enlarged struct will work with both v1 and v2 plugins.\nThe velero Restore process will be modified to reference v2 plugins instead of v1 plugins.\nAn adapter will be created so that any existing RIA v1 plugin can be executed as a v2 plugin when executing a restore.\n\n## Detailed Design\n\n### proto changes (compiled into golang by protoc)\n\nThe v2 RestoreItemAction.proto will be like the current v1 version with the following changes:\nRestoreItemActionExecuteOutput gets three new fields (defined in the current (v1) RestoreItemAction.proto file:\n```\nmessage RestoreItemActionExecuteResponse {\n bytes item = 1;\n repeated ResourceIdentifier additionalItems = 2;\n bool skipRestore = 3;\n string operationID = 4;\n bool waitForAdditionalItems = 5;\n google.protobuf.Duration additionalItemsReadyTimeout = 6;\n}\n\n```\nThe RestoreItemAction service gets three new rpc methods:\n```\nservice RestoreItemAction {\n rpc AppliesTo(RestoreItemActionAppliesToRequest) returns (RestoreItemActionAppliesToResponse);\n rpc Execute(RestoreItemActionExecuteRequest) returns (RestoreItemActionExecuteResponse);\n rpc Progress(RestoreItemActionProgressRequest) returns (RestoreItemActionProgressResponse);\n rpc Cancel(RestoreItemActionCancelRequest) returns (google.protobuf.Empty);\n rpc AreAdditionalItemsReady(RestoreItemActionItemsReadyRequest) returns (RestoreItemActionItemsReadyResponse);\n}\n\n```\nTo support these new rpc methods, we define new request/response message types:\n```\nmessage RestoreItemActionProgressRequest {\n string plugin = 1;\n string operationID = 2;\n bytes restore = 3;\n}\n\nmessage RestoreItemActionProgressResponse {\n generated.OperationProgress progress = 1;\n}\n\nmessage RestoreItemActionCancelRequest {\n string plugin = 1;\n string operationID = 2;\n bytes restore = 3;\n}\n\nmessage RestoreItemActionItemsReadyRequest {\n string plugin = 1;\n bytes restore = 2;\n repeated ResourceIdentifier additionalItems = 3;\n}\nmessage RestoreItemActionItemsReadyResponse {\n bool ready = 1;\n}\n\n```\nOne new shared message type will be needed, as defined in the v2 BackupItemAction design:\n```\nmessage OperationProgress {\n bool completed = 1;\n string err = 2;\n int64 completed = 3;\n int64 total = 4;\n string operationUnits = 5;\n string description = 6;\n google.protobuf.Timestamp started = 7;\n google.protobuf.Timestamp updated = 8;\n}\n```\n\nIn addition to the three new rpc methods added to the RestoreItemAction interface, there is also a new `Name()` method. This one is only actually used internally by Velero to get the name that the plugin was registered with, but it still must be defined in a plugin which implements RestoreItemActionV2 in order to implement the interface. It doesn't really matter what it returns, though, as this particular method is not delegated to the plugin via RPC calls. The new (and modified) interface methods for `RestoreItemAction` are as follows:\n```\ntype BackupItemAction interface {\n...\n\tName() string\n...\n\tProgress(operationID string, restore *api.Restore) (velero.OperationProgress, error)\n\tCancel(operationID string, backup *api.Restore) error\n\tAreAdditionalItemsReady(AdditionalItems []velero.ResourceIdentifier, restore *api.Restore) (bool, error)\n...\n}\ntype RestoreItemActionExecuteOutput struct {\n\tUpdatedItem runtime.Unstructured\n\tAdditionalItems []ResourceIdentifier\n\tSkipRestore bool\n\tOperationID string\n\tWaitForAdditionalItems bool\n}\n\n```\n\nA new PluginKind, `RestoreItemActionV2`, will be created, and the restore process will be modified to use this plugin kind.\nSee [Plugin Versioning](plugin-versioning.md) for more details on implementation plans, including v1 adapters, etc.\n\n\n## Compatibility\nThe included v1 adapter will allow any existing RestoreItemAction plugin to work as expected, with no-op AreAdditionalItemsReady(), Progress(), and Cancel() methods.\n\n## Implementation\nThis will be implemented during the Velero 1.11 development cycle.\n" }, { "path": "design/Implemented/schedule-skip-immediately-config_design.md", "content": "# Schedule Skip Immediately Config Design\n## Abstract\nWhen unpausing schedule, a backup could be due immediately.\nNew Schedules also create new backup immediately.\n\nThis design allows user to *skip **immediately due** backup run upon unpausing or schedule creation*.\n\n## Background\nCurrently, the default behavior of schedule when `.Status.LastBackup` is nil or is due immediately after unpausing, a backup will be created. This may not be a desired by all users (https://github.com/vmware-tanzu/velero/issues/6517)\n\nUser want ability to skip the first immediately due backup when schedule is unpaused and or created.\n\nIf you create a schedule with cron \"45 * * * *\" and pause it at say the 43rd minute and then unpause it at say 50th minute, a backup gets triggered (since .Status.LastBackup is nil or >60min ago).\n\nWith this design, user can skip the first immediately due backup when schedule is unpaused and or created.\n\n## Goals\n- Add an option so user can when unpausing (when immediately due) or creating new schedule, to not create a backup immediately.\n\n## Non Goals\n- Changing the default behavior\n\n## High-Level Design\nAdd a new field with to the schedule spec and as a new cli flags for install, server, schedule commands; allowing user to skip immediately due backup when unpausing or schedule creation.\n\nIf CLI flag is specified during schedule unpause, velero will update the schedule spec accordingly and override prior spec for `skipImmediately``.\n\n## Detailed Design\n### CLI Changes\n`velero schedule unpause` will now take an optional bool flag `--skip-immediately` to allow user to override the behavior configured for velero server (see `velero server` below).\n\n`velero schedule unpause schedule-1 --skip-immediately=false` will unpause the schedule but not skip the backup if due immediately from `Schedule.Status.LastBackup` timestamp. Backup will be run at the next cron schedule.\n\n`velero schedule unpause schedule-1 --skip-immediately=true` will unpause the schedule and skip the backup if due immediately from `Schedule.Status.LastBackup` timestamp. Backup will also be run at the next cron schedule.\n\n`velero schedule unpause schedule-1` will check `.spec.SkipImmediately` in the schedule to determine behavior. This field will default to false to maintain prior behavior.\n\n`velero server` will add a new flag `--schedule-skip-immediately` to configure default value to patch new schedules created without the field. This flag will default to false to maintain prior behavior if not set.\n\n`velero install` will add a new flag `--schedule-skip-immediately` to configure default value to patch new schedules created without the field. This flag will default to false to maintain prior behavior if not set.\n\n### API Changes\n`pkg/apis/velero/v1/schedule_types.go`\n```diff\n// ScheduleSpec defines the specification for a Velero schedule\ntype ScheduleSpec struct {\n\t// Template is the definition of the Backup to be run\n\t// on the provided schedule\n\tTemplate BackupSpec `json:\"template\"`\n\n\t// Schedule is a Cron expression defining when to run\n\t// the Backup.\n\tSchedule string `json:\"schedule\"`\n\n\t// UseOwnerReferencesBackup specifies whether to use\n\t// OwnerReferences on backups created by this Schedule.\n\t// +optional\n\t// +nullable\n\tUseOwnerReferencesInBackup *bool `json:\"useOwnerReferencesInBackup,omitempty\"`\n\n\t// Paused specifies whether the schedule is paused or not\n\t// +optional\n\tPaused bool `json:\"paused,omitempty\"`\n\n+\t// SkipImmediately specifies whether to skip backup if schedule is due immediately from `Schedule.Status.LastBackup` timestamp when schedule is unpaused or if schedule is new.\n+\t// If true, backup will be skipped immediately when schedule is unpaused if it is due based on .Status.LastBackupTimestamp or schedule is new, and will run at next schedule time.\n+\t// If false, backup will not be skipped immediately when schedule is unpaused, but will run at next schedule time.\n+\t// If empty, will follow server configuration (default: false). \n+\t// +optional\n+\tSkipImmediately bool `json:\"skipImmediately,omitempty\"`\n}\n```\n\n**Note:** The Velero server automatically patches the `skipImmediately` field back to `false` after it's been used. This is because `skipImmediately` is designed to be a one-time operation rather than a persistent state. When the controller detects that `skipImmediately` is set to `true`, it:\n1. Sets the flag back to `false`\n2. Records the current time in `schedule.Status.LastSkipped`\n\nThis \"consume and reset\" pattern ensures that after skipping one immediate backup, the schedule returns to normal behavior for subsequent runs. The `LastSkipped` timestamp is then used to determine when the next backup should run.\n\n```go\n// From pkg/controller/schedule_controller.go\nif schedule.Spec.SkipImmediately != nil && *schedule.Spec.SkipImmediately { \n *schedule.Spec.SkipImmediately = false \n schedule.Status.LastSkipped = &metav1.Time{Time: c.clock.Now()} \n} \n```\n\n`LastSkipped` will be added to `ScheduleStatus` struct to track the last time a schedule was skipped.\n```diff\n// ScheduleStatus captures the current state of a Velero schedule\ntype ScheduleStatus struct {\n\t// Phase is the current phase of the Schedule\n\t// +optional\n\tPhase SchedulePhase `json:\"phase,omitempty\"`\n\n\t// LastBackup is the last time a Backup was run for this\n\t// Schedule schedule\n\t// +optional\n\t// +nullable\n\tLastBackup *metav1.Time `json:\"lastBackup,omitempty\"`\n\n+\t// LastSkipped is the last time a Schedule was skipped\n+\t// +optional\n+\t// +nullable\n+\tLastSkipped *metav1.Time `json:\"lastSkipped,omitempty\"`\n\n\t// ValidationErrors is a slice of all validation errors (if\n\t// applicable)\n\t// +optional\n\tValidationErrors []string `json:\"validationErrors,omitempty\"`\n}\n```\n\nThe `LastSkipped` field is crucial for the schedule controller to determine the next run time. When a backup is skipped, this timestamp is used instead of `LastBackup` to calculate when the next backup should occur, ensuring the schedule maintains its intended cadence even after skipping a backup.\n\nWhen `schedule.spec.SkipImmediately` is `true`, `LastSkipped` will be set to the current time, and `schedule.spec.SkipImmediately` set to nil so it can be used again.\n\nThe `getNextRunTime()` function below is updated so `LastSkipped` which is after `LastBackup` will be used to determine next run time.\n\n```go\nfunc getNextRunTime(schedule *velerov1.Schedule, cronSchedule cron.Schedule, asOf time.Time) (bool, time.Time) {\n\tvar lastBackupTime time.Time\n\tif schedule.Status.LastBackup != nil {\n\t\tlastBackupTime = schedule.Status.LastBackup.Time\n\t} else {\n\t\tlastBackupTime = schedule.CreationTimestamp.Time\n\t}\n\tif schedule.Status.LastSkipped != nil && schedule.Status.LastSkipped.After(lastBackupTime) {\n\t\tlastBackupTime = schedule.Status.LastSkipped.Time\n\t}\n\n\tnextRunTime := cronSchedule.Next(lastBackupTime)\n\n\treturn asOf.After(nextRunTime), nextRunTime\n}\n```\n\nWhen schedule is unpaused, and `Schedule.Status.LastBackup` is not nil, if `Schedule.Status.LastSkipped` is recent, a backup will not be created.\n\nWhen schedule is unpaused or created with `Schedule.Status.LastBackup` set to nil or schedule is newly created, normally a backup will be created immediately. If `Schedule.Status.LastSkipped` is recent, a backup will not be created.\n\nBackup will be run at the next cron schedule based on LastBackup or LastSkipped whichever is more recent.\n\n## Alternatives Considered\n\nN/A\n\n\n## Security Considerations\nNone\n\n## Compatibility\nUpon upgrade, the new field will be added to the schedule spec automatically and will default to the prior behavior of running a backup when schedule is unpaused if it is due based on .Status.LastBackup or schedule is new.\n\nSince this is a new field, it will be ignored by older versions of velero.\n\n## Implementation\nTBD\n\n## Open Issues\nN/A\n" }, { "path": "design/Implemented/secrets.md", "content": "# Support for multiple provider credentials\n\nCurrently, Velero only supports a single credential secret per location provider/plugin. Velero creates and stores the plugin credential secret under the hard-coded key `secret.cloud-credentials.data.cloud`.\n\nThis makes it so switching from one plugin to another necessitates overriding the existing credential secret with the appropriate one for the new plugin.\n\n## Goals\n\n- To allow Velero to create and store multiple secrets for provider credentials, even multiple credentials for the same provider\n- To improve the UX for configuring the velero deployment with multiple plugins/providers.\n- Enable use cases such as AWS volume snapshots w/Minio as the object storage\n- Continue to support use cases where multiple Backup Storage Locations are in use simultaneously \n - `velero backup logs` while backup/restore is running\n- Handle changes in configuration while operations are happening as well as they currently are\n\n## Non Goals\n\n- To make any change except what's necessary to handle multiple credentials\n- To allow multiple credentials for or change the UX for node-based authentication (e.g. AWS IAM, GCP Workload Identity, Azure AAD Pod Identity). Node-based authentication will not allow cases such as a mix of AWS snapshots with Minio object storage.\n\n## Design overview\n\nInstead of one credential per Velero deployment, multiple credentials can be added and used with different BSLs.\n \nThere are two aspects to handling multiple credentials:\n\n- Modifying how credentials are configured and specified by the user\n- Modifying how credentials are provided to the plugin processes\n\nEach of these aspects will be discussed in turn.\n\n### Credential configuration\n\nCurrently, Velero creates a secret (`cloud-credentials`) during install with a single entry that contains the contents of the credentials file passed by the user.\n\nInstead of adding new CLI options to Velero to create and manage credentials, users will create their own Kubernetes secrets within the Velero namespace and reference these.\nThis approach is being chosen as it allows users to directly manage Kubernetes secrets objects as they wish and it removes the need for wrapper functions to be created within Velero to manage the creation of secrets. Separate credentials rather than combining credentials in a single secret also avoids issues with maximum size of credentials as well as update in place issues. \n\nTo enable the use of existing Kubernetes secrets, BSLs will be modified to have a new field `Credential`.\nThis field will be a [`SecretKeySelector`](https://godoc.org/k8s.io/api/core/v1#SecretKeySelector) which will enable the user to specify which key within a particular secret the BSL should use.\n\nExisting BackupStorageLocationSpec definition:\n\n\t// BackupStorageLocationSpec defines the desired state of a Velero BackupStorageLocation\n\ttype BackupStorageLocationSpec struct {\n\t\t// Provider is the provider of the backup storage.\n\t\tProvider string `json:\"provider\"`\n\t\n\t\t// Config is for provider-specific configuration fields.\n\t\t// +optional\n\t\tConfig map[string]string `json:\"config,omitempty\"`\n\t\n\t\tStorageType `json:\",inline\"`\n\t\n\t\t// Default indicates this location is the default backup storage location.\n\t\t// +optional\n\t\tDefault bool `json:\"default,omitempty\"`\n\t\n\t\t// AccessMode defines the permissions for the backup storage location.\n\t\t// +optional\n\t\tAccessMode BackupStorageLocationAccessMode `json:\"accessMode,omitempty\"`\n\t\n\t\t// BackupSyncPeriod defines how frequently to sync backup API objects from object storage. A value of 0 disables sync.\n\t\t// +optional\n\t\t// +nullable\n\t\tBackupSyncPeriod *metav1.Duration `json:\"backupSyncPeriod,omitempty\"`\n\t\n\t\t// ValidationFrequency defines how frequently to validate the corresponding object storage. A value of 0 disables validation.\n\t\t// +optional\n\t\t// +nullable\n\t\tValidationFrequency *metav1.Duration `json:\"validationFrequency,omitempty\"`\n\t}\n\nThe following field will be added:\n\n\tCredential *corev1api.SecretKeySelector `json:\"credential,omitempty\"`\n\nThe resulting BackupStorageLocationSpec will be this:\n\n\t// BackupStorageLocationSpec defines the desired state of a Velero BackupStorageLocation\n\ttype BackupStorageLocationSpec struct {\n\t\t// Provider is the provider of the backup storage.\n\t\tProvider string `json:\"provider\"`\n\t\n\t\t// Config is for provider-specific configuration fields.\n\t\t// +optional\n\t\tConfig map[string]string `json:\"config,omitempty\"`\n\t\n\t\t// Credential contains the credential information intended to be used with this location\n\t\t// +optional\n\t\tCredential *corev1api.SecretKeySelector `json:\"credential,omitempty\"`\n\t\n\t\tStorageType `json:\",inline\"`\n\t\n\t\t// Default indicates this location is the default backup storage location.\n\t\t// +optional\n\t\tDefault bool `json:\"default,omitempty\"`\n\t\n\t\t// AccessMode defines the permissions for the backup storage location.\n\t\t// +optional\n\t\tAccessMode BackupStorageLocationAccessMode `json:\"accessMode,omitempty\"`\n\t\n\t\t// BackupSyncPeriod defines how frequently to sync backup API objects from object storage. A value of 0 disables sync.\n\t\t// +optional\n\t\t// +nullable\n\t\tBackupSyncPeriod *metav1.Duration `json:\"backupSyncPeriod,omitempty\"`\n\t\n\t\t// ValidationFrequency defines how frequently to validate the corresponding object storage. A value of 0 disables validation.\n\t\t// +optional\n\t\t// +nullable\n\t\tValidationFrequency *metav1.Duration `json:\"validationFrequency,omitempty\"`\n\t}\n\t\nThe CLI for managing Backup Storage Locations (BSLs) will be modified to allow the user to set these credentials.\nBoth `velero backup-location (create|set)` will have a new flag (`--credential`) to specify the secret and key within the secret to use.\nThis flag will take a key-value pair in the format `=`.\nThe arguments will be validated to ensure that the secret exists in the Velero namespace.\nIf the Credential field is empty in a BSL, the default credentials from `cloud-credentials` will be used as they\nare currently.\n\n### Making credentials available to plugins\n\nThe approach we have chosen is to include the path to the credentials file in the `config` map passed to a plugin.\n\n### Including the credentials file path in the `config` map\n\nPrior to using any secret for a BSL, it will need to be serialized to disk.\nUsing the details in the `Credential` field in the BSL, the contents of the Secret will be read and serialized.\nTo achieve this, we will create a new package, `credentials`, which will introduce new types and functions to manage the fetching of credentials based on a `SecretKeySelector`.\nThis will also be responsible for serializing the fetched credentials to a temporary directory on the Velero pod filesystem.\n\nThe path where a set of credentials will be written to will be a fixed path based on the namespace, name, and key from the secret rather than a randomly named file as is usual with temporary files.\nThe reason for this is that `BackupStore`s are frequently created within the controllers and the credentials must be serialized before any plugin APIs are called, which would result in a quick accumulation of temporary credentials files.\nFor example, the default validation frequency for BackupStorageLocations is one minute.\nThis means that any time a `BackupStore`, or other type which requires credentials, is created, the credentials will be fetched from the API server and may overwrite any existing use of that credential.\n\nIf we instead wanted to use an unique file each time, we could work around the of multiple files being written by cleaning up the temporary files upon completion of the plugin operations, if this information is known.\n\nOnce the credentials have been serialized, this path will be made available to the plugins.\nInstead of setting the necessary environment variable for the plugin process, the `config` map for the BSL will be modified to include an addiitional entry with the path to the credentials file: `credentialsFile`.\nThis will be passed through when [initializing the BSL](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/velero/object_store.go#L27-L30) and it will be the responsibility of the plugin to use the passed credentials when starting a session.\nFor an example of how this would affect the AWS plugin, see [this PR](https://github.com/vmware-tanzu/velero-plugin-for-aws/pull/69).\n\nThe restic controllers will also need to be updated to use the correct credentials.\nThe BackupStorageLocation for a given PVB/PVR will be fetched and the `Credential` field from that BSL will be serialized.\nThe existing setup for the restic commands use the credentials from the environment variables with [some repo provider specific overrides](https://github.com/vmware-tanzu/velero/blob/main/pkg/controller/pod_volume_backup_controller.go#L260-L273).\nInstead of relying on the existing environment variables, if there are credentials for a particular BSL, the environment will be specifically created for each `RepoIdentifier`.\nThis will use a lot of the existing logic with the exception that it will be modified to work with a serialized secret rather than find the secret file from an environment variable.\nCurrently, GCP is the only provider that relies on the existing environment variables with no specific overrides.\nFor GCP, the environment variable will be overwritten with the path of the serialized secret.\n\n\n## Split credentials between VolumeSnapshotter and ObjectStore plugins\n\nOne of the use cases we wish to satisfy is the ability to specify a different object store than the cloud provider offers,\nfor example, using a Minio S3 object store from within AWS. Currently the VolumeSnapshotter and the ObjectStore plugin\nshare the cloud credentials. Each backup/restore has a BackupStorageLocation associated\nwith it. The BackupStorageLocation can optionally specify the credential used by the ObjectStorePlugin and Restic daemons \nwhile the cloud credential will always be used for the VolumeSnapshotter.\n\n## Velero Plugin for vSphere compatibility\n\nThe vSphere plugin is implemented as a BackupItemAction and shares the credentials of the AWS plugin for S3 access.\nThe backup storage location is passed in _Backup.Spec.StorageLocation_. Currently the plugin retrieves the S3 bucket and\nserver from the BSL and creates a BackupRespositoryClaim with that and the credentials retrieved from the cloud credential.\nThe plugin will need to be modified to retrieve the credentials field from the BSL and use that credential in the\nBackupRepositoryClaim.\n\n## Backwards compatibility\n\nFor now, regardless of the approaches used above, we will still support the existing workflow.\n\nUsers will be able to set credentials during install and a secret will be created for them.\nThis secret will still be mounted into the Velero pods and the appropriate environment variables set.\nThis will allow users to use versions of plugins which haven't yet been updated to use credentials directly, such as with many community created plugins.\n\nMultiple credential handling will only be used in the case where a particular BSL has been modified to use an existing secret.\n\n## Security Considerations\n\nAlthough the handling of secrets will be similar to how credentials are currently managed within Velero, care must be taken to ensure that any new code does not leak the contents of secrets, for example, including them within logs.\n\n## Parallelism\nIn order to support parallelism, Velero will need to be able to use multiple credentials simultaneously with the\nObjectStore. Currently backups are single threaded and a single BSL will be used throughout the entire backup. The only\nexisting points of parallelism are when a user downloads logs for a backup or the BackupStorageLocationReconciler \nreconciles while a backup or restore is running. In the current code, `download_request_controller.go` and \n`backup_storage_location_controller.go` create a new plugin manager and hence another ObjectStore plugin in \nparallel with the ObjectStore plugin servicing a backup or restore (if one is running).\n\n## Alternatives Considered\n\nThree different approaches can be taken to provide credentials to plugin processes:\n\n1. Providing the path to the credentials file as an environment variable per plugin. This is how credentials are currently passed.\n1. Include the path to the credentials file in the `config` map passed to a plugin.\n1. Include the details of the secret in the `config` map passed to a plugin.\n\nThe last two options require changes to the plugin as the plugin will need to instantiate a client using the provided credentials.\nThe client libraries used by the plugins will not be able to rely on the credentials details being available in the environment as they currently do.\n\nWe have selected option 2 as the approach to take.\n\nThe approaches that were not selected are detailed below for reference.\n\n#### Providing the credentials via environment variables\n\nTo continue to provide the credentials via the environment, plugins will need to be invoked differently so that the correct credential is used.\nCurrently, there is a single secret, which is mounted into every pod deployed by Velero (the Velero Deployment and the Restic DaemonSet) at the path `/credentials/cloud`.\nThis path is made known to all plugins through provider specific environment variables and all possible provider environment variables are set to this path.\n\nInstead of setting the environment variables for all the pods, we can modify plugin processes are created so that the environment variables are set on a per plugin process basis.\nPrior to using any secret for a BSL, it will need to be serialized to disk.\nUsing the details in the `Credential` field in the BSL, the contents of the Secret will be read and serialized to a file.\nEach plugin process would still have the same set of environment variables set, however the value used for each of these variables would instead be the path to the serialized secret.\n\nTo set the environment variables for a plugin process, the plugin manager must be modified so that when creating an ObjectStore, we pass in the entire BSL object, rather than [just the provider](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/clientmgmt/manager.go#L132-L158).\nThe plugin manager currently stores a map of [plugin executables to an associated `RestartableProcess`](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/clientmgmt/manager.go#L59-L70).\nNew restartable processes are created only [with the executable that the process would run](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/clientmgmt/manager.go#L122).\nThis could be modified to also take the necessary environment variables so that when [underlying go-plugin process is created](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/clientmgmt/client_builder.go#L78), these environment variables could be provided and would be set on the plugin process.\n\nTaking this approach would not require any changes from plugins as the credentials information would be made available to them in the same way.\nHowever, it is quite a significant change in how we initialize and invoke plugins.\n\nWe would also need to ensure that the restic controllers are updated in the same way so that correct credentials are used (when creating a `ResticRepository` or processing `PodVolumeBackup`/`PodVolumeRestore`).\nThis could be achieved by modifying the existing function to [run a restic command](https://github.com/vmware-tanzu/velero/blob/main/pkg/restic/repository_manager.go#L237-L290).\nThis function already sets environment variables for the restic process depending on which storage provider is being used.\n\n#### Include the details of the secret in `config` map passed to a plugin\n\nThis approach is like the selected approach of passing the credentials file via the `config` map, however instead of the Velero process being responsible for serializing the file to disk prior to invoking the plugin, the `Credential SecretKeySelector` details will be passed through to the plugin.\nIt will be the responsibility of the plugin to fetch the secret from the Kubernetes API and perform the necessary steps to make it available for use when creating a session, for example, serializing the contents to disk, or evaluating the contents and adding to the process environment.\n\nThis approach has an additional burden on the plugin author over the previous approach as it requires the author to create a client to communicate with the Kubernetes API to retrieve the secret.\nAlthough it would be the responsibility of the plugin to serialize the credential and use it directly, Velero would still be responsible for serializing the secret so that it could be used with the restic controllers as in the selected approach.\n" }, { "path": "design/Implemented/supporting-volumeattributes-resource-policy.md", "content": "# Adding Support For VolumeAttributes in Resource Policy\n \n## Abstract\nCurrently [Velero Resource policies](https://velero.io/docs/main/resource-filtering/#creating-resource-policies) are only supporting \"Driver\" to be filtered for [CSI volume conditions](https://github.com/vmware-tanzu/velero/blob/8e23752a6ea83f101bd94a69dcf17f519a805388/internal/resourcepolicies/volume_resources_validator.go#L28)\n\nIf user want to skip certain CSI volumes based on other volume attributes like protocol or SKU, etc, they can't do it with the current Velero resource policies. It would be convenient if Velero resource policies could be extended to filter on volume attributes along with existing driver filter in the resource policies `conditions` to handle the backup of volumes just by `some specific volumes attributes conditions`.\n \n## Background\nAs of Today, Velero resource policy already provides us the way to filter volumes based on the `driver` name. But it's not enough to handle the volumes based on other volume attributes like protocol, SKU, etc.\n\n## Example:\n - Provision Azure NFS: Define the Storage class with `protocol: nfs` under storage class parameters to provision [CSI NFS Azure File Shares](https://learn.microsoft.com/en-us/azure/aks/azure-files-csi#nfs-file-shares).\n - User wants to back up AFS (Azure file shares) but only want to backup `SMB` type of file share volumes and not `NFS` file share volumes.\n\n## Goals\n- We are only bringing additional support in the resource policy to only handle volumes during backup.\n- Introducing support for `VolumeAttributes` filter along with `driver` filter in CSI volume conditions to handle volumes.\n \n## Non-Goals\n- Currently, only handles volumes, and does not support other resources.\n \n## Use-cases/Scenarios\n### Skip backup volumes by some volume attributes:\nUsers want to skip PV with the requirements:\n- option to skip specified PV on volume attributes type (like Protocol as NFS, SMB, etc)\n\n### Sample Storage Class Used to create such Volumes\n```\napiVersion: storage.k8s.io/v1\nkind: StorageClass\nmetadata:\n name: azurefile-csi-nfs\nprovisioner: file.csi.azure.com\nallowVolumeExpansion: true\nparameters:\n protocol: nfs \n```\n\n## High-Level Design\nModifying the existing Resource Policies code for [csiVolumeSource](https://github.com/vmware-tanzu/velero/blob/8e23752a6ea83f101bd94a69dcf17f519a805388/internal/resourcepolicies/volume_resources_validator.go#L28C6-L28C22) to add the new `VolumeAttributes` filter for CSI volumes and adding validations in existing [csiCondition](https://github.com/vmware-tanzu/velero/blob/8e23752a6ea83f101bd94a69dcf17f519a805388/internal/resourcepolicies/volume_resources.go#L150) to match with volume attributes in the conditions from Resource Policy config map and original persistent volume.\n\n## Detailed Design\nThe volume resources policies should contain a list of policies which is the combination of conditions and related `action`, when target volumes meet the conditions, the related `action` will take effection.\n\nBelow is the API Design for the user configuration:\n\n### API Design\n```go\ntype csiVolumeSource struct {\n\tDriver string `yaml:\"driver,omitempty\"`\n\t// [NEW] CSI volume attributes\n\tVolumeAttributes map[string]string `yaml:\"volumeAttributes,omitempty\"`\n}\n```\n\nThe policies YAML config file would look like this:\n```yaml\nversion: v1\nvolumePolicies:\n - conditions:\n csi:\n driver: disk.csi.azure.com\n action:\n type: skip\n - conditions:\n csi:\n driver: file.csi.azure.com\n volumeAttributes:\n protocol: nfs\n action:\n type: skip`\n```\n \n### New Supported Conditions\n#### VolumeAttributes\nExisting CSI Volume Condition can now add `volumeAttributes` which will be key and value pairs. \n\n Specify details for the related volume source (currently only csi driver is supported filter)\n ```yaml\n csi: // match volume using `file.csi.azure.com` and with volumeAttributes protocol as nfs\n driver: file.csi.azure.com \n volumeAttributes:\n protocol: nfs\n ```" }, { "path": "design/Implemented/unified-repo-and-kopia-integration/unified-repo-and-kopia-integration.md", "content": "# Unified Repository & Kopia Integration Design\n\n## Glossary & Abbreviation\n\n**BR**: Backup & Restore \n**Backup Storage**: The storage that meets BR requirements, for example, scalable, durable, cost-effective, etc., therefore, Backup Storage is usually implemented as Object storage or File System storage, it may be on-premises or in cloud. Backup Storage is not BR specific necessarily, so it usually doesn’t provide most of the BR related features. On the other hand, storage vendors may provide BR specific storages that include some BR features like deduplication, compression, encryption, etc. For a standalone BR solution (i.e. Velero), the Backup Storage is not part of the solution, it is provided by users, so the BR solution should not assume the BR related features are always available from the Backup Storage. \n**Backup Repository**: Backup repository is layered between BR data movers and Backup Storage to provide BR related features. Backup Repository is a part of BR solution, so generally, BR solution by default leverages the Backup Repository to provide the features because Backup Repository is always available; when Backup Storage provides duplicated features, and the latter is more beneficial (i.e., performance is better), BR solution should have the ability to opt to use the Backup Storage’s implementation. \n**Data Mover**: The BR module to read/write data from/to workloads, the aim is to eliminate the differences of workloads. \n**TCO**: Total Cost of Ownership. This is a general criteria for products/solutions, but also means a lot for BR solutions. For example, this means what kind of backup storage (and its cost) it requires, the retention policy of backup copies, the ways to remove backup data redundancy, etc. \n**RTO**: Recovery Time Objective. This is the duration of time that users’ business can recover after a disaster. \n\n## Background\n\nAs a Kubernetes BR solution, Velero is pursuing the capability to back up data from the volatile and limited production environment into the durable, heterogeneous and scalable backup storage. This relies on two parts: \n\n- Move data from various production workloads. The data mover has this role. Depending on the type of workload, Velero needs different data movers. For example, file system data mover, block data mover, and data movers for specific applications. At present, Velero supports moving file system data from PVs through Restic, which plays the role of the File System Data Mover. \n- Persist data in backup storage. For a BR solution, this is the responsibility of the backup repository. Specifically, the backup repository is required to: \n - Efficiently save data so as to reduce TCO. For example, deduplicate and compress the data before saving it\n - Securely save data so as to meet security criteria. For example, encrypt the data on rest, make the data immutable after backup, and detect/protect from ransomware\n - Efficiently retrieve data during restore so as to meet RTO. For example, restore a small unit of data or data associated with a small span of time \n - Effectively manage data from all kinds of data movers in all kinds of backup storage. This means 2 things: first, apparently, backup storages are different from each other; second, some data movers may save quite different data from others, for example, some data movers save a portion of the logical object for each backup and need to visit and manage the portions as an entire logic object, aka. incremental backup. The backup repository needs to provide unified functionalities to eliminate the differences from the both ends\n - Provide scalabilities so that users could assign resources (CPU, memory, network, etc.) in a flexible way to the backup repository since backup repository contains resource consuming modules\n\nAt present, Velero provides some of these capabilities by leveraging Restic (e.g., deduplication and encryption on rest). This means that in addition to being a data mover for file system level data, Restic also plays the role of a backup repository, albeit one that is incomplete and limited: \n\n- Restic is an inseparable unit made up of a file system data mover and a repository. This means that the repository capabilities are only available for Restic file system backup. We cannot provide the same capabilities to other data movers using Restic.\n- The backup storage Velero supports through our Restic backup path depends on the storage Restic supports. As a result, if there is a requirement to introduce backup storage that Restic doesn’t support, we have no way to make it.\n- There is no way to enhance or extend the repository capabilities, because of the same reason – Restic is an inseparable unit, we cannot insert one or more customized layers to make the enhancements and extensions.\n\nMoreover, as reflected by user-reported issues, Restic seems to have many performance issues on both the file system data mover side and the repository side. \n\nOn the other hand, based on a previous analysis and testing, we found that Kopia has better performance, with more features and more suitable to fulfill Velero’s repository targets (Kopia’s architecture divides modules more clearly according to their responsibilities, every module plays a complete role with clear interfaces. This makes it easier to take individual modules to Velero without losing critical functionalities). \n\n## Goals\n\n- Define a Unified Repository Interface that various data movers could interact with. This is for below purposes:\n - All kinds of data movers acquire the same set of backup repository capabilities very easily\n - Provide the possibility to plugin in different backup repositories/backup storages without affecting the upper layers\n - Provide the possibility to plugin in modules between data mover and backup repository, so as to extend the repository capabilities\n - Provide the possibility to scale the backup repository without affecting the upper layers\n- Use Kopia repository to implement the Unified Repository\n- Use Kopia uploader as the file system data mover for Pod Volume Backup\n- Have Kopia uploader calling the Unified Repository Interface and save/retrieve data to/from the Unified Repository\n- Make Kopia uploader generic enough to move any file system data so that other data movement cases could use it\n- Use the existing logic or add new logic to manage the unified repository and Kopia uploader\n- Preserve the legacy Restic path, this is for the consideration of backward compatibility\n\n## Non-Goals\n\n- The Unified Repository supports all kinds of data movers to save logic objects into it. How these logic objects are organized for a specific data mover (for example, how a volume’s block data is organized and represented by a unified repository object) should be included in the related data mover design.\n- At present, Velero saves Kubernetes resources, backup metedata, debug logs separately. Eventually, we want to save them in the Unified Repository. How to organize these data into the Unified Repository should be included in a separate design.\n- For PodVolume BR, this design focuses on the data path only, other parts beyond the data read/write and data persistency are irrelevant and kept unchanged. \n- Kopia uploader is made generic enough to move any file system data. How it is integrated in other cases, is irrelevant to this design. Take CSI snapshot backup for example, how the snapshot is taken and exposed to Kopia uploader should be included in the related data mover design.\n- The adanced modes of the Unified Repository, for example, backup repository/storage plugin, backup repository extension, etc. are not included in this design. We will have separate designs to cover them whenever necessary.\n\n## Architecture of Unified Repository\n\nBelow shows the primary modules and their responsibilities:\n\n- Kopia uploader, as been well isolated, could move all file system data either from the production PV (as Velero’s PodVolume BR does), or from any kind of snapshot (i.e., CSI snapshot).\n- Unified Repository Interface, data movers call the Unified Repository Interface to write/read data to/from the Unified Repository.\n- Kopia repository layers, CAOS and CABS, work as the backup repository and expose the Kopia Repository interface.\n- A Kopia Repository Library works as an adapter between Unified Repository Interface and Kopia Repository interface. Specifically, it implements Unified Repository Interface and calls Kopia Repository interface.\n- At present, there is only one kind of backup repository -- Kopia Repository. If a new backup repository/storage is required, we need to create a new Library as an adapter to the Unified Repository Interface\n- At present, the Kopia Repository works as a single piece in the same process of the caller, in future, we may run its CABS into a dedicated process or node.\n- At present, we don’t have a requirement to extend the backup repository, if needed, an extra module could be added as an upper layer into the Unified Repository without changing the data movers. \n\nNeither Kopia uploader nor Kopia Repository is invoked through CLI, instead, they are invoked through code interfaces, because we need to do lots of customizations. \n\nThe Unified Repository takes two kinds of data:\n- Unified Repository Object: This is the user's logical data, for example, files/directories, blocks of a volume, data of a database, etc.\n- Unified Repository Manifest: This could include all other data to maintain the object data, for example, snapshot information, etc. \n\nFor Unified Repository Object/Manifest, a brief guidance to data movers are as below:\n- Data movers treat the simple unit of data they recognize as an Object. For example, file system data movers treat a file or a directory as an Object; block data movers treat a volume as an Object. However, it is unnecessary that every data mover has a unique data format in the Unified Repository, to the opposite, it is recommended that data movers could share the data formats unless there is any reason not to, in this way, the data generated by one data mover could be used by other data movers.\n- Data movers don't need to care about the differences between full and incremental backups regarding the data organization. Data movers always have full views of their objects, if an object is partially written, they use the object writer's Seek function to skip the unchanged parts\n- Unified Repository may divide the data movers' logical Object into sub-objects or slices, or append internal metadata, but they are transparent to data movers \n- Every Object has an unified identifier, in order to retrieve the Object later, data movers need to save the identifiers into the snapshot information. The snapshot information is saved as a Manifest.\n- Manifests could hold any kind of small piece data in a K-V manner. Inside the backup repository, these kinds of data may be processed differently from Object data, but it is transparent to data movers.\n- A Manifest also has an unified identifier, the Unified Repository provides the capabilities to list all the Manifests or a specified Manifest by its identifier, or a specified Manifest by its name, or a set of Manifests by their labels.\n \n![A Unified Repository Architecture](unified-repo.png)\n\nVelero by default uses the Unified Repository for all kinds of data movement, it is also able to integrate with other data movement paths from any party, for any purpose. Details are concluded as below:\n\n- Built-in Data Path: this is the default data movement path, which uses Velero built-in data movers to backup/restore workloads, the data is written to/read from the Unified Repository.\n- Data Mover Replacement: Any party could write its own data movers and plug them into Velero. Meanwhile, these plugin data movers could also write/read data to/from Velero’s Unified Repository so that these data movers could expose the same capabilities that provided by the Unified Repository. In order to do this, the data mover providers need to call the Unified Repository Interface from inside their plugin data movers.\n- Data Path Replacement: Some vendors may already have their own data movers and backup repository and they want to replace Velero’s entire data path (including data movers and backup repository). In this case, the providers only need to implement their plugin data movers, all the things downwards are a black box to Velero and managed by providers themselves (including API call, data transport, installation, life cycle management, etc.). Therefore, this case is out of the scope of Unified Repository. \n![A Scope](scope.png) \n\n# Detailed Design\n\n## The Unified Repository Interface \nBelow are the definitions of the Unified Repository Interface. All the functions are synchronization functions. \n```\n// BackupRepoService is used to initialize, open or maintain a backup repository\ntype BackupRepoService interface {\n\t// Init creates a backup repository or connect to an existing backup repository.\n\t// repoOption: option to the backup repository and the underlying backup storage.\n\t// createNew: indicates whether to create a new or connect to an existing backup repository.\n\tInit(ctx context.Context, repoOption RepoOptions, createNew bool) error\n\n\t// Open opens an backup repository that has been created/connected.\n\t// repoOption: options to open the backup repository and the underlying storage.\n\tOpen(ctx context.Context, repoOption RepoOptions) (BackupRepo, error)\n\n\t// Maintain is periodically called to maintain the backup repository to eliminate redundant data.\n\t// repoOption: options to maintain the backup repository.\n\tMaintain(ctx context.Context, repoOption RepoOptions) error\n\n\t// DefaultMaintenanceFrequency returns the defgault frequency of maintenance, callers refer this\n\t// frequency to maintain the backup repository to get the best maintenance performance\n\tDefaultMaintenanceFrequency() time.Duration\n}\n\n// BackupRepo provides the access to the backup repository\ntype BackupRepo interface {\n\t// OpenObject opens an existing object for read.\n\t// id: the object's unified identifier.\n\tOpenObject(ctx context.Context, id ID) (ObjectReader, error)\n\n\t// GetManifest gets a manifest data from the backup repository.\n\tGetManifest(ctx context.Context, id ID, mani *RepoManifest) error\n\n\t// FindManifests gets one or more manifest data that match the given labels\n\tFindManifests(ctx context.Context, filter ManifestFilter) ([]*ManifestEntryMetadata, error)\n\n\t// NewObjectWriter creates a new object and return the object's writer interface.\n\t// return: A unified identifier of the object on success.\n\tNewObjectWriter(ctx context.Context, opt ObjectWriteOptions) ObjectWriter\n\n\t// PutManifest saves a manifest object into the backup repository.\n\tPutManifest(ctx context.Context, mani RepoManifest) (ID, error)\n\n\t// DeleteManifest deletes a manifest object from the backup repository.\n\tDeleteManifest(ctx context.Context, id ID) error\n\n\t// Flush flushes all the backup repository data\n\tFlush(ctx context.Context) error\n\n\t// Time returns the local time of the backup repository. It may be different from the time of the caller\n\tTime() time.Time\n\n\t// Close closes the backup repository\n\tClose(ctx context.Context) error\n\ntype ObjectReader interface {\n\tio.ReadCloser\n\tio.Seeker\n\n\t// Length returns the logical size of the object\n\tLength() int64\n}\n \ntype ObjectWriter interface {\n\tio.WriteCloser\n\n\t// Seeker is used in the cases that the object is not written sequentially\n\tio.Seeker\n\n\t// Checkpoint is periodically called to preserve the state of data written to the repo so far.\n\t// Checkpoint returns a unified identifier that represent the current state.\n\t// An empty ID could be returned on success if the backup repository doesn't support this.\n\tCheckpoint() (ID, error)\n\n\t// Result waits for the completion of the object write.\n\t// Result returns the object's unified identifier after the write completes.\n\tResult() (ID, error)\n}\n```\n \nSome data structure & constants used by the interfaces: \n```\ntype RepoOptions struct {\n\t// StorageType is a repository specific string to identify a backup storage, i.e., \"s3\", \"filesystem\"\n\tStorageType string\n\t// RepoPassword is the backup repository's password, if any\n\tRepoPassword string\n\t// ConfigFilePath is a custom path to save the repository's configuration, if any\n\tConfigFilePath string\n\t// GeneralOptions takes other repository specific options\n\tGeneralOptions map[string]string\n\t// StorageOptions takes storage specific options\n\tStorageOptions map[string]string\n\t// Description is a description of the backup repository/backup repository operation.\n\t// It is for logging/debugging purpose only and doesn't control any behavior of the backup repository.\n\tDescription string\n}\n\n// ObjectWriteOptions defines the options when creating an object for write\ntype ObjectWriteOptions struct {\n\tFullPath string // Full logical path of the object\n\tDataType int // OBJECT_DATA_TYPE_*\n\tDescription string // A description of the object, could be empty\n\tPrefix ID // A prefix of the name used to save the object\n\tAccessMode int // OBJECT_DATA_ACCESS_*\n\tBackupMode int // OBJECT_DATA_BACKUP_*\n}\n\nconst (\n\t// Below consts descrbe the data type of one object.\n\t// Metadata: This type describes how the data is organized.\n\t// For a file system backup, the Metadata describes a Dir or File.\n\t// For a block backup, the Metadata describes a Disk and its incremental link.\n\tObjectDataTypeUnknown int = 0\n\tObjectDataTypeMetadata int = 1\n\tObjectDataTypeData int = 2\n\n\t// Below consts defines the access mode when creating an object for write\n\tObjectDataAccessModeUnknown int = 0\n\tObjectDataAccessModeFile int = 1\n\tObjectDataAccessModeBlock int = 2\n\n\tObjectDataBackupModeUnknown int = 0\n\tObjectDataBackupModeFull int = 1\n\tObjectDataBackupModeInc int = 2\n)\n\n// ManifestEntryMetadata is the metadata describing one manifest data\ntype ManifestEntryMetadata struct {\n\tID ID // The ID of the manifest data\n\tLength int32 // The data size of the manifest data\n\tLabels map[string]string // Labels saved together with the manifest data\n\tModTime time.Time // Modified time of the manifest data\n}\n \ntype RepoManifest struct {\n\tPayload interface{} // The user data of manifest\n\tMetadata *ManifestEntryMetadata // The metadata data of manifest\n}\n \ntype ManifestFilter struct {\n Labels map[string]string\n}\n``` \n\n## Workflow\n\n### Backup & Restore Workflow\n\nWe preserve the bone of the existing BR workflow, that is:\n\n- Still use the Velero Server pod and VeleroNodeAgent daemonSet (originally called Restic daemonset) pods to hold the corresponding controllers and modules\n- Still use the Backup/Restore CR and BackupRepository CR (originally called ResticRepository CR) to drive the BR workflow\n\nThe modules in gray color in below diagram are the existing modules and with no significant changes. \nIn the new design, we will have separate and independent modules/logics for backup repository and uploader (data mover), specifically:\n\n- Repository Provider provides functionalities to manage the backup repository. For example, initialize a repository, connect to a repository, manage the snapshots in the repository, maintain a repository, etc.\n- Uploader Provider provides functionalities to run a backup or restore.\n\nThe Repository Provider and Uploader Provider use options to choose the path --- legacy path vs. new path (Kopia uploader + Unified Repository). Specifically, for legacy path, Repository Provider will manage Restic Repository only, otherwise, it manages Unified Repository only; for legacy path, Uploader Provider calls Restic to do the BR, otherwise, it calls Kopia uploader to do the BR. \n\nIn order to manage Restic Repository, the Repository Provider calls Restic Repository Provider, the latter invokes the existing Restic CLIs. \nIn order to manage Unified Repository, the Repository Provider calls Unified Repository Provider, the latter calls the Unified Repository module through the udmrepo.BackupRepoService interface. It doesn’t know how the Unified Repository is implemented necessarily. \nIn order to use Restic to do BR, the Uploader Provider calls Restic Uploader Provider, the latter invokes the existing Restic CLIs. \nIn order to use Kopia to do BR, the Uploader Provider calls Kopia Uploader Provider, the latter do the following things:\n\n- Call Unified Repository through the udmrepo.BackupRepoService interface to open the unified repository for read/write. Again, it doesn’t know how the Unified Repository is implemented necessarily. It gets a BackupRepo’s read/write handle after the call succeeds\n- Wrap the BackupRepo handle into a Kopia Shim which implements Kopia Repository interface\n- Call the Kopia Uploader. Kopia Uploader is a Kopia module without any change, so it only understands Kopia Repository interface\n- Kopia Uploader starts to backup/restore the corresponding PV’s file system data and write/read data to/from the provided Kopia Repository implementation, that is, Kopia Shim here\n- When read/write calls go into Kopia Shim, it in turn calls the BackupRepo handle for read/write\n- Finally, the read/write calls flow to Unified Repository module\n\nThe Unified Repository provides all-in-one functionalities of a Backup Repository and exposes the Unified Repository Interface. Inside, Kopia Library is an adapter for Kopia Repository to translate the Unified Repository Interface calls to Kopia Repository interface calls. \nBoth Kopia Shim and Kopia Library rely on Kopia Repository interface, so we need to have some Kopia version control. We may need to change Kopia Shim and Kopia Library when upgrading Kopia to a new version and the Kopia Repository interface has some changes in the new version.\n![A BR Workflow](br-workflow.png) \nThe modules in blue color in below diagram represent the newly added modules/logics or reorganized logics. \nThe modules in yellow color in below diagram represent the called Kopia modules without changes. \n\n### Delete Snapshot Workflow\nThe Delete Snapshot workflow follows the similar manner with BR workflow, that is, we preserve the upper-level workflows until the calls reach to BackupDeletionController, then:\n- Leverage Repository Provider to switch between Restic implementation and Unified Repository implementation in the same mechanism as BR\n- For Restic implementation, the Restic Repository Provider invokes the existing “Forget” Restic CLI\n- For Unified Repository implementation, the Unified Repository Provider calls udmrepo.BackupRepo’s DeleteManifest to delete a snapshot\n![A Snapshot Deletion Workflow](snapshot-deletion-workflow.png) \n\n### Maintenance Workflow\nBackup Repository/Backup Storage may need to periodically reorganize its data so that it could guarantee its QOS during the long-time service. Some Backup Repository/Backup Storage does this in background automatically, so the user doesn’t need to interfere; some others need the caller to explicitly call their maintenance interface periodically. Restic and Kopia both go with the second way, that is, Velero needs to periodically call their maintenance interface. \nVelero already has an existing workflow to call Restic maintenance (it is called “Prune” in Restic, so Velero uses the same word). The existing workflow is as follows:\n- The Prune is triggered at the time of the backup\n- When a BackupRepository CR (originally called ResticRepository CR) is created by PodVolumeBackup/Restore Controller, the BackupRepository controller checks if it reaches to the Prune Due Time, if so, it calls PruneRepo \n- In the new design, the Repository Provider implements PruneRepo call, it uses the same way to switch between Restic Repository Provider and Unified Repository Provider, then:\n - For Restic Repository, Restic Repository Provider invokes the existing “Prune” CLI of Restic\n - For Unified Repository, Unified Repository Provider calls udmrepo.BackupRepoService’s Maintain function\n\nKopia has two maintenance modes – the full maintenance and quick maintenance. There are many differences between full and quick mode, but briefly speaking, quick mode only processes the hottest data (primarily, it is the metadata and index data), so quick maintenance is much faster than full maintenance. On the other hand, quick maintenance also scatters the burden of full maintenance so that the full maintenance could finish fastly and make less impact. We will also take this quick maintenance into Velero. \nWe will add a new Due Time to Velero, finally, we have two Prune Due Time:\n- Normal Due Time: For Restic, this will invoke Restic Prune; for Unified Repository, this will invoke udmrepo.BackupRepoService’s Maintain(full) call and finally call Kopia’s full maintenance\n- Quick Due Time: For Restic, this does nothing; for Unified Repository, this will invoke udmrepo.BackupRepoService’s Maintain(quick) call and finally call Kopia’s quick maintenance \n\nWe assign different values to Normal Due Time and Quick Due Time, as a result of which, the quick maintenance happens more frequently than full maintenance. \n![A Maintenance Workflow](maintenance-workflow.png) \n\n### Progress Update\nBecause Kopia Uploader is an unchanged Kopia module, we need to find a way to get its progress during the BR. \nKopia Uploader accepts a Progress interface to update rich information during the BR, so the Kopia Uploader Provider will implement a Kopia’s Progress interface and then pass it to Kopia Uploader during its initialization. \nIn this way, Velero will be able to get the progress as shown in the diagram below. \n![A Progress Update](progress-update.png) \n\n### Logs\nIn the current design, Velero is using two unchanged Kopia modules --- the Kopia Uploader and the Kopia Repository. Both will generate debug logs during their run. Velero will collect these logs in order to aid the debug. \nKopia’s Uploader and Repository both get the Logger information from the current GO Context, therefore, the Kopia Uploader Provider/Kopia Library could set the Logger interface into the current context and pass the context to Kopia Uploader/Kopia Repository. \nVelero will set Logger interfaces separately for Kopia Uploader and Kopia Repository. In this way, the Unified Repository could serve other data movers without losing the debug log capability; and the Kopia Uploader could write to any repository without losing the debug log capability. \nKopia’s debug logs will be written to the same log file as Velero server or VeleroNodeAgent daemonset, so Velero doesn’t need to upload/download these debug logs separately. \n![A Debug Log for Uploader](debug-log-uploader.png) \n![A Debug Log for Repository](debug-log-repository.png) \n\n## Path Switch & Coexist\nAs mentioned above, There will be two paths. The related controllers need to identify the path during runtime and adjust its working mode. \nAccording to the requirements, path changing is fulfilled at the backup/restore level. In order to let the controllers know the path, we need to add some option values. Specifically, there will be option/mode values for path selection in two places:\n- Add the “uploader-type” option as a parameter of the Velero server. The parameters will be set by the installation. Currently the option has two values, either \"restic\" or \"kopia\" (in future, we may add other file system uploaders, then we will have more values).\n- Add a \"uploaderType\" value in the PodVolume Backup/Restore CR and a \"repositoryType\" value in the BackupRepository CR. \"uploaderType\" currently has two values , either \"restic\" or \"kopia\"; \"repositoryType\" currently has two values, either \"restic\" or \"kopia\" (in future, the Unified Repository could opt among multiple backup repository/backup storage, so there may be more values. This is a good reason that repositoryType is a multivariate flag, however, in which way to opt among the backup repository/backup storage is not covered in this PR). If the values are missing in the CRs, it by default means \"uploaderType=restic\" and \"repositoryType=restic\", so the legacy CRs are handled correctly by Restic. \n\nThe corresponding controllers handle the CRs by checking the CRs' path value. Some examples are as below:\n- The PodVolume BR controller checks the \"uploaderType\" value from PodVolume CRs and decide its working path\n- The BackupRepository controller checks the \"repositoryType\" value from BackupRepository CRs and decide its working path\n- The Backup controller that runs in Velero server checks its “uploader-type” parameter to decide the path for the Backup it is going to create and then create the PodVolume Backup CR and BackupRepository CR\n- The Restore controller checks the Backup, from which it is going to restore, for the path and then create the PodVolume Restore CR and BackupRepository CR\n\nAs described above, the “uploader-type” parameter of the Velero server is only used to decide the path when creating a new Backup, for other cases, the path selection is driven by the related CRs. Therefore, we only need to add this parameter to the Velero server. \n\n## Velero CR Name Changes\nWe will change below CRs' name to make them more generic:\n- \"ResticRepository\" CR to \"BackupRepository\" CR \n\nThis means, we add a new CR type and deprecate the old one. As a result, if users upgrade from the old release, the old CRs will be orphaned, Velero will neither refer to it nor manage it, users need to delete these CRs manually. \nAs a side effect, when upgrading from an old release, even though the path is not changed, the BackupRepository gets created all the time, because Velero will not refer to the old CR's status. This seems to cause the repository to initialize more than once, however, it won't happen. In the BackupRepository controller, before initializing a repository, it always tries to connect to the repository first, if it is connectable, it won't do the initialization. \nWhen backing up with the new release, Velero always creates BackupRepository CRs instead of ResticRepository CRs. \nWhen restoring from an old backup, Velero always creates BackupRepository CRs instead of ResticRepository CRs. \nWhen there are already backups or restores running during the upgrade, since after upgrade, the Velero server pods and VeleroNodeAgent daemonset pods are restarted, the existing backups/restores will fail immediately. \n\n## Storage Configuration\nThe backup repository needs some parameters to connect to various backup storage. For example, for a S3 compatible storage, the parameters may include bucket name, region, endpoint, etc. Different backup storage have totally different parameters. BackupRepository CRs, PodVolume Backup CRs and PodVolume Restore CRs save these parameters in their spec, as a string called repoIdentififer. The format of the string is for S3 storage only, it meets Restic CLI's requirements but is not enough for other backup repository. On the other hand, the parameters that are used to generate the repoIdentififer all come from the BackupStorageLocation. The latter has a map structure that could take parameters from any storage kind. \nTherefore, for the new path, Velero uses the information in the BackupStorageLocation directly. That is, whenever Velero needs to initialize/connect to the Unified Repository, it acquires the storage configuration from the corresponding BackupStorageLocation. Then no more elements will be added in BackupRepository CRs, PodVolume Backup CRs or PodVolume Restore CRs. \nThe legacy path will be kept as is. That is, Velero still sets/gets the repoIdentififer in BackupRepository CRs, PodVolume Backup CRs and PodVolume Restore CRs and then passes to Restic CLI. \n\n## Installation\n We will add a new flag \"--uploader-type\" during installation. The flag has 2 meanings:\n - It indicates the file system uploader to be used by PodVolume BR\n - It implies the backup repository type manner, Restic if uploader-type=restic, Unified Repository in all other cases\n\n The flag has below two values: \n **\"Restic\"**: it means Velero will use Restic to do the pod volume backup. Therefore, the Velero server deployment will be created as below:\n ```\n spec:\n containers:\n - args:\n - server\n - --features=\n - --uploader-type=restic\n command:\n - /velero\n```\nThe BackupRepository CRs and PodVolume Backup/Restore CRs created in this case are as below:\n```\nspec:\n backupStorageLocation: default\n maintenanceFrequency: 168h0m0s\n repositoryType: restic\n volumeNamespace: nginx-example\n```\n```\nspec:\n backupStorageLocation: default\n node: aks-agentpool-27359964-vmss000000\n pod:\n kind: Pod\n name: nginx-stateful-0\n namespace: nginx-example\n uid: 86aaec56-2b21-4736-9964-621047717133 \n tags:\n ...\n uploaderType: restic\n volume: nginx-log\n```\n```\nspec:\n backupStorageLocation: default\n pod:\n kind: Pod\n name: nginx-stateful-0\n namespace: nginx-example\n uid: e56d5872-3d94-4125-bfe8-8a222bf0fcf1\n snapshotID: 1741e5f1\n uploaderType: restic\n volume: nginx-log\n```\n **\"Kopia\"**: it means Velero will use Kopia uploader to do the pod volume backup (so it will use Unified Repository as the backup target). Therefore, the Velero server deployment will be created as below:\n ```\n spec:\n containers:\n - args:\n - server\n - --features=\n - --uploader-type=kopia\n command:\n - /velero\n```\nThe BackupRepository CRs created in this case are hard set with \"kopia\" at present, sice Kopia is the only option as a backup repository. The PodVolume Backup/Restore CRs are created with \"kopia\" as well:\n```\nspec:\n backupStorageLocation: default\n maintenanceFrequency: 168h0m0s\n repositoryType: kopia\n volumeNamespace: nginx-example\n```\n```\nspec:\n backupStorageLocation: default\n node: aks-agentpool-27359964-vmss000000\n pod:\n kind: Pod\n name: nginx-stateful-0\n namespace: nginx-example\n uid: 86aaec56-2b21-4736-9964-621047717133 \n tags:\n ...\n uploaderType: kopia\n volume: nginx-log\n```\n```\nspec:\n backupStorageLocation: default\n pod:\n kind: Pod\n name: nginx-stateful-0\n namespace: nginx-example\n uid: e56d5872-3d94-4125-bfe8-8a222bf0fcf1\n snapshotID: 1741e5f1\n uploaderType: kopia\n volume: nginx-log\n```\nWe will add the flag for both CLI installation and Helm Chart Installation. Specifically:\n- Helm Chart Installation: add the \"--uploaderType\" and \"--default-volumes-to-fs-backup\" flag into its value.yaml and then generate the deployments according to the value. Value.yaml is the user-provided configuration file, therefore, users could set this value at the time of installation. The changes in Value.yaml are as below:\n```\n command:\n - /velero\n args:\n - server\n {{- with .Values.configuration }}\n\t\t - --uploader-type={{ default \"restic\" .uploaderType }}\n\t\t\t{{- if .defaultVolumesToFsBackup }}\n - --default-volumes-to-fs-backup\n {{- end }}\n``` \n- CLI Installation: add the \"--uploaderType\" and \"--default-volumes-to-fs-backup\" flag into the installation command line, and then create the two deployments accordingly. Users could change the option at the time of installation. The CLI is as below: \n```velero install --uploader-type=restic --default-volumes-to-fs-backup --use-node-agent``` \n```velero install --uploader-type=kopia --default-volumes-to-fs-backup --use-node-agent``` \n\n## Upgrade\nFor upgrade, we allow users to change the path by specifying \"--uploader-type\" flag in the same way as the fresh installation. Therefore, the flag change should be applied to the Velero server after upgrade. Additionally, We need to add a label to Velero server to indicate the current path, so as to provide an easy for querying it. \nMoreover, if users upgrade from the old release, we need to change the existing Restic Daemonset name to VeleroNodeAgent daemonSet. The name change should be applied after upgrade. \nThe recommended way for upgrade is to modify the related Velero resource directly through kubectl, the above changes will be applied in the same way. We need to modify the Velero doc for all these changes. \n\n## CLI\nBelow Velero CLI or its output needs some changes: \n- ```Velero backup describe```: the output should indicate the path \n- ```Velero restore describe```: the output should indicate the path \n- ```Velero restic repo get```: the name of this CLI should be changed to a generic one, for example, \"Velero repo get\"; the output of this CLI should print all the backup repository if Restic repository and Unified Repository exist at the same time \n\nAt present, we don't have a requirement for selecting the path during backup, so we don't change the ```Velero backup create``` CLI for now. If there is a requirement in future, we could simply add a flag similar to \"--uploader-type\" to select the path. \n\n## CR Example\nBelow sample files demonstrate complete CRs with all the changes mentioned above:\n- BackupRepository CR: https://gist.github.com/Lyndon-Li/f38ad69dd8c4785c046cd7ed0ef2b6ed#file-backup-repository-sample-yaml\n- PodVolumeBackup CR: https://gist.github.com/Lyndon-Li/f38ad69dd8c4785c046cd7ed0ef2b6ed#file-pvb-sample-yaml\n- PodVolumeRestore CR: https://gist.github.com/Lyndon-Li/f38ad69dd8c4785c046cd7ed0ef2b6ed#file-pvr-sample-yaml\n\n## User Perspective\nThis design aims to provide a flexible backup repository layer and a generic file system uploader, which are fundermental for PodVolume and other data movements. Although this will make Velero more capable, at present, we don't pursue to expose differentiated features end to end. Specifically: \n- For a fresh installation, if the \"--uploader-type\" is not specified, there is a default value for PodVolume BR. We will keep it as \"restic\" for at least one release, then we switch the value to \"kopia\"\n- Even when changing to the new path, Velero still allows users to restore from the data backed up by Restic\n- The capability of PodVolume BR under the new path is kept the same as it under Restic path and the same as the existing PodVolume BR\n- The operational experiences are kept the same as much as possible, the known changes are listed below\n\nBelow user experiences are changed for this design:\n- Installation CLI change: a new option is added to the installation CLI, see the Installation section for details\n- CR change: One or more existing CRs have been renamed, see the Velero CR Changes section for details\n- Velero CLI name and output change, see the CLI section for details\n- Velero daemonset name change\n- Wording Alignment: as the existing situation, many places are using the word of \"Restic\", for example, \"default-volume-to-restic\" option, most of them are not accurate anymore, we will change these words and give a detailed list of the changes " }, { "path": "design/Implemented/velero-debug.md", "content": "# `velero debug` command for gathering troubleshooting information\n\n## Abstract\nTo simplify the communication between velero users and developers, this document proposes the `velero debug` command to generate a tarball including the logs needed for debugging.\n\nGithub issue: https://github.com/vmware-tanzu/velero/issues/675\n\n## Background\nGathering information to troubleshoot a Velero deployment is currently spread across multiple commands, and is not very efficient. Logs for the Velero server itself are accessed via a kubectl logs command, while information on specific backups or restores are accessed via a Velero subcommand. Restic logs are even more complicated to retrieve, since one must gather logs for every instance of the daemonset, and there’s currently no good mechanism to locate which node a particular restic backup ran against. \nA dedicated subcommand can lower this effort and reduce back-and-forth between user and developer for collecting the logs.\n\n\n## Goals\n- Enable efficient log collection for Velero and associated components, like plugins and restic.\n\n## Non Goals\n- Collecting logs for components that do not belong to velero such as storage service.\n- Automated log analysis.\n\n## High-Level Design\nWith the introduction of the new command `velero debug`, the command would download all of the following information:\n- velero deployment logs\n- restic DaemonSet logs\n- plugin logs \n- All the resources in the group `velero.io` that are created such as:\n - Backup\n - Restore\n - BackupStorageLocation\n - PodVolumeBackup\n - PodVolumeRestore\n - *etc ...*\n- Log of the backup and restore, if specified in the param\n\nA project called `crash-diagnostics` (or `crashd`) (https://github.com/vmware-tanzu/crash-diagnostics) implements the Kubernetes API queries and provides Starlark scripting language to abstract details, and collect the information into a local copy. It can be used as a standalone CLI executing a Starlark script file.\nWith the capabilities of embedding files in Go 1.16, we can define a Starlark script gathering the necessary information, embed the script at build time, then the velero debug command will invoke `crashd`, passing in the script’s text contents.\n\n## Detailed Design\n### Triggering the script\nThe Starlark script to be called by crashd:\n\n```python\ndef capture_backup_logs(cmd, namespace):\n if args.backup:\n log(\"Collecting log and information for backup: {}\".format(args.backup))\n backupDescCmd = \"{} --namespace={} backup describe {} --details\".format(cmd, namespace, args.backup)\n capture_local(cmd=backupDescCmd, file_name=\"backup_describe_{}.txt\".format(args.backup))\n backupLogsCmd = \"{} --namespace={} backup logs {}\".format(cmd, namespace, args.backup)\n capture_local(cmd=backupLogsCmd, file_name=\"backup_{}.log\".format(args.backup))\ndef capture_restore_logs(cmd, namespace):\n if args.restore:\n log(\"Collecting log and information for restore: {}\".format(args.restore))\n restoreDescCmd = \"{} --namespace={} restore describe {} --details\".format(cmd, namespace, args.restore)\n capture_local(cmd=restoreDescCmd, file_name=\"restore_describe_{}.txt\".format(args.restore))\n restoreLogsCmd = \"{} --namespace={} restore logs {}\".format(cmd, namespace, args.restore)\n capture_local(cmd=restoreLogsCmd, file_name=\"restore_{}.log\".format(args.restore))\n\nns = args.namespace if args.namespace else \"velero\"\noutput = args.output if args.output else \"bundle.tar.gz\"\ncmd = args.cmd if args.cmd else \"velero\"\n# Working dir for writing during script execution\ncrshd = crashd_config(workdir=\"./velero-bundle\")\nset_defaults(kube_config(path=args.kubeconfig, cluster_context=args.kubecontext))\nlog(\"Collecting velero resources in namespace: {}\". format(ns))\nkube_capture(what=\"objects\", namespaces=[ns], groups=['velero.io'])\ncapture_local(cmd=\"{} version -n {}\".format(cmd, ns), file_name=\"version.txt\")\nlog(\"Collecting velero deployment logs in namespace: {}\". format(ns))\nkube_capture(what=\"logs\", namespaces=[ns])\ncapture_backup_logs(cmd, ns)\ncapture_restore_logs(cmd, ns)\narchive(output_file=output, source_paths=[crshd.workdir])\nlog(\"Generated debug information bundle: {}\".format(output))\n```\nThe sample command to trigger the script via crashd:\n```shell\n./crashd run ./velero.cshd --args \n'backup=harbor-backup-2nd,namespace=velero,basedir=,restore=,kubeconfig=/home/.kube/minikube-250-224/config,output='\n```\nTo trigger the script in `velero debug`, in the package `pkg/cmd/cli/debug` a struct `option` will be introduced\n```go\ntype option struct {\n\t// currCmd the velero command\n\tcurrCmd string\n\t// workdir for crashd will be $baseDir/velero-debug\n\tbaseDir string\n\t// the namespace where velero server is installed\n\tnamespace string\n\t// the absolute path for the log bundle to be generated\n\toutputPath string\n\t// the absolute path for the kubeconfig file that will be read by crashd for calling K8S API\n\tkubeconfigPath string\n\t// the kubecontext to be used for calling K8S API\n\tkubeContext string\n\t// optional, the name of the backup resource whose log will be packaged into the debug bundle\n\tbackup string\n\t// optional, the name of the restore resource whose log will be packaged into the debug bundle\n\trestore string\n\t// optional, it controls whether to print the debug log messages when calling crashd\n\tverbose bool\n}\n```\nThe code will consolidate the input parameters and execution context of the `velero` CLI to form the option struct, which can be transformed into the `argsMap` that can be used when calling the func `exec.Execute` in `crashd`:\nhttps://github.com/vmware-tanzu/crash-diagnostics/blob/v0.3.4/exec/executor.go#L17\n\n## Alternatives Considered\nThe collection could be done via the Kubernetes client-go API, but such integration is not necessarily trivial to implement, therefore, `crashd` is preferred approach\n\n## Security Considerations\n- The starlark script will be embedded into the velero binary, and the byte slice will be passed to the `exec.Execute` func directly, so there’s little risk that the script will be modified before being executed.\n\n## Compatibility\nAs the `crashd` project evolves the behavior of the internal functions used in the Starlark script may change. We’ll ensure the correctness of the script via regular E2E tests.\n\n\n## Implementation\n1. Bump up to use Go v1.16 to compile velero\n2. Embed the starlark script\n3. Implement the `velero debug` sub-command to call the script\n4. Add E2E test case\n\n## Open Questions\n- **Command dependencies:** In the Starlark script, for collecting version info and backup logs, it calls the `velero backup logs` and `velero version`, which makes the call stack like velero debug -> crashd -> velero xxx. We need to make sure this works under different PATH settings.\n- **Progress and error handling:** The log collection may take a relatively long time, log messages should be printed to indicate the progress when different items are being downloaded and packaged. Additionally, when an error happens, `crashd` may omit some errors, so before the script is executed we'll do some validation and make sure the `debug` command fail early if some parameters are incorrect.\n" }, { "path": "design/Implemented/velero-uploader-configuration.md", "content": "# Velero Uploader Configuration Integration and Extensibility\n\n## Abstract\nThis design proposal aims to make Velero Uploader configurable by introducing a structured approach for managing Uploader settings. we will define and standardize a data structure to facilitate future additions to Uploader configurations. This enhancement provides a template for extending Uploader-related options. And also includes examples of adding sub-options to the Uploader Configuration.\n\n## Background\nVelero is widely used for backing up and restoring Kubernetes clusters. In various scenarios, optimizing the backup process is essential, future needs may arise for adding more configuration options related to the Uploader component especially when dealing with large datasets. Therefore, a standardized configuration template is required.\n\n## Goals\n1. **Extensible Uploader Configuration**: Provide an extensible approach to manage Uploader configurations, making it easy to add and modify configuration options related to the Velero uploader.\n2. **User-friendliness**: Ensure that the new Uploader configuration options are easy to understand and use for Velero users without introducing excessive complexity.\n\n## Non Goals\n1. Expanding to other Velero components: The primary focus of this design is Uploader configuration and does not include extending to other components or modules within Velero. Configuration changes for other components may require separate design and implementation.\n\n## High-Level Design\nTo achieve extensibility in Velero Uploader configurations, the following key components and changes are proposed:\n\n### UploaderConfig Structure\nTwo new data structures, `UploaderConfigForBackup` and `UploaderConfigForRestore`, will be defined to store Uploader configurations. These structures will include the configuration options related to backup and restore for Uploader:\n\n```go\ntype UploaderConfigForBackup struct {\n}\n\ntype UploaderConfigForRestore struct {\n}\n```\n\n### Integration with Backup & Restore CRD\nThe Velero CLI will support an uploader configuration-related flag, allowing users to set the value when creating backups or restores. This value will be stored in the `UploaderConfig` field within the `Backup` CRD and `Restore` CRD:\n\n```go\ntype BackupSpec struct {\n\t// UploaderConfig specifies the configuration for the uploader.\n\t// +optional\n\t// +nullable\n\tUploaderConfig *UploaderConfigForBackup `json:\"uploaderConfig,omitempty\"`\n}\n\ntype RestoreSpec struct {\n\t// UploaderConfig specifies the configuration for the restore.\n\t// +optional\n\t// +nullable\n\tUploaderConfig *UploaderConfigForRestore `json:\"uploaderConfig,omitempty\"`\n}\n```\n\n### Configuration Propagated to Different CRDs\nThe configuration specified in `UploaderConfig` needs to be effective for backup and restore both by file system way and data-mover way. \nTherefore, the `UploaderConfig` field value from the `Backup` CRD should be propagated to `PodVolumeBackup` and `DataUpload` CRDs.\n\nWe aim for the configurations in PodVolumeBackup to originate not only from UploaderConfig in Backup but also potentially from other sources such as the server or configmap. Simultaneously, to align with the configurations in DataUpload's `DataMoverConfig map[string]string`, we have defined an `UploaderSettings map[string]string` here to record the configurations in PodVolumeBackup.\n\n```go\ntype PodVolumeBackupSpec struct {\n\t// UploaderSettings are a map of key-value pairs that should be applied to the\n\t// uploader configuration.\n\t// +optional\n\t// +nullable\n\tUploaderSettings map[string]string `json:\"uploaderSettings,omitempty\"`\n}\n```\n\n`UploaderConfig` will be stored in DataUpload's `DataMoverConfig map[string]string` field.\n\nAlso the `UploaderConfig` field value from the `Restore` CRD should be propagated to `PodVolumeRestore` and `DataDownload` CRDs:\n\n```go\ntype PodVolumeRestoreSpec struct {\n\t// UploaderSettings are a map of key-value pairs that should be applied to the\n\t// uploader configuration.\n\t// +optional\n\t// +nullable\n\tUploaderSettings map[string]string `json:\"uploaderSettings,omitempty\"`\n}\n```\nAlso `UploaderConfig` will be stored in DataUpload's `DataMoverConfig map[string]string` field.\n\n### Store and Get Configuration\nWe need to store and retrieve configurations in the PodVolumeBackup and DataUpload structs. This involves type conversion based on the configuration type, storing it in a map[string]string, or performing type conversion from this map for retrieval.\n\nPodVolumeRestore and DataDownload are also similar.\n\n## Sub-options in UploaderConfig\nAdding fields above in CRDs can accommodate any future additions to Uploader configurations by adding new fields to the `UploaderConfigForBackup` or `UploaderConfigForRestore` structures.\n\n### Parallel Files Upload\nThis section focuses on enabling the configuration for the number of parallel file uploads during backups.\nbelow are the key steps that should be added to support this new feature.\n\n#### Velero CLI\nThe Velero CLI will support a `--parallel-files-upload` flag, allowing users to set the `ParallelFilesUpload` value when creating backups.\n\n#### UploaderConfig\nbelow the sub-option `ParallelFilesUpload` is added into UploaderConfig:\n\n```go\n// UploaderConfigForBackup defines the configuration for the uploader when doing backup.\ntype UploaderConfigForBackup struct {\n\t// ParallelFilesUpload is the number of files parallel uploads to perform when using the uploader.\n\t// +optional\n\tParallelFilesUpload int `json:\"parallelFilesUpload,omitempty\"`\n}\n```\n\n#### Kopia Parallel Upload Policy\nVelero Uploader can set upload policies when calling Kopia APIs. In the Kopia codebase, the structure for upload policies is defined as follows:\n\n```go\n// UploadPolicy describes the policy to apply when uploading snapshots.\ntype UploadPolicy struct {\n ...\n MaxParallelFileReads *OptionalInt `json:\"maxParallelFileReads,omitempty\"`\n}\n```\n\nVelero can set the `MaxParallelFileReads` parameter for Kopia's upload policy as follows:\n\n```go\ncurPolicy := getDefaultPolicy()\nif parallelUpload > 0 {\n\tcurPolicy.UploadPolicy.MaxParallelFileReads = newOptionalInt(parallelUpload)\n}\n```\n\n#### Restic Parallel Upload Policy\nAs Restic does not support parallel file upload, the configuration would not take effect, so we should output a warning when the user sets the `ParallelFilesUpload` value by using Restic to do a backup.\n\n```go\nif parallelFilesUpload > 0 {\n\tlog.Warnf(\"ParallelFilesUpload is set to %d, but Restic does not support parallel file uploads. Ignoring\", parallelFilesUpload)\n\t}\n```\n\nRoughly, the process is as follows: \n1. Users pass the ParallelFilesUpload parameter and its value through the Velero CLI. This parameter and its value are stored as a sub-option within UploaderConfig and then placed into the Backup CR. \n2. When users perform file system backups, UploaderConfig is passed to the PodVolumeBackup CR. When users use the Data-mover for backups, it is passed to the DataUpload CR. \n3. The configuration will be stored in map[string]string type of field in CR.\n3. Each respective controller within the CRs calls the uploader, and the ParallelFilesUpload from map in CRs is passed to the uploader.\n4. When the uploader subsequently calls the Kopia API, it can use the ParallelFilesUpload to set the MaxParallelFileReads parameter, and if the uploader calls the Restic command it would output one warning log for Restic does not support this feature.\n\n### Sparse Option For Kopia & Restic Restore\nIn many system files, numerous zero bytes or empty blocks persist, occupying physical storage space. Sparse restore employs a more intelligent approach, including appropriately handling empty blocks, thereby achieving the correct system state. This write sparse files mechanism aims to enhance restore efficiency while maintaining restoration accuracy.\nBelow are the key steps that should be added to support this new feature.\n#### Velero CLI\nThe Velero CLI will support a `--write-sparse-files` flag, allowing users to set the `WriteSparseFiles` value when creating restores with Restic or Kopia uploader.\n#### UploaderConfig\nbelow the sub-option `WriteSparseFiles` is added into UploaderConfig:\n```go\n// UploaderConfigForRestore defines the configuration for the restore.\ntype UploaderConfigForRestore struct {\n\t// WriteSparseFiles is a flag to indicate whether write files sparsely or not.\n\t// +optional\n\t// +nullable\n\tWriteSparseFiles *bool `json:\"writeSparseFiles,omitempty\"`\n}\n```\n\n### Enable Sparse in Restic\nFor Restic, it could be enabled by pass the flag `--sparse` in creating restore:\n```bash\nrestic restore create --sparse $snapshotID\n```\n### Enable Sparse in Kopia\nFor Kopia, it could be enabled this feature by the `WriteSparseFiles` field in the [FilesystemOutput](https://pkg.go.dev/github.com/kopia/kopia@v0.13.0/snapshot/restore#FilesystemOutput).\n\n```go\nfsOutput := &restore.FilesystemOutput{\n\t\tWriteSparseFiles: uploaderutil.GetWriteSparseFiles(uploaderCfg),\n\t}\n```\nRoughly, the process is as follows:\n1. Users pass the WriteSparseFiles parameter and its value through the Velero CLI. This parameter and its value are stored as a sub-option within UploaderConfig and then placed into the Restore CR.\n2. When users perform file system restores, UploaderConfig is passed to the PodVolumeRestore CR. When users use the Data-mover for restores, it is passed to the DataDownload CR.\n3. The configuration will be stored in map[string]string type of field in CR.\n4. Each respective controller within the CRs calls the uploader, and the WriteSparseFiles from map in CRs is passed to the uploader.\n5. When the uploader subsequently calls the Kopia API, it can use the WriteSparseFiles to set the WriteSparseFiles parameter, and if the uploader calls the Restic command it would append `--sparse` flag within the restore command.\n\n### Parallel Restore\nSetting the parallelism of restore operations can improve the efficiency and speed of the restore process, especially when dealing with large amounts of data.\n\n### Velero CLI\nThe Velero CLI will support a --parallel-files-download flag, allowing users to set the parallelism value when creating restores. when no value specified, the value of it would be the number of CPUs for the node that the node agent pod is running.\n```bash\n\tvelero restore create --parallel-files-download $num\n```\n\n### UploaderConfig\nbelow the sub-option parallel is added into UploaderConfig:\n\n```go\n\ttype UploaderConfigForRestore struct {\n\t\t// ParallelFilesDownload is the number of parallel for restore.\n\t\t// +optional\n\t\tParallelFilesDownload int `json:\"parallelFilesDownload,omitempty\"`\n\t}\n```\n\n#### Kopia Parallel Restore Policy\n\nVelero Uploader can set restore policies when calling Kopia APIs. In the Kopia codebase, the structure for restore policies is defined as follows:\n\n```go\n\t// first get concurrrency from uploader config\n\trestoreConcurrency, _ := uploaderutil.GetRestoreConcurrency(uploaderCfg)\n\t// set restore concurrency into restore options\n\trestoreOpt := restore.Options{\n\t\tParallel: restoreConcurrency,\n\t}\n\t// do restore with restore option\n\trestore.Entry(..., restoreOpt)\n```\n\n#### Restic Parallel Restore Policy\n\nConfigurable parallel restore is not supported by restic, so we would return one error if the option is configured.\n```go\n\trestoreConcurrency, err := uploaderutil.GetRestoreConcurrency(uploaderCfg)\n\tif err != nil {\n\t\treturn extraFlags, errors.Wrap(err, \"failed to get uploader config\")\n\t}\n\t\n\tif restoreConcurrency > 0 {\n\t\treturn extraFlags, errors.New(\"restic does not support parallel restore\")\n\t}\n```\n\n## Alternatives Considered\nTo enhance extensibility further, the option of storing `UploaderConfig` in a Kubernetes ConfigMap can be explored, this approach would allow the addition and modification of configuration options without the need to modify the CRD.\n" }, { "path": "design/Implemented/vgdp-affinity-enhancement.md", "content": "# Velero Generic Data Path Load Affinity Enhancement Design\r\n\r\n## Glossary & Abbreviation\r\n\r\n**Velero Generic Data Path (VGDP)**: VGDP is the collective modules that is introduced in [Unified Repository design][1]. Velero uses these modules to finish data transfer for various purposes (i.e., PodVolume backup/restore, Volume Snapshot Data Movement). VGDP modules include uploaders and the backup repository. \r\n\r\n**Exposer**: Exposer is a module that is introduced in [Volume Snapshot Data Movement Design][1]. Velero uses this module to expose the volume snapshots to Velero node-agent pods or node-agent associated pods so as to complete the data movement from the snapshots.\r\n\r\n## Background\r\n\r\nThe implemented [VGDP LoadAffinity design][3] already defined the a structure `LoadAffinity` in `--node-agent-configmap` parameter. The parameter is used to set the affinity of the backupPod of VGDP.\r\n\r\nThere are still some limitations of this design:\r\n* The affinity setting is global. Say there are two StorageClasses and the underlying storage can only provision volumes to part of the cluster nodes. The supported nodes don't have intersection. Then the affinity will definitely not work in some cases.\r\n* The old design focuses on the backupPod affinity, but the restorePod also needs the affinity setting.\r\n\r\nAs a result, create this design to address the limitations.\r\n\r\n## Goals\r\n\r\n- Enhance the node affinity of VGDP instances for volume snapshot data movement: add per StorageClass node affinity.\r\n- Enhance the node affinity of VGDP instances for volume snapshot data movement: support the or logic between affinity selectors.\r\n- Define the behaviors of node affinity of VGDP instances in node-agent for volume snapshot data movement restore, when the PVC restore doesn't require delay binding.\r\n\r\n## Non-Goals\r\n\r\n- It is also beneficial to support VGDP instances affinity for PodVolume backup/restore, this will be implemented after the PodVolume micro service completes.\r\n\r\n## Solution\r\n\r\nThis design still uses the ConfigMap specified by `velero node-agent` CLI's parameter `--node-agent-configmap` to host the node affinity configurations.\r\n\r\nUpon the implemented [VGDP LoadAffinity design][3] introduced `[]*LoadAffinity` structure, this design add a new field `StorageClass`. This field is optional.\r\n* If the `LoadAffinity` element's `StorageClass` doesn't have value, it means this element is applied to global, just as the old design.\r\n* If the `LoadAffinity` element's `StorageClass` has value, it means this element is applied to the VGDP instances' PVCs use the specified StorageClass.\r\n* The `LoadAffinity` element whose `StorageClass` has value has higher priority than the `LoadAffinity` element whose `StorageClass` doesn't have value.\r\n\r\n\r\n```go\r\ntype Configs struct {\r\n // LoadConcurrency is the config for load concurrency per node.\r\n LoadConcurrency *LoadConcurrency `json:\"loadConcurrency,omitempty\"`\r\n\r\n // LoadAffinity is the config for data path load affinity.\r\n LoadAffinity []*LoadAffinity `json:\"loadAffinity,omitempty\"` \r\n}\r\n\r\ntype LoadAffinity struct {\r\n // NodeSelector specifies the label selector to match nodes\r\n NodeSelector metav1.LabelSelector `json:\"nodeSelector\"`\r\n}\r\n```\r\n\r\n``` go\r\ntype LoadAffinity struct {\r\n // NodeSelector specifies the label selector to match nodes\r\n NodeSelector metav1.LabelSelector `json:\"nodeSelector\"`\r\n\r\n // StorageClass specifies the VGDPs the LoadAffinity applied to. If the StorageClass doesn't have value, it applies to all. If not, it applies to only the VGDPs that use this StorageClass.\r\n StorageClass string `json:\"storageClass\"`\r\n}\r\n```\r\n\r\n### Decision Tree\r\n\r\n```mermaid\r\nflowchart TD\r\n A[VGDP Pod Needs Scheduling] --> B{Is this a restore operation?}\r\n \r\n B -->|Yes| C{StorageClass has volumeBindingMode: WaitForFirstConsumer?}\r\n B -->|No| D[Backup Operation]\r\n \r\n C -->|Yes| E{restorePVC.ignoreDelayBinding = true?}\r\n C -->|No| F[StorageClass binding mode: Immediate]\r\n \r\n E -->|No| G[Wait for target Pod scheduling
Use Pod's selected node
⚠️ Affinity rules ignored]\r\n E -->|Yes| H[Apply affinity rules
despite WaitForFirstConsumer]\r\n \r\n F --> I{Check StorageClass in loadAffinity by StorageClass field}\r\n H --> I\r\n D --> J{Using backupPVC with different StorageClass?}\r\n \r\n J -->|Yes| K[Use final StorageClass
for affinity lookup]\r\n J -->|No| L[Use original PVC StorageClass
for affinity lookup]\r\n \r\n K --> I\r\n L --> I\r\n \r\n I -->|StorageClass found| N[Filter the LoadAffinity by
the StorageClass
🎯 and apply the LoadAffinity HIGHEST PRIORITY]\r\n I -->|StorageClass not found| O{Check loadAffinity element without StorageClass field}\r\n\r\n O -->|No loadAffinity configured| R[No affinity constraints
Schedule on any available node
🌐 DEFAULT]\r\n \r\n O --> V[Validate node-agent availability
⚠️ Ensure node-agent pods exist on target nodes]\r\n N --> V\r\n \r\n V --> W{Node-agent available on selected nodes?}\r\n W -->|Yes| X[✅ VGDP Pod scheduled successfully]\r\n W -->|No| Y[❌ Pod stays in Pending state
Timeout after 30min
Check node-agent DaemonSet coverage]\r\n \r\n R --> Z[Schedule on any node
✅ Basic scheduling]\r\n \r\n %% Styling\r\n classDef successNode fill:#d4edda,stroke:#155724,color:#155724\r\n classDef warningNode fill:#fff3cd,stroke:#856404,color:#856404\r\n classDef errorNode fill:#f8d7da,stroke:#721c24,color:#721c24\r\n classDef priorityHigh fill:#e7f3ff,stroke:#0066cc,color:#0066cc\r\n classDef priorityMedium fill:#f0f8ff,stroke:#4d94ff,color:#4d94ff\r\n classDef priorityDefault fill:#f8f9fa,stroke:#6c757d,color:#6c757d\r\n \r\n class X,Z successNode\r\n class G,V,Y warningNode\r\n class Y errorNode\r\n class N,T,U priorityHigh\r\n class P,Q priorityMedium\r\n class R priorityDefault\r\n```\r\n\r\n### Examples\r\n\r\n#### LoadAffinity interacts with LoadAffinityPerStorageClass\r\n\r\n``` json\r\n{\r\n \"loadAffinity\": [\r\n {\r\n \"nodeSelector\": {\r\n \"matchLabels\": {\r\n \"beta.kubernetes.io/instance-type\": \"Standard_B4ms\"\r\n }\r\n }\r\n },\r\n {\r\n \"nodeSelector\": {\r\n \"matchExpressions\": [\r\n {\r\n \"key\": \"kubernetes.io/os\",\r\n \"values\": [\r\n \"linux\"\r\n ],\r\n \"operator\": \"In\"\r\n }\r\n ]\r\n },\r\n \"storageClass\": \"kibishii-storage-class\"\r\n },\r\n {\r\n \"nodeSelector\": {\r\n \"matchLabels\": {\r\n \"beta.kubernetes.io/instance-type\": \"Standard_B8ms\"\r\n }\r\n },\r\n \"storageClass\": \"kibishii-storage-class\"\r\n }\r\n ]\r\n}\r\n```\r\n\r\nThis sample demonstrates how the `loadAffinity` elements with `StorageClass` field and without `StorageClass` field setting work together.\r\nIf the VGDP mounting volume is created from StorageClass `kibishii-storage-class`, its pod will run Linux nodes or instance type as `Standard_B8ms`.\r\n\r\nThe other VGDP instances will run on nodes, which instance type is `Standard_B4ms`.\r\n\r\n#### LoadAffinity interacts with BackupPVC\r\n\r\n``` json\r\n{\r\n \"loadAffinity\": [\r\n {\r\n \"nodeSelector\": {\r\n \"matchLabels\": {\r\n \"beta.kubernetes.io/instance-type\": \"Standard_B4ms\"\r\n }\r\n },\r\n \"storageClass\": \"kibishii-storage-class\"\r\n },\r\n {\r\n \"nodeSelector\": {\r\n \"matchLabels\": {\r\n \"beta.kubernetes.io/instance-type\": \"Standard_B2ms\"\r\n }\r\n },\r\n \"storageClass\": \"worker-storagepolicy\"\r\n }\r\n ],\r\n \"backupPVC\": {\r\n \"kibishii-storage-class\": {\r\n \"storageClass\": \"worker-storagepolicy\"\r\n }\r\n }\r\n}\r\n```\r\n\r\nVelero data mover supports to use different StorageClass to create backupPVC by [design](https://github.com/vmware-tanzu/velero/pull/7982).\r\n\r\nIn this example, if the backup target PVC's StorageClass is `kibishii-storage-class`, its backupPVC should use StorageClass `worker-storagepolicy`. Because the final StorageClass is `worker-storagepolicy`, the backupPod uses the loadAffinity specified by `loadAffinity`'s elements with `StorageClass` field set to `worker-storagepolicy`. backupPod will be assigned to nodes, which instance type is `Standard_B2ms`.\r\n\r\n\r\n#### LoadAffinity interacts with RestorePVC\r\n\r\n``` json\r\n{\r\n \"loadAffinity\": [\r\n {\r\n \"nodeSelector\": {\r\n \"matchLabels\": {\r\n \"beta.kubernetes.io/instance-type\": \"Standard_B4ms\"\r\n }\r\n },\r\n \"storageClass\": \"kibishii-storage-class\"\r\n }\r\n ],\r\n \"restorePVC\": {\r\n \"ignoreDelayBinding\": false\r\n }\r\n}\r\n```\r\n\r\n##### StorageClass's bind mode is WaitForFirstConsumer\r\n\r\n``` yaml\r\napiVersion: storage.k8s.io/v1\r\nkind: StorageClass\r\nmetadata:\r\n name: kibishii-storage-class\r\nparameters:\r\n svStorageClass: worker-storagepolicy\r\nprovisioner: csi.vsphere.vmware.com\r\nreclaimPolicy: Delete\r\nvolumeBindingMode: WaitForFirstConsumer\r\n```\r\n\r\nIf restorePVC should be created from StorageClass `kibishii-storage-class`, and it's volumeBindingMode is `WaitForFirstConsumer`.\r\nAlthough `loadAffinityPerStorageClass` has a section matches the StorageClass, the `ignoreDelayBinding` is set `false`, the Velero exposer will wait until the target Pod scheduled to a node, and returns the node as SelectedNode for the restorePVC.\r\nAs a result, the `loadAffinityPerStorageClass` will not take affect.\r\n\r\n##### StorageClass's bind mode is Immediate\r\n\r\n``` yaml\r\napiVersion: storage.k8s.io/v1\r\nkind: StorageClass\r\nmetadata:\r\n name: kibishii-storage-class\r\nparameters:\r\n svStorageClass: worker-storagepolicy\r\nprovisioner: csi.vsphere.vmware.com\r\nreclaimPolicy: Delete\r\nvolumeBindingMode: Immediate\r\n```\r\n\r\nBecause the StorageClass volumeBindingMode is `Immediate`, although `ignoreDelayBinding` is set to `false`, restorePVC will not be created according to the target Pod.\r\n\r\nThe restorePod will be assigned to nodes, which instance type is `Standard_B4ms`.\r\n\r\n[1]: Implemented/unified-repo-and-kopia-integration/unified-repo-and-kopia-integration.md\r\n[2]: Implemented/volume-snapshot-data-movement/volume-snapshot-data-movement.md\r\n[3]: Implemented/node-agent-affinity.md" }, { "path": "design/Implemented/vgdp-micro-service/vgdp-micro-service.md", "content": "# VGDP Micro Service For Volume Snapshot Data Movement\n\n## Glossary & Abbreviation\n\n**VGDP**: Velero Generic Data Path. The collective of modules that is introduced in [Unified Repository design][1]. Velero uses these modules to finish data transmission for various purposes. It includes uploaders and the backup repository. \n**Volume Snapshot Data Movement**: The backup/restore method introduced in [Volume Snapshot Data Movement design][2]. It backs up snapshot data from the volatile and limited production environment into the durable, heterogeneous and scalable backup storage. \n**VBDM**: Velero Built-in Data Mover as introduced in [Volume Snapshot Data Movement design][2], it is the built-in data mover shipped along with Velero. \n**Exposer**: Exposer is introduced in [Volume Snapshot Data Movement design][2] and is used to expose the volume snapshots/target volumes for VGDP to access locally. \n\n## Background\nAs the architecture introduced in [Volume Snapshot Data Movement design][2], VGDP instances are running inside the node-agent pods, however, more and more use cases require to run the VGDP instances in dedicated pods, or in another word, make them as micro services, the benefits are as below:\n- This avoids VGDP to access volume data through host path, while host path access involves privilege escalations in some environments (e.g., must run under privileged mode), which makes challenge to users. \n- This enable users to to control resource (i.e., cpu, memory) request/limit in a granular manner, e.g., control them per backup/restore of a volume\n- This increases the resilience, crash of one VGDP activity won't affect others\n- In the cases that the backup storage must be represented by a Kubernetes persistent volumes (i.e., nfs storage, [COSI][3]), this avoids to dynamically mount the persistent volumes to node-agent pods and cause node-agent pods to restart (this is not accepted since node-agent lose it current state after its pods restart) \n- This prevents unnecessary full backup. Velero's fs uploaders support file level incremental backup by comparing the file name and metadata. However, at present the files are visited by host path, while pod and PVC's ID are part of the host path, so once the pod is recreated, the same file is regarded as a different file since the pod's ID has been changed. If the fs uploader is in a dedicated pod and files are visited by pod's volume path, files' full path are not changed after pod restarts, so incremental backups could continue. \n\n## Goals\n- Create a solution to make VGDP instances as micro services\n- Modify the VBDM to offload the VGDP work from node-agent to the VGDP micro service\n- Create the mechanism for VBDM to control and monitor the VGDP micro services in various scenarios\n\n## Non-Goals\n- The current solution covers Volume Snapshot Data Movement backup/restore type only, even though VGDP is also used by pod volume backup. It is less possible to do this for pod volume backup, since it must run inside the source workload pods. \n- The current solution covers VBDM only. 3rd data movers still follow the **Replacement** section of [Volume Snapshot Data Movement design][2]. That is, 3rd data movers handle the DUCR/DDCR on their own and they are free to make themselves micro service style or monolith service style. \n\n\n## Overview\nThe solution is based on [Volume Snapshot Data Movement design][2], the architecture is followed as is and existing components are not changed unless it is necessary. \nBelow lists the changed components, why and how: \n**Exposer**: Exposer is to expose the snapshot/target volume as a path/device name/endpoint that are recognizable by VGDP. Varying from the type of snapshot/target volume, a pod may be created as part of the expose. Now, since we run the VGDP instance in a separate pod, a pod is created anyway, we assume exposer creates a pod all the time and make the appropriate exposing configurations to the pod so that VGDP instance could access the snapshot/target volume locally inside the pod. The pod is still called as backupPod or restorePod. \nThen we need to change the command the backupPod/restorePod is running, the command launches VGDP-MS (VGDP Micro Service, see below) when the container starts up. \nFor CSI snapshot, the backupPod/restorePod is created as the result of expose, the only thing left is to change the backupPod/restorePod's image. \n**VBDM**: VBDM contains the data mover controller, while the controller calls the Exposer and launches the VGDP instances. Now, since the VGDP instance is launched by the backupPod/restorePod, the controller should not launch the VGDP instance again. However, the controller still needs to monitor and control the VGDP instance. Moreover, in order to avoid any contest situations, the controller is still the only place to update DUCRs and DDCRs. \n\nBesides the changes to above existing components, we need to add below new components: \n**VGDP Watcher**: We create a new module to help the data mover controller to watch activities of the VGDP instance in the backupPod/restorePod. VGDP Watcher is a part of VBDM. \n**VGDP-MS**: VGDP Micro Service is the binary for the command backupPod/restorePod runs. It accepts the parameters and then launches the VGDP instance according to the request type, specifically, backup or restore. VGDP-MS also runs other modules to sync-up with the data mover controller. VGDP-MS is also a part of VBDM. \n\nBelow diagram shows how these components work together: \n![vgdp-ms-1.png](vgdp-ms-1.png) \n\nThe [Node-agent concurrency][4] is still used to control the concurrency of VGDP micro services. When there are too many volumes in the backup/restore, which takes too much computing resources(CPU, memory, etc.) or Kubernetes resources(pods, PVCs, PVs, etc.), users could set the concurrency in each node so as to control the total number of concurrent VGDP micro services in the cluster. \n\n## Detailed Design\n### Exposer\nAt present, the exposer creates backupPod/restorePod and sets ```velero-helper pause``` as the command run by backupPod/restorePod. \nNow, VGDP-MS command will be used, and the ```velero``` image will be running inside the backupPod/restorePod. The command is like below: \n```velero data-mover backup --volume-path xxx --volume-mode xxx --data-upload xxx --resource-timeout xxx --log-format xxx --log-level xxx``` \nOr: \n```velero data-mover restore --volume-path xxx --volume-mode xxx --data-download xxx --resource-timeout xxx --log-format xxx --log-level xxx``` \n\nThe first one is for backup and the other one is for restore. \nBelow are the parameters of the commands: \n**volume-path**: Deliver the full path inside the backupPod/restorePod for the volume to be backed up/restored. \n**volume-mode**: Deliver the mode for the volume be backed up/restored, at present either ```Filesystem``` mode or ```Block``` mode. \n**data-upload**: DUCR for this backup. \n**data-download**: DDCR for this backup. \n**resource-timeout**: resource-timeout is used to control the timeout for operations related to resources. It has the same meaning with the resource-timeout for node-agent. \n**log-format** and **log-level**: This is to control the behavior of log generation inside VGDP-MS. \n\nIn order to have the same capability and permission with node-agent, below pod configurations are inherited from node-agent and set to backupPod/restorePod's spec:\n- Volumes: Some configMaps will be mapped as volumes to node-agent, so we add the same volumes of node-agent to the backupPod/restorePod\n- Environment Variables\n- Security Contexts \nWe may not actually need all the capabilities in the VGDP-MS as the node-agent. At present, we just duplicate all of them, if we find any problem in future, we can filter out the capabilities that are not required by VGDP-MS. \nThe backupPod/restorePod is not run in Privileged mode as it is not required since the volumes are visisted by pod path. \nThe root user is still required, especially by the restore (in order to restore the file system attributes, owners, etc.), so we will use root user for backupPod/restorePod. \nWe set backupPod/restorePod's ```RestartPolicy``` to ```RestartPolicyNever```, so that once VGDP-MS terminates in any reason, backupPod/restorePod won't restart and the DUCR/DDCR is marked as one of the terminal phases (Completed/Failed/Cancelled) accordingly. \n\n\n### VGDP Watcher\n#### Dual mode event watch\nThe primary task of VGDP Watcher is to watch the status change from backupPod/restorePod or the VGDP instance, so as to inform the data mover controller in below situations:\n- backupPod/restorePod starts\n- VGDP instance starts\n- Progress update\n- VGDP instance completes/fails/cancelled\n- backupPod/restorePod stops\n\nWe use two mechanism to make the watch: \n**Pod Phases**: VGDP Watcher watches the backupPod/restorePod's phases updated by Kubernetes. That is, VGDP Watcher creates an informer to watch the pod resource for the backupPod/restorePod and detect that the pod reaches to one of the terminated phases (i.e., PodSucceeded, PodFailed). We also check the availability & status of the backupPod/restorePod at the beginning of the watch so as to detect the starting of the backupPod/restorePod. \n**Custom Kubernetes Events**: VGDP-MS generates Kubernetes events and associates them to the DUCR/DDCR at the time of VGDP instance starting/stopping and progress update, then VGDP Watcher creates another informer to watch the Event resource associated to the DUCR/DDCR. \n\nPod Phases watch covers the entire lifecycle of the backupPod/restorePod, but we don't know the status of the VGDP instance through it; and it can only deliver information by then end of the pod lifecycle. \nCustom Event watch generates details of the VGDP instances and the events could be generated any time; but it cannot generate notifications before VGDP starts or in the case that VGDP crashes or shutdown abnormally. \n\nTherefore, we adopt the both mechanisms to VGDP Watcher. In the end, there will be two sources generating the result of VGDP-MS: \n- The termination message of backupPod/restorePod\n- The message along with the VGDP Instance Completes/Fails/Cancelled event \n\nOn the one hand, in some cases only the backupPod/restorePod's termination message is available, e.g., the backupPod/restorePod crashes or or backupPod/restorePod quits before VGDP instance is started. So we refer to the first mechanism to get the notifications. \nOn the other hand, if they are both available, we have the results from them for mutual verification. \n\nConclusively, under the help of VGDP Watcher, data mover controller starts VGDP-MS controllably and waits until VGDP-MS ends under any circumstances. \n\n#### AsyncBR adapter\nVGDP Watcher needs to notify the data mover controller when one of the watched event happens, so that the controller could do the operations as if it receives the same callbacks from VGDP as the current behavior. In order not to break the existing code logics of data mover controllers, we make VGDP Watcher as an adapter of AsyncBR which is the interface implemented by VGDP and called by the data mover controller. \nSince the parameters to call VGDP Watcher is different from the ones to call VGDP, we change the AsyncBR interface to hide some parameters from one another, the new interface is as below: \n```\ntype AsyncBR interface {\n\t// Init initializes an asynchronous data path instance\n\tInit(ctx context.Context, res *exposer.ExposeResult, param interface{}) error\n\n\t// StartBackup starts an asynchronous data path instance for backup\n\tStartBackup(dataMoverConfig map[string]string, param interface{}) error\n\n\t// StartRestore starts an asynchronous data path instance for restore\n\tStartRestore(snapshotID string, dataMoverConfig map[string]string) error\n\n\t// Cancel cancels an asynchronous data path instance\n\tCancel()\n\n\t// Close closes an asynchronous data path instance\n\tClose(ctx context.Context)\n}\n```\nSome parameters are hidden into ```param```, but the functions and calling logics are not changed. \n\nVGDP Watcher should be launched by the data mover controller before VGDP instance starts, otherwise, multiple corner problems may happen. E.g., VGDP-MS may run the VGDP instance immediately after the backupPod/restorePod is launched and completes it before the data mover controller starts VGDP Watcher, as a result, multiple informs are missed from VGDP Watcher. \nTherefore, the controller launches VGDP Watcher first and then set the DUCR/DDCR to ```InProgress```; on the other hand, VGDP-MS waits DUCR/DDCR turns to ```InProgress``` before running the VGDP instance. \n\n### VGDP-MS\nVGDP-MS is represented by ```velero data-mover``` subcommand and has its own subcommand ```backup``` and ```restore```. \nBelow diagram shows the VGDP-MS workflow: \n![vgdp-ms-2.png](vgdp-ms-2.png) \n\n**Start DUCR/DDCR Watcher**: VGDP-MS needs to watch the corresponding DUCR/DDCR so as to react on some events happening to the DUCR/DDCR. E.g., when the data movement is cancelled, a ```Cancel``` flag is set to the DUCR/DDCR, by watching the DUCR/DDCR, VGDP-MS is able to see it and cancel the VGDP instance. \n**Wait DUCR/DDCR InProgress**: As mentioned above, VGDP-MS won't start the VGDP instance until DUCR/DDCR turns to ```InProgress```, by which time VGDP Watcher has been started. \n**Record VGDP Starts**: This generates the VGDP Instance Starts event. \n**VGDP Callbacks**: When VGDP comes to one of the terminal states (i.e., completed, failed, cancelled), the corresponding callback is called. \n**Record VGDP Ends**: This generates the VGDP Instance Completes/Fails/Cancelled event, and also generates backupPod/restorePod termination message. \n**Record VGDP Progress**: This periodically generates/updates the Progress event with totalBytes/bytesDone to indicate the progress of the data movement. \n**Set VGDP Output**: This writes the termination message to the backupPod/restorePod's termination log (by default, it is written to ```/dev/termination-log```). \n\nIf VGDP completes, VGDP Instance Completes event and backupPod/restorePod termination shares the same message as below: \n```\ntype BackupResult struct {\n\tSnapshotID string `json:\"snapshotID\"`\n\tEmptySnapshot bool `json:\"emptySnapshot\"`\n\tSource exposer.AccessPoint `json:\"source,omitempty\"`\n}\n```\n```\ntype RestoreResult struct {\n\tTarget exposer.AccessPoint `json:\"target,omitempty\"`\n}\n``` \n``` \ntype AccessPoint struct {\n\tByPath string `json:\"byPath\"`\n\tVolMode uploader.PersistentVolumeMode `json:\"volumeMode\"`\n}\n``` \n\nThe existing VGDP result structures are actually being reused, we just add the json markers so that they can be marshalled. \n\nAs mentioned above, once VGDP-MS ends in any way, the backupPod/restorePod terminates and never restarts, so the end of VGDP-MS means the end of DU/DD. \n\nFor Progress update, the existing Progress structure is being reused: \n``` \ntype Progress struct {\n\tTotalBytes int64 `json:\"totalBytes,omitempty\"`\n\tBytesDone int64 `json:\"doneBytes,omitempty\"`\n}\n``` \n\n### Log Collection\nDuring the running of VGDP instance, some logs are generated which are important for troubleshooting. This includes all the logs generated by the uploader and repository. Therefore, it is important to collect these logs. \nOn the other hand, the logs are now generated in the backupPod/restorePod, while the backupPod/restorePod is deleted immediately after the data movement completes. Therefore, by default, ```velero debug``` is not able to collect these logs. \n\nAs a solution, we use logrus's hook mechanism to redirect the backupPod/restorePod's logs into node-agent's log, so that ```velero debug``` could collect VGDP logs as is without any changes. \n\nBelow diagram shows how VGDP logs are redirected: \n![vgdp-ms-3.png](vgdp-ms-3.png) \n\nThis log redirecting mechanism is thread safe since the hook acquires the write lock before writing the log buffer, so it guarantees that in the node-agent log there is no corruptions after redirecting the log, and the redirected logs and the original node-agent logs are not projected into each other. \n\n### Resource Control\nThe CPU/memory resource of backupPod/restorePod is configurable, which means users are allowed to configure resources per volume backup/restore. \nBy default, the [Best Effort policy][5] is used, and users are allowed to change it through the ConfigMap specified by `velero node-agent` CLI's parameter `--node-agent-configmap`. Specifically, we add below structures to the ConfigMap: \n``` \ntype Configs struct {\n\t// PodResources is the resource config for various types of pods launched by node-agent, i.e., data mover pods.\n\tPodResources *PodResources `json:\"podResources,omitempty\"`\n}\n\ntype PodResources struct {\n\tCPURequest string `json:\"cpuRequest,omitempty\"`\n\tMemoryRequest string `json:\"memoryRequest,omitempty\"`\n\tCPULimit string `json:\"cpuLimit,omitempty\"`\n\tMemoryLimit string `json:\"memoryLimit,omitempty\"`\n}\n```\nThe string values must mactch Kubernetes Quantity expressions; for each resource, the \"request\" value must not be larger than the \"limit\" value. Otherwise, if any one of the values fail, all the resource configurations will be ignored. \n\nThe configurations are loaded by node-agent at start time, so users can change the values in the configMap any time, but the changes won't effect until node-agent restarts. \n\n\n## node-agent\nnode-agent is still required. Even though VGDP is now not running inside node-agent, node-agent still hosts the data mover controller which reconciles DUCR/DDCR and operates DUCR/DDCR in other steps before the VGDP instance is started, i.e., Accept, Expose, etc. \nPrivileged mode and root user are not required for node-agent anymore by Volume Snapshot Data Movement, however, they are still required by PVB(PodVolumeBackup) and PVR(PodVolumeRestore). Therefore, we will keep the node-agent deamonset as is, for any users who don't use PVB/PVR and have concern about the privileged mode/root user, they need to manually modify the deamonset spec to remove the dependencies. \n\n## CRD Changes\nThere is no changes to any CRD. \n\n## Installation Changes\nNo changes to installation, the backupPod/restorePod's configurations are all inherited from node-agent. \n\n## Upgrade\nUpgrade is not impacted. \n\n## CLI\nCLI is not changed.\n\n\n\n[1]: ../unified-repo-and-kopia-integration/unified-repo-and-kopia-integration.md\n[2]: ../volume-snapshot-data-movement/volume-snapshot-data-movement.md\n[3]: https://kubernetes.io/blog/2022/09/02/cosi-kubernetes-object-storage-management/\n[4]: ../node-agent-concurrency.md\n[5]: https://kubernetes.io/docs/concepts/workloads/pods/pod-qos/\n\n" }, { "path": "design/Implemented/vgdp-micro-service-for-fs-backup/vgdp-micro-service-for-fs-backup.md", "content": "# VGDP Micro Service For fs-backup\n\n## Glossary & Abbreviation\n\n**VGDP**: Velero Generic Data Path. The collective modules that is introduced in [Unified Repository design][1]. Velero uses these modules to finish data transmission for various purposes. It includes uploaders and the backup repository. \n**fs-backup**: Also known as pod volume backup (PVB)/pod volume restore (PVR). It is one of the primary backup methods built-in with Velero. It has been refactored in [Unified Repository design][1]. \n**PVB**: Pod Volume Backup, the internal name for backup part of fs-backup. \n**PVR**: Pod Volume Restore, the internal name for restore part of fs-backup. \n**Exposer**: Exposer is introduced in [Volume Snapshot Data Movement design][2] and is used to expose the volume snapshots/volumes for VGDP to access locally. \n**VGDP MS**: VGDP Micro Service, it is introduced in [VGDP Micro Service For Volume Snapshot Data Movement][3]. It hosts VGDP instances in dedicated backup/restore pods, instead of in node-agent pods. \n\n## Background\nAs described in [VGDP Micro Service For Volume Snapshot Data Movement][3], hosting VGDP instances in dedicated pods has solved many major problems and brought significant improvements in scalability. These improvements are also effective for fs-backup. And besides the benefits listed in [VGDP Micro Service For Volume Snapshot Data Movement][3], we can also see below ones specifically for fs-backup:\n- This enables fs-backup to support Windows workloads. Windows doesn't support propagate mount, so the current fs-backup solution doesn't work for Windows nodes and Windows workloads. However, if the final host-path for the source volume is mounted to the VGDP MS pods, it should work. \n- This enables fs-backup to reuse the existing VGDP features seamlessly, i.e., concurrency control, node selector, etc.\n\nBy moving all VGDP instances out of node-agent pods, we would further get prepared for below important features and improvements:\n- NFS support: NFS volumes are mounted to VGDP MS pods, so node-agent pods don't need to restart when a new BSL is added.\n- Performance improvement for Kopia uploader restore ([#7725][9]): dedicated cache volumes could be mounted to the VGDP MS pods, without affecting node-agent pods.\n- Controllable resource usage for node-agent: node-agent pods are long running and so not suitable for data path activities as the OS usually reclaim memory in a lazy reclaim behavior, so the unused memory may be shown as occupied by node-agent pods, which misleads Kubernetes or other related sub system. After this change, node-agent pods no longer require large resource (CPU/memory) usage, so no obvious memory retain will be observed.\n- Simplify node-agent configuration: host-path mounts, root user and privileged mode are no longer required by node-agent; and the configuration differences of node-agent for linux and Windows nodes could be eliminated. \n\n## Goals\n- Create a solution to make VGDP instances as micro services for fs-backup\n- Modify the fs-backup workflow to offload the VGDP work from node-agent to the VGDP MS\n- Create the mechanism for fs-backup to control and monitor the VGDP MS in various scenarios\n\n## Non-Goals\n- The current solution covers the VGDP Micro Service for fs-backup itself, the potentional features/improvements that rely on this solution will be covered by further designs and implementations. \n\n\n## Overview\nThe solution is based on [VGDP Micro Service For Volume Snapshot Data Movement][3], the architecture is followed as is and existing components are not changed unless it is necessary. \nBelow diagram shows how these components work together: \n![vgdp-ms-1.png](vgdp-ms-1.png) \n\nBelow lists the changed components, why and how: \n**Pod-Volume Exposer**: A new exposer, pod-volume exposer is added. It retrieves the host path of the specific volume and then creates the backupPod/restorePod and mounts the host path to the pod. The command of the backupPod/restorePod is also changed to launch VGDP MS for PVB/PVR. \n**PVB/PVR Controller**: The PVB/PVR controllers are refactored to work with podVolume exposer, VGDP-MS, etc. The controllers will also support Cancel and resume. So PVB/PVR CRD is also refactored to support these scenarios. \n**PVB/PVR VGDP-MS**: New commands for PVB/PVR VGDP-MS are added. The VGDP instances are started in the backupPod/restorePod as result of the commands. \n\nThe VGDP Watcher and its mechanism are fully reused.\n\nThe [Node-agent concurrency][4] is reused to control the concurrency of VGDP MS for fs-backup. When there are too many volumes in the backup/restore, which takes too much computing resources(CPU, memory, etc.) or Kubernetes resources(pods, PVCs, PVs, etc.), users could set the concurrency in each node so as to control the total number of concurrent VGDP instances in the cluster. \n\n## Detailed Design\n### Exposer\nAs the old behavior, the host path (e.g., `/var/lib/kubelet/pods`) for the Kubernetes pods are mounted to node-agent pods, then the VGDP instances running in the same pods access the data through subdir of the host path for a specific volume, e.g., `/var/lib/kubelet/pods//volumes/kubernetes.io~csi//mount`. Therefore, a node-agent pod could access all volumes attached to the same node. \nFor the new implementation, the exposer retrieves the host path for a specific volume directly, and then mount that host path to the backupPod/restorePod. This also means that the backupPod/restorePod could only access the volume to be backed up or restored. \n\nThe exposer creates backupPod/restorePod and sets ```velero pod-volume``` as the command run by backupPod/restorePod. And `velero` image is used for the backupPod/restorePod. \nThere are sub commands varying from backup and restore: \n```velero pod-volume backup --volume-path xxx --pod-volume-backup xxx --resource-timeout xxx --log-format xxx --log-level xxx``` \nOr: \n```velero pod-volume restore --volume-path xxx --pod-volume-restore xxx --resource-timeout xxx --log-format xxx --log-level xxx``` \n\nBelow are the parameters of the commands: \n**volume-path**: Deliver the full path inside the backupPod/restorePod for the volume to be backed up/restored. \n**pod-volume-backup**: PVB CR for this backup. \n**pod-volume-restore**: PVR CR for this restore. \n**resource-timeout**: resource-timeout is used to control the timeout for operations related to resources. It has the same meaning with the resource-timeout for node-agent. \n**log-format** and **log-level**: This is to control the behavior of log generation inside VGDP-MS. \n\nBelow pod configurations are inherited from node-agent and set to backupPod/restorePod's spec:\n- Volumes: Some configMaps will be mapped as volumes to node-agent, so we add the same volumes of node-agent to the backupPod/restorePod\n- Environment Variables\n- Security Contexts \n\nSince the volume data is still accessed by host path, the backupPod/restorePod may still need to run in Privileged mode in some environments. Therefore, the Privileged mode setting which is a part of Security Contexts will be inherited from node-agent. \nThe root user is still required, especially by the restore (in order to restore the file system attributes, owners, etc.), so we will use root user for backupPod/restorePod. \n\nAs same as [VGDP Micro Service For Volume Snapshot Data Movement][3], the backupPod/restorePods's ```RestartPolicy``` is set to ```RestartPolicyNever```, so that once VGDP-MS terminates for any reason, backupPod/restorePod won't restart and the PVB/PVR is marked as one of the terminal phases (Completed/Failed/Cancelled) accordingly. \n\n### VGDP Watcher\nThe VGDP watcher is fully reused, specifically, we still use the dual mode event watcher to watch the status change from backupPod/restorePod or the VGDP instance. \nThe AsyncBR adapter and its interface is also fully reused. \n\n### VGDP-MS\nThe VGDP-MS that is represented by ```velero pod-volume``` keeps the same workflow as [VGDP Micro Service For Volume Snapshot Data Movement][3]: \n![vgdp-ms-2.png](vgdp-ms-2.png) \n\n**Start DUCR/DDCR Watcher**: The same as [VGDP Micro Service For Volume Snapshot Data Movement][3], except that it watches PVB/PVR CRs. \n**Wait DUCR/DDCR InProgress**: The same as The same as [VGDP Micro Service For Volume Snapshot Data Movement][3], VGDP-MS won't start the VGDP instance until PVB/PVR CR turns to ```InProgress```. \n**Record VGDP Starts**: The same as [VGDP Micro Service For Volume Snapshot Data Movement][3]. \n**VGDP Callbacks**: The same as [VGDP Micro Service For Volume Snapshot Data Movement][3]. \n**Record VGDP Ends**: The same as [VGDP Micro Service For Volume Snapshot Data Movement][3]. \n**Record VGDP Progress**: The same as [VGDP Micro Service For Volume Snapshot Data Movement][3]. \n**Set VGDP Output**: The same as [VGDP Micro Service For Volume Snapshot Data Movement][3]. \n\nThe return message for VGDP completion is also reused, except that `VolMode` is always set to `PersistentVolumeFilesystem`: \n```\ntype BackupResult struct {\n SnapshotID string `json:\"snapshotID\"`\n EmptySnapshot bool `json:\"emptySnapshot\"`\n Source exposer.AccessPoint `json:\"source,omitempty\"`\n}\n```\n```\ntype RestoreResult struct {\n Target exposer.AccessPoint `json:\"target,omitempty\"`\n}\n``` \n``` \ntype AccessPoint struct {\n ByPath string `json:\"byPath\"`\n VolMode uploader.PersistentVolumeMode `json:\"volumeMode\"`\n}\n``` \n\nAnd the mechanism and data struct for Progress update is also reused:\n``` \ntype Progress struct {\n TotalBytes int64 `json:\"totalBytes,omitempty\"`\n BytesDone int64 `json:\"doneBytes,omitempty\"`\n}\n``` \n\n### Log Collection\nThe log collection mechanism is the same as [VGDP Micro Service For Volume Snapshot Data Movement][3]. \n\n### Resource Control\nThe resource control mechanism is the same as [VGDP Micro Service For Volume Snapshot Data Movement][3]. \n\n### Restic Restore\nAs the current Restic path deprecation process, restore is still supported. On the other hand, we don't want to support Restic path for this new VGDP MS implementation. \nTherefore, the legacy PVR controller and workflow is preserved for Restic path restore. The controller watches legacy PVRs only, and then launches the legacy workflow. Meawhile, the new PVR controller should skip legacy PVRs. \nAfter Restic path is full deprecated, the code for the legacy controller and workflow should be removed. \n\n### Velero Server Restarts\nThe backup/restore stays in InProgress phase during the running of PVB/PVR, no phase changes between completion of item iteration and completion of PVB/PVR. As a result, on Velero server restarts, there is no way to resume a backup/restore. \nTherefore, the backup/restore will be be marked as Failed, which is the same as the old behavior. And it is still not as good as CSI snapshot data movement for which the backup/restore could be resumed as long as it has iterated all items. \nBy the meanwhile, there is indeed some improvements. As the old behavior, once the backup/restore is set as Failed on Velero server restart, the running PVB/PVR will be left there, as a result, the VGDP instances may run for a long time and take lots of resource for nothing; for the new implementation, PVB/PVR will be set as Cancel immediately after the backup/restore is set as Failed. \n\n### node-agent Restarts\nAs the old behavior, once a node-agent pod restarts, all the PVBs/PVRs running in the same node will be set as Failed as there is no way to resume the VGDP instances for them. \nFor the new implementation, since the VGDP instances run in dedicated backupPods/restorePods without affected, the PVBs/PVRs will be resumed after node-agent restarts. This includes PVBs/PVRs in all phases. \n\nThe legacy PVRs handling Restic restore are processed by the old workflow, so they will still be set as Failed on node-agent restart. \n\n### Windows Support\nWindows nodes and workloads will be supported by following the same changes for CSI snapshot data movement as listed in [Velero Windows Support][7]. There are some additional changes particularly for PVB/PVR. \n\n#### Restore Helper \nPVR requires an init-container, called `restore-wait`, to run in the workload pod. There are default configurations for the container and users could customize them by the `pod-volume-restore` RIA plugin configMap.\nThe `pod-volume-restore` RIA is used to config the init-container, so it should support Windows pods for all the configurations. \nMeanwhile, the customized options in the configMap should also support Windows pods. If an option is not suitable for Windows pods, it will be ignored by the RIA. \n\nBy default, the init-container uses `velero` image with a binary called `velero-restore-helper` inside, so that binary should be compiled and assembled to the `velero` image for Windows. \n\n#### Privileged mode\nPrivileged pods are implemented by [HostProcess Pods][8] on Windows and need to be specially configured. And there are many constrains for it. \nAs one of the constrains, HostProcess pods supports Windows service accounts only. As a result, restore will not be able to support it until [#8423][10] is fixed, otherwise, the restored files are not usable by workloads which run under genneral container users, e.g., `containerUser` or `containerAdministrator`. \nTherefore, as the current implementation, fs-backup will not support Windows workloads in the environments where Privileged mode is required. A limitation should be documented. \n\n## node-agent\nnode-agent is required to host the PVB/PVR controller which reconciles PVB/PVR and operates PVB/PVR in other steps before the VGDP instance is started, i.e., Accept, Expose, etc. \nnode-agent still requires host path mount because of two deprecating features [in-tree storage provider support deprecation][5] and [emptyDir volume support deprecation][6]. As a result, Privileged mode and root user are still required in some environments. Therefore, we will keep the node-agent deamonset as is, until the two deprecations complete. \n\n## CRD Changes\nIn order to support the VGDP MS workflow, some elements in the PVB/PVR CRDs are added or extended:\n- New phases are added for PVB/PVR: `PodVolumeBackupPhaseAccepted`, `PodVolumeBackupPhasePrepared`, `PodVolumeBackupPhaseCanceling`, `PodVolumeBackupPhaseCanceled`; `PodVolumeRestorePhaseAccepted`, `PodVolumeRestorePhasePrepared`, `PodVolumeRestorePhaseCanceling`, `PodVolumeRestorePhaseCanceled`. \n- New fields are added to PVB/PVR spec to support cancel: `Cancel bool`\n- New fields are added to PVB/PVR spec to support the accept phase and processing: `AcceptedTimestamp *metav1.Time`\n- A new field, which records the node the PVR is running, is added to PVR Status: `Node string`\n\nNew changes happen to Backup/Restore CRDs. \n\nBelow is the new PVB CRD:\n```yaml\napiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: v0.16.5\n name: podvolumebackups.velero.io\nspec:\n group: velero.io\n names:\n kind: PodVolumeBackup\n listKind: PodVolumeBackupList\n plural: podvolumebackups\n singular: podvolumebackup\n scope: Namespaced\n versions:\n - additionalPrinterColumns:\n - description: PodVolumeBackup status such as New/InProgress\n jsonPath: .status.phase\n name: Status\n type: string\n - description: Time duration since this PodVolumeBackup was started\n jsonPath: .status.startTimestamp\n name: Started\n type: date\n - description: Completed bytes\n format: int64\n jsonPath: .status.progress.bytesDone\n name: Bytes Done\n type: integer\n - description: Total bytes\n format: int64\n jsonPath: .status.progress.totalBytes\n name: Total Bytes\n type: integer\n - description: Name of the Backup Storage Location where this backup should be\n stored\n jsonPath: .spec.backupStorageLocation\n name: Storage Location\n type: string\n - description: Time duration since this PodVolumeBackup was created\n jsonPath: .metadata.creationTimestamp\n name: Age\n type: date\n - description: Name of the node where the PodVolumeBackup is processed\n jsonPath: .status.node\n name: Node\n type: string\n - description: The type of the uploader to handle data transfer\n jsonPath: .spec.uploaderType\n name: Uploader\n type: string\n name: v1\n schema:\n openAPIV3Schema:\n properties:\n apiVersion:\n description: |-\n APIVersion defines the versioned schema of this representation of an object.\n Servers should convert recognized schemas to the latest internal value, and\n may reject unrecognized values.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\n type: string\n kind:\n description: |-\n Kind is a string value representing the REST resource this object represents.\n Servers may infer this from the endpoint the client submits requests to.\n Cannot be updated.\n In CamelCase.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\n type: string\n metadata:\n type: object\n spec:\n description: PodVolumeBackupSpec is the specification for a PodVolumeBackup.\n properties:\n backupStorageLocation:\n description: |-\n BackupStorageLocation is the name of the backup storage location\n where the backup repository is stored.\n type: string\n cancel:\n description: |-\n Cancel indicates request to cancel the ongoing PodVolumeBackup. It can be set\n when the PodVolumeBackup is in InProgress phase\n type: boolean\n node:\n description: Node is the name of the node that the Pod is running\n on.\n type: string\n pod:\n description: Pod is a reference to the pod containing the volume to\n be backed up.\n properties:\n apiVersion:\n description: API version of the referent.\n type: string\n fieldPath:\n description: |-\n If referring to a piece of an object instead of an entire object, this string\n should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2].\n For example, if the object reference is to a container within a pod, this would take on a value like:\n \"spec.containers{name}\" (where \"name\" refers to the name of the container that triggered\n the event) or if no container name is specified \"spec.containers[2]\" (container with\n index 2 in this pod). This syntax is chosen only to have some well-defined way of\n referencing a part of an object.\n type: string\n kind:\n description: |-\n Kind of the referent.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\n type: string\n name:\n description: |-\n Name of the referent.\n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\n type: string\n namespace:\n description: |-\n Namespace of the referent.\n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/\n type: string\n resourceVersion:\n description: |-\n Specific resourceVersion to which this reference is made, if any.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency\n type: string\n uid:\n description: |-\n UID of the referent.\n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids\n type: string\n type: object\n x-kubernetes-map-type: atomic\n repoIdentifier:\n description: RepoIdentifier is the backup repository identifier.\n type: string\n tags:\n additionalProperties:\n type: string\n description: |-\n Tags are a map of key-value pairs that should be applied to the\n volume backup as tags.\n type: object\n uploaderSettings:\n additionalProperties:\n type: string\n description: |-\n UploaderSettings are a map of key-value pairs that should be applied to the\n uploader configuration.\n nullable: true\n type: object\n uploaderType:\n description: UploaderType is the type of the uploader to handle the\n data transfer.\n enum:\n - kopia\n - \"\"\n type: string\n volume:\n description: |-\n Volume is the name of the volume within the Pod to be backed\n up.\n type: string\n required:\n - backupStorageLocation\n - node\n - pod\n - repoIdentifier\n - volume\n type: object\n status:\n description: PodVolumeBackupStatus is the current status of a PodVolumeBackup.\n properties:\n acceptedTimestamp:\n description: |-\n AcceptedTimestamp records the time the pod volume backup is to be prepared.\n The server's time is used for AcceptedTimestamp\n format: date-time\n nullable: true\n type: string\n completionTimestamp:\n description: |-\n CompletionTimestamp records the time a backup was completed.\n Completion time is recorded even on failed backups.\n Completion time is recorded before uploading the backup object.\n The server's time is used for CompletionTimestamps\n format: date-time\n nullable: true\n type: string\n message:\n description: Message is a message about the pod volume backup's status.\n type: string\n path:\n description: Path is the full path within the controller pod being\n backed up.\n type: string\n phase:\n description: Phase is the current state of the PodVolumeBackup.\n enum:\n - New\n - Accepted\n - Prepared\n - InProgress\n - Canceling\n - Canceled\n - Completed\n - Failed\n type: string\n progress:\n description: |-\n Progress holds the total number of bytes of the volume and the current\n number of backed up bytes. This can be used to display progress information\n about the backup operation.\n properties:\n bytesDone:\n format: int64\n type: integer\n totalBytes:\n format: int64\n type: integer\n type: object\n snapshotID:\n description: SnapshotID is the identifier for the snapshot of the\n pod volume.\n type: string\n startTimestamp:\n description: |-\n StartTimestamp records the time a backup was started.\n Separate from CreationTimestamp, since that value changes\n on restores.\n The server's time is used for StartTimestamps\n format: date-time\n nullable: true\n type: string\n type: object\n type: object\n served: true\n storage: true\n subresources: {}\n```\n\nBelow is the new PVR CRD:\n```yaml\napiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\nmetadata:\n annotations:\n controller-gen.kubebuilder.io/version: v0.16.5\n name: podvolumerestores.velero.io\nspec:\n group: velero.io\n names:\n kind: PodVolumeRestore\n listKind: PodVolumeRestoreList\n plural: podvolumerestores\n singular: podvolumerestore\n scope: Namespaced\n versions:\n - additionalPrinterColumns:\n - description: PodVolumeRestore status such as New/InProgress\n jsonPath: .status.phase\n name: Status\n type: string\n - description: Time duration since this PodVolumeRestore was started\n jsonPath: .status.startTimestamp\n name: Started\n type: date\n - description: Completed bytes\n format: int64\n jsonPath: .status.progress.bytesDone\n name: Bytes Done\n type: integer\n - description: Total bytes\n format: int64\n jsonPath: .status.progress.totalBytes\n name: Total Bytes\n type: integer\n - description: Name of the Backup Storage Location where the backup data is stored\n jsonPath: .spec.backupStorageLocation\n name: Storage Location\n type: string\n - description: Time duration since this PodVolumeRestore was created\n jsonPath: .metadata.creationTimestamp\n name: Age\n type: date\n - description: Name of the node where the PodVolumeRestore is processed\n jsonPath: .status.node\n name: Node\n type: string\n - description: The type of the uploader to handle data transfer\n jsonPath: .spec.uploaderType\n name: Uploader Type\n type: string\n name: v1\n schema:\n openAPIV3Schema:\n properties:\n apiVersion:\n description: |-\n APIVersion defines the versioned schema of this representation of an object.\n Servers should convert recognized schemas to the latest internal value, and\n may reject unrecognized values.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\n type: string\n kind:\n description: |-\n Kind is a string value representing the REST resource this object represents.\n Servers may infer this from the endpoint the client submits requests to.\n Cannot be updated.\n In CamelCase.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\n type: string\n metadata:\n type: object\n spec:\n description: PodVolumeRestoreSpec is the specification for a PodVolumeRestore.\n properties:\n backupStorageLocation:\n description: |-\n BackupStorageLocation is the name of the backup storage location\n where the backup repository is stored.\n type: string\n cancel:\n description: |-\n Cancel indicates request to cancel the ongoing PodVolumeRestore. It can be set\n when the PodVolumeRestore is in InProgress phase\n type: boolean\n pod:\n description: Pod is a reference to the pod containing the volume to\n be restored.\n properties:\n apiVersion:\n description: API version of the referent.\n type: string\n fieldPath:\n description: |-\n If referring to a piece of an object instead of an entire object, this string\n should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2].\n For example, if the object reference is to a container within a pod, this would take on a value like:\n \"spec.containers{name}\" (where \"name\" refers to the name of the container that triggered\n the event) or if no container name is specified \"spec.containers[2]\" (container with\n index 2 in this pod). This syntax is chosen only to have some well-defined way of\n referencing a part of an object.\n type: string\n kind:\n description: |-\n Kind of the referent.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\n type: string\n name:\n description: |-\n Name of the referent.\n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\n type: string\n namespace:\n description: |-\n Namespace of the referent.\n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/\n type: string\n resourceVersion:\n description: |-\n Specific resourceVersion to which this reference is made, if any.\n More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency\n type: string\n uid:\n description: |-\n UID of the referent.\n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids\n type: string\n type: object\n x-kubernetes-map-type: atomic\n repoIdentifier:\n description: RepoIdentifier is the backup repository identifier.\n type: string\n snapshotID:\n description: SnapshotID is the ID of the volume snapshot to be restored.\n type: string\n sourceNamespace:\n description: SourceNamespace is the original namespace for namespace\n mapping.\n type: string\n uploaderSettings:\n additionalProperties:\n type: string\n description: |-\n UploaderSettings are a map of key-value pairs that should be applied to the\n uploader configuration.\n nullable: true\n type: object\n uploaderType:\n description: UploaderType is the type of the uploader to handle the\n data transfer.\n enum:\n - kopia\n - \"\"\n type: string\n volume:\n description: Volume is the name of the volume within the Pod to be\n restored.\n type: string\n required:\n - backupStorageLocation\n - pod\n - repoIdentifier\n - snapshotID\n - sourceNamespace\n - volume\n type: object\n status:\n description: PodVolumeRestoreStatus is the current status of a PodVolumeRestore.\n properties:\n acceptedTimestamp:\n description: |-\n AcceptedTimestamp records the time the pod volume restore is to be prepared.\n The server's time is used for AcceptedTimestamp\n format: date-time\n nullable: true\n type: string\n completionTimestamp:\n description: |-\n CompletionTimestamp records the time a restore was completed.\n Completion time is recorded even on failed restores.\n The server's time is used for CompletionTimestamps\n format: date-time\n nullable: true\n type: string\n message:\n description: Message is a message about the pod volume restore's status.\n type: string\n node:\n description: Node is name of the node where the pod volume restore\n is processed.\n type: string\n phase:\n description: Phase is the current state of the PodVolumeRestore.\n enum:\n - New\n - Accepted\n - Prepared\n - InProgress\n - Canceling\n - Canceled\n - Completed\n - Failed\n type: string\n progress:\n description: |-\n Progress holds the total number of bytes of the snapshot and the current\n number of restored bytes. This can be used to display progress information\n about the restore operation.\n properties:\n bytesDone:\n format: int64\n type: integer\n totalBytes:\n format: int64\n type: integer\n type: object\n startTimestamp:\n description: |-\n StartTimestamp records the time a restore was started.\n The server's time is used for StartTimestamps\n format: date-time\n nullable: true\n type: string\n type: object\n type: object\n served: true\n storage: true\n subresources: {}\n```\n\n## Installation Changes\nNo changes to installation, the backupPod/restorePod's configurations are either inherited from node-agent or retrieved from node-agent-configmap. \n\n## Upgrade\nUpgrade is not impacted. \n\n## CLI\nCLI is not changed.\n\n\n\n[1]: ../unified-repo-and-kopia-integration/unified-repo-and-kopia-integration.md\n[2]: ../volume-snapshot-data-movement/volume-snapshot-data-movement.md\n[3]: ../vgdp-micro-service/vgdp-micro-service.md\n[4]: ../node-agent-concurrency.md\n[5]: https://github.com/vmware-tanzu/velero/issues/8955\n[6]: https://github.com/vmware-tanzu/velero/issues/8956\n[7]: https://github.com/vmware-tanzu/velero/issues/8289\n[8]: https://kubernetes.io/docs/tasks/configure-pod-container/create-hostprocess-pod/\n[9]: https://github.com/vmware-tanzu/velero/issues/7725\n[10]: https://github.com/vmware-tanzu/velero/issues/8423" }, { "path": "design/Implemented/volume-group-snapshot.md", "content": "# Add Support for VolumeGroupSnapshots\n\nThis proposal outlines the design and implementation plan for incorporating VolumeGroupSnapshot support into Velero. The enhancement will allow Velero to perform consistent, atomic snapshots of groups of Volumes using the new Kubernetes [VolumeGroupSnapshot API](https://kubernetes.io/blog/2024/12/18/kubernetes-1-32-volume-group-snapshot-beta/). This capability is especially critical for stateful applications that rely on multiple volumes to ensure data consistency, such as databases and analytics workloads.\n\n## Glossary & Abbreviation\n\nTerminology used in this document:\n- VGS: VolumeGroupSnapshot\n- VS: VolumeSnapshot\n- VGSC: VolumeGroupSnapshotContent\n- VSC: VolumeSnapshotContent\n- VGSClass: VolumeGroupSnapshotClass\n- VSClass: VolumeSnapshotClass\n\n## Background\n\nVelero currently enables snapshot-based backups on an individual Volume basis through CSI drivers. However, modern stateful applications often require multiple volumes for data, logs, and backups. This distributed data architecture increases the risk of inconsistencies when volumes are captured individually. Kubernetes has introduced the VolumeGroupSnapshot(VGS) API [(KEP-3476)](https://github.com/kubernetes/enhancements/pull/1551), which allows for the atomic snapshotting of multiple volumes in a coordinated manner. By integrating this feature, Velero can offer enhanced disaster recovery for multi-volume applications, ensuring consistency across all related data.\n\n## Goals\n- Ensure that multiple related volumes are snapshotted simultaneously, preserving consistency for stateful applications via VolumeGroupSnapshots(VGS) API.\n- Integrate VolumeGroupSnapshot functionality into Velero’s existing backup and restore workflows.\n- Allow users to opt in to volume group snapshots via specifying the group label.\n\n## Non-Goals\n- The proposal does not require a complete overhaul of Velero’s CSI integration, it will extend the current mechanism to support group snapshots.\n- No any changes pertaining to execution of Restore Hooks\n\n## High-Level Design\n\n### Backup workflow:\n#### Accept the label to be used for VGS from the user:\n - Accept the label from the user, we will do this in 3 ways:\n - Firstly, we will have a hard-coded default label key like `velero.io/volume-group-snapshot` that the users can directly use on their PVCs.\n - Secondly, we will let the users override this default VGS label via a velero server arg, `--volume-group-nsaphot-label-key`, if needed.\n - And Finally we will have the option to override the default label via Backup API spec, `backup.spec.volumeGroupSnapshotLabelKey`\n - In all the instances, the VGS label key will be present on the backup spec, this makes the label key accessible to plugins during the execution of backup operation.\n - This label will enable velero to filter the PVC to be included in the VGS spec.\n - Users will have to label the PVCs before invoking the backup operation.\n - This label would act as a group identifier for the PVCs to be grouped under a specific VGS.\n - It will be used to collect the PVCs to be used for a particular instance of VGS object. \n\n**Note:** \n - Modifying or adding VGS label on PVCs during an active backup operation may lead to unexpected or undesirable backup results. To avoid inconsistencies, ensure PVC labels remain unchanged throughout the backup execution.\n - Label Key Precedence: When determining which label key to use for grouping PVCs into a VolumeGroupSnapshot, Velero applies overrides in the following order (highest to lowest):\n - Backup API spec (`backup.spec.volumeGroupSnapshotLabelKey`)\n - Server flag (`--volume-group-snapshot-label-key`)\n - Built-in default (`velero.io/volume-group-snapshot`)\n\n Whichever key wins this precedence is then injected into the Backup spec so that all Velero plugins can uniformly discover and use it during the backup execution.\n#### Changes to the Existing PVC ItemBlockAction plugin:\n - Currently the PVC IBA plugin is applied to PVCs and adds the RelatedItems for the particular PVC into the ItemBlock.\n - At first it checks whether the PVC is bound and VolumeName is non-empty.\n - Then it adds the related PV under the list of relatedItems.\n - Following on, the plugin adds the pods mounting the PVC as relatedItems.\n - Now we need to extend this PVC IBA plugin to add the PVCs to be grouped for a particular VGS object, so that they are processed together under an ItemBlock by Velero.\n - First we will check if the PVC that is being processed by the plugin has the user specified VGS label.\n - If it is present then we will execute a List call in the namespace with the label as a matching criteria and see if this results in any PVCs (other than the current one).\n - If there are PVCs matching the criteria then we add the PVCs to the relatedItems list.\n - This helps in building the ItemBlock we need for VGS processing, i.e. we have the relevant pods and PVCs in the ItemBlock.\n\n**Note:** The ItemBlock to VGS relationship will not always be 1:1. There might be scenarios when the ItemBlock might have multiple VGS instances associated with it.\nLets go over some ItemBlock/VGS scenarios that we might encounter and visualize them for clarity:\n1. Pod Mounts: Pod1 mounts both PVC1 and PVC2. \n Grouping: PVC1 and PVC2 share the same group label (group: A) \n ItemBlock: The item block includes Pod1, PVC1, and PVC2. \n VolumeGroupSnapshot (VGS): Because PVC1 and PVC2 are grouped together by their label, they trigger the creation of a single VGS (labeled with group: A). \n\n```mermaid\nflowchart TD\n subgraph ItemBlock\n P1[Pod1]\n PVC1[PVC1 group: A]\n PVC2[PVC2 group: A]\n end\n\n P1 -->|mounts| PVC1\n P1 -->|mounts| PVC2\n\n PVC1 --- PVC2\n\n PVC1 -- \"group: A\" --> VGS[VGS group: A]\n PVC2 -- \"group: A\" --> VGS\n\n``` \n2. Pod Mounts: Pod1 mounts each of the four PVCs. \n Grouping: \n Group A: PVC1 and PVC2 share the same grouping label (group: A). \n Group B: PVC3 and PVC4 share the grouping label (group: B) \n ItemBlock: All objects (Pod1, PVC1, PVC2, PVC3, and PVC4) are collected into a single item block. \n VolumeGroupSnapshots: \n PVC1 and PVC2 (group A) point to the same VGS (VGS (group: A)). \n PVC3 and PVC4 (group B) point to a different VGS (VGS (group: B)). \n```mermaid\nflowchart TD\n subgraph ItemBlock\n P1[Pod1]\n PVC1[PVC1 group: A]\n PVC2[PVC2 group: A]\n PVC3[PVC3 group: B]\n PVC4[PVC4 group: B]\n end\n\n %% Pod mounts all PVCs\n P1 -->|mounts| PVC1\n P1 -->|mounts| PVC2\n P1 -->|mounts| PVC3\n P1 -->|mounts| PVC4\n\n %% Group A relationships: PVC1 and PVC2\n PVC1 --- PVC2\n PVC1 -- \"group: A\" --> VGS_A[VGS-A group: A]\n PVC2 -- \"group: A\" --> VGS_A\n\n %% Group B relationships: PVC3 and PVC4\n PVC3 --- PVC4\n PVC3 -- \"group: B\" --> VGS_B[VGS-B group: B]\n PVC4 -- \"group: B\" --> VGS_B\n```\n\n3. Pod Mounts: Pod1 mounts both PVC1 and PVC2, Pod2 mounts PVC1 and PVC3. \n Grouping: \n Group A: PVC1 and PVC2 \n Group B: PVC3 \n ItemBlock: All objects-Pod1, Pod2, PVC1, PVC2, and PVC3, are collected into a single item block. \n VolumeGroupSnapshots: \n PVC1 and PVC2 (group A) point to the same VGS (VGS (group: A)). \n PVC3 (group B) point to a different VGS (VGS (group: B)). \n```mermaid\nflowchart TD\n subgraph ItemBlock\n P1[Pod1]\n P2[Pod2]\n PVC1[PVC1 group: A]\n PVC2[PVC2 group: A]\n PVC3[PVC3 group: B]\n end\n\n %% Pod mount relationships\n P1 -->|mounts| PVC1\n P1 -->|mounts| PVC2\n P2 -->|mounts| PVC1\n P2 -->|mounts| PVC3\n\n %% Grouping for Group A: PVC1 and PVC2 are grouped into VGS_A\n PVC1 --- PVC2\n PVC1 -- \"Group A\" --> VGS_A[VGS Group A]\n PVC2 -- \"Group A\" --> VGS_A\n\n %% Grouping for Group B: PVC3 grouped into VGS_B\n PVC3 -- \"Group B\" --> VGS_B[VGS Group B]\n \n```\n\n#### Updates to CSI PVC plugin:\nThe CSI PVC plugin now supports obtaining a VolumeSnapshot (VS) reference for a PVC in three ways, and then applies common branching for datamover and non‑datamover workflows:\n\n- Scenario 1: PVC has a VGS label and no VS (created via the VGS workflow) exists for its volume group:\n - Determine VGSClass: The plugin will pick `VolumeGroupSnapshotClass` by following the same tier based precedence as it does for individual `VolumeSnapshotClasses`:\n - Default by Label: Use the one VGSClass labeled\n ```yaml\n metadata:\n labels:\n velero.io/csi-volumegroupsnapshot-class: \"true\"\n\n ```\n whose `spec.driver` matches the CSI driver used by the PVCs.\n - Backup‑level Override: If the Backup CR has an annotation\n ```yaml\n metadata:\n annotations:\n velero.io/csi-volumegroupsnapshot-class_: \n ```\n (with equal to the PVCs’ CSI driver), use that class.\n - PVC‑level Override: Finally, if the PVC itself carries an annotation\n ```yaml\n metadata:\n annotations:\n velero.io/csi-volume-group-snapshot-class: \n ```\n and that class exists, use it.\n At each step, if the plugin finds zero or multiple matching classes, VGS creation is skipped and backup fails.\n - Create VGS: The plugin creates a new VolumeGroupSnapshot (VGS) for the PVC’s volume group. This action automatically triggers creation of the corresponding VGSC, VS, and VSC objects.\n - Wait for VS Status: The plugin waits until each VS (one per PVC in the group) has its `volumeGroupSnapshotName` populated. This confirms that the snapshot controller has completed its work. `CSISnapshotTimeout` will be used here.\n - Update VS Objects: Once the VS objects are provisioned, the plugin updates them by removing VGS owner references and VGS-related finalizers, and by adding backup metadata labels (including BackupName, BackupUUID, and PVC name). These labels are later used to detect an existing VS when processing another PVC of the same group.\n - Patch and Cleanup: The plugin patches the deletionPolicy of the VGSC to \"Retain\" (ensuring that deletion of the VGSC does not remove the underlying VSC objects or storage snapshots) and then deletes the temporary VGS and VGSC objects.\n \n- Scenario 2: PVC has a VGS label and a VS created via an earlier VGS workflow already exists:\n - The plugin lists VS objects in the PVC’s namespace using backup metadata labels (BackupUID, BackupName, and PVCName).\n - It verifies that at least one VS has a non‑empty `volumeGroupSnapshotName` in its status.\n - If such a VS exists, the plugin skips creating a new VGS (or VS) and proceeds with the legacy workflow using the existing VS.\n - If a VS is found but its status does not indicate it was created by the VGS workflow (i.e. its `volumeGroupSnapshotName` is empty), the backup for that PVC is failed, resulting in a partially failed backup.\n- Scenario 3: PVC does not have a VGS label:\n - The legacy workflow is followed, and an individual VolumeSnapshot (VS) is created for the PVC.\n- Common Branching for Datamover and Non‑datamover Workflows:\n - Once a VS reference (`vsRef`) is determined—whether through the VGS workflow (Scenario 1 or 2) or the legacy workflow (Scenario 3)—the plugin then applies the common branching:\n - Non‑datamover Case: The VS reference is directly added as an additional backup item.\n \n - Datamover Case: The plugin waits until the VS’s associated VSC snapshot handle is ready (using the configured CSISnapshotTimeout), then creates a DataUpload for the VS–PVC pair. The resulting DataUpload is then added as an additional backup item.\n\n\n```mermaid\nflowchart TD\n %% Section 1: Accept VGS Label from User\n subgraph Accept_Label\n A1[User sets VGS label key using default velero.io/volume-group-snapshot or via server arg or Backup API spec]\n A2[User labels PVCs before backup]\n A1 --> A2\n end\n\n %% Section 2: PVC ItemBlockAction Plugin Extension\n subgraph PVC_ItemBlockAction\n B1[Check PVC is bound and has VolumeName]\n B2[Add related PV to relatedItems]\n B3[Add pods mounting PVC to relatedItems]\n B4[Check if PVC has user-specified VGS label]\n B5[List PVCs in namespace matching label criteria]\n B6[Add matching PVCs to relatedItems]\n B1 --> B2 --> B3 --> B4\n B4 -- Yes --> B5\n B5 --> B6\n end\n\n %% Section 3: CSI PVC Plugin Updates\n subgraph CSI_PVC_Plugin\n C1[For each PVC, check for VGS label]\n C1 -- Has VGS label --> C2[Determine scenario]\n C1 -- No VGS label --> C16[Scenario 3: Legacy workflow - create individual VS]\n\n %% Scenario 1: No existing VS via VGS exists\n subgraph Scenario1[Scenario 1: No existing VS via VGS]\n S1[List grouped PVCs using VGS label]\n S2[Determine CSI driver for grouped PVCs]\n S3[If single CSI driver then select matching VGSClass; else fail backup]\n S4[Create new VGS triggering VGSC, VS, and VSC creation]\n S5[Wait for VS objects to have nonempty volumeGroupSnapshotName]\n S6[Update VS objects; remove VGS owner refs and finalizers; add backup metadata labels]\n S7[Patch VGSC deletionPolicy to Retain]\n S8[Delete transient VGS and VGSC]\n S1 --> S2 --> S3 --> S4 --> S5 --> S6 --> S7 --> S8\n\t\t \n end\n\n %% Scenario 2: Existing VS via VGS exists\n subgraph Scenario2[Scenario 2: Existing VS via VGS exists]\n S9[List VS objects using backup metadata - BackupUID, BackupName, PVCName]\n S10[Check if any VS has nonempty volumeGroupSnapshotName]\n S9 --> S10\n S10 -- Yes --> S11[Use existing VS]\n S10 -- No --> S12[Fail backup for PVC]\n end\n\n C2 -- Scenario1 applies --> S1\n C2 -- Scenario2 applies --> S9\n\n %% Common Branch: After obtaining a VS reference\n subgraph Common_Branch[Common Branch]\n CB1[Obtain VS reference as vsRef]\n CB2[If non-datamover, add vsRef as additional backup item]\n CB3[If datamover, wait for VSC handle and create DataUpload; add DataUpload as additional backup item]\n CB1 --> CB2\n CB1 --> CB3\n end\n \n %% Connect Scenario outcomes and legacy branch to the common branch\n S8 --> CB1\n S11 --> CB1\n C16 --> CB1\n end\n\n %% Overall Flow Connections\n A2 --> B1\n B6 --> C1\n\n```\n\n\nRestore workflow:\n\n- No changes required for the restore workflow.\n\n## Detailed Design\n\nBackup workflow:\n- Accept the label to be used for VGS from the user as a server argument:\n - Set a default VGS label key to be used:\n ```go\n // default VolumeGroupSnapshot Label\n\tdefaultVGSLabelKey = \"velero.io/volume-group-snapshot\"\n \n ```\n - Add this as a server flag and pass it to backup reconciler, so that we can use it during the backup request execution.\n ```go\n flags.StringVar(&c.DefaultVGSLabelKey, \"volume-group-snapshot-label-key\", c.DefaultVGSLabelKey, \"Label key for grouping PVCs into VolumeGroupSnapshot\")\n ```\n\n - Update the Backup CRD to accept the VGS Label Key as a spec value:\n ```go\n // VolumeGroupSnapshotLabelKey specifies the label key to be used for grouping the PVCs under\n\t// an instance of VolumeGroupSnapshot, if left unspecified velero.io/volume-group-snapshot is used\n\t// +optional\n\tVolumeGroupSnapshotLabelKey string `json:\"volumeGroupSnapshotLabelKey,omitempty\"`\n ```\n - Modify the [`prepareBackupRequest` function](https://github.com/openshift/velero/blob/8c8a6cccd78b78bd797e40189b0b9bee46a97f9e/pkg/controller/backup_controller.go#L327) to set the default label key as a backup spec if the user does not specify any value:\n ```go\n if len(request.Spec.VolumeGroupSnapshotLabelKey) == 0 {\n\t\t// set the default key value\n\t\trequest.Spec.VolumeGroupSnapshotLabelKey = b.defaultVGSLabelKey\n\t}\n ```\n\n- Changes to the Existing [PVC ItemBlockAction plugin](https://github.com/vmware-tanzu/velero/blob/512199723ff95d5016b32e91e3bf06b65f57d608/pkg/itemblock/actions/pvc_action.go#L64) (Update the GetRelatedItems function):\n```go\n// Retrieve the VGS label key from the Backup spec.\n\tvgsLabelKey := backup.Spec.VolumeGroupSnapshotLabelKey\n\tif vgsLabelKey != \"\" {\n\t\t// Check if the PVC has the specified VGS label.\n\t\tif groupID, ok := pvc.Labels[vgsLabelKey]; ok {\n\t\t\t// List all PVCs in the namespace with the same label key and value (i.e. same group).\n\t\t\tpvcList := new(corev1api.PersistentVolumeClaimList)\n\t\t\tif err := a.crClient.List(context.Background(), pvcList, crclient.InNamespace(pvc.Namespace), crclient.MatchingLabels{vgsLabelKey: groupID}); err != nil {\n\t\t\t\treturn nil, errors.Wrap(err, \"failed to list PVCs for VGS grouping\")\n\t\t\t}\n\t\t\t// Add each matching PVC (except the current one) to the relatedItems.\n\t\t\tfor _, groupPVC := range pvcList.Items {\n\t\t\t\tif groupPVC.Name == pvc.Name {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\ta.log.Infof(\"Adding grouped PVC %s to relatedItems for PVC %s\", groupPVC.Name, pvc.Name)\n\t\t\t\trelatedItems = append(relatedItems, velero.ResourceIdentifier{\n\t\t\t\t\tGroupResource: kuberesource.PersistentVolumeClaims,\n\t\t\t\t\tNamespace: groupPVC.Namespace,\n\t\t\t\t\tName: groupPVC.Name,\n\t\t\t\t})\n\t\t\t}\n\t\t}\n\t} else {\n\t\ta.log.Info(\"No VolumeGroupSnapshotLabelKey provided in backup spec; skipping PVC grouping\")\n\t}\n```\n\n- Updates to [CSI PVC plugin](https://github.com/vmware-tanzu/velero/blob/512199723ff95d5016b32e91e3bf06b65f57d608/pkg/backup/actions/csi/pvc_action.go#L200) (Update the Execute method):\n```go\nfunc (p *pvcBackupItemAction) Execute(\n item runtime.Unstructured,\n backup *velerov1api.Backup,\n) (\n runtime.Unstructured,\n []velero.ResourceIdentifier,\n string,\n []velero.ResourceIdentifier,\n error,\n) {\n p.log.Info(\"Starting PVCBackupItemAction\")\n\n // Validate backup policy and PVC/PV\n if valid := p.validateBackup(*backup); !valid {\n return item, nil, \"\", nil, nil\n }\n\n var pvc corev1api.PersistentVolumeClaim\n if err := runtime.DefaultUnstructuredConverter.FromUnstructured(item.UnstructuredContent(), &pvc); err != nil {\n return nil, nil, \"\", nil, errors.WithStack(err)\n }\n if valid, item, err := p.validatePVCandPV(pvc, item); !valid {\n if err != nil {\n return nil, nil, \"\", nil, err\n }\n return item, nil, \"\", nil, nil\n }\n\n shouldSnapshot, err := volumehelper.ShouldPerformSnapshotWithBackup(\n item,\n kuberesource.PersistentVolumeClaims,\n *backup,\n p.crClient,\n p.log,\n )\n if err != nil {\n return nil, nil, \"\", nil, err\n }\n if !shouldSnapshot {\n p.log.Debugf(\"CSI plugin skip snapshot for PVC %s according to VolumeHelper setting\", pvc.Namespace+\"/\"+pvc.Name)\n return nil, nil, \"\", nil, nil\n }\n\n var additionalItems []velero.ResourceIdentifier\n var operationID string\n var itemToUpdate []velero.ResourceIdentifier\n\n // vsRef will be our common reference to the VolumeSnapshot (VS)\n var vsRef *corev1api.ObjectReference\n\n // Retrieve the VGS label key from the backup spec.\n vgsLabelKey := backup.Spec.VolumeGroupSnapshotLabelKey\n\n // Check if the PVC has the user-specified VGS label.\n if group, ok := pvc.Labels[vgsLabelKey]; ok && group != \"\" {\n p.log.Infof(\"PVC %s has VGS label with group %s\", pvc.Name, group)\n // --- VGS branch ---\n // 1. Check if a VS created via a VGS workflow exists for this PVC.\n existingVS, err := p.findExistingVSForBackup(backup.UID, backup.Name, pvc.Name, pvc.Namespace)\n if err != nil {\n return nil, nil, \"\", nil, err\n }\n if existingVS != nil && existingVS.Status.VolumeGroupSnapshotName != \"\" {\n p.log.Infof(\"Existing VS %s found for PVC %s in group %s; skipping VGS creation\", existingVS.Name, pvc.Name, group)\n vsRef = &corev1api.ObjectReference{\n Namespace: existingVS.Namespace,\n Name: existingVS.Name,\n }\n } else {\n // 2. No existing VS via VGS; execute VGS creation workflow.\n groupedPVCs, err := p.listGroupedPVCs(backup, pvc.Namespace, vgsLabelKey, group)\n if err != nil {\n return nil, nil, \"\", nil, err\n }\n pvcNames := extractPVCNames(groupedPVCs)\n // Determine the CSI driver used by the grouped PVCs.\n driver, err := p.determineCSIDriver(groupedPVCs)\n if err != nil {\n return nil, nil, \"\", nil, errors.Wrap(err, \"failed to determine CSI driver for grouped PVCs\")\n }\n if driver == \"\" {\n return nil, nil, \"\", nil, errors.New(\"multiple CSI drivers found for grouped PVCs; failing backup\")\n }\n // Retrieve the appropriate VGSClass for the CSI driver.\n vgsClass := p.getVGSClassForDriver(driver)\n p.log.Infof(\"Determined CSI driver %s with VGSClass %s for PVC group %s\", driver, vgsClass, group)\n\n newVGS, err := p.createVolumeGroupSnapshot(backup, pvc, pvcNames, vgsLabelKey, group, vgsClass)\n if err != nil {\n return nil, nil, \"\", nil, err\n }\n p.log.Infof(\"Created new VGS %s for PVC group %s\", newVGS.Name, group)\n \n // Wait for the VS objects created via VGS to have volumeGroupSnapshotName in status.\n if err := p.waitForVGSAssociatedVS(newVGS, pvc.Namespace, backup.Spec.CSISnapshotTimeout.Duration); err != nil {\n return nil, nil, \"\", nil, err\n }\n // Update the VS objects: remove VGS owner references and finalizers; add backup metadata labels.\n if err := p.updateVGSCreatedVS(newVGS, backup); err != nil {\n return nil, nil, \"\", nil, err\n }\n // Patch the VGSC deletionPolicy to Retain.\n if err := p.patchVGSCDeletionPolicy(newVGS, pvc.Namespace); err != nil {\n return nil, nil, \"\", nil, err\n }\n // Delete the VGS and VGSC\n if err := p.deleteVGSAndVGSC(newVGS, pvc.Namespace); err != nil {\n return nil, nil, \"\", nil, err\n }\n // Fetch the VS that was created for this PVC via VGS.\n vs, err := p.getVSForPVC(backup, pvc, vgsLabelKey, group)\n if err != nil {\n return nil, nil, \"\", nil, err\n }\n vsRef = &corev1api.ObjectReference{\n Namespace: vs.Namespace,\n Name: vs.Name,\n }\n }\n } else {\n // Legacy workflow: PVC does not have a VGS label; create an individual VS.\n vs, err := p.createVolumeSnapshot(pvc, backup)\n if err != nil {\n return nil, nil, \"\", nil, err\n }\n vsRef = &corev1api.ObjectReference{\n Namespace: vs.Namespace,\n Name: vs.Name,\n }\n }\n\n // --- Common Branch ---\n // Now we have vsRef populated from one of the above cases.\n // Branch further based on backup.Spec.SnapshotMoveData.\n if boolptr.IsSetToTrue(backup.Spec.SnapshotMoveData) {\n // Datamover case:\n operationID = label.GetValidName(\n string(velerov1api.AsyncOperationIDPrefixDataUpload) + string(backup.UID) + \".\" + string(pvc.UID),\n )\n dataUploadLog := p.log.WithFields(logrus.Fields{\n \"Source PVC\": fmt.Sprintf(\"%s/%s\", pvc.Namespace, pvc.Name),\n \"VolumeSnapshot\": fmt.Sprintf(\"%s/%s\", vsRef.Namespace, vsRef.Name),\n \"Operation ID\": operationID,\n \"Backup\": backup.Name,\n })\n // Retrieve the current VS using vsRef\n vs := &snapshotv1api.VolumeSnapshot{}\n if err := p.crClient.Get(context.TODO(), crclient.ObjectKey{Namespace: vsRef.Namespace, Name: vsRef.Name}, vs); err != nil {\n return nil, nil, \"\", nil, errors.Wrapf(err, \"failed to get VolumeSnapshot %s\", vsRef.Name)\n }\n // Wait until the VS-associated VSC snapshot handle is ready.\n _, err := csi.WaitUntilVSCHandleIsReady(\n vs,\n p.crClient,\n p.log,\n true,\n backup.Spec.CSISnapshotTimeout.Duration,\n )\n if err != nil {\n dataUploadLog.Errorf(\"Failed to wait for VolumeSnapshot to become ReadyToUse: %s\", err.Error())\n csi.CleanupVolumeSnapshot(vs, p.crClient, p.log)\n return nil, nil, \"\", nil, errors.WithStack(err)\n }\n dataUploadLog.Info(\"Starting data upload of backup\")\n dataUpload, err := createDataUpload(\n context.Background(),\n backup,\n p.crClient,\n vs,\n &pvc,\n operationID,\n )\n if err != nil {\n dataUploadLog.WithError(err).Error(\"Failed to submit DataUpload\")\n if deleteErr := p.crClient.Delete(context.TODO(), vs); deleteErr != nil && !apierrors.IsNotFound(deleteErr) {\n dataUploadLog.WithError(deleteErr).Error(\"Failed to delete VolumeSnapshot\")\n }\n return item, nil, \"\", nil, nil\n }\n dataUploadLog.Info(\"DataUpload submitted successfully\")\n itemToUpdate = []velero.ResourceIdentifier{\n {\n GroupResource: schema.GroupResource{\n Group: \"velero.io\",\n Resource: \"datauploads\",\n },\n Namespace: dataUpload.Namespace,\n Name: dataUpload.Name,\n },\n }\n annotations[velerov1api.DataUploadNameAnnotation] = dataUpload.Namespace + \"/\" + dataUpload.Name\n // For the datamover case, add the dataUpload as an additional item directly.\n vsRef = &corev1api.ObjectReference{\n Namespace: dataUpload.Namespace,\n Name: dataUpload.Name,\n }\n additionalItems = append(additionalItems, velero.ResourceIdentifier{\n GroupResource: schema.GroupResource{\n Group: \"velero.io\",\n Resource: \"datauploads\",\n },\n Namespace: dataUpload.Namespace,\n Name: dataUpload.Name,\n })\n } else {\n // Non-datamover case:\n // Use vsRef for snapshot purposes.\n additionalItems = append(additionalItems, convertVSToResourceIdentifiersFromRef(vsRef)...)\n p.log.Infof(\"VolumeSnapshot additional item added for VS %s\", vsRef.Name)\n }\n\n // Update PVC metadata with common labels and annotations.\n labels := map[string]string{\n velerov1api.VolumeSnapshotLabel: vsRef.Name,\n velerov1api.BackupNameLabel: backup.Name,\n }\n annotations := map[string]string{\n velerov1api.VolumeSnapshotLabel: vsRef.Name,\n velerov1api.MustIncludeAdditionalItemAnnotation: \"true\",\n }\n kubeutil.AddAnnotations(&pvc.ObjectMeta, annotations)\n kubeutil.AddLabels(&pvc.ObjectMeta, labels)\n\n p.log.Infof(\"Returning from PVCBackupItemAction with %d additionalItems to backup\", len(additionalItems))\n for _, ai := range additionalItems {\n p.log.Debugf(\"%s: %s\", ai.GroupResource.String(), ai.Name)\n }\n\n pvcMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&pvc)\n if err != nil {\n return nil, nil, \"\", nil, errors.WithStack(err)\n }\n\n return &unstructured.Unstructured{Object: pvcMap},\n additionalItems, operationID, itemToUpdate, nil\n}\n\n\n```\n\n## Implementation\n\nThis design proposal is targeted for velero 1.16.\n\nThe implementation of this proposed design is targeted for velero 1.17.\n\n**Note:**\n- VGS support isn't a requirement on restore. The design does not have any VGS related elements/considerations in the restore workflow.\n\n## Requirements and Assumptions\n- Kubernetes Version:\n - Minimum: v1.32.0 or later, since the VolumeGroupSnapshot API goes beta in 1.32.\n - Assumption: CRDs for `VolumeGroupSnapshot`, `VolumeGroupSnapshotClass`, and `VolumeGroupSnapshotContent` are already installed.\n\n- VolumeGroupSnapshot API Availability:\n - If the VGS API group (`groupsnapshot.storage.k8s.io/v1beta1`) is not present, Velero backup will fail.\n\n- CSI Driver Compatibility\n - Only CSI drivers that implement the VolumeGroupSnapshot admission and controller support this feature.\n - Upon VGS creation, we assume the driver will atomically snapshot all matching PVCs; if it does not, the plugin may time out.\n\n## Performance Considerations\n- Use VGS if you have many similar volumes that must be snapped together and you want to minimize API/server load.\n- Use individual VS if you have only a few volumes, or want one‐volume failures to be isolated.\n\n## Testing Strategy\n\n- Unit tests: We will add targeted unit tests to cover all new code paths—including existing-VS detection, VGS creation, legacy VS fallback, and error scenarios.\n- E2E tests: For E2E we would need, a Kind cluster with a CSI driver that supports group snapshots, deploy an application with multiple PVCs, execute a Velero backup and restore, and verify that VGS is created, all underlying VS objects reach ReadyToUse, and every PVC is restored successfully." }, { "path": "design/Implemented/volume-policy-label-selector-criteria.md", "content": "# Add Label Selector as a criteria for Volume Policy\n\n## Abstract\nVelero’s volume policies currently support several criteria (such as capacity, storage class, and volume source type) to select volumes for backup. This update extends the design by allowing users to specify required labels on the associated PersistentVolumeClaim (PVC) via a simple key/value map. At runtime, Velero looks up the PVC (when a PV has a ClaimRef), extracts its labels, and compares them with the user-specified map. If all key/value pairs match, the volume qualifies for backup.\n\n## Background \nPersistentVolumes (PVs) in Kubernetes are typically bound to PersistentVolumeClaims (PVCs) that include labels (for example, indicating environment, application, or region). Basing backup policies on these PVC labels enables more precise control over which volumes are processed.\n\n## Goals\n- Allow users to specify a simple key/value mapping in the volume policy YAML so that only volumes whose associated PVCs contain those labels are selected.\n- Support policies that target volumes based on criteria such as environment=production or region=us-west.\n\n## Non-Goals\n- No changes will be made to the actions (skip, snapshot, fs-backup) of the volume policy engine. This update focuses solely on how volumes are selected.\n- The design does not support other label selector operations (e.g., NotIn, Exists, DoesNotExist) and only allows for exact key/value matching.\n\n## Use-cases/scenarios\n1. Environment-Specific Backup:\n- A user wishes to back up only those volumes whose associated PVCs have labels such as `environment=production` and `app=database`.\n- The volume policy specifies a pvcLabels map with those key/value pairs; only volumes whose PVCs match are processed.\n```yaml\nvolumePolicies:\n - conditions:\n pvcLabels:\n environment: production\n app: database\n action:\n type: snapshot\n```\n2. Region-Specific Backup:\n- A user operating in multiple regions wants to back up only volumes in the `us-west` region.\n- The policy includes `pvcLabels: { region: us-west }`, so only PVs bound to PVCs with that label are selected.\n```yaml\nvolumePolicies:\n - conditions:\n pvcLabels:\n region: us-west\n action:\n type: snapshot\n```\n3. Automated Label-Based Backups:\n- An external system automatically labels new PVCs (for example, `backup: true`).\n- A volume policy with `pvcLabels: { backup: true }` ensures that any new volume whose PVC contains that label is included in backup operations.\n```yaml\nversion: v1\nvolumePolicies:\n - conditions:\n pvcLabels:\n backup: true\n action:\n type: snapshot\n```\n## High-Level Design\n\n1. Extend Volume Policy Schema:\n- The YAML schema for volume conditions is extended to include an optional field pvcLabels of type `map[string]string`.\n2. Implement New Condition Type:\n- A new condition, `pvcLabelsCondition`, is created. It implements the `volumeCondition` interface and simply compares the user-specified key/value pairs with the actual PVC labels (populated at runtime).\n3. Update Structured Volume:\n- The internal representation of a volume (`structuredVolume`) is extended with a new field `pvcLabels map[string]string` to store the labels from the associated PVC.\n- A new helper function (or an updated parsing function) is used to perform a PVC lookup when a PV has a ClaimRef, populating the pvcLabels field.\n4. Integrate with Policy Engine:\n- The policy builder is updated to create and add a `pvcLabelsCondition` if the policy YAML contains a `pvcLabels` entry.\n- The matching entry point uses the updated `structuredVolume` (populated with PVC labels) to evaluate all conditions, including the new PVC labels condition.\n## Detailed Design\n\n1. Update Volume Conditions Schema: Define the conditions struct with a simple map for PVC labels:\n```go\n// volumeConditions defines the current format of conditions we parse.\ntype volumeConditions struct {\n Capacity string `yaml:\"capacity,omitempty\"`\n StorageClass []string `yaml:\"storageClass,omitempty\"`\n NFS *nFSVolumeSource `yaml:\"nfs,omitempty\"`\n CSI *csiVolumeSource `yaml:\"csi,omitempty\"`\n VolumeTypes []SupportedVolume `yaml:\"volumeTypes,omitempty\"`\n // New field: pvcLabels for simple exact-match filtering.\n PVCLabels map[string]string `yaml:\"pvcLabels,omitempty\"`\n}\n```\n2. New Condition: `pvcLabelsCondition`: Implement a condition that compares expected labels with those on the PVC:\n```go\n// pvcLabelsCondition defines a condition that matches if the PVC's labels contain all the specified key/value pairs.\ntype pvcLabelsCondition struct {\n labels map[string]string\n}\n\nfunc (c *pvcLabelsCondition) match(v *structuredVolume) bool {\n if len(c.labels) == 0 {\n return true // No label condition specified; always match.\n }\n if v.pvcLabels == nil {\n return false // No PVC labels found.\n }\n for key, expectedVal := range c.labels {\n if actualVal, exists := v.pvcLabels[key]; !exists || actualVal != expectedVal {\n return false\n }\n }\n return true\n}\n\nfunc (c *pvcLabelsCondition) validate() error {\n // No extra validation needed for a simple map.\n return nil\n}\n```\n3. Update `structuredVolume`: Extend the internal volume representation with a field for PVC labels:\n```go\n// structuredVolume represents a volume with parsed fields.\ntype structuredVolume struct {\n capacity resource.Quantity\n storageClass string\n // New field: pvcLabels stores labels from the associated PVC.\n pvcLabels map[string]string\n nfs *nFSVolumeSource\n csi *csiVolumeSource\n volumeType SupportedVolume\n}\n```\n4. Update PVC Lookup – `parsePVWithPVC`: Modify the PV parsing function to perform a PVC lookup:\n```go\nfunc (s *structuredVolume) parsePVWithPVC(pv *corev1.PersistentVolume, client crclient.Client) error {\n s.capacity = *pv.Spec.Capacity.Storage()\n s.storageClass = pv.Spec.StorageClassName\n\n if pv.Spec.NFS != nil {\n s.nfs = &nFSVolumeSource{\n Server: pv.Spec.NFS.Server,\n Path: pv.Spec.NFS.Path,\n }\n }\n if pv.Spec.CSI != nil {\n s.csi = &csiVolumeSource{\n Driver: pv.Spec.CSI.Driver,\n VolumeAttributes: pv.Spec.CSI.VolumeAttributes,\n }\n }\n s.volumeType = getVolumeTypeFromPV(pv)\n\n // If the PV is bound to a PVC, look it up and store its labels.\n if pv.Spec.ClaimRef != nil {\n pvc := &corev1.PersistentVolumeClaim{}\n err := client.Get(context.Background(), crclient.ObjectKey{\n Namespace: pv.Spec.ClaimRef.Namespace,\n Name: pv.Spec.ClaimRef.Name,\n }, pvc)\n if err != nil {\n return errors.Wrap(err, \"failed to get PVC for PV\")\n }\n s.pvcLabels = pvc.Labels\n }\n return nil\n}\n```\n5. Update the Policy Builder: Add the new condition to the policy if pvcLabels is provided:\n```go\nfunc (p *Policies) BuildPolicy(resPolicies *ResourcePolicies) error {\n for _, vp := range resPolicies.VolumePolicies {\n con, err := unmarshalVolConditions(vp.Conditions)\n if err != nil {\n return errors.WithStack(err)\n }\n volCap, err := parseCapacity(con.Capacity)\n if err != nil {\n return errors.WithStack(err)\n }\n var volP volPolicy\n volP.action = vp.Action\n volP.conditions = append(volP.conditions, &capacityCondition{capacity: *volCap})\n volP.conditions = append(volP.conditions, &storageClassCondition{storageClass: con.StorageClass})\n volP.conditions = append(volP.conditions, &nfsCondition{nfs: con.NFS})\n volP.conditions = append(volP.conditions, &csiCondition{csi: con.CSI})\n volP.conditions = append(volP.conditions, &volumeTypeCondition{volumeTypes: con.VolumeTypes})\n // If a pvcLabels map is provided, add the pvcLabelsCondition.\n if con.PVCLabels != nil && len(con.PVCLabels) > 0 {\n volP.conditions = append(volP.conditions, &pvcLabelsCondition{labels: con.PVCLabels})\n }\n p.volumePolicies = append(p.volumePolicies, volP)\n }\n p.version = resPolicies.Version\n return nil\n}\n```\n6. Update the Matching Entry Point: Use the updated PV parsing that performs a PVC lookup:\n```go\nfunc (p *Policies) GetMatchAction(res interface{}, client crclient.Client) (*Action, error) {\n volume := &structuredVolume{}\n switch obj := res.(type) {\n case *corev1.PersistentVolume:\n if err := volume.parsePVWithPVC(obj, client); err != nil {\n return nil, errors.Wrap(err, \"failed to parse PV with PVC lookup\")\n }\n case *corev1.Volume:\n volume.parsePodVolume(obj)\n default:\n return nil, errors.New(\"failed to convert object\")\n }\n return p.match(volume), nil\n}\n```\n\nNote: The matching loop (p.match(volume)) iterates over all conditions (including our new pvcLabelsCondition) and returns the corresponding action if all conditions match." }, { "path": "design/Implemented/volume-snapshot-data-movement/volume-snapshot-data-movement.md", "content": "# Volume Snapshot Data Movement Design\n\n## Glossary & Abbreviation\n\n**BR**: Backup & Restore \n**Backup Storage**: See the same definition in [Unified Repository design][1]. \n**Backup Repository**: See the same definition in [Unified Repository design][1]. \n**BIA/RIA V2**: Backup Item Action/Restore Item Action V2 that supports asynchronized operations, see the [general progress monitoring design][2] for details. \n\n## Background\n\nAs a Kubernetes BR solution, Velero is pursuing the capability to back up data from the volatile and limited production environment into the durable, heterogeneous and scalable backup storage. This relies on two parts: \n\n- Data Movement: Move data from various production workloads, including the snapshots of the workloads or volumes of the workloads\n- Data Persistency and Management: Persistent the data in backup storage and manage its security, redundancy, accessibility, etc. through backup repository. This has been covered by the [Unified Repository design][1]\n\nAt present, Velero supports moving file system data from PVs through Pod Volume Backup (a.k.a. file system backup). However, it backs up the data from the live file system, so it should be the last option when more consistent data movement (i.e., moving data from snapshot) is not available. \n\nMoreover, we would like to create a general workflow to variations during the data movement, e.g., data movement plugins, different snapshot types, different snapshot accesses and different data accesses. \n\n## Goals\n\n- Create components and workflows for Velero to move data based on volume snapshots\n- Create components and workflows for Velero built-in data mover\n- Create the mechanism to support data mover plugins from third parties\n- Implement CSI snapshot data movement on file system level\n- Support different data accesses, i.e., file system level and block level\n- Support different snapshot types, i.e., CSI snapshot, volume snapshot API from storage vendors\n- Support different snapshot accesses, i.e., through PV generated from snapshots, and through direct access API from storage vendors\n- Reuse the existing Velero generic data path as created in [Unified Repository design][1]\n\n## Non-Goals\n\n- The current support for block level access is through file system uploader, so it is not aimed to deliver features of an ultimate block level backup. Block level backup will be included in a future design\n- Most of the components are generic, but the Exposer is snapshot type specific or snapshot access specific. The current design covers the implementation details for exposing CSI snapshot to host path access only, for other types or accesses, we may need a separate design\n- The current workflow focuses on snapshot-based data movements. For some application/SaaS level data sources, snapshots may not be taken explicitly. We don’t take them into consideration, though we believe that some workflows or components may still be reusable.\n\n## Architecture of Volume Snapshot Data Movement\n\n### Workflows \n\nHere are the diagrams that illustrate components and workflows for backup and restore respectively. \nFor backup, we intend to create an extensive architecture for various snapshot types, snapshot accesses and various data accesses. For example, the snapshot specific operations are isolated in Data Mover Plugin and Exposer. In this way, we only need to change the two modules for variations. Likely, the data access details are isolated into uploaders, so different uploaders could be plugged into the workflow seamlessly. \n\nFor restore, we intend to create a generic workflow that could for all backups. This means the restore is backup source independent. Therefore, for example, we can restore a CSI snapshot backup to another cluster with no CSI facilities or with CSI facilities different from the source cluster. \nWe still have the Exposer module for restore and it is to expose the target volume to the data path. Therefore, we still have the flexibility to introduce different ways to expose the target volume. \nLikely, the data downloading details are isolated in uploaders, so we can still create multiple types of uploaders. \n\nBelow is the backup workflow: \n![backup-workflow.png](backup-workflow.png) \n\nBelow is the restore workflow: \n![restore-workflow.png](restore-workflow.png) \n\n### Components\nBelow are the generic components in the data movement workflow: \n\n**Velero**: Velero controls the backup/restore workflow, it calls BIA/RIA V2 to backup/restore an object that involves data movement, specifically, a PVC or a PV. \n**BIA/RIA V2**: BIA/RIA V2 are the protocols between Velero and the data mover plugins. They support asynchronized operations so that Velero backup/restore is not marked as completion until the data movement is done and in the meantime, Velero is free to process other backups during the data movement. \n**Data Mover Plugin (DMP)**: DMP implement BIA/RIA V2 and it invokes the corresponding data mover by creating the DataUpload/DataDownload CRs. DMP is also responsible to take snapshot of the source volume, so it is a snapshot type specific module. For CSI snapshot data movement, the CSI plugin could be extended as a DMP, this also means that the CSI plugin will fully implement BIA/RIA V2 and support some more methods like Progress, Cancel, etc. \n**DataUpload CR (DUCR)/ DataDownload CR (DDCR)**: DUCR/DDCR are Kubernetes CRs that act as the protocol between data mover plugins and data movers. The parties who want to provide a data mover need to watch and process these CRs. \n**Data Mover (DM)**: DM is a collective of modules to finish the data movement, specifically, data upload and data download. The modules may include the data mover controllers to reconcile DUCR/DDCR and the data path to transfer data. \n\nDMs take the responsibility to handle DUCR/DDCRs, Velero provides a built-in DM and meanwhile Velero supports plugin DMs. Below shows the components for the built-in DM: \n\n**Velero Built-in Data Mover (VBDM)**: VBDM is the built-in data mover shipped along with Velero, it includes Velero data mover controllers and Velero generic data path. \n**Node-Agent**: Node-Agent is an existing Velero module that will be used to host VBDM. \n**Exposer**: Exposer is to expose the snapshot/target volume as a path/device name/endpoint that are recognizable by Velero generic data path. For different snapshot types/snapshot accesses, the Exposer may be different. This isolation guarantees that when we want to support other snapshot types/snapshot accesses, we only need to replace with a new Exposer and keep other components as is. \n**Velero Generic Data Path (VGDP)**: VGDP is the collective of modules that is introduced in [Unified Repository design][1]. Velero uses these modules to finish data transmission for various purposes. In includes uploaders and the backup repository. \n**Uploader**: Uploader is the module in VGDP that reads data from the source and writes to backup repository for backup; while read data from backup repository and write to the restore target for restore. At present, only file system uploader is supported. In future, the block level uploader will be added. For file system and basic block uploader, only Kopia uploader will be used, Restic will not be integrated with VBDM. \n\n### Replacement\n3rd parties could integrate their own data movement into Velero by replacing VBDM with their own DMs. The DMs should process DUCR/DDCRs appropriately and finally put them into one of the terminal states as shown in the DataUpload CRD and DataDownload CRD sections. \nTheoretically, replacing the DMP is also allowed. In this way, the entire workflow is customized, so this is out of the scope of this design. \n\n## Detailed Design\n\n### Backup Sequence\nBelow are the data movement actions and sequences during backup: \n![backup-sequence.png](backup-sequence.png) \n\nBelow are actions from Velero and DMP:\n\n**BIA Execute** \nThis is the existing logic in Velero. For a source PVC/PV, Velero delivers it to the corresponding BackupItemAction plugin, the plugin then takes the related actions to back it up.\nFor example, the existing CSI plugin takes a CSI snapshot to the volume represented by the PVC and then returns additional items (i.e., VolumeSnapshot, VolumeSnapshotContent and VolumeSnapshotClass) for Velero to further backup.\nTo support data movement, we will use BIA V2 which supports asynchronous operation management. Here is the Execute method from BIA V2: \n```\nExecute(item runtime.Unstructured, backup *api.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error)\n```\nBesides ```additionalItem``` (as the 2nd return value), Execute method will return one more resource list called ```itemToUpdate```, which means the items to be updated and persisted when the async operation completes. For details, visit [general progress monitoring design][2]. \nSpecifically, this mechanism will be used to persist DUCR into the persisted backup data, in another words, DUCR will be returned as ```itemToUpdate``` from Execute method. DUCR contains all the information the restore requires, so during restore, DUCR will be extracted from the backup data. \nAdditionally, in the same way, a DMP could add any other items into the persisted backup data. \nExecute method also returns the ```operationID``` which uniquely identifies the asynchronous operation. This ```operationID``` is generated by plugins. The [general progress monitoring design][2] doesn't restrict the format of the ```operationID```, for Velero CSI plugin, the ```operationID``` is a combination of the backup CR UID and the source PVC (represented by the ```item``` parameter) UID. \n\n**Create Snapshot** \nThe DMP creates a snapshot of the requested volume and deliver it to DM through DUCR. After that, the DMP leaves the snapshot and its related objects (e.g., VolumeSnapshot and VolumeSnapshotContent for CSI snapshot) to the DM, DM then has full control of the snapshot and its related objects, i.e., deciding whether to delete the snapshot or its related objects and when to do it. \nThis also indicates that the DUCR should contain the snapshot type specific information because different snapshot types may have their unique information. \nFor Velero built-in implementation, the existing logics to create the snapshots will be reused, specifically, for CSI snapshot, the related logics in CSI plugin are fully reused. \n\n**Create DataUpload CR** \nA DUCR is created for as the result of each Execute call, then Execute method will return and leave DUCR being processed asynchronously. \n\n**Set Backup As WaitForAsyncOperations** \n**Persist Backup** \nAfter ```Execute``` returns, the backup is set to ```WaitingForPluginOperations```, and then Velero is free to process other items or backups. \nBefore Velero moves to other items/backups, it will persist the backup data. This is the same as the existing behavior. \nThe backup then is left as ```WaitForAsyncOperations``` until the DM completes or timeout. \n\n**BIA Progress** \nVelero keeps monitoring the status of the backup by calling BIA V2’s Progress method. Below is the Progress method from BIA V2:\n```\nProgress(operationID string, backup *api.Backup) (velero.OperationProgress, error)\n```\nOn the call of this method, DMP will query the DUCR’s status. Some critical progress data is transferred from DUCR to the ```OperationProgress``` which is the return value of BIA V2’s Progress method. For example, NCompleted indicates the size/number of data that have been completed and NTotal indicates the total size/number of data. \nWhen the async operation completes, the Progress method returns an OperationProgress with ```Completed``` set as true. Then Velero will persist DUCR as well as any other items returned by DUP as ```itemToUpdate```. \nFinally, then backup is as ```Completed```. \nTo help BIA Progress find the corresponding DUCR, the ```operationID``` is saved along with the DUCR as a label ```velero.io/async-operation-id```.\n\nDUCRs are handled by the data movers, so how to handle them are totally decided by the data movers. Below covers the details of VBDM, plugging data movers should have their own actions and workflows. \n\n**Persist DataUpload CR** \nAs mentioned above, the DUCR will be persisted when it is completed under the help of BIA V2 async operation finalizing mechanism. \nThis means the backup tarball will be uploaded twice, this is as the designed behavior of [general progress monitoring design][2]. \n\nConclusively, as a result of the above executions:\n- A DataUpload CR is created and persisted to the backup tarball. The CR will be left there after the backup completes because the CR includes many information connecting to the backup that may be useful to end users or upper level modules. \n- A snapshot as well as the objects representing it are created. For CSI snapshot, a VolumeSnapshot object and a VolumeSnapshotContent object is created. The DMP leaves the snapshot as well as its related objects to DM for further processing. \n\nVBDM creates a Data Uploader Controller to handle the DUCRs in node-agent daemonset, therefore, on each node, there will be an instance of this controller. The controller connects to the backup repository and calls the uploader. Below are the VBDM actions. \n\n**Acquire Object Lock** \n**Release Object Lock** \nThere are multiple instances of Data Uploader Controllers and when a DUCR is created, there should be only one of the instances handle the CR. \nTherefore, an operation called “Acquired Object Lock” is used to reach a consensus among the controller instances so that only one controller instance takes over the CR and tries the next action – Expose for the CR. \nAfter the CR is completed in the Expose phase, the CR is released with the operation of “Release Object Lock”. \nWe fulfil the “Acquired Object Lock” and “Release Object Lock” under the help of Kubernetes API server and the etcd in the background, which guarantees strong write consistency among all the nodes. \n\n**Expose** \nFor some kinds of snapshot, it may not be usable directly after it is taken. For example, a CSI snapshot is represented by the VolumeSnapshot and VolumeSnapshotContent object, if we don’t do anything, we don’t see any PV really exists in the cluster, so VGDP has no way to access it. Meanwhile, when we have a PV representing the snapshot data, we still need a way to make it accessible by the VGDP. \nThe details of the expose process are snapshot specific, and for one kind of snapshot, we may have different methods to expose it to VGDP. Later, we will have a specific section to explain the current design of the Exposer. \n\n**Backup From Data Path** \nAfter a snapshot is exposed, VGDP will be able to access the snapshot data, so the controller calls the uploader to start the data backup. \nTo support cancellation and concurrent backup, the call to the VGDP is done asynchronously. How this asynchronization is implemented may be related to the Exposer. as the current design of Exposer, the asynchronization is implemented by the controller with go routines. \n\nWe keep VGDP reused for VBDM, so everything inside VGDP are kept as is. For details of VGDP, refer to the [Unified Repository design][1]. \n\n**Update Repo Snapshot ID**\nWhen VGDP completes backup, it returns an ID that represent the root object saved into the backup repository for this backup, through the root object, we will be able to enumerate the entire backup data. \nThis Repo Snapshot ID will be saved along with the DUCR. \n\n### DataUpload CRD\nBelow are the essential fields of DataUpload CRD. The CRD covers below information:\n- The information to manipulate the specified snapshot\n- The information to manipulate the specified data mover\n- The information to manipulate the specified backup repository\n- The progress of the current data upload\n- The result of the current data upload once it finishes\n\nFor snapshot manipulation:\n- ```snapshotType``` indicates the type of the snapshot, at present, the only valid value is ```CSI```. \n- If ```snapshotType``` is ```CSI```, ```csiSnapshot``` which is a pointer to a ```CSISnapshotSpec``` must not be absent. \n- ```CSISnapshotSpec``` specifies the information of the CSI snapshot, e.g., ```volumeSnapshot``` is the name of VolumeSnapshot object representing the CSI snapshot; ```storageClass``` specifies the name of the StorageClass of the source PVC, which will be used to create the backupPVC during the data upload.\n\nFor data mover manipulation:\n- ```datamover``` indicates the name of the data mover, if it is empty or ```velero```, it means the built-in data mover will be used for this data upload \n\nFor backup repository manipulation, ```backupStorageLocation``` is the name of the related BackupStorageLocation, where we can find all the required information.\n\nFor the progress, it includes the ```totalBytes``` and ```doneBytes``` so that other modules could easily cuclulate a progress. \n\nFor data upload result, ```snapshotID``` in the ```status``` field is the Repo Snapshot ID. Data movers may have their private outputs as a result of the DataUpload, they will be put in the ```dataMoverResult``` map of the ```status``` field. \n\nHere are the statuses of DataUpload CRD and their descriptions:\n- New: The DUCR has been created but not processed by a controller\n- Accepted: the Object lock has been acquired for this DUCR and the elected controller is trying to expose the snapshot\n- Prepared: the snapshot has been exposed, the related controller is starting to process the upload\n- InProgress: the data upload is in progress\n- Canceling: the data upload is being canceled\n- Canceled: the data upload has been canceled\n- Completed: the data upload has completed\n- Failed: the data upload has failed\n\nBelow is the full spec of DataUpload CRD:\n```\napiVersion: apiextensions.k8s.io/v1alpha1\nkind: CustomResourceDefinition\nmetadata:\n labels:\n component: velero\n name: datauploads.velero.io\nspec:\n conversion:\n strategy: None\n group: velero.io\n names:\n kind: DataUpload\n listKind: DataUploadList\n plural: datauploads\n singular: dataupload\n scope: Namespaced\n versions:\n - additionalPrinterColumns:\n - description: DataUpload status such as New/InProgress\n jsonPath: .status.phase\n name: Status\n type: string\n - description: Time duration since this DataUpload was started\n jsonPath: .status.startTimestamp\n name: Started\n type: date\n - description: Completed bytes\n format: int64\n jsonPath: .status.progress.bytesDone\n name: Bytes Done\n type: integer\n - description: Total bytes\n format: int64\n jsonPath: .status.progress.totalBytes\n name: Total Bytes\n type: integer\n - description: Name of the Backup Storage Location where this backup should be\n stored\n jsonPath: .spec.backupStorageLocation\n name: Storage Location\n type: string\n - description: Time duration since this DataUpload was created\n jsonPath: .metadata.creationTimestamp\n name: Age\n type: date\n name: v1\n schema:\n openAPIV3Schema:\n properties:\n spec:\n description: DataUploadSpec is the specification for a DataUpload.\n properties:\n backupStorageLocation:\n description: BackupStorageLocation is the name of the backup storage\n location where the backup repository is stored.\n type: string\n csiSnapshot:\n description: If SnapshotType is CSI, CSISnapshot provides the information\n of the CSI snapshot.\n properties:\n snapshotClass:\n description: SnapshotClass is the name of the snapshot class that\n the volume snapshot is created with\n type: string \n storageClass:\n description: StorageClass is the name of the storage class of\n the PVC that the volume snapshot is created from\n type: string\n volumeSnapshot:\n description: VolumeSnapshot is the name of the volume snapshot\n to be backed up\n type: string\n required:\n - storageClass\n - volumeSnapshot\n type: object\n datamover:\n description: DataMover specifies the data mover to be used by the\n backup. If DataMover is \"\" or \"velero\", the built-in data mover\n will be used.\n type: string\n\t\t\t operationTimeout:\n description: OperationTimeout specifies the time used to wait internal \n operations, e.g., wait the CSI snapshot to become readyToUse.\n type: string\n snapshotType:\n description: SnapshotType is the type of the snapshot to be backed\n up.\n type: string\n sourceNamespace:\n description: SourceNamespace is the original namespace where the volume\n is backed up from.\n type: string\n required:\n - backupStorageLocation\n - csiSnapshot\n - snapshotType\n - sourceNamespace\n type: object\n status:\n description: DataUploadStatus is the current status of a DataUpload.\n properties:\n completionTimestamp:\n description: CompletionTimestamp records the time a backup was completed.\n Completion time is recorded even on failed backups. Completion time\n is recorded before uploading the backup object. The server's time\n is used for CompletionTimestamps\n format: date-time\n nullable: true\n type: string\n dataMoverResult:\n additionalProperties:\n type: string\n description: DataMoverResult stores data-mover-specific information\n as a result of the DataUpload.\n nullable: true\n type: object\n message:\n description: Message is a message about the DataUpload's status.\n type: string\n\t\t\t node:\n description: Node is the name of the node where the DataUpload is running.\n type: string\t\t\t \n path:\n description: Path is the full path of the snapshot volume being backed\n up.\n type: string\n phase:\n description: Phase is the current state of the DataUpload.\n enum:\n - New\n - Accepted\n - Prepared\n - InProgress\n - Canceling\n - Canceled\n - Completed\n - Failed\n type: string\n progress:\n description: Progress holds the total number of bytes of the volume\n and the current number of backed up bytes. This can be used to display\n progress information about the backup operation.\n properties:\n bytesDone:\n format: int64\n type: integer\n totalBytes:\n format: int64\n type: integer\n type: object\n snapshotID:\n description: SnapshotID is the identifier for the snapshot in the\n backup repository.\n type: string\n startTimestamp:\n description: StartTimestamp records the time a backup was started.\n Separate from CreationTimestamp, since that value changes on restores.\n The server's time is used for StartTimestamps\n format: date-time\n nullable: true\n type: string\n type: object\n type: object\n\n```\n\n### Restore Sequence\n\nBelow are the data movement actions sequences during restore: \n![restore-sequence.png](restore-sequence.png) \n\nMany of the actions are the same with backup, here are the different ones. \n\n**Query Backup Result** \nThe essential information to be filled into DataDownload all comes from the DataUpload CR. For example, the Repo Snapshot ID is stored in the status fields of DataUpload CR. However, we don't want to restore the DataUpload CR and leave it in the cluster since it is useless after the restore. Therefore, we will retrieve the necessary information from DataUpload CR and store it in a temporary ConfigMap for the DM to use. There is one ConfigMap for each DataDownload CR and the ConfigMaps belong to a restore will be deleted when the restore finishes. \n\n**Prepare Volume Readiness** \nAs the current pattern, Velero delivers an object representing a volume, either a PVC or a PV, to DMP and Velero will create the object after DMP's Execute call returns. However, by this time, DM should have not finished the restore, so the volume is not ready for use. \nIn this step, DMP needs to mark the object as unready to use so as to prevent others from using it, i.e., a pod mounts the volume. Additionlly, DMP needs to provide an approach for DM to mark it as ready when the data movement finishes. \nHow to mark the volume as unready or ready varying from the type of the object, specifically, a PVC or a PV; and there are more than one ways to achieve this. \nBelow show the details of how to do this for CSI snapshot data movement. \nAfter the DMP submits the DataDownload CR, it does below modifications to the PVC spec:\n- Set spec.VolumeName to empty (\"\")\n- Add a selector with a matchLabel ```velero.io/dynamic-pv-restore```\n\nWith these two steps, it tells Kubernetes that the PVC is not bound and it only binds a PV with the ```velero.io/dynamic-pv-restore``` label. As a result, even after the PVC object is created by Velero later and is used by other resources, it is not usable until the DM creates the target PV. \n\n**Expose** \nThe purpose of expose process for restore is to create the target PV and make the PV accessible by VGDP. Later the Expose section will cover the details. \n\n**Finish Volume Readiness** \nBy the data restore finishes, the target PV is ready for use but it is not delivered to the outside world. This step is the follow up of Prepare Volume Readiness, which does necessary work to mark the volume ready to use. \nFor CSI snapshot restore, DM does below steps:\n- Set the target PV's claim reference (the ```claimRef``` filed) to the target PVC\n- Add the ```velero.io/dynamic-pv-restore``` label to the target PV\n\nBy the meantime, the target PVC should have been created in the source user namespace and waiting for binding. \nWhen the above steps are done, the target PVC will be bound immediately by Kubernetes. \nThis also means that Velero should not restore the PV if a data movement restore is involved, this follows the existing CSI snapshot behavior. \n\nFor restore, VBDM doesn’t need to persist anything. \n\n### DataDownload CRD\nBelow are the essential fields of DataDownload CRD. The CRD covers below information:\n- The information to manipulate the target volume\n- The information to manipulate the specified data mover\n- The information to manipulate the specified backup repository\n\nTarget volume information includes PVC and PV that represents the volume and the target namespace. \nThe data mover information and backup repository information are the same with DataUpload CRD.\nDataDownload CRD defines the same status as DataUpload CRD with nearly the same meanings. \n\nBelow is the full spec of DataDownload CRD:\n```\napiVersion: apiextensions.k8s.io/v1alpha1\nkind: CustomResourceDefinition\nmetadata:\n labels:\n component: velero\n name: datadownloads.velero.io\nspec:\n conversion:\n strategy: None\n group: velero.io\n names:\n kind: DataDownload\n listKind: DataDownloadList\n plural: datadownloads\n singular: datadownload\n scope: Namespaced\n versions:\n - DataDownload:\n - description: DataDownload status such as New/InProgress\n jsonPath: .status.phase\n name: Status\n type: string\n - description: Time duration since this DataDownload was started\n jsonPath: .status.startTimestamp\n name: Started\n type: date\n - description: Completed bytes\n format: int64\n jsonPath: .status.progress.bytesDone\n name: Bytes Done\n type: integer\n - description: Total bytes\n format: int64\n jsonPath: .status.progress.totalBytes\n name: Total Bytes\n type: integer\n - description: Time duration since this DataDownload was created\n jsonPath: .metadata.creationTimestamp\n name: Age\n type: date\n name: v1\n schema:\n openAPIV3Schema:\n properties:\n spec:\n description: SnapshotDownloadSpec is the specification for a SnapshotDownload.\n properties:\n backupStorageLocation:\n description: BackupStorageLocation is the name of the backup storage\n location where the backup repository is stored.\n type: string\n datamover:\n description: DataMover specifies the data mover to be used by the\n backup. If DataMover is \"\" or \"velero\", the built-in data mover\n will be used.\n type: string\n operationTimeout:\n description: OperationTimeout specifies the time used to wait internal\n operations, before returning error as timeout.\n type: string\n snapshotID:\n description: SnapshotID is the ID of the Velero backup snapshot to\n be restored from.\n type: string\n sourceNamespace:\n description: SourceNamespace is the original namespace where the volume\n is backed up from.\n type: string\n targetVolume:\n description: TargetVolume is the information of the target PVC and\n PV.\n properties:\n namespace:\n description: Namespace is the target namespace\n type: string\n pv:\n description: PV is the name of the target PV that is created by\n Velero restore\n type: string\n pvc:\n description: PVC is the name of the target PVC that is created\n by Velero restore\n type: string\n required:\n - namespace\n - pv\n - pvc\n type: object\n required:\n - backupStorageLocation\n - restoreName\n - snapshotID\n - sourceNamespace\n - targetVolume\n type: object\n status:\n description: SnapshotRestoreStatus is the current status of a SnapshotRestore.\n properties:\n completionTimestamp:\n description: CompletionTimestamp records the time a restore was completed.\n Completion time is recorded even on failed restores. The server's\n time is used for CompletionTimestamps\n format: date-time\n nullable: true\n type: string\n message:\n description: Message is a message about the snapshot restore's status.\n type: string\n\t\t\t node:\n description: Node is the name of the node where the DataDownload is running.\n type: string\t\t\t\t\n phase:\n description: Phase is the current state of theSnapshotRestore.\n enum:\n - New\n - Accepted\n - Prepared\n - InProgress\n - Canceling\n - Canceled\n - Completed\n - Failed\n type: string\n progress:\n description: Progress holds the total number of bytes of the snapshot\n and the current number of restored bytes. This can be used to display\n progress information about the restore operation.\n properties:\n bytesDone:\n format: int64\n type: integer\n totalBytes:\n format: int64\n type: integer\n type: object\n startTimestamp:\n description: StartTimestamp records the time a restore was started.\n The server's time is used for StartTimestamps\n format: date-time\n nullable: true\n type: string\n type: object\n type: object\n``` \n\n## Expose \n\n### Expose for DataUpload \nAt present, for a file system backup, VGDP accepts a string representing the root path of the snapshot to be backed up, the path should be accessible from the process/pod that VGDP is running. In future, VGDP may accept different access parameters. Anyway, the snapshot should be accessible local. \nTherefore, the first phase for Expose is to expose the snapshot to be locally accessed. This is a snapshot specific operation. \nFor CSI snapshot, the final target is to create below 3 objects in Velero namespace: \n- backupVSC: This is the Volume Snapshot Content object represents the CSI snapshot\n- backupVS: This the Volume Snapshot object for BackupVSC in Velero namespace\n- backupPVC: This is the PVC created from the backupVS in Velero namespace. Specifically, backupPVC’s data source points to backupVS\n- backupPod: This is a pod attaching backupPVC in Velero namespace. As Kubernetes restriction, the PV is not provisioned until the PVC is attached to a pod and the pod is scheduled to a node. Therefore, after the backupPod is running, the backupPV which represents the data of the snapshot will be provisioned\n- backupPV: This is the PV provisioned as a result of backupPod schedule, it has the same data of the snapshot\n\nInitially, the CSI VS object is created in the source user namespace (we call it sourceVS), after the Expose, all the objects will be in Velero namespace, so all the data upload activities happen in the Velero namespace only. \nAs you can see, we have duplicated some objects (sourceVS and sourceVSC), this is due to Kubernetes restriction – the data source reference cannot across namespaces. \nAfter the duplication completes, the objects related to the source user namespace will be deleted. \n\nBelow diagram shows the relationships of the objects: \n![expose-objects.png](expose-objects.png) \n\nAfter the first phase, we will see a backupPod attaching a backupPVC/backupPV which data is the same as the snapshot data. Then the second phase could start, this phase is related to the uploader. \nFor file system uploader, the target of this phase is to get a path that is accessible locally by the uploader. There are some alternatives:\n- Get the path in the backupPod, so that VGDP runs inside the backupPod\n- Get the path on the host, so that VGDP runs inside node-agent, this is similar to the existing PodVolumeBackup\n\nEach option has their pros and cons, in the current design, we will use the second way because it is simpler in implementation and more controllable in workflow. \n\n### Expose for DataDownload\nThe Expose operation for DataDownload still takes two phases, The first phase creates below objects:\n- restorePVC: It is a PVC in Velero namespace with the same specification, it is used to provision the restorePV\n- restorePod: It is used to attach the restorePVC so that the restorePV could be provisioned by Kubernetes\n- restorePV: It is provisioned by Kubernetes and bound to restorePVC\n\nData will be downloaded to the restorePV. No object is created in user source namespace and no activity is done there either. \nThe second phase is the same as DataUpload, that is, we still use the host path to access restorePV and run VGDP in node-agent. \n\n### Expose cleanup\nSome internal objects are created during the expose. Therefore, we need to clean them up to prevent internal objects from rampant growth. The cleanup happens in two cases:\n- When the controller finishes processing the DUCR/DDCR, this includes the cases that the DUCR/DDCR is completed, failed and cancelled. \n- When the DM restarts and the DM doesn't support restart recovery. When the DM comes back, it should detect all the ongoing DUCR/DDCR and clean up the expose. Specifically, VBDM should follow this rule since it doesn't support restart recovery. \n\n\n## Cancellation\nWe will leverage on BIA/RIA V2's Cancel method to implement the cancellation, below are the prototypes from BIA/RIA V2: \n``` \nCancel(operationID string, backup *api.Backup) error \nCancel(operationID string, restore *api.Restore) error\n``` \n\nAt present, Velero doesn’t support canceling an ongoing backup, the current version of BIA/RIA V2 framework has some problems to support the end to end cancellation as well. \nTherefore, the current design doesn’t aim to deliver an end-to-end cancellation workflow but to implement the cancellation workflow inside the data movement, in future, when the other two parts are ready for cancellation, the data movement cancellation workflow could be directly used. \n\nAdditionally, at present, the data movement cancellation will be used in the below scenarios: \n- When a backup is deleted, the backup deletion controller will call DMP’s Cancel method, so that the ongoing data movement will not run after the backup is deleted.\n- In the restart case, the ongoing backups will be marked as ```Failed``` when Velero restarts, at this time, DMP’s Cancel method will also be called when Velero server comes back because Velero will never process these backups. \n\nFor data movement implementation, a ```Cancel``` field is included in the DUCR/DDCR.\nDMP patches the DUCR/DDCR with ```Cancel``` field set to true, then it keeps querying the status of DUCR/DDCR until it comes to Canceled status or timeout, by which time, DMP returns the Cancel call to Velero. \nThen DM needs to handle the cancel request, e.g., stop the data transition. For VBDM, it sets a signal to the uploader and the uploader will abort in a short time. \nThe cancelled DUCR/DDCR is marked as ```Canceled```. \n\nBelow diagram shows VBDM’s cancel workflow (take backup for example, restore is the same). \n![cancel-sequence.png](cancel-sequence.png) \n\nIt is possible that a DM doesn’t support cancellation at all or only support in a specific phase (e.g., during InProgress phase), if the cancellation is requested at an unexpected time or to an unexpecting DM, the behavior is decided by the DMP and DM, below are some recommendations:\n- If a DM doesn't support cancellation at all, DMP should be aware of this, so the DMP could return an error and fail early\n- If the cancellation is requested at an unexpected time, DMP is possibly not aware of this, it could still deliver it to the DM, so both Velero and DMP wait there until DM completes the cancel request or timeout\n\nVBDM's cancellation exactly follows the above rules. \n\n\n## Parallelism\nVelero uses BIA/RIA V2 to launch data movement tasks, so from Velero’s perspective, the DataUpload/DataDownload CRs from the running backups will be submitted in parallel. \nThen how these CRs are handled is decided by data movers, in another words, the specific data mover decides whether to handle them sequentially or in parallel, as well what the parallelism is like. Velero makes no restriction to data movers regarding to this. \nNext, let’s focus on the parallelism of VBDM, which could also be a reference of the plugin data movers. \nVBDM is hosted by Velero node-agent, so there is one data movement controller instance on each Kubernetes node, which also means that these instances could handle the DataUpload/DataDownload CRs in parallel. \nOn the other hand, a volume/volume snapshot may be accessed from only one or multiple nodes varying from its location, the backend storage architecture, etc. Therefore, the first decisive factor of the parallelism is the accessibility of a volume/volume snapshot. \nTherefore, we have below principles:\n- We should spread the data movement activities equally to all the nodes in the cluster. This requires a load balance design from Velero\n- In one node, it doesn’t mean the more concurrency the better, because the data movement activities are high in resource consumption, i.e., CPU, memory, and network throughput. For the same consideration, we should make this configurable because the best number should be set by users according to the bottleneck they detect \n\nWe will address the two principles step by step. As the first step, VBDM’s parallelism is designed as below: \n- We don’t create the load balancing mechanism for the first step, we don’t detect the accessibility of the volume/volume snapshot explicitly. Instead, we create the backupPod/restorePod under the help of Kubernetes, Kubernetes schedules the backupPod/restorePod to the appropriate node, then the data movement controller on that node will handle the DataUpload/DataDownload CR there, so the resource will be consumed from that node.\n- We expose the configurable concurrency value per node, for details of how the concurrency number constraints various backups and restores which share VGDP, check the [node-agent concurrency design][3]. \n\nAs for the resource consumption, it is related to the data scale of the data movement activity and it is charged to node-agent pods, so users should configure enough resource to node-agent pods. \n\n## Progress Report\nWhen a DUCR/DDCR is in InProgress phase, users could check the progress. \nIn DUCR/DDCR’s status, we have fields like ```totalBytes``` and ```doneBytes```, the same values will be displayed as a result of below querires:\n- Call ```kubectl get dataupload -n velero xxx or kubectl get datadownload -n velero xxx```.\n- Call ```velero backup describe –details```. This is implemented as part of BIA/RIA V2, the above values are transferred to async operation and this command retrieve them from the async operation instead of DUCR/DDCR. See [general progress monitoring design][2] for details\n\n\n## Backup Sync\nDUCR contains the information that is required during restore but as mentioned above, it will not be synced because during restore its information is retrieved dynamically. Therefore, we have no change to Backup Sync. \n\n## Backup Deletion\nOnce a backup is deleted, the data in the backup repository should be deleted as well. On the other hand, the data is created by the specific DM, Velero doesn't know how to delete the data. Therefore, Velero relies on the DM to delete the backup data. \nAs the current workflow, when ```velero backup delete``` CLI is called, a ```deletebackuprequests``` CR is submitted; after the backup delete controller finishes all the work, the ```deletebackuprequests``` CR will be deleted. In order to give an opportunity for the DM to delete the backup data, we remedy the workflow as below: \n- During the backup deletion, the backup delete controller retrieves all the DUCRs belong to the backup \n- The backup delete controller then creates the DUCRs into the cluster\n- Before deleting the ```deletebackuprequests``` CR, the backup delete controller adds a ```velero.io/dm-delete-backup``` finalizer to the CR\n- As a result, the ```deletebackuprequests``` CR will not be deleted until the finalizer is removed\n- The DM needs to watch the ```deletebackuprequests``` CRs with the ```velero.io/dm-delete-backup``` finalizer\n- Once the DM finds one, it collects a list of DUCRs that belong to the backup indicating by the ```deletebackuprequests``` CR's spec\n- If the list is not empty, the DM delete the backup data for each of the DUCRs in the list as well as the DUCRs themselves\n- Finally, when all the items in the list are processed successfully, the DM removes the ```velero.io/dm-delete-backup``` finalizer\n- Otherwise, if any error happens during the processing, the ```deletebackuprequests``` CR will be left there with the ```velero.io/dm-delete-backup``` finalizer, as well as the failed DUCRs\n- DMs may use a periodical manner to retry the failed delete requests \n\n## Restarts\nIf Velero restarts during a data movement activity, the backup/restore will be marked as failed when Velero server comes back, by this time, Velero will request a cancellation to the ongoing data movement. \nIf DM restarts, Velero has no way to detect this, DM is expected to:\n- Either recover from the restart and continue the data movement\n- Or if DM doesn’t support recovery, it should cancel the data movement and mark the DUCR/DDCR as failed. DM should also clear any internal objects created during the data movement before and after the restart \n\nAt present, VBDM doesn't support recovery, so it will follow the second rule. \n\n## Kopia For Block Device\nTo work with block devices, VGDP will be updated. Today, when Kopia attempts to create a snapshot of the block device, it will error because kopia does not support this file type. Kopia does have a nice set of interfaces that are able to be extended though.\n\n**Notice**\nThe Kopia block mode uploader only supports non-Windows platforms, because the block mode code invokes some system calls that are not present in the Windows platform.\n\nTo achieve the necessary information to determine the type of volume that is being used, we will need to pass in the volume mode in provider interface.\n\n```go\ntype PersistentVolumeMode string \n\nconst (\n\t// PersistentVolumeBlock means the volume will not be formatted with a filesystem and will remain a raw block device.\n\tPersistentVolumeBlock PersistentVolumeMode = \"Block\"\n\t// PersistentVolumeFilesystem means the volume will be or is formatted with a filesystem.\n\tPersistentVolumeFilesystem PersistentVolumeMode = \"Filesystem\"\n)\n\n// Provider which is designed for one pod volume to do the backup or restore\ntype Provider interface {\n\t// RunBackup which will do backup for one specific volume and return snapshotID, isSnapshotEmpty, error\n\t// updater is used for updating backup progress which implement by third-party\n\tRunBackup(\n\t\tctx context.Context,\n\t\tpath string,\n\t\trealSource string,\n\t\ttags map[string]string,\n\t\tforceFull bool,\n\t\tparentSnapshot string,\n volMode uploader.PersistentVolumeMode,\n\t\tuploaderCfg shared.UploaderConfig,\n\t\tupdater uploader.ProgressUpdater) (string, bool, error)\n\n RunRestore(\n\t\tctx context.Context,\n\t\tsnapshotID string,\n\t\tvolumePath string,\n\t\tvolMode uploader.PersistentVolumeMode,\n\t\tupdater uploader.ProgressUpdater) error\n```\n\nIn this case, we will extend the default kopia uploader to add the ability, when a given volume is for a block mode and is mapped as a device, we will use the [StreamingFile](https://pkg.go.dev/github.com/kopia/kopia@v0.13.0/fs#StreamingFile) to stream the device and backup to the kopia repository. \n\n```go\nfunc getLocalBlockEntry(sourcePath string) (fs.Entry, error) {\n\tsource, err := resolveSymlink(sourcePath)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"resolveSymlink\")\n\t}\n\n\tfileInfo, err := os.Lstat(source)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"unable to get the source device information %s\", source)\n\t}\n\n\tif (fileInfo.Sys().(*syscall.Stat_t).Mode & syscall.S_IFMT) != syscall.S_IFBLK {\n\t\treturn nil, errors.Errorf(\"source path %s is not a block device\", source)\n\t}\n\n\tdevice, err := os.Open(source)\n\tif err != nil {\n\t\tif os.IsPermission(err) || err.Error() == ErrNotPermitted {\n\t\t\treturn nil, errors.Wrapf(err, \"no permission to open the source device %s, make sure that node agent is running in privileged mode\", source)\n\t\t}\n\t\treturn nil, errors.Wrapf(err, \"unable to open the source device %s\", source)\n\t}\n\n\tsf := virtualfs.StreamingFileFromReader(source, device)\n\treturn virtualfs.NewStaticDirectory(source, []fs.Entry{sf}), nil\n}\n```\n\nIn the `pkg/uploader/kopia/snapshot.go` this is used in the Backup call like\n\n```go\n\tif volMode == uploader.PersistentVolumeFilesystem {\n\t\t// to be consistent with restic when backup empty dir returns one error for upper logic handle\n\t\tdirs, err := os.ReadDir(source)\n\t\tif err != nil {\n\t\t\treturn nil, false, errors.Wrapf(err, \"Unable to read dir in path %s\", source)\n\t\t} else if len(dirs) == 0 {\n\t\t\treturn nil, true, nil\n\t\t}\n\t}\n\n source = filepath.Clean(source)\n ...\n\n\tvar sourceEntry fs.Entry\n\n\tif volMode == uploader.PersistentVolumeBlock {\n\t\tsourceEntry, err = getLocalBlockEntry(source)\n\t\tif err != nil {\n\t\t\treturn nil, false, errors.Wrap(err, \"unable to get local block device entry\")\n\t\t}\n\t} else {\n\t\tsourceEntry, err = getLocalFSEntry(source)\n\t\tif err != nil {\n\t\t\treturn nil, false, errors.Wrap(err, \"unable to get local filesystem entry\")\n\t\t}\n\t}\n\n ...\n \tsnapID, snapshotSize, err := SnapshotSource(kopiaCtx, repoWriter, fsUploader, sourceInfo, sourceEntry, forceFull, parentSnapshot, tags, log, \"Kopia Uploader\")\n\n```\n\nTo handle restore, we need to extend the [Output](https://pkg.go.dev/github.com/kopia/kopia@v0.13.0/snapshot/restore#Output) interface and specifically the [FilesystemOutput](https://pkg.go.dev/github.com/kopia/kopia@v0.13.0/snapshot/restore#FilesystemOutput).\n\nWe only need to extend two functions the rest will be passed through.\n\n```go\ntype BlockOutput struct {\n\t*restore.FilesystemOutput\n\n\ttargetFileName string\n}\n\nvar _ restore.Output = &BlockOutput{}\n\nconst bufferSize = 128 * 1024\n\nfunc (o *BlockOutput) WriteFile(ctx context.Context, relativePath string, remoteFile fs.File) error {\n\tremoteReader, err := remoteFile.Open(ctx)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"failed to open remote file %s\", remoteFile.Name())\n\t}\n\tdefer remoteReader.Close()\n\n\ttargetFile, err := os.Create(o.targetFileName)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"failed to open file %s\", o.targetFileName)\n\t}\n\tdefer targetFile.Close()\n\n\tbuffer := make([]byte, bufferSize)\n\n\treadData := true\n\tfor readData {\n\t\tbytesToWrite, err := remoteReader.Read(buffer)\n\t\tif err != nil {\n\t\t\tif err != io.EOF {\n\t\t\t\treturn errors.Wrapf(err, \"failed to read data from remote file %s\", o.targetFileName)\n\t\t\t}\n\t\t\treadData = false\n\t\t}\n\n\t\tif bytesToWrite > 0 {\n\t\t\toffset := 0\n\t\t\tfor bytesToWrite > 0 {\n\t\t\t\tif bytesWritten, err := targetFile.Write(buffer[offset:bytesToWrite]); err == nil {\n\t\t\t\t\tbytesToWrite -= bytesWritten\n\t\t\t\t\toffset += bytesWritten\n\t\t\t\t} else {\n\t\t\t\t\treturn errors.Wrapf(err, \"failed to write data to file %s\", o.targetFileName)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (o *BlockOutput) BeginDirectory(ctx context.Context, relativePath string, e fs.Directory) error {\n\tvar err error\n\to.targetFileName, err = filepath.EvalSymlinks(o.TargetPath)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"unable to evaluate symlinks for %s\", o.targetFileName)\n\t}\n\n\tfileInfo, err := os.Lstat(o.targetFileName)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"unable to get the target device information for %s\", o.TargetPath)\n\t}\n\n\tif (fileInfo.Sys().(*syscall.Stat_t).Mode & syscall.S_IFMT) != syscall.S_IFBLK {\n\t\treturn errors.Errorf(\"target file %s is not a block device\", o.TargetPath)\n\t}\n\n\treturn nil\n}\n```\n\nAdditional mount is required in the node-agent specification to resolve symlinks to the block devices from /host_pods/POD_ID/volumeDevices/kubernetes.io~csi directory.\n\n```yaml\n - mountPath: /var/lib/kubelet/plugins\n mountPropagation: HostToContainer\n name: host-plugins\n....\n - hostPath:\n path: /var/lib/kubelet/plugins\n name: host-plugins\n```\n\nPrivileged mode is required to access the block devices in /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/publish directory as confirmed by testing on EKS and Minikube.\n\n```yaml\n\t\t\t\t\t\t\tSecurityContext: &corev1.SecurityContext{\n\t\t\t\t\t\t\t\tPrivileged: &c.privilegedNodeAgent,\n\t\t\t\t\t\t\t},\n```\n\n## Plugin Data Movers\nThere should be only one DM to handle a specific DUCR/DDCR in all cases. If more than one DMs process a DUCR/DDCR at the same time, there will be a disaster. \nTherefore, a DM should check the dataMover field of DUCR/DDCR and process the CRs belong to it only. \nFor example, VBDM reconciles DUCR/DDCR with their ```dataMover``` field set to \"\" or \"velero\", it will skip all others. \nThis means during the installation, users are allowed to install more than one DMs, but the DMs should follow the above rule. \nWhen creating a backup, we should allow users to specify the data mover, so a new backup CLI option is required. \nFor restore, we should retrieve the same information from the corresponding backup, so that the data mover selection is consistent. \n\nAt present, Velero doesn't have the capability to verify the existence of the specified data mover. As a result, if a wrong data mover name is specified for the backup or the specified data mover is not installed, nothing will fail early, DUCR/DDCR is still created and Velero will wait there until timeout. \n\nPlugin DMs may need some private configurations, the plugin DM providers are recommended to create a self-managed configMap to take the information. Velero doesn't maintain the lifecycle of the configMap. \nBesides, the configMap is recommended to named as the DM's name, in this way, if Velero or DMP recognizes some generic options that varies between DMs, the options could be added into the configMap and visited by Velero or DMP. \n\nConclusively, below are the steps plugin DMs need to do in order to integrate to Velero volume snapshot data movement. \n### Backup\n- Handle and only handle DUCRs with the matching ```dataMover``` value\n- Maintain the phases and progresses of DUCRs correctly\n- If supported, response to the Cancel request of DUCRs\n- Dispose the volume snapshots as well as their related objects after the snapshot data is transferred\n\n### Restore\n- Handle and only handle DDCRs with the matching ```dataMover``` value\n- Maintain the phases and progresses of DDCRs correctly\n- If supported, response to the Cancel request of DDCRs\n- Create the PV with data restored to it\n- Set PV's ```claimRef``` to the provided PVC and set ```velero.io/dynamic-pv-restore``` label\n\n## Working Mode\nIt doesn’t mean that once the data movement feature is enabled users must move every snapshot. We will support below two working modes:\n- Don’t move snapshots. This is same with the existing CSI snapshot feature, that is, native snapshots are taken and kept\n- Move snapshot data and delete native snapshots. This means that once the data movement completes, the native snapshots will be deleted.\n\nFor this purpose, we need to add a new option in the backup command as well as the Backup CRD. \nThe same option for restore will be retrieved from the specified backup, so that the working mode is consistent. \n\n## Backup and Restore CRD Changes\nWe add below new fields in the Backup CRD:\n```\n\t// SnapshotMoveData specifies whether snapshot data should be moved\n\t// +optional\n\t// +nullable\n\tSnapshotMoveData *bool `json:\"snapshotMoveData,omitempty\"`\n\n\t// DataMover specifies the data mover to be used by the backup.\n\t// If DataMover is \"\" or \"velero\", the built-in data mover will be used.\n\t// +optional\n\tDataMover string `json:\"datamover,omitempty\"`\n```\t\nSnapshotMoveData will be used to decide the Working Mode. \nDataMover will be used to decide the data mover to handle the DUCR. DUCR's DataMover value is derived from this value. \n\nAs mentioned in the Plugin Data Movers section, the data movement information for a restore should be the same with the backup. Therefore, the working mode for restore should be decided by checking the corresponding Backup CR; when creating a DDCR, the DataMover value should be retrieved from the corresponding Backup Result. \n\n## Logging\nThe logs during the data movement are categorized as below:\n- Logs generated by Velero\n- Logs generated by DMPs\n- Logs generated by DMs\n\nFor 1 and 2, the existing plugin mechanism guarantees that the logs could be saved into the Velero server log as well as backup/restore persistent log. \nFor 3, Velero leverage on DMs to decide how to save the log, but they will not go to Velero server log or backup/restore persistent log. For VBDM, the logs are saved in the node-agent server log. \n\n## Installation\nDMs need to be configured during installation so that they can be installed. Plugin DMs may have their own configuration, for VGDM, the only requirement is to install Velero node-agent. \nMoreover, the DMP is also required during the installation. \n\nFrom release-1.14, the `github.com/vmware-tanzu/velero-plugin-for-csi` repository, which is the Velero CSI plugin, is merged into the `github.com/vmware-tanzu/velero` repository.\nThe reason to merge the CSI plugin is:\n* The VolumeSnapshot data mover depends on the CSI plugin, it's reasonabe to integrate them.\n* This change reduces the Velero deploying complexity.\n* This makes performance tuning easier in the future.\n\nAs a result, no need to install Velero CSI plugin anymore.\n\nFor example, to move CSI snapshot through VBDM, below is the installation script: \n```\nvelero install \\\n --provider \\\n --image \\\n --features=EnableCSI \\\n --use-node-agent \\ \n```\n\n## Upgrade\nFor VBDM, no new installation option is introduced, so upgrade is not affected.\nIf plugin DMs require new options and so the upgrade is affected, they should explain them in their own documents.\n\n## CLI\nAs explained in the Working Mode section, we add one more flag ```snapshot-move-data``` to indicate whether the snapshot data should be moved. \nAs explained in the Plugin Data Movers section, we add one more flag ```data-mover``` for users to configure the data mover to move the snapshot data. \nExample of backup command are as below. \n\nBelow CLI means to create a backup with volume snapshot data movement enabled and with VBDM as the data mover: \n```\nvelero backup create xxx --include-namespaces --snapshot-move-data\n```\n\nBelow CLI has the same meaning as the first one: \n```\nvelero backup create xxx --include-namespaces --snapshot-move-data --data-mover velero\n```\n\nBelow CLI means to create a backup with volume snapshot data movement enabled and with \"xxx-plugin-dm\" as the data mover: \n```\nvelero backup create xxx --include-namespaces --snapshot-move-data --data-mover xxx-plugin-dm\n```\n\nRestore command is kept as is. \n\n\n\n\n[1]: ../unified-repo-and-kopia-integration/unified-repo-and-kopia-integration.md\n[2]: ../general-progress-monitoring.md\n[3]: ../node-agent-concurrency.md" }, { "path": "design/Implemented/wait-for-additional-items.md", "content": "# Wait for AdditionalItems to be ready on Restore\n\nWhen a velero `RestoreItemAction` plugin returns a list of resources\nvia `AdditionalItems`, velero restores these resources before\nrestoring the current resource. There is a race condition here, as it\nis possible that after running the restore on these returned items,\nthe current item's restore might execute before the additional items\nare available. Depending on the nature of the dependency between the\ncurrent item and the additional items, this could cause the restore of\nthe current item to fail.\n\n## Goals\n\n- Enable Velero to ensure that Additional items returned by a restore\n plugin's `Execute` func are ready before restoring the current item\n- Augment the RestoreItemAction plugin interface to allow the plugins\n to determine when an additional item is ready, since doing so\n requires knowledge specific to the resource type.\n\n## Background\n\nBecause Velero does not wait after restoring additional items to\nrestore the current item, in some cases the current item restore will\nfail if the additional items are not yet ready. Velero (and the\n`RestoreItemAction` plugins) need to implement this \"wait until ready\"\nfunctionality.\n\n## High-Level Design\n\nAfter each RestoreItemAction execute call (and following restore of\nany returned additional items) , we need to wait for these returned\nAdditional Items to be ready before restoring the current item. In\norder to do this, we also need to extend the `RestoreItemActionExecuteOutput`\nstruct to allow the plugin which returned additional items to\ndetermine whether they are ready.\n\n## Detailed Design\n\n### `restoreItem` Changes\n\nWhen each `RestoreItemAction` `Execute()` call returns, the\n`RestoreItemActionExecuteOutput` struct contains a slice of\n`AdditionalItems` which must be restored before this item can be\nrestored. After restoring these items, Velero needs to be able to wait\nfor them to be ready before moving on to the next item. Right after\nlooping over the additional items at\nhttps://github.com/vmware-tanzu/velero/blob/main/pkg/restore/restore.go#L960-L991\n\nwe still have a reference to the additional items (`GroupResource` and\nnamespaced name), as well as a reference to the `RestoreItemAction`\nplugin which required it.\n\nAt this point, if the `RestoreItemActionExecuteOutput`\n`WaitForAdditionalItems` field is set to `true` we need to call a func\nsimilar to `crdAvailable` which we will call `itemsAvailable`\nhttps://github.com/vmware-tanzu/velero/blob/main/pkg/restore/restore.go#L623\nThis func should also be defined within restore.go\n\nInstead of the one minute CRD timeout, we'll use a timeout specific to\nwaiting for additional items. There will be a new field added to\nserverConfig, `additionalItemsReadyTimeout`, with a\n`defaultAdditionalItemsReadyTimeout` const set to 10 minutes. In\naddition, each plugin will be able to define an override for the\nglobal server-level value, which will be added as another optional\nfield in the `RestoreItemActionExecuteOutput` struct. Instead of the\n`IsUnstructuredCRDReady` call, we'll call `AreAdditionalItemsReady` on\nthe plugin, passing in the same `AdditionalItems` slice as an argument\n(with items which failed to restore filtered out). If this func\nreturns an error, then `itemsAvailable` will propagate the error, and\n`restoreItem` will handle it the same way it handles an error return\non restoring an additional item. If the timeout is reached without\nready returning true, velero will continue on to attempt restore of\nthe current item.\n\n### `RestoreItemAction` plugin interface changes\n\nIn order to implement the `AreAdditionalItemsReady` plugin func, a new\nfunction will be added to the `RestoreItemAction` interface.\n```\ntype RestoreItemAction interface {\n // AppliesTo returns information about which resources this action should be invoked for.\n // A RestoreItemAction's Execute function will only be invoked on items that match the returned\n // selector. A zero-valued ResourceSelector matches all resources.\n AppliesTo() (ResourceSelector, error)\n\n // Execute allows the ItemAction to perform arbitrary logic with the item being restored,\n // including mutating the item itself prior to restore. The item (unmodified or modified)\n // should be returned, along with an optional slice of ResourceIdentifiers specifying additional\n // related items that should be restored, a warning (which will be logged but will not prevent\n // the item from being restored) or error (which will be logged and will prevent the item\n // from being restored) if applicable.\n Execute(input *RestoreItemActionExecuteInput) (*RestoreItemActionExecuteOutput, error)\n\n\t// AreAdditionalItemsReady allows the ItemAction to communicate whether the passed-in\n\t// slice of AdditionalItems (previously returned by Execute())\n\t// are ready. Returns true if all items are ready, and false\n\t// otherwise. The second return value is an error string if an\n\t// error occurred.\n\tAreAdditionalItemsReady(restore *api.Restore, AdditionalItems []ResourceIdentifier) (bool, string)\n}\n```\n\n### `RestoreItemActionExecuteOutput` changes\n\nTwo new fields will be added to `RestoreItemActionExecuteOutput`, both\noptional. `AdditionalItemsReadyTimeout`, if non-zero, will override\n`serverConfig.additionalItemsReadyTimeout`. If\n`WaitForAdditionalItems` is true, then `restoreItem` will call\n`itemsAvailable` which will invoke the plugin func\n`AreAdditionalItemsReady` and wait until the func returns true or the\ntimeout is reached. If `WaitForAdditionalItems` is false (the default\ncase), then current velero behavior will be followed. Existing plugins\nwhich do not need to signal to wait for `AdditionalItems` won't need\nto change their `Execute()` functions.\n\nIn addition, a new func, `WithItemsWait()` will\nbe added to `RestoreItemActionExecuteOutput` similar to\n`WithoutRestore()` which will set `WaitForAdditionalItems` to\ntrue. This will allow a plugin to include waiting for\nAdditionalItems like this:\n```\nfunc AreAdditionalItemsReady (restore *api.Restore, additionalItems []ResourceIdentifier) (bool, string) {\n\t...\n\treturn true, \"\"\n}\nfunc (p *RestorePlugin) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\t...\n\treturn velero.NewRestoreItemActionExecuteOutput(input.Item).WithItemsWait(), nil\n}\n\n```\n\n\n```\n// RestoreItemActionExecuteOutput contains the output variables for the ItemAction's Execution function.\ntype RestoreItemActionExecuteOutput struct {\n\t// UpdatedItem is the item being restored mutated by ItemAction.\n\tUpdatedItem runtime.Unstructured\n\n\t// AdditionalItems is a list of additional related items that should\n\t// be restored.\n\tAdditionalItems []ResourceIdentifier\n\n\t// SkipRestore tells velero to stop executing further actions\n\t// on this item, and skip the restore step. When this field's\n\t// value is true, AdditionalItems will be ignored.\n\tSkipRestore bool\n\n // WaitForAdditionalItems determines whether velero will wait\n\t// until AreAdditionalItemsReady returns true before restoring\n\t// this item. If this field's value is true, then after restoring\n\t// the returned AdditionalItems, velero will not restore this item\n\t// until AreAdditionalItemsReady returns true or the timeout is\n // reached. Otherwise, AreAdditionalItemsReady is not called.\n WaitForAdditionalItems bool\n\n\t// AdditionalItemsReadyTimeout will override serverConfig.additionalItemsReadyTimeout\n\t// if specified. This value specifies how long velero will wait\n\t// for additional items to be ready before moving on.\n\tAdditionalItemsReadyTimeout time.Duration\n}\n\n// WithItemsWait returns RestoreItemActionExecuteOutput with WaitForAdditionalItems set to true.\nfunc (r *RestoreItemActionExecuteOutput) WithItemsWait()\n) *RestoreItemActionExecuteOutput {\n\tr.WaitForAdditionalItems = true\n\treturn r\n}\n\n```\n\n## New design iteration (Feb 2021)\n\nIn starting the implementation based on the originally approved\ndesign, I've run into an unexpected snag. When adding the wait func\npointer to the `RestoreItemActionExecuteOutput` struct, I had\nforgotten about the protocol buffer message format that's used for\npassing args to the plugin methods. Funcs are predefined RPC calls\nwith autogenerated go code, so we can't just pass a regular golang\nfunc pointer in the struct. I've modified the above design to instead\nuse an explicit `AreAdditionalItemsReady` func. Since this will break\nbackwards compatibility with current `RestoreItemAction` plugins,\nimplementation of this feature should wait until Velero plugin\nversioning, as described in\nhttps://github.com/vmware-tanzu/velero/issues/3285 is\nimplemented. With plugin versioning in place, existing (non-versioned\nor 1.0-versioned) `RestoreItemAction` plugins which do not define\n`AreAdditionalItemsReady` would be able to coexist with a\nto-be-implemented `RestoreItemAction` plugin version 2.0 (or 1.1,\netc.) which defines this new interface method. Without plugin\nversioning, implementing this feature would break all existing plugins\nuntil they define `AreAdditionalItemsReady`.\n\nAlso note that when moving to the new plugin version, the vast\nmajority of plugins will probably not need to wait for additional\nitems. All they will need to do to react to this plugin interface\nchange would be to define the following in the plugin:\n\n```\nfunc AreAdditionalItemsReady (restore *api.Restore, additionalItems []ResourceIdentifier) (bool, string) {\n\treturn true, \"\"\n}\n```\n\nAs long as they never set `WaitForAdditionalItems` to true, this\nfunction won't be called anyway, but if it is called, there will be no\nwaiting, since it will always return true.\n" }, { "path": "design/Implemented/wildcard-namespace-support-design.md", "content": "\n# Wildcard Namespace Support\n\n## Abstract\n\nVelero currently treats namespace patterns with glob characters as literal strings. This design adds wildcard expansion to support flexible namespace selection using patterns like `app-*` or `test-{dev,staging}`.\n\n## Background\n\nRequested in [#1874](https://github.com/vmware-tanzu/velero/issues/1874) for more flexible namespace selection.\n\n## Goals\n\n- Support glob pattern expansion in namespace includes/excludes\n- Maintain backward compatibility with existing `*` behavior\n\n## Non-Goals\n\n- Complex regex patterns beyond basic globs\n\n## High-Level Design\n\nWildcard expansion occurs early in both backup and restore flows, converting patterns to literal namespace lists before normal processing.\n\n### Backup Flow\n\nExpansion happens in `getResourceItems()` before namespace collection:\n1. Check if wildcards exist using `ShouldExpandWildcards()`\n2. Expand patterns against active cluster namespaces\n3. Replace includes/excludes with expanded literal namespaces\n4. Continue with normal backup processing\n\n### Restore Flow\n\nExpansion occurs in `execute()` after parsing backup contents:\n1. Extract available namespaces from backup tar\n2. Expand patterns against backup namespaces (not cluster namespaces)\n3. Update restore context with expanded namespaces\n4. Continue with normal restore processing\n\nThis ensures restore wildcards match actual backup contents, not current cluster state.\n\n## Detailed Design\n\n### Status Fields\n\nAdd wildcard expansion tracking to backup and restore CRDs:\n\n```go\ntype WildcardNamespaceStatus struct {\n // IncludeWildcardMatches records namespaces that matched include patterns\n // +optional\n IncludeWildcardMatches []string `json:\"includeWildcardMatches,omitempty\"`\n \n // ExcludeWildcardMatches records namespaces that matched exclude patterns \n // +optional\n ExcludeWildcardMatches []string `json:\"excludeWildcardMatches,omitempty\"`\n \n // WildcardResult records final namespaces after wildcard processing\n // +optional\n WildcardResult []string `json:\"wildcardResult,omitempty\"`\n}\n\n// Added to both BackupStatus and RestoreStatus\ntype BackupStatus struct {\n // WildcardNamespaces contains wildcard expansion results\n // +optional\n WildcardNamespaces *WildcardNamespaceStatus `json:\"wildcardNamespaces,omitempty\"`\n}\n```\n\n### Wildcard Expansion Package\n\nNew `pkg/util/wildcard/expand.go` package provides:\n\n- `ShouldExpandWildcards()` - Skip expansion for simple \"*\" case\n- `ExpandWildcards()` - Main expansion function using `github.com/gobwas/glob`\n- Pattern validation rejecting unsupported regex symbols\n\n**Supported patterns**: `*`, `?`, `[abc]`, `{a,b,c}` \n**Unsupported**: `|()`, `**`\n\n### Implementation Details\n\n#### Backup Integration (`pkg/backup/item_collector.go`)\n\nExpansion in `getResourceItems()`:\n- Call `wildcard.ExpandWildcards()` with cluster namespaces\n- Update `NamespaceIncludesExcludes` with expanded results\n- Populate status fields with expansion results\n\n#### Restore Integration (`pkg/restore/restore.go`)\n\nExpansion in `execute()`:\n```go\nif wildcard.ShouldExpandWildcards(includes, excludes) {\n availableNamespaces := extractNamespacesFromBackup(backupResources)\n expandedIncludes, expandedExcludes, err := wildcard.ExpandWildcards(\n availableNamespaces, includes, excludes)\n // Update context and status\n}\n```\n\n## Alternatives Considered\n\n1. **Client-side expansion**: Rejected because it wouldn't work for scheduled backups\n2. **Expansion in `collectNamespaces`**: Rejected because these functions expect literal namespaces\n\n## Compatibility\n\nMaintains full backward compatibility - existing \"*\" behavior unchanged.\n\n## Implementation\n\nTarget: Velero 1.18\n" }, { "path": "design/_template.md", "content": "# Design proposal template ``\n\n_Note_: The preferred style for design documents is one sentence per line.\n*Do not wrap lines*.\nThis aids in review of the document as changes to a line are not obscured by the reflowing those changes caused and has a side effect of avoiding debate about one or two space after a period.\n\n_Note_: The name of the file should follow the name pattern `_design.md`, e.g:\n`listener-design.md`.\n\n## Abstract\nOne to two sentences that describes the goal of this proposal and the problem being solved by the proposed change.\nThe reader should be able to tell by the title, and the opening paragraph, if this document is relevant to them.\n\n## Background\nOne to two paragraphs of exposition to set the context for this proposal.\n\n## Goals\n- A short list of things which will be accomplished by implementing this proposal.\n- Two things is ok.\n- Three is pushing it.\n- More than three goals suggests that the proposal's scope is too large.\n\n## Non Goals\n- A short list of items which are:\n- a. out of scope\n- b. follow on items which are deliberately excluded from this proposal.\n\n\n## High-Level Design\nOne to two paragraphs that describe the high level changes that will be made to implement this proposal.\n\n## Detailed Design\nA detailed design describing how the changes to the product should be made.\n\nThe names of types, fields, interfaces, and methods should be agreed on here, not debated in code review.\nThe same applies to changes in CRDs, YAML examples, and so on.\n\nIdeally the changes should be made in sequence so that the work required to implement this design can be done incrementally, possibly in parallel.\n\n## Alternatives Considered\nIf there are alternative high level or detailed designs that were not pursued they should be called out here with a brief explanation of why they were not pursued.\n\n## Security Considerations\nIf this proposal has an impact to the security of the product, its users, or data stored or transmitted via the product, they must be addressed here.\n\n## Compatibility\nA discussion of any compatibility issues that need to be considered\n\n## Implementation\nA description of the implementation, timelines, and any resources that have agreed to contribute.\n\n## Open Issues\nA discussion of issues relating to this proposal for which the author does not know the solution. This section may be omitted if there are none.\n" }, { "path": "design/cli-install-changes.md", "content": "# Proposal for a more intuitive CLI to install and configure Velero\n\nCurrently, the Velero CLI tool has a `install` command that configures numerous major and minor aspects of Velero. As a result, the combined set of flags for this `install` command makes it hard to intuit and reason about the different Velero components. This document proposes changes to improve the UX for installation and configuration in a way that would make it easier for the user to discover what needs to be configured by looking at what is available in the CLI rather then having to rely heavily on our documentation for the usage. At the same time, it is expected that the documentation update to reflect these changes will also make the documentation flow easier to follow.\n\nThis proposal prioritizes discoverability and self-documentation over minimizing length or number of commands and flags.\n\n## Goals\n\n- Split flags currently under the `velero install` command into multiple commands, and group flags under commands in a way that allows a good level of discovery and self-documentation\n- Maintain compatibility with gitops practices (i.e. ability to generate a full set of yaml for install that can be stored in source control)\n- Have a clear path for deprecating commands\n\n## Non Goals\n\n- Introduce new CLI features \n- Propose changes to the CLI that go beyond the functionality of install and configure\n- Optimize for shorter length or number of commands/flags\n\n## Background\n\nThis document proposes users could benefit from a more intuitive and self-documenting CLI setup as compared to our existing CLI UX. Ultimately, it is proposed that a recipe-style CLI flow for installation, configuration and use would greatly contribute to this purpose.\n\nAlso, the `install` command currently can be reused to update Velero deployment configurations. For server and restic related install and configurations, settings will be moved to under `velero config`.\n\n## High-Level Design\n\nThe naming and organization of the proposed new CLI commands below have been inspired on the `kubectl` commands, particularly `kubectl set` and `kubectl config`.\n\n#### General CLI improvements\n\nThese are improvements that are part of this proposal:\n- Go over all flags and document what is optional, what is required, and default values.\n- Capitalize all help messages\n\n#### Commands\n\nThe organization of the commands follows this format:\n\n```\nvelero [resource] [operation] [flags]\n```\n\nTo conform with Velero's current practice: \n- commands will also work by swapping the operation/resource.\n- the \"object\" of a command is an argument, and flags are strictly for modifiers (example: `backup get my-backup` and not `backup get --name my-backup`)\n\nAll commands will include the `--dry-run` flag, which can be used to output yaml files containing the commands' configuration for resource creation or patching.\n\n`--dry-run generate resources, but don't send them to the cluster. Use with -o. Optional.`\n\nThe `--help` and `--output` flags will also be included for all commands, omitted below for brevity.\n\nBelow is the proposed set of new commands to setup and configure Velero.\n\n1) `velero config`\n\n```\n server Configure up the namespace, RBAC, deployment, etc., but does not add any external plugins, BSL/VSL definitions. This would be the minimum set of commands to get the Velero server up and running and ready to accept other configurations.\n --label-columns stringArray a comma-separated list of labels to be displayed as columns\n --show-labels show labels in the last column\n --image string image to use for the Velero and restic server pods. Optional. (default \"velero/velero:latest\")\n --pod-annotations mapStringString annotations to add to the Velero and restic pods. Optional. Format is key1=value1,key2=value2\n --restore-only run the server in restore-only mode. Optional.\n --pod-cpu-limit string CPU limit for Velero pod. A value of \"0\" is treated as unbounded. Optional. (default \"1000m\")\n --pod-cpu-request string CPU request for Velero pod. A value of \"0\" is treated as unbounded. Optional. (default \"500m\")\n --pod-mem-limit string memory limit for Velero pod. A value of \"0\" is treated as unbounded. Optional. (default \"256Mi\")\n --pod-mem-request string memory request for Velero pod. A value of \"0\" is treated as unbounded. Optional. (default \"128Mi\")\n --client-burst int maximum number of requests by the server to the Kubernetes API in a short period of time (default 30)\n --client-qps float32 maximum number of requests per second by the server to the Kubernetes API once the burst limit has been reached (default 20)\n --default-backup-ttl duration how long to wait by default before backups can be garbage collected (default 720h0m0s)\n --disable-controllers strings list of controllers to disable on startup. Valid values are backup,backup-sync,schedule,gc,backup-deletion,restore,download-request,restic-repo,server-status-request\n --log-format the format for log output. Valid values are text, json. (default text)\n --log-level the level at which to log. Valid values are debug, info, warning, error, fatal, panic. (default info)\n --metrics-address string the address to expose prometheus metrics (default \":8085\")\n --plugin-dir string directory containing Velero plugins (default \"/plugins\")\n --profiler-address string the address to expose the pprof profiler (default \"localhost:6060\")\n --restore-only run in a mode where only restores are allowed; backups, schedules, and garbage-collection are all disabled. DEPRECATED: this flag will be removed in v2.0. Use read-only backup storage locations instead.\n --restore-resource-priorities strings desired order of resource restores; any resource not in the list will be restored alphabetically after the prioritized resources (default [namespaces,storageclasses,persistentvolumes,persistentvolumeclaims,secrets,configmaps,serviceaccounts,limitranges,pods,replicaset,customresourcedefinitions])\n --terminating-resource-timeout duration how long to wait on persistent volumes and namespaces to terminate during a restore before timing out (default 10m0s)\n\n restic Configuration for restic operations.\n --default-prune-frequency duration how often 'restic prune' is run for restic repositories by default. Optional.\n --pod-annotations mapStringString annotations to add to the Velero and restic pods. Optional. Format is key1=value1,key2=value2\n --pod-cpu-limit string CPU limit for restic pod. A value of \"0\" is treated as unbounded. Optional. (default \"0\")\n --pod-cpu-request string CPU request for restic pod. A value of \"0\" is treated as unbounded. Optional. (default \"0\")\n --pod-mem-limit string memory limit for restic pod. A value of \"0\" is treated as unbounded. Optional. (default \"0\")\n --pod-mem-request string memory request for restic pod. A value of \"0\" is treated as unbounded. Optional. (default \"0\")\n --timeout duration how long backups/restores of pod volumes should be allowed to run before timing out (default 1h0m0s)\n repo \n get Get restic repositories\n```\nThe `velero config server` command will create the following resources:\n\n```\nNamespace\nDeployment\nbackups.velero.io\nbackupstoragelocations.velero.io\ndeletebackuprequests.velero.io\ndownloadrequests.velero.io\npodvolumebackups.velero.io\npodvolumerestores.velero.io\nresticrepositories.velero.io\nrestores.velero.io\nschedules.velero.io\nserverstatusrequests.velero.io\nvolumesnapshotlocations.velero.io\n```\n\nNote: Velero will maintain the `velero server` command run by the Velero pod, which starts the Velero server deployment.\n\n2) `velero backup-location`\nCommands/flags for backup locations.\n\n```\n set\n --default string sets the default backup storage location (default \"default\") (NEW, -- was `server --default-backup-storage-location; could be set as an annotation on the BSL)\n --credentials mapStringString sets the name of the corresponding credentials secret for a provider. Format is provider:credentials-secret-name. (NEW)\n --cacert-file mapStringString configuration to use for creating a secret containing a custom certificate for an S3 location of a plugin provider. Format is provider:path-to-file. (NEW)\n\n create NAME [flags]\n --default Sets this new location to be the new default backup location. Default is false. (NEW) \n --access-mode access mode for the backup storage location. Valid values are ReadWrite,ReadOnly (default ReadWrite)\n --backup-sync-period 0s how often to ensure all Velero backups in object storage exist as Backup API objects in the cluster. Optional. Set this to 0s to disable sync\n --bucket string name of the object storage bucket where backups should be stored. Required.\n --config mapStringString configuration to use for creating a backup storage location. Format is key1=value1,key2=value2 (was also in `velero install --backup-location-config`). Required for Azure.\n --provider string provider name for backup storage. Required.\n --label-columns stringArray a comma-separated list of labels to be displayed as columns\n --labels mapStringString labels to apply to the backup storage location\n --prefix string prefix under which all Velero data should be stored within the bucket. Optional.\n --provider string name of the backup storage provider (e.g. aws, azure, gcp)\n --show-labels show labels in the last column\n --credentials mapStringString sets the name of the corresponding credentials secret for a provider. Format is provider:credentials-secret-name. (NEW)\n --cacert-file mapStringString configuration to use for creating a secret containing a custom certificate for an S3 location of a plugin provider. Format is provider:path-to-file. (NEW)\n\n get Display backup storage locations\n --default displays the current default backup storage location (NEW)\n --label-columns stringArray a comma-separated list of labels to be displayed as columns\n -l, --selector string only show items matching this label selector\n --show-labels show labels in the last column\n\n```\n\n3) `velero snapshot-location`\nCommands/flags for snapshot locations.\n\n```\n set\n --default mapStringString sets the list of unique volume providers and default volume snapshot location (provider1:location-01,provider2:location-02,...) (NEW, -- was `server --default-volume-snapshot-locations; could be set as an annotation on the VSL) \n --credentials mapStringString sets the list of name of the corresponding credentials secret for providers. Format is (provider1:credentials-secret-name1,provider2:credentials-secret-name2,...) (NEW)\n\n create NAME [flags]\n --default Sets these new locations to be the new default snapshot locations. Default is false. (NEW)\n --config mapStringString configuration to use for creating a volume snapshot location. Format is key1=value1,key2=value2 (was also in `velero install --`snapshot-location-config`). Required.\n --provider string provider name for volume storage. Required.\n --label-columns stringArray a comma-separated list of labels to be displayed as columns\n --labels mapStringString labels to apply to the volume snapshot location\n --provider string name of the volume snapshot provider (e.g. aws, azure, gcp)\n --show-labels show labels in the last column\n --credentials mapStringString sets the list of name of the corresponding credentials secret for providers. Format is (provider1:credentials-secret-name1,provider2:credentials-secret-name2,...) (NEW)\n\n get Display snapshot locations\n --default list of unique volume providers and default volume snapshot location (provider1:location-01,provider2:location-02,...) (NEW -- was `server --default-volume-snapshot-locations`))\n```\n\n4) `velero plugin`\nConfiguration for plugins.\n\n```\n add stringArray IMAGES [flags] - add plugin container images to install into the Velero Deployment\n\n get get information for all plugins on the velero server (was `get`)\n --timeout duration maximum time to wait for plugin information to be reported (default 5s)\n\n remove Remove a plugin [NAME | IMAGE] \n\n set\n --credentials-file mapStringString configuration to use for creating a secret containing the AIM credentials for a plugin provider. Format is provider:path-to-file. (was `secret-file`)\n --no-secret flag indicating if a secret should be created. Must be used as confirmation if create --secret-file is not provided. Optional. (MOVED FROM install -- not sure we need it?)\n --sa-annotations mapStringString annotations to add to the Velero ServiceAccount for GKE. Add iam.gke.io/gcp-service-account=[GSA_NAME]@[PROJECT_NAME].iam.gserviceaccount.com for workload identity. Optional. Format is key1=value1,key2=value2\n```\n\n#### Example\n\nConsidering this proposal, let's consider what a high-level documentation for getting Velero ready to do backups could look like for Velero users:\n\nAfter installing the Velero CLI:\n```\nvelero config server [flags] (required)\nvelero config restic [flags]\nvelero plugin add IMAGES [flags] (add/config provider plugins)\nvelero backup-location/snapshot-location create NAME [flags] (run `velero plugin --get` to see what kind of plugins are available; create locations)\nvelero backup/restore/schedule create/get/delete NAME [flags]\n```\n\nThe above recipe-style documentation should highlight 1) the main components of Velero, and, 2) the relationship/dependency between the main components\n\n### Deprecation\n\n#### Timeline\n\nIn order to maintain compatibility with the current Velero version for a sufficient amount of time, and give users a chance to upgrade any install scripts they might have, we will keep the current `velero install` command in parallel with the new commands until the next major Velero version, which will be Velero 2.0. In the mean time, ia deprecation warning will be added to the `velero install` command. \n\n#### Commands/flags deprecated or moved\n\n##### Velero Install \n`velero install (DEPRECATED)`\n\nFlags moved to...\n\n...`velero config server`:\n```\n --image string image to use for the Velero and restic server pods. Optional. (default \"velero/velero:latest\")\n --label-columns stringArray a comma-separated list of labels to be displayed as columns\n --pod-annotations mapStringString annotations to add to the Velero and restic pods. Optional. Format is key1=value1,key2=value2\n --show-labels show labels in the last column\n --pod-cpu-limit string CPU limit for Velero pod. A value of \"0\" is treated as unbounded. Optional. (default \"1000m\")\n --pod-cpu-request string CPU request for Velero pod. A value of \"0\" is treated as unbounded. Optional. (default \"500m\")\n --pod-mem-limit string memory limit for Velero pod. A value of \"0\" is treated as unbounded. Optional. (default \"256Mi\")\n --pod-mem-request string memory request for Velero pod. A value of \"0\" is treated as unbounded. Optional. (default \"128Mi\")\n```\n\n...`velero config restic`\n```\n --default-prune-frequency duration how often 'restic prune' is run for restic repositories by default. Optional.\n --pod-cpu-limit string CPU limit for restic pod. A value of \"0\" is treated as unbounded. Optional. (default \"0\")\n --pod-cpu-request string CPU request for restic pod. A value of \"0\" is treated as unbounded. Optional. (default \"0\")\n --pod-mem-limit string memory limit for restic pod. A value of \"0\" is treated as unbounded. Optional. (default \"0\")\n --pod-mem-request string memory request for restic pod. A value of \"0\" is treated as unbounded. Optional. (default \"0\")\n```\n\n...`backup-location create`\n```\n --backup-location-config mapStringString configuration to use for the backup storage location. Format is key1=value1,key2=value2\n --bucket string name of the object storage bucket where backups should be stored\n --prefix string prefix under which all Velero data should be stored within the bucket. Optional.\n```\n\n...`snapshot-location create`\n```\n --snapshot-location-config mapStringString configuration to use for the volume snapshot location. Format is key1=value1,key2=value2\n```\n\n...both `backup-location create` and `snapshot-location create`\n```\n --provider string provider name for backup and volume storage\n```\n\n...`plugin`\n```\n --plugins stringArray Plugin container images to install into the Velero Deployment\n --sa-annotations mapStringString annotations to add to the Velero ServiceAccount. Add iam.gke.io/gcp-service-account=[GSA_NAME]@[PROJECT_NAME].iam.gserviceaccount.com for workload identity. Optional. Format is key1=value1,key2=value2\n --no-secret flag indicating if a secret should be created. Must be used as confirmation if --secret-file is not provided. Optional.\n --secret-file string (renamed `credentials-file`) file containing credentials for backup and volume provider. If not specified, --no-secret must be used for confirmation. Optional.\n```\n\nFlags to deprecate:\n```\n --no-default-backup-location flag indicating if a default backup location should be created. Must be used as confirmation if --bucket or --provider are not provided. Optional.\n --use-volume-snapshots whether or not to create snapshot location automatically. Set to false if you do not plan to create volume snapshots via a storage provider. (default true)\n --wait wait for Velero deployment to be ready. Optional.\n --use-restic (obsolete since now we have `velero config restic`)\n```\n\n##### Velero Server\n\nThese flags will be moved to under `velero config server`:\n\n`velero server --default-backup-storage-location (DEPRECATED)` changed to `velero backup-location set --default`\n\n`velero server --default-volume-snapshot-locations (DEPRECATED)` changed to `velero snapshot-location set --default`\n\nThe value for these flags will be stored as annotations.\n\n## Detailed Design\n\n#### Handling CA certs\n\nIn anticipation of a new configuration implementation to handle custom CA certs (as per design doc https://github.com/vmware-tanzu/velero/blob/main/design/custom-ca-support.md), a new flag `velero storage-location create/set --cacert-file mapStringString` is proposed. It sets the configuration to use for creating a secret containing a custom certificate for an S3 location of a plugin provider. Format is provider:path-to-file.\n\nSee discussion https://github.com/vmware-tanzu/velero/pull/2259#discussion_r384700723 for more clarification.\n\n#### Renaming \"provider\" to \"location-plugin\"\n\nAs part of this change, we should change to use the term `location-plugin` instead of `provider`. The reasoning: in practice, we usually have 1 plugin per provider, and if there is an implementation for both object store and volume snapshotter for that provider, it will all be contained in the same plugin. When we handle plugins, we follow this logic. In other words, there's a plugin name (ex: `velero.io/aws`) and it can contain implementations of kind `ObjectStore` and/or `VolumeSnapshotter`.\n\nBut when we handle BSL or VSL (and the CLI commands/flags that configure them), we use the term `provider`, which can cause ambiguity as if that is a kind of thing different from a plugin. If the plugin is the \"thing\" that contains the implementation for the desired provider, we should make it easier for the user to guess that and change BackupStorageLocation/VolumeSnapshotLocation `Spec.Provider` field to be called `Spec.Location-Plugin` and all related CLI command flags to `location-plugin`, and update the docs accordingly.\n\nThis change will require a CRD version bump and deprecation cycle.\n\n#### GitOps Compatibility\n\nTo maintain compatibility with gitops practices, each of the new commands will generate `yaml` output that can be stored in source control.\n\nFor content examples, please refer to the files here:\n\nhttps://github.com/carlisia/velero/tree/c-cli-design/design/CLI/PoC\n\nNote: actual `yaml` file names are defined by the user.\n\n`velero config server` - base/deployment.yaml\n\n`velero config restic` - overlays/plugins/restic.yaml\n\n`velero backup-location create` - base/backupstoragelocations.yaml\n\n`velero snapshot-location create` - base/volumasnapshotlocations.yaml\n\n`velero plugin add velero/velero-plugin-for-aws:v1.0.1` - overlays/plugins/aws-plugin.yaml\n\n`velero plugin add velero/velero-plugin-for-microsoft-azure:v1.0.1` - overlay/plugins/azure-plugin.yaml\n\nThese resources can be deployed/deleted using the included kustomize setup and running:\n\n```\nkubectl apply -k design/CLI/PoC/overlays/plugins/\n\nkubectl delete -k design/CLI/PoC/overlays/plugins/\n```\n\nNote: All CRDs, including the `ResticRepository`, may continue to be deployed at startup as it is now, or together with their respective instantiation.\n\n\n#### Changes to startup behavior\n\nTo recap, this proposal redesigns the Velero CLI to make `velero install` obsolete, and instead breaks down the installation and configuration into separate commands. These are the major highlights:\n\n- Plugins will only be installed separately via `velero plugin add`\n- BSL/VSL will be continue to be configured separately, and now each will have an associated secret\n\nSince each BSL/VSL will have its own association with a secret, the user will no longer need to upload a new secret whenever changing to, or adding, a BSL/VSL for a provider that is different from the one in use. This will be done at setup time. This will make it easier to support any number of BSL/VSL combinations, with different providers each.\n\nThe user will start up the Velero server on a cluster by using the command `velero config server`. This will create the Velero deployment resource with default values or values overwritten with flags, create the Velero CRDs, and anything else that is not specific to plugins or BSL/VSL.\n\nThe Velero server will start up, verify that the deployment is running, that all CRDs were found, and log a message that it is waiting for a BSL to be configured. at this point, other operations, such as configuring restic, will be allowed. Velero should keep track of its status, ie, if it is ready to create backups or not. This could be a field `ServerStatus` added to `ServerStatusRequest`. Possible values could be [ready|waiting]. \"ready\" would mean there is at least 1 valid BSL, and \"waiting\" would be anything but that.\n\nWhen adding/configuring a BSL or VSL, we will allow creating locations, and continuously verify if there is a corresponding, valid plugin. When a valid match is found, mark the BSL/VSL as \"ready\". This would require adding a field to the BSL/VSL, or using the existing `Phase` field, and keep track of its status, possibly: [ready|waiting].\n\nWith the first approach: the server would transition into \"ready\" (to create backups) as soon as there is one BSL. It would require a set sequence of actions, ie, first install the plugin, only then the user can successfully configure a BSL.\n\nWith the second approach, the Velero server would continue looping and checking all existing BSLs for at least 1 with a \"ready\" status. Once it found that, it would set itself to \"ready\" also.\n\nAnother new behavior that must be added: the server needs to identify when there no longer exists a valid BSL. At this point, it should change its status from \"ready\" to one that indicates it is not ready, maybe \"waiting\". With the first approach above, this would mean checking if there is still at least one BSL. With the second approach, it would require checking the status of all BSLs to find at least one with the status of \"ready\".\n\nAs it is today, a valid VSL would not be required to create backups, unless the backup included a PV.\n\nTo make it easier for the user to identify if their Velero server is ready to create backups or not, a `velero status` command should be added. This issue has been created some time ago for this purpose: https://github.com/vmware-tanzu/velero/issues/1094.\n\n## Alternatives Considered\n\nIt seems that the vast majority of tools document their usage with `kubectl` and `yaml` files to install and configure their Kubernetes resources. Many of them also make use of Helm, and to a lesser extent some of them have their own CLI tools. \n\nAmongst the tools that have their own CLI, not enough examples were found to establish a clear pattern of usage. It seems the most relevant priority should be to have output in `yaml` format.\n\nAny set of `yaml` files can also be arranged to use with Kustomize by creating/updating resources, and patching them using Kustomize functionalities.\n\nThe way the Velero commands were arranged in this proposal with the ability to output corresponding `yaml` files, and the included Kustomize examples, makes it in line with the widely used practices for installation and configuration.\n\nSome CLI tools do not document their usage with Kustomize, one could assume it is because anyone with knowledge of Kustomize and `yaml` files would know how to use it.\n\nHere are some examples:\n\nhttps://github.com/jetstack/kustomize-cert-manager-demo\n\nhttps://github.com/istio/installer/tree/master/kustomize\n\nhttps://github.com/weaveworks/flagger/tree/master/kustomize\n\nhttps://github.com/jpeach/contour/tree/1c575c772e9fd747fba72ae41ab99bdae7a01864/kustomize (RFC)\n\n## Security Considerations\n\nN/A\n" }, { "path": "design/graph-manifest.md", "content": "# Object Graph Manifest for Velero\n\n## Abstract\n\nOne to two sentences that describes the goal of this proposal and the problem being solved by the proposed change.\nThe reader should be able to tell by the title, and the opening paragraph, if this document is relevant to them.\n\nCurrently, Velero does not have a complete manifest of everything in the backup, aside from the backup tarball itself.\nThis change introduces a new data structure to be stored with a backup in object storage which will allow for more efficient operations in reporting of what a backup contains.\nAdditionally, this manifest should enable advancements in Velero's features and architecture, enabling dry-run support, concurrent backup and restore operations, and reliable restoration of complex applications.\n\n## Background\n\nRight now, Velero backs up items one at a time, sorted by API Group and namespace.\nIt also restores items one at a time, using the restoreResourcePriorities flag to indicate which order API Groups should have their objects restored first.\nWhile this does work currently, it presents challenges for more complex applications that have their dependencies in the form of a graph rather than strictly linear.\n\nFor example, Cluster API clusters are a set of complex Kubernetes objects that require that the \"root\" objects are restored first, before their \"leaf\" objects.\nIf a Cluster that a ClusterResourceSetBinding refers to does not exist, then a restore of the CAPI cluster will fail.\n\nAdditionally, Velero does not have a reliable way to communicate what objects will be affected in a backup or restore operation without actually performing the operation.\nThis complicates dry-run tasks, because a user must simply perform the action without knowing what will be touched.\nIt also complicates allowing backups and restores to run in parallel, because there is currently no way to know if a single Kubernetes object is included in multiple backups or restores, which can lead to unreliability, deadlocking, and race conditions were Velero made to be more concurrent today.\n\n## Goals\n\n- Introduce a manifest data structure that defines the contents of a backup.\n- Store the manifest data into object storage alongside existing backup data.\n\n## Non Goals\n\nThis proposal seeks to enable, but not define, the following.\n\n- Implementing concurrency beyond what already exists in Velero.\n- Implementing a dry-run feature.\n- Implementing a new restore ordering procedure.\n\nWhile the data structure should take these scenarios into account, they will not be implemented alongside it.\n\n## High-Level Design\n\nTo uniquely identify a Kubernetes object within a cluster or backup, the following fields are sufficient:\n\n- API Group and Version (example: backup.velero.io/v1)\n- Namespace \n- Name\n- Labels\n\nThis criteria covers the majority of Velero's inclusion or exclusion logic.\nHowever, some additional fields enable further use cases.\n\n- Owners, which are other Kubernetes objects that have some relationship to this object. They may be strict or soft dependencies.\n- Annotations, which provide extra metadata about the object that might be useful for other programs to consume.\n- UUID generated by Kubernetes. This is useful in defining Owner relationships, providing a single, immutable key to find an object. This is _not_ considered at restore time, only internally for defining links.\n\nAll of this information already exists within a Velero backup's tarball of resources, but extracting such data is inefficient.\nThe entire tarball must be downloaded and extracted, and then JSON within parsed to read labels, owners, annotations, and a UUID.\nThe rest of the information is encoded in the file system structure within the Velero backup tarball.\nWhile doable, this is heavyweight in terms of time and potentially memory.\n\nInstead, this proposal suggests adding a new manifest structure that is kept alongside the backup tarball.\nThis structure would contain the above fields only, and could be used to perform inclusion/exclusion logic on a backup, select a resource from within a backup, and do set operations over backup or restore contents to identify overlapping resources.\n\nHere are some use cases that this data structure should enable, that have been difficult to implement prior to its existence:\n\n- A dry-run operation on backup, informing the user what would be selected if they were to perform the operation.\n A manifest could be created and saved, allowing for a user to do a dry-run, then accept it to perform the backup.\n Restore operations can be treated similarly.\n- Efficient, non-overlapping parallelization of backup and restore operations.\n By building or reading a manifest before performing a backup or restore, Velero can determine if there are overlapping resources.\n If there are no overlaps, the operations can proceed in parallel.\n If there are overlaps, the operations can proveed serially.\n- Graph-based restores for non-linear dependencies.\n Not all resources in a Kubernetes cluster can be defined in a strict, linear way.\n They may have multiple owners, and writing BackupItemActions or RestoreItemActions to simply return a chain of owners is not an efficient way to support the many Kubernetes operators/controllers being written.\n Instead, by having a manifest with enough information, Velero can build a discrete list that ensures dependencies are restored before their dependents, with less input from plugin authors.\n\n## Detailed Design\n\nThe Manifest data structure would look like this, in Go type structure:\n\n```golang\n// NamespacedItems maps a given namespace to all of its contained items.\ntype NamespacedItems map[string]*Item\n\n// APIGroupNamespaces maps an API group/version to a map of namespaces and their items.\ntype KindNamespaces map[string]NamespacedItems\n\ntype Manifest struct {\n // Kinds holds the top level map of all resources in a manifest.\n Kinds KindNamespaces\n\n // Index is used to look up an individual item quickly based on UUID.\n // This enables fetching owners out of the maps more efficiently at the cost of memory space.\n Index map[string]*Item\n}\n\n\n// Item represents a Kubernetes resource within a backup based on it's selectable criteria.\n// It is not the whole Kubernetes resource as retrieved from the API server, but rather a collection of important fields needed for filtering.\ntype Item struct {\n // Kubernetes API group which this Item belongs to.\n // Could be a core resource, or a CustomResourceDefinition.\n APIGroup string\n\n // Version of the APIGroup that the Item belongs to.\n APIVersion string\n\n // Kubernetes namespace which contains this item.\n // Empty string for cluster-level resource.\n Namespace string\n\n // Item's given name.\n Name string\n\n // Map of labels that the Item had at backup time.\n Labels map[string]string\n\n // Map of annotations that the Item had at Backup time.\n // Useful for plugins that may decide to process only Items with specific annotations.\n Annotations map[string]string\n\n // Owners is a list of UUIDs to other items that own or refer to this item.\n Owners []string\n\n // Manifest is a pointer to the Manifest in which this object is contained.\n // Useful for getting access to things like the Manifest.Index map.\n Manifest *Manifest\n}\n```\n\nIn addition to the new types, the following Go interfaces would be provided for convenience.\n\n```golang\ntype Itermer interface {\n // Returns the Item as a string, following the current Velero backup version 1.1.0 tarball structure format.\n // ///.json\n String() string\n\n // Owners returns a slice of realized Items that own or refer to the current Item.\n // Useful for building out a full graph of Items to restore.\n // Will use the UUIDs in Item.Owners to look up the owner Items in the Manifest.\n Owners() []*Item\n\n // Kind returns the Kind of an object, which is a combination of the APIGroup and APIVersion.\n // Useful for verifying the needed CustomResourceDefinition exists before actually restoring this Item.\n Kind() *Item\n\n // Children returns a slice of all Items that refer to this item as an Owner.\n Children() []*Items\n}\n\n// This error type is being created in order to make reliable sentinel errors.\n// See https://dave.cheney.net/2019/06/10/constant-time for more details.\ntype ManifestError string\n\nfunc (e ManifestError) Error() string {\n return string(e)\n}\n\nconst ItemAlreadyExists = ManifestError(\"item already exists in manifest\")\n\ntype Manifester interface {\n // Set returns the entire list of resources as a set of strings (using Itemer.String).\n // This is useful for comparing two manifests and determining if they have any overlapping resources.\n // In the future, when implementing concurrent operations, this can be used as a sanity check to ensure resources aren't being backed up or restored by two operations at once.\n Set() sets.String\n\n // Adds an item to the appropriate APIGroup and Namespace within a Manifest\n // Returns (true, nil) if the Item is successfully added to the Manifest,\n // Returns (false, ItemAlreadyExists) if the Item is already in the Manifest.\n Add(*Item) (bool, error)\n}\n```\n\n### Serialization\n\nThe entire `Manifest` should be serialized into the `manifest.json` file within the object storage for a single backup.\nIt is possible that this file could also be compressed for space efficiency.\n\n### Memory Concerns\n\nBecause the `Manifest` is holding a minimal amount of data, memory sizes should not be a concern for most clusters.\nTODO: Document known limits on API group name, resource name, and kind name character limits.\n\n## Security Considerations\n\nIntroducing this manifest does not increase the attack surface of Velero, as this data is already present in the existing backups.\nStoring the manifest.json file next to the existing backup data in the object storage does not change access patterns.\n\n## Compatibility\n\nThe introduction of this file should trigger Velero backup version 1.2.0, but it will not interfere with Velero versions that do not support the `Manifest` as the file will be additive.\nIn time, this file will replace the `-resource-list.json.gz` file, but for compatibility the two will appear side by side.\n\nWhen first implemented, Velero should simply build the `Manifest` as it backs up items, and serialize it at the end.\nAny logic changes that rely on the `Manifest` file must be introduced with their own design document, with their own compatibility concerns.\n\n## Implementation\n\nThe `Manifest` object will _not_ be implemented as a Kubernetes CustomResourceDefinition, but rather one of Velero's own internal constructs.\n\nImplementation for the data structure alone should be minimal - the types will need to be defined in a `manifest` package.\nThen, the backup process should create a `Manifest`, passing it to the various `*Backuppers` in the `backup` package.\nThese methods will insert individual `Items` into the `Manifest`.\nFinally, logic should be added to the `persistence` package to ensure that the new `manifest.json` file is uploadable and allowed.\n\n## Alternatives Considered\n\nNone so far.\n\n## Open Issues\n\n- When should compatibility with the `-resource-list.json.gz` file be dropped?\n- What are some good test case Kubernetes resources and controllers to try this out with?\nCluster API seems like an obvious choice, but are there others?\n- Since it is not implemented as a CustomResourceDefinition, how can a `Manifest` be retained so that users could issue a dry-run command, then perform their actual desire operation?\nCould it be stored in Velero's temp directories?\nNote that this is making Velero itself more stateful.\n" }, { "path": "design/new-prepost-backuprestore-plugin-hooks.md", "content": "# Pre-Backup, Post-Backup, Pre-Restore, and Post-Restore Action Plugin Hooks\n\n## Abstract\n\nVelero should provide a way to trigger actions before and after each backup and restore.\n**Important**: These proposed plugin hooks are fundamentally different from the existing plugin hooks, BackupItemAction and RestoreItemAction, which are triggered per resource item during backup and restore, respectively.\nThe proposed plugin hooks are to be executed only once: pre-backup (before backup starts), post-backup (after the backup is completed and uploaded to object storage, including volumes snapshots), pre-restore (before restore starts) and post-restore (after the restore is completed, including volumes are restored).\n\n### PreBackup and PostBackup Actions\n\nFor the backup, the sequence of events of Velero backup are the following (these sequence depicted is prior upcoming changes for [upload progress #3533](https://github.com/vmware-tanzu/velero/issues/3533) ):\n\n```\nNew Backup Request\n |--> Validation of the request\n |--> Set Backup Phase \"In Progress\"\n | --> Start Backup\n | --> Discover all Plugins\n |--> Check if Backup Exists\n |--> Backup all K8s Resource Items\n |--> Perform all Volumes Snapshots\n |--> Final Backup Phase is determined\n |--> Persist Backup and Logs on Object Storage\n```\nWe propose the pre-backup and post-backup plugin hooks to be executed in this sequence:\n\n```\nNew Backup Request\n |--> Validation of the request\n |--> Set Backup Phase \"In Progress\"\n | --> Start Backup\n | --> Discover all Plugins\n |--> Check if Backup Exists\n |--> **PreBackupActions** are executed, logging actions on existent backup log file\n |--> Backup all K8s Resource Items\n |--> Perform all Volumes Snapshots\n |--> Final Backup Phase is determined\n |--> Persist Backup and logs on Object Storage\n |--> **PostBackupActions** are executed, logging to its own file\n```\nThese plugin hooks will be invoked:\n\n- PreBackupAction: plugin actions are executed after the backup object is created and validated but before the backup is being processed, more precisely _before_ function [c.backupper.Backup](https://github.com/vmware-tanzu/velero/blob/74476db9d791fa91bba0147eac8ec189820adb3d/pkg/controller/backup_controller.go#L590). If the PreBackupActions return an err, the backup object is not processed and the Backup phase will be set as `FailedPreBackupActions`.\n\n- PostBackupAction: plugin actions are executed after the backup is finished and persisted, more precisely _after_ function [c.runBackup](https://github.com/vmware-tanzu/velero/blob/74476db9d791fa91bba0147eac8ec189820adb3d/pkg/controller/backup_controller.go#L274).\n\nThe proposed plugin hooks will execute actions that will have statuses on their own:\n`Backup.Status.PreBackupActionsStatuses` and `Backup.Status.PostBackupActionsStatuses` which will be an array of a proposed struct `ActionStatus` with PluginName, StartTimestamp, CompletionTimestamp and Phase.\n\n### PreRestore and PostRestore Actions\n\nFor the restore, the sequence of events of Velero restore are the following (these sequence depicted is prior upcoming changes for [upload progress #3533](https://github.com/vmware-tanzu/velero/issues/3533) ):\n```\nNew Restore Request\n |--> Validation of the request\n |--> Checks if restore is from a backup or a schedule\n |--> Fetches backup\n |--> Set Restore Phase \"In Progress\"\n |--> Start Restore\n |--> Discover all Plugins\n |--> Download backup file to temp\n |--> Fetch list of volumes snapshots\n |--> Restore K8s items, including PVs\n |--> Final Restore Phase is determined\n |--> Persist Restore logs on Object Storage\n```\nWe propose the pre-restore and post-restore plugin hooks to be executed in this sequence:\n```\nNew Restore Request\n |--> Validation of the request\n |--> Checks if restore is from a backup or a schedule\n |--> Fetches backup\n |--> Set Restore Phase \"In Progress\"\n |--> Start Restore\n |--> Discover all Plugins\n |--> Download backup file to temp\n |--> Fetch list of volumes snapshots\n |--> **PreRestoreActions** are executed, logging actions on existent backup log file\n |--> Restore K8s items, including PVs\n |--> Final Restore Phase is determined\n |--> Persist Restore logs on Object Storage\n |--> **PostRestoreActions** are executed, logging to its own file\n```\n\nThese plugin hooks will be invoked:\n\n- PreRestoreAction: plugin actions are executed after the restore object is created and validated and before the backup object is fetched, more precisely in function `runValidatedRestore` _after_ function [info.backupStore.GetBackupVolumeSnapshots](https://github.com/vmware-tanzu/velero/blob/7c75cd6cf854064c9a454e53ba22cc5881d3f1f0/pkg/controller/restore_controller.go#L460). If the PreRestoreActions return an err, the restore object is not processed and the Restore phase will be set a `FailedPreRestoreActions`.\n \n- PostRestoreAction: plugin actions are executed after the restore finishes processing all items and volumes snapshots are restored and logs persisted, more precisely in function `processRestore` _after_ setting [`restore.Status.CompletionTimestamp`](https://github.com/vmware-tanzu/velero/blob/7c75cd6cf854064c9a454e53ba22cc5881d3f1f0/pkg/controller/restore_controller.go#L273).\n\nThe proposed plugin hooks will execute actions that will have statuses on their own:\n`Restore.Status.PreRestoreActionsStatuses` and `Restore.Status.PostRestoreActionsStatuses` which will be an array of a proposed struct `ActionStatus` with PluginName, StartTimestamp, CompletionTimestamp and Phase.\n\n## Background\n\nIncreasingly, Velero is employed for workload migrations across different Kubernetes clusters.\nUsing Velero for migrations requires an atomic operation involving a Velero backup on a source cluster followed by a Velero restore on a destination cluster.\n\nIt is common during these migrations to perform many actions inside and outside Kubernetes clusters.\n**Attention**: these actions are not per resource item, but they are actions to be executed _once_ before and/or after the migration itself (remember, migration in this context is Velero Backup + Velero Restore).\n\nOne important use case driving this proposal is migrating stateful workloads at scale across different clusters/storage backends.\nToday, Velero's Restic integration is the response for such use cases, but there are some limitations:\n\n- Quiesce/unquiesce workloads: Pod hooks are useful for quiescing/unquiescing workloads, but platform engineers often do not have the luxury/visibility/time/knowledge to go through each pod in order to add specific commands to quiesce/unquiesce workloads.\n- Orphan PVC/PV pairs: PVCs/PVs that do not have associated running pods are not backed up and consequently, are not migrated.\n\nAiming to address these two limitations, and separate from this proposal, we would like to write a Velero plugin that takes advantage of the proposed Pre-Backup plugin hook. This plugin will be executed _once_ (not per resource item) prior backup. It will scale down the applications setting `.spec.replicas=0` to all deployments, statefulsets, daemonsets, replicasets, etc. and will start a small-footprint staging pod that will mount all PVC/PV pairs. Similarly, we would like to write another plugin that will utilize the proposed Post-Restore plugin hook. This plugin will unquiesce migrated applications by killing the staging pod and reinstating original `.spec.replicas` values after the Velero restore is completed.\n\nOther examples of plugins that can use the proposed plugin hooks are:\n\n- PostBackupAction: trigger a Velero Restore after a successful Velero backup (and complete the migration operation).\n- PreRestoreAction: pre-expand the cluster's capacity via Cluster API to avoid starvation of cluster resources before the restore.\n- PostRestoreAction: call actions to be performed outside Kubernetes clusters, such as configure a global load balancer (GLB) that enables the new cluster.\n\nThe post backup actions will be executed after the backup is uploaded (persisted) on the disk. The logs of post-backup actions will be uploaded on the disk once the actions are completed.\n\nThe post restore actions will be executed after the restore is uploaded (persisted) on the disk. The logs of post-restore actions will be uploaded on the disk once the actions are completed.\n\nThis design seeks to provide missing extension points. This proposal's scope is to only add the new plugin hooks, not the plugins themselves.\n\n## Goals\n\n- Provide PreBackupAction, PostBackupAction, PreRestoreAction, and PostRestoreAction APIs for plugins to implement.\n- Update Velero backup and restore creation logic to invoke registered PreBackupAction and PreRestoreAction plugins before processing the backup and restore respectively.\n- Update Velero backup and restore complete logic to invoke registered PostBackupAction and PostRestoreAction plugins the objects are uploaded on disk.\n- Create one `ActionStatus` struct to keep track of execution of the plugin hooks. This struct has PluginName, StartTimestamp, CompletionTimestamp and Phase.\n- Add sub statuses for the plugins on Backup object: `Backup.Status.PreBackupActionsStatuses` and `Backup.Status.PostBackupActionsStatuses`. They will be flagged as optional and nullable. They will be populated only each plugin registered for the PreBackup and PostBackup hooks, respectively.\n- Add sub statuses for the plugins on Restore object: `Backup.Status.PreRestoreActionsStatuses` and `Backup.Status.PostRestoreActionsStatuses`. They will be flagged as optional and nullable. They will be populated only each plugin registered for the PreRestore and PostRestore hooks, respectively.\n- that will be populated optionally if Pre/Post Backup/Restore.\n\n## Non-Goals\n\n- Specific implementations of the PreBackupAction, PostBackupAction, PreRestoreAction and PostRestoreAction API beyond test cases.\n- For migration specific actions (Velero Backup + Velero Restore), add disk synchronization during the validation of the Restore (making sure the newly created backup will show during restore)\n\n## High-Level Design\n\nThe Velero backup controller package will be modified for `PreBackupAction` and `PostBackupAction`.\n\nThe PreBackupAction plugin API will resemble the BackupItemAction plugin hook design, but with the fundamental difference that it will receive only as input the Velero `Backup` object created.\nIt will not receive any resource list items because the backup is not yet running at that stage.\nIn addition, the `PreBackupAction` interface will only have an `Execute()` method since the plugin will be executed once per Backup creation, not per item.\n\nThe Velero backup controller will be modified so that if there are any PreBackupAction plugins registered, they will be\n\nThe PostBackupAction plugin API will resemble the BackupItemAction plugin design, but with the fundamental difference that it will receive only as input the Velero `Backup` object without any resource list items.\nBy this stage, the backup has already been executed, with items backed up and volumes snapshots processed and persisted.\nThe `PostBackupAction` interface will only have an `Execute()` method since the plugin will be executed only once per Backup, not per item.\n\nIf there are any PostBackupAction plugins registered, they will be executed after the backup is finished and persisted, more precisely _after_ function [c.runBackup](https://github.com/vmware-tanzu/velero/blob/74476db9d791fa91bba0147eac8ec189820adb3d/pkg/controller/backup_controller.go#L274).\n\nThe Velero restore controller package will be modified for `PreRestoreAction` and `PostRestoreAction`.\n\nThe PreRestoreAction plugin API will resemble the RestoreItemAction plugin design, but with the fundamental difference that it will receive only as input the Velero `Restore` object created.\nIt will not receive any resource list items because the restore has not yet been running at that stage.\nIn addition, the `PreRestoreAction` interface will only have an `Execute()` method since the plugin will be executed only once per Restore creation, not per item.\n\nThe Velero restore controller will be modified so that if there are any PreRestoreAction plugins registered, they will be executed after the restore object is created and validated and before the backup object is fetched, more precisely in function `runValidatedRestore` _after_ function [info.backupStore.GetBackupVolumeSnapshots](https://github.com/vmware-tanzu/velero/blob/7c75cd6cf854064c9a454e53ba22cc5881d3f1f0/pkg/controller/restore_controller.go#L460). If the PreRestoreActions return an err, the restore object is not processed and the Restore phase will be set a `FailedPreRestoreActions`.\n\nThe PostRestoreAction plugin API will resemble the RestoreItemAction plugin design, but with the fundamental difference that it will receive only as input the Velero `Restore` object without any resource list items.\nAt this stage, the restore has already been executed.\nThe `PostRestoreAction` interface will only have an `Execute()` method since the plugin will be executed only once per Restore, not per item.\n\nIf any PostRestoreAction plugins are registered, they will be executed after the restore finishes processing all items and volumes snapshots are restored and logs persisted, more precisely in function `processRestore` _after_ setting [`restore.Status.CompletionTimestamp`](https://github.com/vmware-tanzu/velero/blob/7c75cd6cf854064c9a454e53ba22cc5881d3f1f0/pkg/controller/restore_controller.go#L273).\n\n## Detailed Design\n\n### New Status struct\n\nTo keep the status of the plugins, we propose the following struct:\n\n```go\ntype ActionStatus struct {\n // PluginName is the name of the registered plugin\n // retrieved by the PluginManager as id.Name\n // +optional\n // +nullable\n PluginName string `json:\"pluginName,omitempty\"`\n\n // StartTimestamp records the time the plugin started.\n // +optional\n // +nullable\n StartTimestamp *metav1.Time `json:\"startTimestamp,omitempty\"`\n\n // CompletionTimestamp records the time the plugin was completed.\n // +optional\n // +nullable\n CompletionTimestamp *metav1.Time `json:\"completionTimestamp,omitempty\"`\n\n // Phase is the current state of the Action.\n // +optional\n // +nullable\n Phase ActionPhase `json:\"phase,omitempty\"`\n}\n\n// ActionPhase is a string representation of the lifecycle phase of an action being executed by a plugin\n// of a Velero backup.\n// +kubebuilder:validation:Enum=InProgress;Completed;Failed\ntype ActionPhase string\n\nconst (\n // ActionPhaseInProgress means the action has being executed\n ActionPhaseInProgress ActionPhase = \"InProgress\"\n\n // ActionPhaseCompleted means the action finished successfully\n ActionPhaseCompleted ActionPhase = \"Completed\"\n\n // ActionPhaseFailed means the action failed\n ActionPhaseFailed ActionPhase = \"Failed\"\n)\n\n```\n\n### Backup Status of the Plugins\n\nThe `Backup` Status section will have the follow:\n\n```go\ntype BackupStatus struct {\n (...)\n // PreBackupActionsStatuses contains information about the pre backup plugins's execution.\n // Note that this information is will be only populated if there are prebackup plugins actions\n // registered\n // +optional\n // +nullable\n PreBackupActionsStatuses *[]ActionStatus `json:\"preBackupActionsStatuses,omitempty\"`\n\n // PostBackupActionsStatuses contains information about the post backup plugins's execution.\n // Note that this information is will be only populated if there are postbackup plugins actions\n // registered\n // +optional\n // +nullable\n PostBackupActionsStatuses *[]ActionStatus `json:\"postBackupActionsStatuses,omitempty\"`\n\n}\n```\n\n### Restore Status of the Plugins\n\nThe `Restore` Status section will have the follow:\n\n```go\ntype RestoreStatus struct {\n (...)\n // PreRestoreActionsStatuses contains information about the pre Restore plugins's execution.\n // Note that this information is will be only populated if there are preRestore plugins actions\n // registered\n // +optional\n // +nullable\n PreRestoreActionsStatuses *[]ActionStatus `json:\"preRestoreActionsStatuses,omitempty\"`\n\n // PostRestoreActionsStatuses contains information about the post restore plugins's execution.\n // Note that this information is will be only populated if there are postrestore plugins actions\n // registered\n // +optional\n // +nullable\n PostRestoreActionsStatuses *[]ActionStatus `json:\"postRestoreActionsStatuses,omitempty\"`\n\n}\n```\n\n### New Backup and Restore Phases\n\n#### New Backup Phase: FailedPreBackupActions\n\nIn case the PreBackupActionsStatuses has at least one `ActionPhase` = `Failed`, it means al least one of the plugins returned an error and consequently, the backup will not move forward. The final status of the Backup object will be set as `FailedPreBackupActions`:\n\n```go\n\n// BackupPhase is a string representation of the lifecycle phase\n// of a Velero backup.\n// +kubebuilder:validation:Enum=New;FailedValidation;FailedPreBackupActions;InProgress;Uploading;UploadingPartialFailure;Completed;PartiallyFailed;Failed;Deleting\ntype BackupPhase string\n\nconst (\n\n (...)\n\n // BackupPhaseFailedPreBackupActions means one or more the Pre Backup Actions has failed\n // and therefore backup will not run.\n BackupPhaseFailedPreBackupActions BackupPhase = \"FailedPreBackupActions\"\n\n (...)\n)\n\n```\n\n#### New Restore Phase FailedPreRestoreActions\n\nIn case the PreRestoreActionsStatuses has at least one `ActionPhase` = `Failed`, it means al least one of the plugins returned an error and consequently, the restore will not move forward. The final status of the Restore object will be set as `FailedPreRestoreActions`:\n\n```go\n\n// RestorePhase is a string representation of the lifecycle phase\n// of a Velero restore\n// +kubebuilder:validation:Enum=New;FailedValidation;FailedPreRestoreActions;InProgress;Completed;PartiallyFailed;Failed\ntype RestorePhase string\n\nconst (\n\n (...)\n\n // RestorePhaseFailedPreRestoreActions means one or more the Pre Restore Actions has failed\n // and therefore restore will not run.\n RestorePhaseFailedPreRestoreActions BackupPhase = \"FailedPreRestoreActions\"\n\n (...)\n)\n\n```\n\n### New Interface types\n\n#### PreBackupAction\n\nThe `PreBackupAction` interface is as follows:\n\n```go\n// PreBackupAction provides a hook into the backup process before it begins.\ntype PreBackupAction interface {\n // Execute the PreBackupAction plugin providing it access to the Backup that\n // is being executed\n Execute(backup *api.Backup) error\n}\n```\n\n`PreBackupAction` will be defined in `pkg/plugin/velero/pre_backup_action.go`.\n\n#### PostBackupAction\n\nThe `PostBackupAction` interface is as follows:\n\n```go\n// PostBackupAction provides a hook into the backup process after it completes.\ntype PostBackupAction interface {\n // Execute the PostBackupAction plugin providing it access to the Backup that\n // has been completed\n Execute(backup *api.Backup) error\n}\n```\n\n`PostBackupAction` will be defined in `pkg/plugin/velero/post_backup_action.go`.\n\n#### PreRestoreAction\n\nThe `PreRestoreAction` interface is as follows:\n\n```go\n// PreRestoreAction provides a hook into the restore process before it begins.\ntype PreRestoreAction interface {\n // Execute the PreRestoreAction plugin providing it access to the Restore that\n // is being executed\n Execute(restore *api.Restore) error\n}\n```\n\n`PreRestoreAction` will be defined in `pkg/plugin/velero/pre_restore_action.go`.\n\n#### PostRestoreAction\n\nThe `PostRestoreAction` interface is as follows:\n\n```go\n// PostRestoreAction provides a hook into the restore process after it completes.\ntype PostRestoreAction interface {\n // Execute the PostRestoreAction plugin providing it access to the Restore that\n // has been completed\n Execute(restore *api.Restore) error\n}\n```\n\n`PostRestoreAction` will be defined in `pkg/plugin/velero/post_restore_action.go`.\n\n### New BackupStore Interface Methods\n\nFor the persistence of the logs originated from the PostBackup and PostRestore plugins, create two additional methods on `BackupStore` interface:\n\n```go\ntype BackupStore interface {\n (...)\n PutPostBackuplog(backup string, log io.Reader) error\n PutPostRestoreLog(backup, restore string, log io.Reader) error\n (...)\n```\n\nThe implementation of these new two methods will go hand-in-hand with the changes of uploading phases rebase.\n\n\n### Generate Protobuf Definitions and Client/Servers\n\nIn `pkg/plugin/proto`, add the following:\n\n1. Protobuf definitions will be necessary for PreBackupAction in `pkg/plugin/proto/PreBackupAction.proto`.\n\n```protobuf\nmessage PreBackupActionExecuteRequest {\n ...\n}\n\nservice PreBackupAction {\n rpc Execute(PreBackupActionExecuteRequest) returns (Empty)\n}\n```\n\nOnce these are written, then a client and server implementation can be written in `pkg/plugin/framework/pre_backup_action_client.go` and `pkg/plugin/framework/pre_backup_action_server.go`, respectively.\n\n2. Protobuf definitions will be necessary for PostBackupAction in `pkg/plugin/proto/PostBackupAction.proto`.\n\n```protobuf\nmessage PostBackupActionExecuteRequest {\n ...\n}\n\nservice PostBackupAction {\n rpc Execute(PostBackupActionExecuteRequest) returns (Empty)\n}\n```\n\nOnce these are written, then a client and server implementation can be written in `pkg/plugin/framework/post_backup_action_client.go` and `pkg/plugin/framework/post_backup_action_server.go`, respectively.\n\n3. Protobuf definitions will be necessary for PreRestoreAction in `pkg/plugin/proto/PreRestoreAction.proto`.\n\n```protobuf\nmessage PreRestoreActionExecuteRequest {\n ...\n}\n\nservice PreRestoreAction {\n rpc Execute(PreRestoreActionExecuteRequest) returns (Empty)\n}\n```\n\nOnce these are written, then a client and server implementation can be written in `pkg/plugin/framework/pre_restore_action_client.go` and `pkg/plugin/framework/pre_restore_action_server.go`, respectively.\n\n4. Protobuf definitions will be necessary for PostRestoreAction in `pkg/plugin/proto/PostRestoreAction.proto`.\n\n```protobuf\nmessage PostRestoreActionExecuteRequest {\n ...\n}\n\nservice PostRestoreAction {\n rpc Execute(PostRestoreActionExecuteRequest) returns (Empty)\n}\n```\n\nOnce these are written, then a client and server implementation can be written in `pkg/plugin/framework/post_restore_action_client.go` and `pkg/plugin/framework/post_restore_action_server.go`, respectively.\n\n### Restartable Delete Plugins\n\nSimilar to the `RestoreItemAction` and `BackupItemAction` plugins, restartable processes will need to be implemented (with the difference that there is no `AppliedTo()` method).\n\nIn `pkg/plugin/clientmgmt/`, add\n\n1. `restartable_pre_backup_action.go`, creating the following unexported type:\n\n```go\ntype restartablePreBackupAction struct {\n key kindAndName\n sharedPluginProcess RestartableProcess\n}\n\nfunc newRestartablePreBackupAction(name string, sharedPluginProcess RestartableProcess) *restartablePreBackupAction {\n // ...\n}\n\nfunc (r *restartablePreBackupAction) getPreBackupAction() (velero.PreBackupAction, error) {\n // ...\n}\n\nfunc (r *restartablePreBackupAction) getDelegate() (velero.PreBackupAction, error) {\n // ...\n}\n\n// Execute restarts the plugin's process if needed, then delegates the call.\nfunc (r *restartablePreBackupAction) Execute(input *velero.PreBackupActionInput) (error) {\n // ...\n}\n```\n\n2. `restartable_post_backup_action.go`, creating the following unexported type:\n\n```go\ntype restartablePostBackupAction struct {\n key kindAndName\n sharedPluginProcess RestartableProcess\n}\n\nfunc newRestartablePostBackupAction(name string, sharedPluginProcess RestartableProcess) *restartablePostBackupAction {\n // ...\n}\n\nfunc (r *restartablePostBackupAction) getPostBackupAction() (velero.PostBackupAction, error) {\n // ...\n}\n\nfunc (r *restartablePostBackupAction) getDelegate() (velero.PostBackupAction, error) {\n // ...\n}\n\n// Execute restarts the plugin's process if needed, then delegates the call.\nfunc (r *restartablePostBackupAction) Execute(input *velero.PostBackupActionInput) (error) {\n // ...\n}\n```\n\n3. `restartable_pre_restore_action.go`, creating the following unexported type:\n\n```go\ntype restartablePreRestoreAction struct {\n key kindAndName\n sharedPluginProcess RestartableProcess\n}\n\nfunc newRestartablePreRestoreAction(name string, sharedPluginProcess RestartableProcess) *restartablePreRestoreAction {\n // ...\n}\n\nfunc (r *restartablePreRestoreAction) getPreRestoreAction() (velero.PreRestoreAction, error) {\n // ...\n}\n\nfunc (r *restartablePreRestoreAction) getDelegate() (velero.PreRestoreAction, error) {\n // ...\n}\n\n// Execute restarts the plugin's process if needed, then delegates the call.\nfunc (r *restartablePreRestoreAction) Execute(input *velero.PreRestoreActionInput) (error) {\n // ...\n}\n```\n\n4. `restartable_post_restore_action.go`, creating the following unexported type:\n\n```go\ntype restartablePostRestoreAction struct {\n key kindAndName\n sharedPluginProcess RestartableProcess\n}\n\nfunc newRestartablePostRestoreAction(name string, sharedPluginProcess RestartableProcess) *restartablePostRestoreAction {\n // ...\n}\n\nfunc (r *restartablePostRestoreAction) getPostRestoreAction() (velero.PostRestoreAction, error) {\n // ...\n}\n\nfunc (r *restartablePostRestoreAction) getDelegate() (velero.PostRestoreAction, error) {\n // ...\n}\n\n// Execute restarts the plugin's process if needed, then delegates the call.\nfunc (r *restartablePostRestoreAction) Execute(input *velero.PostRestoreActionInput) (error) {\n // ...\n}\n```\n\n### Plugin Manager Changes\n\nAdd the following methods to the `Manager` interface in `pkg/plugin/clientmgmt/manager.go`:\n\n```go\ntype Manager interface {\n ...\n // Get PreBackupAction returns a PreBackupAction plugin for name.\n GetPreBackupAction(name string) (PreBackupAction, error)\n\n // Get PreBackupActions returns the all PreBackupAction plugins.\n GetPreBackupActions() ([]PreBackupAction, error)\n\n // Get PostBackupAction returns a PostBackupAction plugin for name.\n GetPostBackupAction(name string) (PostBackupAction, error)\n\n // GetPostBackupActions returns the all PostBackupAction plugins.\n GetPostBackupActions() ([]PostBackupAction, error)\n\n // Get PreRestoreAction returns a PreRestoreAction plugin for name.\n GetPreRestoreAction(name string) (PreRestoreAction, error)\n\n // Get PreRestoreActions returns the all PreRestoreAction plugins.\n GetPreRestoreActions() ([]PreRestoreAction, error)\n\n // Get PostRestoreAction returns a PostRestoreAction plugin for name.\n GetPostRestoreAction(name string) (PostRestoreAction, error)\n\n // GetPostRestoreActions returns the all PostRestoreAction plugins.\n GetPostRestoreActions() ([]PostRestoreAction, error)\n\n}\n```\n\n`GetPreBackupAction` and `GetPreBackupActions` will invoke the `restartablePreBackupAction` implementations.\n`GetPostBackupAction` and `GetPostBackupActions` will invoke the `restartablePostBackupAction` implementations.\n`GetPreRestoreAction` and `GetPreRestoreActions` will invoke the `restartablePreRestoreAction` implementations.\n`GetPostRestoreAction` and `GetPostRestoreActions` will invoke the `restartablePostRestoreAction` implementations.\n\n### How to invoke the Plugins\n\n#### Getting Pre/Post Backup Actions\n\nGetting Actions on `backup_controller.go` in `runBackup`:\n\n```go\n\n backupLog.Info(\"Getting PreBackup actions\")\n preBackupActions, err := pluginManager.GetPreBackupActions()\n if err != nil {\n return err\n }\n\n backupLog.Info(\"Getting PostBackup actions\")\n postBackupActions, err := pluginManager.GetPostBackupActions()\n if err != nil {\n return err\n }\n```\n\n#### Pre Backup Actions Plugins\n\nCalling the Pre Backup actions:\n\n```go\n for _, preBackupAction := range preBackupActions {\n err := preBackupAction.Execute(backup.Backup)\n if err != nil {\n backup.Backup.Status.Phase = velerov1api.BackupPhaseFailedPreBackupActions\n return err\n }\n }\n```\n\n#### Post Backup Actions Plugins\n\nCalling the Post Backup actions:\n\n```go\n for _, postBackupAction := range postBackupActions {\n err := postBackupAction.Execute(backup.Backup)\n if err != nil {\n postBackupLog.Error(err)\n }\n }\n```\n\n#### Getting Pre/Post Restore Actions\n\nGetting Actions on `restore_controller.go` in `runValidatedRestore`:\n\n```go\n\n restoreLog.Info(\"Getting PreRestore actions\")\n preRestoreActions, err := pluginManager.GetPreRestoreActions()\n if err != nil {\n return errors.Wrap(err, \"error getting pre-restore actions\")\n }\n\n restoreLog.Info(\"Getting PostRestore actions\")\n postRestoreActions, err := pluginManager.GetPostRestoreActions()\n if err != nil {\n return errors.Wrap(err, \"error getting post-restore actions\")\n }\n```\n\n#### Pre Restore Actions Plugins\n\nCalling the Pre Restore actions:\n\n```go\n for _, preRestoreAction := range preRestoreActions {\n err := preRestoreAction.Execute(restoreReq.Restore)\n if err != nil {\n restoreReq.Restore.Status.Phase = velerov1api.RestorePhaseFailedPreRestoreActions\n return errors.Wrap(err, \"error executing pre-restore action\")\n }\n }\n```\n\n#### Post Restore Actions Plugins\n\nCalling the Post Restore actions:\n\n```go\n for _, postRestoreAction := range postRestoreActions {\n err := postRestoreAction.Execute(restoreReq.Restore)\n if err != nil {\n postRestoreLog.Error(err.Error())\n }\n }\n```\n\n### Giving the User the Option to Skip the Execution of the Plugins\n\nVelero plugins are loaded as init containers. If plugins are unloaded, they trigger a restart of the Velero controller.\nNot mentioning if one plugin does get loaded for any reason (i.e., docker hub image pace limit), Velero does not start.\nIn other words, the constant load/unload of plugins can disrupt the Velero controller, and they cannot be the only method to run the actions from these plugins selectively.\nAs part of this proposal, we want to give the velero user the ability to skip the execution of the plugins via annotations on the Velero CR backup and restore objects.\nIf one of these exists, the given plugin, referenced below as `plugin-name`, will be skipped.\n\nBackup Object Annotations:\n\n```\n /prebackup=skip\n /postbackup=skip\n```\n\nRestore Object Annotations:\n\n```\n /prerestore=skip\n /postrestore=skip\n```\n\n## Alternatives Considered\n\nAn alternative to these plugin hooks is to implement all the pre/post backup/restore logic _outside_ Velero.\nIn this case, one would need to write an external controller that works similar to what [Konveyor Crane](https://github.com/konveyor/mig-controller/blob/master/pkg/controller/migmigration/quiesce.go) does today when quiescing applications.\nWe find this a viable way, but we think that Velero users can benefit from Velero having greater embedded capabilities, which will allow users to write or load plugins extensions without relying on an external components.\n\n## Security Considerations\n\nThe plugins will only be invoked if loaded per a user's discretion.\nIt is recommended to check security vulnerabilities before execution.\n\n## Compatibility\n\nIn terms of backward compatibility, this design should stay compatible with most Velero installations that are upgrading.\nIf plugins are not present, then the backup/restore process should proceed the same way it worked before their inclusion.\n\n## Implementation\n\nThe implementation dependencies are roughly in the order as they are described in the [Detailed Design](#detailed-design) section.\n\n## Open Issues\n" }, { "path": "design/restore-progress.md", "content": "# Restore progress reporting\n\nVelero _Backup_ resource provides real-time progress of an ongoing backup by means of a _Progress_ field in the CR. Velero _Restore_, on the other hand, only shows one of the phases (InProgress, Completed, PartiallyFailed, Failed) of the ongoing restore. In this document, we propose detailed progress reporting for Velero _Restore_. With the introduction of the proposed _Progress_ field, Velero _Restore_ CR will look like:\n\n```yml\napiVersion: velero.io/v1\nkind: Restore\nmetadata:\n name: test-restore\n namespace: velero\nspec:\n [...]\nstatus:\n phase: InProgress\n progress:\n itemsRestored: 100\n totalItems: 140\n```\n\n## Goals\n\n- Enable progress reporting for Velero Restore\n\n## Non Goals\n\n- Estimate time to completion\n\n## Background\n\nThe current _Restore_ CR lets users know whether a restore is in-progress or completed (failed/succeeded). While this basic piece of information is useful to the end user, there seems to be room for improvement in the user experience. The _Restore_ CR can show detailed progress in terms of the number of resources restored so far and the total number of resources to be restored. This will be particularly useful for restores that run for a longer duration of time. Such progress reporting already exists for Velero _Backup_. This document proposes similar implementation for Velero _Restore_.\n\n## High-Level Design\n\nWe propose to divide the restore process in two steps. The first step will collect all the items to be restored from the backup tarball. It will apply the label selector and include/exclude rules on the resources / items and store them (preserving the priority order) in an in-memory data structure. The second step will read the collected items and restore them. \n\n## Detailed Design\n\n### Progress struct\n\nA new struct will be introduced to store progress information:\n\n```go\ntype RestoreProgress struct {\n TotalItems int `json:\"totalItems,omitempty`\n ItemsRestored int `json:\"itemsRestored,omitempty`\n}\n```\n\n`RestoreStatus` will include the above struct:\n\n```go\ntype RestoreStatus struct {\n [...]\n\n Progress *RestoreProgress `json:\"progress,omitempty\"`\n}\n```\n\n### Modifications to restore.go\n\nCurrently, the restore process works by looping through the resources in the backup tarball and restoring them one-by-one in the same pass:\n\n```go\nfunc (ctx *context) execute(...) {\n [...]\n\n for _, resource := range getOrderedResources(...) {\n [...]\n\n for namespace, items := range resourceList.ItemsByNamespace {\n [...]\n\n for _, item := range items {\n [...]\n\n // restore item here\n w, e := restoreItem(...)\n }\n }\n }\n}\n```\n\nWe propose to remove the call to `restoreItem()` in the inner most loop and instead store the item in a data structure. Once all the items are collected, we loop through the array of collected items and make a call to `restoreItem()`:\n\n```go\nfunc (ctx *context) getOrderedResourceCollection(...) {\n collectedResources := []restoreResource\n for _, resource := range getOrderedResources(...) {\n [...]\n\n for namespace, items := range resourceList.ItemsByNamespace {\n [...]\n collectedResource := restoreResource{}\n for _, item := range items {\n [...]\n\n // store item in a data structure\n collectedResource.itemsByNamespace[originalNamespace] = append(collectedResource.itemsByNamespace[originalNamespace], item)\n }\n }\n collectedResources.append(collectedResources, collectedResource)\n }\n return collectedResources\n}\n\nfunc (ctx *context) execute(...) {\n [...]\n\n // get all items\n resources := ctx.getOrderedResourceCollection(...)\n\n for _, resource := range resources {\n [...]\n\n for _, items := range resource.itemsByNamespace {\n [...]\n\n for _, item := range items {\n [...]\n\n // restore the item\n w, e := restoreItem(...)\n }\n }\n }\n\n [...]\n}\n```\n\nWe introduce two new structs to hold the collected items:\n\n```go\ntype restoreResource struct {\n resource string\n itemsByNamespace map[string][]restoreItem\n totalItems int\n}\n\ntype restoreItem struct {\n targetNamespace string\n name string\n}\n```\n\nEach group resource is represented by `restoreResource`. The map `itemsByNamespace` is indexed by `originalNamespace`, and the values are list of `items` in the original namespace. `totalItems` is simply the count of all items which are present in the nested map of namespace and items. It is updated every time an item is added to the map. Each item represented by `restoreItem` has `name` and the resolved `targetNamespace`.\n\n### Calculating progress\n\nThe total number of items can be calculated by simply adding the number of total items present in the map of all resources.\n\n```go\ntotalItems := 0\n\nfor _, resource := range collectedResources {\n\ttotalItems += resource.totalItems\n}\n```\n\nThe additional items returned by the plugins will still be discovered at the time of plugin execution. The number of `totalItems` will be adjusted to include such additional items. As a result, the number of total items is expected to change whenever plugins execute:\n\n```go\n i := 0\n for _, resource := range resources {\n [...]\n\n for _, items := range resource.itemsByNamespace {\n [...]\n\n for _, item := range items {\n [...]\n\n // restore the item\n w, e := restoreItem(...)\n\t\ti++\n\t\t// calculate the actual count of resources\n\t\tactualTotalItems := len(ctx.restoredItems) + (totalItems - i)\n }\n }\n }\n```\n\n### Updating progress \n\nThe updates to the `progress` field in the CR can be sent on a channel as soon as an item is restored. A goroutine receiving update on that channel can make an `Update()` call to update the _Restore_ CR. This will require us to pass an instance of `RestoresGetter` to the `kubernetesRestorer` struct.\n\n\n## Alternatives Considered\n\nAs an alternative, we have considered an approach which doesn't divide the restore process in two steps. \n\nWith that approach, the total number of items will be read from the Backup CR. We will keep three counters, `totalItems`, `skippedItems` and `restoredItems`:\n\n```yml\nstatus:\n phase: InProgress\n progress:\n totalItems: 100\n skippedItems: 20\n restoredItems: 79\n```\n\nThis approach doesn't require us to find the number of total items beforehand.\n\n## Security Considerations\n\nOmitted\n\n## Compatibility\n\nOmitted\n\n## Implementation\n\nTBD\n\n## Open Issues\n\nhttps://github.com/vmware-tanzu/velero/issues/21" }, { "path": "design/upload-progress.md", "content": "# Upload Progress Monitoring\n\nVolume snapshotter plugin are used by Velero to take snapshots of persistent volume contents. \nDepending on the underlying storage system, those snapshots may be available to use immediately, \nthey may be uploaded to stable storage internally by the plugin or they may need to be uploaded after\nthe snapshot has been taken. We would like for Velero to continue on to the next part of the backup as quickly\nas possible but we would also like the backup to not be marked as complete until it is a usable backup. We'd also\neventually like to bring the control of upload under the control of Velero and allow the user to make decisions\nabout the ultimate destination of backup data independent of the storage system they're using.\n\n\n\n## Examples\nAWS - AWS snapshots return quickly, but are then uploaded in the background and cannot be used until EBS moves\nthe data into S3 internally.\n\nvSphere - The vSphere plugin takes a local snapshot and then the vSphere plugin uploads the data to S3. The local\nsnapshot is usable before the upload completes.\n\nRestic - Does not go through the volume snapshot path. Restic backups will block Velero progress until completed.\n\n## Goals\n\n- Enable monitoring of operations that continue after snapshotting operations have completed\n- Keep non-usable backups (upload/persistence has not finished) from appearing as completed\n- Minimize change to volume snapshot and BackupItemAction plugins\n\n## Non-goals\n- Unification of BackupItemActions and VolumeSnapshotters\n\n## Models\n\n### Internal configuration and management\nIn this model, movement of the snapshot to stable storage is under the control of the snapshot\nplugin. Decisions about where and when the snapshot gets moved to stable storage are not\ndirectly controlled by Velero. This is the model for the current VolumeSnapshot plugins.\n\n### Velero controlled management\nIn this model, the snapshot is moved to external storage under the control of Velero. This\nenables Velero to move data between storage systems. This also allows backup partners to use\nVelero to snapshot data and then move the data into their backup repository.\n\n## Backup phases\n\nVelero currently has backup phases \"InProgress\" and \"Completed\". The backup moves to the Completed\nphase when all of the volume snapshots have completed and the Kubernetes metadata has been written\ninto the object store. However, the actual data movement may be happening in the background\nafter the backup has been marked \"Completed\". The backup is not actually a stable backup until\nthe data has been persisted properly. In some cases (e.g. AWS) the backup cannot be restored from\nuntil the snapshots have been persisted.\n\nOnce the snapshots have been taken, however, it is possible for additional backups to be made without\ninterference. Waiting until all data has been moved before starting the next backup will\nslow the progress of the system without adding any actual benefit to the user.\n\nA new backup phase, \"Uploading\" will be introduced. When a backup has entered this phase, Velero\nis free to start another backup. The backup will remain in the \"Uploading\" phase until all data\nhas been successfully moved to persistent storage. The backup will not fail once it reaches\nthis phase, it will continuously retry moving the data. If the backup is deleted (cancelled), the plugins will\nattempt to delete the snapshots and stop the data movement - this may not be possible with all\nstorage systems.\n\n### State progression\n\n![image](UploadFSM.png)\n### New\nWhen a backup request is initially created, it is in the \"New\" phase. \n\nThe next state is either \"InProgress\" or \"FailedValidation\"\n\n### FailedValidation\nIf the backup request is incorrectly formed, it goes to the \"FailedValidation\" phase and terminates\n\n### InProgress\nWhen work on the backup begins, it moves to the \"InProgress\" phase. It remains in the \"InProgress\"\nphase until all pre/post execution hooks have been executed, all snapshots have been taken and the\nKubernetes metadata and backup info is safely written to the object store plugin.\n\nIn the current implementation, Restic backups will move data during the \"InProgress\" phase.\nIn the future, it may be possible to combine a snapshot with a Restic (or equivalent) backup which\nwould allow for data movement to be handled in the \"Uploading\" phase,\n\nThe next phase is either \"Completed\", \"Uploading\", \"Failed\" or \"PartiallyFailed\". Backups which \nwould have a final phase of \"Completed\" or \"PartiallyFailed\" may move to the \"Uploading\" state.\nA backup which will be marked \"Failed\" will go directly to\nthe \"Failed\" phase. Uploads may continue in the background for snapshots that were taken by a \"Failed\"\nbackup, but no progress will not be monitored or updated. When a \"Failed\" backup is deleted, all snapshots\nwill be deleted and at that point any uploads still in progress should be aborted.\n\n### Uploading (new)\nThe \"Uploading\" phase signifies that the main part of the backup, including snapshotting has completed successfully\nand uploading is continuing. In the event of an error during uploading, the phase will change to \nUploadingPartialFailure. On success, the phase changes to Completed. The backup cannot be\nrestored from when it is in the Uploading state.\n\n### UploadingPartialFailure (new)\nThe \"UploadingPartialFailure\" phase signifies that the main part of the backup, including snapshotting has completed,\nbut there were partial failures either during the main part or during the uploading. The backup cannot be\nrestored from when it is in the UploadingPartialFailure state.\n\n### Failed\nWhen a backup has had fatal errors it is marked as \"Failed\" This backup cannot be restored from.\n\n### Completed\nThe \"Completed\" phase signifies that the backup has completed, all data has been transferred to stable storage\nand the backup is ready to be used in a restore. When the Completed phase has been reached it is safe\nto remove any of the items that were backed up.\n\n### PartiallyFailed\nThe \"PartiallyFailed\" phase signifies that the backup has completed and at least part of the backup is usable.\nRestoration from a PartiallyFailed backup will not result in a complete restoration but pieces may be available.\n\n## Workflow\n\nWhen a BackupAction is executed, any SnapshotItemAction or VolumeSnapshot plugins will return snapshot IDs.\nThe plugin should be able to provide status on\nthe progress for the snapshot and handle cancellation of the upload if the snapshot is deleted.\nIf the plugin is restarted, the snapshot ID should remain valid.\n\nWhen all snapshots have been taken and Kubernetes resources have been persisted to the ObjectStorePlugin\nthe backup will either have fatal errors or will be at least partially usable.\n\nIf the backup has fatal errors it will move to the \"Failed\" state and finish. If a backup fails, the upload will not be\ncancelled but it will not be monitored either. For backups in any phase, all snapshots will be deleted when the backup\nis deleted. Plugins will cancel any data movement and\nremove snapshots and other associated resources when the VolumeSnapshotter DeleteSnapshot method or \nDeleteItemAction Execute method is called.\n\nVelero will poll the plugins for status on the snapshots when the backup exits the \"InProgress\" phase and\nhas no fatal errors.\n\nIf any snapshots are not complete, the backup will move to either Uploading or UploadingPartialFailure or Failed.\n\nPost-snapshot operations may take a long time and Velero and its plugins may be restarted during \nthis time. Once a backup has moved into the Uploading or UploadingPartialFailure phase, another \nbackup may be started.\n\nWhile in the Uploading or UploadingPartialFailure phase, the snapshots and backup items will be periodically polled.\nWhen all of the snapshots and backup items have reported success, the backup will move to the Completed or \nPartiallyFailed phase, depending on whether the backup was in the Uploading or UploadingPartialFailure phase.\n\nThe Backup resources will not be written to object storage until the backup has entered a final phase: \nCompleted, Failed or PartialFailure\n## Reconciliation of InProgress backups\n\nInProgress backups will not have a `velero-backup.json` present in the object store. During reconciliation, backups which\ndo not have a `velero-backup.json` object in the object store will be ignored.\n\n## Plugin API changes\n\n### UploadProgress struct\n\n type UploadProgress struct {\n completed bool // True when the operation has completed, either successfully or with a failure\n err error // Set when the operation has failed\n itemsCompleted, itemsToComplete int64 // The number of items that have been completed and the items to complete\n // For a disk, an item would be a byte and itemsToComplete would be the\n // total size to transfer (may be less than the size of a volume if\n // performing an incremental) and itemsCompleted is the number of bytes\n // transferred. On successful completion, itemsCompleted and itemsToComplete\n // should be the same\n started, updated time.Time // When the upload was started and when the last update was seen. Not all\n // systems retain when the upload was begun, return Time 0 (time.Unix(0, 0))\n // if unknown.\n }\n\n### VolumeSnapshotter changes\n\nA new method will be added to the VolumeSnapshotter interface (details depending on plugin versioning spec)\n\n UploadProgress(snapshotID string) (UploadProgress, error)\n\nUploadProgress will report the current status of a snapshot upload. This should be callable at any time after the snapshot\nhas been taken. In the event a plugin is restarted, if the snapshotID continues to be valid it should be possible to\nretrieve the progress.\n\n`error` is set if there is an issue retrieving progress. If the snapshot is has encountered an error during the upload,\nthe error should be return in UploadProgress and error should be nil.\n\n### SnapshotItemAction plugin\n\nCurrently CSI snapshots and the Velero Plugin for vSphere are implemented as BackupItemAction plugins. The majority of\nBackupItemAction plugins do not take snapshots or upload data so rather than modify BackupItemAction we introduce a new\nplugins, SnapshotItemAction. SnapshotItemAction will be used in place of BackupItemAction for\nthe CSI snapshots and the Velero Plugin for vSphere and will return a snapshot ID in addition to the item itself.\n\nThe SnapshotItemAction plugin identifier as well as the Item and Snapshot ID will be stored in the \n`-itemsnapshots.json.gz`. When checking for progress, this info will be used to select the appropriate\nSnapshotItemAction plugin to query for progress.\n\n_NotApplicable_ should only be returned if the SnapshotItemAction plugin should not be handling the item. If the\nSnapshotItemAction plugin should handle the item but, for example, the item/snapshot ID cannot be found to report progress, a\nUploadProgress struct with the error set appropriately (in this case _NotFound_) should be returned.\n\n // SnapshotItemAction is an actor that snapshots an individual item being backed up (it may also do other\n operations on the item that is returned).\n \n type SnapshotItemAction interface {\n \t// AppliesTo returns information about which resources this action should be invoked for.\n \t// A BackupItemAction's Execute function will only be invoked on items that match the returned\n \t// selector. A zero-valued ResourceSelector matches all resources.\n \tAppliesTo() (ResourceSelector, error)\n \n \t// Execute allows the ItemAction to perform arbitrary logic with the item being backed up,\n \t// including mutating the item itself prior to backup. The item (unmodified or modified)\n \t// should be returned, along with an optional slice of ResourceIdentifiers specifying\n \t// additional related items that should be backed up.\n \tExecute(item runtime.Unstructured, backup *api.Backup) (runtime.Unstructured, snapshotID string,\n \t []ResourceIdentifier, error)\n \t\n \t// Progress \n \tProgress(input *SnapshotItemProgressInput) (UploadProgress, error)\n }\n \n // SnapshotItemProgressInput contains the input parameters for the SnapshotItemAction's Progress function.\n type SnapshotItemProgressInput struct {\n \t// Item is the item that was stored in the backup\n \tItem runtime.Unstructured\n \t// SnapshotID is the snapshot ID returned by SnapshotItemAction\n \tSnapshotID string\n \t// Backup is the representation of the restore resource processed by Velero.\n \tBackup *velerov1api.Backup\n }\n\n\n## Changes in Velero backup format\n\nNo changes to the existing format are introduced by this change. A `-itemsnapshots.json.gz` file will be \nadded that contains the items and snapshot IDs returned by ItemSnapshotAction. Also, the creation of the \n`velero-backup.json` object will not occur until the backup moves to one of the terminal phases (_Completed_, \n_PartiallyFailed_, or _Failed_). Reconciliation should ignore backups that do not have a `velero-backup.json` object.\n\nThe cluster that is creating the backup will have the Backup resource present and will be able to manage the backup\nbefore the backup completes.\n\nIf the Backup resource is removed (e.g. Velero is uninstalled) before a backup completes and writes its \n`velero-backup.json` object, the other objects in the object store for the backup will be effectively orphaned. This \ncan currently happen but the current window is much smaller.\n\n### `-itemsnapshots.json.gz`\nThe itemsnapshots file is similar to the existing `-itemsnapshots.json.gz` Each snapshot taken via\nSnapshotItemAction will have a JSON record in the file. Exact format TBD.\n\n## CSI snapshots\n\nFor systems such as EBS, a snapshot is not available until the storage system has transferred the snapshot to\nstable storage. CSI snapshots expose the _readyToUse_ state that, in the case of EBS, indicates that the snapshot\nhas been transferred to durable storage and is ready to be used. The CSI BackupItemProgress.Progress method will\npoll that field and when completed, return completion.\n\n## vSphere plugin\n\nThe vSphere Plugin for Velero uploads snapshots to S3 in the background. This is also a BackupItemAction plugin,\nit will check the status of the Upload records for the snapshot and return progress.\n\n## Backup workflow changes\n\nThe backup workflow remains the same until we get to the point where the `velero-backup.json` object is written.\nAt this point, we will queue the backup to a finalization go-routine. The next backup may then begin. The finalization\nroutine will run across all of the volume snapshots and call the _UploadProgress_ method on each of them. It will\nthen run across all items and call _BackupItemProgress.Progress_ for any that match with a BackupItemProgress.\n\nIf all snapshots and backup items have finished uploading (either successfully or failed), the backup will be completed\nand the backup will move to the appropriate terminal phase and upload the `velero-backup.json` object to the object store\nand the backup will be complete.\n\nIf any of the snapshots or backup items are still being processed, the phase of the backup will be set to the appropriate\nphase (_Uploading_ or _UploadingPartialFailure_). In the event of any of the upload progress checks return an error, the \nphase will move to _UploadingPartialFailure_. The backup will then be requeued and will be rechecked again after some \ntime has passed.\n\n## Restart workflow\nOn restart, the Velero server will scan all Backup resources. Any Backup resources which are in the _InProgress_ phase\nwill be moved to the _Failed_ phase. Any Backup resources in the _Oploading_ or _OploadingPartialFailure_ phase will\nbe treated as if they have been requeued and progress checked and the backup will be requeued or moved to a terminal\nphase as appropriate.\n\n# Implementation tasks\n\nVolumeSnapshotter new plugin APIs \nBackupItemProgress new plugin interface \nNew backup phases \nDefer uploading `velero-backup.json` \nAWS EBS plugin UploadProgress implementation \nUpload monitoring \nImplementation of `-itemsnapshots.json.gz` file \nRestart logic \nChange in reconciliation logic to ignore backups that have not completed \nCSI plugin BackupItemProgress implementation \nvSphere plugin BackupItemProgress implementation (vSphere plugin team) \n\n# Future Fragile/Durable snapshot tracking\nFutures are here for reference, they may change radically when actually implemented.\n\nSome storage systems have the ability to provide different levels of protection for snapshots. These are termed \"Fragile\"\nand \"Durable\". Currently, Velero expects snapshots to be Durable (they should be able to survive the destruction of the\ncluster and the storage it is using). In the future we would like the ability to take advantage of snapshots that are\nFragile. For example, vSphere snapshots are Fragile (they reside in the same datastore as the virtual disk). The Velero\nPlugin for vSphere uses a vSphere local/fragile snapshot to get a consistent snapshot, then uploads the data to S3 to\nmake it Durable. In the current design, upload progress will not be complete until the snapshot is ready to use and\nDurable. It is possible, however, to restore data from a vSphere snapshot before it has been made Durable, and this is a\ncapability we'd like to expose in the future. Other storage systems implement this functionality as well. We will be moving\nthe control of the data movement from the vSphere plugin into Velero.\n\nSome storage system, such as EBS, are only capable of creating Durable snapshots. There is no usable intermediate Fragile stage.\n\nFor a Velero backup, users should be able to specify whether they want a Durable backup or a Fragile backup (Fragile backups\nmay consume less resources, be quicker to restore from and are suitable for things like backing up a cluster before upgrading\nsoftware). We can introduce three snapshot states - Creating, Fragile and Durable. A snapshot would be created with a\ndesired state, Fragile or Durable. When the snapshot reaches the desired or higher state (e.g. request was for Fragile but\nsnapshot went to Durable as on EBS), then the snapshot would be completed.\n" }, { "path": "design/vsv2-design.md", "content": "# Design for VolumeSnapshotter v2 API\n\n## Abstract\nThis design includes the changes to the VolumeSnapshotter api design as required by the [Item Action Progress Monitoring](general-progress-monitoring.md) feature.\nThe VolumeSnapshotter v2 interface will have two new methods.\nIf there are any additional VolumeSnapshotter API changes that are needed in the same Velero release cycle as this change, those can be added here as well.\n\n## Background\nThis API change is needed to facilitate long-running plugin actions that may not be complete when the Execute() method returns.\nThe existing snapshotID returned by CreateSnapshot will be used as the operation ID.\nThis will allow long-running plugin actions to continue in the background while Velero moves on to the next plugin, the next item, etc.\n\n## Goals\n- Allow for VolumeSnapshotter CreateSnapshot() to initiate a long-running operation and report on operation status.\n\n## Non Goals\n- Allowing velero control over when the long-running operation begins.\n\n\n## High-Level Design\nAs per the [Plugin Versioning](plugin-versioning.md) design, a new VolumeSnapshotterv2 plugin `.proto` file will be created to define the GRPC interface.\nv2 go files will also be created in `plugin/clientmgmt/volumesnapshotter` and `plugin/framework/volumesnapshotter`, and a new PluginKind will be created.\nThe velero Backup process will be modified to reference v2 plugins instead of v1 plugins.\nAn adapter will be created so that any existing VolumeSnapshotter v1 plugin can be executed as a v2 plugin when executing a backup.\n\n## Detailed Design\n\n### proto changes (compiled into golang by protoc)\n\nThe v2 VolumeSnapshotter.proto will be like the current v1 version with the following changes:\nThe VolumeSnapshotter service gets two new rpc methods:\n```\nservice VolumeSnapshotter {\n rpc Init(VolumeSnapshotterInitRequest) returns (Empty);\n rpc CreateVolumeFromSnapshot(CreateVolumeRequest) returns (CreateVolumeResponse);\n rpc GetVolumeInfo(GetVolumeInfoRequest) returns (GetVolumeInfoResponse);\n rpc CreateSnapshot(CreateSnapshotRequest) returns (CreateSnapshotResponse);\n rpc DeleteSnapshot(DeleteSnapshotRequest) returns (Empty);\n rpc GetVolumeID(GetVolumeIDRequest) returns (GetVolumeIDResponse);\n rpc SetVolumeID(SetVolumeIDRequest) returns (SetVolumeIDResponse);\n rpc Progress(VolumeSnapshotterProgressRequest) returns (VolumeSnapshotterProgressResponse);\n rpc Cancel(VolumeSnapshotterCancelRequest) returns (google.protobuf.Empty);\n}\n```\nTo support these new rpc methods, we define new request/response message types:\n```\nmessage VolumeSnapshotterProgressRequest {\n string plugin = 1;\n string snapshotID = 2;\n}\n\nmessage VolumeSnapshotterProgressResponse {\n generated.OperationProgress progress = 1;\n}\n\nmessage VolumeSnapshotterCancelRequest {\n string plugin = 1;\n string operationID = 2;\n}\n\n```\nOne new shared message type will be needed, as defined in the v2 BackupItemAction design:\n```\nmessage OperationProgress {\n bool completed = 1;\n string err = 2;\n int64 completed = 3;\n int64 total = 4;\n string operationUnits = 5;\n string description = 6;\n google.protobuf.Timestamp started = 7;\n google.protobuf.Timestamp updated = 8;\n}\n```\n\nA new PluginKind, `VolumeSnapshotterV2`, will be created, and the backup process will be modified to use this plugin kind.\nSee [Plugin Versioning](plugin-versioning.md) for more details on implementation plans, including v1 adapters, etc.\n\n\n## Compatibility\nThe included v1 adapter will allow any existing VolumeSnapshotter plugin to work as expected, with no-op Progress() and Cancel() methods.\n\n## Implementation\nThis will be implemented during the Velero 1.11 development cycle.\n" }, { "path": "examples/README.md", "content": "# Examples\n\nThis directory contains sample YAML config files that can be used for exploring Velero.\n\n* `minio/`: Used in the [Quickstart][0] to set up [Minio][1], a local S3-compatible object storage service. It provides a convenient way to test Velero without tying you to a specific cloud provider.\n\n* `nginx-app/`: A sample nginx app that can be used to test backups and restores.\n\n\n[0]: https://velero.io/docs/main/contributions/minio/\n[1]: https://github.com/minio/minio\n" }, { "path": "examples/minio/00-minio-deployment.yaml", "content": "# Copyright 2017 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n---\napiVersion: v1\nkind: Namespace\nmetadata:\n name: velero\n\n---\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n namespace: velero\n name: minio\n labels:\n component: minio\nspec:\n strategy:\n type: Recreate\n selector:\n matchLabels:\n component: minio\n template:\n metadata:\n labels:\n component: minio\n spec:\n volumes:\n - name: storage\n emptyDir: {}\n - name: config\n emptyDir: {}\n containers:\n - name: minio\n image: minio/minio:latest\n imagePullPolicy: IfNotPresent\n args:\n - server\n - /storage\n - --config-dir=/config\n env:\n - name: MINIO_ACCESS_KEY\n value: \"minio\"\n - name: MINIO_SECRET_KEY\n value: \"minio123\"\n ports:\n - containerPort: 9000\n volumeMounts:\n - name: storage\n mountPath: \"/storage\"\n - name: config\n mountPath: \"/config\"\n\n---\napiVersion: v1\nkind: Service\nmetadata:\n namespace: velero\n name: minio\n labels:\n component: minio\nspec:\n # ClusterIP is recommended for production environments.\n # Change to NodePort if needed per documentation,\n # but only if you run Minio in a test/trial environment, for example with Minikube.\n type: ClusterIP\n ports:\n - port: 9000\n targetPort: 9000\n protocol: TCP\n selector:\n component: minio\n\n---\napiVersion: batch/v1\nkind: Job\nmetadata:\n namespace: velero\n name: minio-setup\n labels:\n component: minio\nspec:\n template:\n metadata:\n name: minio-setup\n spec:\n restartPolicy: OnFailure\n volumes:\n - name: config\n emptyDir: {}\n containers:\n - name: mc\n image: minio/mc:latest\n imagePullPolicy: IfNotPresent\n command:\n - /bin/sh\n - -c\n - \"mc --config-dir=/config alias set velero http://minio:9000 minio minio123 && mc --config-dir=/config mb -p velero/velero\"\n volumeMounts:\n - name: config\n mountPath: \"/config\"\n" }, { "path": "examples/nginx-app/README.md", "content": "# Files\n\nThis directory contains manifests for two versions of a sample Nginx app under the `nginx-example` namespace.\n\n## `base.yaml`\n\nThis is the most basic version of the Nginx app, which can be used to test Velero's backup and restore functionality.\n\n*This can be deployed as is.*\n\n## `with-pv.yaml`\n\nThis sets up an Nginx app that logs to a persistent volume, so that Velero's PV snapshotting functionality can also be tested.\n\n*This requires you to first replace the placeholder value ``.*\n" }, { "path": "examples/nginx-app/base.yaml", "content": "# Copyright 2017 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n---\napiVersion: v1\nkind: Namespace\nmetadata:\n name: nginx-example\n labels:\n app: nginx\n\n---\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n name: nginx-deployment\n namespace: nginx-example\n labels:\n app: nginx\nspec:\n replicas: 2\n selector:\n matchLabels:\n app: nginx\n template:\n metadata:\n labels:\n app: nginx\n spec:\n containers:\n - image: nginx:1.17.6\n name: nginx\n ports:\n - containerPort: 80\n\n---\napiVersion: v1\nkind: Service\nmetadata:\n labels:\n app: nginx\n name: my-nginx\n namespace: nginx-example\nspec:\n ports:\n - port: 80\n targetPort: 80\n selector:\n app: nginx\n type: LoadBalancer\n" }, { "path": "examples/nginx-app/with-pv.yaml", "content": "# Copyright 2017 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n---\napiVersion: v1\nkind: Namespace\nmetadata:\n name: nginx-example\n labels:\n app: nginx\n\n---\nkind: PersistentVolumeClaim\napiVersion: v1\nmetadata:\n name: nginx-logs\n namespace: nginx-example\n labels:\n app: nginx\nspec:\n # Optional:\n # storageClassName: \n accessModes:\n - ReadWriteOnce\n resources:\n requests:\n storage: 50Mi\n\n---\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n name: nginx-deployment\n namespace: nginx-example\nspec:\n replicas: 1\n selector:\n matchLabels:\n app: nginx\n template:\n metadata:\n labels:\n app: nginx\n annotations:\n pre.hook.backup.velero.io/container: fsfreeze\n pre.hook.backup.velero.io/command: '[\"/sbin/fsfreeze\", \"--freeze\", \"/var/log/nginx\"]'\n post.hook.backup.velero.io/container: fsfreeze\n post.hook.backup.velero.io/command: '[\"/sbin/fsfreeze\", \"--unfreeze\", \"/var/log/nginx\"]'\n spec:\n volumes:\n - name: nginx-logs\n persistentVolumeClaim:\n claimName: nginx-logs\n containers:\n - image: nginx:1.17.6\n name: nginx\n ports:\n - containerPort: 80\n volumeMounts:\n - mountPath: \"/var/log/nginx\"\n name: nginx-logs\n readOnly: false\n - image: ubuntu:bionic\n name: fsfreeze\n securityContext:\n privileged: true\n volumeMounts:\n - mountPath: \"/var/log/nginx\"\n name: nginx-logs\n readOnly: false\n command:\n - \"/bin/bash\"\n - \"-c\"\n - \"sleep infinity\"\n\n \n---\napiVersion: v1\nkind: Service\nmetadata:\n labels:\n app: nginx\n name: my-nginx\n namespace: nginx-example\nspec:\n ports:\n - port: 80\n targetPort: 80\n selector:\n app: nginx\n type: LoadBalancer\n" }, { "path": "go.mod", "content": "module github.com/vmware-tanzu/velero\n\ngo 1.25.0\n\nrequire (\n\tcloud.google.com/go/storage v1.57.2\n\tgithub.com/Azure/azure-sdk-for-go/sdk/azcore v1.20.0\n\tgithub.com/Azure/azure-sdk-for-go/sdk/azidentity v1.13.1\n\tgithub.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v5 v5.6.0\n\tgithub.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storage/armstorage v1.8.1\n\tgithub.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.3\n\tgithub.com/aws/aws-sdk-go-v2 v1.24.1\n\tgithub.com/aws/aws-sdk-go-v2/config v1.26.3\n\tgithub.com/aws/aws-sdk-go-v2/credentials v1.16.14\n\tgithub.com/aws/aws-sdk-go-v2/feature/s3/manager v1.15.11\n\tgithub.com/aws/aws-sdk-go-v2/service/ec2 v1.143.0\n\tgithub.com/aws/aws-sdk-go-v2/service/s3 v1.48.0\n\tgithub.com/aws/aws-sdk-go-v2/service/sts v1.26.7\n\tgithub.com/bombsimon/logrusr/v3 v3.0.0\n\tgithub.com/evanphx/json-patch/v5 v5.9.11\n\tgithub.com/fatih/color v1.18.0\n\tgithub.com/gobwas/glob v0.2.3\n\tgithub.com/google/go-cmp v0.7.0\n\tgithub.com/google/uuid v1.6.0\n\tgithub.com/hashicorp/go-hclog v0.14.1\n\tgithub.com/hashicorp/go-plugin v1.6.0\n\tgithub.com/joho/godotenv v1.3.0\n\tgithub.com/kopia/kopia v0.16.0\n\tgithub.com/kubernetes-csi/external-snapshotter/client/v8 v8.2.0\n\tgithub.com/onsi/ginkgo/v2 v2.22.0\n\tgithub.com/onsi/gomega v1.36.1\n\tgithub.com/petar/GoLLRB v0.0.0-20210522233825-ae3b015fd3e9\n\tgithub.com/pkg/errors v0.9.1\n\tgithub.com/prometheus/client_golang v1.23.2\n\tgithub.com/prometheus/client_model v0.6.2\n\tgithub.com/robfig/cron/v3 v3.0.1\n\tgithub.com/sirupsen/logrus v1.9.3\n\tgithub.com/spf13/afero v1.10.0\n\tgithub.com/spf13/cobra v1.8.1\n\tgithub.com/spf13/pflag v1.0.5\n\tgithub.com/stretchr/testify v1.11.1\n\tgithub.com/vmware-tanzu/crash-diagnostics v0.3.7\n\tgo.uber.org/zap v1.27.1\n\tgolang.org/x/mod v0.30.0\n\tgolang.org/x/oauth2 v0.34.0\n\tgolang.org/x/sys v0.40.0\n\tgolang.org/x/text v0.32.0\n\tgoogle.golang.org/api v0.256.0\n\tgoogle.golang.org/grpc v1.79.3\n\tgoogle.golang.org/protobuf v1.36.10\n\tgopkg.in/yaml.v3 v3.0.1\n\tk8s.io/api v0.33.3\n\tk8s.io/apiextensions-apiserver v0.33.3\n\tk8s.io/apimachinery v0.33.3\n\tk8s.io/cli-runtime v0.33.3\n\tk8s.io/client-go v0.33.3\n\tk8s.io/klog/v2 v2.130.1\n\tk8s.io/kube-aggregator v0.33.3\n\tk8s.io/metrics v0.33.3\n\tk8s.io/utils v0.0.0-20241104100929-3ea5e8cea738\n\tsigs.k8s.io/controller-runtime v0.21.0\n\tsigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3\n\tsigs.k8s.io/yaml v1.4.0\n)\n\nrequire (\n\tcel.dev/expr v0.25.1 // indirect\n\tcloud.google.com/go v0.121.6 // indirect\n\tcloud.google.com/go/auth v0.17.0 // indirect\n\tcloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect\n\tcloud.google.com/go/compute/metadata v0.9.0 // indirect\n\tcloud.google.com/go/iam v1.5.2 // indirect\n\tcloud.google.com/go/monitoring v1.24.2 // indirect\n\tgithub.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2 // indirect\n\tgithub.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 // indirect\n\tgithub.com/AzureAD/microsoft-authentication-library-for-go v1.6.0 // indirect\n\tgithub.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.30.0 // indirect\n\tgithub.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.53.0 // indirect\n\tgithub.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.53.0 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.5.4 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.14.11 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/internal/configsources v1.2.10 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.5.10 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/internal/ini v1.7.2 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/internal/v4a v1.2.10 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.10.4 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/service/internal/checksum v1.2.10 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.10.10 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.16.10 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/service/sso v1.18.6 // indirect\n\tgithub.com/aws/aws-sdk-go-v2/service/ssooidc v1.21.6 // indirect\n\tgithub.com/aws/smithy-go v1.19.0 // indirect\n\tgithub.com/beorn7/perks v1.0.1 // indirect\n\tgithub.com/blang/semver/v4 v4.0.0 // indirect\n\tgithub.com/cespare/xxhash/v2 v2.3.0 // indirect\n\tgithub.com/chmduquesne/rollinghash v4.0.0+incompatible // indirect\n\tgithub.com/cncf/xds/go v0.0.0-20251210132809-ee656c7534f5 // indirect\n\tgithub.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect\n\tgithub.com/dustin/go-humanize v1.0.1 // indirect\n\tgithub.com/edsrzf/mmap-go v1.2.0 // indirect\n\tgithub.com/emicklei/go-restful/v3 v3.11.0 // indirect\n\tgithub.com/envoyproxy/go-control-plane/envoy v1.36.0 // indirect\n\tgithub.com/envoyproxy/protoc-gen-validate v1.3.0 // indirect\n\tgithub.com/felixge/httpsnoop v1.0.4 // indirect\n\tgithub.com/fsnotify/fsnotify v1.7.0 // indirect\n\tgithub.com/fxamacker/cbor/v2 v2.7.0 // indirect\n\tgithub.com/go-ini/ini v1.67.0 // indirect\n\tgithub.com/go-jose/go-jose/v4 v4.1.3 // indirect\n\tgithub.com/go-logr/logr v1.4.3 // indirect\n\tgithub.com/go-logr/stdr v1.2.2 // indirect\n\tgithub.com/go-ole/go-ole v1.3.0 // indirect\n\tgithub.com/go-openapi/jsonpointer v0.21.0 // indirect\n\tgithub.com/go-openapi/jsonreference v0.20.2 // indirect\n\tgithub.com/go-openapi/swag v0.23.0 // indirect\n\tgithub.com/go-task/slim-sprig/v3 v3.0.0 // indirect\n\tgithub.com/gofrs/flock v0.13.0 // indirect\n\tgithub.com/gogo/protobuf v1.3.2 // indirect\n\tgithub.com/golang-jwt/jwt/v5 v5.3.0 // indirect\n\tgithub.com/golang/protobuf v1.5.4 // indirect\n\tgithub.com/google/btree v1.1.3 // indirect\n\tgithub.com/google/gnostic-models v0.6.9 // indirect\n\tgithub.com/google/pprof v0.0.0-20241029153458-d1b30febd7db // indirect\n\tgithub.com/google/s2a-go v0.1.9 // indirect\n\tgithub.com/googleapis/enterprise-certificate-proxy v0.3.7 // indirect\n\tgithub.com/googleapis/gax-go/v2 v2.15.0 // indirect\n\tgithub.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 // indirect\n\tgithub.com/hashicorp/cronexpr v1.1.3 // indirect\n\tgithub.com/hashicorp/yamux v0.1.1 // indirect\n\tgithub.com/inconshreveable/mousetrap v1.1.0 // indirect\n\tgithub.com/jmespath/go-jmespath v0.4.0 // indirect\n\tgithub.com/josharian/intern v1.0.0 // indirect\n\tgithub.com/json-iterator/go v1.1.12 // indirect\n\tgithub.com/klauspost/compress v1.18.2 // indirect\n\tgithub.com/klauspost/cpuid/v2 v2.3.0 // indirect\n\tgithub.com/klauspost/crc32 v1.3.0 // indirect\n\tgithub.com/klauspost/pgzip v1.2.6 // indirect\n\tgithub.com/klauspost/reedsolomon v1.12.6 // indirect\n\tgithub.com/kylelemons/godebug v1.1.0 // indirect\n\tgithub.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de // indirect\n\tgithub.com/mailru/easyjson v0.7.7 // indirect\n\tgithub.com/mattn/go-colorable v0.1.14 // indirect\n\tgithub.com/mattn/go-isatty v0.0.20 // indirect\n\tgithub.com/minio/crc64nvme v1.1.0 // indirect\n\tgithub.com/minio/md5-simd v1.1.2 // indirect\n\tgithub.com/minio/minio-go/v7 v7.0.97 // indirect\n\tgithub.com/mitchellh/go-testing-interface v1.0.0 // indirect\n\tgithub.com/moby/spdystream v0.5.0 // indirect\n\tgithub.com/moby/term v0.5.0 // indirect\n\tgithub.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect\n\tgithub.com/modern-go/reflect2 v1.0.2 // indirect\n\tgithub.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect\n\tgithub.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f // indirect\n\tgithub.com/mxk/go-vss v1.2.0 // indirect\n\tgithub.com/natefinch/atomic v1.0.1 // indirect\n\tgithub.com/nxadm/tail v1.4.8 // indirect\n\tgithub.com/oklog/run v1.0.0 // indirect\n\tgithub.com/philhofer/fwd v1.2.0 // indirect\n\tgithub.com/pierrec/lz4 v2.6.1+incompatible // indirect\n\tgithub.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect\n\tgithub.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect\n\tgithub.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect\n\tgithub.com/prometheus/common v0.67.4 // indirect\n\tgithub.com/prometheus/procfs v0.16.1 // indirect\n\tgithub.com/rs/xid v1.6.0 // indirect\n\tgithub.com/spiffe/go-spiffe/v2 v2.6.0 // indirect\n\tgithub.com/stretchr/objx v0.5.2 // indirect\n\tgithub.com/tinylib/msgp v1.3.0 // indirect\n\tgithub.com/vladimirvivien/gexe v0.1.1 // indirect\n\tgithub.com/x448/float16 v0.8.4 // indirect\n\tgithub.com/zeebo/blake3 v0.2.4 // indirect\n\tgo.opentelemetry.io/auto/sdk v1.2.1 // indirect\n\tgo.opentelemetry.io/contrib/detectors/gcp v1.39.0 // indirect\n\tgo.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0 // indirect\n\tgo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 // indirect\n\tgo.opentelemetry.io/otel v1.40.0 // indirect\n\tgo.opentelemetry.io/otel/metric v1.40.0 // indirect\n\tgo.opentelemetry.io/otel/sdk v1.40.0 // indirect\n\tgo.opentelemetry.io/otel/sdk/metric v1.40.0 // indirect\n\tgo.opentelemetry.io/otel/trace v1.40.0 // indirect\n\tgo.starlark.net v0.0.0-20230525235612-a134d8f9ddca // indirect\n\tgo.uber.org/multierr v1.11.0 // indirect\n\tgo.yaml.in/yaml/v2 v2.4.3 // indirect\n\tgolang.org/x/crypto v0.46.0 // indirect\n\tgolang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 // indirect\n\tgolang.org/x/net v0.48.0 // indirect\n\tgolang.org/x/sync v0.19.0 // indirect\n\tgolang.org/x/term v0.38.0 // indirect\n\tgolang.org/x/time v0.14.0 // indirect\n\tgolang.org/x/tools v0.39.0 // indirect\n\tgomodules.xyz/jsonpatch/v2 v2.4.0 // indirect\n\tgoogle.golang.org/genproto v0.0.0-20250603155806-513f23925822 // indirect\n\tgoogle.golang.org/genproto/googleapis/api v0.0.0-20251202230838-ff82c1b0f217 // indirect\n\tgoogle.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217 // indirect\n\tgopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect\n\tgopkg.in/inf.v0 v0.9.1 // indirect\n\tk8s.io/kube-openapi v0.0.0-20250318190949-c8a335a9a2ff // indirect\n\tsigs.k8s.io/randfill v1.0.0 // indirect\n\tsigs.k8s.io/structured-merge-diff/v4 v4.6.0 // indirect\n)\n\nreplace github.com/kopia/kopia => github.com/project-velero/kopia v0.0.0-20251230033609-d946b1e75197\n" }, { "path": "go.sum", "content": "al.essio.dev/pkg/shellescape v1.5.1 h1:86HrALUujYS/h+GtqoB26SBEdkWfmMI6FubjXlsXyho=\nal.essio.dev/pkg/shellescape v1.5.1/go.mod h1:6sIqp7X2P6mThCQ7twERpZTuigpr6KbZWtls1U8I890=\ncel.dev/expr v0.25.1 h1:1KrZg61W6TWSxuNZ37Xy49ps13NUovb66QLprthtwi4=\ncel.dev/expr v0.25.1/go.mod h1:hrXvqGP6G6gyx8UAHSHJ5RGk//1Oj5nXQ2NI02Nrsg4=\ncloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=\ncloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=\ncloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU=\ncloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU=\ncloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY=\ncloud.google.com/go v0.44.3/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY=\ncloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc=\ncloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0=\ncloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To=\ncloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4=\ncloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M=\ncloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc=\ncloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk=\ncloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs=\ncloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc=\ncloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY=\ncloud.google.com/go v0.72.0/go.mod h1:M+5Vjvlc2wnp6tjzE102Dw08nGShTscUx2nZMufOKPI=\ncloud.google.com/go v0.74.0/go.mod h1:VV1xSbzvo+9QJOxLDaJfTjx5e+MePCpCWwvftOeQmWk=\ncloud.google.com/go v0.75.0/go.mod h1:VGuuCn7PG0dwsd5XPVm2Mm3wlh3EL55/79EKB6hlPTY=\ncloud.google.com/go v0.78.0/go.mod h1:QjdrLG0uq+YwhjoVOLsS1t7TW8fs36kLs4XO5R5ECHg=\ncloud.google.com/go v0.79.0/go.mod h1:3bzgcEeQlzbuEAYu4mrWhKqWjmpprinYgKJLgKHnbb8=\ncloud.google.com/go v0.81.0/go.mod h1:mk/AM35KwGk/Nm2YSeZbxXdrNK3KZOYHmLkOqC2V6E0=\ncloud.google.com/go v0.121.6 h1:waZiuajrI28iAf40cWgycWNgaXPO06dupuS+sgibK6c=\ncloud.google.com/go v0.121.6/go.mod h1:coChdst4Ea5vUpiALcYKXEpR1S9ZgXbhEzzMcMR66vI=\ncloud.google.com/go/auth v0.17.0 h1:74yCm7hCj2rUyyAocqnFzsAYXgJhrG26XCFimrc/Kz4=\ncloud.google.com/go/auth v0.17.0/go.mod h1:6wv/t5/6rOPAX4fJiRjKkJCvswLwdet7G8+UGXt7nCQ=\ncloud.google.com/go/auth/oauth2adapt v0.2.8 h1:keo8NaayQZ6wimpNSmW5OPc283g65QNIiLpZnkHRbnc=\ncloud.google.com/go/auth/oauth2adapt v0.2.8/go.mod h1:XQ9y31RkqZCcwJWNSx2Xvric3RrU88hAYYbjDWYDL+c=\ncloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=\ncloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=\ncloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc=\ncloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg=\ncloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc=\ncloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ=\ncloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdBtwLoEkH9Zs=\ncloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCBSM6qQI1yTBdEb3C10=\ncloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=\ncloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk=\ncloud.google.com/go/firestore v1.1.0/go.mod h1:ulACoGHTpvq5r8rxGJ4ddJZBZqakUQqClKRT5SZwBmk=\ncloud.google.com/go/iam v1.5.2 h1:qgFRAGEmd8z6dJ/qyEchAuL9jpswyODjA2lS+w234g8=\ncloud.google.com/go/iam v1.5.2/go.mod h1:SE1vg0N81zQqLzQEwxL2WI6yhetBdbNQuTvIKCSkUHE=\ncloud.google.com/go/logging v1.13.0 h1:7j0HgAp0B94o1YRDqiqm26w4q1rDMH7XNRU34lJXHYc=\ncloud.google.com/go/logging v1.13.0/go.mod h1:36CoKh6KA/M0PbhPKMq6/qety2DCAErbhXT62TuXALA=\ncloud.google.com/go/longrunning v0.7.0 h1:FV0+SYF1RIj59gyoWDRi45GiYUMM3K1qO51qoboQT1E=\ncloud.google.com/go/longrunning v0.7.0/go.mod h1:ySn2yXmjbK9Ba0zsQqunhDkYi0+9rlXIwnoAf+h+TPY=\ncloud.google.com/go/monitoring v1.24.2 h1:5OTsoJ1dXYIiMiuL+sYscLc9BumrL3CarVLL7dd7lHM=\ncloud.google.com/go/monitoring v1.24.2/go.mod h1:x7yzPWcgDRnPEv3sI+jJGBkwl5qINf+6qY4eq0I9B4U=\ncloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=\ncloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw=\ncloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA=\ncloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU=\ncloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw=\ncloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos=\ncloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=\ncloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs=\ncloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=\ncloud.google.com/go/storage v1.14.0/go.mod h1:GrKmX003DSIwi9o29oFT7YDnHYwZoctc3fOKtUw0Xmo=\ncloud.google.com/go/storage v1.57.2 h1:sVlym3cHGYhrp6XZKkKb+92I1V42ks2qKKpB0CF5Mb4=\ncloud.google.com/go/storage v1.57.2/go.mod h1:n5ijg4yiRXXpCu0sJTD6k+eMf7GRrJmPyr9YxLXGHOk=\ncloud.google.com/go/trace v1.11.6 h1:2O2zjPzqPYAHrn3OKl029qlqG6W8ZdYaOWRyr8NgMT4=\ncloud.google.com/go/trace v1.11.6/go.mod h1:GA855OeDEBiBMzcckLPE2kDunIpC72N+Pq8WFieFjnI=\ndmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=\ngithub.com/Azure/azure-sdk-for-go/sdk/azcore v1.20.0 h1:JXg2dwJUmPB9JmtVmdEB16APJ7jurfbY5jnfXpJoRMc=\ngithub.com/Azure/azure-sdk-for-go/sdk/azcore v1.20.0/go.mod h1:YD5h/ldMsG0XiIw7PdyNhLxaM317eFh5yNLccNfGdyw=\ngithub.com/Azure/azure-sdk-for-go/sdk/azidentity v1.13.1 h1:Hk5QBxZQC1jb2Fwj6mpzme37xbCDdNTxU7O9eb5+LB4=\ngithub.com/Azure/azure-sdk-for-go/sdk/azidentity v1.13.1/go.mod h1:IYus9qsFobWIc2YVwe/WPjcnyCkPKtnHAqUYeebc8z0=\ngithub.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.3.2 h1:yz1bePFlP5Vws5+8ez6T3HWXPmwOK7Yvq8QxDBD3SKY=\ngithub.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.3.2/go.mod h1:Pa9ZNPuoNu/GztvBSKk9J1cDJW6vk/n0zLtV4mgd8N8=\ngithub.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2 h1:9iefClla7iYpfYWdzPCRDozdmndjTm8DXdpCzPajMgA=\ngithub.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2/go.mod h1:XtLgD3ZD34DAaVIIAyG3objl5DynM3CQ/vMcbBNJZGI=\ngithub.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v5 v5.6.0 h1:ui3YNbxfW7J3tTFIZMH6LIGRjCngp+J+nIFlnizfNTE=\ngithub.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v5 v5.6.0/go.mod h1:gZmgV+qBqygoznvqo2J9oKZAFziqhLZ2xE/WVUmzkHA=\ngithub.com/Azure/azure-sdk-for-go/sdk/resourcemanager/internal/v2 v2.0.0 h1:PTFGRSlMKCQelWwxUyYVEUqseBJVemLyqWJjvMyt0do=\ngithub.com/Azure/azure-sdk-for-go/sdk/resourcemanager/internal/v2 v2.0.0/go.mod h1:LRr2FzBTQlONPPa5HREE5+RjSCTXl7BwOvYOaWTqCaI=\ngithub.com/Azure/azure-sdk-for-go/sdk/resourcemanager/internal/v3 v3.1.0 h1:2qsIIvxVT+uE6yrNldntJKlLRgxGbZ85kgtz5SNBhMw=\ngithub.com/Azure/azure-sdk-for-go/sdk/resourcemanager/internal/v3 v3.1.0/go.mod h1:AW8VEadnhw9xox+VaVd9sP7NjzOAnaZBLRH6Tq3cJ38=\ngithub.com/Azure/azure-sdk-for-go/sdk/resourcemanager/resources/armresources v1.2.0 h1:Dd+RhdJn0OTtVGaeDLZpcumkIVCtA/3/Fo42+eoYvVM=\ngithub.com/Azure/azure-sdk-for-go/sdk/resourcemanager/resources/armresources v1.2.0/go.mod h1:5kakwfW5CjC9KK+Q4wjXAg+ShuIm2mBMua0ZFj2C8PE=\ngithub.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storage/armstorage v1.8.1 h1:/Zt+cDPnpC3OVDm/JKLOs7M2DKmLRIIp3XIx9pHHiig=\ngithub.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storage/armstorage v1.8.1/go.mod h1:Ng3urmn6dYe8gnbCMoHHVl5APYz2txho3koEkV2o2HA=\ngithub.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.3 h1:ZJJNFaQ86GVKQ9ehwqyAFE6pIfyicpuJ8IkVaPBc6/4=\ngithub.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.3/go.mod h1:URuDvhmATVKqHBH9/0nOiNKk0+YcwfQ3WkK5PqHKxc8=\ngithub.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 h1:L/gRVlceqvL25UVaW/CKtUDjefjrs0SPonmDGUVOYP0=\ngithub.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E=\ngithub.com/Azure/go-autorest v14.2.0+incompatible/go.mod h1:r+4oMnoxhatjLLJ6zxSWATqVooLgysK6ZNox3g/xq24=\ngithub.com/Azure/go-autorest/autorest v0.11.18/go.mod h1:dSiJPy22c3u0OtOKDNttNgqpNFY/GeWa7GH/Pz56QRA=\ngithub.com/Azure/go-autorest/autorest/adal v0.9.13/go.mod h1:W/MM4U6nLxnIskrw4UwWzlHfGjwUS50aOsc/I3yuU8M=\ngithub.com/Azure/go-autorest/autorest/date v0.3.0/go.mod h1:BI0uouVdmngYNUzGWeSYnokU+TrmwEsOqdt8Y6sso74=\ngithub.com/Azure/go-autorest/autorest/mocks v0.4.1/go.mod h1:LTp+uSrOhSkaKrUy935gNZuuIPPVsHlr9DSOxSayd+k=\ngithub.com/Azure/go-autorest/logger v0.2.1/go.mod h1:T9E3cAhj2VqvPOtCYAvby9aBXkZmbF5NWuPV8+WeEW8=\ngithub.com/Azure/go-autorest/tracing v0.6.0/go.mod h1:+vhtPC754Xsa23ID7GlGsrdKBpUA79WCAKPPZVC2DeU=\ngithub.com/AzureAD/microsoft-authentication-extensions-for-go/cache v0.1.1 h1:WJTmL004Abzc5wDB5VtZG2PJk5ndYDgVacGqfirKxjM=\ngithub.com/AzureAD/microsoft-authentication-extensions-for-go/cache v0.1.1/go.mod h1:tCcJZ0uHAmvjsVYzEFivsRTN00oz5BEsRgQHu5JZ9WE=\ngithub.com/AzureAD/microsoft-authentication-library-for-go v1.6.0 h1:XRzhVemXdgvJqCH0sFfrBUTnUJSBrBf7++ypk+twtRs=\ngithub.com/AzureAD/microsoft-authentication-library-for-go v1.6.0/go.mod h1:HKpQxkWaGLJ+D/5H8QRpyQXA1eKjxkFlOMwck5+33Jk=\ngithub.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=\ngithub.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=\ngithub.com/GehirnInc/crypt v0.0.0-20230320061759-8cc1b52080c5 h1:IEjq88XO4PuBDcvmjQJcQGg+w+UaafSy8G5Kcb5tBhI=\ngithub.com/GehirnInc/crypt v0.0.0-20230320061759-8cc1b52080c5/go.mod h1:exZ0C/1emQJAw5tHOaUDyY1ycttqBAPcxuzf7QbY6ec=\ngithub.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.30.0 h1:sBEjpZlNHzK1voKq9695PJSX2o5NEXl7/OL3coiIY0c=\ngithub.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.30.0/go.mod h1:P4WPRUkOhJC13W//jWpyfJNDAIpvRbAUIYLX/4jtlE0=\ngithub.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.53.0 h1:owcC2UnmsZycprQ5RfRgjydWhuoxg71LUfyiQdijZuM=\ngithub.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.53.0/go.mod h1:ZPpqegjbE99EPKsu3iUWV22A04wzGPcAY/ziSIQEEgs=\ngithub.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/cloudmock v0.53.0 h1:4LP6hvB4I5ouTbGgWtixJhgED6xdf67twf9PoY96Tbg=\ngithub.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/cloudmock v0.53.0/go.mod h1:jUZ5LYlw40WMd07qxcQJD5M40aUxrfwqQX1g7zxYnrQ=\ngithub.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.53.0 h1:Ron4zCA/yk6U7WOBXhTJcDpsUBG9npumK6xw2auFltQ=\ngithub.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.53.0/go.mod h1:cSgYe11MCNYunTnRXrKiR/tHc0eoKjICUuWpNZoVCOo=\ngithub.com/NYTimes/gziphandler v0.0.0-20170623195520-56545f4a5d46/go.mod h1:3wb06e3pkSAbeQ52E9H9iFoQsEEwGN64994WTCIhntQ=\ngithub.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=\ngithub.com/PuerkitoBio/purell v1.1.1/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0=\ngithub.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578/go.mod h1:uGdkoq3SwY9Y+13GIhn11/XLaGBb4BfwItxLd5jeuXE=\ngithub.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=\ngithub.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=\ngithub.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=\ngithub.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o=\ngithub.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=\ngithub.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY=\ngithub.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8=\ngithub.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio=\ngithub.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs=\ngithub.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY=\ngithub.com/aws/aws-sdk-go-v2 v1.24.1 h1:xAojnj+ktS95YZlDf0zxWBkbFtymPeDP+rvUQIH3uAU=\ngithub.com/aws/aws-sdk-go-v2 v1.24.1/go.mod h1:LNh45Br1YAkEKaAqvmE1m8FUx6a5b/V0oAKV7of29b4=\ngithub.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.5.4 h1:OCs21ST2LrepDfD3lwlQiOqIGp6JiEUqG84GzTDoyJs=\ngithub.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.5.4/go.mod h1:usURWEKSNNAcAZuzRn/9ZYPT8aZQkR7xcCtunK/LkJo=\ngithub.com/aws/aws-sdk-go-v2/config v1.26.3 h1:dKuc2jdp10y13dEEvPqWxqLoc0vF3Z9FC45MvuQSxOA=\ngithub.com/aws/aws-sdk-go-v2/config v1.26.3/go.mod h1:Bxgi+DeeswYofcYO0XyGClwlrq3DZEXli0kLf4hkGA0=\ngithub.com/aws/aws-sdk-go-v2/credentials v1.16.14 h1:mMDTwwYO9A0/JbOCOG7EOZHtYM+o7OfGWfu0toa23VE=\ngithub.com/aws/aws-sdk-go-v2/credentials v1.16.14/go.mod h1:cniAUh3ErQPHtCQGPT5ouvSAQ0od8caTO9OOuufZOAE=\ngithub.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.14.11 h1:c5I5iH+DZcH3xOIMlz3/tCKJDaHFwYEmxvlh2fAcFo8=\ngithub.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.14.11/go.mod h1:cRrYDYAMUohBJUtUnOhydaMHtiK/1NZ0Otc9lIb6O0Y=\ngithub.com/aws/aws-sdk-go-v2/feature/s3/manager v1.15.11 h1:I6lAa3wBWfCz/cKkOpAcumsETRkFAl70sWi8ItcMEsM=\ngithub.com/aws/aws-sdk-go-v2/feature/s3/manager v1.15.11/go.mod h1:be1NIO30kJA23ORBLqPo1LttEM6tPNSEcjkd1eKzNW0=\ngithub.com/aws/aws-sdk-go-v2/internal/configsources v1.2.10 h1:vF+Zgd9s+H4vOXd5BMaPWykta2a6Ih0AKLq/X6NYKn4=\ngithub.com/aws/aws-sdk-go-v2/internal/configsources v1.2.10/go.mod h1:6BkRjejp/GR4411UGqkX8+wFMbFbqsUIimfK4XjOKR4=\ngithub.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.5.10 h1:nYPe006ktcqUji8S2mqXf9c/7NdiKriOwMvWQHgYztw=\ngithub.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.5.10/go.mod h1:6UV4SZkVvmODfXKql4LCbaZUpF7HO2BX38FgBf9ZOLw=\ngithub.com/aws/aws-sdk-go-v2/internal/ini v1.7.2 h1:GrSw8s0Gs/5zZ0SX+gX4zQjRnRsMJDJ2sLur1gRBhEM=\ngithub.com/aws/aws-sdk-go-v2/internal/ini v1.7.2/go.mod h1:6fQQgfuGmw8Al/3M2IgIllycxV7ZW7WCdVSqfBeUiCY=\ngithub.com/aws/aws-sdk-go-v2/internal/v4a v1.2.10 h1:5oE2WzJE56/mVveuDZPJESKlg/00AaS2pY2QZcnxg4M=\ngithub.com/aws/aws-sdk-go-v2/internal/v4a v1.2.10/go.mod h1:FHbKWQtRBYUz4vO5WBWjzMD2by126ny5y/1EoaWoLfI=\ngithub.com/aws/aws-sdk-go-v2/service/ec2 v1.143.0 h1:ZAO4y7MSRqU74ZFCA+HC6Ek5fI7dsTdwJg88s72I/gE=\ngithub.com/aws/aws-sdk-go-v2/service/ec2 v1.143.0/go.mod h1:hIsHE0PaWAQakLCshKS7VKWMGXaqrAFp4m95s2W9E6c=\ngithub.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.10.4 h1:/b31bi3YVNlkzkBrm9LfpaKoaYZUxIAj4sHfOTmLfqw=\ngithub.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.10.4/go.mod h1:2aGXHFmbInwgP9ZfpmdIfOELL79zhdNYNmReK8qDfdQ=\ngithub.com/aws/aws-sdk-go-v2/service/internal/checksum v1.2.10 h1:L0ai8WICYHozIKK+OtPzVJBugL7culcuM4E4JOpIEm8=\ngithub.com/aws/aws-sdk-go-v2/service/internal/checksum v1.2.10/go.mod h1:byqfyxJBshFk0fF9YmK0M0ugIO8OWjzH2T3bPG4eGuA=\ngithub.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.10.10 h1:DBYTXwIGQSGs9w4jKm60F5dmCQ3EEruxdc0MFh+3EY4=\ngithub.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.10.10/go.mod h1:wohMUQiFdzo0NtxbBg0mSRGZ4vL3n0dKjLTINdcIino=\ngithub.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.16.10 h1:KOxnQeWy5sXyS37fdKEvAsGHOr9fa/qvwxfJurR/BzE=\ngithub.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.16.10/go.mod h1:jMx5INQFYFYB3lQD9W0D8Ohgq6Wnl7NYOJ2TQndbulI=\ngithub.com/aws/aws-sdk-go-v2/service/s3 v1.48.0 h1:PJTdBMsyvra6FtED7JZtDpQrIAflYDHFoZAu/sKYkwU=\ngithub.com/aws/aws-sdk-go-v2/service/s3 v1.48.0/go.mod h1:4qXHrG1Ne3VGIMZPCB8OjH/pLFO94sKABIusjh0KWPU=\ngithub.com/aws/aws-sdk-go-v2/service/sso v1.18.6 h1:dGrs+Q/WzhsiUKh82SfTVN66QzyulXuMDTV/G8ZxOac=\ngithub.com/aws/aws-sdk-go-v2/service/sso v1.18.6/go.mod h1:+mJNDdF+qiUlNKNC3fxn74WWNN+sOiGOEImje+3ScPM=\ngithub.com/aws/aws-sdk-go-v2/service/ssooidc v1.21.6 h1:Yf2MIo9x+0tyv76GljxzqA3WtC5mw7NmazD2chwjxE4=\ngithub.com/aws/aws-sdk-go-v2/service/ssooidc v1.21.6/go.mod h1:ykf3COxYI0UJmxcfcxcVuz7b6uADi1FkiUz6Eb7AgM8=\ngithub.com/aws/aws-sdk-go-v2/service/sts v1.26.7 h1:NzO4Vrau795RkUdSHKEwiR01FaGzGOH1EETJ+5QHnm0=\ngithub.com/aws/aws-sdk-go-v2/service/sts v1.26.7/go.mod h1:6h2YuIoxaMSCFf5fi1EgZAwdfkGMgDY+DVfa61uLe4U=\ngithub.com/aws/smithy-go v1.19.0 h1:KWFKQV80DpP3vJrrA9sVAHQ5gc2z8i4EzrLhLlWXcBM=\ngithub.com/aws/smithy-go v1.19.0/go.mod h1:NukqUGpCZIILqqiV0NIjeFh24kd/FAa4beRb6nbIUPE=\ngithub.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=\ngithub.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=\ngithub.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=\ngithub.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=\ngithub.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs=\ngithub.com/bketelsen/crypt v0.0.3-0.20200106085610-5cbc8cc4026c/go.mod h1:MKsuJmJgSg28kpZDP6UIiPt0e0Oz0kqKNGyRaWEPv84=\ngithub.com/bketelsen/crypt v0.0.4/go.mod h1:aI6NrJ0pMGgvZKL1iVgXLnfIFJtfV+bKCoqOes/6LfM=\ngithub.com/blang/semver/v4 v4.0.0 h1:1PFHFE6yCCTv8C1TeyNNarDzntLi7wMI5i/pzqYIsAM=\ngithub.com/blang/semver/v4 v4.0.0/go.mod h1:IbckMUScFkM3pff0VJDNKRiT6TG/YpiHIM2yvyW5YoQ=\ngithub.com/bombsimon/logrusr/v3 v3.0.0 h1:tcAoLfuAhKP9npBxWzSdpsvKPQt1XV02nSf2lZA82TQ=\ngithub.com/bombsimon/logrusr/v3 v3.0.0/go.mod h1:PksPPgSFEL2I52pla2glgCyyd2OqOHAnFF5E+g8Ixco=\ngithub.com/bufbuild/protocompile v0.4.0 h1:LbFKd2XowZvQ/kajzguUp2DC9UEIQhIq77fZZlaQsNA=\ngithub.com/bufbuild/protocompile v0.4.0/go.mod h1:3v93+mbWn/v3xzN+31nwkJfrEpAUwp+BagBSZWx+TP8=\ngithub.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=\ngithub.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=\ngithub.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=\ngithub.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=\ngithub.com/chmduquesne/rollinghash v4.0.0+incompatible h1:hnREQO+DXjqIw3rUTzWN7/+Dpw+N5Um8zpKV0JOEgbo=\ngithub.com/chmduquesne/rollinghash v4.0.0+incompatible/go.mod h1:Uc2I36RRfTAf7Dge82bi3RU0OQUmXT9iweIcPqvr8A0=\ngithub.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=\ngithub.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=\ngithub.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=\ngithub.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=\ngithub.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=\ngithub.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=\ngithub.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=\ngithub.com/cncf/xds/go v0.0.0-20251210132809-ee656c7534f5 h1:6xNmx7iTtyBRev0+D/Tv1FZd4SCg8axKApyNyRsAt/w=\ngithub.com/cncf/xds/go v0.0.0-20251210132809-ee656c7534f5/go.mod h1:KdCmV+x/BuvyMxRnYBlmVaq4OLiKW6iRQfvC62cvdkI=\ngithub.com/coreos/bbolt v1.3.2/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk=\ngithub.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=\ngithub.com/coreos/etcd v3.3.13+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=\ngithub.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=\ngithub.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=\ngithub.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=\ngithub.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=\ngithub.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=\ngithub.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=\ngithub.com/cpuguy83/go-md2man/v2 v2.0.4/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=\ngithub.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=\ngithub.com/creack/pty v1.1.18 h1:n56/Zwd5o6whRC5PMGretI4IdRLlmBXYNjScPaBgsbY=\ngithub.com/creack/pty v1.1.18/go.mod h1:MOBLtS5ELjhRRrroQr9kyvTxUAFNvYEK993ew/Vr4O4=\ngithub.com/danieljoos/wincred v1.2.2 h1:774zMFJrqaeYCK2W57BgAem/MLi6mtSE47MB6BOJ0i0=\ngithub.com/danieljoos/wincred v1.2.2/go.mod h1:w7w4Utbrz8lqeMbDAK0lkNJUv5sAOkFi7nd/ogr0Uh8=\ngithub.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=\ngithub.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=\ngithub.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no=\ngithub.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE=\ngithub.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=\ngithub.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=\ngithub.com/edsrzf/mmap-go v1.2.0 h1:hXLYlkbaPzt1SaQk+anYwKSRNhufIDCchSPkUD6dD84=\ngithub.com/edsrzf/mmap-go v1.2.0/go.mod h1:19H/e8pUPLicwkyNgOykDXkJ9F0MHE+Z52B8EIth78Q=\ngithub.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153/go.mod h1:/Zj4wYkgs4iZTTu3o/KG3Itv/qCCa8VVMlb3i9OVuzc=\ngithub.com/emicklei/go-restful v0.0.0-20170410110728-ff4f55a20633/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs=\ngithub.com/emicklei/go-restful/v3 v3.11.0 h1:rAQeMHw1c7zTmncogyy8VvRZwtkmkZ4FxERmMY4rD+g=\ngithub.com/emicklei/go-restful/v3 v3.11.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=\ngithub.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=\ngithub.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=\ngithub.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=\ngithub.com/envoyproxy/go-control-plane v0.9.7/go.mod h1:cwu0lG7PUMfa9snN8LXBig5ynNVH9qI8YYLbd1fK2po=\ngithub.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk=\ngithub.com/envoyproxy/go-control-plane v0.9.9-0.20210217033140-668b12f5399d/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk=\ngithub.com/envoyproxy/go-control-plane v0.14.0 h1:hbG2kr4RuFj222B6+7T83thSPqLjwBIfQawTkC++2HA=\ngithub.com/envoyproxy/go-control-plane v0.14.0/go.mod h1:NcS5X47pLl/hfqxU70yPwL9ZMkUlwlKxtAohpi2wBEU=\ngithub.com/envoyproxy/go-control-plane/envoy v1.36.0 h1:yg/JjO5E7ubRyKX3m07GF3reDNEnfOboJ0QySbH736g=\ngithub.com/envoyproxy/go-control-plane/envoy v1.36.0/go.mod h1:ty89S1YCCVruQAm9OtKeEkQLTb+Lkz0k8v9W0Oxsv98=\ngithub.com/envoyproxy/go-control-plane/ratelimit v0.1.0 h1:/G9QYbddjL25KvtKTv3an9lx6VBE2cnb8wp1vEGNYGI=\ngithub.com/envoyproxy/go-control-plane/ratelimit v0.1.0/go.mod h1:Wk+tMFAFbCXaJPzVVHnPgRKdUdwW/KdbRt94AzgRee4=\ngithub.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=\ngithub.com/envoyproxy/protoc-gen-validate v1.3.0 h1:TvGH1wof4H33rezVKWSpqKz5NXWg5VPuZ0uONDT6eb4=\ngithub.com/envoyproxy/protoc-gen-validate v1.3.0/go.mod h1:HvYl7zwPa5mffgyeTUHA9zHIH36nmrm7oCbo4YKoSWA=\ngithub.com/evanphx/json-patch v4.11.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=\ngithub.com/evanphx/json-patch v5.6.0+incompatible h1:jBYDEEiFBPxA0v50tFdvOzQQTCvpL6mnFh5mB2/l16U=\ngithub.com/evanphx/json-patch v5.6.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=\ngithub.com/evanphx/json-patch/v5 v5.9.11 h1:/8HVnzMq13/3x9TPvjG08wUGqBTmZBsCWzjTM0wiaDU=\ngithub.com/evanphx/json-patch/v5 v5.9.11/go.mod h1:3j+LviiESTElxA4p3EMKAB9HXj3/XEtnUf6OZxqIQTM=\ngithub.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=\ngithub.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM=\ngithub.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU=\ngithub.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=\ngithub.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=\ngithub.com/form3tech-oss/jwt-go v3.2.2+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k=\ngithub.com/form3tech-oss/jwt-go v3.2.3+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k=\ngithub.com/frankban/quicktest v1.13.1 h1:xVm/f9seEhZFL9+n5kv5XLrGwy6elc4V9v/XFY2vmd8=\ngithub.com/frankban/quicktest v1.13.1/go.mod h1:NeW+ay9A/U67EYXNFA1nPE8e/tnQv/09mUdL/ijj8og=\ngithub.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=\ngithub.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=\ngithub.com/fsnotify/fsnotify v1.5.1/go.mod h1:T3375wBYaZdLLcVNkcVbzGHY7f1l/uK5T5Ai1i3InKU=\ngithub.com/fsnotify/fsnotify v1.7.0 h1:8JEhPFa5W2WU7YfeZzPNqzMP6Lwt7L2715Ggo0nosvA=\ngithub.com/fsnotify/fsnotify v1.7.0/go.mod h1:40Bi/Hjc2AVfZrqy+aj+yEI+/bRxZnMJyTJwOpGvigM=\ngithub.com/fxamacker/cbor/v2 v2.7.0 h1:iM5WgngdRBanHcxugY4JySA0nk1wZorNOpTgCMedv5E=\ngithub.com/fxamacker/cbor/v2 v2.7.0/go.mod h1:pxXPTn3joSm21Gbwsv0w9OSA2y1HFR9qXEeXQVeNoDQ=\ngithub.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=\ngithub.com/go-errors/errors v1.0.1/go.mod h1:f4zRHt4oKfwPJE5k8C9vpYG+aDHdBFUsgrm6/TyX73Q=\ngithub.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=\ngithub.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=\ngithub.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=\ngithub.com/go-ini/ini v1.67.0 h1:z6ZrTEZqSWOTyH2FlglNbNgARyHG8oLW9gMELqKr06A=\ngithub.com/go-ini/ini v1.67.0/go.mod h1:ByCAeIL28uOIIG0E3PJtZPDL8WnHpFKFOtgjp+3Ies8=\ngithub.com/go-jose/go-jose/v4 v4.1.3 h1:CVLmWDhDVRa6Mi/IgCgaopNosCaHz7zrMeF9MlZRkrs=\ngithub.com/go-jose/go-jose/v4 v4.1.3/go.mod h1:x4oUasVrzR7071A4TnHLGSPpNOm2a21K9Kf04k1rs08=\ngithub.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=\ngithub.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=\ngithub.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=\ngithub.com/go-logr/logr v0.1.0/go.mod h1:ixOQHD9gLJUVQQ2ZOR7zLEifBX6tGkNJF4QyIY7sIas=\ngithub.com/go-logr/logr v0.4.0/go.mod h1:z6/tIYblkpsD+a4lm/fGIIU9mZ+XfAiaFtq7xTgseGU=\ngithub.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=\ngithub.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=\ngithub.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=\ngithub.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=\ngithub.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=\ngithub.com/go-logr/zapr v1.3.0 h1:XGdV8XW8zdwFiwOA2Dryh1gj2KRQyOOoNmBy4EplIcQ=\ngithub.com/go-logr/zapr v1.3.0/go.mod h1:YKepepNBd1u/oyhd/yQmtjVXmm9uML4IXUgMOwR8/Gg=\ngithub.com/go-ole/go-ole v1.3.0 h1:Dt6ye7+vXGIKZ7Xtk4s6/xVdGDQynvom7xCFEdWr6uE=\ngithub.com/go-ole/go-ole v1.3.0/go.mod h1:5LS6F96DhAwUc7C+1HLexzMXY1xGRSryjyPPKW6zv78=\ngithub.com/go-openapi/jsonpointer v0.19.3/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg=\ngithub.com/go-openapi/jsonpointer v0.19.5/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg=\ngithub.com/go-openapi/jsonpointer v0.19.6/go.mod h1:osyAmYz/mB/C3I+WsTTSgw1ONzaLJoLCyoi6/zppojs=\ngithub.com/go-openapi/jsonpointer v0.21.0 h1:YgdVicSA9vH5RiHs9TZW5oyafXZFc6+2Vc1rr/O9oNQ=\ngithub.com/go-openapi/jsonpointer v0.21.0/go.mod h1:IUyH9l/+uyhIYQ/PXVA41Rexl+kOkAPDdXEYns6fzUY=\ngithub.com/go-openapi/jsonreference v0.19.3/go.mod h1:rjx6GuL8TTa9VaixXglHmQmIL98+wF9xc8zWvFonSJ8=\ngithub.com/go-openapi/jsonreference v0.19.5/go.mod h1:RdybgQwPxbL4UEjuAruzK1x3nE69AqPYEJeo/TWfEeg=\ngithub.com/go-openapi/jsonreference v0.20.2 h1:3sVjiK66+uXK/6oQ8xgcRKcFgQ5KXa2KvnJRumpMGbE=\ngithub.com/go-openapi/jsonreference v0.20.2/go.mod h1:Bl1zwGIM8/wsvqjsOQLJ/SH+En5Ap4rVB5KVcIDZG2k=\ngithub.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk=\ngithub.com/go-openapi/swag v0.19.14/go.mod h1:QYRuS/SOXUCsnplDa677K7+DxSOj6IPNl/eQntq43wQ=\ngithub.com/go-openapi/swag v0.22.3/go.mod h1:UzaqsxGiab7freDnrUUra0MwWfN/q7tE4j+VcZ0yl14=\ngithub.com/go-openapi/swag v0.23.0 h1:vsEVJDUo2hPJ2tu0/Xc+4noaxyEffXNIs3cOULZ+GrE=\ngithub.com/go-openapi/swag v0.23.0/go.mod h1:esZ8ITTYEsH1V2trKHjAN8Ai7xHb8RV+YSZ577vPjgQ=\ngithub.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=\ngithub.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1vB6EwHI=\ngithub.com/go-task/slim-sprig/v3 v3.0.0/go.mod h1:W848ghGpv3Qj3dhTPRyJypKRiqCdHZiAzKg9hl15HA8=\ngithub.com/gobwas/glob v0.2.3 h1:A4xDbljILXROh+kObIiy5kIaPYD8e96x1tgBhUI5J+Y=\ngithub.com/gobwas/glob v0.2.3/go.mod h1:d3Ez4x06l9bZtSvzIay5+Yzi0fmZzPgnTbPcKjJAkT8=\ngithub.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=\ngithub.com/godbus/dbus/v5 v5.1.0 h1:4KLkAxT3aOY8Li4FRJe/KvhoNFFxo0m6fNuFUO8QJUk=\ngithub.com/godbus/dbus/v5 v5.1.0/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=\ngithub.com/gofrs/flock v0.13.0 h1:95JolYOvGMqeH31+FC7D2+uULf6mG61mEZ/A8dRYMzw=\ngithub.com/gofrs/flock v0.13.0/go.mod h1:jxeyy9R1auM5S6JYDBhDt+E2TCo7DkratH4Pgi8P+Z0=\ngithub.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=\ngithub.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=\ngithub.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=\ngithub.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=\ngithub.com/golang-jwt/jwt/v4 v4.5.2 h1:YtQM7lnr8iZ+j5q71MGKkNw9Mn7AjHM68uc9g5fXeUI=\ngithub.com/golang-jwt/jwt/v4 v4.5.2/go.mod h1:m21LjoU+eqJr34lmDMbreY2eSTRJ1cv77w39/MY0Ch0=\ngithub.com/golang-jwt/jwt/v5 v5.3.0 h1:pv4AsKCKKZuqlgs5sUmn4x8UlGa0kEVt/puTpKx9vvo=\ngithub.com/golang-jwt/jwt/v5 v5.3.0/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=\ngithub.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=\ngithub.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\ngithub.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\ngithub.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\ngithub.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\ngithub.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\ngithub.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=\ngithub.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=\ngithub.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y=\ngithub.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=\ngithub.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=\ngithub.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=\ngithub.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4=\ngithub.com/golang/mock v1.5.0/go.mod h1:CWnOUgYIOo4TcNZ0wHX3YZCqsaM1I1Jvs6v3mP3KVu8=\ngithub.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=\ngithub.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=\ngithub.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=\ngithub.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=\ngithub.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=\ngithub.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk=\ngithub.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=\ngithub.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=\ngithub.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=\ngithub.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=\ngithub.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=\ngithub.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=\ngithub.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=\ngithub.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=\ngithub.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=\ngithub.com/golang/protobuf v1.5.1/go.mod h1:DopwsBzvsk0Fs44TXzsVbJyPhcCPeIwnvohx4u74HPM=\ngithub.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=\ngithub.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=\ngithub.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=\ngithub.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=\ngithub.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=\ngithub.com/google/btree v1.0.1/go.mod h1:xXMiIv4Fb/0kKde4SpL7qlzvu5cMJDRkFDxJfI9uaxA=\ngithub.com/google/btree v1.1.3 h1:CVpQJjYgC4VbzxeGVHfvZrv1ctoYCAI8vbl07Fcxlyg=\ngithub.com/google/btree v1.1.3/go.mod h1:qOPhT0dTNdNzV6Z/lhRX0YXUafgPLFUh+gZMl761Gm4=\ngithub.com/google/gnostic-models v0.6.9 h1:MU/8wDLif2qCXZmzncUQ/BOfxWfthHi63KqpoNbWqVw=\ngithub.com/google/gnostic-models v0.6.9/go.mod h1:CiWsm0s6BSQd1hRn8/QmxqB6BesYcbSZxsz9b0KuDBw=\ngithub.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=\ngithub.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=\ngithub.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=\ngithub.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=\ngithub.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=\ngithub.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=\ngithub.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=\ngithub.com/google/gofuzz v1.1.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=\ngithub.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0=\ngithub.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=\ngithub.com/google/martian v2.1.0+incompatible h1:/CP5g8u/VJHijgedC/Legn3BAbAaWPgecwXBIDzw5no=\ngithub.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=\ngithub.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=\ngithub.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=\ngithub.com/google/martian/v3 v3.3.3 h1:DIhPTQrbPkgs2yJYdXU/eNACCG5DVQjySNRNlflZ9Fc=\ngithub.com/google/martian/v3 v3.3.3/go.mod h1:iEPrYcgCF7jA9OtScMFQyAlZZ4YXTKEtJ1E6RWzmBA0=\ngithub.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=\ngithub.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=\ngithub.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=\ngithub.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=\ngithub.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=\ngithub.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=\ngithub.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=\ngithub.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=\ngithub.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=\ngithub.com/google/pprof v0.0.0-20201218002935-b9804c9f04c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=\ngithub.com/google/pprof v0.0.0-20210122040257-d980be63207e/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=\ngithub.com/google/pprof v0.0.0-20210226084205-cbba55b83ad5/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=\ngithub.com/google/pprof v0.0.0-20241029153458-d1b30febd7db h1:097atOisP2aRj7vFgYQBbFN4U4JNXUNYpxael3UzMyo=\ngithub.com/google/pprof v0.0.0-20241029153458-d1b30febd7db/go.mod h1:vavhavw2zAxS5dIdcRluK6cSGGPlZynqzFM8NdvU144=\ngithub.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=\ngithub.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0=\ngithub.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM=\ngithub.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510/go.mod h1:pupxD2MaaD3pAXIBCelhxNneeOaAeabZDe5s4K6zSpQ=\ngithub.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=\ngithub.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=\ngithub.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=\ngithub.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=\ngithub.com/googleapis/enterprise-certificate-proxy v0.3.7 h1:zrn2Ee/nWmHulBx5sAVrGgAa0f2/R35S4DJwfFaUPFQ=\ngithub.com/googleapis/enterprise-certificate-proxy v0.3.7/go.mod h1:MkHOF77EYAE7qfSuSS9PU6g4Nt4e11cnsDUowfwewLA=\ngithub.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=\ngithub.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=\ngithub.com/googleapis/gax-go/v2 v2.15.0 h1:SyjDc1mGgZU5LncH8gimWo9lW1DtIfPibOG81vgd/bo=\ngithub.com/googleapis/gax-go/v2 v2.15.0/go.mod h1:zVVkkxAQHa1RQpg9z2AUCMnKhi0Qld9rcmyfL1OZhoc=\ngithub.com/googleapis/gnostic v0.5.1/go.mod h1:6U4PtQXGIEt/Z3h5MAT7FNofLnw9vXk2cUuW7uA/OeU=\ngithub.com/googleapis/gnostic v0.5.5/go.mod h1:7+EbHbldMins07ALC74bsA81Ovc97DwqyJO1AENw9kA=\ngithub.com/googleapis/google-cloud-go-testing v0.0.0-20200911160855-bcd43fbb19e8/go.mod h1:dvDLG8qkwmyD9a/MJJN3XJcT3xFxOKAvTZGvuZmac9g=\ngithub.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=\ngithub.com/gorilla/mux v1.8.1 h1:TuBL49tXwgrFYWhqrNgrUNEY92u81SPhu7sTdzQEiWY=\ngithub.com/gorilla/mux v1.8.1/go.mod h1:AKf9I4AEqPTmMytcMc0KkNouC66V3BtZ4qD5fmWSiMQ=\ngithub.com/gorilla/websocket v1.4.0/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=\ngithub.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=\ngithub.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 h1:JeSE6pjso5THxAzdVpqr6/geYxZytqFMBCOtn/ujyeo=\ngithub.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674/go.mod h1:r4w70xmWCQKmi1ONH4KIaBptdivuRPyosB9RmPlGEwA=\ngithub.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA=\ngithub.com/grpc-ecosystem/go-grpc-middleware v1.0.0/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs=\ngithub.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk=\ngithub.com/grpc-ecosystem/grpc-gateway v1.9.0/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=\ngithub.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw=\ngithub.com/hanwen/go-fuse/v2 v2.9.0 h1:0AOGUkHtbOVeyGLr0tXupiid1Vg7QB7M6YUcdmVdC58=\ngithub.com/hanwen/go-fuse/v2 v2.9.0/go.mod h1:yE6D2PqWwm3CbYRxFXV9xUd8Md5d6NG0WBs5spCswmI=\ngithub.com/hashicorp/consul/api v1.1.0/go.mod h1:VmuI/Lkw1nC05EYQWNKwWGbkg+FbDBtguAZLlVdkD9Q=\ngithub.com/hashicorp/consul/sdk v0.1.1/go.mod h1:VKf9jXwCTEY1QZP2MOLRhb5i/I/ssyNV1vwHyQBF0x8=\ngithub.com/hashicorp/cronexpr v1.1.3 h1:rl5IkxXN2m681EfivTlccqIryzYJSXRGRNa0xeG7NA4=\ngithub.com/hashicorp/cronexpr v1.1.3/go.mod h1:P4wA0KBl9C5q2hABiMO7cp6jcIg96CDh1Efb3g1PWA4=\ngithub.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=\ngithub.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=\ngithub.com/hashicorp/go-hclog v0.14.1 h1:nQcJDQwIAGnmoUWp8ubocEX40cCml/17YkF6csQLReU=\ngithub.com/hashicorp/go-hclog v0.14.1/go.mod h1:whpDNt7SSdeAju8AWKIWsul05p54N/39EeqMAyrmvFQ=\ngithub.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60=\ngithub.com/hashicorp/go-msgpack v0.5.3/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM=\ngithub.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk=\ngithub.com/hashicorp/go-plugin v1.6.0 h1:wgd4KxHJTVGGqWBq4QPB1i5BZNEx9BR8+OFmHDmTk8A=\ngithub.com/hashicorp/go-plugin v1.6.0/go.mod h1:lBS5MtSSBZk0SHc66KACcjjlU6WzEVP/8pwz68aMkCI=\ngithub.com/hashicorp/go-rootcerts v1.0.0/go.mod h1:K6zTfqpRlCUIjkwsN4Z+hiSfzSTQa6eBIzfwKfwNnHU=\ngithub.com/hashicorp/go-sockaddr v1.0.0/go.mod h1:7Xibr9yA9JjQq1JpNB2Vw7kxv8xerXegt+ozgdvDeDU=\ngithub.com/hashicorp/go-syslog v1.0.0/go.mod h1:qPfqrKkXGihmCqbJM2mZgkZGvKG1dFdvsLplgctolz4=\ngithub.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=\ngithub.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=\ngithub.com/hashicorp/go.net v0.0.1/go.mod h1:hjKkEWcCURg++eb33jQU7oqQcI9XDCnUzHA0oac0k90=\ngithub.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=\ngithub.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=\ngithub.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=\ngithub.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64=\ngithub.com/hashicorp/mdns v1.0.0/go.mod h1:tL+uN++7HEJ6SQLQ2/p+z2pH24WQKWjBPkE0mNTz8vQ=\ngithub.com/hashicorp/memberlist v0.1.3/go.mod h1:ajVTdAv/9Im8oMAAj5G31PhhMCZJV2pPBoIllUwCN7I=\ngithub.com/hashicorp/serf v0.8.2/go.mod h1:6hOLApaqBFA1NXqRQAsxw9QxuDEvNxSQRwA/JwenrHc=\ngithub.com/hashicorp/yamux v0.1.1 h1:yrQxtgseBDrq9Y652vSRDvsKCJKOUD+GzTS4Y0Y8pvE=\ngithub.com/hashicorp/yamux v0.1.1/go.mod h1:CtWFDAQgb7dxtzFs4tWbplKIe2jSi3+5vKbgIO0SLnQ=\ngithub.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=\ngithub.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=\ngithub.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=\ngithub.com/imdario/mergo v0.3.5/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=\ngithub.com/imdario/mergo v0.3.11/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA=\ngithub.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=\ngithub.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=\ngithub.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=\ngithub.com/jhump/protoreflect v1.15.1 h1:HUMERORf3I3ZdX05WaQ6MIpd/NJ434hTp5YiKgfCL6c=\ngithub.com/jhump/protoreflect v1.15.1/go.mod h1:jD/2GMKKE6OqX8qTjhADU1e6DShO+gavG9e0Q693nKo=\ngithub.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg=\ngithub.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=\ngithub.com/jmespath/go-jmespath/internal/testify v1.5.1 h1:shLQSRRSCCPj3f2gpwzGwWFoC7ycTf1rcQZHOlsJ6N8=\ngithub.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U=\ngithub.com/joho/godotenv v1.3.0 h1:Zjp+RcGpHhGlrMbJzXTrZZPrWj+1vfm90La1wgB6Bhc=\ngithub.com/joho/godotenv v1.3.0/go.mod h1:7hK45KPybAkOC6peb+G5yklZfMxEjkZhHbwpqxOKXbg=\ngithub.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo=\ngithub.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY=\ngithub.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=\ngithub.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=\ngithub.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=\ngithub.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=\ngithub.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=\ngithub.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=\ngithub.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=\ngithub.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=\ngithub.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=\ngithub.com/keybase/go-keychain v0.0.1 h1:way+bWYa6lDppZoZcgMbYsvC7GxljxrskdNInRtuthU=\ngithub.com/keybase/go-keychain v0.0.1/go.mod h1:PdEILRW3i9D8JcdM+FmY6RwkHGnhHxXwkPPMeUgOK1k=\ngithub.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=\ngithub.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=\ngithub.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=\ngithub.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk=\ngithub.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4=\ngithub.com/klauspost/cpuid/v2 v2.0.1/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=\ngithub.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=\ngithub.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=\ngithub.com/klauspost/crc32 v1.3.0 h1:sSmTt3gUt81RP655XGZPElI0PelVTZ6YwCRnPSupoFM=\ngithub.com/klauspost/crc32 v1.3.0/go.mod h1:D7kQaZhnkX/Y0tstFGf8VUzv2UofNGqCjnC3zdHB0Hw=\ngithub.com/klauspost/pgzip v1.2.6 h1:8RXeL5crjEUFnR2/Sn6GJNWtSQ3Dk8pq4CL3jvdDyjU=\ngithub.com/klauspost/pgzip v1.2.6/go.mod h1:Ch1tH69qFZu15pkjo5kYi6mth2Zzwzt50oCQKQE9RUs=\ngithub.com/klauspost/reedsolomon v1.12.6 h1:8pqE9aECQG/ZFitiUD1xK/E83zwosBAZtE3UbuZM8TQ=\ngithub.com/klauspost/reedsolomon v1.12.6/go.mod h1:ggJT9lc71Vu+cSOPBlxGvBN6TfAS77qB4fp8vJ05NSA=\ngithub.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=\ngithub.com/kopia/htmluibuild v0.0.1-0.20251125011029-7f1c3f84f29d h1:U3VB/cDMsPW4zB4JRFbVRDzIpPytt889rJUKAG40NPA=\ngithub.com/kopia/htmluibuild v0.0.1-0.20251125011029-7f1c3f84f29d/go.mod h1:h53A5JM3t2qiwxqxusBe+PFgGcgZdS+DWCQvG5PTlto=\ngithub.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=\ngithub.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=\ngithub.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=\ngithub.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=\ngithub.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=\ngithub.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=\ngithub.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=\ngithub.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=\ngithub.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=\ngithub.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=\ngithub.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=\ngithub.com/kubernetes-csi/external-snapshotter/client/v8 v8.2.0 h1:Q3jQ1NkFqv5o+F8dMmHd8SfEmlcwNeo1immFApntEwE=\ngithub.com/kubernetes-csi/external-snapshotter/client/v8 v8.2.0/go.mod h1:E3vdYxHj2C2q6qo8/Da4g7P+IcwqRZyy3gJBzYybV9Y=\ngithub.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=\ngithub.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=\ngithub.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de h1:9TO3cAIGXtEhnIaL+V+BEER86oLrvS+kWobKpbJuye0=\ngithub.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de/go.mod h1:zAbeS9B/r2mtpb6U+EI2rYA5OAXxsYw6wTamcNW+zcE=\ngithub.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=\ngithub.com/magiconair/properties v1.8.1/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=\ngithub.com/magiconair/properties v1.8.5/go.mod h1:y3VJvCyxH9uVvJTWEGAELF3aiYNyPKd5NZ3oSwXrF60=\ngithub.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=\ngithub.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=\ngithub.com/mailru/easyjson v0.7.0/go.mod h1:KAzv3t3aY1NaHWoQz1+4F1ccyAH66Jk7yos7ldAVICs=\ngithub.com/mailru/easyjson v0.7.6/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=\ngithub.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=\ngithub.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=\ngithub.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=\ngithub.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=\ngithub.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=\ngithub.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=\ngithub.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=\ngithub.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=\ngithub.com/mattn/go-isatty v0.0.10/go.mod h1:qgIWMr58cqv1PHHyhnkY9lrL7etaEgOFcMEpPG5Rm84=\ngithub.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=\ngithub.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=\ngithub.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=\ngithub.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=\ngithub.com/minio/crc64nvme v1.1.0 h1:e/tAguZ+4cw32D+IO/8GSf5UVr9y+3eJcxZI2WOO/7Q=\ngithub.com/minio/crc64nvme v1.1.0/go.mod h1:eVfm2fAzLlxMdUGc0EEBGSMmPwmXD5XiNRpnu9J3bvg=\ngithub.com/minio/md5-simd v1.1.2 h1:Gdi1DZK69+ZVMoNHRXJyNcxrMA4dSxoYHZSQbirFg34=\ngithub.com/minio/md5-simd v1.1.2/go.mod h1:MzdKDxYpY2BT9XQFocsiZf/NKVtR7nkE4RoEpN+20RM=\ngithub.com/minio/minio-go/v7 v7.0.97 h1:lqhREPyfgHTB/ciX8k2r8k0D93WaFqxbJX36UZq5occ=\ngithub.com/minio/minio-go/v7 v7.0.97/go.mod h1:re5VXuo0pwEtoNLsNuSr0RrLfT/MBtohwdaSmPPSRSk=\ngithub.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc=\ngithub.com/mitchellh/go-homedir v1.0.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=\ngithub.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=\ngithub.com/mitchellh/go-testing-interface v1.0.0 h1:fzU/JVNcaqHQEcVFAKeR41fkiLdIPrefOvVG1VZ96U0=\ngithub.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI=\ngithub.com/mitchellh/gox v0.4.0/go.mod h1:Sd9lOJ0+aimLBi73mGofS1ycjY8lL3uZM3JPS42BGNg=\ngithub.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0QubkSMEySY=\ngithub.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=\ngithub.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=\ngithub.com/mitchellh/mapstructure v1.4.1/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=\ngithub.com/moby/spdystream v0.2.0/go.mod h1:f7i0iNDQJ059oMTcWxx8MA/zKFIuD/lY+0GqbN2Wy8c=\ngithub.com/moby/spdystream v0.5.0 h1:7r0J1Si3QO/kjRitvSLVVFUjxMEb/YLj6S9FF62JBCU=\ngithub.com/moby/spdystream v0.5.0/go.mod h1:xBAYlnt/ay+11ShkdFKNAG7LsyK/tmNBVvVOwrfMgdI=\ngithub.com/moby/term v0.5.0 h1:xt8Q1nalod/v7BqbG21f8mQPqH+xAaC9C3N3wfWbVP0=\ngithub.com/moby/term v0.5.0/go.mod h1:8FzsFHVUBGZdbDsJw/ot+X+d5HLUbvklYLJ9uGfcI3Y=\ngithub.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=\ngithub.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=\ngithub.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=\ngithub.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=\ngithub.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=\ngithub.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=\ngithub.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=\ngithub.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00/go.mod h1:Pm3mSP3c5uWn86xMLZ5Sa7JB9GsEZySvHYXCTK4E9q4=\ngithub.com/munnerz/goautoneg v0.0.0-20120707110453-a547fc61f48d/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=\ngithub.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=\ngithub.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=\ngithub.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=\ngithub.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f h1:y5//uYreIhSUg3J1GEMiLbxo1LJaP8RfCpH6pymGZus=\ngithub.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f/go.mod h1:ZdcZmHo+o7JKHSa8/e818NopupXU1YMK5fe1lsApnBw=\ngithub.com/mxk/go-vss v1.2.0 h1:JpdOPc/P6B3XyRoddn0iMiG/ADBi3AuEsv8RlTb+JeE=\ngithub.com/mxk/go-vss v1.2.0/go.mod h1:ZQ4yFxCG54vqPnCd+p2IxAe5jwZdz56wSjbwzBXiFd8=\ngithub.com/natefinch/atomic v1.0.1 h1:ZPYKxkqQOx3KZ+RsbnP/YsgvxWQPGxjC0oBt2AhwV0A=\ngithub.com/natefinch/atomic v1.0.1/go.mod h1:N/D/ELrljoqDyT3rZrsUmtsuzvHkeB/wWjHV22AZRbM=\ngithub.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=\ngithub.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=\ngithub.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE=\ngithub.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU=\ngithub.com/oklog/run v1.0.0 h1:Ru7dDtJNOyC66gQ5dQmaCa0qIsAUFY3sFpK1Xk8igrw=\ngithub.com/oklog/run v1.0.0/go.mod h1:dlhp/R75TPv97u0XWUtDeV/lRKWPKSdTuV0TZvrmrQA=\ngithub.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=\ngithub.com/onsi/ginkgo v0.0.0-20170829012221-11459a886d9c/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=\ngithub.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=\ngithub.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk=\ngithub.com/onsi/ginkgo v1.14.0 h1:2mOpI4JVVPBN+WQRa0WKH2eXR+Ey+uK4n7Zj0aYpIQA=\ngithub.com/onsi/ginkgo v1.14.0/go.mod h1:iSB4RoI2tjJc9BBv4NKIKWKya62Rps+oPG/Lv9klQyY=\ngithub.com/onsi/ginkgo/v2 v2.22.0 h1:Yed107/8DjTr0lKCNt7Dn8yQ6ybuDRQoMGrNFKzMfHg=\ngithub.com/onsi/ginkgo/v2 v2.22.0/go.mod h1:7Du3c42kxCUegi0IImZ1wUQzMBVecgIHjR1C+NkhLQo=\ngithub.com/onsi/gomega v0.0.0-20170829124025-dcabb60a477c/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA=\ngithub.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY=\ngithub.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo=\ngithub.com/onsi/gomega v1.36.1 h1:bJDPBO7ibjxcbHMgSCoo4Yj18UWbKDlLwX1x9sybDcw=\ngithub.com/onsi/gomega v1.36.1/go.mod h1:PvZbdDc8J6XJEpDK4HCuRBm8a6Fzp9/DmhC9C7yFlog=\ngithub.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=\ngithub.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=\ngithub.com/pelletier/go-toml v1.9.3/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c=\ngithub.com/petar/GoLLRB v0.0.0-20210522233825-ae3b015fd3e9 h1:1/WtZae0yGtPq+TI6+Tv1WTxkukpXeMlviSxvL7SRgk=\ngithub.com/petar/GoLLRB v0.0.0-20210522233825-ae3b015fd3e9/go.mod h1:x3N5drFsm2uilKKuuYo6LdyD8vZAW55sH/9w+pbo1sw=\ngithub.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU=\ngithub.com/philhofer/fwd v1.2.0 h1:e6DnBTl7vGY+Gz322/ASL4Gyp1FspeMvx1RNDoToZuM=\ngithub.com/philhofer/fwd v1.2.0/go.mod h1:RqIHx9QI14HlwKwm98g9Re5prTQ6LdeRQn+gXJFxsJM=\ngithub.com/pierrec/lz4 v2.6.1+incompatible h1:9UY3+iC23yxF0UfGaYrGplQ+79Rg+h/q9FV9ix19jjM=\ngithub.com/pierrec/lz4 v2.6.1+incompatible/go.mod h1:pdkljMzZIN41W+lC3N2tnIh5sFi+IEE17M5jbnwPHcY=\ngithub.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c h1:+mdjkGKdHQG3305AYmdv1U2eRNDiU2ErMBj1gwrq8eQ=\ngithub.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c/go.mod h1:7rwL4CYBLnjLxUqIJNnCWiEdr3bn6IUYi15bNlnbCCU=\ngithub.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=\ngithub.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=\ngithub.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=\ngithub.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=\ngithub.com/pkg/sftp v1.10.1/go.mod h1:lYOWFsE0bwd1+KfKJaKeuokY15vzFx25BLbzYYoAxZI=\ngithub.com/pkg/sftp v1.13.1/go.mod h1:3HaPG6Dq1ILlpPZRO0HVMrsydcdLt6HRDccSgb87qRg=\ngithub.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 h1:GFCKgmp0tecUJ0sJuv4pzYCqS9+RGSn52M3FUwPs+uo=\ngithub.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10/go.mod h1:t/avpk3KcrXxUnYOhZhMXJlSEyie6gQbtLq5NM3loB8=\ngithub.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=\ngithub.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=\ngithub.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=\ngithub.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI=\ngithub.com/project-velero/kopia v0.0.0-20251230033609-d946b1e75197 h1:iGkfuELGvFCqW+zcrhf2GsOwNH1nWYBsC69IOc57KJk=\ngithub.com/project-velero/kopia v0.0.0-20251230033609-d946b1e75197/go.mod h1:RL4KehCNKEIDNltN7oruSa3ldwBNVPmQbwmN3Schbjc=\ngithub.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=\ngithub.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDft0ttaMvbicHlPoso=\ngithub.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o=\ngithub.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg=\ngithub.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=\ngithub.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=\ngithub.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=\ngithub.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=\ngithub.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=\ngithub.com/prometheus/common v0.0.0-20181113130724-41aa239b4cce/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro=\ngithub.com/prometheus/common v0.4.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=\ngithub.com/prometheus/common v0.67.4 h1:yR3NqWO1/UyO1w2PhUvXlGQs/PtFmoveVO0KZ4+Lvsc=\ngithub.com/prometheus/common v0.67.4/go.mod h1:gP0fq6YjjNCLssJCQp0yk4M8W6ikLURwkdd/YKtTbyI=\ngithub.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=\ngithub.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=\ngithub.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg=\ngithub.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is=\ngithub.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU=\ngithub.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs=\ngithub.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro=\ngithub.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg=\ngithub.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ=\ngithub.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=\ngithub.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=\ngithub.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=\ngithub.com/rs/xid v1.6.0 h1:fV591PaemRlL6JfRxGDEPl69wICngIQ3shQtzfy2gxU=\ngithub.com/rs/xid v1.6.0/go.mod h1:7XoLgs4eV+QndskICGsho+ADou8ySMSjJKDIan90Nz0=\ngithub.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=\ngithub.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=\ngithub.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts=\ngithub.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc=\ngithub.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=\ngithub.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=\ngithub.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=\ngithub.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=\ngithub.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=\ngithub.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=\ngithub.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc=\ngithub.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA=\ngithub.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM=\ngithub.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=\ngithub.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=\ngithub.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk=\ngithub.com/spf13/afero v1.6.0/go.mod h1:Ai8FlHk4v/PARR026UzYexafAt9roJ7LcLMAmO6Z93I=\ngithub.com/spf13/afero v1.10.0 h1:EaGW2JJh15aKOejeuJ+wpFSHnbd7GE6Wvp3TsNhb6LY=\ngithub.com/spf13/afero v1.10.0/go.mod h1:UBogFpq8E9Hx+xc5CNTTEpTnuHVmXDwZcZcE1eb/UhQ=\ngithub.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=\ngithub.com/spf13/cast v1.3.1/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=\ngithub.com/spf13/cobra v1.0.0/go.mod h1:/6GTrnGXV9HjY+aR4k0oJ5tcvakLuG6EuKReYlHNrgE=\ngithub.com/spf13/cobra v1.1.3/go.mod h1:pGADOWyqRD/YMrPZigI/zbliZ2wVD/23d+is3pSWzOo=\ngithub.com/spf13/cobra v1.2.1/go.mod h1:ExllRjgxM/piMAM+3tAZvg8fsklGAf3tPfi+i8t68Nk=\ngithub.com/spf13/cobra v1.8.1 h1:e5/vxKd/rZsfSJMUX1agtjeTDf+qv1/JdBF8gg5k9ZM=\ngithub.com/spf13/cobra v1.8.1/go.mod h1:wHxEcudfqmLYa8iTfL+OuZPbBZkmvliBWKIezN3kD9Y=\ngithub.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo=\ngithub.com/spf13/jwalterweatherman v1.1.0/go.mod h1:aNWZUN0dPAAO/Ljvb5BEdw96iTZ0EXowPYD95IqWIGo=\ngithub.com/spf13/pflag v0.0.0-20170130214245-9ff6c6923cff/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=\ngithub.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=\ngithub.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=\ngithub.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=\ngithub.com/spf13/viper v1.4.0/go.mod h1:PTJ7Z/lr49W6bUbkmS1V3by4uWynFiR9p7+dSq/yZzE=\ngithub.com/spf13/viper v1.7.0/go.mod h1:8WkrPz2fc9jxqZNCJI/76HCieCp4Q8HaLFoCha5qpdg=\ngithub.com/spf13/viper v1.8.1/go.mod h1:o0Pch8wJ9BVSWGQMbra6iw0oQ5oktSIBaujf1rJH9Ns=\ngithub.com/spiffe/go-spiffe/v2 v2.6.0 h1:l+DolpxNWYgruGQVV0xsfeya3CsC7m8iBzDnMpsbLuo=\ngithub.com/spiffe/go-spiffe/v2 v2.6.0/go.mod h1:gm2SeUoMZEtpnzPNs2Csc0D/gX33k1xIx7lEzqblHEs=\ngithub.com/stoewer/go-strcase v1.2.0/go.mod h1:IBiWB2sKIp3wVVQ3Y035++gc+knqhUQag1KpM8ahLw8=\ngithub.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=\ngithub.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=\ngithub.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE=\ngithub.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=\ngithub.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=\ngithub.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=\ngithub.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=\ngithub.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=\ngithub.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=\ngithub.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=\ngithub.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=\ngithub.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=\ngithub.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=\ngithub.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=\ngithub.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=\ngithub.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=\ngithub.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=\ngithub.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=\ngithub.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw=\ngithub.com/tg123/go-htpasswd v1.2.4 h1:HgH8KKCjdmo7jjXWN9k1nefPBd7Be3tFCTjc2jPraPU=\ngithub.com/tg123/go-htpasswd v1.2.4/go.mod h1:EKThQok9xHkun6NBMynNv6Jmu24A33XdZzzl4Q7H1+0=\ngithub.com/tinylib/msgp v1.3.0 h1:ULuf7GPooDaIlbyvgAxBV/FI7ynli6LZ1/nVUNu+0ww=\ngithub.com/tinylib/msgp v1.3.0/go.mod h1:ykjzy2wzgrlvpDCRc4LA8UXy6D8bzMSuAF3WD57Gok0=\ngithub.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=\ngithub.com/ugorji/go v1.1.4/go.mod h1:uQMGLiO92mf5W77hV/PUCpI3pbzQx3CRekS0kk+RGrc=\ngithub.com/vladimirvivien/gexe v0.1.1 h1:2A0SBaOSKH+cwLVdt6H+KkHZotZWRNLlWygANGw5DxE=\ngithub.com/vladimirvivien/gexe v0.1.1/go.mod h1:LHQL00w/7gDUKIak24n801ABp8C+ni6eBht9vGVst8w=\ngithub.com/vmware-tanzu/crash-diagnostics v0.3.7 h1:6gbv/3o1FzyRLS7Dz/+yVg1Lk1oRBQLyI3d1YTtlTT8=\ngithub.com/vmware-tanzu/crash-diagnostics v0.3.7/go.mod h1:gO8670rd+qdjnJVol674snT/A46GQ27u085kKhZznlM=\ngithub.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM=\ngithub.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg=\ngithub.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=\ngithub.com/xlab/treeprint v0.0.0-20181112141820-a009c3971eca/go.mod h1:ce1O1j6UtZfjr22oyGxGLbauSBp2YVXpARAosm7dHBg=\ngithub.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=\ngithub.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=\ngithub.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=\ngithub.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=\ngithub.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=\ngithub.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=\ngithub.com/zalando/go-keyring v0.2.6 h1:r7Yc3+H+Ux0+M72zacZoItR3UDxeWfKTcabvkI8ua9s=\ngithub.com/zalando/go-keyring v0.2.6/go.mod h1:2TCrxYrbUNYfNS/Kgy/LSrkSQzZ5UPVH85RwfczwvcI=\ngithub.com/zeebo/assert v1.1.0 h1:hU1L1vLTHsnO8x8c9KAR5GmM5QscxHg5RNU5z5qbUWY=\ngithub.com/zeebo/assert v1.1.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0=\ngithub.com/zeebo/blake3 v0.2.4 h1:KYQPkhpRtcqh0ssGYcKLG1JYvddkEA8QwCM/yBqhaZI=\ngithub.com/zeebo/blake3 v0.2.4/go.mod h1:7eeQ6d2iXWRGF6npfaxl2CU+xy2Fjo2gxeyZGCRUjcE=\ngithub.com/zeebo/pcg v1.0.1 h1:lyqfGeWiv4ahac6ttHs+I5hwtH/+1mrhlCtVNQM2kHo=\ngithub.com/zeebo/pcg v1.0.1/go.mod h1:09F0S9iiKrwn9rlI5yjLkmrug154/YRW6KnnXVDM/l4=\ngo.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=\ngo.etcd.io/etcd/api/v3 v3.5.0/go.mod h1:cbVKeC6lCfl7j/8jBhAK6aIYO9XOjdptoxU/nLQcPvs=\ngo.etcd.io/etcd/client/pkg/v3 v3.5.0/go.mod h1:IJHfcCEKxYu1Os13ZdwCwIUTUVGYTSAM3YSwc9/Ac1g=\ngo.etcd.io/etcd/client/v2 v2.305.0/go.mod h1:h9puh54ZTgAKtEbut2oe9P4L/oqKCVB6xsXlzd7alYQ=\ngo.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=\ngo.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=\ngo.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=\ngo.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=\ngo.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=\ngo.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=\ngo.opencensus.io v0.23.0/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E=\ngo.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=\ngo.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=\ngo.opentelemetry.io/contrib/detectors/gcp v1.39.0 h1:kWRNZMsfBHZ+uHjiH4y7Etn2FK26LAGkNFw7RHv1DhE=\ngo.opentelemetry.io/contrib/detectors/gcp v1.39.0/go.mod h1:t/OGqzHBa5v6RHZwrDBJ2OirWc+4q/w2fTbLZwAKjTk=\ngo.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0 h1:q4XOmH/0opmeuJtPsbFNivyl7bCt7yRBbeEm2sC/XtQ=\ngo.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0/go.mod h1:snMWehoOh2wsEwnvvwtDyFCxVeDAODenXHtn5vzrKjo=\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 h1:F7Jx+6hwnZ41NSFTO5q4LYDtJRXBf2PD0rNBkeB/lus=\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0/go.mod h1:UHB22Z8QsdRDrnAtX4PntOl36ajSxcdUMt1sF7Y6E7Q=\ngo.opentelemetry.io/otel v1.40.0 h1:oA5YeOcpRTXq6NN7frwmwFR0Cn3RhTVZvXsP4duvCms=\ngo.opentelemetry.io/otel v1.40.0/go.mod h1:IMb+uXZUKkMXdPddhwAHm6UfOwJyh4ct1ybIlV14J0g=\ngo.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.36.0 h1:rixTyDGXFxRy1xzhKrotaHy3/KXdPhlWARrCgK+eqUY=\ngo.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.36.0/go.mod h1:dowW6UsM9MKbJq5JTz2AMVp3/5iW5I/TStsk8S+CfHw=\ngo.opentelemetry.io/otel/metric v1.40.0 h1:rcZe317KPftE2rstWIBitCdVp89A2HqjkxR3c11+p9g=\ngo.opentelemetry.io/otel/metric v1.40.0/go.mod h1:ib/crwQH7N3r5kfiBZQbwrTge743UDc7DTFVZrrXnqc=\ngo.opentelemetry.io/otel/sdk v1.40.0 h1:KHW/jUzgo6wsPh9At46+h4upjtccTmuZCFAc9OJ71f8=\ngo.opentelemetry.io/otel/sdk v1.40.0/go.mod h1:Ph7EFdYvxq72Y8Li9q8KebuYUr2KoeyHx0DRMKrYBUE=\ngo.opentelemetry.io/otel/sdk/metric v1.40.0 h1:mtmdVqgQkeRxHgRv4qhyJduP3fYJRMX4AtAlbuWdCYw=\ngo.opentelemetry.io/otel/sdk/metric v1.40.0/go.mod h1:4Z2bGMf0KSK3uRjlczMOeMhKU2rhUqdWNoKcYrtcBPg=\ngo.opentelemetry.io/otel/trace v1.40.0 h1:WA4etStDttCSYuhwvEa8OP8I5EWu24lkOzp+ZYblVjw=\ngo.opentelemetry.io/otel/trace v1.40.0/go.mod h1:zeAhriXecNGP/s2SEG3+Y8X9ujcJOTqQ5RgdEJcawiA=\ngo.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5/go.mod h1:nmDLcffg48OtT/PSW0Hg7FvpRQsQh5OSqIylirxKC7o=\ngo.starlark.net v0.0.0-20201006213952-227f4aabceb5/go.mod h1:f0znQkUKRrkk36XxWbGjMqQM8wGv/xHBVE2qc3B5oFU=\ngo.starlark.net v0.0.0-20230525235612-a134d8f9ddca h1:VdD38733bfYv5tUZwEIskMM93VanwNIi5bIKnDrJdEY=\ngo.starlark.net v0.0.0-20230525235612-a134d8f9ddca/go.mod h1:jxU+3+j+71eXOW14274+SmmuW82qJzl6iZSeqEtTGds=\ngo.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=\ngo.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=\ngo.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=\ngo.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=\ngo.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0=\ngo.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU=\ngo.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=\ngo.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=\ngo.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q=\ngo.uber.org/zap v1.17.0/go.mod h1:MXVU+bhUf/A7Xi2HNOnopQOrmycQ5Ih87HtOu4q5SSo=\ngo.uber.org/zap v1.27.1 h1:08RqriUEv8+ArZRYSTXy1LeBScaMpVSTBhCeaZYfMYc=\ngo.uber.org/zap v1.27.1/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=\ngo.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0=\ngo.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8=\ngolang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=\ngolang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=\ngolang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=\ngolang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/crypto v0.0.0-20190820162420-60c769a6c586/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=\ngolang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=\ngolang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=\ngolang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=\ngolang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=\ngolang.org/x/crypto v0.46.0 h1:cKRW/pmt1pKAfetfu+RCEvjvZkA9RimPbh7bhFjGVBU=\ngolang.org/x/crypto v0.46.0/go.mod h1:Evb/oLKmMraqjZ2iQTwDwvCtJkczlDuTmdJXoZVzqU0=\ngolang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=\ngolang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=\ngolang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=\ngolang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek=\ngolang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY=\ngolang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=\ngolang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=\ngolang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=\ngolang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=\ngolang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=\ngolang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 h1:2dVuKD2vS7b0QIHQbpyTISPd0LeHDbnYEryqj5Q1ug8=\ngolang.org/x/exp v0.0.0-20240719175910-8a7402abbf56/go.mod h1:M4RDyNAINzryxdtnbRXRL/OHtkFuWGRjvuhBJpk2IlY=\ngolang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=\ngolang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=\ngolang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=\ngolang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=\ngolang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=\ngolang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs=\ngolang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=\ngolang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=\ngolang.org/x/lint v0.0.0-20201208152925-83fdc39ff7b5/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=\ngolang.org/x/lint v0.0.0-20210508222113-6edffad5e616/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=\ngolang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=\ngolang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=\ngolang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=\ngolang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY=\ngolang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=\ngolang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=\ngolang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/mod v0.30.0 h1:fDEXFVZ/fmCKProc/yAXXUijritrDzahmwwefnjoPFk=\ngolang.org/x/mod v0.30.0/go.mod h1:lAsf5O2EvJeSFMiBxXDki7sCgAxEUcZHXoXMKT4GJKc=\ngolang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20181023162649-9b4f9f5ad519/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20181201002055-351d144fa1fc/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20181220203305-927f97764cc3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190522155817-f3200d17e092/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=\ngolang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=\ngolang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20190827160401-ba9fcec4b297/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=\ngolang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=\ngolang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=\ngolang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=\ngolang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=\ngolang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=\ngolang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=\ngolang.org/x/net v0.0.0-20201224014010-6772e930b67b/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=\ngolang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=\ngolang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=\ngolang.org/x/net v0.0.0-20210316092652-d523dce5a7f4/go.mod h1:RBQZq4jEuRlivfhVLdyRGr576XBO4/greRjx4P4O3yc=\ngolang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM=\ngolang.org/x/net v0.0.0-20210520170846-37e1c6afe023/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=\ngolang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=\ngolang.org/x/net v0.48.0 h1:zyQRTTrjc33Lhh0fBgT/H3oZq9WuvRR5gPC70xpDiQU=\ngolang.org/x/net v0.48.0/go.mod h1:+ndRgGjkh8FGtu1w1FGbEC31if4VrNVMuKTgcAAnQRY=\ngolang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=\ngolang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=\ngolang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=\ngolang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=\ngolang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=\ngolang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20201109201403-9fd604954f58/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20201208152858-08078c50e5b5/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20210402161424-2e8d93401602/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=\ngolang.org/x/oauth2 v0.34.0 h1:hqK/t4AKgbqWkdkcAeI8XLmbK+4m4G5YeQRrmiotGlw=\ngolang.org/x/oauth2 v0.34.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA=\ngolang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4=\ngolang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=\ngolang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20181107165924-66b7b1311ac8/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191002063906-3421d5a6bb1c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191008105621-543471e840be/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200519105757-fe76b779f299/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200625212154-ddb9806d33ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210220050731-9a76102bfb43/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210225134936-a50acf3fe073/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210305230114-8fe3ee5dd75b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210315160823-c6e025ad8005/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210403161142-5e06dd20ab57/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210423185535-09eb48e85fd7/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20211019181941-9d821ace8654/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.40.0 h1:DBZZqJ2Rkml6QMQsZywtnjnnGvHza6BTfYFWY9kjEWQ=\ngolang.org/x/sys v0.40.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=\ngolang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=\ngolang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=\ngolang.org/x/term v0.0.0-20210220032956-6a3ed077a48d/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=\ngolang.org/x/term v0.0.0-20220526004731-065cf7ba2467/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=\ngolang.org/x/term v0.38.0 h1:PQ5pkm/rLO6HnxFR7N2lJHOZX6Kez5Y1gDSJla6jo7Q=\ngolang.org/x/term v0.38.0/go.mod h1:bSEAKrOT1W+VSu9TSCMtoGEOUcKxOKgl3LE5QEF/xVg=\ngolang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=\ngolang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=\ngolang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=\ngolang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=\ngolang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\ngolang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\ngolang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\ngolang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\ngolang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=\ngolang.org/x/text v0.32.0 h1:ZD01bjUt1FQ9WJ0ClOL5vxgxOI/sVCNgX1YtKwcY0mU=\ngolang.org/x/text v0.32.0/go.mod h1:o/rUWzghvpD5TXrTIBuJU77MTaN0ljMWE47kxGJQ7jY=\ngolang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=\ngolang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=\ngolang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=\ngolang.org/x/time v0.0.0-20210723032227-1f47c861a9ac/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=\ngolang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI=\ngolang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4=\ngolang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=\ngolang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=\ngolang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=\ngolang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=\ngolang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=\ngolang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=\ngolang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=\ngolang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=\ngolang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=\ngolang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=\ngolang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=\ngolang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191112195655-aa38f8e97acc/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=\ngolang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=\ngolang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8=\ngolang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=\ngolang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=\ngolang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=\ngolang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE=\ngolang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.0.0-20210108195828-e2f9c7f1fc8e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=\ngolang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=\ngolang.org/x/tools v0.1.2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=\ngolang.org/x/tools v0.39.0 h1:ik4ho21kwuQln40uelmciQPp9SipgNDdrafrYA4TmQQ=\ngolang.org/x/tools v0.39.0/go.mod h1:JnefbkDPyD8UU2kI5fuf8ZX4/yUeh9W877ZeBONxUqQ=\ngolang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngomodules.xyz/jsonpatch/v2 v2.4.0 h1:Ci3iUJyx9UeRx7CeFN8ARgGbkESwJK+KB9lLcWxY/Zw=\ngomodules.xyz/jsonpatch/v2 v2.4.0/go.mod h1:AH3dM2RI6uoBZxn3LVrfvJ3E0/9dG4cSrbuBJT4moAY=\ngonum.org/v1/gonum v0.16.0 h1:5+ul4Swaf3ESvrOnidPp4GZbzf0mxVQpDCYUQE7OJfk=\ngonum.org/v1/gonum v0.16.0/go.mod h1:fef3am4MQ93R2HHpKnLk4/Tbh/s0+wqD5nfa6Pnwy4E=\ngoogle.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=\ngoogle.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=\ngoogle.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=\ngoogle.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=\ngoogle.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=\ngoogle.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=\ngoogle.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=\ngoogle.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=\ngoogle.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=\ngoogle.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=\ngoogle.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=\ngoogle.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=\ngoogle.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=\ngoogle.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=\ngoogle.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM=\ngoogle.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc=\ngoogle.golang.org/api v0.35.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg=\ngoogle.golang.org/api v0.36.0/go.mod h1:+z5ficQTmoYpPn8LCUNVpK5I7hwkpjbcgqA7I34qYtE=\ngoogle.golang.org/api v0.40.0/go.mod h1:fYKFpnQN0DsDSKRVRcQSDQNtqWPfM9i+zNPxepjRCQ8=\ngoogle.golang.org/api v0.41.0/go.mod h1:RkxM5lITDfTzmyKFPt+wGrCJbVfniCr2ool8kTBzRTU=\ngoogle.golang.org/api v0.43.0/go.mod h1:nQsDGjRXMo4lvh5hP0TKqF244gqhGcr/YSIykhUk/94=\ngoogle.golang.org/api v0.44.0/go.mod h1:EBOGZqzyhtvMDoxwS97ctnh0zUmYY6CxqXsc1AvkYD8=\ngoogle.golang.org/api v0.256.0 h1:u6Khm8+F9sxbCTYNoBHg6/Hwv0N/i+V94MvkOSor6oI=\ngoogle.golang.org/api v0.256.0/go.mod h1:KIgPhksXADEKJlnEoRa9qAII4rXcy40vfI8HRqcU964=\ngoogle.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=\ngoogle.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=\ngoogle.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=\ngoogle.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0=\ngoogle.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=\ngoogle.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=\ngoogle.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=\ngoogle.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=\ngoogle.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=\ngoogle.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=\ngoogle.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=\ngoogle.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=\ngoogle.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=\ngoogle.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=\ngoogle.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8=\ngoogle.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=\ngoogle.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA=\ngoogle.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200513103714-09dca8ec2884/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U=\ngoogle.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=\ngoogle.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA=\ngoogle.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20201019141844-1ed22bb0c154/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20201109203340-2640f1f9cdfb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20201201144952-b05cb90ed32e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20201210142538-e3217bee35cc/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20201214200347-8c77b98c765d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210108203827-ffc7fda8c3d7/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210222152913-aa3ee6e6a81c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210226172003-ab064af71705/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210303154014-9728d6b83eeb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210310155132-4ce2db91004e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210319143718-93e7006c17a6/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=\ngoogle.golang.org/genproto v0.0.0-20210402141018-6c239bbf2bb1/go.mod h1:9lPAdzaEmUacj36I+k7YKbEc5CXzPIeORRgDAUOu28A=\ngoogle.golang.org/genproto v0.0.0-20210602131652-f16073e35f0c/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0=\ngoogle.golang.org/genproto v0.0.0-20250603155806-513f23925822 h1:rHWScKit0gvAPuOnu87KpaYtjK5zBMLcULh7gxkCXu4=\ngoogle.golang.org/genproto v0.0.0-20250603155806-513f23925822/go.mod h1:HubltRL7rMh0LfnQPkMH4NPDFEWp0jw3vixw7jEM53s=\ngoogle.golang.org/genproto/googleapis/api v0.0.0-20251202230838-ff82c1b0f217 h1:fCvbg86sFXwdrl5LgVcTEvNC+2txB5mgROGmRL5mrls=\ngoogle.golang.org/genproto/googleapis/api v0.0.0-20251202230838-ff82c1b0f217/go.mod h1:+rXWjjaukWZun3mLfjmVnQi18E1AsFbDN9QdJ5YXLto=\ngoogle.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217 h1:gRkg/vSppuSQoDjxyiGfN4Upv/h/DQmIR10ZU8dh4Ww=\ngoogle.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217/go.mod h1:7i2o+ce6H/6BluujYR+kqX3GKH+dChPTQU19wjRPiGk=\ngoogle.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=\ngoogle.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=\ngoogle.golang.org/grpc v1.21.0/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=\ngoogle.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=\ngoogle.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=\ngoogle.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=\ngoogle.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=\ngoogle.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=\ngoogle.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=\ngoogle.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60=\ngoogle.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk=\ngoogle.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=\ngoogle.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=\ngoogle.golang.org/grpc v1.31.1/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=\ngoogle.golang.org/grpc v1.33.1/go.mod h1:fr5YgcSWrqhRRxogOsw7RzIpsmvOZ6IcH4kBYTpR3n0=\ngoogle.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=\ngoogle.golang.org/grpc v1.34.0/go.mod h1:WotjhfgOW/POjDeRt8vscBtXq+2VjORFy659qA51WJ8=\ngoogle.golang.org/grpc v1.35.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=\ngoogle.golang.org/grpc v1.36.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=\ngoogle.golang.org/grpc v1.36.1/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=\ngoogle.golang.org/grpc v1.38.0/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM=\ngoogle.golang.org/grpc v1.79.3 h1:sybAEdRIEtvcD68Gx7dmnwjZKlyfuc61Dyo9pGXXkKE=\ngoogle.golang.org/grpc v1.79.3/go.mod h1:KmT0Kjez+0dde/v2j9vzwoAScgEPx/Bw1CYChhHLrHQ=\ngoogle.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=\ngoogle.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=\ngoogle.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=\ngoogle.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=\ngoogle.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=\ngoogle.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=\ngoogle.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=\ngoogle.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=\ngoogle.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4=\ngoogle.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=\ngoogle.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=\ngoogle.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=\ngoogle.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=\ngoogle.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE=\ngoogle.golang.org/protobuf v1.36.10/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=\ngopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=\ngopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=\ngopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=\ngopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=\ngopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=\ngopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=\ngopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=\ngopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=\ngopkg.in/evanphx/json-patch.v4 v4.12.0 h1:n6jtcsulIzXPJaxegRbvFNNrZDjbij7ny3gmSPG+6V4=\ngopkg.in/evanphx/json-patch.v4 v4.12.0/go.mod h1:p8EYWUEYMpynmqDbY58zCKCFZw8pRWMG4EsWvDvM72M=\ngopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=\ngopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc=\ngopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw=\ngopkg.in/ini.v1 v1.51.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=\ngopkg.in/ini.v1 v1.62.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=\ngopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo=\ngopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ=\ngopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=\ngopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74=\ngopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.2.3/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=\ngopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=\ngopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=\ngopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=\ngopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=\ngopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=\ngopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=\nhonnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nhonnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nhonnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nhonnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nhonnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=\nhonnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=\nhonnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=\nk8s.io/api v0.22.2/go.mod h1:y3ydYpLJAaDI+BbSe2xmGcqxiWHmWjkEeIbiwHvnPR8=\nk8s.io/api v0.33.3 h1:SRd5t//hhkI1buzxb288fy2xvjubstenEKL9K51KBI8=\nk8s.io/api v0.33.3/go.mod h1:01Y/iLUjNBM3TAvypct7DIj0M0NIZc+PzAHCIo0CYGE=\nk8s.io/apiextensions-apiserver v0.33.3 h1:qmOcAHN6DjfD0v9kxL5udB27SRP6SG/MTopmge3MwEs=\nk8s.io/apiextensions-apiserver v0.33.3/go.mod h1:oROuctgo27mUsyp9+Obahos6CWcMISSAPzQ77CAQGz8=\nk8s.io/apimachinery v0.22.2/go.mod h1:O3oNtNadZdeOMxHFVxOreoznohCpy0z6mocxbZr7oJ0=\nk8s.io/apimachinery v0.33.3 h1:4ZSrmNa0c/ZpZJhAgRdcsFcZOw1PQU1bALVQ0B3I5LA=\nk8s.io/apimachinery v0.33.3/go.mod h1:BHW0YOu7n22fFv/JkYOEfkUYNRN0fj0BlvMFWA7b+SM=\nk8s.io/cli-runtime v0.22.2/go.mod h1:tkm2YeORFpbgQHEK/igqttvPTRIHFRz5kATlw53zlMI=\nk8s.io/cli-runtime v0.33.3 h1:Dgy4vPjNIu8LMJBSvs8W0LcdV0PX/8aGG1DA1W8lklA=\nk8s.io/cli-runtime v0.33.3/go.mod h1:yklhLklD4vLS8HNGgC9wGiuHWze4g7x6XQZ+8edsKEo=\nk8s.io/client-go v0.22.2/go.mod h1:sAlhrkVDf50ZHx6z4K0S40wISNTarf1r800F+RlCF6U=\nk8s.io/client-go v0.33.3 h1:M5AfDnKfYmVJif92ngN532gFqakcGi6RvaOF16efrpA=\nk8s.io/client-go v0.33.3/go.mod h1:luqKBQggEf3shbxHY4uVENAxrDISLOarxpTKMiUuujg=\nk8s.io/gengo v0.0.0-20200413195148-3a45101e95ac/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0=\nk8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE=\nk8s.io/klog/v2 v2.9.0/go.mod h1:hy9LJ/NvuK+iVyP4Ehqva4HxZG/oXyIS3n3Jmire4Ec=\nk8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk=\nk8s.io/klog/v2 v2.130.1/go.mod h1:3Jpz1GvMt720eyJH1ckRHK1EDfpxISzJ7I9OYgaDtPE=\nk8s.io/kube-aggregator v0.33.3 h1:Pa6hQpKJMX0p0D2wwcxXJgu02++gYcGWXoW1z1ZJDfo=\nk8s.io/kube-aggregator v0.33.3/go.mod h1:hwvkUoQ8q6gv0+SgNnlmQ3eUue1zHhJKTHsX7BwxwSE=\nk8s.io/kube-openapi v0.0.0-20210421082810-95288971da7e/go.mod h1:vHXdDvt9+2spS2Rx9ql3I8tycm3H9FDfdUoIuKCefvw=\nk8s.io/kube-openapi v0.0.0-20250318190949-c8a335a9a2ff h1:/usPimJzUKKu+m+TE36gUyGcf03XZEP0ZIKgKj35LS4=\nk8s.io/kube-openapi v0.0.0-20250318190949-c8a335a9a2ff/go.mod h1:5jIi+8yX4RIb8wk3XwBo5Pq2ccx4FP10ohkbSKCZoK8=\nk8s.io/metrics v0.33.3 h1:9CcqBz15JZfISqwca33gdHS8I6XfsK1vA8WUdEnG70g=\nk8s.io/metrics v0.33.3/go.mod h1:Aw+cdg4AYHw0HvUY+lCyq40FOO84awrqvJRTw0cmXDs=\nk8s.io/utils v0.0.0-20210819203725-bdf08cb9a70a/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA=\nk8s.io/utils v0.0.0-20241104100929-3ea5e8cea738 h1:M3sRQVHv7vB20Xc2ybTt7ODCeFj6JSWYFzOFnYeS6Ro=\nk8s.io/utils v0.0.0-20241104100929-3ea5e8cea738/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0=\nrsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=\nrsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=\nrsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=\nsigs.k8s.io/controller-runtime v0.21.0 h1:CYfjpEuicjUecRk+KAeyYh+ouUBn4llGyDYytIGcJS8=\nsigs.k8s.io/controller-runtime v0.21.0/go.mod h1:OSg14+F65eWqIu4DceX7k/+QRAbTTvxeQSNSOQpukWM=\nsigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3 h1:/Rv+M11QRah1itp8VhT6HoVx1Ray9eB4DBr+K+/sCJ8=\nsigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3/go.mod h1:18nIHnGi6636UCz6m8i4DhaJ65T6EruyzmoQqI2BVDo=\nsigs.k8s.io/kustomize/api v0.8.11/go.mod h1:a77Ls36JdfCWojpUqR6m60pdGY1AYFix4AH83nJtY1g=\nsigs.k8s.io/kustomize/kyaml v0.11.0/go.mod h1:GNMwjim4Ypgp/MueD3zXHLRJEjz7RvtPae0AwlvEMFM=\nsigs.k8s.io/randfill v0.0.0-20250304075658-069ef1bbf016/go.mod h1:XeLlZ/jmk4i1HRopwe7/aU3H5n1zNUcX6TM94b3QxOY=\nsigs.k8s.io/randfill v1.0.0 h1:JfjMILfT8A6RbawdsK2JXGBR5AQVfd+9TbzrlneTyrU=\nsigs.k8s.io/randfill v1.0.0/go.mod h1:XeLlZ/jmk4i1HRopwe7/aU3H5n1zNUcX6TM94b3QxOY=\nsigs.k8s.io/structured-merge-diff/v4 v4.0.2/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw=\nsigs.k8s.io/structured-merge-diff/v4 v4.1.2/go.mod h1:j/nl6xW8vLS49O8YvXW1ocPhZawJtm+Yrr7PPRQ0Vg4=\nsigs.k8s.io/structured-merge-diff/v4 v4.6.0 h1:IUA9nvMmnKWcj5jl84xn+T5MnlZKThmUW1TdblaLVAc=\nsigs.k8s.io/structured-merge-diff/v4 v4.6.0/go.mod h1:dDy58f92j70zLsuZVuUX5Wp9vtxXpaZnkPGWeqDfCps=\nsigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc=\nsigs.k8s.io/yaml v1.4.0 h1:Mk1wCc2gy/F0THH0TAp1QYyJNzRm2KCLy3o5ASXVI5E=\nsigs.k8s.io/yaml v1.4.0/go.mod h1:Ejl7/uTz7PSA4eKMyQCUTnhZYNmLIl+5c2lQPGR2BPY=\n" }, { "path": "hack/boilerplate.go.txt", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n" }, { "path": "hack/build-image/Dockerfile", "content": "# Copyright 2018, 2019, 2020 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nFROM --platform=$TARGETPLATFORM golang:1.25-bookworm\n\nARG GOPROXY\n\nENV GO111MODULE=on\n# Use a proxy for go modules to reduce the likelihood of various hosts being down and breaking the build\nENV GOPROXY=${GOPROXY}\n\n# kubebuilder test bundle is separated from kubebuilder. Need to setup it for CI test.\n# Using setup-envtest to download envtest binaries\nRUN go install sigs.k8s.io/controller-runtime/tools/setup-envtest@latest && \\\n mkdir -p /usr/local/kubebuilder/bin && \\\n ENVTEST_ASSETS_DIR=$(setup-envtest use 1.33.0 --bin-dir /usr/local/kubebuilder/bin -p path) && \\\n cp -r ${ENVTEST_ASSETS_DIR}/* /usr/local/kubebuilder/bin/\n\nRUN wget --quiet https://github.com/kubernetes-sigs/kubebuilder/releases/download/v3.2.0/kubebuilder_linux_$(go env GOARCH) && \\\n mv kubebuilder_linux_$(go env GOARCH) /usr/local/kubebuilder/bin/kubebuilder && \\\n chmod +x /usr/local/kubebuilder/bin/kubebuilder\n\n# get controller-tools\nRUN go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.16.5\n\n# get goimports (the revision is pinned so we don't indiscriminately update, but the particular commit\n# is not important)\nRUN go install golang.org/x/tools/cmd/goimports@v0.33.0\n\n# get protoc compiler and golang plugin\nWORKDIR /root\nRUN apt-get update && apt-get install -y unzip\n# protobuf uses bazel cpunames except following\n# if cpu == \"systemz\":\n# cpu = \"s390_64\"\n# elif cpu == \"aarch64\":\n# cpu = \"aarch_64\"\n# elif cpu == \"ppc64\":\n# cpu = \"ppcle_64\"\n# snippet from: https://github.com/protocolbuffers/protobuf/blob/d445953603e66eb8992a39b4e10fcafec8501f24/protobuf_release.bzl#L18-L24\n# cpu names: https://github.com/bazelbuild/platforms/blob/main/cpu/BUILD\nRUN ARCH=$(go env GOARCH) && \\\n if [ \"$ARCH\" = \"s390x\" ] ; then \\\n ARCH=\"s390_64\"; \\\n elif [ \"$ARCH\" = \"arm64\" ] ; then \\\n ARCH=\"aarch_64\"; \\\n elif [ \"$ARCH\" = \"ppc64le\" ] ; then \\\n ARCH=\"ppcle_64\"; \\\n elif [ \"$ARCH\" = \"ppc64\" ] ; then \\\n ARCH=\"ppcle_64\"; \\\n else \\\n ARCH=$(uname -m); \\\n fi && echo \"ARCH=$ARCH\" && \\\n wget --quiet https://github.com/protocolbuffers/protobuf/releases/download/v25.2/protoc-25.2-linux-$ARCH.zip && \\\n unzip protoc-25.2-linux-$ARCH.zip; \\\n rm *.zip && \\\n mv bin/protoc /usr/bin/protoc && \\\n mv include/google /usr/include && \\\n chmod a+x /usr/include/google && \\\n chmod a+x /usr/include/google/protobuf && \\\n chmod a+r -R /usr/include/google && \\\n chmod +x /usr/bin/protoc\nRUN go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.33.0 \\\n && go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.3.0\n\n# get goreleaser\n# goreleaser name template per arch is basically goarch except for amd64 and 386 https://github.com/goreleaser/goreleaser/blob/ec8819a95c5527fae65e5cb41673f5bbc3245fda/.goreleaser.yaml#L167C1-L173C42\n# {{- .ProjectName }}_\n# {{- title .Os }}_\n# {{- if eq .Arch \"amd64\" }}x86_64\n# {{- else if eq .Arch \"386\" }}i386\n# {{- else }}{{ .Arch }}{{ end }}\n# {{- if .Arm }}v{{ .Arm }}{{ end -}}\nRUN ARCH=$(go env GOARCH) && \\\n if [ \"$ARCH\" = \"amd64\" ] ; then \\\n ARCH=\"x86_64\"; \\\n elif [ \"$ARCH\" = \"386\" ] ; then \\\n ARCH=\"i386\"; \\\n elif [ \"$ARCH\" = \"ppc64le\" ] ; then \\\n ARCH=\"ppc64\"; \\\n fi && \\\n wget --quiet \"https://github.com/goreleaser/goreleaser/releases/download/v1.26.2/goreleaser_Linux_$ARCH.tar.gz\" && \\\n tar xvf goreleaser_Linux_$ARCH.tar.gz; \\\n mv goreleaser /usr/bin/goreleaser && \\\n chmod +x /usr/bin/goreleaser\n\n# get golangci-lint\nRUN curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/HEAD/install.sh | sh -s -- -b $(go env GOPATH)/bin v2.5.0\n\n# install kubectl\nRUN curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/$(go env GOARCH)/kubectl\nRUN chmod +x ./kubectl\nRUN mv ./kubectl /usr/local/bin\n\n# Fix the \"dubious ownership\" issue from git when running goreleaser.sh\nRUN echo \"[safe] \\n\\t directory = *\" > /.gitconfig\n" }, { "path": "hack/build-restic.sh", "content": "#!/bin/bash\n\n# Copyright 2020 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -o errexit\nset -o nounset\nset -o pipefail\n\n# Use /output/usr/bin/ as the default output directory as this\n# is the path expected by the Velero Dockerfile.\noutput_dir=${OUTPUT_DIR:-/output/usr/bin}\nrestic_bin=${output_dir}/restic\nbuild_path=$(dirname \"$PWD\")\n\nif [[ -z \"${BIN}\" ]]; then\n echo \"BIN must be set\"\n exit 1\nfi\n\nif [[ \"${BIN}\" != \"velero\" ]]; then\n echo \"${BIN} does not need the restic binary\"\n exit 0\nfi\n\nif [[ -z \"${GOOS}\" ]]; then\n echo \"GOOS must be set\"\n exit 1\nfi\nif [[ -z \"${GOARCH}\" ]]; then\n echo \"GOARCH must be set\"\n exit 1\nfi\nif [[ -z \"${RESTIC_VERSION}\" ]]; then\n echo \"RESTIC_VERSION must be set\"\n exit 1\nfi\n\nmkdir ${build_path}/restic\ngit clone -b v${RESTIC_VERSION} https://github.com/restic/restic.git ${build_path}/restic\npushd ${build_path}/restic\ngit apply /go/src/github.com/vmware-tanzu/velero/hack/fix_restic_cve.txt\ngo run build.go --goos \"${GOOS}\" --goarch \"${GOARCH}\" --goarm \"${GOARM}\" -o ${restic_bin}\nchmod +x ${restic_bin}\npopd\n" }, { "path": "hack/build.sh", "content": "#!/bin/bash\n\n# Copyright 2016 The Kubernetes Authors.\n#\n# Modifications Copyright 2020 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -o errexit\nset -o nounset\nset -o pipefail\n\nif [[ -z \"${PKG}\" ]]; then\n echo \"PKG must be set\"\n exit 1\nfi\nif [[ -z \"${BIN}\" ]]; then\n echo \"BIN must be set\"\n exit 1\nfi\nif [[ -z \"${GOOS}\" ]]; then\n echo \"GOOS must be set\"\n exit 1\nfi\nif [[ -z \"${GOBIN}\" ]]; then\n echo \"GOBIN must be set\"\n exit 1\nfi\nif [[ -z \"${GOARCH}\" ]]; then\n echo \"GOARCH must be set\"\n exit 1\nfi\nif [[ -z \"${VERSION}\" ]]; then\n echo \"VERSION must be set\"\n exit 1\nfi\n\nif [[ -z \"${REGISTRY}\" ]]; then\n echo \"REGISTRY must be set\"\n exit 1\nfi\n\nif [[ -z \"${GIT_SHA}\" ]]; then\n echo \"GIT_SHA must be set\"\n exit 1\nfi\n\nif [[ -z \"${GIT_TREE_STATE}\" ]]; then\n echo \"GIT_TREE_STATE must be set\"\n exit 1\nfi\n\nGCFLAGS=\"\"\nif [[ ${DEBUG:-} = \"1\" ]]; then\n GCFLAGS=\"all=-N -l\"\nfi\n\nexport CGO_ENABLED=0\n\nLDFLAGS=\"-X ${PKG}/pkg/buildinfo.Version=${VERSION}\"\nLDFLAGS=\"${LDFLAGS} -X ${PKG}/pkg/buildinfo.ImageRegistry=${REGISTRY}\"\nLDFLAGS=\"${LDFLAGS} -X ${PKG}/pkg/buildinfo.GitSHA=${GIT_SHA}\"\nLDFLAGS=\"${LDFLAGS} -X ${PKG}/pkg/buildinfo.GitTreeState=${GIT_TREE_STATE}\"\n\nif [[ -z \"${OUTPUT_DIR:-}\" ]]; then\n OUTPUT_DIR=.\nfi\nOUTPUT=${OUTPUT_DIR}/${BIN}\nif [[ \"${GOOS}\" = \"windows\" ]]; then\n OUTPUT=\"${OUTPUT}.exe\"\nfi\n\ngo build \\\n -o ${OUTPUT} \\\n -gcflags \"${GCFLAGS}\" \\\n -installsuffix \"static\" \\\n -ldflags \"${LDFLAGS}\" \\\n ${PKG}/cmd/${BIN}\n" }, { "path": "hack/changelog-check.sh", "content": "#!/bin/bash\n\n# Copyright 2020 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\nset +x\n\nif [[ -z \"$CI\" ]]; then\n echo \"This script is intended to be run only on Github Actions.\" >&2\n exit 1\nfi\n\nCHANGELOG_PATH='changelogs/unreleased'\n\n# https://help.github.com/en/actions/reference/events-that-trigger-workflows#pull-request-event-pull_request\n# GITHUB_REF is something like \"refs/pull/:prNumber/merge\"\npr_number=$(echo $GITHUB_REF | cut -d / -f 3)\n\nchange_log_file=\"${CHANGELOG_PATH}/${pr_number}-*\"\n\nif ls ${change_log_file} 1> /dev/null 2>&1; then\n echo \"changelog for PR ${pr_number} exists\"\n exit 0\nelse\n echo \"PR ${pr_number} is missing a changelog. Please refer https://velero.io/docs/main/code-standards/#adding-a-changelog and add a changelog.\"\n exit 1\nfi\n\n" }, { "path": "hack/ci-check.sh", "content": "#!/usr/bin/env bash\n\n# If we're doing push build, as opposed to a PR, always run make ci\nif [ \"$TRAVIS_PULL_REQUEST\" == \"false\" ]; then\n make ci\n # Exit script early, returning make ci's error\n exit $?\nfi\n\n# Only run `make ci` if files outside of the site directory changed in the branch\n# In a PR build, $TRAVIS_BRANCH is the destination branch.\nif [[ $(git diff --name-only $TRAVIS_BRANCH | grep --invert-match site/) ]]; then\n make ci\nelse\n echo \"Skipping make ci since nothing outside of site directory changed.\"\n exit 0\nfi\n" }, { "path": "hack/crd-gen/v1/main.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\n// This code embeds the CRD manifests in ../bases in ../crds/crds.go\n\npackage main\n\nimport (\n\t\"bytes\"\n\t\"compress/gzip\"\n\t\"fmt\"\n\t\"io\"\n\t\"log\"\n\t\"os\"\n\t\"text/template\"\n)\n\n// This is relative to config/crd/crds\nconst goHeaderFile = \"../../../../hack/boilerplate.go.txt\"\n\nconst tpl = `{{.GoHeader}}\n// Code generated by crds_generate.go; DO NOT EDIT.\n\npackage crds\n\nimport (\n\t\"bytes\"\n\t\"compress/gzip\"\n\t\"io\"\n\n\tapiextinstall \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/install\"\n\tapiextv1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1\"\n\t\"k8s.io/client-go/kubernetes/scheme\"\n)\n\nvar rawCRDs = [][]byte{\n{{- range .RawCRDs }}\n\t[]byte({{ . }}),\n{{- end }}\n}\n\nvar CRDs = crds()\n\nfunc crds() []*apiextv1.CustomResourceDefinition {\n\tapiextinstall.Install(scheme.Scheme)\n\tdecode := scheme.Codecs.UniversalDeserializer().Decode\n\tvar objs []*apiextv1.CustomResourceDefinition\n\tfor _, crd := range rawCRDs {\n\t\tgzr, err := gzip.NewReader(bytes.NewReader(crd))\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\tbytes, err := io.ReadAll(gzr)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\tgzr.Close()\n\n\t\tobj, _, err := decode(bytes, nil, nil)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\tobjs = append(objs, obj.(*apiextv1.CustomResourceDefinition))\n\t}\n\treturn objs\n}\n`\n\ntype templateData struct {\n\tGoHeader string\n\tRawCRDs []string\n}\n\nfunc main() {\n\theaderBytes, err := os.ReadFile(goHeaderFile)\n\tif err != nil {\n\t\tlog.Fatalln(err)\n\t}\n\n\tdata := templateData{\n\t\tGoHeader: string(headerBytes),\n\t}\n\n\t// This is relative to config/crd/crds\n\tmanifests, err := os.ReadDir(\"../bases\")\n\tif err != nil {\n\t\tlog.Fatalln(err)\n\t}\n\n\tfor _, crd := range manifests {\n\t\tfile, err := os.Open(\"../bases/\" + crd.Name())\n\t\tif err != nil {\n\t\t\tlog.Fatalln(err)\n\t\t}\n\n\t\t// gzip compress manifest\n\t\tvar buf bytes.Buffer\n\t\tgzw := gzip.NewWriter(&buf)\n\t\tif _, err := io.Copy(gzw, file); err != nil {\n\t\t\tlog.Fatalln(err)\n\t\t}\n\t\tfile.Close()\n\t\tgzw.Close()\n\n\t\tdata.RawCRDs = append(data.RawCRDs, fmt.Sprintf(\"%q\", buf.Bytes()))\n\t}\n\n\tt, err := template.New(\"crd\").Parse(tpl)\n\tif err != nil {\n\t\tlog.Fatalln(err)\n\t}\n\n\tout, err := os.Create(\"crds.go\")\n\tif err != nil {\n\t\tlog.Fatalln(err)\n\t}\n\n\tif err := t.Execute(out, data); err != nil {\n\t\tlog.Fatalln(err)\n\t}\n}\n" }, { "path": "hack/docker-push.sh", "content": "#!/bin/bash\n\n# Copyright 2020 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# docker-push is invoked by the CI/CD system to deploy docker images to Docker Hub.\n# It will build images for all commits to main and all git tags.\n# The highest, non-prerelease semantic version will also be given the `latest` tag.\n\nset +x\n\nif [[ -z \"$CI\" ]]; then\n echo \"This script is intended to be run only on Github Actions.\" >&2\n exit 1\nfi\n\n# Return value is written into HIGHEST\nHIGHEST=\"\"\nfunction highest_release() {\n # Loop through the tags since pre-release versions come before the actual versions.\n # Iterate til we find the first non-pre-release\n\n # This is not necessarily the most recently made tag; instead, we want it to be the highest semantic version.\n # The most recent tag could potentially be a lower semantic version, made as a point release for a previous series.\n # As an example, if v1.3.0 exists and we create v1.2.2, v1.3.0 should still be `latest`.\n # `git describe --tags $(git rev-list --tags --max-count=1)` would return the most recently made tag.\n\n for t in $(git tag -l --sort=-v:refname);\n do\n # If the tag has alpha, beta or rc in it, it's not \"latest\"\n if [[ \"$t\" == *\"beta\"* || \"$t\" == *\"alpha\"* || \"$t\" == *\"rc\"* ]]; then\n continue\n fi\n HIGHEST=\"$t\"\n break\n done\n}\n\ntriggeredBy=$(echo $GITHUB_REF | cut -d / -f 2)\nif [[ \"$triggeredBy\" == \"heads\" ]]; then\n BRANCH=$(echo $GITHUB_REF | cut -d / -f 3)\n TAG=\nelif [[ \"$triggeredBy\" == \"tags\" ]]; then\n BRANCH=\n TAG=$(echo $GITHUB_REF | cut -d / -f 3)\nfi\n\n# if both BRANCH and TAG are empty, then it's triggered by PR. Use target branch instead.\n# BRANCH is needed in docker buildx command to set as image tag.\n# When action is triggered by PR, just build container without pushing, so set type to local.\n# When action is triggered by PUSH, need to push container, so set type to registry.\nif [[ -z $BRANCH && -z $TAG ]]; then\n echo \"Test Velero container build without pushing, when Dockerfile is changed by PR.\"\n BRANCH=\"${GITHUB_BASE_REF}-container\"\n OUTPUT_TYPE=\"tar\"\nelse\n OUTPUT_TYPE=\"registry\"\nfi\n\nTAG_LATEST=false\nif [[ ! -z \"$TAG\" ]]; then\n echo \"We're building tag $TAG\"\n VERSION=\"$TAG\"\n # Explicitly checkout tags when building from a git tag.\n # This is not needed when building from main\n git fetch --tags\n # Calculate the latest release if there's a tag.\n highest_release\n if [[ \"$TAG\" == \"$HIGHEST\" ]]; then\n TAG_LATEST=true\n fi\nelse\n echo \"We're on branch $BRANCH\"\n VERSION=\"$BRANCH\"\n if [[ \"$VERSION\" == release-* ]]; then\n VERSION=${VERSION}-dev\n fi\nfi\n\nif [[ -z \"$BUILD_OS\" ]]; then\n BUILD_OS=\"linux,windows\"\nfi\n\nif [[ -z \"$BUILD_ARCH\" ]]; then\n BUILD_ARCH=\"amd64,arm64\"\nfi\n\n# Debugging info\necho \"Highest tag found: $HIGHEST\"\necho \"BRANCH: $BRANCH\"\necho \"TAG: $TAG\"\necho \"TAG_LATEST: $TAG_LATEST\"\necho \"VERSION: $VERSION\"\necho \"BUILD_OS: $BUILD_OS\"\necho \"BUILD_ARCH: $BUILD_ARCH\"\n\necho \"Building and pushing container images.\"\n\n\nVERSION=\"$VERSION\" \\\nTAG_LATEST=\"$TAG_LATEST\" \\\nBUILD_OS=\"$BUILD_OS\" \\\nBUILD_ARCH=\"$BUILD_ARCH\" \\\nBUILD_OUTPUT_TYPE=$OUTPUT_TYPE \\\nmake all-containers" }, { "path": "hack/fix_restic_cve.txt", "content": "diff --git a/go.mod b/go.mod\nindex 5f939c481..f6205aa3c 100644\n--- a/go.mod\n+++ b/go.mod\n@@ -24,32 +24,31 @@ require (\n \tgithub.com/restic/chunker v0.4.0\n \tgithub.com/spf13/cobra v1.6.1\n \tgithub.com/spf13/pflag v1.0.5\n-\tgolang.org/x/crypto v0.5.0\n-\tgolang.org/x/net v0.5.0\n-\tgolang.org/x/oauth2 v0.4.0\n-\tgolang.org/x/sync v0.1.0\n-\tgolang.org/x/sys v0.4.0\n-\tgolang.org/x/term v0.4.0\n-\tgolang.org/x/text v0.6.0\n-\tgoogle.golang.org/api v0.106.0\n+\tgolang.org/x/crypto v0.45.0\n+\tgolang.org/x/net v0.47.0\n+\tgolang.org/x/oauth2 v0.28.0\n+\tgolang.org/x/sync v0.18.0\n+\tgolang.org/x/sys v0.38.0\n+\tgolang.org/x/term v0.37.0\n+\tgolang.org/x/text v0.31.0\n+\tgoogle.golang.org/api v0.114.0\n )\n \n require (\n-\tcloud.google.com/go v0.108.0 // indirect\n-\tcloud.google.com/go/compute v1.15.1 // indirect\n-\tcloud.google.com/go/compute/metadata v0.2.3 // indirect\n-\tcloud.google.com/go/iam v0.10.0 // indirect\n+\tcloud.google.com/go v0.110.0 // indirect\n+\tcloud.google.com/go/compute/metadata v0.3.0 // indirect\n+\tcloud.google.com/go/iam v0.13.0 // indirect\n \tgithub.com/Azure/azure-sdk-for-go/sdk/internal v1.1.2 // indirect\n \tgithub.com/cpuguy83/go-md2man/v2 v2.0.2 // indirect\n \tgithub.com/dnaeon/go-vcr v1.2.0 // indirect\n \tgithub.com/dustin/go-humanize v1.0.0 // indirect\n \tgithub.com/felixge/fgprof v0.9.3 // indirect\n \tgithub.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect\n-\tgithub.com/golang/protobuf v1.5.2 // indirect\n+\tgithub.com/golang/protobuf v1.5.3 // indirect\n \tgithub.com/google/pprof v0.0.0-20230111200839-76d1ae5aea2b // indirect\n \tgithub.com/google/uuid v1.3.0 // indirect\n-\tgithub.com/googleapis/enterprise-certificate-proxy v0.2.1 // indirect\n-\tgithub.com/googleapis/gax-go/v2 v2.7.0 // indirect\n+\tgithub.com/googleapis/enterprise-certificate-proxy v0.2.3 // indirect\n+\tgithub.com/googleapis/gax-go/v2 v2.7.1 // indirect\n \tgithub.com/inconshreveable/mousetrap v1.1.0 // indirect\n \tgithub.com/json-iterator/go v1.1.12 // indirect\n \tgithub.com/klauspost/cpuid/v2 v2.2.3 // indirect\n@@ -63,11 +62,13 @@ require (\n \tgo.opencensus.io v0.24.0 // indirect\n \tgolang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect\n \tgoogle.golang.org/appengine v1.6.7 // indirect\n-\tgoogle.golang.org/genproto v0.0.0-20230110181048-76db0878b65f // indirect\n-\tgoogle.golang.org/grpc v1.52.0 // indirect\n-\tgoogle.golang.org/protobuf v1.28.1 // indirect\n+\tgoogle.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect\n+\tgoogle.golang.org/grpc v1.56.3 // indirect\n+\tgoogle.golang.org/protobuf v1.33.0 // indirect\n \tgopkg.in/ini.v1 v1.67.0 // indirect\n \tgopkg.in/yaml.v3 v3.0.1 // indirect\n )\n \n-go 1.18\n+go 1.24.0\n+\n+toolchain go1.24.11\ndiff --git a/go.sum b/go.sum\nindex 026e1d2fa..4a37e7ac7 100644\n--- a/go.sum\n+++ b/go.sum\n@@ -1,23 +1,24 @@\n cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=\n-cloud.google.com/go v0.108.0 h1:xntQwnfn8oHGX0crLVinvHM+AhXvi3QHQIEcX/2hiWk=\n-cloud.google.com/go v0.108.0/go.mod h1:lNUfQqusBJp0bgAg6qrHgYFYbTB+dOiob1itwnlD33Q=\n-cloud.google.com/go/compute v1.15.1 h1:7UGq3QknM33pw5xATlpzeoomNxsacIVvTqTTvbfajmE=\n-cloud.google.com/go/compute v1.15.1/go.mod h1:bjjoF/NtFUrkD/urWfdHaKuOPDR5nWIs63rR+SXhcpA=\n-cloud.google.com/go/compute/metadata v0.2.3 h1:mg4jlk7mCAj6xXp9UJ4fjI9VUI5rubuGBW5aJ7UnBMY=\n-cloud.google.com/go/compute/metadata v0.2.3/go.mod h1:VAV5nSsACxMJvgaAuX6Pk2AawlZn8kiOGuCv6gTkwuA=\n-cloud.google.com/go/iam v0.10.0 h1:fpP/gByFs6US1ma53v7VxhvbJpO2Aapng6wabJ99MuI=\n-cloud.google.com/go/iam v0.10.0/go.mod h1:nXAECrMt2qHpF6RZUZseteD6QyanL68reN4OXPw0UWM=\n-cloud.google.com/go/longrunning v0.3.0 h1:NjljC+FYPV3uh5/OwWT6pVU+doBqMg2x/rZlE+CamDs=\n+cloud.google.com/go v0.110.0 h1:Zc8gqp3+a9/Eyph2KDmcGaPtbKRIoqq4YTlL4NMD0Ys=\n+cloud.google.com/go v0.110.0/go.mod h1:SJnCLqQ0FCFGSZMUNUf84MV3Aia54kn7pi8st7tMzaY=\n+cloud.google.com/go/compute/metadata v0.3.0 h1:Tz+eQXMEqDIKRsmY3cHTL6FVaynIjX2QxYC4trgAKZc=\n+cloud.google.com/go/compute/metadata v0.3.0/go.mod h1:zFmK7XCadkQkj6TtorcaGlCW1hT1fIilQDwofLpJ20k=\n+cloud.google.com/go/iam v0.13.0 h1:+CmB+K0J/33d0zSQ9SlFWUeCCEn5XJA0ZMZ3pHE9u8k=\n+cloud.google.com/go/iam v0.13.0/go.mod h1:ljOg+rcNfzZ5d6f1nAUJ8ZIxOaZUVoS14bKCtaLZ/D0=\n+cloud.google.com/go/longrunning v0.4.1 h1:v+yFJOfKC3yZdY6ZUI933pIYdhyhV8S3NpWrXWmg7jM=\n+cloud.google.com/go/longrunning v0.4.1/go.mod h1:4iWDqhBZ70CvZ6BfETbvam3T8FMvLK+eFj0E6AaRQTo=\n cloud.google.com/go/storage v1.28.1 h1:F5QDG5ChchaAVQhINh24U99OWHURqrW8OmQcGKXcbgI=\n cloud.google.com/go/storage v1.28.1/go.mod h1:Qnisd4CqDdo6BGs2AD5LLnEsmSQ80wQ5ogcBBKhU86Y=\n github.com/Azure/azure-sdk-for-go/sdk/azcore v1.3.0 h1:VuHAcMq8pU1IWNT/m5yRaGqbK0BiQKHT8X4DTp9CHdI=\n github.com/Azure/azure-sdk-for-go/sdk/azcore v1.3.0/go.mod h1:tZoQYdDZNOiIjdSn0dVWVfl0NEPGOJqVLzSrcFk4Is0=\n github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.1.0 h1:QkAcEIAKbNL4KoFr4SathZPhDhF4mVwpBMFlYjyAqy8=\n+github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.1.0/go.mod h1:bhXu1AjYL+wutSL/kpSq6s7733q2Rb0yuot9Zgfqa/0=\n github.com/Azure/azure-sdk-for-go/sdk/internal v1.1.2 h1:+5VZ72z0Qan5Bog5C+ZkgSqUbeVUd9wgtHOrIKuc5b8=\n github.com/Azure/azure-sdk-for-go/sdk/internal v1.1.2/go.mod h1:eWRD7oawr1Mu1sLCawqVc0CUiF43ia3qQMxLscsKQ9w=\n github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v0.5.1 h1:BMTdr+ib5ljLa9MxTJK8x/Ds0MbBb4MfuW5BL0zMJnI=\n github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v0.5.1/go.mod h1:c6WvOhtmjNUWbLfOG1qxM/q0SPvQNSVJvolm+C52dIU=\n github.com/AzureAD/microsoft-authentication-library-for-go v0.5.1 h1:BWe8a+f/t+7KY7zH2mqygeUD0t8hNFXe08p1Pb3/jKE=\n+github.com/AzureAD/microsoft-authentication-library-for-go v0.5.1/go.mod h1:Vt9sXTKwMyGcOxSmLDMnGPgqsUg7m8pe215qMLrDXw4=\n github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=\n github.com/Julusian/godocdown v0.0.0-20170816220326-6d19f8ff2df8/go.mod h1:INZr5t32rG59/5xeltqoCJoNY7e5x/3xoY9WSWVWg74=\n github.com/anacrolix/fuse v0.2.0 h1:pc+To78kI2d/WUjIyrsdqeJQAesuwpGxlI3h1nAv3Do=\n@@ -54,6 +55,7 @@ github.com/felixge/fgprof v0.9.3/go.mod h1:RdbpDgzqYVh/T9fPELJyV7EYJuHB55UTEULNu\n github.com/go-ole/go-ole v1.2.6 h1:/Fpf6oFPoeFik9ty7siob0G6Ke8QvQEuVcuChpwXzpY=\n github.com/go-ole/go-ole v1.2.6/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0=\n github.com/golang-jwt/jwt v3.2.1+incompatible h1:73Z+4BJcrTC+KczS6WvTPvRGOp1WmfEP4Q1lOd9Z/+c=\n+github.com/golang-jwt/jwt v3.2.1+incompatible/go.mod h1:8pz2t5EyA70fFQQSrl6XZXzqecmYZeUEB8OUGHkxJ+I=\n github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=\n github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\n github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=\n@@ -70,8 +72,8 @@ github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvq\n github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=\n github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=\n github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=\n-github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw=\n-github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=\n+github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg=\n+github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=\n github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=\n github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=\n github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=\n@@ -82,17 +84,18 @@ github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/\n github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=\n github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=\n github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=\n-github.com/google/martian/v3 v3.2.1 h1:d8MncMlErDFTwQGBK1xhv026j9kqhvw1Qv9IbWT1VLQ=\n+github.com/google/martian/v3 v3.3.2 h1:IqNFLAmvJOgVlpdEBiQbDc2EwKW77amAycfTuWKdfvw=\n+github.com/google/martian/v3 v3.3.2/go.mod h1:oBOf6HBosgwRXnUGWUB05QECsc6uvmMiJ3+6W4l/CUk=\n github.com/google/pprof v0.0.0-20211214055906-6f57359322fd/go.mod h1:KgnwoLYCZ8IQu3XUZ8Nc/bM9CCZFOyjUNOSygVozoDg=\n github.com/google/pprof v0.0.0-20230111200839-76d1ae5aea2b h1:8htHrh2bw9c7Idkb7YNac+ZpTqLMjRpI+FWu51ltaQc=\n github.com/google/pprof v0.0.0-20230111200839-76d1ae5aea2b/go.mod h1:dDKJzRmX4S37WGHujM7tX//fmj1uioxKzKxz3lo4HJo=\n github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=\n github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=\n github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=\n-github.com/googleapis/enterprise-certificate-proxy v0.2.1 h1:RY7tHKZcRlk788d5WSo/e83gOyyy742E8GSs771ySpg=\n-github.com/googleapis/enterprise-certificate-proxy v0.2.1/go.mod h1:AwSRAtLfXpU5Nm3pW+v7rGDHp09LsPtGY9MduiEsR9k=\n-github.com/googleapis/gax-go/v2 v2.7.0 h1:IcsPKeInNvYi7eqSaDjiZqDDKu5rsmunY0Y1YupQSSQ=\n-github.com/googleapis/gax-go/v2 v2.7.0/go.mod h1:TEop28CZZQ2y+c0VxMUmu1lV+fQx57QpBWsYpwqHJx8=\n+github.com/googleapis/enterprise-certificate-proxy v0.2.3 h1:yk9/cqRKtT9wXZSsRH9aurXEpJX+U6FLtpYTdC3R06k=\n+github.com/googleapis/enterprise-certificate-proxy v0.2.3/go.mod h1:AwSRAtLfXpU5Nm3pW+v7rGDHp09LsPtGY9MduiEsR9k=\n+github.com/googleapis/gax-go/v2 v2.7.1 h1:gF4c0zjUP2H/s/hEGyLA3I0fA2ZWjzYiONAD6cvPr8A=\n+github.com/googleapis/gax-go/v2 v2.7.1/go.mod h1:4orTrqY6hXxxaUL4LHIPl6lGo8vAE38/qKbhSAKP6QI=\n github.com/hashicorp/golang-lru/v2 v2.0.1 h1:5pv5N1lT1fjLg2VQ5KWc7kmucp2x/kvFOnxuVTqZ6x4=\n github.com/hashicorp/golang-lru/v2 v2.0.1/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=\n github.com/ianlancetaylor/demangle v0.0.0-20210905161508-09a460cdf81d/go.mod h1:aYm2/VgdVmcIU8iMfdMvDMsRAQjcfZSKFby6HOFvi/w=\n@@ -114,6 +117,7 @@ github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=\n github.com/kurin/blazer v0.5.4-0.20211030221322-ba894c124ac6 h1:nz7i1au+nDzgExfqW5Zl6q85XNTvYoGnM5DHiQC0yYs=\n github.com/kurin/blazer v0.5.4-0.20211030221322-ba894c124ac6/go.mod h1:4FCXMUWo9DllR2Do4TtBd377ezyAJ51vB5uTBjt0pGU=\n github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=\n+github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=\n github.com/minio/md5-simd v1.1.2 h1:Gdi1DZK69+ZVMoNHRXJyNcxrMA4dSxoYHZSQbirFg34=\n github.com/minio/md5-simd v1.1.2/go.mod h1:MzdKDxYpY2BT9XQFocsiZf/NKVtR7nkE4RoEpN+20RM=\n github.com/minio/minio-go/v7 v7.0.46 h1:Vo3tNmNXuj7ME5qrvN4iadO7b4mzu/RSFdUkUhaPldk=\n@@ -129,6 +133,7 @@ github.com/modocache/gover v0.0.0-20171022184752-b58185e213c5/go.mod h1:caMODM3P\n github.com/ncw/swift/v2 v2.0.1 h1:q1IN8hNViXEv8Zvg3Xdis4a3c4IlIGezkYz09zQL5J0=\n github.com/ncw/swift/v2 v2.0.1/go.mod h1:z0A9RVdYPjNjXVo2pDOPxZ4eu3oarO1P91fTItcb+Kg=\n github.com/pkg/browser v0.0.0-20210115035449-ce105d075bb4 h1:Qj1ukM4GlMWXNdMBuXcXfz/Kw9s1qm0CLY32QxuSImI=\n+github.com/pkg/browser v0.0.0-20210115035449-ce105d075bb4/go.mod h1:N6UoU20jOqggOuDwUaBQpluzLNDqif3kq9z2wpdYEfQ=\n github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=\n github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=\n github.com/pkg/profile v1.7.0 h1:hnbDkaNWPCLMO9wGLdBFTIZvzDrDfBM2072E1S9gJkA=\n@@ -172,8 +177,8 @@ golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACk\n golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\n golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=\n golang.org/x/crypto v0.0.0-20211215153901-e495a2d5b3d3/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=\n-golang.org/x/crypto v0.5.0 h1:U/0M97KRkSFvyD/3FSmdP5W5swImpNgle/EHFhOsQPE=\n-golang.org/x/crypto v0.5.0/go.mod h1:NK/OQwhpMQP3MwtdjgLlYHnH9ebylxKWv3e0fK+mkQU=\n+golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q=\n+golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4=\n golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=\n golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=\n golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=\n@@ -189,17 +194,17 @@ golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLL\n golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\n golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=\n golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=\n-golang.org/x/net v0.5.0 h1:GyT4nK/YDHSqa1c4753ouYCDajOYKTja9Xb/OHtgvSw=\n-golang.org/x/net v0.5.0/go.mod h1:DivGGAXEgPSlEBzxGzZI+ZLohi+xUj054jfeKui00ws=\n+golang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY=\n+golang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU=\n golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=\n-golang.org/x/oauth2 v0.4.0 h1:NF0gk8LVPg1Ml7SSbGyySuoxdsXitj7TvgvuRxIMc/M=\n-golang.org/x/oauth2 v0.4.0/go.mod h1:RznEsdpjGAINPTOF0UH/t+xJ75L18YO3Ho6Pyn+uRec=\n+golang.org/x/oauth2 v0.28.0 h1:CrgCKl8PPAVtLnU3c+EDw6x11699EWlsDeWNWKdIOkc=\n+golang.org/x/oauth2 v0.28.0/go.mod h1:onh5ek6nERTohokkhCD/y2cV4Do3fxFHFuAejCkRWT8=\n golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\n golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\n golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\n golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\n-golang.org/x/sync v0.1.0 h1:wsuoTGHzEhffawBOhz5CYhcrV4IdKZbEyZjBMuTp12o=\n-golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\n+golang.org/x/sync v0.18.0 h1:kr88TuHDroi+UVf+0hZnirlk8o8T+4MrK6mr60WkH/I=\n+golang.org/x/sync v0.18.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=\n golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\n golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\n golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\n@@ -214,17 +219,17 @@ golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBc\n golang.org/x/sys v0.0.0-20220408201424-a24fb2fb8a0f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\n golang.org/x/sys v0.0.0-20220704084225-05e143d24a9e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\n golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\n-golang.org/x/sys v0.4.0 h1:Zr2JFtRQNX3BCZ8YtxRE9hNJYC8J6I1MVbMg6owUp18=\n-golang.org/x/sys v0.4.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\n+golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc=\n+golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=\n golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=\n-golang.org/x/term v0.4.0 h1:O7UWfv5+A2qiuulQk30kVinPoMtoIPeVaKLEgLpVkvg=\n-golang.org/x/term v0.4.0/go.mod h1:9P2UbLfCdcvo3p/nzKvsmas4TnlujnuoV9hGgYzW1lQ=\n+golang.org/x/term v0.37.0 h1:8EGAD0qCmHYZg6J17DvsMy9/wJ7/D/4pV/wfnld5lTU=\n+golang.org/x/term v0.37.0/go.mod h1:5pB4lxRNYYVZuTLmy8oR2BH8dflOR+IbTYFD8fi3254=\n golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=\n golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=\n golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\n golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\n-golang.org/x/text v0.6.0 h1:3XmdazWV+ubf7QgHSTWeykHOci5oeekaGJBLkrkaw4k=\n-golang.org/x/text v0.6.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=\n+golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM=\n+golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM=\n golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\n golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\n golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=\n@@ -237,8 +242,8 @@ golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8T\n golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\n golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 h1:H2TDz8ibqkAF6YGhCdN3jS9O0/s90v0rJh3X/OLHEUk=\n golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2/go.mod h1:K8+ghG5WaK9qNqU5K3HdILfMLy1f3aNYFI/wnl100a8=\n-google.golang.org/api v0.106.0 h1:ffmW0faWCwKkpbbtvlY/K/8fUl+JKvNS5CVzRoyfCv8=\n-google.golang.org/api v0.106.0/go.mod h1:2Ts0XTHNVWxypznxWOYUeI4g3WdP9Pk2Qk58+a/O9MY=\n+google.golang.org/api v0.114.0 h1:1xQPji6cO2E2vLiI+C/XiFAnsn1WV3mjaEwGLhi3grE=\n+google.golang.org/api v0.114.0/go.mod h1:ifYI2ZsFK6/uGddGfAD5BMxlnkBqCmqHSDUVi45N5Yg=\n google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=\n google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=\n google.golang.org/appengine v1.6.7 h1:FZR1q0exgwxzPzp/aF+VccGrSfxfPpkBqjIIEq3ru6c=\n@@ -246,15 +251,15 @@ google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCID\n google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=\n google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=\n google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=\n-google.golang.org/genproto v0.0.0-20230110181048-76db0878b65f h1:BWUVssLB0HVOSY78gIdvk1dTVYtT1y8SBWtPYuTJ/6w=\n-google.golang.org/genproto v0.0.0-20230110181048-76db0878b65f/go.mod h1:RGgjbofJ8xD9Sq1VVhDM1Vok1vRONV+rg+CjzG4SZKM=\n+google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 h1:KpwkzHKEF7B9Zxg18WzOa7djJ+Ha5DzthMyZYQfEn2A=\n+google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1/go.mod h1:nKE/iIaLqn2bQwXBg8f1g2Ylh6r5MN5CmZvuzZCgsCU=\n google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=\n google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=\n google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=\n google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=\n google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=\n-google.golang.org/grpc v1.52.0 h1:kd48UiU7EHsV4rnLyOJRuP/Il/UHE7gdDAQ+SZI7nZk=\n-google.golang.org/grpc v1.52.0/go.mod h1:pu6fVzoFb+NBYNAvQL08ic+lvB2IojljRYuun5vorUY=\n+google.golang.org/grpc v1.56.3 h1:8I4C0Yq1EjstUzUJzpcRVbuYA2mODtEmpWiQoN/b2nc=\n+google.golang.org/grpc v1.56.3/go.mod h1:I9bI3vqKfayGqPUAwGdOSu7kt6oIJLixfffKrpXqQ9s=\n google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=\n google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=\n google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=\n@@ -266,14 +271,15 @@ google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpAD\n google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=\n google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=\n google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=\n-google.golang.org/protobuf v1.28.1 h1:d0NfwRgPtno5B1Wa6L2DAG+KivqkdutMf1UhdNx175w=\n-google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=\n+google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI=\n+google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=\n gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=\n gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=\n gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=\n gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=\n gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\n gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=\n+gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=\n gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=\n gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=\n gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=\n" }, { "path": "hack/issue-template-gen/main.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\n// This code renders the IssueTemplate string in pkg/cmd/cli/bug/bug.go to\n// .github/ISSUE_TEMPLATE/bug_report.md via the hack/update-generated-issue-template.sh script.\n\npackage main\n\nimport (\n\t\"log\"\n\t\"os\"\n\t\"text/template\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/bug\"\n)\n\nfunc main() {\n\toutTemplateFilename := os.Args[1]\n\toutFile, err := os.OpenFile(outTemplateFilename, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0644)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer outFile.Close()\n\ttmpl, err := template.New(\"ghissue\").Parse(bug.IssueTemplate)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\terr = tmpl.Execute(outFile, bug.VeleroBugInfo{})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n}\n" }, { "path": "hack/lint.sh", "content": "#!/bin/bash\n#\n# Copyright 2020 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# Printing out cache status\ngolangci-lint cache status\n\n# Enable GL_DEBUG line below for debug messages for golangci-lint\n# export GL_DEBUG=loader,gocritic,env\nCMD=\"golangci-lint run\"\necho \"Running $CMD\"\n\neval $CMD\n" }, { "path": "hack/release-tools/brew-update.sh", "content": "#!/usr/bin/env bash\n\n# This script assumes that 2 environment variables are defined outside of it:\n# VELERO_VERSION - a full version version string, starting with v. example: v1.4.2\n# HOMEBREW_GITHUB_API_TOKEN - the GitHub API token that the brew command will use to create a PR on the user's behalf.\n\n\n# Check if brew is found on the user's $PATH; exit if not.\nif [ -z $(which brew) ];\nthen\n echo \"Homebrew must first be installed to use this script!\"\n exit 1\nfi\n\n# GitHub URL which contains the source code archive for the tagged release\nURL=https://github.com/vmware-tanzu/velero/archive/$VELERO_VERSION.tar.gz\n\n# Update brew so we're sure we have the latest Velero formula\nbrew update\n\n# Invoke brew's helper function, which will run all their tests and end up opening a browser with the resulting PR.\nbrew bump-formula-pr velero --url=$URL\n" }, { "path": "hack/release-tools/changelog.sh", "content": "#!/bin/bash\n\n# Copyright 2018 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -o errexit\nset -o nounset\nset -o pipefail\n\nfunction join { local IFS=\"$1\"; shift; echo \"$*\"; }\n\nCHANGELOG_PATH='changelogs/unreleased'\nUNRELEASED=$(ls -t ${CHANGELOG_PATH})\necho -e \"Generating CHANGELOG markdown from ${CHANGELOG_PATH}\\n\"\nfor entry in $UNRELEASED\ndo\n IFS=$'-' read -ra pruser <<<\"$entry\"\n contents=$(cat ${CHANGELOG_PATH}/${entry})\n pr=${pruser[0]}\n user=$(join '-' ${pruser[@]:1})\n echo \" * ${contents} (#${pr}, @${user})\"\ndone\necho -e \"\\nCopy and paste the list above in to the appropriate CHANGELOG file.\"\necho \"Be sure to run: git rm ${CHANGELOG_PATH}/*\"\n" }, { "path": "hack/release-tools/chk_version.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage main\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"regexp\"\n)\n\n// This regex should match both our GA format (example: v1.4.3) and pre-release formats (v1.2.4-beta.2, v1.5.0-rc.1)\n// The following sub-capture groups are defined:\n//\n//\tmajor\n//\tminor\n//\tpatch\n//\tprerelease (this will be alpha/beta/rc followed by a \".\", followed by 1 or more digits (alpha.5)\nvar releaseRegex = regexp.MustCompile(`^v(?P[[:digit:]]+)\\.(?P[[:digit:]]+)\\.(?P[[:digit:]]+)(-{1}(?P(alpha|beta|rc)\\.[[:digit:]]+))*`)\n\n// This small program exists because checking the VELERO_VERSION rules in bash is difficult, and difficult to test for correctness.\n// Calling it with --verify will verify whether or not the VELERO_VERSION environment variable is a valid version string, without parsing for its components.\n// Calling it without --verify will try to parse the version into its component pieces.\nfunc main() {\n\tveleroVersion := os.Getenv(\"VELERO_VERSION\")\n\n\tsubmatches := reSubMatchMap(releaseRegex, veleroVersion)\n\n\t// Didn't match the regex, exit.\n\tif len(submatches) == 0 {\n\t\tfmt.Printf(\"VELERO_VERSION of %s was not valid. Please correct the value and retry.\", veleroVersion)\n\t\tos.Exit(1)\n\t}\n\n\tif len(os.Args) > 1 && os.Args[1] == \"--verify\" {\n\t\tos.Exit(0)\n\t}\n\n\t// Send these in a bash variable format to stdout, so that they can be consumed by bash scripts that call the go program.\n\tfmt.Printf(\"VELERO_MAJOR=%s\\n\", submatches[\"major\"])\n\tfmt.Printf(\"VELERO_MINOR=%s\\n\", submatches[\"minor\"])\n\tfmt.Printf(\"VELERO_PATCH=%s\\n\", submatches[\"patch\"])\n\tfmt.Printf(\"VELERO_PRERELEASE=%s\\n\", submatches[\"prerelease\"])\n}\n\n// reSubMatchMap returns a map with the named submatches within a regular expression populated as keys, and their matched values within a given string as values.\n// If no matches are found, a nil map is returned\nfunc reSubMatchMap(r *regexp.Regexp, s string) map[string]string {\n\tmatch := r.FindStringSubmatch(s)\n\tsubmatches := make(map[string]string)\n\tif len(match) == 0 {\n\t\treturn submatches\n\t}\n\tfor i, name := range r.SubexpNames() {\n\t\t// 0 will always be empty from the return values of SubexpNames's documentation, so skip it.\n\t\tif i != 0 {\n\t\t\tsubmatches[name] = match[i]\n\t\t}\n\t}\n\n\treturn submatches\n}\n" }, { "path": "hack/release-tools/chk_version_test.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage main\n\nimport (\n\t\"fmt\"\n\t\"testing\"\n)\n\nfunc TestRegexMatching(t *testing.T) {\n\ttests := []struct {\n\t\tversion string\n\t\texpectMatch bool\n\t}{\n\t\t{\n\t\t\tversion: \"v1.4.0\",\n\t\t\texpectMatch: true,\n\t\t},\n\t\t{\n\t\t\tversion: \"v2.0.0\",\n\t\t\texpectMatch: true,\n\t\t},\n\t\t{\n\t\t\tversion: \"v1.5.0-alpha.1\",\n\t\t\texpectMatch: true,\n\t\t},\n\t\t{\n\t\t\tversion: \"v1.16.1320-beta.14\",\n\t\t\texpectMatch: true,\n\t\t},\n\t\t{\n\t\t\tversion: \"1.0.0\",\n\t\t\texpectMatch: false,\n\t\t},\n\t\t{\n\t\t\t// this is true because while the \"--\" is invalid, v1.0.0 is a valid part of the regex\n\t\t\tversion: \"v1.0.0--beta.1\",\n\t\t\texpectMatch: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tname := fmt.Sprintf(\"Testing version string %s\", test.version)\n\t\tt.Run(name, func(t *testing.T) {\n\t\t\tresults := reSubMatchMap(releaseRegex, test.version)\n\n\t\t\tif len(results) == 0 && test.expectMatch {\n\t\t\t\tt.Fail()\n\t\t\t}\n\n\t\t\tif len(results) > 0 && !test.expectMatch {\n\t\t\t\tfmt.Printf(\"%v\", results)\n\t\t\t\tt.Fail()\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "hack/release-tools/gen-docs.sh", "content": "#!/bin/bash\n\n# Copyright 2020 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# gen-docs.sh is used for the \"make gen-docs\" target. It generates a new \n# versioned docs directory under site/content/docs. It follows\n# the following process:\n# 1. Copies the contents of the most recently tagged docs directory into the new\n# directory, to establish a useful baseline to diff against.\n# 2. Adds all copied content from step 1 to git's staging area via 'git add'.\n# 3. Replaces the contents of the new docs directory with the contents of the\n# 'main' docs directory, updating any version-specific links (e.g. to a\n# specific branch of the GitHub repository) to use the new version\n# 4. Copies the previous version's ToC file and runs 'git add' to establish\n# a useful baseline to diff against.\n# 5. Replaces the content of the new ToC file with the main ToC.\n# 6. Update site/config.yaml and site/_data/toc-mapping.yml to include entries\n# for the new version.\n#\n# The unstaged changes in the working directory can now easily be diff'ed against the\n# staged changes using 'git diff' to review all docs changes made since the previous \n# tagged version. Once the unstaged changes are ready, they can be added to the\n# staging area using 'git add' and then committed.\n#\n# NEW_DOCS_VERSION defines the version that the docs will be tagged with \n# (i.e. what’s in the URL, what shows up in the version dropdown on the site). \n# This should be formatted as either v1.4 (for any GA release, including minor), or v1.5.0-beta.1/v1.5.0-rc.1 (for an alpha/beta/RC).\n\n# To run gen-docs: \"VELERO_VERSION=v1.4.0 NEW_DOCS_VERSION=v1.4 PREVIOUS_DOCS_VERSION= make gen-docs\"\n# Note: if PREVIOUS_DOCS_VERSION is not set, the script will copy from the \n# latest version.\n#\n# **NOTE**: there are additional manual steps required to finalize the process of generating\n# a new versioned docs site. The full process is documented in site/README-HUGO.md\n\nset -o errexit\nset -o nounset\nset -o pipefail\n\nDOCS_DIRECTORY=site/content/docs\nDATA_DOCS_DIRECTORY=site/data/docs\nCONFIG_FILE=site/config.yaml\nMAIN_BRANCH=main\n\n# don't run if there's already a directory for the target docs version\nif [[ -d $DOCS_DIRECTORY/$NEW_DOCS_VERSION ]]; then\n echo \"ERROR: $DOCS_DIRECTORY/$NEW_DOCS_VERSION already exists\"\n exit 1\nfi\n\n# get the alphabetically last item in $DOCS_DIRECTORY to use as PREVIOUS_DOCS_VERSION\n# if not explicitly specified by the user\nif [[ -z \"${PREVIOUS_DOCS_VERSION:-}\" ]]; then\n echo \"PREVIOUS_DOCS_VERSION was not specified, getting the latest version\"\n PREVIOUS_DOCS_VERSION=$(ls -1 $DOCS_DIRECTORY/ | tail -n 1)\nfi\n\n# make a copy of the previous versioned docs dir\necho \"Creating copy of docs directory $DOCS_DIRECTORY/$PREVIOUS_DOCS_VERSION in $DOCS_DIRECTORY/$NEW_DOCS_VERSION\"\ncp -r $DOCS_DIRECTORY/${PREVIOUS_DOCS_VERSION}/ $DOCS_DIRECTORY/${NEW_DOCS_VERSION}/\n\n# 'git add' the previous version's docs as-is so we get a useful diff when we copy the $MAIN_BRANCH docs in\necho \"Running 'git add' for previous version's doc contents to use as a base for diff\"\ngit add -f $DOCS_DIRECTORY/${NEW_DOCS_VERSION}\n\n# now copy the contents of $DOCS_DIRECTORY/$MAIN_BRANCH into the same directory so we can get a nice\n# git diff of what changed since previous version\necho \"Copying $DOCS_DIRECTORY/$MAIN_BRANCH/ to $DOCS_DIRECTORY/${NEW_DOCS_VERSION}/\"\nrm -rf $DOCS_DIRECTORY/${NEW_DOCS_VERSION}/ && cp -r $DOCS_DIRECTORY/$MAIN_BRANCH/ $DOCS_DIRECTORY/${NEW_DOCS_VERSION}/\n\n# make a copy of the previous versioned ToC\nNEW_DOCS_TOC=\"$(echo ${NEW_DOCS_VERSION} | tr . -)-toc\"\nPREVIOUS_DOCS_TOC=\"$(echo ${PREVIOUS_DOCS_VERSION} | tr . -)-toc\"\n\necho \"Creating copy of $DATA_DOCS_DIRECTORY/$PREVIOUS_DOCS_TOC.yml at $DATA_DOCS_DIRECTORY/$NEW_DOCS_TOC.yml\"\ncp $DATA_DOCS_DIRECTORY/$PREVIOUS_DOCS_TOC.yml $DATA_DOCS_DIRECTORY/$NEW_DOCS_TOC.yml\n\n# 'git add' the previous version's ToC content as-is so we get a useful diff when we copy the $MAIN_BRANCH ToC in\necho \"Running 'git add' for previous version's ToC to use as a base for diff\"\ngit add $DATA_DOCS_DIRECTORY/$NEW_DOCS_TOC.yml\n\n# now copy the $MAIN_BRANCH ToC so we can get a nice git diff of what changed since previous version\necho \"Copying $DATA_DOCS_DIRECTORY/$MAIN_BRANCH-toc.yml to $DATA_DOCS_DIRECTORY/$NEW_DOCS_TOC.yml\"\nrm $DATA_DOCS_DIRECTORY/$NEW_DOCS_TOC.yml && cp $DATA_DOCS_DIRECTORY/$MAIN_BRANCH-toc.yml $DATA_DOCS_DIRECTORY/$NEW_DOCS_TOC.yml\n\n# replace known version-specific links -- the sed syntax is slightly different in OS X and Linux,\n# so check which OS we're running on.\nif [[ $(uname) == \"Darwin\" ]]; then\n echo \"[OS X] updating version-specific links\"\n find $DOCS_DIRECTORY/${NEW_DOCS_VERSION} -type f -name \"*.md\" | xargs sed -i '' \"s|https://velero.io/docs/$MAIN_BRANCH|https://velero.io/docs/$VELERO_VERSION|g\"\n find $DOCS_DIRECTORY/${NEW_DOCS_VERSION} -type f -name \"*.md\" | xargs sed -i '' \"s|https://github.com/vmware-tanzu/velero/blob/$MAIN_BRANCH|https://github.com/vmware-tanzu/velero/blob/$VELERO_VERSION|g\"\n find $DOCS_DIRECTORY/${NEW_DOCS_VERSION} -type f -name \"_index.md\" | xargs sed -i '' \"s|version: $MAIN_BRANCH|version: $NEW_DOCS_VERSION|g\"\n\n echo \"[OS X] Updating latest version in $CONFIG_FILE\"\n sed -i '' \"s/latest: ${PREVIOUS_DOCS_VERSION}/latest: ${NEW_DOCS_VERSION}/\" $CONFIG_FILE\n\n # newlines and lack of indentation are requirements for this sed syntax\n # which is doing an append\n echo \"[OS X] Adding latest version to versions list in $CONFIG_FILE\"\n sed -i '' \"/- $MAIN_BRANCH/a\\\\\n\\ \\ \\ \\ - ${NEW_DOCS_VERSION}\n\" $CONFIG_FILE\n\n echo \"[OS X] Adding ToC mapping entry\"\n sed -i '' \"/$MAIN_BRANCH: $MAIN_BRANCH-toc/a\\\\\n${NEW_DOCS_VERSION}: ${NEW_DOCS_TOC}\n\" $DATA_DOCS_DIRECTORY/toc-mapping.yml\n\nelse\n echo \"[Linux] updating version-specific links\"\n find $DOCS_DIRECTORY/${NEW_DOCS_VERSION} -type f -name \"*.md\" | xargs sed -i'' \"s|https://velero.io/docs/$MAIN_BRANCH|https://velero.io/docs/$VELERO_VERSION|g\"\n find $DOCS_DIRECTORY/${NEW_DOCS_VERSION} -type f -name \"*.md\" | xargs sed -i'' \"s|https://github.com/vmware-tanzu/velero/blob/$MAIN_BRANCH|https://github.com/vmware-tanzu/velero/blob/$VELERO_VERSION|g\"\n\n echo \"[Linux] Updating latest version in $CONFIG_FILE\"\n sed -i'' \"s/latest: ${PREVIOUS_DOCS_VERSION}/latest: ${NEW_DOCS_VERSION}/\" $CONFIG_FILE\n \n echo \"[Linux] Adding latest version to versions list in $CONFIG_FILE\"\n sed -i'' \"/- $MAIN_BRANCH/a - ${NEW_DOCS_VERSION}\" $CONFIG_FILE\n \n echo \"[Linux] Adding ToC mapping entry\"\n sed -i'' \"/$MAIN_BRANCH: $MAIN_BRANCH-toc/a ${NEW_DOCS_VERSION}: ${NEW_DOCS_TOC}\" $DATA_DOCS_DIRECTORY/toc-mapping.yml\nfi\n\necho \"Success! $DOCS_DIRECTORY/$NEW_DOCS_VERSION has been created.\"\necho \"\"\necho \"The next steps are:\"\necho \" 1. Consult site/README-HUGO.md for further manual steps required to finalize the new versioned docs generation.\"\necho \" 2. Run a 'git diff' to review all changes made to the docs since the previous version.\"\necho \" 3. Make any manual changes/corrections necessary.\"\necho \" 4. Run 'git add' to stage all unstaged changes, then 'git commit'.\"\n" }, { "path": "hack/release-tools/goreleaser.sh", "content": "#!/bin/bash\n\n# Copyright 2018 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -o errexit\nset -o nounset\nset -o pipefail\n\nif [[ -z \"${GITHUB_TOKEN}\" ]]; then\n echo \"GITHUB_TOKEN must be set\"\n exit 1\nfi\n\n# TODO derive this from the major+minor version\nif [ -z \"${RELEASE_NOTES_FILE}\" ]; then\n echo \"RELEASE_NOTES_FILE must be set\"\n exit 1\nfi\n\nif [ -z \"${REGISTRY}\" ]; then\n echo \"REGISTRY must be set\"\n exit 1\nfi\n\nGIT_DIRTY=$(git status --porcelain 2> /dev/null)\nif [[ -z \"${GIT_DIRTY}\" ]]; then\n export GIT_TREE_STATE=clean\nelse\n export GIT_TREE_STATE=dirty\nfi\n\n# Verify .goreleaser.yml format first.\necho \"Start to verify .goreleaser.yml format\"\ngoreleaser check\n\n# $PUBLISH must explicitly be set to 'TRUE' for goreleaser\n# to publish the release to GitHub.\nif [[ \"${PUBLISH:-}\" != \"TRUE\" ]]; then\n echo \"Not set to publish\"\n goreleaser release \\\n --clean \\\n --release-notes=\"${RELEASE_NOTES_FILE}\" \\\n --snapshot # Generate an unversioned snapshot release, skipping all validations and without publishing any artifacts (implies --skip-publish, --skip-announce and --skip-validate)\nelse\n echo \"Getting ready to publish\"\n goreleaser release \\\n --clean \\\n --release-notes=\"${RELEASE_NOTES_FILE}\"\nfi\n" }, { "path": "hack/release-tools/tag-release.sh", "content": "#!/usr/bin/env bash\n\n# Copyright 2020 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\n# This script will do the necessary checks and actions to create a release of Velero. It will:\n# - validate that all prerequisites are met\n# - verify the version string is what the user expects.\n# - create a git tag\n# - push the created git tag to GitHub\n# - run GoReleaser\n\n# The following variables are needed:\n\n# - $VELERO_VERSION: defines the tag of Velero that any https://github.com/vmware-tanzu/velero/...\n# links in the docs should redirect to.\n# - $REMOTE: defines the remote that should be used when pushing tags and branches. Defaults to \"upstream\"\n# - $publish: TRUE/FALSE value where FALSE (or not including it) will indicate a dry-run, and TRUE, or simply adding 'publish',\n# will tag the release with the $VELERO_VERSION and push the tag to a remote named 'upstream'.\n# - $GITHUB_TOKEN: Needed to run the goreleaser process to generate a GitHub release. \n# Use https://github.com/settings/tokens/new?scopes=repo if you don't already have a token.\n# Regenerate an existing token: https://github.com/settings/tokens.\n# You may regenerate the token for every release if you prefer.\n# See https://goreleaser.com/environment/ for more details.\n\n# This script is meant to be a combination of documentation and executable.\n# If you have questions at any point, please stop and ask!\n\n# Fail on any error.\nset -eo pipefail\n\n# Directory in which the script itself resides, so we can use it for calling programs that are in the same directory.\nDIR=\"$( cd \"$( dirname \"${BASH_SOURCE[0]}\" )\" >/dev/null 2>&1 && pwd )\"\n\n# Default to using upstream as the remote\nremote=${REMOTE:-upstream}\n\n# Parse out the branch we're on so we can switch back to it at the end of a dry-run, where we delete the tag. Requires git v1.8.1+\nupstream_branch=$(git symbolic-ref --short HEAD)\n\nfunction tag_and_push() {\n echo \"Tagging $VELERO_VERSION\"\n git tag $VELERO_VERSION || true\n \n if [[ $publish == \"TRUE\" ]]; then\n echo \"Pushing $VELERO_VERSION\"\n git push \"$remote\" $VELERO_VERSION\n fi\n}\n\n# Default to a dry-run mode\npublish=FALSE\nif [[ \"$1\" = \"publish\" ]]; then\n publish=TRUE\nfi\n\n# For now, have the person doing the release pass in the VELERO_VERSION variable as an environment variable.\n# In the future, we might be able to inspect git via `git describe --abbrev=0` to get a hint for it.\nif [[ -z \"$VELERO_VERSION\" ]]; then\n printf \"The \\$VELERO_VERSION environment variable is not set. Please set it with\\n\\texport VELERO_VERSION=v\\nthen try again.\"\n exit 1\nfi\n\n# Make sure the user's provided their github token, so we can give it to goreleaser.\nif [[ -z \"$GITHUB_TOKEN\" ]]; then\n printf \"The GITHUB_TOKEN environment variable is not set. Please set it with\\n\\t export GITHUB_TOKEN=\\n then try again.\"\n exit 1\nfi\n\n# Ensure that we have a clean working tree before we let any changes happen, especially important for cutting release branches.\nif [[ -n $(git status --short) ]]; then \n echo \"Your git working directory is dirty! Please clean up untracked files and stash any changes before proceeding.\"\n exit 3\nfi\n\n# Make sure that there's no issue with the environment variable's format before trying to eval the parsed version.\nif ! go run $DIR/chk_version.go --verify; then\n exit 2\nfi\n# Since we're past the validation of the VELERO_VERSION, parse the version's individual components.\neval $(go run $DIR/chk_version.go)\n\nprintf \"To clarify, you've provided a version string of $VELERO_VERSION.\\n\"\nprintf \"Based on this, the following assumptions have been made: \\n\"\n\n# $VELERO_PATCH gets populated by the chk_version.go script that parses and verifies the given version format\n# If we've got a patch release, we assume the tag is on release branch.\nif [[ \"$VELERO_PATCH\" != 0 ]]; then\n printf \"*\\t This is a patch release.\\n\"\n ON_RELEASE_BRANCH=TRUE\nfi\n\n# $VELERO_PRERELEASE gets populated by the chk_version.go script that parses and verifies the given version format\n# If we've got a GA release, we assume the tag is on release branch.\n# -n is \"string is non-empty\"\n[[ -n $VELERO_PRERELEASE ]] && printf \"*\\t This is a pre-release.\\n\"\n\n# -z is \"string is empty\"\nif [[ -z $VELERO_PRERELEASE ]]; then\n printf \"*\\t This is a GA release.\\n\"\n ON_RELEASE_BRANCH=TRUE\nfi\n\nif [[ \"$ON_RELEASE_BRANCH\" == \"TRUE\" ]]; then\n release_branch_name=release-$VELERO_MAJOR.$VELERO_MINOR\n printf \"*\\t The commit to tag is on branch: %s. Please make sure this branch has been created.\\n\" $release_branch_name\nfi\n\nif [[ $publish == \"TRUE\" ]]; then\n echo \"If this is all correct, press enter/return to proceed to TAG THE RELEASE and UPLOAD THE TAG TO GITHUB.\"\nelse\n echo \"If this is all correct, press enter/return to proceed to TAG THE RELEASE and PROCEED WITH THE DRY-RUN.\"\nfi\n\necho \"Otherwise, press ctrl-c to CANCEL the process without making any changes.\"\n\nread -p \"Ready to continue? \"\n\necho \"Alright, let's go.\"\n\necho \"Pulling down all git tags and branches before doing any work.\"\ngit fetch \"$remote\" --tags\n\nif [[ -n $release_branch_name ]]; then\n # Tag on release branch\n remote_release_branch_name=\"$remote/$release_branch_name\"\n\n # Determine whether the local and remote release branches already exist\n local_branch=$(git branch | { grep \"$release_branch_name\" || true; })\n remote_branch=$(git branch -r | { grep \"$remote_release_branch_name\" || true;})\n if [[ -z $remote_branch ]]; then\n echo \"The branch $remote_release_branch_name must be created before you tag the release.\"\n exit 1\n fi\n if [[ -z $local_branch ]]; then\n # Remote branch exists, but does not exist locally. Checkout and track the remote branch.\n git checkout --track \"$remote_release_branch_name\"\n else\n # Checkout the local release branch and ensure it is up to date with the remote\n git checkout \"$release_branch_name\"\n git pull --set-upstream \"$remote\" \"$release_branch_name\"\n fi\n tag_and_push\nelse\n echo \"Checking out $remote/main.\"\n git checkout \"$remote\"/main\n tag_and_push\nfi\n\n\necho \"Invoking Goreleaser to create the GitHub release.\"\nRELEASE_NOTES_FILE=changelogs/CHANGELOG-$VELERO_MAJOR.$VELERO_MINOR.md \\\n PUBLISH=$publish \\\n make release\n\nif [[ $publish == \"FALSE\" ]]; then\n # Delete the local tag so we don't potentially conflict when it's re-run for real.\n # This also means we won't have to just ignore existing tags in tag_and_push, which could be a problem if there's an existing tag.\n echo \"Dry run complete. Deleting git tag $VELERO_VERSION\"\n git checkout $upstream_branch\n git tag -d $VELERO_VERSION\nfi\n\n" }, { "path": "hack/test.sh", "content": "#!/bin/bash\n\n# Copyright 2016 The Kubernetes Authors.\n# Modifications Copyright 2020 The Velero Contributors\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -o errexit\nset -o nounset\nset -o pipefail\n\nexport CGO_ENABLED=0\n\nTARGETS=($(go list ./pkg/... ./internal/...| grep -vE \"/pkg/builder|pkg/apis|pkg/test|pkg/generated|pkg/plugin/generated|mocks|internal/restartabletest\"))\nTARGETS+=(\n ./cmd/...\n)\n\nif [[ ${#@} -ne 0 ]]; then\n TARGETS=(\"$@\")\nfi\n\necho \"Running all short tests in:\" \"${TARGETS[@]}\"\n\nif [[ -n \"${GOFLAGS:-}\" ]]; then\n echo \"GOFLAGS: ${GOFLAGS}\"\nfi\n\n# After bumping up \"sigs.k8s.io/controller-runtime\" to v0.10.2, get the error \"panic: mkdir /.cache/kubebuilder-envtest: permission denied\"\n# when running this script with \"make test\" command. This is caused by that \"make test\" runs inside a container with user and group specified,\n# but the user and group don't exist inside the container, when the code(https://github.com/kubernetes-sigs/controller-runtime/blob/v0.10.2/pkg/internal/testing/addr/manager.go#L44)\n# tries to get the cache directory, it gets the directory \"/\" and then get the permission error when trying to create directory under \"/\".\n# Specifying the cache directory by environment variable \"XDG_CACHE_HOME\" to workaround it\nXDG_CACHE_HOME=/tmp/ go test -installsuffix \"static\" -short -timeout 1200s -coverprofile=coverage.out \"${TARGETS[@]}\"\necho \"Success!\"\n" }, { "path": "hack/update-1fmt.sh", "content": "#!/bin/bash\n#\n# Copyright 2017 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -o errexit\nset -o nounset\nset -o pipefail\n\nif [[ ${1:-} == '--verify' ]]; then\n # List file diffs that need formatting updates\n MODE='-d'\n ACTION='Verifying'\nelse\n # Write formatting updates to files\n MODE='-w'\n ACTION='Updating'\nfi\n\nif ! command -v goimports > /dev/null; then\n echo 'goimports is missing - please run \"go get golang.org/x/tools/cmd/goimports\"'\n exit 1\nfi\n\nfiles=\"$(find . -type f -name '*.go' -not -path './.go/*' -not -path './vendor/*' -not -path './site/*' -not -path './.git/*' -not -path '*/generated/*' -not -name 'zz_generated*' -not -path '*/mocks/*')\"\necho \"${ACTION} gofmt\"\noutput=$(gofmt \"${MODE}\" -s ${files})\nif [[ -n \"${output}\" ]]; then\n VERIFY_FMT_FAILED=1\n echo \"${output}\"\nfi\nif [[ -n \"${VERIFY_FMT_FAILED:-}\" ]]; then\n echo \"${ACTION} gofmt - failed! Please run 'make update'.\"\nelse\n echo \"${ACTION} gofmt - done!\"\nfi\n\necho \"${ACTION} goimports\"\noutput=$(goimports \"${MODE}\" -local github.com/vmware-tanzu/velero ${files})\nif [[ -n \"${output}\" ]]; then\n VERIFY_IMPORTS_FAILED=1\n echo \"${output}\"\nfi\nif [[ -n \"${VERIFY_IMPORTS_FAILED:-}\" ]]; then\n echo \"${ACTION} goimports - failed! Please run 'make update'.\"\nelse\n echo \"${ACTION} goimports - done!\"\nfi\n\nif [[ -n \"${VERIFY_FMT_FAILED:-}\" || -n \"${VERIFY_IMPORTS_FAILED:-}\" ]]; then\n exit 1\nfi\n" }, { "path": "hack/update-2proto.sh", "content": "#!/bin/bash -e\n#\n# Copyright 2017, 2019 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nHACK_DIR=$(dirname \"${BASH_SOURCE}\")\n\necho \"Updating plugin proto\"\n\necho protoc --version\nprotoc \\\n -I pkg/plugin/proto/ \\\n -I /usr/include \\\n --go_out=pkg/plugin/generated/ \\\n --go_opt=module=github.com/vmware-tanzu/velero/pkg/plugin/generated \\\n --go-grpc_out=pkg/plugin/generated \\\n --go-grpc_opt=paths=source_relative \\\n --go-grpc_opt=require_unimplemented_servers=false \\\n $(find pkg/plugin/proto -name '*.proto')\n\necho \"Updating plugin proto - done!\"\n" }, { "path": "hack/update-3generated-crd-code.sh", "content": "#!/bin/bash\n#\n# Copyright the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nset -o errexit\nset -o nounset\nset -o pipefail\nset -o xtrace\n\n# this script expects to be run from the root of the Velero repo.\n\nif [[ -z \"${GOPATH}\" ]]; then\n GOPATH=~/go\nfi\n\nif ! command -v controller-gen > /dev/null; then\n echo \"controller-gen is missing\"\n exit 1\nfi\n\n# Generate CRD for v1.\ncontroller-gen \\\n crd:crdVersions=v1 \\\n paths=./pkg/apis/velero/v1/... \\\n paths=./pkg/controller/... \\\n output:crd:artifacts:config=config/crd/v1/bases \\\n object \\\n paths=./pkg/apis/velero/v1/...\n\n# Generate CRD for v2alpha1.\ncontroller-gen \\\n crd:crdVersions=v1 \\\n paths=./pkg/apis/velero/v2alpha1/... \\\n paths=./pkg/controller/... \\\n output:crd:artifacts:config=config/crd/v2alpha1/bases \\\n object \\\n paths=./pkg/apis/velero/v2alpha1/...\n\n# Generate RBAC.\ncontroller-gen \\\n paths=./pkg/apis/velero/v1/... \\\n paths=./pkg/apis/velero/v2alpha1/... \\\n paths=./pkg/controller/... \\\n rbac:roleName=velero-perms\n\ngo generate ./config/crd/v1/crds\n\ngo generate ./config/crd/v2alpha1/crds\n" }, { "path": "hack/update-4generated-issue-template.sh", "content": "#!/bin/bash -e\n#\n# Copyright 2018, 2019 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nVELERO_ROOT=$(dirname ${BASH_SOURCE})/..\nBIN=${VELERO_ROOT}/_output/bin\n\nmkdir -p ${BIN}\n\necho \"Updating generated Github issue template\"\ngo build -o ${BIN}/issue-tmpl-gen ./hack/issue-template-gen/main.go\n\nif [[ $# -gt 1 ]]; then\n echo \"usage: ${BASH_SOURCE} [OUTPUT_FILE]\"\n exit 1\nfi\n\nOUTPUT_ISSUE_FILE=\"$1\"\nif [[ -z \"${OUTPUT_ISSUE_FILE}\" ]]; then\n OUTPUT_ISSUE_FILE=${VELERO_ROOT}/.github/ISSUE_TEMPLATE/bug_report.md\nfi\n\n${BIN}/issue-tmpl-gen ${OUTPUT_ISSUE_FILE} \necho \"Success!\"\n" }, { "path": "hack/update-all.sh", "content": "#!/bin/bash -e\n#\n# Copyright 2017 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nHACK_DIR=$(dirname \"${BASH_SOURCE}\")\n\necho \"Running all update scripts\"\n\nfor f in ${HACK_DIR}/update-*.sh; do\n if [[ $f = \"${HACK_DIR}/update-all.sh\" ]]; then\n continue\n fi\n $f\ndone\n" }, { "path": "hack/verify-all.sh", "content": "#!/bin/bash -e\n#\n# Copyright 2017 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nHACK_DIR=$(dirname \"${BASH_SOURCE}\")\n\necho \"Running all verification scripts\"\n\nfor f in ${HACK_DIR}/verify-*.sh; do\n if [[ $f = \"${HACK_DIR}/verify-all.sh\" ]]; then\n continue\n fi\n $f\ndone\n" }, { "path": "hack/verify-fmt.sh", "content": "#!/bin/bash\n#\n# Copyright 2017 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nHACK_DIR=$(dirname \"${BASH_SOURCE[0]}\")\n\"${HACK_DIR}\"/update-1fmt.sh --verify\n" }, { "path": "hack/verify-generated-crd-code.sh", "content": "#!/bin/bash -e\n#\n# Copyright the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nHACK_DIR=$(dirname \"${BASH_SOURCE}\")\n\n${HACK_DIR}/update-3generated-crd-code.sh\n\n# ensure no changes to generated CRDs\nif ! git diff --exit-code config/crd/v1/crds/crds.go config/crd/v2alpha1/crds/crds.go &> /dev/null; then\n # revert changes to state before running CRD generation to stay consistent\n # with code-generator `--verify-only` option which discards generated changes\n git checkout config/crd\n\n echo \"CRD verification - failed! Generated CRDs are out-of-date, please run 'make update' and 'git add' the generated file(s).\"\n exit 1\nfi\n" }, { "path": "hack/verify-generated-issue-template.sh", "content": "#!/bin/bash -e\n#\n# Copyright 2018 the Velero contributors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nVELERO_ROOT=$(dirname ${BASH_SOURCE})/..\nHACK_DIR=$(dirname \"${BASH_SOURCE}\")\nISSUE_TEMPLATE_FILE=${VELERO_ROOT}/.github/ISSUE_TEMPLATE/bug_report.md\nOUT_TMP_FILE=\"$(mktemp -d)\"/bug_report.md\n\n\ntrap cleanup INT TERM HUP EXIT\n\ncleanup() {\n rm -rf ${TMP_DIR}\n}\n\necho \"Verifying generated Github issue template\"\n${HACK_DIR}/update-4generated-issue-template.sh ${OUT_TMP_FILE} > /dev/null\noutput=$(echo \"`diff ${ISSUE_TEMPLATE_FILE} ${OUT_TMP_FILE}`\")\n\nif [[ -n \"${output}\" ]] ; then\n echo \"FAILURE: verification of generated template failed:\"\n echo \"${output}\"\n exit 1\nfi\n\necho \"Success!\"\n" }, { "path": "internal/credentials/file_store.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage credentials\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\n\t\"github.com/pkg/errors\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\n// FileStore defines operations for interacting with credentials\n// that are stored on a file system.\ntype FileStore interface {\n\t// Path returns a path on disk where the secret key defined by\n\t// the given selector is serialized.\n\tPath(selector *corev1api.SecretKeySelector) (string, error)\n}\n\ntype namespacedFileStore struct {\n\tclient kbclient.Client\n\tnamespace string\n\tfsRoot string\n\tfs filesystem.Interface\n}\n\n// NewNamespacedFileStore returns a FileStore which can interact with credentials\n// for the given namespace and will store them under the given fsRoot.\nfunc NewNamespacedFileStore(client kbclient.Client, namespace string, fsRoot string, fs filesystem.Interface) (FileStore, error) {\n\tfsNamespaceRoot := filepath.Join(fsRoot, namespace)\n\n\tif err := fs.MkdirAll(fsNamespaceRoot, 0755); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &namespacedFileStore{\n\t\tclient: client,\n\t\tnamespace: namespace,\n\t\tfsRoot: fsNamespaceRoot,\n\t\tfs: fs,\n\t}, nil\n}\n\n// Path returns a path on disk where the secret key defined by\n// the given selector is serialized.\nfunc (n *namespacedFileStore) Path(selector *corev1api.SecretKeySelector) (string, error) {\n\tcreds, err := kube.GetSecretKey(n.client, n.namespace, selector)\n\tif err != nil {\n\t\treturn \"\", errors.Wrap(err, \"unable to get key for secret\")\n\t}\n\n\tkeyFilePath := filepath.Join(n.fsRoot, fmt.Sprintf(\"%s-%s\", selector.Name, selector.Key))\n\n\t// owner RW perms, group R perms, no public perms\n\tfile, err := n.fs.OpenFile(keyFilePath, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0640)\n\tif err != nil {\n\t\treturn \"\", errors.Wrap(err, \"unable to open credentials file for writing\")\n\t}\n\n\tif _, err := file.Write(creds); err != nil {\n\t\treturn \"\", errors.Wrap(err, \"unable to write credentials to store\")\n\t}\n\n\tif err := file.Close(); err != nil {\n\t\treturn \"\", errors.Wrap(err, \"unable to close credentials file\")\n\t}\n\n\treturn keyFilePath, nil\n}\n" }, { "path": "internal/credentials/file_store_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage credentials\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestNamespacedFileStore(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tnamespace string\n\t\tfsRoot string\n\t\tsecrets []*corev1api.Secret\n\t\tsecretSelector *corev1api.SecretKeySelector\n\t\twantErr string\n\t\texpectedPath string\n\t\texpectedContents string\n\t}{\n\t\t{\n\t\t\tname: \"returns an error if the secret can't be found\",\n\t\t\tsecretSelector: builder.ForSecretKeySelector(\"non-existent-secret\", \"secret-key\").Result(),\n\t\t\twantErr: \"unable to get key for secret: secrets \\\"non-existent-secret\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"returns a filepath formed using fsRoot, namespace, secret name and key, with secret contents\",\n\t\t\tnamespace: \"ns1\",\n\t\t\tfsRoot: \"/tmp/credentials\",\n\t\t\tsecretSelector: builder.ForSecretKeySelector(\"credential\", \"key2\").Result(),\n\t\t\tsecrets: []*corev1api.Secret{\n\t\t\t\tbuilder.ForSecret(\"ns1\", \"credential\").Data(map[string][]byte{\n\t\t\t\t\t\"key1\": []byte(\"ns1-secretdata1\"),\n\t\t\t\t\t\"key2\": []byte(\"ns1-secretdata2\"),\n\t\t\t\t\t\"key3\": []byte(\"ns1-secretdata3\"),\n\t\t\t\t}).Result(),\n\t\t\t\tbuilder.ForSecret(\"ns2\", \"credential\").Data(map[string][]byte{\n\t\t\t\t\t\"key2\": []byte(\"ns2-secretdata2\"),\n\t\t\t\t}).Result(),\n\t\t\t},\n\t\t\texpectedPath: \"/tmp/credentials/ns1/credential-key2\",\n\t\t\texpectedContents: \"ns1-secretdata2\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tclient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\t\t\tfor _, secret := range tc.secrets {\n\t\t\t\trequire.NoError(t, client.Create(t.Context(), secret))\n\t\t\t}\n\n\t\t\tfs := velerotest.NewFakeFileSystem()\n\t\t\tfileStore, err := NewNamespacedFileStore(client, tc.namespace, tc.fsRoot, fs)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tpath, err := fileStore.Path(tc.secretSelector)\n\n\t\t\tif tc.wantErr != \"\" {\n\t\t\t\trequire.EqualError(t, err, tc.wantErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\trequire.Equal(t, tc.expectedPath, path)\n\n\t\t\tcontents, err := fs.ReadFile(path)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.Equal(t, []byte(tc.expectedContents), contents)\n\t\t})\n\t}\n}\n" }, { "path": "internal/credentials/getter.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage credentials\n\n// CredentialGetter is a collection of interfaces for interacting with credentials\n// that are stored in different targets\ntype CredentialGetter struct {\n\tFromFile FileStore\n\tFromSecret SecretStore\n}\n" }, { "path": "internal/credentials/local.go", "content": "package credentials\n\nimport \"os\"\n\nfunc DefaultStoreDirectory() string {\n\treturn os.TempDir() + \"/credentials\"\n}\n" }, { "path": "internal/credentials/mocks/FileStore.go", "content": "// Code generated by mockery v2.14.0. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tmock \"github.com/stretchr/testify/mock\"\n\tcorev1api \"k8s.io/api/core/v1\"\n)\n\n// FileStore is an autogenerated mock type for the FileStore type\ntype FileStore struct {\n\tmock.Mock\n}\n\n// Path provides a mock function with given fields: selector\nfunc (_m *FileStore) Path(selector *corev1api.SecretKeySelector) (string, error) {\n\tret := _m.Called(selector)\n\n\tvar r0 string\n\tif rf, ok := ret.Get(0).(func(*corev1api.SecretKeySelector) string); ok {\n\t\tr0 = rf(selector)\n\t} else {\n\t\tr0 = ret.Get(0).(string)\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(*corev1api.SecretKeySelector) error); ok {\n\t\tr1 = rf(selector)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\ntype mockConstructorTestingTNewFileStore interface {\n\tmock.TestingT\n\tCleanup(func())\n}\n\n// NewFileStore creates a new instance of FileStore. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\nfunc NewFileStore(t mockConstructorTestingTNewFileStore) *FileStore {\n\tmock := &FileStore{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "internal/credentials/mocks/SecretStore.go", "content": "// Code generated by mockery v2.14.0. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tmock \"github.com/stretchr/testify/mock\"\n\tcorev1api \"k8s.io/api/core/v1\"\n)\n\n// SecretStore is an autogenerated mock type for the SecretStore type\ntype SecretStore struct {\n\tmock.Mock\n}\n\n// Get provides a mock function with given fields: selector\nfunc (_m *SecretStore) Get(selector *corev1api.SecretKeySelector) (string, error) {\n\tret := _m.Called(selector)\n\n\tvar r0 string\n\tif rf, ok := ret.Get(0).(func(*corev1api.SecretKeySelector) string); ok {\n\t\tr0 = rf(selector)\n\t} else {\n\t\tr0 = ret.Get(0).(string)\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(*corev1api.SecretKeySelector) error); ok {\n\t\tr1 = rf(selector)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\ntype mockConstructorTestingTNewSecretStore interface {\n\tmock.TestingT\n\tCleanup(func())\n}\n\n// NewSecretStore creates a new instance of SecretStore. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\nfunc NewSecretStore(t mockConstructorTestingTNewSecretStore) *SecretStore {\n\tmock := &SecretStore{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "internal/credentials/secret_store.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage credentials\n\nimport (\n\t\"github.com/pkg/errors\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\n// SecretStore defines operations for interacting with credentials\n// that are stored in Secret.\ntype SecretStore interface {\n\t// Get returns the secret key defined by the given selector\n\tGet(selector *corev1api.SecretKeySelector) (string, error)\n}\n\ntype namespacedSecretStore struct {\n\tclient kbclient.Client\n\tnamespace string\n}\n\n// NewNamespacedSecretStore returns a SecretStore which can interact with credentials\n// for the given namespace.\nfunc NewNamespacedSecretStore(client kbclient.Client, namespace string) (SecretStore, error) {\n\treturn &namespacedSecretStore{\n\t\tclient: client,\n\t\tnamespace: namespace,\n\t}, nil\n}\n\n// Buffer returns the secret key defined by the given selector.\nfunc (n *namespacedSecretStore) Get(selector *corev1api.SecretKeySelector) (string, error) {\n\tcreds, err := kube.GetSecretKey(n.client, n.namespace, selector)\n\tif err != nil {\n\t\treturn \"\", errors.Wrap(err, \"unable to get key for secret\")\n\t}\n\n\treturn string(creds), nil\n}\n" }, { "path": "internal/delete/actions/csi/volumesnapshotcontent_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\t\"github.com/google/uuid\"\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\tcrclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\tplugincommon \"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\tkubeutil \"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\n// volumeSnapshotContentDeleteItemAction is a restore item action plugin for Velero\ntype volumeSnapshotContentDeleteItemAction struct {\n\tlog logrus.FieldLogger\n\tcrClient crclient.Client\n}\n\n// AppliesTo returns information indicating\n// VolumeSnapshotContentRestoreItemAction action should be invoked\n// while restoring VolumeSnapshotContent.snapshot.storage.k8s.io resources\nfunc (p *volumeSnapshotContentDeleteItemAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"volumesnapshotcontents.snapshot.storage.k8s.io\"},\n\t}, nil\n}\n\nfunc (p *volumeSnapshotContentDeleteItemAction) Execute(\n\tinput *velero.DeleteItemActionExecuteInput,\n) error {\n\tp.log.Info(\"Starting VolumeSnapshotContentDeleteItemAction\")\n\n\tvar snapCont snapshotv1api.VolumeSnapshotContent\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(\n\t\tinput.Item.UnstructuredContent(),\n\t\t&snapCont,\n\t); err != nil {\n\t\treturn errors.Wrapf(err, \"failed to convert VolumeSnapshotContent from unstructured\")\n\t}\n\n\t// We don't want this DeleteItemAction plugin to delete\n\t// VolumeSnapshotContent taken outside of Velero.\n\t// So skip deleting VolumeSnapshotContent not have the backup name\n\t// in its labels.\n\tif !kubeutil.HasBackupLabel(&snapCont.ObjectMeta, input.Backup.Name) {\n\t\tp.log.Info(\n\t\t\t\"VolumeSnapshotContent %s was not taken by backup %s, skipping deletion\",\n\t\t\tsnapCont.Name,\n\t\t\tinput.Backup.Name,\n\t\t)\n\t\treturn nil\n\t}\n\n\tp.log.Infof(\"Deleting VolumeSnapshotContent %s\", snapCont.Name)\n\n\tuuid, err := uuid.NewRandom()\n\tif err != nil {\n\t\tp.log.WithError(err).Errorf(\"Fail to generate the UUID to create VSC %s\", snapCont.Name)\n\t\treturn errors.Wrapf(err, \"Fail to generate the UUID to create VSC %s\", snapCont.Name)\n\t}\n\tsnapCont.Name = \"vsc-\" + uuid.String()\n\n\tsnapCont.Spec.DeletionPolicy = snapshotv1api.VolumeSnapshotContentDelete\n\n\tsnapCont.Spec.Source = snapshotv1api.VolumeSnapshotContentSource{\n\t\tSnapshotHandle: snapCont.Status.SnapshotHandle,\n\t}\n\n\tsnapCont.Spec.VolumeSnapshotRef = corev1api.ObjectReference{\n\t\tAPIVersion: snapshotv1api.SchemeGroupVersion.String(),\n\t\tKind: \"VolumeSnapshot\",\n\t\tNamespace: \"ns-\" + string(snapCont.UID),\n\t\tName: \"name-\" + string(snapCont.UID),\n\t}\n\n\tsnapCont.ResourceVersion = \"\"\n\n\tif snapCont.Spec.VolumeSnapshotClassName != nil {\n\t\t// Delete VolumeSnapshotClass from the VolumeSnapshotContent.\n\t\t// This is necessary to make the deletion independent of the VolumeSnapshotClass.\n\t\tsnapCont.Spec.VolumeSnapshotClassName = nil\n\t\tp.log.Debugf(\"Deleted VolumeSnapshotClassName from VolumeSnapshotContent %s to make deletion independent of VolumeSnapshotClass\",\n\t\t\tsnapCont.Name)\n\t}\n\n\tif err := p.crClient.Create(context.TODO(), &snapCont); err != nil {\n\t\treturn errors.Wrapf(err, \"fail to create VolumeSnapshotContent %s\", snapCont.Name)\n\t}\n\n\t// Read resource timeout from backup annotation, if not set, use default value.\n\ttimeout, err := time.ParseDuration(\n\t\tinput.Backup.Annotations[velerov1api.ResourceTimeoutAnnotation])\n\tif err != nil {\n\t\tp.log.Warnf(\"fail to parse resource timeout annotation %s: %s\",\n\t\t\tinput.Backup.Annotations[velerov1api.ResourceTimeoutAnnotation], err.Error())\n\t\ttimeout = 10 * time.Minute\n\t}\n\tp.log.Debugf(\"resource timeout is set to %s\", timeout.String())\n\n\tinterval := 5 * time.Second\n\n\t// Wait until VSC created and ReadyToUse is true.\n\tif err := wait.PollUntilContextTimeout(\n\t\tcontext.Background(),\n\t\tinterval,\n\t\ttimeout,\n\t\ttrue,\n\t\tfunc(ctx context.Context) (bool, error) {\n\t\t\treturn checkVSCReadiness(ctx, &snapCont, p.crClient)\n\t\t},\n\t); err != nil {\n\t\t// Clean up the VSC we created since it can't become ready\n\t\tif deleteErr := p.crClient.Delete(context.TODO(), &snapCont); deleteErr != nil && !apierrors.IsNotFound(deleteErr) {\n\t\t\tp.log.WithError(deleteErr).Errorf(\"Failed to clean up VolumeSnapshotContent %s\", snapCont.Name)\n\t\t}\n\t\treturn errors.Wrapf(err, \"fail to wait VolumeSnapshotContent %s becomes ready.\", snapCont.Name)\n\t}\n\n\tif err := p.crClient.Delete(\n\t\tcontext.TODO(),\n\t\t&snapCont,\n\t); err != nil && !apierrors.IsNotFound(err) {\n\t\tp.log.Infof(\"VolumeSnapshotContent %s not found\", snapCont.Name)\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\nvar checkVSCReadiness = func(\n\tctx context.Context,\n\tvsc *snapshotv1api.VolumeSnapshotContent,\n\tclient crclient.Client,\n) (bool, error) {\n\ttmpVSC := new(snapshotv1api.VolumeSnapshotContent)\n\tif err := client.Get(ctx, crclient.ObjectKeyFromObject(vsc), tmpVSC); err != nil {\n\t\treturn false, errors.Wrapf(\n\t\t\terr, \"failed to get VolumeSnapshotContent %s\", vsc.Name,\n\t\t)\n\t}\n\n\tif tmpVSC.Status != nil && boolptr.IsSetToTrue(tmpVSC.Status.ReadyToUse) {\n\t\treturn true, nil\n\t}\n\n\t// Fail fast on permanent CSI driver errors (e.g., InvalidSnapshot.NotFound)\n\tif tmpVSC.Status != nil && tmpVSC.Status.Error != nil && tmpVSC.Status.Error.Message != nil {\n\t\treturn false, errors.Errorf(\n\t\t\t\"VolumeSnapshotContent %s has error: %s\", vsc.Name, *tmpVSC.Status.Error.Message,\n\t\t)\n\t}\n\n\treturn false, nil\n}\n\nfunc NewVolumeSnapshotContentDeleteItemAction(\n\tf client.Factory,\n) plugincommon.HandlerInitializer {\n\treturn func(logger logrus.FieldLogger) (any, error) {\n\t\tcrClient, err := f.KubebuilderClient()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\treturn &volumeSnapshotContentDeleteItemAction{\n\t\t\tlog: logger,\n\t\t\tcrClient: crClient,\n\t\t}, nil\n\t}\n}\n" }, { "path": "internal/delete/actions/csi/volumesnapshotcontent_action_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"testing\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\tcrclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestVSCExecute(t *testing.T) {\n\tsnapshotHandleStr := \"test\"\n\ttests := []struct {\n\t\tname string\n\t\titem runtime.Unstructured\n\t\tvsc *snapshotv1api.VolumeSnapshotContent\n\t\tbackup *velerov1api.Backup\n\t\tfunction func(\n\t\t\tctx context.Context,\n\t\t\tvsc *snapshotv1api.VolumeSnapshotContent,\n\t\t\tclient crclient.Client,\n\t\t) (bool, error)\n\t\texpectErr bool\n\t}{\n\t\t{\n\t\t\tname: \"VolumeSnapshotContent doesn't have backup label\",\n\t\t\titem: velerotest.UnstructuredOrDie(\n\t\t\t\t`\n\t\t\t\t{\n\t\t\t\t\t\"apiVersion\": \"snapshot.storage.k8s.io/v1\",\n\t\t\t\t\t\"kind\": \"VolumeSnapshotContent\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns\",\n\t\t\t\t\t\t\"name\": \"foo\"\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\t`,\n\t\t\t),\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").Result(),\n\t\t\texpectErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Normal case, VolumeSnapshot should be deleted\",\n\t\t\tvsc: builder.ForVolumeSnapshotContent(\"bar\").ObjectMeta(builder.WithLabelsMap(map[string]string{velerov1api.BackupNameLabel: \"backup\"})).VolumeSnapshotClassName(\"volumesnapshotclass\").Status(&snapshotv1api.VolumeSnapshotContentStatus{SnapshotHandle: &snapshotHandleStr}).Result(),\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").ObjectMeta(builder.WithAnnotationsMap(map[string]string{velerov1api.ResourceTimeoutAnnotation: \"5s\"})).Result(),\n\t\t\texpectErr: false,\n\t\t\tfunction: func(\n\t\t\t\tctx context.Context,\n\t\t\t\tvsc *snapshotv1api.VolumeSnapshotContent,\n\t\t\t\tclient crclient.Client,\n\t\t\t) (bool, error) {\n\t\t\t\treturn true, nil\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Error case, deletion fails\",\n\t\t\tvsc: builder.ForVolumeSnapshotContent(\"bar\").ObjectMeta(builder.WithLabelsMap(map[string]string{velerov1api.BackupNameLabel: \"backup\"})).Status(&snapshotv1api.VolumeSnapshotContentStatus{SnapshotHandle: &snapshotHandleStr}).Result(),\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").ObjectMeta(builder.WithAnnotationsMap(map[string]string{velerov1api.ResourceTimeoutAnnotation: \"5s\"})).Result(),\n\t\t\texpectErr: true,\n\t\t\tfunction: func(\n\t\t\t\tctx context.Context,\n\t\t\t\tvsc *snapshotv1api.VolumeSnapshotContent,\n\t\t\t\tclient crclient.Client,\n\t\t\t) (bool, error) {\n\t\t\t\treturn false, errors.Errorf(\"test error case\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Error case with CSI error, dangling VSC should be cleaned up\",\n\t\t\tvsc: builder.ForVolumeSnapshotContent(\"bar\").ObjectMeta(builder.WithLabelsMap(map[string]string{velerov1api.BackupNameLabel: \"backup\"})).Status(&snapshotv1api.VolumeSnapshotContentStatus{SnapshotHandle: &snapshotHandleStr}).Result(),\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").ObjectMeta(builder.WithAnnotationsMap(map[string]string{velerov1api.ResourceTimeoutAnnotation: \"5s\"})).Result(),\n\t\t\texpectErr: true,\n\t\t\tfunction: func(\n\t\t\t\tctx context.Context,\n\t\t\t\tvsc *snapshotv1api.VolumeSnapshotContent,\n\t\t\t\tclient crclient.Client,\n\t\t\t) (bool, error) {\n\t\t\t\treturn false, errors.Errorf(\"VolumeSnapshotContent %s has error: InvalidSnapshot.NotFound\", vsc.Name)\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tcrClient := velerotest.NewFakeControllerRuntimeClient(t)\n\t\t\tlogger := logrus.StandardLogger()\n\t\t\tcheckVSCReadiness = test.function\n\n\t\t\tp := volumeSnapshotContentDeleteItemAction{log: logger, crClient: crClient}\n\n\t\t\tif test.vsc != nil {\n\t\t\t\tvscMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(test.vsc)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\ttest.item = &unstructured.Unstructured{Object: vscMap}\n\t\t\t}\n\n\t\t\terr := p.Execute(\n\t\t\t\t&velero.DeleteItemActionExecuteInput{\n\t\t\t\t\tItem: test.item,\n\t\t\t\t\tBackup: test.backup,\n\t\t\t\t},\n\t\t\t)\n\n\t\t\tif test.expectErr == false {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestVSCAppliesTo(t *testing.T) {\n\tp := volumeSnapshotContentDeleteItemAction{\n\t\tlog: logrus.StandardLogger(),\n\t}\n\tselector, err := p.AppliesTo()\n\n\trequire.NoError(t, err)\n\n\trequire.Equal(\n\t\tt,\n\t\tvelero.ResourceSelector{\n\t\t\tIncludedResources: []string{\"volumesnapshotcontents.snapshot.storage.k8s.io\"},\n\t\t},\n\t\tselector,\n\t)\n}\n\nfunc TestNewVolumeSnapshotContentDeleteItemAction(t *testing.T) {\n\tlogger := logrus.StandardLogger()\n\tcrClient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\tf := &factorymocks.Factory{}\n\tf.On(\"KubebuilderClient\").Return(nil, fmt.Errorf(\"\"))\n\tplugin := NewVolumeSnapshotContentDeleteItemAction(f)\n\t_, err := plugin(logger)\n\trequire.Error(t, err)\n\n\tf1 := &factorymocks.Factory{}\n\tf1.On(\"KubebuilderClient\").Return(crClient, nil)\n\tplugin1 := NewVolumeSnapshotContentDeleteItemAction(f1)\n\t_, err1 := plugin1(logger)\n\trequire.NoError(t, err1)\n}\n\nfunc TestCheckVSCReadiness(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tvsc *snapshotv1api.VolumeSnapshotContent\n\t\tcreateVSC bool\n\t\texpectErr bool\n\t\tready bool\n\t}{\n\t\t{\n\t\t\tname: \"VSC not exist\",\n\t\t\tvsc: &snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"vsc-1\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tcreateVSC: false,\n\t\t\texpectErr: true,\n\t\t\tready: false,\n\t\t},\n\t\t{\n\t\t\tname: \"VSC not ready\",\n\t\t\tvsc: &snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"vsc-1\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tcreateVSC: true,\n\t\t\texpectErr: false,\n\t\t\tready: false,\n\t\t},\n\t\t{\n\t\t\tname: \"VSC has error from CSI driver\",\n\t\t\tvsc: &snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"vsc-1\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotContentStatus{\n\t\t\t\t\tReadyToUse: boolPtr(false),\n\t\t\t\t\tError: &snapshotv1api.VolumeSnapshotError{\n\t\t\t\t\t\tMessage: stringPtr(\"InvalidSnapshot.NotFound: The snapshot 'snap-0abc123' does not exist.\"),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tcreateVSC: true,\n\t\t\texpectErr: true,\n\t\t\tready: false,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tcrClient := velerotest.NewFakeControllerRuntimeClient(t)\n\t\t\tif test.createVSC {\n\t\t\t\trequire.NoError(t, crClient.Create(t.Context(), test.vsc))\n\t\t\t}\n\n\t\t\tready, err := checkVSCReadiness(t.Context(), test.vsc, crClient)\n\t\t\trequire.Equal(t, test.ready, ready)\n\t\t\tif test.expectErr {\n\t\t\t\trequire.Error(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc boolPtr(b bool) *bool {\n\treturn &b\n}\n\nfunc stringPtr(s string) *string {\n\treturn &s\n}\n" }, { "path": "internal/delete/delete_item_action_handler.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage delete\n\nimport (\n\t\"io\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/archive\"\n\t\"github.com/vmware-tanzu/velero/pkg/discovery\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n)\n\n// Context provides the necessary environment to run DeleteItemAction plugins\ntype Context struct {\n\tBackup *velerov1api.Backup\n\tBackupReader io.Reader\n\tActions []velero.DeleteItemAction\n\tFilesystem filesystem.Interface\n\tLog logrus.FieldLogger\n\tDiscoveryHelper discovery.Helper\n\tresolvedActions []framework.DeleteItemResolvedAction\n}\n\nfunc InvokeDeleteActions(ctx *Context) error {\n\tvar err error\n\tresolver := framework.NewDeleteItemActionResolver(ctx.Actions)\n\tctx.resolvedActions, err = resolver.ResolveActions(ctx.DiscoveryHelper, ctx.Log)\n\t// No actions installed and no error means we don't have to continue;\n\t// just do the backup deletion without worrying about plugins.\n\tif len(ctx.resolvedActions) == 0 && err == nil {\n\t\tctx.Log.Debug(\"No delete item actions present, proceeding with rest of backup deletion process\")\n\t\treturn nil\n\t} else if err != nil {\n\t\treturn errors.Wrapf(err, \"error resolving actions\")\n\t}\n\n\t// get items out of backup tarball into a temp directory\n\tdir, err := archive.NewExtractor(ctx.Log, ctx.Filesystem).UnzipAndExtractBackup(ctx.BackupReader)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error extracting backup\")\n\t}\n\tdefer func() {\n\t\tif err := ctx.Filesystem.RemoveAll(dir); err != nil {\n\t\t\tctx.Log.Errorf(\"error removing temporary directory %s: %s\", dir, err.Error())\n\t\t}\n\t}()\n\n\tctx.Log.Debugf(\"Downloaded and extracted the backup file to: %s\", dir)\n\n\tbackupResources, err := archive.NewParser(ctx.Log, ctx.Filesystem).Parse(dir)\n\tif existErr := errors.Is(err, archive.ErrNotExist); existErr {\n\t\tctx.Log.Debug(\"ignore invoking delete item actions: \", err)\n\t\treturn nil\n\t} else if err != nil {\n\t\treturn errors.Wrapf(err, \"error parsing backup %q\", dir)\n\t}\n\tprocessdResources := sets.NewString()\n\n\tfor resource := range backupResources {\n\t\tgroupResource := schema.ParseGroupResource(resource)\n\n\t\t// We've already seen this group/resource, so don't process it again.\n\t\tif processdResources.Has(groupResource.String()) {\n\t\t\tcontinue\n\t\t}\n\n\t\t// Get a list of all items that exist for this resource\n\t\tresourceList := backupResources[groupResource.String()]\n\t\tif resourceList == nil {\n\t\t\tcontinue\n\t\t}\n\n\t\t// Iterate over all items, grouped by namespace.\n\t\tfor namespace, items := range resourceList.ItemsByNamespace {\n\t\t\tnsLog := ctx.Log.WithField(\"namespace\", namespace)\n\t\t\tnsLog.Info(\"Starting to check for items in namespace\")\n\n\t\t\t// Filter applicable actions based on namespace only once per namespace.\n\t\t\tactions := ctx.getApplicableActions(groupResource, namespace)\n\n\t\t\t// Process individual items from the backup\n\t\t\tfor _, item := range items {\n\t\t\t\titemPath := archive.GetItemFilePath(dir, resource, namespace, item)\n\n\t\t\t\t// obj is the Unstructured item from the backup\n\t\t\t\tobj, err := archive.Unmarshal(ctx.Filesystem, itemPath)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn errors.Wrapf(err, \"Could not unmarshal item: %v\", item)\n\t\t\t\t}\n\n\t\t\t\titemLog := nsLog.WithField(\"item\", obj.GetName())\n\t\t\t\titemLog.Infof(\"invoking DeleteItemAction plugins\")\n\n\t\t\t\tfor _, action := range actions {\n\t\t\t\t\tif !action.Selector.Matches(labels.Set(obj.GetLabels())) {\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\n\t\t\t\t\terr = action.DeleteItemAction.Execute(&velero.DeleteItemActionExecuteInput{\n\t\t\t\t\t\tItem: obj,\n\t\t\t\t\t\tBackup: ctx.Backup,\n\t\t\t\t\t})\n\t\t\t\t\t// Since we want to keep looping even on errors, log them instead of just returning.\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\titemLog.WithError(err).Error(\"plugin error\")\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\treturn nil\n}\n\n// getApplicableActions takes resolved DeleteItemActions and filters them for a given group/resource and namespace.\nfunc (ctx *Context) getApplicableActions(groupResource schema.GroupResource, namespace string) []framework.DeleteItemResolvedAction {\n\tvar actions []framework.DeleteItemResolvedAction\n\tfor _, action := range ctx.resolvedActions {\n\t\tif action.ShouldUse(groupResource, namespace, nil, ctx.Log) {\n\t\t\tactions = append(actions, action)\n\t\t}\n\t}\n\treturn actions\n}\n" }, { "path": "internal/delete/delete_item_action_handler_test.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage delete\n\nimport (\n\t\"io\"\n\t\"sort\"\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/api/meta\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/discovery\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n\tkubeutil \"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nfunc TestInvokeDeleteItemActionsRunForCorrectItems(t *testing.T) {\n\t// Declare test-singleton objects.\n\tfs := test.NewFakeFileSystem()\n\tlog := logrus.StandardLogger()\n\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\tapiResources []*test.APIResource\n\t\ttarball io.Reader\n\t\tactions map[*recordResourcesAction][]string // recordResourceActions are the plugins that will capture item ids, the []string values are the ids we'll test against.\n\t}{\n\t\t{\n\t\t\tname: \"single action with no selector runs for all items\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"velero\").Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result(), builder.ForPod(\"ns-2\", \"pod-2\").Result()).\n\t\t\t\tAddItems(\"persistentvolumes\", builder.ForPersistentVolume(\"pv-1\").Result(), builder.ForPersistentVolume(\"pv-2\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{test.Pods(), test.PVs()},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction): {\"ns-1/pod-1\", \"ns-2/pod-2\", \"pv-1\", \"pv-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"single action with a resource selector for namespaced resources runs only for matching resources\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"velero\").Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result(), builder.ForPod(\"ns-2\", \"pod-2\").Result()).\n\t\t\t\tAddItems(\"persistentvolumes\", builder.ForPersistentVolume(\"pv-1\").Result(), builder.ForPersistentVolume(\"pv-2\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{test.Pods(), test.PVs()},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForResource(\"pods\"): {\"ns-1/pod-1\", \"ns-2/pod-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"single action with a resource selector for cluster-scoped resources runs only for matching resources\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"velero\").Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result(), builder.ForPod(\"ns-2\", \"pod-2\").Result()).\n\t\t\t\tAddItems(\"persistentvolumes\", builder.ForPersistentVolume(\"pv-1\").Result(), builder.ForPersistentVolume(\"pv-2\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{test.Pods(), test.PVs()},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForResource(\"persistentvolumes\"): {\"pv-1\", \"pv-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"single action with a namespace selector runs only for resources in that namespace\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"velero\").Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result(), builder.ForPod(\"ns-2\", \"pod-2\").Result()).\n\t\t\t\tAddItems(\"persistentvolumeclaims\", builder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").Result(), builder.ForPersistentVolumeClaim(\"ns-2\", \"pvc-2\").Result()).\n\t\t\t\tAddItems(\"persistentvolumes\", builder.ForPersistentVolume(\"pv-1\").Result(), builder.ForPersistentVolume(\"pv-2\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{test.Pods(), test.PVCs(), test.PVs()},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForNamespace(\"ns-1\"): {\"ns-1/pod-1\", \"ns-1/pvc-1\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"multiple actions, each with a different resource selector using short name, run for matching resources\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"velero\").Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result(), builder.ForPod(\"ns-2\", \"pod-2\").Result()).\n\t\t\t\tAddItems(\"persistentvolumeclaims\", builder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").Result(), builder.ForPersistentVolumeClaim(\"ns-2\", \"pvc-2\").Result()).\n\t\t\t\tAddItems(\"persistentvolumes\", builder.ForPersistentVolume(\"pv-1\").Result(), builder.ForPersistentVolume(\"pv-2\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{test.Pods(), test.PVCs(), test.PVs()},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForResource(\"po\"): {\"ns-1/pod-1\", \"ns-2/pod-2\"},\n\t\t\t\tnew(recordResourcesAction).ForResource(\"pv\"): {\"pv-1\", \"pv-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"actions with selectors that don't match anything don't run for any resources\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"velero\").Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result()).\n\t\t\t\tAddItems(\"persistentvolumeclaims\", builder.ForPersistentVolumeClaim(\"ns-2\", \"pvc-2\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{test.Pods(), test.PVCs(), test.PVs()},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForNamespace(\"ns-1\").ForResource(\"persistentvolumeclaims\"): nil,\n\t\t\t\tnew(recordResourcesAction).ForNamespace(\"ns-2\").ForResource(\"pods\"): nil,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"single action with label selector runs only for those items\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"velero\").Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").ObjectMeta(builder.WithLabels(\"app\", \"app1\")).Result(), builder.ForPod(\"ns-2\", \"pod-2\").Result()).\n\t\t\t\tAddItems(\"persistentvolumeclaims\", builder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").Result(), builder.ForPersistentVolumeClaim(\"ns-2\", \"pvc-2\").ObjectMeta(builder.WithLabels(\"app\", \"app1\")).Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{test.Pods(), test.PVCs()},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForLabelSelector(\"app=app1\"): {\"ns-1/pod-1\", \"ns-2/pvc-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"success if resources dir does not exist\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"velero\").Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{test.Pods(), test.PVCs()},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForNamespace(\"ns-1\").ForResource(\"persistentvolumeclaims\"): nil,\n\t\t\t\tnew(recordResourcesAction).ForNamespace(\"ns-2\").ForResource(\"pods\"): nil,\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t// test harness contains the fake API server/discovery client\n\t\t\th := newHarness(t)\n\t\t\tfor _, r := range tc.apiResources {\n\t\t\t\th.addResource(t, r)\n\t\t\t}\n\n\t\t\t// Get the plugins out of the map in order to use them.\n\t\t\tvar actions []velero.DeleteItemAction\n\t\t\tfor action := range tc.actions {\n\t\t\t\tactions = append(actions, action)\n\t\t\t}\n\n\t\t\tc := &Context{\n\t\t\t\tBackup: tc.backup,\n\t\t\t\tBackupReader: tc.tarball,\n\t\t\t\tFilesystem: fs,\n\t\t\t\tDiscoveryHelper: h.discoveryHelper,\n\t\t\t\tActions: actions,\n\t\t\t\tLog: log,\n\t\t\t}\n\n\t\t\terr := InvokeDeleteActions(c)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Compare the plugins against the ids that we wanted.\n\t\t\tfor action, want := range tc.actions {\n\t\t\t\tsort.Strings(want)\n\t\t\t\tsort.Strings(action.ids)\n\t\t\t\tassert.Equal(t, want, action.ids)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TODO: unify this with the test harness in pkg/restore/restore_test.go\ntype harness struct {\n\t*test.APIServer\n\tdiscoveryHelper discovery.Helper\n}\n\nfunc newHarness(t *testing.T) *harness {\n\tt.Helper()\n\n\tapiServer := test.NewAPIServer(t)\n\tlog := logrus.StandardLogger()\n\n\tdiscoveryHelper, err := discovery.NewHelper(apiServer.DiscoveryClient, log)\n\trequire.NoError(t, err)\n\n\treturn &harness{\n\t\tAPIServer: apiServer,\n\t\tdiscoveryHelper: discoveryHelper,\n\t}\n}\n\n// addResource adds an APIResource and it's items to a faked API server for testing.\nfunc (h *harness) addResource(t *testing.T, resource *test.APIResource) {\n\tt.Helper()\n\n\th.DiscoveryClient.WithAPIResource(resource)\n\trequire.NoError(t, h.discoveryHelper.Refresh())\n\n\tfor _, item := range resource.Items {\n\t\tobj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(item)\n\t\trequire.NoError(t, err)\n\n\t\tunstructuredObj := &unstructured.Unstructured{Object: obj}\n\t\tif resource.Namespaced {\n\t\t\t_, err = h.DynamicClient.Resource(resource.GVR()).Namespace(item.GetNamespace()).Create(t.Context(), unstructuredObj, metav1.CreateOptions{})\n\t\t} else {\n\t\t\t_, err = h.DynamicClient.Resource(resource.GVR()).Create(t.Context(), unstructuredObj, metav1.CreateOptions{})\n\t\t}\n\t\trequire.NoError(t, err)\n\t}\n}\n\n// recordResourcesAction is a delete item action that can be configured to run\n// for specific resources/namespaces and simply record the items that is\n// executed for.\ntype recordResourcesAction struct {\n\tselector velero.ResourceSelector\n\tids []string\n}\n\nfunc (a *recordResourcesAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn a.selector, nil\n}\n\nfunc (a *recordResourcesAction) Execute(input *velero.DeleteItemActionExecuteInput) error {\n\tmetadata, err := meta.Accessor(input.Item)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\ta.ids = append(a.ids, kubeutil.NamespaceAndName(metadata))\n\n\treturn nil\n}\n\nfunc (a *recordResourcesAction) ForResource(resource string) *recordResourcesAction {\n\ta.selector.IncludedResources = append(a.selector.IncludedResources, resource)\n\treturn a\n}\n\nfunc (a *recordResourcesAction) ForNamespace(namespace string) *recordResourcesAction {\n\ta.selector.IncludedNamespaces = append(a.selector.IncludedNamespaces, namespace)\n\treturn a\n}\n\nfunc (a *recordResourcesAction) ForLabelSelector(selector string) *recordResourcesAction {\n\ta.selector.LabelSelector = selector\n\treturn a\n}\n\nfunc TestInvokeDeleteItemActionsWithNoPlugins(t *testing.T) {\n\tc := &Context{\n\t\tBackup: builder.ForBackup(\"velero\", \"velero\").Result(),\n\t\tLog: logrus.StandardLogger(),\n\t\t// No other fields are set on the assumption that if 0 actions are present,\n\t\t// the backup tarball and file system being empty will produce no errors.\n\t}\n\terr := InvokeDeleteActions(c)\n\trequire.NoError(t, err)\n}\n" }, { "path": "internal/hook/hook_tracker.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage hook\n\nimport (\n\t\"fmt\"\n\t\"sync\"\n)\n\nconst (\n\tHookSourceAnnotation = \"annotation\"\n\tHookSourceSpec = \"spec\"\n)\n\n// hookKey identifies a backup/restore hook\ntype hookKey struct {\n\t// PodNamespace indicates the namespace of pod where hooks are executed.\n\t// For hooks specified in the backup/restore spec, this field is the namespace of an applicable pod.\n\t// For hooks specified in pod annotation, this field is the namespace of pod where hooks are annotated.\n\tpodNamespace string\n\t// PodName indicates the pod where hooks are executed.\n\t// For hooks specified in the backup/restore spec, this field is an applicable pod name.\n\t// For hooks specified in pod annotation, this field is the pod where hooks are annotated.\n\tpodName string\n\t// HookPhase is only for backup hooks, for restore hooks, this field is empty.\n\thookPhase HookPhase\n\t// HookName is only for hooks specified in the backup/restore spec.\n\t// For hooks specified in pod annotation, this field is empty or \"\".\n\thookName string\n\t// HookSource indicates where hooks come from.\n\thookSource string\n\t// Container indicates the container hooks use.\n\t// For hooks specified in the backup/restore spec, the container might be the same under different hookName.\n\tcontainer string\n\t// hookIndex contains the slice index for the specific hook, in order to track multiple hooks\n\t// for the same container\n\thookIndex int\n}\n\n// hookStatus records the execution status of a specific hook.\n// hookStatus is extensible to accommodate additional fields as needs develop.\ntype hookStatus struct {\n\t// HookFailed indicates if hook failed to execute.\n\thookFailed bool\n\t// hookExecuted indicates if hook already execute.\n\thookExecuted bool\n}\n\n// HookTracker tracks all hooks' execution status in a single backup/restore.\ntype HookTracker struct {\n\tlock *sync.RWMutex\n\t// tracker records all hook info for a single backup/restore.\n\ttracker map[hookKey]hookStatus\n\t// hookAttemptedCnt indicates the number of attempted hooks.\n\thookAttemptedCnt int\n\t// hookFailedCnt indicates the number of failed hooks.\n\thookFailedCnt int\n\t// HookExecutedCnt indicates the number of executed hooks.\n\thookExecutedCnt int\n\t// hookErrs records hook execution errors if any.\n\thookErrs []HookErrInfo\n}\n\n// NewHookTracker creates a hookTracker instance.\nfunc NewHookTracker() *HookTracker {\n\treturn &HookTracker{\n\t\tlock: &sync.RWMutex{},\n\t\ttracker: make(map[hookKey]hookStatus),\n\t}\n}\n\n// Add adds a hook to the hook tracker\n// Add must precede the Record for each individual hook.\n// In other words, a hook must be added to the tracker before its execution result is recorded.\nfunc (ht *HookTracker) Add(podNamespace, podName, container, source, hookName string, hookPhase HookPhase, hookIndex int) {\n\tht.lock.Lock()\n\tdefer ht.lock.Unlock()\n\n\tkey := hookKey{\n\t\tpodNamespace: podNamespace,\n\t\tpodName: podName,\n\t\thookSource: source,\n\t\tcontainer: container,\n\t\thookPhase: hookPhase,\n\t\thookName: hookName,\n\t\thookIndex: hookIndex,\n\t}\n\n\tif _, ok := ht.tracker[key]; !ok {\n\t\tht.tracker[key] = hookStatus{\n\t\t\thookFailed: false,\n\t\t\thookExecuted: false,\n\t\t}\n\t\tht.hookAttemptedCnt++\n\t}\n}\n\n// Record records the hook's execution status\n// Add must precede the Record for each individual hook.\n// In other words, a hook must be added to the tracker before its execution result is recorded.\nfunc (ht *HookTracker) Record(podNamespace, podName, container, source, hookName string, hookPhase HookPhase, hookIndex int, hookFailed bool, hookErr error) error {\n\tht.lock.Lock()\n\tdefer ht.lock.Unlock()\n\n\tkey := hookKey{\n\t\tpodNamespace: podNamespace,\n\t\tpodName: podName,\n\t\thookSource: source,\n\t\tcontainer: container,\n\t\thookPhase: hookPhase,\n\t\thookName: hookName,\n\t\thookIndex: hookIndex,\n\t}\n\n\tif _, ok := ht.tracker[key]; !ok {\n\t\treturn fmt.Errorf(\"hook not exist in hook tracker, hook: %+v\", key)\n\t}\n\n\tif !ht.tracker[key].hookExecuted {\n\t\tht.tracker[key] = hookStatus{\n\t\t\thookFailed: hookFailed,\n\t\t\thookExecuted: true,\n\t\t}\n\t\tht.hookExecutedCnt++\n\t\tif hookFailed {\n\t\t\tht.hookFailedCnt++\n\t\t\tht.hookErrs = append(ht.hookErrs, HookErrInfo{Namespace: key.podNamespace, Err: hookErr})\n\t\t}\n\t}\n\treturn nil\n}\n\n// Stat returns the number of attempted hooks and failed hooks\nfunc (ht *HookTracker) Stat() (hookAttemptedCnt int, hookFailedCnt int) {\n\tht.lock.RLock()\n\tdefer ht.lock.RUnlock()\n\n\treturn ht.hookAttemptedCnt, ht.hookFailedCnt\n}\n\n// IsComplete returns whether the execution of all hooks has finished or not\nfunc (ht *HookTracker) IsComplete() bool {\n\tht.lock.RLock()\n\tdefer ht.lock.RUnlock()\n\n\treturn ht.hookAttemptedCnt == ht.hookExecutedCnt\n}\n\n// HooksErr returns hook execution errors\nfunc (ht *HookTracker) HookErrs() []HookErrInfo {\n\tht.lock.RLock()\n\tdefer ht.lock.RUnlock()\n\n\treturn ht.hookErrs\n}\n\n// MultiHookTrackers tracks all hooks' execution status for multiple backups/restores.\ntype MultiHookTracker struct {\n\tlock *sync.RWMutex\n\t// trackers is a map that uses the backup/restore name as the key and stores a HookTracker as value.\n\ttrackers map[string]*HookTracker\n}\n\n// NewMultiHookTracker creates a multiHookTracker instance.\nfunc NewMultiHookTracker() *MultiHookTracker {\n\treturn &MultiHookTracker{\n\t\tlock: &sync.RWMutex{},\n\t\ttrackers: make(map[string]*HookTracker),\n\t}\n}\n\n// Add adds a backup/restore hook to the tracker\nfunc (mht *MultiHookTracker) Add(name, podNamespace, podName, container, source, hookName string, hookPhase HookPhase, hookIndex int) {\n\tmht.lock.Lock()\n\tdefer mht.lock.Unlock()\n\n\tif _, ok := mht.trackers[name]; !ok {\n\t\tmht.trackers[name] = NewHookTracker()\n\t}\n\tmht.trackers[name].Add(podNamespace, podName, container, source, hookName, hookPhase, hookIndex)\n}\n\n// Record records a backup/restore hook execution status\nfunc (mht *MultiHookTracker) Record(name, podNamespace, podName, container, source, hookName string, hookPhase HookPhase, hookIndex int, hookFailed bool, hookErr error) error {\n\tmht.lock.RLock()\n\tdefer mht.lock.RUnlock()\n\n\tvar err error\n\tif _, ok := mht.trackers[name]; ok {\n\t\terr = mht.trackers[name].Record(podNamespace, podName, container, source, hookName, hookPhase, hookIndex, hookFailed, hookErr)\n\t} else {\n\t\terr = fmt.Errorf(\"the backup/restore not exist in hook tracker, backup/restore name: %s\", name)\n\t}\n\treturn err\n}\n\n// Stat returns the number of attempted hooks and failed hooks for a particular backup/restore\nfunc (mht *MultiHookTracker) Stat(name string) (hookAttemptedCnt int, hookFailedCnt int) {\n\tmht.lock.RLock()\n\tdefer mht.lock.RUnlock()\n\n\tif _, ok := mht.trackers[name]; ok {\n\t\treturn mht.trackers[name].Stat()\n\t}\n\treturn\n}\n\n// Delete removes the hook data for a particular backup/restore\nfunc (mht *MultiHookTracker) Delete(name string) {\n\tmht.lock.Lock()\n\tdefer mht.lock.Unlock()\n\n\tdelete(mht.trackers, name)\n}\n\n// IsComplete returns whether the execution of all hooks for a particular backup/restore has finished or not\nfunc (mht *MultiHookTracker) IsComplete(name string) bool {\n\tmht.lock.RLock()\n\tdefer mht.lock.RUnlock()\n\n\tif _, ok := mht.trackers[name]; ok {\n\t\treturn mht.trackers[name].IsComplete()\n\t}\n\treturn true\n}\n\n// HooksErr returns hook execution errors for a particular backup/restore\nfunc (mht *MultiHookTracker) HookErrs(name string) []HookErrInfo {\n\tmht.lock.RLock()\n\tdefer mht.lock.RUnlock()\n\n\tif _, ok := mht.trackers[name]; ok {\n\t\treturn mht.trackers[name].HookErrs()\n\t}\n\treturn nil\n}\n" }, { "path": "internal/hook/hook_tracker_test.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage hook\n\nimport (\n\t\"fmt\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestNewHookTracker(t *testing.T) {\n\ttracker := NewHookTracker()\n\n\tassert.NotNil(t, tracker)\n\tassert.Empty(t, tracker.tracker)\n}\n\nfunc TestHookTracker_Add(t *testing.T) {\n\ttracker := NewHookTracker()\n\n\ttracker.Add(\"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0)\n\n\tkey := hookKey{\n\t\tpodNamespace: \"ns1\",\n\t\tpodName: \"pod1\",\n\t\tcontainer: \"container1\",\n\t\thookPhase: \"\",\n\t\thookSource: HookSourceAnnotation,\n\t\thookName: \"h1\",\n\t}\n\n\t_, ok := tracker.tracker[key]\n\tassert.True(t, ok)\n}\n\nfunc TestHookTracker_Record(t *testing.T) {\n\ttracker := NewHookTracker()\n\ttracker.Add(\"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0)\n\terr := tracker.Record(\"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0, true, fmt.Errorf(\"err\"))\n\n\tkey := hookKey{\n\t\tpodNamespace: \"ns1\",\n\t\tpodName: \"pod1\",\n\t\tcontainer: \"container1\",\n\t\thookPhase: \"\",\n\t\thookSource: HookSourceAnnotation,\n\t\thookName: \"h1\",\n\t}\n\n\tinfo := tracker.tracker[key]\n\tassert.True(t, info.hookFailed)\n\tassert.True(t, info.hookExecuted)\n\trequire.NoError(t, err)\n\n\terr = tracker.Record(\"ns2\", \"pod2\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0, true, fmt.Errorf(\"err\"))\n\trequire.Error(t, err)\n\n\terr = tracker.Record(\"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0, false, nil)\n\trequire.NoError(t, err)\n\tassert.True(t, info.hookFailed)\n}\n\nfunc TestHookTracker_Stat(t *testing.T) {\n\ttracker := NewHookTracker()\n\n\ttracker.Add(\"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0)\n\ttracker.Add(\"ns2\", \"pod2\", \"container1\", HookSourceAnnotation, \"h2\", \"\", 0)\n\ttracker.Add(\"ns2\", \"pod2\", \"container1\", HookSourceAnnotation, \"h2\", \"\", 1)\n\ttracker.Record(\"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0, true, fmt.Errorf(\"err\"))\n\n\tattempted, failed := tracker.Stat()\n\tassert.Equal(t, 3, attempted)\n\tassert.Equal(t, 1, failed)\n}\n\nfunc TestHookTracker_IsComplete(t *testing.T) {\n\ttracker := NewHookTracker()\n\ttracker.Add(\"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", PhasePre, 0)\n\ttracker.Record(\"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", PhasePre, 0, true, fmt.Errorf(\"err\"))\n\tassert.True(t, tracker.IsComplete())\n\n\ttracker.Add(\"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0)\n\tassert.False(t, tracker.IsComplete())\n}\n\nfunc TestHookTracker_HookErrs(t *testing.T) {\n\ttracker := NewHookTracker()\n\ttracker.Add(\"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0)\n\ttracker.Record(\"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0, true, fmt.Errorf(\"err\"))\n\n\thookErrs := tracker.HookErrs()\n\tassert.Len(t, hookErrs, 1)\n}\n\nfunc TestMultiHookTracker_Add(t *testing.T) {\n\tmht := NewMultiHookTracker()\n\n\tmht.Add(\"restore1\", \"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0)\n\n\tkey := hookKey{\n\t\tpodNamespace: \"ns1\",\n\t\tpodName: \"pod1\",\n\t\tcontainer: \"container1\",\n\t\thookPhase: \"\",\n\t\thookSource: HookSourceAnnotation,\n\t\thookName: \"h1\",\n\t\thookIndex: 0,\n\t}\n\n\t_, ok := mht.trackers[\"restore1\"].tracker[key]\n\tassert.True(t, ok)\n}\n\nfunc TestMultiHookTracker_Record(t *testing.T) {\n\tmht := NewMultiHookTracker()\n\tmht.Add(\"restore1\", \"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0)\n\terr := mht.Record(\"restore1\", \"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0, true, fmt.Errorf(\"err\"))\n\n\tkey := hookKey{\n\t\tpodNamespace: \"ns1\",\n\t\tpodName: \"pod1\",\n\t\tcontainer: \"container1\",\n\t\thookPhase: \"\",\n\t\thookSource: HookSourceAnnotation,\n\t\thookName: \"h1\",\n\t\thookIndex: 0,\n\t}\n\n\tinfo := mht.trackers[\"restore1\"].tracker[key]\n\tassert.True(t, info.hookFailed)\n\tassert.True(t, info.hookExecuted)\n\trequire.NoError(t, err)\n\n\terr = mht.Record(\"restore1\", \"ns2\", \"pod2\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0, true, fmt.Errorf(\"err\"))\n\trequire.Error(t, err)\n\n\terr = mht.Record(\"restore2\", \"ns2\", \"pod2\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0, true, fmt.Errorf(\"err\"))\n\tassert.Error(t, err)\n}\n\nfunc TestMultiHookTracker_Stat(t *testing.T) {\n\tmht := NewMultiHookTracker()\n\n\tmht.Add(\"restore1\", \"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0)\n\tmht.Add(\"restore1\", \"ns2\", \"pod2\", \"container1\", HookSourceAnnotation, \"h2\", \"\", 0)\n\tmht.Add(\"restore1\", \"ns2\", \"pod2\", \"container1\", HookSourceAnnotation, \"h2\", \"\", 1)\n\tmht.Record(\"restore1\", \"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0, true, fmt.Errorf(\"err\"))\n\tmht.Record(\"restore1\", \"ns2\", \"pod2\", \"container1\", HookSourceAnnotation, \"h2\", \"\", 0, false, nil)\n\tmht.Record(\"restore1\", \"ns2\", \"pod2\", \"container1\", HookSourceAnnotation, \"h2\", \"\", 1, false, nil)\n\n\tattempted, failed := mht.Stat(\"restore1\")\n\tassert.Equal(t, 3, attempted)\n\tassert.Equal(t, 1, failed)\n}\n\nfunc TestMultiHookTracker_Delete(t *testing.T) {\n\tmht := NewMultiHookTracker()\n\tmht.Add(\"restore1\", \"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0)\n\tmht.Delete(\"restore1\")\n\n\t_, ok := mht.trackers[\"restore1\"]\n\tassert.False(t, ok)\n}\n\nfunc TestMultiHookTracker_IsComplete(t *testing.T) {\n\tmht := NewMultiHookTracker()\n\tmht.Add(\"backup1\", \"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", PhasePre, 0)\n\tmht.Record(\"backup1\", \"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", PhasePre, 0, true, fmt.Errorf(\"err\"))\n\tassert.True(t, mht.IsComplete(\"backup1\"))\n\n\tmht.Add(\"restore1\", \"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0)\n\tassert.False(t, mht.IsComplete(\"restore1\"))\n\n\tassert.True(t, mht.IsComplete(\"restore2\"))\n}\n\nfunc TestMultiHookTracker_HookErrs(t *testing.T) {\n\tmht := NewMultiHookTracker()\n\tmht.Add(\"restore1\", \"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0)\n\tmht.Record(\"restore1\", \"ns1\", \"pod1\", \"container1\", HookSourceAnnotation, \"h1\", \"\", 0, true, fmt.Errorf(\"err\"))\n\n\thookErrs := mht.HookErrs(\"restore1\")\n\tassert.Len(t, hookErrs, 1)\n\n\thookErrs2 := mht.HookErrs(\"restore2\")\n\tassert.Empty(t, hookErrs2)\n}\n" }, { "path": "internal/hook/item_hook_handler.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage hook\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"strconv\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/google/uuid\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/api/meta\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/podexec\"\n\t\"github.com/vmware-tanzu/velero/pkg/restorehelper\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/collections\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\ntype HookPhase string\n\nconst (\n\tPhasePre HookPhase = \"pre\"\n\tPhasePost HookPhase = \"post\"\n)\n\nconst (\n\t// Backup hook annotations\n\tpodBackupHookContainerAnnotationKey = \"hook.backup.velero.io/container\"\n\tpodBackupHookCommandAnnotationKey = \"hook.backup.velero.io/command\"\n\tpodBackupHookOnErrorAnnotationKey = \"hook.backup.velero.io/on-error\"\n\tpodBackupHookTimeoutAnnotationKey = \"hook.backup.velero.io/timeout\"\n\n\t// Restore hook annotations\n\tpodRestoreHookContainerAnnotationKey = \"post.hook.restore.velero.io/container\"\n\tpodRestoreHookCommandAnnotationKey = \"post.hook.restore.velero.io/command\"\n\tpodRestoreHookOnErrorAnnotationKey = \"post.hook.restore.velero.io/on-error\"\n\tpodRestoreHookTimeoutAnnotationKey = \"post.hook.restore.velero.io/exec-timeout\"\n\tpodRestoreHookWaitTimeoutAnnotationKey = \"post.hook.restore.velero.io/wait-timeout\"\n\tpodRestoreHookWaitForReadyAnnotationKey = \"post.hook.restore.velero.io/wait-for-ready\"\n\tpodRestoreHookInitContainerImageAnnotationKey = \"init.hook.restore.velero.io/container-image\"\n\tpodRestoreHookInitContainerNameAnnotationKey = \"init.hook.restore.velero.io/container-name\"\n\tpodRestoreHookInitContainerCommandAnnotationKey = \"init.hook.restore.velero.io/command\"\n\tpodRestoreHookInitContainerTimeoutAnnotationKey = \"init.hook.restore.velero.io/timeout\"\n)\n\n// ItemHookHandler invokes hooks for an item.\ntype ItemHookHandler interface {\n\t// HandleHooks invokes hooks for an item. If the item is a pod and the appropriate annotations exist\n\t// to specify a hook, that is executed. Otherwise, this looks at the backup context's Backup to\n\t// determine if there are any hooks relevant to the item, taking into account the hook spec's\n\t// namespaces, resources, and label selector.\n\tHandleHooks(\n\t\tlog logrus.FieldLogger,\n\t\tgroupResource schema.GroupResource,\n\t\tobj runtime.Unstructured,\n\t\tresourceHooks []ResourceHook,\n\t\tphase HookPhase,\n\t\thookTracker *HookTracker,\n\t) error\n}\n\n// ItemRestoreHookHandler invokes restore hooks for an item\ntype ItemRestoreHookHandler interface {\n\tHandleRestoreHooks(\n\t\tlog logrus.FieldLogger,\n\t\tgroupResource schema.GroupResource,\n\t\tobj runtime.Unstructured,\n\t\trh []ResourceRestoreHook,\n\t) (runtime.Unstructured, error)\n}\n\n// InitContainerRestoreHookHandler is the restore hook handler to add init containers to restored pods.\ntype InitContainerRestoreHookHandler struct{}\n\n// HandleRestoreHooks runs the restore hooks for an item.\n// If the item is a pod, then hooks are chosen to be run as follows:\n// If the pod has the appropriate annotations specifying the hook action, then hooks from the annotation are run\n// Otherwise, the supplied ResourceRestoreHooks are applied.\nfunc (i *InitContainerRestoreHookHandler) HandleRestoreHooks(\n\tlog logrus.FieldLogger,\n\tgroupResource schema.GroupResource,\n\tobj runtime.Unstructured,\n\tresourceRestoreHooks []ResourceRestoreHook,\n\tnamespaceMapping map[string]string,\n) (runtime.Unstructured, error) {\n\t// We only support hooks on pods right now\n\tif groupResource != kuberesource.Pods {\n\t\treturn nil, nil\n\t}\n\n\tmetadata, err := meta.Accessor(obj)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to get a metadata accessor\")\n\t}\n\tpod := new(corev1api.Pod)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(obj.UnstructuredContent(), pod); err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tinitContainers := []corev1api.Container{}\n\t// If this pod is backed up with data movement, then we want to the pod volumes restored prior to\n\t// running the restore hook init containers. This allows the restore hook init containers to prepare the\n\t// restored data to be consumed by the application container(s).\n\t// So if there is a \"restore-wait\" init container already on the pod at index 0, we'll preserve that and run\n\t// it before running any other init container.\n\tif len(pod.Spec.InitContainers) > 0 && pod.Spec.InitContainers[0].Name == restorehelper.WaitInitContainer {\n\t\tinitContainers = append(initContainers, pod.Spec.InitContainers[0])\n\t\tpod.Spec.InitContainers = pod.Spec.InitContainers[1:]\n\t}\n\n\tinitContainerFromAnnotations := getInitContainerFromAnnotation(kube.NamespaceAndName(pod), metadata.GetAnnotations(), log)\n\tif initContainerFromAnnotations != nil {\n\t\tlog.Infof(\"Handling InitRestoreHooks from pod annotations\")\n\t\tinitContainers = append(initContainers, *initContainerFromAnnotations)\n\t} else {\n\t\tlog.Infof(\"Handling InitRestoreHooks from RestoreSpec\")\n\t\t// pod did not have the annotations appropriate for restore hooks\n\t\t// running applicable ResourceRestoreHooks supplied.\n\t\tnamespace := metadata.GetNamespace()\n\t\tlabels := labels.Set(metadata.GetLabels())\n\n\t\t// Apply the hook according to the target namespace in which the pod will be restored\n\t\t// more details see https://github.com/vmware-tanzu/velero/issues/4720\n\t\tif namespaceMapping != nil {\n\t\t\tif n, ok := namespaceMapping[namespace]; ok {\n\t\t\t\tnamespace = n\n\t\t\t}\n\t\t}\n\t\tfor _, rh := range resourceRestoreHooks {\n\t\t\tif !rh.Selector.applicableTo(groupResource, namespace, labels) {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tfor _, hook := range rh.RestoreHooks {\n\t\t\t\tif hook.Init != nil {\n\t\t\t\t\tcontainers := make([]corev1api.Container, 0)\n\t\t\t\t\tfor _, raw := range hook.Init.InitContainers {\n\t\t\t\t\t\tcontainer := corev1api.Container{}\n\t\t\t\t\t\terr := ValidateContainer(raw.Raw)\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\tlog.Errorf(\"invalid Restore Init hook: %s\", err.Error())\n\t\t\t\t\t\t\treturn nil, err\n\t\t\t\t\t\t}\n\t\t\t\t\t\terr = json.Unmarshal(raw.Raw, &container)\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\tlog.Errorf(\"fail to Unmarshal hook Init into container: %s\", err.Error())\n\t\t\t\t\t\t\treturn nil, errors.WithStack(err)\n\t\t\t\t\t\t}\n\t\t\t\t\t\tcontainers = append(containers, container)\n\t\t\t\t\t}\n\t\t\t\t\tinitContainers = append(initContainers, containers...)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\tpod.Spec.InitContainers = append(initContainers, pod.Spec.InitContainers...)\n\tlog.Infof(\"Returning pod %s/%s with %d init container(s)\", pod.Namespace, pod.Name, len(pod.Spec.InitContainers))\n\n\tpodMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&pod)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\treturn &unstructured.Unstructured{Object: podMap}, nil\n}\n\n// DefaultItemHookHandler is the default itemHookHandler.\ntype DefaultItemHookHandler struct {\n\tPodCommandExecutor podexec.PodCommandExecutor\n}\n\nfunc (h *DefaultItemHookHandler) HandleHooks(\n\tlog logrus.FieldLogger,\n\tgroupResource schema.GroupResource,\n\tobj runtime.Unstructured,\n\tresourceHooks []ResourceHook,\n\tphase HookPhase,\n\thookTracker *HookTracker,\n) error {\n\t// We only support hooks on pods right now\n\tif groupResource != kuberesource.Pods {\n\t\treturn nil\n\t}\n\n\tmetadata, err := meta.Accessor(obj)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"unable to get a metadata accessor\")\n\t}\n\n\tnamespace := metadata.GetNamespace()\n\tname := metadata.GetName()\n\n\t// If the pod has the hook specified via annotations, that takes priority.\n\thookFromAnnotations := getPodExecHookFromAnnotations(metadata.GetAnnotations(), phase, log)\n\tif phase == PhasePre && hookFromAnnotations == nil {\n\t\t// See if the pod has the legacy hook annotation keys (i.e. without a phase specified)\n\t\thookFromAnnotations = getPodExecHookFromAnnotations(metadata.GetAnnotations(), \"\", log)\n\t}\n\tif hookFromAnnotations != nil {\n\t\thookTracker.Add(namespace, name, hookFromAnnotations.Container, HookSourceAnnotation, \"\", phase, 0)\n\n\t\thookLog := log.WithFields(\n\t\t\tlogrus.Fields{\n\t\t\t\t\"hookSource\": HookSourceAnnotation,\n\t\t\t\t\"hookType\": \"exec\",\n\t\t\t\t\"hookPhase\": phase,\n\t\t\t},\n\t\t)\n\n\t\thookFailed := false\n\t\tvar errExec error\n\t\tif errExec = h.PodCommandExecutor.ExecutePodCommand(hookLog, obj.UnstructuredContent(), namespace, name, \"\", hookFromAnnotations); errExec != nil {\n\t\t\thookLog.WithError(errExec).Error(\"Error executing hook\")\n\t\t\thookFailed = true\n\t\t}\n\t\terrTracker := hookTracker.Record(namespace, name, hookFromAnnotations.Container, HookSourceAnnotation, \"\", phase, 0, hookFailed, errExec)\n\t\tif errTracker != nil {\n\t\t\thookLog.WithError(errTracker).Warn(\"Error recording the hook in hook tracker\")\n\t\t}\n\n\t\tif errExec != nil && hookFromAnnotations.OnError == velerov1api.HookErrorModeFail {\n\t\t\treturn errExec\n\t\t}\n\n\t\treturn nil\n\t}\n\n\tlabels := labels.Set(metadata.GetLabels())\n\t// Otherwise, check for hooks defined in the backup spec.\n\t// modeFailError records the error from the hook with \"Fail\" error mode\n\tvar modeFailError error\n\tfor _, resourceHook := range resourceHooks {\n\t\tif !resourceHook.Selector.applicableTo(groupResource, namespace, labels) {\n\t\t\tcontinue\n\t\t}\n\n\t\tvar hooks []velerov1api.BackupResourceHook\n\t\tif phase == PhasePre {\n\t\t\thooks = resourceHook.Pre\n\t\t} else {\n\t\t\thooks = resourceHook.Post\n\t\t}\n\n\t\tfor i, hook := range hooks {\n\t\t\tif groupResource == kuberesource.Pods {\n\t\t\t\tif hook.Exec != nil {\n\t\t\t\t\thookTracker.Add(namespace, name, hook.Exec.Container, HookSourceSpec, resourceHook.Name, phase, i)\n\t\t\t\t\t// The remaining hooks will only be executed if modeFailError is nil.\n\t\t\t\t\t// Otherwise, execution will stop and only hook collection will occur.\n\t\t\t\t\tif modeFailError == nil {\n\t\t\t\t\t\thookLog := log.WithFields(\n\t\t\t\t\t\t\tlogrus.Fields{\n\t\t\t\t\t\t\t\t\"hookSource\": HookSourceSpec,\n\t\t\t\t\t\t\t\t\"hookType\": \"exec\",\n\t\t\t\t\t\t\t\t\"hookPhase\": phase,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t)\n\n\t\t\t\t\t\thookFailed := false\n\t\t\t\t\t\terr := h.PodCommandExecutor.ExecutePodCommand(hookLog, obj.UnstructuredContent(), namespace, name, resourceHook.Name, hook.Exec)\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\thookLog.WithError(err).Error(\"Error executing hook\")\n\t\t\t\t\t\t\thookFailed = true\n\t\t\t\t\t\t\tif hook.Exec.OnError == velerov1api.HookErrorModeFail {\n\t\t\t\t\t\t\t\tmodeFailError = err\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}\n\t\t\t\t\t\terrTracker := hookTracker.Record(namespace, name, hook.Exec.Container, HookSourceSpec, resourceHook.Name, phase, i, hookFailed, err)\n\t\t\t\t\t\tif errTracker != nil {\n\t\t\t\t\t\t\thookLog.WithError(errTracker).Warn(\"Error recording the hook in hook tracker\")\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\treturn modeFailError\n}\n\n// NoOpItemHookHandler is the an itemHookHandler for the Finalize controller where hooks don't run\ntype NoOpItemHookHandler struct{}\n\nfunc (h *NoOpItemHookHandler) HandleHooks(\n\tlog logrus.FieldLogger,\n\tgroupResource schema.GroupResource,\n\tobj runtime.Unstructured,\n\tresourceHooks []ResourceHook,\n\tphase HookPhase,\n\thookTracker *HookTracker,\n) error {\n\treturn nil\n}\n\nfunc phasedKey(phase HookPhase, key string) string {\n\tif phase != \"\" {\n\t\treturn fmt.Sprintf(\"%v.%v\", phase, key)\n\t}\n\treturn key\n}\n\nfunc getHookAnnotation(annotations map[string]string, key string, phase HookPhase) string {\n\treturn annotations[phasedKey(phase, key)]\n}\n\n// getPodExecHookFromAnnotations returns an ExecHook based on the annotations, as long as the\n// 'command' annotation is present. If it is absent, this returns nil.\n// If there is an error in parsing a supplied timeout, it is logged.\nfunc getPodExecHookFromAnnotations(annotations map[string]string, phase HookPhase, log logrus.FieldLogger) *velerov1api.ExecHook {\n\tcommandValue := getHookAnnotation(annotations, podBackupHookCommandAnnotationKey, phase)\n\tif commandValue == \"\" {\n\t\treturn nil\n\t}\n\n\tcontainer := getHookAnnotation(annotations, podBackupHookContainerAnnotationKey, phase)\n\n\tonError := velerov1api.HookErrorMode(getHookAnnotation(annotations, podBackupHookOnErrorAnnotationKey, phase))\n\tif onError != velerov1api.HookErrorModeContinue && onError != velerov1api.HookErrorModeFail {\n\t\tonError = \"\"\n\t}\n\n\tvar timeout time.Duration\n\ttimeoutString := getHookAnnotation(annotations, podBackupHookTimeoutAnnotationKey, phase)\n\tif timeoutString != \"\" {\n\t\tif temp, err := time.ParseDuration(timeoutString); err == nil {\n\t\t\ttimeout = temp\n\t\t} else {\n\t\t\tlog.Warn(errors.Wrapf(err, \"Unable to parse provided timeout %s, using default\", timeoutString))\n\t\t}\n\t}\n\n\treturn &velerov1api.ExecHook{\n\t\tContainer: container,\n\t\tCommand: parseStringToCommand(commandValue),\n\t\tOnError: onError,\n\t\tTimeout: metav1.Duration{Duration: timeout},\n\t}\n}\n\nfunc parseStringToCommand(commandValue string) []string {\n\tvar command []string\n\t// check for json array\n\tif commandValue[0] == '[' {\n\t\tif err := json.Unmarshal([]byte(commandValue), &command); err != nil {\n\t\t\tcommand = []string{commandValue}\n\t\t}\n\t} else {\n\t\tcommand = append(command, commandValue)\n\t}\n\treturn command\n}\n\ntype ResourceHookSelector struct {\n\tNamespaces *collections.IncludesExcludes\n\tResources *collections.IncludesExcludes\n\tLabelSelector labels.Selector\n}\n\n// ResourceHook is a hook for a given resource.\ntype ResourceHook struct {\n\tName string\n\tSelector ResourceHookSelector\n\tPre []velerov1api.BackupResourceHook\n\tPost []velerov1api.BackupResourceHook\n}\n\nfunc (r ResourceHookSelector) applicableTo(groupResource schema.GroupResource, namespace string, labels labels.Set) bool {\n\tif r.Namespaces != nil && !r.Namespaces.ShouldInclude(namespace) {\n\t\treturn false\n\t}\n\tif r.Resources != nil && !r.Resources.ShouldInclude(groupResource.String()) {\n\t\treturn false\n\t}\n\tif r.LabelSelector != nil && !r.LabelSelector.Matches(labels) {\n\t\treturn false\n\t}\n\treturn true\n}\n\n// ResourceRestoreHook is a restore hook for a given resource.\ntype ResourceRestoreHook struct {\n\tName string\n\tSelector ResourceHookSelector\n\tRestoreHooks []velerov1api.RestoreResourceHook\n}\n\nfunc getInitContainerFromAnnotation(podName string, annotations map[string]string, log logrus.FieldLogger) *corev1api.Container {\n\tcontainerImage := annotations[podRestoreHookInitContainerImageAnnotationKey]\n\tcontainerName := annotations[podRestoreHookInitContainerNameAnnotationKey]\n\tcommand := annotations[podRestoreHookInitContainerCommandAnnotationKey]\n\tif containerImage == \"\" {\n\t\tlog.Infof(\"Pod %s has no %s annotation, no initRestoreHook in annotation\", podName, podRestoreHookInitContainerImageAnnotationKey)\n\t\treturn nil\n\t}\n\tif command == \"\" {\n\t\tlog.Infof(\"RestoreHook init container for pod %s is using container's default entrypoint\", podName, containerImage)\n\t}\n\tif containerName == \"\" {\n\t\tuid, err := uuid.NewRandom()\n\t\tuuidStr := \"deadfeed\"\n\t\tif err != nil {\n\t\t\tlog.Errorf(\"Failed to generate UUID for container name\")\n\t\t} else {\n\t\t\tuuidStr = strings.Split(uid.String(), \"-\")[0]\n\t\t}\n\t\tcontainerName = fmt.Sprintf(\"velero-restore-init-%s\", uuidStr)\n\t\tlog.Infof(\"Pod %s has no %s annotation, using generated name %s for initContainer\", podName, podRestoreHookInitContainerNameAnnotationKey, containerName)\n\t}\n\n\tinitContainer := corev1api.Container{\n\t\tImage: containerImage,\n\t\tName: containerName,\n\t\tCommand: parseStringToCommand(command),\n\t}\n\n\treturn &initContainer\n}\n\n// GetRestoreHooksFromSpec returns a list of ResourceRestoreHooks from the restore Spec.\nfunc GetRestoreHooksFromSpec(hooksSpec *velerov1api.RestoreHooks) ([]ResourceRestoreHook, error) {\n\tif hooksSpec == nil {\n\t\treturn []ResourceRestoreHook{}, nil\n\t}\n\trestoreHooks := make([]ResourceRestoreHook, 0, len(hooksSpec.Resources))\n\tfor _, rs := range hooksSpec.Resources {\n\t\trh := ResourceRestoreHook{\n\t\t\tName: rs.Name,\n\t\t\tSelector: ResourceHookSelector{\n\t\t\t\tNamespaces: collections.NewIncludesExcludes().Includes(rs.IncludedNamespaces...).Excludes(rs.ExcludedNamespaces...),\n\t\t\t\t// these hooks ara applicable only to pods.\n\t\t\t\t// TODO: resolve the pods resource via discovery?\n\t\t\t\tResources: collections.NewIncludesExcludes().Includes(kuberesource.Pods.Resource),\n\t\t\t},\n\t\t\t// TODO does this work for ExecRestoreHook as well?\n\t\t\tRestoreHooks: rs.PostHooks,\n\t\t}\n\n\t\tif rs.LabelSelector != nil {\n\t\t\tls, err := metav1.LabelSelectorAsSelector(rs.LabelSelector)\n\t\t\tif err != nil {\n\t\t\t\treturn []ResourceRestoreHook{}, errors.WithStack(err)\n\t\t\t}\n\t\t\trh.Selector.LabelSelector = ls\n\t\t}\n\t\trestoreHooks = append(restoreHooks, rh)\n\t}\n\n\treturn restoreHooks, nil\n}\n\n// getPodExecRestoreHookFromAnnotations returns an ExecRestoreHook based on restore annotations, as\n// long as the 'command' annotation is present. If it is absent, this returns nil.\nfunc getPodExecRestoreHookFromAnnotations(annotations map[string]string, log logrus.FieldLogger) *velerov1api.ExecRestoreHook {\n\tcommandValue := annotations[podRestoreHookCommandAnnotationKey]\n\tif commandValue == \"\" {\n\t\treturn nil\n\t}\n\n\tcontainer := annotations[podRestoreHookContainerAnnotationKey]\n\n\tonError := velerov1api.HookErrorMode(annotations[podRestoreHookOnErrorAnnotationKey])\n\tif onError != velerov1api.HookErrorModeContinue && onError != velerov1api.HookErrorModeFail {\n\t\tonError = \"\"\n\t}\n\n\tvar execTimeout time.Duration\n\texecTimeoutString := annotations[podRestoreHookTimeoutAnnotationKey]\n\tif execTimeoutString != \"\" {\n\t\tif temp, err := time.ParseDuration(execTimeoutString); err == nil {\n\t\t\texecTimeout = temp\n\t\t} else {\n\t\t\tlog.Warn(errors.Wrapf(err, \"Unable to parse exec timeout %s, ignoring\", execTimeoutString))\n\t\t}\n\t}\n\n\tvar waitTimeout time.Duration\n\twaitTimeoutString := annotations[podRestoreHookWaitTimeoutAnnotationKey]\n\tif waitTimeoutString != \"\" {\n\t\tif temp, err := time.ParseDuration(waitTimeoutString); err == nil {\n\t\t\twaitTimeout = temp\n\t\t} else {\n\t\t\tlog.Warn(errors.Wrapf(err, \"Unable to parse wait timeout %s, ignoring\", waitTimeoutString))\n\t\t}\n\t}\n\n\twaitForReadyString := annotations[podRestoreHookWaitForReadyAnnotationKey]\n\twaitForReady := boolptr.False()\n\tif waitForReadyString != \"\" {\n\t\tvar err error\n\t\t*waitForReady, err = strconv.ParseBool(waitForReadyString)\n\t\tif err != nil {\n\t\t\tlog.Warn(errors.Wrapf(err, \"Unable to parse wait for ready %s, ignoring\", waitForReadyString))\n\t\t}\n\t}\n\n\treturn &velerov1api.ExecRestoreHook{\n\t\tContainer: container,\n\t\tCommand: parseStringToCommand(commandValue),\n\t\tOnError: onError,\n\t\tExecTimeout: metav1.Duration{Duration: execTimeout},\n\t\tWaitTimeout: metav1.Duration{Duration: waitTimeout},\n\t\tWaitForReady: waitForReady,\n\t}\n}\n\ntype PodExecRestoreHook struct {\n\tHookName string\n\tHookSource string\n\tHook velerov1api.ExecRestoreHook\n\texecuted bool\n\t// hookIndex contains the slice index for the specific hook from the restore spec\n\t// in order to track multiple hooks. Stored here because restore hook results are recorded\n\t// outside of the original slice iteration\n\t// for the same container\n\thookIndex int\n}\n\n// GroupRestoreExecHooks returns a list of hooks to be executed in a pod grouped by\n// container name. If an exec hook is defined in annotation that is used, else applicable exec\n// hooks from the restore resource are accumulated.\nfunc GroupRestoreExecHooks(\n\trestoreName string,\n\tresourceRestoreHooks []ResourceRestoreHook,\n\tpod *corev1api.Pod,\n\tlog logrus.FieldLogger,\n\thookTrack *MultiHookTracker,\n) (map[string][]PodExecRestoreHook, error) {\n\tbyContainer := map[string][]PodExecRestoreHook{}\n\n\tif pod == nil || len(pod.Spec.Containers) == 0 {\n\t\treturn byContainer, nil\n\t}\n\tmetadata, err := meta.Accessor(pod)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\thookFromAnnotation := getPodExecRestoreHookFromAnnotations(metadata.GetAnnotations(), log)\n\tif hookFromAnnotation != nil {\n\t\t// default to first container in pod if unset\n\t\tif hookFromAnnotation.Container == \"\" {\n\t\t\thookFromAnnotation.Container = pod.Spec.Containers[0].Name\n\t\t}\n\t\thookTrack.Add(restoreName, metadata.GetNamespace(), metadata.GetName(), hookFromAnnotation.Container, HookSourceAnnotation, \"\", HookPhase(\"\"), 0)\n\t\tbyContainer[hookFromAnnotation.Container] = []PodExecRestoreHook{\n\t\t\t{\n\t\t\t\tHookName: \"\",\n\t\t\t\tHookSource: HookSourceAnnotation,\n\t\t\t\tHook: *hookFromAnnotation,\n\t\t\t\thookIndex: 0,\n\t\t\t},\n\t\t}\n\t\treturn byContainer, nil\n\t}\n\n\t// No hook found on pod's annotations so check for applicable hooks from the restore spec\n\tlabels := metadata.GetLabels()\n\tnamespace := metadata.GetNamespace()\n\tfor _, rrh := range resourceRestoreHooks {\n\t\tif !rrh.Selector.applicableTo(kuberesource.Pods, namespace, labels) {\n\t\t\tcontinue\n\t\t}\n\t\tfor i, rh := range rrh.RestoreHooks {\n\t\t\tif rh.Exec == nil {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tnamed := PodExecRestoreHook{\n\t\t\t\tHookName: rrh.Name,\n\t\t\t\tHook: *rh.Exec,\n\t\t\t\tHookSource: HookSourceSpec,\n\t\t\t\thookIndex: i,\n\t\t\t}\n\t\t\t// default to false if attr WaitForReady not set\n\t\t\tif named.Hook.WaitForReady == nil {\n\t\t\t\tnamed.Hook.WaitForReady = boolptr.False()\n\t\t\t}\n\t\t\t// default to first container in pod if unset, without mutating resource restore hook\n\t\t\tif named.Hook.Container == \"\" {\n\t\t\t\tnamed.Hook.Container = pod.Spec.Containers[0].Name\n\t\t\t}\n\t\t\thookTrack.Add(restoreName, metadata.GetNamespace(), metadata.GetName(), named.Hook.Container, HookSourceSpec, rrh.Name, HookPhase(\"\"), i)\n\t\t\tbyContainer[named.Hook.Container] = append(byContainer[named.Hook.Container], named)\n\t\t}\n\t}\n\n\treturn byContainer, nil\n}\n\n// ValidateContainer validate whether a map contains mandatory k8s Container fields.\n// mandatory fields include name, image and commands.\nfunc ValidateContainer(raw []byte) error {\n\tcontainer := corev1api.Container{}\n\terr := json.Unmarshal(raw, &container)\n\tif err != nil {\n\t\treturn err\n\t}\n\tif len(container.Command) <= 0 || len(container.Name) <= 0 || len(container.Image) <= 0 {\n\t\treturn fmt.Errorf(\"invalid InitContainer in restore hook, it doesn't have Command, Name or Image field\")\n\t}\n\treturn nil\n}\n" }, { "path": "internal/hook/item_hook_handler_test.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage hook\n\nimport (\n\t\"fmt\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/collections\"\n)\n\nfunc TestHandleHooksSkips(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tgroupResource string\n\t\titem runtime.Unstructured\n\t\thooks []ResourceHook\n\t}{\n\t\t{\n\t\t\tname: \"not a pod\",\n\t\t\tgroupResource: \"widget.group\",\n\t\t},\n\t\t{\n\t\t\tname: \"pod without annotation / no spec hooks\",\n\t\t\titem: velerotest.UnstructuredOrDie(\n\t\t\t\t`\n\t\t\t\t{\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"kind\": \"Pod\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns\",\n\t\t\t\t\t\t\"name\": \"foo\"\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\t`,\n\t\t\t),\n\t\t},\n\t\t{\n\t\t\tname: \"spec hooks not applicable\",\n\t\t\tgroupResource: \"pods\",\n\t\t\titem: velerotest.UnstructuredOrDie(\n\t\t\t\t`\n\t\t\t\t{\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"kind\": \"Pod\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns\",\n\t\t\t\t\t\t\"name\": \"foo\",\n\t\t\t\t\t\t\"labels\": {\n\t\t\t\t\t\t\t\"color\": \"blue\"\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\t`,\n\t\t\t),\n\t\t\thooks: []ResourceHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"ns exclude\",\n\t\t\t\t\tSelector: ResourceHookSelector{Namespaces: collections.NewIncludesExcludes().Excludes(\"ns\")},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"resource exclude\",\n\t\t\t\t\tSelector: ResourceHookSelector{Resources: collections.NewIncludesExcludes().Includes(\"widgets.group\")},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"label selector mismatch\",\n\t\t\t\t\tSelector: ResourceHookSelector{LabelSelector: parseLabelSelectorOrDie(\"color=green\")},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"missing exec hook\",\n\t\t\t\t\tPre: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t{},\n\t\t\t\t\t\t{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\thookTracker := NewHookTracker()\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tpodCommandExecutor := &velerotest.MockPodCommandExecutor{}\n\t\t\tdefer podCommandExecutor.AssertExpectations(t)\n\n\t\t\th := &DefaultItemHookHandler{\n\t\t\t\tPodCommandExecutor: podCommandExecutor,\n\t\t\t}\n\n\t\t\tgroupResource := schema.ParseGroupResource(test.groupResource)\n\t\t\terr := h.HandleHooks(velerotest.NewLogger(), groupResource, test.item, test.hooks, PhasePre, hookTracker)\n\t\t\trequire.NoError(t, err)\n\t\t})\n\t}\n}\n\nfunc TestHandleHooks(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tphase HookPhase\n\t\tgroupResource string\n\t\titem runtime.Unstructured\n\t\thooks []ResourceHook\n\t\thookErrorsByContainer map[string]error\n\t\texpectedError error\n\t\texpectedPodHook *velerov1api.ExecHook\n\t\texpectedPodHookError error\n\t}{\n\t\t{\n\t\t\tname: \"pod, no annotation, spec (multiple pre hooks) = run spec\",\n\t\t\tphase: PhasePre,\n\t\t\tgroupResource: \"pods\",\n\t\t\titem: velerotest.UnstructuredOrDie(`\n\t\t{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"Pod\",\n\t\t\t\"metadata\": {\n\t\t\t\t\"namespace\": \"ns\",\n\t\t\t\t\"name\": \"name\"\n\t\t\t}\n\t\t}`),\n\t\t\thooks: []ResourceHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\tPre: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\tContainer: \"1a\",\n\t\t\t\t\t\t\t\tCommand: []string{\"pre-1a\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\tContainer: \"1b\",\n\t\t\t\t\t\t\t\tCommand: []string{\"pre-1b\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"hook2\",\n\t\t\t\t\tPre: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\tContainer: \"2a\",\n\t\t\t\t\t\t\t\tCommand: []string{\"2a\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\tContainer: \"2b\",\n\t\t\t\t\t\t\t\tCommand: []string{\"2b\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"pod, no annotation, spec (multiple post hooks) = run spec\",\n\t\t\tphase: PhasePost,\n\t\t\tgroupResource: \"pods\",\n\t\t\titem: velerotest.UnstructuredOrDie(`\n\t\t{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"Pod\",\n\t\t\t\"metadata\": {\n\t\t\t\t\"namespace\": \"ns\",\n\t\t\t\t\"name\": \"name\"\n\t\t\t}\n\t\t}`),\n\t\t\thooks: []ResourceHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\tPost: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\tContainer: \"1a\",\n\t\t\t\t\t\t\t\tCommand: []string{\"pre-1a\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\tContainer: \"1b\",\n\t\t\t\t\t\t\t\tCommand: []string{\"pre-1b\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"hook2\",\n\t\t\t\t\tPost: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\tContainer: \"2a\",\n\t\t\t\t\t\t\t\tCommand: []string{\"2a\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\tContainer: \"2b\",\n\t\t\t\t\t\t\t\tCommand: []string{\"2b\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"pod, annotation (legacy), no spec = run annotation\",\n\t\t\tphase: PhasePre,\n\t\t\tgroupResource: \"pods\",\n\t\t\titem: velerotest.UnstructuredOrDie(`\n\t\t{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"Pod\",\n\t\t\t\"metadata\": {\n\t\t\t\t\"namespace\": \"ns\",\n\t\t\t\t\"name\": \"name\",\n\t\t\t\t\"annotations\": {\n\t\t\t\t\t\"hook.backup.velero.io/container\": \"c\",\n\t\t\t\t\t\"hook.backup.velero.io/command\": \"/bin/ls\"\n\t\t\t\t}\n\t\t\t}\n\t\t}`),\n\t\t\texpectedPodHook: &velerov1api.ExecHook{\n\t\t\t\tContainer: \"c\",\n\t\t\t\tCommand: []string{\"/bin/ls\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"pod, annotation (pre), no spec = run annotation\",\n\t\t\tphase: PhasePre,\n\t\t\tgroupResource: \"pods\",\n\t\t\titem: velerotest.UnstructuredOrDie(`\n\t\t{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"Pod\",\n\t\t\t\"metadata\": {\n\t\t\t\t\"namespace\": \"ns\",\n\t\t\t\t\"name\": \"name\",\n\t\t\t\t\"annotations\": {\n\t\t\t\t\t\"pre.hook.backup.velero.io/container\": \"c\",\n\t\t\t\t\t\"pre.hook.backup.velero.io/command\": \"/bin/ls\"\n\t\t\t\t}\n\t\t\t}\n\t\t}`),\n\t\t\texpectedPodHook: &velerov1api.ExecHook{\n\t\t\t\tContainer: \"c\",\n\t\t\t\tCommand: []string{\"/bin/ls\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"pod, annotation (post), no spec = run annotation\",\n\t\t\tphase: PhasePost,\n\t\t\tgroupResource: \"pods\",\n\t\t\titem: velerotest.UnstructuredOrDie(`\n\t\t{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"Pod\",\n\t\t\t\"metadata\": {\n\t\t\t\t\"namespace\": \"ns\",\n\t\t\t\t\"name\": \"name\",\n\t\t\t\t\"annotations\": {\n\t\t\t\t\t\"post.hook.backup.velero.io/container\": \"c\",\n\t\t\t\t\t\"post.hook.backup.velero.io/command\": \"/bin/ls\"\n\t\t\t\t}\n\t\t\t}\n\t\t}`),\n\t\t\texpectedPodHook: &velerov1api.ExecHook{\n\t\t\t\tContainer: \"c\",\n\t\t\t\tCommand: []string{\"/bin/ls\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"pod, annotation & spec = run annotation\",\n\t\t\tphase: PhasePre,\n\t\t\tgroupResource: \"pods\",\n\t\t\titem: velerotest.UnstructuredOrDie(`\n\t\t{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"Pod\",\n\t\t\t\"metadata\": {\n\t\t\t\t\"namespace\": \"ns\",\n\t\t\t\t\"name\": \"name\",\n\t\t\t\t\"annotations\": {\n\t\t\t\t\t\"hook.backup.velero.io/container\": \"c\",\n\t\t\t\t\t\"hook.backup.velero.io/command\": \"/bin/ls\"\n\t\t\t\t}\n\t\t\t}\n\t\t}`),\n\t\t\texpectedPodHook: &velerov1api.ExecHook{\n\t\t\t\tContainer: \"c\",\n\t\t\t\tCommand: []string{\"/bin/ls\"},\n\t\t\t},\n\t\t\thooks: []ResourceHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\tPre: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\tContainer: \"1a\",\n\t\t\t\t\t\t\t\tCommand: []string{\"1a\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"pod, annotation, onError=fail = return error\",\n\t\t\tphase: PhasePre,\n\t\t\tgroupResource: \"pods\",\n\t\t\titem: velerotest.UnstructuredOrDie(`\n\t\t{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"Pod\",\n\t\t\t\"metadata\": {\n\t\t\t\t\"namespace\": \"ns\",\n\t\t\t\t\"name\": \"name\",\n\t\t\t\t\"annotations\": {\n\t\t\t\t\t\"hook.backup.velero.io/container\": \"c\",\n\t\t\t\t\t\"hook.backup.velero.io/command\": \"/bin/ls\",\n\t\t\t\t\t\"hook.backup.velero.io/on-error\": \"Fail\"\n\t\t\t\t}\n\t\t\t}\n\t\t}`),\n\t\t\texpectedPodHook: &velerov1api.ExecHook{\n\t\t\t\tContainer: \"c\",\n\t\t\t\tCommand: []string{\"/bin/ls\"},\n\t\t\t\tOnError: velerov1api.HookErrorModeFail,\n\t\t\t},\n\t\t\texpectedPodHookError: errors.New(\"pod hook error\"),\n\t\t\texpectedError: errors.New(\"pod hook error\"),\n\t\t},\n\t\t{\n\t\t\tname: \"pod, annotation, onError=continue = return nil\",\n\t\t\tphase: PhasePre,\n\t\t\tgroupResource: \"pods\",\n\t\t\titem: velerotest.UnstructuredOrDie(`\n\t\t{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"Pod\",\n\t\t\t\"metadata\": {\n\t\t\t\t\"namespace\": \"ns\",\n\t\t\t\t\"name\": \"name\",\n\t\t\t\t\"annotations\": {\n\t\t\t\t\t\"hook.backup.velero.io/container\": \"c\",\n\t\t\t\t\t\"hook.backup.velero.io/command\": \"/bin/ls\",\n\t\t\t\t\t\"hook.backup.velero.io/on-error\": \"Continue\"\n\t\t\t\t}\n\t\t\t}\n\t\t}`),\n\t\t\texpectedPodHook: &velerov1api.ExecHook{\n\t\t\t\tContainer: \"c\",\n\t\t\t\tCommand: []string{\"/bin/ls\"},\n\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t},\n\t\t\texpectedPodHookError: errors.New(\"pod hook error\"),\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"pod, spec, onError=fail = don't run other hooks\",\n\t\t\tphase: PhasePre,\n\t\t\tgroupResource: \"pods\",\n\t\t\titem: velerotest.UnstructuredOrDie(`\n\t\t{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"Pod\",\n\t\t\t\"metadata\": {\n\t\t\t\t\"namespace\": \"ns\",\n\t\t\t\t\"name\": \"name\"\n\t\t\t}\n\t\t}`),\n\t\t\thooks: []ResourceHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\tPre: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\tContainer: \"1a\",\n\t\t\t\t\t\t\t\tCommand: []string{\"1a\"},\n\t\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\tContainer: \"1b\",\n\t\t\t\t\t\t\t\tCommand: []string{\"1b\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"hook2\",\n\t\t\t\t\tPre: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\tContainer: \"2\",\n\t\t\t\t\t\t\t\tCommand: []string{\"2\"},\n\t\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeFail,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"hook3\",\n\t\t\t\t\tPre: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\tContainer: \"3\",\n\t\t\t\t\t\t\t\tCommand: []string{\"3\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\thookErrorsByContainer: map[string]error{\n\t\t\t\t\"1a\": errors.New(\"1a error, but continue\"),\n\t\t\t\t\"2\": errors.New(\"2 error, fail\"),\n\t\t\t},\n\t\t\texpectedError: errors.New(\"2 error, fail\"),\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tpodCommandExecutor := &velerotest.MockPodCommandExecutor{}\n\t\t\tdefer podCommandExecutor.AssertExpectations(t)\n\n\t\t\th := &DefaultItemHookHandler{\n\t\t\t\tPodCommandExecutor: podCommandExecutor,\n\t\t\t}\n\n\t\t\tif test.expectedPodHook != nil {\n\t\t\t\tpodCommandExecutor.On(\"ExecutePodCommand\", mock.Anything, test.item.UnstructuredContent(), \"ns\", \"name\", \"\", test.expectedPodHook).Return(test.expectedPodHookError)\n\t\t\t} else {\n\t\t\thookLoop:\n\t\t\t\tfor _, resourceHook := range test.hooks {\n\t\t\t\t\tfor _, hook := range resourceHook.Pre {\n\t\t\t\t\t\thookError := test.hookErrorsByContainer[hook.Exec.Container]\n\t\t\t\t\t\tpodCommandExecutor.On(\"ExecutePodCommand\", mock.Anything, test.item.UnstructuredContent(), \"ns\", \"name\", resourceHook.Name, hook.Exec).Return(hookError)\n\t\t\t\t\t\tif hookError != nil && hook.Exec.OnError == velerov1api.HookErrorModeFail {\n\t\t\t\t\t\t\tbreak hookLoop\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t\tfor _, hook := range resourceHook.Post {\n\t\t\t\t\t\thookError := test.hookErrorsByContainer[hook.Exec.Container]\n\t\t\t\t\t\tpodCommandExecutor.On(\"ExecutePodCommand\", mock.Anything, test.item.UnstructuredContent(), \"ns\", \"name\", resourceHook.Name, hook.Exec).Return(hookError)\n\t\t\t\t\t\tif hookError != nil && hook.Exec.OnError == velerov1api.HookErrorModeFail {\n\t\t\t\t\t\t\tbreak hookLoop\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tgroupResource := schema.ParseGroupResource(test.groupResource)\n\t\t\thookTracker := NewHookTracker()\n\t\t\terr := h.HandleHooks(velerotest.NewLogger(), groupResource, test.item, test.hooks, test.phase, hookTracker)\n\n\t\t\tif test.expectedError != nil {\n\t\t\t\tassert.EqualError(t, err, test.expectedError.Error())\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\trequire.NoError(t, err)\n\t\t})\n\t}\n}\n\nfunc TestGetPodExecHookFromAnnotations(t *testing.T) {\n\tphases := []HookPhase{\"\", PhasePre, PhasePost}\n\tfor _, phase := range phases {\n\t\ttests := []struct {\n\t\t\tname string\n\t\t\tannotations map[string]string\n\t\t\texpectedHook *velerov1api.ExecHook\n\t\t}{\n\t\t\t{\n\t\t\t\tname: \"missing command annotation\",\n\t\t\t\texpectedHook: nil,\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"malformed command json array\",\n\t\t\t\tannotations: map[string]string{\n\t\t\t\t\tphasedKey(phase, podBackupHookCommandAnnotationKey): \"[blarg\",\n\t\t\t\t},\n\t\t\t\texpectedHook: &velerov1api.ExecHook{\n\t\t\t\t\tCommand: []string{\"[blarg\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"valid command json array\",\n\t\t\t\tannotations: map[string]string{\n\t\t\t\t\tphasedKey(phase, podBackupHookCommandAnnotationKey): `[\"a\",\"b\",\"c\"]`,\n\t\t\t\t},\n\t\t\t\texpectedHook: &velerov1api.ExecHook{\n\t\t\t\t\tCommand: []string{\"a\", \"b\", \"c\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"command as a string\",\n\t\t\t\tannotations: map[string]string{\n\t\t\t\t\tphasedKey(phase, podBackupHookCommandAnnotationKey): \"/usr/bin/foo\",\n\t\t\t\t},\n\t\t\t\texpectedHook: &velerov1api.ExecHook{\n\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"hook mode set to continue\",\n\t\t\t\tannotations: map[string]string{\n\t\t\t\t\tphasedKey(phase, podBackupHookCommandAnnotationKey): \"/usr/bin/foo\",\n\t\t\t\t\tphasedKey(phase, podBackupHookOnErrorAnnotationKey): string(velerov1api.HookErrorModeContinue),\n\t\t\t\t},\n\t\t\t\texpectedHook: &velerov1api.ExecHook{\n\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"hook mode set to fail\",\n\t\t\t\tannotations: map[string]string{\n\t\t\t\t\tphasedKey(phase, podBackupHookCommandAnnotationKey): \"/usr/bin/foo\",\n\t\t\t\t\tphasedKey(phase, podBackupHookOnErrorAnnotationKey): string(velerov1api.HookErrorModeFail),\n\t\t\t\t},\n\t\t\t\texpectedHook: &velerov1api.ExecHook{\n\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\tOnError: velerov1api.HookErrorModeFail,\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"use the specified timeout\",\n\t\t\t\tannotations: map[string]string{\n\t\t\t\t\tphasedKey(phase, podBackupHookCommandAnnotationKey): \"/usr/bin/foo\",\n\t\t\t\t\tphasedKey(phase, podBackupHookTimeoutAnnotationKey): \"5m3s\",\n\t\t\t\t},\n\t\t\t\texpectedHook: &velerov1api.ExecHook{\n\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\tTimeout: metav1.Duration{Duration: 5*time.Minute + 3*time.Second},\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"invalid timeout is logged\",\n\t\t\t\tannotations: map[string]string{\n\t\t\t\t\tphasedKey(phase, podBackupHookCommandAnnotationKey): \"/usr/bin/foo\",\n\t\t\t\t\tphasedKey(phase, podBackupHookTimeoutAnnotationKey): \"invalid\",\n\t\t\t\t},\n\t\t\t\texpectedHook: &velerov1api.ExecHook{\n\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"use the specified container\",\n\t\t\t\tannotations: map[string]string{\n\t\t\t\t\tphasedKey(phase, podBackupHookContainerAnnotationKey): \"some-container\",\n\t\t\t\t\tphasedKey(phase, podBackupHookCommandAnnotationKey): \"/usr/bin/foo\",\n\t\t\t\t},\n\t\t\t\texpectedHook: &velerov1api.ExecHook{\n\t\t\t\t\tContainer: \"some-container\",\n\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\n\t\tfor _, test := range tests {\n\t\t\tt.Run(fmt.Sprintf(\"%s (phase=%q)\", test.name, phase), func(t *testing.T) {\n\t\t\t\tl := velerotest.NewLogger()\n\t\t\t\thook := getPodExecHookFromAnnotations(test.annotations, phase, l)\n\t\t\t\tassert.Equal(t, test.expectedHook, hook)\n\t\t\t})\n\t\t}\n\t}\n}\n\nfunc TestResourceHookApplicableTo(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tincludedNamespaces []string\n\t\texcludedNamespaces []string\n\t\tincludedResources []string\n\t\texcludedResources []string\n\t\tlabelSelector string\n\t\tnamespace string\n\t\tresource schema.GroupResource\n\t\tlabels labels.Set\n\t\texpected bool\n\t}{\n\t\t{\n\t\t\tname: \"allow anything\",\n\t\t\tnamespace: \"foo\",\n\t\t\tresource: schema.GroupResource{Group: \"foo\", Resource: \"bar\"},\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"namespace in included list\",\n\t\t\tincludedNamespaces: []string{\"a\", \"b\"},\n\t\t\texcludedNamespaces: []string{\"c\", \"d\"},\n\t\t\tnamespace: \"b\",\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"namespace not in included list\",\n\t\t\tincludedNamespaces: []string{\"a\", \"b\"},\n\t\t\tnamespace: \"c\",\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"namespace excluded\",\n\t\t\texcludedNamespaces: []string{\"a\", \"b\"},\n\t\t\tnamespace: \"a\",\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"resource in included list\",\n\t\t\tincludedResources: []string{\"foo.a\", \"bar.b\"},\n\t\t\texcludedResources: []string{\"baz.c\"},\n\t\t\tresource: schema.GroupResource{Group: \"a\", Resource: \"foo\"},\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"resource not in included list\",\n\t\t\tincludedResources: []string{\"foo.a\", \"bar.b\"},\n\t\t\tresource: schema.GroupResource{Group: \"c\", Resource: \"baz\"},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"resource excluded\",\n\t\t\texcludedResources: []string{\"foo.a\", \"bar.b\"},\n\t\t\tresource: schema.GroupResource{Group: \"b\", Resource: \"bar\"},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"label selector matches\",\n\t\t\tlabelSelector: \"a=b\",\n\t\t\tlabels: labels.Set{\"a\": \"b\"},\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"label selector doesn't match\",\n\t\t\tlabelSelector: \"a=b\",\n\t\t\tlabels: labels.Set{\"a\": \"c\"},\n\t\t\texpected: false,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\th := ResourceHook{\n\t\t\t\tSelector: ResourceHookSelector{\n\t\t\t\t\tNamespaces: collections.NewIncludesExcludes().Includes(test.includedNamespaces...).Excludes(test.excludedNamespaces...),\n\t\t\t\t\tResources: collections.NewIncludesExcludes().Includes(test.includedResources...).Excludes(test.excludedResources...),\n\t\t\t\t},\n\t\t\t}\n\t\t\tif test.labelSelector != \"\" {\n\t\t\t\tselector, err := labels.Parse(test.labelSelector)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\th.Selector.LabelSelector = selector\n\t\t\t}\n\n\t\t\tresult := h.Selector.applicableTo(test.resource, test.namespace, test.labels)\n\t\t\tassert.Equal(t, test.expected, result)\n\t\t})\n\t}\n}\n\nfunc parseLabelSelectorOrDie(s string) labels.Selector {\n\tret, err := labels.Parse(s)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn ret\n}\n\nfunc TestGetPodExecRestoreHookFromAnnotations(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tinputAnnotations map[string]string\n\t\texpected *velerov1api.ExecRestoreHook\n\t}{\n\t\t{\n\t\t\tname: \"should return nil when command is missing\",\n\t\t\tinputAnnotations: nil,\n\t\t\texpected: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"should return nil when command is empty string\",\n\t\t\tinputAnnotations: map[string]string{\n\t\t\t\tpodRestoreHookCommandAnnotationKey: \"\",\n\t\t\t},\n\t\t\texpected: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"should return a hook when 1 item command is a string\",\n\t\t\tinputAnnotations: map[string]string{\n\t\t\t\tpodRestoreHookCommandAnnotationKey: \"/usr/bin/foo\",\n\t\t\t},\n\t\t\texpected: &velerov1api.ExecRestoreHook{\n\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\tWaitForReady: boolptr.False(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should return a multi-item hook when command is a json array\",\n\t\t\tinputAnnotations: map[string]string{\n\t\t\t\tpodRestoreHookCommandAnnotationKey: `[\"a\",\"b\",\"c\"]`,\n\t\t\t},\n\t\t\texpected: &velerov1api.ExecRestoreHook{\n\t\t\t\tCommand: []string{\"a\", \"b\", \"c\"},\n\t\t\t\tWaitForReady: boolptr.False(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"error mode continue should be in returned hook when set in annotation\",\n\t\t\tinputAnnotations: map[string]string{\n\t\t\t\tpodRestoreHookCommandAnnotationKey: \"/usr/bin/foo\",\n\t\t\t\tpodRestoreHookOnErrorAnnotationKey: string(velerov1api.HookErrorModeContinue),\n\t\t\t},\n\t\t\texpected: &velerov1api.ExecRestoreHook{\n\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\tWaitForReady: boolptr.False(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"error mode fail should be in returned hook when set in annotation\",\n\t\t\tinputAnnotations: map[string]string{\n\t\t\t\tpodRestoreHookCommandAnnotationKey: \"/usr/bin/foo\",\n\t\t\t\tpodRestoreHookOnErrorAnnotationKey: string(velerov1api.HookErrorModeFail),\n\t\t\t},\n\t\t\texpected: &velerov1api.ExecRestoreHook{\n\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\tOnError: velerov1api.HookErrorModeFail,\n\t\t\t\tWaitForReady: boolptr.False(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"exec and wait timeouts should be in returned hook when set in annotations\",\n\t\t\tinputAnnotations: map[string]string{\n\t\t\t\tpodRestoreHookCommandAnnotationKey: \"/usr/bin/foo\",\n\t\t\t\tpodRestoreHookTimeoutAnnotationKey: \"45s\",\n\t\t\t\tpodRestoreHookWaitTimeoutAnnotationKey: \"1h\",\n\t\t\t},\n\t\t\texpected: &velerov1api.ExecRestoreHook{\n\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\tExecTimeout: metav1.Duration{Duration: 45 * time.Second},\n\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Hour},\n\t\t\t\tWaitForReady: boolptr.False(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"container should be in returned hook when set in annotation\",\n\t\t\tinputAnnotations: map[string]string{\n\t\t\t\tpodRestoreHookCommandAnnotationKey: \"/usr/bin/foo\",\n\t\t\t\tpodRestoreHookContainerAnnotationKey: \"my-app\",\n\t\t\t},\n\t\t\texpected: &velerov1api.ExecRestoreHook{\n\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\tContainer: \"my-app\",\n\t\t\t\tWaitForReady: boolptr.False(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"bad exec timeout should be discarded\",\n\t\t\tinputAnnotations: map[string]string{\n\t\t\t\tpodRestoreHookCommandAnnotationKey: \"/usr/bin/foo\",\n\t\t\t\tpodRestoreHookContainerAnnotationKey: \"my-app\",\n\t\t\t\tpodRestoreHookTimeoutAnnotationKey: \"none\",\n\t\t\t},\n\t\t\texpected: &velerov1api.ExecRestoreHook{\n\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\tContainer: \"my-app\",\n\t\t\t\tExecTimeout: metav1.Duration{Duration: 0},\n\t\t\t\tWaitForReady: boolptr.False(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"bad wait timeout should be discarded\",\n\t\t\tinputAnnotations: map[string]string{\n\t\t\t\tpodRestoreHookCommandAnnotationKey: \"/usr/bin/foo\",\n\t\t\t\tpodRestoreHookContainerAnnotationKey: \"my-app\",\n\t\t\t\tpodRestoreHookWaitTimeoutAnnotationKey: \"none\",\n\t\t\t},\n\t\t\texpected: &velerov1api.ExecRestoreHook{\n\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\tContainer: \"my-app\",\n\t\t\t\tExecTimeout: metav1.Duration{Duration: 0},\n\t\t\t\tWaitForReady: boolptr.False(),\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tl := velerotest.NewLogger()\n\t\t\tactual := getPodExecRestoreHookFromAnnotations(tc.inputAnnotations, l)\n\t\t\tassert.Equal(t, tc.expected, actual)\n\t\t})\n\t}\n}\n\nfunc TestGroupRestoreExecHooks(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tresourceRestoreHooks []ResourceRestoreHook\n\t\tpod *corev1api.Pod\n\t\texpected map[string][]PodExecRestoreHook\n\t}{\n\t\t{\n\t\t\tname: \"should return empty map when neither spec hooks nor annotations hooks are set\",\n\t\t\tresourceRestoreHooks: nil,\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").Result(),\n\t\t\texpected: map[string][]PodExecRestoreHook{},\n\t\t},\n\t\t{\n\t\t\tname: \"should return hook from annotation when no spec hooks are set\",\n\t\t\tresourceRestoreHooks: nil,\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tObjectMeta(builder.WithAnnotations(\n\t\t\t\t\tpodRestoreHookCommandAnnotationKey, \"/usr/bin/foo\",\n\t\t\t\t\tpodRestoreHookContainerAnnotationKey, \"container1\",\n\t\t\t\t\tpodRestoreHookOnErrorAnnotationKey, string(velerov1api.HookErrorModeContinue),\n\t\t\t\t\tpodRestoreHookTimeoutAnnotationKey, \"1s\",\n\t\t\t\t\tpodRestoreHookWaitTimeoutAnnotationKey, \"1m\",\n\t\t\t\t\tpodRestoreHookWaitForReadyAnnotationKey, \"true\",\n\t\t\t\t)).\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\texpected: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"\",\n\t\t\t\t\t\tHookSource: HookSourceAnnotation,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t\tWaitForReady: boolptr.True(),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should default to first pod container when not set in annotation\",\n\t\t\tresourceRestoreHooks: nil,\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tObjectMeta(builder.WithAnnotations(\n\t\t\t\t\tpodRestoreHookCommandAnnotationKey, \"/usr/bin/foo\",\n\t\t\t\t\tpodRestoreHookOnErrorAnnotationKey, string(velerov1api.HookErrorModeContinue),\n\t\t\t\t\tpodRestoreHookTimeoutAnnotationKey, \"1s\",\n\t\t\t\t\tpodRestoreHookWaitTimeoutAnnotationKey, \"1m\",\n\t\t\t\t)).\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\texpected: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"\",\n\t\t\t\t\t\tHookSource: HookSourceAnnotation,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t\tWaitForReady: boolptr.False(),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should return hook from spec for pod with no hook annotations\",\n\t\t\tresourceRestoreHooks: []ResourceRestoreHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\tSelector: ResourceHookSelector{},\n\t\t\t\t\tRestoreHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\texpected: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"hook1\",\n\t\t\t\t\t\tHookSource: HookSourceSpec,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t\tWaitForReady: boolptr.False(),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should default to first container pod when unset in spec hook\",\n\t\t\tresourceRestoreHooks: []ResourceRestoreHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\tSelector: ResourceHookSelector{},\n\t\t\t\t\tRestoreHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\texpected: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"hook1\",\n\t\t\t\t\t\tHookSource: HookSourceSpec,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t\tWaitForReady: boolptr.False(),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should return hook from annotation ignoring hooks in spec\",\n\t\t\tresourceRestoreHooks: []ResourceRestoreHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\tSelector: ResourceHookSelector{},\n\t\t\t\t\tRestoreHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\t\tContainer: \"container2\",\n\t\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/bar\"},\n\t\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeFail,\n\t\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Hour},\n\t\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Hour},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tObjectMeta(builder.WithAnnotations(\n\t\t\t\t\tpodRestoreHookCommandAnnotationKey, \"/usr/bin/foo\",\n\t\t\t\t\tpodRestoreHookContainerAnnotationKey, \"container1\",\n\t\t\t\t\tpodRestoreHookOnErrorAnnotationKey, string(velerov1api.HookErrorModeContinue),\n\t\t\t\t\tpodRestoreHookTimeoutAnnotationKey, \"1s\",\n\t\t\t\t\tpodRestoreHookWaitTimeoutAnnotationKey, \"1m\",\n\t\t\t\t)).\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\texpected: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"\",\n\t\t\t\t\t\tHookSource: HookSourceAnnotation,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t\tWaitForReady: boolptr.False(),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should return empty map when only has init hook and pod has no hook annotations\",\n\t\t\tresourceRestoreHooks: []ResourceRestoreHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\tSelector: ResourceHookSelector{},\n\t\t\t\t\tRestoreHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tInit: &velerov1api.InitRestoreHook{},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\texpected: map[string][]PodExecRestoreHook{},\n\t\t},\n\t\t{\n\t\t\tname: \"should return empty map when spec has exec hook for pod in different namespace and pod has no hook annotations\",\n\t\t\tresourceRestoreHooks: []ResourceRestoreHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\tSelector: ResourceHookSelector{\n\t\t\t\t\t\tNamespaces: collections.NewIncludesExcludes().Includes(\"other\"),\n\t\t\t\t\t},\n\t\t\t\t\tRestoreHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").Result(),\n\t\t\texpected: map[string][]PodExecRestoreHook{},\n\t\t},\n\t\t{\n\t\t\tname: \"should return map with multiple keys when spec hooks apply to multiple containers in pod and has no hook annotations\",\n\t\t\tresourceRestoreHooks: []ResourceRestoreHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\tSelector: ResourceHookSelector{},\n\t\t\t\t\tRestoreHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeFail,\n\t\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\t\tContainer: \"container2\",\n\t\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/baz\"},\n\t\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second * 3},\n\t\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Second * 3},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/bar\"},\n\t\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second * 2},\n\t\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute * 2},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"hook2\",\n\t\t\t\t\tSelector: ResourceHookSelector{},\n\t\t\t\t\tRestoreHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/aaa\"},\n\t\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second * 4},\n\t\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute * 4},\n\t\t\t\t\t\t\t\tWaitForReady: boolptr.True(),\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\texpected: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"hook1\",\n\t\t\t\t\t\tHookSource: HookSourceSpec,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeFail,\n\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t\tWaitForReady: boolptr.False(),\n\t\t\t\t\t\t},\n\t\t\t\t\t\thookIndex: 0,\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"hook1\",\n\t\t\t\t\t\tHookSource: HookSourceSpec,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/bar\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second * 2},\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute * 2},\n\t\t\t\t\t\t\tWaitForReady: boolptr.False(),\n\t\t\t\t\t\t},\n\t\t\t\t\t\thookIndex: 2,\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"hook2\",\n\t\t\t\t\t\tHookSource: HookSourceSpec,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/aaa\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second * 4},\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute * 4},\n\t\t\t\t\t\t\tWaitForReady: boolptr.True(),\n\t\t\t\t\t\t},\n\t\t\t\t\t\thookIndex: 0,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"container2\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"hook1\",\n\t\t\t\t\t\tHookSource: HookSourceSpec,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container2\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/baz\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second * 3},\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Second * 3},\n\t\t\t\t\t\t\tWaitForReady: boolptr.False(),\n\t\t\t\t\t\t},\n\t\t\t\t\t\thookIndex: 1,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\thookTracker := NewMultiHookTracker()\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactual, err := GroupRestoreExecHooks(\"restore1\", tc.resourceRestoreHooks, tc.pod, velerotest.NewLogger(), hookTracker)\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Equal(t, tc.expected, actual)\n\t\t})\n\t}\n}\n\nfunc TestGetInitContainerFromAnnotations(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tinputAnnotations map[string]string\n\t\texpected *corev1api.Container\n\t\texpectNil bool\n\t}{\n\t\t{\n\t\t\tname: \"should return nil when container image is empty\",\n\t\t\texpectNil: true,\n\t\t\tinputAnnotations: map[string]string{\n\t\t\t\tpodRestoreHookInitContainerImageAnnotationKey: \"\",\n\t\t\t\tpodRestoreHookInitContainerNameAnnotationKey: \"restore-init\",\n\t\t\t\tpodRestoreHookInitContainerCommandAnnotationKey: \"/usr/bin/data-populator\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should return nil when container image is missing\",\n\t\t\texpectNil: true,\n\t\t\tinputAnnotations: map[string]string{\n\t\t\t\tpodRestoreHookInitContainerNameAnnotationKey: \"restore-init\",\n\t\t\t\tpodRestoreHookInitContainerCommandAnnotationKey: \"/usr/bin/data-populator\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should generate container name when container name is empty\",\n\t\t\texpectNil: false,\n\t\t\tinputAnnotations: map[string]string{\n\t\t\t\tpodRestoreHookInitContainerImageAnnotationKey: \"busy-box\",\n\t\t\t\tpodRestoreHookInitContainerNameAnnotationKey: \"\",\n\t\t\t\tpodRestoreHookInitContainerCommandAnnotationKey: \"/usr/bin/data-populator /user-data full\",\n\t\t\t},\n\t\t\texpected: builder.ForContainer(\"restore-init1\", \"busy-box\").\n\t\t\t\tCommand([]string{\"/usr/bin/data-populator /user-data full\"}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"should generate container name when container name is missing\",\n\t\t\texpectNil: false,\n\t\t\tinputAnnotations: map[string]string{\n\t\t\t\tpodRestoreHookInitContainerImageAnnotationKey: \"busy-box\",\n\t\t\t\tpodRestoreHookInitContainerCommandAnnotationKey: \"/usr/bin/data-populator /user-data full\",\n\t\t\t},\n\t\t\texpected: builder.ForContainer(\"restore-init1\", \"busy-box\").\n\t\t\t\tCommand([]string{\"/usr/bin/data-populator /user-data full\"}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"should return expected init container when all annotations are specified\",\n\t\t\texpectNil: false,\n\t\t\texpected: builder.ForContainer(\"restore-init1\", \"busy-box\").\n\t\t\t\tCommand([]string{\"/usr/bin/data-populator /user-data full\"}).Result(),\n\t\t\tinputAnnotations: map[string]string{\n\t\t\t\tpodRestoreHookInitContainerImageAnnotationKey: \"busy-box\",\n\t\t\t\tpodRestoreHookInitContainerNameAnnotationKey: \"restore-init\",\n\t\t\t\tpodRestoreHookInitContainerCommandAnnotationKey: \"/usr/bin/data-populator /user-data full\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should return expected init container when all annotations are specified with command as a JSON array\",\n\t\t\texpectNil: false,\n\t\t\texpected: builder.ForContainer(\"restore-init1\", \"busy-box\").\n\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).Result(),\n\t\t\tinputAnnotations: map[string]string{\n\t\t\t\tpodRestoreHookInitContainerImageAnnotationKey: \"busy-box\",\n\t\t\t\tpodRestoreHookInitContainerNameAnnotationKey: \"restore-init\",\n\t\t\t\tpodRestoreHookInitContainerCommandAnnotationKey: `[\"a\",\"b\",\"c\"]`,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should return expected init container when all annotations are specified with command as malformed a JSON array\",\n\t\t\texpectNil: false,\n\t\t\texpected: builder.ForContainer(\"restore-init1\", \"busy-box\").\n\t\t\t\tCommand([]string{\"[foobarbaz\"}).Result(),\n\t\t\tinputAnnotations: map[string]string{\n\t\t\t\tpodRestoreHookInitContainerImageAnnotationKey: \"busy-box\",\n\t\t\t\tpodRestoreHookInitContainerNameAnnotationKey: \"restore-init\",\n\t\t\t\tpodRestoreHookInitContainerCommandAnnotationKey: \"[foobarbaz\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactualInitContainer := getInitContainerFromAnnotation(\"test/pod1\", tc.inputAnnotations, velerotest.NewLogger())\n\t\t\tif tc.expectNil {\n\t\t\t\tassert.Nil(t, actualInitContainer)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tassert.NotEmpty(t, actualInitContainer.Name)\n\t\t\tassert.Equal(t, tc.expected.Image, actualInitContainer.Image)\n\t\t\tassert.Equal(t, tc.expected.Command, actualInitContainer.Command)\n\t\t})\n\t}\n}\n\nfunc TestGetRestoreHooksFromSpec(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\thookSpec *velerov1api.RestoreHooks\n\t\texpected []ResourceRestoreHook\n\t\texpectedError error\n\t}{\n\t\t{\n\t\t\tname: \"should return empty hooks and no error when hookSpec is nil\",\n\t\t\thookSpec: nil,\n\t\t\texpected: []ResourceRestoreHook{},\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"should return empty hooks and no error when hookSpec resources is nil\",\n\t\t\thookSpec: &velerov1api.RestoreHooks{\n\t\t\t\tResources: nil,\n\t\t\t},\n\t\t\texpected: []ResourceRestoreHook{},\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"should return empty hooks and no error when hookSpec resources is empty\",\n\t\t\thookSpec: &velerov1api.RestoreHooks{\n\t\t\t\tResources: []velerov1api.RestoreResourceHookSpec{},\n\t\t\t},\n\t\t\texpected: []ResourceRestoreHook{},\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"should return hooks specified in the hookSpec initContainer hooks only\",\n\t\t\thookSpec: &velerov1api.RestoreHooks{\n\t\t\t\tResources: []velerov1api.RestoreResourceHookSpec{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"h1\",\n\t\t\t\t\t\tIncludedNamespaces: []string{\"ns1\", \"ns2\", \"ns3\"},\n\t\t\t\t\t\tExcludedNamespaces: []string{\"ns4\", \"ns5\", \"ns6\"},\n\t\t\t\t\t\tIncludedResources: []string{kuberesource.Pods.Resource},\n\t\t\t\t\t\tPostHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tInit: &velerov1api.InitRestoreHook{\n\t\t\t\t\t\t\t\t\tInitContainers: []runtime.RawExtension{\n\t\t\t\t\t\t\t\t\t\tbuilder.ForContainer(\"restore-init1\", \"busy-box\").\n\t\t\t\t\t\t\t\t\t\t\tCommand([]string{\"foobarbaz\"}).ResultRawExtension(),\n\t\t\t\t\t\t\t\t\t\tbuilder.ForContainer(\"restore-init2\", \"busy-box\").\n\t\t\t\t\t\t\t\t\t\t\tCommand([]string{\"foobarbaz\"}).ResultRawExtension(),\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: []ResourceRestoreHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"h1\",\n\t\t\t\t\tSelector: ResourceHookSelector{\n\t\t\t\t\t\tNamespaces: collections.NewIncludesExcludes().Includes(\"ns1\", \"ns2\", \"ns3\").Excludes(\"ns4\", \"ns5\", \"ns6\"),\n\t\t\t\t\t\tResources: collections.NewIncludesExcludes().Includes(kuberesource.Pods.Resource),\n\t\t\t\t\t},\n\t\t\t\t\tRestoreHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tInit: &velerov1api.InitRestoreHook{\n\t\t\t\t\t\t\t\tInitContainers: []runtime.RawExtension{\n\t\t\t\t\t\t\t\t\tbuilder.ForContainer(\"restore-init1\", \"busy-box\").\n\t\t\t\t\t\t\t\t\t\tCommand([]string{\"foobarbaz\"}).ResultRawExtension(),\n\t\t\t\t\t\t\t\t\tbuilder.ForContainer(\"restore-init2\", \"busy-box\").\n\t\t\t\t\t\t\t\t\t\tCommand([]string{\"foobarbaz\"}).ResultRawExtension(),\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactual, err := GetRestoreHooksFromSpec(tc.hookSpec)\n\n\t\t\tassert.Equal(t, tc.expected, actual)\n\t\t\tassert.Equal(t, tc.expectedError, err)\n\t\t})\n\t}\n}\n\nfunc TestHandleRestoreHooks(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tpodInput corev1api.Pod\n\t\trestoreHooks []ResourceRestoreHook\n\t\tnamespaceMapping map[string]string\n\t\texpectedPod *corev1api.Pod\n\t\texpectedError error\n\t}{\n\t\t{\n\t\t\tname: \"should handle hook from annotation no hooks in spec on pod with no init containers\",\n\t\t\tpodInput: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tpodRestoreHookInitContainerImageAnnotationKey: \"nginx\",\n\t\t\t\t\t\tpodRestoreHookInitContainerNameAnnotationKey: \"restore-init-container\",\n\t\t\t\t\t\tpodRestoreHookInitContainerCommandAnnotationKey: `[\"a\", \"b\", \"c\"]`,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedError: nil,\n\t\t\texpectedPod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tpodRestoreHookInitContainerImageAnnotationKey: \"nginx\",\n\t\t\t\t\t\tpodRestoreHookInitContainerNameAnnotationKey: \"restore-init-container\",\n\t\t\t\t\t\tpodRestoreHookInitContainerCommandAnnotationKey: `[\"a\", \"b\", \"c\"]`,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t*builder.ForContainer(\"restore-init-container\", \"nginx\").\n\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).Result(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should handle hook from annotation no hooks in spec on pod with init containers\",\n\t\t\tpodInput: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tpodRestoreHookInitContainerImageAnnotationKey: \"nginx\",\n\t\t\t\t\t\tpodRestoreHookInitContainerNameAnnotationKey: \"restore-init-container\",\n\t\t\t\t\t\tpodRestoreHookInitContainerCommandAnnotationKey: `[\"a\", \"b\", \"c\"]`,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step1\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step1\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step2\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step2\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step3\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step3\"}).Result(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedError: nil,\n\t\t\texpectedPod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tpodRestoreHookInitContainerImageAnnotationKey: \"nginx\",\n\t\t\t\t\t\tpodRestoreHookInitContainerNameAnnotationKey: \"restore-init-container\",\n\t\t\t\t\t\tpodRestoreHookInitContainerCommandAnnotationKey: `[\"a\", \"b\", \"c\"]`,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t*builder.ForContainer(\"restore-init-container\", \"nginx\").\n\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step1\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step1\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step2\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step2\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step3\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step3\"}).Result(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should handle hook from annotation ignoring hooks in spec\",\n\t\t\tpodInput: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tpodRestoreHookInitContainerImageAnnotationKey: \"nginx\",\n\t\t\t\t\t\tpodRestoreHookInitContainerNameAnnotationKey: \"restore-init-container\",\n\t\t\t\t\t\tpodRestoreHookInitContainerCommandAnnotationKey: `[\"a\", \"b\", \"c\"]`,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step1\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step1\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step2\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step2\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step3\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step3\"}).Result(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedError: nil,\n\t\t\texpectedPod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tpodRestoreHookInitContainerImageAnnotationKey: \"nginx\",\n\t\t\t\t\t\tpodRestoreHookInitContainerNameAnnotationKey: \"restore-init-container\",\n\t\t\t\t\t\tpodRestoreHookInitContainerCommandAnnotationKey: `[\"a\", \"b\", \"c\"]`,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t*builder.ForContainer(\"restore-init-container\", \"nginx\").\n\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step1\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step1\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step2\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step2\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step3\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step3\"}).Result(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trestoreHooks: []ResourceRestoreHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"ignore-hook1\",\n\t\t\t\t\tSelector: ResourceHookSelector{\n\t\t\t\t\t\tNamespaces: collections.NewIncludesExcludes().Includes(\"default\"),\n\t\t\t\t\t\tResources: collections.NewIncludesExcludes().Includes(kuberesource.Pods.Resource),\n\t\t\t\t\t},\n\t\t\t\t\tRestoreHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tInit: &velerov1api.InitRestoreHook{\n\t\t\t\t\t\t\t\tInitContainers: []runtime.RawExtension{\n\t\t\t\t\t\t\t\t\tbuilder.ForContainer(\"should-not exist\", \"does-not-matter\").\n\t\t\t\t\t\t\t\t\t\tCommand([]string{\"\"}).ResultRawExtension(),\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should handle hook from spec on pod with no init containers\",\n\t\t\tpodInput: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedError: nil,\n\t\t\texpectedPod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t*builder.ForContainer(\"restore-init-container-1\", \"nginx\").\n\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"restore-init-container-2\", \"nginx\").\n\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).Result(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trestoreHooks: []ResourceRestoreHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\tSelector: ResourceHookSelector{\n\t\t\t\t\t\tNamespaces: collections.NewIncludesExcludes().Includes(\"default\"),\n\t\t\t\t\t\tResources: collections.NewIncludesExcludes().Includes(kuberesource.Pods.Resource),\n\t\t\t\t\t},\n\t\t\t\t\tRestoreHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tInit: &velerov1api.InitRestoreHook{\n\t\t\t\t\t\t\t\tInitContainers: []runtime.RawExtension{\n\t\t\t\t\t\t\t\t\tbuilder.ForContainer(\"restore-init-container-1\", \"nginx\").\n\t\t\t\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).ResultRawExtension(),\n\t\t\t\t\t\t\t\t\tbuilder.ForContainer(\"restore-init-container-2\", \"nginx\").\n\t\t\t\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).ResultRawExtension(),\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should handle hook from spec when no restore hook annotation and existing init containers\",\n\t\t\tpodInput: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step1\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step1\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step2\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step2\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step3\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step3\"}).Result(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedError: nil,\n\t\t\texpectedPod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t*builder.ForContainer(\"restore-init-container-1\", \"nginx\").\n\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"restore-init-container-2\", \"nginx\").\n\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step1\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step1\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step2\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step2\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step3\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step3\"}).Result(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trestoreHooks: []ResourceRestoreHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\tSelector: ResourceHookSelector{\n\t\t\t\t\t\tNamespaces: collections.NewIncludesExcludes().Includes(\"default\"),\n\t\t\t\t\t\tResources: collections.NewIncludesExcludes().Includes(kuberesource.Pods.Resource),\n\t\t\t\t\t},\n\t\t\t\t\tRestoreHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tInit: &velerov1api.InitRestoreHook{\n\t\t\t\t\t\t\t\tInitContainers: []runtime.RawExtension{\n\t\t\t\t\t\t\t\t\tbuilder.ForContainer(\"restore-init-container-1\", \"nginx\").\n\t\t\t\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).ResultRawExtension(),\n\t\t\t\t\t\t\t\t\tbuilder.ForContainer(\"restore-init-container-2\", \"nginx\").\n\t\t\t\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).ResultRawExtension(),\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should not apply any restore hook init containers when resource hook selector mismatch\",\n\t\t\tpodInput: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedError: nil,\n\t\t\texpectedPod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t},\n\t\t\trestoreHooks: []ResourceRestoreHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\tSelector: ResourceHookSelector{\n\t\t\t\t\t\tNamespaces: collections.NewIncludesExcludes().Excludes(\"default\"),\n\t\t\t\t\t\tResources: collections.NewIncludesExcludes().Includes(kuberesource.Pods.Resource),\n\t\t\t\t\t},\n\t\t\t\t\tRestoreHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tInit: &velerov1api.InitRestoreHook{\n\t\t\t\t\t\t\t\tInitContainers: []runtime.RawExtension{\n\t\t\t\t\t\t\t\t\tbuilder.ForContainer(\"restore-init-container-1\", \"nginx\").\n\t\t\t\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).ResultRawExtension(),\n\t\t\t\t\t\t\t\t\tbuilder.ForContainer(\"restore-init-container-2\", \"nginx\").\n\t\t\t\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).ResultRawExtension(),\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should preserve restore-wait init container when it is the only existing init container\",\n\t\t\tpodInput: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t*builder.ForContainer(\"restore-wait\", \"bus-box\").\n\t\t\t\t\t\t\tCommand([]string{\"pod-volume-restore\"}).Result(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedError: nil,\n\t\t\texpectedPod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t*builder.ForContainer(\"restore-wait\", \"bus-box\").\n\t\t\t\t\t\t\tCommand([]string{\"pod-volume-restore\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"restore-init-container-1\", \"nginx\").\n\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"restore-init-container-2\", \"nginx\").\n\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).Result(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trestoreHooks: []ResourceRestoreHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\tSelector: ResourceHookSelector{\n\t\t\t\t\t\tNamespaces: collections.NewIncludesExcludes().Includes(\"default\"),\n\t\t\t\t\t\tResources: collections.NewIncludesExcludes().Includes(kuberesource.Pods.Resource),\n\t\t\t\t\t},\n\t\t\t\t\tRestoreHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tInit: &velerov1api.InitRestoreHook{\n\t\t\t\t\t\t\t\tInitContainers: []runtime.RawExtension{\n\t\t\t\t\t\t\t\t\tbuilder.ForContainer(\"restore-init-container-1\", \"nginx\").\n\t\t\t\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).ResultRawExtension(),\n\t\t\t\t\t\t\t\t\tbuilder.ForContainer(\"restore-init-container-2\", \"nginx\").\n\t\t\t\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).ResultRawExtension(),\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\n\t\t{\n\t\t\tname: \"should preserve restore-wait init container when it exits with other init containers\",\n\t\t\tpodInput: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t*builder.ForContainer(\"restore-wait\", \"bus-box\").\n\t\t\t\t\t\t\tCommand([]string{\"pod-volume-restore\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step1\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step1\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step2\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step2\"}).Result(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedError: nil,\n\t\t\texpectedPod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t*builder.ForContainer(\"restore-wait\", \"bus-box\").\n\t\t\t\t\t\t\tCommand([]string{\"pod-volume-restore\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"restore-init-container-1\", \"nginx\").\n\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"restore-init-container-2\", \"nginx\").\n\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step1\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step1\"}).Result(),\n\t\t\t\t\t\t*builder.ForContainer(\"init-app-step2\", \"busy-box\").\n\t\t\t\t\t\t\tCommand([]string{\"init-step2\"}).Result(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trestoreHooks: []ResourceRestoreHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\tSelector: ResourceHookSelector{\n\t\t\t\t\t\tNamespaces: collections.NewIncludesExcludes().Includes(\"default\"),\n\t\t\t\t\t\tResources: collections.NewIncludesExcludes().Includes(kuberesource.Pods.Resource),\n\t\t\t\t\t},\n\t\t\t\t\tRestoreHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tInit: &velerov1api.InitRestoreHook{\n\t\t\t\t\t\t\t\tInitContainers: []runtime.RawExtension{\n\t\t\t\t\t\t\t\t\tbuilder.ForContainer(\"restore-init-container-1\", \"nginx\").\n\t\t\t\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).ResultRawExtension(),\n\t\t\t\t\t\t\t\t\tbuilder.ForContainer(\"restore-init-container-2\", \"nginx\").\n\t\t\t\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).ResultRawExtension(),\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should not apply any restore hook init containers when resource hook is nil\",\n\t\t\tpodInput: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedError: nil,\n\t\t\texpectedPod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t},\n\t\t\trestoreHooks: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"should not apply any restore hook init containers when resource hook is empty\",\n\t\t\tpodInput: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedError: nil,\n\t\t\texpectedPod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t},\n\t\t\trestoreHooks: []ResourceRestoreHook{},\n\t\t},\n\t\t{\n\t\t\tname: \"should not apply init container when the namespace mapping is provided and the hook points to the original namespace\",\n\t\t\tpodInput: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{},\n\t\t\t},\n\t\t\texpectedError: nil,\n\t\t\texpectedPod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{},\n\t\t\t},\n\t\t\trestoreHooks: []ResourceRestoreHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\tSelector: ResourceHookSelector{\n\t\t\t\t\t\tNamespaces: collections.NewIncludesExcludes().Includes(\"default\"),\n\t\t\t\t\t\tResources: collections.NewIncludesExcludes().Includes(kuberesource.Pods.Resource),\n\t\t\t\t\t},\n\t\t\t\t\tRestoreHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tInit: &velerov1api.InitRestoreHook{\n\t\t\t\t\t\t\t\tInitContainers: []runtime.RawExtension{\n\t\t\t\t\t\t\t\t\tbuilder.ForContainer(\"restore-init-container-1\", \"nginx\").\n\t\t\t\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).ResultRawExtension(),\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tnamespaceMapping: map[string]string{\"default\": \"new\"},\n\t\t},\n\t\t{\n\t\t\tname: \"should apply init container when the namespace mapping is provided and the hook points to the new namespace\",\n\t\t\tpodInput: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{},\n\t\t\t},\n\t\t\texpectedError: nil,\n\t\t\texpectedPod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t*builder.ForContainer(\"restore-init-container-1\", \"nginx\").\n\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).Result(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trestoreHooks: []ResourceRestoreHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\tSelector: ResourceHookSelector{\n\t\t\t\t\t\tNamespaces: collections.NewIncludesExcludes().Includes(\"new\"),\n\t\t\t\t\t\tResources: collections.NewIncludesExcludes().Includes(kuberesource.Pods.Resource),\n\t\t\t\t\t},\n\t\t\t\t\tRestoreHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tInit: &velerov1api.InitRestoreHook{\n\t\t\t\t\t\t\t\tInitContainers: []runtime.RawExtension{\n\t\t\t\t\t\t\t\t\tbuilder.ForContainer(\"restore-init-container-1\", \"nginx\").\n\t\t\t\t\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).ResultRawExtension(),\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tnamespaceMapping: map[string]string{\"default\": \"new\"},\n\t\t},\n\t\t{\n\t\t\tname: \"Invalid InitContainer in Restore hook should return nil as pod, and error.\",\n\t\t\tpodInput: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"app1\",\n\t\t\t\t\tNamespace: \"new\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{},\n\t\t\t},\n\t\t\texpectedError: fmt.Errorf(\"invalid InitContainer in restore hook, it doesn't have Command, Name or Image field\"),\n\t\t\texpectedPod: nil,\n\t\t\trestoreHooks: []ResourceRestoreHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\tSelector: ResourceHookSelector{\n\t\t\t\t\t\tNamespaces: collections.NewIncludesExcludes().Includes(\"new\"),\n\t\t\t\t\t\tResources: collections.NewIncludesExcludes().Includes(kuberesource.Pods.Resource),\n\t\t\t\t\t},\n\t\t\t\t\tRestoreHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tInit: &velerov1api.InitRestoreHook{\n\t\t\t\t\t\t\t\tInitContainers: []runtime.RawExtension{\n\t\t\t\t\t\t\t\t\tbuilder.ForContainer(\"restore-init-container-1\", \"nginx\").\n\t\t\t\t\t\t\t\t\t\tResultRawExtension(),\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\thandler := InitContainerRestoreHookHandler{}\n\t\t\tpodMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&tc.podInput)\n\t\t\trequire.NoError(t, err)\n\t\t\tactual, err := handler.HandleRestoreHooks(velerotest.NewLogger(), kuberesource.Pods, &unstructured.Unstructured{Object: podMap}, tc.restoreHooks, tc.namespaceMapping)\n\t\t\tassert.Equal(t, tc.expectedError, err)\n\t\t\tif actual != nil {\n\t\t\t\tactualPod := new(corev1api.Pod)\n\t\t\t\terr = runtime.DefaultUnstructuredConverter.FromUnstructured(actual.UnstructuredContent(), actualPod)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, tc.expectedPod, actualPod)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestValidateContainer(t *testing.T) {\n\tvalid := `{\"name\": \"test\", \"image\": \"busybox\", \"command\": [\"pwd\"]}`\n\tnoName := `{\"image\": \"busybox\", \"command\": [\"pwd\"]}`\n\tnoImage := `{\"name\": \"test\", \"command\": [\"pwd\"]}`\n\tnoCommand := `{\"name\": \"test\", \"image\": \"busybox\"}`\n\texpectedError := fmt.Errorf(\"invalid InitContainer in restore hook, it doesn't have Command, Name or Image field\")\n\n\t// valid string should return nil as result.\n\trequire.NoError(t, ValidateContainer([]byte(valid)))\n\n\t// noName string should return expected error as result.\n\tassert.Equal(t, expectedError, ValidateContainer([]byte(noName)))\n\n\t// noImage string should return expected error as result.\n\tassert.Equal(t, expectedError, ValidateContainer([]byte(noImage)))\n\n\t// noCommand string should return expected error as result.\n\tassert.Equal(t, expectedError, ValidateContainer([]byte(noCommand)))\n}\n\nfunc TestBackupHookTracker(t *testing.T) {\n\ttype podWithHook struct {\n\t\titem runtime.Unstructured\n\t\thooks []ResourceHook\n\t\thookErrorsByContainer map[string]error\n\t\texpectedPodHook *velerov1api.ExecHook\n\t\texpectedPodHookError error\n\t\texpectedError error\n\t}\n\ttest1 := []struct {\n\t\tname string\n\t\tphase HookPhase\n\t\tgroupResource string\n\t\tpods []podWithHook\n\t\thookTracker *HookTracker\n\t\texpectedHookAttempted int\n\t\texpectedHookFailed int\n\t}{\n\t\t{\n\t\t\tname: \"a pod with spec hooks, no error\",\n\t\t\tphase: PhasePre,\n\t\t\tgroupResource: \"pods\",\n\t\t\thookTracker: NewHookTracker(),\n\t\t\texpectedHookAttempted: 2,\n\t\t\texpectedHookFailed: 0,\n\t\t\tpods: []podWithHook{\n\t\t\t\t{\n\t\t\t\t\titem: velerotest.UnstructuredOrDie(`\n\t\t\t\t\t{\n\t\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\t\"kind\": \"Pod\",\n\t\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\t\"namespace\": \"ns\",\n\t\t\t\t\t\t\t\"name\": \"name\"\n\t\t\t\t\t\t}\n\t\t\t\t\t}`),\n\t\t\t\t\thooks: []ResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\t\t\tPre: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\t\t\tContainer: \"1a\",\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"pre-1a\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\t\t\tContainer: \"1b\",\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"pre-1b\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"a pod with spec hooks and same container under different hook name, no error\",\n\t\t\tphase: PhasePre,\n\t\t\tgroupResource: \"pods\",\n\t\t\thookTracker: NewHookTracker(),\n\t\t\texpectedHookAttempted: 4,\n\t\t\texpectedHookFailed: 0,\n\t\t\tpods: []podWithHook{\n\t\t\t\t{\n\t\t\t\t\titem: velerotest.UnstructuredOrDie(`\n\t\t\t\t\t{\n\t\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\t\"kind\": \"Pod\",\n\t\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\t\"namespace\": \"ns\",\n\t\t\t\t\t\t\t\"name\": \"name\"\n\t\t\t\t\t\t}\n\t\t\t\t\t}`),\n\t\t\t\t\thooks: []ResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\t\t\tPre: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\t\t\tContainer: \"1a\",\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"pre-1a\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\t\t\tContainer: \"1b\",\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"pre-1b\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"hook2\",\n\t\t\t\t\t\t\tPre: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\t\t\tContainer: \"1a\",\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"2a\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\t\t\tContainer: \"2b\",\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"2b\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"a pod with spec hooks, on error=fail\",\n\t\t\tphase: PhasePre,\n\t\t\tgroupResource: \"pods\",\n\t\t\thookTracker: NewHookTracker(),\n\t\t\texpectedHookAttempted: 4,\n\t\t\texpectedHookFailed: 2,\n\t\t\tpods: []podWithHook{\n\t\t\t\t{\n\t\t\t\t\titem: velerotest.UnstructuredOrDie(`\n\t\t\t\t\t{\n\t\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\t\"kind\": \"Pod\",\n\t\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\t\"namespace\": \"ns\",\n\t\t\t\t\t\t\t\"name\": \"name\"\n\t\t\t\t\t\t}\n\t\t\t\t\t}`),\n\t\t\t\t\thooks: []ResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\t\t\tPre: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\t\t\tContainer: \"1a\",\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"1a\"},\n\t\t\t\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\t\t\tContainer: \"1b\",\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"1b\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"hook2\",\n\t\t\t\t\t\t\tPre: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\t\t\tContainer: \"2\",\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"2\"},\n\t\t\t\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeFail,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"hook3\",\n\t\t\t\t\t\t\tPre: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\t\t\tContainer: \"3\",\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"3\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\thookErrorsByContainer: map[string]error{\n\t\t\t\t\t\t\"1a\": errors.New(\"1a error, but continue\"),\n\t\t\t\t\t\t\"2\": errors.New(\"2 error, fail\"),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"a pod with annotation and spec hooks\",\n\t\t\tphase: PhasePre,\n\t\t\tgroupResource: \"pods\",\n\t\t\thookTracker: NewHookTracker(),\n\t\t\texpectedHookAttempted: 1,\n\t\t\texpectedHookFailed: 0,\n\t\t\tpods: []podWithHook{\n\t\t\t\t{\n\t\t\t\t\titem: velerotest.UnstructuredOrDie(`\n\t\t\t\t\t{\n\t\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\t\"kind\": \"Pod\",\n\t\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\t\"namespace\": \"ns\",\n\t\t\t\t\t\t\t\"name\": \"name\",\n\t\t\t\t\t\t\t\"annotations\": {\n\t\t\t\t\t\t\t\t\"hook.backup.velero.io/container\": \"c\",\n\t\t\t\t\t\t\t\t\"hook.backup.velero.io/command\": \"/bin/ls\"\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}\n\t\t\t\t\t}`),\n\t\t\t\t\texpectedPodHook: &velerov1api.ExecHook{\n\t\t\t\t\t\tContainer: \"c\",\n\t\t\t\t\t\tCommand: []string{\"/bin/ls\"},\n\t\t\t\t\t},\n\t\t\t\t\thooks: []ResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\t\t\tPre: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\t\t\tContainer: \"1a\",\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"1a\"},\n\t\t\t\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\t\t\tContainer: \"1b\",\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"1b\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"a pod with annotation, on error=fail\",\n\t\t\tphase: PhasePre,\n\t\t\tgroupResource: \"pods\",\n\t\t\thookTracker: NewHookTracker(),\n\t\t\texpectedHookAttempted: 1,\n\t\t\texpectedHookFailed: 1,\n\t\t\tpods: []podWithHook{\n\t\t\t\t{\n\t\t\t\t\titem: velerotest.UnstructuredOrDie(`\n\t\t\t\t\t{\n\t\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\t\"kind\": \"Pod\",\n\t\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\t\"namespace\": \"ns\",\n\t\t\t\t\t\t\t\"name\": \"name\",\n\t\t\t\t\t\t\t\"annotations\": {\n\t\t\t\t\t\t\t\t\"hook.backup.velero.io/container\": \"c\",\n\t\t\t\t\t\t\t\t\"hook.backup.velero.io/command\": \"/bin/ls\",\n\t\t\t\t\t\t\t\t\"hook.backup.velero.io/on-error\": \"Fail\"\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}\n\t\t\t\t\t}`),\n\t\t\t\t\texpectedPodHook: &velerov1api.ExecHook{\n\t\t\t\t\t\tContainer: \"c\",\n\t\t\t\t\t\tCommand: []string{\"/bin/ls\"},\n\t\t\t\t\t\tOnError: velerov1api.HookErrorModeFail,\n\t\t\t\t\t},\n\t\t\t\t\texpectedPodHookError: errors.New(\"pod hook error\"),\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"two pods, one with annotation, the other with spec\",\n\t\t\tphase: PhasePre,\n\t\t\tgroupResource: \"pods\",\n\t\t\thookTracker: NewHookTracker(),\n\t\t\texpectedHookAttempted: 3,\n\t\t\texpectedHookFailed: 1,\n\t\t\tpods: []podWithHook{\n\t\t\t\t{\n\t\t\t\t\titem: velerotest.UnstructuredOrDie(`\n\t\t\t\t\t{\n\t\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\t\"kind\": \"Pod\",\n\t\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\t\"namespace\": \"ns\",\n\t\t\t\t\t\t\t\"name\": \"name\",\n\t\t\t\t\t\t\t\"annotations\": {\n\t\t\t\t\t\t\t\t\"hook.backup.velero.io/container\": \"c\",\n\t\t\t\t\t\t\t\t\"hook.backup.velero.io/command\": \"/bin/ls\",\n\t\t\t\t\t\t\t\t\"hook.backup.velero.io/on-error\": \"Fail\"\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}\n\t\t\t\t\t}`),\n\t\t\t\t\texpectedPodHook: &velerov1api.ExecHook{\n\t\t\t\t\t\tContainer: \"c\",\n\t\t\t\t\t\tCommand: []string{\"/bin/ls\"},\n\t\t\t\t\t\tOnError: velerov1api.HookErrorModeFail,\n\t\t\t\t\t},\n\t\t\t\t\texpectedPodHookError: errors.New(\"pod hook error\"),\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\titem: velerotest.UnstructuredOrDie(`\n\t\t\t\t\t{\n\t\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\t\"kind\": \"Pod\",\n\t\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\t\"namespace\": \"ns\",\n\t\t\t\t\t\t\t\"name\": \"name\"\n\t\t\t\t\t\t}\n\t\t\t\t\t}`),\n\t\t\t\t\thooks: []ResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\t\t\tPre: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\t\t\tContainer: \"1a\",\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"pre-1a\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\t\t\tContainer: \"1b\",\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"pre-1b\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range test1 {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tpodCommandExecutor := &velerotest.MockPodCommandExecutor{}\n\t\t\tdefer podCommandExecutor.AssertExpectations(t)\n\n\t\t\th := &DefaultItemHookHandler{\n\t\t\t\tPodCommandExecutor: podCommandExecutor,\n\t\t\t}\n\n\t\t\tgroupResource := schema.ParseGroupResource(test.groupResource)\n\t\t\thookTracker := test.hookTracker\n\n\t\t\tfor _, pod := range test.pods {\n\t\t\t\tif pod.expectedPodHook != nil {\n\t\t\t\t\tpodCommandExecutor.On(\"ExecutePodCommand\", mock.Anything, pod.item.UnstructuredContent(), \"ns\", \"name\", \"\", pod.expectedPodHook).Return(pod.expectedPodHookError)\n\t\t\t\t} else {\n\t\t\t\thookLoop:\n\t\t\t\t\tfor _, resourceHook := range pod.hooks {\n\t\t\t\t\t\tfor _, hook := range resourceHook.Pre {\n\t\t\t\t\t\t\thookError := pod.hookErrorsByContainer[hook.Exec.Container]\n\t\t\t\t\t\t\tpodCommandExecutor.On(\"ExecutePodCommand\", mock.Anything, pod.item.UnstructuredContent(), \"ns\", \"name\", resourceHook.Name, hook.Exec).Return(hookError)\n\t\t\t\t\t\t\tif hookError != nil && hook.Exec.OnError == velerov1api.HookErrorModeFail {\n\t\t\t\t\t\t\t\tbreak hookLoop\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}\n\t\t\t\t\t\tfor _, hook := range resourceHook.Post {\n\t\t\t\t\t\t\thookError := pod.hookErrorsByContainer[hook.Exec.Container]\n\t\t\t\t\t\t\tpodCommandExecutor.On(\"ExecutePodCommand\", mock.Anything, pod.item.UnstructuredContent(), \"ns\", \"name\", resourceHook.Name, hook.Exec).Return(hookError)\n\t\t\t\t\t\t\tif hookError != nil && hook.Exec.OnError == velerov1api.HookErrorModeFail {\n\t\t\t\t\t\t\t\tbreak hookLoop\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\th.HandleHooks(velerotest.NewLogger(), groupResource, pod.item, pod.hooks, test.phase, hookTracker)\n\t\t\t}\n\t\t\tactualAtemptted, actualFailed := hookTracker.Stat()\n\t\t\tassert.Equal(t, test.expectedHookAttempted, actualAtemptted)\n\t\t\tassert.Equal(t, test.expectedHookFailed, actualFailed)\n\t\t})\n\t}\n}\n\nfunc TestRestoreHookTrackerAdd(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tresourceRestoreHooks []ResourceRestoreHook\n\t\tpod *corev1api.Pod\n\t\thookTracker *MultiHookTracker\n\t\texpectedCnt int\n\t}{\n\t\t{\n\t\t\tname: \"neither spec hooks nor annotations hooks are set\",\n\t\t\tresourceRestoreHooks: nil,\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").Result(),\n\t\t\thookTracker: NewMultiHookTracker(),\n\t\t\texpectedCnt: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"a hook specified in pod annotation\",\n\t\t\tresourceRestoreHooks: nil,\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tObjectMeta(builder.WithAnnotations(\n\t\t\t\t\tpodRestoreHookCommandAnnotationKey, \"/usr/bin/foo\",\n\t\t\t\t\tpodRestoreHookContainerAnnotationKey, \"container1\",\n\t\t\t\t\tpodRestoreHookOnErrorAnnotationKey, string(velerov1api.HookErrorModeContinue),\n\t\t\t\t\tpodRestoreHookTimeoutAnnotationKey, \"1s\",\n\t\t\t\t\tpodRestoreHookWaitTimeoutAnnotationKey, \"1m\",\n\t\t\t\t\tpodRestoreHookWaitForReadyAnnotationKey, \"true\",\n\t\t\t\t)).\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\thookTracker: NewMultiHookTracker(),\n\t\t\texpectedCnt: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"two hooks specified in restore spec\",\n\t\t\tresourceRestoreHooks: []ResourceRestoreHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\tSelector: ResourceHookSelector{},\n\t\t\t\t\tRestoreHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\t\tContainer: \"container2\",\n\t\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}, &corev1api.Container{\n\t\t\t\t\tName: \"container2\",\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\thookTracker: NewMultiHookTracker(),\n\t\t\texpectedCnt: 2,\n\t\t},\n\t\t{\n\t\t\tname: \"both spec hooks and annotations hooks are set\",\n\t\t\tresourceRestoreHooks: []ResourceRestoreHook{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook1\",\n\t\t\t\t\tSelector: ResourceHookSelector{},\n\t\t\t\t\tRestoreHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo2\"},\n\t\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tObjectMeta(builder.WithAnnotations(\n\t\t\t\t\tpodRestoreHookCommandAnnotationKey, \"/usr/bin/foo\",\n\t\t\t\t\tpodRestoreHookContainerAnnotationKey, \"container1\",\n\t\t\t\t\tpodRestoreHookOnErrorAnnotationKey, string(velerov1api.HookErrorModeContinue),\n\t\t\t\t\tpodRestoreHookTimeoutAnnotationKey, \"1s\",\n\t\t\t\t\tpodRestoreHookWaitTimeoutAnnotationKey, \"1m\",\n\t\t\t\t\tpodRestoreHookWaitForReadyAnnotationKey, \"true\",\n\t\t\t\t)).\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\thookTracker: NewMultiHookTracker(),\n\t\t\texpectedCnt: 1,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t_, _ = GroupRestoreExecHooks(\"restore1\", tc.resourceRestoreHooks, tc.pod, velerotest.NewLogger(), tc.hookTracker)\n\t\t\tif _, ok := tc.hookTracker.trackers[\"restore1\"]; !ok {\n\t\t\t\treturn\n\t\t\t}\n\t\t\ttracker := tc.hookTracker.trackers[\"restore1\"].tracker\n\t\t\tassert.Len(t, tracker, tc.expectedCnt)\n\t\t})\n\t}\n}\n" }, { "path": "internal/hook/wait_exec_hook_handler.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage hook\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/fields\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/tools/cache\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/podexec\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\ntype WaitExecHookHandler interface {\n\tHandleHooks(\n\t\tctx context.Context,\n\t\tlog logrus.FieldLogger,\n\t\tpod *corev1api.Pod,\n\t\tbyContainer map[string][]PodExecRestoreHook,\n\t\tmultiHookTracker *MultiHookTracker,\n\t\trestoreName string,\n\t) []error\n}\n\ntype ListWatchFactory interface {\n\tNewListWatch(namespace string, selector fields.Selector) cache.ListerWatcher\n}\n\ntype DefaultListWatchFactory struct {\n\tPodsGetter cache.Getter\n}\n\ntype HookErrInfo struct {\n\tNamespace string\n\tErr error\n}\n\nfunc (d *DefaultListWatchFactory) NewListWatch(namespace string, selector fields.Selector) cache.ListerWatcher {\n\treturn cache.NewListWatchFromClient(d.PodsGetter, \"pods\", namespace, selector)\n}\n\nvar _ ListWatchFactory = &DefaultListWatchFactory{}\n\ntype DefaultWaitExecHookHandler struct {\n\tListWatchFactory ListWatchFactory\n\tPodCommandExecutor podexec.PodCommandExecutor\n}\n\nvar _ WaitExecHookHandler = &DefaultWaitExecHookHandler{}\n\nfunc (e *DefaultWaitExecHookHandler) HandleHooks(\n\tctx context.Context,\n\tlog logrus.FieldLogger,\n\tpod *corev1api.Pod,\n\tbyContainer map[string][]PodExecRestoreHook,\n\tmultiHookTracker *MultiHookTracker,\n\trestoreName string,\n) []error {\n\tif pod == nil {\n\t\treturn nil\n\t}\n\n\t// If hooks are defined for a container that does not exist in the pod log a warning and discard\n\t// those hooks to avoid waiting for a container that will never become ready. After that if\n\t// there are no hooks left to be executed return immediately.\n\tfor containerName := range byContainer {\n\t\tif !podHasContainer(pod, containerName) {\n\t\t\tlog.Warningf(\"Pod %s does not have container %s: discarding post-restore exec hooks\", kube.NamespaceAndName(pod), containerName)\n\t\t\tdelete(byContainer, containerName)\n\t\t}\n\t}\n\tif len(byContainer) == 0 {\n\t\treturn nil\n\t}\n\n\t// Every hook in every container can have its own wait timeout. Rather than setting up separate\n\t// contexts for each, find the largest wait timeout for any hook that should be executed in\n\t// the pod and watch the pod for up to that long. Before executing any hook in a container,\n\t// check if that hook has a timeout and skip execution if expired.\n\tctx, cancel := context.WithCancel(ctx)\n\tmaxWait := maxHookWait(byContainer)\n\t// If no hook has a wait timeout then this function will continue waiting for containers to\n\t// become ready until the shared hook context is canceled.\n\tif maxWait > 0 {\n\t\tctx, cancel = context.WithTimeout(ctx, maxWait)\n\t}\n\twaitStart := time.Now()\n\n\tvar errors []error\n\n\t// The first time this handler is called after a container starts running it will execute all\n\t// pending hooks for that container. Subsequent invocations of this handler will never execute\n\t// hooks in that container. It uses the byContainer map to keep track of which containers have\n\t// not yet been observed to be running. It relies on the Informer not to be called concurrently.\n\t// When a container is observed running and its hooks are executed, the container is deleted\n\t// from the byContainer map. When the map is empty the watch is ended.\n\thandler := func(newObj any) {\n\t\tnewPod, ok := newObj.(*corev1api.Pod)\n\t\tif !ok {\n\t\t\treturn\n\t\t}\n\n\t\tpodLog := log.WithFields(\n\t\t\tlogrus.Fields{\n\t\t\t\t\"pod\": kube.NamespaceAndName(newPod),\n\t\t\t},\n\t\t)\n\n\t\tif newPod.Status.Phase == corev1api.PodSucceeded || newPod.Status.Phase == corev1api.PodFailed {\n\t\t\terr := fmt.Errorf(\"pod entered phase %s before some post-restore exec hooks ran\", newPod.Status.Phase)\n\t\t\tpodLog.Warning(err)\n\t\t\tcancel()\n\t\t\treturn\n\t\t}\n\n\t\tfor containerName, hooks := range byContainer {\n\t\t\tif !isContainerUp(newPod, containerName, hooks) {\n\t\t\t\tpodLog.Infof(\"Container %s is not up: post-restore hooks will not yet be executed\", containerName)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tpodMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(newPod)\n\t\t\tif err != nil {\n\t\t\t\tpodLog.WithError(err).Error(\"error unstructuring pod\")\n\t\t\t\tcancel()\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\t// Sequentially run all hooks for the ready container. The container's hooks are not\n\t\t\t// removed from the byContainer map until all have completed so that if one fails\n\t\t\t// remaining unexecuted hooks can be handled by the outer function.\n\t\t\tfor i, hook := range hooks {\n\t\t\t\t// This indicates to the outer function not to handle this hook as unexecuted in\n\t\t\t\t// case of terminating before deleting this container's slice of hooks from the\n\t\t\t\t// byContainer map.\n\t\t\t\tbyContainer[containerName][i].executed = true\n\n\t\t\t\thookLog := podLog.WithFields(\n\t\t\t\t\tlogrus.Fields{\n\t\t\t\t\t\t\"hookSource\": hook.HookSource,\n\t\t\t\t\t\t\"hookType\": \"exec\",\n\t\t\t\t\t\t\"hookPhase\": \"post\",\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t\t// Check the individual hook's wait timeout is not expired\n\t\t\t\tif hook.Hook.WaitTimeout.Duration != 0 && time.Since(waitStart) > hook.Hook.WaitTimeout.Duration {\n\t\t\t\t\terr := fmt.Errorf(\"hook %s in container %s expired before executing\", hook.HookName, hook.Hook.Container)\n\t\t\t\t\thookLog.Error(err)\n\t\t\t\t\terrors = append(errors, err)\n\n\t\t\t\t\terrTracker := multiHookTracker.Record(restoreName, newPod.Namespace, newPod.Name, hook.Hook.Container, hook.HookSource, hook.HookName, HookPhase(\"\"), hook.hookIndex, true, err)\n\t\t\t\t\tif errTracker != nil {\n\t\t\t\t\t\thookLog.WithError(errTracker).Warn(\"Error recording the hook in hook tracker\")\n\t\t\t\t\t}\n\n\t\t\t\t\tif hook.Hook.OnError == velerov1api.HookErrorModeFail {\n\t\t\t\t\t\tcancel()\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\teh := &velerov1api.ExecHook{\n\t\t\t\t\tContainer: hook.Hook.Container,\n\t\t\t\t\tCommand: hook.Hook.Command,\n\t\t\t\t\tOnError: hook.Hook.OnError,\n\t\t\t\t\tTimeout: hook.Hook.ExecTimeout,\n\t\t\t\t}\n\n\t\t\t\thookFailed := false\n\t\t\t\tvar hookErr error\n\t\t\t\tif hookErr = e.PodCommandExecutor.ExecutePodCommand(hookLog, podMap, pod.Namespace, pod.Name, hook.HookName, eh); hookErr != nil {\n\t\t\t\t\thookLog.WithError(hookErr).Error(\"Error executing hook\")\n\t\t\t\t\thookErr = fmt.Errorf(\"hook %s in container %s failed to execute, err: %v\", hook.HookName, hook.Hook.Container, hookErr)\n\t\t\t\t\terrors = append(errors, hookErr)\n\t\t\t\t\thookFailed = true\n\t\t\t\t}\n\n\t\t\t\terrTracker := multiHookTracker.Record(restoreName, newPod.Namespace, newPod.Name, hook.Hook.Container, hook.HookSource, hook.HookName, HookPhase(\"\"), hook.hookIndex, hookFailed, hookErr)\n\t\t\t\tif errTracker != nil {\n\t\t\t\t\thookLog.WithError(errTracker).Warn(\"Error recording the hook in hook tracker\")\n\t\t\t\t}\n\n\t\t\t\tif hookErr != nil && hook.Hook.OnError == velerov1api.HookErrorModeFail {\n\t\t\t\t\tcancel()\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}\n\t\t\tdelete(byContainer, containerName)\n\t\t}\n\t\tif len(byContainer) == 0 {\n\t\t\tcancel()\n\t\t}\n\t}\n\n\tselector := fields.OneTermEqualSelector(\"metadata.name\", pod.Name)\n\tlw := e.ListWatchFactory.NewListWatch(pod.Namespace, selector)\n\t_, podWatcher := cache.NewInformerWithOptions(cache.InformerOptions{\n\t\tListerWatcher: lw,\n\t\tObjectType: pod,\n\t\tResyncPeriod: 0,\n\t\tHandler: cache.ResourceEventHandlerFuncs{\n\t\t\tAddFunc: handler,\n\t\t\tUpdateFunc: func(_, newObj any) {\n\t\t\t\thandler(newObj)\n\t\t\t},\n\t\t\tDeleteFunc: func(obj any) {\n\t\t\t\terr := fmt.Errorf(\"pod %s deleted before all hooks were executed\", kube.NamespaceAndName(pod))\n\t\t\t\tlog.Error(err)\n\t\t\t\tcancel()\n\t\t\t},\n\t\t},\n\t},\n\t)\n\n\tpodWatcher.Run(ctx.Done())\n\n\t// There are some cases where this function could return with unexecuted hooks: the pod may\n\t// be deleted, a hook could fail, or it may timeout waiting for\n\t// containers to become ready.\n\t// Each unexecuted hook is logged as an error and this error will be returned from this function.\n\tfor _, hooks := range byContainer {\n\t\tfor _, hook := range hooks {\n\t\t\tif hook.executed {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\terr := fmt.Errorf(\"hook %s in container %s in pod %s not executed: %v\", hook.HookName, hook.Hook.Container, kube.NamespaceAndName(pod), ctx.Err())\n\t\t\thookLog := log.WithFields(\n\t\t\t\tlogrus.Fields{\n\t\t\t\t\t\"hookSource\": hook.HookSource,\n\t\t\t\t\t\"hookType\": \"exec\",\n\t\t\t\t\t\"hookPhase\": \"post\",\n\t\t\t\t},\n\t\t\t)\n\n\t\t\terrTracker := multiHookTracker.Record(restoreName, pod.Namespace, pod.Name, hook.Hook.Container, hook.HookSource, hook.HookName, HookPhase(\"\"), hook.hookIndex, true, err)\n\t\t\tif errTracker != nil {\n\t\t\t\thookLog.WithError(errTracker).Warn(\"Error recording the hook in hook tracker\")\n\t\t\t}\n\n\t\t\thookLog.Error(err)\n\t\t\terrors = append(errors, err)\n\t\t}\n\t}\n\n\treturn errors\n}\n\nfunc podHasContainer(pod *corev1api.Pod, containerName string) bool {\n\tif pod == nil {\n\t\treturn false\n\t}\n\tfor _, c := range pod.Spec.Containers {\n\t\tif c.Name == containerName {\n\t\t\treturn true\n\t\t}\n\t}\n\n\treturn false\n}\n\nfunc isContainerUp(pod *corev1api.Pod, containerName string, hooks []PodExecRestoreHook) bool {\n\tif pod == nil {\n\t\treturn false\n\t}\n\tvar waitForReady bool\n\tfor _, hook := range hooks {\n\t\tif boolptr.IsSetToTrue(hook.Hook.WaitForReady) {\n\t\t\twaitForReady = true\n\t\t\tbreak\n\t\t}\n\t}\n\tfor _, cs := range pod.Status.ContainerStatuses {\n\t\tif cs.Name != containerName {\n\t\t\tcontinue\n\t\t}\n\t\tif waitForReady {\n\t\t\treturn cs.Ready\n\t\t}\n\t\treturn cs.State.Running != nil\n\t}\n\n\treturn false\n}\n\n// maxHookWait returns 0 to mean wait indefinitely. Any hook without a wait timeout will cause this\n// function to return 0.\nfunc maxHookWait(byContainer map[string][]PodExecRestoreHook) time.Duration {\n\tvar maxWait time.Duration\n\tfor _, hooks := range byContainer {\n\t\tfor _, hook := range hooks {\n\t\t\tif hook.Hook.WaitTimeout.Duration <= 0 {\n\t\t\t\treturn 0\n\t\t\t}\n\t\t\tif hook.Hook.WaitTimeout.Duration > maxWait {\n\t\t\t\tmaxWait = hook.Hook.WaitTimeout.Duration\n\t\t\t}\n\t\t}\n\t}\n\treturn maxWait\n}\n" }, { "path": "internal/hook/wait_exec_hook_handler_test.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage hook\n\nimport (\n\t\"context\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/fields\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/tools/cache\"\n\tfcache \"k8s.io/client-go/tools/cache/testing\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n)\n\ntype fakeListWatchFactory struct {\n\tlw cache.ListerWatcher\n}\n\nfunc (f *fakeListWatchFactory) NewListWatch(ns string, selector fields.Selector) cache.ListerWatcher {\n\treturn f.lw\n}\n\nvar _ ListWatchFactory = &fakeListWatchFactory{}\n\nfunc TestWaitExecHandleHooks(t *testing.T) {\n\ttype change struct {\n\t\t// delta to wait since last change applied or pod added\n\t\twait time.Duration\n\t\tupdated *corev1api.Pod\n\t}\n\ttype expectedExecution struct {\n\t\thook *velerov1api.ExecHook\n\t\tname string\n\t\terror error\n\t\tpod *corev1api.Pod\n\t}\n\ttests := []struct {\n\t\tname string\n\t\t// Used as argument to HandleHooks and first state added to ListerWatcher\n\t\tinitialPod *corev1api.Pod\n\t\tgroupResource string\n\t\tbyContainer map[string][]PodExecRestoreHook\n\t\texpectedExecutions []expectedExecution\n\t\texpectedErrors []error\n\t\t// changes represents the states of the pod over time. It can be used to test a container\n\t\t// becoming ready at some point after it is first observed by the controller.\n\t\tchanges []change\n\t\tsharedHooksContextTimeout time.Duration\n\t}{\n\t\t{\n\t\t\tname: \"should return no error when hook from annotation executes successfully\",\n\t\t\tinitialPod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tObjectMeta(builder.WithAnnotations(\n\t\t\t\t\tpodRestoreHookCommandAnnotationKey, \"/usr/bin/foo\",\n\t\t\t\t\tpodRestoreHookContainerAnnotationKey, \"container1\",\n\t\t\t\t\tpodRestoreHookOnErrorAnnotationKey, string(velerov1api.HookErrorModeContinue),\n\t\t\t\t\tpodRestoreHookTimeoutAnnotationKey, \"1s\",\n\t\t\t\t\tpodRestoreHookWaitTimeoutAnnotationKey, \"1m\",\n\t\t\t\t)).\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\tgroupResource: \"pods\",\n\t\t\tbyContainer: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"\",\n\t\t\t\t\t\tHookSource: HookSourceAnnotation,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedExecutions: []expectedExecution{\n\t\t\t\t{\n\t\t\t\t\tname: \"\",\n\t\t\t\t\thook: &velerov1api.ExecHook{\n\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\tTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t},\n\t\t\t\t\terror: nil,\n\t\t\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\t\t\tObjectMeta(builder.WithResourceVersion(\"1\")).\n\t\t\t\t\t\tObjectMeta(builder.WithAnnotations(\n\t\t\t\t\t\t\tpodRestoreHookCommandAnnotationKey, \"/usr/bin/foo\",\n\t\t\t\t\t\t\tpodRestoreHookContainerAnnotationKey, \"container1\",\n\t\t\t\t\t\t\tpodRestoreHookOnErrorAnnotationKey, string(velerov1api.HookErrorModeContinue),\n\t\t\t\t\t\t\tpodRestoreHookTimeoutAnnotationKey, \"1s\",\n\t\t\t\t\t\t\tpodRestoreHookWaitTimeoutAnnotationKey, \"1m\",\n\t\t\t\t\t\t)).\n\t\t\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tResult(),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedErrors: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"should return an error when hook from annotation fails with on error mode fail\",\n\t\t\tinitialPod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tObjectMeta(builder.WithAnnotations(\n\t\t\t\t\tpodRestoreHookCommandAnnotationKey, \"/usr/bin/foo\",\n\t\t\t\t\tpodRestoreHookContainerAnnotationKey, \"container1\",\n\t\t\t\t\tpodRestoreHookOnErrorAnnotationKey, string(velerov1api.HookErrorModeFail),\n\t\t\t\t\tpodRestoreHookTimeoutAnnotationKey, \"1s\",\n\t\t\t\t\tpodRestoreHookWaitTimeoutAnnotationKey, \"1m\",\n\t\t\t\t)).\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\tgroupResource: \"pods\",\n\t\t\tbyContainer: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"\",\n\t\t\t\t\t\tHookSource: HookSourceAnnotation,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeFail,\n\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedExecutions: []expectedExecution{\n\t\t\t\t{\n\t\t\t\t\tname: \"\",\n\t\t\t\t\thook: &velerov1api.ExecHook{\n\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\tOnError: velerov1api.HookErrorModeFail,\n\t\t\t\t\t\tTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t},\n\t\t\t\t\terror: errors.New(\"pod hook error\"),\n\t\t\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\t\t\tObjectMeta(builder.WithResourceVersion(\"1\")).\n\t\t\t\t\t\tObjectMeta(builder.WithAnnotations(\n\t\t\t\t\t\t\tpodRestoreHookCommandAnnotationKey, \"/usr/bin/foo\",\n\t\t\t\t\t\t\tpodRestoreHookContainerAnnotationKey, \"container1\",\n\t\t\t\t\t\t\tpodRestoreHookOnErrorAnnotationKey, string(velerov1api.HookErrorModeFail),\n\t\t\t\t\t\t\tpodRestoreHookTimeoutAnnotationKey, \"1s\",\n\t\t\t\t\t\t\tpodRestoreHookWaitTimeoutAnnotationKey, \"1m\",\n\t\t\t\t\t\t)).\n\t\t\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tResult(),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedErrors: []error{errors.New(\"hook in container container1 failed to execute, err: pod hook error\")},\n\t\t},\n\t\t{\n\t\t\tname: \"should return error when hook from annotation fails with on error mode continue\",\n\t\t\tinitialPod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tObjectMeta(builder.WithAnnotations(\n\t\t\t\t\tpodRestoreHookCommandAnnotationKey, \"/usr/bin/foo\",\n\t\t\t\t\tpodRestoreHookContainerAnnotationKey, \"container1\",\n\t\t\t\t\tpodRestoreHookOnErrorAnnotationKey, string(velerov1api.HookErrorModeContinue),\n\t\t\t\t\tpodRestoreHookTimeoutAnnotationKey, \"1s\",\n\t\t\t\t\tpodRestoreHookWaitTimeoutAnnotationKey, \"1m\",\n\t\t\t\t)).\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\tgroupResource: \"pods\",\n\t\t\tbyContainer: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"\",\n\t\t\t\t\t\tHookSource: HookSourceAnnotation,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedExecutions: []expectedExecution{\n\t\t\t\t{\n\t\t\t\t\tname: \"\",\n\t\t\t\t\thook: &velerov1api.ExecHook{\n\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\tTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t},\n\t\t\t\t\terror: errors.New(\"pod hook error\"),\n\t\t\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\t\t\tObjectMeta(builder.WithResourceVersion(\"1\")).\n\t\t\t\t\t\tObjectMeta(builder.WithAnnotations(\n\t\t\t\t\t\t\tpodRestoreHookCommandAnnotationKey, \"/usr/bin/foo\",\n\t\t\t\t\t\t\tpodRestoreHookContainerAnnotationKey, \"container1\",\n\t\t\t\t\t\t\tpodRestoreHookOnErrorAnnotationKey, string(velerov1api.HookErrorModeContinue),\n\t\t\t\t\t\t\tpodRestoreHookTimeoutAnnotationKey, \"1s\",\n\t\t\t\t\t\t\tpodRestoreHookWaitTimeoutAnnotationKey, \"1m\",\n\t\t\t\t\t\t)).\n\t\t\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tResult(),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedErrors: []error{errors.New(\"hook in container container1 failed to execute, err: pod hook error\")},\n\t\t},\n\t\t{\n\t\t\tname: \"should return no error when hook from annotation executes after 10ms wait for container to start\",\n\t\t\tinitialPod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tObjectMeta(builder.WithAnnotations(\n\t\t\t\t\tpodRestoreHookCommandAnnotationKey, \"/usr/bin/foo\",\n\t\t\t\t\tpodRestoreHookContainerAnnotationKey, \"container1\",\n\t\t\t\t\tpodRestoreHookOnErrorAnnotationKey, string(velerov1api.HookErrorModeContinue),\n\t\t\t\t\tpodRestoreHookTimeoutAnnotationKey, \"1s\",\n\t\t\t\t\tpodRestoreHookWaitTimeoutAnnotationKey, \"1m\",\n\t\t\t\t)).\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tWaiting: &corev1api.ContainerStateWaiting{},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\tgroupResource: \"pods\",\n\t\t\tbyContainer: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"\",\n\t\t\t\t\t\tHookSource: HookSourceAnnotation,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedExecutions: []expectedExecution{\n\t\t\t\t{\n\t\t\t\t\tname: \"\",\n\t\t\t\t\thook: &velerov1api.ExecHook{\n\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\tTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t},\n\t\t\t\t\terror: nil,\n\t\t\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\t\t\tObjectMeta(builder.WithResourceVersion(\"2\")).\n\t\t\t\t\t\tObjectMeta(builder.WithAnnotations(\n\t\t\t\t\t\t\tpodRestoreHookCommandAnnotationKey, \"/usr/bin/foo\",\n\t\t\t\t\t\t\tpodRestoreHookContainerAnnotationKey, \"container1\",\n\t\t\t\t\t\t\tpodRestoreHookOnErrorAnnotationKey, string(velerov1api.HookErrorModeContinue),\n\t\t\t\t\t\t\tpodRestoreHookTimeoutAnnotationKey, \"1s\",\n\t\t\t\t\t\t\tpodRestoreHookWaitTimeoutAnnotationKey, \"1m\",\n\t\t\t\t\t\t)).\n\t\t\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tResult(),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedErrors: nil,\n\t\t\tchanges: []change{\n\t\t\t\t{\n\t\t\t\t\twait: 10 * time.Millisecond,\n\t\t\t\t\tupdated: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\t\t\tObjectMeta(builder.WithAnnotations(\n\t\t\t\t\t\t\tpodRestoreHookCommandAnnotationKey, \"/usr/bin/foo\",\n\t\t\t\t\t\t\tpodRestoreHookContainerAnnotationKey, \"container1\",\n\t\t\t\t\t\t\tpodRestoreHookOnErrorAnnotationKey, string(velerov1api.HookErrorModeContinue),\n\t\t\t\t\t\t\tpodRestoreHookTimeoutAnnotationKey, \"1s\",\n\t\t\t\t\t\t\tpodRestoreHookWaitTimeoutAnnotationKey, \"1m\",\n\t\t\t\t\t\t)).\n\t\t\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tResult(),\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should return no error when hook from spec executes successfully\",\n\t\t\tgroupResource: \"pods\",\n\t\t\tinitialPod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\texpectedErrors: nil,\n\t\t\tbyContainer: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"my-hook-1\",\n\t\t\t\t\t\tHookSource: HookSourceSpec,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedExecutions: []expectedExecution{\n\t\t\t\t{\n\t\t\t\t\tname: \"my-hook-1\",\n\t\t\t\t\thook: &velerov1api.ExecHook{\n\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t},\n\t\t\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\t\t\tObjectMeta(builder.WithResourceVersion(\"1\")).\n\t\t\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tResult(),\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should return error when spec hook with wait timeout expires with OnError mode Continue\",\n\t\t\tgroupResource: \"pods\",\n\t\t\tinitialPod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tWaiting: &corev1api.ContainerStateWaiting{},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\texpectedErrors: []error{errors.New(\"hook my-hook-1 in container container1 in pod default/my-pod not executed: context deadline exceeded\")},\n\t\t\tbyContainer: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"my-hook-1\",\n\t\t\t\t\t\tHookSource: HookSourceSpec,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Millisecond},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedExecutions: []expectedExecution{},\n\t\t},\n\t\t{\n\t\t\tname: \"should return an error when spec hook with wait timeout expires with OnError mode Fail\",\n\t\t\tgroupResource: \"pods\",\n\t\t\tinitialPod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tWaiting: &corev1api.ContainerStateWaiting{},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\texpectedErrors: []error{errors.New(\"hook my-hook-1 in container container1 in pod default/my-pod not executed: context deadline exceeded\")},\n\t\t\tbyContainer: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"my-hook-1\",\n\t\t\t\t\t\tHookSource: HookSourceSpec,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeFail,\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Millisecond},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedExecutions: []expectedExecution{},\n\t\t},\n\t\t{\n\t\t\tname: \"should return an error when shared hooks context is canceled before spec hook with OnError mode Fail executes\",\n\t\t\tgroupResource: \"pods\",\n\t\t\tinitialPod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tWaiting: &corev1api.ContainerStateWaiting{},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\texpectedErrors: []error{errors.New(\"hook my-hook-1 in container container1 in pod default/my-pod not executed: context deadline exceeded\")},\n\t\t\tbyContainer: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"my-hook-1\",\n\t\t\t\t\t\tHookSource: HookSourceSpec,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeFail,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedExecutions: []expectedExecution{},\n\t\t\tsharedHooksContextTimeout: time.Millisecond,\n\t\t},\n\t\t{\n\t\t\tname: \"should return error when shared hooks context is canceled before spec hook with OnError mode Continue executes\",\n\t\t\texpectedErrors: []error{errors.New(\"hook my-hook-1 in container container1 in pod default/my-pod not executed: context deadline exceeded\")},\n\t\t\tgroupResource: \"pods\",\n\t\t\tinitialPod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tWaiting: &corev1api.ContainerStateWaiting{},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\tbyContainer: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"my-hook-1\",\n\t\t\t\t\t\tHookSource: HookSourceSpec,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedExecutions: []expectedExecution{},\n\t\t\tsharedHooksContextTimeout: time.Millisecond,\n\t\t},\n\t\t{\n\t\t\tname: \"should return no error with 2 spec hooks in 2 different containers, 1st container starts running after 10ms, 2nd container after 20ms, both succeed\",\n\t\t\tgroupResource: \"pods\",\n\t\t\tinitialPod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container2\",\n\t\t\t\t}).\n\t\t\t\t// initially both are waiting\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tWaiting: &corev1api.ContainerStateWaiting{},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container2\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tWaiting: &corev1api.ContainerStateWaiting{},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\texpectedErrors: nil,\n\t\t\tbyContainer: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"my-hook-1\",\n\t\t\t\t\t\tHookSource: HookSourceSpec,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"container2\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"my-hook-1\",\n\t\t\t\t\t\tHookSource: HookSourceSpec,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container2\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/bar\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedExecutions: []expectedExecution{\n\t\t\t\t{\n\t\t\t\t\tname: \"my-hook-1\",\n\t\t\t\t\thook: &velerov1api.ExecHook{\n\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t},\n\t\t\t\t\terror: nil,\n\t\t\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\t\t\tObjectMeta(builder.WithResourceVersion(\"2\")).\n\t\t\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\t\t\tName: \"container2\",\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}).\n\t\t\t\t\t\t// container 2 is still waiting when the first hook executes in container1\n\t\t\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\t\t\tName: \"container2\",\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tWaiting: &corev1api.ContainerStateWaiting{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tResult(),\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tname: \"my-hook-1\",\n\t\t\t\t\thook: &velerov1api.ExecHook{\n\t\t\t\t\t\tContainer: \"container2\",\n\t\t\t\t\t\tCommand: []string{\"/usr/bin/bar\"},\n\t\t\t\t\t},\n\t\t\t\t\terror: nil,\n\t\t\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\t\t\tObjectMeta(builder.WithResourceVersion(\"3\")).\n\t\t\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\t\t\tName: \"container2\",\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\t\t\tName: \"container2\",\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tResult(),\n\t\t\t\t},\n\t\t\t},\n\t\t\tchanges: []change{\n\t\t\t\t// 1st modification: container1 starts running, resourceVersion 2, container2 still waiting\n\t\t\t\t{\n\t\t\t\t\twait: 10 * time.Millisecond,\n\t\t\t\t\tupdated: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\t\t\tObjectMeta(builder.WithResourceVersion(\"2\")).\n\t\t\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\t\t\tName: \"container2\",\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\t\t\tName: \"container2\",\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tWaiting: &corev1api.ContainerStateWaiting{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tResult(),\n\t\t\t\t},\n\t\t\t\t// 2nd modification: container2 starts running, resourceVersion 3\n\t\t\t\t{\n\t\t\t\t\twait: 10 * time.Millisecond,\n\t\t\t\t\tupdated: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\t\t\tObjectMeta(builder.WithResourceVersion(\"3\")).\n\t\t\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\t\t\tName: \"container2\",\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\t\t\tName: \"container2\",\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tResult(),\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Multiple hooks with non-sequential indices (bug #9359)\",\n\t\t\tinitialPod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\tgroupResource: \"pods\",\n\t\t\tbyContainer: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"first-hook\",\n\t\t\t\t\t\tHookSource: HookSourceAnnotation,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t},\n\t\t\t\t\t\thookIndex: 0,\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"second-hook\",\n\t\t\t\t\t\tHookSource: HookSourceAnnotation,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/bar\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t},\n\t\t\t\t\t\thookIndex: 2,\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"third-hook\",\n\t\t\t\t\t\tHookSource: HookSourceAnnotation,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/third\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t},\n\t\t\t\t\t\thookIndex: 4,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedExecutions: []expectedExecution{\n\t\t\t\t{\n\t\t\t\t\tname: \"first-hook\",\n\t\t\t\t\thook: &velerov1api.ExecHook{\n\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\tTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t},\n\t\t\t\t\terror: nil,\n\t\t\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\t\t\tObjectMeta(builder.WithResourceVersion(\"1\")).\n\t\t\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tResult(),\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tname: \"second-hook\",\n\t\t\t\t\thook: &velerov1api.ExecHook{\n\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\tCommand: []string{\"/usr/bin/bar\"},\n\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\tTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t},\n\t\t\t\t\terror: nil,\n\t\t\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\t\t\tObjectMeta(builder.WithResourceVersion(\"1\")).\n\t\t\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tResult(),\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tname: \"third-hook\",\n\t\t\t\t\thook: &velerov1api.ExecHook{\n\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\tCommand: []string{\"/usr/bin/third\"},\n\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\tTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t},\n\t\t\t\t\terror: nil,\n\t\t\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\t\t\tObjectMeta(builder.WithResourceVersion(\"1\")).\n\t\t\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tResult(),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedErrors: nil,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tsource := fcache.NewFakeControllerSource()\n\t\t\tgo func() {\n\t\t\t\t// This is the state of the pod that will be seen by the AddFunc handler.\n\t\t\t\tsource.Add(test.initialPod)\n\t\t\t\t// Changes holds the versions of the pod over time. Each of these states\n\t\t\t\t// will be seen by the UpdateFunc handler.\n\t\t\t\tfor _, change := range test.changes {\n\t\t\t\t\ttime.Sleep(change.wait)\n\t\t\t\t\tsource.Modify(change.updated)\n\t\t\t\t}\n\t\t\t}()\n\n\t\t\tpodCommandExecutor := &velerotest.MockPodCommandExecutor{}\n\t\t\tdefer podCommandExecutor.AssertExpectations(t)\n\n\t\t\th := &DefaultWaitExecHookHandler{\n\t\t\t\tPodCommandExecutor: podCommandExecutor,\n\t\t\t\tListWatchFactory: &fakeListWatchFactory{source},\n\t\t\t}\n\n\t\t\tfor _, e := range test.expectedExecutions {\n\t\t\t\tobj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(e.pod)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tpodCommandExecutor.On(\"ExecutePodCommand\", mock.Anything, obj, e.pod.Namespace, e.pod.Name, e.name, e.hook).Return(e.error)\n\t\t\t}\n\n\t\t\tctx := t.Context()\n\t\t\tif test.sharedHooksContextTimeout > 0 {\n\t\t\t\tvar ctxCancel context.CancelFunc\n\t\t\t\tctx, ctxCancel = context.WithTimeout(ctx, test.sharedHooksContextTimeout)\n\t\t\t\tdefer ctxCancel()\n\t\t\t}\n\n\t\t\thookTracker := NewMultiHookTracker()\n\t\t\terrs := h.HandleHooks(ctx, velerotest.NewLogger(), test.initialPod, test.byContainer, hookTracker, \"restore1\")\n\n\t\t\t// for i, ee := range test.expectedErrors {\n\t\t\trequire.Len(t, errs, len(test.expectedErrors))\n\t\t\tfor i, ee := range test.expectedErrors {\n\t\t\t\tassert.EqualError(t, errs[i], ee.Error())\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestPodHasContainer(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpod *corev1api.Pod\n\t\tcontainer string\n\t\texpect bool\n\t}{\n\t\t{\n\t\t\tname: \"has container\",\n\t\t\texpect: true,\n\t\t\tcontainer: \"container1\",\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t},\n\t\t{\n\t\t\tname: \"does not have container\",\n\t\t\texpect: false,\n\t\t\tcontainer: \"container1\",\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container2\",\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tactual := podHasContainer(test.pod, test.container)\n\t\t\tassert.Equal(t, actual, test.expect)\n\t\t})\n\t}\n}\n\nfunc TestIsContainerUp(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpod *corev1api.Pod\n\t\tcontainer string\n\t\texpect bool\n\t\thooks []PodExecRestoreHook\n\t}{\n\t\t{\n\t\t\tname: \"should return true when running\",\n\t\t\tcontainer: \"container1\",\n\t\t\texpect: true,\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\thooks: []PodExecRestoreHook{},\n\t\t},\n\t\t{\n\t\t\tname: \"should return false when running but not ready\",\n\t\t\tcontainer: \"container1\",\n\t\t\texpect: false,\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t},\n\t\t\t\t\tReady: false,\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\thooks: []PodExecRestoreHook{\n\t\t\t\t{\n\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\tWaitForReady: boolptr.True(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should return true when running and ready\",\n\t\t\tcontainer: \"container1\",\n\t\t\texpect: true,\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t},\n\t\t\t\t\tReady: true,\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\thooks: []PodExecRestoreHook{\n\t\t\t\t{\n\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\tWaitForReady: boolptr.True(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should return false when no state is set\",\n\t\t\tcontainer: \"container1\",\n\t\t\texpect: false,\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\thooks: []PodExecRestoreHook{},\n\t\t},\n\t\t{\n\t\t\tname: \"should return false when waiting\",\n\t\t\tcontainer: \"container1\",\n\t\t\texpect: false,\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tWaiting: &corev1api.ContainerStateWaiting{},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\thooks: []PodExecRestoreHook{},\n\t\t},\n\t\t{\n\t\t\tname: \"should return true when running and first container is terminated\",\n\t\t\tcontainer: \"container1\",\n\t\t\texpect: true,\n\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container0\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tTerminated: &corev1api.ContainerStateTerminated{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\t&corev1api.ContainerStatus{\n\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t\t},\n\t\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\thooks: []PodExecRestoreHook{},\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tactual := isContainerUp(test.pod, test.container, test.hooks)\n\t\t\tassert.Equal(t, actual, test.expect)\n\t\t})\n\t}\n}\n\nfunc TestMaxHookWait(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbyContainer map[string][]PodExecRestoreHook\n\t\texpect time.Duration\n\t}{\n\t\t{\n\t\t\tname: \"should return 0 for nil map\",\n\t\t\tbyContainer: nil,\n\t\t\texpect: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"should return 0 if all hooks are 0 or negative\",\n\t\t\texpect: 0,\n\t\t\tbyContainer: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: 0},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: -1},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should return biggest wait timeout from multiple hooks in multiple containers\",\n\t\t\texpect: time.Hour,\n\t\t\tbyContainer: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"container2\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Hour},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should return 0 if any hook does not have a wait timeout\",\n\t\t\texpect: 0,\n\t\t\tbyContainer: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: 0},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tactual := maxHookWait(test.byContainer)\n\t\t\tassert.Equal(t, actual, test.expect)\n\t\t})\n\t}\n}\n\nfunc TestRestoreHookTrackerUpdate(t *testing.T) {\n\ttype expectedExecution struct {\n\t\thook *velerov1api.ExecHook\n\t\tname string\n\t\terror error\n\t\tpod *corev1api.Pod\n\t}\n\n\thookTracker1 := NewMultiHookTracker()\n\thookTracker1.Add(\"restore1\", \"default\", \"my-pod\", \"container1\", HookSourceAnnotation, \"\", HookPhase(\"\"), 0)\n\n\thookTracker2 := NewMultiHookTracker()\n\thookTracker2.Add(\"restore1\", \"default\", \"my-pod\", \"container1\", HookSourceSpec, \"my-hook-1\", HookPhase(\"\"), 0)\n\n\thookTracker3 := NewMultiHookTracker()\n\thookTracker3.Add(\"restore1\", \"default\", \"my-pod\", \"container1\", HookSourceSpec, \"my-hook-1\", HookPhase(\"\"), 0)\n\thookTracker3.Add(\"restore1\", \"default\", \"my-pod\", \"container2\", HookSourceSpec, \"my-hook-2\", HookPhase(\"\"), 0)\n\n\thookTracker4 := NewMultiHookTracker()\n\thookTracker4.Add(\"restore1\", \"default\", \"my-pod\", \"container1\", HookSourceSpec, \"my-hook-1\", HookPhase(\"\"), 0)\n\n\ttests1 := []struct {\n\t\tname string\n\t\tinitialPod *corev1api.Pod\n\t\tgroupResource string\n\t\tbyContainer map[string][]PodExecRestoreHook\n\t\texpectedExecutions []expectedExecution\n\t\thookTracker *MultiHookTracker\n\t\texpectedFailed int\n\t}{\n\t\t{\n\t\t\tname: \"a hook executes successfully\",\n\t\t\tinitialPod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tObjectMeta(builder.WithAnnotations(\n\t\t\t\t\tpodRestoreHookCommandAnnotationKey, \"/usr/bin/foo\",\n\t\t\t\t\tpodRestoreHookContainerAnnotationKey, \"container1\",\n\t\t\t\t\tpodRestoreHookOnErrorAnnotationKey, string(velerov1api.HookErrorModeContinue),\n\t\t\t\t\tpodRestoreHookTimeoutAnnotationKey, \"1s\",\n\t\t\t\t\tpodRestoreHookWaitTimeoutAnnotationKey, \"1m\",\n\t\t\t\t)).\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\tgroupResource: \"pods\",\n\t\t\tbyContainer: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"\",\n\t\t\t\t\t\tHookSource: HookSourceAnnotation,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\tExecTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Minute},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedExecutions: []expectedExecution{\n\t\t\t\t{\n\t\t\t\t\tname: \"\",\n\t\t\t\t\thook: &velerov1api.ExecHook{\n\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\tTimeout: metav1.Duration{Duration: time.Second},\n\t\t\t\t\t},\n\t\t\t\t\terror: nil,\n\t\t\t\t\tpod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\t\t\tObjectMeta(builder.WithResourceVersion(\"1\")).\n\t\t\t\t\t\tObjectMeta(builder.WithAnnotations(\n\t\t\t\t\t\t\tpodRestoreHookCommandAnnotationKey, \"/usr/bin/foo\",\n\t\t\t\t\t\t\tpodRestoreHookContainerAnnotationKey, \"container1\",\n\t\t\t\t\t\t\tpodRestoreHookOnErrorAnnotationKey, string(velerov1api.HookErrorModeContinue),\n\t\t\t\t\t\t\tpodRestoreHookTimeoutAnnotationKey, \"1s\",\n\t\t\t\t\t\t\tpodRestoreHookWaitTimeoutAnnotationKey, \"1m\",\n\t\t\t\t\t\t)).\n\t\t\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\t\t\tName: \"container1\",\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tResult(),\n\t\t\t\t},\n\t\t\t},\n\t\t\thookTracker: hookTracker1,\n\t\t\texpectedFailed: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"a hook with OnError mode Fail failed to execute\",\n\t\t\tgroupResource: \"pods\",\n\t\t\tinitialPod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tWaiting: &corev1api.ContainerStateWaiting{},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\tbyContainer: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"my-hook-1\",\n\t\t\t\t\t\tHookSource: HookSourceSpec,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeFail,\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Millisecond},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\thookTracker: hookTracker2,\n\t\t\texpectedFailed: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"a hook with OnError mode Continue failed to execute\",\n\t\t\tgroupResource: \"pods\",\n\t\t\tinitialPod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tWaiting: &corev1api.ContainerStateWaiting{},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\tbyContainer: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"my-hook-1\",\n\t\t\t\t\t\tHookSource: HookSourceSpec,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Millisecond},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\thookTracker: hookTracker4,\n\t\t\texpectedFailed: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"two hooks with OnError mode Continue failed to execute\",\n\t\t\tgroupResource: \"pods\",\n\t\t\tinitialPod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container2\",\n\t\t\t\t}).\n\t\t\t\t// initially both are waiting\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tWaiting: &corev1api.ContainerStateWaiting{},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container2\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tWaiting: &corev1api.ContainerStateWaiting{},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\tbyContainer: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"my-hook-1\",\n\t\t\t\t\t\tHookSource: HookSourceSpec,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Millisecond},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"container2\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"my-hook-2\",\n\t\t\t\t\t\tHookSource: HookSourceSpec,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container2\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/bar\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Millisecond},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\thookTracker: hookTracker3,\n\t\t\texpectedFailed: 2,\n\t\t},\n\t\t{\n\t\t\tname: \"a hook was recorded before added to tracker\",\n\t\t\tgroupResource: \"pods\",\n\t\t\tinitialPod: builder.ForPod(\"default\", \"my-pod\").\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t}).\n\t\t\t\tContainerStatuses(&corev1api.ContainerStatus{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\tWaiting: &corev1api.ContainerStateWaiting{},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\tbyContainer: map[string][]PodExecRestoreHook{\n\t\t\t\t\"container1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tHookName: \"my-hook-1\",\n\t\t\t\t\t\tHookSource: HookSourceSpec,\n\t\t\t\t\t\tHook: velerov1api.ExecRestoreHook{\n\t\t\t\t\t\t\tContainer: \"container1\",\n\t\t\t\t\t\t\tCommand: []string{\"/usr/bin/foo\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\tWaitTimeout: metav1.Duration{Duration: time.Millisecond},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\thookTracker: NewMultiHookTracker(),\n\t\t\texpectedFailed: 0,\n\t\t},\n\t}\n\n\tfor _, test := range tests1 {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tsource := fcache.NewFakeControllerSource()\n\t\t\tgo func() {\n\t\t\t\t// This is the state of the pod that will be seen by the AddFunc handler.\n\t\t\t\tsource.Add(test.initialPod)\n\t\t\t}()\n\n\t\t\tpodCommandExecutor := &velerotest.MockPodCommandExecutor{}\n\t\t\tdefer podCommandExecutor.AssertExpectations(t)\n\n\t\t\th := &DefaultWaitExecHookHandler{\n\t\t\t\tPodCommandExecutor: podCommandExecutor,\n\t\t\t\tListWatchFactory: &fakeListWatchFactory{source},\n\t\t\t}\n\n\t\t\tfor _, e := range test.expectedExecutions {\n\t\t\t\tobj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(e.pod)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tpodCommandExecutor.On(\"ExecutePodCommand\", mock.Anything, obj, e.pod.Namespace, e.pod.Name, e.name, e.hook).Return(e.error)\n\t\t\t}\n\n\t\t\tctx := t.Context()\n\t\t\t_ = h.HandleHooks(ctx, velerotest.NewLogger(), test.initialPod, test.byContainer, test.hookTracker, \"restore1\")\n\t\t\t_, actualFailed := test.hookTracker.Stat(\"restore1\")\n\t\t\tassert.Equal(t, test.expectedFailed, actualFailed)\n\t\t})\n\t}\n}\n" }, { "path": "internal/resourcemodifiers/json_merge_patch.go", "content": "package resourcemodifiers\n\nimport (\n\t\"fmt\"\n\n\tjsonpatch \"github.com/evanphx/json-patch/v5\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"sigs.k8s.io/yaml\"\n)\n\ntype JSONMergePatch struct {\n\tPatchData string `json:\"patchData,omitempty\"`\n}\n\ntype JSONMergePatcher struct {\n\tpatches []JSONMergePatch\n}\n\nfunc (p *JSONMergePatcher) Patch(u *unstructured.Unstructured, _ logrus.FieldLogger) (*unstructured.Unstructured, error) {\n\tobjBytes, err := u.MarshalJSON()\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"error in marshaling object %s\", err)\n\t}\n\n\tfor _, patch := range p.patches {\n\t\tpatchBytes, err := yaml.YAMLToJSON([]byte(patch.PatchData))\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"error in converting YAML to JSON %s\", err)\n\t\t}\n\n\t\tobjBytes, err = jsonpatch.MergePatch(objBytes, patchBytes)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"error in applying JSON Patch: %s\", err.Error())\n\t\t}\n\t}\n\n\tupdated := &unstructured.Unstructured{}\n\terr = updated.UnmarshalJSON(objBytes)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"error in unmarshalling modified object %s\", err.Error())\n\t}\n\n\treturn updated, nil\n}\n" }, { "path": "internal/resourcemodifiers/json_merge_patch_test.go", "content": "package resourcemodifiers\n\nimport (\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\tclientgoscheme \"k8s.io/client-go/kubernetes/scheme\"\n)\n\nfunc TestJsonMergePatchFailure(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdata string\n\t}{\n\t\t{\n\t\t\tname: \"patch with bad yaml\",\n\t\t\tdata: \"a: b:\",\n\t\t},\n\t\t{\n\t\t\tname: \"patch with bad json\",\n\t\t\tdata: `{\"a\"::1}`,\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tscheme := runtime.NewScheme()\n\t\t\terr := clientgoscheme.AddToScheme(scheme)\n\t\t\trequire.NoError(t, err)\n\t\t\tpt := &JSONMergePatcher{\n\t\t\t\tpatches: []JSONMergePatch{{PatchData: tt.data}},\n\t\t\t}\n\n\t\t\tu := &unstructured.Unstructured{}\n\t\t\t_, err = pt.Patch(u, logrus.New())\n\t\t\tassert.Error(t, err)\n\t\t})\n\t}\n}\n" }, { "path": "internal/resourcemodifiers/json_patch.go", "content": "package resourcemodifiers\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"strconv\"\n\t\"strings\"\n\n\tjsonpatch \"github.com/evanphx/json-patch/v5\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n)\n\ntype JSONPatch struct {\n\tOperation string `json:\"operation\"`\n\tFrom string `json:\"from,omitempty\"`\n\tPath string `json:\"path\"`\n\tValue string `json:\"value,omitempty\"`\n}\n\nfunc (p *JSONPatch) ToString() string {\n\tif addQuotes(&p.Value) {\n\t\treturn fmt.Sprintf(`{\"op\": \"%s\", \"from\": \"%s\", \"path\": \"%s\", \"value\": \"%s\"}`, p.Operation, p.From, p.Path, p.Value)\n\t}\n\treturn fmt.Sprintf(`{\"op\": \"%s\", \"from\": \"%s\", \"path\": \"%s\", \"value\": %s}`, p.Operation, p.From, p.Path, p.Value)\n}\n\nfunc addQuotes(value *string) bool {\n\tif *value == \"\" {\n\t\treturn true\n\t}\n\t// if value is escaped, remove escape and add quotes\n\t// this is useful for scenarios where boolean, null and numbers are required to be set as string.\n\tif strings.HasPrefix(*value, \"\\\"\") && strings.HasSuffix(*value, \"\\\"\") {\n\t\t*value = strings.TrimPrefix(*value, \"\\\"\")\n\t\t*value = strings.TrimSuffix(*value, \"\\\"\")\n\t\treturn true\n\t}\n\t// if value is null, then don't add quotes\n\tif *value == \"null\" {\n\t\treturn false\n\t}\n\t// if value is a boolean, then don't add quotes\n\tif strings.ToLower(*value) == \"true\" || strings.ToLower(*value) == \"false\" {\n\t\treturn false\n\t}\n\t// if value is a json object or array, then don't add quotes.\n\tif strings.HasPrefix(*value, \"{\") || strings.HasPrefix(*value, \"[\") {\n\t\treturn false\n\t}\n\t// if value is a number, then don't add quotes\n\tif _, err := strconv.ParseFloat(*value, 64); err == nil {\n\t\treturn false\n\t}\n\treturn true\n}\n\ntype JSONPatcher struct {\n\tpatches []JSONPatch `yaml:\"patches\"`\n}\n\nfunc (p *JSONPatcher) Patch(u *unstructured.Unstructured, logger logrus.FieldLogger) (*unstructured.Unstructured, error) {\n\tmodifiedObjBytes, err := p.applyPatch(u)\n\tif err != nil {\n\t\tif errors.Is(err, jsonpatch.ErrTestFailed) {\n\t\t\tlogger.Infof(\"Test operation failed for JSON Patch %s\", err.Error())\n\t\t\treturn u.DeepCopy(), nil\n\t\t}\n\t\treturn nil, fmt.Errorf(\"error in applying JSON Patch %s\", err.Error())\n\t}\n\n\tupdated := &unstructured.Unstructured{}\n\terr = updated.UnmarshalJSON(modifiedObjBytes)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"error in unmarshalling modified object %s\", err.Error())\n\t}\n\n\treturn updated, nil\n}\n\nfunc (p *JSONPatcher) applyPatch(u *unstructured.Unstructured) ([]byte, error) {\n\tpatchBytes := p.patchArrayToByteArray()\n\tjsonPatch, err := jsonpatch.DecodePatch(patchBytes)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"error in decoding json patch %s\", err.Error())\n\t}\n\n\tobjBytes, err := u.MarshalJSON()\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"error in marshaling object %s\", err.Error())\n\t}\n\n\treturn jsonPatch.Apply(objBytes)\n}\n\nfunc (p *JSONPatcher) patchArrayToByteArray() []byte {\n\tvar patches []string\n\tfor _, patch := range p.patches {\n\t\tpatches = append(patches, patch.ToString())\n\t}\n\tpatchesStr := strings.Join(patches, \",\\n\\t\")\n\treturn []byte(fmt.Sprintf(`[%s]`, patchesStr))\n}\n" }, { "path": "internal/resourcemodifiers/resource_modifiers.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\thttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage resourcemodifiers\n\nimport (\n\t\"fmt\"\n\t\"regexp\"\n\n\tjsonpatch \"github.com/evanphx/json-patch/v5\"\n\t\"github.com/gobwas/glob\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"sigs.k8s.io/yaml\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/collections\"\n)\n\nconst (\n\tConfigmapRefType = \"configmap\"\n\tResourceModifierSupportedVersionV1 = \"v1\"\n)\n\ntype MatchRule struct {\n\tPath string `json:\"path,omitempty\"`\n\tValue string `json:\"value,omitempty\"`\n}\n\ntype Conditions struct {\n\tNamespaces []string `json:\"namespaces,omitempty\"`\n\tGroupResource string `json:\"groupResource\"`\n\tResourceNameRegex string `json:\"resourceNameRegex,omitempty\"`\n\tLabelSelector *metav1.LabelSelector `json:\"labelSelector,omitempty\"`\n\tMatches []MatchRule `json:\"matches,omitempty\"`\n}\n\ntype ResourceModifierRule struct {\n\tConditions Conditions `json:\"conditions\"`\n\tPatches []JSONPatch `json:\"patches,omitempty\"`\n\tMergePatches []JSONMergePatch `json:\"mergePatches,omitempty\"`\n\tStrategicPatches []StrategicMergePatch `json:\"strategicPatches,omitempty\"`\n}\n\ntype ResourceModifiers struct {\n\tVersion string `json:\"version\"`\n\tResourceModifierRules []ResourceModifierRule `json:\"resourceModifierRules\"`\n}\n\nfunc GetResourceModifiersFromConfig(cm *corev1api.ConfigMap) (*ResourceModifiers, error) {\n\tif cm == nil {\n\t\treturn nil, fmt.Errorf(\"could not parse config from nil configmap\")\n\t}\n\tif len(cm.Data) != 1 {\n\t\treturn nil, fmt.Errorf(\"illegal resource modifiers %s/%s configmap\", cm.Namespace, cm.Name)\n\t}\n\n\tvar yamlData string\n\tfor _, v := range cm.Data {\n\t\tyamlData = v\n\t}\n\n\tresModifiers, err := unmarshalResourceModifiers([]byte(yamlData))\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\treturn resModifiers, nil\n}\n\nfunc (p *ResourceModifiers) ApplyResourceModifierRules(obj *unstructured.Unstructured, groupResource string, scheme *runtime.Scheme, log logrus.FieldLogger) []error {\n\tvar errs []error\n\torigin := obj\n\t// If there are more than one rules, we need to keep the original object for condition matching\n\tif len(p.ResourceModifierRules) > 1 {\n\t\torigin = obj.DeepCopy()\n\t}\n\tfor _, rule := range p.ResourceModifierRules {\n\t\tmatched, err := rule.match(origin, groupResource, log)\n\t\tif err != nil {\n\t\t\terrs = append(errs, err)\n\t\t\tcontinue\n\t\t} else if !matched {\n\t\t\tcontinue\n\t\t}\n\n\t\tlog.Infof(\"Applying resource modifier patch on %s/%s\", origin.GetNamespace(), origin.GetName())\n\t\terr = rule.applyPatch(obj, scheme, log)\n\t\tif err != nil {\n\t\t\terrs = append(errs, err)\n\t\t}\n\t}\n\n\treturn errs\n}\n\nfunc (r *ResourceModifierRule) match(obj *unstructured.Unstructured, groupResource string, log logrus.FieldLogger) (bool, error) {\n\tns := obj.GetNamespace()\n\tif ns != \"\" {\n\t\tnamespaceInclusion := collections.NewIncludesExcludes().Includes(r.Conditions.Namespaces...)\n\t\tif !namespaceInclusion.ShouldInclude(ns) {\n\t\t\treturn false, nil\n\t\t}\n\t}\n\n\tg, err := glob.Compile(r.Conditions.GroupResource, '.')\n\tif err != nil {\n\t\tlog.Errorf(\"Bad glob pattern of groupResource in condition, groupResource: %s, err: %s\", r.Conditions.GroupResource, err)\n\t\treturn false, err\n\t}\n\n\tif !g.Match(groupResource) {\n\t\treturn false, nil\n\t}\n\n\tif r.Conditions.ResourceNameRegex != \"\" {\n\t\tmatch, err := regexp.MatchString(r.Conditions.ResourceNameRegex, obj.GetName())\n\t\tif err != nil {\n\t\t\treturn false, errors.Errorf(\"error in matching regex %s\", err.Error())\n\t\t}\n\t\tif !match {\n\t\t\treturn false, nil\n\t\t}\n\t}\n\n\tif r.Conditions.LabelSelector != nil {\n\t\tselector, err := metav1.LabelSelectorAsSelector(r.Conditions.LabelSelector)\n\t\tif err != nil {\n\t\t\treturn false, errors.Errorf(\"error in creating label selector %s\", err.Error())\n\t\t}\n\t\tif !selector.Matches(labels.Set(obj.GetLabels())) {\n\t\t\treturn false, nil\n\t\t}\n\t}\n\n\tmatch, err := matchConditions(obj, r.Conditions.Matches, log)\n\tif err != nil {\n\t\treturn false, err\n\t} else if !match {\n\t\tlog.Info(\"Conditions do not match, skip it\")\n\t\treturn false, nil\n\t}\n\n\treturn true, nil\n}\n\nfunc matchConditions(u *unstructured.Unstructured, rules []MatchRule, _ logrus.FieldLogger) (bool, error) {\n\tif len(rules) == 0 {\n\t\treturn true, nil\n\t}\n\n\tvar fixed []JSONPatch\n\tfor _, rule := range rules {\n\t\tif rule.Path == \"\" {\n\t\t\treturn false, fmt.Errorf(\"path is required for match rule\")\n\t\t}\n\n\t\tfixed = append(fixed, JSONPatch{\n\t\t\tOperation: \"test\",\n\t\t\tPath: rule.Path,\n\t\t\tValue: rule.Value,\n\t\t})\n\t}\n\n\tp := &JSONPatcher{patches: fixed}\n\t_, err := p.applyPatch(u)\n\tif err != nil {\n\t\tif errors.Is(err, jsonpatch.ErrTestFailed) || errors.Is(err, jsonpatch.ErrMissing) {\n\t\t\treturn false, nil\n\t\t}\n\t\treturn false, err\n\t}\n\n\treturn true, nil\n}\n\nfunc unmarshalResourceModifiers(yamlData []byte) (*ResourceModifiers, error) {\n\tresModifiers := &ResourceModifiers{}\n\terr := yaml.UnmarshalStrict(yamlData, resModifiers)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to decode yaml data into resource modifiers, err: %s\", err)\n\t}\n\treturn resModifiers, nil\n}\n\ntype patcher interface {\n\tPatch(u *unstructured.Unstructured, logger logrus.FieldLogger) (*unstructured.Unstructured, error)\n}\n\nfunc (r *ResourceModifierRule) applyPatch(u *unstructured.Unstructured, scheme *runtime.Scheme, logger logrus.FieldLogger) error {\n\tvar p patcher\n\tif len(r.Patches) > 0 {\n\t\tp = &JSONPatcher{patches: r.Patches}\n\t} else if len(r.MergePatches) > 0 {\n\t\tp = &JSONMergePatcher{patches: r.MergePatches}\n\t} else if len(r.StrategicPatches) > 0 {\n\t\tp = &StrategicMergePatcher{patches: r.StrategicPatches, scheme: scheme}\n\t} else {\n\t\treturn fmt.Errorf(\"no patch data found\")\n\t}\n\n\tupdated, err := p.Patch(u, logger)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"error in applying patch %s\", err)\n\t}\n\n\tu.SetUnstructuredContent(updated.Object)\n\treturn nil\n}\n" }, { "path": "internal/resourcemodifiers/resource_modifiers_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\thttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage resourcemodifiers\n\nimport (\n\t\"reflect\"\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/serializer/yaml\"\n\tutilruntime \"k8s.io/apimachinery/pkg/util/runtime\"\n\tclientgoscheme \"k8s.io/client-go/kubernetes/scheme\"\n)\n\nfunc TestGetResourceModifiersFromConfig(t *testing.T) {\n\tcm1 := &corev1api.ConfigMap{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-configmap\",\n\t\t\tNamespace: \"test-namespace\",\n\t\t},\n\t\tData: map[string]string{\n\t\t\t\"sub.yml\": \"version: v1\\nresourceModifierRules:\\n- conditions:\\n groupResource: persistentvolumeclaims\\n resourceNameRegex: \\\".*\\\"\\n namespaces:\\n - bar\\n - foo\\n patches:\\n - operation: replace\\n path: \\\"/spec/storageClassName\\\"\\n value: \\\"premium\\\"\\n - operation: remove\\n path: \\\"/metadata/labels/test\\\"\\n\\n\\n\",\n\t\t},\n\t}\n\n\trules1 := &ResourceModifiers{\n\t\tVersion: \"v1\",\n\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t{\n\t\t\t\tConditions: Conditions{\n\t\t\t\t\tGroupResource: \"persistentvolumeclaims\",\n\t\t\t\t\tResourceNameRegex: \".*\",\n\t\t\t\t\tNamespaces: []string{\"bar\", \"foo\"},\n\t\t\t\t},\n\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t{\n\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\t\t\tValue: \"premium\",\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tOperation: \"remove\",\n\t\t\t\t\t\tPath: \"/metadata/labels/test\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tcm2 := &corev1api.ConfigMap{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-configmap\",\n\t\t\tNamespace: \"test-namespace\",\n\t\t},\n\t\tData: map[string]string{\n\t\t\t\"sub.yml\": \"version: v1\\nresourceModifierRules:\\n- conditions:\\n groupResource: deployments.apps\\n resourceNameRegex: \\\"^test-.*$\\\"\\n namespaces:\\n - bar\\n - foo\\n patches:\\n - operation: add\\n path: \\\"/spec/template/spec/containers/0\\\"\\n value: \\\"{\\\\\\\"name\\\\\\\": \\\\\\\"nginx\\\\\\\", \\\\\\\"image\\\\\\\": \\\\\\\"nginx:1.14.2\\\\\\\", \\\\\\\"ports\\\\\\\": [{\\\\\\\"containerPort\\\\\\\": 80}]}\\\"\\n - operation: copy\\n from: \\\"/spec/template/spec/containers/0\\\"\\n path: \\\"/spec/template/spec/containers/1\\\"\\n\\n\\n\",\n\t\t},\n\t}\n\n\trules2 := &ResourceModifiers{\n\t\tVersion: \"v1\",\n\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t{\n\t\t\t\tConditions: Conditions{\n\t\t\t\t\tGroupResource: \"deployments.apps\",\n\t\t\t\t\tResourceNameRegex: \"^test-.*$\",\n\t\t\t\t\tNamespaces: []string{\"bar\", \"foo\"},\n\t\t\t\t},\n\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t{\n\t\t\t\t\t\tOperation: \"add\",\n\t\t\t\t\t\tPath: \"/spec/template/spec/containers/0\",\n\t\t\t\t\t\tValue: `{\"name\": \"nginx\", \"image\": \"nginx:1.14.2\", \"ports\": [{\"containerPort\": 80}]}`,\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tOperation: \"copy\",\n\t\t\t\t\t\tFrom: \"/spec/template/spec/containers/0\",\n\t\t\t\t\t\tPath: \"/spec/template/spec/containers/1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tcm3 := &corev1api.ConfigMap{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-configmap\",\n\t\t\tNamespace: \"test-namespace\",\n\t\t},\n\t\tData: map[string]string{\n\t\t\t\"sub.yml\": \"version1: v1\\nresourceModifierRules:\\n- conditions:\\n groupResource: deployments.apps\\n resourceNameRegex: \\\"^test-.*$\\\"\\n namespaces:\\n - bar\\n - foo\\n patches:\\n - operation: add\\n path: \\\"/spec/template/spec/containers/0\\\"\\n value: \\\"{\\\\\\\"name\\\\\\\": \\\\\\\"nginx\\\\\\\", \\\\\\\"image\\\\\\\": \\\\\\\"nginx:1.14.2\\\\\\\", \\\\\\\"ports\\\\\\\": [{\\\\\\\"containerPort\\\\\\\": 80}]}\\\"\\n - operation: copy\\n from: \\\"/spec/template/spec/containers/0\\\"\\n path: \\\"/spec/template/spec/containers/1\\\"\\n\\n\\n\",\n\t\t},\n\t}\n\n\tcm4 := &corev1api.ConfigMap{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-configmap\",\n\t\t\tNamespace: \"test-namespace\",\n\t\t},\n\t\tData: map[string]string{\n\t\t\t\"sub.yml\": \"version: v1\\nresourceModifierRules:\\n- conditions:\\n groupResource: deployments.apps\\n labelSelector:\\n matchLabels:\\n a: b\\n\",\n\t\t},\n\t}\n\n\trules4 := &ResourceModifiers{\n\t\tVersion: \"v1\",\n\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t{\n\t\t\t\tConditions: Conditions{\n\t\t\t\t\tGroupResource: \"deployments.apps\",\n\t\t\t\t\tLabelSelector: &metav1.LabelSelector{\n\t\t\t\t\t\tMatchLabels: map[string]string{\n\t\t\t\t\t\t\t\"a\": \"b\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tcm5 := &corev1api.ConfigMap{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-configmap\",\n\t\t\tNamespace: \"test-namespace\",\n\t\t},\n\t\tData: map[string]string{\n\t\t\t\"sub.yml\": \"version: v1\\nresourceModifierRules:\\n- conditions:\\n groupResource: pods\\n namespaces:\\n - ns1\\n matches:\\n - path: /metadata/annotations/foo\\n value: bar\\n mergePatches:\\n - patchData: |\\n metadata:\\n annotations:\\n foo: null\",\n\t\t},\n\t}\n\n\trules5 := &ResourceModifiers{\n\t\tVersion: \"v1\",\n\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t{\n\t\t\t\tConditions: Conditions{\n\t\t\t\t\tGroupResource: \"pods\",\n\t\t\t\t\tNamespaces: []string{\n\t\t\t\t\t\t\"ns1\",\n\t\t\t\t\t},\n\t\t\t\t\tMatches: []MatchRule{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tPath: \"/metadata/annotations/foo\",\n\t\t\t\t\t\t\tValue: \"bar\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tMergePatches: []JSONMergePatch{\n\t\t\t\t\t{\n\t\t\t\t\t\tPatchData: \"metadata:\\n annotations:\\n foo: null\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tcm6 := &corev1api.ConfigMap{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-configmap\",\n\t\t\tNamespace: \"test-namespace\",\n\t\t},\n\t\tData: map[string]string{\n\t\t\t\"sub.yml\": \"version: v1\\nresourceModifierRules:\\n- conditions:\\n groupResource: pods\\n namespaces:\\n - ns1\\n strategicPatches:\\n - patchData: |\\n spec:\\n containers:\\n - name: nginx\\n image: repo2/nginx\",\n\t\t},\n\t}\n\n\trules6 := &ResourceModifiers{\n\t\tVersion: \"v1\",\n\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t{\n\t\t\t\tConditions: Conditions{\n\t\t\t\t\tGroupResource: \"pods\",\n\t\t\t\t\tNamespaces: []string{\n\t\t\t\t\t\t\"ns1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStrategicPatches: []StrategicMergePatch{\n\t\t\t\t\t{\n\t\t\t\t\t\tPatchData: \"spec:\\n containers:\\n - name: nginx\\n image: repo2/nginx\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tcm7 := &corev1api.ConfigMap{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-configmap\",\n\t\t\tNamespace: \"test-namespace\",\n\t\t},\n\t\tData: map[string]string{\n\t\t\t\"sub.yml\": \"version: v1\\nresourceModifierRules:\\n- conditions:\\n groupResource: pods\\n namespaces:\\n - ns1\\n mergePatches:\\n - patchData: |\\n {\\\"metadata\\\":{\\\"annotations\\\":{\\\"foo\\\":null}}}\",\n\t\t},\n\t}\n\n\trules7 := &ResourceModifiers{\n\t\tVersion: \"v1\",\n\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t{\n\t\t\t\tConditions: Conditions{\n\t\t\t\t\tGroupResource: \"pods\",\n\t\t\t\t\tNamespaces: []string{\n\t\t\t\t\t\t\"ns1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tMergePatches: []JSONMergePatch{\n\t\t\t\t\t{\n\t\t\t\t\t\tPatchData: `{\"metadata\":{\"annotations\":{\"foo\":null}}}`,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tcm8 := &corev1api.ConfigMap{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-configmap\",\n\t\t\tNamespace: \"test-namespace\",\n\t\t},\n\t\tData: map[string]string{\n\t\t\t\"sub.yml\": \"version: v1\\nresourceModifierRules:\\n- conditions:\\n groupResource: pods\\n namespaces:\\n - ns1\\n strategicPatches:\\n - patchData: |\\n {\\\"spec\\\":{\\\"containers\\\":[{\\\"name\\\": \\\"nginx\\\",\\\"image\\\": \\\"repo2/nginx\\\"}]}}\",\n\t\t},\n\t}\n\n\trules8 := &ResourceModifiers{\n\t\tVersion: \"v1\",\n\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t{\n\t\t\t\tConditions: Conditions{\n\t\t\t\t\tGroupResource: \"pods\",\n\t\t\t\t\tNamespaces: []string{\n\t\t\t\t\t\t\"ns1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStrategicPatches: []StrategicMergePatch{\n\t\t\t\t\t{\n\t\t\t\t\t\tPatchData: `{\"spec\":{\"containers\":[{\"name\": \"nginx\",\"image\": \"repo2/nginx\"}]}}`,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tcm9 := &corev1api.ConfigMap{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-configmap\",\n\t\t\tNamespace: \"test-namespace\",\n\t\t},\n\t\tData: map[string]string{\n\t\t\t\"sub.yml\": \"version: v1\\nresourceModifierRules:\\n- conditions:\\n groupResource: deployments.apps\\n resourceNameRegex: \\\"^test-.*$\\\"\\n namespaces:\\n - bar\\n - foo\\n patches:\\n - operation: replace\\n path: \\\"/value/bool\\\"\\n value: \\\"\\\\\\\"true\\\\\\\"\\\"\\n\\n\\n\",\n\t\t},\n\t}\n\n\trules9 := &ResourceModifiers{\n\t\tVersion: \"v1\",\n\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t{\n\t\t\t\tConditions: Conditions{\n\t\t\t\t\tGroupResource: \"deployments.apps\",\n\t\t\t\t\tResourceNameRegex: \"^test-.*$\",\n\t\t\t\t\tNamespaces: []string{\"bar\", \"foo\"},\n\t\t\t\t},\n\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t{\n\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\tPath: \"/value/bool\",\n\t\t\t\t\t\tValue: `\"true\"`,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tcm10 := &corev1api.ConfigMap{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-configmap\",\n\t\t\tNamespace: \"test-namespace\",\n\t\t},\n\t\tData: map[string]string{\n\t\t\t\"sub.yml\": \"version: v1\\nresourceModifierRules:\\n- conditions:\\n groupResource: deployments.apps\\n resourceNameRegex: \\\"^test-.*$\\\"\\n namespaces:\\n - bar\\n - foo\\n patches:\\n - operation: replace\\n path: \\\"/value/bool\\\"\\n value: \\\"true\\\"\\n\\n\\n\",\n\t\t},\n\t}\n\n\trules10 := &ResourceModifiers{\n\t\tVersion: \"v1\",\n\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t{\n\t\t\t\tConditions: Conditions{\n\t\t\t\t\tGroupResource: \"deployments.apps\",\n\t\t\t\t\tResourceNameRegex: \"^test-.*$\",\n\t\t\t\t\tNamespaces: []string{\"bar\", \"foo\"},\n\t\t\t\t},\n\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t{\n\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\tPath: \"/value/bool\",\n\t\t\t\t\t\tValue: \"true\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\ttype args struct {\n\t\tcm *corev1api.ConfigMap\n\t}\n\ttests := []struct {\n\t\tname string\n\t\targs args\n\t\twant *ResourceModifiers\n\t\twantErr bool\n\t}{\n\t\t{\n\t\t\tname: \"test 1\",\n\t\t\targs: args{\n\t\t\t\tcm: cm1,\n\t\t\t},\n\t\t\twant: rules1,\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"complex payload in add and copy operator\",\n\t\t\targs: args{\n\t\t\t\tcm: cm2,\n\t\t\t},\n\t\t\twant: rules2,\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"invalid payload version1\",\n\t\t\targs: args{\n\t\t\t\tcm: cm3,\n\t\t\t},\n\t\t\twant: nil,\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"match labels\",\n\t\t\targs: args{\n\t\t\t\tcm: cm4,\n\t\t\t},\n\t\t\twant: rules4,\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"nil configmap\",\n\t\t\targs: args{\n\t\t\t\tcm: nil,\n\t\t\t},\n\t\t\twant: nil,\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"complex yaml data with json merge patch\",\n\t\t\targs: args{\n\t\t\t\tcm: cm5,\n\t\t\t},\n\t\t\twant: rules5,\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"complex yaml data with strategic merge patch\",\n\t\t\targs: args{\n\t\t\t\tcm: cm6,\n\t\t\t},\n\t\t\twant: rules6,\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"complex json data with json merge patch\",\n\t\t\targs: args{\n\t\t\t\tcm: cm7,\n\t\t\t},\n\t\t\twant: rules7,\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"complex json data with strategic merge patch\",\n\t\t\targs: args{\n\t\t\t\tcm: cm8,\n\t\t\t},\n\t\t\twant: rules8,\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"bool value as string\",\n\t\t\targs: args{\n\t\t\t\tcm: cm9,\n\t\t\t},\n\t\t\twant: rules9,\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"bool value as bool\",\n\t\t\targs: args{\n\t\t\t\tcm: cm10,\n\t\t\t},\n\t\t\twant: rules10,\n\t\t\twantErr: false,\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tgot, err := GetResourceModifiersFromConfig(tt.args.cm)\n\t\t\tif (err != nil) != tt.wantErr {\n\t\t\t\tt.Errorf(\"GetResourceModifiersFromConfig() error = %v, wantErr %v\", err, tt.wantErr)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif !reflect.DeepEqual(got, tt.want) {\n\t\t\t\tt.Errorf(\"GetResourceModifiersFromConfig() = %v, want %v\", got, tt.want)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestResourceModifiers_ApplyResourceModifierRules(t *testing.T) {\n\tpvcStandardSc := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"PersistentVolumeClaim\",\n\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\"name\": \"test-pvc\",\n\t\t\t\t\"namespace\": \"foo\",\n\t\t\t},\n\t\t\t\"spec\": map[string]any{\n\t\t\t\t\"storageClassName\": \"standard\",\n\t\t\t},\n\t\t},\n\t}\n\n\tpvcPremiumSc := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"PersistentVolumeClaim\",\n\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\"name\": \"test-pvc\",\n\t\t\t\t\"namespace\": \"foo\",\n\t\t\t},\n\t\t\t\"spec\": map[string]any{\n\t\t\t\t\"storageClassName\": \"premium\",\n\t\t\t},\n\t\t},\n\t}\n\n\tpvcGoldSc := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"PersistentVolumeClaim\",\n\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\"name\": \"test-pvc\",\n\t\t\t\t\"namespace\": \"foo\",\n\t\t\t},\n\t\t\t\"spec\": map[string]any{\n\t\t\t\t\"storageClassName\": \"gold\",\n\t\t\t},\n\t\t},\n\t}\n\n\tdeployNginxOneReplica := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"apiVersion\": \"apps/v1\",\n\t\t\t\"kind\": \"Deployment\",\n\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\"name\": \"test-deployment\",\n\t\t\t\t\"namespace\": \"foo\",\n\t\t\t\t\"labels\": map[string]any{\n\t\t\t\t\t\"app\": \"nginx\",\n\t\t\t\t},\n\t\t\t},\n\t\t\t\"spec\": map[string]any{\n\t\t\t\t\"replicas\": int64(1),\n\t\t\t\t\"template\": map[string]any{\n\t\t\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\t\t\"labels\": map[string]any{\n\t\t\t\t\t\t\t\"app\": \"nginx\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t\"spec\": map[string]any{\n\t\t\t\t\t\t\"containers\": []any{\n\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\"name\": \"nginx\",\n\t\t\t\t\t\t\t\t\"image\": \"nginx:latest\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tdeployNginxTwoReplica := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"apiVersion\": \"apps/v1\",\n\t\t\t\"kind\": \"Deployment\",\n\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\"name\": \"test-deployment\",\n\t\t\t\t\"namespace\": \"foo\",\n\t\t\t\t\"labels\": map[string]any{\n\t\t\t\t\t\"app\": \"nginx\",\n\t\t\t\t},\n\t\t\t},\n\t\t\t\"spec\": map[string]any{\n\t\t\t\t\"replicas\": int64(2),\n\t\t\t\t\"template\": map[string]any{\n\t\t\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\t\t\"labels\": map[string]any{\n\t\t\t\t\t\t\t\"app\": \"nginx\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t\"spec\": map[string]any{\n\t\t\t\t\t\t\"containers\": []any{\n\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\"name\": \"nginx\",\n\t\t\t\t\t\t\t\t\"image\": \"nginx:latest\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tdeployNginxMysql := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"apiVersion\": \"apps/v1\",\n\t\t\t\"kind\": \"Deployment\",\n\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\"name\": \"test-deployment\",\n\t\t\t\t\"namespace\": \"foo\",\n\t\t\t\t\"labels\": map[string]any{\n\t\t\t\t\t\"app\": \"nginx\",\n\t\t\t\t},\n\t\t\t},\n\t\t\t\"spec\": map[string]any{\n\t\t\t\t\"replicas\": int64(1),\n\t\t\t\t\"template\": map[string]any{\n\t\t\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\t\t\"labels\": map[string]any{\n\t\t\t\t\t\t\t\"app\": \"nginx\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t\"spec\": map[string]any{\n\t\t\t\t\t\t\"containers\": []any{\n\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\"name\": \"nginx\",\n\t\t\t\t\t\t\t\t\"image\": \"nginx:latest\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\"name\": \"mysql\",\n\t\t\t\t\t\t\t\t\"image\": \"mysql:latest\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tcmTrue := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"ConfigMap\",\n\t\t\t\"data\": map[string]any{\n\t\t\t\t\"test\": \"true\",\n\t\t\t},\n\t\t},\n\t}\n\tcmFalse := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"ConfigMap\",\n\t\t\t\"data\": map[string]any{\n\t\t\t\t\"test\": \"false\",\n\t\t\t},\n\t\t},\n\t}\n\ttype fields struct {\n\t\tVersion string\n\t\tResourceModifierRules []ResourceModifierRule\n\t}\n\ttype args struct {\n\t\tobj *unstructured.Unstructured\n\t\tgroupResource string\n\t}\n\ttests := []struct {\n\t\tname string\n\t\tfields fields\n\t\targs args\n\t\twantErr bool\n\t\twantObj *unstructured.Unstructured\n\t}{\n\t\t{\n\t\t\tname: \"configmap true false string\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"configmaps\",\n\t\t\t\t\t\t\tResourceNameRegex: \".*\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\t\t\tPath: \"/data/test\",\n\t\t\t\t\t\t\t\tValue: `\"false\"`,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\targs: args{\n\t\t\t\tobj: cmTrue.DeepCopy(),\n\t\t\t\tgroupResource: \"configmaps\",\n\t\t\t},\n\t\t\twantErr: false,\n\t\t\twantObj: cmFalse.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"Invalid Regex throws error\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"persistentvolumeclaims\",\n\t\t\t\t\t\t\tResourceNameRegex: \"[a-z\",\n\t\t\t\t\t\t\tNamespaces: []string{\"foo\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"test\",\n\t\t\t\t\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\t\t\t\t\tValue: \"standard\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\t\t\t\t\tValue: \"premium\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\targs: args{\n\t\t\t\tobj: pvcStandardSc.DeepCopy(),\n\t\t\t\tgroupResource: \"persistentvolumeclaims\",\n\t\t\t},\n\t\t\twantErr: true,\n\t\t\twantObj: pvcStandardSc.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"pvc with standard storage class should be patched to premium\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"persistentvolumeclaims\",\n\t\t\t\t\t\t\tResourceNameRegex: \".*\",\n\t\t\t\t\t\t\tNamespaces: []string{\"foo\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"test\",\n\t\t\t\t\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\t\t\t\t\tValue: \"standard\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\t\t\t\t\tValue: \"premium\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\targs: args{\n\t\t\t\tobj: pvcStandardSc.DeepCopy(),\n\t\t\t\tgroupResource: \"persistentvolumeclaims\",\n\t\t\t},\n\t\t\twantErr: false,\n\t\t\twantObj: pvcPremiumSc.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"pvc with standard storage class should be patched to premium, even when rules are [standard => premium, premium => gold]\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"persistentvolumeclaims\",\n\t\t\t\t\t\t\tResourceNameRegex: \".*\",\n\t\t\t\t\t\t\tMatches: []MatchRule{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\t\t\t\t\t\tValue: \"standard\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\t\t\t\t\tValue: \"premium\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"persistentvolumeclaims\",\n\t\t\t\t\t\t\tResourceNameRegex: \".*\",\n\t\t\t\t\t\t\tMatches: []MatchRule{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\t\t\t\t\t\tValue: \"premium\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\t\t\t\t\tValue: \"gold\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\targs: args{\n\t\t\t\tobj: pvcStandardSc.DeepCopy(),\n\t\t\t\tgroupResource: \"persistentvolumeclaims\",\n\t\t\t},\n\t\t\twantErr: false,\n\t\t\twantObj: pvcPremiumSc.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"pvc with standard storage class should be patched to gold, even when rules are [standard => premium, standard => gold]\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"persistentvolumeclaims\",\n\t\t\t\t\t\t\tResourceNameRegex: \".*\",\n\t\t\t\t\t\t\tMatches: []MatchRule{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\t\t\t\t\t\tValue: \"standard\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\t\t\t\t\tValue: \"premium\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"persistentvolumeclaims\",\n\t\t\t\t\t\t\tResourceNameRegex: \".*\",\n\t\t\t\t\t\t\tMatches: []MatchRule{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\t\t\t\t\t\tValue: \"standard\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\t\t\t\t\tValue: \"gold\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\targs: args{\n\t\t\t\tobj: pvcStandardSc.DeepCopy(),\n\t\t\t\tgroupResource: \"persistentvolumeclaims\",\n\t\t\t},\n\t\t\twantErr: false,\n\t\t\twantObj: pvcGoldSc.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"nginx deployment: 1 -> 2 replicas\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"deployments.apps\",\n\t\t\t\t\t\t\tResourceNameRegex: \"^test-.*$\",\n\t\t\t\t\t\t\tNamespaces: []string{\"foo\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"test\",\n\t\t\t\t\t\t\t\tPath: \"/spec/replicas\",\n\t\t\t\t\t\t\t\tValue: \"1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\t\t\tPath: \"/spec/replicas\",\n\t\t\t\t\t\t\t\tValue: \"2\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\targs: args{\n\t\t\t\tobj: deployNginxOneReplica.DeepCopy(),\n\t\t\t\tgroupResource: \"deployments.apps\",\n\t\t\t},\n\t\t\twantErr: false,\n\t\t\twantObj: deployNginxTwoReplica.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"nginx deployment: test operator fails, skips substitution, no error\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"deployments.apps\",\n\t\t\t\t\t\t\tResourceNameRegex: \"^test-.*$\",\n\t\t\t\t\t\t\tNamespaces: []string{\"foo\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"test\",\n\t\t\t\t\t\t\t\tPath: \"/spec/replicas\",\n\t\t\t\t\t\t\t\tValue: \"5\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\t\t\tPath: \"/spec/replicas\",\n\t\t\t\t\t\t\t\tValue: \"2\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\targs: args{\n\t\t\t\tobj: deployNginxOneReplica.DeepCopy(),\n\t\t\t\tgroupResource: \"deployments.apps\",\n\t\t\t},\n\t\t\twantErr: false,\n\t\t\twantObj: deployNginxOneReplica.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"nginx deployment: Empty Resource Regex\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"deployments.apps\",\n\t\t\t\t\t\t\tNamespaces: []string{\"foo\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"test\",\n\t\t\t\t\t\t\t\tPath: \"/spec/replicas\",\n\t\t\t\t\t\t\t\tValue: \"1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\t\t\tPath: \"/spec/replicas\",\n\t\t\t\t\t\t\t\tValue: \"2\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\targs: args{\n\t\t\t\tobj: deployNginxOneReplica.DeepCopy(),\n\t\t\t\tgroupResource: \"deployments.apps\",\n\t\t\t},\n\t\t\twantErr: false,\n\t\t\twantObj: deployNginxTwoReplica.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"nginx deployment: Empty Resource Regex\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"deployments.apps\",\n\t\t\t\t\t\t\tNamespaces: []string{\"foo\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"test\",\n\t\t\t\t\t\t\t\tPath: \"/spec/replicas\",\n\t\t\t\t\t\t\t\tValue: \"1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\t\t\tPath: \"/spec/replicas\",\n\t\t\t\t\t\t\t\tValue: \"2\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\targs: args{\n\t\t\t\tobj: deployNginxOneReplica.DeepCopy(),\n\t\t\t\tgroupResource: \"deployments.apps\",\n\t\t\t},\n\t\t\twantErr: false,\n\t\t\twantObj: deployNginxTwoReplica.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"nginx deployment: Empty Resource Regex and namespaces list\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"deployments.apps\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"test\",\n\t\t\t\t\t\t\t\tPath: \"/spec/replicas\",\n\t\t\t\t\t\t\t\tValue: \"1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\t\t\tPath: \"/spec/replicas\",\n\t\t\t\t\t\t\t\tValue: \"2\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\targs: args{\n\t\t\t\tobj: deployNginxOneReplica.DeepCopy(),\n\t\t\t\tgroupResource: \"deployments.apps\",\n\t\t\t},\n\t\t\twantErr: false,\n\t\t\twantObj: deployNginxTwoReplica.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"nginx deployment: namespace doesn't match\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"deployments.apps\",\n\t\t\t\t\t\t\tResourceNameRegex: \".*\",\n\t\t\t\t\t\t\tNamespaces: []string{\"bar\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"test\",\n\t\t\t\t\t\t\t\tPath: \"/spec/replicas\",\n\t\t\t\t\t\t\t\tValue: \"1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\t\t\tPath: \"/spec/replicas\",\n\t\t\t\t\t\t\t\tValue: \"2\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\targs: args{\n\t\t\t\tobj: deployNginxOneReplica.DeepCopy(),\n\t\t\t\tgroupResource: \"deployments.apps\",\n\t\t\t},\n\t\t\twantErr: false,\n\t\t\twantObj: deployNginxOneReplica.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"add container mysql to deployment\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"deployments.apps\",\n\t\t\t\t\t\t\tResourceNameRegex: \"^test-.*$\",\n\t\t\t\t\t\t\tNamespaces: []string{\"foo\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"add\",\n\t\t\t\t\t\t\t\tPath: \"/spec/template/spec/containers/1\",\n\t\t\t\t\t\t\t\tValue: `{\"name\": \"mysql\", \"image\": \"mysql:latest\"}`,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\targs: args{\n\t\t\t\tobj: deployNginxOneReplica,\n\t\t\t\tgroupResource: \"deployments.apps\",\n\t\t\t},\n\t\t\twantErr: false,\n\t\t\twantObj: deployNginxMysql,\n\t\t},\n\t\t{\n\t\t\tname: \"Copy container 0 to container 1 and then modify container 1\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"deployments.apps\",\n\t\t\t\t\t\t\tResourceNameRegex: \"^test-.*$\",\n\t\t\t\t\t\t\tNamespaces: []string{\"foo\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"copy\",\n\t\t\t\t\t\t\t\tFrom: \"/spec/template/spec/containers/0\",\n\t\t\t\t\t\t\t\tPath: \"/spec/template/spec/containers/1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"test\",\n\t\t\t\t\t\t\t\tPath: \"/spec/template/spec/containers/1/image\",\n\t\t\t\t\t\t\t\tValue: \"nginx:latest\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\t\t\tPath: \"/spec/template/spec/containers/1/name\",\n\t\t\t\t\t\t\t\tValue: \"mysql\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\t\t\tPath: \"/spec/template/spec/containers/1/image\",\n\t\t\t\t\t\t\t\tValue: \"mysql:latest\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\targs: args{\n\t\t\t\tobj: deployNginxOneReplica.DeepCopy(),\n\t\t\t\tgroupResource: \"deployments.apps\",\n\t\t\t},\n\t\t\twantErr: false,\n\t\t\twantObj: deployNginxMysql.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"nginx deployment: match label selector\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"deployments.apps\",\n\t\t\t\t\t\t\tNamespaces: []string{\"foo\"},\n\t\t\t\t\t\t\tLabelSelector: &metav1.LabelSelector{\n\t\t\t\t\t\t\t\tMatchLabels: map[string]string{\n\t\t\t\t\t\t\t\t\t\"app\": \"nginx\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"test\",\n\t\t\t\t\t\t\t\tPath: \"/spec/replicas\",\n\t\t\t\t\t\t\t\tValue: \"1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\t\t\tPath: \"/spec/replicas\",\n\t\t\t\t\t\t\t\tValue: \"2\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\targs: args{\n\t\t\t\tobj: deployNginxOneReplica.DeepCopy(),\n\t\t\t\tgroupResource: \"deployments.apps\",\n\t\t\t},\n\t\t\twantErr: false,\n\t\t\twantObj: deployNginxTwoReplica.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"nginx deployment: mismatch label selector\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"deployments.apps\",\n\t\t\t\t\t\t\tNamespaces: []string{\"foo\"},\n\t\t\t\t\t\t\tLabelSelector: &metav1.LabelSelector{\n\t\t\t\t\t\t\t\tMatchLabels: map[string]string{\n\t\t\t\t\t\t\t\t\t\"app\": \"nginx-mismatch\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"test\",\n\t\t\t\t\t\t\t\tPath: \"/spec/replicas\",\n\t\t\t\t\t\t\t\tValue: \"1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\t\t\tPath: \"/spec/replicas\",\n\t\t\t\t\t\t\t\tValue: \"2\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\targs: args{\n\t\t\t\tobj: deployNginxOneReplica.DeepCopy(),\n\t\t\t\tgroupResource: \"deployments.apps\",\n\t\t\t},\n\t\t\twantErr: false,\n\t\t\twantObj: deployNginxOneReplica.DeepCopy(),\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tp := &ResourceModifiers{\n\t\t\t\tVersion: tt.fields.Version,\n\t\t\t\tResourceModifierRules: tt.fields.ResourceModifierRules,\n\t\t\t}\n\t\t\tgot := p.ApplyResourceModifierRules(tt.args.obj, tt.args.groupResource, nil, logrus.New())\n\n\t\t\tassert.Equal(t, tt.wantErr, len(got) > 0)\n\t\t\tassert.Equal(t, *tt.wantObj, *tt.args.obj)\n\t\t})\n\t}\n}\n\nvar podYAMLWithNginxImage = `\napiVersion: v1\nkind: Pod\nmetadata:\n name: pod1\n namespace: fake\nspec:\n containers:\n - image: nginx\n name: nginx\n`\n\nvar podYAMLWithNginx1Image = `\napiVersion: v1\nkind: Pod\nmetadata:\n name: pod1\n namespace: fake\nspec:\n containers:\n - image: nginx1\n name: nginx\n`\n\nvar podYAMLWithNFSVolume = `\napiVersion: v1\nkind: Pod\nmetadata:\n name: pod1\n namespace: fake\nspec:\n containers:\n - image: fake\n name: fake\n volumeMounts:\n - mountPath: /fake1\n name: vol1\n - mountPath: /fake2\n name: vol2\n volumes:\n - name: vol1\n nfs:\n path: /fake2\n - name: vol2\n emptyDir: {}\n`\n\nvar podYAMLWithPVCVolume = `\napiVersion: v1\nkind: Pod\nmetadata:\n name: pod1\n namespace: fake\nspec:\n containers:\n - image: fake\n name: fake\n volumeMounts:\n - mountPath: /fake1\n name: vol1\n - mountPath: /fake2\n name: vol2\n volumes:\n - name: vol1\n persistentVolumeClaim:\n claimName: pvc1\n - name: vol2\n emptyDir: {}\n`\n\nvar svcYAMLWithPort8000 = `\napiVersion: v1\nkind: Service\nmetadata:\n name: svc1\n namespace: fake\nspec:\n ports:\n - name: fake1\n port: 8001\n protocol: TCP\n targetPort: 8001\n - name: fake\n port: 8000\n protocol: TCP\n targetPort: 8000\n - name: fake2\n port: 8002\n protocol: TCP\n targetPort: 8002\n`\n\nvar svcYAMLWithPort9000 = `\napiVersion: v1\nkind: Service\nmetadata:\n name: svc1\n namespace: fake\nspec:\n ports:\n - name: fake1\n port: 8001\n protocol: TCP\n targetPort: 8001\n - name: fake\n port: 9000\n protocol: TCP\n targetPort: 9000\n - name: fake2\n port: 8002\n protocol: TCP\n targetPort: 8002\n`\n\nvar cmYAMLWithLabelAToB = `\napiVersion: v1\nkind: ConfigMap\nmetadata:\n name: cm1\n namespace: fake\n labels:\n a: b\n c: d\n`\n\nvar cmYAMLWithLabelAToC = `\napiVersion: v1\nkind: ConfigMap\nmetadata:\n name: cm1\n namespace: fake\n labels:\n a: c\n c: d\n`\n\nvar cmYAMLWithoutLabelA = `\napiVersion: v1\nkind: ConfigMap\nmetadata:\n name: cm1\n namespace: fake\n labels:\n c: d\n`\n\nfunc TestResourceModifiers_ApplyResourceModifierRules_StrategicMergePatch(t *testing.T) {\n\tscheme := runtime.NewScheme()\n\tutilruntime.Must(clientgoscheme.AddToScheme(scheme))\n\tunstructuredSerializer := yaml.NewDecodingSerializer(unstructured.UnstructuredJSONScheme)\n\to1, _, err := unstructuredSerializer.Decode([]byte(podYAMLWithNFSVolume), nil, nil)\n\trequire.NoError(t, err)\n\tpodWithNFSVolume := o1.(*unstructured.Unstructured)\n\n\to2, _, err := unstructuredSerializer.Decode([]byte(podYAMLWithPVCVolume), nil, nil)\n\trequire.NoError(t, err)\n\tpodWithPVCVolume := o2.(*unstructured.Unstructured)\n\n\to3, _, err := unstructuredSerializer.Decode([]byte(svcYAMLWithPort8000), nil, nil)\n\trequire.NoError(t, err)\n\tsvcWithPort8000 := o3.(*unstructured.Unstructured)\n\n\to4, _, err := unstructuredSerializer.Decode([]byte(svcYAMLWithPort9000), nil, nil)\n\trequire.NoError(t, err)\n\tsvcWithPort9000 := o4.(*unstructured.Unstructured)\n\n\to5, _, err := unstructuredSerializer.Decode([]byte(podYAMLWithNginxImage), nil, nil)\n\trequire.NoError(t, err)\n\tpodWithNginxImage := o5.(*unstructured.Unstructured)\n\n\to6, _, err := unstructuredSerializer.Decode([]byte(podYAMLWithNginx1Image), nil, nil)\n\trequire.NoError(t, err)\n\tpodWithNginx1Image := o6.(*unstructured.Unstructured)\n\n\ttests := []struct {\n\t\tname string\n\t\trm *ResourceModifiers\n\t\tobj *unstructured.Unstructured\n\t\tgroupResource string\n\t\twantErr bool\n\t\twantObj *unstructured.Unstructured\n\t}{\n\t\t{\n\t\t\tname: \"update image\",\n\t\t\trm: &ResourceModifiers{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"pods\",\n\t\t\t\t\t\t\tNamespaces: []string{\"fake\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tStrategicPatches: []StrategicMergePatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tPatchData: `{\"spec\":{\"containers\":[{\"name\":\"nginx\",\"image\":\"nginx1\"}]}}`,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tobj: podWithNginxImage.DeepCopy(),\n\t\t\tgroupResource: \"pods\",\n\t\t\twantErr: false,\n\t\t\twantObj: podWithNginx1Image.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"update image with yaml format\",\n\t\t\trm: &ResourceModifiers{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"pods\",\n\t\t\t\t\t\t\tNamespaces: []string{\"fake\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tStrategicPatches: []StrategicMergePatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tPatchData: `spec:\n containers:\n - name: nginx\n image: nginx1`,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tobj: podWithNginxImage.DeepCopy(),\n\t\t\tgroupResource: \"pods\",\n\t\t\twantErr: false,\n\t\t\twantObj: podWithNginx1Image.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"replace nfs with pvc in volume\",\n\t\t\trm: &ResourceModifiers{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"pods\",\n\t\t\t\t\t\t\tNamespaces: []string{\"fake\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tStrategicPatches: []StrategicMergePatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tPatchData: `{\"spec\":{\"volumes\":[{\"nfs\":null,\"name\":\"vol1\",\"persistentVolumeClaim\":{\"claimName\":\"pvc1\"}}]}}`,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tobj: podWithNFSVolume.DeepCopy(),\n\t\t\tgroupResource: \"pods\",\n\t\t\twantErr: false,\n\t\t\twantObj: podWithPVCVolume.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"replace any other volume source with pvc in volume\",\n\t\t\trm: &ResourceModifiers{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"pods\",\n\t\t\t\t\t\t\tNamespaces: []string{\"fake\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tStrategicPatches: []StrategicMergePatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tPatchData: `{\"spec\":{\"volumes\":[{\"$retainKeys\":[\"name\",\"persistentVolumeClaim\"],\"name\":\"vol1\",\"persistentVolumeClaim\":{\"claimName\":\"pvc1\"}}]}}`,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tobj: podWithNFSVolume.DeepCopy(),\n\t\t\tgroupResource: \"pods\",\n\t\t\twantErr: false,\n\t\t\twantObj: podWithPVCVolume.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"update a service port\",\n\t\t\trm: &ResourceModifiers{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"services\",\n\t\t\t\t\t\t\tNamespaces: []string{\"fake\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tStrategicPatches: []StrategicMergePatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tPatchData: `{\"spec\":{\"$setElementOrder/ports\":[{\"port\":8001},{\"port\":9000},{\"port\":8002}],\"ports\":[{\"name\":\"fake\",\"port\":9000,\"protocol\":\"TCP\",\"targetPort\":9000},{\"$patch\":\"delete\",\"port\":8000}]}}`,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tobj: svcWithPort8000.DeepCopy(),\n\t\t\tgroupResource: \"services\",\n\t\t\twantErr: false,\n\t\t\twantObj: svcWithPort9000.DeepCopy(),\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tgot := tt.rm.ApplyResourceModifierRules(tt.obj, tt.groupResource, scheme, logrus.New())\n\n\t\t\tassert.Equal(t, tt.wantErr, len(got) > 0)\n\t\t\tassert.Equal(t, *tt.wantObj, *tt.obj)\n\t\t})\n\t}\n}\n\nfunc TestResourceModifiers_ApplyResourceModifierRules_JSONMergePatch(t *testing.T) {\n\tunstructuredSerializer := yaml.NewDecodingSerializer(unstructured.UnstructuredJSONScheme)\n\to1, _, err := unstructuredSerializer.Decode([]byte(cmYAMLWithLabelAToB), nil, nil)\n\trequire.NoError(t, err)\n\tcmWithLabelAToB := o1.(*unstructured.Unstructured)\n\n\to2, _, err := unstructuredSerializer.Decode([]byte(cmYAMLWithLabelAToC), nil, nil)\n\trequire.NoError(t, err)\n\tcmWithLabelAToC := o2.(*unstructured.Unstructured)\n\n\to3, _, err := unstructuredSerializer.Decode([]byte(cmYAMLWithoutLabelA), nil, nil)\n\trequire.NoError(t, err)\n\tcmWithoutLabelA := o3.(*unstructured.Unstructured)\n\n\ttests := []struct {\n\t\tname string\n\t\trm *ResourceModifiers\n\t\tobj *unstructured.Unstructured\n\t\tgroupResource string\n\t\twantErr bool\n\t\twantObj *unstructured.Unstructured\n\t}{\n\t\t{\n\t\t\tname: \"update labels\",\n\t\t\trm: &ResourceModifiers{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"configmaps\",\n\t\t\t\t\t\t\tNamespaces: []string{\"fake\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tMergePatches: []JSONMergePatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tPatchData: `{\"metadata\":{\"labels\":{\"a\":\"c\"}}}`,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tobj: cmWithLabelAToB.DeepCopy(),\n\t\t\tgroupResource: \"configmaps\",\n\t\t\twantErr: false,\n\t\t\twantObj: cmWithLabelAToC.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"update labels in yaml format\",\n\t\t\trm: &ResourceModifiers{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"configmaps\",\n\t\t\t\t\t\t\tNamespaces: []string{\"fake\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tMergePatches: []JSONMergePatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tPatchData: `metadata:\n labels:\n a: c`,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tobj: cmWithLabelAToB.DeepCopy(),\n\t\t\tgroupResource: \"configmaps\",\n\t\t\twantErr: false,\n\t\t\twantObj: cmWithLabelAToC.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"delete labels\",\n\t\t\trm: &ResourceModifiers{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"configmaps\",\n\t\t\t\t\t\t\tNamespaces: []string{\"fake\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tMergePatches: []JSONMergePatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tPatchData: `{\"metadata\":{\"labels\":{\"a\":null}}}`,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tobj: cmWithLabelAToB.DeepCopy(),\n\t\t\tgroupResource: \"configmaps\",\n\t\t\twantErr: false,\n\t\t\twantObj: cmWithoutLabelA.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"add labels\",\n\t\t\trm: &ResourceModifiers{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"configmaps\",\n\t\t\t\t\t\t\tNamespaces: []string{\"fake\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tMergePatches: []JSONMergePatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tPatchData: `{\"metadata\":{\"labels\":{\"a\":\"b\"}}}`,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tobj: cmWithoutLabelA.DeepCopy(),\n\t\t\tgroupResource: \"configmaps\",\n\t\t\twantErr: false,\n\t\t\twantObj: cmWithLabelAToB.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"delete non-existing labels\",\n\t\t\trm: &ResourceModifiers{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"configmaps\",\n\t\t\t\t\t\t\tNamespaces: []string{\"fake\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tMergePatches: []JSONMergePatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tPatchData: `{\"metadata\":{\"labels\":{\"a\":null}}}`,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tobj: cmWithoutLabelA.DeepCopy(),\n\t\t\tgroupResource: \"configmaps\",\n\t\t\twantErr: false,\n\t\t\twantObj: cmWithoutLabelA.DeepCopy(),\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tgot := tt.rm.ApplyResourceModifierRules(tt.obj, tt.groupResource, nil, logrus.New())\n\n\t\t\tassert.Equal(t, tt.wantErr, len(got) > 0)\n\t\t\tassert.Equal(t, *tt.wantObj, *tt.obj)\n\t\t})\n\t}\n}\n\nfunc TestResourceModifiers_wildcard_in_GroupResource(t *testing.T) {\n\tunstructuredSerializer := yaml.NewDecodingSerializer(unstructured.UnstructuredJSONScheme)\n\to1, _, err := unstructuredSerializer.Decode([]byte(cmYAMLWithLabelAToB), nil, nil)\n\trequire.NoError(t, err)\n\tcmWithLabelAToB := o1.(*unstructured.Unstructured)\n\n\to2, _, err := unstructuredSerializer.Decode([]byte(cmYAMLWithLabelAToC), nil, nil)\n\trequire.NoError(t, err)\n\tcmWithLabelAToC := o2.(*unstructured.Unstructured)\n\n\ttests := []struct {\n\t\tname string\n\t\trm *ResourceModifiers\n\t\tobj *unstructured.Unstructured\n\t\tgroupResource string\n\t\twantErr bool\n\t\twantObj *unstructured.Unstructured\n\t}{\n\t\t{\n\t\t\tname: \"match all groups and resources\",\n\t\t\trm: &ResourceModifiers{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"*\",\n\t\t\t\t\t\t\tNamespaces: []string{\"fake\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tMergePatches: []JSONMergePatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tPatchData: `{\"metadata\":{\"labels\":{\"a\":\"c\"}}}`,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tobj: cmWithLabelAToB.DeepCopy(),\n\t\t\tgroupResource: \"configmaps\",\n\t\t\twantErr: false,\n\t\t\twantObj: cmWithLabelAToC.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"match all resources in group apps\",\n\t\t\trm: &ResourceModifiers{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"*.apps\",\n\t\t\t\t\t\t\tNamespaces: []string{\"fake\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tMergePatches: []JSONMergePatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tPatchData: `{\"metadata\":{\"labels\":{\"a\":\"c\"}}}`,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tobj: cmWithLabelAToB.DeepCopy(),\n\t\t\tgroupResource: \"fake.apps\",\n\t\t\twantErr: false,\n\t\t\twantObj: cmWithLabelAToC.DeepCopy(),\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tgot := tt.rm.ApplyResourceModifierRules(tt.obj, tt.groupResource, nil, logrus.New())\n\n\t\t\tassert.Equal(t, tt.wantErr, len(got) > 0)\n\t\t\tassert.Equal(t, *tt.wantObj, *tt.obj)\n\t\t})\n\t}\n}\n\nfunc TestResourceModifiers_conditional_patches(t *testing.T) {\n\tunstructuredSerializer := yaml.NewDecodingSerializer(unstructured.UnstructuredJSONScheme)\n\to1, _, err := unstructuredSerializer.Decode([]byte(cmYAMLWithLabelAToB), nil, nil)\n\trequire.NoError(t, err)\n\tcmWithLabelAToB := o1.(*unstructured.Unstructured)\n\n\to2, _, err := unstructuredSerializer.Decode([]byte(cmYAMLWithLabelAToC), nil, nil)\n\trequire.NoError(t, err)\n\tcmWithLabelAToC := o2.(*unstructured.Unstructured)\n\n\ttests := []struct {\n\t\tname string\n\t\trm *ResourceModifiers\n\t\tobj *unstructured.Unstructured\n\t\tgroupResource string\n\t\twantErr bool\n\t\twantObj *unstructured.Unstructured\n\t}{\n\t\t{\n\t\t\tname: \"match conditions and apply patches\",\n\t\t\trm: &ResourceModifiers{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"*\",\n\t\t\t\t\t\t\tNamespaces: []string{\"fake\"},\n\t\t\t\t\t\t\tMatches: []MatchRule{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tPath: \"/metadata/labels/a\",\n\t\t\t\t\t\t\t\t\tValue: \"b\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tMergePatches: []JSONMergePatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tPatchData: `{\"metadata\":{\"labels\":{\"a\":\"c\"}}}`,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tobj: cmWithLabelAToB.DeepCopy(),\n\t\t\tgroupResource: \"configmaps\",\n\t\t\twantErr: false,\n\t\t\twantObj: cmWithLabelAToC.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"mismatch conditions and skip patches\",\n\t\t\trm: &ResourceModifiers{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"*\",\n\t\t\t\t\t\t\tNamespaces: []string{\"fake\"},\n\t\t\t\t\t\t\tMatches: []MatchRule{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tPath: \"/metadata/labels/a\",\n\t\t\t\t\t\t\t\t\tValue: \"c\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tMergePatches: []JSONMergePatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tPatchData: `{\"metadata\":{\"labels\":{\"a\":\"c\"}}}`,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tobj: cmWithLabelAToB.DeepCopy(),\n\t\t\tgroupResource: \"configmaps\",\n\t\t\twantErr: false,\n\t\t\twantObj: cmWithLabelAToB.DeepCopy(),\n\t\t},\n\t\t{\n\t\t\tname: \"missing condition path and skip patches\",\n\t\t\trm: &ResourceModifiers{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"*\",\n\t\t\t\t\t\t\tNamespaces: []string{\"fake\"},\n\t\t\t\t\t\t\tMatches: []MatchRule{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tPath: \"/metadata/labels/a/b\",\n\t\t\t\t\t\t\t\t\tValue: \"c\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tMergePatches: []JSONMergePatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tPatchData: `{\"metadata\":{\"labels\":{\"a\":\"c\"}}}`,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tobj: cmWithLabelAToB.DeepCopy(),\n\t\t\tgroupResource: \"configmaps\",\n\t\t\twantErr: false,\n\t\t\twantObj: cmWithLabelAToB.DeepCopy(),\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tgot := tt.rm.ApplyResourceModifierRules(tt.obj, tt.groupResource, nil, logrus.New())\n\n\t\t\tassert.Equal(t, tt.wantErr, len(got) > 0)\n\t\t\tassert.Equal(t, *tt.wantObj, *tt.obj)\n\t\t})\n\t}\n}\n\nfunc TestJSONPatch_ToString(t *testing.T) {\n\ttype fields struct {\n\t\tOperation string\n\t\tFrom string\n\t\tPath string\n\t\tValue string\n\t}\n\ttests := []struct {\n\t\tname string\n\t\tfields fields\n\t\twant string\n\t}{\n\t\t{\n\t\t\tname: \"test\",\n\t\t\tfields: fields{\n\t\t\t\tOperation: \"test\",\n\t\t\t\tPath: \"/spec/replicas\",\n\t\t\t\tValue: \"1\",\n\t\t\t},\n\t\t\twant: `{\"op\": \"test\", \"from\": \"\", \"path\": \"/spec/replicas\", \"value\": 1}`,\n\t\t},\n\t\t{\n\t\t\tname: \"replace integer\",\n\t\t\tfields: fields{\n\t\t\t\tOperation: \"replace\",\n\t\t\t\tPath: \"/spec/replicas\",\n\t\t\t\tValue: \"2\",\n\t\t\t},\n\t\t\twant: `{\"op\": \"replace\", \"from\": \"\", \"path\": \"/spec/replicas\", \"value\": 2}`,\n\t\t},\n\t\t{\n\t\t\tname: \"replace array\",\n\t\t\tfields: fields{\n\t\t\t\tOperation: \"replace\",\n\t\t\t\tPath: \"/spec/template/spec/containers/0/ports\",\n\t\t\t\tValue: `[{\"containerPort\": 80}]`,\n\t\t\t},\n\t\t\twant: `{\"op\": \"replace\", \"from\": \"\", \"path\": \"/spec/template/spec/containers/0/ports\", \"value\": [{\"containerPort\": 80}]}`,\n\t\t},\n\t\t{\n\t\t\tname: \"replace with null\",\n\t\t\tfields: fields{\n\t\t\t\tOperation: \"replace\",\n\t\t\t\tPath: \"/spec/template/spec/containers/0/ports\",\n\t\t\t\tValue: `null`,\n\t\t\t},\n\t\t\twant: `{\"op\": \"replace\", \"from\": \"\", \"path\": \"/spec/template/spec/containers/0/ports\", \"value\": null}`,\n\t\t},\n\t\t{\n\t\t\tname: \"add json object\",\n\t\t\tfields: fields{\n\t\t\t\tOperation: \"add\",\n\t\t\t\tPath: \"/spec/template/spec/containers/0\",\n\t\t\t\tValue: `{\"name\": \"nginx\", \"image\": \"nginx:1.14.2\", \"ports\": [{\"containerPort\": 80}]}`,\n\t\t\t},\n\t\t\twant: `{\"op\": \"add\", \"from\": \"\", \"path\": \"/spec/template/spec/containers/0\", \"value\": {\"name\": \"nginx\", \"image\": \"nginx:1.14.2\", \"ports\": [{\"containerPort\": 80}]}}`,\n\t\t},\n\t\t{\n\t\t\tname: \"remove\",\n\t\t\tfields: fields{\n\t\t\t\tOperation: \"remove\",\n\t\t\t\tPath: \"/spec/template/spec/containers/0\",\n\t\t\t},\n\t\t\twant: `{\"op\": \"remove\", \"from\": \"\", \"path\": \"/spec/template/spec/containers/0\", \"value\": \"\"}`,\n\t\t},\n\t\t{\n\t\t\tname: \"move\",\n\t\t\tfields: fields{\n\t\t\t\tOperation: \"move\",\n\t\t\t\tFrom: \"/spec/template/spec/containers/0\",\n\t\t\t\tPath: \"/spec/template/spec/containers/1\",\n\t\t\t},\n\t\t\twant: `{\"op\": \"move\", \"from\": \"/spec/template/spec/containers/0\", \"path\": \"/spec/template/spec/containers/1\", \"value\": \"\"}`,\n\t\t},\n\t\t{\n\t\t\tname: \"copy\",\n\t\t\tfields: fields{\n\t\t\t\tOperation: \"copy\",\n\t\t\t\tFrom: \"/spec/template/spec/containers/0\",\n\t\t\t\tPath: \"/spec/template/spec/containers/1\",\n\t\t\t},\n\t\t\twant: `{\"op\": \"copy\", \"from\": \"/spec/template/spec/containers/0\", \"path\": \"/spec/template/spec/containers/1\", \"value\": \"\"}`,\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tp := &JSONPatch{\n\t\t\t\tOperation: tt.fields.Operation,\n\t\t\t\tFrom: tt.fields.From,\n\t\t\t\tPath: tt.fields.Path,\n\t\t\t\tValue: tt.fields.Value,\n\t\t\t}\n\t\t\tif got := p.ToString(); got != tt.want {\n\t\t\t\tt.Errorf(\"JSONPatch.ToString() = %v, want %v\", got, tt.want)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "internal/resourcemodifiers/resource_modifiers_validator.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\thttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage resourcemodifiers\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n)\n\nfunc (r *ResourceModifierRule) Validate() error {\n\tif err := r.Conditions.Validate(); err != nil {\n\t\treturn err\n\t}\n\n\tcount := 0\n\tfor _, size := range []int{\n\t\tlen(r.Patches),\n\t\tlen(r.MergePatches),\n\t\tlen(r.StrategicPatches),\n\t} {\n\t\tif size != 0 {\n\t\t\tcount++\n\t\t}\n\t\tif count >= 2 {\n\t\t\treturn fmt.Errorf(\"only one of patches, mergePatches, strategicPatches can be specified\")\n\t\t}\n\t}\n\n\tfor _, patch := range r.Patches {\n\t\tif err := patch.Validate(); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc (p *ResourceModifiers) Validate() error {\n\tif !strings.EqualFold(p.Version, ResourceModifierSupportedVersionV1) {\n\t\treturn fmt.Errorf(\"unsupported resource modifier version %s\", p.Version)\n\t}\n\tif len(p.ResourceModifierRules) == 0 {\n\t\treturn fmt.Errorf(\"resource modifier rules cannot be empty\")\n\t}\n\tfor _, rule := range p.ResourceModifierRules {\n\t\tif err := rule.Validate(); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (p *JSONPatch) Validate() error {\n\tif p.Operation == \"\" {\n\t\treturn fmt.Errorf(\"operation cannot be empty\")\n\t}\n\tif operation := strings.ToLower(p.Operation); operation != \"add\" && operation != \"remove\" && operation != \"replace\" && operation != \"test\" && operation != \"move\" && operation != \"copy\" {\n\t\treturn fmt.Errorf(\"unsupported operation %s\", p.Operation)\n\t}\n\tif p.Path == \"\" {\n\t\treturn fmt.Errorf(\"path cannot be empty\")\n\t}\n\treturn nil\n}\n\nfunc (c *Conditions) Validate() error {\n\tif c.GroupResource == \"\" {\n\t\treturn fmt.Errorf(\"groupkResource cannot be empty\")\n\t}\n\treturn nil\n}\n" }, { "path": "internal/resourcemodifiers/resource_modifiers_validator_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\thttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage resourcemodifiers\n\nimport (\n\t\"testing\"\n)\n\nfunc TestResourceModifiers_Validate(t *testing.T) {\n\ttype fields struct {\n\t\tVersion string\n\t\tResourceModifierRules []ResourceModifierRule\n\t}\n\ttests := []struct {\n\t\tname string\n\t\tfields fields\n\t\twantErr bool\n\t}{\n\t\t{\n\t\t\tname: \"correct version, non 0 length ResourceModifierRules\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"persistentvolumeclaims\",\n\t\t\t\t\t\t\tResourceNameRegex: \".*\",\n\t\t\t\t\t\t\tNamespaces: []string{\"bar\", \"foo\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\t\t\t\t\tValue: \"premium\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"incorrect version, non 0 length ResourceModifierRules\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v2\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"persistentvolumeclaims\",\n\t\t\t\t\t\t\tResourceNameRegex: \".*\",\n\t\t\t\t\t\t\tNamespaces: []string{\"bar\", \"foo\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"replace\",\n\t\t\t\t\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\t\t\t\t\tValue: \"premium\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"correct version, 0 length ResourceModifierRules\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{},\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"patch has invalid operation\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"persistentvolumeclaims\",\n\t\t\t\t\t\t\tResourceNameRegex: \".*\",\n\t\t\t\t\t\t\tNamespaces: []string{\"bar\", \"foo\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"invalid\",\n\t\t\t\t\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\t\t\t\t\tValue: \"premium\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Condition has empty GroupResource\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"\",\n\t\t\t\t\t\t\tResourceNameRegex: \".*\",\n\t\t\t\t\t\t\tNamespaces: []string{\"bar\", \"foo\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"invalid\",\n\t\t\t\t\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\t\t\t\t\tValue: \"premium\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"More than one patch type in a rule\",\n\t\t\tfields: fields{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResourceModifierRules: []ResourceModifierRule{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: Conditions{\n\t\t\t\t\t\t\tGroupResource: \"*\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPatches: []JSONPatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tOperation: \"test\",\n\t\t\t\t\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\t\t\t\t\tValue: \"premium\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tMergePatches: []JSONMergePatch{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tPatchData: `{\"metadata\":{\"labels\":{\"a\":null}}}`,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tp := &ResourceModifiers{\n\t\t\t\tVersion: tt.fields.Version,\n\t\t\t\tResourceModifierRules: tt.fields.ResourceModifierRules,\n\t\t\t}\n\t\t\tif err := p.Validate(); (err != nil) != tt.wantErr {\n\t\t\t\tt.Errorf(\"ResourceModifiers.Validate() error = %v, wantErr %v\", err, tt.wantErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestJsonPatch_Validate(t *testing.T) {\n\ttype fields struct {\n\t\tOperation string\n\t\tPath string\n\t\tValue string\n\t}\n\ttests := []struct {\n\t\tname string\n\t\tfields fields\n\t\twantErr bool\n\t}{\n\t\t{\n\t\t\tname: \"not empty operation, path, and new value, valid scenario\",\n\t\t\tfields: fields{\n\t\t\t\tOperation: \"replace\",\n\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\tValue: \"premium\",\n\t\t\t},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"empty operation throws error\",\n\t\t\tfields: fields{\n\t\t\t\tOperation: \"\",\n\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\tValue: \"premium\",\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"empty path throws error\",\n\t\t\tfields: fields{\n\t\t\t\tOperation: \"replace\",\n\t\t\t\tPath: \"\",\n\t\t\t\tValue: \"premium\",\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"invalid operation throws error\",\n\t\t\tfields: fields{\n\t\t\t\tOperation: \"invalid\",\n\t\t\t\tPath: \"/spec/storageClassName\",\n\t\t\t\tValue: \"premium\",\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tp := &JSONPatch{\n\t\t\t\tOperation: tt.fields.Operation,\n\t\t\t\tPath: tt.fields.Path,\n\t\t\t\tValue: tt.fields.Value,\n\t\t\t}\n\t\t\tif err := p.Validate(); (err != nil) != tt.wantErr {\n\t\t\t\tt.Errorf(\"JsonPatch.Validate() error = %v, wantErr %v\", err, tt.wantErr)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "internal/resourcemodifiers/strategic_merge_patch.go", "content": "package resourcemodifiers\n\nimport (\n\t\"fmt\"\n\t\"net/http\"\n\n\t\"github.com/sirupsen/logrus\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/apimachinery/pkg/util/mergepatch\"\n\t\"k8s.io/apimachinery/pkg/util/strategicpatch\"\n\t\"k8s.io/apimachinery/pkg/util/validation/field\"\n\tkubejson \"sigs.k8s.io/json\"\n\t\"sigs.k8s.io/yaml\"\n)\n\ntype StrategicMergePatch struct {\n\tPatchData string `json:\"patchData,omitempty\"`\n}\n\ntype StrategicMergePatcher struct {\n\tpatches []StrategicMergePatch\n\tscheme *runtime.Scheme\n}\n\nfunc (p *StrategicMergePatcher) Patch(u *unstructured.Unstructured, _ logrus.FieldLogger) (*unstructured.Unstructured, error) {\n\tgvk := u.GetObjectKind().GroupVersionKind()\n\tschemaReferenceObj, err := p.scheme.New(gvk)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\torigin := u.DeepCopy()\n\tupdated := u.DeepCopy()\n\tfor _, patch := range p.patches {\n\t\tpatchBytes, err := yaml.YAMLToJSON([]byte(patch.PatchData))\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"error in converting YAML to JSON %s\", err)\n\t\t}\n\n\t\terr = strategicPatchObject(origin, patchBytes, updated, schemaReferenceObj)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"error in applying Strategic Patch %s\", err.Error())\n\t\t}\n\n\t\torigin = updated.DeepCopy()\n\t}\n\n\treturn updated, nil\n}\n\n// strategicPatchObject applies a strategic merge patch of `patchBytes` to\n// `originalObject` and stores the result in `objToUpdate`.\n// It additionally returns the map[string]any representation of the\n// `originalObject` and `patchBytes`.\n// NOTE: Both `originalObject` and `objToUpdate` are supposed to be versioned.\nfunc strategicPatchObject(\n\toriginalObject runtime.Object,\n\tpatchBytes []byte,\n\tobjToUpdate runtime.Object,\n\tschemaReferenceObj runtime.Object,\n) error {\n\toriginalObjMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(originalObject)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tpatchMap := make(map[string]any)\n\tvar strictErrs []error\n\tstrictErrs, err = kubejson.UnmarshalStrict(patchBytes, &patchMap)\n\tif err != nil {\n\t\treturn apierrors.NewBadRequest(err.Error())\n\t}\n\n\tif err := applyPatchToObject(originalObjMap, patchMap, objToUpdate, schemaReferenceObj, strictErrs); err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n\n// applyPatchToObject applies a strategic merge patch of to\n// and stores the result in .\n// NOTE: must be a versioned object.\nfunc applyPatchToObject(\n\toriginalMap map[string]any,\n\tpatchMap map[string]any,\n\tobjToUpdate runtime.Object,\n\tschemaReferenceObj runtime.Object,\n\tstrictErrs []error,\n) error {\n\tpatchedObjMap, err := strategicpatch.StrategicMergeMapPatch(originalMap, patchMap, schemaReferenceObj)\n\tif err != nil {\n\t\treturn interpretStrategicMergePatchError(err)\n\t}\n\n\t// Rather than serialize the patched map to JSON, then decode it to an object, we go directly from a map to an object\n\tconverter := runtime.DefaultUnstructuredConverter\n\tif err := converter.FromUnstructuredWithValidation(patchedObjMap, objToUpdate, true); err != nil {\n\t\tstrictError, isStrictError := runtime.AsStrictDecodingError(err)\n\t\tswitch {\n\t\tcase !isStrictError:\n\t\t\t// disregard any sttrictErrs, because it's an incomplete\n\t\t\t// list of strict errors given that we don't know what fields were\n\t\t\t// unknown because StrategicMergeMapPatch failed.\n\t\t\t// Non-strict errors trump in this case.\n\t\t\treturn apierrors.NewInvalid(schema.GroupKind{}, \"\", field.ErrorList{\n\t\t\t\tfield.Invalid(field.NewPath(\"patch\"), fmt.Sprintf(\"%+v\", patchMap), err.Error()),\n\t\t\t})\n\t\t//case validationDirective == metav1.FieldValidationWarn:\n\t\t//\taddStrictDecodingWarnings(requestContext, append(strictErrs, strictError.Errors()...))\n\t\tdefault:\n\t\t\tstrictDecodingError := runtime.NewStrictDecodingError(append(strictErrs, strictError.Errors()...))\n\t\t\treturn apierrors.NewInvalid(schema.GroupKind{}, \"\", field.ErrorList{\n\t\t\t\tfield.Invalid(field.NewPath(\"patch\"), fmt.Sprintf(\"%+v\", patchMap), strictDecodingError.Error()),\n\t\t\t})\n\t\t}\n\t} else if len(strictErrs) > 0 {\n\t\treturn apierrors.NewInvalid(schema.GroupKind{}, \"\", field.ErrorList{\n\t\t\tfield.Invalid(field.NewPath(\"patch\"), fmt.Sprintf(\"%+v\", patchMap), runtime.NewStrictDecodingError(strictErrs).Error()),\n\t\t})\n\t}\n\n\treturn nil\n}\n\n// interpretStrategicMergePatchError interprets the error type and returns an error with appropriate HTTP code.\nfunc interpretStrategicMergePatchError(err error) error {\n\tswitch err {\n\tcase mergepatch.ErrBadJSONDoc, mergepatch.ErrBadPatchFormatForPrimitiveList, mergepatch.ErrBadPatchFormatForRetainKeys, mergepatch.ErrBadPatchFormatForSetElementOrderList, mergepatch.ErrUnsupportedStrategicMergePatchFormat:\n\t\treturn apierrors.NewBadRequest(err.Error())\n\tcase mergepatch.ErrNoListOfLists, mergepatch.ErrPatchContentNotMatchRetainKeys:\n\t\treturn apierrors.NewGenericServerResponse(http.StatusUnprocessableEntity, \"\", schema.GroupResource{}, \"\", err.Error(), 0, false)\n\tdefault:\n\t\treturn err\n\t}\n}\n" }, { "path": "internal/resourcemodifiers/strategic_merge_patch_test.go", "content": "package resourcemodifiers\n\nimport (\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\tclientgoscheme \"k8s.io/client-go/kubernetes/scheme\"\n)\n\nfunc TestStrategicMergePatchFailure(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdata string\n\t\tkind string\n\t}{\n\t\t{\n\t\t\tname: \"patch with unknown kind\",\n\t\t\tdata: \"{}\",\n\t\t\tkind: \"BadKind\",\n\t\t},\n\t\t{\n\t\t\tname: \"patch with bad yaml\",\n\t\t\tdata: \"a: b:\",\n\t\t\tkind: \"Pod\",\n\t\t},\n\t\t{\n\t\t\tname: \"patch with bad json\",\n\t\t\tdata: `{\"a\"::1}`,\n\t\t\tkind: \"Pod\",\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tscheme := runtime.NewScheme()\n\t\t\terr := clientgoscheme.AddToScheme(scheme)\n\t\t\trequire.NoError(t, err)\n\t\t\tpt := &StrategicMergePatcher{\n\t\t\t\tpatches: []StrategicMergePatch{{PatchData: tt.data}},\n\t\t\t\tscheme: scheme,\n\t\t\t}\n\n\t\t\tu := &unstructured.Unstructured{}\n\t\t\tu.SetGroupVersionKind(schema.GroupVersionKind{Version: \"v1\", Kind: tt.kind})\n\t\t\t_, err = pt.Patch(u, logrus.New())\n\t\t\tassert.Error(t, err)\n\t\t})\n\t}\n}\n" }, { "path": "internal/resourcepolicies/resource_policies.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\thttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage resourcepolicies\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strings\"\n\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tcrclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\ntype VolumeActionType string\n\nconst (\n\t// currently only support configmap type of resource config\n\tConfigmapRefType string = \"configmap\"\n\t// skip action implies the volume would be skipped from the backup operation\n\tSkip VolumeActionType = \"skip\"\n\t// fs-backup action implies that the volume would be backed up via file system copy method using the uploader(kopia/restic) configured by the user\n\tFSBackup VolumeActionType = \"fs-backup\"\n\t// snapshot action can have 3 different meaning based on velero configuration and backup spec - cloud provider based snapshots, local csi snapshots and datamover snapshots\n\tSnapshot VolumeActionType = \"snapshot\"\n)\n\n// Action defined as one action for a specific way of backup\ntype Action struct {\n\t// Type defined specific type of action, currently only support 'skip'\n\tType VolumeActionType `yaml:\"type\"`\n\t// Parameters defined map of parameters when executing a specific action\n\tParameters map[string]any `yaml:\"parameters,omitempty\"`\n}\n\n// IncludeExcludePolicy defined policy to include or exclude resources based on the names\ntype IncludeExcludePolicy struct {\n\t// The following fields have the same semantics as those from the spec of backup.\n\t// Refer to the comment in the velerov1api.BackupSpec for more details.\n\tIncludedClusterScopedResources []string `yaml:\"includedClusterScopedResources\"`\n\tExcludedClusterScopedResources []string `yaml:\"excludedClusterScopedResources\"`\n\tIncludedNamespaceScopedResources []string `yaml:\"includedNamespaceScopedResources\"`\n\tExcludedNamespaceScopedResources []string `yaml:\"excludedNamespaceScopedResources\"`\n}\n\nfunc (p *IncludeExcludePolicy) Validate() error {\n\tif err := p.validateIncludeExclude(p.IncludedClusterScopedResources, p.ExcludedClusterScopedResources); err != nil {\n\t\treturn err\n\t}\n\treturn p.validateIncludeExclude(p.IncludedNamespaceScopedResources, p.ExcludedNamespaceScopedResources)\n}\n\nfunc (p *IncludeExcludePolicy) validateIncludeExclude(includesList, excludesList []string) error {\n\tincludes := sets.NewString(includesList...)\n\texcludes := sets.NewString(excludesList...)\n\n\tif includes.Has(\"*\") || excludes.Has(\"*\") {\n\t\treturn fmt.Errorf(\"cannot use '*' in includes or excludes filters in the policy\")\n\t}\n\tfor _, itm := range excludes.List() {\n\t\tif includes.Has(itm) {\n\t\t\treturn fmt.Errorf(\"excludes list cannot contain an item in the includes list: %s\", itm)\n\t\t}\n\t}\n\treturn nil\n}\n\n// VolumePolicy defined policy to conditions to match Volumes and related action to handle matched Volumes\ntype VolumePolicy struct {\n\t// Conditions defined list of conditions to match Volumes\n\tConditions map[string]any `yaml:\"conditions\"`\n\tAction Action `yaml:\"action\"`\n}\n\n// ResourcePolicies currently defined slice of volume policies to handle backup\ntype ResourcePolicies struct {\n\tVersion string `yaml:\"version\"`\n\tVolumePolicies []VolumePolicy `yaml:\"volumePolicies\"`\n\tIncludeExcludePolicy *IncludeExcludePolicy `yaml:\"includeExcludePolicy\"`\n\t// we may support other resource policies in the future, and they could be added separately\n\t// OtherResourcePolicies []OtherResourcePolicy\n}\n\ntype Policies struct {\n\tversion string\n\tvolumePolicies []volPolicy\n\tincludeExcludePolicy *IncludeExcludePolicy\n\t// OtherPolicies\n}\n\nfunc unmarshalResourcePolicies(yamlData *string) (*ResourcePolicies, error) {\n\tresPolicies := &ResourcePolicies{}\n\terr := decodeStruct(strings.NewReader(*yamlData), resPolicies)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to decode yaml data into resource policies %v\", err)\n\t}\n\n\tfor _, vp := range resPolicies.VolumePolicies {\n\t\tif raw, ok := vp.Conditions[\"pvcLabels\"]; ok {\n\t\t\tswitch raw.(type) {\n\t\t\tcase map[string]any, map[string]string:\n\t\t\tdefault:\n\t\t\t\treturn nil, fmt.Errorf(\"pvcLabels must be a map of string to string, got %T\", raw)\n\t\t\t}\n\t\t}\n\t}\n\treturn resPolicies, nil\n}\n\nfunc (p *Policies) BuildPolicy(resPolicies *ResourcePolicies) error {\n\tfor _, vp := range resPolicies.VolumePolicies {\n\t\tcon, err := unmarshalVolConditions(vp.Conditions)\n\t\tif err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\t\tvolCap, err := parseCapacity(con.Capacity)\n\t\tif err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\t\tvar volP volPolicy\n\t\tvolP.action = vp.Action\n\t\tvolP.conditions = append(volP.conditions, &capacityCondition{capacity: *volCap})\n\t\tvolP.conditions = append(volP.conditions, &storageClassCondition{storageClass: con.StorageClass})\n\t\tvolP.conditions = append(volP.conditions, &nfsCondition{nfs: con.NFS})\n\t\tvolP.conditions = append(volP.conditions, &csiCondition{csi: con.CSI})\n\t\tvolP.conditions = append(volP.conditions, &volumeTypeCondition{volumeTypes: con.VolumeTypes})\n\t\tif len(con.PVCLabels) > 0 {\n\t\t\tvolP.conditions = append(volP.conditions, &pvcLabelsCondition{labels: con.PVCLabels})\n\t\t}\n\t\tif len(con.PVCPhase) > 0 {\n\t\t\tvolP.conditions = append(volP.conditions, &pvcPhaseCondition{phases: con.PVCPhase})\n\t\t}\n\t\tp.volumePolicies = append(p.volumePolicies, volP)\n\t}\n\n\t// Other resource policies\n\n\tp.version = resPolicies.Version\n\tp.includeExcludePolicy = resPolicies.IncludeExcludePolicy\n\treturn nil\n}\n\nfunc (p *Policies) match(res *structuredVolume) *Action {\n\tfor _, policy := range p.volumePolicies {\n\t\tisAllMatch := false\n\t\tfor _, con := range policy.conditions {\n\t\t\tif !con.match(res) {\n\t\t\t\tisAllMatch = false\n\t\t\t\tbreak\n\t\t\t}\n\t\t\tisAllMatch = true\n\t\t}\n\t\tif isAllMatch {\n\t\t\treturn &policy.action\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc (p *Policies) GetMatchAction(res any) (*Action, error) {\n\tdata, ok := res.(VolumeFilterData)\n\tif !ok {\n\t\treturn nil, errors.New(\"failed to convert input to VolumeFilterData\")\n\t}\n\n\tvolume := &structuredVolume{}\n\tswitch {\n\tcase data.PersistentVolume != nil:\n\t\tvolume.parsePV(data.PersistentVolume)\n\t\tif data.PVC != nil {\n\t\t\tvolume.parsePVC(data.PVC)\n\t\t}\n\tcase data.PodVolume != nil:\n\t\tvolume.parsePodVolume(data.PodVolume)\n\t\tif data.PVC != nil {\n\t\t\tvolume.parsePVC(data.PVC)\n\t\t}\n\tcase data.PVC != nil:\n\t\t// Handle PVC-only scenarios (e.g., unbound PVCs)\n\t\tvolume.parsePVC(data.PVC)\n\tdefault:\n\t\treturn nil, errors.New(\"failed to convert object\")\n\t}\n\n\treturn p.match(volume), nil\n}\n\nfunc (p *Policies) Validate() error {\n\tif p.version != currentSupportDataVersion {\n\t\treturn fmt.Errorf(\"incompatible version number %s with supported version %s\", p.version, currentSupportDataVersion)\n\t}\n\n\tfor _, policy := range p.volumePolicies {\n\t\tif err := policy.action.validate(); err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\t\tfor _, con := range policy.conditions {\n\t\t\tif err := con.validate(); err != nil {\n\t\t\t\treturn errors.WithStack(err)\n\t\t\t}\n\t\t}\n\t}\n\n\tif p.GetIncludeExcludePolicy() != nil {\n\t\tif err := p.GetIncludeExcludePolicy().Validate(); err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (p *Policies) GetIncludeExcludePolicy() *IncludeExcludePolicy {\n\treturn p.includeExcludePolicy\n}\n\nfunc GetResourcePoliciesFromBackup(\n\tbackup velerov1api.Backup,\n\tclient crclient.Client,\n\tlogger logrus.FieldLogger,\n) (resourcePolicies *Policies, err error) {\n\tif backup.Spec.ResourcePolicy != nil &&\n\t\tstrings.EqualFold(backup.Spec.ResourcePolicy.Kind, ConfigmapRefType) {\n\t\tpoliciesConfigMap := &corev1api.ConfigMap{}\n\t\terr = client.Get(\n\t\t\tcontext.Background(),\n\t\t\tcrclient.ObjectKey{Namespace: backup.Namespace, Name: backup.Spec.ResourcePolicy.Name},\n\t\t\tpoliciesConfigMap,\n\t\t)\n\t\tif err != nil {\n\t\t\tlogger.Errorf(\"Fail to get ResourcePolicies %s ConfigMap with error %s.\",\n\t\t\t\tbackup.Namespace+\"/\"+backup.Spec.ResourcePolicy.Name, err.Error())\n\t\t\treturn nil, fmt.Errorf(\"fail to get ResourcePolicies %s ConfigMap with error %s\",\n\t\t\t\tbackup.Namespace+\"/\"+backup.Spec.ResourcePolicy.Name, err.Error())\n\t\t}\n\t\tresourcePolicies, err = getResourcePoliciesFromConfig(policiesConfigMap)\n\t\tif err != nil {\n\t\t\tlogger.Errorf(\"Fail to read ResourcePolicies from ConfigMap %s with error %s.\",\n\t\t\t\tbackup.Namespace+\"/\"+backup.Name, err.Error())\n\t\t\treturn nil, fmt.Errorf(\"fail to read the ResourcePolicies from ConfigMap %s with error %s\",\n\t\t\t\tbackup.Namespace+\"/\"+backup.Name, err.Error())\n\t\t} else if err = resourcePolicies.Validate(); err != nil {\n\t\t\tlogger.Errorf(\"Fail to validate ResourcePolicies in ConfigMap %s with error %s.\",\n\t\t\t\tbackup.Namespace+\"/\"+backup.Name, err.Error())\n\t\t\treturn nil, fmt.Errorf(\"fail to validate ResourcePolicies in ConfigMap %s with error %s\",\n\t\t\t\tbackup.Namespace+\"/\"+backup.Name, err.Error())\n\t\t}\n\t}\n\n\treturn resourcePolicies, nil\n}\n\nfunc getResourcePoliciesFromConfig(cm *corev1api.ConfigMap) (*Policies, error) {\n\tif cm == nil {\n\t\treturn nil, fmt.Errorf(\"could not parse config from nil configmap\")\n\t}\n\tif len(cm.Data) != 1 {\n\t\treturn nil, fmt.Errorf(\"illegal resource policies %s/%s configmap\", cm.Namespace, cm.Name)\n\t}\n\n\tvar yamlData string\n\tfor _, v := range cm.Data {\n\t\tyamlData = v\n\t}\n\n\tresPolicies, err := unmarshalResourcePolicies(&yamlData)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tpolicies := &Policies{}\n\tif err := policies.BuildPolicy(resPolicies); err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\treturn policies, nil\n}\n" }, { "path": "internal/resourcepolicies/resource_policies_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\thttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage resourcepolicies\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/api/resource\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\nfunc TestLoadResourcePolicies(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tyamlData string\n\t\twantErr bool\n\t}{\n\t\t{\n\t\t\tname: \"unknown key in yaml\",\n\t\t\tyamlData: `version: v1\n\tvolumePolicies:\n\t- conditions:\n\t\tcapacity: \"0,100Gi\"\n\t\tunknown: {}\n\t\tstorageClass:\n\t\t- gp2\n\t\t- ebs-sc\n\t action:\n\t\ttype: skip`,\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"reduplicated key in yaml\",\n\t\t\tyamlData: `version: v1\n\tvolumePolicies:\n\t- conditions:\n\t\tcapacity: \"0,100Gi\"\n\t\tcapacity: \"0,100Gi\"\n\t\tstorageClass:\n\t\t- gp2\n\t\t- ebs-sc\n\t action:\n\t\ttype: skip`,\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"error format of storageClass\",\n\t\t\tyamlData: `version: v1\n\tvolumePolicies:\n\t- conditions:\n\t\tcapacity: \"0,100Gi\"\n\t\tstorageClass: gp2\n\t action:\n\t\ttype: skip`,\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"error format of csi\",\n\t\t\tyamlData: `version: v1\n\tvolumePolicies:\n\t- conditions:\n\t\tcapacity: \"0,100Gi\"\n\t\tcsi: gp2\n\t action:\n\t\ttype: skip`,\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"error format of nfs\",\n\t\t\tyamlData: `version: v1\n\tvolumePolicies:\n\t- conditions:\n\t\tcapacity: \"0,100Gi\"\n\t\tcsi: {}\n\t\tnfs: abc\n\t action:\n\t\ttype: skip`,\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"supported format volume policies\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n - conditions:\n capacity: '0,100Gi'\n csi:\n driver: aws.efs.csi.driver\n action:\n type: skip\n`,\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"supported format csi driver with volumeAttributes for volume policies\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n - conditions:\n capacity: '0,100Gi'\n csi:\n driver: aws.efs.csi.driver\n volumeAttributes:\n key1: value1\n action:\n type: skip\n`,\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"supported format pvcLabels\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n - conditions:\n pvcLabels:\n environment: production\n app: database\n action:\n type: skip\n`,\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"error format of pvcLabels (not a map)\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n - conditions:\n pvcLabels: \"production\"\n action:\n type: skip\n`,\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"supported format pvcLabels with extra keys\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n - conditions:\n pvcLabels:\n environment: production\n region: us-west\n action:\n type: skip\n`,\n\t\t\twantErr: false,\n\t\t},\n\t}\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t_, err := unmarshalResourcePolicies(&tc.yamlData)\n\n\t\t\tif (err != nil) != tc.wantErr {\n\t\t\t\tt.Fatalf(\"Expected error %v, but got error %v\", tc.wantErr, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetResourceMatchedAction(t *testing.T) {\n\tresPolicies := &ResourcePolicies{\n\t\tVersion: \"v1\",\n\t\tVolumePolicies: []VolumePolicy{\n\t\t\t{\n\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\"capacity\": \"0,10Gi\",\n\t\t\t\t\t\"storageClass\": []string{\"gp2\", \"ebs-sc\"},\n\t\t\t\t\t\"csi\": any(\n\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\"driver\": \"aws.efs.csi.driver\",\n\t\t\t\t\t\t}),\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\"csi\": any(\n\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\"driver\": \"files.csi.driver\",\n\t\t\t\t\t\t\t\"volumeAttributes\": map[string]string{\"protocol\": \"nfs\"},\n\t\t\t\t\t\t}),\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tAction: Action{Type: \"snapshot\"},\n\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\"capacity\": \"10,100Gi\",\n\t\t\t\t\t\"storageClass\": []string{\"gp2\", \"ebs-sc\"},\n\t\t\t\t\t\"csi\": any(\n\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\"driver\": \"aws.efs.csi.driver\",\n\t\t\t\t\t\t}),\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tAction: Action{Type: \"fs-backup\"},\n\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\"storageClass\": []string{\"gp2\", \"ebs-sc\"},\n\t\t\t\t\t\"csi\": any(\n\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\"driver\": \"aws.efs.csi.driver\",\n\t\t\t\t\t\t}),\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tAction: Action{Type: \"snapshot\"},\n\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\"pvcLabels\": map[string]string{\n\t\t\t\t\t\t\"environment\": \"production\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\ttestCases := []struct {\n\t\tname string\n\t\tvolume *structuredVolume\n\t\texpectedAction *Action\n\t\tresourcePolicies *ResourcePolicies\n\t}{\n\t\t{\n\t\t\tname: \"match policy\",\n\t\t\tvolume: &structuredVolume{\n\t\t\t\tcapacity: *resource.NewQuantity(5<<30, resource.BinarySI),\n\t\t\t\tstorageClass: \"ebs-sc\",\n\t\t\t\tcsi: &csiVolumeSource{Driver: \"aws.efs.csi.driver\"},\n\t\t\t},\n\t\t\texpectedAction: &Action{Type: \"skip\"},\n\t\t},\n\t\t{\n\t\t\tname: \"match policy AFS NFS\",\n\t\t\tvolume: &structuredVolume{\n\t\t\t\tcapacity: *resource.NewQuantity(5<<30, resource.BinarySI),\n\t\t\t\tstorageClass: \"afs-nfs\",\n\t\t\t\tcsi: &csiVolumeSource{Driver: \"files.csi.driver\", VolumeAttributes: map[string]string{\"protocol\": \"nfs\"}},\n\t\t\t},\n\t\t\texpectedAction: &Action{Type: \"skip\"},\n\t\t},\n\t\t{\n\t\t\tname: \"match policy AFS SMB\",\n\t\t\tvolume: &structuredVolume{\n\t\t\t\tcapacity: *resource.NewQuantity(5<<30, resource.BinarySI),\n\t\t\t\tstorageClass: \"afs-smb\",\n\t\t\t\tcsi: &csiVolumeSource{Driver: \"files.csi.driver\"},\n\t\t\t},\n\t\t\texpectedAction: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"both matches return the first policy\",\n\t\t\tvolume: &structuredVolume{\n\t\t\t\tcapacity: *resource.NewQuantity(50<<30, resource.BinarySI),\n\t\t\t\tstorageClass: \"ebs-sc\",\n\t\t\t\tcsi: &csiVolumeSource{Driver: \"aws.efs.csi.driver\"},\n\t\t\t},\n\t\t\texpectedAction: &Action{Type: \"snapshot\"},\n\t\t},\n\t\t{\n\t\t\tname: \"mismatch all policies\",\n\t\t\tvolume: &structuredVolume{\n\t\t\t\tcapacity: *resource.NewQuantity(50<<30, resource.BinarySI),\n\t\t\t\tstorageClass: \"ebs-sc\",\n\t\t\t\tnfs: &nFSVolumeSource{},\n\t\t\t},\n\t\t\texpectedAction: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"match pvcLabels condition\",\n\t\t\tvolume: &structuredVolume{\n\t\t\t\tcapacity: *resource.NewQuantity(5<<30, resource.BinarySI),\n\t\t\t\tstorageClass: \"some-class\",\n\t\t\t\tpvcLabels: map[string]string{\n\t\t\t\t\t\"environment\": \"production\",\n\t\t\t\t\t\"team\": \"backend\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedAction: &Action{Type: \"snapshot\"},\n\t\t},\n\t\t{\n\t\t\tname: \"mismatch pvcLabels condition\",\n\t\t\tvolume: &structuredVolume{\n\t\t\t\tcapacity: *resource.NewQuantity(5<<30, resource.BinarySI),\n\t\t\t\tstorageClass: \"some-class\",\n\t\t\t\tpvcLabels: map[string]string{\n\t\t\t\t\t\"environment\": \"staging\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedAction: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"nil condition always match the action\",\n\t\t\tvolume: &structuredVolume{\n\t\t\t\tcapacity: *resource.NewQuantity(5<<30, resource.BinarySI),\n\t\t\t\tstorageClass: \"some-class\",\n\t\t\t\tpvcLabels: map[string]string{\n\t\t\t\t\t\"environment\": \"staging\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tresourcePolicies: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\t\t\tConditions: map[string]any{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedAction: &Action{Type: \"skip\"},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tpolicies := &Policies{}\n\t\t\tcurrentResourcePolicy := resPolicies\n\t\t\tif tc.resourcePolicies != nil {\n\t\t\t\tcurrentResourcePolicy = tc.resourcePolicies\n\t\t\t}\n\t\t\terr := policies.BuildPolicy(currentResourcePolicy)\n\t\t\tif err != nil {\n\t\t\t\tt.Errorf(\"Failed to build policy with error %v\", err)\n\t\t\t}\n\n\t\t\taction := policies.match(tc.volume)\n\t\t\tif action == nil {\n\t\t\t\tif tc.expectedAction != nil {\n\t\t\t\t\tt.Errorf(\"Expected action %v, but got result nil\", tc.expectedAction.Type)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif tc.expectedAction != nil {\n\t\t\t\t\tif action.Type != tc.expectedAction.Type {\n\t\t\t\t\t\tt.Errorf(\"Expected action %v, but got result %v\", tc.expectedAction.Type, action.Type)\n\t\t\t\t\t}\n\t\t\t\t} else {\n\t\t\t\t\tt.Errorf(\"Expected action nil, but got result %v\", action.Type)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetResourcePoliciesFromConfig(t *testing.T) {\n\t// Create a test ConfigMap\n\tcm := &corev1api.ConfigMap{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-configmap\",\n\t\t\tNamespace: \"test-namespace\",\n\t\t},\n\t\tData: map[string]string{\n\t\t\t\"test-data\": `version: v1\nvolumePolicies:\n - conditions:\n capacity: '0,10Gi'\n csi:\n driver: disks.csi.driver\n action:\n type: skip\n - conditions:\n csi:\n driver: files.csi.driver\n volumeAttributes:\n protocol: nfs\n action:\n type: skip\n - conditions:\n pvcLabels:\n environment: production\n action:\n type: skip\n`,\n\t\t},\n\t}\n\n\t// Call the function and check for errors\n\tresPolicies, err := getResourcePoliciesFromConfig(cm)\n\trequire.NoError(t, err)\n\n\t// Check that the returned resourcePolicies object contains the expected data\n\tassert.Equal(t, \"v1\", resPolicies.version)\n\n\tassert.Len(t, resPolicies.volumePolicies, 3)\n\n\tpolicies := ResourcePolicies{\n\t\tVersion: \"v1\",\n\t\tVolumePolicies: []VolumePolicy{\n\t\t\t{\n\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\"capacity\": \"0,10Gi\",\n\t\t\t\t\t\"csi\": map[string]any{\n\t\t\t\t\t\t\"driver\": \"disks.csi.driver\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tAction: Action{\n\t\t\t\t\tType: Skip,\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\"csi\": map[string]any{\n\t\t\t\t\t\t\"driver\": \"files.csi.driver\",\n\t\t\t\t\t\t\"volumeAttributes\": map[string]string{\"protocol\": \"nfs\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tAction: Action{\n\t\t\t\t\tType: Skip,\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\"pvcLabels\": map[string]string{\n\t\t\t\t\t\t\"environment\": \"production\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tAction: Action{\n\t\t\t\t\tType: Skip,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tp := &Policies{}\n\terr = p.BuildPolicy(&policies)\n\tif err != nil {\n\t\tt.Fatalf(\"failed to build policy: %v\", err)\n\t}\n\n\tassert.Equal(t, p, resPolicies)\n}\n\nfunc TestGetMatchAction(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tyamlData string\n\t\tvol *corev1api.PersistentVolume\n\t\tpodVol *corev1api.Volume\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\tskip bool\n\t}{\n\t\t{\n\t\t\tname: \"empty csi\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n- conditions:\n csi: {}\n action:\n type: skip`,\n\t\t\tvol: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{Driver: \"ebs.csi.aws.com\"},\n\t\t\t\t\t}},\n\t\t\t},\n\t\t\tskip: true,\n\t\t},\n\t\t{\n\t\t\tname: \"empty csi with pv no csi driver\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n- conditions:\n csi: {}\n action:\n type: skip`,\n\t\t\tvol: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tCapacity: corev1api.ResourceList{\n\t\t\t\t\t\tcorev1api.ResourceStorage: resource.MustParse(\"1Gi\"),\n\t\t\t\t\t}},\n\t\t\t},\n\t\t\tskip: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Skip AFS CSI condition with Disk volumes\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n - conditions:\n csi:\n driver: files.csi.driver\n action:\n type: skip`,\n\t\t\tvol: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{Driver: \"disks.csi.driver\"},\n\t\t\t\t\t}},\n\t\t\t},\n\t\t\tskip: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Skip AFS CSI condition with AFS volumes\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n - conditions:\n csi:\n driver: files.csi.driver\n action:\n type: skip`,\n\t\t\tvol: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{Driver: \"files.csi.driver\"},\n\t\t\t\t\t}},\n\t\t\t},\n\t\t\tskip: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Skip AFS NFS CSI condition with Disk volumes\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n - conditions:\n csi:\n driver: files.csi.driver\n volumeAttributes:\n protocol: nfs\n action:\n type: skip\n`,\n\t\t\tvol: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{Driver: \"disks.csi.driver\"},\n\t\t\t\t\t}},\n\t\t\t},\n\t\t\tskip: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Skip AFS NFS CSI condition with AFS SMB volumes\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n - conditions:\n csi:\n driver: files.csi.driver\n volumeAttributes:\n protocol: nfs\n action:\n type: skip\n`,\n\t\t\tvol: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{Driver: \"files.csi.driver\", VolumeAttributes: map[string]string{\"key1\": \"val1\"}},\n\t\t\t\t\t}},\n\t\t\t},\n\t\t\tskip: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Skip AFS NFS CSI condition with AFS NFS volumes\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n - conditions:\n csi:\n driver: files.csi.driver\n volumeAttributes:\n protocol: nfs\n action:\n type: skip\n`,\n\t\t\tvol: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{Driver: \"files.csi.driver\", VolumeAttributes: map[string]string{\"protocol\": \"nfs\"}},\n\t\t\t\t\t}},\n\t\t\t},\n\t\t\tskip: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Skip Disk and AFS NFS CSI condition with Disk volumes\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n - conditions:\n csi:\n driver: disks.csi.driver\n action:\n type: skip\n - conditions:\n csi:\n driver: files.csi.driver\n volumeAttributes:\n protocol: nfs\n action:\n type: skip`,\n\t\t\tvol: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{Driver: \"disks.csi.driver\", VolumeAttributes: map[string]string{\"key1\": \"val1\"}},\n\t\t\t\t\t}},\n\t\t\t},\n\t\t\tskip: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Skip Disk and AFS NFS CSI condition with AFS SMB volumes\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n - conditions:\n csi:\n driver: disks.csi.driver\n action:\n type: skip\n - conditions:\n csi:\n driver: files.csi.driver\n volumeAttributes:\n protocol: nfs\n action:\n type: skip`,\n\t\t\tvol: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{Driver: \"files.csi.driver\", VolumeAttributes: map[string]string{\"key1\": \"val1\"}},\n\t\t\t\t\t}},\n\t\t\t},\n\t\t\tskip: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Skip Disk and AFS NFS CSI condition with AFS NFS volumes\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n - conditions:\n csi:\n driver: disks.csi.driver\n action:\n type: skip\n - conditions:\n csi:\n driver: files.csi.driver\n volumeAttributes:\n protocol: nfs\n action:\n type: skip`,\n\t\t\tvol: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{Driver: \"files.csi.driver\", VolumeAttributes: map[string]string{\"key1\": \"val1\", \"protocol\": \"nfs\"}},\n\t\t\t\t\t}},\n\t\t\t},\n\t\t\tskip: true,\n\t\t},\n\t\t{\n\t\t\tname: \"csi not configured and testing capacity condition\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n- conditions:\n capacity: \"0,100Gi\"\n action:\n type: skip`,\n\t\t\tvol: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tCapacity: corev1api.ResourceList{\n\t\t\t\t\t\tcorev1api.ResourceStorage: resource.MustParse(\"1Gi\"),\n\t\t\t\t\t},\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{Driver: \"ebs.csi.aws.com\"},\n\t\t\t\t\t}},\n\t\t\t},\n\t\t\tskip: true,\n\t\t},\n\t\t{\n\t\t\tname: \"empty nfs\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n- conditions:\n nfs: {}\n action:\n type: skip`,\n\t\t\tvol: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tNFS: &corev1api.NFSVolumeSource{Server: \"192.168.1.20\"},\n\t\t\t\t\t}},\n\t\t\t},\n\t\t\tskip: true,\n\t\t},\n\t\t{\n\t\t\tname: \"nfs not configured\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n- conditions:\n capacity: \"0,100Gi\"\n action:\n type: skip`,\n\t\t\tvol: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tCapacity: corev1api.ResourceList{\n\t\t\t\t\t\tcorev1api.ResourceStorage: resource.MustParse(\"1Gi\"),\n\t\t\t\t\t},\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tNFS: &corev1api.NFSVolumeSource{Server: \"192.168.1.20\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tskip: true,\n\t\t},\n\t\t{\n\t\t\tname: \"empty nfs with pv no nfs volume source\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n- conditions:\n capacity: \"0,100Gi\"\n nfs: {}\n action:\n type: skip`,\n\t\t\tvol: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tCapacity: corev1api.ResourceList{\n\t\t\t\t\t\tcorev1api.ResourceStorage: resource.MustParse(\"1Gi\"),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tskip: false,\n\t\t},\n\t\t{\n\t\t\tname: \"match volume by types\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n- conditions:\n capacity: \"0,100Gi\"\n volumeTypes:\n - local\n - hostPath\n action:\n type: skip`,\n\t\t\tvol: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tCapacity: corev1api.ResourceList{\n\t\t\t\t\t\tcorev1api.ResourceStorage: resource.MustParse(\"1Gi\"),\n\t\t\t\t\t},\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tHostPath: &corev1api.HostPathVolumeSource{Path: \"/mnt/data\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tskip: true,\n\t\t},\n\t\t{\n\t\t\tname: \"mismatch volume by types\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n- conditions:\n capacity: \"0,100Gi\"\n volumeTypes:\n - local\n action:\n type: skip`,\n\t\t\tvol: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tCapacity: corev1api.ResourceList{\n\t\t\t\t\t\tcorev1api.ResourceStorage: resource.MustParse(\"1Gi\"),\n\t\t\t\t\t},\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tHostPath: &corev1api.HostPathVolumeSource{Path: \"/mnt/data\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tskip: false,\n\t\t},\n\t\t{\n\t\t\tname: \"PVC labels match\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n- conditions:\n capacity: \"0,100Gi\"\n pvcLabels:\n environment: production\n action:\n type: skip`,\n\t\t\tvol: &corev1api.PersistentVolume{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"pv-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tCapacity: corev1api.ResourceList{\n\t\t\t\t\t\tcorev1api.ResourceStorage: resource.MustParse(\"1Gi\"),\n\t\t\t\t\t},\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{},\n\t\t\t\t\tClaimRef: &corev1api.ObjectReference{\n\t\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\t\tName: \"pvc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\tName: \"pvc-1\",\n\t\t\t\t\tLabels: map[string]string{\"environment\": \"production\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\tskip: true,\n\t\t},\n\t\t{\n\t\t\tname: \"PVC labels match, criteria label is a subset of the pvc labels\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n- conditions:\n capacity: \"0,100Gi\"\n pvcLabels:\n environment: production\n action:\n type: skip`,\n\t\t\tvol: &corev1api.PersistentVolume{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"pv-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tCapacity: corev1api.ResourceList{\n\t\t\t\t\t\tcorev1api.ResourceStorage: resource.MustParse(\"1Gi\"),\n\t\t\t\t\t},\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{},\n\t\t\t\t\tClaimRef: &corev1api.ObjectReference{\n\t\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\t\tName: \"pvc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\tName: \"pvc-1\",\n\t\t\t\t\tLabels: map[string]string{\"environment\": \"production\", \"app\": \"backend\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\tskip: true,\n\t\t},\n\t\t{\n\t\t\tname: \"PVC labels match don't match exactly\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n- conditions:\n capacity: \"0,100Gi\"\n pvcLabels:\n environment: production\n app: frontend\n action:\n type: skip`,\n\t\t\tvol: &corev1api.PersistentVolume{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"pv-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tCapacity: corev1api.ResourceList{\n\t\t\t\t\t\tcorev1api.ResourceStorage: resource.MustParse(\"1Gi\"),\n\t\t\t\t\t},\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{},\n\t\t\t\t\tClaimRef: &corev1api.ObjectReference{\n\t\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\t\tName: \"pvc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\tName: \"pvc-1\",\n\t\t\t\t\tLabels: map[string]string{\"environment\": \"production\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\tskip: false,\n\t\t},\n\t\t{\n\t\t\tname: \"PVC labels mismatch\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n- conditions:\n capacity: \"0,100Gi\"\n pvcLabels:\n environment: production\n action:\n type: skip`,\n\t\t\tvol: &corev1api.PersistentVolume{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"pv-2\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tCapacity: corev1api.ResourceList{\n\t\t\t\t\t\tcorev1api.ResourceStorage: resource.MustParse(\"1Gi\"),\n\t\t\t\t\t},\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{},\n\t\t\t\t\tClaimRef: &corev1api.ObjectReference{\n\t\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\t\tName: \"pvc-2\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\tName: \"pvc-1\",\n\t\t\t\t\tLabels: map[string]string{\"environment\": \"staging\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\tskip: false,\n\t\t},\n\t\t{\n\t\t\tname: \"PodVolume case with PVC labels match\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n- conditions:\n pvcLabels:\n environment: production\n action:\n type: skip`,\n\t\t\tvol: nil,\n\t\t\tpodVol: &corev1api.Volume{Name: \"pod-vol-1\"},\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\tName: \"pvc-1\",\n\t\t\t\t\tLabels: map[string]string{\"environment\": \"production\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\tskip: true,\n\t\t},\n\t\t{\n\t\t\tname: \"PodVolume case with PVC labels mismatch\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n- conditions:\n pvcLabels:\n environment: production\n action:\n type: skip`,\n\t\t\tvol: nil,\n\t\t\tpodVol: &corev1api.Volume{Name: \"pod-vol-2\"},\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\tName: \"pvc-2\",\n\t\t\t\t\tLabels: map[string]string{\"environment\": \"staging\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\tskip: false,\n\t\t},\n\t\t{\n\t\t\tname: \"PodVolume case with PVC labels match with extra keys on PVC\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n- conditions:\n pvcLabels:\n environment: production\n action:\n type: skip`,\n\t\t\tvol: nil,\n\t\t\tpodVol: &corev1api.Volume{Name: \"pod-vol-3\"},\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\tName: \"pvc-3\",\n\t\t\t\t\tLabels: map[string]string{\"environment\": \"production\", \"app\": \"backend\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\tskip: true,\n\t\t},\n\t\t{\n\t\t\tname: \"PodVolume case with PVC labels don't match exactly\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n- conditions:\n pvcLabels:\n environment: production\n app: frontend\n action:\n type: skip`,\n\t\t\tvol: nil,\n\t\t\tpodVol: &corev1api.Volume{Name: \"pod-vol-4\"},\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\tName: \"pvc-4\",\n\t\t\t\t\tLabels: map[string]string{\"environment\": \"production\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\tskip: false,\n\t\t},\n\t\t{\n\t\t\tname: \"PVC phase matching - Pending phase should skip\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n- conditions:\n pvcPhase: [\"Pending\"]\n action:\n type: skip`,\n\t\t\tvol: nil,\n\t\t\tpodVol: nil,\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\tName: \"pvc-pending\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\t\t\tPhase: corev1api.ClaimPending,\n\t\t\t\t},\n\t\t\t},\n\t\t\tskip: true,\n\t\t},\n\t\t{\n\t\t\tname: \"PVC phase matching - Bound phase should not skip\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n- conditions:\n pvcPhase: [\"Pending\"]\n action:\n type: skip`,\n\t\t\tvol: nil,\n\t\t\tpodVol: nil,\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\tName: \"pvc-bound\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\t\t\tPhase: corev1api.ClaimBound,\n\t\t\t\t},\n\t\t\t},\n\t\t\tskip: false,\n\t\t},\n\t\t{\n\t\t\tname: \"PVC phase matching - Multiple phases (Pending, Lost)\",\n\t\t\tyamlData: `version: v1\nvolumePolicies:\n- conditions:\n pvcPhase: [\"Pending\", \"Lost\"]\n action:\n type: skip`,\n\t\t\tvol: nil,\n\t\t\tpodVol: nil,\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\tName: \"pvc-lost\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\t\t\tPhase: corev1api.ClaimLost,\n\t\t\t\t},\n\t\t\t},\n\t\t\tskip: true,\n\t\t},\n\t}\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tresPolicies, err := unmarshalResourcePolicies(&tc.yamlData)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"got error when get match action %v\", err)\n\t\t\t}\n\t\t\trequire.NoError(t, err)\n\t\t\tpolicies := &Policies{}\n\t\t\terr = policies.BuildPolicy(resPolicies)\n\t\t\trequire.NoError(t, err)\n\t\t\tvfd := VolumeFilterData{}\n\t\t\tif tc.pvc != nil {\n\t\t\t\tvfd.PVC = tc.pvc\n\t\t\t}\n\n\t\t\tif tc.vol != nil {\n\t\t\t\tvfd.PersistentVolume = tc.vol\n\t\t\t}\n\n\t\t\tif tc.podVol != nil {\n\t\t\t\tvfd.PodVolume = tc.podVol\n\t\t\t}\n\n\t\t\taction, err := policies.GetMatchAction(vfd)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tif tc.skip {\n\t\t\t\tif action.Type != Skip {\n\t\t\t\t\tt.Fatalf(\"Expected action skip but is %v\", action.Type)\n\t\t\t\t}\n\t\t\t} else if action != nil && action.Type == Skip {\n\t\t\t\tt.Fatalf(\"Expected action not skip but is %v\", action.Type)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetMatchAction_Errors(t *testing.T) {\n\tp := &Policies{}\n\n\ttestCases := []struct {\n\t\tname string\n\t\tinput any\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"invalid input type\",\n\t\t\tinput: \"invalid input\",\n\t\t\texpectedErr: \"failed to convert input to VolumeFilterData\",\n\t\t},\n\t\t{\n\t\t\tname: \"no volume provided\",\n\t\t\tinput: VolumeFilterData{\n\t\t\t\tPersistentVolume: nil,\n\t\t\t\tPodVolume: nil,\n\t\t\t\tPVC: nil,\n\t\t\t},\n\t\t\texpectedErr: \"failed to convert object\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\taction, err := p.GetMatchAction(tc.input)\n\t\t\trequire.ErrorContains(t, err, tc.expectedErr)\n\t\t\tassert.Nil(t, action)\n\t\t})\n\t}\n}\n\nfunc TestParsePVC(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\texpectedLabels map[string]string\n\t\texpectedPhase string\n\t\texpectErr bool\n\t}{\n\t\t{\n\t\t\tname: \"valid PVC with labels and Pending phase\",\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tLabels: map[string]string{\"env\": \"prod\"},\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\t\t\tPhase: corev1api.ClaimPending,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedLabels: map[string]string{\"env\": \"prod\"},\n\t\t\texpectedPhase: \"Pending\",\n\t\t\texpectErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"valid PVC with Bound phase\",\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tLabels: map[string]string{},\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\t\t\tPhase: corev1api.ClaimBound,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedLabels: nil,\n\t\t\texpectedPhase: \"Bound\",\n\t\t\texpectErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"valid PVC with Lost phase\",\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\t\t\tPhase: corev1api.ClaimLost,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedLabels: nil,\n\t\t\texpectedPhase: \"Lost\",\n\t\t\texpectErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"nil PVC pointer\",\n\t\t\tpvc: (*corev1api.PersistentVolumeClaim)(nil),\n\t\t\texpectedLabels: nil,\n\t\t\texpectedPhase: \"\",\n\t\t\texpectErr: false,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\ts := &structuredVolume{}\n\t\t\ts.parsePVC(tc.pvc)\n\n\t\t\tassert.Equal(t, tc.expectedLabels, s.pvcLabels)\n\t\t\tassert.Equal(t, tc.expectedPhase, s.pvcPhase)\n\t\t})\n\t}\n}\n\nfunc TestPVCPhaseMatch(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tcondition *pvcPhaseCondition\n\t\tvolume *structuredVolume\n\t\texpectedMatch bool\n\t}{\n\t\t{\n\t\t\tname: \"match Pending phase\",\n\t\t\tcondition: &pvcPhaseCondition{phases: []string{\"Pending\"}},\n\t\t\tvolume: &structuredVolume{pvcPhase: \"Pending\"},\n\t\t\texpectedMatch: true,\n\t\t},\n\t\t{\n\t\t\tname: \"match multiple phases - Pending matches\",\n\t\t\tcondition: &pvcPhaseCondition{phases: []string{\"Pending\", \"Bound\"}},\n\t\t\tvolume: &structuredVolume{pvcPhase: \"Pending\"},\n\t\t\texpectedMatch: true,\n\t\t},\n\t\t{\n\t\t\tname: \"match multiple phases - Bound matches\",\n\t\t\tcondition: &pvcPhaseCondition{phases: []string{\"Pending\", \"Bound\"}},\n\t\t\tvolume: &structuredVolume{pvcPhase: \"Bound\"},\n\t\t\texpectedMatch: true,\n\t\t},\n\t\t{\n\t\t\tname: \"no match for different phase\",\n\t\t\tcondition: &pvcPhaseCondition{phases: []string{\"Pending\"}},\n\t\t\tvolume: &structuredVolume{pvcPhase: \"Bound\"},\n\t\t\texpectedMatch: false,\n\t\t},\n\t\t{\n\t\t\tname: \"no match for empty phase\",\n\t\t\tcondition: &pvcPhaseCondition{phases: []string{\"Pending\"}},\n\t\t\tvolume: &structuredVolume{pvcPhase: \"\"},\n\t\t\texpectedMatch: false,\n\t\t},\n\t\t{\n\t\t\tname: \"match with empty phases list (always match)\",\n\t\t\tcondition: &pvcPhaseCondition{phases: []string{}},\n\t\t\tvolume: &structuredVolume{pvcPhase: \"Pending\"},\n\t\t\texpectedMatch: true,\n\t\t},\n\t\t{\n\t\t\tname: \"match with nil phases list (always match)\",\n\t\t\tcondition: &pvcPhaseCondition{phases: nil},\n\t\t\tvolume: &structuredVolume{pvcPhase: \"Pending\"},\n\t\t\texpectedMatch: true,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tresult := tc.condition.match(tc.volume)\n\t\t\tassert.Equal(t, tc.expectedMatch, result)\n\t\t})\n\t}\n}\n" }, { "path": "internal/resourcepolicies/volume_filter_data.go", "content": "package resourcepolicies\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n)\n\n// VolumeFilterData bundles the volume data needed for volume policy filtering\ntype VolumeFilterData struct {\n\tPersistentVolume *corev1api.PersistentVolume\n\tPodVolume *corev1api.Volume\n\tPVC *corev1api.PersistentVolumeClaim\n}\n\n// NewVolumeFilterData constructs a new VolumeFilterData instance.\nfunc NewVolumeFilterData(pv *corev1api.PersistentVolume, podVol *corev1api.Volume, pvc *corev1api.PersistentVolumeClaim) VolumeFilterData {\n\treturn VolumeFilterData{\n\t\tPersistentVolume: pv,\n\t\tPodVolume: podVol,\n\t\tPVC: pvc,\n\t}\n}\n" }, { "path": "internal/resourcepolicies/volume_filter_data_test.go", "content": "package resourcepolicies\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\nfunc TestNewVolumeFilterData(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tpv *corev1api.PersistentVolume\n\t\tpodVol *corev1api.Volume\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\texpectedPVName string\n\t\texpectedPodName string\n\t\texpectedPVCName string\n\t}{\n\t\t{\n\t\t\tname: \"all provided\",\n\t\t\tpv: &corev1api.PersistentVolume{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"pv-test\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tpodVol: &corev1api.Volume{\n\t\t\t\tName: \"pod-vol-test\",\n\t\t\t},\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"pvc-test\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedPVName: \"pv-test\",\n\t\t\texpectedPodName: \"pod-vol-test\",\n\t\t\texpectedPVCName: \"pvc-test\",\n\t\t},\n\t\t{\n\t\t\tname: \"only PV provided\",\n\t\t\tpv: &corev1api.PersistentVolume{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"pv-only\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tpodVol: nil,\n\t\t\tpvc: nil,\n\t\t\texpectedPVName: \"pv-only\",\n\t\t\texpectedPodName: \"\",\n\t\t\texpectedPVCName: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"only PodVolume provided\",\n\t\t\tpv: nil,\n\t\t\tpodVol: &corev1api.Volume{\n\t\t\t\tName: \"pod-only\",\n\t\t\t},\n\t\t\tpvc: nil,\n\t\t\texpectedPVName: \"\",\n\t\t\texpectedPodName: \"pod-only\",\n\t\t\texpectedPVCName: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"only PVC provided\",\n\t\t\tpv: nil,\n\t\t\tpodVol: nil,\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"pvc-only\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedPVName: \"\",\n\t\t\texpectedPodName: \"\",\n\t\t\texpectedPVCName: \"pvc-only\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvfd := NewVolumeFilterData(tc.pv, tc.podVol, tc.pvc)\n\t\t\tif tc.expectedPVName != \"\" {\n\t\t\t\tassert.NotNil(t, vfd.PersistentVolume)\n\t\t\t\tassert.Equal(t, tc.expectedPVName, vfd.PersistentVolume.Name)\n\t\t\t} else {\n\t\t\t\tassert.Nil(t, vfd.PersistentVolume)\n\t\t\t}\n\t\t\tif tc.expectedPodName != \"\" {\n\t\t\t\tassert.NotNil(t, vfd.PodVolume)\n\t\t\t\tassert.Equal(t, tc.expectedPodName, vfd.PodVolume.Name)\n\t\t\t} else {\n\t\t\t\tassert.Nil(t, vfd.PodVolume)\n\t\t\t}\n\t\t\tif tc.expectedPVCName != \"\" {\n\t\t\t\tassert.NotNil(t, vfd.PVC)\n\t\t\t\tassert.Equal(t, tc.expectedPVCName, vfd.PVC.Name)\n\t\t\t} else {\n\t\t\t\tassert.Nil(t, vfd.PVC)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "internal/resourcepolicies/volume_resources.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\thttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage resourcepolicies\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"strings\"\n\n\t\"k8s.io/apimachinery/pkg/labels\"\n\n\t\"github.com/pkg/errors\"\n\t\"gopkg.in/yaml.v3\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/api/resource\"\n)\n\ntype volPolicy struct {\n\taction Action\n\tconditions []volumeCondition\n}\n\ntype volumeCondition interface {\n\tmatch(v *structuredVolume) bool\n\tvalidate() error\n}\n\n// capacity consist of the lower and upper boundary\ntype capacity struct {\n\tlower resource.Quantity\n\tupper resource.Quantity\n}\n\ntype structuredVolume struct {\n\tcapacity resource.Quantity\n\tstorageClass string\n\tnfs *nFSVolumeSource\n\tcsi *csiVolumeSource\n\tvolumeType SupportedVolume\n\tpvcLabels map[string]string\n\tpvcPhase string\n}\n\nfunc (s *structuredVolume) parsePV(pv *corev1api.PersistentVolume) {\n\ts.capacity = *pv.Spec.Capacity.Storage()\n\ts.storageClass = pv.Spec.StorageClassName\n\tnfs := pv.Spec.NFS\n\tif nfs != nil {\n\t\ts.nfs = &nFSVolumeSource{Server: nfs.Server, Path: nfs.Path}\n\t}\n\n\tcsi := pv.Spec.CSI\n\tif csi != nil {\n\t\ts.csi = &csiVolumeSource{Driver: csi.Driver, VolumeAttributes: csi.VolumeAttributes}\n\t}\n\n\ts.volumeType = getVolumeTypeFromPV(pv)\n}\n\nfunc (s *structuredVolume) parsePVC(pvc *corev1api.PersistentVolumeClaim) {\n\tif pvc != nil {\n\t\tif len(pvc.GetLabels()) > 0 {\n\t\t\ts.pvcLabels = pvc.Labels\n\t\t}\n\t\ts.pvcPhase = string(pvc.Status.Phase)\n\t}\n}\n\nfunc (s *structuredVolume) parsePodVolume(vol *corev1api.Volume) {\n\tnfs := vol.NFS\n\tif nfs != nil {\n\t\ts.nfs = &nFSVolumeSource{Server: nfs.Server, Path: nfs.Path}\n\t}\n\n\tcsi := vol.CSI\n\tif csi != nil {\n\t\ts.csi = &csiVolumeSource{Driver: csi.Driver, VolumeAttributes: csi.VolumeAttributes}\n\t}\n\n\ts.volumeType = getVolumeTypeFromVolume(vol)\n}\n\n// pvcLabelsCondition defines a condition that matches if the PVC's labels contain all the provided key/value pairs.\ntype pvcLabelsCondition struct {\n\tlabels map[string]string\n}\n\nfunc (c *pvcLabelsCondition) match(v *structuredVolume) bool {\n\t// No labels specified: always match.\n\tif len(c.labels) == 0 {\n\t\treturn true\n\t}\n\tif v.pvcLabels == nil {\n\t\treturn false\n\t}\n\tselector := labels.SelectorFromSet(c.labels)\n\treturn selector.Matches(labels.Set(v.pvcLabels))\n}\n\nfunc (c *pvcLabelsCondition) validate() error {\n\treturn nil\n}\n\n// pvcPhaseCondition defines a condition that matches if the PVC's phase matches any of the provided phases.\ntype pvcPhaseCondition struct {\n\tphases []string\n}\n\nfunc (c *pvcPhaseCondition) match(v *structuredVolume) bool {\n\t// No phases specified: always match.\n\tif len(c.phases) == 0 {\n\t\treturn true\n\t}\n\tif v.pvcPhase == \"\" {\n\t\treturn false\n\t}\n\tfor _, phase := range c.phases {\n\t\tif v.pvcPhase == phase {\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n\nfunc (c *pvcPhaseCondition) validate() error {\n\treturn nil\n}\n\ntype capacityCondition struct {\n\tcapacity capacity\n}\n\nfunc (c *capacityCondition) match(v *structuredVolume) bool {\n\treturn c.capacity.isInRange(v.capacity)\n}\n\ntype storageClassCondition struct {\n\tstorageClass []string\n}\n\nfunc (s *storageClassCondition) match(v *structuredVolume) bool {\n\tif len(s.storageClass) == 0 {\n\t\treturn true\n\t}\n\n\tif v.storageClass == \"\" {\n\t\treturn false\n\t}\n\n\tfor _, sc := range s.storageClass {\n\t\tif v.storageClass == sc {\n\t\t\treturn true\n\t\t}\n\t}\n\n\treturn false\n}\n\ntype nfsCondition struct {\n\tnfs *nFSVolumeSource\n}\n\nfunc (c *nfsCondition) match(v *structuredVolume) bool {\n\tif c.nfs == nil {\n\t\treturn true\n\t}\n\tif v.nfs == nil {\n\t\treturn false\n\t}\n\n\tif c.nfs.Path == \"\" {\n\t\tif c.nfs.Server == \"\" { // match nfs: {}\n\t\t\treturn v.nfs != nil\n\t\t}\n\t\tif c.nfs.Server != v.nfs.Server {\n\t\t\treturn false\n\t\t}\n\t\treturn true\n\t}\n\tif c.nfs.Path != v.nfs.Path {\n\t\treturn false\n\t}\n\tif c.nfs.Server == \"\" {\n\t\treturn true\n\t}\n\tif c.nfs.Server != v.nfs.Server {\n\t\treturn false\n\t}\n\treturn true\n}\n\ntype csiCondition struct {\n\tcsi *csiVolumeSource\n}\n\nfunc (c *csiCondition) match(v *structuredVolume) bool {\n\tif c.csi == nil {\n\t\treturn true\n\t}\n\n\tif c.csi.Driver == \"\" { // match csi: {}\n\t\treturn v.csi != nil\n\t}\n\n\tif v.csi == nil {\n\t\treturn false\n\t}\n\n\tif c.csi.Driver != v.csi.Driver {\n\t\treturn false\n\t}\n\n\tif len(c.csi.VolumeAttributes) == 0 {\n\t\treturn true\n\t}\n\n\tif len(v.csi.VolumeAttributes) == 0 {\n\t\treturn false\n\t}\n\n\tfor key, value := range c.csi.VolumeAttributes {\n\t\tif value != v.csi.VolumeAttributes[key] {\n\t\t\treturn false\n\t\t}\n\t}\n\n\treturn true\n}\n\n// parseCapacity parse string into capacity format\nfunc parseCapacity(cap string) (*capacity, error) {\n\tif cap == \"\" {\n\t\tcap = \",\"\n\t}\n\tcapacities := strings.Split(cap, \",\")\n\tvar quantities []resource.Quantity\n\tif len(capacities) != 2 {\n\t\treturn nil, fmt.Errorf(\"wrong format of Capacity %v\", cap)\n\t}\n\n\tfor _, v := range capacities {\n\t\tif strings.TrimSpace(v) == \"\" {\n\t\t\t// case similar \"10Gi,\"\n\t\t\t// if empty, the quantity will assigned with 0\n\t\t\tquantities = append(quantities, *resource.NewQuantity(int64(0), resource.DecimalSI))\n\t\t} else {\n\t\t\tquantity, err := resource.ParseQuantity(strings.TrimSpace(v))\n\t\t\tif err != nil {\n\t\t\t\treturn nil, fmt.Errorf(\"wrong format of Capacity %v with err %v\", v, err)\n\t\t\t}\n\t\t\tquantities = append(quantities, quantity)\n\t\t}\n\t}\n\n\treturn &capacity{lower: quantities[0], upper: quantities[1]}, nil\n}\n\n// isInRange returns true if the quantity y is in range of capacity, or it returns false\nfunc (c *capacity) isInRange(y resource.Quantity) bool {\n\tif c.lower.IsZero() && c.upper.Cmp(y) >= 0 {\n\t\t// [0, a] y\n\t\treturn true\n\t}\n\tif c.upper.IsZero() && c.lower.Cmp(y) <= 0 {\n\t\t// [b, 0] y\n\t\treturn true\n\t}\n\tif !c.lower.IsZero() && !c.upper.IsZero() {\n\t\t// [a, b] y\n\t\treturn c.lower.Cmp(y) <= 0 && c.upper.Cmp(y) >= 0\n\t}\n\treturn false\n}\n\n// unmarshalVolConditions parse map[string]any into volumeConditions format\n// and validate key fields of the map.\nfunc unmarshalVolConditions(con map[string]any) (*volumeConditions, error) {\n\tvolConditons := &volumeConditions{}\n\tbuffer := new(bytes.Buffer)\n\terr := yaml.NewEncoder(buffer).Encode(con)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to encode volume conditions\")\n\t}\n\n\tif err := decodeStruct(buffer, volConditons); err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to decode volume conditions\")\n\t}\n\treturn volConditons, nil\n}\n" }, { "path": "internal/resourcepolicies/volume_resources_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\thttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage resourcepolicies\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/api/resource\"\n)\n\nfunc setStructuredVolume(capacity resource.Quantity, sc string, nfs *nFSVolumeSource, csi *csiVolumeSource, pvcLabels map[string]string) *structuredVolume {\n\treturn &structuredVolume{\n\t\tcapacity: capacity,\n\t\tstorageClass: sc,\n\t\tnfs: nfs,\n\t\tcsi: csi,\n\t\tpvcLabels: pvcLabels,\n\t}\n}\n\nfunc TestPVCLabelsMatch(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tcondition *pvcLabelsCondition\n\t\tvolume *structuredVolume\n\t\texpectedMatch bool\n\t}{\n\t\t{\n\t\t\tname: \"match exact label (single)\",\n\t\t\tcondition: &pvcLabelsCondition{\n\t\t\t\tlabels: map[string]string{\"environment\": \"production\"},\n\t\t\t},\n\t\t\tvolume: setStructuredVolume(\n\t\t\t\t*resource.NewQuantity(0, resource.BinarySI),\n\t\t\t\t\"any\",\n\t\t\t\tnil,\n\t\t\t\tnil,\n\t\t\t\tmap[string]string{\"environment\": \"production\", \"app\": \"database\"},\n\t\t\t),\n\t\t\texpectedMatch: true,\n\t\t},\n\t\t{\n\t\t\tname: \"match exact label (multiple)\",\n\t\t\tcondition: &pvcLabelsCondition{\n\t\t\t\tlabels: map[string]string{\"environment\": \"production\", \"app\": \"database\"},\n\t\t\t},\n\t\t\tvolume: setStructuredVolume(\n\t\t\t\t*resource.NewQuantity(0, resource.BinarySI),\n\t\t\t\t\"any\",\n\t\t\t\tnil,\n\t\t\t\tnil,\n\t\t\t\tmap[string]string{\"environment\": \"production\", \"app\": \"database\"},\n\t\t\t),\n\t\t\texpectedMatch: true,\n\t\t},\n\t\t{\n\t\t\tname: \"mismatch label value\",\n\t\t\tcondition: &pvcLabelsCondition{\n\t\t\t\tlabels: map[string]string{\"environment\": \"production\"},\n\t\t\t},\n\t\t\tvolume: setStructuredVolume(\n\t\t\t\t*resource.NewQuantity(0, resource.BinarySI),\n\t\t\t\t\"any\",\n\t\t\t\tnil,\n\t\t\t\tnil,\n\t\t\t\tmap[string]string{\"environment\": \"staging\", \"app\": \"database\"},\n\t\t\t),\n\t\t\texpectedMatch: false,\n\t\t},\n\t\t{\n\t\t\tname: \"missing label key\",\n\t\t\tcondition: &pvcLabelsCondition{\n\t\t\t\tlabels: map[string]string{\"environment\": \"production\", \"region\": \"us-west\"},\n\t\t\t},\n\t\t\tvolume: setStructuredVolume(\n\t\t\t\t*resource.NewQuantity(0, resource.BinarySI),\n\t\t\t\t\"any\",\n\t\t\t\tnil,\n\t\t\t\tnil,\n\t\t\t\tmap[string]string{\"environment\": \"production\", \"app\": \"database\"},\n\t\t\t),\n\t\t\texpectedMatch: false,\n\t\t},\n\t\t{\n\t\t\tname: \"empty condition always matches\",\n\t\t\tcondition: &pvcLabelsCondition{\n\t\t\t\tlabels: map[string]string{},\n\t\t\t},\n\t\t\tvolume: setStructuredVolume(\n\t\t\t\t*resource.NewQuantity(0, resource.BinarySI),\n\t\t\t\t\"any\",\n\t\t\t\tnil,\n\t\t\t\tnil,\n\t\t\t\tmap[string]string{\"environment\": \"staging\"},\n\t\t\t),\n\t\t\texpectedMatch: true,\n\t\t},\n\t\t{\n\t\t\tname: \"nil pvcLabels fails non-empty condition\",\n\t\t\tcondition: &pvcLabelsCondition{\n\t\t\t\tlabels: map[string]string{\"environment\": \"production\"},\n\t\t\t},\n\t\t\tvolume: setStructuredVolume(\n\t\t\t\t*resource.NewQuantity(0, resource.BinarySI),\n\t\t\t\t\"any\",\n\t\t\t\tnil,\n\t\t\t\tnil,\n\t\t\t\tnil,\n\t\t\t),\n\t\t\texpectedMatch: false,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tmatch := tt.condition.match(tt.volume)\n\t\t\tassert.Equal(t, tt.expectedMatch, match, \"expected match %v, got %v\", tt.expectedMatch, match)\n\t\t})\n\t}\n}\n\nfunc TestParseCapacity(t *testing.T) {\n\tvar emptyCapacity capacity\n\ttests := []struct {\n\t\tinput string\n\t\texpected capacity\n\t\texpectedErr error\n\t}{\n\t\t{\"10Gi,20Gi\", capacity{lower: *resource.NewQuantity(10<<30, resource.BinarySI), upper: *resource.NewQuantity(20<<30, resource.BinarySI)}, nil},\n\t\t{\"10Gi,\", capacity{lower: *resource.NewQuantity(10<<30, resource.BinarySI), upper: *resource.NewQuantity(0, resource.DecimalSI)}, nil},\n\t\t{\"10Gi\", emptyCapacity, fmt.Errorf(\"wrong format of Capacity 10Gi\")},\n\t\t{\"\", emptyCapacity, nil},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.input, func(t *testing.T) {\n\t\t\tactual, actualErr := parseCapacity(test.input)\n\t\t\tif test.expected != emptyCapacity {\n\t\t\t\tassert.Equal(t, 0, test.expected.lower.Cmp(actual.lower))\n\t\t\t\tassert.Equal(t, 0, test.expected.upper.Cmp(actual.upper))\n\t\t\t}\n\t\t\tassert.Equal(t, test.expectedErr, actualErr)\n\t\t})\n\t}\n}\n\nfunc TestCapacityIsInRange(t *testing.T) {\n\tt.Parallel()\n\n\ttests := []struct {\n\t\tcapacity *capacity\n\t\tquantity resource.Quantity\n\t\tisInRange bool\n\t}{\n\t\t{&capacity{*resource.NewQuantity(0, resource.BinarySI), *resource.NewQuantity(10<<30, resource.BinarySI)}, *resource.NewQuantity(5<<30, resource.BinarySI), true},\n\t\t{&capacity{*resource.NewQuantity(0, resource.BinarySI), *resource.NewQuantity(10<<30, resource.BinarySI)}, *resource.NewQuantity(15<<30, resource.BinarySI), false},\n\t\t{&capacity{*resource.NewQuantity(20<<30, resource.BinarySI), *resource.NewQuantity(0, resource.DecimalSI)}, *resource.NewQuantity(25<<30, resource.BinarySI), true},\n\t\t{&capacity{*resource.NewQuantity(20<<30, resource.BinarySI), *resource.NewQuantity(0, resource.DecimalSI)}, *resource.NewQuantity(15<<30, resource.BinarySI), false},\n\t\t{&capacity{*resource.NewQuantity(10<<30, resource.BinarySI), *resource.NewQuantity(20<<30, resource.BinarySI)}, *resource.NewQuantity(15<<30, resource.BinarySI), true},\n\t\t{&capacity{*resource.NewQuantity(10<<30, resource.BinarySI), *resource.NewQuantity(20<<30, resource.BinarySI)}, *resource.NewQuantity(5<<30, resource.BinarySI), false},\n\t\t{&capacity{*resource.NewQuantity(10<<30, resource.BinarySI), *resource.NewQuantity(20<<30, resource.BinarySI)}, *resource.NewQuantity(25<<30, resource.BinarySI), false},\n\t\t{&capacity{*resource.NewQuantity(0, resource.BinarySI), *resource.NewQuantity(0, resource.BinarySI)}, *resource.NewQuantity(5<<30, resource.BinarySI), true},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(fmt.Sprintf(\"%v with %v\", test.capacity, test.quantity), func(t *testing.T) {\n\t\t\tt.Parallel()\n\n\t\t\tactual := test.capacity.isInRange(test.quantity)\n\n\t\t\tassert.Equal(t, test.isInRange, actual)\n\t\t})\n\t}\n}\n\nfunc TestStorageClassConditionMatch(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tcondition *storageClassCondition\n\t\tvolume *structuredVolume\n\t\texpectedMatch bool\n\t}{\n\t\t{\n\t\t\tname: \"match single storage class\",\n\t\t\tcondition: &storageClassCondition{[]string{\"gp2\"}},\n\t\t\tvolume: setStructuredVolume(*resource.NewQuantity(0, resource.BinarySI), \"gp2\", nil, nil, nil),\n\t\t\texpectedMatch: true,\n\t\t},\n\t\t{\n\t\t\tname: \"match multiple storage classes\",\n\t\t\tcondition: &storageClassCondition{[]string{\"gp2\", \"ebs-sc\"}},\n\t\t\tvolume: setStructuredVolume(*resource.NewQuantity(0, resource.BinarySI), \"gp2\", nil, nil, nil),\n\t\t\texpectedMatch: true,\n\t\t},\n\t\t{\n\t\t\tname: \"mismatch storage class\",\n\t\t\tcondition: &storageClassCondition{[]string{\"gp2\"}},\n\t\t\tvolume: setStructuredVolume(*resource.NewQuantity(0, resource.BinarySI), \"ebs-sc\", nil, nil, nil),\n\t\t\texpectedMatch: false,\n\t\t},\n\t\t{\n\t\t\tname: \"empty storage class\",\n\t\t\tcondition: &storageClassCondition{[]string{}},\n\t\t\tvolume: setStructuredVolume(*resource.NewQuantity(0, resource.BinarySI), \"ebs-sc\", nil, nil, nil),\n\t\t\texpectedMatch: true,\n\t\t},\n\t\t{\n\t\t\tname: \"empty volume storage class\",\n\t\t\tcondition: &storageClassCondition{[]string{\"gp2\"}},\n\t\t\tvolume: setStructuredVolume(*resource.NewQuantity(0, resource.BinarySI), \"\", nil, nil, nil),\n\t\t\texpectedMatch: false,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tmatch := tt.condition.match(tt.volume)\n\t\t\tif match != tt.expectedMatch {\n\t\t\t\tt.Errorf(\"expected %v, but got %v\", tt.expectedMatch, match)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestNFSConditionMatch(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tcondition *nfsCondition\n\t\tvolume *structuredVolume\n\t\texpectedMatch bool\n\t}{\n\t\t{\n\t\t\tname: \"match nfs condition\",\n\t\t\tcondition: &nfsCondition{&nFSVolumeSource{Server: \"192.168.10.20\"}},\n\t\t\tvolume: setStructuredVolume(*resource.NewQuantity(0, resource.BinarySI), \"\", &nFSVolumeSource{Server: \"192.168.10.20\"}, nil, nil),\n\t\t\texpectedMatch: true,\n\t\t},\n\t\t{\n\t\t\tname: \"empty nfs condition\",\n\t\t\tcondition: &nfsCondition{nil},\n\t\t\tvolume: setStructuredVolume(*resource.NewQuantity(0, resource.BinarySI), \"\", &nFSVolumeSource{Server: \"192.168.10.20\"}, nil, nil),\n\t\t\texpectedMatch: true,\n\t\t},\n\t\t{\n\t\t\tname: \"empty nfs server and path condition\",\n\t\t\tcondition: &nfsCondition{&nFSVolumeSource{Server: \"\", Path: \"\"}},\n\t\t\tvolume: setStructuredVolume(*resource.NewQuantity(0, resource.BinarySI), \"\", &nFSVolumeSource{Server: \"192.168.10.20\"}, nil, nil),\n\t\t\texpectedMatch: true,\n\t\t},\n\t\t{\n\t\t\tname: \"server mismatch\",\n\t\t\tcondition: &nfsCondition{&nFSVolumeSource{Server: \"192.168.10.20\", Path: \"\"}},\n\t\t\tvolume: setStructuredVolume(*resource.NewQuantity(0, resource.BinarySI), \"\", &nFSVolumeSource{Server: \"\"}, nil, nil),\n\t\t\texpectedMatch: false,\n\t\t},\n\t\t{\n\t\t\tname: \"empty nfs server condition\",\n\t\t\tcondition: &nfsCondition{&nFSVolumeSource{Path: \"/mnt/data\"}},\n\t\t\tvolume: setStructuredVolume(*resource.NewQuantity(0, resource.BinarySI), \"\", &nFSVolumeSource{Server: \"192.168.10.20\", Path: \"/mnt/data\"}, nil, nil),\n\t\t\texpectedMatch: true,\n\t\t},\n\t\t{\n\t\t\tname: \"empty nfs volume\",\n\t\t\tcondition: &nfsCondition{&nFSVolumeSource{Server: \"192.168.10.20\"}},\n\t\t\tvolume: setStructuredVolume(*resource.NewQuantity(0, resource.BinarySI), \"\", nil, nil, nil),\n\t\t\texpectedMatch: false,\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tmatch := tt.condition.match(tt.volume)\n\t\t\tif match != tt.expectedMatch {\n\t\t\t\tt.Errorf(\"expected %v, but got %v\", tt.expectedMatch, match)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestCSIConditionMatch(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tcondition *csiCondition\n\t\tvolume *structuredVolume\n\t\texpectedMatch bool\n\t}{\n\t\t{\n\t\t\tname: \"match csi driver condition\",\n\t\t\tcondition: &csiCondition{&csiVolumeSource{Driver: \"test\"}},\n\t\t\tvolume: setStructuredVolume(*resource.NewQuantity(0, resource.BinarySI), \"\", nil, &csiVolumeSource{Driver: \"test\"}, nil),\n\t\t\texpectedMatch: true,\n\t\t},\n\t\t{\n\t\t\tname: \"empty csi driver condition\",\n\t\t\tcondition: &csiCondition{nil},\n\t\t\tvolume: setStructuredVolume(*resource.NewQuantity(0, resource.BinarySI), \"\", nil, &csiVolumeSource{Driver: \"test\"}, nil),\n\t\t\texpectedMatch: true,\n\t\t},\n\t\t{\n\t\t\tname: \"empty csi driver volume\",\n\t\t\tcondition: &csiCondition{&csiVolumeSource{Driver: \"test\"}},\n\t\t\tvolume: setStructuredVolume(*resource.NewQuantity(0, resource.BinarySI), \"\", nil, &csiVolumeSource{}, nil),\n\t\t\texpectedMatch: false,\n\t\t},\n\t\t{\n\t\t\tname: \"match csi volumeAttributes condition\",\n\t\t\tcondition: &csiCondition{&csiVolumeSource{Driver: \"test\", VolumeAttributes: map[string]string{\"protocol\": \"nfs\"}}},\n\t\t\tvolume: setStructuredVolume(*resource.NewQuantity(0, resource.BinarySI), \"\", nil, &csiVolumeSource{Driver: \"test\", VolumeAttributes: map[string]string{\"protocol\": \"nfs\"}}, nil),\n\t\t\texpectedMatch: true,\n\t\t},\n\t\t{\n\t\t\tname: \"empty csi volumeAttributes condition\",\n\t\t\tcondition: &csiCondition{&csiVolumeSource{Driver: \"test\"}},\n\t\t\tvolume: setStructuredVolume(*resource.NewQuantity(0, resource.BinarySI), \"\", nil, &csiVolumeSource{Driver: \"test\", VolumeAttributes: map[string]string{\"protocol\": \"nfs\"}}, nil),\n\t\t\texpectedMatch: true,\n\t\t},\n\t\t{\n\t\t\tname: \"empty csi volumeAttributes volume\",\n\t\t\tcondition: &csiCondition{&csiVolumeSource{Driver: \"test\", VolumeAttributes: map[string]string{\"protocol\": \"nfs\"}}},\n\t\t\tvolume: setStructuredVolume(*resource.NewQuantity(0, resource.BinarySI), \"\", nil, &csiVolumeSource{Driver: \"test\", VolumeAttributes: map[string]string{\"protocol\": \"\"}}, nil),\n\t\t\texpectedMatch: false,\n\t\t},\n\t\t{\n\t\t\tname: \"empty csi volumeAttributes volume\",\n\t\t\tcondition: &csiCondition{&csiVolumeSource{Driver: \"test\", VolumeAttributes: map[string]string{\"protocol\": \"nfs\"}}},\n\t\t\tvolume: setStructuredVolume(*resource.NewQuantity(0, resource.BinarySI), \"\", nil, &csiVolumeSource{Driver: \"test\"}, nil),\n\t\t\texpectedMatch: false,\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tmatch := tt.condition.match(tt.volume)\n\t\t\tif match != tt.expectedMatch {\n\t\t\t\tt.Errorf(\"expected %v, but got %v\", tt.expectedMatch, match)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestUnmarshalVolumeConditions(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tinput map[string]any\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"Valid input\",\n\t\t\tinput: map[string]any{\n\t\t\t\t\"capacity\": \"1Gi,10Gi\",\n\t\t\t\t\"storageClass\": []string{\n\t\t\t\t\t\"gp2\",\n\t\t\t\t\t\"ebs-sc\",\n\t\t\t\t},\n\t\t\t\t\"csi\": &csiVolumeSource{\n\t\t\t\t\tDriver: \"aws.efs.csi.driver\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedError: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"Invalid input: invalid capacity filed name\",\n\t\t\tinput: map[string]any{\n\t\t\t\t\"Capacity\": \"1Gi,10Gi\",\n\t\t\t},\n\t\t\texpectedError: \"field Capacity not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"Invalid input: invalid storage class format\",\n\t\t\tinput: map[string]any{\n\t\t\t\t\"storageClass\": \"ebs-sc\",\n\t\t\t},\n\t\t\texpectedError: \"str `ebs-sc` into []string\",\n\t\t},\n\t\t{\n\t\t\tname: \"Invalid input: invalid csi format\",\n\t\t\tinput: map[string]any{\n\t\t\t\t\"csi\": \"csi.driver\",\n\t\t\t},\n\t\t\texpectedError: \"str `csi.driver` into resourcepolicies.csiVolumeSource\",\n\t\t},\n\t\t{\n\t\t\tname: \"Invalid input: unknown field\",\n\t\t\tinput: map[string]any{\n\t\t\t\t\"unknown\": \"foo\",\n\t\t\t},\n\t\t\texpectedError: \"field unknown not found in type\",\n\t\t},\n\t\t{\n\t\t\tname: \"Valid pvcLabels input as map[string]string\",\n\t\t\tinput: map[string]any{\n\t\t\t\t\"capacity\": \"1Gi,10Gi\",\n\t\t\t\t\"pvcLabels\": map[string]string{\n\t\t\t\t\t\"environment\": \"production\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedError: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"Valid pvcLabels input as map[string]any\",\n\t\t\tinput: map[string]any{\n\t\t\t\t\"capacity\": \"1Gi,10Gi\",\n\t\t\t\t\"pvcLabels\": map[string]any{\n\t\t\t\t\t\"environment\": \"production\",\n\t\t\t\t\t\"app\": \"database\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedError: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"Invalid pvcLabels input: not a map\",\n\t\t\tinput: map[string]any{\n\t\t\t\t\"capacity\": \"1Gi,10Gi\",\n\t\t\t\t\"pvcLabels\": \"production\",\n\t\t\t},\n\t\t\texpectedError: \"!!str `production` into map[string]string\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t_, err := unmarshalVolConditions(tc.input)\n\t\t\tif tc.expectedError != \"\" {\n\t\t\t\tif err == nil {\n\t\t\t\t\tt.Errorf(\"Expected error '%s', but got nil\", tc.expectedError)\n\t\t\t\t} else if !strings.Contains(err.Error(), tc.expectedError) {\n\t\t\t\t\tt.Errorf(\"Expected error '%s', but got '%v'\", tc.expectedError, err)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestParsePodVolume(t *testing.T) {\n\t// Mock data\n\tnfsVolume := corev1api.Volume{}\n\tnfsVolume.NFS = &corev1api.NFSVolumeSource{\n\t\tServer: \"nfs.example.com\",\n\t\tPath: \"/exports/data\",\n\t}\n\tcsiVolume := corev1api.Volume{}\n\tcsiVolume.CSI = &corev1api.CSIVolumeSource{\n\t\tDriver: \"csi.example.com\",\n\t\tVolumeAttributes: map[string]string{\"protocol\": \"nfs\"},\n\t}\n\temptyVolume := corev1api.Volume{}\n\n\t// Test cases\n\ttestCases := []struct {\n\t\tname string\n\t\tinputVolume *corev1api.Volume\n\t\texpectedNFS *nFSVolumeSource\n\t\texpectedCSI *csiVolumeSource\n\t}{\n\t\t{\n\t\t\tname: \"NFS volume\",\n\t\t\tinputVolume: &nfsVolume,\n\t\t\texpectedNFS: &nFSVolumeSource{Server: \"nfs.example.com\", Path: \"/exports/data\"},\n\t\t},\n\t\t{\n\t\t\tname: \"CSI volume\",\n\t\t\tinputVolume: &csiVolume,\n\t\t\texpectedCSI: &csiVolumeSource{Driver: \"csi.example.com\", VolumeAttributes: map[string]string{\"protocol\": \"nfs\"}},\n\t\t},\n\t\t{\n\t\t\tname: \"Empty volume\",\n\t\t\tinputVolume: &emptyVolume,\n\t\t\texpectedNFS: nil,\n\t\t\texpectedCSI: nil,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t// Call the function\n\t\t\tstructuredVolume := &structuredVolume{}\n\t\t\tstructuredVolume.parsePodVolume(tc.inputVolume)\n\n\t\t\t// Check the results\n\t\t\tif tc.expectedNFS != nil {\n\t\t\t\tif structuredVolume.nfs == nil {\n\t\t\t\t\tt.Errorf(\"Expected a non-nil NFS volume source\")\n\t\t\t\t} else if *tc.expectedNFS != *structuredVolume.nfs {\n\t\t\t\t\tt.Errorf(\"NFS volume source does not match expected value\")\n\t\t\t\t}\n\t\t\t}\n\t\t\tif tc.expectedCSI != nil {\n\t\t\t\tif structuredVolume.csi == nil {\n\t\t\t\t\tt.Errorf(\"Expected a non-nil CSI volume source\")\n\t\t\t\t} else if tc.expectedCSI.Driver != structuredVolume.csi.Driver {\n\t\t\t\t\tt.Errorf(\"CSI volume source does not match expected value\")\n\t\t\t\t}\n\t\t\t\t// Check volumeAttributes\n\t\t\t\tif len(tc.expectedCSI.VolumeAttributes) != len(structuredVolume.csi.VolumeAttributes) {\n\t\t\t\t\tt.Errorf(\"CSI volume attributes does not match expected value\")\n\t\t\t\t} else {\n\t\t\t\t\tfor k, v := range tc.expectedCSI.VolumeAttributes {\n\t\t\t\t\t\tif structuredVolume.csi.VolumeAttributes[k] != v {\n\t\t\t\t\t\t\tt.Errorf(\"CSI volume attributes does not match expected value\")\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestParsePV(t *testing.T) {\n\t// Mock data\n\tnfsVolume := corev1api.PersistentVolume{}\n\tnfsVolume.Spec.Capacity = corev1api.ResourceList{corev1api.ResourceStorage: resource.MustParse(\"1Gi\")}\n\tnfsVolume.Spec.NFS = &corev1api.NFSVolumeSource{Server: \"nfs.example.com\", Path: \"/exports/data\"}\n\tcsiVolume := corev1api.PersistentVolume{}\n\tcsiVolume.Spec.Capacity = corev1api.ResourceList{corev1api.ResourceStorage: resource.MustParse(\"2Gi\")}\n\tcsiVolume.Spec.CSI = &corev1api.CSIPersistentVolumeSource{Driver: \"csi.example.com\", VolumeAttributes: map[string]string{\"protocol\": \"nfs\"}}\n\temptyVolume := corev1api.PersistentVolume{}\n\n\t// Test cases\n\ttestCases := []struct {\n\t\tname string\n\t\tinputVolume *corev1api.PersistentVolume\n\t\texpectedNFS *nFSVolumeSource\n\t\texpectedCSI *csiVolumeSource\n\t}{\n\t\t{\n\t\t\tname: \"NFS volume\",\n\t\t\tinputVolume: &nfsVolume,\n\t\t\texpectedNFS: &nFSVolumeSource{Server: \"nfs.example.com\", Path: \"/exports/data\"},\n\t\t\texpectedCSI: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"CSI volume\",\n\t\t\tinputVolume: &csiVolume,\n\t\t\texpectedNFS: nil,\n\t\t\texpectedCSI: &csiVolumeSource{Driver: \"csi.example.com\", VolumeAttributes: map[string]string{\"protocol\": \"nfs\"}},\n\t\t},\n\t\t{\n\t\t\tname: \"Empty volume\",\n\t\t\tinputVolume: &emptyVolume,\n\t\t\texpectedNFS: nil,\n\t\t\texpectedCSI: nil,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t// Call the function\n\t\t\tstructuredVolume := &structuredVolume{}\n\t\t\tstructuredVolume.parsePV(tc.inputVolume)\n\t\t\t// Check the results\n\t\t\tif structuredVolume.capacity != *tc.inputVolume.Spec.Capacity.Storage() {\n\t\t\t\tt.Errorf(\"capacity does not match expected value\")\n\t\t\t}\n\t\t\tif structuredVolume.storageClass != tc.inputVolume.Spec.StorageClassName {\n\t\t\t\tt.Errorf(\"Storage class does not match expected value\")\n\t\t\t}\n\t\t\tif tc.expectedNFS != nil {\n\t\t\t\tif structuredVolume.nfs == nil {\n\t\t\t\t\tt.Errorf(\"Expected a non-nil NFS volume source\")\n\t\t\t\t} else if *tc.expectedNFS != *structuredVolume.nfs {\n\t\t\t\t\tt.Errorf(\"NFS volume source does not match expected value\")\n\t\t\t\t}\n\t\t\t}\n\t\t\tif tc.expectedCSI != nil {\n\t\t\t\tif structuredVolume.csi == nil {\n\t\t\t\t\tt.Errorf(\"Expected a non-nil CSI volume source\")\n\t\t\t\t} else if tc.expectedCSI.Driver != structuredVolume.csi.Driver {\n\t\t\t\t\tt.Errorf(\"CSI volume source does not match expected value\")\n\t\t\t\t}\n\t\t\t\t// Check volumeAttributes\n\t\t\t\tif len(tc.expectedCSI.VolumeAttributes) != len(structuredVolume.csi.VolumeAttributes) {\n\t\t\t\t\tt.Errorf(\"CSI volume attributes does not match expected value\")\n\t\t\t\t} else {\n\t\t\t\t\tfor k, v := range tc.expectedCSI.VolumeAttributes {\n\t\t\t\t\t\tif structuredVolume.csi.VolumeAttributes[k] != v {\n\t\t\t\t\t\t\tt.Errorf(\"CSI volume attributes does not match expected value\")\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "internal/resourcepolicies/volume_resources_validator.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\thttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage resourcepolicies\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\n\t\"github.com/pkg/errors\"\n\t\"gopkg.in/yaml.v3\"\n)\n\nconst currentSupportDataVersion = \"v1\"\n\ntype csiVolumeSource struct {\n\tDriver string `yaml:\"driver,omitempty\"`\n\t// CSI volume attributes\n\tVolumeAttributes map[string]string `yaml:\"volumeAttributes,omitempty\"`\n}\n\ntype nFSVolumeSource struct {\n\t// Server is the hostname or IP address of the NFS server\n\tServer string `yaml:\"server,omitempty\"`\n\t// Path is the exported NFS share\n\tPath string `yaml:\"path,omitempty\"`\n}\n\n// volumeConditions defined the current format of conditions we parsed\ntype volumeConditions struct {\n\tCapacity string `yaml:\"capacity,omitempty\"`\n\tStorageClass []string `yaml:\"storageClass,omitempty\"`\n\tNFS *nFSVolumeSource `yaml:\"nfs,omitempty\"`\n\tCSI *csiVolumeSource `yaml:\"csi,omitempty\"`\n\tVolumeTypes []SupportedVolume `yaml:\"volumeTypes,omitempty\"`\n\tPVCLabels map[string]string `yaml:\"pvcLabels,omitempty\"`\n\tPVCPhase []string `yaml:\"pvcPhase,omitempty\"`\n}\n\nfunc (c *capacityCondition) validate() error {\n\t// [0, a]\n\t// [a, b]\n\t// [b, 0]\n\t// ==> low <= upper or upper is zero\n\tif (c.capacity.upper.Cmp(c.capacity.lower) >= 0) ||\n\t\t(!c.capacity.lower.IsZero() && c.capacity.upper.IsZero()) {\n\t\treturn nil\n\t}\n\treturn errors.Errorf(\"illegal values for capacity %v\", c.capacity)\n}\n\nfunc (s *storageClassCondition) validate() error {\n\t// validate by yamlv3\n\treturn nil\n}\n\nfunc (c *nfsCondition) validate() error {\n\t// validate by yamlv3\n\treturn nil\n}\n\nfunc (c *csiCondition) validate() error {\n\tif c != nil && c.csi != nil && c.csi.Driver == \"\" && c.csi.VolumeAttributes != nil {\n\t\treturn errors.New(\"csi driver should not be empty when filtering by volume attributes\")\n\t}\n\n\treturn nil\n}\n\n// decodeStruct restric validate the keys in decoded mappings to exist as fields in the struct being decoded into\nfunc decodeStruct(r io.Reader, s any) error {\n\tdec := yaml.NewDecoder(r)\n\tdec.KnownFields(true)\n\treturn dec.Decode(s)\n}\n\n// validate check action format\nfunc (a *Action) validate() error {\n\t// validate Type\n\tvalid := false\n\tif a.Type == Skip || a.Type == Snapshot || a.Type == FSBackup {\n\t\tvalid = true\n\t}\n\tif !valid {\n\t\treturn fmt.Errorf(\"invalid action type %s\", a.Type)\n\t}\n\n\t// TODO validate parameters\n\treturn nil\n}\n" }, { "path": "internal/resourcepolicies/volume_resources_validator_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\thttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage resourcepolicies\n\nimport (\n\t\"testing\"\n\n\t\"k8s.io/apimachinery/pkg/api/resource\"\n)\n\nfunc TestCapacityConditionValidate(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tcapacity *capacity\n\t\twantErr bool\n\t}{\n\t\t{\n\t\t\tname: \"lower and upper are both zero\",\n\t\t\tcapacity: &capacity{lower: *resource.NewQuantity(0, resource.DecimalSI), upper: *resource.NewQuantity(0, resource.DecimalSI)},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"lower is zero and upper is greater than zero\",\n\t\t\tcapacity: &capacity{lower: *resource.NewQuantity(0, resource.DecimalSI), upper: *resource.NewQuantity(100, resource.DecimalSI)},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"lower is greater than upper\",\n\t\t\tcapacity: &capacity{lower: *resource.NewQuantity(100, resource.DecimalSI), upper: *resource.NewQuantity(50, resource.DecimalSI)},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"lower and upper are equal\",\n\t\t\tcapacity: &capacity{lower: *resource.NewQuantity(100, resource.DecimalSI), upper: *resource.NewQuantity(100, resource.DecimalSI)},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"lower is greater than zero and upper is zero\",\n\t\t\tcapacity: &capacity{lower: *resource.NewQuantity(100, resource.DecimalSI), upper: *resource.NewQuantity(0, resource.DecimalSI)},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"lower and upper are both not zero and lower is less than upper\",\n\t\t\tcapacity: &capacity{lower: *resource.NewQuantity(100, resource.DecimalSI), upper: *resource.NewQuantity(200, resource.DecimalSI)},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"lower and upper are both not zero and lower is equal to upper\",\n\t\t\tcapacity: &capacity{lower: *resource.NewQuantity(100, resource.DecimalSI), upper: *resource.NewQuantity(100, resource.DecimalSI)},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"lower and upper are both not zero and lower is greater than upper\",\n\t\t\tcapacity: &capacity{lower: *resource.NewQuantity(200, resource.DecimalSI), upper: *resource.NewQuantity(100, resource.DecimalSI)},\n\t\t\twantErr: true,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tc := &capacityCondition{capacity: *tc.capacity}\n\t\t\terr := c.validate()\n\n\t\t\tif (err != nil) != tc.wantErr {\n\t\t\t\tt.Fatalf(\"Expected error %v, but got error %v\", tc.wantErr, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestValidate(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tres *ResourcePolicies\n\t\twantErr bool\n\t}{\n\t\t{\n\t\t\tname: \"unknown key in yaml\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"capacity\": \"0,10Gi\",\n\t\t\t\t\t\t\t\"unknown\": \"\",\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2\", \"ebs-sc\"},\n\t\t\t\t\t\t\t\"csi\": any(\n\t\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\t\"driver\": \"aws.efs.csi.driver\",\n\t\t\t\t\t\t\t\t}),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"error format of capacity\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"capacity\": \"10Gi\",\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2\", \"ebs-sc\"},\n\t\t\t\t\t\t\t\"csi\": any(\n\t\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\t\"driver\": \"aws.efs.csi.driver\",\n\t\t\t\t\t\t\t\t}),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"error format of storageClass\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"capacity\": \"0,10Gi\",\n\t\t\t\t\t\t\t\"storageClass\": \"ebs-sc\",\n\t\t\t\t\t\t\t\"csi\": any(\n\t\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\t\"driver\": \"aws.efs.csi.driver\",\n\t\t\t\t\t\t\t\t}),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"error format of csi\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"capacity\": \"0,10Gi\",\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2\", \"ebs-sc\"},\n\t\t\t\t\t\t\t\"csi\": \"aws.efs.csi.driver\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"error format of csi driver\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"capacity\": \"0,10Gi\",\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2\", \"ebs-sc\"},\n\t\t\t\t\t\t\t\"csi\": any(\n\t\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\t\"driver\": []string{\"aws.efs.csi.driver\"},\n\t\t\t\t\t\t\t\t}),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"error format of csi driver volumeAttributes\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"capacity\": \"0,10Gi\",\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2\", \"ebs-sc\"},\n\t\t\t\t\t\t\t\"csi\": any(\n\t\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\t\"driver\": \"aws.efs.csi.driver\",\n\t\t\t\t\t\t\t\t\t\"volumeAttributes\": \"test\",\n\t\t\t\t\t\t\t\t}),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"unsupported version\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v2\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"capacity\": \"0,10Gi\",\n\t\t\t\t\t\t\t\"csi\": any(\n\t\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\t\"driver\": \"aws.efs.csi.driver\",\n\t\t\t\t\t\t\t\t}),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"unsupported action\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"unsupported\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"capacity\": \"0,10Gi\",\n\t\t\t\t\t\t\t\"csi\": any(\n\t\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\t\"driver\": \"aws.efs.csi.driver\",\n\t\t\t\t\t\t\t\t}),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"error format of nfs\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"capacity\": \"0,10Gi\",\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2\", \"ebs-sc\"},\n\t\t\t\t\t\t\t\"nfs\": \"aws.efs.csi.driver\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"supported format volume policies only csi driver\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"csi\": any(\n\t\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\t\"driver\": \"aws.efs.csi.driver\",\n\t\t\t\t\t\t\t\t}),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"unsupported format volume policies only csi volumeattributes\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"csi\": any(\n\t\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\t\"volumeAttributes\": map[string]string{\n\t\t\t\t\t\t\t\t\t\t\"key1\": \"value1\",\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t}),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"supported format volume policies with csi driver and volumeattributes\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"csi\": any(\n\t\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\t\"driver\": \"aws.efs.csi.driver\",\n\t\t\t\t\t\t\t\t\t\"volumeAttributes\": map[string]string{\n\t\t\t\t\t\t\t\t\t\t\"key1\": \"value1\",\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t}),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"supported format volume policies\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"capacity\": \"0,10Gi\",\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2\", \"ebs-sc\"},\n\t\t\t\t\t\t\t\"csi\": any(\n\t\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\t\"driver\": \"aws.efs.csi.driver\",\n\t\t\t\t\t\t\t\t}),\n\t\t\t\t\t\t\t\"nfs\": any(\n\t\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\t\"server\": \"192.168.20.90\",\n\t\t\t\t\t\t\t\t\t\"path\": \"/mnt/data/\",\n\t\t\t\t\t\t\t\t}),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"supported format volume policies, action type snapshot\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"snapshot\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"capacity\": \"0,10Gi\",\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2\", \"ebs-sc\"},\n\t\t\t\t\t\t\t\"csi\": any(\n\t\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\t\"driver\": \"aws.efs.csi.driver\",\n\t\t\t\t\t\t\t\t}),\n\t\t\t\t\t\t\t\"nfs\": any(\n\t\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\t\"server\": \"192.168.20.90\",\n\t\t\t\t\t\t\t\t\t\"path\": \"/mnt/data/\",\n\t\t\t\t\t\t\t\t}),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"supported format volume policies, action type fs-backup\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"fs-backup\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"capacity\": \"0,10Gi\",\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2\", \"ebs-sc\"},\n\t\t\t\t\t\t\t\"csi\": any(\n\t\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\t\"driver\": \"aws.efs.csi.driver\",\n\t\t\t\t\t\t\t\t}),\n\t\t\t\t\t\t\t\"nfs\": any(\n\t\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\t\"server\": \"192.168.20.90\",\n\t\t\t\t\t\t\t\t\t\"path\": \"/mnt/data/\",\n\t\t\t\t\t\t\t\t}),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"supported format volume policies, action type fs-backup and snapshot\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: Snapshot},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: FSBackup},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"nfs\": any(\n\t\t\t\t\t\t\t\tmap[string]any{\n\t\t\t\t\t\t\t\t\t\"server\": \"192.168.20.90\",\n\t\t\t\t\t\t\t\t\t\"path\": \"/mnt/data/\",\n\t\t\t\t\t\t\t\t}),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"supported format volume policies with pvcLabels (valid map)\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"pvcLabels\": map[string]string{\n\t\t\t\t\t\t\t\t\"environment\": \"production\",\n\t\t\t\t\t\t\t\t\"app\": \"database\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"error format volume policies with pvcLabels (not a map)\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"pvcLabels\": \"production\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \" '*' in the filters of include exclude policy - 1\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"pvcLabels\": map[string]string{\n\t\t\t\t\t\t\t\t\"environment\": \"production\",\n\t\t\t\t\t\t\t\t\"app\": \"database\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tIncludeExcludePolicy: &IncludeExcludePolicy{\n\t\t\t\t\tIncludedClusterScopedResources: []string{\"*\"},\n\t\t\t\t\tExcludedClusterScopedResources: []string{\"crds\"},\n\t\t\t\t\tIncludedNamespaceScopedResources: []string{\"pods\"},\n\t\t\t\t\tExcludedNamespaceScopedResources: []string{\"secrets\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \" '*' in the filters of include exclude policy - 2\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"pvcLabels\": map[string]string{\n\t\t\t\t\t\t\t\t\"environment\": \"production\",\n\t\t\t\t\t\t\t\t\"app\": \"database\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tIncludeExcludePolicy: &IncludeExcludePolicy{\n\t\t\t\t\tIncludedClusterScopedResources: []string{\"persistentvolumes\"},\n\t\t\t\t\tExcludedClusterScopedResources: []string{\"crds\"},\n\t\t\t\t\tIncludedNamespaceScopedResources: []string{\"pods\"},\n\t\t\t\t\tExcludedNamespaceScopedResources: []string{\"*\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \" dup item in both the include and exclude filters of include exclude policy\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"pvcLabels\": map[string]string{\n\t\t\t\t\t\t\t\t\"environment\": \"production\",\n\t\t\t\t\t\t\t\t\"app\": \"database\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tIncludeExcludePolicy: &IncludeExcludePolicy{\n\t\t\t\t\tIncludedClusterScopedResources: []string{\"persistentvolumes\"},\n\t\t\t\t\tExcludedClusterScopedResources: []string{\"crds\"},\n\t\t\t\t\tIncludedNamespaceScopedResources: []string{\"pods\", \"configmaps\"},\n\t\t\t\t\tExcludedNamespaceScopedResources: []string{\"secrets\", \"pods\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \" valid volume policies and valid include/exclude policy\",\n\t\t\tres: &ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: Action{Type: \"skip\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"pvcLabels\": map[string]string{\n\t\t\t\t\t\t\t\t\"environment\": \"production\",\n\t\t\t\t\t\t\t\t\"app\": \"database\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tIncludeExcludePolicy: &IncludeExcludePolicy{\n\t\t\t\t\tIncludedClusterScopedResources: []string{\"persistentvolumes\"},\n\t\t\t\t\tExcludedClusterScopedResources: []string{\"crds\"},\n\t\t\t\t\tIncludedNamespaceScopedResources: []string{\"pods\", \"configmaps\"},\n\t\t\t\t\tExcludedNamespaceScopedResources: []string{\"secrets\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: false,\n\t\t},\n\t}\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tpolicies := &Policies{}\n\t\t\terr1 := policies.BuildPolicy(tc.res)\n\t\t\terr2 := policies.Validate()\n\n\t\t\tif tc.wantErr {\n\t\t\t\tif err1 == nil && err2 == nil {\n\t\t\t\t\tt.Fatalf(\"Expected error %v, but not get error\", tc.wantErr)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif err1 != nil || err2 != nil {\n\t\t\t\t\tt.Fatalf(\"Expected error %v, but got error %v %v\", tc.wantErr, err1, err2)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "internal/resourcepolicies/volume_types_conditions.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage resourcepolicies\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n)\n\ntype volumeTypeCondition struct {\n\tvolumeTypes []SupportedVolume\n}\n\ntype SupportedVolume string\n\nconst (\n\tAWSAzureDisk SupportedVolume = \"awsAzureDisk\"\n\tAWSElasticBlockStore SupportedVolume = \"awsElasticBlockStore\"\n\tAzureDisk SupportedVolume = \"azureDisk\"\n\tAzureFile SupportedVolume = \"azureFile\"\n\tCinder SupportedVolume = \"cinder\"\n\tCephFS SupportedVolume = \"cephfs\"\n\tConfigMap SupportedVolume = \"configMap\"\n\tCSI SupportedVolume = \"csi\"\n\tDownwardAPI SupportedVolume = \"downwardAPI\"\n\tEmptyDir SupportedVolume = \"emptyDir\"\n\tEphemeral SupportedVolume = \"ephemeral\"\n\tFC SupportedVolume = \"fc\"\n\tFlocker SupportedVolume = \"flocker\"\n\tFlexVolume SupportedVolume = \"flexVolume\"\n\tGitRepo SupportedVolume = \"gitRepo\"\n\tGlusterfs SupportedVolume = \"glusterfs\"\n\tGCEPersistentDisk SupportedVolume = \"gcePersistentDisk\"\n\tHostPath SupportedVolume = \"hostPath\"\n\tISCSI SupportedVolume = \"iscsi\"\n\tLocal SupportedVolume = \"local\"\n\tNFS SupportedVolume = \"nfs\"\n\tPhotonPersistentDisk SupportedVolume = \"photonPersistentDisk\"\n\tPortworxVolume SupportedVolume = \"portworxVolume\"\n\tProjected SupportedVolume = \"projected\"\n\tQuobyte SupportedVolume = \"quobyte\"\n\tRBD SupportedVolume = \"rbd\"\n\tScaleIO SupportedVolume = \"scaleIO\"\n\tSecret SupportedVolume = \"secret\"\n\tStorageOS SupportedVolume = \"storageOS\"\n\tVsphereVolume SupportedVolume = \"vsphereVolume\"\n)\n\nfunc (v *volumeTypeCondition) match(s *structuredVolume) bool {\n\tif len(v.volumeTypes) == 0 {\n\t\treturn true\n\t}\n\n\tfor _, vt := range v.volumeTypes {\n\t\tif vt == s.volumeType {\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n\nfunc (v *volumeTypeCondition) validate() error {\n\t// validate by yamlv3\n\treturn nil\n}\n\nfunc getVolumeTypeFromPV(pv *corev1api.PersistentVolume) SupportedVolume {\n\tif pv == nil {\n\t\treturn \"\"\n\t}\n\n\tif pv.Spec.AWSElasticBlockStore != nil {\n\t\treturn AWSElasticBlockStore\n\t}\n\tif pv.Spec.AzureDisk != nil {\n\t\treturn AzureDisk\n\t}\n\tif pv.Spec.AzureFile != nil {\n\t\treturn AzureFile\n\t}\n\tif pv.Spec.CephFS != nil {\n\t\treturn CephFS\n\t}\n\tif pv.Spec.Cinder != nil {\n\t\treturn Cinder\n\t}\n\tif pv.Spec.CSI != nil {\n\t\treturn CSI\n\t}\n\tif pv.Spec.FC != nil {\n\t\treturn FC\n\t}\n\tif pv.Spec.Flocker != nil {\n\t\treturn Flocker\n\t}\n\tif pv.Spec.FlexVolume != nil {\n\t\treturn FlexVolume\n\t}\n\tif pv.Spec.GCEPersistentDisk != nil {\n\t\treturn GCEPersistentDisk\n\t}\n\tif pv.Spec.Glusterfs != nil {\n\t\treturn Glusterfs\n\t}\n\tif pv.Spec.HostPath != nil {\n\t\treturn HostPath\n\t}\n\tif pv.Spec.ISCSI != nil {\n\t\treturn ISCSI\n\t}\n\tif pv.Spec.Local != nil {\n\t\treturn Local\n\t}\n\tif pv.Spec.NFS != nil {\n\t\treturn NFS\n\t}\n\tif pv.Spec.PhotonPersistentDisk != nil {\n\t\treturn PhotonPersistentDisk\n\t}\n\tif pv.Spec.PortworxVolume != nil {\n\t\treturn PortworxVolume\n\t}\n\tif pv.Spec.Quobyte != nil {\n\t\treturn Quobyte\n\t}\n\tif pv.Spec.RBD != nil {\n\t\treturn RBD\n\t}\n\tif pv.Spec.ScaleIO != nil {\n\t\treturn ScaleIO\n\t}\n\tif pv.Spec.StorageOS != nil {\n\t\treturn StorageOS\n\t}\n\tif pv.Spec.VsphereVolume != nil {\n\t\treturn VsphereVolume\n\t}\n\treturn \"\"\n}\n\nfunc getVolumeTypeFromVolume(vol *corev1api.Volume) SupportedVolume {\n\tif vol == nil {\n\t\treturn \"\"\n\t}\n\n\tif vol.AWSElasticBlockStore != nil {\n\t\treturn AWSElasticBlockStore\n\t}\n\tif vol.AzureDisk != nil {\n\t\treturn AzureDisk\n\t}\n\tif vol.AzureFile != nil {\n\t\treturn AzureFile\n\t}\n\tif vol.CephFS != nil {\n\t\treturn CephFS\n\t}\n\tif vol.Cinder != nil {\n\t\treturn Cinder\n\t}\n\tif vol.CSI != nil {\n\t\treturn CSI\n\t}\n\tif vol.FC != nil {\n\t\treturn FC\n\t}\n\tif vol.Flocker != nil {\n\t\treturn Flocker\n\t}\n\tif vol.FlexVolume != nil {\n\t\treturn FlexVolume\n\t}\n\tif vol.GCEPersistentDisk != nil {\n\t\treturn GCEPersistentDisk\n\t}\n\tif vol.GitRepo != nil {\n\t\treturn GitRepo\n\t}\n\tif vol.Glusterfs != nil {\n\t\treturn Glusterfs\n\t}\n\tif vol.ISCSI != nil {\n\t\treturn ISCSI\n\t}\n\tif vol.NFS != nil {\n\t\treturn NFS\n\t}\n\tif vol.Secret != nil {\n\t\treturn Secret\n\t}\n\tif vol.RBD != nil {\n\t\treturn RBD\n\t}\n\tif vol.DownwardAPI != nil {\n\t\treturn DownwardAPI\n\t}\n\tif vol.ConfigMap != nil {\n\t\treturn ConfigMap\n\t}\n\tif vol.Projected != nil {\n\t\treturn Projected\n\t}\n\tif vol.Ephemeral != nil {\n\t\treturn Ephemeral\n\t}\n\tif vol.FC != nil {\n\t\treturn FC\n\t}\n\tif vol.PhotonPersistentDisk != nil {\n\t\treturn PhotonPersistentDisk\n\t}\n\tif vol.PortworxVolume != nil {\n\t\treturn PortworxVolume\n\t}\n\tif vol.Quobyte != nil {\n\t\treturn Quobyte\n\t}\n\tif vol.ScaleIO != nil {\n\t\treturn ScaleIO\n\t}\n\tif vol.StorageOS != nil {\n\t\treturn StorageOS\n\t}\n\tif vol.VsphereVolume != nil {\n\t\treturn VsphereVolume\n\t}\n\tif vol.HostPath != nil {\n\t\treturn HostPath\n\t}\n\tif vol.EmptyDir != nil {\n\t\treturn EmptyDir\n\t}\n\treturn \"\"\n}\n" }, { "path": "internal/resourcepolicies/volume_types_conditions_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\thttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage resourcepolicies\n\nimport (\n\t\"testing\"\n\n\tcorev1api \"k8s.io/api/core/v1\"\n)\n\nfunc TestGetVolumeTypeFromPV(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tinputPV *corev1api.PersistentVolume\n\t\texpected SupportedVolume\n\t}{\n\t\t{\n\t\t\tname: \"nil PersistentVolume\",\n\t\t\tinputPV: nil,\n\t\t\texpected: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"Test GCEPersistentDisk\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tGCEPersistentDisk: &corev1api.GCEPersistentDiskVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: GCEPersistentDisk,\n\t\t},\n\t\t{\n\t\t\tname: \"Test AWSElasticBlockStore\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tAWSElasticBlockStore: &corev1api.AWSElasticBlockStoreVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: AWSElasticBlockStore,\n\t\t},\n\t\t{\n\t\t\tname: \"Test HostPath\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tHostPath: &corev1api.HostPathVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: HostPath,\n\t\t},\n\t\t{\n\t\t\tname: \"Test Glusterfs\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tGlusterfs: &corev1api.GlusterfsPersistentVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: Glusterfs,\n\t\t},\n\t\t{\n\t\t\tname: \"Test NFS\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tNFS: &corev1api.NFSVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: NFS,\n\t\t},\n\t\t{\n\t\t\tname: \"Test RBD\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tRBD: &corev1api.RBDPersistentVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: RBD,\n\t\t},\n\t\t{\n\t\t\tname: \"Test ISCSI\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tISCSI: &corev1api.ISCSIPersistentVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: ISCSI,\n\t\t},\n\t\t{\n\t\t\tname: \"Test Cinder\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tCinder: &corev1api.CinderPersistentVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: Cinder,\n\t\t},\n\t\t{\n\t\t\tname: \"Test CephFS\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tCephFS: &corev1api.CephFSPersistentVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: CephFS,\n\t\t},\n\t\t{\n\t\t\tname: \"Test FC\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tFC: &corev1api.FCVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: FC,\n\t\t},\n\t\t{\n\t\t\tname: \"Test Flocker\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tFlocker: &corev1api.FlockerVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: Flocker,\n\t\t},\n\t\t{\n\t\t\tname: \"Test FlexVolume\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tFlexVolume: &corev1api.FlexPersistentVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: FlexVolume,\n\t\t},\n\t\t{\n\t\t\tname: \"Test AzureFile\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tAzureFile: &corev1api.AzureFilePersistentVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: AzureFile,\n\t\t},\n\t\t{\n\t\t\tname: \"Test VsphereVolume\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tVsphereVolume: &corev1api.VsphereVirtualDiskVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: VsphereVolume,\n\t\t},\n\t\t{\n\t\t\tname: \"Test Quobyte\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tQuobyte: &corev1api.QuobyteVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: Quobyte,\n\t\t},\n\t\t{\n\t\t\tname: \"Test AzureDisk\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tAzureDisk: &corev1api.AzureDiskVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: AzureDisk,\n\t\t},\n\t\t{\n\t\t\tname: \"Test PhotonPersistentDisk\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tPhotonPersistentDisk: &corev1api.PhotonPersistentDiskVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: PhotonPersistentDisk,\n\t\t},\n\t\t{\n\t\t\tname: \"Test PortworxVolume\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tPortworxVolume: &corev1api.PortworxVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: PortworxVolume,\n\t\t},\n\t\t{\n\t\t\tname: \"Test ScaleIO\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tScaleIO: &corev1api.ScaleIOPersistentVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: ScaleIO,\n\t\t},\n\t\t{\n\t\t\tname: \"Test Local\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tLocal: &corev1api.LocalVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: Local,\n\t\t},\n\t\t{\n\t\t\tname: \"Test StorageOS\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tStorageOS: &corev1api.StorageOSPersistentVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: StorageOS,\n\t\t},\n\t\t{\n\t\t\tname: \"Test CSI\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: CSI,\n\t\t},\n\t\t{\n\t\t\tname: \"Test Unknown Source\",\n\t\t\tinputPV: &corev1api.PersistentVolume{\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{},\n\t\t\t},\n\t\t\texpected: \"\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tresult := getVolumeTypeFromPV(tc.inputPV)\n\t\t\tif result != tc.expected {\n\t\t\t\tt.Errorf(\"Expected %s, but got %s\", tc.expected, result)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetVolumeTypeFromVolume(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tinputVol *corev1api.Volume\n\t\texpected SupportedVolume\n\t}{\n\t\t{\n\t\t\tname: \"nil Volume\",\n\t\t\tinputVol: nil,\n\t\t\texpected: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"Test Unknown Source\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{},\n\t\t\t},\n\t\t\texpected: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"Test HostPath\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tHostPath: &corev1api.HostPathVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: HostPath,\n\t\t},\n\t\t{\n\t\t\tname: \"Test EmptyDir\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tEmptyDir: &corev1api.EmptyDirVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: EmptyDir,\n\t\t},\n\t\t{\n\t\t\tname: \"Test GCEPersistentDisk\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tGCEPersistentDisk: &corev1api.GCEPersistentDiskVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: GCEPersistentDisk,\n\t\t},\n\t\t{\n\t\t\tname: \"Test AWSElasticBlockStore\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tAWSElasticBlockStore: &corev1api.AWSElasticBlockStoreVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: AWSElasticBlockStore,\n\t\t},\n\t\t{\n\t\t\tname: \"Test GitRepo\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tGitRepo: &corev1api.GitRepoVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: GitRepo,\n\t\t},\n\t\t{\n\t\t\tname: \"Test Secret\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tSecret: &corev1api.SecretVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: Secret,\n\t\t},\n\t\t{\n\t\t\tname: \"Test NFS\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tNFS: &corev1api.NFSVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: NFS,\n\t\t},\n\t\t{\n\t\t\tname: \"Test ISCSI\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tISCSI: &corev1api.ISCSIVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: ISCSI,\n\t\t},\n\t\t{\n\t\t\tname: \"Test Glusterfs\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tGlusterfs: &corev1api.GlusterfsVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: Glusterfs,\n\t\t},\n\t\t{\n\t\t\tname: \"Test RBD\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tRBD: &corev1api.RBDVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: RBD,\n\t\t},\n\t\t{\n\t\t\tname: \"Test FlexVolume\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tFlexVolume: &corev1api.FlexVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: FlexVolume,\n\t\t},\n\t\t{\n\t\t\tname: \"Test Cinder\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tCinder: &corev1api.CinderVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: Cinder,\n\t\t},\n\t\t{\n\t\t\tname: \"Test CephFS\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tCephFS: &corev1api.CephFSVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: CephFS,\n\t\t},\n\t\t{\n\t\t\tname: \"Test Flocker\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tFlocker: &corev1api.FlockerVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: Flocker,\n\t\t},\n\t\t{\n\t\t\tname: \"Test DownwardAPI\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tDownwardAPI: &corev1api.DownwardAPIVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: DownwardAPI,\n\t\t},\n\t\t{\n\t\t\tname: \"Test FC\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tFC: &corev1api.FCVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: FC,\n\t\t},\n\t\t{\n\t\t\tname: \"Test AzureFile\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tAzureFile: &corev1api.AzureFileVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: AzureFile,\n\t\t},\n\t\t{\n\t\t\tname: \"Test ConfigMap\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tConfigMap: &corev1api.ConfigMapVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: ConfigMap,\n\t\t},\n\t\t{\n\t\t\tname: \"Test VsphereVolume\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tVsphereVolume: &corev1api.VsphereVirtualDiskVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: VsphereVolume,\n\t\t},\n\t\t{\n\t\t\tname: \"Test Quobyte\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tQuobyte: &corev1api.QuobyteVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: Quobyte,\n\t\t},\n\t\t{\n\t\t\tname: \"Test AzureDisk\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tAzureDisk: &corev1api.AzureDiskVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: AzureDisk,\n\t\t},\n\t\t{\n\t\t\tname: \"Test PhotonPersistentDisk\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tPhotonPersistentDisk: &corev1api.PhotonPersistentDiskVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: PhotonPersistentDisk,\n\t\t},\n\t\t{\n\t\t\tname: \"Test Projected\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tProjected: &corev1api.ProjectedVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: Projected,\n\t\t},\n\t\t{\n\t\t\tname: \"Test PortworxVolume\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tPortworxVolume: &corev1api.PortworxVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: PortworxVolume,\n\t\t},\n\t\t{\n\t\t\tname: \"Test ScaleIO\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tScaleIO: &corev1api.ScaleIOVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: ScaleIO,\n\t\t},\n\t\t{\n\t\t\tname: \"Test StorageOS\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tStorageOS: &corev1api.StorageOSVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: StorageOS,\n\t\t},\n\t\t{\n\t\t\tname: \"Test CSI\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tCSI: &corev1api.CSIVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: CSI,\n\t\t},\n\t\t{\n\t\t\tname: \"Test Ephemeral\",\n\t\t\tinputVol: &corev1api.Volume{\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tEphemeral: &corev1api.EphemeralVolumeSource{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: Ephemeral,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tresult := getVolumeTypeFromVolume(tc.inputVol)\n\t\t\tif result != tc.expected {\n\t\t\t\tt.Errorf(\"Expected %s, but got %s\", tc.expected, result)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "internal/restartabletest/restartable_delegate.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage restartabletest\n\nimport (\n\t\"reflect\"\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n)\n\ntype MockRestartableProcess struct {\n\tmock.Mock\n}\n\nfunc (rp *MockRestartableProcess) AddReinitializer(key process.KindAndName, r process.Reinitializer) {\n\trp.Called(key, r)\n}\n\nfunc (rp *MockRestartableProcess) Reset() error {\n\targs := rp.Called()\n\treturn args.Error(0)\n}\n\nfunc (rp *MockRestartableProcess) ResetIfNeeded() error {\n\targs := rp.Called()\n\treturn args.Error(0)\n}\n\nfunc (rp *MockRestartableProcess) GetByKindAndName(key process.KindAndName) (any, error) {\n\targs := rp.Called(key)\n\treturn args.Get(0), args.Error(1)\n}\n\nfunc (rp *MockRestartableProcess) Stop() {\n\trp.Called()\n}\n\ntype RestartableDelegateTest struct {\n\tFunction string\n\tInputs []any\n\tExpectedErrorOutputs []any\n\tExpectedDelegateOutputs []any\n}\n\ntype Mockable interface {\n\tTest(t mock.TestingT)\n\tOn(method string, args ...any) *mock.Call\n\tAssertExpectations(t mock.TestingT) bool\n}\n\nfunc RunRestartableDelegateTests(\n\tt *testing.T,\n\tkind common.PluginKind,\n\tnewRestartable func(key process.KindAndName, p process.RestartableProcess) any,\n\tnewMock func() Mockable,\n\ttests ...RestartableDelegateTest,\n) {\n\tt.Helper()\n\tfor _, tc := range tests {\n\t\tt.Run(tc.Function, func(t *testing.T) {\n\t\t\tp := new(MockRestartableProcess)\n\t\t\tp.Test(t)\n\t\t\tdefer p.AssertExpectations(t)\n\n\t\t\t// getDelegate error\n\t\t\tp.On(\"ResetIfNeeded\").Return(errors.Errorf(\"reset error\")).Once()\n\t\t\tname := \"delegateName\"\n\t\t\tkey := process.KindAndName{Kind: kind, Name: name}\n\t\t\tr := newRestartable(key, p)\n\n\t\t\t// Get the method we're going to call using reflection\n\t\t\tmethod := reflect.ValueOf(r).MethodByName(tc.Function)\n\t\t\trequire.NotEmpty(t, method)\n\n\t\t\t// Convert the test case inputs ([]any) to []reflect.Value\n\t\t\tvar inputValues []reflect.Value\n\t\t\tfor i := range tc.Inputs {\n\t\t\t\tinputValues = append(inputValues, reflect.ValueOf(tc.Inputs[i]))\n\t\t\t}\n\n\t\t\t// Invoke the method being tested\n\t\t\tactual := method.Call(inputValues)\n\n\t\t\t// This Function asserts that the actual outputs match the expected outputs\n\t\t\tcheckOutputs := func(expected []any, actual []reflect.Value) {\n\t\t\t\trequire.Len(t, actual, len(expected))\n\n\t\t\t\tfor i := range actual {\n\t\t\t\t\t// Get the underlying value from the reflect.Value\n\t\t\t\t\ta := actual[i].Interface()\n\n\t\t\t\t\t// Check if it's an error\n\t\t\t\t\tactualErr, actualErrOk := a.(error)\n\t\t\t\t\t// Check if the expected output element is an error\n\t\t\t\t\texpectedErr, expectedErrOk := expected[i].(error)\n\t\t\t\t\t// If both are errors, use EqualError\n\t\t\t\t\tif actualErrOk && expectedErrOk {\n\t\t\t\t\t\trequire.EqualError(t, actualErr, expectedErr.Error())\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\n\t\t\t\t\t// If Function returns nil as struct return type, we cannot just\n\t\t\t\t\t// compare the interface to nil as its type will not be nil,\n\t\t\t\t\t// only the value will be\n\t\t\t\t\tif expected[i] == nil && reflect.ValueOf(a).Kind() == reflect.Ptr {\n\t\t\t\t\t\tassert.True(t, reflect.ValueOf(a).IsNil())\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\n\t\t\t\t\t// Otherwise, use plain Equal\n\t\t\t\t\tassert.Equal(t, expected[i], a)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\t// Make sure we get what we expected when getDelegate returned an error\n\t\t\tcheckOutputs(tc.ExpectedErrorOutputs, actual)\n\n\t\t\t// Invoke delegate, make sure all returned values are passed through\n\t\t\tp.On(\"ResetIfNeeded\").Return(nil)\n\n\t\t\tdelegate := newMock()\n\t\t\tdelegate.Test(t)\n\t\t\tdefer delegate.AssertExpectations(t)\n\n\t\t\tp.On(\"GetByKindAndName\", key).Return(delegate, nil)\n\n\t\t\t// Set up the mocked method in the delegate\n\t\t\tdelegate.On(tc.Function, tc.Inputs...).Return(tc.ExpectedDelegateOutputs...)\n\n\t\t\t// Invoke the method being tested\n\t\t\tactual = method.Call(inputValues)\n\n\t\t\t// Make sure we get what we expected when invoking the delegate\n\t\t\tcheckOutputs(tc.ExpectedDelegateOutputs, actual)\n\t\t})\n\t}\n}\n" }, { "path": "internal/storage/storagelocation.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage storage\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// DefaultBackupLocationInfo holds server default backup storage location information\ntype DefaultBackupLocationInfo struct {\n\t// StorageLocation is the name of the backup storage location designated as the default on the server side.\n\t// Deprecated TODO(2.0)\n\tStorageLocation string\n\t// ServerValidationFrequency is the server default validation frequency for all backup storage locations\n\tServerValidationFrequency time.Duration\n}\n\n// IsReadyToValidate calculates if a given backup storage location is ready to be validated.\n//\n// Rules:\n// Users can choose a validation frequency per location. This will override the server's default value\n// To skip/stop validation, set the frequency to zero\n// This will always return \"true\" for the first attempt at validating a location, regardless of its validation frequency setting\n// Otherwise, it returns \"ready\" only when NOW is equal to or after the next validation time\n// (next validation time: last validation time + validation frequency)\nfunc IsReadyToValidate(bslValidationFrequency *metav1.Duration, lastValidationTime *metav1.Time, serverValidationFrequency time.Duration, log logrus.FieldLogger) bool {\n\tvalidationFrequency := serverValidationFrequency\n\t// If the bsl validation frequency is not specifically set, skip this block and continue, using the server's default\n\tif bslValidationFrequency != nil {\n\t\tvalidationFrequency = bslValidationFrequency.Duration\n\t}\n\n\tif validationFrequency < 0 {\n\t\tlog.Debugf(\"Validation period must be non-negative, changing from %d to %d\", validationFrequency, serverValidationFrequency)\n\t\tvalidationFrequency = serverValidationFrequency\n\t}\n\n\tlastValidation := lastValidationTime\n\tif lastValidation == nil {\n\t\t// Regardless of validation frequency, we want to validate all BSLs at least once.\n\t\treturn true\n\t}\n\n\tif validationFrequency == 0 {\n\t\t// Validation was disabled so return false.\n\t\tlog.Debug(\"Validation period for this backup location is set to 0, skipping validation\")\n\t\treturn false\n\t}\n\n\t// We want to validate BSL only if the set validation frequency/ interval has elapsed.\n\tnextValidation := lastValidation.Add(validationFrequency) // next validation time: last validation time + validation frequency\n\treturn !time.Now().UTC().Before(nextValidation) // ready only when NOW is equal to or after the next validation time\n}\n\n// ListBackupStorageLocations verifies if there are any backup storage locations.\n// For all purposes, if either there is an error while attempting to fetch items or\n// if there are no items an error would be returned since the functioning of the system\n// would be haulted.\nfunc ListBackupStorageLocations(ctx context.Context, kbClient client.Client, namespace string) (velerov1api.BackupStorageLocationList, error) {\n\tvar locations velerov1api.BackupStorageLocationList\n\tif err := kbClient.List(ctx, &locations, &client.ListOptions{\n\t\tNamespace: namespace,\n\t}); err != nil {\n\t\treturn velerov1api.BackupStorageLocationList{}, err\n\t}\n\n\tif len(locations.Items) == 0 {\n\t\treturn velerov1api.BackupStorageLocationList{}, errors.New(\"no backup storage locations found\")\n\t}\n\n\treturn locations, nil\n}\n\nfunc GetDefaultBackupStorageLocations(ctx context.Context, kbClient client.Client, namespace string) (*velerov1api.BackupStorageLocationList, error) {\n\tlocations := new(velerov1api.BackupStorageLocationList)\n\tdefaultLocations := new(velerov1api.BackupStorageLocationList)\n\tif err := kbClient.List(context.Background(), locations, &client.ListOptions{Namespace: namespace}); err != nil {\n\t\treturn defaultLocations, errors.Wrapf(err, \"failed to list backup storage locations in namespace %s\", namespace)\n\t}\n\n\tfor _, location := range locations.Items {\n\t\tif location.Spec.Default {\n\t\t\tdefaultLocations.Items = append(defaultLocations.Items, location)\n\t\t}\n\t}\n\treturn defaultLocations, nil\n}\n" }, { "path": "internal/storage/storagelocation_test.go", "content": "/*Copyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage storage\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t. \"github.com/onsi/gomega\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n)\n\nfunc TestIsReadyToValidate(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbslValidationFrequency *metav1.Duration\n\t\tlastValidationTime *metav1.Time\n\t\tdefaultLocationInfo DefaultBackupLocationInfo\n\t\tready bool\n\t}{\n\t\t{\n\t\t\tname: \"validate when true when validation frequency is zero and lastValidationTime is nil\",\n\t\t\tbslValidationFrequency: &metav1.Duration{Duration: 0},\n\t\t\tdefaultLocationInfo: DefaultBackupLocationInfo{\n\t\t\t\tServerValidationFrequency: 0,\n\t\t\t},\n\t\t\tready: true,\n\t\t},\n\t\t{\n\t\t\tname: \"don't validate when false when validation is disabled and lastValidationTime is not nil\",\n\t\t\tbslValidationFrequency: &metav1.Duration{Duration: 0},\n\t\t\tlastValidationTime: &metav1.Time{Time: time.Now()},\n\t\t\tdefaultLocationInfo: DefaultBackupLocationInfo{\n\t\t\t\tServerValidationFrequency: 0,\n\t\t\t},\n\t\t\tready: false,\n\t\t},\n\t\t{\n\t\t\tname: \"validate as per location setting, as that takes precedence, and always if it has never been validated before regardless of the frequency setting\",\n\t\t\tbslValidationFrequency: &metav1.Duration{Duration: 1 * time.Hour},\n\t\t\tdefaultLocationInfo: DefaultBackupLocationInfo{\n\t\t\t\tServerValidationFrequency: 0,\n\t\t\t},\n\t\t\tready: true,\n\t\t},\n\t\t{\n\t\t\tname: \"don't validate as per location setting, as it is set to zero and that takes precedence\",\n\t\t\tbslValidationFrequency: &metav1.Duration{Duration: 0},\n\t\t\tdefaultLocationInfo: DefaultBackupLocationInfo{\n\t\t\t\tServerValidationFrequency: 1,\n\t\t\t},\n\t\t\tlastValidationTime: &metav1.Time{Time: time.Now()},\n\t\t\tready: false,\n\t\t},\n\t\t{\n\t\t\tname: \"validate as per default setting when location setting is not set\",\n\t\t\tdefaultLocationInfo: DefaultBackupLocationInfo{\n\t\t\t\tServerValidationFrequency: 1,\n\t\t\t},\n\t\t\tready: true,\n\t\t},\n\t\t{\n\t\t\tname: \"don't validate when default setting is set to zero and the location setting is not set\",\n\t\t\tdefaultLocationInfo: DefaultBackupLocationInfo{\n\t\t\t\tServerValidationFrequency: 0,\n\t\t\t},\n\t\t\tlastValidationTime: &metav1.Time{Time: time.Now()},\n\t\t\tready: false,\n\t\t},\n\t\t{\n\t\t\tname: \"don't validate when now is before the NEXT validation time (validation frequency + last validation time)\",\n\t\t\tbslValidationFrequency: &metav1.Duration{Duration: 1 * time.Second},\n\t\t\tlastValidationTime: &metav1.Time{Time: time.Now()},\n\t\t\tdefaultLocationInfo: DefaultBackupLocationInfo{\n\t\t\t\tServerValidationFrequency: 0,\n\t\t\t},\n\t\t\tready: false,\n\t\t},\n\t\t{\n\t\t\tname: \"validate when now is equal to the NEXT validation time (validation frequency + last validation time)\",\n\t\t\tbslValidationFrequency: &metav1.Duration{Duration: 1 * time.Second},\n\t\t\tlastValidationTime: &metav1.Time{Time: time.Now().Add(-1 * time.Second)},\n\t\t\tdefaultLocationInfo: DefaultBackupLocationInfo{\n\t\t\t\tServerValidationFrequency: 0,\n\t\t\t},\n\t\t\tready: true,\n\t\t},\n\t\t{\n\t\t\tname: \"validate when now is after the NEXT validation time (validation frequency + last validation time)\",\n\t\t\tbslValidationFrequency: &metav1.Duration{Duration: 1 * time.Second},\n\t\t\tlastValidationTime: &metav1.Time{Time: time.Now().Add(-2 * time.Second)},\n\t\t\tdefaultLocationInfo: DefaultBackupLocationInfo{\n\t\t\t\tServerValidationFrequency: 0,\n\t\t\t},\n\t\t\tready: true,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tg := NewWithT(t)\n\t\t\tlog := velerotest.NewLogger()\n\t\t\tactual := IsReadyToValidate(tt.bslValidationFrequency, tt.lastValidationTime, tt.defaultLocationInfo.ServerValidationFrequency, log)\n\t\t\tg.Expect(actual).To(BeIdenticalTo(tt.ready))\n\t\t})\n\t}\n}\n\nfunc TestListBackupStorageLocations(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackupLocations *velerov1api.BackupStorageLocationList\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"1 existing location does not return an error\",\n\t\t\tbackupLocations: &velerov1api.BackupStorageLocationList{\n\t\t\t\tItems: []velerov1api.BackupStorageLocation{\n\t\t\t\t\t*builder.ForBackupStorageLocation(\"ns-1\", \"location-1\").Result(),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"multiple existing location does not return an error\",\n\t\t\tbackupLocations: &velerov1api.BackupStorageLocationList{\n\t\t\t\tItems: []velerov1api.BackupStorageLocation{\n\t\t\t\t\t*builder.ForBackupStorageLocation(\"ns-1\", \"location-1\").Result(),\n\t\t\t\t\t*builder.ForBackupStorageLocation(\"ns-1\", \"location-2\").Result(),\n\t\t\t\t\t*builder.ForBackupStorageLocation(\"ns-1\", \"location-3\").Result(),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"no existing locations returns an error\",\n\t\t\tbackupLocations: &velerov1api.BackupStorageLocationList{},\n\t\t\texpectError: true,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tg := NewWithT(t)\n\n\t\t\tclient := fake.NewClientBuilder().WithScheme(util.VeleroScheme).WithRuntimeObjects(tt.backupLocations).Build()\n\t\t\tif tt.expectError {\n\t\t\t\t_, err := ListBackupStorageLocations(t.Context(), client, \"ns-1\")\n\t\t\t\tg.Expect(err).To(HaveOccurred())\n\t\t\t} else {\n\t\t\t\t_, err := ListBackupStorageLocations(t.Context(), client, \"ns-1\")\n\t\t\t\tg.Expect(err).ToNot(HaveOccurred())\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "internal/velero/images.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage velero\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/buildinfo\"\n)\n\n// Use Dockerhub as the default registry if the build process didn't supply a registry\nfunc imageRegistry() string {\n\tif buildinfo.ImageRegistry == \"\" {\n\t\treturn \"velero\"\n\t}\n\treturn buildinfo.ImageRegistry\n}\n\n// ImageTag returns the image tag that should be used by Velero images.\n// It uses the Version from the buildinfo or \"latest\" if the build process didn't supply a version.\nfunc ImageTag() string {\n\tif buildinfo.Version == \"\" {\n\t\treturn \"latest\"\n\t}\n\treturn buildinfo.Version\n}\n\n// DefaultVeleroImage returns the default container image to use for this version of Velero.\nfunc DefaultVeleroImage() string {\n\treturn fmt.Sprintf(\"%s/%s:%s\", imageRegistry(), \"velero\", ImageTag())\n}\n" }, { "path": "internal/velero/images_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage velero\n\nimport (\n\t\"fmt\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/buildinfo\"\n)\n\nfunc TestImageTag(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tbuildInfoVersion string\n\t\twant string\n\t}{\n\t\t{\n\t\t\tname: \"tag is latest when buildinfo.Version is empty\",\n\t\t\twant: \"latest\",\n\t\t},\n\t\t{\n\t\t\tname: \"tag is buildinfo.Version when not empty\",\n\t\t\tbuildInfoVersion: \"custom-build-version\",\n\t\t\twant: \"custom-build-version\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\toriginalVersion := buildinfo.Version\n\t\t\tbuildinfo.Version = tc.buildInfoVersion\n\t\t\tdefer func() {\n\t\t\t\tbuildinfo.Version = originalVersion\n\t\t\t}()\n\n\t\t\tassert.Equal(t, tc.want, ImageTag())\n\t\t})\n\t}\n}\n\nfunc TestImageRegistry(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tbuildInfoRegistry string\n\t\twant string\n\t}{\n\t\t{\n\t\t\tname: \"registry is velero when buildinfo.ImageRegistry is empty\",\n\t\t\twant: \"velero\",\n\t\t},\n\t\t{\n\t\t\tname: \"registry is buildinfo.ImageRegistry when not empty\",\n\t\t\tbuildInfoRegistry: \"custom-build-image-registry\",\n\t\t\twant: \"custom-build-image-registry\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\toriginalImageRegistry := buildinfo.ImageRegistry\n\t\t\tbuildinfo.ImageRegistry = tc.buildInfoRegistry\n\t\t\tdefer func() {\n\t\t\t\tbuildinfo.ImageRegistry = originalImageRegistry\n\t\t\t}()\n\n\t\t\tassert.Equal(t, tc.want, imageRegistry())\n\t\t})\n\t}\n}\n\nfunc testDefaultImage(t *testing.T, defaultImageFn func() string, imageName string) {\n\tt.Helper()\n\ttestCases := []struct {\n\t\tname string\n\t\tbuildInfoVersion string\n\t\tbuildInfoRegistry string\n\t\twant string\n\t}{\n\t\t{\n\t\t\tname: \"image uses velero as registry and latest as tag when buildinfo.ImageRegistry and buildinfo.Version are empty\",\n\t\t\twant: fmt.Sprintf(\"velero/%s:latest\", imageName),\n\t\t},\n\t\t{\n\t\t\tname: \"image uses buildinfo.ImageRegistry as registry when not empty\",\n\t\t\tbuildInfoRegistry: \"custom-build-image-registry\",\n\t\t\twant: fmt.Sprintf(\"custom-build-image-registry/%s:latest\", imageName),\n\t\t},\n\t\t{\n\t\t\tname: \"image uses buildinfo.Version as tag when not empty\",\n\t\t\tbuildInfoVersion: \"custom-build-version\",\n\t\t\twant: fmt.Sprintf(\"velero/%s:custom-build-version\", imageName),\n\t\t},\n\t\t{\n\t\t\tname: \"image uses both buildinfo.ImageRegistry and buildinfo.Version when not empty\",\n\t\t\tbuildInfoRegistry: \"custom-build-image-registry\",\n\t\t\tbuildInfoVersion: \"custom-build-version\",\n\t\t\twant: fmt.Sprintf(\"custom-build-image-registry/%s:custom-build-version\", imageName),\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\toriginalImageRegistry := buildinfo.ImageRegistry\n\t\t\toriginalVersion := buildinfo.Version\n\t\t\tbuildinfo.ImageRegistry = tc.buildInfoRegistry\n\t\t\tbuildinfo.Version = tc.buildInfoVersion\n\t\t\tdefer func() {\n\t\t\t\tbuildinfo.ImageRegistry = originalImageRegistry\n\t\t\t\tbuildinfo.Version = originalVersion\n\t\t\t}()\n\n\t\t\tassert.Equal(t, tc.want, defaultImageFn())\n\t\t})\n\t}\n}\n\nfunc TestDefaultVeleroImage(t *testing.T) {\n\ttestDefaultImage(t, DefaultVeleroImage, \"velero\")\n}\n" }, { "path": "internal/velero/serverstatusrequest.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage velero\n\nimport (\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n)\n\ntype PluginLister interface {\n\t// List returns all PluginIdentifiers for kind.\n\tList(kind common.PluginKind) []framework.PluginIdentifier\n}\n\n// GetInstalledPluginInfo returns a list of installed plugins\nfunc GetInstalledPluginInfo(pluginLister PluginLister) []velerov1api.PluginInfo {\n\tvar plugins []velerov1api.PluginInfo\n\tfor _, v := range common.AllPluginKinds() {\n\t\tlist := pluginLister.List(v)\n\t\tfor _, plugin := range list {\n\t\t\tpluginInfo := velerov1api.PluginInfo{\n\t\t\t\tName: plugin.Name,\n\t\t\t\tKind: plugin.Kind.String(),\n\t\t\t}\n\t\t\tplugins = append(plugins, pluginInfo)\n\t\t}\n\t}\n\treturn plugins\n}\n" }, { "path": "internal/volume/native_snapshot.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage volume\n\n// Snapshot stores information about a persistent volume snapshot taken as\n// part of a Velero backup.\ntype Snapshot struct {\n\tSpec SnapshotSpec `json:\"spec\"`\n\n\tStatus SnapshotStatus `json:\"status\"`\n}\n\ntype SnapshotSpec struct {\n\t// BackupName is the name of the Velero backup this snapshot\n\t// is associated with.\n\tBackupName string `json:\"backupName\"`\n\n\t// BackupUID is the UID of the Velero backup this snapshot\n\t// is associated with.\n\tBackupUID string `json:\"backupUID\"`\n\n\t// Location is the name of the VolumeSnapshotLocation where this snapshot is stored.\n\tLocation string `json:\"location\"`\n\n\t// PersistentVolumeName is the Kubernetes name for the volume.\n\tPersistentVolumeName string `json:\"persistentVolumeName\"`\n\n\t// ProviderVolumeID is the provider's ID for the volume.\n\tProviderVolumeID string `json:\"providerVolumeID\"`\n\n\t// VolumeType is the type of the disk/volume in the cloud provider\n\t// API.\n\tVolumeType string `json:\"volumeType\"`\n\n\t// VolumeAZ is the where the volume is provisioned\n\t// in the cloud provider.\n\tVolumeAZ string `json:\"volumeAZ,omitempty\"`\n\n\t// VolumeIOPS is the optional value of provisioned IOPS for the\n\t// disk/volume in the cloud provider API.\n\tVolumeIOPS *int64 `json:\"volumeIOPS,omitempty\"`\n}\n\ntype SnapshotStatus struct {\n\t// ProviderSnapshotID is the ID of the snapshot taken in the cloud\n\t// provider API of this volume.\n\tProviderSnapshotID string `json:\"providerSnapshotID,omitempty\"`\n\n\t// Phase is the current state of the VolumeSnapshot.\n\tPhase SnapshotPhase `json:\"phase,omitempty\"`\n}\n\n// SnapshotPhase is the lifecycle phase of a Velero volume snapshot.\ntype SnapshotPhase string\n\nconst (\n\t// SnapshotPhaseNew means the volume snapshot has been created but not\n\t// yet processed by the VolumeSnapshotController.\n\tSnapshotPhaseNew SnapshotPhase = \"New\"\n\n\t// SnapshotPhaseCompleted means the volume snapshot was successfully created and can be restored from..\n\tSnapshotPhaseCompleted SnapshotPhase = \"Completed\"\n\n\t// SnapshotPhaseFailed means the volume snapshot was unable to execute.\n\tSnapshotPhaseFailed SnapshotPhase = \"Failed\"\n)\n" }, { "path": "internal/volume/snapshotlocation.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage volume\n\nimport (\n\t\"github.com/pkg/errors\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// UpdateVolumeSnapshotLocationWithCredentialConfig adds the credentials file path to the config\n// if the VSL specifies a credential\nfunc UpdateVolumeSnapshotLocationWithCredentialConfig(location *velerov1api.VolumeSnapshotLocation, credentialStore credentials.FileStore) error {\n\tif location.Spec.Config == nil {\n\t\tlocation.Spec.Config = make(map[string]string)\n\t}\n\t// If the VSL specifies a credential, fetch its path on disk and pass to\n\t// plugin via the config.\n\tif location.Spec.Credential != nil && credentialStore != nil {\n\t\tcredsFile, err := credentialStore.Path(location.Spec.Credential)\n\t\tif err != nil {\n\t\t\treturn errors.Wrap(err, \"unable to get credentials\")\n\t\t}\n\n\t\tlocation.Spec.Config[\"credentialsFile\"] = credsFile\n\t}\n\treturn nil\n}\n" }, { "path": "internal/volume/utils.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage volume\n\nimport (\n\t\"regexp\"\n)\n\n// it has to have the same value as \"github.com/vmware-tanzu/velero/pkg/restore\".ItemRestoreResultCreated\nconst itemRestoreResultCreated = \"created\"\n\n// RestoredPVCFromRestoredResourceList returns a set of PVCs that were restored from the given restoredResourceList.\nfunc RestoredPVCFromRestoredResourceList(restoredResourceList map[string][]string) map[string]struct{} {\n\tpvcKey := \"v1/PersistentVolumeClaim\"\n\tpvcList := make(map[string]struct{})\n\n\tfor _, pvc := range restoredResourceList[pvcKey] {\n\t\t// the format of pvc string in restoredResourceList is like: \"namespace/pvcName(status)\"\n\t\t// extract the substring before \"(created)\" if the status in rightmost Parenthesis is \"created\"\n\t\tr := regexp.MustCompile(`\\(([^)]+)\\)`)\n\t\tmatches := r.FindAllStringSubmatch(pvc, -1)\n\t\tif len(matches) > 0 && matches[len(matches)-1][1] == itemRestoreResultCreated {\n\t\t\tpvcList[pvc[:len(pvc)-len(\"(created)\")]] = struct{}{}\n\t\t}\n\t}\n\n\treturn pvcList\n}\n" }, { "path": "internal/volume/utils_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage volume\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestGetRestoredPVCFromRestoredResourceList(t *testing.T) {\n\t// test empty list\n\trestoredResourceList := map[string][]string{}\n\tactual := RestoredPVCFromRestoredResourceList(restoredResourceList)\n\tassert.Empty(t, actual)\n\n\t// test no match\n\trestoredResourceList = map[string][]string{\n\t\t\"v1/PersistentVolumeClaim\": {\n\t\t\t\"namespace1/pvc1(updated)\",\n\t\t},\n\t\t\"v1/PersistentVolume\": {\n\t\t\t\"namespace1/pv(created)\",\n\t\t},\n\t}\n\tactual = RestoredPVCFromRestoredResourceList(restoredResourceList)\n\tassert.Empty(t, actual)\n\n\t// test matches\n\trestoredResourceList = map[string][]string{\n\t\t\"v1/PersistentVolumeClaim\": {\n\t\t\t\"namespace1/pvc1(created)\",\n\t\t\t\"namespace2/pvc2(updated)\",\n\t\t\t\"namespace3/pvc(3)(created)\",\n\t\t},\n\t}\n\texpected := map[string]struct{}{\n\t\t\"namespace1/pvc1\": {},\n\t\t\"namespace3/pvc(3)\": {},\n\t}\n\tactual = RestoredPVCFromRestoredResourceList(restoredResourceList)\n\tassert.Equal(t, expected, actual)\n}\n" }, { "path": "internal/volume/volumes_information.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage volume\n\nimport (\n\t\"context\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/features\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n)\n\ntype Method string\n\nconst (\n\tNativeSnapshot Method = \"NativeSnapshot\"\n\tPodVolumeBackup Method = \"PodVolumeBackup\"\n\tCSISnapshot Method = \"CSISnapshot\"\n\tPodVolumeRestore Method = \"PodVolumeRestore\"\n)\n\nconst (\n\tFieldValueIsUnknown string = \"unknown\"\n\tveleroDatamover string = \"velero\"\n)\n\ntype BackupVolumeInfo struct {\n\t// The PVC's name.\n\tPVCName string `json:\"pvcName,omitempty\"`\n\n\t// The PVC's namespace\n\tPVCNamespace string `json:\"pvcNamespace,omitempty\"`\n\n\t// The PV name.\n\tPVName string `json:\"pvName,omitempty\"`\n\n\t// The way the volume data is backed up. The valid value includes `VeleroNativeSnapshot`, `PodVolumeBackup` and `CSISnapshot`.\n\tBackupMethod Method `json:\"backupMethod,omitempty\"`\n\n\t// Whether the volume's snapshot data is moved to specified storage.\n\tSnapshotDataMoved bool `json:\"snapshotDataMoved\"`\n\n\t// Whether the local snapshot is preserved after snapshot is moved.\n\t// The local snapshot may be a result of CSI snapshot backup(no data movement)\n\t// or a CSI snapshot data movement plus preserve local snapshot.\n\tPreserveLocalSnapshot bool `json:\"preserveLocalSnapshot\"`\n\n\t// Whether the Volume is skipped in this backup.\n\tSkipped bool `json:\"skipped\"`\n\n\t// The reason for the volume is skipped in the backup.\n\tSkippedReason string `json:\"skippedReason,omitempty\"`\n\n\t// Snapshot starts timestamp.\n\tStartTimestamp *metav1.Time `json:\"startTimestamp,omitempty\"`\n\n\t// Snapshot completes timestamp.\n\tCompletionTimestamp *metav1.Time `json:\"completionTimestamp,omitempty\"`\n\n\t// Whether the volume data is backed up successfully.\n\tResult VolumeResult `json:\"result,omitempty\"`\n\n\tCSISnapshotInfo *CSISnapshotInfo `json:\"csiSnapshotInfo,omitempty\"`\n\tSnapshotDataMovementInfo *SnapshotDataMovementInfo `json:\"snapshotDataMovementInfo,omitempty\"`\n\tNativeSnapshotInfo *NativeSnapshotInfo `json:\"nativeSnapshotInfo,omitempty\"`\n\tPVBInfo *PodVolumeInfo `json:\"pvbInfo,omitempty\"`\n\tPVInfo *PVInfo `json:\"pvInfo,omitempty\"`\n}\n\ntype VolumeResult string\n\nconst (\n\tVolumeResultSucceeded VolumeResult = \"succeeded\"\n\tVolumeResultFailed VolumeResult = \"failed\"\n\t//VolumeResultCanceled VolumeResult = \"canceled\"\n)\n\ntype RestoreVolumeInfo struct {\n\t// The name of the restored PVC\n\tPVCName string `json:\"pvcName,omitempty\"`\n\n\t// The namespace of the restored PVC\n\tPVCNamespace string `json:\"pvcNamespace,omitempty\"`\n\n\t// The name of the restored PV, it is possible that in one item there is only PVC or PV info.\n\t// But if both PVC and PV exist in one item of volume info, they should matched, and if the PV is bound to a PVC,\n\t// they should coexist in one item.\n\tPVName string `json:\"pvName,omitempty\"`\n\n\t// The way the volume data is restored.\n\tRestoreMethod Method `json:\"restoreMethod,omitempty\"`\n\n\t// Whether the volume's data are restored via data movement\n\tSnapshotDataMoved bool `json:\"snapshotDataMoved\"`\n\n\tCSISnapshotInfo *CSISnapshotInfo `json:\"csiSnapshotInfo,omitempty\"`\n\tSnapshotDataMovementInfo *SnapshotDataMovementInfo `json:\"snapshotDataMovementInfo,omitempty\"`\n\tNativeSnapshotInfo *NativeSnapshotInfo `json:\"nativeSnapshotInfo,omitempty\"`\n\tPVRInfo *PodVolumeInfo `json:\"pvrInfo,omitempty\"`\n}\n\n// CSISnapshotInfo is used for displaying the CSI snapshot status\ntype CSISnapshotInfo struct {\n\t// It's the storage provider's snapshot ID for CSI.\n\tSnapshotHandle string `json:\"snapshotHandle\"`\n\n\t// The snapshot corresponding volume size.\n\tSize int64 `json:\"size\"`\n\n\t// The name of the CSI driver.\n\tDriver string `json:\"driver\"`\n\n\t// The name of the VolumeSnapshotContent.\n\tVSCName string `json:\"vscName\"`\n\n\t// The Async Operation's ID.\n\tOperationID string `json:\"operationID,omitempty\"`\n\n\t// The VolumeSnapshot's Status.ReadyToUse value\n\tReadyToUse *bool\n}\n\n// SnapshotDataMovementInfo is used for displaying the snapshot data mover status.\ntype SnapshotDataMovementInfo struct {\n\t// The data mover used by the backup. The valid values are `velero` and ``(equals to `velero`).\n\tDataMover string `json:\"dataMover\"`\n\n\t// The type of the uploader that uploads the snapshot data. The valid values are `kopia` and `restic`.\n\tUploaderType string `json:\"uploaderType\"`\n\n\t// The name or ID of the snapshot associated object(SAO).\n\t// SAO is used to support local snapshots for the snapshot data mover,\n\t// e.g. it could be a VolumeSnapshot for CSI snapshot data movement.\n\tRetainedSnapshot string `json:\"retainedSnapshot,omitempty\"`\n\n\t// It's the filesystem repository's snapshot ID.\n\tSnapshotHandle string `json:\"snapshotHandle\"`\n\n\t// The Async Operation's ID.\n\tOperationID string `json:\"operationID\"`\n\n\t// Moved snapshot data size.\n\tSize int64 `json:\"size\"`\n\n\t// Moved snapshot incremental size.\n\tIncrementalSize int64 `json:\"incrementalSize,omitempty\"`\n\n\t// The DataUpload's Status.Phase value\n\tPhase velerov2alpha1.DataUploadPhase\n}\n\n// NativeSnapshotInfo is used for displaying the Velero native snapshot status.\n// A Velero Native Snapshot is a cloud storage snapshot taken by the Velero native\n// plugins, e.g. velero-plugin-for-aws, velero-plugin-for-gcp, and\n// velero-plugin-for-microsoft-azure.\ntype NativeSnapshotInfo struct {\n\t// It's the storage provider's snapshot ID for the Velero-native snapshot.\n\tSnapshotHandle string `json:\"snapshotHandle\"`\n\n\t// The cloud provider snapshot volume type.\n\tVolumeType string `json:\"volumeType\"`\n\n\t// The cloud provider snapshot volume's availability zones.\n\tVolumeAZ string `json:\"volumeAZ\"`\n\n\t// The cloud provider snapshot volume's IOPS.\n\tIOPS string `json:\"iops\"`\n\n\t// The NativeSnapshot's Status.Phase value\n\tPhase SnapshotPhase\n}\n\nfunc newNativeSnapshotInfo(s *Snapshot) *NativeSnapshotInfo {\n\tvar iops int64\n\tif s.Spec.VolumeIOPS != nil {\n\t\tiops = *s.Spec.VolumeIOPS\n\t}\n\treturn &NativeSnapshotInfo{\n\t\tSnapshotHandle: s.Status.ProviderSnapshotID,\n\t\tVolumeType: s.Spec.VolumeType,\n\t\tVolumeAZ: s.Spec.VolumeAZ,\n\t\tIOPS: strconv.FormatInt(iops, 10),\n\t\tPhase: s.Status.Phase,\n\t}\n}\n\n// PodVolumeInfo is used for displaying the PodVolumeBackup/PodVolumeRestore snapshot status.\ntype PodVolumeInfo struct {\n\t// It's the file-system uploader's snapshot ID for PodVolumeBackup/PodVolumeRestore.\n\tSnapshotHandle string `json:\"snapshotHandle,omitempty\"`\n\n\t// The snapshot corresponding volume size.\n\tSize int64 `json:\"size,omitempty\"`\n\n\t// The incremental snapshot size.\n\tIncrementalSize int64 `json:\"incrementalSize,omitempty\"`\n\n\t// The type of the uploader that uploads the data. The valid values are `kopia` and `restic`.\n\tUploaderType string `json:\"uploaderType\"`\n\n\t// The PVC's corresponding volume name used by Pod\n\t// https://github.com/kubernetes/kubernetes/blob/e4b74dd12fa8cb63c174091d5536a10b8ec19d34/pkg/apis/core/types.go#L48\n\tVolumeName string `json:\"volumeName\"`\n\n\t// The Pod name mounting this PVC.\n\tPodName string `json:\"podName\"`\n\n\t// The Pod namespace\n\tPodNamespace string `json:\"podNamespace\"`\n\n\t// The PVB-taken k8s node's name.\n\t// This field will be empty when the struct is used to represent a podvolumerestore.\n\tNodeName string `json:\"nodeName,omitempty\"`\n\n\t// The PVB's Status.Phase value\n\tPhase velerov1api.PodVolumeBackupPhase\n}\n\nfunc newPodVolumeInfoFromPVB(pvb *velerov1api.PodVolumeBackup) *PodVolumeInfo {\n\treturn &PodVolumeInfo{\n\t\tSnapshotHandle: pvb.Status.SnapshotID,\n\t\tSize: pvb.Status.Progress.TotalBytes,\n\t\tIncrementalSize: pvb.Status.IncrementalBytes,\n\t\tUploaderType: pvb.Spec.UploaderType,\n\t\tVolumeName: pvb.Spec.Volume,\n\t\tPodName: pvb.Spec.Pod.Name,\n\t\tPodNamespace: pvb.Spec.Pod.Namespace,\n\t\tNodeName: pvb.Spec.Node,\n\t\tPhase: pvb.Status.Phase,\n\t}\n}\n\nfunc newPodVolumeInfoFromPVR(pvr *velerov1api.PodVolumeRestore) *PodVolumeInfo {\n\treturn &PodVolumeInfo{\n\t\tSnapshotHandle: pvr.Spec.SnapshotID,\n\t\tSize: pvr.Status.Progress.TotalBytes,\n\t\tUploaderType: pvr.Spec.UploaderType,\n\t\tVolumeName: pvr.Spec.Volume,\n\t\tPodName: pvr.Spec.Pod.Name,\n\t\tPodNamespace: pvr.Spec.Pod.Namespace,\n\t}\n}\n\n// PVInfo is used to store some PV information modified after creation.\n// Those information are lost after PV recreation.\ntype PVInfo struct {\n\t// ReclaimPolicy of PV. It could be different from the referenced StorageClass.\n\tReclaimPolicy string `json:\"reclaimPolicy\"`\n\n\t// The PV's labels should be kept after recreation.\n\tLabels map[string]string `json:\"labels\"`\n}\n\n// BackupVolumesInformation contains the information needs by generating\n// the backup BackupVolumeInfo array.\ntype BackupVolumesInformation struct {\n\t// A map contains the backup-included PV detail content. The key is PV name.\n\tpvMap *pvcPvMap\n\tvolumeInfos []*BackupVolumeInfo\n\n\tlogger logrus.FieldLogger\n\tcrClient kbclient.Client\n\tvolumeSnapshots []snapshotv1api.VolumeSnapshot\n\tvolumeSnapshotContents []snapshotv1api.VolumeSnapshotContent\n\tvolumeSnapshotClasses []snapshotv1api.VolumeSnapshotClass\n\tSkippedPVs map[string]string\n\tNativeSnapshots []*Snapshot\n\tPodVolumeBackups []*velerov1api.PodVolumeBackup\n\tBackupOperations []*itemoperation.BackupOperation\n\tBackupName string\n}\n\ntype pvcPvInfo struct {\n\tPVCName string\n\tPVCNamespace string\n\tPV corev1api.PersistentVolume\n}\n\nfunc (v *BackupVolumesInformation) Init() {\n\tv.pvMap = &pvcPvMap{\n\t\tdata: make(map[string]pvcPvInfo),\n\t}\n\tv.volumeInfos = make([]*BackupVolumeInfo, 0)\n}\n\nfunc (v *BackupVolumesInformation) InsertPVMap(pv corev1api.PersistentVolume, pvcName, pvcNamespace string) {\n\tif v.pvMap == nil {\n\t\tv.Init()\n\t}\n\tv.pvMap.insert(pv, pvcName, pvcNamespace)\n}\n\nfunc (v *BackupVolumesInformation) Result(\n\tcsiVolumeSnapshots []snapshotv1api.VolumeSnapshot,\n\tcsiVolumeSnapshotContents []snapshotv1api.VolumeSnapshotContent,\n\tcsiVolumesnapshotClasses []snapshotv1api.VolumeSnapshotClass,\n\tcrClient kbclient.Client,\n\tlogger logrus.FieldLogger,\n) []*BackupVolumeInfo {\n\tv.logger = logger\n\tv.crClient = crClient\n\tv.volumeSnapshots = csiVolumeSnapshots\n\tv.volumeSnapshotContents = csiVolumeSnapshotContents\n\tv.volumeSnapshotClasses = csiVolumesnapshotClasses\n\n\tv.generateVolumeInfoForSkippedPV()\n\tv.generateVolumeInfoForVeleroNativeSnapshot()\n\tv.generateVolumeInfoForCSIVolumeSnapshot()\n\tv.generateVolumeInfoFromPVB()\n\tv.generateVolumeInfoFromDataUpload()\n\n\treturn v.volumeInfos\n}\n\n// generateVolumeInfoForSkippedPV generate VolumeInfos for SkippedPV.\nfunc (v *BackupVolumesInformation) generateVolumeInfoForSkippedPV() {\n\ttmpVolumeInfos := make([]*BackupVolumeInfo, 0)\n\n\tfor pvName, skippedReason := range v.SkippedPVs {\n\t\tif pvcPVInfo := v.pvMap.retrieve(pvName, \"\", \"\"); pvcPVInfo != nil {\n\t\t\tvolumeInfo := &BackupVolumeInfo{\n\t\t\t\tPVCName: pvcPVInfo.PVCName,\n\t\t\t\tPVCNamespace: pvcPVInfo.PVCNamespace,\n\t\t\t\tPVName: pvName,\n\t\t\t\tSnapshotDataMoved: false,\n\t\t\t\tSkipped: true,\n\t\t\t\tSkippedReason: skippedReason,\n\t\t\t\tPVInfo: &PVInfo{\n\t\t\t\t\tReclaimPolicy: string(pvcPVInfo.PV.Spec.PersistentVolumeReclaimPolicy),\n\t\t\t\t\tLabels: pvcPVInfo.PV.Labels,\n\t\t\t\t},\n\t\t\t}\n\t\t\ttmpVolumeInfos = append(tmpVolumeInfos, volumeInfo)\n\t\t} else {\n\t\t\tv.logger.Warnf(\"Cannot find info for PV %s\", pvName)\n\t\t\tcontinue\n\t\t}\n\t}\n\n\tv.volumeInfos = append(v.volumeInfos, tmpVolumeInfos...)\n}\n\n// generateVolumeInfoForVeleroNativeSnapshot generate VolumeInfos for Velero native snapshot\nfunc (v *BackupVolumesInformation) generateVolumeInfoForVeleroNativeSnapshot() {\n\ttmpVolumeInfos := make([]*BackupVolumeInfo, 0)\n\n\tfor _, nativeSnapshot := range v.NativeSnapshots {\n\t\tif pvcPVInfo := v.pvMap.retrieve(nativeSnapshot.Spec.PersistentVolumeName, \"\", \"\"); pvcPVInfo != nil {\n\t\t\tvolumeResult := VolumeResultFailed\n\t\t\tif nativeSnapshot.Status.Phase == SnapshotPhaseCompleted {\n\t\t\t\tvolumeResult = VolumeResultSucceeded\n\t\t\t}\n\t\t\tvolumeInfo := &BackupVolumeInfo{\n\t\t\t\tBackupMethod: NativeSnapshot,\n\t\t\t\tPVCName: pvcPVInfo.PVCName,\n\t\t\t\tPVCNamespace: pvcPVInfo.PVCNamespace,\n\t\t\t\tPVName: pvcPVInfo.PV.Name,\n\t\t\t\tSnapshotDataMoved: false,\n\t\t\t\tSkipped: false,\n\t\t\t\t// Only set Succeeded to true when the NativeSnapshot's phase is Completed,\n\t\t\t\t// although NativeSnapshot doesn't check whether the snapshot creation result.\n\t\t\t\tResult: volumeResult,\n\t\t\t\tNativeSnapshotInfo: newNativeSnapshotInfo(nativeSnapshot),\n\t\t\t\tPVInfo: &PVInfo{\n\t\t\t\t\tReclaimPolicy: string(pvcPVInfo.PV.Spec.PersistentVolumeReclaimPolicy),\n\t\t\t\t\tLabels: pvcPVInfo.PV.Labels,\n\t\t\t\t},\n\t\t\t}\n\t\t\ttmpVolumeInfos = append(tmpVolumeInfos, volumeInfo)\n\t\t} else {\n\t\t\tv.logger.Warnf(\"cannot find info for PV %s\", nativeSnapshot.Spec.PersistentVolumeName)\n\t\t\tcontinue\n\t\t}\n\t}\n\n\tv.volumeInfos = append(v.volumeInfos, tmpVolumeInfos...)\n}\n\n// generateVolumeInfoForCSIVolumeSnapshot generate VolumeInfos for CSI VolumeSnapshot\nfunc (v *BackupVolumesInformation) generateVolumeInfoForCSIVolumeSnapshot() {\n\ttmpVolumeInfos := make([]*BackupVolumeInfo, 0)\n\n\tfor _, volumeSnapshot := range v.volumeSnapshots {\n\t\tvar volumeSnapshotContent *snapshotv1api.VolumeSnapshotContent\n\n\t\t// This is protective logic. The passed-in VS should be all related\n\t\t// to this backup.\n\t\tif volumeSnapshot.Labels[velerov1api.BackupNameLabel] != v.BackupName {\n\t\t\tcontinue\n\t\t}\n\n\t\tif volumeSnapshot.Status == nil || volumeSnapshot.Status.BoundVolumeSnapshotContentName == nil {\n\t\t\tv.logger.Warnf(\"Cannot fine VolumeSnapshotContent for VolumeSnapshot %s/%s\", volumeSnapshot.Namespace, volumeSnapshot.Name)\n\t\t\tcontinue\n\t\t}\n\n\t\tif volumeSnapshot.Spec.Source.PersistentVolumeClaimName == nil {\n\t\t\tv.logger.Warnf(\"VolumeSnapshot %s/%s doesn't have a source PVC\", volumeSnapshot.Namespace, volumeSnapshot.Name)\n\t\t\tcontinue\n\t\t}\n\n\t\tfor index := range v.volumeSnapshotContents {\n\t\t\tif *volumeSnapshot.Status.BoundVolumeSnapshotContentName == v.volumeSnapshotContents[index].Name {\n\t\t\t\tvolumeSnapshotContent = &v.volumeSnapshotContents[index]\n\t\t\t}\n\t\t}\n\n\t\tif volumeSnapshotContent == nil {\n\t\t\tv.logger.Warnf(\"fail to get VolumeSnapshotContent for VolumeSnapshot: %s/%s\",\n\t\t\t\tvolumeSnapshot.Namespace, volumeSnapshot.Name)\n\t\t\tcontinue\n\t\t}\n\n\t\tvar operation itemoperation.BackupOperation\n\t\tfor _, op := range v.BackupOperations {\n\t\t\tif op.Spec.ResourceIdentifier.GroupResource.String() == kuberesource.VolumeSnapshots.String() &&\n\t\t\t\top.Spec.ResourceIdentifier.Name == volumeSnapshot.Name &&\n\t\t\t\top.Spec.ResourceIdentifier.Namespace == volumeSnapshot.Namespace {\n\t\t\t\toperation = *op\n\t\t\t}\n\t\t}\n\n\t\tvar size int64\n\t\tif volumeSnapshot.Status.RestoreSize != nil {\n\t\t\tsize = volumeSnapshot.Status.RestoreSize.Value()\n\t\t}\n\t\tsnapshotHandle := \"\"\n\t\tif volumeSnapshotContent.Status.SnapshotHandle != nil {\n\t\t\tsnapshotHandle = *volumeSnapshotContent.Status.SnapshotHandle\n\t\t}\n\t\tif pvcPVInfo := v.pvMap.retrieve(\"\", *volumeSnapshot.Spec.Source.PersistentVolumeClaimName, volumeSnapshot.Namespace); pvcPVInfo != nil {\n\t\t\tvolumeInfo := &BackupVolumeInfo{\n\t\t\t\tBackupMethod: CSISnapshot,\n\t\t\t\tPVCName: pvcPVInfo.PVCName,\n\t\t\t\tPVCNamespace: pvcPVInfo.PVCNamespace,\n\t\t\t\tPVName: pvcPVInfo.PV.Name,\n\t\t\t\tSkipped: false,\n\t\t\t\tSnapshotDataMoved: false,\n\t\t\t\tPreserveLocalSnapshot: true,\n\t\t\t\tCSISnapshotInfo: &CSISnapshotInfo{\n\t\t\t\t\tVSCName: *volumeSnapshot.Status.BoundVolumeSnapshotContentName,\n\t\t\t\t\tSize: size,\n\t\t\t\t\tDriver: volumeSnapshotContent.Spec.Driver,\n\t\t\t\t\tSnapshotHandle: snapshotHandle,\n\t\t\t\t\tOperationID: operation.Spec.OperationID,\n\t\t\t\t\tReadyToUse: volumeSnapshot.Status.ReadyToUse,\n\t\t\t\t},\n\t\t\t\tPVInfo: &PVInfo{\n\t\t\t\t\tReclaimPolicy: string(pvcPVInfo.PV.Spec.PersistentVolumeReclaimPolicy),\n\t\t\t\t\tLabels: pvcPVInfo.PV.Labels,\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tif volumeSnapshot.Status.CreationTime != nil {\n\t\t\t\tvolumeInfo.StartTimestamp = volumeSnapshot.Status.CreationTime\n\t\t\t}\n\n\t\t\ttmpVolumeInfos = append(tmpVolumeInfos, volumeInfo)\n\t\t} else {\n\t\t\tv.logger.Warnf(\"cannot find info for PVC %s/%s\", volumeSnapshot.Namespace, volumeSnapshot.Spec.Source.PersistentVolumeClaimName)\n\t\t\tcontinue\n\t\t}\n\t}\n\n\tv.volumeInfos = append(v.volumeInfos, tmpVolumeInfos...)\n}\n\n// generateVolumeInfoFromPVB generate BackupVolumeInfo for PVB.\nfunc (v *BackupVolumesInformation) generateVolumeInfoFromPVB() {\n\ttmpVolumeInfos := make([]*BackupVolumeInfo, 0)\n\tfor _, pvb := range v.PodVolumeBackups {\n\t\tvolumeInfo := &BackupVolumeInfo{\n\t\t\tBackupMethod: PodVolumeBackup,\n\t\t\tSnapshotDataMoved: false,\n\t\t\tSkipped: false,\n\t\t\tStartTimestamp: pvb.Status.StartTimestamp,\n\t\t\tCompletionTimestamp: pvb.Status.CompletionTimestamp,\n\t\t\tPVBInfo: newPodVolumeInfoFromPVB(pvb),\n\t\t}\n\n\t\t// Only set Succeeded to true when the PVB's phase is Completed.\n\t\tif pvb.Status.Phase == velerov1api.PodVolumeBackupPhaseCompleted {\n\t\t\tvolumeInfo.Result = VolumeResultSucceeded\n\t\t} else {\n\t\t\tvolumeInfo.Result = VolumeResultFailed\n\t\t}\n\n\t\tpvcName, err := pvcByPodvolume(context.TODO(), v.crClient, pvb.Spec.Pod.Name, pvb.Spec.Pod.Namespace, pvb.Spec.Volume)\n\t\tif err != nil {\n\t\t\tv.logger.WithError(err).Warn(\"Fail to get PVC from PodVolumeBackup: \", pvb.Name)\n\t\t\tcontinue\n\t\t}\n\t\tif pvcName != \"\" {\n\t\t\tif pvcPVInfo := v.pvMap.retrieve(\"\", pvcName, pvb.Spec.Pod.Namespace); pvcPVInfo != nil {\n\t\t\t\tvolumeInfo.PVCName = pvcPVInfo.PVCName\n\t\t\t\tvolumeInfo.PVCNamespace = pvcPVInfo.PVCNamespace\n\t\t\t\tvolumeInfo.PVName = pvcPVInfo.PV.Name\n\t\t\t\tvolumeInfo.PVInfo = &PVInfo{\n\t\t\t\t\tReclaimPolicy: string(pvcPVInfo.PV.Spec.PersistentVolumeReclaimPolicy),\n\t\t\t\t\tLabels: pvcPVInfo.PV.Labels,\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tv.logger.Warnf(\"Cannot find info for PVC %s/%s\", pvb.Spec.Pod.Namespace, pvcName)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t} else {\n\t\t\tv.logger.Debug(\"The PVB %s doesn't have a corresponding PVC\", pvb.Name)\n\t\t}\n\t\ttmpVolumeInfos = append(tmpVolumeInfos, volumeInfo)\n\t}\n\tv.volumeInfos = append(v.volumeInfos, tmpVolumeInfos...)\n}\n\nfunc (v *BackupVolumesInformation) getVolumeSnapshotClasses() (\n\t[]snapshotv1api.VolumeSnapshotClass,\n\terror,\n) {\n\tvsClassList := new(snapshotv1api.VolumeSnapshotClassList)\n\tif err := v.crClient.List(context.TODO(), vsClassList); err != nil {\n\t\tv.logger.Warnf(\"Cannot list VolumeSnapshotClass with error %s.\", err.Error())\n\t\treturn nil, err\n\t}\n\n\treturn vsClassList.Items, nil\n}\n\n// generateVolumeInfoFromDataUpload generate BackupVolumeInfo for DataUpload.\nfunc (v *BackupVolumesInformation) generateVolumeInfoFromDataUpload() {\n\tif !features.IsEnabled(velerov1api.CSIFeatureFlag) {\n\t\tv.logger.Debug(\"Skip generating BackupVolumeInfo when the CSI feature is disabled.\")\n\t\treturn\n\t}\n\n\t// Retrieve the operations containing DataUpload.\n\tduOperationMap := make(map[kbclient.ObjectKey]*itemoperation.BackupOperation)\n\tfor _, operation := range v.BackupOperations {\n\t\tif operation.Spec.ResourceIdentifier.GroupResource.String() == kuberesource.PersistentVolumeClaims.String() {\n\t\t\tfor _, identifier := range operation.Spec.PostOperationItems {\n\t\t\t\tif identifier.GroupResource.String() == \"datauploads.velero.io\" {\n\t\t\t\t\tduOperationMap[kbclient.ObjectKey{\n\t\t\t\t\t\tNamespace: identifier.Namespace,\n\t\t\t\t\t\tName: identifier.Name,\n\t\t\t\t\t}] = operation\n\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\tif len(duOperationMap) <= 0 {\n\t\t// No DataUpload is found. Return early.\n\t\treturn\n\t}\n\n\ttmpVolumeInfos := make([]*BackupVolumeInfo, 0)\n\tfor duObjectKey, operation := range duOperationMap {\n\t\tdataUpload := new(velerov2alpha1.DataUpload)\n\t\terr := v.crClient.Get(\n\t\t\tcontext.TODO(),\n\t\t\tduObjectKey,\n\t\t\tdataUpload,\n\t\t)\n\t\tif err != nil {\n\t\t\tv.logger.Warnf(\"Fail to get DataUpload %s: %s\",\n\t\t\t\tduObjectKey.Namespace+\"/\"+duObjectKey.Name,\n\t\t\t\terr.Error(),\n\t\t\t)\n\t\t\tcontinue\n\t\t}\n\n\t\tif pvcPVInfo := v.pvMap.retrieve(\n\t\t\t\"\",\n\t\t\toperation.Spec.ResourceIdentifier.Name,\n\t\t\toperation.Spec.ResourceIdentifier.Namespace,\n\t\t); pvcPVInfo != nil {\n\t\t\tdataMover := veleroDatamover\n\t\t\tif dataUpload.Spec.DataMover != \"\" {\n\t\t\t\tdataMover = dataUpload.Spec.DataMover\n\t\t\t}\n\n\t\t\tvolumeInfo := &BackupVolumeInfo{\n\t\t\t\tBackupMethod: CSISnapshot,\n\t\t\t\tPVCName: pvcPVInfo.PVCName,\n\t\t\t\tPVCNamespace: pvcPVInfo.PVCNamespace,\n\t\t\t\tPVName: pvcPVInfo.PV.Name,\n\t\t\t\tSnapshotDataMoved: true,\n\t\t\t\tSkipped: false,\n\t\t\t\tCSISnapshotInfo: &CSISnapshotInfo{\n\t\t\t\t\tSnapshotHandle: FieldValueIsUnknown,\n\t\t\t\t\tVSCName: FieldValueIsUnknown,\n\t\t\t\t\tOperationID: FieldValueIsUnknown,\n\t\t\t\t\tDriver: dataUpload.Spec.CSISnapshot.Driver,\n\t\t\t\t},\n\t\t\t\tSnapshotDataMovementInfo: &SnapshotDataMovementInfo{\n\t\t\t\t\tDataMover: dataMover,\n\t\t\t\t\tUploaderType: velerov1api.BackupRepositoryTypeKopia,\n\t\t\t\t\tOperationID: operation.Spec.OperationID,\n\t\t\t\t\tPhase: dataUpload.Status.Phase,\n\t\t\t\t},\n\t\t\t\tPVInfo: &PVInfo{\n\t\t\t\t\tReclaimPolicy: string(pvcPVInfo.PV.Spec.PersistentVolumeReclaimPolicy),\n\t\t\t\t\tLabels: pvcPVInfo.PV.Labels,\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tif dataUpload.Status.StartTimestamp != nil {\n\t\t\t\tvolumeInfo.StartTimestamp = dataUpload.Status.StartTimestamp\n\t\t\t}\n\n\t\t\ttmpVolumeInfos = append(tmpVolumeInfos, volumeInfo)\n\t\t} else {\n\t\t\tv.logger.Warnf(\"Cannot find info for PVC %s/%s\", operation.Spec.ResourceIdentifier.Namespace, operation.Spec.ResourceIdentifier.Name)\n\t\t\tcontinue\n\t\t}\n\t}\n\n\tv.volumeInfos = append(v.volumeInfos, tmpVolumeInfos...)\n}\n\ntype pvcPvMap struct {\n\tdata map[string]pvcPvInfo\n}\n\nfunc (m *pvcPvMap) insert(pv corev1api.PersistentVolume, pvcName, pvcNamespace string) {\n\tm.data[pv.Name] = pvcPvInfo{\n\t\tPVCName: pvcName,\n\t\tPVCNamespace: pvcNamespace,\n\t\tPV: pv,\n\t}\n}\n\nfunc (m *pvcPvMap) retrieve(pvName, pvcName, pvcNS string) *pvcPvInfo {\n\tif pvName != \"\" {\n\t\tif info, ok := m.data[pvName]; ok {\n\t\t\treturn &info\n\t\t}\n\t\treturn nil\n\t}\n\n\tif pvcNS == \"\" || pvcName == \"\" {\n\t\treturn nil\n\t}\n\n\tfor _, info := range m.data {\n\t\tif pvcNS == info.PVCNamespace && pvcName == info.PVCName {\n\t\t\treturn &info\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc pvcByPodvolume(ctx context.Context, crClient kbclient.Client, podName, podNamespace, volumeName string) (string, error) {\n\tpod := new(corev1api.Pod)\n\terr := crClient.Get(ctx, kbclient.ObjectKey{Namespace: podNamespace, Name: podName}, pod)\n\tif err != nil {\n\t\treturn \"\", errors.Wrap(err, \"failed to get pod\")\n\t}\n\tfor _, volume := range pod.Spec.Volumes {\n\t\tif volume.Name == volumeName && volume.PersistentVolumeClaim != nil {\n\t\t\treturn volume.PersistentVolumeClaim.ClaimName, nil\n\t\t}\n\t}\n\treturn \"\", nil\n}\n\n// RestoreVolumeInfoTracker is used to track the volume information during restore.\n// It is used to generate the RestoreVolumeInfo array.\ntype RestoreVolumeInfoTracker struct {\n\t*sync.Mutex\n\trestore *velerov1api.Restore\n\tlog logrus.FieldLogger\n\tclient kbclient.Client\n\tpvPvc *pvcPvMap\n\n\t// map of PV name to the NativeSnapshotInfo from which the PV is restored\n\tpvNativeSnapshotMap map[string]*NativeSnapshotInfo\n\t// map of PVC object to the CSISnapshot object from which the PV is restored\n\t// the key is in the form of $pvc-ns/$pvc-name\n\tpvcCSISnapshotMap map[string]snapshotv1api.VolumeSnapshot\n\tdatadownloadList *velerov2alpha1.DataDownloadList\n\tpvrs []*velerov1api.PodVolumeRestore\n}\n\n// Populate data objects in the tracker, which will be used to generate the RestoreVolumeInfo array in Result()\n// The input param resourceList should be the final result of the restore.\nfunc (t *RestoreVolumeInfoTracker) Populate(ctx context.Context, restoredResourceList map[string][]string) {\n\tpvcs := RestoredPVCFromRestoredResourceList(restoredResourceList)\n\tt.Lock()\n\tdefer t.Unlock()\n\tfor item := range pvcs {\n\t\tn := strings.Split(item, \"/\")\n\t\tpvcNS, pvcName := n[0], n[1]\n\t\tlog := t.log.WithField(\"namespace\", pvcNS).WithField(\"name\", pvcName)\n\t\tpvc := &corev1api.PersistentVolumeClaim{}\n\t\tif err := t.client.Get(ctx, kbclient.ObjectKey{Namespace: pvcNS, Name: pvcName}, pvc); err != nil {\n\t\t\tlog.WithError(err).Error(\"Failed to get PVC\")\n\t\t\tcontinue\n\t\t}\n\t\t// Collect the CSI VolumeSnapshot objects referenced by the restored PVCs,\n\t\tif pvc.Spec.DataSource != nil && pvc.Spec.DataSource.Kind == \"VolumeSnapshot\" {\n\t\t\tvs := &snapshotv1api.VolumeSnapshot{}\n\t\t\tif err := t.client.Get(ctx, kbclient.ObjectKey{Namespace: pvcNS, Name: pvc.Spec.DataSource.Name}, vs); err != nil {\n\t\t\t\tlog.WithError(err).Error(\"Failed to get VolumeSnapshot\")\n\t\t\t} else {\n\t\t\t\tt.pvcCSISnapshotMap[pvc.Namespace+\"/\"+pvcName] = *vs\n\t\t\t}\n\t\t}\n\t\tif pvc.Status.Phase == corev1api.ClaimBound && pvc.Spec.VolumeName != \"\" {\n\t\t\tpv := &corev1api.PersistentVolume{}\n\t\t\tif err := t.client.Get(ctx, kbclient.ObjectKey{Name: pvc.Spec.VolumeName}, pv); err != nil {\n\t\t\t\tlog.WithError(err).Error(\"Failed to get PV\")\n\t\t\t} else {\n\t\t\t\tt.pvPvc.insert(*pv, pvcName, pvcNS)\n\t\t\t}\n\t\t} else {\n\t\t\tlog.Warn(\"PVC is not bound or has no volume name\")\n\t\t\tcontinue\n\t\t}\n\t}\n\tif err := t.client.List(ctx, t.datadownloadList, &kbclient.ListOptions{\n\t\tNamespace: t.restore.Namespace,\n\t\tLabelSelector: label.NewSelectorForRestore(t.restore.Name),\n\t}); err != nil {\n\t\tt.log.WithError(err).Error(\"Failed to List DataDownloads\")\n\t}\n}\n\n// Result generates the RestoreVolumeInfo array, the data should come from the Tracker itself and it should not connect tokkkk API\n// server again.\nfunc (t *RestoreVolumeInfoTracker) Result() []*RestoreVolumeInfo {\n\tvolumeInfos := make([]*RestoreVolumeInfo, 0)\n\n\t// Generate RestoreVolumeInfo for PVRs\n\tfor _, pvr := range t.pvrs {\n\t\tvolumeInfo := &RestoreVolumeInfo{\n\t\t\tSnapshotDataMoved: false,\n\t\t\tPVRInfo: newPodVolumeInfoFromPVR(pvr),\n\t\t\tRestoreMethod: PodVolumeRestore,\n\t\t}\n\t\tpvcName, err := pvcByPodvolume(context.TODO(), t.client, pvr.Spec.Pod.Name, pvr.Spec.Pod.Namespace, pvr.Spec.Volume)\n\t\tif err != nil {\n\t\t\tt.log.WithError(err).Warn(\"Fail to get PVC from PodVolumeRestore: \", pvr.Name)\n\t\t\tcontinue\n\t\t}\n\t\tif pvcName != \"\" {\n\t\t\tvolumeInfo.PVCName = pvcName\n\t\t\tvolumeInfo.PVCNamespace = pvr.Spec.Pod.Namespace\n\t\t\tif pvcPVInfo := t.pvPvc.retrieve(\"\", pvcName, pvr.Spec.Pod.Namespace); pvcPVInfo != nil {\n\t\t\t\tvolumeInfo.PVName = pvcPVInfo.PV.Name\n\t\t\t}\n\t\t} else {\n\t\t\t// In this case, the volume is not bound to a PVC and\n\t\t\t// the PVR will not be able to populate into the volume, so we'll skip it\n\t\t\tt.log.Warnf(\"unable to get PVC for PodVolumeRestore %s/%s, pod: %s/%s, volume: %s\",\n\t\t\t\tpvr.Namespace, pvr.Name, pvr.Spec.Pod.Namespace, pvr.Spec.Pod.Name, pvr.Spec.Volume)\n\t\t\tcontinue\n\t\t}\n\t\tvolumeInfos = append(volumeInfos, volumeInfo)\n\t}\n\n\t// Generate RestoreVolumeInfo for PVs restored from NativeSnapshots\n\tfor pvName, snapshotInfo := range t.pvNativeSnapshotMap {\n\t\tvolumeInfo := &RestoreVolumeInfo{\n\t\t\tPVName: pvName,\n\t\t\tSnapshotDataMoved: false,\n\t\t\tNativeSnapshotInfo: snapshotInfo,\n\t\t\tRestoreMethod: NativeSnapshot,\n\t\t}\n\t\tif pvcPVInfo := t.pvPvc.retrieve(pvName, \"\", \"\"); pvcPVInfo != nil {\n\t\t\tvolumeInfo.PVCName = pvcPVInfo.PVCName\n\t\t\tvolumeInfo.PVCNamespace = pvcPVInfo.PVCNamespace\n\t\t}\n\t\tvolumeInfos = append(volumeInfos, volumeInfo)\n\t}\n\n\t// Generate RestoreVolumeInfo for PVs restored from CSISnapshots\n\tfor pvc, csiSnapshot := range t.pvcCSISnapshotMap {\n\t\tn := strings.Split(pvc, \"/\")\n\t\tif len(n) != 2 {\n\t\t\tt.log.Warnf(\"Invalid PVC key '%s' in the pvc-CSISnapshot map, skip populating it to volume info\", pvc)\n\t\t\tcontinue\n\t\t}\n\t\tpvcNS, pvcName := n[0], n[1]\n\t\tvar restoreSize int64\n\t\tif csiSnapshot.Status != nil && csiSnapshot.Status.RestoreSize != nil {\n\t\t\trestoreSize = csiSnapshot.Status.RestoreSize.Value()\n\t\t}\n\t\tvscName := \"\"\n\t\tif csiSnapshot.Spec.Source.VolumeSnapshotContentName != nil {\n\t\t\tvscName = *csiSnapshot.Spec.Source.VolumeSnapshotContentName\n\t\t}\n\n\t\tvolumeInfo := &RestoreVolumeInfo{\n\t\t\tPVCNamespace: pvcNS,\n\t\t\tPVCName: pvcName,\n\t\t\tSnapshotDataMoved: false,\n\t\t\tRestoreMethod: CSISnapshot,\n\t\t\tCSISnapshotInfo: &CSISnapshotInfo{\n\t\t\t\tSnapshotHandle: csiSnapshot.Annotations[velerov1api.VolumeSnapshotHandleAnnotation],\n\t\t\t\tSize: restoreSize,\n\t\t\t\tDriver: csiSnapshot.Annotations[velerov1api.DriverNameAnnotation],\n\t\t\t\tVSCName: vscName,\n\t\t\t},\n\t\t}\n\t\tif pvcPVInfo := t.pvPvc.retrieve(\"\", pvcName, pvcNS); pvcPVInfo != nil {\n\t\t\tvolumeInfo.PVName = pvcPVInfo.PV.Name\n\t\t}\n\t\tvolumeInfos = append(volumeInfos, volumeInfo)\n\t}\n\n\tfor _, dd := range t.datadownloadList.Items {\n\t\tvar pvcName, pvcNS, pvName string\n\t\tif pvcPVInfo := t.pvPvc.retrieve(dd.Spec.TargetVolume.PV, dd.Spec.TargetVolume.PVC, dd.Spec.TargetVolume.Namespace); pvcPVInfo != nil {\n\t\t\tpvcName = pvcPVInfo.PVCName\n\t\t\tpvcNS = pvcPVInfo.PVCNamespace\n\t\t\tpvName = pvcPVInfo.PV.Name\n\t\t} else {\n\t\t\tpvcName = dd.Spec.TargetVolume.PVC\n\t\t\tpvName = dd.Spec.TargetVolume.PV\n\t\t\tpvcNS = dd.Spec.TargetVolume.Namespace\n\t\t}\n\t\toperationID := dd.Labels[velerov1api.AsyncOperationIDLabel]\n\t\tdataMover := veleroDatamover\n\t\tif dd.Spec.DataMover != \"\" {\n\t\t\tdataMover = dd.Spec.DataMover\n\t\t}\n\t\tvolumeInfo := &RestoreVolumeInfo{\n\t\t\tPVName: pvName,\n\t\t\tPVCNamespace: pvcNS,\n\t\t\tPVCName: pvcName,\n\t\t\tSnapshotDataMoved: true,\n\t\t\t// The method will be CSI always no CSI related CRs are created during restore, because\n\t\t\t// the datadownload was initiated in CSI plugin\n\t\t\t// For the same reason, no CSI snapshot info will be populated into volumeInfo\n\t\t\tRestoreMethod: CSISnapshot,\n\t\t\tSnapshotDataMovementInfo: &SnapshotDataMovementInfo{\n\t\t\t\tDataMover: dataMover,\n\t\t\t\tUploaderType: velerov1api.BackupRepositoryTypeKopia,\n\t\t\t\tSnapshotHandle: dd.Spec.SnapshotID,\n\t\t\t\tOperationID: operationID,\n\t\t\t},\n\t\t}\n\n\t\tvolumeInfos = append(volumeInfos, volumeInfo)\n\t}\n\n\treturn volumeInfos\n}\n\nfunc NewRestoreVolInfoTracker(restore *velerov1api.Restore, logger logrus.FieldLogger, client kbclient.Client) *RestoreVolumeInfoTracker {\n\treturn &RestoreVolumeInfoTracker{\n\t\tMutex: &sync.Mutex{},\n\t\tclient: client,\n\t\tlog: logger,\n\t\trestore: restore,\n\t\tpvPvc: &pvcPvMap{\n\t\t\tdata: make(map[string]pvcPvInfo),\n\t\t},\n\t\tpvNativeSnapshotMap: make(map[string]*NativeSnapshotInfo),\n\t\tpvcCSISnapshotMap: make(map[string]snapshotv1api.VolumeSnapshot),\n\t\tdatadownloadList: &velerov2alpha1.DataDownloadList{},\n\t}\n}\n\nfunc (t *RestoreVolumeInfoTracker) TrackNativeSnapshot(pvName string, snapshotHandle, volumeType, volumeAZ string, iops int64) {\n\tt.Lock()\n\tdefer t.Unlock()\n\tt.pvNativeSnapshotMap[pvName] = &NativeSnapshotInfo{\n\t\tSnapshotHandle: snapshotHandle,\n\t\tVolumeType: volumeType,\n\t\tVolumeAZ: volumeAZ,\n\t\tIOPS: strconv.FormatInt(iops, 10),\n\t}\n}\n\nfunc (t *RestoreVolumeInfoTracker) RenamePVForNativeSnapshot(oldName, newName string) {\n\tt.Lock()\n\tdefer t.Unlock()\n\tif snapshotInfo, ok := t.pvNativeSnapshotMap[oldName]; ok {\n\t\tt.pvNativeSnapshotMap[newName] = snapshotInfo\n\t\tdelete(t.pvNativeSnapshotMap, oldName)\n\t}\n}\n\nfunc (t *RestoreVolumeInfoTracker) TrackPodVolume(pvr *velerov1api.PodVolumeRestore) {\n\tt.Lock()\n\tdefer t.Unlock()\n\tt.pvrs = append(t.pvrs, pvr)\n}\n" }, { "path": "internal/volume/volumes_information_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage volume\n\nimport (\n\t\"sync\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/api/resource\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/features\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n)\n\nfunc TestGenerateVolumeInfoForSkippedPV(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tskippedPVName string\n\t\tpvMap map[string]pvcPvInfo\n\t\texpectedVolumeInfos []*BackupVolumeInfo\n\t}{\n\t\t{\n\t\t\tname: \"Cannot find info for PV\",\n\t\t\tskippedPVName: \"testPV\",\n\t\t\tpvMap: map[string]pvcPvInfo{\n\t\t\t\t\"velero/testPVC\": {\n\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\tPVCNamespace: \"velero\",\n\t\t\t\t\tPV: corev1api.PersistentVolume{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"testPV\",\n\t\t\t\t\t\t\tLabels: map[string]string{\"a\": \"b\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\t\tPersistentVolumeReclaimPolicy: corev1api.PersistentVolumeReclaimDelete,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{},\n\t\t},\n\t\t{\n\t\t\tname: \"Normal Skipped PV info\",\n\t\t\tskippedPVName: \"testPV\",\n\t\t\tpvMap: map[string]pvcPvInfo{\n\t\t\t\t\"velero/testPVC\": {\n\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\tPVCNamespace: \"velero\",\n\t\t\t\t\tPV: corev1api.PersistentVolume{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"testPV\",\n\t\t\t\t\t\t\tLabels: map[string]string{\"a\": \"b\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\t\tPersistentVolumeReclaimPolicy: corev1api.PersistentVolumeReclaimDelete,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"testPV\": {\n\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\tPVCNamespace: \"velero\",\n\t\t\t\t\tPV: corev1api.PersistentVolume{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"testPV\",\n\t\t\t\t\t\t\tLabels: map[string]string{\"a\": \"b\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\t\tPersistentVolumeReclaimPolicy: corev1api.PersistentVolumeReclaimDelete,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\tPVCNamespace: \"velero\",\n\t\t\t\t\tPVName: \"testPV\",\n\t\t\t\t\tSkipped: true,\n\t\t\t\t\tSkippedReason: \"CSI: skipped for PodVolumeBackup\",\n\t\t\t\t\tPVInfo: &PVInfo{\n\t\t\t\t\t\tReclaimPolicy: \"Delete\",\n\t\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\t\"a\": \"b\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvolumesInfo := BackupVolumesInformation{}\n\t\t\tvolumesInfo.Init()\n\n\t\t\tif tc.skippedPVName != \"\" {\n\t\t\t\tvolumesInfo.SkippedPVs = map[string]string{\n\t\t\t\t\ttc.skippedPVName: \"CSI: skipped for PodVolumeBackup\",\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif tc.pvMap != nil {\n\t\t\t\tfor k, v := range tc.pvMap {\n\t\t\t\t\tif k == v.PV.Name {\n\t\t\t\t\t\tvolumesInfo.pvMap.insert(v.PV, v.PVCName, v.PVCNamespace)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\tvolumesInfo.logger = logging.DefaultLogger(logrus.DebugLevel, logging.FormatJSON)\n\n\t\t\tvolumesInfo.generateVolumeInfoForSkippedPV()\n\t\t\trequire.Equal(t, tc.expectedVolumeInfos, volumesInfo.volumeInfos)\n\t\t})\n\t}\n}\n\nfunc TestGenerateVolumeInfoForVeleroNativeSnapshot(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tnativeSnapshot Snapshot\n\t\tpvMap map[string]pvcPvInfo\n\t\texpectedVolumeInfos []*BackupVolumeInfo\n\t}{\n\t\t{\n\t\t\tname: \"Native snapshot's IPOS pointer is nil\",\n\t\t\tnativeSnapshot: Snapshot{\n\t\t\t\tSpec: SnapshotSpec{\n\t\t\t\t\tPersistentVolumeName: \"testPV\",\n\t\t\t\t\tVolumeIOPS: nil,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{},\n\t\t},\n\t\t{\n\t\t\tname: \"Cannot find info for the PV\",\n\t\t\tnativeSnapshot: Snapshot{\n\t\t\t\tSpec: SnapshotSpec{\n\t\t\t\t\tPersistentVolumeName: \"testPV\",\n\t\t\t\t\tVolumeIOPS: int64Ptr(100),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{},\n\t\t},\n\t\t{\n\t\t\tname: \"Cannot find PV info in pvMap\",\n\t\t\tpvMap: map[string]pvcPvInfo{\n\t\t\t\t\"velero/testPVC\": {\n\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\tPVCNamespace: \"velero\",\n\t\t\t\t\tPV: corev1api.PersistentVolume{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"testPV\",\n\t\t\t\t\t\t\tLabels: map[string]string{\"a\": \"b\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\t\tPersistentVolumeReclaimPolicy: corev1api.PersistentVolumeReclaimDelete,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tnativeSnapshot: Snapshot{\n\t\t\t\tSpec: SnapshotSpec{\n\t\t\t\t\tPersistentVolumeName: \"testPV\",\n\t\t\t\t\tVolumeIOPS: int64Ptr(100),\n\t\t\t\t\tVolumeType: \"ssd\",\n\t\t\t\t\tVolumeAZ: \"us-central1-a\",\n\t\t\t\t},\n\t\t\t\tStatus: SnapshotStatus{\n\t\t\t\t\tProviderSnapshotID: \"pvc-b31e3386-4bbb-4937-95d-7934cd62-b0a1-494b-95d7-0687440e8d0c\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{},\n\t\t},\n\t\t{\n\t\t\tname: \"Normal native snapshot with failed phase\",\n\t\t\tpvMap: map[string]pvcPvInfo{\n\t\t\t\t\"testPV\": {\n\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\tPVCNamespace: \"velero\",\n\t\t\t\t\tPV: corev1api.PersistentVolume{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"testPV\",\n\t\t\t\t\t\t\tLabels: map[string]string{\"a\": \"b\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\t\tPersistentVolumeReclaimPolicy: corev1api.PersistentVolumeReclaimDelete,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tnativeSnapshot: Snapshot{\n\t\t\t\tSpec: SnapshotSpec{\n\t\t\t\t\tPersistentVolumeName: \"testPV\",\n\t\t\t\t\tVolumeIOPS: int64Ptr(100),\n\t\t\t\t\tVolumeType: \"ssd\",\n\t\t\t\t\tVolumeAZ: \"us-central1-a\",\n\t\t\t\t},\n\t\t\t\tStatus: SnapshotStatus{\n\t\t\t\t\tProviderSnapshotID: \"pvc-b31e3386-4bbb-4937-95d-7934cd62-b0a1-494b-95d7-0687440e8d0c\",\n\t\t\t\t\tPhase: SnapshotPhaseFailed,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\tPVCNamespace: \"velero\",\n\t\t\t\t\tPVName: \"testPV\",\n\t\t\t\t\tBackupMethod: NativeSnapshot,\n\t\t\t\t\tResult: VolumeResultFailed,\n\t\t\t\t\tPVInfo: &PVInfo{\n\t\t\t\t\t\tReclaimPolicy: \"Delete\",\n\t\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\t\"a\": \"b\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tNativeSnapshotInfo: &NativeSnapshotInfo{\n\t\t\t\t\t\tSnapshotHandle: \"pvc-b31e3386-4bbb-4937-95d-7934cd62-b0a1-494b-95d7-0687440e8d0c\",\n\t\t\t\t\t\tVolumeType: \"ssd\",\n\t\t\t\t\t\tVolumeAZ: \"us-central1-a\",\n\t\t\t\t\t\tIOPS: \"100\",\n\t\t\t\t\t\tPhase: SnapshotPhaseFailed,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Normal native snapshot\",\n\t\t\tpvMap: map[string]pvcPvInfo{\n\t\t\t\t\"testPV\": {\n\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\tPVCNamespace: \"velero\",\n\t\t\t\t\tPV: corev1api.PersistentVolume{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"testPV\",\n\t\t\t\t\t\t\tLabels: map[string]string{\"a\": \"b\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\t\tPersistentVolumeReclaimPolicy: corev1api.PersistentVolumeReclaimDelete,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tnativeSnapshot: Snapshot{\n\t\t\t\tSpec: SnapshotSpec{\n\t\t\t\t\tPersistentVolumeName: \"testPV\",\n\t\t\t\t\tVolumeIOPS: int64Ptr(100),\n\t\t\t\t\tVolumeType: \"ssd\",\n\t\t\t\t\tVolumeAZ: \"us-central1-a\",\n\t\t\t\t},\n\t\t\t\tStatus: SnapshotStatus{\n\t\t\t\t\tProviderSnapshotID: \"pvc-b31e3386-4bbb-4937-95d-7934cd62-b0a1-494b-95d7-0687440e8d0c\",\n\t\t\t\t\tPhase: SnapshotPhaseCompleted,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\tPVCNamespace: \"velero\",\n\t\t\t\t\tPVName: \"testPV\",\n\t\t\t\t\tBackupMethod: NativeSnapshot,\n\t\t\t\t\tResult: VolumeResultSucceeded,\n\t\t\t\t\tPVInfo: &PVInfo{\n\t\t\t\t\t\tReclaimPolicy: \"Delete\",\n\t\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\t\"a\": \"b\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tNativeSnapshotInfo: &NativeSnapshotInfo{\n\t\t\t\t\t\tSnapshotHandle: \"pvc-b31e3386-4bbb-4937-95d-7934cd62-b0a1-494b-95d7-0687440e8d0c\",\n\t\t\t\t\t\tVolumeType: \"ssd\",\n\t\t\t\t\t\tVolumeAZ: \"us-central1-a\",\n\t\t\t\t\t\tIOPS: \"100\",\n\t\t\t\t\t\tPhase: SnapshotPhaseCompleted,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvolumesInfo := BackupVolumesInformation{}\n\t\t\tvolumesInfo.Init()\n\t\t\tvolumesInfo.NativeSnapshots = append(volumesInfo.NativeSnapshots, &tc.nativeSnapshot)\n\t\t\tif tc.pvMap != nil {\n\t\t\t\tfor k, v := range tc.pvMap {\n\t\t\t\t\tif k == v.PV.Name {\n\t\t\t\t\t\tvolumesInfo.pvMap.insert(v.PV, v.PVCName, v.PVCNamespace)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\tvolumesInfo.logger = logging.DefaultLogger(logrus.DebugLevel, logging.FormatJSON)\n\n\t\t\tvolumesInfo.generateVolumeInfoForVeleroNativeSnapshot()\n\t\t\trequire.Equal(t, tc.expectedVolumeInfos, volumesInfo.volumeInfos)\n\t\t})\n\t}\n}\n\nfunc TestGenerateVolumeInfoForCSIVolumeSnapshot(t *testing.T) {\n\tresourceQuantity := resource.MustParse(\"100Gi\")\n\tnow := metav1.Now()\n\treadyToUse := true\n\ttests := []struct {\n\t\tname string\n\t\tvolumeSnapshot snapshotv1api.VolumeSnapshot\n\t\tvolumeSnapshotContent snapshotv1api.VolumeSnapshotContent\n\t\tvolumeSnapshotClass snapshotv1api.VolumeSnapshotClass\n\t\tpvMap map[string]pvcPvInfo\n\t\toperation *itemoperation.BackupOperation\n\t\texpectedVolumeInfos []*BackupVolumeInfo\n\t}{\n\t\t{\n\t\t\tname: \"VS doesn't have VolumeSnapshotClass name\",\n\t\t\tvolumeSnapshot: snapshotv1api.VolumeSnapshot{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"testVS\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t\tSpec: snapshotv1api.VolumeSnapshotSpec{},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{},\n\t\t},\n\t\t{\n\t\t\tname: \"VS doesn't have status\",\n\t\t\tvolumeSnapshot: snapshotv1api.VolumeSnapshot{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"testVS\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t\tSpec: snapshotv1api.VolumeSnapshotSpec{\n\t\t\t\t\tVolumeSnapshotClassName: stringPtr(\"testClass\"),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{},\n\t\t},\n\t\t{\n\t\t\tname: \"VS doesn't have PVC\",\n\t\t\tvolumeSnapshot: snapshotv1api.VolumeSnapshot{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"testVS\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t\tSpec: snapshotv1api.VolumeSnapshotSpec{\n\t\t\t\t\tVolumeSnapshotClassName: stringPtr(\"testClass\"),\n\t\t\t\t},\n\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\t\t\tBoundVolumeSnapshotContentName: stringPtr(\"testContent\"),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{},\n\t\t},\n\t\t{\n\t\t\tname: \"Cannot find VSC for VS\",\n\t\t\tvolumeSnapshot: snapshotv1api.VolumeSnapshot{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"testVS\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t\tSpec: snapshotv1api.VolumeSnapshotSpec{\n\t\t\t\t\tVolumeSnapshotClassName: stringPtr(\"testClass\"),\n\t\t\t\t\tSource: snapshotv1api.VolumeSnapshotSource{\n\t\t\t\t\t\tPersistentVolumeClaimName: stringPtr(\"testPVC\"),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\t\t\tBoundVolumeSnapshotContentName: stringPtr(\"testContent\"),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{},\n\t\t},\n\t\t{\n\t\t\tname: \"Cannot find BackupVolumeInfo for PVC\",\n\t\t\tvolumeSnapshot: snapshotv1api.VolumeSnapshot{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"testVS\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t\tSpec: snapshotv1api.VolumeSnapshotSpec{\n\t\t\t\t\tVolumeSnapshotClassName: stringPtr(\"testClass\"),\n\t\t\t\t\tSource: snapshotv1api.VolumeSnapshotSource{\n\t\t\t\t\t\tPersistentVolumeClaimName: stringPtr(\"testPVC\"),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\t\t\tBoundVolumeSnapshotContentName: stringPtr(\"testContent\"),\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshotContent: *builder.ForVolumeSnapshotContent(\"testContent\").Status(&snapshotv1api.VolumeSnapshotContentStatus{SnapshotHandle: stringPtr(\"testSnapshotHandle\")}).Result(),\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{},\n\t\t},\n\t\t{\n\t\t\tname: \"Normal VolumeSnapshot case\",\n\t\t\tvolumeSnapshot: snapshotv1api.VolumeSnapshot{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"testVS\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\tCreationTimestamp: now,\n\t\t\t\t},\n\t\t\t\tSpec: snapshotv1api.VolumeSnapshotSpec{\n\t\t\t\t\tVolumeSnapshotClassName: stringPtr(\"testClass\"),\n\t\t\t\t\tSource: snapshotv1api.VolumeSnapshotSource{\n\t\t\t\t\t\tPersistentVolumeClaimName: stringPtr(\"testPVC\"),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\t\t\tBoundVolumeSnapshotContentName: stringPtr(\"testContent\"),\n\t\t\t\t\tCreationTime: &now,\n\t\t\t\t\tRestoreSize: &resourceQuantity,\n\t\t\t\t\tReadyToUse: &readyToUse,\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshotContent: *builder.ForVolumeSnapshotContent(\"testContent\").Driver(\"pd.csi.storage.gke.io\").Status(&snapshotv1api.VolumeSnapshotContentStatus{SnapshotHandle: stringPtr(\"testSnapshotHandle\")}).Result(),\n\t\t\tpvMap: map[string]pvcPvInfo{\n\t\t\t\t\"testPV\": {\n\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\tPVCNamespace: \"velero\",\n\t\t\t\t\tPV: corev1api.PersistentVolume{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"testPV\",\n\t\t\t\t\t\t\tLabels: map[string]string{\"a\": \"b\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\t\tPersistentVolumeReclaimPolicy: corev1api.PersistentVolumeReclaimDelete,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\toperation: &itemoperation.BackupOperation{\n\t\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\t\tOperationID: \"testID\",\n\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\tGroupResource: schema.GroupResource{\n\t\t\t\t\t\t\tGroup: \"snapshot.storage.k8s.io\",\n\t\t\t\t\t\t\tResource: \"volumesnapshots\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\tName: \"testVS\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\tPVCNamespace: \"velero\",\n\t\t\t\t\tPVName: \"testPV\",\n\t\t\t\t\tBackupMethod: CSISnapshot,\n\t\t\t\t\tStartTimestamp: &now,\n\t\t\t\t\tPreserveLocalSnapshot: true,\n\t\t\t\t\tCSISnapshotInfo: &CSISnapshotInfo{\n\t\t\t\t\t\tDriver: \"pd.csi.storage.gke.io\",\n\t\t\t\t\t\tSnapshotHandle: \"testSnapshotHandle\",\n\t\t\t\t\t\tSize: 107374182400,\n\t\t\t\t\t\tVSCName: \"testContent\",\n\t\t\t\t\t\tOperationID: \"testID\",\n\t\t\t\t\t\tReadyToUse: &readyToUse,\n\t\t\t\t\t},\n\t\t\t\t\tPVInfo: &PVInfo{\n\t\t\t\t\t\tReclaimPolicy: \"Delete\",\n\t\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\t\"a\": \"b\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvolumesInfo := BackupVolumesInformation{}\n\t\t\tvolumesInfo.Init()\n\n\t\t\tif tc.pvMap != nil {\n\t\t\t\tfor k, v := range tc.pvMap {\n\t\t\t\t\tif k == v.PV.Name {\n\t\t\t\t\t\tvolumesInfo.pvMap.insert(v.PV, v.PVCName, v.PVCNamespace)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif tc.operation != nil {\n\t\t\t\tvolumesInfo.BackupOperations = append(volumesInfo.BackupOperations, tc.operation)\n\t\t\t}\n\n\t\t\tvolumesInfo.volumeSnapshots = []snapshotv1api.VolumeSnapshot{tc.volumeSnapshot}\n\t\t\tvolumesInfo.volumeSnapshotContents = []snapshotv1api.VolumeSnapshotContent{tc.volumeSnapshotContent}\n\t\t\tvolumesInfo.volumeSnapshotClasses = []snapshotv1api.VolumeSnapshotClass{tc.volumeSnapshotClass}\n\t\t\tvolumesInfo.logger = logging.DefaultLogger(logrus.DebugLevel, logging.FormatJSON)\n\n\t\t\tvolumesInfo.generateVolumeInfoForCSIVolumeSnapshot()\n\t\t\trequire.Equal(t, tc.expectedVolumeInfos, volumesInfo.volumeInfos)\n\t\t})\n\t}\n}\n\nfunc TestGenerateVolumeInfoFromPVB(t *testing.T) {\n\tnow := metav1.Now()\n\ttests := []struct {\n\t\tname string\n\t\tpvb *velerov1api.PodVolumeBackup\n\t\tpod *corev1api.Pod\n\t\tpvMap map[string]pvcPvInfo\n\t\texpectedVolumeInfos []*BackupVolumeInfo\n\t}{\n\t\t{\n\t\t\tname: \"cannot find PVB's pod, should fail\",\n\t\t\tpvb: builder.ForPodVolumeBackup(\"velero\", \"testPVB\").PodName(\"testPod\").PodNamespace(\"velero\").Result(),\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{},\n\t\t},\n\t\t{\n\t\t\tname: \"PVB doesn't have a related PVC\",\n\t\t\tpvb: builder.ForPodVolumeBackup(\"velero\", \"testPVB\").PodName(\"testPod\").PodNamespace(\"velero\").Result(),\n\t\t\tpod: builder.ForPod(\"velero\", \"testPod\").Containers(&corev1api.Container{\n\t\t\t\tName: \"test\",\n\t\t\t\tVolumeMounts: []corev1api.VolumeMount{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"testVolume\",\n\t\t\t\t\t\tMountPath: \"/data\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t}).Volumes(\n\t\t\t\t&corev1api.Volume{\n\t\t\t\t\tName: \"\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tHostPath: &corev1api.HostPathVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t).Result(),\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"\",\n\t\t\t\t\tPVCNamespace: \"\",\n\t\t\t\t\tPVName: \"\",\n\t\t\t\t\tBackupMethod: PodVolumeBackup,\n\t\t\t\t\tResult: VolumeResultFailed,\n\t\t\t\t\tPVBInfo: &PodVolumeInfo{\n\t\t\t\t\t\tPodName: \"testPod\",\n\t\t\t\t\t\tPodNamespace: \"velero\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Backup doesn't have information for PVC\",\n\t\t\tpvb: builder.ForPodVolumeBackup(\"velero\", \"testPVB\").PodName(\"testPod\").PodNamespace(\"velero\").Result(),\n\t\t\tpod: builder.ForPod(\"velero\", \"testPod\").Containers(&corev1api.Container{\n\t\t\t\tName: \"test\",\n\t\t\t\tVolumeMounts: []corev1api.VolumeMount{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"testVolume\",\n\t\t\t\t\t\tMountPath: \"/data\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t}).Volumes(\n\t\t\t\t&corev1api.Volume{\n\t\t\t\t\tName: \"\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\tClaimName: \"testPVC\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t).Result(),\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{},\n\t\t},\n\t\t{\n\t\t\tname: \"PVB's volume has a PVC with failed phase\",\n\t\t\tpvMap: map[string]pvcPvInfo{\n\t\t\t\t\"testPV\": {\n\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\tPVCNamespace: \"velero\",\n\t\t\t\t\tPV: corev1api.PersistentVolume{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"testPV\",\n\t\t\t\t\t\t\tLabels: map[string]string{\"a\": \"b\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\t\tPersistentVolumeReclaimPolicy: corev1api.PersistentVolumeReclaimDelete,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvb: builder.ForPodVolumeBackup(\"velero\", \"testPVB\").\n\t\t\t\tPodName(\"testPod\").\n\t\t\t\tPodNamespace(\"velero\").\n\t\t\t\tStartTimestamp(&now).\n\t\t\t\tCompletionTimestamp(&now).\n\t\t\t\tPhase(velerov1api.PodVolumeBackupPhaseFailed).\n\t\t\t\tResult(),\n\t\t\tpod: builder.ForPod(\"velero\", \"testPod\").Containers(&corev1api.Container{\n\t\t\t\tName: \"test\",\n\t\t\t\tVolumeMounts: []corev1api.VolumeMount{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"testVolume\",\n\t\t\t\t\t\tMountPath: \"/data\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t}).Volumes(\n\t\t\t\t&corev1api.Volume{\n\t\t\t\t\tName: \"\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\tClaimName: \"testPVC\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t).Result(),\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\tPVCNamespace: \"velero\",\n\t\t\t\t\tPVName: \"testPV\",\n\t\t\t\t\tBackupMethod: PodVolumeBackup,\n\t\t\t\t\tStartTimestamp: &now,\n\t\t\t\t\tCompletionTimestamp: &now,\n\t\t\t\t\tResult: VolumeResultFailed,\n\t\t\t\t\tPVBInfo: &PodVolumeInfo{\n\t\t\t\t\t\tPodName: \"testPod\",\n\t\t\t\t\t\tPodNamespace: \"velero\",\n\t\t\t\t\t\tPhase: velerov1api.PodVolumeBackupPhaseFailed,\n\t\t\t\t\t},\n\t\t\t\t\tPVInfo: &PVInfo{\n\t\t\t\t\t\tReclaimPolicy: string(corev1api.PersistentVolumeReclaimDelete),\n\t\t\t\t\t\tLabels: map[string]string{\"a\": \"b\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"PVB's volume has a PVC\",\n\t\t\tpvMap: map[string]pvcPvInfo{\n\t\t\t\t\"testPV\": {\n\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\tPVCNamespace: \"velero\",\n\t\t\t\t\tPV: corev1api.PersistentVolume{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"testPV\",\n\t\t\t\t\t\t\tLabels: map[string]string{\"a\": \"b\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\t\tPersistentVolumeReclaimPolicy: corev1api.PersistentVolumeReclaimDelete,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvb: builder.ForPodVolumeBackup(\"velero\", \"testPVB\").\n\t\t\t\tPodName(\"testPod\").\n\t\t\t\tPodNamespace(\"velero\").\n\t\t\t\tStartTimestamp(&now).\n\t\t\t\tCompletionTimestamp(&now).\n\t\t\t\tPhase(velerov1api.PodVolumeBackupPhaseCompleted).\n\t\t\t\tResult(),\n\t\t\tpod: builder.ForPod(\"velero\", \"testPod\").Containers(&corev1api.Container{\n\t\t\t\tName: \"test\",\n\t\t\t\tVolumeMounts: []corev1api.VolumeMount{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"testVolume\",\n\t\t\t\t\t\tMountPath: \"/data\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t}).Volumes(\n\t\t\t\t&corev1api.Volume{\n\t\t\t\t\tName: \"\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\tClaimName: \"testPVC\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t).Result(),\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\tPVCNamespace: \"velero\",\n\t\t\t\t\tPVName: \"testPV\",\n\t\t\t\t\tBackupMethod: PodVolumeBackup,\n\t\t\t\t\tStartTimestamp: &now,\n\t\t\t\t\tCompletionTimestamp: &now,\n\t\t\t\t\tResult: VolumeResultSucceeded,\n\t\t\t\t\tPVBInfo: &PodVolumeInfo{\n\t\t\t\t\t\tPodName: \"testPod\",\n\t\t\t\t\t\tPodNamespace: \"velero\",\n\t\t\t\t\t\tPhase: velerov1api.PodVolumeBackupPhaseCompleted,\n\t\t\t\t\t},\n\t\t\t\t\tPVInfo: &PVInfo{\n\t\t\t\t\t\tReclaimPolicy: string(corev1api.PersistentVolumeReclaimDelete),\n\t\t\t\t\t\tLabels: map[string]string{\"a\": \"b\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvolumesInfo := BackupVolumesInformation{}\n\t\t\tvolumesInfo.Init()\n\t\t\tvolumesInfo.crClient = velerotest.NewFakeControllerRuntimeClient(t)\n\n\t\t\tvolumesInfo.PodVolumeBackups = append(volumesInfo.PodVolumeBackups, tc.pvb)\n\n\t\t\tif tc.pvMap != nil {\n\t\t\t\tfor k, v := range tc.pvMap {\n\t\t\t\t\tif k == v.PV.Name {\n\t\t\t\t\t\tvolumesInfo.pvMap.insert(v.PV, v.PVCName, v.PVCNamespace)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\tif tc.pod != nil {\n\t\t\t\trequire.NoError(t, volumesInfo.crClient.Create(t.Context(), tc.pod))\n\t\t\t}\n\t\t\tvolumesInfo.logger = logging.DefaultLogger(logrus.DebugLevel, logging.FormatJSON)\n\n\t\t\tvolumesInfo.generateVolumeInfoFromPVB()\n\t\t\trequire.Equal(t, tc.expectedVolumeInfos, volumesInfo.volumeInfos)\n\t\t})\n\t}\n}\n\nfunc TestGenerateVolumeInfoFromDataUpload(t *testing.T) {\n\t// The unstructured conversion will loose the time precision to second\n\t// level. To make test pass. Set the now precision at second at the\n\t// beginning.\n\tnow := metav1.Now().Rfc3339Copy()\n\tfeatures.Enable(velerov1api.CSIFeatureFlag)\n\tdefer features.Disable(velerov1api.CSIFeatureFlag)\n\ttests := []struct {\n\t\tname string\n\t\tvs *snapshotv1api.VolumeSnapshot\n\t\tvsc *snapshotv1api.VolumeSnapshotContent\n\t\tdataUpload *velerov2alpha1.DataUpload\n\t\toperation *itemoperation.BackupOperation\n\t\tpvMap map[string]pvcPvInfo\n\t\texpectedVolumeInfos []*BackupVolumeInfo\n\t}{\n\t\t{\n\t\t\tname: \"Operation is not for PVC\",\n\t\t\toperation: &itemoperation.BackupOperation{\n\t\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\tGroupResource: schema.GroupResource{\n\t\t\t\t\t\t\tGroup: \"\",\n\t\t\t\t\t\t\tResource: \"configmaps\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{},\n\t\t},\n\t\t{\n\t\t\tname: \"Operation doesn't have DataUpload PostItemOperation\",\n\t\t\toperation: &itemoperation.BackupOperation{\n\t\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\tGroupResource: schema.GroupResource{\n\t\t\t\t\t\t\tGroup: \"\",\n\t\t\t\t\t\t\tResource: \"persistentvolumeclaims\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\tName: \"testPVC\",\n\t\t\t\t\t},\n\t\t\t\t\tPostOperationItems: []velero.ResourceIdentifier{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tGroupResource: schema.GroupResource{\n\t\t\t\t\t\t\t\tGroup: \"\",\n\t\t\t\t\t\t\t\tResource: \"configmaps\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{},\n\t\t},\n\t\t{\n\t\t\tname: \"DataUpload cannot be found for operation\",\n\t\t\toperation: &itemoperation.BackupOperation{\n\t\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\t\tOperationID: \"testOperation\",\n\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\tGroupResource: schema.GroupResource{\n\t\t\t\t\t\t\tGroup: \"\",\n\t\t\t\t\t\t\tResource: \"persistentvolumeclaims\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\tName: \"testPVC\",\n\t\t\t\t\t},\n\t\t\t\t\tPostOperationItems: []velero.ResourceIdentifier{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tGroupResource: schema.GroupResource{\n\t\t\t\t\t\t\t\tGroup: \"velero.io\",\n\t\t\t\t\t\t\t\tResource: \"datauploads\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"testDU\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{},\n\t\t},\n\t\t{\n\t\t\tname: \"Normal DataUpload case\",\n\t\t\tdataUpload: builder.ForDataUpload(\"velero\", \"testDU\").\n\t\t\t\tDataMover(\"velero\").\n\t\t\t\tCSISnapshot(&velerov2alpha1.CSISnapshotSpec{\n\t\t\t\t\tVolumeSnapshot: \"vs-01\",\n\t\t\t\t\tSnapshotClass: \"testClass\",\n\t\t\t\t\tDriver: \"pd.csi.storage.gke.io\",\n\t\t\t\t}).SnapshotID(\"testSnapshotHandle\").\n\t\t\t\tStartTimestamp(&now).\n\t\t\t\tPhase(velerov2alpha1.DataUploadPhaseCompleted).\n\t\t\t\tResult(),\n\t\t\tvs: builder.ForVolumeSnapshot(velerov1api.DefaultNamespace, \"vs-01\").Status().BoundVolumeSnapshotContentName(\"vsc-01\").Result(),\n\t\t\tvsc: builder.ForVolumeSnapshotContent(\"vsc-01\").Driver(\"pd.csi.storage.gke.io\").Result(),\n\t\t\toperation: &itemoperation.BackupOperation{\n\t\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\t\tOperationID: \"testOperation\",\n\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\tGroupResource: schema.GroupResource{\n\t\t\t\t\t\t\tGroup: \"\",\n\t\t\t\t\t\t\tResource: \"persistentvolumeclaims\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\tName: \"testPVC\",\n\t\t\t\t\t},\n\t\t\t\t\tPostOperationItems: []velero.ResourceIdentifier{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tGroupResource: schema.GroupResource{\n\t\t\t\t\t\t\t\tGroup: \"velero.io\",\n\t\t\t\t\t\t\t\tResource: \"datauploads\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"testDU\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvMap: map[string]pvcPvInfo{\n\t\t\t\t\"testPV\": {\n\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\tPVCNamespace: \"velero\",\n\t\t\t\t\tPV: corev1api.PersistentVolume{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"testPV\",\n\t\t\t\t\t\t\tLabels: map[string]string{\"a\": \"b\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\t\tPersistentVolumeReclaimPolicy: corev1api.PersistentVolumeReclaimDelete,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\tPVCNamespace: \"velero\",\n\t\t\t\t\tPVName: \"testPV\",\n\t\t\t\t\tBackupMethod: CSISnapshot,\n\t\t\t\t\tSnapshotDataMoved: true,\n\t\t\t\t\tStartTimestamp: &now,\n\t\t\t\t\tCSISnapshotInfo: &CSISnapshotInfo{\n\t\t\t\t\t\tVSCName: FieldValueIsUnknown,\n\t\t\t\t\t\tSnapshotHandle: FieldValueIsUnknown,\n\t\t\t\t\t\tOperationID: FieldValueIsUnknown,\n\t\t\t\t\t\tSize: 0,\n\t\t\t\t\t\tDriver: \"pd.csi.storage.gke.io\",\n\t\t\t\t\t},\n\t\t\t\t\tSnapshotDataMovementInfo: &SnapshotDataMovementInfo{\n\t\t\t\t\t\tDataMover: \"velero\",\n\t\t\t\t\t\tUploaderType: \"kopia\",\n\t\t\t\t\t\tOperationID: \"testOperation\",\n\t\t\t\t\t\tPhase: velerov2alpha1.DataUploadPhaseCompleted,\n\t\t\t\t\t},\n\t\t\t\t\tPVInfo: &PVInfo{\n\t\t\t\t\t\tReclaimPolicy: string(corev1api.PersistentVolumeReclaimDelete),\n\t\t\t\t\t\tLabels: map[string]string{\"a\": \"b\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvolumesInfo := BackupVolumesInformation{}\n\t\t\tvolumesInfo.Init()\n\n\t\t\tif tc.operation != nil {\n\t\t\t\tvolumesInfo.BackupOperations = append(volumesInfo.BackupOperations, tc.operation)\n\t\t\t}\n\n\t\t\tif tc.pvMap != nil {\n\t\t\t\tfor k, v := range tc.pvMap {\n\t\t\t\t\tif k == v.PV.Name {\n\t\t\t\t\t\tvolumesInfo.pvMap.insert(v.PV, v.PVCName, v.PVCNamespace)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tobjects := make([]runtime.Object, 0)\n\t\t\tif tc.dataUpload != nil {\n\t\t\t\tobjects = append(objects, tc.dataUpload)\n\t\t\t}\n\t\t\tif tc.vs != nil {\n\t\t\t\tobjects = append(objects, tc.vs)\n\t\t\t}\n\t\t\tif tc.vsc != nil {\n\t\t\t\tobjects = append(objects, tc.vsc)\n\t\t\t}\n\t\t\tvolumesInfo.crClient = velerotest.NewFakeControllerRuntimeClient(t, objects...)\n\n\t\t\tvolumesInfo.logger = logging.DefaultLogger(logrus.DebugLevel, logging.FormatJSON)\n\n\t\t\tvolumesInfo.generateVolumeInfoFromDataUpload()\n\n\t\t\tif len(tc.expectedVolumeInfos) > 0 {\n\t\t\t\trequire.Equal(t, tc.expectedVolumeInfos[0].PVInfo, volumesInfo.volumeInfos[0].PVInfo)\n\t\t\t\trequire.Equal(t, tc.expectedVolumeInfos[0].SnapshotDataMovementInfo, volumesInfo.volumeInfos[0].SnapshotDataMovementInfo)\n\t\t\t\trequire.Equal(t, tc.expectedVolumeInfos[0].CSISnapshotInfo, volumesInfo.volumeInfos[0].CSISnapshotInfo)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestRestoreVolumeInfoTrackNativeSnapshot(t *testing.T) {\n\tfakeCilent := velerotest.NewFakeControllerRuntimeClient(t)\n\n\trestore := builder.ForRestore(\"velero\", \"testRestore\").Result()\n\ttracker := NewRestoreVolInfoTracker(restore, logrus.New(), fakeCilent)\n\ttracker.TrackNativeSnapshot(\"testPV\", \"snap-001\", \"ebs\", \"us-west-1\", 10000)\n\tassert.Equal(t, NativeSnapshotInfo{\n\t\tSnapshotHandle: \"snap-001\",\n\t\tVolumeType: \"ebs\",\n\t\tVolumeAZ: \"us-west-1\",\n\t\tIOPS: \"10000\",\n\t}, *tracker.pvNativeSnapshotMap[\"testPV\"])\n\ttracker.TrackNativeSnapshot(\"testPV\", \"snap-002\", \"ebs\", \"us-west-2\", 15000)\n\tassert.Equal(t, NativeSnapshotInfo{\n\t\tSnapshotHandle: \"snap-002\",\n\t\tVolumeType: \"ebs\",\n\t\tVolumeAZ: \"us-west-2\",\n\t\tIOPS: \"15000\",\n\t}, *tracker.pvNativeSnapshotMap[\"testPV\"])\n\ttracker.RenamePVForNativeSnapshot(\"testPV\", \"newPV\")\n\t_, ok := tracker.pvNativeSnapshotMap[\"testPV\"]\n\tassert.False(t, ok)\n\tassert.Equal(t, NativeSnapshotInfo{\n\t\tSnapshotHandle: \"snap-002\",\n\t\tVolumeType: \"ebs\",\n\t\tVolumeAZ: \"us-west-2\",\n\t\tIOPS: \"15000\",\n\t}, *tracker.pvNativeSnapshotMap[\"newPV\"])\n}\n\nfunc TestRestoreVolumeInfoResult(t *testing.T) {\n\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t,\n\t\tbuilder.ForPod(\"testNS\", \"testPod\").\n\t\t\tVolumes(builder.ForVolume(\"data-volume-1\").PersistentVolumeClaimSource(\"testPVC2\").Result()).\n\t\t\tResult())\n\ttestRestore := builder.ForRestore(\"velero\", \"testRestore\").Result()\n\ttests := []struct {\n\t\tname string\n\t\ttracker *RestoreVolumeInfoTracker\n\t\texpectResultValues []RestoreVolumeInfo\n\t}{\n\t\t{\n\t\t\tname: \"empty\",\n\t\t\ttracker: &RestoreVolumeInfoTracker{\n\t\t\t\tMutex: &sync.Mutex{},\n\t\t\t\tclient: fakeClient,\n\t\t\t\tlog: logrus.New(),\n\t\t\t\trestore: testRestore,\n\t\t\t\tpvPvc: &pvcPvMap{\n\t\t\t\t\tdata: make(map[string]pvcPvInfo),\n\t\t\t\t},\n\t\t\t\tpvNativeSnapshotMap: map[string]*NativeSnapshotInfo{},\n\t\t\t\tpvcCSISnapshotMap: map[string]snapshotv1api.VolumeSnapshot{},\n\t\t\t\tdatadownloadList: &velerov2alpha1.DataDownloadList{},\n\t\t\t\tpvrs: []*velerov1api.PodVolumeRestore{},\n\t\t\t},\n\t\t\texpectResultValues: []RestoreVolumeInfo{},\n\t\t},\n\t\t{\n\t\t\tname: \"native snapshot and podvolumes\",\n\t\t\ttracker: &RestoreVolumeInfoTracker{\n\t\t\t\tMutex: &sync.Mutex{},\n\t\t\t\tclient: fakeClient,\n\t\t\t\tlog: logrus.New(),\n\t\t\t\trestore: testRestore,\n\t\t\t\tpvPvc: &pvcPvMap{\n\t\t\t\t\tdata: map[string]pvcPvInfo{\n\t\t\t\t\t\t\"testPV\": {\n\t\t\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\t\t\tPVCNamespace: \"testNS\",\n\t\t\t\t\t\t\tPV: *builder.ForPersistentVolume(\"testPV\").Result(),\n\t\t\t\t\t\t},\n\t\t\t\t\t\t\"testPV2\": {\n\t\t\t\t\t\t\tPVCName: \"testPVC2\",\n\t\t\t\t\t\t\tPVCNamespace: \"testNS\",\n\t\t\t\t\t\t\tPV: *builder.ForPersistentVolume(\"testPV2\").Result(),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tpvNativeSnapshotMap: map[string]*NativeSnapshotInfo{\n\t\t\t\t\t\"testPV\": {\n\t\t\t\t\t\tSnapshotHandle: \"snap-001\",\n\t\t\t\t\t\tVolumeType: \"ebs\",\n\t\t\t\t\t\tVolumeAZ: \"us-west-1\",\n\t\t\t\t\t\tIOPS: \"10000\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tpvcCSISnapshotMap: map[string]snapshotv1api.VolumeSnapshot{},\n\t\t\t\tdatadownloadList: &velerov2alpha1.DataDownloadList{},\n\t\t\t\tpvrs: []*velerov1api.PodVolumeRestore{\n\t\t\t\t\tbuilder.ForPodVolumeRestore(\"velero\", \"testRestore-1234\").\n\t\t\t\t\t\tPodNamespace(\"testNS\").\n\t\t\t\t\t\tPodName(\"testPod\").\n\t\t\t\t\t\tVolume(\"data-volume-1\").\n\t\t\t\t\t\tUploaderType(\"kopia\").\n\t\t\t\t\t\tSnapshotID(\"pvr-snap-001\").Result(),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectResultValues: []RestoreVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"testPVC2\",\n\t\t\t\t\tPVCNamespace: \"testNS\",\n\t\t\t\t\tPVName: \"testPV2\",\n\t\t\t\t\tRestoreMethod: PodVolumeRestore,\n\t\t\t\t\tSnapshotDataMoved: false,\n\t\t\t\t\tPVRInfo: &PodVolumeInfo{\n\t\t\t\t\t\tSnapshotHandle: \"pvr-snap-001\",\n\t\t\t\t\t\tPodName: \"testPod\",\n\t\t\t\t\t\tPodNamespace: \"testNS\",\n\t\t\t\t\t\tUploaderType: \"kopia\",\n\t\t\t\t\t\tVolumeName: \"data-volume-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\tPVCNamespace: \"testNS\",\n\t\t\t\t\tPVName: \"testPV\",\n\t\t\t\t\tRestoreMethod: NativeSnapshot,\n\t\t\t\t\tSnapshotDataMoved: false,\n\t\t\t\t\tNativeSnapshotInfo: &NativeSnapshotInfo{\n\t\t\t\t\t\tSnapshotHandle: \"snap-001\",\n\t\t\t\t\t\tVolumeType: \"ebs\",\n\t\t\t\t\t\tVolumeAZ: \"us-west-1\",\n\t\t\t\t\t\tIOPS: \"10000\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"CSI snapshot without datamovement and podvolumes\",\n\t\t\ttracker: &RestoreVolumeInfoTracker{\n\t\t\t\tMutex: &sync.Mutex{},\n\t\t\t\tclient: fakeClient,\n\t\t\t\tlog: logrus.New(),\n\t\t\t\trestore: testRestore,\n\t\t\t\tpvPvc: &pvcPvMap{\n\t\t\t\t\tdata: map[string]pvcPvInfo{\n\t\t\t\t\t\t\"testPV\": {\n\t\t\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\t\t\tPVCNamespace: \"testNS\",\n\t\t\t\t\t\t\tPV: *builder.ForPersistentVolume(\"testPV\").Result(),\n\t\t\t\t\t\t},\n\t\t\t\t\t\t\"testPV2\": {\n\t\t\t\t\t\t\tPVCName: \"testPVC2\",\n\t\t\t\t\t\t\tPVCNamespace: \"testNS\",\n\t\t\t\t\t\t\tPV: *builder.ForPersistentVolume(\"testPV2\").Result(),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tpvNativeSnapshotMap: map[string]*NativeSnapshotInfo{},\n\t\t\t\tpvcCSISnapshotMap: map[string]snapshotv1api.VolumeSnapshot{\n\t\t\t\t\t\"testNS/testPVC\": *builder.ForVolumeSnapshot(\"sourceNS\", \"testCSISnapshot\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithAnnotations(velerov1api.VolumeSnapshotHandleAnnotation, \"csi-snap-001\",\n\t\t\t\t\t\t\t\tvelerov1api.DriverNameAnnotation, \"test-csi-driver\"),\n\t\t\t\t\t\t).SourceVolumeSnapshotContentName(\"test-vsc-001\").\n\t\t\t\t\t\tStatus().RestoreSize(\"1Gi\").Result(),\n\t\t\t\t},\n\t\t\t\tdatadownloadList: &velerov2alpha1.DataDownloadList{},\n\t\t\t\tpvrs: []*velerov1api.PodVolumeRestore{\n\t\t\t\t\tbuilder.ForPodVolumeRestore(\"velero\", \"testRestore-1234\").\n\t\t\t\t\t\tPodNamespace(\"testNS\").\n\t\t\t\t\t\tPodName(\"testPod\").\n\t\t\t\t\t\tVolume(\"data-volume-1\").\n\t\t\t\t\t\tUploaderType(\"kopia\").\n\t\t\t\t\t\tSnapshotID(\"pvr-snap-001\").Result(),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectResultValues: []RestoreVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"testPVC2\",\n\t\t\t\t\tPVCNamespace: \"testNS\",\n\t\t\t\t\tPVName: \"testPV2\",\n\t\t\t\t\tRestoreMethod: PodVolumeRestore,\n\t\t\t\t\tSnapshotDataMoved: false,\n\t\t\t\t\tPVRInfo: &PodVolumeInfo{\n\t\t\t\t\t\tSnapshotHandle: \"pvr-snap-001\",\n\t\t\t\t\t\tPodName: \"testPod\",\n\t\t\t\t\t\tPodNamespace: \"testNS\",\n\t\t\t\t\t\tUploaderType: \"kopia\",\n\t\t\t\t\t\tVolumeName: \"data-volume-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\tPVCNamespace: \"testNS\",\n\t\t\t\t\tPVName: \"testPV\",\n\t\t\t\t\tRestoreMethod: CSISnapshot,\n\t\t\t\t\tSnapshotDataMoved: false,\n\t\t\t\t\tCSISnapshotInfo: &CSISnapshotInfo{\n\t\t\t\t\t\tSnapshotHandle: \"csi-snap-001\",\n\t\t\t\t\t\tVSCName: \"test-vsc-001\",\n\t\t\t\t\t\tSize: 1073741824,\n\t\t\t\t\t\tDriver: \"test-csi-driver\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"CSI snapshot with datamovement\",\n\t\t\ttracker: &RestoreVolumeInfoTracker{\n\t\t\t\tMutex: &sync.Mutex{},\n\t\t\t\tclient: fakeClient,\n\t\t\t\tlog: logrus.New(),\n\t\t\t\trestore: testRestore,\n\t\t\t\tpvPvc: &pvcPvMap{\n\t\t\t\t\tdata: map[string]pvcPvInfo{\n\t\t\t\t\t\t\"testPV\": {\n\t\t\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\t\t\tPVCNamespace: \"testNS\",\n\t\t\t\t\t\t\tPV: *builder.ForPersistentVolume(\"testPV\").Result(),\n\t\t\t\t\t\t},\n\t\t\t\t\t\t\"testPV2\": {\n\t\t\t\t\t\t\tPVCName: \"testPVC2\",\n\t\t\t\t\t\t\tPVCNamespace: \"testNS\",\n\t\t\t\t\t\t\tPV: *builder.ForPersistentVolume(\"testPV2\").Result(),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tpvNativeSnapshotMap: map[string]*NativeSnapshotInfo{},\n\t\t\t\tpvcCSISnapshotMap: map[string]snapshotv1api.VolumeSnapshot{},\n\t\t\t\tdatadownloadList: &velerov2alpha1.DataDownloadList{\n\t\t\t\t\tItems: []velerov2alpha1.DataDownload{\n\t\t\t\t\t\t*builder.ForDataDownload(\"velero\", \"testDataDownload-1\").\n\t\t\t\t\t\t\tObjectMeta(builder.WithLabels(velerov1api.AsyncOperationIDLabel, \"dd-operation-001\")).\n\t\t\t\t\t\t\tSnapshotID(\"dd-snap-001\").\n\t\t\t\t\t\t\tTargetVolume(velerov2alpha1.TargetVolumeSpec{\n\t\t\t\t\t\t\t\tPVC: \"testPVC\",\n\t\t\t\t\t\t\t\tNamespace: \"testNS\",\n\t\t\t\t\t\t\t}).\n\t\t\t\t\t\t\tResult(),\n\t\t\t\t\t\t*builder.ForDataDownload(\"velero\", \"testDataDownload-2\").\n\t\t\t\t\t\t\tObjectMeta(builder.WithLabels(velerov1api.AsyncOperationIDLabel, \"dd-operation-002\")).\n\t\t\t\t\t\t\tSnapshotID(\"dd-snap-002\").\n\t\t\t\t\t\t\tTargetVolume(velerov2alpha1.TargetVolumeSpec{\n\t\t\t\t\t\t\t\tPVC: \"testPVC2\",\n\t\t\t\t\t\t\t\tNamespace: \"testNS\",\n\t\t\t\t\t\t\t}).\n\t\t\t\t\t\t\tResult(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tpvrs: []*velerov1api.PodVolumeRestore{},\n\t\t\t},\n\t\t\texpectResultValues: []RestoreVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"testPVC\",\n\t\t\t\t\tPVCNamespace: \"testNS\",\n\t\t\t\t\tPVName: \"testPV\",\n\t\t\t\t\tRestoreMethod: CSISnapshot,\n\t\t\t\t\tSnapshotDataMoved: true,\n\t\t\t\t\tSnapshotDataMovementInfo: &SnapshotDataMovementInfo{\n\t\t\t\t\t\tDataMover: \"velero\",\n\t\t\t\t\t\tUploaderType: velerov1api.BackupRepositoryTypeKopia,\n\t\t\t\t\t\tSnapshotHandle: \"dd-snap-001\",\n\t\t\t\t\t\tOperationID: \"dd-operation-001\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"testPVC2\",\n\t\t\t\t\tPVCNamespace: \"testNS\",\n\t\t\t\t\tPVName: \"testPV2\",\n\t\t\t\t\tRestoreMethod: CSISnapshot,\n\t\t\t\t\tSnapshotDataMoved: true,\n\t\t\t\t\tSnapshotDataMovementInfo: &SnapshotDataMovementInfo{\n\t\t\t\t\t\tDataMover: \"velero\",\n\t\t\t\t\t\tUploaderType: velerov1api.BackupRepositoryTypeKopia,\n\t\t\t\t\t\tSnapshotHandle: \"dd-snap-002\",\n\t\t\t\t\t\tOperationID: \"dd-operation-002\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tresult := tc.tracker.Result()\n\t\t\tvaluesList := []RestoreVolumeInfo{}\n\t\t\tfor _, item := range result {\n\t\t\t\tvaluesList = append(valuesList, *item)\n\t\t\t}\n\t\t\tassert.Equal(t, tc.expectResultValues, valuesList)\n\t\t})\n\t}\n}\n\nfunc stringPtr(str string) *string {\n\treturn &str\n}\n\nfunc int64Ptr(val int) *int64 {\n\ti := int64(val)\n\treturn &i\n}\n\nfunc TestGetVolumeSnapshotClasses(t *testing.T) {\n\tclass := &snapshotv1api.VolumeSnapshotClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"class\",\n\t\t\tResourceVersion: \"999\",\n\t\t},\n\t}\n\tvolumesInfo := BackupVolumesInformation{\n\t\tlogger: logging.DefaultLogger(logrus.DebugLevel, logging.FormatJSON),\n\t\tcrClient: velerotest.NewFakeControllerRuntimeClient(t, class),\n\t}\n\n\tresult, err := volumesInfo.getVolumeSnapshotClasses()\n\trequire.NoError(t, err)\n\trequire.Equal(t, []snapshotv1api.VolumeSnapshotClass{*class}, result)\n}\n" }, { "path": "internal/volumehelper/volume_policy_helper.go", "content": "package volumehelper\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strings\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\tcrclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/resourcepolicies\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\tkubeutil \"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\tpodvolumeutil \"github.com/vmware-tanzu/velero/pkg/util/podvolume\"\n)\n\ntype VolumeHelper interface {\n\tShouldPerformSnapshot(obj runtime.Unstructured, groupResource schema.GroupResource) (bool, error)\n\tShouldPerformFSBackup(volume corev1api.Volume, pod corev1api.Pod) (bool, error)\n}\n\ntype volumeHelperImpl struct {\n\tvolumePolicy *resourcepolicies.Policies\n\tsnapshotVolumes *bool\n\tlogger logrus.FieldLogger\n\tclient crclient.Client\n\tdefaultVolumesToFSBackup bool\n\t// This parameter is used to align the fs-backup with snapshot action,\n\t// because PVC is already filtered by the resource filter before getting\n\t// to the volume policy check, but fs-backup is based on the pod resource,\n\t// the resource filter on PVC and PV doesn't work on this scenario.\n\tbackupExcludePVC bool\n\t// pvcPodCache provides cached PVC to Pod mappings for improved performance.\n\t// When there are many PVCs and pods, using this cache avoids O(N*M) lookups.\n\tpvcPodCache *podvolumeutil.PVCPodCache\n}\n\n// NewVolumeHelperImpl creates a VolumeHelper without PVC-to-Pod caching.\n//\n// Deprecated: Use NewVolumeHelperImplWithNamespaces or NewVolumeHelperImplWithCache instead\n// for better performance. These functions provide PVC-to-Pod caching which avoids O(N*M)\n// complexity when there are many PVCs and pods. See issue #9179 for details.\nfunc NewVolumeHelperImpl(\n\tvolumePolicy *resourcepolicies.Policies,\n\tsnapshotVolumes *bool,\n\tlogger logrus.FieldLogger,\n\tclient crclient.Client,\n\tdefaultVolumesToFSBackup bool,\n\tbackupExcludePVC bool,\n) VolumeHelper {\n\t// Pass nil namespaces - no cache will be built, so this never fails.\n\t// This is used by plugins that don't need the cache optimization.\n\tvh, _ := NewVolumeHelperImplWithNamespaces(\n\t\tvolumePolicy,\n\t\tsnapshotVolumes,\n\t\tlogger,\n\t\tclient,\n\t\tdefaultVolumesToFSBackup,\n\t\tbackupExcludePVC,\n\t\tnil,\n\t)\n\treturn vh\n}\n\n// NewVolumeHelperImplWithNamespaces creates a VolumeHelper with a PVC-to-Pod cache for improved performance.\n// The cache is built internally from the provided namespaces list.\n// This avoids O(N*M) complexity when there are many PVCs and pods.\n// See issue #9179 for details.\n// Returns an error if cache building fails - callers should not proceed with backup in this case.\nfunc NewVolumeHelperImplWithNamespaces(\n\tvolumePolicy *resourcepolicies.Policies,\n\tsnapshotVolumes *bool,\n\tlogger logrus.FieldLogger,\n\tclient crclient.Client,\n\tdefaultVolumesToFSBackup bool,\n\tbackupExcludePVC bool,\n\tnamespaces []string,\n) (VolumeHelper, error) {\n\tvar pvcPodCache *podvolumeutil.PVCPodCache\n\tif len(namespaces) > 0 {\n\t\tpvcPodCache = podvolumeutil.NewPVCPodCache()\n\t\tif err := pvcPodCache.BuildCacheForNamespaces(context.Background(), namespaces, client); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tlogger.Infof(\"Built PVC-to-Pod cache for %d namespaces\", len(namespaces))\n\t}\n\n\treturn &volumeHelperImpl{\n\t\tvolumePolicy: volumePolicy,\n\t\tsnapshotVolumes: snapshotVolumes,\n\t\tlogger: logger,\n\t\tclient: client,\n\t\tdefaultVolumesToFSBackup: defaultVolumesToFSBackup,\n\t\tbackupExcludePVC: backupExcludePVC,\n\t\tpvcPodCache: pvcPodCache,\n\t}, nil\n}\n\n// NewVolumeHelperImplWithCache creates a VolumeHelper using an externally managed PVC-to-Pod cache.\n// This is used by plugins that build the cache lazily per-namespace (following the pattern from PR #9226).\n// The cache can be nil, in which case PVC-to-Pod lookups will fall back to direct API calls.\nfunc NewVolumeHelperImplWithCache(\n\tbackup velerov1api.Backup,\n\tclient crclient.Client,\n\tlogger logrus.FieldLogger,\n\tpvcPodCache *podvolumeutil.PVCPodCache,\n) (VolumeHelper, error) {\n\tresourcePolicies, err := resourcepolicies.GetResourcePoliciesFromBackup(backup, client, logger)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to get volume policies from backup\")\n\t}\n\n\treturn &volumeHelperImpl{\n\t\tvolumePolicy: resourcePolicies,\n\t\tsnapshotVolumes: backup.Spec.SnapshotVolumes,\n\t\tlogger: logger,\n\t\tclient: client,\n\t\tdefaultVolumesToFSBackup: boolptr.IsSetToTrue(backup.Spec.DefaultVolumesToFsBackup),\n\t\tbackupExcludePVC: boolptr.IsSetToTrue(backup.Spec.SnapshotMoveData),\n\t\tpvcPodCache: pvcPodCache,\n\t}, nil\n}\n\nfunc (v *volumeHelperImpl) ShouldPerformSnapshot(obj runtime.Unstructured, groupResource schema.GroupResource) (bool, error) {\n\t// check if volume policy exists and also check if the object(pv/pvc) fits a volume policy criteria and see if the associated action is snapshot\n\t// if it is not snapshot then skip the code path for snapshotting the PV/PVC\n\tpvc := new(corev1api.PersistentVolumeClaim)\n\tpv := new(corev1api.PersistentVolume)\n\tvar err error\n\n\tvar pvNotFoundErr error\n\tif groupResource == kuberesource.PersistentVolumeClaims {\n\t\tif err = runtime.DefaultUnstructuredConverter.FromUnstructured(obj.UnstructuredContent(), &pvc); err != nil {\n\t\t\tv.logger.WithError(err).Error(\"fail to convert unstructured into PVC\")\n\t\t\treturn false, err\n\t\t}\n\n\t\tpv, err = kubeutil.GetPVForPVC(pvc, v.client)\n\t\tif err != nil {\n\t\t\t// Any error means PV not available - save to return later if no policy matches\n\t\t\tv.logger.Debugf(\"PV not found for PVC %s: %v\", pvc.Namespace+\"/\"+pvc.Name, err)\n\t\t\tpvNotFoundErr = err\n\t\t\tpv = nil\n\t\t}\n\t}\n\n\tif groupResource == kuberesource.PersistentVolumes {\n\t\tif err = runtime.DefaultUnstructuredConverter.FromUnstructured(obj.UnstructuredContent(), &pv); err != nil {\n\t\t\tv.logger.WithError(err).Error(\"fail to convert unstructured into PV\")\n\t\t\treturn false, err\n\t\t}\n\t}\n\n\tif v.volumePolicy != nil {\n\t\tvfd := resourcepolicies.NewVolumeFilterData(pv, nil, pvc)\n\t\taction, err := v.volumePolicy.GetMatchAction(vfd)\n\t\tif err != nil {\n\t\t\tv.logger.WithError(err).Errorf(\"fail to get VolumePolicy match action for %+v\", vfd)\n\t\t\treturn false, err\n\t\t}\n\n\t\t// If there is a match action, and the action type is snapshot, return true,\n\t\t// or the action type is not snapshot, then return false.\n\t\t// If there is no match action, go on to the next check.\n\t\tif action != nil {\n\t\t\tif action.Type == resourcepolicies.Snapshot {\n\t\t\t\tv.logger.Infof(\"performing snapshot action for %+v\", vfd)\n\t\t\t\treturn true, nil\n\t\t\t} else {\n\t\t\t\tv.logger.Infof(\"Skip snapshot action for %+v as the action type is %s\", vfd, action.Type)\n\t\t\t\treturn false, nil\n\t\t\t}\n\t\t}\n\t}\n\n\t// If resource is PVC, and PV is nil (e.g., Pending/Lost PVC with no matching policy), return the original error\n\tif groupResource == kuberesource.PersistentVolumeClaims && pv == nil && pvNotFoundErr != nil {\n\t\tv.logger.WithError(pvNotFoundErr).Errorf(\"fail to get PV for PVC %s\", pvc.Namespace+\"/\"+pvc.Name)\n\t\treturn false, pvNotFoundErr\n\t}\n\n\t// If this PV is claimed, see if we've already taken a (pod volume backup)\n\t// snapshot of the contents of this PV. If so, don't take a snapshot.\n\tif pv.Spec.ClaimRef != nil {\n\t\t// Use cached lookup if available for better performance with many PVCs/pods\n\t\tpods, err := podvolumeutil.GetPodsUsingPVCWithCache(\n\t\t\tpv.Spec.ClaimRef.Namespace,\n\t\t\tpv.Spec.ClaimRef.Name,\n\t\t\tv.client,\n\t\t\tv.pvcPodCache,\n\t\t)\n\t\tif err != nil {\n\t\t\tv.logger.WithError(err).Errorf(\"fail to get pod for PV %s\", pv.Name)\n\t\t\treturn false, err\n\t\t}\n\n\t\tfor _, pod := range pods {\n\t\t\tfor _, vol := range pod.Spec.Volumes {\n\t\t\t\tif vol.PersistentVolumeClaim != nil &&\n\t\t\t\t\tvol.PersistentVolumeClaim.ClaimName == pv.Spec.ClaimRef.Name &&\n\t\t\t\t\tv.shouldPerformFSBackupLegacy(vol, pod) {\n\t\t\t\t\tv.logger.Infof(\"Skipping snapshot of pv %s because it is backed up with PodVolumeBackup.\", pv.Name)\n\t\t\t\t\treturn false, nil\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\tif !boolptr.IsSetToFalse(v.snapshotVolumes) {\n\t\t// If the backup.Spec.SnapshotVolumes is not set, or set to true, then should take the snapshot.\n\t\tv.logger.Infof(\"performing snapshot action for pv %s as the snapshotVolumes is not set to false\", pv.Name)\n\t\treturn true, nil\n\t}\n\n\tv.logger.Infof(\"skipping snapshot action for pv %s possibly due to no volume policy setting or snapshotVolumes is false\", pv.Name)\n\treturn false, nil\n}\n\nfunc (v volumeHelperImpl) ShouldPerformFSBackup(volume corev1api.Volume, pod corev1api.Pod) (bool, error) {\n\tif !v.shouldIncludeVolumeInBackup(volume) {\n\t\tv.logger.Debugf(\"skip fs-backup action for pod %s's volume %s, due to not pass volume check.\", pod.Namespace+\"/\"+pod.Name, volume.Name)\n\t\treturn false, nil\n\t}\n\n\tvar pvNotFoundErr error\n\tif v.volumePolicy != nil {\n\t\tvar resource any\n\t\tvar err error\n\t\tresource = &volume\n\t\tvar pvc = &corev1api.PersistentVolumeClaim{}\n\t\tif volume.VolumeSource.PersistentVolumeClaim != nil {\n\t\t\tpvc, err = kubeutil.GetPVCForPodVolume(&volume, &pod, v.client)\n\t\t\tif err != nil {\n\t\t\t\tv.logger.WithError(err).Errorf(\"fail to get PVC for pod %s\", pod.Namespace+\"/\"+pod.Name)\n\t\t\t\treturn false, err\n\t\t\t}\n\t\t\tpvResource, err := kubeutil.GetPVForPVC(pvc, v.client)\n\t\t\tif err != nil {\n\t\t\t\t// Any error means PV not available - save to return later if no policy matches\n\t\t\t\tv.logger.Debugf(\"PV not found for PVC %s: %v\", pvc.Namespace+\"/\"+pvc.Name, err)\n\t\t\t\tpvNotFoundErr = err\n\t\t\t} else {\n\t\t\t\tresource = pvResource\n\t\t\t}\n\t\t}\n\n\t\tpv, podVolume, err := v.getVolumeFromResource(resource)\n\t\tif err != nil {\n\t\t\treturn false, err\n\t\t}\n\n\t\tvfd := resourcepolicies.NewVolumeFilterData(pv, podVolume, pvc)\n\t\taction, err := v.volumePolicy.GetMatchAction(vfd)\n\t\tif err != nil {\n\t\t\tv.logger.WithError(err).Error(\"fail to get VolumePolicy match action for volume\")\n\t\t\treturn false, err\n\t\t}\n\n\t\tif action != nil {\n\t\t\tif action.Type == resourcepolicies.FSBackup {\n\t\t\t\tv.logger.Infof(\"Perform fs-backup action for volume %s of pod %s due to volume policy match\",\n\t\t\t\t\tvolume.Name, pod.Namespace+\"/\"+pod.Name)\n\t\t\t\treturn true, nil\n\t\t\t} else {\n\t\t\t\tv.logger.Infof(\"Skip fs-backup action for volume %s for pod %s because the action type is %s\",\n\t\t\t\t\tvolume.Name, pod.Namespace+\"/\"+pod.Name, action.Type)\n\t\t\t\treturn false, nil\n\t\t\t}\n\t\t}\n\n\t\t// If no policy matched and PV was not found, return the original error\n\t\tif pvNotFoundErr != nil {\n\t\t\tv.logger.WithError(pvNotFoundErr).Errorf(\"fail to get PV for PVC %s\", pvc.Namespace+\"/\"+pvc.Name)\n\t\t\treturn false, pvNotFoundErr\n\t\t}\n\t}\n\n\tif v.shouldPerformFSBackupLegacy(volume, pod) {\n\t\tv.logger.Infof(\"Perform fs-backup action for volume %s of pod %s due to opt-in/out way\",\n\t\t\tvolume.Name, pod.Namespace+\"/\"+pod.Name)\n\t\treturn true, nil\n\t} else {\n\t\tv.logger.Infof(\"Skip fs-backup action for volume %s of pod %s due to opt-in/out way\",\n\t\t\tvolume.Name, pod.Namespace+\"/\"+pod.Name)\n\t\treturn false, nil\n\t}\n}\n\nfunc (v volumeHelperImpl) shouldPerformFSBackupLegacy(\n\tvolume corev1api.Volume,\n\tpod corev1api.Pod,\n) bool {\n\t// Check volume in opt-in way\n\tif !v.defaultVolumesToFSBackup {\n\t\toptInVolumeNames := podvolumeutil.GetVolumesToBackup(&pod)\n\t\tfor _, volumeName := range optInVolumeNames {\n\t\t\tif volume.Name == volumeName {\n\t\t\t\treturn true\n\t\t\t}\n\t\t}\n\n\t\treturn false\n\t} else {\n\t\t// Check volume in opt-out way\n\t\toptOutVolumeNames := podvolumeutil.GetVolumesToExclude(&pod)\n\t\tfor _, volumeName := range optOutVolumeNames {\n\t\t\tif volume.Name == volumeName {\n\t\t\t\treturn false\n\t\t\t}\n\t\t}\n\n\t\treturn true\n\t}\n}\n\nfunc (v *volumeHelperImpl) shouldIncludeVolumeInBackup(vol corev1api.Volume) bool {\n\tincludeVolumeInBackup := true\n\t// cannot backup hostpath volumes as they are not mounted into /var/lib/kubelet/pods\n\t// and therefore not accessible to the node agent daemon set.\n\tif vol.HostPath != nil {\n\t\tincludeVolumeInBackup = false\n\t}\n\t// don't backup volumes mounting secrets. Secrets will be backed up separately.\n\tif vol.Secret != nil {\n\t\tincludeVolumeInBackup = false\n\t}\n\t// don't backup volumes mounting ConfigMaps. ConfigMaps will be backed up separately.\n\tif vol.ConfigMap != nil {\n\t\tincludeVolumeInBackup = false\n\t}\n\t// don't backup volumes mounted as projected volumes, all data in those come from kube state.\n\tif vol.Projected != nil {\n\t\tincludeVolumeInBackup = false\n\t}\n\t// don't backup DownwardAPI volumes, all data in those come from kube state.\n\tif vol.DownwardAPI != nil {\n\t\tincludeVolumeInBackup = false\n\t}\n\tif vol.PersistentVolumeClaim != nil && v.backupExcludePVC {\n\t\tincludeVolumeInBackup = false\n\t}\n\t// don't include volumes that mount the default service account token.\n\tif strings.HasPrefix(vol.Name, \"default-token\") {\n\t\tincludeVolumeInBackup = false\n\t}\n\treturn includeVolumeInBackup\n}\n\nfunc (v *volumeHelperImpl) getVolumeFromResource(resource any) (*corev1api.PersistentVolume, *corev1api.Volume, error) {\n\tif pv, ok := resource.(*corev1api.PersistentVolume); ok {\n\t\treturn pv, nil, nil\n\t} else if podVol, ok := resource.(*corev1api.Volume); ok {\n\t\treturn nil, podVol, nil\n\t}\n\treturn nil, nil, fmt.Errorf(\"resource is not a PersistentVolume or Volume\")\n}\n" }, { "path": "internal/volumehelper/volume_policy_helper_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\thttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage volumehelper\n\nimport (\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/utils/ptr\"\n\n\t\"github.com/vmware-tanzu/velero/internal/resourcepolicies\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\tpodvolumeutil \"github.com/vmware-tanzu/velero/pkg/util/podvolume\"\n)\n\nfunc TestVolumeHelperImpl_ShouldPerformSnapshot(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tinputObj runtime.Object\n\t\tgroupResource schema.GroupResource\n\t\tpod *corev1api.Pod\n\t\tresourcePolicies *resourcepolicies.ResourcePolicies\n\t\tsnapshotVolumesFlag *bool\n\t\tdefaultVolumesToFSBackup bool\n\t\tshouldSnapshot bool\n\t\texpectedErr bool\n\t}{\n\t\t{\n\t\t\tname: \"VolumePolicy match, returns true and no error\",\n\t\t\tinputObj: builder.ForPersistentVolume(\"example-pv\").StorageClass(\"gp2-csi\").ClaimRef(\"ns\", \"pvc-1\").Result(),\n\t\t\tgroupResource: kuberesource.PersistentVolumes,\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2-csi\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Snapshot,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsnapshotVolumesFlag: ptr.To(true),\n\t\t\tshouldSnapshot: true,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"VolumePolicy match, snapshotVolumes is false, return true and no error\",\n\t\t\tinputObj: builder.ForPersistentVolume(\"example-pv\").StorageClass(\"gp2-csi\").ClaimRef(\"ns\", \"pvc-1\").Result(),\n\t\t\tgroupResource: kuberesource.PersistentVolumes,\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2-csi\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Snapshot,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsnapshotVolumesFlag: ptr.To(false),\n\t\t\tshouldSnapshot: true,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"VolumePolicy match but action is unexpected, return false and no error\",\n\t\t\tinputObj: builder.ForPersistentVolume(\"example-pv\").StorageClass(\"gp2-csi\").ClaimRef(\"ns\", \"pvc-1\").Result(),\n\t\t\tgroupResource: kuberesource.PersistentVolumes,\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2-csi\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.FSBackup,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsnapshotVolumesFlag: ptr.To(true),\n\t\t\tshouldSnapshot: false,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"VolumePolicy not match, not selected by fs-backup as opt-out way, snapshotVolumes is true, returns true and no error\",\n\t\t\tinputObj: builder.ForPersistentVolume(\"example-pv\").StorageClass(\"gp3-csi\").ClaimRef(\"ns\", \"pvc-1\").Result(),\n\t\t\tgroupResource: kuberesource.PersistentVolumes,\n\t\t\tpod: builder.ForPod(\"ns\", \"pod-1\").Result(),\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2-csi\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Snapshot,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsnapshotVolumesFlag: ptr.To(true),\n\t\t\tshouldSnapshot: true,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"VolumePolicy not match, selected by fs-backup as opt-out way, snapshotVolumes is true, returns false and no error\",\n\t\t\tinputObj: builder.ForPersistentVolume(\"example-pv\").StorageClass(\"gp3-csi\").ClaimRef(\"ns\", \"pvc-1\").Result(),\n\t\t\tgroupResource: kuberesource.PersistentVolumes,\n\t\t\tpod: builder.ForPod(\"ns\", \"pod-1\").Volumes(\n\t\t\t\t&corev1api.Volume{\n\t\t\t\t\tName: \"volume\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\tClaimName: \"pvc-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t).Result(),\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2-csi\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Snapshot,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsnapshotVolumesFlag: ptr.To(true),\n\t\t\tdefaultVolumesToFSBackup: true,\n\t\t\tshouldSnapshot: false,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"VolumePolicy not match, selected by fs-backup as opt-out way, snapshotVolumes is true, returns false and no error\",\n\t\t\tinputObj: builder.ForPersistentVolume(\"example-pv\").StorageClass(\"gp3-csi\").ClaimRef(\"ns\", \"pvc-1\").Result(),\n\t\t\tgroupResource: kuberesource.PersistentVolumes,\n\t\t\tpod: builder.ForPod(\"ns\", \"pod-1\").\n\t\t\t\tObjectMeta(builder.WithAnnotations(velerov1api.VolumesToExcludeAnnotation, \"volume\")).\n\t\t\t\tVolumes(\n\t\t\t\t\t&corev1api.Volume{\n\t\t\t\t\t\tName: \"volume\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc-1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t).Result(),\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2-csi\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Snapshot,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsnapshotVolumesFlag: ptr.To(true),\n\t\t\tdefaultVolumesToFSBackup: true,\n\t\t\tshouldSnapshot: true,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"VolumePolicy not match, not selected by fs-backup as opt-in way, snapshotVolumes is true, returns false and no error\",\n\t\t\tinputObj: builder.ForPersistentVolume(\"example-pv\").StorageClass(\"gp3-csi\").ClaimRef(\"ns\", \"pvc-1\").Result(),\n\t\t\tgroupResource: kuberesource.PersistentVolumes,\n\t\t\tpod: builder.ForPod(\"ns\", \"pod-1\").\n\t\t\t\tObjectMeta(builder.WithAnnotations(velerov1api.VolumesToBackupAnnotation, \"volume\")).\n\t\t\t\tVolumes(\n\t\t\t\t\t&corev1api.Volume{\n\t\t\t\t\t\tName: \"volume\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc-1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t).Result(),\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2-csi\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Snapshot,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsnapshotVolumesFlag: ptr.To(true),\n\t\t\tdefaultVolumesToFSBackup: false,\n\t\t\tshouldSnapshot: false,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"VolumePolicy not match, not selected by fs-backup as opt-in way, snapshotVolumes is true, returns true and no error\",\n\t\t\tinputObj: builder.ForPersistentVolume(\"example-pv\").StorageClass(\"gp3-csi\").ClaimRef(\"ns\", \"pvc-1\").Result(),\n\t\t\tgroupResource: kuberesource.PersistentVolumes,\n\t\t\tpod: builder.ForPod(\"ns\", \"pod-1\").\n\t\t\t\tVolumes(\n\t\t\t\t\t&corev1api.Volume{\n\t\t\t\t\t\tName: \"volume\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc-1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t).Result(),\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2-csi\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Snapshot,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsnapshotVolumesFlag: ptr.To(true),\n\t\t\tdefaultVolumesToFSBackup: false,\n\t\t\tshouldSnapshot: true,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"No VolumePolicy, not selected by fs-backup, snapshotVolumes is true, returns true and no error\",\n\t\t\tinputObj: builder.ForPersistentVolume(\"example-pv\").StorageClass(\"gp3-csi\").ClaimRef(\"ns\", \"pvc-1\").Result(),\n\t\t\tgroupResource: kuberesource.PersistentVolumes,\n\t\t\tresourcePolicies: nil,\n\t\t\tsnapshotVolumesFlag: ptr.To(true),\n\t\t\tshouldSnapshot: true,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"No VolumePolicy, not selected by fs-backup, snapshotVolumes is false, returns false and no error\",\n\t\t\tinputObj: builder.ForPersistentVolume(\"example-pv\").StorageClass(\"gp3-csi\").ClaimRef(\"ns\", \"pvc-1\").Result(),\n\t\t\tgroupResource: kuberesource.PersistentVolumes,\n\t\t\tresourcePolicies: nil,\n\t\t\tsnapshotVolumesFlag: ptr.To(false),\n\t\t\tshouldSnapshot: false,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"PVC not having PV, return false and error when no matching policy\",\n\t\t\tinputObj: builder.ForPersistentVolumeClaim(\"default\", \"example-pvc\").StorageClass(\"gp2-csi\").Result(),\n\t\t\tgroupResource: kuberesource.PersistentVolumeClaims,\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t},\n\t\t\tsnapshotVolumesFlag: ptr.To(true),\n\t\t\tshouldSnapshot: false,\n\t\t\texpectedErr: true,\n\t\t},\n\t}\n\n\tobjs := []runtime.Object{\n\t\t&corev1api.PersistentVolumeClaim{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: \"ns\",\n\t\t\t\tName: \"pvc-1\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, objs...)\n\t\t\tif tc.pod != nil {\n\t\t\t\tfakeClient.Create(t.Context(), tc.pod)\n\t\t\t}\n\n\t\t\tvar p *resourcepolicies.Policies\n\t\t\tif tc.resourcePolicies != nil {\n\t\t\t\tp = &resourcepolicies.Policies{}\n\t\t\t\terr := p.BuildPolicy(tc.resourcePolicies)\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Fatalf(\"failed to build policy with error %v\", err)\n\t\t\t\t}\n\t\t\t}\n\t\t\tvh := NewVolumeHelperImpl(\n\t\t\t\tp,\n\t\t\t\ttc.snapshotVolumesFlag,\n\t\t\t\tlogrus.StandardLogger(),\n\t\t\t\tfakeClient,\n\t\t\t\ttc.defaultVolumesToFSBackup,\n\t\t\t\tfalse,\n\t\t\t)\n\n\t\t\tobj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.inputObj)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tactualShouldSnapshot, actualError := vh.ShouldPerformSnapshot(&unstructured.Unstructured{Object: obj}, tc.groupResource)\n\t\t\tif tc.expectedErr {\n\t\t\t\trequire.Error(t, actualError, \"Want error; Got nil error\")\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\trequire.Equalf(t, tc.shouldSnapshot, actualShouldSnapshot, \"Want shouldSnapshot as %t; Got shouldSnapshot as %t\", tc.shouldSnapshot, actualShouldSnapshot)\n\t\t})\n\t}\n}\n\nfunc TestVolumeHelperImpl_ShouldIncludeVolumeInBackup(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tvol corev1api.Volume\n\t\tbackupExcludePVC bool\n\t\tshouldInclude bool\n\t}{\n\t\t{\n\t\t\tname: \"volume has host path so do not include\",\n\t\t\tvol: corev1api.Volume{\n\t\t\t\tName: \"sample-volume\",\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tHostPath: &corev1api.HostPathVolumeSource{\n\t\t\t\t\t\tPath: \"some-path\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tbackupExcludePVC: false,\n\t\t\tshouldInclude: false,\n\t\t},\n\t\t{\n\t\t\tname: \"volume has secret mounted so do not include\",\n\t\t\tvol: corev1api.Volume{\n\t\t\t\tName: \"sample-volume\",\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tSecret: &corev1api.SecretVolumeSource{\n\t\t\t\t\t\tSecretName: \"sample-secret\",\n\t\t\t\t\t\tItems: []corev1api.KeyToPath{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"username\",\n\t\t\t\t\t\t\t\tPath: \"my-username\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tbackupExcludePVC: false,\n\t\t\tshouldInclude: false,\n\t\t},\n\t\t{\n\t\t\tname: \"volume has configmap so do not include\",\n\t\t\tvol: corev1api.Volume{\n\t\t\t\tName: \"sample-volume\",\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tConfigMap: &corev1api.ConfigMapVolumeSource{\n\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\tName: \"sample-cm\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tbackupExcludePVC: false,\n\t\t\tshouldInclude: false,\n\t\t},\n\t\t{\n\t\t\tname: \"volume is mounted as project volume so do not include\",\n\t\t\tvol: corev1api.Volume{\n\t\t\t\tName: \"sample-volume\",\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tProjected: &corev1api.ProjectedVolumeSource{\n\t\t\t\t\t\tSources: []corev1api.VolumeProjection{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tbackupExcludePVC: false,\n\t\t\tshouldInclude: false,\n\t\t},\n\t\t{\n\t\t\tname: \"volume has downwardAPI so do not include\",\n\t\t\tvol: corev1api.Volume{\n\t\t\t\tName: \"sample-volume\",\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tDownwardAPI: &corev1api.DownwardAPIVolumeSource{\n\t\t\t\t\t\tItems: []corev1api.DownwardAPIVolumeFile{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tPath: \"labels\",\n\t\t\t\t\t\t\t\tFieldRef: &corev1api.ObjectFieldSelector{\n\t\t\t\t\t\t\t\t\tFieldPath: \"metadata.labels\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tbackupExcludePVC: false,\n\t\t\tshouldInclude: false,\n\t\t},\n\t\t{\n\t\t\tname: \"volume has pvc and backupExcludePVC is true so do not include\",\n\t\t\tvol: corev1api.Volume{\n\t\t\t\tName: \"sample-volume\",\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\tClaimName: \"sample-pvc\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tbackupExcludePVC: true,\n\t\t\tshouldInclude: false,\n\t\t},\n\t\t{\n\t\t\tname: \"volume name has prefix default-token so do not include\",\n\t\t\tvol: corev1api.Volume{\n\t\t\t\tName: \"default-token-vol-name\",\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\tClaimName: \"sample-pvc\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tbackupExcludePVC: false,\n\t\t\tshouldInclude: false,\n\t\t},\n\t}\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tresourcePolicies := resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2-csi\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Snapshot,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t}\n\t\t\tpolicies := resourcePolicies\n\t\t\tp := &resourcepolicies.Policies{}\n\t\t\terr := p.BuildPolicy(&policies)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"failed to build policy with error %v\", err)\n\t\t\t}\n\t\t\tvh := &volumeHelperImpl{\n\t\t\t\tvolumePolicy: p,\n\t\t\t\tsnapshotVolumes: ptr.To(true),\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t\tbackupExcludePVC: tc.backupExcludePVC,\n\t\t\t}\n\t\t\tactualShouldInclude := vh.shouldIncludeVolumeInBackup(tc.vol)\n\t\t\tassert.Equalf(t, actualShouldInclude, tc.shouldInclude, \"Want shouldInclude as %v; Got actualShouldInclude as %v\", tc.shouldInclude, actualShouldInclude)\n\t\t})\n\t}\n}\n\nfunc TestVolumeHelperImpl_ShouldPerformFSBackup(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tpod *corev1api.Pod\n\t\tresources []runtime.Object\n\t\tresourcePolicies *resourcepolicies.ResourcePolicies\n\t\tsnapshotVolumesFlag *bool\n\t\tdefaultVolumesToFSBackup bool\n\t\tshouldFSBackup bool\n\t\texpectedErr bool\n\t}{\n\t\t{\n\t\t\tname: \"HostPath volume should be skipped.\",\n\t\t\tpod: builder.ForPod(\"ns\", \"pod-1\").\n\t\t\t\tVolumes(\n\t\t\t\t\t&corev1api.Volume{\n\t\t\t\t\t\tName: \"\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tHostPath: &corev1api.HostPathVolumeSource{\n\t\t\t\t\t\t\t\tPath: \"/mnt/test\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t}).Result(),\n\t\t\tshouldFSBackup: false,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"VolumePolicy match, return true and no error\",\n\t\t\tpod: builder.ForPod(\"ns\", \"pod-1\").\n\t\t\t\tVolumes(\n\t\t\t\t\t&corev1api.Volume{\n\t\t\t\t\t\tName: \"\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc-1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t}).Result(),\n\t\t\tresources: []runtime.Object{\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns\", \"pvc-1\").\n\t\t\t\t\tVolumeName(\"pv-1\").\n\t\t\t\t\tStorageClass(\"gp2-csi\").Phase(corev1api.ClaimBound).Result(),\n\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").StorageClass(\"gp2-csi\").Result(),\n\t\t\t},\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2-csi\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.FSBackup,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tshouldFSBackup: true,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Volume source is emptyDir, VolumePolicy match, return true and no error\",\n\t\t\tpod: builder.ForPod(\"ns\", \"pod-1\").\n\t\t\t\tVolumes(\n\t\t\t\t\t&corev1api.Volume{\n\t\t\t\t\t\tName: \"\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tEmptyDir: &corev1api.EmptyDirVolumeSource{},\n\t\t\t\t\t\t},\n\t\t\t\t\t}).Result(),\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"volumeTypes\": []string{\"emptyDir\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.FSBackup,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tshouldFSBackup: true,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"VolumePolicy match, action type is not fs-backup, return false and no error\",\n\t\t\tpod: builder.ForPod(\"ns\", \"pod-1\").\n\t\t\t\tVolumes(\n\t\t\t\t\t&corev1api.Volume{\n\t\t\t\t\t\tName: \"\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc-1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t}).Result(),\n\t\t\tresources: []runtime.Object{\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns\", \"pvc-1\").\n\t\t\t\t\tVolumeName(\"pv-1\").\n\t\t\t\t\tStorageClass(\"gp2-csi\").Phase(corev1api.ClaimBound).Result(),\n\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").StorageClass(\"gp2-csi\").Result(),\n\t\t\t},\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2-csi\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Snapshot,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tshouldFSBackup: false,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"VolumePolicy not match, selected by opt-in way, return true and no error\",\n\t\t\tpod: builder.ForPod(\"ns\", \"pod-1\").\n\t\t\t\tObjectMeta(builder.WithAnnotations(velerov1api.VolumesToBackupAnnotation, \"pvc-1\")).\n\t\t\t\tVolumes(\n\t\t\t\t\t&corev1api.Volume{\n\t\t\t\t\t\tName: \"pvc-1\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc-1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t}).Result(),\n\t\t\tresources: []runtime.Object{\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns\", \"pvc-1\").\n\t\t\t\t\tVolumeName(\"pv-1\").\n\t\t\t\t\tStorageClass(\"gp2-csi\").Phase(corev1api.ClaimBound).Result(),\n\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").StorageClass(\"gp2-csi\").Result(),\n\t\t\t},\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp3-csi\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.FSBackup,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tshouldFSBackup: true,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"No VolumePolicy, not selected by opt-out way, return false and no error\",\n\t\t\tpod: builder.ForPod(\"ns\", \"pod-1\").\n\t\t\t\tObjectMeta(builder.WithAnnotations(velerov1api.VolumesToExcludeAnnotation, \"pvc-1\")).\n\t\t\t\tVolumes(\n\t\t\t\t\t&corev1api.Volume{\n\t\t\t\t\t\tName: \"pvc-1\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc-1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t}).Result(),\n\t\t\tresources: []runtime.Object{\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns\", \"pvc-1\").\n\t\t\t\t\tVolumeName(\"pv-1\").\n\t\t\t\t\tStorageClass(\"gp2-csi\").Phase(corev1api.ClaimBound).Result(),\n\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").StorageClass(\"gp2-csi\").Result(),\n\t\t\t},\n\t\t\tdefaultVolumesToFSBackup: true,\n\t\t\tshouldFSBackup: false,\n\t\t\texpectedErr: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, tc.resources...)\n\t\t\tif tc.pod != nil {\n\t\t\t\tfakeClient.Create(t.Context(), tc.pod)\n\t\t\t}\n\n\t\t\tvar p *resourcepolicies.Policies\n\t\t\tif tc.resourcePolicies != nil {\n\t\t\t\tp = &resourcepolicies.Policies{}\n\t\t\t\terr := p.BuildPolicy(tc.resourcePolicies)\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Fatalf(\"failed to build policy with error %v\", err)\n\t\t\t\t}\n\t\t\t}\n\t\t\tvh := NewVolumeHelperImpl(\n\t\t\t\tp,\n\t\t\t\ttc.snapshotVolumesFlag,\n\t\t\t\tlogrus.StandardLogger(),\n\t\t\t\tfakeClient,\n\t\t\t\ttc.defaultVolumesToFSBackup,\n\t\t\t\tfalse,\n\t\t\t)\n\n\t\t\tactualShouldFSBackup, actualError := vh.ShouldPerformFSBackup(tc.pod.Spec.Volumes[0], *tc.pod)\n\t\t\tif tc.expectedErr {\n\t\t\t\trequire.Error(t, actualError, \"Want error; Got nil error\")\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\trequire.Equalf(t, tc.shouldFSBackup, actualShouldFSBackup, \"Want shouldFSBackup as %t; Got shouldFSBackup as %t\", tc.shouldFSBackup, actualShouldFSBackup)\n\t\t})\n\t}\n}\n\nfunc TestGetVolumeFromResource(t *testing.T) {\n\thelper := &volumeHelperImpl{}\n\n\tt.Run(\"PersistentVolume input\", func(t *testing.T) {\n\t\tpv := &corev1api.PersistentVolume{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"test-pv\",\n\t\t\t},\n\t\t}\n\t\toutPV, outPod, err := helper.getVolumeFromResource(pv)\n\t\trequire.NoError(t, err)\n\t\tassert.NotNil(t, outPV)\n\t\tassert.Nil(t, outPod)\n\t\tassert.Equal(t, \"test-pv\", outPV.Name)\n\t})\n\n\tt.Run(\"Volume input\", func(t *testing.T) {\n\t\tvol := &corev1api.Volume{\n\t\t\tName: \"test-volume\",\n\t\t}\n\t\toutPV, outPod, err := helper.getVolumeFromResource(vol)\n\t\trequire.NoError(t, err)\n\t\tassert.Nil(t, outPV)\n\t\tassert.NotNil(t, outPod)\n\t\tassert.Equal(t, \"test-volume\", outPod.Name)\n\t})\n\n\tt.Run(\"Invalid input\", func(t *testing.T) {\n\t\t_, _, err := helper.getVolumeFromResource(\"invalid\")\n\t\tassert.ErrorContains(t, err, \"resource is not a PersistentVolume or Volume\")\n\t})\n}\n\nfunc TestVolumeHelperImplWithCache_ShouldPerformSnapshot(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tinputObj runtime.Object\n\t\tgroupResource schema.GroupResource\n\t\tpod *corev1api.Pod\n\t\tresourcePolicies *resourcepolicies.ResourcePolicies\n\t\tsnapshotVolumesFlag *bool\n\t\tdefaultVolumesToFSBackup bool\n\t\tbuildCache bool\n\t\tshouldSnapshot bool\n\t\texpectedErr bool\n\t}{\n\t\t{\n\t\t\tname: \"VolumePolicy match with cache, returns true\",\n\t\t\tinputObj: builder.ForPersistentVolume(\"example-pv\").StorageClass(\"gp2-csi\").ClaimRef(\"ns\", \"pvc-1\").Result(),\n\t\t\tgroupResource: kuberesource.PersistentVolumes,\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2-csi\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Snapshot,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsnapshotVolumesFlag: ptr.To(true),\n\t\t\tbuildCache: true,\n\t\t\tshouldSnapshot: true,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"VolumePolicy not match, fs-backup via opt-out with cache, skips snapshot\",\n\t\t\tinputObj: builder.ForPersistentVolume(\"example-pv\").StorageClass(\"gp3-csi\").ClaimRef(\"ns\", \"pvc-1\").Result(),\n\t\t\tgroupResource: kuberesource.PersistentVolumes,\n\t\t\tpod: builder.ForPod(\"ns\", \"pod-1\").Volumes(\n\t\t\t\t&corev1api.Volume{\n\t\t\t\t\tName: \"volume\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\tClaimName: \"pvc-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t).Result(),\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2-csi\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Snapshot,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsnapshotVolumesFlag: ptr.To(true),\n\t\t\tdefaultVolumesToFSBackup: true,\n\t\t\tbuildCache: true,\n\t\t\tshouldSnapshot: false,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Cache not built, falls back to direct lookup\",\n\t\t\tinputObj: builder.ForPersistentVolume(\"example-pv\").StorageClass(\"gp2-csi\").ClaimRef(\"ns\", \"pvc-1\").Result(),\n\t\t\tgroupResource: kuberesource.PersistentVolumes,\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2-csi\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Snapshot,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsnapshotVolumesFlag: ptr.To(true),\n\t\t\tbuildCache: false,\n\t\t\tshouldSnapshot: true,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"No volume policy, defaultVolumesToFSBackup with cache, skips snapshot\",\n\t\t\tinputObj: builder.ForPersistentVolume(\"example-pv\").StorageClass(\"gp2-csi\").ClaimRef(\"ns\", \"pvc-1\").Result(),\n\t\t\tgroupResource: kuberesource.PersistentVolumes,\n\t\t\tpod: builder.ForPod(\"ns\", \"pod-1\").Volumes(\n\t\t\t\t&corev1api.Volume{\n\t\t\t\t\tName: \"volume\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\tClaimName: \"pvc-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t).Result(),\n\t\t\tresourcePolicies: nil,\n\t\t\tsnapshotVolumesFlag: ptr.To(true),\n\t\t\tdefaultVolumesToFSBackup: true,\n\t\t\tbuildCache: true,\n\t\t\tshouldSnapshot: false,\n\t\t\texpectedErr: false,\n\t\t},\n\t}\n\n\tobjs := []runtime.Object{\n\t\t&corev1api.PersistentVolumeClaim{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: \"ns\",\n\t\t\t\tName: \"pvc-1\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, objs...)\n\t\t\tif tc.pod != nil {\n\t\t\t\trequire.NoError(t, fakeClient.Create(t.Context(), tc.pod))\n\t\t\t}\n\n\t\t\tvar p *resourcepolicies.Policies\n\t\t\tif tc.resourcePolicies != nil {\n\t\t\t\tp = &resourcepolicies.Policies{}\n\t\t\t\terr := p.BuildPolicy(tc.resourcePolicies)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tvar namespaces []string\n\t\t\tif tc.buildCache {\n\t\t\t\tnamespaces = []string{\"ns\"}\n\t\t\t}\n\n\t\t\tvh, err := NewVolumeHelperImplWithNamespaces(\n\t\t\t\tp,\n\t\t\t\ttc.snapshotVolumesFlag,\n\t\t\t\tlogrus.StandardLogger(),\n\t\t\t\tfakeClient,\n\t\t\t\ttc.defaultVolumesToFSBackup,\n\t\t\t\tfalse,\n\t\t\t\tnamespaces,\n\t\t\t)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tobj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.inputObj)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tactualShouldSnapshot, actualError := vh.ShouldPerformSnapshot(&unstructured.Unstructured{Object: obj}, tc.groupResource)\n\t\t\tif tc.expectedErr {\n\t\t\t\trequire.Error(t, actualError)\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NoError(t, actualError)\n\t\t\trequire.Equalf(t, tc.shouldSnapshot, actualShouldSnapshot, \"Want shouldSnapshot as %t; Got shouldSnapshot as %t\", tc.shouldSnapshot, actualShouldSnapshot)\n\t\t})\n\t}\n}\n\nfunc TestVolumeHelperImplWithCache_ShouldPerformFSBackup(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tpod *corev1api.Pod\n\t\tresources []runtime.Object\n\t\tresourcePolicies *resourcepolicies.ResourcePolicies\n\t\tsnapshotVolumesFlag *bool\n\t\tdefaultVolumesToFSBackup bool\n\t\tbuildCache bool\n\t\tshouldFSBackup bool\n\t\texpectedErr bool\n\t}{\n\t\t{\n\t\t\tname: \"VolumePolicy match with cache, return true\",\n\t\t\tpod: builder.ForPod(\"ns\", \"pod-1\").\n\t\t\t\tVolumes(\n\t\t\t\t\t&corev1api.Volume{\n\t\t\t\t\t\tName: \"vol-1\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc-1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t}).Result(),\n\t\t\tresources: []runtime.Object{\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns\", \"pvc-1\").\n\t\t\t\t\tVolumeName(\"pv-1\").\n\t\t\t\t\tStorageClass(\"gp2-csi\").Phase(corev1api.ClaimBound).Result(),\n\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").StorageClass(\"gp2-csi\").Result(),\n\t\t\t},\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2-csi\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.FSBackup,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tbuildCache: true,\n\t\t\tshouldFSBackup: true,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"VolumePolicy match with cache, action is snapshot, return false\",\n\t\t\tpod: builder.ForPod(\"ns\", \"pod-1\").\n\t\t\t\tVolumes(\n\t\t\t\t\t&corev1api.Volume{\n\t\t\t\t\t\tName: \"vol-1\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc-1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t}).Result(),\n\t\t\tresources: []runtime.Object{\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns\", \"pvc-1\").\n\t\t\t\t\tVolumeName(\"pv-1\").\n\t\t\t\t\tStorageClass(\"gp2-csi\").Phase(corev1api.ClaimBound).Result(),\n\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").StorageClass(\"gp2-csi\").Result(),\n\t\t\t},\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2-csi\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Snapshot,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tbuildCache: true,\n\t\t\tshouldFSBackup: false,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Cache not built, falls back to direct lookup, opt-in annotation\",\n\t\t\tpod: builder.ForPod(\"ns\", \"pod-1\").\n\t\t\t\tObjectMeta(builder.WithAnnotations(velerov1api.VolumesToBackupAnnotation, \"vol-1\")).\n\t\t\t\tVolumes(\n\t\t\t\t\t&corev1api.Volume{\n\t\t\t\t\t\tName: \"vol-1\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc-1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t}).Result(),\n\t\t\tresources: []runtime.Object{\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns\", \"pvc-1\").\n\t\t\t\t\tVolumeName(\"pv-1\").\n\t\t\t\t\tStorageClass(\"gp2-csi\").Phase(corev1api.ClaimBound).Result(),\n\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").StorageClass(\"gp2-csi\").Result(),\n\t\t\t},\n\t\t\tbuildCache: false,\n\t\t\tdefaultVolumesToFSBackup: false,\n\t\t\tshouldFSBackup: true,\n\t\t\texpectedErr: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, tc.resources...)\n\t\t\tif tc.pod != nil {\n\t\t\t\trequire.NoError(t, fakeClient.Create(t.Context(), tc.pod))\n\t\t\t}\n\n\t\t\tvar p *resourcepolicies.Policies\n\t\t\tif tc.resourcePolicies != nil {\n\t\t\t\tp = &resourcepolicies.Policies{}\n\t\t\t\terr := p.BuildPolicy(tc.resourcePolicies)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tvar namespaces []string\n\t\t\tif tc.buildCache {\n\t\t\t\tnamespaces = []string{\"ns\"}\n\t\t\t}\n\n\t\t\tvh, err := NewVolumeHelperImplWithNamespaces(\n\t\t\t\tp,\n\t\t\t\ttc.snapshotVolumesFlag,\n\t\t\t\tlogrus.StandardLogger(),\n\t\t\t\tfakeClient,\n\t\t\t\ttc.defaultVolumesToFSBackup,\n\t\t\t\tfalse,\n\t\t\t\tnamespaces,\n\t\t\t)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tactualShouldFSBackup, actualError := vh.ShouldPerformFSBackup(tc.pod.Spec.Volumes[0], *tc.pod)\n\t\t\tif tc.expectedErr {\n\t\t\t\trequire.Error(t, actualError)\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NoError(t, actualError)\n\t\t\trequire.Equalf(t, tc.shouldFSBackup, actualShouldFSBackup, \"Want shouldFSBackup as %t; Got shouldFSBackup as %t\", tc.shouldFSBackup, actualShouldFSBackup)\n\t\t})\n\t}\n}\n\n// TestNewVolumeHelperImplWithCache tests the NewVolumeHelperImplWithCache constructor\n// which is used by plugins that build the cache lazily per-namespace.\nfunc TestNewVolumeHelperImplWithCache(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tbackup velerov1api.Backup\n\t\tresourcePolicyConfigMap *corev1api.ConfigMap\n\t\tpvcPodCache bool // whether to pass a cache\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"creates VolumeHelper with nil cache\",\n\t\t\tbackup: velerov1api.Backup{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-backup\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tSnapshotVolumes: ptr.To(true),\n\t\t\t\t\tDefaultVolumesToFsBackup: ptr.To(false),\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvcPodCache: false,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"creates VolumeHelper with non-nil cache\",\n\t\t\tbackup: velerov1api.Backup{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-backup\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tSnapshotVolumes: ptr.To(true),\n\t\t\t\t\tDefaultVolumesToFsBackup: ptr.To(true),\n\t\t\t\t\tSnapshotMoveData: ptr.To(true),\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvcPodCache: true,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"creates VolumeHelper with resource policies\",\n\t\t\tbackup: velerov1api.Backup{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-backup\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tSnapshotVolumes: ptr.To(true),\n\t\t\t\t\tResourcePolicy: &corev1api.TypedLocalObjectReference{\n\t\t\t\t\t\tKind: \"ConfigMap\",\n\t\t\t\t\t\tName: \"resource-policy\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tresourcePolicyConfigMap: &corev1api.ConfigMap{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"resource-policy\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t\tData: map[string]string{\n\t\t\t\t\t\"policy\": `version: v1\nvolumePolicies:\n- conditions:\n storageClass:\n - gp2-csi\n action:\n type: snapshot`,\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvcPodCache: true,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"fails when resource policy ConfigMap not found\",\n\t\t\tbackup: velerov1api.Backup{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-backup\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tResourcePolicy: &corev1api.TypedLocalObjectReference{\n\t\t\t\t\t\tKind: \"ConfigMap\",\n\t\t\t\t\t\tName: \"non-existent-policy\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvcPodCache: false,\n\t\t\texpectError: true,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar objs []runtime.Object\n\t\t\tif tc.resourcePolicyConfigMap != nil {\n\t\t\t\tobjs = append(objs, tc.resourcePolicyConfigMap)\n\t\t\t}\n\t\t\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, objs...)\n\n\t\t\tvar cache *podvolumeutil.PVCPodCache\n\t\t\tif tc.pvcPodCache {\n\t\t\t\tcache = podvolumeutil.NewPVCPodCache()\n\t\t\t}\n\n\t\t\tvh, err := NewVolumeHelperImplWithCache(\n\t\t\t\ttc.backup,\n\t\t\t\tfakeClient,\n\t\t\t\tlogrus.StandardLogger(),\n\t\t\t\tcache,\n\t\t\t)\n\n\t\t\tif tc.expectError {\n\t\t\t\trequire.Error(t, err)\n\t\t\t\trequire.Nil(t, vh)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\trequire.NotNil(t, vh)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestNewVolumeHelperImplWithCache_UsesCache verifies that the VolumeHelper created\n// via NewVolumeHelperImplWithCache actually uses the provided cache for lookups.\nfunc TestNewVolumeHelperImplWithCache_UsesCache(t *testing.T) {\n\t// Create a pod that uses a PVC via opt-out (defaultVolumesToFsBackup=true)\n\tpod := builder.ForPod(\"ns\", \"pod-1\").Volumes(\n\t\t&corev1api.Volume{\n\t\t\tName: \"volume\",\n\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\tClaimName: \"pvc-1\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t).Result()\n\n\tpvc := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"ns\",\n\t\t\tName: \"pvc-1\",\n\t\t},\n\t}\n\n\tpv := builder.ForPersistentVolume(\"example-pv\").StorageClass(\"gp2-csi\").ClaimRef(\"ns\", \"pvc-1\").Result()\n\n\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, pvc, pv, pod)\n\n\t// Build cache for the namespace\n\tcache := podvolumeutil.NewPVCPodCache()\n\terr := cache.BuildCacheForNamespace(t.Context(), \"ns\", fakeClient)\n\trequire.NoError(t, err)\n\n\tbackup := velerov1api.Backup{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-backup\",\n\t\t\tNamespace: \"velero\",\n\t\t},\n\t\tSpec: velerov1api.BackupSpec{\n\t\t\tSnapshotVolumes: ptr.To(true),\n\t\t\tDefaultVolumesToFsBackup: ptr.To(true), // opt-out mode\n\t\t},\n\t}\n\n\tvh, err := NewVolumeHelperImplWithCache(backup, fakeClient, logrus.StandardLogger(), cache)\n\trequire.NoError(t, err)\n\n\t// Convert PV to unstructured\n\tobj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(pv)\n\trequire.NoError(t, err)\n\n\t// ShouldPerformSnapshot should return false because the volume is selected for fs-backup\n\t// This relies on the cache to find the pod using the PVC\n\tshouldSnapshot, err := vh.ShouldPerformSnapshot(&unstructured.Unstructured{Object: obj}, kuberesource.PersistentVolumes)\n\trequire.NoError(t, err)\n\trequire.False(t, shouldSnapshot, \"Expected snapshot to be skipped due to fs-backup selection via cache\")\n}\n\n// TestVolumeHelperImpl_ShouldPerformSnapshot_UnboundPVC tests that Pending and Lost PVCs with\n// phase-based skip policies don't cause errors when GetPVForPVC would fail.\nfunc TestVolumeHelperImpl_ShouldPerformSnapshot_UnboundPVC(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tinputPVC *corev1api.PersistentVolumeClaim\n\t\tresourcePolicies *resourcepolicies.ResourcePolicies\n\t\tshouldSnapshot bool\n\t\texpectedErr bool\n\t}{\n\t\t{\n\t\t\tname: \"Pending PVC with phase-based skip policy should not error and return false\",\n\t\t\tinputPVC: builder.ForPersistentVolumeClaim(\"ns\", \"pvc-pending\").\n\t\t\t\tStorageClass(\"non-existent-class\").\n\t\t\t\tPhase(corev1api.ClaimPending).\n\t\t\t\tResult(),\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"pvcPhase\": []string{\"Pending\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Skip,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tshouldSnapshot: false,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Pending PVC without matching skip policy should error (no PV)\",\n\t\t\tinputPVC: builder.ForPersistentVolumeClaim(\"ns\", \"pvc-pending-no-policy\").\n\t\t\t\tStorageClass(\"non-existent-class\").\n\t\t\t\tPhase(corev1api.ClaimPending).\n\t\t\t\tResult(),\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2-csi\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Skip,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tshouldSnapshot: false,\n\t\t\texpectedErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Lost PVC with phase-based skip policy should not error and return false\",\n\t\t\tinputPVC: builder.ForPersistentVolumeClaim(\"ns\", \"pvc-lost\").\n\t\t\t\tStorageClass(\"some-class\").\n\t\t\t\tPhase(corev1api.ClaimLost).\n\t\t\t\tResult(),\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"pvcPhase\": []string{\"Lost\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Skip,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tshouldSnapshot: false,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Lost PVC with policy for Pending and Lost should not error and return false\",\n\t\t\tinputPVC: builder.ForPersistentVolumeClaim(\"ns\", \"pvc-lost\").\n\t\t\t\tStorageClass(\"some-class\").\n\t\t\t\tPhase(corev1api.ClaimLost).\n\t\t\t\tResult(),\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"pvcPhase\": []string{\"Pending\", \"Lost\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Skip,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tshouldSnapshot: false,\n\t\t\texpectedErr: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\t\t\tvar p *resourcepolicies.Policies\n\t\t\tif tc.resourcePolicies != nil {\n\t\t\t\tp = &resourcepolicies.Policies{}\n\t\t\t\terr := p.BuildPolicy(tc.resourcePolicies)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tvh := NewVolumeHelperImpl(\n\t\t\t\tp,\n\t\t\t\tptr.To(true),\n\t\t\t\tlogrus.StandardLogger(),\n\t\t\t\tfakeClient,\n\t\t\t\tfalse,\n\t\t\t\tfalse,\n\t\t\t)\n\n\t\t\tobj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.inputPVC)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tactualShouldSnapshot, actualError := vh.ShouldPerformSnapshot(&unstructured.Unstructured{Object: obj}, kuberesource.PersistentVolumeClaims)\n\t\t\tif tc.expectedErr {\n\t\t\t\trequire.Error(t, actualError, \"Want error; Got nil error\")\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\trequire.NoError(t, actualError)\n\t\t\trequire.Equalf(t, tc.shouldSnapshot, actualShouldSnapshot, \"Want shouldSnapshot as %t; Got shouldSnapshot as %t\", tc.shouldSnapshot, actualShouldSnapshot)\n\t\t})\n\t}\n}\n\n// TestVolumeHelperImpl_ShouldPerformFSBackup_UnboundPVC tests that Pending and Lost PVCs with\n// phase-based skip policies don't cause errors when GetPVForPVC would fail.\nfunc TestVolumeHelperImpl_ShouldPerformFSBackup_UnboundPVC(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tpod *corev1api.Pod\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\tresourcePolicies *resourcepolicies.ResourcePolicies\n\t\tshouldFSBackup bool\n\t\texpectedErr bool\n\t}{\n\t\t{\n\t\t\tname: \"Pending PVC with phase-based skip policy should not error and return false\",\n\t\t\tpod: builder.ForPod(\"ns\", \"pod-1\").\n\t\t\t\tVolumes(\n\t\t\t\t\t&corev1api.Volume{\n\t\t\t\t\t\tName: \"vol-pending\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc-pending\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t}).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns\", \"pvc-pending\").\n\t\t\t\tStorageClass(\"non-existent-class\").\n\t\t\t\tPhase(corev1api.ClaimPending).\n\t\t\t\tResult(),\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"pvcPhase\": []string{\"Pending\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Skip,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tshouldFSBackup: false,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Pending PVC without matching skip policy should error (no PV)\",\n\t\t\tpod: builder.ForPod(\"ns\", \"pod-1\").\n\t\t\t\tVolumes(\n\t\t\t\t\t&corev1api.Volume{\n\t\t\t\t\t\tName: \"vol-pending\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc-pending-no-policy\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t}).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns\", \"pvc-pending-no-policy\").\n\t\t\t\tStorageClass(\"non-existent-class\").\n\t\t\t\tPhase(corev1api.ClaimPending).\n\t\t\t\tResult(),\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2-csi\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Skip,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tshouldFSBackup: false,\n\t\t\texpectedErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Lost PVC with phase-based skip policy should not error and return false\",\n\t\t\tpod: builder.ForPod(\"ns\", \"pod-1\").\n\t\t\t\tVolumes(\n\t\t\t\t\t&corev1api.Volume{\n\t\t\t\t\t\tName: \"vol-lost\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc-lost\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t}).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns\", \"pvc-lost\").\n\t\t\t\tStorageClass(\"some-class\").\n\t\t\t\tPhase(corev1api.ClaimLost).\n\t\t\t\tResult(),\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"pvcPhase\": []string{\"Lost\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Skip,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tshouldFSBackup: false,\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Lost PVC with policy for Pending and Lost should not error and return false\",\n\t\t\tpod: builder.ForPod(\"ns\", \"pod-1\").\n\t\t\t\tVolumes(\n\t\t\t\t\t&corev1api.Volume{\n\t\t\t\t\t\tName: \"vol-lost\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc-lost\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t}).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns\", \"pvc-lost\").\n\t\t\t\tStorageClass(\"some-class\").\n\t\t\t\tPhase(corev1api.ClaimLost).\n\t\t\t\tResult(),\n\t\t\tresourcePolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"pvcPhase\": []string{\"Pending\", \"Lost\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\t\t\tType: resourcepolicies.Skip,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tshouldFSBackup: false,\n\t\t\texpectedErr: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, tc.pvc)\n\t\t\trequire.NoError(t, fakeClient.Create(t.Context(), tc.pod))\n\n\t\t\tvar p *resourcepolicies.Policies\n\t\t\tif tc.resourcePolicies != nil {\n\t\t\t\tp = &resourcepolicies.Policies{}\n\t\t\t\terr := p.BuildPolicy(tc.resourcePolicies)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tvh := NewVolumeHelperImpl(\n\t\t\t\tp,\n\t\t\t\tptr.To(true),\n\t\t\t\tlogrus.StandardLogger(),\n\t\t\t\tfakeClient,\n\t\t\t\tfalse,\n\t\t\t\tfalse,\n\t\t\t)\n\n\t\t\tactualShouldFSBackup, actualError := vh.ShouldPerformFSBackup(tc.pod.Spec.Volumes[0], *tc.pod)\n\t\t\tif tc.expectedErr {\n\t\t\t\trequire.Error(t, actualError, \"Want error; Got nil error\")\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\trequire.NoError(t, actualError)\n\t\t\trequire.Equalf(t, tc.shouldFSBackup, actualShouldFSBackup, \"Want shouldFSBackup as %t; Got shouldFSBackup as %t\", tc.shouldFSBackup, actualShouldFSBackup)\n\t\t})\n\t}\n}\n" }, { "path": "netlify.toml", "content": "[build]\n base = \"site/\"\n command = \"hugo --gc --minify\"\n publish = \"site/public\"\n\n[context.production.environment]\n HUGO_VERSION = \"0.73.0\"\n\n[context.deploy-preview.environment]\n HUGO_VERSION = \"0.73.0\"\n" }, { "path": "pkg/apis/velero/shared/data_move_operation_progress.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage shared\n\n// DataMoveOperationProgress represents the progress of a\n// data movement operation\n\n// +k8s:deepcopy-gen=true\ntype DataMoveOperationProgress struct {\n\t// +optional\n\tTotalBytes int64 `json:\"totalBytes,omitempty\"`\n\n\t// +optional\n\tBytesDone int64 `json:\"bytesDone,omitempty\"`\n}\n" }, { "path": "pkg/apis/velero/v1/backup_repository_types.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// BackupRepositorySpec is the specification for a BackupRepository.\ntype BackupRepositorySpec struct {\n\t// VolumeNamespace is the namespace this backup repository contains\n\t// pod volume backups for.\n\tVolumeNamespace string `json:\"volumeNamespace\"`\n\n\t// BackupStorageLocation is the name of the BackupStorageLocation\n\t// that should contain this repository.\n\tBackupStorageLocation string `json:\"backupStorageLocation\"`\n\n\t// RepositoryType indicates the type of the backend repository\n\t// +kubebuilder:validation:Enum=kopia;restic;\"\"\n\t// +optional\n\tRepositoryType string `json:\"repositoryType\"`\n\n\t// ResticIdentifier is the full restic-compatible string for identifying\n\t// this repository. This field is only used when RepositoryType is \"restic\".\n\t// +optional\n\tResticIdentifier string `json:\"resticIdentifier,omitempty\"`\n\n\t// MaintenanceFrequency is how often maintenance should be run.\n\tMaintenanceFrequency metav1.Duration `json:\"maintenanceFrequency\"`\n\n\t// RepositoryConfig is for repository-specific configuration fields.\n\t// +optional\n\t// +nullable\n\tRepositoryConfig map[string]string `json:\"repositoryConfig,omitempty\"`\n}\n\n// BackupRepositoryPhase represents the lifecycle phase of a BackupRepository.\n// +kubebuilder:validation:Enum=New;Ready;NotReady\ntype BackupRepositoryPhase string\n\nconst (\n\tBackupRepositoryPhaseNew BackupRepositoryPhase = \"New\"\n\tBackupRepositoryPhaseReady BackupRepositoryPhase = \"Ready\"\n\tBackupRepositoryPhaseNotReady BackupRepositoryPhase = \"NotReady\"\n\n\tBackupRepositoryTypeRestic string = \"restic\"\n\tBackupRepositoryTypeKopia string = \"kopia\"\n)\n\n// BackupRepositoryStatus is the current status of a BackupRepository.\ntype BackupRepositoryStatus struct {\n\t// Phase is the current state of the BackupRepository.\n\t// +optional\n\tPhase BackupRepositoryPhase `json:\"phase,omitempty\"`\n\n\t// Message is a message about the current status of the BackupRepository.\n\t// +optional\n\tMessage string `json:\"message,omitempty\"`\n\n\t// LastMaintenanceTime is the last time repo maintenance succeeded.\n\t// +optional\n\t// +nullable\n\tLastMaintenanceTime *metav1.Time `json:\"lastMaintenanceTime,omitempty\"`\n\n\t// RecentMaintenance is status of the recent repo maintenance.\n\t// +optional\n\tRecentMaintenance []BackupRepositoryMaintenanceStatus `json:\"recentMaintenance,omitempty\"`\n}\n\n// BackupRepositoryMaintenanceResult represents the result of a repo maintenance.\n// +kubebuilder:validation:Enum=Succeeded;Failed\ntype BackupRepositoryMaintenanceResult string\n\nconst (\n\tBackupRepositoryMaintenanceSucceeded BackupRepositoryMaintenanceResult = \"Succeeded\"\n\tBackupRepositoryMaintenanceFailed BackupRepositoryMaintenanceResult = \"Failed\"\n)\n\ntype BackupRepositoryMaintenanceStatus struct {\n\t// Result is the result of the repo maintenance.\n\t// +optional\n\tResult BackupRepositoryMaintenanceResult `json:\"result,omitempty\"`\n\n\t// StartTimestamp is the start time of the repo maintenance.\n\t// +optional\n\t// +nullable\n\tStartTimestamp *metav1.Time `json:\"startTimestamp,omitempty\"`\n\n\t// CompleteTimestamp is the completion time of the repo maintenance.\n\t// +optional\n\t// +nullable\n\tCompleteTimestamp *metav1.Time `json:\"completeTimestamp,omitempty\"`\n\n\t// Message is a message about the current status of the repo maintenance.\n\t// +optional\n\tMessage string `json:\"message,omitempty\"`\n}\n\n// TODO(2.0) After converting all resources to use the runtime-controller client,\n// the genclient and k8s:deepcopy markers will no longer be needed and should be removed.\n// +genclient\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:root=true\n// +kubebuilder:object:generate=true\n// +kubebuilder:storageversion\n// +kubebuilder:printcolumn:name=\"Age\",type=\"date\",JSONPath=\".metadata.creationTimestamp\"\n// +kubebuilder:printcolumn:name=\"Repository Type\",type=\"string\",JSONPath=\".spec.repositoryType\"\n//\n\ntype BackupRepository struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ObjectMeta `json:\"metadata,omitempty\"`\n\n\t// +optional\n\tSpec BackupRepositorySpec `json:\"spec,omitempty\"`\n\n\t// +optional\n\tStatus BackupRepositoryStatus `json:\"status,omitempty\"`\n}\n\n// TODO(2.0) After converting all resources to use the runtime-controller client,\n// the k8s:deepcopy marker will no longer be needed and should be removed.\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:root=true\n// +kubebuilder:rbac:groups=velero.io,resources=backuprepositories,verbs=get;list;watch;create;update;patch;delete\n// +kubebuilder:rbac:groups=velero.io,resources=backuprepositories/status,verbs=get;update;patch\n\n// BackupRepositoryList is a list of BackupRepositories.\ntype BackupRepositoryList struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ListMeta `json:\"metadata,omitempty\"`\n\n\tItems []BackupRepository `json:\"items\"`\n}\n" }, { "path": "pkg/apis/velero/v1/backup_types.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\ntype Metadata struct {\n\tLabels map[string]string `json:\"labels,omitempty\"`\n}\n\n// BackupSpec defines the specification for a Velero backup.\ntype BackupSpec struct {\n\t// +optional\n\tMetadata `json:\"metadata,omitempty\"`\n\t// IncludedNamespaces is a slice of namespace names to include objects\n\t// from. If empty, all namespaces are included.\n\t// +optional\n\t// +nullable\n\tIncludedNamespaces []string `json:\"includedNamespaces,omitempty\"`\n\n\t// ExcludedNamespaces contains a list of namespaces that are not\n\t// included in the backup.\n\t// +optional\n\t// +nullable\n\tExcludedNamespaces []string `json:\"excludedNamespaces,omitempty\"`\n\n\t// IncludedResources is a slice of resource names to include\n\t// in the backup. If empty, all resources are included.\n\t// +optional\n\t// +nullable\n\tIncludedResources []string `json:\"includedResources,omitempty\"`\n\n\t// ExcludedResources is a slice of resource names that are not\n\t// included in the backup.\n\t// +optional\n\t// +nullable\n\tExcludedResources []string `json:\"excludedResources,omitempty\"`\n\n\t// IncludedClusterScopedResources is a slice of cluster-scoped\n\t// resource type names to include in the backup.\n\t// If set to \"*\", all cluster-scoped resource types are included.\n\t// The default value is empty, which means only related\n\t// cluster-scoped resources are included.\n\t// +optional\n\t// +nullable\n\tIncludedClusterScopedResources []string `json:\"includedClusterScopedResources,omitempty\"`\n\n\t// ExcludedClusterScopedResources is a slice of cluster-scoped\n\t// resource type names to exclude from the backup.\n\t// If set to \"*\", all cluster-scoped resource types are excluded.\n\t// The default value is empty.\n\t// +optional\n\t// +nullable\n\tExcludedClusterScopedResources []string `json:\"excludedClusterScopedResources,omitempty\"`\n\n\t// IncludedNamespaceScopedResources is a slice of namespace-scoped\n\t// resource type names to include in the backup.\n\t// The default value is \"*\".\n\t// +optional\n\t// +nullable\n\tIncludedNamespaceScopedResources []string `json:\"includedNamespaceScopedResources,omitempty\"`\n\n\t// ExcludedNamespaceScopedResources is a slice of namespace-scoped\n\t// resource type names to exclude from the backup.\n\t// If set to \"*\", all namespace-scoped resource types are excluded.\n\t// The default value is empty.\n\t// +optional\n\t// +nullable\n\tExcludedNamespaceScopedResources []string `json:\"excludedNamespaceScopedResources,omitempty\"`\n\n\t// LabelSelector is a metav1.LabelSelector to filter with\n\t// when adding individual objects to the backup. If empty\n\t// or nil, all objects are included. Optional.\n\t// +optional\n\t// +nullable\n\tLabelSelector *metav1.LabelSelector `json:\"labelSelector,omitempty\"`\n\n\t// OrLabelSelectors is list of metav1.LabelSelector to filter with\n\t// when adding individual objects to the backup. If multiple provided\n\t// they will be joined by the OR operator. LabelSelector as well as\n\t// OrLabelSelectors cannot co-exist in backup request, only one of them\n\t// can be used.\n\t// +optional\n\t// +nullable\n\tOrLabelSelectors []*metav1.LabelSelector `json:\"orLabelSelectors,omitempty\"`\n\n\t// SnapshotVolumes specifies whether to take snapshots\n\t// of any PV's referenced in the set of objects included\n\t// in the Backup.\n\t// +optional\n\t// +nullable\n\tSnapshotVolumes *bool `json:\"snapshotVolumes,omitempty\"`\n\n\t// TTL is a time.Duration-parseable string describing how long\n\t// the Backup should be retained for.\n\t// +optional\n\tTTL metav1.Duration `json:\"ttl,omitempty\"`\n\n\t// VolumeGroupSnapshotLabelKey specifies the label key to group PVCs under a VGS.\n\t// +optional\n\tVolumeGroupSnapshotLabelKey string `json:\"volumeGroupSnapshotLabelKey,omitempty\"`\n\n\t// IncludeClusterResources specifies whether cluster-scoped resources\n\t// should be included for consideration in the backup.\n\t// +optional\n\t// +nullable\n\tIncludeClusterResources *bool `json:\"includeClusterResources,omitempty\"`\n\n\t// Hooks represent custom behaviors that should be executed at different phases of the backup.\n\t// +optional\n\tHooks BackupHooks `json:\"hooks,omitempty\"`\n\n\t// StorageLocation is a string containing the name of a BackupStorageLocation where the backup should be stored.\n\t// +optional\n\tStorageLocation string `json:\"storageLocation,omitempty\"`\n\n\t// VolumeSnapshotLocations is a list containing names of VolumeSnapshotLocations associated with this backup.\n\t// +optional\n\tVolumeSnapshotLocations []string `json:\"volumeSnapshotLocations,omitempty\"`\n\n\t// DefaultVolumesToRestic specifies whether restic should be used to take a\n\t// backup of all pod volumes by default.\n\t//\n\t// Deprecated: this field is no longer used and will be removed entirely in future. Use DefaultVolumesToFsBackup instead.\n\t// +optional\n\t// +nullable\n\tDefaultVolumesToRestic *bool `json:\"defaultVolumesToRestic,omitempty\"`\n\n\t// DefaultVolumesToFsBackup specifies whether pod volume file system backup should be used\n\t// for all volumes by default.\n\t// +optional\n\t// +nullable\n\tDefaultVolumesToFsBackup *bool `json:\"defaultVolumesToFsBackup,omitempty\"`\n\n\t// OrderedResources specifies the backup order of resources of specific Kind.\n\t// The map key is the resource name and value is a list of object names separated by commas.\n\t// Each resource name has format \"namespace/objectname\". For cluster resources, simply use \"objectname\".\n\t// +optional\n\t// +nullable\n\tOrderedResources map[string]string `json:\"orderedResources,omitempty\"`\n\n\t// CSISnapshotTimeout specifies the time used to wait for CSI VolumeSnapshot status turns to\n\t// ReadyToUse during creation, before returning error as timeout.\n\t// The default value is 10 minute.\n\t// +optional\n\tCSISnapshotTimeout metav1.Duration `json:\"csiSnapshotTimeout,omitempty\"`\n\n\t// ItemOperationTimeout specifies the time used to wait for asynchronous BackupItemAction operations\n\t// The default value is 4 hour.\n\t// +optional\n\tItemOperationTimeout metav1.Duration `json:\"itemOperationTimeout,omitempty\"`\n\t// ResourcePolicy specifies the referenced resource policies that backup should follow\n\t// +optional\n\tResourcePolicy *corev1api.TypedLocalObjectReference `json:\"resourcePolicy,omitempty\"`\n\n\t// SnapshotMoveData specifies whether snapshot data should be moved\n\t// +optional\n\t// +nullable\n\tSnapshotMoveData *bool `json:\"snapshotMoveData,omitempty\"`\n\n\t// DataMover specifies the data mover to be used by the backup.\n\t// If DataMover is \"\" or \"velero\", the built-in data mover will be used.\n\t// +optional\n\tDataMover string `json:\"datamover,omitempty\"`\n\n\t// UploaderConfig specifies the configuration for the uploader.\n\t// +optional\n\t// +nullable\n\tUploaderConfig *UploaderConfigForBackup `json:\"uploaderConfig,omitempty\"`\n}\n\n// UploaderConfigForBackup defines the configuration for the uploader when doing backup.\ntype UploaderConfigForBackup struct {\n\t// ParallelFilesUpload is the number of files parallel uploads to perform when using the uploader.\n\t// +optional\n\tParallelFilesUpload int `json:\"parallelFilesUpload,omitempty\"`\n}\n\n// BackupHooks contains custom behaviors that should be executed at different phases of the backup.\ntype BackupHooks struct {\n\t// Resources are hooks that should be executed when backing up individual instances of a resource.\n\t// +optional\n\t// +nullable\n\tResources []BackupResourceHookSpec `json:\"resources,omitempty\"`\n}\n\n// BackupResourceHookSpec defines one or more BackupResourceHooks that should be executed based on\n// the rules defined for namespaces, resources, and label selector.\ntype BackupResourceHookSpec struct {\n\t// Name is the name of this hook.\n\tName string `json:\"name\"`\n\n\t// IncludedNamespaces specifies the namespaces to which this hook spec applies. If empty, it applies\n\t// to all namespaces.\n\t// +optional\n\t// +nullable\n\tIncludedNamespaces []string `json:\"includedNamespaces,omitempty\"`\n\n\t// ExcludedNamespaces specifies the namespaces to which this hook spec does not apply.\n\t// +optional\n\t// +nullable\n\tExcludedNamespaces []string `json:\"excludedNamespaces,omitempty\"`\n\n\t// IncludedResources specifies the resources to which this hook spec applies. If empty, it applies\n\t// to all resources.\n\t// +optional\n\t// +nullable\n\tIncludedResources []string `json:\"includedResources,omitempty\"`\n\n\t// ExcludedResources specifies the resources to which this hook spec does not apply.\n\t// +optional\n\t// +nullable\n\tExcludedResources []string `json:\"excludedResources,omitempty\"`\n\n\t// LabelSelector, if specified, filters the resources to which this hook spec applies.\n\t// +optional\n\t// +nullable\n\tLabelSelector *metav1.LabelSelector `json:\"labelSelector,omitempty\"`\n\n\t// PreHooks is a list of BackupResourceHooks to execute prior to storing the item in the backup.\n\t// These are executed before any \"additional items\" from item actions are processed.\n\t// +optional\n\tPreHooks []BackupResourceHook `json:\"pre,omitempty\"`\n\n\t// PostHooks is a list of BackupResourceHooks to execute after storing the item in the backup.\n\t// These are executed after all \"additional items\" from item actions are processed.\n\t// +optional\n\tPostHooks []BackupResourceHook `json:\"post,omitempty\"`\n}\n\n// BackupResourceHook defines a hook for a resource.\ntype BackupResourceHook struct {\n\t// Exec defines an exec hook.\n\tExec *ExecHook `json:\"exec\"`\n}\n\n// ExecHook is a hook that uses the pod exec API to execute a command in a container in a pod.\ntype ExecHook struct {\n\t// Container is the container in the pod where the command should be executed. If not specified,\n\t// the pod's first container is used.\n\t// +optional\n\tContainer string `json:\"container,omitempty\"`\n\n\t// Command is the command and arguments to execute.\n\t// +kubebuilder:validation:MinItems=1\n\tCommand []string `json:\"command\"`\n\n\t// OnError specifies how Velero should behave if it encounters an error executing this hook.\n\t// +optional\n\tOnError HookErrorMode `json:\"onError,omitempty\"`\n\n\t// Timeout defines the maximum amount of time Velero should wait for the hook to complete before\n\t// considering the execution a failure.\n\t// +optional\n\tTimeout metav1.Duration `json:\"timeout,omitempty\"`\n}\n\n// HookErrorMode defines how Velero should treat an error from a hook.\n// +kubebuilder:validation:Enum=Continue;Fail\ntype HookErrorMode string\n\nconst (\n\t// HookErrorModeContinue means that an error from a hook is acceptable and the backup/restore can\n\t// proceed with the rest of hooks' execution. This backup/restore should be in `PartiallyFailed` status.\n\tHookErrorModeContinue HookErrorMode = \"Continue\"\n\n\t// HookErrorModeFail means that an error from a hook is problematic and Velero should stop executing following hooks.\n\t// This backup/restore should be in `PartiallyFailed` status.\n\tHookErrorModeFail HookErrorMode = \"Fail\"\n)\n\n// BackupPhase is a string representation of the lifecycle phase\n// of a Velero backup.\n// +kubebuilder:validation:Enum=New;Queued;ReadyToStart;FailedValidation;InProgress;WaitingForPluginOperations;WaitingForPluginOperationsPartiallyFailed;Finalizing;FinalizingPartiallyFailed;Completed;PartiallyFailed;Failed;Deleting\ntype BackupPhase string\n\nconst (\n\t// BackupPhaseNew means the backup has been created but not\n\t// yet processed by the BackupController.\n\tBackupPhaseNew BackupPhase = \"New\"\n\n\t// BackupPhaseQueued means the backup has been added to the queue and is waiting for the Queue to move it out of the queue.\n\tBackupPhaseQueued BackupPhase = \"Queued\"\n\n\t// BackupPhaseReadyToStart means the backup has been pulled from the queue and is ready to start.\n\tBackupPhaseReadyToStart BackupPhase = \"ReadyToStart\"\n\n\t// BackupPhaseFailedValidation means the backup has failed\n\t// the controller's validations and therefore will not run.\n\tBackupPhaseFailedValidation BackupPhase = \"FailedValidation\"\n\n\t// BackupPhaseInProgress means the backup is currently executing.\n\tBackupPhaseInProgress BackupPhase = \"InProgress\"\n\n\t// BackupPhaseWaitingForPluginOperations means the backup of\n\t// Kubernetes resources, creation of snapshots, and other\n\t// async plugin operations was successful and snapshot data is\n\t// currently uploading or other plugin operations are still\n\t// ongoing. The backup is not usable yet.\n\tBackupPhaseWaitingForPluginOperations BackupPhase = \"WaitingForPluginOperations\"\n\n\t// BackupPhaseWaitingForPluginOperationsPartiallyFailed means\n\t// the backup of Kubernetes resources, creation of snapshots,\n\t// and other async plugin operations partially failed (final\n\t// phase will be PartiallyFailed) and snapshot data is\n\t// currently uploading or other plugin operations are still\n\t// ongoing. The backup is not usable yet.\n\tBackupPhaseWaitingForPluginOperationsPartiallyFailed BackupPhase = \"WaitingForPluginOperationsPartiallyFailed\"\n\n\t// BackupPhaseFinalizing means the backup of\n\t// Kubernetes resources, creation of snapshots, and other\n\t// async plugin operations were successful and snapshot upload and\n\t// other plugin operations are now complete, but the Backup is awaiting\n\t// final update of resources modified during async operations.\n\t// The backup is not usable yet.\n\tBackupPhaseFinalizing BackupPhase = \"Finalizing\"\n\n\t// BackupPhaseFinalizingPartiallyFailed means the backup of\n\t// Kubernetes resources, creation of snapshots, and other\n\t// async plugin operations were successful and snapshot upload and\n\t// other plugin operations are now complete, but one or more errors\n\t// occurred during backup or async operation processing, and the\n\t// Backup is awaiting final update of resources modified during async\n\t// operations. The backup is not usable yet.\n\tBackupPhaseFinalizingPartiallyFailed BackupPhase = \"FinalizingPartiallyFailed\"\n\n\t// BackupPhaseCompleted means the backup has run successfully without\n\t// errors.\n\tBackupPhaseCompleted BackupPhase = \"Completed\"\n\n\t// BackupPhasePartiallyFailed means the backup has run to completion\n\t// but encountered 1+ errors backing up individual items.\n\tBackupPhasePartiallyFailed BackupPhase = \"PartiallyFailed\"\n\n\t// BackupPhaseFailed means the backup ran but encountered an error that\n\t// prevented it from completing successfully.\n\tBackupPhaseFailed BackupPhase = \"Failed\"\n\n\t// BackupPhaseDeleting means the backup and all its associated data are being deleted.\n\tBackupPhaseDeleting BackupPhase = \"Deleting\"\n)\n\n// BackupStatus captures the current status of a Velero backup.\ntype BackupStatus struct {\n\t// Version is the backup format major version.\n\t// Deprecated: Please see FormatVersion\n\t// +optional\n\tVersion int `json:\"version,omitempty\"`\n\n\t// FormatVersion is the backup format version, including major, minor, and patch version.\n\t// +optional\n\tFormatVersion string `json:\"formatVersion,omitempty\"`\n\n\t// Expiration is when this Backup is eligible for garbage-collection.\n\t// +optional\n\t// +nullable\n\tExpiration *metav1.Time `json:\"expiration,omitempty\"`\n\n\t// Phase is the current state of the Backup.\n\t// +optional\n\tPhase BackupPhase `json:\"phase,omitempty\"`\n\n\t// QueuePosition is the position of the backup in the queue.\n\t// Only relevant when Phase is \"Queued\"\n\t// +optional\n\tQueuePosition int `json:\"queuePosition,omitempty\"`\n\n\t// ValidationErrors is a slice of all validation errors (if\n\t// applicable).\n\t// +optional\n\t// +nullable\n\tValidationErrors []string `json:\"validationErrors,omitempty\"`\n\n\t// StartTimestamp records the time a backup was started.\n\t// Separate from CreationTimestamp, since that value changes\n\t// on restores.\n\t// The server's time is used for StartTimestamps\n\t// +optional\n\t// +nullable\n\tStartTimestamp *metav1.Time `json:\"startTimestamp,omitempty\"`\n\n\t// CompletionTimestamp records the time a backup was completed.\n\t// Completion time is recorded even on failed backups.\n\t// Completion time is recorded before uploading the backup object.\n\t// The server's time is used for CompletionTimestamps\n\t// +optional\n\t// +nullable\n\tCompletionTimestamp *metav1.Time `json:\"completionTimestamp,omitempty\"`\n\n\t// VolumeSnapshotsAttempted is the total number of attempted\n\t// volume snapshots for this backup.\n\t// +optional\n\tVolumeSnapshotsAttempted int `json:\"volumeSnapshotsAttempted,omitempty\"`\n\n\t// VolumeSnapshotsCompleted is the total number of successfully\n\t// completed volume snapshots for this backup.\n\t// +optional\n\tVolumeSnapshotsCompleted int `json:\"volumeSnapshotsCompleted,omitempty\"`\n\n\t// FailureReason is an error that caused the entire backup to fail.\n\t// +optional\n\tFailureReason string `json:\"failureReason,omitempty\"`\n\n\t// Warnings is a count of all warning messages that were generated during\n\t// execution of the backup. The actual warnings are in the backup's log\n\t// file in object storage.\n\t// +optional\n\tWarnings int `json:\"warnings,omitempty\"`\n\n\t// Errors is a count of all error messages that were generated during\n\t// execution of the backup. The actual errors are in the backup's log\n\t// file in object storage.\n\t// +optional\n\tErrors int `json:\"errors,omitempty\"`\n\n\t// Progress contains information about the backup's execution progress. Note\n\t// that this information is best-effort only -- if Velero fails to update it\n\t// during a backup for any reason, it may be inaccurate/stale.\n\t// +optional\n\t// +nullable\n\tProgress *BackupProgress `json:\"progress,omitempty\"`\n\n\t// CSIVolumeSnapshotsAttempted is the total number of attempted\n\t// CSI VolumeSnapshots for this backup.\n\t// +optional\n\tCSIVolumeSnapshotsAttempted int `json:\"csiVolumeSnapshotsAttempted,omitempty\"`\n\n\t// CSIVolumeSnapshotsCompleted is the total number of successfully\n\t// completed CSI VolumeSnapshots for this backup.\n\t// +optional\n\tCSIVolumeSnapshotsCompleted int `json:\"csiVolumeSnapshotsCompleted,omitempty\"`\n\n\t// BackupItemOperationsAttempted is the total number of attempted\n\t// async BackupItemAction operations for this backup.\n\t// +optional\n\tBackupItemOperationsAttempted int `json:\"backupItemOperationsAttempted,omitempty\"`\n\n\t// BackupItemOperationsCompleted is the total number of successfully completed\n\t// async BackupItemAction operations for this backup.\n\t// +optional\n\tBackupItemOperationsCompleted int `json:\"backupItemOperationsCompleted,omitempty\"`\n\n\t// BackupItemOperationsFailed is the total number of async\n\t// BackupItemAction operations for this backup which ended with an error.\n\t// +optional\n\tBackupItemOperationsFailed int `json:\"backupItemOperationsFailed,omitempty\"`\n\n\t// HookStatus contains information about the status of the hooks.\n\t// +optional\n\t// +nullable\n\tHookStatus *HookStatus `json:\"hookStatus,omitempty\"`\n}\n\n// BackupProgress stores information about the progress of a Backup's execution.\ntype BackupProgress struct {\n\t// TotalItems is the total number of items to be backed up. This number may change\n\t// throughout the execution of the backup due to plugins that return additional related\n\t// items to back up, the velero.io/exclude-from-backup label, and various other\n\t// filters that happen as items are processed.\n\t// +optional\n\tTotalItems int `json:\"totalItems,omitempty\"`\n\n\t// ItemsBackedUp is the number of items that have actually been written to the\n\t// backup tarball so far.\n\t// +optional\n\tItemsBackedUp int `json:\"itemsBackedUp,omitempty\"`\n}\n\n// HookStatus stores information about the status of the hooks.\ntype HookStatus struct {\n\t// HooksAttempted is the total number of attempted hooks\n\t// Specifically, HooksAttempted represents the number of hooks that failed to execute\n\t// and the number of hooks that executed successfully.\n\t// +optional\n\tHooksAttempted int `json:\"hooksAttempted,omitempty\"`\n\n\t// HooksFailed is the total number of hooks which ended with an error\n\t// +optional\n\tHooksFailed int `json:\"hooksFailed,omitempty\"`\n}\n\n// +genclient\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:root=true\n// +kubebuilder:object:generate=true\n// +kubebuilder:storageversion\n// +kubebuilder:rbac:groups=velero.io,resources=backups,verbs=create;delete;get;list;patch;update;watch\n// +kubebuilder:rbac:groups=velero.io,resources=backups/status,verbs=get;update;patch\n\n// Backup is a Velero resource that represents the capture of Kubernetes\n// cluster state at a point in time (API objects and associated volume state).\ntype Backup struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ObjectMeta `json:\"metadata,omitempty\"`\n\n\t// +optional\n\tSpec BackupSpec `json:\"spec,omitempty\"`\n\n\t// +optional\n\tStatus BackupStatus `json:\"status,omitempty\"`\n}\n\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n\n// BackupList is a list of Backups.\ntype BackupList struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ListMeta `json:\"metadata,omitempty\"`\n\n\tItems []Backup `json:\"items\"`\n}\n" }, { "path": "pkg/apis/velero/v1/backupstoragelocation_types.go", "content": "/*\nCopyright 2017, 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\t\"errors\"\n\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n)\n\n// BackupStorageLocationSpec defines the desired state of a Velero BackupStorageLocation\ntype BackupStorageLocationSpec struct {\n\t// Provider is the provider of the backup storage.\n\tProvider string `json:\"provider\"`\n\n\t// Config is for provider-specific configuration fields.\n\t// +optional\n\tConfig map[string]string `json:\"config,omitempty\"`\n\n\t// Credential contains the credential information intended to be used with this location\n\t// +optional\n\tCredential *corev1api.SecretKeySelector `json:\"credential,omitempty\"`\n\n\tStorageType `json:\",inline\"`\n\n\t// Default indicates this location is the default backup storage location.\n\t// +optional\n\tDefault bool `json:\"default,omitempty\"`\n\n\t// AccessMode defines the permissions for the backup storage location.\n\t// +optional\n\tAccessMode BackupStorageLocationAccessMode `json:\"accessMode,omitempty\"`\n\n\t// BackupSyncPeriod defines how frequently to sync backup API objects from object storage. A value of 0 disables sync.\n\t// +optional\n\t// +nullable\n\tBackupSyncPeriod *metav1.Duration `json:\"backupSyncPeriod,omitempty\"`\n\n\t// ValidationFrequency defines how frequently to validate the corresponding object storage. A value of 0 disables validation.\n\t// +optional\n\t// +nullable\n\tValidationFrequency *metav1.Duration `json:\"validationFrequency,omitempty\"`\n}\n\n// BackupStorageLocationStatus defines the observed state of BackupStorageLocation\ntype BackupStorageLocationStatus struct {\n\t// Phase is the current state of the BackupStorageLocation.\n\t// +optional\n\tPhase BackupStorageLocationPhase `json:\"phase,omitempty\"`\n\n\t// LastSyncedTime is the last time the contents of the location were synced into\n\t// the cluster.\n\t// +optional\n\t// +nullable\n\tLastSyncedTime *metav1.Time `json:\"lastSyncedTime,omitempty\"`\n\n\t// LastValidationTime is the last time the backup store location was validated\n\t// the cluster.\n\t// +optional\n\t// +nullable\n\tLastValidationTime *metav1.Time `json:\"lastValidationTime,omitempty\"`\n\n\t// Message is a message about the backup storage location's status.\n\t// +optional\n\tMessage string `json:\"message,omitempty\"`\n\n\t// LastSyncedRevision is the value of the `metadata/revision` file in the backup\n\t// storage location the last time the BSL's contents were synced into the cluster.\n\t//\n\t// Deprecated: this field is no longer updated or used for detecting changes to\n\t// the location's contents and will be removed entirely in v2.0.\n\t// +optional\n\tLastSyncedRevision types.UID `json:\"lastSyncedRevision,omitempty\"`\n\n\t// AccessMode is an unused field.\n\t//\n\t// Deprecated: there is now an AccessMode field on the Spec and this field\n\t// will be removed entirely as of v2.0.\n\t// +optional\n\tAccessMode BackupStorageLocationAccessMode `json:\"accessMode,omitempty\"`\n}\n\n// TODO(2.0) After converting all resources to use the runtime-controller client,\n// the genclient and k8s:deepcopy markers will no longer be needed and should be removed.\n// +genclient\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:root=true\n// +kubebuilder:resource:shortName=bsl\n// +kubebuilder:object:generate=true\n// +kubebuilder:storageversion\n// +kubebuilder:printcolumn:name=\"Phase\",type=\"string\",JSONPath=\".status.phase\",description=\"Backup Storage Location status such as Available/Unavailable\"\n// +kubebuilder:printcolumn:name=\"Last Validated\",type=\"date\",JSONPath=\".status.lastValidationTime\",description=\"LastValidationTime is the last time the backup store location was validated\"\n// +kubebuilder:printcolumn:name=\"Age\",type=\"date\",JSONPath=\".metadata.creationTimestamp\"\n// +kubebuilder:printcolumn:name=\"Default\",type=\"boolean\",JSONPath=\".spec.default\",description=\"Default backup storage location\"\n\n// BackupStorageLocation is a location where Velero stores backup objects\ntype BackupStorageLocation struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\tmetav1.ObjectMeta `json:\"metadata,omitempty\"`\n\n\tSpec BackupStorageLocationSpec `json:\"spec,omitempty\"`\n\tStatus BackupStorageLocationStatus `json:\"status,omitempty\"`\n}\n\n// TODO(2.0) After converting all resources to use the runtime-controller client,\n// the k8s:deepcopy marker will no longer be needed and should be removed.\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:root=true\n// +kubebuilder:rbac:groups=velero.io,resources=backupstoragelocations,verbs=get;list;watch;create;update;patch;delete\n// +kubebuilder:rbac:groups=velero.io,resources=backupstoragelocations/status,verbs=get;update;patch\n\n// BackupStorageLocationList contains a list of BackupStorageLocation\ntype BackupStorageLocationList struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\tmetav1.ListMeta `json:\"metadata,omitempty\"`\n\tItems []BackupStorageLocation `json:\"items\"`\n}\n\n// StorageType represents the type of storage that a backup location uses.\n// ObjectStorage must be non-nil, since it is currently the only supported StorageType.\ntype StorageType struct {\n\tObjectStorage *ObjectStorageLocation `json:\"objectStorage\"`\n}\n\n// ObjectStorageLocation specifies the settings necessary to connect to a provider's object storage.\ntype ObjectStorageLocation struct {\n\t// Bucket is the bucket to use for object storage.\n\tBucket string `json:\"bucket\"`\n\n\t// Prefix is the path inside a bucket to use for Velero storage. Optional.\n\t// +optional\n\tPrefix string `json:\"prefix,omitempty\"`\n\n\t// CACert defines a CA bundle to use when verifying TLS connections to the provider.\n\t// Deprecated: Use CACertRef instead.\n\t// +optional\n\tCACert []byte `json:\"caCert,omitempty\"`\n\n\t// CACertRef is a reference to a Secret containing the CA certificate bundle to use\n\t// when verifying TLS connections to the provider. The Secret must be in the same\n\t// namespace as the BackupStorageLocation.\n\t// +optional\n\tCACertRef *corev1api.SecretKeySelector `json:\"caCertRef,omitempty\"`\n}\n\n// BackupStorageLocationPhase is the lifecycle phase of a Velero BackupStorageLocation.\n// +kubebuilder:validation:Enum=Available;Unavailable\n// +kubebuilder:default=Unavailable\ntype BackupStorageLocationPhase string\n\nconst (\n\t// BackupStorageLocationPhaseAvailable means the location is available to read and write from.\n\tBackupStorageLocationPhaseAvailable BackupStorageLocationPhase = \"Available\"\n\n\t// BackupStorageLocationPhaseUnavailable means the location is unavailable to read and write from.\n\tBackupStorageLocationPhaseUnavailable BackupStorageLocationPhase = \"Unavailable\"\n)\n\n// BackupStorageLocationAccessMode represents the permissions for a BackupStorageLocation.\n// +kubebuilder:validation:Enum=ReadOnly;ReadWrite\ntype BackupStorageLocationAccessMode string\n\nconst (\n\t// BackupStorageLocationAccessModeReadOnly represents read-only access to a BackupStorageLocation.\n\tBackupStorageLocationAccessModeReadOnly BackupStorageLocationAccessMode = \"ReadOnly\"\n\n\t// BackupStorageLocationAccessModeReadWrite represents read and write access to a BackupStorageLocation.\n\tBackupStorageLocationAccessModeReadWrite BackupStorageLocationAccessMode = \"ReadWrite\"\n)\n\n// TODO(2.0): remove the AccessMode field from BackupStorageLocationStatus.\n// TODO(2.0): remove the LastSyncedRevision field from BackupStorageLocationStatus.\n\n// Validate validates the BackupStorageLocation to ensure that only one of CACert or CACertRef is set.\nfunc (bsl *BackupStorageLocation) Validate() error {\n\tif bsl.Spec.ObjectStorage != nil &&\n\t\tbsl.Spec.ObjectStorage.CACert != nil &&\n\t\tbsl.Spec.ObjectStorage.CACertRef != nil {\n\t\treturn errors.New(\"cannot specify both caCert and caCertRef in objectStorage\")\n\t}\n\treturn nil\n}\n" }, { "path": "pkg/apis/velero/v1/backupstoragelocation_types_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\t\"testing\"\n\n\tcorev1api \"k8s.io/api/core/v1\"\n)\n\nfunc TestBackupStorageLocationValidate(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbsl *BackupStorageLocation\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"valid - neither CACert nor CACertRef set\",\n\t\t\tbsl: &BackupStorageLocation{\n\t\t\t\tSpec: BackupStorageLocationSpec{\n\t\t\t\t\tStorageType: StorageType{\n\t\t\t\t\t\tObjectStorage: &ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"test-bucket\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"valid - only CACert set\",\n\t\t\tbsl: &BackupStorageLocation{\n\t\t\t\tSpec: BackupStorageLocationSpec{\n\t\t\t\t\tStorageType: StorageType{\n\t\t\t\t\t\tObjectStorage: &ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"test-bucket\",\n\t\t\t\t\t\t\tCACert: []byte(\"test-cert\"),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"valid - only CACertRef set\",\n\t\t\tbsl: &BackupStorageLocation{\n\t\t\t\tSpec: BackupStorageLocationSpec{\n\t\t\t\t\tStorageType: StorageType{\n\t\t\t\t\t\tObjectStorage: &ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"test-bucket\",\n\t\t\t\t\t\t\tCACertRef: &corev1api.SecretKeySelector{\n\t\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\t\tName: \"ca-cert-secret\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\tKey: \"ca.crt\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"invalid - both CACert and CACertRef set\",\n\t\t\tbsl: &BackupStorageLocation{\n\t\t\t\tSpec: BackupStorageLocationSpec{\n\t\t\t\t\tStorageType: StorageType{\n\t\t\t\t\t\tObjectStorage: &ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"test-bucket\",\n\t\t\t\t\t\t\tCACert: []byte(\"test-cert\"),\n\t\t\t\t\t\t\tCACertRef: &corev1api.SecretKeySelector{\n\t\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\t\tName: \"ca-cert-secret\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\tKey: \"ca.crt\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"valid - no ObjectStorage\",\n\t\t\tbsl: &BackupStorageLocation{\n\t\t\t\tSpec: BackupStorageLocationSpec{\n\t\t\t\t\tStorageType: StorageType{\n\t\t\t\t\t\tObjectStorage: nil,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectError: false,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\terr := test.bsl.Validate()\n\t\t\tif test.expectError && err == nil {\n\t\t\t\tt.Errorf(\"expected error but got none\")\n\t\t\t}\n\t\t\tif !test.expectError && err != nil {\n\t\t\t\tt.Errorf(\"expected no error but got: %v\", err)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/apis/velero/v1/constants.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nconst (\n\t// DefaultNamespace is the Kubernetes namespace that is used by default for\n\t// the Velero server and API objects.\n\tDefaultNamespace = \"velero\"\n\n\t// ResourcesDir is a top-level directory expected in backups which contains sub-directories\n\t// for each resource type in the backup.\n\tResourcesDir = \"resources\"\n\n\t// MetadataDir is a top-level directory expected in backups which contains\n\t// files that store metadata about the backup, such as the backup version.\n\tMetadataDir = \"metadata\"\n\n\t// ClusterScopedDir is the name of the directory containing cluster-scoped\n\t// resources within a Velero backup.\n\tClusterScopedDir = \"cluster\"\n\n\t// NamespaceScopedDir is the name of the directory containing namespace-scoped\n\t// resource within a Velero backup.\n\tNamespaceScopedDir = \"namespaces\"\n\n\t// CSIFeatureFlag is the feature flag string that defines whether or not CSI features are being used.\n\tCSIFeatureFlag = \"EnableCSI\"\n\n\t// PreferredVersionDir is the suffix name of the directory containing the preferred version of the API group\n\t// resource within a Velero backup.\n\tPreferredVersionDir = \"-preferredversion\"\n\n\t// APIGroupVersionsFeatureFlag is the feature flag string that defines whether or not to handle multiple API Group Versions\n\tAPIGroupVersionsFeatureFlag = \"EnableAPIGroupVersions\"\n)\n" }, { "path": "pkg/apis/velero/v1/delete_backup_request_types.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport metav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n// DeleteBackupRequestSpec is the specification for which backups to delete.\ntype DeleteBackupRequestSpec struct {\n\tBackupName string `json:\"backupName\"`\n}\n\n// DeleteBackupRequestPhase represents the lifecycle phase of a DeleteBackupRequest.\n// +kubebuilder:validation:Enum=New;InProgress;Processed\ntype DeleteBackupRequestPhase string\n\nconst (\n\t// DeleteBackupRequestPhaseNew means the DeleteBackupRequest has not been processed yet.\n\tDeleteBackupRequestPhaseNew DeleteBackupRequestPhase = \"New\"\n\n\t// DeleteBackupRequestPhaseInProgress means the DeleteBackupRequest is being processed.\n\tDeleteBackupRequestPhaseInProgress DeleteBackupRequestPhase = \"InProgress\"\n\n\t// DeleteBackupRequestPhaseProcessed means the DeleteBackupRequest has been processed.\n\tDeleteBackupRequestPhaseProcessed DeleteBackupRequestPhase = \"Processed\"\n)\n\n// DeleteBackupRequestStatus is the current status of a DeleteBackupRequest.\ntype DeleteBackupRequestStatus struct {\n\t// Phase is the current state of the DeleteBackupRequest.\n\t// +optional\n\tPhase DeleteBackupRequestPhase `json:\"phase,omitempty\"`\n\n\t// Errors contains any errors that were encountered during the deletion process.\n\t// +optional\n\t// +nullable\n\tErrors []string `json:\"errors,omitempty\"`\n}\n\n// TODO(2.0) After converting all resources to use the runtime-controller client, the genclient and k8s:deepcopy markers will no longer be needed and should be removed.\n// +genclient\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:root=true\n// +kubebuilder:object:generate=true\n// +kubebuilder:storageversion\n// +kubebuilder:printcolumn:name=\"BackupName\",type=\"string\",JSONPath=\".spec.backupName\",description=\"The name of the backup to be deleted\"\n// +kubebuilder:printcolumn:name=\"Status\",type=\"string\",JSONPath=\".status.phase\",description=\"The status of the deletion request\"\n\n// DeleteBackupRequest is a request to delete one or more backups.\ntype DeleteBackupRequest struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ObjectMeta `json:\"metadata,omitempty\"`\n\n\t// +optional\n\tSpec DeleteBackupRequestSpec `json:\"spec,omitempty\"`\n\n\t// +optional\n\tStatus DeleteBackupRequestStatus `json:\"status,omitempty\"`\n}\n\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:root=true\n\n// DeleteBackupRequestList is a list of DeleteBackupRequests.\ntype DeleteBackupRequestList struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ListMeta `json:\"metadata,omitempty\"`\n\n\tItems []DeleteBackupRequest `json:\"items\"`\n}\n" }, { "path": "pkg/apis/velero/v1/doc.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\n// +k8s:deepcopy-gen=package\n\n// Package v1 is the v1 version of the API.\n// +groupName=velero.io\npackage v1\n" }, { "path": "pkg/apis/velero/v1/download_request_types.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport metav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n// DownloadRequestSpec is the specification for a download request.\ntype DownloadRequestSpec struct {\n\t// Target is what to download (e.g. logs for a backup).\n\tTarget DownloadTarget `json:\"target\"`\n}\n\n// DownloadTargetKind represents what type of file to download.\n// +kubebuilder:validation:Enum=BackupLog;BackupContents;BackupVolumeSnapshots;BackupItemOperations;BackupResourceList;BackupResults;RestoreLog;RestoreResults;RestoreResourceList;RestoreItemOperations;CSIBackupVolumeSnapshots;CSIBackupVolumeSnapshotContents;BackupVolumeInfos;RestoreVolumeInfo\ntype DownloadTargetKind string\n\nconst (\n\tDownloadTargetKindBackupLog DownloadTargetKind = \"BackupLog\"\n\tDownloadTargetKindBackupContents DownloadTargetKind = \"BackupContents\"\n\tDownloadTargetKindBackupVolumeSnapshots DownloadTargetKind = \"BackupVolumeSnapshots\"\n\tDownloadTargetKindBackupItemOperations DownloadTargetKind = \"BackupItemOperations\"\n\tDownloadTargetKindBackupResourceList DownloadTargetKind = \"BackupResourceList\"\n\tDownloadTargetKindBackupResults DownloadTargetKind = \"BackupResults\"\n\tDownloadTargetKindRestoreLog DownloadTargetKind = \"RestoreLog\"\n\tDownloadTargetKindRestoreResults DownloadTargetKind = \"RestoreResults\"\n\tDownloadTargetKindRestoreResourceList DownloadTargetKind = \"RestoreResourceList\"\n\tDownloadTargetKindRestoreItemOperations DownloadTargetKind = \"RestoreItemOperations\"\n\tDownloadTargetKindCSIBackupVolumeSnapshots DownloadTargetKind = \"CSIBackupVolumeSnapshots\"\n\tDownloadTargetKindCSIBackupVolumeSnapshotContents DownloadTargetKind = \"CSIBackupVolumeSnapshotContents\"\n\tDownloadTargetKindBackupVolumeInfos DownloadTargetKind = \"BackupVolumeInfos\"\n\tDownloadTargetKindRestoreVolumeInfo DownloadTargetKind = \"RestoreVolumeInfo\"\n)\n\n// DownloadTarget is the specification for what kind of file to download, and the name of the\n// resource with which it's associated.\ntype DownloadTarget struct {\n\t// Kind is the type of file to download.\n\tKind DownloadTargetKind `json:\"kind\"`\n\n\t// Name is the name of the Kubernetes resource with which the file is associated.\n\tName string `json:\"name\"`\n}\n\n// DownloadRequestPhase represents the lifecycle phase of a DownloadRequest.\n// +kubebuilder:validation:Enum=New;Processed\ntype DownloadRequestPhase string\n\nconst (\n\t// DownloadRequestPhaseNew means the DownloadRequest has not been processed by the\n\t// DownloadRequestController yet.\n\tDownloadRequestPhaseNew DownloadRequestPhase = \"New\"\n\n\t// DownloadRequestPhaseProcessed means the DownloadRequest has been processed by the\n\t// DownloadRequestController.\n\tDownloadRequestPhaseProcessed DownloadRequestPhase = \"Processed\"\n)\n\n// DownloadRequestStatus is the current status of a DownloadRequest.\ntype DownloadRequestStatus struct {\n\t// Phase is the current state of the DownloadRequest.\n\t// +optional\n\tPhase DownloadRequestPhase `json:\"phase,omitempty\"`\n\n\t// DownloadURL contains the pre-signed URL for the target file.\n\t// +optional\n\tDownloadURL string `json:\"downloadURL,omitempty\"`\n\n\t// Expiration is when this DownloadRequest expires and can be deleted by the system.\n\t// +optional\n\t// +nullable\n\tExpiration *metav1.Time `json:\"expiration,omitempty\"`\n}\n\n// TODO(2.0) After converting all resources to use the runtime-controller client,\n// the k8s:deepcopy marker will no longer be needed and should be removed.\n// +genclient\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:root=true\n// +kubebuilder:object:generate=true\n// +kubebuilder:storageversion\n\n// DownloadRequest is a request to download an artifact from backup object storage, such as a backup\n// log file.\ntype DownloadRequest struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ObjectMeta `json:\"metadata,omitempty\"`\n\n\t// +optional\n\tSpec DownloadRequestSpec `json:\"spec,omitempty\"`\n\n\t// +optional\n\tStatus DownloadRequestStatus `json:\"status,omitempty\"`\n}\n\n// TODO(2.0) After converting all resources to use the runtime-controller client,\n// the k8s:deepcopy marker will no longer be needed and should be removed.\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:root=true\n// +kubebuilder:rbac:groups=velero.io,resources=downloadrequests,verbs=get;list;watch;create;update;patch;delete\n// +kubebuilder:rbac:groups=velero.io,resources=downloadrequests/status,verbs=get;update;patch\n\n// DownloadRequestList is a list of DownloadRequests.\ntype DownloadRequestList struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ListMeta `json:\"metadata,omitempty\"`\n\n\tItems []DownloadRequest `json:\"items\"`\n}\n" }, { "path": "pkg/apis/velero/v1/groupversion_info.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\n// Package v1 contains API Schema definitions for the velero v1 API group\n// +kubebuilder:object:generate=true\n// +groupName=velero.io\npackage v1\n\nimport (\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n)\n\nvar (\n\t// SchemeGroupVersion is group version used to register these objects\n\tSchemeGroupVersion = schema.GroupVersion{Group: \"velero.io\", Version: \"v1\"}\n\n\t// SchemeBuilder is used to add go types to the GroupVersionKind scheme\n\tSchemeBuilder = runtime.NewSchemeBuilder(addKnownTypes)\n\n\t// AddToScheme adds the types in this group-version to the given scheme.\n\tAddToScheme = SchemeBuilder.AddToScheme\n)\n" }, { "path": "pkg/apis/velero/v1/labels_annotations.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nconst (\n\t// BackupNameLabel is the label key used to identify a backup by name.\n\tBackupNameLabel = \"velero.io/backup-name\"\n\n\t// BackupUIDLabel is the label key used to identify a backup by uid.\n\tBackupUIDLabel = \"velero.io/backup-uid\"\n\n\t// RestoreNameLabel is the label key used to identify a restore by name.\n\tRestoreNameLabel = \"velero.io/restore-name\"\n\n\t// ScheduleNameLabel is the label key used to identify a schedule by name.\n\tScheduleNameLabel = \"velero.io/schedule-name\"\n\n\t// RestoreUIDLabel is the label key used to identify a restore by uid.\n\tRestoreUIDLabel = \"velero.io/restore-uid\"\n\n\t// PodUIDLabel is the label key used to identify a pod by uid.\n\tPodUIDLabel = \"velero.io/pod-uid\"\n\n\t// PVCUIDLabel is the label key used to identify a PVC by uid.\n\tPVCUIDLabel = \"velero.io/pvc-uid\"\n\n\t// PodVolumeOperationTimeoutAnnotation is the annotation key used to apply\n\t// a backup/restore-specific timeout value for pod volume operations (i.e.\n\t// pod volume backups/restores).\n\tPodVolumeOperationTimeoutAnnotation = \"velero.io/pod-volume-timeout\"\n\n\t// StorageLocationLabel is the label key used to identify the storage\n\t// location of a backup.\n\tStorageLocationLabel = \"velero.io/storage-location\"\n\n\t// VolumeNamespaceLabel is the label key used to identify which\n\t// namespace a repository stores backups for.\n\tVolumeNamespaceLabel = \"velero.io/volume-namespace\"\n\n\t// RepositoryTypeLabel is the label key used to identify the type of a repository\n\tRepositoryTypeLabel = \"velero.io/repository-type\"\n\n\t// DataUploadLabel is the label key used to identify the dataupload for snapshot backup pod\n\tDataUploadLabel = \"velero.io/data-upload\"\n\n\t// DataUploadSnapshotInfoLabel is used to identify the configmap that contains the snapshot info of a data upload\n\t// normally the value of the label should the \"true\" or \"false\"\n\tDataUploadSnapshotInfoLabel = \"velero.io/data-upload-snapshot-info\"\n\n\t// DataDownloadLabel is the label key used to identify the datadownload for snapshot restore pod\n\tDataDownloadLabel = \"velero.io/data-download\"\n\n\t// SourceClusterK8sVersionAnnotation is the label key used to identify the k8s\n\t// git version of the backup , i.e. v1.16.4\n\tSourceClusterK8sGitVersionAnnotation = \"velero.io/source-cluster-k8s-gitversion\"\n\n\t// SourceClusterK8sMajorVersionAnnotation is the label key used to identify the k8s\n\t// major version of the backup , i.e. 1\n\tSourceClusterK8sMajorVersionAnnotation = \"velero.io/source-cluster-k8s-major-version\"\n\n\t// SourceClusterK8sMajorVersionAnnotation is the label key used to identify the k8s\n\t// minor version of the backup , i.e. 16\n\tSourceClusterK8sMinorVersionAnnotation = \"velero.io/source-cluster-k8s-minor-version\"\n\n\t// ResourceTimeoutAnnotation is the annotation key used to carry the global resource\n\t// timeout value for backup to plugins.\n\tResourceTimeoutAnnotation = \"velero.io/resource-timeout\"\n\n\t// AsyncOperationIDLabel is the label key used to identify the async operation ID\n\tAsyncOperationIDLabel = \"velero.io/async-operation-id\"\n\n\t// PVCNameLabel is the label key used to identify the PVC's namespace and name.\n\t// The format is /.\n\tPVCNamespaceNameLabel = \"velero.io/pvc-namespace-name\"\n\n\t// ResourceUsageLabel is the label key to explain the Velero resource usage.\n\tResourceUsageLabel = \"velero.io/resource-usage\"\n\n\t// VolumesToBackupAnnotation is the annotation on a pod whose mounted volumes\n\t// need to be backed up using pod volume backup.\n\tVolumesToBackupAnnotation = \"backup.velero.io/backup-volumes\"\n\n\t// VolumesToExcludeAnnotation is the annotation on a pod whose mounted volumes\n\t// should be excluded from pod volume backup.\n\tVolumesToExcludeAnnotation = \"backup.velero.io/backup-volumes-excludes\"\n\n\t// ExcludeFromBackupLabel is the label to exclude k8s resource from backup,\n\t// even if the resource contains a matching selector label.\n\tExcludeFromBackupLabel = \"velero.io/exclude-from-backup\"\n\n\t// SkipFromBackupAnnotation is the annotation used by internal BackupItemActions\n\t// to indicate that a resource should be skipped from backup,\n\t// even if it doesn't have the ExcludeFromBackupLabel.\n\t// This is used in cases where we want to skip backup of a resource based on some logic in a plugin.\n\t//\n\t// Notice: SkipFromBackupAnnotation's priority is higher than MustIncludeAdditionalItemAnnotation.\n\t// If SkipFromBackupAnnotation is set, the resource will be skipped even if MustIncludeAdditionalItemAnnotation is set.\n\tSkipFromBackupAnnotation = \"velero.io/skip-from-backup\"\n\n\t// defaultVGSLabelKey is the default label key used to group PVCs under a VolumeGroupSnapshot\n\tDefaultVGSLabelKey = \"velero.io/volume-group\"\n\n\t// PVBLabel is the label key used to identify the pvb for pvb pod\n\tPVBLabel = \"velero.io/pod-volume-backup\"\n\n\t// PVRLabel is the label key used to identify the pvb for pvr pod\n\tPVRLabel = \"velero.io/pod-volume-restore\"\n)\n\ntype AsyncOperationIDPrefix string\n\nconst (\n\tAsyncOperationIDPrefixDataDownload AsyncOperationIDPrefix = \"dd-\"\n\tAsyncOperationIDPrefixDataUpload AsyncOperationIDPrefix = \"du-\"\n)\n\ntype VeleroResourceUsage string\n\nconst (\n\tVeleroResourceUsageDataUploadResult VeleroResourceUsage = \"DataUpload\"\n)\n\n// CSI related plugin actions' constant variable\nconst (\n\tVolumeSnapshotLabel = \"velero.io/volume-snapshot-name\"\n\tVolumeSnapshotHandleAnnotation = \"velero.io/csi-volumesnapshot-handle\"\n\tVolumeSnapshotRestoreSize = \"velero.io/csi-volumesnapshot-restore-size\"\n\tDriverNameAnnotation = \"velero.io/csi-driver-name\"\n\tVSCDeletionPolicyAnnotation = \"velero.io/csi-vsc-deletion-policy\"\n\tVolumeSnapshotClassSelectorLabel = \"velero.io/csi-volumesnapshot-class\"\n\tVolumeSnapshotClassDriverBackupAnnotationPrefix = \"velero.io/csi-volumesnapshot-class\"\n\tVolumeSnapshotClassDriverPVCAnnotation = \"velero.io/csi-volumesnapshot-class\"\n\n\t// https://kubernetes.io/zh-cn/docs/concepts/storage/volume-snapshot-classes/\n\tVolumeSnapshotClassKubernetesAnnotation = \"snapshot.storage.kubernetes.io/is-default-class\"\n\n\t// There is no release w/ these constants exported. Using the strings for now.\n\t// CSI Annotation volumesnapshotclass\n\t// https://github.com/kubernetes-csi/external-snapshotter/blob/master/pkg/utils/util.go#L59-L60\n\tPrefixedListSecretNameAnnotation = \"csi.storage.k8s.io/snapshotter-list-secret-name\" // #nosec G101\n\tPrefixedListSecretNamespaceAnnotation = \"csi.storage.k8s.io/snapshotter-list-secret-namespace\" // #nosec G101\n\n\t// CSI Annotation volumesnapshotcontents\n\tPrefixedSecretNameAnnotation = \"csi.storage.k8s.io/snapshotter-secret-name\" // #nosec G101\n\tPrefixedSecretNamespaceAnnotation = \"csi.storage.k8s.io/snapshotter-secret-namespace\" // #nosec G101\n\n\t// Velero checks this annotation to determine whether to skip resource excluding check.\n\tMustIncludeAdditionalItemAnnotation = \"backup.velero.io/must-include-additional-items\"\n\t// SkippedNoCSIPVAnnotation - Velero checks this annotation on processed PVC to\n\t// find out if the snapshot was skipped b/c the PV is not provisioned via CSI\n\tSkippedNoCSIPVAnnotation = \"backup.velero.io/skipped-no-csi-pv\"\n\n\t// DynamicPVRestoreLabel is the label key for dynamic PV restore\n\tDynamicPVRestoreLabel = \"velero.io/dynamic-pv-restore\"\n\n\t// DataUploadNameAnnotation is the label key for the DataUpload name\n\tDataUploadNameAnnotation = \"velero.io/data-upload-name\"\n\n\t// Label used on VolumeGroupSnapshotClass to mark it as Velero's default for a CSI driver\n\tVolumeGroupSnapshotClassDefaultLabel = \"velero.io/csi-volumegroupsnapshot-class\"\n\n\t// Annotation on PVC to override the VGS class to use\n\tVolumeGroupSnapshotClassAnnotationPVC = \"velero.io/csi-volume-group-snapshot-class\"\n\n\t// Annotation prefix on Backup to override VGS class per CSI driver\n\tVolumeGroupSnapshotClassAnnotationBackupPrefix = \"velero.io/csi-volumegroupsnapshot-class_\"\n)\n" }, { "path": "pkg/apis/velero/v1/pod_volume_backup_types.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/apis/velero/shared\"\n)\n\n// PodVolumeBackupSpec is the specification for a PodVolumeBackup.\ntype PodVolumeBackupSpec struct {\n\t// Node is the name of the node that the Pod is running on.\n\tNode string `json:\"node\"`\n\n\t// Pod is a reference to the pod containing the volume to be backed up.\n\tPod corev1api.ObjectReference `json:\"pod\"`\n\n\t// Volume is the name of the volume within the Pod to be backed\n\t// up.\n\tVolume string `json:\"volume\"`\n\n\t// BackupStorageLocation is the name of the backup storage location\n\t// where the backup repository is stored.\n\tBackupStorageLocation string `json:\"backupStorageLocation\"`\n\n\t// RepoIdentifier is the backup repository identifier.\n\tRepoIdentifier string `json:\"repoIdentifier\"`\n\n\t// UploaderType is the type of the uploader to handle the data transfer.\n\t// +kubebuilder:validation:Enum=kopia;restic;\"\"\n\t// +optional\n\tUploaderType string `json:\"uploaderType\"`\n\n\t// Tags are a map of key-value pairs that should be applied to the\n\t// volume backup as tags.\n\t// +optional\n\tTags map[string]string `json:\"tags,omitempty\"`\n\n\t// UploaderSettings are a map of key-value pairs that should be applied to the\n\t// uploader configuration.\n\t// +optional\n\t// +nullable\n\tUploaderSettings map[string]string `json:\"uploaderSettings,omitempty\"`\n\n\t// Cancel indicates request to cancel the ongoing PodVolumeBackup. It can be set\n\t// when the PodVolumeBackup is in InProgress phase\n\tCancel bool `json:\"cancel,omitempty\"`\n}\n\n// PodVolumeBackupPhase represents the lifecycle phase of a PodVolumeBackup.\n// +kubebuilder:validation:Enum=New;Accepted;Prepared;InProgress;Canceling;Canceled;Completed;Failed\ntype PodVolumeBackupPhase string\n\nconst (\n\tPodVolumeBackupPhaseNew PodVolumeBackupPhase = \"New\"\n\tPodVolumeBackupPhaseAccepted PodVolumeBackupPhase = \"Accepted\"\n\tPodVolumeBackupPhasePrepared PodVolumeBackupPhase = \"Prepared\"\n\tPodVolumeBackupPhaseInProgress PodVolumeBackupPhase = \"InProgress\"\n\tPodVolumeBackupPhaseCanceling PodVolumeBackupPhase = \"Canceling\"\n\tPodVolumeBackupPhaseCanceled PodVolumeBackupPhase = \"Canceled\"\n\tPodVolumeBackupPhaseCompleted PodVolumeBackupPhase = \"Completed\"\n\tPodVolumeBackupPhaseFailed PodVolumeBackupPhase = \"Failed\"\n)\n\n// PodVolumeBackupStatus is the current status of a PodVolumeBackup.\ntype PodVolumeBackupStatus struct {\n\t// Phase is the current state of the PodVolumeBackup.\n\t// +optional\n\tPhase PodVolumeBackupPhase `json:\"phase,omitempty\"`\n\n\t// Path is the full path within the controller pod being backed up.\n\t// +optional\n\tPath string `json:\"path,omitempty\"`\n\n\t// SnapshotID is the identifier for the snapshot of the pod volume.\n\t// +optional\n\tSnapshotID string `json:\"snapshotID,omitempty\"`\n\n\t// Message is a message about the pod volume backup's status.\n\t// +optional\n\tMessage string `json:\"message,omitempty\"`\n\n\t// StartTimestamp records the time a backup was started.\n\t// Separate from CreationTimestamp, since that value changes\n\t// on restores.\n\t// The server's time is used for StartTimestamps\n\t// +optional\n\t// +nullable\n\tStartTimestamp *metav1.Time `json:\"startTimestamp,omitempty\"`\n\n\t// CompletionTimestamp records the time a backup was completed.\n\t// Completion time is recorded even on failed backups.\n\t// Completion time is recorded before uploading the backup object.\n\t// The server's time is used for CompletionTimestamps\n\t// +optional\n\t// +nullable\n\tCompletionTimestamp *metav1.Time `json:\"completionTimestamp,omitempty\"`\n\n\t// Progress holds the total number of bytes of the volume and the current\n\t// number of backed up bytes. This can be used to display progress information\n\t// about the backup operation.\n\t// +optional\n\tProgress shared.DataMoveOperationProgress `json:\"progress,omitempty\"`\n\n\t// IncrementalBytes holds the number of bytes new or changed since the last backup\n\t// +optional\n\tIncrementalBytes int64 `json:\"incrementalBytes,omitempty\"`\n\n\t// AcceptedTimestamp records the time the pod volume backup is to be prepared.\n\t// The server's time is used for AcceptedTimestamp\n\t// +optional\n\t// +nullable\n\tAcceptedTimestamp *metav1.Time `json:\"acceptedTimestamp,omitempty\"`\n}\n\n// TODO(2.0) After converting all resources to use the runttime-controller client,\n// the genclient and k8s:deepcopy markers will no longer be needed and should be removed.\n// +genclient\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:storageversion\n// +kubebuilder:printcolumn:name=\"Status\",type=\"string\",JSONPath=\".status.phase\",description=\"PodVolumeBackup status such as New/InProgress\"\n// +kubebuilder:printcolumn:name=\"Started\",type=\"date\",JSONPath=\".status.startTimestamp\",description=\"Time duration since this PodVolumeBackup was started\"\n// +kubebuilder:printcolumn:name=\"Bytes Done\",type=\"integer\",format=\"int64\",JSONPath=\".status.progress.bytesDone\",description=\"Completed bytes\"\n// +kubebuilder:printcolumn:name=\"Total Bytes\",type=\"integer\",format=\"int64\",JSONPath=\".status.progress.totalBytes\",description=\"Total bytes\"\n// +kubebuilder:printcolumn:name=\"Incremental Bytes\",type=\"integer\",format=\"int64\",JSONPath=\".status.incrementalBytes\",description=\"Incremental bytes\",priority=10\n// +kubebuilder:printcolumn:name=\"Storage Location\",type=\"string\",JSONPath=\".spec.backupStorageLocation\",description=\"Name of the Backup Storage Location where this backup should be stored\"\n// +kubebuilder:printcolumn:name=\"Age\",type=\"date\",JSONPath=\".metadata.creationTimestamp\",description=\"Time duration since this PodVolumeBackup was created\"\n// +kubebuilder:printcolumn:name=\"Node\",type=\"string\",JSONPath=\".status.node\",description=\"Name of the node where the PodVolumeBackup is processed\"\n// +kubebuilder:printcolumn:name=\"Uploader\",type=\"string\",JSONPath=\".spec.uploaderType\",description=\"The type of the uploader to handle data transfer\"\n// +kubebuilder:object:root=true\n// +kubebuilder:object:generate=true\n\ntype PodVolumeBackup struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ObjectMeta `json:\"metadata,omitempty\"`\n\n\t// +optional\n\tSpec PodVolumeBackupSpec `json:\"spec,omitempty\"`\n\n\t// +optional\n\tStatus PodVolumeBackupStatus `json:\"status,omitempty\"`\n}\n\n// TODO(2.0) After converting all resources to use the runtime-controller client,\n// the k8s:deepcopy marker will no longer be needed and should be removed.\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:root=true\n// +kubebuilder:rbac:groups=velero.io,resources=podvolumebackups,verbs=get;list;watch;create;update;patch;delete\n// +kubebuilder:rbac:groups=velero.io,resources=podvolumebackups/status,verbs=get;update;patch\n\n// PodVolumeBackupList is a list of PodVolumeBackups.\ntype PodVolumeBackupList struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ListMeta `json:\"metadata,omitempty\"`\n\n\tItems []PodVolumeBackup `json:\"items\"`\n}\n" }, { "path": "pkg/apis/velero/v1/pod_volume_restore_type.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/apis/velero/shared\"\n)\n\n// PodVolumeRestoreSpec is the specification for a PodVolumeRestore.\ntype PodVolumeRestoreSpec struct {\n\t// Pod is a reference to the pod containing the volume to be restored.\n\tPod corev1api.ObjectReference `json:\"pod\"`\n\n\t// Volume is the name of the volume within the Pod to be restored.\n\tVolume string `json:\"volume\"`\n\n\t// BackupStorageLocation is the name of the backup storage location\n\t// where the backup repository is stored.\n\tBackupStorageLocation string `json:\"backupStorageLocation\"`\n\n\t// RepoIdentifier is the backup repository identifier.\n\tRepoIdentifier string `json:\"repoIdentifier\"`\n\n\t// UploaderType is the type of the uploader to handle the data transfer.\n\t// +kubebuilder:validation:Enum=kopia;restic;\"\"\n\t// +optional\n\tUploaderType string `json:\"uploaderType\"`\n\n\t// SnapshotID is the ID of the volume snapshot to be restored.\n\tSnapshotID string `json:\"snapshotID\"`\n\n\t// SourceNamespace is the original namespace for namaspace mapping.\n\tSourceNamespace string `json:\"sourceNamespace\"`\n\n\t// UploaderSettings are a map of key-value pairs that should be applied to the\n\t// uploader configuration.\n\t// +optional\n\t// +nullable\n\tUploaderSettings map[string]string `json:\"uploaderSettings,omitempty\"`\n\n\t// Cancel indicates request to cancel the ongoing PodVolumeRestore. It can be set\n\t// when the PodVolumeRestore is in InProgress phase\n\tCancel bool `json:\"cancel,omitempty\"`\n\n\t// SnapshotSize is the logical size in Bytes of the snapshot.\n\t// +optional\n\tSnapshotSize int64 `json:\"snapshotSize,omitempty\"`\n}\n\n// PodVolumeRestorePhase represents the lifecycle phase of a PodVolumeRestore.\n// +kubebuilder:validation:Enum=New;Accepted;Prepared;InProgress;Canceling;Canceled;Completed;Failed\ntype PodVolumeRestorePhase string\n\nconst (\n\tPodVolumeRestorePhaseNew PodVolumeRestorePhase = \"New\"\n\tPodVolumeRestorePhaseAccepted PodVolumeRestorePhase = \"Accepted\"\n\tPodVolumeRestorePhasePrepared PodVolumeRestorePhase = \"Prepared\"\n\tPodVolumeRestorePhaseInProgress PodVolumeRestorePhase = \"InProgress\"\n\tPodVolumeRestorePhaseCanceling PodVolumeRestorePhase = \"Canceling\"\n\tPodVolumeRestorePhaseCanceled PodVolumeRestorePhase = \"Canceled\"\n\tPodVolumeRestorePhaseCompleted PodVolumeRestorePhase = \"Completed\"\n\tPodVolumeRestorePhaseFailed PodVolumeRestorePhase = \"Failed\"\n)\n\n// PodVolumeRestoreStatus is the current status of a PodVolumeRestore.\ntype PodVolumeRestoreStatus struct {\n\t// Phase is the current state of the PodVolumeRestore.\n\t// +optional\n\tPhase PodVolumeRestorePhase `json:\"phase,omitempty\"`\n\n\t// Message is a message about the pod volume restore's status.\n\t// +optional\n\tMessage string `json:\"message,omitempty\"`\n\n\t// StartTimestamp records the time a restore was started.\n\t// The server's time is used for StartTimestamps\n\t// +optional\n\t// +nullable\n\tStartTimestamp *metav1.Time `json:\"startTimestamp,omitempty\"`\n\n\t// CompletionTimestamp records the time a restore was completed.\n\t// Completion time is recorded even on failed restores.\n\t// The server's time is used for CompletionTimestamps\n\t// +optional\n\t// +nullable\n\tCompletionTimestamp *metav1.Time `json:\"completionTimestamp,omitempty\"`\n\n\t// Progress holds the total number of bytes of the snapshot and the current\n\t// number of restored bytes. This can be used to display progress information\n\t// about the restore operation.\n\t// +optional\n\tProgress shared.DataMoveOperationProgress `json:\"progress,omitempty\"`\n\n\t// AcceptedTimestamp records the time the pod volume restore is to be prepared.\n\t// The server's time is used for AcceptedTimestamp\n\t// +optional\n\t// +nullable\n\tAcceptedTimestamp *metav1.Time `json:\"acceptedTimestamp,omitempty\"`\n\n\t// Node is name of the node where the pod volume restore is processed.\n\t// +optional\n\tNode string `json:\"node,omitempty\"`\n}\n\n// TODO(2.0) After converting all resources to use the runtime-controller client, the genclient and k8s:deepcopy markers will no longer be needed and should be removed.\n// +genclient\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:generate=true\n// +kubebuilder:object:root=true\n// +kubebuilder:storageversion\n// +kubebuilder:printcolumn:name=\"Status\",type=\"string\",JSONPath=\".status.phase\",description=\"PodVolumeRestore status such as New/InProgress\"\n// +kubebuilder:printcolumn:name=\"Started\",type=\"date\",JSONPath=\".status.startTimestamp\",description=\"Time duration since this PodVolumeRestore was started\"\n// +kubebuilder:printcolumn:name=\"Bytes Done\",type=\"integer\",format=\"int64\",JSONPath=\".status.progress.bytesDone\",description=\"Completed bytes\"\n// +kubebuilder:printcolumn:name=\"Total Bytes\",type=\"integer\",format=\"int64\",JSONPath=\".status.progress.totalBytes\",description=\"Total bytes\"\n// +kubebuilder:printcolumn:name=\"Storage Location\",type=\"string\",JSONPath=\".spec.backupStorageLocation\",description=\"Name of the Backup Storage Location where the backup data is stored\"\n// +kubebuilder:printcolumn:name=\"Age\",type=\"date\",JSONPath=\".metadata.creationTimestamp\",description=\"Time duration since this PodVolumeRestore was created\"\n// +kubebuilder:printcolumn:name=\"Node\",type=\"string\",JSONPath=\".status.node\",description=\"Name of the node where the PodVolumeRestore is processed\"\n// +kubebuilder:printcolumn:name=\"Uploader Type\",type=\"string\",JSONPath=\".spec.uploaderType\",description=\"The type of the uploader to handle data transfer\"\n\ntype PodVolumeRestore struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ObjectMeta `json:\"metadata,omitempty\"`\n\n\t// +optional\n\tSpec PodVolumeRestoreSpec `json:\"spec,omitempty\"`\n\n\t// +optional\n\tStatus PodVolumeRestoreStatus `json:\"status,omitempty\"`\n}\n\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:generate=true\n// +kubebuilder:object:root=true\n\n// PodVolumeRestoreList is a list of PodVolumeRestores.\ntype PodVolumeRestoreList struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ListMeta `json:\"metadata,omitempty\"`\n\n\tItems []PodVolumeRestore `json:\"items\"`\n}\n" }, { "path": "pkg/apis/velero/v1/register.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n)\n\n// Resource gets a Velero GroupResource for a specified resource\nfunc Resource(resource string) schema.GroupResource {\n\treturn SchemeGroupVersion.WithResource(resource).GroupResource()\n}\n\ntype typeInfo struct {\n\tPluralName string\n\tItemType runtime.Object\n\tItemListType runtime.Object\n}\n\nfunc newTypeInfo(pluralName string, itemType, itemListType runtime.Object) typeInfo {\n\treturn typeInfo{\n\t\tPluralName: pluralName,\n\t\tItemType: itemType,\n\t\tItemListType: itemListType,\n\t}\n}\n\n// CustomResources returns a map of all custom resources within the Velero\n// API group, keyed on Kind.\nfunc CustomResources() map[string]typeInfo {\n\treturn map[string]typeInfo{\n\t\t\"Backup\": newTypeInfo(\"backups\", &Backup{}, &BackupList{}),\n\t\t\"Restore\": newTypeInfo(\"restores\", &Restore{}, &RestoreList{}),\n\t\t\"Schedule\": newTypeInfo(\"schedules\", &Schedule{}, &ScheduleList{}),\n\t\t\"DownloadRequest\": newTypeInfo(\"downloadrequests\", &DownloadRequest{}, &DownloadRequestList{}),\n\t\t\"DeleteBackupRequest\": newTypeInfo(\"deletebackuprequests\", &DeleteBackupRequest{}, &DeleteBackupRequestList{}),\n\t\t\"PodVolumeBackup\": newTypeInfo(\"podvolumebackups\", &PodVolumeBackup{}, &PodVolumeBackupList{}),\n\t\t\"PodVolumeRestore\": newTypeInfo(\"podvolumerestores\", &PodVolumeRestore{}, &PodVolumeRestoreList{}),\n\t\t\"BackupRepository\": newTypeInfo(\"backuprepositories\", &BackupRepository{}, &BackupRepositoryList{}),\n\t\t\"BackupStorageLocation\": newTypeInfo(\"backupstoragelocations\", &BackupStorageLocation{}, &BackupStorageLocationList{}),\n\t\t\"VolumeSnapshotLocation\": newTypeInfo(\"volumesnapshotlocations\", &VolumeSnapshotLocation{}, &VolumeSnapshotLocationList{}),\n\t\t\"ServerStatusRequest\": newTypeInfo(\"serverstatusrequests\", &ServerStatusRequest{}, &ServerStatusRequestList{}),\n\t}\n}\n\n// CustomResourceKinds returns a list of all custom resources kinds within the Velero\nfunc CustomResourceKinds() sets.Set[string] {\n\tkinds := sets.New[string]()\n\n\tresources := CustomResources()\n\tfor kind := range resources {\n\t\tkinds.Insert(kind)\n\t}\n\n\treturn kinds\n}\n\nfunc addKnownTypes(scheme *runtime.Scheme) error {\n\tfor _, typeInfo := range CustomResources() {\n\t\tscheme.AddKnownTypes(SchemeGroupVersion, typeInfo.ItemType, typeInfo.ItemListType)\n\t}\n\n\tmetav1.AddToGroupVersion(scheme, SchemeGroupVersion)\n\treturn nil\n}\n" }, { "path": "pkg/apis/velero/v1/restore_types.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n)\n\n// RestoreSpec defines the specification for a Velero restore.\ntype RestoreSpec struct {\n\t// BackupName is the unique name of the Velero backup to restore\n\t// from.\n\t// +optional\n\tBackupName string `json:\"backupName,omitempty\"`\n\n\t// ScheduleName is the unique name of the Velero schedule to restore\n\t// from. If specified, and BackupName is empty, Velero will restore\n\t// from the most recent successful backup created from this schedule.\n\t// +optional\n\tScheduleName string `json:\"scheduleName,omitempty\"`\n\n\t// IncludedNamespaces is a slice of namespace names to include objects\n\t// from. If empty, all namespaces are included.\n\t// +optional\n\t// +nullable\n\tIncludedNamespaces []string `json:\"includedNamespaces,omitempty\"`\n\n\t// ExcludedNamespaces contains a list of namespaces that are not\n\t// included in the restore.\n\t// +optional\n\t// +nullable\n\tExcludedNamespaces []string `json:\"excludedNamespaces,omitempty\"`\n\n\t// IncludedResources is a slice of resource names to include\n\t// in the restore. If empty, all resources in the backup are included.\n\t// +optional\n\t// +nullable\n\tIncludedResources []string `json:\"includedResources,omitempty\"`\n\n\t// ExcludedResources is a slice of resource names that are not\n\t// included in the restore.\n\t// +optional\n\t// +nullable\n\tExcludedResources []string `json:\"excludedResources,omitempty\"`\n\n\t// NamespaceMapping is a map of source namespace names\n\t// to target namespace names to restore into. Any source\n\t// namespaces not included in the map will be restored into\n\t// namespaces of the same name.\n\t// +optional\n\tNamespaceMapping map[string]string `json:\"namespaceMapping,omitempty\"`\n\n\t// LabelSelector is a metav1.LabelSelector to filter with\n\t// when restoring individual objects from the backup. If empty\n\t// or nil, all objects are included. Optional.\n\t// +optional\n\t// +nullable\n\tLabelSelector *metav1.LabelSelector `json:\"labelSelector,omitempty\"`\n\n\t// OrLabelSelectors is list of metav1.LabelSelector to filter with\n\t// when restoring individual objects from the backup. If multiple provided\n\t// they will be joined by the OR operator. LabelSelector as well as\n\t// OrLabelSelectors cannot co-exist in restore request, only one of them\n\t// can be used\n\t// +optional\n\t// +nullable\n\tOrLabelSelectors []*metav1.LabelSelector `json:\"orLabelSelectors,omitempty\"`\n\n\t// RestorePVs specifies whether to restore all included\n\t// PVs from snapshot\n\t// +optional\n\t// +nullable\n\tRestorePVs *bool `json:\"restorePVs,omitempty\"`\n\n\t// RestoreStatus specifies which resources we should restore the status\n\t// field. If nil, no objects are included. Optional.\n\t// +optional\n\t// +nullable\n\tRestoreStatus *RestoreStatusSpec `json:\"restoreStatus,omitempty\"`\n\n\t// PreserveNodePorts specifies whether to restore old nodePorts from backup.\n\t// +optional\n\t// +nullable\n\tPreserveNodePorts *bool `json:\"preserveNodePorts,omitempty\"`\n\n\t// IncludeClusterResources specifies whether cluster-scoped resources\n\t// should be included for consideration in the restore. If null, defaults\n\t// to true.\n\t// +optional\n\t// +nullable\n\tIncludeClusterResources *bool `json:\"includeClusterResources,omitempty\"`\n\n\t// Hooks represent custom behaviors that should be executed during or post restore.\n\t// +optional\n\tHooks RestoreHooks `json:\"hooks,omitempty\"`\n\n\t// ExistingResourcePolicy specifies the restore behavior for the Kubernetes resource to be restored\n\t// +optional\n\t// +nullable\n\tExistingResourcePolicy PolicyType `json:\"existingResourcePolicy,omitempty\"`\n\n\t// ItemOperationTimeout specifies the time used to wait for RestoreItemAction operations\n\t// The default value is 4 hour.\n\t// +optional\n\tItemOperationTimeout metav1.Duration `json:\"itemOperationTimeout,omitempty\"`\n\n\t// ResourceModifier specifies the reference to JSON resource patches that should be applied to resources before restoration.\n\t// +optional\n\t// +nullable\n\tResourceModifier *corev1api.TypedLocalObjectReference `json:\"resourceModifier,omitempty\"`\n\n\t// UploaderConfig specifies the configuration for the restore.\n\t// +optional\n\t// +nullable\n\tUploaderConfig *UploaderConfigForRestore `json:\"uploaderConfig,omitempty\"`\n}\n\n// UploaderConfigForRestore defines the configuration for the restore.\ntype UploaderConfigForRestore struct {\n\t// WriteSparseFiles is a flag to indicate whether write files sparsely or not.\n\t// +optional\n\t// +nullable\n\tWriteSparseFiles *bool `json:\"writeSparseFiles,omitempty\"`\n\t// ParallelFilesDownload is the concurrency number setting for restore.\n\t// +optional\n\tParallelFilesDownload int `json:\"parallelFilesDownload,omitempty\"`\n}\n\n// RestoreHooks contains custom behaviors that should be executed during or post restore.\ntype RestoreHooks struct {\n\tResources []RestoreResourceHookSpec `json:\"resources,omitempty\"`\n}\n\ntype RestoreStatusSpec struct {\n\t// IncludedResources specifies the resources to which will restore the status.\n\t// If empty, it applies to all resources.\n\t// +optional\n\t// +nullable\n\tIncludedResources []string `json:\"includedResources,omitempty\"`\n\n\t// ExcludedResources specifies the resources to which will not restore the status.\n\t// +optional\n\t// +nullable\n\tExcludedResources []string `json:\"excludedResources,omitempty\"`\n}\n\n// RestoreResourceHookSpec defines one or more RestoreResrouceHooks that should be executed based on\n// the rules defined for namespaces, resources, and label selector.\ntype RestoreResourceHookSpec struct {\n\t// Name is the name of this hook.\n\tName string `json:\"name\"`\n\n\t// IncludedNamespaces specifies the namespaces to which this hook spec applies. If empty, it applies\n\t// to all namespaces.\n\t// +optional\n\t// +nullable\n\tIncludedNamespaces []string `json:\"includedNamespaces,omitempty\"`\n\n\t// ExcludedNamespaces specifies the namespaces to which this hook spec does not apply.\n\t// +optional\n\t// +nullable\n\tExcludedNamespaces []string `json:\"excludedNamespaces,omitempty\"`\n\n\t// IncludedResources specifies the resources to which this hook spec applies. If empty, it applies\n\t// to all resources.\n\t// +optional\n\t// +nullable\n\tIncludedResources []string `json:\"includedResources,omitempty\"`\n\n\t// ExcludedResources specifies the resources to which this hook spec does not apply.\n\t// +optional\n\t// +nullable\n\tExcludedResources []string `json:\"excludedResources,omitempty\"`\n\n\t// LabelSelector, if specified, filters the resources to which this hook spec applies.\n\t// +optional\n\t// +nullable\n\tLabelSelector *metav1.LabelSelector `json:\"labelSelector,omitempty\"`\n\n\t// PostHooks is a list of RestoreResourceHooks to execute during and after restoring a resource.\n\t// +optional\n\tPostHooks []RestoreResourceHook `json:\"postHooks,omitempty\"`\n}\n\n// RestoreResourceHook defines a restore hook for a resource.\ntype RestoreResourceHook struct {\n\t// Exec defines an exec restore hook.\n\tExec *ExecRestoreHook `json:\"exec,omitempty\"`\n\n\t// Init defines an init restore hook.\n\tInit *InitRestoreHook `json:\"init,omitempty\"`\n}\n\n// ExecRestoreHook is a hook that uses pod exec API to execute a command inside a container in a pod\ntype ExecRestoreHook struct {\n\t// Container is the container in the pod where the command should be executed. If not specified,\n\t// the pod's first container is used.\n\t// +optional\n\tContainer string `json:\"container,omitempty\"`\n\n\t// Command is the command and arguments to execute from within a container after a pod has been restored.\n\t// +kubebuilder:validation:MinItems=1\n\tCommand []string `json:\"command\"`\n\n\t// OnError specifies how Velero should behave if it encounters an error executing this hook.\n\t// +optional\n\tOnError HookErrorMode `json:\"onError,omitempty\"`\n\n\t// ExecTimeout defines the maximum amount of time Velero should wait for the hook to complete before\n\t// considering the execution a failure.\n\t// +optional\n\tExecTimeout metav1.Duration `json:\"execTimeout,omitempty\"`\n\n\t// WaitTimeout defines the maximum amount of time Velero should wait for the container to be Ready\n\t// before attempting to run the command.\n\t// +optional\n\tWaitTimeout metav1.Duration `json:\"waitTimeout,omitempty\"`\n\n\t// WaitForReady ensures command will be launched when container is Ready instead of Running.\n\t// +optional\n\t// +nullable\n\tWaitForReady *bool `json:\"waitForReady,omitempty\"`\n}\n\n// InitRestoreHook is a hook that adds an init container to a PodSpec to run commands before the\n// workload pod is able to start.\ntype InitRestoreHook struct {\n\t// +kubebuilder:pruning:PreserveUnknownFields\n\t// InitContainers is list of init containers to be added to a pod during its restore.\n\t// +optional\n\tInitContainers []runtime.RawExtension `json:\"initContainers\"`\n\n\t// Timeout defines the maximum amount of time Velero should wait for the initContainers to complete.\n\t// +optional\n\tTimeout metav1.Duration `json:\"timeout,omitempty\"`\n}\n\n// RestorePhase is a string representation of the lifecycle phase\n// of a Velero restore\n// +kubebuilder:validation:Enum=New;FailedValidation;InProgress;WaitingForPluginOperations;WaitingForPluginOperationsPartiallyFailed;Completed;PartiallyFailed;Failed;Finalizing;FinalizingPartiallyFailed\ntype RestorePhase string\n\nconst (\n\t// RestorePhaseNew means the restore has been created but not\n\t// yet processed by the RestoreController\n\tRestorePhaseNew RestorePhase = \"New\"\n\n\t// RestorePhaseFailedValidation means the restore has failed\n\t// the controller's validations and therefore will not run.\n\tRestorePhaseFailedValidation RestorePhase = \"FailedValidation\"\n\n\t// RestorePhaseInProgress means the restore is currently executing.\n\tRestorePhaseInProgress RestorePhase = \"InProgress\"\n\n\t// RestorePhaseWaitingForPluginOperations means the restore of\n\t// Kubernetes resources and other async plugin operations was\n\t// successful and plugin operations are still ongoing. The\n\t// restore is not complete yet.\n\tRestorePhaseWaitingForPluginOperations RestorePhase = \"WaitingForPluginOperations\"\n\n\t// RestorePhaseWaitingForPluginOperationsPartiallyFailed means\n\t// the restore of Kubernetes resources and other async plugin\n\t// operations partially failed (final phase will be\n\t// PartiallyFailed) and other plugin operations are still\n\t// ongoing. The restore is not complete yet.\n\tRestorePhaseWaitingForPluginOperationsPartiallyFailed RestorePhase = \"WaitingForPluginOperationsPartiallyFailed\"\n\n\t// RestorePhaseFinalizing means the restore of\n\t// Kubernetes resources and other async plugin operations were successful and\n\t// other plugin operations are now complete, but the restore is awaiting\n\t// the completion of wrap-up tasks before the restore process enters terminal phase.\n\tRestorePhaseFinalizing RestorePhase = \"Finalizing\"\n\n\t// RestorePhaseFinalizingPartiallyFailed means the restore of\n\t// Kubernetes resources and other async plugin operations were successful and\n\t// other plugin operations are now complete, but one or more errors\n\t// occurred during restore or async operation processing. The restore is awaiting\n\t// the completion of wrap-up tasks before the restore process enters terminal phase.\n\tRestorePhaseFinalizingPartiallyFailed RestorePhase = \"FinalizingPartiallyFailed\"\n\n\t// RestorePhaseCompleted means the restore has run successfully\n\t// without errors.\n\tRestorePhaseCompleted RestorePhase = \"Completed\"\n\n\t// RestorePhasePartiallyFailed means the restore has run to completion\n\t// but encountered 1+ errors restoring individual items.\n\tRestorePhasePartiallyFailed RestorePhase = \"PartiallyFailed\"\n\n\t// RestorePhaseFailed means the restore was unable to execute.\n\t// The failing error is recorded in status.FailureReason.\n\tRestorePhaseFailed RestorePhase = \"Failed\"\n\n\t// PolicyTypeNone means velero will not overwrite the resource\n\t// in cluster with the one in backup whether changed/unchanged.\n\tPolicyTypeNone PolicyType = \"none\"\n\n\t// PolicyTypeUpdate means velero will try to attempt a patch on\n\t// the changed resources.\n\tPolicyTypeUpdate PolicyType = \"update\"\n)\n\n// RestoreStatus captures the current status of a Velero restore\ntype RestoreStatus struct {\n\t// Phase is the current state of the Restore\n\t// +optional\n\tPhase RestorePhase `json:\"phase,omitempty\"`\n\n\t// ValidationErrors is a slice of all validation errors (if\n\t// applicable)\n\t// +optional\n\t// +nullable\n\tValidationErrors []string `json:\"validationErrors,omitempty\"`\n\n\t// Warnings is a count of all warning messages that were generated during\n\t// execution of the restore. The actual warnings are stored in object storage.\n\t// +optional\n\tWarnings int `json:\"warnings,omitempty\"`\n\n\t// Errors is a count of all error messages that were generated during\n\t// execution of the restore. The actual errors are stored in object storage.\n\t// +optional\n\tErrors int `json:\"errors,omitempty\"`\n\n\t// FailureReason is an error that caused the entire restore to fail.\n\t// +optional\n\tFailureReason string `json:\"failureReason,omitempty\"`\n\n\t// StartTimestamp records the time the restore operation was started.\n\t// The server's time is used for StartTimestamps\n\t// +optional\n\t// +nullable\n\tStartTimestamp *metav1.Time `json:\"startTimestamp,omitempty\"`\n\n\t// CompletionTimestamp records the time the restore operation was completed.\n\t// Completion time is recorded even on failed restore.\n\t// The server's time is used for StartTimestamps\n\t// +optional\n\t// +nullable\n\tCompletionTimestamp *metav1.Time `json:\"completionTimestamp,omitempty\"`\n\n\t// Progress contains information about the restore's execution progress. Note\n\t// that this information is best-effort only -- if Velero fails to update it\n\t// during a restore for any reason, it may be inaccurate/stale.\n\t// +optional\n\t// +nullable\n\tProgress *RestoreProgress `json:\"progress,omitempty\"`\n\n\t// RestoreItemOperationsAttempted is the total number of attempted\n\t// async RestoreItemAction operations for this restore.\n\t// +optional\n\tRestoreItemOperationsAttempted int `json:\"restoreItemOperationsAttempted,omitempty\"`\n\n\t// RestoreItemOperationsCompleted is the total number of successfully completed\n\t// async RestoreItemAction operations for this restore.\n\t// +optional\n\tRestoreItemOperationsCompleted int `json:\"restoreItemOperationsCompleted,omitempty\"`\n\n\t// RestoreItemOperationsFailed is the total number of async\n\t// RestoreItemAction operations for this restore which ended with an error.\n\t// +optional\n\tRestoreItemOperationsFailed int `json:\"restoreItemOperationsFailed,omitempty\"`\n\n\t// HookStatus contains information about the status of the hooks.\n\t// +optional\n\t// +nullable\n\tHookStatus *HookStatus `json:\"hookStatus,omitempty\"`\n}\n\n// RestoreProgress stores information about the restore's execution progress\ntype RestoreProgress struct {\n\t// TotalItems is the total number of items to be restored. This number may change\n\t// throughout the execution of the restore due to plugins that return additional related\n\t// items to restore\n\t// +optional\n\tTotalItems int `json:\"totalItems,omitempty\"`\n\t// ItemsRestored is the number of items that have actually been restored so far\n\t// +optional\n\tItemsRestored int `json:\"itemsRestored,omitempty\"`\n}\n\n// +genclient\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:root=true\n// +kubebuilder:object:generate=true\n// +kubebuilder:storageversion\n// +kubebuilder:rbac:groups=velero.io,resources=restores,verbs=create;delete;get;list;patch;update;watch\n// +kubebuilder:rbac:groups=velero.io,resources=restores/status,verbs=get;update;patch\n\n// Restore is a Velero resource that represents the application of\n// resources from a Velero backup to a target Kubernetes cluster.\ntype Restore struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ObjectMeta `json:\"metadata,omitempty\"`\n\n\t// +optional\n\tSpec RestoreSpec `json:\"spec,omitempty\"`\n\n\t// +optional\n\tStatus RestoreStatus `json:\"status,omitempty\"`\n}\n\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n\n// RestoreList is a list of Restores.\ntype RestoreList struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ListMeta `json:\"metadata\"`\n\n\tItems []Restore `json:\"items\"`\n}\n\n// PolicyType helps specify the ExistingResourcePolicy\ntype PolicyType string\n" }, { "path": "pkg/apis/velero/v1/schedule_types.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\t\"fmt\"\n\t\"time\"\n\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// ScheduleSpec defines the specification for a Velero schedule\ntype ScheduleSpec struct {\n\t// Template is the definition of the Backup to be run\n\t// on the provided schedule\n\tTemplate BackupSpec `json:\"template\"`\n\n\t// Schedule is a Cron expression defining when to run\n\t// the Backup.\n\tSchedule string `json:\"schedule\"`\n\n\t// UseOwnerReferencesBackup specifies whether to use\n\t// OwnerReferences on backups created by this Schedule.\n\t// +optional\n\t// +nullable\n\tUseOwnerReferencesInBackup *bool `json:\"useOwnerReferencesInBackup,omitempty\"`\n\n\t// Paused specifies whether the schedule is paused or not\n\t// +optional\n\tPaused bool `json:\"paused,omitempty\"`\n\n\t// SkipImmediately specifies whether to skip backup if schedule is due immediately from `schedule.status.lastBackup` timestamp when schedule is unpaused or if schedule is new.\n\t// If true, backup will be skipped immediately when schedule is unpaused if it is due based on .Status.LastBackupTimestamp or schedule is new, and will run at next schedule time.\n\t// If false, backup will not be skipped immediately when schedule is unpaused, but will run at next schedule time.\n\t// If empty, will follow server configuration (default: false).\n\t// +optional\n\tSkipImmediately *bool `json:\"skipImmediately,omitempty\"`\n}\n\n// SchedulePhase is a string representation of the lifecycle phase\n// of a Velero schedule\n// +kubebuilder:validation:Enum=New;Enabled;FailedValidation\ntype SchedulePhase string\n\nconst (\n\t// SchedulePhaseNew means the schedule has been created but not\n\t// yet processed by the ScheduleController\n\tSchedulePhaseNew SchedulePhase = \"New\"\n\n\t// SchedulePhaseEnabled means the schedule has been validated and\n\t// will now be triggering backups according to the schedule spec.\n\tSchedulePhaseEnabled SchedulePhase = \"Enabled\"\n\n\t// SchedulePhaseFailedValidation means the schedule has failed\n\t// the controller's validations and therefore will not trigger backups.\n\tSchedulePhaseFailedValidation SchedulePhase = \"FailedValidation\"\n)\n\n// ScheduleStatus captures the current state of a Velero schedule\ntype ScheduleStatus struct {\n\t// Phase is the current phase of the Schedule\n\t// +optional\n\tPhase SchedulePhase `json:\"phase,omitempty\"`\n\n\t// LastBackup is the last time a Backup was run for this\n\t// Schedule schedule\n\t// +optional\n\t// +nullable\n\tLastBackup *metav1.Time `json:\"lastBackup,omitempty\"`\n\n\t// LastSkipped is the last time a Schedule was skipped\n\t// +optional\n\t// +nullable\n\tLastSkipped *metav1.Time `json:\"lastSkipped,omitempty\"`\n\n\t// ValidationErrors is a slice of all validation errors (if\n\t// applicable)\n\t// +optional\n\tValidationErrors []string `json:\"validationErrors,omitempty\"`\n}\n\n// TODO(2.0) After converting all resources to use the runtime-controller client, the genclient and k8s:deepcopy markers will no longer be needed and should be removed.\n// +genclient\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:generate=true\n// +kubebuilder:object:root=true\n// +kubebuilder:storageversion\n// +kubebuilder:printcolumn:name=\"Status\",type=\"string\",JSONPath=\".status.phase\",description=\"Status of the schedule\"\n// +kubebuilder:printcolumn:name=\"Schedule\",type=\"string\",JSONPath=\".spec.schedule\",description=\"A Cron expression defining when to run the Backup\"\n// +kubebuilder:printcolumn:name=\"LastBackup\",type=\"date\",JSONPath=\".status.lastBackup\",description=\"The last time a Backup was run for this schedule\"\n// +kubebuilder:printcolumn:name=\"Age\",type=\"date\",JSONPath=\".metadata.creationTimestamp\"\n// +kubebuilder:printcolumn:name=\"Paused\",type=\"boolean\",JSONPath=\".spec.paused\"\n\n// Schedule is a Velero resource that represents a pre-scheduled or\n// periodic Backup that should be run.\ntype Schedule struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ObjectMeta `json:\"metadata\"`\n\n\t// +optional\n\tSpec ScheduleSpec `json:\"spec,omitempty\"`\n\n\t// +optional\n\tStatus ScheduleStatus `json:\"status,omitempty\"`\n}\n\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:generate=true\n// +kubebuilder:object:root=true\n\n// ScheduleList is a list of Schedules.\ntype ScheduleList struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ListMeta `json:\"metadata,omitempty\"`\n\n\tItems []Schedule `json:\"items\"`\n}\n\n// TimestampedName returns the default backup name format based on the schedule\nfunc (s *Schedule) TimestampedName(timestamp time.Time) string {\n\treturn fmt.Sprintf(\"%s-%s\", s.Name, timestamp.Format(\"20060102150405\"))\n}\n" }, { "path": "pkg/apis/velero/v1/server_status_request_types.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// TODO(2.0) After converting all resources to use the runtime-controller client,\n// the genclient and k8s:deepcopy markers will no longer be needed and should be removed.\n// +genclient\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:root=true\n// +kubebuilder:resource:shortName=ssr\n// +kubebuilder:object:generate=true\n// +kubebuilder:storageversion\n\n// ServerStatusRequest is a request to access current status information about\n// the Velero server.\ntype ServerStatusRequest struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ObjectMeta `json:\"metadata,omitempty\"`\n\n\t// +optional\n\tSpec ServerStatusRequestSpec `json:\"spec,omitempty\"`\n\n\t// +optional\n\tStatus ServerStatusRequestStatus `json:\"status,omitempty\"`\n}\n\n// ServerStatusRequestSpec is the specification for a ServerStatusRequest.\ntype ServerStatusRequestSpec struct {\n}\n\n// ServerStatusRequestPhase represents the lifecycle phase of a ServerStatusRequest.\n// +kubebuilder:validation:Enum=New;Processed\ntype ServerStatusRequestPhase string\n\nconst (\n\t// ServerStatusRequestPhaseNew means the ServerStatusRequest has not been processed yet.\n\tServerStatusRequestPhaseNew ServerStatusRequestPhase = \"New\"\n\t// ServerStatusRequestPhaseProcessed means the ServerStatusRequest has been processed.\n\tServerStatusRequestPhaseProcessed ServerStatusRequestPhase = \"Processed\"\n)\n\n// PluginInfo contains attributes of a Velero plugin\ntype PluginInfo struct {\n\tName string `json:\"name\"`\n\tKind string `json:\"kind\"`\n}\n\n// ServerStatusRequestStatus is the current status of a ServerStatusRequest.\ntype ServerStatusRequestStatus struct {\n\t// Phase is the current lifecycle phase of the ServerStatusRequest.\n\t// +optional\n\tPhase ServerStatusRequestPhase `json:\"phase,omitempty\"`\n\n\t// ProcessedTimestamp is when the ServerStatusRequest was processed\n\t// by the ServerStatusRequestController.\n\t// +optional\n\t// +nullable\n\tProcessedTimestamp *metav1.Time `json:\"processedTimestamp,omitempty\"`\n\n\t// ServerVersion is the Velero server version.\n\t// +optional\n\tServerVersion string `json:\"serverVersion,omitempty\"`\n\n\t// Plugins list information about the plugins running on the Velero server\n\t// +optional\n\t// +nullable\n\tPlugins []PluginInfo `json:\"plugins,omitempty\"`\n}\n\n// TODO(2.0) After converting all resources to use the runtime-controller client,\n// the k8s:deepcopy marker will no longer be needed and should be removed.\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:root=true\n// +kubebuilder:rbac:groups=velero.io,resources=serverstatusrequests,verbs=get;list;watch;create;update;patch;delete\n// +kubebuilder:rbac:groups=velero.io,resources=serverstatusrequests/status,verbs=get;update;patch\n\n// ServerStatusRequestList is a list of ServerStatusRequests.\ntype ServerStatusRequestList struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ListMeta `json:\"metadata,omitempty\"`\n\n\tItems []ServerStatusRequest `json:\"items\"`\n}\n" }, { "path": "pkg/apis/velero/v1/volume_snapshot_location_type.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// +genclient\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:root=true\n// +kubebuilder:resource:shortName=vsl\n// +kubebuilder:object:generate=true\n// +kubebuilder:storageversion\n\n// VolumeSnapshotLocation is a location where Velero stores volume snapshots.\ntype VolumeSnapshotLocation struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ObjectMeta `json:\"metadata,omitempty\"`\n\n\t// +optional\n\tSpec VolumeSnapshotLocationSpec `json:\"spec,omitempty\"`\n\n\t// +optional\n\tStatus VolumeSnapshotLocationStatus `json:\"status,omitempty\"`\n}\n\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:root=true\n// +kubebuilder:rbac:groups=velero.io,resources=volumesnapshotlocations,verbs=get;list;watch;create;update;patch;delete\n\n// VolumeSnapshotLocationList is a list of VolumeSnapshotLocations.\ntype VolumeSnapshotLocationList struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ListMeta `json:\"metadata,omitempty\"`\n\n\tItems []VolumeSnapshotLocation `json:\"items\"`\n}\n\n// VolumeSnapshotLocationSpec defines the specification for a Velero VolumeSnapshotLocation.\ntype VolumeSnapshotLocationSpec struct {\n\t// Provider is the provider of the volume storage.\n\tProvider string `json:\"provider\"`\n\n\t// Config is for provider-specific configuration fields.\n\t// +optional\n\tConfig map[string]string `json:\"config,omitempty\"`\n\n\t// Credential contains the credential information intended to be used with this location\n\t// +optional\n\tCredential *corev1api.SecretKeySelector `json:\"credential,omitempty\"`\n}\n\n// VolumeSnapshotLocationPhase is the lifecycle phase of a Velero VolumeSnapshotLocation.\n// +kubebuilder:validation:Enum=Available;Unavailable\ntype VolumeSnapshotLocationPhase string\n\nconst (\n\t// VolumeSnapshotLocationPhaseAvailable means the location is available to read and write from.\n\tVolumeSnapshotLocationPhaseAvailable VolumeSnapshotLocationPhase = \"Available\"\n\n\t// VolumeSnapshotLocationPhaseUnavailable means the location is unavailable to read and write from.\n\tVolumeSnapshotLocationPhaseUnavailable VolumeSnapshotLocationPhase = \"Unavailable\"\n)\n\n// VolumeSnapshotLocationStatus describes the current status of a Velero VolumeSnapshotLocation.\ntype VolumeSnapshotLocationStatus struct {\n\t// +optional\n\tPhase VolumeSnapshotLocationPhase `json:\"phase,omitempty\"`\n}\n" }, { "path": "pkg/apis/velero/v1/zz_generated.deepcopy.go", "content": "//go:build !ignore_autogenerated\n\n// Code generated by controller-gen. DO NOT EDIT.\n\npackage v1\n\nimport (\n\tcorev1 \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n)\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *Backup) DeepCopyInto(out *Backup) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ObjectMeta.DeepCopyInto(&out.ObjectMeta)\n\tin.Spec.DeepCopyInto(&out.Spec)\n\tin.Status.DeepCopyInto(&out.Status)\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Backup.\nfunc (in *Backup) DeepCopy() *Backup {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(Backup)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *Backup) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *BackupHooks) DeepCopyInto(out *BackupHooks) {\n\t*out = *in\n\tif in.Resources != nil {\n\t\tin, out := &in.Resources, &out.Resources\n\t\t*out = make([]BackupResourceHookSpec, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BackupHooks.\nfunc (in *BackupHooks) DeepCopy() *BackupHooks {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(BackupHooks)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *BackupList) DeepCopyInto(out *BackupList) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ListMeta.DeepCopyInto(&out.ListMeta)\n\tif in.Items != nil {\n\t\tin, out := &in.Items, &out.Items\n\t\t*out = make([]Backup, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BackupList.\nfunc (in *BackupList) DeepCopy() *BackupList {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(BackupList)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *BackupList) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *BackupProgress) DeepCopyInto(out *BackupProgress) {\n\t*out = *in\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BackupProgress.\nfunc (in *BackupProgress) DeepCopy() *BackupProgress {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(BackupProgress)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *BackupRepository) DeepCopyInto(out *BackupRepository) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ObjectMeta.DeepCopyInto(&out.ObjectMeta)\n\tin.Spec.DeepCopyInto(&out.Spec)\n\tin.Status.DeepCopyInto(&out.Status)\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BackupRepository.\nfunc (in *BackupRepository) DeepCopy() *BackupRepository {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(BackupRepository)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *BackupRepository) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *BackupRepositoryList) DeepCopyInto(out *BackupRepositoryList) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ListMeta.DeepCopyInto(&out.ListMeta)\n\tif in.Items != nil {\n\t\tin, out := &in.Items, &out.Items\n\t\t*out = make([]BackupRepository, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BackupRepositoryList.\nfunc (in *BackupRepositoryList) DeepCopy() *BackupRepositoryList {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(BackupRepositoryList)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *BackupRepositoryList) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *BackupRepositoryMaintenanceStatus) DeepCopyInto(out *BackupRepositoryMaintenanceStatus) {\n\t*out = *in\n\tif in.StartTimestamp != nil {\n\t\tin, out := &in.StartTimestamp, &out.StartTimestamp\n\t\t*out = (*in).DeepCopy()\n\t}\n\tif in.CompleteTimestamp != nil {\n\t\tin, out := &in.CompleteTimestamp, &out.CompleteTimestamp\n\t\t*out = (*in).DeepCopy()\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BackupRepositoryMaintenanceStatus.\nfunc (in *BackupRepositoryMaintenanceStatus) DeepCopy() *BackupRepositoryMaintenanceStatus {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(BackupRepositoryMaintenanceStatus)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *BackupRepositorySpec) DeepCopyInto(out *BackupRepositorySpec) {\n\t*out = *in\n\tout.MaintenanceFrequency = in.MaintenanceFrequency\n\tif in.RepositoryConfig != nil {\n\t\tin, out := &in.RepositoryConfig, &out.RepositoryConfig\n\t\t*out = make(map[string]string, len(*in))\n\t\tfor key, val := range *in {\n\t\t\t(*out)[key] = val\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BackupRepositorySpec.\nfunc (in *BackupRepositorySpec) DeepCopy() *BackupRepositorySpec {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(BackupRepositorySpec)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *BackupRepositoryStatus) DeepCopyInto(out *BackupRepositoryStatus) {\n\t*out = *in\n\tif in.LastMaintenanceTime != nil {\n\t\tin, out := &in.LastMaintenanceTime, &out.LastMaintenanceTime\n\t\t*out = (*in).DeepCopy()\n\t}\n\tif in.RecentMaintenance != nil {\n\t\tin, out := &in.RecentMaintenance, &out.RecentMaintenance\n\t\t*out = make([]BackupRepositoryMaintenanceStatus, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BackupRepositoryStatus.\nfunc (in *BackupRepositoryStatus) DeepCopy() *BackupRepositoryStatus {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(BackupRepositoryStatus)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *BackupResourceHook) DeepCopyInto(out *BackupResourceHook) {\n\t*out = *in\n\tif in.Exec != nil {\n\t\tin, out := &in.Exec, &out.Exec\n\t\t*out = new(ExecHook)\n\t\t(*in).DeepCopyInto(*out)\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BackupResourceHook.\nfunc (in *BackupResourceHook) DeepCopy() *BackupResourceHook {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(BackupResourceHook)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *BackupResourceHookSpec) DeepCopyInto(out *BackupResourceHookSpec) {\n\t*out = *in\n\tif in.IncludedNamespaces != nil {\n\t\tin, out := &in.IncludedNamespaces, &out.IncludedNamespaces\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.ExcludedNamespaces != nil {\n\t\tin, out := &in.ExcludedNamespaces, &out.ExcludedNamespaces\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.IncludedResources != nil {\n\t\tin, out := &in.IncludedResources, &out.IncludedResources\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.ExcludedResources != nil {\n\t\tin, out := &in.ExcludedResources, &out.ExcludedResources\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.LabelSelector != nil {\n\t\tin, out := &in.LabelSelector, &out.LabelSelector\n\t\t*out = new(metav1.LabelSelector)\n\t\t(*in).DeepCopyInto(*out)\n\t}\n\tif in.PreHooks != nil {\n\t\tin, out := &in.PreHooks, &out.PreHooks\n\t\t*out = make([]BackupResourceHook, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n\tif in.PostHooks != nil {\n\t\tin, out := &in.PostHooks, &out.PostHooks\n\t\t*out = make([]BackupResourceHook, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BackupResourceHookSpec.\nfunc (in *BackupResourceHookSpec) DeepCopy() *BackupResourceHookSpec {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(BackupResourceHookSpec)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *BackupSpec) DeepCopyInto(out *BackupSpec) {\n\t*out = *in\n\tin.Metadata.DeepCopyInto(&out.Metadata)\n\tif in.IncludedNamespaces != nil {\n\t\tin, out := &in.IncludedNamespaces, &out.IncludedNamespaces\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.ExcludedNamespaces != nil {\n\t\tin, out := &in.ExcludedNamespaces, &out.ExcludedNamespaces\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.IncludedResources != nil {\n\t\tin, out := &in.IncludedResources, &out.IncludedResources\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.ExcludedResources != nil {\n\t\tin, out := &in.ExcludedResources, &out.ExcludedResources\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.IncludedClusterScopedResources != nil {\n\t\tin, out := &in.IncludedClusterScopedResources, &out.IncludedClusterScopedResources\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.ExcludedClusterScopedResources != nil {\n\t\tin, out := &in.ExcludedClusterScopedResources, &out.ExcludedClusterScopedResources\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.IncludedNamespaceScopedResources != nil {\n\t\tin, out := &in.IncludedNamespaceScopedResources, &out.IncludedNamespaceScopedResources\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.ExcludedNamespaceScopedResources != nil {\n\t\tin, out := &in.ExcludedNamespaceScopedResources, &out.ExcludedNamespaceScopedResources\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.LabelSelector != nil {\n\t\tin, out := &in.LabelSelector, &out.LabelSelector\n\t\t*out = new(metav1.LabelSelector)\n\t\t(*in).DeepCopyInto(*out)\n\t}\n\tif in.OrLabelSelectors != nil {\n\t\tin, out := &in.OrLabelSelectors, &out.OrLabelSelectors\n\t\t*out = make([]*metav1.LabelSelector, len(*in))\n\t\tfor i := range *in {\n\t\t\tif (*in)[i] != nil {\n\t\t\t\tin, out := &(*in)[i], &(*out)[i]\n\t\t\t\t*out = new(metav1.LabelSelector)\n\t\t\t\t(*in).DeepCopyInto(*out)\n\t\t\t}\n\t\t}\n\t}\n\tif in.SnapshotVolumes != nil {\n\t\tin, out := &in.SnapshotVolumes, &out.SnapshotVolumes\n\t\t*out = new(bool)\n\t\t**out = **in\n\t}\n\tout.TTL = in.TTL\n\tif in.IncludeClusterResources != nil {\n\t\tin, out := &in.IncludeClusterResources, &out.IncludeClusterResources\n\t\t*out = new(bool)\n\t\t**out = **in\n\t}\n\tin.Hooks.DeepCopyInto(&out.Hooks)\n\tif in.VolumeSnapshotLocations != nil {\n\t\tin, out := &in.VolumeSnapshotLocations, &out.VolumeSnapshotLocations\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.DefaultVolumesToRestic != nil {\n\t\tin, out := &in.DefaultVolumesToRestic, &out.DefaultVolumesToRestic\n\t\t*out = new(bool)\n\t\t**out = **in\n\t}\n\tif in.DefaultVolumesToFsBackup != nil {\n\t\tin, out := &in.DefaultVolumesToFsBackup, &out.DefaultVolumesToFsBackup\n\t\t*out = new(bool)\n\t\t**out = **in\n\t}\n\tif in.OrderedResources != nil {\n\t\tin, out := &in.OrderedResources, &out.OrderedResources\n\t\t*out = make(map[string]string, len(*in))\n\t\tfor key, val := range *in {\n\t\t\t(*out)[key] = val\n\t\t}\n\t}\n\tout.CSISnapshotTimeout = in.CSISnapshotTimeout\n\tout.ItemOperationTimeout = in.ItemOperationTimeout\n\tif in.ResourcePolicy != nil {\n\t\tin, out := &in.ResourcePolicy, &out.ResourcePolicy\n\t\t*out = new(corev1.TypedLocalObjectReference)\n\t\t(*in).DeepCopyInto(*out)\n\t}\n\tif in.SnapshotMoveData != nil {\n\t\tin, out := &in.SnapshotMoveData, &out.SnapshotMoveData\n\t\t*out = new(bool)\n\t\t**out = **in\n\t}\n\tif in.UploaderConfig != nil {\n\t\tin, out := &in.UploaderConfig, &out.UploaderConfig\n\t\t*out = new(UploaderConfigForBackup)\n\t\t**out = **in\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BackupSpec.\nfunc (in *BackupSpec) DeepCopy() *BackupSpec {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(BackupSpec)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *BackupStatus) DeepCopyInto(out *BackupStatus) {\n\t*out = *in\n\tif in.Expiration != nil {\n\t\tin, out := &in.Expiration, &out.Expiration\n\t\t*out = (*in).DeepCopy()\n\t}\n\tif in.ValidationErrors != nil {\n\t\tin, out := &in.ValidationErrors, &out.ValidationErrors\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.StartTimestamp != nil {\n\t\tin, out := &in.StartTimestamp, &out.StartTimestamp\n\t\t*out = (*in).DeepCopy()\n\t}\n\tif in.CompletionTimestamp != nil {\n\t\tin, out := &in.CompletionTimestamp, &out.CompletionTimestamp\n\t\t*out = (*in).DeepCopy()\n\t}\n\tif in.Progress != nil {\n\t\tin, out := &in.Progress, &out.Progress\n\t\t*out = new(BackupProgress)\n\t\t**out = **in\n\t}\n\tif in.HookStatus != nil {\n\t\tin, out := &in.HookStatus, &out.HookStatus\n\t\t*out = new(HookStatus)\n\t\t**out = **in\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BackupStatus.\nfunc (in *BackupStatus) DeepCopy() *BackupStatus {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(BackupStatus)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *BackupStorageLocation) DeepCopyInto(out *BackupStorageLocation) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ObjectMeta.DeepCopyInto(&out.ObjectMeta)\n\tin.Spec.DeepCopyInto(&out.Spec)\n\tin.Status.DeepCopyInto(&out.Status)\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BackupStorageLocation.\nfunc (in *BackupStorageLocation) DeepCopy() *BackupStorageLocation {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(BackupStorageLocation)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *BackupStorageLocation) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *BackupStorageLocationList) DeepCopyInto(out *BackupStorageLocationList) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ListMeta.DeepCopyInto(&out.ListMeta)\n\tif in.Items != nil {\n\t\tin, out := &in.Items, &out.Items\n\t\t*out = make([]BackupStorageLocation, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BackupStorageLocationList.\nfunc (in *BackupStorageLocationList) DeepCopy() *BackupStorageLocationList {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(BackupStorageLocationList)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *BackupStorageLocationList) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *BackupStorageLocationSpec) DeepCopyInto(out *BackupStorageLocationSpec) {\n\t*out = *in\n\tif in.Config != nil {\n\t\tin, out := &in.Config, &out.Config\n\t\t*out = make(map[string]string, len(*in))\n\t\tfor key, val := range *in {\n\t\t\t(*out)[key] = val\n\t\t}\n\t}\n\tif in.Credential != nil {\n\t\tin, out := &in.Credential, &out.Credential\n\t\t*out = new(corev1.SecretKeySelector)\n\t\t(*in).DeepCopyInto(*out)\n\t}\n\tin.StorageType.DeepCopyInto(&out.StorageType)\n\tif in.BackupSyncPeriod != nil {\n\t\tin, out := &in.BackupSyncPeriod, &out.BackupSyncPeriod\n\t\t*out = new(metav1.Duration)\n\t\t**out = **in\n\t}\n\tif in.ValidationFrequency != nil {\n\t\tin, out := &in.ValidationFrequency, &out.ValidationFrequency\n\t\t*out = new(metav1.Duration)\n\t\t**out = **in\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BackupStorageLocationSpec.\nfunc (in *BackupStorageLocationSpec) DeepCopy() *BackupStorageLocationSpec {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(BackupStorageLocationSpec)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *BackupStorageLocationStatus) DeepCopyInto(out *BackupStorageLocationStatus) {\n\t*out = *in\n\tif in.LastSyncedTime != nil {\n\t\tin, out := &in.LastSyncedTime, &out.LastSyncedTime\n\t\t*out = (*in).DeepCopy()\n\t}\n\tif in.LastValidationTime != nil {\n\t\tin, out := &in.LastValidationTime, &out.LastValidationTime\n\t\t*out = (*in).DeepCopy()\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BackupStorageLocationStatus.\nfunc (in *BackupStorageLocationStatus) DeepCopy() *BackupStorageLocationStatus {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(BackupStorageLocationStatus)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *DeleteBackupRequest) DeepCopyInto(out *DeleteBackupRequest) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ObjectMeta.DeepCopyInto(&out.ObjectMeta)\n\tout.Spec = in.Spec\n\tin.Status.DeepCopyInto(&out.Status)\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DeleteBackupRequest.\nfunc (in *DeleteBackupRequest) DeepCopy() *DeleteBackupRequest {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(DeleteBackupRequest)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *DeleteBackupRequest) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *DeleteBackupRequestList) DeepCopyInto(out *DeleteBackupRequestList) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ListMeta.DeepCopyInto(&out.ListMeta)\n\tif in.Items != nil {\n\t\tin, out := &in.Items, &out.Items\n\t\t*out = make([]DeleteBackupRequest, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DeleteBackupRequestList.\nfunc (in *DeleteBackupRequestList) DeepCopy() *DeleteBackupRequestList {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(DeleteBackupRequestList)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *DeleteBackupRequestList) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *DeleteBackupRequestSpec) DeepCopyInto(out *DeleteBackupRequestSpec) {\n\t*out = *in\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DeleteBackupRequestSpec.\nfunc (in *DeleteBackupRequestSpec) DeepCopy() *DeleteBackupRequestSpec {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(DeleteBackupRequestSpec)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *DeleteBackupRequestStatus) DeepCopyInto(out *DeleteBackupRequestStatus) {\n\t*out = *in\n\tif in.Errors != nil {\n\t\tin, out := &in.Errors, &out.Errors\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DeleteBackupRequestStatus.\nfunc (in *DeleteBackupRequestStatus) DeepCopy() *DeleteBackupRequestStatus {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(DeleteBackupRequestStatus)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *DownloadRequest) DeepCopyInto(out *DownloadRequest) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ObjectMeta.DeepCopyInto(&out.ObjectMeta)\n\tout.Spec = in.Spec\n\tin.Status.DeepCopyInto(&out.Status)\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DownloadRequest.\nfunc (in *DownloadRequest) DeepCopy() *DownloadRequest {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(DownloadRequest)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *DownloadRequest) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *DownloadRequestList) DeepCopyInto(out *DownloadRequestList) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ListMeta.DeepCopyInto(&out.ListMeta)\n\tif in.Items != nil {\n\t\tin, out := &in.Items, &out.Items\n\t\t*out = make([]DownloadRequest, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DownloadRequestList.\nfunc (in *DownloadRequestList) DeepCopy() *DownloadRequestList {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(DownloadRequestList)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *DownloadRequestList) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *DownloadRequestSpec) DeepCopyInto(out *DownloadRequestSpec) {\n\t*out = *in\n\tout.Target = in.Target\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DownloadRequestSpec.\nfunc (in *DownloadRequestSpec) DeepCopy() *DownloadRequestSpec {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(DownloadRequestSpec)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *DownloadRequestStatus) DeepCopyInto(out *DownloadRequestStatus) {\n\t*out = *in\n\tif in.Expiration != nil {\n\t\tin, out := &in.Expiration, &out.Expiration\n\t\t*out = (*in).DeepCopy()\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DownloadRequestStatus.\nfunc (in *DownloadRequestStatus) DeepCopy() *DownloadRequestStatus {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(DownloadRequestStatus)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *DownloadTarget) DeepCopyInto(out *DownloadTarget) {\n\t*out = *in\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DownloadTarget.\nfunc (in *DownloadTarget) DeepCopy() *DownloadTarget {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(DownloadTarget)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *ExecHook) DeepCopyInto(out *ExecHook) {\n\t*out = *in\n\tif in.Command != nil {\n\t\tin, out := &in.Command, &out.Command\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tout.Timeout = in.Timeout\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ExecHook.\nfunc (in *ExecHook) DeepCopy() *ExecHook {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(ExecHook)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *ExecRestoreHook) DeepCopyInto(out *ExecRestoreHook) {\n\t*out = *in\n\tif in.Command != nil {\n\t\tin, out := &in.Command, &out.Command\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tout.ExecTimeout = in.ExecTimeout\n\tout.WaitTimeout = in.WaitTimeout\n\tif in.WaitForReady != nil {\n\t\tin, out := &in.WaitForReady, &out.WaitForReady\n\t\t*out = new(bool)\n\t\t**out = **in\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ExecRestoreHook.\nfunc (in *ExecRestoreHook) DeepCopy() *ExecRestoreHook {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(ExecRestoreHook)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *HookStatus) DeepCopyInto(out *HookStatus) {\n\t*out = *in\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new HookStatus.\nfunc (in *HookStatus) DeepCopy() *HookStatus {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(HookStatus)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *InitRestoreHook) DeepCopyInto(out *InitRestoreHook) {\n\t*out = *in\n\tif in.InitContainers != nil {\n\t\tin, out := &in.InitContainers, &out.InitContainers\n\t\t*out = make([]runtime.RawExtension, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n\tout.Timeout = in.Timeout\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new InitRestoreHook.\nfunc (in *InitRestoreHook) DeepCopy() *InitRestoreHook {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(InitRestoreHook)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *Metadata) DeepCopyInto(out *Metadata) {\n\t*out = *in\n\tif in.Labels != nil {\n\t\tin, out := &in.Labels, &out.Labels\n\t\t*out = make(map[string]string, len(*in))\n\t\tfor key, val := range *in {\n\t\t\t(*out)[key] = val\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Metadata.\nfunc (in *Metadata) DeepCopy() *Metadata {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(Metadata)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *ObjectStorageLocation) DeepCopyInto(out *ObjectStorageLocation) {\n\t*out = *in\n\tif in.CACert != nil {\n\t\tin, out := &in.CACert, &out.CACert\n\t\t*out = make([]byte, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.CACertRef != nil {\n\t\tin, out := &in.CACertRef, &out.CACertRef\n\t\t*out = new(corev1.SecretKeySelector)\n\t\t(*in).DeepCopyInto(*out)\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ObjectStorageLocation.\nfunc (in *ObjectStorageLocation) DeepCopy() *ObjectStorageLocation {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(ObjectStorageLocation)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *PluginInfo) DeepCopyInto(out *PluginInfo) {\n\t*out = *in\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PluginInfo.\nfunc (in *PluginInfo) DeepCopy() *PluginInfo {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(PluginInfo)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *PodVolumeBackup) DeepCopyInto(out *PodVolumeBackup) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ObjectMeta.DeepCopyInto(&out.ObjectMeta)\n\tin.Spec.DeepCopyInto(&out.Spec)\n\tin.Status.DeepCopyInto(&out.Status)\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PodVolumeBackup.\nfunc (in *PodVolumeBackup) DeepCopy() *PodVolumeBackup {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(PodVolumeBackup)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *PodVolumeBackup) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *PodVolumeBackupList) DeepCopyInto(out *PodVolumeBackupList) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ListMeta.DeepCopyInto(&out.ListMeta)\n\tif in.Items != nil {\n\t\tin, out := &in.Items, &out.Items\n\t\t*out = make([]PodVolumeBackup, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PodVolumeBackupList.\nfunc (in *PodVolumeBackupList) DeepCopy() *PodVolumeBackupList {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(PodVolumeBackupList)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *PodVolumeBackupList) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *PodVolumeBackupSpec) DeepCopyInto(out *PodVolumeBackupSpec) {\n\t*out = *in\n\tout.Pod = in.Pod\n\tif in.Tags != nil {\n\t\tin, out := &in.Tags, &out.Tags\n\t\t*out = make(map[string]string, len(*in))\n\t\tfor key, val := range *in {\n\t\t\t(*out)[key] = val\n\t\t}\n\t}\n\tif in.UploaderSettings != nil {\n\t\tin, out := &in.UploaderSettings, &out.UploaderSettings\n\t\t*out = make(map[string]string, len(*in))\n\t\tfor key, val := range *in {\n\t\t\t(*out)[key] = val\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PodVolumeBackupSpec.\nfunc (in *PodVolumeBackupSpec) DeepCopy() *PodVolumeBackupSpec {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(PodVolumeBackupSpec)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *PodVolumeBackupStatus) DeepCopyInto(out *PodVolumeBackupStatus) {\n\t*out = *in\n\tif in.StartTimestamp != nil {\n\t\tin, out := &in.StartTimestamp, &out.StartTimestamp\n\t\t*out = (*in).DeepCopy()\n\t}\n\tif in.CompletionTimestamp != nil {\n\t\tin, out := &in.CompletionTimestamp, &out.CompletionTimestamp\n\t\t*out = (*in).DeepCopy()\n\t}\n\tout.Progress = in.Progress\n\tif in.AcceptedTimestamp != nil {\n\t\tin, out := &in.AcceptedTimestamp, &out.AcceptedTimestamp\n\t\t*out = (*in).DeepCopy()\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PodVolumeBackupStatus.\nfunc (in *PodVolumeBackupStatus) DeepCopy() *PodVolumeBackupStatus {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(PodVolumeBackupStatus)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *PodVolumeRestore) DeepCopyInto(out *PodVolumeRestore) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ObjectMeta.DeepCopyInto(&out.ObjectMeta)\n\tin.Spec.DeepCopyInto(&out.Spec)\n\tin.Status.DeepCopyInto(&out.Status)\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PodVolumeRestore.\nfunc (in *PodVolumeRestore) DeepCopy() *PodVolumeRestore {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(PodVolumeRestore)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *PodVolumeRestore) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *PodVolumeRestoreList) DeepCopyInto(out *PodVolumeRestoreList) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ListMeta.DeepCopyInto(&out.ListMeta)\n\tif in.Items != nil {\n\t\tin, out := &in.Items, &out.Items\n\t\t*out = make([]PodVolumeRestore, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PodVolumeRestoreList.\nfunc (in *PodVolumeRestoreList) DeepCopy() *PodVolumeRestoreList {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(PodVolumeRestoreList)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *PodVolumeRestoreList) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *PodVolumeRestoreSpec) DeepCopyInto(out *PodVolumeRestoreSpec) {\n\t*out = *in\n\tout.Pod = in.Pod\n\tif in.UploaderSettings != nil {\n\t\tin, out := &in.UploaderSettings, &out.UploaderSettings\n\t\t*out = make(map[string]string, len(*in))\n\t\tfor key, val := range *in {\n\t\t\t(*out)[key] = val\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PodVolumeRestoreSpec.\nfunc (in *PodVolumeRestoreSpec) DeepCopy() *PodVolumeRestoreSpec {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(PodVolumeRestoreSpec)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *PodVolumeRestoreStatus) DeepCopyInto(out *PodVolumeRestoreStatus) {\n\t*out = *in\n\tif in.StartTimestamp != nil {\n\t\tin, out := &in.StartTimestamp, &out.StartTimestamp\n\t\t*out = (*in).DeepCopy()\n\t}\n\tif in.CompletionTimestamp != nil {\n\t\tin, out := &in.CompletionTimestamp, &out.CompletionTimestamp\n\t\t*out = (*in).DeepCopy()\n\t}\n\tout.Progress = in.Progress\n\tif in.AcceptedTimestamp != nil {\n\t\tin, out := &in.AcceptedTimestamp, &out.AcceptedTimestamp\n\t\t*out = (*in).DeepCopy()\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PodVolumeRestoreStatus.\nfunc (in *PodVolumeRestoreStatus) DeepCopy() *PodVolumeRestoreStatus {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(PodVolumeRestoreStatus)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *Restore) DeepCopyInto(out *Restore) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ObjectMeta.DeepCopyInto(&out.ObjectMeta)\n\tin.Spec.DeepCopyInto(&out.Spec)\n\tin.Status.DeepCopyInto(&out.Status)\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Restore.\nfunc (in *Restore) DeepCopy() *Restore {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(Restore)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *Restore) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *RestoreHooks) DeepCopyInto(out *RestoreHooks) {\n\t*out = *in\n\tif in.Resources != nil {\n\t\tin, out := &in.Resources, &out.Resources\n\t\t*out = make([]RestoreResourceHookSpec, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RestoreHooks.\nfunc (in *RestoreHooks) DeepCopy() *RestoreHooks {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(RestoreHooks)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *RestoreList) DeepCopyInto(out *RestoreList) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ListMeta.DeepCopyInto(&out.ListMeta)\n\tif in.Items != nil {\n\t\tin, out := &in.Items, &out.Items\n\t\t*out = make([]Restore, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RestoreList.\nfunc (in *RestoreList) DeepCopy() *RestoreList {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(RestoreList)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *RestoreList) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *RestoreProgress) DeepCopyInto(out *RestoreProgress) {\n\t*out = *in\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RestoreProgress.\nfunc (in *RestoreProgress) DeepCopy() *RestoreProgress {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(RestoreProgress)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *RestoreResourceHook) DeepCopyInto(out *RestoreResourceHook) {\n\t*out = *in\n\tif in.Exec != nil {\n\t\tin, out := &in.Exec, &out.Exec\n\t\t*out = new(ExecRestoreHook)\n\t\t(*in).DeepCopyInto(*out)\n\t}\n\tif in.Init != nil {\n\t\tin, out := &in.Init, &out.Init\n\t\t*out = new(InitRestoreHook)\n\t\t(*in).DeepCopyInto(*out)\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RestoreResourceHook.\nfunc (in *RestoreResourceHook) DeepCopy() *RestoreResourceHook {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(RestoreResourceHook)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *RestoreResourceHookSpec) DeepCopyInto(out *RestoreResourceHookSpec) {\n\t*out = *in\n\tif in.IncludedNamespaces != nil {\n\t\tin, out := &in.IncludedNamespaces, &out.IncludedNamespaces\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.ExcludedNamespaces != nil {\n\t\tin, out := &in.ExcludedNamespaces, &out.ExcludedNamespaces\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.IncludedResources != nil {\n\t\tin, out := &in.IncludedResources, &out.IncludedResources\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.ExcludedResources != nil {\n\t\tin, out := &in.ExcludedResources, &out.ExcludedResources\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.LabelSelector != nil {\n\t\tin, out := &in.LabelSelector, &out.LabelSelector\n\t\t*out = new(metav1.LabelSelector)\n\t\t(*in).DeepCopyInto(*out)\n\t}\n\tif in.PostHooks != nil {\n\t\tin, out := &in.PostHooks, &out.PostHooks\n\t\t*out = make([]RestoreResourceHook, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RestoreResourceHookSpec.\nfunc (in *RestoreResourceHookSpec) DeepCopy() *RestoreResourceHookSpec {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(RestoreResourceHookSpec)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *RestoreSpec) DeepCopyInto(out *RestoreSpec) {\n\t*out = *in\n\tif in.IncludedNamespaces != nil {\n\t\tin, out := &in.IncludedNamespaces, &out.IncludedNamespaces\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.ExcludedNamespaces != nil {\n\t\tin, out := &in.ExcludedNamespaces, &out.ExcludedNamespaces\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.IncludedResources != nil {\n\t\tin, out := &in.IncludedResources, &out.IncludedResources\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.ExcludedResources != nil {\n\t\tin, out := &in.ExcludedResources, &out.ExcludedResources\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.NamespaceMapping != nil {\n\t\tin, out := &in.NamespaceMapping, &out.NamespaceMapping\n\t\t*out = make(map[string]string, len(*in))\n\t\tfor key, val := range *in {\n\t\t\t(*out)[key] = val\n\t\t}\n\t}\n\tif in.LabelSelector != nil {\n\t\tin, out := &in.LabelSelector, &out.LabelSelector\n\t\t*out = new(metav1.LabelSelector)\n\t\t(*in).DeepCopyInto(*out)\n\t}\n\tif in.OrLabelSelectors != nil {\n\t\tin, out := &in.OrLabelSelectors, &out.OrLabelSelectors\n\t\t*out = make([]*metav1.LabelSelector, len(*in))\n\t\tfor i := range *in {\n\t\t\tif (*in)[i] != nil {\n\t\t\t\tin, out := &(*in)[i], &(*out)[i]\n\t\t\t\t*out = new(metav1.LabelSelector)\n\t\t\t\t(*in).DeepCopyInto(*out)\n\t\t\t}\n\t\t}\n\t}\n\tif in.RestorePVs != nil {\n\t\tin, out := &in.RestorePVs, &out.RestorePVs\n\t\t*out = new(bool)\n\t\t**out = **in\n\t}\n\tif in.RestoreStatus != nil {\n\t\tin, out := &in.RestoreStatus, &out.RestoreStatus\n\t\t*out = new(RestoreStatusSpec)\n\t\t(*in).DeepCopyInto(*out)\n\t}\n\tif in.PreserveNodePorts != nil {\n\t\tin, out := &in.PreserveNodePorts, &out.PreserveNodePorts\n\t\t*out = new(bool)\n\t\t**out = **in\n\t}\n\tif in.IncludeClusterResources != nil {\n\t\tin, out := &in.IncludeClusterResources, &out.IncludeClusterResources\n\t\t*out = new(bool)\n\t\t**out = **in\n\t}\n\tin.Hooks.DeepCopyInto(&out.Hooks)\n\tout.ItemOperationTimeout = in.ItemOperationTimeout\n\tif in.ResourceModifier != nil {\n\t\tin, out := &in.ResourceModifier, &out.ResourceModifier\n\t\t*out = new(corev1.TypedLocalObjectReference)\n\t\t(*in).DeepCopyInto(*out)\n\t}\n\tif in.UploaderConfig != nil {\n\t\tin, out := &in.UploaderConfig, &out.UploaderConfig\n\t\t*out = new(UploaderConfigForRestore)\n\t\t(*in).DeepCopyInto(*out)\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RestoreSpec.\nfunc (in *RestoreSpec) DeepCopy() *RestoreSpec {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(RestoreSpec)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *RestoreStatus) DeepCopyInto(out *RestoreStatus) {\n\t*out = *in\n\tif in.ValidationErrors != nil {\n\t\tin, out := &in.ValidationErrors, &out.ValidationErrors\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.StartTimestamp != nil {\n\t\tin, out := &in.StartTimestamp, &out.StartTimestamp\n\t\t*out = (*in).DeepCopy()\n\t}\n\tif in.CompletionTimestamp != nil {\n\t\tin, out := &in.CompletionTimestamp, &out.CompletionTimestamp\n\t\t*out = (*in).DeepCopy()\n\t}\n\tif in.Progress != nil {\n\t\tin, out := &in.Progress, &out.Progress\n\t\t*out = new(RestoreProgress)\n\t\t**out = **in\n\t}\n\tif in.HookStatus != nil {\n\t\tin, out := &in.HookStatus, &out.HookStatus\n\t\t*out = new(HookStatus)\n\t\t**out = **in\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RestoreStatus.\nfunc (in *RestoreStatus) DeepCopy() *RestoreStatus {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(RestoreStatus)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *RestoreStatusSpec) DeepCopyInto(out *RestoreStatusSpec) {\n\t*out = *in\n\tif in.IncludedResources != nil {\n\t\tin, out := &in.IncludedResources, &out.IncludedResources\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n\tif in.ExcludedResources != nil {\n\t\tin, out := &in.ExcludedResources, &out.ExcludedResources\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RestoreStatusSpec.\nfunc (in *RestoreStatusSpec) DeepCopy() *RestoreStatusSpec {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(RestoreStatusSpec)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *Schedule) DeepCopyInto(out *Schedule) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ObjectMeta.DeepCopyInto(&out.ObjectMeta)\n\tin.Spec.DeepCopyInto(&out.Spec)\n\tin.Status.DeepCopyInto(&out.Status)\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Schedule.\nfunc (in *Schedule) DeepCopy() *Schedule {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(Schedule)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *Schedule) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *ScheduleList) DeepCopyInto(out *ScheduleList) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ListMeta.DeepCopyInto(&out.ListMeta)\n\tif in.Items != nil {\n\t\tin, out := &in.Items, &out.Items\n\t\t*out = make([]Schedule, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ScheduleList.\nfunc (in *ScheduleList) DeepCopy() *ScheduleList {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(ScheduleList)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *ScheduleList) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *ScheduleSpec) DeepCopyInto(out *ScheduleSpec) {\n\t*out = *in\n\tin.Template.DeepCopyInto(&out.Template)\n\tif in.UseOwnerReferencesInBackup != nil {\n\t\tin, out := &in.UseOwnerReferencesInBackup, &out.UseOwnerReferencesInBackup\n\t\t*out = new(bool)\n\t\t**out = **in\n\t}\n\tif in.SkipImmediately != nil {\n\t\tin, out := &in.SkipImmediately, &out.SkipImmediately\n\t\t*out = new(bool)\n\t\t**out = **in\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ScheduleSpec.\nfunc (in *ScheduleSpec) DeepCopy() *ScheduleSpec {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(ScheduleSpec)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *ScheduleStatus) DeepCopyInto(out *ScheduleStatus) {\n\t*out = *in\n\tif in.LastBackup != nil {\n\t\tin, out := &in.LastBackup, &out.LastBackup\n\t\t*out = (*in).DeepCopy()\n\t}\n\tif in.LastSkipped != nil {\n\t\tin, out := &in.LastSkipped, &out.LastSkipped\n\t\t*out = (*in).DeepCopy()\n\t}\n\tif in.ValidationErrors != nil {\n\t\tin, out := &in.ValidationErrors, &out.ValidationErrors\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ScheduleStatus.\nfunc (in *ScheduleStatus) DeepCopy() *ScheduleStatus {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(ScheduleStatus)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *ServerStatusRequest) DeepCopyInto(out *ServerStatusRequest) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ObjectMeta.DeepCopyInto(&out.ObjectMeta)\n\tout.Spec = in.Spec\n\tin.Status.DeepCopyInto(&out.Status)\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ServerStatusRequest.\nfunc (in *ServerStatusRequest) DeepCopy() *ServerStatusRequest {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(ServerStatusRequest)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *ServerStatusRequest) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *ServerStatusRequestList) DeepCopyInto(out *ServerStatusRequestList) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ListMeta.DeepCopyInto(&out.ListMeta)\n\tif in.Items != nil {\n\t\tin, out := &in.Items, &out.Items\n\t\t*out = make([]ServerStatusRequest, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ServerStatusRequestList.\nfunc (in *ServerStatusRequestList) DeepCopy() *ServerStatusRequestList {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(ServerStatusRequestList)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *ServerStatusRequestList) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *ServerStatusRequestSpec) DeepCopyInto(out *ServerStatusRequestSpec) {\n\t*out = *in\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ServerStatusRequestSpec.\nfunc (in *ServerStatusRequestSpec) DeepCopy() *ServerStatusRequestSpec {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(ServerStatusRequestSpec)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *ServerStatusRequestStatus) DeepCopyInto(out *ServerStatusRequestStatus) {\n\t*out = *in\n\tif in.ProcessedTimestamp != nil {\n\t\tin, out := &in.ProcessedTimestamp, &out.ProcessedTimestamp\n\t\t*out = (*in).DeepCopy()\n\t}\n\tif in.Plugins != nil {\n\t\tin, out := &in.Plugins, &out.Plugins\n\t\t*out = make([]PluginInfo, len(*in))\n\t\tcopy(*out, *in)\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ServerStatusRequestStatus.\nfunc (in *ServerStatusRequestStatus) DeepCopy() *ServerStatusRequestStatus {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(ServerStatusRequestStatus)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *StorageType) DeepCopyInto(out *StorageType) {\n\t*out = *in\n\tif in.ObjectStorage != nil {\n\t\tin, out := &in.ObjectStorage, &out.ObjectStorage\n\t\t*out = new(ObjectStorageLocation)\n\t\t(*in).DeepCopyInto(*out)\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new StorageType.\nfunc (in *StorageType) DeepCopy() *StorageType {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(StorageType)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *UploaderConfigForBackup) DeepCopyInto(out *UploaderConfigForBackup) {\n\t*out = *in\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new UploaderConfigForBackup.\nfunc (in *UploaderConfigForBackup) DeepCopy() *UploaderConfigForBackup {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(UploaderConfigForBackup)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *UploaderConfigForRestore) DeepCopyInto(out *UploaderConfigForRestore) {\n\t*out = *in\n\tif in.WriteSparseFiles != nil {\n\t\tin, out := &in.WriteSparseFiles, &out.WriteSparseFiles\n\t\t*out = new(bool)\n\t\t**out = **in\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new UploaderConfigForRestore.\nfunc (in *UploaderConfigForRestore) DeepCopy() *UploaderConfigForRestore {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(UploaderConfigForRestore)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *VolumeSnapshotLocation) DeepCopyInto(out *VolumeSnapshotLocation) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ObjectMeta.DeepCopyInto(&out.ObjectMeta)\n\tin.Spec.DeepCopyInto(&out.Spec)\n\tout.Status = in.Status\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VolumeSnapshotLocation.\nfunc (in *VolumeSnapshotLocation) DeepCopy() *VolumeSnapshotLocation {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(VolumeSnapshotLocation)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *VolumeSnapshotLocation) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *VolumeSnapshotLocationList) DeepCopyInto(out *VolumeSnapshotLocationList) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ListMeta.DeepCopyInto(&out.ListMeta)\n\tif in.Items != nil {\n\t\tin, out := &in.Items, &out.Items\n\t\t*out = make([]VolumeSnapshotLocation, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VolumeSnapshotLocationList.\nfunc (in *VolumeSnapshotLocationList) DeepCopy() *VolumeSnapshotLocationList {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(VolumeSnapshotLocationList)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *VolumeSnapshotLocationList) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *VolumeSnapshotLocationSpec) DeepCopyInto(out *VolumeSnapshotLocationSpec) {\n\t*out = *in\n\tif in.Config != nil {\n\t\tin, out := &in.Config, &out.Config\n\t\t*out = make(map[string]string, len(*in))\n\t\tfor key, val := range *in {\n\t\t\t(*out)[key] = val\n\t\t}\n\t}\n\tif in.Credential != nil {\n\t\tin, out := &in.Credential, &out.Credential\n\t\t*out = new(corev1.SecretKeySelector)\n\t\t(*in).DeepCopyInto(*out)\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VolumeSnapshotLocationSpec.\nfunc (in *VolumeSnapshotLocationSpec) DeepCopy() *VolumeSnapshotLocationSpec {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(VolumeSnapshotLocationSpec)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *VolumeSnapshotLocationStatus) DeepCopyInto(out *VolumeSnapshotLocationStatus) {\n\t*out = *in\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VolumeSnapshotLocationStatus.\nfunc (in *VolumeSnapshotLocationStatus) DeepCopy() *VolumeSnapshotLocationStatus {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(VolumeSnapshotLocationStatus)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n" }, { "path": "pkg/apis/velero/v2alpha1/data_download_types.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v2alpha1\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/apis/velero/shared\"\n)\n\n// DataDownloadSpec is the specification for a DataDownload.\ntype DataDownloadSpec struct {\n\t// TargetVolume is the information of the target PVC and PV.\n\tTargetVolume TargetVolumeSpec `json:\"targetVolume\"`\n\n\t// BackupStorageLocation is the name of the backup storage location\n\t// where the backup repository is stored.\n\tBackupStorageLocation string `json:\"backupStorageLocation\"`\n\n\t// DataMover specifies the data mover to be used by the backup.\n\t// If DataMover is \"\" or \"velero\", the built-in data mover will be used.\n\t// +optional\n\tDataMover string `json:\"datamover,omitempty\"`\n\n\t// SnapshotID is the ID of the Velero backup snapshot to be restored from.\n\tSnapshotID string `json:\"snapshotID\"`\n\n\t// SourceNamespace is the original namespace where the volume is backed up from.\n\t// It may be different from SourcePVC's namespace if namespace is remapped during restore.\n\tSourceNamespace string `json:\"sourceNamespace\"`\n\n\t// DataMoverConfig is for data-mover-specific configuration fields.\n\t// +optional\n\tDataMoverConfig map[string]string `json:\"dataMoverConfig,omitempty\"`\n\n\t// Cancel indicates request to cancel the ongoing DataDownload. It can be set\n\t// when the DataDownload is in InProgress phase\n\tCancel bool `json:\"cancel,omitempty\"`\n\n\t// OperationTimeout specifies the time used to wait internal operations,\n\t// before returning error as timeout.\n\tOperationTimeout metav1.Duration `json:\"operationTimeout\"`\n\n\t// NodeOS is OS of the node where the DataDownload is processed.\n\t// +optional\n\tNodeOS NodeOS `json:\"nodeOS,omitempty\"`\n\n\t// SnapshotSize is the logical size in Bytes of the snapshot.\n\t// +optional\n\tSnapshotSize int64 `json:\"snapshotSize,omitempty\"`\n}\n\n// TargetVolumeSpec is the specification for a target PVC.\ntype TargetVolumeSpec struct {\n\t// PVC is the name of the target PVC that is created by Velero restore\n\tPVC string `json:\"pvc\"`\n\n\t// PV is the name of the target PV that is created by Velero restore\n\tPV string `json:\"pv\"`\n\n\t// Namespace is the target namespace\n\tNamespace string `json:\"namespace\"`\n}\n\n// DataDownloadPhase represents the lifecycle phase of a DataDownload.\n// +kubebuilder:validation:Enum=New;Accepted;Prepared;InProgress;Canceling;Canceled;Completed;Failed\ntype DataDownloadPhase string\n\nconst (\n\tDataDownloadPhaseNew DataDownloadPhase = \"New\"\n\tDataDownloadPhaseAccepted DataDownloadPhase = \"Accepted\"\n\tDataDownloadPhasePrepared DataDownloadPhase = \"Prepared\"\n\tDataDownloadPhaseInProgress DataDownloadPhase = \"InProgress\"\n\tDataDownloadPhaseCanceling DataDownloadPhase = \"Canceling\"\n\tDataDownloadPhaseCanceled DataDownloadPhase = \"Canceled\"\n\tDataDownloadPhaseCompleted DataDownloadPhase = \"Completed\"\n\tDataDownloadPhaseFailed DataDownloadPhase = \"Failed\"\n)\n\n// DataDownloadStatus is the current status of a DataDownload.\ntype DataDownloadStatus struct {\n\t// Phase is the current state of the DataDownload.\n\t// +optional\n\tPhase DataDownloadPhase `json:\"phase,omitempty\"`\n\n\t// Message is a message about the DataDownload's status.\n\t// +optional\n\tMessage string `json:\"message,omitempty\"`\n\n\t// StartTimestamp records the time a restore was started.\n\t// The server's time is used for StartTimestamps\n\t// +optional\n\t// +nullable\n\tStartTimestamp *metav1.Time `json:\"startTimestamp,omitempty\"`\n\n\t// CompletionTimestamp records the time a restore was completed.\n\t// Completion time is recorded even on failed restores.\n\t// The server's time is used for CompletionTimestamps\n\t// +optional\n\t// +nullable\n\tCompletionTimestamp *metav1.Time `json:\"completionTimestamp,omitempty\"`\n\n\t// Progress holds the total number of bytes of the snapshot and the current\n\t// number of restored bytes. This can be used to display progress information\n\t// about the restore operation.\n\t// +optional\n\tProgress shared.DataMoveOperationProgress `json:\"progress,omitempty\"`\n\n\t// Node is name of the node where the DataDownload is processed.\n\t// +optional\n\tNode string `json:\"node,omitempty\"`\n\n\t// Node is name of the node where the DataUpload is prepared.\n\t// +optional\n\tAcceptedByNode string `json:\"acceptedByNode,omitempty\"`\n\n\t// AcceptedTimestamp records the time the DataUpload is to be prepared.\n\t// The server's time is used for AcceptedTimestamp\n\t// +optional\n\t// +nullable\n\tAcceptedTimestamp *metav1.Time `json:\"acceptedTimestamp,omitempty\"`\n}\n\n// TODO(2.0) After converting all resources to use the runtime-controller client, the genclient and k8s:deepcopy markers will no longer be needed and should be removed.\n// +genclient\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:generate=true\n// +kubebuilder:object:root=true\n// +kubebuilder:storageversion\n// +kubebuilder:printcolumn:name=\"Status\",type=\"string\",JSONPath=\".status.phase\",description=\"DataDownload status such as New/InProgress\"\n// +kubebuilder:printcolumn:name=\"Started\",type=\"date\",JSONPath=\".status.startTimestamp\",description=\"Time duration since this DataDownload was started\"\n// +kubebuilder:printcolumn:name=\"Bytes Done\",type=\"integer\",format=\"int64\",JSONPath=\".status.progress.bytesDone\",description=\"Completed bytes\"\n// +kubebuilder:printcolumn:name=\"Total Bytes\",type=\"integer\",format=\"int64\",JSONPath=\".status.progress.totalBytes\",description=\"Total bytes\"\n// +kubebuilder:printcolumn:name=\"Storage Location\",type=\"string\",JSONPath=\".spec.backupStorageLocation\",description=\"Name of the Backup Storage Location where the backup data is stored\"\n// +kubebuilder:printcolumn:name=\"Age\",type=\"date\",JSONPath=\".metadata.creationTimestamp\",description=\"Time duration since this DataDownload was created\"\n// +kubebuilder:printcolumn:name=\"Node\",type=\"string\",JSONPath=\".status.node\",description=\"Name of the node where the DataDownload is processed\"\n\n// DataDownload acts as the protocol between data mover plugins and data mover controller for the datamover restore operation\ntype DataDownload struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ObjectMeta `json:\"metadata,omitempty\"`\n\n\t// +optional\n\tSpec DataDownloadSpec `json:\"spec,omitempty\"`\n\n\t// +optional\n\tStatus DataDownloadStatus `json:\"status,omitempty\"`\n}\n\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:generate=true\n// +kubebuilder:object:root=true\n// +kubebuilder:rbac:groups=velero.io,resources=datadownloads,verbs=get;list;watch;create;update;patch;delete\n// +kubebuilder:rbac:groups=velero.io,resources=datadownloads/status,verbs=get;update;patch\n\n// DataDownloadList is a list of DataDownloads.\ntype DataDownloadList struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ListMeta `json:\"metadata,omitempty\"`\n\n\tItems []DataDownload `json:\"items\"`\n}\n" }, { "path": "pkg/apis/velero/v2alpha1/data_upload_types.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v2alpha1\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/apis/velero/shared\"\n)\n\n// DataUploadSpec is the specification for a DataUpload.\ntype DataUploadSpec struct {\n\t// SnapshotType is the type of the snapshot to be backed up.\n\tSnapshotType SnapshotType `json:\"snapshotType\"`\n\n\t// If SnapshotType is CSI, CSISnapshot provides the information of the CSI snapshot.\n\t// +optional\n\t// +nullable\n\tCSISnapshot *CSISnapshotSpec `json:\"csiSnapshot\"`\n\n\t// SourcePVC is the name of the PVC which the snapshot is taken for.\n\tSourcePVC string `json:\"sourcePVC\"`\n\n\t// DataMover specifies the data mover to be used by the backup.\n\t// If DataMover is \"\" or \"velero\", the built-in data mover will be used.\n\t// +optional\n\tDataMover string `json:\"datamover,omitempty\"`\n\n\t// BackupStorageLocation is the name of the backup storage location\n\t// where the backup repository is stored.\n\tBackupStorageLocation string `json:\"backupStorageLocation\"`\n\n\t// SourceNamespace is the original namespace where the volume is backed up from.\n\t// It is the same namespace for SourcePVC and CSI namespaced objects.\n\tSourceNamespace string `json:\"sourceNamespace\"`\n\n\t// DataMoverConfig is for data-mover-specific configuration fields.\n\t// +optional\n\t// +nullable\n\tDataMoverConfig map[string]string `json:\"dataMoverConfig,omitempty\"`\n\n\t// Cancel indicates request to cancel the ongoing DataUpload. It can be set\n\t// when the DataUpload is in InProgress phase\n\tCancel bool `json:\"cancel,omitempty\"`\n\n\t// OperationTimeout specifies the time used to wait internal operations,\n\t// before returning error as timeout.\n\tOperationTimeout metav1.Duration `json:\"operationTimeout\"`\n}\n\ntype SnapshotType string\n\nconst (\n\tSnapshotTypeCSI SnapshotType = \"CSI\"\n)\n\n// CSISnapshotSpec is the specification for a CSI snapshot.\ntype CSISnapshotSpec struct {\n\t// VolumeSnapshot is the name of the volume snapshot to be backed up\n\tVolumeSnapshot string `json:\"volumeSnapshot\"`\n\n\t// StorageClass is the name of the storage class of the PVC that the volume snapshot is created from\n\tStorageClass string `json:\"storageClass\"`\n\n\t// SnapshotClass is the name of the snapshot class that the volume snapshot is created with\n\t// +optional\n\tSnapshotClass string `json:\"snapshotClass\"`\n\n\t// Driver is the driver used by the VolumeSnapshotContent\n\t// +optional\n\tDriver string `json:\"driver,omitempty\"`\n}\n\n// DataUploadPhase represents the lifecycle phase of a DataUpload.\n// +kubebuilder:validation:Enum=New;Accepted;Prepared;InProgress;Canceling;Canceled;Completed;Failed\ntype DataUploadPhase string\n\nconst (\n\tDataUploadPhaseNew DataUploadPhase = \"New\"\n\tDataUploadPhaseAccepted DataUploadPhase = \"Accepted\"\n\tDataUploadPhasePrepared DataUploadPhase = \"Prepared\"\n\tDataUploadPhaseInProgress DataUploadPhase = \"InProgress\"\n\tDataUploadPhaseCanceling DataUploadPhase = \"Canceling\"\n\tDataUploadPhaseCanceled DataUploadPhase = \"Canceled\"\n\tDataUploadPhaseCompleted DataUploadPhase = \"Completed\"\n\tDataUploadPhaseFailed DataUploadPhase = \"Failed\"\n)\n\n// NodeOS represents OS of a node.\n// +kubebuilder:validation:Enum=auto;linux;windows\ntype NodeOS string\n\nconst (\n\tNodeOSLinux NodeOS = \"linux\"\n\tNodeOSWindows NodeOS = \"windows\"\n\tNodeOSAuto NodeOS = \"auto\"\n)\n\n// DataUploadStatus is the current status of a DataUpload.\ntype DataUploadStatus struct {\n\t// Phase is the current state of the DataUpload.\n\t// +optional\n\tPhase DataUploadPhase `json:\"phase,omitempty\"`\n\n\t// Path is the full path of the snapshot volume being backed up.\n\t// +optional\n\tPath string `json:\"path,omitempty\"`\n\n\t// SnapshotID is the identifier for the snapshot in the backup repository.\n\t// +optional\n\tSnapshotID string `json:\"snapshotID,omitempty\"`\n\n\t// DataMoverResult stores data-mover-specific information as a result of the DataUpload.\n\t// +optional\n\t// +nullable\n\tDataMoverResult *map[string]string `json:\"dataMoverResult,omitempty\"`\n\n\t// Message is a message about the DataUpload's status.\n\t// +optional\n\tMessage string `json:\"message,omitempty\"`\n\n\t// StartTimestamp records the time a backup was started.\n\t// Separate from CreationTimestamp, since that value changes\n\t// on restores.\n\t// The server's time is used for StartTimestamps\n\t// +optional\n\t// +nullable\n\tStartTimestamp *metav1.Time `json:\"startTimestamp,omitempty\"`\n\n\t// CompletionTimestamp records the time a backup was completed.\n\t// Completion time is recorded even on failed backups.\n\t// Completion time is recorded before uploading the backup object.\n\t// The server's time is used for CompletionTimestamps\n\t// +optional\n\t// +nullable\n\tCompletionTimestamp *metav1.Time `json:\"completionTimestamp,omitempty\"`\n\n\t// Progress holds the total number of bytes of the volume and the current\n\t// number of backed up bytes. This can be used to display progress information\n\t// about the backup operation.\n\t// +optional\n\tProgress shared.DataMoveOperationProgress `json:\"progress,omitempty\"`\n\n\t// IncrementalBytes holds the number of bytes new or changed since the last backup\n\t// +optional\n\tIncrementalBytes int64 `json:\"incrementalBytes,omitempty\"`\n\n\t// Node is name of the node where the DataUpload is processed.\n\t// +optional\n\tNode string `json:\"node,omitempty\"`\n\n\t// NodeOS is OS of the node where the DataUpload is processed.\n\t// +optional\n\tNodeOS NodeOS `json:\"nodeOS,omitempty\"`\n\n\t// AcceptedByNode is name of the node where the DataUpload is prepared.\n\t// +optional\n\tAcceptedByNode string `json:\"acceptedByNode,omitempty\"`\n\n\t// AcceptedTimestamp records the time the DataUpload is to be prepared.\n\t// The server's time is used for AcceptedTimestamp\n\t// +optional\n\t// +nullable\n\tAcceptedTimestamp *metav1.Time `json:\"acceptedTimestamp,omitempty\"`\n}\n\n// TODO(2.0) After converting all resources to use the runttime-controller client,\n// the genclient and k8s:deepcopy markers will no longer be needed and should be removed.\n// +genclient\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:root=true\n// +kubebuilder:object:generate=true\n// +kubebuilder:storageversion\n// +kubebuilder:printcolumn:name=\"Status\",type=\"string\",JSONPath=\".status.phase\",description=\"DataUpload status such as New/InProgress\"\n// +kubebuilder:printcolumn:name=\"Started\",type=\"date\",JSONPath=\".status.startTimestamp\",description=\"Time duration since this DataUpload was started\"\n// +kubebuilder:printcolumn:name=\"Bytes Done\",type=\"integer\",format=\"int64\",JSONPath=\".status.progress.bytesDone\",description=\"Completed bytes\"\n// +kubebuilder:printcolumn:name=\"Total Bytes\",type=\"integer\",format=\"int64\",JSONPath=\".status.progress.totalBytes\",description=\"Total bytes\"\n// +kubebuilder:printcolumn:name=\"Incremental Bytes\",type=\"integer\",format=\"int64\",JSONPath=\".status.incrementalBytes\",description=\"Incremental bytes\",priority=10\n// +kubebuilder:printcolumn:name=\"Storage Location\",type=\"string\",JSONPath=\".spec.backupStorageLocation\",description=\"Name of the Backup Storage Location where this backup should be stored\"\n// +kubebuilder:printcolumn:name=\"Age\",type=\"date\",JSONPath=\".metadata.creationTimestamp\",description=\"Time duration since this DataUpload was created\"\n// +kubebuilder:printcolumn:name=\"Node\",type=\"string\",JSONPath=\".status.node\",description=\"Name of the node where the DataUpload is processed\"\n\n// DataUpload acts as the protocol between data mover plugins and data mover controller for the datamover backup operation\ntype DataUpload struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ObjectMeta `json:\"metadata,omitempty\"`\n\n\t// +optional\n\tSpec DataUploadSpec `json:\"spec,omitempty\"`\n\n\t// +optional\n\tStatus DataUploadStatus `json:\"status,omitempty\"`\n}\n\n// TODO(2.0) After converting all resources to use the runtime-controller client,\n// the k8s:deepcopy marker will no longer be needed and should be removed.\n// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object\n// +kubebuilder:object:root=true\n// +kubebuilder:rbac:groups=velero.io,resources=datauploads,verbs=get;list;watch;create;update;patch;delete\n// +kubebuilder:rbac:groups=velero.io,resources=datauploads/status,verbs=get;update;patch\n\n// DataUploadList is a list of DataUploads.\ntype DataUploadList struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ListMeta `json:\"metadata,omitempty\"`\n\n\tItems []DataUpload `json:\"items\"`\n}\n\n// DataUploadResult represents the SnasphotBackup result to be used by DataDownload.\ntype DataUploadResult struct {\n\t// BackupStorageLocation is the name of the backup storage location\n\t// where the backup repository is stored.\n\tBackupStorageLocation string `json:\"backupStorageLocation\"`\n\n\t// DataMover specifies the data mover used by the DataUpload\n\t// +optional\n\tDataMover string `json:\"datamover,omitempty\"`\n\n\t// SnapshotID is the identifier for the snapshot in the backup repository.\n\tSnapshotID string `json:\"snapshotID,omitempty\"`\n\n\t// SourceNamespace is the original namespace where the volume is backed up from.\n\tSourceNamespace string `json:\"sourceNamespace\"`\n\n\t// DataMoverResult stores data-mover-specific information as a result of the DataUpload.\n\t// +optional\n\t// +nullable\n\tDataMoverResult *map[string]string `json:\"dataMoverResult,omitempty\"`\n\n\t// NodeOS is OS of the node where the DataUpload is processed.\n\t// +optional\n\tNodeOS NodeOS `json:\"nodeOS,omitempty\"`\n\n\t// SnapshotSize is the logical size in Bytes of the snapshot.\n\t// +optional\n\tSnapshotSize int64 `json:\"snapshotSize,omitempty\"`\n}\n" }, { "path": "pkg/apis/velero/v2alpha1/doc.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\n// +k8s:deepcopy-gen=package\n\n// Package v2alpha1 is the v2alpha1 version of the API.\n// +groupName=velero.io\npackage v2alpha1\n" }, { "path": "pkg/apis/velero/v2alpha1/groupversion_info.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\n// Package v2alpha1 contains API Schema definitions for the velero v2alpha1 API group\n// +kubebuilder:object:generate=true\n// +groupName=velero.io\npackage v2alpha1\n\nimport (\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n)\n\nvar (\n\t// SchemeGroupVersion is group version used to register these objects\n\tSchemeGroupVersion = schema.GroupVersion{Group: \"velero.io\", Version: \"v2alpha1\"}\n\n\t// SchemeBuilder is used to add go types to the GroupVersionKind scheme\n\tSchemeBuilder = runtime.NewSchemeBuilder(addKnownTypes)\n\n\t// AddToScheme adds the types in this group-version to the given scheme.\n\tAddToScheme = SchemeBuilder.AddToScheme\n)\n" }, { "path": "pkg/apis/velero/v2alpha1/register.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v2alpha1\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n)\n\n// Resource gets a Velero GroupResource for a specified resource\nfunc Resource(resource string) schema.GroupResource {\n\treturn SchemeGroupVersion.WithResource(resource).GroupResource()\n}\n\ntype typeInfo struct {\n\tPluralName string\n\tItemType runtime.Object\n\tItemListType runtime.Object\n}\n\nfunc newTypeInfo(pluralName string, itemType, itemListType runtime.Object) typeInfo {\n\treturn typeInfo{\n\t\tPluralName: pluralName,\n\t\tItemType: itemType,\n\t\tItemListType: itemListType,\n\t}\n}\n\n// CustomResources returns a map of all custom resources within the Velero\n// API group, keyed on Kind.\nfunc CustomResources() map[string]typeInfo {\n\treturn map[string]typeInfo{\n\t\t\"DataUpload\": newTypeInfo(\"datauploads\", &DataUpload{}, &DataUploadList{}),\n\t\t\"DataDownload\": newTypeInfo(\"datadownloads\", &DataDownload{}, &DataDownloadList{}),\n\t}\n}\n\n// CustomResourceKinds returns a list of all custom resources kinds within the Velero\nfunc CustomResourceKinds() sets.Set[string] {\n\tkinds := sets.New[string]()\n\n\tresources := CustomResources()\n\tfor kind := range resources {\n\t\tkinds.Insert(kind)\n\t}\n\n\treturn kinds\n}\n\nfunc addKnownTypes(scheme *runtime.Scheme) error {\n\tfor _, typeInfo := range CustomResources() {\n\t\tscheme.AddKnownTypes(SchemeGroupVersion, typeInfo.ItemType, typeInfo.ItemListType)\n\t}\n\n\tmetav1.AddToGroupVersion(scheme, SchemeGroupVersion)\n\treturn nil\n}\n" }, { "path": "pkg/apis/velero/v2alpha1/zz_generated.deepcopy.go", "content": "//go:build !ignore_autogenerated\n\n// Code generated by controller-gen. DO NOT EDIT.\n\npackage v2alpha1\n\nimport (\n\t\"k8s.io/apimachinery/pkg/runtime\"\n)\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *CSISnapshotSpec) DeepCopyInto(out *CSISnapshotSpec) {\n\t*out = *in\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CSISnapshotSpec.\nfunc (in *CSISnapshotSpec) DeepCopy() *CSISnapshotSpec {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(CSISnapshotSpec)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *DataDownload) DeepCopyInto(out *DataDownload) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ObjectMeta.DeepCopyInto(&out.ObjectMeta)\n\tin.Spec.DeepCopyInto(&out.Spec)\n\tin.Status.DeepCopyInto(&out.Status)\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataDownload.\nfunc (in *DataDownload) DeepCopy() *DataDownload {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(DataDownload)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *DataDownload) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *DataDownloadList) DeepCopyInto(out *DataDownloadList) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ListMeta.DeepCopyInto(&out.ListMeta)\n\tif in.Items != nil {\n\t\tin, out := &in.Items, &out.Items\n\t\t*out = make([]DataDownload, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataDownloadList.\nfunc (in *DataDownloadList) DeepCopy() *DataDownloadList {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(DataDownloadList)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *DataDownloadList) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *DataDownloadSpec) DeepCopyInto(out *DataDownloadSpec) {\n\t*out = *in\n\tout.TargetVolume = in.TargetVolume\n\tif in.DataMoverConfig != nil {\n\t\tin, out := &in.DataMoverConfig, &out.DataMoverConfig\n\t\t*out = make(map[string]string, len(*in))\n\t\tfor key, val := range *in {\n\t\t\t(*out)[key] = val\n\t\t}\n\t}\n\tout.OperationTimeout = in.OperationTimeout\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataDownloadSpec.\nfunc (in *DataDownloadSpec) DeepCopy() *DataDownloadSpec {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(DataDownloadSpec)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *DataDownloadStatus) DeepCopyInto(out *DataDownloadStatus) {\n\t*out = *in\n\tif in.StartTimestamp != nil {\n\t\tin, out := &in.StartTimestamp, &out.StartTimestamp\n\t\t*out = (*in).DeepCopy()\n\t}\n\tif in.CompletionTimestamp != nil {\n\t\tin, out := &in.CompletionTimestamp, &out.CompletionTimestamp\n\t\t*out = (*in).DeepCopy()\n\t}\n\tout.Progress = in.Progress\n\tif in.AcceptedTimestamp != nil {\n\t\tin, out := &in.AcceptedTimestamp, &out.AcceptedTimestamp\n\t\t*out = (*in).DeepCopy()\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataDownloadStatus.\nfunc (in *DataDownloadStatus) DeepCopy() *DataDownloadStatus {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(DataDownloadStatus)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *DataUpload) DeepCopyInto(out *DataUpload) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ObjectMeta.DeepCopyInto(&out.ObjectMeta)\n\tin.Spec.DeepCopyInto(&out.Spec)\n\tin.Status.DeepCopyInto(&out.Status)\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataUpload.\nfunc (in *DataUpload) DeepCopy() *DataUpload {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(DataUpload)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *DataUpload) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *DataUploadList) DeepCopyInto(out *DataUploadList) {\n\t*out = *in\n\tout.TypeMeta = in.TypeMeta\n\tin.ListMeta.DeepCopyInto(&out.ListMeta)\n\tif in.Items != nil {\n\t\tin, out := &in.Items, &out.Items\n\t\t*out = make([]DataUpload, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataUploadList.\nfunc (in *DataUploadList) DeepCopy() *DataUploadList {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(DataUploadList)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.\nfunc (in *DataUploadList) DeepCopyObject() runtime.Object {\n\tif c := in.DeepCopy(); c != nil {\n\t\treturn c\n\t}\n\treturn nil\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *DataUploadResult) DeepCopyInto(out *DataUploadResult) {\n\t*out = *in\n\tif in.DataMoverResult != nil {\n\t\tin, out := &in.DataMoverResult, &out.DataMoverResult\n\t\t*out = new(map[string]string)\n\t\tif **in != nil {\n\t\t\tin, out := *in, *out\n\t\t\t*out = make(map[string]string, len(*in))\n\t\t\tfor key, val := range *in {\n\t\t\t\t(*out)[key] = val\n\t\t\t}\n\t\t}\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataUploadResult.\nfunc (in *DataUploadResult) DeepCopy() *DataUploadResult {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(DataUploadResult)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *DataUploadSpec) DeepCopyInto(out *DataUploadSpec) {\n\t*out = *in\n\tif in.CSISnapshot != nil {\n\t\tin, out := &in.CSISnapshot, &out.CSISnapshot\n\t\t*out = new(CSISnapshotSpec)\n\t\t**out = **in\n\t}\n\tif in.DataMoverConfig != nil {\n\t\tin, out := &in.DataMoverConfig, &out.DataMoverConfig\n\t\t*out = make(map[string]string, len(*in))\n\t\tfor key, val := range *in {\n\t\t\t(*out)[key] = val\n\t\t}\n\t}\n\tout.OperationTimeout = in.OperationTimeout\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataUploadSpec.\nfunc (in *DataUploadSpec) DeepCopy() *DataUploadSpec {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(DataUploadSpec)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *DataUploadStatus) DeepCopyInto(out *DataUploadStatus) {\n\t*out = *in\n\tif in.DataMoverResult != nil {\n\t\tin, out := &in.DataMoverResult, &out.DataMoverResult\n\t\t*out = new(map[string]string)\n\t\tif **in != nil {\n\t\t\tin, out := *in, *out\n\t\t\t*out = make(map[string]string, len(*in))\n\t\t\tfor key, val := range *in {\n\t\t\t\t(*out)[key] = val\n\t\t\t}\n\t\t}\n\t}\n\tif in.StartTimestamp != nil {\n\t\tin, out := &in.StartTimestamp, &out.StartTimestamp\n\t\t*out = (*in).DeepCopy()\n\t}\n\tif in.CompletionTimestamp != nil {\n\t\tin, out := &in.CompletionTimestamp, &out.CompletionTimestamp\n\t\t*out = (*in).DeepCopy()\n\t}\n\tout.Progress = in.Progress\n\tif in.AcceptedTimestamp != nil {\n\t\tin, out := &in.AcceptedTimestamp, &out.AcceptedTimestamp\n\t\t*out = (*in).DeepCopy()\n\t}\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataUploadStatus.\nfunc (in *DataUploadStatus) DeepCopy() *DataUploadStatus {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(DataUploadStatus)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\n// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.\nfunc (in *TargetVolumeSpec) DeepCopyInto(out *TargetVolumeSpec) {\n\t*out = *in\n}\n\n// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TargetVolumeSpec.\nfunc (in *TargetVolumeSpec) DeepCopy() *TargetVolumeSpec {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(TargetVolumeSpec)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n" }, { "path": "pkg/archive/extractor.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage archive\n\nimport (\n\t\"archive/tar\"\n\t\"compress/gzip\"\n\t\"io\"\n\t\"path/filepath\"\n\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n)\n\n// Extractor unzips/extracts a backup tarball to a local\n// temp directory.\ntype Extractor struct {\n\tlog logrus.FieldLogger\n\tfs filesystem.Interface\n}\n\nfunc NewExtractor(log logrus.FieldLogger, fs filesystem.Interface) *Extractor {\n\treturn &Extractor{\n\t\tlog: log,\n\t\tfs: fs,\n\t}\n}\n\n// UnzipAndExtractBackup extracts a reader on a gzipped tarball to a local temp directory\nfunc (e *Extractor) UnzipAndExtractBackup(src io.Reader) (string, error) {\n\tgzr, err := gzip.NewReader(src)\n\tif err != nil {\n\t\te.log.Infof(\"error creating gzip reader: %v\", err)\n\t\treturn \"\", err\n\t}\n\tdefer gzr.Close()\n\n\treturn e.readBackup(tar.NewReader(gzr))\n}\n\nfunc (e *Extractor) writeFile(target string, tarRdr *tar.Reader) error {\n\tfile, err := e.fs.Create(target)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer file.Close()\n\n\tif _, err := io.Copy(file, tarRdr); err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n\nfunc (e *Extractor) readBackup(tarRdr *tar.Reader) (string, error) {\n\tdir, err := e.fs.TempDir(\"\", \"\")\n\tif err != nil {\n\t\te.log.Infof(\"error creating temp dir: %v\", err)\n\t\treturn \"\", err\n\t}\n\n\tfor {\n\t\theader, err := tarRdr.Next()\n\n\t\tif err == io.EOF {\n\t\t\tbreak\n\t\t}\n\t\tif err != nil {\n\t\t\te.log.Infof(\"error reading tar: %v\", err)\n\t\t\treturn \"\", err\n\t\t}\n\n\t\ttarget := filepath.Join(dir, header.Name) //nolint:gosec // Internal usage. No need to check.\n\n\t\tswitch header.Typeflag {\n\t\tcase tar.TypeDir:\n\t\t\terr := e.fs.MkdirAll(target, header.FileInfo().Mode())\n\t\t\tif err != nil {\n\t\t\t\te.log.Infof(\"mkdirall error: %v\", err)\n\t\t\t\treturn \"\", err\n\t\t\t}\n\n\t\tcase tar.TypeReg:\n\t\t\t// make sure we have the directory created\n\t\t\terr := e.fs.MkdirAll(filepath.Dir(target), header.FileInfo().Mode())\n\t\t\tif err != nil {\n\t\t\t\te.log.Infof(\"mkdirall error: %v\", err)\n\t\t\t\treturn \"\", err\n\t\t\t}\n\n\t\t\t// create the file\n\t\t\tif err := e.writeFile(target, tarRdr); err != nil {\n\t\t\t\te.log.Infof(\"error copying: %v\", err)\n\t\t\t\treturn \"\", err\n\t\t\t}\n\t\t}\n\t}\n\n\treturn dir, nil\n}\n" }, { "path": "pkg/archive/extractor_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage archive\n\nimport (\n\t\"archive/tar\"\n\t\"compress/gzip\"\n\t\"io\"\n\t\"os\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n)\n\nfunc TestUnzipAndExtractBackup(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tfiles []string\n\t\tIsTarball bool\n\t\twantErr bool\n\t}{\n\t\t{\n\t\t\tname: \"when the format of backup file is invalid, an error is returned\",\n\t\t\tfiles: []string{},\n\t\t\tIsTarball: false,\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"when the backup tarball is empty, the function should work correctly and returns no error\",\n\t\t\tfiles: []string{},\n\t\t\tIsTarball: true,\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"when the backup tarball includes a mix of items, the function should work correctly and returns no error\",\n\t\t\tfiles: []string{\n\t\t\t\t\"root-dir/resources/namespace/cluster/example.json\",\n\t\t\t\t\"root-dir/resources/pods/namespaces/example.json\",\n\t\t\t\t\"root-dir/metadata/version\",\n\t\t\t},\n\t\t\tIsTarball: true,\n\t\t\twantErr: false,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\text := NewExtractor(test.NewLogger(), test.NewFakeFileSystem())\n\t\t\tvar fileName string\n\t\t\tvar err error\n\t\t\tif tc.IsTarball {\n\t\t\t\tfileName, err = createArchive(tc.files, ext.fs)\n\t\t\t} else {\n\t\t\t\tfileName, err = createRegular(ext.fs)\n\t\t\t}\n\t\t\trequire.NoError(t, err)\n\n\t\t\tfile, err := ext.fs.OpenFile(fileName, os.O_RDWR|os.O_CREATE, 0644)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t_, err = ext.UnzipAndExtractBackup(file.(io.Reader))\n\t\t\tif tc.wantErr && (err == nil) {\n\t\t\t\tt.Errorf(\"%s: wanted error but got nil\", tc.name)\n\t\t\t}\n\n\t\t\tif !tc.wantErr && (err != nil) {\n\t\t\t\tt.Errorf(\"%s: wanted no error but got err: %v\", tc.name, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc createArchive(files []string, fs filesystem.Interface) (string, error) {\n\toutName := \"output.tar.gz\"\n\tout, err := fs.Create(outName)\n\tif err != nil {\n\t\treturn outName, err\n\t}\n\tdefer out.Close()\n\tgw := gzip.NewWriter(out)\n\tdefer gw.Close()\n\ttw := tar.NewWriter(gw)\n\tdefer tw.Close()\n\n\t// Iterate over files and add them to the tar archive\n\tfor _, file := range files {\n\t\terr := addToArchive(tw, file, fs)\n\t\tif err != nil {\n\t\t\treturn outName, err\n\t\t}\n\t}\n\n\treturn outName, nil\n}\n\nfunc addToArchive(tw *tar.Writer, filename string, fs filesystem.Interface) error {\n\t// Create the file\n\tfile, err := fs.Create(filename)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer file.Close()\n\n\t// Get FileInfo about size, mode, etc.\n\tinfo, err := fs.Stat(filename)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Create a tar Header from the FileInfo data\n\theader, err := tar.FileInfoHeader(info, info.Name())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\theader.Name = filename\n\terr = tw.WriteHeader(header)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\nfunc createRegular(fs filesystem.Interface) (string, error) {\n\toutName := \"output\"\n\tout, err := fs.Create(outName)\n\tif err != nil {\n\t\treturn outName, err\n\t}\n\tdefer out.Close()\n\n\treturn outName, nil\n}\n" }, { "path": "pkg/archive/filesystem.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage archive\n\nimport (\n\t\"encoding/json\"\n\t\"path/filepath\"\n\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n)\n\n// GetItemFilePath returns an item's file path once extracted from a Velero backup archive.\nfunc GetItemFilePath(rootDir, groupResource, namespace, name string) string {\n\treturn GetVersionedItemFilePath(rootDir, groupResource, namespace, name, \"\")\n}\n\n// GetVersionedItemFilePath returns an item's file path once extracted from a Velero backup archive, with version included.\nfunc GetVersionedItemFilePath(rootDir, groupResource, namespace, name, versionPath string) string {\n\treturn filepath.Join(rootDir, velerov1api.ResourcesDir, groupResource, versionPath, GetScopeDir(namespace), namespace, name+\".json\")\n}\n\n// GetScopeDir returns NamespaceScopedDir if namespace is present, or ClusterScopedDir if empty\nfunc GetScopeDir(namespace string) string {\n\tif namespace == \"\" {\n\t\treturn velerov1api.ClusterScopedDir\n\t}\n\treturn velerov1api.NamespaceScopedDir\n}\n\n// Unmarshal reads the specified file, unmarshals the JSON contained within it\n// and returns an Unstructured object.\nfunc Unmarshal(fs filesystem.Interface, filePath string) (*unstructured.Unstructured, error) {\n\tvar obj unstructured.Unstructured\n\n\tbytes, err := fs.ReadFile(filePath)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\terr = json.Unmarshal(bytes, &obj)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &obj, nil\n}\n" }, { "path": "pkg/archive/filesystem_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage archive\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestGetItemFilePath(t *testing.T) {\n\tres := GetItemFilePath(\"root\", \"resource\", \"\", \"item\")\n\tassert.Equal(t, \"root/resources/resource/cluster/item.json\", res)\n\n\tres = GetItemFilePath(\"root\", \"resource\", \"namespace\", \"item\")\n\tassert.Equal(t, \"root/resources/resource/namespaces/namespace/item.json\", res)\n\n\tres = GetItemFilePath(\"\", \"resource\", \"\", \"item\")\n\tassert.Equal(t, \"resources/resource/cluster/item.json\", res)\n\n\tres = GetVersionedItemFilePath(\"root\", \"resource\", \"\", \"item\", \"\")\n\tassert.Equal(t, \"root/resources/resource/cluster/item.json\", res)\n\n\tres = GetVersionedItemFilePath(\"root\", \"resource\", \"namespace\", \"item\", \"\")\n\tassert.Equal(t, \"root/resources/resource/namespaces/namespace/item.json\", res)\n\n\tres = GetVersionedItemFilePath(\"root\", \"resource\", \"namespace\", \"item\", \"v1\")\n\tassert.Equal(t, \"root/resources/resource/v1/namespaces/namespace/item.json\", res)\n\n\tres = GetVersionedItemFilePath(\"root\", \"resource\", \"\", \"item\", \"v1\")\n\tassert.Equal(t, \"root/resources/resource/v1/cluster/item.json\", res)\n\n\tres = GetVersionedItemFilePath(\"\", \"resource\", \"\", \"item\", \"\")\n\tassert.Equal(t, \"resources/resource/cluster/item.json\", res)\n}\n\nfunc TestGetScopeDir(t *testing.T) {\n\tres := GetScopeDir(\"\")\n\tassert.Equal(t, velerov1api.ClusterScopedDir, res)\n\n\tres = GetScopeDir(\"test-namespace\")\n\tassert.Equal(t, velerov1api.NamespaceScopedDir, res)\n}\n\nfunc TestUnmarshal(t *testing.T) {\n\tfs := test.NewFakeFileSystem()\n\tfilePath := \"pod.json\"\n\tfileContent := `{\n\t\t\"apiVersion\": \"v1\",\n\t\t\"kind\": \"Pod\",\n\t\t\"metadata\": {\n\t\t\t\"name\": \"example-pod\"\n\t\t},\n\t\t\"spec\": {\n\t\t\t\"containers\": [{\n\t\t\t\t\"name\": \"example-container\",\n\t\t\t\t\"image\": \"example-image\"\n\t\t\t}]\n\t\t}\n\t}`\n\tout, err := fs.Create(filePath)\n\trequire.NoError(t, err)\n\n\t_, err = out.Write([]byte(fileContent))\n\trequire.NoError(t, err)\n\n\t_, err = Unmarshal(fs, filePath)\n\trequire.NoError(t, err)\n}\n" }, { "path": "pkg/archive/parser.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage archive\n\nimport (\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n)\n\nvar ErrNotExist = errors.New(\"does not exist\")\n\n// Parser traverses an extracted archive on disk to validate\n// it and provide a helpful representation of it to consumers.\ntype Parser struct {\n\tlog logrus.FieldLogger\n\tfs filesystem.Interface\n}\n\n// ResourceItems contains the collection of items of a given resource type\n// within a backup, grouped by namespace (or empty string for cluster-scoped\n// resources).\ntype ResourceItems struct {\n\t// GroupResource is API group and resource name,\n\t// formatted as \"resource.group\". For the \"core\"\n\t// API group, the \".group\" suffix is omitted.\n\tGroupResource string\n\n\t// ItemsByNamespace is a map from namespace (or empty string\n\t// for cluster-scoped resources) to a list of individual item\n\t// names contained in the archive. Item names **do not** include\n\t// the file extension.\n\tItemsByNamespace map[string][]string\n}\n\n// NewParser constructs a Parser.\nfunc NewParser(log logrus.FieldLogger, fs filesystem.Interface) *Parser {\n\treturn &Parser{\n\t\tlog: log,\n\t\tfs: fs,\n\t}\n}\n\n// Parse reads an extracted backup on the file system and returns\n// a structured catalog of the resources and items contained within it.\nfunc (p *Parser) Parse(dir string) (map[string]*ResourceItems, error) {\n\t// ensure top-level \"resources\" directory exists, and read subdirectories\n\t// of it, where each one is expected to correspond to a resource.\n\tresourcesDir := filepath.Join(dir, velerov1api.ResourcesDir)\n\tresourceDirs, err := p.checkAndReadDir(resourcesDir)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// loop through each subdirectory (one per resource) and assemble\n\t// catalog of items within it.\n\tresources := map[string]*ResourceItems{}\n\tfor _, resourceDir := range resourceDirs {\n\t\tif !resourceDir.IsDir() {\n\t\t\tp.log.Warnf(\"Ignoring unexpected file %q in directory %q\", resourceDir.Name(), strings.TrimPrefix(resourcesDir, dir+\"/\"))\n\t\t\tcontinue\n\t\t}\n\n\t\tresourceItems := &ResourceItems{\n\t\t\tGroupResource: resourceDir.Name(),\n\t\t\tItemsByNamespace: map[string][]string{},\n\t\t}\n\n\t\t// check for existence of a \"cluster\" subdirectory containing cluster-scoped\n\t\t// instances of this resource, and read its contents if it exists.\n\t\tclusterScopedDir := filepath.Join(resourcesDir, resourceDir.Name(), velerov1api.ClusterScopedDir)\n\t\texists, err := p.fs.DirExists(clusterScopedDir)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"error checking for existence of directory %q\", strings.TrimPrefix(clusterScopedDir, dir+\"/\"))\n\t\t}\n\t\tif exists {\n\t\t\titems, err := p.getResourceItemsForScope(clusterScopedDir, dir)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\n\t\t\tif len(items) > 0 {\n\t\t\t\tresourceItems.ItemsByNamespace[\"\"] = items\n\t\t\t}\n\t\t}\n\n\t\t// check for existence of a \"namespaces\" subdirectory containing further subdirectories,\n\t\t// one per namespace, and read its contents if it exists.\n\t\tnamespaceScopedDir := filepath.Join(resourcesDir, resourceDir.Name(), velerov1api.NamespaceScopedDir)\n\t\texists, err = p.fs.DirExists(namespaceScopedDir)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"error checking for existence of directory %q\", strings.TrimPrefix(namespaceScopedDir, dir+\"/\"))\n\t\t}\n\t\tif exists {\n\t\t\tnamespaceDirs, err := p.fs.ReadDir(namespaceScopedDir)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, errors.Wrapf(err, \"error reading contents of directory %q\", strings.TrimPrefix(namespaceScopedDir, dir+\"/\"))\n\t\t\t}\n\n\t\t\tfor _, namespaceDir := range namespaceDirs {\n\t\t\t\tif !namespaceDir.IsDir() {\n\t\t\t\t\tp.log.Warnf(\"Ignoring unexpected file %q in directory %q\", namespaceDir.Name(), strings.TrimPrefix(namespaceScopedDir, dir+\"/\"))\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\n\t\t\t\titems, err := p.getResourceItemsForScope(filepath.Join(namespaceScopedDir, namespaceDir.Name()), dir)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn nil, err\n\t\t\t\t}\n\n\t\t\t\tif len(items) > 0 {\n\t\t\t\t\tresourceItems.ItemsByNamespace[namespaceDir.Name()] = items\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\tresources[resourceDir.Name()] = resourceItems\n\t}\n\n\treturn resources, nil\n}\n\n// getResourceItemsForScope returns the list of items with a namespace or\n// cluster-scoped subdirectory for a specific resource.\nfunc (p *Parser) getResourceItemsForScope(dir, archiveRootDir string) ([]string, error) {\n\tfiles, err := p.fs.ReadDir(dir)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error reading contents of directory %q\", strings.TrimPrefix(dir, archiveRootDir+\"/\"))\n\t}\n\n\tvar items []string\n\tfor _, file := range files {\n\t\tif file.IsDir() {\n\t\t\tp.log.Warnf(\"Ignoring unexpected subdirectory %q in directory %q\", file.Name(), strings.TrimPrefix(dir, archiveRootDir+\"/\"))\n\t\t\tcontinue\n\t\t}\n\n\t\titems = append(items, strings.TrimSuffix(file.Name(), \".json\"))\n\t}\n\n\treturn items, nil\n}\n\n// checkAndReadDir is a wrapper around fs.DirExists and fs.ReadDir that does checks\n// and returns errors if directory cannot be read.\nfunc (p *Parser) checkAndReadDir(dir string) ([]os.FileInfo, error) {\n\texists, err := p.fs.DirExists(dir)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error checking for existence of directory %q\", filepath.ToSlash(dir))\n\t}\n\tif !exists {\n\t\treturn nil, errors.Wrapf(ErrNotExist, \"directory %q\", filepath.ToSlash(dir))\n\t}\n\n\tcontents, err := p.fs.ReadDir(dir)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"reading contents of %q\", filepath.ToSlash(dir))\n\t}\n\n\treturn contents, nil\n}\n\n// ParseGroupVersions extracts the versions for each API Group from the backup\n// directory names and stores them in a metav1 APIGroup object.\nfunc (p *Parser) ParseGroupVersions(dir string) (map[string]metav1.APIGroup, error) {\n\tresourcesDir := filepath.Join(dir, velerov1api.ResourcesDir)\n\n\t// Get the subdirectories inside the \"resources\" directory. The subdirectories\n\t// will have resource.group names like \"horizontalpodautoscalers.autoscaling\".\n\trgDirs, err := p.checkAndReadDir(resourcesDir)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tresourceAGs := make(map[string]metav1.APIGroup)\n\n\t// Loop through the resource.group directory names.\n\tfor _, rgd := range rgDirs {\n\t\tgroup := metav1.APIGroup{\n\t\t\tName: extractGroupName(rgd.Name()),\n\t\t}\n\n\t\trgdPath := filepath.Join(resourcesDir, rgd.Name())\n\n\t\t// Inside each of the resource.group directories are directories whose\n\t\t// names are API Group versions like \"v1\" or \"v1-preferredversion\"\n\t\tgvDirs, err := p.checkAndReadDir(rgdPath)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tvar supportedVersions []metav1.GroupVersionForDiscovery\n\n\t\tfor _, gvd := range gvDirs {\n\t\t\tgvdName := gvd.Name()\n\n\t\t\t// Don't save the namespaces or clusters directories in list of\n\t\t\t// supported API Group Versions.\n\t\t\tif gvdName == \"namespaces\" || gvdName == \"cluster\" {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tversion := metav1.GroupVersionForDiscovery{\n\t\t\t\tGroupVersion: strings.TrimPrefix(group.Name+\"/\"+gvdName, \"/\"),\n\t\t\t\tVersion: gvdName,\n\t\t\t}\n\n\t\t\tif strings.Contains(gvdName, velerov1api.PreferredVersionDir) {\n\t\t\t\tgvdName = strings.TrimSuffix(gvdName, velerov1api.PreferredVersionDir)\n\n\t\t\t\t// Update version and group version to be without suffix.\n\t\t\t\tversion.Version = gvdName\n\t\t\t\tversion.GroupVersion = strings.TrimPrefix(group.Name+\"/\"+gvdName, \"/\")\n\n\t\t\t\tgroup.PreferredVersion = version\n\t\t\t}\n\n\t\t\tsupportedVersions = append(supportedVersions, version)\n\t\t}\n\n\t\tgroup.Versions = supportedVersions\n\n\t\tresourceAGs[rgd.Name()] = group\n\t}\n\n\treturn resourceAGs, nil\n}\n\n// extractGroupName will take a concatenated resource.group and extract the group,\n// if there is one. Resources like \"pods\" which has no group and will return an\n// empty string.\nfunc extractGroupName(resourceGroupDir string) string {\n\tparts := strings.SplitN(resourceGroupDir, \".\", 2)\n\tvar group string\n\n\tif len(parts) == 2 {\n\t\tgroup = parts[1]\n\t}\n\n\treturn group\n}\n" }, { "path": "pkg/archive/parser_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage archive\n\nimport (\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestParse(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tfiles []string\n\t\tdir string\n\t\twantErrMsg error\n\t\twant map[string]*ResourceItems\n\t}{\n\t\t{\n\t\t\tname: \"when there is no top-level resources directory, an error is returned\",\n\t\t\tdir: \"root-dir\",\n\t\t\twantErrMsg: ErrNotExist,\n\t\t},\n\t\t{\n\t\t\tname: \"when there are no directories under the resources directory, an empty map is returned\",\n\t\t\tdir: \"root-dir\",\n\t\t\tfiles: []string{\"root-dir/resources/\"},\n\t\t\twant: map[string]*ResourceItems{},\n\t\t},\n\t\t{\n\t\t\tname: \"a mix of cluster-scoped and namespaced items across multiple resources are correctly returned\",\n\t\t\tdir: \"root-dir\",\n\t\t\tfiles: []string{\n\t\t\t\t\"root-dir/resources/widgets.foo/cluster/item-1.json\",\n\t\t\t\t\"root-dir/resources/widgets.foo/cluster/item-2.json\",\n\t\t\t\t\"root-dir/resources/widgets.foo/namespaces/ns-1/item-1.json\",\n\t\t\t\t\"root-dir/resources/widgets.foo/namespaces/ns-1/item-2.json\",\n\t\t\t\t\"root-dir/resources/widgets.foo/namespaces/ns-2/item-1.json\",\n\t\t\t\t\"root-dir/resources/widgets.foo/namespaces/ns-2/item-2.json\",\n\n\t\t\t\t\"root-dir/resources/dongles.foo/cluster/item-3.json\",\n\t\t\t\t\"root-dir/resources/dongles.foo/cluster/item-4.json\",\n\n\t\t\t\t\"root-dir/resources/dongles.bar/namespaces/ns-3/item-3.json\",\n\t\t\t\t\"root-dir/resources/dongles.bar/namespaces/ns-3/item-4.json\",\n\t\t\t\t\"root-dir/resources/dongles.bar/namespaces/ns-4/item-5.json\",\n\t\t\t\t\"root-dir/resources/dongles.bar/namespaces/ns-4/item-6.json\",\n\t\t\t},\n\t\t\twant: map[string]*ResourceItems{\n\t\t\t\t\"widgets.foo\": {\n\t\t\t\t\tGroupResource: \"widgets.foo\",\n\t\t\t\t\tItemsByNamespace: map[string][]string{\n\t\t\t\t\t\t\"\": {\"item-1\", \"item-2\"},\n\t\t\t\t\t\t\"ns-1\": {\"item-1\", \"item-2\"},\n\t\t\t\t\t\t\"ns-2\": {\"item-1\", \"item-2\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"dongles.foo\": {\n\t\t\t\t\tGroupResource: \"dongles.foo\",\n\t\t\t\t\tItemsByNamespace: map[string][]string{\n\t\t\t\t\t\t\"\": {\"item-3\", \"item-4\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"dongles.bar\": {\n\t\t\t\t\tGroupResource: \"dongles.bar\",\n\t\t\t\t\tItemsByNamespace: map[string][]string{\n\t\t\t\t\t\t\"ns-3\": {\"item-3\", \"item-4\"},\n\t\t\t\t\t\t\"ns-4\": {\"item-5\", \"item-6\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tp := &Parser{\n\t\t\t\tlog: test.NewLogger(),\n\t\t\t\tfs: test.NewFakeFileSystem(),\n\t\t\t}\n\n\t\t\tfor _, file := range tc.files {\n\t\t\t\trequire.NoError(t, p.fs.MkdirAll(file, 0755))\n\n\t\t\t\tif !strings.HasSuffix(file, \"/\") {\n\t\t\t\t\tres, err := p.fs.Create(file)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\trequire.NoError(t, res.Close())\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tres, err := p.Parse(tc.dir)\n\t\t\tif tc.wantErrMsg != nil {\n\t\t\t\tassert.ErrorIs(t, err, tc.wantErrMsg, \"Error should be: %v, got: %v\", tc.wantErrMsg, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, tc.want, res)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestParseGroupVersions(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tfiles []string\n\t\tbackupDir string\n\t\twantErrMsg error\n\t\twant map[string]metav1.APIGroup\n\t}{\n\t\t{\n\t\t\tname: \"when there is no top-level resources directory, an error is returned\",\n\t\t\tbackupDir: \"/var/folders\",\n\t\t\twantErrMsg: ErrNotExist,\n\t\t},\n\t\t{\n\t\t\tname: \"when there are no directories under the resources directory, an empty map is returned\",\n\t\t\tbackupDir: \"/var/folders\",\n\t\t\tfiles: []string{\"/var/folders/resources/\"},\n\t\t\twant: map[string]metav1.APIGroup{},\n\t\t},\n\t\t{\n\t\t\tname: \"when there is a mix of cluster-scoped and namespaced items for resources with preferred or multiple API groups, all group versions are correctly returned\",\n\t\t\tbackupDir: \"/var/folders\",\n\t\t\tfiles: []string{\n\t\t\t\t\"/var/folders/resources/clusterroles.rbac.authorization.k8s.io/v1-preferredversion/cluster/system/controller/attachdetach-controller.json\",\n\t\t\t\t\"/var/folders/resources/clusterroles.rbac.authorization.k8s.io/cluster/system/controller/attachdetach-controller.json\",\n\n\t\t\t\t\"/var/folders/resources/horizontalpodautoscalers.autoscaling/namespaces/myexample/php-apache-autoscaler.json\",\n\t\t\t\t\"/var/folders/resources/horizontalpodautoscalers.autoscaling/v1-preferredversion/namespaces/myexample/php-apache-autoscaler.json\",\n\t\t\t\t\"/var/folders/resources/horizontalpodautoscalers.autoscaling/v2beta1/namespaces/myexample/php-apache-autoscaler.json\",\n\t\t\t\t\"/var/folders/resources/horizontalpodautoscalers.autoscaling/v2beta2/namespaces/myexample/php-apache-autoscaler.json\",\n\n\t\t\t\t\"/var/folders/resources/pods/namespaces/nginx-example/nginx-deployment-57d5dcb68-wrqsc.json\",\n\t\t\t\t\"/var/folders/resources/pods/v1-preferredversion/namespaces/nginx-example/nginx-deployment-57d5dcb68-wrqsc.json\",\n\t\t\t},\n\t\t\twant: map[string]metav1.APIGroup{\n\t\t\t\t\"clusterroles.rbac.authorization.k8s.io\": {\n\t\t\t\t\tName: \"rbac.authorization.k8s.io\",\n\t\t\t\t\tVersions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tGroupVersion: \"rbac.authorization.k8s.io/v1\",\n\t\t\t\t\t\t\tVersion: \"v1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tPreferredVersion: metav1.GroupVersionForDiscovery{\n\t\t\t\t\t\tGroupVersion: \"rbac.authorization.k8s.io/v1\",\n\t\t\t\t\t\tVersion: \"v1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"horizontalpodautoscalers.autoscaling\": {\n\t\t\t\t\tName: \"autoscaling\",\n\t\t\t\t\tVersions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tGroupVersion: \"autoscaling/v1\",\n\t\t\t\t\t\t\tVersion: \"v1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tGroupVersion: \"autoscaling/v2beta1\",\n\t\t\t\t\t\t\tVersion: \"v2beta1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tGroupVersion: \"autoscaling/v2beta2\",\n\t\t\t\t\t\t\tVersion: \"v2beta2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tPreferredVersion: metav1.GroupVersionForDiscovery{\n\t\t\t\t\t\tGroupVersion: \"autoscaling/v1\",\n\t\t\t\t\t\tVersion: \"v1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"pods\": {\n\t\t\t\t\tName: \"\",\n\t\t\t\t\tVersions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tGroupVersion: \"v1\",\n\t\t\t\t\t\t\tVersion: \"v1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tPreferredVersion: metav1.GroupVersionForDiscovery{\n\t\t\t\t\t\tGroupVersion: \"v1\",\n\t\t\t\t\t\tVersion: \"v1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tp := &Parser{\n\t\t\t\tlog: test.NewLogger(),\n\t\t\t\tfs: test.NewFakeFileSystem(),\n\t\t\t}\n\n\t\t\tfor _, file := range tc.files {\n\t\t\t\trequire.NoError(t, p.fs.MkdirAll(file, 0755))\n\n\t\t\t\tif !strings.HasSuffix(file, \"/\") {\n\t\t\t\t\tres, err := p.fs.Create(file)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\trequire.NoError(t, res.Close())\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tres, err := p.ParseGroupVersions(tc.backupDir)\n\t\t\tif tc.wantErrMsg != nil {\n\t\t\t\tassert.ErrorIs(t, err, tc.wantErrMsg, \"Error should be: %v, got: %v\", tc.wantErrMsg, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, tc.want, res)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestExtractGroupName(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\trgDir string\n\t\twant string\n\t}{\n\t\t{\n\t\t\tname: \"Directory has no dots (only a group name)\",\n\t\t\trgDir: \"pods\",\n\t\t\twant: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"Directory has one concatenation dot (has both resource and group name which have 0 dots\",\n\t\t\trgDir: \"cronjobs.batch\",\n\t\t\twant: \"batch\",\n\t\t},\n\t\t{\n\t\t\tname: \"Directory has 3 dots in name (group has 2 dot)\",\n\t\t\trgDir: \"leases.coordination.k8s.io\",\n\t\t\twant: \"coordination.k8s.io\",\n\t\t},\n\t\t{\n\t\t\tname: \"Directory has 4 dots in name (group has 3 dots)\",\n\t\t\trgDir: \"roles.rbac.authorization.k8s.io\",\n\t\t\twant: \"rbac.authorization.k8s.io\",\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tgrp := extractGroupName(tc.rgDir)\n\n\t\t\tassert.Equal(t, tc.want, grp)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/backup/actions/backup_pv_action.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"strings\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/actionhelpers\"\n)\n\n// PVCAction inspects a PersistentVolumeClaim for the PersistentVolume\n// that it references and backs it up\ntype PVCAction struct {\n\tlog logrus.FieldLogger\n}\n\nfunc NewPVCAction(logger logrus.FieldLogger) *PVCAction {\n\treturn &PVCAction{log: logger}\n}\n\nfunc (a *PVCAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"persistentvolumeclaims\"},\n\t}, nil\n}\n\n// Execute finds the PersistentVolume bound by the provided\n// PersistentVolumeClaim, if any, and backs it up\nfunc (a *PVCAction) Execute(item runtime.Unstructured, backup *v1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, error) {\n\ta.log.Info(\"Executing PVCAction\")\n\n\tpvc := new(corev1api.PersistentVolumeClaim)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(item.UnstructuredContent(), &pvc); err != nil {\n\t\treturn nil, nil, errors.Wrap(err, \"unable to convert unstructured item to persistent volume claim\")\n\t}\n\n\tif pvc.Status.Phase != corev1api.ClaimBound || pvc.Spec.VolumeName == \"\" {\n\t\treturn item, nil, nil\n\t}\n\n\t// remove dataSource if exists from prior restored CSI volumes\n\tif pvc.Spec.DataSource != nil {\n\t\tpvc.Spec.DataSource = nil\n\t}\n\tif pvc.Spec.DataSourceRef != nil {\n\t\tpvc.Spec.DataSourceRef = nil\n\t}\n\n\t// When StorageClassName is set to \"\", it means no StorageClass is specified,\n\t// even the default StorageClass is not used. Only keep the Selector for this case.\n\t// https://kubernetes.io/docs/concepts/storage/persistent-volumes/#reserving-a-persistentvolume\n\tif pvc.Spec.StorageClassName == nil || *pvc.Spec.StorageClassName != \"\" {\n\t\t// Clean the selector to make the PVC to dynamically allocate PV.\n\t\tpvc.Spec.Selector = nil\n\t}\n\n\t// Clean stale Velero labels from PVC metadata and selector\n\ta.cleanupStaleVeleroLabels(pvc, backup)\n\n\tpvcMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&pvc)\n\tif err != nil {\n\t\treturn nil, nil, errors.Wrap(err, \"unable to convert pvc to unstructured item\")\n\t}\n\n\treturn &unstructured.Unstructured{Object: pvcMap}, actionhelpers.RelatedItemsForPVC(pvc, a.log), nil\n}\n\n// cleanupStaleVeleroLabels removes stale Velero labels from both the PVC metadata\n// and the selector's match labels to ensure clean backups\nfunc (a *PVCAction) cleanupStaleVeleroLabels(pvc *corev1api.PersistentVolumeClaim, backup *v1.Backup) {\n\t// Clean stale Velero labels from selector match labels\n\tif pvc.Spec.Selector != nil && pvc.Spec.Selector.MatchLabels != nil {\n\t\tfor k := range pvc.Spec.Selector.MatchLabels {\n\t\t\tif strings.HasPrefix(k, \"velero.io/\") {\n\t\t\t\ta.log.Infof(\"Deleting stale Velero label %s from PVC %s selector\", k, pvc.Name)\n\t\t\t\tdelete(pvc.Spec.Selector.MatchLabels, k)\n\t\t\t}\n\t\t}\n\t}\n\n\t// Clean stale Velero labels from main metadata\n\tif pvc.Labels != nil {\n\t\tfor k, v := range pvc.Labels {\n\t\t\t// Only remove labels that are clearly stale from previous operations\n\t\t\tshouldRemove := false\n\n\t\t\t// Always remove restore-name labels as these are from previous restores\n\t\t\tif k == v1.RestoreNameLabel {\n\t\t\t\tshouldRemove = true\n\t\t\t}\n\n\t\t\tif k == v1.MustIncludeAdditionalItemAnnotation {\n\t\t\t\tshouldRemove = true\n\t\t\t}\n\n\t\t\t// Remove backup-name labels that don't match current backup\n\t\t\tif k == v1.BackupNameLabel && v != backup.Name {\n\t\t\t\tshouldRemove = true\n\t\t\t}\n\n\t\t\t// Remove volume-snapshot-name labels from previous CSI backups\n\t\t\t// Note: If this backup creates new CSI snapshots, the CSI action will add them back\n\t\t\tif k == v1.VolumeSnapshotLabel {\n\t\t\t\tshouldRemove = true\n\t\t\t}\n\n\t\t\tif shouldRemove {\n\t\t\t\ta.log.Infof(\"Deleting stale Velero label %s=%s from PVC %s\", k, v, pvc.Name)\n\t\t\t\tdelete(pvc.Labels, k)\n\t\t\t}\n\t\t}\n\t}\n}\n" }, { "path": "pkg/backup/actions/backup_pv_action_test.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestBackupPVAction(t *testing.T) {\n\tpvc := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"spec\": map[string]any{},\n\t\t\t\"status\": map[string]any{},\n\t\t},\n\t}\n\n\tbackup := &v1.Backup{}\n\n\ta := NewPVCAction(velerotest.NewLogger())\n\n\t// no spec.volumeName should result in no error\n\t// and no additional items\n\t_, additional, err := a.Execute(pvc, backup)\n\trequire.NoError(t, err)\n\tassert.Empty(t, additional)\n\n\t// empty spec.volumeName should result in no error\n\t// and no additional items\n\tpvc.Object[\"spec\"].(map[string]any)[\"volumeName\"] = \"\"\n\t_, additional, err = a.Execute(pvc, backup)\n\trequire.NoError(t, err)\n\tassert.Empty(t, additional)\n\n\t// Action should clean the spec.Selector when the StorageClassName is not set.\n\tinput := builder.ForPersistentVolumeClaim(\"abc\", \"abc\").VolumeName(\"pv\").Selector(&metav1.LabelSelector{MatchLabels: map[string]string{\"abc\": \"abc\"}}).Phase(corev1api.ClaimBound).Result()\n\tinputUnstructured, err := runtime.DefaultUnstructuredConverter.ToUnstructured(input)\n\trequire.NoError(t, err)\n\titem, additional, err := a.Execute(&unstructured.Unstructured{Object: inputUnstructured}, backup)\n\trequire.NoError(t, err)\n\trequire.Len(t, additional, 1)\n\tmodifiedPVC := new(corev1api.PersistentVolumeClaim)\n\trequire.NoError(t, runtime.DefaultUnstructuredConverter.FromUnstructured(item.UnstructuredContent(), modifiedPVC))\n\trequire.Nil(t, modifiedPVC.Spec.Selector)\n\n\t// Action should clean the spec.Selector when the StorageClassName is set to specific StorageClass\n\tinput2 := builder.ForPersistentVolumeClaim(\"abc\", \"abc\").VolumeName(\"pv\").StorageClass(\"sc1\").Selector(&metav1.LabelSelector{MatchLabels: map[string]string{\"abc\": \"abc\"}}).Phase(corev1api.ClaimBound).Result()\n\tinputUnstructured2, err2 := runtime.DefaultUnstructuredConverter.ToUnstructured(input2)\n\trequire.NoError(t, err2)\n\titem2, additional2, err2 := a.Execute(&unstructured.Unstructured{Object: inputUnstructured2}, backup)\n\trequire.NoError(t, err2)\n\trequire.Len(t, additional2, 1)\n\tmodifiedPVC2 := new(corev1api.PersistentVolumeClaim)\n\trequire.NoError(t, runtime.DefaultUnstructuredConverter.FromUnstructured(item2.UnstructuredContent(), modifiedPVC2))\n\trequire.Nil(t, modifiedPVC2.Spec.Selector)\n\n\t// Action should keep the spec.Selector when the StorageClassName is set to \"\"\n\tinput3 := builder.ForPersistentVolumeClaim(\"abc\", \"abc\").StorageClass(\"\").Selector(&metav1.LabelSelector{MatchLabels: map[string]string{\"abc\": \"abc\"}}).VolumeName(\"pv\").Phase(corev1api.ClaimBound).Result()\n\tinputUnstructured3, err3 := runtime.DefaultUnstructuredConverter.ToUnstructured(input3)\n\trequire.NoError(t, err3)\n\titem3, additional3, err3 := a.Execute(&unstructured.Unstructured{Object: inputUnstructured3}, backup)\n\trequire.NoError(t, err3)\n\trequire.Len(t, additional3, 1)\n\tmodifiedPVC3 := new(corev1api.PersistentVolumeClaim)\n\trequire.NoError(t, runtime.DefaultUnstructuredConverter.FromUnstructured(item3.UnstructuredContent(), modifiedPVC3))\n\trequire.Equal(t, input3.Spec.Selector, modifiedPVC3.Spec.Selector)\n\n\t// Action should delete label started with\"velero.io/\" from the spec.Selector when the StorageClassName is set to \"\"\n\tinput4 := builder.ForPersistentVolumeClaim(\"abc\", \"abc\").StorageClass(\"\").Selector(&metav1.LabelSelector{MatchLabels: map[string]string{\"velero.io/abc\": \"abc\", \"abc\": \"abc\"}}).VolumeName(\"pv\").Phase(corev1api.ClaimBound).Result()\n\tinputUnstructured4, err4 := runtime.DefaultUnstructuredConverter.ToUnstructured(input4)\n\trequire.NoError(t, err4)\n\titem4, additional4, err4 := a.Execute(&unstructured.Unstructured{Object: inputUnstructured4}, backup)\n\trequire.NoError(t, err4)\n\trequire.Len(t, additional4, 1)\n\tmodifiedPVC4 := new(corev1api.PersistentVolumeClaim)\n\trequire.NoError(t, runtime.DefaultUnstructuredConverter.FromUnstructured(item4.UnstructuredContent(), modifiedPVC4))\n\trequire.Equal(t, &metav1.LabelSelector{MatchLabels: map[string]string{\"abc\": \"abc\"}}, modifiedPVC4.Spec.Selector)\n\n\t// Action should clean the spec.Selector when the StorageClassName has value\n\tinput5 := builder.ForPersistentVolumeClaim(\"abc\", \"abc\").StorageClass(\"sc1\").Selector(&metav1.LabelSelector{MatchLabels: map[string]string{\"velero.io/abc\": \"abc\", \"abc\": \"abc\"}}).VolumeName(\"pv\").Phase(corev1api.ClaimBound).Result()\n\tinputUnstructured5, err5 := runtime.DefaultUnstructuredConverter.ToUnstructured(input5)\n\trequire.NoError(t, err5)\n\titem5, additional5, err5 := a.Execute(&unstructured.Unstructured{Object: inputUnstructured5}, backup)\n\trequire.NoError(t, err5)\n\trequire.Len(t, additional5, 1)\n\tmodifiedPVC5 := new(corev1api.PersistentVolumeClaim)\n\trequire.NoError(t, runtime.DefaultUnstructuredConverter.FromUnstructured(item5.UnstructuredContent(), modifiedPVC5))\n\trequire.Nil(t, modifiedPVC5.Spec.Selector)\n\n\t// non-empty spec.volumeName when status.phase is empty\n\t// should result in no error and no additional items\n\tpvc.Object[\"spec\"].(map[string]any)[\"volumeName\"] = \"myVolume\"\n\t_, additional, err = a.Execute(pvc, backup)\n\trequire.NoError(t, err)\n\trequire.Empty(t, additional)\n\n\t// non-empty spec.volumeName when status.phase is 'Pending'\n\t// should result in no error and no additional items\n\tpvc.Object[\"status\"].(map[string]any)[\"phase\"] = corev1api.ClaimPending\n\t_, additional, err = a.Execute(pvc, backup)\n\trequire.NoError(t, err)\n\trequire.Empty(t, additional)\n\n\t// non-empty spec.volumeName when status.phase is 'Lost'\n\t// should result in no error and no additional items\n\tpvc.Object[\"status\"].(map[string]any)[\"phase\"] = corev1api.ClaimLost\n\t_, additional, err = a.Execute(pvc, backup)\n\trequire.NoError(t, err)\n\trequire.Empty(t, additional)\n\n\t// non-empty spec.volumeName when status.phase is 'Bound'\n\t// should result in no error and one additional item for the PV\n\tpvc.Object[\"status\"].(map[string]any)[\"phase\"] = corev1api.ClaimBound\n\t_, additional, err = a.Execute(pvc, backup)\n\trequire.NoError(t, err)\n\trequire.Len(t, additional, 1)\n\tassert.Equal(t, velero.ResourceIdentifier{GroupResource: kuberesource.PersistentVolumes, Name: \"myVolume\"}, additional[0])\n\n\t// empty spec.volumeName when status.phase is 'Bound' should\n\t// result in no error and no additional items\n\tpvc.Object[\"spec\"].(map[string]any)[\"volumeName\"] = \"\"\n\t_, additional, err = a.Execute(pvc, backup)\n\trequire.NoError(t, err)\n\tassert.Empty(t, additional)\n}\n\nfunc TestCleanupStaleVeleroLabels(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tinputPVC *corev1api.PersistentVolumeClaim\n\t\tbackup *v1.Backup\n\t\texpectedLabels map[string]string\n\t\texpectedSelector *metav1.LabelSelector\n\t}{\n\t\t{\n\t\t\tname: \"removes restore-name labels\",\n\t\t\tinputPVC: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-pvc\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/restore-name\": \"old-restore\",\n\t\t\t\t\t\t\"app\": \"myapp\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tbackup: &v1.Backup{ObjectMeta: metav1.ObjectMeta{Name: \"current-backup\"}},\n\t\t\texpectedLabels: map[string]string{\n\t\t\t\t\"app\": \"myapp\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"removes backup-name labels that don't match current backup\",\n\t\t\tinputPVC: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-pvc\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/backup-name\": \"old-backup\",\n\t\t\t\t\t\t\"app\": \"myapp\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tbackup: &v1.Backup{ObjectMeta: metav1.ObjectMeta{Name: \"current-backup\"}},\n\t\t\texpectedLabels: map[string]string{\n\t\t\t\t\"app\": \"myapp\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"keeps backup-name labels that match current backup\",\n\t\t\tinputPVC: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-pvc\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/backup-name\": \"current-backup\",\n\t\t\t\t\t\t\"app\": \"myapp\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tbackup: &v1.Backup{ObjectMeta: metav1.ObjectMeta{Name: \"current-backup\"}},\n\t\t\texpectedLabels: map[string]string{\n\t\t\t\t\"velero.io/backup-name\": \"current-backup\",\n\t\t\t\t\"app\": \"myapp\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"removes volume-snapshot-name labels\",\n\t\t\tinputPVC: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-pvc\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/volume-snapshot-name\": \"old-snapshot\",\n\t\t\t\t\t\t\"app\": \"myapp\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tbackup: &v1.Backup{ObjectMeta: metav1.ObjectMeta{Name: \"current-backup\"}},\n\t\t\texpectedLabels: map[string]string{\n\t\t\t\t\"app\": \"myapp\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"removes velero labels from selector match labels\",\n\t\t\tinputPVC: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-pvc\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\tSelector: &metav1.LabelSelector{\n\t\t\t\t\t\tMatchLabels: map[string]string{\n\t\t\t\t\t\t\t\"velero.io/restore-name\": \"old-restore\",\n\t\t\t\t\t\t\t\"velero.io/backup-name\": \"old-backup\",\n\t\t\t\t\t\t\t\"app\": \"myapp\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tbackup: &v1.Backup{ObjectMeta: metav1.ObjectMeta{Name: \"current-backup\"}},\n\t\t\texpectedLabels: nil,\n\t\t\texpectedSelector: &metav1.LabelSelector{\n\t\t\t\tMatchLabels: map[string]string{\n\t\t\t\t\t\"app\": \"myapp\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"handles PVC with no labels\",\n\t\t\tinputPVC: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-pvc\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tbackup: &v1.Backup{ObjectMeta: metav1.ObjectMeta{Name: \"current-backup\"}},\n\t\t\texpectedLabels: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"handles PVC with no selector\",\n\t\t\tinputPVC: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-pvc\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"app\": \"myapp\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tbackup: &v1.Backup{ObjectMeta: metav1.ObjectMeta{Name: \"current-backup\"}},\n\t\t\texpectedLabels: map[string]string{\n\t\t\t\t\"app\": \"myapp\",\n\t\t\t},\n\t\t\texpectedSelector: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"removes multiple stale velero labels\",\n\t\t\tinputPVC: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-pvc\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/restore-name\": \"old-restore\",\n\t\t\t\t\t\t\"velero.io/backup-name\": \"old-backup\",\n\t\t\t\t\t\t\"velero.io/volume-snapshot-name\": \"old-snapshot\",\n\t\t\t\t\t\t\"app\": \"myapp\",\n\t\t\t\t\t\t\"env\": \"prod\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\tSelector: &metav1.LabelSelector{\n\t\t\t\t\t\tMatchLabels: map[string]string{\n\t\t\t\t\t\t\t\"velero.io/restore-name\": \"old-restore\",\n\t\t\t\t\t\t\t\"app\": \"myapp\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tbackup: &v1.Backup{ObjectMeta: metav1.ObjectMeta{Name: \"current-backup\"}},\n\t\t\texpectedLabels: map[string]string{\n\t\t\t\t\"app\": \"myapp\",\n\t\t\t\t\"env\": \"prod\",\n\t\t\t},\n\t\t\texpectedSelector: &metav1.LabelSelector{\n\t\t\t\tMatchLabels: map[string]string{\n\t\t\t\t\t\"app\": \"myapp\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\taction := NewPVCAction(velerotest.NewLogger())\n\n\t\t\t// Create a copy of the input PVC to avoid modifying the test case\n\t\t\tpvcCopy := tc.inputPVC.DeepCopy()\n\n\t\t\taction.cleanupStaleVeleroLabels(pvcCopy, tc.backup)\n\n\t\t\tassert.Equal(t, tc.expectedLabels, pvcCopy.Labels, \"Labels should match expected values\")\n\t\t\tassert.Equal(t, tc.expectedSelector, pvcCopy.Spec.Selector, \"Selector should match expected values\")\n\t\t})\n\t}\n}\n" }, { "path": "pkg/backup/actions/csi/pvc_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strconv\"\n\t\"time\"\n\n\t\"k8s.io/client-go/util/retry\"\n\n\tvolumegroupsnapshotv1beta1 \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumegroupsnapshot/v1beta1\"\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tstoragev1api \"k8s.io/api/storage/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t_ \"k8s.io/client-go/plugin/pkg/client/auth/gcp\"\n\tcrclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil\"\n\n\t\"k8s.io/apimachinery/pkg/api/resource\"\n\n\tinternalvolumehelper \"github.com/vmware-tanzu/velero/internal/volumehelper\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\tveleroclient \"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\tplugincommon \"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/utils/volumehelper\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tbiav2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/backupitemaction/v2\"\n\tuploaderUtil \"github.com/vmware-tanzu/velero/pkg/uploader/util\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/csi\"\n\tkubeutil \"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\tpodvolumeutil \"github.com/vmware-tanzu/velero/pkg/util/podvolume\"\n)\n\n// TODO: Replace hardcoded VolumeSnapshot finalizer strings with constants from\n// \"github.com/kubernetes-csi/external-snapshotter/v8/pkg/utils\"\n// once module/toolchain upgrades are done.\n// Finalizer constants\nconst (\n\tVolumeSnapshotFinalizerGroupProtection = \"snapshot.storage.kubernetes.io/volumesnapshot-in-group-protection\"\n\tVolumeSnapshotFinalizerSourceProtection = \"snapshot.storage.kubernetes.io/volumesnapshot-as-source-protection\"\n)\n\n// pvcBackupItemAction is a backup item action plugin for Velero.\ntype pvcBackupItemAction struct {\n\tlog logrus.FieldLogger\n\tcrClient crclient.Client\n\n\t// pvcPodCache provides lazy per-namespace caching of PVC-to-Pod mappings.\n\t// Since plugin instances are unique per backup (created via newPluginManager and\n\t// cleaned up via CleanupClients at backup completion), we can safely cache this\n\t// without mutex or backup UID tracking.\n\t// This avoids the O(N*M) performance issue when there are many PVCs and pods.\n\t// See issue #9179 and PR #9226 for details.\n\tpvcPodCache *podvolumeutil.PVCPodCache\n}\n\n// AppliesTo returns information indicating that the PVCBackupItemAction\n// should be invoked to backup PVCs.\nfunc (p *pvcBackupItemAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"persistentvolumeclaims\"},\n\t}, nil\n}\n\nfunc (p *pvcBackupItemAction) validateBackup(backup velerov1api.Backup) (valid bool) {\n\tif backup.Status.Phase == velerov1api.BackupPhaseFinalizing ||\n\t\tbackup.Status.Phase == velerov1api.BackupPhaseFinalizingPartiallyFailed {\n\t\tp.log.WithFields(\n\t\t\tlogrus.Fields{\n\t\t\t\t\"Backup\": fmt.Sprintf(\"%s/%s\", backup.Namespace, backup.Name),\n\t\t\t\t\"Phase\": backup.Status.Phase,\n\t\t\t},\n\t\t).Debug(\"Backup is in finalizing phase. Skip this PVC.\")\n\t\treturn false\n\t}\n\n\treturn true\n}\n\n// ensurePVCPodCacheForNamespace ensures the PVC-to-Pod cache is built for the given namespace.\n// This uses lazy per-namespace caching following the pattern from PR #9226.\n// Since plugin instances are unique per backup, we can safely cache without mutex or backup UID tracking.\nfunc (p *pvcBackupItemAction) ensurePVCPodCacheForNamespace(ctx context.Context, namespace string) error {\n\t// Initialize cache if needed\n\tif p.pvcPodCache == nil {\n\t\tp.pvcPodCache = podvolumeutil.NewPVCPodCache()\n\t}\n\n\t// Build cache for namespace if not already done\n\tif !p.pvcPodCache.IsNamespaceBuilt(namespace) {\n\t\tp.log.Debugf(\"Building PVC-to-Pod cache for namespace %s\", namespace)\n\t\tif err := p.pvcPodCache.BuildCacheForNamespace(ctx, namespace, p.crClient); err != nil {\n\t\t\treturn errors.Wrapf(err, \"failed to build PVC-to-Pod cache for namespace %s\", namespace)\n\t\t}\n\t}\n\treturn nil\n}\n\n// getVolumeHelperWithCache creates a VolumeHelper using the pre-built PVC-to-Pod cache.\n// The cache should be ensured for the relevant namespace(s) before calling this.\nfunc (p *pvcBackupItemAction) getVolumeHelperWithCache(backup *velerov1api.Backup) (internalvolumehelper.VolumeHelper, error) {\n\t// Create VolumeHelper with our lazy-built cache\n\tvh, err := internalvolumehelper.NewVolumeHelperImplWithCache(\n\t\t*backup,\n\t\tp.crClient,\n\t\tp.log,\n\t\tp.pvcPodCache,\n\t)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to create VolumeHelper\")\n\t}\n\treturn vh, nil\n}\n\n// getOrCreateVolumeHelper returns a VolumeHelper with lazy per-namespace caching.\n// The VolumeHelper uses the pvcPodCache which is populated lazily as namespaces are encountered.\n// Callers should use ensurePVCPodCacheForNamespace before calling methods that need\n// PVC-to-Pod lookups for a specific namespace.\n// Since plugin instances are unique per backup (created via newPluginManager and\n// cleaned up via CleanupClients at backup completion), we can safely cache this.\n// See issue #9179 and PR #9226 for details.\nfunc (p *pvcBackupItemAction) getOrCreateVolumeHelper(backup *velerov1api.Backup) (internalvolumehelper.VolumeHelper, error) {\n\t// Initialize the PVC-to-Pod cache if needed\n\tif p.pvcPodCache == nil {\n\t\tp.pvcPodCache = podvolumeutil.NewPVCPodCache()\n\t}\n\n\t// Return the VolumeHelper with our lazily-built cache\n\t// The cache will be populated incrementally as namespaces are encountered\n\treturn p.getVolumeHelperWithCache(backup)\n}\n\nfunc (p *pvcBackupItemAction) validatePVCandPV(\n\tpvc corev1api.PersistentVolumeClaim,\n\titem runtime.Unstructured,\n) (\n\tvalid bool,\n\tupdateItem runtime.Unstructured,\n\terr error,\n) {\n\tupdateItem = item\n\n\t// no storage class: we don't know how to map to a VolumeSnapshotClass\n\tif pvc.Spec.StorageClassName == nil {\n\t\treturn false,\n\t\t\tupdateItem,\n\t\t\terrors.Errorf(\n\t\t\t\t\"Cannot snapshot PVC %s/%s, PVC has no storage class.\",\n\t\t\t\tpvc.Namespace, pvc.Name)\n\t}\n\n\tp.log.Debugf(\n\t\t\"Fetching underlying PV for PVC %s\",\n\t\tfmt.Sprintf(\"%s/%s\", pvc.Namespace, pvc.Name),\n\t)\n\n\t// Do nothing if this is not a CSI provisioned volume\n\tpv, err := kubeutil.GetPVForPVC(&pvc, p.crClient)\n\tif err != nil {\n\t\treturn false, updateItem, errors.WithStack(err)\n\t}\n\n\tif pv.Spec.PersistentVolumeSource.CSI == nil {\n\t\tp.log.Infof(\n\t\t\t\"Skipping PVC %s/%s, associated PV %s is not a CSI volume\",\n\t\t\tpvc.Namespace, pvc.Name, pv.Name)\n\n\t\tkubeutil.AddAnnotations(\n\t\t\t&pvc.ObjectMeta,\n\t\t\tmap[string]string{\n\t\t\t\tvelerov1api.SkippedNoCSIPVAnnotation: \"true\",\n\t\t\t})\n\t\tdata, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&pvc)\n\t\tupdateItem = &unstructured.Unstructured{Object: data}\n\t\treturn false, updateItem, err\n\t}\n\n\treturn true, updateItem, nil\n}\n\nfunc (p *pvcBackupItemAction) createVolumeSnapshot(\n\tpvc corev1api.PersistentVolumeClaim,\n\tbackup *velerov1api.Backup,\n) (\n\tvs *snapshotv1api.VolumeSnapshot,\n\terr error,\n) {\n\tp.log.Debugf(\"Fetching storage class for PV %s\", *pvc.Spec.StorageClassName)\n\tstorageClass := new(storagev1api.StorageClass)\n\tif err := p.crClient.Get(\n\t\tcontext.TODO(), crclient.ObjectKey{Name: *pvc.Spec.StorageClassName},\n\t\tstorageClass,\n\t); err != nil {\n\t\treturn nil, errors.Wrap(err, \"error getting storage class\")\n\t}\n\n\tp.log.Debugf(\"Fetching VolumeSnapshotClass for %s\", storageClass.Provisioner)\n\tvsClass, err := csi.GetVolumeSnapshotClass(\n\t\tstorageClass.Provisioner,\n\t\tbackup,\n\t\t&pvc,\n\t\tp.log,\n\t\tp.crClient,\n\t)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(\n\t\t\terr, \"failed to get VolumeSnapshotClass for StorageClass %s\",\n\t\t\tstorageClass.Name,\n\t\t)\n\t}\n\tp.log.Infof(\"VolumeSnapshotClass=%s\", vsClass.Name)\n\n\tvsLabels := map[string]string{}\n\tfor k, v := range pvc.ObjectMeta.Labels {\n\t\tvsLabels[k] = v\n\t}\n\tvsLabels[velerov1api.BackupNameLabel] = label.GetValidName(backup.Name)\n\n\t// Craft the vs object to be created\n\tvs = &snapshotv1api.VolumeSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tGenerateName: \"velero-\" + pvc.Name + \"-\",\n\t\t\tNamespace: pvc.Namespace,\n\t\t\tLabels: vsLabels,\n\t\t},\n\t\tSpec: snapshotv1api.VolumeSnapshotSpec{\n\t\t\tSource: snapshotv1api.VolumeSnapshotSource{\n\t\t\t\tPersistentVolumeClaimName: &pvc.Name,\n\t\t\t},\n\t\t\tVolumeSnapshotClassName: &vsClass.Name,\n\t\t},\n\t}\n\n\tif err := p.crClient.Create(context.TODO(), vs); err != nil {\n\t\treturn nil, errors.Wrapf(\n\t\t\terr, \"error creating volume snapshot\",\n\t\t)\n\t}\n\tp.log.Infof(\n\t\t\"Created VolumeSnapshot %s\",\n\t\tfmt.Sprintf(\"%s/%s\", vs.Namespace, vs.Name),\n\t)\n\n\treturn vs, nil\n}\n\n// Execute recognizes PVCs backed by volumes provisioned by CSI drivers\n// with VolumeSnapshotting capability and creates snapshots of the\n// underlying PVs by creating VolumeSnapshot CSI API objects that will\n// trigger the CSI driver to perform the snapshot operation on the volume.\nfunc (p *pvcBackupItemAction) Execute(\n\titem runtime.Unstructured,\n\tbackup *velerov1api.Backup,\n) (\n\truntime.Unstructured,\n\t[]velero.ResourceIdentifier,\n\tstring,\n\t[]velero.ResourceIdentifier,\n\terror,\n) {\n\tp.log.Info(\"Starting PVCBackupItemAction\")\n\n\tif valid := p.validateBackup(*backup); !valid {\n\t\treturn item, nil, \"\", nil, nil\n\t}\n\n\tvar pvc corev1api.PersistentVolumeClaim\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(\n\t\titem.UnstructuredContent(),\n\t\t&pvc,\n\t); err != nil {\n\t\treturn nil, nil, \"\", nil, errors.WithStack(err)\n\t}\n\tif valid, item, err := p.validatePVCandPV(\n\t\tpvc,\n\t\titem,\n\t); !valid {\n\t\tif err != nil {\n\t\t\treturn nil, nil, \"\", nil, err\n\t\t}\n\t\treturn item, nil, \"\", nil, nil\n\t}\n\n\t// Ensure PVC-to-Pod cache is built for this namespace (lazy per-namespace caching)\n\tif err := p.ensurePVCPodCacheForNamespace(context.TODO(), pvc.Namespace); err != nil {\n\t\treturn nil, nil, \"\", nil, err\n\t}\n\n\t// Get or create the cached VolumeHelper for this backup\n\tvh, err := p.getOrCreateVolumeHelper(backup)\n\tif err != nil {\n\t\treturn nil, nil, \"\", nil, err\n\t}\n\n\tshouldSnapshot, err := volumehelper.ShouldPerformSnapshotWithVolumeHelper(\n\t\titem,\n\t\tkuberesource.PersistentVolumeClaims,\n\t\t*backup,\n\t\tp.crClient,\n\t\tp.log,\n\t\tvh,\n\t)\n\tif err != nil {\n\t\treturn nil, nil, \"\", nil, err\n\t}\n\tif !shouldSnapshot {\n\t\tp.log.Debugf(\"CSI plugin skip snapshot for PVC %s according to the VolumeHelper setting.\",\n\t\t\tpvc.Namespace+\"/\"+pvc.Name)\n\t\treturn nil, nil, \"\", nil, err\n\t}\n\n\tvs, err := p.getVolumeSnapshotReference(context.TODO(), pvc, backup)\n\tif err != nil {\n\t\treturn nil, nil, \"\", nil, err\n\t}\n\n\t// Wait until VS associated VSC snapshot handle created before\n\t// continue.we later require the vsc restore size\n\tvsc, err := csi.WaitUntilVSCHandleIsReady(\n\t\tvs,\n\t\tp.crClient,\n\t\tp.log,\n\t\tbackup.Spec.CSISnapshotTimeout.Duration,\n\t)\n\tif err != nil {\n\t\tp.log.Errorf(\"Failed to wait for VolumeSnapshot %s/%s to become ReadyToUse within timeout %v: %s\",\n\t\t\tvs.Namespace, vs.Name, backup.Spec.CSISnapshotTimeout.Duration, err.Error())\n\t\tcsi.CleanupVolumeSnapshot(vs, p.crClient, p.log)\n\t\treturn nil, nil, \"\", nil, errors.WithStack(err)\n\t}\n\n\tlabels := map[string]string{\n\t\tvelerov1api.VolumeSnapshotLabel: vs.Name,\n\t\tvelerov1api.BackupNameLabel: backup.Name,\n\t}\n\n\tannotations := map[string]string{\n\t\tvelerov1api.VolumeSnapshotLabel: vs.Name,\n\t\tvelerov1api.MustIncludeAdditionalItemAnnotation: \"true\",\n\t}\n\n\tvar additionalItems []velero.ResourceIdentifier\n\toperationID := \"\"\n\tvar itemToUpdate []velero.ResourceIdentifier\n\n\tif boolptr.IsSetToTrue(backup.Spec.SnapshotMoveData) {\n\t\toperationID = label.GetValidName(\n\t\t\tstring(\n\t\t\t\tvelerov1api.AsyncOperationIDPrefixDataUpload,\n\t\t\t) + string(backup.UID) + \".\" + string(pvc.UID),\n\t\t)\n\t\tdataUploadLog := p.log.WithFields(logrus.Fields{\n\t\t\t\"Source PVC\": fmt.Sprintf(\"%s/%s\", pvc.Namespace, pvc.Name),\n\t\t\t\"VolumeSnapshot\": fmt.Sprintf(\"%s/%s\", vs.Namespace, vs.Name),\n\t\t\t\"Operation ID\": operationID,\n\t\t\t\"Backup\": backup.Name,\n\t\t})\n\n\t\tdataUploadLog.Info(\"Starting data upload of backup\")\n\n\t\tdataUpload, err := createDataUpload(\n\t\t\tcontext.Background(),\n\t\t\tbackup,\n\t\t\tp.crClient,\n\t\t\tvs,\n\t\t\t&pvc,\n\t\t\toperationID,\n\t\t\tvsc,\n\t\t)\n\t\tif err != nil {\n\t\t\tdataUploadLog.WithError(err).Error(\"failed to submit DataUpload\")\n\n\t\t\t// TODO: need to use DeleteVolumeSnapshotIfAny, after data mover\n\t\t\t// adopting the controller-runtime client.\n\t\t\tif deleteErr := p.crClient.Delete(context.TODO(), vs); deleteErr != nil {\n\t\t\t\tif !apierrors.IsNotFound(deleteErr) {\n\t\t\t\t\tdataUploadLog.WithError(deleteErr).Error(\"fail to delete VolumeSnapshot\")\n\t\t\t\t}\n\t\t\t}\n\n\t\t\t// Return without modification to not fail the backup,\n\t\t\t// and the above error log makes the backup partially fail.\n\t\t\treturn item, nil, \"\", nil, nil\n\t\t} else {\n\t\t\titemToUpdate = []velero.ResourceIdentifier{\n\t\t\t\t{\n\t\t\t\t\tGroupResource: schema.GroupResource{\n\t\t\t\t\t\tGroup: \"velero.io\",\n\t\t\t\t\t\tResource: \"datauploads\",\n\t\t\t\t\t},\n\t\t\t\t\tNamespace: dataUpload.Namespace,\n\t\t\t\t\tName: dataUpload.Name,\n\t\t\t\t},\n\t\t\t}\n\t\t\t// Set the DataUploadNameLabel, which is used for restore to\n\t\t\t// let CSI plugin check whether it should handle the volume.\n\t\t\t// If volume is CSI migration, PVC doesn't have the annotation.\n\t\t\tannotations[velerov1api.DataUploadNameAnnotation] = dataUpload.Namespace + \"/\" + dataUpload.Name\n\n\t\t\tdataUploadLog.Info(\"DataUpload is submitted successfully.\")\n\t\t}\n\t} else {\n\t\tsetPVCRequestSizeToVSRestoreSize(&pvc, vsc, p.log)\n\n\t\tadditionalItems = []velero.ResourceIdentifier{\n\t\t\t{\n\t\t\t\tGroupResource: kuberesource.VolumeSnapshots,\n\t\t\t\tNamespace: vs.Namespace,\n\t\t\t\tName: vs.Name,\n\t\t\t},\n\t\t}\n\t}\n\n\tkubeutil.AddAnnotations(&pvc.ObjectMeta, annotations)\n\tkubeutil.AddLabels(&pvc.ObjectMeta, labels)\n\n\tp.log.Infof(\"Returning from PVCBackupItemAction with %d additionalItems to backup\",\n\t\tlen(additionalItems))\n\tfor _, ai := range additionalItems {\n\t\tp.log.Debugf(\"%s: %s\", ai.GroupResource.String(), ai.Name)\n\t}\n\n\tpvcMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&pvc)\n\tif err != nil {\n\t\treturn nil, nil, \"\", nil, errors.WithStack(err)\n\t}\n\treturn &unstructured.Unstructured{Object: pvcMap},\n\t\tadditionalItems, operationID, itemToUpdate, nil\n}\n\nfunc (p *pvcBackupItemAction) Name() string {\n\treturn \"PVCBackupItemAction\"\n}\n\nfunc (p *pvcBackupItemAction) Progress(\n\toperationID string,\n\tbackup *velerov1api.Backup,\n) (velero.OperationProgress, error) {\n\tprogress := velero.OperationProgress{}\n\tif operationID == \"\" {\n\t\treturn progress, biav2.InvalidOperationIDError(operationID)\n\t}\n\n\tdataUpload, err := getDataUpload(context.Background(), p.crClient, operationID)\n\tif err != nil {\n\t\tp.log.Errorf(\n\t\t\t\"fail to get DataUpload for backup %s/%s by operation ID %s: %s\",\n\t\t\tbackup.Namespace, backup.Name, operationID, err.Error(),\n\t\t)\n\t\treturn progress, err\n\t}\n\tif dataUpload.Status.Phase == velerov2alpha1.DataUploadPhaseNew ||\n\t\tdataUpload.Status.Phase == \"\" {\n\t\tp.log.Debugf(\"DataUpload is still not processed yet. Skip progress update.\")\n\t\treturn progress, nil\n\t}\n\n\tprogress.Description = string(dataUpload.Status.Phase)\n\tprogress.OperationUnits = \"Bytes\"\n\tprogress.NCompleted = dataUpload.Status.Progress.BytesDone\n\tprogress.NTotal = dataUpload.Status.Progress.TotalBytes\n\n\tif dataUpload.Status.StartTimestamp != nil {\n\t\tprogress.Started = dataUpload.Status.StartTimestamp.Time\n\t}\n\n\tif dataUpload.Status.CompletionTimestamp != nil {\n\t\tprogress.Updated = dataUpload.Status.CompletionTimestamp.Time\n\t}\n\n\tif dataUpload.Status.Phase == velerov2alpha1.DataUploadPhaseCompleted {\n\t\tprogress.Completed = true\n\t} else if dataUpload.Status.Phase == velerov2alpha1.DataUploadPhaseFailed {\n\t\tprogress.Completed = true\n\t\tprogress.Err = dataUpload.Status.Message\n\t} else if dataUpload.Status.Phase == velerov2alpha1.DataUploadPhaseCanceled {\n\t\tprogress.Completed = true\n\t\tprogress.Err = \"DataUpload is canceled\"\n\t}\n\n\treturn progress, nil\n}\n\nfunc (p *pvcBackupItemAction) Cancel(operationID string, backup *velerov1api.Backup) error {\n\tif operationID == \"\" {\n\t\treturn biav2.InvalidOperationIDError(operationID)\n\t}\n\n\tdataUpload, err := getDataUpload(context.Background(), p.crClient, operationID)\n\tif err != nil {\n\t\tp.log.Errorf(\n\t\t\t\"fail to get DataUpload for backup %s/%s: %s\",\n\t\t\tbackup.Namespace, backup.Name, err.Error(),\n\t\t)\n\t\treturn err\n\t}\n\n\treturn cancelDataUpload(context.Background(), p.crClient, dataUpload)\n}\n\nfunc newDataUpload(\n\tbackup *velerov1api.Backup,\n\tvs *snapshotv1api.VolumeSnapshot,\n\tpvc *corev1api.PersistentVolumeClaim,\n\toperationID string,\n\tvsc *snapshotv1api.VolumeSnapshotContent,\n) *velerov2alpha1.DataUpload {\n\tdataUpload := &velerov2alpha1.DataUpload{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: velerov2alpha1.SchemeGroupVersion.String(),\n\t\t\tKind: \"DataUpload\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: backup.Namespace,\n\t\t\tGenerateName: backup.Name + \"-\",\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tName: backup.Name,\n\t\t\t\t\tUID: backup.UID,\n\t\t\t\t\tController: boolptr.True(),\n\t\t\t\t},\n\t\t\t},\n\t\t\tLabels: map[string]string{\n\t\t\t\tvelerov1api.BackupNameLabel: label.GetValidName(backup.Name),\n\t\t\t\tvelerov1api.BackupUIDLabel: string(backup.UID),\n\t\t\t\tvelerov1api.PVCUIDLabel: string(pvc.UID),\n\t\t\t\tvelerov1api.AsyncOperationIDLabel: operationID,\n\t\t\t},\n\t\t},\n\t\tSpec: velerov2alpha1.DataUploadSpec{\n\t\t\tSnapshotType: velerov2alpha1.SnapshotTypeCSI,\n\t\t\tCSISnapshot: &velerov2alpha1.CSISnapshotSpec{\n\t\t\t\tVolumeSnapshot: vs.Name,\n\t\t\t\tStorageClass: *pvc.Spec.StorageClassName,\n\t\t\t\tDriver: vsc.Spec.Driver,\n\t\t\t},\n\t\t\tSourcePVC: pvc.Name,\n\t\t\tDataMover: backup.Spec.DataMover,\n\t\t\tBackupStorageLocation: backup.Spec.StorageLocation,\n\t\t\tSourceNamespace: pvc.Namespace,\n\t\t\tOperationTimeout: backup.Spec.CSISnapshotTimeout,\n\t\t},\n\t}\n\n\tif vs.Spec.VolumeSnapshotClassName != nil {\n\t\tdataUpload.Spec.CSISnapshot.SnapshotClass = *vs.Spec.VolumeSnapshotClassName\n\t}\n\n\tif backup.Spec.UploaderConfig != nil &&\n\t\tbackup.Spec.UploaderConfig.ParallelFilesUpload > 0 {\n\t\tdataUpload.Spec.DataMoverConfig = make(map[string]string)\n\t\tdataUpload.Spec.DataMoverConfig[uploaderUtil.ParallelFilesUpload] = strconv.Itoa(backup.Spec.UploaderConfig.ParallelFilesUpload)\n\t}\n\n\treturn dataUpload\n}\n\nfunc createDataUpload(\n\tctx context.Context,\n\tbackup *velerov1api.Backup,\n\tcrClient crclient.Client,\n\tvs *snapshotv1api.VolumeSnapshot,\n\tpvc *corev1api.PersistentVolumeClaim,\n\toperationID string,\n\tvsc *snapshotv1api.VolumeSnapshotContent,\n) (*velerov2alpha1.DataUpload, error) {\n\tdataUpload := newDataUpload(backup, vs, pvc, operationID, vsc)\n\n\terr := crClient.Create(ctx, dataUpload)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"fail to create DataUpload CR\")\n\t}\n\n\treturn dataUpload, err\n}\n\nfunc getDataUpload(\n\tctx context.Context,\n\tcrClient crclient.Client,\n\toperationID string,\n) (*velerov2alpha1.DataUpload, error) {\n\tdataUploadList := new(velerov2alpha1.DataUploadList)\n\terr := crClient.List(ctx, dataUploadList, &crclient.ListOptions{\n\t\tLabelSelector: labels.SelectorFromSet(\n\t\t\tmap[string]string{velerov1api.AsyncOperationIDLabel: operationID},\n\t\t),\n\t})\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error to list DataUpload\")\n\t}\n\n\tif len(dataUploadList.Items) == 0 {\n\t\treturn nil, errors.Errorf(\"not found DataUpload for operationID %s\", operationID)\n\t}\n\n\tif len(dataUploadList.Items) > 1 {\n\t\treturn nil, errors.Errorf(\"more than one DataUpload found operationID %s\", operationID)\n\t}\n\n\treturn &dataUploadList.Items[0], nil\n}\n\nfunc cancelDataUpload(\n\tctx context.Context,\n\tcrClient crclient.Client,\n\tdataUpload *velerov2alpha1.DataUpload,\n) error {\n\tupdatedDataUpload := dataUpload.DeepCopy()\n\tupdatedDataUpload.Spec.Cancel = true\n\n\terr := crClient.Patch(ctx, updatedDataUpload, crclient.MergeFrom(dataUpload))\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error patch DataUpload\")\n\t}\n\n\treturn nil\n}\n\nfunc NewPvcBackupItemAction(f veleroclient.Factory) plugincommon.HandlerInitializer {\n\treturn func(logger logrus.FieldLogger) (any, error) {\n\t\tcrClient, err := f.KubebuilderClient()\n\t\tif err != nil {\n\t\t\treturn nil, errors.WithStack(err)\n\t\t}\n\n\t\treturn &pvcBackupItemAction{\n\t\t\tlog: logger,\n\t\t\tcrClient: crClient,\n\t\t}, nil\n\t}\n}\n\nfunc (p *pvcBackupItemAction) getVolumeSnapshotReference(\n\tctx context.Context,\n\tpvc corev1api.PersistentVolumeClaim,\n\tbackup *velerov1api.Backup,\n) (*snapshotv1api.VolumeSnapshot, error) {\n\tvgsLabelKey := backup.Spec.VolumeGroupSnapshotLabelKey\n\tgroup, hasLabel := pvc.Labels[vgsLabelKey]\n\n\tif vgsLabelKey != \"\" && hasLabel && group != \"\" {\n\t\tp.log.Infof(\"PVC %s/%s is part of VolumeGroupSnapshot group %q via label %q\", pvc.Namespace, pvc.Name, group, vgsLabelKey)\n\n\t\t// Try to find an existing VS created via a previous VGS in the current backup\n\t\texistingVS, err := p.findExistingVSForBackup(ctx, backup.UID, backup.Name, pvc.Name, pvc.Namespace)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"failed to find existing VolumeSnapshot for PVC %s/%s\", pvc.Namespace, pvc.Name)\n\t\t}\n\n\t\tif existingVS != nil {\n\t\t\tif existingVS.Status != nil && existingVS.Status.VolumeGroupSnapshotName != nil {\n\t\t\t\tp.log.Infof(\"Reusing existing VolumeSnapshot %s for PVC %s\", existingVS.Name, pvc.Name)\n\t\t\t\treturn existingVS, nil\n\t\t\t} else {\n\t\t\t\treturn nil, errors.Errorf(\"found VolumeSnapshot %s for PVC %s, but it was not created via VolumeGroupSnapshot (missing volumeGroupSnapshotName)\", existingVS.Name, pvc.Name)\n\t\t\t}\n\t\t}\n\n\t\tp.log.Infof(\"No existing VS found for PVC %s; creating new VGS\", pvc.Name)\n\n\t\t// List all PVCs in the VGS group\n\t\tgroupedPVCs, err := p.listGroupedPVCs(ctx, pvc.Namespace, vgsLabelKey, group)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"failed to list PVCs in VolumeGroupSnapshot group %q in namespace %q\", group, pvc.Namespace)\n\t\t}\n\n\t\t// Ensure PVC-to-Pod cache is built for this namespace (lazy per-namespace caching)\n\t\tif err := p.ensurePVCPodCacheForNamespace(ctx, pvc.Namespace); err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"failed to build PVC-to-Pod cache for namespace %s\", pvc.Namespace)\n\t\t}\n\n\t\t// Get the cached VolumeHelper for filtering PVCs by volume policy\n\t\tvh, err := p.getOrCreateVolumeHelper(backup)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"failed to get VolumeHelper for filtering PVCs in group %q\", group)\n\t\t}\n\n\t\t// Filter PVCs by volume policy\n\t\tfilteredPVCs, err := p.filterPVCsByVolumePolicy(groupedPVCs, backup, vh)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"failed to filter PVCs by volume policy for VolumeGroupSnapshot group %q\", group)\n\t\t}\n\n\t\t// Warn if any PVCs were filtered out\n\t\tif len(filteredPVCs) < len(groupedPVCs) {\n\t\t\tfor _, originalPVC := range groupedPVCs {\n\t\t\t\tfound := false\n\t\t\t\tfor _, filteredPVC := range filteredPVCs {\n\t\t\t\t\tif originalPVC.Name == filteredPVC.Name {\n\t\t\t\t\t\tfound = true\n\t\t\t\t\t\tbreak\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tif !found {\n\t\t\t\t\tp.log.Warnf(\"PVC %s/%s has VolumeGroupSnapshot label %s=%s but is excluded by volume policy\", originalPVC.Namespace, originalPVC.Name, vgsLabelKey, group)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\t// Determine the CSI driver for the grouped PVCs\n\t\tdriver, err := p.determineCSIDriver(filteredPVCs)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"failed to determine CSI driver for PVCs in VolumeGroupSnapshot group %q\", group)\n\t\t}\n\t\tif driver == \"\" {\n\t\t\treturn nil, errors.New(\"no csi driver found, failing the backup\")\n\t\t}\n\n\t\t// Determine the VGSClass to be used for the VGS object to be created\n\t\tvgsClass, err := p.determineVGSClass(ctx, driver, backup, &pvc)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"failed to determine VolumeGroupSnapshotClass for CSI driver %q\", driver)\n\t\t}\n\n\t\t// Create the VGS object\n\t\tnewVGS, err := p.createVolumeGroupSnapshot(ctx, backup, pvc, vgsLabelKey, group, vgsClass)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"failed to create VolumeGroupSnapshot for PVC %s/%s\", pvc.Namespace, pvc.Name)\n\t\t}\n\n\t\t// Wait for all the VS objects associated with the VGS to have status and VGS Name (VS readiness is checked in legacy flow) and get the PVC-to-VS map\n\t\tvsMap, err := p.waitForVGSAssociatedVS(ctx, filteredPVCs, newVGS, backup.Spec.CSISnapshotTimeout.Duration)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"timeout waiting for VolumeSnapshots to have status created via VolumeGroupSnapshot %s\", newVGS.Name)\n\t\t}\n\n\t\t// Update the VS objects: remove VGS owner references and finalizers; add backup metadata labels.\n\t\terr = p.updateVGSCreatedVS(ctx, vsMap, newVGS, backup)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"failed to update VolumeSnapshots created by VolumeGroupSnapshot %s\", newVGS.Name)\n\t\t}\n\n\t\t// Wait for VGSC binding in the VGS status\n\t\terr = p.waitForVGSCBinding(ctx, newVGS, backup.Spec.CSISnapshotTimeout.Duration)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"timeout waiting for VolumeGroupSnapshotContent binding for VolumeGroupSnapshot %s\", newVGS.Name)\n\t\t}\n\n\t\t// Re-fetch latest VGS to ensure status is populated after VGSC binding\n\t\tlatestVGS := &volumegroupsnapshotv1beta1.VolumeGroupSnapshot{}\n\t\tif err := p.crClient.Get(ctx, crclient.ObjectKeyFromObject(newVGS), latestVGS); err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"failed to re-fetch VolumeGroupSnapshot %s after VGSC binding wait\", newVGS.Name)\n\t\t}\n\n\t\t// Patch the VGSC deletionPolicy to Retain.\n\t\terr = p.patchVGSCDeletionPolicy(ctx, latestVGS)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"failed to patch VolumeGroupSnapshotContent Deletion Policy for VolumeGroupSnapshot %s\", newVGS.Name)\n\t\t}\n\n\t\t// Delete the VGS and VGSC\n\t\terr = p.deleteVGSAndVGSC(ctx, latestVGS)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"failed to get VolumeSnapshot for PVC %s/%s created by VolumeGroupSnapshot %s\", pvc.Namespace, pvc.Name, newVGS.Name)\n\t\t}\n\n\t\t// Use the VS that was created for this PVC via VGS.\n\t\tvs, found := vsMap[pvc.Name]\n\t\tif !found {\n\t\t\treturn nil, errors.Wrapf(err, \"failed to get VolumeSnapshot for PVC %s/%s created by VolumeGroupSnapshot %s\", pvc.Namespace, pvc.Name, newVGS.Name)\n\t\t}\n\n\t\treturn vs, nil\n\t}\n\n\t// Legacy fallback: create individual VS\n\treturn p.createVolumeSnapshot(pvc, backup)\n}\n\nfunc (p *pvcBackupItemAction) findExistingVSForBackup(\n\tctx context.Context,\n\tbackupUID types.UID,\n\tbackupName, pvcName, namespace string,\n) (*snapshotv1api.VolumeSnapshot, error) {\n\tvsList := &snapshotv1api.VolumeSnapshotList{}\n\n\tlabelSelector := labels.SelectorFromSet(map[string]string{\n\t\tvelerov1api.BackupNameLabel: label.GetValidName(backupName),\n\t\tvelerov1api.BackupUIDLabel: string(backupUID),\n\t})\n\n\tif err := p.crClient.List(ctx, vsList,\n\t\tcrclient.InNamespace(namespace),\n\t\tcrclient.MatchingLabelsSelector{Selector: labelSelector},\n\t); err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to list VolumeSnapshots with backup labels\")\n\t}\n\n\tfor _, vs := range vsList.Items {\n\t\tif vs.Spec.Source.PersistentVolumeClaimName != nil &&\n\t\t\t*vs.Spec.Source.PersistentVolumeClaimName == pvcName {\n\t\t\treturn &vs, nil\n\t\t}\n\t}\n\n\treturn nil, nil\n}\n\nfunc (p *pvcBackupItemAction) listGroupedPVCs(ctx context.Context, namespace, labelKey, groupValue string) ([]corev1api.PersistentVolumeClaim, error) {\n\tpvcList := new(corev1api.PersistentVolumeClaimList)\n\tif err := p.crClient.List(\n\t\tctx,\n\t\tpvcList,\n\t\tcrclient.InNamespace(namespace),\n\t\tcrclient.MatchingLabels{labelKey: groupValue},\n\t); err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to list grouped PVCs\")\n\t}\n\n\treturn pvcList.Items, nil\n}\n\nfunc (p *pvcBackupItemAction) filterPVCsByVolumePolicy(\n\tpvcs []corev1api.PersistentVolumeClaim,\n\tbackup *velerov1api.Backup,\n\tvh internalvolumehelper.VolumeHelper,\n) ([]corev1api.PersistentVolumeClaim, error) {\n\tvar filteredPVCs []corev1api.PersistentVolumeClaim\n\n\tfor _, pvc := range pvcs {\n\t\t// Convert PVC to unstructured for ShouldPerformSnapshotWithVolumeHelper\n\t\tpvcMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&pvc)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"failed to convert PVC %s/%s to unstructured\", pvc.Namespace, pvc.Name)\n\t\t}\n\t\tunstructuredPVC := &unstructured.Unstructured{Object: pvcMap}\n\n\t\t// Check if this PVC should be snapshotted according to volume policies\n\t\t// Uses the cached VolumeHelper for better performance with many PVCs/pods\n\t\tshouldSnapshot, err := volumehelper.ShouldPerformSnapshotWithVolumeHelper(\n\t\t\tunstructuredPVC,\n\t\t\tkuberesource.PersistentVolumeClaims,\n\t\t\t*backup,\n\t\t\tp.crClient,\n\t\t\tp.log,\n\t\t\tvh,\n\t\t)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"failed to check volume policy for PVC %s/%s\", pvc.Namespace, pvc.Name)\n\t\t}\n\n\t\tif shouldSnapshot {\n\t\t\tfilteredPVCs = append(filteredPVCs, pvc)\n\t\t}\n\t}\n\n\treturn filteredPVCs, nil\n}\n\nfunc (p *pvcBackupItemAction) determineCSIDriver(\n\tpvcs []corev1api.PersistentVolumeClaim,\n) (string, error) {\n\tvar driver string\n\n\tfor _, pvc := range pvcs {\n\t\tpv, err := kubeutil.GetPVForPVC(&pvc, p.crClient)\n\t\tif err != nil {\n\t\t\treturn \"\", err\n\t\t}\n\t\tif pv.Spec.CSI == nil {\n\t\t\treturn \"\", errors.Errorf(\"PV %s for PVC %s is not CSI provisioned\", pv.Name, pvc.Name)\n\t\t}\n\t\tcurrent := pv.Spec.CSI.Driver\n\t\tif driver == \"\" {\n\t\t\tdriver = current\n\t\t} else if driver != current {\n\t\t\treturn \"\", errors.Errorf(\"found multiple CSI drivers: %s and %s\", driver, current)\n\t\t}\n\t}\n\treturn driver, nil\n}\n\nfunc (p *pvcBackupItemAction) determineVGSClass(\n\tctx context.Context,\n\tdriver string,\n\tbackup *velerov1api.Backup,\n\tpvc *corev1api.PersistentVolumeClaim,\n) (string, error) {\n\t// 1. PVC-level override\n\tif pvc != nil {\n\t\tif val, ok := pvc.Annotations[velerov1api.VolumeGroupSnapshotClassAnnotationPVC]; ok && val != \"\" {\n\t\t\treturn val, nil\n\t\t}\n\t}\n\n\t// 2. Backup-level override\n\tkey := fmt.Sprintf(velerov1api.VolumeGroupSnapshotClassAnnotationBackupPrefix+\"%s\", driver)\n\tif val, ok := backup.Annotations[key]; ok && val != \"\" {\n\t\treturn val, nil\n\t}\n\n\t// 3. Fallback to label-based default\n\tvgsClassList := &volumegroupsnapshotv1beta1.VolumeGroupSnapshotClassList{}\n\tif err := p.crClient.List(ctx, vgsClassList); err != nil {\n\t\treturn \"\", errors.Wrap(err, \"failed to list VolumeGroupSnapshotClasses\")\n\t}\n\n\tvar matched []string\n\tfor _, class := range vgsClassList.Items {\n\t\tif class.Driver != driver {\n\t\t\tcontinue\n\t\t}\n\t\tif val, ok := class.Labels[velerov1api.VolumeGroupSnapshotClassDefaultLabel]; ok && val == \"true\" {\n\t\t\tmatched = append(matched, class.Name)\n\t\t}\n\t}\n\n\tif len(matched) == 1 {\n\t\treturn matched[0], nil\n\t} else if len(matched) == 0 {\n\t\treturn \"\", errors.Errorf(\"no VolumeGroupSnapshotClass found for driver %q for PVC %s\", driver, pvc.Name)\n\t} else {\n\t\treturn \"\", errors.Errorf(\"multiple VolumeGroupSnapshotClasses found for driver %q with label velero.io/csi-volumegroupsnapshot-class=true\", driver)\n\t}\n}\n\nfunc (p *pvcBackupItemAction) createVolumeGroupSnapshot(\n\tctx context.Context,\n\tbackup *velerov1api.Backup,\n\tpvc corev1api.PersistentVolumeClaim,\n\tvgsLabelKey, vgsLabelValue, vgsClassName string,\n) (*volumegroupsnapshotv1beta1.VolumeGroupSnapshot, error) {\n\tvgsLabels := map[string]string{\n\t\tvelerov1api.BackupNameLabel: label.GetValidName(backup.Name),\n\t\tvelerov1api.BackupUIDLabel: string(backup.UID),\n\t\tvgsLabelKey: vgsLabelValue,\n\t}\n\n\tvgs := &volumegroupsnapshotv1beta1.VolumeGroupSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tGenerateName: fmt.Sprintf(\"velero-%s-\", vgsLabelValue),\n\t\t\tNamespace: pvc.Namespace,\n\t\t\tLabels: vgsLabels,\n\t\t},\n\t\tSpec: volumegroupsnapshotv1beta1.VolumeGroupSnapshotSpec{\n\t\t\tVolumeGroupSnapshotClassName: &vgsClassName,\n\t\t\tSource: volumegroupsnapshotv1beta1.VolumeGroupSnapshotSource{\n\t\t\t\tSelector: &metav1.LabelSelector{\n\t\t\t\t\tMatchLabels: map[string]string{\n\t\t\t\t\t\tvgsLabelKey: vgsLabelValue,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tif err := p.crClient.Create(ctx, vgs); err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to create VolumeGroupSnapshot\")\n\t}\n\n\trefetchedVGS, err := p.getVGSByLabels(ctx, pvc.Namespace, vgsLabels)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to re-fetch VGS after creation\")\n\t}\n\n\tp.log.Infof(\"Re-fetched Created VolumeGroupSnapshot %s/%s for PVC group label %s=%s\",\n\t\trefetchedVGS.Namespace, refetchedVGS.Name, vgsLabelKey, vgsLabelValue)\n\n\treturn refetchedVGS, nil\n}\n\nfunc (p *pvcBackupItemAction) waitForVGSAssociatedVS(\n\tctx context.Context,\n\tgroupedPVCs []corev1api.PersistentVolumeClaim,\n\tvgs *volumegroupsnapshotv1beta1.VolumeGroupSnapshot,\n\ttimeout time.Duration,\n) (map[string]*snapshotv1api.VolumeSnapshot, error) {\n\texpected := len(groupedPVCs)\n\n\tvsMap := make(map[string]*snapshotv1api.VolumeSnapshot)\n\n\terr := wait.PollUntilContextTimeout(ctx, time.Second, timeout, true, func(ctx context.Context) (done bool, err error) {\n\t\tvsList := &snapshotv1api.VolumeSnapshotList{}\n\t\tif err := p.crClient.List(ctx, vsList, crclient.InNamespace(vgs.Namespace)); err != nil {\n\t\t\treturn false, err\n\t\t}\n\n\t\tvsMap = make(map[string]*snapshotv1api.VolumeSnapshot)\n\n\t\tfor _, vs := range vsList.Items {\n\t\t\tif !hasOwnerReference(&vs, vgs) {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif vs.Status != nil && vs.Status.VolumeGroupSnapshotName != nil &&\n\t\t\t\t*vs.Status.VolumeGroupSnapshotName == vgs.Name {\n\t\t\t\tif vs.Spec.Source.PersistentVolumeClaimName != nil {\n\t\t\t\t\tvsMap[*vs.Spec.Source.PersistentVolumeClaimName] = vs.DeepCopy()\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\tif expected == 0 {\n\t\t\treturn false, nil\n\t\t}\n\t\tif len(vsMap) == expected {\n\t\t\treturn true, nil\n\t\t}\n\t\treturn false, nil\n\t})\n\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"timeout waiting for VolumeSnapshots associated with VGS %s\", vgs.Name)\n\t}\n\n\treturn vsMap, nil\n}\n\nfunc hasOwnerReference(obj metav1.Object, vgs *volumegroupsnapshotv1beta1.VolumeGroupSnapshot) bool {\n\tfor _, ref := range obj.GetOwnerReferences() {\n\t\tif ref.Kind == kuberesource.VGSKind &&\n\t\t\tref.APIVersion == volumegroupsnapshotv1beta1.GroupName+\"/\"+volumegroupsnapshotv1beta1.SchemeGroupVersion.Version &&\n\t\t\tref.UID == vgs.UID {\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n\nfunc (p *pvcBackupItemAction) updateVGSCreatedVS(\n\tctx context.Context,\n\tvsMap map[string]*snapshotv1api.VolumeSnapshot,\n\tvgs *volumegroupsnapshotv1beta1.VolumeGroupSnapshot,\n\tbackup *velerov1api.Backup,\n) error {\n\tfor pvcName, vs := range vsMap {\n\t\tif vs == nil || vs.Status == nil || vs.Status.VolumeGroupSnapshotName == nil ||\n\t\t\t*vs.Status.VolumeGroupSnapshotName != vgs.Name {\n\t\t\tcontinue\n\t\t}\n\n\t\terr := retry.RetryOnConflict(retry.DefaultRetry, func() error {\n\t\t\t// Re-fetch the latest VS to avoid conflict\n\t\t\tlatestVS := &snapshotv1api.VolumeSnapshot{}\n\t\t\tif err := p.crClient.Get(ctx, crclient.ObjectKeyFromObject(vs), latestVS); err != nil {\n\t\t\t\treturn errors.Wrapf(err, \"failed to get latest VolumeSnapshot %s (PVC %s)\", vs.Name, pvcName)\n\t\t\t}\n\n\t\t\t// Remove VGS owner ref\n\t\t\tif err := controllerutil.RemoveOwnerReference(vgs, latestVS, p.crClient.Scheme()); err != nil {\n\t\t\t\treturn errors.Wrapf(err, \"failed to remove VGS owner reference from VS %s\", vs.Name)\n\t\t\t}\n\n\t\t\t// Remove known finalizers\n\t\t\tcontrollerutil.RemoveFinalizer(latestVS, VolumeSnapshotFinalizerGroupProtection)\n\t\t\tcontrollerutil.RemoveFinalizer(latestVS, VolumeSnapshotFinalizerSourceProtection)\n\n\t\t\t// Add Velero labels\n\t\t\tif latestVS.Labels == nil {\n\t\t\t\tlatestVS.Labels = make(map[string]string)\n\t\t\t}\n\t\t\tlatestVS.Labels[velerov1api.BackupNameLabel] = backup.Name\n\t\t\tlatestVS.Labels[velerov1api.BackupUIDLabel] = string(backup.UID)\n\n\t\t\t// Attempt to update\n\t\t\treturn p.crClient.Update(ctx, latestVS)\n\t\t})\n\t\tif err != nil {\n\t\t\treturn errors.Wrapf(err, \"failed to update VS %s (PVC %s) after retrying on conflict\", vs.Name, pvcName)\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (p *pvcBackupItemAction) patchVGSCDeletionPolicy(ctx context.Context, vgs *volumegroupsnapshotv1beta1.VolumeGroupSnapshot) error {\n\tif vgs == nil || vgs.Status == nil || vgs.Status.BoundVolumeGroupSnapshotContentName == nil {\n\t\treturn errors.New(\"VolumeGroupSnapshotContent name not found in VGS status\")\n\t}\n\n\tvgscName := vgs.Status.BoundVolumeGroupSnapshotContentName\n\n\treturn retry.RetryOnConflict(retry.DefaultBackoff, func() error {\n\t\tvgsc := &volumegroupsnapshotv1beta1.VolumeGroupSnapshotContent{}\n\t\tif err := p.crClient.Get(ctx, crclient.ObjectKey{Name: *vgscName}, vgsc); err != nil {\n\t\t\treturn errors.Wrapf(err, \"failed to get VolumeGroupSnapshotContent %s for VolumeGroupSnapshot %s/%s\", *vgscName, vgs.Namespace, vgs.Name)\n\t\t}\n\n\t\tif vgsc.Spec.DeletionPolicy == snapshotv1api.VolumeSnapshotContentDelete {\n\t\t\tp.log.Infof(\"Patching VGSC %s to Retain deletionPolicy\", *vgscName)\n\t\t\tvgsc.Spec.DeletionPolicy = snapshotv1api.VolumeSnapshotContentRetain\n\t\t\tif err := p.crClient.Update(ctx, vgsc); err != nil {\n\t\t\t\treturn errors.Wrapf(err, \"failed to update VGSC %s deletionPolicy\", *vgscName)\n\t\t\t}\n\t\t} else {\n\t\t\tp.log.Infof(\"VGSC %s already set to deletionPolicy=%s\", *vgscName, vgsc.Spec.DeletionPolicy)\n\t\t}\n\n\t\treturn nil\n\t})\n}\n\nfunc (p *pvcBackupItemAction) deleteVGSAndVGSC(ctx context.Context, vgs *volumegroupsnapshotv1beta1.VolumeGroupSnapshot) error {\n\tif vgs.Status != nil && vgs.Status.BoundVolumeGroupSnapshotContentName != nil {\n\t\tvgsc := &volumegroupsnapshotv1beta1.VolumeGroupSnapshotContent{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: *vgs.Status.BoundVolumeGroupSnapshotContentName,\n\t\t\t},\n\t\t}\n\t\tp.log.Infof(\"Deleting VolumeGroupSnapshotContent %s\", vgsc.Name)\n\t\tif err := p.crClient.Delete(ctx, vgsc); err != nil && !apierrors.IsNotFound(err) {\n\t\t\tp.log.Warnf(\"Failed to delete VolumeGroupSnapshotContent %s: %v\", vgsc.Name, err)\n\t\t\treturn errors.Wrapf(err, \"failed to delete VolumeGroupSnapshotContent %s\", vgsc.Name)\n\t\t}\n\t} else {\n\t\tp.log.Infof(\"No BoundVolumeGroupSnapshotContentName set in VolumeGroupSnapshot %s/%s\", vgs.Namespace, vgs.Name)\n\t}\n\n\tp.log.Infof(\"Deleting VolumeGroupSnapshot %s/%s\", vgs.Namespace, vgs.Name)\n\tif err := p.crClient.Delete(ctx, vgs); err != nil && !apierrors.IsNotFound(err) {\n\t\tp.log.Warnf(\"Failed to delete VolumeGroupSnapshot %s/%s: %v\", vgs.Namespace, vgs.Name, err)\n\t\treturn errors.Wrapf(err, \"failed to delete VolumeGroupSnapshot %s/%s\", vgs.Namespace, vgs.Name)\n\t}\n\n\treturn nil\n}\n\nfunc (p *pvcBackupItemAction) waitForVGSCBinding(\n\tctx context.Context,\n\tvgs *volumegroupsnapshotv1beta1.VolumeGroupSnapshot,\n\ttimeout time.Duration,\n) error {\n\treturn wait.PollUntilContextTimeout(ctx, time.Second, timeout, true, func(ctx context.Context) (bool, error) {\n\t\tvgsRef := &volumegroupsnapshotv1beta1.VolumeGroupSnapshot{}\n\t\tif err := p.crClient.Get(ctx, crclient.ObjectKeyFromObject(vgs), vgsRef); err != nil {\n\t\t\treturn false, err\n\t\t}\n\n\t\tif vgsRef.Status != nil && vgsRef.Status.BoundVolumeGroupSnapshotContentName != nil {\n\t\t\treturn true, nil\n\t\t}\n\n\t\treturn false, nil\n\t})\n}\n\nfunc (p *pvcBackupItemAction) getVGSByLabels(ctx context.Context, namespace string, labels map[string]string) (*volumegroupsnapshotv1beta1.VolumeGroupSnapshot, error) {\n\tvgsList := &volumegroupsnapshotv1beta1.VolumeGroupSnapshotList{}\n\tif err := p.crClient.List(ctx, vgsList,\n\t\tcrclient.InNamespace(namespace),\n\t\tcrclient.MatchingLabels(labels),\n\t); err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to list VolumeGroupSnapshots by labels\")\n\t}\n\n\tif len(vgsList.Items) == 0 {\n\t\treturn nil, errors.New(\"no VolumeGroupSnapshot found matching labels\")\n\t}\n\tif len(vgsList.Items) > 1 {\n\t\treturn nil, errors.New(\"multiple VolumeGroupSnapshots found matching labels\")\n\t}\n\n\treturn &vgsList.Items[0], nil\n}\n\nfunc setPVCRequestSizeToVSRestoreSize(\n\tpvc *corev1api.PersistentVolumeClaim,\n\tvsc *snapshotv1api.VolumeSnapshotContent,\n\tlogger logrus.FieldLogger,\n) {\n\tif vsc.Status.RestoreSize != nil {\n\t\tlogger.Debugf(\"Patching PVC request size to fit the volumesnapshot restore size %d\", vsc.Status.RestoreSize)\n\t\trestoreSize := *resource.NewQuantity(*vsc.Status.RestoreSize, resource.BinarySI)\n\n\t\t// It is possible that the volume provider allocated a larger\n\t\t// capacity volume than what was requested in the backed up PVC.\n\t\t// In this scenario the volumesnapshot of the PVC will end being\n\t\t// larger than its requested storage size. Such a PVC, on restore\n\t\t// as-is, will be stuck attempting to use a VolumeSnapshot as a\n\t\t// data source for a PVC that is not large enough.\n\t\t// To counter that, here we set the storage request on the PVC\n\t\t// to the larger of the PVC's storage request and the size of the\n\t\t// VolumeSnapshot\n\t\tsetPVCStorageResourceRequest(pvc, restoreSize, logger)\n\t}\n}\n\nfunc setPVCStorageResourceRequest(\n\tpvc *corev1api.PersistentVolumeClaim,\n\trestoreSize resource.Quantity,\n\tlog logrus.FieldLogger,\n) {\n\t{\n\t\tif pvc.Spec.Resources.Requests == nil {\n\t\t\tpvc.Spec.Resources.Requests = corev1api.ResourceList{}\n\t\t}\n\n\t\tstorageReq, exists := pvc.Spec.Resources.Requests[corev1api.ResourceStorage]\n\t\tif !exists || storageReq.Cmp(restoreSize) < 0 {\n\t\t\tpvc.Spec.Resources.Requests[corev1api.ResourceStorage] = restoreSize\n\t\t\trs := pvc.Spec.Resources.Requests[corev1api.ResourceStorage]\n\t\t\tlog.Infof(\"Resetting storage requests for PVC %s/%s to %s\",\n\t\t\t\tpvc.Namespace, pvc.Name, rs.String())\n\t\t}\n\t}\n}\n" }, { "path": "pkg/backup/actions/csi/pvc_action_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\n\tvolumegroupsnapshotv1beta1 \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumegroupsnapshot/v1beta1\"\n\t\"github.com/stretchr/testify/assert\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/utils/pointer\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\n\t\"github.com/google/go-cmp/cmp\"\n\t\"github.com/google/go-cmp/cmp/cmpopts\"\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tstoragev1api \"k8s.io/api/storage/v1\"\n\t\"k8s.io/apimachinery/pkg/api/resource\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\tcrclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/apis/velero/shared\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nconst testDriver = \"csi.example.com\"\n\n// errorInjectingClient is a wrapper around a normal client that injects an error\n// when a specific resource type (VolumeSnapshot) is created.\ntype errorInjectingClient struct {\n\tcrclient.Client\n}\n\n// Create overrides the embedded client's Create method.\nfunc (c *errorInjectingClient) Create(ctx context.Context, obj crclient.Object, opts ...crclient.CreateOption) error {\n\t// Check if the object being created is a VolumeSnapshot.\n\tif _, ok := obj.(*snapshotv1api.VolumeSnapshot); ok {\n\t\t// If it is, return our injected error instead of proceeding.\n\t\treturn errors.New(\"injected error on create\")\n\t}\n\t// For all other object types, call the original, embedded Create method.\n\treturn c.Client.Create(ctx, obj, opts...)\n}\n\nfunc TestExecute(t *testing.T) {\n\tboolTrue := true\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\tpv *corev1api.PersistentVolume\n\t\tsc *storagev1api.StorageClass\n\t\tvsClass *snapshotv1api.VolumeSnapshotClass\n\t\toperationID string\n\t\texpectedErr error\n\t\texpectErr bool // Use bool for cases where we just need to check for any error\n\t\texpectedBackup *velerov1api.Backup\n\t\texpectedDataUpload *velerov2alpha1.DataUpload\n\t\texpectedPVC *corev1api.PersistentVolumeClaim\n\t\tresourcePolicy *corev1api.ConfigMap\n\t\tfailVSCreate bool\n\t\tskipVSReadyUpdate bool // New flag to control VS readiness\n\t}{\n\t\t{\n\t\t\tname: \"Skip PVC BIA when backup is in finalizing phase\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"test\").Phase(velerov1api.BackupPhaseFinalizing).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Fail when creating volumesnapshot returns error\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"test\").CSISnapshotTimeout(1 * time.Minute).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").VolumeName(\"testPV\").StorageClass(\"testSC\").Phase(corev1api.ClaimBound).Result(),\n\t\t\tpv: builder.ForPersistentVolume(\"testPV\").CSI(\"hostpath\", \"testVolume\").Result(),\n\t\t\tsc: builder.ForStorageClass(\"testSC\").Provisioner(\"hostpath\").Result(),\n\t\t\tvsClass: builder.ForVolumeSnapshotClass(\"testVSClass\").Driver(\"hostpath\").ObjectMeta(builder.WithLabels(velerov1api.VolumeSnapshotClassSelectorLabel, \"\")).Result(),\n\t\t\tfailVSCreate: true,\n\t\t\texpectedErr: errors.New(\"error creating volume snapshot: injected error on create\"),\n\t\t},\n\t\t{\n\t\t\tname: \"Fail when waiting for VolumeSnapshot to be ready times out\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"test\").CSISnapshotTimeout(20 * time.Millisecond).Result(), // Short timeout\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").VolumeName(\"testPV\").StorageClass(\"testSC\").Phase(corev1api.ClaimBound).Result(),\n\t\t\tpv: builder.ForPersistentVolume(\"testPV\").CSI(\"hostpath\", \"testVolume\").Result(),\n\t\t\tsc: builder.ForStorageClass(\"testSC\").Provisioner(\"hostpath\").Result(),\n\t\t\tvsClass: builder.ForVolumeSnapshotClass(\"testVSClass\").Driver(\"hostpath\").ObjectMeta(builder.WithLabels(velerov1api.VolumeSnapshotClassSelectorLabel, \"\")).Result(),\n\t\t\tskipVSReadyUpdate: true, // This will cause the timeout\n\t\t\texpectErr: true, // Expect an error, but the exact message can vary\n\t\t},\n\t\t{\n\t\t\tname: \"Test SnapshotMoveData\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"test\").SnapshotMoveData(true).CSISnapshotTimeout(1 * time.Minute).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").VolumeName(\"testPV\").StorageClass(\"testSC\").Phase(corev1api.ClaimBound).Result(),\n\t\t\tpv: builder.ForPersistentVolume(\"testPV\").CSI(\"hostpath\", \"testVolume\").Result(),\n\t\t\tsc: builder.ForStorageClass(\"testSC\").Provisioner(\"hostpath\").Result(),\n\t\t\tvsClass: builder.ForVolumeSnapshotClass(\"testVSClass\").Driver(\"hostpath\").ObjectMeta(builder.WithLabels(velerov1api.VolumeSnapshotClassSelectorLabel, \"\")).Result(),\n\t\t\toperationID: \".\",\n\t\t\texpectedDataUpload: &velerov2alpha1.DataUpload{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"DataUpload\",\n\t\t\t\t\tAPIVersion: velerov2alpha1.SchemeGroupVersion.String(),\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tGenerateName: \"test-\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\tvelerov1api.BackupNameLabel: \"test\",\n\t\t\t\t\t\tvelerov1api.BackupUIDLabel: \"\",\n\t\t\t\t\t\tvelerov1api.PVCUIDLabel: \"\",\n\t\t\t\t\t\tvelerov1api.AsyncOperationIDLabel: \"du-.\",\n\t\t\t\t\t},\n\t\t\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tAPIVersion: \"velero.io/v1\",\n\t\t\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\t\t\tName: \"test\",\n\t\t\t\t\t\t\tUID: \"\",\n\t\t\t\t\t\t\tController: &boolTrue,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov2alpha1.DataUploadSpec{\n\t\t\t\t\tSnapshotType: velerov2alpha1.SnapshotTypeCSI,\n\t\t\t\t\tCSISnapshot: &velerov2alpha1.CSISnapshotSpec{\n\t\t\t\t\t\tVolumeSnapshot: \"\",\n\t\t\t\t\t\tStorageClass: \"testSC\",\n\t\t\t\t\t\tSnapshotClass: \"testVSClass\",\n\t\t\t\t\t},\n\t\t\t\t\tSourcePVC: \"testPVC\",\n\t\t\t\t\tSourceNamespace: \"velero\",\n\t\t\t\t\tOperationTimeout: metav1.Duration{Duration: 1 * time.Minute},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Verify PVC is modified as expected\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"test\").SnapshotMoveData(true).CSISnapshotTimeout(1 * time.Minute).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").VolumeName(\"testPV\").StorageClass(\"testSC\").Phase(corev1api.ClaimBound).Result(),\n\t\t\tpv: builder.ForPersistentVolume(\"testPV\").CSI(\"hostpath\", \"testVolume\").Result(),\n\t\t\tsc: builder.ForStorageClass(\"testSC\").Provisioner(\"hostpath\").Result(),\n\t\t\tvsClass: builder.ForVolumeSnapshotClass(\"tescVSClass\").Driver(\"hostpath\").ObjectMeta(builder.WithLabels(velerov1api.VolumeSnapshotClassSelectorLabel, \"\")).Result(),\n\t\t\toperationID: \".\",\n\t\t\texpectedPVC: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").\n\t\t\t\tObjectMeta(builder.WithAnnotations(velerov1api.MustIncludeAdditionalItemAnnotation, \"true\", velerov1api.DataUploadNameAnnotation, \"velero/\"),\n\t\t\t\t\tbuilder.WithLabels(velerov1api.BackupNameLabel, \"test\")).\n\t\t\t\tVolumeName(\"testPV\").StorageClass(\"testSC\").Phase(corev1api.ClaimBound).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Test ResourcePolicy\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"test\").ResourcePolicies(\"resourcePolicy\").SnapshotVolumes(false).CSISnapshotTimeout(time.Duration(3600) * time.Second).Result(),\n\t\t\tresourcePolicy: builder.ForConfigMap(\"velero\", \"resourcePolicy\").Data(\"policy\", \"{\\\"version\\\":\\\"v1\\\", \\\"volumePolicies\\\":[{\\\"conditions\\\":{\\\"csi\\\": {}},\\\"action\\\":{\\\"type\\\":\\\"snapshot\\\"}}]}\").Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").VolumeName(\"testPV\").StorageClass(\"testSC\").Phase(corev1api.ClaimBound).Result(),\n\t\t\tpv: builder.ForPersistentVolume(\"testPV\").CSI(\"hostpath\", \"testVolume\").Result(),\n\t\t\tsc: builder.ForStorageClass(\"testSC\").Provisioner(\"hostpath\").Result(),\n\t\t\tvsClass: builder.ForVolumeSnapshotClass(\"tescVSClass\").Driver(\"hostpath\").ObjectMeta(builder.WithLabels(velerov1api.VolumeSnapshotClassSelectorLabel, \"\")).Result(),\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlogger := logrus.New()\n\t\t\tlogger.Level = logrus.DebugLevel\n\t\t\tobjects := make([]runtime.Object, 0)\n\t\t\tif tc.pvc != nil {\n\t\t\t\tobjects = append(objects, tc.pvc)\n\t\t\t}\n\t\t\tif tc.pv != nil {\n\t\t\t\tobjects = append(objects, tc.pv)\n\t\t\t}\n\t\t\tif tc.sc != nil {\n\t\t\t\tobjects = append(objects, tc.sc)\n\t\t\t}\n\t\t\tif tc.vsClass != nil {\n\t\t\t\tobjects = append(objects, tc.vsClass)\n\t\t\t}\n\t\t\tif tc.resourcePolicy != nil {\n\t\t\t\tobjects = append(objects, tc.resourcePolicy)\n\t\t\t}\n\n\t\t\tvar crClient crclient.Client\n\t\t\tif tc.failVSCreate {\n\t\t\t\trealFakeClient := velerotest.NewFakeControllerRuntimeClient(t, objects...)\n\t\t\t\tcrClient = &errorInjectingClient{Client: realFakeClient}\n\t\t\t} else {\n\t\t\t\tcrClient = velerotest.NewFakeControllerRuntimeClient(t, objects...)\n\t\t\t}\n\n\t\t\tpvcBIA := pvcBackupItemAction{\n\t\t\t\tlog: logger,\n\t\t\t\tcrClient: crClient,\n\t\t\t}\n\n\t\t\tpvcMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&tc.pvc)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tif tc.pvc != nil && !tc.failVSCreate && !tc.skipVSReadyUpdate {\n\t\t\t\tgo func() {\n\t\t\t\t\tvar vsList snapshotv1api.VolumeSnapshotList\n\t\t\t\t\terr := wait.PollUntilContextTimeout(t.Context(), 1*time.Second, 10*time.Second, true, func(ctx context.Context) (bool, error) {\n\t\t\t\t\t\terr = pvcBIA.crClient.List(ctx, &vsList, &crclient.ListOptions{Namespace: tc.pvc.Namespace})\n\n\t\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\t\tif err != nil || len(vsList.Items) == 0 {\n\t\t\t\t\t\t\treturn false, err\n\t\t\t\t\t\t}\n\t\t\t\t\t\treturn true, nil\n\t\t\t\t\t})\n\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tvscName := \"testVSC\"\n\t\t\t\t\treadyToUse := true\n\t\t\t\t\tvsList.Items[0].Status = &snapshotv1api.VolumeSnapshotStatus{\n\t\t\t\t\t\tBoundVolumeSnapshotContentName: &vscName,\n\t\t\t\t\t\tReadyToUse: &readyToUse,\n\t\t\t\t\t}\n\t\t\t\t\terr = pvcBIA.crClient.Update(t.Context(), &vsList.Items[0])\n\t\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\t\thandleName := \"testHandle\"\n\t\t\t\t\tvsc := builder.ForVolumeSnapshotContent(\"testVSC\").Status(&snapshotv1api.VolumeSnapshotContentStatus{SnapshotHandle: &handleName}).Result()\n\t\t\t\t\terr = pvcBIA.crClient.Create(t.Context(), vsc)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t}()\n\t\t\t}\n\n\t\t\tresultUnstructed, _, _, _, err := pvcBIA.Execute(&unstructured.Unstructured{Object: pvcMap}, tc.backup)\n\n\t\t\tif tc.expectedErr != nil {\n\t\t\t\trequire.EqualError(t, err, tc.expectedErr.Error())\n\t\t\t} else if tc.expectErr {\n\t\t\t\trequire.Error(t, err)\n\t\t\t\t// On timeout failure, check that the cleanup logic was called\n\t\t\t\tif tc.skipVSReadyUpdate {\n\t\t\t\t\tvsList := new(snapshotv1api.VolumeSnapshotList)\n\t\t\t\t\terrList := crClient.List(t.Context(), vsList, &crclient.ListOptions{Namespace: tc.pvc.Namespace})\n\t\t\t\t\trequire.NoError(t, errList)\n\t\t\t\t\trequire.Empty(t, vsList.Items, \"VolumeSnapshot should have been cleaned up after readiness check failed\")\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif tc.expectedDataUpload != nil {\n\t\t\t\tdataUploadList := new(velerov2alpha1.DataUploadList)\n\t\t\t\terr := crClient.List(t.Context(), dataUploadList, &crclient.ListOptions{LabelSelector: labels.SelectorFromSet(map[string]string{velerov1api.BackupNameLabel: tc.backup.Name})})\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\trequire.Len(t, dataUploadList.Items, 1)\n\t\t\t\trequire.True(t, cmp.Equal(tc.expectedDataUpload, &dataUploadList.Items[0], cmpopts.IgnoreFields(velerov2alpha1.DataUpload{}, \"ResourceVersion\", \"Name\", \"Spec.CSISnapshot.VolumeSnapshot\")))\n\t\t\t}\n\n\t\t\tif tc.expectedPVC != nil {\n\t\t\t\tresultPVC := new(corev1api.PersistentVolumeClaim)\n\t\t\t\truntime.DefaultUnstructuredConverter.FromUnstructured(resultUnstructed.UnstructuredContent(), resultPVC)\n\t\t\t\trequire.True(t, cmp.Equal(tc.expectedPVC, resultPVC, cmpopts.IgnoreFields(corev1api.PersistentVolumeClaim{}, \"ResourceVersion\", \"Annotations\", \"Labels\")))\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestProgress(t *testing.T) {\n\tcurrentTime := time.Now()\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\tdataUpload *velerov2alpha1.DataUpload\n\t\toperationID string\n\t\texpectedErr string\n\t\texpectedProgress velero.OperationProgress\n\t}{\n\t\t{\n\t\t\tname: \"DataUpload cannot be found\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"test\").Result(),\n\t\t\toperationID: \"testing\",\n\t\t\texpectedErr: \"not found DataUpload for operationID testing\",\n\t\t},\n\t\t{\n\t\t\tname: \"DataUpload is found\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"test\").Result(),\n\t\t\tdataUpload: &velerov2alpha1.DataUpload{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"DataUpload\",\n\t\t\t\t\tAPIVersion: \"v2alpha1\",\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\tName: \"testing\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\tvelerov1api.AsyncOperationIDLabel: \"testing\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStatus: velerov2alpha1.DataUploadStatus{\n\t\t\t\t\tPhase: velerov2alpha1.DataUploadPhaseFailed,\n\t\t\t\t\tProgress: shared.DataMoveOperationProgress{\n\t\t\t\t\t\tBytesDone: 1000,\n\t\t\t\t\t\tTotalBytes: 1000,\n\t\t\t\t\t},\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: currentTime},\n\t\t\t\t\tCompletionTimestamp: &metav1.Time{Time: currentTime},\n\t\t\t\t\tMessage: \"Testing error\",\n\t\t\t\t},\n\t\t\t},\n\t\t\toperationID: \"testing\",\n\t\t\texpectedProgress: velero.OperationProgress{\n\t\t\t\tCompleted: true,\n\t\t\t\tErr: \"Testing error\",\n\t\t\t\tNCompleted: 1000,\n\t\t\t\tNTotal: 1000,\n\t\t\t\tOperationUnits: \"Bytes\",\n\t\t\t\tDescription: \"Failed\",\n\t\t\t\tStarted: currentTime,\n\t\t\t\tUpdated: currentTime,\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tcrClient := velerotest.NewFakeControllerRuntimeClient(t)\n\t\t\tlogger := logrus.New()\n\n\t\t\tpvcBIA := pvcBackupItemAction{\n\t\t\t\tlog: logger,\n\t\t\t\tcrClient: crClient,\n\t\t\t}\n\n\t\t\tif tc.dataUpload != nil {\n\t\t\t\terr := crClient.Create(t.Context(), tc.dataUpload)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tprogress, err := pvcBIA.Progress(tc.operationID, tc.backup)\n\t\t\tif tc.expectedErr != \"\" {\n\t\t\t\trequire.Equal(t, tc.expectedErr, err.Error())\n\t\t\t}\n\t\t\trequire.True(t, cmp.Equal(tc.expectedProgress, progress, cmpopts.IgnoreFields(velero.OperationProgress{}, \"Started\", \"Updated\")))\n\t\t})\n\t}\n}\n\nfunc TestCancel(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\tdataUpload velerov2alpha1.DataUpload\n\t\toperationID string\n\t\texpectedErr error\n\t\texpectedDataUpload velerov2alpha1.DataUpload\n\t}{\n\t\t{\n\t\t\tname: \"Cancel DataUpload\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"test\").Result(),\n\t\t\tdataUpload: velerov2alpha1.DataUpload{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"DataUpload\",\n\t\t\t\t\tAPIVersion: velerov2alpha1.SchemeGroupVersion.String(),\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\tName: \"testing\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\tvelerov1api.AsyncOperationIDLabel: \"testing\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\toperationID: \"testing\",\n\t\t\texpectedDataUpload: velerov2alpha1.DataUpload{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"DataUpload\",\n\t\t\t\t\tAPIVersion: velerov2alpha1.SchemeGroupVersion.String(),\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\tName: \"testing\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\tvelerov1api.AsyncOperationIDLabel: \"testing\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov2alpha1.DataUploadSpec{\n\t\t\t\t\tCancel: true,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tcrClient := velerotest.NewFakeControllerRuntimeClient(t)\n\t\t\tlogger := logrus.New()\n\n\t\t\tpvcBIA := pvcBackupItemAction{\n\t\t\t\tlog: logger,\n\t\t\t\tcrClient: crClient,\n\t\t\t}\n\n\t\t\terr := crClient.Create(t.Context(), &tc.dataUpload)\n\t\t\trequire.NoError(t, err)\n\n\t\t\terr = pvcBIA.Cancel(tc.operationID, tc.backup)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tdu := new(velerov2alpha1.DataUpload)\n\t\t\terr = crClient.Get(t.Context(), crclient.ObjectKey{Namespace: tc.dataUpload.Namespace, Name: tc.dataUpload.Name}, du)\n\t\t\trequire.NoError(t, err)\n\n\t\t\trequire.True(t, cmp.Equal(tc.expectedDataUpload, *du, cmpopts.IgnoreFields(velerov2alpha1.DataUpload{}, \"ResourceVersion\")))\n\t\t})\n\t}\n}\n\nfunc TestPVCAppliesTo(t *testing.T) {\n\tp := pvcBackupItemAction{\n\t\tlog: logrus.StandardLogger(),\n\t}\n\tselector, err := p.AppliesTo()\n\n\trequire.NoError(t, err)\n\n\trequire.Equal(\n\t\tt,\n\t\tvelero.ResourceSelector{\n\t\t\tIncludedResources: []string{\"persistentvolumeclaims\"},\n\t\t},\n\t\tselector,\n\t)\n}\n\nfunc TestNewPVCBackupItemAction(t *testing.T) {\n\tlogger := logrus.StandardLogger()\n\tcrClient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\tf := &factorymocks.Factory{}\n\tf.On(\"KubebuilderClient\").Return(nil, fmt.Errorf(\"\"))\n\tplugin := NewPvcBackupItemAction(f)\n\t_, err := plugin(logger)\n\trequire.Error(t, err)\n\n\tf1 := &factorymocks.Factory{}\n\tf1.On(\"KubebuilderClient\").Return(crClient, nil)\n\tplugin1 := NewPvcBackupItemAction(f1)\n\t_, err1 := plugin1(logger)\n\trequire.NoError(t, err1)\n}\n\nfunc TestListGroupedPVCs(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tnamespace string\n\t\tlabelKey string\n\t\tgroupValue string\n\t\tpvcs []corev1api.PersistentVolumeClaim\n\t\texpectCount int\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"Match single PVC with label\",\n\t\t\tnamespace: \"ns1\",\n\t\t\tlabelKey: \"vgs-key\",\n\t\t\tgroupValue: \"group-a\",\n\t\t\tpvcs: []corev1api.PersistentVolumeClaim{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"pvc1\",\n\t\t\t\t\t\tNamespace: \"ns1\",\n\t\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\t\"vgs-key\": \"group-a\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectCount: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"No matching PVCs\",\n\t\t\tnamespace: \"ns1\",\n\t\t\tlabelKey: \"vgs-key\",\n\t\t\tgroupValue: \"group-b\",\n\t\t\tpvcs: []corev1api.PersistentVolumeClaim{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"pvc1\",\n\t\t\t\t\t\tNamespace: \"ns1\",\n\t\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\t\"vgs-key\": \"group-a\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectCount: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"Match multiple PVCs\",\n\t\t\tnamespace: \"ns1\",\n\t\t\tlabelKey: \"vgs-key\",\n\t\t\tgroupValue: \"group-a\",\n\t\t\tpvcs: []corev1api.PersistentVolumeClaim{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"pvc1\",\n\t\t\t\t\t\tNamespace: \"ns1\",\n\t\t\t\t\t\tLabels: map[string]string{\"vgs-key\": \"group-a\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"pvc2\",\n\t\t\t\t\t\tNamespace: \"ns1\",\n\t\t\t\t\t\tLabels: map[string]string{\"vgs-key\": \"group-a\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectCount: 2,\n\t\t},\n\t\t{\n\t\t\tname: \"Different namespace\",\n\t\t\tnamespace: \"ns2\",\n\t\t\tlabelKey: \"vgs-key\",\n\t\t\tgroupValue: \"group-a\",\n\t\t\tpvcs: []corev1api.PersistentVolumeClaim{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"pvc1\",\n\t\t\t\t\t\tNamespace: \"ns1\",\n\t\t\t\t\t\tLabels: map[string]string{\"vgs-key\": \"group-a\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectCount: 0,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tvar objs []runtime.Object\n\t\t\tfor i := range tt.pvcs {\n\t\t\t\tobjs = append(objs, &tt.pvcs[i])\n\t\t\t}\n\t\t\tclient := velerotest.NewFakeControllerRuntimeClient(t, objs...)\n\n\t\t\taction := &pvcBackupItemAction{\n\t\t\t\tlog: logrus.New(),\n\t\t\t\tcrClient: client,\n\t\t\t}\n\n\t\t\tresult, err := action.listGroupedPVCs(t.Context(), tt.namespace, tt.labelKey, tt.groupValue)\n\t\t\tif tt.expectError {\n\t\t\t\trequire.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\trequire.Len(t, result, tt.expectCount)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestFilterPVCsByVolumePolicy(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvcs []corev1api.PersistentVolumeClaim\n\t\tpvs []corev1api.PersistentVolume\n\t\tvolumePolicyStr string\n\t\texpectCount int\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"All PVCs should be included when no volume policy\",\n\t\t\tpvcs: []corev1api.PersistentVolumeClaim{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pvc-1\", Namespace: \"ns-1\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\t\tVolumeName: \"pv-1\",\n\t\t\t\t\t\tStorageClassName: pointer.String(\"sc-1\"),\n\t\t\t\t\t},\n\t\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{Phase: corev1api.ClaimBound},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pvc-2\", Namespace: \"ns-1\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\t\tVolumeName: \"pv-2\",\n\t\t\t\t\t\tStorageClassName: pointer.String(\"sc-1\"),\n\t\t\t\t\t},\n\t\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{Phase: corev1api.ClaimBound},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvs: []corev1api.PersistentVolume{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pv-1\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{Driver: \"csi-driver-1\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pv-2\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{Driver: \"csi-driver-1\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectCount: 2,\n\t\t},\n\t\t{\n\t\t\tname: \"Filter out NFS PVC by volume policy\",\n\t\t\tpvcs: []corev1api.PersistentVolumeClaim{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pvc-csi\", Namespace: \"ns-1\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\t\tVolumeName: \"pv-csi\",\n\t\t\t\t\t\tStorageClassName: pointer.String(\"sc-1\"),\n\t\t\t\t\t},\n\t\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{Phase: corev1api.ClaimBound},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pvc-nfs\", Namespace: \"ns-1\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\t\tVolumeName: \"pv-nfs\",\n\t\t\t\t\t\tStorageClassName: pointer.String(\"sc-nfs\"),\n\t\t\t\t\t},\n\t\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{Phase: corev1api.ClaimBound},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvs: []corev1api.PersistentVolume{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pv-csi\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{Driver: \"csi-driver\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pv-nfs\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\t\tNFS: &corev1api.NFSVolumeSource{\n\t\t\t\t\t\t\t\tServer: \"nfs-server\",\n\t\t\t\t\t\t\t\tPath: \"/export\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumePolicyStr: `\nversion: v1\nvolumePolicies:\n- conditions:\n nfs: {}\n action:\n type: skip\n`,\n\t\t\texpectCount: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"All PVCs filtered out by volume policy\",\n\t\t\tpvcs: []corev1api.PersistentVolumeClaim{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pvc-nfs-1\", Namespace: \"ns-1\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\t\tVolumeName: \"pv-nfs-1\",\n\t\t\t\t\t\tStorageClassName: pointer.String(\"sc-nfs\"),\n\t\t\t\t\t},\n\t\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{Phase: corev1api.ClaimBound},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pvc-nfs-2\", Namespace: \"ns-1\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\t\tVolumeName: \"pv-nfs-2\",\n\t\t\t\t\t\tStorageClassName: pointer.String(\"sc-nfs\"),\n\t\t\t\t\t},\n\t\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{Phase: corev1api.ClaimBound},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvs: []corev1api.PersistentVolume{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pv-nfs-1\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\t\tNFS: &corev1api.NFSVolumeSource{\n\t\t\t\t\t\t\t\tServer: \"nfs-server\",\n\t\t\t\t\t\t\t\tPath: \"/export/1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pv-nfs-2\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\t\tNFS: &corev1api.NFSVolumeSource{\n\t\t\t\t\t\t\t\tServer: \"nfs-server\",\n\t\t\t\t\t\t\t\tPath: \"/export/2\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumePolicyStr: `\nversion: v1\nvolumePolicies:\n- conditions:\n nfs: {}\n action:\n type: skip\n`,\n\t\t\texpectCount: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"Filter out non-CSI PVCs from mixed driver group\",\n\t\t\tpvcs: []corev1api.PersistentVolumeClaim{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"pvc-linstor\",\n\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\tLabels: map[string]string{\"app.kubernetes.io/instance\": \"myapp\"},\n\t\t\t\t\t},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\t\tVolumeName: \"pv-linstor\",\n\t\t\t\t\t\tStorageClassName: pointer.String(\"sc-linstor\"),\n\t\t\t\t\t},\n\t\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{Phase: corev1api.ClaimBound},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"pvc-nfs\",\n\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\tLabels: map[string]string{\"app.kubernetes.io/instance\": \"myapp\"},\n\t\t\t\t\t},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\t\tVolumeName: \"pv-nfs\",\n\t\t\t\t\t\tStorageClassName: pointer.String(\"sc-nfs\"),\n\t\t\t\t\t},\n\t\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{Phase: corev1api.ClaimBound},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvs: []corev1api.PersistentVolume{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pv-linstor\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{Driver: \"linstor.csi.linbit.com\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pv-nfs\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\t\tNFS: &corev1api.NFSVolumeSource{\n\t\t\t\t\t\t\t\tServer: \"nfs-server\",\n\t\t\t\t\t\t\t\tPath: \"/export\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumePolicyStr: `\nversion: v1\nvolumePolicies:\n- conditions:\n nfs: {}\n action:\n type: skip\n`,\n\t\t\texpectCount: 1,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tobjs := []runtime.Object{}\n\t\t\tfor i := range tt.pvs {\n\t\t\t\tobjs = append(objs, &tt.pvs[i])\n\t\t\t}\n\n\t\t\tclient := velerotest.NewFakeControllerRuntimeClient(t, objs...)\n\n\t\t\tbackup := &velerov1api.Backup{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-backup\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{},\n\t\t\t}\n\n\t\t\t// Add volume policy ConfigMap if specified\n\t\t\tif tt.volumePolicyStr != \"\" {\n\t\t\t\tcm := &corev1api.ConfigMap{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"volume-policy\",\n\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t},\n\t\t\t\t\tData: map[string]string{\n\t\t\t\t\t\t\"volume-policy\": tt.volumePolicyStr,\n\t\t\t\t\t},\n\t\t\t\t}\n\t\t\t\trequire.NoError(t, client.Create(t.Context(), cm))\n\n\t\t\t\tbackup.Spec.ResourcePolicy = &corev1api.TypedLocalObjectReference{\n\t\t\t\t\tKind: \"ConfigMap\",\n\t\t\t\t\tName: \"volume-policy\",\n\t\t\t\t}\n\t\t\t}\n\n\t\t\taction := &pvcBackupItemAction{\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t\tcrClient: client,\n\t\t\t}\n\n\t\t\t// Pass nil for VolumeHelper in tests - it will fall back to creating a new one per call\n\t\t\t// This is the expected behavior for testing and third-party plugins\n\t\t\tresult, err := action.filterPVCsByVolumePolicy(tt.pvcs, backup, nil)\n\t\t\tif tt.expectError {\n\t\t\t\trequire.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\trequire.Len(t, result, tt.expectCount)\n\n\t\t\t\t// For mixed driver scenarios, verify filtered result can determine single CSI driver\n\t\t\t\tif tt.name == \"Filter out non-CSI PVCs from mixed driver group\" && len(result) > 0 {\n\t\t\t\t\tdriver, err := action.determineCSIDriver(result)\n\t\t\t\t\trequire.NoError(t, err, \"After filtering, determineCSIDriver should not fail with multiple drivers error\")\n\t\t\t\t\trequire.Equal(t, \"linstor.csi.linbit.com\", driver, \"Should have the Linstor driver after filtering out NFS\")\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestFilterPVCsByVolumePolicyWithVolumeHelper tests filterPVCsByVolumePolicy when a\n// pre-created VolumeHelper is passed (non-nil). This exercises the cached path used\n// by the CSI PVC BIA plugin for better performance.\nfunc TestFilterPVCsByVolumePolicyWithVolumeHelper(t *testing.T) {\n\t// Create test PVCs and PVs\n\tpvcs := []corev1api.PersistentVolumeClaim{\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pvc-csi\", Namespace: \"ns-1\"},\n\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\tVolumeName: \"pv-csi\",\n\t\t\t\tStorageClassName: pointer.String(\"sc-csi\"),\n\t\t\t},\n\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{Phase: corev1api.ClaimBound},\n\t\t},\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pvc-nfs\", Namespace: \"ns-1\"},\n\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\tVolumeName: \"pv-nfs\",\n\t\t\t\tStorageClassName: pointer.String(\"sc-nfs\"),\n\t\t\t},\n\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{Phase: corev1api.ClaimBound},\n\t\t},\n\t}\n\n\tpvs := []corev1api.PersistentVolume{\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pv-csi\"},\n\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{Driver: \"csi-driver\"},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pv-nfs\"},\n\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\tNFS: &corev1api.NFSVolumeSource{\n\t\t\t\t\t\tServer: \"nfs-server\",\n\t\t\t\t\t\tPath: \"/export\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\t// Create fake client with PVs\n\tobjs := []runtime.Object{}\n\tfor i := range pvs {\n\t\tobjs = append(objs, &pvs[i])\n\t}\n\tclient := velerotest.NewFakeControllerRuntimeClient(t, objs...)\n\n\t// Create backup with volume policy that skips NFS volumes\n\tvolumePolicyStr := `\nversion: v1\nvolumePolicies:\n- conditions:\n nfs: {}\n action:\n type: skip\n`\n\tcm := &corev1api.ConfigMap{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"volume-policy\",\n\t\t\tNamespace: \"velero\",\n\t\t},\n\t\tData: map[string]string{\n\t\t\t\"volume-policy\": volumePolicyStr,\n\t\t},\n\t}\n\trequire.NoError(t, client.Create(t.Context(), cm))\n\n\tbackup := &velerov1api.Backup{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-backup\",\n\t\t\tNamespace: \"velero\",\n\t\t},\n\t\tSpec: velerov1api.BackupSpec{\n\t\t\tResourcePolicy: &corev1api.TypedLocalObjectReference{\n\t\t\t\tKind: \"ConfigMap\",\n\t\t\t\tName: \"volume-policy\",\n\t\t\t},\n\t\t},\n\t}\n\n\taction := &pvcBackupItemAction{\n\t\tlog: velerotest.NewLogger(),\n\t\tcrClient: client,\n\t}\n\n\t// Create a VolumeHelper using the same method the plugin would use\n\tvh, err := action.getOrCreateVolumeHelper(backup)\n\trequire.NoError(t, err)\n\trequire.NotNil(t, vh)\n\n\t// Test with the pre-created VolumeHelper (non-nil path)\n\tresult, err := action.filterPVCsByVolumePolicy(pvcs, backup, vh)\n\trequire.NoError(t, err)\n\n\t// Should filter out the NFS PVC, leaving only the CSI PVC\n\trequire.Len(t, result, 1)\n\trequire.Equal(t, \"pvc-csi\", result[0].Name)\n}\n\nfunc TestDetermineCSIDriver(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvcs []corev1api.PersistentVolumeClaim\n\t\tpvs []corev1api.PersistentVolume\n\t\texpectError bool\n\t\texpectedDriver string\n\t}{\n\t\t{\n\t\t\tname: \"Single PVC with CSI PV\",\n\t\t\tpvcs: []corev1api.PersistentVolumeClaim{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pvc-1\", Namespace: \"ns-1\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{VolumeName: \"pv-1\"},\n\t\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{Phase: corev1api.ClaimBound},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvs: []corev1api.PersistentVolume{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pv-1\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{Driver: \"csi-driver\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedDriver: \"csi-driver\",\n\t\t},\n\t\t{\n\t\t\tname: \"Multiple PVCs with same CSI driver\",\n\t\t\tpvcs: []corev1api.PersistentVolumeClaim{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pvc-1\", Namespace: \"ns-1\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{VolumeName: \"pv-1\"},\n\t\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{Phase: corev1api.ClaimBound},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pvc-2\", Namespace: \"ns-1\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{VolumeName: \"pv-2\"},\n\t\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{Phase: corev1api.ClaimBound},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvs: []corev1api.PersistentVolume{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pv-1\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{Driver: \"csi-driver\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pv-2\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{Driver: \"csi-driver\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedDriver: \"csi-driver\",\n\t\t},\n\t\t{\n\t\t\tname: \"PV not CSI provisioned\",\n\t\t\tpvcs: []corev1api.PersistentVolumeClaim{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pvc-1\", Namespace: \"ns-1\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{VolumeName: \"pv-1\"},\n\t\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{Phase: corev1api.ClaimBound},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvs: []corev1api.PersistentVolume{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pv-1\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Multiple PVCs with different CSI drivers\",\n\t\t\tpvcs: []corev1api.PersistentVolumeClaim{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pvc-1\", Namespace: \"ns-1\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{VolumeName: \"pv-1\"},\n\t\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{Phase: corev1api.ClaimBound},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pvc-2\", Namespace: \"ns-1\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{VolumeName: \"pv-2\"},\n\t\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{Phase: corev1api.ClaimBound},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvs: []corev1api.PersistentVolume{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pv-1\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{Driver: \"csi-driver-1\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pv-2\"},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{Driver: \"csi-driver-2\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectError: true,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tvar initObjs []runtime.Object\n\t\t\tfor i := range tt.pvcs {\n\t\t\t\tpvc := tt.pvcs[i]\n\t\t\t\tinitObjs = append(initObjs, &pvc)\n\t\t\t}\n\t\t\tfor i := range tt.pvs {\n\t\t\t\tpv := tt.pvs[i]\n\t\t\t\tinitObjs = append(initObjs, &pv)\n\t\t\t}\n\n\t\t\tclient := velerotest.NewFakeControllerRuntimeClient(t, initObjs...)\n\t\t\taction := &pvcBackupItemAction{\n\t\t\t\tlog: logrus.New(),\n\t\t\t\tcrClient: client,\n\t\t\t}\n\n\t\t\tdriver, err := action.determineCSIDriver(tt.pvcs)\n\t\t\tif tt.expectError {\n\t\t\t\trequire.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\trequire.Equal(t, tt.expectedDriver, driver)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestDetermineVGSClass(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\texistingVGSClass []volumegroupsnapshotv1beta1.VolumeGroupSnapshotClass\n\t\texpectError bool\n\t\texpectResult string\n\t}{\n\t\t{\n\t\t\tname: \"PVC annotation override\",\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tvelerov1api.VolumeGroupSnapshotClassAnnotationPVC: \"pvc-class\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tbackup: &velerov1api.Backup{},\n\t\t\texpectResult: \"pvc-class\",\n\t\t},\n\t\t{\n\t\t\tname: \"Backup annotation override\",\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{},\n\t\t\tbackup: &velerov1api.Backup{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tfmt.Sprintf(\"%s%s\", velerov1api.VolumeGroupSnapshotClassAnnotationBackupPrefix, testDriver): \"backup-class\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectResult: \"backup-class\",\n\t\t},\n\t\t{\n\t\t\tname: \"Default label-based match\",\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{},\n\t\t\tbackup: &velerov1api.Backup{},\n\t\t\texistingVGSClass: []volumegroupsnapshotv1beta1.VolumeGroupSnapshotClass{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"default-class\",\n\t\t\t\t\t\tLabels: map[string]string{velerov1api.VolumeGroupSnapshotClassDefaultLabel: \"true\"},\n\t\t\t\t\t},\n\t\t\t\t\tDriver: testDriver,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectResult: \"default-class\",\n\t\t},\n\t\t{\n\t\t\tname: \"No matching VGS class\",\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{},\n\t\t\tbackup: &velerov1api.Backup{},\n\t\t\texpectError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Multiple matching VGS classes\",\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{},\n\t\t\tbackup: &velerov1api.Backup{},\n\t\t\texistingVGSClass: []volumegroupsnapshotv1beta1.VolumeGroupSnapshotClass{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"class1\",\n\t\t\t\t\t\tLabels: map[string]string{velerov1api.VolumeGroupSnapshotClassDefaultLabel: \"true\"},\n\t\t\t\t\t},\n\t\t\t\t\tDriver: testDriver,\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"class2\",\n\t\t\t\t\t\tLabels: map[string]string{velerov1api.VolumeGroupSnapshotClassDefaultLabel: \"true\"},\n\t\t\t\t\t},\n\t\t\t\t\tDriver: testDriver,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectError: true,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tvar initObjs []runtime.Object\n\t\t\tfor _, vgsClass := range tt.existingVGSClass {\n\t\t\t\tvgsClassCopy := vgsClass\n\t\t\t\tinitObjs = append(initObjs, &vgsClassCopy)\n\t\t\t}\n\n\t\t\tclient := velerotest.NewFakeControllerRuntimeClient(t, initObjs...)\n\t\t\tlogger := logrus.New()\n\t\t\trequire.NoError(t, volumegroupsnapshotv1beta1.AddToScheme(client.Scheme()))\n\n\t\t\taction := &pvcBackupItemAction{crClient: client, log: logger}\n\n\t\t\tresult, err := action.determineVGSClass(t.Context(), testDriver, tt.backup, tt.pvc)\n\n\t\t\tif tt.expectError {\n\t\t\t\trequire.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\trequire.Equal(t, tt.expectResult, result)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestCreateVolumeGroupSnapshot(t *testing.T) {\n\ttestNamespace := \"test-ns\"\n\ttestLabelKey := \"velero.io/test-vgs-label\"\n\ttestLabelValue := \"group-1\"\n\ttestVGSClass := \"test-class\"\n\ttestBackup := &velerov1api.Backup{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-backup\",\n\t\t\tUID: \"test-uid\",\n\t\t},\n\t}\n\ttestPVC := corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-pvc\",\n\t\t\tNamespace: testNamespace,\n\t\t\tLabels: map[string]string{\n\t\t\t\ttestLabelKey: testLabelValue,\n\t\t\t},\n\t\t},\n\t}\n\n\tcrClient := velerotest.NewFakeControllerRuntimeClient(t)\n\tlog := logrus.New()\n\taction := &pvcBackupItemAction{\n\t\tlog: log,\n\t\tcrClient: crClient,\n\t}\n\n\tvgs, err := action.createVolumeGroupSnapshot(t.Context(), testBackup, testPVC, testLabelKey, testLabelValue, testVGSClass)\n\trequire.NoError(t, err)\n\trequire.NotNil(t, vgs)\n\n\t// Verify VGS fields\n\tassert.Equal(t, testNamespace, vgs.Namespace)\n\tassert.NotEmpty(t, vgs.GenerateName)\n\tassert.Equal(t, testVGSClass, *vgs.Spec.VolumeGroupSnapshotClassName)\n\tassert.NotNil(t, vgs.Spec.Source.Selector)\n\tassert.Equal(t, testLabelValue, vgs.Spec.Source.Selector.MatchLabels[testLabelKey])\n\tassert.Equal(t, testLabelValue, vgs.Labels[testLabelKey])\n\tassert.Equal(t, label.GetValidName(testBackup.Name), vgs.Labels[velerov1api.BackupNameLabel])\n\tassert.Equal(t, string(testBackup.UID), vgs.Labels[velerov1api.BackupUIDLabel])\n\n\t// Check that it exists in fake client\n\tretrieved := &volumegroupsnapshotv1beta1.VolumeGroupSnapshot{}\n\terr = crClient.Get(t.Context(), crclient.ObjectKey{Name: vgs.Name, Namespace: vgs.Namespace}, retrieved)\n\trequire.NoError(t, err)\n}\n\nfunc TestWaitForVGSAssociatedVS(t *testing.T) {\n\tvgs := &volumegroupsnapshotv1beta1.VolumeGroupSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-vgs\",\n\t\t\tNamespace: \"test-ns\",\n\t\t\tUID: types.UID(\"1234-5678-uuid\"),\n\t\t},\n\t}\n\n\tmakeVS := func(name string, hasStatus bool, hasVGSName bool, owned bool, pvcName string) *snapshotv1api.VolumeSnapshot {\n\t\tvar refs []metav1.OwnerReference\n\t\tif owned {\n\t\t\trefs = []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: \"groupsnapshot.storage.k8s.io/v1beta1\",\n\t\t\t\t\tKind: \"VolumeGroupSnapshot\",\n\t\t\t\t\tName: vgs.Name,\n\t\t\t\t\tUID: vgs.UID,\n\t\t\t\t},\n\t\t\t}\n\t\t}\n\n\t\tvs := &snapshotv1api.VolumeSnapshot{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: name,\n\t\t\t\tNamespace: vgs.Namespace,\n\t\t\t\tOwnerReferences: refs,\n\t\t\t},\n\t\t\tSpec: snapshotv1api.VolumeSnapshotSpec{\n\t\t\t\tSource: snapshotv1api.VolumeSnapshotSource{\n\t\t\t\t\tPersistentVolumeClaimName: pointer.String(pvcName),\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\n\t\tif hasStatus {\n\t\t\tvs.Status = &snapshotv1api.VolumeSnapshotStatus{}\n\t\t\tif hasVGSName {\n\t\t\t\tvs.Status.VolumeGroupSnapshotName = pointer.String(vgs.Name)\n\t\t\t}\n\t\t}\n\n\t\treturn vs\n\t}\n\n\tmakePVC := func(name string) corev1api.PersistentVolumeClaim {\n\t\treturn corev1api.PersistentVolumeClaim{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: name,\n\t\t\t\tNamespace: vgs.Namespace,\n\t\t\t},\n\t\t}\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tvsList []*snapshotv1api.VolumeSnapshot\n\t\tgroupedPVCs []corev1api.PersistentVolumeClaim\n\t\texpectErr bool\n\t\texpectVSMap int\n\t}{\n\t\t{\n\t\t\tname: \"all owned VS have VGS name\",\n\t\t\tvsList: []*snapshotv1api.VolumeSnapshot{\n\t\t\t\tmakeVS(\"vs1\", true, true, true, \"pvc1\"),\n\t\t\t\tmakeVS(\"vs2\", true, true, true, \"pvc2\"),\n\t\t\t},\n\t\t\tgroupedPVCs: []corev1api.PersistentVolumeClaim{\n\t\t\t\tmakePVC(\"pvc1\"),\n\t\t\t\tmakePVC(\"pvc2\"),\n\t\t\t},\n\t\t\texpectErr: false,\n\t\t\texpectVSMap: 2,\n\t\t},\n\t\t{\n\t\t\tname: \"one owned VS missing VGS name\",\n\t\t\tvsList: []*snapshotv1api.VolumeSnapshot{\n\t\t\t\tmakeVS(\"vs1\", true, true, true, \"pvc1\"),\n\t\t\t\tmakeVS(\"vs2\", true, false, true, \"pvc2\"),\n\t\t\t},\n\t\t\tgroupedPVCs: []corev1api.PersistentVolumeClaim{\n\t\t\t\tmakePVC(\"pvc1\"),\n\t\t\t\tmakePVC(\"pvc2\"),\n\t\t\t},\n\t\t\texpectErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"owned VS has no status\",\n\t\t\tvsList: []*snapshotv1api.VolumeSnapshot{\n\t\t\t\tmakeVS(\"vs1\", false, false, true, \"pvc1\"),\n\t\t\t},\n\t\t\tgroupedPVCs: []corev1api.PersistentVolumeClaim{\n\t\t\t\tmakePVC(\"pvc1\"),\n\t\t\t},\n\t\t\texpectErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"unrelated VS ignored\",\n\t\t\tvsList: []*snapshotv1api.VolumeSnapshot{\n\t\t\t\tmakeVS(\"vs1\", true, true, false, \"pvc1\"),\n\t\t\t},\n\t\t\tgroupedPVCs: []corev1api.PersistentVolumeClaim{\n\t\t\t\tmakePVC(\"pvc1\"),\n\t\t\t},\n\t\t\texpectErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"no owned VS present\",\n\t\t\tvsList: []*snapshotv1api.VolumeSnapshot{\n\t\t\t\tmakeVS(\"vs1\", true, true, false, \"pvc1\"),\n\t\t\t},\n\t\t\tgroupedPVCs: []corev1api.PersistentVolumeClaim{\n\t\t\t\tmakePVC(\"pvc1\"),\n\t\t\t},\n\t\t\texpectErr: true,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tvar objs []runtime.Object\n\t\t\tobjs = append(objs, vgs)\n\t\t\tfor _, vs := range tt.vsList {\n\t\t\t\tobjs = append(objs, vs)\n\t\t\t}\n\t\t\tfor _, pvc := range tt.groupedPVCs {\n\t\t\t\tobjs = append(objs, &pvc)\n\t\t\t}\n\n\t\t\tclient := velerotest.NewFakeControllerRuntimeClient(t, objs...)\n\t\t\taction := &pvcBackupItemAction{\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t\tcrClient: client,\n\t\t\t}\n\n\t\t\tvsMap, err := action.waitForVGSAssociatedVS(t.Context(), tt.groupedPVCs, vgs, 2*time.Second)\n\n\t\t\tif tt.expectErr {\n\t\t\t\tif err == nil {\n\t\t\t\t\tt.Errorf(\"expected error but got nil\")\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t\t\t}\n\t\t\t\tif len(vsMap) != tt.expectVSMap {\n\t\t\t\t\tt.Errorf(\"expected vsMap length %d, got %d\", tt.expectVSMap, len(vsMap))\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestUpdateVGSCreatedVS(t *testing.T) {\n\tbackup := &velerov1api.Backup{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"backup-1\",\n\t\t\tUID: \"backup-uid-123\",\n\t\t},\n\t}\n\n\tvgs := &volumegroupsnapshotv1beta1.VolumeGroupSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-vgs\",\n\t\t\tNamespace: \"ns\",\n\t\t\tUID: \"vgs-uid-123\",\n\t\t},\n\t}\n\n\tmakeVS := func(name string, withVGSOwner bool, vgsNamePtr *string, pvcName string) *snapshotv1api.VolumeSnapshot {\n\t\tvar refs []metav1.OwnerReference\n\t\tif withVGSOwner {\n\t\t\trefs = []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: \"groupsnapshot.storage.k8s.io/v1beta1\",\n\t\t\t\t\tKind: \"VolumeGroupSnapshot\",\n\t\t\t\t\tName: vgs.Name,\n\t\t\t\t\tUID: vgs.UID,\n\t\t\t\t},\n\t\t\t}\n\t\t}\n\t\treturn &snapshotv1api.VolumeSnapshot{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: name,\n\t\t\t\tNamespace: vgs.Namespace,\n\t\t\t\tOwnerReferences: refs,\n\t\t\t\tFinalizers: []string{\n\t\t\t\t\tVolumeSnapshotFinalizerGroupProtection,\n\t\t\t\t\tVolumeSnapshotFinalizerSourceProtection,\n\t\t\t\t},\n\t\t\t},\n\t\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\t\tReadyToUse: pointer.Bool(true),\n\t\t\t\tVolumeGroupSnapshotName: vgsNamePtr,\n\t\t\t},\n\t\t\tSpec: snapshotv1api.VolumeSnapshotSpec{\n\t\t\t\tSource: snapshotv1api.VolumeSnapshotSource{\n\t\t\t\t\tPersistentVolumeClaimName: pointer.String(pvcName),\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tvs *snapshotv1api.VolumeSnapshot\n\t\texpectOwnerCleared bool\n\t\texpectFinalizersCleared bool\n\t\texpectLabelPatched bool\n\t}{\n\t\t{\n\t\t\tname: \"should update owned VS\",\n\t\t\tvs: makeVS(\"vs-owned\", true, pointer.String(vgs.Name), \"pvc-1\"),\n\t\t\texpectOwnerCleared: true,\n\t\t\texpectFinalizersCleared: true,\n\t\t\texpectLabelPatched: true,\n\t\t},\n\t\t{\n\t\t\tname: \"should skip VS not owned by VGS\",\n\t\t\tvs: makeVS(\"vs-unowned\", false, nil, \"pvc-1\"),\n\t\t\texpectOwnerCleared: false,\n\t\t\texpectFinalizersCleared: false,\n\t\t\texpectLabelPatched: false,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tclient := velerotest.NewFakeControllerRuntimeClient(t, vgs, tt.vs)\n\t\t\taction := &pvcBackupItemAction{\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t\tcrClient: client,\n\t\t\t}\n\n\t\t\t// Build vsMap using the PVC name from the VS\n\t\t\tvsMap := map[string]*snapshotv1api.VolumeSnapshot{\n\t\t\t\t*tt.vs.Spec.Source.PersistentVolumeClaimName: tt.vs,\n\t\t\t}\n\n\t\t\terr := action.updateVGSCreatedVS(t.Context(), vsMap, vgs, backup)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Fetch updated VS\n\t\t\tupdated := &snapshotv1api.VolumeSnapshot{}\n\t\t\terr = client.Get(t.Context(), crclient.ObjectKey{Name: tt.vs.Name, Namespace: tt.vs.Namespace}, updated)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tif tt.expectOwnerCleared {\n\t\t\t\tassert.Empty(t, updated.OwnerReferences, \"expected ownerReferences to be cleared\")\n\t\t\t} else {\n\t\t\t\tassert.Equal(t, tt.vs.OwnerReferences, updated.OwnerReferences, \"expected ownerReferences to remain unchanged\")\n\t\t\t}\n\n\t\t\tif tt.expectFinalizersCleared {\n\t\t\t\tassert.Empty(t, updated.Finalizers, \"expected finalizers to be cleared\")\n\t\t\t} else {\n\t\t\t\tassert.Equal(t, tt.vs.Finalizers, updated.Finalizers, \"expected finalizers to remain unchanged\")\n\t\t\t}\n\n\t\t\tif tt.expectLabelPatched {\n\t\t\t\tassert.Equal(t, \"backup-1\", updated.Labels[velerov1api.BackupNameLabel])\n\t\t\t\tassert.Equal(t, \"backup-uid-123\", updated.Labels[velerov1api.BackupUIDLabel])\n\t\t\t} else {\n\t\t\t\tassert.Nil(t, updated.Labels, \"expected no labels to be patched\")\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestPatchVGSCDeletionPolicy(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tinitialPolicy snapshotv1api.DeletionPolicy\n\t\texpectedPolicy snapshotv1api.DeletionPolicy\n\t\texpectPatch bool\n\t\texpectErr bool\n\t}{\n\t\t{\n\t\t\tname: \"patches Delete to Retain\",\n\t\t\tinitialPolicy: snapshotv1api.VolumeSnapshotContentDelete,\n\t\t\texpectedPolicy: snapshotv1api.VolumeSnapshotContentRetain,\n\t\t\texpectPatch: true,\n\t\t},\n\t\t{\n\t\t\tname: \"no patch if already Retain\",\n\t\t\tinitialPolicy: snapshotv1api.VolumeSnapshotContentRetain,\n\t\t\texpectedPolicy: snapshotv1api.VolumeSnapshotContentRetain,\n\t\t\texpectPatch: false,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tvgsc := &volumegroupsnapshotv1beta1.VolumeGroupSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"test-vgsc\"},\n\t\t\t\tSpec: volumegroupsnapshotv1beta1.VolumeGroupSnapshotContentSpec{\n\t\t\t\t\tDeletionPolicy: tt.initialPolicy,\n\t\t\t\t},\n\t\t\t}\n\t\t\tvgs := &volumegroupsnapshotv1beta1.VolumeGroupSnapshot{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-vgs\",\n\t\t\t\t\tNamespace: \"ns\",\n\t\t\t\t},\n\t\t\t\tStatus: &volumegroupsnapshotv1beta1.VolumeGroupSnapshotStatus{\n\t\t\t\t\tBoundVolumeGroupSnapshotContentName: pointer.String(\"test-vgsc\"),\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tclient := velerotest.NewFakeControllerRuntimeClient(t, vgs, vgsc)\n\t\t\taction := &pvcBackupItemAction{\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t\tcrClient: client,\n\t\t\t}\n\n\t\t\terr := action.patchVGSCDeletionPolicy(t.Context(), vgs)\n\t\t\tif tt.expectErr {\n\t\t\t\trequire.Error(t, err)\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NoError(t, err)\n\n\t\t\tupdated := &volumegroupsnapshotv1beta1.VolumeGroupSnapshotContent{}\n\t\t\terr = client.Get(t.Context(), crclient.ObjectKey{Name: \"test-vgsc\"}, updated)\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.Equal(t, tt.expectedPolicy, updated.Spec.DeletionPolicy)\n\t\t})\n\t}\n}\n\nfunc TestDeleteVGSAndVGSC(t *testing.T) {\n\tmakeVGS := func(name, namespace string, boundVGSCName *string) *volumegroupsnapshotv1beta1.VolumeGroupSnapshot {\n\t\treturn &volumegroupsnapshotv1beta1.VolumeGroupSnapshot{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: name,\n\t\t\t\tNamespace: namespace,\n\t\t\t},\n\t\t\tStatus: &volumegroupsnapshotv1beta1.VolumeGroupSnapshotStatus{\n\t\t\t\tBoundVolumeGroupSnapshotContentName: boundVGSCName,\n\t\t\t},\n\t\t}\n\t}\n\n\tmakeVGSC := func(name string) *volumegroupsnapshotv1beta1.VolumeGroupSnapshotContent {\n\t\treturn &volumegroupsnapshotv1beta1.VolumeGroupSnapshotContent{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: name,\n\t\t\t},\n\t\t}\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tvgs *volumegroupsnapshotv1beta1.VolumeGroupSnapshot\n\t\texistingVGSC *volumegroupsnapshotv1beta1.VolumeGroupSnapshotContent\n\t\texpectVGSCDelete bool\n\t\texpectVGSDelete bool\n\t}{\n\t\t{\n\t\t\tname: \"deletes both VGSC and VGS\",\n\t\t\tvgs: makeVGS(\"test-vgs\", \"ns\", pointer.String(\"test-vgsc\")),\n\t\t\texistingVGSC: makeVGSC(\"test-vgsc\"),\n\t\t\texpectVGSCDelete: true,\n\t\t\texpectVGSDelete: true,\n\t\t},\n\t\t{\n\t\t\tname: \"VGSC not found, still deletes VGS\",\n\t\t\tvgs: makeVGS(\"test-vgs\", \"ns\", pointer.String(\"missing-vgsc\")),\n\t\t\texistingVGSC: nil,\n\t\t\texpectVGSCDelete: false,\n\t\t\texpectVGSDelete: true,\n\t\t},\n\t\t{\n\t\t\tname: \"no BoundVGSCName set, only deletes VGS\",\n\t\t\tvgs: makeVGS(\"test-vgs\", \"ns\", nil),\n\t\t\texistingVGSC: nil,\n\t\t\texpectVGSCDelete: false,\n\t\t\texpectVGSDelete: true,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tvar objs []runtime.Object\n\t\t\tobjs = append(objs, tt.vgs)\n\t\t\tif tt.existingVGSC != nil {\n\t\t\t\tobjs = append(objs, tt.existingVGSC)\n\t\t\t}\n\n\t\t\tclient := velerotest.NewFakeControllerRuntimeClient(t, objs...)\n\t\t\taction := &pvcBackupItemAction{\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t\tcrClient: client,\n\t\t\t}\n\n\t\t\terr := action.deleteVGSAndVGSC(t.Context(), tt.vgs)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Check VGSC is deleted\n\t\t\tif tt.expectVGSCDelete {\n\t\t\t\tgot := &volumegroupsnapshotv1beta1.VolumeGroupSnapshotContent{}\n\t\t\t\terr = client.Get(t.Context(), crclient.ObjectKey{Name: \"test-vgsc\"}, got)\n\t\t\t\tassert.True(t, apierrors.IsNotFound(err), \"expected VGSC to be deleted\")\n\t\t\t}\n\n\t\t\t// Check VGS is deleted\n\t\t\tgotVGS := &volumegroupsnapshotv1beta1.VolumeGroupSnapshot{}\n\t\t\terr = client.Get(t.Context(), crclient.ObjectKey{Name: \"test-vgs\", Namespace: \"ns\"}, gotVGS)\n\t\t\tassert.True(t, apierrors.IsNotFound(err), \"expected VGS to be deleted\")\n\t\t})\n\t}\n}\n\nfunc TestFindExistingVSForBackup(t *testing.T) {\n\tbackupUID := types.UID(\"backup-uid-123\")\n\tbackupName := \"backup-1\"\n\tpvcName := \"pvc-1\"\n\tnamespace := \"ns\"\n\n\tmakeVS := func(name, pvc string, match bool) *snapshotv1api.VolumeSnapshot {\n\t\tlabels := map[string]string{}\n\t\tif match {\n\t\t\tlabels[velerov1api.BackupNameLabel] = label.GetValidName(backupName)\n\t\t\tlabels[velerov1api.BackupUIDLabel] = string(backupUID)\n\t\t}\n\t\treturn &snapshotv1api.VolumeSnapshot{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: name,\n\t\t\t\tNamespace: namespace,\n\t\t\t\tLabels: labels,\n\t\t\t},\n\t\t\tSpec: snapshotv1api.VolumeSnapshotSpec{\n\t\t\t\tSource: snapshotv1api.VolumeSnapshotSource{\n\t\t\t\t\tPersistentVolumeClaimName: pointer.String(pvc),\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tvsList []*snapshotv1api.VolumeSnapshot\n\t\texpectName string\n\t\texpectNil bool\n\t}{\n\t\t{\n\t\t\tname: \"should find matching VS\",\n\t\t\tvsList: []*snapshotv1api.VolumeSnapshot{\n\t\t\t\tmakeVS(\"vs-match\", pvcName, true),\n\t\t\t},\n\t\t\texpectName: \"vs-match\",\n\t\t\texpectNil: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should skip VS with non-matching labels\",\n\t\t\tvsList: []*snapshotv1api.VolumeSnapshot{\n\t\t\t\tmakeVS(\"vs-nolabel\", pvcName, false),\n\t\t\t},\n\t\t\texpectNil: true,\n\t\t},\n\t\t{\n\t\t\tname: \"should skip VS with different PVC name\",\n\t\t\tvsList: []*snapshotv1api.VolumeSnapshot{\n\t\t\t\tmakeVS(\"vs-other-pvc\", \"other-pvc\", true),\n\t\t\t},\n\t\t\texpectNil: true,\n\t\t},\n\t\t{\n\t\t\tname: \"should return nil if VS list is empty\",\n\t\t\tvsList: []*snapshotv1api.VolumeSnapshot{},\n\t\t\texpectNil: true,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tvar objs []runtime.Object\n\t\t\tfor _, vs := range tt.vsList {\n\t\t\t\tobjs = append(objs, vs)\n\t\t\t}\n\n\t\t\tclient := velerotest.NewFakeControllerRuntimeClient(t, objs...)\n\t\t\taction := &pvcBackupItemAction{\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t\tcrClient: client,\n\t\t\t}\n\n\t\t\tvs, err := action.findExistingVSForBackup(t.Context(), backupUID, backupName, pvcName, namespace)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tif tt.expectNil {\n\t\t\t\tassert.Nil(t, vs)\n\t\t\t} else {\n\t\t\t\trequire.NotNil(t, vs)\n\t\t\t\tassert.Equal(t, tt.expectName, vs.Name)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestWaitForVGSCBinding(t *testing.T) {\n\tmakeVGS := func(name string, withStatus bool) *volumegroupsnapshotv1beta1.VolumeGroupSnapshot {\n\t\tvgs := &volumegroupsnapshotv1beta1.VolumeGroupSnapshot{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: name,\n\t\t\t\tNamespace: \"ns\",\n\t\t\t},\n\t\t}\n\t\tif withStatus {\n\t\t\tcontentName := \"vgsc-123\"\n\t\t\tvgs.Status = &volumegroupsnapshotv1beta1.VolumeGroupSnapshotStatus{\n\t\t\t\tBoundVolumeGroupSnapshotContentName: &contentName,\n\t\t\t}\n\t\t}\n\t\treturn vgs\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tvgs *volumegroupsnapshotv1beta1.VolumeGroupSnapshot\n\t\texpectErr bool\n\t}{\n\t\t{\n\t\t\tname: \"status is already bound\",\n\t\t\tvgs: makeVGS(\"vgs1\", true),\n\t\t\texpectErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"status is nil\",\n\t\t\tvgs: makeVGS(\"vgs2\", false),\n\t\t\texpectErr: true,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tclient := velerotest.NewFakeControllerRuntimeClient(t, tt.vgs.DeepCopy())\n\n\t\t\taction := &pvcBackupItemAction{\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t\tcrClient: client,\n\t\t\t}\n\n\t\t\terr := action.waitForVGSCBinding(t.Context(), tt.vgs, 1*time.Second)\n\n\t\t\tif tt.expectErr {\n\t\t\t\trequire.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\trequire.NotNil(t, tt.vgs.Status)\n\t\t\t\trequire.NotNil(t, tt.vgs.Status.BoundVolumeGroupSnapshotContentName)\n\t\t\t\trequire.Equal(t, \"vgsc-123\", *tt.vgs.Status.BoundVolumeGroupSnapshotContentName)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetVGSByLabels(t *testing.T) {\n\tlabelKey := \"velero.io/backup-name\"\n\tlabelVal := \"backup-123\"\n\ttestLabels := map[string]string{labelKey: labelVal}\n\n\tmakeVGS := func(name string, labels map[string]string) *volumegroupsnapshotv1beta1.VolumeGroupSnapshot {\n\t\treturn &volumegroupsnapshotv1beta1.VolumeGroupSnapshot{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: name,\n\t\t\t\tNamespace: \"test-ns\",\n\t\t\t\tLabels: labels,\n\t\t\t},\n\t\t}\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tvgsObjects []runtime.Object\n\t\texpectError string\n\t\texpectName string\n\t}{\n\t\t{\n\t\t\tname: \"exactly one matching VGS\",\n\t\t\tvgsObjects: []runtime.Object{\n\t\t\t\tmakeVGS(\"vgs1\", testLabels),\n\t\t\t},\n\t\t\texpectName: \"vgs1\",\n\t\t},\n\t\t{\n\t\t\tname: \"no matching VGS\",\n\t\t\tvgsObjects: []runtime.Object{},\n\t\t\texpectError: \"no VolumeGroupSnapshot found matching labels\",\n\t\t},\n\t\t{\n\t\t\tname: \"multiple matching VGS\",\n\t\t\tvgsObjects: []runtime.Object{\n\t\t\t\tmakeVGS(\"vgs1\", testLabels),\n\t\t\t\tmakeVGS(\"vgs2\", testLabels),\n\t\t\t},\n\t\t\texpectError: \"multiple VolumeGroupSnapshots found matching labels\",\n\t\t},\n\t\t{\n\t\t\tname: \"client list error\",\n\t\t\tvgsObjects: []runtime.Object{},\n\t\t\texpectError: \"failed to list VolumeGroupSnapshots by labels\",\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tvar client crclient.Client\n\t\t\tif tt.name == \"client list error\" {\n\t\t\t\t// Inject a client that always errors on List\n\t\t\t\tclient = &failingClient{}\n\t\t\t} else {\n\t\t\t\tclient = velerotest.NewFakeControllerRuntimeClient(t, tt.vgsObjects...)\n\t\t\t}\n\n\t\t\taction := &pvcBackupItemAction{\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t\tcrClient: client,\n\t\t\t}\n\n\t\t\tvgs, err := action.getVGSByLabels(t.Context(), \"test-ns\", testLabels)\n\n\t\t\tif tt.expectError != \"\" {\n\t\t\t\tif err == nil || !strings.Contains(err.Error(), tt.expectError) {\n\t\t\t\t\tt.Errorf(\"expected error containing '%s', got: %v\", tt.expectError, err)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t\t\t}\n\t\t\t\tif vgs == nil || vgs.Name != tt.expectName {\n\t\t\t\t\tt.Errorf(\"expected VGS name %s, got %v\", tt.expectName, vgs)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\n// failingClient is a dummy client that fails on List\ntype failingClient struct {\n\tcrclient.Client\n}\n\nfunc (f *failingClient) List(ctx context.Context, list crclient.ObjectList, opts ...crclient.ListOption) error {\n\treturn fmt.Errorf(\"simulated list error\")\n}\n\nfunc TestHasOwnerReference(t *testing.T) {\n\tvgs := &volumegroupsnapshotv1beta1.VolumeGroupSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-vgs\",\n\t\t\tNamespace: \"test-ns\",\n\t\t\tUID: types.UID(\"1234-uid\"),\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\townerRef metav1.OwnerReference\n\t\texpect bool\n\t}{\n\t\t{\n\t\t\tname: \"match kind, apiversion, uid\",\n\t\t\townerRef: metav1.OwnerReference{\n\t\t\t\tKind: kuberesource.VGSKind,\n\t\t\t\tAPIVersion: volumegroupsnapshotv1beta1.GroupName + \"/\" + volumegroupsnapshotv1beta1.SchemeGroupVersion.Version,\n\t\t\t\tUID: vgs.UID,\n\t\t\t},\n\t\t\texpect: true,\n\t\t},\n\t\t{\n\t\t\tname: \"mismatch kind\",\n\t\t\townerRef: metav1.OwnerReference{\n\t\t\t\tKind: \"other-kind\",\n\t\t\t\tAPIVersion: volumegroupsnapshotv1beta1.GroupName + \"/\" + volumegroupsnapshotv1beta1.SchemeGroupVersion.Version,\n\t\t\t\tUID: vgs.UID,\n\t\t\t},\n\t\t\texpect: false,\n\t\t},\n\t\t{\n\t\t\tname: \"mismatch apiversion\",\n\t\t\townerRef: metav1.OwnerReference{\n\t\t\t\tKind: kuberesource.VGSKind,\n\t\t\t\tAPIVersion: \"wrong.group/v1\",\n\t\t\t\tUID: vgs.UID,\n\t\t\t},\n\t\t\texpect: false,\n\t\t},\n\t\t{\n\t\t\tname: \"mismatch uid\",\n\t\t\townerRef: metav1.OwnerReference{\n\t\t\t\tKind: kuberesource.VGSKind,\n\t\t\t\tAPIVersion: volumegroupsnapshotv1beta1.GroupName + \"/\" + volumegroupsnapshotv1beta1.SchemeGroupVersion.Version,\n\t\t\t\tUID: \"wrong-uid\",\n\t\t\t},\n\t\t\texpect: false,\n\t\t},\n\t\t{\n\t\t\tname: \"no owner references\",\n\t\t\townerRef: metav1.OwnerReference{},\n\t\t\texpect: false,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tobj := &metav1.ObjectMeta{\n\t\t\t\tName: \"dummy\",\n\t\t\t\tNamespace: \"test-ns\",\n\t\t\t}\n\n\t\t\tif tt.name != \"no owner references\" {\n\t\t\t\tobj.OwnerReferences = []metav1.OwnerReference{tt.ownerRef}\n\t\t\t}\n\n\t\t\tfound := hasOwnerReference(obj, vgs)\n\t\t\tassert.Equal(t, tt.expect, found)\n\t\t})\n\t}\n}\n\nfunc TestPVCRequestSize(t *testing.T) {\n\tlogger := logrus.New()\n\n\ttests := []struct {\n\t\tname string\n\t\tpvcInitial *corev1api.PersistentVolumeClaim // Use full PVC to allow for nil Requests\n\t\trestoreSize string\n\t\texpectedSize string\n\t}{\n\t\t{\n\t\t\tname: \"UpdateRequired: PVC request is lower than restore size\",\n\t\t\tpvcInitial: func() *corev1api.PersistentVolumeClaim {\n\t\t\t\tpvc := builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").Result()\n\t\t\t\tpvc.Spec.Resources.Requests = corev1api.ResourceList{\n\t\t\t\t\tcorev1api.ResourceStorage: resource.MustParse(\"1Gi\"),\n\t\t\t\t}\n\t\t\t\treturn pvc\n\t\t\t}(),\n\t\t\trestoreSize: \"2Gi\",\n\t\t\texpectedSize: \"2Gi\",\n\t\t},\n\t\t{\n\t\t\tname: \"NoUpdateRequired: PVC request is larger than restore size\",\n\t\t\tpvcInitial: func() *corev1api.PersistentVolumeClaim {\n\t\t\t\tpvc := builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").Result()\n\t\t\t\tpvc.Spec.Resources.Requests = corev1api.ResourceList{\n\t\t\t\t\tcorev1api.ResourceStorage: resource.MustParse(\"3Gi\"),\n\t\t\t\t}\n\t\t\t\treturn pvc\n\t\t\t}(),\n\t\t\trestoreSize: \"2Gi\",\n\t\t\texpectedSize: \"3Gi\",\n\t\t},\n\t\t{\n\t\t\tname: \"PVC has no initial storage request\",\n\t\t\tpvcInitial: func() *corev1api.PersistentVolumeClaim {\n\t\t\t\tpvc := builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").Result()\n\t\t\t\tpvc.Spec.Resources.Requests = corev1api.ResourceList{} // Empty request list\n\t\t\t\treturn pvc\n\t\t\t}(),\n\t\t\trestoreSize: \"2Gi\",\n\t\t\texpectedSize: \"2Gi\",\n\t\t},\n\t\t{\n\t\t\tname: \"PVC has no initial Resources.Requests map\",\n\t\t\tpvcInitial: func() *corev1api.PersistentVolumeClaim {\n\t\t\t\tpvc := builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").Result()\n\t\t\t\tpvc.Spec.Resources.Requests = nil // This will trigger the line to be covered\n\t\t\t\treturn pvc\n\t\t\t}(),\n\t\t\trestoreSize: \"2Gi\",\n\t\t\texpectedSize: \"2Gi\",\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t// Create a VolumeSnapshotContent with restore size\n\t\t\trsQty := resource.MustParse(tc.restoreSize)\n\n\t\t\tvsc := &snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"testVSC\",\n\t\t\t\t},\n\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotContentStatus{\n\t\t\t\t\tRestoreSize: pointer.Int64(rsQty.Value()),\n\t\t\t\t},\n\t\t\t}\n\n\t\t\t// Call the function under test\n\t\t\tpvc := tc.pvcInitial\n\t\t\tsetPVCRequestSizeToVSRestoreSize(pvc, vsc, logger)\n\n\t\t\t// Verify that the PVC storage request is updated as expected.\n\t\t\tupdatedSize := pvc.Spec.Resources.Requests[corev1api.ResourceStorage]\n\t\t\texpected := resource.MustParse(tc.expectedSize)\n\t\t\t// Corrected line below:\n\t\t\trequire.Equal(t, 0, expected.Cmp(updatedSize), \"Expected size %s, but got %s\", expected.String(), updatedSize.String())\n\t\t})\n\t}\n}\n\n// TestGetOrCreateVolumeHelper tests the VolumeHelper and PVC-to-Pod cache behavior.\n// Since plugin instances are unique per backup (created via newPluginManager and\n// cleaned up via CleanupClients at backup completion), we verify that the pvcPodCache\n// is properly initialized and reused across calls.\nfunc TestGetOrCreateVolumeHelper(t *testing.T) {\n\tclient := velerotest.NewFakeControllerRuntimeClient(t)\n\taction := &pvcBackupItemAction{\n\t\tlog: velerotest.NewLogger(),\n\t\tcrClient: client,\n\t}\n\tbackup := &velerov1api.Backup{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-backup\",\n\t\t\tNamespace: \"velero\",\n\t\t\tUID: types.UID(\"test-uid-1\"),\n\t\t},\n\t}\n\n\t// Initially, pvcPodCache should be nil\n\trequire.Nil(t, action.pvcPodCache, \"pvcPodCache should be nil initially\")\n\n\t// Get VolumeHelper first time - should create new cache and VolumeHelper\n\tvh1, err := action.getOrCreateVolumeHelper(backup)\n\trequire.NoError(t, err)\n\trequire.NotNil(t, vh1)\n\n\t// pvcPodCache should now be initialized\n\trequire.NotNil(t, action.pvcPodCache, \"pvcPodCache should be initialized after first call\")\n\tcache1 := action.pvcPodCache\n\n\t// Get VolumeHelper second time - should reuse the same cache\n\tvh2, err := action.getOrCreateVolumeHelper(backup)\n\trequire.NoError(t, err)\n\trequire.NotNil(t, vh2)\n\n\t// The pvcPodCache should be the same instance\n\trequire.Same(t, cache1, action.pvcPodCache, \"Expected same pvcPodCache instance on repeated calls\")\n}\n" }, { "path": "pkg/backup/actions/csi/volumesnapshot_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strings\"\n\t\"time\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/api/resource\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\tcrclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\tplugincommon \"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tbiav2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/backupitemaction/v2\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/csi\"\n\tkubeutil \"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\n// volumeSnapshotBackupItemAction is a backup item action plugin to backup\n// CSI VolumeSnapshot objects using Velero\ntype volumeSnapshotBackupItemAction struct {\n\tlog logrus.FieldLogger\n\tcrClient crclient.Client\n}\n\n// AppliesTo returns information indicating that the\n// VolumeSnapshotBackupItemAction should be invoked to\n// backup VolumeSnapshots.\nfunc (p *volumeSnapshotBackupItemAction) AppliesTo() (\n\tvelero.ResourceSelector,\n\terror,\n) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"volumesnapshots.snapshot.storage.k8s.io\"},\n\t}, nil\n}\n\n// Execute backs up a CSI VolumeSnapshot object and captures, as labels and annotations,\n// information from its associated VolumeSnapshotContents such as CSI driver name,\n// storage snapshot handle and namespace and name of the snapshot delete secret, if any.\n// It returns the VolumeSnapshotClass and the VolumeSnapshotContents as additional items\n// to be backed up.\nfunc (p *volumeSnapshotBackupItemAction) Execute(\n\titem runtime.Unstructured,\n\tbackup *velerov1api.Backup,\n) (\n\truntime.Unstructured,\n\t[]velero.ResourceIdentifier,\n\tstring,\n\t[]velero.ResourceIdentifier,\n\terror,\n) {\n\tp.log.Infof(\"Executing VolumeSnapshotBackupItemAction\")\n\n\tvs := new(snapshotv1api.VolumeSnapshot)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(\n\t\titem.UnstructuredContent(), vs); err != nil {\n\t\treturn nil, nil, \"\", nil, errors.WithStack(err)\n\t}\n\n\tif backup.Status.Phase == velerov1api.BackupPhaseFinalizing ||\n\t\tbackup.Status.Phase == velerov1api.BackupPhaseFinalizingPartiallyFailed {\n\t\tp.log.\n\t\t\tWithField(\"Backup\", fmt.Sprintf(\"%s/%s\", backup.Namespace, backup.Name)).\n\t\t\tWithField(\"BackupPhase\", backup.Status.Phase).Debugf(\"Cleaning VolumeSnapshots.\")\n\n\t\tcsi.DeleteReadyVolumeSnapshot(*vs, p.crClient, p.log)\n\t\treturn item, nil, \"\", nil, nil\n\t}\n\n\tadditionalItems := make([]velero.ResourceIdentifier, 0)\n\n\tif vs.Spec.VolumeSnapshotClassName != nil {\n\t\t// This is still needed to add the VolumeSnapshotClass to the backup.\n\t\t// The secret with VolumeSnapshotClass is still relevant to backup.\n\t\tadditionalItems = append(\n\t\t\tadditionalItems,\n\t\t\tvelero.ResourceIdentifier{\n\t\t\t\tGroupResource: kuberesource.VolumeSnapshotClasses,\n\t\t\t\tName: *vs.Spec.VolumeSnapshotClassName,\n\t\t\t},\n\t\t)\n\n\t\t// Because async operation will update VolumeSnapshot during finalizing phase.\n\t\t// No matter what we do, VolumeSnapshotClass cannot be deleted. So skip it.\n\t\t// Just deleting VolumeSnapshotClass during restore and delete is enough.\n\t}\n\n\tp.log.Infof(\"Getting VolumesnapshotContent for Volumesnapshot %s/%s\",\n\t\tvs.Namespace, vs.Name)\n\n\tctx := context.TODO()\n\n\tvsc, err := csi.GetVSCForVS(ctx, vs, p.crClient)\n\tif err != nil {\n\t\tcsi.CleanupVolumeSnapshot(vs, p.crClient, p.log)\n\t\treturn nil, nil, \"\", nil, errors.WithStack(err)\n\t}\n\n\tannotations := make(map[string]string)\n\n\tif vsc != nil {\n\t\t// when we are backing up VolumeSnapshots created outside of velero, we\n\t\t// will not await VolumeSnapshot reconciliation and in this case\n\t\t// GetVolumeSnapshotContentForVolumeSnapshot may not find the associated\n\t\t// VolumeSnapshotContents to add to the backup. This is not an error\n\t\t// encountered in the backup process. So we add the VolumeSnapshotContent\n\t\t// to the backup only if one is found.\n\t\tadditionalItems = append(additionalItems, velero.ResourceIdentifier{\n\t\t\tGroupResource: kuberesource.VolumeSnapshotContents,\n\t\t\tName: vsc.Name,\n\t\t})\n\t\tannotations[velerov1api.VSCDeletionPolicyAnnotation] = string(vsc.Spec.DeletionPolicy)\n\n\t\tif vsc.Status != nil {\n\t\t\tif vsc.Status.SnapshotHandle != nil {\n\t\t\t\t// Capture storage provider snapshot handle and CSI driver name\n\t\t\t\t// to be used on restore to create a static VolumeSnapshotContent\n\t\t\t\t// that will be the source of the VolumeSnapshot.\n\t\t\t\tannotations[velerov1api.VolumeSnapshotHandleAnnotation] = *vsc.Status.SnapshotHandle\n\t\t\t\tannotations[velerov1api.DriverNameAnnotation] = vsc.Spec.Driver\n\t\t\t}\n\n\t\t\tif vsc.Status.RestoreSize != nil {\n\t\t\t\tannotations[velerov1api.VolumeSnapshotRestoreSize] = resource.NewQuantity(\n\t\t\t\t\t*vsc.Status.RestoreSize, resource.BinarySI).String()\n\t\t\t}\n\t\t}\n\n\t\tp.log.Infof(\"Patching VolumeSnapshotContent %s with velero BackupNameLabel\",\n\t\t\tvsc.Name)\n\t\t// If we created the VolumeSnapshotContent object during this ongoing backup,\n\t\t// we would have created it with a DeletionPolicy of Retain.\n\t\t// But, we want to retain these VolumeSnapshotContent ONLY for the lifetime\n\t\t// of the backup. To that effect, during velero backup\n\t\t// deletion, we will update the DeletionPolicy of the VolumeSnapshotContent\n\t\t// and then delete the VolumeSnapshot object which will cascade delete the\n\t\t// VolumeSnapshotContent and the associated snapshot in the storage\n\t\t// provider (handled by the CSI driver and the CSI common controller).\n\t\t// However, in the event that the VolumeSnapshot object is deleted outside\n\t\t// of the backup deletion process, it is possible that the dynamically created\n\t\t// VolumeSnapshotContent object will be left as an orphaned and non-discoverable\n\t\t// resource in the cluster as well as in the storage provider. To avoid piling\n\t\t// up of such orphaned resources, we will want to discover and delete the\n\t\t// dynamically created VolumeSnapshotContent. We do that by adding\n\t\t// the \"velero.io/backup-name\" label on the VolumeSnapshotContent.\n\t\t// Further, we want to add this label only on VolumeSnapshotContents that\n\t\t// were created during an ongoing velero backup.\n\t\toriginVSC := vsc.DeepCopy()\n\t\tkubeutil.AddLabels(\n\t\t\t&vsc.ObjectMeta,\n\t\t\tmap[string]string{\n\t\t\t\tvelerov1api.BackupNameLabel: label.GetValidName(backup.Name),\n\t\t\t},\n\t\t)\n\n\t\tif vscPatchError := p.crClient.Patch(\n\t\t\tcontext.TODO(),\n\t\t\tvsc,\n\t\t\tcrclient.MergeFrom(originVSC),\n\t\t); vscPatchError != nil {\n\t\t\tp.log.Warnf(\"Failed to patch VolumeSnapshotContent %s: %v\",\n\t\t\t\tvsc.Name, vscPatchError)\n\t\t}\n\t}\n\n\t// Before applying the BIA v2, the in-cluster VS state is not persisted into backup.\n\t// After the change, because the final state of VS will be stored in backup as the\n\t// result of async operation result, need to patch the annotations into VS to work,\n\t// because restore will check the annotations information.\n\toriginVS := vs.DeepCopy()\n\tkubeutil.AddAnnotations(&vs.ObjectMeta, annotations)\n\tif err := p.crClient.Patch(\n\t\tcontext.TODO(),\n\t\tvs,\n\t\tcrclient.MergeFrom(originVS),\n\t); err != nil {\n\t\tp.log.Errorf(\"Fail to patch VolumeSnapshot: %s.\", err.Error())\n\t\treturn nil, nil, \"\", nil, errors.WithStack(err)\n\t}\n\n\tannotations[velerov1api.MustIncludeAdditionalItemAnnotation] = \"true\"\n\t// save newly applied annotations into the backed-up VolumeSnapshot item\n\tkubeutil.AddAnnotations(&vs.ObjectMeta, annotations)\n\n\tvsMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(vs)\n\tif err != nil {\n\t\treturn nil, nil, \"\", nil, errors.WithStack(err)\n\t}\n\n\tp.log.Infof(\"Returning from VolumeSnapshotBackupItemAction with %d additionalItems to backup\",\n\t\tlen(additionalItems))\n\tfor _, ai := range additionalItems {\n\t\tp.log.Debugf(\"%s: %s\", ai.GroupResource.String(), ai.Name)\n\t}\n\n\toperationID := \"\"\n\tvar itemToUpdate []velero.ResourceIdentifier\n\n\t// Only return Async operation for VSC created for this backup.\n\t// The operationID is of the form //\n\toperationID = vs.Namespace + \"/\" + vs.Name + \"/\" + time.Now().Format(time.RFC3339)\n\titemToUpdate = []velero.ResourceIdentifier{\n\t\t{\n\t\t\tGroupResource: kuberesource.VolumeSnapshots,\n\t\t\tNamespace: vs.Namespace,\n\t\t\tName: vs.Name,\n\t\t},\n\t\t{\n\t\t\tGroupResource: kuberesource.VolumeSnapshotContents,\n\t\t\tName: vsc.Name,\n\t\t},\n\t}\n\n\treturn &unstructured.Unstructured{Object: vsMap},\n\t\tadditionalItems, operationID, itemToUpdate, nil\n}\n\n// Name returns the plugin's name.\nfunc (p *volumeSnapshotBackupItemAction) Name() string {\n\treturn \"VolumeSnapshotBackupItemAction\"\n}\n\nfunc (p *volumeSnapshotBackupItemAction) Progress(\n\toperationID string,\n\tbackup *velerov1api.Backup,\n) (velero.OperationProgress, error) {\n\tprogress := velero.OperationProgress{}\n\tif operationID == \"\" {\n\t\treturn progress, biav2.InvalidOperationIDError(operationID)\n\t}\n\t// The operationID is of the form //\n\toperationIDParts := strings.Split(operationID, \"/\")\n\tif len(operationIDParts) != 3 {\n\t\tp.log.Errorf(\"invalid operation ID %s\", operationID)\n\t\treturn progress, biav2.InvalidOperationIDError(operationID)\n\t}\n\tvar err error\n\tif progress.Started, err = time.Parse(time.RFC3339, operationIDParts[2]); err != nil {\n\t\tp.log.Errorf(\"error parsing operation ID's StartedTime\",\n\t\t\t\"part into time %s: %s\", operationID, err.Error())\n\t\treturn progress, errors.WithStack(err)\n\t}\n\n\tvs := new(snapshotv1api.VolumeSnapshot)\n\tif err := p.crClient.Get(\n\t\tcontext.Background(),\n\t\tcrclient.ObjectKey{\n\t\t\tNamespace: operationIDParts[0],\n\t\t\tName: operationIDParts[1],\n\t\t},\n\t\tvs); err != nil {\n\t\tp.log.Errorf(\"error getting volumesnapshot %s/%s: %s\",\n\t\t\toperationIDParts[0], operationIDParts[1], err.Error())\n\t\treturn progress, errors.WithStack(err)\n\t}\n\n\tif vs.Status == nil {\n\t\tp.log.Debugf(\"VolumeSnapshot %s/%s has an empty status. Skip progress update.\", vs.Namespace, vs.Name)\n\t\treturn progress, nil\n\t}\n\n\tif boolptr.IsSetToTrue(vs.Status.ReadyToUse) {\n\t\tp.log.Debugf(\"VolumeSnapshot %s/%s is ReadyToUse. Continue on querying corresponding VolumeSnapshotContent.\",\n\t\t\tvs.Namespace, vs.Name)\n\t} else if vs.Status.Error != nil {\n\t\terrorMessage := \"\"\n\t\tif vs.Status.Error.Message != nil {\n\t\t\terrorMessage = *vs.Status.Error.Message\n\t\t}\n\t\tp.log.Warnf(\"VolumeSnapshot has a temporary error %s. Snapshot controller will retry later.\",\n\t\t\terrorMessage)\n\n\t\treturn progress, nil\n\t}\n\n\tif vs.Status != nil && vs.Status.BoundVolumeSnapshotContentName != nil {\n\t\tvsc := new(snapshotv1api.VolumeSnapshotContent)\n\t\terr := p.crClient.Get(\n\t\t\tcontext.Background(),\n\t\t\tcrclient.ObjectKey{Name: *vs.Status.BoundVolumeSnapshotContentName},\n\t\t\tvsc,\n\t\t)\n\t\tif err != nil {\n\t\t\tp.log.Errorf(\"error getting VolumeSnapshotContent %s: %s\",\n\t\t\t\t*vs.Status.BoundVolumeSnapshotContentName, err.Error())\n\t\t\treturn progress, errors.WithStack(err)\n\t\t}\n\n\t\tif vsc.Status == nil {\n\t\t\tp.log.Debugf(\"VolumeSnapshotContent %s has an empty Status. Skip progress update.\",\n\t\t\t\tvsc.Name)\n\t\t\treturn progress, nil\n\t\t}\n\n\t\tnow := time.Now()\n\n\t\tif boolptr.IsSetToTrue(vsc.Status.ReadyToUse) {\n\t\t\tprogress.Completed = true\n\t\t\tprogress.Updated = now\n\t\t} else if vsc.Status.Error != nil {\n\t\t\tprogress.Completed = true\n\t\t\tprogress.Updated = now\n\t\t\tif vsc.Status.Error.Message != nil {\n\t\t\t\tprogress.Err = *vsc.Status.Error.Message\n\t\t\t}\n\t\t\tp.log.Warnf(\"VolumeSnapshotContent meets an error %s.\", progress.Err)\n\t\t}\n\t}\n\n\treturn progress, nil\n}\n\n// Cancel is not implemented for VolumeSnapshotBackupItemAction\nfunc (p *volumeSnapshotBackupItemAction) Cancel(\n\toperationID string,\n\tbackup *velerov1api.Backup,\n) error {\n\t// CSI Specification doesn't support canceling a snapshot creation.\n\treturn nil\n}\n\n// NewVolumeSnapshotBackupItemAction returns\n// VolumeSnapshotBackupItemAction instance.\nfunc NewVolumeSnapshotBackupItemAction(\n\tf client.Factory,\n) plugincommon.HandlerInitializer {\n\treturn func(logger logrus.FieldLogger) (any, error) {\n\t\tcrClient, err := f.KubebuilderClient()\n\t\tif err != nil {\n\t\t\treturn nil, errors.WithStack(err)\n\t\t}\n\n\t\treturn &volumeSnapshotBackupItemAction{\n\t\t\tlog: logger,\n\t\t\tcrClient: crClient,\n\t\t}, nil\n\t}\n}\n" }, { "path": "pkg/backup/actions/csi/volumesnapshot_action_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\t\"fmt\"\n\t\"testing\"\n\n\t\"github.com/google/go-cmp/cmp\"\n\t\"github.com/google/go-cmp/cmp/cmpopts\"\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestVSExecute(t *testing.T) {\n\tsnapshotHandle := \"handle\"\n\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\tvs *snapshotv1api.VolumeSnapshot\n\t\tvsc *snapshotv1api.VolumeSnapshotContent\n\t\texpectedErr string\n\t\texpectedAdditionalItems []velero.ResourceIdentifier\n\t\texpectedItemToUpdate []velero.ResourceIdentifier\n\t}{\n\t\t{\n\t\t\tname: \"Normal case\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").\n\t\t\t\tPhase(velerov1api.BackupPhaseInProgress).Result(),\n\t\t\tvs: builder.ForVolumeSnapshot(\"velero\", \"vs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\n\t\t\t\t\tvelerov1api.BackupNameLabel, \"backup\")).\n\t\t\t\tVolumeSnapshotClass(\"class\").Status().\n\t\t\t\tBoundVolumeSnapshotContentName(\"vsc\").Result(),\n\t\t\tvsc: builder.ForVolumeSnapshotContent(\"vsc\").Status(\n\t\t\t\t&snapshotv1api.VolumeSnapshotContentStatus{\n\t\t\t\t\tSnapshotHandle: &snapshotHandle,\n\t\t\t\t},\n\t\t\t).Result(),\n\t\t\texpectedErr: \"\",\n\t\t\texpectedAdditionalItems: []velero.ResourceIdentifier{\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.VolumeSnapshotClasses,\n\t\t\t\t\tName: \"class\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.VolumeSnapshotContents,\n\t\t\t\t\tName: \"vsc\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedItemToUpdate: []velero.ResourceIdentifier{\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.VolumeSnapshots,\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\tName: \"vs\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.VolumeSnapshotContents,\n\t\t\t\t\tName: \"vsc\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"VS not have VSClass\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").\n\t\t\t\tPhase(velerov1api.BackupPhaseInProgress).Result(),\n\t\t\tvs: builder.ForVolumeSnapshot(\"velero\", \"vs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\n\t\t\t\t\tvelerov1api.BackupNameLabel, \"backup\")).\n\t\t\t\tStatus().\n\t\t\t\tBoundVolumeSnapshotContentName(\"vsc\").Result(),\n\t\t\tvsc: builder.ForVolumeSnapshotContent(\"vsc\").Status(\n\t\t\t\t&snapshotv1api.VolumeSnapshotContentStatus{\n\t\t\t\t\tSnapshotHandle: &snapshotHandle,\n\t\t\t\t},\n\t\t\t).Result(),\n\t\t\texpectedErr: \"\",\n\t\t\texpectedAdditionalItems: []velero.ResourceIdentifier{\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.VolumeSnapshotContents,\n\t\t\t\t\tName: \"vsc\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedItemToUpdate: []velero.ResourceIdentifier{\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.VolumeSnapshots,\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\tName: \"vs\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.VolumeSnapshotContents,\n\t\t\t\t\tName: \"vsc\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Backup in finalizing phase - skip VSC lookup\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").\n\t\t\t\tPhase(velerov1api.BackupPhaseFinalizing).Result(),\n\t\t\tvs: builder.ForVolumeSnapshot(\"velero\", \"vs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\n\t\t\t\t\tvelerov1api.BackupNameLabel, \"backup\")).\n\t\t\t\tStatus().\n\t\t\t\tBoundVolumeSnapshotContentName(\"vsc\").Result(),\n\t\t\tvsc: nil, // VSC won't be created/fetched\n\t\t\texpectedErr: \"\",\n\t\t\texpectedAdditionalItems: nil,\n\t\t\texpectedItemToUpdate: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"Backup in finalizing partially failed phase - skip VSC lookup\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").\n\t\t\t\tPhase(velerov1api.BackupPhaseFinalizingPartiallyFailed).Result(),\n\t\t\tvs: builder.ForVolumeSnapshot(\"velero\", \"vs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\n\t\t\t\t\tvelerov1api.BackupNameLabel, \"backup\")).\n\t\t\t\tStatus().\n\t\t\t\tBoundVolumeSnapshotContentName(\"vsc\").Result(),\n\t\t\tvsc: nil, // VSC won't be created/fetched\n\t\t\texpectedErr: \"\",\n\t\t\texpectedAdditionalItems: nil,\n\t\t\texpectedItemToUpdate: nil,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(*testing.T) {\n\t\t\tvsBIA := volumeSnapshotBackupItemAction{\n\t\t\t\tlog: logrus.New(),\n\t\t\t\tcrClient: velerotest.NewFakeControllerRuntimeClient(t, tc.vs),\n\t\t\t}\n\n\t\t\titem, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.vs)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tif tc.vsc != nil {\n\t\t\t\trequire.NoError(t, vsBIA.crClient.Create(t.Context(), tc.vsc))\n\t\t\t}\n\n\t\t\t_, additionalItems, _, itemToUpdate, err := vsBIA.Execute(&unstructured.UnstructuredList{Object: item}, tc.backup)\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t} else {\n\t\t\t\trequire.Equal(t, tc.expectedErr, err.Error())\n\t\t\t}\n\n\t\t\trequire.ElementsMatch(t, tc.expectedAdditionalItems, additionalItems)\n\t\t\trequire.ElementsMatch(t, tc.expectedItemToUpdate, itemToUpdate)\n\t\t})\n\t}\n}\n\nfunc TestVSProgress(t *testing.T) {\n\terrorStr := \"error\"\n\treadyToUse := true\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\tvs *snapshotv1api.VolumeSnapshot\n\t\tvsc *snapshotv1api.VolumeSnapshotContent\n\t\toperationID string\n\t\texpectedErr bool\n\t\texpectedProgress *velero.OperationProgress\n\t}{\n\t\t{\n\t\t\tname: \"Empty OperationID\",\n\t\t\toperationID: \"\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").Result(),\n\t\t\texpectedErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"OperationID doesn't have slash\",\n\t\t\toperationID: \"invalid\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").Result(),\n\t\t\texpectedErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"OperationID doesn't have valid timestamp\",\n\t\t\toperationID: \"ns/name/invalid\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").Result(),\n\t\t\texpectedErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"OperationID represents VS does not exist\",\n\t\t\toperationID: \"ns/name/2024-04-11T18:49:00+08:00\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").Result(),\n\t\t\texpectedErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"VS status is nil\",\n\t\t\toperationID: \"ns/name/2024-04-11T18:49:00+08:00\",\n\t\t\tvs: builder.ForVolumeSnapshot(\"ns\", \"name\").Result(),\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").Result(),\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"VS status has error\",\n\t\t\toperationID: \"ns/name/2024-04-11T18:49:00+08:00\",\n\t\t\tvs: builder.ForVolumeSnapshot(\"ns\", \"name\").Status().\n\t\t\t\tStatusError(snapshotv1api.VolumeSnapshotError{\n\t\t\t\t\tMessage: &errorStr,\n\t\t\t\t}).Result(),\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").Result(),\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Fail to get VSC\",\n\t\t\toperationID: \"ns/name/2024-04-11T18:49:00+08:00\",\n\t\t\tvs: builder.ForVolumeSnapshot(\"ns\", \"name\").Status().\n\t\t\t\tReadyToUse(true).BoundVolumeSnapshotContentName(\"vsc\").Result(),\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").Result(),\n\t\t\texpectedErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"VSC status is nil\",\n\t\t\toperationID: \"ns/name/2024-04-11T18:49:00+08:00\",\n\t\t\tvs: builder.ForVolumeSnapshot(\"ns\", \"name\").Status().\n\t\t\t\tReadyToUse(true).BoundVolumeSnapshotContentName(\"vsc\").Result(),\n\t\t\tvsc: builder.ForVolumeSnapshotContent(\"vsc\").Result(),\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").Result(),\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"VSC is ReadyToUse\",\n\t\t\toperationID: \"ns/name/2024-04-11T18:49:00+08:00\",\n\t\t\tvs: builder.ForVolumeSnapshot(\"ns\", \"name\").Status().\n\t\t\t\tReadyToUse(true).BoundVolumeSnapshotContentName(\"vsc\").Result(),\n\t\t\tvsc: builder.ForVolumeSnapshotContent(\"vsc\").\n\t\t\t\tStatus(&snapshotv1api.VolumeSnapshotContentStatus{\n\t\t\t\t\tReadyToUse: &readyToUse,\n\t\t\t\t}).Result(),\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").Result(),\n\t\t\texpectedErr: false,\n\t\t\texpectedProgress: &velero.OperationProgress{Completed: true},\n\t\t},\n\t\t{\n\t\t\tname: \"VSC status has error\",\n\t\t\toperationID: \"ns/name/2024-04-11T18:49:00+08:00\",\n\t\t\tvs: builder.ForVolumeSnapshot(\"ns\", \"name\").Status().\n\t\t\t\tReadyToUse(true).BoundVolumeSnapshotContentName(\"vsc\").Result(),\n\t\t\tvsc: builder.ForVolumeSnapshotContent(\"vsc\").\n\t\t\t\tStatus(&snapshotv1api.VolumeSnapshotContentStatus{\n\t\t\t\t\tError: &snapshotv1api.VolumeSnapshotError{\n\t\t\t\t\t\tMessage: &errorStr,\n\t\t\t\t\t},\n\t\t\t\t}).Result(),\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").Result(),\n\t\t\texpectedErr: false,\n\t\t\texpectedProgress: &velero.OperationProgress{\n\t\t\t\tCompleted: true,\n\t\t\t\tErr: \"error\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(*testing.T) {\n\t\t\tcrClient := velerotest.NewFakeControllerRuntimeClient(t)\n\t\t\tlogger := logrus.New()\n\n\t\t\tvsBIA := volumeSnapshotBackupItemAction{\n\t\t\t\tlog: logger,\n\t\t\t\tcrClient: crClient,\n\t\t\t}\n\n\t\t\tif tc.vs != nil {\n\t\t\t\terr := crClient.Create(t.Context(), tc.vs)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif tc.vsc != nil {\n\t\t\t\trequire.NoError(t, crClient.Create(t.Context(), tc.vsc))\n\t\t\t}\n\n\t\t\tprogress, err := vsBIA.Progress(tc.operationID, tc.backup)\n\t\t\tif tc.expectedErr == false {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif tc.expectedProgress != nil {\n\t\t\t\trequire.True(\n\t\t\t\t\tt,\n\t\t\t\t\tcmp.Equal(\n\t\t\t\t\t\t*tc.expectedProgress,\n\t\t\t\t\t\tprogress,\n\t\t\t\t\t\tcmpopts.IgnoreFields(\n\t\t\t\t\t\t\tvelero.OperationProgress{},\n\t\t\t\t\t\t\t\"Started\",\n\t\t\t\t\t\t\t\"Updated\",\n\t\t\t\t\t\t),\n\t\t\t\t\t),\n\t\t\t\t)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestVSAppliesTo(t *testing.T) {\n\tp := volumeSnapshotBackupItemAction{\n\t\tlog: logrus.StandardLogger(),\n\t}\n\tselector, err := p.AppliesTo()\n\n\trequire.NoError(t, err)\n\n\trequire.Equal(\n\t\tt,\n\t\tvelero.ResourceSelector{\n\t\t\tIncludedResources: []string{\"volumesnapshots.snapshot.storage.k8s.io\"},\n\t\t},\n\t\tselector,\n\t)\n}\n\nfunc TestNewVolumeSnapshotBackupItemAction(t *testing.T) {\n\tlogger := logrus.StandardLogger()\n\tcrClient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\tf := &factorymocks.Factory{}\n\tf.On(\"KubebuilderClient\").Return(nil, fmt.Errorf(\"\"))\n\tplugin := NewVolumeSnapshotBackupItemAction(f)\n\t_, err := plugin(logger)\n\trequire.Error(t, err)\n\n\tf1 := &factorymocks.Factory{}\n\tf1.On(\"KubebuilderClient\").Return(crClient, nil)\n\tplugin1 := NewVolumeSnapshotBackupItemAction(f1)\n\t_, err1 := plugin1(logger)\n\trequire.NoError(t, err1)\n}\n" }, { "path": "pkg/backup/actions/csi/volumesnapshotclass_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tcsiutil \"github.com/vmware-tanzu/velero/pkg/util/csi\"\n\tkubeutil \"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\n// volumeSnapshotClassBackupItemAction is a backup item action plugin to\n// backup CSI VolumeSnapshotClass objects using Velero\ntype volumeSnapshotClassBackupItemAction struct {\n\tlog logrus.FieldLogger\n}\n\n// AppliesTo returns information indicating that the\n// VolumeSnapshotClassBackupItemAction action should be invoked\n// to backup VolumeSnapshotClass.\nfunc (p *volumeSnapshotClassBackupItemAction) AppliesTo() (\n\tvelero.ResourceSelector,\n\terror,\n) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"volumesnapshotclasses.snapshot.storage.k8s.io\"},\n\t}, nil\n}\n\n// Execute backs up a VolumeSnapshotClass object and returns as additional\n// items any snapshot lister secret that may be referenced in its annotations.\nfunc (p *volumeSnapshotClassBackupItemAction) Execute(\n\titem runtime.Unstructured,\n\tbackup *velerov1api.Backup,\n) (\n\truntime.Unstructured,\n\t[]velero.ResourceIdentifier,\n\tstring,\n\t[]velero.ResourceIdentifier,\n\terror,\n) {\n\tp.log.Infof(\"Executing VolumeSnapshotClassBackupItemAction\")\n\n\tvar snapClass snapshotv1api.VolumeSnapshotClass\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(\n\t\titem.UnstructuredContent(),\n\t\t&snapClass,\n\t); err != nil {\n\t\treturn nil, nil, \"\", nil, errors.WithStack(err)\n\t}\n\n\tadditionalItems := []velero.ResourceIdentifier{}\n\tif csiutil.IsVolumeSnapshotClassHasListerSecret(&snapClass) {\n\t\tadditionalItems = append(additionalItems, velero.ResourceIdentifier{\n\t\t\tGroupResource: kuberesource.Secrets,\n\t\t\tName: snapClass.Annotations[velerov1api.PrefixedListSecretNameAnnotation],\n\t\t\tNamespace: snapClass.Annotations[velerov1api.PrefixedListSecretNamespaceAnnotation],\n\t\t})\n\n\t\tkubeutil.AddAnnotations(&snapClass.ObjectMeta, map[string]string{\n\t\t\tvelerov1api.MustIncludeAdditionalItemAnnotation: \"true\",\n\t\t})\n\t}\n\n\tsnapClassMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&snapClass)\n\tif err != nil {\n\t\treturn nil, nil, \"\", nil, errors.WithStack(err)\n\t}\n\n\tp.log.Infof(\n\t\t\"Returning from VolumeSnapshotClassBackupItemAction with %d additionalItems to backup\",\n\t\tlen(additionalItems),\n\t)\n\treturn &unstructured.Unstructured{Object: snapClassMap}, additionalItems, \"\", nil, nil\n}\n\n// Name returns the plugin's name.\nfunc (p *volumeSnapshotClassBackupItemAction) Name() string {\n\treturn \"VolumeSnapshotClassBackupItemAction\"\n}\n\n// Progress is not implemented for VolumeSnapshotClassBackupItemAction\nfunc (p *volumeSnapshotClassBackupItemAction) Progress(\n\toperationID string,\n\tbackup *velerov1api.Backup,\n) (velero.OperationProgress, error) {\n\treturn velero.OperationProgress{}, nil\n}\n\n// Cancel is not implemented for VolumeSnapshotClassBackupItemAction\nfunc (p *volumeSnapshotClassBackupItemAction) Cancel(\n\toperationID string,\n\tbackup *velerov1api.Backup,\n) error {\n\treturn nil\n}\n\n// NewVolumeSnapshotClassBackupItemAction returns a\n// VolumeSnapshotClassBackupItemAction instance.\nfunc NewVolumeSnapshotClassBackupItemAction(logger logrus.FieldLogger) (any, error) {\n\treturn &volumeSnapshotClassBackupItemAction{log: logger}, nil\n}\n" }, { "path": "pkg/backup/actions/csi/volumesnapshotclass_action_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\t\"testing\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\nfunc TestVSClassExecute(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\titem runtime.Unstructured\n\t\tvsClass *snapshotv1api.VolumeSnapshotClass\n\t\tbackup *velerov1api.Backup\n\t\texpectErr bool\n\t\texpectedItems []velero.ResourceIdentifier\n\t}{\n\t\t{\n\t\t\tname: \"No Secret in the VS Class, no return additional items\",\n\t\t\tvsClass: builder.ForVolumeSnapshotClass(\"test\").Result(),\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").Result(),\n\t\t\texpectErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Normal case, additional items should return\",\n\t\t\tvsClass: builder.ForVolumeSnapshotClass(\"test\").ObjectMeta(builder.WithAnnotationsMap(\n\t\t\t\tmap[string]string{\n\t\t\t\t\tvelerov1api.PrefixedListSecretNameAnnotation: \"name\",\n\t\t\t\t\tvelerov1api.PrefixedListSecretNamespaceAnnotation: \"namespace\",\n\t\t\t\t},\n\t\t\t)).Result(),\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").Result(),\n\t\t\texpectErr: false,\n\t\t\texpectedItems: []velero.ResourceIdentifier{\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.Secrets,\n\t\t\t\t\tNamespace: \"namespace\",\n\t\t\t\t\tName: \"name\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tp, err := NewVolumeSnapshotClassBackupItemAction(logrus.StandardLogger())\n\t\t\trequire.NoError(t, err)\n\n\t\t\taction := p.(*volumeSnapshotClassBackupItemAction)\n\n\t\t\tif test.vsClass != nil {\n\t\t\t\tvsMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(test.vsClass)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\ttest.item = &unstructured.Unstructured{Object: vsMap}\n\t\t\t}\n\n\t\t\t_, additionalItems, _, _, err := action.Execute(\n\t\t\t\ttest.item,\n\t\t\t\ttest.backup,\n\t\t\t)\n\n\t\t\tif test.expectErr == false {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif len(test.expectedItems) > 0 {\n\t\t\t\trequire.Equal(t, test.expectedItems, additionalItems)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestVSClassAppliesTo(t *testing.T) {\n\tp := volumeSnapshotClassBackupItemAction{\n\t\tlog: logrus.StandardLogger(),\n\t}\n\tselector, err := p.AppliesTo()\n\n\trequire.NoError(t, err)\n\n\trequire.Equal(\n\t\tt,\n\t\tvelero.ResourceSelector{\n\t\t\tIncludedResources: []string{\"volumesnapshotclasses.snapshot.storage.k8s.io\"},\n\t\t},\n\t\tselector,\n\t)\n}\n" }, { "path": "pkg/backup/actions/csi/volumesnapshotcontent_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\t\"fmt\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tcsiutil \"github.com/vmware-tanzu/velero/pkg/util/csi\"\n\tkubeutil \"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\n// volumeSnapshotContentBackupItemAction is a backup item action plugin to backup\n// CSI VolumeSnapshotContent objects using Velero\ntype volumeSnapshotContentBackupItemAction struct {\n\tlog logrus.FieldLogger\n}\n\n// AppliesTo returns information indicating that the\n// VolumeSnapshotContentBackupItemAction action should be invoked to\n// backup VolumeSnapshotContents.\nfunc (p *volumeSnapshotContentBackupItemAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"volumesnapshotcontents.snapshot.storage.k8s.io\"},\n\t}, nil\n}\n\n// Execute returns the unmodified VolumeSnapshotContent object along\n// with the snapshot deletion secret, if any, from its annotation\n// as additional items to backup.\nfunc (p *volumeSnapshotContentBackupItemAction) Execute(\n\titem runtime.Unstructured,\n\tbackup *velerov1api.Backup,\n) (\n\truntime.Unstructured,\n\t[]velero.ResourceIdentifier,\n\tstring,\n\t[]velero.ResourceIdentifier,\n\terror,\n) {\n\tp.log.Infof(\"Executing VolumeSnapshotContentBackupItemAction\")\n\n\tif backup.Status.Phase == velerov1api.BackupPhaseFinalizing ||\n\t\tbackup.Status.Phase == velerov1api.BackupPhaseFinalizingPartiallyFailed {\n\t\tp.log.WithField(\"Backup\", fmt.Sprintf(\"%s/%s\", backup.Namespace, backup.Name)).\n\t\t\tWithField(\"BackupPhase\", backup.Status.Phase).\n\t\t\tDebug(\"Skipping VolumeSnapshotContentBackupItemAction\",\n\t\t\t\t\"as backup is in finalizing phase.\")\n\t\treturn item, nil, \"\", nil, nil\n\t}\n\n\tvar snapCont snapshotv1api.VolumeSnapshotContent\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(\n\t\titem.UnstructuredContent(),\n\t\t&snapCont,\n\t); err != nil {\n\t\treturn nil, nil, \"\", nil, errors.WithStack(err)\n\t}\n\n\tadditionalItems := []velero.ResourceIdentifier{}\n\n\t// we should backup the snapshot deletion secrets that may be referenced\n\t// in the VolumeSnapshotContent's annotation\n\tif csiutil.IsVolumeSnapshotContentHasDeleteSecret(&snapCont) {\n\t\tadditionalItems = append(\n\t\t\tadditionalItems,\n\t\t\tvelero.ResourceIdentifier{\n\t\t\t\tGroupResource: kuberesource.Secrets,\n\t\t\t\tName: snapCont.Annotations[velerov1api.PrefixedSecretNameAnnotation],\n\t\t\t\tNamespace: snapCont.Annotations[velerov1api.PrefixedSecretNamespaceAnnotation],\n\t\t\t})\n\n\t\tkubeutil.AddAnnotations(&snapCont.ObjectMeta, map[string]string{\n\t\t\tvelerov1api.MustIncludeAdditionalItemAnnotation: \"true\",\n\t\t})\n\t}\n\n\t// Because async operation will update VolumeSnapshotContent during finalizing phase.\n\t// No matter what we do, VolumeSnapshotClass cannot be deleted. So skip it.\n\t// Just deleting VolumeSnapshotClass during restore and delete is enough.\n\n\tsnapContMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&snapCont)\n\tif err != nil {\n\t\treturn nil, nil, \"\", nil, errors.WithStack(err)\n\t}\n\n\tp.log.Infof(\n\t\t\"Returning from VolumeSnapshotContentBackupItemAction\",\n\t\t\"with %d additionalItems to backup\",\n\t\tlen(additionalItems),\n\t)\n\treturn &unstructured.Unstructured{Object: snapContMap}, additionalItems, \"\", nil, nil\n}\n\n// Name returns the plugin's name.\nfunc (p *volumeSnapshotContentBackupItemAction) Name() string {\n\treturn \"VolumeSnapshotContentBackupItemAction\"\n}\n\n// Progress is not implemented for VolumeSnapshotContentBackupItemAction.\nfunc (p *volumeSnapshotContentBackupItemAction) Progress(\n\toperationID string,\n\tbackup *velerov1api.Backup,\n) (velero.OperationProgress, error) {\n\treturn velero.OperationProgress{}, nil\n}\n\n// Cancel is not implemented for VolumeSnapshotContentBackupItemAction.\nfunc (p *volumeSnapshotContentBackupItemAction) Cancel(\n\toperationID string,\n\tbackup *velerov1api.Backup,\n) error {\n\t// CSI Specification doesn't support canceling a snapshot creation.\n\treturn nil\n}\n\n// NewVolumeSnapshotContentBackupItemAction returns a\n// VolumeSnapshotContentBackupItemAction instance.\nfunc NewVolumeSnapshotContentBackupItemAction(\n\tlogger logrus.FieldLogger,\n) (any, error) {\n\treturn &volumeSnapshotContentBackupItemAction{log: logger}, nil\n}\n" }, { "path": "pkg/backup/actions/csi/volumesnapshotcontent_action_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\t\"testing\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestVSCExecute(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\titem runtime.Unstructured\n\t\tvsc *snapshotv1api.VolumeSnapshotContent\n\t\tbackup *velerov1api.Backup\n\t\texpectErr bool\n\t\texpectedItems []velero.ResourceIdentifier\n\t}{\n\t\t{\n\t\t\tname: \"Invalid VolumeSnapshotContent\",\n\t\t\titem: velerotest.UnstructuredOrDie(\n\t\t\t\t`\n\t\t\t\t{\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"kind\": \"Pod\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns\",\n\t\t\t\t\t\t\"name\": \"foo\"\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\t`,\n\t\t\t),\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").Result(),\n\t\t\texpectErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Normal case, additional items should return\",\n\t\t\tvsc: builder.ForVolumeSnapshotContent(\"test\").ObjectMeta(builder.WithAnnotationsMap(\n\t\t\t\tmap[string]string{\n\t\t\t\t\tvelerov1api.PrefixedSecretNameAnnotation: \"name\",\n\t\t\t\t\tvelerov1api.PrefixedSecretNamespaceAnnotation: \"namespace\",\n\t\t\t\t},\n\t\t\t)).Result(),\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").Result(),\n\t\t\texpectErr: false,\n\t\t\texpectedItems: []velero.ResourceIdentifier{\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.Secrets,\n\t\t\t\t\tNamespace: \"namespace\",\n\t\t\t\t\tName: \"name\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tp, err := NewVolumeSnapshotContentBackupItemAction(logrus.StandardLogger())\n\t\t\trequire.NoError(t, err)\n\n\t\t\taction := p.(*volumeSnapshotContentBackupItemAction)\n\n\t\t\tif test.vsc != nil {\n\t\t\t\tvsMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(test.vsc)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\ttest.item = &unstructured.Unstructured{Object: vsMap}\n\t\t\t}\n\n\t\t\t_, additionalItems, _, _, err := action.Execute(\n\t\t\t\ttest.item,\n\t\t\t\ttest.backup,\n\t\t\t)\n\n\t\t\tif test.expectErr == false {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif len(test.expectedItems) > 0 {\n\t\t\t\trequire.Equal(t, test.expectedItems, additionalItems)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestVSCAppliesTo(t *testing.T) {\n\tp := volumeSnapshotContentBackupItemAction{\n\t\tlog: logrus.StandardLogger(),\n\t}\n\tselector, err := p.AppliesTo()\n\n\trequire.NoError(t, err)\n\n\trequire.Equal(\n\t\tt,\n\t\tvelero.ResourceSelector{\n\t\t\tIncludedResources: []string{\"volumesnapshotcontents.snapshot.storage.k8s.io\"},\n\t\t},\n\t\tselector,\n\t)\n}\n" }, { "path": "pkg/backup/actions/pod_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/actionhelpers\"\n)\n\n// PodAction implements ItemAction.\ntype PodAction struct {\n\tlog logrus.FieldLogger\n}\n\n// NewPodAction creates a new ItemAction for pods.\nfunc NewPodAction(logger logrus.FieldLogger) *PodAction {\n\treturn &PodAction{log: logger}\n}\n\n// AppliesTo returns a ResourceSelector that applies only to pods.\nfunc (a *PodAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"pods\"},\n\t}, nil\n}\n\n// Execute scans the pod's spec.volumes for persistentVolumeClaim volumes and returns a\n// ResourceIdentifier list containing references to all of the persistentVolumeClaim volumes used by\n// the pod. This ensures that when a pod is backed up, all referenced PVCs are backed up too.\nfunc (a *PodAction) Execute(item runtime.Unstructured, backup *v1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, error) {\n\ta.log.Info(\"Executing podAction\")\n\tdefer a.log.Info(\"Done executing podAction\")\n\n\tpod := new(corev1api.Pod)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(item.UnstructuredContent(), pod); err != nil {\n\t\treturn nil, nil, errors.WithStack(err)\n\t}\n\treturn item, actionhelpers.RelatedItemsForPod(pod, a.log), nil\n}\n" }, { "path": "pkg/backup/actions/pod_action_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestPodActionAppliesTo(t *testing.T) {\n\ta := NewPodAction(velerotest.NewLogger())\n\n\tactual, err := a.AppliesTo()\n\trequire.NoError(t, err)\n\n\texpected := velero.ResourceSelector{\n\t\tIncludedResources: []string{\"pods\"},\n\t}\n\tassert.Equal(t, expected, actual)\n}\n\nfunc TestPodActionExecute(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpod runtime.Unstructured\n\t\texpected []velero.ResourceIdentifier\n\t}{\n\t\t{\n\t\t\tname: \"no spec.volumes\",\n\t\t\tpod: velerotest.UnstructuredOrDie(`\n\t\t\t{\n\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\"kind\": \"Pod\",\n\t\t\t\t\"metadata\": {\n\t\t\t\t\t\"namespace\": \"foo\",\n\t\t\t\t\t\"name\": \"bar\"\n\t\t\t\t}\n\t\t\t}\n\t\t\t`),\n\t\t},\n\t\t{\n\t\t\tname: \"persistentVolumeClaim without claimName\",\n\t\t\tpod: velerotest.UnstructuredOrDie(`\n\t\t\t{\n\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\"kind\": \"Pod\",\n\t\t\t\t\"metadata\": {\n\t\t\t\t\t\"namespace\": \"foo\",\n\t\t\t\t\t\"name\": \"bar\"\n\t\t\t\t},\n\t\t\t\t\"spec\": {\n\t\t\t\t\t\"volumes\": [\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\"persistentVolumeClaim\": {}\n\t\t\t\t\t\t}\n\t\t\t\t\t]\n\t\t\t\t}\n\t\t\t}\n\t\t\t`),\n\t\t},\n\t\t{\n\t\t\tname: \"full test, mix of volume types\",\n\t\t\tpod: velerotest.UnstructuredOrDie(`\n\t\t\t{\n\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\"kind\": \"Pod\",\n\t\t\t\t\"metadata\": {\n\t\t\t\t\t\"namespace\": \"foo\",\n\t\t\t\t\t\"name\": \"bar\"\n\t\t\t\t},\n\t\t\t\t\"spec\": {\n\t\t\t\t\t\"volumes\": [\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\"persistentVolumeClaim\": {}\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\"emptyDir\": {}\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\"persistentVolumeClaim\": {\"claimName\": \"claim1\"}\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\"emptyDir\": {}\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\"persistentVolumeClaim\": {\"claimName\": \"claim2\"}\n\t\t\t\t\t\t}\n\t\t\t\t\t]\n\t\t\t\t}\n\t\t\t}\n\t\t\t`),\n\t\t\texpected: []velero.ResourceIdentifier{\n\t\t\t\t{GroupResource: kuberesource.PersistentVolumeClaims, Namespace: \"foo\", Name: \"claim1\"},\n\t\t\t\t{GroupResource: kuberesource.PersistentVolumeClaims, Namespace: \"foo\", Name: \"claim2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"test priority class\",\n\t\t\tpod: velerotest.UnstructuredOrDie(`\n\t\t\t{\n\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\"kind\": \"Pod\",\n\t\t\t\t\"metadata\": {\n\t\t\t\t\t\"namespace\": \"foo\",\n\t\t\t\t\t\"name\": \"bar\"\n\t\t\t\t},\n\t\t\t\t\"spec\": {\n\t\t\t\t\t\"priorityClassName\": \"testPriorityClass\"\n\t\t\t\t}\n\t\t\t}\n\t\t\t`),\n\t\t\texpected: []velero.ResourceIdentifier{\n\t\t\t\t{GroupResource: kuberesource.PriorityClasses, Name: \"testPriorityClass\"},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\ta := NewPodAction(velerotest.NewLogger())\n\n\t\t\tupdated, additionalItems, err := a.Execute(test.pod, nil)\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Equal(t, test.pod, updated)\n\t\t\tassert.Equal(t, test.expected, additionalItems)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/backup/actions/remap_crd_version_action.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tapiextv1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1\"\n\tapiextv1beta1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1\"\n\n\tapiextv1beta1client \"k8s.io/apiextensions-apiserver/pkg/client/clientset/clientset/typed/apiextensions/v1beta1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tvelerodiscovery \"github.com/vmware-tanzu/velero/pkg/discovery\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// RemapCRDVersionAction inspects CustomResourceDefinition and decides if it is a v1\n// CRD that needs to be backed up as v1beta1.\ntype RemapCRDVersionAction struct {\n\tlogger logrus.FieldLogger\n\tbetaCRDClient apiextv1beta1client.CustomResourceDefinitionInterface\n\tdiscoveryHelper velerodiscovery.Helper\n}\n\n// NewRemapCRDVersionAction instantiates a new RemapCRDVersionAction plugin.\nfunc NewRemapCRDVersionAction(logger logrus.FieldLogger, betaCRDClient apiextv1beta1client.CustomResourceDefinitionInterface, discoveryHelper velerodiscovery.Helper) *RemapCRDVersionAction {\n\treturn &RemapCRDVersionAction{logger: logger, betaCRDClient: betaCRDClient, discoveryHelper: discoveryHelper}\n}\n\n// AppliesTo selects the resources the plugin should run against. In this case, CustomResourceDefinitions.\nfunc (a *RemapCRDVersionAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"customresourcedefinition.apiextensions.k8s.io\"},\n\t}, nil\n}\n\n// Execute executes logic necessary to check a CustomResourceDefinition and inspect it for characteristics that necessitate saving it as v1beta1 instead of v1.\nfunc (a *RemapCRDVersionAction) Execute(item runtime.Unstructured, backup *v1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, error) {\n\ta.logger.Info(\"Executing RemapCRDVersionAction\")\n\n\t// This plugin is only relevant for CRDs retrieved from the v1 endpoint that were installed via the v1beta1\n\t// endpoint, so we can exit immediately if the resource in question isn't v1.\n\tapiVersion, ok, err := unstructured.NestedString(item.UnstructuredContent(), \"apiVersion\")\n\tif err != nil {\n\t\treturn nil, nil, errors.Wrap(err, \"unable to read apiVersion from CRD\")\n\t}\n\tif ok && apiVersion != \"apiextensions.k8s.io/v1\" {\n\t\ta.logger.Info(\"Exiting RemapCRDVersionAction, CRD is not v1\")\n\t\treturn item, nil, nil\n\t}\n\n\t// This plugin will exit if the CRD was installed via v1beta1 but the cluster does not support v1beta1 CRD\n\tsupportv1b1 := false\nCheckVersion:\n\tfor _, g := range a.discoveryHelper.APIGroups() {\n\t\tif g.Name == apiextv1.GroupName {\n\t\t\tfor _, v := range g.Versions {\n\t\t\t\tif v.Version == apiextv1beta1.SchemeGroupVersion.Version {\n\t\t\t\t\tsupportv1b1 = true\n\t\t\t\t\tbreak CheckVersion\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\tif !supportv1b1 {\n\t\ta.logger.Info(\"Exiting RemapCRDVersionAction, the cluster does not support v1beta1 CRD\")\n\t\treturn item, nil, nil\n\t}\n\n\t// We've got a v1 CRD and the cluster supports v1beta1 CRD, so proceed.\n\tvar crd apiextv1.CustomResourceDefinition\n\n\t// Do not use runtime.DefaultUnstructuredConverter.FromUnstructured here because it has a bug when converting integers/whole\n\t// numbers in float fields (https://github.com/kubernetes/kubernetes/issues/87675).\n\t// Using JSON as a go-between avoids this issue, without adding a bunch of type conversion by using unstructured helper functions\n\t// to inspect the fields we want to look at.\n\tjs, err := json.Marshal(item.UnstructuredContent())\n\tif err != nil {\n\t\treturn nil, nil, errors.Wrap(err, \"unable to convert unstructured item to JSON\")\n\t}\n\n\tif err = json.Unmarshal(js, &crd); err != nil {\n\t\treturn nil, nil, errors.Wrap(err, \"unable to convert JSON to CRD Go type\")\n\t}\n\n\tlog := a.logger.WithField(\"plugin\", \"RemapCRDVersionAction\").WithField(\"CRD\", crd.Name)\n\n\tswitch {\n\tcase hasSingleVersion(crd), hasNonStructuralSchema(crd), hasPreserveUnknownFields(crd):\n\t\tlog.Infof(\"CustomResourceDefinition %s appears to be v1beta1, fetching the v1beta version\", crd.Name)\n\t\titem, err = fetchV1beta1CRD(crd.Name, a.betaCRDClient)\n\t\tif err != nil {\n\t\t\treturn nil, nil, err\n\t\t}\n\tdefault:\n\t\tlog.Infof(\"CustomResourceDefinition %s does not appear to be v1beta1, backing up as v1\", crd.Name)\n\t}\n\n\treturn item, nil, nil\n}\n\nfunc fetchV1beta1CRD(name string, betaCRDClient apiextv1beta1client.CustomResourceDefinitionInterface) (*unstructured.Unstructured, error) {\n\tbetaCRD, err := betaCRDClient.Get(context.TODO(), name, metav1.GetOptions{})\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error fetching v1beta1 version of %s\", name)\n\t}\n\n\t// Individual items fetched from the API don't always have the kind/API version set\n\t// See https://github.com/kubernetes/kubernetes/issues/3030. Unsure why this is happening here and not in main Velero;\n\t// probably has to do with List calls and Dynamic client vs typed client\n\t// Set these all the time, since they shouldn't ever be different, anyway\n\tbetaCRD.Kind = \"CustomResourceDefinition\"\n\tbetaCRD.APIVersion = apiextv1beta1.SchemeGroupVersion.String()\n\n\tm, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&betaCRD)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error converting v1beta1 version of %s to unstructured\", name)\n\t}\n\titem := &unstructured.Unstructured{Object: m}\n\n\treturn item, nil\n}\n\n// hasPreserveUnknownFields determines whether or not a CRD is set to preserve unknown fields or not.\nfunc hasPreserveUnknownFields(crd apiextv1.CustomResourceDefinition) bool {\n\treturn crd.Spec.PreserveUnknownFields\n}\n\n// hasNonStructuralSchema determines whether or not a CRD has had a nonstructural schema condition applied.\nfunc hasNonStructuralSchema(crd apiextv1.CustomResourceDefinition) bool {\n\tvar ret bool\n\tfor _, c := range crd.Status.Conditions {\n\t\tif c.Type == apiextv1.NonStructuralSchema {\n\t\t\tret = true\n\t\t\tbreak\n\t\t}\n\t}\n\treturn ret\n}\n\n// hasSingleVersion checks a CRD to see if it has a single version with no schema information.\nfunc hasSingleVersion(crd apiextv1.CustomResourceDefinition) bool {\n\t// Looking for 1 version should be enough to tell if it's a v1beta1 CRD, as all v1beta1 CRD versions share the same schema.\n\t// v1 CRDs can have different schemas per version\n\t// The silently upgraded versions will often have a `versions` entry that looks like this:\n\t// versions:\n\t// - name: v1\n\t// served: true\n\t// storage: true\n\t// This is acceptable when re-submitted to a v1beta1 endpoint on restore.\n\tvar ret bool\n\tif len(crd.Spec.Versions) > 0 {\n\t\tif crd.Spec.Versions[0].Schema == nil || crd.Spec.Versions[0].Schema.OpenAPIV3Schema == nil {\n\t\t\tret = true\n\t\t}\n\t}\n\treturn ret\n}\n" }, { "path": "pkg/backup/actions/remap_crd_version_action_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"os\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tapiextv1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1\"\n\tapiextv1beta1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1\"\n\tapiextfakes \"k8s.io/apiextensions-apiserver/pkg/client/clientset/clientset/fake\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tvelerodiscovery \"github.com/vmware-tanzu/velero/pkg/discovery\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestRemapCRDVersionAction(t *testing.T) {\n\tbackup := &v1.Backup{}\n\tclientset := apiextfakes.NewSimpleClientset()\n\tbetaClient := clientset.ApiextensionsV1beta1().CustomResourceDefinitions()\n\n\t// build a v1beta1 CRD with the same name and add it to the fake client that the plugin is going to call.\n\t// keep the same one for all 3 tests, since there's little value in recreating it\n\tb := builder.ForCustomResourceDefinitionV1Beta1(\"test.velero.io\")\n\tc := b.Result()\n\t_, err := betaClient.Create(t.Context(), c, metav1.CreateOptions{})\n\trequire.NoError(t, err)\n\ta := NewRemapCRDVersionAction(velerotest.NewLogger(), betaClient, fakeDiscoveryHelper())\n\n\tt.Run(\"Test a v1 CRD without any Schema information\", func(t *testing.T) {\n\t\tb := builder.ForV1CustomResourceDefinition(\"test.velero.io\")\n\t\t// Set a version that does not include and schema information.\n\t\tb.Version(builder.ForV1CustomResourceDefinitionVersion(\"v1\").Served(true).Storage(true).Result())\n\t\tc := b.Result()\n\n\t\tobj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&c)\n\t\trequire.NoError(t, err)\n\n\t\t// Execute the plugin, which will call the fake client\n\t\titem, _, err := a.Execute(&unstructured.Unstructured{Object: obj}, backup)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, \"apiextensions.k8s.io/v1beta1\", item.UnstructuredContent()[\"apiVersion\"])\n\t})\n\n\tt.Run(\"Test a v1 CRD with a NonStructuralSchema Condition\", func(t *testing.T) {\n\t\tb := builder.ForV1CustomResourceDefinition(\"test.velero.io\")\n\t\tb.Condition(builder.ForV1CustomResourceDefinitionCondition().Type(apiextv1.NonStructuralSchema).Result())\n\t\tc := b.Result()\n\t\tobj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&c)\n\t\trequire.NoError(t, err)\n\n\t\titem, _, err := a.Execute(&unstructured.Unstructured{Object: obj}, backup)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, \"apiextensions.k8s.io/v1beta1\", item.UnstructuredContent()[\"apiVersion\"])\n\t})\n\n\tt.Run(\"Having an integer on a float64 field should work (issue 2319)\", func(t *testing.T) {\n\t\tb := builder.ForV1CustomResourceDefinition(\"test.velero.io\")\n\t\t// 5 here is just an int value, it could be any other whole number.\n\t\tschema := builder.ForJSONSchemaPropsBuilder().Maximum(5).Result()\n\t\tb.Version(builder.ForV1CustomResourceDefinitionVersion(\"v1\").Served(true).Storage(true).Schema(schema).Result())\n\t\tc := b.Result()\n\n\t\t// Marshall in and out of JSON because the problem doesn't manifest when we use ToUnstructured directly\n\t\t// This should simulate the JSON passing over the wire in an HTTP request/response with a dynamic client\n\t\tjs, err := json.Marshal(c)\n\t\trequire.NoError(t, err)\n\n\t\tvar u unstructured.Unstructured\n\t\terr = json.Unmarshal(js, &u)\n\t\trequire.NoError(t, err)\n\n\t\t_, _, err = a.Execute(&u, backup)\n\t\trequire.NoError(t, err)\n\t})\n\n\tt.Run(\"Having Spec.PreserveUnknownFields set to true will return a v1beta1 version of the CRD\", func(t *testing.T) {\n\t\tb := builder.ForV1CustomResourceDefinition(\"test.velero.io\")\n\t\tb.PreserveUnknownFields(true)\n\t\tc := b.Result()\n\t\tobj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&c)\n\t\trequire.NoError(t, err)\n\n\t\titem, _, err := a.Execute(&unstructured.Unstructured{Object: obj}, backup)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, \"apiextensions.k8s.io/v1beta1\", item.UnstructuredContent()[\"apiVersion\"])\n\t})\n\n\tt.Run(\"When the cluster only supports v1 CRD, v1 CRD will be returned even the input has Spec.PreserveUnknownFields set to true (issue 4080)\", func(t *testing.T) {\n\t\ta.discoveryHelper = &velerotest.FakeDiscoveryHelper{\n\t\t\tAPIGroupsList: []metav1.APIGroup{\n\t\t\t\t{\n\t\t\t\t\tName: apiextv1.GroupName,\n\t\t\t\t\tVersions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tVersion: apiextv1.SchemeGroupVersion.Version,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\t\tb := builder.ForV1CustomResourceDefinition(\"test.velero.io\")\n\t\tb.PreserveUnknownFields(true)\n\t\tc := b.Result()\n\t\tobj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&c)\n\t\trequire.NoError(t, err)\n\n\t\titem, _, err := a.Execute(&unstructured.Unstructured{Object: obj}, backup)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, \"apiextensions.k8s.io/v1\", item.UnstructuredContent()[\"apiVersion\"])\n\t\t// set it back to the default one\n\t\ta.discoveryHelper = fakeDiscoveryHelper()\n\t})\n}\n\n// TestRemapCRDVersionActionData tests the RemapCRDVersionAction plugin against actual CRD to confirm that the v1beta1 version is returned when the v1 version is passed in to the plugin.\nfunc TestRemapCRDVersionActionData(t *testing.T) {\n\tbackup := &v1.Backup{}\n\tclientset := apiextfakes.NewSimpleClientset()\n\tbetaClient := clientset.ApiextensionsV1beta1().CustomResourceDefinitions()\n\ta := NewRemapCRDVersionAction(velerotest.NewLogger(), betaClient, fakeDiscoveryHelper())\n\n\ttests := []struct {\n\t\tcrd string\n\t\texpectAdditionalColumns bool\n\t}{\n\t\t{\n\t\t\tcrd: \"elasticsearches.elasticsearch.k8s.elastic.co\",\n\t\t\texpectAdditionalColumns: true,\n\t\t},\n\t\t{\n\t\t\tcrd: \"kibanas.kibana.k8s.elastic.co\",\n\t\t\texpectAdditionalColumns: true,\n\t\t},\n\t\t{\n\t\t\tcrd: \"gcpsamples.gcp.stacks.crossplane.io\",\n\t\t},\n\t\t{\n\t\t\tcrd: \"alertmanagers.monitoring.coreos.com\",\n\t\t},\n\t\t{\n\t\t\tcrd: \"prometheuses.monitoring.coreos.com\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\ttName := fmt.Sprintf(\"%s CRD passed in as v1 should be returned as v1beta1\", test.crd)\n\t\tt.Run(tName, func(t *testing.T) {\n\t\t\t// We don't need a Go struct of the v1 data, just an unstructured to pass into the plugin.\n\t\t\tv1File := fmt.Sprintf(\"testdata/v1/%s.json\", test.crd)\n\t\t\tf, err := os.ReadFile(v1File)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tvar obj unstructured.Unstructured\n\t\t\terr = json.Unmarshal(f, &obj)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Load a v1beta1 struct into the beta client to be returned\n\t\t\tv1beta1File := fmt.Sprintf(\"testdata/v1beta1/%s.json\", test.crd)\n\t\t\tf, err = os.ReadFile(v1beta1File)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tvar crd apiextv1beta1.CustomResourceDefinition\n\t\t\terr = json.Unmarshal(f, &crd)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t_, err = betaClient.Create(t.Context(), &crd, metav1.CreateOptions{})\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Run method under test\n\t\t\titem, _, err := a.Execute(&obj, backup)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.Equal(t, \"apiextensions.k8s.io/v1beta1\", item.UnstructuredContent()[\"apiVersion\"])\n\t\t\tassert.Equal(t, crd.Kind, item.GetObjectKind().GroupVersionKind().GroupKind().Kind)\n\t\t\tname, _, err := unstructured.NestedString(item.UnstructuredContent(), \"metadata\", \"name\")\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Equal(t, crd.Name, name)\n\t\t\tuid, _, err := unstructured.NestedString(item.UnstructuredContent(), \"metadata\", \"uid\")\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Equal(t, string(crd.UID), uid)\n\n\t\t\t// For ElasticSearch and Kibana, problems manifested when additionalPrinterColumns was moved from the top-level spec down to the\n\t\t\t// versions slice.\n\t\t\tif test.expectAdditionalColumns {\n\t\t\t\t_, ok := item.UnstructuredContent()[\"spec\"].(map[string]any)[\"additionalPrinterColumns\"]\n\t\t\t\tassert.True(t, ok)\n\t\t\t}\n\n\t\t\t// Clean up the item created in the test.\n\t\t\tbetaClient.Delete(t.Context(), crd.Name, metav1.DeleteOptions{})\n\t\t})\n\t}\n}\n\nfunc fakeDiscoveryHelper() velerodiscovery.Helper {\n\treturn &velerotest.FakeDiscoveryHelper{\n\t\tAPIGroupsList: []metav1.APIGroup{\n\t\t\t{\n\t\t\t\tName: apiextv1.GroupName,\n\t\t\t\tVersions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t\t{\n\t\t\t\t\t\tVersion: apiextv1beta1.SchemeGroupVersion.Version,\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tVersion: apiextv1.SchemeGroupVersion.Version,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n}\n" }, { "path": "pkg/backup/actions/service_account_action.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/api/meta\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerodiscovery \"github.com/vmware-tanzu/velero/pkg/discovery\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/actionhelpers\"\n)\n\n// ServiceAccountAction implements ItemAction.\ntype ServiceAccountAction struct {\n\tlog logrus.FieldLogger\n\tclusterRoleBindings []actionhelpers.ClusterRoleBinding\n}\n\n// NewServiceAccountAction creates a new ItemAction for service accounts.\nfunc NewServiceAccountAction(logger logrus.FieldLogger, clusterRoleBindingListers map[string]actionhelpers.ClusterRoleBindingLister, discoveryHelper velerodiscovery.Helper) (*ServiceAccountAction, error) {\n\tcrbs, err := actionhelpers.ClusterRoleBindingsForAction(clusterRoleBindingListers, discoveryHelper)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &ServiceAccountAction{\n\t\tlog: logger,\n\t\tclusterRoleBindings: crbs,\n\t}, nil\n}\n\n// AppliesTo returns a ResourceSelector that applies only to service accounts.\nfunc (a *ServiceAccountAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"serviceaccounts\"},\n\t}, nil\n}\n\n// Execute checks for any ClusterRoleBindings that have this service account as a subject, and\n// adds the ClusterRoleBinding and associated ClusterRole to the list of additional items to\n// be backed up.\nfunc (a *ServiceAccountAction) Execute(item runtime.Unstructured, backup *v1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, error) {\n\ta.log.Info(\"Running ServiceAccountAction\")\n\tdefer a.log.Info(\"Done running ServiceAccountAction\")\n\n\tobjectMeta, err := meta.Accessor(item)\n\tif err != nil {\n\t\treturn nil, nil, errors.WithStack(err)\n\t}\n\n\treturn item, actionhelpers.RelatedItemsForServiceAccount(objectMeta, a.clusterRoleBindings, a.log), nil\n}\n" }, { "path": "pkg/backup/actions/service_account_action_test.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"fmt\"\n\t\"sort\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\trbacv1 \"k8s.io/api/rbac/v1\"\n\trbacbeta \"k8s.io/api/rbac/v1beta1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/actionhelpers\"\n)\n\nfunc newV1ClusterRoleBindingList(rbacCRBList []rbacv1.ClusterRoleBinding) []actionhelpers.ClusterRoleBinding {\n\tvar crbs []actionhelpers.ClusterRoleBinding\n\tfor _, c := range rbacCRBList {\n\t\tcrbs = append(crbs, actionhelpers.V1ClusterRoleBinding{Crb: c})\n\t}\n\n\treturn crbs\n}\n\nfunc newV1beta1ClusterRoleBindingList(rbacCRBList []rbacbeta.ClusterRoleBinding) []actionhelpers.ClusterRoleBinding {\n\tvar crbs []actionhelpers.ClusterRoleBinding\n\tfor _, c := range rbacCRBList {\n\t\tcrbs = append(crbs, actionhelpers.V1beta1ClusterRoleBinding{Crb: c})\n\t}\n\n\treturn crbs\n}\n\ntype FakeV1ClusterRoleBindingLister struct {\n\tv1crbs []rbacv1.ClusterRoleBinding\n}\n\nfunc (f FakeV1ClusterRoleBindingLister) List() ([]actionhelpers.ClusterRoleBinding, error) {\n\tvar crbs []actionhelpers.ClusterRoleBinding\n\tfor _, c := range f.v1crbs {\n\t\tcrbs = append(crbs, actionhelpers.V1ClusterRoleBinding{Crb: c})\n\t}\n\treturn crbs, nil\n}\n\ntype FakeV1beta1ClusterRoleBindingLister struct {\n\tv1beta1crbs []rbacbeta.ClusterRoleBinding\n}\n\nfunc (f FakeV1beta1ClusterRoleBindingLister) List() ([]actionhelpers.ClusterRoleBinding, error) {\n\tvar crbs []actionhelpers.ClusterRoleBinding\n\tfor _, c := range f.v1beta1crbs {\n\t\tcrbs = append(crbs, actionhelpers.V1beta1ClusterRoleBinding{Crb: c})\n\t}\n\treturn crbs, nil\n}\n\nfunc TestServiceAccountActionAppliesTo(t *testing.T) {\n\t// Instantiating the struct directly since using\n\t// NewServiceAccountAction requires a full Kubernetes clientset\n\ta := &ServiceAccountAction{}\n\n\tactual, err := a.AppliesTo()\n\trequire.NoError(t, err)\n\n\texpected := velero.ResourceSelector{\n\t\tIncludedResources: []string{\"serviceaccounts\"},\n\t}\n\tassert.Equal(t, expected, actual)\n}\n\nfunc TestNewServiceAccountAction(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tversion string\n\t\texpectedCRBs []actionhelpers.ClusterRoleBinding\n\t}{\n\t\t{\n\t\t\tname: \"rbac v1 API instantiates an saAction\",\n\t\t\tversion: rbacv1.SchemeGroupVersion.Version,\n\t\t\texpectedCRBs: []actionhelpers.ClusterRoleBinding{\n\t\t\t\tactionhelpers.V1ClusterRoleBinding{\n\t\t\t\t\tCrb: rbacv1.ClusterRoleBinding{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"v1crb-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tactionhelpers.V1ClusterRoleBinding{\n\t\t\t\t\tCrb: rbacv1.ClusterRoleBinding{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"v1crb-2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"rbac v1beta1 API instantiates an saAction\",\n\t\t\tversion: rbacbeta.SchemeGroupVersion.Version,\n\t\t\texpectedCRBs: []actionhelpers.ClusterRoleBinding{\n\t\t\t\tactionhelpers.V1beta1ClusterRoleBinding{\n\t\t\t\t\tCrb: rbacbeta.ClusterRoleBinding{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"v1beta1crb-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tactionhelpers.V1beta1ClusterRoleBinding{\n\t\t\t\t\tCrb: rbacbeta.ClusterRoleBinding{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"v1beta1crb-2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"no RBAC API instantiates an saAction with empty slice\",\n\t\t\tversion: \"\",\n\t\t\texpectedCRBs: []actionhelpers.ClusterRoleBinding{},\n\t\t},\n\t}\n\t// Set up all of our fakes outside the test loop\n\tdiscoveryHelper := velerotest.FakeDiscoveryHelper{}\n\tlogger := velerotest.NewLogger()\n\n\tv1crbs := []rbacv1.ClusterRoleBinding{\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"v1crb-1\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"v1crb-2\",\n\t\t\t},\n\t\t},\n\t}\n\n\tv1beta1crbs := []rbacbeta.ClusterRoleBinding{\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"v1beta1crb-1\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"v1beta1crb-2\",\n\t\t\t},\n\t\t},\n\t}\n\n\tclusterRoleBindingListers := map[string]actionhelpers.ClusterRoleBindingLister{\n\t\trbacv1.SchemeGroupVersion.Version: FakeV1ClusterRoleBindingLister{v1crbs: v1crbs},\n\t\trbacbeta.SchemeGroupVersion.Version: FakeV1beta1ClusterRoleBindingLister{v1beta1crbs: v1beta1crbs},\n\t\t\"\": actionhelpers.NoopClusterRoleBindingLister{},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\t// We only care about the preferred version, nothing else in the list\n\t\t\tdiscoveryHelper.APIGroupsList = []metav1.APIGroup{\n\t\t\t\t{\n\t\t\t\t\tName: rbacv1.GroupName,\n\t\t\t\t\tPreferredVersion: metav1.GroupVersionForDiscovery{\n\t\t\t\t\t\tVersion: test.version,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t}\n\t\t\taction, err := NewServiceAccountAction(logger, clusterRoleBindingListers, &discoveryHelper)\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Equal(t, test.expectedCRBs, action.clusterRoleBindings)\n\t\t})\n\t}\n}\n\nfunc TestServiceAccountActionExecute(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tserviceAccount runtime.Unstructured\n\t\tcrbs []rbacv1.ClusterRoleBinding\n\t\texpectedAdditionalItems []velero.ResourceIdentifier\n\t}{\n\t\t{\n\t\t\tname: \"no crbs\",\n\t\t\tserviceAccount: velerotest.UnstructuredOrDie(`\n\t\t\t{\n\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\"metadata\": {\n\t\t\t\t\t\"namespace\": \"velero\",\n\t\t\t\t\t\"name\": \"velero\"\n\t\t\t\t}\n\t\t\t}\n\t\t\t`),\n\t\t\tcrbs: nil,\n\t\t\texpectedAdditionalItems: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"no matching crbs\",\n\t\t\tserviceAccount: velerotest.UnstructuredOrDie(`\n\t\t\t{\n\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\"metadata\": {\n\t\t\t\t\t\"namespace\": \"velero\",\n\t\t\t\t\t\"name\": \"velero\"\n\t\t\t\t}\n\t\t\t}\n\t\t\t`),\n\t\t\tcrbs: []rbacv1.ClusterRoleBinding{\n\t\t\t\t{\n\t\t\t\t\tSubjects: []rbacv1.Subject{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: \"non-matching-kind\",\n\t\t\t\t\t\t\tNamespace: \"non-matching-ns\",\n\t\t\t\t\t\t\tName: \"non-matching-name\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: \"non-matching-kind\",\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: rbacv1.ServiceAccountKind,\n\t\t\t\t\t\t\tNamespace: \"non-matching-ns\",\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: rbacv1.ServiceAccountKind,\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"non-matching-name\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tRoleRef: rbacv1.RoleRef{\n\t\t\t\t\t\tName: \"role\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedAdditionalItems: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"some matching crbs\",\n\t\t\tserviceAccount: velerotest.UnstructuredOrDie(`\n\t\t\t{\n\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\"metadata\": {\n\t\t\t\t\t\"namespace\": \"velero\",\n\t\t\t\t\t\"name\": \"velero\"\n\t\t\t\t}\n\t\t\t}\n\t\t\t`),\n\t\t\tcrbs: []rbacv1.ClusterRoleBinding{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"crb-1\",\n\t\t\t\t\t},\n\t\t\t\t\tSubjects: []rbacv1.Subject{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: \"non-matching-kind\",\n\t\t\t\t\t\t\tNamespace: \"non-matching-ns\",\n\t\t\t\t\t\t\tName: \"non-matching-name\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tRoleRef: rbacv1.RoleRef{\n\t\t\t\t\t\tName: \"role-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"crb-2\",\n\t\t\t\t\t},\n\t\t\t\t\tSubjects: []rbacv1.Subject{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: \"non-matching-kind\",\n\t\t\t\t\t\t\tNamespace: \"non-matching-ns\",\n\t\t\t\t\t\t\tName: \"non-matching-name\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: rbacv1.ServiceAccountKind,\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tRoleRef: rbacv1.RoleRef{\n\t\t\t\t\t\tName: \"role-2\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"crb-3\",\n\t\t\t\t\t},\n\t\t\t\t\tSubjects: []rbacv1.Subject{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: rbacv1.ServiceAccountKind,\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tRoleRef: rbacv1.RoleRef{\n\t\t\t\t\t\tName: \"role-3\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"crb-4\",\n\t\t\t\t\t},\n\t\t\t\t\tSubjects: []rbacv1.Subject{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: rbacv1.ServiceAccountKind,\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: \"non-matching-kind\",\n\t\t\t\t\t\t\tNamespace: \"non-matching-ns\",\n\t\t\t\t\t\t\tName: \"non-matching-name\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tRoleRef: rbacv1.RoleRef{\n\t\t\t\t\t\tName: \"role-4\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedAdditionalItems: []velero.ResourceIdentifier{\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoleBindings,\n\t\t\t\t\tName: \"crb-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoleBindings,\n\t\t\t\t\tName: \"crb-3\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoleBindings,\n\t\t\t\t\tName: \"crb-4\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoles,\n\t\t\t\t\tName: \"role-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoles,\n\t\t\t\t\tName: \"role-3\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoles,\n\t\t\t\t\tName: \"role-4\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\t// Create the action struct directly so we don't need to mock a clientset\n\t\t\taction := &ServiceAccountAction{\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t\tclusterRoleBindings: newV1ClusterRoleBindingList(test.crbs),\n\t\t\t}\n\n\t\t\tres, additional, err := action.Execute(test.serviceAccount, nil)\n\n\t\t\tassert.Equal(t, test.serviceAccount, res)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// ensure slices are ordered for valid comparison\n\t\t\tsort.Slice(test.expectedAdditionalItems, func(i, j int) bool {\n\t\t\t\treturn fmt.Sprintf(\"%s.%s\", test.expectedAdditionalItems[i].GroupResource.String(), test.expectedAdditionalItems[i].Name) <\n\t\t\t\t\tfmt.Sprintf(\"%s.%s\", test.expectedAdditionalItems[j].GroupResource.String(), test.expectedAdditionalItems[j].Name)\n\t\t\t})\n\n\t\t\tsort.Slice(additional, func(i, j int) bool {\n\t\t\t\treturn fmt.Sprintf(\"%s.%s\", additional[i].GroupResource.String(), additional[i].Name) <\n\t\t\t\t\tfmt.Sprintf(\"%s.%s\", additional[j].GroupResource.String(), additional[j].Name)\n\t\t\t})\n\n\t\t\tassert.Equal(t, test.expectedAdditionalItems, additional)\n\t\t})\n\t}\n}\n\nfunc TestServiceAccountActionExecuteOnBeta1(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tserviceAccount runtime.Unstructured\n\t\tcrbs []rbacbeta.ClusterRoleBinding\n\t\texpectedAdditionalItems []velero.ResourceIdentifier\n\t}{\n\t\t{\n\t\t\tname: \"no crbs\",\n\t\t\tserviceAccount: velerotest.UnstructuredOrDie(`\n\t\t\t{\n\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\"metadata\": {\n\t\t\t\t\t\"namespace\": \"velero\",\n\t\t\t\t\t\"name\": \"velero\"\n\t\t\t\t}\n\t\t\t}\n\t\t\t`),\n\t\t\tcrbs: nil,\n\t\t\texpectedAdditionalItems: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"no matching crbs\",\n\t\t\tserviceAccount: velerotest.UnstructuredOrDie(`\n\t\t\t{\n\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\"metadata\": {\n\t\t\t\t\t\"namespace\": \"velero\",\n\t\t\t\t\t\"name\": \"velero\"\n\t\t\t\t}\n\t\t\t}\n\t\t\t`),\n\t\t\tcrbs: []rbacbeta.ClusterRoleBinding{\n\t\t\t\t{\n\t\t\t\t\tSubjects: []rbacbeta.Subject{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: \"non-matching-kind\",\n\t\t\t\t\t\t\tNamespace: \"non-matching-ns\",\n\t\t\t\t\t\t\tName: \"non-matching-name\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: \"non-matching-kind\",\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: rbacbeta.ServiceAccountKind,\n\t\t\t\t\t\t\tNamespace: \"non-matching-ns\",\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: rbacbeta.ServiceAccountKind,\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"non-matching-name\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tRoleRef: rbacbeta.RoleRef{\n\t\t\t\t\t\tName: \"role\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedAdditionalItems: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"some matching crbs\",\n\t\t\tserviceAccount: velerotest.UnstructuredOrDie(`\n\t\t\t{\n\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\"metadata\": {\n\t\t\t\t\t\"namespace\": \"velero\",\n\t\t\t\t\t\"name\": \"velero\"\n\t\t\t\t}\n\t\t\t}\n\t\t\t`),\n\t\t\tcrbs: []rbacbeta.ClusterRoleBinding{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"crb-1\",\n\t\t\t\t\t},\n\t\t\t\t\tSubjects: []rbacbeta.Subject{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: \"non-matching-kind\",\n\t\t\t\t\t\t\tNamespace: \"non-matching-ns\",\n\t\t\t\t\t\t\tName: \"non-matching-name\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tRoleRef: rbacbeta.RoleRef{\n\t\t\t\t\t\tName: \"role-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"crb-2\",\n\t\t\t\t\t},\n\t\t\t\t\tSubjects: []rbacbeta.Subject{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: \"non-matching-kind\",\n\t\t\t\t\t\t\tNamespace: \"non-matching-ns\",\n\t\t\t\t\t\t\tName: \"non-matching-name\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: rbacbeta.ServiceAccountKind,\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tRoleRef: rbacbeta.RoleRef{\n\t\t\t\t\t\tName: \"role-2\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"crb-3\",\n\t\t\t\t\t},\n\t\t\t\t\tSubjects: []rbacbeta.Subject{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: rbacbeta.ServiceAccountKind,\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tRoleRef: rbacbeta.RoleRef{\n\t\t\t\t\t\tName: \"role-3\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"crb-4\",\n\t\t\t\t\t},\n\t\t\t\t\tSubjects: []rbacbeta.Subject{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: rbacbeta.ServiceAccountKind,\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: \"non-matching-kind\",\n\t\t\t\t\t\t\tNamespace: \"non-matching-ns\",\n\t\t\t\t\t\t\tName: \"non-matching-name\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tRoleRef: rbacbeta.RoleRef{\n\t\t\t\t\t\tName: \"role-4\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedAdditionalItems: []velero.ResourceIdentifier{\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoleBindings,\n\t\t\t\t\tName: \"crb-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoleBindings,\n\t\t\t\t\tName: \"crb-3\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoleBindings,\n\t\t\t\t\tName: \"crb-4\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoles,\n\t\t\t\t\tName: \"role-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoles,\n\t\t\t\t\tName: \"role-3\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoles,\n\t\t\t\t\tName: \"role-4\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\t// Create the action struct directly so we don't need to mock a clientset\n\t\t\taction := &ServiceAccountAction{\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t\tclusterRoleBindings: newV1beta1ClusterRoleBindingList(test.crbs),\n\t\t\t}\n\n\t\t\tres, additional, err := action.Execute(test.serviceAccount, nil)\n\n\t\t\tassert.Equal(t, test.serviceAccount, res)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// ensure slices are ordered for valid comparison\n\t\t\tsort.Slice(test.expectedAdditionalItems, func(i, j int) bool {\n\t\t\t\treturn fmt.Sprintf(\"%s.%s\", test.expectedAdditionalItems[i].GroupResource.String(), test.expectedAdditionalItems[i].Name) <\n\t\t\t\t\tfmt.Sprintf(\"%s.%s\", test.expectedAdditionalItems[j].GroupResource.String(), test.expectedAdditionalItems[j].Name)\n\t\t\t})\n\n\t\t\tsort.Slice(additional, func(i, j int) bool {\n\t\t\t\treturn fmt.Sprintf(\"%s.%s\", additional[i].GroupResource.String(), additional[i].Name) <\n\t\t\t\t\tfmt.Sprintf(\"%s.%s\", additional[j].GroupResource.String(), additional[j].Name)\n\t\t\t})\n\n\t\t\tassert.Equal(t, test.expectedAdditionalItems, additional)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/backup/actions/testdata/v1/alertmanagers.monitoring.coreos.com.json", "content": "{\n \"apiVersion\": \"apiextensions.k8s.io/v1\",\n \"kind\": \"CustomResourceDefinition\",\n \"metadata\": {\n \"annotations\": {\n \"helm.sh/hook\": \"crd-install\",\n \"kubectl.kubernetes.io/last-applied-configuration\": \"{\\\"apiVersion\\\":\\\"apiextensions.k8s.io/v1beta1\\\",\\\"kind\\\":\\\"CustomResourceDefinition\\\",\\\"metadata\\\":{\\\"annotations\\\":{\\\"helm.sh/hook\\\":\\\"crd-install\\\"},\\\"creationTimestamp\\\":null,\\\"labels\\\":{\\\"app\\\":\\\"prometheus-operator\\\"},\\\"name\\\":\\\"alertmanagers.monitoring.coreos.com\\\"},\\\"spec\\\":{\\\"group\\\":\\\"monitoring.coreos.com\\\",\\\"names\\\":{\\\"kind\\\":\\\"Alertmanager\\\",\\\"plural\\\":\\\"alertmanagers\\\"},\\\"scope\\\":\\\"Namespaced\\\",\\\"validation\\\":{\\\"openAPIV3Schema\\\":{\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"spec\\\":{\\\"description\\\":\\\"AlertmanagerSpec is a specification of the desired behavior of the Alertmanager cluster. More info: https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#spec-and-status\\\",\\\"properties\\\":{\\\"additionalPeers\\\":{\\\"description\\\":\\\"AdditionalPeers allows injecting a set of additional Alertmanagers to peer with to form a highly available cluster.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"affinity\\\":{\\\"description\\\":\\\"Affinity is a group of affinity scheduling rules.\\\",\\\"properties\\\":{\\\"nodeAffinity\\\":{\\\"description\\\":\\\"Node affinity is a group of node affinity scheduling rules.\\\",\\\"properties\\\":{\\\"preferredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\\\\\"weight\\\\\\\" to the sum if the node matches the corresponding matchExpressions; the node(s) with the highest sum are the most preferred.\\\",\\\"items\\\":{\\\"description\\\":\\\"An empty preferred scheduling term matches all objects with implicit weight 0 (i.e. it's a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op).\\\",\\\"properties\\\":{\\\"preference\\\":{\\\"description\\\":\\\"A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"A list of node selector requirements by node's labels.\\\",\\\"items\\\":{\\\"description\\\":\\\"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"]},\\\"type\\\":\\\"array\\\"},\\\"matchFields\\\":{\\\"description\\\":\\\"A list of node selector requirements by node's fields.\\\",\\\"items\\\":{\\\"description\\\":\\\"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"]},\\\"type\\\":\\\"array\\\"}}},\\\"weight\\\":{\\\"description\\\":\\\"Weight associated with matching the corresponding nodeSelectorTerm, in the range 1-100.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"required\\\":[\\\"weight\\\",\\\"preference\\\"]},\\\"type\\\":\\\"array\\\"},\\\"requiredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"A node selector represents the union of the results of one or more label queries over a set of nodes; that is, it represents the OR of the selectors represented by the node selector terms.\\\",\\\"properties\\\":{\\\"nodeSelectorTerms\\\":{\\\"description\\\":\\\"Required. A list of node selector terms. The terms are ORed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"A list of node selector requirements by node's labels.\\\",\\\"items\\\":{\\\"description\\\":\\\"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"]},\\\"type\\\":\\\"array\\\"},\\\"matchFields\\\":{\\\"description\\\":\\\"A list of node selector requirements by node's fields.\\\",\\\"items\\\":{\\\"description\\\":\\\"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"]},\\\"type\\\":\\\"array\\\"}}},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"nodeSelectorTerms\\\"]}}},\\\"podAffinity\\\":{\\\"description\\\":\\\"Pod affinity is a group of inter pod affinity scheduling rules.\\\",\\\"properties\\\":{\\\"preferredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\\\\\"weight\\\\\\\" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.\\\",\\\"items\\\":{\\\"description\\\":\\\"The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)\\\",\\\"properties\\\":{\\\"podAffinityTerm\\\":{\\\"description\\\":\\\"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\\\u003ctopologyKey\\\\u003e matches that of any node on which a pod of the set of pods is running\\\",\\\"properties\\\":{\\\"labelSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"]},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}}},\\\"namespaces\\\":{\\\"description\\\":\\\"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\\\\\"this pod's namespace\\\\\\\"\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"topologyKey\\\":{\\\"description\\\":\\\"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"topologyKey\\\"]},\\\"weight\\\":{\\\"description\\\":\\\"weight associated with matching the corresponding podAffinityTerm, in the range 1-100.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"required\\\":[\\\"weight\\\",\\\"podAffinityTerm\\\"]},\\\"type\\\":\\\"array\\\"},\\\"requiredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.\\\",\\\"items\\\":{\\\"description\\\":\\\"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\\\u003ctopologyKey\\\\u003e matches that of any node on which a pod of the set of pods is running\\\",\\\"properties\\\":{\\\"labelSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"]},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}}},\\\"namespaces\\\":{\\\"description\\\":\\\"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\\\\\"this pod's namespace\\\\\\\"\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"topologyKey\\\":{\\\"description\\\":\\\"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"topologyKey\\\"]},\\\"type\\\":\\\"array\\\"}}},\\\"podAntiAffinity\\\":{\\\"description\\\":\\\"Pod anti affinity is a group of inter pod anti affinity scheduling rules.\\\",\\\"properties\\\":{\\\"preferredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"The scheduler will prefer to schedule pods to nodes that satisfy the anti-affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling anti-affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\\\\\"weight\\\\\\\" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.\\\",\\\"items\\\":{\\\"description\\\":\\\"The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)\\\",\\\"properties\\\":{\\\"podAffinityTerm\\\":{\\\"description\\\":\\\"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\\\u003ctopologyKey\\\\u003e matches that of any node on which a pod of the set of pods is running\\\",\\\"properties\\\":{\\\"labelSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"]},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}}},\\\"namespaces\\\":{\\\"description\\\":\\\"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\\\\\"this pod's namespace\\\\\\\"\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"topologyKey\\\":{\\\"description\\\":\\\"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"topologyKey\\\"]},\\\"weight\\\":{\\\"description\\\":\\\"weight associated with matching the corresponding podAffinityTerm, in the range 1-100.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"required\\\":[\\\"weight\\\",\\\"podAffinityTerm\\\"]},\\\"type\\\":\\\"array\\\"},\\\"requiredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"If the anti-affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the anti-affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.\\\",\\\"items\\\":{\\\"description\\\":\\\"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\\\u003ctopologyKey\\\\u003e matches that of any node on which a pod of the set of pods is running\\\",\\\"properties\\\":{\\\"labelSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"]},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}}},\\\"namespaces\\\":{\\\"description\\\":\\\"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\\\\\"this pod's namespace\\\\\\\"\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"topologyKey\\\":{\\\"description\\\":\\\"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"topologyKey\\\"]},\\\"type\\\":\\\"array\\\"}}}}},\\\"baseImage\\\":{\\\"description\\\":\\\"Base image that is used to deploy pods, without tag.\\\",\\\"type\\\":\\\"string\\\"},\\\"configMaps\\\":{\\\"description\\\":\\\"ConfigMaps is a list of ConfigMaps in the same namespace as the Alertmanager object, which shall be mounted into the Alertmanager Pods. The ConfigMaps are mounted into /etc/alertmanager/configmaps/\\\\u003cconfigmap-name\\\\u003e.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"configSecret\\\":{\\\"description\\\":\\\"ConfigSecret is the name of a Kubernetes Secret in the same namespace as the Alertmanager object, which contains configuration for this Alertmanager instance. Defaults to 'alertmanager-' The secret is mounted into /etc/alertmanager/config.\\\",\\\"type\\\":\\\"string\\\"},\\\"containers\\\":{\\\"description\\\":\\\"Containers allows injecting additional containers. This is meant to allow adding an authentication proxy to an Alertmanager pod.\\\",\\\"items\\\":{\\\"description\\\":\\\"A single application container that you want to run within a pod.\\\",\\\"properties\\\":{\\\"args\\\":{\\\"description\\\":\\\"Arguments to the entrypoint. The docker image's CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"command\\\":{\\\"description\\\":\\\"Entrypoint array. Not executed within a shell. The docker image's ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"env\\\":{\\\"description\\\":\\\"List of environment variables to set in the container. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"EnvVar represents an environment variable present in a Container.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the environment variable. Must be a C_IDENTIFIER.\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to \\\\\\\"\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"},\\\"valueFrom\\\":{\\\"description\\\":\\\"EnvVarSource represents a source for the value of an EnvVar.\\\",\\\"properties\\\":{\\\"configMapKeyRef\\\":{\\\"description\\\":\\\"Selects a key from a ConfigMap.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key to select.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"]},\\\"fieldRef\\\":{\\\"description\\\":\\\"ObjectFieldSelector selects an APIVersioned field of an object.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"Version of the schema the FieldPath is written in terms of, defaults to \\\\\\\"v1\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"},\\\"fieldPath\\\":{\\\"description\\\":\\\"Path of the field to select in the specified API version.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"fieldPath\\\"]},\\\"resourceFieldRef\\\":{\\\"description\\\":\\\"ResourceFieldSelector represents container resources (cpu, memory) and their output format\\\",\\\"properties\\\":{\\\"containerName\\\":{\\\"description\\\":\\\"Container name: required for volumes, optional for env vars\\\",\\\"type\\\":\\\"string\\\"},\\\"divisor\\\":{},\\\"resource\\\":{\\\"description\\\":\\\"Required: resource to select\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"resource\\\"]},\\\"secretKeyRef\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"]}}}},\\\"required\\\":[\\\"name\\\"]},\\\"type\\\":\\\"array\\\"},\\\"envFrom\\\":{\\\"description\\\":\\\"List of sources to populate environment variables in the container. The keys defined within a source must be a C_IDENTIFIER. All invalid keys will be reported as an event when the container is starting. When a key exists in multiple sources, the value associated with the last source will take precedence. Values defined by an Env with a duplicate key will take precedence. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"EnvFromSource represents the source of a set of ConfigMaps\\\",\\\"properties\\\":{\\\"configMapRef\\\":{\\\"description\\\":\\\"ConfigMapEnvSource selects a ConfigMap to populate the environment variables with.\\\\nThe contents of the target ConfigMap's Data field will represent the key-value pairs as environment variables.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap must be defined\\\",\\\"type\\\":\\\"boolean\\\"}}},\\\"prefix\\\":{\\\"description\\\":\\\"An optional identifier to prepend to each key in the ConfigMap. Must be a C_IDENTIFIER.\\\",\\\"type\\\":\\\"string\\\"},\\\"secretRef\\\":{\\\"description\\\":\\\"SecretEnvSource selects a Secret to populate the environment variables with.\\\\nThe contents of the target Secret's Data field will represent the key-value pairs as environment variables.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret must be defined\\\",\\\"type\\\":\\\"boolean\\\"}}}}},\\\"type\\\":\\\"array\\\"},\\\"image\\\":{\\\"description\\\":\\\"Docker image name. More info: https://kubernetes.io/docs/concepts/containers/images This field is optional to allow higher level config management to default or override container images in workload controllers like Deployments and StatefulSets.\\\",\\\"type\\\":\\\"string\\\"},\\\"imagePullPolicy\\\":{\\\"description\\\":\\\"Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images\\\",\\\"type\\\":\\\"string\\\"},\\\"lifecycle\\\":{\\\"description\\\":\\\"Lifecycle describes actions that the management system should take in response to container lifecycle events. For the PostStart and PreStop lifecycle handlers, management of the container blocks until the action is complete, unless the container process fails, in which case the handler is aborted.\\\",\\\"properties\\\":{\\\"postStart\\\":{\\\"description\\\":\\\"Handler defines a specific action that should be taken\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"ExecAction describes a \\\\\\\"run in container\\\\\\\" action.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}}},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGetAction describes an action based on HTTP Get requests.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"]},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"]},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocketAction describes an action based on opening a socket\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]}},\\\"required\\\":[\\\"port\\\"]}}},\\\"preStop\\\":{\\\"description\\\":\\\"Handler defines a specific action that should be taken\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"ExecAction describes a \\\\\\\"run in container\\\\\\\" action.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}}},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGetAction describes an action based on HTTP Get requests.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"]},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"]},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocketAction describes an action based on opening a socket\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]}},\\\"required\\\":[\\\"port\\\"]}}}}},\\\"livenessProbe\\\":{\\\"description\\\":\\\"Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic.\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"ExecAction describes a \\\\\\\"run in container\\\\\\\" action.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}}},\\\"failureThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGetAction describes an action based on HTTP Get requests.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"]},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"]},\\\"initialDelaySeconds\\\":{\\\"description\\\":\\\"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"periodSeconds\\\":{\\\"description\\\":\\\"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"successThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocketAction describes an action based on opening a socket\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]}},\\\"required\\\":[\\\"port\\\"]},\\\"timeoutSeconds\\\":{\\\"description\\\":\\\"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}}},\\\"name\\\":{\\\"description\\\":\\\"Name of the container specified as a DNS_LABEL. Each container in a pod must have a unique name (DNS_LABEL). Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"},\\\"ports\\\":{\\\"description\\\":\\\"List of ports to expose from the container. Exposing a port here gives the system additional information about the network connections a container uses, but is primarily informational. Not specifying a port here DOES NOT prevent that port from being exposed. Any port which is listening on the default \\\\\\\"0.0.0.0\\\\\\\" address inside a container will be accessible from the network. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"ContainerPort represents a network port in a single container.\\\",\\\"properties\\\":{\\\"containerPort\\\":{\\\"description\\\":\\\"Number of port to expose on the pod's IP address. This must be a valid port number, 0 \\\\u003c x \\\\u003c 65536.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"hostIP\\\":{\\\"description\\\":\\\"What host IP to bind the external port to.\\\",\\\"type\\\":\\\"string\\\"},\\\"hostPort\\\":{\\\"description\\\":\\\"Number of port to expose on the host. If specified, this must be a valid port number, 0 \\\\u003c x \\\\u003c 65536. If HostNetwork is specified, this must match ContainerPort. Most containers do not need this.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"name\\\":{\\\"description\\\":\\\"If specified, this must be an IANA_SVC_NAME and unique within the pod. Each named port in a pod must have a unique name. Name for the port that can be referred to by services.\\\",\\\"type\\\":\\\"string\\\"},\\\"protocol\\\":{\\\"description\\\":\\\"Protocol for port. Must be UDP, TCP, or SCTP. Defaults to \\\\\\\"TCP\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"containerPort\\\"]},\\\"type\\\":\\\"array\\\"},\\\"readinessProbe\\\":{\\\"description\\\":\\\"Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic.\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"ExecAction describes a \\\\\\\"run in container\\\\\\\" action.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}}},\\\"failureThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGetAction describes an action based on HTTP Get requests.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"]},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"]},\\\"initialDelaySeconds\\\":{\\\"description\\\":\\\"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"periodSeconds\\\":{\\\"description\\\":\\\"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"successThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocketAction describes an action based on opening a socket\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]}},\\\"required\\\":[\\\"port\\\"]},\\\"timeoutSeconds\\\":{\\\"description\\\":\\\"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}}},\\\"resources\\\":{\\\"description\\\":\\\"ResourceRequirements describes the compute resource requirements.\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}}},\\\"securityContext\\\":{\\\"description\\\":\\\"SecurityContext holds security configuration that will be applied to a container. Some fields are present in both SecurityContext and PodSecurityContext. When both are set, the values in SecurityContext take precedence.\\\",\\\"properties\\\":{\\\"allowPrivilegeEscalation\\\":{\\\"description\\\":\\\"AllowPrivilegeEscalation controls whether a process can gain more privileges than its parent process. This bool directly controls if the no_new_privs flag will be set on the container process. AllowPrivilegeEscalation is true always when the container is: 1) run as Privileged 2) has CAP_SYS_ADMIN\\\",\\\"type\\\":\\\"boolean\\\"},\\\"capabilities\\\":{\\\"description\\\":\\\"Adds and removes POSIX capabilities from running containers.\\\",\\\"properties\\\":{\\\"add\\\":{\\\"description\\\":\\\"Added capabilities\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"drop\\\":{\\\"description\\\":\\\"Removed capabilities\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}}},\\\"privileged\\\":{\\\"description\\\":\\\"Run container in privileged mode. Processes in privileged containers are essentially equivalent to root on the host. Defaults to false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"procMount\\\":{\\\"description\\\":\\\"procMount denotes the type of proc mount to use for the containers. The default is DefaultProcMount which uses the container runtime defaults for readonly paths and masked paths. This requires the ProcMountType feature flag to be enabled.\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnlyRootFilesystem\\\":{\\\"description\\\":\\\"Whether this container has a read-only root filesystem. Default is false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"runAsGroup\\\":{\\\"description\\\":\\\"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"runAsNonRoot\\\":{\\\"description\\\":\\\"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"runAsUser\\\":{\\\"description\\\":\\\"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"seLinuxOptions\\\":{\\\"description\\\":\\\"SELinuxOptions are the labels to be applied to the container\\\",\\\"properties\\\":{\\\"level\\\":{\\\"description\\\":\\\"Level is SELinux level label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"role\\\":{\\\"description\\\":\\\"Role is a SELinux role label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"type\\\":{\\\"description\\\":\\\"Type is a SELinux type label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"user\\\":{\\\"description\\\":\\\"User is a SELinux user label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"}}}}},\\\"stdin\\\":{\\\"description\\\":\\\"Whether this container should allocate a buffer for stdin in the container runtime. If this is not set, reads from stdin in the container will always result in EOF. Default is false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"stdinOnce\\\":{\\\"description\\\":\\\"Whether the container runtime should close the stdin channel after it has been opened by a single attach. When stdin is true the stdin stream will remain open across multiple attach sessions. If stdinOnce is set to true, stdin is opened on container start, is empty until the first client attaches to stdin, and then remains open and accepts data until the client disconnects, at which time stdin is closed and remains closed until the container is restarted. If this flag is false, a container processes that reads from stdin will never receive an EOF. Default is false\\\",\\\"type\\\":\\\"boolean\\\"},\\\"terminationMessagePath\\\":{\\\"description\\\":\\\"Optional: Path at which the file to which the container's termination message will be written is mounted into the container's filesystem. Message written is intended to be brief final status, such as an assertion failure message. Will be truncated by the node if greater than 4096 bytes. The total message length across all containers will be limited to 12kb. Defaults to /dev/termination-log. Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"},\\\"terminationMessagePolicy\\\":{\\\"description\\\":\\\"Indicate how the termination message should be populated. File will use the contents of terminationMessagePath to populate the container status message on both success and failure. FallbackToLogsOnError will use the last chunk of container log output if the termination message file is empty and the container exited with an error. The log output is limited to 2048 bytes or 80 lines, whichever is smaller. Defaults to File. Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"},\\\"tty\\\":{\\\"description\\\":\\\"Whether this container should allocate a TTY for itself, also requires 'stdin' to be true. Default is false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"volumeDevices\\\":{\\\"description\\\":\\\"volumeDevices is the list of block devices to be used by the container. This is a beta feature.\\\",\\\"items\\\":{\\\"description\\\":\\\"volumeDevice describes a mapping of a raw block device within a container.\\\",\\\"properties\\\":{\\\"devicePath\\\":{\\\"description\\\":\\\"devicePath is the path inside of the container that the device will be mapped to.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"name must match the name of a persistentVolumeClaim in the pod\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"devicePath\\\"]},\\\"type\\\":\\\"array\\\"},\\\"volumeMounts\\\":{\\\"description\\\":\\\"Pod volumes to mount into the container's filesystem. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"VolumeMount describes a mounting of a Volume within a container.\\\",\\\"properties\\\":{\\\"mountPath\\\":{\\\"description\\\":\\\"Path within the container at which the volume should be mounted. Must not contain ':'.\\\",\\\"type\\\":\\\"string\\\"},\\\"mountPropagation\\\":{\\\"description\\\":\\\"mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"This must match the Name of a Volume.\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"subPath\\\":{\\\"description\\\":\\\"Path within the volume from which the container's volume should be mounted. Defaults to \\\\\\\"\\\\\\\" (volume's root).\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"mountPath\\\"]},\\\"type\\\":\\\"array\\\"},\\\"workingDir\\\":{\\\"description\\\":\\\"Container's working directory. If not specified, the container runtime's default will be used, which might be configured in the container image. Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\"]},\\\"type\\\":\\\"array\\\"},\\\"externalUrl\\\":{\\\"description\\\":\\\"The external URL the Alertmanager instances will be available under. This is necessary to generate correct URLs. This is necessary if Alertmanager is not served from root of a DNS name.\\\",\\\"type\\\":\\\"string\\\"},\\\"image\\\":{\\\"description\\\":\\\"Image if specified has precedence over baseImage, tag and sha combinations. Specifying the version is still necessary to ensure the Prometheus Operator knows what version of Alertmanager is being configured.\\\",\\\"type\\\":\\\"string\\\"},\\\"imagePullSecrets\\\":{\\\"description\\\":\\\"An optional list of references to secrets in the same namespace to use for pulling prometheus and alertmanager images from registries see http://kubernetes.io/docs/user-guide/images#specifying-imagepullsecrets-on-a-pod\\\",\\\"items\\\":{\\\"description\\\":\\\"LocalObjectReference contains enough information to let you locate the referenced object inside the same namespace.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"}}},\\\"type\\\":\\\"array\\\"},\\\"listenLocal\\\":{\\\"description\\\":\\\"ListenLocal makes the Alertmanager server listen on loopback, so that it does not bind against the Pod IP. Note this is only for the Alertmanager UI, not the gossip communication.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"logLevel\\\":{\\\"description\\\":\\\"Log level for Alertmanager to be configured with.\\\",\\\"type\\\":\\\"string\\\"},\\\"nodeSelector\\\":{\\\"description\\\":\\\"Define which Nodes the Pods are scheduled on.\\\",\\\"type\\\":\\\"object\\\"},\\\"paused\\\":{\\\"description\\\":\\\"If set to true all actions on the underlying managed objects are not goint to be performed, except for delete actions.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"podMetadata\\\":{\\\"description\\\":\\\"ObjectMeta is metadata that all persisted resources must have, which includes all objects users must create.\\\",\\\"properties\\\":{\\\"annotations\\\":{\\\"description\\\":\\\"Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations\\\",\\\"type\\\":\\\"object\\\"},\\\"clusterName\\\":{\\\"description\\\":\\\"The name of the cluster which the object belongs to. This is used to distinguish resources with same name and namespace in different clusters. This field is not set anywhere right now and apiserver is going to ignore it if set in create or update request.\\\",\\\"type\\\":\\\"string\\\"},\\\"creationTimestamp\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"deletionGracePeriodSeconds\\\":{\\\"description\\\":\\\"Number of seconds allowed for this object to gracefully terminate before it will be removed from the system. Only set when deletionTimestamp is also set. May only be shortened. Read-only.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"deletionTimestamp\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"finalizers\\\":{\\\"description\\\":\\\"Must be empty before the object is deleted from the registry. Each entry is an identifier for the responsible component that will remove the entry from the list. If the deletionTimestamp of the object is non-nil, entries in this list can only be removed.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"generateName\\\":{\\\"description\\\":\\\"GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server.\\\\nIf this field is specified and the generated name exists, the server will NOT return a 409 - instead, it will either return 201 Created or 500 with Reason ServerTimeout indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header).\\\\nApplied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#idempotency\\\",\\\"type\\\":\\\"string\\\"},\\\"generation\\\":{\\\"description\\\":\\\"A sequence number representing a specific generation of the desired state. Populated by the system. Read-only.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"initializers\\\":{\\\"description\\\":\\\"Initializers tracks the progress of initialization.\\\",\\\"properties\\\":{\\\"pending\\\":{\\\"description\\\":\\\"Pending is a list of initializers that must execute in order before this object is visible. When the last pending initializer is removed, and no failing result is set, the initializers struct will be set to nil and the object is considered as initialized and visible to all clients.\\\",\\\"items\\\":{\\\"description\\\":\\\"Initializer is information about an initializer that has not yet completed.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"name of the process that is responsible for initializing this object.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\"]},\\\"type\\\":\\\"array\\\"},\\\"result\\\":{\\\"description\\\":\\\"Status is a return value for calls that don't return other objects.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"code\\\":{\\\"description\\\":\\\"Suggested HTTP return code for this status, 0 if not set.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"details\\\":{\\\"description\\\":\\\"StatusDetails is a set of additional properties that MAY be set by the server to provide additional information about a response. The Reason field of a Status object defines what attributes will be set. Clients must ignore fields that do not match the defined type of each attribute, and should assume that any attribute may be empty, invalid, or under defined.\\\",\\\"properties\\\":{\\\"causes\\\":{\\\"description\\\":\\\"The Causes array includes more details associated with the StatusReason failure. Not all StatusReasons may provide detailed causes.\\\",\\\"items\\\":{\\\"description\\\":\\\"StatusCause provides more information about an api.Status failure, including cases when multiple errors are encountered.\\\",\\\"properties\\\":{\\\"field\\\":{\\\"description\\\":\\\"The field of the resource that has caused this error, as named by its JSON serialization. May include dot and postfix notation for nested attributes. Arrays are zero-indexed. Fields may appear more than once in an array of causes due to fields having multiple errors. Optional.\\\\nExamples:\\\\n \\\\\\\"name\\\\\\\" - the field \\\\\\\"name\\\\\\\" on the current resource\\\\n \\\\\\\"items[0].name\\\\\\\" - the field \\\\\\\"name\\\\\\\" on the first array entry in \\\\\\\"items\\\\\\\"\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"A human-readable description of the cause of the error. This field may be presented as-is to a reader.\\\",\\\"type\\\":\\\"string\\\"},\\\"reason\\\":{\\\"description\\\":\\\"A machine-readable description of the cause of the error. If this value is empty there is no information available.\\\",\\\"type\\\":\\\"string\\\"}}},\\\"type\\\":\\\"array\\\"},\\\"group\\\":{\\\"description\\\":\\\"The group attribute of the resource associated with the status StatusReason.\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"The kind attribute of the resource associated with the status StatusReason. On some operations may differ from the requested resource Kind. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"The name attribute of the resource associated with the status StatusReason (when there is a single name which can be described).\\\",\\\"type\\\":\\\"string\\\"},\\\"retryAfterSeconds\\\":{\\\"description\\\":\\\"If specified, the time in seconds before the operation should be retried. Some errors may indicate the client must take an alternate action - for those errors this field may indicate how long to wait before taking the alternate action.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID of the resource. (when there is a single resource which can be described). More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}}},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"A human-readable description of the status of this operation.\\\",\\\"type\\\":\\\"string\\\"},\\\"metadata\\\":{\\\"description\\\":\\\"ListMeta describes metadata that synthetic resources must have, including lists and various status objects. A resource may have only one of {ObjectMeta, ListMeta}.\\\",\\\"properties\\\":{\\\"continue\\\":{\\\"description\\\":\\\"continue may be set if the user set a limit on the number of items returned, and indicates that the server has more data available. The value is opaque and may be used to issue another request to the endpoint that served this list to retrieve the next set of available objects. Continuing a consistent list may not be possible if the server configuration has changed or more than a few minutes have passed. The resourceVersion field returned when using this continue value will be identical to the value in the first response, unless you have received this token from an error message.\\\",\\\"type\\\":\\\"string\\\"},\\\"resourceVersion\\\":{\\\"description\\\":\\\"String that identifies the server's internal version of this object that can be used by clients to determine when objects have changed. Value must be treated as opaque by clients and passed unmodified back to the server. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\\\",\\\"type\\\":\\\"string\\\"},\\\"selfLink\\\":{\\\"description\\\":\\\"selfLink is a URL representing this object. Populated by the system. Read-only.\\\",\\\"type\\\":\\\"string\\\"}}},\\\"reason\\\":{\\\"description\\\":\\\"A machine-readable description of why this operation is in the \\\\\\\"Failure\\\\\\\" status. If this value is empty there is no information available. A Reason clarifies an HTTP status code but does not override it.\\\",\\\"type\\\":\\\"string\\\"},\\\"status\\\":{\\\"description\\\":\\\"Status of the operation. One of: \\\\\\\"Success\\\\\\\" or \\\\\\\"Failure\\\\\\\". More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status\\\",\\\"type\\\":\\\"string\\\"}}}},\\\"required\\\":[\\\"pending\\\"]},\\\"labels\\\":{\\\"description\\\":\\\"Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels\\\",\\\"type\\\":\\\"object\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names\\\",\\\"type\\\":\\\"string\\\"},\\\"namespace\\\":{\\\"description\\\":\\\"Namespace defines the space within each name must be unique. An empty namespace is equivalent to the \\\\\\\"default\\\\\\\" namespace, but \\\\\\\"default\\\\\\\" is the canonical representation. Not all objects are required to be scoped to a namespace - the value of this field for those objects will be empty.\\\\nMust be a DNS_LABEL. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/namespaces\\\",\\\"type\\\":\\\"string\\\"},\\\"ownerReferences\\\":{\\\"description\\\":\\\"List of objects depended by this object. If ALL objects in the list have been deleted, this object will be garbage collected. If this object is managed by a controller, then an entry in this list will point to this controller, with the controller field set to true. There cannot be more than one managing controller.\\\",\\\"items\\\":{\\\"description\\\":\\\"OwnerReference contains enough information to let you identify an owning object. An owning object must be in the same namespace as the dependent, or be cluster-scoped, so there is no namespace field.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"API version of the referent.\\\",\\\"type\\\":\\\"string\\\"},\\\"blockOwnerDeletion\\\":{\\\"description\\\":\\\"If true, AND if the owner has the \\\\\\\"foregroundDeletion\\\\\\\" finalizer, then the owner cannot be deleted from the key-value store until this reference is removed. Defaults to false. To set this field, a user needs \\\\\\\"delete\\\\\\\" permission of the owner, otherwise 422 (Unprocessable Entity) will be returned.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"controller\\\":{\\\"description\\\":\\\"If true, this reference points to the managing controller.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#names\\\",\\\"type\\\":\\\"string\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"apiVersion\\\",\\\"kind\\\",\\\"name\\\",\\\"uid\\\"]},\\\"type\\\":\\\"array\\\"},\\\"resourceVersion\\\":{\\\"description\\\":\\\"An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources.\\\\nPopulated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\\\",\\\"type\\\":\\\"string\\\"},\\\"selfLink\\\":{\\\"description\\\":\\\"SelfLink is a URL representing this object. Populated by the system. Read-only.\\\",\\\"type\\\":\\\"string\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID is the unique in time and space value for this object. It is typically generated by the server on successful creation of a resource and is not allowed to change on PUT operations.\\\\nPopulated by the system. Read-only. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}}},\\\"priorityClassName\\\":{\\\"description\\\":\\\"Priority class assigned to the Pods\\\",\\\"type\\\":\\\"string\\\"},\\\"replicas\\\":{\\\"description\\\":\\\"Size is the expected size of the alertmanager cluster. The controller will eventually make the size of the running cluster equal to the expected size.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"resources\\\":{\\\"description\\\":\\\"ResourceRequirements describes the compute resource requirements.\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}}},\\\"retention\\\":{\\\"description\\\":\\\"Time duration Alertmanager shall retain data for. Default is '120h', and must match the regular expression `[0-9]+(ms|s|m|h)` (milliseconds seconds minutes hours).\\\",\\\"type\\\":\\\"string\\\"},\\\"routePrefix\\\":{\\\"description\\\":\\\"The route prefix Alertmanager registers HTTP handlers for. This is useful, if using ExternalURL and a proxy is rewriting HTTP routes of a request, and the actual ExternalURL is still true, but the server serves requests under a different route prefix. For example for use with `kubectl proxy`.\\\",\\\"type\\\":\\\"string\\\"},\\\"secrets\\\":{\\\"description\\\":\\\"Secrets is a list of Secrets in the same namespace as the Alertmanager object, which shall be mounted into the Alertmanager Pods. The Secrets are mounted into /etc/alertmanager/secrets/\\\\u003csecret-name\\\\u003e.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"securityContext\\\":{\\\"description\\\":\\\"PodSecurityContext holds pod-level security attributes and common container settings. Some fields are also present in container.securityContext. Field values of container.securityContext take precedence over field values of PodSecurityContext.\\\",\\\"properties\\\":{\\\"fsGroup\\\":{\\\"description\\\":\\\"A special supplemental group that applies to all containers in a pod. Some volume types allow the Kubelet to change the ownership of that volume to be owned by the pod:\\\\n1. The owning GID will be the FSGroup 2. The setgid bit is set (new files created in the volume will be owned by FSGroup) 3. The permission bits are OR'd with rw-rw----\\\\nIf unset, the Kubelet will not modify the ownership and permissions of any volume.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"runAsGroup\\\":{\\\"description\\\":\\\"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"runAsNonRoot\\\":{\\\"description\\\":\\\"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"runAsUser\\\":{\\\"description\\\":\\\"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"seLinuxOptions\\\":{\\\"description\\\":\\\"SELinuxOptions are the labels to be applied to the container\\\",\\\"properties\\\":{\\\"level\\\":{\\\"description\\\":\\\"Level is SELinux level label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"role\\\":{\\\"description\\\":\\\"Role is a SELinux role label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"type\\\":{\\\"description\\\":\\\"Type is a SELinux type label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"user\\\":{\\\"description\\\":\\\"User is a SELinux user label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"}}},\\\"supplementalGroups\\\":{\\\"description\\\":\\\"A list of groups applied to the first process run in each container, in addition to the container's primary GID. If unspecified, no groups will be added to any container.\\\",\\\"items\\\":{\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"type\\\":\\\"array\\\"},\\\"sysctls\\\":{\\\"description\\\":\\\"Sysctls hold a list of namespaced sysctls used for the pod. Pods with unsupported sysctls (by the container runtime) might fail to launch.\\\",\\\"items\\\":{\\\"description\\\":\\\"Sysctl defines a kernel parameter to be set\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of a property to set\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"Value of a property to set\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"]},\\\"type\\\":\\\"array\\\"}}},\\\"serviceAccountName\\\":{\\\"description\\\":\\\"ServiceAccountName is the name of the ServiceAccount to use to run the Prometheus Pods.\\\",\\\"type\\\":\\\"string\\\"},\\\"sha\\\":{\\\"description\\\":\\\"SHA of Alertmanager container image to be deployed. Defaults to the value of `version`. Similar to a tag, but the SHA explicitly deploys an immutable container image. Version and Tag are ignored if SHA is set.\\\",\\\"type\\\":\\\"string\\\"},\\\"storage\\\":{\\\"description\\\":\\\"StorageSpec defines the configured storage for a group Prometheus servers. If neither `emptyDir` nor `volumeClaimTemplate` is specified, then by default an [EmptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir) will be used.\\\",\\\"properties\\\":{\\\"emptyDir\\\":{\\\"description\\\":\\\"Represents an empty directory for a pod. Empty directory volumes support ownership management and SELinux relabeling.\\\",\\\"properties\\\":{\\\"medium\\\":{\\\"description\\\":\\\"What type of storage medium should back this directory. The default is \\\\\\\"\\\\\\\" which means to use the node's default medium. Must be an empty string (default) or Memory. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir\\\",\\\"type\\\":\\\"string\\\"},\\\"sizeLimit\\\":{}}},\\\"volumeClaimTemplate\\\":{\\\"description\\\":\\\"PersistentVolumeClaim is a user's request for and claim to a persistent volume\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"metadata\\\":{\\\"description\\\":\\\"ObjectMeta is metadata that all persisted resources must have, which includes all objects users must create.\\\",\\\"properties\\\":{\\\"annotations\\\":{\\\"description\\\":\\\"Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations\\\",\\\"type\\\":\\\"object\\\"},\\\"clusterName\\\":{\\\"description\\\":\\\"The name of the cluster which the object belongs to. This is used to distinguish resources with same name and namespace in different clusters. This field is not set anywhere right now and apiserver is going to ignore it if set in create or update request.\\\",\\\"type\\\":\\\"string\\\"},\\\"creationTimestamp\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"deletionGracePeriodSeconds\\\":{\\\"description\\\":\\\"Number of seconds allowed for this object to gracefully terminate before it will be removed from the system. Only set when deletionTimestamp is also set. May only be shortened. Read-only.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"deletionTimestamp\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"finalizers\\\":{\\\"description\\\":\\\"Must be empty before the object is deleted from the registry. Each entry is an identifier for the responsible component that will remove the entry from the list. If the deletionTimestamp of the object is non-nil, entries in this list can only be removed.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"generateName\\\":{\\\"description\\\":\\\"GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server.\\\\nIf this field is specified and the generated name exists, the server will NOT return a 409 - instead, it will either return 201 Created or 500 with Reason ServerTimeout indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header).\\\\nApplied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#idempotency\\\",\\\"type\\\":\\\"string\\\"},\\\"generation\\\":{\\\"description\\\":\\\"A sequence number representing a specific generation of the desired state. Populated by the system. Read-only.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"initializers\\\":{\\\"description\\\":\\\"Initializers tracks the progress of initialization.\\\",\\\"properties\\\":{\\\"pending\\\":{\\\"description\\\":\\\"Pending is a list of initializers that must execute in order before this object is visible. When the last pending initializer is removed, and no failing result is set, the initializers struct will be set to nil and the object is considered as initialized and visible to all clients.\\\",\\\"items\\\":{\\\"description\\\":\\\"Initializer is information about an initializer that has not yet completed.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"name of the process that is responsible for initializing this object.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\"]},\\\"type\\\":\\\"array\\\"},\\\"result\\\":{\\\"description\\\":\\\"Status is a return value for calls that don't return other objects.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"code\\\":{\\\"description\\\":\\\"Suggested HTTP return code for this status, 0 if not set.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"details\\\":{\\\"description\\\":\\\"StatusDetails is a set of additional properties that MAY be set by the server to provide additional information about a response. The Reason field of a Status object defines what attributes will be set. Clients must ignore fields that do not match the defined type of each attribute, and should assume that any attribute may be empty, invalid, or under defined.\\\",\\\"properties\\\":{\\\"causes\\\":{\\\"description\\\":\\\"The Causes array includes more details associated with the StatusReason failure. Not all StatusReasons may provide detailed causes.\\\",\\\"items\\\":{\\\"description\\\":\\\"StatusCause provides more information about an api.Status failure, including cases when multiple errors are encountered.\\\",\\\"properties\\\":{\\\"field\\\":{\\\"description\\\":\\\"The field of the resource that has caused this error, as named by its JSON serialization. May include dot and postfix notation for nested attributes. Arrays are zero-indexed. Fields may appear more than once in an array of causes due to fields having multiple errors. Optional.\\\\nExamples:\\\\n \\\\\\\"name\\\\\\\" - the field \\\\\\\"name\\\\\\\" on the current resource\\\\n \\\\\\\"items[0].name\\\\\\\" - the field \\\\\\\"name\\\\\\\" on the first array entry in \\\\\\\"items\\\\\\\"\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"A human-readable description of the cause of the error. This field may be presented as-is to a reader.\\\",\\\"type\\\":\\\"string\\\"},\\\"reason\\\":{\\\"description\\\":\\\"A machine-readable description of the cause of the error. If this value is empty there is no information available.\\\",\\\"type\\\":\\\"string\\\"}}},\\\"type\\\":\\\"array\\\"},\\\"group\\\":{\\\"description\\\":\\\"The group attribute of the resource associated with the status StatusReason.\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"The kind attribute of the resource associated with the status StatusReason. On some operations may differ from the requested resource Kind. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"The name attribute of the resource associated with the status StatusReason (when there is a single name which can be described).\\\",\\\"type\\\":\\\"string\\\"},\\\"retryAfterSeconds\\\":{\\\"description\\\":\\\"If specified, the time in seconds before the operation should be retried. Some errors may indicate the client must take an alternate action - for those errors this field may indicate how long to wait before taking the alternate action.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID of the resource. (when there is a single resource which can be described). More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}}},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"A human-readable description of the status of this operation.\\\",\\\"type\\\":\\\"string\\\"},\\\"metadata\\\":{\\\"description\\\":\\\"ListMeta describes metadata that synthetic resources must have, including lists and various status objects. A resource may have only one of {ObjectMeta, ListMeta}.\\\",\\\"properties\\\":{\\\"continue\\\":{\\\"description\\\":\\\"continue may be set if the user set a limit on the number of items returned, and indicates that the server has more data available. The value is opaque and may be used to issue another request to the endpoint that served this list to retrieve the next set of available objects. Continuing a consistent list may not be possible if the server configuration has changed or more than a few minutes have passed. The resourceVersion field returned when using this continue value will be identical to the value in the first response, unless you have received this token from an error message.\\\",\\\"type\\\":\\\"string\\\"},\\\"resourceVersion\\\":{\\\"description\\\":\\\"String that identifies the server's internal version of this object that can be used by clients to determine when objects have changed. Value must be treated as opaque by clients and passed unmodified back to the server. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\\\",\\\"type\\\":\\\"string\\\"},\\\"selfLink\\\":{\\\"description\\\":\\\"selfLink is a URL representing this object. Populated by the system. Read-only.\\\",\\\"type\\\":\\\"string\\\"}}},\\\"reason\\\":{\\\"description\\\":\\\"A machine-readable description of why this operation is in the \\\\\\\"Failure\\\\\\\" status. If this value is empty there is no information available. A Reason clarifies an HTTP status code but does not override it.\\\",\\\"type\\\":\\\"string\\\"},\\\"status\\\":{\\\"description\\\":\\\"Status of the operation. One of: \\\\\\\"Success\\\\\\\" or \\\\\\\"Failure\\\\\\\". More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status\\\",\\\"type\\\":\\\"string\\\"}}}},\\\"required\\\":[\\\"pending\\\"]},\\\"labels\\\":{\\\"description\\\":\\\"Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels\\\",\\\"type\\\":\\\"object\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names\\\",\\\"type\\\":\\\"string\\\"},\\\"namespace\\\":{\\\"description\\\":\\\"Namespace defines the space within each name must be unique. An empty namespace is equivalent to the \\\\\\\"default\\\\\\\" namespace, but \\\\\\\"default\\\\\\\" is the canonical representation. Not all objects are required to be scoped to a namespace - the value of this field for those objects will be empty.\\\\nMust be a DNS_LABEL. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/namespaces\\\",\\\"type\\\":\\\"string\\\"},\\\"ownerReferences\\\":{\\\"description\\\":\\\"List of objects depended by this object. If ALL objects in the list have been deleted, this object will be garbage collected. If this object is managed by a controller, then an entry in this list will point to this controller, with the controller field set to true. There cannot be more than one managing controller.\\\",\\\"items\\\":{\\\"description\\\":\\\"OwnerReference contains enough information to let you identify an owning object. An owning object must be in the same namespace as the dependent, or be cluster-scoped, so there is no namespace field.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"API version of the referent.\\\",\\\"type\\\":\\\"string\\\"},\\\"blockOwnerDeletion\\\":{\\\"description\\\":\\\"If true, AND if the owner has the \\\\\\\"foregroundDeletion\\\\\\\" finalizer, then the owner cannot be deleted from the key-value store until this reference is removed. Defaults to false. To set this field, a user needs \\\\\\\"delete\\\\\\\" permission of the owner, otherwise 422 (Unprocessable Entity) will be returned.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"controller\\\":{\\\"description\\\":\\\"If true, this reference points to the managing controller.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#names\\\",\\\"type\\\":\\\"string\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"apiVersion\\\",\\\"kind\\\",\\\"name\\\",\\\"uid\\\"]},\\\"type\\\":\\\"array\\\"},\\\"resourceVersion\\\":{\\\"description\\\":\\\"An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources.\\\\nPopulated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\\\",\\\"type\\\":\\\"string\\\"},\\\"selfLink\\\":{\\\"description\\\":\\\"SelfLink is a URL representing this object. Populated by the system. Read-only.\\\",\\\"type\\\":\\\"string\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID is the unique in time and space value for this object. It is typically generated by the server on successful creation of a resource and is not allowed to change on PUT operations.\\\\nPopulated by the system. Read-only. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}}},\\\"spec\\\":{\\\"description\\\":\\\"PersistentVolumeClaimSpec describes the common attributes of storage devices and allows a Source for provider-specific attributes\\\",\\\"properties\\\":{\\\"accessModes\\\":{\\\"description\\\":\\\"AccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"dataSource\\\":{\\\"description\\\":\\\"TypedLocalObjectReference contains enough information to let you locate the typed referenced object inside the same namespace.\\\",\\\"properties\\\":{\\\"apiGroup\\\":{\\\"description\\\":\\\"APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is the type of resource being referenced\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name is the name of resource being referenced\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"kind\\\",\\\"name\\\"]},\\\"resources\\\":{\\\"description\\\":\\\"ResourceRequirements describes the compute resource requirements.\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}}},\\\"selector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"]},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}}},\\\"storageClassName\\\":{\\\"description\\\":\\\"Name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1\\\",\\\"type\\\":\\\"string\\\"},\\\"volumeMode\\\":{\\\"description\\\":\\\"volumeMode defines what type of volume is required by the claim. Value of Filesystem is implied when not included in claim spec. This is a beta feature.\\\",\\\"type\\\":\\\"string\\\"},\\\"volumeName\\\":{\\\"description\\\":\\\"VolumeName is the binding reference to the PersistentVolume backing this claim.\\\",\\\"type\\\":\\\"string\\\"}}},\\\"status\\\":{\\\"description\\\":\\\"PersistentVolumeClaimStatus is the current status of a persistent volume claim.\\\",\\\"properties\\\":{\\\"accessModes\\\":{\\\"description\\\":\\\"AccessModes contains the actual access modes the volume backing the PVC has. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"capacity\\\":{\\\"description\\\":\\\"Represents the actual resources of the underlying volume.\\\",\\\"type\\\":\\\"object\\\"},\\\"conditions\\\":{\\\"description\\\":\\\"Current Condition of persistent volume claim. If underlying persistent volume is being resized then the Condition will be set to 'ResizeStarted'.\\\",\\\"items\\\":{\\\"description\\\":\\\"PersistentVolumeClaimCondition contains details about state of pvc\\\",\\\"properties\\\":{\\\"lastProbeTime\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"lastTransitionTime\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"Human-readable message indicating details about last transition.\\\",\\\"type\\\":\\\"string\\\"},\\\"reason\\\":{\\\"description\\\":\\\"Unique, this should be a short, machine understandable string that gives the reason for condition's last transition. If it reports \\\\\\\"ResizeStarted\\\\\\\" that means the underlying persistent volume is being resized.\\\",\\\"type\\\":\\\"string\\\"},\\\"status\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":{\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"type\\\",\\\"status\\\"]},\\\"type\\\":\\\"array\\\"},\\\"phase\\\":{\\\"description\\\":\\\"Phase represents the current phase of PersistentVolumeClaim.\\\",\\\"type\\\":\\\"string\\\"}}}}}}},\\\"tag\\\":{\\\"description\\\":\\\"Tag of Alertmanager container image to be deployed. Defaults to the value of `version`. Version is ignored if Tag is set.\\\",\\\"type\\\":\\\"string\\\"},\\\"tolerations\\\":{\\\"description\\\":\\\"If specified, the pod's tolerations.\\\",\\\"items\\\":{\\\"description\\\":\\\"The pod this Toleration is attached to tolerates any taint that matches the triple \\\\u003ckey,value,effect\\\\u003e using the matching operator \\\\u003coperator\\\\u003e.\\\",\\\"properties\\\":{\\\"effect\\\":{\\\"description\\\":\\\"Effect indicates the taint effect to match. Empty means match all taint effects. When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.\\\",\\\"type\\\":\\\"string\\\"},\\\"key\\\":{\\\"description\\\":\\\"Key is the taint key that the toleration applies to. Empty means match all taint keys. If the key is empty, operator must be Exists; this combination means to match all values and all keys.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Operator represents a key's relationship to the value. Valid operators are Exists and Equal. Defaults to Equal. Exists is equivalent to wildcard for value, so that a pod can tolerate all taints of a particular category.\\\",\\\"type\\\":\\\"string\\\"},\\\"tolerationSeconds\\\":{\\\"description\\\":\\\"TolerationSeconds represents the period of time the toleration (which must be of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately) by the system.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"value\\\":{\\\"description\\\":\\\"Value is the taint value the toleration matches to. If the operator is Exists, the value should be empty, otherwise just a regular string.\\\",\\\"type\\\":\\\"string\\\"}}},\\\"type\\\":\\\"array\\\"},\\\"version\\\":{\\\"description\\\":\\\"Version the cluster should be on.\\\",\\\"type\\\":\\\"string\\\"}}},\\\"status\\\":{\\\"description\\\":\\\"AlertmanagerStatus is the most recent observed status of the Alertmanager cluster. Read-only. Not included when requesting from the apiserver, only from the Prometheus Operator API itself. More info: https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#spec-and-status\\\",\\\"properties\\\":{\\\"availableReplicas\\\":{\\\"description\\\":\\\"Total number of available pods (ready for at least minReadySeconds) targeted by this Alertmanager cluster.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"paused\\\":{\\\"description\\\":\\\"Represents whether any actions on the underlying managed objects are being performed. Only delete actions will be performed.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"replicas\\\":{\\\"description\\\":\\\"Total number of non-terminated pods targeted by this Alertmanager cluster (their labels match the selector).\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"unavailableReplicas\\\":{\\\"description\\\":\\\"Total number of unavailable pods targeted by this Alertmanager cluster.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"updatedReplicas\\\":{\\\"description\\\":\\\"Total number of non-terminated pods targeted by this Alertmanager cluster that have the desired version spec.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"required\\\":[\\\"paused\\\",\\\"replicas\\\",\\\"updatedReplicas\\\",\\\"availableReplicas\\\",\\\"unavailableReplicas\\\"]}}}},\\\"version\\\":\\\"v1\\\"}}\\n\"\n },\n \"creationTimestamp\": \"2020-05-05T16:51:39Z\",\n \"generation\": 1,\n \"labels\": {\n \"app\": \"prometheus-operator\"\n },\n \"name\": \"alertmanagers.monitoring.coreos.com\",\n \"resourceVersion\": \"206434\",\n \"selfLink\": \"/apis/apiextensions.k8s.io/v1/customresourcedefinitions/alertmanagers.monitoring.coreos.com\",\n \"uid\": \"768a7255-d97a-4762-a400-f359fb24f4c8\"\n },\n \"spec\": {\n \"conversion\": {\n \"strategy\": \"None\"\n },\n \"group\": \"monitoring.coreos.com\",\n \"names\": {\n \"kind\": \"Alertmanager\",\n \"listKind\": \"AlertmanagerList\",\n \"plural\": \"alertmanagers\",\n \"singular\": \"alertmanager\"\n },\n \"preserveUnknownFields\": true,\n \"scope\": \"Namespaced\",\n \"versions\": [\n {\n \"name\": \"v1\",\n \"schema\": {\n \"openAPIV3Schema\": {\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"spec\": {\n \"description\": \"AlertmanagerSpec is a specification of the desired behavior of the Alertmanager cluster. More info: https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#spec-and-status\",\n \"properties\": {\n \"additionalPeers\": {\n \"description\": \"AdditionalPeers allows injecting a set of additional Alertmanagers to peer with to form a highly available cluster.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"affinity\": {\n \"description\": \"Affinity is a group of affinity scheduling rules.\",\n \"properties\": {\n \"nodeAffinity\": {\n \"description\": \"Node affinity is a group of node affinity scheduling rules.\",\n \"properties\": {\n \"preferredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\"weight\\\" to the sum if the node matches the corresponding matchExpressions; the node(s) with the highest sum are the most preferred.\",\n \"items\": {\n \"description\": \"An empty preferred scheduling term matches all objects with implicit weight 0 (i.e. it's a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op).\",\n \"properties\": {\n \"preference\": {\n \"description\": \"A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"A list of node selector requirements by node's labels.\",\n \"items\": {\n \"description\": \"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ]\n },\n \"type\": \"array\"\n },\n \"matchFields\": {\n \"description\": \"A list of node selector requirements by node's fields.\",\n \"items\": {\n \"description\": \"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ]\n },\n \"type\": \"array\"\n }\n }\n },\n \"weight\": {\n \"description\": \"Weight associated with matching the corresponding nodeSelectorTerm, in the range 1-100.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"weight\",\n \"preference\"\n ]\n },\n \"type\": \"array\"\n },\n \"requiredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"A node selector represents the union of the results of one or more label queries over a set of nodes; that is, it represents the OR of the selectors represented by the node selector terms.\",\n \"properties\": {\n \"nodeSelectorTerms\": {\n \"description\": \"Required. A list of node selector terms. The terms are ORed.\",\n \"items\": {\n \"description\": \"A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"A list of node selector requirements by node's labels.\",\n \"items\": {\n \"description\": \"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ]\n },\n \"type\": \"array\"\n },\n \"matchFields\": {\n \"description\": \"A list of node selector requirements by node's fields.\",\n \"items\": {\n \"description\": \"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ]\n },\n \"type\": \"array\"\n }\n }\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"nodeSelectorTerms\"\n ]\n }\n }\n },\n \"podAffinity\": {\n \"description\": \"Pod affinity is a group of inter pod affinity scheduling rules.\",\n \"properties\": {\n \"preferredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\"weight\\\" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.\",\n \"items\": {\n \"description\": \"The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)\",\n \"properties\": {\n \"podAffinityTerm\": {\n \"description\": \"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\u003ctopologyKey\\u003e matches that of any node on which a pod of the set of pods is running\",\n \"properties\": {\n \"labelSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ]\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"namespaces\": {\n \"description\": \"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\"this pod's namespace\\\"\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"topologyKey\": {\n \"description\": \"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"topologyKey\"\n ]\n },\n \"weight\": {\n \"description\": \"weight associated with matching the corresponding podAffinityTerm, in the range 1-100.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"weight\",\n \"podAffinityTerm\"\n ]\n },\n \"type\": \"array\"\n },\n \"requiredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.\",\n \"items\": {\n \"description\": \"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\u003ctopologyKey\\u003e matches that of any node on which a pod of the set of pods is running\",\n \"properties\": {\n \"labelSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ]\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"namespaces\": {\n \"description\": \"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\"this pod's namespace\\\"\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"topologyKey\": {\n \"description\": \"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"topologyKey\"\n ]\n },\n \"type\": \"array\"\n }\n }\n },\n \"podAntiAffinity\": {\n \"description\": \"Pod anti affinity is a group of inter pod anti affinity scheduling rules.\",\n \"properties\": {\n \"preferredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"The scheduler will prefer to schedule pods to nodes that satisfy the anti-affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling anti-affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\"weight\\\" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.\",\n \"items\": {\n \"description\": \"The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)\",\n \"properties\": {\n \"podAffinityTerm\": {\n \"description\": \"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\u003ctopologyKey\\u003e matches that of any node on which a pod of the set of pods is running\",\n \"properties\": {\n \"labelSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ]\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"namespaces\": {\n \"description\": \"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\"this pod's namespace\\\"\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"topologyKey\": {\n \"description\": \"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"topologyKey\"\n ]\n },\n \"weight\": {\n \"description\": \"weight associated with matching the corresponding podAffinityTerm, in the range 1-100.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"weight\",\n \"podAffinityTerm\"\n ]\n },\n \"type\": \"array\"\n },\n \"requiredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"If the anti-affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the anti-affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.\",\n \"items\": {\n \"description\": \"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\u003ctopologyKey\\u003e matches that of any node on which a pod of the set of pods is running\",\n \"properties\": {\n \"labelSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ]\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"namespaces\": {\n \"description\": \"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\"this pod's namespace\\\"\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"topologyKey\": {\n \"description\": \"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"topologyKey\"\n ]\n },\n \"type\": \"array\"\n }\n }\n }\n }\n },\n \"baseImage\": {\n \"description\": \"Base image that is used to deploy pods, without tag.\",\n \"type\": \"string\"\n },\n \"configMaps\": {\n \"description\": \"ConfigMaps is a list of ConfigMaps in the same namespace as the Alertmanager object, which shall be mounted into the Alertmanager Pods. The ConfigMaps are mounted into /etc/alertmanager/configmaps/\\u003cconfigmap-name\\u003e.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"configSecret\": {\n \"description\": \"ConfigSecret is the name of a Kubernetes Secret in the same namespace as the Alertmanager object, which contains configuration for this Alertmanager instance. Defaults to 'alertmanager-' The secret is mounted into /etc/alertmanager/config.\",\n \"type\": \"string\"\n },\n \"containers\": {\n \"description\": \"Containers allows injecting additional containers. This is meant to allow adding an authentication proxy to an Alertmanager pod.\",\n \"items\": {\n \"description\": \"A single application container that you want to run within a pod.\",\n \"properties\": {\n \"args\": {\n \"description\": \"Arguments to the entrypoint. The docker image's CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"command\": {\n \"description\": \"Entrypoint array. Not executed within a shell. The docker image's ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"env\": {\n \"description\": \"List of environment variables to set in the container. Cannot be updated.\",\n \"items\": {\n \"description\": \"EnvVar represents an environment variable present in a Container.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the environment variable. Must be a C_IDENTIFIER.\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to \\\"\\\".\",\n \"type\": \"string\"\n },\n \"valueFrom\": {\n \"description\": \"EnvVarSource represents a source for the value of an EnvVar.\",\n \"properties\": {\n \"configMapKeyRef\": {\n \"description\": \"Selects a key from a ConfigMap.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key to select.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap or it's key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ]\n },\n \"fieldRef\": {\n \"description\": \"ObjectFieldSelector selects an APIVersioned field of an object.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"Version of the schema the FieldPath is written in terms of, defaults to \\\"v1\\\".\",\n \"type\": \"string\"\n },\n \"fieldPath\": {\n \"description\": \"Path of the field to select in the specified API version.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"fieldPath\"\n ]\n },\n \"resourceFieldRef\": {\n \"description\": \"ResourceFieldSelector represents container resources (cpu, memory) and their output format\",\n \"properties\": {\n \"containerName\": {\n \"description\": \"Container name: required for volumes, optional for env vars\",\n \"type\": \"string\"\n },\n \"divisor\": {},\n \"resource\": {\n \"description\": \"Required: resource to select\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"resource\"\n ]\n },\n \"secretKeyRef\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ]\n }\n }\n }\n },\n \"required\": [\n \"name\"\n ]\n },\n \"type\": \"array\"\n },\n \"envFrom\": {\n \"description\": \"List of sources to populate environment variables in the container. The keys defined within a source must be a C_IDENTIFIER. All invalid keys will be reported as an event when the container is starting. When a key exists in multiple sources, the value associated with the last source will take precedence. Values defined by an Env with a duplicate key will take precedence. Cannot be updated.\",\n \"items\": {\n \"description\": \"EnvFromSource represents the source of a set of ConfigMaps\",\n \"properties\": {\n \"configMapRef\": {\n \"description\": \"ConfigMapEnvSource selects a ConfigMap to populate the environment variables with.\\nThe contents of the target ConfigMap's Data field will represent the key-value pairs as environment variables.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap must be defined\",\n \"type\": \"boolean\"\n }\n }\n },\n \"prefix\": {\n \"description\": \"An optional identifier to prepend to each key in the ConfigMap. Must be a C_IDENTIFIER.\",\n \"type\": \"string\"\n },\n \"secretRef\": {\n \"description\": \"SecretEnvSource selects a Secret to populate the environment variables with.\\nThe contents of the target Secret's Data field will represent the key-value pairs as environment variables.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret must be defined\",\n \"type\": \"boolean\"\n }\n }\n }\n }\n },\n \"type\": \"array\"\n },\n \"image\": {\n \"description\": \"Docker image name. More info: https://kubernetes.io/docs/concepts/containers/images This field is optional to allow higher level config management to default or override container images in workload controllers like Deployments and StatefulSets.\",\n \"type\": \"string\"\n },\n \"imagePullPolicy\": {\n \"description\": \"Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images\",\n \"type\": \"string\"\n },\n \"lifecycle\": {\n \"description\": \"Lifecycle describes actions that the management system should take in response to container lifecycle events. For the PostStart and PreStop lifecycle handlers, management of the container blocks until the action is complete, unless the container process fails, in which case the handler is aborted.\",\n \"properties\": {\n \"postStart\": {\n \"description\": \"Handler defines a specific action that should be taken\",\n \"properties\": {\n \"exec\": {\n \"description\": \"ExecAction describes a \\\"run in container\\\" action.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n }\n },\n \"httpGet\": {\n \"description\": \"HTTPGetAction describes an action based on HTTP Get requests.\",\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ]\n },\n \"type\": \"array\"\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"port\"\n ]\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocketAction describes an action based on opening a socket\",\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n }\n },\n \"required\": [\n \"port\"\n ]\n }\n }\n },\n \"preStop\": {\n \"description\": \"Handler defines a specific action that should be taken\",\n \"properties\": {\n \"exec\": {\n \"description\": \"ExecAction describes a \\\"run in container\\\" action.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n }\n },\n \"httpGet\": {\n \"description\": \"HTTPGetAction describes an action based on HTTP Get requests.\",\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ]\n },\n \"type\": \"array\"\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"port\"\n ]\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocketAction describes an action based on opening a socket\",\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n }\n },\n \"required\": [\n \"port\"\n ]\n }\n }\n }\n }\n },\n \"livenessProbe\": {\n \"description\": \"Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic.\",\n \"properties\": {\n \"exec\": {\n \"description\": \"ExecAction describes a \\\"run in container\\\" action.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n }\n },\n \"failureThreshold\": {\n \"description\": \"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"httpGet\": {\n \"description\": \"HTTPGetAction describes an action based on HTTP Get requests.\",\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ]\n },\n \"type\": \"array\"\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"port\"\n ]\n },\n \"initialDelaySeconds\": {\n \"description\": \"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"periodSeconds\": {\n \"description\": \"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"successThreshold\": {\n \"description\": \"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocketAction describes an action based on opening a socket\",\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n }\n },\n \"required\": [\n \"port\"\n ]\n },\n \"timeoutSeconds\": {\n \"description\": \"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n }\n },\n \"name\": {\n \"description\": \"Name of the container specified as a DNS_LABEL. Each container in a pod must have a unique name (DNS_LABEL). Cannot be updated.\",\n \"type\": \"string\"\n },\n \"ports\": {\n \"description\": \"List of ports to expose from the container. Exposing a port here gives the system additional information about the network connections a container uses, but is primarily informational. Not specifying a port here DOES NOT prevent that port from being exposed. Any port which is listening on the default \\\"0.0.0.0\\\" address inside a container will be accessible from the network. Cannot be updated.\",\n \"items\": {\n \"description\": \"ContainerPort represents a network port in a single container.\",\n \"properties\": {\n \"containerPort\": {\n \"description\": \"Number of port to expose on the pod's IP address. This must be a valid port number, 0 \\u003c x \\u003c 65536.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"hostIP\": {\n \"description\": \"What host IP to bind the external port to.\",\n \"type\": \"string\"\n },\n \"hostPort\": {\n \"description\": \"Number of port to expose on the host. If specified, this must be a valid port number, 0 \\u003c x \\u003c 65536. If HostNetwork is specified, this must match ContainerPort. Most containers do not need this.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"name\": {\n \"description\": \"If specified, this must be an IANA_SVC_NAME and unique within the pod. Each named port in a pod must have a unique name. Name for the port that can be referred to by services.\",\n \"type\": \"string\"\n },\n \"protocol\": {\n \"description\": \"Protocol for port. Must be UDP, TCP, or SCTP. Defaults to \\\"TCP\\\".\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"containerPort\"\n ]\n },\n \"type\": \"array\"\n },\n \"readinessProbe\": {\n \"description\": \"Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic.\",\n \"properties\": {\n \"exec\": {\n \"description\": \"ExecAction describes a \\\"run in container\\\" action.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n }\n },\n \"failureThreshold\": {\n \"description\": \"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"httpGet\": {\n \"description\": \"HTTPGetAction describes an action based on HTTP Get requests.\",\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ]\n },\n \"type\": \"array\"\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"port\"\n ]\n },\n \"initialDelaySeconds\": {\n \"description\": \"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"periodSeconds\": {\n \"description\": \"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"successThreshold\": {\n \"description\": \"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocketAction describes an action based on opening a socket\",\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n }\n },\n \"required\": [\n \"port\"\n ]\n },\n \"timeoutSeconds\": {\n \"description\": \"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n }\n },\n \"resources\": {\n \"description\": \"ResourceRequirements describes the compute resource requirements.\",\n \"properties\": {\n \"limits\": {\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n }\n },\n \"securityContext\": {\n \"description\": \"SecurityContext holds security configuration that will be applied to a container. Some fields are present in both SecurityContext and PodSecurityContext. When both are set, the values in SecurityContext take precedence.\",\n \"properties\": {\n \"allowPrivilegeEscalation\": {\n \"description\": \"AllowPrivilegeEscalation controls whether a process can gain more privileges than its parent process. This bool directly controls if the no_new_privs flag will be set on the container process. AllowPrivilegeEscalation is true always when the container is: 1) run as Privileged 2) has CAP_SYS_ADMIN\",\n \"type\": \"boolean\"\n },\n \"capabilities\": {\n \"description\": \"Adds and removes POSIX capabilities from running containers.\",\n \"properties\": {\n \"add\": {\n \"description\": \"Added capabilities\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"drop\": {\n \"description\": \"Removed capabilities\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n }\n },\n \"privileged\": {\n \"description\": \"Run container in privileged mode. Processes in privileged containers are essentially equivalent to root on the host. Defaults to false.\",\n \"type\": \"boolean\"\n },\n \"procMount\": {\n \"description\": \"procMount denotes the type of proc mount to use for the containers. The default is DefaultProcMount which uses the container runtime defaults for readonly paths and masked paths. This requires the ProcMountType feature flag to be enabled.\",\n \"type\": \"string\"\n },\n \"readOnlyRootFilesystem\": {\n \"description\": \"Whether this container has a read-only root filesystem. Default is false.\",\n \"type\": \"boolean\"\n },\n \"runAsGroup\": {\n \"description\": \"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"runAsNonRoot\": {\n \"description\": \"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"type\": \"boolean\"\n },\n \"runAsUser\": {\n \"description\": \"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"seLinuxOptions\": {\n \"description\": \"SELinuxOptions are the labels to be applied to the container\",\n \"properties\": {\n \"level\": {\n \"description\": \"Level is SELinux level label that applies to the container.\",\n \"type\": \"string\"\n },\n \"role\": {\n \"description\": \"Role is a SELinux role label that applies to the container.\",\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"Type is a SELinux type label that applies to the container.\",\n \"type\": \"string\"\n },\n \"user\": {\n \"description\": \"User is a SELinux user label that applies to the container.\",\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"stdin\": {\n \"description\": \"Whether this container should allocate a buffer for stdin in the container runtime. If this is not set, reads from stdin in the container will always result in EOF. Default is false.\",\n \"type\": \"boolean\"\n },\n \"stdinOnce\": {\n \"description\": \"Whether the container runtime should close the stdin channel after it has been opened by a single attach. When stdin is true the stdin stream will remain open across multiple attach sessions. If stdinOnce is set to true, stdin is opened on container start, is empty until the first client attaches to stdin, and then remains open and accepts data until the client disconnects, at which time stdin is closed and remains closed until the container is restarted. If this flag is false, a container processes that reads from stdin will never receive an EOF. Default is false\",\n \"type\": \"boolean\"\n },\n \"terminationMessagePath\": {\n \"description\": \"Optional: Path at which the file to which the container's termination message will be written is mounted into the container's filesystem. Message written is intended to be brief final status, such as an assertion failure message. Will be truncated by the node if greater than 4096 bytes. The total message length across all containers will be limited to 12kb. Defaults to /dev/termination-log. Cannot be updated.\",\n \"type\": \"string\"\n },\n \"terminationMessagePolicy\": {\n \"description\": \"Indicate how the termination message should be populated. File will use the contents of terminationMessagePath to populate the container status message on both success and failure. FallbackToLogsOnError will use the last chunk of container log output if the termination message file is empty and the container exited with an error. The log output is limited to 2048 bytes or 80 lines, whichever is smaller. Defaults to File. Cannot be updated.\",\n \"type\": \"string\"\n },\n \"tty\": {\n \"description\": \"Whether this container should allocate a TTY for itself, also requires 'stdin' to be true. Default is false.\",\n \"type\": \"boolean\"\n },\n \"volumeDevices\": {\n \"description\": \"volumeDevices is the list of block devices to be used by the container. This is a beta feature.\",\n \"items\": {\n \"description\": \"volumeDevice describes a mapping of a raw block device within a container.\",\n \"properties\": {\n \"devicePath\": {\n \"description\": \"devicePath is the path inside of the container that the device will be mapped to.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"name must match the name of a persistentVolumeClaim in the pod\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"devicePath\"\n ]\n },\n \"type\": \"array\"\n },\n \"volumeMounts\": {\n \"description\": \"Pod volumes to mount into the container's filesystem. Cannot be updated.\",\n \"items\": {\n \"description\": \"VolumeMount describes a mounting of a Volume within a container.\",\n \"properties\": {\n \"mountPath\": {\n \"description\": \"Path within the container at which the volume should be mounted. Must not contain ':'.\",\n \"type\": \"string\"\n },\n \"mountPropagation\": {\n \"description\": \"mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"This must match the Name of a Volume.\",\n \"type\": \"string\"\n },\n \"readOnly\": {\n \"description\": \"Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.\",\n \"type\": \"boolean\"\n },\n \"subPath\": {\n \"description\": \"Path within the volume from which the container's volume should be mounted. Defaults to \\\"\\\" (volume's root).\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"mountPath\"\n ]\n },\n \"type\": \"array\"\n },\n \"workingDir\": {\n \"description\": \"Container's working directory. If not specified, the container runtime's default will be used, which might be configured in the container image. Cannot be updated.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\"\n ]\n },\n \"type\": \"array\"\n },\n \"externalUrl\": {\n \"description\": \"The external URL the Alertmanager instances will be available under. This is necessary to generate correct URLs. This is necessary if Alertmanager is not served from root of a DNS name.\",\n \"type\": \"string\"\n },\n \"image\": {\n \"description\": \"Image if specified has precedence over baseImage, tag and sha combinations. Specifying the version is still necessary to ensure the Prometheus Operator knows what version of Alertmanager is being configured.\",\n \"type\": \"string\"\n },\n \"imagePullSecrets\": {\n \"description\": \"An optional list of references to secrets in the same namespace to use for pulling prometheus and alertmanager images from registries see http://kubernetes.io/docs/user-guide/images#specifying-imagepullsecrets-on-a-pod\",\n \"items\": {\n \"description\": \"LocalObjectReference contains enough information to let you locate the referenced object inside the same namespace.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n }\n }\n },\n \"type\": \"array\"\n },\n \"listenLocal\": {\n \"description\": \"ListenLocal makes the Alertmanager server listen on loopback, so that it does not bind against the Pod IP. Note this is only for the Alertmanager UI, not the gossip communication.\",\n \"type\": \"boolean\"\n },\n \"logLevel\": {\n \"description\": \"Log level for Alertmanager to be configured with.\",\n \"type\": \"string\"\n },\n \"nodeSelector\": {\n \"description\": \"Define which Nodes the Pods are scheduled on.\",\n \"type\": \"object\"\n },\n \"paused\": {\n \"description\": \"If set to true all actions on the underlying managed objects are not goint to be performed, except for delete actions.\",\n \"type\": \"boolean\"\n },\n \"podMetadata\": {\n \"description\": \"ObjectMeta is metadata that all persisted resources must have, which includes all objects users must create.\",\n \"properties\": {\n \"annotations\": {\n \"description\": \"Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations\",\n \"type\": \"object\"\n },\n \"clusterName\": {\n \"description\": \"The name of the cluster which the object belongs to. This is used to distinguish resources with same name and namespace in different clusters. This field is not set anywhere right now and apiserver is going to ignore it if set in create or update request.\",\n \"type\": \"string\"\n },\n \"creationTimestamp\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n },\n \"deletionGracePeriodSeconds\": {\n \"description\": \"Number of seconds allowed for this object to gracefully terminate before it will be removed from the system. Only set when deletionTimestamp is also set. May only be shortened. Read-only.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"deletionTimestamp\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n },\n \"finalizers\": {\n \"description\": \"Must be empty before the object is deleted from the registry. Each entry is an identifier for the responsible component that will remove the entry from the list. If the deletionTimestamp of the object is non-nil, entries in this list can only be removed.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"generateName\": {\n \"description\": \"GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server.\\nIf this field is specified and the generated name exists, the server will NOT return a 409 - instead, it will either return 201 Created or 500 with Reason ServerTimeout indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header).\\nApplied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#idempotency\",\n \"type\": \"string\"\n },\n \"generation\": {\n \"description\": \"A sequence number representing a specific generation of the desired state. Populated by the system. Read-only.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"initializers\": {\n \"description\": \"Initializers tracks the progress of initialization.\",\n \"properties\": {\n \"pending\": {\n \"description\": \"Pending is a list of initializers that must execute in order before this object is visible. When the last pending initializer is removed, and no failing result is set, the initializers struct will be set to nil and the object is considered as initialized and visible to all clients.\",\n \"items\": {\n \"description\": \"Initializer is information about an initializer that has not yet completed.\",\n \"properties\": {\n \"name\": {\n \"description\": \"name of the process that is responsible for initializing this object.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\"\n ]\n },\n \"type\": \"array\"\n },\n \"result\": {\n \"description\": \"Status is a return value for calls that don't return other objects.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"code\": {\n \"description\": \"Suggested HTTP return code for this status, 0 if not set.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"details\": {\n \"description\": \"StatusDetails is a set of additional properties that MAY be set by the server to provide additional information about a response. The Reason field of a Status object defines what attributes will be set. Clients must ignore fields that do not match the defined type of each attribute, and should assume that any attribute may be empty, invalid, or under defined.\",\n \"properties\": {\n \"causes\": {\n \"description\": \"The Causes array includes more details associated with the StatusReason failure. Not all StatusReasons may provide detailed causes.\",\n \"items\": {\n \"description\": \"StatusCause provides more information about an api.Status failure, including cases when multiple errors are encountered.\",\n \"properties\": {\n \"field\": {\n \"description\": \"The field of the resource that has caused this error, as named by its JSON serialization. May include dot and postfix notation for nested attributes. Arrays are zero-indexed. Fields may appear more than once in an array of causes due to fields having multiple errors. Optional.\\nExamples:\\n \\\"name\\\" - the field \\\"name\\\" on the current resource\\n \\\"items[0].name\\\" - the field \\\"name\\\" on the first array entry in \\\"items\\\"\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"A human-readable description of the cause of the error. This field may be presented as-is to a reader.\",\n \"type\": \"string\"\n },\n \"reason\": {\n \"description\": \"A machine-readable description of the cause of the error. If this value is empty there is no information available.\",\n \"type\": \"string\"\n }\n }\n },\n \"type\": \"array\"\n },\n \"group\": {\n \"description\": \"The group attribute of the resource associated with the status StatusReason.\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"The kind attribute of the resource associated with the status StatusReason. On some operations may differ from the requested resource Kind. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"The name attribute of the resource associated with the status StatusReason (when there is a single name which can be described).\",\n \"type\": \"string\"\n },\n \"retryAfterSeconds\": {\n \"description\": \"If specified, the time in seconds before the operation should be retried. Some errors may indicate the client must take an alternate action - for those errors this field may indicate how long to wait before taking the alternate action.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"uid\": {\n \"description\": \"UID of the resource. (when there is a single resource which can be described). More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n }\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"A human-readable description of the status of this operation.\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"description\": \"ListMeta describes metadata that synthetic resources must have, including lists and various status objects. A resource may have only one of {ObjectMeta, ListMeta}.\",\n \"properties\": {\n \"continue\": {\n \"description\": \"continue may be set if the user set a limit on the number of items returned, and indicates that the server has more data available. The value is opaque and may be used to issue another request to the endpoint that served this list to retrieve the next set of available objects. Continuing a consistent list may not be possible if the server configuration has changed or more than a few minutes have passed. The resourceVersion field returned when using this continue value will be identical to the value in the first response, unless you have received this token from an error message.\",\n \"type\": \"string\"\n },\n \"resourceVersion\": {\n \"description\": \"String that identifies the server's internal version of this object that can be used by clients to determine when objects have changed. Value must be treated as opaque by clients and passed unmodified back to the server. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\",\n \"type\": \"string\"\n },\n \"selfLink\": {\n \"description\": \"selfLink is a URL representing this object. Populated by the system. Read-only.\",\n \"type\": \"string\"\n }\n }\n },\n \"reason\": {\n \"description\": \"A machine-readable description of why this operation is in the \\\"Failure\\\" status. If this value is empty there is no information available. A Reason clarifies an HTTP status code but does not override it.\",\n \"type\": \"string\"\n },\n \"status\": {\n \"description\": \"Status of the operation. One of: \\\"Success\\\" or \\\"Failure\\\". More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"required\": [\n \"pending\"\n ]\n },\n \"labels\": {\n \"description\": \"Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels\",\n \"type\": \"object\"\n },\n \"name\": {\n \"description\": \"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names\",\n \"type\": \"string\"\n },\n \"namespace\": {\n \"description\": \"Namespace defines the space within each name must be unique. An empty namespace is equivalent to the \\\"default\\\" namespace, but \\\"default\\\" is the canonical representation. Not all objects are required to be scoped to a namespace - the value of this field for those objects will be empty.\\nMust be a DNS_LABEL. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/namespaces\",\n \"type\": \"string\"\n },\n \"ownerReferences\": {\n \"description\": \"List of objects depended by this object. If ALL objects in the list have been deleted, this object will be garbage collected. If this object is managed by a controller, then an entry in this list will point to this controller, with the controller field set to true. There cannot be more than one managing controller.\",\n \"items\": {\n \"description\": \"OwnerReference contains enough information to let you identify an owning object. An owning object must be in the same namespace as the dependent, or be cluster-scoped, so there is no namespace field.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"API version of the referent.\",\n \"type\": \"string\"\n },\n \"blockOwnerDeletion\": {\n \"description\": \"If true, AND if the owner has the \\\"foregroundDeletion\\\" finalizer, then the owner cannot be deleted from the key-value store until this reference is removed. Defaults to false. To set this field, a user needs \\\"delete\\\" permission of the owner, otherwise 422 (Unprocessable Entity) will be returned.\",\n \"type\": \"boolean\"\n },\n \"controller\": {\n \"description\": \"If true, this reference points to the managing controller.\",\n \"type\": \"boolean\"\n },\n \"kind\": {\n \"description\": \"Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#names\",\n \"type\": \"string\"\n },\n \"uid\": {\n \"description\": \"UID of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"apiVersion\",\n \"kind\",\n \"name\",\n \"uid\"\n ]\n },\n \"type\": \"array\"\n },\n \"resourceVersion\": {\n \"description\": \"An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources.\\nPopulated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\",\n \"type\": \"string\"\n },\n \"selfLink\": {\n \"description\": \"SelfLink is a URL representing this object. Populated by the system. Read-only.\",\n \"type\": \"string\"\n },\n \"uid\": {\n \"description\": \"UID is the unique in time and space value for this object. It is typically generated by the server on successful creation of a resource and is not allowed to change on PUT operations.\\nPopulated by the system. Read-only. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n }\n },\n \"priorityClassName\": {\n \"description\": \"Priority class assigned to the Pods\",\n \"type\": \"string\"\n },\n \"replicas\": {\n \"description\": \"Size is the expected size of the alertmanager cluster. The controller will eventually make the size of the running cluster equal to the expected size.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"resources\": {\n \"description\": \"ResourceRequirements describes the compute resource requirements.\",\n \"properties\": {\n \"limits\": {\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n }\n },\n \"retention\": {\n \"description\": \"Time duration Alertmanager shall retain data for. Default is '120h', and must match the regular expression `[0-9]+(ms|s|m|h)` (milliseconds seconds minutes hours).\",\n \"type\": \"string\"\n },\n \"routePrefix\": {\n \"description\": \"The route prefix Alertmanager registers HTTP handlers for. This is useful, if using ExternalURL and a proxy is rewriting HTTP routes of a request, and the actual ExternalURL is still true, but the server serves requests under a different route prefix. For example for use with `kubectl proxy`.\",\n \"type\": \"string\"\n },\n \"secrets\": {\n \"description\": \"Secrets is a list of Secrets in the same namespace as the Alertmanager object, which shall be mounted into the Alertmanager Pods. The Secrets are mounted into /etc/alertmanager/secrets/\\u003csecret-name\\u003e.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"securityContext\": {\n \"description\": \"PodSecurityContext holds pod-level security attributes and common container settings. Some fields are also present in container.securityContext. Field values of container.securityContext take precedence over field values of PodSecurityContext.\",\n \"properties\": {\n \"fsGroup\": {\n \"description\": \"A special supplemental group that applies to all containers in a pod. Some volume types allow the Kubelet to change the ownership of that volume to be owned by the pod:\\n1. The owning GID will be the FSGroup 2. The setgid bit is set (new files created in the volume will be owned by FSGroup) 3. The permission bits are OR'd with rw-rw----\\nIf unset, the Kubelet will not modify the ownership and permissions of any volume.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"runAsGroup\": {\n \"description\": \"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"runAsNonRoot\": {\n \"description\": \"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"type\": \"boolean\"\n },\n \"runAsUser\": {\n \"description\": \"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"seLinuxOptions\": {\n \"description\": \"SELinuxOptions are the labels to be applied to the container\",\n \"properties\": {\n \"level\": {\n \"description\": \"Level is SELinux level label that applies to the container.\",\n \"type\": \"string\"\n },\n \"role\": {\n \"description\": \"Role is a SELinux role label that applies to the container.\",\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"Type is a SELinux type label that applies to the container.\",\n \"type\": \"string\"\n },\n \"user\": {\n \"description\": \"User is a SELinux user label that applies to the container.\",\n \"type\": \"string\"\n }\n }\n },\n \"supplementalGroups\": {\n \"description\": \"A list of groups applied to the first process run in each container, in addition to the container's primary GID. If unspecified, no groups will be added to any container.\",\n \"items\": {\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"type\": \"array\"\n },\n \"sysctls\": {\n \"description\": \"Sysctls hold a list of namespaced sysctls used for the pod. Pods with unsupported sysctls (by the container runtime) might fail to launch.\",\n \"items\": {\n \"description\": \"Sysctl defines a kernel parameter to be set\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of a property to set\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"Value of a property to set\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ]\n },\n \"type\": \"array\"\n }\n }\n },\n \"serviceAccountName\": {\n \"description\": \"ServiceAccountName is the name of the ServiceAccount to use to run the Prometheus Pods.\",\n \"type\": \"string\"\n },\n \"sha\": {\n \"description\": \"SHA of Alertmanager container image to be deployed. Defaults to the value of `version`. Similar to a tag, but the SHA explicitly deploys an immutable container image. Version and Tag are ignored if SHA is set.\",\n \"type\": \"string\"\n },\n \"storage\": {\n \"description\": \"StorageSpec defines the configured storage for a group Prometheus servers. If neither `emptyDir` nor `volumeClaimTemplate` is specified, then by default an [EmptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir) will be used.\",\n \"properties\": {\n \"emptyDir\": {\n \"description\": \"Represents an empty directory for a pod. Empty directory volumes support ownership management and SELinux relabeling.\",\n \"properties\": {\n \"medium\": {\n \"description\": \"What type of storage medium should back this directory. The default is \\\"\\\" which means to use the node's default medium. Must be an empty string (default) or Memory. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir\",\n \"type\": \"string\"\n },\n \"sizeLimit\": {}\n }\n },\n \"volumeClaimTemplate\": {\n \"description\": \"PersistentVolumeClaim is a user's request for and claim to a persistent volume\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"description\": \"ObjectMeta is metadata that all persisted resources must have, which includes all objects users must create.\",\n \"properties\": {\n \"annotations\": {\n \"description\": \"Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations\",\n \"type\": \"object\"\n },\n \"clusterName\": {\n \"description\": \"The name of the cluster which the object belongs to. This is used to distinguish resources with same name and namespace in different clusters. This field is not set anywhere right now and apiserver is going to ignore it if set in create or update request.\",\n \"type\": \"string\"\n },\n \"creationTimestamp\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n },\n \"deletionGracePeriodSeconds\": {\n \"description\": \"Number of seconds allowed for this object to gracefully terminate before it will be removed from the system. Only set when deletionTimestamp is also set. May only be shortened. Read-only.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"deletionTimestamp\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n },\n \"finalizers\": {\n \"description\": \"Must be empty before the object is deleted from the registry. Each entry is an identifier for the responsible component that will remove the entry from the list. If the deletionTimestamp of the object is non-nil, entries in this list can only be removed.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"generateName\": {\n \"description\": \"GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server.\\nIf this field is specified and the generated name exists, the server will NOT return a 409 - instead, it will either return 201 Created or 500 with Reason ServerTimeout indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header).\\nApplied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#idempotency\",\n \"type\": \"string\"\n },\n \"generation\": {\n \"description\": \"A sequence number representing a specific generation of the desired state. Populated by the system. Read-only.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"initializers\": {\n \"description\": \"Initializers tracks the progress of initialization.\",\n \"properties\": {\n \"pending\": {\n \"description\": \"Pending is a list of initializers that must execute in order before this object is visible. When the last pending initializer is removed, and no failing result is set, the initializers struct will be set to nil and the object is considered as initialized and visible to all clients.\",\n \"items\": {\n \"description\": \"Initializer is information about an initializer that has not yet completed.\",\n \"properties\": {\n \"name\": {\n \"description\": \"name of the process that is responsible for initializing this object.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\"\n ]\n },\n \"type\": \"array\"\n },\n \"result\": {\n \"description\": \"Status is a return value for calls that don't return other objects.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"code\": {\n \"description\": \"Suggested HTTP return code for this status, 0 if not set.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"details\": {\n \"description\": \"StatusDetails is a set of additional properties that MAY be set by the server to provide additional information about a response. The Reason field of a Status object defines what attributes will be set. Clients must ignore fields that do not match the defined type of each attribute, and should assume that any attribute may be empty, invalid, or under defined.\",\n \"properties\": {\n \"causes\": {\n \"description\": \"The Causes array includes more details associated with the StatusReason failure. Not all StatusReasons may provide detailed causes.\",\n \"items\": {\n \"description\": \"StatusCause provides more information about an api.Status failure, including cases when multiple errors are encountered.\",\n \"properties\": {\n \"field\": {\n \"description\": \"The field of the resource that has caused this error, as named by its JSON serialization. May include dot and postfix notation for nested attributes. Arrays are zero-indexed. Fields may appear more than once in an array of causes due to fields having multiple errors. Optional.\\nExamples:\\n \\\"name\\\" - the field \\\"name\\\" on the current resource\\n \\\"items[0].name\\\" - the field \\\"name\\\" on the first array entry in \\\"items\\\"\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"A human-readable description of the cause of the error. This field may be presented as-is to a reader.\",\n \"type\": \"string\"\n },\n \"reason\": {\n \"description\": \"A machine-readable description of the cause of the error. If this value is empty there is no information available.\",\n \"type\": \"string\"\n }\n }\n },\n \"type\": \"array\"\n },\n \"group\": {\n \"description\": \"The group attribute of the resource associated with the status StatusReason.\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"The kind attribute of the resource associated with the status StatusReason. On some operations may differ from the requested resource Kind. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"The name attribute of the resource associated with the status StatusReason (when there is a single name which can be described).\",\n \"type\": \"string\"\n },\n \"retryAfterSeconds\": {\n \"description\": \"If specified, the time in seconds before the operation should be retried. Some errors may indicate the client must take an alternate action - for those errors this field may indicate how long to wait before taking the alternate action.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"uid\": {\n \"description\": \"UID of the resource. (when there is a single resource which can be described). More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n }\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"A human-readable description of the status of this operation.\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"description\": \"ListMeta describes metadata that synthetic resources must have, including lists and various status objects. A resource may have only one of {ObjectMeta, ListMeta}.\",\n \"properties\": {\n \"continue\": {\n \"description\": \"continue may be set if the user set a limit on the number of items returned, and indicates that the server has more data available. The value is opaque and may be used to issue another request to the endpoint that served this list to retrieve the next set of available objects. Continuing a consistent list may not be possible if the server configuration has changed or more than a few minutes have passed. The resourceVersion field returned when using this continue value will be identical to the value in the first response, unless you have received this token from an error message.\",\n \"type\": \"string\"\n },\n \"resourceVersion\": {\n \"description\": \"String that identifies the server's internal version of this object that can be used by clients to determine when objects have changed. Value must be treated as opaque by clients and passed unmodified back to the server. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\",\n \"type\": \"string\"\n },\n \"selfLink\": {\n \"description\": \"selfLink is a URL representing this object. Populated by the system. Read-only.\",\n \"type\": \"string\"\n }\n }\n },\n \"reason\": {\n \"description\": \"A machine-readable description of why this operation is in the \\\"Failure\\\" status. If this value is empty there is no information available. A Reason clarifies an HTTP status code but does not override it.\",\n \"type\": \"string\"\n },\n \"status\": {\n \"description\": \"Status of the operation. One of: \\\"Success\\\" or \\\"Failure\\\". More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"required\": [\n \"pending\"\n ]\n },\n \"labels\": {\n \"description\": \"Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels\",\n \"type\": \"object\"\n },\n \"name\": {\n \"description\": \"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names\",\n \"type\": \"string\"\n },\n \"namespace\": {\n \"description\": \"Namespace defines the space within each name must be unique. An empty namespace is equivalent to the \\\"default\\\" namespace, but \\\"default\\\" is the canonical representation. Not all objects are required to be scoped to a namespace - the value of this field for those objects will be empty.\\nMust be a DNS_LABEL. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/namespaces\",\n \"type\": \"string\"\n },\n \"ownerReferences\": {\n \"description\": \"List of objects depended by this object. If ALL objects in the list have been deleted, this object will be garbage collected. If this object is managed by a controller, then an entry in this list will point to this controller, with the controller field set to true. There cannot be more than one managing controller.\",\n \"items\": {\n \"description\": \"OwnerReference contains enough information to let you identify an owning object. An owning object must be in the same namespace as the dependent, or be cluster-scoped, so there is no namespace field.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"API version of the referent.\",\n \"type\": \"string\"\n },\n \"blockOwnerDeletion\": {\n \"description\": \"If true, AND if the owner has the \\\"foregroundDeletion\\\" finalizer, then the owner cannot be deleted from the key-value store until this reference is removed. Defaults to false. To set this field, a user needs \\\"delete\\\" permission of the owner, otherwise 422 (Unprocessable Entity) will be returned.\",\n \"type\": \"boolean\"\n },\n \"controller\": {\n \"description\": \"If true, this reference points to the managing controller.\",\n \"type\": \"boolean\"\n },\n \"kind\": {\n \"description\": \"Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#names\",\n \"type\": \"string\"\n },\n \"uid\": {\n \"description\": \"UID of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"apiVersion\",\n \"kind\",\n \"name\",\n \"uid\"\n ]\n },\n \"type\": \"array\"\n },\n \"resourceVersion\": {\n \"description\": \"An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources.\\nPopulated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\",\n \"type\": \"string\"\n },\n \"selfLink\": {\n \"description\": \"SelfLink is a URL representing this object. Populated by the system. Read-only.\",\n \"type\": \"string\"\n },\n \"uid\": {\n \"description\": \"UID is the unique in time and space value for this object. It is typically generated by the server on successful creation of a resource and is not allowed to change on PUT operations.\\nPopulated by the system. Read-only. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n }\n },\n \"spec\": {\n \"description\": \"PersistentVolumeClaimSpec describes the common attributes of storage devices and allows a Source for provider-specific attributes\",\n \"properties\": {\n \"accessModes\": {\n \"description\": \"AccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"dataSource\": {\n \"description\": \"TypedLocalObjectReference contains enough information to let you locate the typed referenced object inside the same namespace.\",\n \"properties\": {\n \"apiGroup\": {\n \"description\": \"APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is the type of resource being referenced\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name is the name of resource being referenced\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"kind\",\n \"name\"\n ]\n },\n \"resources\": {\n \"description\": \"ResourceRequirements describes the compute resource requirements.\",\n \"properties\": {\n \"limits\": {\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n }\n },\n \"selector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ]\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"storageClassName\": {\n \"description\": \"Name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1\",\n \"type\": \"string\"\n },\n \"volumeMode\": {\n \"description\": \"volumeMode defines what type of volume is required by the claim. Value of Filesystem is implied when not included in claim spec. This is a beta feature.\",\n \"type\": \"string\"\n },\n \"volumeName\": {\n \"description\": \"VolumeName is the binding reference to the PersistentVolume backing this claim.\",\n \"type\": \"string\"\n }\n }\n },\n \"status\": {\n \"description\": \"PersistentVolumeClaimStatus is the current status of a persistent volume claim.\",\n \"properties\": {\n \"accessModes\": {\n \"description\": \"AccessModes contains the actual access modes the volume backing the PVC has. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"capacity\": {\n \"description\": \"Represents the actual resources of the underlying volume.\",\n \"type\": \"object\"\n },\n \"conditions\": {\n \"description\": \"Current Condition of persistent volume claim. If underlying persistent volume is being resized then the Condition will be set to 'ResizeStarted'.\",\n \"items\": {\n \"description\": \"PersistentVolumeClaimCondition contains details about state of pvc\",\n \"properties\": {\n \"lastProbeTime\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n },\n \"lastTransitionTime\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"Human-readable message indicating details about last transition.\",\n \"type\": \"string\"\n },\n \"reason\": {\n \"description\": \"Unique, this should be a short, machine understandable string that gives the reason for condition's last transition. If it reports \\\"ResizeStarted\\\" that means the underlying persistent volume is being resized.\",\n \"type\": \"string\"\n },\n \"status\": {\n \"type\": \"string\"\n },\n \"type\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"type\",\n \"status\"\n ]\n },\n \"type\": \"array\"\n },\n \"phase\": {\n \"description\": \"Phase represents the current phase of PersistentVolumeClaim.\",\n \"type\": \"string\"\n }\n }\n }\n }\n }\n }\n },\n \"tag\": {\n \"description\": \"Tag of Alertmanager container image to be deployed. Defaults to the value of `version`. Version is ignored if Tag is set.\",\n \"type\": \"string\"\n },\n \"tolerations\": {\n \"description\": \"If specified, the pod's tolerations.\",\n \"items\": {\n \"description\": \"The pod this Toleration is attached to tolerates any taint that matches the triple \\u003ckey,value,effect\\u003e using the matching operator \\u003coperator\\u003e.\",\n \"properties\": {\n \"effect\": {\n \"description\": \"Effect indicates the taint effect to match. Empty means match all taint effects. When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.\",\n \"type\": \"string\"\n },\n \"key\": {\n \"description\": \"Key is the taint key that the toleration applies to. Empty means match all taint keys. If the key is empty, operator must be Exists; this combination means to match all values and all keys.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Operator represents a key's relationship to the value. Valid operators are Exists and Equal. Defaults to Equal. Exists is equivalent to wildcard for value, so that a pod can tolerate all taints of a particular category.\",\n \"type\": \"string\"\n },\n \"tolerationSeconds\": {\n \"description\": \"TolerationSeconds represents the period of time the toleration (which must be of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately) by the system.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"value\": {\n \"description\": \"Value is the taint value the toleration matches to. If the operator is Exists, the value should be empty, otherwise just a regular string.\",\n \"type\": \"string\"\n }\n }\n },\n \"type\": \"array\"\n },\n \"version\": {\n \"description\": \"Version the cluster should be on.\",\n \"type\": \"string\"\n }\n }\n },\n \"status\": {\n \"description\": \"AlertmanagerStatus is the most recent observed status of the Alertmanager cluster. Read-only. Not included when requesting from the apiserver, only from the Prometheus Operator API itself. More info: https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#spec-and-status\",\n \"properties\": {\n \"availableReplicas\": {\n \"description\": \"Total number of available pods (ready for at least minReadySeconds) targeted by this Alertmanager cluster.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"paused\": {\n \"description\": \"Represents whether any actions on the underlying managed objects are being performed. Only delete actions will be performed.\",\n \"type\": \"boolean\"\n },\n \"replicas\": {\n \"description\": \"Total number of non-terminated pods targeted by this Alertmanager cluster (their labels match the selector).\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"unavailableReplicas\": {\n \"description\": \"Total number of unavailable pods targeted by this Alertmanager cluster.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"updatedReplicas\": {\n \"description\": \"Total number of non-terminated pods targeted by this Alertmanager cluster that have the desired version spec.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"paused\",\n \"replicas\",\n \"updatedReplicas\",\n \"availableReplicas\",\n \"unavailableReplicas\"\n ]\n }\n }\n }\n },\n \"served\": true,\n \"storage\": true\n }\n ]\n },\n \"status\": {\n \"acceptedNames\": {\n \"kind\": \"Alertmanager\",\n \"listKind\": \"AlertmanagerList\",\n \"plural\": \"alertmanagers\",\n \"singular\": \"alertmanager\"\n },\n \"conditions\": [\n {\n \"lastTransitionTime\": \"2020-05-05T16:51:39Z\",\n \"message\": \"no conflicts found\",\n \"reason\": \"NoConflicts\",\n \"status\": \"True\",\n \"type\": \"NamesAccepted\"\n },\n {\n \"lastTransitionTime\": \"2020-05-05T16:51:39Z\",\n \"message\": \"the initial names have been accepted\",\n \"reason\": \"InitialNamesAccepted\",\n \"status\": \"True\",\n \"type\": \"Established\"\n },\n {\n \"lastTransitionTime\": \"2020-05-05T16:51:39Z\",\n \"message\": \"[spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[nodeAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.properties[preference].properties[matchExpressions].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[nodeAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.properties[preference].properties[matchFields].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[nodeAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.properties[preference].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[nodeAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[nodeAffinity].properties[requiredDuringSchedulingIgnoredDuringExecution].properties[nodeSelectorTerms].items.properties[matchExpressions].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[nodeAffinity].properties[requiredDuringSchedulingIgnoredDuringExecution].properties[nodeSelectorTerms].items.properties[matchFields].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[nodeAffinity].properties[requiredDuringSchedulingIgnoredDuringExecution].properties[nodeSelectorTerms].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[nodeAffinity].properties[requiredDuringSchedulingIgnoredDuringExecution].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[nodeAffinity].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.properties[podAffinityTerm].properties[labelSelector].properties[matchExpressions].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.properties[podAffinityTerm].properties[labelSelector].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.properties[podAffinityTerm].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAffinity].properties[requiredDuringSchedulingIgnoredDuringExecution].items.properties[labelSelector].properties[matchExpressions].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAffinity].properties[requiredDuringSchedulingIgnoredDuringExecution].items.properties[labelSelector].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAffinity].properties[requiredDuringSchedulingIgnoredDuringExecution].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAffinity].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAntiAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.properties[podAffinityTerm].properties[labelSelector].properties[matchExpressions].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAntiAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.properties[podAffinityTerm].properties[labelSelector].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAntiAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.properties[podAffinityTerm].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAntiAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAntiAffinity].properties[requiredDuringSchedulingIgnoredDuringExecution].items.properties[labelSelector].properties[matchExpressions].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAntiAffinity].properties[requiredDuringSchedulingIgnoredDuringExecution].items.properties[labelSelector].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAntiAffinity].properties[requiredDuringSchedulingIgnoredDuringExecution].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAntiAffinity].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[envFrom].items.properties[configMapRef].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[envFrom].items.properties[secretRef].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[envFrom].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[env].items.properties[valueFrom].properties[configMapKeyRef].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[env].items.properties[valueFrom].properties[fieldRef].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[env].items.properties[valueFrom].properties[resourceFieldRef].properties[divisor].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[env].items.properties[valueFrom].properties[resourceFieldRef].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[env].items.properties[valueFrom].properties[secretKeyRef].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[env].items.properties[valueFrom].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[env].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[exec].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[httpGet].properties[httpHeaders].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[httpGet].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[httpGet].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[httpGet].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[httpGet].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[tcpSocket].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[tcpSocket].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[tcpSocket].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[tcpSocket].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[exec].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[httpGet].properties[httpHeaders].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[httpGet].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[httpGet].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[httpGet].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[httpGet].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[tcpSocket].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[tcpSocket].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[tcpSocket].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[tcpSocket].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[exec].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[httpGet].properties[httpHeaders].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[httpGet].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[httpGet].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[httpGet].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[httpGet].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[tcpSocket].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[tcpSocket].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[tcpSocket].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[tcpSocket].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[ports].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[exec].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[httpGet].properties[httpHeaders].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[httpGet].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[httpGet].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[httpGet].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[httpGet].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[tcpSocket].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[tcpSocket].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[tcpSocket].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[tcpSocket].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[resources].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[securityContext].properties[capabilities].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[securityContext].properties[seLinuxOptions].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[securityContext].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[volumeDevices].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[volumeMounts].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[imagePullSecrets].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[podMetadata].properties[initializers].properties[pending].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[podMetadata].properties[initializers].properties[result].properties[details].properties[causes].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[podMetadata].properties[initializers].properties[result].properties[details].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[podMetadata].properties[initializers].properties[result].properties[metadata].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[podMetadata].properties[initializers].properties[result].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[podMetadata].properties[initializers].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[podMetadata].properties[ownerReferences].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[podMetadata].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[resources].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[securityContext].properties[seLinuxOptions].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[securityContext].properties[sysctls].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[securityContext].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[emptyDir].properties[sizeLimit].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[emptyDir].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[metadata].properties[initializers].properties[pending].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[metadata].properties[initializers].properties[result].properties[details].properties[causes].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[metadata].properties[initializers].properties[result].properties[details].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[metadata].properties[initializers].properties[result].properties[metadata].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[metadata].properties[initializers].properties[result].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[metadata].properties[initializers].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[metadata].properties[ownerReferences].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[metadata].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[spec].properties[dataSource].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[spec].properties[resources].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[spec].properties[selector].properties[matchExpressions].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[spec].properties[selector].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[spec].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[status].properties[conditions].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[status].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[tolerations].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[status].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.type: Required value: must not be empty at the root]\",\n \"reason\": \"Violations\",\n \"status\": \"True\",\n \"type\": \"NonStructuralSchema\"\n }\n ],\n \"storedVersions\": [\n \"v1\"\n ]\n }\n}\n" }, { "path": "pkg/backup/actions/testdata/v1/elasticsearches.elasticsearch.k8s.elastic.co.json", "content": "{\n \"apiVersion\": \"apiextensions.k8s.io/v1\",\n \"kind\": \"CustomResourceDefinition\",\n \"metadata\": {\n \"annotations\": {\n \"controller-gen.kubebuilder.io/version\": \"v0.2.5\",\n \"kubectl.kubernetes.io/last-applied-configuration\": \"{\\\"apiVersion\\\":\\\"apiextensions.k8s.io/v1beta1\\\",\\\"kind\\\":\\\"CustomResourceDefinition\\\",\\\"metadata\\\":{\\\"annotations\\\":{\\\"controller-gen.kubebuilder.io/version\\\":\\\"v0.2.5\\\"},\\\"creationTimestamp\\\":null,\\\"name\\\":\\\"elasticsearches.elasticsearch.k8s.elastic.co\\\"},\\\"spec\\\":{\\\"additionalPrinterColumns\\\":[{\\\"JSONPath\\\":\\\".status.health\\\",\\\"name\\\":\\\"health\\\",\\\"type\\\":\\\"string\\\"},{\\\"JSONPath\\\":\\\".status.availableNodes\\\",\\\"description\\\":\\\"Available nodes\\\",\\\"name\\\":\\\"nodes\\\",\\\"type\\\":\\\"integer\\\"},{\\\"JSONPath\\\":\\\".spec.version\\\",\\\"description\\\":\\\"Elasticsearch version\\\",\\\"name\\\":\\\"version\\\",\\\"type\\\":\\\"string\\\"},{\\\"JSONPath\\\":\\\".status.phase\\\",\\\"name\\\":\\\"phase\\\",\\\"type\\\":\\\"string\\\"},{\\\"JSONPath\\\":\\\".metadata.creationTimestamp\\\",\\\"name\\\":\\\"age\\\",\\\"type\\\":\\\"date\\\"}],\\\"group\\\":\\\"elasticsearch.k8s.elastic.co\\\",\\\"names\\\":{\\\"categories\\\":[\\\"elastic\\\"],\\\"kind\\\":\\\"Elasticsearch\\\",\\\"listKind\\\":\\\"ElasticsearchList\\\",\\\"plural\\\":\\\"elasticsearches\\\",\\\"shortNames\\\":[\\\"es\\\"],\\\"singular\\\":\\\"elasticsearch\\\"},\\\"scope\\\":\\\"Namespaced\\\",\\\"subresources\\\":{\\\"status\\\":{}},\\\"validation\\\":{\\\"openAPIV3Schema\\\":{\\\"description\\\":\\\"Elasticsearch represents an Elasticsearch resource in a Kubernetes cluster.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"metadata\\\":{\\\"type\\\":\\\"object\\\"},\\\"spec\\\":{\\\"description\\\":\\\"ElasticsearchSpec holds the specification of an Elasticsearch cluster.\\\",\\\"properties\\\":{\\\"auth\\\":{\\\"description\\\":\\\"Auth contains user authentication and authorization security settings for Elasticsearch.\\\",\\\"properties\\\":{\\\"fileRealm\\\":{\\\"description\\\":\\\"FileRealm to propagate to the Elasticsearch cluster.\\\",\\\"items\\\":{\\\"description\\\":\\\"FileRealmSource references users to create in the Elasticsearch cluster.\\\",\\\"properties\\\":{\\\"secretName\\\":{\\\"description\\\":\\\"SecretName is the name of the secret.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"roles\\\":{\\\"description\\\":\\\"Roles to propagate to the Elasticsearch cluster.\\\",\\\"items\\\":{\\\"description\\\":\\\"RoleSource references roles to create in the Elasticsearch cluster.\\\",\\\"properties\\\":{\\\"secretName\\\":{\\\"description\\\":\\\"SecretName is the name of the secret.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"http\\\":{\\\"description\\\":\\\"HTTP holds HTTP layer settings for Elasticsearch.\\\",\\\"properties\\\":{\\\"service\\\":{\\\"description\\\":\\\"Service defines the template for the associated Kubernetes Service object.\\\",\\\"properties\\\":{\\\"metadata\\\":{\\\"description\\\":\\\"ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored.\\\",\\\"type\\\":\\\"object\\\"},\\\"spec\\\":{\\\"description\\\":\\\"Spec is the specification of the service.\\\",\\\"properties\\\":{\\\"clusterIP\\\":{\\\"description\\\":\\\"clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are \\\\\\\"None\\\\\\\", empty string (\\\\\\\"\\\\\\\"), or a valid IP address. \\\\\\\"None\\\\\\\" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\\\",\\\"type\\\":\\\"string\\\"},\\\"externalIPs\\\":{\\\"description\\\":\\\"externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"externalName\\\":{\\\"description\\\":\\\"externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName.\\\",\\\"type\\\":\\\"string\\\"},\\\"externalTrafficPolicy\\\":{\\\"description\\\":\\\"externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. \\\\\\\"Local\\\\\\\" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. \\\\\\\"Cluster\\\\\\\" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading.\\\",\\\"type\\\":\\\"string\\\"},\\\"healthCheckNodePort\\\":{\\\"description\\\":\\\"healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"ipFamily\\\":{\\\"description\\\":\\\"ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment.\\\",\\\"type\\\":\\\"string\\\"},\\\"loadBalancerIP\\\":{\\\"description\\\":\\\"Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.\\\",\\\"type\\\":\\\"string\\\"},\\\"loadBalancerSourceRanges\\\":{\\\"description\\\":\\\"If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature.\\\\\\\" More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"ports\\\":{\\\"description\\\":\\\"The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\\\",\\\"items\\\":{\\\"description\\\":\\\"ServicePort contains information on service's port.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service.\\\",\\\"type\\\":\\\"string\\\"},\\\"nodePort\\\":{\\\"description\\\":\\\"The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"port\\\":{\\\"description\\\":\\\"The port that will be exposed by this service.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"protocol\\\":{\\\"description\\\":\\\"The IP protocol for this port. Supports \\\\\\\"TCP\\\\\\\", \\\\\\\"UDP\\\\\\\", and \\\\\\\"SCTP\\\\\\\". Default is TCP.\\\",\\\"type\\\":\\\"string\\\"},\\\"targetPort\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod's container ports. If this is not specified, the value of the 'port' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the 'port' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"publishNotReadyAddresses\\\":{\\\"description\\\":\\\"publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"selector\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/\\\",\\\"type\\\":\\\"object\\\"},\\\"sessionAffinity\\\":{\\\"description\\\":\\\"Supports \\\\\\\"ClientIP\\\\\\\" and \\\\\\\"None\\\\\\\". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\\\",\\\"type\\\":\\\"string\\\"},\\\"sessionAffinityConfig\\\":{\\\"description\\\":\\\"sessionAffinityConfig contains the configurations of session affinity.\\\",\\\"properties\\\":{\\\"clientIP\\\":{\\\"description\\\":\\\"clientIP contains the configurations of Client IP based session affinity.\\\",\\\"properties\\\":{\\\"timeoutSeconds\\\":{\\\"description\\\":\\\"timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be \\\\u003e0 \\\\u0026\\\\u0026 \\\\u003c=86400(for 1 day) if ServiceAffinity == \\\\\\\"ClientIP\\\\\\\". Default value is 10800(for 3 hours).\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"topologyKeys\\\":{\\\"description\\\":\\\"topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value \\\\\\\"*\\\\\\\" may be used to mean \\\\\\\"any topology\\\\\\\". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"type\\\":{\\\"description\\\":\\\"type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. \\\\\\\"ExternalName\\\\\\\" maps to the specified externalName. \\\\\\\"ClusterIP\\\\\\\" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is \\\\\\\"None\\\\\\\", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. \\\\\\\"NodePort\\\\\\\" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. \\\\\\\"LoadBalancer\\\\\\\" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"tls\\\":{\\\"description\\\":\\\"TLS defines options for configuring TLS for HTTP.\\\",\\\"properties\\\":{\\\"certificate\\\":{\\\"description\\\":\\\"Certificate is a reference to a Kubernetes secret that contains the certificate and private key for enabling TLS. The referenced secret should contain the following: \\\\n - `ca.crt`: The certificate authority (optional). - `tls.crt`: The certificate (or a chain). - `tls.key`: The private key to the first certificate in the certificate chain.\\\",\\\"properties\\\":{\\\"secretName\\\":{\\\"description\\\":\\\"SecretName is the name of the secret.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"selfSignedCertificate\\\":{\\\"description\\\":\\\"SelfSignedCertificate allows configuring the self-signed certificate generated by the operator.\\\",\\\"properties\\\":{\\\"disabled\\\":{\\\"description\\\":\\\"Disabled indicates that the provisioning of the self-signed certificate should be disabled.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"subjectAltNames\\\":{\\\"description\\\":\\\"SubjectAlternativeNames is a list of SANs to include in the generated HTTP TLS certificate.\\\",\\\"items\\\":{\\\"description\\\":\\\"SubjectAlternativeName represents a SAN entry in a x509 certificate.\\\",\\\"properties\\\":{\\\"dns\\\":{\\\"description\\\":\\\"DNS is the DNS name of the subject.\\\",\\\"type\\\":\\\"string\\\"},\\\"ip\\\":{\\\"description\\\":\\\"IP is the IP address of the subject.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"image\\\":{\\\"description\\\":\\\"Image is the Elasticsearch Docker image to deploy.\\\",\\\"type\\\":\\\"string\\\"},\\\"nodeSets\\\":{\\\"description\\\":\\\"NodeSets allow specifying groups of Elasticsearch nodes sharing the same configuration and Pod templates. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-orchestration.html\\\",\\\"items\\\":{\\\"description\\\":\\\"NodeSet is the specification for a group of Elasticsearch nodes sharing the same configuration and a Pod template.\\\",\\\"properties\\\":{\\\"config\\\":{\\\"description\\\":\\\"Config holds the Elasticsearch configuration.\\\",\\\"type\\\":\\\"object\\\"},\\\"count\\\":{\\\"description\\\":\\\"Count of Elasticsearch nodes to deploy.\\\",\\\"format\\\":\\\"int32\\\",\\\"minimum\\\":1,\\\"type\\\":\\\"integer\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of this set of nodes. Becomes a part of the Elasticsearch node.name setting.\\\",\\\"maxLength\\\":23,\\\"pattern\\\":\\\"[a-zA-Z0-9-]+\\\",\\\"type\\\":\\\"string\\\"},\\\"podTemplate\\\":{\\\"description\\\":\\\"PodTemplate provides customisation options (labels, annotations, affinity rules, resource requests, and so on) for the Pods belonging to this NodeSet.\\\",\\\"type\\\":\\\"object\\\"},\\\"volumeClaimTemplates\\\":{\\\"description\\\":\\\"VolumeClaimTemplates is a list of persistent volume claims to be used by each Pod in this NodeSet. Every claim in this list must have a matching volumeMount in one of the containers defined in the PodTemplate. Items defined here take precedence over any default claims added by the operator with the same name. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-volume-claim-templates.html\\\",\\\"items\\\":{\\\"description\\\":\\\"PersistentVolumeClaim is a user's request for and claim to a persistent volume\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"metadata\\\":{\\\"description\\\":\\\"Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata\\\",\\\"type\\\":\\\"object\\\"},\\\"spec\\\":{\\\"description\\\":\\\"Spec defines the desired characteristics of a volume requested by a pod author. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims\\\",\\\"properties\\\":{\\\"accessModes\\\":{\\\"description\\\":\\\"AccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"dataSource\\\":{\\\"description\\\":\\\"This field requires the VolumeSnapshotDataSource alpha feature gate to be enabled and currently VolumeSnapshot is the only supported data source. If the provisioner can support VolumeSnapshot data source, it will create a new volume and data will be restored to the volume at the same time. If the provisioner does not support VolumeSnapshot data source, volume will not be created and the failure will be reported as an event. In the future, we plan to support more data source types and the behavior of the provisioner may change.\\\",\\\"properties\\\":{\\\"apiGroup\\\":{\\\"description\\\":\\\"APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is the type of resource being referenced\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name is the name of resource being referenced\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"kind\\\",\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"resources\\\":{\\\"description\\\":\\\"Resources represents the minimum resources the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"additionalProperties\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"pattern\\\":\\\"^(\\\\\\\\+|-)?(([0-9]+(\\\\\\\\.[0-9]*)?)|(\\\\\\\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\\\\\\\+|-)?(([0-9]+(\\\\\\\\.[0-9]*)?)|(\\\\\\\\.[0-9]+))))?$\\\"},\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"additionalProperties\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"pattern\\\":\\\"^(\\\\\\\\+|-)?(([0-9]+(\\\\\\\\.[0-9]*)?)|(\\\\\\\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\\\\\\\+|-)?(([0-9]+(\\\\\\\\.[0-9]*)?)|(\\\\\\\\.[0-9]+))))?$\\\"},\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"selector\\\":{\\\"description\\\":\\\"A label query over volumes to consider for binding.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"storageClassName\\\":{\\\"description\\\":\\\"Name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1\\\",\\\"type\\\":\\\"string\\\"},\\\"volumeMode\\\":{\\\"description\\\":\\\"volumeMode defines what type of volume is required by the claim. Value of Filesystem is implied when not included in claim spec. This is a beta feature.\\\",\\\"type\\\":\\\"string\\\"},\\\"volumeName\\\":{\\\"description\\\":\\\"VolumeName is the binding reference to the PersistentVolume backing this claim.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"status\\\":{\\\"description\\\":\\\"Status represents the current information/status of a persistent volume claim. Read-only. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims\\\",\\\"properties\\\":{\\\"accessModes\\\":{\\\"description\\\":\\\"AccessModes contains the actual access modes the volume backing the PVC has. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"capacity\\\":{\\\"additionalProperties\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"pattern\\\":\\\"^(\\\\\\\\+|-)?(([0-9]+(\\\\\\\\.[0-9]*)?)|(\\\\\\\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\\\\\\\+|-)?(([0-9]+(\\\\\\\\.[0-9]*)?)|(\\\\\\\\.[0-9]+))))?$\\\"},\\\"description\\\":\\\"Represents the actual resources of the underlying volume.\\\",\\\"type\\\":\\\"object\\\"},\\\"conditions\\\":{\\\"description\\\":\\\"Current Condition of persistent volume claim. If underlying persistent volume is being resized then the Condition will be set to 'ResizeStarted'.\\\",\\\"items\\\":{\\\"description\\\":\\\"PersistentVolumeClaimCondition contains details about state of pvc\\\",\\\"properties\\\":{\\\"lastProbeTime\\\":{\\\"description\\\":\\\"Last time we probed the condition.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"lastTransitionTime\\\":{\\\"description\\\":\\\"Last time the condition transitioned from one status to another.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"Human-readable message indicating details about last transition.\\\",\\\"type\\\":\\\"string\\\"},\\\"reason\\\":{\\\"description\\\":\\\"Unique, this should be a short, machine understandable string that gives the reason for condition's last transition. If it reports \\\\\\\"ResizeStarted\\\\\\\" that means the underlying persistent volume is being resized.\\\",\\\"type\\\":\\\"string\\\"},\\\"status\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":{\\\"description\\\":\\\"PersistentVolumeClaimConditionType is a valid value of PersistentVolumeClaimCondition.Type\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"status\\\",\\\"type\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"phase\\\":{\\\"description\\\":\\\"Phase represents the current phase of PersistentVolumeClaim.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"count\\\",\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"minItems\\\":1,\\\"type\\\":\\\"array\\\"},\\\"podDisruptionBudget\\\":{\\\"description\\\":\\\"PodDisruptionBudget provides access to the default pod disruption budget for the Elasticsearch cluster. The default budget selects all cluster pods and sets `maxUnavailable` to 1. To disable, set `PodDisruptionBudget` to the empty value (`{}` in YAML).\\\",\\\"properties\\\":{\\\"metadata\\\":{\\\"description\\\":\\\"ObjectMeta is the metadata of the PDB. The name and namespace provided here are managed by ECK and will be ignored.\\\",\\\"type\\\":\\\"object\\\"},\\\"spec\\\":{\\\"description\\\":\\\"Spec is the specification of the PDB.\\\",\\\"properties\\\":{\\\"maxUnavailable\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"An eviction is allowed if at most \\\\\\\"maxUnavailable\\\\\\\" pods selected by \\\\\\\"selector\\\\\\\" are unavailable after the eviction, i.e. even in absence of the evicted pod. For example, one can prevent all voluntary evictions by specifying 0. This is a mutually exclusive setting with \\\\\\\"minAvailable\\\\\\\".\\\"},\\\"minAvailable\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"An eviction is allowed if at least \\\\\\\"minAvailable\\\\\\\" pods selected by \\\\\\\"selector\\\\\\\" will still be available after the eviction, i.e. even in the absence of the evicted pod. So for example you can prevent all voluntary evictions by specifying \\\\\\\"100%\\\\\\\".\\\"},\\\"selector\\\":{\\\"description\\\":\\\"Label query over pods whose evictions are managed by the disruption budget.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"remoteClusters\\\":{\\\"description\\\":\\\"RemoteClusters enables you to establish uni-directional connections to a remote Elasticsearch cluster.\\\",\\\"items\\\":{\\\"description\\\":\\\"RemoteCluster declares a remote Elasticsearch cluster connection.\\\",\\\"properties\\\":{\\\"elasticsearchRef\\\":{\\\"description\\\":\\\"ElasticsearchRef is a reference to an Elasticsearch cluster running within the same k8s cluster.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the Kubernetes object.\\\",\\\"type\\\":\\\"string\\\"},\\\"namespace\\\":{\\\"description\\\":\\\"Namespace of the Kubernetes object. If empty, defaults to the current namespace.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name is the name of the remote cluster as it is set in the Elasticsearch settings. The name is expected to be unique for each remote clusters.\\\",\\\"minLength\\\":1,\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"secureSettings\\\":{\\\"description\\\":\\\"SecureSettings is a list of references to Kubernetes secrets containing sensitive configuration options for Elasticsearch. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-es-secure-settings.html\\\",\\\"items\\\":{\\\"description\\\":\\\"SecretSource defines a data source based on a Kubernetes Secret.\\\",\\\"properties\\\":{\\\"entries\\\":{\\\"description\\\":\\\"Entries define how to project each key-value pair in the secret to filesystem paths. If not defined, all keys will be projected to similarly named paths in the filesystem. If defined, only the specified keys will be projected to the corresponding paths.\\\",\\\"items\\\":{\\\"description\\\":\\\"KeyToPath defines how to map a key in a Secret object to a filesystem path.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"Key is the key contained in the secret.\\\",\\\"type\\\":\\\"string\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path is the relative file path to map the key to. Path must not be an absolute file path and must not contain any \\\\\\\"..\\\\\\\" components.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"secretName\\\":{\\\"description\\\":\\\"SecretName is the name of the secret.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"secretName\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"serviceAccountName\\\":{\\\"description\\\":\\\"ServiceAccountName is used to check access from the current resource to a resource (eg. a remote Elasticsearch cluster) in a different namespace. Can only be used if ECK is enforcing RBAC on references.\\\",\\\"type\\\":\\\"string\\\"},\\\"transport\\\":{\\\"description\\\":\\\"Transport holds transport layer settings for Elasticsearch.\\\",\\\"properties\\\":{\\\"service\\\":{\\\"description\\\":\\\"Service defines the template for the associated Kubernetes Service object.\\\",\\\"properties\\\":{\\\"metadata\\\":{\\\"description\\\":\\\"ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored.\\\",\\\"type\\\":\\\"object\\\"},\\\"spec\\\":{\\\"description\\\":\\\"Spec is the specification of the service.\\\",\\\"properties\\\":{\\\"clusterIP\\\":{\\\"description\\\":\\\"clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are \\\\\\\"None\\\\\\\", empty string (\\\\\\\"\\\\\\\"), or a valid IP address. \\\\\\\"None\\\\\\\" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\\\",\\\"type\\\":\\\"string\\\"},\\\"externalIPs\\\":{\\\"description\\\":\\\"externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"externalName\\\":{\\\"description\\\":\\\"externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName.\\\",\\\"type\\\":\\\"string\\\"},\\\"externalTrafficPolicy\\\":{\\\"description\\\":\\\"externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. \\\\\\\"Local\\\\\\\" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. \\\\\\\"Cluster\\\\\\\" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading.\\\",\\\"type\\\":\\\"string\\\"},\\\"healthCheckNodePort\\\":{\\\"description\\\":\\\"healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"ipFamily\\\":{\\\"description\\\":\\\"ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment.\\\",\\\"type\\\":\\\"string\\\"},\\\"loadBalancerIP\\\":{\\\"description\\\":\\\"Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.\\\",\\\"type\\\":\\\"string\\\"},\\\"loadBalancerSourceRanges\\\":{\\\"description\\\":\\\"If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature.\\\\\\\" More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"ports\\\":{\\\"description\\\":\\\"The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\\\",\\\"items\\\":{\\\"description\\\":\\\"ServicePort contains information on service's port.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service.\\\",\\\"type\\\":\\\"string\\\"},\\\"nodePort\\\":{\\\"description\\\":\\\"The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"port\\\":{\\\"description\\\":\\\"The port that will be exposed by this service.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"protocol\\\":{\\\"description\\\":\\\"The IP protocol for this port. Supports \\\\\\\"TCP\\\\\\\", \\\\\\\"UDP\\\\\\\", and \\\\\\\"SCTP\\\\\\\". Default is TCP.\\\",\\\"type\\\":\\\"string\\\"},\\\"targetPort\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod's container ports. If this is not specified, the value of the 'port' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the 'port' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"publishNotReadyAddresses\\\":{\\\"description\\\":\\\"publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"selector\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/\\\",\\\"type\\\":\\\"object\\\"},\\\"sessionAffinity\\\":{\\\"description\\\":\\\"Supports \\\\\\\"ClientIP\\\\\\\" and \\\\\\\"None\\\\\\\". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\\\",\\\"type\\\":\\\"string\\\"},\\\"sessionAffinityConfig\\\":{\\\"description\\\":\\\"sessionAffinityConfig contains the configurations of session affinity.\\\",\\\"properties\\\":{\\\"clientIP\\\":{\\\"description\\\":\\\"clientIP contains the configurations of Client IP based session affinity.\\\",\\\"properties\\\":{\\\"timeoutSeconds\\\":{\\\"description\\\":\\\"timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be \\\\u003e0 \\\\u0026\\\\u0026 \\\\u003c=86400(for 1 day) if ServiceAffinity == \\\\\\\"ClientIP\\\\\\\". Default value is 10800(for 3 hours).\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"topologyKeys\\\":{\\\"description\\\":\\\"topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value \\\\\\\"*\\\\\\\" may be used to mean \\\\\\\"any topology\\\\\\\". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"type\\\":{\\\"description\\\":\\\"type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. \\\\\\\"ExternalName\\\\\\\" maps to the specified externalName. \\\\\\\"ClusterIP\\\\\\\" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is \\\\\\\"None\\\\\\\", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. \\\\\\\"NodePort\\\\\\\" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. \\\\\\\"LoadBalancer\\\\\\\" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"updateStrategy\\\":{\\\"description\\\":\\\"UpdateStrategy specifies how updates to the cluster should be performed.\\\",\\\"properties\\\":{\\\"changeBudget\\\":{\\\"description\\\":\\\"ChangeBudget defines the constraints to consider when applying changes to the Elasticsearch cluster.\\\",\\\"properties\\\":{\\\"maxSurge\\\":{\\\"description\\\":\\\"MaxSurge is the maximum number of new pods that can be created exceeding the original number of pods defined in the specification. MaxSurge is only taken into consideration when scaling up. Setting a negative value will disable the restriction. Defaults to unbounded if not specified.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"maxUnavailable\\\":{\\\"description\\\":\\\"MaxUnavailable is the maximum number of pods that can be unavailable (not ready) during the update due to circumstances under the control of the operator. Setting a negative value will disable this restriction. Defaults to 1 if not specified.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"version\\\":{\\\"description\\\":\\\"Version of Elasticsearch.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"nodeSets\\\",\\\"version\\\"],\\\"type\\\":\\\"object\\\"},\\\"status\\\":{\\\"description\\\":\\\"ElasticsearchStatus defines the observed state of Elasticsearch\\\",\\\"properties\\\":{\\\"availableNodes\\\":{\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"health\\\":{\\\"description\\\":\\\"ElasticsearchHealth is the health of the cluster as returned by the health API.\\\",\\\"type\\\":\\\"string\\\"},\\\"phase\\\":{\\\"description\\\":\\\"ElasticsearchOrchestrationPhase is the phase Elasticsearch is in from the controller point of view.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}}}},\\\"version\\\":\\\"v1\\\",\\\"versions\\\":[{\\\"name\\\":\\\"v1\\\",\\\"served\\\":true,\\\"storage\\\":true},{\\\"name\\\":\\\"v1beta1\\\",\\\"served\\\":true,\\\"storage\\\":false},{\\\"name\\\":\\\"v1alpha1\\\",\\\"served\\\":false,\\\"storage\\\":false}]},\\\"status\\\":{\\\"acceptedNames\\\":{\\\"kind\\\":\\\"\\\",\\\"plural\\\":\\\"\\\"},\\\"conditions\\\":[],\\\"storedVersions\\\":[]}}\\n\"\n },\n \"creationTimestamp\": \"2020-04-28T23:31:51Z\",\n \"generation\": 1,\n \"labels\": {\n \"velero.io/backup-name\": \"es\",\n \"velero.io/restore-name\": \"es-crds\"\n },\n \"name\": \"elasticsearches.elasticsearch.k8s.elastic.co\",\n \"resourceVersion\": \"1703536\",\n \"selfLink\": \"/apis/apiextensions.k8s.io/v1/customresourcedefinitions/elasticsearches.elasticsearch.k8s.elastic.co\",\n \"uid\": \"e8596856-29ae-47e4-8b14-5f7f027adf4a\"\n },\n \"spec\": {\n \"conversion\": {\n \"strategy\": \"None\"\n },\n \"group\": \"elasticsearch.k8s.elastic.co\",\n \"names\": {\n \"categories\": [\n \"elastic\"\n ],\n \"kind\": \"Elasticsearch\",\n \"listKind\": \"ElasticsearchList\",\n \"plural\": \"elasticsearches\",\n \"shortNames\": [\n \"es\"\n ],\n \"singular\": \"elasticsearch\"\n },\n \"preserveUnknownFields\": true,\n \"scope\": \"Namespaced\",\n \"versions\": [\n {\n \"additionalPrinterColumns\": [\n {\n \"jsonPath\": \".status.health\",\n \"name\": \"health\",\n \"type\": \"string\"\n },\n {\n \"description\": \"Available nodes\",\n \"jsonPath\": \".status.availableNodes\",\n \"name\": \"nodes\",\n \"type\": \"integer\"\n },\n {\n \"description\": \"Elasticsearch version\",\n \"jsonPath\": \".spec.version\",\n \"name\": \"version\",\n \"type\": \"string\"\n },\n {\n \"jsonPath\": \".status.phase\",\n \"name\": \"phase\",\n \"type\": \"string\"\n },\n {\n \"jsonPath\": \".metadata.creationTimestamp\",\n \"name\": \"age\",\n \"type\": \"date\"\n }\n ],\n \"name\": \"v1\",\n \"schema\": {\n \"openAPIV3Schema\": {\n \"description\": \"Elasticsearch represents an Elasticsearch resource in a Kubernetes cluster.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"ElasticsearchSpec holds the specification of an Elasticsearch cluster.\",\n \"properties\": {\n \"auth\": {\n \"description\": \"Auth contains user authentication and authorization security settings for Elasticsearch.\",\n \"properties\": {\n \"fileRealm\": {\n \"description\": \"FileRealm to propagate to the Elasticsearch cluster.\",\n \"items\": {\n \"description\": \"FileRealmSource references users to create in the Elasticsearch cluster.\",\n \"properties\": {\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"roles\": {\n \"description\": \"Roles to propagate to the Elasticsearch cluster.\",\n \"items\": {\n \"description\": \"RoleSource references roles to create in the Elasticsearch cluster.\",\n \"properties\": {\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"http\": {\n \"description\": \"HTTP holds HTTP layer settings for Elasticsearch.\",\n \"properties\": {\n \"service\": {\n \"description\": \"Service defines the template for the associated Kubernetes Service object.\",\n \"properties\": {\n \"metadata\": {\n \"description\": \"ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored.\",\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec is the specification of the service.\",\n \"properties\": {\n \"clusterIP\": {\n \"description\": \"clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are \\\"None\\\", empty string (\\\"\\\"), or a valid IP address. \\\"None\\\" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"externalIPs\": {\n \"description\": \"externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"externalName\": {\n \"description\": \"externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName.\",\n \"type\": \"string\"\n },\n \"externalTrafficPolicy\": {\n \"description\": \"externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. \\\"Local\\\" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. \\\"Cluster\\\" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading.\",\n \"type\": \"string\"\n },\n \"healthCheckNodePort\": {\n \"description\": \"healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"ipFamily\": {\n \"description\": \"ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment.\",\n \"type\": \"string\"\n },\n \"loadBalancerIP\": {\n \"description\": \"Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.\",\n \"type\": \"string\"\n },\n \"loadBalancerSourceRanges\": {\n \"description\": \"If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature.\\\" More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"ports\": {\n \"description\": \"The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"items\": {\n \"description\": \"ServicePort contains information on service's port.\",\n \"properties\": {\n \"name\": {\n \"description\": \"The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service.\",\n \"type\": \"string\"\n },\n \"nodePort\": {\n \"description\": \"The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"port\": {\n \"description\": \"The port that will be exposed by this service.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"protocol\": {\n \"description\": \"The IP protocol for this port. Supports \\\"TCP\\\", \\\"UDP\\\", and \\\"SCTP\\\". Default is TCP.\",\n \"type\": \"string\"\n },\n \"targetPort\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod's container ports. If this is not specified, the value of the 'port' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the 'port' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"publishNotReadyAddresses\": {\n \"description\": \"publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery.\",\n \"type\": \"boolean\"\n },\n \"selector\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/\",\n \"type\": \"object\"\n },\n \"sessionAffinity\": {\n \"description\": \"Supports \\\"ClientIP\\\" and \\\"None\\\". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"sessionAffinityConfig\": {\n \"description\": \"sessionAffinityConfig contains the configurations of session affinity.\",\n \"properties\": {\n \"clientIP\": {\n \"description\": \"clientIP contains the configurations of Client IP based session affinity.\",\n \"properties\": {\n \"timeoutSeconds\": {\n \"description\": \"timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be \\u003e0 \\u0026\\u0026 \\u003c=86400(for 1 day) if ServiceAffinity == \\\"ClientIP\\\". Default value is 10800(for 3 hours).\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"topologyKeys\": {\n \"description\": \"topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value \\\"*\\\" may be used to mean \\\"any topology\\\". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"type\": {\n \"description\": \"type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. \\\"ExternalName\\\" maps to the specified externalName. \\\"ClusterIP\\\" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is \\\"None\\\", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. \\\"NodePort\\\" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. \\\"LoadBalancer\\\" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"tls\": {\n \"description\": \"TLS defines options for configuring TLS for HTTP.\",\n \"properties\": {\n \"certificate\": {\n \"description\": \"Certificate is a reference to a Kubernetes secret that contains the certificate and private key for enabling TLS. The referenced secret should contain the following: \\n - `ca.crt`: The certificate authority (optional). - `tls.crt`: The certificate (or a chain). - `tls.key`: The private key to the first certificate in the certificate chain.\",\n \"properties\": {\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"selfSignedCertificate\": {\n \"description\": \"SelfSignedCertificate allows configuring the self-signed certificate generated by the operator.\",\n \"properties\": {\n \"disabled\": {\n \"description\": \"Disabled indicates that the provisioning of the self-signed certificate should be disabled.\",\n \"type\": \"boolean\"\n },\n \"subjectAltNames\": {\n \"description\": \"SubjectAlternativeNames is a list of SANs to include in the generated HTTP TLS certificate.\",\n \"items\": {\n \"description\": \"SubjectAlternativeName represents a SAN entry in a x509 certificate.\",\n \"properties\": {\n \"dns\": {\n \"description\": \"DNS is the DNS name of the subject.\",\n \"type\": \"string\"\n },\n \"ip\": {\n \"description\": \"IP is the IP address of the subject.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"image\": {\n \"description\": \"Image is the Elasticsearch Docker image to deploy.\",\n \"type\": \"string\"\n },\n \"nodeSets\": {\n \"description\": \"NodeSets allow specifying groups of Elasticsearch nodes sharing the same configuration and Pod templates. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-orchestration.html\",\n \"items\": {\n \"description\": \"NodeSet is the specification for a group of Elasticsearch nodes sharing the same configuration and a Pod template.\",\n \"properties\": {\n \"config\": {\n \"description\": \"Config holds the Elasticsearch configuration.\",\n \"type\": \"object\"\n },\n \"count\": {\n \"description\": \"Count of Elasticsearch nodes to deploy.\",\n \"format\": \"int32\",\n \"minimum\": 1,\n \"type\": \"integer\"\n },\n \"name\": {\n \"description\": \"Name of this set of nodes. Becomes a part of the Elasticsearch node.name setting.\",\n \"maxLength\": 23,\n \"pattern\": \"[a-zA-Z0-9-]+\",\n \"type\": \"string\"\n },\n \"podTemplate\": {\n \"description\": \"PodTemplate provides customisation options (labels, annotations, affinity rules, resource requests, and so on) for the Pods belonging to this NodeSet.\",\n \"type\": \"object\"\n },\n \"volumeClaimTemplates\": {\n \"description\": \"VolumeClaimTemplates is a list of persistent volume claims to be used by each Pod in this NodeSet. Every claim in this list must have a matching volumeMount in one of the containers defined in the PodTemplate. Items defined here take precedence over any default claims added by the operator with the same name. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-volume-claim-templates.html\",\n \"items\": {\n \"description\": \"PersistentVolumeClaim is a user's request for and claim to a persistent volume\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"description\": \"Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata\",\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec defines the desired characteristics of a volume requested by a pod author. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims\",\n \"properties\": {\n \"accessModes\": {\n \"description\": \"AccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"dataSource\": {\n \"description\": \"This field requires the VolumeSnapshotDataSource alpha feature gate to be enabled and currently VolumeSnapshot is the only supported data source. If the provisioner can support VolumeSnapshot data source, it will create a new volume and data will be restored to the volume at the same time. If the provisioner does not support VolumeSnapshot data source, volume will not be created and the failure will be reported as an event. In the future, we plan to support more data source types and the behavior of the provisioner may change.\",\n \"properties\": {\n \"apiGroup\": {\n \"description\": \"APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is the type of resource being referenced\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name is the name of resource being referenced\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"kind\",\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"resources\": {\n \"description\": \"Resources represents the minimum resources the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources\",\n \"properties\": {\n \"limits\": {\n \"additionalProperties\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"pattern\": \"^(\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))))?$\"\n },\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"additionalProperties\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"pattern\": \"^(\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))))?$\"\n },\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"selector\": {\n \"description\": \"A label query over volumes to consider for binding.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"storageClassName\": {\n \"description\": \"Name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1\",\n \"type\": \"string\"\n },\n \"volumeMode\": {\n \"description\": \"volumeMode defines what type of volume is required by the claim. Value of Filesystem is implied when not included in claim spec. This is a beta feature.\",\n \"type\": \"string\"\n },\n \"volumeName\": {\n \"description\": \"VolumeName is the binding reference to the PersistentVolume backing this claim.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"status\": {\n \"description\": \"Status represents the current information/status of a persistent volume claim. Read-only. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims\",\n \"properties\": {\n \"accessModes\": {\n \"description\": \"AccessModes contains the actual access modes the volume backing the PVC has. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"capacity\": {\n \"additionalProperties\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"pattern\": \"^(\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))))?$\"\n },\n \"description\": \"Represents the actual resources of the underlying volume.\",\n \"type\": \"object\"\n },\n \"conditions\": {\n \"description\": \"Current Condition of persistent volume claim. If underlying persistent volume is being resized then the Condition will be set to 'ResizeStarted'.\",\n \"items\": {\n \"description\": \"PersistentVolumeClaimCondition contains details about state of pvc\",\n \"properties\": {\n \"lastProbeTime\": {\n \"description\": \"Last time we probed the condition.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n },\n \"lastTransitionTime\": {\n \"description\": \"Last time the condition transitioned from one status to another.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"Human-readable message indicating details about last transition.\",\n \"type\": \"string\"\n },\n \"reason\": {\n \"description\": \"Unique, this should be a short, machine understandable string that gives the reason for condition's last transition. If it reports \\\"ResizeStarted\\\" that means the underlying persistent volume is being resized.\",\n \"type\": \"string\"\n },\n \"status\": {\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"PersistentVolumeClaimConditionType is a valid value of PersistentVolumeClaimCondition.Type\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"status\",\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"phase\": {\n \"description\": \"Phase represents the current phase of PersistentVolumeClaim.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"count\",\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"minItems\": 1,\n \"type\": \"array\"\n },\n \"podDisruptionBudget\": {\n \"description\": \"PodDisruptionBudget provides access to the default pod disruption budget for the Elasticsearch cluster. The default budget selects all cluster pods and sets `maxUnavailable` to 1. To disable, set `PodDisruptionBudget` to the empty value (`{}` in YAML).\",\n \"properties\": {\n \"metadata\": {\n \"description\": \"ObjectMeta is the metadata of the PDB. The name and namespace provided here are managed by ECK and will be ignored.\",\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec is the specification of the PDB.\",\n \"properties\": {\n \"maxUnavailable\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"An eviction is allowed if at most \\\"maxUnavailable\\\" pods selected by \\\"selector\\\" are unavailable after the eviction, i.e. even in absence of the evicted pod. For example, one can prevent all voluntary evictions by specifying 0. This is a mutually exclusive setting with \\\"minAvailable\\\".\"\n },\n \"minAvailable\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"An eviction is allowed if at least \\\"minAvailable\\\" pods selected by \\\"selector\\\" will still be available after the eviction, i.e. even in the absence of the evicted pod. So for example you can prevent all voluntary evictions by specifying \\\"100%\\\".\"\n },\n \"selector\": {\n \"description\": \"Label query over pods whose evictions are managed by the disruption budget.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"remoteClusters\": {\n \"description\": \"RemoteClusters enables you to establish uni-directional connections to a remote Elasticsearch cluster.\",\n \"items\": {\n \"description\": \"RemoteCluster declares a remote Elasticsearch cluster connection.\",\n \"properties\": {\n \"elasticsearchRef\": {\n \"description\": \"ElasticsearchRef is a reference to an Elasticsearch cluster running within the same k8s cluster.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the Kubernetes object.\",\n \"type\": \"string\"\n },\n \"namespace\": {\n \"description\": \"Namespace of the Kubernetes object. If empty, defaults to the current namespace.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"name\": {\n \"description\": \"Name is the name of the remote cluster as it is set in the Elasticsearch settings. The name is expected to be unique for each remote clusters.\",\n \"minLength\": 1,\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"secureSettings\": {\n \"description\": \"SecureSettings is a list of references to Kubernetes secrets containing sensitive configuration options for Elasticsearch. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-es-secure-settings.html\",\n \"items\": {\n \"description\": \"SecretSource defines a data source based on a Kubernetes Secret.\",\n \"properties\": {\n \"entries\": {\n \"description\": \"Entries define how to project each key-value pair in the secret to filesystem paths. If not defined, all keys will be projected to similarly named paths in the filesystem. If defined, only the specified keys will be projected to the corresponding paths.\",\n \"items\": {\n \"description\": \"KeyToPath defines how to map a key in a Secret object to a filesystem path.\",\n \"properties\": {\n \"key\": {\n \"description\": \"Key is the key contained in the secret.\",\n \"type\": \"string\"\n },\n \"path\": {\n \"description\": \"Path is the relative file path to map the key to. Path must not be an absolute file path and must not contain any \\\"..\\\" components.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"secretName\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"serviceAccountName\": {\n \"description\": \"ServiceAccountName is used to check access from the current resource to a resource (eg. a remote Elasticsearch cluster) in a different namespace. Can only be used if ECK is enforcing RBAC on references.\",\n \"type\": \"string\"\n },\n \"transport\": {\n \"description\": \"Transport holds transport layer settings for Elasticsearch.\",\n \"properties\": {\n \"service\": {\n \"description\": \"Service defines the template for the associated Kubernetes Service object.\",\n \"properties\": {\n \"metadata\": {\n \"description\": \"ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored.\",\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec is the specification of the service.\",\n \"properties\": {\n \"clusterIP\": {\n \"description\": \"clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are \\\"None\\\", empty string (\\\"\\\"), or a valid IP address. \\\"None\\\" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"externalIPs\": {\n \"description\": \"externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"externalName\": {\n \"description\": \"externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName.\",\n \"type\": \"string\"\n },\n \"externalTrafficPolicy\": {\n \"description\": \"externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. \\\"Local\\\" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. \\\"Cluster\\\" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading.\",\n \"type\": \"string\"\n },\n \"healthCheckNodePort\": {\n \"description\": \"healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"ipFamily\": {\n \"description\": \"ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment.\",\n \"type\": \"string\"\n },\n \"loadBalancerIP\": {\n \"description\": \"Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.\",\n \"type\": \"string\"\n },\n \"loadBalancerSourceRanges\": {\n \"description\": \"If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature.\\\" More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"ports\": {\n \"description\": \"The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"items\": {\n \"description\": \"ServicePort contains information on service's port.\",\n \"properties\": {\n \"name\": {\n \"description\": \"The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service.\",\n \"type\": \"string\"\n },\n \"nodePort\": {\n \"description\": \"The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"port\": {\n \"description\": \"The port that will be exposed by this service.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"protocol\": {\n \"description\": \"The IP protocol for this port. Supports \\\"TCP\\\", \\\"UDP\\\", and \\\"SCTP\\\". Default is TCP.\",\n \"type\": \"string\"\n },\n \"targetPort\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod's container ports. If this is not specified, the value of the 'port' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the 'port' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"publishNotReadyAddresses\": {\n \"description\": \"publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery.\",\n \"type\": \"boolean\"\n },\n \"selector\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/\",\n \"type\": \"object\"\n },\n \"sessionAffinity\": {\n \"description\": \"Supports \\\"ClientIP\\\" and \\\"None\\\". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"sessionAffinityConfig\": {\n \"description\": \"sessionAffinityConfig contains the configurations of session affinity.\",\n \"properties\": {\n \"clientIP\": {\n \"description\": \"clientIP contains the configurations of Client IP based session affinity.\",\n \"properties\": {\n \"timeoutSeconds\": {\n \"description\": \"timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be \\u003e0 \\u0026\\u0026 \\u003c=86400(for 1 day) if ServiceAffinity == \\\"ClientIP\\\". Default value is 10800(for 3 hours).\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"topologyKeys\": {\n \"description\": \"topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value \\\"*\\\" may be used to mean \\\"any topology\\\". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"type\": {\n \"description\": \"type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. \\\"ExternalName\\\" maps to the specified externalName. \\\"ClusterIP\\\" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is \\\"None\\\", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. \\\"NodePort\\\" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. \\\"LoadBalancer\\\" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"updateStrategy\": {\n \"description\": \"UpdateStrategy specifies how updates to the cluster should be performed.\",\n \"properties\": {\n \"changeBudget\": {\n \"description\": \"ChangeBudget defines the constraints to consider when applying changes to the Elasticsearch cluster.\",\n \"properties\": {\n \"maxSurge\": {\n \"description\": \"MaxSurge is the maximum number of new pods that can be created exceeding the original number of pods defined in the specification. MaxSurge is only taken into consideration when scaling up. Setting a negative value will disable the restriction. Defaults to unbounded if not specified.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"maxUnavailable\": {\n \"description\": \"MaxUnavailable is the maximum number of pods that can be unavailable (not ready) during the update due to circumstances under the control of the operator. Setting a negative value will disable this restriction. Defaults to 1 if not specified.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"version\": {\n \"description\": \"Version of Elasticsearch.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"nodeSets\",\n \"version\"\n ],\n \"type\": \"object\"\n },\n \"status\": {\n \"description\": \"ElasticsearchStatus defines the observed state of Elasticsearch\",\n \"properties\": {\n \"availableNodes\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"health\": {\n \"description\": \"ElasticsearchHealth is the health of the cluster as returned by the health API.\",\n \"type\": \"string\"\n },\n \"phase\": {\n \"description\": \"ElasticsearchOrchestrationPhase is the phase Elasticsearch is in from the controller point of view.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n }\n }\n },\n \"served\": true,\n \"storage\": true,\n \"subresources\": {\n \"status\": {}\n }\n },\n {\n \"additionalPrinterColumns\": [\n {\n \"jsonPath\": \".status.health\",\n \"name\": \"health\",\n \"type\": \"string\"\n },\n {\n \"description\": \"Available nodes\",\n \"jsonPath\": \".status.availableNodes\",\n \"name\": \"nodes\",\n \"type\": \"integer\"\n },\n {\n \"description\": \"Elasticsearch version\",\n \"jsonPath\": \".spec.version\",\n \"name\": \"version\",\n \"type\": \"string\"\n },\n {\n \"jsonPath\": \".status.phase\",\n \"name\": \"phase\",\n \"type\": \"string\"\n },\n {\n \"jsonPath\": \".metadata.creationTimestamp\",\n \"name\": \"age\",\n \"type\": \"date\"\n }\n ],\n \"name\": \"v1beta1\",\n \"schema\": {\n \"openAPIV3Schema\": {\n \"description\": \"Elasticsearch represents an Elasticsearch resource in a Kubernetes cluster.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"ElasticsearchSpec holds the specification of an Elasticsearch cluster.\",\n \"properties\": {\n \"auth\": {\n \"description\": \"Auth contains user authentication and authorization security settings for Elasticsearch.\",\n \"properties\": {\n \"fileRealm\": {\n \"description\": \"FileRealm to propagate to the Elasticsearch cluster.\",\n \"items\": {\n \"description\": \"FileRealmSource references users to create in the Elasticsearch cluster.\",\n \"properties\": {\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"roles\": {\n \"description\": \"Roles to propagate to the Elasticsearch cluster.\",\n \"items\": {\n \"description\": \"RoleSource references roles to create in the Elasticsearch cluster.\",\n \"properties\": {\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"http\": {\n \"description\": \"HTTP holds HTTP layer settings for Elasticsearch.\",\n \"properties\": {\n \"service\": {\n \"description\": \"Service defines the template for the associated Kubernetes Service object.\",\n \"properties\": {\n \"metadata\": {\n \"description\": \"ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored.\",\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec is the specification of the service.\",\n \"properties\": {\n \"clusterIP\": {\n \"description\": \"clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are \\\"None\\\", empty string (\\\"\\\"), or a valid IP address. \\\"None\\\" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"externalIPs\": {\n \"description\": \"externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"externalName\": {\n \"description\": \"externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName.\",\n \"type\": \"string\"\n },\n \"externalTrafficPolicy\": {\n \"description\": \"externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. \\\"Local\\\" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. \\\"Cluster\\\" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading.\",\n \"type\": \"string\"\n },\n \"healthCheckNodePort\": {\n \"description\": \"healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"ipFamily\": {\n \"description\": \"ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment.\",\n \"type\": \"string\"\n },\n \"loadBalancerIP\": {\n \"description\": \"Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.\",\n \"type\": \"string\"\n },\n \"loadBalancerSourceRanges\": {\n \"description\": \"If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature.\\\" More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"ports\": {\n \"description\": \"The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"items\": {\n \"description\": \"ServicePort contains information on service's port.\",\n \"properties\": {\n \"name\": {\n \"description\": \"The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service.\",\n \"type\": \"string\"\n },\n \"nodePort\": {\n \"description\": \"The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"port\": {\n \"description\": \"The port that will be exposed by this service.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"protocol\": {\n \"description\": \"The IP protocol for this port. Supports \\\"TCP\\\", \\\"UDP\\\", and \\\"SCTP\\\". Default is TCP.\",\n \"type\": \"string\"\n },\n \"targetPort\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod's container ports. If this is not specified, the value of the 'port' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the 'port' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"publishNotReadyAddresses\": {\n \"description\": \"publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery.\",\n \"type\": \"boolean\"\n },\n \"selector\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/\",\n \"type\": \"object\"\n },\n \"sessionAffinity\": {\n \"description\": \"Supports \\\"ClientIP\\\" and \\\"None\\\". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"sessionAffinityConfig\": {\n \"description\": \"sessionAffinityConfig contains the configurations of session affinity.\",\n \"properties\": {\n \"clientIP\": {\n \"description\": \"clientIP contains the configurations of Client IP based session affinity.\",\n \"properties\": {\n \"timeoutSeconds\": {\n \"description\": \"timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be \\u003e0 \\u0026\\u0026 \\u003c=86400(for 1 day) if ServiceAffinity == \\\"ClientIP\\\". Default value is 10800(for 3 hours).\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"topologyKeys\": {\n \"description\": \"topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value \\\"*\\\" may be used to mean \\\"any topology\\\". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"type\": {\n \"description\": \"type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. \\\"ExternalName\\\" maps to the specified externalName. \\\"ClusterIP\\\" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is \\\"None\\\", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. \\\"NodePort\\\" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. \\\"LoadBalancer\\\" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"tls\": {\n \"description\": \"TLS defines options for configuring TLS for HTTP.\",\n \"properties\": {\n \"certificate\": {\n \"description\": \"Certificate is a reference to a Kubernetes secret that contains the certificate and private key for enabling TLS. The referenced secret should contain the following: \\n - `ca.crt`: The certificate authority (optional). - `tls.crt`: The certificate (or a chain). - `tls.key`: The private key to the first certificate in the certificate chain.\",\n \"properties\": {\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"selfSignedCertificate\": {\n \"description\": \"SelfSignedCertificate allows configuring the self-signed certificate generated by the operator.\",\n \"properties\": {\n \"disabled\": {\n \"description\": \"Disabled indicates that the provisioning of the self-signed certificate should be disabled.\",\n \"type\": \"boolean\"\n },\n \"subjectAltNames\": {\n \"description\": \"SubjectAlternativeNames is a list of SANs to include in the generated HTTP TLS certificate.\",\n \"items\": {\n \"description\": \"SubjectAlternativeName represents a SAN entry in a x509 certificate.\",\n \"properties\": {\n \"dns\": {\n \"description\": \"DNS is the DNS name of the subject.\",\n \"type\": \"string\"\n },\n \"ip\": {\n \"description\": \"IP is the IP address of the subject.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"image\": {\n \"description\": \"Image is the Elasticsearch Docker image to deploy.\",\n \"type\": \"string\"\n },\n \"nodeSets\": {\n \"description\": \"NodeSets allow specifying groups of Elasticsearch nodes sharing the same configuration and Pod templates. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-orchestration.html\",\n \"items\": {\n \"description\": \"NodeSet is the specification for a group of Elasticsearch nodes sharing the same configuration and a Pod template.\",\n \"properties\": {\n \"config\": {\n \"description\": \"Config holds the Elasticsearch configuration.\",\n \"type\": \"object\"\n },\n \"count\": {\n \"description\": \"Count of Elasticsearch nodes to deploy.\",\n \"format\": \"int32\",\n \"minimum\": 1,\n \"type\": \"integer\"\n },\n \"name\": {\n \"description\": \"Name of this set of nodes. Becomes a part of the Elasticsearch node.name setting.\",\n \"maxLength\": 23,\n \"pattern\": \"[a-zA-Z0-9-]+\",\n \"type\": \"string\"\n },\n \"podTemplate\": {\n \"description\": \"PodTemplate provides customisation options (labels, annotations, affinity rules, resource requests, and so on) for the Pods belonging to this NodeSet.\",\n \"type\": \"object\"\n },\n \"volumeClaimTemplates\": {\n \"description\": \"VolumeClaimTemplates is a list of persistent volume claims to be used by each Pod in this NodeSet. Every claim in this list must have a matching volumeMount in one of the containers defined in the PodTemplate. Items defined here take precedence over any default claims added by the operator with the same name. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-volume-claim-templates.html\",\n \"items\": {\n \"description\": \"PersistentVolumeClaim is a user's request for and claim to a persistent volume\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"description\": \"Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata\",\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec defines the desired characteristics of a volume requested by a pod author. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims\",\n \"properties\": {\n \"accessModes\": {\n \"description\": \"AccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"dataSource\": {\n \"description\": \"This field requires the VolumeSnapshotDataSource alpha feature gate to be enabled and currently VolumeSnapshot is the only supported data source. If the provisioner can support VolumeSnapshot data source, it will create a new volume and data will be restored to the volume at the same time. If the provisioner does not support VolumeSnapshot data source, volume will not be created and the failure will be reported as an event. In the future, we plan to support more data source types and the behavior of the provisioner may change.\",\n \"properties\": {\n \"apiGroup\": {\n \"description\": \"APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is the type of resource being referenced\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name is the name of resource being referenced\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"kind\",\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"resources\": {\n \"description\": \"Resources represents the minimum resources the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources\",\n \"properties\": {\n \"limits\": {\n \"additionalProperties\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"pattern\": \"^(\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))))?$\"\n },\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"additionalProperties\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"pattern\": \"^(\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))))?$\"\n },\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"selector\": {\n \"description\": \"A label query over volumes to consider for binding.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"storageClassName\": {\n \"description\": \"Name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1\",\n \"type\": \"string\"\n },\n \"volumeMode\": {\n \"description\": \"volumeMode defines what type of volume is required by the claim. Value of Filesystem is implied when not included in claim spec. This is a beta feature.\",\n \"type\": \"string\"\n },\n \"volumeName\": {\n \"description\": \"VolumeName is the binding reference to the PersistentVolume backing this claim.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"status\": {\n \"description\": \"Status represents the current information/status of a persistent volume claim. Read-only. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims\",\n \"properties\": {\n \"accessModes\": {\n \"description\": \"AccessModes contains the actual access modes the volume backing the PVC has. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"capacity\": {\n \"additionalProperties\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"pattern\": \"^(\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))))?$\"\n },\n \"description\": \"Represents the actual resources of the underlying volume.\",\n \"type\": \"object\"\n },\n \"conditions\": {\n \"description\": \"Current Condition of persistent volume claim. If underlying persistent volume is being resized then the Condition will be set to 'ResizeStarted'.\",\n \"items\": {\n \"description\": \"PersistentVolumeClaimCondition contains details about state of pvc\",\n \"properties\": {\n \"lastProbeTime\": {\n \"description\": \"Last time we probed the condition.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n },\n \"lastTransitionTime\": {\n \"description\": \"Last time the condition transitioned from one status to another.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"Human-readable message indicating details about last transition.\",\n \"type\": \"string\"\n },\n \"reason\": {\n \"description\": \"Unique, this should be a short, machine understandable string that gives the reason for condition's last transition. If it reports \\\"ResizeStarted\\\" that means the underlying persistent volume is being resized.\",\n \"type\": \"string\"\n },\n \"status\": {\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"PersistentVolumeClaimConditionType is a valid value of PersistentVolumeClaimCondition.Type\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"status\",\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"phase\": {\n \"description\": \"Phase represents the current phase of PersistentVolumeClaim.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"count\",\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"minItems\": 1,\n \"type\": \"array\"\n },\n \"podDisruptionBudget\": {\n \"description\": \"PodDisruptionBudget provides access to the default pod disruption budget for the Elasticsearch cluster. The default budget selects all cluster pods and sets `maxUnavailable` to 1. To disable, set `PodDisruptionBudget` to the empty value (`{}` in YAML).\",\n \"properties\": {\n \"metadata\": {\n \"description\": \"ObjectMeta is the metadata of the PDB. The name and namespace provided here are managed by ECK and will be ignored.\",\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec is the specification of the PDB.\",\n \"properties\": {\n \"maxUnavailable\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"An eviction is allowed if at most \\\"maxUnavailable\\\" pods selected by \\\"selector\\\" are unavailable after the eviction, i.e. even in absence of the evicted pod. For example, one can prevent all voluntary evictions by specifying 0. This is a mutually exclusive setting with \\\"minAvailable\\\".\"\n },\n \"minAvailable\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"An eviction is allowed if at least \\\"minAvailable\\\" pods selected by \\\"selector\\\" will still be available after the eviction, i.e. even in the absence of the evicted pod. So for example you can prevent all voluntary evictions by specifying \\\"100%\\\".\"\n },\n \"selector\": {\n \"description\": \"Label query over pods whose evictions are managed by the disruption budget.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"remoteClusters\": {\n \"description\": \"RemoteClusters enables you to establish uni-directional connections to a remote Elasticsearch cluster.\",\n \"items\": {\n \"description\": \"RemoteCluster declares a remote Elasticsearch cluster connection.\",\n \"properties\": {\n \"elasticsearchRef\": {\n \"description\": \"ElasticsearchRef is a reference to an Elasticsearch cluster running within the same k8s cluster.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the Kubernetes object.\",\n \"type\": \"string\"\n },\n \"namespace\": {\n \"description\": \"Namespace of the Kubernetes object. If empty, defaults to the current namespace.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"name\": {\n \"description\": \"Name is the name of the remote cluster as it is set in the Elasticsearch settings. The name is expected to be unique for each remote clusters.\",\n \"minLength\": 1,\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"secureSettings\": {\n \"description\": \"SecureSettings is a list of references to Kubernetes secrets containing sensitive configuration options for Elasticsearch. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-es-secure-settings.html\",\n \"items\": {\n \"description\": \"SecretSource defines a data source based on a Kubernetes Secret.\",\n \"properties\": {\n \"entries\": {\n \"description\": \"Entries define how to project each key-value pair in the secret to filesystem paths. If not defined, all keys will be projected to similarly named paths in the filesystem. If defined, only the specified keys will be projected to the corresponding paths.\",\n \"items\": {\n \"description\": \"KeyToPath defines how to map a key in a Secret object to a filesystem path.\",\n \"properties\": {\n \"key\": {\n \"description\": \"Key is the key contained in the secret.\",\n \"type\": \"string\"\n },\n \"path\": {\n \"description\": \"Path is the relative file path to map the key to. Path must not be an absolute file path and must not contain any \\\"..\\\" components.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"secretName\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"serviceAccountName\": {\n \"description\": \"ServiceAccountName is used to check access from the current resource to a resource (eg. a remote Elasticsearch cluster) in a different namespace. Can only be used if ECK is enforcing RBAC on references.\",\n \"type\": \"string\"\n },\n \"transport\": {\n \"description\": \"Transport holds transport layer settings for Elasticsearch.\",\n \"properties\": {\n \"service\": {\n \"description\": \"Service defines the template for the associated Kubernetes Service object.\",\n \"properties\": {\n \"metadata\": {\n \"description\": \"ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored.\",\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec is the specification of the service.\",\n \"properties\": {\n \"clusterIP\": {\n \"description\": \"clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are \\\"None\\\", empty string (\\\"\\\"), or a valid IP address. \\\"None\\\" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"externalIPs\": {\n \"description\": \"externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"externalName\": {\n \"description\": \"externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName.\",\n \"type\": \"string\"\n },\n \"externalTrafficPolicy\": {\n \"description\": \"externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. \\\"Local\\\" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. \\\"Cluster\\\" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading.\",\n \"type\": \"string\"\n },\n \"healthCheckNodePort\": {\n \"description\": \"healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"ipFamily\": {\n \"description\": \"ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment.\",\n \"type\": \"string\"\n },\n \"loadBalancerIP\": {\n \"description\": \"Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.\",\n \"type\": \"string\"\n },\n \"loadBalancerSourceRanges\": {\n \"description\": \"If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature.\\\" More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"ports\": {\n \"description\": \"The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"items\": {\n \"description\": \"ServicePort contains information on service's port.\",\n \"properties\": {\n \"name\": {\n \"description\": \"The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service.\",\n \"type\": \"string\"\n },\n \"nodePort\": {\n \"description\": \"The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"port\": {\n \"description\": \"The port that will be exposed by this service.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"protocol\": {\n \"description\": \"The IP protocol for this port. Supports \\\"TCP\\\", \\\"UDP\\\", and \\\"SCTP\\\". Default is TCP.\",\n \"type\": \"string\"\n },\n \"targetPort\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod's container ports. If this is not specified, the value of the 'port' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the 'port' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"publishNotReadyAddresses\": {\n \"description\": \"publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery.\",\n \"type\": \"boolean\"\n },\n \"selector\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/\",\n \"type\": \"object\"\n },\n \"sessionAffinity\": {\n \"description\": \"Supports \\\"ClientIP\\\" and \\\"None\\\". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"sessionAffinityConfig\": {\n \"description\": \"sessionAffinityConfig contains the configurations of session affinity.\",\n \"properties\": {\n \"clientIP\": {\n \"description\": \"clientIP contains the configurations of Client IP based session affinity.\",\n \"properties\": {\n \"timeoutSeconds\": {\n \"description\": \"timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be \\u003e0 \\u0026\\u0026 \\u003c=86400(for 1 day) if ServiceAffinity == \\\"ClientIP\\\". Default value is 10800(for 3 hours).\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"topologyKeys\": {\n \"description\": \"topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value \\\"*\\\" may be used to mean \\\"any topology\\\". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"type\": {\n \"description\": \"type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. \\\"ExternalName\\\" maps to the specified externalName. \\\"ClusterIP\\\" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is \\\"None\\\", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. \\\"NodePort\\\" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. \\\"LoadBalancer\\\" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"updateStrategy\": {\n \"description\": \"UpdateStrategy specifies how updates to the cluster should be performed.\",\n \"properties\": {\n \"changeBudget\": {\n \"description\": \"ChangeBudget defines the constraints to consider when applying changes to the Elasticsearch cluster.\",\n \"properties\": {\n \"maxSurge\": {\n \"description\": \"MaxSurge is the maximum number of new pods that can be created exceeding the original number of pods defined in the specification. MaxSurge is only taken into consideration when scaling up. Setting a negative value will disable the restriction. Defaults to unbounded if not specified.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"maxUnavailable\": {\n \"description\": \"MaxUnavailable is the maximum number of pods that can be unavailable (not ready) during the update due to circumstances under the control of the operator. Setting a negative value will disable this restriction. Defaults to 1 if not specified.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"version\": {\n \"description\": \"Version of Elasticsearch.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"nodeSets\",\n \"version\"\n ],\n \"type\": \"object\"\n },\n \"status\": {\n \"description\": \"ElasticsearchStatus defines the observed state of Elasticsearch\",\n \"properties\": {\n \"availableNodes\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"health\": {\n \"description\": \"ElasticsearchHealth is the health of the cluster as returned by the health API.\",\n \"type\": \"string\"\n },\n \"phase\": {\n \"description\": \"ElasticsearchOrchestrationPhase is the phase Elasticsearch is in from the controller point of view.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n }\n }\n },\n \"served\": true,\n \"storage\": false,\n \"subresources\": {\n \"status\": {}\n }\n },\n {\n \"additionalPrinterColumns\": [\n {\n \"jsonPath\": \".status.health\",\n \"name\": \"health\",\n \"type\": \"string\"\n },\n {\n \"description\": \"Available nodes\",\n \"jsonPath\": \".status.availableNodes\",\n \"name\": \"nodes\",\n \"type\": \"integer\"\n },\n {\n \"description\": \"Elasticsearch version\",\n \"jsonPath\": \".spec.version\",\n \"name\": \"version\",\n \"type\": \"string\"\n },\n {\n \"jsonPath\": \".status.phase\",\n \"name\": \"phase\",\n \"type\": \"string\"\n },\n {\n \"jsonPath\": \".metadata.creationTimestamp\",\n \"name\": \"age\",\n \"type\": \"date\"\n }\n ],\n \"name\": \"v1alpha1\",\n \"schema\": {\n \"openAPIV3Schema\": {\n \"description\": \"Elasticsearch represents an Elasticsearch resource in a Kubernetes cluster.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"ElasticsearchSpec holds the specification of an Elasticsearch cluster.\",\n \"properties\": {\n \"auth\": {\n \"description\": \"Auth contains user authentication and authorization security settings for Elasticsearch.\",\n \"properties\": {\n \"fileRealm\": {\n \"description\": \"FileRealm to propagate to the Elasticsearch cluster.\",\n \"items\": {\n \"description\": \"FileRealmSource references users to create in the Elasticsearch cluster.\",\n \"properties\": {\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"roles\": {\n \"description\": \"Roles to propagate to the Elasticsearch cluster.\",\n \"items\": {\n \"description\": \"RoleSource references roles to create in the Elasticsearch cluster.\",\n \"properties\": {\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"http\": {\n \"description\": \"HTTP holds HTTP layer settings for Elasticsearch.\",\n \"properties\": {\n \"service\": {\n \"description\": \"Service defines the template for the associated Kubernetes Service object.\",\n \"properties\": {\n \"metadata\": {\n \"description\": \"ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored.\",\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec is the specification of the service.\",\n \"properties\": {\n \"clusterIP\": {\n \"description\": \"clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are \\\"None\\\", empty string (\\\"\\\"), or a valid IP address. \\\"None\\\" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"externalIPs\": {\n \"description\": \"externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"externalName\": {\n \"description\": \"externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName.\",\n \"type\": \"string\"\n },\n \"externalTrafficPolicy\": {\n \"description\": \"externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. \\\"Local\\\" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. \\\"Cluster\\\" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading.\",\n \"type\": \"string\"\n },\n \"healthCheckNodePort\": {\n \"description\": \"healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"ipFamily\": {\n \"description\": \"ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment.\",\n \"type\": \"string\"\n },\n \"loadBalancerIP\": {\n \"description\": \"Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.\",\n \"type\": \"string\"\n },\n \"loadBalancerSourceRanges\": {\n \"description\": \"If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature.\\\" More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"ports\": {\n \"description\": \"The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"items\": {\n \"description\": \"ServicePort contains information on service's port.\",\n \"properties\": {\n \"name\": {\n \"description\": \"The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service.\",\n \"type\": \"string\"\n },\n \"nodePort\": {\n \"description\": \"The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"port\": {\n \"description\": \"The port that will be exposed by this service.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"protocol\": {\n \"description\": \"The IP protocol for this port. Supports \\\"TCP\\\", \\\"UDP\\\", and \\\"SCTP\\\". Default is TCP.\",\n \"type\": \"string\"\n },\n \"targetPort\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod's container ports. If this is not specified, the value of the 'port' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the 'port' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"publishNotReadyAddresses\": {\n \"description\": \"publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery.\",\n \"type\": \"boolean\"\n },\n \"selector\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/\",\n \"type\": \"object\"\n },\n \"sessionAffinity\": {\n \"description\": \"Supports \\\"ClientIP\\\" and \\\"None\\\". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"sessionAffinityConfig\": {\n \"description\": \"sessionAffinityConfig contains the configurations of session affinity.\",\n \"properties\": {\n \"clientIP\": {\n \"description\": \"clientIP contains the configurations of Client IP based session affinity.\",\n \"properties\": {\n \"timeoutSeconds\": {\n \"description\": \"timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be \\u003e0 \\u0026\\u0026 \\u003c=86400(for 1 day) if ServiceAffinity == \\\"ClientIP\\\". Default value is 10800(for 3 hours).\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"topologyKeys\": {\n \"description\": \"topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value \\\"*\\\" may be used to mean \\\"any topology\\\". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"type\": {\n \"description\": \"type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. \\\"ExternalName\\\" maps to the specified externalName. \\\"ClusterIP\\\" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is \\\"None\\\", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. \\\"NodePort\\\" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. \\\"LoadBalancer\\\" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"tls\": {\n \"description\": \"TLS defines options for configuring TLS for HTTP.\",\n \"properties\": {\n \"certificate\": {\n \"description\": \"Certificate is a reference to a Kubernetes secret that contains the certificate and private key for enabling TLS. The referenced secret should contain the following: \\n - `ca.crt`: The certificate authority (optional). - `tls.crt`: The certificate (or a chain). - `tls.key`: The private key to the first certificate in the certificate chain.\",\n \"properties\": {\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"selfSignedCertificate\": {\n \"description\": \"SelfSignedCertificate allows configuring the self-signed certificate generated by the operator.\",\n \"properties\": {\n \"disabled\": {\n \"description\": \"Disabled indicates that the provisioning of the self-signed certificate should be disabled.\",\n \"type\": \"boolean\"\n },\n \"subjectAltNames\": {\n \"description\": \"SubjectAlternativeNames is a list of SANs to include in the generated HTTP TLS certificate.\",\n \"items\": {\n \"description\": \"SubjectAlternativeName represents a SAN entry in a x509 certificate.\",\n \"properties\": {\n \"dns\": {\n \"description\": \"DNS is the DNS name of the subject.\",\n \"type\": \"string\"\n },\n \"ip\": {\n \"description\": \"IP is the IP address of the subject.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"image\": {\n \"description\": \"Image is the Elasticsearch Docker image to deploy.\",\n \"type\": \"string\"\n },\n \"nodeSets\": {\n \"description\": \"NodeSets allow specifying groups of Elasticsearch nodes sharing the same configuration and Pod templates. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-orchestration.html\",\n \"items\": {\n \"description\": \"NodeSet is the specification for a group of Elasticsearch nodes sharing the same configuration and a Pod template.\",\n \"properties\": {\n \"config\": {\n \"description\": \"Config holds the Elasticsearch configuration.\",\n \"type\": \"object\"\n },\n \"count\": {\n \"description\": \"Count of Elasticsearch nodes to deploy.\",\n \"format\": \"int32\",\n \"minimum\": 1,\n \"type\": \"integer\"\n },\n \"name\": {\n \"description\": \"Name of this set of nodes. Becomes a part of the Elasticsearch node.name setting.\",\n \"maxLength\": 23,\n \"pattern\": \"[a-zA-Z0-9-]+\",\n \"type\": \"string\"\n },\n \"podTemplate\": {\n \"description\": \"PodTemplate provides customisation options (labels, annotations, affinity rules, resource requests, and so on) for the Pods belonging to this NodeSet.\",\n \"type\": \"object\"\n },\n \"volumeClaimTemplates\": {\n \"description\": \"VolumeClaimTemplates is a list of persistent volume claims to be used by each Pod in this NodeSet. Every claim in this list must have a matching volumeMount in one of the containers defined in the PodTemplate. Items defined here take precedence over any default claims added by the operator with the same name. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-volume-claim-templates.html\",\n \"items\": {\n \"description\": \"PersistentVolumeClaim is a user's request for and claim to a persistent volume\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"description\": \"Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata\",\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec defines the desired characteristics of a volume requested by a pod author. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims\",\n \"properties\": {\n \"accessModes\": {\n \"description\": \"AccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"dataSource\": {\n \"description\": \"This field requires the VolumeSnapshotDataSource alpha feature gate to be enabled and currently VolumeSnapshot is the only supported data source. If the provisioner can support VolumeSnapshot data source, it will create a new volume and data will be restored to the volume at the same time. If the provisioner does not support VolumeSnapshot data source, volume will not be created and the failure will be reported as an event. In the future, we plan to support more data source types and the behavior of the provisioner may change.\",\n \"properties\": {\n \"apiGroup\": {\n \"description\": \"APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is the type of resource being referenced\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name is the name of resource being referenced\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"kind\",\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"resources\": {\n \"description\": \"Resources represents the minimum resources the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources\",\n \"properties\": {\n \"limits\": {\n \"additionalProperties\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"pattern\": \"^(\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))))?$\"\n },\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"additionalProperties\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"pattern\": \"^(\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))))?$\"\n },\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"selector\": {\n \"description\": \"A label query over volumes to consider for binding.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"storageClassName\": {\n \"description\": \"Name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1\",\n \"type\": \"string\"\n },\n \"volumeMode\": {\n \"description\": \"volumeMode defines what type of volume is required by the claim. Value of Filesystem is implied when not included in claim spec. This is a beta feature.\",\n \"type\": \"string\"\n },\n \"volumeName\": {\n \"description\": \"VolumeName is the binding reference to the PersistentVolume backing this claim.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"status\": {\n \"description\": \"Status represents the current information/status of a persistent volume claim. Read-only. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims\",\n \"properties\": {\n \"accessModes\": {\n \"description\": \"AccessModes contains the actual access modes the volume backing the PVC has. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"capacity\": {\n \"additionalProperties\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"pattern\": \"^(\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))))?$\"\n },\n \"description\": \"Represents the actual resources of the underlying volume.\",\n \"type\": \"object\"\n },\n \"conditions\": {\n \"description\": \"Current Condition of persistent volume claim. If underlying persistent volume is being resized then the Condition will be set to 'ResizeStarted'.\",\n \"items\": {\n \"description\": \"PersistentVolumeClaimCondition contains details about state of pvc\",\n \"properties\": {\n \"lastProbeTime\": {\n \"description\": \"Last time we probed the condition.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n },\n \"lastTransitionTime\": {\n \"description\": \"Last time the condition transitioned from one status to another.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"Human-readable message indicating details about last transition.\",\n \"type\": \"string\"\n },\n \"reason\": {\n \"description\": \"Unique, this should be a short, machine understandable string that gives the reason for condition's last transition. If it reports \\\"ResizeStarted\\\" that means the underlying persistent volume is being resized.\",\n \"type\": \"string\"\n },\n \"status\": {\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"PersistentVolumeClaimConditionType is a valid value of PersistentVolumeClaimCondition.Type\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"status\",\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"phase\": {\n \"description\": \"Phase represents the current phase of PersistentVolumeClaim.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"count\",\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"minItems\": 1,\n \"type\": \"array\"\n },\n \"podDisruptionBudget\": {\n \"description\": \"PodDisruptionBudget provides access to the default pod disruption budget for the Elasticsearch cluster. The default budget selects all cluster pods and sets `maxUnavailable` to 1. To disable, set `PodDisruptionBudget` to the empty value (`{}` in YAML).\",\n \"properties\": {\n \"metadata\": {\n \"description\": \"ObjectMeta is the metadata of the PDB. The name and namespace provided here are managed by ECK and will be ignored.\",\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec is the specification of the PDB.\",\n \"properties\": {\n \"maxUnavailable\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"An eviction is allowed if at most \\\"maxUnavailable\\\" pods selected by \\\"selector\\\" are unavailable after the eviction, i.e. even in absence of the evicted pod. For example, one can prevent all voluntary evictions by specifying 0. This is a mutually exclusive setting with \\\"minAvailable\\\".\"\n },\n \"minAvailable\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"An eviction is allowed if at least \\\"minAvailable\\\" pods selected by \\\"selector\\\" will still be available after the eviction, i.e. even in the absence of the evicted pod. So for example you can prevent all voluntary evictions by specifying \\\"100%\\\".\"\n },\n \"selector\": {\n \"description\": \"Label query over pods whose evictions are managed by the disruption budget.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"remoteClusters\": {\n \"description\": \"RemoteClusters enables you to establish uni-directional connections to a remote Elasticsearch cluster.\",\n \"items\": {\n \"description\": \"RemoteCluster declares a remote Elasticsearch cluster connection.\",\n \"properties\": {\n \"elasticsearchRef\": {\n \"description\": \"ElasticsearchRef is a reference to an Elasticsearch cluster running within the same k8s cluster.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the Kubernetes object.\",\n \"type\": \"string\"\n },\n \"namespace\": {\n \"description\": \"Namespace of the Kubernetes object. If empty, defaults to the current namespace.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"name\": {\n \"description\": \"Name is the name of the remote cluster as it is set in the Elasticsearch settings. The name is expected to be unique for each remote clusters.\",\n \"minLength\": 1,\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"secureSettings\": {\n \"description\": \"SecureSettings is a list of references to Kubernetes secrets containing sensitive configuration options for Elasticsearch. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-es-secure-settings.html\",\n \"items\": {\n \"description\": \"SecretSource defines a data source based on a Kubernetes Secret.\",\n \"properties\": {\n \"entries\": {\n \"description\": \"Entries define how to project each key-value pair in the secret to filesystem paths. If not defined, all keys will be projected to similarly named paths in the filesystem. If defined, only the specified keys will be projected to the corresponding paths.\",\n \"items\": {\n \"description\": \"KeyToPath defines how to map a key in a Secret object to a filesystem path.\",\n \"properties\": {\n \"key\": {\n \"description\": \"Key is the key contained in the secret.\",\n \"type\": \"string\"\n },\n \"path\": {\n \"description\": \"Path is the relative file path to map the key to. Path must not be an absolute file path and must not contain any \\\"..\\\" components.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"secretName\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"serviceAccountName\": {\n \"description\": \"ServiceAccountName is used to check access from the current resource to a resource (eg. a remote Elasticsearch cluster) in a different namespace. Can only be used if ECK is enforcing RBAC on references.\",\n \"type\": \"string\"\n },\n \"transport\": {\n \"description\": \"Transport holds transport layer settings for Elasticsearch.\",\n \"properties\": {\n \"service\": {\n \"description\": \"Service defines the template for the associated Kubernetes Service object.\",\n \"properties\": {\n \"metadata\": {\n \"description\": \"ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored.\",\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec is the specification of the service.\",\n \"properties\": {\n \"clusterIP\": {\n \"description\": \"clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are \\\"None\\\", empty string (\\\"\\\"), or a valid IP address. \\\"None\\\" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"externalIPs\": {\n \"description\": \"externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"externalName\": {\n \"description\": \"externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName.\",\n \"type\": \"string\"\n },\n \"externalTrafficPolicy\": {\n \"description\": \"externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. \\\"Local\\\" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. \\\"Cluster\\\" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading.\",\n \"type\": \"string\"\n },\n \"healthCheckNodePort\": {\n \"description\": \"healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"ipFamily\": {\n \"description\": \"ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment.\",\n \"type\": \"string\"\n },\n \"loadBalancerIP\": {\n \"description\": \"Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.\",\n \"type\": \"string\"\n },\n \"loadBalancerSourceRanges\": {\n \"description\": \"If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature.\\\" More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"ports\": {\n \"description\": \"The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"items\": {\n \"description\": \"ServicePort contains information on service's port.\",\n \"properties\": {\n \"name\": {\n \"description\": \"The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service.\",\n \"type\": \"string\"\n },\n \"nodePort\": {\n \"description\": \"The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"port\": {\n \"description\": \"The port that will be exposed by this service.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"protocol\": {\n \"description\": \"The IP protocol for this port. Supports \\\"TCP\\\", \\\"UDP\\\", and \\\"SCTP\\\". Default is TCP.\",\n \"type\": \"string\"\n },\n \"targetPort\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod's container ports. If this is not specified, the value of the 'port' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the 'port' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"publishNotReadyAddresses\": {\n \"description\": \"publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery.\",\n \"type\": \"boolean\"\n },\n \"selector\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/\",\n \"type\": \"object\"\n },\n \"sessionAffinity\": {\n \"description\": \"Supports \\\"ClientIP\\\" and \\\"None\\\". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"sessionAffinityConfig\": {\n \"description\": \"sessionAffinityConfig contains the configurations of session affinity.\",\n \"properties\": {\n \"clientIP\": {\n \"description\": \"clientIP contains the configurations of Client IP based session affinity.\",\n \"properties\": {\n \"timeoutSeconds\": {\n \"description\": \"timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be \\u003e0 \\u0026\\u0026 \\u003c=86400(for 1 day) if ServiceAffinity == \\\"ClientIP\\\". Default value is 10800(for 3 hours).\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"topologyKeys\": {\n \"description\": \"topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value \\\"*\\\" may be used to mean \\\"any topology\\\". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"type\": {\n \"description\": \"type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. \\\"ExternalName\\\" maps to the specified externalName. \\\"ClusterIP\\\" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is \\\"None\\\", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. \\\"NodePort\\\" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. \\\"LoadBalancer\\\" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"updateStrategy\": {\n \"description\": \"UpdateStrategy specifies how updates to the cluster should be performed.\",\n \"properties\": {\n \"changeBudget\": {\n \"description\": \"ChangeBudget defines the constraints to consider when applying changes to the Elasticsearch cluster.\",\n \"properties\": {\n \"maxSurge\": {\n \"description\": \"MaxSurge is the maximum number of new pods that can be created exceeding the original number of pods defined in the specification. MaxSurge is only taken into consideration when scaling up. Setting a negative value will disable the restriction. Defaults to unbounded if not specified.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"maxUnavailable\": {\n \"description\": \"MaxUnavailable is the maximum number of pods that can be unavailable (not ready) during the update due to circumstances under the control of the operator. Setting a negative value will disable this restriction. Defaults to 1 if not specified.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"version\": {\n \"description\": \"Version of Elasticsearch.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"nodeSets\",\n \"version\"\n ],\n \"type\": \"object\"\n },\n \"status\": {\n \"description\": \"ElasticsearchStatus defines the observed state of Elasticsearch\",\n \"properties\": {\n \"availableNodes\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"health\": {\n \"description\": \"ElasticsearchHealth is the health of the cluster as returned by the health API.\",\n \"type\": \"string\"\n },\n \"phase\": {\n \"description\": \"ElasticsearchOrchestrationPhase is the phase Elasticsearch is in from the controller point of view.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n }\n }\n },\n \"served\": false,\n \"storage\": false,\n \"subresources\": {\n \"status\": {}\n }\n }\n ]\n },\n \"status\": {\n \"acceptedNames\": {\n \"categories\": [\n \"elastic\"\n ],\n \"kind\": \"Elasticsearch\",\n \"listKind\": \"ElasticsearchList\",\n \"plural\": \"elasticsearches\",\n \"shortNames\": [\n \"es\"\n ],\n \"singular\": \"elasticsearch\"\n },\n \"conditions\": [\n {\n \"lastTransitionTime\": \"2020-04-28T23:31:51Z\",\n \"message\": \"[spec.validation.openAPIV3Schema.properties[spec].properties[http].properties[service].properties[spec].properties[ports].items.properties[targetPort].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[nodeSets].items.properties[volumeClaimTemplates].items.properties[spec].properties[resources].properties[limits].additionalProperties.type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[nodeSets].items.properties[volumeClaimTemplates].items.properties[spec].properties[resources].properties[requests].additionalProperties.type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[nodeSets].items.properties[volumeClaimTemplates].items.properties[status].properties[capacity].additionalProperties.type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[podDisruptionBudget].properties[spec].properties[maxUnavailable].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[podDisruptionBudget].properties[spec].properties[minAvailable].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[transport].properties[service].properties[spec].properties[ports].items.properties[targetPort].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.type: Required value: must not be empty at the root]\",\n \"reason\": \"Violations\",\n \"status\": \"True\",\n \"type\": \"NonStructuralSchema\"\n },\n {\n \"lastTransitionTime\": \"2020-04-28T23:31:51Z\",\n \"message\": \"no conflicts found\",\n \"reason\": \"NoConflicts\",\n \"status\": \"True\",\n \"type\": \"NamesAccepted\"\n },\n {\n \"lastTransitionTime\": \"2020-04-28T23:31:51Z\",\n \"message\": \"the initial names have been accepted\",\n \"reason\": \"InitialNamesAccepted\",\n \"status\": \"True\",\n \"type\": \"Established\"\n }\n ],\n \"storedVersions\": [\n \"v1\"\n ]\n }\n}\n" }, { "path": "pkg/backup/actions/testdata/v1/gcpsamples.gcp.stacks.crossplane.io.json", "content": "{\n \"apiVersion\": \"apiextensions.k8s.io/v1\",\n \"kind\": \"CustomResourceDefinition\",\n \"metadata\": {\n \"annotations\": {\n \"kubectl.kubernetes.io/last-applied-configuration\": \"{\\\"apiVersion\\\":\\\"apiextensions.k8s.io/v1beta1\\\",\\\"kind\\\":\\\"CustomResourceDefinition\\\",\\\"metadata\\\":{\\\"annotations\\\":{},\\\"creationTimestamp\\\":\\\"2020-04-20T16:57:37Z\\\",\\\"generation\\\":1,\\\"name\\\":\\\"gcpsamples.gcp.stacks.crossplane.io\\\",\\\"resourceVersion\\\":\\\"549\\\",\\\"selfLink\\\":\\\"/apis/apiextensions.k8s.io/v1/customresourcedefinitions/gcpsamples.gcp.stacks.crossplane.io\\\",\\\"uid\\\":\\\"db5f4321-3226-44b0-8247-66fd7ef59dc8\\\"},\\\"spec\\\":{\\\"conversion\\\":{\\\"strategy\\\":\\\"None\\\"},\\\"group\\\":\\\"gcp.stacks.crossplane.io\\\",\\\"names\\\":{\\\"kind\\\":\\\"GCPSample\\\",\\\"listKind\\\":\\\"GCPSampleList\\\",\\\"plural\\\":\\\"gcpsamples\\\",\\\"singular\\\":\\\"gcpsample\\\"},\\\"preserveUnknownFields\\\":true,\\\"scope\\\":\\\"Cluster\\\",\\\"versions\\\":[{\\\"name\\\":\\\"v1alpha1\\\",\\\"served\\\":true,\\\"storage\\\":true}]},\\\"status\\\":{\\\"acceptedNames\\\":{\\\"kind\\\":\\\"GCPSample\\\",\\\"listKind\\\":\\\"GCPSampleList\\\",\\\"plural\\\":\\\"gcpsamples\\\",\\\"singular\\\":\\\"gcpsample\\\"},\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2020-04-20T16:57:37Z\\\",\\\"message\\\":\\\"no conflicts found\\\",\\\"reason\\\":\\\"NoConflicts\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"NamesAccepted\\\"},{\\\"lastTransitionTime\\\":\\\"2020-04-20T16:57:37Z\\\",\\\"message\\\":\\\"the initial names have been accepted\\\",\\\"reason\\\":\\\"InitialNamesAccepted\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Established\\\"}],\\\"storedVersions\\\":[\\\"v1alpha1\\\"]}}\\n\"\n },\n \"creationTimestamp\": \"2020-04-20T17:27:56Z\",\n \"generation\": 1,\n \"name\": \"gcpsamples.gcp.stacks.crossplane.io\",\n \"resourceVersion\": \"5567\",\n \"selfLink\": \"/apis/apiextensions.k8s.io/v1/customresourcedefinitions/gcpsamples.gcp.stacks.crossplane.io\",\n \"uid\": \"c0bbac74-acab-4620-b628-1d5f91b19040\"\n },\n \"spec\": {\n \"conversion\": {\n \"strategy\": \"None\"\n },\n \"group\": \"gcp.stacks.crossplane.io\",\n \"names\": {\n \"kind\": \"GCPSample\",\n \"listKind\": \"GCPSampleList\",\n \"plural\": \"gcpsamples\",\n \"singular\": \"gcpsample\"\n },\n \"preserveUnknownFields\": true,\n \"scope\": \"Cluster\",\n \"versions\": [\n {\n \"name\": \"v1alpha1\",\n \"served\": true,\n \"storage\": true\n }\n ]\n },\n \"status\": {\n \"acceptedNames\": {\n \"kind\": \"GCPSample\",\n \"listKind\": \"GCPSampleList\",\n \"plural\": \"gcpsamples\",\n \"singular\": \"gcpsample\"\n },\n \"conditions\": [\n {\n \"lastTransitionTime\": \"2020-04-20T17:27:56Z\",\n \"message\": \"no conflicts found\",\n \"reason\": \"NoConflicts\",\n \"status\": \"True\",\n \"type\": \"NamesAccepted\"\n },\n {\n \"lastTransitionTime\": \"2020-04-20T17:27:56Z\",\n \"message\": \"the initial names have been accepted\",\n \"reason\": \"InitialNamesAccepted\",\n \"status\": \"True\",\n \"type\": \"Established\"\n }\n ],\n \"storedVersions\": [\n \"v1alpha1\"\n ]\n }\n}\n" }, { "path": "pkg/backup/actions/testdata/v1/kibanas.kibana.k8s.elastic.co.json", "content": "{\n \"apiVersion\": \"apiextensions.k8s.io/v1\",\n \"kind\": \"CustomResourceDefinition\",\n \"metadata\": {\n \"annotations\": {\n \"controller-gen.kubebuilder.io/version\": \"v0.2.5\",\n \"kubectl.kubernetes.io/last-applied-configuration\": \"{\\\"apiVersion\\\":\\\"apiextensions.k8s.io/v1beta1\\\",\\\"kind\\\":\\\"CustomResourceDefinition\\\",\\\"metadata\\\":{\\\"annotations\\\":{\\\"controller-gen.kubebuilder.io/version\\\":\\\"v0.2.5\\\"},\\\"creationTimestamp\\\":null,\\\"name\\\":\\\"kibanas.kibana.k8s.elastic.co\\\"},\\\"spec\\\":{\\\"additionalPrinterColumns\\\":[{\\\"JSONPath\\\":\\\".status.health\\\",\\\"name\\\":\\\"health\\\",\\\"type\\\":\\\"string\\\"},{\\\"JSONPath\\\":\\\".status.availableNodes\\\",\\\"description\\\":\\\"Available nodes\\\",\\\"name\\\":\\\"nodes\\\",\\\"type\\\":\\\"integer\\\"},{\\\"JSONPath\\\":\\\".spec.version\\\",\\\"description\\\":\\\"Kibana version\\\",\\\"name\\\":\\\"version\\\",\\\"type\\\":\\\"string\\\"},{\\\"JSONPath\\\":\\\".metadata.creationTimestamp\\\",\\\"name\\\":\\\"age\\\",\\\"type\\\":\\\"date\\\"}],\\\"group\\\":\\\"kibana.k8s.elastic.co\\\",\\\"names\\\":{\\\"categories\\\":[\\\"elastic\\\"],\\\"kind\\\":\\\"Kibana\\\",\\\"listKind\\\":\\\"KibanaList\\\",\\\"plural\\\":\\\"kibanas\\\",\\\"shortNames\\\":[\\\"kb\\\"],\\\"singular\\\":\\\"kibana\\\"},\\\"scope\\\":\\\"Namespaced\\\",\\\"subresources\\\":{\\\"status\\\":{}},\\\"validation\\\":{\\\"openAPIV3Schema\\\":{\\\"description\\\":\\\"Kibana represents a Kibana resource in a Kubernetes cluster.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"metadata\\\":{\\\"type\\\":\\\"object\\\"},\\\"spec\\\":{\\\"description\\\":\\\"KibanaSpec holds the specification of a Kibana instance.\\\",\\\"properties\\\":{\\\"config\\\":{\\\"description\\\":\\\"Config holds the Kibana configuration. See: https://www.elastic.co/guide/en/kibana/current/settings.html\\\",\\\"type\\\":\\\"object\\\"},\\\"count\\\":{\\\"description\\\":\\\"Count of Kibana instances to deploy.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"elasticsearchRef\\\":{\\\"description\\\":\\\"ElasticsearchRef is a reference to an Elasticsearch cluster running in the same Kubernetes cluster.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the Kubernetes object.\\\",\\\"type\\\":\\\"string\\\"},\\\"namespace\\\":{\\\"description\\\":\\\"Namespace of the Kubernetes object. If empty, defaults to the current namespace.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"http\\\":{\\\"description\\\":\\\"HTTP holds the HTTP layer configuration for Kibana.\\\",\\\"properties\\\":{\\\"service\\\":{\\\"description\\\":\\\"Service defines the template for the associated Kubernetes Service object.\\\",\\\"properties\\\":{\\\"metadata\\\":{\\\"description\\\":\\\"ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored.\\\",\\\"type\\\":\\\"object\\\"},\\\"spec\\\":{\\\"description\\\":\\\"Spec is the specification of the service.\\\",\\\"properties\\\":{\\\"clusterIP\\\":{\\\"description\\\":\\\"clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are \\\\\\\"None\\\\\\\", empty string (\\\\\\\"\\\\\\\"), or a valid IP address. \\\\\\\"None\\\\\\\" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\\\",\\\"type\\\":\\\"string\\\"},\\\"externalIPs\\\":{\\\"description\\\":\\\"externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"externalName\\\":{\\\"description\\\":\\\"externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName.\\\",\\\"type\\\":\\\"string\\\"},\\\"externalTrafficPolicy\\\":{\\\"description\\\":\\\"externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. \\\\\\\"Local\\\\\\\" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. \\\\\\\"Cluster\\\\\\\" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading.\\\",\\\"type\\\":\\\"string\\\"},\\\"healthCheckNodePort\\\":{\\\"description\\\":\\\"healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"ipFamily\\\":{\\\"description\\\":\\\"ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment.\\\",\\\"type\\\":\\\"string\\\"},\\\"loadBalancerIP\\\":{\\\"description\\\":\\\"Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.\\\",\\\"type\\\":\\\"string\\\"},\\\"loadBalancerSourceRanges\\\":{\\\"description\\\":\\\"If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature.\\\\\\\" More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"ports\\\":{\\\"description\\\":\\\"The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\\\",\\\"items\\\":{\\\"description\\\":\\\"ServicePort contains information on service's port.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service.\\\",\\\"type\\\":\\\"string\\\"},\\\"nodePort\\\":{\\\"description\\\":\\\"The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"port\\\":{\\\"description\\\":\\\"The port that will be exposed by this service.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"protocol\\\":{\\\"description\\\":\\\"The IP protocol for this port. Supports \\\\\\\"TCP\\\\\\\", \\\\\\\"UDP\\\\\\\", and \\\\\\\"SCTP\\\\\\\". Default is TCP.\\\",\\\"type\\\":\\\"string\\\"},\\\"targetPort\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod's container ports. If this is not specified, the value of the 'port' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the 'port' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"publishNotReadyAddresses\\\":{\\\"description\\\":\\\"publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"selector\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/\\\",\\\"type\\\":\\\"object\\\"},\\\"sessionAffinity\\\":{\\\"description\\\":\\\"Supports \\\\\\\"ClientIP\\\\\\\" and \\\\\\\"None\\\\\\\". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\\\",\\\"type\\\":\\\"string\\\"},\\\"sessionAffinityConfig\\\":{\\\"description\\\":\\\"sessionAffinityConfig contains the configurations of session affinity.\\\",\\\"properties\\\":{\\\"clientIP\\\":{\\\"description\\\":\\\"clientIP contains the configurations of Client IP based session affinity.\\\",\\\"properties\\\":{\\\"timeoutSeconds\\\":{\\\"description\\\":\\\"timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be \\\\u003e0 \\\\u0026\\\\u0026 \\\\u003c=86400(for 1 day) if ServiceAffinity == \\\\\\\"ClientIP\\\\\\\". Default value is 10800(for 3 hours).\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"topologyKeys\\\":{\\\"description\\\":\\\"topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value \\\\\\\"*\\\\\\\" may be used to mean \\\\\\\"any topology\\\\\\\". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"type\\\":{\\\"description\\\":\\\"type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. \\\\\\\"ExternalName\\\\\\\" maps to the specified externalName. \\\\\\\"ClusterIP\\\\\\\" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is \\\\\\\"None\\\\\\\", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. \\\\\\\"NodePort\\\\\\\" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. \\\\\\\"LoadBalancer\\\\\\\" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"tls\\\":{\\\"description\\\":\\\"TLS defines options for configuring TLS for HTTP.\\\",\\\"properties\\\":{\\\"certificate\\\":{\\\"description\\\":\\\"Certificate is a reference to a Kubernetes secret that contains the certificate and private key for enabling TLS. The referenced secret should contain the following: \\\\n - `ca.crt`: The certificate authority (optional). - `tls.crt`: The certificate (or a chain). - `tls.key`: The private key to the first certificate in the certificate chain.\\\",\\\"properties\\\":{\\\"secretName\\\":{\\\"description\\\":\\\"SecretName is the name of the secret.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"selfSignedCertificate\\\":{\\\"description\\\":\\\"SelfSignedCertificate allows configuring the self-signed certificate generated by the operator.\\\",\\\"properties\\\":{\\\"disabled\\\":{\\\"description\\\":\\\"Disabled indicates that the provisioning of the self-signed certificate should be disabled.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"subjectAltNames\\\":{\\\"description\\\":\\\"SubjectAlternativeNames is a list of SANs to include in the generated HTTP TLS certificate.\\\",\\\"items\\\":{\\\"description\\\":\\\"SubjectAlternativeName represents a SAN entry in a x509 certificate.\\\",\\\"properties\\\":{\\\"dns\\\":{\\\"description\\\":\\\"DNS is the DNS name of the subject.\\\",\\\"type\\\":\\\"string\\\"},\\\"ip\\\":{\\\"description\\\":\\\"IP is the IP address of the subject.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"image\\\":{\\\"description\\\":\\\"Image is the Kibana Docker image to deploy.\\\",\\\"type\\\":\\\"string\\\"},\\\"podTemplate\\\":{\\\"description\\\":\\\"PodTemplate provides customisation options (labels, annotations, affinity rules, resource requests, and so on) for the Kibana pods\\\",\\\"type\\\":\\\"object\\\"},\\\"secureSettings\\\":{\\\"description\\\":\\\"SecureSettings is a list of references to Kubernetes secrets containing sensitive configuration options for Kibana. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-kibana.html#k8s-kibana-secure-settings\\\",\\\"items\\\":{\\\"description\\\":\\\"SecretSource defines a data source based on a Kubernetes Secret.\\\",\\\"properties\\\":{\\\"entries\\\":{\\\"description\\\":\\\"Entries define how to project each key-value pair in the secret to filesystem paths. If not defined, all keys will be projected to similarly named paths in the filesystem. If defined, only the specified keys will be projected to the corresponding paths.\\\",\\\"items\\\":{\\\"description\\\":\\\"KeyToPath defines how to map a key in a Secret object to a filesystem path.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"Key is the key contained in the secret.\\\",\\\"type\\\":\\\"string\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path is the relative file path to map the key to. Path must not be an absolute file path and must not contain any \\\\\\\"..\\\\\\\" components.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"secretName\\\":{\\\"description\\\":\\\"SecretName is the name of the secret.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"secretName\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"serviceAccountName\\\":{\\\"description\\\":\\\"ServiceAccountName is used to check access from the current resource to a resource (eg. Elasticsearch) in a different namespace. Can only be used if ECK is enforcing RBAC on references.\\\",\\\"type\\\":\\\"string\\\"},\\\"version\\\":{\\\"description\\\":\\\"Version of Kibana.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"version\\\"],\\\"type\\\":\\\"object\\\"},\\\"status\\\":{\\\"description\\\":\\\"KibanaStatus defines the observed state of Kibana\\\",\\\"properties\\\":{\\\"associationStatus\\\":{\\\"description\\\":\\\"AssociationStatus is the status of an association resource.\\\",\\\"type\\\":\\\"string\\\"},\\\"availableNodes\\\":{\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"health\\\":{\\\"description\\\":\\\"KibanaHealth expresses the status of the Kibana instances.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}}}},\\\"version\\\":\\\"v1\\\",\\\"versions\\\":[{\\\"name\\\":\\\"v1\\\",\\\"served\\\":true,\\\"storage\\\":true},{\\\"name\\\":\\\"v1beta1\\\",\\\"served\\\":true,\\\"storage\\\":false},{\\\"name\\\":\\\"v1alpha1\\\",\\\"served\\\":false,\\\"storage\\\":false}]},\\\"status\\\":{\\\"acceptedNames\\\":{\\\"kind\\\":\\\"\\\",\\\"plural\\\":\\\"\\\"},\\\"conditions\\\":[],\\\"storedVersions\\\":[]}}\\n\"\n },\n \"creationTimestamp\": \"2020-04-28T23:31:53Z\",\n \"generation\": 1,\n \"labels\": {\n \"velero.io/backup-name\": \"es\",\n \"velero.io/restore-name\": \"es-crds\"\n },\n \"name\": \"kibanas.kibana.k8s.elastic.co\",\n \"resourceVersion\": \"1703552\",\n \"selfLink\": \"/apis/apiextensions.k8s.io/v1/customresourcedefinitions/kibanas.kibana.k8s.elastic.co\",\n \"uid\": \"95f42a77-654f-4380-a6b1-1fe2587f0713\"\n },\n \"spec\": {\n \"conversion\": {\n \"strategy\": \"None\"\n },\n \"group\": \"kibana.k8s.elastic.co\",\n \"names\": {\n \"categories\": [\n \"elastic\"\n ],\n \"kind\": \"Kibana\",\n \"listKind\": \"KibanaList\",\n \"plural\": \"kibanas\",\n \"shortNames\": [\n \"kb\"\n ],\n \"singular\": \"kibana\"\n },\n \"preserveUnknownFields\": true,\n \"scope\": \"Namespaced\",\n \"versions\": [\n {\n \"additionalPrinterColumns\": [\n {\n \"jsonPath\": \".status.health\",\n \"name\": \"health\",\n \"type\": \"string\"\n },\n {\n \"description\": \"Available nodes\",\n \"jsonPath\": \".status.availableNodes\",\n \"name\": \"nodes\",\n \"type\": \"integer\"\n },\n {\n \"description\": \"Kibana version\",\n \"jsonPath\": \".spec.version\",\n \"name\": \"version\",\n \"type\": \"string\"\n },\n {\n \"jsonPath\": \".metadata.creationTimestamp\",\n \"name\": \"age\",\n \"type\": \"date\"\n }\n ],\n \"name\": \"v1\",\n \"schema\": {\n \"openAPIV3Schema\": {\n \"description\": \"Kibana represents a Kibana resource in a Kubernetes cluster.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"KibanaSpec holds the specification of a Kibana instance.\",\n \"properties\": {\n \"config\": {\n \"description\": \"Config holds the Kibana configuration. See: https://www.elastic.co/guide/en/kibana/current/settings.html\",\n \"type\": \"object\"\n },\n \"count\": {\n \"description\": \"Count of Kibana instances to deploy.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"elasticsearchRef\": {\n \"description\": \"ElasticsearchRef is a reference to an Elasticsearch cluster running in the same Kubernetes cluster.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the Kubernetes object.\",\n \"type\": \"string\"\n },\n \"namespace\": {\n \"description\": \"Namespace of the Kubernetes object. If empty, defaults to the current namespace.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"http\": {\n \"description\": \"HTTP holds the HTTP layer configuration for Kibana.\",\n \"properties\": {\n \"service\": {\n \"description\": \"Service defines the template for the associated Kubernetes Service object.\",\n \"properties\": {\n \"metadata\": {\n \"description\": \"ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored.\",\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec is the specification of the service.\",\n \"properties\": {\n \"clusterIP\": {\n \"description\": \"clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are \\\"None\\\", empty string (\\\"\\\"), or a valid IP address. \\\"None\\\" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"externalIPs\": {\n \"description\": \"externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"externalName\": {\n \"description\": \"externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName.\",\n \"type\": \"string\"\n },\n \"externalTrafficPolicy\": {\n \"description\": \"externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. \\\"Local\\\" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. \\\"Cluster\\\" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading.\",\n \"type\": \"string\"\n },\n \"healthCheckNodePort\": {\n \"description\": \"healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"ipFamily\": {\n \"description\": \"ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment.\",\n \"type\": \"string\"\n },\n \"loadBalancerIP\": {\n \"description\": \"Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.\",\n \"type\": \"string\"\n },\n \"loadBalancerSourceRanges\": {\n \"description\": \"If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature.\\\" More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"ports\": {\n \"description\": \"The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"items\": {\n \"description\": \"ServicePort contains information on service's port.\",\n \"properties\": {\n \"name\": {\n \"description\": \"The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service.\",\n \"type\": \"string\"\n },\n \"nodePort\": {\n \"description\": \"The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"port\": {\n \"description\": \"The port that will be exposed by this service.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"protocol\": {\n \"description\": \"The IP protocol for this port. Supports \\\"TCP\\\", \\\"UDP\\\", and \\\"SCTP\\\". Default is TCP.\",\n \"type\": \"string\"\n },\n \"targetPort\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod's container ports. If this is not specified, the value of the 'port' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the 'port' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"publishNotReadyAddresses\": {\n \"description\": \"publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery.\",\n \"type\": \"boolean\"\n },\n \"selector\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/\",\n \"type\": \"object\"\n },\n \"sessionAffinity\": {\n \"description\": \"Supports \\\"ClientIP\\\" and \\\"None\\\". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"sessionAffinityConfig\": {\n \"description\": \"sessionAffinityConfig contains the configurations of session affinity.\",\n \"properties\": {\n \"clientIP\": {\n \"description\": \"clientIP contains the configurations of Client IP based session affinity.\",\n \"properties\": {\n \"timeoutSeconds\": {\n \"description\": \"timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be \\u003e0 \\u0026\\u0026 \\u003c=86400(for 1 day) if ServiceAffinity == \\\"ClientIP\\\". Default value is 10800(for 3 hours).\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"topologyKeys\": {\n \"description\": \"topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value \\\"*\\\" may be used to mean \\\"any topology\\\". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"type\": {\n \"description\": \"type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. \\\"ExternalName\\\" maps to the specified externalName. \\\"ClusterIP\\\" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is \\\"None\\\", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. \\\"NodePort\\\" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. \\\"LoadBalancer\\\" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"tls\": {\n \"description\": \"TLS defines options for configuring TLS for HTTP.\",\n \"properties\": {\n \"certificate\": {\n \"description\": \"Certificate is a reference to a Kubernetes secret that contains the certificate and private key for enabling TLS. The referenced secret should contain the following: \\n - `ca.crt`: The certificate authority (optional). - `tls.crt`: The certificate (or a chain). - `tls.key`: The private key to the first certificate in the certificate chain.\",\n \"properties\": {\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"selfSignedCertificate\": {\n \"description\": \"SelfSignedCertificate allows configuring the self-signed certificate generated by the operator.\",\n \"properties\": {\n \"disabled\": {\n \"description\": \"Disabled indicates that the provisioning of the self-signed certificate should be disabled.\",\n \"type\": \"boolean\"\n },\n \"subjectAltNames\": {\n \"description\": \"SubjectAlternativeNames is a list of SANs to include in the generated HTTP TLS certificate.\",\n \"items\": {\n \"description\": \"SubjectAlternativeName represents a SAN entry in a x509 certificate.\",\n \"properties\": {\n \"dns\": {\n \"description\": \"DNS is the DNS name of the subject.\",\n \"type\": \"string\"\n },\n \"ip\": {\n \"description\": \"IP is the IP address of the subject.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"image\": {\n \"description\": \"Image is the Kibana Docker image to deploy.\",\n \"type\": \"string\"\n },\n \"podTemplate\": {\n \"description\": \"PodTemplate provides customisation options (labels, annotations, affinity rules, resource requests, and so on) for the Kibana pods\",\n \"type\": \"object\"\n },\n \"secureSettings\": {\n \"description\": \"SecureSettings is a list of references to Kubernetes secrets containing sensitive configuration options for Kibana. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-kibana.html#k8s-kibana-secure-settings\",\n \"items\": {\n \"description\": \"SecretSource defines a data source based on a Kubernetes Secret.\",\n \"properties\": {\n \"entries\": {\n \"description\": \"Entries define how to project each key-value pair in the secret to filesystem paths. If not defined, all keys will be projected to similarly named paths in the filesystem. If defined, only the specified keys will be projected to the corresponding paths.\",\n \"items\": {\n \"description\": \"KeyToPath defines how to map a key in a Secret object to a filesystem path.\",\n \"properties\": {\n \"key\": {\n \"description\": \"Key is the key contained in the secret.\",\n \"type\": \"string\"\n },\n \"path\": {\n \"description\": \"Path is the relative file path to map the key to. Path must not be an absolute file path and must not contain any \\\"..\\\" components.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"secretName\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"serviceAccountName\": {\n \"description\": \"ServiceAccountName is used to check access from the current resource to a resource (eg. Elasticsearch) in a different namespace. Can only be used if ECK is enforcing RBAC on references.\",\n \"type\": \"string\"\n },\n \"version\": {\n \"description\": \"Version of Kibana.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"version\"\n ],\n \"type\": \"object\"\n },\n \"status\": {\n \"description\": \"KibanaStatus defines the observed state of Kibana\",\n \"properties\": {\n \"associationStatus\": {\n \"description\": \"AssociationStatus is the status of an association resource.\",\n \"type\": \"string\"\n },\n \"availableNodes\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"health\": {\n \"description\": \"KibanaHealth expresses the status of the Kibana instances.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n }\n }\n },\n \"served\": true,\n \"storage\": true,\n \"subresources\": {\n \"status\": {}\n }\n },\n {\n \"additionalPrinterColumns\": [\n {\n \"jsonPath\": \".status.health\",\n \"name\": \"health\",\n \"type\": \"string\"\n },\n {\n \"description\": \"Available nodes\",\n \"jsonPath\": \".status.availableNodes\",\n \"name\": \"nodes\",\n \"type\": \"integer\"\n },\n {\n \"description\": \"Kibana version\",\n \"jsonPath\": \".spec.version\",\n \"name\": \"version\",\n \"type\": \"string\"\n },\n {\n \"jsonPath\": \".metadata.creationTimestamp\",\n \"name\": \"age\",\n \"type\": \"date\"\n }\n ],\n \"name\": \"v1beta1\",\n \"schema\": {\n \"openAPIV3Schema\": {\n \"description\": \"Kibana represents a Kibana resource in a Kubernetes cluster.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"KibanaSpec holds the specification of a Kibana instance.\",\n \"properties\": {\n \"config\": {\n \"description\": \"Config holds the Kibana configuration. See: https://www.elastic.co/guide/en/kibana/current/settings.html\",\n \"type\": \"object\"\n },\n \"count\": {\n \"description\": \"Count of Kibana instances to deploy.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"elasticsearchRef\": {\n \"description\": \"ElasticsearchRef is a reference to an Elasticsearch cluster running in the same Kubernetes cluster.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the Kubernetes object.\",\n \"type\": \"string\"\n },\n \"namespace\": {\n \"description\": \"Namespace of the Kubernetes object. If empty, defaults to the current namespace.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"http\": {\n \"description\": \"HTTP holds the HTTP layer configuration for Kibana.\",\n \"properties\": {\n \"service\": {\n \"description\": \"Service defines the template for the associated Kubernetes Service object.\",\n \"properties\": {\n \"metadata\": {\n \"description\": \"ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored.\",\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec is the specification of the service.\",\n \"properties\": {\n \"clusterIP\": {\n \"description\": \"clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are \\\"None\\\", empty string (\\\"\\\"), or a valid IP address. \\\"None\\\" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"externalIPs\": {\n \"description\": \"externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"externalName\": {\n \"description\": \"externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName.\",\n \"type\": \"string\"\n },\n \"externalTrafficPolicy\": {\n \"description\": \"externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. \\\"Local\\\" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. \\\"Cluster\\\" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading.\",\n \"type\": \"string\"\n },\n \"healthCheckNodePort\": {\n \"description\": \"healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"ipFamily\": {\n \"description\": \"ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment.\",\n \"type\": \"string\"\n },\n \"loadBalancerIP\": {\n \"description\": \"Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.\",\n \"type\": \"string\"\n },\n \"loadBalancerSourceRanges\": {\n \"description\": \"If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature.\\\" More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"ports\": {\n \"description\": \"The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"items\": {\n \"description\": \"ServicePort contains information on service's port.\",\n \"properties\": {\n \"name\": {\n \"description\": \"The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service.\",\n \"type\": \"string\"\n },\n \"nodePort\": {\n \"description\": \"The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"port\": {\n \"description\": \"The port that will be exposed by this service.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"protocol\": {\n \"description\": \"The IP protocol for this port. Supports \\\"TCP\\\", \\\"UDP\\\", and \\\"SCTP\\\". Default is TCP.\",\n \"type\": \"string\"\n },\n \"targetPort\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod's container ports. If this is not specified, the value of the 'port' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the 'port' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"publishNotReadyAddresses\": {\n \"description\": \"publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery.\",\n \"type\": \"boolean\"\n },\n \"selector\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/\",\n \"type\": \"object\"\n },\n \"sessionAffinity\": {\n \"description\": \"Supports \\\"ClientIP\\\" and \\\"None\\\". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"sessionAffinityConfig\": {\n \"description\": \"sessionAffinityConfig contains the configurations of session affinity.\",\n \"properties\": {\n \"clientIP\": {\n \"description\": \"clientIP contains the configurations of Client IP based session affinity.\",\n \"properties\": {\n \"timeoutSeconds\": {\n \"description\": \"timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be \\u003e0 \\u0026\\u0026 \\u003c=86400(for 1 day) if ServiceAffinity == \\\"ClientIP\\\". Default value is 10800(for 3 hours).\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"topologyKeys\": {\n \"description\": \"topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value \\\"*\\\" may be used to mean \\\"any topology\\\". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"type\": {\n \"description\": \"type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. \\\"ExternalName\\\" maps to the specified externalName. \\\"ClusterIP\\\" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is \\\"None\\\", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. \\\"NodePort\\\" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. \\\"LoadBalancer\\\" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"tls\": {\n \"description\": \"TLS defines options for configuring TLS for HTTP.\",\n \"properties\": {\n \"certificate\": {\n \"description\": \"Certificate is a reference to a Kubernetes secret that contains the certificate and private key for enabling TLS. The referenced secret should contain the following: \\n - `ca.crt`: The certificate authority (optional). - `tls.crt`: The certificate (or a chain). - `tls.key`: The private key to the first certificate in the certificate chain.\",\n \"properties\": {\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"selfSignedCertificate\": {\n \"description\": \"SelfSignedCertificate allows configuring the self-signed certificate generated by the operator.\",\n \"properties\": {\n \"disabled\": {\n \"description\": \"Disabled indicates that the provisioning of the self-signed certificate should be disabled.\",\n \"type\": \"boolean\"\n },\n \"subjectAltNames\": {\n \"description\": \"SubjectAlternativeNames is a list of SANs to include in the generated HTTP TLS certificate.\",\n \"items\": {\n \"description\": \"SubjectAlternativeName represents a SAN entry in a x509 certificate.\",\n \"properties\": {\n \"dns\": {\n \"description\": \"DNS is the DNS name of the subject.\",\n \"type\": \"string\"\n },\n \"ip\": {\n \"description\": \"IP is the IP address of the subject.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"image\": {\n \"description\": \"Image is the Kibana Docker image to deploy.\",\n \"type\": \"string\"\n },\n \"podTemplate\": {\n \"description\": \"PodTemplate provides customisation options (labels, annotations, affinity rules, resource requests, and so on) for the Kibana pods\",\n \"type\": \"object\"\n },\n \"secureSettings\": {\n \"description\": \"SecureSettings is a list of references to Kubernetes secrets containing sensitive configuration options for Kibana. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-kibana.html#k8s-kibana-secure-settings\",\n \"items\": {\n \"description\": \"SecretSource defines a data source based on a Kubernetes Secret.\",\n \"properties\": {\n \"entries\": {\n \"description\": \"Entries define how to project each key-value pair in the secret to filesystem paths. If not defined, all keys will be projected to similarly named paths in the filesystem. If defined, only the specified keys will be projected to the corresponding paths.\",\n \"items\": {\n \"description\": \"KeyToPath defines how to map a key in a Secret object to a filesystem path.\",\n \"properties\": {\n \"key\": {\n \"description\": \"Key is the key contained in the secret.\",\n \"type\": \"string\"\n },\n \"path\": {\n \"description\": \"Path is the relative file path to map the key to. Path must not be an absolute file path and must not contain any \\\"..\\\" components.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"secretName\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"serviceAccountName\": {\n \"description\": \"ServiceAccountName is used to check access from the current resource to a resource (eg. Elasticsearch) in a different namespace. Can only be used if ECK is enforcing RBAC on references.\",\n \"type\": \"string\"\n },\n \"version\": {\n \"description\": \"Version of Kibana.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"version\"\n ],\n \"type\": \"object\"\n },\n \"status\": {\n \"description\": \"KibanaStatus defines the observed state of Kibana\",\n \"properties\": {\n \"associationStatus\": {\n \"description\": \"AssociationStatus is the status of an association resource.\",\n \"type\": \"string\"\n },\n \"availableNodes\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"health\": {\n \"description\": \"KibanaHealth expresses the status of the Kibana instances.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n }\n }\n },\n \"served\": true,\n \"storage\": false,\n \"subresources\": {\n \"status\": {}\n }\n },\n {\n \"additionalPrinterColumns\": [\n {\n \"jsonPath\": \".status.health\",\n \"name\": \"health\",\n \"type\": \"string\"\n },\n {\n \"description\": \"Available nodes\",\n \"jsonPath\": \".status.availableNodes\",\n \"name\": \"nodes\",\n \"type\": \"integer\"\n },\n {\n \"description\": \"Kibana version\",\n \"jsonPath\": \".spec.version\",\n \"name\": \"version\",\n \"type\": \"string\"\n },\n {\n \"jsonPath\": \".metadata.creationTimestamp\",\n \"name\": \"age\",\n \"type\": \"date\"\n }\n ],\n \"name\": \"v1alpha1\",\n \"schema\": {\n \"openAPIV3Schema\": {\n \"description\": \"Kibana represents a Kibana resource in a Kubernetes cluster.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"KibanaSpec holds the specification of a Kibana instance.\",\n \"properties\": {\n \"config\": {\n \"description\": \"Config holds the Kibana configuration. See: https://www.elastic.co/guide/en/kibana/current/settings.html\",\n \"type\": \"object\"\n },\n \"count\": {\n \"description\": \"Count of Kibana instances to deploy.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"elasticsearchRef\": {\n \"description\": \"ElasticsearchRef is a reference to an Elasticsearch cluster running in the same Kubernetes cluster.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the Kubernetes object.\",\n \"type\": \"string\"\n },\n \"namespace\": {\n \"description\": \"Namespace of the Kubernetes object. If empty, defaults to the current namespace.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"http\": {\n \"description\": \"HTTP holds the HTTP layer configuration for Kibana.\",\n \"properties\": {\n \"service\": {\n \"description\": \"Service defines the template for the associated Kubernetes Service object.\",\n \"properties\": {\n \"metadata\": {\n \"description\": \"ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored.\",\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec is the specification of the service.\",\n \"properties\": {\n \"clusterIP\": {\n \"description\": \"clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are \\\"None\\\", empty string (\\\"\\\"), or a valid IP address. \\\"None\\\" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"externalIPs\": {\n \"description\": \"externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"externalName\": {\n \"description\": \"externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName.\",\n \"type\": \"string\"\n },\n \"externalTrafficPolicy\": {\n \"description\": \"externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. \\\"Local\\\" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. \\\"Cluster\\\" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading.\",\n \"type\": \"string\"\n },\n \"healthCheckNodePort\": {\n \"description\": \"healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"ipFamily\": {\n \"description\": \"ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment.\",\n \"type\": \"string\"\n },\n \"loadBalancerIP\": {\n \"description\": \"Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.\",\n \"type\": \"string\"\n },\n \"loadBalancerSourceRanges\": {\n \"description\": \"If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature.\\\" More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"ports\": {\n \"description\": \"The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"items\": {\n \"description\": \"ServicePort contains information on service's port.\",\n \"properties\": {\n \"name\": {\n \"description\": \"The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service.\",\n \"type\": \"string\"\n },\n \"nodePort\": {\n \"description\": \"The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"port\": {\n \"description\": \"The port that will be exposed by this service.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"protocol\": {\n \"description\": \"The IP protocol for this port. Supports \\\"TCP\\\", \\\"UDP\\\", and \\\"SCTP\\\". Default is TCP.\",\n \"type\": \"string\"\n },\n \"targetPort\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod's container ports. If this is not specified, the value of the 'port' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the 'port' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"publishNotReadyAddresses\": {\n \"description\": \"publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery.\",\n \"type\": \"boolean\"\n },\n \"selector\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/\",\n \"type\": \"object\"\n },\n \"sessionAffinity\": {\n \"description\": \"Supports \\\"ClientIP\\\" and \\\"None\\\". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"sessionAffinityConfig\": {\n \"description\": \"sessionAffinityConfig contains the configurations of session affinity.\",\n \"properties\": {\n \"clientIP\": {\n \"description\": \"clientIP contains the configurations of Client IP based session affinity.\",\n \"properties\": {\n \"timeoutSeconds\": {\n \"description\": \"timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be \\u003e0 \\u0026\\u0026 \\u003c=86400(for 1 day) if ServiceAffinity == \\\"ClientIP\\\". Default value is 10800(for 3 hours).\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"topologyKeys\": {\n \"description\": \"topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value \\\"*\\\" may be used to mean \\\"any topology\\\". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"type\": {\n \"description\": \"type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. \\\"ExternalName\\\" maps to the specified externalName. \\\"ClusterIP\\\" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is \\\"None\\\", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. \\\"NodePort\\\" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. \\\"LoadBalancer\\\" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"tls\": {\n \"description\": \"TLS defines options for configuring TLS for HTTP.\",\n \"properties\": {\n \"certificate\": {\n \"description\": \"Certificate is a reference to a Kubernetes secret that contains the certificate and private key for enabling TLS. The referenced secret should contain the following: \\n - `ca.crt`: The certificate authority (optional). - `tls.crt`: The certificate (or a chain). - `tls.key`: The private key to the first certificate in the certificate chain.\",\n \"properties\": {\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"selfSignedCertificate\": {\n \"description\": \"SelfSignedCertificate allows configuring the self-signed certificate generated by the operator.\",\n \"properties\": {\n \"disabled\": {\n \"description\": \"Disabled indicates that the provisioning of the self-signed certificate should be disabled.\",\n \"type\": \"boolean\"\n },\n \"subjectAltNames\": {\n \"description\": \"SubjectAlternativeNames is a list of SANs to include in the generated HTTP TLS certificate.\",\n \"items\": {\n \"description\": \"SubjectAlternativeName represents a SAN entry in a x509 certificate.\",\n \"properties\": {\n \"dns\": {\n \"description\": \"DNS is the DNS name of the subject.\",\n \"type\": \"string\"\n },\n \"ip\": {\n \"description\": \"IP is the IP address of the subject.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"image\": {\n \"description\": \"Image is the Kibana Docker image to deploy.\",\n \"type\": \"string\"\n },\n \"podTemplate\": {\n \"description\": \"PodTemplate provides customisation options (labels, annotations, affinity rules, resource requests, and so on) for the Kibana pods\",\n \"type\": \"object\"\n },\n \"secureSettings\": {\n \"description\": \"SecureSettings is a list of references to Kubernetes secrets containing sensitive configuration options for Kibana. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-kibana.html#k8s-kibana-secure-settings\",\n \"items\": {\n \"description\": \"SecretSource defines a data source based on a Kubernetes Secret.\",\n \"properties\": {\n \"entries\": {\n \"description\": \"Entries define how to project each key-value pair in the secret to filesystem paths. If not defined, all keys will be projected to similarly named paths in the filesystem. If defined, only the specified keys will be projected to the corresponding paths.\",\n \"items\": {\n \"description\": \"KeyToPath defines how to map a key in a Secret object to a filesystem path.\",\n \"properties\": {\n \"key\": {\n \"description\": \"Key is the key contained in the secret.\",\n \"type\": \"string\"\n },\n \"path\": {\n \"description\": \"Path is the relative file path to map the key to. Path must not be an absolute file path and must not contain any \\\"..\\\" components.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"secretName\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"serviceAccountName\": {\n \"description\": \"ServiceAccountName is used to check access from the current resource to a resource (eg. Elasticsearch) in a different namespace. Can only be used if ECK is enforcing RBAC on references.\",\n \"type\": \"string\"\n },\n \"version\": {\n \"description\": \"Version of Kibana.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"version\"\n ],\n \"type\": \"object\"\n },\n \"status\": {\n \"description\": \"KibanaStatus defines the observed state of Kibana\",\n \"properties\": {\n \"associationStatus\": {\n \"description\": \"AssociationStatus is the status of an association resource.\",\n \"type\": \"string\"\n },\n \"availableNodes\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"health\": {\n \"description\": \"KibanaHealth expresses the status of the Kibana instances.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n }\n }\n },\n \"served\": false,\n \"storage\": false,\n \"subresources\": {\n \"status\": {}\n }\n }\n ]\n },\n \"status\": {\n \"acceptedNames\": {\n \"categories\": [\n \"elastic\"\n ],\n \"kind\": \"Kibana\",\n \"listKind\": \"KibanaList\",\n \"plural\": \"kibanas\",\n \"shortNames\": [\n \"kb\"\n ],\n \"singular\": \"kibana\"\n },\n \"conditions\": [\n {\n \"lastTransitionTime\": \"2020-04-28T23:31:53Z\",\n \"message\": \"[spec.validation.openAPIV3Schema.properties[spec].properties[http].properties[service].properties[spec].properties[ports].items.properties[targetPort].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.type: Required value: must not be empty at the root]\",\n \"reason\": \"Violations\",\n \"status\": \"True\",\n \"type\": \"NonStructuralSchema\"\n },\n {\n \"lastTransitionTime\": \"2020-04-28T23:31:53Z\",\n \"message\": \"no conflicts found\",\n \"reason\": \"NoConflicts\",\n \"status\": \"True\",\n \"type\": \"NamesAccepted\"\n },\n {\n \"lastTransitionTime\": \"2020-04-28T23:31:53Z\",\n \"message\": \"the initial names have been accepted\",\n \"reason\": \"InitialNamesAccepted\",\n \"status\": \"True\",\n \"type\": \"Established\"\n }\n ],\n \"storedVersions\": [\n \"v1\"\n ]\n }\n}\n" }, { "path": "pkg/backup/actions/testdata/v1/pprometheuses.monitoring.coreos.com.json", "content": "{\n \"apiVersion\": \"apiextensions.k8s.io/v1\",\n \"kind\": \"CustomResourceDefinition\",\n \"metadata\": {\n \"annotations\": {\n \"controller-gen.kubebuilder.io/version\": \"v0.2.4\",\n \"kubectl.kubernetes.io/last-applied-configuration\": \"{\\\"apiVersion\\\":\\\"apiextensions.k8s.io/v1beta1\\\",\\\"kind\\\":\\\"CustomResourceDefinition\\\",\\\"metadata\\\":{\\\"annotations\\\":{\\\"controller-gen.kubebuilder.io/version\\\":\\\"v0.2.4\\\"},\\\"creationTimestamp\\\":null,\\\"name\\\":\\\"prometheuses.monitoring.coreos.com\\\"},\\\"spec\\\":{\\\"additionalPrinterColumns\\\":[{\\\"JSONPath\\\":\\\".spec.version\\\",\\\"description\\\":\\\"The version of Prometheus\\\",\\\"name\\\":\\\"Version\\\",\\\"type\\\":\\\"string\\\"},{\\\"JSONPath\\\":\\\".spec.replicas\\\",\\\"description\\\":\\\"The desired replicas number of Prometheuses\\\",\\\"name\\\":\\\"Replicas\\\",\\\"type\\\":\\\"integer\\\"},{\\\"JSONPath\\\":\\\".metadata.creationTimestamp\\\",\\\"name\\\":\\\"Age\\\",\\\"type\\\":\\\"date\\\"}],\\\"group\\\":\\\"monitoring.coreos.com\\\",\\\"names\\\":{\\\"kind\\\":\\\"Prometheus\\\",\\\"listKind\\\":\\\"PrometheusList\\\",\\\"plural\\\":\\\"prometheuses\\\",\\\"singular\\\":\\\"prometheus\\\"},\\\"preserveUnknownFields\\\":false,\\\"scope\\\":\\\"Namespaced\\\",\\\"subresources\\\":{},\\\"validation\\\":{\\\"openAPIV3Schema\\\":{\\\"description\\\":\\\"Prometheus defines a Prometheus deployment.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"metadata\\\":{\\\"type\\\":\\\"object\\\"},\\\"spec\\\":{\\\"description\\\":\\\"Specification of the desired behavior of the Prometheus cluster. More info: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#spec-and-status\\\",\\\"properties\\\":{\\\"additionalAlertManagerConfigs\\\":{\\\"description\\\":\\\"AdditionalAlertManagerConfigs allows specifying a key of a Secret containing additional Prometheus AlertManager configurations. AlertManager configurations specified are appended to the configurations generated by the Prometheus Operator. Job configurations specified must have the form as specified in the official Prometheus documentation: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#alertmanager_config. As AlertManager configs are appended, the user is responsible to make sure it is valid. Note that using this feature may expose the possibility to break upgrades of Prometheus. It is advised to review Prometheus release notes to ensure that no incompatible AlertManager configs are going to break Prometheus after the upgrade.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"additionalAlertRelabelConfigs\\\":{\\\"description\\\":\\\"AdditionalAlertRelabelConfigs allows specifying a key of a Secret containing additional Prometheus alert relabel configurations. Alert relabel configurations specified are appended to the configurations generated by the Prometheus Operator. Alert relabel configurations specified must have the form as specified in the official Prometheus documentation: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#alert_relabel_configs. As alert relabel configs are appended, the user is responsible to make sure it is valid. Note that using this feature may expose the possibility to break upgrades of Prometheus. It is advised to review Prometheus release notes to ensure that no incompatible alert relabel configs are going to break Prometheus after the upgrade.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"additionalScrapeConfigs\\\":{\\\"description\\\":\\\"AdditionalScrapeConfigs allows specifying a key of a Secret containing additional Prometheus scrape configurations. Scrape configurations specified are appended to the configurations generated by the Prometheus Operator. Job configurations specified must have the form as specified in the official Prometheus documentation: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config. As scrape configs are appended, the user is responsible to make sure it is valid. Note that using this feature may expose the possibility to break upgrades of Prometheus. It is advised to review Prometheus release notes to ensure that no incompatible scrape configs are going to break Prometheus after the upgrade.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"affinity\\\":{\\\"description\\\":\\\"If specified, the pod's scheduling constraints.\\\",\\\"properties\\\":{\\\"nodeAffinity\\\":{\\\"description\\\":\\\"Describes node affinity scheduling rules for the pod.\\\",\\\"properties\\\":{\\\"preferredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\\\\\"weight\\\\\\\" to the sum if the node matches the corresponding matchExpressions; the node(s) with the highest sum are the most preferred.\\\",\\\"items\\\":{\\\"description\\\":\\\"An empty preferred scheduling term matches all objects with implicit weight 0 (i.e. it's a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op).\\\",\\\"properties\\\":{\\\"preference\\\":{\\\"description\\\":\\\"A node selector term, associated with the corresponding weight.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"A list of node selector requirements by node's labels.\\\",\\\"items\\\":{\\\"description\\\":\\\"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchFields\\\":{\\\"description\\\":\\\"A list of node selector requirements by node's fields.\\\",\\\"items\\\":{\\\"description\\\":\\\"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"weight\\\":{\\\"description\\\":\\\"Weight associated with matching the corresponding nodeSelectorTerm, in the range 1-100.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"required\\\":[\\\"preference\\\",\\\"weight\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"requiredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to an update), the system may or may not try to eventually evict the pod from its node.\\\",\\\"properties\\\":{\\\"nodeSelectorTerms\\\":{\\\"description\\\":\\\"Required. A list of node selector terms. The terms are ORed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"A list of node selector requirements by node's labels.\\\",\\\"items\\\":{\\\"description\\\":\\\"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchFields\\\":{\\\"description\\\":\\\"A list of node selector requirements by node's fields.\\\",\\\"items\\\":{\\\"description\\\":\\\"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"nodeSelectorTerms\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"podAffinity\\\":{\\\"description\\\":\\\"Describes pod affinity scheduling rules (e.g. co-locate this pod in the same node, zone, etc. as some other pod(s)).\\\",\\\"properties\\\":{\\\"preferredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\\\\\"weight\\\\\\\" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.\\\",\\\"items\\\":{\\\"description\\\":\\\"The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)\\\",\\\"properties\\\":{\\\"podAffinityTerm\\\":{\\\"description\\\":\\\"Required. A pod affinity term, associated with the corresponding weight.\\\",\\\"properties\\\":{\\\"labelSelector\\\":{\\\"description\\\":\\\"A label query over a set of resources, in this case pods.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"namespaces\\\":{\\\"description\\\":\\\"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\\\\\"this pod's namespace\\\\\\\"\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"topologyKey\\\":{\\\"description\\\":\\\"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"topologyKey\\\"],\\\"type\\\":\\\"object\\\"},\\\"weight\\\":{\\\"description\\\":\\\"weight associated with matching the corresponding podAffinityTerm, in the range 1-100.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"required\\\":[\\\"podAffinityTerm\\\",\\\"weight\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"requiredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.\\\",\\\"items\\\":{\\\"description\\\":\\\"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\\\u003ctopologyKey\\\\u003e matches that of any node on which a pod of the set of pods is running\\\",\\\"properties\\\":{\\\"labelSelector\\\":{\\\"description\\\":\\\"A label query over a set of resources, in this case pods.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"namespaces\\\":{\\\"description\\\":\\\"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\\\\\"this pod's namespace\\\\\\\"\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"topologyKey\\\":{\\\"description\\\":\\\"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"topologyKey\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"podAntiAffinity\\\":{\\\"description\\\":\\\"Describes pod anti-affinity scheduling rules (e.g. avoid putting this pod in the same node, zone, etc. as some other pod(s)).\\\",\\\"properties\\\":{\\\"preferredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"The scheduler will prefer to schedule pods to nodes that satisfy the anti-affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling anti-affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\\\\\"weight\\\\\\\" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.\\\",\\\"items\\\":{\\\"description\\\":\\\"The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)\\\",\\\"properties\\\":{\\\"podAffinityTerm\\\":{\\\"description\\\":\\\"Required. A pod affinity term, associated with the corresponding weight.\\\",\\\"properties\\\":{\\\"labelSelector\\\":{\\\"description\\\":\\\"A label query over a set of resources, in this case pods.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"namespaces\\\":{\\\"description\\\":\\\"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\\\\\"this pod's namespace\\\\\\\"\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"topologyKey\\\":{\\\"description\\\":\\\"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"topologyKey\\\"],\\\"type\\\":\\\"object\\\"},\\\"weight\\\":{\\\"description\\\":\\\"weight associated with matching the corresponding podAffinityTerm, in the range 1-100.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"required\\\":[\\\"podAffinityTerm\\\",\\\"weight\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"requiredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"If the anti-affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the anti-affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.\\\",\\\"items\\\":{\\\"description\\\":\\\"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\\\u003ctopologyKey\\\\u003e matches that of any node on which a pod of the set of pods is running\\\",\\\"properties\\\":{\\\"labelSelector\\\":{\\\"description\\\":\\\"A label query over a set of resources, in this case pods.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"namespaces\\\":{\\\"description\\\":\\\"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\\\\\"this pod's namespace\\\\\\\"\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"topologyKey\\\":{\\\"description\\\":\\\"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"topologyKey\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"alerting\\\":{\\\"description\\\":\\\"Define details regarding alerting.\\\",\\\"properties\\\":{\\\"alertmanagers\\\":{\\\"description\\\":\\\"AlertmanagerEndpoints Prometheus should fire alerts against.\\\",\\\"items\\\":{\\\"description\\\":\\\"AlertmanagerEndpoints defines a selection of a single Endpoints object containing alertmanager IPs to fire alerts against.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"Version of the Alertmanager API that Prometheus uses to send alerts. It can be \\\\\\\"v1\\\\\\\" or \\\\\\\"v2\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"},\\\"bearerTokenFile\\\":{\\\"description\\\":\\\"BearerTokenFile to read from filesystem to use when authenticating to Alertmanager.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of Endpoints object in Namespace.\\\",\\\"type\\\":\\\"string\\\"},\\\"namespace\\\":{\\\"description\\\":\\\"Namespace of Endpoints object.\\\",\\\"type\\\":\\\"string\\\"},\\\"pathPrefix\\\":{\\\"description\\\":\\\"Prefix for the HTTP path alerts are pushed to.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Port the Alertmanager API is exposed on.\\\",\\\"x-kubernetes-int-or-string\\\":true},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use when firing alerts.\\\",\\\"type\\\":\\\"string\\\"},\\\"tlsConfig\\\":{\\\"description\\\":\\\"TLS Config to use for alertmanager connection.\\\",\\\"properties\\\":{\\\"ca\\\":{\\\"description\\\":\\\"Struct containing the CA cert to use for the targets.\\\",\\\"properties\\\":{\\\"configMap\\\":{\\\"description\\\":\\\"ConfigMap containing data to use for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key to select.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"secret\\\":{\\\"description\\\":\\\"Secret containing data to use for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"caFile\\\":{\\\"description\\\":\\\"Path to the CA cert in the Prometheus container to use for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"cert\\\":{\\\"description\\\":\\\"Struct containing the client cert file for the targets.\\\",\\\"properties\\\":{\\\"configMap\\\":{\\\"description\\\":\\\"ConfigMap containing data to use for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key to select.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"secret\\\":{\\\"description\\\":\\\"Secret containing data to use for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"certFile\\\":{\\\"description\\\":\\\"Path to the client cert file in the Prometheus container for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"insecureSkipVerify\\\":{\\\"description\\\":\\\"Disable target certificate validation.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"keyFile\\\":{\\\"description\\\":\\\"Path to the client key file in the Prometheus container for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"keySecret\\\":{\\\"description\\\":\\\"Secret containing the client key file for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"serverName\\\":{\\\"description\\\":\\\"Used to verify the hostname for the targets.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"required\\\":[\\\"name\\\",\\\"namespace\\\",\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"alertmanagers\\\"],\\\"type\\\":\\\"object\\\"},\\\"apiserverConfig\\\":{\\\"description\\\":\\\"APIServerConfig allows specifying a host and auth methods to access apiserver. If left empty, Prometheus is assumed to run inside of the cluster and will discover API servers automatically and use the pod's CA certificate and bearer token file at /var/run/secrets/kubernetes.io/serviceaccount/.\\\",\\\"properties\\\":{\\\"basicAuth\\\":{\\\"description\\\":\\\"BasicAuth allow an endpoint to authenticate over basic authentication\\\",\\\"properties\\\":{\\\"password\\\":{\\\"description\\\":\\\"The secret in the service monitor namespace that contains the password for authentication.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"username\\\":{\\\"description\\\":\\\"The secret in the service monitor namespace that contains the username for authentication.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"bearerToken\\\":{\\\"description\\\":\\\"Bearer token for accessing apiserver.\\\",\\\"type\\\":\\\"string\\\"},\\\"bearerTokenFile\\\":{\\\"description\\\":\\\"File to read bearer token for accessing apiserver.\\\",\\\"type\\\":\\\"string\\\"},\\\"host\\\":{\\\"description\\\":\\\"Host of apiserver. A valid string consisting of a hostname or IP followed by an optional port number\\\",\\\"type\\\":\\\"string\\\"},\\\"tlsConfig\\\":{\\\"description\\\":\\\"TLS Config to use for accessing apiserver.\\\",\\\"properties\\\":{\\\"ca\\\":{\\\"description\\\":\\\"Struct containing the CA cert to use for the targets.\\\",\\\"properties\\\":{\\\"configMap\\\":{\\\"description\\\":\\\"ConfigMap containing data to use for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key to select.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"secret\\\":{\\\"description\\\":\\\"Secret containing data to use for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"caFile\\\":{\\\"description\\\":\\\"Path to the CA cert in the Prometheus container to use for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"cert\\\":{\\\"description\\\":\\\"Struct containing the client cert file for the targets.\\\",\\\"properties\\\":{\\\"configMap\\\":{\\\"description\\\":\\\"ConfigMap containing data to use for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key to select.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"secret\\\":{\\\"description\\\":\\\"Secret containing data to use for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"certFile\\\":{\\\"description\\\":\\\"Path to the client cert file in the Prometheus container for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"insecureSkipVerify\\\":{\\\"description\\\":\\\"Disable target certificate validation.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"keyFile\\\":{\\\"description\\\":\\\"Path to the client key file in the Prometheus container for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"keySecret\\\":{\\\"description\\\":\\\"Secret containing the client key file for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"serverName\\\":{\\\"description\\\":\\\"Used to verify the hostname for the targets.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"required\\\":[\\\"host\\\"],\\\"type\\\":\\\"object\\\"},\\\"arbitraryFSAccessThroughSMs\\\":{\\\"description\\\":\\\"ArbitraryFSAccessThroughSMs configures whether configuration based on a service monitor can access arbitrary files on the file system of the Prometheus container e.g. bearer token files.\\\",\\\"properties\\\":{\\\"deny\\\":{\\\"type\\\":\\\"boolean\\\"}},\\\"type\\\":\\\"object\\\"},\\\"baseImage\\\":{\\\"description\\\":\\\"Base image to use for a Prometheus deployment.\\\",\\\"type\\\":\\\"string\\\"},\\\"configMaps\\\":{\\\"description\\\":\\\"ConfigMaps is a list of ConfigMaps in the same namespace as the Prometheus object, which shall be mounted into the Prometheus Pods. The ConfigMaps are mounted into /etc/prometheus/configmaps/\\\\u003cconfigmap-name\\\\u003e.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"containers\\\":{\\\"description\\\":\\\"Containers allows injecting additional containers or modifying operator generated containers. This can be used to allow adding an authentication proxy to a Prometheus pod or to change the behavior of an operator generated container. Containers described here modify an operator generated container if they share the same name and modifications are done via a strategic merge patch. The current container names are: `prometheus`, `prometheus-config-reloader`, `rules-configmap-reloader`, and `thanos-sidecar`. Overriding containers is entirely outside the scope of what the maintainers will support and by doing so, you accept that this behaviour may break at any time without notice.\\\",\\\"items\\\":{\\\"description\\\":\\\"A single application container that you want to run within a pod.\\\",\\\"properties\\\":{\\\"args\\\":{\\\"description\\\":\\\"Arguments to the entrypoint. The docker image's CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"command\\\":{\\\"description\\\":\\\"Entrypoint array. Not executed within a shell. The docker image's ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"env\\\":{\\\"description\\\":\\\"List of environment variables to set in the container. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"EnvVar represents an environment variable present in a Container.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the environment variable. Must be a C_IDENTIFIER.\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to \\\\\\\"\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"},\\\"valueFrom\\\":{\\\"description\\\":\\\"Source for the environment variable's value. Cannot be used if value is not empty.\\\",\\\"properties\\\":{\\\"configMapKeyRef\\\":{\\\"description\\\":\\\"Selects a key of a ConfigMap.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key to select.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"fieldRef\\\":{\\\"description\\\":\\\"Selects a field of the pod: supports metadata.name, metadata.namespace, metadata.labels, metadata.annotations, spec.nodeName, spec.serviceAccountName, status.hostIP, status.podIP, status.podIPs.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"Version of the schema the FieldPath is written in terms of, defaults to \\\\\\\"v1\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"},\\\"fieldPath\\\":{\\\"description\\\":\\\"Path of the field to select in the specified API version.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"fieldPath\\\"],\\\"type\\\":\\\"object\\\"},\\\"resourceFieldRef\\\":{\\\"description\\\":\\\"Selects a resource of the container: only resources limits and requests (limits.cpu, limits.memory, limits.ephemeral-storage, requests.cpu, requests.memory and requests.ephemeral-storage) are currently supported.\\\",\\\"properties\\\":{\\\"containerName\\\":{\\\"description\\\":\\\"Container name: required for volumes, optional for env vars\\\",\\\"type\\\":\\\"string\\\"},\\\"divisor\\\":{\\\"description\\\":\\\"Specifies the output format of the exposed resources, defaults to \\\\\\\"1\\\\\\\"\\\",\\\"type\\\":\\\"string\\\"},\\\"resource\\\":{\\\"description\\\":\\\"Required: resource to select\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"resource\\\"],\\\"type\\\":\\\"object\\\"},\\\"secretKeyRef\\\":{\\\"description\\\":\\\"Selects a key of a secret in the pod's namespace\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"required\\\":[\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"envFrom\\\":{\\\"description\\\":\\\"List of sources to populate environment variables in the container. The keys defined within a source must be a C_IDENTIFIER. All invalid keys will be reported as an event when the container is starting. When a key exists in multiple sources, the value associated with the last source will take precedence. Values defined by an Env with a duplicate key will take precedence. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"EnvFromSource represents the source of a set of ConfigMaps\\\",\\\"properties\\\":{\\\"configMapRef\\\":{\\\"description\\\":\\\"The ConfigMap to select from\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"type\\\":\\\"object\\\"},\\\"prefix\\\":{\\\"description\\\":\\\"An optional identifier to prepend to each key in the ConfigMap. Must be a C_IDENTIFIER.\\\",\\\"type\\\":\\\"string\\\"},\\\"secretRef\\\":{\\\"description\\\":\\\"The Secret to select from\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"image\\\":{\\\"description\\\":\\\"Docker image name. More info: https://kubernetes.io/docs/concepts/containers/images This field is optional to allow higher level config management to default or override container images in workload controllers like Deployments and StatefulSets.\\\",\\\"type\\\":\\\"string\\\"},\\\"imagePullPolicy\\\":{\\\"description\\\":\\\"Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images\\\",\\\"type\\\":\\\"string\\\"},\\\"lifecycle\\\":{\\\"description\\\":\\\"Actions that the management system should take in response to container lifecycle events. Cannot be updated.\\\",\\\"properties\\\":{\\\"postStart\\\":{\\\"description\\\":\\\"PostStart is called immediately after a container is created. If the handler fails, the container is terminated and restarted according to its restart policy. Other management of the container blocks until the hook completes. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"One and only one of the following should be specified. Exec specifies the action to take.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGet specifies the http request to perform.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\\\",\\\"x-kubernetes-int-or-string\\\":true},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported TODO: implement a realistic TCP lifecycle hook\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Number or name of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\\\",\\\"x-kubernetes-int-or-string\\\":true}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"preStop\\\":{\\\"description\\\":\\\"PreStop is called immediately before a container is terminated due to an API request or management event such as liveness/startup probe failure, preemption, resource contention, etc. The handler is not called if the container crashes or exits. The reason for termination is passed to the handler. The Pod's termination grace period countdown begins before the PreStop hooked is executed. Regardless of the outcome of the handler, the container will eventually terminate within the Pod's termination grace period. Other management of the container blocks until the hook completes or until the termination grace period is reached. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"One and only one of the following should be specified. Exec specifies the action to take.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGet specifies the http request to perform.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\\\",\\\"x-kubernetes-int-or-string\\\":true},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported TODO: implement a realistic TCP lifecycle hook\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Number or name of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\\\",\\\"x-kubernetes-int-or-string\\\":true}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"livenessProbe\\\":{\\\"description\\\":\\\"Periodic probe of container liveness. Container will be restarted if the probe fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"One and only one of the following should be specified. Exec specifies the action to take.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"failureThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGet specifies the http request to perform.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\\\",\\\"x-kubernetes-int-or-string\\\":true},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"initialDelaySeconds\\\":{\\\"description\\\":\\\"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"periodSeconds\\\":{\\\"description\\\":\\\"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"successThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness and startup. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported TODO: implement a realistic TCP lifecycle hook\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Number or name of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\\\",\\\"x-kubernetes-int-or-string\\\":true}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"timeoutSeconds\\\":{\\\"description\\\":\\\"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the container specified as a DNS_LABEL. Each container in a pod must have a unique name (DNS_LABEL). Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"},\\\"ports\\\":{\\\"description\\\":\\\"List of ports to expose from the container. Exposing a port here gives the system additional information about the network connections a container uses, but is primarily informational. Not specifying a port here DOES NOT prevent that port from being exposed. Any port which is listening on the default \\\\\\\"0.0.0.0\\\\\\\" address inside a container will be accessible from the network. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"ContainerPort represents a network port in a single container.\\\",\\\"properties\\\":{\\\"containerPort\\\":{\\\"description\\\":\\\"Number of port to expose on the pod's IP address. This must be a valid port number, 0 \\\\u003c x \\\\u003c 65536.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"hostIP\\\":{\\\"description\\\":\\\"What host IP to bind the external port to.\\\",\\\"type\\\":\\\"string\\\"},\\\"hostPort\\\":{\\\"description\\\":\\\"Number of port to expose on the host. If specified, this must be a valid port number, 0 \\\\u003c x \\\\u003c 65536. If HostNetwork is specified, this must match ContainerPort. Most containers do not need this.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"name\\\":{\\\"description\\\":\\\"If specified, this must be an IANA_SVC_NAME and unique within the pod. Each named port in a pod must have a unique name. Name for the port that can be referred to by services.\\\",\\\"type\\\":\\\"string\\\"},\\\"protocol\\\":{\\\"description\\\":\\\"Protocol for port. Must be UDP, TCP, or SCTP. Defaults to \\\\\\\"TCP\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"containerPort\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"readinessProbe\\\":{\\\"description\\\":\\\"Periodic probe of container service readiness. Container will be removed from service endpoints if the probe fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"One and only one of the following should be specified. Exec specifies the action to take.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"failureThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGet specifies the http request to perform.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\\\",\\\"x-kubernetes-int-or-string\\\":true},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"initialDelaySeconds\\\":{\\\"description\\\":\\\"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"periodSeconds\\\":{\\\"description\\\":\\\"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"successThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness and startup. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported TODO: implement a realistic TCP lifecycle hook\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Number or name of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\\\",\\\"x-kubernetes-int-or-string\\\":true}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"timeoutSeconds\\\":{\\\"description\\\":\\\"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"},\\\"resources\\\":{\\\"description\\\":\\\"Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"securityContext\\\":{\\\"description\\\":\\\"Security options the pod should run with. More info: https://kubernetes.io/docs/concepts/policy/security-context/ More info: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/\\\",\\\"properties\\\":{\\\"allowPrivilegeEscalation\\\":{\\\"description\\\":\\\"AllowPrivilegeEscalation controls whether a process can gain more privileges than its parent process. This bool directly controls if the no_new_privs flag will be set on the container process. AllowPrivilegeEscalation is true always when the container is: 1) run as Privileged 2) has CAP_SYS_ADMIN\\\",\\\"type\\\":\\\"boolean\\\"},\\\"capabilities\\\":{\\\"description\\\":\\\"The capabilities to add/drop when running containers. Defaults to the default set of capabilities granted by the container runtime.\\\",\\\"properties\\\":{\\\"add\\\":{\\\"description\\\":\\\"Added capabilities\\\",\\\"items\\\":{\\\"description\\\":\\\"Capability represent POSIX capabilities type\\\",\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"drop\\\":{\\\"description\\\":\\\"Removed capabilities\\\",\\\"items\\\":{\\\"description\\\":\\\"Capability represent POSIX capabilities type\\\",\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"privileged\\\":{\\\"description\\\":\\\"Run container in privileged mode. Processes in privileged containers are essentially equivalent to root on the host. Defaults to false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"procMount\\\":{\\\"description\\\":\\\"procMount denotes the type of proc mount to use for the containers. The default is DefaultProcMount which uses the container runtime defaults for readonly paths and masked paths. This requires the ProcMountType feature flag to be enabled.\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnlyRootFilesystem\\\":{\\\"description\\\":\\\"Whether this container has a read-only root filesystem. Default is false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"runAsGroup\\\":{\\\"description\\\":\\\"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"runAsNonRoot\\\":{\\\"description\\\":\\\"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"runAsUser\\\":{\\\"description\\\":\\\"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"seLinuxOptions\\\":{\\\"description\\\":\\\"The SELinux context to be applied to the container. If unspecified, the container runtime will allocate a random SELinux context for each container. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"properties\\\":{\\\"level\\\":{\\\"description\\\":\\\"Level is SELinux level label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"role\\\":{\\\"description\\\":\\\"Role is a SELinux role label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"type\\\":{\\\"description\\\":\\\"Type is a SELinux type label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"user\\\":{\\\"description\\\":\\\"User is a SELinux user label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"windowsOptions\\\":{\\\"description\\\":\\\"The Windows specific settings applied to all containers. If unspecified, the options from the PodSecurityContext will be used. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"properties\\\":{\\\"gmsaCredentialSpec\\\":{\\\"description\\\":\\\"GMSACredentialSpec is where the GMSA admission webhook (https://github.com/kubernetes-sigs/windows-gmsa) inlines the contents of the GMSA credential spec named by the GMSACredentialSpecName field.\\\",\\\"type\\\":\\\"string\\\"},\\\"gmsaCredentialSpecName\\\":{\\\"description\\\":\\\"GMSACredentialSpecName is the name of the GMSA credential spec to use.\\\",\\\"type\\\":\\\"string\\\"},\\\"runAsUserName\\\":{\\\"description\\\":\\\"The UserName in Windows to run the entrypoint of the container process. Defaults to the user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"startupProbe\\\":{\\\"description\\\":\\\"StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod's lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. This is a beta feature enabled by the StartupProbe feature flag. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"One and only one of the following should be specified. Exec specifies the action to take.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"failureThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGet specifies the http request to perform.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\\\",\\\"x-kubernetes-int-or-string\\\":true},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"initialDelaySeconds\\\":{\\\"description\\\":\\\"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"periodSeconds\\\":{\\\"description\\\":\\\"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"successThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness and startup. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported TODO: implement a realistic TCP lifecycle hook\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Number or name of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\\\",\\\"x-kubernetes-int-or-string\\\":true}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"timeoutSeconds\\\":{\\\"description\\\":\\\"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"},\\\"stdin\\\":{\\\"description\\\":\\\"Whether this container should allocate a buffer for stdin in the container runtime. If this is not set, reads from stdin in the container will always result in EOF. Default is false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"stdinOnce\\\":{\\\"description\\\":\\\"Whether the container runtime should close the stdin channel after it has been opened by a single attach. When stdin is true the stdin stream will remain open across multiple attach sessions. If stdinOnce is set to true, stdin is opened on container start, is empty until the first client attaches to stdin, and then remains open and accepts data until the client disconnects, at which time stdin is closed and remains closed until the container is restarted. If this flag is false, a container processes that reads from stdin will never receive an EOF. Default is false\\\",\\\"type\\\":\\\"boolean\\\"},\\\"terminationMessagePath\\\":{\\\"description\\\":\\\"Optional: Path at which the file to which the container's termination message will be written is mounted into the container's filesystem. Message written is intended to be brief final status, such as an assertion failure message. Will be truncated by the node if greater than 4096 bytes. The total message length across all containers will be limited to 12kb. Defaults to /dev/termination-log. Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"},\\\"terminationMessagePolicy\\\":{\\\"description\\\":\\\"Indicate how the termination message should be populated. File will use the contents of terminationMessagePath to populate the container status message on both success and failure. FallbackToLogsOnError will use the last chunk of container log output if the termination message file is empty and the container exited with an error. The log output is limited to 2048 bytes or 80 lines, whichever is smaller. Defaults to File. Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"},\\\"tty\\\":{\\\"description\\\":\\\"Whether this container should allocate a TTY for itself, also requires 'stdin' to be true. Default is false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"volumeDevices\\\":{\\\"description\\\":\\\"volumeDevices is the list of block devices to be used by the container.\\\",\\\"items\\\":{\\\"description\\\":\\\"volumeDevice describes a mapping of a raw block device within a container.\\\",\\\"properties\\\":{\\\"devicePath\\\":{\\\"description\\\":\\\"devicePath is the path inside of the container that the device will be mapped to.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"name must match the name of a persistentVolumeClaim in the pod\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"devicePath\\\",\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"volumeMounts\\\":{\\\"description\\\":\\\"Pod volumes to mount into the container's filesystem. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"VolumeMount describes a mounting of a Volume within a container.\\\",\\\"properties\\\":{\\\"mountPath\\\":{\\\"description\\\":\\\"Path within the container at which the volume should be mounted. Must not contain ':'.\\\",\\\"type\\\":\\\"string\\\"},\\\"mountPropagation\\\":{\\\"description\\\":\\\"mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"This must match the Name of a Volume.\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"subPath\\\":{\\\"description\\\":\\\"Path within the volume from which the container's volume should be mounted. Defaults to \\\\\\\"\\\\\\\" (volume's root).\\\",\\\"type\\\":\\\"string\\\"},\\\"subPathExpr\\\":{\\\"description\\\":\\\"Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to \\\\\\\"\\\\\\\" (volume's root). SubPathExpr and SubPath are mutually exclusive.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"mountPath\\\",\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"workingDir\\\":{\\\"description\\\":\\\"Container's working directory. If not specified, the container runtime's default will be used, which might be configured in the container image. Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"disableCompaction\\\":{\\\"description\\\":\\\"Disable prometheus compaction.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"enableAdminAPI\\\":{\\\"description\\\":\\\"Enable access to prometheus web admin API. Defaults to the value of `false`. WARNING: Enabling the admin APIs enables mutating endpoints, to delete data, shutdown Prometheus, and more. Enabling this should be done with care and the user is advised to add additional authentication authorization via a proxy to ensure only clients authorized to perform these actions can do so. For more information see https://prometheus.io/docs/prometheus/latest/querying/api/#tsdb-admin-apis\\\",\\\"type\\\":\\\"boolean\\\"},\\\"enforcedNamespaceLabel\\\":{\\\"description\\\":\\\"EnforcedNamespaceLabel enforces adding a namespace label of origin for each alert and metric that is user created. The label value will always be the namespace of the object that is being created.\\\",\\\"type\\\":\\\"string\\\"},\\\"evaluationInterval\\\":{\\\"description\\\":\\\"Interval between consecutive evaluations.\\\",\\\"type\\\":\\\"string\\\"},\\\"externalLabels\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"The labels to add to any time series or alerts when communicating with external systems (federation, remote storage, Alertmanager).\\\",\\\"type\\\":\\\"object\\\"},\\\"externalUrl\\\":{\\\"description\\\":\\\"The external URL the Prometheus instances will be available under. This is necessary to generate correct URLs. This is necessary if Prometheus is not served from root of a DNS name.\\\",\\\"type\\\":\\\"string\\\"},\\\"ignoreNamespaceSelectors\\\":{\\\"description\\\":\\\"IgnoreNamespaceSelectors if set to true will ignore NamespaceSelector settings from the podmonitor and servicemonitor configs, and they will only discover endpoints within their current namespace. Defaults to false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"image\\\":{\\\"description\\\":\\\"Image if specified has precedence over baseImage, tag and sha combinations. Specifying the version is still necessary to ensure the Prometheus Operator knows what version of Prometheus is being configured.\\\",\\\"type\\\":\\\"string\\\"},\\\"imagePullSecrets\\\":{\\\"description\\\":\\\"An optional list of references to secrets in the same namespace to use for pulling prometheus and alertmanager images from registries see http://kubernetes.io/docs/user-guide/images#specifying-imagepullsecrets-on-a-pod\\\",\\\"items\\\":{\\\"description\\\":\\\"LocalObjectReference contains enough information to let you locate the referenced object inside the same namespace.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"initContainers\\\":{\\\"description\\\":\\\"InitContainers allows adding initContainers to the pod definition. Those can be used to e.g. fetch secrets for injection into the Prometheus configuration from external sources. Any errors during the execution of an initContainer will lead to a restart of the Pod. More info: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/ Using initContainers for any use case other then secret fetching is entirely outside the scope of what the maintainers will support and by doing so, you accept that this behaviour may break at any time without notice.\\\",\\\"items\\\":{\\\"description\\\":\\\"A single application container that you want to run within a pod.\\\",\\\"properties\\\":{\\\"args\\\":{\\\"description\\\":\\\"Arguments to the entrypoint. The docker image's CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"command\\\":{\\\"description\\\":\\\"Entrypoint array. Not executed within a shell. The docker image's ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"env\\\":{\\\"description\\\":\\\"List of environment variables to set in the container. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"EnvVar represents an environment variable present in a Container.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the environment variable. Must be a C_IDENTIFIER.\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to \\\\\\\"\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"},\\\"valueFrom\\\":{\\\"description\\\":\\\"Source for the environment variable's value. Cannot be used if value is not empty.\\\",\\\"properties\\\":{\\\"configMapKeyRef\\\":{\\\"description\\\":\\\"Selects a key of a ConfigMap.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key to select.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"fieldRef\\\":{\\\"description\\\":\\\"Selects a field of the pod: supports metadata.name, metadata.namespace, metadata.labels, metadata.annotations, spec.nodeName, spec.serviceAccountName, status.hostIP, status.podIP, status.podIPs.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"Version of the schema the FieldPath is written in terms of, defaults to \\\\\\\"v1\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"},\\\"fieldPath\\\":{\\\"description\\\":\\\"Path of the field to select in the specified API version.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"fieldPath\\\"],\\\"type\\\":\\\"object\\\"},\\\"resourceFieldRef\\\":{\\\"description\\\":\\\"Selects a resource of the container: only resources limits and requests (limits.cpu, limits.memory, limits.ephemeral-storage, requests.cpu, requests.memory and requests.ephemeral-storage) are currently supported.\\\",\\\"properties\\\":{\\\"containerName\\\":{\\\"description\\\":\\\"Container name: required for volumes, optional for env vars\\\",\\\"type\\\":\\\"string\\\"},\\\"divisor\\\":{\\\"description\\\":\\\"Specifies the output format of the exposed resources, defaults to \\\\\\\"1\\\\\\\"\\\",\\\"type\\\":\\\"string\\\"},\\\"resource\\\":{\\\"description\\\":\\\"Required: resource to select\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"resource\\\"],\\\"type\\\":\\\"object\\\"},\\\"secretKeyRef\\\":{\\\"description\\\":\\\"Selects a key of a secret in the pod's namespace\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"required\\\":[\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"envFrom\\\":{\\\"description\\\":\\\"List of sources to populate environment variables in the container. The keys defined within a source must be a C_IDENTIFIER. All invalid keys will be reported as an event when the container is starting. When a key exists in multiple sources, the value associated with the last source will take precedence. Values defined by an Env with a duplicate key will take precedence. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"EnvFromSource represents the source of a set of ConfigMaps\\\",\\\"properties\\\":{\\\"configMapRef\\\":{\\\"description\\\":\\\"The ConfigMap to select from\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"type\\\":\\\"object\\\"},\\\"prefix\\\":{\\\"description\\\":\\\"An optional identifier to prepend to each key in the ConfigMap. Must be a C_IDENTIFIER.\\\",\\\"type\\\":\\\"string\\\"},\\\"secretRef\\\":{\\\"description\\\":\\\"The Secret to select from\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"image\\\":{\\\"description\\\":\\\"Docker image name. More info: https://kubernetes.io/docs/concepts/containers/images This field is optional to allow higher level config management to default or override container images in workload controllers like Deployments and StatefulSets.\\\",\\\"type\\\":\\\"string\\\"},\\\"imagePullPolicy\\\":{\\\"description\\\":\\\"Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images\\\",\\\"type\\\":\\\"string\\\"},\\\"lifecycle\\\":{\\\"description\\\":\\\"Actions that the management system should take in response to container lifecycle events. Cannot be updated.\\\",\\\"properties\\\":{\\\"postStart\\\":{\\\"description\\\":\\\"PostStart is called immediately after a container is created. If the handler fails, the container is terminated and restarted according to its restart policy. Other management of the container blocks until the hook completes. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"One and only one of the following should be specified. Exec specifies the action to take.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGet specifies the http request to perform.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\\\",\\\"x-kubernetes-int-or-string\\\":true},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported TODO: implement a realistic TCP lifecycle hook\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Number or name of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\\\",\\\"x-kubernetes-int-or-string\\\":true}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"preStop\\\":{\\\"description\\\":\\\"PreStop is called immediately before a container is terminated due to an API request or management event such as liveness/startup probe failure, preemption, resource contention, etc. The handler is not called if the container crashes or exits. The reason for termination is passed to the handler. The Pod's termination grace period countdown begins before the PreStop hooked is executed. Regardless of the outcome of the handler, the container will eventually terminate within the Pod's termination grace period. Other management of the container blocks until the hook completes or until the termination grace period is reached. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"One and only one of the following should be specified. Exec specifies the action to take.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGet specifies the http request to perform.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\\\",\\\"x-kubernetes-int-or-string\\\":true},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported TODO: implement a realistic TCP lifecycle hook\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Number or name of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\\\",\\\"x-kubernetes-int-or-string\\\":true}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"livenessProbe\\\":{\\\"description\\\":\\\"Periodic probe of container liveness. Container will be restarted if the probe fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"One and only one of the following should be specified. Exec specifies the action to take.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"failureThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGet specifies the http request to perform.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\\\",\\\"x-kubernetes-int-or-string\\\":true},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"initialDelaySeconds\\\":{\\\"description\\\":\\\"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"periodSeconds\\\":{\\\"description\\\":\\\"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"successThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness and startup. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported TODO: implement a realistic TCP lifecycle hook\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Number or name of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\\\",\\\"x-kubernetes-int-or-string\\\":true}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"timeoutSeconds\\\":{\\\"description\\\":\\\"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the container specified as a DNS_LABEL. Each container in a pod must have a unique name (DNS_LABEL). Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"},\\\"ports\\\":{\\\"description\\\":\\\"List of ports to expose from the container. Exposing a port here gives the system additional information about the network connections a container uses, but is primarily informational. Not specifying a port here DOES NOT prevent that port from being exposed. Any port which is listening on the default \\\\\\\"0.0.0.0\\\\\\\" address inside a container will be accessible from the network. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"ContainerPort represents a network port in a single container.\\\",\\\"properties\\\":{\\\"containerPort\\\":{\\\"description\\\":\\\"Number of port to expose on the pod's IP address. This must be a valid port number, 0 \\\\u003c x \\\\u003c 65536.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"hostIP\\\":{\\\"description\\\":\\\"What host IP to bind the external port to.\\\",\\\"type\\\":\\\"string\\\"},\\\"hostPort\\\":{\\\"description\\\":\\\"Number of port to expose on the host. If specified, this must be a valid port number, 0 \\\\u003c x \\\\u003c 65536. If HostNetwork is specified, this must match ContainerPort. Most containers do not need this.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"name\\\":{\\\"description\\\":\\\"If specified, this must be an IANA_SVC_NAME and unique within the pod. Each named port in a pod must have a unique name. Name for the port that can be referred to by services.\\\",\\\"type\\\":\\\"string\\\"},\\\"protocol\\\":{\\\"description\\\":\\\"Protocol for port. Must be UDP, TCP, or SCTP. Defaults to \\\\\\\"TCP\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"containerPort\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"readinessProbe\\\":{\\\"description\\\":\\\"Periodic probe of container service readiness. Container will be removed from service endpoints if the probe fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"One and only one of the following should be specified. Exec specifies the action to take.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"failureThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGet specifies the http request to perform.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\\\",\\\"x-kubernetes-int-or-string\\\":true},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"initialDelaySeconds\\\":{\\\"description\\\":\\\"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"periodSeconds\\\":{\\\"description\\\":\\\"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"successThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness and startup. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported TODO: implement a realistic TCP lifecycle hook\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Number or name of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\\\",\\\"x-kubernetes-int-or-string\\\":true}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"timeoutSeconds\\\":{\\\"description\\\":\\\"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"},\\\"resources\\\":{\\\"description\\\":\\\"Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"securityContext\\\":{\\\"description\\\":\\\"Security options the pod should run with. More info: https://kubernetes.io/docs/concepts/policy/security-context/ More info: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/\\\",\\\"properties\\\":{\\\"allowPrivilegeEscalation\\\":{\\\"description\\\":\\\"AllowPrivilegeEscalation controls whether a process can gain more privileges than its parent process. This bool directly controls if the no_new_privs flag will be set on the container process. AllowPrivilegeEscalation is true always when the container is: 1) run as Privileged 2) has CAP_SYS_ADMIN\\\",\\\"type\\\":\\\"boolean\\\"},\\\"capabilities\\\":{\\\"description\\\":\\\"The capabilities to add/drop when running containers. Defaults to the default set of capabilities granted by the container runtime.\\\",\\\"properties\\\":{\\\"add\\\":{\\\"description\\\":\\\"Added capabilities\\\",\\\"items\\\":{\\\"description\\\":\\\"Capability represent POSIX capabilities type\\\",\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"drop\\\":{\\\"description\\\":\\\"Removed capabilities\\\",\\\"items\\\":{\\\"description\\\":\\\"Capability represent POSIX capabilities type\\\",\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"privileged\\\":{\\\"description\\\":\\\"Run container in privileged mode. Processes in privileged containers are essentially equivalent to root on the host. Defaults to false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"procMount\\\":{\\\"description\\\":\\\"procMount denotes the type of proc mount to use for the containers. The default is DefaultProcMount which uses the container runtime defaults for readonly paths and masked paths. This requires the ProcMountType feature flag to be enabled.\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnlyRootFilesystem\\\":{\\\"description\\\":\\\"Whether this container has a read-only root filesystem. Default is false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"runAsGroup\\\":{\\\"description\\\":\\\"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"runAsNonRoot\\\":{\\\"description\\\":\\\"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"runAsUser\\\":{\\\"description\\\":\\\"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"seLinuxOptions\\\":{\\\"description\\\":\\\"The SELinux context to be applied to the container. If unspecified, the container runtime will allocate a random SELinux context for each container. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"properties\\\":{\\\"level\\\":{\\\"description\\\":\\\"Level is SELinux level label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"role\\\":{\\\"description\\\":\\\"Role is a SELinux role label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"type\\\":{\\\"description\\\":\\\"Type is a SELinux type label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"user\\\":{\\\"description\\\":\\\"User is a SELinux user label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"windowsOptions\\\":{\\\"description\\\":\\\"The Windows specific settings applied to all containers. If unspecified, the options from the PodSecurityContext will be used. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"properties\\\":{\\\"gmsaCredentialSpec\\\":{\\\"description\\\":\\\"GMSACredentialSpec is where the GMSA admission webhook (https://github.com/kubernetes-sigs/windows-gmsa) inlines the contents of the GMSA credential spec named by the GMSACredentialSpecName field.\\\",\\\"type\\\":\\\"string\\\"},\\\"gmsaCredentialSpecName\\\":{\\\"description\\\":\\\"GMSACredentialSpecName is the name of the GMSA credential spec to use.\\\",\\\"type\\\":\\\"string\\\"},\\\"runAsUserName\\\":{\\\"description\\\":\\\"The UserName in Windows to run the entrypoint of the container process. Defaults to the user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"startupProbe\\\":{\\\"description\\\":\\\"StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod's lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. This is a beta feature enabled by the StartupProbe feature flag. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"One and only one of the following should be specified. Exec specifies the action to take.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"failureThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGet specifies the http request to perform.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\\\",\\\"x-kubernetes-int-or-string\\\":true},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"initialDelaySeconds\\\":{\\\"description\\\":\\\"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"periodSeconds\\\":{\\\"description\\\":\\\"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"successThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness and startup. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported TODO: implement a realistic TCP lifecycle hook\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Number or name of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\\\",\\\"x-kubernetes-int-or-string\\\":true}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"timeoutSeconds\\\":{\\\"description\\\":\\\"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"},\\\"stdin\\\":{\\\"description\\\":\\\"Whether this container should allocate a buffer for stdin in the container runtime. If this is not set, reads from stdin in the container will always result in EOF. Default is false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"stdinOnce\\\":{\\\"description\\\":\\\"Whether the container runtime should close the stdin channel after it has been opened by a single attach. When stdin is true the stdin stream will remain open across multiple attach sessions. If stdinOnce is set to true, stdin is opened on container start, is empty until the first client attaches to stdin, and then remains open and accepts data until the client disconnects, at which time stdin is closed and remains closed until the container is restarted. If this flag is false, a container processes that reads from stdin will never receive an EOF. Default is false\\\",\\\"type\\\":\\\"boolean\\\"},\\\"terminationMessagePath\\\":{\\\"description\\\":\\\"Optional: Path at which the file to which the container's termination message will be written is mounted into the container's filesystem. Message written is intended to be brief final status, such as an assertion failure message. Will be truncated by the node if greater than 4096 bytes. The total message length across all containers will be limited to 12kb. Defaults to /dev/termination-log. Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"},\\\"terminationMessagePolicy\\\":{\\\"description\\\":\\\"Indicate how the termination message should be populated. File will use the contents of terminationMessagePath to populate the container status message on both success and failure. FallbackToLogsOnError will use the last chunk of container log output if the termination message file is empty and the container exited with an error. The log output is limited to 2048 bytes or 80 lines, whichever is smaller. Defaults to File. Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"},\\\"tty\\\":{\\\"description\\\":\\\"Whether this container should allocate a TTY for itself, also requires 'stdin' to be true. Default is false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"volumeDevices\\\":{\\\"description\\\":\\\"volumeDevices is the list of block devices to be used by the container.\\\",\\\"items\\\":{\\\"description\\\":\\\"volumeDevice describes a mapping of a raw block device within a container.\\\",\\\"properties\\\":{\\\"devicePath\\\":{\\\"description\\\":\\\"devicePath is the path inside of the container that the device will be mapped to.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"name must match the name of a persistentVolumeClaim in the pod\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"devicePath\\\",\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"volumeMounts\\\":{\\\"description\\\":\\\"Pod volumes to mount into the container's filesystem. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"VolumeMount describes a mounting of a Volume within a container.\\\",\\\"properties\\\":{\\\"mountPath\\\":{\\\"description\\\":\\\"Path within the container at which the volume should be mounted. Must not contain ':'.\\\",\\\"type\\\":\\\"string\\\"},\\\"mountPropagation\\\":{\\\"description\\\":\\\"mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"This must match the Name of a Volume.\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"subPath\\\":{\\\"description\\\":\\\"Path within the volume from which the container's volume should be mounted. Defaults to \\\\\\\"\\\\\\\" (volume's root).\\\",\\\"type\\\":\\\"string\\\"},\\\"subPathExpr\\\":{\\\"description\\\":\\\"Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to \\\\\\\"\\\\\\\" (volume's root). SubPathExpr and SubPath are mutually exclusive.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"mountPath\\\",\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"workingDir\\\":{\\\"description\\\":\\\"Container's working directory. If not specified, the container runtime's default will be used, which might be configured in the container image. Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"listenLocal\\\":{\\\"description\\\":\\\"ListenLocal makes the Prometheus server listen on loopback, so that it does not bind against the Pod IP.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"logFormat\\\":{\\\"description\\\":\\\"Log format for Prometheus to be configured with.\\\",\\\"type\\\":\\\"string\\\"},\\\"logLevel\\\":{\\\"description\\\":\\\"Log level for Prometheus to be configured with.\\\",\\\"type\\\":\\\"string\\\"},\\\"nodeSelector\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Define which Nodes the Pods are scheduled on.\\\",\\\"type\\\":\\\"object\\\"},\\\"overrideHonorLabels\\\":{\\\"description\\\":\\\"OverrideHonorLabels if set to true overrides all user configured honor_labels. If HonorLabels is set in ServiceMonitor or PodMonitor to true, this overrides honor_labels to false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"overrideHonorTimestamps\\\":{\\\"description\\\":\\\"OverrideHonorTimestamps allows to globally enforce honoring timestamps in all scrape configs.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"paused\\\":{\\\"description\\\":\\\"When a Prometheus deployment is paused, no actions except for deletion will be performed on the underlying objects.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"podMetadata\\\":{\\\"description\\\":\\\"PodMetadata configures Labels and Annotations which are propagated to the prometheus pods.\\\",\\\"properties\\\":{\\\"annotations\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations\\\",\\\"type\\\":\\\"object\\\"},\\\"labels\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels\\\",\\\"type\\\":\\\"object\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"podMonitorNamespaceSelector\\\":{\\\"description\\\":\\\"Namespaces to be selected for PodMonitor discovery. If nil, only check own namespace.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"podMonitorSelector\\\":{\\\"description\\\":\\\"*Experimental* PodMonitors to be selected for target discovery. *Deprecated:* if neither this nor serviceMonitorSelector are specified, configuration is unmanaged.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"portName\\\":{\\\"description\\\":\\\"Port name used for the pods and governing service. This defaults to web\\\",\\\"type\\\":\\\"string\\\"},\\\"priorityClassName\\\":{\\\"description\\\":\\\"Priority class assigned to the Pods\\\",\\\"type\\\":\\\"string\\\"},\\\"prometheusExternalLabelName\\\":{\\\"description\\\":\\\"Name of Prometheus external label used to denote Prometheus instance name. Defaults to the value of `prometheus`. External label will _not_ be added when value is set to empty string (`\\\\\\\"\\\\\\\"`).\\\",\\\"type\\\":\\\"string\\\"},\\\"query\\\":{\\\"description\\\":\\\"QuerySpec defines the query command line flags when starting Prometheus.\\\",\\\"properties\\\":{\\\"lookbackDelta\\\":{\\\"description\\\":\\\"The delta difference allowed for retrieving metrics during expression evaluations.\\\",\\\"type\\\":\\\"string\\\"},\\\"maxConcurrency\\\":{\\\"description\\\":\\\"Number of concurrent queries that can be run at once.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"maxSamples\\\":{\\\"description\\\":\\\"Maximum number of samples a single query can load into memory. Note that queries will fail if they would load more samples than this into memory, so this also limits the number of samples a query can return.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"timeout\\\":{\\\"description\\\":\\\"Maximum time a query may take before being aborted.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"queryLogFile\\\":{\\\"description\\\":\\\"QueryLogFile specifies the file to which PromQL queries are logged. Note that this location must be writable, and can be persisted using an attached volume. Alternatively, the location can be set to a stdout location such as `/dev/stdout` to log querie information to the default Prometheus log stream. This is only available in versions of Prometheus \\\\u003e= 2.16.0. For more details, see the Prometheus docs (https://prometheus.io/docs/guides/query-log/)\\\",\\\"type\\\":\\\"string\\\"},\\\"remoteRead\\\":{\\\"description\\\":\\\"If specified, the remote_read spec. This is an experimental feature, it may change in any upcoming release in a breaking way.\\\",\\\"items\\\":{\\\"description\\\":\\\"RemoteReadSpec defines the remote_read configuration for prometheus.\\\",\\\"properties\\\":{\\\"basicAuth\\\":{\\\"description\\\":\\\"BasicAuth for the URL.\\\",\\\"properties\\\":{\\\"password\\\":{\\\"description\\\":\\\"The secret in the service monitor namespace that contains the password for authentication.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"username\\\":{\\\"description\\\":\\\"The secret in the service monitor namespace that contains the username for authentication.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"bearerToken\\\":{\\\"description\\\":\\\"bearer token for remote read.\\\",\\\"type\\\":\\\"string\\\"},\\\"bearerTokenFile\\\":{\\\"description\\\":\\\"File to read bearer token for remote read.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"The name of the remote read queue, must be unique if specified. The name is used in metrics and logging in order to differentiate read configurations. Only valid in Prometheus versions 2.15.0 and newer.\\\",\\\"type\\\":\\\"string\\\"},\\\"proxyUrl\\\":{\\\"description\\\":\\\"Optional ProxyURL\\\",\\\"type\\\":\\\"string\\\"},\\\"readRecent\\\":{\\\"description\\\":\\\"Whether reads should be made for queries for time ranges that the local storage should have complete data for.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"remoteTimeout\\\":{\\\"description\\\":\\\"Timeout for requests to the remote read endpoint.\\\",\\\"type\\\":\\\"string\\\"},\\\"requiredMatchers\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"An optional list of equality matchers which have to be present in a selector to query the remote read endpoint.\\\",\\\"type\\\":\\\"object\\\"},\\\"tlsConfig\\\":{\\\"description\\\":\\\"TLS Config to use for remote read.\\\",\\\"properties\\\":{\\\"ca\\\":{\\\"description\\\":\\\"Struct containing the CA cert to use for the targets.\\\",\\\"properties\\\":{\\\"configMap\\\":{\\\"description\\\":\\\"ConfigMap containing data to use for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key to select.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"secret\\\":{\\\"description\\\":\\\"Secret containing data to use for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"caFile\\\":{\\\"description\\\":\\\"Path to the CA cert in the Prometheus container to use for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"cert\\\":{\\\"description\\\":\\\"Struct containing the client cert file for the targets.\\\",\\\"properties\\\":{\\\"configMap\\\":{\\\"description\\\":\\\"ConfigMap containing data to use for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key to select.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"secret\\\":{\\\"description\\\":\\\"Secret containing data to use for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"certFile\\\":{\\\"description\\\":\\\"Path to the client cert file in the Prometheus container for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"insecureSkipVerify\\\":{\\\"description\\\":\\\"Disable target certificate validation.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"keyFile\\\":{\\\"description\\\":\\\"Path to the client key file in the Prometheus container for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"keySecret\\\":{\\\"description\\\":\\\"Secret containing the client key file for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"serverName\\\":{\\\"description\\\":\\\"Used to verify the hostname for the targets.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"url\\\":{\\\"description\\\":\\\"The URL of the endpoint to send samples to.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"url\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"remoteWrite\\\":{\\\"description\\\":\\\"If specified, the remote_write spec. This is an experimental feature, it may change in any upcoming release in a breaking way.\\\",\\\"items\\\":{\\\"description\\\":\\\"RemoteWriteSpec defines the remote_write configuration for prometheus.\\\",\\\"properties\\\":{\\\"basicAuth\\\":{\\\"description\\\":\\\"BasicAuth for the URL.\\\",\\\"properties\\\":{\\\"password\\\":{\\\"description\\\":\\\"The secret in the service monitor namespace that contains the password for authentication.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"username\\\":{\\\"description\\\":\\\"The secret in the service monitor namespace that contains the username for authentication.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"bearerToken\\\":{\\\"description\\\":\\\"File to read bearer token for remote write.\\\",\\\"type\\\":\\\"string\\\"},\\\"bearerTokenFile\\\":{\\\"description\\\":\\\"File to read bearer token for remote write.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"The name of the remote write queue, must be unique if specified. The name is used in metrics and logging in order to differentiate queues. Only valid in Prometheus versions 2.15.0 and newer.\\\",\\\"type\\\":\\\"string\\\"},\\\"proxyUrl\\\":{\\\"description\\\":\\\"Optional ProxyURL\\\",\\\"type\\\":\\\"string\\\"},\\\"queueConfig\\\":{\\\"description\\\":\\\"QueueConfig allows tuning of the remote write queue parameters.\\\",\\\"properties\\\":{\\\"batchSendDeadline\\\":{\\\"description\\\":\\\"BatchSendDeadline is the maximum time a sample will wait in buffer.\\\",\\\"type\\\":\\\"string\\\"},\\\"capacity\\\":{\\\"description\\\":\\\"Capacity is the number of samples to buffer per shard before we start dropping them.\\\",\\\"type\\\":\\\"integer\\\"},\\\"maxBackoff\\\":{\\\"description\\\":\\\"MaxBackoff is the maximum retry delay.\\\",\\\"type\\\":\\\"string\\\"},\\\"maxRetries\\\":{\\\"description\\\":\\\"MaxRetries is the maximum number of times to retry a batch on recoverable errors.\\\",\\\"type\\\":\\\"integer\\\"},\\\"maxSamplesPerSend\\\":{\\\"description\\\":\\\"MaxSamplesPerSend is the maximum number of samples per send.\\\",\\\"type\\\":\\\"integer\\\"},\\\"maxShards\\\":{\\\"description\\\":\\\"MaxShards is the maximum number of shards, i.e. amount of concurrency.\\\",\\\"type\\\":\\\"integer\\\"},\\\"minBackoff\\\":{\\\"description\\\":\\\"MinBackoff is the initial retry delay. Gets doubled for every retry.\\\",\\\"type\\\":\\\"string\\\"},\\\"minShards\\\":{\\\"description\\\":\\\"MinShards is the minimum number of shards, i.e. amount of concurrency.\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"},\\\"remoteTimeout\\\":{\\\"description\\\":\\\"Timeout for requests to the remote write endpoint.\\\",\\\"type\\\":\\\"string\\\"},\\\"tlsConfig\\\":{\\\"description\\\":\\\"TLS Config to use for remote write.\\\",\\\"properties\\\":{\\\"ca\\\":{\\\"description\\\":\\\"Struct containing the CA cert to use for the targets.\\\",\\\"properties\\\":{\\\"configMap\\\":{\\\"description\\\":\\\"ConfigMap containing data to use for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key to select.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"secret\\\":{\\\"description\\\":\\\"Secret containing data to use for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"caFile\\\":{\\\"description\\\":\\\"Path to the CA cert in the Prometheus container to use for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"cert\\\":{\\\"description\\\":\\\"Struct containing the client cert file for the targets.\\\",\\\"properties\\\":{\\\"configMap\\\":{\\\"description\\\":\\\"ConfigMap containing data to use for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key to select.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"secret\\\":{\\\"description\\\":\\\"Secret containing data to use for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"certFile\\\":{\\\"description\\\":\\\"Path to the client cert file in the Prometheus container for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"insecureSkipVerify\\\":{\\\"description\\\":\\\"Disable target certificate validation.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"keyFile\\\":{\\\"description\\\":\\\"Path to the client key file in the Prometheus container for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"keySecret\\\":{\\\"description\\\":\\\"Secret containing the client key file for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"serverName\\\":{\\\"description\\\":\\\"Used to verify the hostname for the targets.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"url\\\":{\\\"description\\\":\\\"The URL of the endpoint to send samples to.\\\",\\\"type\\\":\\\"string\\\"},\\\"writeRelabelConfigs\\\":{\\\"description\\\":\\\"The list of remote write relabel configurations.\\\",\\\"items\\\":{\\\"description\\\":\\\"RelabelConfig allows dynamic rewriting of the label set, being applied to samples before ingestion. It defines `\\\\u003cmetric_relabel_configs\\\\u003e`-section of Prometheus configuration. More info: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#metric_relabel_configs\\\",\\\"properties\\\":{\\\"action\\\":{\\\"description\\\":\\\"Action to perform based on regex matching. Default is 'replace'\\\",\\\"type\\\":\\\"string\\\"},\\\"modulus\\\":{\\\"description\\\":\\\"Modulus to take of the hash of the source label values.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"regex\\\":{\\\"description\\\":\\\"Regular expression against which the extracted value is matched. Default is '(.*)'\\\",\\\"type\\\":\\\"string\\\"},\\\"replacement\\\":{\\\"description\\\":\\\"Replacement value against which a regex replace is performed if the regular expression matches. Regex capture groups are available. Default is '$1'\\\",\\\"type\\\":\\\"string\\\"},\\\"separator\\\":{\\\"description\\\":\\\"Separator placed between concatenated source label values. default is ';'.\\\",\\\"type\\\":\\\"string\\\"},\\\"sourceLabels\\\":{\\\"description\\\":\\\"The source labels select values from existing labels. Their content is concatenated using the configured separator and matched against the configured regular expression for the replace, keep, and drop actions.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"targetLabel\\\":{\\\"description\\\":\\\"Label to which the resulting value is written in a replace action. It is mandatory for replace actions. Regex capture groups are available.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"url\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"replicaExternalLabelName\\\":{\\\"description\\\":\\\"Name of Prometheus external label used to denote replica name. Defaults to the value of `prometheus_replica`. External label will _not_ be added when value is set to empty string (`\\\\\\\"\\\\\\\"`).\\\",\\\"type\\\":\\\"string\\\"},\\\"replicas\\\":{\\\"description\\\":\\\"Number of instances to deploy for a Prometheus deployment.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"resources\\\":{\\\"description\\\":\\\"Define resources requests and limits for single Pods.\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"retention\\\":{\\\"description\\\":\\\"Time duration Prometheus shall retain data for. Default is '24h', and must match the regular expression `[0-9]+(ms|s|m|h|d|w|y)` (milliseconds seconds minutes hours days weeks years).\\\",\\\"type\\\":\\\"string\\\"},\\\"retentionSize\\\":{\\\"description\\\":\\\"Maximum amount of disk space used by blocks.\\\",\\\"type\\\":\\\"string\\\"},\\\"routePrefix\\\":{\\\"description\\\":\\\"The route prefix Prometheus registers HTTP handlers for. This is useful, if using ExternalURL and a proxy is rewriting HTTP routes of a request, and the actual ExternalURL is still true, but the server serves requests under a different route prefix. For example for use with `kubectl proxy`.\\\",\\\"type\\\":\\\"string\\\"},\\\"ruleNamespaceSelector\\\":{\\\"description\\\":\\\"Namespaces to be selected for PrometheusRules discovery. If unspecified, only the same namespace as the Prometheus object is in is used.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"ruleSelector\\\":{\\\"description\\\":\\\"A selector to select which PrometheusRules to mount for loading alerting/recording rules from. Until (excluding) Prometheus Operator v0.24.0 Prometheus Operator will migrate any legacy rule ConfigMaps to PrometheusRule custom resources selected by RuleSelector. Make sure it does not match any config maps that you do not want to be migrated.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"rules\\\":{\\\"description\\\":\\\"/--rules.*/ command-line arguments.\\\",\\\"properties\\\":{\\\"alert\\\":{\\\"description\\\":\\\"/--rules.alert.*/ command-line arguments\\\",\\\"properties\\\":{\\\"forGracePeriod\\\":{\\\"description\\\":\\\"Minimum duration between alert and restored 'for' state. This is maintained only for alerts with configured 'for' time greater than grace period.\\\",\\\"type\\\":\\\"string\\\"},\\\"forOutageTolerance\\\":{\\\"description\\\":\\\"Max time to tolerate prometheus outage for restoring 'for' state of alert.\\\",\\\"type\\\":\\\"string\\\"},\\\"resendDelay\\\":{\\\"description\\\":\\\"Minimum amount of time to wait before resending an alert to Alertmanager.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"scrapeInterval\\\":{\\\"description\\\":\\\"Interval between consecutive scrapes.\\\",\\\"type\\\":\\\"string\\\"},\\\"secrets\\\":{\\\"description\\\":\\\"Secrets is a list of Secrets in the same namespace as the Prometheus object, which shall be mounted into the Prometheus Pods. The Secrets are mounted into /etc/prometheus/secrets/\\\\u003csecret-name\\\\u003e.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"securityContext\\\":{\\\"description\\\":\\\"SecurityContext holds pod-level security attributes and common container settings. This defaults to the default PodSecurityContext.\\\",\\\"properties\\\":{\\\"fsGroup\\\":{\\\"description\\\":\\\"A special supplemental group that applies to all containers in a pod. Some volume types allow the Kubelet to change the ownership of that volume to be owned by the pod: \\\\n 1. The owning GID will be the FSGroup 2. The setgid bit is set (new files created in the volume will be owned by FSGroup) 3. The permission bits are OR'd with rw-rw---- \\\\n If unset, the Kubelet will not modify the ownership and permissions of any volume.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"fsGroupChangePolicy\\\":{\\\"description\\\":\\\"fsGroupChangePolicy defines behavior of changing ownership and permission of the volume before being exposed inside Pod. This field will only apply to volume types which support fsGroup based ownership(and permissions). It will have no effect on ephemeral volume types such as: secret, configmaps and emptydir. Valid values are \\\\\\\"OnRootMismatch\\\\\\\" and \\\\\\\"Always\\\\\\\". If not specified defaults to \\\\\\\"Always\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"},\\\"runAsGroup\\\":{\\\"description\\\":\\\"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"runAsNonRoot\\\":{\\\"description\\\":\\\"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"runAsUser\\\":{\\\"description\\\":\\\"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"seLinuxOptions\\\":{\\\"description\\\":\\\"The SELinux context to be applied to all containers. If unspecified, the container runtime will allocate a random SELinux context for each container. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\\\",\\\"properties\\\":{\\\"level\\\":{\\\"description\\\":\\\"Level is SELinux level label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"role\\\":{\\\"description\\\":\\\"Role is a SELinux role label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"type\\\":{\\\"description\\\":\\\"Type is a SELinux type label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"user\\\":{\\\"description\\\":\\\"User is a SELinux user label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"supplementalGroups\\\":{\\\"description\\\":\\\"A list of groups applied to the first process run in each container, in addition to the container's primary GID. If unspecified, no groups will be added to any container.\\\",\\\"items\\\":{\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"type\\\":\\\"array\\\"},\\\"sysctls\\\":{\\\"description\\\":\\\"Sysctls hold a list of namespaced sysctls used for the pod. Pods with unsupported sysctls (by the container runtime) might fail to launch.\\\",\\\"items\\\":{\\\"description\\\":\\\"Sysctl defines a kernel parameter to be set\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of a property to set\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"Value of a property to set\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"windowsOptions\\\":{\\\"description\\\":\\\"The Windows specific settings applied to all containers. If unspecified, the options within a container's SecurityContext will be used. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"properties\\\":{\\\"gmsaCredentialSpec\\\":{\\\"description\\\":\\\"GMSACredentialSpec is where the GMSA admission webhook (https://github.com/kubernetes-sigs/windows-gmsa) inlines the contents of the GMSA credential spec named by the GMSACredentialSpecName field.\\\",\\\"type\\\":\\\"string\\\"},\\\"gmsaCredentialSpecName\\\":{\\\"description\\\":\\\"GMSACredentialSpecName is the name of the GMSA credential spec to use.\\\",\\\"type\\\":\\\"string\\\"},\\\"runAsUserName\\\":{\\\"description\\\":\\\"The UserName in Windows to run the entrypoint of the container process. Defaults to the user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"serviceAccountName\\\":{\\\"description\\\":\\\"ServiceAccountName is the name of the ServiceAccount to use to run the Prometheus Pods.\\\",\\\"type\\\":\\\"string\\\"},\\\"serviceMonitorNamespaceSelector\\\":{\\\"description\\\":\\\"Namespaces to be selected for ServiceMonitor discovery. If nil, only check own namespace.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"serviceMonitorSelector\\\":{\\\"description\\\":\\\"ServiceMonitors to be selected for target discovery. *Deprecated:* if neither this nor podMonitorSelector are specified, configuration is unmanaged.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"sha\\\":{\\\"description\\\":\\\"SHA of Prometheus container image to be deployed. Defaults to the value of `version`. Similar to a tag, but the SHA explicitly deploys an immutable container image. Version and Tag are ignored if SHA is set.\\\",\\\"type\\\":\\\"string\\\"},\\\"storage\\\":{\\\"description\\\":\\\"Storage spec to specify how storage shall be used.\\\",\\\"properties\\\":{\\\"disableMountSubPath\\\":{\\\"description\\\":\\\"Deprecated: subPath usage will be disabled by default in a future release, this option will become unnecessary. DisableMountSubPath allows to remove any subPath usage in volume mounts.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"emptyDir\\\":{\\\"description\\\":\\\"EmptyDirVolumeSource to be used by the Prometheus StatefulSets. If specified, used in place of any volumeClaimTemplate. More info: https://kubernetes.io/docs/concepts/storage/volumes/#emptydir\\\",\\\"properties\\\":{\\\"medium\\\":{\\\"description\\\":\\\"What type of storage medium should back this directory. The default is \\\\\\\"\\\\\\\" which means to use the node's default medium. Must be an empty string (default) or Memory. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir\\\",\\\"type\\\":\\\"string\\\"},\\\"sizeLimit\\\":{\\\"description\\\":\\\"Total amount of local storage required for this EmptyDir volume. The size limit is also applicable for memory medium. The maximum usage on memory medium EmptyDir would be the minimum value between the SizeLimit specified here and the sum of memory limits of all containers in a pod. The default is nil which means that the limit is undefined. More info: http://kubernetes.io/docs/user-guide/volumes#emptydir\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"volumeClaimTemplate\\\":{\\\"description\\\":\\\"A PVC spec to be used by the Prometheus StatefulSets.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"metadata\\\":{\\\"description\\\":\\\"EmbeddedMetadata contains metadata relevant to an EmbeddedResource.\\\",\\\"properties\\\":{\\\"annotations\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations\\\",\\\"type\\\":\\\"object\\\"},\\\"labels\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels\\\",\\\"type\\\":\\\"object\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"spec\\\":{\\\"description\\\":\\\"Spec defines the desired characteristics of a volume requested by a pod author. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims\\\",\\\"properties\\\":{\\\"accessModes\\\":{\\\"description\\\":\\\"AccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"dataSource\\\":{\\\"description\\\":\\\"This field can be used to specify either: * An existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot - Beta) * An existing PVC (PersistentVolumeClaim) * An existing custom resource/object that implements data population (Alpha) In order to use VolumeSnapshot object types, the appropriate feature gate must be enabled (VolumeSnapshotDataSource or AnyVolumeDataSource) If the provisioner or an external controller can support the specified data source, it will create a new volume based on the contents of the specified data source. If the specified data source is not supported, the volume will not be created and the failure will be reported as an event. In the future, we plan to support more data source types and the behavior of the provisioner may change.\\\",\\\"properties\\\":{\\\"apiGroup\\\":{\\\"description\\\":\\\"APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is the type of resource being referenced\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name is the name of resource being referenced\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"kind\\\",\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"resources\\\":{\\\"description\\\":\\\"Resources represents the minimum resources the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"selector\\\":{\\\"description\\\":\\\"A label query over volumes to consider for binding.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"storageClassName\\\":{\\\"description\\\":\\\"Name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1\\\",\\\"type\\\":\\\"string\\\"},\\\"volumeMode\\\":{\\\"description\\\":\\\"volumeMode defines what type of volume is required by the claim. Value of Filesystem is implied when not included in claim spec.\\\",\\\"type\\\":\\\"string\\\"},\\\"volumeName\\\":{\\\"description\\\":\\\"VolumeName is the binding reference to the PersistentVolume backing this claim.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"status\\\":{\\\"description\\\":\\\"Status represents the current information/status of a persistent volume claim. Read-only. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims\\\",\\\"properties\\\":{\\\"accessModes\\\":{\\\"description\\\":\\\"AccessModes contains the actual access modes the volume backing the PVC has. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"capacity\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Represents the actual resources of the underlying volume.\\\",\\\"type\\\":\\\"object\\\"},\\\"conditions\\\":{\\\"description\\\":\\\"Current Condition of persistent volume claim. If underlying persistent volume is being resized then the Condition will be set to 'ResizeStarted'.\\\",\\\"items\\\":{\\\"description\\\":\\\"PersistentVolumeClaimCondition contains details about state of pvc\\\",\\\"properties\\\":{\\\"lastProbeTime\\\":{\\\"description\\\":\\\"Last time we probed the condition.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"lastTransitionTime\\\":{\\\"description\\\":\\\"Last time the condition transitioned from one status to another.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"Human-readable message indicating details about last transition.\\\",\\\"type\\\":\\\"string\\\"},\\\"reason\\\":{\\\"description\\\":\\\"Unique, this should be a short, machine understandable string that gives the reason for condition's last transition. If it reports \\\\\\\"ResizeStarted\\\\\\\" that means the underlying persistent volume is being resized.\\\",\\\"type\\\":\\\"string\\\"},\\\"status\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":{\\\"description\\\":\\\"PersistentVolumeClaimConditionType is a valid value of PersistentVolumeClaimCondition.Type\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"status\\\",\\\"type\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"phase\\\":{\\\"description\\\":\\\"Phase represents the current phase of PersistentVolumeClaim.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"tag\\\":{\\\"description\\\":\\\"Tag of Prometheus container image to be deployed. Defaults to the value of `version`. Version is ignored if Tag is set.\\\",\\\"type\\\":\\\"string\\\"},\\\"thanos\\\":{\\\"description\\\":\\\"Thanos configuration allows configuring various aspects of a Prometheus server in a Thanos environment. \\\\n This section is experimental, it may change significantly without deprecation notice in any release. \\\\n This is experimental and may change significantly without backward compatibility in any release.\\\",\\\"properties\\\":{\\\"baseImage\\\":{\\\"description\\\":\\\"Thanos base image if other than default.\\\",\\\"type\\\":\\\"string\\\"},\\\"grpcServerTlsConfig\\\":{\\\"description\\\":\\\"GRPCServerTLSConfig configures the gRPC server from which Thanos Querier reads recorded rule data. Note: Currently only the CAFile, CertFile, and KeyFile fields are supported. Maps to the '--grpc-server-tls-*' CLI args.\\\",\\\"properties\\\":{\\\"ca\\\":{\\\"description\\\":\\\"Struct containing the CA cert to use for the targets.\\\",\\\"properties\\\":{\\\"configMap\\\":{\\\"description\\\":\\\"ConfigMap containing data to use for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key to select.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"secret\\\":{\\\"description\\\":\\\"Secret containing data to use for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"caFile\\\":{\\\"description\\\":\\\"Path to the CA cert in the Prometheus container to use for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"cert\\\":{\\\"description\\\":\\\"Struct containing the client cert file for the targets.\\\",\\\"properties\\\":{\\\"configMap\\\":{\\\"description\\\":\\\"ConfigMap containing data to use for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key to select.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"secret\\\":{\\\"description\\\":\\\"Secret containing data to use for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"certFile\\\":{\\\"description\\\":\\\"Path to the client cert file in the Prometheus container for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"insecureSkipVerify\\\":{\\\"description\\\":\\\"Disable target certificate validation.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"keyFile\\\":{\\\"description\\\":\\\"Path to the client key file in the Prometheus container for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"keySecret\\\":{\\\"description\\\":\\\"Secret containing the client key file for the targets.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"serverName\\\":{\\\"description\\\":\\\"Used to verify the hostname for the targets.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"image\\\":{\\\"description\\\":\\\"Image if specified has precedence over baseImage, tag and sha combinations. Specifying the version is still necessary to ensure the Prometheus Operator knows what version of Thanos is being configured.\\\",\\\"type\\\":\\\"string\\\"},\\\"listenLocal\\\":{\\\"description\\\":\\\"ListenLocal makes the Thanos sidecar listen on loopback, so that it does not bind against the Pod IP.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"logFormat\\\":{\\\"description\\\":\\\"LogFormat for Thanos sidecar to be configured with.\\\",\\\"type\\\":\\\"string\\\"},\\\"logLevel\\\":{\\\"description\\\":\\\"LogLevel for Thanos sidecar to be configured with.\\\",\\\"type\\\":\\\"string\\\"},\\\"objectStorageConfig\\\":{\\\"description\\\":\\\"ObjectStorageConfig configures object storage in Thanos.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"resources\\\":{\\\"description\\\":\\\"Resources defines the resource requirements for the Thanos sidecar. If not provided, no requests/limits will be set\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"sha\\\":{\\\"description\\\":\\\"SHA of Thanos container image to be deployed. Defaults to the value of `version`. Similar to a tag, but the SHA explicitly deploys an immutable container image. Version and Tag are ignored if SHA is set.\\\",\\\"type\\\":\\\"string\\\"},\\\"tag\\\":{\\\"description\\\":\\\"Tag of Thanos sidecar container image to be deployed. Defaults to the value of `version`. Version is ignored if Tag is set.\\\",\\\"type\\\":\\\"string\\\"},\\\"tracingConfig\\\":{\\\"description\\\":\\\"TracingConfig configures tracing in Thanos. This is an experimental feature, it may change in any upcoming release in a breaking way.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"version\\\":{\\\"description\\\":\\\"Version describes the version of Thanos to use.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"tolerations\\\":{\\\"description\\\":\\\"If specified, the pod's tolerations.\\\",\\\"items\\\":{\\\"description\\\":\\\"The pod this Toleration is attached to tolerates any taint that matches the triple \\\\u003ckey,value,effect\\\\u003e using the matching operator \\\\u003coperator\\\\u003e.\\\",\\\"properties\\\":{\\\"effect\\\":{\\\"description\\\":\\\"Effect indicates the taint effect to match. Empty means match all taint effects. When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.\\\",\\\"type\\\":\\\"string\\\"},\\\"key\\\":{\\\"description\\\":\\\"Key is the taint key that the toleration applies to. Empty means match all taint keys. If the key is empty, operator must be Exists; this combination means to match all values and all keys.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Operator represents a key's relationship to the value. Valid operators are Exists and Equal. Defaults to Equal. Exists is equivalent to wildcard for value, so that a pod can tolerate all taints of a particular category.\\\",\\\"type\\\":\\\"string\\\"},\\\"tolerationSeconds\\\":{\\\"description\\\":\\\"TolerationSeconds represents the period of time the toleration (which must be of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately) by the system.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"value\\\":{\\\"description\\\":\\\"Value is the taint value the toleration matches to. If the operator is Exists, the value should be empty, otherwise just a regular string.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"version\\\":{\\\"description\\\":\\\"Version of Prometheus to be deployed.\\\",\\\"type\\\":\\\"string\\\"},\\\"volumeMounts\\\":{\\\"description\\\":\\\"VolumeMounts allows configuration of additional VolumeMounts on the output StatefulSet definition. VolumeMounts specified will be appended to other VolumeMounts in the prometheus container, that are generated as a result of StorageSpec objects.\\\",\\\"items\\\":{\\\"description\\\":\\\"VolumeMount describes a mounting of a Volume within a container.\\\",\\\"properties\\\":{\\\"mountPath\\\":{\\\"description\\\":\\\"Path within the container at which the volume should be mounted. Must not contain ':'.\\\",\\\"type\\\":\\\"string\\\"},\\\"mountPropagation\\\":{\\\"description\\\":\\\"mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"This must match the Name of a Volume.\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"subPath\\\":{\\\"description\\\":\\\"Path within the volume from which the container's volume should be mounted. Defaults to \\\\\\\"\\\\\\\" (volume's root).\\\",\\\"type\\\":\\\"string\\\"},\\\"subPathExpr\\\":{\\\"description\\\":\\\"Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to \\\\\\\"\\\\\\\" (volume's root). SubPathExpr and SubPath are mutually exclusive.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"mountPath\\\",\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"volumes\\\":{\\\"description\\\":\\\"Volumes allows configuration of additional volumes on the output StatefulSet definition. Volumes specified will be appended to other volumes that are generated as a result of StorageSpec objects.\\\",\\\"items\\\":{\\\"description\\\":\\\"Volume represents a named volume in a pod that may be accessed by any container in the pod.\\\",\\\"properties\\\":{\\\"awsElasticBlockStore\\\":{\\\"description\\\":\\\"AWSElasticBlockStore represents an AWS Disk resource that is attached to a kubelet's host machine and then exposed to the pod. More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore\\\",\\\"properties\\\":{\\\"fsType\\\":{\\\"description\\\":\\\"Filesystem type of the volume that you want to mount. Tip: Ensure that the filesystem type is supported by the host operating system. Examples: \\\\\\\"ext4\\\\\\\", \\\\\\\"xfs\\\\\\\", \\\\\\\"ntfs\\\\\\\". Implicitly inferred to be \\\\\\\"ext4\\\\\\\" if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore TODO: how do we prevent errors in the filesystem from compromising the machine\\\",\\\"type\\\":\\\"string\\\"},\\\"partition\\\":{\\\"description\\\":\\\"The partition in the volume that you want to mount. If omitted, the default is to mount by volume name. Examples: For volume /dev/sda1, you specify the partition as \\\\\\\"1\\\\\\\". Similarly, the volume partition for /dev/sda is \\\\\\\"0\\\\\\\" (or you can leave the property empty).\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"Specify \\\\\\\"true\\\\\\\" to force and set the ReadOnly property in VolumeMounts to \\\\\\\"true\\\\\\\". If omitted, the default is \\\\\\\"false\\\\\\\". More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore\\\",\\\"type\\\":\\\"boolean\\\"},\\\"volumeID\\\":{\\\"description\\\":\\\"Unique ID of the persistent disk resource in AWS (Amazon EBS volume). More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"volumeID\\\"],\\\"type\\\":\\\"object\\\"},\\\"azureDisk\\\":{\\\"description\\\":\\\"AzureDisk represents an Azure Data Disk mount on the host and bind mount to the pod.\\\",\\\"properties\\\":{\\\"cachingMode\\\":{\\\"description\\\":\\\"Host Caching mode: None, Read Only, Read Write.\\\",\\\"type\\\":\\\"string\\\"},\\\"diskName\\\":{\\\"description\\\":\\\"The Name of the data disk in the blob storage\\\",\\\"type\\\":\\\"string\\\"},\\\"diskURI\\\":{\\\"description\\\":\\\"The URI the data disk in the blob storage\\\",\\\"type\\\":\\\"string\\\"},\\\"fsType\\\":{\\\"description\\\":\\\"Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. \\\\\\\"ext4\\\\\\\", \\\\\\\"xfs\\\\\\\", \\\\\\\"ntfs\\\\\\\". Implicitly inferred to be \\\\\\\"ext4\\\\\\\" if unspecified.\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Expected values Shared: multiple blob disks per storage account Dedicated: single blob disk per storage account Managed: azure managed data disk (only in managed availability set). defaults to shared\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts.\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"diskName\\\",\\\"diskURI\\\"],\\\"type\\\":\\\"object\\\"},\\\"azureFile\\\":{\\\"description\\\":\\\"AzureFile represents an Azure File Service mount on the host and bind mount to the pod.\\\",\\\"properties\\\":{\\\"readOnly\\\":{\\\"description\\\":\\\"Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"secretName\\\":{\\\"description\\\":\\\"the name of secret that contains Azure Storage Account Name and Key\\\",\\\"type\\\":\\\"string\\\"},\\\"shareName\\\":{\\\"description\\\":\\\"Share Name\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"secretName\\\",\\\"shareName\\\"],\\\"type\\\":\\\"object\\\"},\\\"cephfs\\\":{\\\"description\\\":\\\"CephFS represents a Ceph FS mount on the host that shares a pod's lifetime\\\",\\\"properties\\\":{\\\"monitors\\\":{\\\"description\\\":\\\"Required: Monitors is a collection of Ceph monitors More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Optional: Used as the mounted root, rather than the full Ceph tree, default is /\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"Optional: Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts. More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it\\\",\\\"type\\\":\\\"boolean\\\"},\\\"secretFile\\\":{\\\"description\\\":\\\"Optional: SecretFile is the path to key ring for User, default is /etc/ceph/user.secret More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it\\\",\\\"type\\\":\\\"string\\\"},\\\"secretRef\\\":{\\\"description\\\":\\\"Optional: SecretRef is reference to the authentication secret for User, default is empty. More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"user\\\":{\\\"description\\\":\\\"Optional: User is the rados user name, default is admin More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"monitors\\\"],\\\"type\\\":\\\"object\\\"},\\\"cinder\\\":{\\\"description\\\":\\\"Cinder represents a cinder volume attached and mounted on kubelets host machine. More info: https://examples.k8s.io/mysql-cinder-pd/README.md\\\",\\\"properties\\\":{\\\"fsType\\\":{\\\"description\\\":\\\"Filesystem type to mount. Must be a filesystem type supported by the host operating system. Examples: \\\\\\\"ext4\\\\\\\", \\\\\\\"xfs\\\\\\\", \\\\\\\"ntfs\\\\\\\". Implicitly inferred to be \\\\\\\"ext4\\\\\\\" if unspecified. More info: https://examples.k8s.io/mysql-cinder-pd/README.md\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"Optional: Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts. More info: https://examples.k8s.io/mysql-cinder-pd/README.md\\\",\\\"type\\\":\\\"boolean\\\"},\\\"secretRef\\\":{\\\"description\\\":\\\"Optional: points to a secret object containing parameters used to connect to OpenStack.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"volumeID\\\":{\\\"description\\\":\\\"volume id used to identify the volume in cinder. More info: https://examples.k8s.io/mysql-cinder-pd/README.md\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"volumeID\\\"],\\\"type\\\":\\\"object\\\"},\\\"configMap\\\":{\\\"description\\\":\\\"ConfigMap represents a configMap that should populate this volume\\\",\\\"properties\\\":{\\\"defaultMode\\\":{\\\"description\\\":\\\"Optional: mode bits to use on created files by default. Must be a value between 0 and 0777. Defaults to 0644. Directories within the path are not affected by this setting. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"items\\\":{\\\"description\\\":\\\"If unspecified, each key-value pair in the Data field of the referenced ConfigMap will be projected into the volume as a file whose name is the key and content is the value. If specified, the listed keys will be projected into the specified paths, and unlisted keys will not be present. If a key is specified which is not present in the ConfigMap, the volume setup will error unless it is marked optional. Paths must be relative and may not contain the '..' path or start with '..'.\\\",\\\"items\\\":{\\\"description\\\":\\\"Maps a string key to a path within a volume.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key to project.\\\",\\\"type\\\":\\\"string\\\"},\\\"mode\\\":{\\\"description\\\":\\\"Optional: mode bits to use on this file, must be a value between 0 and 0777. If not specified, the volume defaultMode will be used. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"path\\\":{\\\"description\\\":\\\"The relative path of the file to map the key to. May not be an absolute path. May not contain the path element '..'. May not start with the string '..'.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"key\\\",\\\"path\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap or its keys must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"type\\\":\\\"object\\\"},\\\"csi\\\":{\\\"description\\\":\\\"CSI (Container Storage Interface) represents storage that is handled by an external CSI driver (Alpha feature).\\\",\\\"properties\\\":{\\\"driver\\\":{\\\"description\\\":\\\"Driver is the name of the CSI driver that handles this volume. Consult with your admin for the correct name as registered in the cluster.\\\",\\\"type\\\":\\\"string\\\"},\\\"fsType\\\":{\\\"description\\\":\\\"Filesystem type to mount. Ex. \\\\\\\"ext4\\\\\\\", \\\\\\\"xfs\\\\\\\", \\\\\\\"ntfs\\\\\\\". If not provided, the empty value is passed to the associated CSI driver which will determine the default filesystem to apply.\\\",\\\"type\\\":\\\"string\\\"},\\\"nodePublishSecretRef\\\":{\\\"description\\\":\\\"NodePublishSecretRef is a reference to the secret object containing sensitive information to pass to the CSI driver to complete the CSI NodePublishVolume and NodeUnpublishVolume calls. This field is optional, and may be empty if no secret is required. If the secret object contains more than one secret, all secret references are passed.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"Specifies a read-only configuration for the volume. Defaults to false (read/write).\\\",\\\"type\\\":\\\"boolean\\\"},\\\"volumeAttributes\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"VolumeAttributes stores driver-specific properties that are passed to the CSI driver. Consult your driver's documentation for supported values.\\\",\\\"type\\\":\\\"object\\\"}},\\\"required\\\":[\\\"driver\\\"],\\\"type\\\":\\\"object\\\"},\\\"downwardAPI\\\":{\\\"description\\\":\\\"DownwardAPI represents downward API about the pod that should populate this volume\\\",\\\"properties\\\":{\\\"defaultMode\\\":{\\\"description\\\":\\\"Optional: mode bits to use on created files by default. Must be a value between 0 and 0777. Defaults to 0644. Directories within the path are not affected by this setting. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"items\\\":{\\\"description\\\":\\\"Items is a list of downward API volume file\\\",\\\"items\\\":{\\\"description\\\":\\\"DownwardAPIVolumeFile represents information to create the file containing the pod field\\\",\\\"properties\\\":{\\\"fieldRef\\\":{\\\"description\\\":\\\"Required: Selects a field of the pod: only annotations, labels, name and namespace are supported.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"Version of the schema the FieldPath is written in terms of, defaults to \\\\\\\"v1\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"},\\\"fieldPath\\\":{\\\"description\\\":\\\"Path of the field to select in the specified API version.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"fieldPath\\\"],\\\"type\\\":\\\"object\\\"},\\\"mode\\\":{\\\"description\\\":\\\"Optional: mode bits to use on this file, must be a value between 0 and 0777. If not specified, the volume defaultMode will be used. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"path\\\":{\\\"description\\\":\\\"Required: Path is the relative path name of the file to be created. Must not be absolute or contain the '..' path. Must be utf-8 encoded. The first item of the relative path must not start with '..'\\\",\\\"type\\\":\\\"string\\\"},\\\"resourceFieldRef\\\":{\\\"description\\\":\\\"Selects a resource of the container: only resources limits and requests (limits.cpu, limits.memory, requests.cpu and requests.memory) are currently supported.\\\",\\\"properties\\\":{\\\"containerName\\\":{\\\"description\\\":\\\"Container name: required for volumes, optional for env vars\\\",\\\"type\\\":\\\"string\\\"},\\\"divisor\\\":{\\\"description\\\":\\\"Specifies the output format of the exposed resources, defaults to \\\\\\\"1\\\\\\\"\\\",\\\"type\\\":\\\"string\\\"},\\\"resource\\\":{\\\"description\\\":\\\"Required: resource to select\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"resource\\\"],\\\"type\\\":\\\"object\\\"}},\\\"required\\\":[\\\"path\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"emptyDir\\\":{\\\"description\\\":\\\"EmptyDir represents a temporary directory that shares a pod's lifetime. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir\\\",\\\"properties\\\":{\\\"medium\\\":{\\\"description\\\":\\\"What type of storage medium should back this directory. The default is \\\\\\\"\\\\\\\" which means to use the node's default medium. Must be an empty string (default) or Memory. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir\\\",\\\"type\\\":\\\"string\\\"},\\\"sizeLimit\\\":{\\\"description\\\":\\\"Total amount of local storage required for this EmptyDir volume. The size limit is also applicable for memory medium. The maximum usage on memory medium EmptyDir would be the minimum value between the SizeLimit specified here and the sum of memory limits of all containers in a pod. The default is nil which means that the limit is undefined. More info: http://kubernetes.io/docs/user-guide/volumes#emptydir\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"fc\\\":{\\\"description\\\":\\\"FC represents a Fibre Channel resource that is attached to a kubelet's host machine and then exposed to the pod.\\\",\\\"properties\\\":{\\\"fsType\\\":{\\\"description\\\":\\\"Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. \\\\\\\"ext4\\\\\\\", \\\\\\\"xfs\\\\\\\", \\\\\\\"ntfs\\\\\\\". Implicitly inferred to be \\\\\\\"ext4\\\\\\\" if unspecified. TODO: how do we prevent errors in the filesystem from compromising the machine\\\",\\\"type\\\":\\\"string\\\"},\\\"lun\\\":{\\\"description\\\":\\\"Optional: FC target lun number\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"Optional: Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"targetWWNs\\\":{\\\"description\\\":\\\"Optional: FC target worldwide names (WWNs)\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"wwids\\\":{\\\"description\\\":\\\"Optional: FC volume world wide identifiers (wwids) Either wwids or combination of targetWWNs and lun must be set, but not both simultaneously.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"flexVolume\\\":{\\\"description\\\":\\\"FlexVolume represents a generic volume resource that is provisioned/attached using an exec based plugin.\\\",\\\"properties\\\":{\\\"driver\\\":{\\\"description\\\":\\\"Driver is the name of the driver to use for this volume.\\\",\\\"type\\\":\\\"string\\\"},\\\"fsType\\\":{\\\"description\\\":\\\"Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. \\\\\\\"ext4\\\\\\\", \\\\\\\"xfs\\\\\\\", \\\\\\\"ntfs\\\\\\\". The default filesystem depends on FlexVolume script.\\\",\\\"type\\\":\\\"string\\\"},\\\"options\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Optional: Extra command options if any.\\\",\\\"type\\\":\\\"object\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"Optional: Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"secretRef\\\":{\\\"description\\\":\\\"Optional: SecretRef is reference to the secret object containing sensitive information to pass to the plugin scripts. This may be empty if no secret object is specified. If the secret object contains more than one secret, all secrets are passed to the plugin scripts.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"required\\\":[\\\"driver\\\"],\\\"type\\\":\\\"object\\\"},\\\"flocker\\\":{\\\"description\\\":\\\"Flocker represents a Flocker volume attached to a kubelet's host machine. This depends on the Flocker control service being running\\\",\\\"properties\\\":{\\\"datasetName\\\":{\\\"description\\\":\\\"Name of the dataset stored as metadata -\\\\u003e name on the dataset for Flocker should be considered as deprecated\\\",\\\"type\\\":\\\"string\\\"},\\\"datasetUUID\\\":{\\\"description\\\":\\\"UUID of the dataset. This is unique identifier of a Flocker dataset\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"gcePersistentDisk\\\":{\\\"description\\\":\\\"GCEPersistentDisk represents a GCE Disk resource that is attached to a kubelet's host machine and then exposed to the pod. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk\\\",\\\"properties\\\":{\\\"fsType\\\":{\\\"description\\\":\\\"Filesystem type of the volume that you want to mount. Tip: Ensure that the filesystem type is supported by the host operating system. Examples: \\\\\\\"ext4\\\\\\\", \\\\\\\"xfs\\\\\\\", \\\\\\\"ntfs\\\\\\\". Implicitly inferred to be \\\\\\\"ext4\\\\\\\" if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk TODO: how do we prevent errors in the filesystem from compromising the machine\\\",\\\"type\\\":\\\"string\\\"},\\\"partition\\\":{\\\"description\\\":\\\"The partition in the volume that you want to mount. If omitted, the default is to mount by volume name. Examples: For volume /dev/sda1, you specify the partition as \\\\\\\"1\\\\\\\". Similarly, the volume partition for /dev/sda is \\\\\\\"0\\\\\\\" (or you can leave the property empty). More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"pdName\\\":{\\\"description\\\":\\\"Unique name of the PD resource in GCE. Used to identify the disk in GCE. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"ReadOnly here will force the ReadOnly setting in VolumeMounts. Defaults to false. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"pdName\\\"],\\\"type\\\":\\\"object\\\"},\\\"gitRepo\\\":{\\\"description\\\":\\\"GitRepo represents a git repository at a particular revision. DEPRECATED: GitRepo is deprecated. To provision a container with a git repo, mount an EmptyDir into an InitContainer that clones the repo using git, then mount the EmptyDir into the Pod's container.\\\",\\\"properties\\\":{\\\"directory\\\":{\\\"description\\\":\\\"Target directory name. Must not contain or start with '..'. If '.' is supplied, the volume directory will be the git repository. Otherwise, if specified, the volume will contain the git repository in the subdirectory with the given name.\\\",\\\"type\\\":\\\"string\\\"},\\\"repository\\\":{\\\"description\\\":\\\"Repository URL\\\",\\\"type\\\":\\\"string\\\"},\\\"revision\\\":{\\\"description\\\":\\\"Commit hash for the specified revision.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"repository\\\"],\\\"type\\\":\\\"object\\\"},\\\"glusterfs\\\":{\\\"description\\\":\\\"Glusterfs represents a Glusterfs mount on the host that shares a pod's lifetime. More info: https://examples.k8s.io/volumes/glusterfs/README.md\\\",\\\"properties\\\":{\\\"endpoints\\\":{\\\"description\\\":\\\"EndpointsName is the endpoint name that details Glusterfs topology. More info: https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod\\\",\\\"type\\\":\\\"string\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path is the Glusterfs volume path. More info: https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"ReadOnly here will force the Glusterfs volume to be mounted with read-only permissions. Defaults to false. More info: https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"endpoints\\\",\\\"path\\\"],\\\"type\\\":\\\"object\\\"},\\\"hostPath\\\":{\\\"description\\\":\\\"HostPath represents a pre-existing file or directory on the host machine that is directly exposed to the container. This is generally used for system agents or other privileged things that are allowed to see the host machine. Most containers will NOT need this. More info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath --- TODO(jonesdl) We need to restrict who can use host directory mounts and who can/can not mount host directories as read/write.\\\",\\\"properties\\\":{\\\"path\\\":{\\\"description\\\":\\\"Path of the directory on the host. If the path is a symlink, it will follow the link to the real path. More info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath\\\",\\\"type\\\":\\\"string\\\"},\\\"type\\\":{\\\"description\\\":\\\"Type for HostPath Volume Defaults to \\\\\\\"\\\\\\\" More info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"path\\\"],\\\"type\\\":\\\"object\\\"},\\\"iscsi\\\":{\\\"description\\\":\\\"ISCSI represents an ISCSI Disk resource that is attached to a kubelet's host machine and then exposed to the pod. More info: https://examples.k8s.io/volumes/iscsi/README.md\\\",\\\"properties\\\":{\\\"chapAuthDiscovery\\\":{\\\"description\\\":\\\"whether support iSCSI Discovery CHAP authentication\\\",\\\"type\\\":\\\"boolean\\\"},\\\"chapAuthSession\\\":{\\\"description\\\":\\\"whether support iSCSI Session CHAP authentication\\\",\\\"type\\\":\\\"boolean\\\"},\\\"fsType\\\":{\\\"description\\\":\\\"Filesystem type of the volume that you want to mount. Tip: Ensure that the filesystem type is supported by the host operating system. Examples: \\\\\\\"ext4\\\\\\\", \\\\\\\"xfs\\\\\\\", \\\\\\\"ntfs\\\\\\\". Implicitly inferred to be \\\\\\\"ext4\\\\\\\" if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#iscsi TODO: how do we prevent errors in the filesystem from compromising the machine\\\",\\\"type\\\":\\\"string\\\"},\\\"initiatorName\\\":{\\\"description\\\":\\\"Custom iSCSI Initiator Name. If initiatorName is specified with iscsiInterface simultaneously, new iSCSI interface \\\\u003ctarget portal\\\\u003e:\\\\u003cvolume name\\\\u003e will be created for the connection.\\\",\\\"type\\\":\\\"string\\\"},\\\"iqn\\\":{\\\"description\\\":\\\"Target iSCSI Qualified Name.\\\",\\\"type\\\":\\\"string\\\"},\\\"iscsiInterface\\\":{\\\"description\\\":\\\"iSCSI Interface Name that uses an iSCSI transport. Defaults to 'default' (tcp).\\\",\\\"type\\\":\\\"string\\\"},\\\"lun\\\":{\\\"description\\\":\\\"iSCSI Target Lun number.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"portals\\\":{\\\"description\\\":\\\"iSCSI Target Portal List. The portal is either an IP or ip_addr:port if the port is other than default (typically TCP ports 860 and 3260).\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"ReadOnly here will force the ReadOnly setting in VolumeMounts. Defaults to false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"secretRef\\\":{\\\"description\\\":\\\"CHAP Secret for iSCSI target and initiator authentication\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"targetPortal\\\":{\\\"description\\\":\\\"iSCSI Target Portal. The Portal is either an IP or ip_addr:port if the port is other than default (typically TCP ports 860 and 3260).\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"iqn\\\",\\\"lun\\\",\\\"targetPortal\\\"],\\\"type\\\":\\\"object\\\"},\\\"name\\\":{\\\"description\\\":\\\"Volume's name. Must be a DNS_LABEL and unique within the pod. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"nfs\\\":{\\\"description\\\":\\\"NFS represents an NFS mount on the host that shares a pod's lifetime More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs\\\",\\\"properties\\\":{\\\"path\\\":{\\\"description\\\":\\\"Path that is exported by the NFS server. More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"ReadOnly here will force the NFS export to be mounted with read-only permissions. Defaults to false. More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs\\\",\\\"type\\\":\\\"boolean\\\"},\\\"server\\\":{\\\"description\\\":\\\"Server is the hostname or IP address of the NFS server. More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"path\\\",\\\"server\\\"],\\\"type\\\":\\\"object\\\"},\\\"persistentVolumeClaim\\\":{\\\"description\\\":\\\"PersistentVolumeClaimVolumeSource represents a reference to a PersistentVolumeClaim in the same namespace. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims\\\",\\\"properties\\\":{\\\"claimName\\\":{\\\"description\\\":\\\"ClaimName is the name of a PersistentVolumeClaim in the same namespace as the pod using this volume. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"Will force the ReadOnly setting in VolumeMounts. Default false.\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"claimName\\\"],\\\"type\\\":\\\"object\\\"},\\\"photonPersistentDisk\\\":{\\\"description\\\":\\\"PhotonPersistentDisk represents a PhotonController persistent disk attached and mounted on kubelets host machine\\\",\\\"properties\\\":{\\\"fsType\\\":{\\\"description\\\":\\\"Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. \\\\\\\"ext4\\\\\\\", \\\\\\\"xfs\\\\\\\", \\\\\\\"ntfs\\\\\\\". Implicitly inferred to be \\\\\\\"ext4\\\\\\\" if unspecified.\\\",\\\"type\\\":\\\"string\\\"},\\\"pdID\\\":{\\\"description\\\":\\\"ID that identifies Photon Controller persistent disk\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"pdID\\\"],\\\"type\\\":\\\"object\\\"},\\\"portworxVolume\\\":{\\\"description\\\":\\\"PortworxVolume represents a portworx volume attached and mounted on kubelets host machine\\\",\\\"properties\\\":{\\\"fsType\\\":{\\\"description\\\":\\\"FSType represents the filesystem type to mount Must be a filesystem type supported by the host operating system. Ex. \\\\\\\"ext4\\\\\\\", \\\\\\\"xfs\\\\\\\". Implicitly inferred to be \\\\\\\"ext4\\\\\\\" if unspecified.\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"volumeID\\\":{\\\"description\\\":\\\"VolumeID uniquely identifies a Portworx volume\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"volumeID\\\"],\\\"type\\\":\\\"object\\\"},\\\"projected\\\":{\\\"description\\\":\\\"Items for all in one resources secrets, configmaps, and downward API\\\",\\\"properties\\\":{\\\"defaultMode\\\":{\\\"description\\\":\\\"Mode bits to use on created files by default. Must be a value between 0 and 0777. Directories within the path are not affected by this setting. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"sources\\\":{\\\"description\\\":\\\"list of volume projections\\\",\\\"items\\\":{\\\"description\\\":\\\"Projection that may be projected along with other supported volume types\\\",\\\"properties\\\":{\\\"configMap\\\":{\\\"description\\\":\\\"information about the configMap data to project\\\",\\\"properties\\\":{\\\"items\\\":{\\\"description\\\":\\\"If unspecified, each key-value pair in the Data field of the referenced ConfigMap will be projected into the volume as a file whose name is the key and content is the value. If specified, the listed keys will be projected into the specified paths, and unlisted keys will not be present. If a key is specified which is not present in the ConfigMap, the volume setup will error unless it is marked optional. Paths must be relative and may not contain the '..' path or start with '..'.\\\",\\\"items\\\":{\\\"description\\\":\\\"Maps a string key to a path within a volume.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key to project.\\\",\\\"type\\\":\\\"string\\\"},\\\"mode\\\":{\\\"description\\\":\\\"Optional: mode bits to use on this file, must be a value between 0 and 0777. If not specified, the volume defaultMode will be used. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"path\\\":{\\\"description\\\":\\\"The relative path of the file to map the key to. May not be an absolute path. May not contain the path element '..'. May not start with the string '..'.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"key\\\",\\\"path\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap or its keys must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"type\\\":\\\"object\\\"},\\\"downwardAPI\\\":{\\\"description\\\":\\\"information about the downwardAPI data to project\\\",\\\"properties\\\":{\\\"items\\\":{\\\"description\\\":\\\"Items is a list of DownwardAPIVolume file\\\",\\\"items\\\":{\\\"description\\\":\\\"DownwardAPIVolumeFile represents information to create the file containing the pod field\\\",\\\"properties\\\":{\\\"fieldRef\\\":{\\\"description\\\":\\\"Required: Selects a field of the pod: only annotations, labels, name and namespace are supported.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"Version of the schema the FieldPath is written in terms of, defaults to \\\\\\\"v1\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"},\\\"fieldPath\\\":{\\\"description\\\":\\\"Path of the field to select in the specified API version.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"fieldPath\\\"],\\\"type\\\":\\\"object\\\"},\\\"mode\\\":{\\\"description\\\":\\\"Optional: mode bits to use on this file, must be a value between 0 and 0777. If not specified, the volume defaultMode will be used. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"path\\\":{\\\"description\\\":\\\"Required: Path is the relative path name of the file to be created. Must not be absolute or contain the '..' path. Must be utf-8 encoded. The first item of the relative path must not start with '..'\\\",\\\"type\\\":\\\"string\\\"},\\\"resourceFieldRef\\\":{\\\"description\\\":\\\"Selects a resource of the container: only resources limits and requests (limits.cpu, limits.memory, requests.cpu and requests.memory) are currently supported.\\\",\\\"properties\\\":{\\\"containerName\\\":{\\\"description\\\":\\\"Container name: required for volumes, optional for env vars\\\",\\\"type\\\":\\\"string\\\"},\\\"divisor\\\":{\\\"description\\\":\\\"Specifies the output format of the exposed resources, defaults to \\\\\\\"1\\\\\\\"\\\",\\\"type\\\":\\\"string\\\"},\\\"resource\\\":{\\\"description\\\":\\\"Required: resource to select\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"resource\\\"],\\\"type\\\":\\\"object\\\"}},\\\"required\\\":[\\\"path\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"secret\\\":{\\\"description\\\":\\\"information about the secret data to project\\\",\\\"properties\\\":{\\\"items\\\":{\\\"description\\\":\\\"If unspecified, each key-value pair in the Data field of the referenced Secret will be projected into the volume as a file whose name is the key and content is the value. If specified, the listed keys will be projected into the specified paths, and unlisted keys will not be present. If a key is specified which is not present in the Secret, the volume setup will error unless it is marked optional. Paths must be relative and may not contain the '..' path or start with '..'.\\\",\\\"items\\\":{\\\"description\\\":\\\"Maps a string key to a path within a volume.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key to project.\\\",\\\"type\\\":\\\"string\\\"},\\\"mode\\\":{\\\"description\\\":\\\"Optional: mode bits to use on this file, must be a value between 0 and 0777. If not specified, the volume defaultMode will be used. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"path\\\":{\\\"description\\\":\\\"The relative path of the file to map the key to. May not be an absolute path. May not contain the path element '..'. May not start with the string '..'.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"key\\\",\\\"path\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"type\\\":\\\"object\\\"},\\\"serviceAccountToken\\\":{\\\"description\\\":\\\"information about the serviceAccountToken data to project\\\",\\\"properties\\\":{\\\"audience\\\":{\\\"description\\\":\\\"Audience is the intended audience of the token. A recipient of a token must identify itself with an identifier specified in the audience of the token, and otherwise should reject the token. The audience defaults to the identifier of the apiserver.\\\",\\\"type\\\":\\\"string\\\"},\\\"expirationSeconds\\\":{\\\"description\\\":\\\"ExpirationSeconds is the requested duration of validity of the service account token. As the token approaches expiration, the kubelet volume plugin will proactively rotate the service account token. The kubelet will start trying to rotate the token if the token is older than 80 percent of its time to live or if the token is older than 24 hours.Defaults to 1 hour and must be at least 10 minutes.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path is the path relative to the mount point of the file to project the token into.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"path\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"sources\\\"],\\\"type\\\":\\\"object\\\"},\\\"quobyte\\\":{\\\"description\\\":\\\"Quobyte represents a Quobyte mount on the host that shares a pod's lifetime\\\",\\\"properties\\\":{\\\"group\\\":{\\\"description\\\":\\\"Group to map volume access to Default is no group\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"ReadOnly here will force the Quobyte volume to be mounted with read-only permissions. Defaults to false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"registry\\\":{\\\"description\\\":\\\"Registry represents a single or multiple Quobyte Registry services specified as a string as host:port pair (multiple entries are separated with commas) which acts as the central registry for volumes\\\",\\\"type\\\":\\\"string\\\"},\\\"tenant\\\":{\\\"description\\\":\\\"Tenant owning the given Quobyte volume in the Backend Used with dynamically provisioned Quobyte volumes, value is set by the plugin\\\",\\\"type\\\":\\\"string\\\"},\\\"user\\\":{\\\"description\\\":\\\"User to map volume access to Defaults to serivceaccount user\\\",\\\"type\\\":\\\"string\\\"},\\\"volume\\\":{\\\"description\\\":\\\"Volume is a string that references an already created Quobyte volume by name.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"registry\\\",\\\"volume\\\"],\\\"type\\\":\\\"object\\\"},\\\"rbd\\\":{\\\"description\\\":\\\"RBD represents a Rados Block Device mount on the host that shares a pod's lifetime. More info: https://examples.k8s.io/volumes/rbd/README.md\\\",\\\"properties\\\":{\\\"fsType\\\":{\\\"description\\\":\\\"Filesystem type of the volume that you want to mount. Tip: Ensure that the filesystem type is supported by the host operating system. Examples: \\\\\\\"ext4\\\\\\\", \\\\\\\"xfs\\\\\\\", \\\\\\\"ntfs\\\\\\\". Implicitly inferred to be \\\\\\\"ext4\\\\\\\" if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#rbd TODO: how do we prevent errors in the filesystem from compromising the machine\\\",\\\"type\\\":\\\"string\\\"},\\\"image\\\":{\\\"description\\\":\\\"The rados image name. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it\\\",\\\"type\\\":\\\"string\\\"},\\\"keyring\\\":{\\\"description\\\":\\\"Keyring is the path to key ring for RBDUser. Default is /etc/ceph/keyring. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it\\\",\\\"type\\\":\\\"string\\\"},\\\"monitors\\\":{\\\"description\\\":\\\"A collection of Ceph monitors. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"pool\\\":{\\\"description\\\":\\\"The rados pool name. Default is rbd. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"ReadOnly here will force the ReadOnly setting in VolumeMounts. Defaults to false. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it\\\",\\\"type\\\":\\\"boolean\\\"},\\\"secretRef\\\":{\\\"description\\\":\\\"SecretRef is name of the authentication secret for RBDUser. If provided overrides keyring. Default is nil. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"user\\\":{\\\"description\\\":\\\"The rados user name. Default is admin. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"image\\\",\\\"monitors\\\"],\\\"type\\\":\\\"object\\\"},\\\"scaleIO\\\":{\\\"description\\\":\\\"ScaleIO represents a ScaleIO persistent volume attached and mounted on Kubernetes nodes.\\\",\\\"properties\\\":{\\\"fsType\\\":{\\\"description\\\":\\\"Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. \\\\\\\"ext4\\\\\\\", \\\\\\\"xfs\\\\\\\", \\\\\\\"ntfs\\\\\\\". Default is \\\\\\\"xfs\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"},\\\"gateway\\\":{\\\"description\\\":\\\"The host address of the ScaleIO API Gateway.\\\",\\\"type\\\":\\\"string\\\"},\\\"protectionDomain\\\":{\\\"description\\\":\\\"The name of the ScaleIO Protection Domain for the configured storage.\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"secretRef\\\":{\\\"description\\\":\\\"SecretRef references to the secret for ScaleIO user and other sensitive information. If this is not provided, Login operation will fail.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"sslEnabled\\\":{\\\"description\\\":\\\"Flag to enable/disable SSL communication with Gateway, default false\\\",\\\"type\\\":\\\"boolean\\\"},\\\"storageMode\\\":{\\\"description\\\":\\\"Indicates whether the storage for a volume should be ThickProvisioned or ThinProvisioned. Default is ThinProvisioned.\\\",\\\"type\\\":\\\"string\\\"},\\\"storagePool\\\":{\\\"description\\\":\\\"The ScaleIO Storage Pool associated with the protection domain.\\\",\\\"type\\\":\\\"string\\\"},\\\"system\\\":{\\\"description\\\":\\\"The name of the storage system as configured in ScaleIO.\\\",\\\"type\\\":\\\"string\\\"},\\\"volumeName\\\":{\\\"description\\\":\\\"The name of a volume already created in the ScaleIO system that is associated with this volume source.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"gateway\\\",\\\"secretRef\\\",\\\"system\\\"],\\\"type\\\":\\\"object\\\"},\\\"secret\\\":{\\\"description\\\":\\\"Secret represents a secret that should populate this volume. More info: https://kubernetes.io/docs/concepts/storage/volumes#secret\\\",\\\"properties\\\":{\\\"defaultMode\\\":{\\\"description\\\":\\\"Optional: mode bits to use on created files by default. Must be a value between 0 and 0777. Defaults to 0644. Directories within the path are not affected by this setting. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"items\\\":{\\\"description\\\":\\\"If unspecified, each key-value pair in the Data field of the referenced Secret will be projected into the volume as a file whose name is the key and content is the value. If specified, the listed keys will be projected into the specified paths, and unlisted keys will not be present. If a key is specified which is not present in the Secret, the volume setup will error unless it is marked optional. Paths must be relative and may not contain the '..' path or start with '..'.\\\",\\\"items\\\":{\\\"description\\\":\\\"Maps a string key to a path within a volume.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key to project.\\\",\\\"type\\\":\\\"string\\\"},\\\"mode\\\":{\\\"description\\\":\\\"Optional: mode bits to use on this file, must be a value between 0 and 0777. If not specified, the volume defaultMode will be used. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"path\\\":{\\\"description\\\":\\\"The relative path of the file to map the key to. May not be an absolute path. May not contain the path element '..'. May not start with the string '..'.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"key\\\",\\\"path\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or its keys must be defined\\\",\\\"type\\\":\\\"boolean\\\"},\\\"secretName\\\":{\\\"description\\\":\\\"Name of the secret in the pod's namespace to use. More info: https://kubernetes.io/docs/concepts/storage/volumes#secret\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"storageos\\\":{\\\"description\\\":\\\"StorageOS represents a StorageOS volume attached and mounted on Kubernetes nodes.\\\",\\\"properties\\\":{\\\"fsType\\\":{\\\"description\\\":\\\"Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. \\\\\\\"ext4\\\\\\\", \\\\\\\"xfs\\\\\\\", \\\\\\\"ntfs\\\\\\\". Implicitly inferred to be \\\\\\\"ext4\\\\\\\" if unspecified.\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"secretRef\\\":{\\\"description\\\":\\\"SecretRef specifies the secret to use for obtaining the StorageOS API credentials. If not specified, default values will be attempted.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"volumeName\\\":{\\\"description\\\":\\\"VolumeName is the human-readable name of the StorageOS volume. Volume names are only unique within a namespace.\\\",\\\"type\\\":\\\"string\\\"},\\\"volumeNamespace\\\":{\\\"description\\\":\\\"VolumeNamespace specifies the scope of the volume within StorageOS. If no namespace is specified then the Pod's namespace will be used. This allows the Kubernetes name scoping to be mirrored within StorageOS for tighter integration. Set VolumeName to any name to override the default behaviour. Set to \\\\\\\"default\\\\\\\" if you are not using namespaces within StorageOS. Namespaces that do not pre-exist within StorageOS will be created.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"vsphereVolume\\\":{\\\"description\\\":\\\"VsphereVolume represents a vSphere volume attached and mounted on kubelets host machine\\\",\\\"properties\\\":{\\\"fsType\\\":{\\\"description\\\":\\\"Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. \\\\\\\"ext4\\\\\\\", \\\\\\\"xfs\\\\\\\", \\\\\\\"ntfs\\\\\\\". Implicitly inferred to be \\\\\\\"ext4\\\\\\\" if unspecified.\\\",\\\"type\\\":\\\"string\\\"},\\\"storagePolicyID\\\":{\\\"description\\\":\\\"Storage Policy Based Management (SPBM) profile ID associated with the StoragePolicyName.\\\",\\\"type\\\":\\\"string\\\"},\\\"storagePolicyName\\\":{\\\"description\\\":\\\"Storage Policy Based Management (SPBM) profile name.\\\",\\\"type\\\":\\\"string\\\"},\\\"volumePath\\\":{\\\"description\\\":\\\"Path that identifies vSphere volume vmdk\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"volumePath\\\"],\\\"type\\\":\\\"object\\\"}},\\\"required\\\":[\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"walCompression\\\":{\\\"description\\\":\\\"Enable compression of the write-ahead log using Snappy. This flag is only available in versions of Prometheus \\\\u003e= 2.11.0.\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"type\\\":\\\"object\\\"},\\\"status\\\":{\\\"description\\\":\\\"Most recent observed status of the Prometheus cluster. Read-only. Not included when requesting from the apiserver, only from the Prometheus Operator API itself. More info: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#spec-and-status\\\",\\\"properties\\\":{\\\"availableReplicas\\\":{\\\"description\\\":\\\"Total number of available pods (ready for at least minReadySeconds) targeted by this Prometheus deployment.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"paused\\\":{\\\"description\\\":\\\"Represents whether any actions on the underlying managed objects are being performed. Only delete actions will be performed.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"replicas\\\":{\\\"description\\\":\\\"Total number of non-terminated pods targeted by this Prometheus deployment (their labels match the selector).\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"unavailableReplicas\\\":{\\\"description\\\":\\\"Total number of unavailable pods targeted by this Prometheus deployment.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"updatedReplicas\\\":{\\\"description\\\":\\\"Total number of non-terminated pods targeted by this Prometheus deployment that have the desired version spec.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"required\\\":[\\\"availableReplicas\\\",\\\"paused\\\",\\\"replicas\\\",\\\"unavailableReplicas\\\",\\\"updatedReplicas\\\"],\\\"type\\\":\\\"object\\\"}},\\\"required\\\":[\\\"spec\\\"],\\\"type\\\":\\\"object\\\"}},\\\"version\\\":\\\"v1\\\",\\\"versions\\\":[{\\\"name\\\":\\\"v1\\\",\\\"served\\\":true,\\\"storage\\\":true}]},\\\"status\\\":{\\\"acceptedNames\\\":{\\\"kind\\\":\\\"\\\",\\\"plural\\\":\\\"\\\"},\\\"conditions\\\":[],\\\"storedVersions\\\":[]}}\\n\"\n },\n \"creationTimestamp\": \"2020-05-04T19:47:40Z\",\n \"generation\": 1,\n \"name\": \"prometheuses.monitoring.coreos.com\",\n \"resourceVersion\": \"1013\",\n \"selfLink\": \"/apis/apiextensions.k8s.io/v1/customresourcedefinitions/prometheuses.monitoring.coreos.com\",\n \"uid\": \"ff0864c8-712f-4b03-844c-339abc00cb15\"\n },\n \"spec\": {\n \"conversion\": {\n \"strategy\": \"None\"\n },\n \"group\": \"monitoring.coreos.com\",\n \"names\": {\n \"kind\": \"Prometheus\",\n \"listKind\": \"PrometheusList\",\n \"plural\": \"prometheuses\",\n \"singular\": \"prometheus\"\n },\n \"scope\": \"Namespaced\",\n \"versions\": [\n {\n \"additionalPrinterColumns\": [\n {\n \"description\": \"The version of Prometheus\",\n \"jsonPath\": \".spec.version\",\n \"name\": \"Version\",\n \"type\": \"string\"\n },\n {\n \"description\": \"The desired replicas number of Prometheuses\",\n \"jsonPath\": \".spec.replicas\",\n \"name\": \"Replicas\",\n \"type\": \"integer\"\n },\n {\n \"jsonPath\": \".metadata.creationTimestamp\",\n \"name\": \"Age\",\n \"type\": \"date\"\n }\n ],\n \"name\": \"v1\",\n \"schema\": {\n \"openAPIV3Schema\": {\n \"description\": \"Prometheus defines a Prometheus deployment.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Specification of the desired behavior of the Prometheus cluster. More info: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#spec-and-status\",\n \"properties\": {\n \"additionalAlertManagerConfigs\": {\n \"description\": \"AdditionalAlertManagerConfigs allows specifying a key of a Secret containing additional Prometheus AlertManager configurations. AlertManager configurations specified are appended to the configurations generated by the Prometheus Operator. Job configurations specified must have the form as specified in the official Prometheus documentation: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#alertmanager_config. As AlertManager configs are appended, the user is responsible to make sure it is valid. Note that using this feature may expose the possibility to break upgrades of Prometheus. It is advised to review Prometheus release notes to ensure that no incompatible AlertManager configs are going to break Prometheus after the upgrade.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"additionalAlertRelabelConfigs\": {\n \"description\": \"AdditionalAlertRelabelConfigs allows specifying a key of a Secret containing additional Prometheus alert relabel configurations. Alert relabel configurations specified are appended to the configurations generated by the Prometheus Operator. Alert relabel configurations specified must have the form as specified in the official Prometheus documentation: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#alert_relabel_configs. As alert relabel configs are appended, the user is responsible to make sure it is valid. Note that using this feature may expose the possibility to break upgrades of Prometheus. It is advised to review Prometheus release notes to ensure that no incompatible alert relabel configs are going to break Prometheus after the upgrade.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"additionalScrapeConfigs\": {\n \"description\": \"AdditionalScrapeConfigs allows specifying a key of a Secret containing additional Prometheus scrape configurations. Scrape configurations specified are appended to the configurations generated by the Prometheus Operator. Job configurations specified must have the form as specified in the official Prometheus documentation: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config. As scrape configs are appended, the user is responsible to make sure it is valid. Note that using this feature may expose the possibility to break upgrades of Prometheus. It is advised to review Prometheus release notes to ensure that no incompatible scrape configs are going to break Prometheus after the upgrade.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"affinity\": {\n \"description\": \"If specified, the pod's scheduling constraints.\",\n \"properties\": {\n \"nodeAffinity\": {\n \"description\": \"Describes node affinity scheduling rules for the pod.\",\n \"properties\": {\n \"preferredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\"weight\\\" to the sum if the node matches the corresponding matchExpressions; the node(s) with the highest sum are the most preferred.\",\n \"items\": {\n \"description\": \"An empty preferred scheduling term matches all objects with implicit weight 0 (i.e. it's a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op).\",\n \"properties\": {\n \"preference\": {\n \"description\": \"A node selector term, associated with the corresponding weight.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"A list of node selector requirements by node's labels.\",\n \"items\": {\n \"description\": \"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchFields\": {\n \"description\": \"A list of node selector requirements by node's fields.\",\n \"items\": {\n \"description\": \"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"weight\": {\n \"description\": \"Weight associated with matching the corresponding nodeSelectorTerm, in the range 1-100.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"preference\",\n \"weight\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"requiredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to an update), the system may or may not try to eventually evict the pod from its node.\",\n \"properties\": {\n \"nodeSelectorTerms\": {\n \"description\": \"Required. A list of node selector terms. The terms are ORed.\",\n \"items\": {\n \"description\": \"A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"A list of node selector requirements by node's labels.\",\n \"items\": {\n \"description\": \"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchFields\": {\n \"description\": \"A list of node selector requirements by node's fields.\",\n \"items\": {\n \"description\": \"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"nodeSelectorTerms\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"podAffinity\": {\n \"description\": \"Describes pod affinity scheduling rules (e.g. co-locate this pod in the same node, zone, etc. as some other pod(s)).\",\n \"properties\": {\n \"preferredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\"weight\\\" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.\",\n \"items\": {\n \"description\": \"The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)\",\n \"properties\": {\n \"podAffinityTerm\": {\n \"description\": \"Required. A pod affinity term, associated with the corresponding weight.\",\n \"properties\": {\n \"labelSelector\": {\n \"description\": \"A label query over a set of resources, in this case pods.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"namespaces\": {\n \"description\": \"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\"this pod's namespace\\\"\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"topologyKey\": {\n \"description\": \"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"topologyKey\"\n ],\n \"type\": \"object\"\n },\n \"weight\": {\n \"description\": \"weight associated with matching the corresponding podAffinityTerm, in the range 1-100.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"podAffinityTerm\",\n \"weight\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"requiredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.\",\n \"items\": {\n \"description\": \"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\u003ctopologyKey\\u003e matches that of any node on which a pod of the set of pods is running\",\n \"properties\": {\n \"labelSelector\": {\n \"description\": \"A label query over a set of resources, in this case pods.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"namespaces\": {\n \"description\": \"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\"this pod's namespace\\\"\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"topologyKey\": {\n \"description\": \"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"topologyKey\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"podAntiAffinity\": {\n \"description\": \"Describes pod anti-affinity scheduling rules (e.g. avoid putting this pod in the same node, zone, etc. as some other pod(s)).\",\n \"properties\": {\n \"preferredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"The scheduler will prefer to schedule pods to nodes that satisfy the anti-affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling anti-affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\"weight\\\" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.\",\n \"items\": {\n \"description\": \"The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)\",\n \"properties\": {\n \"podAffinityTerm\": {\n \"description\": \"Required. A pod affinity term, associated with the corresponding weight.\",\n \"properties\": {\n \"labelSelector\": {\n \"description\": \"A label query over a set of resources, in this case pods.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"namespaces\": {\n \"description\": \"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\"this pod's namespace\\\"\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"topologyKey\": {\n \"description\": \"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"topologyKey\"\n ],\n \"type\": \"object\"\n },\n \"weight\": {\n \"description\": \"weight associated with matching the corresponding podAffinityTerm, in the range 1-100.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"podAffinityTerm\",\n \"weight\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"requiredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"If the anti-affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the anti-affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.\",\n \"items\": {\n \"description\": \"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\u003ctopologyKey\\u003e matches that of any node on which a pod of the set of pods is running\",\n \"properties\": {\n \"labelSelector\": {\n \"description\": \"A label query over a set of resources, in this case pods.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"namespaces\": {\n \"description\": \"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\"this pod's namespace\\\"\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"topologyKey\": {\n \"description\": \"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"topologyKey\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"alerting\": {\n \"description\": \"Define details regarding alerting.\",\n \"properties\": {\n \"alertmanagers\": {\n \"description\": \"AlertmanagerEndpoints Prometheus should fire alerts against.\",\n \"items\": {\n \"description\": \"AlertmanagerEndpoints defines a selection of a single Endpoints object containing alertmanager IPs to fire alerts against.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"Version of the Alertmanager API that Prometheus uses to send alerts. It can be \\\"v1\\\" or \\\"v2\\\".\",\n \"type\": \"string\"\n },\n \"bearerTokenFile\": {\n \"description\": \"BearerTokenFile to read from filesystem to use when authenticating to Alertmanager.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of Endpoints object in Namespace.\",\n \"type\": \"string\"\n },\n \"namespace\": {\n \"description\": \"Namespace of Endpoints object.\",\n \"type\": \"string\"\n },\n \"pathPrefix\": {\n \"description\": \"Prefix for the HTTP path alerts are pushed to.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Port the Alertmanager API is exposed on.\",\n \"x-kubernetes-int-or-string\": true\n },\n \"scheme\": {\n \"description\": \"Scheme to use when firing alerts.\",\n \"type\": \"string\"\n },\n \"tlsConfig\": {\n \"description\": \"TLS Config to use for alertmanager connection.\",\n \"properties\": {\n \"ca\": {\n \"description\": \"Struct containing the CA cert to use for the targets.\",\n \"properties\": {\n \"configMap\": {\n \"description\": \"ConfigMap containing data to use for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key to select.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"secret\": {\n \"description\": \"Secret containing data to use for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"caFile\": {\n \"description\": \"Path to the CA cert in the Prometheus container to use for the targets.\",\n \"type\": \"string\"\n },\n \"cert\": {\n \"description\": \"Struct containing the client cert file for the targets.\",\n \"properties\": {\n \"configMap\": {\n \"description\": \"ConfigMap containing data to use for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key to select.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"secret\": {\n \"description\": \"Secret containing data to use for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"certFile\": {\n \"description\": \"Path to the client cert file in the Prometheus container for the targets.\",\n \"type\": \"string\"\n },\n \"insecureSkipVerify\": {\n \"description\": \"Disable target certificate validation.\",\n \"type\": \"boolean\"\n },\n \"keyFile\": {\n \"description\": \"Path to the client key file in the Prometheus container for the targets.\",\n \"type\": \"string\"\n },\n \"keySecret\": {\n \"description\": \"Secret containing the client key file for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"serverName\": {\n \"description\": \"Used to verify the hostname for the targets.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"required\": [\n \"name\",\n \"namespace\",\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"alertmanagers\"\n ],\n \"type\": \"object\"\n },\n \"apiserverConfig\": {\n \"description\": \"APIServerConfig allows specifying a host and auth methods to access apiserver. If left empty, Prometheus is assumed to run inside of the cluster and will discover API servers automatically and use the pod's CA certificate and bearer token file at /var/run/secrets/kubernetes.io/serviceaccount/.\",\n \"properties\": {\n \"basicAuth\": {\n \"description\": \"BasicAuth allow an endpoint to authenticate over basic authentication\",\n \"properties\": {\n \"password\": {\n \"description\": \"The secret in the service monitor namespace that contains the password for authentication.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"username\": {\n \"description\": \"The secret in the service monitor namespace that contains the username for authentication.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"bearerToken\": {\n \"description\": \"Bearer token for accessing apiserver.\",\n \"type\": \"string\"\n },\n \"bearerTokenFile\": {\n \"description\": \"File to read bearer token for accessing apiserver.\",\n \"type\": \"string\"\n },\n \"host\": {\n \"description\": \"Host of apiserver. A valid string consisting of a hostname or IP followed by an optional port number\",\n \"type\": \"string\"\n },\n \"tlsConfig\": {\n \"description\": \"TLS Config to use for accessing apiserver.\",\n \"properties\": {\n \"ca\": {\n \"description\": \"Struct containing the CA cert to use for the targets.\",\n \"properties\": {\n \"configMap\": {\n \"description\": \"ConfigMap containing data to use for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key to select.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"secret\": {\n \"description\": \"Secret containing data to use for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"caFile\": {\n \"description\": \"Path to the CA cert in the Prometheus container to use for the targets.\",\n \"type\": \"string\"\n },\n \"cert\": {\n \"description\": \"Struct containing the client cert file for the targets.\",\n \"properties\": {\n \"configMap\": {\n \"description\": \"ConfigMap containing data to use for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key to select.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"secret\": {\n \"description\": \"Secret containing data to use for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"certFile\": {\n \"description\": \"Path to the client cert file in the Prometheus container for the targets.\",\n \"type\": \"string\"\n },\n \"insecureSkipVerify\": {\n \"description\": \"Disable target certificate validation.\",\n \"type\": \"boolean\"\n },\n \"keyFile\": {\n \"description\": \"Path to the client key file in the Prometheus container for the targets.\",\n \"type\": \"string\"\n },\n \"keySecret\": {\n \"description\": \"Secret containing the client key file for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"serverName\": {\n \"description\": \"Used to verify the hostname for the targets.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"required\": [\n \"host\"\n ],\n \"type\": \"object\"\n },\n \"arbitraryFSAccessThroughSMs\": {\n \"description\": \"ArbitraryFSAccessThroughSMs configures whether configuration based on a service monitor can access arbitrary files on the file system of the Prometheus container e.g. bearer token files.\",\n \"properties\": {\n \"deny\": {\n \"type\": \"boolean\"\n }\n },\n \"type\": \"object\"\n },\n \"baseImage\": {\n \"description\": \"Base image to use for a Prometheus deployment.\",\n \"type\": \"string\"\n },\n \"configMaps\": {\n \"description\": \"ConfigMaps is a list of ConfigMaps in the same namespace as the Prometheus object, which shall be mounted into the Prometheus Pods. The ConfigMaps are mounted into /etc/prometheus/configmaps/\\u003cconfigmap-name\\u003e.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"containers\": {\n \"description\": \"Containers allows injecting additional containers or modifying operator generated containers. This can be used to allow adding an authentication proxy to a Prometheus pod or to change the behavior of an operator generated container. Containers described here modify an operator generated container if they share the same name and modifications are done via a strategic merge patch. The current container names are: `prometheus`, `prometheus-config-reloader`, `rules-configmap-reloader`, and `thanos-sidecar`. Overriding containers is entirely outside the scope of what the maintainers will support and by doing so, you accept that this behaviour may break at any time without notice.\",\n \"items\": {\n \"description\": \"A single application container that you want to run within a pod.\",\n \"properties\": {\n \"args\": {\n \"description\": \"Arguments to the entrypoint. The docker image's CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"command\": {\n \"description\": \"Entrypoint array. Not executed within a shell. The docker image's ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"env\": {\n \"description\": \"List of environment variables to set in the container. Cannot be updated.\",\n \"items\": {\n \"description\": \"EnvVar represents an environment variable present in a Container.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the environment variable. Must be a C_IDENTIFIER.\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to \\\"\\\".\",\n \"type\": \"string\"\n },\n \"valueFrom\": {\n \"description\": \"Source for the environment variable's value. Cannot be used if value is not empty.\",\n \"properties\": {\n \"configMapKeyRef\": {\n \"description\": \"Selects a key of a ConfigMap.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key to select.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"fieldRef\": {\n \"description\": \"Selects a field of the pod: supports metadata.name, metadata.namespace, metadata.labels, metadata.annotations, spec.nodeName, spec.serviceAccountName, status.hostIP, status.podIP, status.podIPs.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"Version of the schema the FieldPath is written in terms of, defaults to \\\"v1\\\".\",\n \"type\": \"string\"\n },\n \"fieldPath\": {\n \"description\": \"Path of the field to select in the specified API version.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"fieldPath\"\n ],\n \"type\": \"object\"\n },\n \"resourceFieldRef\": {\n \"description\": \"Selects a resource of the container: only resources limits and requests (limits.cpu, limits.memory, limits.ephemeral-storage, requests.cpu, requests.memory and requests.ephemeral-storage) are currently supported.\",\n \"properties\": {\n \"containerName\": {\n \"description\": \"Container name: required for volumes, optional for env vars\",\n \"type\": \"string\"\n },\n \"divisor\": {\n \"description\": \"Specifies the output format of the exposed resources, defaults to \\\"1\\\"\",\n \"type\": \"string\"\n },\n \"resource\": {\n \"description\": \"Required: resource to select\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"resource\"\n ],\n \"type\": \"object\"\n },\n \"secretKeyRef\": {\n \"description\": \"Selects a key of a secret in the pod's namespace\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"envFrom\": {\n \"description\": \"List of sources to populate environment variables in the container. The keys defined within a source must be a C_IDENTIFIER. All invalid keys will be reported as an event when the container is starting. When a key exists in multiple sources, the value associated with the last source will take precedence. Values defined by an Env with a duplicate key will take precedence. Cannot be updated.\",\n \"items\": {\n \"description\": \"EnvFromSource represents the source of a set of ConfigMaps\",\n \"properties\": {\n \"configMapRef\": {\n \"description\": \"The ConfigMap to select from\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"type\": \"object\"\n },\n \"prefix\": {\n \"description\": \"An optional identifier to prepend to each key in the ConfigMap. Must be a C_IDENTIFIER.\",\n \"type\": \"string\"\n },\n \"secretRef\": {\n \"description\": \"The Secret to select from\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"image\": {\n \"description\": \"Docker image name. More info: https://kubernetes.io/docs/concepts/containers/images This field is optional to allow higher level config management to default or override container images in workload controllers like Deployments and StatefulSets.\",\n \"type\": \"string\"\n },\n \"imagePullPolicy\": {\n \"description\": \"Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images\",\n \"type\": \"string\"\n },\n \"lifecycle\": {\n \"description\": \"Actions that the management system should take in response to container lifecycle events. Cannot be updated.\",\n \"properties\": {\n \"postStart\": {\n \"description\": \"PostStart is called immediately after a container is created. If the handler fails, the container is terminated and restarted according to its restart policy. Other management of the container blocks until the hook completes. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks\",\n \"properties\": {\n \"exec\": {\n \"description\": \"One and only one of the following should be specified. Exec specifies the action to take.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"httpGet\": {\n \"description\": \"HTTPGet specifies the http request to perform.\",\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\",\n \"x-kubernetes-int-or-string\": true\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported TODO: implement a realistic TCP lifecycle hook\",\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Number or name of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\",\n \"x-kubernetes-int-or-string\": true\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"preStop\": {\n \"description\": \"PreStop is called immediately before a container is terminated due to an API request or management event such as liveness/startup probe failure, preemption, resource contention, etc. The handler is not called if the container crashes or exits. The reason for termination is passed to the handler. The Pod's termination grace period countdown begins before the PreStop hooked is executed. Regardless of the outcome of the handler, the container will eventually terminate within the Pod's termination grace period. Other management of the container blocks until the hook completes or until the termination grace period is reached. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks\",\n \"properties\": {\n \"exec\": {\n \"description\": \"One and only one of the following should be specified. Exec specifies the action to take.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"httpGet\": {\n \"description\": \"HTTPGet specifies the http request to perform.\",\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\",\n \"x-kubernetes-int-or-string\": true\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported TODO: implement a realistic TCP lifecycle hook\",\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Number or name of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\",\n \"x-kubernetes-int-or-string\": true\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"livenessProbe\": {\n \"description\": \"Periodic probe of container liveness. Container will be restarted if the probe fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"properties\": {\n \"exec\": {\n \"description\": \"One and only one of the following should be specified. Exec specifies the action to take.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"failureThreshold\": {\n \"description\": \"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"httpGet\": {\n \"description\": \"HTTPGet specifies the http request to perform.\",\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\",\n \"x-kubernetes-int-or-string\": true\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"initialDelaySeconds\": {\n \"description\": \"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"periodSeconds\": {\n \"description\": \"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"successThreshold\": {\n \"description\": \"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness and startup. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported TODO: implement a realistic TCP lifecycle hook\",\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Number or name of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\",\n \"x-kubernetes-int-or-string\": true\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"timeoutSeconds\": {\n \"description\": \"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n },\n \"name\": {\n \"description\": \"Name of the container specified as a DNS_LABEL. Each container in a pod must have a unique name (DNS_LABEL). Cannot be updated.\",\n \"type\": \"string\"\n },\n \"ports\": {\n \"description\": \"List of ports to expose from the container. Exposing a port here gives the system additional information about the network connections a container uses, but is primarily informational. Not specifying a port here DOES NOT prevent that port from being exposed. Any port which is listening on the default \\\"0.0.0.0\\\" address inside a container will be accessible from the network. Cannot be updated.\",\n \"items\": {\n \"description\": \"ContainerPort represents a network port in a single container.\",\n \"properties\": {\n \"containerPort\": {\n \"description\": \"Number of port to expose on the pod's IP address. This must be a valid port number, 0 \\u003c x \\u003c 65536.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"hostIP\": {\n \"description\": \"What host IP to bind the external port to.\",\n \"type\": \"string\"\n },\n \"hostPort\": {\n \"description\": \"Number of port to expose on the host. If specified, this must be a valid port number, 0 \\u003c x \\u003c 65536. If HostNetwork is specified, this must match ContainerPort. Most containers do not need this.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"name\": {\n \"description\": \"If specified, this must be an IANA_SVC_NAME and unique within the pod. Each named port in a pod must have a unique name. Name for the port that can be referred to by services.\",\n \"type\": \"string\"\n },\n \"protocol\": {\n \"description\": \"Protocol for port. Must be UDP, TCP, or SCTP. Defaults to \\\"TCP\\\".\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"containerPort\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"readinessProbe\": {\n \"description\": \"Periodic probe of container service readiness. Container will be removed from service endpoints if the probe fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"properties\": {\n \"exec\": {\n \"description\": \"One and only one of the following should be specified. Exec specifies the action to take.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"failureThreshold\": {\n \"description\": \"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"httpGet\": {\n \"description\": \"HTTPGet specifies the http request to perform.\",\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\",\n \"x-kubernetes-int-or-string\": true\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"initialDelaySeconds\": {\n \"description\": \"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"periodSeconds\": {\n \"description\": \"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"successThreshold\": {\n \"description\": \"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness and startup. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported TODO: implement a realistic TCP lifecycle hook\",\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Number or name of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\",\n \"x-kubernetes-int-or-string\": true\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"timeoutSeconds\": {\n \"description\": \"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n },\n \"resources\": {\n \"description\": \"Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"properties\": {\n \"limits\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"securityContext\": {\n \"description\": \"Security options the pod should run with. More info: https://kubernetes.io/docs/concepts/policy/security-context/ More info: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/\",\n \"properties\": {\n \"allowPrivilegeEscalation\": {\n \"description\": \"AllowPrivilegeEscalation controls whether a process can gain more privileges than its parent process. This bool directly controls if the no_new_privs flag will be set on the container process. AllowPrivilegeEscalation is true always when the container is: 1) run as Privileged 2) has CAP_SYS_ADMIN\",\n \"type\": \"boolean\"\n },\n \"capabilities\": {\n \"description\": \"The capabilities to add/drop when running containers. Defaults to the default set of capabilities granted by the container runtime.\",\n \"properties\": {\n \"add\": {\n \"description\": \"Added capabilities\",\n \"items\": {\n \"description\": \"Capability represent POSIX capabilities type\",\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"drop\": {\n \"description\": \"Removed capabilities\",\n \"items\": {\n \"description\": \"Capability represent POSIX capabilities type\",\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"privileged\": {\n \"description\": \"Run container in privileged mode. Processes in privileged containers are essentially equivalent to root on the host. Defaults to false.\",\n \"type\": \"boolean\"\n },\n \"procMount\": {\n \"description\": \"procMount denotes the type of proc mount to use for the containers. The default is DefaultProcMount which uses the container runtime defaults for readonly paths and masked paths. This requires the ProcMountType feature flag to be enabled.\",\n \"type\": \"string\"\n },\n \"readOnlyRootFilesystem\": {\n \"description\": \"Whether this container has a read-only root filesystem. Default is false.\",\n \"type\": \"boolean\"\n },\n \"runAsGroup\": {\n \"description\": \"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"runAsNonRoot\": {\n \"description\": \"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"type\": \"boolean\"\n },\n \"runAsUser\": {\n \"description\": \"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"seLinuxOptions\": {\n \"description\": \"The SELinux context to be applied to the container. If unspecified, the container runtime will allocate a random SELinux context for each container. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"properties\": {\n \"level\": {\n \"description\": \"Level is SELinux level label that applies to the container.\",\n \"type\": \"string\"\n },\n \"role\": {\n \"description\": \"Role is a SELinux role label that applies to the container.\",\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"Type is a SELinux type label that applies to the container.\",\n \"type\": \"string\"\n },\n \"user\": {\n \"description\": \"User is a SELinux user label that applies to the container.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"windowsOptions\": {\n \"description\": \"The Windows specific settings applied to all containers. If unspecified, the options from the PodSecurityContext will be used. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"properties\": {\n \"gmsaCredentialSpec\": {\n \"description\": \"GMSACredentialSpec is where the GMSA admission webhook (https://github.com/kubernetes-sigs/windows-gmsa) inlines the contents of the GMSA credential spec named by the GMSACredentialSpecName field.\",\n \"type\": \"string\"\n },\n \"gmsaCredentialSpecName\": {\n \"description\": \"GMSACredentialSpecName is the name of the GMSA credential spec to use.\",\n \"type\": \"string\"\n },\n \"runAsUserName\": {\n \"description\": \"The UserName in Windows to run the entrypoint of the container process. Defaults to the user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"startupProbe\": {\n \"description\": \"StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod's lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. This is a beta feature enabled by the StartupProbe feature flag. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"properties\": {\n \"exec\": {\n \"description\": \"One and only one of the following should be specified. Exec specifies the action to take.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"failureThreshold\": {\n \"description\": \"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"httpGet\": {\n \"description\": \"HTTPGet specifies the http request to perform.\",\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\",\n \"x-kubernetes-int-or-string\": true\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"initialDelaySeconds\": {\n \"description\": \"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"periodSeconds\": {\n \"description\": \"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"successThreshold\": {\n \"description\": \"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness and startup. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported TODO: implement a realistic TCP lifecycle hook\",\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Number or name of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\",\n \"x-kubernetes-int-or-string\": true\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"timeoutSeconds\": {\n \"description\": \"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n },\n \"stdin\": {\n \"description\": \"Whether this container should allocate a buffer for stdin in the container runtime. If this is not set, reads from stdin in the container will always result in EOF. Default is false.\",\n \"type\": \"boolean\"\n },\n \"stdinOnce\": {\n \"description\": \"Whether the container runtime should close the stdin channel after it has been opened by a single attach. When stdin is true the stdin stream will remain open across multiple attach sessions. If stdinOnce is set to true, stdin is opened on container start, is empty until the first client attaches to stdin, and then remains open and accepts data until the client disconnects, at which time stdin is closed and remains closed until the container is restarted. If this flag is false, a container processes that reads from stdin will never receive an EOF. Default is false\",\n \"type\": \"boolean\"\n },\n \"terminationMessagePath\": {\n \"description\": \"Optional: Path at which the file to which the container's termination message will be written is mounted into the container's filesystem. Message written is intended to be brief final status, such as an assertion failure message. Will be truncated by the node if greater than 4096 bytes. The total message length across all containers will be limited to 12kb. Defaults to /dev/termination-log. Cannot be updated.\",\n \"type\": \"string\"\n },\n \"terminationMessagePolicy\": {\n \"description\": \"Indicate how the termination message should be populated. File will use the contents of terminationMessagePath to populate the container status message on both success and failure. FallbackToLogsOnError will use the last chunk of container log output if the termination message file is empty and the container exited with an error. The log output is limited to 2048 bytes or 80 lines, whichever is smaller. Defaults to File. Cannot be updated.\",\n \"type\": \"string\"\n },\n \"tty\": {\n \"description\": \"Whether this container should allocate a TTY for itself, also requires 'stdin' to be true. Default is false.\",\n \"type\": \"boolean\"\n },\n \"volumeDevices\": {\n \"description\": \"volumeDevices is the list of block devices to be used by the container.\",\n \"items\": {\n \"description\": \"volumeDevice describes a mapping of a raw block device within a container.\",\n \"properties\": {\n \"devicePath\": {\n \"description\": \"devicePath is the path inside of the container that the device will be mapped to.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"name must match the name of a persistentVolumeClaim in the pod\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"devicePath\",\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"volumeMounts\": {\n \"description\": \"Pod volumes to mount into the container's filesystem. Cannot be updated.\",\n \"items\": {\n \"description\": \"VolumeMount describes a mounting of a Volume within a container.\",\n \"properties\": {\n \"mountPath\": {\n \"description\": \"Path within the container at which the volume should be mounted. Must not contain ':'.\",\n \"type\": \"string\"\n },\n \"mountPropagation\": {\n \"description\": \"mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"This must match the Name of a Volume.\",\n \"type\": \"string\"\n },\n \"readOnly\": {\n \"description\": \"Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.\",\n \"type\": \"boolean\"\n },\n \"subPath\": {\n \"description\": \"Path within the volume from which the container's volume should be mounted. Defaults to \\\"\\\" (volume's root).\",\n \"type\": \"string\"\n },\n \"subPathExpr\": {\n \"description\": \"Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to \\\"\\\" (volume's root). SubPathExpr and SubPath are mutually exclusive.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"mountPath\",\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"workingDir\": {\n \"description\": \"Container's working directory. If not specified, the container runtime's default will be used, which might be configured in the container image. Cannot be updated.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"disableCompaction\": {\n \"description\": \"Disable prometheus compaction.\",\n \"type\": \"boolean\"\n },\n \"enableAdminAPI\": {\n \"description\": \"Enable access to prometheus web admin API. Defaults to the value of `false`. WARNING: Enabling the admin APIs enables mutating endpoints, to delete data, shutdown Prometheus, and more. Enabling this should be done with care and the user is advised to add additional authentication authorization via a proxy to ensure only clients authorized to perform these actions can do so. For more information see https://prometheus.io/docs/prometheus/latest/querying/api/#tsdb-admin-apis\",\n \"type\": \"boolean\"\n },\n \"enforcedNamespaceLabel\": {\n \"description\": \"EnforcedNamespaceLabel enforces adding a namespace label of origin for each alert and metric that is user created. The label value will always be the namespace of the object that is being created.\",\n \"type\": \"string\"\n },\n \"evaluationInterval\": {\n \"description\": \"Interval between consecutive evaluations.\",\n \"type\": \"string\"\n },\n \"externalLabels\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"The labels to add to any time series or alerts when communicating with external systems (federation, remote storage, Alertmanager).\",\n \"type\": \"object\"\n },\n \"externalUrl\": {\n \"description\": \"The external URL the Prometheus instances will be available under. This is necessary to generate correct URLs. This is necessary if Prometheus is not served from root of a DNS name.\",\n \"type\": \"string\"\n },\n \"ignoreNamespaceSelectors\": {\n \"description\": \"IgnoreNamespaceSelectors if set to true will ignore NamespaceSelector settings from the podmonitor and servicemonitor configs, and they will only discover endpoints within their current namespace. Defaults to false.\",\n \"type\": \"boolean\"\n },\n \"image\": {\n \"description\": \"Image if specified has precedence over baseImage, tag and sha combinations. Specifying the version is still necessary to ensure the Prometheus Operator knows what version of Prometheus is being configured.\",\n \"type\": \"string\"\n },\n \"imagePullSecrets\": {\n \"description\": \"An optional list of references to secrets in the same namespace to use for pulling prometheus and alertmanager images from registries see http://kubernetes.io/docs/user-guide/images#specifying-imagepullsecrets-on-a-pod\",\n \"items\": {\n \"description\": \"LocalObjectReference contains enough information to let you locate the referenced object inside the same namespace.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"initContainers\": {\n \"description\": \"InitContainers allows adding initContainers to the pod definition. Those can be used to e.g. fetch secrets for injection into the Prometheus configuration from external sources. Any errors during the execution of an initContainer will lead to a restart of the Pod. More info: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/ Using initContainers for any use case other then secret fetching is entirely outside the scope of what the maintainers will support and by doing so, you accept that this behaviour may break at any time without notice.\",\n \"items\": {\n \"description\": \"A single application container that you want to run within a pod.\",\n \"properties\": {\n \"args\": {\n \"description\": \"Arguments to the entrypoint. The docker image's CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"command\": {\n \"description\": \"Entrypoint array. Not executed within a shell. The docker image's ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"env\": {\n \"description\": \"List of environment variables to set in the container. Cannot be updated.\",\n \"items\": {\n \"description\": \"EnvVar represents an environment variable present in a Container.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the environment variable. Must be a C_IDENTIFIER.\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to \\\"\\\".\",\n \"type\": \"string\"\n },\n \"valueFrom\": {\n \"description\": \"Source for the environment variable's value. Cannot be used if value is not empty.\",\n \"properties\": {\n \"configMapKeyRef\": {\n \"description\": \"Selects a key of a ConfigMap.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key to select.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"fieldRef\": {\n \"description\": \"Selects a field of the pod: supports metadata.name, metadata.namespace, metadata.labels, metadata.annotations, spec.nodeName, spec.serviceAccountName, status.hostIP, status.podIP, status.podIPs.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"Version of the schema the FieldPath is written in terms of, defaults to \\\"v1\\\".\",\n \"type\": \"string\"\n },\n \"fieldPath\": {\n \"description\": \"Path of the field to select in the specified API version.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"fieldPath\"\n ],\n \"type\": \"object\"\n },\n \"resourceFieldRef\": {\n \"description\": \"Selects a resource of the container: only resources limits and requests (limits.cpu, limits.memory, limits.ephemeral-storage, requests.cpu, requests.memory and requests.ephemeral-storage) are currently supported.\",\n \"properties\": {\n \"containerName\": {\n \"description\": \"Container name: required for volumes, optional for env vars\",\n \"type\": \"string\"\n },\n \"divisor\": {\n \"description\": \"Specifies the output format of the exposed resources, defaults to \\\"1\\\"\",\n \"type\": \"string\"\n },\n \"resource\": {\n \"description\": \"Required: resource to select\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"resource\"\n ],\n \"type\": \"object\"\n },\n \"secretKeyRef\": {\n \"description\": \"Selects a key of a secret in the pod's namespace\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"envFrom\": {\n \"description\": \"List of sources to populate environment variables in the container. The keys defined within a source must be a C_IDENTIFIER. All invalid keys will be reported as an event when the container is starting. When a key exists in multiple sources, the value associated with the last source will take precedence. Values defined by an Env with a duplicate key will take precedence. Cannot be updated.\",\n \"items\": {\n \"description\": \"EnvFromSource represents the source of a set of ConfigMaps\",\n \"properties\": {\n \"configMapRef\": {\n \"description\": \"The ConfigMap to select from\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"type\": \"object\"\n },\n \"prefix\": {\n \"description\": \"An optional identifier to prepend to each key in the ConfigMap. Must be a C_IDENTIFIER.\",\n \"type\": \"string\"\n },\n \"secretRef\": {\n \"description\": \"The Secret to select from\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"image\": {\n \"description\": \"Docker image name. More info: https://kubernetes.io/docs/concepts/containers/images This field is optional to allow higher level config management to default or override container images in workload controllers like Deployments and StatefulSets.\",\n \"type\": \"string\"\n },\n \"imagePullPolicy\": {\n \"description\": \"Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images\",\n \"type\": \"string\"\n },\n \"lifecycle\": {\n \"description\": \"Actions that the management system should take in response to container lifecycle events. Cannot be updated.\",\n \"properties\": {\n \"postStart\": {\n \"description\": \"PostStart is called immediately after a container is created. If the handler fails, the container is terminated and restarted according to its restart policy. Other management of the container blocks until the hook completes. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks\",\n \"properties\": {\n \"exec\": {\n \"description\": \"One and only one of the following should be specified. Exec specifies the action to take.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"httpGet\": {\n \"description\": \"HTTPGet specifies the http request to perform.\",\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\",\n \"x-kubernetes-int-or-string\": true\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported TODO: implement a realistic TCP lifecycle hook\",\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Number or name of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\",\n \"x-kubernetes-int-or-string\": true\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"preStop\": {\n \"description\": \"PreStop is called immediately before a container is terminated due to an API request or management event such as liveness/startup probe failure, preemption, resource contention, etc. The handler is not called if the container crashes or exits. The reason for termination is passed to the handler. The Pod's termination grace period countdown begins before the PreStop hooked is executed. Regardless of the outcome of the handler, the container will eventually terminate within the Pod's termination grace period. Other management of the container blocks until the hook completes or until the termination grace period is reached. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks\",\n \"properties\": {\n \"exec\": {\n \"description\": \"One and only one of the following should be specified. Exec specifies the action to take.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"httpGet\": {\n \"description\": \"HTTPGet specifies the http request to perform.\",\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\",\n \"x-kubernetes-int-or-string\": true\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported TODO: implement a realistic TCP lifecycle hook\",\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Number or name of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\",\n \"x-kubernetes-int-or-string\": true\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"livenessProbe\": {\n \"description\": \"Periodic probe of container liveness. Container will be restarted if the probe fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"properties\": {\n \"exec\": {\n \"description\": \"One and only one of the following should be specified. Exec specifies the action to take.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"failureThreshold\": {\n \"description\": \"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"httpGet\": {\n \"description\": \"HTTPGet specifies the http request to perform.\",\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\",\n \"x-kubernetes-int-or-string\": true\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"initialDelaySeconds\": {\n \"description\": \"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"periodSeconds\": {\n \"description\": \"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"successThreshold\": {\n \"description\": \"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness and startup. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported TODO: implement a realistic TCP lifecycle hook\",\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Number or name of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\",\n \"x-kubernetes-int-or-string\": true\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"timeoutSeconds\": {\n \"description\": \"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n },\n \"name\": {\n \"description\": \"Name of the container specified as a DNS_LABEL. Each container in a pod must have a unique name (DNS_LABEL). Cannot be updated.\",\n \"type\": \"string\"\n },\n \"ports\": {\n \"description\": \"List of ports to expose from the container. Exposing a port here gives the system additional information about the network connections a container uses, but is primarily informational. Not specifying a port here DOES NOT prevent that port from being exposed. Any port which is listening on the default \\\"0.0.0.0\\\" address inside a container will be accessible from the network. Cannot be updated.\",\n \"items\": {\n \"description\": \"ContainerPort represents a network port in a single container.\",\n \"properties\": {\n \"containerPort\": {\n \"description\": \"Number of port to expose on the pod's IP address. This must be a valid port number, 0 \\u003c x \\u003c 65536.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"hostIP\": {\n \"description\": \"What host IP to bind the external port to.\",\n \"type\": \"string\"\n },\n \"hostPort\": {\n \"description\": \"Number of port to expose on the host. If specified, this must be a valid port number, 0 \\u003c x \\u003c 65536. If HostNetwork is specified, this must match ContainerPort. Most containers do not need this.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"name\": {\n \"description\": \"If specified, this must be an IANA_SVC_NAME and unique within the pod. Each named port in a pod must have a unique name. Name for the port that can be referred to by services.\",\n \"type\": \"string\"\n },\n \"protocol\": {\n \"description\": \"Protocol for port. Must be UDP, TCP, or SCTP. Defaults to \\\"TCP\\\".\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"containerPort\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"readinessProbe\": {\n \"description\": \"Periodic probe of container service readiness. Container will be removed from service endpoints if the probe fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"properties\": {\n \"exec\": {\n \"description\": \"One and only one of the following should be specified. Exec specifies the action to take.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"failureThreshold\": {\n \"description\": \"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"httpGet\": {\n \"description\": \"HTTPGet specifies the http request to perform.\",\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\",\n \"x-kubernetes-int-or-string\": true\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"initialDelaySeconds\": {\n \"description\": \"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"periodSeconds\": {\n \"description\": \"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"successThreshold\": {\n \"description\": \"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness and startup. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported TODO: implement a realistic TCP lifecycle hook\",\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Number or name of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\",\n \"x-kubernetes-int-or-string\": true\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"timeoutSeconds\": {\n \"description\": \"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n },\n \"resources\": {\n \"description\": \"Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"properties\": {\n \"limits\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"securityContext\": {\n \"description\": \"Security options the pod should run with. More info: https://kubernetes.io/docs/concepts/policy/security-context/ More info: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/\",\n \"properties\": {\n \"allowPrivilegeEscalation\": {\n \"description\": \"AllowPrivilegeEscalation controls whether a process can gain more privileges than its parent process. This bool directly controls if the no_new_privs flag will be set on the container process. AllowPrivilegeEscalation is true always when the container is: 1) run as Privileged 2) has CAP_SYS_ADMIN\",\n \"type\": \"boolean\"\n },\n \"capabilities\": {\n \"description\": \"The capabilities to add/drop when running containers. Defaults to the default set of capabilities granted by the container runtime.\",\n \"properties\": {\n \"add\": {\n \"description\": \"Added capabilities\",\n \"items\": {\n \"description\": \"Capability represent POSIX capabilities type\",\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"drop\": {\n \"description\": \"Removed capabilities\",\n \"items\": {\n \"description\": \"Capability represent POSIX capabilities type\",\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"privileged\": {\n \"description\": \"Run container in privileged mode. Processes in privileged containers are essentially equivalent to root on the host. Defaults to false.\",\n \"type\": \"boolean\"\n },\n \"procMount\": {\n \"description\": \"procMount denotes the type of proc mount to use for the containers. The default is DefaultProcMount which uses the container runtime defaults for readonly paths and masked paths. This requires the ProcMountType feature flag to be enabled.\",\n \"type\": \"string\"\n },\n \"readOnlyRootFilesystem\": {\n \"description\": \"Whether this container has a read-only root filesystem. Default is false.\",\n \"type\": \"boolean\"\n },\n \"runAsGroup\": {\n \"description\": \"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"runAsNonRoot\": {\n \"description\": \"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"type\": \"boolean\"\n },\n \"runAsUser\": {\n \"description\": \"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"seLinuxOptions\": {\n \"description\": \"The SELinux context to be applied to the container. If unspecified, the container runtime will allocate a random SELinux context for each container. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"properties\": {\n \"level\": {\n \"description\": \"Level is SELinux level label that applies to the container.\",\n \"type\": \"string\"\n },\n \"role\": {\n \"description\": \"Role is a SELinux role label that applies to the container.\",\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"Type is a SELinux type label that applies to the container.\",\n \"type\": \"string\"\n },\n \"user\": {\n \"description\": \"User is a SELinux user label that applies to the container.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"windowsOptions\": {\n \"description\": \"The Windows specific settings applied to all containers. If unspecified, the options from the PodSecurityContext will be used. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"properties\": {\n \"gmsaCredentialSpec\": {\n \"description\": \"GMSACredentialSpec is where the GMSA admission webhook (https://github.com/kubernetes-sigs/windows-gmsa) inlines the contents of the GMSA credential spec named by the GMSACredentialSpecName field.\",\n \"type\": \"string\"\n },\n \"gmsaCredentialSpecName\": {\n \"description\": \"GMSACredentialSpecName is the name of the GMSA credential spec to use.\",\n \"type\": \"string\"\n },\n \"runAsUserName\": {\n \"description\": \"The UserName in Windows to run the entrypoint of the container process. Defaults to the user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"startupProbe\": {\n \"description\": \"StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod's lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. This is a beta feature enabled by the StartupProbe feature flag. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"properties\": {\n \"exec\": {\n \"description\": \"One and only one of the following should be specified. Exec specifies the action to take.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"failureThreshold\": {\n \"description\": \"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"httpGet\": {\n \"description\": \"HTTPGet specifies the http request to perform.\",\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\",\n \"x-kubernetes-int-or-string\": true\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"initialDelaySeconds\": {\n \"description\": \"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"periodSeconds\": {\n \"description\": \"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"successThreshold\": {\n \"description\": \"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness and startup. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocket specifies an action involving a TCP port. TCP hooks not yet supported TODO: implement a realistic TCP lifecycle hook\",\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ],\n \"description\": \"Number or name of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.\",\n \"x-kubernetes-int-or-string\": true\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"timeoutSeconds\": {\n \"description\": \"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n },\n \"stdin\": {\n \"description\": \"Whether this container should allocate a buffer for stdin in the container runtime. If this is not set, reads from stdin in the container will always result in EOF. Default is false.\",\n \"type\": \"boolean\"\n },\n \"stdinOnce\": {\n \"description\": \"Whether the container runtime should close the stdin channel after it has been opened by a single attach. When stdin is true the stdin stream will remain open across multiple attach sessions. If stdinOnce is set to true, stdin is opened on container start, is empty until the first client attaches to stdin, and then remains open and accepts data until the client disconnects, at which time stdin is closed and remains closed until the container is restarted. If this flag is false, a container processes that reads from stdin will never receive an EOF. Default is false\",\n \"type\": \"boolean\"\n },\n \"terminationMessagePath\": {\n \"description\": \"Optional: Path at which the file to which the container's termination message will be written is mounted into the container's filesystem. Message written is intended to be brief final status, such as an assertion failure message. Will be truncated by the node if greater than 4096 bytes. The total message length across all containers will be limited to 12kb. Defaults to /dev/termination-log. Cannot be updated.\",\n \"type\": \"string\"\n },\n \"terminationMessagePolicy\": {\n \"description\": \"Indicate how the termination message should be populated. File will use the contents of terminationMessagePath to populate the container status message on both success and failure. FallbackToLogsOnError will use the last chunk of container log output if the termination message file is empty and the container exited with an error. The log output is limited to 2048 bytes or 80 lines, whichever is smaller. Defaults to File. Cannot be updated.\",\n \"type\": \"string\"\n },\n \"tty\": {\n \"description\": \"Whether this container should allocate a TTY for itself, also requires 'stdin' to be true. Default is false.\",\n \"type\": \"boolean\"\n },\n \"volumeDevices\": {\n \"description\": \"volumeDevices is the list of block devices to be used by the container.\",\n \"items\": {\n \"description\": \"volumeDevice describes a mapping of a raw block device within a container.\",\n \"properties\": {\n \"devicePath\": {\n \"description\": \"devicePath is the path inside of the container that the device will be mapped to.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"name must match the name of a persistentVolumeClaim in the pod\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"devicePath\",\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"volumeMounts\": {\n \"description\": \"Pod volumes to mount into the container's filesystem. Cannot be updated.\",\n \"items\": {\n \"description\": \"VolumeMount describes a mounting of a Volume within a container.\",\n \"properties\": {\n \"mountPath\": {\n \"description\": \"Path within the container at which the volume should be mounted. Must not contain ':'.\",\n \"type\": \"string\"\n },\n \"mountPropagation\": {\n \"description\": \"mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"This must match the Name of a Volume.\",\n \"type\": \"string\"\n },\n \"readOnly\": {\n \"description\": \"Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.\",\n \"type\": \"boolean\"\n },\n \"subPath\": {\n \"description\": \"Path within the volume from which the container's volume should be mounted. Defaults to \\\"\\\" (volume's root).\",\n \"type\": \"string\"\n },\n \"subPathExpr\": {\n \"description\": \"Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to \\\"\\\" (volume's root). SubPathExpr and SubPath are mutually exclusive.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"mountPath\",\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"workingDir\": {\n \"description\": \"Container's working directory. If not specified, the container runtime's default will be used, which might be configured in the container image. Cannot be updated.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"listenLocal\": {\n \"description\": \"ListenLocal makes the Prometheus server listen on loopback, so that it does not bind against the Pod IP.\",\n \"type\": \"boolean\"\n },\n \"logFormat\": {\n \"description\": \"Log format for Prometheus to be configured with.\",\n \"type\": \"string\"\n },\n \"logLevel\": {\n \"description\": \"Log level for Prometheus to be configured with.\",\n \"type\": \"string\"\n },\n \"nodeSelector\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Define which Nodes the Pods are scheduled on.\",\n \"type\": \"object\"\n },\n \"overrideHonorLabels\": {\n \"description\": \"OverrideHonorLabels if set to true overrides all user configured honor_labels. If HonorLabels is set in ServiceMonitor or PodMonitor to true, this overrides honor_labels to false.\",\n \"type\": \"boolean\"\n },\n \"overrideHonorTimestamps\": {\n \"description\": \"OverrideHonorTimestamps allows to globally enforce honoring timestamps in all scrape configs.\",\n \"type\": \"boolean\"\n },\n \"paused\": {\n \"description\": \"When a Prometheus deployment is paused, no actions except for deletion will be performed on the underlying objects.\",\n \"type\": \"boolean\"\n },\n \"podMetadata\": {\n \"description\": \"PodMetadata configures Labels and Annotations which are propagated to the prometheus pods.\",\n \"properties\": {\n \"annotations\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations\",\n \"type\": \"object\"\n },\n \"labels\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels\",\n \"type\": \"object\"\n },\n \"name\": {\n \"description\": \"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"podMonitorNamespaceSelector\": {\n \"description\": \"Namespaces to be selected for PodMonitor discovery. If nil, only check own namespace.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"podMonitorSelector\": {\n \"description\": \"*Experimental* PodMonitors to be selected for target discovery. *Deprecated:* if neither this nor serviceMonitorSelector are specified, configuration is unmanaged.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"portName\": {\n \"description\": \"Port name used for the pods and governing service. This defaults to web\",\n \"type\": \"string\"\n },\n \"priorityClassName\": {\n \"description\": \"Priority class assigned to the Pods\",\n \"type\": \"string\"\n },\n \"prometheusExternalLabelName\": {\n \"description\": \"Name of Prometheus external label used to denote Prometheus instance name. Defaults to the value of `prometheus`. External label will _not_ be added when value is set to empty string (`\\\"\\\"`).\",\n \"type\": \"string\"\n },\n \"query\": {\n \"description\": \"QuerySpec defines the query command line flags when starting Prometheus.\",\n \"properties\": {\n \"lookbackDelta\": {\n \"description\": \"The delta difference allowed for retrieving metrics during expression evaluations.\",\n \"type\": \"string\"\n },\n \"maxConcurrency\": {\n \"description\": \"Number of concurrent queries that can be run at once.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"maxSamples\": {\n \"description\": \"Maximum number of samples a single query can load into memory. Note that queries will fail if they would load more samples than this into memory, so this also limits the number of samples a query can return.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"timeout\": {\n \"description\": \"Maximum time a query may take before being aborted.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"queryLogFile\": {\n \"description\": \"QueryLogFile specifies the file to which PromQL queries are logged. Note that this location must be writable, and can be persisted using an attached volume. Alternatively, the location can be set to a stdout location such as `/dev/stdout` to log querie information to the default Prometheus log stream. This is only available in versions of Prometheus \\u003e= 2.16.0. For more details, see the Prometheus docs (https://prometheus.io/docs/guides/query-log/)\",\n \"type\": \"string\"\n },\n \"remoteRead\": {\n \"description\": \"If specified, the remote_read spec. This is an experimental feature, it may change in any upcoming release in a breaking way.\",\n \"items\": {\n \"description\": \"RemoteReadSpec defines the remote_read configuration for prometheus.\",\n \"properties\": {\n \"basicAuth\": {\n \"description\": \"BasicAuth for the URL.\",\n \"properties\": {\n \"password\": {\n \"description\": \"The secret in the service monitor namespace that contains the password for authentication.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"username\": {\n \"description\": \"The secret in the service monitor namespace that contains the username for authentication.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"bearerToken\": {\n \"description\": \"bearer token for remote read.\",\n \"type\": \"string\"\n },\n \"bearerTokenFile\": {\n \"description\": \"File to read bearer token for remote read.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"The name of the remote read queue, must be unique if specified. The name is used in metrics and logging in order to differentiate read configurations. Only valid in Prometheus versions 2.15.0 and newer.\",\n \"type\": \"string\"\n },\n \"proxyUrl\": {\n \"description\": \"Optional ProxyURL\",\n \"type\": \"string\"\n },\n \"readRecent\": {\n \"description\": \"Whether reads should be made for queries for time ranges that the local storage should have complete data for.\",\n \"type\": \"boolean\"\n },\n \"remoteTimeout\": {\n \"description\": \"Timeout for requests to the remote read endpoint.\",\n \"type\": \"string\"\n },\n \"requiredMatchers\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"An optional list of equality matchers which have to be present in a selector to query the remote read endpoint.\",\n \"type\": \"object\"\n },\n \"tlsConfig\": {\n \"description\": \"TLS Config to use for remote read.\",\n \"properties\": {\n \"ca\": {\n \"description\": \"Struct containing the CA cert to use for the targets.\",\n \"properties\": {\n \"configMap\": {\n \"description\": \"ConfigMap containing data to use for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key to select.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"secret\": {\n \"description\": \"Secret containing data to use for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"caFile\": {\n \"description\": \"Path to the CA cert in the Prometheus container to use for the targets.\",\n \"type\": \"string\"\n },\n \"cert\": {\n \"description\": \"Struct containing the client cert file for the targets.\",\n \"properties\": {\n \"configMap\": {\n \"description\": \"ConfigMap containing data to use for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key to select.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"secret\": {\n \"description\": \"Secret containing data to use for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"certFile\": {\n \"description\": \"Path to the client cert file in the Prometheus container for the targets.\",\n \"type\": \"string\"\n },\n \"insecureSkipVerify\": {\n \"description\": \"Disable target certificate validation.\",\n \"type\": \"boolean\"\n },\n \"keyFile\": {\n \"description\": \"Path to the client key file in the Prometheus container for the targets.\",\n \"type\": \"string\"\n },\n \"keySecret\": {\n \"description\": \"Secret containing the client key file for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"serverName\": {\n \"description\": \"Used to verify the hostname for the targets.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"url\": {\n \"description\": \"The URL of the endpoint to send samples to.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"url\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"remoteWrite\": {\n \"description\": \"If specified, the remote_write spec. This is an experimental feature, it may change in any upcoming release in a breaking way.\",\n \"items\": {\n \"description\": \"RemoteWriteSpec defines the remote_write configuration for prometheus.\",\n \"properties\": {\n \"basicAuth\": {\n \"description\": \"BasicAuth for the URL.\",\n \"properties\": {\n \"password\": {\n \"description\": \"The secret in the service monitor namespace that contains the password for authentication.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"username\": {\n \"description\": \"The secret in the service monitor namespace that contains the username for authentication.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"bearerToken\": {\n \"description\": \"File to read bearer token for remote write.\",\n \"type\": \"string\"\n },\n \"bearerTokenFile\": {\n \"description\": \"File to read bearer token for remote write.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"The name of the remote write queue, must be unique if specified. The name is used in metrics and logging in order to differentiate queues. Only valid in Prometheus versions 2.15.0 and newer.\",\n \"type\": \"string\"\n },\n \"proxyUrl\": {\n \"description\": \"Optional ProxyURL\",\n \"type\": \"string\"\n },\n \"queueConfig\": {\n \"description\": \"QueueConfig allows tuning of the remote write queue parameters.\",\n \"properties\": {\n \"batchSendDeadline\": {\n \"description\": \"BatchSendDeadline is the maximum time a sample will wait in buffer.\",\n \"type\": \"string\"\n },\n \"capacity\": {\n \"description\": \"Capacity is the number of samples to buffer per shard before we start dropping them.\",\n \"type\": \"integer\"\n },\n \"maxBackoff\": {\n \"description\": \"MaxBackoff is the maximum retry delay.\",\n \"type\": \"string\"\n },\n \"maxRetries\": {\n \"description\": \"MaxRetries is the maximum number of times to retry a batch on recoverable errors.\",\n \"type\": \"integer\"\n },\n \"maxSamplesPerSend\": {\n \"description\": \"MaxSamplesPerSend is the maximum number of samples per send.\",\n \"type\": \"integer\"\n },\n \"maxShards\": {\n \"description\": \"MaxShards is the maximum number of shards, i.e. amount of concurrency.\",\n \"type\": \"integer\"\n },\n \"minBackoff\": {\n \"description\": \"MinBackoff is the initial retry delay. Gets doubled for every retry.\",\n \"type\": \"string\"\n },\n \"minShards\": {\n \"description\": \"MinShards is the minimum number of shards, i.e. amount of concurrency.\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n },\n \"remoteTimeout\": {\n \"description\": \"Timeout for requests to the remote write endpoint.\",\n \"type\": \"string\"\n },\n \"tlsConfig\": {\n \"description\": \"TLS Config to use for remote write.\",\n \"properties\": {\n \"ca\": {\n \"description\": \"Struct containing the CA cert to use for the targets.\",\n \"properties\": {\n \"configMap\": {\n \"description\": \"ConfigMap containing data to use for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key to select.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"secret\": {\n \"description\": \"Secret containing data to use for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"caFile\": {\n \"description\": \"Path to the CA cert in the Prometheus container to use for the targets.\",\n \"type\": \"string\"\n },\n \"cert\": {\n \"description\": \"Struct containing the client cert file for the targets.\",\n \"properties\": {\n \"configMap\": {\n \"description\": \"ConfigMap containing data to use for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key to select.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"secret\": {\n \"description\": \"Secret containing data to use for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"certFile\": {\n \"description\": \"Path to the client cert file in the Prometheus container for the targets.\",\n \"type\": \"string\"\n },\n \"insecureSkipVerify\": {\n \"description\": \"Disable target certificate validation.\",\n \"type\": \"boolean\"\n },\n \"keyFile\": {\n \"description\": \"Path to the client key file in the Prometheus container for the targets.\",\n \"type\": \"string\"\n },\n \"keySecret\": {\n \"description\": \"Secret containing the client key file for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"serverName\": {\n \"description\": \"Used to verify the hostname for the targets.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"url\": {\n \"description\": \"The URL of the endpoint to send samples to.\",\n \"type\": \"string\"\n },\n \"writeRelabelConfigs\": {\n \"description\": \"The list of remote write relabel configurations.\",\n \"items\": {\n \"description\": \"RelabelConfig allows dynamic rewriting of the label set, being applied to samples before ingestion. It defines `\\u003cmetric_relabel_configs\\u003e`-section of Prometheus configuration. More info: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#metric_relabel_configs\",\n \"properties\": {\n \"action\": {\n \"description\": \"Action to perform based on regex matching. Default is 'replace'\",\n \"type\": \"string\"\n },\n \"modulus\": {\n \"description\": \"Modulus to take of the hash of the source label values.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"regex\": {\n \"description\": \"Regular expression against which the extracted value is matched. Default is '(.*)'\",\n \"type\": \"string\"\n },\n \"replacement\": {\n \"description\": \"Replacement value against which a regex replace is performed if the regular expression matches. Regex capture groups are available. Default is '$1'\",\n \"type\": \"string\"\n },\n \"separator\": {\n \"description\": \"Separator placed between concatenated source label values. default is ';'.\",\n \"type\": \"string\"\n },\n \"sourceLabels\": {\n \"description\": \"The source labels select values from existing labels. Their content is concatenated using the configured separator and matched against the configured regular expression for the replace, keep, and drop actions.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"targetLabel\": {\n \"description\": \"Label to which the resulting value is written in a replace action. It is mandatory for replace actions. Regex capture groups are available.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"url\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"replicaExternalLabelName\": {\n \"description\": \"Name of Prometheus external label used to denote replica name. Defaults to the value of `prometheus_replica`. External label will _not_ be added when value is set to empty string (`\\\"\\\"`).\",\n \"type\": \"string\"\n },\n \"replicas\": {\n \"description\": \"Number of instances to deploy for a Prometheus deployment.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"resources\": {\n \"description\": \"Define resources requests and limits for single Pods.\",\n \"properties\": {\n \"limits\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"retention\": {\n \"description\": \"Time duration Prometheus shall retain data for. Default is '24h', and must match the regular expression `[0-9]+(ms|s|m|h|d|w|y)` (milliseconds seconds minutes hours days weeks years).\",\n \"type\": \"string\"\n },\n \"retentionSize\": {\n \"description\": \"Maximum amount of disk space used by blocks.\",\n \"type\": \"string\"\n },\n \"routePrefix\": {\n \"description\": \"The route prefix Prometheus registers HTTP handlers for. This is useful, if using ExternalURL and a proxy is rewriting HTTP routes of a request, and the actual ExternalURL is still true, but the server serves requests under a different route prefix. For example for use with `kubectl proxy`.\",\n \"type\": \"string\"\n },\n \"ruleNamespaceSelector\": {\n \"description\": \"Namespaces to be selected for PrometheusRules discovery. If unspecified, only the same namespace as the Prometheus object is in is used.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"ruleSelector\": {\n \"description\": \"A selector to select which PrometheusRules to mount for loading alerting/recording rules from. Until (excluding) Prometheus Operator v0.24.0 Prometheus Operator will migrate any legacy rule ConfigMaps to PrometheusRule custom resources selected by RuleSelector. Make sure it does not match any config maps that you do not want to be migrated.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"rules\": {\n \"description\": \"/--rules.*/ command-line arguments.\",\n \"properties\": {\n \"alert\": {\n \"description\": \"/--rules.alert.*/ command-line arguments\",\n \"properties\": {\n \"forGracePeriod\": {\n \"description\": \"Minimum duration between alert and restored 'for' state. This is maintained only for alerts with configured 'for' time greater than grace period.\",\n \"type\": \"string\"\n },\n \"forOutageTolerance\": {\n \"description\": \"Max time to tolerate prometheus outage for restoring 'for' state of alert.\",\n \"type\": \"string\"\n },\n \"resendDelay\": {\n \"description\": \"Minimum amount of time to wait before resending an alert to Alertmanager.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"scrapeInterval\": {\n \"description\": \"Interval between consecutive scrapes.\",\n \"type\": \"string\"\n },\n \"secrets\": {\n \"description\": \"Secrets is a list of Secrets in the same namespace as the Prometheus object, which shall be mounted into the Prometheus Pods. The Secrets are mounted into /etc/prometheus/secrets/\\u003csecret-name\\u003e.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"securityContext\": {\n \"description\": \"SecurityContext holds pod-level security attributes and common container settings. This defaults to the default PodSecurityContext.\",\n \"properties\": {\n \"fsGroup\": {\n \"description\": \"A special supplemental group that applies to all containers in a pod. Some volume types allow the Kubelet to change the ownership of that volume to be owned by the pod: \\n 1. The owning GID will be the FSGroup 2. The setgid bit is set (new files created in the volume will be owned by FSGroup) 3. The permission bits are OR'd with rw-rw---- \\n If unset, the Kubelet will not modify the ownership and permissions of any volume.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"fsGroupChangePolicy\": {\n \"description\": \"fsGroupChangePolicy defines behavior of changing ownership and permission of the volume before being exposed inside Pod. This field will only apply to volume types which support fsGroup based ownership(and permissions). It will have no effect on ephemeral volume types such as: secret, configmaps and emptydir. Valid values are \\\"OnRootMismatch\\\" and \\\"Always\\\". If not specified defaults to \\\"Always\\\".\",\n \"type\": \"string\"\n },\n \"runAsGroup\": {\n \"description\": \"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"runAsNonRoot\": {\n \"description\": \"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"type\": \"boolean\"\n },\n \"runAsUser\": {\n \"description\": \"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"seLinuxOptions\": {\n \"description\": \"The SELinux context to be applied to all containers. If unspecified, the container runtime will allocate a random SELinux context for each container. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\",\n \"properties\": {\n \"level\": {\n \"description\": \"Level is SELinux level label that applies to the container.\",\n \"type\": \"string\"\n },\n \"role\": {\n \"description\": \"Role is a SELinux role label that applies to the container.\",\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"Type is a SELinux type label that applies to the container.\",\n \"type\": \"string\"\n },\n \"user\": {\n \"description\": \"User is a SELinux user label that applies to the container.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"supplementalGroups\": {\n \"description\": \"A list of groups applied to the first process run in each container, in addition to the container's primary GID. If unspecified, no groups will be added to any container.\",\n \"items\": {\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"type\": \"array\"\n },\n \"sysctls\": {\n \"description\": \"Sysctls hold a list of namespaced sysctls used for the pod. Pods with unsupported sysctls (by the container runtime) might fail to launch.\",\n \"items\": {\n \"description\": \"Sysctl defines a kernel parameter to be set\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of a property to set\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"Value of a property to set\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"windowsOptions\": {\n \"description\": \"The Windows specific settings applied to all containers. If unspecified, the options within a container's SecurityContext will be used. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"properties\": {\n \"gmsaCredentialSpec\": {\n \"description\": \"GMSACredentialSpec is where the GMSA admission webhook (https://github.com/kubernetes-sigs/windows-gmsa) inlines the contents of the GMSA credential spec named by the GMSACredentialSpecName field.\",\n \"type\": \"string\"\n },\n \"gmsaCredentialSpecName\": {\n \"description\": \"GMSACredentialSpecName is the name of the GMSA credential spec to use.\",\n \"type\": \"string\"\n },\n \"runAsUserName\": {\n \"description\": \"The UserName in Windows to run the entrypoint of the container process. Defaults to the user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"serviceAccountName\": {\n \"description\": \"ServiceAccountName is the name of the ServiceAccount to use to run the Prometheus Pods.\",\n \"type\": \"string\"\n },\n \"serviceMonitorNamespaceSelector\": {\n \"description\": \"Namespaces to be selected for ServiceMonitor discovery. If nil, only check own namespace.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"serviceMonitorSelector\": {\n \"description\": \"ServiceMonitors to be selected for target discovery. *Deprecated:* if neither this nor podMonitorSelector are specified, configuration is unmanaged.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"sha\": {\n \"description\": \"SHA of Prometheus container image to be deployed. Defaults to the value of `version`. Similar to a tag, but the SHA explicitly deploys an immutable container image. Version and Tag are ignored if SHA is set.\",\n \"type\": \"string\"\n },\n \"storage\": {\n \"description\": \"Storage spec to specify how storage shall be used.\",\n \"properties\": {\n \"disableMountSubPath\": {\n \"description\": \"Deprecated: subPath usage will be disabled by default in a future release, this option will become unnecessary. DisableMountSubPath allows to remove any subPath usage in volume mounts.\",\n \"type\": \"boolean\"\n },\n \"emptyDir\": {\n \"description\": \"EmptyDirVolumeSource to be used by the Prometheus StatefulSets. If specified, used in place of any volumeClaimTemplate. More info: https://kubernetes.io/docs/concepts/storage/volumes/#emptydir\",\n \"properties\": {\n \"medium\": {\n \"description\": \"What type of storage medium should back this directory. The default is \\\"\\\" which means to use the node's default medium. Must be an empty string (default) or Memory. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir\",\n \"type\": \"string\"\n },\n \"sizeLimit\": {\n \"description\": \"Total amount of local storage required for this EmptyDir volume. The size limit is also applicable for memory medium. The maximum usage on memory medium EmptyDir would be the minimum value between the SizeLimit specified here and the sum of memory limits of all containers in a pod. The default is nil which means that the limit is undefined. More info: http://kubernetes.io/docs/user-guide/volumes#emptydir\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"volumeClaimTemplate\": {\n \"description\": \"A PVC spec to be used by the Prometheus StatefulSets.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"description\": \"EmbeddedMetadata contains metadata relevant to an EmbeddedResource.\",\n \"properties\": {\n \"annotations\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations\",\n \"type\": \"object\"\n },\n \"labels\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels\",\n \"type\": \"object\"\n },\n \"name\": {\n \"description\": \"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec defines the desired characteristics of a volume requested by a pod author. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims\",\n \"properties\": {\n \"accessModes\": {\n \"description\": \"AccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"dataSource\": {\n \"description\": \"This field can be used to specify either: * An existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot - Beta) * An existing PVC (PersistentVolumeClaim) * An existing custom resource/object that implements data population (Alpha) In order to use VolumeSnapshot object types, the appropriate feature gate must be enabled (VolumeSnapshotDataSource or AnyVolumeDataSource) If the provisioner or an external controller can support the specified data source, it will create a new volume based on the contents of the specified data source. If the specified data source is not supported, the volume will not be created and the failure will be reported as an event. In the future, we plan to support more data source types and the behavior of the provisioner may change.\",\n \"properties\": {\n \"apiGroup\": {\n \"description\": \"APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is the type of resource being referenced\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name is the name of resource being referenced\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"kind\",\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"resources\": {\n \"description\": \"Resources represents the minimum resources the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources\",\n \"properties\": {\n \"limits\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"selector\": {\n \"description\": \"A label query over volumes to consider for binding.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"storageClassName\": {\n \"description\": \"Name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1\",\n \"type\": \"string\"\n },\n \"volumeMode\": {\n \"description\": \"volumeMode defines what type of volume is required by the claim. Value of Filesystem is implied when not included in claim spec.\",\n \"type\": \"string\"\n },\n \"volumeName\": {\n \"description\": \"VolumeName is the binding reference to the PersistentVolume backing this claim.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"status\": {\n \"description\": \"Status represents the current information/status of a persistent volume claim. Read-only. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims\",\n \"properties\": {\n \"accessModes\": {\n \"description\": \"AccessModes contains the actual access modes the volume backing the PVC has. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"capacity\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Represents the actual resources of the underlying volume.\",\n \"type\": \"object\"\n },\n \"conditions\": {\n \"description\": \"Current Condition of persistent volume claim. If underlying persistent volume is being resized then the Condition will be set to 'ResizeStarted'.\",\n \"items\": {\n \"description\": \"PersistentVolumeClaimCondition contains details about state of pvc\",\n \"properties\": {\n \"lastProbeTime\": {\n \"description\": \"Last time we probed the condition.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n },\n \"lastTransitionTime\": {\n \"description\": \"Last time the condition transitioned from one status to another.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"Human-readable message indicating details about last transition.\",\n \"type\": \"string\"\n },\n \"reason\": {\n \"description\": \"Unique, this should be a short, machine understandable string that gives the reason for condition's last transition. If it reports \\\"ResizeStarted\\\" that means the underlying persistent volume is being resized.\",\n \"type\": \"string\"\n },\n \"status\": {\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"PersistentVolumeClaimConditionType is a valid value of PersistentVolumeClaimCondition.Type\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"status\",\n \"type\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"phase\": {\n \"description\": \"Phase represents the current phase of PersistentVolumeClaim.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"tag\": {\n \"description\": \"Tag of Prometheus container image to be deployed. Defaults to the value of `version`. Version is ignored if Tag is set.\",\n \"type\": \"string\"\n },\n \"thanos\": {\n \"description\": \"Thanos configuration allows configuring various aspects of a Prometheus server in a Thanos environment. \\n This section is experimental, it may change significantly without deprecation notice in any release. \\n This is experimental and may change significantly without backward compatibility in any release.\",\n \"properties\": {\n \"baseImage\": {\n \"description\": \"Thanos base image if other than default.\",\n \"type\": \"string\"\n },\n \"grpcServerTlsConfig\": {\n \"description\": \"GRPCServerTLSConfig configures the gRPC server from which Thanos Querier reads recorded rule data. Note: Currently only the CAFile, CertFile, and KeyFile fields are supported. Maps to the '--grpc-server-tls-*' CLI args.\",\n \"properties\": {\n \"ca\": {\n \"description\": \"Struct containing the CA cert to use for the targets.\",\n \"properties\": {\n \"configMap\": {\n \"description\": \"ConfigMap containing data to use for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key to select.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"secret\": {\n \"description\": \"Secret containing data to use for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"caFile\": {\n \"description\": \"Path to the CA cert in the Prometheus container to use for the targets.\",\n \"type\": \"string\"\n },\n \"cert\": {\n \"description\": \"Struct containing the client cert file for the targets.\",\n \"properties\": {\n \"configMap\": {\n \"description\": \"ConfigMap containing data to use for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key to select.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"secret\": {\n \"description\": \"Secret containing data to use for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"certFile\": {\n \"description\": \"Path to the client cert file in the Prometheus container for the targets.\",\n \"type\": \"string\"\n },\n \"insecureSkipVerify\": {\n \"description\": \"Disable target certificate validation.\",\n \"type\": \"boolean\"\n },\n \"keyFile\": {\n \"description\": \"Path to the client key file in the Prometheus container for the targets.\",\n \"type\": \"string\"\n },\n \"keySecret\": {\n \"description\": \"Secret containing the client key file for the targets.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"serverName\": {\n \"description\": \"Used to verify the hostname for the targets.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"image\": {\n \"description\": \"Image if specified has precedence over baseImage, tag and sha combinations. Specifying the version is still necessary to ensure the Prometheus Operator knows what version of Thanos is being configured.\",\n \"type\": \"string\"\n },\n \"listenLocal\": {\n \"description\": \"ListenLocal makes the Thanos sidecar listen on loopback, so that it does not bind against the Pod IP.\",\n \"type\": \"boolean\"\n },\n \"logFormat\": {\n \"description\": \"LogFormat for Thanos sidecar to be configured with.\",\n \"type\": \"string\"\n },\n \"logLevel\": {\n \"description\": \"LogLevel for Thanos sidecar to be configured with.\",\n \"type\": \"string\"\n },\n \"objectStorageConfig\": {\n \"description\": \"ObjectStorageConfig configures object storage in Thanos.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"resources\": {\n \"description\": \"Resources defines the resource requirements for the Thanos sidecar. If not provided, no requests/limits will be set\",\n \"properties\": {\n \"limits\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"sha\": {\n \"description\": \"SHA of Thanos container image to be deployed. Defaults to the value of `version`. Similar to a tag, but the SHA explicitly deploys an immutable container image. Version and Tag are ignored if SHA is set.\",\n \"type\": \"string\"\n },\n \"tag\": {\n \"description\": \"Tag of Thanos sidecar container image to be deployed. Defaults to the value of `version`. Version is ignored if Tag is set.\",\n \"type\": \"string\"\n },\n \"tracingConfig\": {\n \"description\": \"TracingConfig configures tracing in Thanos. This is an experimental feature, it may change in any upcoming release in a breaking way.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"version\": {\n \"description\": \"Version describes the version of Thanos to use.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"tolerations\": {\n \"description\": \"If specified, the pod's tolerations.\",\n \"items\": {\n \"description\": \"The pod this Toleration is attached to tolerates any taint that matches the triple \\u003ckey,value,effect\\u003e using the matching operator \\u003coperator\\u003e.\",\n \"properties\": {\n \"effect\": {\n \"description\": \"Effect indicates the taint effect to match. Empty means match all taint effects. When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.\",\n \"type\": \"string\"\n },\n \"key\": {\n \"description\": \"Key is the taint key that the toleration applies to. Empty means match all taint keys. If the key is empty, operator must be Exists; this combination means to match all values and all keys.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Operator represents a key's relationship to the value. Valid operators are Exists and Equal. Defaults to Equal. Exists is equivalent to wildcard for value, so that a pod can tolerate all taints of a particular category.\",\n \"type\": \"string\"\n },\n \"tolerationSeconds\": {\n \"description\": \"TolerationSeconds represents the period of time the toleration (which must be of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately) by the system.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"value\": {\n \"description\": \"Value is the taint value the toleration matches to. If the operator is Exists, the value should be empty, otherwise just a regular string.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"version\": {\n \"description\": \"Version of Prometheus to be deployed.\",\n \"type\": \"string\"\n },\n \"volumeMounts\": {\n \"description\": \"VolumeMounts allows configuration of additional VolumeMounts on the output StatefulSet definition. VolumeMounts specified will be appended to other VolumeMounts in the prometheus container, that are generated as a result of StorageSpec objects.\",\n \"items\": {\n \"description\": \"VolumeMount describes a mounting of a Volume within a container.\",\n \"properties\": {\n \"mountPath\": {\n \"description\": \"Path within the container at which the volume should be mounted. Must not contain ':'.\",\n \"type\": \"string\"\n },\n \"mountPropagation\": {\n \"description\": \"mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"This must match the Name of a Volume.\",\n \"type\": \"string\"\n },\n \"readOnly\": {\n \"description\": \"Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.\",\n \"type\": \"boolean\"\n },\n \"subPath\": {\n \"description\": \"Path within the volume from which the container's volume should be mounted. Defaults to \\\"\\\" (volume's root).\",\n \"type\": \"string\"\n },\n \"subPathExpr\": {\n \"description\": \"Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to \\\"\\\" (volume's root). SubPathExpr and SubPath are mutually exclusive.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"mountPath\",\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"volumes\": {\n \"description\": \"Volumes allows configuration of additional volumes on the output StatefulSet definition. Volumes specified will be appended to other volumes that are generated as a result of StorageSpec objects.\",\n \"items\": {\n \"description\": \"Volume represents a named volume in a pod that may be accessed by any container in the pod.\",\n \"properties\": {\n \"awsElasticBlockStore\": {\n \"description\": \"AWSElasticBlockStore represents an AWS Disk resource that is attached to a kubelet's host machine and then exposed to the pod. More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore\",\n \"properties\": {\n \"fsType\": {\n \"description\": \"Filesystem type of the volume that you want to mount. Tip: Ensure that the filesystem type is supported by the host operating system. Examples: \\\"ext4\\\", \\\"xfs\\\", \\\"ntfs\\\". Implicitly inferred to be \\\"ext4\\\" if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore TODO: how do we prevent errors in the filesystem from compromising the machine\",\n \"type\": \"string\"\n },\n \"partition\": {\n \"description\": \"The partition in the volume that you want to mount. If omitted, the default is to mount by volume name. Examples: For volume /dev/sda1, you specify the partition as \\\"1\\\". Similarly, the volume partition for /dev/sda is \\\"0\\\" (or you can leave the property empty).\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"readOnly\": {\n \"description\": \"Specify \\\"true\\\" to force and set the ReadOnly property in VolumeMounts to \\\"true\\\". If omitted, the default is \\\"false\\\". More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore\",\n \"type\": \"boolean\"\n },\n \"volumeID\": {\n \"description\": \"Unique ID of the persistent disk resource in AWS (Amazon EBS volume). More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"volumeID\"\n ],\n \"type\": \"object\"\n },\n \"azureDisk\": {\n \"description\": \"AzureDisk represents an Azure Data Disk mount on the host and bind mount to the pod.\",\n \"properties\": {\n \"cachingMode\": {\n \"description\": \"Host Caching mode: None, Read Only, Read Write.\",\n \"type\": \"string\"\n },\n \"diskName\": {\n \"description\": \"The Name of the data disk in the blob storage\",\n \"type\": \"string\"\n },\n \"diskURI\": {\n \"description\": \"The URI the data disk in the blob storage\",\n \"type\": \"string\"\n },\n \"fsType\": {\n \"description\": \"Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. \\\"ext4\\\", \\\"xfs\\\", \\\"ntfs\\\". Implicitly inferred to be \\\"ext4\\\" if unspecified.\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Expected values Shared: multiple blob disks per storage account Dedicated: single blob disk per storage account Managed: azure managed data disk (only in managed availability set). defaults to shared\",\n \"type\": \"string\"\n },\n \"readOnly\": {\n \"description\": \"Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts.\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"diskName\",\n \"diskURI\"\n ],\n \"type\": \"object\"\n },\n \"azureFile\": {\n \"description\": \"AzureFile represents an Azure File Service mount on the host and bind mount to the pod.\",\n \"properties\": {\n \"readOnly\": {\n \"description\": \"Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts.\",\n \"type\": \"boolean\"\n },\n \"secretName\": {\n \"description\": \"the name of secret that contains Azure Storage Account Name and Key\",\n \"type\": \"string\"\n },\n \"shareName\": {\n \"description\": \"Share Name\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"secretName\",\n \"shareName\"\n ],\n \"type\": \"object\"\n },\n \"cephfs\": {\n \"description\": \"CephFS represents a Ceph FS mount on the host that shares a pod's lifetime\",\n \"properties\": {\n \"monitors\": {\n \"description\": \"Required: Monitors is a collection of Ceph monitors More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"path\": {\n \"description\": \"Optional: Used as the mounted root, rather than the full Ceph tree, default is /\",\n \"type\": \"string\"\n },\n \"readOnly\": {\n \"description\": \"Optional: Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts. More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it\",\n \"type\": \"boolean\"\n },\n \"secretFile\": {\n \"description\": \"Optional: SecretFile is the path to key ring for User, default is /etc/ceph/user.secret More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it\",\n \"type\": \"string\"\n },\n \"secretRef\": {\n \"description\": \"Optional: SecretRef is reference to the authentication secret for User, default is empty. More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"user\": {\n \"description\": \"Optional: User is the rados user name, default is admin More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"monitors\"\n ],\n \"type\": \"object\"\n },\n \"cinder\": {\n \"description\": \"Cinder represents a cinder volume attached and mounted on kubelets host machine. More info: https://examples.k8s.io/mysql-cinder-pd/README.md\",\n \"properties\": {\n \"fsType\": {\n \"description\": \"Filesystem type to mount. Must be a filesystem type supported by the host operating system. Examples: \\\"ext4\\\", \\\"xfs\\\", \\\"ntfs\\\". Implicitly inferred to be \\\"ext4\\\" if unspecified. More info: https://examples.k8s.io/mysql-cinder-pd/README.md\",\n \"type\": \"string\"\n },\n \"readOnly\": {\n \"description\": \"Optional: Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts. More info: https://examples.k8s.io/mysql-cinder-pd/README.md\",\n \"type\": \"boolean\"\n },\n \"secretRef\": {\n \"description\": \"Optional: points to a secret object containing parameters used to connect to OpenStack.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"volumeID\": {\n \"description\": \"volume id used to identify the volume in cinder. More info: https://examples.k8s.io/mysql-cinder-pd/README.md\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"volumeID\"\n ],\n \"type\": \"object\"\n },\n \"configMap\": {\n \"description\": \"ConfigMap represents a configMap that should populate this volume\",\n \"properties\": {\n \"defaultMode\": {\n \"description\": \"Optional: mode bits to use on created files by default. Must be a value between 0 and 0777. Defaults to 0644. Directories within the path are not affected by this setting. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"items\": {\n \"description\": \"If unspecified, each key-value pair in the Data field of the referenced ConfigMap will be projected into the volume as a file whose name is the key and content is the value. If specified, the listed keys will be projected into the specified paths, and unlisted keys will not be present. If a key is specified which is not present in the ConfigMap, the volume setup will error unless it is marked optional. Paths must be relative and may not contain the '..' path or start with '..'.\",\n \"items\": {\n \"description\": \"Maps a string key to a path within a volume.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key to project.\",\n \"type\": \"string\"\n },\n \"mode\": {\n \"description\": \"Optional: mode bits to use on this file, must be a value between 0 and 0777. If not specified, the volume defaultMode will be used. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"path\": {\n \"description\": \"The relative path of the file to map the key to. May not be an absolute path. May not contain the path element '..'. May not start with the string '..'.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"key\",\n \"path\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap or its keys must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"type\": \"object\"\n },\n \"csi\": {\n \"description\": \"CSI (Container Storage Interface) represents storage that is handled by an external CSI driver (Alpha feature).\",\n \"properties\": {\n \"driver\": {\n \"description\": \"Driver is the name of the CSI driver that handles this volume. Consult with your admin for the correct name as registered in the cluster.\",\n \"type\": \"string\"\n },\n \"fsType\": {\n \"description\": \"Filesystem type to mount. Ex. \\\"ext4\\\", \\\"xfs\\\", \\\"ntfs\\\". If not provided, the empty value is passed to the associated CSI driver which will determine the default filesystem to apply.\",\n \"type\": \"string\"\n },\n \"nodePublishSecretRef\": {\n \"description\": \"NodePublishSecretRef is a reference to the secret object containing sensitive information to pass to the CSI driver to complete the CSI NodePublishVolume and NodeUnpublishVolume calls. This field is optional, and may be empty if no secret is required. If the secret object contains more than one secret, all secret references are passed.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"readOnly\": {\n \"description\": \"Specifies a read-only configuration for the volume. Defaults to false (read/write).\",\n \"type\": \"boolean\"\n },\n \"volumeAttributes\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"VolumeAttributes stores driver-specific properties that are passed to the CSI driver. Consult your driver's documentation for supported values.\",\n \"type\": \"object\"\n }\n },\n \"required\": [\n \"driver\"\n ],\n \"type\": \"object\"\n },\n \"downwardAPI\": {\n \"description\": \"DownwardAPI represents downward API about the pod that should populate this volume\",\n \"properties\": {\n \"defaultMode\": {\n \"description\": \"Optional: mode bits to use on created files by default. Must be a value between 0 and 0777. Defaults to 0644. Directories within the path are not affected by this setting. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"items\": {\n \"description\": \"Items is a list of downward API volume file\",\n \"items\": {\n \"description\": \"DownwardAPIVolumeFile represents information to create the file containing the pod field\",\n \"properties\": {\n \"fieldRef\": {\n \"description\": \"Required: Selects a field of the pod: only annotations, labels, name and namespace are supported.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"Version of the schema the FieldPath is written in terms of, defaults to \\\"v1\\\".\",\n \"type\": \"string\"\n },\n \"fieldPath\": {\n \"description\": \"Path of the field to select in the specified API version.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"fieldPath\"\n ],\n \"type\": \"object\"\n },\n \"mode\": {\n \"description\": \"Optional: mode bits to use on this file, must be a value between 0 and 0777. If not specified, the volume defaultMode will be used. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"path\": {\n \"description\": \"Required: Path is the relative path name of the file to be created. Must not be absolute or contain the '..' path. Must be utf-8 encoded. The first item of the relative path must not start with '..'\",\n \"type\": \"string\"\n },\n \"resourceFieldRef\": {\n \"description\": \"Selects a resource of the container: only resources limits and requests (limits.cpu, limits.memory, requests.cpu and requests.memory) are currently supported.\",\n \"properties\": {\n \"containerName\": {\n \"description\": \"Container name: required for volumes, optional for env vars\",\n \"type\": \"string\"\n },\n \"divisor\": {\n \"description\": \"Specifies the output format of the exposed resources, defaults to \\\"1\\\"\",\n \"type\": \"string\"\n },\n \"resource\": {\n \"description\": \"Required: resource to select\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"resource\"\n ],\n \"type\": \"object\"\n }\n },\n \"required\": [\n \"path\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"emptyDir\": {\n \"description\": \"EmptyDir represents a temporary directory that shares a pod's lifetime. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir\",\n \"properties\": {\n \"medium\": {\n \"description\": \"What type of storage medium should back this directory. The default is \\\"\\\" which means to use the node's default medium. Must be an empty string (default) or Memory. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir\",\n \"type\": \"string\"\n },\n \"sizeLimit\": {\n \"description\": \"Total amount of local storage required for this EmptyDir volume. The size limit is also applicable for memory medium. The maximum usage on memory medium EmptyDir would be the minimum value between the SizeLimit specified here and the sum of memory limits of all containers in a pod. The default is nil which means that the limit is undefined. More info: http://kubernetes.io/docs/user-guide/volumes#emptydir\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"fc\": {\n \"description\": \"FC represents a Fibre Channel resource that is attached to a kubelet's host machine and then exposed to the pod.\",\n \"properties\": {\n \"fsType\": {\n \"description\": \"Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. \\\"ext4\\\", \\\"xfs\\\", \\\"ntfs\\\". Implicitly inferred to be \\\"ext4\\\" if unspecified. TODO: how do we prevent errors in the filesystem from compromising the machine\",\n \"type\": \"string\"\n },\n \"lun\": {\n \"description\": \"Optional: FC target lun number\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"readOnly\": {\n \"description\": \"Optional: Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts.\",\n \"type\": \"boolean\"\n },\n \"targetWWNs\": {\n \"description\": \"Optional: FC target worldwide names (WWNs)\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"wwids\": {\n \"description\": \"Optional: FC volume world wide identifiers (wwids) Either wwids or combination of targetWWNs and lun must be set, but not both simultaneously.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"flexVolume\": {\n \"description\": \"FlexVolume represents a generic volume resource that is provisioned/attached using an exec based plugin.\",\n \"properties\": {\n \"driver\": {\n \"description\": \"Driver is the name of the driver to use for this volume.\",\n \"type\": \"string\"\n },\n \"fsType\": {\n \"description\": \"Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. \\\"ext4\\\", \\\"xfs\\\", \\\"ntfs\\\". The default filesystem depends on FlexVolume script.\",\n \"type\": \"string\"\n },\n \"options\": {\n \"additionalProperties\": {\n \"type\": \"string\"\n },\n \"description\": \"Optional: Extra command options if any.\",\n \"type\": \"object\"\n },\n \"readOnly\": {\n \"description\": \"Optional: Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts.\",\n \"type\": \"boolean\"\n },\n \"secretRef\": {\n \"description\": \"Optional: SecretRef is reference to the secret object containing sensitive information to pass to the plugin scripts. This may be empty if no secret object is specified. If the secret object contains more than one secret, all secrets are passed to the plugin scripts.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"required\": [\n \"driver\"\n ],\n \"type\": \"object\"\n },\n \"flocker\": {\n \"description\": \"Flocker represents a Flocker volume attached to a kubelet's host machine. This depends on the Flocker control service being running\",\n \"properties\": {\n \"datasetName\": {\n \"description\": \"Name of the dataset stored as metadata -\\u003e name on the dataset for Flocker should be considered as deprecated\",\n \"type\": \"string\"\n },\n \"datasetUUID\": {\n \"description\": \"UUID of the dataset. This is unique identifier of a Flocker dataset\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"gcePersistentDisk\": {\n \"description\": \"GCEPersistentDisk represents a GCE Disk resource that is attached to a kubelet's host machine and then exposed to the pod. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk\",\n \"properties\": {\n \"fsType\": {\n \"description\": \"Filesystem type of the volume that you want to mount. Tip: Ensure that the filesystem type is supported by the host operating system. Examples: \\\"ext4\\\", \\\"xfs\\\", \\\"ntfs\\\". Implicitly inferred to be \\\"ext4\\\" if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk TODO: how do we prevent errors in the filesystem from compromising the machine\",\n \"type\": \"string\"\n },\n \"partition\": {\n \"description\": \"The partition in the volume that you want to mount. If omitted, the default is to mount by volume name. Examples: For volume /dev/sda1, you specify the partition as \\\"1\\\". Similarly, the volume partition for /dev/sda is \\\"0\\\" (or you can leave the property empty). More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"pdName\": {\n \"description\": \"Unique name of the PD resource in GCE. Used to identify the disk in GCE. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk\",\n \"type\": \"string\"\n },\n \"readOnly\": {\n \"description\": \"ReadOnly here will force the ReadOnly setting in VolumeMounts. Defaults to false. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"pdName\"\n ],\n \"type\": \"object\"\n },\n \"gitRepo\": {\n \"description\": \"GitRepo represents a git repository at a particular revision. DEPRECATED: GitRepo is deprecated. To provision a container with a git repo, mount an EmptyDir into an InitContainer that clones the repo using git, then mount the EmptyDir into the Pod's container.\",\n \"properties\": {\n \"directory\": {\n \"description\": \"Target directory name. Must not contain or start with '..'. If '.' is supplied, the volume directory will be the git repository. Otherwise, if specified, the volume will contain the git repository in the subdirectory with the given name.\",\n \"type\": \"string\"\n },\n \"repository\": {\n \"description\": \"Repository URL\",\n \"type\": \"string\"\n },\n \"revision\": {\n \"description\": \"Commit hash for the specified revision.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"repository\"\n ],\n \"type\": \"object\"\n },\n \"glusterfs\": {\n \"description\": \"Glusterfs represents a Glusterfs mount on the host that shares a pod's lifetime. More info: https://examples.k8s.io/volumes/glusterfs/README.md\",\n \"properties\": {\n \"endpoints\": {\n \"description\": \"EndpointsName is the endpoint name that details Glusterfs topology. More info: https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod\",\n \"type\": \"string\"\n },\n \"path\": {\n \"description\": \"Path is the Glusterfs volume path. More info: https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod\",\n \"type\": \"string\"\n },\n \"readOnly\": {\n \"description\": \"ReadOnly here will force the Glusterfs volume to be mounted with read-only permissions. Defaults to false. More info: https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"endpoints\",\n \"path\"\n ],\n \"type\": \"object\"\n },\n \"hostPath\": {\n \"description\": \"HostPath represents a pre-existing file or directory on the host machine that is directly exposed to the container. This is generally used for system agents or other privileged things that are allowed to see the host machine. Most containers will NOT need this. More info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath --- TODO(jonesdl) We need to restrict who can use host directory mounts and who can/can not mount host directories as read/write.\",\n \"properties\": {\n \"path\": {\n \"description\": \"Path of the directory on the host. If the path is a symlink, it will follow the link to the real path. More info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath\",\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"Type for HostPath Volume Defaults to \\\"\\\" More info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"path\"\n ],\n \"type\": \"object\"\n },\n \"iscsi\": {\n \"description\": \"ISCSI represents an ISCSI Disk resource that is attached to a kubelet's host machine and then exposed to the pod. More info: https://examples.k8s.io/volumes/iscsi/README.md\",\n \"properties\": {\n \"chapAuthDiscovery\": {\n \"description\": \"whether support iSCSI Discovery CHAP authentication\",\n \"type\": \"boolean\"\n },\n \"chapAuthSession\": {\n \"description\": \"whether support iSCSI Session CHAP authentication\",\n \"type\": \"boolean\"\n },\n \"fsType\": {\n \"description\": \"Filesystem type of the volume that you want to mount. Tip: Ensure that the filesystem type is supported by the host operating system. Examples: \\\"ext4\\\", \\\"xfs\\\", \\\"ntfs\\\". Implicitly inferred to be \\\"ext4\\\" if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#iscsi TODO: how do we prevent errors in the filesystem from compromising the machine\",\n \"type\": \"string\"\n },\n \"initiatorName\": {\n \"description\": \"Custom iSCSI Initiator Name. If initiatorName is specified with iscsiInterface simultaneously, new iSCSI interface \\u003ctarget portal\\u003e:\\u003cvolume name\\u003e will be created for the connection.\",\n \"type\": \"string\"\n },\n \"iqn\": {\n \"description\": \"Target iSCSI Qualified Name.\",\n \"type\": \"string\"\n },\n \"iscsiInterface\": {\n \"description\": \"iSCSI Interface Name that uses an iSCSI transport. Defaults to 'default' (tcp).\",\n \"type\": \"string\"\n },\n \"lun\": {\n \"description\": \"iSCSI Target Lun number.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"portals\": {\n \"description\": \"iSCSI Target Portal List. The portal is either an IP or ip_addr:port if the port is other than default (typically TCP ports 860 and 3260).\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"readOnly\": {\n \"description\": \"ReadOnly here will force the ReadOnly setting in VolumeMounts. Defaults to false.\",\n \"type\": \"boolean\"\n },\n \"secretRef\": {\n \"description\": \"CHAP Secret for iSCSI target and initiator authentication\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"targetPortal\": {\n \"description\": \"iSCSI Target Portal. The Portal is either an IP or ip_addr:port if the port is other than default (typically TCP ports 860 and 3260).\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"iqn\",\n \"lun\",\n \"targetPortal\"\n ],\n \"type\": \"object\"\n },\n \"name\": {\n \"description\": \"Volume's name. Must be a DNS_LABEL and unique within the pod. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"nfs\": {\n \"description\": \"NFS represents an NFS mount on the host that shares a pod's lifetime More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs\",\n \"properties\": {\n \"path\": {\n \"description\": \"Path that is exported by the NFS server. More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs\",\n \"type\": \"string\"\n },\n \"readOnly\": {\n \"description\": \"ReadOnly here will force the NFS export to be mounted with read-only permissions. Defaults to false. More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs\",\n \"type\": \"boolean\"\n },\n \"server\": {\n \"description\": \"Server is the hostname or IP address of the NFS server. More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"path\",\n \"server\"\n ],\n \"type\": \"object\"\n },\n \"persistentVolumeClaim\": {\n \"description\": \"PersistentVolumeClaimVolumeSource represents a reference to a PersistentVolumeClaim in the same namespace. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims\",\n \"properties\": {\n \"claimName\": {\n \"description\": \"ClaimName is the name of a PersistentVolumeClaim in the same namespace as the pod using this volume. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims\",\n \"type\": \"string\"\n },\n \"readOnly\": {\n \"description\": \"Will force the ReadOnly setting in VolumeMounts. Default false.\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"claimName\"\n ],\n \"type\": \"object\"\n },\n \"photonPersistentDisk\": {\n \"description\": \"PhotonPersistentDisk represents a PhotonController persistent disk attached and mounted on kubelets host machine\",\n \"properties\": {\n \"fsType\": {\n \"description\": \"Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. \\\"ext4\\\", \\\"xfs\\\", \\\"ntfs\\\". Implicitly inferred to be \\\"ext4\\\" if unspecified.\",\n \"type\": \"string\"\n },\n \"pdID\": {\n \"description\": \"ID that identifies Photon Controller persistent disk\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"pdID\"\n ],\n \"type\": \"object\"\n },\n \"portworxVolume\": {\n \"description\": \"PortworxVolume represents a portworx volume attached and mounted on kubelets host machine\",\n \"properties\": {\n \"fsType\": {\n \"description\": \"FSType represents the filesystem type to mount Must be a filesystem type supported by the host operating system. Ex. \\\"ext4\\\", \\\"xfs\\\". Implicitly inferred to be \\\"ext4\\\" if unspecified.\",\n \"type\": \"string\"\n },\n \"readOnly\": {\n \"description\": \"Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts.\",\n \"type\": \"boolean\"\n },\n \"volumeID\": {\n \"description\": \"VolumeID uniquely identifies a Portworx volume\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"volumeID\"\n ],\n \"type\": \"object\"\n },\n \"projected\": {\n \"description\": \"Items for all in one resources secrets, configmaps, and downward API\",\n \"properties\": {\n \"defaultMode\": {\n \"description\": \"Mode bits to use on created files by default. Must be a value between 0 and 0777. Directories within the path are not affected by this setting. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"sources\": {\n \"description\": \"list of volume projections\",\n \"items\": {\n \"description\": \"Projection that may be projected along with other supported volume types\",\n \"properties\": {\n \"configMap\": {\n \"description\": \"information about the configMap data to project\",\n \"properties\": {\n \"items\": {\n \"description\": \"If unspecified, each key-value pair in the Data field of the referenced ConfigMap will be projected into the volume as a file whose name is the key and content is the value. If specified, the listed keys will be projected into the specified paths, and unlisted keys will not be present. If a key is specified which is not present in the ConfigMap, the volume setup will error unless it is marked optional. Paths must be relative and may not contain the '..' path or start with '..'.\",\n \"items\": {\n \"description\": \"Maps a string key to a path within a volume.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key to project.\",\n \"type\": \"string\"\n },\n \"mode\": {\n \"description\": \"Optional: mode bits to use on this file, must be a value between 0 and 0777. If not specified, the volume defaultMode will be used. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"path\": {\n \"description\": \"The relative path of the file to map the key to. May not be an absolute path. May not contain the path element '..'. May not start with the string '..'.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"key\",\n \"path\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap or its keys must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"type\": \"object\"\n },\n \"downwardAPI\": {\n \"description\": \"information about the downwardAPI data to project\",\n \"properties\": {\n \"items\": {\n \"description\": \"Items is a list of DownwardAPIVolume file\",\n \"items\": {\n \"description\": \"DownwardAPIVolumeFile represents information to create the file containing the pod field\",\n \"properties\": {\n \"fieldRef\": {\n \"description\": \"Required: Selects a field of the pod: only annotations, labels, name and namespace are supported.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"Version of the schema the FieldPath is written in terms of, defaults to \\\"v1\\\".\",\n \"type\": \"string\"\n },\n \"fieldPath\": {\n \"description\": \"Path of the field to select in the specified API version.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"fieldPath\"\n ],\n \"type\": \"object\"\n },\n \"mode\": {\n \"description\": \"Optional: mode bits to use on this file, must be a value between 0 and 0777. If not specified, the volume defaultMode will be used. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"path\": {\n \"description\": \"Required: Path is the relative path name of the file to be created. Must not be absolute or contain the '..' path. Must be utf-8 encoded. The first item of the relative path must not start with '..'\",\n \"type\": \"string\"\n },\n \"resourceFieldRef\": {\n \"description\": \"Selects a resource of the container: only resources limits and requests (limits.cpu, limits.memory, requests.cpu and requests.memory) are currently supported.\",\n \"properties\": {\n \"containerName\": {\n \"description\": \"Container name: required for volumes, optional for env vars\",\n \"type\": \"string\"\n },\n \"divisor\": {\n \"description\": \"Specifies the output format of the exposed resources, defaults to \\\"1\\\"\",\n \"type\": \"string\"\n },\n \"resource\": {\n \"description\": \"Required: resource to select\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"resource\"\n ],\n \"type\": \"object\"\n }\n },\n \"required\": [\n \"path\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"secret\": {\n \"description\": \"information about the secret data to project\",\n \"properties\": {\n \"items\": {\n \"description\": \"If unspecified, each key-value pair in the Data field of the referenced Secret will be projected into the volume as a file whose name is the key and content is the value. If specified, the listed keys will be projected into the specified paths, and unlisted keys will not be present. If a key is specified which is not present in the Secret, the volume setup will error unless it is marked optional. Paths must be relative and may not contain the '..' path or start with '..'.\",\n \"items\": {\n \"description\": \"Maps a string key to a path within a volume.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key to project.\",\n \"type\": \"string\"\n },\n \"mode\": {\n \"description\": \"Optional: mode bits to use on this file, must be a value between 0 and 0777. If not specified, the volume defaultMode will be used. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"path\": {\n \"description\": \"The relative path of the file to map the key to. May not be an absolute path. May not contain the path element '..'. May not start with the string '..'.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"key\",\n \"path\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"type\": \"object\"\n },\n \"serviceAccountToken\": {\n \"description\": \"information about the serviceAccountToken data to project\",\n \"properties\": {\n \"audience\": {\n \"description\": \"Audience is the intended audience of the token. A recipient of a token must identify itself with an identifier specified in the audience of the token, and otherwise should reject the token. The audience defaults to the identifier of the apiserver.\",\n \"type\": \"string\"\n },\n \"expirationSeconds\": {\n \"description\": \"ExpirationSeconds is the requested duration of validity of the service account token. As the token approaches expiration, the kubelet volume plugin will proactively rotate the service account token. The kubelet will start trying to rotate the token if the token is older than 80 percent of its time to live or if the token is older than 24 hours.Defaults to 1 hour and must be at least 10 minutes.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"path\": {\n \"description\": \"Path is the path relative to the mount point of the file to project the token into.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"path\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"sources\"\n ],\n \"type\": \"object\"\n },\n \"quobyte\": {\n \"description\": \"Quobyte represents a Quobyte mount on the host that shares a pod's lifetime\",\n \"properties\": {\n \"group\": {\n \"description\": \"Group to map volume access to Default is no group\",\n \"type\": \"string\"\n },\n \"readOnly\": {\n \"description\": \"ReadOnly here will force the Quobyte volume to be mounted with read-only permissions. Defaults to false.\",\n \"type\": \"boolean\"\n },\n \"registry\": {\n \"description\": \"Registry represents a single or multiple Quobyte Registry services specified as a string as host:port pair (multiple entries are separated with commas) which acts as the central registry for volumes\",\n \"type\": \"string\"\n },\n \"tenant\": {\n \"description\": \"Tenant owning the given Quobyte volume in the Backend Used with dynamically provisioned Quobyte volumes, value is set by the plugin\",\n \"type\": \"string\"\n },\n \"user\": {\n \"description\": \"User to map volume access to Defaults to serivceaccount user\",\n \"type\": \"string\"\n },\n \"volume\": {\n \"description\": \"Volume is a string that references an already created Quobyte volume by name.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"registry\",\n \"volume\"\n ],\n \"type\": \"object\"\n },\n \"rbd\": {\n \"description\": \"RBD represents a Rados Block Device mount on the host that shares a pod's lifetime. More info: https://examples.k8s.io/volumes/rbd/README.md\",\n \"properties\": {\n \"fsType\": {\n \"description\": \"Filesystem type of the volume that you want to mount. Tip: Ensure that the filesystem type is supported by the host operating system. Examples: \\\"ext4\\\", \\\"xfs\\\", \\\"ntfs\\\". Implicitly inferred to be \\\"ext4\\\" if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#rbd TODO: how do we prevent errors in the filesystem from compromising the machine\",\n \"type\": \"string\"\n },\n \"image\": {\n \"description\": \"The rados image name. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it\",\n \"type\": \"string\"\n },\n \"keyring\": {\n \"description\": \"Keyring is the path to key ring for RBDUser. Default is /etc/ceph/keyring. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it\",\n \"type\": \"string\"\n },\n \"monitors\": {\n \"description\": \"A collection of Ceph monitors. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"pool\": {\n \"description\": \"The rados pool name. Default is rbd. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it\",\n \"type\": \"string\"\n },\n \"readOnly\": {\n \"description\": \"ReadOnly here will force the ReadOnly setting in VolumeMounts. Defaults to false. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it\",\n \"type\": \"boolean\"\n },\n \"secretRef\": {\n \"description\": \"SecretRef is name of the authentication secret for RBDUser. If provided overrides keyring. Default is nil. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"user\": {\n \"description\": \"The rados user name. Default is admin. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"image\",\n \"monitors\"\n ],\n \"type\": \"object\"\n },\n \"scaleIO\": {\n \"description\": \"ScaleIO represents a ScaleIO persistent volume attached and mounted on Kubernetes nodes.\",\n \"properties\": {\n \"fsType\": {\n \"description\": \"Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. \\\"ext4\\\", \\\"xfs\\\", \\\"ntfs\\\". Default is \\\"xfs\\\".\",\n \"type\": \"string\"\n },\n \"gateway\": {\n \"description\": \"The host address of the ScaleIO API Gateway.\",\n \"type\": \"string\"\n },\n \"protectionDomain\": {\n \"description\": \"The name of the ScaleIO Protection Domain for the configured storage.\",\n \"type\": \"string\"\n },\n \"readOnly\": {\n \"description\": \"Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts.\",\n \"type\": \"boolean\"\n },\n \"secretRef\": {\n \"description\": \"SecretRef references to the secret for ScaleIO user and other sensitive information. If this is not provided, Login operation will fail.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"sslEnabled\": {\n \"description\": \"Flag to enable/disable SSL communication with Gateway, default false\",\n \"type\": \"boolean\"\n },\n \"storageMode\": {\n \"description\": \"Indicates whether the storage for a volume should be ThickProvisioned or ThinProvisioned. Default is ThinProvisioned.\",\n \"type\": \"string\"\n },\n \"storagePool\": {\n \"description\": \"The ScaleIO Storage Pool associated with the protection domain.\",\n \"type\": \"string\"\n },\n \"system\": {\n \"description\": \"The name of the storage system as configured in ScaleIO.\",\n \"type\": \"string\"\n },\n \"volumeName\": {\n \"description\": \"The name of a volume already created in the ScaleIO system that is associated with this volume source.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"gateway\",\n \"secretRef\",\n \"system\"\n ],\n \"type\": \"object\"\n },\n \"secret\": {\n \"description\": \"Secret represents a secret that should populate this volume. More info: https://kubernetes.io/docs/concepts/storage/volumes#secret\",\n \"properties\": {\n \"defaultMode\": {\n \"description\": \"Optional: mode bits to use on created files by default. Must be a value between 0 and 0777. Defaults to 0644. Directories within the path are not affected by this setting. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"items\": {\n \"description\": \"If unspecified, each key-value pair in the Data field of the referenced Secret will be projected into the volume as a file whose name is the key and content is the value. If specified, the listed keys will be projected into the specified paths, and unlisted keys will not be present. If a key is specified which is not present in the Secret, the volume setup will error unless it is marked optional. Paths must be relative and may not contain the '..' path or start with '..'.\",\n \"items\": {\n \"description\": \"Maps a string key to a path within a volume.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key to project.\",\n \"type\": \"string\"\n },\n \"mode\": {\n \"description\": \"Optional: mode bits to use on this file, must be a value between 0 and 0777. If not specified, the volume defaultMode will be used. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"path\": {\n \"description\": \"The relative path of the file to map the key to. May not be an absolute path. May not contain the path element '..'. May not start with the string '..'.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"key\",\n \"path\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or its keys must be defined\",\n \"type\": \"boolean\"\n },\n \"secretName\": {\n \"description\": \"Name of the secret in the pod's namespace to use. More info: https://kubernetes.io/docs/concepts/storage/volumes#secret\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"storageos\": {\n \"description\": \"StorageOS represents a StorageOS volume attached and mounted on Kubernetes nodes.\",\n \"properties\": {\n \"fsType\": {\n \"description\": \"Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. \\\"ext4\\\", \\\"xfs\\\", \\\"ntfs\\\". Implicitly inferred to be \\\"ext4\\\" if unspecified.\",\n \"type\": \"string\"\n },\n \"readOnly\": {\n \"description\": \"Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts.\",\n \"type\": \"boolean\"\n },\n \"secretRef\": {\n \"description\": \"SecretRef specifies the secret to use for obtaining the StorageOS API credentials. If not specified, default values will be attempted.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names TODO: Add other useful fields. apiVersion, kind, uid?\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"volumeName\": {\n \"description\": \"VolumeName is the human-readable name of the StorageOS volume. Volume names are only unique within a namespace.\",\n \"type\": \"string\"\n },\n \"volumeNamespace\": {\n \"description\": \"VolumeNamespace specifies the scope of the volume within StorageOS. If no namespace is specified then the Pod's namespace will be used. This allows the Kubernetes name scoping to be mirrored within StorageOS for tighter integration. Set VolumeName to any name to override the default behaviour. Set to \\\"default\\\" if you are not using namespaces within StorageOS. Namespaces that do not pre-exist within StorageOS will be created.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"vsphereVolume\": {\n \"description\": \"VsphereVolume represents a vSphere volume attached and mounted on kubelets host machine\",\n \"properties\": {\n \"fsType\": {\n \"description\": \"Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. \\\"ext4\\\", \\\"xfs\\\", \\\"ntfs\\\". Implicitly inferred to be \\\"ext4\\\" if unspecified.\",\n \"type\": \"string\"\n },\n \"storagePolicyID\": {\n \"description\": \"Storage Policy Based Management (SPBM) profile ID associated with the StoragePolicyName.\",\n \"type\": \"string\"\n },\n \"storagePolicyName\": {\n \"description\": \"Storage Policy Based Management (SPBM) profile name.\",\n \"type\": \"string\"\n },\n \"volumePath\": {\n \"description\": \"Path that identifies vSphere volume vmdk\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"volumePath\"\n ],\n \"type\": \"object\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"walCompression\": {\n \"description\": \"Enable compression of the write-ahead log using Snappy. This flag is only available in versions of Prometheus \\u003e= 2.11.0.\",\n \"type\": \"boolean\"\n }\n },\n \"type\": \"object\"\n },\n \"status\": {\n \"description\": \"Most recent observed status of the Prometheus cluster. Read-only. Not included when requesting from the apiserver, only from the Prometheus Operator API itself. More info: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#spec-and-status\",\n \"properties\": {\n \"availableReplicas\": {\n \"description\": \"Total number of available pods (ready for at least minReadySeconds) targeted by this Prometheus deployment.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"paused\": {\n \"description\": \"Represents whether any actions on the underlying managed objects are being performed. Only delete actions will be performed.\",\n \"type\": \"boolean\"\n },\n \"replicas\": {\n \"description\": \"Total number of non-terminated pods targeted by this Prometheus deployment (their labels match the selector).\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"unavailableReplicas\": {\n \"description\": \"Total number of unavailable pods targeted by this Prometheus deployment.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"updatedReplicas\": {\n \"description\": \"Total number of non-terminated pods targeted by this Prometheus deployment that have the desired version spec.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"availableReplicas\",\n \"paused\",\n \"replicas\",\n \"unavailableReplicas\",\n \"updatedReplicas\"\n ],\n \"type\": \"object\"\n }\n },\n \"required\": [\n \"spec\"\n ],\n \"type\": \"object\"\n }\n },\n \"served\": true,\n \"storage\": true,\n \"subresources\": {}\n }\n ]\n },\n \"status\": {\n \"acceptedNames\": {\n \"kind\": \"Prometheus\",\n \"listKind\": \"PrometheusList\",\n \"plural\": \"prometheuses\",\n \"singular\": \"prometheus\"\n },\n \"conditions\": [\n {\n \"lastTransitionTime\": \"2020-05-04T19:47:40Z\",\n \"message\": \"no conflicts found\",\n \"reason\": \"NoConflicts\",\n \"status\": \"True\",\n \"type\": \"NamesAccepted\"\n },\n {\n \"lastTransitionTime\": \"2020-05-04T19:47:40Z\",\n \"message\": \"the initial names have been accepted\",\n \"reason\": \"InitialNamesAccepted\",\n \"status\": \"True\",\n \"type\": \"Established\"\n }\n ],\n \"storedVersions\": [\n \"v1\"\n ]\n }\n}\n" }, { "path": "pkg/backup/actions/testdata/v1/prometheuses.monitoring.coreos.com.json", "content": "{\n \"apiVersion\": \"apiextensions.k8s.io/v1\",\n \"kind\": \"CustomResourceDefinition\",\n \"metadata\": {\n \"annotations\": {\n \"helm.sh/hook\": \"crd-install\",\n \"kubectl.kubernetes.io/last-applied-configuration\": \"{\\\"apiVersion\\\":\\\"apiextensions.k8s.io/v1beta1\\\",\\\"kind\\\":\\\"CustomResourceDefinition\\\",\\\"metadata\\\":{\\\"annotations\\\":{\\\"helm.sh/hook\\\":\\\"crd-install\\\"},\\\"creationTimestamp\\\":null,\\\"labels\\\":{\\\"app\\\":\\\"prometheus-operator\\\"},\\\"name\\\":\\\"prometheuses.monitoring.coreos.com\\\"},\\\"spec\\\":{\\\"group\\\":\\\"monitoring.coreos.com\\\",\\\"names\\\":{\\\"kind\\\":\\\"Prometheus\\\",\\\"plural\\\":\\\"prometheuses\\\"},\\\"scope\\\":\\\"Namespaced\\\",\\\"validation\\\":{\\\"openAPIV3Schema\\\":{\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"spec\\\":{\\\"description\\\":\\\"PrometheusSpec is a specification of the desired behavior of the Prometheus cluster. More info: https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#spec-and-status\\\",\\\"properties\\\":{\\\"additionalAlertManagerConfigs\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"additionalAlertRelabelConfigs\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"additionalScrapeConfigs\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"affinity\\\":{\\\"description\\\":\\\"Affinity is a group of affinity scheduling rules.\\\",\\\"properties\\\":{\\\"nodeAffinity\\\":{\\\"description\\\":\\\"Node affinity is a group of node affinity scheduling rules.\\\",\\\"properties\\\":{\\\"preferredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\\\\\"weight\\\\\\\" to the sum if the node matches the corresponding matchExpressions; the node(s) with the highest sum are the most preferred.\\\",\\\"items\\\":{\\\"description\\\":\\\"An empty preferred scheduling term matches all objects with implicit weight 0 (i.e. it's a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op).\\\",\\\"properties\\\":{\\\"preference\\\":{\\\"description\\\":\\\"A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"A list of node selector requirements by node's labels.\\\",\\\"items\\\":{\\\"description\\\":\\\"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchFields\\\":{\\\"description\\\":\\\"A list of node selector requirements by node's fields.\\\",\\\"items\\\":{\\\"description\\\":\\\"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"weight\\\":{\\\"description\\\":\\\"Weight associated with matching the corresponding nodeSelectorTerm, in the range 1-100.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"required\\\":[\\\"weight\\\",\\\"preference\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"requiredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"A node selector represents the union of the results of one or more label queries over a set of nodes; that is, it represents the OR of the selectors represented by the node selector terms.\\\",\\\"properties\\\":{\\\"nodeSelectorTerms\\\":{\\\"description\\\":\\\"Required. A list of node selector terms. The terms are ORed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"A list of node selector requirements by node's labels.\\\",\\\"items\\\":{\\\"description\\\":\\\"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchFields\\\":{\\\"description\\\":\\\"A list of node selector requirements by node's fields.\\\",\\\"items\\\":{\\\"description\\\":\\\"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"nodeSelectorTerms\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"podAffinity\\\":{\\\"description\\\":\\\"Pod affinity is a group of inter pod affinity scheduling rules.\\\",\\\"properties\\\":{\\\"preferredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\\\\\"weight\\\\\\\" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.\\\",\\\"items\\\":{\\\"description\\\":\\\"The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)\\\",\\\"properties\\\":{\\\"podAffinityTerm\\\":{\\\"description\\\":\\\"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\\\u003ctopologyKey\\\\u003e matches that of any node on which a pod of the set of pods is running\\\",\\\"properties\\\":{\\\"labelSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"namespaces\\\":{\\\"description\\\":\\\"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\\\\\"this pod's namespace\\\\\\\"\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"topologyKey\\\":{\\\"description\\\":\\\"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"topologyKey\\\"],\\\"type\\\":\\\"object\\\"},\\\"weight\\\":{\\\"description\\\":\\\"weight associated with matching the corresponding podAffinityTerm, in the range 1-100.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"required\\\":[\\\"weight\\\",\\\"podAffinityTerm\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"requiredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.\\\",\\\"items\\\":{\\\"description\\\":\\\"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\\\u003ctopologyKey\\\\u003e matches that of any node on which a pod of the set of pods is running\\\",\\\"properties\\\":{\\\"labelSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"namespaces\\\":{\\\"description\\\":\\\"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\\\\\"this pod's namespace\\\\\\\"\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"topologyKey\\\":{\\\"description\\\":\\\"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"topologyKey\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"podAntiAffinity\\\":{\\\"description\\\":\\\"Pod anti affinity is a group of inter pod anti affinity scheduling rules.\\\",\\\"properties\\\":{\\\"preferredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"The scheduler will prefer to schedule pods to nodes that satisfy the anti-affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling anti-affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\\\\\"weight\\\\\\\" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.\\\",\\\"items\\\":{\\\"description\\\":\\\"The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)\\\",\\\"properties\\\":{\\\"podAffinityTerm\\\":{\\\"description\\\":\\\"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\\\u003ctopologyKey\\\\u003e matches that of any node on which a pod of the set of pods is running\\\",\\\"properties\\\":{\\\"labelSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"namespaces\\\":{\\\"description\\\":\\\"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\\\\\"this pod's namespace\\\\\\\"\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"topologyKey\\\":{\\\"description\\\":\\\"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"topologyKey\\\"],\\\"type\\\":\\\"object\\\"},\\\"weight\\\":{\\\"description\\\":\\\"weight associated with matching the corresponding podAffinityTerm, in the range 1-100.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"required\\\":[\\\"weight\\\",\\\"podAffinityTerm\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"requiredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"If the anti-affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the anti-affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.\\\",\\\"items\\\":{\\\"description\\\":\\\"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\\\u003ctopologyKey\\\\u003e matches that of any node on which a pod of the set of pods is running\\\",\\\"properties\\\":{\\\"labelSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"namespaces\\\":{\\\"description\\\":\\\"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\\\\\"this pod's namespace\\\\\\\"\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"topologyKey\\\":{\\\"description\\\":\\\"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"topologyKey\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"alerting\\\":{\\\"description\\\":\\\"AlertingSpec defines parameters for alerting configuration of Prometheus servers.\\\",\\\"properties\\\":{\\\"alertmanagers\\\":{\\\"description\\\":\\\"AlertmanagerEndpoints Prometheus should fire alerts against.\\\",\\\"items\\\":{\\\"description\\\":\\\"AlertmanagerEndpoints defines a selection of a single Endpoints object containing alertmanager IPs to fire alerts against.\\\",\\\"properties\\\":{\\\"bearerTokenFile\\\":{\\\"description\\\":\\\"BearerTokenFile to read from filesystem to use when authenticating to Alertmanager.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of Endpoints object in Namespace.\\\",\\\"type\\\":\\\"string\\\"},\\\"namespace\\\":{\\\"description\\\":\\\"Namespace of Endpoints object.\\\",\\\"type\\\":\\\"string\\\"},\\\"pathPrefix\\\":{\\\"description\\\":\\\"Prefix for the HTTP path alerts are pushed to.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use when firing alerts.\\\",\\\"type\\\":\\\"string\\\"},\\\"tlsConfig\\\":{\\\"description\\\":\\\"TLSConfig specifies TLS configuration parameters.\\\",\\\"properties\\\":{\\\"caFile\\\":{\\\"description\\\":\\\"The CA cert to use for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"certFile\\\":{\\\"description\\\":\\\"The client cert file for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"insecureSkipVerify\\\":{\\\"description\\\":\\\"Disable target certificate validation.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"keyFile\\\":{\\\"description\\\":\\\"The client key file for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"serverName\\\":{\\\"description\\\":\\\"Used to verify the hostname for the targets.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"required\\\":[\\\"namespace\\\",\\\"name\\\",\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"alertmanagers\\\"],\\\"type\\\":\\\"object\\\"},\\\"apiserverConfig\\\":{\\\"description\\\":\\\"APIServerConfig defines a host and auth methods to access apiserver. More info: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#kubernetes_sd_config\\\",\\\"properties\\\":{\\\"basicAuth\\\":{\\\"description\\\":\\\"BasicAuth allow an endpoint to authenticate over basic authentication More info: https://prometheus.io/docs/operating/configuration/#endpoints\\\",\\\"properties\\\":{\\\"password\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"username\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"bearerToken\\\":{\\\"description\\\":\\\"Bearer token for accessing apiserver.\\\",\\\"type\\\":\\\"string\\\"},\\\"bearerTokenFile\\\":{\\\"description\\\":\\\"File to read bearer token for accessing apiserver.\\\",\\\"type\\\":\\\"string\\\"},\\\"host\\\":{\\\"description\\\":\\\"Host of apiserver. A valid string consisting of a hostname or IP followed by an optional port number\\\",\\\"type\\\":\\\"string\\\"},\\\"tlsConfig\\\":{\\\"description\\\":\\\"TLSConfig specifies TLS configuration parameters.\\\",\\\"properties\\\":{\\\"caFile\\\":{\\\"description\\\":\\\"The CA cert to use for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"certFile\\\":{\\\"description\\\":\\\"The client cert file for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"insecureSkipVerify\\\":{\\\"description\\\":\\\"Disable target certificate validation.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"keyFile\\\":{\\\"description\\\":\\\"The client key file for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"serverName\\\":{\\\"description\\\":\\\"Used to verify the hostname for the targets.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"required\\\":[\\\"host\\\"],\\\"type\\\":\\\"object\\\"},\\\"baseImage\\\":{\\\"description\\\":\\\"Base image to use for a Prometheus deployment.\\\",\\\"type\\\":\\\"string\\\"},\\\"configMaps\\\":{\\\"description\\\":\\\"ConfigMaps is a list of ConfigMaps in the same namespace as the Prometheus object, which shall be mounted into the Prometheus Pods. The ConfigMaps are mounted into /etc/prometheus/configmaps/\\\\u003cconfigmap-name\\\\u003e.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"containers\\\":{\\\"description\\\":\\\"Containers allows injecting additional containers or modifying operator generated containers. This can be used to allow adding an authentication proxy to a Prometheus pod or to change the behavior of an operator generated container. Containers described here modify an operator generated container if they share the same name and modifications are done via a strategic merge patch. The current container names are: `prometheus`, `prometheus-config-reloader`, `rules-configmap-reloader`, and `thanos-sidecar`. Overriding containers is entirely outside the scope of what the maintainers will support and by doing so, you accept that this behaviour may break at any time without notice.\\\",\\\"items\\\":{\\\"description\\\":\\\"A single application container that you want to run within a pod.\\\",\\\"properties\\\":{\\\"args\\\":{\\\"description\\\":\\\"Arguments to the entrypoint. The docker image's CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"command\\\":{\\\"description\\\":\\\"Entrypoint array. Not executed within a shell. The docker image's ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"env\\\":{\\\"description\\\":\\\"List of environment variables to set in the container. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"EnvVar represents an environment variable present in a Container.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the environment variable. Must be a C_IDENTIFIER.\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to \\\\\\\"\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"},\\\"valueFrom\\\":{\\\"description\\\":\\\"EnvVarSource represents a source for the value of an EnvVar.\\\",\\\"properties\\\":{\\\"configMapKeyRef\\\":{\\\"description\\\":\\\"Selects a key from a ConfigMap.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key to select.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"fieldRef\\\":{\\\"description\\\":\\\"ObjectFieldSelector selects an APIVersioned field of an object.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"Version of the schema the FieldPath is written in terms of, defaults to \\\\\\\"v1\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"},\\\"fieldPath\\\":{\\\"description\\\":\\\"Path of the field to select in the specified API version.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"fieldPath\\\"],\\\"type\\\":\\\"object\\\"},\\\"resourceFieldRef\\\":{\\\"description\\\":\\\"ResourceFieldSelector represents container resources (cpu, memory) and their output format\\\",\\\"properties\\\":{\\\"containerName\\\":{\\\"description\\\":\\\"Container name: required for volumes, optional for env vars\\\",\\\"type\\\":\\\"string\\\"},\\\"divisor\\\":{},\\\"resource\\\":{\\\"description\\\":\\\"Required: resource to select\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"resource\\\"],\\\"type\\\":\\\"object\\\"},\\\"secretKeyRef\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"required\\\":[\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"envFrom\\\":{\\\"description\\\":\\\"List of sources to populate environment variables in the container. The keys defined within a source must be a C_IDENTIFIER. All invalid keys will be reported as an event when the container is starting. When a key exists in multiple sources, the value associated with the last source will take precedence. Values defined by an Env with a duplicate key will take precedence. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"EnvFromSource represents the source of a set of ConfigMaps\\\",\\\"properties\\\":{\\\"configMapRef\\\":{\\\"description\\\":\\\"ConfigMapEnvSource selects a ConfigMap to populate the environment variables with.\\\\n\\\\nThe contents of the target ConfigMap's Data field will represent the key-value pairs as environment variables.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"type\\\":\\\"object\\\"},\\\"prefix\\\":{\\\"description\\\":\\\"An optional identifier to prepend to each key in the ConfigMap. Must be a C_IDENTIFIER.\\\",\\\"type\\\":\\\"string\\\"},\\\"secretRef\\\":{\\\"description\\\":\\\"SecretEnvSource selects a Secret to populate the environment variables with.\\\\n\\\\nThe contents of the target Secret's Data field will represent the key-value pairs as environment variables.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"image\\\":{\\\"description\\\":\\\"Docker image name. More info: https://kubernetes.io/docs/concepts/containers/images This field is optional to allow higher level config management to default or override container images in workload controllers like Deployments and StatefulSets.\\\",\\\"type\\\":\\\"string\\\"},\\\"imagePullPolicy\\\":{\\\"description\\\":\\\"Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images\\\",\\\"type\\\":\\\"string\\\"},\\\"lifecycle\\\":{\\\"description\\\":\\\"Lifecycle describes actions that the management system should take in response to container lifecycle events. For the PostStart and PreStop lifecycle handlers, management of the container blocks until the action is complete, unless the container process fails, in which case the handler is aborted.\\\",\\\"properties\\\":{\\\"postStart\\\":{\\\"description\\\":\\\"Handler defines a specific action that should be taken\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"ExecAction describes a \\\\\\\"run in container\\\\\\\" action.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGetAction describes an action based on HTTP Get requests.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocketAction describes an action based on opening a socket\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"preStop\\\":{\\\"description\\\":\\\"Handler defines a specific action that should be taken\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"ExecAction describes a \\\\\\\"run in container\\\\\\\" action.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGetAction describes an action based on HTTP Get requests.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocketAction describes an action based on opening a socket\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"livenessProbe\\\":{\\\"description\\\":\\\"Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic.\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"ExecAction describes a \\\\\\\"run in container\\\\\\\" action.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"failureThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGetAction describes an action based on HTTP Get requests.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"initialDelaySeconds\\\":{\\\"description\\\":\\\"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"periodSeconds\\\":{\\\"description\\\":\\\"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"successThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocketAction describes an action based on opening a socket\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"timeoutSeconds\\\":{\\\"description\\\":\\\"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the container specified as a DNS_LABEL. Each container in a pod must have a unique name (DNS_LABEL). Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"},\\\"ports\\\":{\\\"description\\\":\\\"List of ports to expose from the container. Exposing a port here gives the system additional information about the network connections a container uses, but is primarily informational. Not specifying a port here DOES NOT prevent that port from being exposed. Any port which is listening on the default \\\\\\\"0.0.0.0\\\\\\\" address inside a container will be accessible from the network. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"ContainerPort represents a network port in a single container.\\\",\\\"properties\\\":{\\\"containerPort\\\":{\\\"description\\\":\\\"Number of port to expose on the pod's IP address. This must be a valid port number, 0 \\\\u003c x \\\\u003c 65536.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"hostIP\\\":{\\\"description\\\":\\\"What host IP to bind the external port to.\\\",\\\"type\\\":\\\"string\\\"},\\\"hostPort\\\":{\\\"description\\\":\\\"Number of port to expose on the host. If specified, this must be a valid port number, 0 \\\\u003c x \\\\u003c 65536. If HostNetwork is specified, this must match ContainerPort. Most containers do not need this.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"name\\\":{\\\"description\\\":\\\"If specified, this must be an IANA_SVC_NAME and unique within the pod. Each named port in a pod must have a unique name. Name for the port that can be referred to by services.\\\",\\\"type\\\":\\\"string\\\"},\\\"protocol\\\":{\\\"description\\\":\\\"Protocol for port. Must be UDP, TCP, or SCTP. Defaults to \\\\\\\"TCP\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"containerPort\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"readinessProbe\\\":{\\\"description\\\":\\\"Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic.\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"ExecAction describes a \\\\\\\"run in container\\\\\\\" action.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"failureThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGetAction describes an action based on HTTP Get requests.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"initialDelaySeconds\\\":{\\\"description\\\":\\\"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"periodSeconds\\\":{\\\"description\\\":\\\"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"successThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocketAction describes an action based on opening a socket\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"timeoutSeconds\\\":{\\\"description\\\":\\\"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"},\\\"resources\\\":{\\\"description\\\":\\\"ResourceRequirements describes the compute resource requirements.\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"securityContext\\\":{\\\"description\\\":\\\"SecurityContext holds security configuration that will be applied to a container. Some fields are present in both SecurityContext and PodSecurityContext. When both are set, the values in SecurityContext take precedence.\\\",\\\"properties\\\":{\\\"allowPrivilegeEscalation\\\":{\\\"description\\\":\\\"AllowPrivilegeEscalation controls whether a process can gain more privileges than its parent process. This bool directly controls if the no_new_privs flag will be set on the container process. AllowPrivilegeEscalation is true always when the container is: 1) run as Privileged 2) has CAP_SYS_ADMIN\\\",\\\"type\\\":\\\"boolean\\\"},\\\"capabilities\\\":{\\\"description\\\":\\\"Adds and removes POSIX capabilities from running containers.\\\",\\\"properties\\\":{\\\"add\\\":{\\\"description\\\":\\\"Added capabilities\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"drop\\\":{\\\"description\\\":\\\"Removed capabilities\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"privileged\\\":{\\\"description\\\":\\\"Run container in privileged mode. Processes in privileged containers are essentially equivalent to root on the host. Defaults to false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"procMount\\\":{\\\"description\\\":\\\"procMount denotes the type of proc mount to use for the containers. The default is DefaultProcMount which uses the container runtime defaults for readonly paths and masked paths. This requires the ProcMountType feature flag to be enabled.\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnlyRootFilesystem\\\":{\\\"description\\\":\\\"Whether this container has a read-only root filesystem. Default is false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"runAsGroup\\\":{\\\"description\\\":\\\"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"runAsNonRoot\\\":{\\\"description\\\":\\\"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"runAsUser\\\":{\\\"description\\\":\\\"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"seLinuxOptions\\\":{\\\"description\\\":\\\"SELinuxOptions are the labels to be applied to the container\\\",\\\"properties\\\":{\\\"level\\\":{\\\"description\\\":\\\"Level is SELinux level label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"role\\\":{\\\"description\\\":\\\"Role is a SELinux role label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"type\\\":{\\\"description\\\":\\\"Type is a SELinux type label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"user\\\":{\\\"description\\\":\\\"User is a SELinux user label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"stdin\\\":{\\\"description\\\":\\\"Whether this container should allocate a buffer for stdin in the container runtime. If this is not set, reads from stdin in the container will always result in EOF. Default is false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"stdinOnce\\\":{\\\"description\\\":\\\"Whether the container runtime should close the stdin channel after it has been opened by a single attach. When stdin is true the stdin stream will remain open across multiple attach sessions. If stdinOnce is set to true, stdin is opened on container start, is empty until the first client attaches to stdin, and then remains open and accepts data until the client disconnects, at which time stdin is closed and remains closed until the container is restarted. If this flag is false, a container processes that reads from stdin will never receive an EOF. Default is false\\\",\\\"type\\\":\\\"boolean\\\"},\\\"terminationMessagePath\\\":{\\\"description\\\":\\\"Optional: Path at which the file to which the container's termination message will be written is mounted into the container's filesystem. Message written is intended to be brief final status, such as an assertion failure message. Will be truncated by the node if greater than 4096 bytes. The total message length across all containers will be limited to 12kb. Defaults to /dev/termination-log. Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"},\\\"terminationMessagePolicy\\\":{\\\"description\\\":\\\"Indicate how the termination message should be populated. File will use the contents of terminationMessagePath to populate the container status message on both success and failure. FallbackToLogsOnError will use the last chunk of container log output if the termination message file is empty and the container exited with an error. The log output is limited to 2048 bytes or 80 lines, whichever is smaller. Defaults to File. Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"},\\\"tty\\\":{\\\"description\\\":\\\"Whether this container should allocate a TTY for itself, also requires 'stdin' to be true. Default is false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"volumeDevices\\\":{\\\"description\\\":\\\"volumeDevices is the list of block devices to be used by the container. This is a beta feature.\\\",\\\"items\\\":{\\\"description\\\":\\\"volumeDevice describes a mapping of a raw block device within a container.\\\",\\\"properties\\\":{\\\"devicePath\\\":{\\\"description\\\":\\\"devicePath is the path inside of the container that the device will be mapped to.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"name must match the name of a persistentVolumeClaim in the pod\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"devicePath\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"volumeMounts\\\":{\\\"description\\\":\\\"Pod volumes to mount into the container's filesystem. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"VolumeMount describes a mounting of a Volume within a container.\\\",\\\"properties\\\":{\\\"mountPath\\\":{\\\"description\\\":\\\"Path within the container at which the volume should be mounted. Must not contain ':'.\\\",\\\"type\\\":\\\"string\\\"},\\\"mountPropagation\\\":{\\\"description\\\":\\\"mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"This must match the Name of a Volume.\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"subPath\\\":{\\\"description\\\":\\\"Path within the volume from which the container's volume should be mounted. Defaults to \\\\\\\"\\\\\\\" (volume's root).\\\",\\\"type\\\":\\\"string\\\"},\\\"subPathExpr\\\":{\\\"description\\\":\\\"Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to \\\\\\\"\\\\\\\" (volume's root). SubPathExpr and SubPath are mutually exclusive. This field is alpha in 1.14.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"mountPath\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"workingDir\\\":{\\\"description\\\":\\\"Container's working directory. If not specified, the container runtime's default will be used, which might be configured in the container image. Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"enableAdminAPI\\\":{\\\"description\\\":\\\"Enable access to prometheus web admin API. Defaults to the value of `false`. WARNING: Enabling the admin APIs enables mutating endpoints, to delete data, shutdown Prometheus, and more. Enabling this should be done with care and the user is advised to add additional authentication authorization via a proxy to ensure only clients authorized to perform these actions can do so. For more information see https://prometheus.io/docs/prometheus/latest/querying/api/#tsdb-admin-apis\\\",\\\"type\\\":\\\"boolean\\\"},\\\"evaluationInterval\\\":{\\\"description\\\":\\\"Interval between consecutive evaluations.\\\",\\\"type\\\":\\\"string\\\"},\\\"externalLabels\\\":{\\\"description\\\":\\\"The labels to add to any time series or alerts when communicating with external systems (federation, remote storage, Alertmanager).\\\",\\\"type\\\":\\\"object\\\"},\\\"externalUrl\\\":{\\\"description\\\":\\\"The external URL the Prometheus instances will be available under. This is necessary to generate correct URLs. This is necessary if Prometheus is not served from root of a DNS name.\\\",\\\"type\\\":\\\"string\\\"},\\\"image\\\":{\\\"description\\\":\\\"Image if specified has precedence over baseImage, tag and sha combinations. Specifying the version is still necessary to ensure the Prometheus Operator knows what version of Prometheus is being configured.\\\",\\\"type\\\":\\\"string\\\"},\\\"imagePullSecrets\\\":{\\\"description\\\":\\\"An optional list of references to secrets in the same namespace to use for pulling prometheus and alertmanager images from registries see http://kubernetes.io/docs/user-guide/images#specifying-imagepullsecrets-on-a-pod\\\",\\\"items\\\":{\\\"description\\\":\\\"LocalObjectReference contains enough information to let you locate the referenced object inside the same namespace.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"listenLocal\\\":{\\\"description\\\":\\\"ListenLocal makes the Prometheus server listen on loopback, so that it does not bind against the Pod IP.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"logFormat\\\":{\\\"description\\\":\\\"Log format for Prometheus to be configured with.\\\",\\\"type\\\":\\\"string\\\"},\\\"logLevel\\\":{\\\"description\\\":\\\"Log level for Prometheus to be configured with.\\\",\\\"type\\\":\\\"string\\\"},\\\"nodeSelector\\\":{\\\"description\\\":\\\"Define which Nodes the Pods are scheduled on.\\\",\\\"type\\\":\\\"object\\\"},\\\"paused\\\":{\\\"description\\\":\\\"When a Prometheus deployment is paused, no actions except for deletion will be performed on the underlying objects.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"podMetadata\\\":{\\\"description\\\":\\\"ObjectMeta is metadata that all persisted resources must have, which includes all objects users must create.\\\",\\\"properties\\\":{\\\"annotations\\\":{\\\"description\\\":\\\"Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations\\\",\\\"type\\\":\\\"object\\\"},\\\"clusterName\\\":{\\\"description\\\":\\\"The name of the cluster which the object belongs to. This is used to distinguish resources with same name and namespace in different clusters. This field is not set anywhere right now and apiserver is going to ignore it if set in create or update request.\\\",\\\"type\\\":\\\"string\\\"},\\\"creationTimestamp\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"deletionGracePeriodSeconds\\\":{\\\"description\\\":\\\"Number of seconds allowed for this object to gracefully terminate before it will be removed from the system. Only set when deletionTimestamp is also set. May only be shortened. Read-only.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"deletionTimestamp\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"finalizers\\\":{\\\"description\\\":\\\"Must be empty before the object is deleted from the registry. Each entry is an identifier for the responsible component that will remove the entry from the list. If the deletionTimestamp of the object is non-nil, entries in this list can only be removed.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"generateName\\\":{\\\"description\\\":\\\"GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server.\\\\n\\\\nIf this field is specified and the generated name exists, the server will NOT return a 409 - instead, it will either return 201 Created or 500 with Reason ServerTimeout indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header).\\\\n\\\\nApplied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#idempotency\\\",\\\"type\\\":\\\"string\\\"},\\\"generation\\\":{\\\"description\\\":\\\"A sequence number representing a specific generation of the desired state. Populated by the system. Read-only.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"initializers\\\":{\\\"description\\\":\\\"Initializers tracks the progress of initialization.\\\",\\\"properties\\\":{\\\"pending\\\":{\\\"description\\\":\\\"Pending is a list of initializers that must execute in order before this object is visible. When the last pending initializer is removed, and no failing result is set, the initializers struct will be set to nil and the object is considered as initialized and visible to all clients.\\\",\\\"items\\\":{\\\"description\\\":\\\"Initializer is information about an initializer that has not yet completed.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"name of the process that is responsible for initializing this object.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"result\\\":{\\\"description\\\":\\\"Status is a return value for calls that don't return other objects.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"code\\\":{\\\"description\\\":\\\"Suggested HTTP return code for this status, 0 if not set.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"details\\\":{\\\"description\\\":\\\"StatusDetails is a set of additional properties that MAY be set by the server to provide additional information about a response. The Reason field of a Status object defines what attributes will be set. Clients must ignore fields that do not match the defined type of each attribute, and should assume that any attribute may be empty, invalid, or under defined.\\\",\\\"properties\\\":{\\\"causes\\\":{\\\"description\\\":\\\"The Causes array includes more details associated with the StatusReason failure. Not all StatusReasons may provide detailed causes.\\\",\\\"items\\\":{\\\"description\\\":\\\"StatusCause provides more information about an api.Status failure, including cases when multiple errors are encountered.\\\",\\\"properties\\\":{\\\"field\\\":{\\\"description\\\":\\\"The field of the resource that has caused this error, as named by its JSON serialization. May include dot and postfix notation for nested attributes. Arrays are zero-indexed. Fields may appear more than once in an array of causes due to fields having multiple errors. Optional.\\\\n\\\\nExamples:\\\\n \\\\\\\"name\\\\\\\" - the field \\\\\\\"name\\\\\\\" on the current resource\\\\n \\\\\\\"items[0].name\\\\\\\" - the field \\\\\\\"name\\\\\\\" on the first array entry in \\\\\\\"items\\\\\\\"\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"A human-readable description of the cause of the error. This field may be presented as-is to a reader.\\\",\\\"type\\\":\\\"string\\\"},\\\"reason\\\":{\\\"description\\\":\\\"A machine-readable description of the cause of the error. If this value is empty there is no information available.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"group\\\":{\\\"description\\\":\\\"The group attribute of the resource associated with the status StatusReason.\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"The kind attribute of the resource associated with the status StatusReason. On some operations may differ from the requested resource Kind. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"The name attribute of the resource associated with the status StatusReason (when there is a single name which can be described).\\\",\\\"type\\\":\\\"string\\\"},\\\"retryAfterSeconds\\\":{\\\"description\\\":\\\"If specified, the time in seconds before the operation should be retried. Some errors may indicate the client must take an alternate action - for those errors this field may indicate how long to wait before taking the alternate action.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID of the resource. (when there is a single resource which can be described). More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"A human-readable description of the status of this operation.\\\",\\\"type\\\":\\\"string\\\"},\\\"metadata\\\":{\\\"description\\\":\\\"ListMeta describes metadata that synthetic resources must have, including lists and various status objects. A resource may have only one of {ObjectMeta, ListMeta}.\\\",\\\"properties\\\":{\\\"continue\\\":{\\\"description\\\":\\\"continue may be set if the user set a limit on the number of items returned, and indicates that the server has more data available. The value is opaque and may be used to issue another request to the endpoint that served this list to retrieve the next set of available objects. Continuing a consistent list may not be possible if the server configuration has changed or more than a few minutes have passed. The resourceVersion field returned when using this continue value will be identical to the value in the first response, unless you have received this token from an error message.\\\",\\\"type\\\":\\\"string\\\"},\\\"resourceVersion\\\":{\\\"description\\\":\\\"String that identifies the server's internal version of this object that can be used by clients to determine when objects have changed. Value must be treated as opaque by clients and passed unmodified back to the server. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\\\",\\\"type\\\":\\\"string\\\"},\\\"selfLink\\\":{\\\"description\\\":\\\"selfLink is a URL representing this object. Populated by the system. Read-only.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"reason\\\":{\\\"description\\\":\\\"A machine-readable description of why this operation is in the \\\\\\\"Failure\\\\\\\" status. If this value is empty there is no information available. A Reason clarifies an HTTP status code but does not override it.\\\",\\\"type\\\":\\\"string\\\"},\\\"status\\\":{\\\"description\\\":\\\"Status of the operation. One of: \\\\\\\"Success\\\\\\\" or \\\\\\\"Failure\\\\\\\". More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"required\\\":[\\\"pending\\\"],\\\"type\\\":\\\"object\\\"},\\\"labels\\\":{\\\"description\\\":\\\"Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels\\\",\\\"type\\\":\\\"object\\\"},\\\"managedFields\\\":{\\\"description\\\":\\\"ManagedFields maps workflow-id and version to the set of fields that are managed by that workflow. This is mostly for internal housekeeping, and users typically shouldn't need to set or understand this field. A workflow can be the user's name, a controller's name, or the name of a specific apply path like \\\\\\\"ci-cd\\\\\\\". The set of fields is always in the version that the workflow used when modifying the object.\\\\n\\\\nThis field is alpha and can be changed or removed without notice.\\\",\\\"items\\\":{\\\"description\\\":\\\"ManagedFieldsEntry is a workflow-id, a FieldSet and the group version of the resource that the fieldset applies to.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the version of this resource that this field set applies to. The format is \\\\\\\"group/version\\\\\\\" just like the top-level APIVersion field. It is necessary to track the version of a field set because it cannot be automatically converted.\\\",\\\"type\\\":\\\"string\\\"},\\\"fields\\\":{\\\"description\\\":\\\"Fields stores a set of fields in a data structure like a Trie. To understand how this is used, see: https://github.com/kubernetes-sigs/structured-merge-diff\\\",\\\"type\\\":\\\"object\\\"},\\\"manager\\\":{\\\"description\\\":\\\"Manager is an identifier of the workflow managing these fields.\\\",\\\"type\\\":\\\"string\\\"},\\\"operation\\\":{\\\"description\\\":\\\"Operation is the type of operation which lead to this ManagedFieldsEntry being created. The only valid values for this field are 'Apply' and 'Update'.\\\",\\\"type\\\":\\\"string\\\"},\\\"time\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names\\\",\\\"type\\\":\\\"string\\\"},\\\"namespace\\\":{\\\"description\\\":\\\"Namespace defines the space within each name must be unique. An empty namespace is equivalent to the \\\\\\\"default\\\\\\\" namespace, but \\\\\\\"default\\\\\\\" is the canonical representation. Not all objects are required to be scoped to a namespace - the value of this field for those objects will be empty.\\\\n\\\\nMust be a DNS_LABEL. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/namespaces\\\",\\\"type\\\":\\\"string\\\"},\\\"ownerReferences\\\":{\\\"description\\\":\\\"List of objects depended by this object. If ALL objects in the list have been deleted, this object will be garbage collected. If this object is managed by a controller, then an entry in this list will point to this controller, with the controller field set to true. There cannot be more than one managing controller.\\\",\\\"items\\\":{\\\"description\\\":\\\"OwnerReference contains enough information to let you identify an owning object. An owning object must be in the same namespace as the dependent, or be cluster-scoped, so there is no namespace field.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"API version of the referent.\\\",\\\"type\\\":\\\"string\\\"},\\\"blockOwnerDeletion\\\":{\\\"description\\\":\\\"If true, AND if the owner has the \\\\\\\"foregroundDeletion\\\\\\\" finalizer, then the owner cannot be deleted from the key-value store until this reference is removed. Defaults to false. To set this field, a user needs \\\\\\\"delete\\\\\\\" permission of the owner, otherwise 422 (Unprocessable Entity) will be returned.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"controller\\\":{\\\"description\\\":\\\"If true, this reference points to the managing controller.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#names\\\",\\\"type\\\":\\\"string\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"apiVersion\\\",\\\"kind\\\",\\\"name\\\",\\\"uid\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"resourceVersion\\\":{\\\"description\\\":\\\"An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources.\\\\n\\\\nPopulated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\\\",\\\"type\\\":\\\"string\\\"},\\\"selfLink\\\":{\\\"description\\\":\\\"SelfLink is a URL representing this object. Populated by the system. Read-only.\\\",\\\"type\\\":\\\"string\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID is the unique in time and space value for this object. It is typically generated by the server on successful creation of a resource and is not allowed to change on PUT operations.\\\\n\\\\nPopulated by the system. Read-only. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"podMonitorNamespaceSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"podMonitorSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"priorityClassName\\\":{\\\"description\\\":\\\"Priority class assigned to the Pods\\\",\\\"type\\\":\\\"string\\\"},\\\"prometheusExternalLabelName\\\":{\\\"description\\\":\\\"Name of Prometheus external label used to denote Prometheus instance name. Defaults to the value of `prometheus`. External label will _not_ be added when value is set to empty string (`\\\\\\\"\\\\\\\"`).\\\",\\\"type\\\":\\\"string\\\"},\\\"query\\\":{\\\"description\\\":\\\"QuerySpec defines the query command line flags when starting Prometheus.\\\",\\\"properties\\\":{\\\"lookbackDelta\\\":{\\\"description\\\":\\\"The delta difference allowed for retrieving metrics during expression evaluations.\\\",\\\"type\\\":\\\"string\\\"},\\\"maxConcurrency\\\":{\\\"description\\\":\\\"Number of concurrent queries that can be run at once.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"maxSamples\\\":{\\\"description\\\":\\\"Maximum number of samples a single query can load into memory. Note that queries will fail if they would load more samples than this into memory, so this also limits the number of samples a query can return.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"timeout\\\":{\\\"description\\\":\\\"Maximum time a query may take before being aborted.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"remoteRead\\\":{\\\"description\\\":\\\"If specified, the remote_read spec. This is an experimental feature, it may change in any upcoming release in a breaking way.\\\",\\\"items\\\":{\\\"description\\\":\\\"RemoteReadSpec defines the remote_read configuration for prometheus.\\\",\\\"properties\\\":{\\\"basicAuth\\\":{\\\"description\\\":\\\"BasicAuth allow an endpoint to authenticate over basic authentication More info: https://prometheus.io/docs/operating/configuration/#endpoints\\\",\\\"properties\\\":{\\\"password\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"username\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"bearerToken\\\":{\\\"description\\\":\\\"bearer token for remote read.\\\",\\\"type\\\":\\\"string\\\"},\\\"bearerTokenFile\\\":{\\\"description\\\":\\\"File to read bearer token for remote read.\\\",\\\"type\\\":\\\"string\\\"},\\\"proxyUrl\\\":{\\\"description\\\":\\\"Optional ProxyURL\\\",\\\"type\\\":\\\"string\\\"},\\\"readRecent\\\":{\\\"description\\\":\\\"Whether reads should be made for queries for time ranges that the local storage should have complete data for.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"remoteTimeout\\\":{\\\"description\\\":\\\"Timeout for requests to the remote read endpoint.\\\",\\\"type\\\":\\\"string\\\"},\\\"requiredMatchers\\\":{\\\"description\\\":\\\"An optional list of equality matchers which have to be present in a selector to query the remote read endpoint.\\\",\\\"type\\\":\\\"object\\\"},\\\"tlsConfig\\\":{\\\"description\\\":\\\"TLSConfig specifies TLS configuration parameters.\\\",\\\"properties\\\":{\\\"caFile\\\":{\\\"description\\\":\\\"The CA cert to use for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"certFile\\\":{\\\"description\\\":\\\"The client cert file for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"insecureSkipVerify\\\":{\\\"description\\\":\\\"Disable target certificate validation.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"keyFile\\\":{\\\"description\\\":\\\"The client key file for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"serverName\\\":{\\\"description\\\":\\\"Used to verify the hostname for the targets.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"url\\\":{\\\"description\\\":\\\"The URL of the endpoint to send samples to.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"url\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"remoteWrite\\\":{\\\"description\\\":\\\"If specified, the remote_write spec. This is an experimental feature, it may change in any upcoming release in a breaking way.\\\",\\\"items\\\":{\\\"description\\\":\\\"RemoteWriteSpec defines the remote_write configuration for prometheus.\\\",\\\"properties\\\":{\\\"basicAuth\\\":{\\\"description\\\":\\\"BasicAuth allow an endpoint to authenticate over basic authentication More info: https://prometheus.io/docs/operating/configuration/#endpoints\\\",\\\"properties\\\":{\\\"password\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"username\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"bearerToken\\\":{\\\"description\\\":\\\"File to read bearer token for remote write.\\\",\\\"type\\\":\\\"string\\\"},\\\"bearerTokenFile\\\":{\\\"description\\\":\\\"File to read bearer token for remote write.\\\",\\\"type\\\":\\\"string\\\"},\\\"proxyUrl\\\":{\\\"description\\\":\\\"Optional ProxyURL\\\",\\\"type\\\":\\\"string\\\"},\\\"queueConfig\\\":{\\\"description\\\":\\\"QueueConfig allows the tuning of remote_write queue_config parameters. This object is referenced in the RemoteWriteSpec object.\\\",\\\"properties\\\":{\\\"batchSendDeadline\\\":{\\\"description\\\":\\\"BatchSendDeadline is the maximum time a sample will wait in buffer.\\\",\\\"type\\\":\\\"string\\\"},\\\"capacity\\\":{\\\"description\\\":\\\"Capacity is the number of samples to buffer per shard before we start dropping them.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"maxBackoff\\\":{\\\"description\\\":\\\"MaxBackoff is the maximum retry delay.\\\",\\\"type\\\":\\\"string\\\"},\\\"maxRetries\\\":{\\\"description\\\":\\\"MaxRetries is the maximum number of times to retry a batch on recoverable errors.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"maxSamplesPerSend\\\":{\\\"description\\\":\\\"MaxSamplesPerSend is the maximum number of samples per send.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"maxShards\\\":{\\\"description\\\":\\\"MaxShards is the maximum number of shards, i.e. amount of concurrency.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"minBackoff\\\":{\\\"description\\\":\\\"MinBackoff is the initial retry delay. Gets doubled for every retry.\\\",\\\"type\\\":\\\"string\\\"},\\\"minShards\\\":{\\\"description\\\":\\\"MinShards is the minimum number of shards, i.e. amount of concurrency.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"},\\\"remoteTimeout\\\":{\\\"description\\\":\\\"Timeout for requests to the remote write endpoint.\\\",\\\"type\\\":\\\"string\\\"},\\\"tlsConfig\\\":{\\\"description\\\":\\\"TLSConfig specifies TLS configuration parameters.\\\",\\\"properties\\\":{\\\"caFile\\\":{\\\"description\\\":\\\"The CA cert to use for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"certFile\\\":{\\\"description\\\":\\\"The client cert file for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"insecureSkipVerify\\\":{\\\"description\\\":\\\"Disable target certificate validation.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"keyFile\\\":{\\\"description\\\":\\\"The client key file for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"serverName\\\":{\\\"description\\\":\\\"Used to verify the hostname for the targets.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"url\\\":{\\\"description\\\":\\\"The URL of the endpoint to send samples to.\\\",\\\"type\\\":\\\"string\\\"},\\\"writeRelabelConfigs\\\":{\\\"description\\\":\\\"The list of remote write relabel configurations.\\\",\\\"items\\\":{\\\"description\\\":\\\"RelabelConfig allows dynamic rewriting of the label set, being applied to samples before ingestion. It defines `\\\\u003cmetric_relabel_configs\\\\u003e`-section of Prometheus configuration. More info: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#metric_relabel_configs\\\",\\\"properties\\\":{\\\"action\\\":{\\\"description\\\":\\\"Action to perform based on regex matching. Default is 'replace'\\\",\\\"type\\\":\\\"string\\\"},\\\"modulus\\\":{\\\"description\\\":\\\"Modulus to take of the hash of the source label values.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"regex\\\":{\\\"description\\\":\\\"Regular expression against which the extracted value is matched. default is '(.*)'\\\",\\\"type\\\":\\\"string\\\"},\\\"replacement\\\":{\\\"description\\\":\\\"Replacement value against which a regex replace is performed if the regular expression matches. Regex capture groups are available. Default is '$1'\\\",\\\"type\\\":\\\"string\\\"},\\\"separator\\\":{\\\"description\\\":\\\"Separator placed between concatenated source label values. default is ';'.\\\",\\\"type\\\":\\\"string\\\"},\\\"sourceLabels\\\":{\\\"description\\\":\\\"The source labels select values from existing labels. Their content is concatenated using the configured separator and matched against the configured regular expression for the replace, keep, and drop actions.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"targetLabel\\\":{\\\"description\\\":\\\"Label to which the resulting value is written in a replace action. It is mandatory for replace actions. Regex capture groups are available.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"url\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"replicaExternalLabelName\\\":{\\\"description\\\":\\\"Name of Prometheus external label used to denote replica name. Defaults to the value of `prometheus_replica`. External label will _not_ be added when value is set to empty string (`\\\\\\\"\\\\\\\"`).\\\",\\\"type\\\":\\\"string\\\"},\\\"replicas\\\":{\\\"description\\\":\\\"Number of instances to deploy for a Prometheus deployment.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"resources\\\":{\\\"description\\\":\\\"ResourceRequirements describes the compute resource requirements.\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"retention\\\":{\\\"description\\\":\\\"Time duration Prometheus shall retain data for. Default is '24h', and must match the regular expression `[0-9]+(ms|s|m|h|d|w|y)` (milliseconds seconds minutes hours days weeks years).\\\",\\\"type\\\":\\\"string\\\"},\\\"retentionSize\\\":{\\\"description\\\":\\\"Maximum amount of disk space used by blocks.\\\",\\\"type\\\":\\\"string\\\"},\\\"routePrefix\\\":{\\\"description\\\":\\\"The route prefix Prometheus registers HTTP handlers for. This is useful, if using ExternalURL and a proxy is rewriting HTTP routes of a request, and the actual ExternalURL is still true, but the server serves requests under a different route prefix. For example for use with `kubectl proxy`.\\\",\\\"type\\\":\\\"string\\\"},\\\"ruleNamespaceSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"ruleSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"rules\\\":{\\\"description\\\":\\\"/--rules.*/ command-line arguments\\\",\\\"properties\\\":{\\\"alert\\\":{\\\"description\\\":\\\"/--rules.alert.*/ command-line arguments\\\",\\\"properties\\\":{\\\"forGracePeriod\\\":{\\\"description\\\":\\\"Minimum duration between alert and restored 'for' state. This is maintained only for alerts with configured 'for' time greater than grace period.\\\",\\\"type\\\":\\\"string\\\"},\\\"forOutageTolerance\\\":{\\\"description\\\":\\\"Max time to tolerate prometheus outage for restoring 'for' state of alert.\\\",\\\"type\\\":\\\"string\\\"},\\\"resendDelay\\\":{\\\"description\\\":\\\"Minimum amount of time to wait before resending an alert to Alertmanager.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"scrapeInterval\\\":{\\\"description\\\":\\\"Interval between consecutive scrapes.\\\",\\\"type\\\":\\\"string\\\"},\\\"secrets\\\":{\\\"description\\\":\\\"Secrets is a list of Secrets in the same namespace as the Prometheus object, which shall be mounted into the Prometheus Pods. The Secrets are mounted into /etc/prometheus/secrets/\\\\u003csecret-name\\\\u003e.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"securityContext\\\":{\\\"description\\\":\\\"PodSecurityContext holds pod-level security attributes and common container settings. Some fields are also present in container.securityContext. Field values of container.securityContext take precedence over field values of PodSecurityContext.\\\",\\\"properties\\\":{\\\"fsGroup\\\":{\\\"description\\\":\\\"A special supplemental group that applies to all containers in a pod. Some volume types allow the Kubelet to change the ownership of that volume to be owned by the pod:\\\\n\\\\n1. The owning GID will be the FSGroup 2. The setgid bit is set (new files created in the volume will be owned by FSGroup) 3. The permission bits are OR'd with rw-rw----\\\\n\\\\nIf unset, the Kubelet will not modify the ownership and permissions of any volume.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"runAsGroup\\\":{\\\"description\\\":\\\"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"runAsNonRoot\\\":{\\\"description\\\":\\\"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"runAsUser\\\":{\\\"description\\\":\\\"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"seLinuxOptions\\\":{\\\"description\\\":\\\"SELinuxOptions are the labels to be applied to the container\\\",\\\"properties\\\":{\\\"level\\\":{\\\"description\\\":\\\"Level is SELinux level label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"role\\\":{\\\"description\\\":\\\"Role is a SELinux role label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"type\\\":{\\\"description\\\":\\\"Type is a SELinux type label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"user\\\":{\\\"description\\\":\\\"User is a SELinux user label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"supplementalGroups\\\":{\\\"description\\\":\\\"A list of groups applied to the first process run in each container, in addition to the container's primary GID. If unspecified, no groups will be added to any container.\\\",\\\"items\\\":{\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"type\\\":\\\"array\\\"},\\\"sysctls\\\":{\\\"description\\\":\\\"Sysctls hold a list of namespaced sysctls used for the pod. Pods with unsupported sysctls (by the container runtime) might fail to launch.\\\",\\\"items\\\":{\\\"description\\\":\\\"Sysctl defines a kernel parameter to be set\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of a property to set\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"Value of a property to set\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"serviceAccountName\\\":{\\\"description\\\":\\\"ServiceAccountName is the name of the ServiceAccount to use to run the Prometheus Pods.\\\",\\\"type\\\":\\\"string\\\"},\\\"serviceMonitorNamespaceSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"serviceMonitorSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"sha\\\":{\\\"description\\\":\\\"SHA of Prometheus container image to be deployed. Defaults to the value of `version`. Similar to a tag, but the SHA explicitly deploys an immutable container image. Version and Tag are ignored if SHA is set.\\\",\\\"type\\\":\\\"string\\\"},\\\"storage\\\":{\\\"description\\\":\\\"StorageSpec defines the configured storage for a group Prometheus servers. If neither `emptyDir` nor `volumeClaimTemplate` is specified, then by default an [EmptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir) will be used.\\\",\\\"properties\\\":{\\\"emptyDir\\\":{\\\"description\\\":\\\"Represents an empty directory for a pod. Empty directory volumes support ownership management and SELinux relabeling.\\\",\\\"properties\\\":{\\\"medium\\\":{\\\"description\\\":\\\"What type of storage medium should back this directory. The default is \\\\\\\"\\\\\\\" which means to use the node's default medium. Must be an empty string (default) or Memory. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir\\\",\\\"type\\\":\\\"string\\\"},\\\"sizeLimit\\\":{}},\\\"type\\\":\\\"object\\\"},\\\"volumeClaimTemplate\\\":{\\\"description\\\":\\\"PersistentVolumeClaim is a user's request for and claim to a persistent volume\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"metadata\\\":{\\\"description\\\":\\\"ObjectMeta is metadata that all persisted resources must have, which includes all objects users must create.\\\",\\\"properties\\\":{\\\"annotations\\\":{\\\"description\\\":\\\"Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations\\\",\\\"type\\\":\\\"object\\\"},\\\"clusterName\\\":{\\\"description\\\":\\\"The name of the cluster which the object belongs to. This is used to distinguish resources with same name and namespace in different clusters. This field is not set anywhere right now and apiserver is going to ignore it if set in create or update request.\\\",\\\"type\\\":\\\"string\\\"},\\\"creationTimestamp\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"deletionGracePeriodSeconds\\\":{\\\"description\\\":\\\"Number of seconds allowed for this object to gracefully terminate before it will be removed from the system. Only set when deletionTimestamp is also set. May only be shortened. Read-only.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"deletionTimestamp\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"finalizers\\\":{\\\"description\\\":\\\"Must be empty before the object is deleted from the registry. Each entry is an identifier for the responsible component that will remove the entry from the list. If the deletionTimestamp of the object is non-nil, entries in this list can only be removed.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"generateName\\\":{\\\"description\\\":\\\"GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server.\\\\n\\\\nIf this field is specified and the generated name exists, the server will NOT return a 409 - instead, it will either return 201 Created or 500 with Reason ServerTimeout indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header).\\\\n\\\\nApplied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#idempotency\\\",\\\"type\\\":\\\"string\\\"},\\\"generation\\\":{\\\"description\\\":\\\"A sequence number representing a specific generation of the desired state. Populated by the system. Read-only.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"initializers\\\":{\\\"description\\\":\\\"Initializers tracks the progress of initialization.\\\",\\\"properties\\\":{\\\"pending\\\":{\\\"description\\\":\\\"Pending is a list of initializers that must execute in order before this object is visible. When the last pending initializer is removed, and no failing result is set, the initializers struct will be set to nil and the object is considered as initialized and visible to all clients.\\\",\\\"items\\\":{\\\"description\\\":\\\"Initializer is information about an initializer that has not yet completed.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"name of the process that is responsible for initializing this object.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"result\\\":{\\\"description\\\":\\\"Status is a return value for calls that don't return other objects.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"code\\\":{\\\"description\\\":\\\"Suggested HTTP return code for this status, 0 if not set.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"details\\\":{\\\"description\\\":\\\"StatusDetails is a set of additional properties that MAY be set by the server to provide additional information about a response. The Reason field of a Status object defines what attributes will be set. Clients must ignore fields that do not match the defined type of each attribute, and should assume that any attribute may be empty, invalid, or under defined.\\\",\\\"properties\\\":{\\\"causes\\\":{\\\"description\\\":\\\"The Causes array includes more details associated with the StatusReason failure. Not all StatusReasons may provide detailed causes.\\\",\\\"items\\\":{\\\"description\\\":\\\"StatusCause provides more information about an api.Status failure, including cases when multiple errors are encountered.\\\",\\\"properties\\\":{\\\"field\\\":{\\\"description\\\":\\\"The field of the resource that has caused this error, as named by its JSON serialization. May include dot and postfix notation for nested attributes. Arrays are zero-indexed. Fields may appear more than once in an array of causes due to fields having multiple errors. Optional.\\\\n\\\\nExamples:\\\\n \\\\\\\"name\\\\\\\" - the field \\\\\\\"name\\\\\\\" on the current resource\\\\n \\\\\\\"items[0].name\\\\\\\" - the field \\\\\\\"name\\\\\\\" on the first array entry in \\\\\\\"items\\\\\\\"\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"A human-readable description of the cause of the error. This field may be presented as-is to a reader.\\\",\\\"type\\\":\\\"string\\\"},\\\"reason\\\":{\\\"description\\\":\\\"A machine-readable description of the cause of the error. If this value is empty there is no information available.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"group\\\":{\\\"description\\\":\\\"The group attribute of the resource associated with the status StatusReason.\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"The kind attribute of the resource associated with the status StatusReason. On some operations may differ from the requested resource Kind. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"The name attribute of the resource associated with the status StatusReason (when there is a single name which can be described).\\\",\\\"type\\\":\\\"string\\\"},\\\"retryAfterSeconds\\\":{\\\"description\\\":\\\"If specified, the time in seconds before the operation should be retried. Some errors may indicate the client must take an alternate action - for those errors this field may indicate how long to wait before taking the alternate action.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID of the resource. (when there is a single resource which can be described). More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"A human-readable description of the status of this operation.\\\",\\\"type\\\":\\\"string\\\"},\\\"metadata\\\":{\\\"description\\\":\\\"ListMeta describes metadata that synthetic resources must have, including lists and various status objects. A resource may have only one of {ObjectMeta, ListMeta}.\\\",\\\"properties\\\":{\\\"continue\\\":{\\\"description\\\":\\\"continue may be set if the user set a limit on the number of items returned, and indicates that the server has more data available. The value is opaque and may be used to issue another request to the endpoint that served this list to retrieve the next set of available objects. Continuing a consistent list may not be possible if the server configuration has changed or more than a few minutes have passed. The resourceVersion field returned when using this continue value will be identical to the value in the first response, unless you have received this token from an error message.\\\",\\\"type\\\":\\\"string\\\"},\\\"resourceVersion\\\":{\\\"description\\\":\\\"String that identifies the server's internal version of this object that can be used by clients to determine when objects have changed. Value must be treated as opaque by clients and passed unmodified back to the server. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\\\",\\\"type\\\":\\\"string\\\"},\\\"selfLink\\\":{\\\"description\\\":\\\"selfLink is a URL representing this object. Populated by the system. Read-only.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"reason\\\":{\\\"description\\\":\\\"A machine-readable description of why this operation is in the \\\\\\\"Failure\\\\\\\" status. If this value is empty there is no information available. A Reason clarifies an HTTP status code but does not override it.\\\",\\\"type\\\":\\\"string\\\"},\\\"status\\\":{\\\"description\\\":\\\"Status of the operation. One of: \\\\\\\"Success\\\\\\\" or \\\\\\\"Failure\\\\\\\". More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"required\\\":[\\\"pending\\\"],\\\"type\\\":\\\"object\\\"},\\\"labels\\\":{\\\"description\\\":\\\"Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels\\\",\\\"type\\\":\\\"object\\\"},\\\"managedFields\\\":{\\\"description\\\":\\\"ManagedFields maps workflow-id and version to the set of fields that are managed by that workflow. This is mostly for internal housekeeping, and users typically shouldn't need to set or understand this field. A workflow can be the user's name, a controller's name, or the name of a specific apply path like \\\\\\\"ci-cd\\\\\\\". The set of fields is always in the version that the workflow used when modifying the object.\\\\n\\\\nThis field is alpha and can be changed or removed without notice.\\\",\\\"items\\\":{\\\"description\\\":\\\"ManagedFieldsEntry is a workflow-id, a FieldSet and the group version of the resource that the fieldset applies to.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the version of this resource that this field set applies to. The format is \\\\\\\"group/version\\\\\\\" just like the top-level APIVersion field. It is necessary to track the version of a field set because it cannot be automatically converted.\\\",\\\"type\\\":\\\"string\\\"},\\\"fields\\\":{\\\"description\\\":\\\"Fields stores a set of fields in a data structure like a Trie. To understand how this is used, see: https://github.com/kubernetes-sigs/structured-merge-diff\\\",\\\"type\\\":\\\"object\\\"},\\\"manager\\\":{\\\"description\\\":\\\"Manager is an identifier of the workflow managing these fields.\\\",\\\"type\\\":\\\"string\\\"},\\\"operation\\\":{\\\"description\\\":\\\"Operation is the type of operation which lead to this ManagedFieldsEntry being created. The only valid values for this field are 'Apply' and 'Update'.\\\",\\\"type\\\":\\\"string\\\"},\\\"time\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names\\\",\\\"type\\\":\\\"string\\\"},\\\"namespace\\\":{\\\"description\\\":\\\"Namespace defines the space within each name must be unique. An empty namespace is equivalent to the \\\\\\\"default\\\\\\\" namespace, but \\\\\\\"default\\\\\\\" is the canonical representation. Not all objects are required to be scoped to a namespace - the value of this field for those objects will be empty.\\\\n\\\\nMust be a DNS_LABEL. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/namespaces\\\",\\\"type\\\":\\\"string\\\"},\\\"ownerReferences\\\":{\\\"description\\\":\\\"List of objects depended by this object. If ALL objects in the list have been deleted, this object will be garbage collected. If this object is managed by a controller, then an entry in this list will point to this controller, with the controller field set to true. There cannot be more than one managing controller.\\\",\\\"items\\\":{\\\"description\\\":\\\"OwnerReference contains enough information to let you identify an owning object. An owning object must be in the same namespace as the dependent, or be cluster-scoped, so there is no namespace field.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"API version of the referent.\\\",\\\"type\\\":\\\"string\\\"},\\\"blockOwnerDeletion\\\":{\\\"description\\\":\\\"If true, AND if the owner has the \\\\\\\"foregroundDeletion\\\\\\\" finalizer, then the owner cannot be deleted from the key-value store until this reference is removed. Defaults to false. To set this field, a user needs \\\\\\\"delete\\\\\\\" permission of the owner, otherwise 422 (Unprocessable Entity) will be returned.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"controller\\\":{\\\"description\\\":\\\"If true, this reference points to the managing controller.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#names\\\",\\\"type\\\":\\\"string\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"apiVersion\\\",\\\"kind\\\",\\\"name\\\",\\\"uid\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"resourceVersion\\\":{\\\"description\\\":\\\"An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources.\\\\n\\\\nPopulated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\\\",\\\"type\\\":\\\"string\\\"},\\\"selfLink\\\":{\\\"description\\\":\\\"SelfLink is a URL representing this object. Populated by the system. Read-only.\\\",\\\"type\\\":\\\"string\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID is the unique in time and space value for this object. It is typically generated by the server on successful creation of a resource and is not allowed to change on PUT operations.\\\\n\\\\nPopulated by the system. Read-only. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"spec\\\":{\\\"description\\\":\\\"PersistentVolumeClaimSpec describes the common attributes of storage devices and allows a Source for provider-specific attributes\\\",\\\"properties\\\":{\\\"accessModes\\\":{\\\"description\\\":\\\"AccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"dataSource\\\":{\\\"description\\\":\\\"TypedLocalObjectReference contains enough information to let you locate the typed referenced object inside the same namespace.\\\",\\\"properties\\\":{\\\"apiGroup\\\":{\\\"description\\\":\\\"APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is the type of resource being referenced\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name is the name of resource being referenced\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"kind\\\",\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"resources\\\":{\\\"description\\\":\\\"ResourceRequirements describes the compute resource requirements.\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"selector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"storageClassName\\\":{\\\"description\\\":\\\"Name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1\\\",\\\"type\\\":\\\"string\\\"},\\\"volumeMode\\\":{\\\"description\\\":\\\"volumeMode defines what type of volume is required by the claim. Value of Filesystem is implied when not included in claim spec. This is a beta feature.\\\",\\\"type\\\":\\\"string\\\"},\\\"volumeName\\\":{\\\"description\\\":\\\"VolumeName is the binding reference to the PersistentVolume backing this claim.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"status\\\":{\\\"description\\\":\\\"PersistentVolumeClaimStatus is the current status of a persistent volume claim.\\\",\\\"properties\\\":{\\\"accessModes\\\":{\\\"description\\\":\\\"AccessModes contains the actual access modes the volume backing the PVC has. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"capacity\\\":{\\\"description\\\":\\\"Represents the actual resources of the underlying volume.\\\",\\\"type\\\":\\\"object\\\"},\\\"conditions\\\":{\\\"description\\\":\\\"Current Condition of persistent volume claim. If underlying persistent volume is being resized then the Condition will be set to 'ResizeStarted'.\\\",\\\"items\\\":{\\\"description\\\":\\\"PersistentVolumeClaimCondition contains details about state of pvc\\\",\\\"properties\\\":{\\\"lastProbeTime\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"lastTransitionTime\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"Human-readable message indicating details about last transition.\\\",\\\"type\\\":\\\"string\\\"},\\\"reason\\\":{\\\"description\\\":\\\"Unique, this should be a short, machine understandable string that gives the reason for condition's last transition. If it reports \\\\\\\"ResizeStarted\\\\\\\" that means the underlying persistent volume is being resized.\\\",\\\"type\\\":\\\"string\\\"},\\\"status\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":{\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"type\\\",\\\"status\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"phase\\\":{\\\"description\\\":\\\"Phase represents the current phase of PersistentVolumeClaim.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"tag\\\":{\\\"description\\\":\\\"Tag of Prometheus container image to be deployed. Defaults to the value of `version`. Version is ignored if Tag is set.\\\",\\\"type\\\":\\\"string\\\"},\\\"thanos\\\":{\\\"description\\\":\\\"ThanosSpec defines parameters for a Prometheus server within a Thanos deployment.\\\",\\\"properties\\\":{\\\"baseImage\\\":{\\\"description\\\":\\\"Thanos base image if other than default.\\\",\\\"type\\\":\\\"string\\\"},\\\"image\\\":{\\\"description\\\":\\\"Image if specified has precedence over baseImage, tag and sha combinations. Specifying the version is still necessary to ensure the Prometheus Operator knows what version of Thanos is being configured.\\\",\\\"type\\\":\\\"string\\\"},\\\"objectStorageConfig\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"resources\\\":{\\\"description\\\":\\\"ResourceRequirements describes the compute resource requirements.\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"sha\\\":{\\\"description\\\":\\\"SHA of Thanos container image to be deployed. Defaults to the value of `version`. Similar to a tag, but the SHA explicitly deploys an immutable container image. Version and Tag are ignored if SHA is set.\\\",\\\"type\\\":\\\"string\\\"},\\\"tag\\\":{\\\"description\\\":\\\"Tag of Thanos sidecar container image to be deployed. Defaults to the value of `version`. Version is ignored if Tag is set.\\\",\\\"type\\\":\\\"string\\\"},\\\"version\\\":{\\\"description\\\":\\\"Version describes the version of Thanos to use.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"tolerations\\\":{\\\"description\\\":\\\"If specified, the pod's tolerations.\\\",\\\"items\\\":{\\\"description\\\":\\\"The pod this Toleration is attached to tolerates any taint that matches the triple \\\\u003ckey,value,effect\\\\u003e using the matching operator \\\\u003coperator\\\\u003e.\\\",\\\"properties\\\":{\\\"effect\\\":{\\\"description\\\":\\\"Effect indicates the taint effect to match. Empty means match all taint effects. When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.\\\",\\\"type\\\":\\\"string\\\"},\\\"key\\\":{\\\"description\\\":\\\"Key is the taint key that the toleration applies to. Empty means match all taint keys. If the key is empty, operator must be Exists; this combination means to match all values and all keys.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Operator represents a key's relationship to the value. Valid operators are Exists and Equal. Defaults to Equal. Exists is equivalent to wildcard for value, so that a pod can tolerate all taints of a particular category.\\\",\\\"type\\\":\\\"string\\\"},\\\"tolerationSeconds\\\":{\\\"description\\\":\\\"TolerationSeconds represents the period of time the toleration (which must be of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately) by the system.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"value\\\":{\\\"description\\\":\\\"Value is the taint value the toleration matches to. If the operator is Exists, the value should be empty, otherwise just a regular string.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"version\\\":{\\\"description\\\":\\\"Version of Prometheus to be deployed.\\\",\\\"type\\\":\\\"string\\\"},\\\"walCompression\\\":{\\\"description\\\":\\\"Enable compression of the write-ahead log using Snappy.\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"type\\\":\\\"object\\\"},\\\"status\\\":{\\\"description\\\":\\\"PrometheusStatus is the most recent observed status of the Prometheus cluster. Read-only. Not included when requesting from the apiserver, only from the Prometheus Operator API itself. More info: https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#spec-and-status\\\",\\\"properties\\\":{\\\"availableReplicas\\\":{\\\"description\\\":\\\"Total number of available pods (ready for at least minReadySeconds) targeted by this Prometheus deployment.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"paused\\\":{\\\"description\\\":\\\"Represents whether any actions on the underlying managed objects are being performed. Only delete actions will be performed.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"replicas\\\":{\\\"description\\\":\\\"Total number of non-terminated pods targeted by this Prometheus deployment (their labels match the selector).\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"unavailableReplicas\\\":{\\\"description\\\":\\\"Total number of unavailable pods targeted by this Prometheus deployment.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"updatedReplicas\\\":{\\\"description\\\":\\\"Total number of non-terminated pods targeted by this Prometheus deployment that have the desired version spec.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"required\\\":[\\\"paused\\\",\\\"replicas\\\",\\\"updatedReplicas\\\",\\\"availableReplicas\\\",\\\"unavailableReplicas\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"version\\\":\\\"v1\\\"}}\\n\"\n },\n \"creationTimestamp\": \"2020-05-05T16:58:10Z\",\n \"generation\": 1,\n \"labels\": {\n \"app\": \"prometheus-operator\"\n },\n \"name\": \"prometheuses.monitoring.coreos.com\",\n \"resourceVersion\": \"207497\",\n \"selfLink\": \"/apis/apiextensions.k8s.io/v1/customresourcedefinitions/prometheuses.monitoring.coreos.com\",\n \"uid\": \"472b9025-d931-4d48-8a6f-77c3b7c33f6e\"\n },\n \"spec\": {\n \"conversion\": {\n \"strategy\": \"None\"\n },\n \"group\": \"monitoring.coreos.com\",\n \"names\": {\n \"kind\": \"Prometheus\",\n \"listKind\": \"PrometheusList\",\n \"plural\": \"prometheuses\",\n \"singular\": \"prometheus\"\n },\n \"preserveUnknownFields\": true,\n \"scope\": \"Namespaced\",\n \"versions\": [\n {\n \"name\": \"v1\",\n \"schema\": {\n \"openAPIV3Schema\": {\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"spec\": {\n \"description\": \"PrometheusSpec is a specification of the desired behavior of the Prometheus cluster. More info: https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#spec-and-status\",\n \"properties\": {\n \"additionalAlertManagerConfigs\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"additionalAlertRelabelConfigs\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"additionalScrapeConfigs\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"affinity\": {\n \"description\": \"Affinity is a group of affinity scheduling rules.\",\n \"properties\": {\n \"nodeAffinity\": {\n \"description\": \"Node affinity is a group of node affinity scheduling rules.\",\n \"properties\": {\n \"preferredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\"weight\\\" to the sum if the node matches the corresponding matchExpressions; the node(s) with the highest sum are the most preferred.\",\n \"items\": {\n \"description\": \"An empty preferred scheduling term matches all objects with implicit weight 0 (i.e. it's a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op).\",\n \"properties\": {\n \"preference\": {\n \"description\": \"A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"A list of node selector requirements by node's labels.\",\n \"items\": {\n \"description\": \"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchFields\": {\n \"description\": \"A list of node selector requirements by node's fields.\",\n \"items\": {\n \"description\": \"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"weight\": {\n \"description\": \"Weight associated with matching the corresponding nodeSelectorTerm, in the range 1-100.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"weight\",\n \"preference\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"requiredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"A node selector represents the union of the results of one or more label queries over a set of nodes; that is, it represents the OR of the selectors represented by the node selector terms.\",\n \"properties\": {\n \"nodeSelectorTerms\": {\n \"description\": \"Required. A list of node selector terms. The terms are ORed.\",\n \"items\": {\n \"description\": \"A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"A list of node selector requirements by node's labels.\",\n \"items\": {\n \"description\": \"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchFields\": {\n \"description\": \"A list of node selector requirements by node's fields.\",\n \"items\": {\n \"description\": \"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"nodeSelectorTerms\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"podAffinity\": {\n \"description\": \"Pod affinity is a group of inter pod affinity scheduling rules.\",\n \"properties\": {\n \"preferredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\"weight\\\" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.\",\n \"items\": {\n \"description\": \"The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)\",\n \"properties\": {\n \"podAffinityTerm\": {\n \"description\": \"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\u003ctopologyKey\\u003e matches that of any node on which a pod of the set of pods is running\",\n \"properties\": {\n \"labelSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"namespaces\": {\n \"description\": \"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\"this pod's namespace\\\"\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"topologyKey\": {\n \"description\": \"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"topologyKey\"\n ],\n \"type\": \"object\"\n },\n \"weight\": {\n \"description\": \"weight associated with matching the corresponding podAffinityTerm, in the range 1-100.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"weight\",\n \"podAffinityTerm\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"requiredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.\",\n \"items\": {\n \"description\": \"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\u003ctopologyKey\\u003e matches that of any node on which a pod of the set of pods is running\",\n \"properties\": {\n \"labelSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"namespaces\": {\n \"description\": \"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\"this pod's namespace\\\"\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"topologyKey\": {\n \"description\": \"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"topologyKey\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"podAntiAffinity\": {\n \"description\": \"Pod anti affinity is a group of inter pod anti affinity scheduling rules.\",\n \"properties\": {\n \"preferredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"The scheduler will prefer to schedule pods to nodes that satisfy the anti-affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling anti-affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\"weight\\\" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.\",\n \"items\": {\n \"description\": \"The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)\",\n \"properties\": {\n \"podAffinityTerm\": {\n \"description\": \"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\u003ctopologyKey\\u003e matches that of any node on which a pod of the set of pods is running\",\n \"properties\": {\n \"labelSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"namespaces\": {\n \"description\": \"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\"this pod's namespace\\\"\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"topologyKey\": {\n \"description\": \"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"topologyKey\"\n ],\n \"type\": \"object\"\n },\n \"weight\": {\n \"description\": \"weight associated with matching the corresponding podAffinityTerm, in the range 1-100.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"weight\",\n \"podAffinityTerm\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"requiredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"If the anti-affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the anti-affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.\",\n \"items\": {\n \"description\": \"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\u003ctopologyKey\\u003e matches that of any node on which a pod of the set of pods is running\",\n \"properties\": {\n \"labelSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"namespaces\": {\n \"description\": \"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\"this pod's namespace\\\"\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"topologyKey\": {\n \"description\": \"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"topologyKey\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"alerting\": {\n \"description\": \"AlertingSpec defines parameters for alerting configuration of Prometheus servers.\",\n \"properties\": {\n \"alertmanagers\": {\n \"description\": \"AlertmanagerEndpoints Prometheus should fire alerts against.\",\n \"items\": {\n \"description\": \"AlertmanagerEndpoints defines a selection of a single Endpoints object containing alertmanager IPs to fire alerts against.\",\n \"properties\": {\n \"bearerTokenFile\": {\n \"description\": \"BearerTokenFile to read from filesystem to use when authenticating to Alertmanager.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of Endpoints object in Namespace.\",\n \"type\": \"string\"\n },\n \"namespace\": {\n \"description\": \"Namespace of Endpoints object.\",\n \"type\": \"string\"\n },\n \"pathPrefix\": {\n \"description\": \"Prefix for the HTTP path alerts are pushed to.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n },\n \"scheme\": {\n \"description\": \"Scheme to use when firing alerts.\",\n \"type\": \"string\"\n },\n \"tlsConfig\": {\n \"description\": \"TLSConfig specifies TLS configuration parameters.\",\n \"properties\": {\n \"caFile\": {\n \"description\": \"The CA cert to use for the targets.\",\n \"type\": \"string\"\n },\n \"certFile\": {\n \"description\": \"The client cert file for the targets.\",\n \"type\": \"string\"\n },\n \"insecureSkipVerify\": {\n \"description\": \"Disable target certificate validation.\",\n \"type\": \"boolean\"\n },\n \"keyFile\": {\n \"description\": \"The client key file for the targets.\",\n \"type\": \"string\"\n },\n \"serverName\": {\n \"description\": \"Used to verify the hostname for the targets.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"required\": [\n \"namespace\",\n \"name\",\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"alertmanagers\"\n ],\n \"type\": \"object\"\n },\n \"apiserverConfig\": {\n \"description\": \"APIServerConfig defines a host and auth methods to access apiserver. More info: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#kubernetes_sd_config\",\n \"properties\": {\n \"basicAuth\": {\n \"description\": \"BasicAuth allow an endpoint to authenticate over basic authentication More info: https://prometheus.io/docs/operating/configuration/#endpoints\",\n \"properties\": {\n \"password\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"username\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"bearerToken\": {\n \"description\": \"Bearer token for accessing apiserver.\",\n \"type\": \"string\"\n },\n \"bearerTokenFile\": {\n \"description\": \"File to read bearer token for accessing apiserver.\",\n \"type\": \"string\"\n },\n \"host\": {\n \"description\": \"Host of apiserver. A valid string consisting of a hostname or IP followed by an optional port number\",\n \"type\": \"string\"\n },\n \"tlsConfig\": {\n \"description\": \"TLSConfig specifies TLS configuration parameters.\",\n \"properties\": {\n \"caFile\": {\n \"description\": \"The CA cert to use for the targets.\",\n \"type\": \"string\"\n },\n \"certFile\": {\n \"description\": \"The client cert file for the targets.\",\n \"type\": \"string\"\n },\n \"insecureSkipVerify\": {\n \"description\": \"Disable target certificate validation.\",\n \"type\": \"boolean\"\n },\n \"keyFile\": {\n \"description\": \"The client key file for the targets.\",\n \"type\": \"string\"\n },\n \"serverName\": {\n \"description\": \"Used to verify the hostname for the targets.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"required\": [\n \"host\"\n ],\n \"type\": \"object\"\n },\n \"baseImage\": {\n \"description\": \"Base image to use for a Prometheus deployment.\",\n \"type\": \"string\"\n },\n \"configMaps\": {\n \"description\": \"ConfigMaps is a list of ConfigMaps in the same namespace as the Prometheus object, which shall be mounted into the Prometheus Pods. The ConfigMaps are mounted into /etc/prometheus/configmaps/\\u003cconfigmap-name\\u003e.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"containers\": {\n \"description\": \"Containers allows injecting additional containers or modifying operator generated containers. This can be used to allow adding an authentication proxy to a Prometheus pod or to change the behavior of an operator generated container. Containers described here modify an operator generated container if they share the same name and modifications are done via a strategic merge patch. The current container names are: `prometheus`, `prometheus-config-reloader`, `rules-configmap-reloader`, and `thanos-sidecar`. Overriding containers is entirely outside the scope of what the maintainers will support and by doing so, you accept that this behaviour may break at any time without notice.\",\n \"items\": {\n \"description\": \"A single application container that you want to run within a pod.\",\n \"properties\": {\n \"args\": {\n \"description\": \"Arguments to the entrypoint. The docker image's CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"command\": {\n \"description\": \"Entrypoint array. Not executed within a shell. The docker image's ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"env\": {\n \"description\": \"List of environment variables to set in the container. Cannot be updated.\",\n \"items\": {\n \"description\": \"EnvVar represents an environment variable present in a Container.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the environment variable. Must be a C_IDENTIFIER.\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to \\\"\\\".\",\n \"type\": \"string\"\n },\n \"valueFrom\": {\n \"description\": \"EnvVarSource represents a source for the value of an EnvVar.\",\n \"properties\": {\n \"configMapKeyRef\": {\n \"description\": \"Selects a key from a ConfigMap.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key to select.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap or it's key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"fieldRef\": {\n \"description\": \"ObjectFieldSelector selects an APIVersioned field of an object.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"Version of the schema the FieldPath is written in terms of, defaults to \\\"v1\\\".\",\n \"type\": \"string\"\n },\n \"fieldPath\": {\n \"description\": \"Path of the field to select in the specified API version.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"fieldPath\"\n ],\n \"type\": \"object\"\n },\n \"resourceFieldRef\": {\n \"description\": \"ResourceFieldSelector represents container resources (cpu, memory) and their output format\",\n \"properties\": {\n \"containerName\": {\n \"description\": \"Container name: required for volumes, optional for env vars\",\n \"type\": \"string\"\n },\n \"divisor\": {},\n \"resource\": {\n \"description\": \"Required: resource to select\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"resource\"\n ],\n \"type\": \"object\"\n },\n \"secretKeyRef\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"envFrom\": {\n \"description\": \"List of sources to populate environment variables in the container. The keys defined within a source must be a C_IDENTIFIER. All invalid keys will be reported as an event when the container is starting. When a key exists in multiple sources, the value associated with the last source will take precedence. Values defined by an Env with a duplicate key will take precedence. Cannot be updated.\",\n \"items\": {\n \"description\": \"EnvFromSource represents the source of a set of ConfigMaps\",\n \"properties\": {\n \"configMapRef\": {\n \"description\": \"ConfigMapEnvSource selects a ConfigMap to populate the environment variables with.\\n\\nThe contents of the target ConfigMap's Data field will represent the key-value pairs as environment variables.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"type\": \"object\"\n },\n \"prefix\": {\n \"description\": \"An optional identifier to prepend to each key in the ConfigMap. Must be a C_IDENTIFIER.\",\n \"type\": \"string\"\n },\n \"secretRef\": {\n \"description\": \"SecretEnvSource selects a Secret to populate the environment variables with.\\n\\nThe contents of the target Secret's Data field will represent the key-value pairs as environment variables.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"image\": {\n \"description\": \"Docker image name. More info: https://kubernetes.io/docs/concepts/containers/images This field is optional to allow higher level config management to default or override container images in workload controllers like Deployments and StatefulSets.\",\n \"type\": \"string\"\n },\n \"imagePullPolicy\": {\n \"description\": \"Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images\",\n \"type\": \"string\"\n },\n \"lifecycle\": {\n \"description\": \"Lifecycle describes actions that the management system should take in response to container lifecycle events. For the PostStart and PreStop lifecycle handlers, management of the container blocks until the action is complete, unless the container process fails, in which case the handler is aborted.\",\n \"properties\": {\n \"postStart\": {\n \"description\": \"Handler defines a specific action that should be taken\",\n \"properties\": {\n \"exec\": {\n \"description\": \"ExecAction describes a \\\"run in container\\\" action.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"httpGet\": {\n \"description\": \"HTTPGetAction describes an action based on HTTP Get requests.\",\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocketAction describes an action based on opening a socket\",\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"preStop\": {\n \"description\": \"Handler defines a specific action that should be taken\",\n \"properties\": {\n \"exec\": {\n \"description\": \"ExecAction describes a \\\"run in container\\\" action.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"httpGet\": {\n \"description\": \"HTTPGetAction describes an action based on HTTP Get requests.\",\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocketAction describes an action based on opening a socket\",\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"livenessProbe\": {\n \"description\": \"Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic.\",\n \"properties\": {\n \"exec\": {\n \"description\": \"ExecAction describes a \\\"run in container\\\" action.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"failureThreshold\": {\n \"description\": \"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"httpGet\": {\n \"description\": \"HTTPGetAction describes an action based on HTTP Get requests.\",\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"initialDelaySeconds\": {\n \"description\": \"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"periodSeconds\": {\n \"description\": \"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"successThreshold\": {\n \"description\": \"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocketAction describes an action based on opening a socket\",\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"timeoutSeconds\": {\n \"description\": \"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n },\n \"name\": {\n \"description\": \"Name of the container specified as a DNS_LABEL. Each container in a pod must have a unique name (DNS_LABEL). Cannot be updated.\",\n \"type\": \"string\"\n },\n \"ports\": {\n \"description\": \"List of ports to expose from the container. Exposing a port here gives the system additional information about the network connections a container uses, but is primarily informational. Not specifying a port here DOES NOT prevent that port from being exposed. Any port which is listening on the default \\\"0.0.0.0\\\" address inside a container will be accessible from the network. Cannot be updated.\",\n \"items\": {\n \"description\": \"ContainerPort represents a network port in a single container.\",\n \"properties\": {\n \"containerPort\": {\n \"description\": \"Number of port to expose on the pod's IP address. This must be a valid port number, 0 \\u003c x \\u003c 65536.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"hostIP\": {\n \"description\": \"What host IP to bind the external port to.\",\n \"type\": \"string\"\n },\n \"hostPort\": {\n \"description\": \"Number of port to expose on the host. If specified, this must be a valid port number, 0 \\u003c x \\u003c 65536. If HostNetwork is specified, this must match ContainerPort. Most containers do not need this.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"name\": {\n \"description\": \"If specified, this must be an IANA_SVC_NAME and unique within the pod. Each named port in a pod must have a unique name. Name for the port that can be referred to by services.\",\n \"type\": \"string\"\n },\n \"protocol\": {\n \"description\": \"Protocol for port. Must be UDP, TCP, or SCTP. Defaults to \\\"TCP\\\".\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"containerPort\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"readinessProbe\": {\n \"description\": \"Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic.\",\n \"properties\": {\n \"exec\": {\n \"description\": \"ExecAction describes a \\\"run in container\\\" action.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"failureThreshold\": {\n \"description\": \"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"httpGet\": {\n \"description\": \"HTTPGetAction describes an action based on HTTP Get requests.\",\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"initialDelaySeconds\": {\n \"description\": \"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"periodSeconds\": {\n \"description\": \"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"successThreshold\": {\n \"description\": \"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocketAction describes an action based on opening a socket\",\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n }\n },\n \"required\": [\n \"port\"\n ],\n \"type\": \"object\"\n },\n \"timeoutSeconds\": {\n \"description\": \"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n },\n \"resources\": {\n \"description\": \"ResourceRequirements describes the compute resource requirements.\",\n \"properties\": {\n \"limits\": {\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"securityContext\": {\n \"description\": \"SecurityContext holds security configuration that will be applied to a container. Some fields are present in both SecurityContext and PodSecurityContext. When both are set, the values in SecurityContext take precedence.\",\n \"properties\": {\n \"allowPrivilegeEscalation\": {\n \"description\": \"AllowPrivilegeEscalation controls whether a process can gain more privileges than its parent process. This bool directly controls if the no_new_privs flag will be set on the container process. AllowPrivilegeEscalation is true always when the container is: 1) run as Privileged 2) has CAP_SYS_ADMIN\",\n \"type\": \"boolean\"\n },\n \"capabilities\": {\n \"description\": \"Adds and removes POSIX capabilities from running containers.\",\n \"properties\": {\n \"add\": {\n \"description\": \"Added capabilities\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"drop\": {\n \"description\": \"Removed capabilities\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"privileged\": {\n \"description\": \"Run container in privileged mode. Processes in privileged containers are essentially equivalent to root on the host. Defaults to false.\",\n \"type\": \"boolean\"\n },\n \"procMount\": {\n \"description\": \"procMount denotes the type of proc mount to use for the containers. The default is DefaultProcMount which uses the container runtime defaults for readonly paths and masked paths. This requires the ProcMountType feature flag to be enabled.\",\n \"type\": \"string\"\n },\n \"readOnlyRootFilesystem\": {\n \"description\": \"Whether this container has a read-only root filesystem. Default is false.\",\n \"type\": \"boolean\"\n },\n \"runAsGroup\": {\n \"description\": \"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"runAsNonRoot\": {\n \"description\": \"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"type\": \"boolean\"\n },\n \"runAsUser\": {\n \"description\": \"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"seLinuxOptions\": {\n \"description\": \"SELinuxOptions are the labels to be applied to the container\",\n \"properties\": {\n \"level\": {\n \"description\": \"Level is SELinux level label that applies to the container.\",\n \"type\": \"string\"\n },\n \"role\": {\n \"description\": \"Role is a SELinux role label that applies to the container.\",\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"Type is a SELinux type label that applies to the container.\",\n \"type\": \"string\"\n },\n \"user\": {\n \"description\": \"User is a SELinux user label that applies to the container.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"stdin\": {\n \"description\": \"Whether this container should allocate a buffer for stdin in the container runtime. If this is not set, reads from stdin in the container will always result in EOF. Default is false.\",\n \"type\": \"boolean\"\n },\n \"stdinOnce\": {\n \"description\": \"Whether the container runtime should close the stdin channel after it has been opened by a single attach. When stdin is true the stdin stream will remain open across multiple attach sessions. If stdinOnce is set to true, stdin is opened on container start, is empty until the first client attaches to stdin, and then remains open and accepts data until the client disconnects, at which time stdin is closed and remains closed until the container is restarted. If this flag is false, a container processes that reads from stdin will never receive an EOF. Default is false\",\n \"type\": \"boolean\"\n },\n \"terminationMessagePath\": {\n \"description\": \"Optional: Path at which the file to which the container's termination message will be written is mounted into the container's filesystem. Message written is intended to be brief final status, such as an assertion failure message. Will be truncated by the node if greater than 4096 bytes. The total message length across all containers will be limited to 12kb. Defaults to /dev/termination-log. Cannot be updated.\",\n \"type\": \"string\"\n },\n \"terminationMessagePolicy\": {\n \"description\": \"Indicate how the termination message should be populated. File will use the contents of terminationMessagePath to populate the container status message on both success and failure. FallbackToLogsOnError will use the last chunk of container log output if the termination message file is empty and the container exited with an error. The log output is limited to 2048 bytes or 80 lines, whichever is smaller. Defaults to File. Cannot be updated.\",\n \"type\": \"string\"\n },\n \"tty\": {\n \"description\": \"Whether this container should allocate a TTY for itself, also requires 'stdin' to be true. Default is false.\",\n \"type\": \"boolean\"\n },\n \"volumeDevices\": {\n \"description\": \"volumeDevices is the list of block devices to be used by the container. This is a beta feature.\",\n \"items\": {\n \"description\": \"volumeDevice describes a mapping of a raw block device within a container.\",\n \"properties\": {\n \"devicePath\": {\n \"description\": \"devicePath is the path inside of the container that the device will be mapped to.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"name must match the name of a persistentVolumeClaim in the pod\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"devicePath\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"volumeMounts\": {\n \"description\": \"Pod volumes to mount into the container's filesystem. Cannot be updated.\",\n \"items\": {\n \"description\": \"VolumeMount describes a mounting of a Volume within a container.\",\n \"properties\": {\n \"mountPath\": {\n \"description\": \"Path within the container at which the volume should be mounted. Must not contain ':'.\",\n \"type\": \"string\"\n },\n \"mountPropagation\": {\n \"description\": \"mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"This must match the Name of a Volume.\",\n \"type\": \"string\"\n },\n \"readOnly\": {\n \"description\": \"Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.\",\n \"type\": \"boolean\"\n },\n \"subPath\": {\n \"description\": \"Path within the volume from which the container's volume should be mounted. Defaults to \\\"\\\" (volume's root).\",\n \"type\": \"string\"\n },\n \"subPathExpr\": {\n \"description\": \"Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to \\\"\\\" (volume's root). SubPathExpr and SubPath are mutually exclusive. This field is alpha in 1.14.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"mountPath\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"workingDir\": {\n \"description\": \"Container's working directory. If not specified, the container runtime's default will be used, which might be configured in the container image. Cannot be updated.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"enableAdminAPI\": {\n \"description\": \"Enable access to prometheus web admin API. Defaults to the value of `false`. WARNING: Enabling the admin APIs enables mutating endpoints, to delete data, shutdown Prometheus, and more. Enabling this should be done with care and the user is advised to add additional authentication authorization via a proxy to ensure only clients authorized to perform these actions can do so. For more information see https://prometheus.io/docs/prometheus/latest/querying/api/#tsdb-admin-apis\",\n \"type\": \"boolean\"\n },\n \"evaluationInterval\": {\n \"description\": \"Interval between consecutive evaluations.\",\n \"type\": \"string\"\n },\n \"externalLabels\": {\n \"description\": \"The labels to add to any time series or alerts when communicating with external systems (federation, remote storage, Alertmanager).\",\n \"type\": \"object\"\n },\n \"externalUrl\": {\n \"description\": \"The external URL the Prometheus instances will be available under. This is necessary to generate correct URLs. This is necessary if Prometheus is not served from root of a DNS name.\",\n \"type\": \"string\"\n },\n \"image\": {\n \"description\": \"Image if specified has precedence over baseImage, tag and sha combinations. Specifying the version is still necessary to ensure the Prometheus Operator knows what version of Prometheus is being configured.\",\n \"type\": \"string\"\n },\n \"imagePullSecrets\": {\n \"description\": \"An optional list of references to secrets in the same namespace to use for pulling prometheus and alertmanager images from registries see http://kubernetes.io/docs/user-guide/images#specifying-imagepullsecrets-on-a-pod\",\n \"items\": {\n \"description\": \"LocalObjectReference contains enough information to let you locate the referenced object inside the same namespace.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"listenLocal\": {\n \"description\": \"ListenLocal makes the Prometheus server listen on loopback, so that it does not bind against the Pod IP.\",\n \"type\": \"boolean\"\n },\n \"logFormat\": {\n \"description\": \"Log format for Prometheus to be configured with.\",\n \"type\": \"string\"\n },\n \"logLevel\": {\n \"description\": \"Log level for Prometheus to be configured with.\",\n \"type\": \"string\"\n },\n \"nodeSelector\": {\n \"description\": \"Define which Nodes the Pods are scheduled on.\",\n \"type\": \"object\"\n },\n \"paused\": {\n \"description\": \"When a Prometheus deployment is paused, no actions except for deletion will be performed on the underlying objects.\",\n \"type\": \"boolean\"\n },\n \"podMetadata\": {\n \"description\": \"ObjectMeta is metadata that all persisted resources must have, which includes all objects users must create.\",\n \"properties\": {\n \"annotations\": {\n \"description\": \"Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations\",\n \"type\": \"object\"\n },\n \"clusterName\": {\n \"description\": \"The name of the cluster which the object belongs to. This is used to distinguish resources with same name and namespace in different clusters. This field is not set anywhere right now and apiserver is going to ignore it if set in create or update request.\",\n \"type\": \"string\"\n },\n \"creationTimestamp\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n },\n \"deletionGracePeriodSeconds\": {\n \"description\": \"Number of seconds allowed for this object to gracefully terminate before it will be removed from the system. Only set when deletionTimestamp is also set. May only be shortened. Read-only.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"deletionTimestamp\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n },\n \"finalizers\": {\n \"description\": \"Must be empty before the object is deleted from the registry. Each entry is an identifier for the responsible component that will remove the entry from the list. If the deletionTimestamp of the object is non-nil, entries in this list can only be removed.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"generateName\": {\n \"description\": \"GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server.\\n\\nIf this field is specified and the generated name exists, the server will NOT return a 409 - instead, it will either return 201 Created or 500 with Reason ServerTimeout indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header).\\n\\nApplied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#idempotency\",\n \"type\": \"string\"\n },\n \"generation\": {\n \"description\": \"A sequence number representing a specific generation of the desired state. Populated by the system. Read-only.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"initializers\": {\n \"description\": \"Initializers tracks the progress of initialization.\",\n \"properties\": {\n \"pending\": {\n \"description\": \"Pending is a list of initializers that must execute in order before this object is visible. When the last pending initializer is removed, and no failing result is set, the initializers struct will be set to nil and the object is considered as initialized and visible to all clients.\",\n \"items\": {\n \"description\": \"Initializer is information about an initializer that has not yet completed.\",\n \"properties\": {\n \"name\": {\n \"description\": \"name of the process that is responsible for initializing this object.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"result\": {\n \"description\": \"Status is a return value for calls that don't return other objects.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"code\": {\n \"description\": \"Suggested HTTP return code for this status, 0 if not set.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"details\": {\n \"description\": \"StatusDetails is a set of additional properties that MAY be set by the server to provide additional information about a response. The Reason field of a Status object defines what attributes will be set. Clients must ignore fields that do not match the defined type of each attribute, and should assume that any attribute may be empty, invalid, or under defined.\",\n \"properties\": {\n \"causes\": {\n \"description\": \"The Causes array includes more details associated with the StatusReason failure. Not all StatusReasons may provide detailed causes.\",\n \"items\": {\n \"description\": \"StatusCause provides more information about an api.Status failure, including cases when multiple errors are encountered.\",\n \"properties\": {\n \"field\": {\n \"description\": \"The field of the resource that has caused this error, as named by its JSON serialization. May include dot and postfix notation for nested attributes. Arrays are zero-indexed. Fields may appear more than once in an array of causes due to fields having multiple errors. Optional.\\n\\nExamples:\\n \\\"name\\\" - the field \\\"name\\\" on the current resource\\n \\\"items[0].name\\\" - the field \\\"name\\\" on the first array entry in \\\"items\\\"\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"A human-readable description of the cause of the error. This field may be presented as-is to a reader.\",\n \"type\": \"string\"\n },\n \"reason\": {\n \"description\": \"A machine-readable description of the cause of the error. If this value is empty there is no information available.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"group\": {\n \"description\": \"The group attribute of the resource associated with the status StatusReason.\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"The kind attribute of the resource associated with the status StatusReason. On some operations may differ from the requested resource Kind. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"The name attribute of the resource associated with the status StatusReason (when there is a single name which can be described).\",\n \"type\": \"string\"\n },\n \"retryAfterSeconds\": {\n \"description\": \"If specified, the time in seconds before the operation should be retried. Some errors may indicate the client must take an alternate action - for those errors this field may indicate how long to wait before taking the alternate action.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"uid\": {\n \"description\": \"UID of the resource. (when there is a single resource which can be described). More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"A human-readable description of the status of this operation.\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"description\": \"ListMeta describes metadata that synthetic resources must have, including lists and various status objects. A resource may have only one of {ObjectMeta, ListMeta}.\",\n \"properties\": {\n \"continue\": {\n \"description\": \"continue may be set if the user set a limit on the number of items returned, and indicates that the server has more data available. The value is opaque and may be used to issue another request to the endpoint that served this list to retrieve the next set of available objects. Continuing a consistent list may not be possible if the server configuration has changed or more than a few minutes have passed. The resourceVersion field returned when using this continue value will be identical to the value in the first response, unless you have received this token from an error message.\",\n \"type\": \"string\"\n },\n \"resourceVersion\": {\n \"description\": \"String that identifies the server's internal version of this object that can be used by clients to determine when objects have changed. Value must be treated as opaque by clients and passed unmodified back to the server. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\",\n \"type\": \"string\"\n },\n \"selfLink\": {\n \"description\": \"selfLink is a URL representing this object. Populated by the system. Read-only.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"reason\": {\n \"description\": \"A machine-readable description of why this operation is in the \\\"Failure\\\" status. If this value is empty there is no information available. A Reason clarifies an HTTP status code but does not override it.\",\n \"type\": \"string\"\n },\n \"status\": {\n \"description\": \"Status of the operation. One of: \\\"Success\\\" or \\\"Failure\\\". More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"required\": [\n \"pending\"\n ],\n \"type\": \"object\"\n },\n \"labels\": {\n \"description\": \"Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels\",\n \"type\": \"object\"\n },\n \"managedFields\": {\n \"description\": \"ManagedFields maps workflow-id and version to the set of fields that are managed by that workflow. This is mostly for internal housekeeping, and users typically shouldn't need to set or understand this field. A workflow can be the user's name, a controller's name, or the name of a specific apply path like \\\"ci-cd\\\". The set of fields is always in the version that the workflow used when modifying the object.\\n\\nThis field is alpha and can be changed or removed without notice.\",\n \"items\": {\n \"description\": \"ManagedFieldsEntry is a workflow-id, a FieldSet and the group version of the resource that the fieldset applies to.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the version of this resource that this field set applies to. The format is \\\"group/version\\\" just like the top-level APIVersion field. It is necessary to track the version of a field set because it cannot be automatically converted.\",\n \"type\": \"string\"\n },\n \"fields\": {\n \"description\": \"Fields stores a set of fields in a data structure like a Trie. To understand how this is used, see: https://github.com/kubernetes-sigs/structured-merge-diff\",\n \"type\": \"object\"\n },\n \"manager\": {\n \"description\": \"Manager is an identifier of the workflow managing these fields.\",\n \"type\": \"string\"\n },\n \"operation\": {\n \"description\": \"Operation is the type of operation which lead to this ManagedFieldsEntry being created. The only valid values for this field are 'Apply' and 'Update'.\",\n \"type\": \"string\"\n },\n \"time\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"name\": {\n \"description\": \"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names\",\n \"type\": \"string\"\n },\n \"namespace\": {\n \"description\": \"Namespace defines the space within each name must be unique. An empty namespace is equivalent to the \\\"default\\\" namespace, but \\\"default\\\" is the canonical representation. Not all objects are required to be scoped to a namespace - the value of this field for those objects will be empty.\\n\\nMust be a DNS_LABEL. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/namespaces\",\n \"type\": \"string\"\n },\n \"ownerReferences\": {\n \"description\": \"List of objects depended by this object. If ALL objects in the list have been deleted, this object will be garbage collected. If this object is managed by a controller, then an entry in this list will point to this controller, with the controller field set to true. There cannot be more than one managing controller.\",\n \"items\": {\n \"description\": \"OwnerReference contains enough information to let you identify an owning object. An owning object must be in the same namespace as the dependent, or be cluster-scoped, so there is no namespace field.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"API version of the referent.\",\n \"type\": \"string\"\n },\n \"blockOwnerDeletion\": {\n \"description\": \"If true, AND if the owner has the \\\"foregroundDeletion\\\" finalizer, then the owner cannot be deleted from the key-value store until this reference is removed. Defaults to false. To set this field, a user needs \\\"delete\\\" permission of the owner, otherwise 422 (Unprocessable Entity) will be returned.\",\n \"type\": \"boolean\"\n },\n \"controller\": {\n \"description\": \"If true, this reference points to the managing controller.\",\n \"type\": \"boolean\"\n },\n \"kind\": {\n \"description\": \"Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#names\",\n \"type\": \"string\"\n },\n \"uid\": {\n \"description\": \"UID of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"apiVersion\",\n \"kind\",\n \"name\",\n \"uid\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"resourceVersion\": {\n \"description\": \"An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources.\\n\\nPopulated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\",\n \"type\": \"string\"\n },\n \"selfLink\": {\n \"description\": \"SelfLink is a URL representing this object. Populated by the system. Read-only.\",\n \"type\": \"string\"\n },\n \"uid\": {\n \"description\": \"UID is the unique in time and space value for this object. It is typically generated by the server on successful creation of a resource and is not allowed to change on PUT operations.\\n\\nPopulated by the system. Read-only. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"podMonitorNamespaceSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"podMonitorSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"priorityClassName\": {\n \"description\": \"Priority class assigned to the Pods\",\n \"type\": \"string\"\n },\n \"prometheusExternalLabelName\": {\n \"description\": \"Name of Prometheus external label used to denote Prometheus instance name. Defaults to the value of `prometheus`. External label will _not_ be added when value is set to empty string (`\\\"\\\"`).\",\n \"type\": \"string\"\n },\n \"query\": {\n \"description\": \"QuerySpec defines the query command line flags when starting Prometheus.\",\n \"properties\": {\n \"lookbackDelta\": {\n \"description\": \"The delta difference allowed for retrieving metrics during expression evaluations.\",\n \"type\": \"string\"\n },\n \"maxConcurrency\": {\n \"description\": \"Number of concurrent queries that can be run at once.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"maxSamples\": {\n \"description\": \"Maximum number of samples a single query can load into memory. Note that queries will fail if they would load more samples than this into memory, so this also limits the number of samples a query can return.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"timeout\": {\n \"description\": \"Maximum time a query may take before being aborted.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"remoteRead\": {\n \"description\": \"If specified, the remote_read spec. This is an experimental feature, it may change in any upcoming release in a breaking way.\",\n \"items\": {\n \"description\": \"RemoteReadSpec defines the remote_read configuration for prometheus.\",\n \"properties\": {\n \"basicAuth\": {\n \"description\": \"BasicAuth allow an endpoint to authenticate over basic authentication More info: https://prometheus.io/docs/operating/configuration/#endpoints\",\n \"properties\": {\n \"password\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"username\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"bearerToken\": {\n \"description\": \"bearer token for remote read.\",\n \"type\": \"string\"\n },\n \"bearerTokenFile\": {\n \"description\": \"File to read bearer token for remote read.\",\n \"type\": \"string\"\n },\n \"proxyUrl\": {\n \"description\": \"Optional ProxyURL\",\n \"type\": \"string\"\n },\n \"readRecent\": {\n \"description\": \"Whether reads should be made for queries for time ranges that the local storage should have complete data for.\",\n \"type\": \"boolean\"\n },\n \"remoteTimeout\": {\n \"description\": \"Timeout for requests to the remote read endpoint.\",\n \"type\": \"string\"\n },\n \"requiredMatchers\": {\n \"description\": \"An optional list of equality matchers which have to be present in a selector to query the remote read endpoint.\",\n \"type\": \"object\"\n },\n \"tlsConfig\": {\n \"description\": \"TLSConfig specifies TLS configuration parameters.\",\n \"properties\": {\n \"caFile\": {\n \"description\": \"The CA cert to use for the targets.\",\n \"type\": \"string\"\n },\n \"certFile\": {\n \"description\": \"The client cert file for the targets.\",\n \"type\": \"string\"\n },\n \"insecureSkipVerify\": {\n \"description\": \"Disable target certificate validation.\",\n \"type\": \"boolean\"\n },\n \"keyFile\": {\n \"description\": \"The client key file for the targets.\",\n \"type\": \"string\"\n },\n \"serverName\": {\n \"description\": \"Used to verify the hostname for the targets.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"url\": {\n \"description\": \"The URL of the endpoint to send samples to.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"url\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"remoteWrite\": {\n \"description\": \"If specified, the remote_write spec. This is an experimental feature, it may change in any upcoming release in a breaking way.\",\n \"items\": {\n \"description\": \"RemoteWriteSpec defines the remote_write configuration for prometheus.\",\n \"properties\": {\n \"basicAuth\": {\n \"description\": \"BasicAuth allow an endpoint to authenticate over basic authentication More info: https://prometheus.io/docs/operating/configuration/#endpoints\",\n \"properties\": {\n \"password\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"username\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"bearerToken\": {\n \"description\": \"File to read bearer token for remote write.\",\n \"type\": \"string\"\n },\n \"bearerTokenFile\": {\n \"description\": \"File to read bearer token for remote write.\",\n \"type\": \"string\"\n },\n \"proxyUrl\": {\n \"description\": \"Optional ProxyURL\",\n \"type\": \"string\"\n },\n \"queueConfig\": {\n \"description\": \"QueueConfig allows the tuning of remote_write queue_config parameters. This object is referenced in the RemoteWriteSpec object.\",\n \"properties\": {\n \"batchSendDeadline\": {\n \"description\": \"BatchSendDeadline is the maximum time a sample will wait in buffer.\",\n \"type\": \"string\"\n },\n \"capacity\": {\n \"description\": \"Capacity is the number of samples to buffer per shard before we start dropping them.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"maxBackoff\": {\n \"description\": \"MaxBackoff is the maximum retry delay.\",\n \"type\": \"string\"\n },\n \"maxRetries\": {\n \"description\": \"MaxRetries is the maximum number of times to retry a batch on recoverable errors.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"maxSamplesPerSend\": {\n \"description\": \"MaxSamplesPerSend is the maximum number of samples per send.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"maxShards\": {\n \"description\": \"MaxShards is the maximum number of shards, i.e. amount of concurrency.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"minBackoff\": {\n \"description\": \"MinBackoff is the initial retry delay. Gets doubled for every retry.\",\n \"type\": \"string\"\n },\n \"minShards\": {\n \"description\": \"MinShards is the minimum number of shards, i.e. amount of concurrency.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"type\": \"object\"\n },\n \"remoteTimeout\": {\n \"description\": \"Timeout for requests to the remote write endpoint.\",\n \"type\": \"string\"\n },\n \"tlsConfig\": {\n \"description\": \"TLSConfig specifies TLS configuration parameters.\",\n \"properties\": {\n \"caFile\": {\n \"description\": \"The CA cert to use for the targets.\",\n \"type\": \"string\"\n },\n \"certFile\": {\n \"description\": \"The client cert file for the targets.\",\n \"type\": \"string\"\n },\n \"insecureSkipVerify\": {\n \"description\": \"Disable target certificate validation.\",\n \"type\": \"boolean\"\n },\n \"keyFile\": {\n \"description\": \"The client key file for the targets.\",\n \"type\": \"string\"\n },\n \"serverName\": {\n \"description\": \"Used to verify the hostname for the targets.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"url\": {\n \"description\": \"The URL of the endpoint to send samples to.\",\n \"type\": \"string\"\n },\n \"writeRelabelConfigs\": {\n \"description\": \"The list of remote write relabel configurations.\",\n \"items\": {\n \"description\": \"RelabelConfig allows dynamic rewriting of the label set, being applied to samples before ingestion. It defines `\\u003cmetric_relabel_configs\\u003e`-section of Prometheus configuration. More info: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#metric_relabel_configs\",\n \"properties\": {\n \"action\": {\n \"description\": \"Action to perform based on regex matching. Default is 'replace'\",\n \"type\": \"string\"\n },\n \"modulus\": {\n \"description\": \"Modulus to take of the hash of the source label values.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"regex\": {\n \"description\": \"Regular expression against which the extracted value is matched. default is '(.*)'\",\n \"type\": \"string\"\n },\n \"replacement\": {\n \"description\": \"Replacement value against which a regex replace is performed if the regular expression matches. Regex capture groups are available. Default is '$1'\",\n \"type\": \"string\"\n },\n \"separator\": {\n \"description\": \"Separator placed between concatenated source label values. default is ';'.\",\n \"type\": \"string\"\n },\n \"sourceLabels\": {\n \"description\": \"The source labels select values from existing labels. Their content is concatenated using the configured separator and matched against the configured regular expression for the replace, keep, and drop actions.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"targetLabel\": {\n \"description\": \"Label to which the resulting value is written in a replace action. It is mandatory for replace actions. Regex capture groups are available.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"url\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"replicaExternalLabelName\": {\n \"description\": \"Name of Prometheus external label used to denote replica name. Defaults to the value of `prometheus_replica`. External label will _not_ be added when value is set to empty string (`\\\"\\\"`).\",\n \"type\": \"string\"\n },\n \"replicas\": {\n \"description\": \"Number of instances to deploy for a Prometheus deployment.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"resources\": {\n \"description\": \"ResourceRequirements describes the compute resource requirements.\",\n \"properties\": {\n \"limits\": {\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"retention\": {\n \"description\": \"Time duration Prometheus shall retain data for. Default is '24h', and must match the regular expression `[0-9]+(ms|s|m|h|d|w|y)` (milliseconds seconds minutes hours days weeks years).\",\n \"type\": \"string\"\n },\n \"retentionSize\": {\n \"description\": \"Maximum amount of disk space used by blocks.\",\n \"type\": \"string\"\n },\n \"routePrefix\": {\n \"description\": \"The route prefix Prometheus registers HTTP handlers for. This is useful, if using ExternalURL and a proxy is rewriting HTTP routes of a request, and the actual ExternalURL is still true, but the server serves requests under a different route prefix. For example for use with `kubectl proxy`.\",\n \"type\": \"string\"\n },\n \"ruleNamespaceSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"ruleSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"rules\": {\n \"description\": \"/--rules.*/ command-line arguments\",\n \"properties\": {\n \"alert\": {\n \"description\": \"/--rules.alert.*/ command-line arguments\",\n \"properties\": {\n \"forGracePeriod\": {\n \"description\": \"Minimum duration between alert and restored 'for' state. This is maintained only for alerts with configured 'for' time greater than grace period.\",\n \"type\": \"string\"\n },\n \"forOutageTolerance\": {\n \"description\": \"Max time to tolerate prometheus outage for restoring 'for' state of alert.\",\n \"type\": \"string\"\n },\n \"resendDelay\": {\n \"description\": \"Minimum amount of time to wait before resending an alert to Alertmanager.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"scrapeInterval\": {\n \"description\": \"Interval between consecutive scrapes.\",\n \"type\": \"string\"\n },\n \"secrets\": {\n \"description\": \"Secrets is a list of Secrets in the same namespace as the Prometheus object, which shall be mounted into the Prometheus Pods. The Secrets are mounted into /etc/prometheus/secrets/\\u003csecret-name\\u003e.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"securityContext\": {\n \"description\": \"PodSecurityContext holds pod-level security attributes and common container settings. Some fields are also present in container.securityContext. Field values of container.securityContext take precedence over field values of PodSecurityContext.\",\n \"properties\": {\n \"fsGroup\": {\n \"description\": \"A special supplemental group that applies to all containers in a pod. Some volume types allow the Kubelet to change the ownership of that volume to be owned by the pod:\\n\\n1. The owning GID will be the FSGroup 2. The setgid bit is set (new files created in the volume will be owned by FSGroup) 3. The permission bits are OR'd with rw-rw----\\n\\nIf unset, the Kubelet will not modify the ownership and permissions of any volume.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"runAsGroup\": {\n \"description\": \"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"runAsNonRoot\": {\n \"description\": \"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"type\": \"boolean\"\n },\n \"runAsUser\": {\n \"description\": \"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"seLinuxOptions\": {\n \"description\": \"SELinuxOptions are the labels to be applied to the container\",\n \"properties\": {\n \"level\": {\n \"description\": \"Level is SELinux level label that applies to the container.\",\n \"type\": \"string\"\n },\n \"role\": {\n \"description\": \"Role is a SELinux role label that applies to the container.\",\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"Type is a SELinux type label that applies to the container.\",\n \"type\": \"string\"\n },\n \"user\": {\n \"description\": \"User is a SELinux user label that applies to the container.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"supplementalGroups\": {\n \"description\": \"A list of groups applied to the first process run in each container, in addition to the container's primary GID. If unspecified, no groups will be added to any container.\",\n \"items\": {\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"type\": \"array\"\n },\n \"sysctls\": {\n \"description\": \"Sysctls hold a list of namespaced sysctls used for the pod. Pods with unsupported sysctls (by the container runtime) might fail to launch.\",\n \"items\": {\n \"description\": \"Sysctl defines a kernel parameter to be set\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of a property to set\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"Value of a property to set\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n }\n },\n \"type\": \"object\"\n },\n \"serviceAccountName\": {\n \"description\": \"ServiceAccountName is the name of the ServiceAccount to use to run the Prometheus Pods.\",\n \"type\": \"string\"\n },\n \"serviceMonitorNamespaceSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"serviceMonitorSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"sha\": {\n \"description\": \"SHA of Prometheus container image to be deployed. Defaults to the value of `version`. Similar to a tag, but the SHA explicitly deploys an immutable container image. Version and Tag are ignored if SHA is set.\",\n \"type\": \"string\"\n },\n \"storage\": {\n \"description\": \"StorageSpec defines the configured storage for a group Prometheus servers. If neither `emptyDir` nor `volumeClaimTemplate` is specified, then by default an [EmptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir) will be used.\",\n \"properties\": {\n \"emptyDir\": {\n \"description\": \"Represents an empty directory for a pod. Empty directory volumes support ownership management and SELinux relabeling.\",\n \"properties\": {\n \"medium\": {\n \"description\": \"What type of storage medium should back this directory. The default is \\\"\\\" which means to use the node's default medium. Must be an empty string (default) or Memory. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir\",\n \"type\": \"string\"\n },\n \"sizeLimit\": {}\n },\n \"type\": \"object\"\n },\n \"volumeClaimTemplate\": {\n \"description\": \"PersistentVolumeClaim is a user's request for and claim to a persistent volume\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"description\": \"ObjectMeta is metadata that all persisted resources must have, which includes all objects users must create.\",\n \"properties\": {\n \"annotations\": {\n \"description\": \"Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations\",\n \"type\": \"object\"\n },\n \"clusterName\": {\n \"description\": \"The name of the cluster which the object belongs to. This is used to distinguish resources with same name and namespace in different clusters. This field is not set anywhere right now and apiserver is going to ignore it if set in create or update request.\",\n \"type\": \"string\"\n },\n \"creationTimestamp\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n },\n \"deletionGracePeriodSeconds\": {\n \"description\": \"Number of seconds allowed for this object to gracefully terminate before it will be removed from the system. Only set when deletionTimestamp is also set. May only be shortened. Read-only.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"deletionTimestamp\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n },\n \"finalizers\": {\n \"description\": \"Must be empty before the object is deleted from the registry. Each entry is an identifier for the responsible component that will remove the entry from the list. If the deletionTimestamp of the object is non-nil, entries in this list can only be removed.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"generateName\": {\n \"description\": \"GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server.\\n\\nIf this field is specified and the generated name exists, the server will NOT return a 409 - instead, it will either return 201 Created or 500 with Reason ServerTimeout indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header).\\n\\nApplied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#idempotency\",\n \"type\": \"string\"\n },\n \"generation\": {\n \"description\": \"A sequence number representing a specific generation of the desired state. Populated by the system. Read-only.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"initializers\": {\n \"description\": \"Initializers tracks the progress of initialization.\",\n \"properties\": {\n \"pending\": {\n \"description\": \"Pending is a list of initializers that must execute in order before this object is visible. When the last pending initializer is removed, and no failing result is set, the initializers struct will be set to nil and the object is considered as initialized and visible to all clients.\",\n \"items\": {\n \"description\": \"Initializer is information about an initializer that has not yet completed.\",\n \"properties\": {\n \"name\": {\n \"description\": \"name of the process that is responsible for initializing this object.\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"result\": {\n \"description\": \"Status is a return value for calls that don't return other objects.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"code\": {\n \"description\": \"Suggested HTTP return code for this status, 0 if not set.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"details\": {\n \"description\": \"StatusDetails is a set of additional properties that MAY be set by the server to provide additional information about a response. The Reason field of a Status object defines what attributes will be set. Clients must ignore fields that do not match the defined type of each attribute, and should assume that any attribute may be empty, invalid, or under defined.\",\n \"properties\": {\n \"causes\": {\n \"description\": \"The Causes array includes more details associated with the StatusReason failure. Not all StatusReasons may provide detailed causes.\",\n \"items\": {\n \"description\": \"StatusCause provides more information about an api.Status failure, including cases when multiple errors are encountered.\",\n \"properties\": {\n \"field\": {\n \"description\": \"The field of the resource that has caused this error, as named by its JSON serialization. May include dot and postfix notation for nested attributes. Arrays are zero-indexed. Fields may appear more than once in an array of causes due to fields having multiple errors. Optional.\\n\\nExamples:\\n \\\"name\\\" - the field \\\"name\\\" on the current resource\\n \\\"items[0].name\\\" - the field \\\"name\\\" on the first array entry in \\\"items\\\"\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"A human-readable description of the cause of the error. This field may be presented as-is to a reader.\",\n \"type\": \"string\"\n },\n \"reason\": {\n \"description\": \"A machine-readable description of the cause of the error. If this value is empty there is no information available.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"group\": {\n \"description\": \"The group attribute of the resource associated with the status StatusReason.\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"The kind attribute of the resource associated with the status StatusReason. On some operations may differ from the requested resource Kind. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"The name attribute of the resource associated with the status StatusReason (when there is a single name which can be described).\",\n \"type\": \"string\"\n },\n \"retryAfterSeconds\": {\n \"description\": \"If specified, the time in seconds before the operation should be retried. Some errors may indicate the client must take an alternate action - for those errors this field may indicate how long to wait before taking the alternate action.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"uid\": {\n \"description\": \"UID of the resource. (when there is a single resource which can be described). More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"A human-readable description of the status of this operation.\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"description\": \"ListMeta describes metadata that synthetic resources must have, including lists and various status objects. A resource may have only one of {ObjectMeta, ListMeta}.\",\n \"properties\": {\n \"continue\": {\n \"description\": \"continue may be set if the user set a limit on the number of items returned, and indicates that the server has more data available. The value is opaque and may be used to issue another request to the endpoint that served this list to retrieve the next set of available objects. Continuing a consistent list may not be possible if the server configuration has changed or more than a few minutes have passed. The resourceVersion field returned when using this continue value will be identical to the value in the first response, unless you have received this token from an error message.\",\n \"type\": \"string\"\n },\n \"resourceVersion\": {\n \"description\": \"String that identifies the server's internal version of this object that can be used by clients to determine when objects have changed. Value must be treated as opaque by clients and passed unmodified back to the server. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\",\n \"type\": \"string\"\n },\n \"selfLink\": {\n \"description\": \"selfLink is a URL representing this object. Populated by the system. Read-only.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"reason\": {\n \"description\": \"A machine-readable description of why this operation is in the \\\"Failure\\\" status. If this value is empty there is no information available. A Reason clarifies an HTTP status code but does not override it.\",\n \"type\": \"string\"\n },\n \"status\": {\n \"description\": \"Status of the operation. One of: \\\"Success\\\" or \\\"Failure\\\". More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"required\": [\n \"pending\"\n ],\n \"type\": \"object\"\n },\n \"labels\": {\n \"description\": \"Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels\",\n \"type\": \"object\"\n },\n \"managedFields\": {\n \"description\": \"ManagedFields maps workflow-id and version to the set of fields that are managed by that workflow. This is mostly for internal housekeeping, and users typically shouldn't need to set or understand this field. A workflow can be the user's name, a controller's name, or the name of a specific apply path like \\\"ci-cd\\\". The set of fields is always in the version that the workflow used when modifying the object.\\n\\nThis field is alpha and can be changed or removed without notice.\",\n \"items\": {\n \"description\": \"ManagedFieldsEntry is a workflow-id, a FieldSet and the group version of the resource that the fieldset applies to.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the version of this resource that this field set applies to. The format is \\\"group/version\\\" just like the top-level APIVersion field. It is necessary to track the version of a field set because it cannot be automatically converted.\",\n \"type\": \"string\"\n },\n \"fields\": {\n \"description\": \"Fields stores a set of fields in a data structure like a Trie. To understand how this is used, see: https://github.com/kubernetes-sigs/structured-merge-diff\",\n \"type\": \"object\"\n },\n \"manager\": {\n \"description\": \"Manager is an identifier of the workflow managing these fields.\",\n \"type\": \"string\"\n },\n \"operation\": {\n \"description\": \"Operation is the type of operation which lead to this ManagedFieldsEntry being created. The only valid values for this field are 'Apply' and 'Update'.\",\n \"type\": \"string\"\n },\n \"time\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"name\": {\n \"description\": \"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names\",\n \"type\": \"string\"\n },\n \"namespace\": {\n \"description\": \"Namespace defines the space within each name must be unique. An empty namespace is equivalent to the \\\"default\\\" namespace, but \\\"default\\\" is the canonical representation. Not all objects are required to be scoped to a namespace - the value of this field for those objects will be empty.\\n\\nMust be a DNS_LABEL. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/namespaces\",\n \"type\": \"string\"\n },\n \"ownerReferences\": {\n \"description\": \"List of objects depended by this object. If ALL objects in the list have been deleted, this object will be garbage collected. If this object is managed by a controller, then an entry in this list will point to this controller, with the controller field set to true. There cannot be more than one managing controller.\",\n \"items\": {\n \"description\": \"OwnerReference contains enough information to let you identify an owning object. An owning object must be in the same namespace as the dependent, or be cluster-scoped, so there is no namespace field.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"API version of the referent.\",\n \"type\": \"string\"\n },\n \"blockOwnerDeletion\": {\n \"description\": \"If true, AND if the owner has the \\\"foregroundDeletion\\\" finalizer, then the owner cannot be deleted from the key-value store until this reference is removed. Defaults to false. To set this field, a user needs \\\"delete\\\" permission of the owner, otherwise 422 (Unprocessable Entity) will be returned.\",\n \"type\": \"boolean\"\n },\n \"controller\": {\n \"description\": \"If true, this reference points to the managing controller.\",\n \"type\": \"boolean\"\n },\n \"kind\": {\n \"description\": \"Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#names\",\n \"type\": \"string\"\n },\n \"uid\": {\n \"description\": \"UID of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"apiVersion\",\n \"kind\",\n \"name\",\n \"uid\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"resourceVersion\": {\n \"description\": \"An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources.\\n\\nPopulated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\",\n \"type\": \"string\"\n },\n \"selfLink\": {\n \"description\": \"SelfLink is a URL representing this object. Populated by the system. Read-only.\",\n \"type\": \"string\"\n },\n \"uid\": {\n \"description\": \"UID is the unique in time and space value for this object. It is typically generated by the server on successful creation of a resource and is not allowed to change on PUT operations.\\n\\nPopulated by the system. Read-only. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"PersistentVolumeClaimSpec describes the common attributes of storage devices and allows a Source for provider-specific attributes\",\n \"properties\": {\n \"accessModes\": {\n \"description\": \"AccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"dataSource\": {\n \"description\": \"TypedLocalObjectReference contains enough information to let you locate the typed referenced object inside the same namespace.\",\n \"properties\": {\n \"apiGroup\": {\n \"description\": \"APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is the type of resource being referenced\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name is the name of resource being referenced\",\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"kind\",\n \"name\"\n ],\n \"type\": \"object\"\n },\n \"resources\": {\n \"description\": \"ResourceRequirements describes the compute resource requirements.\",\n \"properties\": {\n \"limits\": {\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"selector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"storageClassName\": {\n \"description\": \"Name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1\",\n \"type\": \"string\"\n },\n \"volumeMode\": {\n \"description\": \"volumeMode defines what type of volume is required by the claim. Value of Filesystem is implied when not included in claim spec. This is a beta feature.\",\n \"type\": \"string\"\n },\n \"volumeName\": {\n \"description\": \"VolumeName is the binding reference to the PersistentVolume backing this claim.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"status\": {\n \"description\": \"PersistentVolumeClaimStatus is the current status of a persistent volume claim.\",\n \"properties\": {\n \"accessModes\": {\n \"description\": \"AccessModes contains the actual access modes the volume backing the PVC has. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\",\n \"items\": {\n \"type\": \"string\"\n },\n \"type\": \"array\"\n },\n \"capacity\": {\n \"description\": \"Represents the actual resources of the underlying volume.\",\n \"type\": \"object\"\n },\n \"conditions\": {\n \"description\": \"Current Condition of persistent volume claim. If underlying persistent volume is being resized then the Condition will be set to 'ResizeStarted'.\",\n \"items\": {\n \"description\": \"PersistentVolumeClaimCondition contains details about state of pvc\",\n \"properties\": {\n \"lastProbeTime\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n },\n \"lastTransitionTime\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"format\": \"date-time\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"Human-readable message indicating details about last transition.\",\n \"type\": \"string\"\n },\n \"reason\": {\n \"description\": \"Unique, this should be a short, machine understandable string that gives the reason for condition's last transition. If it reports \\\"ResizeStarted\\\" that means the underlying persistent volume is being resized.\",\n \"type\": \"string\"\n },\n \"status\": {\n \"type\": \"string\"\n },\n \"type\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"type\",\n \"status\"\n ],\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"phase\": {\n \"description\": \"Phase represents the current phase of PersistentVolumeClaim.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"tag\": {\n \"description\": \"Tag of Prometheus container image to be deployed. Defaults to the value of `version`. Version is ignored if Tag is set.\",\n \"type\": \"string\"\n },\n \"thanos\": {\n \"description\": \"ThanosSpec defines parameters for a Prometheus server within a Thanos deployment.\",\n \"properties\": {\n \"baseImage\": {\n \"description\": \"Thanos base image if other than default.\",\n \"type\": \"string\"\n },\n \"image\": {\n \"description\": \"Image if specified has precedence over baseImage, tag and sha combinations. Specifying the version is still necessary to ensure the Prometheus Operator knows what version of Thanos is being configured.\",\n \"type\": \"string\"\n },\n \"objectStorageConfig\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n },\n \"required\": [\n \"key\"\n ],\n \"type\": \"object\"\n },\n \"resources\": {\n \"description\": \"ResourceRequirements describes the compute resource requirements.\",\n \"properties\": {\n \"limits\": {\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n },\n \"sha\": {\n \"description\": \"SHA of Thanos container image to be deployed. Defaults to the value of `version`. Similar to a tag, but the SHA explicitly deploys an immutable container image. Version and Tag are ignored if SHA is set.\",\n \"type\": \"string\"\n },\n \"tag\": {\n \"description\": \"Tag of Thanos sidecar container image to be deployed. Defaults to the value of `version`. Version is ignored if Tag is set.\",\n \"type\": \"string\"\n },\n \"version\": {\n \"description\": \"Version describes the version of Thanos to use.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"tolerations\": {\n \"description\": \"If specified, the pod's tolerations.\",\n \"items\": {\n \"description\": \"The pod this Toleration is attached to tolerates any taint that matches the triple \\u003ckey,value,effect\\u003e using the matching operator \\u003coperator\\u003e.\",\n \"properties\": {\n \"effect\": {\n \"description\": \"Effect indicates the taint effect to match. Empty means match all taint effects. When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.\",\n \"type\": \"string\"\n },\n \"key\": {\n \"description\": \"Key is the taint key that the toleration applies to. Empty means match all taint keys. If the key is empty, operator must be Exists; this combination means to match all values and all keys.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Operator represents a key's relationship to the value. Valid operators are Exists and Equal. Defaults to Equal. Exists is equivalent to wildcard for value, so that a pod can tolerate all taints of a particular category.\",\n \"type\": \"string\"\n },\n \"tolerationSeconds\": {\n \"description\": \"TolerationSeconds represents the period of time the toleration (which must be of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately) by the system.\",\n \"format\": \"int64\",\n \"type\": \"integer\"\n },\n \"value\": {\n \"description\": \"Value is the taint value the toleration matches to. If the operator is Exists, the value should be empty, otherwise just a regular string.\",\n \"type\": \"string\"\n }\n },\n \"type\": \"object\"\n },\n \"type\": \"array\"\n },\n \"version\": {\n \"description\": \"Version of Prometheus to be deployed.\",\n \"type\": \"string\"\n },\n \"walCompression\": {\n \"description\": \"Enable compression of the write-ahead log using Snappy.\",\n \"type\": \"boolean\"\n }\n },\n \"type\": \"object\"\n },\n \"status\": {\n \"description\": \"PrometheusStatus is the most recent observed status of the Prometheus cluster. Read-only. Not included when requesting from the apiserver, only from the Prometheus Operator API itself. More info: https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#spec-and-status\",\n \"properties\": {\n \"availableReplicas\": {\n \"description\": \"Total number of available pods (ready for at least minReadySeconds) targeted by this Prometheus deployment.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"paused\": {\n \"description\": \"Represents whether any actions on the underlying managed objects are being performed. Only delete actions will be performed.\",\n \"type\": \"boolean\"\n },\n \"replicas\": {\n \"description\": \"Total number of non-terminated pods targeted by this Prometheus deployment (their labels match the selector).\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"unavailableReplicas\": {\n \"description\": \"Total number of unavailable pods targeted by this Prometheus deployment.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"updatedReplicas\": {\n \"description\": \"Total number of non-terminated pods targeted by this Prometheus deployment that have the desired version spec.\",\n \"format\": \"int32\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"paused\",\n \"replicas\",\n \"updatedReplicas\",\n \"availableReplicas\",\n \"unavailableReplicas\"\n ],\n \"type\": \"object\"\n }\n },\n \"type\": \"object\"\n }\n },\n \"served\": true,\n \"storage\": true\n }\n ]\n },\n \"status\": {\n \"acceptedNames\": {\n \"kind\": \"Prometheus\",\n \"listKind\": \"PrometheusList\",\n \"plural\": \"prometheuses\",\n \"singular\": \"prometheus\"\n },\n \"conditions\": [\n {\n \"lastTransitionTime\": \"2020-05-05T16:58:10Z\",\n \"message\": \"no conflicts found\",\n \"reason\": \"NoConflicts\",\n \"status\": \"True\",\n \"type\": \"NamesAccepted\"\n },\n {\n \"lastTransitionTime\": \"2020-05-05T16:58:10Z\",\n \"message\": \"the initial names have been accepted\",\n \"reason\": \"InitialNamesAccepted\",\n \"status\": \"True\",\n \"type\": \"Established\"\n },\n {\n \"lastTransitionTime\": \"2020-05-05T16:58:10Z\",\n \"message\": \"[spec.validation.openAPIV3Schema.properties[spec].properties[alerting].properties[alertmanagers].items.properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[alerting].properties[alertmanagers].items.properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[alerting].properties[alertmanagers].items.properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[env].items.properties[valueFrom].properties[resourceFieldRef].properties[divisor].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[httpGet].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[httpGet].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[httpGet].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[tcpSocket].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[tcpSocket].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[tcpSocket].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[httpGet].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[httpGet].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[httpGet].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[tcpSocket].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[tcpSocket].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[tcpSocket].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[httpGet].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[httpGet].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[httpGet].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[tcpSocket].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[tcpSocket].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[tcpSocket].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[httpGet].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[httpGet].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[httpGet].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[tcpSocket].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[tcpSocket].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[tcpSocket].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[emptyDir].properties[sizeLimit].type: Required value: must not be empty for specified object fields]\",\n \"reason\": \"Violations\",\n \"status\": \"True\",\n \"type\": \"NonStructuralSchema\"\n }\n ],\n \"storedVersions\": [\n \"v1\"\n ]\n }\n}\n" }, { "path": "pkg/backup/actions/testdata/v1beta1/alertmanagers.monitoring.coreos.com.json", "content": "{\n \"kind\": \"CustomResourceDefinition\",\n \"apiVersion\": \"apiextensions.k8s.io/v1beta1\",\n \"metadata\": {\n \"name\": \"alertmanagers.monitoring.coreos.com\",\n \"selfLink\": \"/apis/apiextensions.k8s.io/v1beta1/customresourcedefinitions/alertmanagers.monitoring.coreos.com\",\n \"uid\": \"768a7255-d97a-4762-a400-f359fb24f4c8\",\n \"resourceVersion\": \"206434\",\n \"generation\": 1,\n \"creationTimestamp\": \"2020-05-05T16:51:39Z\",\n \"labels\": {\n \"app\": \"prometheus-operator\"\n },\n \"annotations\": {\n \"helm.sh/hook\": \"crd-install\",\n \"kubectl.kubernetes.io/last-applied-configuration\": \"{\\\"apiVersion\\\":\\\"apiextensions.k8s.io/v1beta1\\\",\\\"kind\\\":\\\"CustomResourceDefinition\\\",\\\"metadata\\\":{\\\"annotations\\\":{\\\"helm.sh/hook\\\":\\\"crd-install\\\"},\\\"creationTimestamp\\\":null,\\\"labels\\\":{\\\"app\\\":\\\"prometheus-operator\\\"},\\\"name\\\":\\\"alertmanagers.monitoring.coreos.com\\\"},\\\"spec\\\":{\\\"group\\\":\\\"monitoring.coreos.com\\\",\\\"names\\\":{\\\"kind\\\":\\\"Alertmanager\\\",\\\"plural\\\":\\\"alertmanagers\\\"},\\\"scope\\\":\\\"Namespaced\\\",\\\"validation\\\":{\\\"openAPIV3Schema\\\":{\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"spec\\\":{\\\"description\\\":\\\"AlertmanagerSpec is a specification of the desired behavior of the Alertmanager cluster. More info: https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#spec-and-status\\\",\\\"properties\\\":{\\\"additionalPeers\\\":{\\\"description\\\":\\\"AdditionalPeers allows injecting a set of additional Alertmanagers to peer with to form a highly available cluster.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"affinity\\\":{\\\"description\\\":\\\"Affinity is a group of affinity scheduling rules.\\\",\\\"properties\\\":{\\\"nodeAffinity\\\":{\\\"description\\\":\\\"Node affinity is a group of node affinity scheduling rules.\\\",\\\"properties\\\":{\\\"preferredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\\\\\"weight\\\\\\\" to the sum if the node matches the corresponding matchExpressions; the node(s) with the highest sum are the most preferred.\\\",\\\"items\\\":{\\\"description\\\":\\\"An empty preferred scheduling term matches all objects with implicit weight 0 (i.e. it's a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op).\\\",\\\"properties\\\":{\\\"preference\\\":{\\\"description\\\":\\\"A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"A list of node selector requirements by node's labels.\\\",\\\"items\\\":{\\\"description\\\":\\\"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"]},\\\"type\\\":\\\"array\\\"},\\\"matchFields\\\":{\\\"description\\\":\\\"A list of node selector requirements by node's fields.\\\",\\\"items\\\":{\\\"description\\\":\\\"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"]},\\\"type\\\":\\\"array\\\"}}},\\\"weight\\\":{\\\"description\\\":\\\"Weight associated with matching the corresponding nodeSelectorTerm, in the range 1-100.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"required\\\":[\\\"weight\\\",\\\"preference\\\"]},\\\"type\\\":\\\"array\\\"},\\\"requiredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"A node selector represents the union of the results of one or more label queries over a set of nodes; that is, it represents the OR of the selectors represented by the node selector terms.\\\",\\\"properties\\\":{\\\"nodeSelectorTerms\\\":{\\\"description\\\":\\\"Required. A list of node selector terms. The terms are ORed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"A list of node selector requirements by node's labels.\\\",\\\"items\\\":{\\\"description\\\":\\\"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"]},\\\"type\\\":\\\"array\\\"},\\\"matchFields\\\":{\\\"description\\\":\\\"A list of node selector requirements by node's fields.\\\",\\\"items\\\":{\\\"description\\\":\\\"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"]},\\\"type\\\":\\\"array\\\"}}},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"nodeSelectorTerms\\\"]}}},\\\"podAffinity\\\":{\\\"description\\\":\\\"Pod affinity is a group of inter pod affinity scheduling rules.\\\",\\\"properties\\\":{\\\"preferredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\\\\\"weight\\\\\\\" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.\\\",\\\"items\\\":{\\\"description\\\":\\\"The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)\\\",\\\"properties\\\":{\\\"podAffinityTerm\\\":{\\\"description\\\":\\\"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\\\u003ctopologyKey\\\\u003e matches that of any node on which a pod of the set of pods is running\\\",\\\"properties\\\":{\\\"labelSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"]},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}}},\\\"namespaces\\\":{\\\"description\\\":\\\"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\\\\\"this pod's namespace\\\\\\\"\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"topologyKey\\\":{\\\"description\\\":\\\"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"topologyKey\\\"]},\\\"weight\\\":{\\\"description\\\":\\\"weight associated with matching the corresponding podAffinityTerm, in the range 1-100.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"required\\\":[\\\"weight\\\",\\\"podAffinityTerm\\\"]},\\\"type\\\":\\\"array\\\"},\\\"requiredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.\\\",\\\"items\\\":{\\\"description\\\":\\\"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\\\u003ctopologyKey\\\\u003e matches that of any node on which a pod of the set of pods is running\\\",\\\"properties\\\":{\\\"labelSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"]},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}}},\\\"namespaces\\\":{\\\"description\\\":\\\"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\\\\\"this pod's namespace\\\\\\\"\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"topologyKey\\\":{\\\"description\\\":\\\"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"topologyKey\\\"]},\\\"type\\\":\\\"array\\\"}}},\\\"podAntiAffinity\\\":{\\\"description\\\":\\\"Pod anti affinity is a group of inter pod anti affinity scheduling rules.\\\",\\\"properties\\\":{\\\"preferredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"The scheduler will prefer to schedule pods to nodes that satisfy the anti-affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling anti-affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\\\\\"weight\\\\\\\" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.\\\",\\\"items\\\":{\\\"description\\\":\\\"The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)\\\",\\\"properties\\\":{\\\"podAffinityTerm\\\":{\\\"description\\\":\\\"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\\\u003ctopologyKey\\\\u003e matches that of any node on which a pod of the set of pods is running\\\",\\\"properties\\\":{\\\"labelSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"]},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}}},\\\"namespaces\\\":{\\\"description\\\":\\\"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\\\\\"this pod's namespace\\\\\\\"\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"topologyKey\\\":{\\\"description\\\":\\\"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"topologyKey\\\"]},\\\"weight\\\":{\\\"description\\\":\\\"weight associated with matching the corresponding podAffinityTerm, in the range 1-100.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"required\\\":[\\\"weight\\\",\\\"podAffinityTerm\\\"]},\\\"type\\\":\\\"array\\\"},\\\"requiredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"If the anti-affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the anti-affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.\\\",\\\"items\\\":{\\\"description\\\":\\\"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\\\u003ctopologyKey\\\\u003e matches that of any node on which a pod of the set of pods is running\\\",\\\"properties\\\":{\\\"labelSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"]},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}}},\\\"namespaces\\\":{\\\"description\\\":\\\"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\\\\\"this pod's namespace\\\\\\\"\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"topologyKey\\\":{\\\"description\\\":\\\"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"topologyKey\\\"]},\\\"type\\\":\\\"array\\\"}}}}},\\\"baseImage\\\":{\\\"description\\\":\\\"Base image that is used to deploy pods, without tag.\\\",\\\"type\\\":\\\"string\\\"},\\\"configMaps\\\":{\\\"description\\\":\\\"ConfigMaps is a list of ConfigMaps in the same namespace as the Alertmanager object, which shall be mounted into the Alertmanager Pods. The ConfigMaps are mounted into /etc/alertmanager/configmaps/\\\\u003cconfigmap-name\\\\u003e.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"configSecret\\\":{\\\"description\\\":\\\"ConfigSecret is the name of a Kubernetes Secret in the same namespace as the Alertmanager object, which contains configuration for this Alertmanager instance. Defaults to 'alertmanager-' The secret is mounted into /etc/alertmanager/config.\\\",\\\"type\\\":\\\"string\\\"},\\\"containers\\\":{\\\"description\\\":\\\"Containers allows injecting additional containers. This is meant to allow adding an authentication proxy to an Alertmanager pod.\\\",\\\"items\\\":{\\\"description\\\":\\\"A single application container that you want to run within a pod.\\\",\\\"properties\\\":{\\\"args\\\":{\\\"description\\\":\\\"Arguments to the entrypoint. The docker image's CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"command\\\":{\\\"description\\\":\\\"Entrypoint array. Not executed within a shell. The docker image's ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"env\\\":{\\\"description\\\":\\\"List of environment variables to set in the container. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"EnvVar represents an environment variable present in a Container.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the environment variable. Must be a C_IDENTIFIER.\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to \\\\\\\"\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"},\\\"valueFrom\\\":{\\\"description\\\":\\\"EnvVarSource represents a source for the value of an EnvVar.\\\",\\\"properties\\\":{\\\"configMapKeyRef\\\":{\\\"description\\\":\\\"Selects a key from a ConfigMap.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key to select.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"]},\\\"fieldRef\\\":{\\\"description\\\":\\\"ObjectFieldSelector selects an APIVersioned field of an object.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"Version of the schema the FieldPath is written in terms of, defaults to \\\\\\\"v1\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"},\\\"fieldPath\\\":{\\\"description\\\":\\\"Path of the field to select in the specified API version.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"fieldPath\\\"]},\\\"resourceFieldRef\\\":{\\\"description\\\":\\\"ResourceFieldSelector represents container resources (cpu, memory) and their output format\\\",\\\"properties\\\":{\\\"containerName\\\":{\\\"description\\\":\\\"Container name: required for volumes, optional for env vars\\\",\\\"type\\\":\\\"string\\\"},\\\"divisor\\\":{},\\\"resource\\\":{\\\"description\\\":\\\"Required: resource to select\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"resource\\\"]},\\\"secretKeyRef\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"]}}}},\\\"required\\\":[\\\"name\\\"]},\\\"type\\\":\\\"array\\\"},\\\"envFrom\\\":{\\\"description\\\":\\\"List of sources to populate environment variables in the container. The keys defined within a source must be a C_IDENTIFIER. All invalid keys will be reported as an event when the container is starting. When a key exists in multiple sources, the value associated with the last source will take precedence. Values defined by an Env with a duplicate key will take precedence. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"EnvFromSource represents the source of a set of ConfigMaps\\\",\\\"properties\\\":{\\\"configMapRef\\\":{\\\"description\\\":\\\"ConfigMapEnvSource selects a ConfigMap to populate the environment variables with.\\\\nThe contents of the target ConfigMap's Data field will represent the key-value pairs as environment variables.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap must be defined\\\",\\\"type\\\":\\\"boolean\\\"}}},\\\"prefix\\\":{\\\"description\\\":\\\"An optional identifier to prepend to each key in the ConfigMap. Must be a C_IDENTIFIER.\\\",\\\"type\\\":\\\"string\\\"},\\\"secretRef\\\":{\\\"description\\\":\\\"SecretEnvSource selects a Secret to populate the environment variables with.\\\\nThe contents of the target Secret's Data field will represent the key-value pairs as environment variables.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret must be defined\\\",\\\"type\\\":\\\"boolean\\\"}}}}},\\\"type\\\":\\\"array\\\"},\\\"image\\\":{\\\"description\\\":\\\"Docker image name. More info: https://kubernetes.io/docs/concepts/containers/images This field is optional to allow higher level config management to default or override container images in workload controllers like Deployments and StatefulSets.\\\",\\\"type\\\":\\\"string\\\"},\\\"imagePullPolicy\\\":{\\\"description\\\":\\\"Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images\\\",\\\"type\\\":\\\"string\\\"},\\\"lifecycle\\\":{\\\"description\\\":\\\"Lifecycle describes actions that the management system should take in response to container lifecycle events. For the PostStart and PreStop lifecycle handlers, management of the container blocks until the action is complete, unless the container process fails, in which case the handler is aborted.\\\",\\\"properties\\\":{\\\"postStart\\\":{\\\"description\\\":\\\"Handler defines a specific action that should be taken\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"ExecAction describes a \\\\\\\"run in container\\\\\\\" action.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}}},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGetAction describes an action based on HTTP Get requests.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"]},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"]},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocketAction describes an action based on opening a socket\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]}},\\\"required\\\":[\\\"port\\\"]}}},\\\"preStop\\\":{\\\"description\\\":\\\"Handler defines a specific action that should be taken\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"ExecAction describes a \\\\\\\"run in container\\\\\\\" action.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}}},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGetAction describes an action based on HTTP Get requests.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"]},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"]},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocketAction describes an action based on opening a socket\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]}},\\\"required\\\":[\\\"port\\\"]}}}}},\\\"livenessProbe\\\":{\\\"description\\\":\\\"Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic.\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"ExecAction describes a \\\\\\\"run in container\\\\\\\" action.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}}},\\\"failureThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGetAction describes an action based on HTTP Get requests.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"]},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"]},\\\"initialDelaySeconds\\\":{\\\"description\\\":\\\"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"periodSeconds\\\":{\\\"description\\\":\\\"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"successThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocketAction describes an action based on opening a socket\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]}},\\\"required\\\":[\\\"port\\\"]},\\\"timeoutSeconds\\\":{\\\"description\\\":\\\"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}}},\\\"name\\\":{\\\"description\\\":\\\"Name of the container specified as a DNS_LABEL. Each container in a pod must have a unique name (DNS_LABEL). Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"},\\\"ports\\\":{\\\"description\\\":\\\"List of ports to expose from the container. Exposing a port here gives the system additional information about the network connections a container uses, but is primarily informational. Not specifying a port here DOES NOT prevent that port from being exposed. Any port which is listening on the default \\\\\\\"0.0.0.0\\\\\\\" address inside a container will be accessible from the network. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"ContainerPort represents a network port in a single container.\\\",\\\"properties\\\":{\\\"containerPort\\\":{\\\"description\\\":\\\"Number of port to expose on the pod's IP address. This must be a valid port number, 0 \\\\u003c x \\\\u003c 65536.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"hostIP\\\":{\\\"description\\\":\\\"What host IP to bind the external port to.\\\",\\\"type\\\":\\\"string\\\"},\\\"hostPort\\\":{\\\"description\\\":\\\"Number of port to expose on the host. If specified, this must be a valid port number, 0 \\\\u003c x \\\\u003c 65536. If HostNetwork is specified, this must match ContainerPort. Most containers do not need this.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"name\\\":{\\\"description\\\":\\\"If specified, this must be an IANA_SVC_NAME and unique within the pod. Each named port in a pod must have a unique name. Name for the port that can be referred to by services.\\\",\\\"type\\\":\\\"string\\\"},\\\"protocol\\\":{\\\"description\\\":\\\"Protocol for port. Must be UDP, TCP, or SCTP. Defaults to \\\\\\\"TCP\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"containerPort\\\"]},\\\"type\\\":\\\"array\\\"},\\\"readinessProbe\\\":{\\\"description\\\":\\\"Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic.\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"ExecAction describes a \\\\\\\"run in container\\\\\\\" action.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}}},\\\"failureThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGetAction describes an action based on HTTP Get requests.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"]},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"]},\\\"initialDelaySeconds\\\":{\\\"description\\\":\\\"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"periodSeconds\\\":{\\\"description\\\":\\\"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"successThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocketAction describes an action based on opening a socket\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]}},\\\"required\\\":[\\\"port\\\"]},\\\"timeoutSeconds\\\":{\\\"description\\\":\\\"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}}},\\\"resources\\\":{\\\"description\\\":\\\"ResourceRequirements describes the compute resource requirements.\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}}},\\\"securityContext\\\":{\\\"description\\\":\\\"SecurityContext holds security configuration that will be applied to a container. Some fields are present in both SecurityContext and PodSecurityContext. When both are set, the values in SecurityContext take precedence.\\\",\\\"properties\\\":{\\\"allowPrivilegeEscalation\\\":{\\\"description\\\":\\\"AllowPrivilegeEscalation controls whether a process can gain more privileges than its parent process. This bool directly controls if the no_new_privs flag will be set on the container process. AllowPrivilegeEscalation is true always when the container is: 1) run as Privileged 2) has CAP_SYS_ADMIN\\\",\\\"type\\\":\\\"boolean\\\"},\\\"capabilities\\\":{\\\"description\\\":\\\"Adds and removes POSIX capabilities from running containers.\\\",\\\"properties\\\":{\\\"add\\\":{\\\"description\\\":\\\"Added capabilities\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"drop\\\":{\\\"description\\\":\\\"Removed capabilities\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}}},\\\"privileged\\\":{\\\"description\\\":\\\"Run container in privileged mode. Processes in privileged containers are essentially equivalent to root on the host. Defaults to false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"procMount\\\":{\\\"description\\\":\\\"procMount denotes the type of proc mount to use for the containers. The default is DefaultProcMount which uses the container runtime defaults for readonly paths and masked paths. This requires the ProcMountType feature flag to be enabled.\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnlyRootFilesystem\\\":{\\\"description\\\":\\\"Whether this container has a read-only root filesystem. Default is false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"runAsGroup\\\":{\\\"description\\\":\\\"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"runAsNonRoot\\\":{\\\"description\\\":\\\"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"runAsUser\\\":{\\\"description\\\":\\\"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"seLinuxOptions\\\":{\\\"description\\\":\\\"SELinuxOptions are the labels to be applied to the container\\\",\\\"properties\\\":{\\\"level\\\":{\\\"description\\\":\\\"Level is SELinux level label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"role\\\":{\\\"description\\\":\\\"Role is a SELinux role label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"type\\\":{\\\"description\\\":\\\"Type is a SELinux type label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"user\\\":{\\\"description\\\":\\\"User is a SELinux user label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"}}}}},\\\"stdin\\\":{\\\"description\\\":\\\"Whether this container should allocate a buffer for stdin in the container runtime. If this is not set, reads from stdin in the container will always result in EOF. Default is false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"stdinOnce\\\":{\\\"description\\\":\\\"Whether the container runtime should close the stdin channel after it has been opened by a single attach. When stdin is true the stdin stream will remain open across multiple attach sessions. If stdinOnce is set to true, stdin is opened on container start, is empty until the first client attaches to stdin, and then remains open and accepts data until the client disconnects, at which time stdin is closed and remains closed until the container is restarted. If this flag is false, a container processes that reads from stdin will never receive an EOF. Default is false\\\",\\\"type\\\":\\\"boolean\\\"},\\\"terminationMessagePath\\\":{\\\"description\\\":\\\"Optional: Path at which the file to which the container's termination message will be written is mounted into the container's filesystem. Message written is intended to be brief final status, such as an assertion failure message. Will be truncated by the node if greater than 4096 bytes. The total message length across all containers will be limited to 12kb. Defaults to /dev/termination-log. Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"},\\\"terminationMessagePolicy\\\":{\\\"description\\\":\\\"Indicate how the termination message should be populated. File will use the contents of terminationMessagePath to populate the container status message on both success and failure. FallbackToLogsOnError will use the last chunk of container log output if the termination message file is empty and the container exited with an error. The log output is limited to 2048 bytes or 80 lines, whichever is smaller. Defaults to File. Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"},\\\"tty\\\":{\\\"description\\\":\\\"Whether this container should allocate a TTY for itself, also requires 'stdin' to be true. Default is false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"volumeDevices\\\":{\\\"description\\\":\\\"volumeDevices is the list of block devices to be used by the container. This is a beta feature.\\\",\\\"items\\\":{\\\"description\\\":\\\"volumeDevice describes a mapping of a raw block device within a container.\\\",\\\"properties\\\":{\\\"devicePath\\\":{\\\"description\\\":\\\"devicePath is the path inside of the container that the device will be mapped to.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"name must match the name of a persistentVolumeClaim in the pod\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"devicePath\\\"]},\\\"type\\\":\\\"array\\\"},\\\"volumeMounts\\\":{\\\"description\\\":\\\"Pod volumes to mount into the container's filesystem. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"VolumeMount describes a mounting of a Volume within a container.\\\",\\\"properties\\\":{\\\"mountPath\\\":{\\\"description\\\":\\\"Path within the container at which the volume should be mounted. Must not contain ':'.\\\",\\\"type\\\":\\\"string\\\"},\\\"mountPropagation\\\":{\\\"description\\\":\\\"mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"This must match the Name of a Volume.\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"subPath\\\":{\\\"description\\\":\\\"Path within the volume from which the container's volume should be mounted. Defaults to \\\\\\\"\\\\\\\" (volume's root).\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"mountPath\\\"]},\\\"type\\\":\\\"array\\\"},\\\"workingDir\\\":{\\\"description\\\":\\\"Container's working directory. If not specified, the container runtime's default will be used, which might be configured in the container image. Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\"]},\\\"type\\\":\\\"array\\\"},\\\"externalUrl\\\":{\\\"description\\\":\\\"The external URL the Alertmanager instances will be available under. This is necessary to generate correct URLs. This is necessary if Alertmanager is not served from root of a DNS name.\\\",\\\"type\\\":\\\"string\\\"},\\\"image\\\":{\\\"description\\\":\\\"Image if specified has precedence over baseImage, tag and sha combinations. Specifying the version is still necessary to ensure the Prometheus Operator knows what version of Alertmanager is being configured.\\\",\\\"type\\\":\\\"string\\\"},\\\"imagePullSecrets\\\":{\\\"description\\\":\\\"An optional list of references to secrets in the same namespace to use for pulling prometheus and alertmanager images from registries see http://kubernetes.io/docs/user-guide/images#specifying-imagepullsecrets-on-a-pod\\\",\\\"items\\\":{\\\"description\\\":\\\"LocalObjectReference contains enough information to let you locate the referenced object inside the same namespace.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"}}},\\\"type\\\":\\\"array\\\"},\\\"listenLocal\\\":{\\\"description\\\":\\\"ListenLocal makes the Alertmanager server listen on loopback, so that it does not bind against the Pod IP. Note this is only for the Alertmanager UI, not the gossip communication.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"logLevel\\\":{\\\"description\\\":\\\"Log level for Alertmanager to be configured with.\\\",\\\"type\\\":\\\"string\\\"},\\\"nodeSelector\\\":{\\\"description\\\":\\\"Define which Nodes the Pods are scheduled on.\\\",\\\"type\\\":\\\"object\\\"},\\\"paused\\\":{\\\"description\\\":\\\"If set to true all actions on the underlying managed objects are not goint to be performed, except for delete actions.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"podMetadata\\\":{\\\"description\\\":\\\"ObjectMeta is metadata that all persisted resources must have, which includes all objects users must create.\\\",\\\"properties\\\":{\\\"annotations\\\":{\\\"description\\\":\\\"Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations\\\",\\\"type\\\":\\\"object\\\"},\\\"clusterName\\\":{\\\"description\\\":\\\"The name of the cluster which the object belongs to. This is used to distinguish resources with same name and namespace in different clusters. This field is not set anywhere right now and apiserver is going to ignore it if set in create or update request.\\\",\\\"type\\\":\\\"string\\\"},\\\"creationTimestamp\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"deletionGracePeriodSeconds\\\":{\\\"description\\\":\\\"Number of seconds allowed for this object to gracefully terminate before it will be removed from the system. Only set when deletionTimestamp is also set. May only be shortened. Read-only.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"deletionTimestamp\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"finalizers\\\":{\\\"description\\\":\\\"Must be empty before the object is deleted from the registry. Each entry is an identifier for the responsible component that will remove the entry from the list. If the deletionTimestamp of the object is non-nil, entries in this list can only be removed.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"generateName\\\":{\\\"description\\\":\\\"GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server.\\\\nIf this field is specified and the generated name exists, the server will NOT return a 409 - instead, it will either return 201 Created or 500 with Reason ServerTimeout indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header).\\\\nApplied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#idempotency\\\",\\\"type\\\":\\\"string\\\"},\\\"generation\\\":{\\\"description\\\":\\\"A sequence number representing a specific generation of the desired state. Populated by the system. Read-only.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"initializers\\\":{\\\"description\\\":\\\"Initializers tracks the progress of initialization.\\\",\\\"properties\\\":{\\\"pending\\\":{\\\"description\\\":\\\"Pending is a list of initializers that must execute in order before this object is visible. When the last pending initializer is removed, and no failing result is set, the initializers struct will be set to nil and the object is considered as initialized and visible to all clients.\\\",\\\"items\\\":{\\\"description\\\":\\\"Initializer is information about an initializer that has not yet completed.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"name of the process that is responsible for initializing this object.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\"]},\\\"type\\\":\\\"array\\\"},\\\"result\\\":{\\\"description\\\":\\\"Status is a return value for calls that don't return other objects.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"code\\\":{\\\"description\\\":\\\"Suggested HTTP return code for this status, 0 if not set.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"details\\\":{\\\"description\\\":\\\"StatusDetails is a set of additional properties that MAY be set by the server to provide additional information about a response. The Reason field of a Status object defines what attributes will be set. Clients must ignore fields that do not match the defined type of each attribute, and should assume that any attribute may be empty, invalid, or under defined.\\\",\\\"properties\\\":{\\\"causes\\\":{\\\"description\\\":\\\"The Causes array includes more details associated with the StatusReason failure. Not all StatusReasons may provide detailed causes.\\\",\\\"items\\\":{\\\"description\\\":\\\"StatusCause provides more information about an api.Status failure, including cases when multiple errors are encountered.\\\",\\\"properties\\\":{\\\"field\\\":{\\\"description\\\":\\\"The field of the resource that has caused this error, as named by its JSON serialization. May include dot and postfix notation for nested attributes. Arrays are zero-indexed. Fields may appear more than once in an array of causes due to fields having multiple errors. Optional.\\\\nExamples:\\\\n \\\\\\\"name\\\\\\\" - the field \\\\\\\"name\\\\\\\" on the current resource\\\\n \\\\\\\"items[0].name\\\\\\\" - the field \\\\\\\"name\\\\\\\" on the first array entry in \\\\\\\"items\\\\\\\"\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"A human-readable description of the cause of the error. This field may be presented as-is to a reader.\\\",\\\"type\\\":\\\"string\\\"},\\\"reason\\\":{\\\"description\\\":\\\"A machine-readable description of the cause of the error. If this value is empty there is no information available.\\\",\\\"type\\\":\\\"string\\\"}}},\\\"type\\\":\\\"array\\\"},\\\"group\\\":{\\\"description\\\":\\\"The group attribute of the resource associated with the status StatusReason.\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"The kind attribute of the resource associated with the status StatusReason. On some operations may differ from the requested resource Kind. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"The name attribute of the resource associated with the status StatusReason (when there is a single name which can be described).\\\",\\\"type\\\":\\\"string\\\"},\\\"retryAfterSeconds\\\":{\\\"description\\\":\\\"If specified, the time in seconds before the operation should be retried. Some errors may indicate the client must take an alternate action - for those errors this field may indicate how long to wait before taking the alternate action.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID of the resource. (when there is a single resource which can be described). More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}}},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"A human-readable description of the status of this operation.\\\",\\\"type\\\":\\\"string\\\"},\\\"metadata\\\":{\\\"description\\\":\\\"ListMeta describes metadata that synthetic resources must have, including lists and various status objects. A resource may have only one of {ObjectMeta, ListMeta}.\\\",\\\"properties\\\":{\\\"continue\\\":{\\\"description\\\":\\\"continue may be set if the user set a limit on the number of items returned, and indicates that the server has more data available. The value is opaque and may be used to issue another request to the endpoint that served this list to retrieve the next set of available objects. Continuing a consistent list may not be possible if the server configuration has changed or more than a few minutes have passed. The resourceVersion field returned when using this continue value will be identical to the value in the first response, unless you have received this token from an error message.\\\",\\\"type\\\":\\\"string\\\"},\\\"resourceVersion\\\":{\\\"description\\\":\\\"String that identifies the server's internal version of this object that can be used by clients to determine when objects have changed. Value must be treated as opaque by clients and passed unmodified back to the server. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\\\",\\\"type\\\":\\\"string\\\"},\\\"selfLink\\\":{\\\"description\\\":\\\"selfLink is a URL representing this object. Populated by the system. Read-only.\\\",\\\"type\\\":\\\"string\\\"}}},\\\"reason\\\":{\\\"description\\\":\\\"A machine-readable description of why this operation is in the \\\\\\\"Failure\\\\\\\" status. If this value is empty there is no information available. A Reason clarifies an HTTP status code but does not override it.\\\",\\\"type\\\":\\\"string\\\"},\\\"status\\\":{\\\"description\\\":\\\"Status of the operation. One of: \\\\\\\"Success\\\\\\\" or \\\\\\\"Failure\\\\\\\". More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status\\\",\\\"type\\\":\\\"string\\\"}}}},\\\"required\\\":[\\\"pending\\\"]},\\\"labels\\\":{\\\"description\\\":\\\"Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels\\\",\\\"type\\\":\\\"object\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names\\\",\\\"type\\\":\\\"string\\\"},\\\"namespace\\\":{\\\"description\\\":\\\"Namespace defines the space within each name must be unique. An empty namespace is equivalent to the \\\\\\\"default\\\\\\\" namespace, but \\\\\\\"default\\\\\\\" is the canonical representation. Not all objects are required to be scoped to a namespace - the value of this field for those objects will be empty.\\\\nMust be a DNS_LABEL. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/namespaces\\\",\\\"type\\\":\\\"string\\\"},\\\"ownerReferences\\\":{\\\"description\\\":\\\"List of objects depended by this object. If ALL objects in the list have been deleted, this object will be garbage collected. If this object is managed by a controller, then an entry in this list will point to this controller, with the controller field set to true. There cannot be more than one managing controller.\\\",\\\"items\\\":{\\\"description\\\":\\\"OwnerReference contains enough information to let you identify an owning object. An owning object must be in the same namespace as the dependent, or be cluster-scoped, so there is no namespace field.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"API version of the referent.\\\",\\\"type\\\":\\\"string\\\"},\\\"blockOwnerDeletion\\\":{\\\"description\\\":\\\"If true, AND if the owner has the \\\\\\\"foregroundDeletion\\\\\\\" finalizer, then the owner cannot be deleted from the key-value store until this reference is removed. Defaults to false. To set this field, a user needs \\\\\\\"delete\\\\\\\" permission of the owner, otherwise 422 (Unprocessable Entity) will be returned.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"controller\\\":{\\\"description\\\":\\\"If true, this reference points to the managing controller.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#names\\\",\\\"type\\\":\\\"string\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"apiVersion\\\",\\\"kind\\\",\\\"name\\\",\\\"uid\\\"]},\\\"type\\\":\\\"array\\\"},\\\"resourceVersion\\\":{\\\"description\\\":\\\"An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources.\\\\nPopulated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\\\",\\\"type\\\":\\\"string\\\"},\\\"selfLink\\\":{\\\"description\\\":\\\"SelfLink is a URL representing this object. Populated by the system. Read-only.\\\",\\\"type\\\":\\\"string\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID is the unique in time and space value for this object. It is typically generated by the server on successful creation of a resource and is not allowed to change on PUT operations.\\\\nPopulated by the system. Read-only. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}}},\\\"priorityClassName\\\":{\\\"description\\\":\\\"Priority class assigned to the Pods\\\",\\\"type\\\":\\\"string\\\"},\\\"replicas\\\":{\\\"description\\\":\\\"Size is the expected size of the alertmanager cluster. The controller will eventually make the size of the running cluster equal to the expected size.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"resources\\\":{\\\"description\\\":\\\"ResourceRequirements describes the compute resource requirements.\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}}},\\\"retention\\\":{\\\"description\\\":\\\"Time duration Alertmanager shall retain data for. Default is '120h', and must match the regular expression `[0-9]+(ms|s|m|h)` (milliseconds seconds minutes hours).\\\",\\\"type\\\":\\\"string\\\"},\\\"routePrefix\\\":{\\\"description\\\":\\\"The route prefix Alertmanager registers HTTP handlers for. This is useful, if using ExternalURL and a proxy is rewriting HTTP routes of a request, and the actual ExternalURL is still true, but the server serves requests under a different route prefix. For example for use with `kubectl proxy`.\\\",\\\"type\\\":\\\"string\\\"},\\\"secrets\\\":{\\\"description\\\":\\\"Secrets is a list of Secrets in the same namespace as the Alertmanager object, which shall be mounted into the Alertmanager Pods. The Secrets are mounted into /etc/alertmanager/secrets/\\\\u003csecret-name\\\\u003e.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"securityContext\\\":{\\\"description\\\":\\\"PodSecurityContext holds pod-level security attributes and common container settings. Some fields are also present in container.securityContext. Field values of container.securityContext take precedence over field values of PodSecurityContext.\\\",\\\"properties\\\":{\\\"fsGroup\\\":{\\\"description\\\":\\\"A special supplemental group that applies to all containers in a pod. Some volume types allow the Kubelet to change the ownership of that volume to be owned by the pod:\\\\n1. The owning GID will be the FSGroup 2. The setgid bit is set (new files created in the volume will be owned by FSGroup) 3. The permission bits are OR'd with rw-rw----\\\\nIf unset, the Kubelet will not modify the ownership and permissions of any volume.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"runAsGroup\\\":{\\\"description\\\":\\\"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"runAsNonRoot\\\":{\\\"description\\\":\\\"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"runAsUser\\\":{\\\"description\\\":\\\"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"seLinuxOptions\\\":{\\\"description\\\":\\\"SELinuxOptions are the labels to be applied to the container\\\",\\\"properties\\\":{\\\"level\\\":{\\\"description\\\":\\\"Level is SELinux level label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"role\\\":{\\\"description\\\":\\\"Role is a SELinux role label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"type\\\":{\\\"description\\\":\\\"Type is a SELinux type label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"user\\\":{\\\"description\\\":\\\"User is a SELinux user label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"}}},\\\"supplementalGroups\\\":{\\\"description\\\":\\\"A list of groups applied to the first process run in each container, in addition to the container's primary GID. If unspecified, no groups will be added to any container.\\\",\\\"items\\\":{\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"type\\\":\\\"array\\\"},\\\"sysctls\\\":{\\\"description\\\":\\\"Sysctls hold a list of namespaced sysctls used for the pod. Pods with unsupported sysctls (by the container runtime) might fail to launch.\\\",\\\"items\\\":{\\\"description\\\":\\\"Sysctl defines a kernel parameter to be set\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of a property to set\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"Value of a property to set\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"]},\\\"type\\\":\\\"array\\\"}}},\\\"serviceAccountName\\\":{\\\"description\\\":\\\"ServiceAccountName is the name of the ServiceAccount to use to run the Prometheus Pods.\\\",\\\"type\\\":\\\"string\\\"},\\\"sha\\\":{\\\"description\\\":\\\"SHA of Alertmanager container image to be deployed. Defaults to the value of `version`. Similar to a tag, but the SHA explicitly deploys an immutable container image. Version and Tag are ignored if SHA is set.\\\",\\\"type\\\":\\\"string\\\"},\\\"storage\\\":{\\\"description\\\":\\\"StorageSpec defines the configured storage for a group Prometheus servers. If neither `emptyDir` nor `volumeClaimTemplate` is specified, then by default an [EmptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir) will be used.\\\",\\\"properties\\\":{\\\"emptyDir\\\":{\\\"description\\\":\\\"Represents an empty directory for a pod. Empty directory volumes support ownership management and SELinux relabeling.\\\",\\\"properties\\\":{\\\"medium\\\":{\\\"description\\\":\\\"What type of storage medium should back this directory. The default is \\\\\\\"\\\\\\\" which means to use the node's default medium. Must be an empty string (default) or Memory. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir\\\",\\\"type\\\":\\\"string\\\"},\\\"sizeLimit\\\":{}}},\\\"volumeClaimTemplate\\\":{\\\"description\\\":\\\"PersistentVolumeClaim is a user's request for and claim to a persistent volume\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"metadata\\\":{\\\"description\\\":\\\"ObjectMeta is metadata that all persisted resources must have, which includes all objects users must create.\\\",\\\"properties\\\":{\\\"annotations\\\":{\\\"description\\\":\\\"Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations\\\",\\\"type\\\":\\\"object\\\"},\\\"clusterName\\\":{\\\"description\\\":\\\"The name of the cluster which the object belongs to. This is used to distinguish resources with same name and namespace in different clusters. This field is not set anywhere right now and apiserver is going to ignore it if set in create or update request.\\\",\\\"type\\\":\\\"string\\\"},\\\"creationTimestamp\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"deletionGracePeriodSeconds\\\":{\\\"description\\\":\\\"Number of seconds allowed for this object to gracefully terminate before it will be removed from the system. Only set when deletionTimestamp is also set. May only be shortened. Read-only.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"deletionTimestamp\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"finalizers\\\":{\\\"description\\\":\\\"Must be empty before the object is deleted from the registry. Each entry is an identifier for the responsible component that will remove the entry from the list. If the deletionTimestamp of the object is non-nil, entries in this list can only be removed.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"generateName\\\":{\\\"description\\\":\\\"GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server.\\\\nIf this field is specified and the generated name exists, the server will NOT return a 409 - instead, it will either return 201 Created or 500 with Reason ServerTimeout indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header).\\\\nApplied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#idempotency\\\",\\\"type\\\":\\\"string\\\"},\\\"generation\\\":{\\\"description\\\":\\\"A sequence number representing a specific generation of the desired state. Populated by the system. Read-only.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"initializers\\\":{\\\"description\\\":\\\"Initializers tracks the progress of initialization.\\\",\\\"properties\\\":{\\\"pending\\\":{\\\"description\\\":\\\"Pending is a list of initializers that must execute in order before this object is visible. When the last pending initializer is removed, and no failing result is set, the initializers struct will be set to nil and the object is considered as initialized and visible to all clients.\\\",\\\"items\\\":{\\\"description\\\":\\\"Initializer is information about an initializer that has not yet completed.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"name of the process that is responsible for initializing this object.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\"]},\\\"type\\\":\\\"array\\\"},\\\"result\\\":{\\\"description\\\":\\\"Status is a return value for calls that don't return other objects.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"code\\\":{\\\"description\\\":\\\"Suggested HTTP return code for this status, 0 if not set.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"details\\\":{\\\"description\\\":\\\"StatusDetails is a set of additional properties that MAY be set by the server to provide additional information about a response. The Reason field of a Status object defines what attributes will be set. Clients must ignore fields that do not match the defined type of each attribute, and should assume that any attribute may be empty, invalid, or under defined.\\\",\\\"properties\\\":{\\\"causes\\\":{\\\"description\\\":\\\"The Causes array includes more details associated with the StatusReason failure. Not all StatusReasons may provide detailed causes.\\\",\\\"items\\\":{\\\"description\\\":\\\"StatusCause provides more information about an api.Status failure, including cases when multiple errors are encountered.\\\",\\\"properties\\\":{\\\"field\\\":{\\\"description\\\":\\\"The field of the resource that has caused this error, as named by its JSON serialization. May include dot and postfix notation for nested attributes. Arrays are zero-indexed. Fields may appear more than once in an array of causes due to fields having multiple errors. Optional.\\\\nExamples:\\\\n \\\\\\\"name\\\\\\\" - the field \\\\\\\"name\\\\\\\" on the current resource\\\\n \\\\\\\"items[0].name\\\\\\\" - the field \\\\\\\"name\\\\\\\" on the first array entry in \\\\\\\"items\\\\\\\"\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"A human-readable description of the cause of the error. This field may be presented as-is to a reader.\\\",\\\"type\\\":\\\"string\\\"},\\\"reason\\\":{\\\"description\\\":\\\"A machine-readable description of the cause of the error. If this value is empty there is no information available.\\\",\\\"type\\\":\\\"string\\\"}}},\\\"type\\\":\\\"array\\\"},\\\"group\\\":{\\\"description\\\":\\\"The group attribute of the resource associated with the status StatusReason.\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"The kind attribute of the resource associated with the status StatusReason. On some operations may differ from the requested resource Kind. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"The name attribute of the resource associated with the status StatusReason (when there is a single name which can be described).\\\",\\\"type\\\":\\\"string\\\"},\\\"retryAfterSeconds\\\":{\\\"description\\\":\\\"If specified, the time in seconds before the operation should be retried. Some errors may indicate the client must take an alternate action - for those errors this field may indicate how long to wait before taking the alternate action.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID of the resource. (when there is a single resource which can be described). More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}}},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"A human-readable description of the status of this operation.\\\",\\\"type\\\":\\\"string\\\"},\\\"metadata\\\":{\\\"description\\\":\\\"ListMeta describes metadata that synthetic resources must have, including lists and various status objects. A resource may have only one of {ObjectMeta, ListMeta}.\\\",\\\"properties\\\":{\\\"continue\\\":{\\\"description\\\":\\\"continue may be set if the user set a limit on the number of items returned, and indicates that the server has more data available. The value is opaque and may be used to issue another request to the endpoint that served this list to retrieve the next set of available objects. Continuing a consistent list may not be possible if the server configuration has changed or more than a few minutes have passed. The resourceVersion field returned when using this continue value will be identical to the value in the first response, unless you have received this token from an error message.\\\",\\\"type\\\":\\\"string\\\"},\\\"resourceVersion\\\":{\\\"description\\\":\\\"String that identifies the server's internal version of this object that can be used by clients to determine when objects have changed. Value must be treated as opaque by clients and passed unmodified back to the server. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\\\",\\\"type\\\":\\\"string\\\"},\\\"selfLink\\\":{\\\"description\\\":\\\"selfLink is a URL representing this object. Populated by the system. Read-only.\\\",\\\"type\\\":\\\"string\\\"}}},\\\"reason\\\":{\\\"description\\\":\\\"A machine-readable description of why this operation is in the \\\\\\\"Failure\\\\\\\" status. If this value is empty there is no information available. A Reason clarifies an HTTP status code but does not override it.\\\",\\\"type\\\":\\\"string\\\"},\\\"status\\\":{\\\"description\\\":\\\"Status of the operation. One of: \\\\\\\"Success\\\\\\\" or \\\\\\\"Failure\\\\\\\". More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status\\\",\\\"type\\\":\\\"string\\\"}}}},\\\"required\\\":[\\\"pending\\\"]},\\\"labels\\\":{\\\"description\\\":\\\"Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels\\\",\\\"type\\\":\\\"object\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names\\\",\\\"type\\\":\\\"string\\\"},\\\"namespace\\\":{\\\"description\\\":\\\"Namespace defines the space within each name must be unique. An empty namespace is equivalent to the \\\\\\\"default\\\\\\\" namespace, but \\\\\\\"default\\\\\\\" is the canonical representation. Not all objects are required to be scoped to a namespace - the value of this field for those objects will be empty.\\\\nMust be a DNS_LABEL. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/namespaces\\\",\\\"type\\\":\\\"string\\\"},\\\"ownerReferences\\\":{\\\"description\\\":\\\"List of objects depended by this object. If ALL objects in the list have been deleted, this object will be garbage collected. If this object is managed by a controller, then an entry in this list will point to this controller, with the controller field set to true. There cannot be more than one managing controller.\\\",\\\"items\\\":{\\\"description\\\":\\\"OwnerReference contains enough information to let you identify an owning object. An owning object must be in the same namespace as the dependent, or be cluster-scoped, so there is no namespace field.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"API version of the referent.\\\",\\\"type\\\":\\\"string\\\"},\\\"blockOwnerDeletion\\\":{\\\"description\\\":\\\"If true, AND if the owner has the \\\\\\\"foregroundDeletion\\\\\\\" finalizer, then the owner cannot be deleted from the key-value store until this reference is removed. Defaults to false. To set this field, a user needs \\\\\\\"delete\\\\\\\" permission of the owner, otherwise 422 (Unprocessable Entity) will be returned.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"controller\\\":{\\\"description\\\":\\\"If true, this reference points to the managing controller.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#names\\\",\\\"type\\\":\\\"string\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"apiVersion\\\",\\\"kind\\\",\\\"name\\\",\\\"uid\\\"]},\\\"type\\\":\\\"array\\\"},\\\"resourceVersion\\\":{\\\"description\\\":\\\"An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources.\\\\nPopulated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\\\",\\\"type\\\":\\\"string\\\"},\\\"selfLink\\\":{\\\"description\\\":\\\"SelfLink is a URL representing this object. Populated by the system. Read-only.\\\",\\\"type\\\":\\\"string\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID is the unique in time and space value for this object. It is typically generated by the server on successful creation of a resource and is not allowed to change on PUT operations.\\\\nPopulated by the system. Read-only. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}}},\\\"spec\\\":{\\\"description\\\":\\\"PersistentVolumeClaimSpec describes the common attributes of storage devices and allows a Source for provider-specific attributes\\\",\\\"properties\\\":{\\\"accessModes\\\":{\\\"description\\\":\\\"AccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"dataSource\\\":{\\\"description\\\":\\\"TypedLocalObjectReference contains enough information to let you locate the typed referenced object inside the same namespace.\\\",\\\"properties\\\":{\\\"apiGroup\\\":{\\\"description\\\":\\\"APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is the type of resource being referenced\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name is the name of resource being referenced\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"kind\\\",\\\"name\\\"]},\\\"resources\\\":{\\\"description\\\":\\\"ResourceRequirements describes the compute resource requirements.\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}}},\\\"selector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"]},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}}},\\\"storageClassName\\\":{\\\"description\\\":\\\"Name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1\\\",\\\"type\\\":\\\"string\\\"},\\\"volumeMode\\\":{\\\"description\\\":\\\"volumeMode defines what type of volume is required by the claim. Value of Filesystem is implied when not included in claim spec. This is a beta feature.\\\",\\\"type\\\":\\\"string\\\"},\\\"volumeName\\\":{\\\"description\\\":\\\"VolumeName is the binding reference to the PersistentVolume backing this claim.\\\",\\\"type\\\":\\\"string\\\"}}},\\\"status\\\":{\\\"description\\\":\\\"PersistentVolumeClaimStatus is the current status of a persistent volume claim.\\\",\\\"properties\\\":{\\\"accessModes\\\":{\\\"description\\\":\\\"AccessModes contains the actual access modes the volume backing the PVC has. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"capacity\\\":{\\\"description\\\":\\\"Represents the actual resources of the underlying volume.\\\",\\\"type\\\":\\\"object\\\"},\\\"conditions\\\":{\\\"description\\\":\\\"Current Condition of persistent volume claim. If underlying persistent volume is being resized then the Condition will be set to 'ResizeStarted'.\\\",\\\"items\\\":{\\\"description\\\":\\\"PersistentVolumeClaimCondition contains details about state of pvc\\\",\\\"properties\\\":{\\\"lastProbeTime\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"lastTransitionTime\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"Human-readable message indicating details about last transition.\\\",\\\"type\\\":\\\"string\\\"},\\\"reason\\\":{\\\"description\\\":\\\"Unique, this should be a short, machine understandable string that gives the reason for condition's last transition. If it reports \\\\\\\"ResizeStarted\\\\\\\" that means the underlying persistent volume is being resized.\\\",\\\"type\\\":\\\"string\\\"},\\\"status\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":{\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"type\\\",\\\"status\\\"]},\\\"type\\\":\\\"array\\\"},\\\"phase\\\":{\\\"description\\\":\\\"Phase represents the current phase of PersistentVolumeClaim.\\\",\\\"type\\\":\\\"string\\\"}}}}}}},\\\"tag\\\":{\\\"description\\\":\\\"Tag of Alertmanager container image to be deployed. Defaults to the value of `version`. Version is ignored if Tag is set.\\\",\\\"type\\\":\\\"string\\\"},\\\"tolerations\\\":{\\\"description\\\":\\\"If specified, the pod's tolerations.\\\",\\\"items\\\":{\\\"description\\\":\\\"The pod this Toleration is attached to tolerates any taint that matches the triple \\\\u003ckey,value,effect\\\\u003e using the matching operator \\\\u003coperator\\\\u003e.\\\",\\\"properties\\\":{\\\"effect\\\":{\\\"description\\\":\\\"Effect indicates the taint effect to match. Empty means match all taint effects. When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.\\\",\\\"type\\\":\\\"string\\\"},\\\"key\\\":{\\\"description\\\":\\\"Key is the taint key that the toleration applies to. Empty means match all taint keys. If the key is empty, operator must be Exists; this combination means to match all values and all keys.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Operator represents a key's relationship to the value. Valid operators are Exists and Equal. Defaults to Equal. Exists is equivalent to wildcard for value, so that a pod can tolerate all taints of a particular category.\\\",\\\"type\\\":\\\"string\\\"},\\\"tolerationSeconds\\\":{\\\"description\\\":\\\"TolerationSeconds represents the period of time the toleration (which must be of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately) by the system.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"value\\\":{\\\"description\\\":\\\"Value is the taint value the toleration matches to. If the operator is Exists, the value should be empty, otherwise just a regular string.\\\",\\\"type\\\":\\\"string\\\"}}},\\\"type\\\":\\\"array\\\"},\\\"version\\\":{\\\"description\\\":\\\"Version the cluster should be on.\\\",\\\"type\\\":\\\"string\\\"}}},\\\"status\\\":{\\\"description\\\":\\\"AlertmanagerStatus is the most recent observed status of the Alertmanager cluster. Read-only. Not included when requesting from the apiserver, only from the Prometheus Operator API itself. More info: https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#spec-and-status\\\",\\\"properties\\\":{\\\"availableReplicas\\\":{\\\"description\\\":\\\"Total number of available pods (ready for at least minReadySeconds) targeted by this Alertmanager cluster.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"paused\\\":{\\\"description\\\":\\\"Represents whether any actions on the underlying managed objects are being performed. Only delete actions will be performed.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"replicas\\\":{\\\"description\\\":\\\"Total number of non-terminated pods targeted by this Alertmanager cluster (their labels match the selector).\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"unavailableReplicas\\\":{\\\"description\\\":\\\"Total number of unavailable pods targeted by this Alertmanager cluster.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"updatedReplicas\\\":{\\\"description\\\":\\\"Total number of non-terminated pods targeted by this Alertmanager cluster that have the desired version spec.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"required\\\":[\\\"paused\\\",\\\"replicas\\\",\\\"updatedReplicas\\\",\\\"availableReplicas\\\",\\\"unavailableReplicas\\\"]}}}},\\\"version\\\":\\\"v1\\\"}}\\n\"\n }\n },\n \"spec\": {\n \"group\": \"monitoring.coreos.com\",\n \"version\": \"v1\",\n \"names\": {\n \"plural\": \"alertmanagers\",\n \"singular\": \"alertmanager\",\n \"kind\": \"Alertmanager\",\n \"listKind\": \"AlertmanagerList\"\n },\n \"scope\": \"Namespaced\",\n \"validation\": {\n \"openAPIV3Schema\": {\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"spec\": {\n \"description\": \"AlertmanagerSpec is a specification of the desired behavior of the Alertmanager cluster. More info: https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#spec-and-status\",\n \"properties\": {\n \"additionalPeers\": {\n \"description\": \"AdditionalPeers allows injecting a set of additional Alertmanagers to peer with to form a highly available cluster.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"affinity\": {\n \"description\": \"Affinity is a group of affinity scheduling rules.\",\n \"properties\": {\n \"nodeAffinity\": {\n \"description\": \"Node affinity is a group of node affinity scheduling rules.\",\n \"properties\": {\n \"preferredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\"weight\\\" to the sum if the node matches the corresponding matchExpressions; the node(s) with the highest sum are the most preferred.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"An empty preferred scheduling term matches all objects with implicit weight 0 (i.e. it's a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op).\",\n \"required\": [\n \"weight\",\n \"preference\"\n ],\n \"properties\": {\n \"preference\": {\n \"description\": \"A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"A list of node selector requirements by node's labels.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchFields\": {\n \"description\": \"A list of node selector requirements by node's fields.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n }\n }\n },\n \"weight\": {\n \"description\": \"Weight associated with matching the corresponding nodeSelectorTerm, in the range 1-100.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n }\n }\n }\n },\n \"requiredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"A node selector represents the union of the results of one or more label queries over a set of nodes; that is, it represents the OR of the selectors represented by the node selector terms.\",\n \"required\": [\n \"nodeSelectorTerms\"\n ],\n \"properties\": {\n \"nodeSelectorTerms\": {\n \"description\": \"Required. A list of node selector terms. The terms are ORed.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"A list of node selector requirements by node's labels.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchFields\": {\n \"description\": \"A list of node selector requirements by node's fields.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n }\n }\n }\n }\n }\n }\n }\n },\n \"podAffinity\": {\n \"description\": \"Pod affinity is a group of inter pod affinity scheduling rules.\",\n \"properties\": {\n \"preferredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\"weight\\\" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)\",\n \"required\": [\n \"weight\",\n \"podAffinityTerm\"\n ],\n \"properties\": {\n \"podAffinityTerm\": {\n \"description\": \"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key matches that of any node on which a pod of the set of pods is running\",\n \"required\": [\n \"topologyKey\"\n ],\n \"properties\": {\n \"labelSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"namespaces\": {\n \"description\": \"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\"this pod's namespace\\\"\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"topologyKey\": {\n \"description\": \"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\",\n \"type\": \"string\"\n }\n }\n },\n \"weight\": {\n \"description\": \"weight associated with matching the corresponding podAffinityTerm, in the range 1-100.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n }\n }\n }\n },\n \"requiredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key matches that of any node on which a pod of the set of pods is running\",\n \"required\": [\n \"topologyKey\"\n ],\n \"properties\": {\n \"labelSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"namespaces\": {\n \"description\": \"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\"this pod's namespace\\\"\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"topologyKey\": {\n \"description\": \"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\",\n \"type\": \"string\"\n }\n }\n }\n }\n }\n },\n \"podAntiAffinity\": {\n \"description\": \"Pod anti affinity is a group of inter pod anti affinity scheduling rules.\",\n \"properties\": {\n \"preferredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"The scheduler will prefer to schedule pods to nodes that satisfy the anti-affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling anti-affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\"weight\\\" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)\",\n \"required\": [\n \"weight\",\n \"podAffinityTerm\"\n ],\n \"properties\": {\n \"podAffinityTerm\": {\n \"description\": \"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key matches that of any node on which a pod of the set of pods is running\",\n \"required\": [\n \"topologyKey\"\n ],\n \"properties\": {\n \"labelSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"namespaces\": {\n \"description\": \"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\"this pod's namespace\\\"\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"topologyKey\": {\n \"description\": \"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\",\n \"type\": \"string\"\n }\n }\n },\n \"weight\": {\n \"description\": \"weight associated with matching the corresponding podAffinityTerm, in the range 1-100.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n }\n }\n }\n },\n \"requiredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"If the anti-affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the anti-affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key matches that of any node on which a pod of the set of pods is running\",\n \"required\": [\n \"topologyKey\"\n ],\n \"properties\": {\n \"labelSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"namespaces\": {\n \"description\": \"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\"this pod's namespace\\\"\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"topologyKey\": {\n \"description\": \"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\",\n \"type\": \"string\"\n }\n }\n }\n }\n }\n }\n }\n },\n \"baseImage\": {\n \"description\": \"Base image that is used to deploy pods, without tag.\",\n \"type\": \"string\"\n },\n \"configMaps\": {\n \"description\": \"ConfigMaps is a list of ConfigMaps in the same namespace as the Alertmanager object, which shall be mounted into the Alertmanager Pods. The ConfigMaps are mounted into /etc/alertmanager/configmaps/.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"configSecret\": {\n \"description\": \"ConfigSecret is the name of a Kubernetes Secret in the same namespace as the Alertmanager object, which contains configuration for this Alertmanager instance. Defaults to 'alertmanager-' The secret is mounted into /etc/alertmanager/config.\",\n \"type\": \"string\"\n },\n \"containers\": {\n \"description\": \"Containers allows injecting additional containers. This is meant to allow adding an authentication proxy to an Alertmanager pod.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A single application container that you want to run within a pod.\",\n \"required\": [\n \"name\"\n ],\n \"properties\": {\n \"args\": {\n \"description\": \"Arguments to the entrypoint. The docker image's CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"command\": {\n \"description\": \"Entrypoint array. Not executed within a shell. The docker image's ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"env\": {\n \"description\": \"List of environment variables to set in the container. Cannot be updated.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"EnvVar represents an environment variable present in a Container.\",\n \"required\": [\n \"name\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the environment variable. Must be a C_IDENTIFIER.\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to \\\"\\\".\",\n \"type\": \"string\"\n },\n \"valueFrom\": {\n \"description\": \"EnvVarSource represents a source for the value of an EnvVar.\",\n \"properties\": {\n \"configMapKeyRef\": {\n \"description\": \"Selects a key from a ConfigMap.\",\n \"required\": [\n \"key\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The key to select.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap or it's key must be defined\",\n \"type\": \"boolean\"\n }\n }\n },\n \"fieldRef\": {\n \"description\": \"ObjectFieldSelector selects an APIVersioned field of an object.\",\n \"required\": [\n \"fieldPath\"\n ],\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"Version of the schema the FieldPath is written in terms of, defaults to \\\"v1\\\".\",\n \"type\": \"string\"\n },\n \"fieldPath\": {\n \"description\": \"Path of the field to select in the specified API version.\",\n \"type\": \"string\"\n }\n }\n },\n \"resourceFieldRef\": {\n \"description\": \"ResourceFieldSelector represents container resources (cpu, memory) and their output format\",\n \"required\": [\n \"resource\"\n ],\n \"properties\": {\n \"containerName\": {\n \"description\": \"Container name: required for volumes, optional for env vars\",\n \"type\": \"string\"\n },\n \"divisor\": {},\n \"resource\": {\n \"description\": \"Required: resource to select\",\n \"type\": \"string\"\n }\n }\n },\n \"secretKeyRef\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"required\": [\n \"key\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n }\n }\n }\n }\n }\n }\n },\n \"envFrom\": {\n \"description\": \"List of sources to populate environment variables in the container. The keys defined within a source must be a C_IDENTIFIER. All invalid keys will be reported as an event when the container is starting. When a key exists in multiple sources, the value associated with the last source will take precedence. Values defined by an Env with a duplicate key will take precedence. Cannot be updated.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"EnvFromSource represents the source of a set of ConfigMaps\",\n \"properties\": {\n \"configMapRef\": {\n \"description\": \"ConfigMapEnvSource selects a ConfigMap to populate the environment variables with.\\nThe contents of the target ConfigMap's Data field will represent the key-value pairs as environment variables.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap must be defined\",\n \"type\": \"boolean\"\n }\n }\n },\n \"prefix\": {\n \"description\": \"An optional identifier to prepend to each key in the ConfigMap. Must be a C_IDENTIFIER.\",\n \"type\": \"string\"\n },\n \"secretRef\": {\n \"description\": \"SecretEnvSource selects a Secret to populate the environment variables with.\\nThe contents of the target Secret's Data field will represent the key-value pairs as environment variables.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret must be defined\",\n \"type\": \"boolean\"\n }\n }\n }\n }\n }\n },\n \"image\": {\n \"description\": \"Docker image name. More info: https://kubernetes.io/docs/concepts/containers/images This field is optional to allow higher level config management to default or override container images in workload controllers like Deployments and StatefulSets.\",\n \"type\": \"string\"\n },\n \"imagePullPolicy\": {\n \"description\": \"Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images\",\n \"type\": \"string\"\n },\n \"lifecycle\": {\n \"description\": \"Lifecycle describes actions that the management system should take in response to container lifecycle events. For the PostStart and PreStop lifecycle handlers, management of the container blocks until the action is complete, unless the container process fails, in which case the handler is aborted.\",\n \"properties\": {\n \"postStart\": {\n \"description\": \"Handler defines a specific action that should be taken\",\n \"properties\": {\n \"exec\": {\n \"description\": \"ExecAction describes a \\\"run in container\\\" action.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n },\n \"httpGet\": {\n \"description\": \"HTTPGetAction describes an action based on HTTP Get requests.\",\n \"required\": [\n \"port\"\n ],\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n }\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocketAction describes an action based on opening a socket\",\n \"required\": [\n \"port\"\n ],\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n }\n }\n }\n }\n },\n \"preStop\": {\n \"description\": \"Handler defines a specific action that should be taken\",\n \"properties\": {\n \"exec\": {\n \"description\": \"ExecAction describes a \\\"run in container\\\" action.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n },\n \"httpGet\": {\n \"description\": \"HTTPGetAction describes an action based on HTTP Get requests.\",\n \"required\": [\n \"port\"\n ],\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n }\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocketAction describes an action based on opening a socket\",\n \"required\": [\n \"port\"\n ],\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n }\n }\n }\n }\n }\n }\n },\n \"livenessProbe\": {\n \"description\": \"Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic.\",\n \"properties\": {\n \"exec\": {\n \"description\": \"ExecAction describes a \\\"run in container\\\" action.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n },\n \"failureThreshold\": {\n \"description\": \"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"httpGet\": {\n \"description\": \"HTTPGetAction describes an action based on HTTP Get requests.\",\n \"required\": [\n \"port\"\n ],\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n }\n },\n \"initialDelaySeconds\": {\n \"description\": \"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"periodSeconds\": {\n \"description\": \"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"successThreshold\": {\n \"description\": \"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocketAction describes an action based on opening a socket\",\n \"required\": [\n \"port\"\n ],\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n }\n }\n },\n \"timeoutSeconds\": {\n \"description\": \"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n }\n }\n },\n \"name\": {\n \"description\": \"Name of the container specified as a DNS_LABEL. Each container in a pod must have a unique name (DNS_LABEL). Cannot be updated.\",\n \"type\": \"string\"\n },\n \"ports\": {\n \"description\": \"List of ports to expose from the container. Exposing a port here gives the system additional information about the network connections a container uses, but is primarily informational. Not specifying a port here DOES NOT prevent that port from being exposed. Any port which is listening on the default \\\"0.0.0.0\\\" address inside a container will be accessible from the network. Cannot be updated.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"ContainerPort represents a network port in a single container.\",\n \"required\": [\n \"containerPort\"\n ],\n \"properties\": {\n \"containerPort\": {\n \"description\": \"Number of port to expose on the pod's IP address. This must be a valid port number, 0 < x < 65536.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"hostIP\": {\n \"description\": \"What host IP to bind the external port to.\",\n \"type\": \"string\"\n },\n \"hostPort\": {\n \"description\": \"Number of port to expose on the host. If specified, this must be a valid port number, 0 < x < 65536. If HostNetwork is specified, this must match ContainerPort. Most containers do not need this.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"name\": {\n \"description\": \"If specified, this must be an IANA_SVC_NAME and unique within the pod. Each named port in a pod must have a unique name. Name for the port that can be referred to by services.\",\n \"type\": \"string\"\n },\n \"protocol\": {\n \"description\": \"Protocol for port. Must be UDP, TCP, or SCTP. Defaults to \\\"TCP\\\".\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"readinessProbe\": {\n \"description\": \"Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic.\",\n \"properties\": {\n \"exec\": {\n \"description\": \"ExecAction describes a \\\"run in container\\\" action.\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n },\n \"failureThreshold\": {\n \"description\": \"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"httpGet\": {\n \"description\": \"HTTPGetAction describes an action based on HTTP Get requests.\",\n \"required\": [\n \"port\"\n ],\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n }\n },\n \"initialDelaySeconds\": {\n \"description\": \"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"periodSeconds\": {\n \"description\": \"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"successThreshold\": {\n \"description\": \"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocketAction describes an action based on opening a socket\",\n \"required\": [\n \"port\"\n ],\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n }\n }\n },\n \"timeoutSeconds\": {\n \"description\": \"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n }\n }\n },\n \"resources\": {\n \"description\": \"ResourceRequirements describes the compute resource requirements.\",\n \"properties\": {\n \"limits\": {\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n }\n },\n \"securityContext\": {\n \"description\": \"SecurityContext holds security configuration that will be applied to a container. Some fields are present in both SecurityContext and PodSecurityContext. When both are set, the values in SecurityContext take precedence.\",\n \"properties\": {\n \"allowPrivilegeEscalation\": {\n \"description\": \"AllowPrivilegeEscalation controls whether a process can gain more privileges than its parent process. This bool directly controls if the no_new_privs flag will be set on the container process. AllowPrivilegeEscalation is true always when the container is: 1) run as Privileged 2) has CAP_SYS_ADMIN\",\n \"type\": \"boolean\"\n },\n \"capabilities\": {\n \"description\": \"Adds and removes POSIX capabilities from running containers.\",\n \"properties\": {\n \"add\": {\n \"description\": \"Added capabilities\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"drop\": {\n \"description\": \"Removed capabilities\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n },\n \"privileged\": {\n \"description\": \"Run container in privileged mode. Processes in privileged containers are essentially equivalent to root on the host. Defaults to false.\",\n \"type\": \"boolean\"\n },\n \"procMount\": {\n \"description\": \"procMount denotes the type of proc mount to use for the containers. The default is DefaultProcMount which uses the container runtime defaults for readonly paths and masked paths. This requires the ProcMountType feature flag to be enabled.\",\n \"type\": \"string\"\n },\n \"readOnlyRootFilesystem\": {\n \"description\": \"Whether this container has a read-only root filesystem. Default is false.\",\n \"type\": \"boolean\"\n },\n \"runAsGroup\": {\n \"description\": \"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"runAsNonRoot\": {\n \"description\": \"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"type\": \"boolean\"\n },\n \"runAsUser\": {\n \"description\": \"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"seLinuxOptions\": {\n \"description\": \"SELinuxOptions are the labels to be applied to the container\",\n \"properties\": {\n \"level\": {\n \"description\": \"Level is SELinux level label that applies to the container.\",\n \"type\": \"string\"\n },\n \"role\": {\n \"description\": \"Role is a SELinux role label that applies to the container.\",\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"Type is a SELinux type label that applies to the container.\",\n \"type\": \"string\"\n },\n \"user\": {\n \"description\": \"User is a SELinux user label that applies to the container.\",\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"stdin\": {\n \"description\": \"Whether this container should allocate a buffer for stdin in the container runtime. If this is not set, reads from stdin in the container will always result in EOF. Default is false.\",\n \"type\": \"boolean\"\n },\n \"stdinOnce\": {\n \"description\": \"Whether the container runtime should close the stdin channel after it has been opened by a single attach. When stdin is true the stdin stream will remain open across multiple attach sessions. If stdinOnce is set to true, stdin is opened on container start, is empty until the first client attaches to stdin, and then remains open and accepts data until the client disconnects, at which time stdin is closed and remains closed until the container is restarted. If this flag is false, a container processes that reads from stdin will never receive an EOF. Default is false\",\n \"type\": \"boolean\"\n },\n \"terminationMessagePath\": {\n \"description\": \"Optional: Path at which the file to which the container's termination message will be written is mounted into the container's filesystem. Message written is intended to be brief final status, such as an assertion failure message. Will be truncated by the node if greater than 4096 bytes. The total message length across all containers will be limited to 12kb. Defaults to /dev/termination-log. Cannot be updated.\",\n \"type\": \"string\"\n },\n \"terminationMessagePolicy\": {\n \"description\": \"Indicate how the termination message should be populated. File will use the contents of terminationMessagePath to populate the container status message on both success and failure. FallbackToLogsOnError will use the last chunk of container log output if the termination message file is empty and the container exited with an error. The log output is limited to 2048 bytes or 80 lines, whichever is smaller. Defaults to File. Cannot be updated.\",\n \"type\": \"string\"\n },\n \"tty\": {\n \"description\": \"Whether this container should allocate a TTY for itself, also requires 'stdin' to be true. Default is false.\",\n \"type\": \"boolean\"\n },\n \"volumeDevices\": {\n \"description\": \"volumeDevices is the list of block devices to be used by the container. This is a beta feature.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"volumeDevice describes a mapping of a raw block device within a container.\",\n \"required\": [\n \"name\",\n \"devicePath\"\n ],\n \"properties\": {\n \"devicePath\": {\n \"description\": \"devicePath is the path inside of the container that the device will be mapped to.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"name must match the name of a persistentVolumeClaim in the pod\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"volumeMounts\": {\n \"description\": \"Pod volumes to mount into the container's filesystem. Cannot be updated.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"VolumeMount describes a mounting of a Volume within a container.\",\n \"required\": [\n \"name\",\n \"mountPath\"\n ],\n \"properties\": {\n \"mountPath\": {\n \"description\": \"Path within the container at which the volume should be mounted. Must not contain ':'.\",\n \"type\": \"string\"\n },\n \"mountPropagation\": {\n \"description\": \"mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"This must match the Name of a Volume.\",\n \"type\": \"string\"\n },\n \"readOnly\": {\n \"description\": \"Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.\",\n \"type\": \"boolean\"\n },\n \"subPath\": {\n \"description\": \"Path within the volume from which the container's volume should be mounted. Defaults to \\\"\\\" (volume's root).\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"workingDir\": {\n \"description\": \"Container's working directory. If not specified, the container runtime's default will be used, which might be configured in the container image. Cannot be updated.\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"externalUrl\": {\n \"description\": \"The external URL the Alertmanager instances will be available under. This is necessary to generate correct URLs. This is necessary if Alertmanager is not served from root of a DNS name.\",\n \"type\": \"string\"\n },\n \"image\": {\n \"description\": \"Image if specified has precedence over baseImage, tag and sha combinations. Specifying the version is still necessary to ensure the Prometheus Operator knows what version of Alertmanager is being configured.\",\n \"type\": \"string\"\n },\n \"imagePullSecrets\": {\n \"description\": \"An optional list of references to secrets in the same namespace to use for pulling prometheus and alertmanager images from registries see http://kubernetes.io/docs/user-guide/images#specifying-imagepullsecrets-on-a-pod\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"LocalObjectReference contains enough information to let you locate the referenced object inside the same namespace.\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"listenLocal\": {\n \"description\": \"ListenLocal makes the Alertmanager server listen on loopback, so that it does not bind against the Pod IP. Note this is only for the Alertmanager UI, not the gossip communication.\",\n \"type\": \"boolean\"\n },\n \"logLevel\": {\n \"description\": \"Log level for Alertmanager to be configured with.\",\n \"type\": \"string\"\n },\n \"nodeSelector\": {\n \"description\": \"Define which Nodes the Pods are scheduled on.\",\n \"type\": \"object\"\n },\n \"paused\": {\n \"description\": \"If set to true all actions on the underlying managed objects are not goint to be performed, except for delete actions.\",\n \"type\": \"boolean\"\n },\n \"podMetadata\": {\n \"description\": \"ObjectMeta is metadata that all persisted resources must have, which includes all objects users must create.\",\n \"properties\": {\n \"annotations\": {\n \"description\": \"Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations\",\n \"type\": \"object\"\n },\n \"clusterName\": {\n \"description\": \"The name of the cluster which the object belongs to. This is used to distinguish resources with same name and namespace in different clusters. This field is not set anywhere right now and apiserver is going to ignore it if set in create or update request.\",\n \"type\": \"string\"\n },\n \"creationTimestamp\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"type\": \"string\",\n \"format\": \"date-time\"\n },\n \"deletionGracePeriodSeconds\": {\n \"description\": \"Number of seconds allowed for this object to gracefully terminate before it will be removed from the system. Only set when deletionTimestamp is also set. May only be shortened. Read-only.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"deletionTimestamp\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"type\": \"string\",\n \"format\": \"date-time\"\n },\n \"finalizers\": {\n \"description\": \"Must be empty before the object is deleted from the registry. Each entry is an identifier for the responsible component that will remove the entry from the list. If the deletionTimestamp of the object is non-nil, entries in this list can only be removed.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"generateName\": {\n \"description\": \"GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server.\\nIf this field is specified and the generated name exists, the server will NOT return a 409 - instead, it will either return 201 Created or 500 with Reason ServerTimeout indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header).\\nApplied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#idempotency\",\n \"type\": \"string\"\n },\n \"generation\": {\n \"description\": \"A sequence number representing a specific generation of the desired state. Populated by the system. Read-only.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"initializers\": {\n \"description\": \"Initializers tracks the progress of initialization.\",\n \"required\": [\n \"pending\"\n ],\n \"properties\": {\n \"pending\": {\n \"description\": \"Pending is a list of initializers that must execute in order before this object is visible. When the last pending initializer is removed, and no failing result is set, the initializers struct will be set to nil and the object is considered as initialized and visible to all clients.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"Initializer is information about an initializer that has not yet completed.\",\n \"required\": [\n \"name\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"name of the process that is responsible for initializing this object.\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"result\": {\n \"description\": \"Status is a return value for calls that don't return other objects.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"code\": {\n \"description\": \"Suggested HTTP return code for this status, 0 if not set.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"details\": {\n \"description\": \"StatusDetails is a set of additional properties that MAY be set by the server to provide additional information about a response. The Reason field of a Status object defines what attributes will be set. Clients must ignore fields that do not match the defined type of each attribute, and should assume that any attribute may be empty, invalid, or under defined.\",\n \"properties\": {\n \"causes\": {\n \"description\": \"The Causes array includes more details associated with the StatusReason failure. Not all StatusReasons may provide detailed causes.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"StatusCause provides more information about an api.Status failure, including cases when multiple errors are encountered.\",\n \"properties\": {\n \"field\": {\n \"description\": \"The field of the resource that has caused this error, as named by its JSON serialization. May include dot and postfix notation for nested attributes. Arrays are zero-indexed. Fields may appear more than once in an array of causes due to fields having multiple errors. Optional.\\nExamples:\\n \\\"name\\\" - the field \\\"name\\\" on the current resource\\n \\\"items[0].name\\\" - the field \\\"name\\\" on the first array entry in \\\"items\\\"\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"A human-readable description of the cause of the error. This field may be presented as-is to a reader.\",\n \"type\": \"string\"\n },\n \"reason\": {\n \"description\": \"A machine-readable description of the cause of the error. If this value is empty there is no information available.\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"group\": {\n \"description\": \"The group attribute of the resource associated with the status StatusReason.\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"The kind attribute of the resource associated with the status StatusReason. On some operations may differ from the requested resource Kind. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"The name attribute of the resource associated with the status StatusReason (when there is a single name which can be described).\",\n \"type\": \"string\"\n },\n \"retryAfterSeconds\": {\n \"description\": \"If specified, the time in seconds before the operation should be retried. Some errors may indicate the client must take an alternate action - for those errors this field may indicate how long to wait before taking the alternate action.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"uid\": {\n \"description\": \"UID of the resource. (when there is a single resource which can be described). More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n }\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"A human-readable description of the status of this operation.\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"description\": \"ListMeta describes metadata that synthetic resources must have, including lists and various status objects. A resource may have only one of {ObjectMeta, ListMeta}.\",\n \"properties\": {\n \"continue\": {\n \"description\": \"continue may be set if the user set a limit on the number of items returned, and indicates that the server has more data available. The value is opaque and may be used to issue another request to the endpoint that served this list to retrieve the next set of available objects. Continuing a consistent list may not be possible if the server configuration has changed or more than a few minutes have passed. The resourceVersion field returned when using this continue value will be identical to the value in the first response, unless you have received this token from an error message.\",\n \"type\": \"string\"\n },\n \"resourceVersion\": {\n \"description\": \"String that identifies the server's internal version of this object that can be used by clients to determine when objects have changed. Value must be treated as opaque by clients and passed unmodified back to the server. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\",\n \"type\": \"string\"\n },\n \"selfLink\": {\n \"description\": \"selfLink is a URL representing this object. Populated by the system. Read-only.\",\n \"type\": \"string\"\n }\n }\n },\n \"reason\": {\n \"description\": \"A machine-readable description of why this operation is in the \\\"Failure\\\" status. If this value is empty there is no information available. A Reason clarifies an HTTP status code but does not override it.\",\n \"type\": \"string\"\n },\n \"status\": {\n \"description\": \"Status of the operation. One of: \\\"Success\\\" or \\\"Failure\\\". More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status\",\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"labels\": {\n \"description\": \"Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels\",\n \"type\": \"object\"\n },\n \"name\": {\n \"description\": \"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names\",\n \"type\": \"string\"\n },\n \"namespace\": {\n \"description\": \"Namespace defines the space within each name must be unique. An empty namespace is equivalent to the \\\"default\\\" namespace, but \\\"default\\\" is the canonical representation. Not all objects are required to be scoped to a namespace - the value of this field for those objects will be empty.\\nMust be a DNS_LABEL. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/namespaces\",\n \"type\": \"string\"\n },\n \"ownerReferences\": {\n \"description\": \"List of objects depended by this object. If ALL objects in the list have been deleted, this object will be garbage collected. If this object is managed by a controller, then an entry in this list will point to this controller, with the controller field set to true. There cannot be more than one managing controller.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"OwnerReference contains enough information to let you identify an owning object. An owning object must be in the same namespace as the dependent, or be cluster-scoped, so there is no namespace field.\",\n \"required\": [\n \"apiVersion\",\n \"kind\",\n \"name\",\n \"uid\"\n ],\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"API version of the referent.\",\n \"type\": \"string\"\n },\n \"blockOwnerDeletion\": {\n \"description\": \"If true, AND if the owner has the \\\"foregroundDeletion\\\" finalizer, then the owner cannot be deleted from the key-value store until this reference is removed. Defaults to false. To set this field, a user needs \\\"delete\\\" permission of the owner, otherwise 422 (Unprocessable Entity) will be returned.\",\n \"type\": \"boolean\"\n },\n \"controller\": {\n \"description\": \"If true, this reference points to the managing controller.\",\n \"type\": \"boolean\"\n },\n \"kind\": {\n \"description\": \"Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#names\",\n \"type\": \"string\"\n },\n \"uid\": {\n \"description\": \"UID of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"resourceVersion\": {\n \"description\": \"An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources.\\nPopulated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\",\n \"type\": \"string\"\n },\n \"selfLink\": {\n \"description\": \"SelfLink is a URL representing this object. Populated by the system. Read-only.\",\n \"type\": \"string\"\n },\n \"uid\": {\n \"description\": \"UID is the unique in time and space value for this object. It is typically generated by the server on successful creation of a resource and is not allowed to change on PUT operations.\\nPopulated by the system. Read-only. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n }\n },\n \"priorityClassName\": {\n \"description\": \"Priority class assigned to the Pods\",\n \"type\": \"string\"\n },\n \"replicas\": {\n \"description\": \"Size is the expected size of the alertmanager cluster. The controller will eventually make the size of the running cluster equal to the expected size.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"resources\": {\n \"description\": \"ResourceRequirements describes the compute resource requirements.\",\n \"properties\": {\n \"limits\": {\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n }\n },\n \"retention\": {\n \"description\": \"Time duration Alertmanager shall retain data for. Default is '120h', and must match the regular expression `[0-9]+(ms|s|m|h)` (milliseconds seconds minutes hours).\",\n \"type\": \"string\"\n },\n \"routePrefix\": {\n \"description\": \"The route prefix Alertmanager registers HTTP handlers for. This is useful, if using ExternalURL and a proxy is rewriting HTTP routes of a request, and the actual ExternalURL is still true, but the server serves requests under a different route prefix. For example for use with `kubectl proxy`.\",\n \"type\": \"string\"\n },\n \"secrets\": {\n \"description\": \"Secrets is a list of Secrets in the same namespace as the Alertmanager object, which shall be mounted into the Alertmanager Pods. The Secrets are mounted into /etc/alertmanager/secrets/.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"securityContext\": {\n \"description\": \"PodSecurityContext holds pod-level security attributes and common container settings. Some fields are also present in container.securityContext. Field values of container.securityContext take precedence over field values of PodSecurityContext.\",\n \"properties\": {\n \"fsGroup\": {\n \"description\": \"A special supplemental group that applies to all containers in a pod. Some volume types allow the Kubelet to change the ownership of that volume to be owned by the pod:\\n1. The owning GID will be the FSGroup 2. The setgid bit is set (new files created in the volume will be owned by FSGroup) 3. The permission bits are OR'd with rw-rw----\\nIf unset, the Kubelet will not modify the ownership and permissions of any volume.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"runAsGroup\": {\n \"description\": \"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"runAsNonRoot\": {\n \"description\": \"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"type\": \"boolean\"\n },\n \"runAsUser\": {\n \"description\": \"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"seLinuxOptions\": {\n \"description\": \"SELinuxOptions are the labels to be applied to the container\",\n \"properties\": {\n \"level\": {\n \"description\": \"Level is SELinux level label that applies to the container.\",\n \"type\": \"string\"\n },\n \"role\": {\n \"description\": \"Role is a SELinux role label that applies to the container.\",\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"Type is a SELinux type label that applies to the container.\",\n \"type\": \"string\"\n },\n \"user\": {\n \"description\": \"User is a SELinux user label that applies to the container.\",\n \"type\": \"string\"\n }\n }\n },\n \"supplementalGroups\": {\n \"description\": \"A list of groups applied to the first process run in each container, in addition to the container's primary GID. If unspecified, no groups will be added to any container.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"integer\",\n \"format\": \"int64\"\n }\n },\n \"sysctls\": {\n \"description\": \"Sysctls hold a list of namespaced sysctls used for the pod. Pods with unsupported sysctls (by the container runtime) might fail to launch.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"Sysctl defines a kernel parameter to be set\",\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"Name of a property to set\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"Value of a property to set\",\n \"type\": \"string\"\n }\n }\n }\n }\n }\n },\n \"serviceAccountName\": {\n \"description\": \"ServiceAccountName is the name of the ServiceAccount to use to run the Prometheus Pods.\",\n \"type\": \"string\"\n },\n \"sha\": {\n \"description\": \"SHA of Alertmanager container image to be deployed. Defaults to the value of `version`. Similar to a tag, but the SHA explicitly deploys an immutable container image. Version and Tag are ignored if SHA is set.\",\n \"type\": \"string\"\n },\n \"storage\": {\n \"description\": \"StorageSpec defines the configured storage for a group Prometheus servers. If neither `emptyDir` nor `volumeClaimTemplate` is specified, then by default an [EmptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir) will be used.\",\n \"properties\": {\n \"emptyDir\": {\n \"description\": \"Represents an empty directory for a pod. Empty directory volumes support ownership management and SELinux relabeling.\",\n \"properties\": {\n \"medium\": {\n \"description\": \"What type of storage medium should back this directory. The default is \\\"\\\" which means to use the node's default medium. Must be an empty string (default) or Memory. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir\",\n \"type\": \"string\"\n },\n \"sizeLimit\": {}\n }\n },\n \"volumeClaimTemplate\": {\n \"description\": \"PersistentVolumeClaim is a user's request for and claim to a persistent volume\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"description\": \"ObjectMeta is metadata that all persisted resources must have, which includes all objects users must create.\",\n \"properties\": {\n \"annotations\": {\n \"description\": \"Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations\",\n \"type\": \"object\"\n },\n \"clusterName\": {\n \"description\": \"The name of the cluster which the object belongs to. This is used to distinguish resources with same name and namespace in different clusters. This field is not set anywhere right now and apiserver is going to ignore it if set in create or update request.\",\n \"type\": \"string\"\n },\n \"creationTimestamp\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"type\": \"string\",\n \"format\": \"date-time\"\n },\n \"deletionGracePeriodSeconds\": {\n \"description\": \"Number of seconds allowed for this object to gracefully terminate before it will be removed from the system. Only set when deletionTimestamp is also set. May only be shortened. Read-only.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"deletionTimestamp\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"type\": \"string\",\n \"format\": \"date-time\"\n },\n \"finalizers\": {\n \"description\": \"Must be empty before the object is deleted from the registry. Each entry is an identifier for the responsible component that will remove the entry from the list. If the deletionTimestamp of the object is non-nil, entries in this list can only be removed.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"generateName\": {\n \"description\": \"GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server.\\nIf this field is specified and the generated name exists, the server will NOT return a 409 - instead, it will either return 201 Created or 500 with Reason ServerTimeout indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header).\\nApplied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#idempotency\",\n \"type\": \"string\"\n },\n \"generation\": {\n \"description\": \"A sequence number representing a specific generation of the desired state. Populated by the system. Read-only.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"initializers\": {\n \"description\": \"Initializers tracks the progress of initialization.\",\n \"required\": [\n \"pending\"\n ],\n \"properties\": {\n \"pending\": {\n \"description\": \"Pending is a list of initializers that must execute in order before this object is visible. When the last pending initializer is removed, and no failing result is set, the initializers struct will be set to nil and the object is considered as initialized and visible to all clients.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"Initializer is information about an initializer that has not yet completed.\",\n \"required\": [\n \"name\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"name of the process that is responsible for initializing this object.\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"result\": {\n \"description\": \"Status is a return value for calls that don't return other objects.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"code\": {\n \"description\": \"Suggested HTTP return code for this status, 0 if not set.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"details\": {\n \"description\": \"StatusDetails is a set of additional properties that MAY be set by the server to provide additional information about a response. The Reason field of a Status object defines what attributes will be set. Clients must ignore fields that do not match the defined type of each attribute, and should assume that any attribute may be empty, invalid, or under defined.\",\n \"properties\": {\n \"causes\": {\n \"description\": \"The Causes array includes more details associated with the StatusReason failure. Not all StatusReasons may provide detailed causes.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"StatusCause provides more information about an api.Status failure, including cases when multiple errors are encountered.\",\n \"properties\": {\n \"field\": {\n \"description\": \"The field of the resource that has caused this error, as named by its JSON serialization. May include dot and postfix notation for nested attributes. Arrays are zero-indexed. Fields may appear more than once in an array of causes due to fields having multiple errors. Optional.\\nExamples:\\n \\\"name\\\" - the field \\\"name\\\" on the current resource\\n \\\"items[0].name\\\" - the field \\\"name\\\" on the first array entry in \\\"items\\\"\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"A human-readable description of the cause of the error. This field may be presented as-is to a reader.\",\n \"type\": \"string\"\n },\n \"reason\": {\n \"description\": \"A machine-readable description of the cause of the error. If this value is empty there is no information available.\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"group\": {\n \"description\": \"The group attribute of the resource associated with the status StatusReason.\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"The kind attribute of the resource associated with the status StatusReason. On some operations may differ from the requested resource Kind. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"The name attribute of the resource associated with the status StatusReason (when there is a single name which can be described).\",\n \"type\": \"string\"\n },\n \"retryAfterSeconds\": {\n \"description\": \"If specified, the time in seconds before the operation should be retried. Some errors may indicate the client must take an alternate action - for those errors this field may indicate how long to wait before taking the alternate action.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"uid\": {\n \"description\": \"UID of the resource. (when there is a single resource which can be described). More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n }\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"A human-readable description of the status of this operation.\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"description\": \"ListMeta describes metadata that synthetic resources must have, including lists and various status objects. A resource may have only one of {ObjectMeta, ListMeta}.\",\n \"properties\": {\n \"continue\": {\n \"description\": \"continue may be set if the user set a limit on the number of items returned, and indicates that the server has more data available. The value is opaque and may be used to issue another request to the endpoint that served this list to retrieve the next set of available objects. Continuing a consistent list may not be possible if the server configuration has changed or more than a few minutes have passed. The resourceVersion field returned when using this continue value will be identical to the value in the first response, unless you have received this token from an error message.\",\n \"type\": \"string\"\n },\n \"resourceVersion\": {\n \"description\": \"String that identifies the server's internal version of this object that can be used by clients to determine when objects have changed. Value must be treated as opaque by clients and passed unmodified back to the server. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\",\n \"type\": \"string\"\n },\n \"selfLink\": {\n \"description\": \"selfLink is a URL representing this object. Populated by the system. Read-only.\",\n \"type\": \"string\"\n }\n }\n },\n \"reason\": {\n \"description\": \"A machine-readable description of why this operation is in the \\\"Failure\\\" status. If this value is empty there is no information available. A Reason clarifies an HTTP status code but does not override it.\",\n \"type\": \"string\"\n },\n \"status\": {\n \"description\": \"Status of the operation. One of: \\\"Success\\\" or \\\"Failure\\\". More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status\",\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"labels\": {\n \"description\": \"Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels\",\n \"type\": \"object\"\n },\n \"name\": {\n \"description\": \"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names\",\n \"type\": \"string\"\n },\n \"namespace\": {\n \"description\": \"Namespace defines the space within each name must be unique. An empty namespace is equivalent to the \\\"default\\\" namespace, but \\\"default\\\" is the canonical representation. Not all objects are required to be scoped to a namespace - the value of this field for those objects will be empty.\\nMust be a DNS_LABEL. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/namespaces\",\n \"type\": \"string\"\n },\n \"ownerReferences\": {\n \"description\": \"List of objects depended by this object. If ALL objects in the list have been deleted, this object will be garbage collected. If this object is managed by a controller, then an entry in this list will point to this controller, with the controller field set to true. There cannot be more than one managing controller.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"OwnerReference contains enough information to let you identify an owning object. An owning object must be in the same namespace as the dependent, or be cluster-scoped, so there is no namespace field.\",\n \"required\": [\n \"apiVersion\",\n \"kind\",\n \"name\",\n \"uid\"\n ],\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"API version of the referent.\",\n \"type\": \"string\"\n },\n \"blockOwnerDeletion\": {\n \"description\": \"If true, AND if the owner has the \\\"foregroundDeletion\\\" finalizer, then the owner cannot be deleted from the key-value store until this reference is removed. Defaults to false. To set this field, a user needs \\\"delete\\\" permission of the owner, otherwise 422 (Unprocessable Entity) will be returned.\",\n \"type\": \"boolean\"\n },\n \"controller\": {\n \"description\": \"If true, this reference points to the managing controller.\",\n \"type\": \"boolean\"\n },\n \"kind\": {\n \"description\": \"Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#names\",\n \"type\": \"string\"\n },\n \"uid\": {\n \"description\": \"UID of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"resourceVersion\": {\n \"description\": \"An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources.\\nPopulated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\",\n \"type\": \"string\"\n },\n \"selfLink\": {\n \"description\": \"SelfLink is a URL representing this object. Populated by the system. Read-only.\",\n \"type\": \"string\"\n },\n \"uid\": {\n \"description\": \"UID is the unique in time and space value for this object. It is typically generated by the server on successful creation of a resource and is not allowed to change on PUT operations.\\nPopulated by the system. Read-only. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n }\n },\n \"spec\": {\n \"description\": \"PersistentVolumeClaimSpec describes the common attributes of storage devices and allows a Source for provider-specific attributes\",\n \"properties\": {\n \"accessModes\": {\n \"description\": \"AccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"dataSource\": {\n \"description\": \"TypedLocalObjectReference contains enough information to let you locate the typed referenced object inside the same namespace.\",\n \"required\": [\n \"kind\",\n \"name\"\n ],\n \"properties\": {\n \"apiGroup\": {\n \"description\": \"APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is the type of resource being referenced\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name is the name of resource being referenced\",\n \"type\": \"string\"\n }\n }\n },\n \"resources\": {\n \"description\": \"ResourceRequirements describes the compute resource requirements.\",\n \"properties\": {\n \"limits\": {\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n }\n },\n \"selector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"storageClassName\": {\n \"description\": \"Name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1\",\n \"type\": \"string\"\n },\n \"volumeMode\": {\n \"description\": \"volumeMode defines what type of volume is required by the claim. Value of Filesystem is implied when not included in claim spec. This is a beta feature.\",\n \"type\": \"string\"\n },\n \"volumeName\": {\n \"description\": \"VolumeName is the binding reference to the PersistentVolume backing this claim.\",\n \"type\": \"string\"\n }\n }\n },\n \"status\": {\n \"description\": \"PersistentVolumeClaimStatus is the current status of a persistent volume claim.\",\n \"properties\": {\n \"accessModes\": {\n \"description\": \"AccessModes contains the actual access modes the volume backing the PVC has. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"capacity\": {\n \"description\": \"Represents the actual resources of the underlying volume.\",\n \"type\": \"object\"\n },\n \"conditions\": {\n \"description\": \"Current Condition of persistent volume claim. If underlying persistent volume is being resized then the Condition will be set to 'ResizeStarted'.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"PersistentVolumeClaimCondition contains details about state of pvc\",\n \"required\": [\n \"type\",\n \"status\"\n ],\n \"properties\": {\n \"lastProbeTime\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"type\": \"string\",\n \"format\": \"date-time\"\n },\n \"lastTransitionTime\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"type\": \"string\",\n \"format\": \"date-time\"\n },\n \"message\": {\n \"description\": \"Human-readable message indicating details about last transition.\",\n \"type\": \"string\"\n },\n \"reason\": {\n \"description\": \"Unique, this should be a short, machine understandable string that gives the reason for condition's last transition. If it reports \\\"ResizeStarted\\\" that means the underlying persistent volume is being resized.\",\n \"type\": \"string\"\n },\n \"status\": {\n \"type\": \"string\"\n },\n \"type\": {\n \"type\": \"string\"\n }\n }\n }\n },\n \"phase\": {\n \"description\": \"Phase represents the current phase of PersistentVolumeClaim.\",\n \"type\": \"string\"\n }\n }\n }\n }\n }\n }\n },\n \"tag\": {\n \"description\": \"Tag of Alertmanager container image to be deployed. Defaults to the value of `version`. Version is ignored if Tag is set.\",\n \"type\": \"string\"\n },\n \"tolerations\": {\n \"description\": \"If specified, the pod's tolerations.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"The pod this Toleration is attached to tolerates any taint that matches the triple using the matching operator .\",\n \"properties\": {\n \"effect\": {\n \"description\": \"Effect indicates the taint effect to match. Empty means match all taint effects. When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.\",\n \"type\": \"string\"\n },\n \"key\": {\n \"description\": \"Key is the taint key that the toleration applies to. Empty means match all taint keys. If the key is empty, operator must be Exists; this combination means to match all values and all keys.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Operator represents a key's relationship to the value. Valid operators are Exists and Equal. Defaults to Equal. Exists is equivalent to wildcard for value, so that a pod can tolerate all taints of a particular category.\",\n \"type\": \"string\"\n },\n \"tolerationSeconds\": {\n \"description\": \"TolerationSeconds represents the period of time the toleration (which must be of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately) by the system.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"value\": {\n \"description\": \"Value is the taint value the toleration matches to. If the operator is Exists, the value should be empty, otherwise just a regular string.\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"version\": {\n \"description\": \"Version the cluster should be on.\",\n \"type\": \"string\"\n }\n }\n },\n \"status\": {\n \"description\": \"AlertmanagerStatus is the most recent observed status of the Alertmanager cluster. Read-only. Not included when requesting from the apiserver, only from the Prometheus Operator API itself. More info: https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#spec-and-status\",\n \"required\": [\n \"paused\",\n \"replicas\",\n \"updatedReplicas\",\n \"availableReplicas\",\n \"unavailableReplicas\"\n ],\n \"properties\": {\n \"availableReplicas\": {\n \"description\": \"Total number of available pods (ready for at least minReadySeconds) targeted by this Alertmanager cluster.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"paused\": {\n \"description\": \"Represents whether any actions on the underlying managed objects are being performed. Only delete actions will be performed.\",\n \"type\": \"boolean\"\n },\n \"replicas\": {\n \"description\": \"Total number of non-terminated pods targeted by this Alertmanager cluster (their labels match the selector).\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"unavailableReplicas\": {\n \"description\": \"Total number of unavailable pods targeted by this Alertmanager cluster.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"updatedReplicas\": {\n \"description\": \"Total number of non-terminated pods targeted by this Alertmanager cluster that have the desired version spec.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n }\n }\n }\n }\n }\n },\n \"versions\": [\n {\n \"name\": \"v1\",\n \"served\": true,\n \"storage\": true\n }\n ],\n \"conversion\": {\n \"strategy\": \"None\"\n },\n \"preserveUnknownFields\": true\n },\n \"status\": {\n \"conditions\": [\n {\n \"type\": \"NamesAccepted\",\n \"status\": \"True\",\n \"lastTransitionTime\": \"2020-05-05T16:51:39Z\",\n \"reason\": \"NoConflicts\",\n \"message\": \"no conflicts found\"\n },\n {\n \"type\": \"Established\",\n \"status\": \"True\",\n \"lastTransitionTime\": \"2020-05-05T16:51:39Z\",\n \"reason\": \"InitialNamesAccepted\",\n \"message\": \"the initial names have been accepted\"\n },\n {\n \"type\": \"NonStructuralSchema\",\n \"status\": \"True\",\n \"lastTransitionTime\": \"2020-05-05T16:51:39Z\",\n \"reason\": \"Violations\",\n \"message\": \"[spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[nodeAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.properties[preference].properties[matchExpressions].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[nodeAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.properties[preference].properties[matchFields].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[nodeAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.properties[preference].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[nodeAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[nodeAffinity].properties[requiredDuringSchedulingIgnoredDuringExecution].properties[nodeSelectorTerms].items.properties[matchExpressions].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[nodeAffinity].properties[requiredDuringSchedulingIgnoredDuringExecution].properties[nodeSelectorTerms].items.properties[matchFields].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[nodeAffinity].properties[requiredDuringSchedulingIgnoredDuringExecution].properties[nodeSelectorTerms].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[nodeAffinity].properties[requiredDuringSchedulingIgnoredDuringExecution].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[nodeAffinity].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.properties[podAffinityTerm].properties[labelSelector].properties[matchExpressions].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.properties[podAffinityTerm].properties[labelSelector].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.properties[podAffinityTerm].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAffinity].properties[requiredDuringSchedulingIgnoredDuringExecution].items.properties[labelSelector].properties[matchExpressions].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAffinity].properties[requiredDuringSchedulingIgnoredDuringExecution].items.properties[labelSelector].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAffinity].properties[requiredDuringSchedulingIgnoredDuringExecution].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAffinity].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAntiAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.properties[podAffinityTerm].properties[labelSelector].properties[matchExpressions].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAntiAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.properties[podAffinityTerm].properties[labelSelector].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAntiAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.properties[podAffinityTerm].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAntiAffinity].properties[preferredDuringSchedulingIgnoredDuringExecution].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAntiAffinity].properties[requiredDuringSchedulingIgnoredDuringExecution].items.properties[labelSelector].properties[matchExpressions].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAntiAffinity].properties[requiredDuringSchedulingIgnoredDuringExecution].items.properties[labelSelector].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAntiAffinity].properties[requiredDuringSchedulingIgnoredDuringExecution].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].properties[podAntiAffinity].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[affinity].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[envFrom].items.properties[configMapRef].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[envFrom].items.properties[secretRef].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[envFrom].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[env].items.properties[valueFrom].properties[configMapKeyRef].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[env].items.properties[valueFrom].properties[fieldRef].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[env].items.properties[valueFrom].properties[resourceFieldRef].properties[divisor].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[env].items.properties[valueFrom].properties[resourceFieldRef].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[env].items.properties[valueFrom].properties[secretKeyRef].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[env].items.properties[valueFrom].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[env].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[exec].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[httpGet].properties[httpHeaders].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[httpGet].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[httpGet].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[httpGet].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[httpGet].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[tcpSocket].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[tcpSocket].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[tcpSocket].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[tcpSocket].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[exec].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[httpGet].properties[httpHeaders].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[httpGet].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[httpGet].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[httpGet].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[httpGet].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[tcpSocket].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[tcpSocket].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[tcpSocket].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[tcpSocket].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[exec].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[httpGet].properties[httpHeaders].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[httpGet].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[httpGet].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[httpGet].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[httpGet].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[tcpSocket].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[tcpSocket].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[tcpSocket].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[tcpSocket].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[ports].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[exec].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[httpGet].properties[httpHeaders].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[httpGet].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[httpGet].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[httpGet].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[httpGet].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[tcpSocket].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[tcpSocket].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[tcpSocket].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[tcpSocket].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[resources].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[securityContext].properties[capabilities].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[securityContext].properties[seLinuxOptions].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[securityContext].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[volumeDevices].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[volumeMounts].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[imagePullSecrets].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[podMetadata].properties[initializers].properties[pending].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[podMetadata].properties[initializers].properties[result].properties[details].properties[causes].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[podMetadata].properties[initializers].properties[result].properties[details].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[podMetadata].properties[initializers].properties[result].properties[metadata].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[podMetadata].properties[initializers].properties[result].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[podMetadata].properties[initializers].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[podMetadata].properties[ownerReferences].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[podMetadata].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[resources].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[securityContext].properties[seLinuxOptions].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[securityContext].properties[sysctls].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[securityContext].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[emptyDir].properties[sizeLimit].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[emptyDir].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[metadata].properties[initializers].properties[pending].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[metadata].properties[initializers].properties[result].properties[details].properties[causes].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[metadata].properties[initializers].properties[result].properties[details].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[metadata].properties[initializers].properties[result].properties[metadata].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[metadata].properties[initializers].properties[result].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[metadata].properties[initializers].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[metadata].properties[ownerReferences].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[metadata].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[spec].properties[dataSource].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[spec].properties[resources].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[spec].properties[selector].properties[matchExpressions].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[spec].properties[selector].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[spec].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[status].properties[conditions].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].properties[status].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[volumeClaimTemplate].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[tolerations].items.type: Required value: must not be empty for specified array items, spec.validation.openAPIV3Schema.properties[spec].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[status].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.type: Required value: must not be empty at the root]\"\n }\n ],\n \"acceptedNames\": {\n \"plural\": \"alertmanagers\",\n \"singular\": \"alertmanager\",\n \"kind\": \"Alertmanager\",\n \"listKind\": \"AlertmanagerList\"\n },\n \"storedVersions\": [\n \"v1\"\n ]\n }\n}\n" }, { "path": "pkg/backup/actions/testdata/v1beta1/elasticsearches.elasticsearch.k8s.elastic.co.json", "content": "{\n \"kind\": \"CustomResourceDefinition\",\n \"apiVersion\": \"apiextensions.k8s.io/v1beta1\",\n \"metadata\": {\n \"name\": \"elasticsearches.elasticsearch.k8s.elastic.co\",\n \"selfLink\": \"/apis/apiextensions.k8s.io/v1beta1/customresourcedefinitions/elasticsearches.elasticsearch.k8s.elastic.co\",\n \"uid\": \"e8596856-29ae-47e4-8b14-5f7f027adf4a\",\n \"resourceVersion\": \"1703536\",\n \"generation\": 1,\n \"creationTimestamp\": \"2020-04-28T23:31:51Z\",\n \"labels\": {\n \"velero.io/backup-name\": \"es\",\n \"velero.io/restore-name\": \"es-crds\"\n },\n \"annotations\": {\n \"controller-gen.kubebuilder.io/version\": \"v0.2.5\",\n \"kubectl.kubernetes.io/last-applied-configuration\": \"{\\\"apiVersion\\\":\\\"apiextensions.k8s.io/v1beta1\\\",\\\"kind\\\":\\\"CustomResourceDefinition\\\",\\\"metadata\\\":{\\\"annotations\\\":{\\\"controller-gen.kubebuilder.io/version\\\":\\\"v0.2.5\\\"},\\\"creationTimestamp\\\":null,\\\"name\\\":\\\"elasticsearches.elasticsearch.k8s.elastic.co\\\"},\\\"spec\\\":{\\\"additionalPrinterColumns\\\":[{\\\"JSONPath\\\":\\\".status.health\\\",\\\"name\\\":\\\"health\\\",\\\"type\\\":\\\"string\\\"},{\\\"JSONPath\\\":\\\".status.availableNodes\\\",\\\"description\\\":\\\"Available nodes\\\",\\\"name\\\":\\\"nodes\\\",\\\"type\\\":\\\"integer\\\"},{\\\"JSONPath\\\":\\\".spec.version\\\",\\\"description\\\":\\\"Elasticsearch version\\\",\\\"name\\\":\\\"version\\\",\\\"type\\\":\\\"string\\\"},{\\\"JSONPath\\\":\\\".status.phase\\\",\\\"name\\\":\\\"phase\\\",\\\"type\\\":\\\"string\\\"},{\\\"JSONPath\\\":\\\".metadata.creationTimestamp\\\",\\\"name\\\":\\\"age\\\",\\\"type\\\":\\\"date\\\"}],\\\"group\\\":\\\"elasticsearch.k8s.elastic.co\\\",\\\"names\\\":{\\\"categories\\\":[\\\"elastic\\\"],\\\"kind\\\":\\\"Elasticsearch\\\",\\\"listKind\\\":\\\"ElasticsearchList\\\",\\\"plural\\\":\\\"elasticsearches\\\",\\\"shortNames\\\":[\\\"es\\\"],\\\"singular\\\":\\\"elasticsearch\\\"},\\\"scope\\\":\\\"Namespaced\\\",\\\"subresources\\\":{\\\"status\\\":{}},\\\"validation\\\":{\\\"openAPIV3Schema\\\":{\\\"description\\\":\\\"Elasticsearch represents an Elasticsearch resource in a Kubernetes cluster.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"metadata\\\":{\\\"type\\\":\\\"object\\\"},\\\"spec\\\":{\\\"description\\\":\\\"ElasticsearchSpec holds the specification of an Elasticsearch cluster.\\\",\\\"properties\\\":{\\\"auth\\\":{\\\"description\\\":\\\"Auth contains user authentication and authorization security settings for Elasticsearch.\\\",\\\"properties\\\":{\\\"fileRealm\\\":{\\\"description\\\":\\\"FileRealm to propagate to the Elasticsearch cluster.\\\",\\\"items\\\":{\\\"description\\\":\\\"FileRealmSource references users to create in the Elasticsearch cluster.\\\",\\\"properties\\\":{\\\"secretName\\\":{\\\"description\\\":\\\"SecretName is the name of the secret.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"roles\\\":{\\\"description\\\":\\\"Roles to propagate to the Elasticsearch cluster.\\\",\\\"items\\\":{\\\"description\\\":\\\"RoleSource references roles to create in the Elasticsearch cluster.\\\",\\\"properties\\\":{\\\"secretName\\\":{\\\"description\\\":\\\"SecretName is the name of the secret.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"http\\\":{\\\"description\\\":\\\"HTTP holds HTTP layer settings for Elasticsearch.\\\",\\\"properties\\\":{\\\"service\\\":{\\\"description\\\":\\\"Service defines the template for the associated Kubernetes Service object.\\\",\\\"properties\\\":{\\\"metadata\\\":{\\\"description\\\":\\\"ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored.\\\",\\\"type\\\":\\\"object\\\"},\\\"spec\\\":{\\\"description\\\":\\\"Spec is the specification of the service.\\\",\\\"properties\\\":{\\\"clusterIP\\\":{\\\"description\\\":\\\"clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are \\\\\\\"None\\\\\\\", empty string (\\\\\\\"\\\\\\\"), or a valid IP address. \\\\\\\"None\\\\\\\" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\\\",\\\"type\\\":\\\"string\\\"},\\\"externalIPs\\\":{\\\"description\\\":\\\"externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"externalName\\\":{\\\"description\\\":\\\"externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName.\\\",\\\"type\\\":\\\"string\\\"},\\\"externalTrafficPolicy\\\":{\\\"description\\\":\\\"externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. \\\\\\\"Local\\\\\\\" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. \\\\\\\"Cluster\\\\\\\" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading.\\\",\\\"type\\\":\\\"string\\\"},\\\"healthCheckNodePort\\\":{\\\"description\\\":\\\"healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"ipFamily\\\":{\\\"description\\\":\\\"ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment.\\\",\\\"type\\\":\\\"string\\\"},\\\"loadBalancerIP\\\":{\\\"description\\\":\\\"Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.\\\",\\\"type\\\":\\\"string\\\"},\\\"loadBalancerSourceRanges\\\":{\\\"description\\\":\\\"If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature.\\\\\\\" More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"ports\\\":{\\\"description\\\":\\\"The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\\\",\\\"items\\\":{\\\"description\\\":\\\"ServicePort contains information on service's port.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service.\\\",\\\"type\\\":\\\"string\\\"},\\\"nodePort\\\":{\\\"description\\\":\\\"The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"port\\\":{\\\"description\\\":\\\"The port that will be exposed by this service.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"protocol\\\":{\\\"description\\\":\\\"The IP protocol for this port. Supports \\\\\\\"TCP\\\\\\\", \\\\\\\"UDP\\\\\\\", and \\\\\\\"SCTP\\\\\\\". Default is TCP.\\\",\\\"type\\\":\\\"string\\\"},\\\"targetPort\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod's container ports. If this is not specified, the value of the 'port' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the 'port' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"publishNotReadyAddresses\\\":{\\\"description\\\":\\\"publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"selector\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/\\\",\\\"type\\\":\\\"object\\\"},\\\"sessionAffinity\\\":{\\\"description\\\":\\\"Supports \\\\\\\"ClientIP\\\\\\\" and \\\\\\\"None\\\\\\\". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\\\",\\\"type\\\":\\\"string\\\"},\\\"sessionAffinityConfig\\\":{\\\"description\\\":\\\"sessionAffinityConfig contains the configurations of session affinity.\\\",\\\"properties\\\":{\\\"clientIP\\\":{\\\"description\\\":\\\"clientIP contains the configurations of Client IP based session affinity.\\\",\\\"properties\\\":{\\\"timeoutSeconds\\\":{\\\"description\\\":\\\"timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be \\\\u003e0 \\\\u0026\\\\u0026 \\\\u003c=86400(for 1 day) if ServiceAffinity == \\\\\\\"ClientIP\\\\\\\". Default value is 10800(for 3 hours).\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"topologyKeys\\\":{\\\"description\\\":\\\"topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value \\\\\\\"*\\\\\\\" may be used to mean \\\\\\\"any topology\\\\\\\". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"type\\\":{\\\"description\\\":\\\"type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. \\\\\\\"ExternalName\\\\\\\" maps to the specified externalName. \\\\\\\"ClusterIP\\\\\\\" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is \\\\\\\"None\\\\\\\", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. \\\\\\\"NodePort\\\\\\\" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. \\\\\\\"LoadBalancer\\\\\\\" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"tls\\\":{\\\"description\\\":\\\"TLS defines options for configuring TLS for HTTP.\\\",\\\"properties\\\":{\\\"certificate\\\":{\\\"description\\\":\\\"Certificate is a reference to a Kubernetes secret that contains the certificate and private key for enabling TLS. The referenced secret should contain the following: \\\\n - `ca.crt`: The certificate authority (optional). - `tls.crt`: The certificate (or a chain). - `tls.key`: The private key to the first certificate in the certificate chain.\\\",\\\"properties\\\":{\\\"secretName\\\":{\\\"description\\\":\\\"SecretName is the name of the secret.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"selfSignedCertificate\\\":{\\\"description\\\":\\\"SelfSignedCertificate allows configuring the self-signed certificate generated by the operator.\\\",\\\"properties\\\":{\\\"disabled\\\":{\\\"description\\\":\\\"Disabled indicates that the provisioning of the self-signed certificate should be disabled.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"subjectAltNames\\\":{\\\"description\\\":\\\"SubjectAlternativeNames is a list of SANs to include in the generated HTTP TLS certificate.\\\",\\\"items\\\":{\\\"description\\\":\\\"SubjectAlternativeName represents a SAN entry in a x509 certificate.\\\",\\\"properties\\\":{\\\"dns\\\":{\\\"description\\\":\\\"DNS is the DNS name of the subject.\\\",\\\"type\\\":\\\"string\\\"},\\\"ip\\\":{\\\"description\\\":\\\"IP is the IP address of the subject.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"image\\\":{\\\"description\\\":\\\"Image is the Elasticsearch Docker image to deploy.\\\",\\\"type\\\":\\\"string\\\"},\\\"nodeSets\\\":{\\\"description\\\":\\\"NodeSets allow specifying groups of Elasticsearch nodes sharing the same configuration and Pod templates. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-orchestration.html\\\",\\\"items\\\":{\\\"description\\\":\\\"NodeSet is the specification for a group of Elasticsearch nodes sharing the same configuration and a Pod template.\\\",\\\"properties\\\":{\\\"config\\\":{\\\"description\\\":\\\"Config holds the Elasticsearch configuration.\\\",\\\"type\\\":\\\"object\\\"},\\\"count\\\":{\\\"description\\\":\\\"Count of Elasticsearch nodes to deploy.\\\",\\\"format\\\":\\\"int32\\\",\\\"minimum\\\":1,\\\"type\\\":\\\"integer\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of this set of nodes. Becomes a part of the Elasticsearch node.name setting.\\\",\\\"maxLength\\\":23,\\\"pattern\\\":\\\"[a-zA-Z0-9-]+\\\",\\\"type\\\":\\\"string\\\"},\\\"podTemplate\\\":{\\\"description\\\":\\\"PodTemplate provides customisation options (labels, annotations, affinity rules, resource requests, and so on) for the Pods belonging to this NodeSet.\\\",\\\"type\\\":\\\"object\\\"},\\\"volumeClaimTemplates\\\":{\\\"description\\\":\\\"VolumeClaimTemplates is a list of persistent volume claims to be used by each Pod in this NodeSet. Every claim in this list must have a matching volumeMount in one of the containers defined in the PodTemplate. Items defined here take precedence over any default claims added by the operator with the same name. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-volume-claim-templates.html\\\",\\\"items\\\":{\\\"description\\\":\\\"PersistentVolumeClaim is a user's request for and claim to a persistent volume\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"metadata\\\":{\\\"description\\\":\\\"Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata\\\",\\\"type\\\":\\\"object\\\"},\\\"spec\\\":{\\\"description\\\":\\\"Spec defines the desired characteristics of a volume requested by a pod author. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims\\\",\\\"properties\\\":{\\\"accessModes\\\":{\\\"description\\\":\\\"AccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"dataSource\\\":{\\\"description\\\":\\\"This field requires the VolumeSnapshotDataSource alpha feature gate to be enabled and currently VolumeSnapshot is the only supported data source. If the provisioner can support VolumeSnapshot data source, it will create a new volume and data will be restored to the volume at the same time. If the provisioner does not support VolumeSnapshot data source, volume will not be created and the failure will be reported as an event. In the future, we plan to support more data source types and the behavior of the provisioner may change.\\\",\\\"properties\\\":{\\\"apiGroup\\\":{\\\"description\\\":\\\"APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is the type of resource being referenced\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name is the name of resource being referenced\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"kind\\\",\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"resources\\\":{\\\"description\\\":\\\"Resources represents the minimum resources the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"additionalProperties\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"pattern\\\":\\\"^(\\\\\\\\+|-)?(([0-9]+(\\\\\\\\.[0-9]*)?)|(\\\\\\\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\\\\\\\+|-)?(([0-9]+(\\\\\\\\.[0-9]*)?)|(\\\\\\\\.[0-9]+))))?$\\\"},\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"additionalProperties\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"pattern\\\":\\\"^(\\\\\\\\+|-)?(([0-9]+(\\\\\\\\.[0-9]*)?)|(\\\\\\\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\\\\\\\+|-)?(([0-9]+(\\\\\\\\.[0-9]*)?)|(\\\\\\\\.[0-9]+))))?$\\\"},\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"selector\\\":{\\\"description\\\":\\\"A label query over volumes to consider for binding.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"storageClassName\\\":{\\\"description\\\":\\\"Name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1\\\",\\\"type\\\":\\\"string\\\"},\\\"volumeMode\\\":{\\\"description\\\":\\\"volumeMode defines what type of volume is required by the claim. Value of Filesystem is implied when not included in claim spec. This is a beta feature.\\\",\\\"type\\\":\\\"string\\\"},\\\"volumeName\\\":{\\\"description\\\":\\\"VolumeName is the binding reference to the PersistentVolume backing this claim.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"status\\\":{\\\"description\\\":\\\"Status represents the current information/status of a persistent volume claim. Read-only. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims\\\",\\\"properties\\\":{\\\"accessModes\\\":{\\\"description\\\":\\\"AccessModes contains the actual access modes the volume backing the PVC has. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"capacity\\\":{\\\"additionalProperties\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"pattern\\\":\\\"^(\\\\\\\\+|-)?(([0-9]+(\\\\\\\\.[0-9]*)?)|(\\\\\\\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\\\\\\\+|-)?(([0-9]+(\\\\\\\\.[0-9]*)?)|(\\\\\\\\.[0-9]+))))?$\\\"},\\\"description\\\":\\\"Represents the actual resources of the underlying volume.\\\",\\\"type\\\":\\\"object\\\"},\\\"conditions\\\":{\\\"description\\\":\\\"Current Condition of persistent volume claim. If underlying persistent volume is being resized then the Condition will be set to 'ResizeStarted'.\\\",\\\"items\\\":{\\\"description\\\":\\\"PersistentVolumeClaimCondition contains details about state of pvc\\\",\\\"properties\\\":{\\\"lastProbeTime\\\":{\\\"description\\\":\\\"Last time we probed the condition.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"lastTransitionTime\\\":{\\\"description\\\":\\\"Last time the condition transitioned from one status to another.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"Human-readable message indicating details about last transition.\\\",\\\"type\\\":\\\"string\\\"},\\\"reason\\\":{\\\"description\\\":\\\"Unique, this should be a short, machine understandable string that gives the reason for condition's last transition. If it reports \\\\\\\"ResizeStarted\\\\\\\" that means the underlying persistent volume is being resized.\\\",\\\"type\\\":\\\"string\\\"},\\\"status\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":{\\\"description\\\":\\\"PersistentVolumeClaimConditionType is a valid value of PersistentVolumeClaimCondition.Type\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"status\\\",\\\"type\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"phase\\\":{\\\"description\\\":\\\"Phase represents the current phase of PersistentVolumeClaim.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"count\\\",\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"minItems\\\":1,\\\"type\\\":\\\"array\\\"},\\\"podDisruptionBudget\\\":{\\\"description\\\":\\\"PodDisruptionBudget provides access to the default pod disruption budget for the Elasticsearch cluster. The default budget selects all cluster pods and sets `maxUnavailable` to 1. To disable, set `PodDisruptionBudget` to the empty value (`{}` in YAML).\\\",\\\"properties\\\":{\\\"metadata\\\":{\\\"description\\\":\\\"ObjectMeta is the metadata of the PDB. The name and namespace provided here are managed by ECK and will be ignored.\\\",\\\"type\\\":\\\"object\\\"},\\\"spec\\\":{\\\"description\\\":\\\"Spec is the specification of the PDB.\\\",\\\"properties\\\":{\\\"maxUnavailable\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"An eviction is allowed if at most \\\\\\\"maxUnavailable\\\\\\\" pods selected by \\\\\\\"selector\\\\\\\" are unavailable after the eviction, i.e. even in absence of the evicted pod. For example, one can prevent all voluntary evictions by specifying 0. This is a mutually exclusive setting with \\\\\\\"minAvailable\\\\\\\".\\\"},\\\"minAvailable\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"An eviction is allowed if at least \\\\\\\"minAvailable\\\\\\\" pods selected by \\\\\\\"selector\\\\\\\" will still be available after the eviction, i.e. even in the absence of the evicted pod. So for example you can prevent all voluntary evictions by specifying \\\\\\\"100%\\\\\\\".\\\"},\\\"selector\\\":{\\\"description\\\":\\\"Label query over pods whose evictions are managed by the disruption budget.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"remoteClusters\\\":{\\\"description\\\":\\\"RemoteClusters enables you to establish uni-directional connections to a remote Elasticsearch cluster.\\\",\\\"items\\\":{\\\"description\\\":\\\"RemoteCluster declares a remote Elasticsearch cluster connection.\\\",\\\"properties\\\":{\\\"elasticsearchRef\\\":{\\\"description\\\":\\\"ElasticsearchRef is a reference to an Elasticsearch cluster running within the same k8s cluster.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the Kubernetes object.\\\",\\\"type\\\":\\\"string\\\"},\\\"namespace\\\":{\\\"description\\\":\\\"Namespace of the Kubernetes object. If empty, defaults to the current namespace.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name is the name of the remote cluster as it is set in the Elasticsearch settings. The name is expected to be unique for each remote clusters.\\\",\\\"minLength\\\":1,\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"secureSettings\\\":{\\\"description\\\":\\\"SecureSettings is a list of references to Kubernetes secrets containing sensitive configuration options for Elasticsearch. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-es-secure-settings.html\\\",\\\"items\\\":{\\\"description\\\":\\\"SecretSource defines a data source based on a Kubernetes Secret.\\\",\\\"properties\\\":{\\\"entries\\\":{\\\"description\\\":\\\"Entries define how to project each key-value pair in the secret to filesystem paths. If not defined, all keys will be projected to similarly named paths in the filesystem. If defined, only the specified keys will be projected to the corresponding paths.\\\",\\\"items\\\":{\\\"description\\\":\\\"KeyToPath defines how to map a key in a Secret object to a filesystem path.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"Key is the key contained in the secret.\\\",\\\"type\\\":\\\"string\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path is the relative file path to map the key to. Path must not be an absolute file path and must not contain any \\\\\\\"..\\\\\\\" components.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"secretName\\\":{\\\"description\\\":\\\"SecretName is the name of the secret.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"secretName\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"serviceAccountName\\\":{\\\"description\\\":\\\"ServiceAccountName is used to check access from the current resource to a resource (eg. a remote Elasticsearch cluster) in a different namespace. Can only be used if ECK is enforcing RBAC on references.\\\",\\\"type\\\":\\\"string\\\"},\\\"transport\\\":{\\\"description\\\":\\\"Transport holds transport layer settings for Elasticsearch.\\\",\\\"properties\\\":{\\\"service\\\":{\\\"description\\\":\\\"Service defines the template for the associated Kubernetes Service object.\\\",\\\"properties\\\":{\\\"metadata\\\":{\\\"description\\\":\\\"ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored.\\\",\\\"type\\\":\\\"object\\\"},\\\"spec\\\":{\\\"description\\\":\\\"Spec is the specification of the service.\\\",\\\"properties\\\":{\\\"clusterIP\\\":{\\\"description\\\":\\\"clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are \\\\\\\"None\\\\\\\", empty string (\\\\\\\"\\\\\\\"), or a valid IP address. \\\\\\\"None\\\\\\\" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\\\",\\\"type\\\":\\\"string\\\"},\\\"externalIPs\\\":{\\\"description\\\":\\\"externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"externalName\\\":{\\\"description\\\":\\\"externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName.\\\",\\\"type\\\":\\\"string\\\"},\\\"externalTrafficPolicy\\\":{\\\"description\\\":\\\"externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. \\\\\\\"Local\\\\\\\" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. \\\\\\\"Cluster\\\\\\\" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading.\\\",\\\"type\\\":\\\"string\\\"},\\\"healthCheckNodePort\\\":{\\\"description\\\":\\\"healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"ipFamily\\\":{\\\"description\\\":\\\"ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment.\\\",\\\"type\\\":\\\"string\\\"},\\\"loadBalancerIP\\\":{\\\"description\\\":\\\"Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.\\\",\\\"type\\\":\\\"string\\\"},\\\"loadBalancerSourceRanges\\\":{\\\"description\\\":\\\"If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature.\\\\\\\" More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"ports\\\":{\\\"description\\\":\\\"The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\\\",\\\"items\\\":{\\\"description\\\":\\\"ServicePort contains information on service's port.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service.\\\",\\\"type\\\":\\\"string\\\"},\\\"nodePort\\\":{\\\"description\\\":\\\"The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"port\\\":{\\\"description\\\":\\\"The port that will be exposed by this service.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"protocol\\\":{\\\"description\\\":\\\"The IP protocol for this port. Supports \\\\\\\"TCP\\\\\\\", \\\\\\\"UDP\\\\\\\", and \\\\\\\"SCTP\\\\\\\". Default is TCP.\\\",\\\"type\\\":\\\"string\\\"},\\\"targetPort\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod's container ports. If this is not specified, the value of the 'port' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the 'port' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"publishNotReadyAddresses\\\":{\\\"description\\\":\\\"publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"selector\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/\\\",\\\"type\\\":\\\"object\\\"},\\\"sessionAffinity\\\":{\\\"description\\\":\\\"Supports \\\\\\\"ClientIP\\\\\\\" and \\\\\\\"None\\\\\\\". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\\\",\\\"type\\\":\\\"string\\\"},\\\"sessionAffinityConfig\\\":{\\\"description\\\":\\\"sessionAffinityConfig contains the configurations of session affinity.\\\",\\\"properties\\\":{\\\"clientIP\\\":{\\\"description\\\":\\\"clientIP contains the configurations of Client IP based session affinity.\\\",\\\"properties\\\":{\\\"timeoutSeconds\\\":{\\\"description\\\":\\\"timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be \\\\u003e0 \\\\u0026\\\\u0026 \\\\u003c=86400(for 1 day) if ServiceAffinity == \\\\\\\"ClientIP\\\\\\\". Default value is 10800(for 3 hours).\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"topologyKeys\\\":{\\\"description\\\":\\\"topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value \\\\\\\"*\\\\\\\" may be used to mean \\\\\\\"any topology\\\\\\\". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"type\\\":{\\\"description\\\":\\\"type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. \\\\\\\"ExternalName\\\\\\\" maps to the specified externalName. \\\\\\\"ClusterIP\\\\\\\" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is \\\\\\\"None\\\\\\\", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. \\\\\\\"NodePort\\\\\\\" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. \\\\\\\"LoadBalancer\\\\\\\" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"updateStrategy\\\":{\\\"description\\\":\\\"UpdateStrategy specifies how updates to the cluster should be performed.\\\",\\\"properties\\\":{\\\"changeBudget\\\":{\\\"description\\\":\\\"ChangeBudget defines the constraints to consider when applying changes to the Elasticsearch cluster.\\\",\\\"properties\\\":{\\\"maxSurge\\\":{\\\"description\\\":\\\"MaxSurge is the maximum number of new pods that can be created exceeding the original number of pods defined in the specification. MaxSurge is only taken into consideration when scaling up. Setting a negative value will disable the restriction. Defaults to unbounded if not specified.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"maxUnavailable\\\":{\\\"description\\\":\\\"MaxUnavailable is the maximum number of pods that can be unavailable (not ready) during the update due to circumstances under the control of the operator. Setting a negative value will disable this restriction. Defaults to 1 if not specified.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"version\\\":{\\\"description\\\":\\\"Version of Elasticsearch.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"nodeSets\\\",\\\"version\\\"],\\\"type\\\":\\\"object\\\"},\\\"status\\\":{\\\"description\\\":\\\"ElasticsearchStatus defines the observed state of Elasticsearch\\\",\\\"properties\\\":{\\\"availableNodes\\\":{\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"health\\\":{\\\"description\\\":\\\"ElasticsearchHealth is the health of the cluster as returned by the health API.\\\",\\\"type\\\":\\\"string\\\"},\\\"phase\\\":{\\\"description\\\":\\\"ElasticsearchOrchestrationPhase is the phase Elasticsearch is in from the controller point of view.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}}}},\\\"version\\\":\\\"v1\\\",\\\"versions\\\":[{\\\"name\\\":\\\"v1\\\",\\\"served\\\":true,\\\"storage\\\":true},{\\\"name\\\":\\\"v1beta1\\\",\\\"served\\\":true,\\\"storage\\\":false},{\\\"name\\\":\\\"v1alpha1\\\",\\\"served\\\":false,\\\"storage\\\":false}]},\\\"status\\\":{\\\"acceptedNames\\\":{\\\"kind\\\":\\\"\\\",\\\"plural\\\":\\\"\\\"},\\\"conditions\\\":[],\\\"storedVersions\\\":[]}}\\n\"\n }\n },\n \"spec\": {\n \"group\": \"elasticsearch.k8s.elastic.co\",\n \"version\": \"v1\",\n \"names\": {\n \"plural\": \"elasticsearches\",\n \"singular\": \"elasticsearch\",\n \"shortNames\": [\n \"es\"\n ],\n \"kind\": \"Elasticsearch\",\n \"listKind\": \"ElasticsearchList\",\n \"categories\": [\n \"elastic\"\n ]\n },\n \"scope\": \"Namespaced\",\n \"validation\": {\n \"openAPIV3Schema\": {\n \"description\": \"Elasticsearch represents an Elasticsearch resource in a Kubernetes cluster.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"ElasticsearchSpec holds the specification of an Elasticsearch cluster.\",\n \"type\": \"object\",\n \"required\": [\n \"nodeSets\",\n \"version\"\n ],\n \"properties\": {\n \"auth\": {\n \"description\": \"Auth contains user authentication and authorization security settings for Elasticsearch.\",\n \"type\": \"object\",\n \"properties\": {\n \"fileRealm\": {\n \"description\": \"FileRealm to propagate to the Elasticsearch cluster.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"FileRealmSource references users to create in the Elasticsearch cluster.\",\n \"type\": \"object\",\n \"properties\": {\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"roles\": {\n \"description\": \"Roles to propagate to the Elasticsearch cluster.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"RoleSource references roles to create in the Elasticsearch cluster.\",\n \"type\": \"object\",\n \"properties\": {\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n }\n }\n }\n }\n },\n \"http\": {\n \"description\": \"HTTP holds HTTP layer settings for Elasticsearch.\",\n \"type\": \"object\",\n \"properties\": {\n \"service\": {\n \"description\": \"Service defines the template for the associated Kubernetes Service object.\",\n \"type\": \"object\",\n \"properties\": {\n \"metadata\": {\n \"description\": \"ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored.\",\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec is the specification of the service.\",\n \"type\": \"object\",\n \"properties\": {\n \"clusterIP\": {\n \"description\": \"clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are \\\"None\\\", empty string (\\\"\\\"), or a valid IP address. \\\"None\\\" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"externalIPs\": {\n \"description\": \"externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"externalName\": {\n \"description\": \"externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName.\",\n \"type\": \"string\"\n },\n \"externalTrafficPolicy\": {\n \"description\": \"externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. \\\"Local\\\" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. \\\"Cluster\\\" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading.\",\n \"type\": \"string\"\n },\n \"healthCheckNodePort\": {\n \"description\": \"healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"ipFamily\": {\n \"description\": \"ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment.\",\n \"type\": \"string\"\n },\n \"loadBalancerIP\": {\n \"description\": \"Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.\",\n \"type\": \"string\"\n },\n \"loadBalancerSourceRanges\": {\n \"description\": \"If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature.\\\" More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"ports\": {\n \"description\": \"The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"ServicePort contains information on service's port.\",\n \"type\": \"object\",\n \"required\": [\n \"port\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service.\",\n \"type\": \"string\"\n },\n \"nodePort\": {\n \"description\": \"The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"port\": {\n \"description\": \"The port that will be exposed by this service.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"protocol\": {\n \"description\": \"The IP protocol for this port. Supports \\\"TCP\\\", \\\"UDP\\\", and \\\"SCTP\\\". Default is TCP.\",\n \"type\": \"string\"\n },\n \"targetPort\": {\n \"description\": \"Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod's container ports. If this is not specified, the value of the 'port' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the 'port' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service\",\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ]\n }\n }\n }\n },\n \"publishNotReadyAddresses\": {\n \"description\": \"publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery.\",\n \"type\": \"boolean\"\n },\n \"selector\": {\n \"description\": \"Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"type\": \"string\"\n }\n },\n \"sessionAffinity\": {\n \"description\": \"Supports \\\"ClientIP\\\" and \\\"None\\\". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"sessionAffinityConfig\": {\n \"description\": \"sessionAffinityConfig contains the configurations of session affinity.\",\n \"type\": \"object\",\n \"properties\": {\n \"clientIP\": {\n \"description\": \"clientIP contains the configurations of Client IP based session affinity.\",\n \"type\": \"object\",\n \"properties\": {\n \"timeoutSeconds\": {\n \"description\": \"timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be >0 && <=86400(for 1 day) if ServiceAffinity == \\\"ClientIP\\\". Default value is 10800(for 3 hours).\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n }\n }\n }\n }\n },\n \"topologyKeys\": {\n \"description\": \"topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value \\\"*\\\" may be used to mean \\\"any topology\\\". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"type\": {\n \"description\": \"type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. \\\"ExternalName\\\" maps to the specified externalName. \\\"ClusterIP\\\" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is \\\"None\\\", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. \\\"NodePort\\\" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. \\\"LoadBalancer\\\" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types\",\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"tls\": {\n \"description\": \"TLS defines options for configuring TLS for HTTP.\",\n \"type\": \"object\",\n \"properties\": {\n \"certificate\": {\n \"description\": \"Certificate is a reference to a Kubernetes secret that contains the certificate and private key for enabling TLS. The referenced secret should contain the following: \\n - `ca.crt`: The certificate authority (optional). - `tls.crt`: The certificate (or a chain). - `tls.key`: The private key to the first certificate in the certificate chain.\",\n \"type\": \"object\",\n \"properties\": {\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n }\n },\n \"selfSignedCertificate\": {\n \"description\": \"SelfSignedCertificate allows configuring the self-signed certificate generated by the operator.\",\n \"type\": \"object\",\n \"properties\": {\n \"disabled\": {\n \"description\": \"Disabled indicates that the provisioning of the self-signed certificate should be disabled.\",\n \"type\": \"boolean\"\n },\n \"subjectAltNames\": {\n \"description\": \"SubjectAlternativeNames is a list of SANs to include in the generated HTTP TLS certificate.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"SubjectAlternativeName represents a SAN entry in a x509 certificate.\",\n \"type\": \"object\",\n \"properties\": {\n \"dns\": {\n \"description\": \"DNS is the DNS name of the subject.\",\n \"type\": \"string\"\n },\n \"ip\": {\n \"description\": \"IP is the IP address of the subject.\",\n \"type\": \"string\"\n }\n }\n }\n }\n }\n }\n }\n }\n }\n },\n \"image\": {\n \"description\": \"Image is the Elasticsearch Docker image to deploy.\",\n \"type\": \"string\"\n },\n \"nodeSets\": {\n \"description\": \"NodeSets allow specifying groups of Elasticsearch nodes sharing the same configuration and Pod templates. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-orchestration.html\",\n \"type\": \"array\",\n \"minItems\": 1,\n \"items\": {\n \"description\": \"NodeSet is the specification for a group of Elasticsearch nodes sharing the same configuration and a Pod template.\",\n \"type\": \"object\",\n \"required\": [\n \"count\",\n \"name\"\n ],\n \"properties\": {\n \"config\": {\n \"description\": \"Config holds the Elasticsearch configuration.\",\n \"type\": \"object\"\n },\n \"count\": {\n \"description\": \"Count of Elasticsearch nodes to deploy.\",\n \"type\": \"integer\",\n \"format\": \"int32\",\n \"minimum\": 1\n },\n \"name\": {\n \"description\": \"Name of this set of nodes. Becomes a part of the Elasticsearch node.name setting.\",\n \"type\": \"string\",\n \"maxLength\": 23,\n \"pattern\": \"[a-zA-Z0-9-]+\"\n },\n \"podTemplate\": {\n \"description\": \"PodTemplate provides customisation options (labels, annotations, affinity rules, resource requests, and so on) for the Pods belonging to this NodeSet.\",\n \"type\": \"object\"\n },\n \"volumeClaimTemplates\": {\n \"description\": \"VolumeClaimTemplates is a list of persistent volume claims to be used by each Pod in this NodeSet. Every claim in this list must have a matching volumeMount in one of the containers defined in the PodTemplate. Items defined here take precedence over any default claims added by the operator with the same name. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-volume-claim-templates.html\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"PersistentVolumeClaim is a user's request for and claim to a persistent volume\",\n \"type\": \"object\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"description\": \"Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata\",\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec defines the desired characteristics of a volume requested by a pod author. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims\",\n \"type\": \"object\",\n \"properties\": {\n \"accessModes\": {\n \"description\": \"AccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"dataSource\": {\n \"description\": \"This field requires the VolumeSnapshotDataSource alpha feature gate to be enabled and currently VolumeSnapshot is the only supported data source. If the provisioner can support VolumeSnapshot data source, it will create a new volume and data will be restored to the volume at the same time. If the provisioner does not support VolumeSnapshot data source, volume will not be created and the failure will be reported as an event. In the future, we plan to support more data source types and the behavior of the provisioner may change.\",\n \"type\": \"object\",\n \"required\": [\n \"kind\",\n \"name\"\n ],\n \"properties\": {\n \"apiGroup\": {\n \"description\": \"APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is the type of resource being referenced\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name is the name of resource being referenced\",\n \"type\": \"string\"\n }\n }\n },\n \"resources\": {\n \"description\": \"Resources represents the minimum resources the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources\",\n \"type\": \"object\",\n \"properties\": {\n \"limits\": {\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"pattern\": \"^(\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))))?$\",\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ]\n }\n },\n \"requests\": {\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"pattern\": \"^(\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))))?$\",\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ]\n }\n }\n }\n },\n \"selector\": {\n \"description\": \"A label query over volumes to consider for binding.\",\n \"type\": \"object\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"type\": \"object\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"type\": \"string\"\n }\n }\n }\n },\n \"storageClassName\": {\n \"description\": \"Name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1\",\n \"type\": \"string\"\n },\n \"volumeMode\": {\n \"description\": \"volumeMode defines what type of volume is required by the claim. Value of Filesystem is implied when not included in claim spec. This is a beta feature.\",\n \"type\": \"string\"\n },\n \"volumeName\": {\n \"description\": \"VolumeName is the binding reference to the PersistentVolume backing this claim.\",\n \"type\": \"string\"\n }\n }\n },\n \"status\": {\n \"description\": \"Status represents the current information/status of a persistent volume claim. Read-only. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims\",\n \"type\": \"object\",\n \"properties\": {\n \"accessModes\": {\n \"description\": \"AccessModes contains the actual access modes the volume backing the PVC has. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"capacity\": {\n \"description\": \"Represents the actual resources of the underlying volume.\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"pattern\": \"^(\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\\\+|-)?(([0-9]+(\\\\.[0-9]*)?)|(\\\\.[0-9]+))))?$\",\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ]\n }\n },\n \"conditions\": {\n \"description\": \"Current Condition of persistent volume claim. If underlying persistent volume is being resized then the Condition will be set to 'ResizeStarted'.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"PersistentVolumeClaimCondition contains details about state of pvc\",\n \"type\": \"object\",\n \"required\": [\n \"status\",\n \"type\"\n ],\n \"properties\": {\n \"lastProbeTime\": {\n \"description\": \"Last time we probed the condition.\",\n \"type\": \"string\",\n \"format\": \"date-time\"\n },\n \"lastTransitionTime\": {\n \"description\": \"Last time the condition transitioned from one status to another.\",\n \"type\": \"string\",\n \"format\": \"date-time\"\n },\n \"message\": {\n \"description\": \"Human-readable message indicating details about last transition.\",\n \"type\": \"string\"\n },\n \"reason\": {\n \"description\": \"Unique, this should be a short, machine understandable string that gives the reason for condition's last transition. If it reports \\\"ResizeStarted\\\" that means the underlying persistent volume is being resized.\",\n \"type\": \"string\"\n },\n \"status\": {\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"PersistentVolumeClaimConditionType is a valid value of PersistentVolumeClaimCondition.Type\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"phase\": {\n \"description\": \"Phase represents the current phase of PersistentVolumeClaim.\",\n \"type\": \"string\"\n }\n }\n }\n }\n }\n }\n }\n }\n },\n \"podDisruptionBudget\": {\n \"description\": \"PodDisruptionBudget provides access to the default pod disruption budget for the Elasticsearch cluster. The default budget selects all cluster pods and sets `maxUnavailable` to 1. To disable, set `PodDisruptionBudget` to the empty value (`{}` in YAML).\",\n \"type\": \"object\",\n \"properties\": {\n \"metadata\": {\n \"description\": \"ObjectMeta is the metadata of the PDB. The name and namespace provided here are managed by ECK and will be ignored.\",\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec is the specification of the PDB.\",\n \"type\": \"object\",\n \"properties\": {\n \"maxUnavailable\": {\n \"description\": \"An eviction is allowed if at most \\\"maxUnavailable\\\" pods selected by \\\"selector\\\" are unavailable after the eviction, i.e. even in absence of the evicted pod. For example, one can prevent all voluntary evictions by specifying 0. This is a mutually exclusive setting with \\\"minAvailable\\\".\",\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ]\n },\n \"minAvailable\": {\n \"description\": \"An eviction is allowed if at least \\\"minAvailable\\\" pods selected by \\\"selector\\\" will still be available after the eviction, i.e. even in the absence of the evicted pod. So for example you can prevent all voluntary evictions by specifying \\\"100%\\\".\",\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ]\n },\n \"selector\": {\n \"description\": \"Label query over pods whose evictions are managed by the disruption budget.\",\n \"type\": \"object\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"type\": \"object\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"type\": \"string\"\n }\n }\n }\n }\n }\n }\n }\n },\n \"remoteClusters\": {\n \"description\": \"RemoteClusters enables you to establish uni-directional connections to a remote Elasticsearch cluster.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"RemoteCluster declares a remote Elasticsearch cluster connection.\",\n \"type\": \"object\",\n \"required\": [\n \"name\"\n ],\n \"properties\": {\n \"elasticsearchRef\": {\n \"description\": \"ElasticsearchRef is a reference to an Elasticsearch cluster running within the same k8s cluster.\",\n \"type\": \"object\",\n \"required\": [\n \"name\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the Kubernetes object.\",\n \"type\": \"string\"\n },\n \"namespace\": {\n \"description\": \"Namespace of the Kubernetes object. If empty, defaults to the current namespace.\",\n \"type\": \"string\"\n }\n }\n },\n \"name\": {\n \"description\": \"Name is the name of the remote cluster as it is set in the Elasticsearch settings. The name is expected to be unique for each remote clusters.\",\n \"type\": \"string\",\n \"minLength\": 1\n }\n }\n }\n },\n \"secureSettings\": {\n \"description\": \"SecureSettings is a list of references to Kubernetes secrets containing sensitive configuration options for Elasticsearch. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-es-secure-settings.html\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"SecretSource defines a data source based on a Kubernetes Secret.\",\n \"type\": \"object\",\n \"required\": [\n \"secretName\"\n ],\n \"properties\": {\n \"entries\": {\n \"description\": \"Entries define how to project each key-value pair in the secret to filesystem paths. If not defined, all keys will be projected to similarly named paths in the filesystem. If defined, only the specified keys will be projected to the corresponding paths.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"KeyToPath defines how to map a key in a Secret object to a filesystem path.\",\n \"type\": \"object\",\n \"required\": [\n \"key\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"Key is the key contained in the secret.\",\n \"type\": \"string\"\n },\n \"path\": {\n \"description\": \"Path is the relative file path to map the key to. Path must not be an absolute file path and must not contain any \\\"..\\\" components.\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"serviceAccountName\": {\n \"description\": \"ServiceAccountName is used to check access from the current resource to a resource (eg. a remote Elasticsearch cluster) in a different namespace. Can only be used if ECK is enforcing RBAC on references.\",\n \"type\": \"string\"\n },\n \"transport\": {\n \"description\": \"Transport holds transport layer settings for Elasticsearch.\",\n \"type\": \"object\",\n \"properties\": {\n \"service\": {\n \"description\": \"Service defines the template for the associated Kubernetes Service object.\",\n \"type\": \"object\",\n \"properties\": {\n \"metadata\": {\n \"description\": \"ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored.\",\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec is the specification of the service.\",\n \"type\": \"object\",\n \"properties\": {\n \"clusterIP\": {\n \"description\": \"clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are \\\"None\\\", empty string (\\\"\\\"), or a valid IP address. \\\"None\\\" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"externalIPs\": {\n \"description\": \"externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"externalName\": {\n \"description\": \"externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName.\",\n \"type\": \"string\"\n },\n \"externalTrafficPolicy\": {\n \"description\": \"externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. \\\"Local\\\" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. \\\"Cluster\\\" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading.\",\n \"type\": \"string\"\n },\n \"healthCheckNodePort\": {\n \"description\": \"healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"ipFamily\": {\n \"description\": \"ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment.\",\n \"type\": \"string\"\n },\n \"loadBalancerIP\": {\n \"description\": \"Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.\",\n \"type\": \"string\"\n },\n \"loadBalancerSourceRanges\": {\n \"description\": \"If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature.\\\" More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"ports\": {\n \"description\": \"The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"ServicePort contains information on service's port.\",\n \"type\": \"object\",\n \"required\": [\n \"port\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service.\",\n \"type\": \"string\"\n },\n \"nodePort\": {\n \"description\": \"The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"port\": {\n \"description\": \"The port that will be exposed by this service.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"protocol\": {\n \"description\": \"The IP protocol for this port. Supports \\\"TCP\\\", \\\"UDP\\\", and \\\"SCTP\\\". Default is TCP.\",\n \"type\": \"string\"\n },\n \"targetPort\": {\n \"description\": \"Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod's container ports. If this is not specified, the value of the 'port' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the 'port' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service\",\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ]\n }\n }\n }\n },\n \"publishNotReadyAddresses\": {\n \"description\": \"publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery.\",\n \"type\": \"boolean\"\n },\n \"selector\": {\n \"description\": \"Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"type\": \"string\"\n }\n },\n \"sessionAffinity\": {\n \"description\": \"Supports \\\"ClientIP\\\" and \\\"None\\\". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"sessionAffinityConfig\": {\n \"description\": \"sessionAffinityConfig contains the configurations of session affinity.\",\n \"type\": \"object\",\n \"properties\": {\n \"clientIP\": {\n \"description\": \"clientIP contains the configurations of Client IP based session affinity.\",\n \"type\": \"object\",\n \"properties\": {\n \"timeoutSeconds\": {\n \"description\": \"timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be >0 && <=86400(for 1 day) if ServiceAffinity == \\\"ClientIP\\\". Default value is 10800(for 3 hours).\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n }\n }\n }\n }\n },\n \"topologyKeys\": {\n \"description\": \"topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value \\\"*\\\" may be used to mean \\\"any topology\\\". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"type\": {\n \"description\": \"type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. \\\"ExternalName\\\" maps to the specified externalName. \\\"ClusterIP\\\" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is \\\"None\\\", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. \\\"NodePort\\\" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. \\\"LoadBalancer\\\" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types\",\n \"type\": \"string\"\n }\n }\n }\n }\n }\n }\n },\n \"updateStrategy\": {\n \"description\": \"UpdateStrategy specifies how updates to the cluster should be performed.\",\n \"type\": \"object\",\n \"properties\": {\n \"changeBudget\": {\n \"description\": \"ChangeBudget defines the constraints to consider when applying changes to the Elasticsearch cluster.\",\n \"type\": \"object\",\n \"properties\": {\n \"maxSurge\": {\n \"description\": \"MaxSurge is the maximum number of new pods that can be created exceeding the original number of pods defined in the specification. MaxSurge is only taken into consideration when scaling up. Setting a negative value will disable the restriction. Defaults to unbounded if not specified.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"maxUnavailable\": {\n \"description\": \"MaxUnavailable is the maximum number of pods that can be unavailable (not ready) during the update due to circumstances under the control of the operator. Setting a negative value will disable this restriction. Defaults to 1 if not specified.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n }\n }\n }\n }\n },\n \"version\": {\n \"description\": \"Version of Elasticsearch.\",\n \"type\": \"string\"\n }\n }\n },\n \"status\": {\n \"description\": \"ElasticsearchStatus defines the observed state of Elasticsearch\",\n \"type\": \"object\",\n \"properties\": {\n \"availableNodes\": {\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"health\": {\n \"description\": \"ElasticsearchHealth is the health of the cluster as returned by the health API.\",\n \"type\": \"string\"\n },\n \"phase\": {\n \"description\": \"ElasticsearchOrchestrationPhase is the phase Elasticsearch is in from the controller point of view.\",\n \"type\": \"string\"\n }\n }\n }\n }\n }\n },\n \"subresources\": {\n \"status\": {}\n },\n \"versions\": [\n {\n \"name\": \"v1\",\n \"served\": true,\n \"storage\": true\n },\n {\n \"name\": \"v1beta1\",\n \"served\": true,\n \"storage\": false\n },\n {\n \"name\": \"v1alpha1\",\n \"served\": false,\n \"storage\": false\n }\n ],\n \"additionalPrinterColumns\": [\n {\n \"name\": \"health\",\n \"type\": \"string\",\n \"JSONPath\": \".status.health\"\n },\n {\n \"name\": \"nodes\",\n \"type\": \"integer\",\n \"description\": \"Available nodes\",\n \"JSONPath\": \".status.availableNodes\"\n },\n {\n \"name\": \"version\",\n \"type\": \"string\",\n \"description\": \"Elasticsearch version\",\n \"JSONPath\": \".spec.version\"\n },\n {\n \"name\": \"phase\",\n \"type\": \"string\",\n \"JSONPath\": \".status.phase\"\n },\n {\n \"name\": \"age\",\n \"type\": \"date\",\n \"JSONPath\": \".metadata.creationTimestamp\"\n }\n ],\n \"conversion\": {\n \"strategy\": \"None\"\n },\n \"preserveUnknownFields\": true\n },\n \"status\": {\n \"conditions\": [\n {\n \"type\": \"NonStructuralSchema\",\n \"status\": \"True\",\n \"lastTransitionTime\": \"2020-04-28T23:31:51Z\",\n \"reason\": \"Violations\",\n \"message\": \"[spec.validation.openAPIV3Schema.properties[spec].properties[http].properties[service].properties[spec].properties[ports].items.properties[targetPort].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[nodeSets].items.properties[volumeClaimTemplates].items.properties[spec].properties[resources].properties[limits].additionalProperties.type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[nodeSets].items.properties[volumeClaimTemplates].items.properties[spec].properties[resources].properties[requests].additionalProperties.type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[nodeSets].items.properties[volumeClaimTemplates].items.properties[status].properties[capacity].additionalProperties.type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[podDisruptionBudget].properties[spec].properties[maxUnavailable].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[podDisruptionBudget].properties[spec].properties[minAvailable].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[transport].properties[service].properties[spec].properties[ports].items.properties[targetPort].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.type: Required value: must not be empty at the root]\"\n },\n {\n \"type\": \"NamesAccepted\",\n \"status\": \"True\",\n \"lastTransitionTime\": \"2020-04-28T23:31:51Z\",\n \"reason\": \"NoConflicts\",\n \"message\": \"no conflicts found\"\n },\n {\n \"type\": \"Established\",\n \"status\": \"True\",\n \"lastTransitionTime\": \"2020-04-28T23:31:51Z\",\n \"reason\": \"InitialNamesAccepted\",\n \"message\": \"the initial names have been accepted\"\n }\n ],\n \"acceptedNames\": {\n \"plural\": \"elasticsearches\",\n \"singular\": \"elasticsearch\",\n \"shortNames\": [\n \"es\"\n ],\n \"kind\": \"Elasticsearch\",\n \"listKind\": \"ElasticsearchList\",\n \"categories\": [\n \"elastic\"\n ]\n },\n \"storedVersions\": [\n \"v1\"\n ]\n }\n}\n" }, { "path": "pkg/backup/actions/testdata/v1beta1/gcpsamples.gcp.stacks.crossplane.io.json", "content": "{\n \"kind\": \"CustomResourceDefinition\",\n \"apiVersion\": \"apiextensions.k8s.io/v1beta1\",\n \"metadata\": {\n \"name\": \"gcpsamples.gcp.stacks.crossplane.io\",\n \"selfLink\": \"/apis/apiextensions.k8s.io/v1beta1/customresourcedefinitions/gcpsamples.gcp.stacks.crossplane.io\",\n \"uid\": \"c0bbac74-acab-4620-b628-1d5f91b19040\",\n \"resourceVersion\": \"5567\",\n \"generation\": 1,\n \"creationTimestamp\": \"2020-04-20T17:27:56Z\",\n \"annotations\": {\n \"kubectl.kubernetes.io/last-applied-configuration\": \"{\\\"apiVersion\\\":\\\"apiextensions.k8s.io/v1beta1\\\",\\\"kind\\\":\\\"CustomResourceDefinition\\\",\\\"metadata\\\":{\\\"annotations\\\":{},\\\"creationTimestamp\\\":\\\"2020-04-20T16:57:37Z\\\",\\\"generation\\\":1,\\\"name\\\":\\\"gcpsamples.gcp.stacks.crossplane.io\\\",\\\"resourceVersion\\\":\\\"549\\\",\\\"selfLink\\\":\\\"/apis/apiextensions.k8s.io/v1/customresourcedefinitions/gcpsamples.gcp.stacks.crossplane.io\\\",\\\"uid\\\":\\\"db5f4321-3226-44b0-8247-66fd7ef59dc8\\\"},\\\"spec\\\":{\\\"conversion\\\":{\\\"strategy\\\":\\\"None\\\"},\\\"group\\\":\\\"gcp.stacks.crossplane.io\\\",\\\"names\\\":{\\\"kind\\\":\\\"GCPSample\\\",\\\"listKind\\\":\\\"GCPSampleList\\\",\\\"plural\\\":\\\"gcpsamples\\\",\\\"singular\\\":\\\"gcpsample\\\"},\\\"preserveUnknownFields\\\":true,\\\"scope\\\":\\\"Cluster\\\",\\\"versions\\\":[{\\\"name\\\":\\\"v1alpha1\\\",\\\"served\\\":true,\\\"storage\\\":true}]},\\\"status\\\":{\\\"acceptedNames\\\":{\\\"kind\\\":\\\"GCPSample\\\",\\\"listKind\\\":\\\"GCPSampleList\\\",\\\"plural\\\":\\\"gcpsamples\\\",\\\"singular\\\":\\\"gcpsample\\\"},\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2020-04-20T16:57:37Z\\\",\\\"message\\\":\\\"no conflicts found\\\",\\\"reason\\\":\\\"NoConflicts\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"NamesAccepted\\\"},{\\\"lastTransitionTime\\\":\\\"2020-04-20T16:57:37Z\\\",\\\"message\\\":\\\"the initial names have been accepted\\\",\\\"reason\\\":\\\"InitialNamesAccepted\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Established\\\"}],\\\"storedVersions\\\":[\\\"v1alpha1\\\"]}}\\n\"\n }\n },\n \"spec\": {\n \"group\": \"gcp.stacks.crossplane.io\",\n \"version\": \"v1alpha1\",\n \"names\": {\n \"plural\": \"gcpsamples\",\n \"singular\": \"gcpsample\",\n \"kind\": \"GCPSample\",\n \"listKind\": \"GCPSampleList\"\n },\n \"scope\": \"Cluster\",\n \"versions\": [\n {\n \"name\": \"v1alpha1\",\n \"served\": true,\n \"storage\": true\n }\n ],\n \"conversion\": {\n \"strategy\": \"None\"\n },\n \"preserveUnknownFields\": true\n },\n \"status\": {\n \"conditions\": [\n {\n \"type\": \"NamesAccepted\",\n \"status\": \"True\",\n \"lastTransitionTime\": \"2020-04-20T17:27:56Z\",\n \"reason\": \"NoConflicts\",\n \"message\": \"no conflicts found\"\n },\n {\n \"type\": \"Established\",\n \"status\": \"True\",\n \"lastTransitionTime\": \"2020-04-20T17:27:56Z\",\n \"reason\": \"InitialNamesAccepted\",\n \"message\": \"the initial names have been accepted\"\n }\n ],\n \"acceptedNames\": {\n \"plural\": \"gcpsamples\",\n \"singular\": \"gcpsample\",\n \"kind\": \"GCPSample\",\n \"listKind\": \"GCPSampleList\"\n },\n \"storedVersions\": [\n \"v1alpha1\"\n ]\n }\n}\n" }, { "path": "pkg/backup/actions/testdata/v1beta1/kibanas.kibana.k8s.elastic.co.json", "content": "{\n \"kind\": \"CustomResourceDefinition\",\n \"apiVersion\": \"apiextensions.k8s.io/v1beta1\",\n \"metadata\": {\n \"name\": \"kibanas.kibana.k8s.elastic.co\",\n \"selfLink\": \"/apis/apiextensions.k8s.io/v1beta1/customresourcedefinitions/kibanas.kibana.k8s.elastic.co\",\n \"uid\": \"95f42a77-654f-4380-a6b1-1fe2587f0713\",\n \"resourceVersion\": \"1703552\",\n \"generation\": 1,\n \"creationTimestamp\": \"2020-04-28T23:31:53Z\",\n \"labels\": {\n \"velero.io/backup-name\": \"es\",\n \"velero.io/restore-name\": \"es-crds\"\n },\n \"annotations\": {\n \"controller-gen.kubebuilder.io/version\": \"v0.2.5\",\n \"kubectl.kubernetes.io/last-applied-configuration\": \"{\\\"apiVersion\\\":\\\"apiextensions.k8s.io/v1beta1\\\",\\\"kind\\\":\\\"CustomResourceDefinition\\\",\\\"metadata\\\":{\\\"annotations\\\":{\\\"controller-gen.kubebuilder.io/version\\\":\\\"v0.2.5\\\"},\\\"creationTimestamp\\\":null,\\\"name\\\":\\\"kibanas.kibana.k8s.elastic.co\\\"},\\\"spec\\\":{\\\"additionalPrinterColumns\\\":[{\\\"JSONPath\\\":\\\".status.health\\\",\\\"name\\\":\\\"health\\\",\\\"type\\\":\\\"string\\\"},{\\\"JSONPath\\\":\\\".status.availableNodes\\\",\\\"description\\\":\\\"Available nodes\\\",\\\"name\\\":\\\"nodes\\\",\\\"type\\\":\\\"integer\\\"},{\\\"JSONPath\\\":\\\".spec.version\\\",\\\"description\\\":\\\"Kibana version\\\",\\\"name\\\":\\\"version\\\",\\\"type\\\":\\\"string\\\"},{\\\"JSONPath\\\":\\\".metadata.creationTimestamp\\\",\\\"name\\\":\\\"age\\\",\\\"type\\\":\\\"date\\\"}],\\\"group\\\":\\\"kibana.k8s.elastic.co\\\",\\\"names\\\":{\\\"categories\\\":[\\\"elastic\\\"],\\\"kind\\\":\\\"Kibana\\\",\\\"listKind\\\":\\\"KibanaList\\\",\\\"plural\\\":\\\"kibanas\\\",\\\"shortNames\\\":[\\\"kb\\\"],\\\"singular\\\":\\\"kibana\\\"},\\\"scope\\\":\\\"Namespaced\\\",\\\"subresources\\\":{\\\"status\\\":{}},\\\"validation\\\":{\\\"openAPIV3Schema\\\":{\\\"description\\\":\\\"Kibana represents a Kibana resource in a Kubernetes cluster.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"metadata\\\":{\\\"type\\\":\\\"object\\\"},\\\"spec\\\":{\\\"description\\\":\\\"KibanaSpec holds the specification of a Kibana instance.\\\",\\\"properties\\\":{\\\"config\\\":{\\\"description\\\":\\\"Config holds the Kibana configuration. See: https://www.elastic.co/guide/en/kibana/current/settings.html\\\",\\\"type\\\":\\\"object\\\"},\\\"count\\\":{\\\"description\\\":\\\"Count of Kibana instances to deploy.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"elasticsearchRef\\\":{\\\"description\\\":\\\"ElasticsearchRef is a reference to an Elasticsearch cluster running in the same Kubernetes cluster.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the Kubernetes object.\\\",\\\"type\\\":\\\"string\\\"},\\\"namespace\\\":{\\\"description\\\":\\\"Namespace of the Kubernetes object. If empty, defaults to the current namespace.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"http\\\":{\\\"description\\\":\\\"HTTP holds the HTTP layer configuration for Kibana.\\\",\\\"properties\\\":{\\\"service\\\":{\\\"description\\\":\\\"Service defines the template for the associated Kubernetes Service object.\\\",\\\"properties\\\":{\\\"metadata\\\":{\\\"description\\\":\\\"ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored.\\\",\\\"type\\\":\\\"object\\\"},\\\"spec\\\":{\\\"description\\\":\\\"Spec is the specification of the service.\\\",\\\"properties\\\":{\\\"clusterIP\\\":{\\\"description\\\":\\\"clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are \\\\\\\"None\\\\\\\", empty string (\\\\\\\"\\\\\\\"), or a valid IP address. \\\\\\\"None\\\\\\\" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\\\",\\\"type\\\":\\\"string\\\"},\\\"externalIPs\\\":{\\\"description\\\":\\\"externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"externalName\\\":{\\\"description\\\":\\\"externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName.\\\",\\\"type\\\":\\\"string\\\"},\\\"externalTrafficPolicy\\\":{\\\"description\\\":\\\"externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. \\\\\\\"Local\\\\\\\" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. \\\\\\\"Cluster\\\\\\\" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading.\\\",\\\"type\\\":\\\"string\\\"},\\\"healthCheckNodePort\\\":{\\\"description\\\":\\\"healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"ipFamily\\\":{\\\"description\\\":\\\"ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment.\\\",\\\"type\\\":\\\"string\\\"},\\\"loadBalancerIP\\\":{\\\"description\\\":\\\"Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.\\\",\\\"type\\\":\\\"string\\\"},\\\"loadBalancerSourceRanges\\\":{\\\"description\\\":\\\"If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature.\\\\\\\" More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"ports\\\":{\\\"description\\\":\\\"The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\\\",\\\"items\\\":{\\\"description\\\":\\\"ServicePort contains information on service's port.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service.\\\",\\\"type\\\":\\\"string\\\"},\\\"nodePort\\\":{\\\"description\\\":\\\"The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"port\\\":{\\\"description\\\":\\\"The port that will be exposed by this service.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"protocol\\\":{\\\"description\\\":\\\"The IP protocol for this port. Supports \\\\\\\"TCP\\\\\\\", \\\\\\\"UDP\\\\\\\", and \\\\\\\"SCTP\\\\\\\". Default is TCP.\\\",\\\"type\\\":\\\"string\\\"},\\\"targetPort\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"integer\\\"},{\\\"type\\\":\\\"string\\\"}],\\\"description\\\":\\\"Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod's container ports. If this is not specified, the value of the 'port' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the 'port' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"publishNotReadyAddresses\\\":{\\\"description\\\":\\\"publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"selector\\\":{\\\"additionalProperties\\\":{\\\"type\\\":\\\"string\\\"},\\\"description\\\":\\\"Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/\\\",\\\"type\\\":\\\"object\\\"},\\\"sessionAffinity\\\":{\\\"description\\\":\\\"Supports \\\\\\\"ClientIP\\\\\\\" and \\\\\\\"None\\\\\\\". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\\\",\\\"type\\\":\\\"string\\\"},\\\"sessionAffinityConfig\\\":{\\\"description\\\":\\\"sessionAffinityConfig contains the configurations of session affinity.\\\",\\\"properties\\\":{\\\"clientIP\\\":{\\\"description\\\":\\\"clientIP contains the configurations of Client IP based session affinity.\\\",\\\"properties\\\":{\\\"timeoutSeconds\\\":{\\\"description\\\":\\\"timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be \\\\u003e0 \\\\u0026\\\\u0026 \\\\u003c=86400(for 1 day) if ServiceAffinity == \\\\\\\"ClientIP\\\\\\\". Default value is 10800(for 3 hours).\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"topologyKeys\\\":{\\\"description\\\":\\\"topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value \\\\\\\"*\\\\\\\" may be used to mean \\\\\\\"any topology\\\\\\\". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"type\\\":{\\\"description\\\":\\\"type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. \\\\\\\"ExternalName\\\\\\\" maps to the specified externalName. \\\\\\\"ClusterIP\\\\\\\" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is \\\\\\\"None\\\\\\\", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. \\\\\\\"NodePort\\\\\\\" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. \\\\\\\"LoadBalancer\\\\\\\" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"tls\\\":{\\\"description\\\":\\\"TLS defines options for configuring TLS for HTTP.\\\",\\\"properties\\\":{\\\"certificate\\\":{\\\"description\\\":\\\"Certificate is a reference to a Kubernetes secret that contains the certificate and private key for enabling TLS. The referenced secret should contain the following: \\\\n - `ca.crt`: The certificate authority (optional). - `tls.crt`: The certificate (or a chain). - `tls.key`: The private key to the first certificate in the certificate chain.\\\",\\\"properties\\\":{\\\"secretName\\\":{\\\"description\\\":\\\"SecretName is the name of the secret.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"selfSignedCertificate\\\":{\\\"description\\\":\\\"SelfSignedCertificate allows configuring the self-signed certificate generated by the operator.\\\",\\\"properties\\\":{\\\"disabled\\\":{\\\"description\\\":\\\"Disabled indicates that the provisioning of the self-signed certificate should be disabled.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"subjectAltNames\\\":{\\\"description\\\":\\\"SubjectAlternativeNames is a list of SANs to include in the generated HTTP TLS certificate.\\\",\\\"items\\\":{\\\"description\\\":\\\"SubjectAlternativeName represents a SAN entry in a x509 certificate.\\\",\\\"properties\\\":{\\\"dns\\\":{\\\"description\\\":\\\"DNS is the DNS name of the subject.\\\",\\\"type\\\":\\\"string\\\"},\\\"ip\\\":{\\\"description\\\":\\\"IP is the IP address of the subject.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"image\\\":{\\\"description\\\":\\\"Image is the Kibana Docker image to deploy.\\\",\\\"type\\\":\\\"string\\\"},\\\"podTemplate\\\":{\\\"description\\\":\\\"PodTemplate provides customisation options (labels, annotations, affinity rules, resource requests, and so on) for the Kibana pods\\\",\\\"type\\\":\\\"object\\\"},\\\"secureSettings\\\":{\\\"description\\\":\\\"SecureSettings is a list of references to Kubernetes secrets containing sensitive configuration options for Kibana. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-kibana.html#k8s-kibana-secure-settings\\\",\\\"items\\\":{\\\"description\\\":\\\"SecretSource defines a data source based on a Kubernetes Secret.\\\",\\\"properties\\\":{\\\"entries\\\":{\\\"description\\\":\\\"Entries define how to project each key-value pair in the secret to filesystem paths. If not defined, all keys will be projected to similarly named paths in the filesystem. If defined, only the specified keys will be projected to the corresponding paths.\\\",\\\"items\\\":{\\\"description\\\":\\\"KeyToPath defines how to map a key in a Secret object to a filesystem path.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"Key is the key contained in the secret.\\\",\\\"type\\\":\\\"string\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path is the relative file path to map the key to. Path must not be an absolute file path and must not contain any \\\\\\\"..\\\\\\\" components.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"secretName\\\":{\\\"description\\\":\\\"SecretName is the name of the secret.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"secretName\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"serviceAccountName\\\":{\\\"description\\\":\\\"ServiceAccountName is used to check access from the current resource to a resource (eg. Elasticsearch) in a different namespace. Can only be used if ECK is enforcing RBAC on references.\\\",\\\"type\\\":\\\"string\\\"},\\\"version\\\":{\\\"description\\\":\\\"Version of Kibana.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"version\\\"],\\\"type\\\":\\\"object\\\"},\\\"status\\\":{\\\"description\\\":\\\"KibanaStatus defines the observed state of Kibana\\\",\\\"properties\\\":{\\\"associationStatus\\\":{\\\"description\\\":\\\"AssociationStatus is the status of an association resource.\\\",\\\"type\\\":\\\"string\\\"},\\\"availableNodes\\\":{\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"health\\\":{\\\"description\\\":\\\"KibanaHealth expresses the status of the Kibana instances.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}}}},\\\"version\\\":\\\"v1\\\",\\\"versions\\\":[{\\\"name\\\":\\\"v1\\\",\\\"served\\\":true,\\\"storage\\\":true},{\\\"name\\\":\\\"v1beta1\\\",\\\"served\\\":true,\\\"storage\\\":false},{\\\"name\\\":\\\"v1alpha1\\\",\\\"served\\\":false,\\\"storage\\\":false}]},\\\"status\\\":{\\\"acceptedNames\\\":{\\\"kind\\\":\\\"\\\",\\\"plural\\\":\\\"\\\"},\\\"conditions\\\":[],\\\"storedVersions\\\":[]}}\\n\"\n }\n },\n \"spec\": {\n \"group\": \"kibana.k8s.elastic.co\",\n \"version\": \"v1\",\n \"names\": {\n \"plural\": \"kibanas\",\n \"singular\": \"kibana\",\n \"shortNames\": [\n \"kb\"\n ],\n \"kind\": \"Kibana\",\n \"listKind\": \"KibanaList\",\n \"categories\": [\n \"elastic\"\n ]\n },\n \"scope\": \"Namespaced\",\n \"validation\": {\n \"openAPIV3Schema\": {\n \"description\": \"Kibana represents a Kibana resource in a Kubernetes cluster.\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"KibanaSpec holds the specification of a Kibana instance.\",\n \"type\": \"object\",\n \"required\": [\n \"version\"\n ],\n \"properties\": {\n \"config\": {\n \"description\": \"Config holds the Kibana configuration. See: https://www.elastic.co/guide/en/kibana/current/settings.html\",\n \"type\": \"object\"\n },\n \"count\": {\n \"description\": \"Count of Kibana instances to deploy.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"elasticsearchRef\": {\n \"description\": \"ElasticsearchRef is a reference to an Elasticsearch cluster running in the same Kubernetes cluster.\",\n \"type\": \"object\",\n \"required\": [\n \"name\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the Kubernetes object.\",\n \"type\": \"string\"\n },\n \"namespace\": {\n \"description\": \"Namespace of the Kubernetes object. If empty, defaults to the current namespace.\",\n \"type\": \"string\"\n }\n }\n },\n \"http\": {\n \"description\": \"HTTP holds the HTTP layer configuration for Kibana.\",\n \"type\": \"object\",\n \"properties\": {\n \"service\": {\n \"description\": \"Service defines the template for the associated Kubernetes Service object.\",\n \"type\": \"object\",\n \"properties\": {\n \"metadata\": {\n \"description\": \"ObjectMeta is the metadata of the service. The name and namespace provided here are managed by ECK and will be ignored.\",\n \"type\": \"object\"\n },\n \"spec\": {\n \"description\": \"Spec is the specification of the service.\",\n \"type\": \"object\",\n \"properties\": {\n \"clusterIP\": {\n \"description\": \"clusterIP is the IP address of the service and is usually assigned randomly by the master. If an address is specified manually and is not in use by others, it will be allocated to the service; otherwise, creation of the service will fail. This field can not be changed through updates. Valid values are \\\"None\\\", empty string (\\\"\\\"), or a valid IP address. \\\"None\\\" can be specified for headless services when proxying is not required. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"externalIPs\": {\n \"description\": \"externalIPs is a list of IP addresses for which nodes in the cluster will also accept traffic for this service. These IPs are not managed by Kubernetes. The user is responsible for ensuring that traffic arrives at a node with this IP. A common example is external load-balancers that are not part of the Kubernetes system.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"externalName\": {\n \"description\": \"externalName is the external reference that kubedns or equivalent will return as a CNAME record for this service. No proxying will be involved. Must be a valid RFC-1123 hostname (https://tools.ietf.org/html/rfc1123) and requires Type to be ExternalName.\",\n \"type\": \"string\"\n },\n \"externalTrafficPolicy\": {\n \"description\": \"externalTrafficPolicy denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. \\\"Local\\\" preserves the client source IP and avoids a second hop for LoadBalancer and Nodeport type services, but risks potentially imbalanced traffic spreading. \\\"Cluster\\\" obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading.\",\n \"type\": \"string\"\n },\n \"healthCheckNodePort\": {\n \"description\": \"healthCheckNodePort specifies the healthcheck nodePort for the service. If not specified, HealthCheckNodePort is created by the service api backend with the allocated nodePort. Will use user-specified nodePort value if specified by the client. Only effects when Type is set to LoadBalancer and ExternalTrafficPolicy is set to Local.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"ipFamily\": {\n \"description\": \"ipFamily specifies whether this Service has a preference for a particular IP family (e.g. IPv4 vs. IPv6). If a specific IP family is requested, the clusterIP field will be allocated from that family, if it is available in the cluster. If no IP family is requested, the cluster's primary IP family will be used. Other IP fields (loadBalancerIP, loadBalancerSourceRanges, externalIPs) and controllers which allocate external load-balancers should use the same IP family. Endpoints for this Service will be of this family. This field is immutable after creation. Assigning a ServiceIPFamily not available in the cluster (e.g. IPv6 in IPv4 only cluster) is an error condition and will fail during clusterIP assignment.\",\n \"type\": \"string\"\n },\n \"loadBalancerIP\": {\n \"description\": \"Only applies to Service Type: LoadBalancer LoadBalancer will get created with the IP specified in this field. This feature depends on whether the underlying cloud-provider supports specifying the loadBalancerIP when a load balancer is created. This field will be ignored if the cloud-provider does not support the feature.\",\n \"type\": \"string\"\n },\n \"loadBalancerSourceRanges\": {\n \"description\": \"If specified and supported by the platform, this will restrict traffic through the cloud-provider load-balancer will be restricted to the specified client IPs. This field will be ignored if the cloud-provider does not support the feature.\\\" More info: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"ports\": {\n \"description\": \"The list of ports that are exposed by this service. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"ServicePort contains information on service's port.\",\n \"type\": \"object\",\n \"required\": [\n \"port\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"The name of this port within the service. This must be a DNS_LABEL. All ports within a ServiceSpec must have unique names. When considering the endpoints for a Service, this must match the 'name' field in the EndpointPort. Optional if only one ServicePort is defined on this service.\",\n \"type\": \"string\"\n },\n \"nodePort\": {\n \"description\": \"The port on each node on which this service is exposed when type=NodePort or LoadBalancer. Usually assigned by the system. If specified, it will be allocated to the service if unused or else creation of the service will fail. Default is to auto-allocate a port if the ServiceType of this Service requires one. More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"port\": {\n \"description\": \"The port that will be exposed by this service.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"protocol\": {\n \"description\": \"The IP protocol for this port. Supports \\\"TCP\\\", \\\"UDP\\\", and \\\"SCTP\\\". Default is TCP.\",\n \"type\": \"string\"\n },\n \"targetPort\": {\n \"description\": \"Number or name of the port to access on the pods targeted by the service. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME. If this is a string, it will be looked up as a named port in the target Pod's container ports. If this is not specified, the value of the 'port' field is used (an identity map). This field is ignored for services with clusterIP=None, and should be omitted or set equal to the 'port' field. More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service\",\n \"anyOf\": [\n {\n \"type\": \"integer\"\n },\n {\n \"type\": \"string\"\n }\n ]\n }\n }\n }\n },\n \"publishNotReadyAddresses\": {\n \"description\": \"publishNotReadyAddresses, when set to true, indicates that DNS implementations must publish the notReadyAddresses of subsets for the Endpoints associated with the Service. The default value is false. The primary use case for setting this field is to use a StatefulSet's Headless Service to propagate SRV records for its Pods without respect to their readiness for purpose of peer discovery.\",\n \"type\": \"boolean\"\n },\n \"selector\": {\n \"description\": \"Route service traffic to pods with label keys and values matching this selector. If empty or not present, the service is assumed to have an external process managing its endpoints, which Kubernetes will not modify. Only applies to types ClusterIP, NodePort, and LoadBalancer. Ignored if type is ExternalName. More info: https://kubernetes.io/docs/concepts/services-networking/service/\",\n \"type\": \"object\",\n \"additionalProperties\": {\n \"type\": \"string\"\n }\n },\n \"sessionAffinity\": {\n \"description\": \"Supports \\\"ClientIP\\\" and \\\"None\\\". Used to maintain session affinity. Enable client IP based session affinity. Must be ClientIP or None. Defaults to None. More info: https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies\",\n \"type\": \"string\"\n },\n \"sessionAffinityConfig\": {\n \"description\": \"sessionAffinityConfig contains the configurations of session affinity.\",\n \"type\": \"object\",\n \"properties\": {\n \"clientIP\": {\n \"description\": \"clientIP contains the configurations of Client IP based session affinity.\",\n \"type\": \"object\",\n \"properties\": {\n \"timeoutSeconds\": {\n \"description\": \"timeoutSeconds specifies the seconds of ClientIP type session sticky time. The value must be >0 && <=86400(for 1 day) if ServiceAffinity == \\\"ClientIP\\\". Default value is 10800(for 3 hours).\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n }\n }\n }\n }\n },\n \"topologyKeys\": {\n \"description\": \"topologyKeys is a preference-order list of topology keys which implementations of services should use to preferentially sort endpoints when accessing this Service, it can not be used at the same time as externalTrafficPolicy=Local. Topology keys must be valid label keys and at most 16 keys may be specified. Endpoints are chosen based on the first topology key with available backends. If this field is specified and all entries have no backends that match the topology of the client, the service has no backends for that client and connections should fail. The special value \\\"*\\\" may be used to mean \\\"any topology\\\". This catch-all value, if used, only makes sense as the last value in the list. If this is not specified or empty, no topology constraints will be applied.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"type\": {\n \"description\": \"type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. \\\"ExternalName\\\" maps to the specified externalName. \\\"ClusterIP\\\" allocates a cluster-internal IP address for load-balancing to endpoints. Endpoints are determined by the selector or if that is not specified, by manual construction of an Endpoints object. If clusterIP is \\\"None\\\", no virtual IP is allocated and the endpoints are published as a set of endpoints rather than a stable IP. \\\"NodePort\\\" builds on ClusterIP and allocates a port on every node which routes to the clusterIP. \\\"LoadBalancer\\\" builds on NodePort and creates an external load-balancer (if supported in the current cloud) which routes to the clusterIP. More info: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types\",\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"tls\": {\n \"description\": \"TLS defines options for configuring TLS for HTTP.\",\n \"type\": \"object\",\n \"properties\": {\n \"certificate\": {\n \"description\": \"Certificate is a reference to a Kubernetes secret that contains the certificate and private key for enabling TLS. The referenced secret should contain the following: \\n - `ca.crt`: The certificate authority (optional). - `tls.crt`: The certificate (or a chain). - `tls.key`: The private key to the first certificate in the certificate chain.\",\n \"type\": \"object\",\n \"properties\": {\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n }\n },\n \"selfSignedCertificate\": {\n \"description\": \"SelfSignedCertificate allows configuring the self-signed certificate generated by the operator.\",\n \"type\": \"object\",\n \"properties\": {\n \"disabled\": {\n \"description\": \"Disabled indicates that the provisioning of the self-signed certificate should be disabled.\",\n \"type\": \"boolean\"\n },\n \"subjectAltNames\": {\n \"description\": \"SubjectAlternativeNames is a list of SANs to include in the generated HTTP TLS certificate.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"SubjectAlternativeName represents a SAN entry in a x509 certificate.\",\n \"type\": \"object\",\n \"properties\": {\n \"dns\": {\n \"description\": \"DNS is the DNS name of the subject.\",\n \"type\": \"string\"\n },\n \"ip\": {\n \"description\": \"IP is the IP address of the subject.\",\n \"type\": \"string\"\n }\n }\n }\n }\n }\n }\n }\n }\n }\n },\n \"image\": {\n \"description\": \"Image is the Kibana Docker image to deploy.\",\n \"type\": \"string\"\n },\n \"podTemplate\": {\n \"description\": \"PodTemplate provides customisation options (labels, annotations, affinity rules, resource requests, and so on) for the Kibana pods\",\n \"type\": \"object\"\n },\n \"secureSettings\": {\n \"description\": \"SecureSettings is a list of references to Kubernetes secrets containing sensitive configuration options for Kibana. See: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-kibana.html#k8s-kibana-secure-settings\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"SecretSource defines a data source based on a Kubernetes Secret.\",\n \"type\": \"object\",\n \"required\": [\n \"secretName\"\n ],\n \"properties\": {\n \"entries\": {\n \"description\": \"Entries define how to project each key-value pair in the secret to filesystem paths. If not defined, all keys will be projected to similarly named paths in the filesystem. If defined, only the specified keys will be projected to the corresponding paths.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"KeyToPath defines how to map a key in a Secret object to a filesystem path.\",\n \"type\": \"object\",\n \"required\": [\n \"key\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"Key is the key contained in the secret.\",\n \"type\": \"string\"\n },\n \"path\": {\n \"description\": \"Path is the relative file path to map the key to. Path must not be an absolute file path and must not contain any \\\"..\\\" components.\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"secretName\": {\n \"description\": \"SecretName is the name of the secret.\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"serviceAccountName\": {\n \"description\": \"ServiceAccountName is used to check access from the current resource to a resource (eg. Elasticsearch) in a different namespace. Can only be used if ECK is enforcing RBAC on references.\",\n \"type\": \"string\"\n },\n \"version\": {\n \"description\": \"Version of Kibana.\",\n \"type\": \"string\"\n }\n }\n },\n \"status\": {\n \"description\": \"KibanaStatus defines the observed state of Kibana\",\n \"type\": \"object\",\n \"properties\": {\n \"associationStatus\": {\n \"description\": \"AssociationStatus is the status of an association resource.\",\n \"type\": \"string\"\n },\n \"availableNodes\": {\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"health\": {\n \"description\": \"KibanaHealth expresses the status of the Kibana instances.\",\n \"type\": \"string\"\n }\n }\n }\n }\n }\n },\n \"subresources\": {\n \"status\": {}\n },\n \"versions\": [\n {\n \"name\": \"v1\",\n \"served\": true,\n \"storage\": true\n },\n {\n \"name\": \"v1beta1\",\n \"served\": true,\n \"storage\": false\n },\n {\n \"name\": \"v1alpha1\",\n \"served\": false,\n \"storage\": false\n }\n ],\n \"additionalPrinterColumns\": [\n {\n \"name\": \"health\",\n \"type\": \"string\",\n \"JSONPath\": \".status.health\"\n },\n {\n \"name\": \"nodes\",\n \"type\": \"integer\",\n \"description\": \"Available nodes\",\n \"JSONPath\": \".status.availableNodes\"\n },\n {\n \"name\": \"version\",\n \"type\": \"string\",\n \"description\": \"Kibana version\",\n \"JSONPath\": \".spec.version\"\n },\n {\n \"name\": \"age\",\n \"type\": \"date\",\n \"JSONPath\": \".metadata.creationTimestamp\"\n }\n ],\n \"conversion\": {\n \"strategy\": \"None\"\n },\n \"preserveUnknownFields\": true\n },\n \"status\": {\n \"conditions\": [\n {\n \"type\": \"NonStructuralSchema\",\n \"status\": \"True\",\n \"lastTransitionTime\": \"2020-04-28T23:31:53Z\",\n \"reason\": \"Violations\",\n \"message\": \"[spec.validation.openAPIV3Schema.properties[spec].properties[http].properties[service].properties[spec].properties[ports].items.properties[targetPort].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.type: Required value: must not be empty at the root]\"\n },\n {\n \"type\": \"NamesAccepted\",\n \"status\": \"True\",\n \"lastTransitionTime\": \"2020-04-28T23:31:53Z\",\n \"reason\": \"NoConflicts\",\n \"message\": \"no conflicts found\"\n },\n {\n \"type\": \"Established\",\n \"status\": \"True\",\n \"lastTransitionTime\": \"2020-04-28T23:31:53Z\",\n \"reason\": \"InitialNamesAccepted\",\n \"message\": \"the initial names have been accepted\"\n }\n ],\n \"acceptedNames\": {\n \"plural\": \"kibanas\",\n \"singular\": \"kibana\",\n \"shortNames\": [\n \"kb\"\n ],\n \"kind\": \"Kibana\",\n \"listKind\": \"KibanaList\",\n \"categories\": [\n \"elastic\"\n ]\n },\n \"storedVersions\": [\n \"v1\"\n ]\n }\n}\n" }, { "path": "pkg/backup/actions/testdata/v1beta1/prometheuses.monitoring.coreos.com.json", "content": "{\n \"kind\": \"CustomResourceDefinition\",\n \"apiVersion\": \"apiextensions.k8s.io/v1beta1\",\n \"metadata\": {\n \"name\": \"prometheuses.monitoring.coreos.com\",\n \"selfLink\": \"/apis/apiextensions.k8s.io/v1beta1/customresourcedefinitions/prometheuses.monitoring.coreos.com\",\n \"uid\": \"472b9025-d931-4d48-8a6f-77c3b7c33f6e\",\n \"resourceVersion\": \"207497\",\n \"generation\": 1,\n \"creationTimestamp\": \"2020-05-05T16:58:10Z\",\n \"labels\": {\n \"app\": \"prometheus-operator\"\n },\n \"annotations\": {\n \"helm.sh/hook\": \"crd-install\",\n \"kubectl.kubernetes.io/last-applied-configuration\": \"{\\\"apiVersion\\\":\\\"apiextensions.k8s.io/v1beta1\\\",\\\"kind\\\":\\\"CustomResourceDefinition\\\",\\\"metadata\\\":{\\\"annotations\\\":{\\\"helm.sh/hook\\\":\\\"crd-install\\\"},\\\"creationTimestamp\\\":null,\\\"labels\\\":{\\\"app\\\":\\\"prometheus-operator\\\"},\\\"name\\\":\\\"prometheuses.monitoring.coreos.com\\\"},\\\"spec\\\":{\\\"group\\\":\\\"monitoring.coreos.com\\\",\\\"names\\\":{\\\"kind\\\":\\\"Prometheus\\\",\\\"plural\\\":\\\"prometheuses\\\"},\\\"scope\\\":\\\"Namespaced\\\",\\\"validation\\\":{\\\"openAPIV3Schema\\\":{\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"spec\\\":{\\\"description\\\":\\\"PrometheusSpec is a specification of the desired behavior of the Prometheus cluster. More info: https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#spec-and-status\\\",\\\"properties\\\":{\\\"additionalAlertManagerConfigs\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"additionalAlertRelabelConfigs\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"additionalScrapeConfigs\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"affinity\\\":{\\\"description\\\":\\\"Affinity is a group of affinity scheduling rules.\\\",\\\"properties\\\":{\\\"nodeAffinity\\\":{\\\"description\\\":\\\"Node affinity is a group of node affinity scheduling rules.\\\",\\\"properties\\\":{\\\"preferredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\\\\\"weight\\\\\\\" to the sum if the node matches the corresponding matchExpressions; the node(s) with the highest sum are the most preferred.\\\",\\\"items\\\":{\\\"description\\\":\\\"An empty preferred scheduling term matches all objects with implicit weight 0 (i.e. it's a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op).\\\",\\\"properties\\\":{\\\"preference\\\":{\\\"description\\\":\\\"A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"A list of node selector requirements by node's labels.\\\",\\\"items\\\":{\\\"description\\\":\\\"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchFields\\\":{\\\"description\\\":\\\"A list of node selector requirements by node's fields.\\\",\\\"items\\\":{\\\"description\\\":\\\"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"weight\\\":{\\\"description\\\":\\\"Weight associated with matching the corresponding nodeSelectorTerm, in the range 1-100.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"required\\\":[\\\"weight\\\",\\\"preference\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"requiredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"A node selector represents the union of the results of one or more label queries over a set of nodes; that is, it represents the OR of the selectors represented by the node selector terms.\\\",\\\"properties\\\":{\\\"nodeSelectorTerms\\\":{\\\"description\\\":\\\"Required. A list of node selector terms. The terms are ORed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"A list of node selector requirements by node's labels.\\\",\\\"items\\\":{\\\"description\\\":\\\"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchFields\\\":{\\\"description\\\":\\\"A list of node selector requirements by node's fields.\\\",\\\"items\\\":{\\\"description\\\":\\\"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"nodeSelectorTerms\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"podAffinity\\\":{\\\"description\\\":\\\"Pod affinity is a group of inter pod affinity scheduling rules.\\\",\\\"properties\\\":{\\\"preferredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\\\\\"weight\\\\\\\" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.\\\",\\\"items\\\":{\\\"description\\\":\\\"The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)\\\",\\\"properties\\\":{\\\"podAffinityTerm\\\":{\\\"description\\\":\\\"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\\\u003ctopologyKey\\\\u003e matches that of any node on which a pod of the set of pods is running\\\",\\\"properties\\\":{\\\"labelSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"namespaces\\\":{\\\"description\\\":\\\"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\\\\\"this pod's namespace\\\\\\\"\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"topologyKey\\\":{\\\"description\\\":\\\"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"topologyKey\\\"],\\\"type\\\":\\\"object\\\"},\\\"weight\\\":{\\\"description\\\":\\\"weight associated with matching the corresponding podAffinityTerm, in the range 1-100.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"required\\\":[\\\"weight\\\",\\\"podAffinityTerm\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"requiredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.\\\",\\\"items\\\":{\\\"description\\\":\\\"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\\\u003ctopologyKey\\\\u003e matches that of any node on which a pod of the set of pods is running\\\",\\\"properties\\\":{\\\"labelSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"namespaces\\\":{\\\"description\\\":\\\"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\\\\\"this pod's namespace\\\\\\\"\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"topologyKey\\\":{\\\"description\\\":\\\"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"topologyKey\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"podAntiAffinity\\\":{\\\"description\\\":\\\"Pod anti affinity is a group of inter pod anti affinity scheduling rules.\\\",\\\"properties\\\":{\\\"preferredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"The scheduler will prefer to schedule pods to nodes that satisfy the anti-affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling anti-affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\\\\\"weight\\\\\\\" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.\\\",\\\"items\\\":{\\\"description\\\":\\\"The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)\\\",\\\"properties\\\":{\\\"podAffinityTerm\\\":{\\\"description\\\":\\\"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\\\u003ctopologyKey\\\\u003e matches that of any node on which a pod of the set of pods is running\\\",\\\"properties\\\":{\\\"labelSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"namespaces\\\":{\\\"description\\\":\\\"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\\\\\"this pod's namespace\\\\\\\"\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"topologyKey\\\":{\\\"description\\\":\\\"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"topologyKey\\\"],\\\"type\\\":\\\"object\\\"},\\\"weight\\\":{\\\"description\\\":\\\"weight associated with matching the corresponding podAffinityTerm, in the range 1-100.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"required\\\":[\\\"weight\\\",\\\"podAffinityTerm\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"requiredDuringSchedulingIgnoredDuringExecution\\\":{\\\"description\\\":\\\"If the anti-affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the anti-affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.\\\",\\\"items\\\":{\\\"description\\\":\\\"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key \\\\u003ctopologyKey\\\\u003e matches that of any node on which a pod of the set of pods is running\\\",\\\"properties\\\":{\\\"labelSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"namespaces\\\":{\\\"description\\\":\\\"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\\\\\"this pod's namespace\\\\\\\"\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"topologyKey\\\":{\\\"description\\\":\\\"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"topologyKey\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"alerting\\\":{\\\"description\\\":\\\"AlertingSpec defines parameters for alerting configuration of Prometheus servers.\\\",\\\"properties\\\":{\\\"alertmanagers\\\":{\\\"description\\\":\\\"AlertmanagerEndpoints Prometheus should fire alerts against.\\\",\\\"items\\\":{\\\"description\\\":\\\"AlertmanagerEndpoints defines a selection of a single Endpoints object containing alertmanager IPs to fire alerts against.\\\",\\\"properties\\\":{\\\"bearerTokenFile\\\":{\\\"description\\\":\\\"BearerTokenFile to read from filesystem to use when authenticating to Alertmanager.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of Endpoints object in Namespace.\\\",\\\"type\\\":\\\"string\\\"},\\\"namespace\\\":{\\\"description\\\":\\\"Namespace of Endpoints object.\\\",\\\"type\\\":\\\"string\\\"},\\\"pathPrefix\\\":{\\\"description\\\":\\\"Prefix for the HTTP path alerts are pushed to.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use when firing alerts.\\\",\\\"type\\\":\\\"string\\\"},\\\"tlsConfig\\\":{\\\"description\\\":\\\"TLSConfig specifies TLS configuration parameters.\\\",\\\"properties\\\":{\\\"caFile\\\":{\\\"description\\\":\\\"The CA cert to use for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"certFile\\\":{\\\"description\\\":\\\"The client cert file for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"insecureSkipVerify\\\":{\\\"description\\\":\\\"Disable target certificate validation.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"keyFile\\\":{\\\"description\\\":\\\"The client key file for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"serverName\\\":{\\\"description\\\":\\\"Used to verify the hostname for the targets.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"required\\\":[\\\"namespace\\\",\\\"name\\\",\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"alertmanagers\\\"],\\\"type\\\":\\\"object\\\"},\\\"apiserverConfig\\\":{\\\"description\\\":\\\"APIServerConfig defines a host and auth methods to access apiserver. More info: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#kubernetes_sd_config\\\",\\\"properties\\\":{\\\"basicAuth\\\":{\\\"description\\\":\\\"BasicAuth allow an endpoint to authenticate over basic authentication More info: https://prometheus.io/docs/operating/configuration/#endpoints\\\",\\\"properties\\\":{\\\"password\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"username\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"bearerToken\\\":{\\\"description\\\":\\\"Bearer token for accessing apiserver.\\\",\\\"type\\\":\\\"string\\\"},\\\"bearerTokenFile\\\":{\\\"description\\\":\\\"File to read bearer token for accessing apiserver.\\\",\\\"type\\\":\\\"string\\\"},\\\"host\\\":{\\\"description\\\":\\\"Host of apiserver. A valid string consisting of a hostname or IP followed by an optional port number\\\",\\\"type\\\":\\\"string\\\"},\\\"tlsConfig\\\":{\\\"description\\\":\\\"TLSConfig specifies TLS configuration parameters.\\\",\\\"properties\\\":{\\\"caFile\\\":{\\\"description\\\":\\\"The CA cert to use for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"certFile\\\":{\\\"description\\\":\\\"The client cert file for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"insecureSkipVerify\\\":{\\\"description\\\":\\\"Disable target certificate validation.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"keyFile\\\":{\\\"description\\\":\\\"The client key file for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"serverName\\\":{\\\"description\\\":\\\"Used to verify the hostname for the targets.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"required\\\":[\\\"host\\\"],\\\"type\\\":\\\"object\\\"},\\\"baseImage\\\":{\\\"description\\\":\\\"Base image to use for a Prometheus deployment.\\\",\\\"type\\\":\\\"string\\\"},\\\"configMaps\\\":{\\\"description\\\":\\\"ConfigMaps is a list of ConfigMaps in the same namespace as the Prometheus object, which shall be mounted into the Prometheus Pods. The ConfigMaps are mounted into /etc/prometheus/configmaps/\\\\u003cconfigmap-name\\\\u003e.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"containers\\\":{\\\"description\\\":\\\"Containers allows injecting additional containers or modifying operator generated containers. This can be used to allow adding an authentication proxy to a Prometheus pod or to change the behavior of an operator generated container. Containers described here modify an operator generated container if they share the same name and modifications are done via a strategic merge patch. The current container names are: `prometheus`, `prometheus-config-reloader`, `rules-configmap-reloader`, and `thanos-sidecar`. Overriding containers is entirely outside the scope of what the maintainers will support and by doing so, you accept that this behaviour may break at any time without notice.\\\",\\\"items\\\":{\\\"description\\\":\\\"A single application container that you want to run within a pod.\\\",\\\"properties\\\":{\\\"args\\\":{\\\"description\\\":\\\"Arguments to the entrypoint. The docker image's CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"command\\\":{\\\"description\\\":\\\"Entrypoint array. Not executed within a shell. The docker image's ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"env\\\":{\\\"description\\\":\\\"List of environment variables to set in the container. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"EnvVar represents an environment variable present in a Container.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the environment variable. Must be a C_IDENTIFIER.\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to \\\\\\\"\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"},\\\"valueFrom\\\":{\\\"description\\\":\\\"EnvVarSource represents a source for the value of an EnvVar.\\\",\\\"properties\\\":{\\\"configMapKeyRef\\\":{\\\"description\\\":\\\"Selects a key from a ConfigMap.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key to select.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"fieldRef\\\":{\\\"description\\\":\\\"ObjectFieldSelector selects an APIVersioned field of an object.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"Version of the schema the FieldPath is written in terms of, defaults to \\\\\\\"v1\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"},\\\"fieldPath\\\":{\\\"description\\\":\\\"Path of the field to select in the specified API version.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"fieldPath\\\"],\\\"type\\\":\\\"object\\\"},\\\"resourceFieldRef\\\":{\\\"description\\\":\\\"ResourceFieldSelector represents container resources (cpu, memory) and their output format\\\",\\\"properties\\\":{\\\"containerName\\\":{\\\"description\\\":\\\"Container name: required for volumes, optional for env vars\\\",\\\"type\\\":\\\"string\\\"},\\\"divisor\\\":{},\\\"resource\\\":{\\\"description\\\":\\\"Required: resource to select\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"resource\\\"],\\\"type\\\":\\\"object\\\"},\\\"secretKeyRef\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"required\\\":[\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"envFrom\\\":{\\\"description\\\":\\\"List of sources to populate environment variables in the container. The keys defined within a source must be a C_IDENTIFIER. All invalid keys will be reported as an event when the container is starting. When a key exists in multiple sources, the value associated with the last source will take precedence. Values defined by an Env with a duplicate key will take precedence. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"EnvFromSource represents the source of a set of ConfigMaps\\\",\\\"properties\\\":{\\\"configMapRef\\\":{\\\"description\\\":\\\"ConfigMapEnvSource selects a ConfigMap to populate the environment variables with.\\\\n\\\\nThe contents of the target ConfigMap's Data field will represent the key-value pairs as environment variables.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the ConfigMap must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"type\\\":\\\"object\\\"},\\\"prefix\\\":{\\\"description\\\":\\\"An optional identifier to prepend to each key in the ConfigMap. Must be a C_IDENTIFIER.\\\",\\\"type\\\":\\\"string\\\"},\\\"secretRef\\\":{\\\"description\\\":\\\"SecretEnvSource selects a Secret to populate the environment variables with.\\\\n\\\\nThe contents of the target Secret's Data field will represent the key-value pairs as environment variables.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"image\\\":{\\\"description\\\":\\\"Docker image name. More info: https://kubernetes.io/docs/concepts/containers/images This field is optional to allow higher level config management to default or override container images in workload controllers like Deployments and StatefulSets.\\\",\\\"type\\\":\\\"string\\\"},\\\"imagePullPolicy\\\":{\\\"description\\\":\\\"Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images\\\",\\\"type\\\":\\\"string\\\"},\\\"lifecycle\\\":{\\\"description\\\":\\\"Lifecycle describes actions that the management system should take in response to container lifecycle events. For the PostStart and PreStop lifecycle handlers, management of the container blocks until the action is complete, unless the container process fails, in which case the handler is aborted.\\\",\\\"properties\\\":{\\\"postStart\\\":{\\\"description\\\":\\\"Handler defines a specific action that should be taken\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"ExecAction describes a \\\\\\\"run in container\\\\\\\" action.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGetAction describes an action based on HTTP Get requests.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocketAction describes an action based on opening a socket\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"preStop\\\":{\\\"description\\\":\\\"Handler defines a specific action that should be taken\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"ExecAction describes a \\\\\\\"run in container\\\\\\\" action.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGetAction describes an action based on HTTP Get requests.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocketAction describes an action based on opening a socket\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"livenessProbe\\\":{\\\"description\\\":\\\"Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic.\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"ExecAction describes a \\\\\\\"run in container\\\\\\\" action.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"failureThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGetAction describes an action based on HTTP Get requests.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"initialDelaySeconds\\\":{\\\"description\\\":\\\"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"periodSeconds\\\":{\\\"description\\\":\\\"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"successThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocketAction describes an action based on opening a socket\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"timeoutSeconds\\\":{\\\"description\\\":\\\"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the container specified as a DNS_LABEL. Each container in a pod must have a unique name (DNS_LABEL). Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"},\\\"ports\\\":{\\\"description\\\":\\\"List of ports to expose from the container. Exposing a port here gives the system additional information about the network connections a container uses, but is primarily informational. Not specifying a port here DOES NOT prevent that port from being exposed. Any port which is listening on the default \\\\\\\"0.0.0.0\\\\\\\" address inside a container will be accessible from the network. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"ContainerPort represents a network port in a single container.\\\",\\\"properties\\\":{\\\"containerPort\\\":{\\\"description\\\":\\\"Number of port to expose on the pod's IP address. This must be a valid port number, 0 \\\\u003c x \\\\u003c 65536.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"hostIP\\\":{\\\"description\\\":\\\"What host IP to bind the external port to.\\\",\\\"type\\\":\\\"string\\\"},\\\"hostPort\\\":{\\\"description\\\":\\\"Number of port to expose on the host. If specified, this must be a valid port number, 0 \\\\u003c x \\\\u003c 65536. If HostNetwork is specified, this must match ContainerPort. Most containers do not need this.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"name\\\":{\\\"description\\\":\\\"If specified, this must be an IANA_SVC_NAME and unique within the pod. Each named port in a pod must have a unique name. Name for the port that can be referred to by services.\\\",\\\"type\\\":\\\"string\\\"},\\\"protocol\\\":{\\\"description\\\":\\\"Protocol for port. Must be UDP, TCP, or SCTP. Defaults to \\\\\\\"TCP\\\\\\\".\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"containerPort\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"readinessProbe\\\":{\\\"description\\\":\\\"Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic.\\\",\\\"properties\\\":{\\\"exec\\\":{\\\"description\\\":\\\"ExecAction describes a \\\\\\\"run in container\\\\\\\" action.\\\",\\\"properties\\\":{\\\"command\\\":{\\\"description\\\":\\\"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"failureThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"httpGet\\\":{\\\"description\\\":\\\"HTTPGetAction describes an action based on HTTP Get requests.\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Host name to connect to, defaults to the pod IP. You probably want to set \\\\\\\"Host\\\\\\\" in httpHeaders instead.\\\",\\\"type\\\":\\\"string\\\"},\\\"httpHeaders\\\":{\\\"description\\\":\\\"Custom headers to set in the request. HTTP allows repeated headers.\\\",\\\"items\\\":{\\\"description\\\":\\\"HTTPHeader describes a custom header to be used in HTTP probes\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"The header field name\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"The header field value\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"path\\\":{\\\"description\\\":\\\"Path to access on the HTTP server.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]},\\\"scheme\\\":{\\\"description\\\":\\\"Scheme to use for connecting to the host. Defaults to HTTP.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"initialDelaySeconds\\\":{\\\"description\\\":\\\"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"periodSeconds\\\":{\\\"description\\\":\\\"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"successThreshold\\\":{\\\"description\\\":\\\"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"tcpSocket\\\":{\\\"description\\\":\\\"TCPSocketAction describes an action based on opening a socket\\\",\\\"properties\\\":{\\\"host\\\":{\\\"description\\\":\\\"Optional: Host name to connect to, defaults to the pod IP.\\\",\\\"type\\\":\\\"string\\\"},\\\"port\\\":{\\\"anyOf\\\":[{\\\"type\\\":\\\"string\\\"},{\\\"type\\\":\\\"integer\\\"}]}},\\\"required\\\":[\\\"port\\\"],\\\"type\\\":\\\"object\\\"},\\\"timeoutSeconds\\\":{\\\"description\\\":\\\"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"},\\\"resources\\\":{\\\"description\\\":\\\"ResourceRequirements describes the compute resource requirements.\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"securityContext\\\":{\\\"description\\\":\\\"SecurityContext holds security configuration that will be applied to a container. Some fields are present in both SecurityContext and PodSecurityContext. When both are set, the values in SecurityContext take precedence.\\\",\\\"properties\\\":{\\\"allowPrivilegeEscalation\\\":{\\\"description\\\":\\\"AllowPrivilegeEscalation controls whether a process can gain more privileges than its parent process. This bool directly controls if the no_new_privs flag will be set on the container process. AllowPrivilegeEscalation is true always when the container is: 1) run as Privileged 2) has CAP_SYS_ADMIN\\\",\\\"type\\\":\\\"boolean\\\"},\\\"capabilities\\\":{\\\"description\\\":\\\"Adds and removes POSIX capabilities from running containers.\\\",\\\"properties\\\":{\\\"add\\\":{\\\"description\\\":\\\"Added capabilities\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"drop\\\":{\\\"description\\\":\\\"Removed capabilities\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"privileged\\\":{\\\"description\\\":\\\"Run container in privileged mode. Processes in privileged containers are essentially equivalent to root on the host. Defaults to false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"procMount\\\":{\\\"description\\\":\\\"procMount denotes the type of proc mount to use for the containers. The default is DefaultProcMount which uses the container runtime defaults for readonly paths and masked paths. This requires the ProcMountType feature flag to be enabled.\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnlyRootFilesystem\\\":{\\\"description\\\":\\\"Whether this container has a read-only root filesystem. Default is false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"runAsGroup\\\":{\\\"description\\\":\\\"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"runAsNonRoot\\\":{\\\"description\\\":\\\"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"runAsUser\\\":{\\\"description\\\":\\\"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"seLinuxOptions\\\":{\\\"description\\\":\\\"SELinuxOptions are the labels to be applied to the container\\\",\\\"properties\\\":{\\\"level\\\":{\\\"description\\\":\\\"Level is SELinux level label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"role\\\":{\\\"description\\\":\\\"Role is a SELinux role label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"type\\\":{\\\"description\\\":\\\"Type is a SELinux type label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"user\\\":{\\\"description\\\":\\\"User is a SELinux user label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"stdin\\\":{\\\"description\\\":\\\"Whether this container should allocate a buffer for stdin in the container runtime. If this is not set, reads from stdin in the container will always result in EOF. Default is false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"stdinOnce\\\":{\\\"description\\\":\\\"Whether the container runtime should close the stdin channel after it has been opened by a single attach. When stdin is true the stdin stream will remain open across multiple attach sessions. If stdinOnce is set to true, stdin is opened on container start, is empty until the first client attaches to stdin, and then remains open and accepts data until the client disconnects, at which time stdin is closed and remains closed until the container is restarted. If this flag is false, a container processes that reads from stdin will never receive an EOF. Default is false\\\",\\\"type\\\":\\\"boolean\\\"},\\\"terminationMessagePath\\\":{\\\"description\\\":\\\"Optional: Path at which the file to which the container's termination message will be written is mounted into the container's filesystem. Message written is intended to be brief final status, such as an assertion failure message. Will be truncated by the node if greater than 4096 bytes. The total message length across all containers will be limited to 12kb. Defaults to /dev/termination-log. Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"},\\\"terminationMessagePolicy\\\":{\\\"description\\\":\\\"Indicate how the termination message should be populated. File will use the contents of terminationMessagePath to populate the container status message on both success and failure. FallbackToLogsOnError will use the last chunk of container log output if the termination message file is empty and the container exited with an error. The log output is limited to 2048 bytes or 80 lines, whichever is smaller. Defaults to File. Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"},\\\"tty\\\":{\\\"description\\\":\\\"Whether this container should allocate a TTY for itself, also requires 'stdin' to be true. Default is false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"volumeDevices\\\":{\\\"description\\\":\\\"volumeDevices is the list of block devices to be used by the container. This is a beta feature.\\\",\\\"items\\\":{\\\"description\\\":\\\"volumeDevice describes a mapping of a raw block device within a container.\\\",\\\"properties\\\":{\\\"devicePath\\\":{\\\"description\\\":\\\"devicePath is the path inside of the container that the device will be mapped to.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"name must match the name of a persistentVolumeClaim in the pod\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"devicePath\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"volumeMounts\\\":{\\\"description\\\":\\\"Pod volumes to mount into the container's filesystem. Cannot be updated.\\\",\\\"items\\\":{\\\"description\\\":\\\"VolumeMount describes a mounting of a Volume within a container.\\\",\\\"properties\\\":{\\\"mountPath\\\":{\\\"description\\\":\\\"Path within the container at which the volume should be mounted. Must not contain ':'.\\\",\\\"type\\\":\\\"string\\\"},\\\"mountPropagation\\\":{\\\"description\\\":\\\"mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"This must match the Name of a Volume.\\\",\\\"type\\\":\\\"string\\\"},\\\"readOnly\\\":{\\\"description\\\":\\\"Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"subPath\\\":{\\\"description\\\":\\\"Path within the volume from which the container's volume should be mounted. Defaults to \\\\\\\"\\\\\\\" (volume's root).\\\",\\\"type\\\":\\\"string\\\"},\\\"subPathExpr\\\":{\\\"description\\\":\\\"Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to \\\\\\\"\\\\\\\" (volume's root). SubPathExpr and SubPath are mutually exclusive. This field is alpha in 1.14.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"mountPath\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"workingDir\\\":{\\\"description\\\":\\\"Container's working directory. If not specified, the container runtime's default will be used, which might be configured in the container image. Cannot be updated.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"enableAdminAPI\\\":{\\\"description\\\":\\\"Enable access to prometheus web admin API. Defaults to the value of `false`. WARNING: Enabling the admin APIs enables mutating endpoints, to delete data, shutdown Prometheus, and more. Enabling this should be done with care and the user is advised to add additional authentication authorization via a proxy to ensure only clients authorized to perform these actions can do so. For more information see https://prometheus.io/docs/prometheus/latest/querying/api/#tsdb-admin-apis\\\",\\\"type\\\":\\\"boolean\\\"},\\\"evaluationInterval\\\":{\\\"description\\\":\\\"Interval between consecutive evaluations.\\\",\\\"type\\\":\\\"string\\\"},\\\"externalLabels\\\":{\\\"description\\\":\\\"The labels to add to any time series or alerts when communicating with external systems (federation, remote storage, Alertmanager).\\\",\\\"type\\\":\\\"object\\\"},\\\"externalUrl\\\":{\\\"description\\\":\\\"The external URL the Prometheus instances will be available under. This is necessary to generate correct URLs. This is necessary if Prometheus is not served from root of a DNS name.\\\",\\\"type\\\":\\\"string\\\"},\\\"image\\\":{\\\"description\\\":\\\"Image if specified has precedence over baseImage, tag and sha combinations. Specifying the version is still necessary to ensure the Prometheus Operator knows what version of Prometheus is being configured.\\\",\\\"type\\\":\\\"string\\\"},\\\"imagePullSecrets\\\":{\\\"description\\\":\\\"An optional list of references to secrets in the same namespace to use for pulling prometheus and alertmanager images from registries see http://kubernetes.io/docs/user-guide/images#specifying-imagepullsecrets-on-a-pod\\\",\\\"items\\\":{\\\"description\\\":\\\"LocalObjectReference contains enough information to let you locate the referenced object inside the same namespace.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"listenLocal\\\":{\\\"description\\\":\\\"ListenLocal makes the Prometheus server listen on loopback, so that it does not bind against the Pod IP.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"logFormat\\\":{\\\"description\\\":\\\"Log format for Prometheus to be configured with.\\\",\\\"type\\\":\\\"string\\\"},\\\"logLevel\\\":{\\\"description\\\":\\\"Log level for Prometheus to be configured with.\\\",\\\"type\\\":\\\"string\\\"},\\\"nodeSelector\\\":{\\\"description\\\":\\\"Define which Nodes the Pods are scheduled on.\\\",\\\"type\\\":\\\"object\\\"},\\\"paused\\\":{\\\"description\\\":\\\"When a Prometheus deployment is paused, no actions except for deletion will be performed on the underlying objects.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"podMetadata\\\":{\\\"description\\\":\\\"ObjectMeta is metadata that all persisted resources must have, which includes all objects users must create.\\\",\\\"properties\\\":{\\\"annotations\\\":{\\\"description\\\":\\\"Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations\\\",\\\"type\\\":\\\"object\\\"},\\\"clusterName\\\":{\\\"description\\\":\\\"The name of the cluster which the object belongs to. This is used to distinguish resources with same name and namespace in different clusters. This field is not set anywhere right now and apiserver is going to ignore it if set in create or update request.\\\",\\\"type\\\":\\\"string\\\"},\\\"creationTimestamp\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"deletionGracePeriodSeconds\\\":{\\\"description\\\":\\\"Number of seconds allowed for this object to gracefully terminate before it will be removed from the system. Only set when deletionTimestamp is also set. May only be shortened. Read-only.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"deletionTimestamp\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"finalizers\\\":{\\\"description\\\":\\\"Must be empty before the object is deleted from the registry. Each entry is an identifier for the responsible component that will remove the entry from the list. If the deletionTimestamp of the object is non-nil, entries in this list can only be removed.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"generateName\\\":{\\\"description\\\":\\\"GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server.\\\\n\\\\nIf this field is specified and the generated name exists, the server will NOT return a 409 - instead, it will either return 201 Created or 500 with Reason ServerTimeout indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header).\\\\n\\\\nApplied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#idempotency\\\",\\\"type\\\":\\\"string\\\"},\\\"generation\\\":{\\\"description\\\":\\\"A sequence number representing a specific generation of the desired state. Populated by the system. Read-only.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"initializers\\\":{\\\"description\\\":\\\"Initializers tracks the progress of initialization.\\\",\\\"properties\\\":{\\\"pending\\\":{\\\"description\\\":\\\"Pending is a list of initializers that must execute in order before this object is visible. When the last pending initializer is removed, and no failing result is set, the initializers struct will be set to nil and the object is considered as initialized and visible to all clients.\\\",\\\"items\\\":{\\\"description\\\":\\\"Initializer is information about an initializer that has not yet completed.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"name of the process that is responsible for initializing this object.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"result\\\":{\\\"description\\\":\\\"Status is a return value for calls that don't return other objects.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"code\\\":{\\\"description\\\":\\\"Suggested HTTP return code for this status, 0 if not set.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"details\\\":{\\\"description\\\":\\\"StatusDetails is a set of additional properties that MAY be set by the server to provide additional information about a response. The Reason field of a Status object defines what attributes will be set. Clients must ignore fields that do not match the defined type of each attribute, and should assume that any attribute may be empty, invalid, or under defined.\\\",\\\"properties\\\":{\\\"causes\\\":{\\\"description\\\":\\\"The Causes array includes more details associated with the StatusReason failure. Not all StatusReasons may provide detailed causes.\\\",\\\"items\\\":{\\\"description\\\":\\\"StatusCause provides more information about an api.Status failure, including cases when multiple errors are encountered.\\\",\\\"properties\\\":{\\\"field\\\":{\\\"description\\\":\\\"The field of the resource that has caused this error, as named by its JSON serialization. May include dot and postfix notation for nested attributes. Arrays are zero-indexed. Fields may appear more than once in an array of causes due to fields having multiple errors. Optional.\\\\n\\\\nExamples:\\\\n \\\\\\\"name\\\\\\\" - the field \\\\\\\"name\\\\\\\" on the current resource\\\\n \\\\\\\"items[0].name\\\\\\\" - the field \\\\\\\"name\\\\\\\" on the first array entry in \\\\\\\"items\\\\\\\"\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"A human-readable description of the cause of the error. This field may be presented as-is to a reader.\\\",\\\"type\\\":\\\"string\\\"},\\\"reason\\\":{\\\"description\\\":\\\"A machine-readable description of the cause of the error. If this value is empty there is no information available.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"group\\\":{\\\"description\\\":\\\"The group attribute of the resource associated with the status StatusReason.\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"The kind attribute of the resource associated with the status StatusReason. On some operations may differ from the requested resource Kind. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"The name attribute of the resource associated with the status StatusReason (when there is a single name which can be described).\\\",\\\"type\\\":\\\"string\\\"},\\\"retryAfterSeconds\\\":{\\\"description\\\":\\\"If specified, the time in seconds before the operation should be retried. Some errors may indicate the client must take an alternate action - for those errors this field may indicate how long to wait before taking the alternate action.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID of the resource. (when there is a single resource which can be described). More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"A human-readable description of the status of this operation.\\\",\\\"type\\\":\\\"string\\\"},\\\"metadata\\\":{\\\"description\\\":\\\"ListMeta describes metadata that synthetic resources must have, including lists and various status objects. A resource may have only one of {ObjectMeta, ListMeta}.\\\",\\\"properties\\\":{\\\"continue\\\":{\\\"description\\\":\\\"continue may be set if the user set a limit on the number of items returned, and indicates that the server has more data available. The value is opaque and may be used to issue another request to the endpoint that served this list to retrieve the next set of available objects. Continuing a consistent list may not be possible if the server configuration has changed or more than a few minutes have passed. The resourceVersion field returned when using this continue value will be identical to the value in the first response, unless you have received this token from an error message.\\\",\\\"type\\\":\\\"string\\\"},\\\"resourceVersion\\\":{\\\"description\\\":\\\"String that identifies the server's internal version of this object that can be used by clients to determine when objects have changed. Value must be treated as opaque by clients and passed unmodified back to the server. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\\\",\\\"type\\\":\\\"string\\\"},\\\"selfLink\\\":{\\\"description\\\":\\\"selfLink is a URL representing this object. Populated by the system. Read-only.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"reason\\\":{\\\"description\\\":\\\"A machine-readable description of why this operation is in the \\\\\\\"Failure\\\\\\\" status. If this value is empty there is no information available. A Reason clarifies an HTTP status code but does not override it.\\\",\\\"type\\\":\\\"string\\\"},\\\"status\\\":{\\\"description\\\":\\\"Status of the operation. One of: \\\\\\\"Success\\\\\\\" or \\\\\\\"Failure\\\\\\\". More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"required\\\":[\\\"pending\\\"],\\\"type\\\":\\\"object\\\"},\\\"labels\\\":{\\\"description\\\":\\\"Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels\\\",\\\"type\\\":\\\"object\\\"},\\\"managedFields\\\":{\\\"description\\\":\\\"ManagedFields maps workflow-id and version to the set of fields that are managed by that workflow. This is mostly for internal housekeeping, and users typically shouldn't need to set or understand this field. A workflow can be the user's name, a controller's name, or the name of a specific apply path like \\\\\\\"ci-cd\\\\\\\". The set of fields is always in the version that the workflow used when modifying the object.\\\\n\\\\nThis field is alpha and can be changed or removed without notice.\\\",\\\"items\\\":{\\\"description\\\":\\\"ManagedFieldsEntry is a workflow-id, a FieldSet and the group version of the resource that the fieldset applies to.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the version of this resource that this field set applies to. The format is \\\\\\\"group/version\\\\\\\" just like the top-level APIVersion field. It is necessary to track the version of a field set because it cannot be automatically converted.\\\",\\\"type\\\":\\\"string\\\"},\\\"fields\\\":{\\\"description\\\":\\\"Fields stores a set of fields in a data structure like a Trie. To understand how this is used, see: https://github.com/kubernetes-sigs/structured-merge-diff\\\",\\\"type\\\":\\\"object\\\"},\\\"manager\\\":{\\\"description\\\":\\\"Manager is an identifier of the workflow managing these fields.\\\",\\\"type\\\":\\\"string\\\"},\\\"operation\\\":{\\\"description\\\":\\\"Operation is the type of operation which lead to this ManagedFieldsEntry being created. The only valid values for this field are 'Apply' and 'Update'.\\\",\\\"type\\\":\\\"string\\\"},\\\"time\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names\\\",\\\"type\\\":\\\"string\\\"},\\\"namespace\\\":{\\\"description\\\":\\\"Namespace defines the space within each name must be unique. An empty namespace is equivalent to the \\\\\\\"default\\\\\\\" namespace, but \\\\\\\"default\\\\\\\" is the canonical representation. Not all objects are required to be scoped to a namespace - the value of this field for those objects will be empty.\\\\n\\\\nMust be a DNS_LABEL. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/namespaces\\\",\\\"type\\\":\\\"string\\\"},\\\"ownerReferences\\\":{\\\"description\\\":\\\"List of objects depended by this object. If ALL objects in the list have been deleted, this object will be garbage collected. If this object is managed by a controller, then an entry in this list will point to this controller, with the controller field set to true. There cannot be more than one managing controller.\\\",\\\"items\\\":{\\\"description\\\":\\\"OwnerReference contains enough information to let you identify an owning object. An owning object must be in the same namespace as the dependent, or be cluster-scoped, so there is no namespace field.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"API version of the referent.\\\",\\\"type\\\":\\\"string\\\"},\\\"blockOwnerDeletion\\\":{\\\"description\\\":\\\"If true, AND if the owner has the \\\\\\\"foregroundDeletion\\\\\\\" finalizer, then the owner cannot be deleted from the key-value store until this reference is removed. Defaults to false. To set this field, a user needs \\\\\\\"delete\\\\\\\" permission of the owner, otherwise 422 (Unprocessable Entity) will be returned.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"controller\\\":{\\\"description\\\":\\\"If true, this reference points to the managing controller.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#names\\\",\\\"type\\\":\\\"string\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"apiVersion\\\",\\\"kind\\\",\\\"name\\\",\\\"uid\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"resourceVersion\\\":{\\\"description\\\":\\\"An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources.\\\\n\\\\nPopulated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\\\",\\\"type\\\":\\\"string\\\"},\\\"selfLink\\\":{\\\"description\\\":\\\"SelfLink is a URL representing this object. Populated by the system. Read-only.\\\",\\\"type\\\":\\\"string\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID is the unique in time and space value for this object. It is typically generated by the server on successful creation of a resource and is not allowed to change on PUT operations.\\\\n\\\\nPopulated by the system. Read-only. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"podMonitorNamespaceSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"podMonitorSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"priorityClassName\\\":{\\\"description\\\":\\\"Priority class assigned to the Pods\\\",\\\"type\\\":\\\"string\\\"},\\\"prometheusExternalLabelName\\\":{\\\"description\\\":\\\"Name of Prometheus external label used to denote Prometheus instance name. Defaults to the value of `prometheus`. External label will _not_ be added when value is set to empty string (`\\\\\\\"\\\\\\\"`).\\\",\\\"type\\\":\\\"string\\\"},\\\"query\\\":{\\\"description\\\":\\\"QuerySpec defines the query command line flags when starting Prometheus.\\\",\\\"properties\\\":{\\\"lookbackDelta\\\":{\\\"description\\\":\\\"The delta difference allowed for retrieving metrics during expression evaluations.\\\",\\\"type\\\":\\\"string\\\"},\\\"maxConcurrency\\\":{\\\"description\\\":\\\"Number of concurrent queries that can be run at once.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"maxSamples\\\":{\\\"description\\\":\\\"Maximum number of samples a single query can load into memory. Note that queries will fail if they would load more samples than this into memory, so this also limits the number of samples a query can return.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"timeout\\\":{\\\"description\\\":\\\"Maximum time a query may take before being aborted.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"remoteRead\\\":{\\\"description\\\":\\\"If specified, the remote_read spec. This is an experimental feature, it may change in any upcoming release in a breaking way.\\\",\\\"items\\\":{\\\"description\\\":\\\"RemoteReadSpec defines the remote_read configuration for prometheus.\\\",\\\"properties\\\":{\\\"basicAuth\\\":{\\\"description\\\":\\\"BasicAuth allow an endpoint to authenticate over basic authentication More info: https://prometheus.io/docs/operating/configuration/#endpoints\\\",\\\"properties\\\":{\\\"password\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"username\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"bearerToken\\\":{\\\"description\\\":\\\"bearer token for remote read.\\\",\\\"type\\\":\\\"string\\\"},\\\"bearerTokenFile\\\":{\\\"description\\\":\\\"File to read bearer token for remote read.\\\",\\\"type\\\":\\\"string\\\"},\\\"proxyUrl\\\":{\\\"description\\\":\\\"Optional ProxyURL\\\",\\\"type\\\":\\\"string\\\"},\\\"readRecent\\\":{\\\"description\\\":\\\"Whether reads should be made for queries for time ranges that the local storage should have complete data for.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"remoteTimeout\\\":{\\\"description\\\":\\\"Timeout for requests to the remote read endpoint.\\\",\\\"type\\\":\\\"string\\\"},\\\"requiredMatchers\\\":{\\\"description\\\":\\\"An optional list of equality matchers which have to be present in a selector to query the remote read endpoint.\\\",\\\"type\\\":\\\"object\\\"},\\\"tlsConfig\\\":{\\\"description\\\":\\\"TLSConfig specifies TLS configuration parameters.\\\",\\\"properties\\\":{\\\"caFile\\\":{\\\"description\\\":\\\"The CA cert to use for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"certFile\\\":{\\\"description\\\":\\\"The client cert file for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"insecureSkipVerify\\\":{\\\"description\\\":\\\"Disable target certificate validation.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"keyFile\\\":{\\\"description\\\":\\\"The client key file for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"serverName\\\":{\\\"description\\\":\\\"Used to verify the hostname for the targets.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"url\\\":{\\\"description\\\":\\\"The URL of the endpoint to send samples to.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"url\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"remoteWrite\\\":{\\\"description\\\":\\\"If specified, the remote_write spec. This is an experimental feature, it may change in any upcoming release in a breaking way.\\\",\\\"items\\\":{\\\"description\\\":\\\"RemoteWriteSpec defines the remote_write configuration for prometheus.\\\",\\\"properties\\\":{\\\"basicAuth\\\":{\\\"description\\\":\\\"BasicAuth allow an endpoint to authenticate over basic authentication More info: https://prometheus.io/docs/operating/configuration/#endpoints\\\",\\\"properties\\\":{\\\"password\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"username\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"bearerToken\\\":{\\\"description\\\":\\\"File to read bearer token for remote write.\\\",\\\"type\\\":\\\"string\\\"},\\\"bearerTokenFile\\\":{\\\"description\\\":\\\"File to read bearer token for remote write.\\\",\\\"type\\\":\\\"string\\\"},\\\"proxyUrl\\\":{\\\"description\\\":\\\"Optional ProxyURL\\\",\\\"type\\\":\\\"string\\\"},\\\"queueConfig\\\":{\\\"description\\\":\\\"QueueConfig allows the tuning of remote_write queue_config parameters. This object is referenced in the RemoteWriteSpec object.\\\",\\\"properties\\\":{\\\"batchSendDeadline\\\":{\\\"description\\\":\\\"BatchSendDeadline is the maximum time a sample will wait in buffer.\\\",\\\"type\\\":\\\"string\\\"},\\\"capacity\\\":{\\\"description\\\":\\\"Capacity is the number of samples to buffer per shard before we start dropping them.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"maxBackoff\\\":{\\\"description\\\":\\\"MaxBackoff is the maximum retry delay.\\\",\\\"type\\\":\\\"string\\\"},\\\"maxRetries\\\":{\\\"description\\\":\\\"MaxRetries is the maximum number of times to retry a batch on recoverable errors.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"maxSamplesPerSend\\\":{\\\"description\\\":\\\"MaxSamplesPerSend is the maximum number of samples per send.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"maxShards\\\":{\\\"description\\\":\\\"MaxShards is the maximum number of shards, i.e. amount of concurrency.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"minBackoff\\\":{\\\"description\\\":\\\"MinBackoff is the initial retry delay. Gets doubled for every retry.\\\",\\\"type\\\":\\\"string\\\"},\\\"minShards\\\":{\\\"description\\\":\\\"MinShards is the minimum number of shards, i.e. amount of concurrency.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"type\\\":\\\"object\\\"},\\\"remoteTimeout\\\":{\\\"description\\\":\\\"Timeout for requests to the remote write endpoint.\\\",\\\"type\\\":\\\"string\\\"},\\\"tlsConfig\\\":{\\\"description\\\":\\\"TLSConfig specifies TLS configuration parameters.\\\",\\\"properties\\\":{\\\"caFile\\\":{\\\"description\\\":\\\"The CA cert to use for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"certFile\\\":{\\\"description\\\":\\\"The client cert file for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"insecureSkipVerify\\\":{\\\"description\\\":\\\"Disable target certificate validation.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"keyFile\\\":{\\\"description\\\":\\\"The client key file for the targets.\\\",\\\"type\\\":\\\"string\\\"},\\\"serverName\\\":{\\\"description\\\":\\\"Used to verify the hostname for the targets.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"url\\\":{\\\"description\\\":\\\"The URL of the endpoint to send samples to.\\\",\\\"type\\\":\\\"string\\\"},\\\"writeRelabelConfigs\\\":{\\\"description\\\":\\\"The list of remote write relabel configurations.\\\",\\\"items\\\":{\\\"description\\\":\\\"RelabelConfig allows dynamic rewriting of the label set, being applied to samples before ingestion. It defines `\\\\u003cmetric_relabel_configs\\\\u003e`-section of Prometheus configuration. More info: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#metric_relabel_configs\\\",\\\"properties\\\":{\\\"action\\\":{\\\"description\\\":\\\"Action to perform based on regex matching. Default is 'replace'\\\",\\\"type\\\":\\\"string\\\"},\\\"modulus\\\":{\\\"description\\\":\\\"Modulus to take of the hash of the source label values.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"regex\\\":{\\\"description\\\":\\\"Regular expression against which the extracted value is matched. default is '(.*)'\\\",\\\"type\\\":\\\"string\\\"},\\\"replacement\\\":{\\\"description\\\":\\\"Replacement value against which a regex replace is performed if the regular expression matches. Regex capture groups are available. Default is '$1'\\\",\\\"type\\\":\\\"string\\\"},\\\"separator\\\":{\\\"description\\\":\\\"Separator placed between concatenated source label values. default is ';'.\\\",\\\"type\\\":\\\"string\\\"},\\\"sourceLabels\\\":{\\\"description\\\":\\\"The source labels select values from existing labels. Their content is concatenated using the configured separator and matched against the configured regular expression for the replace, keep, and drop actions.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"targetLabel\\\":{\\\"description\\\":\\\"Label to which the resulting value is written in a replace action. It is mandatory for replace actions. Regex capture groups are available.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"url\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"replicaExternalLabelName\\\":{\\\"description\\\":\\\"Name of Prometheus external label used to denote replica name. Defaults to the value of `prometheus_replica`. External label will _not_ be added when value is set to empty string (`\\\\\\\"\\\\\\\"`).\\\",\\\"type\\\":\\\"string\\\"},\\\"replicas\\\":{\\\"description\\\":\\\"Number of instances to deploy for a Prometheus deployment.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"resources\\\":{\\\"description\\\":\\\"ResourceRequirements describes the compute resource requirements.\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"retention\\\":{\\\"description\\\":\\\"Time duration Prometheus shall retain data for. Default is '24h', and must match the regular expression `[0-9]+(ms|s|m|h|d|w|y)` (milliseconds seconds minutes hours days weeks years).\\\",\\\"type\\\":\\\"string\\\"},\\\"retentionSize\\\":{\\\"description\\\":\\\"Maximum amount of disk space used by blocks.\\\",\\\"type\\\":\\\"string\\\"},\\\"routePrefix\\\":{\\\"description\\\":\\\"The route prefix Prometheus registers HTTP handlers for. This is useful, if using ExternalURL and a proxy is rewriting HTTP routes of a request, and the actual ExternalURL is still true, but the server serves requests under a different route prefix. For example for use with `kubectl proxy`.\\\",\\\"type\\\":\\\"string\\\"},\\\"ruleNamespaceSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"ruleSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"rules\\\":{\\\"description\\\":\\\"/--rules.*/ command-line arguments\\\",\\\"properties\\\":{\\\"alert\\\":{\\\"description\\\":\\\"/--rules.alert.*/ command-line arguments\\\",\\\"properties\\\":{\\\"forGracePeriod\\\":{\\\"description\\\":\\\"Minimum duration between alert and restored 'for' state. This is maintained only for alerts with configured 'for' time greater than grace period.\\\",\\\"type\\\":\\\"string\\\"},\\\"forOutageTolerance\\\":{\\\"description\\\":\\\"Max time to tolerate prometheus outage for restoring 'for' state of alert.\\\",\\\"type\\\":\\\"string\\\"},\\\"resendDelay\\\":{\\\"description\\\":\\\"Minimum amount of time to wait before resending an alert to Alertmanager.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"scrapeInterval\\\":{\\\"description\\\":\\\"Interval between consecutive scrapes.\\\",\\\"type\\\":\\\"string\\\"},\\\"secrets\\\":{\\\"description\\\":\\\"Secrets is a list of Secrets in the same namespace as the Prometheus object, which shall be mounted into the Prometheus Pods. The Secrets are mounted into /etc/prometheus/secrets/\\\\u003csecret-name\\\\u003e.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"securityContext\\\":{\\\"description\\\":\\\"PodSecurityContext holds pod-level security attributes and common container settings. Some fields are also present in container.securityContext. Field values of container.securityContext take precedence over field values of PodSecurityContext.\\\",\\\"properties\\\":{\\\"fsGroup\\\":{\\\"description\\\":\\\"A special supplemental group that applies to all containers in a pod. Some volume types allow the Kubelet to change the ownership of that volume to be owned by the pod:\\\\n\\\\n1. The owning GID will be the FSGroup 2. The setgid bit is set (new files created in the volume will be owned by FSGroup) 3. The permission bits are OR'd with rw-rw----\\\\n\\\\nIf unset, the Kubelet will not modify the ownership and permissions of any volume.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"runAsGroup\\\":{\\\"description\\\":\\\"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"runAsNonRoot\\\":{\\\"description\\\":\\\"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"runAsUser\\\":{\\\"description\\\":\\\"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"seLinuxOptions\\\":{\\\"description\\\":\\\"SELinuxOptions are the labels to be applied to the container\\\",\\\"properties\\\":{\\\"level\\\":{\\\"description\\\":\\\"Level is SELinux level label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"role\\\":{\\\"description\\\":\\\"Role is a SELinux role label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"type\\\":{\\\"description\\\":\\\"Type is a SELinux type label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"},\\\"user\\\":{\\\"description\\\":\\\"User is a SELinux user label that applies to the container.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"supplementalGroups\\\":{\\\"description\\\":\\\"A list of groups applied to the first process run in each container, in addition to the container's primary GID. If unspecified, no groups will be added to any container.\\\",\\\"items\\\":{\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"type\\\":\\\"array\\\"},\\\"sysctls\\\":{\\\"description\\\":\\\"Sysctls hold a list of namespaced sysctls used for the pod. Pods with unsupported sysctls (by the container runtime) might fail to launch.\\\",\\\"items\\\":{\\\"description\\\":\\\"Sysctl defines a kernel parameter to be set\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"Name of a property to set\\\",\\\"type\\\":\\\"string\\\"},\\\"value\\\":{\\\"description\\\":\\\"Value of a property to set\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\",\\\"value\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"}},\\\"type\\\":\\\"object\\\"},\\\"serviceAccountName\\\":{\\\"description\\\":\\\"ServiceAccountName is the name of the ServiceAccount to use to run the Prometheus Pods.\\\",\\\"type\\\":\\\"string\\\"},\\\"serviceMonitorNamespaceSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"serviceMonitorSelector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"sha\\\":{\\\"description\\\":\\\"SHA of Prometheus container image to be deployed. Defaults to the value of `version`. Similar to a tag, but the SHA explicitly deploys an immutable container image. Version and Tag are ignored if SHA is set.\\\",\\\"type\\\":\\\"string\\\"},\\\"storage\\\":{\\\"description\\\":\\\"StorageSpec defines the configured storage for a group Prometheus servers. If neither `emptyDir` nor `volumeClaimTemplate` is specified, then by default an [EmptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir) will be used.\\\",\\\"properties\\\":{\\\"emptyDir\\\":{\\\"description\\\":\\\"Represents an empty directory for a pod. Empty directory volumes support ownership management and SELinux relabeling.\\\",\\\"properties\\\":{\\\"medium\\\":{\\\"description\\\":\\\"What type of storage medium should back this directory. The default is \\\\\\\"\\\\\\\" which means to use the node's default medium. Must be an empty string (default) or Memory. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir\\\",\\\"type\\\":\\\"string\\\"},\\\"sizeLimit\\\":{}},\\\"type\\\":\\\"object\\\"},\\\"volumeClaimTemplate\\\":{\\\"description\\\":\\\"PersistentVolumeClaim is a user's request for and claim to a persistent volume\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"metadata\\\":{\\\"description\\\":\\\"ObjectMeta is metadata that all persisted resources must have, which includes all objects users must create.\\\",\\\"properties\\\":{\\\"annotations\\\":{\\\"description\\\":\\\"Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations\\\",\\\"type\\\":\\\"object\\\"},\\\"clusterName\\\":{\\\"description\\\":\\\"The name of the cluster which the object belongs to. This is used to distinguish resources with same name and namespace in different clusters. This field is not set anywhere right now and apiserver is going to ignore it if set in create or update request.\\\",\\\"type\\\":\\\"string\\\"},\\\"creationTimestamp\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"deletionGracePeriodSeconds\\\":{\\\"description\\\":\\\"Number of seconds allowed for this object to gracefully terminate before it will be removed from the system. Only set when deletionTimestamp is also set. May only be shortened. Read-only.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"deletionTimestamp\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"finalizers\\\":{\\\"description\\\":\\\"Must be empty before the object is deleted from the registry. Each entry is an identifier for the responsible component that will remove the entry from the list. If the deletionTimestamp of the object is non-nil, entries in this list can only be removed.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"generateName\\\":{\\\"description\\\":\\\"GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server.\\\\n\\\\nIf this field is specified and the generated name exists, the server will NOT return a 409 - instead, it will either return 201 Created or 500 with Reason ServerTimeout indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header).\\\\n\\\\nApplied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#idempotency\\\",\\\"type\\\":\\\"string\\\"},\\\"generation\\\":{\\\"description\\\":\\\"A sequence number representing a specific generation of the desired state. Populated by the system. Read-only.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"initializers\\\":{\\\"description\\\":\\\"Initializers tracks the progress of initialization.\\\",\\\"properties\\\":{\\\"pending\\\":{\\\"description\\\":\\\"Pending is a list of initializers that must execute in order before this object is visible. When the last pending initializer is removed, and no failing result is set, the initializers struct will be set to nil and the object is considered as initialized and visible to all clients.\\\",\\\"items\\\":{\\\"description\\\":\\\"Initializer is information about an initializer that has not yet completed.\\\",\\\"properties\\\":{\\\"name\\\":{\\\"description\\\":\\\"name of the process that is responsible for initializing this object.\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"result\\\":{\\\"description\\\":\\\"Status is a return value for calls that don't return other objects.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\\\",\\\"type\\\":\\\"string\\\"},\\\"code\\\":{\\\"description\\\":\\\"Suggested HTTP return code for this status, 0 if not set.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"details\\\":{\\\"description\\\":\\\"StatusDetails is a set of additional properties that MAY be set by the server to provide additional information about a response. The Reason field of a Status object defines what attributes will be set. Clients must ignore fields that do not match the defined type of each attribute, and should assume that any attribute may be empty, invalid, or under defined.\\\",\\\"properties\\\":{\\\"causes\\\":{\\\"description\\\":\\\"The Causes array includes more details associated with the StatusReason failure. Not all StatusReasons may provide detailed causes.\\\",\\\"items\\\":{\\\"description\\\":\\\"StatusCause provides more information about an api.Status failure, including cases when multiple errors are encountered.\\\",\\\"properties\\\":{\\\"field\\\":{\\\"description\\\":\\\"The field of the resource that has caused this error, as named by its JSON serialization. May include dot and postfix notation for nested attributes. Arrays are zero-indexed. Fields may appear more than once in an array of causes due to fields having multiple errors. Optional.\\\\n\\\\nExamples:\\\\n \\\\\\\"name\\\\\\\" - the field \\\\\\\"name\\\\\\\" on the current resource\\\\n \\\\\\\"items[0].name\\\\\\\" - the field \\\\\\\"name\\\\\\\" on the first array entry in \\\\\\\"items\\\\\\\"\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"A human-readable description of the cause of the error. This field may be presented as-is to a reader.\\\",\\\"type\\\":\\\"string\\\"},\\\"reason\\\":{\\\"description\\\":\\\"A machine-readable description of the cause of the error. If this value is empty there is no information available.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"group\\\":{\\\"description\\\":\\\"The group attribute of the resource associated with the status StatusReason.\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"The kind attribute of the resource associated with the status StatusReason. On some operations may differ from the requested resource Kind. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"The name attribute of the resource associated with the status StatusReason (when there is a single name which can be described).\\\",\\\"type\\\":\\\"string\\\"},\\\"retryAfterSeconds\\\":{\\\"description\\\":\\\"If specified, the time in seconds before the operation should be retried. Some errors may indicate the client must take an alternate action - for those errors this field may indicate how long to wait before taking the alternate action.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID of the resource. (when there is a single resource which can be described). More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"A human-readable description of the status of this operation.\\\",\\\"type\\\":\\\"string\\\"},\\\"metadata\\\":{\\\"description\\\":\\\"ListMeta describes metadata that synthetic resources must have, including lists and various status objects. A resource may have only one of {ObjectMeta, ListMeta}.\\\",\\\"properties\\\":{\\\"continue\\\":{\\\"description\\\":\\\"continue may be set if the user set a limit on the number of items returned, and indicates that the server has more data available. The value is opaque and may be used to issue another request to the endpoint that served this list to retrieve the next set of available objects. Continuing a consistent list may not be possible if the server configuration has changed or more than a few minutes have passed. The resourceVersion field returned when using this continue value will be identical to the value in the first response, unless you have received this token from an error message.\\\",\\\"type\\\":\\\"string\\\"},\\\"resourceVersion\\\":{\\\"description\\\":\\\"String that identifies the server's internal version of this object that can be used by clients to determine when objects have changed. Value must be treated as opaque by clients and passed unmodified back to the server. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\\\",\\\"type\\\":\\\"string\\\"},\\\"selfLink\\\":{\\\"description\\\":\\\"selfLink is a URL representing this object. Populated by the system. Read-only.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"reason\\\":{\\\"description\\\":\\\"A machine-readable description of why this operation is in the \\\\\\\"Failure\\\\\\\" status. If this value is empty there is no information available. A Reason clarifies an HTTP status code but does not override it.\\\",\\\"type\\\":\\\"string\\\"},\\\"status\\\":{\\\"description\\\":\\\"Status of the operation. One of: \\\\\\\"Success\\\\\\\" or \\\\\\\"Failure\\\\\\\". More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"required\\\":[\\\"pending\\\"],\\\"type\\\":\\\"object\\\"},\\\"labels\\\":{\\\"description\\\":\\\"Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels\\\",\\\"type\\\":\\\"object\\\"},\\\"managedFields\\\":{\\\"description\\\":\\\"ManagedFields maps workflow-id and version to the set of fields that are managed by that workflow. This is mostly for internal housekeeping, and users typically shouldn't need to set or understand this field. A workflow can be the user's name, a controller's name, or the name of a specific apply path like \\\\\\\"ci-cd\\\\\\\". The set of fields is always in the version that the workflow used when modifying the object.\\\\n\\\\nThis field is alpha and can be changed or removed without notice.\\\",\\\"items\\\":{\\\"description\\\":\\\"ManagedFieldsEntry is a workflow-id, a FieldSet and the group version of the resource that the fieldset applies to.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"APIVersion defines the version of this resource that this field set applies to. The format is \\\\\\\"group/version\\\\\\\" just like the top-level APIVersion field. It is necessary to track the version of a field set because it cannot be automatically converted.\\\",\\\"type\\\":\\\"string\\\"},\\\"fields\\\":{\\\"description\\\":\\\"Fields stores a set of fields in a data structure like a Trie. To understand how this is used, see: https://github.com/kubernetes-sigs/structured-merge-diff\\\",\\\"type\\\":\\\"object\\\"},\\\"manager\\\":{\\\"description\\\":\\\"Manager is an identifier of the workflow managing these fields.\\\",\\\"type\\\":\\\"string\\\"},\\\"operation\\\":{\\\"description\\\":\\\"Operation is the type of operation which lead to this ManagedFieldsEntry being created. The only valid values for this field are 'Apply' and 'Update'.\\\",\\\"type\\\":\\\"string\\\"},\\\"time\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names\\\",\\\"type\\\":\\\"string\\\"},\\\"namespace\\\":{\\\"description\\\":\\\"Namespace defines the space within each name must be unique. An empty namespace is equivalent to the \\\\\\\"default\\\\\\\" namespace, but \\\\\\\"default\\\\\\\" is the canonical representation. Not all objects are required to be scoped to a namespace - the value of this field for those objects will be empty.\\\\n\\\\nMust be a DNS_LABEL. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/namespaces\\\",\\\"type\\\":\\\"string\\\"},\\\"ownerReferences\\\":{\\\"description\\\":\\\"List of objects depended by this object. If ALL objects in the list have been deleted, this object will be garbage collected. If this object is managed by a controller, then an entry in this list will point to this controller, with the controller field set to true. There cannot be more than one managing controller.\\\",\\\"items\\\":{\\\"description\\\":\\\"OwnerReference contains enough information to let you identify an owning object. An owning object must be in the same namespace as the dependent, or be cluster-scoped, so there is no namespace field.\\\",\\\"properties\\\":{\\\"apiVersion\\\":{\\\"description\\\":\\\"API version of the referent.\\\",\\\"type\\\":\\\"string\\\"},\\\"blockOwnerDeletion\\\":{\\\"description\\\":\\\"If true, AND if the owner has the \\\\\\\"foregroundDeletion\\\\\\\" finalizer, then the owner cannot be deleted from the key-value store until this reference is removed. Defaults to false. To set this field, a user needs \\\\\\\"delete\\\\\\\" permission of the owner, otherwise 422 (Unprocessable Entity) will be returned.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"controller\\\":{\\\"description\\\":\\\"If true, this reference points to the managing controller.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#names\\\",\\\"type\\\":\\\"string\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"apiVersion\\\",\\\"kind\\\",\\\"name\\\",\\\"uid\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"resourceVersion\\\":{\\\"description\\\":\\\"An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources.\\\\n\\\\nPopulated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\\\",\\\"type\\\":\\\"string\\\"},\\\"selfLink\\\":{\\\"description\\\":\\\"SelfLink is a URL representing this object. Populated by the system. Read-only.\\\",\\\"type\\\":\\\"string\\\"},\\\"uid\\\":{\\\"description\\\":\\\"UID is the unique in time and space value for this object. It is typically generated by the server on successful creation of a resource and is not allowed to change on PUT operations.\\\\n\\\\nPopulated by the system. Read-only. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"spec\\\":{\\\"description\\\":\\\"PersistentVolumeClaimSpec describes the common attributes of storage devices and allows a Source for provider-specific attributes\\\",\\\"properties\\\":{\\\"accessModes\\\":{\\\"description\\\":\\\"AccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"dataSource\\\":{\\\"description\\\":\\\"TypedLocalObjectReference contains enough information to let you locate the typed referenced object inside the same namespace.\\\",\\\"properties\\\":{\\\"apiGroup\\\":{\\\"description\\\":\\\"APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.\\\",\\\"type\\\":\\\"string\\\"},\\\"kind\\\":{\\\"description\\\":\\\"Kind is the type of resource being referenced\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name is the name of resource being referenced\\\",\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"kind\\\",\\\"name\\\"],\\\"type\\\":\\\"object\\\"},\\\"resources\\\":{\\\"description\\\":\\\"ResourceRequirements describes the compute resource requirements.\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"selector\\\":{\\\"description\\\":\\\"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\\\",\\\"properties\\\":{\\\"matchExpressions\\\":{\\\"description\\\":\\\"matchExpressions is a list of label selector requirements. The requirements are ANDed.\\\",\\\"items\\\":{\\\"description\\\":\\\"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"key is the label key that the selector applies to.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\\\",\\\"type\\\":\\\"string\\\"},\\\"values\\\":{\\\"description\\\":\\\"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"}},\\\"required\\\":[\\\"key\\\",\\\"operator\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"matchLabels\\\":{\\\"description\\\":\\\"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\\\\\"key\\\\\\\", the operator is \\\\\\\"In\\\\\\\", and the values array contains only \\\\\\\"value\\\\\\\". The requirements are ANDed.\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"storageClassName\\\":{\\\"description\\\":\\\"Name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1\\\",\\\"type\\\":\\\"string\\\"},\\\"volumeMode\\\":{\\\"description\\\":\\\"volumeMode defines what type of volume is required by the claim. Value of Filesystem is implied when not included in claim spec. This is a beta feature.\\\",\\\"type\\\":\\\"string\\\"},\\\"volumeName\\\":{\\\"description\\\":\\\"VolumeName is the binding reference to the PersistentVolume backing this claim.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"status\\\":{\\\"description\\\":\\\"PersistentVolumeClaimStatus is the current status of a persistent volume claim.\\\",\\\"properties\\\":{\\\"accessModes\\\":{\\\"description\\\":\\\"AccessModes contains the actual access modes the volume backing the PVC has. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\\\",\\\"items\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":\\\"array\\\"},\\\"capacity\\\":{\\\"description\\\":\\\"Represents the actual resources of the underlying volume.\\\",\\\"type\\\":\\\"object\\\"},\\\"conditions\\\":{\\\"description\\\":\\\"Current Condition of persistent volume claim. If underlying persistent volume is being resized then the Condition will be set to 'ResizeStarted'.\\\",\\\"items\\\":{\\\"description\\\":\\\"PersistentVolumeClaimCondition contains details about state of pvc\\\",\\\"properties\\\":{\\\"lastProbeTime\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"lastTransitionTime\\\":{\\\"description\\\":\\\"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\\\",\\\"format\\\":\\\"date-time\\\",\\\"type\\\":\\\"string\\\"},\\\"message\\\":{\\\"description\\\":\\\"Human-readable message indicating details about last transition.\\\",\\\"type\\\":\\\"string\\\"},\\\"reason\\\":{\\\"description\\\":\\\"Unique, this should be a short, machine understandable string that gives the reason for condition's last transition. If it reports \\\\\\\"ResizeStarted\\\\\\\" that means the underlying persistent volume is being resized.\\\",\\\"type\\\":\\\"string\\\"},\\\"status\\\":{\\\"type\\\":\\\"string\\\"},\\\"type\\\":{\\\"type\\\":\\\"string\\\"}},\\\"required\\\":[\\\"type\\\",\\\"status\\\"],\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"phase\\\":{\\\"description\\\":\\\"Phase represents the current phase of PersistentVolumeClaim.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"tag\\\":{\\\"description\\\":\\\"Tag of Prometheus container image to be deployed. Defaults to the value of `version`. Version is ignored if Tag is set.\\\",\\\"type\\\":\\\"string\\\"},\\\"thanos\\\":{\\\"description\\\":\\\"ThanosSpec defines parameters for a Prometheus server within a Thanos deployment.\\\",\\\"properties\\\":{\\\"baseImage\\\":{\\\"description\\\":\\\"Thanos base image if other than default.\\\",\\\"type\\\":\\\"string\\\"},\\\"image\\\":{\\\"description\\\":\\\"Image if specified has precedence over baseImage, tag and sha combinations. Specifying the version is still necessary to ensure the Prometheus Operator knows what version of Thanos is being configured.\\\",\\\"type\\\":\\\"string\\\"},\\\"objectStorageConfig\\\":{\\\"description\\\":\\\"SecretKeySelector selects a key of a Secret.\\\",\\\"properties\\\":{\\\"key\\\":{\\\"description\\\":\\\"The key of the secret to select from. Must be a valid secret key.\\\",\\\"type\\\":\\\"string\\\"},\\\"name\\\":{\\\"description\\\":\\\"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\\\",\\\"type\\\":\\\"string\\\"},\\\"optional\\\":{\\\"description\\\":\\\"Specify whether the Secret or it's key must be defined\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"required\\\":[\\\"key\\\"],\\\"type\\\":\\\"object\\\"},\\\"resources\\\":{\\\"description\\\":\\\"ResourceRequirements describes the compute resource requirements.\\\",\\\"properties\\\":{\\\"limits\\\":{\\\"description\\\":\\\"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"},\\\"requests\\\":{\\\"description\\\":\\\"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\\\",\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"},\\\"sha\\\":{\\\"description\\\":\\\"SHA of Thanos container image to be deployed. Defaults to the value of `version`. Similar to a tag, but the SHA explicitly deploys an immutable container image. Version and Tag are ignored if SHA is set.\\\",\\\"type\\\":\\\"string\\\"},\\\"tag\\\":{\\\"description\\\":\\\"Tag of Thanos sidecar container image to be deployed. Defaults to the value of `version`. Version is ignored if Tag is set.\\\",\\\"type\\\":\\\"string\\\"},\\\"version\\\":{\\\"description\\\":\\\"Version describes the version of Thanos to use.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"tolerations\\\":{\\\"description\\\":\\\"If specified, the pod's tolerations.\\\",\\\"items\\\":{\\\"description\\\":\\\"The pod this Toleration is attached to tolerates any taint that matches the triple \\\\u003ckey,value,effect\\\\u003e using the matching operator \\\\u003coperator\\\\u003e.\\\",\\\"properties\\\":{\\\"effect\\\":{\\\"description\\\":\\\"Effect indicates the taint effect to match. Empty means match all taint effects. When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.\\\",\\\"type\\\":\\\"string\\\"},\\\"key\\\":{\\\"description\\\":\\\"Key is the taint key that the toleration applies to. Empty means match all taint keys. If the key is empty, operator must be Exists; this combination means to match all values and all keys.\\\",\\\"type\\\":\\\"string\\\"},\\\"operator\\\":{\\\"description\\\":\\\"Operator represents a key's relationship to the value. Valid operators are Exists and Equal. Defaults to Equal. Exists is equivalent to wildcard for value, so that a pod can tolerate all taints of a particular category.\\\",\\\"type\\\":\\\"string\\\"},\\\"tolerationSeconds\\\":{\\\"description\\\":\\\"TolerationSeconds represents the period of time the toleration (which must be of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately) by the system.\\\",\\\"format\\\":\\\"int64\\\",\\\"type\\\":\\\"integer\\\"},\\\"value\\\":{\\\"description\\\":\\\"Value is the taint value the toleration matches to. If the operator is Exists, the value should be empty, otherwise just a regular string.\\\",\\\"type\\\":\\\"string\\\"}},\\\"type\\\":\\\"object\\\"},\\\"type\\\":\\\"array\\\"},\\\"version\\\":{\\\"description\\\":\\\"Version of Prometheus to be deployed.\\\",\\\"type\\\":\\\"string\\\"},\\\"walCompression\\\":{\\\"description\\\":\\\"Enable compression of the write-ahead log using Snappy.\\\",\\\"type\\\":\\\"boolean\\\"}},\\\"type\\\":\\\"object\\\"},\\\"status\\\":{\\\"description\\\":\\\"PrometheusStatus is the most recent observed status of the Prometheus cluster. Read-only. Not included when requesting from the apiserver, only from the Prometheus Operator API itself. More info: https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#spec-and-status\\\",\\\"properties\\\":{\\\"availableReplicas\\\":{\\\"description\\\":\\\"Total number of available pods (ready for at least minReadySeconds) targeted by this Prometheus deployment.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"paused\\\":{\\\"description\\\":\\\"Represents whether any actions on the underlying managed objects are being performed. Only delete actions will be performed.\\\",\\\"type\\\":\\\"boolean\\\"},\\\"replicas\\\":{\\\"description\\\":\\\"Total number of non-terminated pods targeted by this Prometheus deployment (their labels match the selector).\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"unavailableReplicas\\\":{\\\"description\\\":\\\"Total number of unavailable pods targeted by this Prometheus deployment.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"},\\\"updatedReplicas\\\":{\\\"description\\\":\\\"Total number of non-terminated pods targeted by this Prometheus deployment that have the desired version spec.\\\",\\\"format\\\":\\\"int32\\\",\\\"type\\\":\\\"integer\\\"}},\\\"required\\\":[\\\"paused\\\",\\\"replicas\\\",\\\"updatedReplicas\\\",\\\"availableReplicas\\\",\\\"unavailableReplicas\\\"],\\\"type\\\":\\\"object\\\"}},\\\"type\\\":\\\"object\\\"}},\\\"version\\\":\\\"v1\\\"}}\\n\"\n }\n },\n \"spec\": {\n \"group\": \"monitoring.coreos.com\",\n \"version\": \"v1\",\n \"names\": {\n \"plural\": \"prometheuses\",\n \"singular\": \"prometheus\",\n \"kind\": \"Prometheus\",\n \"listKind\": \"PrometheusList\"\n },\n \"scope\": \"Namespaced\",\n \"validation\": {\n \"openAPIV3Schema\": {\n \"type\": \"object\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"spec\": {\n \"description\": \"PrometheusSpec is a specification of the desired behavior of the Prometheus cluster. More info: https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#spec-and-status\",\n \"type\": \"object\",\n \"properties\": {\n \"additionalAlertManagerConfigs\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"type\": \"object\",\n \"required\": [\n \"key\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n }\n },\n \"additionalAlertRelabelConfigs\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"type\": \"object\",\n \"required\": [\n \"key\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n }\n },\n \"additionalScrapeConfigs\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"type\": \"object\",\n \"required\": [\n \"key\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n }\n },\n \"affinity\": {\n \"description\": \"Affinity is a group of affinity scheduling rules.\",\n \"type\": \"object\",\n \"properties\": {\n \"nodeAffinity\": {\n \"description\": \"Node affinity is a group of node affinity scheduling rules.\",\n \"type\": \"object\",\n \"properties\": {\n \"preferredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\"weight\\\" to the sum if the node matches the corresponding matchExpressions; the node(s) with the highest sum are the most preferred.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"An empty preferred scheduling term matches all objects with implicit weight 0 (i.e. it's a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op).\",\n \"type\": \"object\",\n \"required\": [\n \"weight\",\n \"preference\"\n ],\n \"properties\": {\n \"preference\": {\n \"description\": \"A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.\",\n \"type\": \"object\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"A list of node selector requirements by node's labels.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"type\": \"object\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchFields\": {\n \"description\": \"A list of node selector requirements by node's fields.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"type\": \"object\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n }\n }\n },\n \"weight\": {\n \"description\": \"Weight associated with matching the corresponding nodeSelectorTerm, in the range 1-100.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n }\n }\n }\n },\n \"requiredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"A node selector represents the union of the results of one or more label queries over a set of nodes; that is, it represents the OR of the selectors represented by the node selector terms.\",\n \"type\": \"object\",\n \"required\": [\n \"nodeSelectorTerms\"\n ],\n \"properties\": {\n \"nodeSelectorTerms\": {\n \"description\": \"Required. A list of node selector terms. The terms are ORed.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.\",\n \"type\": \"object\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"A list of node selector requirements by node's labels.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"type\": \"object\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchFields\": {\n \"description\": \"A list of node selector requirements by node's fields.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"type\": \"object\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n }\n }\n }\n }\n }\n }\n }\n },\n \"podAffinity\": {\n \"description\": \"Pod affinity is a group of inter pod affinity scheduling rules.\",\n \"type\": \"object\",\n \"properties\": {\n \"preferredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\"weight\\\" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)\",\n \"type\": \"object\",\n \"required\": [\n \"weight\",\n \"podAffinityTerm\"\n ],\n \"properties\": {\n \"podAffinityTerm\": {\n \"description\": \"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key matches that of any node on which a pod of the set of pods is running\",\n \"type\": \"object\",\n \"required\": [\n \"topologyKey\"\n ],\n \"properties\": {\n \"labelSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"type\": \"object\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"type\": \"object\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"namespaces\": {\n \"description\": \"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\"this pod's namespace\\\"\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"topologyKey\": {\n \"description\": \"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\",\n \"type\": \"string\"\n }\n }\n },\n \"weight\": {\n \"description\": \"weight associated with matching the corresponding podAffinityTerm, in the range 1-100.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n }\n }\n }\n },\n \"requiredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key matches that of any node on which a pod of the set of pods is running\",\n \"type\": \"object\",\n \"required\": [\n \"topologyKey\"\n ],\n \"properties\": {\n \"labelSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"type\": \"object\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"type\": \"object\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"namespaces\": {\n \"description\": \"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\"this pod's namespace\\\"\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"topologyKey\": {\n \"description\": \"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\",\n \"type\": \"string\"\n }\n }\n }\n }\n }\n },\n \"podAntiAffinity\": {\n \"description\": \"Pod anti affinity is a group of inter pod anti affinity scheduling rules.\",\n \"type\": \"object\",\n \"properties\": {\n \"preferredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"The scheduler will prefer to schedule pods to nodes that satisfy the anti-affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling anti-affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding \\\"weight\\\" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"The weights of all of the matched WeightedPodAffinityTerm fields are added per-node to find the most preferred node(s)\",\n \"type\": \"object\",\n \"required\": [\n \"weight\",\n \"podAffinityTerm\"\n ],\n \"properties\": {\n \"podAffinityTerm\": {\n \"description\": \"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key matches that of any node on which a pod of the set of pods is running\",\n \"type\": \"object\",\n \"required\": [\n \"topologyKey\"\n ],\n \"properties\": {\n \"labelSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"type\": \"object\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"type\": \"object\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"namespaces\": {\n \"description\": \"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\"this pod's namespace\\\"\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"topologyKey\": {\n \"description\": \"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\",\n \"type\": \"string\"\n }\n }\n },\n \"weight\": {\n \"description\": \"weight associated with matching the corresponding podAffinityTerm, in the range 1-100.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n }\n }\n }\n },\n \"requiredDuringSchedulingIgnoredDuringExecution\": {\n \"description\": \"If the anti-affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the anti-affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key matches that of any node on which a pod of the set of pods is running\",\n \"type\": \"object\",\n \"required\": [\n \"topologyKey\"\n ],\n \"properties\": {\n \"labelSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"type\": \"object\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"type\": \"object\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"namespaces\": {\n \"description\": \"namespaces specifies which namespaces the labelSelector applies to (matches against); null or empty list means \\\"this pod's namespace\\\"\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"topologyKey\": {\n \"description\": \"This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.\",\n \"type\": \"string\"\n }\n }\n }\n }\n }\n }\n }\n },\n \"alerting\": {\n \"description\": \"AlertingSpec defines parameters for alerting configuration of Prometheus servers.\",\n \"type\": \"object\",\n \"required\": [\n \"alertmanagers\"\n ],\n \"properties\": {\n \"alertmanagers\": {\n \"description\": \"AlertmanagerEndpoints Prometheus should fire alerts against.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"AlertmanagerEndpoints defines a selection of a single Endpoints object containing alertmanager IPs to fire alerts against.\",\n \"type\": \"object\",\n \"required\": [\n \"namespace\",\n \"name\",\n \"port\"\n ],\n \"properties\": {\n \"bearerTokenFile\": {\n \"description\": \"BearerTokenFile to read from filesystem to use when authenticating to Alertmanager.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of Endpoints object in Namespace.\",\n \"type\": \"string\"\n },\n \"namespace\": {\n \"description\": \"Namespace of Endpoints object.\",\n \"type\": \"string\"\n },\n \"pathPrefix\": {\n \"description\": \"Prefix for the HTTP path alerts are pushed to.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n },\n \"scheme\": {\n \"description\": \"Scheme to use when firing alerts.\",\n \"type\": \"string\"\n },\n \"tlsConfig\": {\n \"description\": \"TLSConfig specifies TLS configuration parameters.\",\n \"type\": \"object\",\n \"properties\": {\n \"caFile\": {\n \"description\": \"The CA cert to use for the targets.\",\n \"type\": \"string\"\n },\n \"certFile\": {\n \"description\": \"The client cert file for the targets.\",\n \"type\": \"string\"\n },\n \"insecureSkipVerify\": {\n \"description\": \"Disable target certificate validation.\",\n \"type\": \"boolean\"\n },\n \"keyFile\": {\n \"description\": \"The client key file for the targets.\",\n \"type\": \"string\"\n },\n \"serverName\": {\n \"description\": \"Used to verify the hostname for the targets.\",\n \"type\": \"string\"\n }\n }\n }\n }\n }\n }\n }\n },\n \"apiserverConfig\": {\n \"description\": \"APIServerConfig defines a host and auth methods to access apiserver. More info: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#kubernetes_sd_config\",\n \"type\": \"object\",\n \"required\": [\n \"host\"\n ],\n \"properties\": {\n \"basicAuth\": {\n \"description\": \"BasicAuth allow an endpoint to authenticate over basic authentication More info: https://prometheus.io/docs/operating/configuration/#endpoints\",\n \"type\": \"object\",\n \"properties\": {\n \"password\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"type\": \"object\",\n \"required\": [\n \"key\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n }\n },\n \"username\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"type\": \"object\",\n \"required\": [\n \"key\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n }\n }\n }\n },\n \"bearerToken\": {\n \"description\": \"Bearer token for accessing apiserver.\",\n \"type\": \"string\"\n },\n \"bearerTokenFile\": {\n \"description\": \"File to read bearer token for accessing apiserver.\",\n \"type\": \"string\"\n },\n \"host\": {\n \"description\": \"Host of apiserver. A valid string consisting of a hostname or IP followed by an optional port number\",\n \"type\": \"string\"\n },\n \"tlsConfig\": {\n \"description\": \"TLSConfig specifies TLS configuration parameters.\",\n \"type\": \"object\",\n \"properties\": {\n \"caFile\": {\n \"description\": \"The CA cert to use for the targets.\",\n \"type\": \"string\"\n },\n \"certFile\": {\n \"description\": \"The client cert file for the targets.\",\n \"type\": \"string\"\n },\n \"insecureSkipVerify\": {\n \"description\": \"Disable target certificate validation.\",\n \"type\": \"boolean\"\n },\n \"keyFile\": {\n \"description\": \"The client key file for the targets.\",\n \"type\": \"string\"\n },\n \"serverName\": {\n \"description\": \"Used to verify the hostname for the targets.\",\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"baseImage\": {\n \"description\": \"Base image to use for a Prometheus deployment.\",\n \"type\": \"string\"\n },\n \"configMaps\": {\n \"description\": \"ConfigMaps is a list of ConfigMaps in the same namespace as the Prometheus object, which shall be mounted into the Prometheus Pods. The ConfigMaps are mounted into /etc/prometheus/configmaps/.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"containers\": {\n \"description\": \"Containers allows injecting additional containers or modifying operator generated containers. This can be used to allow adding an authentication proxy to a Prometheus pod or to change the behavior of an operator generated container. Containers described here modify an operator generated container if they share the same name and modifications are done via a strategic merge patch. The current container names are: `prometheus`, `prometheus-config-reloader`, `rules-configmap-reloader`, and `thanos-sidecar`. Overriding containers is entirely outside the scope of what the maintainers will support and by doing so, you accept that this behaviour may break at any time without notice.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A single application container that you want to run within a pod.\",\n \"type\": \"object\",\n \"required\": [\n \"name\"\n ],\n \"properties\": {\n \"args\": {\n \"description\": \"Arguments to the entrypoint. The docker image's CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"command\": {\n \"description\": \"Entrypoint array. Not executed within a shell. The docker image's ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"env\": {\n \"description\": \"List of environment variables to set in the container. Cannot be updated.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"EnvVar represents an environment variable present in a Container.\",\n \"type\": \"object\",\n \"required\": [\n \"name\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the environment variable. Must be a C_IDENTIFIER.\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"Variable references $(VAR_NAME) are expanded using the previous defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to \\\"\\\".\",\n \"type\": \"string\"\n },\n \"valueFrom\": {\n \"description\": \"EnvVarSource represents a source for the value of an EnvVar.\",\n \"type\": \"object\",\n \"properties\": {\n \"configMapKeyRef\": {\n \"description\": \"Selects a key from a ConfigMap.\",\n \"type\": \"object\",\n \"required\": [\n \"key\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The key to select.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap or it's key must be defined\",\n \"type\": \"boolean\"\n }\n }\n },\n \"fieldRef\": {\n \"description\": \"ObjectFieldSelector selects an APIVersioned field of an object.\",\n \"type\": \"object\",\n \"required\": [\n \"fieldPath\"\n ],\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"Version of the schema the FieldPath is written in terms of, defaults to \\\"v1\\\".\",\n \"type\": \"string\"\n },\n \"fieldPath\": {\n \"description\": \"Path of the field to select in the specified API version.\",\n \"type\": \"string\"\n }\n }\n },\n \"resourceFieldRef\": {\n \"description\": \"ResourceFieldSelector represents container resources (cpu, memory) and their output format\",\n \"type\": \"object\",\n \"required\": [\n \"resource\"\n ],\n \"properties\": {\n \"containerName\": {\n \"description\": \"Container name: required for volumes, optional for env vars\",\n \"type\": \"string\"\n },\n \"divisor\": {},\n \"resource\": {\n \"description\": \"Required: resource to select\",\n \"type\": \"string\"\n }\n }\n },\n \"secretKeyRef\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"type\": \"object\",\n \"required\": [\n \"key\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n }\n }\n }\n }\n }\n }\n },\n \"envFrom\": {\n \"description\": \"List of sources to populate environment variables in the container. The keys defined within a source must be a C_IDENTIFIER. All invalid keys will be reported as an event when the container is starting. When a key exists in multiple sources, the value associated with the last source will take precedence. Values defined by an Env with a duplicate key will take precedence. Cannot be updated.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"EnvFromSource represents the source of a set of ConfigMaps\",\n \"type\": \"object\",\n \"properties\": {\n \"configMapRef\": {\n \"description\": \"ConfigMapEnvSource selects a ConfigMap to populate the environment variables with.\\n\\nThe contents of the target ConfigMap's Data field will represent the key-value pairs as environment variables.\",\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the ConfigMap must be defined\",\n \"type\": \"boolean\"\n }\n }\n },\n \"prefix\": {\n \"description\": \"An optional identifier to prepend to each key in the ConfigMap. Must be a C_IDENTIFIER.\",\n \"type\": \"string\"\n },\n \"secretRef\": {\n \"description\": \"SecretEnvSource selects a Secret to populate the environment variables with.\\n\\nThe contents of the target Secret's Data field will represent the key-value pairs as environment variables.\",\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret must be defined\",\n \"type\": \"boolean\"\n }\n }\n }\n }\n }\n },\n \"image\": {\n \"description\": \"Docker image name. More info: https://kubernetes.io/docs/concepts/containers/images This field is optional to allow higher level config management to default or override container images in workload controllers like Deployments and StatefulSets.\",\n \"type\": \"string\"\n },\n \"imagePullPolicy\": {\n \"description\": \"Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images\",\n \"type\": \"string\"\n },\n \"lifecycle\": {\n \"description\": \"Lifecycle describes actions that the management system should take in response to container lifecycle events. For the PostStart and PreStop lifecycle handlers, management of the container blocks until the action is complete, unless the container process fails, in which case the handler is aborted.\",\n \"type\": \"object\",\n \"properties\": {\n \"postStart\": {\n \"description\": \"Handler defines a specific action that should be taken\",\n \"type\": \"object\",\n \"properties\": {\n \"exec\": {\n \"description\": \"ExecAction describes a \\\"run in container\\\" action.\",\n \"type\": \"object\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n },\n \"httpGet\": {\n \"description\": \"HTTPGetAction describes an action based on HTTP Get requests.\",\n \"type\": \"object\",\n \"required\": [\n \"port\"\n ],\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"type\": \"object\",\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n }\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocketAction describes an action based on opening a socket\",\n \"type\": \"object\",\n \"required\": [\n \"port\"\n ],\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n }\n }\n }\n }\n },\n \"preStop\": {\n \"description\": \"Handler defines a specific action that should be taken\",\n \"type\": \"object\",\n \"properties\": {\n \"exec\": {\n \"description\": \"ExecAction describes a \\\"run in container\\\" action.\",\n \"type\": \"object\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n },\n \"httpGet\": {\n \"description\": \"HTTPGetAction describes an action based on HTTP Get requests.\",\n \"type\": \"object\",\n \"required\": [\n \"port\"\n ],\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"type\": \"object\",\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n }\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocketAction describes an action based on opening a socket\",\n \"type\": \"object\",\n \"required\": [\n \"port\"\n ],\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n }\n }\n }\n }\n }\n }\n },\n \"livenessProbe\": {\n \"description\": \"Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic.\",\n \"type\": \"object\",\n \"properties\": {\n \"exec\": {\n \"description\": \"ExecAction describes a \\\"run in container\\\" action.\",\n \"type\": \"object\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n },\n \"failureThreshold\": {\n \"description\": \"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"httpGet\": {\n \"description\": \"HTTPGetAction describes an action based on HTTP Get requests.\",\n \"type\": \"object\",\n \"required\": [\n \"port\"\n ],\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"type\": \"object\",\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n }\n },\n \"initialDelaySeconds\": {\n \"description\": \"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"periodSeconds\": {\n \"description\": \"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"successThreshold\": {\n \"description\": \"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocketAction describes an action based on opening a socket\",\n \"type\": \"object\",\n \"required\": [\n \"port\"\n ],\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n }\n }\n },\n \"timeoutSeconds\": {\n \"description\": \"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n }\n }\n },\n \"name\": {\n \"description\": \"Name of the container specified as a DNS_LABEL. Each container in a pod must have a unique name (DNS_LABEL). Cannot be updated.\",\n \"type\": \"string\"\n },\n \"ports\": {\n \"description\": \"List of ports to expose from the container. Exposing a port here gives the system additional information about the network connections a container uses, but is primarily informational. Not specifying a port here DOES NOT prevent that port from being exposed. Any port which is listening on the default \\\"0.0.0.0\\\" address inside a container will be accessible from the network. Cannot be updated.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"ContainerPort represents a network port in a single container.\",\n \"type\": \"object\",\n \"required\": [\n \"containerPort\"\n ],\n \"properties\": {\n \"containerPort\": {\n \"description\": \"Number of port to expose on the pod's IP address. This must be a valid port number, 0 < x < 65536.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"hostIP\": {\n \"description\": \"What host IP to bind the external port to.\",\n \"type\": \"string\"\n },\n \"hostPort\": {\n \"description\": \"Number of port to expose on the host. If specified, this must be a valid port number, 0 < x < 65536. If HostNetwork is specified, this must match ContainerPort. Most containers do not need this.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"name\": {\n \"description\": \"If specified, this must be an IANA_SVC_NAME and unique within the pod. Each named port in a pod must have a unique name. Name for the port that can be referred to by services.\",\n \"type\": \"string\"\n },\n \"protocol\": {\n \"description\": \"Protocol for port. Must be UDP, TCP, or SCTP. Defaults to \\\"TCP\\\".\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"readinessProbe\": {\n \"description\": \"Probe describes a health check to be performed against a container to determine whether it is alive or ready to receive traffic.\",\n \"type\": \"object\",\n \"properties\": {\n \"exec\": {\n \"description\": \"ExecAction describes a \\\"run in container\\\" action.\",\n \"type\": \"object\",\n \"properties\": {\n \"command\": {\n \"description\": \"Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use a shell, you need to explicitly call out to that shell. Exit status of 0 is treated as live/healthy and non-zero is unhealthy.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n },\n \"failureThreshold\": {\n \"description\": \"Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"httpGet\": {\n \"description\": \"HTTPGetAction describes an action based on HTTP Get requests.\",\n \"type\": \"object\",\n \"required\": [\n \"port\"\n ],\n \"properties\": {\n \"host\": {\n \"description\": \"Host name to connect to, defaults to the pod IP. You probably want to set \\\"Host\\\" in httpHeaders instead.\",\n \"type\": \"string\"\n },\n \"httpHeaders\": {\n \"description\": \"Custom headers to set in the request. HTTP allows repeated headers.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"HTTPHeader describes a custom header to be used in HTTP probes\",\n \"type\": \"object\",\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"The header field name\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"The header field value\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"path\": {\n \"description\": \"Path to access on the HTTP server.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n },\n \"scheme\": {\n \"description\": \"Scheme to use for connecting to the host. Defaults to HTTP.\",\n \"type\": \"string\"\n }\n }\n },\n \"initialDelaySeconds\": {\n \"description\": \"Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"periodSeconds\": {\n \"description\": \"How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"successThreshold\": {\n \"description\": \"Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"tcpSocket\": {\n \"description\": \"TCPSocketAction describes an action based on opening a socket\",\n \"type\": \"object\",\n \"required\": [\n \"port\"\n ],\n \"properties\": {\n \"host\": {\n \"description\": \"Optional: Host name to connect to, defaults to the pod IP.\",\n \"type\": \"string\"\n },\n \"port\": {\n \"anyOf\": [\n {\n \"type\": \"string\"\n },\n {\n \"type\": \"integer\"\n }\n ]\n }\n }\n },\n \"timeoutSeconds\": {\n \"description\": \"Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n }\n }\n },\n \"resources\": {\n \"description\": \"ResourceRequirements describes the compute resource requirements.\",\n \"type\": \"object\",\n \"properties\": {\n \"limits\": {\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n }\n },\n \"securityContext\": {\n \"description\": \"SecurityContext holds security configuration that will be applied to a container. Some fields are present in both SecurityContext and PodSecurityContext. When both are set, the values in SecurityContext take precedence.\",\n \"type\": \"object\",\n \"properties\": {\n \"allowPrivilegeEscalation\": {\n \"description\": \"AllowPrivilegeEscalation controls whether a process can gain more privileges than its parent process. This bool directly controls if the no_new_privs flag will be set on the container process. AllowPrivilegeEscalation is true always when the container is: 1) run as Privileged 2) has CAP_SYS_ADMIN\",\n \"type\": \"boolean\"\n },\n \"capabilities\": {\n \"description\": \"Adds and removes POSIX capabilities from running containers.\",\n \"type\": \"object\",\n \"properties\": {\n \"add\": {\n \"description\": \"Added capabilities\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"drop\": {\n \"description\": \"Removed capabilities\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n },\n \"privileged\": {\n \"description\": \"Run container in privileged mode. Processes in privileged containers are essentially equivalent to root on the host. Defaults to false.\",\n \"type\": \"boolean\"\n },\n \"procMount\": {\n \"description\": \"procMount denotes the type of proc mount to use for the containers. The default is DefaultProcMount which uses the container runtime defaults for readonly paths and masked paths. This requires the ProcMountType feature flag to be enabled.\",\n \"type\": \"string\"\n },\n \"readOnlyRootFilesystem\": {\n \"description\": \"Whether this container has a read-only root filesystem. Default is false.\",\n \"type\": \"boolean\"\n },\n \"runAsGroup\": {\n \"description\": \"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"runAsNonRoot\": {\n \"description\": \"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"type\": \"boolean\"\n },\n \"runAsUser\": {\n \"description\": \"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"seLinuxOptions\": {\n \"description\": \"SELinuxOptions are the labels to be applied to the container\",\n \"type\": \"object\",\n \"properties\": {\n \"level\": {\n \"description\": \"Level is SELinux level label that applies to the container.\",\n \"type\": \"string\"\n },\n \"role\": {\n \"description\": \"Role is a SELinux role label that applies to the container.\",\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"Type is a SELinux type label that applies to the container.\",\n \"type\": \"string\"\n },\n \"user\": {\n \"description\": \"User is a SELinux user label that applies to the container.\",\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"stdin\": {\n \"description\": \"Whether this container should allocate a buffer for stdin in the container runtime. If this is not set, reads from stdin in the container will always result in EOF. Default is false.\",\n \"type\": \"boolean\"\n },\n \"stdinOnce\": {\n \"description\": \"Whether the container runtime should close the stdin channel after it has been opened by a single attach. When stdin is true the stdin stream will remain open across multiple attach sessions. If stdinOnce is set to true, stdin is opened on container start, is empty until the first client attaches to stdin, and then remains open and accepts data until the client disconnects, at which time stdin is closed and remains closed until the container is restarted. If this flag is false, a container processes that reads from stdin will never receive an EOF. Default is false\",\n \"type\": \"boolean\"\n },\n \"terminationMessagePath\": {\n \"description\": \"Optional: Path at which the file to which the container's termination message will be written is mounted into the container's filesystem. Message written is intended to be brief final status, such as an assertion failure message. Will be truncated by the node if greater than 4096 bytes. The total message length across all containers will be limited to 12kb. Defaults to /dev/termination-log. Cannot be updated.\",\n \"type\": \"string\"\n },\n \"terminationMessagePolicy\": {\n \"description\": \"Indicate how the termination message should be populated. File will use the contents of terminationMessagePath to populate the container status message on both success and failure. FallbackToLogsOnError will use the last chunk of container log output if the termination message file is empty and the container exited with an error. The log output is limited to 2048 bytes or 80 lines, whichever is smaller. Defaults to File. Cannot be updated.\",\n \"type\": \"string\"\n },\n \"tty\": {\n \"description\": \"Whether this container should allocate a TTY for itself, also requires 'stdin' to be true. Default is false.\",\n \"type\": \"boolean\"\n },\n \"volumeDevices\": {\n \"description\": \"volumeDevices is the list of block devices to be used by the container. This is a beta feature.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"volumeDevice describes a mapping of a raw block device within a container.\",\n \"type\": \"object\",\n \"required\": [\n \"name\",\n \"devicePath\"\n ],\n \"properties\": {\n \"devicePath\": {\n \"description\": \"devicePath is the path inside of the container that the device will be mapped to.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"name must match the name of a persistentVolumeClaim in the pod\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"volumeMounts\": {\n \"description\": \"Pod volumes to mount into the container's filesystem. Cannot be updated.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"VolumeMount describes a mounting of a Volume within a container.\",\n \"type\": \"object\",\n \"required\": [\n \"name\",\n \"mountPath\"\n ],\n \"properties\": {\n \"mountPath\": {\n \"description\": \"Path within the container at which the volume should be mounted. Must not contain ':'.\",\n \"type\": \"string\"\n },\n \"mountPropagation\": {\n \"description\": \"mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"This must match the Name of a Volume.\",\n \"type\": \"string\"\n },\n \"readOnly\": {\n \"description\": \"Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.\",\n \"type\": \"boolean\"\n },\n \"subPath\": {\n \"description\": \"Path within the volume from which the container's volume should be mounted. Defaults to \\\"\\\" (volume's root).\",\n \"type\": \"string\"\n },\n \"subPathExpr\": {\n \"description\": \"Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to \\\"\\\" (volume's root). SubPathExpr and SubPath are mutually exclusive. This field is alpha in 1.14.\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"workingDir\": {\n \"description\": \"Container's working directory. If not specified, the container runtime's default will be used, which might be configured in the container image. Cannot be updated.\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"enableAdminAPI\": {\n \"description\": \"Enable access to prometheus web admin API. Defaults to the value of `false`. WARNING: Enabling the admin APIs enables mutating endpoints, to delete data, shutdown Prometheus, and more. Enabling this should be done with care and the user is advised to add additional authentication authorization via a proxy to ensure only clients authorized to perform these actions can do so. For more information see https://prometheus.io/docs/prometheus/latest/querying/api/#tsdb-admin-apis\",\n \"type\": \"boolean\"\n },\n \"evaluationInterval\": {\n \"description\": \"Interval between consecutive evaluations.\",\n \"type\": \"string\"\n },\n \"externalLabels\": {\n \"description\": \"The labels to add to any time series or alerts when communicating with external systems (federation, remote storage, Alertmanager).\",\n \"type\": \"object\"\n },\n \"externalUrl\": {\n \"description\": \"The external URL the Prometheus instances will be available under. This is necessary to generate correct URLs. This is necessary if Prometheus is not served from root of a DNS name.\",\n \"type\": \"string\"\n },\n \"image\": {\n \"description\": \"Image if specified has precedence over baseImage, tag and sha combinations. Specifying the version is still necessary to ensure the Prometheus Operator knows what version of Prometheus is being configured.\",\n \"type\": \"string\"\n },\n \"imagePullSecrets\": {\n \"description\": \"An optional list of references to secrets in the same namespace to use for pulling prometheus and alertmanager images from registries see http://kubernetes.io/docs/user-guide/images#specifying-imagepullsecrets-on-a-pod\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"LocalObjectReference contains enough information to let you locate the referenced object inside the same namespace.\",\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"listenLocal\": {\n \"description\": \"ListenLocal makes the Prometheus server listen on loopback, so that it does not bind against the Pod IP.\",\n \"type\": \"boolean\"\n },\n \"logFormat\": {\n \"description\": \"Log format for Prometheus to be configured with.\",\n \"type\": \"string\"\n },\n \"logLevel\": {\n \"description\": \"Log level for Prometheus to be configured with.\",\n \"type\": \"string\"\n },\n \"nodeSelector\": {\n \"description\": \"Define which Nodes the Pods are scheduled on.\",\n \"type\": \"object\"\n },\n \"paused\": {\n \"description\": \"When a Prometheus deployment is paused, no actions except for deletion will be performed on the underlying objects.\",\n \"type\": \"boolean\"\n },\n \"podMetadata\": {\n \"description\": \"ObjectMeta is metadata that all persisted resources must have, which includes all objects users must create.\",\n \"type\": \"object\",\n \"properties\": {\n \"annotations\": {\n \"description\": \"Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations\",\n \"type\": \"object\"\n },\n \"clusterName\": {\n \"description\": \"The name of the cluster which the object belongs to. This is used to distinguish resources with same name and namespace in different clusters. This field is not set anywhere right now and apiserver is going to ignore it if set in create or update request.\",\n \"type\": \"string\"\n },\n \"creationTimestamp\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"type\": \"string\",\n \"format\": \"date-time\"\n },\n \"deletionGracePeriodSeconds\": {\n \"description\": \"Number of seconds allowed for this object to gracefully terminate before it will be removed from the system. Only set when deletionTimestamp is also set. May only be shortened. Read-only.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"deletionTimestamp\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"type\": \"string\",\n \"format\": \"date-time\"\n },\n \"finalizers\": {\n \"description\": \"Must be empty before the object is deleted from the registry. Each entry is an identifier for the responsible component that will remove the entry from the list. If the deletionTimestamp of the object is non-nil, entries in this list can only be removed.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"generateName\": {\n \"description\": \"GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server.\\n\\nIf this field is specified and the generated name exists, the server will NOT return a 409 - instead, it will either return 201 Created or 500 with Reason ServerTimeout indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header).\\n\\nApplied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#idempotency\",\n \"type\": \"string\"\n },\n \"generation\": {\n \"description\": \"A sequence number representing a specific generation of the desired state. Populated by the system. Read-only.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"initializers\": {\n \"description\": \"Initializers tracks the progress of initialization.\",\n \"type\": \"object\",\n \"required\": [\n \"pending\"\n ],\n \"properties\": {\n \"pending\": {\n \"description\": \"Pending is a list of initializers that must execute in order before this object is visible. When the last pending initializer is removed, and no failing result is set, the initializers struct will be set to nil and the object is considered as initialized and visible to all clients.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"Initializer is information about an initializer that has not yet completed.\",\n \"type\": \"object\",\n \"required\": [\n \"name\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"name of the process that is responsible for initializing this object.\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"result\": {\n \"description\": \"Status is a return value for calls that don't return other objects.\",\n \"type\": \"object\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"code\": {\n \"description\": \"Suggested HTTP return code for this status, 0 if not set.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"details\": {\n \"description\": \"StatusDetails is a set of additional properties that MAY be set by the server to provide additional information about a response. The Reason field of a Status object defines what attributes will be set. Clients must ignore fields that do not match the defined type of each attribute, and should assume that any attribute may be empty, invalid, or under defined.\",\n \"type\": \"object\",\n \"properties\": {\n \"causes\": {\n \"description\": \"The Causes array includes more details associated with the StatusReason failure. Not all StatusReasons may provide detailed causes.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"StatusCause provides more information about an api.Status failure, including cases when multiple errors are encountered.\",\n \"type\": \"object\",\n \"properties\": {\n \"field\": {\n \"description\": \"The field of the resource that has caused this error, as named by its JSON serialization. May include dot and postfix notation for nested attributes. Arrays are zero-indexed. Fields may appear more than once in an array of causes due to fields having multiple errors. Optional.\\n\\nExamples:\\n \\\"name\\\" - the field \\\"name\\\" on the current resource\\n \\\"items[0].name\\\" - the field \\\"name\\\" on the first array entry in \\\"items\\\"\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"A human-readable description of the cause of the error. This field may be presented as-is to a reader.\",\n \"type\": \"string\"\n },\n \"reason\": {\n \"description\": \"A machine-readable description of the cause of the error. If this value is empty there is no information available.\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"group\": {\n \"description\": \"The group attribute of the resource associated with the status StatusReason.\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"The kind attribute of the resource associated with the status StatusReason. On some operations may differ from the requested resource Kind. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"The name attribute of the resource associated with the status StatusReason (when there is a single name which can be described).\",\n \"type\": \"string\"\n },\n \"retryAfterSeconds\": {\n \"description\": \"If specified, the time in seconds before the operation should be retried. Some errors may indicate the client must take an alternate action - for those errors this field may indicate how long to wait before taking the alternate action.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"uid\": {\n \"description\": \"UID of the resource. (when there is a single resource which can be described). More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n }\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"A human-readable description of the status of this operation.\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"description\": \"ListMeta describes metadata that synthetic resources must have, including lists and various status objects. A resource may have only one of {ObjectMeta, ListMeta}.\",\n \"type\": \"object\",\n \"properties\": {\n \"continue\": {\n \"description\": \"continue may be set if the user set a limit on the number of items returned, and indicates that the server has more data available. The value is opaque and may be used to issue another request to the endpoint that served this list to retrieve the next set of available objects. Continuing a consistent list may not be possible if the server configuration has changed or more than a few minutes have passed. The resourceVersion field returned when using this continue value will be identical to the value in the first response, unless you have received this token from an error message.\",\n \"type\": \"string\"\n },\n \"resourceVersion\": {\n \"description\": \"String that identifies the server's internal version of this object that can be used by clients to determine when objects have changed. Value must be treated as opaque by clients and passed unmodified back to the server. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\",\n \"type\": \"string\"\n },\n \"selfLink\": {\n \"description\": \"selfLink is a URL representing this object. Populated by the system. Read-only.\",\n \"type\": \"string\"\n }\n }\n },\n \"reason\": {\n \"description\": \"A machine-readable description of why this operation is in the \\\"Failure\\\" status. If this value is empty there is no information available. A Reason clarifies an HTTP status code but does not override it.\",\n \"type\": \"string\"\n },\n \"status\": {\n \"description\": \"Status of the operation. One of: \\\"Success\\\" or \\\"Failure\\\". More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status\",\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"labels\": {\n \"description\": \"Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels\",\n \"type\": \"object\"\n },\n \"managedFields\": {\n \"description\": \"ManagedFields maps workflow-id and version to the set of fields that are managed by that workflow. This is mostly for internal housekeeping, and users typically shouldn't need to set or understand this field. A workflow can be the user's name, a controller's name, or the name of a specific apply path like \\\"ci-cd\\\". The set of fields is always in the version that the workflow used when modifying the object.\\n\\nThis field is alpha and can be changed or removed without notice.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"ManagedFieldsEntry is a workflow-id, a FieldSet and the group version of the resource that the fieldset applies to.\",\n \"type\": \"object\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the version of this resource that this field set applies to. The format is \\\"group/version\\\" just like the top-level APIVersion field. It is necessary to track the version of a field set because it cannot be automatically converted.\",\n \"type\": \"string\"\n },\n \"fields\": {\n \"description\": \"Fields stores a set of fields in a data structure like a Trie. To understand how this is used, see: https://github.com/kubernetes-sigs/structured-merge-diff\",\n \"type\": \"object\"\n },\n \"manager\": {\n \"description\": \"Manager is an identifier of the workflow managing these fields.\",\n \"type\": \"string\"\n },\n \"operation\": {\n \"description\": \"Operation is the type of operation which lead to this ManagedFieldsEntry being created. The only valid values for this field are 'Apply' and 'Update'.\",\n \"type\": \"string\"\n },\n \"time\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"type\": \"string\",\n \"format\": \"date-time\"\n }\n }\n }\n },\n \"name\": {\n \"description\": \"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names\",\n \"type\": \"string\"\n },\n \"namespace\": {\n \"description\": \"Namespace defines the space within each name must be unique. An empty namespace is equivalent to the \\\"default\\\" namespace, but \\\"default\\\" is the canonical representation. Not all objects are required to be scoped to a namespace - the value of this field for those objects will be empty.\\n\\nMust be a DNS_LABEL. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/namespaces\",\n \"type\": \"string\"\n },\n \"ownerReferences\": {\n \"description\": \"List of objects depended by this object. If ALL objects in the list have been deleted, this object will be garbage collected. If this object is managed by a controller, then an entry in this list will point to this controller, with the controller field set to true. There cannot be more than one managing controller.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"OwnerReference contains enough information to let you identify an owning object. An owning object must be in the same namespace as the dependent, or be cluster-scoped, so there is no namespace field.\",\n \"type\": \"object\",\n \"required\": [\n \"apiVersion\",\n \"kind\",\n \"name\",\n \"uid\"\n ],\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"API version of the referent.\",\n \"type\": \"string\"\n },\n \"blockOwnerDeletion\": {\n \"description\": \"If true, AND if the owner has the \\\"foregroundDeletion\\\" finalizer, then the owner cannot be deleted from the key-value store until this reference is removed. Defaults to false. To set this field, a user needs \\\"delete\\\" permission of the owner, otherwise 422 (Unprocessable Entity) will be returned.\",\n \"type\": \"boolean\"\n },\n \"controller\": {\n \"description\": \"If true, this reference points to the managing controller.\",\n \"type\": \"boolean\"\n },\n \"kind\": {\n \"description\": \"Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#names\",\n \"type\": \"string\"\n },\n \"uid\": {\n \"description\": \"UID of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"resourceVersion\": {\n \"description\": \"An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources.\\n\\nPopulated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\",\n \"type\": \"string\"\n },\n \"selfLink\": {\n \"description\": \"SelfLink is a URL representing this object. Populated by the system. Read-only.\",\n \"type\": \"string\"\n },\n \"uid\": {\n \"description\": \"UID is the unique in time and space value for this object. It is typically generated by the server on successful creation of a resource and is not allowed to change on PUT operations.\\n\\nPopulated by the system. Read-only. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n }\n },\n \"podMonitorNamespaceSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"type\": \"object\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"type\": \"object\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"podMonitorSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"type\": \"object\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"type\": \"object\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"priorityClassName\": {\n \"description\": \"Priority class assigned to the Pods\",\n \"type\": \"string\"\n },\n \"prometheusExternalLabelName\": {\n \"description\": \"Name of Prometheus external label used to denote Prometheus instance name. Defaults to the value of `prometheus`. External label will _not_ be added when value is set to empty string (`\\\"\\\"`).\",\n \"type\": \"string\"\n },\n \"query\": {\n \"description\": \"QuerySpec defines the query command line flags when starting Prometheus.\",\n \"type\": \"object\",\n \"properties\": {\n \"lookbackDelta\": {\n \"description\": \"The delta difference allowed for retrieving metrics during expression evaluations.\",\n \"type\": \"string\"\n },\n \"maxConcurrency\": {\n \"description\": \"Number of concurrent queries that can be run at once.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"maxSamples\": {\n \"description\": \"Maximum number of samples a single query can load into memory. Note that queries will fail if they would load more samples than this into memory, so this also limits the number of samples a query can return.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"timeout\": {\n \"description\": \"Maximum time a query may take before being aborted.\",\n \"type\": \"string\"\n }\n }\n },\n \"remoteRead\": {\n \"description\": \"If specified, the remote_read spec. This is an experimental feature, it may change in any upcoming release in a breaking way.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"RemoteReadSpec defines the remote_read configuration for prometheus.\",\n \"type\": \"object\",\n \"required\": [\n \"url\"\n ],\n \"properties\": {\n \"basicAuth\": {\n \"description\": \"BasicAuth allow an endpoint to authenticate over basic authentication More info: https://prometheus.io/docs/operating/configuration/#endpoints\",\n \"type\": \"object\",\n \"properties\": {\n \"password\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"type\": \"object\",\n \"required\": [\n \"key\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n }\n },\n \"username\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"type\": \"object\",\n \"required\": [\n \"key\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n }\n }\n }\n },\n \"bearerToken\": {\n \"description\": \"bearer token for remote read.\",\n \"type\": \"string\"\n },\n \"bearerTokenFile\": {\n \"description\": \"File to read bearer token for remote read.\",\n \"type\": \"string\"\n },\n \"proxyUrl\": {\n \"description\": \"Optional ProxyURL\",\n \"type\": \"string\"\n },\n \"readRecent\": {\n \"description\": \"Whether reads should be made for queries for time ranges that the local storage should have complete data for.\",\n \"type\": \"boolean\"\n },\n \"remoteTimeout\": {\n \"description\": \"Timeout for requests to the remote read endpoint.\",\n \"type\": \"string\"\n },\n \"requiredMatchers\": {\n \"description\": \"An optional list of equality matchers which have to be present in a selector to query the remote read endpoint.\",\n \"type\": \"object\"\n },\n \"tlsConfig\": {\n \"description\": \"TLSConfig specifies TLS configuration parameters.\",\n \"type\": \"object\",\n \"properties\": {\n \"caFile\": {\n \"description\": \"The CA cert to use for the targets.\",\n \"type\": \"string\"\n },\n \"certFile\": {\n \"description\": \"The client cert file for the targets.\",\n \"type\": \"string\"\n },\n \"insecureSkipVerify\": {\n \"description\": \"Disable target certificate validation.\",\n \"type\": \"boolean\"\n },\n \"keyFile\": {\n \"description\": \"The client key file for the targets.\",\n \"type\": \"string\"\n },\n \"serverName\": {\n \"description\": \"Used to verify the hostname for the targets.\",\n \"type\": \"string\"\n }\n }\n },\n \"url\": {\n \"description\": \"The URL of the endpoint to send samples to.\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"remoteWrite\": {\n \"description\": \"If specified, the remote_write spec. This is an experimental feature, it may change in any upcoming release in a breaking way.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"RemoteWriteSpec defines the remote_write configuration for prometheus.\",\n \"type\": \"object\",\n \"required\": [\n \"url\"\n ],\n \"properties\": {\n \"basicAuth\": {\n \"description\": \"BasicAuth allow an endpoint to authenticate over basic authentication More info: https://prometheus.io/docs/operating/configuration/#endpoints\",\n \"type\": \"object\",\n \"properties\": {\n \"password\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"type\": \"object\",\n \"required\": [\n \"key\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n }\n },\n \"username\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"type\": \"object\",\n \"required\": [\n \"key\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n }\n }\n }\n },\n \"bearerToken\": {\n \"description\": \"File to read bearer token for remote write.\",\n \"type\": \"string\"\n },\n \"bearerTokenFile\": {\n \"description\": \"File to read bearer token for remote write.\",\n \"type\": \"string\"\n },\n \"proxyUrl\": {\n \"description\": \"Optional ProxyURL\",\n \"type\": \"string\"\n },\n \"queueConfig\": {\n \"description\": \"QueueConfig allows the tuning of remote_write queue_config parameters. This object is referenced in the RemoteWriteSpec object.\",\n \"type\": \"object\",\n \"properties\": {\n \"batchSendDeadline\": {\n \"description\": \"BatchSendDeadline is the maximum time a sample will wait in buffer.\",\n \"type\": \"string\"\n },\n \"capacity\": {\n \"description\": \"Capacity is the number of samples to buffer per shard before we start dropping them.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"maxBackoff\": {\n \"description\": \"MaxBackoff is the maximum retry delay.\",\n \"type\": \"string\"\n },\n \"maxRetries\": {\n \"description\": \"MaxRetries is the maximum number of times to retry a batch on recoverable errors.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"maxSamplesPerSend\": {\n \"description\": \"MaxSamplesPerSend is the maximum number of samples per send.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"maxShards\": {\n \"description\": \"MaxShards is the maximum number of shards, i.e. amount of concurrency.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"minBackoff\": {\n \"description\": \"MinBackoff is the initial retry delay. Gets doubled for every retry.\",\n \"type\": \"string\"\n },\n \"minShards\": {\n \"description\": \"MinShards is the minimum number of shards, i.e. amount of concurrency.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n }\n }\n },\n \"remoteTimeout\": {\n \"description\": \"Timeout for requests to the remote write endpoint.\",\n \"type\": \"string\"\n },\n \"tlsConfig\": {\n \"description\": \"TLSConfig specifies TLS configuration parameters.\",\n \"type\": \"object\",\n \"properties\": {\n \"caFile\": {\n \"description\": \"The CA cert to use for the targets.\",\n \"type\": \"string\"\n },\n \"certFile\": {\n \"description\": \"The client cert file for the targets.\",\n \"type\": \"string\"\n },\n \"insecureSkipVerify\": {\n \"description\": \"Disable target certificate validation.\",\n \"type\": \"boolean\"\n },\n \"keyFile\": {\n \"description\": \"The client key file for the targets.\",\n \"type\": \"string\"\n },\n \"serverName\": {\n \"description\": \"Used to verify the hostname for the targets.\",\n \"type\": \"string\"\n }\n }\n },\n \"url\": {\n \"description\": \"The URL of the endpoint to send samples to.\",\n \"type\": \"string\"\n },\n \"writeRelabelConfigs\": {\n \"description\": \"The list of remote write relabel configurations.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"RelabelConfig allows dynamic rewriting of the label set, being applied to samples before ingestion. It defines ``-section of Prometheus configuration. More info: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#metric_relabel_configs\",\n \"type\": \"object\",\n \"properties\": {\n \"action\": {\n \"description\": \"Action to perform based on regex matching. Default is 'replace'\",\n \"type\": \"string\"\n },\n \"modulus\": {\n \"description\": \"Modulus to take of the hash of the source label values.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"regex\": {\n \"description\": \"Regular expression against which the extracted value is matched. default is '(.*)'\",\n \"type\": \"string\"\n },\n \"replacement\": {\n \"description\": \"Replacement value against which a regex replace is performed if the regular expression matches. Regex capture groups are available. Default is '$1'\",\n \"type\": \"string\"\n },\n \"separator\": {\n \"description\": \"Separator placed between concatenated source label values. default is ';'.\",\n \"type\": \"string\"\n },\n \"sourceLabels\": {\n \"description\": \"The source labels select values from existing labels. Their content is concatenated using the configured separator and matched against the configured regular expression for the replace, keep, and drop actions.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"targetLabel\": {\n \"description\": \"Label to which the resulting value is written in a replace action. It is mandatory for replace actions. Regex capture groups are available.\",\n \"type\": \"string\"\n }\n }\n }\n }\n }\n }\n },\n \"replicaExternalLabelName\": {\n \"description\": \"Name of Prometheus external label used to denote replica name. Defaults to the value of `prometheus_replica`. External label will _not_ be added when value is set to empty string (`\\\"\\\"`).\",\n \"type\": \"string\"\n },\n \"replicas\": {\n \"description\": \"Number of instances to deploy for a Prometheus deployment.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"resources\": {\n \"description\": \"ResourceRequirements describes the compute resource requirements.\",\n \"type\": \"object\",\n \"properties\": {\n \"limits\": {\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n }\n },\n \"retention\": {\n \"description\": \"Time duration Prometheus shall retain data for. Default is '24h', and must match the regular expression `[0-9]+(ms|s|m|h|d|w|y)` (milliseconds seconds minutes hours days weeks years).\",\n \"type\": \"string\"\n },\n \"retentionSize\": {\n \"description\": \"Maximum amount of disk space used by blocks.\",\n \"type\": \"string\"\n },\n \"routePrefix\": {\n \"description\": \"The route prefix Prometheus registers HTTP handlers for. This is useful, if using ExternalURL and a proxy is rewriting HTTP routes of a request, and the actual ExternalURL is still true, but the server serves requests under a different route prefix. For example for use with `kubectl proxy`.\",\n \"type\": \"string\"\n },\n \"ruleNamespaceSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"type\": \"object\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"type\": \"object\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"ruleSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"type\": \"object\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"type\": \"object\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"rules\": {\n \"description\": \"/--rules.*/ command-line arguments\",\n \"type\": \"object\",\n \"properties\": {\n \"alert\": {\n \"description\": \"/--rules.alert.*/ command-line arguments\",\n \"type\": \"object\",\n \"properties\": {\n \"forGracePeriod\": {\n \"description\": \"Minimum duration between alert and restored 'for' state. This is maintained only for alerts with configured 'for' time greater than grace period.\",\n \"type\": \"string\"\n },\n \"forOutageTolerance\": {\n \"description\": \"Max time to tolerate prometheus outage for restoring 'for' state of alert.\",\n \"type\": \"string\"\n },\n \"resendDelay\": {\n \"description\": \"Minimum amount of time to wait before resending an alert to Alertmanager.\",\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"scrapeInterval\": {\n \"description\": \"Interval between consecutive scrapes.\",\n \"type\": \"string\"\n },\n \"secrets\": {\n \"description\": \"Secrets is a list of Secrets in the same namespace as the Prometheus object, which shall be mounted into the Prometheus Pods. The Secrets are mounted into /etc/prometheus/secrets/.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"securityContext\": {\n \"description\": \"PodSecurityContext holds pod-level security attributes and common container settings. Some fields are also present in container.securityContext. Field values of container.securityContext take precedence over field values of PodSecurityContext.\",\n \"type\": \"object\",\n \"properties\": {\n \"fsGroup\": {\n \"description\": \"A special supplemental group that applies to all containers in a pod. Some volume types allow the Kubelet to change the ownership of that volume to be owned by the pod:\\n\\n1. The owning GID will be the FSGroup 2. The setgid bit is set (new files created in the volume will be owned by FSGroup) 3. The permission bits are OR'd with rw-rw----\\n\\nIf unset, the Kubelet will not modify the ownership and permissions of any volume.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"runAsGroup\": {\n \"description\": \"The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"runAsNonRoot\": {\n \"description\": \"Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.\",\n \"type\": \"boolean\"\n },\n \"runAsUser\": {\n \"description\": \"The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"seLinuxOptions\": {\n \"description\": \"SELinuxOptions are the labels to be applied to the container\",\n \"type\": \"object\",\n \"properties\": {\n \"level\": {\n \"description\": \"Level is SELinux level label that applies to the container.\",\n \"type\": \"string\"\n },\n \"role\": {\n \"description\": \"Role is a SELinux role label that applies to the container.\",\n \"type\": \"string\"\n },\n \"type\": {\n \"description\": \"Type is a SELinux type label that applies to the container.\",\n \"type\": \"string\"\n },\n \"user\": {\n \"description\": \"User is a SELinux user label that applies to the container.\",\n \"type\": \"string\"\n }\n }\n },\n \"supplementalGroups\": {\n \"description\": \"A list of groups applied to the first process run in each container, in addition to the container's primary GID. If unspecified, no groups will be added to any container.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"integer\",\n \"format\": \"int64\"\n }\n },\n \"sysctls\": {\n \"description\": \"Sysctls hold a list of namespaced sysctls used for the pod. Pods with unsupported sysctls (by the container runtime) might fail to launch.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"Sysctl defines a kernel parameter to be set\",\n \"type\": \"object\",\n \"required\": [\n \"name\",\n \"value\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"Name of a property to set\",\n \"type\": \"string\"\n },\n \"value\": {\n \"description\": \"Value of a property to set\",\n \"type\": \"string\"\n }\n }\n }\n }\n }\n },\n \"serviceAccountName\": {\n \"description\": \"ServiceAccountName is the name of the ServiceAccount to use to run the Prometheus Pods.\",\n \"type\": \"string\"\n },\n \"serviceMonitorNamespaceSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"type\": \"object\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"type\": \"object\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"serviceMonitorSelector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"type\": \"object\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"type\": \"object\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"sha\": {\n \"description\": \"SHA of Prometheus container image to be deployed. Defaults to the value of `version`. Similar to a tag, but the SHA explicitly deploys an immutable container image. Version and Tag are ignored if SHA is set.\",\n \"type\": \"string\"\n },\n \"storage\": {\n \"description\": \"StorageSpec defines the configured storage for a group Prometheus servers. If neither `emptyDir` nor `volumeClaimTemplate` is specified, then by default an [EmptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir) will be used.\",\n \"type\": \"object\",\n \"properties\": {\n \"emptyDir\": {\n \"description\": \"Represents an empty directory for a pod. Empty directory volumes support ownership management and SELinux relabeling.\",\n \"type\": \"object\",\n \"properties\": {\n \"medium\": {\n \"description\": \"What type of storage medium should back this directory. The default is \\\"\\\" which means to use the node's default medium. Must be an empty string (default) or Memory. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir\",\n \"type\": \"string\"\n },\n \"sizeLimit\": {}\n }\n },\n \"volumeClaimTemplate\": {\n \"description\": \"PersistentVolumeClaim is a user's request for and claim to a persistent volume\",\n \"type\": \"object\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"description\": \"ObjectMeta is metadata that all persisted resources must have, which includes all objects users must create.\",\n \"type\": \"object\",\n \"properties\": {\n \"annotations\": {\n \"description\": \"Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations\",\n \"type\": \"object\"\n },\n \"clusterName\": {\n \"description\": \"The name of the cluster which the object belongs to. This is used to distinguish resources with same name and namespace in different clusters. This field is not set anywhere right now and apiserver is going to ignore it if set in create or update request.\",\n \"type\": \"string\"\n },\n \"creationTimestamp\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"type\": \"string\",\n \"format\": \"date-time\"\n },\n \"deletionGracePeriodSeconds\": {\n \"description\": \"Number of seconds allowed for this object to gracefully terminate before it will be removed from the system. Only set when deletionTimestamp is also set. May only be shortened. Read-only.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"deletionTimestamp\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"type\": \"string\",\n \"format\": \"date-time\"\n },\n \"finalizers\": {\n \"description\": \"Must be empty before the object is deleted from the registry. Each entry is an identifier for the responsible component that will remove the entry from the list. If the deletionTimestamp of the object is non-nil, entries in this list can only be removed.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"generateName\": {\n \"description\": \"GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server.\\n\\nIf this field is specified and the generated name exists, the server will NOT return a 409 - instead, it will either return 201 Created or 500 with Reason ServerTimeout indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header).\\n\\nApplied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#idempotency\",\n \"type\": \"string\"\n },\n \"generation\": {\n \"description\": \"A sequence number representing a specific generation of the desired state. Populated by the system. Read-only.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"initializers\": {\n \"description\": \"Initializers tracks the progress of initialization.\",\n \"type\": \"object\",\n \"required\": [\n \"pending\"\n ],\n \"properties\": {\n \"pending\": {\n \"description\": \"Pending is a list of initializers that must execute in order before this object is visible. When the last pending initializer is removed, and no failing result is set, the initializers struct will be set to nil and the object is considered as initialized and visible to all clients.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"Initializer is information about an initializer that has not yet completed.\",\n \"type\": \"object\",\n \"required\": [\n \"name\"\n ],\n \"properties\": {\n \"name\": {\n \"description\": \"name of the process that is responsible for initializing this object.\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"result\": {\n \"description\": \"Status is a return value for calls that don't return other objects.\",\n \"type\": \"object\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources\",\n \"type\": \"string\"\n },\n \"code\": {\n \"description\": \"Suggested HTTP return code for this status, 0 if not set.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"details\": {\n \"description\": \"StatusDetails is a set of additional properties that MAY be set by the server to provide additional information about a response. The Reason field of a Status object defines what attributes will be set. Clients must ignore fields that do not match the defined type of each attribute, and should assume that any attribute may be empty, invalid, or under defined.\",\n \"type\": \"object\",\n \"properties\": {\n \"causes\": {\n \"description\": \"The Causes array includes more details associated with the StatusReason failure. Not all StatusReasons may provide detailed causes.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"StatusCause provides more information about an api.Status failure, including cases when multiple errors are encountered.\",\n \"type\": \"object\",\n \"properties\": {\n \"field\": {\n \"description\": \"The field of the resource that has caused this error, as named by its JSON serialization. May include dot and postfix notation for nested attributes. Arrays are zero-indexed. Fields may appear more than once in an array of causes due to fields having multiple errors. Optional.\\n\\nExamples:\\n \\\"name\\\" - the field \\\"name\\\" on the current resource\\n \\\"items[0].name\\\" - the field \\\"name\\\" on the first array entry in \\\"items\\\"\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"A human-readable description of the cause of the error. This field may be presented as-is to a reader.\",\n \"type\": \"string\"\n },\n \"reason\": {\n \"description\": \"A machine-readable description of the cause of the error. If this value is empty there is no information available.\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"group\": {\n \"description\": \"The group attribute of the resource associated with the status StatusReason.\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"The kind attribute of the resource associated with the status StatusReason. On some operations may differ from the requested resource Kind. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"The name attribute of the resource associated with the status StatusReason (when there is a single name which can be described).\",\n \"type\": \"string\"\n },\n \"retryAfterSeconds\": {\n \"description\": \"If specified, the time in seconds before the operation should be retried. Some errors may indicate the client must take an alternate action - for those errors this field may indicate how long to wait before taking the alternate action.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"uid\": {\n \"description\": \"UID of the resource. (when there is a single resource which can be described). More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n }\n },\n \"kind\": {\n \"description\": \"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"message\": {\n \"description\": \"A human-readable description of the status of this operation.\",\n \"type\": \"string\"\n },\n \"metadata\": {\n \"description\": \"ListMeta describes metadata that synthetic resources must have, including lists and various status objects. A resource may have only one of {ObjectMeta, ListMeta}.\",\n \"type\": \"object\",\n \"properties\": {\n \"continue\": {\n \"description\": \"continue may be set if the user set a limit on the number of items returned, and indicates that the server has more data available. The value is opaque and may be used to issue another request to the endpoint that served this list to retrieve the next set of available objects. Continuing a consistent list may not be possible if the server configuration has changed or more than a few minutes have passed. The resourceVersion field returned when using this continue value will be identical to the value in the first response, unless you have received this token from an error message.\",\n \"type\": \"string\"\n },\n \"resourceVersion\": {\n \"description\": \"String that identifies the server's internal version of this object that can be used by clients to determine when objects have changed. Value must be treated as opaque by clients and passed unmodified back to the server. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\",\n \"type\": \"string\"\n },\n \"selfLink\": {\n \"description\": \"selfLink is a URL representing this object. Populated by the system. Read-only.\",\n \"type\": \"string\"\n }\n }\n },\n \"reason\": {\n \"description\": \"A machine-readable description of why this operation is in the \\\"Failure\\\" status. If this value is empty there is no information available. A Reason clarifies an HTTP status code but does not override it.\",\n \"type\": \"string\"\n },\n \"status\": {\n \"description\": \"Status of the operation. One of: \\\"Success\\\" or \\\"Failure\\\". More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status\",\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"labels\": {\n \"description\": \"Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels\",\n \"type\": \"object\"\n },\n \"managedFields\": {\n \"description\": \"ManagedFields maps workflow-id and version to the set of fields that are managed by that workflow. This is mostly for internal housekeeping, and users typically shouldn't need to set or understand this field. A workflow can be the user's name, a controller's name, or the name of a specific apply path like \\\"ci-cd\\\". The set of fields is always in the version that the workflow used when modifying the object.\\n\\nThis field is alpha and can be changed or removed without notice.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"ManagedFieldsEntry is a workflow-id, a FieldSet and the group version of the resource that the fieldset applies to.\",\n \"type\": \"object\",\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"APIVersion defines the version of this resource that this field set applies to. The format is \\\"group/version\\\" just like the top-level APIVersion field. It is necessary to track the version of a field set because it cannot be automatically converted.\",\n \"type\": \"string\"\n },\n \"fields\": {\n \"description\": \"Fields stores a set of fields in a data structure like a Trie. To understand how this is used, see: https://github.com/kubernetes-sigs/structured-merge-diff\",\n \"type\": \"object\"\n },\n \"manager\": {\n \"description\": \"Manager is an identifier of the workflow managing these fields.\",\n \"type\": \"string\"\n },\n \"operation\": {\n \"description\": \"Operation is the type of operation which lead to this ManagedFieldsEntry being created. The only valid values for this field are 'Apply' and 'Update'.\",\n \"type\": \"string\"\n },\n \"time\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"type\": \"string\",\n \"format\": \"date-time\"\n }\n }\n }\n },\n \"name\": {\n \"description\": \"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names\",\n \"type\": \"string\"\n },\n \"namespace\": {\n \"description\": \"Namespace defines the space within each name must be unique. An empty namespace is equivalent to the \\\"default\\\" namespace, but \\\"default\\\" is the canonical representation. Not all objects are required to be scoped to a namespace - the value of this field for those objects will be empty.\\n\\nMust be a DNS_LABEL. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/namespaces\",\n \"type\": \"string\"\n },\n \"ownerReferences\": {\n \"description\": \"List of objects depended by this object. If ALL objects in the list have been deleted, this object will be garbage collected. If this object is managed by a controller, then an entry in this list will point to this controller, with the controller field set to true. There cannot be more than one managing controller.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"OwnerReference contains enough information to let you identify an owning object. An owning object must be in the same namespace as the dependent, or be cluster-scoped, so there is no namespace field.\",\n \"type\": \"object\",\n \"required\": [\n \"apiVersion\",\n \"kind\",\n \"name\",\n \"uid\"\n ],\n \"properties\": {\n \"apiVersion\": {\n \"description\": \"API version of the referent.\",\n \"type\": \"string\"\n },\n \"blockOwnerDeletion\": {\n \"description\": \"If true, AND if the owner has the \\\"foregroundDeletion\\\" finalizer, then the owner cannot be deleted from the key-value store until this reference is removed. Defaults to false. To set this field, a user needs \\\"delete\\\" permission of the owner, otherwise 422 (Unprocessable Entity) will be returned.\",\n \"type\": \"boolean\"\n },\n \"controller\": {\n \"description\": \"If true, this reference points to the managing controller.\",\n \"type\": \"boolean\"\n },\n \"kind\": {\n \"description\": \"Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#names\",\n \"type\": \"string\"\n },\n \"uid\": {\n \"description\": \"UID of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"resourceVersion\": {\n \"description\": \"An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources.\\n\\nPopulated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency\",\n \"type\": \"string\"\n },\n \"selfLink\": {\n \"description\": \"SelfLink is a URL representing this object. Populated by the system. Read-only.\",\n \"type\": \"string\"\n },\n \"uid\": {\n \"description\": \"UID is the unique in time and space value for this object. It is typically generated by the server on successful creation of a resource and is not allowed to change on PUT operations.\\n\\nPopulated by the system. Read-only. More info: http://kubernetes.io/docs/user-guide/identifiers#uids\",\n \"type\": \"string\"\n }\n }\n },\n \"spec\": {\n \"description\": \"PersistentVolumeClaimSpec describes the common attributes of storage devices and allows a Source for provider-specific attributes\",\n \"type\": \"object\",\n \"properties\": {\n \"accessModes\": {\n \"description\": \"AccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"dataSource\": {\n \"description\": \"TypedLocalObjectReference contains enough information to let you locate the typed referenced object inside the same namespace.\",\n \"type\": \"object\",\n \"required\": [\n \"kind\",\n \"name\"\n ],\n \"properties\": {\n \"apiGroup\": {\n \"description\": \"APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.\",\n \"type\": \"string\"\n },\n \"kind\": {\n \"description\": \"Kind is the type of resource being referenced\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name is the name of resource being referenced\",\n \"type\": \"string\"\n }\n }\n },\n \"resources\": {\n \"description\": \"ResourceRequirements describes the compute resource requirements.\",\n \"type\": \"object\",\n \"properties\": {\n \"limits\": {\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n }\n },\n \"selector\": {\n \"description\": \"A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.\",\n \"type\": \"object\",\n \"properties\": {\n \"matchExpressions\": {\n \"description\": \"matchExpressions is a list of label selector requirements. The requirements are ANDed.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.\",\n \"type\": \"object\",\n \"required\": [\n \"key\",\n \"operator\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"key is the label key that the selector applies to.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.\",\n \"type\": \"string\"\n },\n \"values\": {\n \"description\": \"values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n }\n }\n }\n },\n \"matchLabels\": {\n \"description\": \"matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is \\\"key\\\", the operator is \\\"In\\\", and the values array contains only \\\"value\\\". The requirements are ANDed.\",\n \"type\": \"object\"\n }\n }\n },\n \"storageClassName\": {\n \"description\": \"Name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1\",\n \"type\": \"string\"\n },\n \"volumeMode\": {\n \"description\": \"volumeMode defines what type of volume is required by the claim. Value of Filesystem is implied when not included in claim spec. This is a beta feature.\",\n \"type\": \"string\"\n },\n \"volumeName\": {\n \"description\": \"VolumeName is the binding reference to the PersistentVolume backing this claim.\",\n \"type\": \"string\"\n }\n }\n },\n \"status\": {\n \"description\": \"PersistentVolumeClaimStatus is the current status of a persistent volume claim.\",\n \"type\": \"object\",\n \"properties\": {\n \"accessModes\": {\n \"description\": \"AccessModes contains the actual access modes the volume backing the PVC has. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"string\"\n }\n },\n \"capacity\": {\n \"description\": \"Represents the actual resources of the underlying volume.\",\n \"type\": \"object\"\n },\n \"conditions\": {\n \"description\": \"Current Condition of persistent volume claim. If underlying persistent volume is being resized then the Condition will be set to 'ResizeStarted'.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"PersistentVolumeClaimCondition contains details about state of pvc\",\n \"type\": \"object\",\n \"required\": [\n \"type\",\n \"status\"\n ],\n \"properties\": {\n \"lastProbeTime\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"type\": \"string\",\n \"format\": \"date-time\"\n },\n \"lastTransitionTime\": {\n \"description\": \"Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.\",\n \"type\": \"string\",\n \"format\": \"date-time\"\n },\n \"message\": {\n \"description\": \"Human-readable message indicating details about last transition.\",\n \"type\": \"string\"\n },\n \"reason\": {\n \"description\": \"Unique, this should be a short, machine understandable string that gives the reason for condition's last transition. If it reports \\\"ResizeStarted\\\" that means the underlying persistent volume is being resized.\",\n \"type\": \"string\"\n },\n \"status\": {\n \"type\": \"string\"\n },\n \"type\": {\n \"type\": \"string\"\n }\n }\n }\n },\n \"phase\": {\n \"description\": \"Phase represents the current phase of PersistentVolumeClaim.\",\n \"type\": \"string\"\n }\n }\n }\n }\n }\n }\n },\n \"tag\": {\n \"description\": \"Tag of Prometheus container image to be deployed. Defaults to the value of `version`. Version is ignored if Tag is set.\",\n \"type\": \"string\"\n },\n \"thanos\": {\n \"description\": \"ThanosSpec defines parameters for a Prometheus server within a Thanos deployment.\",\n \"type\": \"object\",\n \"properties\": {\n \"baseImage\": {\n \"description\": \"Thanos base image if other than default.\",\n \"type\": \"string\"\n },\n \"image\": {\n \"description\": \"Image if specified has precedence over baseImage, tag and sha combinations. Specifying the version is still necessary to ensure the Prometheus Operator knows what version of Thanos is being configured.\",\n \"type\": \"string\"\n },\n \"objectStorageConfig\": {\n \"description\": \"SecretKeySelector selects a key of a Secret.\",\n \"type\": \"object\",\n \"required\": [\n \"key\"\n ],\n \"properties\": {\n \"key\": {\n \"description\": \"The key of the secret to select from. Must be a valid secret key.\",\n \"type\": \"string\"\n },\n \"name\": {\n \"description\": \"Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names\",\n \"type\": \"string\"\n },\n \"optional\": {\n \"description\": \"Specify whether the Secret or it's key must be defined\",\n \"type\": \"boolean\"\n }\n }\n },\n \"resources\": {\n \"description\": \"ResourceRequirements describes the compute resource requirements.\",\n \"type\": \"object\",\n \"properties\": {\n \"limits\": {\n \"description\": \"Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n },\n \"requests\": {\n \"description\": \"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/\",\n \"type\": \"object\"\n }\n }\n },\n \"sha\": {\n \"description\": \"SHA of Thanos container image to be deployed. Defaults to the value of `version`. Similar to a tag, but the SHA explicitly deploys an immutable container image. Version and Tag are ignored if SHA is set.\",\n \"type\": \"string\"\n },\n \"tag\": {\n \"description\": \"Tag of Thanos sidecar container image to be deployed. Defaults to the value of `version`. Version is ignored if Tag is set.\",\n \"type\": \"string\"\n },\n \"version\": {\n \"description\": \"Version describes the version of Thanos to use.\",\n \"type\": \"string\"\n }\n }\n },\n \"tolerations\": {\n \"description\": \"If specified, the pod's tolerations.\",\n \"type\": \"array\",\n \"items\": {\n \"description\": \"The pod this Toleration is attached to tolerates any taint that matches the triple using the matching operator .\",\n \"type\": \"object\",\n \"properties\": {\n \"effect\": {\n \"description\": \"Effect indicates the taint effect to match. Empty means match all taint effects. When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.\",\n \"type\": \"string\"\n },\n \"key\": {\n \"description\": \"Key is the taint key that the toleration applies to. Empty means match all taint keys. If the key is empty, operator must be Exists; this combination means to match all values and all keys.\",\n \"type\": \"string\"\n },\n \"operator\": {\n \"description\": \"Operator represents a key's relationship to the value. Valid operators are Exists and Equal. Defaults to Equal. Exists is equivalent to wildcard for value, so that a pod can tolerate all taints of a particular category.\",\n \"type\": \"string\"\n },\n \"tolerationSeconds\": {\n \"description\": \"TolerationSeconds represents the period of time the toleration (which must be of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately) by the system.\",\n \"type\": \"integer\",\n \"format\": \"int64\"\n },\n \"value\": {\n \"description\": \"Value is the taint value the toleration matches to. If the operator is Exists, the value should be empty, otherwise just a regular string.\",\n \"type\": \"string\"\n }\n }\n }\n },\n \"version\": {\n \"description\": \"Version of Prometheus to be deployed.\",\n \"type\": \"string\"\n },\n \"walCompression\": {\n \"description\": \"Enable compression of the write-ahead log using Snappy.\",\n \"type\": \"boolean\"\n }\n }\n },\n \"status\": {\n \"description\": \"PrometheusStatus is the most recent observed status of the Prometheus cluster. Read-only. Not included when requesting from the apiserver, only from the Prometheus Operator API itself. More info: https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#spec-and-status\",\n \"type\": \"object\",\n \"required\": [\n \"paused\",\n \"replicas\",\n \"updatedReplicas\",\n \"availableReplicas\",\n \"unavailableReplicas\"\n ],\n \"properties\": {\n \"availableReplicas\": {\n \"description\": \"Total number of available pods (ready for at least minReadySeconds) targeted by this Prometheus deployment.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"paused\": {\n \"description\": \"Represents whether any actions on the underlying managed objects are being performed. Only delete actions will be performed.\",\n \"type\": \"boolean\"\n },\n \"replicas\": {\n \"description\": \"Total number of non-terminated pods targeted by this Prometheus deployment (their labels match the selector).\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"unavailableReplicas\": {\n \"description\": \"Total number of unavailable pods targeted by this Prometheus deployment.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n },\n \"updatedReplicas\": {\n \"description\": \"Total number of non-terminated pods targeted by this Prometheus deployment that have the desired version spec.\",\n \"type\": \"integer\",\n \"format\": \"int32\"\n }\n }\n }\n }\n }\n },\n \"versions\": [\n {\n \"name\": \"v1\",\n \"served\": true,\n \"storage\": true\n }\n ],\n \"conversion\": {\n \"strategy\": \"None\"\n },\n \"preserveUnknownFields\": true\n },\n \"status\": {\n \"conditions\": [\n {\n \"type\": \"NamesAccepted\",\n \"status\": \"True\",\n \"lastTransitionTime\": \"2020-05-05T16:58:10Z\",\n \"reason\": \"NoConflicts\",\n \"message\": \"no conflicts found\"\n },\n {\n \"type\": \"Established\",\n \"status\": \"True\",\n \"lastTransitionTime\": \"2020-05-05T16:58:10Z\",\n \"reason\": \"InitialNamesAccepted\",\n \"message\": \"the initial names have been accepted\"\n },\n {\n \"type\": \"NonStructuralSchema\",\n \"status\": \"True\",\n \"lastTransitionTime\": \"2020-05-05T16:58:10Z\",\n \"reason\": \"Violations\",\n \"message\": \"[spec.validation.openAPIV3Schema.properties[spec].properties[alerting].properties[alertmanagers].items.properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[alerting].properties[alertmanagers].items.properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[alerting].properties[alertmanagers].items.properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[env].items.properties[valueFrom].properties[resourceFieldRef].properties[divisor].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[httpGet].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[httpGet].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[httpGet].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[tcpSocket].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[tcpSocket].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[postStart].properties[tcpSocket].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[httpGet].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[httpGet].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[httpGet].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[tcpSocket].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[tcpSocket].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[lifecycle].properties[preStop].properties[tcpSocket].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[httpGet].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[httpGet].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[httpGet].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[tcpSocket].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[tcpSocket].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[livenessProbe].properties[tcpSocket].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[httpGet].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[httpGet].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[httpGet].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[tcpSocket].properties[port].anyOf[0].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[tcpSocket].properties[port].anyOf[1].type: Forbidden: must be empty to be structural, spec.validation.openAPIV3Schema.properties[spec].properties[containers].items.properties[readinessProbe].properties[tcpSocket].properties[port].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[storage].properties[emptyDir].properties[sizeLimit].type: Required value: must not be empty for specified object fields]\"\n }\n ],\n \"acceptedNames\": {\n \"plural\": \"prometheuses\",\n \"singular\": \"prometheus\",\n \"kind\": \"Prometheus\",\n \"listKind\": \"PrometheusList\"\n },\n \"storedVersions\": [\n \"v1\"\n ]\n }\n}\n" }, { "path": "pkg/backup/backed_up_items_map.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"fmt\"\n\t\"sort\"\n\t\"sync\"\n)\n\n// backedUpItemsMap keeps track of the items already backed up for the current Velero Backup\ntype backedUpItemsMap struct {\n\t*sync.RWMutex\n\tbackedUpItems map[itemKey]struct{}\n\ttotalItems map[itemKey]struct{}\n}\n\nfunc NewBackedUpItemsMap() *backedUpItemsMap {\n\treturn &backedUpItemsMap{\n\t\tRWMutex: &sync.RWMutex{},\n\t\tbackedUpItems: make(map[itemKey]struct{}),\n\t\ttotalItems: make(map[itemKey]struct{}),\n\t}\n}\n\nfunc (m *backedUpItemsMap) CopyItemMap() map[itemKey]struct{} {\n\tm.RLock()\n\tdefer m.RUnlock()\n\treturnMap := make(map[itemKey]struct{}, len(m.backedUpItems))\n\tfor key, val := range m.backedUpItems {\n\t\treturnMap[key] = val\n\t}\n\treturn returnMap\n}\n\n// ResourceMap returns a map of the backed up items.\n// For each map entry, the key is the resource type,\n// and the value is a list of namespaced names for the resource.\nfunc (m *backedUpItemsMap) ResourceMap() map[string][]string {\n\tm.RLock()\n\tdefer m.RUnlock()\n\n\tresources := map[string][]string{}\n\tfor i := range m.backedUpItems {\n\t\tentry := i.name\n\t\tif i.namespace != \"\" {\n\t\t\tentry = fmt.Sprintf(\"%s/%s\", i.namespace, i.name)\n\t\t}\n\t\tresources[i.resource] = append(resources[i.resource], entry)\n\t}\n\n\t// sort namespace/name entries for each GVK\n\tfor _, v := range resources {\n\t\tsort.Strings(v)\n\t}\n\n\treturn resources\n}\n\nfunc (m *backedUpItemsMap) Len() int {\n\tm.RLock()\n\tdefer m.RUnlock()\n\treturn len(m.backedUpItems)\n}\n\nfunc (m *backedUpItemsMap) BackedUpAndTotalLen() (int, int) {\n\tm.RLock()\n\tdefer m.RUnlock()\n\treturn len(m.backedUpItems), len(m.totalItems)\n}\n\nfunc (m *backedUpItemsMap) Has(key itemKey) bool {\n\tm.RLock()\n\tdefer m.RUnlock()\n\n\t_, exists := m.backedUpItems[key]\n\treturn exists\n}\n\nfunc (m *backedUpItemsMap) AddItem(key itemKey) {\n\tm.Lock()\n\tdefer m.Unlock()\n\tm.backedUpItems[key] = struct{}{}\n\tm.totalItems[key] = struct{}{}\n}\n\nfunc (m *backedUpItemsMap) DeleteItem(key itemKey) {\n\tm.Lock()\n\tdefer m.Unlock()\n\n\tdelete(m.backedUpItems, key)\n\tdelete(m.totalItems, key)\n}\n\nfunc (m *backedUpItemsMap) AddItemToTotal(key itemKey) {\n\tm.Lock()\n\tdefer m.Unlock()\n\tm.totalItems[key] = struct{}{}\n}\n" }, { "path": "pkg/backup/backup.go", "content": "/*\nCopyright the Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"archive/tar\"\n\t\"bytes\"\n\t\"compress/gzip\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\t\"k8s.io/apimachinery/pkg/api/meta\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\tkubeerrs \"k8s.io/apimachinery/pkg/util/errors\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/hook\"\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\t\"github.com/vmware-tanzu/velero/internal/volumehelper\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/discovery\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemblock\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/persistence\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tbiav2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/backupitemaction/v2\"\n\tibav1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/itemblockaction/v1\"\n\tvsv1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/volumesnapshotter/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/podexec\"\n\t\"github.com/vmware-tanzu/velero/pkg/podvolume\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/collections\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\n// BackupVersion is the current backup major version for Velero.\n// Deprecated, use BackupFormatVersion\nconst BackupVersion = 1\n\n// BackupFormatVersion is the current backup version for Velero, including major, minor, and patch.\nconst BackupFormatVersion = \"1.1.0\"\n\n// ArgoCD managed by namespace label key\nconst ArgoCDManagedByNamespaceLabel = \"argocd.argoproj.io/managed-by\"\n\n// Backupper performs backups.\ntype Backupper interface {\n\t// Backup takes a backup using the specification in the velerov1api.Backup and writes backup and log data\n\t// to the given writers.\n\tBackup(\n\t\tlogger logrus.FieldLogger,\n\t\tbackup *Request,\n\t\tbackupFile io.Writer,\n\t\tactions []biav2.BackupItemAction,\n\t\titemBlockActions []ibav1.ItemBlockAction,\n\t\tvolumeSnapshotterGetter VolumeSnapshotterGetter,\n\t) error\n\n\tBackupWithResolvers(\n\t\tlog logrus.FieldLogger,\n\t\tbackupRequest *Request,\n\t\tbackupFile io.Writer,\n\t\tbackupItemActionResolver framework.BackupItemActionResolverV2,\n\t\titemBlockActionResolver framework.ItemBlockActionResolver,\n\t\tvolumeSnapshotterGetter VolumeSnapshotterGetter,\n\t) error\n\n\tFinalizeBackup(\n\t\tlog logrus.FieldLogger,\n\t\tbackupRequest *Request,\n\t\tinBackupFile io.Reader,\n\t\toutBackupFile io.Writer,\n\t\tbackupItemActionResolver framework.BackupItemActionResolverV2,\n\t\tasyncBIAOperations []*itemoperation.BackupOperation,\n\t\tbackupStore persistence.BackupStore,\n\t) error\n}\n\n// kubernetesBackupper implements Backupper.\ntype kubernetesBackupper struct {\n\tkbClient kbclient.Client\n\tdynamicFactory client.DynamicFactory\n\tdiscoveryHelper discovery.Helper\n\tpodCommandExecutor podexec.PodCommandExecutor\n\tpodVolumeBackupperFactory podvolume.BackupperFactory\n\tpodVolumeTimeout time.Duration\n\tdefaultVolumesToFsBackup bool\n\tclientPageSize int\n\tuploaderType string\n\tpluginManager func(logrus.FieldLogger) clientmgmt.Manager\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter\n}\n\nfunc (i *itemKey) String() string {\n\treturn fmt.Sprintf(\"resource=%s,namespace=%s,name=%s\", i.resource, i.namespace, i.name)\n}\n\nfunc cohabitatingResources() map[string]*cohabitatingResource {\n\treturn map[string]*cohabitatingResource{\n\t\t\"deployments\": newCohabitatingResource(\"deployments\", \"extensions\", \"apps\"),\n\t\t\"daemonsets\": newCohabitatingResource(\"daemonsets\", \"extensions\", \"apps\"),\n\t\t\"replicasets\": newCohabitatingResource(\"replicasets\", \"extensions\", \"apps\"),\n\t\t\"networkpolicies\": newCohabitatingResource(\"networkpolicies\", \"extensions\", \"networking.k8s.io\"),\n\t\t\"events\": newCohabitatingResource(\"events\", \"\", \"events.k8s.io\"),\n\t}\n}\n\n// NewKubernetesBackupper creates a new kubernetesBackupper.\nfunc NewKubernetesBackupper(\n\tkbClient kbclient.Client,\n\tdiscoveryHelper discovery.Helper,\n\tdynamicFactory client.DynamicFactory,\n\tpodCommandExecutor podexec.PodCommandExecutor,\n\tpodVolumeBackupperFactory podvolume.BackupperFactory,\n\tpodVolumeTimeout time.Duration,\n\tdefaultVolumesToFsBackup bool,\n\tclientPageSize int,\n\tuploaderType string,\n\tpluginManager func(logrus.FieldLogger) clientmgmt.Manager,\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter,\n) (Backupper, error) {\n\treturn &kubernetesBackupper{\n\t\tkbClient: kbClient,\n\t\tdiscoveryHelper: discoveryHelper,\n\t\tdynamicFactory: dynamicFactory,\n\t\tpodCommandExecutor: podCommandExecutor,\n\t\tpodVolumeBackupperFactory: podVolumeBackupperFactory,\n\t\tpodVolumeTimeout: podVolumeTimeout,\n\t\tdefaultVolumesToFsBackup: defaultVolumesToFsBackup,\n\t\tclientPageSize: clientPageSize,\n\t\tuploaderType: uploaderType,\n\t\tpluginManager: pluginManager,\n\t\tbackupStoreGetter: backupStoreGetter,\n\t}, nil\n}\n\n// getNamespaceIncludesExcludesAndArgoCDNamespaces returns an IncludesExcludes list containing which namespaces to\n// include and exclude from the backup and a list of namespaces managed by ArgoCD.\nfunc getNamespaceIncludesExcludesAndArgoCDNamespaces(backup *velerov1api.Backup, kbClient kbclient.Client) (*collections.NamespaceIncludesExcludes, []string, error) {\n\tnsList := corev1api.NamespaceList{}\n\tactiveNamespaces := []string{}\n\tnsManagedByArgoCD := []string{}\n\tif err := kbClient.List(context.Background(), &nsList); err != nil {\n\t\treturn nil, nsManagedByArgoCD, err\n\t}\n\tfor _, ns := range nsList.Items {\n\t\tactiveNamespaces = append(activeNamespaces, ns.Name)\n\t}\n\n\t// Set ActiveNamespaces first, then set includes/excludes\n\tincludesExcludes := collections.NewNamespaceIncludesExcludes().\n\t\tActiveNamespaces(activeNamespaces).\n\t\tIncludes(backup.Spec.IncludedNamespaces...).\n\t\tExcludes(backup.Spec.ExcludedNamespaces...)\n\n\t// Expand wildcards if needed\n\tif err := includesExcludes.ExpandIncludesExcludes(); err != nil {\n\t\treturn nil, []string{}, err\n\t}\n\n\t// Check for ArgoCD managed namespaces in the namespaces that will be included\n\tfor _, ns := range nsList.Items {\n\t\tnsLabels := ns.GetLabels()\n\t\tif len(nsLabels[ArgoCDManagedByNamespaceLabel]) > 0 && includesExcludes.ShouldInclude(ns.Name) {\n\t\t\tnsManagedByArgoCD = append(nsManagedByArgoCD, ns.Name)\n\t\t}\n\t}\n\n\treturn includesExcludes, nsManagedByArgoCD, nil\n}\n\nfunc getResourceHooks(hookSpecs []velerov1api.BackupResourceHookSpec, discoveryHelper discovery.Helper) ([]hook.ResourceHook, error) {\n\tresourceHooks := make([]hook.ResourceHook, 0, len(hookSpecs))\n\n\tfor _, s := range hookSpecs {\n\t\th, err := getResourceHook(s, discoveryHelper)\n\t\tif err != nil {\n\t\t\treturn []hook.ResourceHook{}, err\n\t\t}\n\n\t\tresourceHooks = append(resourceHooks, h)\n\t}\n\n\treturn resourceHooks, nil\n}\n\nfunc getResourceHook(hookSpec velerov1api.BackupResourceHookSpec, discoveryHelper discovery.Helper) (hook.ResourceHook, error) {\n\th := hook.ResourceHook{\n\t\tName: hookSpec.Name,\n\t\tSelector: hook.ResourceHookSelector{\n\t\t\tNamespaces: collections.NewIncludesExcludes().Includes(hookSpec.IncludedNamespaces...).Excludes(hookSpec.ExcludedNamespaces...),\n\t\t\tResources: collections.GetResourceIncludesExcludes(discoveryHelper, hookSpec.IncludedResources, hookSpec.ExcludedResources),\n\t\t},\n\t\tPre: hookSpec.PreHooks,\n\t\tPost: hookSpec.PostHooks,\n\t}\n\n\tif hookSpec.LabelSelector != nil {\n\t\tlabelSelector, err := metav1.LabelSelectorAsSelector(hookSpec.LabelSelector)\n\t\tif err != nil {\n\t\t\treturn hook.ResourceHook{}, errors.WithStack(err)\n\t\t}\n\t\th.Selector.LabelSelector = labelSelector\n\t}\n\n\treturn h, nil\n}\n\ntype VolumeSnapshotterGetter interface {\n\tGetVolumeSnapshotter(name string) (vsv1.VolumeSnapshotter, error)\n}\n\n// Backup backs up the items specified in the Backup, placing them in a gzip-compressed tar file\n// written to backupFile. The finalized velerov1api.Backup is written to metadata. Any error that represents\n// a complete backup failure is returned. Errors that constitute partial failures (i.e. failures to\n// back up individual resources that don't prevent the backup from continuing to be processed) are logged\n// to the backup log.\nfunc (kb *kubernetesBackupper) Backup(log logrus.FieldLogger, backupRequest *Request, backupFile io.Writer,\n\tactions []biav2.BackupItemAction, itemBlockActions []ibav1.ItemBlockAction, volumeSnapshotterGetter VolumeSnapshotterGetter) error {\n\tbackupItemActions := framework.NewBackupItemActionResolverV2(actions)\n\titemBlockActionResolver := framework.NewItemBlockActionResolver(itemBlockActions)\n\treturn kb.BackupWithResolvers(log, backupRequest, backupFile, backupItemActions, itemBlockActionResolver, volumeSnapshotterGetter)\n}\n\nfunc (kb *kubernetesBackupper) BackupWithResolvers(\n\tlog logrus.FieldLogger,\n\tbackupRequest *Request,\n\tbackupFile io.Writer,\n\tbackupItemActionResolver framework.BackupItemActionResolverV2,\n\titemBlockActionResolver framework.ItemBlockActionResolver,\n\tvolumeSnapshotterGetter VolumeSnapshotterGetter,\n) error {\n\tgzippedData := gzip.NewWriter(backupFile)\n\tdefer gzippedData.Close()\n\n\ttw := NewTarWriter(tar.NewWriter(gzippedData))\n\tdefer tw.Close()\n\n\tlog.Info(\"Writing backup version file\")\n\tif err := kb.writeBackupVersion(tw); err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\tvar err error\n\tvar nsManagedByArgoCD []string\n\tbackupRequest.NamespaceIncludesExcludes, nsManagedByArgoCD, err = getNamespaceIncludesExcludesAndArgoCDNamespaces(backupRequest.Backup, kb.kbClient)\n\tif err != nil {\n\t\tlog.WithError(err).Errorf(\"error getting namespace includes/excludes\")\n\t\treturn err\n\t}\n\n\tif backupRequest.NamespaceIncludesExcludes.IsWildcardExpanded() {\n\t\texpandedIncludes := backupRequest.NamespaceIncludesExcludes.GetIncludes()\n\t\texpandedExcludes := backupRequest.NamespaceIncludesExcludes.GetExcludes()\n\n\t\t// Get the final namespace list after wildcard expansion\n\t\twildcardResult, err := backupRequest.NamespaceIncludesExcludes.ResolveNamespaceList()\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Errorf(\"error resolving namespace list\")\n\t\t\treturn err\n\t\t}\n\n\t\tlog.WithFields(logrus.Fields{\n\t\t\t\"expandedIncludes\": expandedIncludes,\n\t\t\t\"expandedExcludes\": expandedExcludes,\n\t\t\t\"wildcardResult\": wildcardResult,\n\t\t\t\"includedCount\": len(expandedIncludes),\n\t\t\t\"excludedCount\": len(expandedExcludes),\n\t\t\t\"resultCount\": len(wildcardResult),\n\t\t}).Info(\"Successfully expanded wildcard patterns\")\n\t}\n\n\tlog.Infof(\"Including namespaces: %s\", backupRequest.NamespaceIncludesExcludes.IncludesString())\n\tlog.Infof(\"Excluding namespaces: %s\", backupRequest.NamespaceIncludesExcludes.ExcludesString())\n\n\t// check if there are any namespaces included in the backup which are managed by argoCD\n\t// We will check for the existence of a ArgoCD label in the includedNamespaces and add a warning\n\t// so that users are at least aware about the existence of argoCD managed ns in their backup\n\t// Related Issue: https://github.com/vmware-tanzu/velero/issues/7905\n\tif len(nsManagedByArgoCD) > 0 {\n\t\tlog.Warnf(\"backup operation may encounter complications and potentially produce undesirable results due to the inclusion of namespaces %v managed by ArgoCD in the backup.\", nsManagedByArgoCD)\n\t}\n\n\tif collections.UseOldResourceFilters(backupRequest.Spec) {\n\t\tbackupRequest.ResourceIncludesExcludes = collections.GetGlobalResourceIncludesExcludes(kb.discoveryHelper, log,\n\t\t\tbackupRequest.Spec.IncludedResources,\n\t\t\tbackupRequest.Spec.ExcludedResources,\n\t\t\tbackupRequest.Spec.IncludeClusterResources,\n\t\t\t*backupRequest.NamespaceIncludesExcludes)\n\t} else {\n\t\tsrie := collections.GetScopeResourceIncludesExcludes(kb.discoveryHelper, log,\n\t\t\tbackupRequest.Spec.IncludedNamespaceScopedResources,\n\t\t\tbackupRequest.Spec.ExcludedNamespaceScopedResources,\n\t\t\tbackupRequest.Spec.IncludedClusterScopedResources,\n\t\t\tbackupRequest.Spec.ExcludedClusterScopedResources,\n\t\t\t*backupRequest.NamespaceIncludesExcludes,\n\t\t)\n\t\tif backupRequest.ResPolicies != nil {\n\t\t\tsrie.CombineWithPolicy(backupRequest.ResPolicies.GetIncludeExcludePolicy())\n\t\t}\n\t\tbackupRequest.ResourceIncludesExcludes = srie\n\t}\n\n\tlog.Infof(\"Backing up all volumes using pod volume backup: %t\", boolptr.IsSetToTrue(backupRequest.Backup.Spec.DefaultVolumesToFsBackup))\n\n\tbackupRequest.ResourceHooks, err = getResourceHooks(backupRequest.Spec.Hooks.Resources, kb.discoveryHelper)\n\tif err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Debugf(\"Error from getResourceHooks\")\n\t\treturn err\n\t}\n\n\tbackupRequest.ResolvedActions, err = backupItemActionResolver.ResolveActions(kb.discoveryHelper, log)\n\tif err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Debugf(\"Error from backupItemActionResolver.ResolveActions\")\n\t\treturn err\n\t}\n\n\tbackupRequest.ResolvedItemBlockActions, err = itemBlockActionResolver.ResolveActions(kb.discoveryHelper, log)\n\tif err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Errorf(\"Error from itemBlockActionResolver.ResolveActions\")\n\t\treturn err\n\t}\n\n\tpodVolumeTimeout := kb.podVolumeTimeout\n\tif val := backupRequest.Annotations[velerov1api.PodVolumeOperationTimeoutAnnotation]; val != \"\" {\n\t\tparsed, err := time.ParseDuration(val)\n\t\tif err != nil {\n\t\t\tlog.WithError(errors.WithStack(err)).Errorf(\"Unable to parse pod volume timeout annotation %s, using server value.\", val)\n\t\t} else {\n\t\t\tpodVolumeTimeout = parsed\n\t\t}\n\t}\n\n\tvar podVolumeCancelFunc context.CancelFunc\n\tpodVolumeContext, podVolumeCancelFunc := context.WithTimeout(context.Background(), podVolumeTimeout)\n\tdefer podVolumeCancelFunc()\n\n\tvar podVolumeBackupper podvolume.Backupper\n\tif kb.podVolumeBackupperFactory != nil {\n\t\tpodVolumeBackupper, err = kb.podVolumeBackupperFactory.NewBackupper(podVolumeContext, log, backupRequest.Backup, kb.uploaderType)\n\t\tif err != nil {\n\t\t\tlog.WithError(errors.WithStack(err)).Debugf(\"Error from NewBackupper\")\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\t}\n\n\t// set up a temp dir for the itemCollector to use to temporarily\n\t// store items as they're scraped from the API.\n\ttempDir, err := os.MkdirTemp(\"\", \"\")\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error creating temp dir for backup\")\n\t}\n\tdefer os.RemoveAll(tempDir)\n\n\tcollector := &itemCollector{\n\t\tlog: log,\n\t\tbackupRequest: backupRequest,\n\t\tdiscoveryHelper: kb.discoveryHelper,\n\t\tdynamicFactory: kb.dynamicFactory,\n\t\tcohabitatingResources: cohabitatingResources(),\n\t\tdir: tempDir,\n\t\tpageSize: kb.clientPageSize,\n\t}\n\n\titems := collector.getAllItems()\n\tlog.WithField(\"progress\", \"\").Infof(\"Collected %d items matching the backup spec from the Kubernetes API (actual number of items backed up may be more or less depending on velero.io/exclude-from-backup annotation, plugins returning additional related items to back up, etc.)\", len(items))\n\n\tupdated := backupRequest.Backup.DeepCopy()\n\tif updated.Status.Progress == nil {\n\t\tupdated.Status.Progress = &velerov1api.BackupProgress{}\n\t}\n\n\tupdated.Status.Progress.TotalItems = len(items)\n\tif err := kube.PatchResource(backupRequest.Backup, updated, kb.kbClient); err != nil {\n\t\tlog.WithError(errors.WithStack((err))).Warn(\"Got error trying to update backup's status.progress.totalItems\")\n\t}\n\tbackupRequest.Status.Progress = &velerov1api.BackupProgress{TotalItems: len(items)}\n\n\t// Resolve namespaces for PVC-to-Pod cache building in volumehelper.\n\t// See issue #9179 for details.\n\tnamespaces, err := backupRequest.NamespaceIncludesExcludes.ResolveNamespaceList()\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"Failed to resolve namespace list for PVC-to-Pod cache\")\n\t\treturn err\n\t}\n\n\tvolumeHelperImpl, err := volumehelper.NewVolumeHelperImplWithNamespaces(\n\t\tbackupRequest.ResPolicies,\n\t\tbackupRequest.Spec.SnapshotVolumes,\n\t\tlog,\n\t\tkb.kbClient,\n\t\tboolptr.IsSetToTrue(backupRequest.Spec.DefaultVolumesToFsBackup),\n\t\t!backupRequest.ResourceIncludesExcludes.ShouldInclude(kuberesource.PersistentVolumeClaims.String()),\n\t\tnamespaces,\n\t)\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"Failed to build PVC-to-Pod cache for volume policy lookups\")\n\t\treturn err\n\t}\n\n\titemBackupper := &itemBackupper{\n\t\tbackupRequest: backupRequest,\n\t\ttarWriter: tw,\n\t\tdynamicFactory: kb.dynamicFactory,\n\t\tkbClient: kb.kbClient,\n\t\tdiscoveryHelper: kb.discoveryHelper,\n\t\tpodVolumeBackupper: podVolumeBackupper,\n\t\tpodVolumeContext: podVolumeContext,\n\t\tpodVolumeSnapshotTracker: podvolume.NewTracker(),\n\t\tvolumeSnapshotterCache: NewVolumeSnapshotterCache(volumeSnapshotterGetter),\n\t\titemHookHandler: &hook.DefaultItemHookHandler{\n\t\t\tPodCommandExecutor: kb.podCommandExecutor,\n\t\t},\n\t\thookTracker: hook.NewHookTracker(),\n\t\tvolumeHelperImpl: volumeHelperImpl,\n\t\tkubernetesBackupper: kb,\n\t}\n\n\t// helper struct to send current progress between the main\n\t// backup loop and the gouroutine that periodically patches\n\t// the backup CR with progress updates\n\ttype progressUpdate struct {\n\t\ttotalItems, itemsBackedUp int\n\t}\n\n\t// the main backup process will send on this channel once\n\t// for every item it processes.\n\tupdate := make(chan progressUpdate)\n\n\t// the main backup process will send on this channel when\n\t// it's done sending progress updates\n\tquit := make(chan struct{})\n\n\t// This is the progress updater goroutine that receives\n\t// progress updates on the 'update' channel. It patches\n\t// the backup CR with progress updates at most every second,\n\t// but it will not issue a patch if it hasn't received a new\n\t// update since the previous patch. This goroutine exits\n\t// when it receives on the 'quit' channel.\n\tgo func() {\n\t\tticker := time.NewTicker(1 * time.Second)\n\t\tvar lastUpdate *progressUpdate\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-quit:\n\t\t\t\tticker.Stop()\n\t\t\t\treturn\n\t\t\tcase val := <-update:\n\t\t\t\tlastUpdate = &val\n\t\t\tcase <-ticker.C:\n\t\t\t\tif lastUpdate != nil {\n\t\t\t\t\tupdated := backupRequest.Backup.DeepCopy()\n\t\t\t\t\tif updated.Status.Progress == nil {\n\t\t\t\t\t\tupdated.Status.Progress = &velerov1api.BackupProgress{}\n\t\t\t\t\t}\n\t\t\t\t\tupdated.Status.Progress.TotalItems = lastUpdate.totalItems\n\t\t\t\t\tupdated.Status.Progress.ItemsBackedUp = lastUpdate.itemsBackedUp\n\t\t\t\t\tif err := kube.PatchResource(backupRequest.Backup, updated, kb.kbClient); err != nil {\n\t\t\t\t\t\tlog.WithError(errors.WithStack((err))).Warn(\"Got error trying to update backup's status.progress\")\n\t\t\t\t\t}\n\t\t\t\t\tbackupRequest.Status.Progress = &velerov1api.BackupProgress{TotalItems: lastUpdate.totalItems, ItemsBackedUp: lastUpdate.itemsBackedUp}\n\t\t\t\t\tlastUpdate = nil\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}()\n\n\tresponseCtx, responseCancel := context.WithCancel(context.Background())\n\n\tbackedUpGroupResources := map[schema.GroupResource]bool{}\n\t// Maps items in the item list from GR+NamespacedName to a slice of pointers to kubernetesResources\n\t// We need the slice value since if the EnableAPIGroupVersions feature flag is set, there may\n\t// be more than one resource to back up for the given item.\n\titemsMap := make(map[velero.ResourceIdentifier][]*kubernetesResource)\n\tfor i := range items {\n\t\tkey := velero.ResourceIdentifier{\n\t\t\tGroupResource: items[i].groupResource,\n\t\t\tNamespace: items[i].namespace,\n\t\t\tName: items[i].name,\n\t\t}\n\t\titemsMap[key] = append(itemsMap[key], items[i])\n\t\t// add to total items for progress reporting\n\t\tif items[i].kind != \"\" {\n\t\t\tbackupRequest.BackedUpItems.AddItemToTotal(itemKey{\n\t\t\t\tresource: fmt.Sprintf(\"%s/%s\", items[i].preferredGVR.GroupVersion().String(), items[i].kind),\n\t\t\t\tnamespace: items[i].namespace,\n\t\t\t\tname: items[i].name,\n\t\t\t})\n\t\t}\n\t}\n\n\tvar itemBlock *BackupItemBlock\n\titemBlockReturn := make(chan ItemBlockReturn, 100)\n\twg := &sync.WaitGroup{}\n\t// Handle returns from worker pool processing ItemBlocks\n\tgo func() {\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase response := <-itemBlockReturn: // process each BackupItemBlock response\n\t\t\t\tfunc() {\n\t\t\t\t\tdefer wg.Done()\n\t\t\t\t\tif response.err != nil {\n\t\t\t\t\t\tlog.WithError(errors.WithStack((response.err))).Error(\"Got error in BackupItemBlock.\")\n\t\t\t\t\t}\n\t\t\t\t\tfor _, backedUpGR := range response.resources {\n\t\t\t\t\t\tbackedUpGroupResources[backedUpGR] = true\n\t\t\t\t\t}\n\t\t\t\t\t// We could eventually track which itemBlocks have finished\n\t\t\t\t\t// using response.itemBlock\n\n\t\t\t\t\t// updated total is computed as \"how many items we've backed up so far,\n\t\t\t\t\t// plus how many items are processed but not yet backed up plus how many\n\t\t\t\t\t// we know of that are remaining to be processed\"\n\t\t\t\t\tbackedUpItems, totalItems := backupRequest.BackedUpItems.BackedUpAndTotalLen()\n\n\t\t\t\t\t// send a progress update\n\t\t\t\t\tupdate <- progressUpdate{\n\t\t\t\t\t\ttotalItems: totalItems,\n\t\t\t\t\t\titemsBackedUp: backedUpItems,\n\t\t\t\t\t}\n\n\t\t\t\t\tif len(response.itemBlock.Items) > 0 {\n\t\t\t\t\t\tlog.WithFields(map[string]any{\n\t\t\t\t\t\t\t\"progress\": \"\",\n\t\t\t\t\t\t\t\"kind\": response.itemBlock.Items[0].Item.GroupVersionKind().GroupKind().String(),\n\t\t\t\t\t\t\t\"namespace\": response.itemBlock.Items[0].Item.GetNamespace(),\n\t\t\t\t\t\t\t\"name\": response.itemBlock.Items[0].Item.GetName(),\n\t\t\t\t\t\t}).Infof(\"Backed up %d items out of an estimated total of %d (estimate will change throughout the backup)\", backedUpItems, totalItems)\n\t\t\t\t\t}\n\t\t\t\t}()\n\t\t\tcase <-responseCtx.Done():\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}()\n\n\tfor i := range items {\n\t\tlog.WithFields(map[string]any{\n\t\t\t\"progress\": \"\",\n\t\t\t\"resource\": items[i].groupResource.String(),\n\t\t\t\"namespace\": items[i].namespace,\n\t\t\t\"name\": items[i].name,\n\t\t}).Infof(\"Processing item\")\n\n\t\t// Skip if this item has already been processed (in a block or previously excluded)\n\t\tif items[i].inItemBlockOrExcluded {\n\t\t\tlog.Debugf(\"Not creating new ItemBlock for %s %s/%s because it's already in an ItemBlock\", items[i].groupResource.String(), items[i].namespace, items[i].name)\n\t\t} else {\n\t\t\tif itemBlock == nil {\n\t\t\t\titemBlock = NewBackupItemBlock(log, itemBackupper)\n\t\t\t}\n\t\t\tvar newBlockItem *unstructured.Unstructured\n\n\t\t\t// If the EnableAPIGroupVersions feature flag is set, there could be multiple versions\n\t\t\t// of this item to be backed up. Include all of them in the same ItemBlock\n\t\t\tkey := velero.ResourceIdentifier{\n\t\t\t\tGroupResource: items[i].groupResource,\n\t\t\t\tNamespace: items[i].namespace,\n\t\t\t\tName: items[i].name,\n\t\t\t}\n\t\t\tallVersionsOfItem := itemsMap[key]\n\t\t\tfor _, itemVersion := range allVersionsOfItem {\n\t\t\t\tunstructured := itemBlock.addKubernetesResource(itemVersion, log)\n\t\t\t\tif newBlockItem == nil {\n\t\t\t\t\tnewBlockItem = unstructured\n\t\t\t\t}\n\t\t\t}\n\t\t\t// call GetRelatedItems, add found items to block if not in block, recursively until no more items\n\t\t\tif newBlockItem != nil {\n\t\t\t\tkb.executeItemBlockActions(log, newBlockItem, items[i].groupResource, items[i].name, items[i].namespace, itemsMap, itemBlock)\n\t\t\t}\n\t\t}\n\n\t\t// We skip calling backupItemBlock here so that we will add the next item to the current ItemBlock if:\n\t\t// 1) This is not the last item to be processed\n\t\t// 2) Both current and next item are ordered resources\n\t\t// 3) Both current and next item are for the same GroupResource\n\t\taddNextToBlock := i < len(items)-1 && items[i].orderedResource && items[i+1].orderedResource && items[i].groupResource == items[i+1].groupResource\n\t\tif itemBlock != nil && len(itemBlock.Items) > 0 && !addNextToBlock {\n\t\t\tlog.Infof(\"Backing Up Item Block including %s %s/%s (%v items in block)\", items[i].groupResource.String(), items[i].namespace, items[i].name, len(itemBlock.Items))\n\n\t\t\twg.Add(1)\n\t\t\tbackupRequest.WorkerPool.GetInputChannel() <- ItemBlockInput{\n\t\t\t\titemBlock: itemBlock,\n\t\t\t\treturnChan: itemBlockReturn,\n\t\t\t}\n\t\t\titemBlock = nil\n\t\t}\n\t}\n\n\tdone := make(chan struct{})\n\tgo func() {\n\t\tdefer close(done)\n\t\twg.Wait()\n\t}()\n\n\t// Wait for all the ItemBlocks to be processed\n\tselect {\n\tcase <-done:\n\t\tlog.Info(\"done processing ItemBlocks\")\n\tcase <-responseCtx.Done():\n\t\tlog.Info(\"ItemBlock processing canceled\")\n\t}\n\t// cancel response-processing goroutine\n\tresponseCancel()\n\n\t// no more progress updates will be sent on the 'update' channel\n\tquit <- struct{}{}\n\n\t// back up CRD(this is a CRD definition of the resource, it's a CRD instance) for resource if found.\n\t// We should only need to do this if we've backed up at least one item for the resource\n\t// and the CRD type(this is the CRD type itself) is neither included or excluded.\n\t// When it's included, the resource's CRD is already handled. When it's excluded, no need to check.\n\tif !backupRequest.ResourceIncludesExcludes.ShouldExclude(kuberesource.CustomResourceDefinitions.String()) &&\n\t\t!backupRequest.ResourceIncludesExcludes.ShouldInclude(kuberesource.CustomResourceDefinitions.String()) {\n\t\tfor gr := range backedUpGroupResources {\n\t\t\tkb.backupCRD(log, gr, itemBackupper)\n\t\t}\n\t}\n\n\tprocessedPVBs := itemBackupper.podVolumeBackupper.WaitAllPodVolumesProcessed(log)\n\tbackupRequest.PodVolumeBackups = append(backupRequest.PodVolumeBackups, processedPVBs...)\n\n\t// do a final update on progress since we may have just added some CRDs and may not have updated\n\t// for the last few processed items.\n\tupdated = backupRequest.Backup.DeepCopy()\n\tif updated.Status.Progress == nil {\n\t\tupdated.Status.Progress = &velerov1api.BackupProgress{}\n\t}\n\tbackedUpItems := backupRequest.BackedUpItems.Len()\n\tupdated.Status.Progress.TotalItems = backedUpItems\n\tupdated.Status.Progress.ItemsBackedUp = backedUpItems\n\n\t// update the hooks execution status\n\tif updated.Status.HookStatus == nil {\n\t\tupdated.Status.HookStatus = &velerov1api.HookStatus{}\n\t}\n\tupdated.Status.HookStatus.HooksAttempted, updated.Status.HookStatus.HooksFailed = itemBackupper.hookTracker.Stat()\n\tlog.Debugf(\"hookAttempted: %d, hookFailed: %d\", updated.Status.HookStatus.HooksAttempted, updated.Status.HookStatus.HooksFailed)\n\n\tif err := kube.PatchResource(backupRequest.Backup, updated, kb.kbClient); err != nil {\n\t\tlog.WithError(errors.WithStack((err))).Warn(\"Got error trying to update backup's status.progress and hook status\")\n\t}\n\n\tif skippedPVSummary, err := json.Marshal(backupRequest.SkippedPVTracker.Summary()); err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Warn(\"Fail to generate skipped PV summary.\")\n\t} else {\n\t\tlog.Infof(\"Summary for skipped PVs: %s\", skippedPVSummary)\n\t}\n\n\tbackupRequest.Status.Progress = &velerov1api.BackupProgress{TotalItems: backedUpItems, ItemsBackedUp: backedUpItems}\n\tlog.WithField(\"progress\", \"\").Infof(\"Backed up a total of %d items\", backedUpItems)\n\n\treturn nil\n}\n\nfunc (kb *kubernetesBackupper) executeItemBlockActions(\n\tlog logrus.FieldLogger,\n\tobj runtime.Unstructured,\n\tgroupResource schema.GroupResource,\n\tname, namespace string,\n\titemsMap map[velero.ResourceIdentifier][]*kubernetesResource,\n\titemBlock *BackupItemBlock,\n) {\n\tmetadata, err := meta.Accessor(obj)\n\tif err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Warn(\"Failed to get object metadata.\")\n\t\treturn\n\t}\n\tfor _, action := range itemBlock.itemBackupper.backupRequest.ResolvedItemBlockActions {\n\t\tif !action.ShouldUse(groupResource, namespace, metadata, log) {\n\t\t\tcontinue\n\t\t}\n\t\tlog.Info(\"Executing ItemBlock action\")\n\n\t\trelatedItems, err := action.GetRelatedItems(obj, itemBlock.itemBackupper.backupRequest.Backup)\n\t\tif err != nil {\n\t\t\tlog.Error(errors.Wrapf(err, \"error executing ItemBlock action (groupResource=%s, namespace=%s, name=%s)\", groupResource.String(), namespace, name))\n\t\t\tcontinue\n\t\t}\n\n\t\tfor _, relatedItem := range relatedItems {\n\t\t\tvar newBlockItem *unstructured.Unstructured\n\t\t\t// Look for item in itemsMap\n\t\t\titemsToAdd := itemsMap[relatedItem]\n\t\t\t// if item is in the item collector list, we'll have at least one element.\n\t\t\t// If EnableAPIGroupVersions is set, we may have more than one.\n\t\t\t// If we get an unstructured obj back from addKubernetesResource, then it wasn't\n\t\t\t// already in a block and we recursively look for related items in the returned item.\n\t\t\tif len(itemsToAdd) > 0 {\n\t\t\t\tfor _, itemToAdd := range itemsToAdd {\n\t\t\t\t\tunstructured := itemBlock.addKubernetesResource(itemToAdd, log)\n\t\t\t\t\tif newBlockItem == nil {\n\t\t\t\t\t\tnewBlockItem = unstructured\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tif newBlockItem != nil {\n\t\t\t\t\tkb.executeItemBlockActions(log, newBlockItem, relatedItem.GroupResource, relatedItem.Name, relatedItem.Namespace, itemsMap, itemBlock)\n\t\t\t\t}\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\t// Item wasn't found in item collector list, get from cluster\n\t\t\tgvr, resource, err := itemBlock.itemBackupper.discoveryHelper.ResourceFor(relatedItem.GroupResource.WithVersion(\"\"))\n\t\t\tif err != nil {\n\t\t\t\tlog.Error(errors.Wrapf(err, \"Unable to obtain gvr and resource for related item %s %s/%s\", relatedItem.GroupResource.String(), relatedItem.Namespace, relatedItem.Name))\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tclient, err := itemBlock.itemBackupper.dynamicFactory.ClientForGroupVersionResource(gvr.GroupVersion(), resource, relatedItem.Namespace)\n\t\t\tif err != nil {\n\t\t\t\tlog.Error(errors.Wrapf(err, \"Unable to obtain client for gvr %s %s (%s)\", gvr.GroupVersion(), resource.Name, relatedItem.Namespace))\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\titem, err := client.Get(relatedItem.Name, metav1.GetOptions{})\n\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\tlog.WithFields(logrus.Fields{\n\t\t\t\t\t\"groupResource\": relatedItem.GroupResource,\n\t\t\t\t\t\"namespace\": relatedItem.Namespace,\n\t\t\t\t\t\"name\": relatedItem.Name,\n\t\t\t\t}).Warnf(\"Related item was not found in Kubernetes API, can't add to item block\")\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif err != nil {\n\t\t\t\tlog.Error(errors.Wrapf(err, \"Error while trying to get related item %s %s/%s from cluster\", relatedItem.GroupResource.String(), relatedItem.Namespace, relatedItem.Name))\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\titemsMap[relatedItem] = append(itemsMap[relatedItem], &kubernetesResource{\n\t\t\t\tgroupResource: relatedItem.GroupResource,\n\t\t\t\tpreferredGVR: gvr,\n\t\t\t\tnamespace: relatedItem.Namespace,\n\t\t\t\tname: relatedItem.Name,\n\t\t\t\tinItemBlockOrExcluded: true,\n\t\t\t})\n\n\t\t\trelatedItemMetadata, err := meta.Accessor(item)\n\t\t\tif err != nil {\n\t\t\t\tlog.WithError(errors.WithStack(err)).Warn(\"Failed to get object metadata.\")\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\t// Don't add to ItemBlock if item is excluded\n\t\t\t// itemInclusionChecks logs the reason\n\t\t\tif !itemBlock.itemBackupper.itemInclusionChecks(log, false, relatedItemMetadata, item, relatedItem.GroupResource) {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tlog.Infof(\"adding %s %s/%s to ItemBlock\", relatedItem.GroupResource, relatedItem.Namespace, relatedItem.Name)\n\t\t\titemBlock.AddUnstructured(relatedItem.GroupResource, item, gvr)\n\t\t\tkb.executeItemBlockActions(log, item, relatedItem.GroupResource, relatedItem.Name, relatedItem.Namespace, itemsMap, itemBlock)\n\t\t}\n\t}\n}\n\nfunc (kb *kubernetesBackupper) backupItemBlock(itemBlock *BackupItemBlock) []schema.GroupResource {\n\t// find pods in ItemBlock\n\t// filter pods based on whether they still need to be backed up\n\t// this list will be used to run pre/post hooks\n\tvar preHookPods []itemblock.ItemBlockItem\n\titemBlock.Log.Debug(\"Executing pre hooks\")\n\tfor _, item := range itemBlock.Items {\n\t\tif item.Gr == kuberesource.Pods {\n\t\t\tkey, err := kb.getItemKey(item)\n\t\t\tif err != nil {\n\t\t\t\titemBlock.Log.WithError(errors.WithStack(err)).Error(\"Error accessing pod metadata\")\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\t// Don't run hooks if pod has already been backed up\n\t\t\tif !itemBlock.itemBackupper.backupRequest.BackedUpItems.Has(key) {\n\t\t\t\tpreHookPods = append(preHookPods, item)\n\t\t\t}\n\t\t}\n\t}\n\tpostHookPods, failedPods, errs := kb.handleItemBlockPreHooks(itemBlock, preHookPods)\n\tfor i, pod := range failedPods {\n\t\titemBlock.Log.WithError(errs[i]).WithField(\"name\", pod.Item.GetName()).Error(\"Error running pre hooks for pod\")\n\t\t// if pre hook fails, flag pod as backed-up and move on\n\t\tkey, err := kb.getItemKey(pod)\n\t\tif err != nil {\n\t\t\titemBlock.Log.WithError(errors.WithStack(err)).Error(\"Error accessing pod metadata\")\n\t\t\tcontinue\n\t\t}\n\t\titemBlock.itemBackupper.backupRequest.BackedUpItems.AddItem(key)\n\t}\n\n\titemBlock.Log.Debug(\"Backing up items in BackupItemBlock\")\n\tvar grList []schema.GroupResource\n\tfor _, item := range itemBlock.Items {\n\t\tif backedUp := kb.backupItem(itemBlock.Log, item.Gr, itemBlock.itemBackupper, item.Item, item.PreferredGVR, itemBlock); backedUp {\n\t\t\tgrList = append(grList, item.Gr)\n\t\t}\n\t}\n\n\tif len(postHookPods) > 0 {\n\t\titemBlock.Log.Debug(\"Executing post hooks\")\n\t\tkb.handleItemBlockPostHooks(itemBlock, postHookPods)\n\t}\n\n\treturn grList\n}\n\nfunc (kb *kubernetesBackupper) getItemKey(item itemblock.ItemBlockItem) (itemKey, error) {\n\tmetadata, err := meta.Accessor(item.Item)\n\tif err != nil {\n\t\treturn itemKey{}, err\n\t}\n\tkey := itemKey{\n\t\tresource: resourceKey(item.Item),\n\t\tnamespace: metadata.GetNamespace(),\n\t\tname: metadata.GetName(),\n\t}\n\treturn key, nil\n}\n\nfunc (kb *kubernetesBackupper) handleItemBlockPreHooks(itemBlock *BackupItemBlock, hookPods []itemblock.ItemBlockItem) ([]itemblock.ItemBlockItem, []itemblock.ItemBlockItem, []error) {\n\tvar successPods []itemblock.ItemBlockItem\n\tvar failedPods []itemblock.ItemBlockItem\n\tvar errs []error\n\tfor _, pod := range hookPods {\n\t\terr := itemBlock.itemBackupper.itemHookHandler.HandleHooks(itemBlock.Log, pod.Gr, pod.Item, itemBlock.itemBackupper.backupRequest.ResourceHooks, hook.PhasePre, itemBlock.itemBackupper.hookTracker)\n\t\tif err == nil {\n\t\t\tsuccessPods = append(successPods, pod)\n\t\t} else {\n\t\t\tfailedPods = append(failedPods, pod)\n\t\t\terrs = append(errs, err)\n\t\t}\n\t}\n\treturn successPods, failedPods, errs\n}\n\n// The hooks cannot execute until the PVBs to be processed\nfunc (kb *kubernetesBackupper) handleItemBlockPostHooks(itemBlock *BackupItemBlock, hookPods []itemblock.ItemBlockItem) {\n\tlog := itemBlock.Log\n\n\t// the post hooks will not execute until all PVBs of the item block pods are processed\n\tif err := kb.waitUntilPVBsProcessed(itemBlock.itemBackupper.podVolumeContext, log, itemBlock, hookPods); err != nil {\n\t\tlog.WithError(err).Error(\"failed to wait PVBs processed for the ItemBlock\")\n\t\treturn\n\t}\n\n\tfor _, pod := range hookPods {\n\t\tif err := itemBlock.itemBackupper.itemHookHandler.HandleHooks(itemBlock.Log, pod.Gr, pod.Item, itemBlock.itemBackupper.backupRequest.ResourceHooks,\n\t\t\thook.PhasePost, itemBlock.itemBackupper.hookTracker); err != nil {\n\t\t\tlog.WithError(err).WithField(\"name\", pod.Item.GetName()).Error(\"Error running post hooks for pod\")\n\t\t}\n\t}\n}\n\n// wait all PVBs of the item block pods to be processed\nfunc (kb *kubernetesBackupper) waitUntilPVBsProcessed(ctx context.Context, log logrus.FieldLogger, itemBlock *BackupItemBlock, pods []itemblock.ItemBlockItem) error {\n\tpvbMap := map[*velerov1api.PodVolumeBackup]bool{}\n\tfor _, pod := range pods {\n\t\tnamespace, name := pod.Item.GetNamespace(), pod.Item.GetName()\n\t\tpvbs, err := itemBlock.itemBackupper.podVolumeBackupper.ListPodVolumeBackupsByPod(namespace, name)\n\t\tif err != nil {\n\t\t\treturn errors.Wrapf(err, \"failed to list PodVolumeBackups for pod %s/%s\", namespace, name)\n\t\t}\n\t\tfor _, pvb := range pvbs {\n\t\t\tpvbMap[pvb] = pvb.Status.Phase == velerov1api.PodVolumeBackupPhaseCompleted ||\n\t\t\t\tpvb.Status.Phase == velerov1api.PodVolumeBackupPhaseFailed ||\n\t\t\t\tpvb.Status.Phase == velerov1api.PodVolumeBackupPhaseCanceled\n\t\t}\n\t}\n\n\tcheckFunc := func(context.Context) (done bool, err error) {\n\t\tallProcessed := true\n\t\tfor pvb, processed := range pvbMap {\n\t\t\tif processed {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tupdatedPVB, err := itemBlock.itemBackupper.podVolumeBackupper.GetPodVolumeBackupByPodAndVolume(pvb.Spec.Pod.Namespace, pvb.Spec.Pod.Name, pvb.Spec.Volume)\n\t\t\tif err != nil {\n\t\t\t\tallProcessed = false\n\t\t\t\tlog.Infof(\"failed to get PVB: %v\", err)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif updatedPVB.Status.Phase == velerov1api.PodVolumeBackupPhaseCompleted ||\n\t\t\t\tupdatedPVB.Status.Phase == velerov1api.PodVolumeBackupPhaseFailed ||\n\t\t\t\tupdatedPVB.Status.Phase == velerov1api.PodVolumeBackupPhaseCanceled {\n\t\t\t\tpvbMap[pvb] = true\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tallProcessed = false\n\t\t}\n\n\t\treturn allProcessed, nil\n\t}\n\n\treturn wait.PollUntilContextCancel(ctx, 5*time.Second, true, checkFunc)\n}\n\nfunc (kb *kubernetesBackupper) backupItem(log logrus.FieldLogger, gr schema.GroupResource, itemBackupper *itemBackupper, unstructured *unstructured.Unstructured, preferredGVR schema.GroupVersionResource, itemBlock *BackupItemBlock) bool {\n\tbackedUpItem, _, err := itemBackupper.backupItem(log, unstructured, gr, preferredGVR, false, false, itemBlock)\n\tif aggregate, ok := err.(kubeerrs.Aggregate); ok {\n\t\tlog.WithField(\"name\", unstructured.GetName()).Infof(\"%d errors encountered backup up item\", len(aggregate.Errors()))\n\t\t// log each error separately so we get error location info in the log, and an\n\t\t// accurate count of errors\n\t\tfor _, err = range aggregate.Errors() {\n\t\t\tlog.WithError(err).WithField(\"name\", unstructured.GetName()).Error(\"Error backing up item\")\n\t\t}\n\n\t\treturn false\n\t}\n\tif err != nil {\n\t\tlog.WithError(err).WithField(\"name\", unstructured.GetName()).Error(\"Error backing up item\")\n\t\treturn false\n\t}\n\treturn backedUpItem\n}\n\nfunc (kb *kubernetesBackupper) finalizeItem(\n\tlog logrus.FieldLogger,\n\tgr schema.GroupResource,\n\titemBackupper *itemBackupper,\n\tunstructured *unstructured.Unstructured,\n\tpreferredGVR schema.GroupVersionResource,\n) (bool, []FileForArchive) {\n\tbackedUpItem, updateFiles, err := itemBackupper.backupItem(log, unstructured, gr, preferredGVR, true, true, nil)\n\tif aggregate, ok := err.(kubeerrs.Aggregate); ok {\n\t\tlog.WithField(\"name\", unstructured.GetName()).Infof(\"%d errors encountered backup up item\", len(aggregate.Errors()))\n\t\t// log each error separately so we get error location info in the log, and an\n\t\t// accurate count of errors\n\t\tfor _, err = range aggregate.Errors() {\n\t\t\tlog.WithError(err).WithField(\"name\", unstructured.GetName()).Error(\"Error backing up item\")\n\t\t}\n\n\t\treturn false, updateFiles\n\t}\n\tif err != nil {\n\t\tlog.WithError(err).WithField(\"name\", unstructured.GetName()).Error(\"Error backing up item\")\n\t\treturn false, updateFiles\n\t}\n\treturn backedUpItem, updateFiles\n}\n\n// backupCRD checks if the resource is a custom resource, and if so, backs up the custom resource definition\n// associated with it.\nfunc (kb *kubernetesBackupper) backupCRD(log logrus.FieldLogger, gr schema.GroupResource, itemBackupper *itemBackupper) {\n\tcrdGroupResource := kuberesource.CustomResourceDefinitions\n\n\tlog.Debugf(\"Getting server preferred API version for %s\", crdGroupResource)\n\tgvr, apiResource, err := kb.discoveryHelper.ResourceFor(crdGroupResource.WithVersion(\"\"))\n\tif err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Errorf(\"Error getting resolved resource for %s\", crdGroupResource)\n\t\treturn\n\t}\n\tlog.Debugf(\"Got server preferred API version %s for %s\", gvr.Version, crdGroupResource)\n\n\tlog.Debugf(\"Getting dynamic client for %s\", gvr.String())\n\tcrdClient, err := kb.dynamicFactory.ClientForGroupVersionResource(gvr.GroupVersion(), apiResource, \"\")\n\tif err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Errorf(\"Error getting dynamic client for %s\", crdGroupResource)\n\t\treturn\n\t}\n\tlog.Debugf(\"Got dynamic client for %s\", gvr.String())\n\n\t// try to get a CRD whose name matches the provided GroupResource\n\tunstructured, err := crdClient.Get(gr.String(), metav1.GetOptions{})\n\tif apierrors.IsNotFound(err) {\n\t\t// not found: this means the GroupResource provided was not a\n\t\t// custom resource, so there's no CRD to back up.\n\t\tlog.Debugf(\"No CRD found for GroupResource %s\", gr.String())\n\t\treturn\n\t}\n\tif err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Errorf(\"Error getting CRD %s\", gr.String())\n\t\treturn\n\t}\n\n\tlog.Infof(\"Found associated CRD %s to add to backup\", gr.String())\n\n\tkb.backupItem(log, gvr.GroupResource(), itemBackupper, unstructured, gvr, nil)\n}\n\nfunc (kb *kubernetesBackupper) writeBackupVersion(tw tarWriter) error {\n\tversionFile := filepath.Join(velerov1api.MetadataDir, \"version\")\n\tversionString := fmt.Sprintf(\"%s\\n\", BackupFormatVersion)\n\n\thdr := &tar.Header{\n\t\tName: versionFile,\n\t\tSize: int64(len(versionString)),\n\t\tTypeflag: tar.TypeReg,\n\t\tMode: 0755,\n\t\tModTime: time.Now(),\n\t}\n\tif err := tw.WriteHeader(hdr); err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\tif _, err := tw.Write([]byte(versionString)); err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\treturn nil\n}\n\nfunc (kb *kubernetesBackupper) FinalizeBackup(\n\tlog logrus.FieldLogger,\n\tbackupRequest *Request,\n\tinBackupFile io.Reader,\n\toutBackupFile io.Writer,\n\tbackupItemActionResolver framework.BackupItemActionResolverV2,\n\tasyncBIAOperations []*itemoperation.BackupOperation,\n\tbackupStore persistence.BackupStore,\n) error {\n\tgzw := gzip.NewWriter(outBackupFile)\n\tdefer gzw.Close()\n\ttw := NewTarWriter(tar.NewWriter(gzw))\n\tdefer tw.Close()\n\n\tgzr, err := gzip.NewReader(inBackupFile)\n\tif err != nil {\n\t\tlog.Infof(\"error creating gzip reader: %v\", err)\n\t\treturn err\n\t}\n\tdefer gzr.Close()\n\ttr := tar.NewReader(gzr)\n\n\tbackupRequest.ResolvedActions, err = backupItemActionResolver.ResolveActions(kb.discoveryHelper, log)\n\tif err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Debugf(\"Error from backupItemActionResolver.ResolveActions\")\n\t\treturn err\n\t}\n\n\t// set up a temp dir for the itemCollector to use to temporarily\n\t// store items as they're scraped from the API.\n\ttempDir, err := os.MkdirTemp(\"\", \"\")\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error creating temp dir for backup\")\n\t}\n\tdefer os.RemoveAll(tempDir)\n\n\tcollector := &itemCollector{\n\t\tlog: log,\n\t\tbackupRequest: backupRequest,\n\t\tdiscoveryHelper: kb.discoveryHelper,\n\t\tdynamicFactory: kb.dynamicFactory,\n\t\tcohabitatingResources: cohabitatingResources(),\n\t\tdir: tempDir,\n\t\tpageSize: kb.clientPageSize,\n\t}\n\n\t// Get item list from itemoperation.BackupOperation.Spec.PostOperationItems\n\tvar resourceIDs []velero.ResourceIdentifier\n\tfor _, operation := range asyncBIAOperations {\n\t\tif len(operation.Spec.PostOperationItems) != 0 {\n\t\t\tresourceIDs = append(resourceIDs, operation.Spec.PostOperationItems...)\n\t\t}\n\t}\n\titems := collector.getItemsFromResourceIdentifiers(resourceIDs)\n\tlog.WithField(\"progress\", \"\").Infof(\"Collected %d items from the async BIA operations PostOperationItems list\", len(items))\n\n\titemBackupper := &itemBackupper{\n\t\tbackupRequest: backupRequest,\n\t\ttarWriter: tw,\n\t\tdynamicFactory: kb.dynamicFactory,\n\t\tkbClient: kb.kbClient,\n\t\tdiscoveryHelper: kb.discoveryHelper,\n\t\titemHookHandler: &hook.NoOpItemHookHandler{},\n\t\tpodVolumeSnapshotTracker: podvolume.NewTracker(),\n\t\thookTracker: hook.NewHookTracker(),\n\t\tkubernetesBackupper: kb,\n\t}\n\tupdateFiles := make(map[string]FileForArchive)\n\tbackedUpGroupResources := map[schema.GroupResource]bool{}\n\n\tunstructuredDataUploads := make([]unstructured.Unstructured, 0)\n\n\tfor i, item := range items {\n\t\tlog.WithFields(map[string]any{\n\t\t\t\"progress\": \"\",\n\t\t\t\"resource\": item.groupResource.String(),\n\t\t\t\"namespace\": item.namespace,\n\t\t\t\"name\": item.name,\n\t\t}).Infof(\"Processing item\")\n\n\t\t// use an anonymous func so we can defer-close/remove the file\n\t\t// as soon as we're done with it\n\t\tfunc() {\n\t\t\tvar unstructured unstructured.Unstructured\n\n\t\t\tf, err := os.Open(item.path)\n\t\t\tif err != nil {\n\t\t\t\tlog.WithError(errors.WithStack(err)).Error(\"Error opening file containing item\")\n\t\t\t\treturn\n\t\t\t}\n\t\t\tdefer f.Close()\n\t\t\tdefer os.Remove(f.Name())\n\n\t\t\tif err := json.NewDecoder(f).Decode(&unstructured); err != nil {\n\t\t\t\tlog.WithError(errors.WithStack(err)).Error(\"Error decoding JSON from file\")\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tif item.groupResource == kuberesource.DataUploads {\n\t\t\t\tunstructuredDataUploads = append(unstructuredDataUploads, unstructured)\n\t\t\t}\n\n\t\t\tbackedUp, itemFiles := kb.finalizeItem(log, item.groupResource, itemBackupper, &unstructured, item.preferredGVR)\n\t\t\tif backedUp {\n\t\t\t\tbackedUpGroupResources[item.groupResource] = true\n\t\t\t\tfor _, itemFile := range itemFiles {\n\t\t\t\t\tupdateFiles[itemFile.FilePath] = itemFile\n\t\t\t\t}\n\t\t\t}\n\t\t}()\n\n\t\t// updated total is computed as \"how many items we've backed up so far, plus\n\t\t// how many items we know of that are remaining\"\n\t\tbackedUpItems := backupRequest.BackedUpItems.Len()\n\t\ttotalItems := backedUpItems + (len(items) - (i + 1))\n\n\t\tlog.WithFields(map[string]any{\n\t\t\t\"progress\": \"\",\n\t\t\t\"resource\": item.groupResource.String(),\n\t\t\t\"namespace\": item.namespace,\n\t\t\t\"name\": item.name,\n\t\t}).Infof(\"Updated %d items out of an estimated total of %d (estimate will change throughout the backup finalizer)\", backedUpItems, totalItems)\n\t}\n\n\tvolumeInfos, err := backupStore.GetBackupVolumeInfos(backupRequest.Backup.Name)\n\tif err != nil {\n\t\tlog.WithError(err).Errorf(\"fail to get the backup VolumeInfos for backup %s\", backupRequest.Name)\n\t\treturn err\n\t}\n\n\tif err := updateVolumeInfos(volumeInfos, unstructuredDataUploads, asyncBIAOperations, log); err != nil {\n\t\tlog.WithError(err).Errorf(\"fail to update VolumeInfos for backup %s\", backupRequest.Name)\n\t\treturn err\n\t}\n\n\tif err := putVolumeInfos(backupRequest.Name, volumeInfos, backupStore); err != nil {\n\t\tlog.WithError(err).Errorf(\"fail to put the VolumeInfos for backup %s\", backupRequest.Name)\n\t\treturn err\n\t}\n\n\t// write new tar archive replacing files in original with content updateFiles for matches\n\tif err := buildFinalTarball(tr, tw, updateFiles); err != nil {\n\t\tlog.Errorf(\"Error building final tarball: %s\", err.Error())\n\t\treturn err\n\t}\n\n\tlog.WithField(\"progress\", \"\").Infof(\"Updated a total of %d items\", backupRequest.BackedUpItems.Len())\n\n\treturn nil\n}\n\nfunc buildFinalTarball(tr *tar.Reader, tw tarWriter, updateFiles map[string]FileForArchive) error {\n\ttw.Lock()\n\tdefer tw.Unlock()\n\tfor {\n\t\theader, err := tr.Next()\n\t\tif err == io.EOF {\n\t\t\tbreak\n\t\t}\n\t\tif err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\t\tnewFile, ok := updateFiles[header.Name]\n\t\tif ok {\n\t\t\t// add updated file to archive, skip over tr file content\n\t\t\tif err := tw.WriteHeader(newFile.Header); err != nil {\n\t\t\t\treturn errors.WithStack(err)\n\t\t\t}\n\t\t\tif _, err := tw.Write(newFile.FileBytes); err != nil {\n\t\t\t\treturn errors.WithStack(err)\n\t\t\t}\n\t\t\tdelete(updateFiles, header.Name)\n\t\t\t// skip over file contents from old tarball\n\t\t\t_, err := io.ReadAll(tr)\n\t\t\tif err != nil {\n\t\t\t\treturn errors.WithStack(err)\n\t\t\t}\n\t\t} else {\n\t\t\t// Add original content to new tarball, as item wasn't updated\n\t\t\toldContents, err := io.ReadAll(tr)\n\t\t\tif err != nil {\n\t\t\t\treturn errors.WithStack(err)\n\t\t\t}\n\t\t\tif err := tw.WriteHeader(header); err != nil {\n\t\t\t\treturn errors.WithStack(err)\n\t\t\t}\n\t\t\tif _, err := tw.Write(oldContents); err != nil {\n\t\t\t\treturn errors.WithStack(err)\n\t\t\t}\n\t\t}\n\t}\n\t// iterate over any remaining map entries, which represent updated items that\n\t// were not in the original backup tarball\n\tfor _, newFile := range updateFiles {\n\t\tif err := tw.WriteHeader(newFile.Header); err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\t\tif _, err := tw.Write(newFile.FileBytes); err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\t}\n\treturn nil\n}\n\ntype tarWriter struct {\n\t*tar.Writer\n\t*sync.Mutex\n}\n\nfunc NewTarWriter(writer *tar.Writer) tarWriter {\n\treturn tarWriter{\n\t\tWriter: writer,\n\t\tMutex: &sync.Mutex{},\n\t}\n}\n\n// updateVolumeInfos update the VolumeInfos according to the AsyncOperations\nfunc updateVolumeInfos(\n\tvolumeInfos []*volume.BackupVolumeInfo,\n\tunstructuredItems []unstructured.Unstructured,\n\toperations []*itemoperation.BackupOperation,\n\tlog logrus.FieldLogger,\n) error {\n\tfor _, unstructured := range unstructuredItems {\n\t\tvar dataUpload velerov2alpha1.DataUpload\n\t\terr := runtime.DefaultUnstructuredConverter.FromUnstructured(unstructured.UnstructuredContent(), &dataUpload)\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Errorf(\"fail to convert DataUpload: %s/%s\",\n\t\t\t\tunstructured.GetNamespace(), unstructured.GetName())\n\t\t\treturn err\n\t\t}\n\n\t\tfor index := range volumeInfos {\n\t\t\tif volumeInfos[index].PVCName == dataUpload.Spec.SourcePVC &&\n\t\t\t\tvolumeInfos[index].PVCNamespace == dataUpload.Spec.SourceNamespace &&\n\t\t\t\tvolumeInfos[index].SnapshotDataMovementInfo != nil {\n\t\t\t\tif dataUpload.Status.CompletionTimestamp != nil {\n\t\t\t\t\tvolumeInfos[index].CompletionTimestamp = dataUpload.Status.CompletionTimestamp\n\t\t\t\t}\n\t\t\t\tvolumeInfos[index].SnapshotDataMovementInfo.SnapshotHandle = dataUpload.Status.SnapshotID\n\t\t\t\tvolumeInfos[index].SnapshotDataMovementInfo.RetainedSnapshot = dataUpload.Spec.CSISnapshot.VolumeSnapshot\n\t\t\t\tvolumeInfos[index].SnapshotDataMovementInfo.Size = dataUpload.Status.Progress.TotalBytes\n\t\t\t\tvolumeInfos[index].SnapshotDataMovementInfo.IncrementalSize = dataUpload.Status.IncrementalBytes\n\t\t\t\tvolumeInfos[index].SnapshotDataMovementInfo.Phase = dataUpload.Status.Phase\n\n\t\t\t\tif dataUpload.Status.Phase == velerov2alpha1.DataUploadPhaseCompleted {\n\t\t\t\t\tvolumeInfos[index].Result = volume.VolumeResultSucceeded\n\t\t\t\t} else {\n\t\t\t\t\tvolumeInfos[index].Result = volume.VolumeResultFailed\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\t// Update CSI snapshot VolumeInfo's CompletionTimestamp by the operation update time.\n\tfor volumeIndex := range volumeInfos {\n\t\tif volumeInfos[volumeIndex].BackupMethod == volume.CSISnapshot &&\n\t\t\tvolumeInfos[volumeIndex].CSISnapshotInfo != nil {\n\t\t\tfor opIndex := range operations {\n\t\t\t\tif volumeInfos[volumeIndex].CSISnapshotInfo.OperationID == operations[opIndex].Spec.OperationID {\n\t\t\t\t\t// The VolumeSnapshot and VolumeSnapshotContent don't have a completion timestamp,\n\t\t\t\t\t// so use the operation.Status.Updated as the alternative. It is not the exact time\n\t\t\t\t\t// when the snapshot turns ready, but the operation controller periodically watch the\n\t\t\t\t\t// VSC and VS status. When the controller finds they reach to the ReadyToUse state,\n\t\t\t\t\t// The operation.Status.Updated is set as the found time.\n\t\t\t\t\tvolumeInfos[volumeIndex].CompletionTimestamp = operations[opIndex].Status.Updated\n\n\t\t\t\t\t// Set Succeeded to true when the operation has no error.\n\t\t\t\t\tif operations[opIndex].Status.Error == \"\" {\n\t\t\t\t\t\tvolumeInfos[volumeIndex].Result = volume.VolumeResultSucceeded\n\t\t\t\t\t} else {\n\t\t\t\t\t\tvolumeInfos[volumeIndex].Result = volume.VolumeResultFailed\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc putVolumeInfos(\n\tbackupName string,\n\tvolumeInfos []*volume.BackupVolumeInfo,\n\tbackupStore persistence.BackupStore,\n) error {\n\tbackupVolumeInfoBuf := new(bytes.Buffer)\n\tgzw := gzip.NewWriter(backupVolumeInfoBuf)\n\tdefer gzw.Close()\n\n\tif err := json.NewEncoder(gzw).Encode(volumeInfos); err != nil {\n\t\treturn errors.Wrap(err, \"error encoding restore results to JSON\")\n\t}\n\n\tif err := gzw.Close(); err != nil {\n\t\treturn errors.Wrap(err, \"error closing gzip writer\")\n\t}\n\n\treturn backupStore.PutBackupVolumeInfos(backupName, backupVolumeInfoBuf)\n}\n" }, { "path": "pkg/backup/backup_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"archive/tar\"\n\t\"bytes\"\n\t\"compress/gzip\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"sort\"\n\t\"strings\"\n\t\"sync\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/api/meta\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/internal/resourcepolicies\"\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\t\"github.com/vmware-tanzu/velero/pkg/apis/velero/shared\"\n\tvelerov1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/discovery\"\n\t\"github.com/vmware-tanzu/velero/pkg/features\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/persistence\"\n\tpersistencemocks \"github.com/vmware-tanzu/velero/pkg/persistence/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tbiav2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/backupitemaction/v2\"\n\tibav1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/itemblockaction/v1\"\n\tvsv1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/volumesnapshotter/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/podvolume\"\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n\tkubeutil \"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nfunc TestBackedUpItemsMatchesTarballContents(t *testing.T) {\n\t// TODO: figure out if this can be replaced with the restmapper\n\t// (https://github.com/kubernetes/apimachinery/blob/035e418f1ad9b6da47c4e01906a0cfe32f4ee2e7/pkg/api/meta/restmapper.go)\n\tgvkToResource := map[string]string{\n\t\t\"v1/Pod\": \"pods\",\n\t\t\"apps/v1/Deployment\": \"deployments.apps\",\n\t\t\"v1/PersistentVolume\": \"persistentvolumes\",\n\t}\n\n\th := newHarness(t, nil)\n\tdefer h.itemBlockPool.Stop()\n\n\treq := &Request{\n\t\tBackup: defaultBackup().Result(),\n\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\tWorkerPool: &h.itemBlockPool,\n\t}\n\n\tbackupFile := bytes.NewBuffer([]byte{})\n\n\tapiResources := []*test.APIResource{\n\t\ttest.Pods(\n\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t),\n\t\ttest.Deployments(\n\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t),\n\t\ttest.PVs(\n\t\t\tbuilder.ForPersistentVolume(\"bar\").ClaimRef(\"foo\", \"pvc1\").Result(),\n\t\t\tbuilder.ForPersistentVolume(\"baz\").ClaimRef(\"bar\", \"pvc2\").Result(),\n\t\t),\n\t}\n\tfor _, resource := range apiResources {\n\t\th.addItems(t, resource)\n\t}\n\n\th.backupper.Backup(h.log, req, backupFile, nil, nil, nil)\n\n\t// go through BackedUpItems after the backup to assemble the list of files we\n\t// expect to see in the tarball and compare to see if they match\n\tvar expectedFiles []string\n\tfor item := range req.BackedUpItems.CopyItemMap() {\n\t\tfile := \"resources/\" + gvkToResource[item.resource]\n\t\tif item.namespace != \"\" {\n\t\t\tfile = file + \"/namespaces/\" + item.namespace\n\t\t} else {\n\t\t\tfile = file + \"/cluster\"\n\t\t}\n\t\tfile = file + \"/\" + item.name + \".json\"\n\t\texpectedFiles = append(expectedFiles, file)\n\n\t\tfileWithVersion := \"resources/\" + gvkToResource[item.resource]\n\t\tif item.namespace != \"\" {\n\t\t\tfileWithVersion = fileWithVersion + \"/v1-preferredversion/\" + \"namespaces/\" + item.namespace\n\t\t} else {\n\t\t\tfileWithVersion = fileWithVersion + \"/v1-preferredversion\" + \"/cluster\"\n\t\t}\n\t\tfileWithVersion = fileWithVersion + \"/\" + item.name + \".json\"\n\t\texpectedFiles = append(expectedFiles, fileWithVersion)\n\t}\n\n\tassertTarballContents(t, backupFile, append(expectedFiles, \"metadata/version\")...)\n}\n\n// TestBackupProgressIsUpdated verifies that after a backup has run, its\n// status.progress fields are updated to reflect the total number of items\n// backed up. It validates this by comparing their values to the length of\n// the request's BackedUpItems field.\nfunc TestBackupProgressIsUpdated(t *testing.T) {\n\th := newHarness(t, nil)\n\tdefer h.itemBlockPool.Stop()\n\treq := &Request{\n\t\tBackup: defaultBackup().Result(),\n\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\tWorkerPool: &h.itemBlockPool,\n\t}\n\tbackupFile := bytes.NewBuffer([]byte{})\n\n\tapiResources := []*test.APIResource{\n\t\ttest.Pods(\n\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t),\n\t\ttest.Deployments(\n\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t),\n\t\ttest.PVs(\n\t\t\tbuilder.ForPersistentVolume(\"bar\").Result(),\n\t\t\tbuilder.ForPersistentVolume(\"baz\").Result(),\n\t\t),\n\t}\n\tfor _, resource := range apiResources {\n\t\th.addItems(t, resource)\n\t}\n\n\th.backupper.Backup(h.log, req, backupFile, nil, nil, nil)\n\n\trequire.NotNil(t, req.Status.Progress)\n\tassert.Equal(t, req.BackedUpItems.Len(), req.Status.Progress.TotalItems)\n\tassert.Equal(t, req.BackedUpItems.Len(), req.Status.Progress.ItemsBackedUp)\n}\n\n// TestBackupOldResourceFiltering runs backups with different combinations\n// of resource filters (included/excluded resources, included/excluded\n// namespaces, label selectors, \"include cluster resources\" flag), and\n// verifies that the set of items written to the backup tarball are\n// correct. Validation is done by looking at the names of the files in\n// the backup tarball; the contents of the files are not checked.\nfunc TestBackupOldResourceFiltering(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1.Backup\n\t\tapiResources []*test.APIResource\n\t\twant []string\n\t\tactions []biav2.BackupItemAction\n\t}{\n\t\t{\n\t\t\tname: \"no filters backs up everything\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"included resources filter only backs up resources of those types\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tIncludedResources(\"pods\").\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"excluded resources filter only backs up resources not of those types\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tExcludedResources(\"deployments\").\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"included namespaces filter only backs up resources in those namespaces\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tIncludedNamespaces(\"foo\").\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"excluded namespaces filter only backs up resources not in those namespaces\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tExcludedNamespaces(\"zoo\").\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"IncludeClusterResources=false only backs up namespaced resources\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tIncludeClusterResources(false).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"bar\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"baz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"label selector only backs up matching resources\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tLabelSelector(&metav1.LabelSelector{MatchLabels: map[string]string{\"a\": \"b\"}}).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").ObjectMeta(builder.WithLabels(\"a\", \"b\")).Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").ObjectMeta(builder.WithLabels(\"a\", \"b\")).Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"bar\").ObjectMeta(builder.WithLabels(\"a\", \"b\")).Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"baz\").ObjectMeta(builder.WithLabels(\"a\", \"c\")).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/bar.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"OrLabelSelector only backs up matching resources\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tOrLabelSelector([]*metav1.LabelSelector{{MatchLabels: map[string]string{\"a1\": \"b1\"}}, {MatchLabels: map[string]string{\"a2\": \"b2\"}},\n\t\t\t\t\t{MatchLabels: map[string]string{\"a3\": \"b3\"}}, {MatchLabels: map[string]string{\"a4\": \"b4\"}}}).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").ObjectMeta(builder.WithLabels(\"a1\", \"b1\")).Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").ObjectMeta(builder.WithLabels(\"a2\", \"b2\")).Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"bar\").ObjectMeta(builder.WithLabels(\"a4\", \"b4\")).Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"baz\").ObjectMeta(builder.WithLabels(\"a5\", \"b5\")).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/bar.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"resources with velero.io/exclude-from-backup=true label are not included\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").ObjectMeta(builder.WithLabels(velerov1.ExcludeFromBackupLabel, \"true\")).Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").ObjectMeta(builder.WithLabels(velerov1.ExcludeFromBackupLabel, \"true\")).Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"bar\").ObjectMeta(builder.WithLabels(\"a\", \"b\")).Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"baz\").ObjectMeta(builder.WithLabels(velerov1.ExcludeFromBackupLabel, \"true\")).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/bar.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"resources with velero.io/exclude-from-backup=true label are not included even if matching label selector\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tLabelSelector(&metav1.LabelSelector{MatchLabels: map[string]string{\"a\": \"b\"}}).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").ObjectMeta(builder.WithLabels(velerov1.ExcludeFromBackupLabel, \"true\", \"a\", \"b\")).Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").ObjectMeta(builder.WithLabels(\"a\", \"b\")).Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").ObjectMeta(builder.WithLabels(velerov1.ExcludeFromBackupLabel, \"true\", \"a\", \"b\")).Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"bar\").ObjectMeta(builder.WithLabels(\"a\", \"b\")).Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"baz\").ObjectMeta(builder.WithLabels(\"a\", \"b\", velerov1.ExcludeFromBackupLabel, \"true\")).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/bar.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"resources with velero.io/exclude-from-backup label specified but not 'true' are included\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").ObjectMeta(builder.WithLabels(velerov1.ExcludeFromBackupLabel, \"false\")).Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").ObjectMeta(builder.WithLabels(velerov1.ExcludeFromBackupLabel, \"1\")).Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"bar\").ObjectMeta(builder.WithLabels(\"a\", \"b\")).Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"baz\").ObjectMeta(builder.WithLabels(velerov1.ExcludeFromBackupLabel, \"\")).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/bar.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/baz.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/bar.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/baz.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should include cluster-scoped resources if backing up subset of namespaces and IncludeClusterResources=true\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tIncludedNamespaces(\"ns-1\", \"ns-2\").\n\t\t\t\tIncludeClusterResources(true).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-3\", \"pod-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-2/pod-1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/pv-1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/pv-2.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-2/pod-1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/pv-1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/pv-2.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should not include cluster-scoped resource if backing up subset of namespaces and IncludeClusterResources=false\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tIncludedNamespaces(\"ns-1\", \"ns-2\").\n\t\t\t\tIncludeClusterResources(false).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-3\", \"pod-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-2/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-2/pod-1.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should not include cluster-scoped resource if backing up subset of namespaces and IncludeClusterResources=nil\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tIncludedNamespaces(\"ns-1\", \"ns-2\").\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-3\", \"pod-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-2/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-2/pod-1.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should include cluster-scoped resources if backing up all namespaces and IncludeClusterResources=true\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tIncludeClusterResources(true).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-3\", \"pod-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-2/pod-1.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-3/pod-1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/pv-1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/pv-2.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-2/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-3/pod-1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/pv-1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/pv-2.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should not include cluster-scoped resources if backing up all namespaces and IncludeClusterResources=false\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tIncludeClusterResources(false).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-3\", \"pod-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-2/pod-1.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-3/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-2/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-3/pod-1.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should include cluster-scoped resources if backing up all namespaces and IncludeClusterResources=nil\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-3\", \"pod-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-2/pod-1.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-3/pod-1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/pv-1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/pv-2.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-2/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-3/pod-1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/pv-1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/pv-2.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when a wildcard and a specific resource are included, the wildcard takes precedence\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tIncludedResources(\"*\", \"pods\").\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard excludes are ignored\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tExcludedResources(\"*\").\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"unresolvable included resources are ignored\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tIncludedResources(\"pods\", \"unresolvable\").\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when all included resources are unresolvable, nothing is included\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tIncludedResources(\"unresolvable-1\", \"unresolvable-2\").\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"unresolvable excluded resources are ignored\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tExcludedResources(\"deployments\", \"unresolvable\").\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when all excluded resources are unresolvable, nothing is excluded\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tIncludedResources(\"*\").\n\t\t\t\tExcludedResources(\"unresolvable-1\", \"unresolvable-2\").\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"terminating resources are not backed up\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").ObjectMeta(builder.WithDeletionTimestamp(time.Now())).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"new filters' default value should not impact the old filters' function\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"foo\").IncludeClusterResources(true).Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Volumes(builder.ForVolume(\"foo\").PersistentVolumeClaimSource(\"test-1\").Result()).Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"foo\", \"test-1\").VolumeName(\"test1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/persistentvolumeclaims/namespaces/foo/test-1.json\",\n\t\t\t\t\"resources/persistentvolumeclaims/v1-preferredversion/namespaces/foo/test-1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/test1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/test2.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/test1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/test2.json\",\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\t&pluggableAction{\n\t\t\t\t\tselector: velero.ResourceSelector{IncludedResources: []string{\"persistentvolumeclaims\"}},\n\t\t\t\t\texecuteFunc: func(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\t\t\t\t\t\tadditionalItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"test1\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn item, additionalItems, \"\", nil, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Resource's CRD should be included\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"foo\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"volumesnapshotlocations.velero.io\").Result(),\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"test.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.VSLs(\n\t\t\t\t\tbuilder.ForVolumeSnapshotLocation(\"foo\", \"bar\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Backups(\n\t\t\t\t\tbuilder.ForBackup(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/cluster/volumesnapshotlocations.velero.io.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/v1beta1-preferredversion/cluster/volumesnapshotlocations.velero.io.json\",\n\t\t\t\t\"resources/volumesnapshotlocations.velero.io/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/volumesnapshotlocations.velero.io/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Resource's CRD is not included, when CRD is excluded.\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"foo\").ExcludedResources(\"customresourcedefinitions.apiextensions.k8s.io\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"volumesnapshotlocations.velero.io\").Result(),\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"test.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.VSLs(\n\t\t\t\t\tbuilder.ForVolumeSnapshotLocation(\"foo\", \"bar\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Backups(\n\t\t\t\t\tbuilder.ForBackup(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/volumesnapshotlocations.velero.io/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/volumesnapshotlocations.velero.io/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t},\n\t\t},\n\t}\n\n\titemBlockPool := StartItemBlockWorkerPool(t.Context(), 1, logrus.StandardLogger())\n\tdefer itemBlockPool.Stop()\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\th = newHarness(t, itemBlockPool)\n\t\t\t\treq = &Request{\n\t\t\t\t\tBackup: tc.backup,\n\t\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t\t}\n\t\t\t\tbackupFile = bytes.NewBuffer([]byte{})\n\t\t\t)\n\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.addItems(t, resource)\n\t\t\t}\n\n\t\t\th.backupper.Backup(h.log, req, backupFile, tc.actions, nil, nil)\n\n\t\t\tassertTarballContents(t, backupFile, append(tc.want, \"metadata/version\")...)\n\t\t})\n\t}\n}\n\n// TestCRDInclusion tests whether related CRDs are included, based on\n// backed-up resources and \"include cluster resources\" flag, and\n// verifies that the set of items written to the backup tarball are\n// correct. Validation is done by looking at the names of the files in\n// the backup tarball; the contents of the files are not checked.\nfunc TestCRDInclusion(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1.Backup\n\t\tapiResources []*test.APIResource\n\t\twant []string\n\t}{\n\t\t{\n\t\t\tname: \"include cluster resources=auto includes all CRDs when running a full-cluster backup\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"volumesnapshotlocations.velero.io\").Result(),\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"test.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.VSLs(\n\t\t\t\t\tbuilder.ForVolumeSnapshotLocation(\"foo\", \"vsl-1\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/cluster/backups.velero.io.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/cluster/volumesnapshotlocations.velero.io.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/cluster/test.velero.io.json\",\n\t\t\t\t\"resources/volumesnapshotlocations.velero.io/namespaces/foo/vsl-1.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/v1beta1-preferredversion/cluster/backups.velero.io.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/v1beta1-preferredversion/cluster/volumesnapshotlocations.velero.io.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/v1beta1-preferredversion/cluster/test.velero.io.json\",\n\t\t\t\t\"resources/volumesnapshotlocations.velero.io/v1-preferredversion/namespaces/foo/vsl-1.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"include cluster resources=false excludes all CRDs when backing up all namespaces\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tIncludeClusterResources(false).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"volumesnapshotlocations.velero.io\").Result(),\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"test.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.VSLs(\n\t\t\t\t\tbuilder.ForVolumeSnapshotLocation(\"foo\", \"vsl-1\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/volumesnapshotlocations.velero.io/namespaces/foo/vsl-1.json\",\n\t\t\t\t\"resources/volumesnapshotlocations.velero.io/v1-preferredversion/namespaces/foo/vsl-1.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"include cluster resources=true includes all CRDs when running a full-cluster backup\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tIncludeClusterResources(true).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"volumesnapshotlocations.velero.io\").Result(),\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"test.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.VSLs(\n\t\t\t\t\tbuilder.ForVolumeSnapshotLocation(\"foo\", \"vsl-1\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/cluster/backups.velero.io.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/cluster/volumesnapshotlocations.velero.io.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/cluster/test.velero.io.json\",\n\t\t\t\t\"resources/volumesnapshotlocations.velero.io/namespaces/foo/vsl-1.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/v1beta1-preferredversion/cluster/backups.velero.io.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/v1beta1-preferredversion/cluster/volumesnapshotlocations.velero.io.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/v1beta1-preferredversion/cluster/test.velero.io.json\",\n\t\t\t\t\"resources/volumesnapshotlocations.velero.io/v1-preferredversion/namespaces/foo/vsl-1.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"include cluster resources=auto includes CRDs with CRs when backing up selected namespaces\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tIncludedNamespaces(\"foo\").\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"volumesnapshotlocations.velero.io\").Result(),\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"test.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.VSLs(\n\t\t\t\t\tbuilder.ForVolumeSnapshotLocation(\"foo\", \"vsl-1\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/cluster/volumesnapshotlocations.velero.io.json\",\n\t\t\t\t\"resources/volumesnapshotlocations.velero.io/namespaces/foo/vsl-1.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/v1beta1-preferredversion/cluster/volumesnapshotlocations.velero.io.json\",\n\t\t\t\t\"resources/volumesnapshotlocations.velero.io/v1-preferredversion/namespaces/foo/vsl-1.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"include-cluster-resources=false excludes all CRDs when backing up selected namespaces\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tIncludeClusterResources(false).\n\t\t\t\tIncludedNamespaces(\"foo\").\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"volumesnapshotlocations.velero.io\").Result(),\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"test.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.VSLs(\n\t\t\t\t\tbuilder.ForVolumeSnapshotLocation(\"foo\", \"bar\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/volumesnapshotlocations.velero.io/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/volumesnapshotlocations.velero.io/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"include cluster resources=true includes all CRDs when backing up selected namespaces\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tIncludeClusterResources(true).\n\t\t\t\tIncludedNamespaces(\"foo\").\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"volumesnapshotlocations.velero.io\").Result(),\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"test.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.VSLs(\n\t\t\t\t\tbuilder.ForVolumeSnapshotLocation(\"foo\", \"vsl-1\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/cluster/backups.velero.io.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/cluster/volumesnapshotlocations.velero.io.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/cluster/test.velero.io.json\",\n\t\t\t\t\"resources/volumesnapshotlocations.velero.io/namespaces/foo/vsl-1.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/v1beta1-preferredversion/cluster/backups.velero.io.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/v1beta1-preferredversion/cluster/volumesnapshotlocations.velero.io.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/v1beta1-preferredversion/cluster/test.velero.io.json\",\n\t\t\t\t\"resources/volumesnapshotlocations.velero.io/v1-preferredversion/namespaces/foo/vsl-1.json\",\n\t\t\t},\n\t\t},\n\t}\n\n\titemBlockPool := StartItemBlockWorkerPool(t.Context(), 1, logrus.StandardLogger())\n\tdefer itemBlockPool.Stop()\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\th = newHarness(t, itemBlockPool)\n\t\t\t\treq = &Request{\n\t\t\t\t\tBackup: tc.backup,\n\t\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t\t}\n\t\t\t\tbackupFile = bytes.NewBuffer([]byte{})\n\t\t\t)\n\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.addItems(t, resource)\n\t\t\t}\n\n\t\t\th.backupper.Backup(h.log, req, backupFile, nil, nil, nil)\n\n\t\t\tassertTarballContents(t, backupFile, append(tc.want, \"metadata/version\")...)\n\t\t})\n\t}\n}\n\n// TestBackupResourceCohabitation runs backups for resources that \"cohabitate\",\n// meaning they exist in multiple API groups (e.g. deployments.extensions and\n// deployments.apps), and verifies that only one copy of each resource is backed\n// up, with preference for the non-\"extensions\" API group.\nfunc TestBackupResourceCohabitation(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1.Backup\n\t\tapiResources []*test.APIResource\n\t\twant []string\n\t}{\n\t\t{\n\t\t\tname: \"when deployments exist only in extensions, they're backed up\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.ExtensionsDeployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/deployments.extensions/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.extensions/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.extensions/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.extensions/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when deployments exist in both apps and extensions, only apps/deployments are backed up\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.ExtensionsDeployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when deployments exist that are not in the cohabiting groups those are backed up along with apps/deployments\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.VeleroDeployments(\n\t\t\t\t\tbuilder.ForTestCR(\"Deployment\", \"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForTestCR(\"Deployment\", \"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.velero.io/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.velero.io/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.velero.io/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.velero.io/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t},\n\t\t},\n\t}\n\n\titemBlockPool := StartItemBlockWorkerPool(t.Context(), 1, logrus.StandardLogger())\n\tdefer itemBlockPool.Stop()\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\th = newHarness(t, itemBlockPool)\n\t\t\t\treq = &Request{\n\t\t\t\t\tBackup: tc.backup,\n\t\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t\t}\n\t\t\t\tbackupFile = bytes.NewBuffer([]byte{})\n\t\t\t)\n\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.addItems(t, resource)\n\t\t\t}\n\n\t\t\th.backupper.Backup(h.log, req, backupFile, nil, nil, nil)\n\n\t\t\tassertTarballContents(t, backupFile, append(tc.want, \"metadata/version\")...)\n\t\t})\n\t}\n}\n\n// TestBackupUsesNewCohabitatingResourcesForEachBackup ensures that when two backups are\n// run that each include cohabiting resources, one copy of the relevant resources is\n// backed up in each backup. Verification is done by looking at the contents of the backup\n// tarball. This covers a specific issue that was fixed by https://github.com/vmware-tanzu/velero/pull/485.\nfunc TestBackupUsesNewCohabitatingResourcesForEachBackup(t *testing.T) {\n\th := newHarness(t, nil)\n\tdefer h.itemBlockPool.Stop()\n\n\t// run and verify backup 1\n\tbackup1 := &Request{\n\t\tBackup: defaultBackup().Result(),\n\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\tWorkerPool: &h.itemBlockPool,\n\t}\n\tbackup1File := bytes.NewBuffer([]byte{})\n\n\th.addItems(t, test.Deployments(builder.ForDeployment(\"ns-1\", \"deploy-1\").Result()))\n\th.addItems(t, test.ExtensionsDeployments(builder.ForDeployment(\"ns-1\", \"deploy-1\").Result()))\n\n\th.backupper.Backup(h.log, backup1, backup1File, nil, nil, nil)\n\n\tassertTarballContents(t, backup1File, \"metadata/version\", \"resources/deployments.apps/namespaces/ns-1/deploy-1.json\", \"resources/deployments.apps/v1-preferredversion/namespaces/ns-1/deploy-1.json\")\n\n\t// run and verify backup 2\n\tbackup2 := &Request{\n\t\tBackup: defaultBackup().Result(),\n\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\tWorkerPool: &h.itemBlockPool,\n\t}\n\tbackup2File := bytes.NewBuffer([]byte{})\n\n\th.backupper.Backup(h.log, backup2, backup2File, nil, nil, nil)\n\n\tassertTarballContents(t, backup2File, \"metadata/version\", \"resources/deployments.apps/namespaces/ns-1/deploy-1.json\", \"resources/deployments.apps/v1-preferredversion/namespaces/ns-1/deploy-1.json\")\n}\n\n// TestBackupResourceOrdering runs backups of the core API group and ensures that items are backed\n// up in the expected order (pods, PVCs, PVs, everything else). Verification is done by looking\n// at the order of files written to the backup tarball.\nfunc TestBackupResourceOrdering(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1.Backup\n\t\tapiResources []*test.APIResource\n\t}{\n\t\t{\n\t\t\tname: \"core API group: pods come before pvcs, pvcs come before pvs, pvs come before anything else\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tSnapshotVolumes(false).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"bar\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"baz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Secrets(\n\t\t\t\t\tbuilder.ForSecret(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForSecret(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t},\n\t}\n\n\titemBlockPool := StartItemBlockWorkerPool(t.Context(), 1, logrus.StandardLogger())\n\tdefer itemBlockPool.Stop()\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\th = newHarness(t, itemBlockPool)\n\t\t\t\treq = &Request{\n\t\t\t\t\tBackup: tc.backup,\n\t\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t\t}\n\t\t\t\tbackupFile = bytes.NewBuffer([]byte{})\n\t\t\t)\n\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.addItems(t, resource)\n\t\t\t}\n\n\t\t\th.backupper.Backup(h.log, req, backupFile, nil, nil, nil)\n\n\t\t\tassertTarballOrdering(t, backupFile, \"pods\", \"persistentvolumeclaims\", \"persistentvolumes\")\n\t\t})\n\t}\n}\n\n// recordResourcesAction is a backup item action that can be configured\n// to run for specific resources/namespaces and simply records the items\n// that it is executed for.\ntype recordResourcesAction struct {\n\tname string\n\tselector velero.ResourceSelector\n\tids []string\n\tbackups []velerov1.Backup\n\texecutionErr error\n\tadditionalItems []velero.ResourceIdentifier\n\toperationID string\n\tpostOperationItems []velero.ResourceIdentifier\n\tskippedCSISnapshot bool\n}\n\nfunc (a *recordResourcesAction) Execute(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\tmetadata, err := meta.Accessor(item)\n\tif err != nil {\n\t\treturn item, a.additionalItems, a.operationID, a.postOperationItems, err\n\t}\n\ta.ids = append(a.ids, kubeutil.NamespaceAndName(metadata))\n\ta.backups = append(a.backups, *backup)\n\tif a.skippedCSISnapshot {\n\t\tu := &unstructured.Unstructured{Object: item.UnstructuredContent()}\n\t\tu.SetAnnotations(map[string]string{velerov1.SkippedNoCSIPVAnnotation: \"true\"})\n\t\titem = u\n\t\ta.additionalItems = nil\n\t}\n\treturn item, a.additionalItems, a.operationID, a.postOperationItems, a.executionErr\n}\n\nfunc (a *recordResourcesAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn a.selector, nil\n}\n\nfunc (a *recordResourcesAction) Progress(operationID string, backup *velerov1.Backup) (velero.OperationProgress, error) {\n\treturn velero.OperationProgress{}, nil\n}\n\nfunc (a *recordResourcesAction) Cancel(operationID string, backup *velerov1.Backup) error {\n\treturn nil\n}\n\nfunc (a *recordResourcesAction) Name() string {\n\treturn a.name\n}\n\nfunc (a *recordResourcesAction) ForResource(resource string) *recordResourcesAction {\n\ta.selector.IncludedResources = append(a.selector.IncludedResources, resource)\n\treturn a\n}\n\nfunc (a *recordResourcesAction) ForNamespace(namespace string) *recordResourcesAction {\n\ta.selector.IncludedNamespaces = append(a.selector.IncludedNamespaces, namespace)\n\treturn a\n}\n\nfunc (a *recordResourcesAction) ForLabelSelector(selector string) *recordResourcesAction {\n\ta.selector.LabelSelector = selector\n\treturn a\n}\n\nfunc (a *recordResourcesAction) WithAdditionalItems(items []velero.ResourceIdentifier) *recordResourcesAction {\n\ta.additionalItems = items\n\treturn a\n}\n\nfunc (a *recordResourcesAction) WithName(name string) *recordResourcesAction {\n\ta.name = name\n\treturn a\n}\n\nfunc (a *recordResourcesAction) WithExecutionErr(executionErr error) *recordResourcesAction {\n\ta.executionErr = executionErr\n\treturn a\n}\n\nfunc (a *recordResourcesAction) WithSkippedCSISnapshotFlag(flag bool) *recordResourcesAction {\n\ta.skippedCSISnapshot = flag\n\treturn a\n}\n\n// TestBackupItemActionsForSkippedPV runs backups with backup item actions, and\n// verifies that the data in SkippedPVTracker is updated as expected.\nfunc TestBackupItemActionsForSkippedPV(t *testing.T) {\n\titemBlockPool := StartItemBlockWorkerPool(t.Context(), 1, logrus.StandardLogger())\n\tdefer itemBlockPool.Stop()\n\n\ttests := []struct {\n\t\tname string\n\t\tbackupReq *Request\n\t\tapiResources []*test.APIResource\n\t\truntimeResources []runtime.Object\n\t\tactions []*recordResourcesAction\n\t\tresPolicies *resourcepolicies.ResourcePolicies\n\t\t// {pvName:{approach: reason}}\n\t\texpectSkippedPVs map[string]map[string]string\n\t\texpectNotSkippedPVs []string\n\t}{\n\t\t{\n\t\t\tname: \"backup item action returns the 'not a CSI volume' error and the PV should be tracked as skippedPV\",\n\t\t\tbackupReq: &Request{\n\t\t\t\tBackup: defaultBackup().SnapshotVolumes(false).Result(),\n\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t},\n\t\t\tresPolicies: &resourcepolicies.ResourcePolicies{\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t\t\t{\n\t\t\t\t\t\tAction: resourcepolicies.Action{Type: \"snapshot\"},\n\t\t\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\t\t\"storageClass\": []string{\"gp2\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").StorageClass(\"gp2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").VolumeName(\"pv-1\").StorageClass(\"gp2\").Phase(corev1api.ClaimBound).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\truntimeResources: []runtime.Object{\n\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").StorageClass(\"gp2\").Result(),\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").VolumeName(\"pv-1\").StorageClass(\"gp2\").Phase(corev1api.ClaimBound).Result(),\n\t\t\t},\n\t\t\tactions: []*recordResourcesAction{\n\t\t\t\tnew(recordResourcesAction).WithName(csiBIAPluginName).ForNamespace(\"ns-1\").ForResource(\"persistentvolumeclaims\").WithSkippedCSISnapshotFlag(true),\n\t\t\t},\n\t\t\texpectSkippedPVs: map[string]map[string]string{\n\t\t\t\t\"pv-1\": {\n\t\t\t\t\tcsiSnapshotApproach: \"skipped b/c it's not a CSI volume\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"backup item action named as CSI plugin executed successfully and the PV will be removed from the skipped PV tracker\",\n\t\t\tbackupReq: &Request{\n\t\t\t\tBackup: defaultBackup().Result(),\n\t\t\t\tSkippedPVTracker: &skipPVTracker{\n\t\t\t\t\tRWMutex: &sync.RWMutex{},\n\t\t\t\t\tpvs: map[string]map[string]string{\n\t\t\t\t\t\t\"pv-1\": {\n\t\t\t\t\t\t\t\"any\": \"whatever reason\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tincludedPVs: map[string]struct{}{},\n\t\t\t\t},\n\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t},\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").VolumeName(\"pv-1\").Phase(corev1api.ClaimBound).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\truntimeResources: []runtime.Object{\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").VolumeName(\"pv-1\").Phase(corev1api.ClaimBound).Result(),\n\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").StorageClass(\"gp2\").Result(),\n\t\t\t},\n\t\t\tactions: []*recordResourcesAction{\n\t\t\t\tnew(recordResourcesAction).ForNamespace(\"ns-1\").ForResource(\"persistentvolumeclaims\").WithName(csiBIAPluginName),\n\t\t\t},\n\t\t\texpectNotSkippedPVs: []string{\"pv-1\"},\n\t\t},\n\t}\n\t// Enable CSI feature before running the test, because Velero will check whether\n\t// CSI feature is enabled before executing CSI plugin actions.\n\tfeatures.NewFeatureFlagSet(\"EnableCSI\")\n\tdefer func() {\n\t\tfeatures.NewFeatureFlagSet(\"\")\n\t}()\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(tt *testing.T) {\n\t\t\tvar (\n\t\t\t\th = newHarness(t, itemBlockPool)\n\t\t\t\tbackupFile = bytes.NewBuffer([]byte{})\n\t\t\t\tfakeClient = test.NewFakeControllerRuntimeClient(t, tc.runtimeResources...)\n\t\t\t)\n\t\t\th.backupper.kbClient = fakeClient\n\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.addItems(t, resource)\n\t\t\t}\n\n\t\t\tactions := []biav2.BackupItemAction{}\n\t\t\tfor _, action := range tc.actions {\n\t\t\t\tactions = append(actions, action)\n\t\t\t}\n\n\t\t\tif tc.resPolicies != nil {\n\t\t\t\ttc.backupReq.ResPolicies = new(resourcepolicies.Policies)\n\t\t\t\trequire.NoError(t, tc.backupReq.ResPolicies.BuildPolicy(tc.resPolicies))\n\t\t\t}\n\n\t\t\terr := h.backupper.Backup(h.log, tc.backupReq, backupFile, actions, nil, nil)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tif tc.expectSkippedPVs != nil {\n\t\t\t\tfor pvName, reasons := range tc.expectSkippedPVs {\n\t\t\t\t\tv, ok := tc.backupReq.SkippedPVTracker.pvs[pvName]\n\t\t\t\t\tassert.True(tt, ok)\n\t\t\t\t\tfor approach, reason := range reasons {\n\t\t\t\t\t\tassert.Equal(tt, reason, v[approach])\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\tfor _, pvName := range tc.expectNotSkippedPVs {\n\t\t\t\t_, ok := tc.backupReq.SkippedPVTracker.pvs[pvName]\n\t\t\t\tassert.False(tt, ok)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestBackupActionsRunForCorrectItems runs backups with backup item actions, and\n// verifies that each backup item action is run for the correct set of resources based on its\n// AppliesTo() resource selector. Verification is done by using the recordResourcesAction struct,\n// which records which resources it's executed for.\nfunc TestBackupActionsRunForCorrectItems(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1.Backup\n\t\tapiResources []*test.APIResource\n\n\t\t// actions is a map from a recordResourcesAction (which will record the items it was called for)\n\t\t// to a slice of expected items, formatted as {namespace}/{name}.\n\t\tactions map[*recordResourcesAction][]string\n\t}{\n\t\t{\n\t\t\tname: \"single action with no selector runs for all items\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction): {\"ns-1/pod-1\", \"ns-2/pod-2\", \"pv-1\", \"pv-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"single action with a resource selector for namespaced resources runs only for matching resources\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForResource(\"pods\"): {\"ns-1/pod-1\", \"ns-2/pod-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"single action with a resource selector for cluster-scoped resources runs only for matching resources\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForResource(\"persistentvolumes\"): {\"pv-1\", \"pv-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"single action with a namespace selector runs only for resources in that namespace\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns-2\", \"pvc-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Namespaces(\n\t\t\t\t\tbuilder.ForNamespace(\"ns-1\").Result(),\n\t\t\t\t\tbuilder.ForNamespace(\"ns-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForNamespace(\"ns-1\"): {\"ns-1/pod-1\", \"ns-1/pvc-1\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"single action with a resource and namespace selector runs only for matching resources\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForResource(\"pods\").ForNamespace(\"ns-1\"): {\"ns-1/pod-1\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"multiple actions, each with a different resource selector using short name, run for matching resources\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForResource(\"po\"): {\"ns-1/pod-1\", \"ns-2/pod-2\"},\n\t\t\t\tnew(recordResourcesAction).ForResource(\"pv\"): {\"pv-1\", \"pv-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"actions with selectors that don't match anything don't run for any resources\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns-2\", \"pvc-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForNamespace(\"ns-1\").ForResource(\"persistentvolumeclaims\"): nil,\n\t\t\t\tnew(recordResourcesAction).ForNamespace(\"ns-2\").ForResource(\"pods\"): nil,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"action with a selector that has unresolvable resources doesn't run for any resources\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns-2\", \"pvc-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForResource(\"unresolvable\"): nil,\n\t\t\t},\n\t\t},\n\t}\n\n\titemBlockPool := StartItemBlockWorkerPool(t.Context(), 1, logrus.StandardLogger())\n\tdefer itemBlockPool.Stop()\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\th = newHarness(t, itemBlockPool)\n\t\t\t\treq = &Request{\n\t\t\t\t\tBackup: tc.backup,\n\t\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t\t}\n\t\t\t\tbackupFile = bytes.NewBuffer([]byte{})\n\t\t\t)\n\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.addItems(t, resource)\n\t\t\t}\n\n\t\t\tactions := []biav2.BackupItemAction{}\n\t\t\tfor action := range tc.actions {\n\t\t\t\tactions = append(actions, action)\n\t\t\t}\n\n\t\t\terr := h.backupper.Backup(h.log, req, backupFile, actions, nil, nil)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tfor action, want := range tc.actions {\n\t\t\t\tassert.Equal(t, want, action.ids)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestBackupWithInvalidActions runs backups with backup item actions that are invalid\n// in some way (e.g. an invalid label selector returned from AppliesTo(), an error returned\n// from AppliesTo()) and verifies that this causes the backupper.Backup(...) method to\n// return an error.\nfunc TestBackupWithInvalidActions(t *testing.T) {\n\t// all test cases in this function are expected to cause the method under test\n\t// to return an error, so no expected results need to be set up.\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1.Backup\n\t\tapiResources []*test.APIResource\n\t\tactions []biav2.BackupItemAction\n\t}{\n\t\t{\n\t\t\tname: \"action with invalid label selector results in an error\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"bar\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"baz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\tnew(recordResourcesAction).ForLabelSelector(\"=invalid-selector\"),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"action returning an error from AppliesTo results in an error\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"bar\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"baz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\t&appliesToErrorAction{},\n\t\t\t},\n\t\t},\n\t}\n\n\titemBlockPool := StartItemBlockWorkerPool(t.Context(), 1, logrus.StandardLogger())\n\tdefer itemBlockPool.Stop()\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\th = newHarness(t, itemBlockPool)\n\t\t\t\treq = &Request{\n\t\t\t\t\tBackup: tc.backup,\n\t\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t\t}\n\t\t\t\tbackupFile = bytes.NewBuffer([]byte{})\n\t\t\t)\n\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.addItems(t, resource)\n\t\t\t}\n\n\t\t\tassert.Error(t, h.backupper.Backup(h.log, req, backupFile, tc.actions, nil, nil))\n\t\t})\n\t}\n}\n\n// appliesToErrorAction is a backup item action that always returns\n// an error when AppliesTo() is called.\ntype appliesToErrorAction struct{}\n\nfunc (a *appliesToErrorAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{}, errors.New(\"error calling AppliesTo\")\n}\n\nfunc (a *appliesToErrorAction) Execute(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\tpanic(\"not implemented\")\n}\n\nfunc (a *appliesToErrorAction) GetRelatedItems(item runtime.Unstructured, backup *velerov1.Backup) ([]velero.ResourceIdentifier, error) {\n\tpanic(\"not implemented\")\n}\n\nfunc (a *appliesToErrorAction) Progress(operationID string, backup *velerov1.Backup) (velero.OperationProgress, error) {\n\tpanic(\"not implemented\")\n}\n\nfunc (a *appliesToErrorAction) Cancel(operationID string, backup *velerov1.Backup) error {\n\tpanic(\"not implemented\")\n}\n\nfunc (a *appliesToErrorAction) Name() string {\n\treturn \"\"\n}\n\n// TestBackupActionModifications runs backups with backup item actions that make modifications\n// to items in their Execute(...) methods and verifies that these modifications are\n// persisted to the backup tarball. Verification is done by inspecting the file contents\n// of the tarball.\nfunc TestBackupActionModifications(t *testing.T) {\n\t// modifyingActionGetter is a helper function that returns a *pluggableAction, whose Execute(...)\n\t// method modifies the item being passed in by calling the 'modify' function on it.\n\tmodifyingActionGetter := func(modify func(*unstructured.Unstructured)) *pluggableAction {\n\t\treturn &pluggableAction{\n\t\t\texecuteFunc: func(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\t\t\t\tobj, ok := item.(*unstructured.Unstructured)\n\t\t\t\tif !ok {\n\t\t\t\t\treturn nil, nil, \"\", nil, errors.Errorf(\"unexpected type %T\", item)\n\t\t\t\t}\n\n\t\t\t\tres := obj.DeepCopy()\n\t\t\t\tmodify(res)\n\n\t\t\t\treturn res, nil, \"\", nil, nil\n\t\t\t},\n\t\t}\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1.Backup\n\t\tapiResources []*test.APIResource\n\t\tactions []biav2.BackupItemAction\n\t\twant map[string]unstructuredObject\n\t}{\n\t\t{\n\t\t\tname: \"action that adds a label to item gets persisted\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\tmodifyingActionGetter(func(item *unstructured.Unstructured) {\n\t\t\t\t\titem.SetLabels(map[string]string{\"updated\": \"true\"})\n\t\t\t\t}),\n\t\t\t},\n\t\t\twant: map[string]unstructuredObject{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\": toUnstructuredOrFail(t, builder.ForPod(\"ns-1\", \"pod-1\").ObjectMeta(builder.WithLabels(\"updated\", \"true\")).Result()),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"action that removes labels from item gets persisted\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").ObjectMeta(builder.WithLabels(\"should-be-removed\", \"true\")).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\tmodifyingActionGetter(func(item *unstructured.Unstructured) {\n\t\t\t\t\titem.SetLabels(nil)\n\t\t\t\t}),\n\t\t\t},\n\t\t\twant: map[string]unstructuredObject{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\": toUnstructuredOrFail(t, builder.ForPod(\"ns-1\", \"pod-1\").Result()),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"action that sets a spec field on item gets persisted\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\tmodifyingActionGetter(func(item *unstructured.Unstructured) {\n\t\t\t\t\titem.Object[\"spec\"].(map[string]any)[\"nodeName\"] = \"foo\"\n\t\t\t\t}),\n\t\t\t},\n\t\t\twant: map[string]unstructuredObject{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\": toUnstructuredOrFail(t, builder.ForPod(\"ns-1\", \"pod-1\").NodeName(\"foo\").Result()),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"modifications to name and namespace in an action are persisted in JSON and in filename\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\tmodifyingActionGetter(func(item *unstructured.Unstructured) {\n\t\t\t\t\titem.SetName(item.GetName() + \"-updated\")\n\t\t\t\t\titem.SetNamespace(item.GetNamespace() + \"-updated\")\n\t\t\t\t}),\n\t\t\t},\n\t\t\twant: map[string]unstructuredObject{\n\t\t\t\t\"resources/pods/namespaces/ns-1-updated/pod-1-updated.json\": toUnstructuredOrFail(t, builder.ForPod(\"ns-1-updated\", \"pod-1-updated\").Result()),\n\t\t\t},\n\t\t},\n\t}\n\n\titemBlockPool := StartItemBlockWorkerPool(t.Context(), 1, logrus.StandardLogger())\n\tdefer itemBlockPool.Stop()\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\th = newHarness(t, itemBlockPool)\n\t\t\t\treq = &Request{\n\t\t\t\t\tBackup: tc.backup,\n\t\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t\t}\n\t\t\t\tbackupFile = bytes.NewBuffer([]byte{})\n\t\t\t)\n\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.addItems(t, resource)\n\t\t\t}\n\n\t\t\terr := h.backupper.Backup(h.log, req, backupFile, tc.actions, nil, nil)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassertTarballFileContents(t, backupFile, tc.want)\n\t\t})\n\t}\n}\n\n// TestBackupActionAdditionalItems runs backups with backup item actions that return\n// additional items to be backed up, and verifies that those items are included in the\n// backup tarball as appropriate. Verification is done by looking at the files that exist\n// in the backup tarball.\nfunc TestBackupActionAdditionalItems(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1.Backup\n\t\tapiResources []*test.APIResource\n\t\tactions []biav2.BackupItemAction\n\t\tibActions []ibav1.ItemBlockAction\n\t\twant []string\n\t}{\n\t\t{\n\t\t\tname: \"additional items that are already being backed up are not backed up twice\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-3\", \"pod-3\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\t&pluggableAction{\n\t\t\t\t\tselector: velero.ResourceSelector{IncludedNamespaces: []string{\"ns-1\"}},\n\t\t\t\t\texecuteFunc: func(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\t\t\t\t\t\tadditionalItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"ns-2\", Name: \"pod-2\"},\n\t\t\t\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"ns-3\", Name: \"pod-3\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn item, additionalItems, \"\", nil, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-2/pod-2.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-3/pod-3.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-2/pod-2.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-3/pod-3.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when using a backup namespace filter, additional items that are in a non-included namespace are not backed up\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"ns-1\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-3\", \"pod-3\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\t&pluggableAction{\n\t\t\t\t\texecuteFunc: func(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\t\t\t\t\t\tadditionalItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"ns-2\", Name: \"pod-2\"},\n\t\t\t\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"ns-3\", Name: \"pod-3\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn item, additionalItems, \"\", nil, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when using a backup namespace filter, additional items that are cluster-scoped are backed up\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"ns-1\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\t&pluggableAction{\n\t\t\t\t\texecuteFunc: func(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\t\t\t\t\t\tadditionalItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"pv-1\"},\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"pv-2\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn item, additionalItems, \"\", nil, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/pv-1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/pv-2.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/pv-1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/pv-2.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when using a backup resource filter, additional items that are non-included resources are not backed up\",\n\t\t\tbackup: defaultBackup().IncludedResources(\"pods\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\t&pluggableAction{\n\t\t\t\t\texecuteFunc: func(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\t\t\t\t\t\tadditionalItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"pv-1\"},\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"pv-2\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn item, additionalItems, \"\", nil, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when IncludeClusterResources=false, additional items that are cluster-scoped are not backed up\",\n\t\t\tbackup: defaultBackup().IncludeClusterResources(false).Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\t&pluggableAction{\n\t\t\t\t\texecuteFunc: func(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\t\t\t\t\t\tadditionalItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"pv-1\"},\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"pv-2\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn item, additionalItems, \"\", nil, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-2/pod-2.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-2/pod-2.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"additional items with the velero.io/exclude-from-backup label are not backed up\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"ns-1\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ObjectMeta(builder.WithLabels(velerov1.ExcludeFromBackupLabel, \"true\")).Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\t&pluggableAction{\n\t\t\t\t\texecuteFunc: func(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\t\t\t\t\t\tadditionalItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"pv-1\"},\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"pv-2\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn item, additionalItems, \"\", nil, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/pv-2.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/pv-2.json\",\n\t\t\t},\n\t\t},\n\n\t\t{\n\t\t\tname: \"if additional items aren't found in the API, they're skipped and the original item is still backed up\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-3\", \"pod-3\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\t&pluggableAction{\n\t\t\t\t\tselector: velero.ResourceSelector{IncludedNamespaces: []string{\"ns-1\"}},\n\t\t\t\t\texecuteFunc: func(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\t\t\t\t\t\tadditionalItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"ns-4\", Name: \"pod-4\"},\n\t\t\t\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"ns-5\", Name: \"pod-5\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn item, additionalItems, \"\", nil, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-2/pod-2.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-3/pod-3.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-2/pod-2.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-3/pod-3.json\",\n\t\t\t},\n\t\t},\n\t}\n\n\titemBlockPool := StartItemBlockWorkerPool(t.Context(), 1, logrus.StandardLogger())\n\tdefer itemBlockPool.Stop()\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\th = newHarness(t, itemBlockPool)\n\t\t\t\treq = &Request{\n\t\t\t\t\tBackup: tc.backup,\n\t\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t\t}\n\t\t\t\tbackupFile = bytes.NewBuffer([]byte{})\n\t\t\t)\n\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.addItems(t, resource)\n\t\t\t}\n\n\t\t\terr := h.backupper.Backup(h.log, req, backupFile, tc.actions, nil, nil)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassertTarballContents(t, backupFile, append(tc.want, \"metadata/version\")...)\n\t\t})\n\t}\n}\n\n// recordResourcesIBA is an ItemBlock item action that can be configured\n// to run for specific resources/namespaces and simply records the items\n// that it is executed for.\ntype recordResourcesIBA struct {\n\tname string\n\tselector velero.ResourceSelector\n\tids []string\n\tbackups []velerov1.Backup\n\texecutionErr error\n\trelatedItems []velero.ResourceIdentifier\n}\n\nfunc (a *recordResourcesIBA) GetRelatedItems(item runtime.Unstructured, backup *velerov1.Backup) ([]velero.ResourceIdentifier, error) {\n\tmetadata, err := meta.Accessor(item)\n\tif err != nil {\n\t\treturn a.relatedItems, err\n\t}\n\ta.ids = append(a.ids, kubeutil.NamespaceAndName(metadata))\n\ta.backups = append(a.backups, *backup)\n\treturn a.relatedItems, a.executionErr\n}\n\nfunc (a *recordResourcesIBA) AppliesTo() (velero.ResourceSelector, error) {\n\treturn a.selector, nil\n}\n\nfunc (a *recordResourcesIBA) Name() string {\n\treturn a.name\n}\n\nfunc (a *recordResourcesIBA) ForResource(resource string) *recordResourcesIBA {\n\ta.selector.IncludedResources = append(a.selector.IncludedResources, resource)\n\treturn a\n}\n\nfunc (a *recordResourcesIBA) ForNamespace(namespace string) *recordResourcesIBA {\n\ta.selector.IncludedNamespaces = append(a.selector.IncludedNamespaces, namespace)\n\treturn a\n}\n\nfunc (a *recordResourcesIBA) ForLabelSelector(selector string) *recordResourcesIBA {\n\ta.selector.LabelSelector = selector\n\treturn a\n}\n\nfunc (a *recordResourcesIBA) WithRelatedItems(items []velero.ResourceIdentifier) *recordResourcesIBA {\n\ta.relatedItems = items\n\treturn a\n}\n\nfunc (a *recordResourcesIBA) WithName(name string) *recordResourcesIBA {\n\ta.name = name\n\treturn a\n}\n\nfunc (a *recordResourcesIBA) WithExecutionErr(executionErr error) *recordResourcesIBA {\n\ta.executionErr = executionErr\n\treturn a\n}\n\n// TestItemBlockActionsRunForCorrectItems runs backups with ItemBlock actions, and\n// verifies that each action is run for the correct set of resources based on its\n// AppliesTo() resource selector. Verification is done by using the recordResourcesIBA struct,\n// which records which resources it's executed for.\nfunc TestItemBlockActionsRunForCorrectItems(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1.Backup\n\t\tapiResources []*test.APIResource\n\n\t\t// actions is a map from a recordResourcesIBA (which will record the items it was called for)\n\t\t// to a slice of expected items, formatted as {namespace}/{name}.\n\t\tactions map[*recordResourcesIBA][]string\n\t}{\n\t\t{\n\t\t\tname: \"single action with no selector runs for all items\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: map[*recordResourcesIBA][]string{\n\t\t\t\tnew(recordResourcesIBA): {\"ns-1/pod-1\", \"ns-2/pod-2\", \"pv-1\", \"pv-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"single action with a resource selector for namespaced resources runs only for matching resources\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: map[*recordResourcesIBA][]string{\n\t\t\t\tnew(recordResourcesIBA).ForResource(\"pods\"): {\"ns-1/pod-1\", \"ns-2/pod-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"single action with a resource selector for cluster-scoped resources runs only for matching resources\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: map[*recordResourcesIBA][]string{\n\t\t\t\tnew(recordResourcesIBA).ForResource(\"persistentvolumes\"): {\"pv-1\", \"pv-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"single action with a namespace selector runs only for resources in that namespace\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns-2\", \"pvc-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Namespaces(\n\t\t\t\t\tbuilder.ForNamespace(\"ns-1\").Result(),\n\t\t\t\t\tbuilder.ForNamespace(\"ns-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: map[*recordResourcesIBA][]string{\n\t\t\t\tnew(recordResourcesIBA).ForNamespace(\"ns-1\"): {\"ns-1/pod-1\", \"ns-1/pvc-1\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"single action with a resource and namespace selector runs only for matching resources\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: map[*recordResourcesIBA][]string{\n\t\t\t\tnew(recordResourcesIBA).ForResource(\"pods\").ForNamespace(\"ns-1\"): {\"ns-1/pod-1\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"multiple actions, each with a different resource selector using short name, run for matching resources\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: map[*recordResourcesIBA][]string{\n\t\t\t\tnew(recordResourcesIBA).ForResource(\"po\"): {\"ns-1/pod-1\", \"ns-2/pod-2\"},\n\t\t\t\tnew(recordResourcesIBA).ForResource(\"pv\"): {\"pv-1\", \"pv-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"actions with selectors that don't match anything don't run for any resources\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns-2\", \"pvc-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: map[*recordResourcesIBA][]string{\n\t\t\t\tnew(recordResourcesIBA).ForNamespace(\"ns-1\").ForResource(\"persistentvolumeclaims\"): nil,\n\t\t\t\tnew(recordResourcesIBA).ForNamespace(\"ns-2\").ForResource(\"pods\"): nil,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"action with a selector that has unresolvable resources doesn't run for any resources\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns-2\", \"pvc-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: map[*recordResourcesIBA][]string{\n\t\t\t\tnew(recordResourcesIBA).ForResource(\"unresolvable\"): nil,\n\t\t\t},\n\t\t},\n\t}\n\n\titemBlockPool := StartItemBlockWorkerPool(t.Context(), 1, logrus.StandardLogger())\n\tdefer itemBlockPool.Stop()\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\th = newHarness(t, itemBlockPool)\n\t\t\t\treq = &Request{\n\t\t\t\t\tBackup: tc.backup,\n\t\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t\t}\n\t\t\t\tbackupFile = bytes.NewBuffer([]byte{})\n\t\t\t)\n\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.addItems(t, resource)\n\t\t\t}\n\n\t\t\tactions := []ibav1.ItemBlockAction{}\n\t\t\tfor action := range tc.actions {\n\t\t\t\tactions = append(actions, action)\n\t\t\t}\n\n\t\t\terr := h.backupper.Backup(h.log, req, backupFile, nil, actions, nil)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tfor action, want := range tc.actions {\n\t\t\t\tassert.Equal(t, want, action.ids)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestBackupWithInvalidItemBlockActions runs backups with ItemBlock actions that are invalid\n// in some way (e.g. an invalid label selector returned from AppliesTo(), an error returned\n// from AppliesTo()) and verifies that this causes the backupper.Backup(...) method to\n// return an error.\nfunc TestBackupWithInvalidItemBlockActions(t *testing.T) {\n\t// all test cases in this function are expected to cause the method under test\n\t// to return an error, so no expected results need to be set up.\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1.Backup\n\t\tapiResources []*test.APIResource\n\t\tactions []ibav1.ItemBlockAction\n\t}{\n\t\t{\n\t\t\tname: \"action with invalid label selector results in an error\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"bar\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"baz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []ibav1.ItemBlockAction{\n\t\t\t\tnew(recordResourcesIBA).ForLabelSelector(\"=invalid-selector\"),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"action returning an error from AppliesTo results in an error\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"bar\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"baz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []ibav1.ItemBlockAction{\n\t\t\t\t&appliesToErrorAction{},\n\t\t\t},\n\t\t},\n\t}\n\n\titemBlockPool := StartItemBlockWorkerPool(t.Context(), 1, logrus.StandardLogger())\n\tdefer itemBlockPool.Stop()\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\th = newHarness(t, itemBlockPool)\n\t\t\t\treq = &Request{\n\t\t\t\t\tBackup: tc.backup,\n\t\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t\t}\n\t\t\t\tbackupFile = bytes.NewBuffer([]byte{})\n\t\t\t)\n\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.addItems(t, resource)\n\t\t\t}\n\n\t\t\tassert.Error(t, h.backupper.Backup(h.log, req, backupFile, nil, tc.actions, nil))\n\t\t})\n\t}\n}\n\n// TestItemBlockActionRelatedItems runs backups with ItemBlock actions that return\n// related items, and verifies that those items are included in the\n// backup tarball as appropriate. Verification is done by looking at the files that exist\n// in the backup tarball.\nfunc TestItemBlockActionRelatedItems(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1.Backup\n\t\tapiResources []*test.APIResource\n\t\tactions []ibav1.ItemBlockAction\n\t\twant []string\n\t}{\n\t\t{\n\t\t\tname: \"related items that are already being backed up are not backed up twice\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-3\", \"pod-3\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []ibav1.ItemBlockAction{\n\t\t\t\t&pluggableIBA{\n\t\t\t\t\tselector: velero.ResourceSelector{IncludedNamespaces: []string{\"ns-1\"}},\n\t\t\t\t\tgetRelatedItemsFunc: func(item runtime.Unstructured, backup *velerov1.Backup) ([]velero.ResourceIdentifier, error) {\n\t\t\t\t\t\trelatedItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"ns-2\", Name: \"pod-2\"},\n\t\t\t\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"ns-3\", Name: \"pod-3\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn relatedItems, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-2/pod-2.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-3/pod-3.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-2/pod-2.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-3/pod-3.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when using a backup namespace filter, related items that are in a non-included namespace are not backed up\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"ns-1\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-3\", \"pod-3\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []ibav1.ItemBlockAction{\n\t\t\t\t&pluggableIBA{\n\t\t\t\t\tgetRelatedItemsFunc: func(item runtime.Unstructured, backup *velerov1.Backup) ([]velero.ResourceIdentifier, error) {\n\t\t\t\t\t\trelatedItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"ns-2\", Name: \"pod-2\"},\n\t\t\t\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"ns-3\", Name: \"pod-3\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn relatedItems, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when using a backup namespace filter, related items that are cluster-scoped are backed up\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"ns-1\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []ibav1.ItemBlockAction{\n\t\t\t\t&pluggableIBA{\n\t\t\t\t\tgetRelatedItemsFunc: func(item runtime.Unstructured, backup *velerov1.Backup) ([]velero.ResourceIdentifier, error) {\n\t\t\t\t\t\trelatedItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"pv-1\"},\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"pv-2\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn relatedItems, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/pv-1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/pv-2.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/pv-1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/pv-2.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when using a backup resource filter, related items that are non-included resources are not backed up\",\n\t\t\tbackup: defaultBackup().IncludedResources(\"pods\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []ibav1.ItemBlockAction{\n\t\t\t\t&pluggableIBA{\n\t\t\t\t\tgetRelatedItemsFunc: func(item runtime.Unstructured, backup *velerov1.Backup) ([]velero.ResourceIdentifier, error) {\n\t\t\t\t\t\trelatedItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"pv-1\"},\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"pv-2\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn relatedItems, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when IncludeClusterResources=false, related items that are cluster-scoped are not backed up\",\n\t\t\tbackup: defaultBackup().IncludeClusterResources(false).Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []ibav1.ItemBlockAction{\n\t\t\t\t&pluggableIBA{\n\t\t\t\t\tgetRelatedItemsFunc: func(item runtime.Unstructured, backup *velerov1.Backup) ([]velero.ResourceIdentifier, error) {\n\t\t\t\t\t\trelatedItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"pv-1\"},\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"pv-2\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn relatedItems, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-2/pod-2.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-2/pod-2.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"related items with the velero.io/exclude-from-backup label are not backed up\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"ns-1\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ObjectMeta(builder.WithLabels(velerov1.ExcludeFromBackupLabel, \"true\")).Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []ibav1.ItemBlockAction{\n\t\t\t\t&pluggableIBA{\n\t\t\t\t\tgetRelatedItemsFunc: func(item runtime.Unstructured, backup *velerov1.Backup) ([]velero.ResourceIdentifier, error) {\n\t\t\t\t\t\trelatedItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"pv-1\"},\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"pv-2\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn relatedItems, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/pv-2.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/pv-2.json\",\n\t\t\t},\n\t\t},\n\n\t\t{\n\t\t\tname: \"if related items aren't found in the API, they're skipped and the original item is still backed up\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-3\", \"pod-3\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []ibav1.ItemBlockAction{\n\t\t\t\t&pluggableIBA{\n\t\t\t\t\tselector: velero.ResourceSelector{IncludedNamespaces: []string{\"ns-1\"}},\n\t\t\t\t\tgetRelatedItemsFunc: func(item runtime.Unstructured, backup *velerov1.Backup) ([]velero.ResourceIdentifier, error) {\n\t\t\t\t\t\trelatedItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"ns-4\", Name: \"pod-4\"},\n\t\t\t\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"ns-5\", Name: \"pod-5\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn relatedItems, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-2/pod-2.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-3/pod-3.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-2/pod-2.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-3/pod-3.json\",\n\t\t\t},\n\t\t},\n\t}\n\n\titemBlockPool := StartItemBlockWorkerPool(t.Context(), 1, logrus.StandardLogger())\n\tdefer itemBlockPool.Stop()\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\th = newHarness(t, itemBlockPool)\n\t\t\t\treq = &Request{\n\t\t\t\t\tBackup: tc.backup,\n\t\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t\t}\n\t\t\t\tbackupFile = bytes.NewBuffer([]byte{})\n\t\t\t)\n\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.addItems(t, resource)\n\t\t\t}\n\n\t\t\terr := h.backupper.Backup(h.log, req, backupFile, nil, tc.actions, nil)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassertTarballContents(t, backupFile, append(tc.want, \"metadata/version\")...)\n\t\t})\n\t}\n}\n\n// volumeSnapshotterGetter is a simple implementation of the VolumeSnapshotterGetter\n// interface that returns vsv1.VolumeSnapshotters from a map if they exist.\ntype volumeSnapshotterGetter map[string]vsv1.VolumeSnapshotter\n\nfunc (vsg volumeSnapshotterGetter) GetVolumeSnapshotter(name string) (vsv1.VolumeSnapshotter, error) {\n\tsnapshotter, ok := vsg[name]\n\tif !ok {\n\t\treturn nil, errors.New(\"volume snapshotter not found\")\n\t}\n\n\treturn snapshotter, nil\n}\n\nfunc int64Ptr(val int) *int64 {\n\ti := int64(val)\n\treturn &i\n}\n\ntype volumeIdentifier struct {\n\tvolumeID string\n\tvolumeAZ string\n}\n\ntype volumeInfo struct {\n\tvolumeType string\n\tiops *int64\n\tsnapshotErr bool\n}\n\n// fakeVolumeSnapshotter is a test fake for the vsv1.VolumeSnapshotter interface.\ntype fakeVolumeSnapshotter struct {\n\t// PVVolumeNames is a map from PV name to volume ID, used as the basis\n\t// for the GetVolumeID method.\n\tPVVolumeNames map[string]string\n\n\t// Volumes is a map from volume identifier (volume ID + AZ) to a struct\n\t// of volume info, used for the GetVolumeInfo and CreateSnapshot methods.\n\tVolumes map[volumeIdentifier]*volumeInfo\n}\n\n// WithVolume is a test helper for registering persistent volumes that the\n// fakeVolumeSnapshotter should handle.\nfunc (vs *fakeVolumeSnapshotter) WithVolume(pvName, id, az, volumeType string, iops int, snapshotErr bool) *fakeVolumeSnapshotter {\n\tif vs.PVVolumeNames == nil {\n\t\tvs.PVVolumeNames = make(map[string]string)\n\t}\n\tvs.PVVolumeNames[pvName] = id\n\n\tif vs.Volumes == nil {\n\t\tvs.Volumes = make(map[volumeIdentifier]*volumeInfo)\n\t}\n\n\tidentifier := volumeIdentifier{\n\t\tvolumeID: id,\n\t\tvolumeAZ: az,\n\t}\n\n\tvs.Volumes[identifier] = &volumeInfo{\n\t\tvolumeType: volumeType,\n\t\tiops: int64Ptr(iops),\n\t\tsnapshotErr: snapshotErr,\n\t}\n\n\treturn vs\n}\n\n// Init is a no-op.\nfunc (*fakeVolumeSnapshotter) Init(config map[string]string) error {\n\treturn nil\n}\n\n// GetVolumeID looks up the PV name in the PVVolumeNames map and returns the result\n// if found, or an error otherwise.\nfunc (vs *fakeVolumeSnapshotter) GetVolumeID(pv runtime.Unstructured) (string, error) {\n\tobj := pv.(*unstructured.Unstructured)\n\n\tvolumeID, ok := vs.PVVolumeNames[obj.GetName()]\n\tif !ok {\n\t\treturn \"\", errors.New(\"unsupported volume type\")\n\t}\n\n\treturn volumeID, nil\n}\n\n// CreateSnapshot looks up the volume in the Volume map. If it's not found, an error is\n// returned; if snapshotErr is true on the result, an error is returned; otherwise,\n// a snapshotID of \"-snapshot\" is returned.\nfunc (vs *fakeVolumeSnapshotter) CreateSnapshot(volumeID, volumeAZ string, tags map[string]string) (snapshotID string, err error) {\n\tvi, ok := vs.Volumes[volumeIdentifier{volumeID: volumeID, volumeAZ: volumeAZ}]\n\tif !ok {\n\t\treturn \"\", errors.New(\"volume not found\")\n\t}\n\n\tif vi.snapshotErr {\n\t\treturn \"\", errors.New(\"error calling CreateSnapshot\")\n\t}\n\n\treturn volumeID + \"-snapshot\", nil\n}\n\n// GetVolumeInfo returns volume info if it exists in the Volumes map\n// for the specified volume ID and AZ, or an error otherwise.\nfunc (vs *fakeVolumeSnapshotter) GetVolumeInfo(volumeID, volumeAZ string) (string, *int64, error) {\n\tvi, ok := vs.Volumes[volumeIdentifier{volumeID: volumeID, volumeAZ: volumeAZ}]\n\tif !ok {\n\t\treturn \"\", nil, errors.New(\"volume not found\")\n\t}\n\n\treturn vi.volumeType, vi.iops, nil\n}\n\n// CreateVolumeFromSnapshot panics because it's not expected to be used for backups.\nfunc (*fakeVolumeSnapshotter) CreateVolumeFromSnapshot(snapshotID, volumeType, volumeAZ string, iops *int64) (volumeID string, err error) {\n\tpanic(\"CreateVolumeFromSnapshot should not be used for backups\")\n}\n\n// SetVolumeID panics because it's not expected to be used for backups.\nfunc (*fakeVolumeSnapshotter) SetVolumeID(pv runtime.Unstructured, volumeID string) (runtime.Unstructured, error) {\n\tpanic(\"SetVolumeID should not be used for backups\")\n}\n\n// DeleteSnapshot panics because it's not expected to be used for backups.\nfunc (*fakeVolumeSnapshotter) DeleteSnapshot(snapshotID string) error {\n\tpanic(\"DeleteSnapshot should not be used for backups\")\n}\n\n// TestBackupWithSnapshots runs backups with volume snapshot locations and volume snapshotters\n// configured and verifies that snapshots are created as appropriate. Verification is done by\n// looking at the backup request's VolumeSnapshots field. This test uses the fakeVolumeSnapshotter\n// struct in place of real volume snapshotters.\nfunc TestBackupWithSnapshots(t *testing.T) {\n\t// TODO: add more verification for skippedPVTracker\n\titemBlockPool := StartItemBlockWorkerPool(t.Context(), 1, logrus.StandardLogger())\n\tdefer itemBlockPool.Stop()\n\ttests := []struct {\n\t\tname string\n\t\treq *Request\n\t\tvsls []*velerov1.VolumeSnapshotLocation\n\t\tapiResources []*test.APIResource\n\t\tsnapshotterGetter volumeSnapshotterGetter\n\t\twant []*volume.Snapshot\n\t}{\n\t\t{\n\t\t\tname: \"persistent volume with no zone annotation creates a snapshot\",\n\t\t\treq: &Request{\n\t\t\t\tBackup: defaultBackup().Result(),\n\t\t\t\tSnapshotLocations: []*velerov1.VolumeSnapshotLocation{\n\t\t\t\t\tnewSnapshotLocation(\"velero\", \"default\", \"default\"),\n\t\t\t\t},\n\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t},\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tsnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t\"default\": new(fakeVolumeSnapshotter).WithVolume(\"pv-1\", \"vol-1\", \"\", \"type-1\", 100, false),\n\t\t\t},\n\t\t\twant: []*volume.Snapshot{\n\t\t\t\t{\n\t\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\t\tBackupName: \"backup-1\",\n\t\t\t\t\t\tLocation: \"default\",\n\t\t\t\t\t\tPersistentVolumeName: \"pv-1\",\n\t\t\t\t\t\tProviderVolumeID: \"vol-1\",\n\t\t\t\t\t\tVolumeType: \"type-1\",\n\t\t\t\t\t\tVolumeIOPS: int64Ptr(100),\n\t\t\t\t\t},\n\t\t\t\t\tStatus: volume.SnapshotStatus{\n\t\t\t\t\t\tPhase: volume.SnapshotPhaseCompleted,\n\t\t\t\t\t\tProviderSnapshotID: \"vol-1-snapshot\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"persistent volume with deprecated zone annotation creates a snapshot\",\n\t\t\treq: &Request{\n\t\t\t\tBackup: defaultBackup().Result(),\n\t\t\t\tSnapshotLocations: []*velerov1.VolumeSnapshotLocation{\n\t\t\t\t\tnewSnapshotLocation(\"velero\", \"default\", \"default\"),\n\t\t\t\t},\n\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t},\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ObjectMeta(builder.WithLabels(\"failure-domain.beta.kubernetes.io/zone\", \"zone-1\")).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tsnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t\"default\": new(fakeVolumeSnapshotter).WithVolume(\"pv-1\", \"vol-1\", \"zone-1\", \"type-1\", 100, false),\n\t\t\t},\n\t\t\twant: []*volume.Snapshot{\n\t\t\t\t{\n\t\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\t\tBackupName: \"backup-1\",\n\t\t\t\t\t\tLocation: \"default\",\n\t\t\t\t\t\tPersistentVolumeName: \"pv-1\",\n\t\t\t\t\t\tProviderVolumeID: \"vol-1\",\n\t\t\t\t\t\tVolumeAZ: \"zone-1\",\n\t\t\t\t\t\tVolumeType: \"type-1\",\n\t\t\t\t\t\tVolumeIOPS: int64Ptr(100),\n\t\t\t\t\t},\n\t\t\t\t\tStatus: volume.SnapshotStatus{\n\t\t\t\t\t\tPhase: volume.SnapshotPhaseCompleted,\n\t\t\t\t\t\tProviderSnapshotID: \"vol-1-snapshot\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"persistent volume with GA zone annotation creates a snapshot\",\n\t\t\treq: &Request{\n\t\t\t\tBackup: defaultBackup().Result(),\n\t\t\t\tSnapshotLocations: []*velerov1.VolumeSnapshotLocation{\n\t\t\t\t\tnewSnapshotLocation(\"velero\", \"default\", \"default\"),\n\t\t\t\t},\n\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t},\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ObjectMeta(builder.WithLabels(\"topology.kubernetes.io/zone\", \"zone-1\")).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tsnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t\"default\": new(fakeVolumeSnapshotter).WithVolume(\"pv-1\", \"vol-1\", \"zone-1\", \"type-1\", 100, false),\n\t\t\t},\n\t\t\twant: []*volume.Snapshot{\n\t\t\t\t{\n\t\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\t\tBackupName: \"backup-1\",\n\t\t\t\t\t\tLocation: \"default\",\n\t\t\t\t\t\tPersistentVolumeName: \"pv-1\",\n\t\t\t\t\t\tProviderVolumeID: \"vol-1\",\n\t\t\t\t\t\tVolumeAZ: \"zone-1\",\n\t\t\t\t\t\tVolumeType: \"type-1\",\n\t\t\t\t\t\tVolumeIOPS: int64Ptr(100),\n\t\t\t\t\t},\n\t\t\t\t\tStatus: volume.SnapshotStatus{\n\t\t\t\t\t\tPhase: volume.SnapshotPhaseCompleted,\n\t\t\t\t\t\tProviderSnapshotID: \"vol-1-snapshot\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"persistent volume with both GA and deprecated zone annotation creates a snapshot and should use the GA\",\n\t\t\treq: &Request{\n\t\t\t\tBackup: defaultBackup().Result(),\n\t\t\t\tSnapshotLocations: []*velerov1.VolumeSnapshotLocation{\n\t\t\t\t\tnewSnapshotLocation(\"velero\", \"default\", \"default\"),\n\t\t\t\t},\n\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t},\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ObjectMeta(builder.WithLabelsMap(map[string]string{\"failure-domain.beta.kubernetes.io/zone\": \"zone-1-deprecated\", \"topology.kubernetes.io/zone\": \"zone-1-ga\"})).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tsnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t\"default\": new(fakeVolumeSnapshotter).WithVolume(\"pv-1\", \"vol-1\", \"zone-1-ga\", \"type-1\", 100, false),\n\t\t\t},\n\t\t\twant: []*volume.Snapshot{\n\t\t\t\t{\n\t\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\t\tBackupName: \"backup-1\",\n\t\t\t\t\t\tLocation: \"default\",\n\t\t\t\t\t\tPersistentVolumeName: \"pv-1\",\n\t\t\t\t\t\tProviderVolumeID: \"vol-1\",\n\t\t\t\t\t\tVolumeAZ: \"zone-1-ga\",\n\t\t\t\t\t\tVolumeType: \"type-1\",\n\t\t\t\t\t\tVolumeIOPS: int64Ptr(100),\n\t\t\t\t\t},\n\t\t\t\t\tStatus: volume.SnapshotStatus{\n\t\t\t\t\t\tPhase: volume.SnapshotPhaseCompleted,\n\t\t\t\t\t\tProviderSnapshotID: \"vol-1-snapshot\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"error returned from CreateSnapshot results in a failed snapshot\",\n\t\t\treq: &Request{\n\t\t\t\tBackup: defaultBackup().Result(),\n\t\t\t\tSnapshotLocations: []*velerov1.VolumeSnapshotLocation{\n\t\t\t\t\tnewSnapshotLocation(\"velero\", \"default\", \"default\"),\n\t\t\t\t},\n\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t},\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tsnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t\"default\": new(fakeVolumeSnapshotter).WithVolume(\"pv-1\", \"vol-1\", \"\", \"type-1\", 100, true),\n\t\t\t},\n\t\t\twant: []*volume.Snapshot{\n\t\t\t\t{\n\t\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\t\tBackupName: \"backup-1\",\n\t\t\t\t\t\tLocation: \"default\",\n\t\t\t\t\t\tPersistentVolumeName: \"pv-1\",\n\t\t\t\t\t\tProviderVolumeID: \"vol-1\",\n\t\t\t\t\t\tVolumeType: \"type-1\",\n\t\t\t\t\t\tVolumeIOPS: int64Ptr(100),\n\t\t\t\t\t},\n\t\t\t\t\tStatus: volume.SnapshotStatus{\n\t\t\t\t\t\tPhase: volume.SnapshotPhaseFailed,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"backup with SnapshotVolumes=false does not create any snapshots\",\n\t\t\treq: &Request{\n\t\t\t\tBackup: defaultBackup().SnapshotVolumes(false).Result(),\n\t\t\t\tSnapshotLocations: []*velerov1.VolumeSnapshotLocation{\n\t\t\t\t\tnewSnapshotLocation(\"velero\", \"default\", \"default\"),\n\t\t\t\t},\n\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t},\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tsnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t\"default\": new(fakeVolumeSnapshotter).WithVolume(\"pv-1\", \"vol-1\", \"\", \"type-1\", 100, false),\n\t\t\t},\n\t\t\twant: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"backup with no volume snapshot locations does not create any snapshots\",\n\t\t\treq: &Request{\n\t\t\t\tBackup: defaultBackup().Result(),\n\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t},\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tsnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t\"default\": new(fakeVolumeSnapshotter).WithVolume(\"pv-1\", \"vol-1\", \"\", \"type-1\", 100, false),\n\t\t\t},\n\t\t\twant: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"backup with no volume snapshotters does not create any snapshots\",\n\t\t\treq: &Request{\n\t\t\t\tBackup: defaultBackup().Result(),\n\t\t\t\tSnapshotLocations: []*velerov1.VolumeSnapshotLocation{\n\t\t\t\t\tnewSnapshotLocation(\"velero\", \"default\", \"default\"),\n\t\t\t\t},\n\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t},\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tsnapshotterGetter: map[string]vsv1.VolumeSnapshotter{},\n\t\t\twant: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"unsupported persistent volume type does not create any snapshots\",\n\t\t\treq: &Request{\n\t\t\t\tBackup: defaultBackup().Result(),\n\t\t\t\tSnapshotLocations: []*velerov1.VolumeSnapshotLocation{\n\t\t\t\t\tnewSnapshotLocation(\"velero\", \"default\", \"default\"),\n\t\t\t\t},\n\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t},\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tsnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t\"default\": new(fakeVolumeSnapshotter),\n\t\t\t},\n\t\t\twant: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"when there are multiple volumes, snapshot locations, and snapshotters, volumes are matched to the right snapshotters\",\n\t\t\treq: &Request{\n\t\t\t\tBackup: defaultBackup().Result(),\n\t\t\t\tSnapshotLocations: []*velerov1.VolumeSnapshotLocation{\n\t\t\t\t\tnewSnapshotLocation(\"velero\", \"default\", \"default\"),\n\t\t\t\t\tnewSnapshotLocation(\"velero\", \"another\", \"another\"),\n\t\t\t\t},\n\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t},\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tsnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t\"default\": new(fakeVolumeSnapshotter).WithVolume(\"pv-1\", \"vol-1\", \"\", \"type-1\", 100, false),\n\t\t\t\t\"another\": new(fakeVolumeSnapshotter).WithVolume(\"pv-2\", \"vol-2\", \"\", \"type-2\", 100, false),\n\t\t\t},\n\t\t\twant: []*volume.Snapshot{\n\t\t\t\t{\n\t\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\t\tBackupName: \"backup-1\",\n\t\t\t\t\t\tLocation: \"default\",\n\t\t\t\t\t\tPersistentVolumeName: \"pv-1\",\n\t\t\t\t\t\tProviderVolumeID: \"vol-1\",\n\t\t\t\t\t\tVolumeType: \"type-1\",\n\t\t\t\t\t\tVolumeIOPS: int64Ptr(100),\n\t\t\t\t\t},\n\t\t\t\t\tStatus: volume.SnapshotStatus{\n\t\t\t\t\t\tPhase: volume.SnapshotPhaseCompleted,\n\t\t\t\t\t\tProviderSnapshotID: \"vol-1-snapshot\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\t\tBackupName: \"backup-1\",\n\t\t\t\t\t\tLocation: \"another\",\n\t\t\t\t\t\tPersistentVolumeName: \"pv-2\",\n\t\t\t\t\t\tProviderVolumeID: \"vol-2\",\n\t\t\t\t\t\tVolumeType: \"type-2\",\n\t\t\t\t\t\tVolumeIOPS: int64Ptr(100),\n\t\t\t\t\t},\n\t\t\t\t\tStatus: volume.SnapshotStatus{\n\t\t\t\t\t\tPhase: volume.SnapshotPhaseCompleted,\n\t\t\t\t\t\tProviderSnapshotID: \"vol-2-snapshot\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\th = newHarness(t, itemBlockPool)\n\t\t\t\tbackupFile = bytes.NewBuffer([]byte{})\n\t\t\t)\n\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.addItems(t, resource)\n\t\t\t}\n\n\t\t\terr := h.backupper.Backup(h.log, tc.req, backupFile, nil, nil, tc.snapshotterGetter)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.Equal(t, tc.want, tc.req.VolumeSnapshots.Get())\n\t\t})\n\t}\n}\n\n// TestBackupWithAsyncOperations runs backups which return operationIDs and\n// verifies that the itemoperations are tracked as appropriate. Verification is done by\n// looking at the backup request's itemOperationsList field.\nfunc TestBackupWithAsyncOperations(t *testing.T) {\n\t// completedOperationAction is a *pluggableAction, whose Execute(...)\n\t// method returns an operationID which will always be done when calling Progress.\n\tcompletedOperationAction := &pluggableAction{\n\t\texecuteFunc: func(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\t\t\tobj, ok := item.(*unstructured.Unstructured)\n\t\t\tif !ok {\n\t\t\t\treturn nil, nil, \"\", nil, errors.Errorf(\"unexpected type %T\", item)\n\t\t\t}\n\n\t\t\treturn obj, nil, obj.GetName() + \"-1\", nil, nil\n\t\t},\n\t\tprogressFunc: func(operationID string, backup *velerov1.Backup) (velero.OperationProgress, error) {\n\t\t\treturn velero.OperationProgress{\n\t\t\t\tCompleted: true,\n\t\t\t\tDescription: \"Done!\",\n\t\t\t}, nil\n\t\t},\n\t}\n\n\t// incompleteOperationAction is a *pluggableAction, whose Execute(...)\n\t// method returns an operationID which will never be done when calling Progress.\n\tincompleteOperationAction := &pluggableAction{\n\t\texecuteFunc: func(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\t\t\tobj, ok := item.(*unstructured.Unstructured)\n\t\t\tif !ok {\n\t\t\t\treturn nil, nil, \"\", nil, errors.Errorf(\"unexpected type %T\", item)\n\t\t\t}\n\n\t\t\treturn obj, nil, obj.GetName() + \"-1\", nil, nil\n\t\t},\n\t\tprogressFunc: func(operationID string, backup *velerov1.Backup) (velero.OperationProgress, error) {\n\t\t\treturn velero.OperationProgress{\n\t\t\t\tCompleted: false,\n\t\t\t\tDescription: \"Working...\",\n\t\t\t}, nil\n\t\t},\n\t}\n\n\t// noOperationAction is a *pluggableAction, whose Execute(...)\n\t// method does not return an operationID.\n\tnoOperationAction := &pluggableAction{\n\t\texecuteFunc: func(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\t\t\tobj, ok := item.(*unstructured.Unstructured)\n\t\t\tif !ok {\n\t\t\t\treturn nil, nil, \"\", nil, errors.Errorf(\"unexpected type %T\", item)\n\t\t\t}\n\n\t\t\treturn obj, nil, \"\", nil, nil\n\t\t},\n\t}\n\n\titemBlockPool := StartItemBlockWorkerPool(t.Context(), 1, logrus.StandardLogger())\n\tdefer itemBlockPool.Stop()\n\ttests := []struct {\n\t\tname string\n\t\treq *Request\n\t\tapiResources []*test.APIResource\n\t\tactions []biav2.BackupItemAction\n\t\twant []*itemoperation.BackupOperation\n\t}{\n\t\t{\n\t\t\tname: \"action that starts a short-running process records operation\",\n\t\t\treq: &Request{\n\t\t\t\tBackup: defaultBackup().Result(),\n\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t},\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\tcompletedOperationAction,\n\t\t\t},\n\t\t\twant: []*itemoperation.BackupOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\t\t\tBackupName: \"backup-1\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pod-1\"},\n\t\t\t\t\t\tOperationID: \"pod-1-1\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: \"New\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"action that starts a long-running process records operation\",\n\t\t\treq: &Request{\n\t\t\t\tBackup: defaultBackup().Result(),\n\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t},\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\tincompleteOperationAction,\n\t\t\t},\n\t\t\twant: []*itemoperation.BackupOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\t\t\tBackupName: \"backup-1\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pod-2\"},\n\t\t\t\t\t\tOperationID: \"pod-2-1\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: \"New\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"action that has no operation doesn't record one\",\n\t\t\treq: &Request{\n\t\t\t\tBackup: defaultBackup().Result(),\n\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t},\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-3\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\tnoOperationAction,\n\t\t\t},\n\t\t\twant: []*itemoperation.BackupOperation{},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\th = newHarness(t, itemBlockPool)\n\t\t\t\tbackupFile = bytes.NewBuffer([]byte{})\n\t\t\t)\n\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.addItems(t, resource)\n\t\t\t}\n\n\t\t\terr := h.backupper.Backup(h.log, tc.req, backupFile, tc.actions, nil, nil)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tresultOper := *tc.req.GetItemOperationsList()\n\t\t\t// set want Created times so it won't fail the assert.Equal test\n\t\t\tfor i, wantOper := range tc.want {\n\t\t\t\twantOper.Status.Created = resultOper[i].Status.Created\n\t\t\t}\n\t\t\tassert.Equal(t, tc.want, *tc.req.GetItemOperationsList())\n\t\t})\n\t}\n}\n\n// TestBackupWithInvalidHooks runs backups with invalid hook specifications and verifies\n// that an error is returned.\nfunc TestBackupWithInvalidHooks(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1.Backup\n\t\tapiResources []*test.APIResource\n\t\twant error\n\t}{\n\t\t{\n\t\t\tname: \"hook with invalid label selector causes backup to fail\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tHooks(velerov1.BackupHooks{\n\t\t\t\t\tResources: []velerov1.BackupResourceHookSpec{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"hook-with-invalid-label-selector\",\n\t\t\t\t\t\t\tLabelSelector: &metav1.LabelSelector{\n\t\t\t\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"foo\",\n\t\t\t\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOperator(\"nonexistent-operator\"),\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"bar\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: errors.New(\"\\\"nonexistent-operator\\\" is not a valid label selector operator\"),\n\t\t},\n\t}\n\n\titemBlockPool := StartItemBlockWorkerPool(t.Context(), 1, logrus.StandardLogger())\n\tdefer itemBlockPool.Stop()\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\th = newHarness(t, itemBlockPool)\n\t\t\t\treq = &Request{\n\t\t\t\t\tBackup: tc.backup,\n\t\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t\t}\n\t\t\t\tbackupFile = bytes.NewBuffer([]byte{})\n\t\t\t)\n\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.addItems(t, resource)\n\t\t\t}\n\n\t\t\tassert.EqualError(t, h.backupper.Backup(h.log, req, backupFile, nil, nil, nil), tc.want.Error())\n\t\t})\n\t}\n}\n\n// TestBackupWithHooks runs backups with valid hook specifications and verifies that the\n// hooks are run. It uses a MockPodCommandExecutor since hooks can't actually be executed\n// in running pods during the unit test. Verification is done by asserting expected method\n// calls on the mock object.\nfunc TestBackupWithHooks(t *testing.T) {\n\ttype expectedCall struct {\n\t\tpodNamespace string\n\t\tpodName string\n\t\thookName string\n\t\thook *velerov1.ExecHook\n\t\terr error\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1.Backup\n\t\tapiResources []*test.APIResource\n\t\tactions []ibav1.ItemBlockAction\n\t\twantExecutePodCommandCalls []*expectedCall\n\t\twantBackedUp []string\n\t\twantHookExecutionLog []test.HookExecutionEntry\n\t}{\n\t\t{\n\t\t\tname: \"pre hook with no resource filters runs for all pods\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tHooks(velerov1.BackupHooks{\n\t\t\t\t\tResources: []velerov1.BackupResourceHookSpec{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"hook-1\",\n\t\t\t\t\t\t\tPreHooks: []velerov1.BackupResourceHook{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1.ExecHook{\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"ls\", \"/tmp\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twantExecutePodCommandCalls: []*expectedCall{\n\t\t\t\t{\n\t\t\t\t\tpodNamespace: \"ns-1\",\n\t\t\t\t\tpodName: \"pod-1\",\n\t\t\t\t\thookName: \"hook-1\",\n\t\t\t\t\thook: &velerov1.ExecHook{\n\t\t\t\t\t\tCommand: []string{\"ls\", \"/tmp\"},\n\t\t\t\t\t},\n\t\t\t\t\terr: nil,\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tpodNamespace: \"ns-2\",\n\t\t\t\t\tpodName: \"pod-2\",\n\t\t\t\t\thookName: \"hook-1\",\n\t\t\t\t\thook: &velerov1.ExecHook{\n\t\t\t\t\t\tCommand: []string{\"ls\", \"/tmp\"},\n\t\t\t\t\t},\n\t\t\t\t\terr: nil,\n\t\t\t\t},\n\t\t\t},\n\t\t\twantBackedUp: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-2/pod-2.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-2/pod-2.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"post hook with no resource filters runs for all pods\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tHooks(velerov1.BackupHooks{\n\t\t\t\t\tResources: []velerov1.BackupResourceHookSpec{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"hook-1\",\n\t\t\t\t\t\t\tPostHooks: []velerov1.BackupResourceHook{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1.ExecHook{\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"ls\", \"/tmp\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twantExecutePodCommandCalls: []*expectedCall{\n\t\t\t\t{\n\t\t\t\t\tpodNamespace: \"ns-1\",\n\t\t\t\t\tpodName: \"pod-1\",\n\t\t\t\t\thookName: \"hook-1\",\n\t\t\t\t\thook: &velerov1.ExecHook{\n\t\t\t\t\t\tCommand: []string{\"ls\", \"/tmp\"},\n\t\t\t\t\t},\n\t\t\t\t\terr: nil,\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tpodNamespace: \"ns-2\",\n\t\t\t\t\tpodName: \"pod-2\",\n\t\t\t\t\thookName: \"hook-1\",\n\t\t\t\t\thook: &velerov1.ExecHook{\n\t\t\t\t\t\tCommand: []string{\"ls\", \"/tmp\"},\n\t\t\t\t\t},\n\t\t\t\t\terr: nil,\n\t\t\t\t},\n\t\t\t},\n\t\t\twantBackedUp: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-2/pod-2.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-2/pod-2.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"pre and post hooks run for a pod\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tHooks(velerov1.BackupHooks{\n\t\t\t\t\tResources: []velerov1.BackupResourceHookSpec{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"hook-1\",\n\t\t\t\t\t\t\tPreHooks: []velerov1.BackupResourceHook{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1.ExecHook{\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"pre\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tPostHooks: []velerov1.BackupResourceHook{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1.ExecHook{\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"post\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twantExecutePodCommandCalls: []*expectedCall{\n\t\t\t\t{\n\t\t\t\t\tpodNamespace: \"ns-1\",\n\t\t\t\t\tpodName: \"pod-1\",\n\t\t\t\t\thookName: \"hook-1\",\n\t\t\t\t\thook: &velerov1.ExecHook{\n\t\t\t\t\t\tCommand: []string{\"pre\"},\n\t\t\t\t\t},\n\t\t\t\t\terr: nil,\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tpodNamespace: \"ns-1\",\n\t\t\t\t\tpodName: \"pod-1\",\n\t\t\t\t\thookName: \"hook-1\",\n\t\t\t\t\thook: &velerov1.ExecHook{\n\t\t\t\t\t\tCommand: []string{\"post\"},\n\t\t\t\t\t},\n\t\t\t\t\terr: nil,\n\t\t\t\t},\n\t\t\t},\n\t\t\twantBackedUp: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"pre and post hooks run for two pods sequentially by pods in different ItemBlocks\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tHooks(velerov1.BackupHooks{\n\t\t\t\t\tResources: []velerov1.BackupResourceHookSpec{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"hook-1\",\n\t\t\t\t\t\t\tPreHooks: []velerov1.BackupResourceHook{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1.ExecHook{\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"pre\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tPostHooks: []velerov1.BackupResourceHook{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1.ExecHook{\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"post\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twantExecutePodCommandCalls: []*expectedCall{\n\t\t\t\t{\n\t\t\t\t\tpodNamespace: \"ns-1\",\n\t\t\t\t\tpodName: \"pod-1\",\n\t\t\t\t\thookName: \"hook-1\",\n\t\t\t\t\thook: &velerov1.ExecHook{\n\t\t\t\t\t\tCommand: []string{\"pre\"},\n\t\t\t\t\t},\n\t\t\t\t\terr: nil,\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tpodNamespace: \"ns-1\",\n\t\t\t\t\tpodName: \"pod-1\",\n\t\t\t\t\thookName: \"hook-1\",\n\t\t\t\t\thook: &velerov1.ExecHook{\n\t\t\t\t\t\tCommand: []string{\"post\"},\n\t\t\t\t\t},\n\t\t\t\t\terr: nil,\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tpodNamespace: \"ns-1\",\n\t\t\t\t\tpodName: \"pod-2\",\n\t\t\t\t\thookName: \"hook-1\",\n\t\t\t\t\thook: &velerov1.ExecHook{\n\t\t\t\t\t\tCommand: []string{\"pre\"},\n\t\t\t\t\t},\n\t\t\t\t\terr: nil,\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tpodNamespace: \"ns-1\",\n\t\t\t\t\tpodName: \"pod-2\",\n\t\t\t\t\thookName: \"hook-1\",\n\t\t\t\t\thook: &velerov1.ExecHook{\n\t\t\t\t\t\tCommand: []string{\"post\"},\n\t\t\t\t\t},\n\t\t\t\t\terr: nil,\n\t\t\t\t},\n\t\t\t},\n\t\t\twantHookExecutionLog: []test.HookExecutionEntry{\n\t\t\t\t{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\tHookName: \"hook-1\",\n\t\t\t\t\tHookCommand: []string{\"pre\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\tHookName: \"hook-1\",\n\t\t\t\t\tHookCommand: []string{\"post\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-2\",\n\t\t\t\t\tHookName: \"hook-1\",\n\t\t\t\t\tHookCommand: []string{\"pre\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-2\",\n\t\t\t\t\tHookName: \"hook-1\",\n\t\t\t\t\tHookCommand: []string{\"post\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantBackedUp: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-2.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-2.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"both pre hooks run before both post hooks for pods in the same ItemBlock\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tHooks(velerov1.BackupHooks{\n\t\t\t\t\tResources: []velerov1.BackupResourceHookSpec{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"hook-1\",\n\t\t\t\t\t\t\tPreHooks: []velerov1.BackupResourceHook{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1.ExecHook{\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"pre\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tPostHooks: []velerov1.BackupResourceHook{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1.ExecHook{\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"post\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tactions: []ibav1.ItemBlockAction{\n\t\t\t\t&pluggableIBA{\n\t\t\t\t\tselector: velero.ResourceSelector{IncludedNamespaces: []string{\"ns-1\"}},\n\t\t\t\t\tgetRelatedItemsFunc: func(item runtime.Unstructured, backup *velerov1.Backup) ([]velero.ResourceIdentifier, error) {\n\t\t\t\t\t\trelatedItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"ns-1\", Name: \"pod-1\"},\n\t\t\t\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"ns-1\", Name: \"pod-2\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn relatedItems, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantExecutePodCommandCalls: []*expectedCall{\n\t\t\t\t{\n\t\t\t\t\tpodNamespace: \"ns-1\",\n\t\t\t\t\tpodName: \"pod-1\",\n\t\t\t\t\thookName: \"hook-1\",\n\t\t\t\t\thook: &velerov1.ExecHook{\n\t\t\t\t\t\tCommand: []string{\"pre\"},\n\t\t\t\t\t},\n\t\t\t\t\terr: nil,\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tpodNamespace: \"ns-1\",\n\t\t\t\t\tpodName: \"pod-1\",\n\t\t\t\t\thookName: \"hook-1\",\n\t\t\t\t\thook: &velerov1.ExecHook{\n\t\t\t\t\t\tCommand: []string{\"post\"},\n\t\t\t\t\t},\n\t\t\t\t\terr: nil,\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tpodNamespace: \"ns-1\",\n\t\t\t\t\tpodName: \"pod-2\",\n\t\t\t\t\thookName: \"hook-1\",\n\t\t\t\t\thook: &velerov1.ExecHook{\n\t\t\t\t\t\tCommand: []string{\"pre\"},\n\t\t\t\t\t},\n\t\t\t\t\terr: nil,\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tpodNamespace: \"ns-1\",\n\t\t\t\t\tpodName: \"pod-2\",\n\t\t\t\t\thookName: \"hook-1\",\n\t\t\t\t\thook: &velerov1.ExecHook{\n\t\t\t\t\t\tCommand: []string{\"post\"},\n\t\t\t\t\t},\n\t\t\t\t\terr: nil,\n\t\t\t\t},\n\t\t\t},\n\t\t\twantHookExecutionLog: []test.HookExecutionEntry{\n\t\t\t\t{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\tHookName: \"hook-1\",\n\t\t\t\t\tHookCommand: []string{\"pre\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-2\",\n\t\t\t\t\tHookName: \"hook-1\",\n\t\t\t\t\tHookCommand: []string{\"pre\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\tHookName: \"hook-1\",\n\t\t\t\t\tHookCommand: []string{\"post\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-2\",\n\t\t\t\t\tHookName: \"hook-1\",\n\t\t\t\t\tHookCommand: []string{\"post\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantBackedUp: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/namespaces/ns-1/pod-2.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-1.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-1/pod-2.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"item is not backed up if hook returns an error when OnError=Fail\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tHooks(velerov1.BackupHooks{\n\t\t\t\t\tResources: []velerov1.BackupResourceHookSpec{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"hook-1\",\n\t\t\t\t\t\t\tPreHooks: []velerov1.BackupResourceHook{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tExec: &velerov1.ExecHook{\n\t\t\t\t\t\t\t\t\t\tCommand: []string{\"ls\", \"/tmp\"},\n\t\t\t\t\t\t\t\t\t\tOnError: velerov1.HookErrorModeFail,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twantExecutePodCommandCalls: []*expectedCall{\n\t\t\t\t{\n\t\t\t\t\tpodNamespace: \"ns-1\",\n\t\t\t\t\tpodName: \"pod-1\",\n\t\t\t\t\thookName: \"hook-1\",\n\t\t\t\t\thook: &velerov1.ExecHook{\n\t\t\t\t\t\tCommand: []string{\"ls\", \"/tmp\"},\n\t\t\t\t\t\tOnError: velerov1.HookErrorModeFail,\n\t\t\t\t\t},\n\t\t\t\t\terr: errors.New(\"exec hook error\"),\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tpodNamespace: \"ns-2\",\n\t\t\t\t\tpodName: \"pod-2\",\n\t\t\t\t\thookName: \"hook-1\",\n\t\t\t\t\thook: &velerov1.ExecHook{\n\t\t\t\t\t\tCommand: []string{\"ls\", \"/tmp\"},\n\t\t\t\t\t\tOnError: velerov1.HookErrorModeFail,\n\t\t\t\t\t},\n\t\t\t\t\terr: nil,\n\t\t\t\t},\n\t\t\t},\n\t\t\twantBackedUp: []string{\n\t\t\t\t\"resources/pods/namespaces/ns-2/pod-2.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/ns-2/pod-2.json\",\n\t\t\t},\n\t\t},\n\t}\n\n\titemBlockPool := StartItemBlockWorkerPool(t.Context(), 1, logrus.StandardLogger())\n\tdefer itemBlockPool.Stop()\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\th = newHarness(t, itemBlockPool)\n\t\t\t\treq = &Request{\n\t\t\t\t\tBackup: tc.backup,\n\t\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t\t}\n\t\t\t\tbackupFile = bytes.NewBuffer([]byte{})\n\t\t\t\tpodCommandExecutor = new(test.MockPodCommandExecutor)\n\t\t\t)\n\n\t\t\th.backupper.podCommandExecutor = podCommandExecutor\n\t\t\tdefer podCommandExecutor.AssertExpectations(t)\n\n\t\t\tfor _, expect := range tc.wantExecutePodCommandCalls {\n\t\t\t\tpodCommandExecutor.On(\"ExecutePodCommand\",\n\t\t\t\t\tmock.Anything,\n\t\t\t\t\tmock.Anything,\n\t\t\t\t\texpect.podNamespace,\n\t\t\t\t\texpect.podName,\n\t\t\t\t\texpect.hookName,\n\t\t\t\t\texpect.hook,\n\t\t\t\t).Return(expect.err)\n\t\t\t}\n\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.addItems(t, resource)\n\t\t\t}\n\n\t\t\trequire.NoError(t, h.backupper.Backup(h.log, req, backupFile, nil, tc.actions, nil))\n\n\t\t\tif tc.wantHookExecutionLog != nil {\n\t\t\t\tassert.Equal(t, tc.wantHookExecutionLog, podCommandExecutor.HookExecutionLog)\n\t\t\t}\n\t\t\tassertTarballContents(t, backupFile, append(tc.wantBackedUp, \"metadata/version\")...)\n\t\t})\n\t}\n}\n\ntype fakePodVolumeBackupperFactory struct{}\n\nfunc (f *fakePodVolumeBackupperFactory) NewBackupper(context.Context, logrus.FieldLogger, *velerov1.Backup, string) (podvolume.Backupper, error) {\n\treturn &fakePodVolumeBackupper{}, nil\n}\n\ntype fakePodVolumeBackupper struct {\n\tpvbs []*velerov1.PodVolumeBackup\n}\n\n// BackupPodVolumes returns one pod volume backup per entry in volumes, with namespace \"velero\"\n// and name \"pvb---\".\nfunc (b *fakePodVolumeBackupper) BackupPodVolumes(backup *velerov1.Backup, pod *corev1api.Pod, volumes []string, _ *resourcepolicies.Policies, _ logrus.FieldLogger) ([]*velerov1.PodVolumeBackup, *podvolume.PVCBackupSummary, []error) {\n\tvar res []*velerov1.PodVolumeBackup\n\tpvcSummary := podvolume.NewPVCBackupSummary()\n\n\tanno := pod.GetAnnotations()\n\tif anno != nil && anno[\"backup.velero.io/bakupper-skip\"] != \"\" {\n\t\treturn res, pvcSummary, nil\n\t}\n\n\tfor _, vol := range volumes {\n\t\tpvb := builder.ForPodVolumeBackup(\"velero\", fmt.Sprintf(\"pvb-%s-%s-%s\", pod.Namespace, pod.Name, vol)).Volume(vol).Result()\n\t\tres = append(res, pvb)\n\t}\n\n\tb.pvbs = res\n\n\treturn res, pvcSummary, nil\n}\n\nfunc (b *fakePodVolumeBackupper) WaitAllPodVolumesProcessed(log logrus.FieldLogger) []*velerov1.PodVolumeBackup {\n\treturn b.pvbs\n}\n\nfunc (b *fakePodVolumeBackupper) GetPodVolumeBackupByPodAndVolume(podNamespace, podName, volume string) (*velerov1.PodVolumeBackup, error) {\n\tfor _, pvb := range b.pvbs {\n\t\tif pvb.Spec.Pod.Namespace == podNamespace && pvb.Spec.Pod.Name == podName && pvb.Spec.Volume == volume {\n\t\t\treturn pvb, nil\n\t\t}\n\t}\n\treturn nil, nil\n}\nfunc (b *fakePodVolumeBackupper) ListPodVolumeBackupsByPod(podNamespace, podName string) ([]*velerov1.PodVolumeBackup, error) {\n\tvar pvbs []*velerov1.PodVolumeBackup\n\tfor _, pvb := range b.pvbs {\n\t\tif pvb.Spec.Pod.Namespace == podNamespace && pvb.Spec.Pod.Name == podName {\n\t\t\tpvbs = append(pvbs, pvb)\n\t\t}\n\t}\n\treturn pvbs, nil\n}\n\n// TestBackupWithPodVolume runs backups of pods that are annotated for PodVolume backup,\n// and ensures that the pod volume backupper is called, that the returned PodVolumeBackups\n// are added to the Request object, and that when PVCs are backed up with PodVolume, the\n// claimed PVs are not also snapshotted using a VolumeSnapshotter.\nfunc TestBackupWithPodVolume(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1.Backup\n\t\tapiResources []*test.APIResource\n\t\tpod *corev1api.Pod\n\t\tvsl *velerov1.VolumeSnapshotLocation\n\t\tsnapshotterGetter volumeSnapshotterGetter\n\t\twant []*velerov1.PodVolumeBackup\n\t}{\n\t\t{\n\t\t\tname: \"a pod annotated for pod volume backup should result in pod volume backups being returned\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").\n\t\t\t\t\t\tObjectMeta(builder.WithAnnotations(\"backup.velero.io/backup-volumes\", \"foo\")).\n\t\t\t\t\t\tVolumes(&corev1api.Volume{\n\t\t\t\t\t\t\tName: \"foo\",\n\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\t\tClaimName: \"foo\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}).\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(),\n\t\t\t},\n\t\t\twant: []*velerov1.PodVolumeBackup{\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-ns-1-pod-1-foo\").Volume(\"foo\").Result(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when a PVC is used by two pods and annotated for pod volume backup on both, only one should be backed up\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").\n\t\t\t\t\t\tObjectMeta(builder.WithAnnotations(\"backup.velero.io/backup-volumes\", \"foo\")).\n\t\t\t\t\t\tVolumes(builder.ForVolume(\"foo\").PersistentVolumeClaimSource(\"pvc-1\").Result()).\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-2\").\n\t\t\t\t\t\tObjectMeta(builder.WithAnnotations(\"backup.velero.io/backup-volumes\", \"bar\")).\n\t\t\t\t\t\tVolumes(builder.ForVolume(\"bar\").PersistentVolumeClaimSource(\"pvc-1\").Result()).\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(),\n\t\t\t},\n\t\t\twant: []*velerov1.PodVolumeBackup{\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-ns-1-pod-1-foo\").Volume(\"foo\").Result(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when a PVC is used by two pods and annotated for pod volume backup on both, the backup for pod1 gives up the PVC, the backup for pod2 should include it\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").\n\t\t\t\t\t\tObjectMeta(builder.WithAnnotations(\"backup.velero.io/backup-volumes\", \"foo\", \"backup.velero.io/bakupper-skip\", \"foo\")).\n\t\t\t\t\t\tVolumes(builder.ForVolume(\"foo\").PersistentVolumeClaimSource(\"pvc-1\").Result()).\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-2\").\n\t\t\t\t\t\tObjectMeta(builder.WithAnnotations(\"backup.velero.io/backup-volumes\", \"bar\")).\n\t\t\t\t\t\tVolumes(builder.ForVolume(\"bar\").PersistentVolumeClaimSource(\"pvc-1\").Result()).\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(),\n\t\t\t},\n\t\t\twant: []*velerov1.PodVolumeBackup{\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-ns-1-pod-2-bar\").Volume(\"bar\").Result(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when PVC pod volumes are backed up using pod volume backup, their claimed PVs are not also snapshotted\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").\n\t\t\t\t\t\tVolumes(\n\t\t\t\t\t\t\tbuilder.ForVolume(\"vol-1\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t\t\t\tbuilder.ForVolume(\"vol-2\").PersistentVolumeClaimSource(\"pvc-2\").Result(),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithAnnotations(\"backup.velero.io/backup-volumes\", \"vol-1,vol-2\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").VolumeName(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-2\").VolumeName(\"pv-2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ClaimRef(\"ns-1\", \"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").ClaimRef(\"ns-1\", \"pvc-2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\tpod: builder.ForPod(\"ns-1\", \"pod-1\").\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"vol-1\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForVolume(\"vol-2\").PersistentVolumeClaimSource(\"pvc-2\").Result(),\n\t\t\t\t).\n\t\t\t\tObjectMeta(\n\t\t\t\t\tbuilder.WithAnnotations(\"backup.velero.io/backup-volumes\", \"vol-1,vol-2\"),\n\t\t\t\t).\n\t\t\t\tResult(),\n\t\t\tvsl: newSnapshotLocation(\"velero\", \"default\", \"default\"),\n\t\t\tsnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t\"default\": new(fakeVolumeSnapshotter).\n\t\t\t\t\tWithVolume(\"pv-1\", \"vol-1\", \"\", \"type-1\", 100, false).\n\t\t\t\t\tWithVolume(\"pv-2\", \"vol-2\", \"\", \"type-1\", 100, false),\n\t\t\t},\n\t\t\twant: []*velerov1.PodVolumeBackup{\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-ns-1-pod-1-vol-1\").Volume(\"vol-1\").Result(),\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-ns-1-pod-1-vol-2\").Volume(\"vol-2\").Result(),\n\t\t\t},\n\t\t},\n\t}\n\n\titemBlockPool := StartItemBlockWorkerPool(t.Context(), 1, logrus.StandardLogger())\n\tdefer itemBlockPool.Stop()\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\th = newHarness(t, itemBlockPool)\n\t\t\t\treq = &Request{\n\t\t\t\t\tBackup: tc.backup,\n\t\t\t\t\tSnapshotLocations: []*velerov1.VolumeSnapshotLocation{tc.vsl},\n\t\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t\t}\n\t\t\t\tbackupFile = bytes.NewBuffer([]byte{})\n\t\t\t)\n\n\t\t\tif tc.pod != nil {\n\t\t\t\trequire.NoError(t, h.backupper.kbClient.Create(t.Context(), tc.pod))\n\t\t\t}\n\n\t\t\th.backupper.podVolumeBackupperFactory = new(fakePodVolumeBackupperFactory)\n\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.addItems(t, resource)\n\t\t\t}\n\n\t\t\trequire.NoError(t, h.backupper.Backup(h.log, req, backupFile, nil, nil, tc.snapshotterGetter))\n\n\t\t\tassert.Equal(t, tc.want, req.PodVolumeBackups)\n\n\t\t\t// this assumes that we don't have any test cases where some PVs should be snapshotted using a VolumeSnapshotter\n\t\t\tassert.Nil(t, req.VolumeSnapshots.Get())\n\t\t})\n\t}\n}\n\n// pluggableAction is a backup item action that can be plugged with Execute\n// and Progress function bodies at runtime.\ntype pluggableAction struct {\n\tselector velero.ResourceSelector\n\texecuteFunc func(runtime.Unstructured, *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error)\n\tprogressFunc func(string, *velerov1.Backup) (velero.OperationProgress, error)\n}\n\nfunc (a *pluggableAction) Execute(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\tif a.executeFunc == nil {\n\t\treturn item, nil, \"\", nil, nil\n\t}\n\n\treturn a.executeFunc(item, backup)\n}\n\nfunc (a *pluggableAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn a.selector, nil\n}\n\nfunc (a *pluggableAction) Progress(operationID string, backup *velerov1.Backup) (velero.OperationProgress, error) {\n\tif a.progressFunc == nil {\n\t\treturn velero.OperationProgress{}, nil\n\t}\n\n\treturn a.progressFunc(operationID, backup)\n}\n\nfunc (a *pluggableAction) Cancel(operationID string, backup *velerov1.Backup) error {\n\treturn nil\n}\n\nfunc (a *pluggableAction) Name() string {\n\treturn \"\"\n}\n\n// pluggableIBA is an ItemBlock action that can be plugged with GetRelatedItems function bodies at runtime.\ntype pluggableIBA struct {\n\tselector velero.ResourceSelector\n\tgetRelatedItemsFunc func(runtime.Unstructured, *velerov1.Backup) ([]velero.ResourceIdentifier, error)\n}\n\nfunc (a *pluggableIBA) GetRelatedItems(item runtime.Unstructured, backup *velerov1.Backup) ([]velero.ResourceIdentifier, error) {\n\tif a.getRelatedItemsFunc == nil {\n\t\treturn nil, nil\n\t}\n\n\treturn a.getRelatedItemsFunc(item, backup)\n}\n\nfunc (a *pluggableIBA) AppliesTo() (velero.ResourceSelector, error) {\n\treturn a.selector, nil\n}\n\nfunc (a *pluggableIBA) Name() string {\n\treturn \"\"\n}\n\ntype harness struct {\n\t*test.APIServer\n\tbackupper *kubernetesBackupper\n\tlog logrus.FieldLogger\n\titemBlockPool ItemBlockWorkerPool\n}\n\nfunc (h *harness) addItems(t *testing.T, resource *test.APIResource) {\n\tt.Helper()\n\n\th.DiscoveryClient.WithAPIResource(resource)\n\trequire.NoError(t, h.backupper.discoveryHelper.Refresh())\n\n\tfor _, item := range resource.Items {\n\t\tobj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(item)\n\t\trequire.NoError(t, err)\n\n\t\tunstructuredObj := &unstructured.Unstructured{Object: obj}\n\n\t\tif resource.Namespaced {\n\t\t\t_, err = h.DynamicClient.Resource(resource.GVR()).Namespace(item.GetNamespace()).Create(t.Context(), unstructuredObj, metav1.CreateOptions{})\n\t\t} else {\n\t\t\t_, err = h.DynamicClient.Resource(resource.GVR()).Create(t.Context(), unstructuredObj, metav1.CreateOptions{})\n\t\t}\n\t\trequire.NoError(t, err)\n\t}\n}\n\nfunc newHarness(t *testing.T, itemBlockPool *ItemBlockWorkerPool) *harness {\n\tt.Helper()\n\n\tapiServer := test.NewAPIServer(t)\n\tlog := logrus.StandardLogger()\n\n\tdiscoveryHelper, err := discovery.NewHelper(apiServer.DiscoveryClient, log)\n\trequire.NoError(t, err)\n\n\tif itemBlockPool == nil {\n\t\titemBlockPool = StartItemBlockWorkerPool(t.Context(), 1, log)\n\t}\n\treturn &harness{\n\t\tAPIServer: apiServer,\n\t\tbackupper: &kubernetesBackupper{\n\t\t\tkbClient: test.NewFakeControllerRuntimeClient(t),\n\t\t\tdynamicFactory: client.NewDynamicFactory(apiServer.DynamicClient),\n\t\t\tdiscoveryHelper: discoveryHelper,\n\n\t\t\t// unsupported\n\t\t\tpodCommandExecutor: nil,\n\t\t\tpodVolumeBackupperFactory: new(fakePodVolumeBackupperFactory),\n\t\t\tpodVolumeTimeout: 60 * time.Second,\n\t\t},\n\t\tlog: log,\n\t\titemBlockPool: *itemBlockPool,\n\t}\n}\n\nfunc newSnapshotLocation(ns, name, provider string) *velerov1.VolumeSnapshotLocation {\n\treturn &velerov1.VolumeSnapshotLocation{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: ns,\n\t\t\tName: name,\n\t\t},\n\t\tSpec: velerov1.VolumeSnapshotLocationSpec{\n\t\t\tProvider: provider,\n\t\t},\n\t}\n}\n\nfunc defaultBackup() *builder.BackupBuilder {\n\treturn builder.ForBackup(velerov1.DefaultNamespace, \"backup-1\").DefaultVolumesToFsBackup(false)\n}\n\nfunc toUnstructuredOrFail(t *testing.T, obj any) map[string]any {\n\tt.Helper()\n\n\tres, err := runtime.DefaultUnstructuredConverter.ToUnstructured(obj)\n\trequire.NoError(t, err)\n\n\treturn res\n}\n\n// assertTarballContents verifies that the gzipped tarball stored in the provided\n// backupFile contains exactly the file names specified.\nfunc assertTarballContents(t *testing.T, backupFile io.Reader, items ...string) {\n\tt.Helper()\n\n\tgzr, err := gzip.NewReader(backupFile)\n\trequire.NoError(t, err)\n\n\tr := tar.NewReader(gzr)\n\n\tvar files []string\n\tfor {\n\t\thdr, err := r.Next()\n\t\tif err == io.EOF {\n\t\t\tbreak\n\t\t}\n\t\trequire.NoError(t, err)\n\n\t\tfiles = append(files, hdr.Name)\n\t}\n\n\tsort.Strings(files)\n\tsort.Strings(items)\n\tassert.Equal(t, items, files)\n}\n\n// unstructuredObject is a type alias to improve readability.\ntype unstructuredObject map[string]any\n\n// assertTarballFileContents verifies that the gzipped tarball stored in the provided\n// backupFile contains the files specified as keys in 'want', and for each of those\n// files verifies that the content of the file is JSON and is equivalent to the JSON\n// content stored as values in 'want'.\nfunc assertTarballFileContents(t *testing.T, backupFile io.Reader, want map[string]unstructuredObject) {\n\tt.Helper()\n\n\tgzr, err := gzip.NewReader(backupFile)\n\trequire.NoError(t, err)\n\n\tr := tar.NewReader(gzr)\n\titems := make(map[string][]byte)\n\n\tfor {\n\t\thdr, err := r.Next()\n\t\tif err == io.EOF {\n\t\t\tbreak\n\t\t}\n\t\trequire.NoError(t, err)\n\n\t\tbytes, err := io.ReadAll(r)\n\t\trequire.NoError(t, err)\n\n\t\titems[hdr.Name] = bytes\n\t}\n\n\tfor name, wantItem := range want {\n\t\tgotData, ok := items[name]\n\t\tassert.True(t, ok, \"did not find item %s in tarball\", name)\n\t\tif !ok {\n\t\t\tcontinue\n\t\t}\n\n\t\t// json-unmarshal the data from the tarball\n\t\tvar got unstructuredObject\n\t\terr := json.Unmarshal(gotData, &got)\n\t\trequire.NoError(t, err)\n\t\tif err != nil {\n\t\t\tcontinue\n\t\t}\n\n\t\tassert.Equal(t, wantItem, got)\n\t}\n}\n\n// assertTarballOrdering ensures that resources were written to the tarball in the expected\n// order. Any resources *not* in orderedResources are required to come *after* all resources\n// in orderedResources, in any order.\nfunc assertTarballOrdering(t *testing.T, backupFile io.Reader, orderedResources ...string) {\n\tt.Helper()\n\n\tgzr, err := gzip.NewReader(backupFile)\n\trequire.NoError(t, err)\n\n\tr := tar.NewReader(gzr)\n\n\t// lastSeen tracks the index in 'orderedResources' of the last resource type\n\t// we saw in the tarball. Once we've seen a resource in 'orderedResources',\n\t// we should never see another instance of a prior resource.\n\tlastSeen := 0\n\n\tfor {\n\t\thdr, err := r.Next()\n\t\tif err == io.EOF {\n\t\t\tbreak\n\t\t}\n\t\trequire.NoError(t, err)\n\n\t\t// ignore files like metadata/version\n\t\tif !strings.HasPrefix(hdr.Name, \"resources/\") {\n\t\t\tcontinue\n\t\t}\n\n\t\t// get the resource name\n\t\tparts := strings.Split(hdr.Name, \"/\")\n\t\trequire.GreaterOrEqual(t, len(parts), 2)\n\t\tresourceName := parts[1]\n\n\t\t// Find the index in 'orderedResources' of the resource type for\n\t\t// the current tar item, if it exists. This index ('current') *must*\n\t\t// be greater than or equal to 'lastSeen', which was the last resource\n\t\t// we saw, since otherwise the current resource would be out of order. By\n\t\t// initializing current to len(ordered), we're saying that if the resource\n\t\t// is not explicitly in orederedResources, then it must come *after*\n\t\t// all orderedResources.\n\t\tcurrent := len(orderedResources)\n\t\tfor i, item := range orderedResources {\n\t\t\tif item == resourceName {\n\t\t\t\tcurrent = i\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\n\t\t// the index of the current resource must be the same as or greater than the index of\n\t\t// the last resource we saw for the backed-up order to be correct.\n\t\tassert.GreaterOrEqual(t, current, lastSeen, \"%s was backed up out of order\", resourceName)\n\t\tlastSeen = current\n\t}\n}\n\nfunc TestBackupNewResourceFiltering(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1.Backup\n\t\tapiResources []*test.APIResource\n\t\twant []string\n\t\tactions []biav2.BackupItemAction\n\t}{\n\t\t{\n\t\t\tname: \"no namespace-scoped resources + some cluster-scoped resources\",\n\t\t\tbackup: defaultBackup().IncludedClusterScopedResources(\"persistentvolumes\").ExcludedNamespaceScopedResources(\"*\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"testing\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/persistentvolumes/cluster/testing.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/testing.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"no namespace-scoped resources + all cluster-scoped resources\",\n\t\t\tbackup: defaultBackup().IncludedClusterScopedResources(\"*\").ExcludedNamespaceScopedResources(\"*\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/cluster/backups.velero.io.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/test1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/test2.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/v1beta1-preferredversion/cluster/backups.velero.io.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/test1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/test2.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"some namespace-scoped resources + no cluster-scoped resources 1\",\n\t\t\tbackup: defaultBackup().ExcludedClusterScopedResources(\"*\").IncludedNamespaces(\"foo\", \"zoo\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"some namespace-scoped resources + no cluster-scoped resources 2\",\n\t\t\tbackup: defaultBackup().ExcludedClusterScopedResources(\"*\").IncludedNamespaceScopedResources(\"pods\", \"deployments\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"some namespace-scoped resources + no cluster-scoped resources 3\",\n\t\t\tbackup: defaultBackup().ExcludedClusterScopedResources(\"*\").IncludedNamespaces(\"foo\").IncludedNamespaceScopedResources(\"pods\", \"deployments\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"some namespace-scoped resources + no cluster-scoped resources 4\",\n\t\t\tbackup: defaultBackup().ExcludedClusterScopedResources(\"*\").ExcludedNamespaceScopedResources(\"pods\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"some namespace-scoped resources + only related cluster-scoped resources 2\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"foo\").IncludedNamespaceScopedResources(\"pods\", \"persistentvolumeclaims\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Volumes(builder.ForVolume(\"foo\").PersistentVolumeClaimSource(\"test-1\").Result()).Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"foo\", \"test-1\").VolumeName(\"test1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/persistentvolumeclaims/namespaces/foo/test-1.json\",\n\t\t\t\t\"resources/persistentvolumeclaims/v1-preferredversion/namespaces/foo/test-1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/test1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/test1.json\",\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\t&pluggableAction{\n\t\t\t\t\tselector: velero.ResourceSelector{IncludedResources: []string{\"persistentvolumeclaims\"}},\n\t\t\t\t\texecuteFunc: func(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\t\t\t\t\t\tadditionalItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"test1\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn item, additionalItems, \"\", nil, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"some namespace-scoped resources + only related cluster-scoped resources 3\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"foo\").ExcludedNamespaceScopedResources(\"deployments\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Volumes(builder.ForVolume(\"foo\").PersistentVolumeClaimSource(\"test-1\").Result()).Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"foo\", \"test-1\").VolumeName(\"test1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/persistentvolumeclaims/namespaces/foo/test-1.json\",\n\t\t\t\t\"resources/persistentvolumeclaims/v1-preferredversion/namespaces/foo/test-1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/test1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/test1.json\",\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\t&pluggableAction{\n\t\t\t\t\tselector: velero.ResourceSelector{IncludedResources: []string{\"persistentvolumeclaims\"}},\n\t\t\t\t\texecuteFunc: func(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\t\t\t\t\t\tadditionalItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"test1\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn item, additionalItems, \"\", nil, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"some namespace-scoped resources + some additional cluster-scoped resources 1\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"foo\").IncludedClusterScopedResources(\"customresourcedefinitions\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"foo\", \"test-1\").VolumeName(\"test1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/cluster/backups.velero.io.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/v1beta1-preferredversion/cluster/backups.velero.io.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/persistentvolumeclaims/namespaces/foo/test-1.json\",\n\t\t\t\t\"resources/persistentvolumeclaims/v1-preferredversion/namespaces/foo/test-1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/test1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/test1.json\",\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\t&pluggableAction{\n\t\t\t\t\tselector: velero.ResourceSelector{IncludedResources: []string{\"persistentvolumeclaims\"}},\n\t\t\t\t\texecuteFunc: func(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\t\t\t\t\t\tadditionalItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"test1\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn item, additionalItems, \"\", nil, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"some namespace-scoped resources + some additional cluster-scoped resources 2\",\n\t\t\tbackup: defaultBackup().IncludedNamespaceScopedResources(\"persistentvolumeclaims\").IncludedClusterScopedResources(\"customresourcedefinitions\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"foo\", \"test-1\").VolumeName(\"test1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/cluster/backups.velero.io.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/v1beta1-preferredversion/cluster/backups.velero.io.json\",\n\t\t\t\t\"resources/persistentvolumeclaims/namespaces/foo/test-1.json\",\n\t\t\t\t\"resources/persistentvolumeclaims/v1-preferredversion/namespaces/foo/test-1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/test1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/test1.json\",\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\t&pluggableAction{\n\t\t\t\t\tselector: velero.ResourceSelector{IncludedResources: []string{\"persistentvolumeclaims\"}},\n\t\t\t\t\texecuteFunc: func(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\t\t\t\t\t\tadditionalItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"test1\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn item, additionalItems, \"\", nil, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"some namespace-scoped resources + some additional cluster-scoped resources 3\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"foo\").IncludedNamespaceScopedResources(\"pods\", \"persistentvolumeclaims\").IncludedClusterScopedResources(\"customresourcedefinitions\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"foo\", \"test-1\").VolumeName(\"test1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/cluster/backups.velero.io.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/v1beta1-preferredversion/cluster/backups.velero.io.json\",\n\t\t\t\t\"resources/persistentvolumeclaims/namespaces/foo/test-1.json\",\n\t\t\t\t\"resources/persistentvolumeclaims/v1-preferredversion/namespaces/foo/test-1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/test1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/test1.json\",\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\t&pluggableAction{\n\t\t\t\t\tselector: velero.ResourceSelector{IncludedResources: []string{\"persistentvolumeclaims\"}},\n\t\t\t\t\texecuteFunc: func(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\t\t\t\t\t\tadditionalItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"test1\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn item, additionalItems, \"\", nil, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"some namespace-scoped resources + some additional cluster-scoped resources 4\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"foo\").IncludedNamespaceScopedResources(\"pods\", \"persistentvolumeclaims\").IncludedClusterScopedResources(\"*\").ExcludedClusterScopedResources(\"customresourcedefinitions.apiextensions.k8s.io\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"foo\", \"test-1\").VolumeName(\"test1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/persistentvolumeclaims/namespaces/foo/test-1.json\",\n\t\t\t\t\"resources/persistentvolumeclaims/v1-preferredversion/namespaces/foo/test-1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/test1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/test2.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/test1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/test2.json\",\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t},\n\t\t\tactions: []biav2.BackupItemAction{\n\t\t\t\t&pluggableAction{\n\t\t\t\t\tselector: velero.ResourceSelector{IncludedResources: []string{\"persistentvolumeclaims\"}},\n\t\t\t\t\texecuteFunc: func(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\t\t\t\t\t\tadditionalItems := []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"test1\"},\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\treturn item, additionalItems, \"\", nil, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"some namespace-scoped resources + all cluster-scoped resources 1\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"foo\").IncludedClusterScopedResources(\"*\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"foo\", \"test-1\").VolumeName(\"test1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test2\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/persistentvolumeclaims/namespaces/foo/test-1.json\",\n\t\t\t\t\"resources/persistentvolumeclaims/v1-preferredversion/namespaces/foo/test-1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/test1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/test1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/test2.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/test2.json\",\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"some namespace-scoped resources + all cluster-scoped resources 2\",\n\t\t\tbackup: defaultBackup().IncludedNamespaceScopedResources(\"pods\").IncludedClusterScopedResources(\"*\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/cluster/backups.velero.io.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/v1beta1-preferredversion/cluster/backups.velero.io.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/test1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/test2.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/test1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/test2.json\",\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"some namespace-scoped resources + all cluster-scoped resources 3\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"foo\").IncludedNamespaceScopedResources(\"pods\").IncludedClusterScopedResources(\"*\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/cluster/backups.velero.io.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/v1beta1-preferredversion/cluster/backups.velero.io.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/test1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/test2.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/test1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/test2.json\",\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"all namespace-scoped resources + no cluster-scoped resources\",\n\t\t\tbackup: defaultBackup().ExcludedClusterScopedResources(\"*\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"all namespace-scoped resources + all cluster-scoped resources\",\n\t\t\tbackup: defaultBackup().IncludedClusterScopedResources(\"*\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"test2\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/cluster/backups.velero.io.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/v1beta1-preferredversion/cluster/backups.velero.io.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/test1.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/test2.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/test1.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/test2.json\",\n\t\t\t\t\"resources/pods/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/pods/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"namespace resource should be included even it's not specified in the include list, when IncludedNamespaces has specified value 1\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"foo\").IncludedNamespaceScopedResources(\"Secrets\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Secrets(\n\t\t\t\t\tbuilder.ForSecret(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForSecret(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"foo\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Namespaces(\n\t\t\t\t\tbuilder.ForNamespace(\"foo\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/namespaces/cluster/foo.json\",\n\t\t\t\t\"resources/namespaces/v1-preferredversion/cluster/foo.json\",\n\t\t\t\t\"resources/secrets/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/secrets/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"namespace resource should be included even it's not specified in the include list, when IncludedNamespaces has specified value 2\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"foo\").IncludedClusterScopedResources(\"persistentvolumes\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Secrets(\n\t\t\t\t\tbuilder.ForSecret(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForSecret(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"foo\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Namespaces(\n\t\t\t\t\tbuilder.ForNamespace(\"foo\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/namespaces/cluster/foo.json\",\n\t\t\t\t\"resources/namespaces/v1-preferredversion/cluster/foo.json\",\n\t\t\t\t\"resources/secrets/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/secrets/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/foo.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/foo.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"namespace resource should be included even it's not specified in the include list, when IncludedNamespaces is asterisk.\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"*\").IncludedClusterScopedResources(\"persistentvolumes\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Secrets(\n\t\t\t\t\tbuilder.ForSecret(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForSecret(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"foo\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Namespaces(\n\t\t\t\t\tbuilder.ForNamespace(\"foo\").Result(),\n\t\t\t\t\tbuilder.ForNamespace(\"zoo\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/namespaces/cluster/foo.json\",\n\t\t\t\t\"resources/namespaces/v1-preferredversion/cluster/foo.json\",\n\t\t\t\t\"resources/namespaces/cluster/zoo.json\",\n\t\t\t\t\"resources/namespaces/v1-preferredversion/cluster/zoo.json\",\n\t\t\t\t\"resources/secrets/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/secrets/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/secrets/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/secrets/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/foo.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/foo.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when all namespace-scoped resources are involved, cluster-scoped resources should be included too\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"*\").IncludedNamespaceScopedResources(\"*\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Secrets(\n\t\t\t\t\tbuilder.ForSecret(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForSecret(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"foo\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"bar\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Namespaces(\n\t\t\t\t\tbuilder.ForNamespace(\"foo\").Result(),\n\t\t\t\t\tbuilder.ForNamespace(\"zoo\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/namespaces/cluster/foo.json\",\n\t\t\t\t\"resources/namespaces/v1-preferredversion/cluster/foo.json\",\n\t\t\t\t\"resources/namespaces/cluster/zoo.json\",\n\t\t\t\t\"resources/namespaces/v1-preferredversion/cluster/zoo.json\",\n\t\t\t\t\"resources/secrets/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/secrets/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/secrets/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/secrets/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/foo.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/foo.json\",\n\t\t\t\t\"resources/persistentvolumes/cluster/bar.json\",\n\t\t\t\t\"resources/persistentvolumes/v1-preferredversion/cluster/bar.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"IncludedNamespaces is asterisk, but not all namespace-scoped resource types are include, additional cluster-scoped resources should not be included.\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"*\").IncludedNamespaceScopedResources(\"secrets\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Secrets(\n\t\t\t\t\tbuilder.ForSecret(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForSecret(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"foo\", \"bar\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"foo\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"bar\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Namespaces(\n\t\t\t\t\tbuilder.ForNamespace(\"foo\").Result(),\n\t\t\t\t\tbuilder.ForNamespace(\"zoo\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/namespaces/cluster/foo.json\",\n\t\t\t\t\"resources/namespaces/v1-preferredversion/cluster/foo.json\",\n\t\t\t\t\"resources/namespaces/cluster/zoo.json\",\n\t\t\t\t\"resources/namespaces/v1-preferredversion/cluster/zoo.json\",\n\t\t\t\t\"resources/secrets/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/secrets/namespaces/zoo/raz.json\",\n\t\t\t\t\"resources/secrets/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/secrets/v1-preferredversion/namespaces/zoo/raz.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Resource's CRD should be included\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"foo\").IncludedNamespaceScopedResources(\"volumesnapshotlocations.velero.io\", \"backups.velero.io\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"volumesnapshotlocations.velero.io\").Result(),\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"test.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.VSLs(\n\t\t\t\t\tbuilder.ForVolumeSnapshotLocation(\"foo\", \"bar\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Backups(\n\t\t\t\t\tbuilder.ForBackup(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/cluster/volumesnapshotlocations.velero.io.json\",\n\t\t\t\t\"resources/customresourcedefinitions.apiextensions.k8s.io/v1beta1-preferredversion/cluster/volumesnapshotlocations.velero.io.json\",\n\t\t\t\t\"resources/volumesnapshotlocations.velero.io/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/volumesnapshotlocations.velero.io/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Resource's CRD is not included, when CRD is excluded.\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"foo\").IncludedNamespaceScopedResources(\"volumesnapshotlocations.velero.io\", \"backups.velero.io\").ExcludedClusterScopedResources(\"customresourcedefinitions.apiextensions.k8s.io\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.CRDs(\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"backups.velero.io\").Result(),\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"volumesnapshotlocations.velero.io\").Result(),\n\t\t\t\t\tbuilder.ForCustomResourceDefinitionV1Beta1(\"test.velero.io\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.VSLs(\n\t\t\t\t\tbuilder.ForVolumeSnapshotLocation(\"foo\", \"bar\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.Backups(\n\t\t\t\t\tbuilder.ForBackup(\"zoo\", \"raz\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/volumesnapshotlocations.velero.io/namespaces/foo/bar.json\",\n\t\t\t\t\"resources/volumesnapshotlocations.velero.io/v1-preferredversion/namespaces/foo/bar.json\",\n\t\t\t},\n\t\t},\n\t}\n\n\titemBlockPool := StartItemBlockWorkerPool(t.Context(), 1, logrus.StandardLogger())\n\tdefer itemBlockPool.Stop()\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\th = newHarness(t, itemBlockPool)\n\t\t\t\treq = &Request{\n\t\t\t\t\tBackup: tc.backup,\n\t\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t\t}\n\t\t\t\tbackupFile = bytes.NewBuffer([]byte{})\n\t\t\t)\n\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.addItems(t, resource)\n\t\t\t}\n\n\t\t\th.backupper.Backup(h.log, req, backupFile, tc.actions, nil, nil)\n\n\t\t\tassertTarballContents(t, backupFile, append(tc.want, \"metadata/version\")...)\n\t\t})\n\t}\n}\n\nfunc TestBackupNamespaces(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1.Backup\n\t\tapiResources []*test.APIResource\n\t\twant []string\n\t}{\n\t\t{\n\t\t\tname: \"LabelSelector test\",\n\t\t\tbackup: defaultBackup().LabelSelector(&metav1.LabelSelector{MatchLabels: map[string]string{\"a\": \"b\"}}).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Namespaces(\n\t\t\t\t\tbuilder.ForNamespace(\"ns-1\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\t\tbuilder.ForNamespace(\"ns-2\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\t\tbuilder.ForNamespace(\"ns-3\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").ObjectMeta(builder.WithLabels(\"a\", \"b\")).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/namespaces/cluster/ns-1.json\",\n\t\t\t\t\"resources/namespaces/v1-preferredversion/cluster/ns-1.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/ns-1/deploy-1.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/ns-1/deploy-1.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"OrLabelSelector test\",\n\t\t\tbackup: defaultBackup().OrLabelSelector([]*metav1.LabelSelector{\n\t\t\t\t{MatchLabels: map[string]string{\"a\": \"b\"}},\n\t\t\t\t{MatchLabels: map[string]string{\"c\": \"d\"}},\n\t\t\t}).Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Namespaces(\n\t\t\t\t\tbuilder.ForNamespace(\"ns-1\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\t\tbuilder.ForNamespace(\"ns-2\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\t\tbuilder.ForNamespace(\"ns-3\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").ObjectMeta(builder.WithLabels(\"a\", \"b\")).Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"ns-2\", \"deploy-2\").ObjectMeta(builder.WithLabels(\"c\", \"d\")).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/namespaces/cluster/ns-1.json\",\n\t\t\t\t\"resources/namespaces/v1-preferredversion/cluster/ns-1.json\",\n\t\t\t\t\"resources/namespaces/cluster/ns-2.json\",\n\t\t\t\t\"resources/namespaces/v1-preferredversion/cluster/ns-2.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/ns-1/deploy-1.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/ns-1/deploy-1.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/ns-2/deploy-2.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/ns-2/deploy-2.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"LabelSelector and Namespace filtering test\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"ns-1\").LabelSelector(&metav1.LabelSelector{MatchLabels: map[string]string{\"a\": \"b\"}}).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Namespaces(\n\t\t\t\t\tbuilder.ForNamespace(\"ns-1\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\t\tbuilder.ForNamespace(\"ns-2\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\t\tbuilder.ForNamespace(\"ns-3\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").ObjectMeta(builder.WithLabels(\"a\", \"b\")).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/namespaces/cluster/ns-1.json\",\n\t\t\t\t\"resources/namespaces/v1-preferredversion/cluster/ns-1.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/ns-1/deploy-1.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/ns-1/deploy-1.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"LabelSelector and Namespace exclude filtering test\",\n\t\t\tbackup: defaultBackup().ExcludedNamespaces(\"ns-1\", \"ns-2\").LabelSelector(&metav1.LabelSelector{MatchLabels: map[string]string{\"a\": \"b\"}}).\n\t\t\t\tResult(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Namespaces(\n\t\t\t\t\tbuilder.ForNamespace(\"ns-1\").ObjectMeta(builder.WithLabels(\"a\", \"b\")).Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\t\tbuilder.ForNamespace(\"ns-2\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\t\tbuilder.ForNamespace(\"ns-3\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").ObjectMeta(builder.WithLabels(\"a\", \"b\")).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/namespaces/cluster/ns-1.json\",\n\t\t\t\t\"resources/namespaces/v1-preferredversion/cluster/ns-1.json\",\n\t\t\t\t\"resources/namespaces/cluster/ns-3.json\",\n\t\t\t\t\"resources/namespaces/v1-preferredversion/cluster/ns-3.json\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Empty namespace test\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"invalid*\").Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Namespaces(\n\t\t\t\t\tbuilder.ForNamespace(\"ns-1\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\t\tbuilder.ForNamespace(\"ns-2\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\t\tbuilder.ForNamespace(\"ns-3\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").ObjectMeta(builder.WithLabels(\"a\", \"b\")).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"Default namespace filter test\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Namespaces(\n\t\t\t\t\tbuilder.ForNamespace(\"ns-1\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\t\tbuilder.ForNamespace(\"ns-2\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\t\tbuilder.ForNamespace(\"ns-3\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\t),\n\t\t\t\ttest.Deployments(\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").ObjectMeta(builder.WithLabels(\"a\", \"b\")).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: []string{\n\t\t\t\t\"resources/namespaces/cluster/ns-1.json\",\n\t\t\t\t\"resources/namespaces/v1-preferredversion/cluster/ns-1.json\",\n\t\t\t\t\"resources/namespaces/cluster/ns-2.json\",\n\t\t\t\t\"resources/namespaces/v1-preferredversion/cluster/ns-2.json\",\n\t\t\t\t\"resources/namespaces/cluster/ns-3.json\",\n\t\t\t\t\"resources/namespaces/v1-preferredversion/cluster/ns-3.json\",\n\t\t\t\t\"resources/deployments.apps/namespaces/ns-1/deploy-1.json\",\n\t\t\t\t\"resources/deployments.apps/v1-preferredversion/namespaces/ns-1/deploy-1.json\",\n\t\t\t},\n\t\t},\n\t}\n\n\titemBlockPool := StartItemBlockWorkerPool(t.Context(), 1, logrus.StandardLogger())\n\tdefer itemBlockPool.Stop()\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\th = newHarness(t, itemBlockPool)\n\t\t\t\treq = &Request{\n\t\t\t\t\tBackup: tc.backup,\n\t\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\t\tBackedUpItems: NewBackedUpItemsMap(),\n\t\t\t\t\tWorkerPool: itemBlockPool,\n\t\t\t\t}\n\t\t\t\tbackupFile = bytes.NewBuffer([]byte{})\n\t\t\t)\n\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.addItems(t, resource)\n\t\t\t}\n\n\t\t\th.backupper.Backup(h.log, req, backupFile, nil, nil, nil)\n\n\t\t\tassertTarballContents(t, backupFile, append(tc.want, \"metadata/version\")...)\n\t\t})\n\t}\n}\n\nfunc TestUpdateVolumeInfos(t *testing.T) {\n\ttimeExample := time.Date(2014, 6, 5, 11, 56, 45, 0, time.Local)\n\tnow := metav1.NewTime(timeExample)\n\tlogger := logrus.StandardLogger()\n\n\ttests := []struct {\n\t\tname string\n\t\toperations []*itemoperation.BackupOperation\n\t\tdataUpload *velerov2alpha1.DataUpload\n\t\tvolumeInfos []*volume.BackupVolumeInfo\n\t\texpectedVolumeInfos []*volume.BackupVolumeInfo\n\t}{\n\t\t{\n\t\t\tname: \"CSISnapshot VolumeInfo update with Operation fails\",\n\t\t\toperations: []*itemoperation.BackupOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\t\t\tOperationID: \"test-operation\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tError: \"failed\",\n\t\t\t\t\t\tUpdated: &now,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeInfos: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tBackupMethod: volume.CSISnapshot,\n\t\t\t\t\tCompletionTimestamp: &metav1.Time{},\n\t\t\t\t\tCSISnapshotInfo: &volume.CSISnapshotInfo{\n\t\t\t\t\t\tOperationID: \"test-operation\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tBackupMethod: volume.CSISnapshot,\n\t\t\t\t\tCompletionTimestamp: &now,\n\t\t\t\t\tResult: volume.VolumeResultFailed,\n\t\t\t\t\tCSISnapshotInfo: &volume.CSISnapshotInfo{\n\t\t\t\t\t\tOperationID: \"test-operation\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"CSISnapshot VolumeInfo update\",\n\t\t\toperations: []*itemoperation.BackupOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\t\t\tOperationID: \"test-operation\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tUpdated: &now,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeInfos: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tBackupMethod: volume.CSISnapshot,\n\t\t\t\t\tCompletionTimestamp: &metav1.Time{},\n\t\t\t\t\tCSISnapshotInfo: &volume.CSISnapshotInfo{\n\t\t\t\t\t\tOperationID: \"test-operation\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tBackupMethod: volume.CSISnapshot,\n\t\t\t\t\tCompletionTimestamp: &now,\n\t\t\t\t\tResult: volume.VolumeResultSucceeded,\n\t\t\t\t\tCSISnapshotInfo: &volume.CSISnapshotInfo{\n\t\t\t\t\t\tOperationID: \"test-operation\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"DataUpload VolumeInfo update with fail phase\",\n\t\t\toperations: []*itemoperation.BackupOperation{},\n\t\t\tdataUpload: builder.ForDataUpload(\"velero\", \"du-1\").\n\t\t\t\tCompletionTimestamp(&now).\n\t\t\t\tCSISnapshot(&velerov2alpha1.CSISnapshotSpec{VolumeSnapshot: \"vs-1\"}).\n\t\t\t\tSnapshotID(\"snapshot-id\").\n\t\t\t\tProgress(shared.DataMoveOperationProgress{TotalBytes: 1000}).\n\t\t\t\tIncrementalBytes(500).\n\t\t\t\tPhase(velerov2alpha1.DataUploadPhaseFailed).\n\t\t\t\tSourceNamespace(\"ns-1\").\n\t\t\t\tSourcePVC(\"pvc-1\").\n\t\t\t\tResult(),\n\t\t\tvolumeInfos: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"pvc-1\",\n\t\t\t\t\tPVCNamespace: \"ns-1\",\n\t\t\t\t\tCompletionTimestamp: &metav1.Time{},\n\t\t\t\t\tSnapshotDataMovementInfo: &volume.SnapshotDataMovementInfo{\n\t\t\t\t\t\tDataMover: \"velero\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"pvc-1\",\n\t\t\t\t\tPVCNamespace: \"ns-1\",\n\t\t\t\t\tCompletionTimestamp: &now,\n\t\t\t\t\tResult: volume.VolumeResultFailed,\n\t\t\t\t\tSnapshotDataMovementInfo: &volume.SnapshotDataMovementInfo{\n\t\t\t\t\t\tDataMover: \"velero\",\n\t\t\t\t\t\tRetainedSnapshot: \"vs-1\",\n\t\t\t\t\t\tSnapshotHandle: \"snapshot-id\",\n\t\t\t\t\t\tSize: 1000,\n\t\t\t\t\t\tIncrementalSize: 500,\n\t\t\t\t\t\tPhase: velerov2alpha1.DataUploadPhaseFailed,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"DataUpload VolumeInfo update\",\n\t\t\toperations: []*itemoperation.BackupOperation{},\n\t\t\tdataUpload: builder.ForDataUpload(\"velero\", \"du-1\").\n\t\t\t\tCompletionTimestamp(&now).\n\t\t\t\tCSISnapshot(&velerov2alpha1.CSISnapshotSpec{VolumeSnapshot: \"vs-1\"}).\n\t\t\t\tSnapshotID(\"snapshot-id\").\n\t\t\t\tProgress(shared.DataMoveOperationProgress{TotalBytes: 1000}).\n\t\t\t\tIncrementalBytes(500).\n\t\t\t\tPhase(velerov2alpha1.DataUploadPhaseCompleted).\n\t\t\t\tSourceNamespace(\"ns-1\").\n\t\t\t\tSourcePVC(\"pvc-1\").\n\t\t\t\tResult(),\n\t\t\tvolumeInfos: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"pvc-1\",\n\t\t\t\t\tPVCNamespace: \"ns-1\",\n\t\t\t\t\tCompletionTimestamp: &metav1.Time{},\n\t\t\t\t\tSnapshotDataMovementInfo: &volume.SnapshotDataMovementInfo{\n\t\t\t\t\t\tDataMover: \"velero\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"pvc-1\",\n\t\t\t\t\tPVCNamespace: \"ns-1\",\n\t\t\t\t\tCompletionTimestamp: &now,\n\t\t\t\t\tResult: volume.VolumeResultSucceeded,\n\t\t\t\t\tSnapshotDataMovementInfo: &volume.SnapshotDataMovementInfo{\n\t\t\t\t\t\tDataMover: \"velero\",\n\t\t\t\t\t\tRetainedSnapshot: \"vs-1\",\n\t\t\t\t\t\tSnapshotHandle: \"snapshot-id\",\n\t\t\t\t\t\tSize: 1000,\n\t\t\t\t\t\tIncrementalSize: 500,\n\t\t\t\t\t\tPhase: velerov2alpha1.DataUploadPhaseCompleted,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\t// This is an error case. No crash happen here is good enough.\n\t\t\tname: \"VolumeInfo doesn't have SnapshotDataMovementInfo when there is a matching DataUpload\",\n\t\t\toperations: []*itemoperation.BackupOperation{},\n\t\t\tdataUpload: builder.ForDataUpload(\"velero\", \"du-1\").\n\t\t\t\tCompletionTimestamp(&now).\n\t\t\t\tCSISnapshot(&velerov2alpha1.CSISnapshotSpec{VolumeSnapshot: \"vs-1\"}).\n\t\t\t\tSnapshotID(\"snapshot-id\").\n\t\t\t\tProgress(shared.DataMoveOperationProgress{TotalBytes: 1000}).\n\t\t\t\tIncrementalBytes(500).\n\t\t\t\tPhase(velerov2alpha1.DataUploadPhaseCompleted).\n\t\t\t\tSourceNamespace(\"ns-1\").\n\t\t\t\tSourcePVC(\"pvc-1\").\n\t\t\t\tResult(),\n\t\t\tvolumeInfos: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"pvc-1\",\n\t\t\t\t\tPVCNamespace: \"ns-1\",\n\t\t\t\t\tCompletionTimestamp: &metav1.Time{},\n\t\t\t\t\tSnapshotDataMovementInfo: nil,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumeInfos: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"pvc-1\",\n\t\t\t\t\tPVCNamespace: \"ns-1\",\n\t\t\t\t\tCompletionTimestamp: &metav1.Time{},\n\t\t\t\t\tSnapshotDataMovementInfo: nil,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tunstructures := []unstructured.Unstructured{}\n\t\t\tif tc.dataUpload != nil {\n\t\t\t\tduMap, error := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.dataUpload)\n\t\t\t\trequire.NoError(t, error)\n\t\t\t\tunstructures = append(unstructures,\n\t\t\t\t\tunstructured.Unstructured{\n\t\t\t\t\t\tObject: duMap,\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\n\t\t\trequire.NoError(t, updateVolumeInfos(tc.volumeInfos, unstructures, tc.operations, logger))\n\t\t\tif len(tc.expectedVolumeInfos) > 0 {\n\t\t\t\trequire.Equal(t, tc.expectedVolumeInfos[0].CompletionTimestamp, tc.volumeInfos[0].CompletionTimestamp)\n\t\t\t\trequire.Equal(t, tc.expectedVolumeInfos[0].SnapshotDataMovementInfo, tc.volumeInfos[0].SnapshotDataMovementInfo)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestPutVolumeInfos(t *testing.T) {\n\tbackupName := \"backup-01\"\n\n\tbackupStore := new(persistencemocks.BackupStore)\n\n\tbackupStore.On(\"PutBackupVolumeInfos\", mock.Anything, mock.Anything).Return(nil)\n\n\trequire.NoError(t, putVolumeInfos(backupName, []*volume.BackupVolumeInfo{}, backupStore))\n}\n\ntype fakeSingleObjectBackupStoreGetter struct {\n\tstore persistence.BackupStore\n}\n\nfunc (f *fakeSingleObjectBackupStoreGetter) Get(*velerov1.BackupStorageLocation, persistence.ObjectStoreGetter, logrus.FieldLogger) (persistence.BackupStore, error) {\n\treturn f.store, nil\n}\n\n// NewFakeSingleObjectBackupStoreGetter returns an ObjectBackupStoreGetter\n// that will return only the given BackupStore.\nfunc NewFakeSingleObjectBackupStoreGetter(store persistence.BackupStore) persistence.ObjectBackupStoreGetter {\n\treturn &fakeSingleObjectBackupStoreGetter{store: store}\n}\n" }, { "path": "pkg/backup/delete_helpers.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"fmt\"\n\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n)\n\n// NewDeleteBackupRequest creates a DeleteBackupRequest for the backup identified by name and uid.\nfunc NewDeleteBackupRequest(name string, uid string) *velerov1api.DeleteBackupRequest {\n\treturn &velerov1api.DeleteBackupRequest{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tGenerateName: name + \"-\",\n\t\t\tLabels: map[string]string{\n\t\t\t\tvelerov1api.BackupNameLabel: label.GetValidName(name),\n\t\t\t\tvelerov1api.BackupUIDLabel: uid,\n\t\t\t},\n\t\t},\n\t\tSpec: velerov1api.DeleteBackupRequestSpec{\n\t\t\tBackupName: name,\n\t\t},\n\t}\n}\n\n// NewDeleteBackupRequestListOptions creates a ListOptions with a label selector configured to\n// find DeleteBackupRequests for the backup identified by name and uid.\nfunc NewDeleteBackupRequestListOptions(name, uid string) metav1.ListOptions {\n\treturn metav1.ListOptions{\n\t\tLabelSelector: fmt.Sprintf(\"%s=%s,%s=%s\", velerov1api.BackupNameLabel, label.GetValidName(name), velerov1api.BackupUIDLabel, uid),\n\t}\n}\n" }, { "path": "pkg/backup/item_backupper.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"archive/tar\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\t\"k8s.io/apimachinery/pkg/api/meta\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\tkubeerrs \"k8s.io/apimachinery/pkg/util/errors\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\tkbClient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/hook\"\n\t\"github.com/vmware-tanzu/velero/internal/resourcepolicies\"\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\t\"github.com/vmware-tanzu/velero/internal/volumehelper\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/archive\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/discovery\"\n\t\"github.com/vmware-tanzu/velero/pkg/features\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemblock\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvsv1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/volumesnapshotter/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/podvolume\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\tcsiutil \"github.com/vmware-tanzu/velero/pkg/util/csi\"\n)\n\nconst (\n\tcsiBIAPluginName = \"velero.io/csi-pvc-backupper\"\n\tvsphereBIAPluginName = \"velero.io/vsphere-pvc-backupper\"\n)\n\n// itemBackupper can back up individual items to a tar writer.\ntype itemBackupper struct {\n\tbackupRequest *Request\n\ttarWriter tarWriter\n\tdynamicFactory client.DynamicFactory\n\tkbClient kbClient.Client\n\tdiscoveryHelper discovery.Helper\n\tpodVolumeBackupper podvolume.Backupper\n\tpodVolumeContext context.Context\n\tpodVolumeSnapshotTracker *podvolume.Tracker\n\tkubernetesBackupper *kubernetesBackupper\n\tvolumeSnapshotterCache *VolumeSnapshotterCache\n\titemHookHandler hook.ItemHookHandler\n\thookTracker *hook.HookTracker\n\tvolumeHelperImpl volumehelper.VolumeHelper\n}\n\ntype FileForArchive struct {\n\tFilePath string\n\tHeader *tar.Header\n\tFileBytes []byte\n}\n\n// backupItem backs up an individual item to tarWriter. The item may be excluded based on the\n// namespaces IncludesExcludes list.\n// If finalize is true, then it returns the bytes instead of writing them to the tarWriter\n// In addition to the error return, backupItem also returns a bool indicating whether the item\n// was actually backed up.\nfunc (ib *itemBackupper) backupItem(logger logrus.FieldLogger, obj runtime.Unstructured, groupResource schema.GroupResource, preferredGVR schema.GroupVersionResource, mustInclude, finalize bool, itemBlock *BackupItemBlock) (bool, []FileForArchive, error) {\n\tselectedForBackup, files, err := ib.backupItemInternal(logger, obj, groupResource, preferredGVR, mustInclude, finalize, itemBlock)\n\t// return if not selected, an error occurred, there are no files to add, or for finalize\n\tif !selectedForBackup || err != nil || len(files) == 0 || finalize {\n\t\treturn selectedForBackup, files, err\n\t}\n\tib.tarWriter.Lock()\n\tdefer ib.tarWriter.Unlock()\n\tfor _, file := range files {\n\t\tif err := ib.tarWriter.WriteHeader(file.Header); err != nil {\n\t\t\treturn false, []FileForArchive{}, errors.WithStack(err)\n\t\t}\n\n\t\tif _, err := ib.tarWriter.Write(file.FileBytes); err != nil {\n\t\t\treturn false, []FileForArchive{}, errors.WithStack(err)\n\t\t}\n\t}\n\treturn true, []FileForArchive{}, nil\n}\n\nfunc (ib *itemBackupper) itemInclusionChecks(log logrus.FieldLogger, mustInclude bool, metadata metav1.Object, obj runtime.Unstructured, groupResource schema.GroupResource) bool {\n\tif mustInclude {\n\t\tlog.Infof(\"Skipping the exclusion checks for this resource\")\n\t} else {\n\t\tif metadata.GetLabels()[velerov1api.ExcludeFromBackupLabel] == \"true\" {\n\t\t\tlog.Infof(\"Excluding item because it has label %s=true\", velerov1api.ExcludeFromBackupLabel)\n\t\t\tib.trackSkippedPV(obj, groupResource, \"\", fmt.Sprintf(\"item has label %s=true\", velerov1api.ExcludeFromBackupLabel), log)\n\t\t\treturn false\n\t\t}\n\t\t// NOTE: we have to re-check namespace & resource includes/excludes because it's possible that\n\t\t// backupItem can be invoked by a custom action.\n\t\tnamespace := metadata.GetNamespace()\n\t\tif namespace != \"\" && !ib.backupRequest.NamespaceIncludesExcludes.ShouldInclude(namespace) {\n\t\t\tlog.Info(\"Excluding item because namespace is excluded\")\n\t\t\treturn false\n\t\t}\n\n\t\t// NOTE: we specifically allow namespaces to be backed up even if it's excluded.\n\t\t// This check is more permissive for cluster resources to let those passed in by\n\t\t// plugins' additional items to get involved.\n\t\t// Only expel cluster resource when it's specifically listed in the excluded list here.\n\t\tif namespace == \"\" && groupResource != kuberesource.Namespaces &&\n\t\t\tib.backupRequest.ResourceIncludesExcludes.ShouldExclude(groupResource.String()) {\n\t\t\tlog.Info(\"Excluding item because resource is cluster-scoped and is excluded by cluster filter.\")\n\t\t\treturn false\n\t\t}\n\n\t\t// Only check namespace-scoped resource to avoid expelling cluster resources\n\t\t// are not specified in included list.\n\t\tif namespace != \"\" && !ib.backupRequest.ResourceIncludesExcludes.ShouldInclude(groupResource.String()) {\n\t\t\tlog.Info(\"Excluding item because resource is excluded\")\n\t\t\treturn false\n\t\t}\n\t}\n\n\tif metadata.GetDeletionTimestamp() != nil {\n\t\tlog.Info(\"Skipping item because it's being deleted.\")\n\t\treturn false\n\t}\n\treturn true\n}\n\nfunc (ib *itemBackupper) backupItemInternal(logger logrus.FieldLogger, obj runtime.Unstructured, groupResource schema.GroupResource, preferredGVR schema.GroupVersionResource, mustInclude, finalize bool, itemBlock *BackupItemBlock) (bool, []FileForArchive, error) {\n\tvar itemFiles []FileForArchive\n\tmetadata, err := meta.Accessor(obj)\n\tif err != nil {\n\t\treturn false, itemFiles, err\n\t}\n\n\tnamespace := metadata.GetNamespace()\n\tname := metadata.GetName()\n\n\tlog := logger.WithFields(map[string]any{\n\t\t\"name\": name,\n\t\t\"resource\": groupResource.String(),\n\t\t\"namespace\": namespace,\n\t})\n\n\tif !ib.itemInclusionChecks(log, mustInclude, metadata, obj, groupResource) {\n\t\treturn false, itemFiles, nil\n\t}\n\n\tkey := itemKey{\n\t\tresource: resourceKey(obj),\n\t\tnamespace: namespace,\n\t\tname: name,\n\t}\n\n\tif ib.backupRequest.BackedUpItems.Has(key) {\n\t\tlog.Info(\"Skipping item because it's already been backed up.\")\n\t\t// returning true since this item *is* in the backup, even though we're not backing it up here\n\t\treturn true, itemFiles, nil\n\t}\n\tib.backupRequest.BackedUpItems.AddItem(key)\n\tlog.Info(\"Backing up item\")\n\n\tvar (\n\t\tbackupErrs []error\n\t\tpod *corev1api.Pod\n\t\tpvbVolumes []string\n\t)\n\n\tif optedOut, podName := ib.podVolumeSnapshotTracker.OptedoutByPod(namespace, name); optedOut {\n\t\tib.trackSkippedPV(obj, groupResource, podVolumeApproach, fmt.Sprintf(\"opted out due to annotation in pod %s\", podName), log)\n\t}\n\n\tif groupResource == kuberesource.Pods {\n\t\t// pod needs to be initialized for the unstructured converter\n\t\tpod = new(corev1api.Pod)\n\t\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(obj.UnstructuredContent(), pod); err != nil {\n\t\t\tbackupErrs = append(backupErrs, errors.WithStack(err))\n\t\t\t// nil it on error since it's not valid\n\t\t\tpod = nil\n\t\t} else {\n\t\t\t// Get the list of volumes to back up using pod volume backup from the pod's annotations\n\t\t\t// or volume policy approach. Remove from this list any volumes that use a PVC that we've\n\t\t\t// already backed up (this would be in a read-write-many scenario,\n\t\t\t// where it's been backed up from another pod), since we don't need >1 backup per PVC.\n\t\t\tfor _, volume := range pod.Spec.Volumes {\n\t\t\t\tshouldDoFSBackup, err := ib.volumeHelperImpl.ShouldPerformFSBackup(volume, *pod)\n\t\t\t\tif err != nil {\n\t\t\t\t\tbackupErrs = append(backupErrs, errors.WithStack(err))\n\t\t\t\t}\n\n\t\t\t\tif shouldDoFSBackup {\n\t\t\t\t\t// track the volumes backing up by PVB , so that when we backup PVCs/PVs\n\t\t\t\t\t// via an item action in the next step, we don't snapshot PVs that will have their data backed up\n\t\t\t\t\t// with pod volume backup.\n\t\t\t\t\tib.podVolumeSnapshotTracker.Track(pod, volume.Name)\n\n\t\t\t\t\tif found, pvcName := ib.podVolumeSnapshotTracker.TakenForPodVolume(pod, volume.Name); found {\n\t\t\t\t\t\tlog.WithFields(map[string]any{\n\t\t\t\t\t\t\t\"podVolume\": volume,\n\t\t\t\t\t\t\t\"pvcName\": pvcName,\n\t\t\t\t\t\t}).Info(\"Pod volume uses a persistent volume claim which has already been backed up from another pod, skipping.\")\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\t\t\t\t\tpvbVolumes = append(pvbVolumes, volume.Name)\n\t\t\t\t} else {\n\t\t\t\t\tib.podVolumeSnapshotTracker.Optout(pod, volume.Name)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\t// capture the version of the object before invoking plugin actions as the plugin may update\n\t// the group version of the object.\n\tversionPath := resourceVersion(obj)\n\n\tupdatedObj, additionalItemFiles, err := ib.executeActions(log, obj, groupResource, name, namespace, metadata, finalize, itemBlock)\n\tif err != nil {\n\t\tbackupErrs = append(backupErrs, err)\n\t\treturn false, itemFiles, kubeerrs.NewAggregate(backupErrs)\n\t}\n\n\t// If err is nil and updatedObj is nil, it means the item is skipped by plugin action,\n\t// we should return here to avoid backing up the item, and avoid potential NPE in the following code.\n\tif updatedObj == nil {\n\t\tlog.Infof(\"Remove item from the backup's backupItems list and totalItems list because it's skipped by plugin action.\")\n\t\tib.backupRequest.BackedUpItems.DeleteItem(key)\n\t\treturn false, itemFiles, nil\n\t}\n\n\titemFiles = append(itemFiles, additionalItemFiles...)\n\tobj = updatedObj\n\tif metadata, err = meta.Accessor(obj); err != nil {\n\t\treturn false, itemFiles, errors.WithStack(err)\n\t}\n\t// update name and namespace in case they were modified in an action\n\tname = metadata.GetName()\n\tnamespace = metadata.GetNamespace()\n\n\tif groupResource == kuberesource.PersistentVolumes {\n\t\tif err := ib.addVolumeInfo(obj, log); err != nil {\n\t\t\tbackupErrs = append(backupErrs, err)\n\t\t}\n\n\t\tif err := ib.takePVSnapshot(obj, log); err != nil {\n\t\t\tbackupErrs = append(backupErrs, err)\n\t\t}\n\t}\n\n\tif groupResource == kuberesource.Pods && pod != nil {\n\t\t// this function will return partial results, so process podVolumeBackups\n\t\t// even if there are errors.\n\t\tpodVolumeBackups, podVolumePVCBackupSummary, errs := ib.backupPodVolumes(log, pod, pvbVolumes)\n\n\t\tbackupErrs = append(backupErrs, errs...)\n\n\t\t// Mark the volumes that has been processed by pod volume backup as Taken in the tracker.\n\t\tfor _, pvb := range podVolumeBackups {\n\t\t\tib.podVolumeSnapshotTracker.Take(pod, pvb.Spec.Volume)\n\t\t}\n\n\t\t// Track/Untrack the volumes based on podVolumePVCBackupSummary\n\t\tif podVolumePVCBackupSummary != nil {\n\t\t\tfor _, skippedPVC := range podVolumePVCBackupSummary.Skipped {\n\t\t\t\tif obj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(skippedPVC.PVC); err != nil {\n\t\t\t\t\tbackupErrs = append(backupErrs, errors.WithStack(err))\n\t\t\t\t} else {\n\t\t\t\t\tib.trackSkippedPV(&unstructured.Unstructured{Object: obj}, kuberesource.PersistentVolumeClaims,\n\t\t\t\t\t\tpodVolumeApproach, skippedPVC.Reason, log)\n\t\t\t\t}\n\t\t\t}\n\t\t\tfor _, pvc := range podVolumePVCBackupSummary.Backedup {\n\t\t\t\tif obj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(pvc); err != nil {\n\t\t\t\t\tbackupErrs = append(backupErrs, errors.WithStack(err))\n\t\t\t\t} else {\n\t\t\t\t\tib.unTrackSkippedPV(&unstructured.Unstructured{Object: obj}, kuberesource.PersistentVolumeClaims, log)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\tif len(backupErrs) != 0 {\n\t\treturn false, itemFiles, kubeerrs.NewAggregate(backupErrs)\n\t}\n\n\titemBytes, err := json.Marshal(obj.UnstructuredContent())\n\tif err != nil {\n\t\treturn false, itemFiles, errors.WithStack(err)\n\t}\n\n\tif versionPath == preferredGVR.Version {\n\t\t// backing up preferred version backup without API Group version - for backward compatibility\n\t\tlog.Debugf(\"Resource %s/%s, version= %s, preferredVersion=%s\", groupResource.String(), name, versionPath, preferredGVR.Version)\n\t\titemFiles = append(itemFiles, getFileForArchive(namespace, name, groupResource.String(), \"\", itemBytes))\n\t\tversionPath = versionPath + velerov1api.PreferredVersionDir\n\t}\n\n\titemFiles = append(itemFiles, getFileForArchive(namespace, name, groupResource.String(), versionPath, itemBytes))\n\treturn true, itemFiles, nil\n}\n\nfunc getFileForArchive(namespace, name, groupResource, versionPath string, itemBytes []byte) FileForArchive {\n\tfilePath := archive.GetVersionedItemFilePath(\"\", groupResource, namespace, name, versionPath)\n\thdr := &tar.Header{\n\t\tName: filePath,\n\t\tSize: int64(len(itemBytes)),\n\t\tTypeflag: tar.TypeReg,\n\t\tMode: 0755,\n\t\tModTime: time.Now(),\n\t}\n\treturn FileForArchive{FilePath: filePath, Header: hdr, FileBytes: itemBytes}\n}\n\n// backupPodVolumes triggers pod volume backups of the specified pod volumes, and returns a list of PodVolumeBackups\n// for volumes that were successfully backed up, and a slice of any errors that were encountered.\nfunc (ib *itemBackupper) backupPodVolumes(log logrus.FieldLogger, pod *corev1api.Pod, volumes []string) ([]*velerov1api.PodVolumeBackup, *podvolume.PVCBackupSummary, []error) {\n\tif len(volumes) == 0 {\n\t\treturn nil, nil, nil\n\t}\n\n\tif ib.podVolumeBackupper == nil {\n\t\tlog.Warn(\"No pod volume backupper, not backing up pod's volumes\")\n\t\treturn nil, nil, nil\n\t}\n\n\treturn ib.podVolumeBackupper.BackupPodVolumes(ib.backupRequest.Backup, pod, volumes, ib.backupRequest.ResPolicies, log)\n}\n\nfunc (ib *itemBackupper) executeActions(\n\tlog logrus.FieldLogger,\n\tobj runtime.Unstructured,\n\tgroupResource schema.GroupResource,\n\tname, namespace string,\n\tmetadata metav1.Object,\n\tfinalize bool,\n\titemBlock *BackupItemBlock,\n) (runtime.Unstructured, []FileForArchive, error) {\n\tvar itemFiles []FileForArchive\n\tfor _, action := range ib.backupRequest.ResolvedActions {\n\t\tif !action.ShouldUse(groupResource, namespace, metadata, log) {\n\t\t\tcontinue\n\t\t}\n\t\tlog.Info(\"Executing custom action\")\n\t\tactionName := action.Name()\n\t\tif act, err := ib.getMatchAction(obj, groupResource, actionName); err != nil {\n\t\t\treturn nil, itemFiles, errors.WithStack(err)\n\t\t} else if act != nil && act.Type == resourcepolicies.Skip {\n\t\t\tlog.Infof(\"Skip executing Backup Item Action: %s of resource %s: %s/%s for the matched resource policies\", actionName, groupResource, namespace, name)\n\t\t\tib.trackSkippedPV(obj, groupResource, \"\", \"skipped due to resource policy \", log)\n\t\t\tcontinue\n\t\t}\n\n\t\t// If the EnableCSI feature is not enabled, but the executing action is from CSI plugin, skip the action.\n\t\tif csiutil.ShouldSkipAction(actionName) {\n\t\t\tlog.Infof(\"Skip action %s for resource %s:%s/%s, because the CSI feature is not enabled. Feature setting is %s.\",\n\t\t\t\tactionName, groupResource.String(), metadata.GetNamespace(), metadata.GetName(), features.Serialize())\n\t\t\tcontinue\n\t\t}\n\n\t\tif groupResource == kuberesource.PersistentVolumeClaims &&\n\t\t\tactionName == csiBIAPluginName {\n\t\t\tsnapshotVolume, err := ib.volumeHelperImpl.ShouldPerformSnapshot(obj, kuberesource.PersistentVolumeClaims)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, itemFiles, errors.WithStack(err)\n\t\t\t}\n\n\t\t\tif !snapshotVolume {\n\t\t\t\tib.trackSkippedPV(\n\t\t\t\t\tobj,\n\t\t\t\t\tkuberesource.PersistentVolumeClaims,\n\t\t\t\t\tvolumeSnapshotApproach,\n\t\t\t\t\t\"not satisfy the criteria for VolumePolicy or the legacy snapshot way\",\n\t\t\t\t\tlog,\n\t\t\t\t)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t}\n\n\t\tupdatedItem, additionalItemIdentifiers, operationID, postOperationItems, err := action.Execute(obj, ib.backupRequest.Backup)\n\t\tif err != nil {\n\t\t\treturn nil, itemFiles, errors.Wrapf(err, \"error executing custom action (groupResource=%s, namespace=%s, name=%s)\", groupResource.String(), namespace, name)\n\t\t}\n\n\t\tu := &unstructured.Unstructured{Object: updatedItem.UnstructuredContent()}\n\n\t\tif _, ok := u.GetAnnotations()[velerov1api.SkipFromBackupAnnotation]; ok {\n\t\t\tlog.Infof(\"Resource (groupResource=%s, namespace=%s, name=%s) is skipped from backup by action %s.\",\n\t\t\t\tgroupResource.String(), namespace, name, actionName)\n\t\t\treturn nil, itemFiles, nil\n\t\t}\n\n\t\tif actionName == csiBIAPluginName {\n\t\t\tif additionalItemIdentifiers == nil && u.GetAnnotations()[velerov1api.SkippedNoCSIPVAnnotation] == \"true\" {\n\t\t\t\t// snapshot was skipped by CSI plugin\n\t\t\t\tlog.Infof(\"skip CSI snapshot for PVC %s as it's not a CSI compatible volume\", namespace+\"/\"+name)\n\t\t\t\tib.trackSkippedPV(obj, groupResource, csiSnapshotApproach, \"skipped b/c it's not a CSI volume\", log)\n\t\t\t\tdelete(u.GetAnnotations(), velerov1api.SkippedNoCSIPVAnnotation)\n\t\t\t} else {\n\t\t\t\t// the snapshot has been taken by the BIA plugin\n\t\t\t\tlog.Infof(\"Untrack the PVC %s, because it's backed up by CSI BIA.\", namespace+\"/\"+name)\n\t\t\t\tib.unTrackSkippedPV(obj, kuberesource.PersistentVolumeClaims, log)\n\t\t\t}\n\t\t}\n\n\t\tmustInclude := u.GetAnnotations()[velerov1api.MustIncludeAdditionalItemAnnotation] == \"true\" || finalize\n\t\t// remove the annotation as it's for communication between BIA and velero server,\n\t\t// we don't want the resource be restored with this annotation.\n\t\tdelete(u.GetAnnotations(), velerov1api.MustIncludeAdditionalItemAnnotation)\n\t\tobj = u\n\n\t\t// If async plugin started async operation, add it to the ItemOperations list\n\t\t// ignore during finalize phase\n\t\tif operationID != \"\" {\n\t\t\tif finalize {\n\t\t\t\treturn nil, itemFiles, fmt.Errorf(\"backup Item Action created operation during finalize (groupResource=%s, namespace=%s, name=%s)\", groupResource.String(), namespace, name)\n\t\t\t}\n\t\t\tresourceIdentifier := velero.ResourceIdentifier{\n\t\t\t\tGroupResource: groupResource,\n\t\t\t\tNamespace: namespace,\n\t\t\t\tName: name,\n\t\t\t}\n\t\t\tnow := metav1.Now()\n\t\t\tnewOperation := itemoperation.BackupOperation{\n\t\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\t\tBackupName: ib.backupRequest.Backup.Name,\n\t\t\t\t\tBackupUID: string(ib.backupRequest.Backup.UID),\n\t\t\t\t\tBackupItemAction: action.Name(),\n\t\t\t\t\tResourceIdentifier: resourceIdentifier,\n\t\t\t\t\tOperationID: operationID,\n\t\t\t\t},\n\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\tPhase: itemoperation.OperationPhaseNew,\n\t\t\t\t\tCreated: &now,\n\t\t\t\t},\n\t\t\t}\n\t\t\tnewOperation.Spec.PostOperationItems = postOperationItems\n\t\t\titemOperList := ib.backupRequest.GetItemOperationsList()\n\t\t\t*itemOperList = append(*itemOperList, &newOperation)\n\t\t}\n\n\t\tfor _, additionalItem := range additionalItemIdentifiers {\n\t\t\tvar itemList []itemblock.ItemBlockItem\n\n\t\t\t// get item content from itemBlock if it's there to avoid the additional APIServer call\n\t\t\t// We could have multiple versions to back up if EnableAPIGroupVersions is set\n\t\t\tif itemBlock != nil {\n\t\t\t\titemList = itemBlock.FindItem(additionalItem.GroupResource, additionalItem.Namespace, additionalItem.Name)\n\t\t\t}\n\t\t\t// if item is not in itemblock, pull it from the cluster\n\t\t\tif len(itemList) == 0 {\n\t\t\t\tlog.Infof(\"Additional Item %s %s/%s not found in ItemBlock, getting from cluster\", additionalItem.GroupResource, additionalItem.Namespace, additionalItem.Name)\n\n\t\t\t\tgvr, resource, err := ib.discoveryHelper.ResourceFor(additionalItem.GroupResource.WithVersion(\"\"))\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn nil, itemFiles, err\n\t\t\t\t}\n\n\t\t\t\tclient, err := ib.dynamicFactory.ClientForGroupVersionResource(gvr.GroupVersion(), resource, additionalItem.Namespace)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn nil, itemFiles, err\n\t\t\t\t}\n\n\t\t\t\titem, err := client.Get(additionalItem.Name, metav1.GetOptions{})\n\n\t\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\t\tlog.WithFields(logrus.Fields{\n\t\t\t\t\t\t\"groupResource\": additionalItem.GroupResource,\n\t\t\t\t\t\t\"namespace\": additionalItem.Namespace,\n\t\t\t\t\t\t\"name\": additionalItem.Name,\n\t\t\t\t\t}).Warnf(\"Additional item was not found in Kubernetes API, can't back it up\")\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn nil, itemFiles, errors.WithStack(err)\n\t\t\t\t}\n\t\t\t\titemList = append(itemList, itemblock.ItemBlockItem{\n\t\t\t\t\tGr: additionalItem.GroupResource,\n\t\t\t\t\tItem: item,\n\t\t\t\t\tPreferredGVR: gvr,\n\t\t\t\t})\n\t\t\t}\n\n\t\t\tfor _, item := range itemList {\n\t\t\t\t_, additionalItemFiles, err := ib.backupItem(log, item.Item, additionalItem.GroupResource, item.PreferredGVR, mustInclude, finalize, itemBlock)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn nil, itemFiles, err\n\t\t\t\t}\n\t\t\t\titemFiles = append(itemFiles, additionalItemFiles...)\n\t\t\t}\n\t\t}\n\t}\n\treturn obj, itemFiles, nil\n}\n\n// zoneLabelDeprecated is the label that stores availability-zone info\n// on PVs this is deprecated on Kubernetes >= 1.17.0\n// zoneLabel is the label that stores availability-zone info\n// on PVs\nconst (\n\tzoneLabelDeprecated = \"failure-domain.beta.kubernetes.io/zone\"\n\t// this is reused for nodeAffinity requirements\n\tzoneLabel = \"topology.kubernetes.io/zone\"\n\n\tawsEbsCsiZoneKey = \"topology.ebs.csi.aws.com/zone\"\n\tazureCsiZoneKey = \"topology.disk.csi.azure.com/zone\"\n\tgkeCsiZoneKey = \"topology.gke.io/zone\"\n\tgkeZoneSeparator = \"__\"\n\n\t// OpenStack CSI drivers topology keys\n\tcinderCsiZoneKey = \"topology.manila.csi.openstack.org/zone\"\n\tmanilaCsiZoneKey = \"topology.cinder.csi.openstack.org/zone\"\n)\n\n// takePVSnapshot triggers a snapshot for the volume/disk underlying a PersistentVolume if the provided\n// backup has volume snapshots enabled and the PV is of a compatible type. Also records cloud\n// disk type and IOPS (if applicable) to be able to restore to current state later.\nfunc (ib *itemBackupper) takePVSnapshot(obj runtime.Unstructured, log logrus.FieldLogger) error {\n\tlog.Info(\"Executing takePVSnapshot\")\n\n\tpv := new(corev1api.PersistentVolume)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(obj.UnstructuredContent(), pv); err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\tlog = log.WithField(\"persistentVolume\", pv.Name)\n\n\tsnapshotVolume, err := ib.volumeHelperImpl.ShouldPerformSnapshot(obj, kuberesource.PersistentVolumes)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif !snapshotVolume {\n\t\tib.trackSkippedPV(\n\t\t\tobj,\n\t\t\tkuberesource.PersistentVolumes,\n\t\t\tvolumeSnapshotApproach,\n\t\t\t\"not satisfy the criteria for VolumePolicy or the legacy snapshot way\",\n\t\t\tlog,\n\t\t)\n\t\treturn nil\n\t}\n\n\t// #4758 Do not take snapshot for CSI PV to avoid duplicated snapshotting, when CSI feature is enabled.\n\tif features.IsEnabled(velerov1api.CSIFeatureFlag) && pv.Spec.CSI != nil {\n\t\tlog.Infof(\"Skipping snapshot of persistent volume %s, because it's handled by CSI plugin.\", pv.Name)\n\t\treturn nil\n\t}\n\n\t// TODO: Snapshot data mover is only supported for CSI plugin scenario by now.\n\t// Need to add a mechanism to choose running which plugin for resources.\n\t// After that, this warning can be removed.\n\tif boolptr.IsSetToTrue(ib.backupRequest.Spec.SnapshotMoveData) {\n\t\tlog.Warnf(\"VolumeSnapshotter plugin doesn't support data movement.\")\n\n\t\tif features.IsEnabled(velerov1api.CSIFeatureFlag) && pv.Spec.CSI == nil {\n\t\t\tlog.Warn(\"Cannot use CSI data mover to handle PV, because PV doesn't contain CSI in spec.\",\n\t\t\t\t\" Fall back to Velero native snapshot.\")\n\t\t}\n\t}\n\n\tif ib.backupRequest.ResPolicies != nil {\n\t\tpvc := new(corev1api.PersistentVolumeClaim)\n\t\tif pv.Spec.ClaimRef != nil {\n\t\t\terr = ib.kbClient.Get(context.Background(), kbClient.ObjectKey{\n\t\t\t\tNamespace: pv.Spec.ClaimRef.Namespace,\n\t\t\t\tName: pv.Spec.ClaimRef.Name},\n\t\t\t\tpvc)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t\tvfd := resourcepolicies.NewVolumeFilterData(pv, nil, pvc)\n\t\tif action, err := ib.backupRequest.ResPolicies.GetMatchAction(vfd); err != nil {\n\t\t\tlog.WithError(err).Errorf(\"Error getting matched resource policies for pv %s\", pv.Name)\n\t\t\treturn nil\n\t\t} else if action != nil && action.Type == resourcepolicies.Skip {\n\t\t\tlog.Infof(\"skip snapshot of pv %s for the matched resource policies\", pv.Name)\n\t\t\t// at this point we are sure this object is PV therefore we'll call the tracker directly\n\t\t\tib.backupRequest.SkippedPVTracker.Track(pv.Name, volumeSnapshotApproach, \"matched action is 'skip' in chosen resource policies\")\n\t\t\treturn nil\n\t\t}\n\t}\n\n\t// TODO: -- once failure-domain.beta.kubernetes.io/zone is no longer\n\t// supported in any velero-supported version of Kubernetes, remove fallback checking of it\n\tpvFailureDomainZone, labelFound := pv.Labels[zoneLabel]\n\tif !labelFound {\n\t\tlog.Infof(\"label %q is not present on PersistentVolume, checking deprecated label...\", zoneLabel)\n\t\tpvFailureDomainZone, labelFound = pv.Labels[zoneLabelDeprecated]\n\t\tif !labelFound {\n\t\t\tvar k string\n\t\t\tlog.Infof(\"label %q is not present on PersistentVolume\", zoneLabelDeprecated)\n\t\t\tk, pvFailureDomainZone = zoneFromPVNodeAffinity(pv, awsEbsCsiZoneKey, azureCsiZoneKey, gkeCsiZoneKey, cinderCsiZoneKey, manilaCsiZoneKey, zoneLabel, zoneLabelDeprecated)\n\t\t\tif pvFailureDomainZone != \"\" {\n\t\t\t\tlog.Infof(\"zone info from nodeAffinity requirements: %s, key: %s\", pvFailureDomainZone, k)\n\t\t\t} else {\n\t\t\t\tlog.Infof(\"zone info not available in nodeAffinity requirements\")\n\t\t\t}\n\t\t}\n\t}\n\n\tvar (\n\t\tvolumeID, location string\n\t\tvolumeSnapshotter vsv1.VolumeSnapshotter\n\t)\n\n\tfor _, snapshotLocation := range ib.backupRequest.SnapshotLocations {\n\t\tlog := log.WithField(\"volumeSnapshotLocation\", snapshotLocation.Name)\n\n\t\tbs, err := ib.volumeSnapshotterCache.SetNX(snapshotLocation)\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Error(\"Error getting volume snapshotter for volume snapshot location\")\n\t\t\tcontinue\n\t\t}\n\n\t\tif volumeID, err = bs.GetVolumeID(obj); err != nil {\n\t\t\tlog.WithError(err).Errorf(\"Error attempting to get volume ID for persistent volume\")\n\t\t\tcontinue\n\t\t}\n\t\tif volumeID == \"\" {\n\t\t\tlog.Warn(\"No volume ID returned by volume snapshotter for persistent volume\")\n\t\t\tcontinue\n\t\t}\n\n\t\tlog.Infof(\"Got volume ID for persistent volume\")\n\t\tvolumeSnapshotter = bs\n\t\tlocation = snapshotLocation.Name\n\t\tbreak\n\t}\n\n\tif volumeSnapshotter == nil {\n\t\t// the PV may still has change to be snapshotted by CSI plugin's `PVCBackupItemAction` in PVC backup logic\n\t\tlog.Info(\"Persistent volume is not a supported volume type for Velero-native volumeSnapshotter snapshot, skipping.\")\n\t\tib.backupRequest.SkippedPVTracker.Track(pv.Name, volumeSnapshotApproach, \"no applicable volumesnapshotter found\")\n\t\treturn nil\n\t}\n\n\tlog = log.WithField(\"volumeID\", volumeID)\n\n\t// create tags from the backup's labels\n\ttags := map[string]string{}\n\tfor k, v := range ib.backupRequest.GetLabels() {\n\t\ttags[k] = v\n\t}\n\ttags[\"velero.io/backup\"] = ib.backupRequest.Name\n\ttags[\"velero.io/pv\"] = pv.Name\n\n\tlog.Info(\"Getting volume information\")\n\tvolumeType, iops, err := volumeSnapshotter.GetVolumeInfo(volumeID, pvFailureDomainZone)\n\tif err != nil {\n\t\treturn errors.WithMessage(err, \"error getting volume info\")\n\t}\n\n\tlog.Info(\"Snapshotting persistent volume\")\n\tsnapshot := volumeSnapshot(ib.backupRequest.Backup, pv.Name, volumeID, volumeType, pvFailureDomainZone, location, iops)\n\n\tvar errs []error\n\tlog.Info(\"Untrack the PV %s from the skipped volumes, because it's backed by Velero native snapshot.\", pv.Name)\n\tib.backupRequest.SkippedPVTracker.Untrack(pv.Name)\n\tsnapshotID, err := volumeSnapshotter.CreateSnapshot(snapshot.Spec.ProviderVolumeID, snapshot.Spec.VolumeAZ, tags)\n\tif err != nil {\n\t\terrs = append(errs, errors.Wrap(err, \"error taking snapshot of volume\"))\n\t\tsnapshot.Status.Phase = volume.SnapshotPhaseFailed\n\t} else {\n\t\tsnapshot.Status.Phase = volume.SnapshotPhaseCompleted\n\t\tsnapshot.Status.ProviderSnapshotID = snapshotID\n\t}\n\tib.backupRequest.VolumeSnapshots.Add(snapshot)\n\n\t// nil errors are automatically removed\n\treturn kubeerrs.NewAggregate(errs)\n}\n\nfunc (ib *itemBackupper) getMatchAction(obj runtime.Unstructured, groupResource schema.GroupResource, backupItemActionName string) (*resourcepolicies.Action, error) {\n\tif ib.backupRequest.ResPolicies != nil && groupResource == kuberesource.PersistentVolumeClaims && (backupItemActionName == csiBIAPluginName || backupItemActionName == vsphereBIAPluginName) {\n\t\tpvc := &corev1api.PersistentVolumeClaim{}\n\t\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(obj.UnstructuredContent(), pvc); err != nil {\n\t\t\treturn nil, errors.WithStack(err)\n\t\t}\n\n\t\tvar pv *corev1api.PersistentVolume\n\t\tif pvName := pvc.Spec.VolumeName; pvName != \"\" {\n\t\t\tpv = &corev1api.PersistentVolume{}\n\t\t\tif err := ib.kbClient.Get(context.Background(), kbClient.ObjectKey{Name: pvName}, pv); err != nil {\n\t\t\t\treturn nil, errors.WithStack(err)\n\t\t\t}\n\t\t}\n\t\t// If pv is nil for unbound PVCs - policy matching will use PVC-only conditions\n\t\tvfd := resourcepolicies.NewVolumeFilterData(pv, nil, pvc)\n\t\treturn ib.backupRequest.ResPolicies.GetMatchAction(vfd)\n\t}\n\n\treturn nil, nil\n}\n\n// trackSkippedPV tracks the skipped PV based on the object and the given approach and reason\n// this function will be called throughout the process of backup, it needs to handle any object\nfunc (ib *itemBackupper) trackSkippedPV(obj runtime.Unstructured, groupResource schema.GroupResource, approach string, reason string, log logrus.FieldLogger) {\n\tif name, err := getPVName(obj, groupResource); len(name) > 0 && err == nil {\n\t\tib.backupRequest.SkippedPVTracker.Track(name, approach, reason)\n\t} else if err != nil {\n\t\t// Log at info level for tracking purposes. This is not an error because\n\t\t// it's expected for some resources (e.g., PVCs in Pending or Lost phase)\n\t\t// to not have a PV name. This occurs when volume policy skips unbound PVCs.\n\t\tlog.WithError(err).Infof(\"unable to get PV name, skip tracking.\")\n\t}\n}\n\n// unTrackSkippedPV removes skipped PV based on the object from the tracker\n// this function will be called throughout the process of backup, it needs to handle any object\nfunc (ib *itemBackupper) unTrackSkippedPV(obj runtime.Unstructured, groupResource schema.GroupResource, log logrus.FieldLogger) {\n\tif name, err := getPVName(obj, groupResource); len(name) > 0 && err == nil {\n\t\tib.backupRequest.SkippedPVTracker.Untrack(name)\n\t} else if err != nil {\n\t\t// For PVCs in Pending or Lost phase, it's expected that there's no PV name.\n\t\t// Log at debug level instead of warning to reduce noise.\n\t\tif groupResource == kuberesource.PersistentVolumeClaims {\n\t\t\tpvc := new(corev1api.PersistentVolumeClaim)\n\t\t\tif convErr := runtime.DefaultUnstructuredConverter.FromUnstructured(obj.UnstructuredContent(), pvc); convErr == nil {\n\t\t\t\tif pvc.Status.Phase == corev1api.ClaimPending || pvc.Status.Phase == corev1api.ClaimLost {\n\t\t\t\t\tlog.WithError(err).Debugf(\"unable to get PV name for %s PVC, skip untracking.\", pvc.Status.Phase)\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\tlog.WithError(err).Warnf(\"unable to get PV name, skip untracking.\")\n\t}\n}\n\nfunc (ib *itemBackupper) addVolumeInfo(obj runtime.Unstructured, log logrus.FieldLogger) error {\n\tpv := new(corev1api.PersistentVolume)\n\terr := runtime.DefaultUnstructuredConverter.FromUnstructured(obj.UnstructuredContent(), pv)\n\tif err != nil {\n\t\tlog.WithError(err).Warnf(\"Fail to convert PV\")\n\t\treturn err\n\t}\n\n\tpvcName := \"\"\n\tpvcNamespace := \"\"\n\tif pv.Spec.ClaimRef != nil {\n\t\tpvcName = pv.Spec.ClaimRef.Name\n\t\tpvcNamespace = pv.Spec.ClaimRef.Namespace\n\t}\n\n\tib.backupRequest.VolumesInformation.InsertPVMap(*pv, pvcName, pvcNamespace)\n\n\treturn nil\n}\n\n// convert the input object to PV/PVC and get the PV name\nfunc getPVName(obj runtime.Unstructured, groupResource schema.GroupResource) (string, error) {\n\tif groupResource == kuberesource.PersistentVolumes {\n\t\tpv := new(corev1api.PersistentVolume)\n\t\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(obj.UnstructuredContent(), pv); err != nil {\n\t\t\treturn \"\", fmt.Errorf(\"failed to convert object to PV: %w\", err)\n\t\t}\n\t\treturn pv.Name, nil\n\t}\n\tif groupResource == kuberesource.PersistentVolumeClaims {\n\t\tpvc := new(corev1api.PersistentVolumeClaim)\n\t\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(obj.UnstructuredContent(), pvc); err != nil {\n\t\t\treturn \"\", fmt.Errorf(\"failed to convert object to PVC: %w\", err)\n\t\t}\n\t\tif pvc.Spec.VolumeName == \"\" {\n\t\t\treturn \"\", fmt.Errorf(\"PV name is not set in PVC\")\n\t\t}\n\t\treturn pvc.Spec.VolumeName, nil\n\t}\n\treturn \"\", nil\n}\n\nfunc volumeSnapshot(backup *velerov1api.Backup, volumeName, volumeID, volumeType, az, location string, iops *int64) *volume.Snapshot {\n\treturn &volume.Snapshot{\n\t\tSpec: volume.SnapshotSpec{\n\t\t\tBackupName: backup.Name,\n\t\t\tBackupUID: string(backup.UID),\n\t\t\tLocation: location,\n\t\t\tPersistentVolumeName: volumeName,\n\t\t\tProviderVolumeID: volumeID,\n\t\t\tVolumeType: volumeType,\n\t\t\tVolumeAZ: az,\n\t\t\tVolumeIOPS: iops,\n\t\t},\n\t\tStatus: volume.SnapshotStatus{\n\t\t\tPhase: volume.SnapshotPhaseNew,\n\t\t},\n\t}\n}\n\n// resourceKey returns a string representing the object's GroupVersionKind (e.g.\n// apps/v1/Deployment).\nfunc resourceKey(obj runtime.Unstructured) string {\n\tgvk := obj.GetObjectKind().GroupVersionKind()\n\treturn fmt.Sprintf(\"%s/%s\", gvk.GroupVersion().String(), gvk.Kind)\n}\n\n// resourceVersion returns a string representing the object's API Version (e.g.\n// v1 if item belongs to apps/v1\nfunc resourceVersion(obj runtime.Unstructured) string {\n\tgvk := obj.GetObjectKind().GroupVersionKind()\n\treturn gvk.Version\n}\n\n// zoneFromPVNodeAffinity iterates the node affinity requirement of a PV to\n// get its availability zone, it returns the key merely for logging.\nfunc zoneFromPVNodeAffinity(res *corev1api.PersistentVolume, topologyKeys ...string) (string, string) {\n\tnodeAffinity := res.Spec.NodeAffinity\n\tif nodeAffinity == nil {\n\t\treturn \"\", \"\"\n\t}\n\tkeySet := sets.NewString(topologyKeys...)\n\tproviderGke := false\n\tzones := make([]string, 0)\n\tfor _, term := range nodeAffinity.Required.NodeSelectorTerms {\n\t\tif term.MatchExpressions == nil {\n\t\t\tcontinue\n\t\t}\n\t\tfor _, exp := range term.MatchExpressions {\n\t\t\tif keySet.Has(exp.Key) && exp.Operator == \"In\" && len(exp.Values) > 0 {\n\t\t\t\tif exp.Key == gkeCsiZoneKey {\n\t\t\t\t\tproviderGke = true\n\t\t\t\t\tzones = append(zones, exp.Values[0])\n\t\t\t\t} else {\n\t\t\t\t\treturn exp.Key, exp.Values[0]\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\tif providerGke {\n\t\treturn gkeCsiZoneKey, strings.Join(zones, gkeZoneSeparator)\n\t}\n\n\treturn \"\", \"\"\n}\n" }, { "path": "pkg/backup/item_backupper_test.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"bytes\"\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\tctrlfake \"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\t\"github.com/vmware-tanzu/velero/internal/resourcepolicies\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\n\t\"github.com/stretchr/testify/assert\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n)\n\nfunc Test_resourceKey(t *testing.T) {\n\ttests := []struct {\n\t\tresource metav1.Object\n\t\twant string\n\t}{\n\t\t{resource: builder.ForPod(\"default\", \"test\").Result(), want: \"v1/Pod\"},\n\t\t{resource: builder.ForDeployment(\"default\", \"test\").Result(), want: \"apps/v1/Deployment\"},\n\t\t{resource: builder.ForPersistentVolume(\"test\").Result(), want: \"v1/PersistentVolume\"},\n\t\t{resource: builder.ForRole(\"default\", \"test\").Result(), want: \"rbac.authorization.k8s.io/v1/Role\"},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.want, func(t *testing.T) {\n\t\t\tcontent, _ := runtime.DefaultUnstructuredConverter.ToUnstructured(tt.resource)\n\t\t\tunstructured := &unstructured.Unstructured{Object: content}\n\t\t\tassert.Equal(t, tt.want, resourceKey(unstructured))\n\t\t})\n\t}\n}\n\nfunc Test_zoneFromPVNodeAffinity(t *testing.T) {\n\tkeys := []string{\n\t\tawsEbsCsiZoneKey,\n\t\tazureCsiZoneKey,\n\t\tgkeCsiZoneKey,\n\t\tzoneLabel,\n\t\tzoneLabelDeprecated,\n\t}\n\ttests := []struct {\n\t\tname string\n\t\tpv *corev1api.PersistentVolume\n\t\twantKey string\n\t\twantValue string\n\t}{\n\t\t{\n\t\t\tname: \"AWS CSI Volume\",\n\t\t\tpv: builder.ForPersistentVolume(\"awscsi\").NodeAffinityRequired(\n\t\t\t\tbuilder.ForNodeSelector(\n\t\t\t\t\t*builder.NewNodeSelectorTermBuilder().WithMatchExpression(\"topology.ebs.csi.aws.com/zone\",\n\t\t\t\t\t\t\"In\", \"us-east-2c\").Result(),\n\t\t\t\t).Result(),\n\t\t\t).Result(),\n\t\t\twantKey: \"topology.ebs.csi.aws.com/zone\",\n\t\t\twantValue: \"us-east-2c\",\n\t\t},\n\t\t{\n\t\t\tname: \"Azure CSI Volume\",\n\t\t\tpv: builder.ForPersistentVolume(\"azurecsi\").NodeAffinityRequired(\n\t\t\t\tbuilder.ForNodeSelector(\n\t\t\t\t\t*builder.NewNodeSelectorTermBuilder().WithMatchExpression(\"topology.disk.csi.azure.com/zone\",\n\t\t\t\t\t\t\"In\", \"us-central\").Result(),\n\t\t\t\t).Result(),\n\t\t\t).Result(),\n\t\t\twantKey: \"topology.disk.csi.azure.com/zone\",\n\t\t\twantValue: \"us-central\",\n\t\t},\n\t\t{\n\t\t\tname: \"GCP CSI Volume\",\n\t\t\tpv: builder.ForPersistentVolume(\"gcpcsi\").NodeAffinityRequired(\n\t\t\t\tbuilder.ForNodeSelector(\n\t\t\t\t\t*builder.NewNodeSelectorTermBuilder().WithMatchExpression(\"topology.gke.io/zone\",\n\t\t\t\t\t\t\"In\", \"us-west1-a\").Result(),\n\t\t\t\t).Result(),\n\t\t\t).Result(),\n\t\t\twantKey: \"topology.gke.io/zone\",\n\t\t\twantValue: \"us-west1-a\",\n\t\t},\n\t\t{\n\t\t\tname: \"AWS CSI Volume with multiple zone value, returns the first\",\n\t\t\tpv: builder.ForPersistentVolume(\"awscsi\").NodeAffinityRequired(\n\t\t\t\tbuilder.ForNodeSelector(\n\t\t\t\t\t*builder.NewNodeSelectorTermBuilder().WithMatchExpression(\"topology.ebs.csi.aws.com/zone\",\n\t\t\t\t\t\t\"In\", \"us-east-2c\", \"us-west\").Result(),\n\t\t\t\t).Result(),\n\t\t\t).Result(),\n\t\t\twantKey: \"topology.ebs.csi.aws.com/zone\",\n\t\t\twantValue: \"us-east-2c\",\n\t\t},\n\t\t{\n\t\t\tname: \"Volume with no matching key\",\n\t\t\tpv: builder.ForPersistentVolume(\"no-matching-pv\").NodeAffinityRequired(\n\t\t\t\tbuilder.ForNodeSelector(\n\t\t\t\t\t*builder.NewNodeSelectorTermBuilder().WithMatchExpression(\"some-key\",\n\t\t\t\t\t\t\"In\", \"us-west\").Result(),\n\t\t\t\t).Result(),\n\t\t\t).Result(),\n\t\t\twantKey: \"\",\n\t\t\twantValue: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"Volume with multiple valid keys, returns the first match\", // it should never happen\n\t\t\tpv: builder.ForPersistentVolume(\"multi-matching-pv\").NodeAffinityRequired(\n\t\t\t\tbuilder.ForNodeSelector(\n\t\t\t\t\t*builder.NewNodeSelectorTermBuilder().WithMatchExpression(\"topology.disk.csi.azure.com/zone\",\n\t\t\t\t\t\t\"In\", \"us-central\").Result(),\n\t\t\t\t\t*builder.NewNodeSelectorTermBuilder().WithMatchExpression(\"topology.ebs.csi.aws.com/zone\",\n\t\t\t\t\t\t\"In\", \"us-east-2c\", \"us-west\").Result(),\n\t\t\t\t\t*builder.NewNodeSelectorTermBuilder().WithMatchExpression(\"topology.ebs.csi.aws.com/zone\",\n\t\t\t\t\t\t\"In\", \"unknown\").Result(),\n\t\t\t\t).Result(),\n\t\t\t).Result(),\n\t\t\twantKey: \"topology.disk.csi.azure.com/zone\",\n\t\t\twantValue: \"us-central\",\n\t\t},\n\t\t{\n\t\t\t/* an valid example of node affinity in a GKE's regional PV\n\t\t\tnodeAffinity:\n\t\t\t required:\n\t\t\t nodeSelectorTerms:\n\t\t\t - matchExpressions:\n\t\t\t - key: topology.gke.io/zone\n\t\t\t operator: In\n\t\t\t values:\n\t\t\t - us-central1-a\n\t\t\t - matchExpressions:\n\t\t\t - key: topology.gke.io/zone\n\t\t\t operator: In\n\t\t\t values:\n\t\t\t - us-central1-c\n\t\t\t*/\n\t\t\tname: \"Volume with multiple valid keys, and provider is gke, returns all valid entries's first zone value\",\n\t\t\tpv: builder.ForPersistentVolume(\"multi-matching-pv\").NodeAffinityRequired(\n\t\t\t\tbuilder.ForNodeSelector(\n\t\t\t\t\t*builder.NewNodeSelectorTermBuilder().WithMatchExpression(\"topology.gke.io/zone\",\n\t\t\t\t\t\t\"In\", \"us-central1-c\").Result(),\n\t\t\t\t\t*builder.NewNodeSelectorTermBuilder().WithMatchExpression(\"topology.gke.io/zone\",\n\t\t\t\t\t\t\"In\", \"us-east-2c\", \"us-east-2b\").Result(),\n\t\t\t\t\t*builder.NewNodeSelectorTermBuilder().WithMatchExpression(\"topology.gke.io/zone\",\n\t\t\t\t\t\t\"In\", \"europe-north1-a\").Result(),\n\t\t\t\t).Result(),\n\t\t\t).Result(),\n\t\t\twantKey: \"topology.gke.io/zone\",\n\t\t\twantValue: \"us-central1-c__us-east-2c__europe-north1-a\",\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tk, v := zoneFromPVNodeAffinity(tt.pv, keys...)\n\t\t\tassert.Equal(t, tt.wantKey, k)\n\t\t\tassert.Equal(t, tt.wantValue, v)\n\t\t})\n\t}\n}\n\nfunc TestGetPVName(t *testing.T) {\n\ttestcases := []struct {\n\t\tname string\n\t\tobj metav1.Object\n\t\tgroupResource schema.GroupResource\n\t\tpvName string\n\t\thasErr bool\n\t}{\n\t\t{\n\t\t\tname: \"pv should return pv name\",\n\t\t\tobj: builder.ForPersistentVolume(\"test-pv\").Result(),\n\t\t\tgroupResource: kuberesource.PersistentVolumes,\n\t\t\tpvName: \"test-pv\",\n\t\t\thasErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"pvc without volumeName should return error\",\n\t\t\tobj: builder.ForPersistentVolumeClaim(\"ns\", \"pvc-1\").Result(),\n\t\t\tgroupResource: kuberesource.PersistentVolumeClaims,\n\t\t\tpvName: \"\",\n\t\t\thasErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"pvc with volumeName should return pv name\",\n\t\t\tobj: builder.ForPersistentVolumeClaim(\"ns\", \"pvc-1\").VolumeName(\"test-pv-2\").Result(),\n\t\t\tgroupResource: kuberesource.PersistentVolumeClaims,\n\t\t\tpvName: \"test-pv-2\",\n\t\t\thasErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"unsupported group resource should return empty pv name\",\n\t\t\tobj: builder.ForPod(\"ns\", \"pod1\").Result(),\n\t\t\tgroupResource: kuberesource.Pods,\n\t\t\tpvName: \"\",\n\t\t\thasErr: false,\n\t\t},\n\t}\n\tfor _, tc := range testcases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\to := &unstructured.Unstructured{Object: nil}\n\t\t\tif tc.obj != nil {\n\t\t\t\tdata, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.obj)\n\t\t\t\to = &unstructured.Unstructured{Object: data}\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\t\t\tname, err2 := getPVName(o, tc.groupResource)\n\t\t\tassert.Equal(t, tc.pvName, name)\n\t\t\tassert.Equal(t, tc.hasErr, err2 != nil)\n\t\t})\n\t}\n}\n\nfunc TestRandom(t *testing.T) {\n\tpv := new(corev1api.PersistentVolume)\n\tpvc := new(corev1api.PersistentVolumeClaim)\n\tobj := builder.ForPod(\"ns1\", \"pod1\").ServiceAccount(\"sa\").Result()\n\to, _ := runtime.DefaultUnstructuredConverter.ToUnstructured(obj)\n\terr1 := runtime.DefaultUnstructuredConverter.FromUnstructured(o, pv)\n\terr2 := runtime.DefaultUnstructuredConverter.FromUnstructured(o, pvc)\n\tt.Logf(\"err1: %v, err2: %v\", err1, err2)\n}\n\nfunc TestAddVolumeInfo(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpv *corev1api.PersistentVolume\n\t}{\n\t\t{\n\t\t\tname: \"PV has ClaimRef\",\n\t\t\tpv: builder.ForPersistentVolume(\"testPV\").ClaimRef(\"testNS\", \"testPVC\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"PV has no ClaimRef\",\n\t\t\tpv: builder.ForPersistentVolume(\"testPV\").Result(),\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tib := itemBackupper{}\n\t\t\tib.backupRequest = new(Request)\n\t\t\tib.backupRequest.VolumesInformation.Init()\n\n\t\t\tpvObj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.pv)\n\t\t\trequire.NoError(t, err)\n\t\t\tlogger := logrus.StandardLogger()\n\n\t\t\terr = ib.addVolumeInfo(&unstructured.Unstructured{Object: pvObj}, logger)\n\t\t\trequire.NoError(t, err)\n\t\t})\n\t}\n}\n\nfunc TestGetMatchAction_PendingLostPVC(t *testing.T) {\n\tscheme := runtime.NewScheme()\n\trequire.NoError(t, corev1api.AddToScheme(scheme))\n\n\t// Create resource policies that skip Pending/Lost PVCs\n\tresPolicies := &resourcepolicies.ResourcePolicies{\n\t\tVersion: \"v1\",\n\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t{\n\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\"pvcPhase\": []string{\"Pending\", \"Lost\"},\n\t\t\t\t},\n\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\tType: resourcepolicies.Skip,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tpolicies := &resourcepolicies.Policies{}\n\terr := policies.BuildPolicy(resPolicies)\n\trequire.NoError(t, err)\n\n\ttestCases := []struct {\n\t\tname string\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\tpv *corev1api.PersistentVolume\n\t\texpectedAction *resourcepolicies.Action\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"Pending PVC with no VolumeName should match pvcPhase policy\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns\", \"pending-pvc\").\n\t\t\t\tStorageClass(\"test-sc\").\n\t\t\t\tPhase(corev1api.ClaimPending).\n\t\t\t\tResult(),\n\t\t\tpv: nil,\n\t\t\texpectedAction: &resourcepolicies.Action{Type: resourcepolicies.Skip},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Lost PVC with no VolumeName should match pvcPhase policy\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns\", \"lost-pvc\").\n\t\t\t\tStorageClass(\"test-sc\").\n\t\t\t\tPhase(corev1api.ClaimLost).\n\t\t\t\tResult(),\n\t\t\tpv: nil,\n\t\t\texpectedAction: &resourcepolicies.Action{Type: resourcepolicies.Skip},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Bound PVC with VolumeName and matching PV should not match pvcPhase policy\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns\", \"bound-pvc\").\n\t\t\t\tStorageClass(\"test-sc\").\n\t\t\t\tVolumeName(\"test-pv\").\n\t\t\t\tPhase(corev1api.ClaimBound).\n\t\t\t\tResult(),\n\t\t\tpv: builder.ForPersistentVolume(\"test-pv\").StorageClass(\"test-sc\").Result(),\n\t\t\texpectedAction: nil,\n\t\t\texpectError: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t// Build fake client with PV if present\n\t\t\tclientBuilder := ctrlfake.NewClientBuilder().WithScheme(scheme)\n\t\t\tif tc.pv != nil {\n\t\t\t\tclientBuilder = clientBuilder.WithObjects(tc.pv)\n\t\t\t}\n\t\t\tfakeClient := clientBuilder.Build()\n\n\t\t\tib := &itemBackupper{\n\t\t\t\tkbClient: fakeClient,\n\t\t\t\tbackupRequest: &Request{\n\t\t\t\t\tResPolicies: policies,\n\t\t\t\t},\n\t\t\t}\n\n\t\t\t// Convert PVC to unstructured\n\t\t\tpvcData, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.pvc)\n\t\t\trequire.NoError(t, err)\n\t\t\tobj := &unstructured.Unstructured{Object: pvcData}\n\n\t\t\taction, err := ib.getMatchAction(obj, kuberesource.PersistentVolumeClaims, csiBIAPluginName)\n\t\t\tif tc.expectError {\n\t\t\t\trequire.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif tc.expectedAction == nil {\n\t\t\t\tassert.Nil(t, action)\n\t\t\t} else {\n\t\t\t\trequire.NotNil(t, action)\n\t\t\t\tassert.Equal(t, tc.expectedAction.Type, action.Type)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestTrackSkippedPV_PendingLostPVC(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t}{\n\t\t{\n\t\t\tname: \"Pending PVC should log at info level\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns\", \"pending-pvc\").\n\t\t\t\tPhase(corev1api.ClaimPending).\n\t\t\t\tResult(),\n\t\t},\n\t\t{\n\t\t\tname: \"Lost PVC should log at info level\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns\", \"lost-pvc\").\n\t\t\t\tPhase(corev1api.ClaimLost).\n\t\t\t\tResult(),\n\t\t},\n\t\t{\n\t\t\tname: \"Bound PVC without VolumeName should log at info level\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns\", \"bound-pvc\").\n\t\t\t\tPhase(corev1api.ClaimBound).\n\t\t\t\tResult(),\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tib := &itemBackupper{\n\t\t\t\tbackupRequest: &Request{\n\t\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\t},\n\t\t\t}\n\n\t\t\t// Set up log capture\n\t\t\tlogOutput := &bytes.Buffer{}\n\t\t\tlogger := logrus.New()\n\t\t\tlogger.SetOutput(logOutput)\n\t\t\tlogger.SetLevel(logrus.DebugLevel)\n\n\t\t\t// Convert PVC to unstructured\n\t\t\tpvcData, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.pvc)\n\t\t\trequire.NoError(t, err)\n\t\t\tobj := &unstructured.Unstructured{Object: pvcData}\n\n\t\t\tib.trackSkippedPV(obj, kuberesource.PersistentVolumeClaims, \"\", \"test reason\", logger)\n\n\t\t\tlogStr := logOutput.String()\n\t\t\tassert.Contains(t, logStr, \"level=info\")\n\t\t\tassert.Contains(t, logStr, \"unable to get PV name, skip tracking.\")\n\t\t})\n\t}\n}\n\nfunc TestUnTrackSkippedPV_PendingLostPVC(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\texpectWarningLog bool\n\t\texpectDebugMessage string\n\t}{\n\t\t{\n\t\t\tname: \"Pending PVC should log at debug level, not warning\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns\", \"pending-pvc\").\n\t\t\t\tPhase(corev1api.ClaimPending).\n\t\t\t\tResult(),\n\t\t\texpectWarningLog: false,\n\t\t\texpectDebugMessage: \"unable to get PV name for Pending PVC, skip untracking.\",\n\t\t},\n\t\t{\n\t\t\tname: \"Lost PVC should log at debug level, not warning\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns\", \"lost-pvc\").\n\t\t\t\tPhase(corev1api.ClaimLost).\n\t\t\t\tResult(),\n\t\t\texpectWarningLog: false,\n\t\t\texpectDebugMessage: \"unable to get PV name for Lost PVC, skip untracking.\",\n\t\t},\n\t\t{\n\t\t\tname: \"Bound PVC without VolumeName should log warning\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns\", \"bound-pvc\").\n\t\t\t\tPhase(corev1api.ClaimBound).\n\t\t\t\tResult(),\n\t\t\texpectWarningLog: true,\n\t\t\texpectDebugMessage: \"\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tib := &itemBackupper{\n\t\t\t\tbackupRequest: &Request{\n\t\t\t\t\tSkippedPVTracker: NewSkipPVTracker(),\n\t\t\t\t},\n\t\t\t}\n\n\t\t\t// Set up log capture\n\t\t\tlogOutput := &bytes.Buffer{}\n\t\t\tlogger := logrus.New()\n\t\t\tlogger.SetOutput(logOutput)\n\t\t\tlogger.SetLevel(logrus.DebugLevel)\n\n\t\t\t// Convert PVC to unstructured\n\t\t\tpvcData, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.pvc)\n\t\t\trequire.NoError(t, err)\n\t\t\tobj := &unstructured.Unstructured{Object: pvcData}\n\n\t\t\tib.unTrackSkippedPV(obj, kuberesource.PersistentVolumeClaims, logger)\n\n\t\t\tlogStr := logOutput.String()\n\t\t\tif tc.expectWarningLog {\n\t\t\t\tassert.Contains(t, logStr, \"level=warning\")\n\t\t\t\tassert.Contains(t, logStr, \"unable to get PV name, skip untracking.\")\n\t\t\t} else {\n\t\t\t\tassert.NotContains(t, logStr, \"level=warning\")\n\t\t\t\tif tc.expectDebugMessage != \"\" {\n\t\t\t\t\tassert.Contains(t, logStr, \"level=debug\")\n\t\t\t\t\tassert.Contains(t, logStr, tc.expectDebugMessage)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/backup/item_block_worker_pool.go", "content": "/*\nCopyright the Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"context\"\n\t\"sync\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n)\n\ntype ItemBlockWorkerPool struct {\n\tinputChannel chan ItemBlockInput\n\twg *sync.WaitGroup\n\tlogger logrus.FieldLogger\n\tcancelFunc context.CancelFunc\n}\n\ntype ItemBlockInput struct {\n\titemBlock *BackupItemBlock\n\treturnChan chan ItemBlockReturn\n}\n\ntype ItemBlockReturn struct {\n\titemBlock *BackupItemBlock\n\tresources []schema.GroupResource\n\terr error\n}\n\nfunc (p *ItemBlockWorkerPool) GetInputChannel() chan ItemBlockInput {\n\treturn p.inputChannel\n}\n\nfunc StartItemBlockWorkerPool(ctx context.Context, workers int, log logrus.FieldLogger) *ItemBlockWorkerPool {\n\t// Buffer will hold up to 10 ItemBlocks waiting for processing\n\tinputChannel := make(chan ItemBlockInput, max(workers, 10))\n\n\tctx, cancelFunc := context.WithCancel(ctx)\n\twg := &sync.WaitGroup{}\n\n\tfor i := 0; i < workers; i++ {\n\t\tlogger := log.WithField(\"worker\", i)\n\t\twg.Add(1)\n\t\tgo processItemBlockWorker(ctx, inputChannel, logger, wg)\n\t}\n\n\tpool := &ItemBlockWorkerPool{\n\t\tinputChannel: inputChannel,\n\t\tcancelFunc: cancelFunc,\n\t\tlogger: log,\n\t\twg: wg,\n\t}\n\treturn pool\n}\n\nfunc (p *ItemBlockWorkerPool) Stop() {\n\tp.cancelFunc()\n\tp.logger.Info(\"ItemBlock worker stopping\")\n\tp.wg.Wait()\n\tp.logger.Info(\"ItemBlock worker stopped\")\n}\n\nfunc processItemBlockWorker(ctx context.Context,\n\tinputChannel chan ItemBlockInput,\n\tlogger logrus.FieldLogger,\n\twg *sync.WaitGroup) {\n\tfor {\n\t\tselect {\n\t\tcase m := <-inputChannel:\n\t\t\tlogger.Infof(\"processing ItemBlock for backup %v\", m.itemBlock.itemBackupper.backupRequest.Name)\n\t\t\tgrList := m.itemBlock.itemBackupper.kubernetesBackupper.backupItemBlock(m.itemBlock)\n\t\t\tlogger.Infof(\"finished processing ItemBlock for backup %v\", m.itemBlock.itemBackupper.backupRequest.Name)\n\t\t\tm.returnChan <- ItemBlockReturn{\n\t\t\t\titemBlock: m.itemBlock,\n\t\t\t\tresources: grList,\n\t\t\t\terr: nil,\n\t\t\t}\n\t\tcase <-ctx.Done():\n\t\t\tlogger.Info(\"stopping ItemBlock worker\")\n\t\t\twg.Done()\n\t\t\treturn\n\t\t}\n\t}\n}\n" }, { "path": "pkg/backup/item_collector.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"os\"\n\t\"sort\"\n\t\"strings\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/api/meta\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/client-go/tools/pager\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/discovery\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/collections\"\n)\n\n// itemCollector collects items from the Kubernetes API according to\n// the backup spec and writes them to files inside dir.\ntype itemCollector struct {\n\tlog logrus.FieldLogger\n\tbackupRequest *Request\n\tdiscoveryHelper discovery.Helper\n\tdynamicFactory client.DynamicFactory\n\tcohabitatingResources map[string]*cohabitatingResource\n\tdir string\n\tpageSize int\n\tnsTracker nsTracker\n}\n\n// nsTracker is used to integrate several namespace filters together.\n// 1. Backup's namespace Include/Exclude filters;\n// 2. Backup's (Or)LabelSelector selected namespace;\n// 3. Backup's (Or)LabelSelector selected resources' namespaces.\n//\n// Rules:\n//\n//\ta. When backup namespace Include/Exclude filters get everything,\n//\tThe namespaces, which do not have backup including resources,\n//\tare not collected.\n//\n//\tb. If the namespace I/E filters and the (Or)LabelSelectors selected\n//\tnamespaces are different. The tracker takes the union of them.\ntype nsTracker struct {\n\tsingleLabelSelector labels.Selector\n\torLabelSelector []labels.Selector\n\tnamespaceFilter *collections.NamespaceIncludesExcludes\n\tlogger logrus.FieldLogger\n\n\tnamespaceMap map[string]bool\n}\n\n// track add the namespace into the namespaceMap.\nfunc (nt *nsTracker) track(ns string) {\n\tif nt.namespaceMap == nil {\n\t\tnt.namespaceMap = make(map[string]bool)\n\t}\n\n\tif _, ok := nt.namespaceMap[ns]; !ok {\n\t\tnt.namespaceMap[ns] = true\n\t}\n}\n\n// isTracked check whether the namespace's name exists in\n// namespaceMap.\nfunc (nt *nsTracker) isTracked(ns string) bool {\n\tif nt.namespaceMap != nil {\n\t\treturn nt.namespaceMap[ns]\n\t}\n\treturn false\n}\n\n// init initialize the namespaceMap, and add elements according to\n// namespace include/exclude filters and the backup label selectors.\nfunc (nt *nsTracker) init(\n\tunstructuredNSs []unstructured.Unstructured,\n\tsingleLabelSelector labels.Selector,\n\torLabelSelector []labels.Selector,\n\tnamespaceFilter *collections.NamespaceIncludesExcludes,\n\tlogger logrus.FieldLogger,\n) {\n\tif nt.namespaceMap == nil {\n\t\tnt.namespaceMap = make(map[string]bool)\n\t}\n\tnt.singleLabelSelector = singleLabelSelector\n\tnt.orLabelSelector = orLabelSelector\n\tnt.namespaceFilter = namespaceFilter\n\tnt.logger = logger\n\n\tfor _, namespace := range unstructuredNSs {\n\t\tns := new(corev1api.Namespace)\n\t\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(namespace.UnstructuredContent(), ns); err != nil {\n\t\t\tnt.logger.WithError(err).Errorf(\"Fail to convert unstructured into namespace %s\", namespace.GetName())\n\t\t\tcontinue\n\t\t}\n\t\tif ns.Status.Phase != corev1api.NamespaceActive {\n\t\t\tnt.logger.Infof(\"Skip namespace %s because it's not in Active phase.\", namespace.GetName())\n\t\t\tcontinue\n\t\t}\n\n\t\tif nt.singleLabelSelector != nil &&\n\t\t\tnt.singleLabelSelector.Matches(labels.Set(namespace.GetLabels())) {\n\t\t\tnt.logger.Debugf(\"Track namespace %s, because its labels match backup LabelSelector.\",\n\t\t\t\tnamespace.GetName(),\n\t\t\t)\n\n\t\t\tnt.track(namespace.GetName())\n\t\t\tcontinue\n\t\t}\n\n\t\tif len(nt.orLabelSelector) > 0 {\n\t\t\tfor _, selector := range nt.orLabelSelector {\n\t\t\t\tif selector.Matches(labels.Set(namespace.GetLabels())) {\n\t\t\t\t\tnt.logger.Debugf(\"Track namespace %s, because its labels match the backup OrLabelSelector.\",\n\t\t\t\t\t\tnamespace.GetName(),\n\t\t\t\t\t)\n\t\t\t\t\tnt.track(namespace.GetName())\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\t// Skip the backup when the backup's namespace filter has\n\t\t// default value, and the namespace doesn't match backup\n\t\t// LabelSelector and OrLabelSelector.\n\t\t// https://github.com/vmware-tanzu/velero/issues/7105\n\t\tif nt.namespaceFilter.IncludeEverything() &&\n\t\t\t(nt.singleLabelSelector != nil || len(nt.orLabelSelector) > 0) {\n\t\t\tcontinue\n\t\t}\n\n\t\tif nt.namespaceFilter.ShouldInclude(namespace.GetName()) {\n\t\t\tnt.logger.Debugf(\"Track namespace %s, because its name match the backup namespace filter.\",\n\t\t\t\tnamespace.GetName(),\n\t\t\t)\n\t\t\tnt.track(namespace.GetName())\n\t\t}\n\t}\n}\n\n// filterNamespaces filters the input resource list to remove the\n// namespaces not tracked by the nsTracker.\nfunc (nt *nsTracker) filterNamespaces(\n\tresources []*kubernetesResource,\n) []*kubernetesResource {\n\tresult := make([]*kubernetesResource, 0)\n\n\tfor _, resource := range resources {\n\t\tif resource.groupResource != kuberesource.Namespaces ||\n\t\t\tnt.isTracked(resource.name) {\n\t\t\tresult = append(result, resource)\n\t\t}\n\t}\n\n\treturn result\n}\n\ntype kubernetesResource struct {\n\tgroupResource schema.GroupResource\n\tpreferredGVR schema.GroupVersionResource\n\tnamespace, name, path string\n\torderedResource bool\n\t// set to true during backup processing when added to an ItemBlock\n\t// or if the item is excluded from backup.\n\tinItemBlockOrExcluded bool\n\t// Kind is added to facilitate creating an itemKey for progress tracking\n\tkind string\n}\n\n// getItemsFromResourceIdentifiers get the kubernetesResources\n// specified by the input parameter resourceIDs.\nfunc (r *itemCollector) getItemsFromResourceIdentifiers(\n\tresourceIDs []velero.ResourceIdentifier,\n) []*kubernetesResource {\n\tgrResourceIDsMap := make(map[schema.GroupResource][]velero.ResourceIdentifier)\n\tfor _, resourceID := range resourceIDs {\n\t\tgrResourceIDsMap[resourceID.GroupResource] = append(\n\t\t\tgrResourceIDsMap[resourceID.GroupResource], resourceID)\n\t}\n\treturn r.getItems(grResourceIDsMap)\n}\n\n// getAllItems gets all backup-relevant items from all API groups.\nfunc (r *itemCollector) getAllItems() []*kubernetesResource {\n\tresources := r.getItems(nil)\n\n\treturn r.nsTracker.filterNamespaces(resources)\n}\n\n// getItems gets all backup-relevant items from all API groups,\n//\n// If resourceIDsMap is nil, then all items from the cluster are\n// pulled for each API group, subject to include/exclude rules,\n// except the namespace, because the namespace filtering depends on\n// all namespaced-scoped resources.\n//\n// If resourceIDsMap is supplied, then only those resources are\n// returned, with the appropriate APIGroup information filled in. In\n// this case, include/exclude rules are not invoked, since we already\n// have the list of items, we just need the item collector/discovery\n// helper to fill in the missing GVR, etc. context.\nfunc (r *itemCollector) getItems(\n\tresourceIDsMap map[schema.GroupResource][]velero.ResourceIdentifier,\n) []*kubernetesResource {\n\tvar resources []*kubernetesResource\n\tfor _, group := range r.discoveryHelper.Resources() {\n\t\tgroupItems, err := r.getGroupItems(r.log, group, resourceIDsMap)\n\t\tif err != nil {\n\t\t\tr.log.WithError(err).WithField(\"apiGroup\", group.String()).\n\t\t\t\tError(\"Error collecting resources from API group\")\n\t\t\tcontinue\n\t\t}\n\n\t\tresources = append(resources, groupItems...)\n\t}\n\n\treturn resources\n}\n\n// getGroupItems collects all relevant items from a single API group.\n// If resourceIDsMap is supplied, then only those items are returned,\n// with GVR/APIResource metadata supplied.\nfunc (r *itemCollector) getGroupItems(\n\tlog logrus.FieldLogger,\n\tgroup *metav1.APIResourceList,\n\tresourceIDsMap map[schema.GroupResource][]velero.ResourceIdentifier,\n) ([]*kubernetesResource, error) {\n\tlog = log.WithField(\"group\", group.GroupVersion)\n\n\tlog.Infof(\"Getting items for group\")\n\n\t// Parse so we can check if this is the core group\n\tgv, err := schema.ParseGroupVersion(group.GroupVersion)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error parsing GroupVersion %q\",\n\t\t\tgroup.GroupVersion)\n\t}\n\tif gv.Group == \"\" {\n\t\t// This is the core group, so make sure we process in the following order:\n\t\t// pods, pvcs, pvs, everything else.\n\t\tsortCoreGroup(group)\n\t}\n\n\tvar items []*kubernetesResource\n\tfor _, resource := range group.APIResources {\n\t\tresourceItems, err := r.getResourceItems(log, gv, resource, resourceIDsMap)\n\t\tif err != nil {\n\t\t\tlog.WithError(err).WithField(\"resource\", resource.String()).\n\t\t\t\tError(\"Error getting items for resource\")\n\t\t\tcontinue\n\t\t}\n\n\t\titems = append(items, resourceItems...)\n\t}\n\n\treturn items, nil\n}\n\n// sortResourcesByOrder sorts items by the names specified in \"order\".\n// Items are not in order will be put at the end in original order.\nfunc sortResourcesByOrder(\n\tlog logrus.FieldLogger,\n\titems []*kubernetesResource,\n\torder []string,\n) []*kubernetesResource {\n\tif len(order) == 0 {\n\t\treturn items\n\t}\n\tlog.Debugf(\"Sorting resources using the following order %v...\", order)\n\titemMap := make(map[string]*kubernetesResource)\n\tfor _, item := range items {\n\t\tvar fullname string\n\t\tif item.namespace != \"\" {\n\t\t\tfullname = fmt.Sprintf(\"%s/%s\", item.namespace, item.name)\n\t\t} else {\n\t\t\tfullname = item.name\n\t\t}\n\t\titemMap[fullname] = item\n\t}\n\tvar sortedItems []*kubernetesResource\n\t// First select items from the order\n\tfor _, name := range order {\n\t\tif item, ok := itemMap[name]; ok {\n\t\t\titem.orderedResource = true\n\t\t\tsortedItems = append(sortedItems, item)\n\t\t\tlog.Debugf(\"%s added to sorted resource list.\", item.name)\n\t\t\tdelete(itemMap, name)\n\t\t} else {\n\t\t\tlog.Warnf(\"Cannot find resource '%s'.\", name)\n\t\t}\n\t}\n\t// Now append the rest in sortedGroupItems, maintain the original order\n\tfor _, item := range items {\n\t\tvar fullname string\n\t\tif item.namespace != \"\" {\n\t\t\tfullname = fmt.Sprintf(\"%s/%s\", item.namespace, item.name)\n\t\t} else {\n\t\t\tfullname = item.name\n\t\t}\n\t\tif _, ok := itemMap[fullname]; !ok {\n\t\t\t//This item has been inserted in the result\n\t\t\tcontinue\n\t\t}\n\t\tsortedItems = append(sortedItems, item)\n\t\tlog.Debugf(\"%s added to sorted resource list.\", item.name)\n\t}\n\treturn sortedItems\n}\n\n// getOrderedResourcesForType gets order of resourceType from orderResources.\nfunc getOrderedResourcesForType(\n\torderedResources map[string]string,\n\tresourceType string,\n) []string {\n\tif orderedResources == nil {\n\t\treturn nil\n\t}\n\torderStr, ok := orderedResources[resourceType]\n\tif !ok || len(orderStr) == 0 {\n\t\treturn nil\n\t}\n\torders := strings.Split(orderStr, \",\")\n\treturn orders\n}\n\n// getResourceItems collects all relevant items for a given group-version-resource.\n// If resourceIDsMap is supplied, the items will be pulled from here\n// rather than from the cluster and applying include/exclude rules.\nfunc (r *itemCollector) getResourceItems(\n\tlog logrus.FieldLogger,\n\tgv schema.GroupVersion,\n\tresource metav1.APIResource,\n\tresourceIDsMap map[schema.GroupResource][]velero.ResourceIdentifier,\n) ([]*kubernetesResource, error) {\n\tlog = log.WithField(\"resource\", resource.Name)\n\n\tlog.Info(\"Getting items for resource\")\n\n\tvar (\n\t\tgvr = gv.WithResource(resource.Name)\n\t\tgr = gvr.GroupResource()\n\t)\n\n\torders := getOrderedResourcesForType(\n\t\tr.backupRequest.Backup.Spec.OrderedResources,\n\t\tresource.Name,\n\t)\n\t// Getting the preferred group version of this resource\n\tpreferredGVR, _, err := r.discoveryHelper.ResourceFor(gr.WithVersion(\"\"))\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\t// If we have a resourceIDs map, then only return items listed in it\n\tif resourceIDsMap != nil {\n\t\tresourceIDs, ok := resourceIDsMap[gr]\n\t\tif !ok {\n\t\t\tlog.Info(\"Skipping resource because no items found in supplied ResourceIdentifier list\")\n\t\t\treturn nil, nil\n\t\t}\n\t\tvar items []*kubernetesResource\n\t\tfor _, resourceID := range resourceIDs {\n\t\t\tlog.WithFields(\n\t\t\t\tlogrus.Fields{\n\t\t\t\t\t\"namespace\": resourceID.Namespace,\n\t\t\t\t\t\"name\": resourceID.Name,\n\t\t\t\t},\n\t\t\t).Infof(\"Getting item\")\n\t\t\tresourceClient, err := r.dynamicFactory.ClientForGroupVersionResource(\n\t\t\t\tgv,\n\t\t\t\tresource,\n\t\t\t\tresourceID.Namespace,\n\t\t\t)\n\t\t\tif err != nil {\n\t\t\t\tlog.WithError(errors.WithStack(err)).Error(\"Error getting client for resource\")\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tunstructured, err := resourceClient.Get(resourceID.Name, metav1.GetOptions{})\n\t\t\tif err != nil {\n\t\t\t\tlog.WithError(errors.WithStack(err)).Error(\"Error getting item\")\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tpath, err := r.writeToFile(unstructured)\n\t\t\tif err != nil {\n\t\t\t\tlog.WithError(err).Error(\"Error writing item to file\")\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\titems = append(items, &kubernetesResource{\n\t\t\t\tgroupResource: gr,\n\t\t\t\tpreferredGVR: preferredGVR,\n\t\t\t\tnamespace: resourceID.Namespace,\n\t\t\t\tname: resourceID.Name,\n\t\t\t\tpath: path,\n\t\t\t\tkind: resource.Kind,\n\t\t\t})\n\t\t}\n\n\t\treturn items, nil\n\t}\n\n\tif !r.backupRequest.ResourceIncludesExcludes.ShouldInclude(gr.String()) {\n\t\tlog.Infof(\"Skipping resource because it's excluded\")\n\t\treturn nil, nil\n\t}\n\n\tif cohabitator, found := r.cohabitatingResources[resource.Name]; found {\n\t\tif gv.Group == cohabitator.groupResource1.Group ||\n\t\t\tgv.Group == cohabitator.groupResource2.Group {\n\t\t\tif cohabitator.seen {\n\t\t\t\tlog.WithFields(\n\t\t\t\t\tlogrus.Fields{\n\t\t\t\t\t\t\"cohabitatingResource1\": cohabitator.groupResource1.String(),\n\t\t\t\t\t\t\"cohabitatingResource2\": cohabitator.groupResource2.String(),\n\t\t\t\t\t},\n\t\t\t\t).Infof(\"Skipping resource because it cohabitates and we've already processed it\")\n\t\t\t\treturn nil, nil\n\t\t\t}\n\t\t\tcohabitator.seen = true\n\t\t}\n\t}\n\n\t// Handle namespace resource here.\n\t// Namespace are filtered by namespace include/exclude filters,\n\t// backup LabelSelectors and OrLabelSelectors are checked too.\n\tif gr == kuberesource.Namespaces {\n\t\treturn r.collectNamespaces(\n\t\t\tresource,\n\t\t\tgv,\n\t\t\tgr,\n\t\t\tpreferredGVR,\n\t\t\tlog,\n\t\t)\n\t}\n\n\tclusterScoped := !resource.Namespaced\n\tnamespacesToList := getNamespacesToList(r.backupRequest.NamespaceIncludesExcludes)\n\n\t// If we get here, we're backing up something other than namespaces\n\tif clusterScoped {\n\t\tnamespacesToList = []string{\"\"}\n\t}\n\n\tvar items []*kubernetesResource\n\n\tfor _, namespace := range namespacesToList {\n\t\tunstructuredItems, err := r.listResourceByLabelsPerNamespace(\n\t\t\tnamespace, gr, gv, resource, log)\n\t\tif err != nil {\n\t\t\tcontinue\n\t\t}\n\n\t\t// Collect items in included Namespaces\n\t\tfor i := range unstructuredItems {\n\t\t\titem := &unstructuredItems[i]\n\n\t\t\tpath, err := r.writeToFile(item)\n\t\t\tif err != nil {\n\t\t\t\tlog.WithError(err).Error(\"Error writing item to file\")\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\titems = append(items, &kubernetesResource{\n\t\t\t\tgroupResource: gr,\n\t\t\t\tpreferredGVR: preferredGVR,\n\t\t\t\tnamespace: item.GetNamespace(),\n\t\t\t\tname: item.GetName(),\n\t\t\t\tpath: path,\n\t\t\t\tkind: resource.Kind,\n\t\t\t})\n\n\t\t\tif item.GetNamespace() != \"\" {\n\t\t\t\tlog.Debugf(\"Track namespace %s in nsTracker\", item.GetNamespace())\n\t\t\t\tr.nsTracker.track(item.GetNamespace())\n\t\t\t}\n\t\t}\n\t}\n\n\tif len(orders) > 0 {\n\t\titems = sortResourcesByOrder(r.log, items, orders)\n\t}\n\n\treturn items, nil\n}\n\nfunc (r *itemCollector) listResourceByLabelsPerNamespace(\n\tnamespace string,\n\tgr schema.GroupResource,\n\tgv schema.GroupVersion,\n\tresource metav1.APIResource,\n\tlogger logrus.FieldLogger,\n) ([]unstructured.Unstructured, error) {\n\t// List items from Kubernetes API\n\tlogger = logger.WithField(\"namespace\", namespace)\n\n\tresourceClient, err := r.dynamicFactory.ClientForGroupVersionResource(gv, resource, namespace)\n\tif err != nil {\n\t\tlogger.WithError(err).Error(\"Error getting dynamic client\")\n\t\treturn nil, err\n\t}\n\n\tvar orLabelSelectors []string\n\tif r.backupRequest.Spec.OrLabelSelectors != nil {\n\t\tfor _, s := range r.backupRequest.Spec.OrLabelSelectors {\n\t\t\torLabelSelectors = append(orLabelSelectors, metav1.FormatLabelSelector(s))\n\t\t}\n\t} else {\n\t\torLabelSelectors = []string{}\n\t}\n\n\tlogger.Info(\"Listing items\")\n\tunstructuredItems := make([]unstructured.Unstructured, 0)\n\n\t// Listing items for orLabelSelectors\n\terrListingForNS := false\n\tfor _, label := range orLabelSelectors {\n\t\tunstructuredItems, err = r.listItemsForLabel(unstructuredItems, gr, label, resourceClient)\n\t\tif err != nil {\n\t\t\terrListingForNS = true\n\t\t}\n\t}\n\n\tif errListingForNS {\n\t\tlogger.WithError(err).Error(\"Error listing items\")\n\t\treturn nil, err\n\t}\n\n\tvar labelSelector string\n\tif selector := r.backupRequest.Spec.LabelSelector; selector != nil {\n\t\tlabelSelector = metav1.FormatLabelSelector(selector)\n\t}\n\n\t// Listing items for labelSelector (singular)\n\tif len(orLabelSelectors) == 0 {\n\t\tunstructuredItems, err = r.listItemsForLabel(\n\t\t\tunstructuredItems,\n\t\t\tgr,\n\t\t\tlabelSelector,\n\t\t\tresourceClient,\n\t\t)\n\t\tif err != nil {\n\t\t\tlogger.WithError(err).Error(\"Error listing items\")\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tlogger.Infof(\"Retrieved %d items\", len(unstructuredItems))\n\treturn unstructuredItems, nil\n}\n\nfunc (r *itemCollector) writeToFile(item *unstructured.Unstructured) (string, error) {\n\tf, err := os.CreateTemp(r.dir, \"\")\n\tif err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error creating temp file\")\n\t}\n\tdefer f.Close()\n\n\tjsonBytes, err := json.Marshal(item)\n\tif err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error converting item to JSON\")\n\t}\n\n\tif _, err := f.Write(jsonBytes); err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error writing JSON to file\")\n\t}\n\n\tif err := f.Close(); err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error closing file\")\n\t}\n\n\treturn f.Name(), nil\n}\n\n// sortCoreGroup sorts the core API group.\nfunc sortCoreGroup(group *metav1.APIResourceList) {\n\tsort.SliceStable(group.APIResources, func(i, j int) bool {\n\t\treturn coreGroupResourcePriority(group.APIResources[i].Name) < coreGroupResourcePriority(group.APIResources[j].Name)\n\t})\n}\n\n// These constants represent the relative priorities for resources in the core API group. We want to\n// ensure that we process pods, then pvcs, then pvs, then anything else. This ensures that when a\n// pod is backed up, we can perform a pre hook, then process pvcs and pvs (including taking a\n// snapshot), then perform a post hook on the pod.\nconst (\n\tpod = iota\n\tpvc\n\tpv\n\tother\n)\n\n// coreGroupResourcePriority returns the relative priority of the resource, in the following order:\n// pods, pvcs, pvs, everything else.\nfunc coreGroupResourcePriority(resource string) int {\n\tswitch strings.ToLower(resource) {\n\tcase \"pods\":\n\t\treturn pod\n\tcase \"persistentvolumeclaims\":\n\t\treturn pvc\n\tcase \"persistentvolumes\":\n\t\treturn pv\n\t}\n\n\treturn other\n}\n\n// getNamespacesToList examines ie and resolves the includes and excludes to a full list of\n// namespaces to list. If ie is nil or it includes *, the result is just \"\" (list across all\n// namespaces). Otherwise, the result is a list of every included namespace minus all excluded ones.\nfunc getNamespacesToList(ie *collections.NamespaceIncludesExcludes) []string {\n\tif ie == nil {\n\t\treturn []string{\"\"}\n\t}\n\n\tif ie.ShouldInclude(\"*\") {\n\t\t// \"\" means all namespaces\n\t\treturn []string{\"\"}\n\t}\n\n\tvar list []string\n\tfor _, i := range ie.GetIncludes() {\n\t\tif ie.ShouldInclude(i) {\n\t\t\tlist = append(list, i)\n\t\t}\n\t}\n\n\treturn list\n}\n\ntype cohabitatingResource struct {\n\tresource string\n\tgroupResource1 schema.GroupResource\n\tgroupResource2 schema.GroupResource\n\tseen bool\n}\n\nfunc newCohabitatingResource(resource, group1, group2 string) *cohabitatingResource {\n\treturn &cohabitatingResource{\n\t\tresource: resource,\n\t\tgroupResource1: schema.GroupResource{Group: group1, Resource: resource},\n\t\tgroupResource2: schema.GroupResource{Group: group2, Resource: resource},\n\t\tseen: false,\n\t}\n}\n\n// function to process pager client calls when the pageSize is specified\nfunc (r *itemCollector) processPagerClientCalls(\n\tgr schema.GroupResource,\n\tlabel string,\n\tresourceClient client.Dynamic,\n) (runtime.Object, error) {\n\t// If limit is positive, use a pager to split list over multiple requests\n\t// Use Velero's dynamic list function instead of the default\n\tlistPager := pager.New(pager.SimplePageFunc(func(opts metav1.ListOptions) (runtime.Object, error) {\n\t\treturn resourceClient.List(opts)\n\t}))\n\t// Use the page size defined in the server config\n\t// TODO allow configuration of page buffer size\n\tlistPager.PageSize = int64(r.pageSize)\n\t// Add each item to temporary slice\n\tlist, paginated, err := listPager.List(context.Background(), metav1.ListOptions{LabelSelector: label})\n\n\tif err != nil {\n\t\tr.log.WithError(errors.WithStack(err)).Error(\"Error listing resources\")\n\t\treturn list, err\n\t}\n\n\tif !paginated {\n\t\tr.log.Infof(\"list for groupResource %s was not paginated\", gr)\n\t}\n\n\treturn list, nil\n}\n\nfunc (r *itemCollector) listItemsForLabel(\n\tunstructuredItems []unstructured.Unstructured,\n\tgr schema.GroupResource,\n\tlabel string,\n\tresourceClient client.Dynamic,\n) ([]unstructured.Unstructured, error) {\n\tif r.pageSize > 0 {\n\t\t// process pager client calls\n\t\tlist, err := r.processPagerClientCalls(gr, label, resourceClient)\n\t\tif err != nil {\n\t\t\treturn unstructuredItems, err\n\t\t}\n\n\t\terr = meta.EachListItem(list, func(object runtime.Object) error {\n\t\t\tu, ok := object.(*unstructured.Unstructured)\n\t\t\tif !ok {\n\t\t\t\tr.log.WithError(errors.WithStack(fmt.Errorf(\"expected *unstructured.Unstructured but got %T\", u))).\n\t\t\t\t\tError(\"unable to understand entry in the list\")\n\t\t\t\treturn fmt.Errorf(\"expected *unstructured.Unstructured but got %T\", u)\n\t\t\t}\n\t\t\tunstructuredItems = append(unstructuredItems, *u)\n\t\t\treturn nil\n\t\t})\n\t\tif err != nil {\n\t\t\tr.log.WithError(errors.WithStack(err)).Error(\"unable to understand paginated list\")\n\t\t\treturn unstructuredItems, err\n\t\t}\n\t} else {\n\t\tunstructuredList, err := resourceClient.List(metav1.ListOptions{LabelSelector: label})\n\t\tif err != nil {\n\t\t\tr.log.WithError(errors.WithStack(err)).Error(\"Error listing items\")\n\t\t\treturn unstructuredItems, err\n\t\t}\n\t\tunstructuredItems = append(unstructuredItems, unstructuredList.Items...)\n\t}\n\treturn unstructuredItems, nil\n}\n\n// collectNamespaces process namespace resource according to namespace filters.\nfunc (r *itemCollector) collectNamespaces(\n\tresource metav1.APIResource,\n\tgv schema.GroupVersion,\n\tgr schema.GroupResource,\n\tpreferredGVR schema.GroupVersionResource,\n\tlog logrus.FieldLogger,\n) ([]*kubernetesResource, error) {\n\tresourceClient, err := r.dynamicFactory.ClientForGroupVersionResource(gv, resource, \"\")\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"Error getting dynamic client\")\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tunstructuredList, err := resourceClient.List(metav1.ListOptions{})\n\n\tactiveNamespacesHashSet := make(map[string]bool)\n\tfor _, namespace := range unstructuredList.Items {\n\t\tactiveNamespacesHashSet[namespace.GetName()] = true\n\t}\n\n\tif err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Error(\"error list namespaces\")\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\t// Change to look at the struct includes/excludes\n\t// In case wildcards are expanded, we need to look at the struct includes/excludes\n\tfor _, includedNSName := range r.backupRequest.NamespaceIncludesExcludes.GetIncludes() {\n\t\tnsExists := false\n\t\t// Skip checking the namespace existing when it's \"*\".\n\t\tif includedNSName == \"*\" {\n\t\t\tcontinue\n\t\t}\n\n\t\tif _, ok := activeNamespacesHashSet[includedNSName]; ok {\n\t\t\tnsExists = true\n\t\t}\n\n\t\tif !nsExists {\n\t\t\tlog.Errorf(\"fail to get the namespace %s specified in backup.Spec.IncludedNamespaces\", includedNSName)\n\t\t}\n\t}\n\n\tvar singleSelector labels.Selector\n\tvar orSelectors []labels.Selector\n\n\tif r.backupRequest.Backup.Spec.LabelSelector != nil {\n\t\tvar err error\n\t\tsingleSelector, err = metav1.LabelSelectorAsSelector(\n\t\t\tr.backupRequest.Backup.Spec.LabelSelector)\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Errorf(\"Fail to convert backup LabelSelector %s into selector.\",\n\t\t\t\tmetav1.FormatLabelSelector(r.backupRequest.Backup.Spec.LabelSelector))\n\t\t}\n\t}\n\tif r.backupRequest.Backup.Spec.OrLabelSelectors != nil {\n\t\tfor _, ls := range r.backupRequest.Backup.Spec.OrLabelSelectors {\n\t\t\torSelector, err := metav1.LabelSelectorAsSelector(ls)\n\t\t\tif err != nil {\n\t\t\t\tlog.WithError(err).Errorf(\"Fail to convert backup OrLabelSelector %s into selector.\",\n\t\t\t\t\tmetav1.FormatLabelSelector(ls))\n\t\t\t}\n\t\t\torSelectors = append(orSelectors, orSelector)\n\t\t}\n\t}\n\n\tr.nsTracker.init(\n\t\tunstructuredList.Items,\n\t\tsingleSelector,\n\t\torSelectors,\n\t\tr.backupRequest.NamespaceIncludesExcludes,\n\t\tlog,\n\t)\n\n\tvar items []*kubernetesResource\n\n\tfor index := range unstructuredList.Items {\n\t\tnsName := unstructuredList.Items[index].GetName()\n\n\t\tpath, err := r.writeToFile(&unstructuredList.Items[index])\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Errorf(\"Error writing item %s to file\", nsName)\n\t\t\tcontinue\n\t\t}\n\n\t\titems = append(items, &kubernetesResource{\n\t\t\tgroupResource: gr,\n\t\t\tpreferredGVR: preferredGVR,\n\t\t\tname: nsName,\n\t\t\tpath: path,\n\t\t\tkind: resource.Kind,\n\t\t})\n\t}\n\n\treturn items, nil\n}\n" }, { "path": "pkg/backup/item_collector_test.go", "content": "/*\nCopyright 2017, 2019, 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/collections\"\n)\n\nfunc TestSortCoreGroup(t *testing.T) {\n\tgroup := &metav1.APIResourceList{\n\t\tGroupVersion: \"v1\",\n\t\tAPIResources: []metav1.APIResource{\n\t\t\t{Name: \"persistentvolumes\"},\n\t\t\t{Name: \"configmaps\"},\n\t\t\t{Name: \"antelopes\"},\n\t\t\t{Name: \"persistentvolumeclaims\"},\n\t\t\t{Name: \"pods\"},\n\t\t},\n\t}\n\n\tsortCoreGroup(group)\n\n\texpected := []string{\n\t\t\"pods\",\n\t\t\"persistentvolumeclaims\",\n\t\t\"persistentvolumes\",\n\t\t\"configmaps\",\n\t\t\"antelopes\",\n\t}\n\tfor i, r := range group.APIResources {\n\t\tassert.Equal(t, expected[i], r.Name)\n\t}\n}\n\nfunc TestSortOrderedResource(t *testing.T) {\n\tlog := logrus.StandardLogger()\n\tpodResources := []*kubernetesResource{\n\t\t{namespace: \"ns1\", name: \"pod3\"},\n\t\t{namespace: \"ns1\", name: \"pod1\"},\n\t\t{namespace: \"ns1\", name: \"pod2\"},\n\t}\n\torder := []string{\"ns1/pod2\", \"ns1/pod1\"}\n\texpectedResources := []*kubernetesResource{\n\t\t{namespace: \"ns1\", name: \"pod2\", orderedResource: true},\n\t\t{namespace: \"ns1\", name: \"pod1\", orderedResource: true},\n\t\t{namespace: \"ns1\", name: \"pod3\"},\n\t}\n\tsortedResources := sortResourcesByOrder(log, podResources, order)\n\tassert.Equal(t, expectedResources, sortedResources)\n\n\t// Test cluster resources\n\tpvResources := []*kubernetesResource{\n\t\t{name: \"pv1\"},\n\t\t{name: \"pv2\"},\n\t\t{name: \"pv3\"},\n\t}\n\tpvOrder := []string{\"pv5\", \"pv2\", \"pv1\"}\n\texpectedPvResources := []*kubernetesResource{\n\t\t{name: \"pv2\", orderedResource: true},\n\t\t{name: \"pv1\", orderedResource: true},\n\t\t{name: \"pv3\"},\n\t}\n\tsortedPvResources := sortResourcesByOrder(log, pvResources, pvOrder)\n\tassert.Equal(t, expectedPvResources, sortedPvResources)\n}\n\nfunc TestFilterNamespaces(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tresources []*kubernetesResource\n\t\tneedToTrack string\n\t\texpectedResources []*kubernetesResource\n\t}{\n\t\t{\n\t\t\tname: \"Namespace include by the filter but not in namespacesContainResource\",\n\t\t\tresources: []*kubernetesResource{\n\t\t\t\t{\n\t\t\t\t\tgroupResource: kuberesource.Namespaces,\n\t\t\t\t\tpreferredGVR: kuberesource.Namespaces.WithVersion(\"v1\"),\n\t\t\t\t\tname: \"ns1\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tgroupResource: kuberesource.Namespaces,\n\t\t\t\t\tpreferredGVR: kuberesource.Namespaces.WithVersion(\"v1\"),\n\t\t\t\t\tname: \"ns2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tgroupResource: kuberesource.Pods,\n\t\t\t\t\tpreferredGVR: kuberesource.Namespaces.WithVersion(\"v1\"),\n\t\t\t\t\tname: \"pod1\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tneedToTrack: \"ns1\",\n\t\t\texpectedResources: []*kubernetesResource{\n\t\t\t\t{\n\t\t\t\t\tgroupResource: kuberesource.Namespaces,\n\t\t\t\t\tpreferredGVR: kuberesource.Namespaces.WithVersion(\"v1\"),\n\t\t\t\t\tname: \"ns1\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tgroupResource: kuberesource.Pods,\n\t\t\t\t\tpreferredGVR: kuberesource.Namespaces.WithVersion(\"v1\"),\n\t\t\t\t\tname: \"pod1\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(*testing.T) {\n\t\t\tr := itemCollector{\n\t\t\t\tbackupRequest: &Request{},\n\t\t\t}\n\n\t\t\tif tc.needToTrack != \"\" {\n\t\t\t\tr.nsTracker.track(tc.needToTrack)\n\t\t\t}\n\n\t\t\trequire.Equal(t, tc.expectedResources, r.nsTracker.filterNamespaces(tc.resources))\n\t\t})\n\t}\n}\n\nfunc TestItemCollectorBackupNamespaces(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tie *collections.NamespaceIncludesExcludes\n\t\tnamespaces []*corev1api.Namespace\n\t\tbackup *velerov1api.Backup\n\t\texpectedTrackedNS []string\n\t\tconverter runtime.UnstructuredConverter\n\t}{\n\t\t{\n\t\t\tname: \"ns filter by namespace IE filter\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").Result(),\n\t\t\tie: collections.NewNamespaceIncludesExcludes().Includes(\"ns1\"),\n\t\t\tnamespaces: []*corev1api.Namespace{\n\t\t\t\tbuilder.ForNamespace(\"ns1\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\tbuilder.ForNamespace(\"ns2\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t},\n\t\t\texpectedTrackedNS: []string{\"ns1\"},\n\t\t},\n\t\t{\n\t\t\tname: \"ns filter by backup labelSelector\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").LabelSelector(&metav1.LabelSelector{\n\t\t\t\tMatchLabels: map[string]string{\"name\": \"ns1\"},\n\t\t\t}).Result(),\n\t\t\tie: collections.NewNamespaceIncludesExcludes().Includes(\"*\"),\n\t\t\tnamespaces: []*corev1api.Namespace{\n\t\t\t\tbuilder.ForNamespace(\"ns1\").ObjectMeta(builder.WithLabels(\"name\", \"ns1\")).Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\tbuilder.ForNamespace(\"ns2\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t},\n\t\t\texpectedTrackedNS: []string{\"ns1\"},\n\t\t},\n\t\t{\n\t\t\tname: \"ns filter by backup orLabelSelector\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").OrLabelSelector([]*metav1.LabelSelector{\n\t\t\t\t{MatchLabels: map[string]string{\"name\": \"ns1\"}},\n\t\t\t}).Result(),\n\t\t\tie: collections.NewNamespaceIncludesExcludes().Includes(\"*\"),\n\t\t\tnamespaces: []*corev1api.Namespace{\n\t\t\t\tbuilder.ForNamespace(\"ns1\").ObjectMeta(builder.WithLabels(\"name\", \"ns1\")).Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\tbuilder.ForNamespace(\"ns2\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t},\n\t\t\texpectedTrackedNS: []string{\"ns1\"},\n\t\t},\n\t\t{\n\t\t\tname: \"ns not included by IE filter, but included by labelSelector\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").LabelSelector(&metav1.LabelSelector{\n\t\t\t\tMatchLabels: map[string]string{\"name\": \"ns1\"},\n\t\t\t}).Result(),\n\t\t\tie: collections.NewNamespaceIncludesExcludes().Excludes(\"ns1\"),\n\t\t\tnamespaces: []*corev1api.Namespace{\n\t\t\t\tbuilder.ForNamespace(\"ns1\").ObjectMeta(builder.WithLabels(\"name\", \"ns1\")).Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\tbuilder.ForNamespace(\"ns2\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t},\n\t\t\texpectedTrackedNS: []string{\"ns1\"},\n\t\t},\n\t\t{\n\t\t\tname: \"ns not included by IE filter, but included by orLabelSelector\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").OrLabelSelector([]*metav1.LabelSelector{\n\t\t\t\t{MatchLabels: map[string]string{\"name\": \"ns1\"}},\n\t\t\t}).Result(),\n\t\t\tie: collections.NewNamespaceIncludesExcludes().Excludes(\"ns1\", \"ns2\"),\n\t\t\tnamespaces: []*corev1api.Namespace{\n\t\t\t\tbuilder.ForNamespace(\"ns1\").ObjectMeta(builder.WithLabels(\"name\", \"ns1\")).Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\tbuilder.ForNamespace(\"ns2\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\tbuilder.ForNamespace(\"ns3\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t},\n\t\t\texpectedTrackedNS: []string{\"ns1\", \"ns3\"},\n\t\t},\n\t\t{\n\t\t\tname: \"No ns filters\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").Result(),\n\t\t\tie: collections.NewNamespaceIncludesExcludes().Includes(\"*\"),\n\t\t\tnamespaces: []*corev1api.Namespace{\n\t\t\t\tbuilder.ForNamespace(\"ns1\").ObjectMeta(builder.WithLabels(\"name\", \"ns1\")).Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\tbuilder.ForNamespace(\"ns2\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t},\n\t\t\texpectedTrackedNS: []string{\"ns1\", \"ns2\"},\n\t\t},\n\t\t{\n\t\t\tname: \"ns specified by the IncludeNamespaces cannot be found\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").IncludedNamespaces(\"ns1\", \"invalid\", \"*\").Result(),\n\t\t\tie: collections.NewNamespaceIncludesExcludes().Includes(\"ns1\", \"invalid\", \"*\"),\n\t\t\tnamespaces: []*corev1api.Namespace{\n\t\t\t\tbuilder.ForNamespace(\"ns1\").ObjectMeta(builder.WithLabels(\"name\", \"ns1\")).Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\tbuilder.ForNamespace(\"ns2\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t\tbuilder.ForNamespace(\"ns3\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t},\n\t\t\texpectedTrackedNS: []string{\"ns1\"},\n\t\t},\n\t\t{\n\t\t\tname: \"terminating ns should not tracked\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"backup\").Result(),\n\t\t\tie: collections.NewNamespaceIncludesExcludes().Includes(\"ns1\", \"ns2\"),\n\t\t\tnamespaces: []*corev1api.Namespace{\n\t\t\t\tbuilder.ForNamespace(\"ns1\").Phase(corev1api.NamespaceTerminating).Result(),\n\t\t\t\tbuilder.ForNamespace(\"ns2\").Phase(corev1api.NamespaceActive).Result(),\n\t\t\t},\n\t\t\texpectedTrackedNS: []string{\"ns2\"},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(*testing.T) {\n\t\t\ttempDir := t.TempDir()\n\n\t\t\tvar unstructuredNSList unstructured.UnstructuredList\n\t\t\tfor _, ns := range tc.namespaces {\n\t\t\t\tunstructuredNS, err := runtime.DefaultUnstructuredConverter.ToUnstructured(ns)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tunstructuredNSList.Items = append(unstructuredNSList.Items,\n\t\t\t\t\tunstructured.Unstructured{Object: unstructuredNS})\n\t\t\t}\n\n\t\t\tdc := &test.FakeDynamicClient{}\n\t\t\tdc.On(\"List\", mock.Anything).Return(&unstructuredNSList, nil)\n\n\t\t\tfactory := &test.FakeDynamicFactory{}\n\t\t\tfactory.On(\n\t\t\t\t\"ClientForGroupVersionResource\",\n\t\t\t\tmock.Anything,\n\t\t\t\tmock.Anything,\n\t\t\t\tmock.Anything,\n\t\t\t).Return(dc, nil)\n\n\t\t\tr := itemCollector{\n\t\t\t\tbackupRequest: &Request{\n\t\t\t\t\tBackup: tc.backup,\n\t\t\t\t\tNamespaceIncludesExcludes: tc.ie,\n\t\t\t\t},\n\t\t\t\tdynamicFactory: factory,\n\t\t\t\tdir: tempDir,\n\t\t\t}\n\n\t\t\tif tc.converter == nil {\n\t\t\t\ttc.converter = runtime.DefaultUnstructuredConverter\n\t\t\t}\n\n\t\t\tr.collectNamespaces(\n\t\t\t\tmetav1.APIResource{\n\t\t\t\t\tName: \"Namespace\",\n\t\t\t\t\tKind: \"Namespace\",\n\t\t\t\t\tNamespaced: false,\n\t\t\t\t},\n\t\t\t\tkuberesource.Namespaces.WithVersion(\"\").GroupVersion(),\n\t\t\t\tkuberesource.Namespaces,\n\t\t\t\tkuberesource.Namespaces.WithVersion(\"\"),\n\t\t\t\tlogrus.StandardLogger(),\n\t\t\t)\n\n\t\t\tfor _, ns := range tc.expectedTrackedNS {\n\t\t\t\trequire.True(t, r.nsTracker.isTracked(ns))\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/backup/itemblock.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"encoding/json\"\n\t\"os\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/api/meta\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/itemblock\"\n)\n\ntype BackupItemBlock struct {\n\titemblock.ItemBlock\n\t// This is a reference to the shared itemBackupper for the backup\n\titemBackupper *itemBackupper\n}\n\nfunc NewBackupItemBlock(log logrus.FieldLogger, itemBackupper *itemBackupper) *BackupItemBlock {\n\treturn &BackupItemBlock{\n\t\tItemBlock: itemblock.ItemBlock{Log: log},\n\t\titemBackupper: itemBackupper,\n\t}\n}\n\nfunc (b *BackupItemBlock) addKubernetesResource(item *kubernetesResource, log logrus.FieldLogger) *unstructured.Unstructured {\n\t// no-op if item has already been processed (in a block or previously excluded)\n\tif item.inItemBlockOrExcluded {\n\t\treturn nil\n\t}\n\tvar unstructured unstructured.Unstructured\n\titem.inItemBlockOrExcluded = true\n\n\tf, err := os.Open(item.path)\n\tif err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Error(\"Error opening file containing item\")\n\t\treturn nil\n\t}\n\tdefer f.Close()\n\tdefer os.Remove(f.Name())\n\n\tif err := json.NewDecoder(f).Decode(&unstructured); err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Error(\"Error decoding JSON from file\")\n\t\treturn nil\n\t}\n\n\tmetadata, err := meta.Accessor(&unstructured)\n\tif err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Warn(\"Error accessing item metadata\")\n\t\treturn nil\n\t}\n\t// Don't add to ItemBlock if item is excluded\n\t// itemInclusionChecks logs the reason\n\tif !b.itemBackupper.itemInclusionChecks(log, false, metadata, &unstructured, item.groupResource) {\n\t\treturn nil\n\t}\n\n\tlog.Infof(\"adding %s %s/%s to ItemBlock\", item.groupResource, item.namespace, item.name)\n\tb.AddUnstructured(item.groupResource, &unstructured, item.preferredGVR)\n\treturn &unstructured\n}\n" }, { "path": "pkg/backup/pv_skip_tracker.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"sort\"\n\t\"sync\"\n)\n\ntype SkippedPV struct {\n\tName string `json:\"name\"`\n\tReasons []PVSkipReason `json:\"reasons\"`\n}\n\nfunc (s *SkippedPV) SerializeSkipReasons() string {\n\tret := \"\"\n\tfor _, reason := range s.Reasons {\n\t\tret = ret + reason.Approach + \": \" + reason.Reason + \";\"\n\t}\n\treturn ret\n}\n\ntype PVSkipReason struct {\n\tApproach string `json:\"approach\"`\n\tReason string `json:\"reason\"`\n}\n\n// skipPVTracker keeps track of persistent volumes that have been skipped and the reason why they are skipped.\ntype skipPVTracker struct {\n\t*sync.RWMutex\n\t// pvs is a map of name of the pv to the list of reasons why it is skipped.\n\t// The reasons are stored in a map each key of the map is the backup approach, each approach can have one reason\n\tpvs map[string]map[string]string\n\t// includedPVs is a set of pv to be included in the backup, the element in this set should not be in the \"pvs\" map\n\tincludedPVs map[string]struct{}\n}\n\nconst (\n\tpodVolumeApproach = \"podvolume\"\n\tcsiSnapshotApproach = \"csiSnapshot\"\n\tvolumeSnapshotApproach = \"volumeSnapshot\"\n\tvsphereSnapshotApproach = \"vsphereSnapshot\"\n\tanyApproach = \"any\"\n)\n\nfunc NewSkipPVTracker() *skipPVTracker {\n\treturn &skipPVTracker{\n\t\tRWMutex: &sync.RWMutex{},\n\t\tpvs: make(map[string]map[string]string),\n\t\tincludedPVs: make(map[string]struct{}),\n\t}\n}\n\n// Track tracks the pv with the specified name and the reason why it is skipped\nfunc (pt *skipPVTracker) Track(name, approach, reason string) {\n\tpt.Lock()\n\tdefer pt.Unlock()\n\tif name == \"\" || reason == \"\" {\n\t\treturn\n\t}\n\tif _, ok := pt.includedPVs[name]; ok {\n\t\treturn\n\t}\n\tskipReasons := pt.pvs[name]\n\tif skipReasons == nil {\n\t\tskipReasons = make(map[string]string)\n\t\tpt.pvs[name] = skipReasons\n\t}\n\tif approach == \"\" {\n\t\tapproach = anyApproach\n\t}\n\tskipReasons[approach] = reason\n}\n\n// Untrack removes the pvc with the specified namespace and name.\n// This func should be called when the PV is taken for snapshot, regardless native snapshot, CSI snapshot or fsb backup\n// therefore, in one backup processed if a PV is Untracked once, it will not be tracked again.\nfunc (pt *skipPVTracker) Untrack(name string) {\n\tpt.Lock()\n\tdefer pt.Unlock()\n\tpt.includedPVs[name] = struct{}{}\n\tdelete(pt.pvs, name)\n}\n\n// Summary returns the summary of the tracked pvcs.\nfunc (pt *skipPVTracker) Summary() []SkippedPV {\n\tpt.RLock()\n\tdefer pt.RUnlock()\n\tkeys := make([]string, 0, len(pt.pvs))\n\tfor key := range pt.pvs {\n\t\tkeys = append(keys, key)\n\t}\n\tsort.Strings(keys)\n\tres := make([]SkippedPV, 0, len(keys))\n\tfor _, key := range keys {\n\t\tif skipReasons := pt.pvs[key]; len(skipReasons) > 0 {\n\t\t\tentry := SkippedPV{\n\t\t\t\tName: key,\n\t\t\t\tReasons: make([]PVSkipReason, 0, len(skipReasons)),\n\t\t\t}\n\t\t\tapproaches := make([]string, 0, len(skipReasons))\n\t\t\tfor a := range skipReasons {\n\t\t\t\tapproaches = append(approaches, a)\n\t\t\t}\n\t\t\tsort.Strings(approaches)\n\t\t\tfor _, a := range approaches {\n\t\t\t\tentry.Reasons = append(entry.Reasons, PVSkipReason{\n\t\t\t\t\tApproach: a,\n\t\t\t\t\tReason: skipReasons[a],\n\t\t\t\t})\n\t\t\t}\n\t\t\tres = append(res, entry)\n\t\t}\n\t}\n\treturn res\n}\n" }, { "path": "pkg/backup/pv_skip_tracker_test.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestSummary(t *testing.T) {\n\ttracker := NewSkipPVTracker()\n\ttracker.Track(\"pv5\", \"\", \"skipped due to policy\")\n\ttracker.Track(\"pv3\", podVolumeApproach, \"it's set to opt-out\")\n\ttracker.Track(\"pv3\", csiSnapshotApproach, \"not applicable for CSI \")\n\t// shouldn't be added\n\ttracker.Track(\"\", podVolumeApproach, \"pvc3 is set to be skipped\")\n\ttracker.Track(\"pv10\", volumeSnapshotApproach, \"added by mistake\")\n\ttracker.Untrack(\"pv10\")\n\texpected := []SkippedPV{\n\t\t{\n\t\t\tName: \"pv3\",\n\t\t\tReasons: []PVSkipReason{\n\t\t\t\t{\n\t\t\t\t\tApproach: csiSnapshotApproach,\n\t\t\t\t\tReason: \"not applicable for CSI \",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tApproach: podVolumeApproach,\n\t\t\t\t\tReason: \"it's set to opt-out\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tName: \"pv5\",\n\t\t\tReasons: []PVSkipReason{\n\t\t\t\t{\n\t\t\t\t\tApproach: anyApproach,\n\t\t\t\t\tReason: \"skipped due to policy\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tassert.Equal(t, expected, tracker.Summary())\n}\n\nfunc TestSerializeSkipReasons(t *testing.T) {\n\ttracker := NewSkipPVTracker()\n\t//tracker.Track(\"pv5\", \"\", \"skipped due to policy\")\n\ttracker.Track(\"pv3\", podVolumeApproach, \"it's set to opt-out\")\n\ttracker.Track(\"pv3\", csiSnapshotApproach, \"not applicable for CSI \")\n\n\tfor _, skippedPV := range tracker.Summary() {\n\t\trequire.Equal(t, \"csiSnapshot: not applicable for CSI ;podvolume: it's set to opt-out;\", skippedPV.SerializeSkipReasons())\n\t}\n}\n\nfunc TestTrackUntrack(t *testing.T) {\n\t// If a pv is untracked explicitly it can't be Tracked again, b/c the pv is considered backed up already.\n\ttracker := NewSkipPVTracker()\n\ttracker.Track(\"pv3\", podVolumeApproach, \"it's set to opt-out\")\n\ttracker.Untrack(\"pv3\")\n\ttracker.Track(\"pv3\", csiSnapshotApproach, \"not applicable for CSI \")\n\tassert.Empty(t, tracker.Summary())\n}\n" }, { "path": "pkg/backup/request.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"sync\"\n\n\t\"github.com/vmware-tanzu/velero/internal/hook\"\n\t\"github.com/vmware-tanzu/velero/internal/resourcepolicies\"\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/collections\"\n)\n\ntype itemKey struct {\n\tresource string\n\tnamespace string\n\tname string\n}\n\ntype SynchronizedVSList struct {\n\tsync.Mutex\n\tVolumeSnapshotList []*volume.Snapshot\n}\n\nfunc (s *SynchronizedVSList) Add(vs *volume.Snapshot) {\n\ts.Lock()\n\tdefer s.Unlock()\n\ts.VolumeSnapshotList = append(s.VolumeSnapshotList, vs)\n}\n\nfunc (s *SynchronizedVSList) Get() []*volume.Snapshot {\n\ts.Lock()\n\tdefer s.Unlock()\n\treturn s.VolumeSnapshotList\n}\n\n// Request is a request for a backup, with all references to other objects\n// materialized (e.g. backup/snapshot locations, includes/excludes, etc.)\ntype Request struct {\n\t*velerov1api.Backup\n\tStorageLocation *velerov1api.BackupStorageLocation\n\tSnapshotLocations []*velerov1api.VolumeSnapshotLocation\n\tNamespaceIncludesExcludes *collections.NamespaceIncludesExcludes\n\tResourceIncludesExcludes collections.IncludesExcludesInterface\n\tResourceHooks []hook.ResourceHook\n\tResolvedActions []framework.BackupItemResolvedActionV2\n\tResolvedItemBlockActions []framework.ItemBlockResolvedAction\n\tVolumeSnapshots SynchronizedVSList\n\tPodVolumeBackups []*velerov1api.PodVolumeBackup\n\tBackedUpItems *backedUpItemsMap\n\titemOperationsList *[]*itemoperation.BackupOperation\n\tResPolicies *resourcepolicies.Policies\n\tSkippedPVTracker *skipPVTracker\n\tVolumesInformation volume.BackupVolumesInformation\n\tWorkerPool *ItemBlockWorkerPool\n}\n\n// BackupVolumesInformation contains the information needs by generating\n// the backup BackupVolumeInfo array.\n\n// GetItemOperationsList returns ItemOperationsList, initializing it if necessary\nfunc (r *Request) GetItemOperationsList() *[]*itemoperation.BackupOperation {\n\tif r.itemOperationsList == nil {\n\t\tlist := []*itemoperation.BackupOperation{}\n\t\tr.itemOperationsList = &list\n\t}\n\treturn r.itemOperationsList\n}\n\n// BackupResourceList returns the list of backed up resources grouped by the API\n// Version and Kind\nfunc (r *Request) BackupResourceList() map[string][]string {\n\treturn r.BackedUpItems.ResourceMap()\n}\n\nfunc (r *Request) FillVolumesInformation() {\n\tskippedPVMap := make(map[string]string)\n\n\tfor _, skippedPV := range r.SkippedPVTracker.Summary() {\n\t\tskippedPVMap[skippedPV.Name] = skippedPV.SerializeSkipReasons()\n\t}\n\n\tr.VolumesInformation.SkippedPVs = skippedPVMap\n\tr.VolumesInformation.NativeSnapshots = r.VolumeSnapshots.Get()\n\tr.VolumesInformation.PodVolumeBackups = r.PodVolumeBackups\n\tr.VolumesInformation.BackupOperations = *r.GetItemOperationsList()\n\tr.VolumesInformation.BackupName = r.Backup.Name\n}\n\nfunc (r *Request) StopWorkerPool() {\n\tr.WorkerPool.Stop()\n}\n" }, { "path": "pkg/backup/request_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestRequest_BackupResourceList(t *testing.T) {\n\titems := []itemKey{\n\t\t{\n\t\t\tresource: \"apps/v1/Deployment\",\n\t\t\tname: \"my-deploy\",\n\t\t\tnamespace: \"default\",\n\t\t},\n\t\t{\n\t\t\tresource: \"v1/Pod\",\n\t\t\tname: \"pod1\",\n\t\t\tnamespace: \"ns1\",\n\t\t},\n\t\t{\n\t\t\tresource: \"v1/Pod\",\n\t\t\tname: \"pod2\",\n\t\t\tnamespace: \"ns2\",\n\t\t},\n\t\t{\n\t\t\tresource: \"v1/PersistentVolume\",\n\t\t\tname: \"my-pv\",\n\t\t},\n\t}\n\tbackedUpItems := NewBackedUpItemsMap()\n\tfor _, it := range items {\n\t\tbackedUpItems.AddItem(it)\n\t}\n\n\treq := Request{BackedUpItems: backedUpItems}\n\tassert.Equal(t, map[string][]string{\n\t\t\"apps/v1/Deployment\": {\"default/my-deploy\"},\n\t\t\"v1/Pod\": {\"ns1/pod1\", \"ns2/pod2\"},\n\t\t\"v1/PersistentVolume\": {\"my-pv\"},\n\t}, req.BackupResourceList())\n}\n\nfunc TestRequest_BackupResourceListEntriesSorted(t *testing.T) {\n\titems := []itemKey{\n\t\t{\n\t\t\tresource: \"v1/Pod\",\n\t\t\tname: \"pod2\",\n\t\t\tnamespace: \"ns2\",\n\t\t},\n\t\t{\n\t\t\tresource: \"v1/Pod\",\n\t\t\tname: \"pod1\",\n\t\t\tnamespace: \"ns1\",\n\t\t},\n\t}\n\tbackedUpItems := NewBackedUpItemsMap()\n\tfor _, it := range items {\n\t\tbackedUpItems.AddItem(it)\n\t}\n\n\treq := Request{BackedUpItems: backedUpItems}\n\tassert.Equal(t, map[string][]string{\n\t\t\"v1/Pod\": {\"ns1/pod1\", \"ns2/pod2\"},\n\t}, req.BackupResourceList())\n}\n" }, { "path": "pkg/backup/snapshots.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"context\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/features\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n)\n\n// GetBackupCSIResources is used to get CSI snapshot related resources.\n// Returns VolumeSnapshot, VolumeSnapshotContent, VolumeSnapshotClasses referenced\nfunc GetBackupCSIResources(\n\tclient kbclient.Client,\n\tglobalCRClient kbclient.Client,\n\tbackup *velerov1api.Backup,\n\tbackupLog logrus.FieldLogger,\n) (\n\tvolumeSnapshots []snapshotv1api.VolumeSnapshot,\n\tvolumeSnapshotContents []snapshotv1api.VolumeSnapshotContent,\n\tvolumeSnapshotClasses []snapshotv1api.VolumeSnapshotClass,\n) {\n\tif boolptr.IsSetToTrue(backup.Spec.SnapshotMoveData) {\n\t\tbackupLog.Info(\"backup SnapshotMoveData is set to true, skip VolumeSnapshot resource persistence.\")\n\t} else if features.IsEnabled(velerov1api.CSIFeatureFlag) {\n\t\tselector := label.NewSelectorForBackup(backup.Name)\n\t\tvscList := &snapshotv1api.VolumeSnapshotContentList{}\n\n\t\tvsList := new(snapshotv1api.VolumeSnapshotList)\n\t\terr := globalCRClient.List(context.TODO(), vsList, &kbclient.ListOptions{\n\t\t\tLabelSelector: label.NewSelectorForBackup(backup.Name),\n\t\t})\n\t\tif err != nil {\n\t\t\tbackupLog.Error(err)\n\t\t}\n\t\tvolumeSnapshots = append(volumeSnapshots, vsList.Items...)\n\n\t\tif err := client.List(context.Background(), vscList, &kbclient.ListOptions{LabelSelector: selector}); err != nil {\n\t\t\tbackupLog.Error(err)\n\t\t}\n\t\tif len(vscList.Items) >= 0 {\n\t\t\tvolumeSnapshotContents = vscList.Items\n\t\t}\n\n\t\tvsClassSet := sets.NewString()\n\t\tfor index := range volumeSnapshotContents {\n\t\t\t// persist the volumesnapshotclasses referenced by vsc\n\t\t\tif volumeSnapshotContents[index].Spec.VolumeSnapshotClassName != nil && !vsClassSet.Has(*volumeSnapshotContents[index].Spec.VolumeSnapshotClassName) {\n\t\t\t\tvsClass := &snapshotv1api.VolumeSnapshotClass{}\n\t\t\t\tif err := client.Get(context.TODO(), kbclient.ObjectKey{Name: *volumeSnapshotContents[index].Spec.VolumeSnapshotClassName}, vsClass); err != nil {\n\t\t\t\t\tbackupLog.Error(err)\n\t\t\t\t} else {\n\t\t\t\t\tvsClassSet.Insert(*volumeSnapshotContents[index].Spec.VolumeSnapshotClassName)\n\t\t\t\t\tvolumeSnapshotClasses = append(volumeSnapshotClasses, *vsClass)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\tbackup.Status.CSIVolumeSnapshotsAttempted = len(volumeSnapshots)\n\t}\n\n\treturn volumeSnapshots, volumeSnapshotContents, volumeSnapshotClasses\n}\n" }, { "path": "pkg/backup/volume_snapshotter_cache.go", "content": "package backup\n\nimport (\n\t\"sync\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvsv1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/volumesnapshotter/v1\"\n)\n\ntype VolumeSnapshotterCache struct {\n\tcache map[string]vsv1.VolumeSnapshotter\n\tmutex sync.Mutex\n\tgetter VolumeSnapshotterGetter\n}\n\nfunc NewVolumeSnapshotterCache(getter VolumeSnapshotterGetter) *VolumeSnapshotterCache {\n\treturn &VolumeSnapshotterCache{\n\t\tcache: make(map[string]vsv1.VolumeSnapshotter),\n\t\tgetter: getter,\n\t}\n}\n\nfunc (c *VolumeSnapshotterCache) SetNX(location *velerov1api.VolumeSnapshotLocation) (vsv1.VolumeSnapshotter, error) {\n\tc.mutex.Lock()\n\tdefer c.mutex.Unlock()\n\n\tif snapshotter, exists := c.cache[location.Name]; exists {\n\t\treturn snapshotter, nil\n\t}\n\n\tsnapshotter, err := c.getter.GetVolumeSnapshotter(location.Spec.Provider)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := snapshotter.Init(location.Spec.Config); err != nil {\n\t\treturn nil, err\n\t}\n\n\tc.cache[location.Name] = snapshotter\n\treturn snapshotter, nil\n}\n" }, { "path": "pkg/builder/backup_builder.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\t\"fmt\"\n\t\"time\"\n\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\t\"github.com/vmware-tanzu/velero/internal/resourcepolicies\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n)\n\n/*\n\nExample usage:\n\nvar backup = builder.ForBackup(\"velero\", \"backup-1\").\n\tObjectMeta(\n\t\tbuilder.WithLabels(\"foo\", \"bar\"),\n\t\tbuilder.WithClusterName(\"cluster-1\"),\n\t).\n\tSnapshotVolumes(true).\n\tResult()\n\n*/\n\n// BackupBuilder builds Backup objects.\ntype BackupBuilder struct {\n\tobject *velerov1api.Backup\n}\n\n// ForBackup is the constructor for a BackupBuilder.\nfunc ForBackup(ns, name string) *BackupBuilder {\n\treturn &BackupBuilder{\n\t\tobject: &velerov1api.Backup{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"Backup\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built Backup.\nfunc (b *BackupBuilder) Result() *velerov1api.Backup {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the Backup's ObjectMeta.\nfunc (b *BackupBuilder) ObjectMeta(opts ...ObjectMetaOpt) *BackupBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\n\treturn b\n}\n\n// FromSchedule sets the Backup's spec and labels from the Schedule template\nfunc (b *BackupBuilder) FromSchedule(schedule *velerov1api.Schedule) *BackupBuilder {\n\tvar labels map[string]string\n\n\t// Check if there's explicit Labels defined in the Schedule object template\n\t// and if present then copy it to the backup object.\n\tif schedule.Spec.Template.Metadata.Labels != nil {\n\t\tlogger := logging.DefaultLogger(logging.LogLevelFlag(logrus.InfoLevel).Parse(), logging.NewFormatFlag().Parse())\n\t\tlabels = schedule.Spec.Template.Metadata.Labels\n\t\tlogger.WithFields(logrus.Fields{\n\t\t\t\"backup\": fmt.Sprintf(\"%s/%s\", b.object.GetNamespace(), b.object.GetName()),\n\t\t\t\"labels\": schedule.Spec.Template.Metadata.Labels,\n\t\t}).Info(\"Schedule.template.metadata.labels set - using those labels instead of schedule.labels for backup object\")\n\t} else {\n\t\tlabels = schedule.Labels\n\t\tlogrus.WithFields(logrus.Fields{\n\t\t\t\"backup\": fmt.Sprintf(\"%s/%s\", b.object.GetNamespace(), b.object.GetName()),\n\t\t\t\"labels\": schedule.Labels,\n\t\t}).Info(\"No Schedule.template.metadata.labels set - using Schedule.labels for backup object\")\n\t}\n\tif labels == nil {\n\t\tlabels = make(map[string]string)\n\t}\n\tlabels[velerov1api.ScheduleNameLabel] = schedule.Name\n\n\tb.object.Spec = schedule.Spec.Template\n\tb.ObjectMeta(WithLabelsMap(labels))\n\n\tif schedule.Annotations != nil {\n\t\tb.ObjectMeta(WithAnnotationsMap(schedule.Annotations))\n\t}\n\n\tif boolptr.IsSetToTrue(schedule.Spec.UseOwnerReferencesInBackup) {\n\t\tb.object.SetOwnerReferences([]metav1.OwnerReference{\n\t\t\t{\n\t\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"Schedule\",\n\t\t\t\tName: schedule.Name,\n\t\t\t\tUID: schedule.UID,\n\t\t\t\tController: boolptr.True(),\n\t\t\t},\n\t\t})\n\t}\n\n\tif schedule.Spec.Template.ResourcePolicy != nil {\n\t\tb.ResourcePolicies(schedule.Spec.Template.ResourcePolicy.Name)\n\t}\n\n\treturn b\n}\n\n// IncludedNamespaces sets the Backup's included namespaces.\nfunc (b *BackupBuilder) IncludedNamespaces(namespaces ...string) *BackupBuilder {\n\tb.object.Spec.IncludedNamespaces = namespaces\n\treturn b\n}\n\n// ExcludedNamespaces sets the Backup's excluded namespaces.\nfunc (b *BackupBuilder) ExcludedNamespaces(namespaces ...string) *BackupBuilder {\n\tb.object.Spec.ExcludedNamespaces = namespaces\n\treturn b\n}\n\n// IncludedResources sets the Backup's included resources.\nfunc (b *BackupBuilder) IncludedResources(resources ...string) *BackupBuilder {\n\tb.object.Spec.IncludedResources = resources\n\treturn b\n}\n\n// ExcludedResources sets the Backup's excluded resources.\nfunc (b *BackupBuilder) ExcludedResources(resources ...string) *BackupBuilder {\n\tb.object.Spec.ExcludedResources = resources\n\treturn b\n}\n\n// IncludedClusterScopedResources sets the Backup's included cluster resources.\nfunc (b *BackupBuilder) IncludedClusterScopedResources(resources ...string) *BackupBuilder {\n\tb.object.Spec.IncludedClusterScopedResources = resources\n\treturn b\n}\n\n// ExcludedClusterScopedResources sets the Backup's excluded cluster resources.\nfunc (b *BackupBuilder) ExcludedClusterScopedResources(resources ...string) *BackupBuilder {\n\tb.object.Spec.ExcludedClusterScopedResources = resources\n\treturn b\n}\n\n// IncludedNamespaceScopedResources sets the Backup's included namespaced resources.\nfunc (b *BackupBuilder) IncludedNamespaceScopedResources(resources ...string) *BackupBuilder {\n\tb.object.Spec.IncludedNamespaceScopedResources = resources\n\treturn b\n}\n\n// ExcludedNamespaceScopedResources sets the Backup's excluded namespaced resources.\nfunc (b *BackupBuilder) ExcludedNamespaceScopedResources(resources ...string) *BackupBuilder {\n\tb.object.Spec.ExcludedNamespaceScopedResources = resources\n\treturn b\n}\n\n// IncludeClusterResources sets the Backup's \"include cluster resources\" flag.\nfunc (b *BackupBuilder) IncludeClusterResources(val bool) *BackupBuilder {\n\tb.object.Spec.IncludeClusterResources = &val\n\treturn b\n}\n\n// LabelSelector sets the Backup's label selector.\nfunc (b *BackupBuilder) LabelSelector(selector *metav1.LabelSelector) *BackupBuilder {\n\tb.object.Spec.LabelSelector = selector\n\treturn b\n}\n\n// OrLabelSelector sets the Backup's orLabelSelector set.\nfunc (b *BackupBuilder) OrLabelSelector(orSelectors []*metav1.LabelSelector) *BackupBuilder {\n\tb.object.Spec.OrLabelSelectors = orSelectors\n\treturn b\n}\n\n// SnapshotVolumes sets the Backup's \"snapshot volumes\" flag.\nfunc (b *BackupBuilder) SnapshotVolumes(val bool) *BackupBuilder {\n\tb.object.Spec.SnapshotVolumes = &val\n\treturn b\n}\n\n// DefaultVolumesToFsBackup sets the Backup's \"DefaultVolumesToFsBackup\" flag.\nfunc (b *BackupBuilder) DefaultVolumesToFsBackup(val bool) *BackupBuilder {\n\tb.object.Spec.DefaultVolumesToFsBackup = &val\n\treturn b\n}\n\n// DefaultVolumesToRestic sets the Backup's \"DefaultVolumesToRestic\" flag.\nfunc (b *BackupBuilder) DefaultVolumesToRestic(val bool) *BackupBuilder {\n\tb.object.Spec.DefaultVolumesToRestic = &val\n\treturn b\n}\n\n// Phase sets the Backup's phase.\nfunc (b *BackupBuilder) Phase(phase velerov1api.BackupPhase) *BackupBuilder {\n\tb.object.Status.Phase = phase\n\treturn b\n}\n\n// Phase sets the Backup's queue position.\nfunc (b *BackupBuilder) QueuePosition(queuePos int) *BackupBuilder {\n\tb.object.Status.QueuePosition = queuePos\n\treturn b\n}\n\n// StorageLocation sets the Backup's storage location.\nfunc (b *BackupBuilder) StorageLocation(location string) *BackupBuilder {\n\tb.object.Spec.StorageLocation = location\n\treturn b\n}\n\n// VolumeSnapshotLocations sets the Backup's volume snapshot locations.\nfunc (b *BackupBuilder) VolumeSnapshotLocations(locations ...string) *BackupBuilder {\n\tb.object.Spec.VolumeSnapshotLocations = locations\n\treturn b\n}\n\n// TTL sets the Backup's TTL.\nfunc (b *BackupBuilder) TTL(ttl time.Duration) *BackupBuilder {\n\tb.object.Spec.TTL.Duration = ttl\n\treturn b\n}\n\n// VolumeGroupSnapshotLabelKey sets the label key to group PVCs for VolumeGroupSnapshot.\nfunc (b *BackupBuilder) VolumeGroupSnapshotLabelKey(labelKey string) *BackupBuilder {\n\tb.object.Spec.VolumeGroupSnapshotLabelKey = labelKey\n\treturn b\n}\n\n// Expiration sets the Backup's expiration.\nfunc (b *BackupBuilder) Expiration(val time.Time) *BackupBuilder {\n\tb.object.Status.Expiration = &metav1.Time{Time: val}\n\treturn b\n}\n\n// StartTimestamp sets the Backup's start timestamp.\nfunc (b *BackupBuilder) StartTimestamp(val time.Time) *BackupBuilder {\n\tb.object.Status.StartTimestamp = &metav1.Time{Time: val}\n\treturn b\n}\n\n// CompletionTimestamp sets the Backup's completion timestamp.\nfunc (b *BackupBuilder) CompletionTimestamp(val time.Time) *BackupBuilder {\n\tb.object.Status.CompletionTimestamp = &metav1.Time{Time: val}\n\treturn b\n}\n\n// Hooks sets the Backup's hooks.\nfunc (b *BackupBuilder) Hooks(hooks velerov1api.BackupHooks) *BackupBuilder {\n\tb.object.Spec.Hooks = hooks\n\treturn b\n}\n\n// OrderedResources sets the Backup's OrderedResources\nfunc (b *BackupBuilder) OrderedResources(orders map[string]string) *BackupBuilder {\n\tb.object.Spec.OrderedResources = orders\n\treturn b\n}\n\n// CSISnapshotTimeout sets the Backup's CSISnapshotTimeout\nfunc (b *BackupBuilder) CSISnapshotTimeout(timeout time.Duration) *BackupBuilder {\n\tb.object.Spec.CSISnapshotTimeout.Duration = timeout\n\treturn b\n}\n\n// ItemOperationTimeout sets the Backup's ItemOperationTimeout\nfunc (b *BackupBuilder) ItemOperationTimeout(timeout time.Duration) *BackupBuilder {\n\tb.object.Spec.ItemOperationTimeout.Duration = timeout\n\treturn b\n}\n\n// ResourcePolicies sets the Backup's resource polices.\nfunc (b *BackupBuilder) ResourcePolicies(name string) *BackupBuilder {\n\tb.object.Spec.ResourcePolicy = &corev1api.TypedLocalObjectReference{Kind: resourcepolicies.ConfigmapRefType, Name: name}\n\treturn b\n}\n\n// SnapshotMoveData sets the Backup's \"snapshot move data\" flag.\nfunc (b *BackupBuilder) SnapshotMoveData(val bool) *BackupBuilder {\n\tb.object.Spec.SnapshotMoveData = &val\n\treturn b\n}\n\n// DataMover sets the Backup's data mover\nfunc (b *BackupBuilder) DataMover(name string) *BackupBuilder {\n\tb.object.Spec.DataMover = name\n\treturn b\n}\n\n// ParallelFilesUpload sets the Backup's uploader parallel uploads\nfunc (b *BackupBuilder) ParallelFilesUpload(parallel int) *BackupBuilder {\n\tif b.object.Spec.UploaderConfig == nil {\n\t\tb.object.Spec.UploaderConfig = &velerov1api.UploaderConfigForBackup{}\n\t}\n\tb.object.Spec.UploaderConfig.ParallelFilesUpload = parallel\n\treturn b\n}\n\n// WithStatus sets the Backup's status.\nfunc (b *BackupBuilder) WithStatus(status velerov1api.BackupStatus) *BackupBuilder {\n\tb.object.Status = status\n\treturn b\n}\n" }, { "path": "pkg/builder/backup_storage_location_builder.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\t\"time\"\n\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// BackupStorageLocationBuilder builds BackupStorageLocation objects.\ntype BackupStorageLocationBuilder struct {\n\tobject *velerov1api.BackupStorageLocation\n}\n\n// ForBackupStorageLocation is the constructor for a BackupStorageLocationBuilder.\nfunc ForBackupStorageLocation(ns, name string) *BackupStorageLocationBuilder {\n\treturn &BackupStorageLocationBuilder{\n\t\tobject: &velerov1api.BackupStorageLocation{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"BackupStorageLocation\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built BackupStorageLocation.\nfunc (b *BackupStorageLocationBuilder) Result() *velerov1api.BackupStorageLocation {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the BackupStorageLocation's ObjectMeta.\nfunc (b *BackupStorageLocationBuilder) ObjectMeta(opts ...ObjectMetaOpt) *BackupStorageLocationBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\n\treturn b\n}\n\n// Provider sets the BackupStorageLocation's provider.\nfunc (b *BackupStorageLocationBuilder) Provider(name string) *BackupStorageLocationBuilder {\n\tb.object.Spec.Provider = name\n\treturn b\n}\n\n// Bucket sets the BackupStorageLocation's object storage bucket.\nfunc (b *BackupStorageLocationBuilder) Bucket(val string) *BackupStorageLocationBuilder {\n\tif b.object.Spec.StorageType.ObjectStorage == nil {\n\t\tb.object.Spec.StorageType.ObjectStorage = new(velerov1api.ObjectStorageLocation)\n\t}\n\tb.object.Spec.ObjectStorage.Bucket = val\n\treturn b\n}\n\n// Prefix sets the BackupStorageLocation's object storage prefix.\nfunc (b *BackupStorageLocationBuilder) Prefix(val string) *BackupStorageLocationBuilder {\n\tif b.object.Spec.StorageType.ObjectStorage == nil {\n\t\tb.object.Spec.StorageType.ObjectStorage = new(velerov1api.ObjectStorageLocation)\n\t}\n\tb.object.Spec.ObjectStorage.Prefix = val\n\treturn b\n}\n\n// CACert sets the BackupStorageLocation's object storage CACert.\nfunc (b *BackupStorageLocationBuilder) CACert(val []byte) *BackupStorageLocationBuilder {\n\tif b.object.Spec.StorageType.ObjectStorage == nil {\n\t\tb.object.Spec.StorageType.ObjectStorage = new(velerov1api.ObjectStorageLocation)\n\t}\n\tb.object.Spec.ObjectStorage.CACert = val\n\treturn b\n}\n\n// CACertRef sets the BackupStorageLocation's object storage CACertRef (Secret reference).\nfunc (b *BackupStorageLocationBuilder) CACertRef(selector *corev1api.SecretKeySelector) *BackupStorageLocationBuilder {\n\tif b.object.Spec.StorageType.ObjectStorage == nil {\n\t\tb.object.Spec.StorageType.ObjectStorage = new(velerov1api.ObjectStorageLocation)\n\t}\n\tb.object.Spec.ObjectStorage.CACertRef = selector\n\treturn b\n}\n\n// Default sets the BackupStorageLocation's is default or not\nfunc (b *BackupStorageLocationBuilder) Default(isDefault bool) *BackupStorageLocationBuilder {\n\tb.object.Spec.Default = isDefault\n\treturn b\n}\n\n// AccessMode sets the BackupStorageLocation's access mode.\nfunc (b *BackupStorageLocationBuilder) AccessMode(accessMode velerov1api.BackupStorageLocationAccessMode) *BackupStorageLocationBuilder {\n\tb.object.Spec.AccessMode = accessMode\n\treturn b\n}\n\n// ValidationFrequency sets the BackupStorageLocation's validation frequency.\nfunc (b *BackupStorageLocationBuilder) ValidationFrequency(frequency time.Duration) *BackupStorageLocationBuilder {\n\tb.object.Spec.ValidationFrequency = &metav1.Duration{Duration: frequency}\n\treturn b\n}\n\n// LastValidationTime sets the BackupStorageLocation's last validated time.\nfunc (b *BackupStorageLocationBuilder) LastValidationTime(lastValidated time.Time) *BackupStorageLocationBuilder {\n\tb.object.Status.LastValidationTime = &metav1.Time{Time: lastValidated}\n\treturn b\n}\n\n// Phase sets the BackupStorageLocation's status phase.\nfunc (b *BackupStorageLocationBuilder) Phase(phase velerov1api.BackupStorageLocationPhase) *BackupStorageLocationBuilder {\n\tb.object.Status.Phase = phase\n\treturn b\n}\n\n// Credential sets the BackupStorageLocation's credential selector.\nfunc (b *BackupStorageLocationBuilder) Credential(selector *corev1api.SecretKeySelector) *BackupStorageLocationBuilder {\n\tb.object.Spec.Credential = selector\n\treturn b\n}\n" }, { "path": "pkg/builder/config_map_builder.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// ConfigMapBuilder builds ConfigMap objects.\ntype ConfigMapBuilder struct {\n\tobject *corev1api.ConfigMap\n}\n\n// ForConfigMap is the constructor for a ConfigMapBuilder.\nfunc ForConfigMap(ns, name string) *ConfigMapBuilder {\n\treturn &ConfigMapBuilder{\n\t\tobject: &corev1api.ConfigMap{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: corev1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"ConfigMap\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built ConfigMap.\nfunc (b *ConfigMapBuilder) Result() *corev1api.ConfigMap {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the ConfigMap's ObjectMeta.\nfunc (b *ConfigMapBuilder) ObjectMeta(opts ...ObjectMetaOpt) *ConfigMapBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\n\treturn b\n}\n\n// Data set's the ConfigMap's data.\nfunc (b *ConfigMapBuilder) Data(vals ...string) *ConfigMapBuilder {\n\tb.object.Data = setMapEntries(b.object.Data, vals...)\n\treturn b\n}\n" }, { "path": "pkg/builder/container_builder.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\t\"encoding/json\"\n\t\"strings\"\n\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapimachineryRuntime \"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n)\n\n// ContainerBuilder builds Container objects\ntype ContainerBuilder struct {\n\tobject *corev1api.Container\n}\n\n// ForContainer is the constructor for ContainerBuilder.\nfunc ForContainer(name, image string) *ContainerBuilder {\n\treturn &ContainerBuilder{\n\t\tobject: &corev1api.Container{\n\t\t\tName: name,\n\t\t\tImage: image,\n\t\t},\n\t}\n}\n\n// ForPluginContainer is a helper builder specifically for plugin init containers\nfunc ForPluginContainer(image string, pullPolicy corev1api.PullPolicy) *ContainerBuilder {\n\tvolumeMount := ForVolumeMount(\"plugins\", \"/target\").Result()\n\treturn ForContainer(getName(image), image).PullPolicy(pullPolicy).VolumeMounts(volumeMount)\n}\n\n// getName returns the 'name' component of a docker image that includes the entire string\n// except the registry name, and transforms the combined string into a DNS-1123 compatible name\n// that fits within the 63-character limit for Kubernetes container names.\nfunc getName(image string) string {\n\tslashIndex := strings.Index(image, \"/\")\n\tslashCount := 0\n\tif slashIndex >= 0 {\n\t\tslashCount = strings.Count(image[slashIndex:], \"/\")\n\t}\n\n\tstart := 0\n\tif slashCount > 1 || slashIndex == 0 {\n\t\t// always start after the first slash when there is a registry name\n\t\t// or if the string starts with a slash.\n\t\tstart = slashIndex + 1\n\t}\n\n\t// If the image spec is by digest, remove the digest.\n\t// If it is by tag, remove the tag.\n\t// Otherwise (implicit :latest) leave it alone.\n\tend := len(image)\n\tatIndex := strings.LastIndex(image, \"@\")\n\tif atIndex > 0 {\n\t\tend = atIndex\n\t} else {\n\t\tcolonIndex := strings.LastIndex(image, \":\")\n\t\tif colonIndex > 0 {\n\t\t\tend = colonIndex\n\t\t}\n\t}\n\n\t// https://github.com/distribution/distribution/blob/main/docs/spec/api.md#overview\n\t// valid repository names match the regex [a-z0-9]+(?:[._-][a-z0-9]+)*\n\t// image repository names can container [._] but [._] are not allowed in RFC-1123 labels.\n\t// replace '/', '_' and '.' with '-'\n\tre := strings.NewReplacer(\"/\", \"-\",\n\t\t\"_\", \"-\",\n\t\t\".\", \"-\")\n\tname := re.Replace(image[start:end])\n\n\t// Ensure the name doesn't exceed Kubernetes container name length limit\n\treturn label.GetValidName(name)\n}\n\n// Result returns the built Container.\nfunc (b *ContainerBuilder) Result() *corev1api.Container {\n\treturn b.object\n}\n\n// ResultRawExtension returns the Container as runtime.RawExtension.\nfunc (b *ContainerBuilder) ResultRawExtension() apimachineryRuntime.RawExtension {\n\tresult, err := json.Marshal(b.object)\n\tif err != nil {\n\t\treturn apimachineryRuntime.RawExtension{}\n\t}\n\treturn apimachineryRuntime.RawExtension{\n\t\tRaw: result,\n\t}\n}\n\n// Args sets the container's Args.\nfunc (b *ContainerBuilder) Args(args ...string) *ContainerBuilder {\n\tb.object.Args = append(b.object.Args, args...)\n\treturn b\n}\n\n// VolumeMounts sets the container's VolumeMounts.\nfunc (b *ContainerBuilder) VolumeMounts(volumeMounts ...*corev1api.VolumeMount) *ContainerBuilder {\n\tfor _, v := range volumeMounts {\n\t\tb.object.VolumeMounts = append(b.object.VolumeMounts, *v)\n\t}\n\treturn b\n}\n\n// Resources sets the container's Resources.\nfunc (b *ContainerBuilder) Resources(resources *corev1api.ResourceRequirements) *ContainerBuilder {\n\tb.object.Resources = *resources\n\treturn b\n}\n\n// SecurityContext sets the container's SecurityContext.\nfunc (b *ContainerBuilder) SecurityContext(securityContext *corev1api.SecurityContext) *ContainerBuilder {\n\tb.object.SecurityContext = securityContext\n\treturn b\n}\n\nfunc (b *ContainerBuilder) Env(vars ...*corev1api.EnvVar) *ContainerBuilder {\n\tfor _, v := range vars {\n\t\tb.object.Env = append(b.object.Env, *v)\n\t}\n\treturn b\n}\n\nfunc (b *ContainerBuilder) PullPolicy(pullPolicy corev1api.PullPolicy) *ContainerBuilder {\n\tb.object.ImagePullPolicy = pullPolicy\n\treturn b\n}\n\nfunc (b *ContainerBuilder) Command(command []string) *ContainerBuilder {\n\tif b.object.Command == nil {\n\t\tb.object.Command = []string{}\n\t}\n\n\tb.object.Command = append(b.object.Command, command...)\n\n\treturn b\n}\n" }, { "path": "pkg/builder/container_builder_test.go", "content": "/*\nCopyright 2018, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage builder\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestGetName(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\timage string\n\t\texpected string\n\t}{\n\t\t{\n\t\t\tname: \"image name with registry hostname and tag\",\n\t\t\timage: \"gcr.io/my-repo/my-image:latest\",\n\t\t\texpected: \"my-repo-my-image\",\n\t\t},\n\t\t{\n\t\t\tname: \"image name with registry hostname, without tag\",\n\t\t\timage: \"gcr.io/my-repo/my-image\",\n\t\t\texpected: \"my-repo-my-image\",\n\t\t},\n\t\t{\n\t\t\tname: \"image name without registry hostname, with tag\",\n\t\t\timage: \"my-repo/my-image:latest\",\n\t\t\texpected: \"my-repo-my-image\",\n\t\t},\n\t\t{\n\t\t\tname: \"image name without registry hostname, without tag\",\n\t\t\timage: \"my-repo/my-image\",\n\t\t\texpected: \"my-repo-my-image\",\n\t\t},\n\t\t{\n\t\t\tname: \"image name with registry hostname and port, and tag\",\n\t\t\timage: \"mycustomregistry.io:8080/my-repo/my-image:latest\",\n\t\t\texpected: \"my-repo-my-image\",\n\t\t},\n\t\t{\n\t\t\tname: \"image name with no / in it\",\n\t\t\timage: \"my-image\",\n\t\t\texpected: \"my-image\",\n\t\t},\n\t\t{\n\t\t\tname: \"image name starting with / in it\",\n\t\t\timage: \"/my-image\",\n\t\t\texpected: \"my-image\",\n\t\t},\n\t\t{\n\t\t\tname: \"image name with repo starting with a / as first char\",\n\t\t\timage: \"/my-repo/my-image\",\n\t\t\texpected: \"my-repo-my-image\",\n\t\t},\n\t\t{\n\t\t\tname: \"image name with registry hostname, etoomany slashes, without tag\",\n\t\t\timage: \"gcr.io/my-repo/mystery/another/my-image\",\n\t\t\texpected: \"my-repo-mystery-another-my-image\",\n\t\t},\n\t\t{\n\t\t\tname: \"image name with registry hostname starting with a / will include the registry name ¯\\\\_(ツ)_/¯\",\n\t\t\timage: \"/gcr.io/my-repo/mystery/another/my-image\",\n\t\t\texpected: \"gcr-io-my-repo-mystery-another-my-image\",\n\t\t},\n\t\t{\n\t\t\tname: \"image repository names containing _ \",\n\t\t\timage: \"projects.registry.vmware.com/tanzu_migrator/route-2-httpproxy:myTag\",\n\t\t\texpected: \"tanzu-migrator-route-2-httpproxy\",\n\t\t},\n\t\t{\n\t\t\tname: \"image repository names containing . \",\n\t\t\timage: \"projects.registry.vmware.com/tanzu.migrator/route-2-httpproxy:myTag\",\n\t\t\texpected: \"tanzu-migrator-route-2-httpproxy\",\n\t\t},\n\t\t{\n\t\t\tname: \"pull by digest\",\n\t\t\timage: \"quay.io/vmware-tanzu/velero@sha256:a75f9e8c3ced3943515f249597be389f8233e1258d289b11184796edceaa7dab\",\n\t\t\texpected: \"vmware-tanzu-velero\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tassert.Equal(t, test.expected, getName(test.image))\n\t\t})\n\t}\n}\n\nfunc TestGetNameWithLongPaths(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\timage string\n\t\tvalidate func(t *testing.T, result string)\n\t}{\n\t\t{\n\t\t\tname: \"plugin with deeply nested repository path exceeding 63 characters\",\n\t\t\timage: \"arohcpsvcdev.azurecr.io/redhat-user-workloads/ocp-art-tenant/oadp-hypershift-oadp-plugin-main@sha256:adb840bf3890b4904a8cdda1a74c82cf8d96c52eba9944ac10e795335d6fd450\",\n\t\t\tvalidate: func(t *testing.T, result string) {\n\t\t\t\tt.Helper()\n\t\t\t\t// Should not exceed DNS-1123 label limit of 63 characters\n\t\t\t\tassert.LessOrEqual(t, len(result), 63, \"Container name must satisfy DNS-1123 label constraints (max 63 chars)\")\n\t\t\t\t// Should be exactly 63 characters (truncated with hash)\n\t\t\t\tassert.Len(t, result, 63)\n\t\t\t\t// Should be deterministic\n\t\t\t\tresult2 := getName(\"arohcpsvcdev.azurecr.io/redhat-user-workloads/ocp-art-tenant/oadp-hypershift-oadp-plugin-main@sha256:adb840bf3890b4904a8cdda1a74c82cf8d96c52eba9944ac10e795335d6fd450\")\n\t\t\t\tassert.Equal(t, result, result2)\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"plugin with normal path length (should remain unchanged)\",\n\t\t\timage: \"arohcpsvcdev.azurecr.io/konveyor/velero-plugin-for-microsoft-azure@sha256:b2db5f09da514e817a74c992dcca5f90b77c2ab0b2797eba947d224271d6070e\",\n\t\t\tvalidate: func(t *testing.T, result string) {\n\t\t\t\tt.Helper()\n\t\t\t\tassert.Equal(t, \"konveyor-velero-plugin-for-microsoft-azure\", result)\n\t\t\t\tassert.LessOrEqual(t, len(result), 63)\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"very long nested path\",\n\t\t\timage: \"registry.example.com/org/team/project/subproject/component/service/application-name-with-many-words:v1.2.3\",\n\t\t\tvalidate: func(t *testing.T, result string) {\n\t\t\t\tt.Helper()\n\t\t\t\tassert.LessOrEqual(t, len(result), 63)\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tresult := getName(test.image)\n\t\t\ttest.validate(t, result)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/builder/customresourcedefinition_v1beta1_builder.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tapiextv1beta1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// CustomResourceDefinitionV1Beta1Builder builds v1beta1 CustomResourceDefinition objects.\ntype CustomResourceDefinitionV1Beta1Builder struct {\n\tobject *apiextv1beta1.CustomResourceDefinition\n}\n\n// ForCustomResourceDefinitionV1Beta1 is the constructor for a CustomResourceDefinitionV1Beta1Builder.\nfunc ForCustomResourceDefinitionV1Beta1(name string) *CustomResourceDefinitionV1Beta1Builder {\n\treturn &CustomResourceDefinitionV1Beta1Builder{\n\t\tobject: &apiextv1beta1.CustomResourceDefinition{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: apiextv1beta1.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"CustomResourceDefinition\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Condition adds a CustomResourceDefinitionCondition objects to a CustomResourceDefinitionV1Beta1Builder.\nfunc (c *CustomResourceDefinitionV1Beta1Builder) Condition(cond apiextv1beta1.CustomResourceDefinitionCondition) *CustomResourceDefinitionV1Beta1Builder {\n\tc.object.Status.Conditions = append(c.object.Status.Conditions, cond)\n\treturn c\n}\n\n// Result returns the built CustomResourceDefinition.\nfunc (c *CustomResourceDefinitionV1Beta1Builder) Result() *apiextv1beta1.CustomResourceDefinition {\n\treturn c.object\n}\n\n// ObjectMeta applies functional options to the CustomResourceDefinition's ObjectMeta.\nfunc (c *CustomResourceDefinitionV1Beta1Builder) ObjectMeta(opts ...ObjectMetaOpt) *CustomResourceDefinitionV1Beta1Builder {\n\tfor _, opt := range opts {\n\t\topt(c.object)\n\t}\n\n\treturn c\n}\n\n// CustomResourceDefinitionV1Beta1ConditionBuilder builds CustomResourceDefinitionV1Beta1Condition objects.\ntype CustomResourceDefinitionV1Beta1ConditionBuilder struct {\n\tobject apiextv1beta1.CustomResourceDefinitionCondition\n}\n\n// ForCustomResourceDefinitionV1Beta1Condition is the construction for a CustomResourceDefinitionV1Beta1ConditionBuilder.\nfunc ForCustomResourceDefinitionV1Beta1Condition() *CustomResourceDefinitionV1Beta1ConditionBuilder {\n\treturn &CustomResourceDefinitionV1Beta1ConditionBuilder{\n\t\tobject: apiextv1beta1.CustomResourceDefinitionCondition{},\n\t}\n}\n\n// Type sets the Condition's type.\nfunc (c *CustomResourceDefinitionV1Beta1ConditionBuilder) Type(t apiextv1beta1.CustomResourceDefinitionConditionType) *CustomResourceDefinitionV1Beta1ConditionBuilder {\n\tc.object.Type = t\n\treturn c\n}\n\n// Status sets the Condition's status.\nfunc (c *CustomResourceDefinitionV1Beta1ConditionBuilder) Status(cs apiextv1beta1.ConditionStatus) *CustomResourceDefinitionV1Beta1ConditionBuilder {\n\tc.object.Status = cs\n\treturn c\n}\n\n// Result returns the built v1beta1 CustomResourceDefinitionCondition.\nfunc (c *CustomResourceDefinitionV1Beta1ConditionBuilder) Result() apiextv1beta1.CustomResourceDefinitionCondition {\n\treturn c.object\n}\n" }, { "path": "pkg/builder/data_download_builder.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/apis/velero/shared\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n)\n\n// DataDownloadBuilder builds DataDownload objects.\ntype DataDownloadBuilder struct {\n\tobject *velerov2alpha1api.DataDownload\n}\n\n// ForDataDownload is the constructor of DataDownloadBuilder\nfunc ForDataDownload(namespace, name string) *DataDownloadBuilder {\n\treturn &DataDownloadBuilder{\n\t\tobject: &velerov2alpha1api.DataDownload{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tKind: \"DataDownload\",\n\t\t\t\tAPIVersion: velerov2alpha1api.SchemeGroupVersion.String(),\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: name,\n\t\t\t\tNamespace: namespace,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built DataDownload.\nfunc (d *DataDownloadBuilder) Result() *velerov2alpha1api.DataDownload {\n\treturn d.object\n}\n\n// BackupStorageLocation sets the DataDownload's backup storage location.\nfunc (d *DataDownloadBuilder) BackupStorageLocation(name string) *DataDownloadBuilder {\n\td.object.Spec.BackupStorageLocation = name\n\treturn d\n}\n\n// Phase sets the DataDownload's phase.\nfunc (d *DataDownloadBuilder) Phase(phase velerov2alpha1api.DataDownloadPhase) *DataDownloadBuilder {\n\td.object.Status.Phase = phase\n\treturn d\n}\n\n// SnapshotID sets the DataDownload's SnapshotID.\nfunc (d *DataDownloadBuilder) SnapshotID(id string) *DataDownloadBuilder {\n\td.object.Spec.SnapshotID = id\n\treturn d\n}\n\n// DataMover sets the DataDownload's DataMover.\nfunc (d *DataDownloadBuilder) DataMover(dataMover string) *DataDownloadBuilder {\n\td.object.Spec.DataMover = dataMover\n\treturn d\n}\n\n// SourceNamespace sets the DataDownload's SourceNamespace.\nfunc (d *DataDownloadBuilder) SourceNamespace(sourceNamespace string) *DataDownloadBuilder {\n\td.object.Spec.SourceNamespace = sourceNamespace\n\treturn d\n}\n\n// TargetVolume sets the DataDownload's TargetVolume.\nfunc (d *DataDownloadBuilder) TargetVolume(targetVolume velerov2alpha1api.TargetVolumeSpec) *DataDownloadBuilder {\n\td.object.Spec.TargetVolume = targetVolume\n\treturn d\n}\n\n// Cancel sets the DataDownload's Cancel.\nfunc (d *DataDownloadBuilder) Cancel(cancel bool) *DataDownloadBuilder {\n\td.object.Spec.Cancel = cancel\n\treturn d\n}\n\n// OperationTimeout sets the DataDownload's OperationTimeout.\nfunc (d *DataDownloadBuilder) OperationTimeout(timeout metav1.Duration) *DataDownloadBuilder {\n\td.object.Spec.OperationTimeout = timeout\n\treturn d\n}\n\n// DataMoverConfig sets the DataDownload's DataMoverConfig.\nfunc (d *DataDownloadBuilder) DataMoverConfig(config *map[string]string) *DataDownloadBuilder {\n\td.object.Spec.DataMoverConfig = *config\n\treturn d\n}\n\n// ObjectMeta applies functional options to the DataDownload's ObjectMeta.\nfunc (d *DataDownloadBuilder) ObjectMeta(opts ...ObjectMetaOpt) *DataDownloadBuilder {\n\tfor _, opt := range opts {\n\t\topt(d.object)\n\t}\n\n\treturn d\n}\n\n// Labels sets the DataDownload's Labels.\nfunc (d *DataDownloadBuilder) Labels(labels map[string]string) *DataDownloadBuilder {\n\td.object.Labels = labels\n\treturn d\n}\n\n// Annotations sets the DataDownload's Annotations.\nfunc (d *DataDownloadBuilder) Annotations(annotations map[string]string) *DataDownloadBuilder {\n\td.object.Annotations = annotations\n\treturn d\n}\n\n// StartTimestamp sets the DataDownload's StartTimestamp.\nfunc (d *DataDownloadBuilder) StartTimestamp(startTime *metav1.Time) *DataDownloadBuilder {\n\td.object.Status.StartTimestamp = startTime\n\treturn d\n}\n\n// CompletionTimestamp sets the DataDownload's StartTimestamp.\nfunc (d *DataDownloadBuilder) CompletionTimestamp(completionTimestamp *metav1.Time) *DataDownloadBuilder {\n\td.object.Status.CompletionTimestamp = completionTimestamp\n\treturn d\n}\n\n// Progress sets the DataDownload's Progress.\nfunc (d *DataDownloadBuilder) Progress(progress shared.DataMoveOperationProgress) *DataDownloadBuilder {\n\td.object.Status.Progress = progress\n\treturn d\n}\n\n// Node sets the DataDownload's Node.\nfunc (d *DataDownloadBuilder) Node(node string) *DataDownloadBuilder {\n\td.object.Status.Node = node\n\treturn d\n}\n\n// NodeOS sets the DataDownload's Node OS.\nfunc (d *DataDownloadBuilder) NodeOS(nodeOS velerov2alpha1api.NodeOS) *DataDownloadBuilder {\n\td.object.Spec.NodeOS = nodeOS\n\treturn d\n}\n\n// AcceptedByNode sets the DataDownload's AcceptedByNode.\nfunc (d *DataDownloadBuilder) AcceptedByNode(node string) *DataDownloadBuilder {\n\td.object.Status.AcceptedByNode = node\n\treturn d\n}\n\n// AcceptedTimestamp sets the DataDownload's AcceptedTimestamp.\nfunc (d *DataDownloadBuilder) AcceptedTimestamp(acceptedTimestamp *metav1.Time) *DataDownloadBuilder {\n\td.object.Status.AcceptedTimestamp = acceptedTimestamp\n\treturn d\n}\n\n// Finalizers sets the DataDownload's Finalizers.\nfunc (d *DataDownloadBuilder) Finalizers(finalizers []string) *DataDownloadBuilder {\n\td.object.Finalizers = finalizers\n\treturn d\n}\n\n// Message sets the DataDownload's Message.\nfunc (d *DataDownloadBuilder) Message(msg string) *DataDownloadBuilder {\n\td.object.Status.Message = msg\n\treturn d\n}\n" }, { "path": "pkg/builder/data_upload_builder.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/apis/velero/shared\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n)\n\n// DataUploadBuilder builds DataUpload objects\ntype DataUploadBuilder struct {\n\tobject *velerov2alpha1api.DataUpload\n}\n\n// ForDataUpload is the constructor for a DataUploadBuilder.\nfunc ForDataUpload(ns, name string) *DataUploadBuilder {\n\treturn &DataUploadBuilder{\n\t\tobject: &velerov2alpha1api.DataUpload{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: velerov2alpha1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"DataUpload\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built DataUpload.\nfunc (d *DataUploadBuilder) Result() *velerov2alpha1api.DataUpload {\n\treturn d.object\n}\n\n// BackupStorageLocation sets the DataUpload's backup storage location.\nfunc (d *DataUploadBuilder) BackupStorageLocation(name string) *DataUploadBuilder {\n\td.object.Spec.BackupStorageLocation = name\n\treturn d\n}\n\n// Phase sets the DataUpload's phase.\nfunc (d *DataUploadBuilder) Phase(phase velerov2alpha1api.DataUploadPhase) *DataUploadBuilder {\n\td.object.Status.Phase = phase\n\treturn d\n}\n\n// SnapshotID sets the DataUpload's SnapshotID.\nfunc (d *DataUploadBuilder) SnapshotID(id string) *DataUploadBuilder {\n\td.object.Status.SnapshotID = id\n\treturn d\n}\n\n// DataMover sets the DataUpload's DataMover.\nfunc (d *DataUploadBuilder) DataMover(dataMover string) *DataUploadBuilder {\n\td.object.Spec.DataMover = dataMover\n\treturn d\n}\n\n// SourceNamespace sets the DataUpload's SourceNamespace.\nfunc (d *DataUploadBuilder) SourceNamespace(sourceNamespace string) *DataUploadBuilder {\n\td.object.Spec.SourceNamespace = sourceNamespace\n\treturn d\n}\n\n// SourcePVC sets the DataUpload's SourcePVC.\nfunc (d *DataUploadBuilder) SourcePVC(sourcePVC string) *DataUploadBuilder {\n\td.object.Spec.SourcePVC = sourcePVC\n\treturn d\n}\n\n// SnapshotType sets the DataUpload's SnapshotType.\nfunc (d *DataUploadBuilder) SnapshotType(SnapshotType velerov2alpha1api.SnapshotType) *DataUploadBuilder {\n\td.object.Spec.SnapshotType = SnapshotType\n\treturn d\n}\n\n// Cancel sets the DataUpload's Cancel.\nfunc (d *DataUploadBuilder) Cancel(cancel bool) *DataUploadBuilder {\n\td.object.Spec.Cancel = cancel\n\treturn d\n}\n\n// OperationTimeout sets the DataUpload's OperationTimeout.\nfunc (d *DataUploadBuilder) OperationTimeout(timeout metav1.Duration) *DataUploadBuilder {\n\td.object.Spec.OperationTimeout = timeout\n\treturn d\n}\n\n// DataMoverConfig sets the DataUpload's DataMoverConfig.\nfunc (d *DataUploadBuilder) DataMoverConfig(config map[string]string) *DataUploadBuilder {\n\td.object.Spec.DataMoverConfig = config\n\treturn d\n}\n\n// CSISnapshot sets the DataUpload's CSISnapshot.\nfunc (d *DataUploadBuilder) CSISnapshot(cSISnapshot *velerov2alpha1api.CSISnapshotSpec) *DataUploadBuilder {\n\td.object.Spec.CSISnapshot = cSISnapshot\n\treturn d\n}\n\n// StartTimestamp sets the DataUpload's StartTimestamp.\nfunc (d *DataUploadBuilder) StartTimestamp(startTimestamp *metav1.Time) *DataUploadBuilder {\n\td.object.Status.StartTimestamp = startTimestamp\n\treturn d\n}\n\n// CompletionTimestamp sets the DataUpload's StartTimestamp.\nfunc (d *DataUploadBuilder) CompletionTimestamp(completionTimestamp *metav1.Time) *DataUploadBuilder {\n\td.object.Status.CompletionTimestamp = completionTimestamp\n\treturn d\n}\n\n// Labels sets the DataUpload's Labels.\nfunc (d *DataUploadBuilder) Labels(labels map[string]string) *DataUploadBuilder {\n\td.object.Labels = labels\n\treturn d\n}\n\n// Annotations sets the DataUpload's Annotations.\nfunc (d *DataUploadBuilder) Annotations(annotations map[string]string) *DataUploadBuilder {\n\td.object.Annotations = annotations\n\treturn d\n}\n\n// Progress sets the DataUpload's Progress.\nfunc (d *DataUploadBuilder) Progress(progress shared.DataMoveOperationProgress) *DataUploadBuilder {\n\td.object.Status.Progress = progress\n\treturn d\n}\n\n// IncrementalBytes sets the DataUpload's IncrementalBytes.\nfunc (d *DataUploadBuilder) IncrementalBytes(incrementalBytes int64) *DataUploadBuilder {\n\td.object.Status.IncrementalBytes = incrementalBytes\n\treturn d\n}\n\n// Node sets the DataUpload's Node.\nfunc (d *DataUploadBuilder) Node(node string) *DataUploadBuilder {\n\td.object.Status.Node = node\n\treturn d\n}\n\n// NodeOS sets the DataUpload's Node OS.\nfunc (d *DataUploadBuilder) NodeOS(nodeOS velerov2alpha1api.NodeOS) *DataUploadBuilder {\n\td.object.Status.NodeOS = nodeOS\n\treturn d\n}\n\n// AcceptedByNode sets the DataUpload's AcceptedByNode.\nfunc (d *DataUploadBuilder) AcceptedByNode(node string) *DataUploadBuilder {\n\td.object.Status.AcceptedByNode = node\n\treturn d\n}\n\n// AcceptedTimestamp sets the DataUpload's AcceptedTimestamp.\nfunc (d *DataUploadBuilder) AcceptedTimestamp(acceptedTimestamp *metav1.Time) *DataUploadBuilder {\n\td.object.Status.AcceptedTimestamp = acceptedTimestamp\n\treturn d\n}\n\n// Finalizers sets the DataUpload's Finalizers.\nfunc (d *DataUploadBuilder) Finalizers(finalizers []string) *DataUploadBuilder {\n\td.object.Finalizers = finalizers\n\treturn d\n}\n\n// Message sets the DataUpload's Message.\nfunc (d *DataUploadBuilder) Message(msg string) *DataUploadBuilder {\n\td.object.Status.Message = msg\n\treturn d\n}\n\n// TotalBytes sets the DataUpload's TotalBytes.\nfunc (d *DataUploadBuilder) TotalBytes(size int64) *DataUploadBuilder {\n\td.object.Status.Progress.TotalBytes = size\n\treturn d\n}\n" }, { "path": "pkg/builder/delete_backup_request_builder.go", "content": "/*\nCopyright 2023 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// DeleteBackupRequestBuilder builds DeleteBackupRequest objects\ntype DeleteBackupRequestBuilder struct {\n\tobject *velerov1api.DeleteBackupRequest\n}\n\n// ForDeleteBackupRequest is the constructor for a DeleteBackupRequestBuilder.\nfunc ForDeleteBackupRequest(ns, name string) *DeleteBackupRequestBuilder {\n\treturn &DeleteBackupRequestBuilder{\n\t\tobject: &velerov1api.DeleteBackupRequest{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"DeleteBackupRequest\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built DeleteBackupRequest.\nfunc (b *DeleteBackupRequestBuilder) Result() *velerov1api.DeleteBackupRequest {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the DeleteBackupRequest's ObjectMeta.\nfunc (b *DeleteBackupRequestBuilder) ObjectMeta(opts ...ObjectMetaOpt) *DeleteBackupRequestBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\treturn b\n}\n\n// BackupName sets the DeleteBackupRequest's backup name.\nfunc (b *DeleteBackupRequestBuilder) BackupName(name string) *DeleteBackupRequestBuilder {\n\tb.object.Spec.BackupName = name\n\treturn b\n}\n\n// Phase sets the DeleteBackupRequest's phase.\nfunc (b *DeleteBackupRequestBuilder) Phase(phase velerov1api.DeleteBackupRequestPhase) *DeleteBackupRequestBuilder {\n\tb.object.Status.Phase = phase\n\treturn b\n}\n\n// Errors sets the DeleteBackupRequest's errors.\nfunc (b *DeleteBackupRequestBuilder) Errors(errors ...string) *DeleteBackupRequestBuilder {\n\tb.object.Status.Errors = errors\n\treturn b\n}\n" }, { "path": "pkg/builder/deployment_builder.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// DeploymentBuilder builds Deployment objects.\ntype DeploymentBuilder struct {\n\tobject *appsv1api.Deployment\n}\n\n// ForDeployment is the constructor for a DeploymentBuilder.\nfunc ForDeployment(ns, name string) *DeploymentBuilder {\n\treturn &DeploymentBuilder{\n\t\tobject: &appsv1api.Deployment{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: appsv1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"Deployment\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built Deployment.\nfunc (b *DeploymentBuilder) Result() *appsv1api.Deployment {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the Deployment's ObjectMeta.\nfunc (b *DeploymentBuilder) ObjectMeta(opts ...ObjectMetaOpt) *DeploymentBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\n\treturn b\n}\n" }, { "path": "pkg/builder/download_request_builder.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// DownloadRequestBuilder builds DownloadRequest objects.\ntype DownloadRequestBuilder struct {\n\tobject *velerov1api.DownloadRequest\n}\n\n// ForDownloadRequest is the constructor for a DownloadRequestBuilder.\nfunc ForDownloadRequest(ns, name string) *DownloadRequestBuilder {\n\treturn &DownloadRequestBuilder{\n\t\tobject: &velerov1api.DownloadRequest{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"DownloadRequest\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built DownloadRequest.\nfunc (b *DownloadRequestBuilder) Result() *velerov1api.DownloadRequest {\n\treturn b.object\n}\n\n// Phase sets the DownloadRequest's status phase.\nfunc (b *DownloadRequestBuilder) Phase(phase velerov1api.DownloadRequestPhase) *DownloadRequestBuilder {\n\tb.object.Status.Phase = phase\n\treturn b\n}\n\n// Target sets the DownloadRequest's target kind and target name.\nfunc (b *DownloadRequestBuilder) Target(targetKind velerov1api.DownloadTargetKind, targetName string) *DownloadRequestBuilder {\n\tb.object.Spec.Target.Kind = targetKind\n\tb.object.Spec.Target.Name = targetName\n\treturn b\n}\n" }, { "path": "pkg/builder/item_operation_builder.go", "content": "/*\nCopyright 2023 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\t\"time\"\n\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// OperationStatusBuilder builds OperationStatus objects\ntype OperationStatusBuilder struct {\n\tobject *itemoperation.OperationStatus\n}\n\n// ForOperationStatus is the constructor for a OperationStatusBuilder.\nfunc ForOperationStatus() *OperationStatusBuilder {\n\treturn &OperationStatusBuilder{\n\t\tobject: &itemoperation.OperationStatus{},\n\t}\n}\n\n// Result returns the built OperationStatus.\nfunc (osb *OperationStatusBuilder) Result() *itemoperation.OperationStatus {\n\treturn osb.object\n}\n\n// Phase sets the OperationStatus's phase.\nfunc (osb *OperationStatusBuilder) Phase(phase itemoperation.OperationPhase) *OperationStatusBuilder {\n\tosb.object.Phase = phase\n\treturn osb\n}\n\n// Error sets the OperationStatus's error.\nfunc (osb *OperationStatusBuilder) Error(err string) *OperationStatusBuilder {\n\tosb.object.Error = err\n\treturn osb\n}\n\n// Progress sets the OperationStatus's progress.\nfunc (osb *OperationStatusBuilder) Progress(nComplete int64, nTotal int64, operationUnits string) *OperationStatusBuilder {\n\tosb.object.NCompleted = nComplete\n\tosb.object.NTotal = nTotal\n\tosb.object.OperationUnits = operationUnits\n\treturn osb\n}\n\n// Description sets the OperationStatus's description.\nfunc (osb *OperationStatusBuilder) Description(desc string) *OperationStatusBuilder {\n\tosb.object.Description = desc\n\treturn osb\n}\n\n// Created sets the OperationStatus's creation timestamp.\nfunc (osb *OperationStatusBuilder) Created(t time.Time) *OperationStatusBuilder {\n\tosb.object.Created = &metav1.Time{Time: t}\n\treturn osb\n}\n\n// Updated sets the OperationStatus's last update timestamp.\nfunc (osb *OperationStatusBuilder) Updated(t time.Time) *OperationStatusBuilder {\n\tosb.object.Updated = &metav1.Time{Time: t}\n\treturn osb\n}\n\n// Started sets the OperationStatus's start timestamp.\nfunc (osb *OperationStatusBuilder) Started(t time.Time) *OperationStatusBuilder {\n\tosb.object.Started = &metav1.Time{Time: t}\n\treturn osb\n}\n\n// BackupOperationBuilder builds BackupOperation objects\ntype BackupOperationBuilder struct {\n\tobject *itemoperation.BackupOperation\n}\n\n// ForBackupOperation is the constructor for a BackupOperationBuilder.\nfunc ForBackupOperation() *BackupOperationBuilder {\n\treturn &BackupOperationBuilder{\n\t\tobject: &itemoperation.BackupOperation{},\n\t}\n}\n\n// Result returns the built BackupOperation.\nfunc (bb *BackupOperationBuilder) Result() *itemoperation.BackupOperation {\n\treturn bb.object\n}\n\n// BackupName sets the BackupOperation's backup name.\nfunc (bb *BackupOperationBuilder) BackupName(name string) *BackupOperationBuilder {\n\tbb.object.Spec.BackupName = name\n\treturn bb\n}\n\n// OperationID sets the BackupOperation's operation ID.\nfunc (bb *BackupOperationBuilder) OperationID(id string) *BackupOperationBuilder {\n\tbb.object.Spec.OperationID = id\n\treturn bb\n}\n\n// Status sets the BackupOperation's status.\nfunc (bb *BackupOperationBuilder) Status(status itemoperation.OperationStatus) *BackupOperationBuilder {\n\tbb.object.Status = status\n\treturn bb\n}\n\n// ResourceIdentifier sets the BackupOperation's resource identifier.\nfunc (bb *BackupOperationBuilder) ResourceIdentifier(group, resource, ns, name string) *BackupOperationBuilder {\n\tbb.object.Spec.ResourceIdentifier = velero.ResourceIdentifier{\n\t\tGroupResource: schema.GroupResource{\n\t\t\tGroup: group,\n\t\t\tResource: resource,\n\t\t},\n\t\tNamespace: ns,\n\t\tName: name,\n\t}\n\treturn bb\n}\n\n// BackupItemAction sets the BackupOperation's backup item action.\nfunc (bb *BackupOperationBuilder) BackupItemAction(bia string) *BackupOperationBuilder {\n\tbb.object.Spec.BackupItemAction = bia\n\treturn bb\n}\n\n// PostOperationItem adds a post-operation item to the BackupOperation's list of post-operation items.\nfunc (bb *BackupOperationBuilder) PostOperationItem(group, resource, ns, name string) *BackupOperationBuilder {\n\tbb.object.Spec.PostOperationItems = append(bb.object.Spec.PostOperationItems, velero.ResourceIdentifier{\n\t\tGroupResource: schema.GroupResource{\n\t\t\tGroup: group,\n\t\t\tResource: resource,\n\t\t},\n\t\tNamespace: ns,\n\t\tName: name,\n\t})\n\treturn bb\n}\n\n// RestoreOperationBuilder builds RestoreOperation objects\ntype RestoreOperationBuilder struct {\n\tobject *itemoperation.RestoreOperation\n}\n\n// ForRestoreOperation is the constructor for a RestoreOperationBuilder.\nfunc ForRestoreOperation() *RestoreOperationBuilder {\n\treturn &RestoreOperationBuilder{\n\t\tobject: &itemoperation.RestoreOperation{},\n\t}\n}\n\n// Result returns the built RestoreOperation.\nfunc (rb *RestoreOperationBuilder) Result() *itemoperation.RestoreOperation {\n\treturn rb.object\n}\n\n// RestoreName sets the RestoreOperation's restore name.\nfunc (rb *RestoreOperationBuilder) RestoreName(name string) *RestoreOperationBuilder {\n\trb.object.Spec.RestoreName = name\n\treturn rb\n}\n\n// OperationID sets the RestoreOperation's operation ID.\nfunc (rb *RestoreOperationBuilder) OperationID(id string) *RestoreOperationBuilder {\n\trb.object.Spec.OperationID = id\n\treturn rb\n}\n\n// RestoreItemAction sets the RestoreOperation's restore item action.\nfunc (rb *RestoreOperationBuilder) RestoreItemAction(ria string) *RestoreOperationBuilder {\n\trb.object.Spec.RestoreItemAction = ria\n\treturn rb\n}\n\n// Status sets the RestoreOperation's status.\nfunc (rb *RestoreOperationBuilder) Status(status itemoperation.OperationStatus) *RestoreOperationBuilder {\n\trb.object.Status = status\n\treturn rb\n}\n\n// ResourceIdentifier sets the RestoreOperation's resource identifier.\nfunc (rb *RestoreOperationBuilder) ResourceIdentifier(group, resource, ns, name string) *RestoreOperationBuilder {\n\trb.object.Spec.ResourceIdentifier = velero.ResourceIdentifier{\n\t\tGroupResource: schema.GroupResource{\n\t\t\tGroup: group,\n\t\t\tResource: resource,\n\t\t},\n\t\tNamespace: ns,\n\t\tName: name,\n\t}\n\treturn rb\n}\n" }, { "path": "pkg/builder/job_builder.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tbatchv1api \"k8s.io/api/batch/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\ntype JobBuilder struct {\n\tobject *batchv1api.Job\n}\n\n// ForJob is the constructor for a JobBuilder.\nfunc ForJob(ns, name string) *JobBuilder {\n\treturn &JobBuilder{\n\t\tobject: &batchv1api.Job{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: batchv1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"Job\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built Job.\nfunc (j *JobBuilder) Result() *batchv1api.Job {\n\treturn j.object\n}\n\n// ObjectMeta applies functional options to the Job's ObjectMeta.\nfunc (j *JobBuilder) ObjectMeta(opts ...ObjectMetaOpt) *JobBuilder {\n\tfor _, opt := range opts {\n\t\topt(j.object)\n\t}\n\n\treturn j\n}\n\n// Succeeded sets Succeeded on the Job's Status.\nfunc (j *JobBuilder) Succeeded(succeeded int) *JobBuilder {\n\tj.object.Status.Succeeded = int32(succeeded)\n\treturn j\n}\n" }, { "path": "pkg/builder/json_schema_props_builder.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage builder\n\nimport (\n\tapiextv1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1\"\n)\n\n// JSONSchemaPropsBuilder builds JSONSchemaProps objects.\ntype JSONSchemaPropsBuilder struct {\n\tobject *apiextv1.JSONSchemaProps\n}\n\n// ForJSONSchemaPropsBuilder is the constructor for a JSONSchemaPropsBuilder.\nfunc ForJSONSchemaPropsBuilder() *JSONSchemaPropsBuilder {\n\treturn &JSONSchemaPropsBuilder{\n\t\tobject: &apiextv1.JSONSchemaProps{},\n\t}\n}\n\n// Maximum sets the Maximum field on a JSONSchemaPropsBuilder object.\nfunc (b *JSONSchemaPropsBuilder) Maximum(f float64) *JSONSchemaPropsBuilder {\n\tb.object.Maximum = &f\n\treturn b\n}\n\n// Result returns the built object.\nfunc (b *JSONSchemaPropsBuilder) Result() *apiextv1.JSONSchemaProps {\n\treturn b.object\n}\n" }, { "path": "pkg/builder/namespace_builder.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// NamespaceBuilder builds Namespace objects.\ntype NamespaceBuilder struct {\n\tobject *corev1api.Namespace\n}\n\n// ForNamespace is the constructor for a NamespaceBuilder.\nfunc ForNamespace(name string) *NamespaceBuilder {\n\treturn &NamespaceBuilder{\n\t\tobject: &corev1api.Namespace{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: corev1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"Namespace\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built Namespace.\nfunc (b *NamespaceBuilder) Result() *corev1api.Namespace {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the Namespace's ObjectMeta.\nfunc (b *NamespaceBuilder) ObjectMeta(opts ...ObjectMetaOpt) *NamespaceBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\n\treturn b\n}\n\n// Phase sets the namespace's phase\nfunc (b *NamespaceBuilder) Phase(val corev1api.NamespacePhase) *NamespaceBuilder {\n\tb.object.Status.Phase = val\n\treturn b\n}\n" }, { "path": "pkg/builder/node_builder.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// NodeBuilder builds Node objects.\ntype NodeBuilder struct {\n\tobject *corev1api.Node\n}\n\n// ForNode is the constructor for a NodeBuilder.\nfunc ForNode(name string) *NodeBuilder {\n\treturn &NodeBuilder{\n\t\tobject: &corev1api.Node{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: corev1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"Node\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\nfunc (b *NodeBuilder) Labels(labels map[string]string) *NodeBuilder {\n\tb.object.Labels = labels\n\treturn b\n}\n\n// Result returns the built Node.\nfunc (b *NodeBuilder) Result() *corev1api.Node {\n\treturn b.object\n}\n" }, { "path": "pkg/builder/node_selector_builder.go", "content": "/*\nCopyright 2023 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport corev1api \"k8s.io/api/core/v1\"\n\n// NodeSelectorBuilder builds NodeSelector objects\ntype NodeSelectorBuilder struct {\n\tobject *corev1api.NodeSelector\n}\n\n// ForNodeSelector returns the NodeSelectorBuilder instance with given terms\nfunc ForNodeSelector(term ...corev1api.NodeSelectorTerm) *NodeSelectorBuilder {\n\treturn &NodeSelectorBuilder{\n\t\tobject: &corev1api.NodeSelector{\n\t\t\tNodeSelectorTerms: term,\n\t\t},\n\t}\n}\n\n// Result returns the built NodeSelector\nfunc (b *NodeSelectorBuilder) Result() *corev1api.NodeSelector {\n\treturn b.object\n}\n\n// NodeSelectorTermBuilder builds NodeSelectorTerm objects.\ntype NodeSelectorTermBuilder struct {\n\tobject *corev1api.NodeSelectorTerm\n}\n\n// NewNodeSelectorTermBuilder initializes an instance of NodeSelectorTermBuilder\nfunc NewNodeSelectorTermBuilder() *NodeSelectorTermBuilder {\n\treturn &NodeSelectorTermBuilder{\n\t\tobject: &corev1api.NodeSelectorTerm{\n\t\t\tMatchExpressions: make([]corev1api.NodeSelectorRequirement, 0),\n\t\t\tMatchFields: make([]corev1api.NodeSelectorRequirement, 0),\n\t\t},\n\t}\n}\n\n// WithMatchExpression appends the MatchExpression to the NodeSelectorTerm\nfunc (ntb *NodeSelectorTermBuilder) WithMatchExpression(key string, op string, values ...string) *NodeSelectorTermBuilder {\n\treq := corev1api.NodeSelectorRequirement{\n\t\tKey: key,\n\t\tOperator: corev1api.NodeSelectorOperator(op),\n\t\tValues: values,\n\t}\n\tntb.object.MatchExpressions = append(ntb.object.MatchExpressions, req)\n\treturn ntb\n}\n\n// WithMatchField appends the MatchField to the NodeSelectorTerm\nfunc (ntb *NodeSelectorTermBuilder) WithMatchField(key string, op string, values ...string) *NodeSelectorTermBuilder {\n\treq := corev1api.NodeSelectorRequirement{\n\t\tKey: key,\n\t\tOperator: corev1api.NodeSelectorOperator(op),\n\t\tValues: values,\n\t}\n\tntb.object.MatchFields = append(ntb.object.MatchFields, req)\n\treturn ntb\n}\n\n// Result returns the built NodeSelectorTerm\nfunc (ntb *NodeSelectorTermBuilder) Result() *corev1api.NodeSelectorTerm {\n\treturn ntb.object\n}\n" }, { "path": "pkg/builder/object_meta.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\t\"time\"\n\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n)\n\n// ObjectMetaOpt is a functional option for ObjectMeta.\ntype ObjectMetaOpt func(metav1.Object)\n\n// WithName is a functional option that applies the specified\n// name to an object.\nfunc WithName(val string) func(obj metav1.Object) {\n\treturn func(obj metav1.Object) {\n\t\tobj.SetName(val)\n\t}\n}\n\n// WithResourceVersion is a functional option that applies the specified\n// resourceVersion to an object\nfunc WithResourceVersion(val string) func(obj metav1.Object) {\n\treturn func(obj metav1.Object) {\n\t\tobj.SetResourceVersion(val)\n\t}\n}\n\n// WithLabels is a functional option that applies the specified\n// label keys/values to an object.\nfunc WithLabels(vals ...string) func(obj metav1.Object) {\n\treturn func(obj metav1.Object) {\n\t\tobj.SetLabels(setMapEntries(obj.GetLabels(), vals...))\n\t}\n}\n\n// WithLabelsMap is a functional option that applies the specified labels map to\n// an object.\nfunc WithLabelsMap(labels map[string]string) func(obj metav1.Object) {\n\treturn func(obj metav1.Object) {\n\t\tobjLabels := obj.GetLabels()\n\t\tif objLabels == nil {\n\t\t\tobjLabels = make(map[string]string)\n\t\t}\n\n\t\t// If the label already exists in the object, it will be overwritten\n\t\tfor k, v := range labels {\n\t\t\tobjLabels[k] = v\n\t\t}\n\n\t\tobj.SetLabels(objLabels)\n\t}\n}\n\n// WithAnnotations is a functional option that applies the specified\n// annotation keys/values to an object.\nfunc WithAnnotations(vals ...string) func(obj metav1.Object) {\n\treturn func(obj metav1.Object) {\n\t\tobj.SetAnnotations(setMapEntries(obj.GetAnnotations(), vals...))\n\t}\n}\n\n// WithAnnotationsMap is a functional option that applies the specified annotations map to\n// an object.\nfunc WithAnnotationsMap(annotations map[string]string) func(obj metav1.Object) {\n\treturn func(obj metav1.Object) {\n\t\tobjAnnotations := obj.GetAnnotations()\n\t\tif objAnnotations == nil {\n\t\t\tobjAnnotations = make(map[string]string)\n\t\t}\n\n\t\t// If the label already exists in the object, it will be overwritten\n\t\tfor k, v := range annotations {\n\t\t\tobjAnnotations[k] = v\n\t\t}\n\n\t\tobj.SetAnnotations(objAnnotations)\n\t}\n}\n\nfunc setMapEntries(m map[string]string, vals ...string) map[string]string {\n\tif m == nil {\n\t\tm = make(map[string]string)\n\t}\n\n\t// if we don't have a value for every key, add an empty\n\t// string at the end to serve as the value for the last\n\t// key.\n\tif len(vals)%2 != 0 {\n\t\tvals = append(vals, \"\")\n\t}\n\n\tfor i := 0; i < len(vals); i += 2 {\n\t\tkey := vals[i]\n\t\tval := vals[i+1]\n\n\t\t// If the label already exists in the object, it will be overwritten\n\t\tm[key] = val\n\t}\n\n\treturn m\n}\n\n// WithFinalizers is a functional option that applies the specified\n// finalizers to an object.\nfunc WithFinalizers(vals ...string) func(obj metav1.Object) {\n\treturn func(obj metav1.Object) {\n\t\tobj.SetFinalizers(vals)\n\t}\n}\n\n// WithDeletionTimestamp is a functional option that applies the specified\n// deletion timestamp to an object.\nfunc WithDeletionTimestamp(val time.Time) func(obj metav1.Object) {\n\treturn func(obj metav1.Object) {\n\t\tobj.SetDeletionTimestamp(&metav1.Time{Time: val})\n\t}\n}\n\n// WithUID is a functional option that applies the specified UID to an object.\nfunc WithUID(val string) func(obj metav1.Object) {\n\treturn func(obj metav1.Object) {\n\t\tobj.SetUID(types.UID(val))\n\t}\n}\n\n// WithGenerateName is a functional option that applies the specified generate name to an object.\nfunc WithGenerateName(val string) func(obj metav1.Object) {\n\treturn func(obj metav1.Object) {\n\t\tobj.SetGenerateName(val)\n\t}\n}\n\n// WithManagedFields is a functional option that applies the specified managed fields to an object.\nfunc WithManagedFields(val []metav1.ManagedFieldsEntry) func(obj metav1.Object) {\n\treturn func(obj metav1.Object) {\n\t\tobj.SetManagedFields(val)\n\t}\n}\n\n// WithCreationTimestamp is a functional option that applies the specified creationTimestamp\nfunc WithCreationTimestamp(t time.Time) func(obj metav1.Object) {\n\treturn func(obj metav1.Object) {\n\t\tobj.SetCreationTimestamp(metav1.Time{Time: t})\n\t}\n}\n\n// WithOwnerReference is a functional option that applies the specified OwnerReference to an object.\nfunc WithOwnerReference(val []metav1.OwnerReference) func(obj metav1.Object) {\n\treturn func(obj metav1.Object) {\n\t\tobj.SetOwnerReferences(val)\n\t}\n}\n" }, { "path": "pkg/builder/persistent_volume_builder.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// PersistentVolumeBuilder builds PersistentVolume objects.\ntype PersistentVolumeBuilder struct {\n\tobject *corev1api.PersistentVolume\n}\n\n// ForPersistentVolume is the constructor for a PersistentVolumeBuilder.\nfunc ForPersistentVolume(name string) *PersistentVolumeBuilder {\n\treturn &PersistentVolumeBuilder{\n\t\tobject: &corev1api.PersistentVolume{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: corev1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"PersistentVolume\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built PersistentVolume.\nfunc (b *PersistentVolumeBuilder) Result() *corev1api.PersistentVolume {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the PersistentVolume's ObjectMeta.\nfunc (b *PersistentVolumeBuilder) ObjectMeta(opts ...ObjectMetaOpt) *PersistentVolumeBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\n\treturn b\n}\n\n// ReclaimPolicy sets the PersistentVolume's reclaim policy.\nfunc (b *PersistentVolumeBuilder) ReclaimPolicy(policy corev1api.PersistentVolumeReclaimPolicy) *PersistentVolumeBuilder {\n\tb.object.Spec.PersistentVolumeReclaimPolicy = policy\n\treturn b\n}\n\n// ClaimRef sets the PersistentVolume's claim ref.\nfunc (b *PersistentVolumeBuilder) ClaimRef(ns, name string) *PersistentVolumeBuilder {\n\tb.object.Spec.ClaimRef = &corev1api.ObjectReference{\n\t\tNamespace: ns,\n\t\tName: name,\n\t}\n\treturn b\n}\n\n// AWSEBSVolumeID sets the PersistentVolume's AWSElasticBlockStore volume ID.\nfunc (b *PersistentVolumeBuilder) AWSEBSVolumeID(volumeID string) *PersistentVolumeBuilder {\n\tif b.object.Spec.AWSElasticBlockStore == nil {\n\t\tb.object.Spec.AWSElasticBlockStore = new(corev1api.AWSElasticBlockStoreVolumeSource)\n\t}\n\tb.object.Spec.AWSElasticBlockStore.VolumeID = volumeID\n\treturn b\n}\n\n// CSI sets the PersistentVolume's CSI.\nfunc (b *PersistentVolumeBuilder) CSI(driver, volumeHandle string) *PersistentVolumeBuilder {\n\tif b.object.Spec.CSI == nil {\n\t\tb.object.Spec.CSI = new(corev1api.CSIPersistentVolumeSource)\n\t}\n\tb.object.Spec.CSI.Driver = driver\n\tb.object.Spec.CSI.VolumeHandle = volumeHandle\n\treturn b\n}\n\n// StorageClass sets the PersistentVolume's storage class name.\nfunc (b *PersistentVolumeBuilder) StorageClass(name string) *PersistentVolumeBuilder {\n\tb.object.Spec.StorageClassName = name\n\treturn b\n}\n\n// VolumeMode sets the PersistentVolume's volume mode.\nfunc (b *PersistentVolumeBuilder) VolumeMode(volMode corev1api.PersistentVolumeMode) *PersistentVolumeBuilder {\n\tb.object.Spec.VolumeMode = &volMode\n\treturn b\n}\n\n// NodeAffinityRequired sets the PersistentVolume's NodeAffinity Requirement.\nfunc (b *PersistentVolumeBuilder) NodeAffinityRequired(req *corev1api.NodeSelector) *PersistentVolumeBuilder {\n\tb.object.Spec.NodeAffinity = &corev1api.VolumeNodeAffinity{\n\t\tRequired: req,\n\t}\n\treturn b\n}\n\n// Phase sets the PersistentVolume's phase.\nfunc (b *PersistentVolumeBuilder) Phase(phase corev1api.PersistentVolumePhase) *PersistentVolumeBuilder {\n\tb.object.Status.Phase = phase\n\treturn b\n}\n" }, { "path": "pkg/builder/persistent_volume_claim_builder.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/api/resource\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// PersistentVolumeClaimBuilder builds PersistentVolumeClaim objects.\ntype PersistentVolumeClaimBuilder struct {\n\tobject *corev1api.PersistentVolumeClaim\n}\n\n// ForPersistentVolumeClaim is the constructor for a PersistentVolumeClaimBuilder.\nfunc ForPersistentVolumeClaim(ns, name string) *PersistentVolumeClaimBuilder {\n\treturn &PersistentVolumeClaimBuilder{\n\t\tobject: &corev1api.PersistentVolumeClaim{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: corev1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"PersistentVolumeClaim\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built PersistentVolumeClaim.\nfunc (b *PersistentVolumeClaimBuilder) Result() *corev1api.PersistentVolumeClaim {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the PersistentVolumeClaim's ObjectMeta.\nfunc (b *PersistentVolumeClaimBuilder) ObjectMeta(opts ...ObjectMetaOpt) *PersistentVolumeClaimBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\n\treturn b\n}\n\n// VolumeName sets the PersistentVolumeClaim's volume name.\nfunc (b *PersistentVolumeClaimBuilder) VolumeName(name string) *PersistentVolumeClaimBuilder {\n\tb.object.Spec.VolumeName = name\n\treturn b\n}\n\n// StorageClass sets the PersistentVolumeClaim's storage class name.\nfunc (b *PersistentVolumeClaimBuilder) StorageClass(name string) *PersistentVolumeClaimBuilder {\n\tb.object.Spec.StorageClassName = &name\n\treturn b\n}\n\n// Phase sets the PersistentVolumeClaim's status Phase.\nfunc (b *PersistentVolumeClaimBuilder) Phase(phase corev1api.PersistentVolumeClaimPhase) *PersistentVolumeClaimBuilder {\n\tb.object.Status.Phase = phase\n\treturn b\n}\n\n// RequestResource sets the PersistentVolumeClaim's spec.Resources.Requests.\nfunc (b *PersistentVolumeClaimBuilder) RequestResource(requests corev1api.ResourceList) *PersistentVolumeClaimBuilder {\n\tif b.object.Spec.Resources.Requests == nil {\n\t\tb.object.Spec.Resources.Requests = make(map[corev1api.ResourceName]resource.Quantity)\n\t}\n\tb.object.Spec.Resources.Requests = requests\n\treturn b\n}\n\n// LimitResource sets the PersistentVolumeClaim's spec.Resources.Limits.\nfunc (b *PersistentVolumeClaimBuilder) LimitResource(limits corev1api.ResourceList) *PersistentVolumeClaimBuilder {\n\tif b.object.Spec.Resources.Limits == nil {\n\t\tb.object.Spec.Resources.Limits = make(map[corev1api.ResourceName]resource.Quantity)\n\t}\n\tb.object.Spec.Resources.Limits = limits\n\treturn b\n}\n\n// DataSource sets the PersistentVolumeClaim's spec.DataSource.\nfunc (b *PersistentVolumeClaimBuilder) DataSource(dataSource *corev1api.TypedLocalObjectReference) *PersistentVolumeClaimBuilder {\n\tb.object.Spec.DataSource = dataSource\n\treturn b\n}\n\n// DataSourceRef sets the PersistentVolumeClaim's spec.DataSourceRef.\nfunc (b *PersistentVolumeClaimBuilder) DataSourceRef(dataSourceRef *corev1api.TypedObjectReference) *PersistentVolumeClaimBuilder {\n\tb.object.Spec.DataSourceRef = dataSourceRef\n\treturn b\n}\n\n// Selector sets the PersistentVolumeClaim's spec.Selector.\nfunc (b *PersistentVolumeClaimBuilder) Selector(labelSelector *metav1.LabelSelector) *PersistentVolumeClaimBuilder {\n\tb.object.Spec.Selector = labelSelector\n\treturn b\n}\n" }, { "path": "pkg/builder/pod_builder.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// PodBuilder builds Pod objects.\ntype PodBuilder struct {\n\tobject *corev1api.Pod\n}\n\n// ForPod is the constructor for a PodBuilder.\nfunc ForPod(ns, name string) *PodBuilder {\n\treturn &PodBuilder{\n\t\tobject: &corev1api.Pod{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: corev1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"Pod\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built Pod.\nfunc (b *PodBuilder) Result() *corev1api.Pod {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the Pod's ObjectMeta.\nfunc (b *PodBuilder) ObjectMeta(opts ...ObjectMetaOpt) *PodBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\n\treturn b\n}\n\n// ServiceAccount sets serviceAccounts on pod.\nfunc (b *PodBuilder) ServiceAccount(sa string) *PodBuilder {\n\tb.object.Spec.ServiceAccountName = sa\n\treturn b\n}\n\n// Volumes appends to the pod's volumes\nfunc (b *PodBuilder) Volumes(volumes ...*corev1api.Volume) *PodBuilder {\n\tfor _, v := range volumes {\n\t\tb.object.Spec.Volumes = append(b.object.Spec.Volumes, *v)\n\t}\n\treturn b\n}\n\n// NodeName sets the pod's node name\nfunc (b *PodBuilder) NodeName(val string) *PodBuilder {\n\tb.object.Spec.NodeName = val\n\treturn b\n}\n\nfunc (b *PodBuilder) Labels(labels map[string]string) *PodBuilder {\n\tb.object.Labels = labels\n\treturn b\n}\n\nfunc (b *PodBuilder) InitContainers(containers ...*corev1api.Container) *PodBuilder {\n\tfor _, c := range containers {\n\t\tb.object.Spec.InitContainers = append(b.object.Spec.InitContainers, *c)\n\t}\n\treturn b\n}\n\nfunc (b *PodBuilder) InitContainerState(state corev1api.ContainerState) *PodBuilder {\n\tb.object.Status.InitContainerStatuses = append(b.object.Status.InitContainerStatuses, corev1api.ContainerStatus{State: state})\n\treturn b\n}\n\nfunc (b *PodBuilder) Containers(containers ...*corev1api.Container) *PodBuilder {\n\tfor _, c := range containers {\n\t\tb.object.Spec.Containers = append(b.object.Spec.Containers, *c)\n\t}\n\treturn b\n}\n\nfunc (b *PodBuilder) ContainerStatuses(containerStatuses ...*corev1api.ContainerStatus) *PodBuilder {\n\tfor _, c := range containerStatuses {\n\t\tb.object.Status.ContainerStatuses = append(b.object.Status.ContainerStatuses, *c)\n\t}\n\treturn b\n}\n\nfunc (b *PodBuilder) Phase(phase corev1api.PodPhase) *PodBuilder {\n\tb.object.Status.Phase = phase\n\treturn b\n}\n\nfunc (b *PodBuilder) Status(status corev1api.PodStatus) *PodBuilder {\n\tb.object.Status = status\n\treturn b\n}\n" }, { "path": "pkg/builder/pod_volume_backup_builder.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// PodVolumeBackupBuilder builds PodVolumeBackup objects\ntype PodVolumeBackupBuilder struct {\n\tobject *velerov1api.PodVolumeBackup\n}\n\n// ForPodVolumeBackup is the constructor for a PodVolumeBackupBuilder.\nfunc ForPodVolumeBackup(ns, name string) *PodVolumeBackupBuilder {\n\treturn &PodVolumeBackupBuilder{\n\t\tobject: &velerov1api.PodVolumeBackup{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"PodVolumeBackup\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built PodVolumeBackup.\nfunc (b *PodVolumeBackupBuilder) Result() *velerov1api.PodVolumeBackup {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the PodVolumeBackup's ObjectMeta.\nfunc (b *PodVolumeBackupBuilder) ObjectMeta(opts ...ObjectMetaOpt) *PodVolumeBackupBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\treturn b\n}\n\n// Phase sets the PodVolumeBackup's phase.\nfunc (b *PodVolumeBackupBuilder) Phase(phase velerov1api.PodVolumeBackupPhase) *PodVolumeBackupBuilder {\n\tb.object.Status.Phase = phase\n\treturn b\n}\n\n// Node sets the PodVolumeBackup's node name.\nfunc (b *PodVolumeBackupBuilder) Node(name string) *PodVolumeBackupBuilder {\n\tb.object.Spec.Node = name\n\treturn b\n}\n\n// BackupStorageLocation sets the PodVolumeBackup's backup storage location.\nfunc (b *PodVolumeBackupBuilder) BackupStorageLocation(name string) *PodVolumeBackupBuilder {\n\tb.object.Spec.BackupStorageLocation = name\n\treturn b\n}\n\n// SnapshotID sets the PodVolumeBackup's snapshot ID.\nfunc (b *PodVolumeBackupBuilder) SnapshotID(snapshotID string) *PodVolumeBackupBuilder {\n\tb.object.Status.SnapshotID = snapshotID\n\treturn b\n}\n\nfunc (b *PodVolumeBackupBuilder) StartTimestamp(startTimestamp *metav1.Time) *PodVolumeBackupBuilder {\n\tb.object.Status.StartTimestamp = startTimestamp\n\treturn b\n}\n\nfunc (b *PodVolumeBackupBuilder) CompletionTimestamp(completionTimestamp *metav1.Time) *PodVolumeBackupBuilder {\n\tb.object.Status.CompletionTimestamp = completionTimestamp\n\treturn b\n}\n\n// PodName sets the name of the pod associated with this PodVolumeBackup.\nfunc (b *PodVolumeBackupBuilder) PodName(name string) *PodVolumeBackupBuilder {\n\tb.object.Spec.Pod.Name = name\n\treturn b\n}\n\n// PodNamespace sets the name of the pod associated with this PodVolumeBackup.\nfunc (b *PodVolumeBackupBuilder) PodNamespace(ns string) *PodVolumeBackupBuilder {\n\tb.object.Spec.Pod.Namespace = ns\n\treturn b\n}\n\n// Volume sets the name of the volume associated with this PodVolumeBackup.\nfunc (b *PodVolumeBackupBuilder) Volume(volume string) *PodVolumeBackupBuilder {\n\tb.object.Spec.Volume = volume\n\treturn b\n}\n\n// UploaderType sets the type of uploader to use for this PodVolumeBackup.\nfunc (b *PodVolumeBackupBuilder) UploaderType(uploaderType string) *PodVolumeBackupBuilder {\n\tb.object.Spec.UploaderType = uploaderType\n\treturn b\n}\n\n// Annotations sets the PodVolumeBackup's Annotations.\nfunc (b *PodVolumeBackupBuilder) Annotations(annotations map[string]string) *PodVolumeBackupBuilder {\n\tb.object.Annotations = annotations\n\treturn b\n}\n\n// Cancel sets the PodVolumeBackup's Cancel.\nfunc (b *PodVolumeBackupBuilder) Cancel(cancel bool) *PodVolumeBackupBuilder {\n\tb.object.Spec.Cancel = cancel\n\treturn b\n}\n\n// AcceptedTimestamp sets the PodVolumeBackup's AcceptedTimestamp.\nfunc (b *PodVolumeBackupBuilder) AcceptedTimestamp(acceptedTimestamp *metav1.Time) *PodVolumeBackupBuilder {\n\tb.object.Status.AcceptedTimestamp = acceptedTimestamp\n\treturn b\n}\n\n// Finalizers sets the PodVolumeBackup's Finalizers.\nfunc (b *PodVolumeBackupBuilder) Finalizers(finalizers []string) *PodVolumeBackupBuilder {\n\tb.object.Finalizers = finalizers\n\treturn b\n}\n\n// Message sets the PodVolumeBackup's Message.\nfunc (b *PodVolumeBackupBuilder) Message(msg string) *PodVolumeBackupBuilder {\n\tb.object.Status.Message = msg\n\treturn b\n}\n\n// OwnerReference sets the PodVolumeBackup's OwnerReference.\nfunc (b *PodVolumeBackupBuilder) OwnerReference(ref metav1.OwnerReference) *PodVolumeBackupBuilder {\n\tb.object.OwnerReferences = append(b.object.OwnerReferences, ref)\n\treturn b\n}\n\n// Labels sets the PodVolumeBackup's Labels.\nfunc (b *PodVolumeBackupBuilder) Labels(label map[string]string) *PodVolumeBackupBuilder {\n\tb.object.Labels = label\n\treturn b\n}\n" }, { "path": "pkg/builder/pod_volume_restore_builder.go", "content": "/*\nCopyright 2023 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// PodVolumeRestoreBuilder builds PodVolumeRestore objects.\ntype PodVolumeRestoreBuilder struct {\n\tobject *velerov1api.PodVolumeRestore\n}\n\n// ForPodVolumeRestore is the constructor for a PodVolumeRestoreBuilder.\nfunc ForPodVolumeRestore(ns, name string) *PodVolumeRestoreBuilder {\n\treturn &PodVolumeRestoreBuilder{\n\t\tobject: &velerov1api.PodVolumeRestore{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"PodVolumeRestore\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built PodVolumeRestore.\nfunc (b *PodVolumeRestoreBuilder) Result() *velerov1api.PodVolumeRestore {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the PodVolumeRestore's ObjectMeta.\nfunc (b *PodVolumeRestoreBuilder) ObjectMeta(opts ...ObjectMetaOpt) *PodVolumeRestoreBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\treturn b\n}\n\n// Phase sets the PodVolumeRestore's phase.\nfunc (b *PodVolumeRestoreBuilder) Phase(phase velerov1api.PodVolumeRestorePhase) *PodVolumeRestoreBuilder {\n\tb.object.Status.Phase = phase\n\treturn b\n}\n\n// BackupStorageLocation sets the PodVolumeRestore's backup storage location.\nfunc (b *PodVolumeRestoreBuilder) BackupStorageLocation(name string) *PodVolumeRestoreBuilder {\n\tb.object.Spec.BackupStorageLocation = name\n\treturn b\n}\n\n// SnapshotID sets the PodVolumeRestore's snapshot ID.\nfunc (b *PodVolumeRestoreBuilder) SnapshotID(snapshotID string) *PodVolumeRestoreBuilder {\n\tb.object.Spec.SnapshotID = snapshotID\n\treturn b\n}\n\n// PodName sets the name of the pod associated with this PodVolumeRestore.\nfunc (b *PodVolumeRestoreBuilder) PodName(name string) *PodVolumeRestoreBuilder {\n\tb.object.Spec.Pod.Name = name\n\treturn b\n}\n\n// PodNamespace sets the name of the pod associated with this PodVolumeRestore.\nfunc (b *PodVolumeRestoreBuilder) PodNamespace(ns string) *PodVolumeRestoreBuilder {\n\tb.object.Spec.Pod.Namespace = ns\n\treturn b\n}\n\n// Volume sets the name of the volume associated with this PodVolumeRestore.\nfunc (b *PodVolumeRestoreBuilder) Volume(volume string) *PodVolumeRestoreBuilder {\n\tb.object.Spec.Volume = volume\n\treturn b\n}\n\n// UploaderType sets the type of uploader to use for this PodVolumeRestore.\nfunc (b *PodVolumeRestoreBuilder) UploaderType(uploaderType string) *PodVolumeRestoreBuilder {\n\tb.object.Spec.UploaderType = uploaderType\n\treturn b\n}\n\n// OwnerReference sets the OwnerReference for this PodVolumeRestore.\nfunc (b *PodVolumeRestoreBuilder) OwnerReference(ownerRef []metav1.OwnerReference) *PodVolumeRestoreBuilder {\n\tb.object.OwnerReferences = ownerRef\n\treturn b\n}\n\n// Cancel sets the DataDownload's Cancel.\nfunc (b *PodVolumeRestoreBuilder) Cancel(cancel bool) *PodVolumeRestoreBuilder {\n\tb.object.Spec.Cancel = cancel\n\treturn b\n}\n\n// AcceptedTimestamp sets the PodVolumeRestore's AcceptedTimestamp.\nfunc (b *PodVolumeRestoreBuilder) AcceptedTimestamp(acceptedTimestamp *metav1.Time) *PodVolumeRestoreBuilder {\n\tb.object.Status.AcceptedTimestamp = acceptedTimestamp\n\treturn b\n}\n\n// Finalizers sets the PodVolumeRestore's Finalizers.\nfunc (b *PodVolumeRestoreBuilder) Finalizers(finalizers []string) *PodVolumeRestoreBuilder {\n\tb.object.Finalizers = finalizers\n\treturn b\n}\n\n// Message sets the PodVolumeRestore's Message.\nfunc (b *PodVolumeRestoreBuilder) Message(msg string) *PodVolumeRestoreBuilder {\n\tb.object.Status.Message = msg\n\treturn b\n}\n\n// Message sets the PodVolumeRestore's Node.\nfunc (b *PodVolumeRestoreBuilder) Node(node string) *PodVolumeRestoreBuilder {\n\tb.object.Status.Node = node\n\treturn b\n}\n\n// Labels sets the PodVolumeRestoreBuilder's Labels.\nfunc (b *PodVolumeRestoreBuilder) Labels(label map[string]string) *PodVolumeRestoreBuilder {\n\tb.object.Labels = label\n\treturn b\n}\n" }, { "path": "pkg/builder/priority_class_builder.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n\tschedulingv1api \"k8s.io/api/scheduling/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\ntype PriorityClassBuilder struct {\n\tobject *schedulingv1api.PriorityClass\n}\n\nfunc ForPriorityClass(name string) *PriorityClassBuilder {\n\treturn &PriorityClassBuilder{\n\t\tobject: &schedulingv1api.PriorityClass{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\nfunc (p *PriorityClassBuilder) Value(value int) *PriorityClassBuilder {\n\tp.object.Value = int32(value)\n\treturn p\n}\n\nfunc (p *PriorityClassBuilder) PreemptionPolicy(policy string) *PriorityClassBuilder {\n\tpreemptionPolicy := corev1api.PreemptionPolicy(policy)\n\tp.object.PreemptionPolicy = &preemptionPolicy\n\treturn p\n}\n\nfunc (p *PriorityClassBuilder) Result() *schedulingv1api.PriorityClass {\n\treturn p.object\n}\n" }, { "path": "pkg/builder/restore_builder.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\t\"time\"\n\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// RestoreBuilder builds Restore objects.\ntype RestoreBuilder struct {\n\tobject *velerov1api.Restore\n}\n\n// ForRestore is the constructor for a RestoreBuilder.\nfunc ForRestore(ns, name string) *RestoreBuilder {\n\treturn &RestoreBuilder{\n\t\tobject: &velerov1api.Restore{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"Restore\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built Restore.\nfunc (b *RestoreBuilder) Result() *velerov1api.Restore {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the Restore's ObjectMeta.\nfunc (b *RestoreBuilder) ObjectMeta(opts ...ObjectMetaOpt) *RestoreBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\n\treturn b\n}\n\n// Backup sets the Restore's backup name.\nfunc (b *RestoreBuilder) Backup(name string) *RestoreBuilder {\n\tb.object.Spec.BackupName = name\n\treturn b\n}\n\n// Schedule sets the Restore's schedule name.\nfunc (b *RestoreBuilder) Schedule(name string) *RestoreBuilder {\n\tb.object.Spec.ScheduleName = name\n\treturn b\n}\n\n// IncludedNamespaces appends to the Restore's included namespaces.\nfunc (b *RestoreBuilder) IncludedNamespaces(namespaces ...string) *RestoreBuilder {\n\tb.object.Spec.IncludedNamespaces = append(b.object.Spec.IncludedNamespaces, namespaces...)\n\treturn b\n}\n\n// ExcludedNamespaces appends to the Restore's excluded namespaces.\nfunc (b *RestoreBuilder) ExcludedNamespaces(namespaces ...string) *RestoreBuilder {\n\tb.object.Spec.ExcludedNamespaces = append(b.object.Spec.ExcludedNamespaces, namespaces...)\n\treturn b\n}\n\n// IncludedResources appends to the Restore's included resources.\nfunc (b *RestoreBuilder) IncludedResources(resources ...string) *RestoreBuilder {\n\tb.object.Spec.IncludedResources = append(b.object.Spec.IncludedResources, resources...)\n\treturn b\n}\n\n// ExcludedResources appends to the Restore's excluded resources.\nfunc (b *RestoreBuilder) ExcludedResources(resources ...string) *RestoreBuilder {\n\tb.object.Spec.ExcludedResources = append(b.object.Spec.ExcludedResources, resources...)\n\treturn b\n}\n\n// ExistingResourcePolicy sets the Restore's resource policy.\nfunc (b *RestoreBuilder) ExistingResourcePolicy(policy string) *RestoreBuilder {\n\tb.object.Spec.ExistingResourcePolicy = velerov1api.PolicyType(policy)\n\treturn b\n}\n\n// IncludeClusterResources sets the Restore's \"include cluster resources\" flag.\nfunc (b *RestoreBuilder) IncludeClusterResources(val bool) *RestoreBuilder {\n\tb.object.Spec.IncludeClusterResources = &val\n\treturn b\n}\n\n// LabelSelector sets the Restore's label selector.\nfunc (b *RestoreBuilder) LabelSelector(selector *metav1.LabelSelector) *RestoreBuilder {\n\tb.object.Spec.LabelSelector = selector\n\treturn b\n}\n\n// OrLabelSelector sets the Restore's orLabelSelector set.\nfunc (b *RestoreBuilder) OrLabelSelector(orSelectors []*metav1.LabelSelector) *RestoreBuilder {\n\tb.object.Spec.OrLabelSelectors = orSelectors\n\treturn b\n}\n\n// NamespaceMappings sets the Restore's namespace mappings.\nfunc (b *RestoreBuilder) NamespaceMappings(mapping ...string) *RestoreBuilder {\n\tif b.object.Spec.NamespaceMapping == nil {\n\t\tb.object.Spec.NamespaceMapping = make(map[string]string)\n\t}\n\n\tif len(mapping)%2 != 0 {\n\t\tpanic(\"mapping must contain an even number of values\")\n\t}\n\n\tfor i := 0; i < len(mapping); i += 2 {\n\t\tb.object.Spec.NamespaceMapping[mapping[i]] = mapping[i+1]\n\t}\n\n\treturn b\n}\n\n// Phase sets the Restore's phase.\nfunc (b *RestoreBuilder) Phase(phase velerov1api.RestorePhase) *RestoreBuilder {\n\tb.object.Status.Phase = phase\n\treturn b\n}\n\n// RestorePVs sets the Restore's restore PVs.\nfunc (b *RestoreBuilder) RestorePVs(val bool) *RestoreBuilder {\n\tb.object.Spec.RestorePVs = &val\n\treturn b\n}\n\n// PreserveNodePorts sets the Restore's preserved NodePorts.\nfunc (b *RestoreBuilder) PreserveNodePorts(val bool) *RestoreBuilder {\n\tb.object.Spec.PreserveNodePorts = &val\n\treturn b\n}\n\n// StartTimestamp sets the Restore's start timestamp.\nfunc (b *RestoreBuilder) StartTimestamp(val time.Time) *RestoreBuilder {\n\tb.object.Status.StartTimestamp = &metav1.Time{Time: val}\n\treturn b\n}\n\n// CompletionTimestamp sets the Restore's completion timestamp.\nfunc (b *RestoreBuilder) CompletionTimestamp(val time.Time) *RestoreBuilder {\n\tb.object.Status.CompletionTimestamp = &metav1.Time{Time: val}\n\treturn b\n}\n\n// ItemOperationTimeout sets the Restore's ItemOperationTimeout\nfunc (b *RestoreBuilder) ItemOperationTimeout(timeout time.Duration) *RestoreBuilder {\n\tb.object.Spec.ItemOperationTimeout.Duration = timeout\n\treturn b\n}\n" }, { "path": "pkg/builder/role_builder.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\trbacv1 \"k8s.io/api/rbac/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// RoleBuilder builds Role objects.\ntype RoleBuilder struct {\n\tobject *rbacv1.Role\n}\n\n// ForRole is the constructor for a RoleBuilder.\nfunc ForRole(ns, name string) *RoleBuilder {\n\treturn &RoleBuilder{\n\t\tobject: &rbacv1.Role{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: rbacv1.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"Role\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built Role.\nfunc (b *RoleBuilder) Result() *rbacv1.Role {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the Role's ObjectMeta.\nfunc (b *RoleBuilder) ObjectMeta(opts ...ObjectMetaOpt) *RoleBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\n\treturn b\n}\n" }, { "path": "pkg/builder/schedule_builder.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\t\"time\"\n\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// ScheduleBuilder builds Schedule objects.\ntype ScheduleBuilder struct {\n\tobject *velerov1api.Schedule\n}\n\n// ForSchedule is the constructor for a ScheduleBuilder.\nfunc ForSchedule(ns, name string) *ScheduleBuilder {\n\treturn &ScheduleBuilder{\n\t\tobject: &velerov1api.Schedule{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"Schedule\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built Schedule.\nfunc (b *ScheduleBuilder) Result() *velerov1api.Schedule {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the Schedule's ObjectMeta.\nfunc (b *ScheduleBuilder) ObjectMeta(opts ...ObjectMetaOpt) *ScheduleBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\n\treturn b\n}\n\n// Phase sets the Schedule's phase.\nfunc (b *ScheduleBuilder) Phase(phase velerov1api.SchedulePhase) *ScheduleBuilder {\n\tb.object.Status.Phase = phase\n\treturn b\n}\n\n// ValidationError appends to the Schedule's validation errors.\nfunc (b *ScheduleBuilder) ValidationError(err string) *ScheduleBuilder {\n\tb.object.Status.ValidationErrors = append(b.object.Status.ValidationErrors, err)\n\treturn b\n}\n\n// CronSchedule sets the Schedule's cron schedule.\nfunc (b *ScheduleBuilder) CronSchedule(expression string) *ScheduleBuilder {\n\tb.object.Spec.Schedule = expression\n\treturn b\n}\n\n// LastBackupTime sets the Schedule's last backup time.\nfunc (b *ScheduleBuilder) LastBackupTime(val string) *ScheduleBuilder {\n\tt, _ := time.Parse(\"2006-01-02 15:04:05\", val)\n\tb.object.Status.LastBackup = &metav1.Time{Time: t}\n\treturn b\n}\n\n// Template sets the Schedule's template.\nfunc (b *ScheduleBuilder) Template(spec velerov1api.BackupSpec) *ScheduleBuilder {\n\tb.object.Spec.Template = spec\n\treturn b\n}\n\n// SkipImmediately sets the Schedule's SkipImmediately.\nfunc (b *ScheduleBuilder) SkipImmediately(skip *bool) *ScheduleBuilder {\n\tb.object.Spec.SkipImmediately = skip\n\treturn b\n}\n" }, { "path": "pkg/builder/secret_builder.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// SecretBuilder builds Secret objects.\ntype SecretBuilder struct {\n\tobject *corev1api.Secret\n}\n\n// ForSecret is the constructor for a SecretBuilder.\nfunc ForSecret(ns, name string) *SecretBuilder {\n\treturn &SecretBuilder{\n\t\tobject: &corev1api.Secret{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: corev1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"Secret\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built Secret.\nfunc (b *SecretBuilder) Result() *corev1api.Secret {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the Secret's ObjectMeta.\nfunc (b *SecretBuilder) ObjectMeta(opts ...ObjectMetaOpt) *SecretBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\n\treturn b\n}\n\n// Data sets the Secret data.\nfunc (b *SecretBuilder) Data(data map[string][]byte) *SecretBuilder {\n\tb.object.Data = data\n\treturn b\n}\n" }, { "path": "pkg/builder/secret_key_selector_builder.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n)\n\n// SecretKeySelectorBuilder builds SecretKeySelector objects.\ntype SecretKeySelectorBuilder struct {\n\tobject *corev1api.SecretKeySelector\n}\n\n// ForSecretKeySelector is the constructor for a SecretKeySelectorBuilder.\nfunc ForSecretKeySelector(name string, key string) *SecretKeySelectorBuilder {\n\treturn &SecretKeySelectorBuilder{\n\t\tobject: &corev1api.SecretKeySelector{\n\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\tName: name,\n\t\t\t},\n\t\t\tKey: key,\n\t\t},\n\t}\n}\n\n// Result returns the built SecretKeySelector.\nfunc (b *SecretKeySelectorBuilder) Result() *corev1api.SecretKeySelector {\n\treturn b.object\n}\n" }, { "path": "pkg/builder/server_status_request_builder.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\t\"time\"\n\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// ServerStatusRequestBuilder builds ServerStatusRequest objects.\ntype ServerStatusRequestBuilder struct {\n\tobject *velerov1api.ServerStatusRequest\n}\n\n// ForServerStatusRequest is the constructor for a ServerStatusRequestBuilder.\nfunc ForServerStatusRequest(ns, name, resourceVersion string) *ServerStatusRequestBuilder {\n\treturn &ServerStatusRequestBuilder{\n\t\tobject: &velerov1api.ServerStatusRequest{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"ServerStatusRequest\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t\tResourceVersion: resourceVersion,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built ServerStatusRequest.\nfunc (b *ServerStatusRequestBuilder) Result() *velerov1api.ServerStatusRequest {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the ServerStatusRequest's ObjectMeta.\nfunc (b *ServerStatusRequestBuilder) ObjectMeta(opts ...ObjectMetaOpt) *ServerStatusRequestBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\n\treturn b\n}\n\n// Phase sets the ServerStatusRequest's phase.\nfunc (b *ServerStatusRequestBuilder) Phase(phase velerov1api.ServerStatusRequestPhase) *ServerStatusRequestBuilder {\n\tb.object.Status.Phase = phase\n\treturn b\n}\n\n// ProcessedTimestamp sets the ServerStatusRequest's processed timestamp.\nfunc (b *ServerStatusRequestBuilder) ProcessedTimestamp(time time.Time) *ServerStatusRequestBuilder {\n\tb.object.Status.ProcessedTimestamp = &metav1.Time{Time: time}\n\treturn b\n}\n\n// ServerVersion sets the ServerStatusRequest's server version.\nfunc (b *ServerStatusRequestBuilder) ServerVersion(version string) *ServerStatusRequestBuilder {\n\tb.object.Status.ServerVersion = version\n\treturn b\n}\n\n// Plugins sets the ServerStatusRequest's plugins.\nfunc (b *ServerStatusRequestBuilder) Plugins(plugins []velerov1api.PluginInfo) *ServerStatusRequestBuilder {\n\tb.object.Status.Plugins = plugins\n\treturn b\n}\n" }, { "path": "pkg/builder/service_account_builder.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// ServiceAccountBuilder builds ServiceAccount objects.\ntype ServiceAccountBuilder struct {\n\tobject *corev1api.ServiceAccount\n}\n\n// ForServiceAccount is the constructor for a ServiceAccountBuilder.\nfunc ForServiceAccount(ns, name string) *ServiceAccountBuilder {\n\treturn &ServiceAccountBuilder{\n\t\tobject: &corev1api.ServiceAccount{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: corev1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"ServiceAccount\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built ServiceAccount.\nfunc (b *ServiceAccountBuilder) Result() *corev1api.ServiceAccount {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the ServiceAccount's ObjectMeta.\nfunc (b *ServiceAccountBuilder) ObjectMeta(opts ...ObjectMetaOpt) *ServiceAccountBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\n\treturn b\n}\n" }, { "path": "pkg/builder/service_builder.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// ServiceBuilder builds Service objects.\ntype ServiceBuilder struct {\n\tobject *corev1api.Service\n}\n\n// ForService is the constructor for a ServiceBuilder.\nfunc ForService(ns, name string) *ServiceBuilder {\n\treturn &ServiceBuilder{\n\t\tobject: &corev1api.Service{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: corev1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"Service\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built Service.\nfunc (s *ServiceBuilder) Result() *corev1api.Service {\n\treturn s.object\n}\n\n// ObjectMeta applies functional options to the Service's ObjectMeta.\nfunc (s *ServiceBuilder) ObjectMeta(opts ...ObjectMetaOpt) *ServiceBuilder {\n\tfor _, opt := range opts {\n\t\topt(s.object)\n\t}\n\n\treturn s\n}\n" }, { "path": "pkg/builder/statefulset_builder.go", "content": "/*\nCopyright 2021 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// StatefulSetBuilder builds StatefulSet objects.\ntype StatefulSetBuilder struct {\n\tobject *appsv1api.StatefulSet\n}\n\n// ForStatefulSet is the constructor for a StatefulSetBuilder.\nfunc ForStatefulSet(ns, name string) *StatefulSetBuilder {\n\treturn &StatefulSetBuilder{\n\t\tobject: &appsv1api.StatefulSet{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: appsv1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"StatefulSet\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t\tSpec: appsv1api.StatefulSetSpec{\n\t\t\t\tVolumeClaimTemplates: []corev1api.PersistentVolumeClaim{},\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built StatefulSet.\nfunc (b *StatefulSetBuilder) Result() *appsv1api.StatefulSet {\n\treturn b.object\n}\n\n// StorageClass sets the StatefulSet's VolumeClaimTemplates storage class name.\nfunc (b *StatefulSetBuilder) StorageClass(names ...string) *StatefulSetBuilder {\n\tfor _, name := range names {\n\t\tnameTmp := name\n\t\tb.object.Spec.VolumeClaimTemplates = append(b.object.Spec.VolumeClaimTemplates,\n\t\t\tcorev1api.PersistentVolumeClaim{Spec: corev1api.PersistentVolumeClaimSpec{StorageClassName: &nameTmp}})\n\t}\n\treturn b\n}\n" }, { "path": "pkg/builder/storage_class_builder.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n\tstoragev1api \"k8s.io/api/storage/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// StorageClassBuilder builds StorageClass objects.\ntype StorageClassBuilder struct {\n\tobject *storagev1api.StorageClass\n\tobjectSlice []*storagev1api.StorageClass\n}\n\n// ForStorageClass is the constructor for a StorageClassBuilder.\nfunc ForStorageClass(name string) *StorageClassBuilder {\n\treturn &StorageClassBuilder{\n\t\tobject: &storagev1api.StorageClass{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: storagev1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"StorageClass\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built StorageClass.\nfunc (b *StorageClassBuilder) Result() *storagev1api.StorageClass {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the StorageClass's ObjectMeta.\nfunc (b *StorageClassBuilder) ObjectMeta(opts ...ObjectMetaOpt) *StorageClassBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\n\treturn b\n}\n\n// ForStorageClassSlice is the constructor for a storageClassSlice in StorageClassBuilder.\nfunc ForStorageClassSlice(names ...string) *StorageClassBuilder {\n\tvar storageClassSlice []*storagev1api.StorageClass\n\tfor _, name := range names {\n\t\tstorageClass := &storagev1api.StorageClass{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: storagev1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"StorageClass\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: name,\n\t\t\t},\n\t\t}\n\t\tstorageClassSlice = append(storageClassSlice, storageClass)\n\t}\n\n\treturn &StorageClassBuilder{\n\t\tobjectSlice: storageClassSlice,\n\t}\n}\n\n// SliceResult returns the built StorageClass slice.\nfunc (b *StorageClassBuilder) SliceResult() []*storagev1api.StorageClass {\n\treturn b.objectSlice\n}\n\n// Provisioner sets StorageClass's provisioner.\nfunc (b *StorageClassBuilder) Provisioner(provisioner string) *StorageClassBuilder {\n\tb.object.Provisioner = provisioner\n\treturn b\n}\n\n// ReclaimPolicy sets StorageClass's reclaimPolicy.\nfunc (b *StorageClassBuilder) ReclaimPolicy(policy corev1api.PersistentVolumeReclaimPolicy) *StorageClassBuilder {\n\tb.object.ReclaimPolicy = &policy\n\treturn b\n}\n" }, { "path": "pkg/builder/testcr_builder.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// CustomResourceBuilder builds objects based on velero APIVersion CRDs.\ntype TestCRBuilder struct {\n\tobject *TestCR\n}\n\n// ForTestCR is the constructor for a TestCRBuilder.\nfunc ForTestCR(crdKind, ns, name string) *TestCRBuilder {\n\treturn &TestCRBuilder{\n\t\tobject: &TestCR{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: crdKind,\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built TestCR.\nfunc (b *TestCRBuilder) Result() *TestCR {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the TestCR's ObjectMeta.\nfunc (b *TestCRBuilder) ObjectMeta(opts ...ObjectMetaOpt) *TestCRBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\n\treturn b\n}\n\ntype TestCR struct {\n\tmetav1.TypeMeta `json:\",inline\"`\n\n\t// +optional\n\tmetav1.ObjectMeta `json:\"metadata,omitempty\"`\n\n\t// +optional\n\tSpec TestCRSpec `json:\"spec,omitempty\"`\n\n\t// +optional\n\tStatus TestCRStatus `json:\"status,omitempty\"`\n}\n\ntype TestCRSpec struct {\n}\n\ntype TestCRStatus struct {\n}\n" }, { "path": "pkg/builder/v1_customresourcedefinition_builder.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tapiextv1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// V1CustomResourceDefinitionBuilder builds CustomResourceDefinition objects.\ntype V1CustomResourceDefinitionBuilder struct {\n\tobject *apiextv1.CustomResourceDefinition\n}\n\n// ForV1CustomResourceDefinition is the constructor for a V1CustomResourceDefinitionBuilder.\nfunc ForV1CustomResourceDefinition(name string) *V1CustomResourceDefinitionBuilder {\n\treturn &V1CustomResourceDefinitionBuilder{\n\t\tobject: &apiextv1.CustomResourceDefinition{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: apiextv1.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"CustomResourceDefinition\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Condition adds a CustomResourceDefinitionCondition objects to a V1CustomResourceDefinitionBuilder.\nfunc (b *V1CustomResourceDefinitionBuilder) Condition(cond apiextv1.CustomResourceDefinitionCondition) *V1CustomResourceDefinitionBuilder {\n\tb.object.Status.Conditions = append(b.object.Status.Conditions, cond)\n\treturn b\n}\n\n// Version adds a CustomResourceDefinitionVersion object to a V1CustomResourceDefinitionBuilder.\nfunc (b *V1CustomResourceDefinitionBuilder) Version(ver apiextv1.CustomResourceDefinitionVersion) *V1CustomResourceDefinitionBuilder {\n\tb.object.Spec.Versions = append(b.object.Spec.Versions, ver)\n\treturn b\n}\n\n// PreserveUnknownFields sets PreserveUnknownFields on a CustomResourceDefinition.\nfunc (b *V1CustomResourceDefinitionBuilder) PreserveUnknownFields(preserve bool) *V1CustomResourceDefinitionBuilder {\n\tb.object.Spec.PreserveUnknownFields = preserve\n\treturn b\n}\n\n// Result returns the built CustomResourceDefinition.\nfunc (b *V1CustomResourceDefinitionBuilder) Result() *apiextv1.CustomResourceDefinition {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the CustomResourceDefinition's ObjectMeta.\nfunc (b *V1CustomResourceDefinitionBuilder) ObjectMeta(opts ...ObjectMetaOpt) *V1CustomResourceDefinitionBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\n\treturn b\n}\n\n// V1CustomResourceDefinitionConditionBuilder builds CustomResourceDefinitionCondition objects.\ntype V1CustomResourceDefinitionConditionBuilder struct {\n\tobject apiextv1.CustomResourceDefinitionCondition\n}\n\n// ForV1CustomResourceDefinitionCondition is the constructor for a V1CustomResourceDefinitionConditionBuilder.\nfunc ForV1CustomResourceDefinitionCondition() *V1CustomResourceDefinitionConditionBuilder {\n\treturn &V1CustomResourceDefinitionConditionBuilder{\n\t\tobject: apiextv1.CustomResourceDefinitionCondition{},\n\t}\n}\n\n// Type sets the Condition's type.\nfunc (c *V1CustomResourceDefinitionConditionBuilder) Type(t apiextv1.CustomResourceDefinitionConditionType) *V1CustomResourceDefinitionConditionBuilder {\n\tc.object.Type = t\n\treturn c\n}\n\n// Status sets the Condition's status.\nfunc (c *V1CustomResourceDefinitionConditionBuilder) Status(cs apiextv1.ConditionStatus) *V1CustomResourceDefinitionConditionBuilder {\n\tc.object.Status = cs\n\treturn c\n}\n\n// Result returns the built CustomResourceDefinitionCondition.\nfunc (c *V1CustomResourceDefinitionConditionBuilder) Result() apiextv1.CustomResourceDefinitionCondition {\n\treturn c.object\n}\n\n// V1CustomResourceDefinitionVersionBuilder builds CustomResourceDefinitionVersion objects.\ntype V1CustomResourceDefinitionVersionBuilder struct {\n\tobject apiextv1.CustomResourceDefinitionVersion\n}\n\n// ForV1CustomResourceDefinitionVersion is the constructor for a V1CustomResourceDefinitionVersionBuilder.\nfunc ForV1CustomResourceDefinitionVersion(name string) *V1CustomResourceDefinitionVersionBuilder {\n\treturn &V1CustomResourceDefinitionVersionBuilder{\n\t\tobject: apiextv1.CustomResourceDefinitionVersion{Name: name},\n\t}\n}\n\n// Served sets the Served field on a CustomResourceDefinitionVersion.\nfunc (c *V1CustomResourceDefinitionVersionBuilder) Served(s bool) *V1CustomResourceDefinitionVersionBuilder {\n\tc.object.Served = s\n\treturn c\n}\n\n// Storage sets the Storage field on a CustomResourceDefinitionVersion.\nfunc (c *V1CustomResourceDefinitionVersionBuilder) Storage(s bool) *V1CustomResourceDefinitionVersionBuilder {\n\tc.object.Storage = s\n\treturn c\n}\n\nfunc (c *V1CustomResourceDefinitionVersionBuilder) Schema(s *apiextv1.JSONSchemaProps) *V1CustomResourceDefinitionVersionBuilder {\n\tif c.object.Schema == nil {\n\t\tc.object.Schema = new(apiextv1.CustomResourceValidation)\n\t}\n\tc.object.Schema.OpenAPIV3Schema = s\n\treturn c\n}\n\n// Result returns the built CustomResourceDefinitionVersion.\nfunc (c *V1CustomResourceDefinitionVersionBuilder) Result() apiextv1.CustomResourceDefinitionVersion {\n\treturn c.object\n}\n" }, { "path": "pkg/builder/volume_builder.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n)\n\n// VolumeBuilder builds Volume objects.\ntype VolumeBuilder struct {\n\tobject *corev1api.Volume\n}\n\n// ForVolume is the constructor for a VolumeBuilder.\nfunc ForVolume(name string) *VolumeBuilder {\n\treturn &VolumeBuilder{\n\t\tobject: &corev1api.Volume{\n\t\t\tName: name,\n\t\t},\n\t}\n}\n\n// Result returns the built Volume.\nfunc (b *VolumeBuilder) Result() *corev1api.Volume {\n\treturn b.object\n}\n\n// PersistentVolumeClaimSource sets the Volume's persistent volume claim source.\nfunc (b *VolumeBuilder) PersistentVolumeClaimSource(claimName string) *VolumeBuilder {\n\tb.object.PersistentVolumeClaim = &corev1api.PersistentVolumeClaimVolumeSource{\n\t\tClaimName: claimName,\n\t}\n\treturn b\n}\n\n// CSISource sets the Volume's CSI source.\nfunc (b *VolumeBuilder) CSISource(driver string) *VolumeBuilder {\n\tb.object.CSI = &corev1api.CSIVolumeSource{\n\t\tDriver: driver,\n\t}\n\treturn b\n}\n" }, { "path": "pkg/builder/volume_mount_builder.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n)\n\n// VolumeMountBuilder builds VolumeMount objects.\ntype VolumeMountBuilder struct {\n\tobject *corev1api.VolumeMount\n}\n\n// ForVolumeMount is the constructor for a VolumeMountBuilder.\nfunc ForVolumeMount(name, mountPath string) *VolumeMountBuilder {\n\treturn &VolumeMountBuilder{\n\t\tobject: &corev1api.VolumeMount{\n\t\t\tName: name,\n\t\t\tMountPath: mountPath,\n\t\t},\n\t}\n}\n\n// Result returns the built VolumeMount.\nfunc (b *VolumeMountBuilder) Result() *corev1api.VolumeMount {\n\treturn b.object\n}\n" }, { "path": "pkg/builder/volume_snapshot_builder.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"k8s.io/apimachinery/pkg/api/resource\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// VolumeSnapshotBuilder builds VolumeSnapshot objects.\ntype VolumeSnapshotBuilder struct {\n\tobject *snapshotv1api.VolumeSnapshot\n}\n\n// ForVolumeSnapshot is the constructor for VolumeSnapshotBuilder.\nfunc ForVolumeSnapshot(ns, name string) *VolumeSnapshotBuilder {\n\treturn &VolumeSnapshotBuilder{\n\t\tobject: &snapshotv1api.VolumeSnapshot{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: snapshotv1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"VolumeSnapshot\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: name,\n\t\t\t\tNamespace: ns,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// ObjectMeta applies functional options to the VolumeSnapshot's ObjectMeta.\nfunc (v *VolumeSnapshotBuilder) ObjectMeta(opts ...ObjectMetaOpt) *VolumeSnapshotBuilder {\n\tfor _, opt := range opts {\n\t\topt(v.object)\n\t}\n\n\treturn v\n}\n\n// Result return the built VolumeSnapshot.\nfunc (v *VolumeSnapshotBuilder) Result() *snapshotv1api.VolumeSnapshot {\n\treturn v.object\n}\n\n// Status init the built VolumeSnapshot's status.\nfunc (v *VolumeSnapshotBuilder) Status() *VolumeSnapshotBuilder {\n\tv.object.Status = &snapshotv1api.VolumeSnapshotStatus{}\n\treturn v\n}\n\n// BoundVolumeSnapshotContentName set built VolumeSnapshot's status BoundVolumeSnapshotContentName field.\nfunc (v *VolumeSnapshotBuilder) BoundVolumeSnapshotContentName(vscName string) *VolumeSnapshotBuilder {\n\tv.object.Status.BoundVolumeSnapshotContentName = &vscName\n\treturn v\n}\n\n// SourcePVC set the built VolumeSnapshot's spec.Source.PersistentVolumeClaimName.\nfunc (v *VolumeSnapshotBuilder) SourcePVC(name string) *VolumeSnapshotBuilder {\n\tv.object.Spec.Source.PersistentVolumeClaimName = &name\n\treturn v\n}\n\n// SourceVolumeSnapshotContentName set the built VolumeSnapshot's spec.Source.VolumeSnapshotContentName\nfunc (v *VolumeSnapshotBuilder) SourceVolumeSnapshotContentName(name string) *VolumeSnapshotBuilder {\n\tv.object.Spec.Source.VolumeSnapshotContentName = &name\n\treturn v\n}\n\n// RestoreSize set the built VolumeSnapshot's status.RestoreSize.\nfunc (v *VolumeSnapshotBuilder) RestoreSize(size string) *VolumeSnapshotBuilder {\n\tresourceSize := resource.MustParse(size)\n\tv.object.Status.RestoreSize = &resourceSize\n\treturn v\n}\n\n// VolumeSnapshotClass set the built VolumeSnapshot's spec.VolumeSnapshotClassName value.\nfunc (v *VolumeSnapshotBuilder) VolumeSnapshotClass(name string) *VolumeSnapshotBuilder {\n\tv.object.Spec.VolumeSnapshotClassName = &name\n\treturn v\n}\n\n// StatusError set the built VolumeSnapshot's status.Error value.\nfunc (v *VolumeSnapshotBuilder) StatusError(snapshotError snapshotv1api.VolumeSnapshotError) *VolumeSnapshotBuilder {\n\tv.object.Status.Error = &snapshotError\n\treturn v\n}\n\n// ReadyToUse set the built VolumeSnapshot's status.ReadyToUse value.\nfunc (v *VolumeSnapshotBuilder) ReadyToUse(readyToUse bool) *VolumeSnapshotBuilder {\n\tv.object.Status.ReadyToUse = &readyToUse\n\treturn v\n}\n" }, { "path": "pkg/builder/volume_snapshot_class_builder.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// VolumeSnapshotClassBuilder builds VolumeSnapshotClass objects.\ntype VolumeSnapshotClassBuilder struct {\n\tobject *snapshotv1api.VolumeSnapshotClass\n}\n\n// ForVolumeSnapshotClass is the constructor of VolumeSnapshotClassBuilder.\nfunc ForVolumeSnapshotClass(name string) *VolumeSnapshotClassBuilder {\n\treturn &VolumeSnapshotClassBuilder{\n\t\tobject: &snapshotv1api.VolumeSnapshotClass{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tKind: \"VolumeSnapshotClass\",\n\t\t\t\tAPIVersion: snapshotv1api.SchemeGroupVersion.String(),\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built VolumeSnapshotClass.\nfunc (b *VolumeSnapshotClassBuilder) Result() *snapshotv1api.VolumeSnapshotClass {\n\treturn b.object\n}\n\n// Driver sets the driver of built VolumeSnapshotClass.\nfunc (b *VolumeSnapshotClassBuilder) Driver(driver string) *VolumeSnapshotClassBuilder {\n\tb.object.Driver = driver\n\treturn b\n}\n\n// ObjectMeta applies functional options to the VolumeSnapshotClass's ObjectMeta.\nfunc (b *VolumeSnapshotClassBuilder) ObjectMeta(opts ...ObjectMetaOpt) *VolumeSnapshotClassBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\n\treturn b\n}\n" }, { "path": "pkg/builder/volume_snapshot_content_builder.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n)\n\n// VolumeSnapshotContentBuilder builds VolumeSnapshotContent object.\ntype VolumeSnapshotContentBuilder struct {\n\tobject *snapshotv1api.VolumeSnapshotContent\n}\n\n// ForVolumeSnapshotContent is the constructor of VolumeSnapshotContentBuilder.\nfunc ForVolumeSnapshotContent(name string) *VolumeSnapshotContentBuilder {\n\treturn &VolumeSnapshotContentBuilder{\n\t\tobject: &snapshotv1api.VolumeSnapshotContent{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: snapshotv1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"VolumeSnapshotContent\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built VolumeSnapshotContent.\nfunc (v *VolumeSnapshotContentBuilder) Result() *snapshotv1api.VolumeSnapshotContent {\n\treturn v.object\n}\n\n// Status initiates VolumeSnapshotContent's status.\nfunc (v *VolumeSnapshotContentBuilder) Status(status *snapshotv1api.VolumeSnapshotContentStatus) *VolumeSnapshotContentBuilder {\n\tv.object.Status = status\n\treturn v\n}\n\n// DeletionPolicy sets built VolumeSnapshotContent's spec.DeletionPolicy value.\nfunc (v *VolumeSnapshotContentBuilder) DeletionPolicy(policy snapshotv1api.DeletionPolicy) *VolumeSnapshotContentBuilder {\n\tv.object.Spec.DeletionPolicy = policy\n\treturn v\n}\n\n// VolumeSnapshotRef sets the built VolumeSnapshotContent's spec.VolumeSnapshotRef value.\nfunc (v *VolumeSnapshotContentBuilder) VolumeSnapshotRef(namespace, name, uid string) *VolumeSnapshotContentBuilder {\n\tv.object.Spec.VolumeSnapshotRef = corev1api.ObjectReference{\n\t\tAPIVersion: \"snapshot.storage.k8s.io/v1\",\n\t\tKind: \"VolumeSnapshot\",\n\t\tNamespace: namespace,\n\t\tName: name,\n\t\tUID: types.UID(uid),\n\t}\n\treturn v\n}\n\n// VolumeSnapshotClassName sets the built VolumeSnapshotContent's spec.VolumeSnapshotClassName value.\nfunc (v *VolumeSnapshotContentBuilder) VolumeSnapshotClassName(name string) *VolumeSnapshotContentBuilder {\n\tv.object.Spec.VolumeSnapshotClassName = &name\n\treturn v\n}\n\n// ObjectMeta applies functional options to the VolumeSnapshotContent's ObjectMeta.\nfunc (v *VolumeSnapshotContentBuilder) ObjectMeta(opts ...ObjectMetaOpt) *VolumeSnapshotContentBuilder {\n\tfor _, opt := range opts {\n\t\topt(v.object)\n\t}\n\n\treturn v\n}\n\nfunc (v *VolumeSnapshotContentBuilder) Driver(driver string) *VolumeSnapshotContentBuilder {\n\tv.object.Spec.Driver = driver\n\treturn v\n}\n\nfunc (v *VolumeSnapshotContentBuilder) Source(source snapshotv1api.VolumeSnapshotContentSource) *VolumeSnapshotContentBuilder {\n\tv.object.Spec.Source = source\n\treturn v\n}\n" }, { "path": "pkg/builder/volume_snapshot_location_builder.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage builder\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tcorev1api \"k8s.io/api/core/v1\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// VolumeSnapshotLocationBuilder builds VolumeSnapshotLocation objects.\ntype VolumeSnapshotLocationBuilder struct {\n\tobject *velerov1api.VolumeSnapshotLocation\n}\n\n// ForVolumeSnapshotLocation is the constructor for a VolumeSnapshotLocationBuilder.\nfunc ForVolumeSnapshotLocation(ns, name string) *VolumeSnapshotLocationBuilder {\n\treturn &VolumeSnapshotLocationBuilder{\n\t\tobject: &velerov1api.VolumeSnapshotLocation{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"VolumeSnapshotLocation\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: ns,\n\t\t\t\tName: name,\n\t\t\t},\n\t\t},\n\t}\n}\n\n// Result returns the built VolumeSnapshotLocation.\nfunc (b *VolumeSnapshotLocationBuilder) Result() *velerov1api.VolumeSnapshotLocation {\n\treturn b.object\n}\n\n// ObjectMeta applies functional options to the VolumeSnapshotLocation's ObjectMeta.\nfunc (b *VolumeSnapshotLocationBuilder) ObjectMeta(opts ...ObjectMetaOpt) *VolumeSnapshotLocationBuilder {\n\tfor _, opt := range opts {\n\t\topt(b.object)\n\t}\n\n\treturn b\n}\n\n// Provider sets the VolumeSnapshotLocation's provider.\nfunc (b *VolumeSnapshotLocationBuilder) Provider(name string) *VolumeSnapshotLocationBuilder {\n\tb.object.Spec.Provider = name\n\treturn b\n}\n\n// Credential sets the VolumeSnapshotLocation's credential selector.\nfunc (b *VolumeSnapshotLocationBuilder) Credential(selector *corev1api.SecretKeySelector) *VolumeSnapshotLocationBuilder {\n\tb.object.Spec.Credential = selector\n\treturn b\n}\n" }, { "path": "pkg/buildinfo/buildinfo.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\n// Package buildinfo holds build-time information like the velero version.\n// This is a separate package so that other packages can import it without\n// worrying about introducing circular dependencies.\npackage buildinfo\n\nimport \"fmt\"\n\nvar (\n\t// Version is the current version of Velero, set by the go linker's -X flag at build time.\n\tVersion string\n\n\t// GitSHA is the actual commit that is being built, set by the go linker's -X flag at build time.\n\tGitSHA string\n\n\t// GitTreeState indicates if the git tree is clean or dirty, set by the go linker's -X flag at build\n\t// time.\n\tGitTreeState string\n\n\t// ImageRegistry is the image registry that this build of Velero should use by default to pull the\n\t// Velero and Restore Helper images from.\n\tImageRegistry string\n)\n\n// FormattedGitSHA renders the Git SHA with an indicator of the tree state.\nfunc FormattedGitSHA() string {\n\tif GitTreeState != \"clean\" {\n\t\treturn fmt.Sprintf(\"%s-%s\", GitSHA, GitTreeState)\n\t}\n\treturn GitSHA\n}\n" }, { "path": "pkg/buildinfo/buildinfo_test.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage buildinfo\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestFormattedGitSHA(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tsha string\n\t\tstate string\n\t\texpected string\n\t}{\n\t\t{\n\t\t\t\"Clean git state has no suffix\",\n\t\t\t\"abc123\",\n\t\t\t\"clean\",\n\t\t\t\"abc123\",\n\t\t},\n\t\t{\n\t\t\t\"Dirty git status includes suffix\",\n\t\t\t\"abc123\",\n\t\t\t\"dirty\",\n\t\t\t\"abc123-dirty\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tGitSHA = test.sha\n\t\t\tGitTreeState = test.state\n\t\t\tassert.Equal(t, test.expected, FormattedGitSHA())\n\t\t})\n\t}\n}\n" }, { "path": "pkg/client/auth_providers.go", "content": "/*\nCopyright 2017 the Velero contributors.\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n http://www.apache.org/licenses/LICENSE-2.0\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage client\n\nimport (\n\t// Make sure we import the client-go auth provider plugins.\n\t_ \"k8s.io/client-go/plugin/pkg/client/auth/azure\"\n\t_ \"k8s.io/client-go/plugin/pkg/client/auth/gcp\"\n\t_ \"k8s.io/client-go/plugin/pkg/client/auth/oidc\"\n)\n" }, { "path": "pkg/client/client.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage client\n\nimport (\n\t\"fmt\"\n\t\"runtime\"\n\n\t\"github.com/pkg/errors\"\n\t\"k8s.io/client-go/rest\"\n\t\"k8s.io/client-go/tools/clientcmd\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/buildinfo\"\n)\n\n// Config returns a *rest.Config, using either the kubeconfig (if specified) or an in-cluster\n// configuration.\nfunc Config(kubeconfig, kubecontext, baseName string, qps float32, burst int) (*rest.Config, error) {\n\tloadingRules := clientcmd.NewDefaultClientConfigLoadingRules()\n\tloadingRules.ExplicitPath = kubeconfig\n\tconfigOverrides := &clientcmd.ConfigOverrides{CurrentContext: kubecontext}\n\tkubeConfig := clientcmd.NewNonInteractiveDeferredLoadingClientConfig(loadingRules, configOverrides)\n\n\tclientConfig, err := kubeConfig.ClientConfig()\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error finding Kubernetes API server config in --kubeconfig, $KUBECONFIG, or in-cluster configuration\")\n\t}\n\n\tif qps > 0.0 {\n\t\tclientConfig.QPS = qps\n\t}\n\tif burst > 0 {\n\t\tclientConfig.Burst = burst\n\t}\n\n\tclientConfig.UserAgent = buildUserAgent(\n\t\tbaseName,\n\t\tbuildinfo.Version,\n\t\tbuildinfo.FormattedGitSHA(),\n\t\truntime.GOOS,\n\t\truntime.GOARCH,\n\t)\n\n\treturn clientConfig, nil\n}\n\n// buildUserAgent builds a User-Agent string from given args.\nfunc buildUserAgent(command, version, formattedSha, os, arch string) string {\n\treturn fmt.Sprintf(\n\t\t\"%s/%s (%s/%s) %s\", command, version, os, arch, formattedSha)\n}\n" }, { "path": "pkg/client/client_test.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage client\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestBuildUserAgent(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tcommand string\n\t\tos string\n\t\tarch string\n\t\tgitSha string\n\t\tversion string\n\t\texpected string\n\t}{\n\t\t{\n\t\t\tname: \"Test general interpolation in correct order\",\n\t\t\tcommand: \"velero\",\n\t\t\tos: \"darwin\",\n\t\t\tarch: \"amd64\",\n\t\t\tgitSha: \"abc123\",\n\t\t\tversion: \"v0.1.1\",\n\t\t\texpected: \"velero/v0.1.1 (darwin/amd64) abc123\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tresp := buildUserAgent(test.command, test.version, test.gitSha, test.os, test.arch)\n\t\t\tassert.Equal(t, test.expected, resp)\n\t\t})\n\t}\n}\n\nfunc TestConfig(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tkubeconfig string\n\t\tkubecontext string\n\t\tQPS float32\n\t\tburst int\n\t\texpectedHost string\n\t}{\n\t\t{\n\t\t\tname: \"Test using the right cluster as context indexed\",\n\t\t\tkubeconfig: \"kubeconfig\",\n\t\t\tkubecontext: \"federal-context\",\n\t\t\tQPS: 1.0,\n\t\t\tburst: 1,\n\t\t\texpectedHost: \"https://horse.org:4443\",\n\t\t},\n\t\t{\n\t\t\tname: \"Test using the right cluster as context indexed\",\n\t\t\tkubeconfig: \"kubeconfig\",\n\t\t\tkubecontext: \"queen-anne-context\",\n\t\t\tQPS: 200.0,\n\t\t\tburst: 20,\n\t\t\texpectedHost: \"https://pig.org:443\",\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tclient, err := Config(test.kubeconfig, test.kubecontext, \"velero\", test.QPS, test.burst)\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Equal(t, test.expectedHost, client.Host)\n\t\t\tassert.Equal(t, test.QPS, client.QPS)\n\t\t\tassert.Equal(t, test.burst, client.Burst)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/client/config.go", "content": "/*\nCopyright 2021 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage client\n\nimport (\n\t\"encoding/json\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strconv\"\n\t\"strings\"\n\n\t\"github.com/pkg/errors\"\n)\n\nconst (\n\tConfigKeyNamespace = \"namespace\"\n\tConfigKeyFeatures = \"features\"\n\tConfigKeyCACert = \"cacert\"\n\tConfigKeyColorized = \"colorized\"\n)\n\n// VeleroConfig is a map of strings to any for deserializing Velero client config options.\n// The alias is a way to attach type-asserting convenience methods.\ntype VeleroConfig map[string]any\n\n// LoadConfig loads the Velero client configuration file and returns it as a VeleroConfig. If the\n// file does not exist, an empty map is returned.\nfunc LoadConfig() (VeleroConfig, error) {\n\tfileName := configFileName()\n\n\t_, err := os.Stat(fileName)\n\tif os.IsNotExist(err) {\n\t\t// If the file isn't there, just return an empty map\n\t\treturn VeleroConfig{}, nil\n\t}\n\tif err != nil {\n\t\t// For any other Stat() error, return it\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tconfigFile, err := os.Open(fileName)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\tdefer configFile.Close()\n\n\tvar config VeleroConfig\n\tif err := json.NewDecoder(configFile).Decode(&config); err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\treturn config, nil\n}\n\n// SaveConfig saves the passed in config map to the Velero client configuration file.\nfunc SaveConfig(config VeleroConfig) error {\n\tfileName := configFileName()\n\n\t// Try to make the directory in case it doesn't exist\n\tdir := filepath.Dir(fileName)\n\tif err := os.MkdirAll(dir, 0700); err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\tconfigFile, err := os.OpenFile(fileName, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0600)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\tdefer configFile.Close()\n\n\treturn json.NewEncoder(configFile).Encode(&config)\n}\n\nfunc (c VeleroConfig) Namespace() string {\n\tval, ok := c[ConfigKeyNamespace]\n\tif !ok {\n\t\treturn \"\"\n\t}\n\n\tns, ok := val.(string)\n\tif !ok {\n\t\treturn \"\"\n\t}\n\n\treturn ns\n}\n\nfunc (c VeleroConfig) Features() []string {\n\tval, ok := c[ConfigKeyFeatures]\n\tif !ok {\n\t\treturn []string{}\n\t}\n\n\tfeatures, ok := val.(string)\n\tif !ok {\n\t\treturn []string{}\n\t}\n\n\treturn strings.Split(features, \",\")\n}\n\nfunc (c VeleroConfig) Colorized() bool {\n\tval, ok := c[ConfigKeyColorized]\n\tif !ok {\n\t\treturn true\n\t}\n\n\tvalString, ok := val.(string)\n\tif !ok {\n\t\treturn true\n\t}\n\n\tcolorized, err := strconv.ParseBool(valString)\n\tif err != nil {\n\t\treturn true\n\t}\n\n\treturn colorized\n}\n\nfunc (c VeleroConfig) CACertFile() string {\n\tval, ok := c[ConfigKeyCACert]\n\tif !ok {\n\t\treturn \"\"\n\t}\n\tcaCertFile, ok := val.(string)\n\tif !ok {\n\t\treturn \"\"\n\t}\n\n\treturn caCertFile\n}\n\nfunc configFileName() string {\n\treturn filepath.Join(os.Getenv(\"HOME\"), \".config\", \"velero\", \"config.json\")\n}\n" }, { "path": "pkg/client/config_test.go", "content": "/*\nCopyright 2021 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage client\n\nimport (\n\t\"os\"\n\t\"reflect\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestVeleroConfig(t *testing.T) {\n\tc := VeleroConfig{\n\t\t\"namespace\": \"foo\",\n\t\t\"features\": \"feature1,feature2\",\n\t}\n\n\tassert.Equal(t, \"foo\", c.Namespace())\n\tassert.Equal(t, []string{\"feature1\", \"feature2\"}, c.Features())\n\tassert.True(t, c.Colorized())\n}\n\nfunc removeConfigfileName() error {\n\t// Remove config file if it exist\n\tconfigFile := configFileName()\n\te := os.Remove(configFile)\n\tif e != nil {\n\t\tif !os.IsNotExist(e) {\n\t\t\treturn e\n\t\t}\n\t}\n\treturn nil\n}\nfunc TestConfigOperations(t *testing.T) {\n\tpreHomeEnv := \"\"\n\tprevEnv := os.Environ()\n\tfor _, entry := range prevEnv {\n\t\tparts := strings.SplitN(entry, \"=\", 2)\n\t\tif len(parts) == 2 && parts[0] == \"HOME\" {\n\t\t\tpreHomeEnv = parts[1]\n\t\t\tbreak\n\t\t}\n\t}\n\tos.Unsetenv(\"HOME\")\n\tos.Setenv(\"HOME\", \".\")\n\n\t// Remove config file if it exists\n\terr := removeConfigfileName()\n\trequire.NoError(t, err)\n\n\t// Test LoadConfig: expect an empty velero config\n\texpectedConfig := VeleroConfig{}\n\tconfig, err := LoadConfig()\n\n\trequire.NoError(t, err)\n\tassert.True(t, reflect.DeepEqual(expectedConfig, config))\n\n\t// Test savedConfig\n\texpectedFeature := \"EnableCSI\"\n\texpectedColorized := true\n\texpectedNamespace := \"ns-velero\"\n\texpectedCACert := \"ca-cert\"\n\n\tconfig[ConfigKeyFeatures] = expectedFeature\n\tconfig[ConfigKeyColorized] = expectedColorized\n\tconfig[ConfigKeyNamespace] = expectedNamespace\n\tconfig[ConfigKeyCACert] = expectedCACert\n\n\terr = SaveConfig(config)\n\n\trequire.NoError(t, err)\n\tsavedConfig, err := LoadConfig()\n\trequire.NoError(t, err)\n\n\t// Test Features\n\tfeature := savedConfig.Features()\n\tassert.Len(t, feature, 1)\n\tassert.Equal(t, expectedFeature, feature[0])\n\n\t// Test Colorized\n\tcolorized := savedConfig.Colorized()\n\tassert.Equal(t, expectedColorized, colorized)\n\n\t// Test Namespace\n\tnamespace := savedConfig.Namespace()\n\tassert.Equal(t, expectedNamespace, namespace)\n\n\t// Test Features\n\tcaCertFile := savedConfig.CACertFile()\n\tassert.Equal(t, expectedCACert, caCertFile)\n\n\tt.Cleanup(func() {\n\t\terr = removeConfigfileName()\n\t\trequire.NoError(t, err)\n\t\tos.Unsetenv(\"HOME\")\n\t\tos.Setenv(\"HOME\", preHomeEnv)\n\t})\n}\n" }, { "path": "pkg/client/dynamic.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage client\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/watch\"\n\t\"k8s.io/client-go/dynamic\"\n\t\"k8s.io/client-go/dynamic/dynamicinformer\"\n)\n\n// DynamicFactory contains methods for retrieving dynamic clients for GroupVersionResources and\n// GroupVersionKinds.\ntype DynamicFactory interface {\n\t// ClientForGroupVersionResource returns a Dynamic client for the given group/version\n\t// and resource for the given namespace.\n\tClientForGroupVersionResource(gv schema.GroupVersion, resource metav1.APIResource, namespace string) (Dynamic, error)\n\t// DynamicSharedInformerFactory returns a DynamicSharedInformerFactory.\n\tDynamicSharedInformerFactory() dynamicinformer.DynamicSharedInformerFactory\n}\n\n// dynamicFactory implements DynamicFactory.\ntype dynamicFactory struct {\n\tdynamicClient dynamic.Interface\n}\n\n// NewDynamicFactory returns a new ClientPool-based dynamic factory.\nfunc NewDynamicFactory(dynamicClient dynamic.Interface) DynamicFactory {\n\treturn &dynamicFactory{dynamicClient: dynamicClient}\n}\n\nfunc (f *dynamicFactory) ClientForGroupVersionResource(gv schema.GroupVersion, resource metav1.APIResource, namespace string) (Dynamic, error) {\n\treturn &dynamicResourceClient{\n\t\tresourceClient: f.dynamicClient.Resource(gv.WithResource(resource.Name)).Namespace(namespace),\n\t}, nil\n}\n\nfunc (f *dynamicFactory) DynamicSharedInformerFactory() dynamicinformer.DynamicSharedInformerFactory {\n\treturn dynamicinformer.NewDynamicSharedInformerFactory(f.dynamicClient, time.Minute)\n}\n\n// Creator creates an object.\ntype Creator interface {\n\t// Create creates an object.\n\tCreate(obj *unstructured.Unstructured) (*unstructured.Unstructured, error)\n}\n\n// Lister lists objects.\ntype Lister interface {\n\t// List lists all the objects of a given resource.\n\tList(metav1.ListOptions) (*unstructured.UnstructuredList, error)\n}\n\n// Watcher watches objects.\ntype Watcher interface {\n\t// Watch watches for changes to objects of a given resource.\n\tWatch(metav1.ListOptions) (watch.Interface, error)\n}\n\n// Getter gets an object.\ntype Getter interface {\n\t// Get fetches an object by name.\n\tGet(name string, opts metav1.GetOptions) (*unstructured.Unstructured, error)\n}\n\n// Patcher patches an object.\ntype Patcher interface {\n\t//Patch patches the named object using the provided patch bytes, which are expected to be in JSON merge patch format. The patched object is returned.\n\n\tPatch(name string, data []byte) (*unstructured.Unstructured, error)\n}\n\n// Deletor deletes an object.\ntype Deletor interface {\n\t//Patch patches the named object using the provided patch bytes, which are expected to be in JSON merge patch format. The patched object is returned.\n\n\tDelete(name string, opts metav1.DeleteOptions) error\n}\n\n// StatusUpdater updates status field of a object\ntype StatusUpdater interface {\n\tUpdateStatus(obj *unstructured.Unstructured, opts metav1.UpdateOptions) (*unstructured.Unstructured, error)\n}\n\n// Applier applies changes to an object using server-side apply\ntype Applier interface {\n\tApply(name string, obj *unstructured.Unstructured, opts metav1.ApplyOptions) (*unstructured.Unstructured, error)\n}\n\n// Dynamic contains client methods that Velero needs for backing up and restoring resources.\ntype Dynamic interface {\n\tCreator\n\tLister\n\tWatcher\n\tGetter\n\tPatcher\n\tDeletor\n\tStatusUpdater\n\tApplier\n}\n\n// dynamicResourceClient implements Dynamic.\ntype dynamicResourceClient struct {\n\tresourceClient dynamic.ResourceInterface\n}\n\nvar _ Dynamic = &dynamicResourceClient{}\n\nfunc (d *dynamicResourceClient) Create(obj *unstructured.Unstructured) (*unstructured.Unstructured, error) {\n\treturn d.resourceClient.Create(context.TODO(), obj, metav1.CreateOptions{})\n}\n\nfunc (d *dynamicResourceClient) List(options metav1.ListOptions) (*unstructured.UnstructuredList, error) {\n\treturn d.resourceClient.List(context.TODO(), options)\n}\n\nfunc (d *dynamicResourceClient) Watch(options metav1.ListOptions) (watch.Interface, error) {\n\treturn d.resourceClient.Watch(context.TODO(), options)\n}\n\nfunc (d *dynamicResourceClient) Get(name string, opts metav1.GetOptions) (*unstructured.Unstructured, error) {\n\treturn d.resourceClient.Get(context.TODO(), name, opts)\n}\n\nfunc (d *dynamicResourceClient) Apply(name string, obj *unstructured.Unstructured, opts metav1.ApplyOptions) (*unstructured.Unstructured, error) {\n\treturn d.resourceClient.Apply(context.TODO(), name, obj, opts)\n}\n\nfunc (d *dynamicResourceClient) Patch(name string, data []byte) (*unstructured.Unstructured, error) {\n\treturn d.resourceClient.Patch(context.TODO(), name, types.MergePatchType, data, metav1.PatchOptions{})\n}\n\nfunc (d *dynamicResourceClient) Delete(name string, opts metav1.DeleteOptions) error {\n\treturn d.resourceClient.Delete(context.TODO(), name, opts)\n}\n\nfunc (d *dynamicResourceClient) UpdateStatus(obj *unstructured.Unstructured, opts metav1.UpdateOptions) (*unstructured.Unstructured, error) {\n\treturn d.resourceClient.UpdateStatus(context.TODO(), obj, opts)\n}\n" }, { "path": "pkg/client/factory.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage client\n\nimport (\n\t\"os\"\n\n\tvolumegroupsnapshotv1beta1 \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumegroupsnapshot/v1beta1\"\n\n\tapiextv1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1\"\n\tapiextv1beta1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1\"\n\t\"k8s.io/client-go/discovery\"\n\tk8scheme \"k8s.io/client-go/kubernetes/scheme\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/spf13/pflag\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/dynamic\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"k8s.io/client-go/rest\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n)\n\n//go:generate mockery --name Factory\n\n// Factory knows how to create a VeleroClient and Kubernetes client.\ntype Factory interface {\n\t// BindFlags binds common flags (--kubeconfig, --namespace) to the passed-in FlagSet.\n\tBindFlags(flags *pflag.FlagSet)\n\t// KubeClient returns a Kubernetes client. It uses the following priority to specify the cluster\n\t// configuration: --kubeconfig flag, KUBECONFIG environment variable, in-cluster configuration.\n\tKubeClient() (kubernetes.Interface, error)\n\t// DynamicClient returns a Kubernetes dynamic client. It uses the following priority to specify the cluster\n\t// configuration: --kubeconfig flag, KUBECONFIG environment variable, in-cluster configuration.\n\tDynamicClient() (dynamic.Interface, error)\n\t// KubebuilderClient returns a client for the controller runtime framework. It adds Kubernetes and Velero\n\t// types to its scheme. It uses the following priority to specify the cluster\n\t// configuration: --kubeconfig flag, KUBECONFIG environment variable, in-cluster configuration.\n\tKubebuilderClient() (kbclient.Client, error)\n\t// KubebuilderWatchClient returns a client with watcher for the controller runtime framework.\n\t// It adds Kubernetes and Velero types to its scheme. It uses the following priority to specify the cluster\n\t// configuration: --kubeconfig flag, KUBECONFIG environment variable, in-cluster configuration.\n\tKubebuilderWatchClient() (kbclient.WithWatch, error)\n\t// DiscoveryClient returns a Kubernetes discovery client. It uses the following priority to specify the cluster\n\t// configuration: --kubeconfig flag, KUBECONFIG environment variable, in-cluster configuration.\n\tDiscoveryClient() (discovery.AggregatedDiscoveryInterface, error)\n\t// SetBasename changes the basename for an already-constructed client.\n\t// This is useful for generating clients that require a different user-agent string below the root `velero`\n\t// command, such as the server subcommand.\n\tSetBasename(string)\n\t// SetClientQPS sets the Queries Per Second for a client.\n\tSetClientQPS(float32)\n\t// SetClientBurst sets the Burst for a client.\n\tSetClientBurst(int)\n\t// ClientConfig returns a rest.Config struct used for client-go clients.\n\tClientConfig() (*rest.Config, error)\n\t// Namespace returns the namespace which the Factory will create clients for.\n\tNamespace() string\n}\n\ntype factory struct {\n\tflags *pflag.FlagSet\n\tkubeconfig string\n\tkubecontext string\n\tbaseName string\n\tnamespace string\n\tclientQPS float32\n\tclientBurst int\n}\n\n// NewFactory returns a Factory.\nfunc NewFactory(baseName string, config VeleroConfig) Factory {\n\tf := &factory{\n\t\tflags: pflag.NewFlagSet(\"\", pflag.ContinueOnError),\n\t\tbaseName: baseName,\n\t}\n\n\tf.namespace = os.Getenv(\"VELERO_NAMESPACE\")\n\tif config.Namespace() != \"\" {\n\t\tf.namespace = config.Namespace()\n\t}\n\n\t// We didn't get the namespace via env var or config file, so use the default.\n\t// Command line flags will override when BindFlags is called.\n\tif f.namespace == \"\" {\n\t\tf.namespace = velerov1api.DefaultNamespace\n\t}\n\n\tf.flags.StringVar(&f.kubeconfig, \"kubeconfig\", \"\", \"Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration\")\n\tf.flags.StringVarP(&f.namespace, \"namespace\", \"n\", f.namespace, \"The namespace in which Velero should operate\")\n\tf.flags.StringVar(&f.kubecontext, \"kubecontext\", \"\", \"The context to use to talk to the Kubernetes apiserver. If unset defaults to whatever your current-context is (kubectl config current-context)\")\n\treturn f\n}\n\nfunc (f *factory) BindFlags(flags *pflag.FlagSet) {\n\tflags.AddFlagSet(f.flags)\n}\n\nfunc (f *factory) ClientConfig() (*rest.Config, error) {\n\treturn Config(f.kubeconfig, f.kubecontext, f.baseName, f.clientQPS, f.clientBurst)\n}\n\nfunc (f *factory) KubeClient() (kubernetes.Interface, error) {\n\tclientConfig, err := f.ClientConfig()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tkubeClient, err := kubernetes.NewForConfig(clientConfig)\n\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\treturn kubeClient, nil\n}\n\nfunc (f *factory) DynamicClient() (dynamic.Interface, error) {\n\tclientConfig, err := f.ClientConfig()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdynamicClient, err := dynamic.NewForConfig(clientConfig)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\treturn dynamicClient, nil\n}\n\nfunc (f *factory) KubebuilderClient() (kbclient.Client, error) {\n\tclientConfig, err := f.ClientConfig()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tscheme := runtime.NewScheme()\n\tif err := velerov1api.AddToScheme(scheme); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := velerov2alpha1api.AddToScheme(scheme); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := k8scheme.AddToScheme(scheme); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := apiextv1beta1.AddToScheme(scheme); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := apiextv1.AddToScheme(scheme); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := snapshotv1api.AddToScheme(scheme); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := volumegroupsnapshotv1beta1.AddToScheme(scheme); err != nil {\n\t\treturn nil, err\n\t}\n\tkubebuilderClient, err := kbclient.New(clientConfig, kbclient.Options{\n\t\tScheme: scheme,\n\t})\n\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn kubebuilderClient, nil\n}\n\nfunc (f *factory) KubebuilderWatchClient() (kbclient.WithWatch, error) {\n\tclientConfig, err := f.ClientConfig()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tscheme := runtime.NewScheme()\n\tif err := velerov1api.AddToScheme(scheme); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := velerov2alpha1api.AddToScheme(scheme); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := k8scheme.AddToScheme(scheme); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := apiextv1beta1.AddToScheme(scheme); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := apiextv1.AddToScheme(scheme); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := snapshotv1api.AddToScheme(scheme); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := volumegroupsnapshotv1beta1.AddToScheme(scheme); err != nil {\n\t\treturn nil, err\n\t}\n\tkubebuilderWatchClient, err := kbclient.NewWithWatch(clientConfig, kbclient.Options{\n\t\tScheme: scheme,\n\t})\n\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn kubebuilderWatchClient, nil\n}\n\nfunc (f *factory) DiscoveryClient() (discovery.AggregatedDiscoveryInterface, error) {\n\tclientConfig, err := f.ClientConfig()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn discovery.NewDiscoveryClientForConfig(clientConfig)\n}\n\nfunc (f *factory) SetBasename(name string) {\n\tf.baseName = name\n}\n\nfunc (f *factory) SetClientQPS(qps float32) {\n\tf.clientQPS = qps\n}\n\nfunc (f *factory) SetClientBurst(burst int) {\n\tf.clientBurst = burst\n}\n\nfunc (f *factory) Namespace() string {\n\treturn f.namespace\n}\n" }, { "path": "pkg/client/factory_test.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage client\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"strings\"\n\t\"testing\"\n\n\tflag \"github.com/spf13/pflag\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n)\n\n// TestFactory tests the client.Factory interface.\nfunc TestFactory(t *testing.T) {\n\t// Velero client configuration is currently omitted due to requiring a\n\t// test filesystem in pkg/test. This causes an import cycle as pkg/test\n\t// uses pkg/client's interfaces to implement fakes\n\n\t// Env variable should set the namespace if no config or argument are used\n\tos.Setenv(\"VELERO_NAMESPACE\", \"env-velero\")\n\tf := NewFactory(\"velero\", make(map[string]any))\n\n\tassert.Equal(t, \"env-velero\", f.Namespace())\n\n\tos.Unsetenv(\"VELERO_NAMESPACE\")\n\n\t// Argument should change the namespace\n\tf = NewFactory(\"velero\", make(map[string]any))\n\ts := \"flag-velero\"\n\tflags := new(flag.FlagSet)\n\n\tf.BindFlags(flags)\n\n\tflags.Parse([]string{\"--namespace\", s})\n\n\tassert.Equal(t, s, f.Namespace())\n\n\t// An argument overrides the env variable if both are set.\n\tos.Setenv(\"VELERO_NAMESPACE\", \"env-velero\")\n\tf = NewFactory(\"velero\", make(map[string]any))\n\tflags = new(flag.FlagSet)\n\n\tf.BindFlags(flags)\n\tflags.Parse([]string{\"--namespace\", s})\n\tassert.Equal(t, s, f.Namespace())\n\n\tos.Unsetenv(\"VELERO_NAMESPACE\")\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeconfig string\n\t\tkubecontext string\n\t\tQPS float32\n\t\tburst int\n\t\tbaseName string\n\t\texpectedHost string\n\t}{\n\t\t{\n\t\t\tname: \"Test flag setting in factory ClientConfig (test data #1)\",\n\t\t\tkubeconfig: \"kubeconfig\",\n\t\t\tkubecontext: \"federal-context\",\n\t\t\tQPS: 1.0,\n\t\t\tburst: 1,\n\t\t\tbaseName: \"bn-velero-1\",\n\t\t\texpectedHost: \"https://horse.org:4443\",\n\t\t},\n\t\t{\n\t\t\tname: \"Test flag setting in factory ClientConfig (test data #2)\",\n\t\t\tkubeconfig: \"kubeconfig\",\n\t\t\tkubecontext: \"queen-anne-context\",\n\t\t\tQPS: 200.0,\n\t\t\tburst: 20,\n\t\t\tbaseName: \"bn-velero-2\",\n\t\t\texpectedHost: \"https://pig.org:443\",\n\t\t},\n\t}\n\n\tbaseName := \"velero-bn\"\n\tconfig, err := LoadConfig()\n\trequire.NoError(t, err)\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tf = NewFactory(baseName, config)\n\t\t\tf.SetClientBurst(test.burst)\n\t\t\tf.SetClientQPS(test.QPS)\n\t\t\tf.SetBasename(test.baseName)\n\n\t\t\tflags = new(flag.FlagSet)\n\t\t\tf.BindFlags(flags)\n\t\t\tflags.Parse([]string{\"--kubeconfig\", test.kubeconfig, \"--kubecontext\", test.kubecontext})\n\t\t\tclientConfig, _ := f.ClientConfig()\n\t\t\tassert.Equal(t, test.expectedHost, clientConfig.Host)\n\t\t\tassert.Equal(t, test.QPS, clientConfig.QPS)\n\t\t\tassert.Equal(t, test.burst, clientConfig.Burst)\n\t\t\tstrings.Contains(clientConfig.UserAgent, test.baseName)\n\n\t\t\tkubeClient, _ := f.KubeClient()\n\t\t\tgroup := kubeClient.NodeV1().RESTClient().APIVersion().Group\n\t\t\tassert.NotNil(t, kubeClient)\n\t\t\tassert.Equal(t, \"node.k8s.io\", group)\n\n\t\t\tnamespace := \"ns1\"\n\t\t\tdynamicClient, _ := f.DynamicClient()\n\t\t\tresource := &schema.GroupVersionResource{\n\t\t\t\tGroup: \"group_test\",\n\t\t\t\tVersion: \"verion_test\",\n\t\t\t}\n\t\t\tlist, e := dynamicClient.Resource(*resource).Namespace(namespace).List(\n\t\t\t\tt.Context(),\n\t\t\t\tmetav1.ListOptions{\n\t\t\t\t\tLabelSelector: \"none\",\n\t\t\t\t},\n\t\t\t)\n\t\t\trequire.ErrorContains(t, e, fmt.Sprintf(\"Get \\\"%s/apis/%s/%s/namespaces/%s\", test.expectedHost, resource.Group, resource.Version, namespace))\n\t\t\tassert.Nil(t, list)\n\t\t\tassert.NotNil(t, dynamicClient)\n\n\t\t\tkubebuilderClient, e := f.KubebuilderClient()\n\t\t\trequire.NoError(t, e)\n\t\t\tassert.NotNil(t, kubebuilderClient)\n\n\t\t\tkbClientWithWatch, e := f.KubebuilderWatchClient()\n\t\t\trequire.NoError(t, e)\n\t\t\tassert.NotNil(t, kbClientWithWatch)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/client/kubeconfig", "content": "\ncurrent-context: federal-context\napiVersion: v1\nclusters:\n- cluster:\n api-version: v1\n server: http://cow.org:8080\n name: cow-cluster\n- cluster:\n server: https://horse.org:4443\n name: horse-cluster\n- cluster:\n insecure-skip-tls-verify: true\n server: https://pig.org:443\n name: pig-cluster\ncontexts:\n- context:\n cluster: horse-cluster\n namespace: chisel-ns\n user: green-user\n name: federal-context\n- context:\n cluster: pig-cluster\n namespace: saw-ns\n user: black-user\n name: queen-anne-context\nkind: Config\npreferences:\n colors: true\nusers:\n- name: blue-user\n user:\n token: blue-token\n- name: green-user\n user:\n" }, { "path": "pkg/client/mocks/Factory.go", "content": "// Code generated by mockery v2.28.1. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tdiscovery \"k8s.io/client-go/discovery\"\n\tdynamic \"k8s.io/client-go/dynamic\"\n\n\tkubernetes \"k8s.io/client-go/kubernetes\"\n\n\tmock \"github.com/stretchr/testify/mock\"\n\n\tpflag \"github.com/spf13/pflag\"\n\n\tpkgclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\trest \"k8s.io/client-go/rest\"\n)\n\n// Factory is an autogenerated mock type for the Factory type\ntype Factory struct {\n\tmock.Mock\n}\n\n// BindFlags provides a mock function with given fields: flags\nfunc (_m *Factory) BindFlags(flags *pflag.FlagSet) {\n\t_m.Called(flags)\n}\n\n// ClientConfig provides a mock function with given fields:\nfunc (_m *Factory) ClientConfig() (*rest.Config, error) {\n\tret := _m.Called()\n\n\tvar r0 *rest.Config\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func() (*rest.Config, error)); ok {\n\t\treturn rf()\n\t}\n\tif rf, ok := ret.Get(0).(func() *rest.Config); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(*rest.Config)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// DiscoveryClient provides a mock function with given fields:\nfunc (_m *Factory) DiscoveryClient() (discovery.AggregatedDiscoveryInterface, error) {\n\tret := _m.Called()\n\n\tvar r0 discovery.AggregatedDiscoveryInterface\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func() (discovery.AggregatedDiscoveryInterface, error)); ok {\n\t\treturn rf()\n\t}\n\tif rf, ok := ret.Get(0).(func() discovery.AggregatedDiscoveryInterface); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(discovery.AggregatedDiscoveryInterface)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// DynamicClient provides a mock function with given fields:\nfunc (_m *Factory) DynamicClient() (dynamic.Interface, error) {\n\tret := _m.Called()\n\n\tvar r0 dynamic.Interface\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func() (dynamic.Interface, error)); ok {\n\t\treturn rf()\n\t}\n\tif rf, ok := ret.Get(0).(func() dynamic.Interface); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(dynamic.Interface)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// KubeClient provides a mock function with given fields:\nfunc (_m *Factory) KubeClient() (kubernetes.Interface, error) {\n\tret := _m.Called()\n\n\tvar r0 kubernetes.Interface\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func() (kubernetes.Interface, error)); ok {\n\t\treturn rf()\n\t}\n\tif rf, ok := ret.Get(0).(func() kubernetes.Interface); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(kubernetes.Interface)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// KubebuilderClient provides a mock function with given fields:\nfunc (_m *Factory) KubebuilderClient() (pkgclient.Client, error) {\n\tret := _m.Called()\n\n\tvar r0 pkgclient.Client\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func() (pkgclient.Client, error)); ok {\n\t\treturn rf()\n\t}\n\tif rf, ok := ret.Get(0).(func() pkgclient.Client); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(pkgclient.Client)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// KubebuilderWatchClient provides a mock function with given fields:\nfunc (_m *Factory) KubebuilderWatchClient() (pkgclient.WithWatch, error) {\n\tret := _m.Called()\n\n\tvar r0 pkgclient.WithWatch\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func() (pkgclient.WithWatch, error)); ok {\n\t\treturn rf()\n\t}\n\tif rf, ok := ret.Get(0).(func() pkgclient.WithWatch); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(pkgclient.WithWatch)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Namespace provides a mock function with given fields:\nfunc (_m *Factory) Namespace() string {\n\tret := _m.Called()\n\n\tvar r0 string\n\tif rf, ok := ret.Get(0).(func() string); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(string)\n\t}\n\n\treturn r0\n}\n\n// SetBasename provides a mock function with given fields: _a0\nfunc (_m *Factory) SetBasename(_a0 string) {\n\t_m.Called(_a0)\n}\n\n// SetClientBurst provides a mock function with given fields: _a0\nfunc (_m *Factory) SetClientBurst(_a0 int) {\n\t_m.Called(_a0)\n}\n\n// SetClientQPS provides a mock function with given fields: _a0\nfunc (_m *Factory) SetClientQPS(_a0 float32) {\n\t_m.Called(_a0)\n}\n\ntype mockConstructorTestingTNewFactory interface {\n\tmock.TestingT\n\tCleanup(func())\n}\n\n// NewFactory creates a new instance of Factory. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\nfunc NewFactory(t mockConstructorTestingTNewFactory) *Factory {\n\tmock := &Factory{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/client/retry.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage client\n\nimport (\n\t\"context\"\n\t\"math\"\n\t\"time\"\n\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"k8s.io/client-go/util/retry\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n)\n\nfunc CreateRetryGenerateName(client kbclient.Client, ctx context.Context, obj kbclient.Object) error {\n\tretryCreateFn := func() error {\n\t\t// needed to ensure that the name from the failed create isn't left on the object between retries\n\t\tobj.SetName(\"\")\n\t\treturn client.Create(ctx, obj, &kbclient.CreateOptions{})\n\t}\n\tif obj.GetGenerateName() != \"\" && obj.GetName() == \"\" {\n\t\treturn retry.OnError(retry.DefaultRetry, apierrors.IsAlreadyExists, retryCreateFn)\n\t} else {\n\t\treturn client.Create(ctx, obj, &kbclient.CreateOptions{})\n\t}\n}\n\n// CapBackoff provides a backoff with a set backoff cap\nfunc CapBackoff(cap time.Duration) wait.Backoff {\n\tif cap < 0 {\n\t\tcap = 0\n\t}\n\treturn wait.Backoff{\n\t\tSteps: math.MaxInt,\n\t\tDuration: 10 * time.Millisecond,\n\t\tCap: cap,\n\t\tFactor: retry.DefaultBackoff.Factor,\n\t\tJitter: retry.DefaultBackoff.Jitter,\n\t}\n}\n\n// RetryOnRetriableMaxBackOff accepts a patch function param, retrying when the provided retriable function returns true.\nfunc RetryOnRetriableMaxBackOff(maxDuration time.Duration, fn func() error, retriable func(error) bool) error {\n\treturn retry.OnError(CapBackoff(maxDuration), func(err error) bool { return retriable(err) }, fn)\n}\n\n// RetryOnErrorMaxBackOff accepts a patch function param, retrying when the error is not nil.\nfunc RetryOnErrorMaxBackOff(maxDuration time.Duration, fn func() error) error {\n\treturn RetryOnRetriableMaxBackOff(maxDuration, fn, func(err error) bool { return err != nil })\n}\n" }, { "path": "pkg/cmd/cli/backup/backup.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"github.com/spf13/cobra\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n)\n\nfunc NewCommand(f client.Factory) *cobra.Command {\n\tc := &cobra.Command{\n\t\tUse: \"backup\",\n\t\tShort: \"Work with backups\",\n\t\tLong: \"Work with backups\",\n\t}\n\n\tc.AddCommand(\n\t\tNewCreateCommand(f, \"create\"),\n\t\tNewGetCommand(f, \"get\"),\n\t\tNewLogsCommand(f),\n\t\tNewDescribeCommand(f, \"describe\"),\n\t\tNewDownloadCommand(f),\n\t\tNewDeleteCommand(f, \"delete\"),\n\t)\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/backup/backup_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n)\n\nfunc TestNewBackupCommand(t *testing.T) {\n\t// create a factory\n\tf := &factorymocks.Factory{}\n\n\t// create command\n\tcmd := NewCommand(f)\n\tassert.Equal(t, \"Work with backups\", cmd.Short)\n}\n" }, { "path": "pkg/cmd/cli/backup/create.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/spf13/cobra\"\n\t\"github.com/spf13/pflag\"\n\tkubeerrs \"k8s.io/apimachinery/pkg/util/errors\"\n\t\"k8s.io/client-go/tools/cache\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/flag\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/output\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/collections\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nfunc NewCreateCommand(f client.Factory, use string) *cobra.Command {\n\to := NewCreateOptions()\n\n\tc := &cobra.Command{\n\t\tUse: use + \" NAME\",\n\t\tShort: \"Create a backup\",\n\t\tArgs: cobra.MaximumNArgs(1),\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tcmd.CheckError(o.Complete(args, f))\n\t\t\tcmd.CheckError(o.Validate(c, args, f))\n\t\t\tcmd.CheckError(o.Run(c, f))\n\t\t},\n\t\tExample: ` # Create a backup containing all resources.\n velero backup create backup1\n\n # Create a backup including only the nginx namespace.\n velero backup create nginx-backup --include-namespaces nginx\n\n # Create a backup excluding the velero and default namespaces.\n velero backup create backup2 --exclude-namespaces velero,default\n\n # Create a backup based on a schedule named daily-backup.\n velero backup create --from-schedule daily-backup\n\n # View the YAML for a backup that doesn't snapshot volumes, without sending it to the server.\n velero backup create backup3 --snapshot-volumes=false -o yaml\n\n # Wait for a backup to complete before returning from the command.\n velero backup create backup4 --wait`,\n\t}\n\n\to.BindFlags(c.Flags())\n\to.BindWait(c.Flags())\n\to.BindFromSchedule(c.Flags())\n\toutput.BindFlags(c.Flags())\n\toutput.ClearOutputFlagDefault(c)\n\n\treturn c\n}\n\ntype CreateOptions struct {\n\tName string\n\tTTL time.Duration\n\tSnapshotVolumes flag.OptionalBool\n\tSnapshotMoveData flag.OptionalBool\n\tDataMover string\n\tDefaultVolumesToFsBackup flag.OptionalBool\n\tIncludeNamespaces flag.StringArray\n\tExcludeNamespaces flag.StringArray\n\tIncludeResources flag.StringArray\n\tExcludeResources flag.StringArray\n\tIncludeClusterScopedResources flag.StringArray\n\tExcludeClusterScopedResources flag.StringArray\n\tIncludeNamespaceScopedResources flag.StringArray\n\tExcludeNamespaceScopedResources flag.StringArray\n\tLabels flag.Map\n\tAnnotations flag.Map\n\tSelector flag.LabelSelector\n\tOrSelector flag.OrLabelSelector\n\tIncludeClusterResources flag.OptionalBool\n\tWait bool\n\tStorageLocation string\n\tSnapshotLocations []string\n\tFromSchedule string\n\tOrderedResources string\n\tCSISnapshotTimeout time.Duration\n\tItemOperationTimeout time.Duration\n\tResPoliciesConfigmap string\n\tclient kbclient.WithWatch\n\tParallelFilesUpload int\n}\n\nfunc NewCreateOptions() *CreateOptions {\n\treturn &CreateOptions{\n\t\tIncludeNamespaces: flag.NewStringArray(\"*\"),\n\t\tLabels: flag.NewMap(),\n\t\tAnnotations: flag.NewMap(),\n\t\tSnapshotVolumes: flag.NewOptionalBool(nil),\n\t\tIncludeClusterResources: flag.NewOptionalBool(nil),\n\t}\n}\n\nfunc (o *CreateOptions) BindFlags(flags *pflag.FlagSet) {\n\tflags.DurationVar(&o.TTL, \"ttl\", o.TTL, \"How long before the backup can be garbage collected.\")\n\tflags.Var(&o.IncludeNamespaces, \"include-namespaces\", \"Namespaces to include in the backup (use '*' for all namespaces).\")\n\tflags.Var(&o.ExcludeNamespaces, \"exclude-namespaces\", \"Namespaces to exclude from the backup.\")\n\tflags.Var(&o.IncludeResources, \"include-resources\", \"Resources to include in the backup, formatted as resource.group, such as storageclasses.storage.k8s.io (use '*' for all resources). Cannot work with include-cluster-scoped-resources, exclude-cluster-scoped-resources, include-namespace-scoped-resources and exclude-namespace-scoped-resources.\")\n\tflags.Var(&o.ExcludeResources, \"exclude-resources\", \"Resources to exclude from the backup, formatted as resource.group, such as storageclasses.storage.k8s.io. Cannot work with include-cluster-scoped-resources, exclude-cluster-scoped-resources, include-namespace-scoped-resources and exclude-namespace-scoped-resources.\")\n\tflags.Var(&o.IncludeClusterScopedResources, \"include-cluster-scoped-resources\", \"Cluster-scoped resources to include in the backup, formatted as resource.group, such as storageclasses.storage.k8s.io(use '*' for all resources). Cannot work with include-resources, exclude-resources and include-cluster-resources.\")\n\tflags.Var(&o.ExcludeClusterScopedResources, \"exclude-cluster-scoped-resources\", \"Cluster-scoped resources to exclude from the backup, formatted as resource.group, such as storageclasses.storage.k8s.io(use '*' for all resources). Cannot work with include-resources, exclude-resources and include-cluster-resources.\")\n\tflags.Var(&o.IncludeNamespaceScopedResources, \"include-namespace-scoped-resources\", \"Namespaced resources to include in the backup, formatted as resource.group, such as deployments.apps(use '*' for all resources). Cannot work with include-resources, exclude-resources and include-cluster-resources.\")\n\tflags.Var(&o.ExcludeNamespaceScopedResources, \"exclude-namespace-scoped-resources\", \"Namespaced resources to exclude from the backup, formatted as resource.group, such as deployments.apps(use '*' for all resources). Cannot work with include-resources, exclude-resources and include-cluster-resources.\")\n\tflags.Var(&o.Labels, \"labels\", \"Labels to apply to the backup.\")\n\tflags.Var(&o.Annotations, \"annotations\", \"Annotations to apply to the backup.\")\n\tflags.StringVar(&o.StorageLocation, \"storage-location\", \"\", \"Location in which to store the backup.\")\n\tflags.StringSliceVar(&o.SnapshotLocations, \"volume-snapshot-locations\", o.SnapshotLocations, \"List of locations (at most one per provider) where volume snapshots should be stored.\")\n\tflags.VarP(&o.Selector, \"selector\", \"l\", \"Only back up resources matching this label selector.\")\n\tflags.Var(&o.OrSelector, \"or-selector\", \"Backup resources matching at least one of the label selector from the list. Label selectors should be separated by ' or '. For example, foo=bar or app=nginx\")\n\tflags.StringVar(&o.OrderedResources, \"ordered-resources\", \"\", \"Mapping Kinds to an ordered list of specific resources of that Kind. Resource names are separated by commas and their names are in format 'namespace/resourcename'. For cluster scope resource, simply use resource name. Key-value pairs in the mapping are separated by semi-colon. Example: 'pods=ns1/pod1,ns1/pod2;persistentvolumeclaims=ns1/pvc4,ns1/pvc8'. Optional.\")\n\tflags.DurationVar(&o.CSISnapshotTimeout, \"csi-snapshot-timeout\", o.CSISnapshotTimeout, \"How long to wait for CSI snapshot creation before timeout.\")\n\tflags.DurationVar(&o.ItemOperationTimeout, \"item-operation-timeout\", o.ItemOperationTimeout, \"How long to wait for async plugin operations before timeout.\")\n\tf := flags.VarPF(&o.SnapshotVolumes, \"snapshot-volumes\", \"\", \"Take snapshots of PersistentVolumes as part of the backup. If the parameter is not set, it is treated as setting to 'true'.\")\n\t// this allows the user to just specify \"--snapshot-volumes\" as shorthand for \"--snapshot-volumes=true\"\n\t// like a normal bool flag\n\tf.NoOptDefVal = cmd.TRUE\n\n\tf = flags.VarPF(&o.SnapshotMoveData, \"snapshot-move-data\", \"\", \"Specify whether snapshot data should be moved\")\n\tf.NoOptDefVal = cmd.TRUE\n\n\tf = flags.VarPF(&o.IncludeClusterResources, \"include-cluster-resources\", \"\", \"Include cluster-scoped resources in the backup. Cannot work with include-cluster-scoped-resources, exclude-cluster-scoped-resources, include-namespace-scoped-resources and exclude-namespace-scoped-resources.\")\n\tf.NoOptDefVal = cmd.TRUE\n\n\tf = flags.VarPF(&o.DefaultVolumesToFsBackup, \"default-volumes-to-fs-backup\", \"\", \"Use pod volume file system backup by default for volumes\")\n\tf.NoOptDefVal = cmd.TRUE\n\n\tflags.StringVar(&o.ResPoliciesConfigmap, \"resource-policies-configmap\", \"\", \"Reference to the resource policies configmap that backup should use\")\n\tflags.StringVar(&o.DataMover, \"data-mover\", \"\", \"Specify the data mover to be used by the backup. If the parameter is not set or set as 'velero', the built-in data mover will be used\")\n\tflags.IntVar(&o.ParallelFilesUpload, \"parallel-files-upload\", 0, \"Number of files uploads simultaneously when running a backup. This is only applicable for the kopia uploader\")\n}\n\n// BindWait binds the wait flag separately so it is not called by other create\n// commands that reuse CreateOptions's BindFlags method.\nfunc (o *CreateOptions) BindWait(flags *pflag.FlagSet) {\n\tflags.BoolVarP(&o.Wait, \"wait\", \"w\", o.Wait, \"Wait for the operation to complete.\")\n}\n\n// BindFromSchedule binds the from-schedule flag separately so it is not called\n// by other create commands that reuse CreateOptions's BindFlags method.\nfunc (o *CreateOptions) BindFromSchedule(flags *pflag.FlagSet) {\n\tflags.StringVar(&o.FromSchedule, \"from-schedule\", \"\", \"Create a backup from the template of an existing schedule. Cannot be used with any other filters. Backup name is optional if used.\")\n}\n\nfunc (o *CreateOptions) Validate(c *cobra.Command, args []string, f client.Factory) error {\n\tif err := output.ValidateFlags(c); err != nil {\n\t\treturn err\n\t}\n\n\tif o.Selector.LabelSelector != nil && o.OrSelector.OrLabelSelectors != nil {\n\t\treturn fmt.Errorf(\"either a 'selector' or an 'or-selector' can be specified, but not both\")\n\t}\n\n\t// Ensure if FromSchedule is set, it has a non-empty value\n\tif err := o.validateFromScheduleFlag(c); err != nil {\n\t\treturn err\n\t}\n\n\t// Ensure that unless FromSchedule is set, args contains a backup name\n\tif o.FromSchedule == \"\" && len(args) != 1 {\n\t\treturn fmt.Errorf(\"a backup name is required, unless you are creating based on a schedule\")\n\t}\n\n\terrs := collections.ValidateNamespaceIncludesExcludes(o.IncludeNamespaces, o.ExcludeNamespaces)\n\tif len(errs) > 0 {\n\t\treturn kubeerrs.NewAggregate(errs)\n\t}\n\n\tif o.oldAndNewFilterParametersUsedTogether() {\n\t\treturn fmt.Errorf(\"include-resources, exclude-resources and include-cluster-resources are old filter parameters.\\n\" +\n\t\t\t\"include-cluster-scoped-resources, exclude-cluster-scoped-resources, include-namespace-scoped-resources and exclude-namespace-scoped-resources are new filter parameters.\\n\" +\n\t\t\t\"They cannot be used together\")\n\t}\n\n\tif o.StorageLocation != \"\" {\n\t\tlocation := &velerov1api.BackupStorageLocation{}\n\t\tif err := o.client.Get(context.Background(), kbclient.ObjectKey{\n\t\t\tNamespace: f.Namespace(),\n\t\t\tName: o.StorageLocation,\n\t\t}, location); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tfor _, loc := range o.SnapshotLocations {\n\t\tsnapshotLocation := new(velerov1api.VolumeSnapshotLocation)\n\t\tif err := o.client.Get(context.Background(), kbclient.ObjectKey{Namespace: f.Namespace(), Name: loc}, snapshotLocation); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (o *CreateOptions) validateFromScheduleFlag(c *cobra.Command) error {\n\ttrimmed := strings.TrimSpace(o.FromSchedule)\n\tif c.Flags().Changed(\"from-schedule\") && trimmed == \"\" {\n\t\treturn fmt.Errorf(\"flag must have a non-empty value: --from-schedule\")\n\t}\n\n\t// Assign the trimmed value back\n\to.FromSchedule = trimmed\n\treturn nil\n}\n\nfunc (o *CreateOptions) Complete(args []string, f client.Factory) error {\n\t// If an explicit name is specified, use that name\n\tif len(args) > 0 {\n\t\to.Name = args[0]\n\t}\n\tclient, err := f.KubebuilderWatchClient()\n\tif err != nil {\n\t\treturn err\n\t}\n\to.client = client\n\treturn nil\n}\n\nfunc (o *CreateOptions) Run(c *cobra.Command, f client.Factory) error {\n\tbackup, err := o.BuildBackup(f.Namespace())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif printed, err := output.PrintWithFormat(c, backup); printed || err != nil {\n\t\treturn err\n\t}\n\n\tif o.FromSchedule != \"\" {\n\t\tfmt.Println(\"Creating backup from schedule, all other filters are ignored.\")\n\t}\n\n\tvar updates chan *velerov1api.Backup\n\tif o.Wait {\n\t\tstop := make(chan struct{})\n\t\tdefer close(stop)\n\n\t\tupdates = make(chan *velerov1api.Backup)\n\n\t\tlw := kube.InternalLW{\n\t\t\tClient: o.client,\n\t\t\tNamespace: f.Namespace(),\n\t\t\tObjectList: new(velerov1api.BackupList),\n\t\t}\n\t\tbackupInformer := cache.NewSharedInformer(&lw, &velerov1api.Backup{}, time.Second)\n\t\t_, _ = backupInformer.AddEventHandler(\n\t\t\tcache.FilteringResourceEventHandler{\n\t\t\t\tFilterFunc: func(obj any) bool {\n\t\t\t\t\tbackup, ok := obj.(*velerov1api.Backup)\n\n\t\t\t\t\tif !ok {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\t\t\t\t\treturn backup.Name == o.Name\n\t\t\t\t},\n\t\t\t\tHandler: cache.ResourceEventHandlerFuncs{\n\t\t\t\t\tUpdateFunc: func(_, obj any) {\n\t\t\t\t\t\tbackup, ok := obj.(*velerov1api.Backup)\n\t\t\t\t\t\tif !ok {\n\t\t\t\t\t\t\treturn\n\t\t\t\t\t\t}\n\t\t\t\t\t\tupdates <- backup\n\t\t\t\t\t},\n\t\t\t\t\tDeleteFunc: func(obj any) {\n\t\t\t\t\t\tbackup, ok := obj.(*velerov1api.Backup)\n\t\t\t\t\t\tif !ok {\n\t\t\t\t\t\t\treturn\n\t\t\t\t\t\t}\n\t\t\t\t\t\tupdates <- backup\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t)\n\n\t\tgo backupInformer.Run(stop)\n\t}\n\n\terr = o.client.Create(context.TODO(), backup, &kbclient.CreateOptions{})\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfmt.Printf(\"Backup request %q submitted successfully.\\n\", backup.Name)\n\tif o.Wait {\n\t\tfmt.Println(\"Waiting for backup to complete. You may safely press ctrl-c to stop waiting - your backup will continue in the background.\")\n\t\tticker := time.NewTicker(time.Second)\n\t\tdefer ticker.Stop()\n\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-ticker.C:\n\t\t\t\tfmt.Print(\".\")\n\t\t\tcase backup, ok := <-updates:\n\t\t\t\tif !ok {\n\t\t\t\t\tfmt.Println(\"\\nError waiting: unable to watch backups.\")\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\n\t\t\t\tif backup.Status.Phase == velerov1api.BackupPhaseFailedValidation || backup.Status.Phase == velerov1api.BackupPhaseCompleted ||\n\t\t\t\t\tbackup.Status.Phase == velerov1api.BackupPhasePartiallyFailed || backup.Status.Phase == velerov1api.BackupPhaseFailed {\n\t\t\t\t\tfmt.Printf(\"\\nBackup completed with status: %s. You may check for more information using the commands `velero backup describe %s` and `velero backup logs %s`.\\n\", backup.Status.Phase, backup.Name, backup.Name)\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\t// Not waiting\n\n\tfmt.Printf(\"Run `velero backup describe %s` or `velero backup logs %s` for more details.\\n\", backup.Name, backup.Name)\n\n\treturn nil\n}\n\n// ParseOrderedResources converts to map of Kinds to an ordered list of specific resources of that Kind.\n// Resource names in the list are in format 'namespace/resourcename' and separated by commas.\n// Key-value pairs in the mapping are separated by semi-colon.\n// Ex: 'pods=ns1/pod1,ns1/pod2;persistentvolumeclaims=ns1/pvc4,ns1/pvc8'.\nfunc ParseOrderedResources(orderMapStr string) (map[string]string, error) {\n\tentries := strings.Split(orderMapStr, \";\")\n\tif len(entries) == 0 {\n\t\treturn nil, fmt.Errorf(\"invalid OrderedResources '%s'\", orderMapStr)\n\t}\n\torderedResources := make(map[string]string)\n\tfor _, entry := range entries {\n\t\tkv := strings.Split(entry, \"=\")\n\t\tif len(kv) != 2 {\n\t\t\treturn nil, fmt.Errorf(\"invalid OrderedResources '%s'\", entry)\n\t\t}\n\t\tkind := strings.TrimSpace(kv[0])\n\t\torder := strings.TrimSpace(kv[1])\n\t\torderedResources[kind] = order\n\t}\n\treturn orderedResources, nil\n}\n\nfunc (o *CreateOptions) BuildBackup(namespace string) (*velerov1api.Backup, error) {\n\tvar backupBuilder *builder.BackupBuilder\n\n\tif o.FromSchedule != \"\" {\n\t\tschedule := new(velerov1api.Schedule)\n\t\terr := o.client.Get(context.TODO(), kbclient.ObjectKey{Namespace: namespace, Name: o.FromSchedule}, schedule)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tif o.Name == \"\" {\n\t\t\to.Name = schedule.TimestampedName(time.Now().UTC())\n\t\t}\n\t\tbackupBuilder = builder.ForBackup(namespace, o.Name).\n\t\t\tFromSchedule(schedule)\n\t} else {\n\t\tbackupBuilder = builder.ForBackup(namespace, o.Name).\n\t\t\tIncludedNamespaces(o.IncludeNamespaces...).\n\t\t\tExcludedNamespaces(o.ExcludeNamespaces...).\n\t\t\tIncludedResources(o.IncludeResources...).\n\t\t\tExcludedResources(o.ExcludeResources...).\n\t\t\tIncludedClusterScopedResources(o.IncludeClusterScopedResources...).\n\t\t\tExcludedClusterScopedResources(o.ExcludeClusterScopedResources...).\n\t\t\tIncludedNamespaceScopedResources(o.IncludeNamespaceScopedResources...).\n\t\t\tExcludedNamespaceScopedResources(o.ExcludeNamespaceScopedResources...).\n\t\t\tLabelSelector(o.Selector.LabelSelector).\n\t\t\tOrLabelSelector(o.OrSelector.OrLabelSelectors).\n\t\t\tTTL(o.TTL).\n\t\t\tStorageLocation(o.StorageLocation).\n\t\t\tVolumeSnapshotLocations(o.SnapshotLocations...).\n\t\t\tCSISnapshotTimeout(o.CSISnapshotTimeout).\n\t\t\tItemOperationTimeout(o.ItemOperationTimeout).\n\t\t\tDataMover(o.DataMover)\n\t\tif len(o.OrderedResources) > 0 {\n\t\t\torders, err := ParseOrderedResources(o.OrderedResources)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t\tbackupBuilder.OrderedResources(orders)\n\t\t}\n\n\t\tif o.SnapshotVolumes.Value != nil {\n\t\t\tbackupBuilder.SnapshotVolumes(*o.SnapshotVolumes.Value)\n\t\t}\n\t\tif o.SnapshotMoveData.Value != nil {\n\t\t\tbackupBuilder.SnapshotMoveData(*o.SnapshotMoveData.Value)\n\t\t}\n\t\tif o.IncludeClusterResources.Value != nil {\n\t\t\tbackupBuilder.IncludeClusterResources(*o.IncludeClusterResources.Value)\n\t\t}\n\t\tif o.DefaultVolumesToFsBackup.Value != nil {\n\t\t\tbackupBuilder.DefaultVolumesToFsBackup(*o.DefaultVolumesToFsBackup.Value)\n\t\t}\n\t\tif o.ResPoliciesConfigmap != \"\" {\n\t\t\tbackupBuilder.ResourcePolicies(o.ResPoliciesConfigmap)\n\t\t}\n\t\tif o.ParallelFilesUpload > 0 {\n\t\t\tbackupBuilder.ParallelFilesUpload(o.ParallelFilesUpload)\n\t\t}\n\t}\n\n\tbackup := backupBuilder.ObjectMeta(builder.WithLabelsMap(o.Labels.Data()), builder.WithAnnotationsMap(o.Annotations.Data())).Result()\n\treturn backup, nil\n}\n\nfunc (o *CreateOptions) oldAndNewFilterParametersUsedTogether() bool {\n\thaveOldResourceFilterParameters := len(o.IncludeResources) > 0 ||\n\t\tlen(o.ExcludeResources) > 0 ||\n\t\to.IncludeClusterResources.Value != nil\n\thaveNewResourceFilterParameters := len(o.IncludeClusterScopedResources) > 0 ||\n\t\t(len(o.ExcludeClusterScopedResources) > 0) ||\n\t\t(len(o.IncludeNamespaceScopedResources) > 0) ||\n\t\t(len(o.ExcludeNamespaceScopedResources) > 0)\n\n\treturn haveOldResourceFilterParameters && haveNewResourceFilterParameters\n}\n" }, { "path": "pkg/cmd/cli/backup/create_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"fmt\"\n\t\"strconv\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/spf13/cobra\"\n\tflag \"github.com/spf13/pflag\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\tcontrollerclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\tcmdtest \"github.com/vmware-tanzu/velero/pkg/cmd/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestCreateOptions_BuildBackup(t *testing.T) {\n\to := NewCreateOptions()\n\to.Labels.Set(\"velero.io/test=true\")\n\to.Annotations.Set(\"velero.io/annTest=true\")\n\to.OrderedResources = \"pods=p1,p2;persistentvolumeclaims=pvc1,pvc2\"\n\torders, err := ParseOrderedResources(o.OrderedResources)\n\to.CSISnapshotTimeout = 20 * time.Minute\n\to.ItemOperationTimeout = 20 * time.Minute\n\torLabelSelectors := []*metav1.LabelSelector{\n\t\t{\n\t\t\tMatchLabels: map[string]string{\"k1\": \"v1\", \"k2\": \"v2\"},\n\t\t},\n\t\t{\n\t\t\tMatchLabels: map[string]string{\"a1\": \"b1\", \"a2\": \"b2\"},\n\t\t},\n\t}\n\to.OrSelector.OrLabelSelectors = orLabelSelectors\n\trequire.NoError(t, err)\n\n\tbackup, err := o.BuildBackup(cmdtest.VeleroNameSpace)\n\trequire.NoError(t, err)\n\n\tassert.Equal(t, velerov1api.BackupSpec{\n\t\tTTL: metav1.Duration{Duration: o.TTL},\n\t\tIncludedNamespaces: []string(o.IncludeNamespaces),\n\t\tSnapshotVolumes: o.SnapshotVolumes.Value,\n\t\tIncludeClusterResources: o.IncludeClusterResources.Value,\n\t\tOrderedResources: orders,\n\t\tOrLabelSelectors: orLabelSelectors,\n\t\tCSISnapshotTimeout: metav1.Duration{Duration: o.CSISnapshotTimeout},\n\t\tItemOperationTimeout: metav1.Duration{Duration: o.ItemOperationTimeout},\n\t}, backup.Spec)\n\n\tassert.Equal(t, map[string]string{\n\t\t\"velero.io/test\": \"true\",\n\t}, backup.GetLabels())\n\tassert.Equal(t, map[string]string{\n\t\t\"velero.io/annTest\": \"true\",\n\t}, backup.GetAnnotations())\n\tassert.Equal(t, map[string]string{\n\t\t\"pods\": \"p1,p2\",\n\t\t\"persistentvolumeclaims\": \"pvc1,pvc2\",\n\t}, backup.Spec.OrderedResources)\n}\n\nfunc TestCreateOptions_ValidateFromScheduleFlag(t *testing.T) {\n\tcmd := &cobra.Command{}\n\to := NewCreateOptions()\n\to.BindFromSchedule(cmd.Flags())\n\n\tt.Run(\"from-schedule with empty or no value\", func(t *testing.T) {\n\t\tcmd.Flags().Set(\"from-schedule\", \"\")\n\t\terr := o.validateFromScheduleFlag(cmd)\n\t\trequire.True(t, cmd.Flags().Changed(\"from-schedule\"))\n\t\trequire.Error(t, err)\n\t\trequire.Equal(t, \"flag must have a non-empty value: --from-schedule\", err.Error())\n\t})\n\n\tt.Run(\"from-schedule with spaces only\", func(t *testing.T) {\n\t\tcmd.Flags().Set(\"from-schedule\", \" \")\n\t\terr := o.validateFromScheduleFlag(cmd)\n\t\trequire.True(t, cmd.Flags().Changed(\"from-schedule\"))\n\t\trequire.Error(t, err)\n\t\trequire.Equal(t, \"flag must have a non-empty value: --from-schedule\", err.Error())\n\t})\n\n\tt.Run(\"from-schedule with valid value\", func(t *testing.T) {\n\t\tcmd.Flags().Set(\"from-schedule\", \"daily\")\n\t\terr := o.validateFromScheduleFlag(cmd)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, \"daily\", o.FromSchedule)\n\t})\n\n\tt.Run(\"from-schedule with leading and trailing spaces\", func(t *testing.T) {\n\t\tcmd.Flags().Set(\"from-schedule\", \" daily \")\n\t\terr := o.validateFromScheduleFlag(cmd)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, \"daily\", o.FromSchedule)\n\t})\n}\n\nfunc TestCreateOptions_BuildBackupFromSchedule(t *testing.T) {\n\to := NewCreateOptions()\n\to.FromSchedule = \"test\"\n\n\tscheme := runtime.NewScheme()\n\terr := velerov1api.AddToScheme(scheme)\n\trequire.NoError(t, err)\n\to.client = velerotest.NewFakeControllerRuntimeClient(t).(controllerclient.WithWatch)\n\n\tt.Run(\"inexistent schedule\", func(t *testing.T) {\n\t\t_, err := o.BuildBackup(cmdtest.VeleroNameSpace)\n\t\trequire.Error(t, err)\n\t})\n\n\texpectedBackupSpec := builder.ForBackup(\"test\", cmdtest.VeleroNameSpace).IncludedNamespaces(\"test\").Result().Spec\n\tschedule := builder.ForSchedule(cmdtest.VeleroNameSpace, \"test\").Template(expectedBackupSpec).ObjectMeta(builder.WithLabels(\"velero.io/test\", \"true\"), builder.WithAnnotations(\"velero.io/test\", \"true\")).Result()\n\to.client.Create(t.Context(), schedule, &kbclient.CreateOptions{})\n\n\tt.Run(\"existing schedule\", func(t *testing.T) {\n\t\tbackup, err := o.BuildBackup(cmdtest.VeleroNameSpace)\n\t\trequire.NoError(t, err)\n\n\t\trequire.Equal(t, expectedBackupSpec, backup.Spec)\n\t\trequire.Equal(t, map[string]string{\n\t\t\t\"velero.io/test\": \"true\",\n\t\t\tvelerov1api.ScheduleNameLabel: \"test\",\n\t\t}, backup.GetLabels())\n\t\trequire.Equal(t, map[string]string{\n\t\t\t\"velero.io/test\": \"true\",\n\t\t}, backup.GetAnnotations())\n\t})\n\n\tt.Run(\"command line labels and annotations take precedence over scheduled ones\", func(t *testing.T) {\n\t\to.Labels.Set(\"velero.io/test=yes,custom-label=true\")\n\t\to.Annotations.Set(\"velero.io/test=yes,custom-annotation=true\")\n\t\tbackup, err := o.BuildBackup(cmdtest.VeleroNameSpace)\n\t\trequire.NoError(t, err)\n\n\t\tassert.Equal(t, expectedBackupSpec, backup.Spec)\n\t\tassert.Equal(t, map[string]string{\n\t\t\t\"velero.io/test\": \"yes\",\n\t\t\tvelerov1api.ScheduleNameLabel: \"test\",\n\t\t\t\"custom-label\": \"true\",\n\t\t}, backup.GetLabels())\n\t\tassert.Equal(t, map[string]string{\n\t\t\t\"velero.io/test\": \"yes\",\n\t\t\t\"custom-annotation\": \"true\",\n\t\t}, backup.GetAnnotations())\n\t})\n}\n\nfunc TestCreateOptions_OrderedResources(t *testing.T) {\n\t_, err := ParseOrderedResources(\"pods= ns1/p1; ns1/p2; persistentvolumeclaims=ns2/pvc1, ns2/pvc2\")\n\trequire.Error(t, err)\n\n\torderedResources, err := ParseOrderedResources(\"pods= ns1/p1,ns1/p2 ; persistentvolumeclaims=ns2/pvc1,ns2/pvc2\")\n\trequire.NoError(t, err)\n\n\texpectedResources := map[string]string{\n\t\t\"pods\": \"ns1/p1,ns1/p2\",\n\t\t\"persistentvolumeclaims\": \"ns2/pvc1,ns2/pvc2\",\n\t}\n\tassert.Equal(t, expectedResources, orderedResources)\n\n\torderedResources, err = ParseOrderedResources(\"pods= ns1/p1,ns1/p2 ; persistentvolumes=pv1,pv2\")\n\trequire.NoError(t, err)\n\n\texpectedMixedResources := map[string]string{\n\t\t\"pods\": \"ns1/p1,ns1/p2\",\n\t\t\"persistentvolumes\": \"pv1,pv2\",\n\t}\n\tassert.Equal(t, expectedMixedResources, orderedResources)\n}\n\nfunc TestCreateCommand(t *testing.T) {\n\tname := \"nameToBeCreated\"\n\targs := []string{name}\n\n\tt.Run(\"create a backup create command with full options except fromSchedule and wait, then run by create option\", func(t *testing.T) {\n\t\t// create a factory\n\t\tf := &factorymocks.Factory{}\n\n\t\t// create command\n\t\tcmd := NewCreateCommand(f, \"\")\n\t\tassert.Equal(t, \"Create a backup\", cmd.Short)\n\n\t\tincludeNamespaces := \"app1,app2\"\n\t\texcludeNamespaces := \"pod1,pod2,pod3\"\n\t\tincludeResources := \"sc,sts\"\n\t\texcludeResources := \"job\"\n\t\tincludeClusterScopedResources := \"pv,ComponentStatus\"\n\t\texcludeClusterScopedResources := \"MutatingWebhookConfiguration,APIService\"\n\t\tincludeNamespaceScopedResources := \"Endpoints,Event,PodTemplate\"\n\t\texcludeNamespaceScopedResources := \"Secret,MultiClusterIngress\"\n\t\tlabels := \"c=foo\"\n\t\tannotations := \"ann=foo\"\n\t\tstorageLocation := \"bsl-name-1\"\n\t\tsnapshotLocations := \"region=minio\"\n\t\tselector := \"a=pod\"\n\t\torderedResources := \"pod=pod1,pod2,pod3\"\n\t\tcsiSnapshotTimeout := \"8m30s\"\n\t\titemOperationTimeout := \"99h1m6s\"\n\t\tsnapshotVolumes := \"false\"\n\t\tsnapshotMoveData := \"true\"\n\t\tincludeClusterResources := \"true\"\n\t\tdefaultVolumesToFsBackup := \"true\"\n\t\tresPoliciesConfigmap := \"cm-name-2\"\n\t\tdataMover := \"velero\"\n\t\tparallelFilesUpload := 10\n\t\tflags := new(flag.FlagSet)\n\t\to := NewCreateOptions()\n\t\to.BindFlags(flags)\n\t\to.BindWait(flags)\n\t\to.BindFromSchedule(flags)\n\n\t\tflags.Parse([]string{\"--include-namespaces\", includeNamespaces})\n\t\tflags.Parse([]string{\"--exclude-namespaces\", excludeNamespaces})\n\t\tflags.Parse([]string{\"--include-resources\", includeResources})\n\t\tflags.Parse([]string{\"--exclude-resources\", excludeResources})\n\t\tflags.Parse([]string{\"--include-cluster-scoped-resources\", includeClusterScopedResources})\n\t\tflags.Parse([]string{\"--exclude-cluster-scoped-resources\", excludeClusterScopedResources})\n\t\tflags.Parse([]string{\"--include-namespace-scoped-resources\", includeNamespaceScopedResources})\n\t\tflags.Parse([]string{\"--exclude-namespace-scoped-resources\", excludeNamespaceScopedResources})\n\t\tflags.Parse([]string{\"--labels\", labels})\n\t\tflags.Parse([]string{\"--annotations\", annotations})\n\t\tflags.Parse([]string{\"--storage-location\", storageLocation})\n\t\tflags.Parse([]string{\"--volume-snapshot-locations\", snapshotLocations})\n\t\tflags.Parse([]string{\"--selector\", selector})\n\t\tflags.Parse([]string{\"--ordered-resources\", orderedResources})\n\t\tflags.Parse([]string{\"--csi-snapshot-timeout\", csiSnapshotTimeout})\n\t\tflags.Parse([]string{\"--item-operation-timeout\", itemOperationTimeout})\n\t\tflags.Parse([]string{fmt.Sprintf(\"--snapshot-volumes=%s\", snapshotVolumes)})\n\t\tflags.Parse([]string{fmt.Sprintf(\"--snapshot-move-data=%s\", snapshotMoveData)})\n\t\tflags.Parse([]string{\"--include-cluster-resources\", includeClusterResources})\n\t\tflags.Parse([]string{\"--default-volumes-to-fs-backup\", defaultVolumesToFsBackup})\n\t\tflags.Parse([]string{\"--resource-policies-configmap\", resPoliciesConfigmap})\n\t\tflags.Parse([]string{\"--data-mover\", dataMover})\n\t\tflags.Parse([]string{\"--parallel-files-upload\", strconv.Itoa(parallelFilesUpload)})\n\t\t//flags.Parse([]string{\"--wait\"})\n\n\t\tclient := velerotest.NewFakeControllerRuntimeClient(t).(kbclient.WithWatch)\n\n\t\tf.On(\"Namespace\").Return(mock.Anything)\n\t\tf.On(\"KubebuilderWatchClient\").Return(client, nil)\n\n\t\t//Complete\n\t\te := o.Complete(args, f)\n\t\trequire.NoError(t, e)\n\n\t\t//Validate\n\t\te = o.Validate(cmd, args, f)\n\t\trequire.ErrorContains(t, e, \"include-resources, exclude-resources and include-cluster-resources are old filter parameters\")\n\t\trequire.ErrorContains(t, e, \"include-cluster-scoped-resources, exclude-cluster-scoped-resources, include-namespace-scoped-resources and exclude-namespace-scoped-resources are new filter parameters.\\nThey cannot be used together\")\n\n\t\t//cmd\n\t\te = o.Run(cmd, f)\n\t\trequire.NoError(t, e)\n\n\t\t//Execute\n\t\tcmd.SetArgs([]string{\"bk-name-exe\"})\n\t\te = cmd.Execute()\n\t\trequire.NoError(t, e)\n\n\t\t// verify all options are set as expected\n\t\trequire.Equal(t, name, o.Name)\n\t\trequire.Equal(t, includeNamespaces, o.IncludeNamespaces.String())\n\t\trequire.Equal(t, excludeNamespaces, o.ExcludeNamespaces.String())\n\t\trequire.Equal(t, includeResources, o.IncludeResources.String())\n\t\trequire.Equal(t, excludeResources, o.ExcludeResources.String())\n\t\trequire.Equal(t, includeClusterScopedResources, o.IncludeClusterScopedResources.String())\n\t\trequire.Equal(t, excludeClusterScopedResources, o.ExcludeClusterScopedResources.String())\n\t\trequire.Equal(t, includeNamespaceScopedResources, o.IncludeNamespaceScopedResources.String())\n\t\trequire.Equal(t, excludeNamespaceScopedResources, o.ExcludeNamespaceScopedResources.String())\n\t\trequire.True(t, test.CompareSlice(strings.Split(labels, \",\"), strings.Split(o.Labels.String(), \",\")))\n\t\trequire.True(t, test.CompareSlice(strings.Split(annotations, \",\"), strings.Split(o.Annotations.String(), \",\")))\n\t\trequire.Equal(t, storageLocation, o.StorageLocation)\n\t\trequire.Equal(t, snapshotLocations, strings.Split(o.SnapshotLocations[0], \",\")[0])\n\t\trequire.Equal(t, selector, o.Selector.String())\n\t\trequire.Equal(t, orderedResources, o.OrderedResources)\n\t\trequire.Equal(t, csiSnapshotTimeout, o.CSISnapshotTimeout.String())\n\t\trequire.Equal(t, itemOperationTimeout, o.ItemOperationTimeout.String())\n\t\trequire.Equal(t, snapshotVolumes, o.SnapshotVolumes.String())\n\t\trequire.Equal(t, snapshotMoveData, o.SnapshotMoveData.String())\n\t\trequire.Equal(t, includeClusterResources, o.IncludeClusterResources.String())\n\t\trequire.Equal(t, defaultVolumesToFsBackup, o.DefaultVolumesToFsBackup.String())\n\t\trequire.Equal(t, resPoliciesConfigmap, o.ResPoliciesConfigmap)\n\t\trequire.Equal(t, dataMover, o.DataMover)\n\t\trequire.Equal(t, parallelFilesUpload, o.ParallelFilesUpload)\n\t\t//assert.Equal(t, true, o.Wait)\n\n\t\t// verify oldAndNewFilterParametersUsedTogether\n\t\tmix := o.oldAndNewFilterParametersUsedTogether()\n\t\trequire.True(t, mix)\n\t})\n\n\tt.Run(\"create a backup create command with specific storage-location setting\", func(t *testing.T) {\n\t\tbsl := \"bsl-1\"\n\t\t// create a factory\n\t\tf := &factorymocks.Factory{}\n\t\tcmd := NewCreateCommand(f, \"\")\n\t\tkbclient := velerotest.NewFakeControllerRuntimeClient(t).(kbclient.WithWatch)\n\n\t\tf.On(\"Namespace\").Return(mock.Anything)\n\t\tf.On(\"KubebuilderWatchClient\").Return(kbclient, nil)\n\n\t\tflags := new(flag.FlagSet)\n\t\to := NewCreateOptions()\n\t\to.BindFlags(flags)\n\t\to.BindWait(flags)\n\t\to.BindFromSchedule(flags)\n\t\tflags.Parse([]string{\"--include-namespaces\", \"ns-1\"})\n\t\tflags.Parse([]string{\"--storage-location\", bsl})\n\n\t\t// Complete\n\t\te := o.Complete(args, f)\n\t\trequire.NoError(t, e)\n\n\t\t// Validate\n\t\te = o.Validate(cmd, args, f)\n\t\tassert.ErrorContains(t, e, fmt.Sprintf(\"backupstoragelocations.velero.io \\\"%s\\\" not found\", bsl))\n\t})\n\n\tt.Run(\"create a backup create command with specific volume-snapshot-locations setting\", func(t *testing.T) {\n\t\tvslName := \"vsl-1\"\n\t\t// create a factory\n\t\tf := &factorymocks.Factory{}\n\t\tcmd := NewCreateCommand(f, \"\")\n\t\tkbclient := velerotest.NewFakeControllerRuntimeClient(t).(kbclient.WithWatch)\n\n\t\tvsl := builder.ForVolumeSnapshotLocation(cmdtest.VeleroNameSpace, vslName).Result()\n\n\t\tkbclient.Create(cmd.Context(), vsl, &controllerclient.CreateOptions{})\n\n\t\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\t\tf.On(\"KubebuilderWatchClient\").Return(kbclient, nil)\n\n\t\tflags := new(flag.FlagSet)\n\t\to := NewCreateOptions()\n\t\to.BindFlags(flags)\n\t\to.BindWait(flags)\n\t\to.BindFromSchedule(flags)\n\t\tflags.Parse([]string{\"--include-namespaces\", \"ns-1\"})\n\t\tflags.Parse([]string{\"--volume-snapshot-locations\", vslName})\n\n\t\t// Complete\n\t\te := o.Complete(args, f)\n\t\trequire.NoError(t, e)\n\n\t\t// Validate\n\t\te = o.Validate(cmd, args, f)\n\t\tassert.NoError(t, e)\n\t})\n\n\tt.Run(\"create the other create command with fromSchedule option for Run() other branches\", func(t *testing.T) {\n\t\tf := &factorymocks.Factory{}\n\t\tc := NewCreateCommand(f, \"\")\n\t\tassert.Equal(t, \"Create a backup\", c.Short)\n\t\tflags := new(flag.FlagSet)\n\t\to := NewCreateOptions()\n\t\to.BindFlags(flags)\n\t\to.BindWait(flags)\n\t\to.BindFromSchedule(flags)\n\t\tfromSchedule := \"schedule-name-1\"\n\t\tflags.Parse([]string{\"--from-schedule\", fromSchedule})\n\n\t\tkbclient := velerotest.NewFakeControllerRuntimeClient(t).(kbclient.WithWatch)\n\n\t\tschedule := builder.ForSchedule(cmdtest.VeleroNameSpace, fromSchedule).Result()\n\t\tkbclient.Create(t.Context(), schedule, &controllerclient.CreateOptions{})\n\n\t\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\t\tf.On(\"KubebuilderWatchClient\").Return(kbclient, nil)\n\n\t\te := o.Complete(args, f)\n\t\trequire.NoError(t, e)\n\n\t\te = o.Run(c, f)\n\t\trequire.NoError(t, e)\n\n\t\tc.SetArgs([]string{\"bk-1\"})\n\t\te = c.Execute()\n\t\tassert.NoError(t, e)\n\t})\n}\n" }, { "path": "pkg/cmd/cli/backup/delete.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/spf13/cobra\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\tkubeerrs \"k8s.io/apimachinery/pkg/util/errors\"\n\tcontrollerclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/confirm\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n)\n\n// NewDeleteCommand creates a new command that deletes a backup.\nfunc NewDeleteCommand(f client.Factory, use string) *cobra.Command {\n\to := cli.NewDeleteOptions(\"backup\")\n\n\tc := &cobra.Command{\n\t\tUse: fmt.Sprintf(\"%s [NAMES]\", use),\n\t\tShort: \"Delete backups\",\n\t\tExample: ` # Delete a backup named \"backup-1\".\n velero backup delete backup-1\n\n # Delete a backup named \"backup-1\" without prompting for confirmation.\n velero backup delete backup-1 --confirm\n\n # Delete backups named \"backup-1\" and \"backup-2\".\n velero backup delete backup-1 backup-2\n\n # Delete all backups triggered by schedule \"schedule-1\".\n velero backup delete --selector velero.io/schedule-name=schedule-1\n \n # Delete all backups.\n velero backup delete --all`,\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tcmd.CheckError(o.Complete(f, args))\n\t\t\tcmd.CheckError(o.Validate(c, f, args))\n\t\t\tcmd.CheckError(Run(o))\n\t\t},\n\t}\n\n\to.BindFlags(c.Flags())\n\n\treturn c\n}\n\n// Run performs the delete backup operation.\nfunc Run(o *cli.DeleteOptions) error {\n\tif !o.Confirm && !confirm.GetConfirmation() {\n\t\t// Don't do anything unless we get confirmation\n\t\treturn nil\n\t}\n\n\tvar (\n\t\tbackups []*velerov1api.Backup\n\t\terrs []error\n\t)\n\n\t// get the list of backups to delete\n\tswitch {\n\tcase len(o.Names) > 0:\n\t\tfor _, name := range o.Names {\n\t\t\tbackup := new(velerov1api.Backup)\n\t\t\terr := o.Client.Get(context.TODO(), controllerclient.ObjectKey{Namespace: o.Namespace, Name: name}, backup)\n\t\t\tif err != nil {\n\t\t\t\terrs = append(errs, errors.WithStack(err))\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tbackups = append(backups, backup)\n\t\t}\n\tdefault:\n\t\tselector := labels.Everything()\n\t\tif o.Selector.LabelSelector != nil {\n\t\t\tconvertedSelector, err := metav1.LabelSelectorAsSelector(o.Selector.LabelSelector)\n\t\t\tif err != nil {\n\t\t\t\treturn errors.WithStack(err)\n\t\t\t}\n\t\t\tselector = convertedSelector\n\t\t}\n\n\t\tbackupList := new(velerov1api.BackupList)\n\t\terr := o.Client.List(context.TODO(), backupList, &controllerclient.ListOptions{LabelSelector: selector, Namespace: o.Namespace})\n\t\tif err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\t\tfor i := range backupList.Items {\n\t\t\tbackups = append(backups, &backupList.Items[i])\n\t\t}\n\t}\n\n\tif len(backups) == 0 {\n\t\tfmt.Println(\"No backups found\")\n\t\treturn nil\n\t}\n\n\t// create a backup deletion request for each\n\tfor _, b := range backups {\n\t\tdeleteRequest := builder.ForDeleteBackupRequest(o.Namespace, \"\").BackupName(b.Name).\n\t\t\tObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, label.GetValidName(b.Name),\n\t\t\t\tvelerov1api.BackupUIDLabel, string(b.UID)), builder.WithGenerateName(b.Name+\"-\")).Result()\n\n\t\tif err := client.CreateRetryGenerateName(o.Client, context.TODO(), deleteRequest); err != nil {\n\t\t\terrs = append(errs, err)\n\t\t\tcontinue\n\t\t}\n\n\t\tfmt.Printf(\"Request to delete backup %q submitted successfully.\\nThe backup will be fully deleted after all associated data (disk snapshots, backup files, restores) are removed.\\n\", b.Name)\n\t}\n\n\treturn kubeerrs.NewAggregate(errs)\n}\n" }, { "path": "pkg/cmd/cli/backup/delete_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"testing\"\n\n\tflag \"github.com/spf13/pflag\"\n\t\"github.com/stretchr/testify/require\"\n\tcontrollerclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli\"\n\tcmdtest \"github.com/vmware-tanzu/velero/pkg/cmd/test\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\tveleroexec \"github.com/vmware-tanzu/velero/pkg/util/exec\"\n)\n\nfunc TestDeleteCommand(t *testing.T) {\n\tbackup1 := \"backup-name-1\"\n\tbackup2 := \"backup-name-2\"\n\n\t// create a factory\n\tf := &factorymocks.Factory{}\n\n\tclient := velerotest.NewFakeControllerRuntimeClient(t)\n\tclient.Create(t.Context(), builder.ForBackup(cmdtest.VeleroNameSpace, backup1).Result(), &controllerclient.CreateOptions{})\n\tclient.Create(t.Context(), builder.ForBackup(\"default\", backup2).Result(), &controllerclient.CreateOptions{})\n\n\tf.On(\"KubebuilderClient\").Return(client, nil)\n\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\n\t// create command\n\tc := NewDeleteCommand(f, \"velero backup delete\")\n\tc.SetArgs([]string{backup1, backup2})\n\trequire.Equal(t, \"Delete backups\", c.Short)\n\n\to := cli.NewDeleteOptions(\"backup\")\n\tflags := new(flag.FlagSet)\n\to.BindFlags(flags)\n\tflags.Parse([]string{\"--confirm\"})\n\n\targs := []string{backup1, backup2}\n\n\te := o.Complete(f, args)\n\trequire.NoError(t, e)\n\n\te = o.Validate(c, f, args)\n\trequire.NoError(t, e)\n\n\tRun(o)\n\n\te = c.Execute()\n\trequire.NoError(t, e)\n\n\tif os.Getenv(cmdtest.CaptureFlag) == \"1\" {\n\t\treturn\n\t}\n\n\tcmd := exec.CommandContext(t.Context(), os.Args[0], []string{\"-test.run=TestDeleteCommand\"}...)\n\tcmd.Env = append(os.Environ(), fmt.Sprintf(\"%s=1\", cmdtest.CaptureFlag))\n\tstdout, _, err := veleroexec.RunCommand(cmd)\n\tif err != nil {\n\t\trequire.Contains(t, stdout, fmt.Sprintf(\"backups.velero.io \\\"%s\\\" not found.\", backup2))\n\t}\n}\n" }, { "path": "pkg/cmd/cli/backup/describe.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\n\t\"github.com/spf13/cobra\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\tcontrollerclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/output\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n)\n\nfunc NewDescribeCommand(f client.Factory, use string) *cobra.Command {\n\tvar (\n\t\tlistOptions metav1.ListOptions\n\t\tdetails bool\n\t\tinsecureSkipTLSVerify bool\n\t\toutputFormat = \"plaintext\"\n\t)\n\n\tconfig, err := client.LoadConfig()\n\tif err != nil {\n\t\tfmt.Fprintf(os.Stderr, \"WARNING: Error reading config file: %v\\n\", err)\n\t}\n\tcaCertFile := config.CACertFile()\n\n\tc := &cobra.Command{\n\t\tUse: use + \" [NAME1] [NAME2] [NAME...]\",\n\t\tShort: \"Describe backups\",\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tkbClient, err := f.KubebuilderClient()\n\t\t\tcmd.CheckError(err)\n\n\t\t\tif outputFormat != \"plaintext\" && outputFormat != \"json\" {\n\t\t\t\tcmd.CheckError(fmt.Errorf(\"invalid output format '%s'. valid value are 'plaintext, json'\", outputFormat))\n\t\t\t}\n\n\t\t\tbackups := new(velerov1api.BackupList)\n\t\t\tif len(args) > 0 {\n\t\t\t\tfor _, name := range args {\n\t\t\t\t\tbackup := new(velerov1api.Backup)\n\t\t\t\t\terr := kbClient.Get(context.Background(), controllerclient.ObjectKey{Namespace: f.Namespace(), Name: name}, backup)\n\t\t\t\t\tcmd.CheckError(err)\n\t\t\t\t\tbackups.Items = append(backups.Items, *backup)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tparsedSelector, err := labels.Parse(listOptions.LabelSelector)\n\t\t\t\tcmd.CheckError(err)\n\t\t\t\terr = kbClient.List(context.Background(), backups, &controllerclient.ListOptions{LabelSelector: parsedSelector, Namespace: f.Namespace()})\n\t\t\t\tcmd.CheckError(err)\n\t\t\t}\n\n\t\t\tfirst := true\n\t\t\tfor i, backup := range backups.Items {\n\t\t\t\tdeleteRequestList := new(velerov1api.DeleteBackupRequestList)\n\t\t\t\terr := kbClient.List(context.Background(), deleteRequestList, &controllerclient.ListOptions{\n\t\t\t\t\tNamespace: f.Namespace(),\n\t\t\t\t\tLabelSelector: labels.SelectorFromSet(map[string]string{velerov1api.BackupNameLabel: label.GetValidName(backup.Name), velerov1api.BackupUIDLabel: string(backup.UID)}),\n\t\t\t\t})\n\t\t\t\tif err != nil {\n\t\t\t\t\tfmt.Fprintf(os.Stderr, \"error getting DeleteBackupRequests for backup %s: %v\\n\", backup.Name, err)\n\t\t\t\t}\n\n\t\t\t\tpodVolumeBackupList := new(velerov1api.PodVolumeBackupList)\n\t\t\t\terr = kbClient.List(context.Background(), podVolumeBackupList, &controllerclient.ListOptions{\n\t\t\t\t\tNamespace: f.Namespace(),\n\t\t\t\t\tLabelSelector: labels.SelectorFromSet(map[string]string{velerov1api.BackupNameLabel: label.GetValidName(backup.Name)}),\n\t\t\t\t})\n\t\t\t\tif err != nil {\n\t\t\t\t\tfmt.Fprintf(os.Stderr, \"error getting PodVolumeBackups for backup %s: %v\\n\", backup.Name, err)\n\t\t\t\t}\n\n\t\t\t\t// structured output only applies to a single backup in case of OOM\n\t\t\t\t// To describe the list of backups in structured format, users could iterate over the list and describe backup one after another.\n\t\t\t\tif len(backups.Items) == 1 && outputFormat != \"plaintext\" {\n\t\t\t\t\ts := output.DescribeBackupInSF(context.Background(), kbClient, &backups.Items[i], deleteRequestList.Items, podVolumeBackupList.Items, details, insecureSkipTLSVerify, caCertFile, outputFormat)\n\t\t\t\t\tfmt.Print(s)\n\t\t\t\t} else {\n\t\t\t\t\ts := output.DescribeBackup(context.Background(), kbClient, &backups.Items[i], deleteRequestList.Items, podVolumeBackupList.Items, details, insecureSkipTLSVerify, caCertFile)\n\t\t\t\t\tif first {\n\t\t\t\t\t\tfirst = false\n\t\t\t\t\t\tfmt.Print(s)\n\t\t\t\t\t} else {\n\t\t\t\t\t\tfmt.Printf(\"\\n\\n%s\", s)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\tcmd.CheckError(err)\n\t\t},\n\t}\n\n\tc.Flags().StringVarP(&listOptions.LabelSelector, \"selector\", \"l\", listOptions.LabelSelector, \"Only show items matching this label selector.\")\n\tc.Flags().BoolVar(&details, \"details\", details, \"Display additional detail in the command output.\")\n\tc.Flags().BoolVar(&insecureSkipTLSVerify, \"insecure-skip-tls-verify\", insecureSkipTLSVerify, \"If true, the object store's TLS certificate will not be checked for validity. This is insecure and susceptible to man-in-the-middle attacks. Not recommended for production.\")\n\tc.Flags().StringVar(&caCertFile, \"cacert\", caCertFile, \"Path to a certificate bundle to use when verifying TLS connections. If not specified, the CA certificate from the BackupStorageLocation will be used if available.\")\n\tc.Flags().StringVarP(&outputFormat, \"output\", \"o\", outputFormat, \"Output display format. Valid formats are 'plaintext, json'. 'json' only applies to a single backup\")\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/backup/describe_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/client-go/rest\"\n\tcontrollerclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\tcmdtest \"github.com/vmware-tanzu/velero/pkg/cmd/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/features\"\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n\tveleroexec \"github.com/vmware-tanzu/velero/pkg/util/exec\"\n)\n\nfunc TestNewDescribeCommand(t *testing.T) {\n\t// create a factory\n\tf := &factorymocks.Factory{}\n\tbackupName := \"bk-describe-1\"\n\ttestBackup := builder.ForBackup(cmdtest.VeleroNameSpace, backupName).SnapshotVolumes(false).Result()\n\n\tclientConfig := rest.Config{}\n\tkbClient := test.NewFakeControllerRuntimeClient(t)\n\tkbClient.Create(t.Context(), testBackup, &controllerclient.CreateOptions{})\n\n\tf.On(\"ClientConfig\").Return(&clientConfig, nil)\n\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\tf.On(\"KubebuilderClient\").Return(kbClient, nil)\n\n\t// create command\n\tc := NewDescribeCommand(f, \"velero backup describe\")\n\tassert.Equal(t, \"Describe backups\", c.Short)\n\n\tfeatures.NewFeatureFlagSet(\"EnableCSI\")\n\tdefer features.NewFeatureFlagSet()\n\n\tc.SetArgs([]string{backupName})\n\te := c.Execute()\n\trequire.NoError(t, e)\n\n\tif os.Getenv(cmdtest.CaptureFlag) == \"1\" {\n\t\treturn\n\t}\n\tcmd := exec.CommandContext(t.Context(), os.Args[0], []string{\"-test.run=TestNewDescribeCommand\"}...)\n\tcmd.Env = append(os.Environ(), fmt.Sprintf(\"%s=1\", cmdtest.CaptureFlag))\n\tstdout, _, err := veleroexec.RunCommand(cmd)\n\n\tif err == nil {\n\t\tassert.Contains(t, stdout, \"Backup Volumes:\")\n\t\tassert.Contains(t, stdout, \"Or label selector: \")\n\t\tassert.Contains(t, stdout, fmt.Sprintf(\"Name: %s\", backupName))\n\t\treturn\n\t}\n\tt.Fatalf(\"process ran with err %v, want backups by get()\", err)\n}\n" }, { "path": "pkg/cmd/cli/backup/download.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/spf13/cobra\"\n\t\"github.com/spf13/pflag\"\n\tcontrollerclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/cacert\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/downloadrequest\"\n)\n\nfunc NewDownloadCommand(f client.Factory) *cobra.Command {\n\tconfig, err := client.LoadConfig()\n\tif err != nil {\n\t\tfmt.Fprintf(os.Stderr, \"WARNING: Error reading config file: %v\\n\", err)\n\t}\n\to := NewDownloadOptions()\n\to.caCertFile = config.CACertFile()\n\n\tc := &cobra.Command{\n\t\tUse: \"download NAME\",\n\t\tShort: \"Download all Kubernetes manifests for a backup\",\n\t\tLong: \"Download all Kubernetes manifests for a backup. Contents of persistent volume snapshots are not included.\",\n\t\tArgs: cobra.ExactArgs(1),\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tcmd.CheckError(o.Complete(args))\n\t\t\tcmd.CheckError(o.Validate(c, args, f))\n\t\t\tcmd.CheckError(o.Run(c, f))\n\t\t},\n\t}\n\n\to.BindFlags(c.Flags())\n\n\treturn c\n}\n\ntype DownloadOptions struct {\n\tName string\n\tOutput string\n\tForce bool\n\tTimeout time.Duration\n\tInsecureSkipTLSVerify bool\n\twriteOptions int\n\tcaCertFile string\n}\n\nfunc NewDownloadOptions() *DownloadOptions {\n\treturn &DownloadOptions{\n\t\tTimeout: time.Minute,\n\t}\n}\n\nfunc (o *DownloadOptions) BindFlags(flags *pflag.FlagSet) {\n\tflags.StringVarP(&o.Output, \"output\", \"o\", o.Output, \"Path to output file. Defaults to -data.tar.gz in the current directory.\")\n\tflags.BoolVar(&o.Force, \"force\", o.Force, \"Forces the download and will overwrite file if it exists already.\")\n\tflags.DurationVar(&o.Timeout, \"timeout\", o.Timeout, \"Maximum time to wait to process download request.\")\n\tflags.BoolVar(&o.InsecureSkipTLSVerify, \"insecure-skip-tls-verify\", o.InsecureSkipTLSVerify, \"If true, the object store's TLS certificate will not be checked for validity. This is insecure and susceptible to man-in-the-middle attacks. Not recommended for production.\")\n\tflags.StringVar(&o.caCertFile, \"cacert\", o.caCertFile, \"Path to a certificate bundle to use when verifying TLS connections. If not specified, the CA certificate from the BackupStorageLocation will be used if available.\")\n}\n\nfunc (o *DownloadOptions) Validate(c *cobra.Command, args []string, f client.Factory) error {\n\tkbClient, err := f.KubebuilderClient()\n\tcmd.CheckError(err)\n\n\tbackup := new(velerov1api.Backup)\n\tif err := kbClient.Get(context.Background(), controllerclient.ObjectKey{Namespace: f.Namespace(), Name: o.Name}, backup); err != nil {\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\nfunc (o *DownloadOptions) Complete(args []string) error {\n\to.Name = args[0]\n\n\to.writeOptions = os.O_RDWR | os.O_CREATE | os.O_EXCL\n\tif o.Force {\n\t\to.writeOptions = os.O_RDWR | os.O_CREATE | os.O_TRUNC\n\t}\n\n\tif o.Output == \"\" {\n\t\tpath, err := os.Getwd()\n\t\tif err != nil {\n\t\t\treturn errors.Wrapf(err, \"error getting current directory\")\n\t\t}\n\t\to.Output = filepath.Join(path, fmt.Sprintf(\"%s-data.tar.gz\", o.Name))\n\t}\n\n\treturn nil\n}\n\nfunc (o *DownloadOptions) Run(c *cobra.Command, f client.Factory) error {\n\tkbClient, err := f.KubebuilderClient()\n\tcmd.CheckError(err)\n\n\t// Get the backup to fetch BSL cacert\n\tbackup := new(velerov1api.Backup)\n\tif err := kbClient.Get(context.Background(), controllerclient.ObjectKey{Namespace: f.Namespace(), Name: o.Name}, backup); err != nil {\n\t\treturn err\n\t}\n\n\t// Get BSL cacert if available\n\tbslCACert, err := cacert.GetCACertFromBackup(context.Background(), kbClient, f.Namespace(), backup)\n\tif err != nil {\n\t\t// Log the error but don't fail - we can still try to download without the BSL cacert\n\t\tfmt.Fprintf(os.Stderr, \"WARNING: Error getting cacert from BSL: %v\\n\", err)\n\t\tbslCACert = \"\"\n\t}\n\n\tbackupDest, err := os.OpenFile(o.Output, o.writeOptions, 0600)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer backupDest.Close()\n\n\terr = downloadrequest.StreamWithBSLCACert(context.Background(), kbClient, f.Namespace(), o.Name, velerov1api.DownloadTargetKindBackupContents, backupDest, o.Timeout, o.InsecureSkipTLSVerify, o.caCertFile, bslCACert)\n\tif err != nil {\n\t\tos.Remove(o.Output)\n\t\tcmd.CheckError(err)\n\t}\n\n\tfmt.Printf(\"Backup %s has been successfully downloaded to %s\\n\", o.Name, backupDest.Name())\n\treturn nil\n}\n" }, { "path": "pkg/cmd/cli/backup/download_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"strconv\"\n\t\"testing\"\n\n\tflag \"github.com/spf13/pflag\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\tcmdtest \"github.com/vmware-tanzu/velero/pkg/cmd/test\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\tveleroexec \"github.com/vmware-tanzu/velero/pkg/util/exec\"\n)\n\nfunc TestNewDownloadCommand(t *testing.T) {\n\t// create a factory\n\tf := &factorymocks.Factory{}\n\n\tbackupName := \"backup-1\"\n\tkbclient := velerotest.NewFakeControllerRuntimeClient(t)\n\terr := kbclient.Create(t.Context(), builder.ForBackup(cmdtest.VeleroNameSpace, backupName).Result())\n\trequire.NoError(t, err)\n\terr = kbclient.Create(t.Context(), builder.ForBackup(cmdtest.VeleroNameSpace, \"bk-to-be-download\").Result())\n\trequire.NoError(t, err)\n\n\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\tf.On(\"KubebuilderClient\").Return(kbclient, nil)\n\n\t// create command\n\tc := NewDownloadCommand(f)\n\tc.SetArgs([]string{\"bk-to-be-download\"})\n\tassert.Equal(t, \"Download all Kubernetes manifests for a backup\", c.Short)\n\n\t// create a DownloadOptions with full options set and then run this backup command\n\toutput := \"path/to/download/bk.json\"\n\tforce := true\n\ttimeout := \"1m30s\"\n\tinsecureSkipTlsVerify := false\n\tcacert := \"secret=YHJKKS\"\n\n\tflags := new(flag.FlagSet)\n\to := NewDownloadOptions()\n\to.BindFlags(flags)\n\n\tflags.Parse([]string{\"--output\", output})\n\tflags.Parse([]string{\"--force\"})\n\tflags.Parse([]string{\"--timeout\", timeout})\n\tflags.Parse([]string{fmt.Sprintf(\"--insecure-skip-tls-verify=%s\", strconv.FormatBool(insecureSkipTlsVerify))})\n\tflags.Parse([]string{\"--cacert\", cacert})\n\n\targs := []string{backupName, \"arg2\"}\n\n\te := o.Complete(args)\n\trequire.NoError(t, e)\n\n\te = o.Validate(c, args, f)\n\trequire.NoError(t, e)\n\n\t// verify all options are set as expected\n\tassert.Equal(t, output, o.Output)\n\tassert.Equal(t, force, o.Force)\n\tassert.Equal(t, timeout, o.Timeout.String())\n\tassert.Equal(t, insecureSkipTlsVerify, o.InsecureSkipTLSVerify)\n\tassert.Equal(t, cacert, o.caCertFile)\n\n\tif os.Getenv(cmdtest.CaptureFlag) == \"1\" {\n\t\te = c.Execute()\n\t\tdefer os.Remove(\"bk-to-be-download-data.tar.gz\")\n\t\tassert.NoError(t, e)\n\t\treturn\n\t}\n\tcmd := exec.CommandContext(t.Context(), os.Args[0], []string{\"-test.run=TestNewDownloadCommand\"}...)\n\tcmd.Env = append(os.Environ(), fmt.Sprintf(\"%s=1\", cmdtest.CaptureFlag))\n\t_, stderr, err := veleroexec.RunCommand(cmd)\n\n\tif err != nil {\n\t\trequire.Contains(t, stderr, \"download request download url timeout\")\n\t\treturn\n\t}\n\tt.Fatalf(\"process ran with err %v, want backup delete successfully\", err)\n}\n" }, { "path": "pkg/cmd/cli/backup/get.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"context\"\n\n\t\"github.com/spf13/cobra\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/output\"\n)\n\nfunc NewGetCommand(f client.Factory, use string) *cobra.Command {\n\tvar listOptions metav1.ListOptions\n\n\tc := &cobra.Command{\n\t\tUse: use,\n\t\tShort: \"Get backups\",\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\terr := output.ValidateFlags(c)\n\t\t\tcmd.CheckError(err)\n\n\t\t\tkbClient, err := f.KubebuilderClient()\n\t\t\tcmd.CheckError(err)\n\n\t\t\tbackups := new(api.BackupList)\n\t\t\tif len(args) > 0 {\n\t\t\t\tfor _, name := range args {\n\t\t\t\t\tbackup := new(api.Backup)\n\t\t\t\t\terr := kbClient.Get(context.TODO(), kbclient.ObjectKey{Namespace: f.Namespace(), Name: name}, backup)\n\t\t\t\t\tcmd.CheckError(err)\n\t\t\t\t\tbackups.Items = append(backups.Items, *backup)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tparsedSelector, err := labels.Parse(listOptions.LabelSelector)\n\t\t\t\tcmd.CheckError(err)\n\t\t\t\terr = kbClient.List(context.TODO(), backups, &kbclient.ListOptions{\n\t\t\t\t\tLabelSelector: parsedSelector,\n\t\t\t\t\tNamespace: f.Namespace(),\n\t\t\t\t})\n\t\t\t\tcmd.CheckError(err)\n\t\t\t}\n\n\t\t\t_, err = output.PrintWithFormat(c, backups)\n\t\t\tcmd.CheckError(err)\n\t\t},\n\t}\n\n\tc.Flags().StringVarP(&listOptions.LabelSelector, \"selector\", \"l\", listOptions.LabelSelector, \"Only show items matching this label selector\")\n\n\toutput.BindFlags(c.Flags())\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/backup/get_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\tcmdtest \"github.com/vmware-tanzu/velero/pkg/cmd/test\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\tveleroexec \"github.com/vmware-tanzu/velero/pkg/util/exec\"\n)\n\nfunc TestNewGetCommand(t *testing.T) {\n\targs := []string{\"b1\", \"b2\", \"b3\"}\n\n\t// create a factory\n\tf := &factorymocks.Factory{}\n\n\tclient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\tfor _, backupName := range args {\n\t\tbackup := builder.ForBackup(cmdtest.VeleroNameSpace, backupName).ObjectMeta(builder.WithLabels(\"abc\", \"abc\")).Result()\n\t\terr := client.Create(t.Context(), backup, &kbclient.CreateOptions{})\n\t\trequire.NoError(t, err)\n\t}\n\n\tf.On(\"KubebuilderClient\").Return(client, nil)\n\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\n\t// create command\n\tc := NewGetCommand(f, \"velero backup get\")\n\tassert.Equal(t, \"Get backups\", c.Short)\n\n\tc.SetArgs(args)\n\te := c.Execute()\n\trequire.NoError(t, e)\n\n\tif os.Getenv(cmdtest.CaptureFlag) == \"1\" {\n\t\treturn\n\t}\n\n\tcmd := exec.CommandContext(t.Context(), os.Args[0], []string{\"-test.run=TestNewGetCommand\"}...)\n\tcmd.Env = append(os.Environ(), fmt.Sprintf(\"%s=1\", cmdtest.CaptureFlag))\n\tstdout, _, err := veleroexec.RunCommand(cmd)\n\trequire.NoError(t, err)\n\n\tif err == nil {\n\t\toutput := strings.Split(stdout, \"\\n\")\n\t\ti := 0\n\t\tfor _, line := range output {\n\t\t\tif strings.Contains(line, \"New\") {\n\t\t\t\ti++\n\t\t\t}\n\t\t}\n\t\tassert.Len(t, args, i)\n\t}\n\n\td := NewGetCommand(f, \"velero backup get\")\n\tc.SetArgs([]string{\"-l\", \"abc=abc\"})\n\te = d.Execute()\n\trequire.NoError(t, e)\n\n\tcmd = exec.CommandContext(t.Context(), os.Args[0], []string{\"-test.run=TestNewGetCommand\"}...)\n\tcmd.Env = append(os.Environ(), fmt.Sprintf(\"%s=1\", cmdtest.CaptureFlag))\n\tstdout, _, err = veleroexec.RunCommand(cmd)\n\trequire.NoError(t, err)\n\n\tif err == nil {\n\t\toutput := strings.Split(stdout, \"\\n\")\n\t\ti := 0\n\t\tfor _, line := range output {\n\t\t\tif strings.Contains(line, \"New\") {\n\t\t\t\ti++\n\t\t\t}\n\t\t}\n\t\tassert.Len(t, args, i)\n\t}\n}\n" }, { "path": "pkg/cmd/cli/backup/logs.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"time\"\n\n\t\"github.com/spf13/cobra\"\n\t\"github.com/spf13/pflag\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/cacert\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/downloadrequest\"\n)\n\ntype LogsOptions struct {\n\tTimeout time.Duration\n\tInsecureSkipTLSVerify bool\n\tCaCertFile string\n\tClient kbclient.Client\n\tBackupName string\n}\n\nfunc NewLogsOptions() LogsOptions {\n\tconfig, err := client.LoadConfig()\n\tif err != nil {\n\t\tfmt.Fprintf(os.Stderr, \"WARNING: Error reading config file: %v\\n\", err)\n\t}\n\n\treturn LogsOptions{\n\t\tTimeout: time.Minute,\n\t\tInsecureSkipTLSVerify: false,\n\t\tCaCertFile: config.CACertFile(),\n\t}\n}\n\nfunc (l *LogsOptions) BindFlags(flags *pflag.FlagSet) {\n\tflags.DurationVar(&l.Timeout, \"timeout\", l.Timeout, \"How long to wait to receive logs.\")\n\tflags.BoolVar(&l.InsecureSkipTLSVerify, \"insecure-skip-tls-verify\", l.InsecureSkipTLSVerify, \"If true, the object store's TLS certificate will not be checked for validity. This is insecure and susceptible to man-in-the-middle attacks. Not recommended for production.\")\n\tflags.StringVar(&l.CaCertFile, \"cacert\", l.CaCertFile, \"Path to a certificate bundle to use when verifying TLS connections.\")\n}\n\nfunc (l *LogsOptions) Run(c *cobra.Command, f client.Factory) error {\n\tbackup := new(velerov1api.Backup)\n\terr := l.Client.Get(context.Background(), kbclient.ObjectKey{Namespace: f.Namespace(), Name: l.BackupName}, backup)\n\tif apierrors.IsNotFound(err) {\n\t\treturn fmt.Errorf(\"backup %q does not exist\", l.BackupName)\n\t} else if err != nil {\n\t\treturn fmt.Errorf(\"error checking for backup %q: %v\", l.BackupName, err)\n\t}\n\n\tswitch backup.Status.Phase {\n\tcase velerov1api.BackupPhaseCompleted, velerov1api.BackupPhasePartiallyFailed, velerov1api.BackupPhaseFailed, velerov1api.BackupPhaseWaitingForPluginOperations, velerov1api.BackupPhaseWaitingForPluginOperationsPartiallyFailed:\n\t\t// terminal and waiting for plugin operations phases, do nothing.\n\tdefault:\n\t\treturn fmt.Errorf(\"logs for backup %q are not available until it's finished processing, please wait \"+\n\t\t\t\"until the backup has a phase of Completed or Failed and try again\", l.BackupName)\n\t}\n\n\t// Get BSL cacert if available\n\tbslCACert, err := cacert.GetCACertFromBackup(context.Background(), l.Client, f.Namespace(), backup)\n\tif err != nil {\n\t\t// Log the error but don't fail - we can still try to download without the BSL cacert\n\t\tfmt.Fprintf(os.Stderr, \"WARNING: Error getting cacert from BSL: %v\\n\", err)\n\t\tbslCACert = \"\"\n\t}\n\n\terr = downloadrequest.StreamWithBSLCACert(context.Background(), l.Client, f.Namespace(), l.BackupName, velerov1api.DownloadTargetKindBackupLog, os.Stdout, l.Timeout, l.InsecureSkipTLSVerify, l.CaCertFile, bslCACert)\n\treturn err\n}\n\nfunc (l *LogsOptions) Complete(args []string, f client.Factory) error {\n\tif len(args) > 0 {\n\t\tl.BackupName = args[0]\n\t}\n\n\tkbClient, err := f.KubebuilderClient()\n\tif err != nil {\n\t\treturn err\n\t}\n\tl.Client = kbClient\n\treturn nil\n}\n\nfunc NewLogsCommand(f client.Factory) *cobra.Command {\n\tl := NewLogsOptions()\n\n\tc := &cobra.Command{\n\t\tUse: \"logs BACKUP\",\n\t\tShort: \"Get backup logs\",\n\t\tArgs: cobra.ExactArgs(1),\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\terr := l.Complete(args, f)\n\t\t\tcmd.CheckError(err)\n\n\t\t\terr = l.Run(c, f)\n\t\t\tcmd.CheckError(err)\n\t\t},\n\t}\n\n\tl.BindFlags(c.Flags())\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/backup/logs_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backup\n\nimport (\n\t\"bytes\"\n\t\"compress/gzip\"\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"os\"\n\t\"strconv\"\n\t\"testing\"\n\t\"time\"\n\n\tflag \"github.com/spf13/pflag\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\tcmdtest \"github.com/vmware-tanzu/velero/pkg/cmd/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/cacert\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestNewLogsCommand(t *testing.T) {\n\tt.Run(\"Flag test\", func(t *testing.T) {\n\t\tl := NewLogsOptions()\n\t\tflags := new(flag.FlagSet)\n\t\tl.BindFlags(flags)\n\n\t\ttimeout := \"1m0s\"\n\t\tinsecureSkipTLSVerify := \"true\"\n\t\tcaCertFile := \"testing\"\n\n\t\tflags.Parse([]string{\"--timeout\", timeout})\n\t\tflags.Parse([]string{\"--insecure-skip-tls-verify\", insecureSkipTLSVerify})\n\t\tflags.Parse([]string{\"--cacert\", caCertFile})\n\n\t\trequire.Equal(t, timeout, l.Timeout.String())\n\t\trequire.Equal(t, insecureSkipTLSVerify, strconv.FormatBool(l.InsecureSkipTLSVerify))\n\t\trequire.Equal(t, caCertFile, l.CaCertFile)\n\t})\n\n\tt.Run(\"Backup not complete test\", func(t *testing.T) {\n\t\tbackupName := \"bk-logs-1\"\n\n\t\t// create a factory\n\t\tf := &factorymocks.Factory{}\n\n\t\tkbClient := velerotest.NewFakeControllerRuntimeClient(t)\n\t\tbackup := builder.ForBackup(cmdtest.VeleroNameSpace, backupName).Result()\n\t\terr := kbClient.Create(t.Context(), backup, &kbclient.CreateOptions{})\n\t\trequire.NoError(t, err)\n\n\t\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\t\tf.On(\"KubebuilderClient\").Return(kbClient, nil)\n\n\t\tc := NewLogsCommand(f)\n\t\tassert.Equal(t, \"Get backup logs\", c.Short)\n\n\t\tl := NewLogsOptions()\n\t\tflags := new(flag.FlagSet)\n\t\tl.BindFlags(flags)\n\t\terr = l.Complete([]string{backupName}, f)\n\t\trequire.NoError(t, err)\n\n\t\terr = l.Run(c, f)\n\t\trequire.Error(t, err)\n\t\trequire.ErrorContains(t, err, fmt.Sprintf(\"logs for backup \\\"%s\\\" are not available until it's finished processing\", backupName))\n\t})\n\n\tt.Run(\"Backup not exist test\", func(t *testing.T) {\n\t\tbackupName := \"not-exist\"\n\t\t// create a factory\n\t\tf := &factorymocks.Factory{}\n\n\t\tkbClient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\t\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\t\tf.On(\"KubebuilderClient\").Return(kbClient, nil)\n\n\t\tc := NewLogsCommand(f)\n\t\tassert.Equal(t, \"Get backup logs\", c.Short)\n\n\t\tl := NewLogsOptions()\n\t\tflags := new(flag.FlagSet)\n\t\tl.BindFlags(flags)\n\t\terr := l.Complete([]string{backupName}, f)\n\t\trequire.NoError(t, err)\n\n\t\terr = l.Run(c, f)\n\t\trequire.Error(t, err)\n\n\t\trequire.Equal(t, fmt.Sprintf(\"backup \\\"%s\\\" does not exist\", backupName), err.Error())\n\n\t\tc.Execute()\n\t})\n\n\tt.Run(\"Normal backup log test\", func(t *testing.T) {\n\t\tbackupName := \"bk-logs-1\"\n\n\t\t// create a factory\n\t\tf := &factorymocks.Factory{}\n\n\t\tkbClient := velerotest.NewFakeControllerRuntimeClient(t)\n\t\tbackup := builder.ForBackup(cmdtest.VeleroNameSpace, backupName).Phase(velerov1api.BackupPhaseCompleted).Result()\n\t\terr := kbClient.Create(t.Context(), backup, &kbclient.CreateOptions{})\n\t\trequire.NoError(t, err)\n\n\t\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\t\tf.On(\"KubebuilderClient\").Return(kbClient, nil)\n\n\t\tc := NewLogsCommand(f)\n\t\tassert.Equal(t, \"Get backup logs\", c.Short)\n\n\t\tl := NewLogsOptions()\n\t\tflags := new(flag.FlagSet)\n\t\tl.BindFlags(flags)\n\t\terr = l.Complete([]string{backupName}, f)\n\t\trequire.NoError(t, err)\n\n\t\ttimeout := time.After(3 * time.Second)\n\t\tdone := make(chan bool)\n\t\tgo func() {\n\t\t\terr = l.Run(c, f)\n\t\t\trequire.Error(t, err)\n\t\t}()\n\n\t\tselect {\n\t\tcase <-timeout:\n\t\t\tt.Skip(\"Test didn't finish in time, because BSL is not in Available state.\")\n\t\tcase <-done:\n\t\t}\n\t})\n\n\tt.Run(\"Invalid client test\", func(t *testing.T) {\n\t\t// create a factory\n\t\tf := &factorymocks.Factory{}\n\n\t\tkbClient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\t\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\n\t\tc := NewLogsCommand(f)\n\t\tassert.Equal(t, \"Get backup logs\", c.Short)\n\n\t\tl := NewLogsOptions()\n\t\tflags := new(flag.FlagSet)\n\t\tl.BindFlags(flags)\n\n\t\tf.On(\"KubebuilderClient\").Return(kbClient, fmt.Errorf(\"test error\"))\n\t\terr := l.Complete([]string{\"\"}, f)\n\t\trequire.Equal(t, \"test error\", err.Error())\n\t})\n\n\tt.Run(\"Backup with BSL cacert test\", func(t *testing.T) {\n\t\tbackupName := \"bk-logs-with-cacert\"\n\t\tbslName := \"test-bsl\"\n\t\texpectedCACert := \"test-cacert-content\"\n\t\texpectedLogContent := \"test backup log content\"\n\n\t\t// create a factory\n\t\tf := &factorymocks.Factory{}\n\n\t\tkbClient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\t\t// Create BSL with cacert\n\t\tbsl := builder.ForBackupStorageLocation(cmdtest.VeleroNameSpace, bslName).\n\t\t\tProvider(\"aws\").\n\t\t\tBucket(\"test-bucket\").\n\t\t\tCACert([]byte(expectedCACert)).\n\t\t\tResult()\n\t\terr := kbClient.Create(t.Context(), bsl, &kbclient.CreateOptions{})\n\t\trequire.NoError(t, err)\n\n\t\t// Create backup referencing the BSL\n\t\tbackup := builder.ForBackup(cmdtest.VeleroNameSpace, backupName).\n\t\t\tPhase(velerov1api.BackupPhaseCompleted).\n\t\t\tStorageLocation(bslName).\n\t\t\tResult()\n\t\terr = kbClient.Create(t.Context(), backup, &kbclient.CreateOptions{})\n\t\trequire.NoError(t, err)\n\n\t\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\t\tf.On(\"KubebuilderClient\").Return(kbClient, nil)\n\n\t\tc := NewLogsCommand(f)\n\t\tassert.Equal(t, \"Get backup logs\", c.Short)\n\n\t\tl := NewLogsOptions()\n\t\tflags := new(flag.FlagSet)\n\t\tl.BindFlags(flags)\n\t\terr = l.Complete([]string{backupName}, f)\n\t\trequire.NoError(t, err)\n\n\t\t// Verify that the BSL cacert can be fetched correctly before running the command\n\t\tfetchedBackup := &velerov1api.Backup{}\n\t\terr = kbClient.Get(t.Context(), kbclient.ObjectKey{Namespace: cmdtest.VeleroNameSpace, Name: backupName}, fetchedBackup)\n\t\trequire.NoError(t, err)\n\n\t\t// Test the cacert fetching logic directly\n\t\tcacertValue, err := cacert.GetCACertFromBackup(t.Context(), kbClient, cmdtest.VeleroNameSpace, fetchedBackup)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, expectedCACert, cacertValue, \"BSL cacert should be retrieved correctly\")\n\n\t\t// Create a mock HTTP server to serve the log content\n\t\tmockServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t\t// For logs, we need to gzip the content\n\t\t\tgzipWriter := gzip.NewWriter(w)\n\t\t\tdefer gzipWriter.Close()\n\t\t\tgzipWriter.Write([]byte(expectedLogContent))\n\t\t}))\n\t\tdefer mockServer.Close()\n\n\t\t// Mock the download request controller by updating DownloadRequests\n\t\tgo func() {\n\t\t\ttime.Sleep(50 * time.Millisecond) // Wait a bit for the request to be created\n\n\t\t\t// List all DownloadRequests\n\t\t\tdownloadRequestList := &velerov1api.DownloadRequestList{}\n\t\t\tif err := kbClient.List(t.Context(), downloadRequestList, &kbclient.ListOptions{\n\t\t\t\tNamespace: cmdtest.VeleroNameSpace,\n\t\t\t}); err == nil {\n\t\t\t\t// Update each download request with the mock server URL\n\t\t\t\tfor _, dr := range downloadRequestList.Items {\n\t\t\t\t\tif dr.Spec.Target.Kind == velerov1api.DownloadTargetKindBackupLog &&\n\t\t\t\t\t\tdr.Spec.Target.Name == backupName {\n\t\t\t\t\t\tdr.Status.DownloadURL = mockServer.URL\n\t\t\t\t\t\tdr.Status.Phase = velerov1api.DownloadRequestPhaseProcessed\n\t\t\t\t\t\tkbClient.Update(t.Context(), &dr)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}()\n\n\t\t// Capture the output\n\t\tvar logOutput bytes.Buffer\n\t\t// Temporarily redirect stdout\n\t\toldStdout := os.Stdout\n\t\tr, w, _ := os.Pipe()\n\t\tos.Stdout = w\n\n\t\t// Run the logs command - it should now succeed\n\t\tl.Timeout = 5 * time.Second\n\t\terr = l.Run(c, f)\n\n\t\t// Restore stdout and read the output\n\t\tw.Close()\n\t\tos.Stdout = oldStdout\n\t\tio.Copy(&logOutput, r)\n\n\t\t// Verify the command succeeded and output is correct\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, expectedLogContent, logOutput.String())\n\t})\n}\n\nfunc TestBSLCACertBehavior(t *testing.T) {\n\tt.Run(\"Backup with BSL without cacert test\", func(t *testing.T) {\n\t\tbackupName := \"bk-logs-without-cacert\"\n\t\tbslName := \"test-bsl-no-cacert\"\n\n\t\t// create a factory\n\t\tf := &factorymocks.Factory{}\n\n\t\tkbClient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\t\t// Create BSL without cacert\n\t\tbsl := builder.ForBackupStorageLocation(cmdtest.VeleroNameSpace, bslName).\n\t\t\tProvider(\"aws\").\n\t\t\tBucket(\"test-bucket\").\n\t\t\t// No CACert() call - BSL will have no cacert\n\t\t\tResult()\n\t\terr := kbClient.Create(t.Context(), bsl, &kbclient.CreateOptions{})\n\t\trequire.NoError(t, err)\n\n\t\t// Create backup referencing the BSL\n\t\tbackup := builder.ForBackup(cmdtest.VeleroNameSpace, backupName).\n\t\t\tPhase(velerov1api.BackupPhaseCompleted).\n\t\t\tStorageLocation(bslName).\n\t\t\tResult()\n\t\terr = kbClient.Create(t.Context(), backup, &kbclient.CreateOptions{})\n\t\trequire.NoError(t, err)\n\n\t\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\t\tf.On(\"KubebuilderClient\").Return(kbClient, nil)\n\n\t\tc := NewLogsCommand(f)\n\t\tassert.Equal(t, \"Get backup logs\", c.Short)\n\n\t\tl := NewLogsOptions()\n\t\tflags := new(flag.FlagSet)\n\t\tl.BindFlags(flags)\n\t\terr = l.Complete([]string{backupName}, f)\n\t\trequire.NoError(t, err)\n\n\t\t// Verify that the BSL cacert returns empty string when not present\n\t\tfetchedBackup := &velerov1api.Backup{}\n\t\terr = kbClient.Get(t.Context(), kbclient.ObjectKey{Namespace: cmdtest.VeleroNameSpace, Name: backupName}, fetchedBackup)\n\t\trequire.NoError(t, err)\n\n\t\t// Test the cacert fetching logic directly\n\t\tcacertValue, err := cacert.GetCACertFromBackup(t.Context(), kbClient, cmdtest.VeleroNameSpace, fetchedBackup)\n\t\trequire.NoError(t, err)\n\t\tassert.Empty(t, cacertValue, \"BSL cacert should be empty when not configured\")\n\n\t\t// The command should still work without cacert\n\t\tl.Timeout = 100 * time.Millisecond\n\t\terr = l.Run(c, f)\n\t\trequire.Error(t, err)\n\t\t// The error should be about download request timeout, not about cacert fetching\n\t\tassert.Contains(t, err.Error(), \"download\")\n\t})\n\n\tt.Run(\"Backup with nonexistent BSL test\", func(t *testing.T) {\n\t\tbackupName := \"bk-logs-with-missing-bsl\"\n\t\tbslName := \"nonexistent-bsl\"\n\n\t\t// create a factory\n\t\tf := &factorymocks.Factory{}\n\n\t\tkbClient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\t\t// Create backup referencing a BSL that doesn't exist\n\t\tbackup := builder.ForBackup(cmdtest.VeleroNameSpace, backupName).\n\t\t\tPhase(velerov1api.BackupPhaseCompleted).\n\t\t\tStorageLocation(bslName).\n\t\t\tResult()\n\t\terr := kbClient.Create(t.Context(), backup, &kbclient.CreateOptions{})\n\t\trequire.NoError(t, err)\n\n\t\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\t\tf.On(\"KubebuilderClient\").Return(kbClient, nil)\n\n\t\tc := NewLogsCommand(f)\n\t\tl := NewLogsOptions()\n\t\tflags := new(flag.FlagSet)\n\t\tl.BindFlags(flags)\n\t\terr = l.Complete([]string{backupName}, f)\n\t\trequire.NoError(t, err)\n\n\t\t// Verify that the BSL cacert returns empty string when BSL doesn't exist\n\t\tfetchedBackup := &velerov1api.Backup{}\n\t\terr = kbClient.Get(t.Context(), kbclient.ObjectKey{Namespace: cmdtest.VeleroNameSpace, Name: backupName}, fetchedBackup)\n\t\trequire.NoError(t, err)\n\n\t\t// Test the cacert fetching logic directly - should not error when BSL is missing\n\t\tcacertValue, err := cacert.GetCACertFromBackup(t.Context(), kbClient, cmdtest.VeleroNameSpace, fetchedBackup)\n\t\trequire.NoError(t, err)\n\t\tassert.Empty(t, cacertValue, \"BSL cacert should be empty when BSL doesn't exist\")\n\n\t\t// The command should still try to run even without BSL\n\t\tl.Timeout = 100 * time.Millisecond\n\t\terr = l.Run(c, f)\n\t\trequire.Error(t, err)\n\t\tassert.Contains(t, err.Error(), \"download\")\n\t})\n}\n" }, { "path": "pkg/cmd/cli/backuplocation/backup_location.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backuplocation\n\nimport (\n\t\"github.com/spf13/cobra\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n)\n\nfunc NewCommand(f client.Factory) *cobra.Command {\n\tc := &cobra.Command{\n\t\tUse: \"backup-location\",\n\t\tShort: \"Work with backup storage locations\",\n\t\tLong: \"Work with backup storage locations\",\n\t}\n\n\tc.AddCommand(\n\t\tNewCreateCommand(f, \"create\"),\n\t\tNewDeleteCommand(f, \"delete\"),\n\t\tNewGetCommand(f, \"get\"),\n\t\tNewSetCommand(f, \"set\"),\n\t)\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/backuplocation/backup_location_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backuplocation\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n)\n\nfunc TestNewCommand(t *testing.T) {\n\t// create a factory\n\tf := &factorymocks.Factory{}\n\n\t// create command\n\tc := NewCommand(f)\n\tassert.Equal(t, \"Work with backup storage locations\", c.Short)\n}\n" }, { "path": "pkg/cmd/cli/backuplocation/create.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backuplocation\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/spf13/cobra\"\n\t\"github.com/spf13/pflag\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/storage\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/flag\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/output\"\n)\n\nfunc NewCreateCommand(f client.Factory, use string) *cobra.Command {\n\to := NewCreateOptions()\n\n\tc := &cobra.Command{\n\t\tUse: use + \" NAME\",\n\t\tShort: \"Create a backup storage location\",\n\t\tArgs: cobra.ExactArgs(1),\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tcmd.CheckError(o.Complete(args, f))\n\t\t\tcmd.CheckError(o.Validate(c, args, f))\n\t\t\tcmd.CheckError(o.Run(c, f))\n\t\t},\n\t}\n\n\to.BindFlags(c.Flags())\n\toutput.BindFlags(c.Flags())\n\toutput.ClearOutputFlagDefault(c)\n\n\treturn c\n}\n\ntype CreateOptions struct {\n\tName string\n\tProvider string\n\tBucket string\n\tCredential flag.Map\n\tDefaultBackupStorageLocation bool\n\tPrefix string\n\tBackupSyncPeriod, ValidationFrequency time.Duration\n\tConfig flag.Map\n\tLabels flag.Map\n\tCACertFile string\n\tAccessMode *flag.Enum\n}\n\nfunc NewCreateOptions() *CreateOptions {\n\treturn &CreateOptions{\n\t\tCredential: flag.NewMap(),\n\t\tConfig: flag.NewMap(),\n\t\tLabels: flag.NewMap(),\n\t\tAccessMode: flag.NewEnum(\n\t\t\tstring(velerov1api.BackupStorageLocationAccessModeReadWrite),\n\t\t\tstring(velerov1api.BackupStorageLocationAccessModeReadWrite),\n\t\t\tstring(velerov1api.BackupStorageLocationAccessModeReadOnly),\n\t\t),\n\t}\n}\n\nfunc (o *CreateOptions) BindFlags(flags *pflag.FlagSet) {\n\tflags.StringVar(&o.Provider, \"provider\", o.Provider, \"Name of the backup storage provider (e.g. aws, azure, gcp).\")\n\tflags.StringVar(&o.Bucket, \"bucket\", o.Bucket, \"Name of the object storage bucket where backups should be stored.\")\n\tflags.Var(&o.Credential, \"credential\", \"The credential to be used by this location as a key-value pair, where the key is the Kubernetes Secret name, and the value is the data key name within the Secret. Optional, one value only.\")\n\tflags.BoolVar(&o.DefaultBackupStorageLocation, \"default\", o.DefaultBackupStorageLocation, \"Sets this new location to be the new default backup storage location. Optional.\")\n\tflags.StringVar(&o.Prefix, \"prefix\", o.Prefix, \"Prefix under which all Velero data should be stored within the bucket. Optional.\")\n\tflags.DurationVar(&o.BackupSyncPeriod, \"backup-sync-period\", o.BackupSyncPeriod, \"How often to ensure all Velero backups in object storage exist as Backup API objects in the cluster. Optional. Set this to `0s` to disable sync. Default: 1 minute.\")\n\tflags.DurationVar(&o.ValidationFrequency, \"validation-frequency\", o.ValidationFrequency, \"How often to verify if the backup storage location is valid. Optional. Set this to `0s` to disable sync. Default 1 minute.\")\n\tflags.Var(&o.Config, \"config\", \"Configuration key-value pairs.\")\n\tflags.Var(&o.Labels, \"labels\", \"Labels to apply to the backup storage location.\")\n\tflags.StringVar(&o.CACertFile, \"cacert\", o.CACertFile, \"File containing a certificate bundle to use when verifying TLS connections to the object store. Optional.\")\n\tflags.Var(\n\t\to.AccessMode,\n\t\t\"access-mode\",\n\t\tfmt.Sprintf(\"Access mode for the backup storage location. Valid values are %s\", strings.Join(o.AccessMode.AllowedValues(), \",\")),\n\t)\n}\n\nfunc (o *CreateOptions) Validate(c *cobra.Command, args []string, f client.Factory) error {\n\tif err := output.ValidateFlags(c); err != nil {\n\t\treturn err\n\t}\n\n\tif o.Provider == \"\" {\n\t\treturn errors.New(\"--provider is required\")\n\t}\n\n\tif o.Bucket == \"\" {\n\t\treturn errors.New(\"--bucket is required\")\n\t}\n\n\tif o.BackupSyncPeriod < 0 {\n\t\treturn errors.New(\"--backup-sync-period must be non-negative\")\n\t}\n\n\tif len(o.Credential.Data()) > 1 {\n\t\treturn errors.New(\"--credential can only contain 1 key/value pair\")\n\t}\n\n\treturn nil\n}\n\nfunc (o *CreateOptions) Complete(args []string, f client.Factory) error {\n\to.Name = args[0]\n\treturn nil\n}\n\nfunc (o *CreateOptions) BuildBackupStorageLocation(namespace string, setBackupSyncPeriod, setValidationFrequency bool) (*velerov1api.BackupStorageLocation, error) {\n\tvar caCertData []byte\n\tif o.CACertFile != \"\" {\n\t\trealPath, err := filepath.Abs(o.CACertFile)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tcaCertData, err = os.ReadFile(realPath)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tbackupStorageLocation := &velerov1api.BackupStorageLocation{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: namespace,\n\t\t\tName: o.Name,\n\t\t\tLabels: o.Labels.Data(),\n\t\t},\n\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\tProvider: o.Provider,\n\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\tBucket: o.Bucket,\n\t\t\t\t\tPrefix: o.Prefix,\n\t\t\t\t\tCACert: caCertData,\n\t\t\t\t},\n\t\t\t},\n\t\t\tConfig: o.Config.Data(),\n\t\t\tDefault: o.DefaultBackupStorageLocation,\n\t\t\tAccessMode: velerov1api.BackupStorageLocationAccessMode(o.AccessMode.String()),\n\t\t},\n\t}\n\n\tif setBackupSyncPeriod {\n\t\tbackupStorageLocation.Spec.BackupSyncPeriod = &metav1.Duration{Duration: o.BackupSyncPeriod}\n\t}\n\n\tif setValidationFrequency {\n\t\tbackupStorageLocation.Spec.ValidationFrequency = &metav1.Duration{Duration: o.ValidationFrequency}\n\t}\n\n\tfor secretName, secretKey := range o.Credential.Data() {\n\t\tbackupStorageLocation.Spec.Credential = builder.ForSecretKeySelector(secretName, secretKey).Result()\n\t\tbreak\n\t}\n\n\treturn backupStorageLocation, nil\n}\n\nfunc (o *CreateOptions) Run(c *cobra.Command, f client.Factory) error {\n\tsetBackupSyncPeriod := c.Flags().Changed(\"backup-sync-period\")\n\tsetValidationFrequency := c.Flags().Changed(\"validation-frequency\")\n\n\tbackupStorageLocation, err := o.BuildBackupStorageLocation(f.Namespace(), setBackupSyncPeriod, setValidationFrequency)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif printed, err := output.PrintWithFormat(c, backupStorageLocation); printed || err != nil {\n\t\treturn err\n\t}\n\n\tkbClient, err := f.KubebuilderClient()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif o.DefaultBackupStorageLocation {\n\t\t// There is one and only one default backup storage location.\n\t\t// Disable any existing default backup storage location first.\n\t\tdefalutBSLs, err := storage.GetDefaultBackupStorageLocations(context.Background(), kbClient, f.Namespace())\n\t\tif err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\t\tif len(defalutBSLs.Items) > 0 {\n\t\t\treturn errors.New(\"there is already exist default backup storage location, please unset it first or do not set --default flag\")\n\t\t}\n\t}\n\n\tif err := kbClient.Create(context.Background(), backupStorageLocation, &kbclient.CreateOptions{}); err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\tfmt.Printf(\"Backup storage location %q configured successfully.\\n\", backupStorageLocation.Name)\n\treturn nil\n}\n" }, { "path": "pkg/cmd/cli/backuplocation/create_test.go", "content": "/*\nCopyright the Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backuplocation\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\tflag \"github.com/spf13/pflag\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\tveleroflag \"github.com/vmware-tanzu/velero/pkg/cmd/util/flag\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestBuildBackupStorageLocationSetsNamespace(t *testing.T) {\n\to := NewCreateOptions()\n\n\tbsl, err := o.BuildBackupStorageLocation(\"velero-test-ns\", false, false)\n\trequire.NoError(t, err)\n\tassert.Equal(t, \"velero-test-ns\", bsl.Namespace)\n}\n\nfunc TestBuildBackupStorageLocationSetsSyncPeriod(t *testing.T) {\n\to := NewCreateOptions()\n\to.BackupSyncPeriod = 2 * time.Minute\n\n\tbsl, err := o.BuildBackupStorageLocation(\"velero-test-ns\", false, false)\n\trequire.NoError(t, err)\n\tassert.Nil(t, bsl.Spec.BackupSyncPeriod)\n\n\tbsl, err = o.BuildBackupStorageLocation(\"velero-test-ns\", true, false)\n\trequire.NoError(t, err)\n\tassert.Equal(t, &metav1.Duration{Duration: 2 * time.Minute}, bsl.Spec.BackupSyncPeriod)\n}\n\nfunc TestBuildBackupStorageLocationSetsValidationFrequency(t *testing.T) {\n\to := NewCreateOptions()\n\to.ValidationFrequency = 2 * time.Minute\n\n\tbsl, err := o.BuildBackupStorageLocation(\"velero-test-ns\", false, false)\n\trequire.NoError(t, err)\n\tassert.Nil(t, bsl.Spec.ValidationFrequency)\n\n\tbsl, err = o.BuildBackupStorageLocation(\"velero-test-ns\", false, true)\n\trequire.NoError(t, err)\n\tassert.Equal(t, &metav1.Duration{Duration: 2 * time.Minute}, bsl.Spec.ValidationFrequency)\n}\n\nfunc TestBuildBackupStorageLocationSetsCredential(t *testing.T) {\n\to := NewCreateOptions()\n\n\tbsl, err := o.BuildBackupStorageLocation(\"velero-test-ns\", false, false)\n\trequire.NoError(t, err)\n\tassert.Nil(t, bsl.Spec.Credential)\n\n\tsetErr := o.Credential.Set(\"my-secret=key-from-secret\")\n\trequire.NoError(t, setErr)\n\n\tbsl, err = o.BuildBackupStorageLocation(\"velero-test-ns\", false, true)\n\trequire.NoError(t, err)\n\tassert.Equal(t, &corev1api.SecretKeySelector{\n\t\tLocalObjectReference: corev1api.LocalObjectReference{Name: \"my-secret\"},\n\t\tKey: \"key-from-secret\",\n\t}, bsl.Spec.Credential)\n}\n\nfunc TestBuildBackupStorageLocationSetsLabels(t *testing.T) {\n\to := NewCreateOptions()\n\n\terr := o.Labels.Set(\"key=value\")\n\trequire.NoError(t, err)\n\n\tbsl, err := o.BuildBackupStorageLocation(\"velero-test-ns\", false, false)\n\trequire.NoError(t, err)\n\tassert.Equal(t, map[string]string{\"key\": \"value\"}, bsl.Labels)\n}\n\nfunc TestCreateCommand_Run(t *testing.T) {\n\t// create a factory\n\tf := &factorymocks.Factory{}\n\n\t// create command\n\tc := NewCreateCommand(f, \"\")\n\tassert.Equal(t, \"Create a backup storage location\", c.Short)\n\n\t// create a CreateOptions with full options set and then run this backup command\n\tname := \"bsl-name-to-be-created\"\n\tprovider := \"aws\"\n\tbucket := \"velero123456\"\n\tcredential := veleroflag.NewMap()\n\tcredential.Set(\"secret=a\")\n\n\tdefaultBackupStorageLocation := true\n\tprefix := \"builds\"\n\tbackupSyncPeriod := \"1m30s\"\n\tvalidationFrequency := \"128h1m6s\"\n\tbslConfig := veleroflag.NewMap()\n\tbslConfigStr := \"region=minio\"\n\tbslConfig.Set(bslConfigStr)\n\n\tlabels := \"a=too,b=woo\"\n\tcaCertFile := \"bsl-name-1\"\n\taccessMode := \"ReadWrite\"\n\n\tflags := new(flag.FlagSet)\n\to := NewCreateOptions()\n\to.BindFlags(flags)\n\n\tflags.Parse([]string{\"--provider\", provider})\n\tflags.Parse([]string{\"--bucket\", bucket})\n\tflags.Parse([]string{\"--credential\", credential.String()})\n\tflags.Parse([]string{\"--default\"})\n\tflags.Parse([]string{\"--prefix\", prefix})\n\tflags.Parse([]string{\"--backup-sync-period\", backupSyncPeriod})\n\tflags.Parse([]string{\"--validation-frequency\", validationFrequency})\n\tflags.Parse([]string{\"--config\", bslConfigStr})\n\tflags.Parse([]string{\"--labels\", labels})\n\tflags.Parse([]string{\"--cacert\", caCertFile})\n\tflags.Parse([]string{\"--access-mode\", accessMode})\n\n\targs := []string{name, \"arg2\"}\n\n\tkbclient := velerotest.NewFakeControllerRuntimeClient(t)\n\tf.On(\"Namespace\").Return(mock.Anything)\n\tf.On(\"KubebuilderClient\").Return(kbclient, nil)\n\n\to.Complete(args, f)\n\te := o.Validate(c, args, f)\n\trequire.NoError(t, e)\n\n\te = o.Run(c, f)\n\trequire.ErrorContains(t, e, fmt.Sprintf(\"%s: no such file or directory\", caCertFile))\n\n\t// verify all options are set as expected\n\tassert.Equal(t, name, o.Name)\n\tassert.Equal(t, provider, o.Provider)\n\tassert.Equal(t, bucket, o.Bucket)\n\tassert.True(t, reflect.DeepEqual(credential, o.Credential))\n\tassert.Equal(t, defaultBackupStorageLocation, o.DefaultBackupStorageLocation)\n\tassert.Equal(t, prefix, o.Prefix)\n\tassert.Equal(t, backupSyncPeriod, o.BackupSyncPeriod.String())\n\tassert.Equal(t, validationFrequency, o.ValidationFrequency.String())\n\tassert.True(t, reflect.DeepEqual(bslConfig, o.Config))\n\tassert.True(t, velerotest.CompareSlice(strings.Split(labels, \",\"), strings.Split(o.Labels.String(), \",\")))\n\tassert.Equal(t, caCertFile, o.CACertFile)\n\tassert.Equal(t, accessMode, o.AccessMode.String())\n\n\t// create the other create command without fromSchedule option for Run() other branches\n\tc = NewCreateCommand(f, \"velero backup-location create\")\n\tassert.Equal(t, \"Create a backup storage location\", c.Short)\n\n\to = NewCreateOptions()\n\to.Labels.Set(\"velero.io/test=true\")\n\n\targs = []string{\"backup-name-2\", \"arg2\"}\n\to.Complete(args, f)\n\n\te = o.Run(c, f)\n\trequire.NoError(t, e)\n\tc.SetArgs([]string{\"bsl-1\", \"--provider=aws\", \"--bucket=bk1\", \"--default\"})\n\te = c.Execute()\n\tassert.NoError(t, e)\n}\n" }, { "path": "pkg/cmd/cli/backuplocation/delete.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backuplocation\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/spf13/cobra\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\tkubeerrs \"k8s.io/apimachinery/pkg/util/errors\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/confirm\"\n)\n\n// NewDeleteCommand creates and returns a new cobra command for deleting backup-locations.\nfunc NewDeleteCommand(f client.Factory, use string) *cobra.Command {\n\to := cli.NewDeleteOptions(\"backup-location\")\n\n\tc := &cobra.Command{\n\t\tUse: fmt.Sprintf(\"%s [NAMES]\", use),\n\t\tShort: \"Delete backup storage locations\",\n\t\tExample: ` # Delete a backup storage location named \"backup-location-1\".\n velero backup-location delete backup-location-1\n\n # Delete a backup storage location named \"backup-location-1\" without prompting for confirmation.\n velero backup-location delete backup-location-1 --confirm\n\n # Delete backup storage locations named \"backup-location-1\" and \"backup-location-2\".\n velero backup-location delete backup-location-1 backup-location-2\n\t\t\n # Delete all backup storage locations labeled with \"foo=bar\".\n velero backup-location delete --selector foo=bar\n\n # Delete all backup storage locations.\n velero backup-location delete --all`,\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tcmd.CheckError(o.Complete(f, args))\n\t\t\tcmd.CheckError(o.Validate(c, f, args))\n\t\t\tcmd.CheckError(Run(f, o))\n\t\t},\n\t}\n\n\to.BindFlags(c.Flags())\n\treturn c\n}\n\n// Run performs the delete backup-location operation.\nfunc Run(f client.Factory, o *cli.DeleteOptions) error {\n\tif !o.Confirm && !confirm.GetConfirmation() {\n\t\t// Don't do anything unless we get confirmation\n\t\treturn nil\n\t}\n\n\tkbClient, err := f.KubebuilderClient()\n\tcmd.CheckError(err)\n\n\tlocations := new(velerov1api.BackupStorageLocationList)\n\tvar errs []error\n\tswitch {\n\tcase len(o.Names) > 0:\n\t\tfor _, name := range o.Names {\n\t\t\tlocation := &velerov1api.BackupStorageLocation{}\n\t\t\terr = kbClient.Get(context.Background(), kbclient.ObjectKey{\n\t\t\t\tNamespace: f.Namespace(),\n\t\t\t\tName: name,\n\t\t\t}, location)\n\t\t\tif err != nil {\n\t\t\t\terrs = append(errs, errors.WithStack(err))\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tlocations.Items = append(locations.Items, *location)\n\t\t}\n\tdefault:\n\t\tselector := labels.Everything().String()\n\t\tif o.Selector.LabelSelector != nil {\n\t\t\tselector = o.Selector.String()\n\t\t}\n\n\t\terr := kbClient.List(context.Background(), locations, &kbclient.ListOptions{\n\t\t\tNamespace: f.Namespace(),\n\t\t\tRaw: &metav1.ListOptions{LabelSelector: selector},\n\t\t})\n\t\tif err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\t}\n\n\tif len(locations.Items) == 0 {\n\t\tfmt.Println(\"No backup-locations found\")\n\t\treturn nil\n\t}\n\n\t// Create a backup-location deletion request for each\n\tfor i, location := range locations.Items {\n\t\tif err := kbClient.Delete(context.Background(), &locations.Items[i], &kbclient.DeleteOptions{}); err != nil {\n\t\t\terrs = append(errs, errors.WithStack(err))\n\t\t\tcontinue\n\t\t}\n\t\tfmt.Printf(\"Backup storage location %q deleted successfully.\\n\", location.Name)\n\n\t\t// Delete backups associated with the deleted BSL.\n\t\tbackupList, err := findAssociatedBackups(kbClient, location.Name, f.Namespace())\n\t\tif err != nil {\n\t\t\terrs = append(errs, fmt.Errorf(\"find backups associated with BSL %q: %w\", location.Name, err))\n\t\t} else if deleteErrs := deleteBackups(kbClient, backupList); deleteErrs != nil {\n\t\t\terrs = append(errs, deleteErrs...)\n\t\t}\n\n\t\t// Delete backup repositories associated with the deleted BSL.\n\t\tbackupRepoList, err := findAssociatedBackupRepos(kbClient, location.Name, f.Namespace())\n\t\tif err != nil {\n\t\t\terrs = append(errs, fmt.Errorf(\"find backup repositories associated with BSL %q: %w\", location.Name, err))\n\t\t} else if deleteErrs := deleteBackupRepos(kbClient, backupRepoList); deleteErrs != nil {\n\t\t\terrs = append(errs, deleteErrs...)\n\t\t}\n\t}\n\n\treturn kubeerrs.NewAggregate(errs)\n}\n\nfunc findAssociatedBackups(client kbclient.Client, bslName, ns string) (velerov1api.BackupList, error) {\n\tvar backups velerov1api.BackupList\n\terr := client.List(context.Background(), &backups, &kbclient.ListOptions{\n\t\tNamespace: ns,\n\t\tRaw: &metav1.ListOptions{LabelSelector: velerov1api.StorageLocationLabel + \"=\" + bslName},\n\t})\n\treturn backups, err\n}\n\nfunc findAssociatedBackupRepos(client kbclient.Client, bslName, ns string) (velerov1api.BackupRepositoryList, error) {\n\tvar repos velerov1api.BackupRepositoryList\n\terr := client.List(context.Background(), &repos, &kbclient.ListOptions{\n\t\tNamespace: ns,\n\t\tRaw: &metav1.ListOptions{LabelSelector: velerov1api.StorageLocationLabel + \"=\" + bslName},\n\t})\n\treturn repos, err\n}\n\nfunc deleteBackups(client kbclient.Client, backups velerov1api.BackupList) []error {\n\tvar errs []error\n\tfor i, backup := range backups.Items {\n\t\tif err := client.Delete(context.Background(), &backups.Items[i], &kbclient.DeleteOptions{}); err != nil {\n\t\t\terrs = append(errs, errors.WithStack(fmt.Errorf(\"delete backup %q associated with deleted BSL: %w\", backup.Name, err)))\n\t\t\tcontinue\n\t\t}\n\t\tfmt.Printf(\"Backup associated with deleted BSL(s) %q deleted successfully.\\n\", backup.Name)\n\t}\n\treturn errs\n}\n\nfunc deleteBackupRepos(client kbclient.Client, repos velerov1api.BackupRepositoryList) []error {\n\tvar errs []error\n\tfor i, repo := range repos.Items {\n\t\tif err := client.Delete(context.Background(), &repos.Items[i], &kbclient.DeleteOptions{}); err != nil {\n\t\t\terrs = append(errs, errors.WithStack(fmt.Errorf(\"delete backup repository %q associated with deleted BSL: %w\", repo.Name, err)))\n\t\t\tcontinue\n\t\t}\n\t\tfmt.Printf(\"Backup repository associated with deleted BSL(s) %q deleted successfully.\\n\", repo.Name)\n\t}\n\treturn errs\n}\n" }, { "path": "pkg/cmd/cli/backuplocation/delete_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backuplocation\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"testing\"\n\n\tflag \"github.com/spf13/pflag\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli\"\n\tcmdtest \"github.com/vmware-tanzu/velero/pkg/cmd/test\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\tveleroexec \"github.com/vmware-tanzu/velero/pkg/util/exec\"\n)\n\nfunc TestNewDeleteCommand(t *testing.T) {\n\t// create a factory\n\tf := &factorymocks.Factory{}\n\tkbclient := velerotest.NewFakeControllerRuntimeClient(t)\n\tf.On(\"Namespace\").Return(mock.Anything)\n\tf.On(\"KubebuilderClient\").Return(kbclient, nil)\n\n\t// create command\n\tc := NewDeleteCommand(f, \"velero backup-location delete\")\n\tassert.Equal(t, \"Delete backup storage locations\", c.Short)\n\n\to := cli.NewDeleteOptions(\"backup\")\n\tflags := new(flag.FlagSet)\n\to.BindFlags(flags)\n\tflags.Parse([]string{\"--confirm\"})\n\n\targs := []string{\"bk-loc-1\", \"bk-loc-2\"}\n\te := o.Complete(f, args)\n\trequire.NoError(t, e)\n\te = o.Validate(c, f, args)\n\trequire.NoError(t, e)\n\tc.SetArgs([]string{\"bk-1\", \"--confirm\"})\n\te = c.Execute()\n\trequire.NoError(t, e)\n\n\te = Run(f, o)\n\trequire.NoError(t, e)\n\tif os.Getenv(cmdtest.CaptureFlag) == \"1\" {\n\t\treturn\n\t}\n\n\tcmd := exec.CommandContext(t.Context(), os.Args[0], []string{\"-test.run=TestNewDeleteCommand\"}...)\n\tcmd.Env = append(os.Environ(), fmt.Sprintf(\"%s=1\", cmdtest.CaptureFlag))\n\tstdout, _, err := veleroexec.RunCommand(cmd)\n\n\tif err == nil {\n\t\tassert.Contains(t, stdout, \"No backup-locations found\")\n\t\treturn\n\t}\n\tt.Fatalf(\"process ran with err %v, want backups by get()\", err)\n}\nfunc TestDeleteFunctions(t *testing.T) {\n\t//t.Run(\"create the other create command with fromSchedule option for Run() other branches\", func(t *testing.T) {\n\t// create a factory\n\tf := &factorymocks.Factory{}\n\tkbclient := velerotest.NewFakeControllerRuntimeClient(t)\n\tf.On(\"Namespace\").Return(mock.Anything)\n\tf.On(\"KubebuilderClient\").Return(kbclient, nil)\n\n\tbkList := velerov1api.BackupList{}\n\tbkrepoList := velerov1api.BackupRepositoryList{}\n\tt.Run(\"findAssociatedBackups\", func(t *testing.T) {\n\t\tbkList, e := findAssociatedBackups(kbclient, \"bk-loc-1\", \"ns1\")\n\t\tassert.Empty(t, bkList.Items)\n\t\tassert.NoError(t, e)\n\t})\n\n\tt.Run(\"findAssociatedBackupRepos\", func(t *testing.T) {\n\t\tbkrepoList, e := findAssociatedBackupRepos(kbclient, \"bk-loc-1\", \"ns1\")\n\t\tassert.Empty(t, bkrepoList.Items)\n\t\tassert.NoError(t, e)\n\t})\n\n\tt.Run(\"deleteBackups\", func(t *testing.T) {\n\t\tbk := velerov1api.Backup{}\n\t\tbk.Name = \"bk-name-last\"\n\t\tbkList.Items = append(bkList.Items, bk)\n\t\terrList := deleteBackups(kbclient, bkList)\n\t\tassert.Len(t, errList, 1)\n\t\tassert.ErrorContains(t, errList[0], fmt.Sprintf(\"delete backup \\\"%s\\\" associated with deleted BSL: backups.velero.io \\\"%s\\\" not found\", bk.Name, bk.Name))\n\t})\n\tt.Run(\"deleteBackupRepos\", func(t *testing.T) {\n\t\tbkrepo := velerov1api.BackupRepository{}\n\t\tbkrepo.Name = \"bk-repo-name-last\"\n\t\tbkrepoList.Items = append(bkrepoList.Items, bkrepo)\n\t\terrList := deleteBackupRepos(kbclient, bkrepoList)\n\t\tassert.Len(t, errList, 1)\n\t\tassert.ErrorContains(t, errList[0], fmt.Sprintf(\"delete backup repository \\\"%s\\\" associated with deleted BSL: backuprepositories.velero.io \\\"%s\\\" not found\", bkrepo.Name, bkrepo.Name))\n\t})\n}\n" }, { "path": "pkg/cmd/cli/backuplocation/get.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backuplocation\n\nimport (\n\t\"context\"\n\n\t\"github.com/spf13/cobra\"\n\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/output\"\n)\n\nfunc NewGetCommand(f client.Factory, use string) *cobra.Command {\n\tvar listOptions metav1.ListOptions\n\tvar showDefaultOnly bool\n\n\tc := &cobra.Command{\n\t\tUse: use,\n\t\tShort: \"Get backup storage locations\",\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\terr := output.ValidateFlags(c)\n\t\t\tcmd.CheckError(err)\n\n\t\t\tkbClient, err := f.KubebuilderClient()\n\t\t\tcmd.CheckError(err)\n\n\t\t\tlocations := new(velerov1api.BackupStorageLocationList)\n\t\t\tif len(args) > 0 {\n\t\t\t\tfor _, name := range args {\n\t\t\t\t\tlocation := &velerov1api.BackupStorageLocation{}\n\t\t\t\t\terr = kbClient.Get(context.Background(), kbclient.ObjectKey{\n\t\t\t\t\t\tNamespace: f.Namespace(),\n\t\t\t\t\t\tName: name,\n\t\t\t\t\t}, location)\n\t\t\t\t\tcmd.CheckError(err)\n\n\t\t\t\t\tif showDefaultOnly {\n\t\t\t\t\t\tif location.Spec.Default {\n\t\t\t\t\t\t\tlocations.Items = append(locations.Items, *location)\n\t\t\t\t\t\t\tbreak\n\t\t\t\t\t\t}\n\t\t\t\t\t} else {\n\t\t\t\t\t\tlocations.Items = append(locations.Items, *location)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\terr := kbClient.List(context.Background(), locations, &kbclient.ListOptions{\n\t\t\t\t\tNamespace: f.Namespace(),\n\t\t\t\t\tRaw: &listOptions,\n\t\t\t\t})\n\t\t\t\tcmd.CheckError(err)\n\n\t\t\t\tif showDefaultOnly {\n\t\t\t\t\tfor i := 0; i < len(locations.Items); i++ {\n\t\t\t\t\t\tif locations.Items[i].Spec.Default {\n\t\t\t\t\t\t\tcontinue\n\t\t\t\t\t\t}\n\t\t\t\t\t\tif i != len(locations.Items)-1 {\n\t\t\t\t\t\t\tcopy(locations.Items[i:], locations.Items[i+1:])\n\t\t\t\t\t\t\ti = i - 1\n\t\t\t\t\t\t}\n\t\t\t\t\t\tlocations.Items = locations.Items[:len(locations.Items)-1]\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\n\t\t\t_, err = output.PrintWithFormat(c, locations)\n\t\t\tcmd.CheckError(err)\n\t\t},\n\t}\n\n\tc.Flags().BoolVar(&showDefaultOnly, \"default\", false, \"Displays the current default backup storage location.\")\n\tc.Flags().StringVarP(&listOptions.LabelSelector, \"selector\", \"l\", listOptions.LabelSelector, \"Only show items matching this label selector.\")\n\n\toutput.BindFlags(c.Flags())\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/backuplocation/get_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backuplocation\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\tcmdtest \"github.com/vmware-tanzu/velero/pkg/cmd/test\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\tveleroexec \"github.com/vmware-tanzu/velero/pkg/util/exec\"\n)\n\nfunc TestNewGetCommand(t *testing.T) {\n\tbkList := []string{\"b1\", \"b2\"}\n\n\tf := &factorymocks.Factory{}\n\tkbclient := velerotest.NewFakeControllerRuntimeClient(t)\n\tf.On(\"Namespace\").Return(mock.Anything)\n\tf.On(\"KubebuilderClient\").Return(kbclient, nil)\n\n\t// get command\n\tc := NewGetCommand(f, \"velero backup-location get\")\n\tassert.Equal(t, \"Get backup storage locations\", c.Short)\n\n\tc.Execute()\n\n\tif os.Getenv(cmdtest.CaptureFlag) == \"1\" {\n\t\tc.SetArgs([]string{\"b1\", \"b2\", \"--default\"})\n\t\tc.Execute()\n\t\treturn\n\t}\n\tcmd := exec.CommandContext(t.Context(), os.Args[0], []string{\"-test.run=TestNewGetCommand\"}...)\n\tcmd.Env = append(os.Environ(), fmt.Sprintf(\"%s=1\", cmdtest.CaptureFlag))\n\t_, stderr, err := veleroexec.RunCommand(cmd)\n\n\tif err != nil {\n\t\tassert.Contains(t, stderr, fmt.Sprintf(\"backupstoragelocations.velero.io \\\"%s\\\" not found\", bkList[0]))\n\t\treturn\n\t}\n\tt.Fatalf(\"process ran with err %v, want backup delete successfully\", err)\n}\n" }, { "path": "pkg/cmd/cli/backuplocation/set.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backuplocation\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/spf13/cobra\"\n\t\"github.com/spf13/pflag\"\n\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/storage\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/flag\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n)\n\nfunc NewSetCommand(f client.Factory, use string) *cobra.Command {\n\to := NewSetOptions()\n\n\tc := &cobra.Command{\n\t\tUse: use + \" NAME\",\n\t\tShort: \"Set specific features for a backup storage location\",\n\t\tArgs: cobra.ExactArgs(1),\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tcmd.CheckError(o.Complete(args, f))\n\t\t\tcmd.CheckError(o.Validate(c, args, f))\n\t\t\tcmd.CheckError(o.Run(c, f))\n\t\t},\n\t}\n\n\to.BindFlags(c.Flags())\n\n\treturn c\n}\n\ntype SetOptions struct {\n\tName string\n\tCACertFile string\n\tCredential flag.Map\n\tDefaultBackupStorageLocation flag.OptionalBool\n}\n\nfunc NewSetOptions() *SetOptions {\n\treturn &SetOptions{\n\t\tCredential: flag.NewMap(),\n\t}\n}\n\nfunc (o *SetOptions) BindFlags(flags *pflag.FlagSet) {\n\tflags.StringVar(&o.CACertFile, \"cacert\", o.CACertFile, \"File containing a certificate bundle to use when verifying TLS connections to the object store. Optional.\")\n\tflags.Var(&o.Credential, \"credential\", \"Sets the credential to be used by this location as a key-value pair, where the key is the Kubernetes Secret name, and the value is the data key name within the Secret. Optional, one value only.\")\n\tf := flags.VarPF(&o.DefaultBackupStorageLocation, \"default\", \"\", \"Sets this new location to be the new default backup storage location. Optional.\")\n\tf.NoOptDefVal = cmd.TRUE\n}\n\nfunc (o *SetOptions) Validate(c *cobra.Command, args []string, f client.Factory) error {\n\tif len(o.Credential.Data()) > 1 {\n\t\treturn errors.New(\"--credential can only contain 1 key/value pair\")\n\t}\n\n\treturn nil\n}\n\nfunc (o *SetOptions) Complete(args []string, f client.Factory) error {\n\to.Name = args[0]\n\treturn nil\n}\n\nfunc (o *SetOptions) Run(c *cobra.Command, f client.Factory) error {\n\tkbClient, err := f.KubebuilderClient()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tvar caCertData []byte\n\tif o.CACertFile != \"\" {\n\t\trealPath, err := filepath.Abs(o.CACertFile)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tcaCertData, err = os.ReadFile(realPath)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tlocation := &velerov1api.BackupStorageLocation{}\n\terr = kbClient.Get(context.Background(), kbclient.ObjectKey{\n\t\tNamespace: f.Namespace(),\n\t\tName: o.Name,\n\t}, location)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\tdefaultOpt := o.DefaultBackupStorageLocation.Value\n\tif defaultOpt != nil {\n\t\tif *defaultOpt { // set default backup storage location\n\t\t\t// need first check if there is already a default backup storage location\n\t\t\tdefalutBSLs, err := storage.GetDefaultBackupStorageLocations(context.Background(), kbClient, f.Namespace())\n\t\t\tif err != nil {\n\t\t\t\treturn errors.WithStack(err)\n\t\t\t}\n\t\t\tif len(defalutBSLs.Items) > 0 {\n\t\t\t\tif len(defalutBSLs.Items) == 1 && defalutBSLs.Items[0].Name == o.Name {\n\t\t\t\t\t// the default backup storage location is the one we want to set\n\t\t\t\t\t// so we do not need to do anything\n\t\t\t\t\tfmt.Printf(\"Backup storage location %q is already the default backup storage location.\\n\", o.Name)\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\t\t\t\treturn errors.New(\"there are already exist default backup storage locations, please unset them first\")\n\t\t\t}\n\t\t}\n\t} else {\n\t\t// user do not specify default option\n\t\t// we should keep the original default option\n\t\to.DefaultBackupStorageLocation = flag.OptionalBool{Value: &location.Spec.Default}\n\t}\n\n\tlocation.Spec.Default = boolptr.IsSetToTrue(o.DefaultBackupStorageLocation.Value)\n\tlocation.Spec.StorageType.ObjectStorage.CACert = caCertData\n\n\tfor name, key := range o.Credential.Data() {\n\t\tlocation.Spec.Credential = builder.ForSecretKeySelector(name, key).Result()\n\t\tbreak\n\t}\n\n\tif err := kbClient.Update(context.Background(), location, &kbclient.UpdateOptions{}); err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\tfmt.Printf(\"Backup storage location %q configured successfully.\\n\", o.Name)\n\treturn nil\n}\n" }, { "path": "pkg/cmd/cli/backuplocation/set_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backuplocation\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"reflect\"\n\t\"testing\"\n\n\tflag \"github.com/spf13/pflag\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\tcmdtest \"github.com/vmware-tanzu/velero/pkg/cmd/test\"\n\tveleroflag \"github.com/vmware-tanzu/velero/pkg/cmd/util/flag\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\tveleroexec \"github.com/vmware-tanzu/velero/pkg/util/exec\"\n)\n\nfunc TestNewSetCommand(t *testing.T) {\n\tbackupName := \"arg2\"\n\t// create a config for factory\n\tf := &factorymocks.Factory{}\n\n\tkbclient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\tf.On(\"Namespace\").Return(mock.Anything)\n\tf.On(\"KubebuilderClient\").Return(kbclient, nil)\n\n\t// create command\n\tc := NewSetCommand(f, \"\")\n\tassert.Equal(t, \"Set specific features for a backup storage location\", c.Short)\n\n\t// create a SetOptions with full options set and then run this backup command\n\tcacert := \"a/b/c/ut-cert.ca\"\n\tdefaultBackupStorageLocation := true\n\tcredential := veleroflag.NewMap()\n\tcredential.Set(\"secret=a\")\n\n\tflags := new(flag.FlagSet)\n\to := NewSetOptions()\n\to.BindFlags(flags)\n\n\tflags.Parse([]string{\"--cacert\", cacert})\n\tflags.Parse([]string{\"--credential\", credential.String()})\n\tflags.Parse([]string{\"--default\"})\n\n\targs := []string{backupName}\n\to.Complete(args, f)\n\te := o.Validate(c, args, f)\n\trequire.NoError(t, e)\n\n\te = o.Run(c, f)\n\trequire.ErrorContains(t, e, fmt.Sprintf(\"%s: no such file or directory\", cacert))\n\n\t// verify all options are set as expected\n\tassert.Equal(t, backupName, o.Name)\n\tassert.Equal(t, cacert, o.CACertFile)\n\tassert.Equal(t, defaultBackupStorageLocation, boolptr.IsSetToTrue(o.DefaultBackupStorageLocation.Value))\n\tassert.True(t, reflect.DeepEqual(credential, o.Credential))\n\n\tassert.ErrorContains(t, e, fmt.Sprintf(\"%s: no such file or directory\", cacert))\n}\n\nfunc TestSetCommand_Execute(t *testing.T) {\n\tbsl := \"bsl-1\"\n\tif os.Getenv(cmdtest.CaptureFlag) == \"1\" {\n\t\t// create a config for factory\n\t\tf := &factorymocks.Factory{}\n\n\t\tkbclient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\t\tf.On(\"Namespace\").Return(mock.Anything)\n\t\tf.On(\"KubebuilderClient\").Return(kbclient, nil)\n\n\t\t// create command\n\t\tc := NewSetCommand(f, \"velero backup-location set\")\n\t\tc.SetArgs([]string{bsl})\n\t\tc.Execute()\n\t\treturn\n\t}\n\n\tcmd := exec.CommandContext(t.Context(), os.Args[0], []string{\"-test.run=TestSetCommand_Execute\"}...)\n\tcmd.Env = append(os.Environ(), fmt.Sprintf(\"%s=1\", cmdtest.CaptureFlag))\n\t_, stderr, err := veleroexec.RunCommand(cmd)\n\n\tif err != nil {\n\t\tassert.Contains(t, stderr, \"backupstoragelocations.velero.io \\\"bsl-1\\\" not found\")\n\t\treturn\n\t}\n\tt.Fatalf(\"process ran with err %v, want backup delete successfully\", err)\n}\n" }, { "path": "pkg/cmd/cli/bug/bug.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage bug\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"net/url\"\n\t\"os\"\n\t\"os/exec\"\n\t\"runtime\"\n\t\"strings\"\n\t\"text/template\"\n\t\"time\"\n\n\t\"github.com/spf13/cobra\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/buildinfo\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/features\"\n)\n\nconst (\n\t// kubectlTimeout is how long we wait in seconds for `kubectl version`\n\t// before killing the process\n\tkubectlTimeout = 5 * time.Second\n\tissueURL = \"https://github.com/vmware-tanzu/velero/issues/new\"\n\t// IssueTemplate is used to generate .github/ISSUE_TEMPLATE/bug_report.md\n\t// as well as the initial text that's place in a new Github issue as\n\t// the result of running `velero bug`.\n\tIssueTemplate = `---\nname: Bug report\nabout: Tell us about a problem you are experiencing\n\n---\n\n**What steps did you take and what happened:**\n\n\n\n**What did you expect to happen:**\n\n**The following information will help us better understand what's going on**:\n\n_If you are using velero v1.7.0+:_ \nPlease use ` + \"`velero debug --backup --restore ` \" +\n\t\t`to generate the support bundle, and attach to this issue, more options please refer to ` +\n\t\t\"`velero debug --help` \" + `\n\n_If you are using earlier versions:_ \nPlease provide the output of the following commands (Pasting long output into a [GitHub gist](https://gist.github.com) or other pastebin is fine.)\n- ` + \"`kubectl logs deployment/velero -n velero`\" + `\n- ` + \"`velero backup describe ` or `kubectl get backup/ -n velero -o yaml`\" + `\n- ` + \"`velero backup logs `\" + `\n- ` + \"`velero restore describe ` or `kubectl get restore/ -n velero -o yaml`\" + `\n- ` + \"`velero restore logs `\" + `\n\n\n**Anything else you would like to add:**\n\n\n\n**Environment:**\n\n- Velero version (use ` + \"`velero version`\" + `):{{.VeleroVersion}} {{.GitCommit}}\n- Velero features (use ` + \"`velero client config get features`\" + `): {{.Features}}\n- Kubernetes version (use ` + \"`kubectl version`\" + `): \n{{- if .KubectlVersion}}\n` + \"```\" + `\n{{.KubectlVersion}}\n` + \"```\" + `\n{{end}}\n- Kubernetes installer & version:\n- Cloud provider or hardware configuration:\n- OS (e.g. from ` + \"`/etc/os-release`\" + `):\n{{- if .RuntimeOS}} - RuntimeOS: {{.RuntimeOS}}{{end}}\n{{- if .RuntimeArch}} - RuntimeArch: {{.RuntimeArch}}{{end}}\n\n\n**Vote on this issue!**\n\nThis is an invitation to the Velero community to vote on issues, you can see the project's [top voted issues listed here](https://github.com/vmware-tanzu/velero/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc). \nUse the \"reaction smiley face\" up to the right of this comment to vote.\n\n- :+1: for \"I would like to see this bug fixed as soon as possible\"\n- :-1: for \"There are more important bugs to focus on right now\"\n`\n)\n\nfunc NewCommand() *cobra.Command {\n\tc := &cobra.Command{\n\t\tUse: \"bug\",\n\t\tShort: \"Report a Velero bug\",\n\t\tLong: \"Open a browser window to report a Velero bug\",\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tkubectlVersion, err := getKubectlVersion()\n\t\t\tif err != nil {\n\t\t\t\t// we don't want to prevent the user from submitting a bug\n\t\t\t\t// if we can't get the kubectl version, so just display a warning\n\t\t\t\tfmt.Fprintf(os.Stderr, \"WARNING: can't get kubectl version: %v\\n\", err)\n\t\t\t}\n\t\t\tbody, err := renderToString(newBugInfo(kubectlVersion))\n\t\t\tcmd.CheckError(err)\n\t\t\tcmd.CheckError(showIssueInBrowser(body))\n\t\t},\n\t}\n\treturn c\n}\n\ntype VeleroBugInfo struct {\n\tVeleroVersion string\n\tGitCommit string\n\tRuntimeOS string\n\tRuntimeArch string\n\tKubectlVersion string\n\tFeatures string\n}\n\n// cmdExistsOnPath checks to see if an executable is available on the current PATH\nfunc cmdExistsOnPath(name string) bool {\n\tif _, err := exec.LookPath(name); err != nil {\n\t\treturn false\n\t}\n\treturn true\n}\n\n// getKubectlVersion makes a best-effort to run `kubectl version`\n// and return it's output. This func will timeout and return an empty\n// string after kubectlTimeout if we're not connected to a cluster.\nfunc getKubectlVersion() (string, error) {\n\tif !cmdExistsOnPath(\"kubectl\") {\n\t\treturn \"\", errors.New(\"kubectl not found on PATH\")\n\t}\n\n\tkubectlCmd := exec.CommandContext(context.Background(), \"kubectl\", \"version\")\n\tvar outbuf bytes.Buffer\n\tkubectlCmd.Stdout = &outbuf\n\tif err := kubectlCmd.Start(); err != nil {\n\t\treturn \"\", errors.New(\"can't start kubectl\")\n\t}\n\n\tdone := make(chan error, 1)\n\tgo func() {\n\t\tdone <- kubectlCmd.Wait()\n\t}()\n\tselect {\n\tcase <-time.After(kubectlTimeout):\n\t\t// we don't care about the possible error returned from Kill() here,\n\t\t// just return an empty string\n\t\tif err := kubectlCmd.Process.Kill(); err != nil {\n\t\t\treturn \"\", fmt.Errorf(\"error killing kubectl process: %w\", err)\n\t\t}\n\t\treturn \"\", errors.New(\"timeout waiting for kubectl version\")\n\n\tcase err := <-done:\n\t\tif err != nil {\n\t\t\treturn \"\", errors.New(\"error waiting for kubectl process\")\n\t\t}\n\t}\n\tversionOut := outbuf.String()\n\tkubectlVersion := strings.TrimSpace(versionOut)\n\treturn kubectlVersion, nil\n}\n\nfunc newBugInfo(kubectlVersion string) *VeleroBugInfo {\n\treturn &VeleroBugInfo{\n\t\tVeleroVersion: buildinfo.Version,\n\t\tGitCommit: buildinfo.FormattedGitSHA(),\n\t\tRuntimeOS: runtime.GOOS,\n\t\tRuntimeArch: runtime.GOARCH,\n\t\tKubectlVersion: kubectlVersion,\n\t\tFeatures: features.Serialize(),\n\t}\n}\n\n// renderToString renders IssueTemplate to a string using the\n// supplied *VeleroBugInfo\nfunc renderToString(bugInfo *VeleroBugInfo) (string, error) {\n\toutputTemplate, err := template.New(\"ghissue\").Parse(IssueTemplate)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\tvar buf bytes.Buffer\n\terr = outputTemplate.Execute(&buf, bugInfo)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\treturn buf.String(), nil\n}\n\n// showIssueInBrowser opens a browser window to submit a Github issue using\n// a platform specific binary.\nfunc showIssueInBrowser(body string) error {\n\turl := issueURL + \"?body=\" + url.QueryEscape(body)\n\tctx := context.Background()\n\tswitch runtime.GOOS {\n\tcase \"darwin\":\n\t\treturn exec.CommandContext(ctx, \"open\", url).Start()\n\tcase \"linux\":\n\t\tif cmdExistsOnPath(\"xdg-open\") {\n\t\t\treturn exec.CommandContext(ctx, \"xdg-open\", url).Start()\n\t\t}\n\t\treturn fmt.Errorf(\"velero can't open a browser window using the command '%s'\", \"xdg-open\")\n\tcase \"windows\":\n\t\treturn exec.CommandContext(ctx, \"rundll32\", \"url.dll,FileProtocolHandler\", url).Start()\n\tdefault:\n\t\treturn fmt.Errorf(\"velero can't open a browser window on platform %s\", runtime.GOOS)\n\t}\n}\n" }, { "path": "pkg/cmd/cli/client/client.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage client\n\nimport (\n\t\"github.com/spf13/cobra\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/client/config\"\n)\n\nfunc NewCommand() *cobra.Command {\n\tc := &cobra.Command{\n\t\tUse: \"client\",\n\t\tShort: \"Velero client related commands\",\n\t}\n\n\tc.AddCommand(\n\t\tconfig.NewCommand(),\n\t)\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/client/config/config.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage config\n\nimport (\n\t\"github.com/spf13/cobra\"\n)\n\nfunc NewCommand() *cobra.Command {\n\tc := &cobra.Command{\n\t\tUse: \"config\",\n\t\tShort: \"Get and set client configuration file values\",\n\t}\n\n\tc.AddCommand(\n\t\tNewGetCommand(),\n\t\tNewSetCommand(),\n\t)\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/client/config/get.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage config\n\nimport (\n\t\"fmt\"\n\t\"sort\"\n\n\t\"github.com/spf13/cobra\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n)\n\nfunc NewGetCommand() *cobra.Command {\n\tc := &cobra.Command{\n\t\tUse: \"get [KEY 1] [KEY 2] [...]\",\n\t\tShort: \"Get client configuration file values\",\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tconfig, err := client.LoadConfig()\n\t\t\tcmd.CheckError(err)\n\n\t\t\tif len(args) == 0 {\n\t\t\t\tkeys := make([]string, 0, len(config))\n\t\t\t\tfor key := range config {\n\t\t\t\t\tkeys = append(keys, key)\n\t\t\t\t}\n\n\t\t\t\tsort.Strings(keys)\n\n\t\t\t\tfor _, key := range keys {\n\t\t\t\t\tfmt.Printf(\"%s: %s\\n\", key, config[key])\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tfor _, key := range args {\n\t\t\t\t\tvalue, found := config[key]\n\t\t\t\t\tif !found {\n\t\t\t\t\t\tvalue = \"\"\n\t\t\t\t\t}\n\t\t\t\t\tfmt.Printf(\"%s: %s\\n\", key, value)\n\t\t\t\t}\n\t\t\t}\n\t\t},\n\t}\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/client/config/set.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage config\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"strings\"\n\n\t\"github.com/spf13/cobra\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n)\n\nfunc NewSetCommand() *cobra.Command {\n\tc := &cobra.Command{\n\t\tUse: \"set KEY=VALUE [KEY=VALUE]...\",\n\t\tShort: \"Set client configuration file values\",\n\t\tArgs: cobra.MinimumNArgs(1),\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tconfig, err := client.LoadConfig()\n\t\t\tcmd.CheckError(err)\n\n\t\t\tfor _, arg := range args {\n\t\t\t\tpair := strings.Split(arg, \"=\")\n\t\t\t\tif len(pair) != 2 {\n\t\t\t\t\tfmt.Fprintf(os.Stderr, \"WARNING: invalid KEY=VALUE: %q\\n\", arg)\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tkey, value := pair[0], pair[1]\n\n\t\t\t\tif value == \"\" {\n\t\t\t\t\tdelete(config, key)\n\t\t\t\t} else {\n\t\t\t\t\tconfig[key] = value\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tcmd.CheckError(client.SaveConfig(config))\n\t\t},\n\t}\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/completion/completion.go", "content": "/*\nCopyright 2021 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage completion\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\n\t\"github.com/spf13/cobra\"\n)\n\nfunc NewCommand() *cobra.Command {\n\tc := &cobra.Command{\n\t\tUse: \"completion [bash|zsh|fish]\",\n\t\tShort: \"Generate completion script\",\n\t\tLong: `To load completions:\n\nBash:\n\n$ source <(velero completion bash)\n\n# To load completions for each session, execute once:\nLinux:\n\t$ velero completion bash > /etc/bash_completion.d/velero\nMacOS:\n\t$ velero completion bash > /usr/local/etc/bash_completion.d/velero\n\nZsh:\n\n# If shell completion is not already enabled in your environment you will need\n# to enable it. You can execute the following once:\n\n$ echo \"autoload -U compinit; compinit\" >> ~/.zshrc\n\n# To load completions for each session, execute once:\n$ velero completion zsh > \"${fpath[1]}/_velero\"\n\n# You will need to start a new shell for this setup to take effect.\n\nFish:\n\n$ velero completion fish | source\n\n# To load completions for each session, execute once:\n$ velero completion fish > ~/.config/fish/completions/velero.fish\n`,\n\t\tArgs: cobra.ExactArgs(1),\n\t\tValidArgs: []string{\"bash\", \"zsh\", \"fish\"},\n\t\tRun: func(cmd *cobra.Command, args []string) {\n\t\t\tshell := args[0]\n\t\t\tswitch shell {\n\t\t\tcase \"bash\":\n\t\t\t\tif err := cmd.Root().GenBashCompletion(os.Stdout); err != nil {\n\t\t\t\t\tfmt.Println(\"fail to generate bash completion script\", err)\n\t\t\t\t\tos.Exit(1)\n\t\t\t\t}\n\t\t\tcase \"zsh\":\n\t\t\t\t// # fix #4912\n\t\t\t\t// cobra does not support zsh completion ouptput used by source command\n\t\t\t\t// according to https://github.com/spf13/cobra/issues/1529\n\t\t\t\t// Need to append compdef manually to do that.\n\t\t\t\tzshHead := \"#compdef velero\\ncompdef _velero velero\\n\"\n\t\t\t\tout := os.Stdout\n\t\t\t\tif _, err := out.Write([]byte(zshHead)); err != nil {\n\t\t\t\t\tfmt.Println(\"fail to append compdef command into zsh completion script: \", err)\n\t\t\t\t\tos.Exit(1)\n\t\t\t\t}\n\n\t\t\t\tif err := cmd.Root().GenZshCompletion(out); err != nil {\n\t\t\t\t\tfmt.Println(\"fail to generate zsh completion script: \", err)\n\t\t\t\t\tos.Exit(1)\n\t\t\t\t}\n\t\t\tcase \"fish\":\n\t\t\t\tif err := cmd.Root().GenFishCompletion(os.Stdout, true); err != nil {\n\t\t\t\t\tfmt.Println(\"fail to generate fish completion script: \", err)\n\t\t\t\t\tos.Exit(1)\n\t\t\t\t}\n\t\t\tdefault:\n\t\t\t\tfmt.Println(\"Invalid shell specified, specify bash, zsh, or fish\")\n\t\t\t\tos.Exit(1)\n\t\t\t}\n\t\t},\n\t}\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/create/create.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage create\n\nimport (\n\t\"github.com/spf13/cobra\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/backup\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/backuplocation\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/restore\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/schedule\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/snapshotlocation\"\n)\n\nfunc NewCommand(f client.Factory) *cobra.Command {\n\tc := &cobra.Command{\n\t\tUse: \"create\",\n\t\tShort: \"Create velero resources\",\n\t\tLong: \"Create velero resources\",\n\t}\n\n\tc.AddCommand(\n\t\tbackup.NewCreateCommand(f, \"backup\"),\n\t\tschedule.NewCreateCommand(f, \"schedule\"),\n\t\trestore.NewCreateCommand(f, \"restore\"),\n\t\tbackuplocation.NewCreateCommand(f, \"backup-location\"),\n\t\tsnapshotlocation.NewCreateCommand(f, \"snapshot-location\"),\n\t)\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/datamover/backup.go", "content": "/*\nCopyright The Velero Contributors.\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n http://www.apache.org/licenses/LICENSE-2.0\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage datamover\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/bombsimon/logrusr/v3\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/spf13/cobra\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/fields\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"k8s.io/klog/v2\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\t\"github.com/vmware-tanzu/velero/pkg/buildinfo\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/signals\"\n\t\"github.com/vmware-tanzu/velero/pkg/datamover\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\n\tctlcache \"sigs.k8s.io/controller-runtime/pkg/cache\"\n\tctlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n)\n\ntype dataMoverBackupConfig struct {\n\tvolumePath string\n\tvolumeMode string\n\tduName string\n\tresourceTimeout time.Duration\n}\n\nfunc NewBackupCommand(f client.Factory) *cobra.Command {\n\tconfig := dataMoverBackupConfig{}\n\n\tlogLevelFlag := logging.LogLevelFlag(logrus.InfoLevel)\n\tformatFlag := logging.NewFormatFlag()\n\n\tcommand := &cobra.Command{\n\t\tUse: \"backup\",\n\t\tShort: \"Run the velero data-mover backup\",\n\t\tLong: \"Run the velero data-mover backup\",\n\t\tHidden: true,\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tlogLevel := logLevelFlag.Parse()\n\t\t\tlogrus.Infof(\"Setting log-level to %s\", strings.ToUpper(logLevel.String()))\n\n\t\t\tlogger := logging.DefaultLogger(logLevel, formatFlag.Parse())\n\t\t\tlogger.Infof(\"Starting Velero data-mover backup %s (%s)\", buildinfo.Version, buildinfo.FormattedGitSHA())\n\n\t\t\tf.SetBasename(fmt.Sprintf(\"%s-%s\", c.Parent().Name(), c.Name()))\n\t\t\ts, err := newdataMoverBackup(logger, f, config)\n\t\t\tif err != nil {\n\t\t\t\tkube.ExitPodWithMessage(logger, false, \"Failed to create data mover backup, %v\", err)\n\t\t\t}\n\n\t\t\ts.run()\n\t\t},\n\t}\n\n\tcommand.Flags().Var(logLevelFlag, \"log-level\", fmt.Sprintf(\"The level at which to log. Valid values are %s.\", strings.Join(logLevelFlag.AllowedValues(), \", \")))\n\tcommand.Flags().Var(formatFlag, \"log-format\", fmt.Sprintf(\"The format for log output. Valid values are %s.\", strings.Join(formatFlag.AllowedValues(), \", \")))\n\tcommand.Flags().StringVar(&config.volumePath, \"volume-path\", config.volumePath, \"The full path of the volume to be backed up\")\n\tcommand.Flags().StringVar(&config.volumeMode, \"volume-mode\", config.volumeMode, \"The mode of the volume to be backed up\")\n\tcommand.Flags().StringVar(&config.duName, \"data-upload\", config.duName, \"The data upload name\")\n\tcommand.Flags().DurationVar(&config.resourceTimeout, \"resource-timeout\", config.resourceTimeout, \"How long to wait for resource processes which are not covered by other specific timeout parameters.\")\n\n\t_ = command.MarkFlagRequired(\"volume-path\")\n\t_ = command.MarkFlagRequired(\"volume-mode\")\n\t_ = command.MarkFlagRequired(\"data-upload\")\n\t_ = command.MarkFlagRequired(\"resource-timeout\")\n\n\treturn command\n}\n\ntype dataMoverBackup struct {\n\tlogger logrus.FieldLogger\n\tctx context.Context\n\tcancelFunc context.CancelFunc\n\tclient ctlclient.Client\n\tcache ctlcache.Cache\n\tnamespace string\n\tnodeName string\n\tconfig dataMoverBackupConfig\n\tkubeClient kubernetes.Interface\n\tdataPathMgr *datapath.Manager\n}\n\nfunc newdataMoverBackup(logger logrus.FieldLogger, factory client.Factory, config dataMoverBackupConfig) (*dataMoverBackup, error) {\n\tctx, cancelFunc := context.WithCancel(context.Background())\n\n\tclientConfig, err := factory.ClientConfig()\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to create client config\")\n\t}\n\n\tctrl.SetLogger(logrusr.New(logger))\n\tklog.SetLogger(logrusr.New(logger)) // klog.Logger is used by k8s.io/client-go\n\n\tscheme := runtime.NewScheme()\n\tif err := velerov1api.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to add velero v1 scheme\")\n\t}\n\n\tif err := velerov2alpha1api.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to add velero v2alpha1 scheme\")\n\t}\n\n\tif err := corev1api.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to add core v1 scheme\")\n\t}\n\n\tnodeName := os.Getenv(\"NODE_NAME\")\n\n\t// use a field selector to filter to only pods scheduled on this node.\n\tcacheOption := ctlcache.Options{\n\t\tScheme: scheme,\n\t\tByObject: map[ctlclient.Object]ctlcache.ByObject{\n\t\t\t&corev1api.Pod{}: {\n\t\t\t\tField: fields.Set{\"spec.nodeName\": nodeName}.AsSelector(),\n\t\t\t},\n\t\t\t&velerov2alpha1api.DataUpload{}: {\n\t\t\t\tField: fields.Set{\"metadata.namespace\": factory.Namespace()}.AsSelector(),\n\t\t\t},\n\t\t},\n\t}\n\n\tcli, err := ctlclient.New(clientConfig, ctlclient.Options{\n\t\tScheme: scheme,\n\t})\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to create client\")\n\t}\n\n\tvar cache ctlcache.Cache\n\tretry := 10\n\tfor {\n\t\tcache, err = ctlcache.New(clientConfig, cacheOption)\n\t\tif err == nil {\n\t\t\tbreak\n\t\t}\n\n\t\tretry--\n\t\tif retry == 0 {\n\t\t\tbreak\n\t\t}\n\n\t\tlogger.WithError(err).Warn(\"Failed to create client cache, need retry\")\n\n\t\ttime.Sleep(time.Second)\n\t}\n\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to create client cache\")\n\t}\n\n\ts := &dataMoverBackup{\n\t\tlogger: logger,\n\t\tctx: ctx,\n\t\tcancelFunc: cancelFunc,\n\t\tclient: cli,\n\t\tcache: cache,\n\t\tconfig: config,\n\t\tnamespace: factory.Namespace(),\n\t\tnodeName: nodeName,\n\t}\n\n\ts.kubeClient, err = factory.KubeClient()\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to create kube client\")\n\t}\n\n\ts.dataPathMgr = datapath.NewManager(1)\n\n\treturn s, nil\n}\n\nvar funcExitWithMessage = kube.ExitPodWithMessage\nvar funcCreateDataPathService = (*dataMoverBackup).createDataPathService\n\nfunc (s *dataMoverBackup) run() {\n\tsignals.CancelOnShutdown(s.cancelFunc, s.logger)\n\tgo func() {\n\t\tif err := s.cache.Start(s.ctx); err != nil {\n\t\t\ts.logger.WithError(err).Warn(\"error starting cache\")\n\t\t}\n\t}()\n\n\ts.runDataPath()\n}\n\nfunc (s *dataMoverBackup) runDataPath() {\n\ts.logger.Infof(\"Starting micro service in node %s for du %s\", s.nodeName, s.config.duName)\n\n\tdpService, err := funcCreateDataPathService(s)\n\tif err != nil {\n\t\ts.cancelFunc()\n\t\tfuncExitWithMessage(s.logger, false, \"Failed to create data path service for DataUpload %s: %v\", s.config.duName, err)\n\t\treturn\n\t}\n\n\ts.logger.Infof(\"Starting data path service %s\", s.config.duName)\n\n\terr = dpService.Init()\n\tif err != nil {\n\t\tdpService.Shutdown()\n\t\ts.cancelFunc()\n\t\tfuncExitWithMessage(s.logger, false, \"Failed to init data path service for DataUpload %s: %v\", s.config.duName, err)\n\t\treturn\n\t}\n\n\ts.logger.Infof(\"Running data path service %s\", s.config.duName)\n\n\tresult, err := dpService.RunCancelableDataPath(s.ctx)\n\tif err != nil {\n\t\tdpService.Shutdown()\n\t\ts.cancelFunc()\n\t\tfuncExitWithMessage(s.logger, false, \"Failed to run data path service for DataUpload %s: %v\", s.config.duName, err)\n\t\treturn\n\t}\n\n\ts.logger.WithField(\"du\", s.config.duName).Info(\"Data path service completed\")\n\n\tdpService.Shutdown()\n\n\ts.logger.WithField(\"du\", s.config.duName).Info(\"Data path service is shut down\")\n\n\ts.cancelFunc()\n\n\tfuncExitWithMessage(s.logger, true, result)\n}\n\nvar funcNewCredentialFileStore = credentials.NewNamespacedFileStore\nvar funcNewCredentialSecretStore = credentials.NewNamespacedSecretStore\n\nfunc (s *dataMoverBackup) createDataPathService() (dataPathService, error) {\n\tcredentialFileStore, err := funcNewCredentialFileStore(\n\t\ts.client,\n\t\ts.namespace,\n\t\tcredentials.DefaultStoreDirectory(),\n\t\tfilesystem.NewFileSystem(),\n\t)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error to create credential file store\")\n\t}\n\n\tcredSecretStore, err := funcNewCredentialSecretStore(s.client, s.namespace)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error to create credential secret store\")\n\t}\n\n\tcredGetter := &credentials.CredentialGetter{FromFile: credentialFileStore, FromSecret: credSecretStore}\n\n\tduInformer, err := s.cache.GetInformer(s.ctx, &velerov2alpha1api.DataUpload{})\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error to get controller-runtime informer from manager\")\n\t}\n\n\trepoEnsurer := repository.NewEnsurer(s.client, s.logger, s.config.resourceTimeout)\n\n\treturn datamover.NewBackupMicroService(s.ctx, s.client, s.kubeClient, s.config.duName, s.namespace, s.nodeName, datapath.AccessPoint{\n\t\tByPath: s.config.volumePath,\n\t\tVolMode: uploader.PersistentVolumeMode(s.config.volumeMode),\n\t}, s.dataPathMgr, repoEnsurer, credGetter, duInformer, s.logger), nil\n}\n" }, { "path": "pkg/cmd/cli/datamover/backup_test.go", "content": "/*\nCopyright The Velero Contributors.\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n http://www.apache.org/licenses/LICENSE-2.0\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage datamover\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\tctlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\tcacheMock \"github.com/vmware-tanzu/velero/pkg/cmd/cli/datamover/mocks\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n)\n\nfunc fakeCreateDataPathServiceWithErr(_ *dataMoverBackup) (dataPathService, error) {\n\treturn nil, errors.New(\"fake-create-data-path-error\")\n}\n\nvar frHelper *fakeRunHelper\n\nfunc fakeCreateDataPathService(_ *dataMoverBackup) (dataPathService, error) {\n\treturn frHelper, nil\n}\n\ntype fakeRunHelper struct {\n\tinitErr error\n\trunCancelableDataPathErr error\n\trunCancelableDataPathResult string\n\texitMessage string\n\tsucceed bool\n}\n\nfunc (fr *fakeRunHelper) Init() error {\n\treturn fr.initErr\n}\n\nfunc (fr *fakeRunHelper) RunCancelableDataPath(_ context.Context) (string, error) {\n\tif fr.runCancelableDataPathErr != nil {\n\t\treturn \"\", fr.runCancelableDataPathErr\n\t} else {\n\t\treturn fr.runCancelableDataPathResult, nil\n\t}\n}\n\nfunc (fr *fakeRunHelper) Shutdown() {\n\n}\n\nfunc (fr *fakeRunHelper) ExitWithMessage(logger logrus.FieldLogger, succeed bool, message string, a ...any) {\n\tfr.succeed = succeed\n\tfr.exitMessage = fmt.Sprintf(message, a...)\n}\n\nfunc TestRunDataPath(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tduName string\n\t\tcreateDataPathFail bool\n\t\tinitDataPathErr error\n\t\trunCancelableDataPathErr error\n\t\trunCancelableDataPathResult string\n\t\texpectedMessage string\n\t\texpectedSucceed bool\n\t}{\n\t\t{\n\t\t\tname: \"create data path failed\",\n\t\t\tduName: \"fake-name\",\n\t\t\tcreateDataPathFail: true,\n\t\t\texpectedMessage: \"Failed to create data path service for DataUpload fake-name: fake-create-data-path-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"init data path failed\",\n\t\t\tduName: \"fake-name\",\n\t\t\tinitDataPathErr: errors.New(\"fake-init-data-path-error\"),\n\t\t\texpectedMessage: \"Failed to init data path service for DataUpload fake-name: fake-init-data-path-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"run data path failed\",\n\t\t\tduName: \"fake-name\",\n\t\t\trunCancelableDataPathErr: errors.New(\"fake-run-data-path-error\"),\n\t\t\texpectedMessage: \"Failed to run data path service for DataUpload fake-name: fake-run-data-path-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tduName: \"fake-name\",\n\t\t\trunCancelableDataPathResult: \"fake-run-data-path-result\",\n\t\t\texpectedMessage: \"fake-run-data-path-result\",\n\t\t\texpectedSucceed: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfrHelper = &fakeRunHelper{\n\t\t\t\tinitErr: test.initDataPathErr,\n\t\t\t\trunCancelableDataPathErr: test.runCancelableDataPathErr,\n\t\t\t\trunCancelableDataPathResult: test.runCancelableDataPathResult,\n\t\t\t}\n\n\t\t\tif test.createDataPathFail {\n\t\t\t\tfuncCreateDataPathService = fakeCreateDataPathServiceWithErr\n\t\t\t} else {\n\t\t\t\tfuncCreateDataPathService = fakeCreateDataPathService\n\t\t\t}\n\n\t\t\tfuncExitWithMessage = frHelper.ExitWithMessage\n\n\t\t\ts := &dataMoverBackup{\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t\tcancelFunc: func() {},\n\t\t\t\tconfig: dataMoverBackupConfig{\n\t\t\t\t\tduName: test.duName,\n\t\t\t\t},\n\t\t\t}\n\n\t\t\ts.runDataPath()\n\n\t\t\tassert.Equal(t, test.expectedMessage, frHelper.exitMessage)\n\t\t\tassert.Equal(t, test.expectedSucceed, frHelper.succeed)\n\t\t})\n\t}\n}\n\ntype fakeCreateDataPathServiceHelper struct {\n\tfileStoreErr error\n\tsecretStoreErr error\n}\n\nfunc (fc *fakeCreateDataPathServiceHelper) NewNamespacedFileStore(_ ctlclient.Client, _ string, _ string, _ filesystem.Interface) (credentials.FileStore, error) {\n\treturn nil, fc.fileStoreErr\n}\n\nfunc (fc *fakeCreateDataPathServiceHelper) NewNamespacedSecretStore(_ ctlclient.Client, _ string) (credentials.SecretStore, error) {\n\treturn nil, fc.secretStoreErr\n}\n\nfunc TestCreateDataPathService(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tfileStoreErr error\n\t\tsecretStoreErr error\n\t\tmockGetInformer bool\n\t\tgetInformerErr error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"create credential file store error\",\n\t\t\tfileStoreErr: errors.New(\"fake-file-store-error\"),\n\t\t\texpectedError: \"error to create credential file store: fake-file-store-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"create credential secret store\",\n\t\t\tsecretStoreErr: errors.New(\"fake-secret-store-error\"),\n\t\t\texpectedError: \"error to create credential secret store: fake-secret-store-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"get informer error\",\n\t\t\tmockGetInformer: true,\n\t\t\tgetInformerErr: errors.New(\"fake-get-informer-error\"),\n\t\t\texpectedError: \"error to get controller-runtime informer from manager: fake-get-informer-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tmockGetInformer: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfcHelper := &fakeCreateDataPathServiceHelper{\n\t\t\t\tfileStoreErr: test.fileStoreErr,\n\t\t\t\tsecretStoreErr: test.secretStoreErr,\n\t\t\t}\n\n\t\t\tfuncNewCredentialFileStore = fcHelper.NewNamespacedFileStore\n\t\t\tfuncNewCredentialSecretStore = fcHelper.NewNamespacedSecretStore\n\n\t\t\tcache := cacheMock.NewCache(t)\n\t\t\tif test.mockGetInformer {\n\t\t\t\tcache.On(\"GetInformer\", mock.Anything, mock.Anything).Return(nil, test.getInformerErr)\n\t\t\t}\n\n\t\t\tfuncExitWithMessage = frHelper.ExitWithMessage\n\n\t\t\ts := &dataMoverBackup{\n\t\t\t\tcache: cache,\n\t\t\t}\n\n\t\t\t_, err := s.createDataPathService()\n\n\t\t\tif test.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.expectedError)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/cmd/cli/datamover/data_mover.go", "content": "/*\nCopyright The Velero Contributors.\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n http://www.apache.org/licenses/LICENSE-2.0\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage datamover\n\nimport (\n\t\"context\"\n\n\t\"github.com/spf13/cobra\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n)\n\nfunc NewCommand(f client.Factory) *cobra.Command {\n\tcommand := &cobra.Command{\n\t\tUse: \"data-mover\",\n\t\tShort: \"Run the velero data-mover\",\n\t\tLong: \"Run the velero data-mover\",\n\t\tHidden: true,\n\t}\n\n\tcommand.AddCommand(\n\t\tNewBackupCommand(f),\n\t\tNewRestoreCommand(f),\n\t)\n\n\treturn command\n}\n\ntype dataPathService interface {\n\tInit() error\n\tRunCancelableDataPath(context.Context) (string, error)\n\tShutdown()\n}\n" }, { "path": "pkg/cmd/cli/datamover/mocks/Cache.go", "content": "// Code generated by mockery v2.39.1. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tcache \"sigs.k8s.io/controller-runtime/pkg/cache\"\n\tclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tcontext \"context\"\n\n\tmock \"github.com/stretchr/testify/mock\"\n\n\tschema \"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\ttypes \"k8s.io/apimachinery/pkg/types\"\n)\n\n// Cache is an autogenerated mock type for the Cache type\ntype Cache struct {\n\tmock.Mock\n}\n\n// Get provides a mock function with given fields: ctx, key, obj, opts\nfunc (_m *Cache) Get(ctx context.Context, key types.NamespacedName, obj client.Object, opts ...client.GetOption) error {\n\t_va := make([]interface{}, len(opts))\n\tfor _i := range opts {\n\t\t_va[_i] = opts[_i]\n\t}\n\tvar _ca []interface{}\n\t_ca = append(_ca, ctx, key, obj)\n\t_ca = append(_ca, _va...)\n\tret := _m.Called(_ca...)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Get\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, types.NamespacedName, client.Object, ...client.GetOption) error); ok {\n\t\tr0 = rf(ctx, key, obj, opts...)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// GetInformer provides a mock function with given fields: ctx, obj, opts\nfunc (_m *Cache) GetInformer(ctx context.Context, obj client.Object, opts ...cache.InformerGetOption) (cache.Informer, error) {\n\t_va := make([]interface{}, len(opts))\n\tfor _i := range opts {\n\t\t_va[_i] = opts[_i]\n\t}\n\tvar _ca []interface{}\n\t_ca = append(_ca, ctx, obj)\n\t_ca = append(_ca, _va...)\n\tret := _m.Called(_ca...)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetInformer\")\n\t}\n\n\tvar r0 cache.Informer\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(context.Context, client.Object, ...cache.InformerGetOption) (cache.Informer, error)); ok {\n\t\treturn rf(ctx, obj, opts...)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, client.Object, ...cache.InformerGetOption) cache.Informer); ok {\n\t\tr0 = rf(ctx, obj, opts...)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(cache.Informer)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, client.Object, ...cache.InformerGetOption) error); ok {\n\t\tr1 = rf(ctx, obj, opts...)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetInformerForKind provides a mock function with given fields: ctx, gvk, opts\nfunc (_m *Cache) GetInformerForKind(ctx context.Context, gvk schema.GroupVersionKind, opts ...cache.InformerGetOption) (cache.Informer, error) {\n\t_va := make([]interface{}, len(opts))\n\tfor _i := range opts {\n\t\t_va[_i] = opts[_i]\n\t}\n\tvar _ca []interface{}\n\t_ca = append(_ca, ctx, gvk)\n\t_ca = append(_ca, _va...)\n\tret := _m.Called(_ca...)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetInformerForKind\")\n\t}\n\n\tvar r0 cache.Informer\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(context.Context, schema.GroupVersionKind, ...cache.InformerGetOption) (cache.Informer, error)); ok {\n\t\treturn rf(ctx, gvk, opts...)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, schema.GroupVersionKind, ...cache.InformerGetOption) cache.Informer); ok {\n\t\tr0 = rf(ctx, gvk, opts...)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(cache.Informer)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, schema.GroupVersionKind, ...cache.InformerGetOption) error); ok {\n\t\tr1 = rf(ctx, gvk, opts...)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// IndexField provides a mock function with given fields: ctx, obj, field, extractValue\nfunc (_m *Cache) IndexField(ctx context.Context, obj client.Object, field string, extractValue client.IndexerFunc) error {\n\tret := _m.Called(ctx, obj, field, extractValue)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for IndexField\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, client.Object, string, client.IndexerFunc) error); ok {\n\t\tr0 = rf(ctx, obj, field, extractValue)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// List provides a mock function with given fields: ctx, list, opts\nfunc (_m *Cache) List(ctx context.Context, list client.ObjectList, opts ...client.ListOption) error {\n\t_va := make([]interface{}, len(opts))\n\tfor _i := range opts {\n\t\t_va[_i] = opts[_i]\n\t}\n\tvar _ca []interface{}\n\t_ca = append(_ca, ctx, list)\n\t_ca = append(_ca, _va...)\n\tret := _m.Called(_ca...)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for List\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, client.ObjectList, ...client.ListOption) error); ok {\n\t\tr0 = rf(ctx, list, opts...)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// RemoveInformer provides a mock function with given fields: ctx, obj\nfunc (_m *Cache) RemoveInformer(ctx context.Context, obj client.Object) error {\n\tret := _m.Called(ctx, obj)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for RemoveInformer\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, client.Object) error); ok {\n\t\tr0 = rf(ctx, obj)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// Start provides a mock function with given fields: ctx\nfunc (_m *Cache) Start(ctx context.Context) error {\n\tret := _m.Called(ctx)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Start\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context) error); ok {\n\t\tr0 = rf(ctx)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// WaitForCacheSync provides a mock function with given fields: ctx\nfunc (_m *Cache) WaitForCacheSync(ctx context.Context) bool {\n\tret := _m.Called(ctx)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for WaitForCacheSync\")\n\t}\n\n\tvar r0 bool\n\tif rf, ok := ret.Get(0).(func(context.Context) bool); ok {\n\t\tr0 = rf(ctx)\n\t} else {\n\t\tr0 = ret.Get(0).(bool)\n\t}\n\n\treturn r0\n}\n\n// NewCache creates a new instance of Cache. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\n// The first argument is typically a *testing.T value.\nfunc NewCache(t interface {\n\tmock.TestingT\n\tCleanup(func())\n}) *Cache {\n\tmock := &Cache{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/cmd/cli/datamover/restore.go", "content": "/*\nCopyright The Velero Contributors.\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n http://www.apache.org/licenses/LICENSE-2.0\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage datamover\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/bombsimon/logrusr/v3\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/spf13/cobra\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/fields\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"k8s.io/klog/v2\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/buildinfo\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/signals\"\n\t\"github.com/vmware-tanzu/velero/pkg/datamover\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n\n\tctlcache \"sigs.k8s.io/controller-runtime/pkg/cache\"\n\tctlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n)\n\ntype dataMoverRestoreConfig struct {\n\tvolumePath string\n\tvolumeMode string\n\tddName string\n\tcacheDir string\n\tresourceTimeout time.Duration\n}\n\nfunc NewRestoreCommand(f client.Factory) *cobra.Command {\n\tlogLevelFlag := logging.LogLevelFlag(logrus.InfoLevel)\n\tformatFlag := logging.NewFormatFlag()\n\n\tconfig := dataMoverRestoreConfig{}\n\n\tcommand := &cobra.Command{\n\t\tUse: \"restore\",\n\t\tShort: \"Run the velero data-mover restore\",\n\t\tLong: \"Run the velero data-mover restore\",\n\t\tHidden: true,\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tlogLevel := logLevelFlag.Parse()\n\t\t\tlogrus.Infof(\"Setting log-level to %s\", strings.ToUpper(logLevel.String()))\n\n\t\t\tlogger := logging.DefaultLogger(logLevel, formatFlag.Parse())\n\t\t\tlogger.Infof(\"Starting Velero data-mover restore %s (%s)\", buildinfo.Version, buildinfo.FormattedGitSHA())\n\n\t\t\tf.SetBasename(fmt.Sprintf(\"%s-%s\", c.Parent().Name(), c.Name()))\n\t\t\ts, err := newdataMoverRestore(logger, f, config)\n\t\t\tif err != nil {\n\t\t\t\tkube.ExitPodWithMessage(logger, false, \"Failed to create data mover restore, %v\", err)\n\t\t\t}\n\n\t\t\ts.run()\n\t\t},\n\t}\n\n\tcommand.Flags().Var(logLevelFlag, \"log-level\", fmt.Sprintf(\"The level at which to log. Valid values are %s.\", strings.Join(logLevelFlag.AllowedValues(), \", \")))\n\tcommand.Flags().Var(formatFlag, \"log-format\", fmt.Sprintf(\"The format for log output. Valid values are %s.\", strings.Join(formatFlag.AllowedValues(), \", \")))\n\tcommand.Flags().StringVar(&config.volumePath, \"volume-path\", config.volumePath, \"The full path of the volume to be restored\")\n\tcommand.Flags().StringVar(&config.volumeMode, \"volume-mode\", config.volumeMode, \"The mode of the volume to be restored\")\n\tcommand.Flags().StringVar(&config.ddName, \"data-download\", config.ddName, \"The data download name\")\n\tcommand.Flags().StringVar(&config.cacheDir, \"cache-volume-path\", config.cacheDir, \"The full path of the cache volume\")\n\tcommand.Flags().DurationVar(&config.resourceTimeout, \"resource-timeout\", config.resourceTimeout, \"How long to wait for resource processes which are not covered by other specific timeout parameters.\")\n\n\t_ = command.MarkFlagRequired(\"volume-path\")\n\t_ = command.MarkFlagRequired(\"volume-mode\")\n\t_ = command.MarkFlagRequired(\"data-download\")\n\t_ = command.MarkFlagRequired(\"resource-timeout\")\n\n\treturn command\n}\n\ntype dataMoverRestore struct {\n\tlogger logrus.FieldLogger\n\tctx context.Context\n\tcancelFunc context.CancelFunc\n\tclient ctlclient.Client\n\tcache ctlcache.Cache\n\tnamespace string\n\tnodeName string\n\tconfig dataMoverRestoreConfig\n\tkubeClient kubernetes.Interface\n\tdataPathMgr *datapath.Manager\n}\n\nfunc newdataMoverRestore(logger logrus.FieldLogger, factory client.Factory, config dataMoverRestoreConfig) (*dataMoverRestore, error) {\n\tctx, cancelFunc := context.WithCancel(context.Background())\n\n\tclientConfig, err := factory.ClientConfig()\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to create client config\")\n\t}\n\n\tctrl.SetLogger(logrusr.New(logger))\n\tklog.SetLogger(logrusr.New(logger)) // klog.Logger is used by k8s.io/client-go\n\n\tscheme := runtime.NewScheme()\n\tif err := velerov1api.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to add velero v1 scheme\")\n\t}\n\n\tif err := velerov2alpha1api.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to add velero v2alpha1 scheme\")\n\t}\n\n\tif err := corev1api.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to add core v1 scheme\")\n\t}\n\n\tnodeName := os.Getenv(\"NODE_NAME\")\n\n\t// use a field selector to filter to only pods scheduled on this node.\n\tcacheOption := ctlcache.Options{\n\t\tScheme: scheme,\n\t\tByObject: map[ctlclient.Object]ctlcache.ByObject{\n\t\t\t&corev1api.Pod{}: {\n\t\t\t\tField: fields.Set{\"spec.nodeName\": nodeName}.AsSelector(),\n\t\t\t},\n\t\t\t&velerov2alpha1api.DataDownload{}: {\n\t\t\t\tField: fields.Set{\"metadata.namespace\": factory.Namespace()}.AsSelector(),\n\t\t\t},\n\t\t},\n\t}\n\n\tcli, err := ctlclient.New(clientConfig, ctlclient.Options{\n\t\tScheme: scheme,\n\t})\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to create client\")\n\t}\n\n\tvar cache ctlcache.Cache\n\tretry := 10\n\tfor {\n\t\tcache, err = ctlcache.New(clientConfig, cacheOption)\n\t\tif err == nil {\n\t\t\tbreak\n\t\t}\n\n\t\tretry--\n\t\tif retry == 0 {\n\t\t\tbreak\n\t\t}\n\n\t\tlogger.WithError(err).Warn(\"Failed to create client cache, need retry\")\n\n\t\ttime.Sleep(time.Second)\n\t}\n\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to create client cache\")\n\t}\n\n\ts := &dataMoverRestore{\n\t\tlogger: logger,\n\t\tctx: ctx,\n\t\tcancelFunc: cancelFunc,\n\t\tclient: cli,\n\t\tcache: cache,\n\t\tconfig: config,\n\t\tnamespace: factory.Namespace(),\n\t\tnodeName: nodeName,\n\t}\n\n\ts.kubeClient, err = factory.KubeClient()\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to create kube client\")\n\t}\n\n\ts.dataPathMgr = datapath.NewManager(1)\n\n\treturn s, nil\n}\n\nvar funcCreateDataPathRestore = (*dataMoverRestore).createDataPathService\n\nfunc (s *dataMoverRestore) run() {\n\tsignals.CancelOnShutdown(s.cancelFunc, s.logger)\n\tgo func() {\n\t\tif err := s.cache.Start(s.ctx); err != nil {\n\t\t\ts.logger.WithError(err).Warn(\"error starting cache\")\n\t\t}\n\t}()\n\n\ts.runDataPath()\n}\n\nfunc (s *dataMoverRestore) runDataPath() {\n\ts.logger.Infof(\"Starting micro service in node %s for dd %s\", s.nodeName, s.config.ddName)\n\n\tdpService, err := funcCreateDataPathRestore(s)\n\tif err != nil {\n\t\ts.cancelFunc()\n\t\tfuncExitWithMessage(s.logger, false, \"Failed to create data path service for DataDownload %s: %v\", s.config.ddName, err)\n\t\treturn\n\t}\n\n\ts.logger.Infof(\"Starting data path service %s\", s.config.ddName)\n\n\terr = dpService.Init()\n\tif err != nil {\n\t\tdpService.Shutdown()\n\t\ts.cancelFunc()\n\t\tfuncExitWithMessage(s.logger, false, \"Failed to init data path service for DataDownload %s: %v\", s.config.ddName, err)\n\t\treturn\n\t}\n\n\tresult, err := dpService.RunCancelableDataPath(s.ctx)\n\tif err != nil {\n\t\tdpService.Shutdown()\n\t\ts.cancelFunc()\n\t\tfuncExitWithMessage(s.logger, false, \"Failed to run data path service for DataDownload %s: %v\", s.config.ddName, err)\n\t\treturn\n\t}\n\n\ts.logger.WithField(\"dd\", s.config.ddName).Info(\"Data path service completed\")\n\n\tdpService.Shutdown()\n\n\ts.logger.WithField(\"dd\", s.config.ddName).Info(\"Data path service is shut down\")\n\n\ts.cancelFunc()\n\n\tfuncExitWithMessage(s.logger, true, result)\n}\n\nfunc (s *dataMoverRestore) createDataPathService() (dataPathService, error) {\n\tcredentialFileStore, err := funcNewCredentialFileStore(\n\t\ts.client,\n\t\ts.namespace,\n\t\tcredentials.DefaultStoreDirectory(),\n\t\tfilesystem.NewFileSystem(),\n\t)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error to create credential file store\")\n\t}\n\n\tcredSecretStore, err := funcNewCredentialSecretStore(s.client, s.namespace)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error to create credential secret store\")\n\t}\n\n\tcredGetter := &credentials.CredentialGetter{FromFile: credentialFileStore, FromSecret: credSecretStore}\n\n\tduInformer, err := s.cache.GetInformer(s.ctx, &velerov2alpha1api.DataDownload{})\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error to get controller-runtime informer from manager\")\n\t}\n\n\trepoEnsurer := repository.NewEnsurer(s.client, s.logger, s.config.resourceTimeout)\n\n\treturn datamover.NewRestoreMicroService(s.ctx, s.client, s.kubeClient, s.config.ddName, s.namespace, s.nodeName, datapath.AccessPoint{\n\t\tByPath: s.config.volumePath,\n\t\tVolMode: uploader.PersistentVolumeMode(s.config.volumeMode),\n\t}, s.dataPathMgr, repoEnsurer, credGetter, duInformer, s.config.cacheDir, s.logger), nil\n}\n" }, { "path": "pkg/cmd/cli/datamover/restore_test.go", "content": "/*\nCopyright The Velero Contributors.\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n http://www.apache.org/licenses/LICENSE-2.0\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage datamover\n\nimport (\n\t\"errors\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\n\tcacheMock \"github.com/vmware-tanzu/velero/pkg/cmd/cli/datamover/mocks\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc fakeCreateDataPathRestoreWithErr(_ *dataMoverRestore) (dataPathService, error) {\n\treturn nil, errors.New(\"fake-create-data-path-error\")\n}\n\nfunc fakeCreateDataPathRestore(_ *dataMoverRestore) (dataPathService, error) {\n\treturn frHelper, nil\n}\n\nfunc TestRunDataPathRestore(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tddName string\n\t\tcreateDataPathFail bool\n\t\tinitDataPathErr error\n\t\trunCancelableDataPathErr error\n\t\trunCancelableDataPathResult string\n\t\texpectedMessage string\n\t\texpectedSucceed bool\n\t}{\n\t\t{\n\t\t\tname: \"create data path failed\",\n\t\t\tddName: \"fake-name\",\n\t\t\tcreateDataPathFail: true,\n\t\t\texpectedMessage: \"Failed to create data path service for DataDownload fake-name: fake-create-data-path-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"init data path failed\",\n\t\t\tddName: \"fake-name\",\n\t\t\tinitDataPathErr: errors.New(\"fake-init-data-path-error\"),\n\t\t\texpectedMessage: \"Failed to init data path service for DataDownload fake-name: fake-init-data-path-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"run data path failed\",\n\t\t\tddName: \"fake-name\",\n\t\t\trunCancelableDataPathErr: errors.New(\"fake-run-data-path-error\"),\n\t\t\texpectedMessage: \"Failed to run data path service for DataDownload fake-name: fake-run-data-path-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tddName: \"fake-name\",\n\t\t\trunCancelableDataPathResult: \"fake-run-data-path-result\",\n\t\t\texpectedMessage: \"fake-run-data-path-result\",\n\t\t\texpectedSucceed: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfrHelper = &fakeRunHelper{\n\t\t\t\tinitErr: test.initDataPathErr,\n\t\t\t\trunCancelableDataPathErr: test.runCancelableDataPathErr,\n\t\t\t\trunCancelableDataPathResult: test.runCancelableDataPathResult,\n\t\t\t}\n\n\t\t\tif test.createDataPathFail {\n\t\t\t\tfuncCreateDataPathRestore = fakeCreateDataPathRestoreWithErr\n\t\t\t} else {\n\t\t\t\tfuncCreateDataPathRestore = fakeCreateDataPathRestore\n\t\t\t}\n\n\t\t\tfuncExitWithMessage = frHelper.ExitWithMessage\n\n\t\t\ts := &dataMoverRestore{\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t\tcancelFunc: func() {},\n\t\t\t\tconfig: dataMoverRestoreConfig{\n\t\t\t\t\tddName: test.ddName,\n\t\t\t\t},\n\t\t\t}\n\n\t\t\ts.runDataPath()\n\n\t\t\tassert.Equal(t, test.expectedMessage, frHelper.exitMessage)\n\t\t\tassert.Equal(t, test.expectedSucceed, frHelper.succeed)\n\t\t})\n\t}\n}\n\nfunc TestCreateDataPathRestore(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tfileStoreErr error\n\t\tsecretStoreErr error\n\t\tmockGetInformer bool\n\t\tgetInformerErr error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"create credential file store error\",\n\t\t\tfileStoreErr: errors.New(\"fake-file-store-error\"),\n\t\t\texpectedError: \"error to create credential file store: fake-file-store-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"create credential secret store\",\n\t\t\tsecretStoreErr: errors.New(\"fake-secret-store-error\"),\n\t\t\texpectedError: \"error to create credential secret store: fake-secret-store-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"get informer error\",\n\t\t\tmockGetInformer: true,\n\t\t\tgetInformerErr: errors.New(\"fake-get-informer-error\"),\n\t\t\texpectedError: \"error to get controller-runtime informer from manager: fake-get-informer-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tmockGetInformer: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfcHelper := &fakeCreateDataPathServiceHelper{\n\t\t\t\tfileStoreErr: test.fileStoreErr,\n\t\t\t\tsecretStoreErr: test.secretStoreErr,\n\t\t\t}\n\n\t\t\tfuncNewCredentialFileStore = fcHelper.NewNamespacedFileStore\n\t\t\tfuncNewCredentialSecretStore = fcHelper.NewNamespacedSecretStore\n\n\t\t\tcache := cacheMock.NewCache(t)\n\t\t\tif test.mockGetInformer {\n\t\t\t\tcache.On(\"GetInformer\", mock.Anything, mock.Anything).Return(nil, test.getInformerErr)\n\t\t\t}\n\n\t\t\tfuncExitWithMessage = frHelper.ExitWithMessage\n\n\t\t\ts := &dataMoverRestore{\n\t\t\t\tcache: cache,\n\t\t\t}\n\n\t\t\t_, err := s.createDataPathService()\n\n\t\t\tif test.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.expectedError)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/cmd/cli/debug/cshd-scripts/velero.cshd", "content": "def capture_backup_logs(cmd, namespace):\n if args.backup:\n log(\"Collecting log and information for backup: {}\".format(args.backup))\n backupDescCmd = \"{} --namespace={} backup describe {} --details\".format(cmd, namespace, args.backup)\n capture_local(cmd=backupDescCmd, file_name=\"backup_describe_{}.txt\".format(args.backup))\n backupLogsCmd = \"{} --namespace={} backup logs {}\".format(cmd, namespace, args.backup)\n capture_local(cmd=backupLogsCmd, file_name=\"backup_{}.log\".format(args.backup))\ndef capture_restore_logs(cmd, namespace):\n if args.restore:\n log(\"Collecting log and information for restore: {}\".format(args.restore))\n restoreDescCmd = \"{} --namespace={} restore describe {} --details\".format(cmd, namespace, args.restore)\n capture_local(cmd=restoreDescCmd, file_name=\"restore_describe_{}.txt\".format(args.restore))\n restoreLogsCmd = \"{} --namespace={} restore logs {}\".format(cmd, namespace, args.restore)\n capture_local(cmd=restoreLogsCmd, file_name=\"restore_{}.log\".format(args.restore))\n\nns = args.namespace if args.namespace else \"velero\"\noutput = args.output if args.output else \"bundle.tar.gz\"\ncmd = args.cmd if args.cmd else \"velero\"\n# Working dir for writing during script execution\ncrshd = crashd_config(workdir=\"./velero-bundle\")\nset_defaults(kube_config(path=args.kubeconfig, cluster_context=args.kubecontext))\nlog(\"Collecting velero resources in namespace: {}\". format(ns))\nkube_capture(what=\"objects\", namespaces=[ns], groups=['velero.io'])\ncapture_local(cmd=\"{} version -n {}\".format(cmd, ns), file_name=\"version.txt\")\nlog(\"Collecting velero deployment logs in namespace: {}\". format(ns))\nkube_capture(what=\"logs\", namespaces=[ns])\ncapture_backup_logs(cmd, ns)\ncapture_restore_logs(cmd, ns)\narchive(output_file=output, source_paths=[crshd.workdir])\nlog(\"Generated debug information bundle: {}\".format(output))\n\n \n" }, { "path": "pkg/cmd/cli/debug/debug.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage debug\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t_ \"embed\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/spf13/cobra\"\n\t\"github.com/spf13/pflag\"\n\t\"github.com/vmware-tanzu/crash-diagnostics/exec\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/client-go/tools/clientcmd\"\n\tctrlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n)\n\n//go:embed cshd-scripts/velero.cshd\nvar scriptBytes []byte\n\ntype option struct {\n\t// currCmd the velero command\n\tcurrCmd string\n\t// workdir for crashd will be $baseDir/velero-debug\n\tbaseDir string\n\t// the namespace where velero server is installed\n\tnamespace string\n\t// the absolute path for the log bundle to be generated\n\toutputPath string\n\t// the absolute path for the kubeconfig file that will be read by crashd for calling K8S API\n\tkubeconfigPath string\n\t// the kubecontext to be used for calling K8S API\n\tkubeContext string\n\t// optional, the name of the backup resource whose log will be packaged into the debug bundle\n\tbackup string\n\t// optional, the name of the restore resource whose log will be packaged into the debug bundle\n\trestore string\n\t// optional, it controls whether to print the debug log messages when calling crashd\n\tverbose bool\n}\n\nfunc (o *option) bindFlags(flags *pflag.FlagSet) {\n\tflags.StringVar(&o.outputPath, \"output\", \"\", \"The path of the bundle tarball, by default it's ./bundle---
---.tar.gz. Optional\")\n\tflags.StringVar(&o.backup, \"backup\", \"\", \"The name of the backup resource whose log will be collected, no backup logs will be collected if it's not set. Optional\")\n\tflags.StringVar(&o.restore, \"restore\", \"\", \"The name of the restore resource whose log will be collected, no restore logs will be collected if it's not set. Optional\")\n\tflags.BoolVar(&o.verbose, \"verbose\", false, \"When it's set to true the debug messages by crashd will be printed during execution. Default value is false.\")\n}\n\nfunc (o *option) asCrashdArgMap() exec.ArgMap {\n\treturn exec.ArgMap{\n\t\t\"cmd\": o.currCmd,\n\t\t\"output\": o.outputPath,\n\t\t\"namespace\": o.namespace,\n\t\t\"basedir\": o.baseDir,\n\t\t\"backup\": o.backup,\n\t\t\"restore\": o.restore,\n\t\t\"kubeconfig\": o.kubeconfigPath,\n\t\t\"kubecontext\": o.kubeContext,\n\t}\n}\n\nfunc (o *option) complete(f client.Factory, fs *pflag.FlagSet) error {\n\tif len(o.outputPath) == 0 {\n\t\to.outputPath = fmt.Sprintf(\"./bundle-%s.tar.gz\", time.Now().Format(\"2006-01-02-15-04-05\"))\n\t}\n\tabsOutputPath, err := filepath.Abs(o.outputPath)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"invalid output path: %v\", err)\n\t}\n\to.outputPath = absOutputPath\n\ttmpDir, err := os.MkdirTemp(\"\", \"crashd\")\n\tif err != nil {\n\t\treturn err\n\t}\n\to.baseDir = tmpDir\n\to.namespace = f.Namespace()\n\tkp, kc := kubeconfigAndContext(fs)\n\to.currCmd, err = os.Executable()\n\tif err != nil {\n\t\treturn err\n\t}\n\to.kubeconfigPath, err = filepath.Abs(kp)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"invalid kubeconfig path: %s, %v\", kp, err)\n\t}\n\to.kubeContext = kc\n\treturn nil\n}\n\nfunc (o *option) validate(f client.Factory) error {\n\tcrClient, err := f.KubebuilderClient()\n\tif err != nil {\n\t\treturn err\n\t}\n\tdeploymentList := new(appsv1api.DeploymentList)\n\tselector, err := labels.Parse(\"component=velero\")\n\tcmd.CheckError(err)\n\terr = crClient.List(context.TODO(), deploymentList, &ctrlclient.ListOptions{\n\t\tNamespace: o.namespace,\n\t\tLabelSelector: selector,\n\t})\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"failed to check velero deployment\")\n\t}\n\tif len(deploymentList.Items) == 0 {\n\t\treturn fmt.Errorf(\"velero deployment does not exist in namespace: %s\", o.namespace)\n\t}\n\tif len(o.backup) > 0 {\n\t\tbackup := new(velerov1api.Backup)\n\t\tif err := crClient.Get(context.TODO(), ctrlclient.ObjectKey{Namespace: o.namespace, Name: o.backup}, backup); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\tif len(o.restore) > 0 {\n\t\trestore := new(velerov1api.Restore)\n\t\tif err := crClient.Get(context.TODO(), ctrlclient.ObjectKey{Namespace: o.namespace, Name: o.restore}, restore); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n\n// NewCommand creates a cobra command.\nfunc NewCommand(f client.Factory) *cobra.Command {\n\to := &option{}\n\tc := &cobra.Command{\n\t\tUse: \"debug\",\n\t\tShort: \"Generate debug bundle\",\n\t\tLong: `Generate a tarball containing the logs of velero deployment, plugin logs, node-agent DaemonSet, \nspecs of resources created by velero server, and optionally the logs of backup and restore.`,\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tflags := c.Flags()\n\t\t\terr := o.complete(f, flags)\n\t\t\tcmd.CheckError(err)\n\t\t\tdefer func(opt *option) {\n\t\t\t\tif len(opt.baseDir) > 0 {\n\t\t\t\t\tif err := os.RemoveAll(opt.baseDir); err != nil {\n\t\t\t\t\t\tfmt.Fprintf(os.Stderr, \"Failed to remove temp dir: %s: %v\\n\", opt.baseDir, err)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}(o)\n\t\t\terr = o.validate(f)\n\t\t\tcmd.CheckError(err)\n\t\t\terr = runCrashd(o)\n\t\t\tcmd.CheckError(err)\n\t\t},\n\t}\n\to.bindFlags(c.Flags())\n\treturn c\n}\n\nfunc runCrashd(o *option) error {\n\tpwd, err := os.Getwd()\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer func() {\n\t\tif err := os.Chdir(pwd); err != nil {\n\t\t\tfmt.Fprintf(os.Stderr, \"Failed to go back to workdir: %v\", err)\n\t\t}\n\t}()\n\tif err := os.Chdir(o.baseDir); err != nil {\n\t\treturn err\n\t}\n\tlogrus.SetOutput(os.Stdout)\n\tif o.verbose {\n\t\tlogrus.SetLevel(logrus.DebugLevel)\n\t}\n\treturn exec.Execute(\"velero-debug-collector\", bytes.NewReader(scriptBytes), o.asCrashdArgMap())\n}\n\nfunc kubeconfigAndContext(fs *pflag.FlagSet) (string, string) {\n\tpathOpt := clientcmd.NewDefaultPathOptions()\n\tkubeconfig, _ := fs.GetString(\"kubeconfig\")\n\tif len(kubeconfig) > 0 {\n\t\tpathOpt.LoadingRules.ExplicitPath = kubeconfig\n\t}\n\tkubecontext, _ := fs.GetString(\"kubecontext\")\n\treturn pathOpt.GetDefaultFilename(), kubecontext\n}\n" }, { "path": "pkg/cmd/cli/delete/delete.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage delete\n\nimport (\n\t\"github.com/spf13/cobra\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/backup\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/backuplocation\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/restore\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/schedule\"\n)\n\nfunc NewCommand(f client.Factory) *cobra.Command {\n\tc := &cobra.Command{\n\t\tUse: \"delete\",\n\t\tShort: \"Delete velero resources\",\n\t\tLong: \"Delete velero resources\",\n\t}\n\n\tbackupCommand := backup.NewDeleteCommand(f, \"backup\")\n\tbackupCommand.Aliases = []string{\"backups\"}\n\n\tbackuplocationCommand := backuplocation.NewDeleteCommand(f, \"backup-location\")\n\tbackuplocationCommand.Aliases = []string{\"backup-locations\"}\n\n\trestoreCommand := restore.NewDeleteCommand(f, \"restore\")\n\trestoreCommand.Aliases = []string{\"restores\"}\n\n\tscheduleCommand := schedule.NewDeleteCommand(f, \"schedule\")\n\tscheduleCommand.Aliases = []string{\"schedules\"}\n\n\tc.AddCommand(\n\t\tbackupCommand,\n\t\tbackuplocationCommand,\n\t\trestoreCommand,\n\t\tscheduleCommand,\n\t)\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/delete_options.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage cli\n\nimport (\n\t\"errors\"\n\n\t\"github.com/spf13/cobra\"\n\t\"github.com/spf13/pflag\"\n\tcontrollerclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/confirm\"\n)\n\n// DeleteOptions contains parameters used for deleting a restore.\ntype DeleteOptions struct {\n\t*SelectOptions\n\tconfirm.ConfirmOptions\n\tClient controllerclient.Client\n\tNamespace string\n}\n\nfunc NewDeleteOptions(singularTypeName string) *DeleteOptions {\n\to := &DeleteOptions{}\n\to.ConfirmOptions = *confirm.NewConfirmOptionsWithDescription(\"Confirm deletion\")\n\to.SelectOptions = NewSelectOptions(\"delete\", singularTypeName)\n\treturn o\n}\n\n// Complete fills in the correct values for all the options.\nfunc (o *DeleteOptions) Complete(f client.Factory, args []string) error {\n\to.Namespace = f.Namespace()\n\tclient, err := f.KubebuilderClient()\n\tif err != nil {\n\t\treturn err\n\t}\n\to.Client = client\n\treturn o.SelectOptions.Complete(args)\n}\n\n// Validate validates the fields of the DeleteOptions struct.\nfunc (o *DeleteOptions) Validate(c *cobra.Command, f client.Factory, args []string) error {\n\tif o.Client == nil {\n\t\treturn errors.New(\"velero client is not set; unable to proceed\")\n\t}\n\n\treturn o.SelectOptions.Validate()\n}\n\n// BindFlags binds options for this command to flags.\nfunc (o *DeleteOptions) BindFlags(flags *pflag.FlagSet) {\n\to.ConfirmOptions.BindFlags(flags)\n\to.SelectOptions.BindFlags(flags)\n}\n\n// Xor returns true if exactly one of the provided values is true,\n// or false otherwise.\nfunc xor(val bool, vals ...bool) bool {\n\tres := val\n\n\tfor _, v := range vals {\n\t\tif res && v {\n\t\t\treturn false\n\t\t}\n\t\tres = res || v\n\t}\n\treturn res\n}\n" }, { "path": "pkg/cmd/cli/describe/describe.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage describe\n\nimport (\n\t\"github.com/spf13/cobra\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/backup\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/restore\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/schedule\"\n)\n\nfunc NewCommand(f client.Factory) *cobra.Command {\n\tc := &cobra.Command{\n\t\tUse: \"describe\",\n\t\tShort: \"Describe velero resources\",\n\t\tLong: \"Describe velero resources\",\n\t}\n\n\tbackupCommand := backup.NewDescribeCommand(f, \"backups\")\n\tbackupCommand.Aliases = []string{\"backup\"}\n\n\tscheduleCommand := schedule.NewDescribeCommand(f, \"schedules\")\n\tscheduleCommand.Aliases = []string{\"schedule\"}\n\n\trestoreCommand := restore.NewDescribeCommand(f, \"restores\")\n\trestoreCommand.Aliases = []string{\"restore\"}\n\n\tc.AddCommand(\n\t\tbackupCommand,\n\t\tscheduleCommand,\n\t\trestoreCommand,\n\t)\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/get/get.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage get\n\nimport (\n\t\"github.com/spf13/cobra\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/backup\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/backuplocation\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/plugin\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/restore\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/schedule\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/snapshotlocation\"\n)\n\nfunc NewCommand(f client.Factory) *cobra.Command {\n\tc := &cobra.Command{\n\t\tUse: \"get\",\n\t\tShort: \"Get velero resources\",\n\t\tLong: \"Get velero resources\",\n\t}\n\n\tbackupCommand := backup.NewGetCommand(f, \"backups\")\n\tbackupCommand.Aliases = []string{\"backup\"}\n\n\tscheduleCommand := schedule.NewGetCommand(f, \"schedules\")\n\tscheduleCommand.Aliases = []string{\"schedule\"}\n\n\trestoreCommand := restore.NewGetCommand(f, \"restores\")\n\trestoreCommand.Aliases = []string{\"restore\"}\n\n\tbackupLocationCommand := backuplocation.NewGetCommand(f, \"backup-locations\")\n\tbackupLocationCommand.Aliases = []string{\"backup-location\"}\n\n\tsnapshotLocationCommand := snapshotlocation.NewGetCommand(f, \"snapshot-locations\")\n\tsnapshotLocationCommand.Aliases = []string{\"snapshot-location\"}\n\n\tpluginCommand := plugin.NewGetCommand(f, \"plugins\")\n\tpluginCommand.Aliases = []string{\"plugin\"}\n\n\tc.AddCommand(\n\t\tbackupCommand,\n\t\tscheduleCommand,\n\t\trestoreCommand,\n\t\tbackupLocationCommand,\n\t\tsnapshotLocationCommand,\n\t\tpluginCommand,\n\t)\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/install/install.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage install\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/spf13/cobra\"\n\t\"github.com/spf13/pflag\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\n\t\"github.com/vmware-tanzu/velero/internal/velero\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/flag\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/output\"\n\t\"github.com/vmware-tanzu/velero/pkg/install\"\n\tvelerotypes \"github.com/vmware-tanzu/velero/pkg/types\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\tkubeutil \"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\n// Options collects all the options for installing Velero into a Kubernetes cluster.\ntype Options struct {\n\tNamespace string\n\tImage string\n\tBucketName string\n\tPrefix string\n\tProviderName string\n\tPodAnnotations flag.Map\n\tPodLabels flag.Map\n\tServiceAccountAnnotations flag.Map\n\tServiceAccountName string\n\tVeleroPodCPURequest string\n\tVeleroPodMemRequest string\n\tVeleroPodCPULimit string\n\tVeleroPodMemLimit string\n\tNodeAgentPodCPURequest string\n\tNodeAgentPodMemRequest string\n\tNodeAgentPodCPULimit string\n\tNodeAgentPodMemLimit string\n\tRestoreOnly bool\n\tSecretFile string\n\tNoSecret bool\n\tDryRun bool\n\tBackupStorageConfig flag.Map\n\tVolumeSnapshotConfig flag.Map\n\tUseNodeAgent bool\n\tUseNodeAgentWindows bool\n\tPrivilegedNodeAgent bool\n\tWait bool\n\tUseVolumeSnapshots bool\n\tDefaultRepoMaintenanceFrequency time.Duration\n\tGarbageCollectionFrequency time.Duration\n\tPodVolumeOperationTimeout time.Duration\n\tPlugins flag.StringArray\n\tNoDefaultBackupLocation bool\n\tCRDsOnly bool\n\tCACertFile string\n\tFeatures string\n\tDefaultVolumesToFsBackup bool\n\tUploaderType string\n\tDefaultSnapshotMoveData bool\n\tDisableInformerCache bool\n\tScheduleSkipImmediately bool\n\tPodResources kubeutil.PodResources\n\tKeepLatestMaintenanceJobs int\n\tBackupRepoConfigMap string\n\tRepoMaintenanceJobConfigMap string\n\tNodeAgentConfigMap string\n\tItemBlockWorkerCount int\n\tConcurrentBackups int\n\tNodeAgentDisableHostPath bool\n\tkubeletRootDir string\n\tApply bool\n\tServerPriorityClassName string\n\tNodeAgentPriorityClassName string\n}\n\n// BindFlags adds command line values to the options struct.\nfunc (o *Options) BindFlags(flags *pflag.FlagSet) {\n\tflags.StringVar(&o.ProviderName, \"provider\", o.ProviderName, \"Provider name for backup and volume storage\")\n\tflags.StringVar(&o.BucketName, \"bucket\", o.BucketName, \"Name of the object storage bucket where backups should be stored\")\n\tflags.StringVar(&o.SecretFile, \"secret-file\", o.SecretFile, \"File containing credentials for backup and volume provider. If not specified, --no-secret must be used for confirmation. Optional.\")\n\tflags.BoolVar(&o.NoSecret, \"no-secret\", o.NoSecret, \"Flag indicating if a secret should be created. Must be used as confirmation if --secret-file is not provided. Optional.\")\n\tflags.BoolVar(&o.Apply, \"apply\", o.Apply, \"Flag indicating if resources should be applied instead of created. This can be used for updating existing resources.\")\n\tflags.BoolVar(&o.NoDefaultBackupLocation, \"no-default-backup-location\", o.NoDefaultBackupLocation, \"Flag indicating if a default backup location should be created. Must be used as confirmation if --bucket or --provider are not provided. Optional.\")\n\tflags.StringVar(&o.Image, \"image\", o.Image, \"Image to use for the Velero and node agent pods. Optional.\")\n\tflags.StringVar(&o.Prefix, \"prefix\", o.Prefix, \"Prefix under which all Velero data should be stored within the bucket. Optional.\")\n\tflags.Var(&o.PodAnnotations, \"pod-annotations\", \"Annotations to add to the Velero and node agent pods. Optional. Format is key1=value1,key2=value2\")\n\tflags.Var(&o.PodLabels, \"pod-labels\", \"Labels to add to the Velero and node agent pods. Optional. Format is key1=value1,key2=value2\")\n\tflags.Var(&o.ServiceAccountAnnotations, \"sa-annotations\", \"Annotations to add to the Velero ServiceAccount. Add iam.gke.io/gcp-service-account=[GSA_NAME]@[PROJECT_NAME].iam.gserviceaccount.com for workload identity. Optional. Format is key1=value1,key2=value2\")\n\tflags.StringVar(&o.ServiceAccountName, \"service-account-name\", o.ServiceAccountName, \"ServiceAccountName to be set to the Velero and node agent pods, it should be created before the installation, and the user also needs to create the rolebinding for it.\"+\n\t\t\" Optional, if this attribute is set, the default service account 'velero' will not be created, and the flag --sa-annotations will be disregarded.\")\n\tflags.StringVar(&o.VeleroPodCPURequest, \"velero-pod-cpu-request\", o.VeleroPodCPURequest, `CPU request for Velero pod. A value of \"0\" is treated as unbounded. Optional.`)\n\tflags.StringVar(&o.VeleroPodMemRequest, \"velero-pod-mem-request\", o.VeleroPodMemRequest, `Memory request for Velero pod. A value of \"0\" is treated as unbounded. Optional.`)\n\tflags.StringVar(&o.VeleroPodCPULimit, \"velero-pod-cpu-limit\", o.VeleroPodCPULimit, `CPU limit for Velero pod. A value of \"0\" is treated as unbounded. Optional.`)\n\tflags.StringVar(&o.VeleroPodMemLimit, \"velero-pod-mem-limit\", o.VeleroPodMemLimit, `Memory limit for Velero pod. A value of \"0\" is treated as unbounded. Optional.`)\n\tflags.StringVar(&o.NodeAgentPodCPURequest, \"node-agent-pod-cpu-request\", o.NodeAgentPodCPURequest, `CPU request for node-agent pod. A value of \"0\" is treated as unbounded. Optional.`)\n\tflags.StringVar(&o.NodeAgentPodMemRequest, \"node-agent-pod-mem-request\", o.NodeAgentPodMemRequest, `Memory request for node-agent pod. A value of \"0\" is treated as unbounded. Optional.`)\n\tflags.StringVar(&o.NodeAgentPodCPULimit, \"node-agent-pod-cpu-limit\", o.NodeAgentPodCPULimit, `CPU limit for node-agent pod. A value of \"0\" is treated as unbounded. Optional.`)\n\tflags.StringVar(&o.NodeAgentPodMemLimit, \"node-agent-pod-mem-limit\", o.NodeAgentPodMemLimit, `Memory limit for node-agent pod. A value of \"0\" is treated as unbounded. Optional.`)\n\tflags.StringVar(&o.kubeletRootDir, \"kubelet-root-dir\", o.kubeletRootDir, `Kubelet root directory for the node agent. Optional.`)\n\n\tflags.Var(&o.BackupStorageConfig, \"backup-location-config\", \"Configuration to use for the backup storage location. Format is key1=value1,key2=value2\")\n\tflags.Var(&o.VolumeSnapshotConfig, \"snapshot-location-config\", \"Configuration to use for the volume snapshot location. Format is key1=value1,key2=value2\")\n\tflags.BoolVar(&o.UseVolumeSnapshots, \"use-volume-snapshots\", o.UseVolumeSnapshots, \"Whether or not to create snapshot location automatically. Set to false if you do not plan to create volume snapshots via a storage provider.\")\n\tflags.BoolVar(&o.RestoreOnly, \"restore-only\", o.RestoreOnly, \"Run the server in restore-only mode. Optional.\")\n\tflags.BoolVar(&o.DryRun, \"dry-run\", o.DryRun, \"Generate resources, but don't send them to the cluster. Use with -o. Optional.\")\n\tflags.BoolVar(&o.UseNodeAgent, \"use-node-agent\", o.UseNodeAgent, \"Create Velero node-agent daemonset. Optional. Velero node-agent hosts and associates Velero modules that need to run in one or more Linux nodes.\")\n\tflags.BoolVar(&o.UseNodeAgentWindows, \"use-node-agent-windows\", o.UseNodeAgentWindows, \"Create Velero node-agent-windows daemonset. Optional. Velero node-agent-windows hosts and associates Velero modules that need to run in one or more Windows nodes.\")\n\tflags.BoolVar(&o.PrivilegedNodeAgent, \"privileged-node-agent\", o.PrivilegedNodeAgent, \"Use privileged mode for the node agent. Optional. Required to backup block devices.\")\n\tflags.BoolVar(&o.Wait, \"wait\", o.Wait, \"Wait for Velero deployment to be ready. Optional.\")\n\tflags.DurationVar(&o.DefaultRepoMaintenanceFrequency, \"default-repo-maintain-frequency\", o.DefaultRepoMaintenanceFrequency, \"How often 'maintain' is run for backup repositories by default. Optional.\")\n\tflags.DurationVar(&o.GarbageCollectionFrequency, \"garbage-collection-frequency\", o.GarbageCollectionFrequency, \"How often the garbage collection runs for expired backups.(default 1h)\")\n\tflags.DurationVar(&o.PodVolumeOperationTimeout, \"pod-volume-operation-timeout\", o.PodVolumeOperationTimeout, \"How long to wait for pod volume operations to complete before timing out(default 4h). Optional.\")\n\tflags.Var(&o.Plugins, \"plugins\", \"Plugin container images to install into the Velero Deployment\")\n\tflags.BoolVar(&o.CRDsOnly, \"crds-only\", o.CRDsOnly, \"Only generate CustomResourceDefinition resources. Useful for updating CRDs for an existing Velero install.\")\n\tflags.StringVar(&o.CACertFile, \"cacert\", o.CACertFile, \"File containing a certificate bundle to use when verifying TLS connections to the object store. Optional.\")\n\tflags.StringVar(&o.Features, \"features\", o.Features, \"Comma separated list of Velero feature flags to be set on the Velero deployment and the node-agent daemonset, if node-agent is enabled\")\n\tflags.BoolVar(&o.DefaultVolumesToFsBackup, \"default-volumes-to-fs-backup\", o.DefaultVolumesToFsBackup, \"Bool flag to configure Velero server to use pod volume file system backup by default for all volumes on all backups. Optional.\")\n\tflags.StringVar(&o.UploaderType, \"uploader-type\", o.UploaderType, fmt.Sprintf(\"The type of uploader to transfer the data of pod volumes, supported value: '%s'\", uploader.KopiaType))\n\tflags.BoolVar(&o.DefaultSnapshotMoveData, \"default-snapshot-move-data\", o.DefaultSnapshotMoveData, \"Bool flag to configure Velero server to move data by default for all snapshots supporting data movement. Optional.\")\n\tflags.BoolVar(&o.DisableInformerCache, \"disable-informer-cache\", o.DisableInformerCache, \"Disable informer cache for Get calls on restore. With this enabled, it will speed up restore in cases where there are backup resources which already exist in the cluster, but for very large clusters this will increase velero memory usage. Default is false (don't disable). Optional.\")\n\tflags.BoolVar(&o.ScheduleSkipImmediately, \"schedule-skip-immediately\", o.ScheduleSkipImmediately, \"Skip the first scheduled backup immediately after creating a schedule. Default is false (don't skip).\")\n\tflags.BoolVar(&o.NodeAgentDisableHostPath, \"node-agent-disable-host-path\", o.NodeAgentDisableHostPath, \"Don't mount the pod volume host path to node-agent. Optional. Pod volume host path mount is required by fs-backup but could be disabled for other backup methods.\")\n\n\tflags.IntVar(\n\t\t&o.KeepLatestMaintenanceJobs,\n\t\t\"keep-latest-maintenance-jobs\",\n\t\to.KeepLatestMaintenanceJobs,\n\t\t\"Number of latest maintenance jobs to keep each repository. Optional.\",\n\t)\n\tflags.StringVar(\n\t\t&o.PodResources.CPURequest,\n\t\t\"maintenance-job-cpu-request\",\n\t\to.PodResources.CPURequest,\n\t\t\"CPU request for maintenance jobs. Default is no limit.\",\n\t)\n\tflags.StringVar(\n\t\t&o.PodResources.MemoryRequest,\n\t\t\"maintenance-job-mem-request\",\n\t\to.PodResources.MemoryRequest,\n\t\t\"Memory request for maintenance jobs. Default is no limit.\",\n\t)\n\tflags.StringVar(\n\t\t&o.PodResources.CPULimit,\n\t\t\"maintenance-job-cpu-limit\",\n\t\to.PodResources.CPULimit,\n\t\t\"CPU limit for maintenance jobs. Default is no limit.\",\n\t)\n\tflags.StringVar(\n\t\t&o.PodResources.MemoryLimit,\n\t\t\"maintenance-job-mem-limit\",\n\t\to.PodResources.MemoryLimit,\n\t\t\"Memory limit for maintenance jobs. Default is no limit.\",\n\t)\n\tflags.StringVar(\n\t\t&o.BackupRepoConfigMap,\n\t\t\"backup-repository-configmap\",\n\t\to.BackupRepoConfigMap,\n\t\t\"The name of configMap containing backup repository configurations.\",\n\t)\n\tflags.StringVar(\n\t\t&o.RepoMaintenanceJobConfigMap,\n\t\t\"repo-maintenance-job-configmap\",\n\t\to.RepoMaintenanceJobConfigMap,\n\t\t\"The name of ConfigMap containing repository maintenance Job configurations.\",\n\t)\n\tflags.StringVar(\n\t\t&o.NodeAgentConfigMap,\n\t\t\"node-agent-configmap\",\n\t\to.NodeAgentConfigMap,\n\t\t\"The name of ConfigMap containing node-agent configurations.\",\n\t)\n\tflags.IntVar(\n\t\t&o.ItemBlockWorkerCount,\n\t\t\"item-block-worker-count\",\n\t\to.ItemBlockWorkerCount,\n\t\t\"Number of worker threads to process ItemBlocks. Default is one. Optional.\",\n\t)\n\tflags.IntVar(\n\t\t&o.ConcurrentBackups,\n\t\t\"concurrent-backups\",\n\t\to.ConcurrentBackups,\n\t\t\"Number of backups to process concurrently. Default is one. Optional.\",\n\t)\n\tflags.StringVar(\n\t\t&o.ServerPriorityClassName,\n\t\t\"server-priority-class-name\",\n\t\to.ServerPriorityClassName,\n\t\t\"Priority class name for the Velero server deployment. Optional.\",\n\t)\n\tflags.StringVar(\n\t\t&o.NodeAgentPriorityClassName,\n\t\t\"node-agent-priority-class-name\",\n\t\to.NodeAgentPriorityClassName,\n\t\t\"Priority class name for the node agent daemonset. Optional.\",\n\t)\n}\n\n// NewInstallOptions instantiates a new, default InstallOptions struct.\nfunc NewInstallOptions() *Options {\n\treturn &Options{\n\t\tNamespace: velerov1api.DefaultNamespace,\n\t\tImage: velero.DefaultVeleroImage(),\n\t\tBackupStorageConfig: flag.NewMap(),\n\t\tVolumeSnapshotConfig: flag.NewMap(),\n\t\tPodAnnotations: flag.NewMap(),\n\t\tPodLabels: flag.NewMap(),\n\t\tServiceAccountAnnotations: flag.NewMap(),\n\t\tVeleroPodCPURequest: install.DefaultVeleroPodCPURequest,\n\t\tVeleroPodMemRequest: install.DefaultVeleroPodMemRequest,\n\t\tVeleroPodCPULimit: install.DefaultVeleroPodCPULimit,\n\t\tVeleroPodMemLimit: install.DefaultVeleroPodMemLimit,\n\t\tNodeAgentPodCPURequest: install.DefaultNodeAgentPodCPURequest,\n\t\tNodeAgentPodMemRequest: install.DefaultNodeAgentPodMemRequest,\n\t\tNodeAgentPodCPULimit: install.DefaultNodeAgentPodCPULimit,\n\t\tNodeAgentPodMemLimit: install.DefaultNodeAgentPodMemLimit,\n\t\t// Default to creating a VSL unless we're told otherwise\n\t\tUseVolumeSnapshots: true,\n\t\tNoDefaultBackupLocation: false,\n\t\tCRDsOnly: false,\n\t\tDefaultVolumesToFsBackup: false,\n\t\tUploaderType: uploader.KopiaType,\n\t\tDefaultSnapshotMoveData: false,\n\t\tDisableInformerCache: false,\n\t\tScheduleSkipImmediately: false,\n\t\tkubeletRootDir: install.DefaultKubeletRootDir,\n\t\tNodeAgentDisableHostPath: false,\n\t}\n}\n\n// AsVeleroOptions translates the values provided at the command line into values used to instantiate Kubernetes resources\nfunc (o *Options) AsVeleroOptions() (*install.VeleroOptions, error) {\n\tvar secretData []byte\n\tif o.SecretFile != \"\" && !o.NoSecret {\n\t\trealPath, err := filepath.Abs(o.SecretFile)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tsecretData, err = os.ReadFile(realPath)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\tvar caCertData []byte\n\tif o.CACertFile != \"\" {\n\t\trealPath, err := filepath.Abs(o.CACertFile)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tcaCertData, err = os.ReadFile(realPath)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\tveleroPodResources, err := kubeutil.ParseCPUAndMemoryResources(\n\t\to.VeleroPodCPURequest,\n\t\to.VeleroPodMemRequest,\n\t\to.VeleroPodCPULimit,\n\t\to.VeleroPodMemLimit,\n\t)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tnodeAgentPodResources, err := kubeutil.ParseCPUAndMemoryResources(\n\t\to.NodeAgentPodCPURequest,\n\t\to.NodeAgentPodMemRequest,\n\t\to.NodeAgentPodCPULimit,\n\t\to.NodeAgentPodMemLimit,\n\t)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &install.VeleroOptions{\n\t\tNamespace: o.Namespace,\n\t\tImage: o.Image,\n\t\tProviderName: o.ProviderName,\n\t\tBucket: o.BucketName,\n\t\tPrefix: o.Prefix,\n\t\tPodAnnotations: o.PodAnnotations.Data(),\n\t\tPodLabels: o.PodLabels.Data(),\n\t\tServiceAccountAnnotations: o.ServiceAccountAnnotations.Data(),\n\t\tServiceAccountName: o.ServiceAccountName,\n\t\tVeleroPodResources: veleroPodResources,\n\t\tNodeAgentPodResources: nodeAgentPodResources,\n\t\tSecretData: secretData,\n\t\tRestoreOnly: o.RestoreOnly,\n\t\tUseNodeAgent: o.UseNodeAgent,\n\t\tUseNodeAgentWindows: o.UseNodeAgentWindows,\n\t\tPrivilegedNodeAgent: o.PrivilegedNodeAgent,\n\t\tUseVolumeSnapshots: o.UseVolumeSnapshots,\n\t\tBSLConfig: o.BackupStorageConfig.Data(),\n\t\tVSLConfig: o.VolumeSnapshotConfig.Data(),\n\t\tDefaultRepoMaintenanceFrequency: o.DefaultRepoMaintenanceFrequency,\n\t\tGarbageCollectionFrequency: o.GarbageCollectionFrequency,\n\t\tPodVolumeOperationTimeout: o.PodVolumeOperationTimeout,\n\t\tPlugins: o.Plugins,\n\t\tNoDefaultBackupLocation: o.NoDefaultBackupLocation,\n\t\tCACertData: caCertData,\n\t\tFeatures: strings.Split(o.Features, \",\"),\n\t\tDefaultVolumesToFsBackup: o.DefaultVolumesToFsBackup,\n\t\tUploaderType: o.UploaderType,\n\t\tDefaultSnapshotMoveData: o.DefaultSnapshotMoveData,\n\t\tDisableInformerCache: o.DisableInformerCache,\n\t\tScheduleSkipImmediately: o.ScheduleSkipImmediately,\n\t\tPodResources: o.PodResources,\n\t\tKeepLatestMaintenanceJobs: o.KeepLatestMaintenanceJobs,\n\t\tBackupRepoConfigMap: o.BackupRepoConfigMap,\n\t\tRepoMaintenanceJobConfigMap: o.RepoMaintenanceJobConfigMap,\n\t\tNodeAgentConfigMap: o.NodeAgentConfigMap,\n\t\tItemBlockWorkerCount: o.ItemBlockWorkerCount,\n\t\tConcurrentBackups: o.ConcurrentBackups,\n\t\tKubeletRootDir: o.kubeletRootDir,\n\t\tNodeAgentDisableHostPath: o.NodeAgentDisableHostPath,\n\t\tServerPriorityClassName: o.ServerPriorityClassName,\n\t\tNodeAgentPriorityClassName: o.NodeAgentPriorityClassName,\n\t}, nil\n}\n\n// NewCommand creates a cobra command.\nfunc NewCommand(f client.Factory) *cobra.Command {\n\to := NewInstallOptions()\n\tc := &cobra.Command{\n\t\tUse: \"install\",\n\t\tShort: \"Install Velero\",\n\t\tLong: `Install Velero onto a Kubernetes cluster using the supplied provider information, such as\nthe provider's name, a bucket name, and a file containing the credentials to access that bucket.\nA prefix within the bucket and configuration for the backup store location may also be supplied.\nAdditionally, volume snapshot information for the same provider may be supplied.\n\nAll required CustomResourceDefinitions will be installed to the server, as well as the\nVelero Deployment and associated node-agent DaemonSet.\n\nThe provided secret data will be created in a Secret named 'cloud-credentials'.\n\nAll namespaced resources will be placed in the 'velero' namespace by default. \n\nThe '--namespace' flag can be used to specify a different namespace to install into.\n\nUse '--wait' to wait for the Velero Deployment to be ready before proceeding.\n\nUse '-o yaml' or '-o json' with '--dry-run' to output all generated resources as text instead of sending the resources to the server.\nThis is useful as a starting point for more customized installations.\n\t\t`,\n\t\tExample: ` # velero install --provider gcp --plugins velero/velero-plugin-for-gcp:v1.0.0 --bucket mybucket --secret-file ./gcp-service-account.json\n\n # velero install --provider aws --plugins velero/velero-plugin-for-aws:v1.0.0 --bucket backups --secret-file ./aws-iam-creds --backup-location-config region=us-east-2 --snapshot-location-config region=us-east-2\n\n # velero install --provider aws --plugins velero/velero-plugin-for-aws:v1.0.0 --bucket backups --secret-file ./aws-iam-creds --backup-location-config region=us-east-2 --snapshot-location-config region=us-east-2 --use-node-agent\n\n # velero install --provider gcp --plugins velero/velero-plugin-for-gcp:v1.0.0 --bucket gcp-backups --secret-file ./gcp-creds.json --wait\n\n # velero install --provider aws --plugins velero/velero-plugin-for-aws:v1.0.0 --bucket backups --backup-location-config region=us-west-2 --snapshot-location-config region=us-west-2 --no-secret --pod-annotations iam.amazonaws.com/role=arn:aws:iam:::role/\n\n # velero install --provider gcp --plugins velero/velero-plugin-for-gcp:v1.0.0 --bucket gcp-backups --secret-file ./gcp-creds.json --velero-pod-cpu-request=1000m --velero-pod-cpu-limit=5000m --velero-pod-mem-request=512Mi --velero-pod-mem-limit=1024Mi\n\n # velero install --provider gcp --plugins velero/velero-plugin-for-gcp:v1.0.0 --bucket gcp-backups --secret-file ./gcp-creds.json --node-agent-pod-cpu-request=1000m --node-agent-pod-cpu-limit=5000m --node-agent-pod-mem-request=512Mi --node-agent-pod-mem-limit=1024Mi\n\n # velero install --provider azure --plugins velero/velero-plugin-for-microsoft-azure:v1.0.0 --bucket $BLOB_CONTAINER --secret-file ./credentials-velero --backup-location-config resourceGroup=$AZURE_BACKUP_RESOURCE_GROUP,storageAccount=$AZURE_STORAGE_ACCOUNT_ID[,subscriptionId=$AZURE_BACKUP_SUBSCRIPTION_ID] --snapshot-location-config apiTimeout=[,resourceGroup=$AZURE_BACKUP_RESOURCE_GROUP,subscriptionId=$AZURE_BACKUP_SUBSCRIPTION_ID]`,\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tcmd.CheckError(o.Validate(c, args, f))\n\t\t\tcmd.CheckError(o.Complete(args, f))\n\t\t\tcmd.CheckError(o.Run(c, f))\n\t\t},\n\t}\n\n\to.BindFlags(c.Flags())\n\toutput.BindFlags(c.Flags())\n\toutput.ClearOutputFlagDefault(c)\n\n\treturn c\n}\n\n// Run executes a command in the context of the provided arguments.\nfunc (o *Options) Run(c *cobra.Command, f client.Factory) error {\n\tvar resources *unstructured.UnstructuredList\n\tif o.CRDsOnly {\n\t\tresources = install.AllCRDs()\n\t} else {\n\t\tvo, err := o.AsVeleroOptions()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tresources = install.AllResources(vo)\n\t}\n\n\tif _, err := output.PrintWithFormat(c, resources); err != nil {\n\t\treturn err\n\t}\n\n\tif o.DryRun {\n\t\treturn nil\n\t}\n\tdynamicClient, err := f.DynamicClient()\n\tif err != nil {\n\t\treturn err\n\t}\n\tdynamicFactory := client.NewDynamicFactory(dynamicClient)\n\n\tkbClient, err := f.KubebuilderClient()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\terrorMsg := fmt.Sprintf(\"\\n\\nError installing Velero. Use `kubectl logs deploy/velero -n %s` to check the deploy logs\", o.Namespace)\n\n\terr = install.Install(dynamicFactory, kbClient, resources, os.Stdout, o.Apply)\n\tif err != nil {\n\t\treturn errors.Wrap(err, errorMsg)\n\t}\n\n\tif o.Wait {\n\t\tfmt.Println(\"Waiting for Velero deployment to be ready.\")\n\t\tif _, err = install.DeploymentIsReady(dynamicFactory, o.Namespace); err != nil {\n\t\t\treturn errors.Wrap(err, errorMsg)\n\t\t}\n\n\t\tif o.UseNodeAgent {\n\t\t\tfmt.Println(\"Waiting for node-agent daemonset to be ready.\")\n\t\t\tif _, err = install.NodeAgentIsReady(dynamicFactory, o.Namespace); err != nil {\n\t\t\t\treturn errors.Wrap(err, errorMsg)\n\t\t\t}\n\t\t}\n\n\t\tif o.UseNodeAgentWindows {\n\t\t\tfmt.Println(\"Waiting for node-agent-windows daemonset to be ready.\")\n\t\t\tif _, err = install.NodeAgentWindowsIsReady(dynamicFactory, o.Namespace); err != nil {\n\t\t\t\treturn errors.Wrap(err, errorMsg)\n\t\t\t}\n\t\t}\n\t}\n\tif o.SecretFile == \"\" {\n\t\tfmt.Printf(\"\\nNo secret file was specified, no Secret created.\\n\\n\")\n\t}\n\n\tif o.NoDefaultBackupLocation {\n\t\tfmt.Printf(\"\\nNo bucket and provider were specified, no default backup storage location created.\\n\\n\")\n\t}\n\n\tfmt.Printf(\"Velero is installed! ⛵ Use 'kubectl logs deployment/velero -n %s' to view the status.\\n\", o.Namespace)\n\treturn nil\n}\n\n// Complete completes options for a command.\nfunc (o *Options) Complete(args []string, f client.Factory) error {\n\to.Namespace = f.Namespace()\n\treturn nil\n}\n\n// Validate validates options provided to a command.\nfunc (o *Options) Validate(c *cobra.Command, args []string, f client.Factory) error {\n\tif err := output.ValidateFlags(c); err != nil {\n\t\treturn err\n\t}\n\n\t// If we're only installing CRDs, we can skip the rest of the validation.\n\tif o.CRDsOnly {\n\t\treturn nil\n\t}\n\n\tif msg, err := uploader.ValidateUploaderType(o.UploaderType); err != nil {\n\t\treturn err\n\t} else if msg != \"\" {\n\t\tfmt.Printf(\"⚠️ %s\\n\", msg)\n\t}\n\n\t// Our main 3 providers don't support bucket names starting with a dash, and a bucket name starting with one\n\t// can indicate that an environment variable was left blank.\n\t// This case will help catch that error\n\tif strings.HasPrefix(o.BucketName, \"-\") {\n\t\treturn errors.Errorf(\"Bucket names cannot begin with a dash. Bucket name was: %s\", o.BucketName)\n\t}\n\n\tif o.NoDefaultBackupLocation {\n\t\tif o.BucketName != \"\" {\n\t\t\treturn errors.New(\"Cannot use both --bucket and --no-default-backup-location at the same time\")\n\t\t}\n\n\t\tif o.Prefix != \"\" {\n\t\t\treturn errors.New(\"Cannot use both --prefix and --no-default-backup-location at the same time\")\n\t\t}\n\n\t\tif o.BackupStorageConfig.String() != \"\" {\n\t\t\treturn errors.New(\"Cannot use both --backup-location-config and --no-default-backup-location at the same time\")\n\t\t}\n\t} else {\n\t\tif o.ProviderName == \"\" {\n\t\t\treturn errors.New(\"--provider is required\")\n\t\t}\n\n\t\tif o.BucketName == \"\" {\n\t\t\treturn errors.New(\"--bucket is required\")\n\t\t}\n\t}\n\n\tif o.UseVolumeSnapshots {\n\t\tif o.ProviderName == \"\" {\n\t\t\treturn errors.New(\"--provider is required when --use-volume-snapshots is set to true\")\n\t\t}\n\t} else {\n\t\tif o.VolumeSnapshotConfig.String() != \"\" {\n\t\t\treturn errors.New(\"--snapshot-location-config must be empty when --use-volume-snapshots=false\")\n\t\t}\n\t}\n\n\tif o.NoDefaultBackupLocation && !o.UseVolumeSnapshots {\n\t\tif o.ProviderName != \"\" {\n\t\t\treturn errors.New(\"--provider must be empty when using --no-default-backup-location and --use-volume-snapshots=false\")\n\t\t}\n\t} else {\n\t\tif len(o.Plugins) == 0 {\n\t\t\treturn errors.New(\"--plugins flag is required\")\n\t\t}\n\t}\n\n\tif o.DefaultVolumesToFsBackup && !o.UseNodeAgent {\n\t\treturn errors.New(\"--use-node-agent is required when using --default-volumes-to-fs-backup\")\n\t}\n\n\tswitch {\n\tcase o.SecretFile == \"\" && !o.NoSecret:\n\t\treturn errors.New(\"One of --secret-file or --no-secret is required\")\n\tcase o.SecretFile != \"\" && o.NoSecret:\n\t\treturn errors.New(\"Cannot use both --secret-file and --no-secret\")\n\t}\n\n\tif o.DefaultRepoMaintenanceFrequency < 0 {\n\t\treturn errors.New(\"--default-repo-maintain-frequency must be non-negative\")\n\t}\n\n\tif o.GarbageCollectionFrequency < 0 {\n\t\treturn errors.New(\"--garbage-collection-frequency must be non-negative\")\n\t}\n\n\tif o.PodVolumeOperationTimeout < 0 {\n\t\treturn errors.New(\"--pod-volume-operation-timeout must be non-negative\")\n\t}\n\n\tcrClient, err := f.KubebuilderClient()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"fail to create go-client %w\", err)\n\t}\n\n\tif len(o.NodeAgentConfigMap) > 0 {\n\t\tif err := kubeutil.VerifyJSONConfigs(c.Context(), o.Namespace, crClient, o.NodeAgentConfigMap, &velerotypes.NodeAgentConfigs{}); err != nil {\n\t\t\treturn fmt.Errorf(\"--node-agent-configmap specified ConfigMap %s is invalid: %w\", o.NodeAgentConfigMap, err)\n\t\t}\n\t}\n\n\tif len(o.RepoMaintenanceJobConfigMap) > 0 {\n\t\tif err := kubeutil.VerifyJSONConfigs(c.Context(), o.Namespace, crClient, o.RepoMaintenanceJobConfigMap, &velerotypes.JobConfigs{}); err != nil {\n\t\t\treturn fmt.Errorf(\"--repo-maintenance-job-configmap specified ConfigMap %s is invalid: %w\", o.RepoMaintenanceJobConfigMap, err)\n\t\t}\n\t}\n\n\tif len(o.BackupRepoConfigMap) > 0 {\n\t\tconfig := make(map[string]any)\n\t\tif err := kubeutil.VerifyJSONConfigs(c.Context(), o.Namespace, crClient, o.BackupRepoConfigMap, &config); err != nil {\n\t\t\treturn fmt.Errorf(\"--backup-repository-configmap specified ConfigMap %s is invalid: %w\", o.BackupRepoConfigMap, err)\n\t\t}\n\t}\n\n\treturn nil\n}\n" }, { "path": "pkg/cmd/cli/install/install_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage install\n\nimport (\n\t\"testing\"\n\n\t\"github.com/spf13/pflag\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestPriorityClassNameFlag(t *testing.T) {\n\t// Test that the flag is properly defined\n\to := NewInstallOptions()\n\tflags := pflag.NewFlagSet(\"test\", pflag.ContinueOnError)\n\to.BindFlags(flags)\n\n\t// Verify the server priority class flag exists\n\tserverFlag := flags.Lookup(\"server-priority-class-name\")\n\tassert.NotNil(t, serverFlag, \"server-priority-class-name flag should exist\")\n\tassert.Equal(t, \"Priority class name for the Velero server deployment. Optional.\", serverFlag.Usage)\n\n\t// Verify the node agent priority class flag exists\n\tnodeAgentFlag := flags.Lookup(\"node-agent-priority-class-name\")\n\tassert.NotNil(t, nodeAgentFlag, \"node-agent-priority-class-name flag should exist\")\n\tassert.Equal(t, \"Priority class name for the node agent daemonset. Optional.\", nodeAgentFlag.Usage)\n\n\t// Test with values for both server and node agent\n\ttestCases := []struct {\n\t\tname string\n\t\tserverPriorityClassName string\n\t\tnodeAgentPriorityClassName string\n\t\texpectedServerValue string\n\t\texpectedNodeAgentValue string\n\t}{\n\t\t{\n\t\t\tname: \"with both priority class names\",\n\t\t\tserverPriorityClassName: \"high-priority\",\n\t\t\tnodeAgentPriorityClassName: \"medium-priority\",\n\t\t\texpectedServerValue: \"high-priority\",\n\t\t\texpectedNodeAgentValue: \"medium-priority\",\n\t\t},\n\t\t{\n\t\t\tname: \"with only server priority class name\",\n\t\t\tserverPriorityClassName: \"high-priority\",\n\t\t\tnodeAgentPriorityClassName: \"\",\n\t\t\texpectedServerValue: \"high-priority\",\n\t\t\texpectedNodeAgentValue: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"with only node agent priority class name\",\n\t\t\tserverPriorityClassName: \"\",\n\t\t\tnodeAgentPriorityClassName: \"medium-priority\",\n\t\t\texpectedServerValue: \"\",\n\t\t\texpectedNodeAgentValue: \"medium-priority\",\n\t\t},\n\t\t{\n\t\t\tname: \"without priority class names\",\n\t\t\tserverPriorityClassName: \"\",\n\t\t\tnodeAgentPriorityClassName: \"\",\n\t\t\texpectedServerValue: \"\",\n\t\t\texpectedNodeAgentValue: \"\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\to := NewInstallOptions()\n\t\t\to.ServerPriorityClassName = tc.serverPriorityClassName\n\t\t\to.NodeAgentPriorityClassName = tc.nodeAgentPriorityClassName\n\n\t\t\tveleroOptions, err := o.AsVeleroOptions()\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Equal(t, tc.expectedServerValue, veleroOptions.ServerPriorityClassName)\n\t\t\tassert.Equal(t, tc.expectedNodeAgentValue, veleroOptions.NodeAgentPriorityClassName)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/cmd/cli/nodeagent/node_agent.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage nodeagent\n\nimport (\n\t\"github.com/spf13/cobra\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n)\n\nfunc NewCommand(f client.Factory) *cobra.Command {\n\tc := &cobra.Command{\n\t\tUse: \"node-agent\",\n\t\tShort: \"Work with node-agent\",\n\t\tLong: \"Work with node-agent\",\n\t}\n\n\tc.AddCommand(\n\t\tNewServerCommand(f),\n\t)\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/nodeagent/server.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage nodeagent\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"math\"\n\t\"net/http\"\n\t\"os\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/bombsimon/logrusr/v3\"\n\tsnapshotv1client \"github.com/kubernetes-csi/external-snapshotter/client/v8/clientset/versioned\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/prometheus/client_golang/prometheus/promhttp\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/spf13/cobra\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tstoragev1api \"k8s.io/api/storage/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/fields\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\t\"k8s.io/client-go/kubernetes\"\n\tcacheutil \"k8s.io/client-go/tools/cache\"\n\t\"k8s.io/klog/v2\"\n\t\"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/cache\"\n\tctrlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/manager\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/buildinfo\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/signals\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/controller\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\t\"github.com/vmware-tanzu/velero/pkg/exposer\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\t\"github.com/vmware-tanzu/velero/pkg/nodeagent\"\n\trepository \"github.com/vmware-tanzu/velero/pkg/repository/manager\"\n\tvelerotypes \"github.com/vmware-tanzu/velero/pkg/types\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n)\n\nvar (\n\tscheme = runtime.NewScheme()\n)\n\nconst (\n\t// the port where prometheus metrics are exposed\n\tdefaultMetricsAddress = \":8085\"\n\n\tdefaultResourceTimeout = 10 * time.Minute\n\tdefaultDataMoverPrepareTimeout = 30 * time.Minute\n\tdefaultDataPathConcurrentNum = 1\n)\n\ntype nodeAgentServerConfig struct {\n\tmetricsAddress string\n\tresourceTimeout time.Duration\n\tdataMoverPrepareTimeout time.Duration\n\tnodeAgentConfig string\n\tbackupRepoConfig string\n}\n\nfunc NewServerCommand(f client.Factory) *cobra.Command {\n\tlogLevelFlag := logging.LogLevelFlag(logrus.InfoLevel)\n\tformatFlag := logging.NewFormatFlag()\n\tconfig := nodeAgentServerConfig{\n\t\tmetricsAddress: defaultMetricsAddress,\n\t\tresourceTimeout: defaultResourceTimeout,\n\t\tdataMoverPrepareTimeout: defaultDataMoverPrepareTimeout,\n\t}\n\n\tcommand := &cobra.Command{\n\t\tUse: \"server\",\n\t\tShort: \"Run the velero node-agent server\",\n\t\tLong: \"Run the velero node-agent server\",\n\t\tHidden: true,\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tlogLevel := logLevelFlag.Parse()\n\t\t\tlogrus.Infof(\"Setting log-level to %s\", strings.ToUpper(logLevel.String()))\n\n\t\t\tlogger := logging.DefaultMergeLogger(logLevel, formatFlag.Parse())\n\t\t\tlogger.Infof(\"Starting Velero node-agent server %s (%s)\", buildinfo.Version, buildinfo.FormattedGitSHA())\n\n\t\t\tf.SetBasename(fmt.Sprintf(\"%s-%s\", c.Parent().Name(), c.Name()))\n\t\t\ts, err := newNodeAgentServer(logger, f, config)\n\t\t\tcmd.CheckError(err)\n\n\t\t\ts.run()\n\t\t},\n\t}\n\n\tcommand.Flags().Var(logLevelFlag, \"log-level\", fmt.Sprintf(\"The level at which to log. Valid values are %s.\", strings.Join(logLevelFlag.AllowedValues(), \", \")))\n\tcommand.Flags().Var(formatFlag, \"log-format\", fmt.Sprintf(\"The format for log output. Valid values are %s.\", strings.Join(formatFlag.AllowedValues(), \", \")))\n\tcommand.Flags().DurationVar(&config.resourceTimeout, \"resource-timeout\", config.resourceTimeout, \"How long to wait for resource processes which are not covered by other specific timeout parameters. Default is 10 minutes.\")\n\tcommand.Flags().DurationVar(&config.dataMoverPrepareTimeout, \"data-mover-prepare-timeout\", config.dataMoverPrepareTimeout, \"How long to wait for preparing a DataUpload/DataDownload. Default is 30 minutes.\")\n\tcommand.Flags().StringVar(&config.metricsAddress, \"metrics-address\", config.metricsAddress, \"The address to expose prometheus metrics\")\n\tcommand.Flags().StringVar(&config.nodeAgentConfig, \"node-agent-configmap\", config.nodeAgentConfig, \"The name of ConfigMap containing node-agent configurations.\")\n\tcommand.Flags().StringVar(&config.backupRepoConfig, \"backup-repository-configmap\", config.backupRepoConfig, \"The name of ConfigMap containing backup repository configurations.\")\n\n\treturn command\n}\n\ntype nodeAgentServer struct {\n\tlogger logrus.FieldLogger\n\tctx context.Context\n\tcancelFunc context.CancelFunc\n\tfileSystem filesystem.Interface\n\tmgr manager.Manager\n\tmetrics *metrics.ServerMetrics\n\tmetricsAddress string\n\tnamespace string\n\tnodeName string\n\tconfig nodeAgentServerConfig\n\tkubeClient kubernetes.Interface\n\tcsiSnapshotClient *snapshotv1client.Clientset\n\tdataPathMgr *datapath.Manager\n\tdataPathConfigs *velerotypes.NodeAgentConfigs\n\tbackupRepoConfigs map[string]string\n\tvgdpCounter *exposer.VgdpCounter\n\trepoConfigMgr repository.ConfigManager\n}\n\nfunc newNodeAgentServer(logger logrus.FieldLogger, factory client.Factory, config nodeAgentServerConfig) (*nodeAgentServer, error) {\n\tctx, cancelFunc := context.WithCancel(context.Background())\n\n\tclientConfig, err := factory.ClientConfig()\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, err\n\t}\n\n\tctrl.SetLogger(logrusr.New(logger))\n\tklog.SetLogger(logrusr.New(logger)) // klog.Logger is used by k8s.io/client-go\n\n\tif err := velerov1api.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, err\n\t}\n\tif err := velerov2alpha1api.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, err\n\t}\n\tif err := corev1api.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, err\n\t}\n\tif err := storagev1api.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, err\n\t}\n\n\tnodeName := os.Getenv(\"NODE_NAME\")\n\n\t// use a field selector to filter to only pods scheduled on this node.\n\tcacheOption := cache.Options{\n\t\tByObject: map[ctrlclient.Object]cache.ByObject{\n\t\t\t&corev1api.Pod{}: {\n\t\t\t\tField: fields.Set{\"spec.nodeName\": nodeName}.AsSelector(),\n\t\t\t},\n\t\t\t&velerov1api.PodVolumeBackup{}: {\n\t\t\t\tField: fields.Set{\"metadata.namespace\": factory.Namespace()}.AsSelector(),\n\t\t\t},\n\t\t\t&velerov1api.PodVolumeRestore{}: {\n\t\t\t\tField: fields.Set{\"metadata.namespace\": factory.Namespace()}.AsSelector(),\n\t\t\t},\n\t\t\t&velerov2alpha1api.DataUpload{}: {\n\t\t\t\tField: fields.Set{\"metadata.namespace\": factory.Namespace()}.AsSelector(),\n\t\t\t},\n\t\t\t&velerov2alpha1api.DataDownload{}: {\n\t\t\t\tField: fields.Set{\"metadata.namespace\": factory.Namespace()}.AsSelector(),\n\t\t\t},\n\t\t\t&corev1api.Event{}: {\n\t\t\t\tField: fields.Set{\"metadata.namespace\": factory.Namespace()}.AsSelector(),\n\t\t\t},\n\t\t},\n\t}\n\n\tvar mgr manager.Manager\n\tretry := 10\n\tfor {\n\t\tmgr, err = ctrl.NewManager(clientConfig, ctrl.Options{\n\t\t\tScheme: scheme,\n\t\t\tCache: cacheOption,\n\t\t})\n\t\tif err == nil {\n\t\t\tbreak\n\t\t}\n\n\t\tretry--\n\t\tif retry == 0 {\n\t\t\tbreak\n\t\t}\n\n\t\tlogger.WithError(err).Warn(\"Failed to create controller manager, need retry\")\n\n\t\ttime.Sleep(time.Second)\n\t}\n\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error creating controller manager\")\n\t}\n\n\ts := &nodeAgentServer{\n\t\tlogger: logger,\n\t\tctx: ctx,\n\t\tcancelFunc: cancelFunc,\n\t\tfileSystem: filesystem.NewFileSystem(),\n\t\tmgr: mgr,\n\t\tconfig: config,\n\t\tnamespace: factory.Namespace(),\n\t\tnodeName: nodeName,\n\t\tmetricsAddress: config.metricsAddress,\n\t\trepoConfigMgr: repository.NewConfigManager(logger),\n\t}\n\n\t// the cache isn't initialized yet when \"validatePodVolumesHostPath\" is called, the client returned by the manager cannot\n\t// be used, so we need the kube client here\n\ts.kubeClient, err = factory.KubeClient()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif err := s.validatePodVolumesHostPath(s.kubeClient); err != nil {\n\t\treturn nil, err\n\t}\n\n\ts.csiSnapshotClient, err = snapshotv1client.NewForConfig(clientConfig)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := s.getDataPathConfigs(); err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := s.getBackupRepoConfigs(); err != nil {\n\t\treturn nil, err\n\t}\n\n\ts.dataPathMgr = datapath.NewManager(s.getDataPathConcurrentNum(defaultDataPathConcurrentNum))\n\n\treturn s, nil\n}\n\nfunc (s *nodeAgentServer) run() {\n\tsignals.CancelOnShutdown(s.cancelFunc, s.logger)\n\n\tgo func() {\n\t\tmetricsMux := http.NewServeMux()\n\t\tmetricsMux.Handle(\"/metrics\", promhttp.Handler())\n\t\ts.logger.Infof(\"Starting metric server for node agent at address [%s]\", s.metricsAddress)\n\t\tserver := &http.Server{\n\t\t\tAddr: s.metricsAddress,\n\t\t\tHandler: metricsMux,\n\t\t\tReadHeaderTimeout: 3 * time.Second,\n\t\t}\n\t\tif err := server.ListenAndServe(); err != nil {\n\t\t\ts.logger.Fatalf(\"Failed to start metric server for node agent at [%s]: %v\", s.metricsAddress, err)\n\t\t}\n\t}()\n\ts.metrics = metrics.NewNodeMetrics()\n\ts.metrics.RegisterAllMetrics()\n\ts.metrics.InitMetricsForNode(s.nodeName)\n\n\ts.logger.Info(\"Starting controllers\")\n\n\t// Get priority class from dataPathConfigs if available\n\tdataMovePriorityClass := \"\"\n\tif s.dataPathConfigs != nil && s.dataPathConfigs.PriorityClassName != \"\" {\n\t\tpriorityClass := s.dataPathConfigs.PriorityClassName\n\t\t// Validate the priority class exists in the cluster\n\t\tctx, cancel := context.WithTimeout(s.ctx, time.Second*30)\n\t\tdefer cancel()\n\t\tif kube.ValidatePriorityClass(ctx, s.kubeClient, priorityClass, s.logger.WithField(\"component\", \"data-mover\")) {\n\t\t\tdataMovePriorityClass = priorityClass\n\t\t\ts.logger.WithField(\"priorityClassName\", priorityClass).Info(\"Using priority class for data mover pods\")\n\t\t} else {\n\t\t\ts.logger.WithField(\"priorityClassName\", priorityClass).Warn(\"Priority class not found in cluster, data mover pods will use default priority\")\n\t\t}\n\t}\n\n\tvar loadAffinity []*kube.LoadAffinity\n\tif s.dataPathConfigs != nil && len(s.dataPathConfigs.LoadAffinity) > 0 {\n\t\tloadAffinity = s.dataPathConfigs.LoadAffinity\n\t\ts.logger.Infof(\"Using customized loadAffinity %v\", loadAffinity)\n\t}\n\n\tvar backupPVCConfig map[string]velerotypes.BackupPVC\n\tif s.dataPathConfigs != nil && s.dataPathConfigs.BackupPVCConfig != nil {\n\t\tbackupPVCConfig = s.dataPathConfigs.BackupPVCConfig\n\t\ts.logger.Infof(\"Using customized backupPVC config %v\", backupPVCConfig)\n\t}\n\n\tprivilegedFsBackup := s.dataPathConfigs != nil && s.dataPathConfigs.PrivilegedFsBackup\n\n\tpodResources := corev1api.ResourceRequirements{}\n\tif s.dataPathConfigs != nil && s.dataPathConfigs.PodResources != nil {\n\t\t// To make the PodResources ConfigMap without ephemeral storage request/limit backward compatible,\n\t\t// need to avoid set value as empty, because empty string will cause parsing error.\n\t\tephemeralStorageRequest := constant.DefaultEphemeralStorageRequest\n\t\tif s.dataPathConfigs.PodResources.EphemeralStorageRequest != \"\" {\n\t\t\tephemeralStorageRequest = s.dataPathConfigs.PodResources.EphemeralStorageRequest\n\t\t}\n\t\tephemeralStorageLimit := constant.DefaultEphemeralStorageLimit\n\t\tif s.dataPathConfigs.PodResources.EphemeralStorageLimit != \"\" {\n\t\t\tephemeralStorageLimit = s.dataPathConfigs.PodResources.EphemeralStorageLimit\n\t\t}\n\n\t\tif res, err := kube.ParseResourceRequirements(\n\t\t\ts.dataPathConfigs.PodResources.CPURequest,\n\t\t\ts.dataPathConfigs.PodResources.MemoryRequest,\n\t\t\tephemeralStorageRequest,\n\t\t\ts.dataPathConfigs.PodResources.CPULimit,\n\t\t\ts.dataPathConfigs.PodResources.MemoryLimit,\n\t\t\tephemeralStorageLimit,\n\t\t); err != nil {\n\t\t\ts.logger.WithError(err).Warn(\"Pod resource requirements are invalid, ignore\")\n\t\t} else {\n\t\t\tpodResources = res\n\t\t\ts.logger.Infof(\"Using customized pod resource requirements %v\", s.dataPathConfigs.PodResources)\n\t\t}\n\t}\n\n\tif s.dataPathConfigs != nil && s.dataPathConfigs.LoadConcurrency != nil && s.dataPathConfigs.LoadConcurrency.PrepareQueueLength > 0 {\n\t\tif counter, err := exposer.StartVgdpCounter(s.ctx, s.mgr, s.dataPathConfigs.LoadConcurrency.PrepareQueueLength); err != nil {\n\t\t\ts.logger.WithError(err).Warnf(\"Failed to start VGDP counter, VDGP loads are not constrained\")\n\t\t} else {\n\t\t\ts.vgdpCounter = counter\n\t\t\ts.logger.Infof(\"VGDP loads are constrained with %d\", s.dataPathConfigs.LoadConcurrency.PrepareQueueLength)\n\t\t}\n\t}\n\n\tvar cachePVCConfig *velerotypes.CachePVC\n\tif s.dataPathConfigs != nil && s.dataPathConfigs.CachePVCConfig != nil {\n\t\tif err := s.validateCachePVCConfig(*s.dataPathConfigs.CachePVCConfig); err != nil {\n\t\t\ts.logger.WithError(err).Warnf(\"Ignore cache config %v\", s.dataPathConfigs.CachePVCConfig)\n\t\t} else {\n\t\t\tcachePVCConfig = s.dataPathConfigs.CachePVCConfig\n\t\t\ts.logger.Infof(\"Using cache volume configs %v\", s.dataPathConfigs.CachePVCConfig)\n\t\t}\n\t}\n\n\tvar podLabels map[string]string\n\tif s.dataPathConfigs != nil && len(s.dataPathConfigs.PodLabels) > 0 {\n\t\tpodLabels = s.dataPathConfigs.PodLabels\n\t\ts.logger.Infof(\"Using customized pod labels %+v\", podLabels)\n\t}\n\n\tvar podAnnotations map[string]string\n\tif s.dataPathConfigs != nil && len(s.dataPathConfigs.PodAnnotations) > 0 {\n\t\tpodAnnotations = s.dataPathConfigs.PodAnnotations\n\t\ts.logger.Infof(\"Using customized pod annotations %+v\", podAnnotations)\n\t}\n\n\tif s.backupRepoConfigs != nil {\n\t\ts.logger.Infof(\"Using backup repo config %v\", s.backupRepoConfigs)\n\t} else if cachePVCConfig != nil {\n\t\ts.logger.Info(\"Backup repo config is not provided, using default values for cache volume configs\")\n\t}\n\n\tpvbReconciler := controller.NewPodVolumeBackupReconciler(\n\t\ts.mgr.GetClient(),\n\t\ts.mgr,\n\t\ts.kubeClient,\n\t\ts.dataPathMgr,\n\t\ts.vgdpCounter,\n\t\ts.nodeName,\n\t\ts.config.dataMoverPrepareTimeout,\n\t\ts.config.resourceTimeout,\n\t\tpodResources,\n\t\ts.metrics,\n\t\ts.logger,\n\t\tdataMovePriorityClass,\n\t\tprivilegedFsBackup,\n\t\tpodLabels,\n\t\tpodAnnotations,\n\t)\n\tif err := pvbReconciler.SetupWithManager(s.mgr); err != nil {\n\t\ts.logger.Fatal(err, \"unable to create controller\", \"controller\", constant.ControllerPodVolumeBackup)\n\t}\n\n\tpvrReconciler := controller.NewPodVolumeRestoreReconciler(\n\t\ts.mgr.GetClient(),\n\t\ts.mgr,\n\t\ts.kubeClient,\n\t\ts.dataPathMgr,\n\t\ts.vgdpCounter,\n\t\ts.nodeName,\n\t\ts.config.dataMoverPrepareTimeout,\n\t\ts.config.resourceTimeout,\n\t\ts.backupRepoConfigs,\n\t\tcachePVCConfig,\n\t\tpodResources,\n\t\ts.logger,\n\t\tdataMovePriorityClass,\n\t\tprivilegedFsBackup,\n\t\ts.repoConfigMgr,\n\t\tpodLabels,\n\t\tpodAnnotations,\n\t)\n\tif err := pvrReconciler.SetupWithManager(s.mgr); err != nil {\n\t\ts.logger.WithError(err).Fatal(\"Unable to create the pod volume restore controller\")\n\t}\n\n\tif err := controller.InitLegacyPodVolumeRestoreReconciler(s.mgr.GetClient(), s.mgr, s.kubeClient, s.dataPathMgr, s.namespace, s.config.resourceTimeout, s.logger); err != nil {\n\t\ts.logger.WithError(err).Fatal(\"Unable to create the legacy pod volume restore controller\")\n\t}\n\n\tdataUploadReconciler := controller.NewDataUploadReconciler(\n\t\ts.mgr.GetClient(),\n\t\ts.mgr,\n\t\ts.kubeClient,\n\t\ts.csiSnapshotClient.SnapshotV1(),\n\t\ts.dataPathMgr,\n\t\ts.vgdpCounter,\n\t\tloadAffinity,\n\t\tbackupPVCConfig,\n\t\tpodResources,\n\t\tclock.RealClock{},\n\t\ts.nodeName,\n\t\ts.config.dataMoverPrepareTimeout,\n\t\ts.logger,\n\t\ts.metrics,\n\t\tdataMovePriorityClass,\n\t\tpodLabels,\n\t\tpodAnnotations,\n\t)\n\tif err := dataUploadReconciler.SetupWithManager(s.mgr); err != nil {\n\t\ts.logger.WithError(err).Fatal(\"Unable to create the data upload controller\")\n\t}\n\n\tvar restorePVCConfig velerotypes.RestorePVC\n\tif s.dataPathConfigs != nil && s.dataPathConfigs.RestorePVCConfig != nil {\n\t\trestorePVCConfig = *s.dataPathConfigs.RestorePVCConfig\n\t\ts.logger.Infof(\"Using customized restorePVC config %v\", restorePVCConfig)\n\t}\n\n\tdataDownloadReconciler := controller.NewDataDownloadReconciler(\n\t\ts.mgr.GetClient(),\n\t\ts.mgr,\n\t\ts.kubeClient,\n\t\ts.dataPathMgr,\n\t\ts.vgdpCounter,\n\t\tloadAffinity,\n\t\trestorePVCConfig,\n\t\ts.backupRepoConfigs,\n\t\tcachePVCConfig,\n\t\tpodResources,\n\t\ts.nodeName,\n\t\ts.config.dataMoverPrepareTimeout,\n\t\ts.logger,\n\t\ts.metrics,\n\t\tdataMovePriorityClass,\n\t\ts.repoConfigMgr,\n\t\tpodLabels,\n\t\tpodAnnotations,\n\t)\n\n\tif err := dataDownloadReconciler.SetupWithManager(s.mgr); err != nil {\n\t\ts.logger.WithError(err).Fatal(\"Unable to create the data download controller\")\n\t}\n\n\tgo func() {\n\t\tif err := s.waitCacheForResume(); err != nil {\n\t\t\ts.logger.WithError(err).Error(\"Failed to wait cache for resume, will not resume DU/DD\")\n\t\t\treturn\n\t\t}\n\n\t\tif err := dataUploadReconciler.AttemptDataUploadResume(s.ctx, s.logger.WithField(\"node\", s.nodeName), s.namespace); err != nil {\n\t\t\ts.logger.WithError(errors.WithStack(err)).Error(\"Failed to attempt data upload resume\")\n\t\t}\n\n\t\tif err := dataDownloadReconciler.AttemptDataDownloadResume(s.ctx, s.logger.WithField(\"node\", s.nodeName), s.namespace); err != nil {\n\t\t\ts.logger.WithError(errors.WithStack(err)).Error(\"Failed to attempt data download resume\")\n\t\t}\n\n\t\tif err := pvbReconciler.AttemptPVBResume(s.ctx, s.logger.WithField(\"node\", s.nodeName), s.namespace); err != nil {\n\t\t\ts.logger.WithError(errors.WithStack(err)).Error(\"Failed to attempt PVB resume\")\n\t\t}\n\n\t\tif err := pvrReconciler.AttemptPVRResume(s.ctx, s.logger.WithField(\"node\", s.nodeName), s.namespace); err != nil {\n\t\t\ts.logger.WithError(errors.WithStack(err)).Error(\"Failed to attempt PVR resume\")\n\t\t}\n\n\t\ts.markLegacyPVRsFailed(s.mgr.GetClient())\n\t}()\n\n\ts.logger.Info(\"Controllers starting...\")\n\n\tif err := s.mgr.Start(ctrl.SetupSignalHandler()); err != nil {\n\t\ts.logger.Fatal(\"Problem starting manager\", err)\n\t}\n}\n\nfunc (s *nodeAgentServer) waitCacheForResume() error {\n\tpodInformer, err := s.mgr.GetCache().GetInformer(s.ctx, &corev1api.Pod{})\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error getting pod informer\")\n\t}\n\n\tduInformer, err := s.mgr.GetCache().GetInformer(s.ctx, &velerov2alpha1api.DataUpload{})\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error getting du informer\")\n\t}\n\n\tddInformer, err := s.mgr.GetCache().GetInformer(s.ctx, &velerov2alpha1api.DataDownload{})\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error getting dd informer\")\n\t}\n\n\tpvbInformer, err := s.mgr.GetCache().GetInformer(s.ctx, &velerov1api.PodVolumeBackup{})\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error getting PVB informer\")\n\t}\n\n\tpvrInformer, err := s.mgr.GetCache().GetInformer(s.ctx, &velerov1api.PodVolumeRestore{})\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error getting PVR informer\")\n\t}\n\n\tif !cacheutil.WaitForCacheSync(s.ctx.Done(), podInformer.HasSynced, duInformer.HasSynced, ddInformer.HasSynced, pvbInformer.HasSynced, pvrInformer.HasSynced) {\n\t\treturn errors.New(\"error waiting informer synced\")\n\t}\n\n\treturn nil\n}\n\n// validatePodVolumesHostPath validates that the pod volumes path contains a\n// directory for each Pod running on this node\nfunc (s *nodeAgentServer) validatePodVolumesHostPath(client kubernetes.Interface) error {\n\tfiles, err := s.fileSystem.ReadDir(nodeagent.HostPodVolumeMountPath())\n\tif err != nil {\n\t\tif errors.Is(err, os.ErrNotExist) {\n\t\t\ts.logger.Warnf(\"Pod volumes host path [%s] doesn't exist, fs-backup is disabled\", nodeagent.HostPodVolumeMountPath())\n\t\t\treturn nil\n\t\t}\n\t\treturn errors.Wrap(err, \"could not read pod volumes host path\")\n\t}\n\n\t// create a map of directory names inside the pod volumes path\n\tdirs := sets.NewString()\n\tfor _, f := range files {\n\t\tif f.IsDir() {\n\t\t\tdirs.Insert(f.Name())\n\t\t}\n\t}\n\n\tpods, err := client.CoreV1().Pods(\"\").List(s.ctx, metav1.ListOptions{FieldSelector: fmt.Sprintf(\"spec.nodeName=%s,status.phase=Running\", s.nodeName)})\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\tvalid := true\n\tfor _, pod := range pods.Items {\n\t\tdirName := string(pod.GetUID())\n\n\t\t// if the pod is a mirror pod, the directory name is the hash value of the\n\t\t// mirror pod annotation\n\t\tif hash, ok := pod.GetAnnotations()[corev1api.MirrorPodAnnotationKey]; ok {\n\t\t\tdirName = hash\n\t\t}\n\n\t\tif !dirs.Has(dirName) {\n\t\t\tvalid = false\n\t\t\ts.logger.WithFields(logrus.Fields{\n\t\t\t\t\"pod\": fmt.Sprintf(\"%s/%s\", pod.GetNamespace(), pod.GetName()),\n\t\t\t\t\"path\": nodeagent.HostPodVolumeMountPath() + \"/\" + dirName,\n\t\t\t}).Debug(\"could not find volumes for pod in host path\")\n\t\t}\n\t}\n\n\tif !valid {\n\t\treturn errors.New(\"unexpected directory structure for host-pods volume, ensure that the host-pods volume corresponds to the pods subdirectory of the kubelet root directory\")\n\t}\n\n\treturn nil\n}\n\nfunc (s *nodeAgentServer) markLegacyPVRsFailed(client ctrlclient.Client) {\n\tpvrs := &velerov1api.PodVolumeRestoreList{}\n\tif err := client.List(s.ctx, pvrs, &ctrlclient.ListOptions{Namespace: s.namespace}); err != nil {\n\t\ts.logger.WithError(errors.WithStack(err)).Error(\"failed to list podvolumerestores\")\n\t\treturn\n\t}\n\n\tfor i, pvr := range pvrs.Items {\n\t\tif !controller.IsLegacyPVR(&pvr) {\n\t\t\tcontinue\n\t\t}\n\n\t\tif pvr.Status.Phase != velerov1api.PodVolumeRestorePhaseInProgress {\n\t\t\ts.logger.Debugf(\"the status of podvolumerestore %q is %q, skip\", pvr.GetName(), pvr.Status.Phase)\n\t\t\tcontinue\n\t\t}\n\n\t\tpod := &corev1api.Pod{}\n\t\tif err := client.Get(s.ctx, types.NamespacedName{\n\t\t\tNamespace: pvr.Spec.Pod.Namespace,\n\t\t\tName: pvr.Spec.Pod.Name,\n\t\t}, pod); err != nil {\n\t\t\ts.logger.WithError(errors.WithStack(err)).Errorf(\"failed to get pod \\\"%s/%s\\\" of podvolumerestore %q\",\n\t\t\t\tpvr.Spec.Pod.Namespace, pvr.Spec.Pod.Name, pvr.GetName())\n\t\t\tcontinue\n\t\t}\n\t\tif pod.Spec.NodeName != s.nodeName {\n\t\t\ts.logger.Debugf(\"the node of pod referenced by podvolumerestore %q is %q, not %q, skip\", pvr.GetName(), pod.Spec.NodeName, s.nodeName)\n\t\t\tcontinue\n\t\t}\n\n\t\tif err := controller.UpdatePVRStatusToFailed(s.ctx, client, &pvrs.Items[i], errors.New(\"cannot survive from node-agent restart\"),\n\t\t\tfmt.Sprintf(\"get a legacy podvolumerestore with status %q during the server starting, mark it as %q\", velerov1api.PodVolumeRestorePhaseInProgress, velerov1api.PodVolumeRestorePhaseFailed),\n\t\t\ttime.Now(), s.logger); err != nil {\n\t\t\ts.logger.WithError(errors.WithStack(err)).Errorf(\"failed to patch podvolumerestore %q\", pvr.GetName())\n\t\t\tcontinue\n\t\t}\n\t\ts.logger.WithField(\"podvolumerestore\", pvr.GetName()).Warn(pvr.Status.Message)\n\t}\n}\n\nvar getConfigsFunc = nodeagent.GetConfigs\n\nfunc (s *nodeAgentServer) getDataPathConfigs() error {\n\tif s.config.nodeAgentConfig == \"\" {\n\t\ts.logger.Info(\"No node-agent configMap is specified\")\n\t\treturn nil\n\t}\n\n\tconfigs, err := getConfigsFunc(s.ctx, s.namespace, s.kubeClient, s.config.nodeAgentConfig)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error getting node agent configs from configMap %s\", s.config.nodeAgentConfig)\n\t}\n\n\ts.dataPathConfigs = configs\n\treturn nil\n}\n\nfunc (s *nodeAgentServer) getBackupRepoConfigs() error {\n\tif s.config.backupRepoConfig == \"\" {\n\t\ts.logger.Info(\"No backup repo configMap is specified\")\n\t\treturn nil\n\t}\n\n\tcm, err := s.kubeClient.CoreV1().ConfigMaps(s.namespace).Get(s.ctx, s.config.backupRepoConfig, metav1.GetOptions{})\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error getting backup repo configs from configMap %s\", s.config.backupRepoConfig)\n\t}\n\n\tif cm.Data == nil {\n\t\treturn errors.Errorf(\"no data is in the backup repo configMap %s\", s.config.backupRepoConfig)\n\t}\n\n\ts.backupRepoConfigs = cm.Data\n\treturn nil\n}\n\nfunc (s *nodeAgentServer) getDataPathConcurrentNum(defaultNum int) int {\n\tconfigs := s.dataPathConfigs\n\n\tif configs == nil || configs.LoadConcurrency == nil {\n\t\ts.logger.Infof(\"Concurrency configs are not found, use the default number %v\", defaultNum)\n\t\treturn defaultNum\n\t}\n\n\tglobalNum := configs.LoadConcurrency.GlobalConfig\n\n\tif globalNum <= 0 {\n\t\ts.logger.Warnf(\"Global number %v is invalid, use the default value %v\", globalNum, defaultNum)\n\t\tglobalNum = defaultNum\n\t}\n\n\tif len(configs.LoadConcurrency.PerNodeConfig) == 0 {\n\t\treturn globalNum\n\t}\n\n\tcurNode, err := s.kubeClient.CoreV1().Nodes().Get(s.ctx, s.nodeName, metav1.GetOptions{})\n\tif err != nil {\n\t\ts.logger.WithError(err).Warnf(\"Failed to get node info for %s, use the global number %v\", s.nodeName, globalNum)\n\t\treturn globalNum\n\t}\n\n\tconcurrentNum := math.MaxInt32\n\n\tfor _, rule := range configs.LoadConcurrency.PerNodeConfig {\n\t\tselector, err := metav1.LabelSelectorAsSelector(&(rule.NodeSelector))\n\t\tif err != nil {\n\t\t\ts.logger.WithError(err).Warnf(\"Failed to parse rule with label selector %s, skip it\", rule.NodeSelector.String())\n\t\t\tcontinue\n\t\t}\n\n\t\tif rule.Number <= 0 {\n\t\t\ts.logger.Warnf(\"Rule with label selector %s is with an invalid number %v, skip it\", rule.NodeSelector.String(), rule.Number)\n\t\t\tcontinue\n\t\t}\n\n\t\tif selector.Matches(labels.Set(curNode.GetLabels())) {\n\t\t\tif concurrentNum > rule.Number {\n\t\t\t\tconcurrentNum = rule.Number\n\t\t\t}\n\t\t}\n\t}\n\n\tif concurrentNum == math.MaxInt32 {\n\t\ts.logger.Infof(\"Per node number for node %s is not found, use the global number %v\", s.nodeName, globalNum)\n\t\tconcurrentNum = globalNum\n\t} else {\n\t\ts.logger.Infof(\"Use the per node number %v over global number %v for node %s\", concurrentNum, globalNum, s.nodeName)\n\t}\n\n\treturn concurrentNum\n}\n\nfunc (s *nodeAgentServer) validateCachePVCConfig(config velerotypes.CachePVC) error {\n\tif config.StorageClass == \"\" {\n\t\treturn errors.New(\"storage class is absent\")\n\t}\n\n\tsc, err := s.kubeClient.StorageV1().StorageClasses().Get(s.ctx, config.StorageClass, metav1.GetOptions{})\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error getting storage class %s\", config.StorageClass)\n\t}\n\n\tif sc.ReclaimPolicy != nil && *sc.ReclaimPolicy != corev1api.PersistentVolumeReclaimDelete {\n\t\treturn errors.Errorf(\"unexpected storage class reclaim policy %v\", *sc.ReclaimPolicy)\n\t}\n\n\treturn nil\n}\n" }, { "path": "pkg/cmd/cli/nodeagent/server_test.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage nodeagent\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"k8s.io/client-go/kubernetes/fake\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/nodeagent\"\n\ttestutil \"github.com/vmware-tanzu/velero/pkg/test\"\n\tvelerotypes \"github.com/vmware-tanzu/velero/pkg/types\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\nfunc Test_validatePodVolumesHostPath(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpods []*corev1api.Pod\n\t\tdirs []string\n\t\tcreateDir bool\n\t\twantErr bool\n\t}{\n\t\t{\n\t\t\tname: \"no error when pod volumes are present\",\n\t\t\tpods: []*corev1api.Pod{\n\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").ObjectMeta(builder.WithUID(\"foo\")).Result(),\n\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").ObjectMeta(builder.WithUID(\"zoo\")).Result(),\n\t\t\t},\n\t\t\tdirs: []string{\"foo\", \"zoo\"},\n\t\t\tcreateDir: true,\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"no error when pod volumes are present and there are mirror pods\",\n\t\t\tpods: []*corev1api.Pod{\n\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").ObjectMeta(builder.WithUID(\"foo\")).Result(),\n\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").ObjectMeta(builder.WithUID(\"zoo\"), builder.WithAnnotations(corev1api.MirrorPodAnnotationKey, \"baz\")).Result(),\n\t\t\t},\n\t\t\tdirs: []string{\"foo\", \"baz\"},\n\t\t\tcreateDir: true,\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"error when all pod volumes missing\",\n\t\t\tpods: []*corev1api.Pod{\n\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").ObjectMeta(builder.WithUID(\"foo\")).Result(),\n\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").ObjectMeta(builder.WithUID(\"zoo\")).Result(),\n\t\t\t},\n\t\t\tdirs: []string{\"unexpected-dir\"},\n\t\t\tcreateDir: true,\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"error when some pod volumes missing\",\n\t\t\tpods: []*corev1api.Pod{\n\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").ObjectMeta(builder.WithUID(\"foo\")).Result(),\n\t\t\t\tbuilder.ForPod(\"zoo\", \"raz\").ObjectMeta(builder.WithUID(\"zoo\")).Result(),\n\t\t\t},\n\t\t\tdirs: []string{\"foo\"},\n\t\t\tcreateDir: true,\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"no error when pod volumes are not present\",\n\t\t\tpods: []*corev1api.Pod{\n\t\t\t\tbuilder.ForPod(\"foo\", \"bar\").ObjectMeta(builder.WithUID(\"foo\")).Result(),\n\t\t\t},\n\t\t\tdirs: []string{\"foo\"},\n\t\t\tcreateDir: false,\n\t\t\twantErr: false,\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tfs := testutil.NewFakeFileSystem()\n\n\t\t\tfor _, dir := range tt.dirs {\n\t\t\t\tif tt.createDir {\n\t\t\t\t\terr := fs.MkdirAll(filepath.Join(nodeagent.HostPodVolumeMountPath(), dir), os.ModePerm)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\tt.Error(err)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tkubeClient := fake.NewSimpleClientset()\n\t\t\tfor _, pod := range tt.pods {\n\t\t\t\t_, err := kubeClient.CoreV1().Pods(pod.GetNamespace()).Create(t.Context(), pod, metav1.CreateOptions{})\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Error(err)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\ts := &nodeAgentServer{\n\t\t\t\tlogger: testutil.NewLogger(),\n\t\t\t\tfileSystem: fs,\n\t\t\t}\n\n\t\t\terr := s.validatePodVolumesHostPath(kubeClient)\n\t\t\tif tt.wantErr {\n\t\t\t\tassert.Error(t, err)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc Test_getDataPathConfigs(t *testing.T) {\n\tconfigs := &velerotypes.NodeAgentConfigs{\n\t\tLoadConcurrency: &velerotypes.LoadConcurrency{\n\t\t\tGlobalConfig: -1,\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tgetFunc func(context.Context, string, kubernetes.Interface, string) (*velerotypes.NodeAgentConfigs, error)\n\t\tconfigMapName string\n\t\texpectConfigs *velerotypes.NodeAgentConfigs\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"no config specified\",\n\t\t},\n\t\t{\n\t\t\tname: \"failed to get configs\",\n\t\t\tconfigMapName: \"node-agent-config\",\n\t\t\tgetFunc: func(context.Context, string, kubernetes.Interface, string) (*velerotypes.NodeAgentConfigs, error) {\n\t\t\t\treturn nil, errors.New(\"fake-get-error\")\n\t\t\t},\n\t\t\texpectedErr: \"error getting node agent configs from configMap node-agent-config: fake-get-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"configs cm not found\",\n\t\t\tconfigMapName: \"node-agent-config\",\n\t\t\tgetFunc: func(context.Context, string, kubernetes.Interface, string) (*velerotypes.NodeAgentConfigs, error) {\n\t\t\t\treturn nil, errors.New(\"fake-not-found-error\")\n\t\t\t},\n\t\t\texpectedErr: \"error getting node agent configs from configMap node-agent-config: fake-not-found-error\",\n\t\t},\n\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tconfigMapName: \"node-agent-config\",\n\t\t\tgetFunc: func(context.Context, string, kubernetes.Interface, string) (*velerotypes.NodeAgentConfigs, error) {\n\t\t\t\treturn configs, nil\n\t\t\t},\n\t\t\texpectConfigs: configs,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\ts := &nodeAgentServer{\n\t\t\t\tconfig: nodeAgentServerConfig{\n\t\t\t\t\tnodeAgentConfig: test.configMapName,\n\t\t\t\t},\n\t\t\t\tlogger: testutil.NewLogger(),\n\t\t\t}\n\n\t\t\tgetConfigsFunc = test.getFunc\n\n\t\t\terr := s.getDataPathConfigs()\n\t\t\tif test.expectedErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, test.expectConfigs, s.dataPathConfigs)\n\t\t\t} else {\n\t\t\t\trequire.EqualError(t, err, test.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc Test_getDataPathConcurrentNum(t *testing.T) {\n\tdefaultNum := 100001\n\tglobalNum := 6\n\tnodeName := \"node-agent-node\"\n\tnode1 := builder.ForNode(\"node-agent-node\").Result()\n\tnode2 := builder.ForNode(\"node-agent-node\").Labels(map[string]string{\n\t\t\"host-name\": \"node-1\",\n\t\t\"xxxx\": \"yyyyy\",\n\t}).Result()\n\n\tinvalidLabelSelector := metav1.LabelSelector{\n\t\tMatchLabels: map[string]string{\n\t\t\t\"inva/lid\": \"inva/lid\",\n\t\t},\n\t}\n\tvalidLabelSelector1 := metav1.LabelSelector{\n\t\tMatchLabels: map[string]string{\n\t\t\t\"host-name\": \"node-1\",\n\t\t},\n\t}\n\tvalidLabelSelector2 := metav1.LabelSelector{\n\t\tMatchLabels: map[string]string{\n\t\t\t\"xxxx\": \"yyyyy\",\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tconfigs velerotypes.NodeAgentConfigs\n\t\tsetKubeClient bool\n\t\tkubeClientObj []runtime.Object\n\t\texpectNum int\n\t\texpectLog string\n\t}{\n\t\t{\n\t\t\tname: \"configs cm's data path concurrency is nil\",\n\t\t\texpectLog: fmt.Sprintf(\"Concurrency configs are not found, use the default number %v\", defaultNum),\n\t\t\texpectNum: defaultNum,\n\t\t},\n\t\t{\n\t\t\tname: \"global number is invalid\",\n\t\t\tconfigs: velerotypes.NodeAgentConfigs{\n\t\t\t\tLoadConcurrency: &velerotypes.LoadConcurrency{\n\t\t\t\t\tGlobalConfig: -1,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectLog: fmt.Sprintf(\"Global number %v is invalid, use the default value %v\", -1, defaultNum),\n\t\t\texpectNum: defaultNum,\n\t\t},\n\t\t{\n\t\t\tname: \"global number is valid\",\n\t\t\tconfigs: velerotypes.NodeAgentConfigs{\n\t\t\t\tLoadConcurrency: &velerotypes.LoadConcurrency{\n\t\t\t\t\tGlobalConfig: globalNum,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectNum: globalNum,\n\t\t},\n\t\t{\n\t\t\tname: \"node is not found\",\n\t\t\tconfigs: velerotypes.NodeAgentConfigs{\n\t\t\t\tLoadConcurrency: &velerotypes.LoadConcurrency{\n\t\t\t\t\tGlobalConfig: globalNum,\n\t\t\t\t\tPerNodeConfig: []velerotypes.RuledConfigs{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tNumber: 100,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsetKubeClient: true,\n\t\t\texpectLog: fmt.Sprintf(\"Failed to get node info for %s, use the global number %v\", nodeName, globalNum),\n\t\t\texpectNum: globalNum,\n\t\t},\n\t\t{\n\t\t\tname: \"failed to get selector\",\n\t\t\tconfigs: velerotypes.NodeAgentConfigs{\n\t\t\t\tLoadConcurrency: &velerotypes.LoadConcurrency{\n\t\t\t\t\tGlobalConfig: globalNum,\n\t\t\t\t\tPerNodeConfig: []velerotypes.RuledConfigs{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tNodeSelector: invalidLabelSelector,\n\t\t\t\t\t\t\tNumber: 100,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsetKubeClient: true,\n\t\t\tkubeClientObj: []runtime.Object{node1},\n\t\t\texpectLog: fmt.Sprintf(\"Failed to parse rule with label selector %s, skip it\", invalidLabelSelector.String()),\n\t\t\texpectNum: globalNum,\n\t\t},\n\t\t{\n\t\t\tname: \"rule number is invalid\",\n\t\t\tconfigs: velerotypes.NodeAgentConfigs{\n\t\t\t\tLoadConcurrency: &velerotypes.LoadConcurrency{\n\t\t\t\t\tGlobalConfig: globalNum,\n\t\t\t\t\tPerNodeConfig: []velerotypes.RuledConfigs{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tNodeSelector: validLabelSelector1,\n\t\t\t\t\t\t\tNumber: -1,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsetKubeClient: true,\n\t\t\tkubeClientObj: []runtime.Object{node1},\n\t\t\texpectLog: fmt.Sprintf(\"Rule with label selector %s is with an invalid number %v, skip it\", validLabelSelector1.String(), -1),\n\t\t\texpectNum: globalNum,\n\t\t},\n\t\t{\n\t\t\tname: \"label doesn't match\",\n\t\t\tconfigs: velerotypes.NodeAgentConfigs{\n\t\t\t\tLoadConcurrency: &velerotypes.LoadConcurrency{\n\t\t\t\t\tGlobalConfig: globalNum,\n\t\t\t\t\tPerNodeConfig: []velerotypes.RuledConfigs{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tNodeSelector: validLabelSelector1,\n\t\t\t\t\t\t\tNumber: -1,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsetKubeClient: true,\n\t\t\tkubeClientObj: []runtime.Object{node1},\n\t\t\texpectLog: fmt.Sprintf(\"Per node number for node %s is not found, use the global number %v\", nodeName, globalNum),\n\t\t\texpectNum: globalNum,\n\t\t},\n\t\t{\n\t\t\tname: \"match one rule\",\n\t\t\tconfigs: velerotypes.NodeAgentConfigs{\n\t\t\t\tLoadConcurrency: &velerotypes.LoadConcurrency{\n\t\t\t\t\tGlobalConfig: globalNum,\n\t\t\t\t\tPerNodeConfig: []velerotypes.RuledConfigs{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tNodeSelector: validLabelSelector1,\n\t\t\t\t\t\t\tNumber: 66,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsetKubeClient: true,\n\t\t\tkubeClientObj: []runtime.Object{node2},\n\t\t\texpectLog: fmt.Sprintf(\"Use the per node number %v over global number %v for node %s\", 66, globalNum, nodeName),\n\t\t\texpectNum: 66,\n\t\t},\n\t\t{\n\t\t\tname: \"match multiple rules\",\n\t\t\tconfigs: velerotypes.NodeAgentConfigs{\n\t\t\t\tLoadConcurrency: &velerotypes.LoadConcurrency{\n\t\t\t\t\tGlobalConfig: globalNum,\n\t\t\t\t\tPerNodeConfig: []velerotypes.RuledConfigs{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tNodeSelector: validLabelSelector1,\n\t\t\t\t\t\t\tNumber: 66,\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tNodeSelector: validLabelSelector2,\n\t\t\t\t\t\t\tNumber: 36,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsetKubeClient: true,\n\t\t\tkubeClientObj: []runtime.Object{node2},\n\t\t\texpectLog: fmt.Sprintf(\"Use the per node number %v over global number %v for node %s\", 36, globalNum, nodeName),\n\t\t\texpectNum: 36,\n\t\t},\n\t\t{\n\t\t\tname: \"match multiple rules 2\",\n\t\t\tconfigs: velerotypes.NodeAgentConfigs{\n\t\t\t\tLoadConcurrency: &velerotypes.LoadConcurrency{\n\t\t\t\t\tGlobalConfig: globalNum,\n\t\t\t\t\tPerNodeConfig: []velerotypes.RuledConfigs{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tNodeSelector: validLabelSelector1,\n\t\t\t\t\t\t\tNumber: 36,\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tNodeSelector: validLabelSelector2,\n\t\t\t\t\t\t\tNumber: 66,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsetKubeClient: true,\n\t\t\tkubeClientObj: []runtime.Object{node2},\n\t\t\texpectLog: fmt.Sprintf(\"Use the per node number %v over global number %v for node %s\", 36, globalNum, nodeName),\n\t\t\texpectNum: 36,\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tlogBuffer := \"\"\n\n\t\t\ts := &nodeAgentServer{\n\t\t\t\tnodeName: nodeName,\n\t\t\t\tdataPathConfigs: &test.configs,\n\t\t\t\tlogger: testutil.NewSingleLogger(&logBuffer),\n\t\t\t}\n\n\t\t\tif test.setKubeClient {\n\t\t\t\ts.kubeClient = fakeKubeClient\n\t\t\t}\n\n\t\t\tnum := s.getDataPathConcurrentNum(defaultNum)\n\t\t\tassert.Equal(t, test.expectNum, num)\n\t\t\tif test.expectLog == \"\" {\n\t\t\t\tassert.Empty(t, logBuffer)\n\t\t\t} else {\n\t\t\t\tassert.Contains(t, logBuffer, test.expectLog)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetBackupRepoConfigs(t *testing.T) {\n\tcmNoData := builder.ForConfigMap(velerov1api.DefaultNamespace, \"backup-repo-config\").Result()\n\tcmWithData := builder.ForConfigMap(velerov1api.DefaultNamespace, \"backup-repo-config\").Data(\"cacheLimit\", \"100\").Result()\n\n\ttests := []struct {\n\t\tname string\n\t\tconfigMapName string\n\t\tkubeClientObj []runtime.Object\n\t\texpectConfigs map[string]string\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"no config specified\",\n\t\t},\n\t\t{\n\t\t\tname: \"failed to get configs\",\n\t\t\tconfigMapName: \"backup-repo-config\",\n\t\t\texpectedErr: \"error getting backup repo configs from configMap backup-repo-config: configmaps \\\"backup-repo-config\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"configs data not found\",\n\t\t\tkubeClientObj: []runtime.Object{cmNoData},\n\t\t\tconfigMapName: \"backup-repo-config\",\n\t\t\texpectedErr: \"no data is in the backup repo configMap backup-repo-config\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tconfigMapName: \"backup-repo-config\",\n\t\t\tkubeClientObj: []runtime.Object{cmWithData},\n\t\t\texpectConfigs: map[string]string{\"cacheLimit\": \"100\"},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\ts := &nodeAgentServer{\n\t\t\t\tnamespace: velerov1api.DefaultNamespace,\n\t\t\t\tkubeClient: fakeKubeClient,\n\t\t\t\tconfig: nodeAgentServerConfig{\n\t\t\t\t\tbackupRepoConfig: test.configMapName,\n\t\t\t\t},\n\t\t\t\tlogger: testutil.NewLogger(),\n\t\t\t}\n\n\t\t\terr := s.getBackupRepoConfigs()\n\t\t\tif test.expectedErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\trequire.Equal(t, test.expectConfigs, s.backupRepoConfigs)\n\t\t\t} else {\n\t\t\t\trequire.EqualError(t, err, test.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestValidateCachePVCConfig(t *testing.T) {\n\tscWithRetainPolicy := builder.ForStorageClass(\"fake-storage-class\").ReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).Result()\n\tscWithDeletePolicy := builder.ForStorageClass(\"fake-storage-class\").ReclaimPolicy(corev1api.PersistentVolumeReclaimDelete).Result()\n\tscWithNoPolicy := builder.ForStorageClass(\"fake-storage-class\").Result()\n\n\ttests := []struct {\n\t\tname string\n\t\tconfig velerotypes.CachePVC\n\t\tkubeClientObj []runtime.Object\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"no storage class\",\n\t\t\texpectedErr: \"storage class is absent\",\n\t\t},\n\t\t{\n\t\t\tname: \"failed to get storage class\",\n\t\t\tconfig: velerotypes.CachePVC{StorageClass: \"fake-storage-class\"},\n\t\t\texpectedErr: \"error getting storage class fake-storage-class: storageclasses.storage.k8s.io \\\"fake-storage-class\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"storage class reclaim policy is not expected\",\n\t\t\tconfig: velerotypes.CachePVC{StorageClass: \"fake-storage-class\"},\n\t\t\tkubeClientObj: []runtime.Object{scWithRetainPolicy},\n\t\t\texpectedErr: \"unexpected storage class reclaim policy Retain\",\n\t\t},\n\t\t{\n\t\t\tname: \"storage class reclaim policy is delete\",\n\t\t\tconfig: velerotypes.CachePVC{StorageClass: \"fake-storage-class\"},\n\t\t\tkubeClientObj: []runtime.Object{scWithDeletePolicy},\n\t\t},\n\t\t{\n\t\t\tname: \"storage class with no reclaim policy\",\n\t\t\tconfig: velerotypes.CachePVC{StorageClass: \"fake-storage-class\"},\n\t\t\tkubeClientObj: []runtime.Object{scWithNoPolicy},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\ts := &nodeAgentServer{\n\t\t\t\tkubeClient: fakeKubeClient,\n\t\t\t}\n\n\t\t\terr := s.validateCachePVCConfig(test.config)\n\n\t\t\tif test.expectedErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t} else {\n\t\t\t\trequire.EqualError(t, err, test.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/cmd/cli/plugin/add.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage plugin\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"strings\"\n\n\tjsonpatch \"github.com/evanphx/json-patch/v5\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/spf13/cobra\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/confirm\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/flag\"\n)\n\nconst (\n\tpluginsVolumeName = \"plugins\"\n\tveleroContainer = \"velero\"\n)\n\nfunc NewAddCommand(f client.Factory) *cobra.Command {\n\tvar (\n\t\timagePullPolicies = []string{string(corev1api.PullAlways), string(corev1api.PullIfNotPresent), string(corev1api.PullNever)}\n\t\timagePullPolicyFlag = flag.NewEnum(string(corev1api.PullIfNotPresent), imagePullPolicies...)\n\t)\n\to := confirm.NewConfirmOptionsWithDescription(\"Confirm add?, may cause the Velero server pod restart which will fail all ongoing jobs\")\n\tc := &cobra.Command{\n\t\tUse: \"add IMAGE\",\n\t\tShort: \"Add a plugin\",\n\t\tArgs: cobra.ExactArgs(1),\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tif !o.Confirm && !confirm.GetConfirmation(\"velero plugin add may cause the Velero server pod restart, so it is a dangerous operation\",\n\t\t\t\t\"once Velero server restarts, all the ongoing jobs will fail.\") {\n\t\t\t\t// Don't do anything unless we get confirmation\n\t\t\t\treturn\n\t\t\t}\n\t\t\tkubeClient, err := f.KubeClient()\n\t\t\tif err != nil {\n\t\t\t\tcmd.CheckError(err)\n\t\t\t}\n\n\t\t\tveleroDeploy, err := veleroDeployment(context.TODO(), kubeClient, f.Namespace())\n\t\t\tif err != nil {\n\t\t\t\tcmd.CheckError(err)\n\t\t\t}\n\n\t\t\toriginal, err := json.Marshal(veleroDeploy)\n\t\t\tcmd.CheckError(err)\n\n\t\t\t// ensure the plugins volume & mount exist\n\t\t\tvolumeExists := false\n\t\t\tfor _, volume := range veleroDeploy.Spec.Template.Spec.Volumes {\n\t\t\t\tif volume.Name == pluginsVolumeName {\n\t\t\t\t\tvolumeExists = true\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif !volumeExists {\n\t\t\t\tvolume := corev1api.Volume{\n\t\t\t\t\tName: pluginsVolumeName,\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tEmptyDir: &corev1api.EmptyDirVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t}\n\n\t\t\t\tvolumeMount := corev1api.VolumeMount{\n\t\t\t\t\tName: pluginsVolumeName,\n\t\t\t\t\tMountPath: \"/plugins\",\n\t\t\t\t}\n\n\t\t\t\tveleroDeploy.Spec.Template.Spec.Volumes = append(veleroDeploy.Spec.Template.Spec.Volumes, volume)\n\n\t\t\t\tcontainers := veleroDeploy.Spec.Template.Spec.Containers\n\t\t\t\tcontainerIndex := -1\n\t\t\t\tfor x, container := range containers {\n\t\t\t\t\tif container.Name == veleroContainer {\n\t\t\t\t\t\tcontainerIndex = x\n\t\t\t\t\t\tbreak\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\tif containerIndex < 0 {\n\t\t\t\t\tcmd.CheckError(errors.New(\"velero container not found in velero deployment\"))\n\t\t\t\t}\n\n\t\t\t\tcontainers[containerIndex].VolumeMounts = append(containers[containerIndex].VolumeMounts, volumeMount)\n\t\t\t}\n\n\t\t\t// add the plugin as an init container\n\t\t\tplugin := *builder.ForPluginContainer(args[0], corev1api.PullPolicy(imagePullPolicyFlag.String())).Result()\n\n\t\t\tveleroDeploy.Spec.Template.Spec.InitContainers = append(veleroDeploy.Spec.Template.Spec.InitContainers, plugin)\n\n\t\t\t// create & apply the patch\n\t\t\tupdated, err := json.Marshal(veleroDeploy)\n\t\t\tcmd.CheckError(err)\n\n\t\t\tpatchBytes, err := jsonpatch.CreateMergePatch(original, updated)\n\t\t\tcmd.CheckError(err)\n\n\t\t\t_, err = kubeClient.AppsV1().Deployments(veleroDeploy.Namespace).Patch(context.TODO(), veleroDeploy.Name, types.MergePatchType, patchBytes, metav1.PatchOptions{})\n\t\t\tcmd.CheckError(err)\n\t\t},\n\t}\n\n\tc.Flags().Var(imagePullPolicyFlag, \"image-pull-policy\", fmt.Sprintf(\"The imagePullPolicy for the plugin container. Valid values are %s.\", strings.Join(imagePullPolicies, \", \")))\n\to.BindFlags(c.Flags())\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/plugin/get.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage plugin\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"time\"\n\n\t\"github.com/spf13/cobra\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/serverstatus\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/output\"\n)\n\nfunc NewGetCommand(f client.Factory, use string) *cobra.Command {\n\ttimeout := 5 * time.Second\n\n\tc := &cobra.Command{\n\t\tUse: use,\n\t\tShort: \"Get information for all plugins on the velero server\",\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\terr := output.ValidateFlags(c)\n\t\t\tcmd.CheckError(err)\n\n\t\t\tkbClient, err := f.KubebuilderClient()\n\t\t\tcmd.CheckError(err)\n\n\t\t\tctx, cancel := context.WithTimeout(context.Background(), timeout)\n\t\t\tdefer cancel()\n\n\t\t\tserverStatusGetter := &serverstatus.DefaultServerStatusGetter{\n\t\t\t\tNamespace: f.Namespace(),\n\t\t\t\tContext: ctx,\n\t\t\t}\n\n\t\t\tserverStatus, err := serverStatusGetter.GetServerStatus(kbClient)\n\t\t\tif err != nil {\n\t\t\t\tfmt.Fprintf(os.Stdout, \"\\n\", err)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\t_, err = output.PrintWithFormat(c, serverStatus)\n\t\t\tcmd.CheckError(err)\n\t\t},\n\t}\n\n\tc.Flags().DurationVar(&timeout, \"timeout\", timeout, \"Maximum time to wait for plugin information to be reported. Default is 5 seconds.\")\n\toutput.BindFlagsSimple(c.Flags())\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/plugin/helpers.go", "content": "/*\nCopyright The Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage plugin\n\nimport (\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/client-go/kubernetes\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/install\"\n)\n\n// veleroDeployment returns a Velero deployment object, selected with label and container name,\n// refer to https://github.com/vmware-tanzu/velero/issues/3961 for more information\nfunc veleroDeployment(ctx context.Context, kubeClient kubernetes.Interface, namespace string) (*appsv1api.Deployment, error) {\n\tveleroLabels := labels.FormatLabels(install.Labels())\n\n\tdeployList, err := kubeClient.\n\t\tAppsV1().\n\t\tDeployments(namespace).\n\t\tList(ctx, metav1.ListOptions{\n\t\t\tLabelSelector: veleroLabels,\n\t\t})\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfor _, deploy := range deployList.Items {\n\t\tfor _, container := range deploy.Spec.Template.Spec.Containers {\n\t\t\tif container.Name == \"velero\" {\n\t\t\t\treturn &deploy, nil\n\t\t\t}\n\t\t}\n\t}\n\n\treturn nil, errors.New(\"Velero deployment not found\")\n}\n" }, { "path": "pkg/cmd/cli/plugin/plugin.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage plugin\n\nimport (\n\t\"github.com/spf13/cobra\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n)\n\nfunc NewCommand(f client.Factory) *cobra.Command {\n\tc := &cobra.Command{\n\t\tUse: \"plugin\",\n\t\tShort: \"Work with plugins\",\n\t\tLong: \"Work with plugins\",\n\t}\n\n\tc.AddCommand(\n\t\tNewAddCommand(f),\n\t\tNewRemoveCommand(f),\n\t\tNewGetCommand(f, \"get\"),\n\t)\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/plugin/remove.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage plugin\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\n\tjsonpatch \"github.com/evanphx/json-patch/v5\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/spf13/cobra\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n)\n\nfunc NewRemoveCommand(f client.Factory) *cobra.Command {\n\tc := &cobra.Command{\n\t\tUse: \"remove [NAME | IMAGE]\",\n\t\tShort: \"Remove a plugin\",\n\t\tArgs: cobra.ExactArgs(1),\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tkubeClient, err := f.KubeClient()\n\t\t\tif err != nil {\n\t\t\t\tcmd.CheckError(err)\n\t\t\t}\n\n\t\t\tveleroDeploy, err := veleroDeployment(context.TODO(), kubeClient, f.Namespace())\n\t\t\tif err != nil {\n\t\t\t\tcmd.CheckError(err)\n\t\t\t}\n\n\t\t\toriginal, err := json.Marshal(veleroDeploy)\n\t\t\tcmd.CheckError(err)\n\n\t\t\tvar (\n\t\t\t\tinitContainers = veleroDeploy.Spec.Template.Spec.InitContainers\n\t\t\t\tindex = -1\n\t\t\t)\n\n\t\t\tfor x, container := range initContainers {\n\t\t\t\tif container.Name == args[0] || container.Image == args[0] {\n\t\t\t\t\tindex = x\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif index == -1 {\n\t\t\t\tcmd.CheckError(errors.Errorf(\"init container %s not found in Velero server deployment\", args[0]))\n\t\t\t}\n\n\t\t\tveleroDeploy.Spec.Template.Spec.InitContainers = append(initContainers[0:index], initContainers[index+1:]...)\n\n\t\t\tupdated, err := json.Marshal(veleroDeploy)\n\t\t\tcmd.CheckError(err)\n\n\t\t\tpatchBytes, err := jsonpatch.CreateMergePatch(original, updated)\n\t\t\tcmd.CheckError(err)\n\n\t\t\t_, err = kubeClient.AppsV1().Deployments(veleroDeploy.Namespace).Patch(context.TODO(), veleroDeploy.Name, types.MergePatchType, patchBytes, metav1.PatchOptions{})\n\t\t\tcmd.CheckError(err)\n\t\t},\n\t}\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/podvolume/backup.go", "content": "/*\nCopyright The Velero Contributors.\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n http://www.apache.org/licenses/LICENSE-2.0\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/bombsimon/logrusr/v3\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/spf13/cobra\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/fields\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"k8s.io/klog/v2\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\t\"github.com/vmware-tanzu/velero/pkg/buildinfo\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/signals\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\t\"github.com/vmware-tanzu/velero/pkg/podvolume\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\n\tctlcache \"sigs.k8s.io/controller-runtime/pkg/cache\"\n\tctlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n)\n\ntype podVolumeBackupConfig struct {\n\tvolumePath string\n\tpvbName string\n\tresourceTimeout time.Duration\n}\n\nfunc NewBackupCommand(f client.Factory) *cobra.Command {\n\tconfig := podVolumeBackupConfig{}\n\n\tlogLevelFlag := logging.LogLevelFlag(logrus.InfoLevel)\n\tformatFlag := logging.NewFormatFlag()\n\n\tcommand := &cobra.Command{\n\t\tUse: \"backup\",\n\t\tShort: \"Run the velero pod volume backup\",\n\t\tLong: \"Run the velero pod volume backup\",\n\t\tHidden: true,\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tlogLevel := logLevelFlag.Parse()\n\t\t\tlogrus.Infof(\"Setting log-level to %s\", strings.ToUpper(logLevel.String()))\n\n\t\t\tlogger := logging.DefaultLogger(logLevel, formatFlag.Parse())\n\t\t\tlogger.Infof(\"Starting Velero pod volume backup %s (%s)\", buildinfo.Version, buildinfo.FormattedGitSHA())\n\n\t\t\tf.SetBasename(fmt.Sprintf(\"%s-%s\", c.Parent().Name(), c.Name()))\n\t\t\ts, err := newPodVolumeBackup(logger, f, config)\n\t\t\tif err != nil {\n\t\t\t\tkube.ExitPodWithMessage(logger, false, \"Failed to create pod volume backup, %v\", err)\n\t\t\t}\n\n\t\t\ts.run()\n\t\t},\n\t}\n\n\tcommand.Flags().Var(logLevelFlag, \"log-level\", fmt.Sprintf(\"The level at which to log. Valid values are %s.\", strings.Join(logLevelFlag.AllowedValues(), \", \")))\n\tcommand.Flags().Var(formatFlag, \"log-format\", fmt.Sprintf(\"The format for log output. Valid values are %s.\", strings.Join(formatFlag.AllowedValues(), \", \")))\n\tcommand.Flags().StringVar(&config.volumePath, \"volume-path\", config.volumePath, \"The full path of the volume to be backed up\")\n\tcommand.Flags().StringVar(&config.pvbName, \"pod-volume-backup\", config.pvbName, \"The PVB name\")\n\tcommand.Flags().DurationVar(&config.resourceTimeout, \"resource-timeout\", config.resourceTimeout, \"How long to wait for resource processes which are not covered by other specific timeout parameters.\")\n\n\t_ = command.MarkFlagRequired(\"volume-path\")\n\t_ = command.MarkFlagRequired(\"pod-volume-backup\")\n\t_ = command.MarkFlagRequired(\"resource-timeout\")\n\n\treturn command\n}\n\ntype podVolumeBackup struct {\n\tlogger logrus.FieldLogger\n\tctx context.Context\n\tcancelFunc context.CancelFunc\n\tclient ctlclient.Client\n\tcache ctlcache.Cache\n\tnamespace string\n\tnodeName string\n\tconfig podVolumeBackupConfig\n\tkubeClient kubernetes.Interface\n\tdataPathMgr *datapath.Manager\n}\n\nfunc newPodVolumeBackup(logger logrus.FieldLogger, factory client.Factory, config podVolumeBackupConfig) (*podVolumeBackup, error) {\n\tctx, cancelFunc := context.WithCancel(context.Background())\n\n\tclientConfig, err := factory.ClientConfig()\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to create client config\")\n\t}\n\n\tctrl.SetLogger(logrusr.New(logger))\n\tklog.SetLogger(logrusr.New(logger)) // klog.Logger is used by k8s.io/client-go\n\n\tscheme := runtime.NewScheme()\n\tif err := velerov1api.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to add velero v1 scheme\")\n\t}\n\n\tif err := corev1api.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to add core v1 scheme\")\n\t}\n\n\tnodeName := os.Getenv(\"NODE_NAME\")\n\n\t// use a field selector to filter to only pods scheduled on this node.\n\tcacheOption := ctlcache.Options{\n\t\tScheme: scheme,\n\t\tByObject: map[ctlclient.Object]ctlcache.ByObject{\n\t\t\t&corev1api.Pod{}: {\n\t\t\t\tField: fields.Set{\"spec.nodeName\": nodeName}.AsSelector(),\n\t\t\t},\n\t\t\t&velerov1api.PodVolumeBackup{}: {\n\t\t\t\tField: fields.Set{\"metadata.namespace\": factory.Namespace()}.AsSelector(),\n\t\t\t},\n\t\t},\n\t}\n\n\tcli, err := ctlclient.New(clientConfig, ctlclient.Options{\n\t\tScheme: scheme,\n\t})\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to create client\")\n\t}\n\n\tvar cache ctlcache.Cache\n\tretry := 10\n\tfor {\n\t\tcache, err = ctlcache.New(clientConfig, cacheOption)\n\t\tif err == nil {\n\t\t\tbreak\n\t\t}\n\n\t\tretry--\n\t\tif retry == 0 {\n\t\t\tbreak\n\t\t}\n\n\t\tlogger.WithError(err).Warn(\"Failed to create client cache, need retry\")\n\n\t\ttime.Sleep(time.Second)\n\t}\n\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to create client cache\")\n\t}\n\n\ts := &podVolumeBackup{\n\t\tlogger: logger,\n\t\tctx: ctx,\n\t\tcancelFunc: cancelFunc,\n\t\tclient: cli,\n\t\tcache: cache,\n\t\tconfig: config,\n\t\tnamespace: factory.Namespace(),\n\t\tnodeName: nodeName,\n\t}\n\n\ts.kubeClient, err = factory.KubeClient()\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to create kube client\")\n\t}\n\n\ts.dataPathMgr = datapath.NewManager(1)\n\n\treturn s, nil\n}\n\nvar funcExitWithMessage = kube.ExitPodWithMessage\nvar funcCreateDataPathService = (*podVolumeBackup).createDataPathService\n\nfunc (s *podVolumeBackup) run() {\n\tsignals.CancelOnShutdown(s.cancelFunc, s.logger)\n\tgo func() {\n\t\tif err := s.cache.Start(s.ctx); err != nil {\n\t\t\ts.logger.WithError(err).Warn(\"error starting cache\")\n\t\t}\n\t}()\n\n\ts.runDataPath()\n}\n\nfunc (s *podVolumeBackup) runDataPath() {\n\ts.logger.Infof(\"Starting micro service in node %s for PVB %s\", s.nodeName, s.config.pvbName)\n\n\tdpService, err := funcCreateDataPathService(s)\n\tif err != nil {\n\t\ts.cancelFunc()\n\t\tfuncExitWithMessage(s.logger, false, \"Failed to create data path service for PVB %s: %v\", s.config.pvbName, err)\n\t\treturn\n\t}\n\n\ts.logger.Infof(\"Starting data path service %s\", s.config.pvbName)\n\n\terr = dpService.Init()\n\tif err != nil {\n\t\tdpService.Shutdown()\n\t\ts.cancelFunc()\n\t\tfuncExitWithMessage(s.logger, false, \"Failed to init data path service for PVB %s: %v\", s.config.pvbName, err)\n\t\treturn\n\t}\n\n\ts.logger.Infof(\"Running data path service %s\", s.config.pvbName)\n\n\tresult, err := dpService.RunCancelableDataPath(s.ctx)\n\tif err != nil {\n\t\tdpService.Shutdown()\n\t\ts.cancelFunc()\n\t\tfuncExitWithMessage(s.logger, false, \"Failed to run data path service for PVB %s: %v\", s.config.pvbName, err)\n\t\treturn\n\t}\n\n\ts.logger.WithField(\"PVB\", s.config.pvbName).Info(\"Data path service completed\")\n\n\tdpService.Shutdown()\n\n\ts.logger.WithField(\"PVB\", s.config.pvbName).Info(\"Data path service is shut down\")\n\n\ts.cancelFunc()\n\n\tfuncExitWithMessage(s.logger, true, result)\n}\n\nvar funcNewCredentialFileStore = credentials.NewNamespacedFileStore\nvar funcNewCredentialSecretStore = credentials.NewNamespacedSecretStore\n\nfunc (s *podVolumeBackup) createDataPathService() (dataPathService, error) {\n\tcredentialFileStore, err := funcNewCredentialFileStore(\n\t\ts.client,\n\t\ts.namespace,\n\t\tcredentials.DefaultStoreDirectory(),\n\t\tfilesystem.NewFileSystem(),\n\t)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error to create credential file store\")\n\t}\n\n\tcredSecretStore, err := funcNewCredentialSecretStore(s.client, s.namespace)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error to create credential secret store\")\n\t}\n\n\tcredGetter := &credentials.CredentialGetter{FromFile: credentialFileStore, FromSecret: credSecretStore}\n\n\tpvbInformer, err := s.cache.GetInformer(s.ctx, &velerov1api.PodVolumeBackup{})\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error to get controller-runtime informer from manager\")\n\t}\n\n\trepoEnsurer := repository.NewEnsurer(s.client, s.logger, s.config.resourceTimeout)\n\n\treturn podvolume.NewBackupMicroService(s.ctx, s.client, s.kubeClient, s.config.pvbName, s.namespace, s.nodeName, datapath.AccessPoint{\n\t\tByPath: s.config.volumePath,\n\t\tVolMode: uploader.PersistentVolumeFilesystem,\n\t}, s.dataPathMgr, repoEnsurer, credGetter, pvbInformer, s.logger), nil\n}\n" }, { "path": "pkg/cmd/cli/podvolume/backup_test.go", "content": "/*\nCopyright The Velero Contributors.\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n http://www.apache.org/licenses/LICENSE-2.0\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\tctlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\tcacheMock \"github.com/vmware-tanzu/velero/pkg/cmd/cli/datamover/mocks\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n)\n\nfunc fakeCreateDataPathServiceWithErr(_ *podVolumeBackup) (dataPathService, error) {\n\treturn nil, errors.New(\"fake-create-data-path-error\")\n}\n\nvar frHelper *fakeRunHelper\n\nfunc fakeCreateDataPathService(_ *podVolumeBackup) (dataPathService, error) {\n\treturn frHelper, nil\n}\n\ntype fakeRunHelper struct {\n\tinitErr error\n\trunCancelableDataPathErr error\n\trunCancelableDataPathResult string\n\texitMessage string\n\tsucceed bool\n}\n\nfunc (fr *fakeRunHelper) Init() error {\n\treturn fr.initErr\n}\n\nfunc (fr *fakeRunHelper) RunCancelableDataPath(_ context.Context) (string, error) {\n\tif fr.runCancelableDataPathErr != nil {\n\t\treturn \"\", fr.runCancelableDataPathErr\n\t} else {\n\t\treturn fr.runCancelableDataPathResult, nil\n\t}\n}\n\nfunc (fr *fakeRunHelper) Shutdown() {\n\n}\n\nfunc (fr *fakeRunHelper) ExitWithMessage(logger logrus.FieldLogger, succeed bool, message string, a ...any) {\n\tfr.succeed = succeed\n\tfr.exitMessage = fmt.Sprintf(message, a...)\n}\n\nfunc TestRunDataPath(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvbName string\n\t\tcreateDataPathFail bool\n\t\tinitDataPathErr error\n\t\trunCancelableDataPathErr error\n\t\trunCancelableDataPathResult string\n\t\texpectedMessage string\n\t\texpectedSucceed bool\n\t}{\n\t\t{\n\t\t\tname: \"create data path failed\",\n\t\t\tpvbName: \"fake-name\",\n\t\t\tcreateDataPathFail: true,\n\t\t\texpectedMessage: \"Failed to create data path service for PVB fake-name: fake-create-data-path-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"init data path failed\",\n\t\t\tpvbName: \"fake-name\",\n\t\t\tinitDataPathErr: errors.New(\"fake-init-data-path-error\"),\n\t\t\texpectedMessage: \"Failed to init data path service for PVB fake-name: fake-init-data-path-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"run data path failed\",\n\t\t\tpvbName: \"fake-name\",\n\t\t\trunCancelableDataPathErr: errors.New(\"fake-run-data-path-error\"),\n\t\t\texpectedMessage: \"Failed to run data path service for PVB fake-name: fake-run-data-path-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tpvbName: \"fake-name\",\n\t\t\trunCancelableDataPathResult: \"fake-run-data-path-result\",\n\t\t\texpectedMessage: \"fake-run-data-path-result\",\n\t\t\texpectedSucceed: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfrHelper = &fakeRunHelper{\n\t\t\t\tinitErr: test.initDataPathErr,\n\t\t\t\trunCancelableDataPathErr: test.runCancelableDataPathErr,\n\t\t\t\trunCancelableDataPathResult: test.runCancelableDataPathResult,\n\t\t\t}\n\n\t\t\tif test.createDataPathFail {\n\t\t\t\tfuncCreateDataPathService = fakeCreateDataPathServiceWithErr\n\t\t\t} else {\n\t\t\t\tfuncCreateDataPathService = fakeCreateDataPathService\n\t\t\t}\n\n\t\t\tfuncExitWithMessage = frHelper.ExitWithMessage\n\n\t\t\ts := &podVolumeBackup{\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t\tcancelFunc: func() {},\n\t\t\t\tconfig: podVolumeBackupConfig{\n\t\t\t\t\tpvbName: test.pvbName,\n\t\t\t\t},\n\t\t\t}\n\n\t\t\ts.runDataPath()\n\n\t\t\tassert.Equal(t, test.expectedMessage, frHelper.exitMessage)\n\t\t\tassert.Equal(t, test.expectedSucceed, frHelper.succeed)\n\t\t})\n\t}\n}\n\ntype fakeCreateDataPathServiceHelper struct {\n\tfileStoreErr error\n\tsecretStoreErr error\n}\n\nfunc (fc *fakeCreateDataPathServiceHelper) NewNamespacedFileStore(_ ctlclient.Client, _ string, _ string, _ filesystem.Interface) (credentials.FileStore, error) {\n\treturn nil, fc.fileStoreErr\n}\n\nfunc (fc *fakeCreateDataPathServiceHelper) NewNamespacedSecretStore(_ ctlclient.Client, _ string) (credentials.SecretStore, error) {\n\treturn nil, fc.secretStoreErr\n}\n\nfunc TestCreateDataPathService(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tfileStoreErr error\n\t\tsecretStoreErr error\n\t\tmockGetInformer bool\n\t\tgetInformerErr error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"create credential file store error\",\n\t\t\tfileStoreErr: errors.New(\"fake-file-store-error\"),\n\t\t\texpectedError: \"error to create credential file store: fake-file-store-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"create credential secret store\",\n\t\t\tsecretStoreErr: errors.New(\"fake-secret-store-error\"),\n\t\t\texpectedError: \"error to create credential secret store: fake-secret-store-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"get informer error\",\n\t\t\tmockGetInformer: true,\n\t\t\tgetInformerErr: errors.New(\"fake-get-informer-error\"),\n\t\t\texpectedError: \"error to get controller-runtime informer from manager: fake-get-informer-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tmockGetInformer: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfcHelper := &fakeCreateDataPathServiceHelper{\n\t\t\t\tfileStoreErr: test.fileStoreErr,\n\t\t\t\tsecretStoreErr: test.secretStoreErr,\n\t\t\t}\n\n\t\t\tfuncNewCredentialFileStore = fcHelper.NewNamespacedFileStore\n\t\t\tfuncNewCredentialSecretStore = fcHelper.NewNamespacedSecretStore\n\n\t\t\tcache := cacheMock.NewCache(t)\n\t\t\tif test.mockGetInformer {\n\t\t\t\tcache.On(\"GetInformer\", mock.Anything, mock.Anything).Return(nil, test.getInformerErr)\n\t\t\t}\n\n\t\t\tfuncExitWithMessage = frHelper.ExitWithMessage\n\n\t\t\ts := &podVolumeBackup{\n\t\t\t\tcache: cache,\n\t\t\t}\n\n\t\t\t_, err := s.createDataPathService()\n\n\t\t\tif test.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.expectedError)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/cmd/cli/podvolume/podvolume.go", "content": "/*\nCopyright The Velero Contributors.\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n http://www.apache.org/licenses/LICENSE-2.0\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"context\"\n\n\t\"github.com/spf13/cobra\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n)\n\nfunc NewCommand(f client.Factory) *cobra.Command {\n\tcommand := &cobra.Command{\n\t\tUse: \"pod-volume\",\n\t\tShort: \"Run the velero pod volume backup/restore\",\n\t\tLong: \"Run the velero pod volume backup/restore\",\n\t\tHidden: true,\n\t}\n\n\tcommand.AddCommand(\n\t\tNewBackupCommand(f),\n\t\tNewRestoreCommand(f),\n\t)\n\n\treturn command\n}\n\ntype dataPathService interface {\n\tInit() error\n\tRunCancelableDataPath(context.Context) (string, error)\n\tShutdown()\n}\n" }, { "path": "pkg/cmd/cli/podvolume/restore.go", "content": "/*\nCopyright The Velero Contributors.\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n http://www.apache.org/licenses/LICENSE-2.0\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/bombsimon/logrusr/v3\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/spf13/cobra\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/fields\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"k8s.io/klog/v2\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/buildinfo\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/signals\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\t\"github.com/vmware-tanzu/velero/pkg/podvolume\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n\n\tctlcache \"sigs.k8s.io/controller-runtime/pkg/cache\"\n\tctlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n)\n\ntype podVolumeRestoreConfig struct {\n\tvolumePath string\n\tpvrName string\n\tcacheDir string\n\tresourceTimeout time.Duration\n}\n\nfunc NewRestoreCommand(f client.Factory) *cobra.Command {\n\tlogLevelFlag := logging.LogLevelFlag(logrus.InfoLevel)\n\tformatFlag := logging.NewFormatFlag()\n\n\tconfig := podVolumeRestoreConfig{}\n\n\tcommand := &cobra.Command{\n\t\tUse: \"restore\",\n\t\tShort: \"Run the velero pod volume restore\",\n\t\tLong: \"Run the velero pod volume restore\",\n\t\tHidden: true,\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tlogLevel := logLevelFlag.Parse()\n\t\t\tlogrus.Infof(\"Setting log-level to %s\", strings.ToUpper(logLevel.String()))\n\n\t\t\tlogger := logging.DefaultLogger(logLevel, formatFlag.Parse())\n\t\t\tlogger.Infof(\"Starting Velero pod volume restore %s (%s)\", buildinfo.Version, buildinfo.FormattedGitSHA())\n\n\t\t\tf.SetBasename(fmt.Sprintf(\"%s-%s\", c.Parent().Name(), c.Name()))\n\t\t\ts, err := newPodVolumeRestore(logger, f, config)\n\t\t\tif err != nil {\n\t\t\t\tkube.ExitPodWithMessage(logger, false, \"Failed to create pod volume restore, %v\", err)\n\t\t\t}\n\n\t\t\ts.run()\n\t\t},\n\t}\n\n\tcommand.Flags().Var(logLevelFlag, \"log-level\", fmt.Sprintf(\"The level at which to log. Valid values are %s.\", strings.Join(logLevelFlag.AllowedValues(), \", \")))\n\tcommand.Flags().Var(formatFlag, \"log-format\", fmt.Sprintf(\"The format for log output. Valid values are %s.\", strings.Join(formatFlag.AllowedValues(), \", \")))\n\tcommand.Flags().StringVar(&config.volumePath, \"volume-path\", config.volumePath, \"The full path of the volume to be restored\")\n\tcommand.Flags().StringVar(&config.pvrName, \"pod-volume-restore\", config.pvrName, \"The PVR name\")\n\tcommand.Flags().StringVar(&config.cacheDir, \"cache-volume-path\", config.cacheDir, \"The full path of the cache volume\")\n\tcommand.Flags().DurationVar(&config.resourceTimeout, \"resource-timeout\", config.resourceTimeout, \"How long to wait for resource processes which are not covered by other specific timeout parameters.\")\n\n\t_ = command.MarkFlagRequired(\"volume-path\")\n\t_ = command.MarkFlagRequired(\"pod-volume-restore\")\n\t_ = command.MarkFlagRequired(\"resource-timeout\")\n\n\tcommand.PreRunE = func(cmd *cobra.Command, args []string) error {\n\t\tif config.resourceTimeout <= 0 {\n\t\t\treturn errors.New(\"resource-timeout must be greater than 0\")\n\t\t}\n\t\tif config.volumePath == \"\" {\n\t\t\treturn errors.New(\"volume-path cannot be empty\")\n\t\t}\n\t\tif config.pvrName == \"\" {\n\t\t\treturn errors.New(\"pod-volume-restore name cannot be empty\")\n\t\t}\n\t\treturn nil\n\t}\n\n\treturn command\n}\n\ntype podVolumeRestore struct {\n\tlogger logrus.FieldLogger\n\tctx context.Context\n\tcancelFunc context.CancelFunc\n\tclient ctlclient.Client\n\tcache ctlcache.Cache\n\tnamespace string\n\tnodeName string\n\tconfig podVolumeRestoreConfig\n\tkubeClient kubernetes.Interface\n\tdataPathMgr *datapath.Manager\n}\n\nfunc newPodVolumeRestore(logger logrus.FieldLogger, factory client.Factory, config podVolumeRestoreConfig) (*podVolumeRestore, error) {\n\tctx, cancelFunc := context.WithCancel(context.Background())\n\n\tclientConfig, err := factory.ClientConfig()\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to create client config\")\n\t}\n\n\tctrl.SetLogger(logrusr.New(logger))\n\tklog.SetLogger(logrusr.New(logger)) // klog.Logger is used by k8s.io/client-go\n\n\tscheme := runtime.NewScheme()\n\tif err := velerov1api.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to add velero v1 scheme\")\n\t}\n\n\tif err := corev1api.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to add core v1 scheme\")\n\t}\n\n\tnodeName := os.Getenv(\"NODE_NAME\")\n\n\t// use a field selector to filter to only pods scheduled on this node.\n\tcacheOption := ctlcache.Options{\n\t\tScheme: scheme,\n\t\tByObject: map[ctlclient.Object]ctlcache.ByObject{\n\t\t\t&corev1api.Pod{}: {\n\t\t\t\tField: fields.Set{\"spec.nodeName\": nodeName}.AsSelector(),\n\t\t\t},\n\t\t\t&velerov1api.PodVolumeRestore{}: {\n\t\t\t\tField: fields.Set{\"metadata.namespace\": factory.Namespace()}.AsSelector(),\n\t\t\t},\n\t\t},\n\t}\n\n\tcli, err := ctlclient.New(clientConfig, ctlclient.Options{\n\t\tScheme: scheme,\n\t})\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to create client\")\n\t}\n\n\tvar cache ctlcache.Cache\n\tretry := 10\n\tfor {\n\t\tcache, err = ctlcache.New(clientConfig, cacheOption)\n\t\tif err == nil {\n\t\t\tbreak\n\t\t}\n\n\t\tretry--\n\t\tif retry == 0 {\n\t\t\tbreak\n\t\t}\n\n\t\tlogger.WithError(err).Warn(\"Failed to create client cache, need retry\")\n\n\t\ttime.Sleep(time.Second)\n\t}\n\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to create client cache\")\n\t}\n\n\ts := &podVolumeRestore{\n\t\tlogger: logger,\n\t\tctx: ctx,\n\t\tcancelFunc: cancelFunc,\n\t\tclient: cli,\n\t\tcache: cache,\n\t\tconfig: config,\n\t\tnamespace: factory.Namespace(),\n\t\tnodeName: nodeName,\n\t}\n\n\ts.kubeClient, err = factory.KubeClient()\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error to create kube client\")\n\t}\n\n\ts.dataPathMgr = datapath.NewManager(1)\n\n\treturn s, nil\n}\n\nvar funcCreateDataPathRestore = (*podVolumeRestore).createDataPathService\n\nfunc (s *podVolumeRestore) run() {\n\tsignals.CancelOnShutdown(s.cancelFunc, s.logger)\n\tgo func() {\n\t\tif err := s.cache.Start(s.ctx); err != nil {\n\t\t\ts.logger.WithError(err).Warn(\"error starting cache\")\n\t\t}\n\t}()\n\n\ts.runDataPath()\n}\n\nfunc (s *podVolumeRestore) runDataPath() {\n\ts.logger.Infof(\"Starting micro service in node %s for PVR %s\", s.nodeName, s.config.pvrName)\n\n\tdpService, err := funcCreateDataPathRestore(s)\n\tif err != nil {\n\t\ts.cancelFunc()\n\t\tfuncExitWithMessage(s.logger, false, \"Failed to create data path service for PVR %s: %v\", s.config.pvrName, err)\n\t\treturn\n\t}\n\n\ts.logger.Infof(\"Starting data path service %s\", s.config.pvrName)\n\n\terr = dpService.Init()\n\tif err != nil {\n\t\tdpService.Shutdown()\n\t\ts.cancelFunc()\n\t\tfuncExitWithMessage(s.logger, false, \"Failed to init data path service for PVR %s: %v\", s.config.pvrName, err)\n\t\treturn\n\t}\n\n\ts.logger.Infof(\"Running data path service %s\", s.config.pvrName)\n\n\tresult, err := dpService.RunCancelableDataPath(s.ctx)\n\tif err != nil {\n\t\tdpService.Shutdown()\n\t\ts.cancelFunc()\n\t\tfuncExitWithMessage(s.logger, false, \"Failed to run data path service for PVR %s: %v\", s.config.pvrName, err)\n\t\treturn\n\t}\n\n\ts.logger.WithField(\"PVR\", s.config.pvrName).Info(\"Data path service completed\")\n\n\tdpService.Shutdown()\n\n\ts.logger.WithField(\"PVR\", s.config.pvrName).Info(\"Data path service is shut down\")\n\n\ts.cancelFunc()\n\n\tfuncExitWithMessage(s.logger, true, result)\n}\n\nfunc (s *podVolumeRestore) createDataPathService() (dataPathService, error) {\n\tcredentialFileStore, err := funcNewCredentialFileStore(\n\t\ts.client,\n\t\ts.namespace,\n\t\tcredentials.DefaultStoreDirectory(),\n\t\tfilesystem.NewFileSystem(),\n\t)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error to create credential file store\")\n\t}\n\n\tcredSecretStore, err := funcNewCredentialSecretStore(s.client, s.namespace)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error to create credential secret store\")\n\t}\n\n\tcredGetter := &credentials.CredentialGetter{FromFile: credentialFileStore, FromSecret: credSecretStore}\n\n\tpvrInformer, err := s.cache.GetInformer(s.ctx, &velerov1api.PodVolumeRestore{})\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error to get controller-runtime informer from manager\")\n\t}\n\n\trepoEnsurer := repository.NewEnsurer(s.client, s.logger, s.config.resourceTimeout)\n\n\treturn podvolume.NewRestoreMicroService(s.ctx, s.client, s.kubeClient, s.config.pvrName, s.namespace, s.nodeName, datapath.AccessPoint{\n\t\tByPath: s.config.volumePath,\n\t\tVolMode: uploader.PersistentVolumeFilesystem,\n\t}, s.dataPathMgr, repoEnsurer, credGetter, pvrInformer, s.config.cacheDir, s.logger), nil\n}\n" }, { "path": "pkg/cmd/cli/podvolume/restore_test.go", "content": "/*\nCopyright The Velero Contributors.\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n http://www.apache.org/licenses/LICENSE-2.0\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"errors\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\n\tcacheMock \"github.com/vmware-tanzu/velero/pkg/cmd/cli/datamover/mocks\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc fakeCreateRestoreDataPathServiceWithErr(_ *podVolumeRestore) (dataPathService, error) {\n\treturn nil, errors.New(\"fake-create-data-path-error\")\n}\n\nfunc fakeCreateRestoreDataPathService(_ *podVolumeRestore) (dataPathService, error) {\n\treturn frHelper, nil\n}\n\nfunc TestRunRestoreDataPath(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvrName string\n\t\tcreateDataPathFail bool\n\t\tinitDataPathErr error\n\t\trunCancelableDataPathErr error\n\t\trunCancelableDataPathResult string\n\t\texpectedMessage string\n\t\texpectedSucceed bool\n\t}{\n\t\t{\n\t\t\tname: \"create data path failed\",\n\t\t\tpvrName: \"fake-name\",\n\t\t\tcreateDataPathFail: true,\n\t\t\texpectedMessage: \"Failed to create data path service for PVR fake-name: fake-create-data-path-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"init data path failed\",\n\t\t\tpvrName: \"fake-name\",\n\t\t\tinitDataPathErr: errors.New(\"fake-init-data-path-error\"),\n\t\t\texpectedMessage: \"Failed to init data path service for PVR fake-name: fake-init-data-path-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"run data path failed\",\n\t\t\tpvrName: \"fake-name\",\n\t\t\trunCancelableDataPathErr: errors.New(\"fake-run-data-path-error\"),\n\t\t\texpectedMessage: \"Failed to run data path service for PVR fake-name: fake-run-data-path-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tpvrName: \"fake-name\",\n\t\t\trunCancelableDataPathResult: \"fake-run-data-path-result\",\n\t\t\texpectedMessage: \"fake-run-data-path-result\",\n\t\t\texpectedSucceed: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfrHelper = &fakeRunHelper{\n\t\t\t\tinitErr: test.initDataPathErr,\n\t\t\t\trunCancelableDataPathErr: test.runCancelableDataPathErr,\n\t\t\t\trunCancelableDataPathResult: test.runCancelableDataPathResult,\n\t\t\t}\n\n\t\t\tif test.createDataPathFail {\n\t\t\t\tfuncCreateDataPathRestore = fakeCreateRestoreDataPathServiceWithErr\n\t\t\t} else {\n\t\t\t\tfuncCreateDataPathRestore = fakeCreateRestoreDataPathService\n\t\t\t}\n\n\t\t\tfuncExitWithMessage = frHelper.ExitWithMessage\n\n\t\t\ts := &podVolumeRestore{\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t\tcancelFunc: func() {},\n\t\t\t\tconfig: podVolumeRestoreConfig{\n\t\t\t\t\tpvrName: test.pvrName,\n\t\t\t\t},\n\t\t\t}\n\n\t\t\ts.runDataPath()\n\n\t\t\tassert.Equal(t, test.expectedMessage, frHelper.exitMessage)\n\t\t\tassert.Equal(t, test.expectedSucceed, frHelper.succeed)\n\t\t})\n\t}\n}\n\nfunc TestCreateRestoreDataPathService(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tfileStoreErr error\n\t\tsecretStoreErr error\n\t\tmockGetInformer bool\n\t\tgetInformerErr error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"create credential file store error\",\n\t\t\tfileStoreErr: errors.New(\"fake-file-store-error\"),\n\t\t\texpectedError: \"error to create credential file store: fake-file-store-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"create credential secret store\",\n\t\t\tsecretStoreErr: errors.New(\"fake-secret-store-error\"),\n\t\t\texpectedError: \"error to create credential secret store: fake-secret-store-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"get informer error\",\n\t\t\tmockGetInformer: true,\n\t\t\tgetInformerErr: errors.New(\"fake-get-informer-error\"),\n\t\t\texpectedError: \"error to get controller-runtime informer from manager: fake-get-informer-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tmockGetInformer: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfcHelper := &fakeCreateDataPathServiceHelper{\n\t\t\t\tfileStoreErr: test.fileStoreErr,\n\t\t\t\tsecretStoreErr: test.secretStoreErr,\n\t\t\t}\n\n\t\t\tfuncNewCredentialFileStore = fcHelper.NewNamespacedFileStore\n\t\t\tfuncNewCredentialSecretStore = fcHelper.NewNamespacedSecretStore\n\n\t\t\tcache := cacheMock.NewCache(t)\n\t\t\tif test.mockGetInformer {\n\t\t\t\tcache.On(\"GetInformer\", mock.Anything, mock.Anything).Return(nil, test.getInformerErr)\n\t\t\t}\n\n\t\t\tfuncExitWithMessage = frHelper.ExitWithMessage\n\n\t\t\ts := &podVolumeRestore{\n\t\t\t\tcache: cache,\n\t\t\t}\n\n\t\t\t_, err := s.createDataPathService()\n\n\t\t\tif test.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.expectedError)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/cmd/cli/repo/get.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage repo\n\nimport (\n\t\"context\"\n\n\t\"github.com/spf13/cobra\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\tctrlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/output\"\n)\n\nfunc NewGetCommand(f client.Factory, use string) *cobra.Command {\n\tvar listOptions metav1.ListOptions\n\n\tc := &cobra.Command{\n\t\tUse: use,\n\t\tShort: \"Get repositories\",\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\terr := output.ValidateFlags(c)\n\t\t\tcmd.CheckError(err)\n\n\t\t\tcrClient, err := f.KubebuilderClient()\n\t\t\tcmd.CheckError(err)\n\n\t\t\trepos := new(api.BackupRepositoryList)\n\t\t\tif len(args) > 0 {\n\t\t\t\tfor _, name := range args {\n\t\t\t\t\trepo := new(api.BackupRepository)\n\t\t\t\t\terr := crClient.Get(context.TODO(), ctrlclient.ObjectKey{Namespace: f.Namespace(), Name: name}, repo)\n\t\t\t\t\tcmd.CheckError(err)\n\t\t\t\t\trepos.Items = append(repos.Items, *repo)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tselector := labels.NewSelector()\n\t\t\t\tif listOptions.LabelSelector != \"\" {\n\t\t\t\t\tselector, err = labels.Parse(listOptions.LabelSelector)\n\t\t\t\t\tcmd.CheckError(err)\n\t\t\t\t}\n\t\t\t\terr = crClient.List(context.TODO(), repos, &ctrlclient.ListOptions{LabelSelector: selector})\n\t\t\t\tcmd.CheckError(err)\n\t\t\t}\n\n\t\t\t_, err = output.PrintWithFormat(c, repos)\n\t\t\tcmd.CheckError(err)\n\t\t},\n\t}\n\n\tc.Flags().StringVarP(&listOptions.LabelSelector, \"selector\", \"l\", listOptions.LabelSelector, \"Only show items matching this label selector.\")\n\n\toutput.BindFlags(c.Flags())\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/repo/repo.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage repo\n\nimport (\n\t\"github.com/spf13/cobra\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n)\n\nfunc NewCommand(f client.Factory) *cobra.Command {\n\tc := &cobra.Command{\n\t\tUse: \"repo\",\n\t\tShort: \"Work with repositories\",\n\t\tLong: \"Work with repositories\",\n\t}\n\n\tc.AddCommand(\n\t\tNewGetCommand(f, \"get\"),\n\t)\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/repomantenance/maintenance.go", "content": "package repomantenance\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/bombsimon/logrusr/v3\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/spf13/cobra\"\n\t\"github.com/spf13/pflag\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerocli \"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n\n\trepokey \"github.com/vmware-tanzu/velero/pkg/repository/keys\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/maintenance\"\n\trepomanager \"github.com/vmware-tanzu/velero/pkg/repository/manager\"\n)\n\ntype Options struct {\n\tRepoName string\n\tBackupStorageLocation string\n\tRepoType string\n\tResourceTimeout time.Duration\n\tLogLevelFlag *logging.LevelFlag\n\tFormatFlag *logging.FormatFlag\n}\n\nfunc (o *Options) BindFlags(flags *pflag.FlagSet) {\n\tflags.StringVar(&o.RepoName, \"repo-name\", \"\", \"namespace of the pod/volume that the snapshot is for\")\n\tflags.StringVar(&o.BackupStorageLocation, \"backup-storage-location\", \"\", \"backup's storage location name\")\n\tflags.StringVar(&o.RepoType, \"repo-type\", velerov1api.BackupRepositoryTypeKopia, \"type of the repository where the snapshot is stored\")\n\tflags.Var(o.LogLevelFlag, \"log-level\", fmt.Sprintf(\"The level at which to log. Valid values are %s.\", strings.Join(o.LogLevelFlag.AllowedValues(), \", \")))\n\tflags.Var(o.FormatFlag, \"log-format\", fmt.Sprintf(\"The format for log output. Valid values are %s.\", strings.Join(o.FormatFlag.AllowedValues(), \", \")))\n}\n\nfunc NewCommand(f velerocli.Factory) *cobra.Command {\n\to := &Options{\n\t\tLogLevelFlag: logging.LogLevelFlag(logrus.InfoLevel),\n\t\tFormatFlag: logging.NewFormatFlag(),\n\t}\n\tcmd := &cobra.Command{\n\t\tUse: \"repo-maintenance\",\n\t\tHidden: true,\n\t\tShort: \"VELERO INTERNAL COMMAND ONLY - not intended to be run directly by users\",\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\to.Run(f)\n\t\t},\n\t}\n\n\to.BindFlags(cmd.Flags())\n\treturn cmd\n}\n\nfunc (o *Options) Run(f velerocli.Factory) {\n\tlogger := logging.DefaultLogger(o.LogLevelFlag.Parse(), o.FormatFlag.Parse())\n\tlogger.SetOutput(os.Stdout)\n\n\tctrl.SetLogger(logrusr.New(logger))\n\n\tpruneError := o.runRepoPrune(f, f.Namespace(), logger)\n\tdefer func() {\n\t\tif pruneError != nil {\n\t\t\tos.Exit(1)\n\t\t}\n\t}()\n\n\tif pruneError != nil {\n\t\tos.Stdout.WriteString(fmt.Sprintf(\"%s%v\", maintenance.TerminationLogIndicator, pruneError))\n\t}\n}\n\nfunc (o *Options) initClient(f velerocli.Factory) (client.Client, error) {\n\tscheme := runtime.NewScheme()\n\terr := velerov1api.AddToScheme(scheme)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to add velero scheme\")\n\t}\n\n\terr = corev1api.AddToScheme(scheme)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to add api core scheme\")\n\t}\n\n\tconfig, err := f.ClientConfig()\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to get client config\")\n\t}\n\n\tcli, err := client.New(config, client.Options{\n\t\tScheme: scheme,\n\t})\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to create client\")\n\t}\n\n\treturn cli, nil\n}\n\nfunc initRepoManager(namespace string, cli client.Client, kubeClient kubernetes.Interface, logger logrus.FieldLogger) (repomanager.Manager, error) {\n\t// ensure the repo key secret is set up\n\tif err := repokey.EnsureCommonRepositoryKey(kubeClient.CoreV1(), namespace); err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to ensure repository key\")\n\t}\n\n\trepoLocker := repository.NewRepoLocker()\n\n\tcredentialFileStore, err := credentials.NewNamespacedFileStore(\n\t\tcli,\n\t\tnamespace,\n\t\tcredentials.DefaultStoreDirectory(),\n\t\tfilesystem.NewFileSystem(),\n\t)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to create namespaced file store\")\n\t}\n\n\tcredentialSecretStore, err := credentials.NewNamespacedSecretStore(cli, namespace)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to create namespaced secret store\")\n\t}\n\n\treturn repomanager.NewManager(\n\t\tnamespace,\n\t\tcli,\n\t\trepoLocker,\n\t\tcredentialFileStore,\n\t\tcredentialSecretStore,\n\t\tlogger,\n\t), nil\n}\n\nfunc (o *Options) runRepoPrune(f velerocli.Factory, namespace string, logger logrus.FieldLogger) error {\n\tcli, err := o.initClient(f)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tkubeClient, err := f.KubeClient()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tvar repo *velerov1api.BackupRepository\n\tretry := 10\n\tfor {\n\t\trepo, err = repository.GetBackupRepository(context.Background(), cli, namespace,\n\t\t\trepository.BackupRepositoryKey{\n\t\t\t\tVolumeNamespace: o.RepoName,\n\t\t\t\tBackupLocation: o.BackupStorageLocation,\n\t\t\t\tRepositoryType: o.RepoType,\n\t\t\t}, true)\n\t\tif err == nil {\n\t\t\tbreak\n\t\t}\n\n\t\tretry--\n\t\tif retry == 0 {\n\t\t\tbreak\n\t\t}\n\n\t\tlogger.WithError(err).Warn(\"Failed to retrieve backup repo, need retry\")\n\n\t\ttime.Sleep(time.Second)\n\t}\n\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"failed to get backup repository\")\n\t}\n\n\tmanager, err := initRepoManager(namespace, cli, kubeClient, logger)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\terr = manager.PruneRepo(repo)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"failed to prune repo\")\n\t}\n\n\treturn nil\n}\n" }, { "path": "pkg/cmd/cli/restore/create.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"sort\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/spf13/cobra\"\n\t\"github.com/spf13/pflag\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/client-go/tools/cache\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/resourcemodifiers\"\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/flag\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/output\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/velero/restore\"\n)\n\nfunc NewCreateCommand(f client.Factory, use string) *cobra.Command {\n\to := NewCreateOptions()\n\n\tc := &cobra.Command{\n\t\tUse: use + \" [RESTORE_NAME] [--from-backup BACKUP_NAME | --from-schedule SCHEDULE_NAME]\",\n\t\tShort: \"Create a restore\",\n\t\tExample: ` # Create a restore named \"restore-1\" from backup \"backup-1\".\n velero restore create restore-1 --from-backup backup-1\n\n # Create a restore with a default name (\"backup-1-\") from backup \"backup-1\".\n velero restore create --from-backup backup-1\n \n # Create a restore from the latest successful backup triggered by schedule \"schedule-1\".\n velero restore create --from-schedule schedule-1\n\n # Create a restore from the latest successful OR partially-failed backup triggered by schedule \"schedule-1\".\n velero restore create --from-schedule schedule-1 --allow-partially-failed\n\n # Create a restore for only persistentvolumeclaims and persistentvolumes within a backup.\n velero restore create --from-backup backup-2 --include-resources persistentvolumeclaims,persistentvolumes`,\n\t\tArgs: cobra.MaximumNArgs(1),\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tcmd.CheckError(o.Complete(args, f))\n\t\t\tcmd.CheckError(o.Validate(c, args, f))\n\t\t\tcmd.CheckError(o.Run(c, f))\n\t\t},\n\t}\n\n\to.BindFlags(c.Flags())\n\toutput.BindFlags(c.Flags())\n\toutput.ClearOutputFlagDefault(c)\n\n\treturn c\n}\n\ntype CreateOptions struct {\n\tBackupName string\n\tScheduleName string\n\tRestoreName string\n\tRestoreVolumes flag.OptionalBool\n\tPreserveNodePorts flag.OptionalBool\n\tLabels flag.Map\n\tAnnotations flag.Map\n\tIncludeNamespaces flag.StringArray\n\tExcludeNamespaces flag.StringArray\n\tExistingResourcePolicy string\n\tIncludeResources flag.StringArray\n\tExcludeResources flag.StringArray\n\tStatusIncludeResources flag.StringArray\n\tStatusExcludeResources flag.StringArray\n\tNamespaceMappings flag.Map\n\tSelector flag.LabelSelector\n\tOrSelector flag.OrLabelSelector\n\tIncludeClusterResources flag.OptionalBool\n\tWait bool\n\tAllowPartiallyFailed flag.OptionalBool\n\tItemOperationTimeout time.Duration\n\tResourceModifierConfigMap string\n\tWriteSparseFiles flag.OptionalBool\n\tParallelFilesDownload int\n\tclient kbclient.WithWatch\n}\n\nfunc NewCreateOptions() *CreateOptions {\n\treturn &CreateOptions{\n\t\tLabels: flag.NewMap(),\n\t\tAnnotations: flag.NewMap(),\n\t\tIncludeNamespaces: flag.NewStringArray(\"*\"),\n\t\tNamespaceMappings: flag.NewMap().WithEntryDelimiter(',').WithKeyValueDelimiter(':'),\n\t\tRestoreVolumes: flag.NewOptionalBool(nil),\n\t\tPreserveNodePorts: flag.NewOptionalBool(nil),\n\t\tIncludeClusterResources: flag.NewOptionalBool(nil),\n\t\tWriteSparseFiles: flag.NewOptionalBool(nil),\n\t}\n}\n\nfunc (o *CreateOptions) BindFlags(flags *pflag.FlagSet) {\n\tflags.StringVar(&o.BackupName, \"from-backup\", \"\", \"Backup to restore from\")\n\tflags.StringVar(&o.ScheduleName, \"from-schedule\", \"\", \"Schedule to restore from\")\n\tflags.Var(&o.IncludeNamespaces, \"include-namespaces\", \"Namespaces to include in the restore (use '*' for all namespaces)\")\n\tflags.Var(&o.ExcludeNamespaces, \"exclude-namespaces\", \"Namespaces to exclude from the restore.\")\n\tflags.Var(&o.NamespaceMappings, \"namespace-mappings\", \"Namespace mappings from name in the backup to desired restored name in the form src1:dst1,src2:dst2,...\")\n\tflags.Var(&o.Labels, \"labels\", \"Labels to apply to the restore.\")\n\tflags.Var(&o.Annotations, \"annotations\", \"Annotations to apply to the restore.\")\n\tflags.Var(&o.IncludeResources, \"include-resources\", \"Resources to include in the restore, formatted as resource.group, such as storageclasses.storage.k8s.io (use '*' for all resources).\")\n\tflags.Var(&o.ExcludeResources, \"exclude-resources\", \"Resources to exclude from the restore, formatted as resource.group, such as storageclasses.storage.k8s.io.\")\n\tflags.StringVar(&o.ExistingResourcePolicy, \"existing-resource-policy\", \"\", \"Restore Policy to be used during the restore workflow, can be - none or update\")\n\tflags.Var(&o.StatusIncludeResources, \"status-include-resources\", \"Resources to include in the restore status, formatted as resource.group, such as storageclasses.storage.k8s.io.\")\n\tflags.Var(&o.StatusExcludeResources, \"status-exclude-resources\", \"Resources to exclude from the restore status, formatted as resource.group, such as storageclasses.storage.k8s.io.\")\n\tflags.VarP(&o.Selector, \"selector\", \"l\", \"Only restore resources matching this label selector.\")\n\tflags.Var(&o.OrSelector, \"or-selector\", \"Restore resources matching at least one of the label selector from the list. Label selectors should be separated by ' or '. For example, foo=bar or app=nginx\")\n\tflags.DurationVar(&o.ItemOperationTimeout, \"item-operation-timeout\", o.ItemOperationTimeout, \"How long to wait for async plugin operations before timeout.\")\n\tf := flags.VarPF(&o.RestoreVolumes, \"restore-volumes\", \"\", \"Whether to restore volumes from snapshots.\")\n\t// this allows the user to just specify \"--restore-volumes\" as shorthand for \"--restore-volumes=true\"\n\t// like a normal bool flag\n\tf.NoOptDefVal = cmd.TRUE\n\n\tf = flags.VarPF(&o.PreserveNodePorts, \"preserve-nodeports\", \"\", \"Whether to preserve nodeports of Services when restoring.\")\n\t// this allows the user to just specify \"--preserve-nodeports\" as shorthand for \"--preserve-nodeports=true\"\n\t// like a normal bool flag\n\tf.NoOptDefVal = cmd.TRUE\n\n\tf = flags.VarPF(&o.IncludeClusterResources, \"include-cluster-resources\", \"\", \"Include cluster-scoped resources in the restore.\")\n\tf.NoOptDefVal = cmd.TRUE\n\n\tf = flags.VarPF(&o.AllowPartiallyFailed, \"allow-partially-failed\", \"\", \"If using --from-schedule, whether to consider PartiallyFailed backups when looking for the most recent one. This flag has no effect if not using --from-schedule.\")\n\tf.NoOptDefVal = cmd.TRUE\n\n\tflags.BoolVarP(&o.Wait, \"wait\", \"w\", o.Wait, \"Wait for the operation to complete.\")\n\n\tflags.StringVar(&o.ResourceModifierConfigMap, \"resource-modifier-configmap\", \"\", \"Reference to the resource modifier configmap that restore will use\")\n\n\tf = flags.VarPF(&o.WriteSparseFiles, \"write-sparse-files\", \"\", \"Whether to write sparse files during restoring volumes\")\n\tf.NoOptDefVal = cmd.TRUE\n\n\tflags.IntVar(&o.ParallelFilesDownload, \"parallel-files-download\", 0, \"The number of restore operations to run in parallel. If set to 0, the default parallelism will be the number of CPUs for the node that node agent pod is running.\")\n}\n\nfunc (o *CreateOptions) Complete(args []string, f client.Factory) error {\n\tif len(args) == 1 {\n\t\to.RestoreName = args[0]\n\t} else {\n\t\tsourceName := o.BackupName\n\t\tif o.ScheduleName != \"\" {\n\t\t\tsourceName = o.ScheduleName\n\t\t}\n\n\t\to.RestoreName = fmt.Sprintf(\"%s-%s\", sourceName, time.Now().Format(\"20060102150405\"))\n\t}\n\n\tclient, err := f.KubebuilderWatchClient()\n\tif err != nil {\n\t\treturn err\n\t}\n\to.client = client\n\n\treturn nil\n}\n\nfunc (o *CreateOptions) Validate(c *cobra.Command, args []string, f client.Factory) error {\n\tif o.BackupName != \"\" && o.ScheduleName != \"\" {\n\t\treturn errors.New(\"either a backup or schedule must be specified, but not both\")\n\t}\n\n\tif o.BackupName == \"\" && o.ScheduleName == \"\" {\n\t\treturn errors.New(\"either a backup or schedule must be specified, but not both\")\n\t}\n\n\tif err := output.ValidateFlags(c); err != nil {\n\t\treturn err\n\t}\n\n\tif o.client == nil {\n\t\t// This should never happen\n\t\treturn errors.New(\"Velero client is not set; unable to proceed\")\n\t}\n\n\tif o.Selector.LabelSelector != nil && o.OrSelector.OrLabelSelectors != nil {\n\t\treturn errors.New(\"either a 'selector' or an 'or-selector' can be specified, but not both\")\n\t}\n\n\tif len(o.ExistingResourcePolicy) > 0 && !restore.IsResourcePolicyValid(o.ExistingResourcePolicy) {\n\t\treturn errors.New(\"existing-resource-policy has invalid value, it accepts only none, update as value\")\n\t}\n\n\tif o.ParallelFilesDownload < 0 {\n\t\treturn errors.New(\"parallel-files-download cannot be negative\")\n\t}\n\n\tswitch {\n\tcase o.BackupName != \"\":\n\t\tbackup := new(api.Backup)\n\t\tif err := o.client.Get(context.TODO(), kbclient.ObjectKey{Namespace: f.Namespace(), Name: o.BackupName}, backup); err != nil {\n\t\t\treturn err\n\t\t}\n\tcase o.ScheduleName != \"\":\n\t\tbackupList := new(api.BackupList)\n\t\terr := o.client.List(context.TODO(), backupList, &kbclient.ListOptions{\n\t\t\tLabelSelector: labels.SelectorFromSet(map[string]string{api.ScheduleNameLabel: o.ScheduleName}),\n\t\t\tNamespace: f.Namespace(),\n\t\t})\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif len(backupList.Items) == 0 {\n\t\t\treturn errors.Errorf(\"No backups found for the schedule %s\", o.ScheduleName)\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// mostRecentBackup returns the backup with the most recent start timestamp that has a phase that's\n// in the provided list of allowed phases.\nfunc mostRecentBackup(backups []api.Backup, allowedPhases ...api.BackupPhase) *api.Backup {\n\t// sort the backups in descending order of start time (i.e. most recent to least recent)\n\tsort.Slice(backups, func(i, j int) bool {\n\t\t// Use .After() because we want descending sort.\n\n\t\tvar iStartTime, jStartTime time.Time\n\t\tif backups[i].Status.StartTimestamp != nil {\n\t\t\tiStartTime = backups[i].Status.StartTimestamp.Time\n\t\t}\n\t\tif backups[j].Status.StartTimestamp != nil {\n\t\t\tjStartTime = backups[j].Status.StartTimestamp.Time\n\t\t}\n\t\treturn iStartTime.After(jStartTime)\n\t})\n\n\t// create a map of the allowed phases for easy lookup below\n\tphases := map[api.BackupPhase]struct{}{}\n\tfor _, phase := range allowedPhases {\n\t\tphases[phase] = struct{}{}\n\t}\n\n\tvar res *api.Backup\n\tfor i, backup := range backups {\n\t\t// if the backup's phase is one of the allowable ones, record\n\t\t// the backup and break the loop so we can return it\n\t\tif _, ok := phases[backup.Status.Phase]; ok {\n\t\t\tres = &backups[i]\n\t\t\tbreak\n\t\t}\n\t}\n\n\treturn res\n}\n\nfunc (o *CreateOptions) Run(c *cobra.Command, f client.Factory) error {\n\tif o.client == nil {\n\t\t// This should never happen\n\t\treturn errors.New(\"Velero client is not set; unable to proceed\")\n\t}\n\n\t// if --allow-partially-failed was specified, look up the most recent Completed or\n\t// PartiallyFailed backup for the provided schedule, and use that specific backup\n\t// to restore from.\n\tif o.ScheduleName != \"\" && boolptr.IsSetToTrue(o.AllowPartiallyFailed.Value) {\n\t\tbackupList := new(api.BackupList)\n\t\terr := o.client.List(context.TODO(), backupList, &kbclient.ListOptions{\n\t\t\tLabelSelector: labels.SelectorFromSet(map[string]string{api.ScheduleNameLabel: o.ScheduleName}),\n\t\t\tNamespace: f.Namespace(),\n\t\t})\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// if we find a Completed or PartiallyFailed backup for the schedule, restore specifically from that backup. If we don't\n\t\t// find one, proceed as-is -- the Velero server will handle validation.\n\t\tif backup := mostRecentBackup(backupList.Items, api.BackupPhaseCompleted, api.BackupPhasePartiallyFailed); backup != nil {\n\t\t\t// TODO(sk): this is kind of a hack -- we should revisit this and probably\n\t\t\t// move this logic to the server side or otherwise solve this problem.\n\t\t\to.BackupName = backup.Name\n\t\t\to.ScheduleName = \"\"\n\t\t}\n\t}\n\n\tvar resModifiers *corev1api.TypedLocalObjectReference\n\n\tif o.ResourceModifierConfigMap != \"\" {\n\t\tresModifiers = &corev1api.TypedLocalObjectReference{\n\t\t\t// Group for core API is \"\"\n\t\t\tAPIGroup: &corev1api.SchemeGroupVersion.Group,\n\t\t\tKind: resourcemodifiers.ConfigmapRefType,\n\t\t\tName: o.ResourceModifierConfigMap,\n\t\t}\n\t}\n\n\trestore := &api.Restore{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: f.Namespace(),\n\t\t\tName: o.RestoreName,\n\t\t\tLabels: o.Labels.Data(),\n\t\t\tAnnotations: o.Annotations.Data(),\n\t\t},\n\t\tSpec: api.RestoreSpec{\n\t\t\tBackupName: o.BackupName,\n\t\t\tScheduleName: o.ScheduleName,\n\t\t\tIncludedNamespaces: o.IncludeNamespaces,\n\t\t\tExcludedNamespaces: o.ExcludeNamespaces,\n\t\t\tIncludedResources: o.IncludeResources,\n\t\t\tExcludedResources: o.ExcludeResources,\n\t\t\tExistingResourcePolicy: api.PolicyType(o.ExistingResourcePolicy),\n\t\t\tNamespaceMapping: o.NamespaceMappings.Data(),\n\t\t\tLabelSelector: o.Selector.LabelSelector,\n\t\t\tOrLabelSelectors: o.OrSelector.OrLabelSelectors,\n\t\t\tRestorePVs: o.RestoreVolumes.Value,\n\t\t\tPreserveNodePorts: o.PreserveNodePorts.Value,\n\t\t\tIncludeClusterResources: o.IncludeClusterResources.Value,\n\t\t\tResourceModifier: resModifiers,\n\t\t\tItemOperationTimeout: metav1.Duration{\n\t\t\t\tDuration: o.ItemOperationTimeout,\n\t\t\t},\n\t\t\tUploaderConfig: &api.UploaderConfigForRestore{\n\t\t\t\tWriteSparseFiles: o.WriteSparseFiles.Value,\n\t\t\t\tParallelFilesDownload: o.ParallelFilesDownload,\n\t\t\t},\n\t\t},\n\t}\n\n\tif len([]string(o.StatusIncludeResources)) > 0 {\n\t\trestore.Spec.RestoreStatus = &api.RestoreStatusSpec{\n\t\t\tIncludedResources: o.StatusIncludeResources,\n\t\t\tExcludedResources: o.StatusExcludeResources,\n\t\t}\n\t}\n\n\tif printed, err := output.PrintWithFormat(c, restore); printed || err != nil {\n\t\treturn err\n\t}\n\n\tvar updates chan *api.Restore\n\tif o.Wait {\n\t\tstop := make(chan struct{})\n\t\tdefer close(stop)\n\n\t\tupdates = make(chan *api.Restore)\n\n\t\tlw := kube.InternalLW{\n\t\t\tClient: o.client,\n\t\t\tNamespace: f.Namespace(),\n\t\t\tObjectList: new(api.RestoreList),\n\t\t}\n\t\trestoreInformer := cache.NewSharedInformer(&lw, &api.Restore{}, time.Second)\n\n\t\t_, _ = restoreInformer.AddEventHandler(\n\t\t\tcache.FilteringResourceEventHandler{\n\t\t\t\tFilterFunc: func(obj any) bool {\n\t\t\t\t\trestore, ok := obj.(*api.Restore)\n\t\t\t\t\tif !ok {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\t\t\t\t\treturn restore.Name == o.RestoreName\n\t\t\t\t},\n\t\t\t\tHandler: cache.ResourceEventHandlerFuncs{\n\t\t\t\t\tUpdateFunc: func(_, obj any) {\n\t\t\t\t\t\trestore, ok := obj.(*api.Restore)\n\t\t\t\t\t\tif !ok {\n\t\t\t\t\t\t\treturn\n\t\t\t\t\t\t}\n\t\t\t\t\t\tupdates <- restore\n\t\t\t\t\t},\n\t\t\t\t\tDeleteFunc: func(obj any) {\n\t\t\t\t\t\trestore, ok := obj.(*api.Restore)\n\t\t\t\t\t\tif !ok {\n\t\t\t\t\t\t\treturn\n\t\t\t\t\t\t}\n\t\t\t\t\t\tupdates <- restore\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t)\n\t\tgo restoreInformer.Run(stop)\n\t}\n\n\terr := o.client.Create(context.TODO(), restore, &kbclient.CreateOptions{})\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfmt.Printf(\"Restore request %q submitted successfully.\\n\", restore.Name)\n\tif o.Wait {\n\t\tfmt.Println(\"Waiting for restore to complete. You may safely press ctrl-c to stop waiting - your restore will continue in the background.\")\n\t\tticker := time.NewTicker(time.Second)\n\t\tdefer ticker.Stop()\n\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-ticker.C:\n\t\t\t\tfmt.Print(\".\")\n\t\t\tcase restore, ok := <-updates:\n\t\t\t\tif !ok {\n\t\t\t\t\tfmt.Println(\"\\nError waiting: unable to watch restores.\")\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\n\t\t\t\tif restore.Status.Phase == api.RestorePhaseFailedValidation || restore.Status.Phase == api.RestorePhaseCompleted ||\n\t\t\t\t\trestore.Status.Phase == api.RestorePhasePartiallyFailed || restore.Status.Phase == api.RestorePhaseFailed {\n\t\t\t\t\tfmt.Printf(\"\\nRestore completed with status: %s. You may check for more information using the commands `velero restore describe %s` and `velero restore logs %s`.\\n\", restore.Status.Phase, restore.Name, restore.Name)\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\t// Not waiting\n\n\tfmt.Printf(\"Run `velero restore describe %s` or `velero restore logs %s` for more details.\\n\", restore.Name, restore.Name)\n\n\treturn nil\n}\n" }, { "path": "pkg/cmd/cli/restore/create_test.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/spf13/pflag\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tcontrollerclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\tcmdtest \"github.com/vmware-tanzu/velero/pkg/cmd/test\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestMostRecentBackup(t *testing.T) {\n\tbackups := []velerov1api.Backup{\n\t\t*builder.ForBackup(cmdtest.VeleroNameSpace, \"backup0\").StartTimestamp(time.Now().Add(3 * time.Second)).Phase(velerov1api.BackupPhaseDeleting).Result(),\n\t\t*builder.ForBackup(cmdtest.VeleroNameSpace, \"backup1\").StartTimestamp(time.Now().Add(time.Second)).Phase(velerov1api.BackupPhaseCompleted).Result(),\n\t\t*builder.ForBackup(cmdtest.VeleroNameSpace, \"backup2\").StartTimestamp(time.Now().Add(2 * time.Second)).Phase(velerov1api.BackupPhasePartiallyFailed).Result(),\n\t}\n\n\texpectedBackup := builder.ForBackup(cmdtest.VeleroNameSpace, \"backup2\").StartTimestamp(time.Now().Add(2 * time.Second)).Phase(velerov1api.BackupPhasePartiallyFailed).Result()\n\n\tresultBackup := mostRecentBackup(backups, velerov1api.BackupPhaseCompleted, velerov1api.BackupPhasePartiallyFailed)\n\n\trequire.Equal(t, expectedBackup.Name, resultBackup.Name)\n}\n\nfunc TestCreateCommand(t *testing.T) {\n\tname := \"nameToBeCreated\"\n\targs := []string{name}\n\n\tt.Run(\"create a backup create command with full options except fromSchedule and wait, then run by create option\", func(t *testing.T) {\n\t\t// create a factory\n\t\tf := &factorymocks.Factory{}\n\n\t\t// create command\n\t\tcmd := NewCreateCommand(f, \"\")\n\t\trequire.Equal(t, \"Create a restore\", cmd.Short)\n\n\t\tbackupName := \"backup1\"\n\t\tscheduleName := \"schedule1\"\n\t\trestoreVolumes := \"true\"\n\t\tpreserveNodePorts := \"true\"\n\t\tlabels := \"c=foo\"\n\t\tannotations := \"ann=foo\"\n\t\tincludeNamespaces := \"app1,app2\"\n\t\texcludeNamespaces := \"pod1,pod2,pod3\"\n\t\texistingResourcePolicy := \"none\"\n\t\tincludeResources := \"sc,sts\"\n\t\texcludeResources := \"job\"\n\t\tstatusIncludeResources := \"sc,sts\"\n\t\tstatusExcludeResources := \"job\"\n\t\tnamespaceMappings := \"a:b\"\n\t\tselector := \"foo=bar\"\n\t\tincludeClusterResources := \"true\"\n\t\tallowPartiallyFailed := \"true\"\n\t\titemOperationTimeout := \"10m0s\"\n\t\twriteSparseFiles := \"true\"\n\t\tparallel := 2\n\t\tflags := new(pflag.FlagSet)\n\t\to := NewCreateOptions()\n\t\to.BindFlags(flags)\n\n\t\tflags.Parse([]string{\"--from-backup\", backupName})\n\t\tflags.Parse([]string{\"--from-schedule\", scheduleName})\n\t\tflags.Parse([]string{\"--restore-volumes\", restoreVolumes})\n\t\tflags.Parse([]string{\"--preserve-nodeports\", preserveNodePorts})\n\t\tflags.Parse([]string{\"--labels\", labels})\n\t\tflags.Parse([]string{\"--annotations\", annotations})\n\t\tflags.Parse([]string{\"--existing-resource-policy\", existingResourcePolicy})\n\t\tflags.Parse([]string{\"--include-namespaces\", includeNamespaces})\n\t\tflags.Parse([]string{\"--exclude-namespaces\", excludeNamespaces})\n\t\tflags.Parse([]string{\"--include-resources\", includeResources})\n\t\tflags.Parse([]string{\"--exclude-resources\", excludeResources})\n\t\tflags.Parse([]string{\"--status-include-resources\", statusIncludeResources})\n\t\tflags.Parse([]string{\"--status-exclude-resources\", statusExcludeResources})\n\t\tflags.Parse([]string{\"--namespace-mappings\", namespaceMappings})\n\t\tflags.Parse([]string{\"--selector\", selector})\n\t\tflags.Parse([]string{\"--include-cluster-resources\", includeClusterResources})\n\t\tflags.Parse([]string{\"--allow-partially-failed\", allowPartiallyFailed})\n\t\tflags.Parse([]string{\"--item-operation-timeout\", itemOperationTimeout})\n\t\tflags.Parse([]string{\"--write-sparse-files\", writeSparseFiles})\n\t\tflags.Parse([]string{\"--parallel-files-download\", \"2\"})\n\t\tclient := velerotest.NewFakeControllerRuntimeClient(t).(kbclient.WithWatch)\n\n\t\tf.On(\"Namespace\").Return(mock.Anything)\n\t\tf.On(\"KubebuilderWatchClient\").Return(client, nil)\n\n\t\t//Complete\n\t\te := o.Complete(args, f)\n\t\trequire.NoError(t, e)\n\n\t\t//Validate\n\t\te = o.Validate(cmd, args, f)\n\t\trequire.ErrorContains(t, e, \"either a backup or schedule must be specified, but not both\")\n\n\t\t//cmd\n\t\te = o.Run(cmd, f)\n\t\trequire.NoError(t, e)\n\n\t\trequire.Equal(t, backupName, o.BackupName)\n\t\trequire.Equal(t, scheduleName, o.ScheduleName)\n\t\trequire.Equal(t, restoreVolumes, o.RestoreVolumes.String())\n\t\trequire.Equal(t, preserveNodePorts, o.PreserveNodePorts.String())\n\t\trequire.Equal(t, labels, o.Labels.String())\n\t\trequire.Equal(t, annotations, o.Annotations.String())\n\t\trequire.Equal(t, includeNamespaces, o.IncludeNamespaces.String())\n\t\trequire.Equal(t, excludeNamespaces, o.ExcludeNamespaces.String())\n\t\trequire.Equal(t, existingResourcePolicy, o.ExistingResourcePolicy)\n\t\trequire.Equal(t, includeResources, o.IncludeResources.String())\n\t\trequire.Equal(t, excludeResources, o.ExcludeResources.String())\n\n\t\trequire.Equal(t, statusIncludeResources, o.StatusIncludeResources.String())\n\t\trequire.Equal(t, statusExcludeResources, o.StatusExcludeResources.String())\n\t\trequire.Equal(t, namespaceMappings, o.NamespaceMappings.String())\n\t\trequire.Equal(t, selector, o.Selector.String())\n\t\trequire.Equal(t, includeClusterResources, o.IncludeClusterResources.String())\n\t\trequire.Equal(t, allowPartiallyFailed, o.AllowPartiallyFailed.String())\n\t\trequire.Equal(t, itemOperationTimeout, o.ItemOperationTimeout.String())\n\t\trequire.Equal(t, writeSparseFiles, o.WriteSparseFiles.String())\n\t\trequire.Equal(t, parallel, o.ParallelFilesDownload)\n\t})\n\n\tt.Run(\"create a restore from schedule\", func(t *testing.T) {\n\t\tf := &factorymocks.Factory{}\n\t\tc := NewCreateCommand(f, \"\")\n\t\trequire.Equal(t, \"Create a restore\", c.Short)\n\t\tflags := new(pflag.FlagSet)\n\t\to := NewCreateOptions()\n\t\to.BindFlags(flags)\n\n\t\tfromSchedule := \"schedule-name-1\"\n\t\tflags.Parse([]string{\"--from-schedule\", fromSchedule})\n\n\t\tkbclient := velerotest.NewFakeControllerRuntimeClient(t).(kbclient.WithWatch)\n\t\tschedule := builder.ForSchedule(cmdtest.VeleroNameSpace, fromSchedule).Result()\n\t\trequire.NoError(t, kbclient.Create(t.Context(), schedule, &controllerclient.CreateOptions{}))\n\t\tbackup := builder.ForBackup(cmdtest.VeleroNameSpace, \"test-backup\").FromSchedule(schedule).Phase(velerov1api.BackupPhaseCompleted).Result()\n\t\trequire.NoError(t, kbclient.Create(t.Context(), backup, &controllerclient.CreateOptions{}))\n\n\t\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\t\tf.On(\"KubebuilderWatchClient\").Return(kbclient, nil)\n\n\t\trequire.NoError(t, o.Complete(args, f))\n\t\trequire.NoError(t, o.Validate(c, []string{}, f))\n\t\trequire.NoError(t, o.Run(c, f))\n\t})\n\n\tt.Run(\"create a restore from not-existed backup\", func(t *testing.T) {\n\t\tf := &factorymocks.Factory{}\n\t\tc := NewCreateCommand(f, \"\")\n\t\trequire.Equal(t, \"Create a restore\", c.Short)\n\t\tflags := new(pflag.FlagSet)\n\t\to := NewCreateOptions()\n\t\to.BindFlags(flags)\n\t\tnonExistedBackupName := \"not-exist\"\n\n\t\tflags.Parse([]string{\"--wait\", \"true\"})\n\t\tflags.Parse([]string{\"--from-backup\", nonExistedBackupName})\n\n\t\tkbclient := velerotest.NewFakeControllerRuntimeClient(t).(kbclient.WithWatch)\n\n\t\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\t\tf.On(\"KubebuilderWatchClient\").Return(kbclient, nil)\n\n\t\trequire.NoError(t, o.Complete(nil, f))\n\t\terr := o.Validate(c, []string{}, f)\n\t\trequire.Equal(t, \"backups.velero.io \\\"not-exist\\\" not found\", err.Error())\n\t})\n}\n" }, { "path": "pkg/cmd/cli/restore/delete.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/spf13/cobra\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\tkubeerrs \"k8s.io/apimachinery/pkg/util/errors\"\n\tcontrollerclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/confirm\"\n)\n\n// NewDeleteCommand creates and returns a new cobra command for deleting restores.\nfunc NewDeleteCommand(f client.Factory, use string) *cobra.Command {\n\to := cli.NewDeleteOptions(\"restore\")\n\n\tc := &cobra.Command{\n\t\tUse: fmt.Sprintf(\"%s [NAMES]\", use),\n\t\tShort: \"Delete restores\",\n\t\tExample: ` # Delete a restore named \"restore-1\".\n velero restore delete restore-1\n\n # Delete a restore named \"restore-1\" without prompting for confirmation.\n velero restore delete restore-1 --confirm\n\n # Delete restores named \"restore-1\" and \"restore-2\".\n velero restore delete restore-1 restore-2\n\n # Delete all restores labeled with \"foo=bar\".\n velero restore delete --selector foo=bar\n\t\n # Delete all restores.\n velero restore delete --all`,\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tcmd.CheckError(o.Complete(f, args))\n\t\t\tcmd.CheckError(o.Validate(c, f, args))\n\t\t\tcmd.CheckError(Run(o))\n\t\t},\n\t}\n\to.BindFlags(c.Flags())\n\treturn c\n}\n\n// Run performs the deletion of restore(s).\nfunc Run(o *cli.DeleteOptions) error {\n\tif !o.Confirm && !confirm.GetConfirmation() {\n\t\treturn nil\n\t}\n\tvar (\n\t\trestores []*velerov1api.Restore\n\t\terrs []error\n\t)\n\n\tswitch {\n\tcase len(o.Names) > 0:\n\t\tfor _, name := range o.Names {\n\t\t\trestore := new(velerov1api.Restore)\n\t\t\terr := o.Client.Get(context.TODO(), controllerclient.ObjectKey{Namespace: o.Namespace, Name: name}, restore)\n\t\t\tif err != nil {\n\t\t\t\terrs = append(errs, errors.WithStack(err))\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\trestores = append(restores, restore)\n\t\t}\n\tdefault:\n\t\tselector := labels.Everything()\n\t\tif o.Selector.LabelSelector != nil {\n\t\t\tconvertedSelector, err := metav1.LabelSelectorAsSelector(o.Selector.LabelSelector)\n\t\t\tif err != nil {\n\t\t\t\treturn errors.WithStack(err)\n\t\t\t}\n\t\t\tselector = convertedSelector\n\t\t}\n\t\trestoreList := new(velerov1api.RestoreList)\n\t\terr := o.Client.List(context.TODO(), restoreList, &controllerclient.ListOptions{\n\t\t\tNamespace: o.Namespace,\n\t\t\tLabelSelector: selector,\n\t\t})\n\t\tif err != nil {\n\t\t\terrs = append(errs, errors.WithStack(err))\n\t\t}\n\n\t\tfor i := range restoreList.Items {\n\t\t\trestores = append(restores, &restoreList.Items[i])\n\t\t}\n\t}\n\n\tif len(errs) > 0 {\n\t\tfmt.Println(\"errs: \", errs)\n\t\treturn kubeerrs.NewAggregate(errs)\n\t}\n\n\tif len(restores) == 0 {\n\t\tfmt.Println(\"No restores found\")\n\t\treturn nil\n\t}\n\tfor _, r := range restores {\n\t\terr := o.Client.Delete(context.TODO(), r, &controllerclient.DeleteOptions{})\n\t\tif err != nil {\n\t\t\terrs = append(errs, errors.WithStack(err))\n\t\t\tcontinue\n\t\t}\n\t\tfmt.Printf(\"Request to delete restore %q submitted successfully.\\nThe restore will be fully deleted after all associated data (restore files in object storage) are removed.\\n\", r.Name)\n\t}\n\treturn kubeerrs.NewAggregate(errs)\n}\n" }, { "path": "pkg/cmd/cli/restore/delete_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"testing\"\n\n\tflag \"github.com/spf13/pflag\"\n\t\"github.com/stretchr/testify/require\"\n\tcontrollerclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli\"\n\tcmdtest \"github.com/vmware-tanzu/velero/pkg/cmd/test\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\tveleroexec \"github.com/vmware-tanzu/velero/pkg/util/exec\"\n)\n\nfunc TestDeleteCommand(t *testing.T) {\n\trestore1 := \"restore-name-1\"\n\trestore2 := \"restore-name-2\"\n\n\t// create a factory\n\tf := &factorymocks.Factory{}\n\n\tclient := velerotest.NewFakeControllerRuntimeClient(t)\n\tclient.Create(t.Context(), builder.ForRestore(cmdtest.VeleroNameSpace, restore1).Result(), &controllerclient.CreateOptions{})\n\tclient.Create(t.Context(), builder.ForRestore(\"default\", restore2).Result(), &controllerclient.CreateOptions{})\n\n\tf.On(\"KubebuilderClient\").Return(client, nil)\n\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\n\t// create command\n\tc := NewDeleteCommand(f, \"velero restore delete\")\n\tc.SetArgs([]string{restore1, restore2})\n\trequire.Equal(t, \"Delete restores\", c.Short)\n\n\to := cli.NewDeleteOptions(\"restore\")\n\tflags := new(flag.FlagSet)\n\to.BindFlags(flags)\n\tflags.Parse([]string{\"--confirm\"})\n\n\targs := []string{restore1, restore2}\n\n\te := o.Complete(f, args)\n\trequire.NoError(t, e)\n\n\te = o.Validate(c, f, args)\n\trequire.NoError(t, e)\n\n\tRun(o)\n\n\te = c.Execute()\n\trequire.NoError(t, e)\n\n\tif os.Getenv(cmdtest.CaptureFlag) == \"1\" {\n\t\treturn\n\t}\n\n\tcmd := exec.CommandContext(t.Context(), os.Args[0], []string{\"-test.run=TestDeleteCommand\"}...)\n\tcmd.Env = append(os.Environ(), fmt.Sprintf(\"%s=1\", cmdtest.CaptureFlag))\n\tstdout, _, err := veleroexec.RunCommand(cmd)\n\tif err != nil {\n\t\trequire.Contains(t, stdout, fmt.Sprintf(\"restores.velero.io \\\"%s\\\" not found.\", restore2))\n\t}\n}\n" }, { "path": "pkg/cmd/cli/restore/describe.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\n\t\"github.com/spf13/cobra\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\tcontrollerclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/output\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n)\n\nfunc NewDescribeCommand(f client.Factory, use string) *cobra.Command {\n\tvar (\n\t\tlistOptions metav1.ListOptions\n\t\tdetails bool\n\t\tinsecureSkipTLSVerify bool\n\t)\n\n\tconfig, err := client.LoadConfig()\n\tif err != nil {\n\t\tfmt.Fprintf(os.Stderr, \"WARNING: Error reading config file: %v\\n\", err)\n\t}\n\tcaCertFile := config.CACertFile()\n\n\tc := &cobra.Command{\n\t\tUse: use + \" [NAME1] [NAME2] [NAME...]\",\n\t\tShort: \"Describe restores\",\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tkbClient, err := f.KubebuilderClient()\n\t\t\tcmd.CheckError(err)\n\n\t\t\trestoreList := new(velerov1api.RestoreList)\n\t\t\tif len(args) > 0 {\n\t\t\t\tfor _, name := range args {\n\t\t\t\t\trestore := new(velerov1api.Restore)\n\t\t\t\t\terr := kbClient.Get(context.TODO(), controllerclient.ObjectKey{Namespace: f.Namespace(), Name: name}, restore)\n\t\t\t\t\tcmd.CheckError(err)\n\t\t\t\t\trestoreList.Items = append(restoreList.Items, *restore)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tparsedSelector, err := labels.Parse(listOptions.LabelSelector)\n\t\t\t\tcmd.CheckError(err)\n\n\t\t\t\terr = kbClient.List(context.TODO(), restoreList, &controllerclient.ListOptions{LabelSelector: parsedSelector, Namespace: f.Namespace()})\n\t\t\t\tcmd.CheckError(err)\n\t\t\t}\n\n\t\t\tfirst := true\n\t\t\tfor i, restore := range restoreList.Items {\n\t\t\t\tpodVolumeRestoreList := new(velerov1api.PodVolumeRestoreList)\n\t\t\t\terr = kbClient.List(context.TODO(), podVolumeRestoreList, &controllerclient.ListOptions{\n\t\t\t\t\tNamespace: f.Namespace(),\n\t\t\t\t\tLabelSelector: labels.SelectorFromSet(map[string]string{velerov1api.RestoreNameLabel: label.GetValidName(restore.Name)}),\n\t\t\t\t})\n\t\t\t\tif err != nil {\n\t\t\t\t\tfmt.Fprintf(os.Stderr, \"error getting PodVolumeRestores for restore %s: %v\\n\", restore.Name, err)\n\t\t\t\t}\n\n\t\t\t\ts := output.DescribeRestore(context.Background(), kbClient, &restoreList.Items[i], podVolumeRestoreList.Items, details, insecureSkipTLSVerify, caCertFile)\n\t\t\t\tif first {\n\t\t\t\t\tfirst = false\n\t\t\t\t\tfmt.Print(s)\n\t\t\t\t} else {\n\t\t\t\t\tfmt.Printf(\"\\n\\n%s\", s)\n\t\t\t\t}\n\t\t\t}\n\t\t\tcmd.CheckError(err)\n\t\t},\n\t}\n\n\tc.Flags().StringVarP(&listOptions.LabelSelector, \"selector\", \"l\", listOptions.LabelSelector, \"Only show items matching this label selector.\")\n\tc.Flags().BoolVar(&details, \"details\", details, \"Display additional detail in the command output.\")\n\tc.Flags().BoolVar(&insecureSkipTLSVerify, \"insecure-skip-tls-verify\", insecureSkipTLSVerify, \"If true, the object store's TLS certificate will not be checked for validity. This is insecure and susceptible to man-in-the-middle attacks. Not recommended for production.\")\n\tc.Flags().StringVar(&caCertFile, \"cacert\", caCertFile, \"Path to a certificate bundle to use when verifying TLS connections.\")\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/restore/describe_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/client-go/rest\"\n\tcontrollerclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\tcmdtest \"github.com/vmware-tanzu/velero/pkg/cmd/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/features\"\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n\tveleroexec \"github.com/vmware-tanzu/velero/pkg/util/exec\"\n)\n\nfunc TestNewDescribeCommand(t *testing.T) {\n\t// create a factory\n\tf := &factorymocks.Factory{}\n\trestoreName := \"restore-describe-1\"\n\ttestRestore := builder.ForRestore(cmdtest.VeleroNameSpace, restoreName).Result()\n\n\tclientConfig := rest.Config{}\n\tkbClient := test.NewFakeControllerRuntimeClient(t)\n\tkbClient.Create(t.Context(), testRestore, &controllerclient.CreateOptions{})\n\n\tf.On(\"ClientConfig\").Return(&clientConfig, nil)\n\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\tf.On(\"KubebuilderClient\").Return(kbClient, nil)\n\n\t// create command\n\tc := NewDescribeCommand(f, \"velero restore describe\")\n\tassert.Equal(t, \"Describe restores\", c.Short)\n\n\tfeatures.NewFeatureFlagSet(\"EnableCSI\")\n\tdefer features.NewFeatureFlagSet()\n\n\tc.SetArgs([]string{restoreName})\n\te := c.Execute()\n\trequire.NoError(t, e)\n\n\tif os.Getenv(cmdtest.CaptureFlag) == \"1\" {\n\t\treturn\n\t}\n\tcmd := exec.CommandContext(t.Context(), os.Args[0], []string{\"-test.run=TestNewDescribeCommand\"}...)\n\tcmd.Env = append(os.Environ(), fmt.Sprintf(\"%s=1\", cmdtest.CaptureFlag))\n\tstdout, _, err := veleroexec.RunCommand(cmd)\n\n\tif err == nil {\n\t\tassert.Contains(t, stdout, fmt.Sprintf(\"Name: %s\", restoreName))\n\t\treturn\n\t}\n\tt.Fatalf(\"process ran with err %v, want backups by get()\", err)\n}\n" }, { "path": "pkg/cmd/cli/restore/get.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"context\"\n\n\t\"github.com/spf13/cobra\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\tcontrollerclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/output\"\n)\n\nfunc NewGetCommand(f client.Factory, use string) *cobra.Command {\n\tvar listOptions metav1.ListOptions\n\n\tc := &cobra.Command{\n\t\tUse: use,\n\t\tShort: \"Get restores\",\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\terr := output.ValidateFlags(c)\n\t\t\tcmd.CheckError(err)\n\n\t\t\tkbClient, err := f.KubebuilderClient()\n\t\t\tcmd.CheckError(err)\n\n\t\t\trestores := new(api.RestoreList)\n\t\t\tif len(args) > 0 {\n\t\t\t\tfor _, name := range args {\n\t\t\t\t\trestore := new(api.Restore)\n\t\t\t\t\terr := kbClient.Get(context.TODO(), controllerclient.ObjectKey{Namespace: f.Namespace(), Name: name}, restore)\n\t\t\t\t\tcmd.CheckError(err)\n\t\t\t\t\trestores.Items = append(restores.Items, *restore)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tparsedSelector, err := labels.Parse(listOptions.LabelSelector)\n\t\t\t\tcmd.CheckError(err)\n\n\t\t\t\terr = kbClient.List(context.TODO(), restores, &controllerclient.ListOptions{LabelSelector: parsedSelector, Namespace: f.Namespace()})\n\t\t\t\tcmd.CheckError(err)\n\t\t\t}\n\n\t\t\t// Append \"(Deleting)\" to phase if deletionTimestamp is marked.\n\t\t\tfor i := range restores.Items {\n\t\t\t\tif !restores.Items[i].DeletionTimestamp.IsZero() {\n\t\t\t\t\trestores.Items[i].Status.Phase += \" (Deleting)\"\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif printed, err := output.PrintWithFormat(c, restores); printed || err != nil {\n\t\t\t\tcmd.CheckError(err)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\t_, err = output.PrintWithFormat(c, restores)\n\t\t\tcmd.CheckError(err)\n\t\t},\n\t}\n\n\tc.Flags().StringVarP(&listOptions.LabelSelector, \"selector\", \"l\", listOptions.LabelSelector, \"Only show items matching this label selector.\")\n\n\toutput.BindFlags(c.Flags())\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/restore/get_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\tcmdtest \"github.com/vmware-tanzu/velero/pkg/cmd/test\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\tveleroexec \"github.com/vmware-tanzu/velero/pkg/util/exec\"\n)\n\nfunc TestNewGetCommand(t *testing.T) {\n\targs := []string{\"b1\", \"b2\", \"b3\"}\n\n\t// create a factory\n\tf := &factorymocks.Factory{}\n\n\tclient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\tfor _, restoreName := range args {\n\t\trestore := builder.ForRestore(cmdtest.VeleroNameSpace, restoreName).ObjectMeta(builder.WithLabels(\"abc\", \"abc\")).Result()\n\t\terr := client.Create(t.Context(), restore, &kbclient.CreateOptions{})\n\t\trequire.NoError(t, err)\n\t}\n\n\tf.On(\"KubebuilderClient\").Return(client, nil)\n\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\n\t// create command\n\tc := NewGetCommand(f, \"velero restore get\")\n\trequire.Equal(t, \"Get restores\", c.Short)\n\n\tc.SetArgs(args)\n\te := c.Execute()\n\trequire.NoError(t, e)\n\n\tif os.Getenv(cmdtest.CaptureFlag) == \"1\" {\n\t\treturn\n\t}\n\n\tcmd := exec.CommandContext(t.Context(), os.Args[0], []string{\"-test.run=TestNewGetCommand\"}...)\n\tcmd.Env = append(os.Environ(), fmt.Sprintf(\"%s=1\", cmdtest.CaptureFlag))\n\tstdout, _, err := veleroexec.RunCommand(cmd)\n\trequire.NoError(t, err)\n\n\tif err == nil {\n\t\toutput := strings.Split(stdout, \"\\n\")\n\t\ti := 0\n\t\tfor _, line := range output {\n\t\t\tif strings.Contains(line, \"New\") {\n\t\t\t\ti++\n\t\t\t}\n\t\t}\n\t\trequire.Len(t, args, i)\n\t}\n}\n" }, { "path": "pkg/cmd/cli/restore/logs.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"time\"\n\n\t\"github.com/spf13/cobra\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tctrlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/cacert\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/downloadrequest\"\n)\n\nfunc NewLogsCommand(f client.Factory) *cobra.Command {\n\tconfig, err := client.LoadConfig()\n\tif err != nil {\n\t\tfmt.Fprintf(os.Stderr, \"WARNING: Error reading config file: %v\\n\", err)\n\t}\n\n\ttimeout := time.Minute\n\tinsecureSkipTLSVerify := false\n\tcaCertFile := config.CACertFile()\n\n\tc := &cobra.Command{\n\t\tUse: \"logs RESTORE\",\n\t\tShort: \"Get restore logs\",\n\t\tArgs: cobra.ExactArgs(1),\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\trestoreName := args[0]\n\n\t\t\tkbClient, err := f.KubebuilderClient()\n\t\t\tcmd.CheckError(err)\n\n\t\t\trestore := new(velerov1api.Restore)\n\t\t\terr = kbClient.Get(context.Background(), ctrlclient.ObjectKey{Namespace: f.Namespace(), Name: restoreName}, restore)\n\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\tcmd.Exit(\"Restore %q does not exist.\", restoreName)\n\t\t\t} else if err != nil {\n\t\t\t\tcmd.Exit(\"Error checking for restore %q: %v\", restoreName, err)\n\t\t\t}\n\n\t\t\tswitch restore.Status.Phase {\n\t\t\tcase velerov1api.RestorePhaseCompleted, velerov1api.RestorePhaseFailed, velerov1api.RestorePhasePartiallyFailed, velerov1api.RestorePhaseWaitingForPluginOperations, velerov1api.RestorePhaseWaitingForPluginOperationsPartiallyFailed:\n\t\t\t\t// terminal and waiting for plugin operations phases, don't exit.\n\t\t\tdefault:\n\t\t\t\tcmd.Exit(\"Logs for restore %q are not available until it's finished processing. Please wait \"+\n\t\t\t\t\t\"until the restore has a phase of Completed or Failed and try again.\", restoreName)\n\t\t\t}\n\n\t\t\t// Get BSL cacert if available\n\t\t\tbslCACert, err := cacert.GetCACertFromRestore(context.Background(), kbClient, f.Namespace(), restore)\n\t\t\tif err != nil {\n\t\t\t\t// Log the error but don't fail - we can still try to download without the BSL cacert\n\t\t\t\tfmt.Fprintf(os.Stderr, \"WARNING: Error getting cacert from BSL: %v\\n\", err)\n\t\t\t\tbslCACert = \"\"\n\t\t\t}\n\n\t\t\terr = downloadrequest.StreamWithBSLCACert(context.Background(), kbClient, f.Namespace(), restoreName, velerov1api.DownloadTargetKindRestoreLog, os.Stdout, timeout, insecureSkipTLSVerify, caCertFile, bslCACert)\n\t\t\tcmd.CheckError(err)\n\t\t},\n\t}\n\n\tc.Flags().DurationVar(&timeout, \"timeout\", timeout, \"How long to wait to receive logs.\")\n\tc.Flags().BoolVar(&insecureSkipTLSVerify, \"insecure-skip-tls-verify\", insecureSkipTLSVerify, \"If true, the object store's TLS certificate will not be checked for validity. This is insecure and susceptible to man-in-the-middle attacks. Not recommended for production.\")\n\tc.Flags().StringVar(&caCertFile, \"cacert\", caCertFile, \"Path to a certificate bundle to use when verifying TLS connections. If not specified, the CA certificate from the BackupStorageLocation will be used if available.\")\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/restore/logs_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"os\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\tcmdtest \"github.com/vmware-tanzu/velero/pkg/cmd/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/cacert\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestNewLogsCommand(t *testing.T) {\n\tt.Run(\"Flag test\", func(t *testing.T) {\n\t\t// create a factory\n\t\tf := &factorymocks.Factory{}\n\n\t\tc := NewLogsCommand(f)\n\t\trequire.Equal(t, \"Get restore logs\", c.Short)\n\n\t\t// Test flag parsing\n\t\ttimeout := \"1s\"\n\t\tinsecureSkipTLSVerify := \"true\"\n\t\tcaCertFile := \"testing\"\n\n\t\tc.Flags().Set(\"timeout\", timeout)\n\t\tc.Flags().Set(\"insecure-skip-tls-verify\", insecureSkipTLSVerify)\n\t\tc.Flags().Set(\"cacert\", caCertFile)\n\n\t\ttimeoutFlag, _ := c.Flags().GetDuration(\"timeout\")\n\t\trequire.Equal(t, 1*time.Second, timeoutFlag)\n\n\t\tinsecureFlag, _ := c.Flags().GetBool(\"insecure-skip-tls-verify\")\n\t\trequire.True(t, insecureFlag)\n\n\t\tcaCertFlag, _ := c.Flags().GetString(\"cacert\")\n\t\trequire.Equal(t, caCertFile, caCertFlag)\n\t})\n\n\tt.Run(\"Restore not complete test\", func(t *testing.T) {\n\t\trestoreName := \"rs-logs-1\"\n\n\t\t// create a factory\n\t\tf := &factorymocks.Factory{}\n\n\t\tkbClient := velerotest.NewFakeControllerRuntimeClient(t)\n\t\trestore := builder.ForRestore(cmdtest.VeleroNameSpace, restoreName).Result()\n\t\terr := kbClient.Create(t.Context(), restore, &kbclient.CreateOptions{})\n\t\trequire.NoError(t, err)\n\n\t\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\t\tf.On(\"KubebuilderClient\").Return(kbClient, nil)\n\n\t\tc := NewLogsCommand(f)\n\t\tassert.Equal(t, \"Get restore logs\", c.Short)\n\n\t\t// The restore command exits with an error message when restore is not complete\n\t\t// We can't easily test this since it calls cmd.Exit, which exits the process\n\t\t// So we'll skip this test case\n\t\tt.Skip(\"Cannot test restore not complete case due to cmd.Exit() call\")\n\t})\n\n\tt.Run(\"Restore not exist test\", func(t *testing.T) {\n\t\t// create a factory\n\t\tf := &factorymocks.Factory{}\n\n\t\tkbClient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\t\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\t\tf.On(\"KubebuilderClient\").Return(kbClient, nil)\n\n\t\tc := NewLogsCommand(f)\n\t\tassert.Equal(t, \"Get restore logs\", c.Short)\n\n\t\t// The restore command exits with an error message when restore doesn't exist\n\t\t// We can't easily test this since it calls cmd.Exit, which exits the process\n\t\t// So we'll skip this test case\n\t\tt.Skip(\"Cannot test restore not exist case due to cmd.Exit() call\")\n\t})\n\n\tt.Run(\"Restore with BSL cacert test\", func(t *testing.T) {\n\t\trestoreName := \"rs-logs-with-cacert\"\n\t\tbackupName := \"bk-for-restore\"\n\t\tbslName := \"test-bsl\"\n\n\t\t// create a factory\n\t\tf := &factorymocks.Factory{}\n\n\t\tkbClient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\t\t// Create BSL with cacert\n\t\tbsl := builder.ForBackupStorageLocation(cmdtest.VeleroNameSpace, bslName).\n\t\t\tProvider(\"aws\").\n\t\t\tBucket(\"test-bucket\").\n\t\t\tCACert([]byte(\"test-cacert-content\")).\n\t\t\tResult()\n\t\terr := kbClient.Create(t.Context(), bsl, &kbclient.CreateOptions{})\n\t\trequire.NoError(t, err)\n\n\t\t// Create backup referencing the BSL\n\t\tbackup := builder.ForBackup(cmdtest.VeleroNameSpace, backupName).\n\t\t\tStorageLocation(bslName).\n\t\t\tResult()\n\t\terr = kbClient.Create(t.Context(), backup, &kbclient.CreateOptions{})\n\t\trequire.NoError(t, err)\n\n\t\t// Create restore referencing the backup\n\t\trestore := builder.ForRestore(cmdtest.VeleroNameSpace, restoreName).\n\t\t\tPhase(velerov1api.RestorePhaseCompleted).\n\t\t\tBackup(backupName).\n\t\t\tResult()\n\t\terr = kbClient.Create(t.Context(), restore, &kbclient.CreateOptions{})\n\t\trequire.NoError(t, err)\n\n\t\tf.On(\"Namespace\").Return(cmdtest.VeleroNameSpace)\n\t\tf.On(\"KubebuilderClient\").Return(kbClient, nil)\n\n\t\tc := NewLogsCommand(f)\n\t\tassert.Equal(t, \"Get restore logs\", c.Short)\n\n\t\t// We can verify that BSL cacert fetching logic is in place\n\t\t// The actual command will call downloadrequest which requires a controller\n\t\t// to be running, so we'll just verify the command structure\n\t\trequire.NotNil(t, c.Run)\n\n\t\t// Verify the BSL cacert can be fetched\n\t\tcacertValue, err := cacert.GetCACertFromRestore(t.Context(), kbClient, f.Namespace(), restore)\n\t\trequire.NoError(t, err)\n\t\trequire.Equal(t, \"test-cacert-content\", cacertValue)\n\t})\n\n\tt.Run(\"CLI execution test\", func(t *testing.T) {\n\t\t// create a factory\n\t\tf := &factorymocks.Factory{}\n\n\t\tc := NewLogsCommand(f)\n\t\trequire.Equal(t, \"Get restore logs\", c.Short)\n\n\t\tif os.Getenv(cmdtest.CaptureFlag) == \"1\" {\n\t\t\tc.SetArgs([]string{\"test\"})\n\t\t\te := c.Execute()\n\t\t\tassert.NoError(t, e)\n\t\t\treturn\n\t\t}\n\t})\n}\n" }, { "path": "pkg/cmd/cli/restore/restore.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"github.com/spf13/cobra\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n)\n\nfunc NewCommand(f client.Factory) *cobra.Command {\n\tc := &cobra.Command{\n\t\tUse: \"restore\",\n\t\tShort: \"Work with restores\",\n\t\tLong: \"Work with restores\",\n\t}\n\n\tc.AddCommand(\n\t\tNewCreateCommand(f, \"create\"),\n\t\tNewGetCommand(f, \"get\"),\n\t\tNewLogsCommand(f),\n\t\tNewDescribeCommand(f, \"describe\"),\n\t\tNewDeleteCommand(f, \"delete\"),\n\t)\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/restore/restore_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n)\n\nfunc TestNewRestoreCommand(t *testing.T) {\n\t// create a factory\n\tf := &factorymocks.Factory{}\n\n\t// create command\n\tcmd := NewCommand(f)\n\tassert.Equal(t, \"Work with restores\", cmd.Short)\n}\n" }, { "path": "pkg/cmd/cli/schedule/create.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage schedule\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/spf13/cobra\"\n\t\"github.com/spf13/pflag\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\t\"github.com/vmware-tanzu/velero/internal/resourcepolicies\"\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/backup\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/output\"\n)\n\nfunc NewCreateCommand(f client.Factory, use string) *cobra.Command {\n\to := NewCreateOptions()\n\n\tc := &cobra.Command{\n\t\tUse: use + \" NAME --schedule\",\n\t\tShort: \"Create a schedule\",\n\t\tLong: `The --schedule flag is required, in cron notation, using UTC time:\n\n| Character Position | Character Period | Acceptable Values |\n| -------------------|:----------------:| -----------------:|\n| 1 | Minute | 0-59,* |\n| 2 | Hour | 0-23,* |\n| 3 | Day of Month | 1-31,* |\n| 4 | Month | 1-12,* |\n| 5 | Day of Week | 0-6,* |\n\nThe schedule can also be expressed using \"@every \" syntax. The duration\ncan be specified using a combination of seconds (s), minutes (m), and hours (h), for\nexample: \"@every 2h30m\".`,\n\n\t\tExample: ` # Create a backup every 6 hours.\n velero create schedule NAME --schedule=\"0 */6 * * *\"\n\n # Create a backup every 6 hours with the @every notation.\n velero create schedule NAME --schedule=\"@every 6h\"\n\n # Create a daily backup of the web namespace.\n velero create schedule NAME --schedule=\"@every 24h\" --include-namespaces web\n\n # Create a weekly backup, each living for 90 days (2160 hours).\n velero create schedule NAME --schedule=\"@every 168h\" --ttl 2160h0m0s`,\n\t\tArgs: cobra.ExactArgs(1),\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tcmd.CheckError(o.Complete(args, f))\n\t\t\tcmd.CheckError(o.Validate(c, args, f))\n\t\t\tcmd.CheckError(o.Run(c, f))\n\t\t},\n\t}\n\n\to.BindFlags(c.Flags())\n\toutput.BindFlags(c.Flags())\n\toutput.ClearOutputFlagDefault(c)\n\n\treturn c\n}\n\ntype CreateOptions struct {\n\tBackupOptions *backup.CreateOptions\n\tSkipOptions *SkipOptions\n\tSchedule string\n\tUseOwnerReferencesInBackup bool\n\tPaused bool\n}\n\nfunc NewCreateOptions() *CreateOptions {\n\treturn &CreateOptions{\n\t\tBackupOptions: backup.NewCreateOptions(),\n\t\tSkipOptions: NewSkipOptions(),\n\t}\n}\n\nfunc (o *CreateOptions) BindFlags(flags *pflag.FlagSet) {\n\to.BackupOptions.BindFlags(flags)\n\to.SkipOptions.BindFlags(flags)\n\tflags.StringVar(&o.Schedule, \"schedule\", o.Schedule, \"A cron expression specifying a recurring schedule for this backup to run\")\n\tflags.BoolVar(&o.UseOwnerReferencesInBackup, \"use-owner-references-in-backup\", o.UseOwnerReferencesInBackup, \"Specifies whether to use OwnerReferences on backups created by this Schedule. Notice: if set to true, when schedule is deleted, backups will be deleted too.\")\n\tflags.BoolVar(&o.Paused, \"paused\", o.Paused, \"Specifies whether the newly created schedule is paused or not.\")\n}\n\nfunc (o *CreateOptions) Validate(c *cobra.Command, args []string, f client.Factory) error {\n\tif len(o.Schedule) == 0 {\n\t\treturn errors.New(\"--schedule is required\")\n\t}\n\n\treturn o.BackupOptions.Validate(c, args, f)\n}\n\nfunc (o *CreateOptions) Complete(args []string, f client.Factory) error {\n\treturn o.BackupOptions.Complete(args, f)\n}\n\nfunc (o *CreateOptions) Run(c *cobra.Command, f client.Factory) error {\n\tvar orders map[string]string\n\n\tcrClient, err := f.KubebuilderClient()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif len(o.BackupOptions.OrderedResources) > 0 {\n\t\torders, err = backup.ParseOrderedResources(o.BackupOptions.OrderedResources)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tschedule := &api.Schedule{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: f.Namespace(),\n\t\t\tName: o.BackupOptions.Name,\n\t\t\tLabels: o.BackupOptions.Labels.Data(),\n\t\t},\n\t\tSpec: api.ScheduleSpec{\n\t\t\tTemplate: api.BackupSpec{\n\t\t\t\tIncludedNamespaces: o.BackupOptions.IncludeNamespaces,\n\t\t\t\tExcludedNamespaces: o.BackupOptions.ExcludeNamespaces,\n\t\t\t\tIncludedResources: o.BackupOptions.IncludeResources,\n\t\t\t\tExcludedResources: o.BackupOptions.ExcludeResources,\n\t\t\t\tIncludedClusterScopedResources: o.BackupOptions.IncludeClusterScopedResources,\n\t\t\t\tExcludedClusterScopedResources: o.BackupOptions.ExcludeClusterScopedResources,\n\t\t\t\tIncludedNamespaceScopedResources: o.BackupOptions.IncludeNamespaceScopedResources,\n\t\t\t\tExcludedNamespaceScopedResources: o.BackupOptions.ExcludeNamespaceScopedResources,\n\t\t\t\tIncludeClusterResources: o.BackupOptions.IncludeClusterResources.Value,\n\t\t\t\tLabelSelector: o.BackupOptions.Selector.LabelSelector,\n\t\t\t\tOrLabelSelectors: o.BackupOptions.OrSelector.OrLabelSelectors,\n\t\t\t\tSnapshotVolumes: o.BackupOptions.SnapshotVolumes.Value,\n\t\t\t\tTTL: metav1.Duration{Duration: o.BackupOptions.TTL},\n\t\t\t\tStorageLocation: o.BackupOptions.StorageLocation,\n\t\t\t\tVolumeSnapshotLocations: o.BackupOptions.SnapshotLocations,\n\t\t\t\tDefaultVolumesToFsBackup: o.BackupOptions.DefaultVolumesToFsBackup.Value,\n\t\t\t\tOrderedResources: orders,\n\t\t\t\tCSISnapshotTimeout: metav1.Duration{Duration: o.BackupOptions.CSISnapshotTimeout},\n\t\t\t\tItemOperationTimeout: metav1.Duration{Duration: o.BackupOptions.ItemOperationTimeout},\n\t\t\t\tDataMover: o.BackupOptions.DataMover,\n\t\t\t\tSnapshotMoveData: o.BackupOptions.SnapshotMoveData.Value,\n\t\t\t},\n\t\t\tSchedule: o.Schedule,\n\t\t\tUseOwnerReferencesInBackup: &o.UseOwnerReferencesInBackup,\n\t\t\tPaused: o.Paused,\n\t\t\tSkipImmediately: o.SkipOptions.SkipImmediately.Value,\n\t\t},\n\t}\n\n\tif o.BackupOptions.ResPoliciesConfigmap != \"\" {\n\t\tschedule.Spec.Template.ResourcePolicy = &corev1api.TypedLocalObjectReference{Kind: resourcepolicies.ConfigmapRefType, Name: o.BackupOptions.ResPoliciesConfigmap}\n\t}\n\n\tif o.BackupOptions.ParallelFilesUpload > 0 {\n\t\tschedule.Spec.Template.UploaderConfig = &api.UploaderConfigForBackup{\n\t\t\tParallelFilesUpload: o.BackupOptions.ParallelFilesUpload,\n\t\t}\n\t}\n\n\tif printed, err := output.PrintWithFormat(c, schedule); printed || err != nil {\n\t\treturn err\n\t}\n\n\terr = crClient.Create(context.TODO(), schedule)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfmt.Printf(\"Schedule %q created successfully.\\n\", schedule.Name)\n\treturn nil\n}\n" }, { "path": "pkg/cmd/cli/schedule/delete.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage schedule\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/spf13/cobra\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\tkubeerrs \"k8s.io/apimachinery/pkg/util/errors\"\n\tcontrollerclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/confirm\"\n)\n\n// NewDeleteCommand creates and returns a new cobra command for deleting schedules.\nfunc NewDeleteCommand(f client.Factory, use string) *cobra.Command {\n\to := cli.NewDeleteOptions(\"schedule\")\n\n\tc := &cobra.Command{\n\t\tUse: fmt.Sprintf(\"%s [NAMES]\", use),\n\t\tShort: \"Delete schedules\",\n\t\tExample: ` # Delete a schedule named \"schedule-1\".\n velero schedule delete schedule-1\n\n # Delete a schedule named \"schedule-1\" without prompting for confirmation.\n velero schedule delete schedule-1 --confirm\n\n # Delete schedules named \"schedule-1\" and \"schedule-2\".\n velero schedule delete schedule-1 schedule-2\n\n # Delete all schedules labeled with \"foo=bar\".\n velero schedule delete --selector foo=bar\n\n # Delete all schedules.\n velero schedule delete --all`,\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tcmd.CheckError(o.Complete(f, args))\n\t\t\tcmd.CheckError(o.Validate(c, f, args))\n\t\t\tcmd.CheckError(Run(o))\n\t\t},\n\t}\n\n\to.BindFlags(c.Flags())\n\treturn c\n}\n\n// Run performs the deletion of schedules.\nfunc Run(o *cli.DeleteOptions) error {\n\tif !o.Confirm && !confirm.GetConfirmation() {\n\t\treturn nil\n\t}\n\tvar (\n\t\tschedules []*velerov1api.Schedule\n\t\terrs []error\n\t)\n\tswitch {\n\tcase len(o.Names) > 0:\n\t\tfor _, name := range o.Names {\n\t\t\tschedule := new(velerov1api.Schedule)\n\t\t\terr := o.Client.Get(context.TODO(), controllerclient.ObjectKey{Namespace: o.Namespace, Name: name}, schedule)\n\t\t\tif err != nil {\n\t\t\t\terrs = append(errs, errors.WithStack(err))\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tschedules = append(schedules, schedule)\n\t\t}\n\tdefault:\n\t\tselector := labels.Everything()\n\t\tif o.Selector.LabelSelector != nil {\n\t\t\tconvertedSelector, err := metav1.LabelSelectorAsSelector(o.Selector.LabelSelector)\n\t\t\tif err != nil {\n\t\t\t\treturn errors.WithStack(err)\n\t\t\t}\n\t\t\tselector = convertedSelector\n\t\t}\n\t\tscheduleList := new(velerov1api.ScheduleList)\n\t\terr := o.Client.List(context.TODO(), scheduleList, &controllerclient.ListOptions{\n\t\t\tNamespace: o.Namespace,\n\t\t\tLabelSelector: selector,\n\t\t})\n\t\tif err != nil {\n\t\t\terrs = append(errs, errors.WithStack(err))\n\t\t}\n\n\t\tfor i := range scheduleList.Items {\n\t\t\tschedules = append(schedules, &scheduleList.Items[i])\n\t\t}\n\t}\n\tif len(schedules) == 0 {\n\t\tfmt.Println(\"No schedules found\")\n\t\treturn nil\n\t}\n\n\tfor _, s := range schedules {\n\t\terr := o.Client.Delete(context.TODO(), s, &controllerclient.DeleteOptions{})\n\t\tif err != nil {\n\t\t\terrs = append(errs, errors.WithStack(err))\n\t\t\tcontinue\n\t\t}\n\t\tfmt.Printf(\"Schedule deleted: %v\\n\", s.Name)\n\t}\n\treturn kubeerrs.NewAggregate(errs)\n}\n" }, { "path": "pkg/cmd/cli/schedule/describe.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage schedule\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\t\"github.com/spf13/cobra\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\tctrlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/output\"\n)\n\nfunc NewDescribeCommand(f client.Factory, use string) *cobra.Command {\n\tvar listOptions metav1.ListOptions\n\n\tc := &cobra.Command{\n\t\tUse: use + \" [NAME1] [NAME2] [NAME...]\",\n\t\tShort: \"Describe schedules\",\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tcrClient, err := f.KubebuilderClient()\n\t\t\tcmd.CheckError(err)\n\n\t\t\tschedules := new(v1.ScheduleList)\n\t\t\tif len(args) > 0 {\n\t\t\t\tfor _, name := range args {\n\t\t\t\t\tschedule := new(v1.Schedule)\n\t\t\t\t\terr := crClient.Get(context.TODO(), ctrlclient.ObjectKey{Namespace: f.Namespace(), Name: name}, schedule)\n\t\t\t\t\tcmd.CheckError(err)\n\t\t\t\t\tschedules.Items = append(schedules.Items, *schedule)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tselector := labels.NewSelector()\n\t\t\t\tif listOptions.LabelSelector != \"\" {\n\t\t\t\t\tselector, err = labels.Parse(listOptions.LabelSelector)\n\t\t\t\t\tcmd.CheckError(err)\n\t\t\t\t}\n\t\t\t\terr = crClient.List(context.TODO(), schedules, &ctrlclient.ListOptions{LabelSelector: selector})\n\t\t\t\tcmd.CheckError(err)\n\t\t\t}\n\n\t\t\tfirst := true\n\t\t\tfor i := range schedules.Items {\n\t\t\t\ts := output.DescribeSchedule(&schedules.Items[i])\n\t\t\t\tif first {\n\t\t\t\t\tfirst = false\n\t\t\t\t\tfmt.Print(s)\n\t\t\t\t} else {\n\t\t\t\t\tfmt.Printf(\"\\n\\n%s\", s)\n\t\t\t\t}\n\t\t\t}\n\t\t\tcmd.CheckError(err)\n\t\t},\n\t}\n\n\tc.Flags().StringVarP(&listOptions.LabelSelector, \"selector\", \"l\", listOptions.LabelSelector, \"Only show items matching this label selector.\")\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/schedule/get.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage schedule\n\nimport (\n\t\"context\"\n\n\t\"github.com/spf13/cobra\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\tctrlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/output\"\n)\n\nfunc NewGetCommand(f client.Factory, use string) *cobra.Command {\n\tvar listOptions metav1.ListOptions\n\n\tc := &cobra.Command{\n\t\tUse: use,\n\t\tShort: \"Get schedules\",\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\terr := output.ValidateFlags(c)\n\t\t\tcmd.CheckError(err)\n\n\t\t\tcrClient, err := f.KubebuilderClient()\n\t\t\tcmd.CheckError(err)\n\n\t\t\tschedules := new(api.ScheduleList)\n\t\t\tif len(args) > 0 {\n\t\t\t\tfor _, name := range args {\n\t\t\t\t\tschedule := new(api.Schedule)\n\t\t\t\t\terr := crClient.Get(context.TODO(), ctrlclient.ObjectKey{Name: name, Namespace: f.Namespace()}, schedule)\n\t\t\t\t\tcmd.CheckError(err)\n\t\t\t\t\tschedules.Items = append(schedules.Items, *schedule)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tselector := labels.NewSelector()\n\t\t\t\tif listOptions.LabelSelector != \"\" {\n\t\t\t\t\tselector, err = labels.Parse(listOptions.LabelSelector)\n\t\t\t\t\tcmd.CheckError(err)\n\t\t\t\t}\n\t\t\t\terr := crClient.List(context.TODO(), schedules, &ctrlclient.ListOptions{LabelSelector: selector})\n\t\t\t\tcmd.CheckError(err)\n\t\t\t}\n\n\t\t\tif printed, err := output.PrintWithFormat(c, schedules); printed || err != nil {\n\t\t\t\tcmd.CheckError(err)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\t_, err = output.PrintWithFormat(c, schedules)\n\t\t\tcmd.CheckError(err)\n\t\t},\n\t}\n\n\tc.Flags().StringVarP(&listOptions.LabelSelector, \"selector\", \"l\", listOptions.LabelSelector, \"Only show items matching this label selector.\")\n\n\toutput.BindFlags(c.Flags())\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/schedule/pause.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage schedule\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/spf13/cobra\"\n\t\"github.com/spf13/pflag\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\tkubeerrs \"k8s.io/apimachinery/pkg/util/errors\"\n\tctrlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli\"\n)\n\n// NewPauseCommand creates the command for pause\nfunc NewPauseCommand(f client.Factory, use string) *cobra.Command {\n\to := cli.NewSelectOptions(\"pause\", \"schedule\")\n\tpauseOpts := NewPauseOptions()\n\n\tc := &cobra.Command{\n\t\tUse: use,\n\t\tShort: \"Pause schedules\",\n\t\tExample: ` # Pause a schedule named \"schedule-1\".\n velero schedule pause schedule-1\n\n # Pause schedules named \"schedule-1\" and \"schedule-2\".\n velero schedule pause schedule-1 schedule-2\n\n # Pause all schedules labeled with \"foo=bar\".\n velero schedule pause --selector foo=bar\n\n # Pause all schedules.\n velero schedule pause --all`,\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tcmd.CheckError(o.Complete(args))\n\t\t\tcmd.CheckError(o.Validate())\n\t\t\tcmd.CheckError(runPause(f, o, true, pauseOpts.SkipOptions.SkipImmediately.Value))\n\t\t},\n\t}\n\n\to.BindFlags(c.Flags())\n\tpauseOpts.BindFlags(c.Flags())\n\n\treturn c\n}\n\ntype PauseOptions struct {\n\tSkipOptions *SkipOptions\n}\n\nfunc NewPauseOptions() *PauseOptions {\n\treturn &PauseOptions{\n\t\tSkipOptions: NewSkipOptions(),\n\t}\n}\n\nfunc (o *PauseOptions) BindFlags(flags *pflag.FlagSet) {\n\to.SkipOptions.BindFlags(flags)\n}\n\nfunc runPause(f client.Factory, o *cli.SelectOptions, paused bool, skipImmediately *bool) error {\n\tcrClient, err := f.KubebuilderClient()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tvar (\n\t\tschedules []*velerov1api.Schedule\n\t\terrs []error\n\t)\n\tswitch {\n\tcase len(o.Names) > 0:\n\t\tfor _, name := range o.Names {\n\t\t\tschedule := new(velerov1api.Schedule)\n\t\t\terr := crClient.Get(context.TODO(), ctrlclient.ObjectKey{Name: name, Namespace: f.Namespace()}, schedule)\n\t\t\tif err != nil {\n\t\t\t\terrs = append(errs, errors.WithStack(err))\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tschedules = append(schedules, schedule)\n\t\t}\n\tdefault:\n\t\tselector := labels.Everything()\n\t\tif o.Selector.LabelSelector != nil {\n\t\t\tconvertedSelector, err := metav1.LabelSelectorAsSelector(o.Selector.LabelSelector)\n\t\t\tif err != nil {\n\t\t\t\treturn errors.WithStack(err)\n\t\t\t}\n\t\t\tselector = convertedSelector\n\t\t}\n\t\tres := new(velerov1api.ScheduleList)\n\t\terr := crClient.List(context.TODO(), res, &ctrlclient.ListOptions{\n\t\t\tLabelSelector: selector,\n\t\t})\n\t\tif err != nil {\n\t\t\terrs = append(errs, errors.WithStack(err))\n\t\t}\n\n\t\tfor i := range res.Items {\n\t\t\tschedules = append(schedules, &res.Items[i])\n\t\t}\n\t}\n\tif len(schedules) == 0 {\n\t\tfmt.Println(\"No schedules found\")\n\t\treturn nil\n\t}\n\n\tmsg := \"paused\"\n\tif !paused {\n\t\tmsg = \"unpaused\"\n\t}\n\tfor _, schedule := range schedules {\n\t\tif schedule.Spec.Paused == paused {\n\t\t\tfmt.Printf(\"Schedule %s is already %s, skip\\n\", schedule.Name, msg)\n\t\t\tcontinue\n\t\t}\n\t\tschedule.Spec.Paused = paused\n\t\tschedule.Spec.SkipImmediately = skipImmediately\n\t\tif err := crClient.Update(context.TODO(), schedule); err != nil {\n\t\t\treturn errors.Wrapf(err, \"failed to update schedule %s\", schedule.Name)\n\t\t}\n\t\tfmt.Printf(\"Schedule %s %s successfully\\n\", schedule.Name, msg)\n\t}\n\treturn kubeerrs.NewAggregate(errs)\n}\n" }, { "path": "pkg/cmd/cli/schedule/schedule.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage schedule\n\nimport (\n\t\"github.com/spf13/cobra\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n)\n\nfunc NewCommand(f client.Factory) *cobra.Command {\n\tc := &cobra.Command{\n\t\tUse: \"schedule\",\n\t\tShort: \"Work with schedules\",\n\t\tLong: \"Work with schedules\",\n\t}\n\n\tc.AddCommand(\n\t\tNewCreateCommand(f, \"create\"),\n\t\tNewGetCommand(f, \"get\"),\n\t\tNewDescribeCommand(f, \"describe\"),\n\t\tNewDeleteCommand(f, \"delete\"),\n\t\tNewPauseCommand(f, \"pause\"),\n\t\tNewUnpauseCommand(f, \"unpause\"),\n\t)\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/schedule/skip_options.go", "content": "package schedule\n\nimport (\n\t\"github.com/spf13/pflag\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/flag\"\n)\n\ntype SkipOptions struct {\n\tSkipImmediately flag.OptionalBool\n}\n\nfunc NewSkipOptions() *SkipOptions {\n\treturn &SkipOptions{}\n}\n\nfunc (o *SkipOptions) BindFlags(flags *pflag.FlagSet) {\n\tf := flags.VarPF(&o.SkipImmediately, \"skip-immediately\", \"\", \"Skip the next scheduled backup immediately\")\n\tf.NoOptDefVal = \"\" // default to nil so server options can take precedence\n}\n" }, { "path": "pkg/cmd/cli/schedule/unpause.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage schedule\n\nimport (\n\t\"github.com/spf13/cobra\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli\"\n)\n\n// NewUnpauseCommand creates the command for unpause\nfunc NewUnpauseCommand(f client.Factory, use string) *cobra.Command {\n\to := cli.NewSelectOptions(\"pause\", \"schedule\")\n\tpauseOpts := NewPauseOptions()\n\tc := &cobra.Command{\n\t\tUse: use,\n\t\tShort: \"Unpause schedules\",\n\t\tExample: ` # Unpause a schedule named \"schedule-1\".\n velero schedule unpause schedule-1\n\n # Unpause schedules named \"schedule-1\" and \"schedule-2\".\n velero schedule unpause schedule-1 schedule-2\n\n # Unpause all schedules labeled with \"foo=bar\".\n velero schedule unpause --selector foo=bar\n\n # Unpause all schedules.\n velero schedule unpause --all`,\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tcmd.CheckError(o.Complete(args))\n\t\t\tcmd.CheckError(o.Validate())\n\t\t\tcmd.CheckError(runPause(f, o, false, pauseOpts.SkipOptions.SkipImmediately.Value))\n\t\t},\n\t}\n\n\to.BindFlags(c.Flags())\n\tpauseOpts.BindFlags(c.Flags())\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/select_option.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage cli\n\nimport (\n\t\"errors\"\n\n\t\"github.com/spf13/pflag\"\n\t\"golang.org/x/text/cases\"\n\t\"golang.org/x/text/language\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/flag\"\n)\n\n// SelectOptions defines the options for selecting resources\ntype SelectOptions struct {\n\tNames []string\n\tAll bool\n\tSelector flag.LabelSelector\n\tCMD string\n\tSingularTypeName string\n}\n\n// NewSelectOptions creates a new option for selector\nfunc NewSelectOptions(cmd, singularTypeName string) *SelectOptions {\n\treturn &SelectOptions{\n\t\tCMD: cmd,\n\t\tSingularTypeName: singularTypeName,\n\t}\n}\n\n// Complete fills in the correct values for all the options.\nfunc (o *SelectOptions) Complete(args []string) error {\n\to.Names = args\n\treturn nil\n}\n\n// Validate validates the fields of the SelectOptions struct.\nfunc (o *SelectOptions) Validate() error {\n\tvar (\n\t\thasNames = len(o.Names) > 0\n\t\thasAll = o.All\n\t\thasSelector = o.Selector.LabelSelector != nil\n\t)\n\tif !xor(hasNames, hasAll, hasSelector) {\n\t\treturn errors.New(\"you must specify exactly one of: specific \" + o.SingularTypeName + \" name(s), the --all flag, or the --selector flag\")\n\t}\n\n\treturn nil\n}\n\n// BindFlags binds options for this command to flags.\nfunc (o *SelectOptions) BindFlags(flags *pflag.FlagSet) {\n\tflags.BoolVar(&o.All, \"all\", o.All, cases.Title(language.Und).String(o.CMD)+\" all \"+o.SingularTypeName+\"s\")\n\tflags.VarP(&o.Selector, \"selector\", \"l\", cases.Title(language.Und).String(o.CMD)+\" all \"+o.SingularTypeName+\"s matching this label selector.\")\n}\n" }, { "path": "pkg/cmd/cli/select_option_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage cli\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/flag\"\n)\n\nfunc TestCompleteOfSelectOption(t *testing.T) {\n\toption := &SelectOptions{}\n\targs := []string{\"arg1\", \"arg2\"}\n\trequire.NoError(t, option.Complete(args))\n\tassert.Equal(t, args, option.Names)\n}\n\nfunc TestValidateOfSelectOption(t *testing.T) {\n\toption := &SelectOptions{\n\t\tNames: nil,\n\t\tSelector: flag.LabelSelector{},\n\t\tAll: false,\n\t}\n\trequire.Error(t, option.Validate())\n\n\toption.All = true\n\tassert.NoError(t, option.Validate())\n}\n" }, { "path": "pkg/cmd/cli/serverstatus/server_status.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage serverstatus\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tveleroclient \"github.com/vmware-tanzu/velero/pkg/client\"\n)\n\ntype Getter interface {\n\tGetServerStatus(kbClient kbclient.Client) (*velerov1api.ServerStatusRequest, error)\n}\n\ntype DefaultServerStatusGetter struct {\n\tNamespace string\n\tContext context.Context\n}\n\nfunc (g *DefaultServerStatusGetter) GetServerStatus(kbClient kbclient.Client) (*velerov1api.ServerStatusRequest, error) {\n\tcreated := builder.ForServerStatusRequest(g.Namespace, \"\", \"0\").ObjectMeta(builder.WithGenerateName(\"velero-cli-\")).Result()\n\n\tif err := veleroclient.CreateRetryGenerateName(kbClient, context.Background(), created); err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tctx, cancel := context.WithCancel(g.Context)\n\tdefer cancel()\n\n\tkey := kbclient.ObjectKey{Name: created.Name, Namespace: g.Namespace}\n\tcheckFunc := func() {\n\t\tupdated := &velerov1api.ServerStatusRequest{}\n\t\tif err := kbClient.Get(ctx, key, updated); err != nil {\n\t\t\treturn\n\t\t}\n\n\t\t// TODO: once the minimum supported Kubernetes version is v1.9.0, remove the following check.\n\t\t// See http://issue.k8s.io/51046 for details.\n\t\tif updated.Name != created.Name {\n\t\t\treturn\n\t\t}\n\n\t\tif updated.Status.Phase == velerov1api.ServerStatusRequestPhaseProcessed {\n\t\t\tcreated = updated\n\t\t\tcancel()\n\t\t}\n\t}\n\n\twait.Until(checkFunc, 250*time.Millisecond, ctx.Done())\n\n\terr := ctx.Err()\n\t// context.Canceled error means we have received a processed ServerStatusRequest\n\tif err == context.Canceled {\n\t\terr = nil\n\t}\n\n\treturn created, err\n}\n" }, { "path": "pkg/cmd/cli/snapshotlocation/create.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage snapshotlocation\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/spf13/cobra\"\n\t\"github.com/spf13/pflag\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/flag\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/output\"\n)\n\nfunc NewCreateCommand(f client.Factory, use string) *cobra.Command {\n\to := NewCreateOptions()\n\n\tc := &cobra.Command{\n\t\tUse: use + \" NAME\",\n\t\tShort: \"Create a volume snapshot location\",\n\t\tArgs: cobra.ExactArgs(1),\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tcmd.CheckError(o.Complete(args, f))\n\t\t\tcmd.CheckError(o.Validate(c, args, f))\n\t\t\tcmd.CheckError(o.Run(c, f))\n\t\t},\n\t}\n\n\to.BindFlags(c.Flags())\n\toutput.BindFlags(c.Flags())\n\toutput.ClearOutputFlagDefault(c)\n\n\treturn c\n}\n\ntype CreateOptions struct {\n\tName string\n\tProvider string\n\tConfig flag.Map\n\tLabels flag.Map\n\tCredential flag.Map\n}\n\nfunc NewCreateOptions() *CreateOptions {\n\treturn &CreateOptions{\n\t\tConfig: flag.NewMap(),\n\t\tLabels: flag.NewMap(),\n\t\tCredential: flag.NewMap(),\n\t}\n}\n\nfunc (o *CreateOptions) BindFlags(flags *pflag.FlagSet) {\n\tflags.StringVar(&o.Provider, \"provider\", o.Provider, \"Name of the volume snapshot provider (e.g. aws, azure, gcp).\")\n\tflags.Var(&o.Config, \"config\", \"Configuration key-value pairs.\")\n\tflags.Var(&o.Labels, \"labels\", \"Labels to apply to the volume snapshot location.\")\n\tflags.Var(&o.Credential, \"credential\", \"The credential to be used by this location as a key-value pair, where the key is the Kubernetes Secret name, and the value is the data key name within the Secret. Optional, one value only.\")\n}\n\nfunc (o *CreateOptions) Validate(c *cobra.Command, args []string, f client.Factory) error {\n\tif err := output.ValidateFlags(c); err != nil {\n\t\treturn err\n\t}\n\n\tif o.Provider == \"\" {\n\t\treturn errors.New(\"--provider is required\")\n\t}\n\n\tif len(o.Credential.Data()) > 1 {\n\t\treturn errors.New(\"--credential can only contain 1 key/value pair\")\n\t}\n\n\treturn nil\n}\n\nfunc (o *CreateOptions) Complete(args []string, f client.Factory) error {\n\to.Name = args[0]\n\treturn nil\n}\n\nfunc (o *CreateOptions) BuildVolumeSnapshotLocation(namespace string) *api.VolumeSnapshotLocation {\n\tvolumeSnapshotLocation := &api.VolumeSnapshotLocation{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: namespace,\n\t\t\tName: o.Name,\n\t\t\tLabels: o.Labels.Data(),\n\t\t},\n\t\tSpec: api.VolumeSnapshotLocationSpec{\n\t\t\tProvider: o.Provider,\n\t\t\tConfig: o.Config.Data(),\n\t\t},\n\t}\n\tfor secretName, secretKey := range o.Credential.Data() {\n\t\tvolumeSnapshotLocation.Spec.Credential = builder.ForSecretKeySelector(secretName, secretKey).Result()\n\t\tbreak\n\t}\n\treturn volumeSnapshotLocation\n}\n\nfunc (o *CreateOptions) Run(c *cobra.Command, f client.Factory) error {\n\tvolumeSnapshotLocation := o.BuildVolumeSnapshotLocation(f.Namespace())\n\n\tif printed, err := output.PrintWithFormat(c, volumeSnapshotLocation); printed || err != nil {\n\t\treturn err\n\t}\n\n\tclient, err := f.KubebuilderClient()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif err := client.Create(context.TODO(), volumeSnapshotLocation); err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\tfmt.Printf(\"Snapshot volume location %q configured successfully.\\n\", volumeSnapshotLocation.Name)\n\treturn nil\n}\n" }, { "path": "pkg/cmd/cli/snapshotlocation/get.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\n\nYou may obtain a copy of the License at\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage snapshotlocation\n\nimport (\n\t\"context\"\n\n\t\"github.com/spf13/cobra\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/output\"\n)\n\nfunc NewGetCommand(f client.Factory, use string) *cobra.Command {\n\tvar listOptions metav1.ListOptions\n\tc := &cobra.Command{\n\t\tUse: use,\n\t\tShort: \"Get snapshot locations\",\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\terr := output.ValidateFlags(c)\n\t\t\tcmd.CheckError(err)\n\t\t\tclient, err := f.KubebuilderClient()\n\t\t\tcmd.CheckError(err)\n\t\t\tlocations := new(api.VolumeSnapshotLocationList)\n\n\t\t\tif len(args) > 0 {\n\t\t\t\tfor _, name := range args {\n\t\t\t\t\tlocation := new(api.VolumeSnapshotLocation)\n\t\t\t\t\terr := client.Get(context.TODO(), kbclient.ObjectKey{Namespace: f.Namespace(), Name: name}, location)\n\t\t\t\t\tcmd.CheckError(err)\n\t\t\t\t\tlocations.Items = append(locations.Items, *location)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\terr = client.List(context.TODO(), locations, &kbclient.ListOptions{Namespace: f.Namespace()})\n\t\t\t\tcmd.CheckError(err)\n\t\t\t}\n\t\t\t_, err = output.PrintWithFormat(c, locations)\n\t\t\tcmd.CheckError(err)\n\t\t},\n\t}\n\tc.Flags().StringVarP(&listOptions.LabelSelector, \"selector\", \"l\", listOptions.LabelSelector, \"Only show items matching this label selector\")\n\toutput.BindFlags(c.Flags())\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/snapshotlocation/set.go", "content": "/*\nCopyright The Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage snapshotlocation\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/spf13/cobra\"\n\t\"github.com/spf13/pflag\"\n\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/flag\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/output\"\n)\n\nfunc NewSetCommand(f client.Factory, use string) *cobra.Command {\n\to := NewSetOptions()\n\n\tc := &cobra.Command{\n\t\tUse: use + \" NAME\",\n\t\tShort: \"Set specific features for a snapshot location\",\n\t\tArgs: cobra.ExactArgs(1),\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tcmd.CheckError(o.Complete(args, f))\n\t\t\tcmd.CheckError(o.Validate(c, args, f))\n\t\t\tcmd.CheckError(o.Run(c, f))\n\t\t},\n\t}\n\n\to.BindFlags(c.Flags())\n\treturn c\n}\n\ntype SetOptions struct {\n\tName string\n\tCredential flag.Map\n}\n\nfunc NewSetOptions() *SetOptions {\n\treturn &SetOptions{\n\t\tCredential: flag.NewMap(),\n\t}\n}\n\nfunc (o *SetOptions) BindFlags(flags *pflag.FlagSet) {\n\tflags.Var(&o.Credential, \"credential\", \"Sets the credential to be used by this location as a key-value pair, where the key is the Kubernetes Secret name, and the value is the data key name within the Secret. Optional, one value only.\")\n}\n\nfunc (o *SetOptions) Validate(c *cobra.Command, args []string, f client.Factory) error {\n\tif err := output.ValidateFlags(c); err != nil {\n\t\treturn err\n\t}\n\n\tif len(o.Credential.Data()) > 1 {\n\t\treturn errors.New(\"--credential can only contain 1 key/value pair\")\n\t}\n\n\treturn nil\n}\n\nfunc (o *SetOptions) Complete(args []string, f client.Factory) error {\n\to.Name = args[0]\n\treturn nil\n}\n\nfunc (o *SetOptions) Run(c *cobra.Command, f client.Factory) error {\n\tkbClient, err := f.KubebuilderClient()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tlocation := &velerov1api.VolumeSnapshotLocation{}\n\terr = kbClient.Get(context.Background(), kbclient.ObjectKey{\n\t\tNamespace: f.Namespace(),\n\t\tName: o.Name,\n\t}, location)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\tfor name, key := range o.Credential.Data() {\n\t\tlocation.Spec.Credential = builder.ForSecretKeySelector(name, key).Result()\n\t\tbreak\n\t}\n\n\tif err := kbClient.Update(context.Background(), location, &kbclient.UpdateOptions{}); err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\tfmt.Printf(\"Volume snapshot location %q configured successfully.\\n\", o.Name)\n\treturn nil\n}\n" }, { "path": "pkg/cmd/cli/snapshotlocation/snapshot_location.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage snapshotlocation\n\nimport (\n\t\"github.com/spf13/cobra\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n)\n\nfunc NewCommand(f client.Factory) *cobra.Command {\n\tc := &cobra.Command{\n\t\tUse: \"snapshot-location\",\n\t\tShort: \"Work with snapshot locations\",\n\t\tLong: \"Work with snapshot locations\",\n\t}\n\n\tc.AddCommand(\n\t\tNewCreateCommand(f, \"create\"),\n\t\tNewGetCommand(f, \"get\"),\n\t\tNewSetCommand(f, \"set\"),\n\t)\n\n\treturn c\n}\n" }, { "path": "pkg/cmd/cli/uninstall/uninstall.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage uninstall\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"reflect\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/spf13/cobra\"\n\t\"github.com/spf13/pflag\"\n\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapiextv1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1\"\n\tapiextv1beta1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\t\"k8s.io/apimachinery/pkg/api/meta\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\tkubeerrs \"k8s.io/apimachinery/pkg/util/errors\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/confirm\"\n\t\"github.com/vmware-tanzu/velero/pkg/controller\"\n\t\"github.com/vmware-tanzu/velero/pkg/install\"\n\tkubeutil \"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nvar gracefulDeletionMaximumDuration = 1 * time.Minute\n\nvar resToDelete = []kbclient.ObjectList{}\n\n// uninstallOptions collects all the options for uninstalling Velero from a Kubernetes cluster.\ntype uninstallOptions struct {\n\twait bool // deprecated\n\tforce bool\n}\n\n// BindFlags adds command line values to the options struct.\nfunc (o *uninstallOptions) BindFlags(flags *pflag.FlagSet) {\n\tflags.BoolVar(&o.wait, \"wait\", o.wait, \"Wait for Velero uninstall to be ready. Optional. Deprecated.\")\n\tflags.BoolVar(&o.force, \"force\", o.force, \"Forces the Velero uninstall. Optional.\")\n}\n\n// NewCommand creates a cobra command.\nfunc NewCommand(f client.Factory) *cobra.Command {\n\to := &uninstallOptions{}\n\n\tc := &cobra.Command{\n\t\tUse: \"uninstall\",\n\t\tShort: \"Uninstall Velero\",\n\t\tLong: `Uninstall Velero along with the CRDs and clusterrolebinding.\n\nThe '--namespace' flag can be used to specify the namespace where velero is installed (default: velero).\nUse '--force' to skip the prompt confirming if you want to uninstall Velero.\n\t\t`,\n\t\tExample: ` # velero uninstall --namespace staging`,\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tif o.wait {\n\t\t\t\tfmt.Println(\"Warning: the \\\"--wait\\\" option is deprecated and will be removed in a future release. The uninstall command always waits for the uninstall to complete.\")\n\t\t\t}\n\n\t\t\t// Confirm if not asked to force-skip confirmation\n\t\t\tif !o.force {\n\t\t\t\tfmt.Println(\"You are about to uninstall Velero.\")\n\t\t\t\tif !confirm.GetConfirmation() {\n\t\t\t\t\t// Don't do anything unless we get confirmation\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tkbClient, err := f.KubebuilderClient()\n\t\t\tcmd.CheckError(err)\n\t\t\tcmd.CheckError(Run(context.Background(), kbClient, f.Namespace()))\n\t\t},\n\t}\n\n\to.BindFlags(c.Flags())\n\treturn c\n}\n\n// Run removes all components that were deployed using the Velero install command\nfunc Run(ctx context.Context, kbClient kbclient.Client, namespace string) error {\n\t// The CRDs cannot be removed until the namespace is deleted to avoid the problem in issue #3974 so if the namespace deletion fails we error out here\n\tif err := deleteNamespace(ctx, kbClient, namespace); err != nil {\n\t\tfmt.Printf(\"Errors while attempting to uninstall Velero: %q \\n\", err)\n\t\treturn err\n\t}\n\n\tvar errs []error\n\n\t// ClusterRoleBinding\n\tcrb := install.ClusterRoleBinding(namespace)\n\tkey := kbclient.ObjectKey{Name: crb.Name}\n\tif err := kbClient.Get(ctx, key, crb); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tfmt.Printf(\"Velero ClusterRoleBinding %q does not exist, skipping.\\n\", crb.Name)\n\t\t} else {\n\t\t\terrs = append(errs, errors.WithStack(err))\n\t\t}\n\t} else {\n\t\tif err := kbClient.Delete(ctx, crb); err != nil {\n\t\t\terrs = append(errs, errors.WithStack(err))\n\t\t}\n\t}\n\n\t// CRDs\n\tveleroLabelSelector := labels.SelectorFromSet(install.Labels())\n\topts := []kbclient.DeleteAllOfOption{\n\t\tkbclient.InNamespace(namespace),\n\t\tkbclient.MatchingLabelsSelector{\n\t\t\tSelector: veleroLabelSelector,\n\t\t},\n\t}\n\tv1CRDsRemoved := false\n\tv1crd := &apiextv1.CustomResourceDefinition{}\n\tif err := kbClient.DeleteAllOf(ctx, v1crd, opts...); err != nil {\n\t\tif meta.IsNoMatchError(err) {\n\t\t\tfmt.Println(\"V1 Velero CRDs not found, skipping...\")\n\t\t} else {\n\t\t\terrs = append(errs, errors.WithStack(err))\n\t\t}\n\t} else {\n\t\tv1CRDsRemoved = true\n\t}\n\n\t// Remove any old Velero v1beta1 CRDs hanging around.\n\tv1beta1crd := &apiextv1beta1.CustomResourceDefinition{}\n\tif err := kbClient.DeleteAllOf(ctx, v1beta1crd, opts...); err != nil {\n\t\tif meta.IsNoMatchError(err) {\n\t\t\tif !v1CRDsRemoved {\n\t\t\t\t// Only mention this if there were no V1 CRDs removed\n\t\t\t\tfmt.Println(\"V1Beta1 Velero CRDs not found, skipping...\")\n\t\t\t}\n\t\t} else {\n\t\t\terrs = append(errs, errors.WithStack(err))\n\t\t}\n\t}\n\n\tif kubeerrs.NewAggregate(errs) != nil {\n\t\tfmt.Printf(\"Errors while attempting to uninstall Velero: %q \\n\", kubeerrs.NewAggregate(errs))\n\t\treturn kubeerrs.NewAggregate(errs)\n\t}\n\n\tfmt.Println(\"Velero uninstalled ⛵\")\n\treturn nil\n}\n\nfunc deleteNamespace(ctx context.Context, kbClient kbclient.Client, namespace string) error {\n\t// First check if it's already been deleted\n\tns := &corev1api.Namespace{}\n\tkey := kbclient.ObjectKey{Name: namespace}\n\tif err := kbClient.Get(ctx, key, ns); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tfmt.Printf(\"Velero namespace %q does not exist, skipping.\\n\", namespace)\n\t\t\treturn nil\n\t\t}\n\t\treturn err\n\t}\n\n\t// Deal with resources with attached finalizers to ensure proper handling of those finalizers.\n\tif err := deleteResourcesWithFinalizer(ctx, kbClient, namespace); err != nil {\n\t\treturn errors.Wrap(err, \"Fail to remove finalizer from restores\")\n\t}\n\n\tif err := kbClient.Delete(ctx, ns); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tfmt.Printf(\"Velero namespace %q does not exist, skipping.\\n\", namespace)\n\t\t\treturn nil\n\t\t}\n\t\treturn err\n\t}\n\tfmt.Println()\n\tfmt.Printf(\"Waiting for velero namespace %q to be deleted\\n\", namespace)\n\tctx, cancel := context.WithCancel(ctx)\n\tdefer cancel()\n\n\tvar err error\n\tcheckFunc := func() {\n\t\tif err = kbClient.Get(ctx, key, ns); err != nil {\n\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\tfmt.Print(\"\\n\")\n\t\t\t\terr = nil\n\t\t\t}\n\t\t\tcancel()\n\t\t\treturn\n\t\t}\n\t\tfmt.Print(\".\")\n\t}\n\n\t// Must wait until the namespace is deleted to avoid the issue https://github.com/vmware-tanzu/velero/issues/3974\n\twait.Until(checkFunc, 5*time.Millisecond, ctx.Done())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfmt.Printf(\"Velero namespace %q deleted\\n\", namespace)\n\treturn nil\n}\n\n// A few things needed to be noticed here:\n// 1. When we delete resources with attached finalizers, the corresponding controller will deal with the finalizer then resources can be deleted successfully.\n// So it is important to delete these resources before deleting the pod that runs that controller.\n// 2. The controller may encounter errors while handling the finalizer, in such case, the controller will keep trying until it succeeds.\n// So it is important to set a timeout, once the process exceed the timeout, we will forcedly delete the resources by removing the finalizer from them,\n// otherwise the deletion process may get stuck indefinitely.\n// 3. There is only resources finalizer supported as of v1.12. If any new finalizers are added in the future, the corresponding deletion logic can be\n// incorporated into this function.\nfunc deleteResourcesWithFinalizer(ctx context.Context, kbClient kbclient.Client, namespace string) error {\n\tfmt.Println(\"Waiting for resource with attached finalizer to be deleted\")\n\treturn deleteResources(ctx, kbClient, namespace)\n}\n\nfunc checkResources(ctx context.Context, kbClient kbclient.Client) error {\n\tcheckCRDs := []string{\"restores.velero.io\", \"datauploads.velero.io\", \"datadownloads.velero.io\"}\n\tvar err error\n\tv1crd := &apiextv1.CustomResourceDefinition{}\n\tfor _, crd := range checkCRDs {\n\t\tkey := kbclient.ObjectKey{Name: crd}\n\t\tif err = kbClient.Get(ctx, key, v1crd); err != nil {\n\t\t\tif !apierrors.IsNotFound(err) {\n\t\t\t\treturn errors.Wrapf(err, \"Error getting %s crd\", crd)\n\t\t\t}\n\t\t} else {\n\t\t\t// no error with found CRD that we should delete\n\t\t\tswitch crd {\n\t\t\tcase \"restores.velero.io\":\n\t\t\t\tresToDelete = append(resToDelete, &velerov1api.RestoreList{})\n\t\t\tcase \"datauploads.velero.io\":\n\t\t\t\tresToDelete = append(resToDelete, &velerov2alpha1api.DataUploadList{})\n\t\t\tcase \"datadownloads.velero.io\":\n\t\t\t\tresToDelete = append(resToDelete, &velerov2alpha1api.DataDownloadList{})\n\t\t\tdefault:\n\t\t\t\tfmt.Printf(\"Unsupported type %s to be cleared\\n\", crd)\n\t\t\t}\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc deleteResources(ctx context.Context, kbClient kbclient.Client, namespace string) error {\n\t// Check if resources crd exists, if it does not exist, return immediately.\n\terr := checkResources(ctx, kbClient)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// First attempt to gracefully delete all the resources within a specified time frame, If the process exceeds the timeout limit,\n\t// it is likely that there may be errors during the finalization of restores. In such cases, we should proceed with forcefully deleting the restores.\n\terr = gracefullyDeleteResources(ctx, kbClient, namespace)\n\tif err != nil && !wait.Interrupted(err) {\n\t\treturn errors.Wrap(err, \"Error deleting resources\")\n\t}\n\n\tif wait.Interrupted(err) {\n\t\terr = forcedlyDeleteResources(ctx, kbClient, namespace)\n\t\tif err != nil {\n\t\t\treturn errors.Wrap(err, \"Error deleting resources forcedly\")\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc gracefullyDeleteResources(ctx context.Context, kbClient kbclient.Client, namespace string) error {\n\terrorChan := make(chan error)\n\n\tvar wg sync.WaitGroup\n\twg.Add(len(resToDelete))\n\n\tfor i := range resToDelete {\n\t\tgo func(index int) {\n\t\t\tdefer wg.Done()\n\t\t\terrorChan <- gracefullyDeleteResource(ctx, kbClient, namespace, resToDelete[index])\n\t\t}(i)\n\t}\n\n\tgo func() {\n\t\twg.Wait()\n\t\tclose(errorChan)\n\t}()\n\n\tfor err := range errorChan {\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn waitDeletingResources(ctx, kbClient, namespace)\n}\n\nfunc gracefullyDeleteResource(ctx context.Context, kbClient kbclient.Client, namespace string, list kbclient.ObjectList) error {\n\tif err := kbClient.List(ctx, list, &kbclient.ListOptions{Namespace: namespace}); err != nil {\n\t\treturn errors.Wrap(err, \"Error getting resources during graceful deletion\")\n\t}\n\n\tvar objectsToDelete []kbclient.Object\n\titems := reflect.ValueOf(list).Elem().FieldByName(\"Items\")\n\n\tfor i := 0; i < items.Len(); i++ {\n\t\titem := items.Index(i).Addr().Interface()\n\t\t// Type assertion to cast item to the appropriate type\n\t\tswitch typedItem := item.(type) {\n\t\tcase *velerov1api.Restore:\n\t\t\tobjectsToDelete = append(objectsToDelete, typedItem)\n\t\tcase *velerov2alpha1api.DataUpload:\n\t\t\tobjectsToDelete = append(objectsToDelete, typedItem)\n\t\tcase *velerov2alpha1api.DataDownload:\n\t\t\tobjectsToDelete = append(objectsToDelete, typedItem)\n\t\tdefault:\n\t\t\treturn errors.New(\"Unsupported resource type\")\n\t\t}\n\t}\n\n\t// Delete collected resources in a batch\n\tfor _, resource := range objectsToDelete {\n\t\tif err := kbClient.Delete(ctx, resource); err != nil {\n\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\treturn errors.Wrap(err, \"Error deleting resources during graceful deletion\")\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc waitDeletingResources(ctx context.Context, kbClient kbclient.Client, namespace string) error {\n\t// Wait for the deletion of all the restores within a specified time frame\n\terr := wait.PollUntilContextTimeout(ctx, time.Second, gracefulDeletionMaximumDuration, true, func(ctx context.Context) (bool, error) {\n\t\titemsCount := 0\n\t\tfor i := range resToDelete {\n\t\t\tif errList := kbClient.List(ctx, resToDelete[i], &kbclient.ListOptions{Namespace: namespace}); errList != nil {\n\t\t\t\treturn false, errList\n\t\t\t}\n\t\t\titemsCount += reflect.ValueOf(resToDelete[i]).Elem().FieldByName(\"Items\").Len()\n\t\t\tif itemsCount > 0 {\n\t\t\t\tfmt.Print(\".\")\n\t\t\t\treturn false, nil\n\t\t\t}\n\t\t}\n\n\t\treturn true, nil\n\t})\n\n\treturn err\n}\n\nfunc forcedlyDeleteResources(ctx context.Context, kbClient kbclient.Client, namespace string) error {\n\t// Delete velero deployment first in case:\n\t// 1. finalizers will be added back by resources related controller after they are removed at next step;\n\t// 2. new resources attached with finalizer will be created by controller after we remove all the resources' finalizer at next step;\n\tdeploy := &appsv1api.Deployment{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"velero\",\n\t\t\tName: namespace,\n\t\t},\n\t}\n\n\terr := kbClient.Delete(ctx, deploy)\n\tif err != nil && !apierrors.IsNotFound(err) {\n\t\treturn errors.Wrap(err, \"Error deleting velero deployment during force deletion\")\n\t}\n\n\tctxc, cancel := context.WithCancel(ctx)\n\tdefer cancel()\n\n\tcheckFunc := func() {\n\t\tdeploy := &appsv1api.Deployment{}\n\t\tkey := kbclient.ObjectKey{Namespace: namespace, Name: \"velero\"}\n\n\t\tif err = kbClient.Get(ctxc, key, deploy); err != nil {\n\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\terr = nil\n\t\t\t}\n\t\t\tcancel()\n\t\t\treturn\n\t\t}\n\t}\n\n\t// Wait until velero deployment are deleted.\n\twait.Until(checkFunc, 100*time.Millisecond, ctxc.Done())\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"Error deleting velero deployment during force deletion\")\n\t}\n\treturn removeResourcesFinalizer(ctx, kbClient, namespace)\n}\n\nfunc removeResourcesFinalizer(ctx context.Context, kbClient kbclient.Client, namespace string) error {\n\tfor i := range resToDelete {\n\t\tif err := removeResourceFinalizer(ctx, kbClient, namespace, resToDelete[i]); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc removeResourceFinalizer(ctx context.Context, kbClient kbclient.Client, namespace string, resourceList kbclient.ObjectList) error {\n\tlistOptions := &kbclient.ListOptions{Namespace: namespace}\n\tif err := kbClient.List(ctx, resourceList, listOptions); err != nil {\n\t\treturn errors.Wrap(err, fmt.Sprintf(\"Error getting resources of type %T during force deletion\", resourceList))\n\t}\n\n\titems := reflect.ValueOf(resourceList).Elem().FieldByName(\"Items\")\n\tvar err error\n\tfor i := 0; i < items.Len(); i++ {\n\t\titem := items.Index(i).Addr().Interface()\n\t\t// Type assertion to cast item to the appropriate type\n\t\tswitch typedItem := item.(type) {\n\t\tcase *velerov1api.Restore:\n\t\t\terr = removeFinalizerForObject(typedItem, controller.ExternalResourcesFinalizer, kbClient)\n\t\tcase *velerov2alpha1api.DataUpload:\n\t\t\terr = removeFinalizerForObject(typedItem, controller.DataUploadDownloadFinalizer, kbClient)\n\t\tcase *velerov2alpha1api.DataDownload:\n\t\t\terr = removeFinalizerForObject(typedItem, controller.DataUploadDownloadFinalizer, kbClient)\n\t\tdefault:\n\t\t\terr = errors.Errorf(\"Unsupported resource type %T\", typedItem)\n\t\t}\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc removeFinalizerForObject(obj kbclient.Object, finalizer string, kbClient kbclient.Client) error {\n\tif controllerutil.ContainsFinalizer(obj, finalizer) {\n\t\tupdate := obj.DeepCopyObject().(kbclient.Object)\n\t\toriginal := obj.DeepCopyObject().(kbclient.Object)\n\n\t\tcontrollerutil.RemoveFinalizer(update, finalizer)\n\t\tif err := kubeutil.PatchResource(original, update, kbClient); err != nil {\n\t\t\treturn errors.Wrap(err, fmt.Sprintf(\"Error removing finalizer %q during force deletion\", finalizer))\n\t\t}\n\t}\n\treturn nil\n}\n" }, { "path": "pkg/cmd/cli/version/version.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage version\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"time\"\n\n\t\"github.com/spf13/cobra\"\n\t\"golang.org/x/mod/semver\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/buildinfo\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/serverstatus\"\n)\n\nfunc NewCommand(f client.Factory) *cobra.Command {\n\tvar clientOnly bool\n\ttimeout := 5 * time.Second\n\n\tc := &cobra.Command{\n\t\tUse: \"version\",\n\t\tShort: \"Print the velero version and associated image\",\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tvar kbClient kbclient.Client\n\t\t\tif !clientOnly {\n\t\t\t\tvar err error\n\t\t\t\tkbClient, err = f.KubebuilderClient()\n\t\t\t\tcmd.CheckError(err)\n\t\t\t}\n\n\t\t\tctx, cancel := context.WithTimeout(context.Background(), timeout)\n\t\t\tdefer cancel()\n\n\t\t\tserverStatusGetter := &serverstatus.DefaultServerStatusGetter{\n\t\t\t\tNamespace: f.Namespace(),\n\t\t\t\tContext: ctx,\n\t\t\t}\n\t\t\tprintVersion(os.Stdout, clientOnly, kbClient, serverStatusGetter)\n\t\t},\n\t}\n\n\tc.Flags().DurationVar(&timeout, \"timeout\", timeout, \"Maximum time to wait for server version to be reported. Default is 5 seconds.\")\n\tc.Flags().BoolVar(&clientOnly, \"client-only\", clientOnly, \"Only get velero client version, not server version\")\n\n\treturn c\n}\n\nfunc printVersion(w io.Writer, clientOnly bool, kbClient kbclient.Client, serverStatusGetter serverstatus.Getter) {\n\tfmt.Fprintln(w, \"Client:\")\n\tfmt.Fprintf(w, \"\\tVersion: %s\\n\", buildinfo.Version)\n\tfmt.Fprintf(w, \"\\tGit commit: %s\\n\", buildinfo.FormattedGitSHA())\n\n\tif clientOnly {\n\t\treturn\n\t}\n\n\tserverStatus, err := serverStatusGetter.GetServerStatus(kbClient)\n\tif err != nil {\n\t\tfmt.Fprintf(w, \"\\n\", err)\n\t\treturn\n\t}\n\n\tfmt.Fprintln(w, \"Server:\")\n\tfmt.Fprintf(w, \"\\tVersion: %s\\n\", serverStatus.Status.ServerVersion)\n\n\tserverSemVer := semver.MajorMinor(serverStatus.Status.ServerVersion)\n\tcliSemVer := semver.MajorMinor(buildinfo.Version)\n\tif serverSemVer != cliSemVer {\n\t\tupgrade := \"client\"\n\t\tcmp := semver.Compare(cliSemVer, serverSemVer)\n\t\tif cmp == 1 {\n\t\t\tupgrade = \"server\"\n\t\t}\n\t\tfmt.Fprintf(w, \"# WARNING: the client version does not match the server version. Please update %s\\n\", upgrade)\n\t}\n}\n" }, { "path": "pkg/cmd/cli/version/version_test.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage version\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/buildinfo\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestPrintVersion(t *testing.T) {\n\t// set up some non-empty buildinfo values, but put them back to their\n\t// defaults at the end of the test\n\tvar (\n\t\torigVersion = buildinfo.Version\n\t\torigGitSHA = buildinfo.GitSHA\n\t\torigGitTreeState = buildinfo.GitTreeState\n\t)\n\tdefer func() {\n\t\tbuildinfo.Version = origVersion\n\t\tbuildinfo.GitSHA = origGitSHA\n\t\tbuildinfo.GitTreeState = origGitTreeState\n\t}()\n\tbuildinfo.Version = \"v1.0.0\"\n\tbuildinfo.GitSHA = \"somegitsha\"\n\tbuildinfo.GitTreeState = \"dirty\"\n\n\tclientVersion := fmt.Sprintf(\"Client:\\n\\tVersion: %s\\n\\tGit commit: %s\\n\", buildinfo.Version, buildinfo.FormattedGitSHA())\n\n\ttests := []struct {\n\t\tname string\n\t\tclientOnly bool\n\t\tserverStatusRequest *velerov1.ServerStatusRequest\n\t\tgetterError error\n\t\twant string\n\t}{\n\t\t{\n\t\t\tname: \"client-only\",\n\t\t\tclientOnly: true,\n\t\t\twant: clientVersion,\n\t\t},\n\t\t{\n\t\t\tname: \"server status getter error\",\n\t\t\tclientOnly: false,\n\t\t\tserverStatusRequest: nil,\n\t\t\tgetterError: errors.New(\"an error\"),\n\t\t\twant: clientVersion + \"\\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"server status getter returns normally\",\n\t\t\tclientOnly: false,\n\t\t\tserverStatusRequest: builder.ForServerStatusRequest(\"velero\", \"ssr-1\", \"0\").ServerVersion(\"v1.0.1\").Result(),\n\t\t\tgetterError: nil,\n\t\t\twant: clientVersion + \"Server:\\n\\tVersion: v1.0.1\\n\",\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\tkbClient = velerotest.NewFakeControllerRuntimeClient(t)\n\t\t\t\tserverStatusGetter = new(mockServerStatusGetter)\n\t\t\t\tbuf = new(bytes.Buffer)\n\t\t\t)\n\t\t\tdefer serverStatusGetter.AssertExpectations(t)\n\n\t\t\t// GetServerStatus should only be called when clientOnly = false\n\t\t\tif !tc.clientOnly {\n\t\t\t\tserverStatusGetter.On(\"GetServerStatus\", kbClient).Return(tc.serverStatusRequest, tc.getterError)\n\t\t\t}\n\n\t\t\tprintVersion(buf, tc.clientOnly, kbClient, serverStatusGetter)\n\n\t\t\tassert.Equal(t, tc.want, buf.String())\n\t\t})\n\t}\n}\n\n// mockServerStatusGetter is an autogenerated mock type for the serverStatusGetter type\n// Code generated by mockery v2.2.1.\ntype mockServerStatusGetter struct {\n\tmock.Mock\n}\n\n// GetServerStatus provides a mock function with given fields: mgr\nfunc (_m *mockServerStatusGetter) GetServerStatus(kbClient kbclient.Client) (*velerov1.ServerStatusRequest, error) {\n\tret := _m.Called(kbClient)\n\n\tvar r0 *velerov1.ServerStatusRequest\n\tif rf, ok := ret.Get(0).(func(kbclient.Client) *velerov1.ServerStatusRequest); ok {\n\t\tr0 = rf(kbClient)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(*velerov1.ServerStatusRequest)\n\t\t}\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(kbclient.Client) error); ok {\n\t\tr1 = rf(kbClient)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n" }, { "path": "pkg/cmd/const.go", "content": "package cmd\n\nvar TRUE = \"true\"\n" }, { "path": "pkg/cmd/errors.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage cmd\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n)\n\n// CheckError prints err to stderr and exits with code 1 if err is not nil. Otherwise, it is a\n// no-op.\nfunc CheckError(err error) {\n\tif err != nil {\n\t\tif err != context.Canceled {\n\t\t\tfmt.Fprintf(os.Stderr, \"An error occurred: %v\\n\", err)\n\t\t}\n\t\tos.Exit(1)\n\t}\n}\n\n// Exit prints msg (with optional args), plus a newline, to stderr and exits with code 1.\nfunc Exit(msg string, args ...any) {\n\tfmt.Fprintf(os.Stderr, msg+\"\\n\", args...)\n\tos.Exit(1)\n}\n" }, { "path": "pkg/cmd/server/config/config.go", "content": "package config\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n\t\"time\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/spf13/pflag\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/flag\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\tpodvolumeconfigs \"github.com/vmware-tanzu/velero/pkg/podvolume/configs\"\n\t\"github.com/vmware-tanzu/velero/pkg/types\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n)\n\nconst (\n\t// the port where prometheus metrics are exposed\n\tdefaultMetricsAddress = \":8085\"\n\n\tdefaultBackupSyncPeriod = time.Minute\n\tdefaultStoreValidationFrequency = time.Minute\n\tdefaultPodVolumeOperationTimeout = 240 * time.Minute\n\tdefaultResourceTerminatingTimeout = 10 * time.Minute\n\n\t// server's client default qps and burst\n\tdefaultClientQPS float32 = 100.0\n\tdefaultClientBurst int = 100\n\tdefaultClientPageSize int = 500\n\n\tdefaultProfilerAddress = \"localhost:6060\"\n\n\t// the default TTL for a backup\n\tdefaultBackupTTL = 30 * 24 * time.Hour\n\n\tdefaultCSISnapshotTimeout = 10 * time.Minute\n\tdefaultItemOperationTimeout = 4 * time.Hour\n\n\tresourceTimeout = 10 * time.Minute\n\n\tdefaultMaxConcurrentK8SConnections = 30\n\tdefaultDisableInformerCache = false\n\n\tDefaultItemBlockWorkerCount = 1\n\tDefaultConcurrentBackups = 1\n)\n\nvar (\n\t// DisableableControllers is a list of controllers that can be disabled\n\tDisableableControllers = []string{\n\t\tconstant.ControllerBackupQueue,\n\t\tconstant.ControllerBackup,\n\t\tconstant.ControllerBackupOperations,\n\t\tconstant.ControllerBackupDeletion,\n\t\tconstant.ControllerBackupFinalizer,\n\t\tconstant.ControllerBackupSync,\n\t\tconstant.ControllerDownloadRequest,\n\t\tconstant.ControllerGarbageCollection,\n\t\tconstant.ControllerBackupRepo,\n\t\tconstant.ControllerRestore,\n\t\tconstant.ControllerRestoreOperations,\n\t\tconstant.ControllerSchedule,\n\t\tconstant.ControllerServerStatusRequest,\n\t\tconstant.ControllerRestoreFinalizer,\n\t}\n\n\t/*\n\t\tHigh priorities:\n\t\t - Custom Resource Definitions come before Custom Resource so that they can be\n\t\t restored with their corresponding CRD.\n\t\t - Namespaces go second because all namespaced resources depend on them.\n\t\t - Storage Classes are needed to create PVs and PVCs correctly.\n\t\t - VolumeSnapshotClasses are needed to provision volumes using volumesnapshots\n\t\t - VolumeSnapshotContents are needed as they contain the handle to the volume snapshot in the\n\t\t storage provider\n\t\t - VolumeSnapshots are needed to create PVCs using the VolumeSnapshot as their data source.\n\t\t - DataUploads need to restore before PVC for Snapshot DataMover to work, because PVC needs the DataUploadResults to create DataDownloads.\n\t\t - PVs go before PVCs because PVCs depend on them.\n\t\t - PVCs go before pods or controllers so they can be mounted as volumes.\n\t\t - Service accounts go before secrets so service account token secrets can be filled automatically.\n\t\t - Secrets and ConfigMaps go before pods or controllers so they can be mounted\n\t\t as volumes.\n\t\t - Limit ranges go before pods or controllers so pods can use them.\n\t\t - Pods go before controllers so they can be explicitly restored and potentially\n\t\t have pod volume restores run before controllers adopt the pods.\n\t\t - Replica sets go before deployments/other controllers so they can be explicitly\n\t\t restored and be adopted by controllers.\n\t\t - CAPI ClusterClasses go before Clusters.\n\t\t - Endpoints go before Services so no new Endpoints will be created\n\t\t - Services go before Clusters so they can be adopted by AKO-operator and no new Services will be created\n\t\t for the same clusters\n\n\t\tLow priorities:\n\t\t - Tanzu ClusterBootstraps go last as it can reference any other kind of resources.\n\t\t - ClusterBootstraps go before CAPI Clusters otherwise a new default ClusterBootstrap object is created for the cluster\n\t\t - CAPI Clusters come before ClusterResourceSets because failing to do so means the CAPI controller-manager will panic.\n\t\t Both Clusters and ClusterResourceSets need to come before ClusterResourceSetBinding in order to properly restore workload clusters.\n\t\t See https://github.com/kubernetes-sigs/cluster-api/issues/4105\n\t\t - apps.kappctrl.k14s.io and packageinstalls.packaging.carvel.dev go after workloads(pod/replicaset/etc.), otherwise the controller may\n\t\t creates new workloads before restoring them\n\t*/\n\tdefaultRestorePriorities = types.Priorities{\n\t\tHighPriorities: []string{\n\t\t\t\"customresourcedefinitions\",\n\t\t\t\"namespaces\",\n\t\t\t\"storageclasses\",\n\t\t\t\"volumesnapshotclass.snapshot.storage.k8s.io\",\n\t\t\t\"volumesnapshotcontents.snapshot.storage.k8s.io\",\n\t\t\t\"volumesnapshots.snapshot.storage.k8s.io\",\n\t\t\t\"datauploads.velero.io\",\n\t\t\t\"persistentvolumes\",\n\t\t\t\"persistentvolumeclaims\",\n\t\t\t\"clusterroles\",\n\t\t\t\"roles\",\n\t\t\t\"serviceaccounts\",\n\t\t\t\"clusterrolebindings\",\n\t\t\t\"rolebindings\",\n\t\t\t\"secrets\",\n\t\t\t\"configmaps\",\n\t\t\t\"limitranges\",\n\t\t\t\"priorityclasses\",\n\t\t\t\"pods\",\n\t\t\t// we fully qualify replicasets.apps because prior to Kubernetes 1.16, replicasets also\n\t\t\t// existed in the extensions API group, but we back up replicasets from \"apps\" so we want\n\t\t\t// to ensure that we prioritize restoring from \"apps\" too, since this is how they're stored\n\t\t\t// in the backup.\n\t\t\t\"replicasets.apps\",\n\t\t\t\"clusterclasses.cluster.x-k8s.io\",\n\t\t\t\"endpoints\",\n\t\t\t\"services\",\n\t\t},\n\t\tLowPriorities: []string{\n\t\t\t\"clusterbootstraps.run.tanzu.vmware.com\",\n\t\t\t\"clusters.cluster.x-k8s.io\",\n\t\t\t\"clusterresourcesets.addons.cluster.x-k8s.io\",\n\t\t\t\"apps.kappctrl.k14s.io\",\n\t\t\t\"packageinstalls.packaging.carvel.dev\",\n\t\t},\n\t}\n)\n\ntype Config struct {\n\tPluginDir string\n\tMetricsAddress string\n\tDefaultBackupLocation string // TODO(2.0) Deprecate defaultBackupLocation\n\tBackupSyncPeriod time.Duration\n\tPodVolumeOperationTimeout time.Duration\n\tResourceTerminatingTimeout time.Duration\n\tDefaultBackupTTL time.Duration\n\tDefaultVGSLabelKey string\n\tStoreValidationFrequency time.Duration\n\tDefaultCSISnapshotTimeout time.Duration\n\tDefaultItemOperationTimeout time.Duration\n\tResourceTimeout time.Duration\n\tRestoreResourcePriorities types.Priorities\n\tDefaultVolumeSnapshotLocations flag.Map\n\tRestoreOnly bool\n\tDisabledControllers []string\n\tClientQPS float32\n\tClientBurst int\n\tClientPageSize int\n\tProfilerAddress string\n\tLogLevel *logging.LevelFlag\n\tLogFormat *logging.FormatFlag\n\tRepoMaintenanceFrequency time.Duration\n\tGarbageCollectionFrequency time.Duration\n\tItemOperationSyncFrequency time.Duration\n\tDefaultVolumesToFsBackup bool\n\tUploaderType string\n\tMaxConcurrentK8SConnections int\n\tDefaultSnapshotMoveData bool\n\tDisableInformerCache bool\n\tScheduleSkipImmediately bool\n\tCredentialsDirectory string\n\tBackupRepoConfig string\n\tRepoMaintenanceJobConfig string\n\tItemBlockWorkerCount int\n\tConcurrentBackups int\n}\n\nfunc GetDefaultConfig() *Config {\n\tconfig := &Config{\n\t\tPluginDir: \"/plugins\",\n\t\tMetricsAddress: defaultMetricsAddress,\n\t\tDefaultBackupLocation: \"default\",\n\t\tDefaultVolumeSnapshotLocations: flag.NewMap().WithKeyValueDelimiter(':'),\n\t\tBackupSyncPeriod: defaultBackupSyncPeriod,\n\t\tDefaultBackupTTL: defaultBackupTTL,\n\t\tDefaultVGSLabelKey: velerov1api.DefaultVGSLabelKey,\n\t\tDefaultCSISnapshotTimeout: defaultCSISnapshotTimeout,\n\t\tDefaultItemOperationTimeout: defaultItemOperationTimeout,\n\t\tResourceTimeout: resourceTimeout,\n\t\tStoreValidationFrequency: defaultStoreValidationFrequency,\n\t\tPodVolumeOperationTimeout: defaultPodVolumeOperationTimeout,\n\t\tRestoreResourcePriorities: defaultRestorePriorities,\n\t\tClientQPS: defaultClientQPS,\n\t\tClientBurst: defaultClientBurst,\n\t\tClientPageSize: defaultClientPageSize,\n\t\tProfilerAddress: defaultProfilerAddress,\n\t\tResourceTerminatingTimeout: defaultResourceTerminatingTimeout,\n\t\tLogLevel: logging.LogLevelFlag(logrus.InfoLevel),\n\t\tLogFormat: logging.NewFormatFlag(),\n\t\tDefaultVolumesToFsBackup: podvolumeconfigs.DefaultVolumesToFsBackup,\n\t\tUploaderType: uploader.KopiaType,\n\t\tMaxConcurrentK8SConnections: defaultMaxConcurrentK8SConnections,\n\t\tDefaultSnapshotMoveData: false,\n\t\tDisableInformerCache: defaultDisableInformerCache,\n\t\tScheduleSkipImmediately: false,\n\t\tCredentialsDirectory: credentials.DefaultStoreDirectory(),\n\t\tItemBlockWorkerCount: DefaultItemBlockWorkerCount,\n\t\tConcurrentBackups: DefaultConcurrentBackups,\n\t}\n\n\treturn config\n}\n\nfunc (c *Config) BindFlags(flags *pflag.FlagSet) {\n\tflags.Var(c.LogLevel, \"log-level\", fmt.Sprintf(\"The level at which to log. Valid values are %s.\", strings.Join(c.LogLevel.AllowedValues(), \", \")))\n\tflags.Var(c.LogFormat, \"log-format\", fmt.Sprintf(\"The format for log output. Valid values are %s.\", strings.Join(c.LogFormat.AllowedValues(), \", \")))\n\tflags.StringVar(&c.PluginDir, \"plugin-dir\", c.PluginDir, \"Directory containing Velero plugins\")\n\tflags.StringVar(&c.MetricsAddress, \"metrics-address\", c.MetricsAddress, \"The address to expose prometheus metrics\")\n\tflags.DurationVar(&c.BackupSyncPeriod, \"backup-sync-period\", c.BackupSyncPeriod, \"How often to ensure all Velero backups in object storage exist as Backup API objects in the cluster. This is the default sync period if none is explicitly specified for a backup storage location.\")\n\tflags.DurationVar(&c.PodVolumeOperationTimeout, \"fs-backup-timeout\", c.PodVolumeOperationTimeout, \"How long pod volume file system backups/restores should be allowed to run before timing out.\")\n\tflags.BoolVar(&c.RestoreOnly, \"restore-only\", c.RestoreOnly, \"Run in a mode where only restores are allowed; backups, schedules, and garbage-collection are all disabled. DEPRECATED: this flag will be removed in v2.0. Use read-only backup storage locations instead.\")\n\tflags.StringSliceVar(&c.DisabledControllers, \"disable-controllers\", c.DisabledControllers, fmt.Sprintf(\"List of controllers to disable on startup. Valid values are %s\", strings.Join(DisableableControllers, \",\")))\n\tflags.Var(&c.RestoreResourcePriorities, \"restore-resource-priorities\", \"Desired order of resource restores, the priority list contains two parts which are split by \\\"-\\\" element. The resources before \\\"-\\\" element are restored first as high priorities, the resources after \\\"-\\\" element are restored last as low priorities, and any resource not in the list will be restored alphabetically between the high and low priorities.\")\n\tflags.StringVar(&c.DefaultBackupLocation, \"default-backup-storage-location\", c.DefaultBackupLocation, \"Name of the default backup storage location. DEPRECATED: this flag will be removed in v2.0. Use \\\"velero backup-location set --default\\\" instead.\")\n\tflags.DurationVar(&c.StoreValidationFrequency, \"store-validation-frequency\", c.StoreValidationFrequency, \"How often to verify if the storage is valid. Optional. Set this to `0s` to disable sync. Default 1 minute.\")\n\tflags.Float32Var(&c.ClientQPS, \"client-qps\", c.ClientQPS, \"Maximum number of requests per second by the server to the Kubernetes API once the burst limit has been reached.\")\n\tflags.IntVar(&c.ClientBurst, \"client-burst\", c.ClientBurst, \"Maximum number of requests by the server to the Kubernetes API in a short period of time.\")\n\tflags.IntVar(&c.ClientPageSize, \"client-page-size\", c.ClientPageSize, \"Page size of requests by the server to the Kubernetes API when listing objects during a backup. Set to 0 to disable paging.\")\n\tflags.StringVar(&c.ProfilerAddress, \"profiler-address\", c.ProfilerAddress, \"The address to expose the pprof profiler.\")\n\tflags.DurationVar(&c.ResourceTerminatingTimeout, \"terminating-resource-timeout\", c.ResourceTerminatingTimeout, \"How long to wait on persistent volumes and namespaces to terminate during a restore before timing out.\")\n\tflags.DurationVar(&c.DefaultBackupTTL, \"default-backup-ttl\", c.DefaultBackupTTL, \"How long to wait by default before backups can be garbage collected.\")\n\tflags.StringVar(&c.DefaultVGSLabelKey, \"volume-group-snapshot-label-key\", c.DefaultVGSLabelKey, \"Label key for grouping PVCs into VolumeGroupSnapshot. Default value is 'velero.io/volume-group'\")\n\tflags.DurationVar(&c.RepoMaintenanceFrequency, \"default-repo-maintain-frequency\", c.RepoMaintenanceFrequency, \"How often 'maintain' is run for backup repositories by default.\")\n\tflags.DurationVar(&c.GarbageCollectionFrequency, \"garbage-collection-frequency\", c.GarbageCollectionFrequency, \"How often garbage collection is run for expired backups.\")\n\tflags.DurationVar(&c.ItemOperationSyncFrequency, \"item-operation-sync-frequency\", c.ItemOperationSyncFrequency, \"How often to check status on backup/restore operations after backup/restore processing. Default is 10 seconds\")\n\tflags.BoolVar(&c.DefaultVolumesToFsBackup, \"default-volumes-to-fs-backup\", c.DefaultVolumesToFsBackup, \"Backup all volumes with pod volume file system backup by default.\")\n\tflags.StringVar(&c.UploaderType, \"uploader-type\", c.UploaderType, \"Type of uploader to handle the transfer of data of pod volumes\")\n\tflags.DurationVar(&c.DefaultItemOperationTimeout, \"default-item-operation-timeout\", c.DefaultItemOperationTimeout, \"How long to wait on asynchronous BackupItemActions and RestoreItemActions to complete before timing out. Default is 4 hours\")\n\tflags.DurationVar(&c.ResourceTimeout, \"resource-timeout\", c.ResourceTimeout, \"How long to wait for resource processes which are not covered by other specific timeout parameters. Default is 10 minutes.\")\n\tflags.IntVar(&c.MaxConcurrentK8SConnections, \"max-concurrent-k8s-connections\", c.MaxConcurrentK8SConnections, \"Max concurrent connections number that Velero can create with kube-apiserver. Default is 30.\")\n\tflags.BoolVar(&c.DefaultSnapshotMoveData, \"default-snapshot-move-data\", c.DefaultSnapshotMoveData, \"Move data by default for all snapshots supporting data movement.\")\n\tflags.BoolVar(&c.DisableInformerCache, \"disable-informer-cache\", c.DisableInformerCache, \"Disable informer cache for Get calls on restore. With this enabled, it will speed up restore in cases where there are backup resources which already exist in the cluster, but for very large clusters this will increase velero memory usage. Default is false (don't disable).\")\n\tflags.BoolVar(&c.ScheduleSkipImmediately, \"schedule-skip-immediately\", c.ScheduleSkipImmediately, \"Skip the first scheduled backup immediately after creating a schedule. Default is false (don't skip).\")\n\tflags.Var(&c.DefaultVolumeSnapshotLocations, \"default-volume-snapshot-locations\", \"List of unique volume providers and default volume snapshot location (provider1:location-01,provider2:location-02,...)\")\n\n\tflags.StringVar(\n\t\t&c.BackupRepoConfig,\n\t\t\"backup-repository-configmap\",\n\t\tc.BackupRepoConfig,\n\t\t\"The name of ConfigMap containing backup repository configurations.\",\n\t)\n\tflags.StringVar(\n\t\t&c.RepoMaintenanceJobConfig,\n\t\t\"repo-maintenance-job-configmap\",\n\t\tc.RepoMaintenanceJobConfig,\n\t\t\"The name of ConfigMap containing repository maintenance Job configurations.\",\n\t)\n\tflags.IntVar(\n\t\t&c.ItemBlockWorkerCount,\n\t\t\"item-block-worker-count\",\n\t\tc.ItemBlockWorkerCount,\n\t\t\"Number of worker threads to process ItemBlocks. Default is one. Optional.\",\n\t)\n\tflags.IntVar(\n\t\t&c.ConcurrentBackups,\n\t\t\"concurrent-backups\",\n\t\tc.ConcurrentBackups,\n\t\t\"Number of backups to process concurrently. Default is one. Optional.\",\n\t)\n}\n" }, { "path": "pkg/cmd/server/config/config_test.go", "content": "package config\n\nimport (\n\t\"testing\"\n\n\t\"github.com/spf13/pflag\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestGetDefaultConfig(t *testing.T) {\n\tconfig := GetDefaultConfig()\n\tassert.Equal(t, 1, config.ItemBlockWorkerCount)\n}\n\nfunc TestBindFlags(t *testing.T) {\n\tconfig := GetDefaultConfig()\n\tconfig.BindFlags(pflag.CommandLine)\n\tassert.Equal(t, 1, config.ItemBlockWorkerCount)\n}\n" }, { "path": "pkg/cmd/server/plugin/plugin.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage plugin\n\nimport (\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/spf13/cobra\"\n\tapiextensions \"k8s.io/apiextensions-apiserver/pkg/client/clientset/clientset\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/datamover\"\n\n\tdia \"github.com/vmware-tanzu/velero/internal/delete/actions/csi\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tbia \"github.com/vmware-tanzu/velero/pkg/backup/actions\"\n\tcsibia \"github.com/vmware-tanzu/velero/pkg/backup/actions/csi\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\tvelerodiscovery \"github.com/vmware-tanzu/velero/pkg/discovery\"\n\t\"github.com/vmware-tanzu/velero/pkg/features\"\n\tiba \"github.com/vmware-tanzu/velero/pkg/itemblock/actions\"\n\tveleroplugin \"github.com/vmware-tanzu/velero/pkg/plugin/framework\"\n\tplugincommon \"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tria \"github.com/vmware-tanzu/velero/pkg/restore/actions\"\n\tcsiria \"github.com/vmware-tanzu/velero/pkg/restore/actions/csi\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/actionhelpers\"\n)\n\nfunc NewCommand(f client.Factory) *cobra.Command {\n\tpluginServer := veleroplugin.NewServer()\n\tc := &cobra.Command{\n\t\tUse: \"run-plugins\",\n\t\tHidden: true,\n\t\tShort: \"INTERNAL COMMAND ONLY - not intended to be run directly by users\",\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\tconfig := pluginServer.GetConfig()\n\t\t\tf.SetClientQPS(config.ClientQPS)\n\t\t\tf.SetClientBurst(config.ClientBurst)\n\t\t\tpluginServer = pluginServer.\n\t\t\t\tRegisterBackupItemAction(\n\t\t\t\t\t\"velero.io/pv\",\n\t\t\t\t\tnewPVBackupItemAction,\n\t\t\t\t).\n\t\t\t\tRegisterBackupItemAction(\n\t\t\t\t\t\"velero.io/pod\",\n\t\t\t\t\tnewPodBackupItemAction,\n\t\t\t\t).\n\t\t\t\tRegisterBackupItemAction(\n\t\t\t\t\t\"velero.io/service-account\",\n\t\t\t\t\tnewServiceAccountBackupItemAction(f),\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemAction(\n\t\t\t\t\t\"velero.io/job\",\n\t\t\t\t\tnewJobRestoreItemAction,\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemAction(\n\t\t\t\t\t\"velero.io/pod\",\n\t\t\t\t\tnewPodRestoreItemAction,\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemAction(\n\t\t\t\t\t\"velero.io/pod-volume-restore\",\n\t\t\t\t\tnewPodVolumeRestoreItemAction(f),\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemAction(\n\t\t\t\t\t\"velero.io/init-restore-hook\",\n\t\t\t\t\tnewInitRestoreHookPodAction,\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemAction(\n\t\t\t\t\t\"velero.io/service\",\n\t\t\t\t\tnewServiceRestoreItemAction,\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemAction(\n\t\t\t\t\t\"velero.io/service-account\",\n\t\t\t\t\tnewServiceAccountRestoreItemAction,\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemAction(\n\t\t\t\t\t\"velero.io/add-pvc-from-pod\",\n\t\t\t\t\tnewAddPVCFromPodRestoreItemAction,\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemAction(\n\t\t\t\t\t\"velero.io/change-storage-class\",\n\t\t\t\t\tnewChangeStorageClassRestoreItemAction(f),\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemAction(\n\t\t\t\t\t\"velero.io/change-image-name\",\n\t\t\t\t\tnewChangeImageNameRestoreItemAction(f),\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemAction(\n\t\t\t\t\t\"velero.io/role-bindings\",\n\t\t\t\t\tnewRoleBindingItemAction,\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemAction(\n\t\t\t\t\t\"velero.io/cluster-role-bindings\",\n\t\t\t\t\tnewClusterRoleBindingItemAction,\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemAction(\n\t\t\t\t\t\"velero.io/crd-preserve-fields\",\n\t\t\t\t\tnewCRDV1PreserveUnknownFieldsItemAction,\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemAction(\n\t\t\t\t\t\"velero.io/pvc\",\n\t\t\t\t\tnewPVCRestoreItemAction(f),\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemAction(\n\t\t\t\t\t\"velero.io/apiservice\",\n\t\t\t\t\tnewAPIServiceRestoreItemAction,\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemAction(\n\t\t\t\t\t\"velero.io/admission-webhook-configuration\",\n\t\t\t\t\tnewAdmissionWebhookConfigurationAction,\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemAction(\n\t\t\t\t\t\"velero.io/secret\",\n\t\t\t\t\tnewSecretRestoreItemAction(f),\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemAction(\n\t\t\t\t\t\"velero.io/dataupload\",\n\t\t\t\t\tnewDataUploadRetrieveAction(f),\n\t\t\t\t).\n\t\t\t\tRegisterDeleteItemAction(\n\t\t\t\t\t\"velero.io/dataupload-delete\",\n\t\t\t\t\tnewDateUploadDeleteItemAction(f),\n\t\t\t\t).\n\t\t\t\tRegisterDeleteItemAction(\n\t\t\t\t\t\"velero.io/csi-volumesnapshotcontent-delete\",\n\t\t\t\t\tnewVolumeSnapshotContentDeleteItemAction(f),\n\t\t\t\t).\n\t\t\t\tRegisterBackupItemActionV2(\n\t\t\t\t\t\"velero.io/csi-pvc-backupper\",\n\t\t\t\t\tnewPvcBackupItemAction(f),\n\t\t\t\t).\n\t\t\t\tRegisterBackupItemActionV2(\n\t\t\t\t\t\"velero.io/csi-volumesnapshot-backupper\",\n\t\t\t\t\tnewVolumeSnapshotBackupItemAction(f),\n\t\t\t\t).\n\t\t\t\tRegisterBackupItemActionV2(\n\t\t\t\t\t\"velero.io/csi-volumesnapshotcontent-backupper\",\n\t\t\t\t\tnewVolumeSnapshotContentBackupItemAction,\n\t\t\t\t).\n\t\t\t\tRegisterBackupItemActionV2(\n\t\t\t\t\t\"velero.io/csi-volumesnapshotclass-backupper\",\n\t\t\t\t\tnewVolumeSnapshotClassBackupItemAction,\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemActionV2(\n\t\t\t\t\tconstant.PluginCSIPVCRestoreRIA,\n\t\t\t\t\tnewPvcRestoreItemAction(f),\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemActionV2(\n\t\t\t\t\tconstant.PluginCsiVolumeSnapshotRestoreRIA,\n\t\t\t\t\tnewVolumeSnapshotRestoreItemAction(f),\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemActionV2(\n\t\t\t\t\t\"velero.io/csi-volumesnapshotcontent-restorer\",\n\t\t\t\t\tnewVolumeSnapshotContentRestoreItemAction(f),\n\t\t\t\t).\n\t\t\t\tRegisterRestoreItemActionV2(\n\t\t\t\t\t\"velero.io/csi-volumesnapshotclass-restorer\",\n\t\t\t\t\tnewVolumeSnapshotClassRestoreItemAction,\n\t\t\t\t).\n\t\t\t\tRegisterItemBlockAction(\n\t\t\t\t\t\"velero.io/pvc\",\n\t\t\t\t\tnewPVCItemBlockAction(f),\n\t\t\t\t).\n\t\t\t\tRegisterItemBlockAction(\n\t\t\t\t\t\"velero.io/pod\",\n\t\t\t\t\tnewPodItemBlockAction,\n\t\t\t\t).\n\t\t\t\tRegisterItemBlockAction(\n\t\t\t\t\t\"velero.io/service-account\",\n\t\t\t\t\tnewServiceAccountItemBlockAction(f),\n\t\t\t\t)\n\n\t\t\tif !features.IsEnabled(velerov1api.APIGroupVersionsFeatureFlag) {\n\t\t\t\t// Do not register crd-remap-version BIA if the API Group feature\n\t\t\t\t// flag is enabled, so that the v1 CRD can be backed up.\n\t\t\t\tpluginServer = pluginServer.RegisterBackupItemAction(\n\t\t\t\t\t\"velero.io/crd-remap-version\",\n\t\t\t\t\tnewRemapCRDVersionAction(f),\n\t\t\t\t)\n\t\t\t}\n\t\t\tpluginServer.Serve()\n\t\t},\n\t\tFParseErrWhitelist: cobra.FParseErrWhitelist{ // Velero.io word list : ignore\n\t\t\tUnknownFlags: true,\n\t\t},\n\t}\n\tpluginServer.BindFlags(c.Flags())\n\treturn c\n}\n\nfunc newPVBackupItemAction(logger logrus.FieldLogger) (any, error) {\n\treturn bia.NewPVCAction(logger), nil\n}\n\nfunc newPodBackupItemAction(logger logrus.FieldLogger) (any, error) {\n\treturn bia.NewPodAction(logger), nil\n}\n\nfunc newServiceAccountBackupItemAction(f client.Factory) plugincommon.HandlerInitializer {\n\treturn func(logger logrus.FieldLogger) (any, error) {\n\t\t// TODO(ncdc): consider a k8s style WantsKubernetesClientSet initialization approach\n\t\tclientset, err := f.KubeClient()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tdiscoveryClient, err := f.DiscoveryClient()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tdiscoveryHelper, err := velerodiscovery.NewHelper(discoveryClient, logger)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\taction, err := bia.NewServiceAccountAction(\n\t\t\tlogger,\n\t\t\tactionhelpers.NewClusterRoleBindingListerMap(clientset),\n\t\t\tdiscoveryHelper)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\treturn action, nil\n\t}\n}\n\nfunc newRemapCRDVersionAction(f client.Factory) plugincommon.HandlerInitializer {\n\treturn func(logger logrus.FieldLogger) (any, error) {\n\t\tconfig, err := f.ClientConfig()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tclient, err := apiextensions.NewForConfig(config)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tdiscoveryClient, err := f.DiscoveryClient()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tdiscoveryHelper, err := velerodiscovery.NewHelper(discoveryClient, logger)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\treturn bia.NewRemapCRDVersionAction(logger, client.ApiextensionsV1beta1().CustomResourceDefinitions(), discoveryHelper), nil\n\t}\n}\n\nfunc newJobRestoreItemAction(logger logrus.FieldLogger) (any, error) {\n\treturn ria.NewJobAction(logger), nil\n}\n\nfunc newPodRestoreItemAction(logger logrus.FieldLogger) (any, error) {\n\treturn ria.NewPodAction(logger), nil\n}\n\nfunc newInitRestoreHookPodAction(logger logrus.FieldLogger) (any, error) {\n\treturn ria.NewInitRestoreHookPodAction(logger), nil\n}\n\nfunc newPodVolumeRestoreItemAction(f client.Factory) plugincommon.HandlerInitializer {\n\treturn func(logger logrus.FieldLogger) (any, error) {\n\t\tclient, err := f.KubeClient()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tcrClient, err := f.KubebuilderClient()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\treturn ria.NewPodVolumeRestoreAction(logger, client.CoreV1().ConfigMaps(f.Namespace()), crClient, f.Namespace())\n\t}\n}\n\nfunc newServiceRestoreItemAction(logger logrus.FieldLogger) (any, error) {\n\treturn ria.NewServiceAction(logger), nil\n}\n\nfunc newServiceAccountRestoreItemAction(logger logrus.FieldLogger) (any, error) {\n\treturn ria.NewServiceAccountAction(logger), nil\n}\n\nfunc newAddPVCFromPodRestoreItemAction(logger logrus.FieldLogger) (any, error) {\n\treturn ria.NewAddPVCFromPodAction(logger), nil\n}\n\nfunc newCRDV1PreserveUnknownFieldsItemAction(logger logrus.FieldLogger) (any, error) {\n\treturn ria.NewCRDV1PreserveUnknownFieldsAction(logger), nil\n}\n\nfunc newChangeStorageClassRestoreItemAction(f client.Factory) plugincommon.HandlerInitializer {\n\treturn func(logger logrus.FieldLogger) (any, error) {\n\t\tclient, err := f.KubeClient()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\treturn ria.NewChangeStorageClassAction(\n\t\t\tlogger,\n\t\t\tclient.CoreV1().ConfigMaps(f.Namespace()),\n\t\t\tclient.StorageV1().StorageClasses(),\n\t\t), nil\n\t}\n}\n\nfunc newChangeImageNameRestoreItemAction(f client.Factory) plugincommon.HandlerInitializer {\n\treturn func(logger logrus.FieldLogger) (any, error) {\n\t\tclient, err := f.KubeClient()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\treturn ria.NewChangeImageNameAction(\n\t\t\tlogger,\n\t\t\tclient.CoreV1().ConfigMaps(f.Namespace()),\n\t\t), nil\n\t}\n}\nfunc newRoleBindingItemAction(logger logrus.FieldLogger) (any, error) {\n\treturn ria.NewRoleBindingAction(logger), nil\n}\n\nfunc newClusterRoleBindingItemAction(logger logrus.FieldLogger) (any, error) {\n\treturn ria.NewClusterRoleBindingAction(logger), nil\n}\n\nfunc newPVCRestoreItemAction(f client.Factory) plugincommon.HandlerInitializer {\n\treturn func(logger logrus.FieldLogger) (any, error) {\n\t\tclient, err := f.KubeClient()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\treturn ria.NewPVCAction(\n\t\t\tlogger,\n\t\t\tclient.CoreV1().ConfigMaps(f.Namespace()),\n\t\t\tclient.CoreV1().Nodes(),\n\t\t), nil\n\t}\n}\n\nfunc newAPIServiceRestoreItemAction(logger logrus.FieldLogger) (any, error) {\n\treturn ria.NewAPIServiceAction(logger), nil\n}\n\nfunc newAdmissionWebhookConfigurationAction(logger logrus.FieldLogger) (any, error) {\n\treturn ria.NewAdmissionWebhookConfigurationAction(logger), nil\n}\n\nfunc newSecretRestoreItemAction(f client.Factory) plugincommon.HandlerInitializer {\n\treturn func(logger logrus.FieldLogger) (any, error) {\n\t\tclient, err := f.KubebuilderClient()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\treturn ria.NewSecretAction(logger, client), nil\n\t}\n}\n\nfunc newDataUploadRetrieveAction(f client.Factory) plugincommon.HandlerInitializer {\n\treturn func(logger logrus.FieldLogger) (any, error) {\n\t\tclient, err := f.KubebuilderClient()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\treturn ria.NewDataUploadRetrieveAction(logger, client), nil\n\t}\n}\n\nfunc newDateUploadDeleteItemAction(f client.Factory) plugincommon.HandlerInitializer {\n\treturn func(logger logrus.FieldLogger) (any, error) {\n\t\tclient, err := f.KubebuilderClient()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\treturn datamover.NewDataUploadDeleteAction(logger, client), nil\n\t}\n}\n\n// CSI plugin init functions.\n\n// BackupItemAction plugins\n\nfunc newPvcBackupItemAction(f client.Factory) plugincommon.HandlerInitializer {\n\treturn csibia.NewPvcBackupItemAction(f)\n}\n\nfunc newVolumeSnapshotBackupItemAction(f client.Factory) plugincommon.HandlerInitializer {\n\treturn csibia.NewVolumeSnapshotBackupItemAction(f)\n}\n\nfunc newVolumeSnapshotContentBackupItemAction(logger logrus.FieldLogger) (any, error) {\n\treturn csibia.NewVolumeSnapshotContentBackupItemAction(logger)\n}\n\nfunc newVolumeSnapshotClassBackupItemAction(logger logrus.FieldLogger) (any, error) {\n\treturn csibia.NewVolumeSnapshotClassBackupItemAction(logger)\n}\n\n// DeleteItemAction plugins\n\nfunc newVolumeSnapshotContentDeleteItemAction(f client.Factory) plugincommon.HandlerInitializer {\n\treturn dia.NewVolumeSnapshotContentDeleteItemAction(f)\n}\n\n// RestoreItemAction plugins\n\nfunc newPvcRestoreItemAction(f client.Factory) plugincommon.HandlerInitializer {\n\treturn csiria.NewPvcRestoreItemAction(f)\n}\n\nfunc newVolumeSnapshotRestoreItemAction(f client.Factory) plugincommon.HandlerInitializer {\n\treturn csiria.NewVolumeSnapshotRestoreItemAction(f)\n}\n\nfunc newVolumeSnapshotContentRestoreItemAction(f client.Factory) plugincommon.HandlerInitializer {\n\treturn csiria.NewVolumeSnapshotContentRestoreItemAction(f)\n}\n\nfunc newVolumeSnapshotClassRestoreItemAction(logger logrus.FieldLogger) (any, error) {\n\treturn csiria.NewVolumeSnapshotClassRestoreItemAction(logger)\n}\n\n// ItemBlockAction plugins\n\nfunc newPVCItemBlockAction(f client.Factory) plugincommon.HandlerInitializer {\n\treturn iba.NewPVCAction(f)\n}\n\nfunc newPodItemBlockAction(logger logrus.FieldLogger) (any, error) {\n\treturn iba.NewPodAction(logger), nil\n}\n\nfunc newServiceAccountItemBlockAction(f client.Factory) plugincommon.HandlerInitializer {\n\treturn func(logger logrus.FieldLogger) (any, error) {\n\t\t// TODO(ncdc): consider a k8s style WantsKubernetesClientSet initialization approach\n\t\tclientset, err := f.KubeClient()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tdiscoveryClient, err := f.DiscoveryClient()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tdiscoveryHelper, err := velerodiscovery.NewHelper(discoveryClient, logger)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\taction, err := iba.NewServiceAccountAction(\n\t\t\tlogger,\n\t\t\tactionhelpers.NewClusterRoleBindingListerMap(clientset),\n\t\t\tdiscoveryHelper)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\treturn action, nil\n\t}\n}\n" }, { "path": "pkg/cmd/server/server.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage server\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\t\"net/http\"\n\t\"net/http/pprof\"\n\t\"os\"\n\t\"strings\"\n\t\"time\"\n\n\tlogrusr \"github.com/bombsimon/logrusr/v3\"\n\tvolumegroupsnapshotv1beta1 \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumegroupsnapshot/v1beta1\"\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/prometheus/client_golang/prometheus/promhttp\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/spf13/cobra\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tbatchv1api \"k8s.io/api/batch/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\tkubeerrs \"k8s.io/apimachinery/pkg/util/errors\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"k8s.io/client-go/discovery\"\n\t\"k8s.io/client-go/dynamic\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"k8s.io/client-go/rest\"\n\t\"k8s.io/klog/v2\"\n\t\"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/cache\"\n\tctrlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/manager\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\t\"github.com/vmware-tanzu/velero/internal/hook\"\n\t\"github.com/vmware-tanzu/velero/internal/storage\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/backup\"\n\t\"github.com/vmware-tanzu/velero/pkg/buildinfo\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/server/config\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/signals\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/controller\"\n\tvelerodiscovery \"github.com/vmware-tanzu/velero/pkg/discovery\"\n\t\"github.com/vmware-tanzu/velero/pkg/features\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperationmap\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\t\"github.com/vmware-tanzu/velero/pkg/nodeagent\"\n\t\"github.com/vmware-tanzu/velero/pkg/persistence\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n\t\"github.com/vmware-tanzu/velero/pkg/podexec\"\n\t\"github.com/vmware-tanzu/velero/pkg/podvolume\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\trepokey \"github.com/vmware-tanzu/velero/pkg/repository/keys\"\n\trepomanager \"github.com/vmware-tanzu/velero/pkg/repository/manager\"\n\t\"github.com/vmware-tanzu/velero/pkg/restore\"\n\tvelerotypes \"github.com/vmware-tanzu/velero/pkg/types\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n)\n\nfunc NewCommand(f client.Factory) *cobra.Command {\n\tconfig := config.GetDefaultConfig()\n\n\tvar command = &cobra.Command{\n\t\tUse: \"server\",\n\t\tShort: \"Run the velero server\",\n\t\tLong: \"Run the velero server\",\n\t\tHidden: true,\n\t\tRun: func(c *cobra.Command, args []string) {\n\t\t\t// go-plugin uses log.Println to log when it's waiting for all plugin processes to complete so we need to\n\t\t\t// set its output to stdout.\n\t\t\tlog.SetOutput(os.Stdout)\n\n\t\t\tlogLevel := config.LogLevel.Parse()\n\t\t\tformat := config.LogFormat.Parse()\n\n\t\t\t// Make sure we log to stdout so cloud log dashboards don't show this as an error.\n\t\t\tlogrus.SetOutput(os.Stdout)\n\n\t\t\t// Velero's DefaultLogger logs to stdout, so all is good there.\n\t\t\tlogger := logging.DefaultLogger(logLevel, format)\n\n\t\t\tlogger.Infof(\"setting log-level to %s\", strings.ToUpper(logLevel.String()))\n\n\t\t\tlogger.Infof(\"Starting Velero server %s (%s)\", buildinfo.Version, buildinfo.FormattedGitSHA())\n\t\t\tif len(features.All()) > 0 {\n\t\t\t\tlogger.Infof(\"%d feature flags enabled %s\", len(features.All()), features.All())\n\t\t\t} else {\n\t\t\t\tlogger.Info(\"No feature flags enabled\")\n\t\t\t}\n\n\t\t\tf.SetBasename(fmt.Sprintf(\"%s-%s\", c.Parent().Name(), c.Name()))\n\n\t\t\ts, err := newServer(f, config, logger)\n\t\t\tcmd.CheckError(err)\n\n\t\t\tcmd.CheckError(s.run())\n\t\t},\n\t}\n\n\tconfig.BindFlags(command.Flags())\n\n\treturn command\n}\n\ntype server struct {\n\tnamespace string\n\tmetricsAddress string\n\tkubeClientConfig *rest.Config\n\tkubeClient kubernetes.Interface\n\tdiscoveryClient discovery.AggregatedDiscoveryInterface\n\tdiscoveryHelper velerodiscovery.Helper\n\tdynamicClient dynamic.Interface\n\t// controller-runtime client. the difference from the controller-manager's client\n\t// is that the controller-manager's client is limited to list namespaced-scoped\n\t// resources in the namespace where Velero is installed, or the cluster-scoped\n\t// resources. The crClient doesn't have the limitation.\n\tcrClient ctrlclient.Client\n\tctx context.Context\n\tcancelFunc context.CancelFunc\n\tlogger logrus.FieldLogger\n\tlogLevel logrus.Level\n\tpluginRegistry process.Registry\n\trepoManager repomanager.Manager\n\trepoLocker *repository.RepoLocker\n\trepoEnsurer *repository.Ensurer\n\tmetrics *metrics.ServerMetrics\n\tconfig *config.Config\n\tmgr manager.Manager\n\tcredentialFileStore credentials.FileStore\n\tcredentialSecretStore credentials.SecretStore\n}\n\nfunc newServer(f client.Factory, config *config.Config, logger *logrus.Logger) (*server, error) {\n\tif msg, err := uploader.ValidateUploaderType(config.UploaderType); err != nil {\n\t\treturn nil, err\n\t} else if msg != \"\" {\n\t\tlogger.Warn(msg)\n\t}\n\n\tif config.ClientQPS < 0.0 {\n\t\treturn nil, errors.New(\"client-qps must be positive\")\n\t}\n\tf.SetClientQPS(config.ClientQPS)\n\n\tif config.ClientBurst <= 0 {\n\t\treturn nil, errors.New(\"client-burst must be positive\")\n\t}\n\tf.SetClientBurst(config.ClientBurst)\n\n\tif config.ClientPageSize < 0 {\n\t\treturn nil, errors.New(\"client-page-size must not be negative\")\n\t}\n\n\tkubeClient, err := f.KubeClient()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdynamicClient, err := f.DynamicClient()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcrClient, err := f.KubebuilderClient()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tpluginRegistry := process.NewRegistry(config.PluginDir, logger, logger.Level)\n\tif err := pluginRegistry.DiscoverPlugins(); err != nil {\n\t\treturn nil, err\n\t}\n\n\t// cancelFunc is not deferred here because if it was, then ctx would immediately\n\t// be canceled once this function exited, making it useless to any informers using later.\n\t// That, in turn, causes the velero server to halt when the first informer tries to use it.\n\t// Therefore, we must explicitly call it on the error paths in this function.\n\tctx, cancelFunc := context.WithCancel(context.Background())\n\n\tif len(config.BackupRepoConfig) > 0 {\n\t\trepoConfig := make(map[string]any)\n\t\tif err := kube.VerifyJSONConfigs(ctx, f.Namespace(), crClient, config.BackupRepoConfig, &repoConfig); err != nil {\n\t\t\tcancelFunc()\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tif len(config.RepoMaintenanceJobConfig) > 0 {\n\t\tif err := kube.VerifyJSONConfigs(ctx, f.Namespace(), crClient, config.RepoMaintenanceJobConfig, &velerotypes.JobConfigs{}); err != nil {\n\t\t\tcancelFunc()\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tclientConfig, err := f.ClientConfig()\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, err\n\t}\n\n\tscheme := runtime.NewScheme()\n\tif err := velerov1api.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, err\n\t}\n\tif err := velerov2alpha1api.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, err\n\t}\n\tif err := corev1api.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, err\n\t}\n\tif err := snapshotv1api.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, err\n\t}\n\tif err := volumegroupsnapshotv1beta1.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, err\n\t}\n\tif err := batchv1api.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, err\n\t}\n\tif err := appsv1api.AddToScheme(scheme); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, err\n\t}\n\n\tctrl.SetLogger(logrusr.New(logger))\n\tklog.SetLogger(logrusr.New(logger)) // klog.Logger is used by k8s.io/client-go\n\n\tvar mgr manager.Manager\n\tretry := 10\n\tfor {\n\t\tmgr, err = ctrl.NewManager(clientConfig, ctrl.Options{\n\t\t\tScheme: scheme,\n\t\t\tCache: cache.Options{\n\t\t\t\tDefaultNamespaces: map[string]cache.Config{\n\t\t\t\t\tf.Namespace(): {},\n\t\t\t\t},\n\t\t\t},\n\t\t})\n\t\tif err == nil {\n\t\t\tbreak\n\t\t}\n\n\t\tretry--\n\t\tif retry == 0 {\n\t\t\tbreak\n\t\t}\n\n\t\tlogger.WithError(err).Warn(\"Failed to create controller manager, need retry\")\n\n\t\ttime.Sleep(time.Second)\n\t}\n\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, errors.Wrap(err, \"error creating controller manager\")\n\t}\n\n\tcredentialFileStore, err := credentials.NewNamespacedFileStore(\n\t\tmgr.GetClient(),\n\t\tf.Namespace(),\n\t\tconfig.CredentialsDirectory,\n\t\tfilesystem.NewFileSystem(),\n\t)\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, err\n\t}\n\n\tcredentialSecretStore, err := credentials.NewNamespacedSecretStore(mgr.GetClient(), f.Namespace())\n\tif err != nil {\n\t\tcancelFunc()\n\t\treturn nil, err\n\t}\n\n\tvar discoveryClient discovery.AggregatedDiscoveryInterface\n\tif discoveryClient, err = f.DiscoveryClient(); err != nil {\n\t\tcancelFunc()\n\t\treturn nil, err\n\t}\n\n\ts := &server{\n\t\tnamespace: f.Namespace(),\n\t\tmetricsAddress: config.MetricsAddress,\n\t\tkubeClientConfig: clientConfig,\n\t\tkubeClient: kubeClient,\n\t\tdiscoveryClient: discoveryClient,\n\t\tdynamicClient: dynamicClient,\n\t\tcrClient: crClient,\n\t\tctx: ctx,\n\t\tcancelFunc: cancelFunc,\n\t\tlogger: logger,\n\t\tlogLevel: logger.Level,\n\t\tpluginRegistry: pluginRegistry,\n\t\tconfig: config,\n\t\tmgr: mgr,\n\t\tcredentialFileStore: credentialFileStore,\n\t\tcredentialSecretStore: credentialSecretStore,\n\t}\n\n\treturn s, nil\n}\n\nfunc (s *server) run() error {\n\tsignals.CancelOnShutdown(s.cancelFunc, s.logger)\n\n\tif s.config.ProfilerAddress != \"\" {\n\t\tgo s.runProfiler()\n\t}\n\n\t// Since s.namespace, which specifies where backups/restores/schedules/etc. should live,\n\t// *could* be different from the namespace where the Velero server pod runs, check to make\n\t// sure it exists, and fail fast if it doesn't.\n\tif err := s.namespaceExists(s.namespace); err != nil {\n\t\treturn err\n\t}\n\n\tif err := s.initDiscoveryHelper(); err != nil {\n\t\treturn err\n\t}\n\n\tif err := s.veleroResourcesExist(); err != nil {\n\t\treturn err\n\t}\n\n\ts.checkNodeAgent()\n\n\tif err := s.initRepoManager(); err != nil {\n\t\treturn err\n\t}\n\n\tif err := s.setupBeforeControllerRun(); err != nil {\n\t\treturn err\n\t}\n\n\tif err := s.runControllers(s.config.DefaultVolumeSnapshotLocations.Data()); err != nil {\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\n// setupBeforeControllerRun do any setup that needs to happen before the controllers are started.\nfunc (s *server) setupBeforeControllerRun() error {\n\tclient, err := ctrlclient.New(s.mgr.GetConfig(), ctrlclient.Options{Scheme: s.mgr.GetScheme()})\n\t// the function is called before starting the controller manager, the embedded client isn't ready to use, so create a new one here\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\tmarkInProgressCRsFailed(s.ctx, client, s.namespace, s.logger)\n\n\tif err := setDefaultBackupLocation(s.ctx, client, s.namespace, s.config.DefaultBackupLocation, s.logger); err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n\n// setDefaultBackupLocation set the BSL that matches the \"velero server --default-backup-storage-location\"\nfunc setDefaultBackupLocation(ctx context.Context, client ctrlclient.Client, namespace, defaultBackupLocation string, logger logrus.FieldLogger) error {\n\tif defaultBackupLocation == \"\" {\n\t\tlogger.Debug(\"No default backup storage location specified. Velero will not automatically select a backup storage location for new backups.\")\n\t\treturn nil\n\t}\n\n\tbackupLocation := &velerov1api.BackupStorageLocation{}\n\tif err := client.Get(ctx, types.NamespacedName{Namespace: namespace, Name: defaultBackupLocation}, backupLocation); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlogger.WithField(\"backupStorageLocation\", defaultBackupLocation).WithError(err).Warn(\"Failed to set default backup storage location at server start\")\n\t\t\treturn nil\n\t\t} else {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\t}\n\n\tif !backupLocation.Spec.Default {\n\t\tbackupLocation.Spec.Default = true\n\t\tif err := client.Update(ctx, backupLocation); err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\n\t\tlogger.WithField(\"backupStorageLocation\", defaultBackupLocation).Info(\"Set backup storage location as default\")\n\t}\n\n\treturn nil\n}\n\n// namespaceExists returns nil if namespace can be successfully\n// gotten from the Kubernetes API, or an error otherwise.\nfunc (s *server) namespaceExists(namespace string) error {\n\ts.logger.WithField(\"namespace\", namespace).Info(\"Checking existence of namespace.\")\n\n\tif _, err := s.kubeClient.CoreV1().Namespaces().Get(s.ctx, namespace, metav1.GetOptions{}); err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\ts.logger.WithField(\"namespace\", namespace).Info(\"Namespace exists\")\n\treturn nil\n}\n\n// initDiscoveryHelper instantiates the server's discovery helper and spawns a\n// goroutine to call Refresh() every 5 minutes.\nfunc (s *server) initDiscoveryHelper() error {\n\tdiscoveryHelper, err := velerodiscovery.NewHelper(s.discoveryClient, s.logger)\n\tif err != nil {\n\t\treturn err\n\t}\n\ts.discoveryHelper = discoveryHelper\n\n\tgo wait.Until(\n\t\tfunc() {\n\t\t\tif err := discoveryHelper.Refresh(); err != nil {\n\t\t\t\ts.logger.WithError(err).Error(\"Error refreshing discovery\")\n\t\t\t}\n\t\t},\n\t\t5*time.Minute,\n\t\ts.ctx.Done(),\n\t)\n\n\treturn nil\n}\n\n// veleroResourcesExist checks for the existence of each Velero CRD via discovery\n// and returns an error if any of them don't exist.\nfunc (s *server) veleroResourcesExist() error {\n\ts.logger.Info(\"Checking existence of Velero custom resource definitions\")\n\n\t// add more group versions whenever available\n\tgvResources := map[string]sets.Set[string]{\n\t\tvelerov1api.SchemeGroupVersion.String(): velerov1api.CustomResourceKinds(),\n\t\tvelerov2alpha1api.SchemeGroupVersion.String(): velerov2alpha1api.CustomResourceKinds(),\n\t}\n\n\tfor _, lst := range s.discoveryHelper.Resources() {\n\t\tif resources, found := gvResources[lst.GroupVersion]; found {\n\t\t\tfor _, resource := range lst.APIResources {\n\t\t\t\ts.logger.WithField(\"kind\", resource.Kind).Info(\"Found custom resource\")\n\t\t\t\tresources.Delete(resource.Kind)\n\t\t\t}\n\t\t}\n\t}\n\n\tvar errs []error\n\tfor gv, resources := range gvResources {\n\t\tfor kind := range resources {\n\t\t\terrs = append(errs, errors.Errorf(\"custom resource %s not found in Velero API group %s\", kind, gv))\n\t\t}\n\t}\n\n\tif len(errs) > 0 {\n\t\terrs = append(errs, errors.New(\"Velero custom resources not found - apply config/crd/v1/bases/*.yaml,config/crd/v2alpha1/bases*.yaml, to update the custom resource definitions\"))\n\t\treturn kubeerrs.NewAggregate(errs)\n\t}\n\n\ts.logger.Info(\"All Velero custom resource definitions exist\")\n\treturn nil\n}\n\nfunc (s *server) checkNodeAgent() {\n\t// warn if node agent does not exist\n\tif kube.WithLinuxNode(s.ctx, s.crClient, s.logger) {\n\t\tif err := nodeagent.IsRunningOnLinux(s.ctx, s.kubeClient, s.namespace); err == nodeagent.ErrDaemonSetNotFound {\n\t\t\ts.logger.Warn(\"Velero node agent not found for linux nodes; pod volume backups/restores and data mover backups/restores will not work until it's created\")\n\t\t} else if err != nil {\n\t\t\ts.logger.WithError(errors.WithStack(err)).Warn(\"Error checking for existence of velero node agent for linux nodes\")\n\t\t}\n\t}\n\n\tif kube.WithWindowsNode(s.ctx, s.crClient, s.logger) {\n\t\tif err := nodeagent.IsRunningOnWindows(s.ctx, s.kubeClient, s.namespace); err == nodeagent.ErrDaemonSetNotFound {\n\t\t\ts.logger.Warn(\"Velero node agent not found for Windows nodes; pod volume backups/restores and data mover backups/restores will not work until it's created\")\n\t\t} else if err != nil {\n\t\t\ts.logger.WithError(errors.WithStack(err)).Warn(\"Error checking for existence of velero node agent for Windows nodes\")\n\t\t}\n\t}\n}\n\nfunc (s *server) initRepoManager() error {\n\t// ensure the repo key secret is set up\n\tif err := repokey.EnsureCommonRepositoryKey(s.kubeClient.CoreV1(), s.namespace); err != nil {\n\t\treturn err\n\t}\n\n\ts.repoLocker = repository.NewRepoLocker()\n\ts.repoEnsurer = repository.NewEnsurer(s.mgr.GetClient(), s.logger, s.config.ResourceTimeout)\n\n\ts.repoManager = repomanager.NewManager(\n\t\ts.namespace,\n\t\ts.mgr.GetClient(),\n\t\ts.repoLocker,\n\t\ts.credentialFileStore,\n\t\ts.credentialSecretStore,\n\t\ts.logger,\n\t)\n\n\treturn nil\n}\n\nfunc (s *server) runControllers(defaultVolumeSnapshotLocations map[string]string) error {\n\ts.logger.Info(\"Starting controllers\")\n\n\tgo func() {\n\t\tmetricsMux := http.NewServeMux()\n\t\tmetricsMux.Handle(\"/metrics\", promhttp.Handler())\n\t\ts.logger.Infof(\"Starting metric server at address [%s]\", s.metricsAddress)\n\t\tserver := &http.Server{\n\t\t\tAddr: s.metricsAddress,\n\t\t\tHandler: metricsMux,\n\t\t\tReadHeaderTimeout: 3 * time.Second,\n\t\t}\n\t\tif err := server.ListenAndServe(); err != nil {\n\t\t\ts.logger.Fatalf(\"Failed to start metric server at [%s]: %v\", s.metricsAddress, err)\n\t\t}\n\t}()\n\ts.metrics = metrics.NewServerMetrics()\n\ts.metrics.RegisterAllMetrics()\n\t// Initialize manual backup metrics\n\ts.metrics.InitSchedule(\"\")\n\n\tnewPluginManager := func(logger logrus.FieldLogger) clientmgmt.Manager {\n\t\treturn clientmgmt.NewManager(logger, s.logLevel, s.pluginRegistry)\n\t}\n\n\tbackupStoreGetter := persistence.NewObjectBackupStoreGetterWithSecretStore(s.credentialFileStore, s.credentialSecretStore)\n\n\tbackupTracker := controller.NewBackupTracker()\n\n\t// By far, PodVolumeBackup, PodVolumeRestore, BackupStorageLocation controllers\n\t// are not included in --disable-controllers list.\n\t// This is because of PVB and PVR are used by node agent DaemonSet,\n\t// and BSL controller is mandatory for Velero to work.\n\t// Note: all runtime type controllers that can be disabled are grouped separately, below:\n\tenabledRuntimeControllers := map[string]struct{}{\n\t\tconstant.ControllerBackup: {},\n\t\tconstant.ControllerBackupDeletion: {},\n\t\tconstant.ControllerBackupFinalizer: {},\n\t\tconstant.ControllerBackupOperations: {},\n\t\tconstant.ControllerBackupRepo: {},\n\t\tconstant.ControllerBackupSync: {},\n\t\tconstant.ControllerDownloadRequest: {},\n\t\tconstant.ControllerGarbageCollection: {},\n\t\tconstant.ControllerRestore: {},\n\t\tconstant.ControllerRestoreOperations: {},\n\t\tconstant.ControllerSchedule: {},\n\t\tconstant.ControllerServerStatusRequest: {},\n\t\tconstant.ControllerRestoreFinalizer: {},\n\t\tconstant.ControllerBackupQueue: {},\n\t}\n\n\tif s.config.RestoreOnly {\n\t\ts.logger.Info(\"Restore only mode - not starting the backup, schedule, delete-backup, or GC controllers\")\n\t\ts.config.DisabledControllers = append(s.config.DisabledControllers,\n\t\t\tconstant.ControllerBackup,\n\t\t\tconstant.ControllerBackupDeletion,\n\t\t\tconstant.ControllerBackupFinalizer,\n\t\t\tconstant.ControllerBackupOperations,\n\t\t\tconstant.ControllerGarbageCollection,\n\t\t\tconstant.ControllerSchedule,\n\t\t)\n\t}\n\n\t// Remove disabled controllers so they are not initialized. If a match is not found we want\n\t// to halt the system so the user knows this operation was not possible.\n\tif err := removeControllers(s.config.DisabledControllers, enabledRuntimeControllers, s.logger); err != nil {\n\t\tlog.Fatal(err, \"unable to disable a controller\")\n\t}\n\n\t// Enable BSL controller. No need to check whether it's enabled or not.\n\tbslr := controller.NewBackupStorageLocationReconciler(\n\t\ts.ctx,\n\t\ts.mgr.GetClient(),\n\t\tstorage.DefaultBackupLocationInfo{\n\t\t\tStorageLocation: s.config.DefaultBackupLocation,\n\t\t\tServerValidationFrequency: s.config.StoreValidationFrequency,\n\t\t},\n\t\tnewPluginManager,\n\t\tbackupStoreGetter,\n\t\ts.metrics,\n\t\ts.logger,\n\t)\n\tif err := bslr.SetupWithManager(s.mgr); err != nil {\n\t\ts.logger.Fatal(err, \"unable to create controller\", \"controller\", constant.ControllerBackupStorageLocation)\n\t}\n\n\tpvbInformer, err := s.mgr.GetCache().GetInformer(s.ctx, &velerov1api.PodVolumeBackup{})\n\tif err != nil {\n\t\ts.logger.Fatal(err, \"fail to get controller-runtime informer from manager for PVB\")\n\t}\n\n\tif _, ok := enabledRuntimeControllers[constant.ControllerBackup]; ok {\n\t\tbackupper, err := backup.NewKubernetesBackupper(\n\t\t\ts.crClient,\n\t\t\ts.discoveryHelper,\n\t\t\tclient.NewDynamicFactory(s.dynamicClient),\n\t\t\tpodexec.NewPodCommandExecutor(s.kubeClientConfig, s.kubeClient.CoreV1().RESTClient()),\n\t\t\tpodvolume.NewBackupperFactory(\n\t\t\t\ts.repoLocker,\n\t\t\t\ts.repoEnsurer,\n\t\t\t\ts.crClient,\n\t\t\t\tpvbInformer,\n\t\t\t\ts.logger,\n\t\t\t),\n\t\t\ts.config.PodVolumeOperationTimeout,\n\t\t\ts.config.DefaultVolumesToFsBackup,\n\t\t\ts.config.ClientPageSize,\n\t\t\ts.config.UploaderType,\n\t\t\tnewPluginManager,\n\t\t\tbackupStoreGetter,\n\t\t)\n\t\tcmd.CheckError(err)\n\t\tif err := controller.NewBackupReconciler(\n\t\t\ts.ctx,\n\t\t\ts.discoveryHelper,\n\t\t\tbackupper,\n\t\t\ts.logger,\n\t\t\ts.logLevel,\n\t\t\tnewPluginManager,\n\t\t\tbackupTracker,\n\t\t\ts.mgr.GetClient(),\n\t\t\ts.config.DefaultBackupLocation,\n\t\t\ts.config.DefaultVolumesToFsBackup,\n\t\t\ts.config.DefaultBackupTTL,\n\t\t\ts.config.DefaultVGSLabelKey,\n\t\t\ts.config.DefaultCSISnapshotTimeout,\n\t\t\ts.config.ResourceTimeout,\n\t\t\ts.config.DefaultItemOperationTimeout,\n\t\t\tdefaultVolumeSnapshotLocations,\n\t\t\ts.metrics,\n\t\t\tbackupStoreGetter,\n\t\t\ts.config.LogFormat.Parse(),\n\t\t\ts.credentialFileStore,\n\t\t\ts.config.MaxConcurrentK8SConnections,\n\t\t\ts.config.DefaultSnapshotMoveData,\n\t\t\ts.config.ItemBlockWorkerCount,\n\t\t\ts.config.ConcurrentBackups,\n\t\t\ts.crClient,\n\t\t).SetupWithManager(s.mgr); err != nil {\n\t\t\ts.logger.Fatal(err, \"unable to create controller\", \"controller\", constant.ControllerBackup)\n\t\t}\n\t}\n\n\tif _, ok := enabledRuntimeControllers[constant.ControllerBackupDeletion]; ok {\n\t\tif err := controller.NewBackupDeletionReconciler(\n\t\t\ts.logger,\n\t\t\ts.mgr.GetClient(),\n\t\t\tbackupTracker,\n\t\t\ts.repoManager,\n\t\t\ts.metrics,\n\t\t\ts.discoveryHelper,\n\t\t\tnewPluginManager,\n\t\t\tbackupStoreGetter,\n\t\t\ts.credentialFileStore,\n\t\t\ts.repoEnsurer,\n\t\t).SetupWithManager(s.mgr); err != nil {\n\t\t\ts.logger.Fatal(err, \"unable to create controller\", \"controller\", constant.ControllerBackupDeletion)\n\t\t}\n\t}\n\n\tbackupOpsMap := itemoperationmap.NewBackupItemOperationsMap()\n\tif _, ok := enabledRuntimeControllers[constant.ControllerBackupOperations]; ok {\n\t\tr := controller.NewBackupOperationsReconciler(\n\t\t\ts.logger,\n\t\t\ts.mgr.GetClient(),\n\t\t\ts.config.ItemOperationSyncFrequency,\n\t\t\tnewPluginManager,\n\t\t\tbackupStoreGetter,\n\t\t\ts.metrics,\n\t\t\tbackupOpsMap,\n\t\t)\n\t\tif err := r.SetupWithManager(s.mgr); err != nil {\n\t\t\ts.logger.Fatal(err, \"unable to create controller\", \"controller\", constant.ControllerBackupOperations)\n\t\t}\n\t}\n\n\tif _, ok := enabledRuntimeControllers[constant.ControllerBackupFinalizer]; ok {\n\t\tbackupper, err := backup.NewKubernetesBackupper(\n\t\t\ts.mgr.GetClient(),\n\t\t\ts.discoveryHelper,\n\t\t\tclient.NewDynamicFactory(s.dynamicClient),\n\t\t\tpodexec.NewPodCommandExecutor(s.kubeClientConfig, s.kubeClient.CoreV1().RESTClient()),\n\t\t\tpodvolume.NewBackupperFactory(\n\t\t\t\ts.repoLocker,\n\t\t\t\ts.repoEnsurer,\n\t\t\t\ts.crClient,\n\t\t\t\tpvbInformer,\n\t\t\t\ts.logger,\n\t\t\t),\n\t\t\ts.config.PodVolumeOperationTimeout,\n\t\t\ts.config.DefaultVolumesToFsBackup,\n\t\t\ts.config.ClientPageSize,\n\t\t\ts.config.UploaderType,\n\t\t\tnewPluginManager,\n\t\t\tbackupStoreGetter,\n\t\t)\n\t\tcmd.CheckError(err)\n\t\tr := controller.NewBackupFinalizerReconciler(\n\t\t\ts.mgr.GetClient(),\n\t\t\ts.crClient,\n\t\t\tclock.RealClock{},\n\t\t\tbackupper,\n\t\t\tnewPluginManager,\n\t\t\tbackupTracker,\n\t\t\tbackupStoreGetter,\n\t\t\ts.logger,\n\t\t\ts.metrics,\n\t\t\ts.config.ResourceTimeout,\n\t\t)\n\t\tif err := r.SetupWithManager(s.mgr); err != nil {\n\t\t\ts.logger.Fatal(err, \"unable to create controller\", \"controller\", constant.ControllerBackupFinalizer)\n\t\t}\n\t}\n\n\tif _, ok := enabledRuntimeControllers[constant.ControllerBackupRepo]; ok {\n\t\tif err := controller.NewBackupRepoReconciler(\n\t\t\ts.namespace,\n\t\t\ts.logger,\n\t\t\ts.mgr.GetClient(),\n\t\t\ts.repoManager,\n\t\t\ts.config.RepoMaintenanceFrequency,\n\t\t\ts.config.BackupRepoConfig,\n\t\t\ts.config.RepoMaintenanceJobConfig,\n\t\t\ts.logLevel,\n\t\t\ts.config.LogFormat,\n\t\t\ts.metrics,\n\t\t).SetupWithManager(s.mgr); err != nil {\n\t\t\ts.logger.Fatal(err, \"unable to create controller\", \"controller\", constant.ControllerBackupRepo)\n\t\t}\n\t}\n\n\tif _, ok := enabledRuntimeControllers[constant.ControllerBackupSync]; ok {\n\t\tsyncPeriod := s.config.BackupSyncPeriod\n\t\tif syncPeriod <= 0 {\n\t\t\tsyncPeriod = time.Minute\n\t\t}\n\n\t\tbackupSyncReconciler := controller.NewBackupSyncReconciler(\n\t\t\ts.mgr.GetClient(),\n\t\t\ts.namespace,\n\t\t\tsyncPeriod,\n\t\t\tnewPluginManager,\n\t\t\tbackupStoreGetter,\n\t\t\ts.logger,\n\t\t)\n\t\tif err := backupSyncReconciler.SetupWithManager(s.mgr); err != nil {\n\t\t\ts.logger.Fatal(err, \" unable to create controller \", \"controller \", constant.ControllerBackupSync)\n\t\t}\n\t}\n\n\trestoreOpsMap := itemoperationmap.NewRestoreItemOperationsMap()\n\tif _, ok := enabledRuntimeControllers[constant.ControllerRestoreOperations]; ok {\n\t\tr := controller.NewRestoreOperationsReconciler(\n\t\t\ts.logger,\n\t\t\ts.namespace,\n\t\t\ts.mgr.GetClient(),\n\t\t\ts.config.ItemOperationSyncFrequency,\n\t\t\tnewPluginManager,\n\t\t\tbackupStoreGetter,\n\t\t\ts.metrics,\n\t\t\trestoreOpsMap,\n\t\t)\n\t\tif err := r.SetupWithManager(s.mgr); err != nil {\n\t\t\ts.logger.Fatal(err, \"unable to create controller\", \"controller\", constant.ControllerRestoreOperations)\n\t\t}\n\t}\n\n\tif _, ok := enabledRuntimeControllers[constant.ControllerDownloadRequest]; ok {\n\t\tr := controller.NewDownloadRequestReconciler(\n\t\t\ts.mgr.GetClient(),\n\t\t\tclock.RealClock{},\n\t\t\tnewPluginManager,\n\t\t\tbackupStoreGetter,\n\t\t\ts.logger,\n\t\t\tbackupOpsMap,\n\t\t\trestoreOpsMap,\n\t\t)\n\t\tif err := r.SetupWithManager(s.mgr); err != nil {\n\t\t\ts.logger.Fatal(err, \"unable to create controller\", \"controller\", constant.ControllerDownloadRequest)\n\t\t}\n\t}\n\n\tif _, ok := enabledRuntimeControllers[constant.ControllerGarbageCollection]; ok {\n\t\tr := controller.NewGCReconciler(s.logger, s.mgr.GetClient(), s.config.GarbageCollectionFrequency)\n\t\tif err := r.SetupWithManager(s.mgr); err != nil {\n\t\t\ts.logger.Fatal(err, \"unable to create controller\", \"controller\", constant.ControllerGarbageCollection)\n\t\t}\n\t}\n\n\tpvrInformer, err := s.mgr.GetCache().GetInformer(s.ctx, &velerov1api.PodVolumeRestore{})\n\tif err != nil {\n\t\ts.logger.Fatal(err, \"fail to get controller-runtime informer from manager for PVR\")\n\t}\n\n\tmultiHookTracker := hook.NewMultiHookTracker()\n\n\tif _, ok := enabledRuntimeControllers[constant.ControllerRestore]; ok {\n\t\trestorer, err := restore.NewKubernetesRestorer(\n\t\t\ts.discoveryHelper,\n\t\t\tclient.NewDynamicFactory(s.dynamicClient),\n\t\t\ts.config.RestoreResourcePriorities,\n\t\t\ts.kubeClient.CoreV1().Namespaces(),\n\t\t\tpodvolume.NewRestorerFactory(\n\t\t\t\ts.repoLocker,\n\t\t\t\ts.repoEnsurer,\n\t\t\t\ts.kubeClient,\n\t\t\t\ts.crClient,\n\t\t\t\tpvrInformer,\n\t\t\t\ts.logger,\n\t\t\t),\n\t\t\ts.config.PodVolumeOperationTimeout,\n\t\t\ts.config.ResourceTerminatingTimeout,\n\t\t\ts.config.ResourceTimeout,\n\t\t\ts.logger,\n\t\t\tpodexec.NewPodCommandExecutor(s.kubeClientConfig, s.kubeClient.CoreV1().RESTClient()),\n\t\t\ts.kubeClient.CoreV1().RESTClient(),\n\t\t\ts.credentialFileStore,\n\t\t\ts.mgr.GetClient(),\n\t\t\tmultiHookTracker,\n\t\t)\n\n\t\tcmd.CheckError(err)\n\n\t\tr := controller.NewRestoreReconciler(\n\t\t\ts.ctx,\n\t\t\ts.namespace,\n\t\t\trestorer,\n\t\t\ts.mgr.GetClient(),\n\t\t\ts.logger,\n\t\t\ts.logLevel,\n\t\t\tnewPluginManager,\n\t\t\tbackupStoreGetter,\n\t\t\ts.metrics,\n\t\t\ts.config.LogFormat.Parse(),\n\t\t\ts.config.DefaultItemOperationTimeout,\n\t\t\ts.config.DisableInformerCache,\n\t\t\ts.crClient,\n\t\t\ts.config.ResourceTimeout,\n\t\t)\n\n\t\tif err = r.SetupWithManager(s.mgr); err != nil {\n\t\t\ts.logger.Fatal(err, \"fail to create controller\", \"controller\", constant.ControllerRestore)\n\t\t}\n\t}\n\n\tif _, ok := enabledRuntimeControllers[constant.ControllerSchedule]; ok {\n\t\tif err := controller.NewScheduleReconciler(s.namespace, s.logger, s.mgr.GetClient(), s.metrics, s.config.ScheduleSkipImmediately).SetupWithManager(s.mgr); err != nil {\n\t\t\ts.logger.Fatal(err, \"unable to create controller\", \"controller\", constant.ControllerSchedule)\n\t\t}\n\t}\n\n\tif _, ok := enabledRuntimeControllers[constant.ControllerServerStatusRequest]; ok {\n\t\tif err := controller.NewServerStatusRequestReconciler(\n\t\t\ts.ctx,\n\t\t\ts.mgr.GetClient(),\n\t\t\ts.pluginRegistry,\n\t\t\tclock.RealClock{},\n\t\t\ts.logger,\n\t\t).SetupWithManager(s.mgr); err != nil {\n\t\t\ts.logger.Fatal(err, \"unable to create controller\", \"controller\", constant.ControllerServerStatusRequest)\n\t\t}\n\t}\n\n\tif _, ok := enabledRuntimeControllers[constant.ControllerRestoreFinalizer]; ok {\n\t\tif err := controller.NewRestoreFinalizerReconciler(\n\t\t\ts.logger,\n\t\t\ts.namespace,\n\t\t\ts.mgr.GetClient(),\n\t\t\tnewPluginManager,\n\t\t\tbackupStoreGetter,\n\t\t\ts.metrics,\n\t\t\ts.crClient,\n\t\t\tmultiHookTracker,\n\t\t\ts.config.ResourceTimeout,\n\t\t).SetupWithManager(s.mgr); err != nil {\n\t\t\ts.logger.Fatal(err, \"unable to create controller\", \"controller\", constant.ControllerRestoreFinalizer)\n\t\t}\n\t}\n\n\tif _, ok := enabledRuntimeControllers[constant.ControllerBackupQueue]; ok {\n\t\tif err := controller.NewBackupQueueReconciler(\n\t\t\ts.mgr.GetClient(),\n\t\t\ts.mgr.GetScheme(),\n\t\t\ts.logger,\n\t\t\ts.config.ConcurrentBackups,\n\t\t\tbackupTracker,\n\t\t).SetupWithManager(s.mgr); err != nil {\n\t\t\ts.logger.Fatal(err, \"unable to create controller\", \"controller\", constant.ControllerBackupQueue)\n\t\t}\n\t}\n\n\ts.logger.Info(\"Server starting...\")\n\n\tif err := s.mgr.Start(s.ctx); err != nil {\n\t\ts.logger.Fatal(\"Problem starting manager\", err)\n\t}\n\treturn nil\n}\n\n// removeControllers will remove any controller listed to be disabled from the list\n// of controllers to be initialized. It will check the runtime controllers. If a match\n// wasn't found and it returns an error.\nfunc removeControllers(disabledControllers []string, enabledRuntimeControllers map[string]struct{}, logger logrus.FieldLogger) error {\n\tfor _, controllerName := range disabledControllers {\n\t\tif _, ok := enabledRuntimeControllers[controllerName]; ok {\n\t\t\tlogger.Infof(\"Disabling controller: %s\", controllerName)\n\t\t\tdelete(enabledRuntimeControllers, controllerName)\n\t\t} else {\n\t\t\tmsg := fmt.Sprintf(\"Invalid value for --disable-controllers flag provided: %s. Valid values are: %s\", controllerName, strings.Join(config.DisableableControllers, \",\"))\n\t\t\treturn errors.New(msg)\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc (s *server) runProfiler() {\n\tmux := http.NewServeMux()\n\tmux.HandleFunc(\"/debug/pprof/\", pprof.Index)\n\tmux.HandleFunc(\"/debug/pprof/cmdline\", pprof.Cmdline)\n\tmux.HandleFunc(\"/debug/pprof/profile\", pprof.Profile)\n\tmux.HandleFunc(\"/debug/pprof/symbol\", pprof.Symbol)\n\tmux.HandleFunc(\"/debug/pprof/trace\", pprof.Trace)\n\n\tserver := &http.Server{\n\t\tAddr: s.config.ProfilerAddress,\n\t\tHandler: mux,\n\t\tReadHeaderTimeout: 3 * time.Second,\n\t}\n\tif err := server.ListenAndServe(); err != nil {\n\t\ts.logger.WithError(errors.WithStack(err)).Error(\"error running profiler http server\")\n\t}\n}\n\n// if there is a restarting during the reconciling of backups/restores/etc, these CRs may be stuck in progress status\n// markInProgressCRsFailed tries to mark the in progress CRs as failed when starting the server to avoid the issue\nfunc markInProgressCRsFailed(ctx context.Context, client ctrlclient.Client, namespace string, log logrus.FieldLogger) {\n\tmarkInProgressBackupsFailed(ctx, client, namespace, log)\n\n\tmarkInProgressRestoresFailed(ctx, client, namespace, log)\n}\n\nfunc markInProgressBackupsFailed(ctx context.Context, client ctrlclient.Client, namespace string, log logrus.FieldLogger) {\n\tbackups := &velerov1api.BackupList{}\n\tif err := client.List(ctx, backups, &ctrlclient.ListOptions{Namespace: namespace}); err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Error(\"failed to list backups\")\n\t\treturn\n\t}\n\n\tfor i, backup := range backups.Items {\n\t\tif backup.Status.Phase != velerov1api.BackupPhaseInProgress {\n\t\t\tlog.Debugf(\"the status of backup %q is %q, skip\", backup.GetName(), backup.Status.Phase)\n\t\t\tcontinue\n\t\t}\n\t\tupdated := backup.DeepCopy()\n\t\tupdated.Status.Phase = velerov1api.BackupPhaseFailed\n\t\tupdated.Status.FailureReason = fmt.Sprintf(\"found a backup with status %q during the server starting, mark it as %q\", backup.Status.Phase, updated.Status.Phase)\n\t\tupdated.Status.CompletionTimestamp = &metav1.Time{Time: time.Now()}\n\t\tif err := client.Patch(ctx, updated, ctrlclient.MergeFrom(&backups.Items[i])); err != nil {\n\t\t\tlog.WithError(errors.WithStack(err)).Errorf(\"failed to patch backup %q\", backup.GetName())\n\t\t\tcontinue\n\t\t}\n\t\tlog.WithField(\"backup\", backup.GetName()).Warn(updated.Status.FailureReason)\n\t\tmarkDataUploadsCancel(ctx, client, backup, log)\n\t\tmarkPodVolumeBackupsCancel(ctx, client, backup, log)\n\t}\n}\n\nfunc markInProgressRestoresFailed(ctx context.Context, client ctrlclient.Client, namespace string, log logrus.FieldLogger) {\n\trestores := &velerov1api.RestoreList{}\n\tif err := client.List(ctx, restores, &ctrlclient.ListOptions{Namespace: namespace}); err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Error(\"failed to list restores\")\n\t\treturn\n\t}\n\tfor i, restore := range restores.Items {\n\t\tif restore.Status.Phase != velerov1api.RestorePhaseInProgress {\n\t\t\tlog.Debugf(\"the status of restore %q is %q, skip\", restore.GetName(), restore.Status.Phase)\n\t\t\tcontinue\n\t\t}\n\t\tupdated := restore.DeepCopy()\n\t\tupdated.Status.Phase = velerov1api.RestorePhaseFailed\n\t\tupdated.Status.FailureReason = fmt.Sprintf(\"found a restore with status %q during the server starting, mark it as %q\", restore.Status.Phase, updated.Status.Phase)\n\t\tupdated.Status.CompletionTimestamp = &metav1.Time{Time: time.Now()}\n\t\tif err := client.Patch(ctx, updated, ctrlclient.MergeFrom(&restores.Items[i])); err != nil {\n\t\t\tlog.WithError(errors.WithStack(err)).Errorf(\"failed to patch restore %q\", restore.GetName())\n\t\t\tcontinue\n\t\t}\n\n\t\tlog.WithField(\"restore\", restore.GetName()).Warn(updated.Status.FailureReason)\n\t\tmarkDataDownloadsCancel(ctx, client, restore, log)\n\t\tmarkPodVolumeRestoresCancel(ctx, client, restore, log)\n\t}\n}\n\nfunc markDataUploadsCancel(ctx context.Context, client ctrlclient.Client, backup velerov1api.Backup, log logrus.FieldLogger) {\n\tdataUploads := &velerov2alpha1api.DataUploadList{}\n\n\tif err := client.List(ctx, dataUploads, &ctrlclient.ListOptions{\n\t\tNamespace: backup.GetNamespace(),\n\t\tLabelSelector: labels.Set(map[string]string{\n\t\t\tvelerov1api.BackupUIDLabel: string(backup.GetUID()),\n\t\t}).AsSelector(),\n\t}); err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Error(\"failed to list dataUploads\")\n\t\treturn\n\t}\n\n\tfor i := range dataUploads.Items {\n\t\tdu := dataUploads.Items[i]\n\t\tif du.Status.Phase == velerov2alpha1api.DataUploadPhaseAccepted ||\n\t\t\tdu.Status.Phase == velerov2alpha1api.DataUploadPhasePrepared ||\n\t\t\tdu.Status.Phase == velerov2alpha1api.DataUploadPhaseInProgress ||\n\t\t\tdu.Status.Phase == velerov2alpha1api.DataUploadPhaseNew ||\n\t\t\tdu.Status.Phase == \"\" {\n\t\t\terr := controller.UpdateDataUploadWithRetry(ctx, client, types.NamespacedName{Namespace: du.Namespace, Name: du.Name}, log.WithField(\"dataupload\", du.Name),\n\t\t\t\tfunc(dataUpload *velerov2alpha1api.DataUpload) bool {\n\t\t\t\t\tif dataUpload.Spec.Cancel {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\n\t\t\t\t\tdataUpload.Spec.Cancel = true\n\t\t\t\t\tdataUpload.Status.Message = fmt.Sprintf(\"Dataupload is in status %q during the velero server starting, mark it as cancel\", du.Status.Phase)\n\n\t\t\t\t\treturn true\n\t\t\t\t})\n\n\t\t\tif err != nil {\n\t\t\t\tlog.WithError(errors.WithStack(err)).Errorf(\"failed to mark dataupload %q cancel\", du.GetName())\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tlog.WithField(\"dataupload\", du.GetName()).Warn(du.Status.Message)\n\t\t}\n\t}\n}\n\nfunc markDataDownloadsCancel(ctx context.Context, client ctrlclient.Client, restore velerov1api.Restore, log logrus.FieldLogger) {\n\tdataDownloads := &velerov2alpha1api.DataDownloadList{}\n\n\tif err := client.List(ctx, dataDownloads, &ctrlclient.ListOptions{\n\t\tNamespace: restore.GetNamespace(),\n\t\tLabelSelector: labels.Set(map[string]string{\n\t\t\tvelerov1api.RestoreUIDLabel: string(restore.GetUID()),\n\t\t}).AsSelector(),\n\t}); err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Error(\"failed to list dataDownloads\")\n\t\treturn\n\t}\n\n\tfor i := range dataDownloads.Items {\n\t\tdd := dataDownloads.Items[i]\n\t\tif dd.Status.Phase == velerov2alpha1api.DataDownloadPhaseAccepted ||\n\t\t\tdd.Status.Phase == velerov2alpha1api.DataDownloadPhasePrepared ||\n\t\t\tdd.Status.Phase == velerov2alpha1api.DataDownloadPhaseInProgress ||\n\t\t\tdd.Status.Phase == velerov2alpha1api.DataDownloadPhaseNew ||\n\t\t\tdd.Status.Phase == \"\" {\n\t\t\terr := controller.UpdateDataDownloadWithRetry(ctx, client, types.NamespacedName{Namespace: dd.Namespace, Name: dd.Name}, log.WithField(\"datadownload\", dd.Name),\n\t\t\t\tfunc(dataDownload *velerov2alpha1api.DataDownload) bool {\n\t\t\t\t\tif dataDownload.Spec.Cancel {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\n\t\t\t\t\tdataDownload.Spec.Cancel = true\n\t\t\t\t\tdataDownload.Status.Message = fmt.Sprintf(\"Datadownload is in status %q during the velero server starting, mark it as cancel\", dd.Status.Phase)\n\n\t\t\t\t\treturn true\n\t\t\t\t})\n\n\t\t\tif err != nil {\n\t\t\t\tlog.WithError(errors.WithStack(err)).Errorf(\"failed to mark datadownload %q cancel\", dd.GetName())\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tlog.WithField(\"datadownload\", dd.GetName()).Warn(dd.Status.Message)\n\t\t}\n\t}\n}\n\nfunc markPodVolumeBackupsCancel(ctx context.Context, client ctrlclient.Client, backup velerov1api.Backup, log logrus.FieldLogger) {\n\tpvbs := &velerov1api.PodVolumeBackupList{}\n\n\tif err := client.List(ctx, pvbs, &ctrlclient.ListOptions{\n\t\tNamespace: backup.GetNamespace(),\n\t\tLabelSelector: labels.Set(map[string]string{\n\t\t\tvelerov1api.BackupUIDLabel: string(backup.GetUID()),\n\t\t}).AsSelector(),\n\t}); err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Error(\"failed to list PVBs\")\n\t\treturn\n\t}\n\n\tfor i := range pvbs.Items {\n\t\tpvb := pvbs.Items[i]\n\t\tif pvb.Status.Phase == velerov1api.PodVolumeBackupPhaseAccepted ||\n\t\t\tpvb.Status.Phase == velerov1api.PodVolumeBackupPhasePrepared ||\n\t\t\tpvb.Status.Phase == velerov1api.PodVolumeBackupPhaseInProgress ||\n\t\t\tpvb.Status.Phase == velerov1api.PodVolumeBackupPhaseNew ||\n\t\t\tpvb.Status.Phase == \"\" {\n\t\t\terr := controller.UpdatePVBWithRetry(ctx, client, types.NamespacedName{Namespace: pvb.Namespace, Name: pvb.Name}, log.WithField(\"PVB\", pvb.Name),\n\t\t\t\tfunc(pvb *velerov1api.PodVolumeBackup) bool {\n\t\t\t\t\tif pvb.Spec.Cancel {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\n\t\t\t\t\tpvb.Spec.Cancel = true\n\t\t\t\t\tpvb.Status.Message = fmt.Sprintf(\"PVB is in status %q during the velero server starting, mark it as cancel\", pvb.Status.Phase)\n\n\t\t\t\t\treturn true\n\t\t\t\t})\n\n\t\t\tif err != nil {\n\t\t\t\tlog.WithError(errors.WithStack(err)).Errorf(\"failed to mark PVB %q cancel\", pvb.GetName())\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tlog.WithField(\"PVB is mark for cancel due to server restart\", pvb.GetName()).Warn(pvb.Status.Message)\n\t\t}\n\t}\n}\n\nfunc markPodVolumeRestoresCancel(ctx context.Context, client ctrlclient.Client, restore velerov1api.Restore, log logrus.FieldLogger) {\n\tpvrs := &velerov1api.PodVolumeRestoreList{}\n\n\tif err := client.List(ctx, pvrs, &ctrlclient.ListOptions{\n\t\tNamespace: restore.GetNamespace(),\n\t\tLabelSelector: labels.Set(map[string]string{\n\t\t\tvelerov1api.RestoreUIDLabel: string(restore.GetUID()),\n\t\t}).AsSelector(),\n\t}); err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Error(\"failed to list PVRs\")\n\t\treturn\n\t}\n\n\tfor i := range pvrs.Items {\n\t\tpvr := pvrs.Items[i]\n\t\tif controller.IsLegacyPVR(&pvr) {\n\t\t\tlog.WithField(\"PVR\", pvr.GetName()).Warn(\"Found a legacy PVR during velero server restart, cannot stop it\")\n\t\t\tcontinue\n\t\t}\n\n\t\tif pvr.Status.Phase == velerov1api.PodVolumeRestorePhaseAccepted ||\n\t\t\tpvr.Status.Phase == velerov1api.PodVolumeRestorePhasePrepared ||\n\t\t\tpvr.Status.Phase == velerov1api.PodVolumeRestorePhaseInProgress ||\n\t\t\tpvr.Status.Phase == velerov1api.PodVolumeRestorePhaseNew ||\n\t\t\tpvr.Status.Phase == \"\" {\n\t\t\terr := controller.UpdatePVRWithRetry(ctx, client, types.NamespacedName{Namespace: pvr.Namespace, Name: pvr.Name}, log.WithField(\"PVR\", pvr.Name),\n\t\t\t\tfunc(pvr *velerov1api.PodVolumeRestore) bool {\n\t\t\t\t\tif pvr.Spec.Cancel {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\n\t\t\t\t\tpvr.Spec.Cancel = true\n\t\t\t\t\tpvr.Status.Message = fmt.Sprintf(\"PVR is in status %q during the velero server starting, mark it as cancel\", pvr.Status.Phase)\n\n\t\t\t\t\treturn true\n\t\t\t\t})\n\n\t\t\tif err != nil {\n\t\t\t\tlog.WithError(errors.WithStack(err)).Errorf(\"failed to mark PVR %q cancel\", pvr.GetName())\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tlog.WithField(\"PVR is mark for cancel due to server restart\", pvr.GetName()).Warn(pvr.Status.Message)\n\t\t}\n\t}\n}\n" }, { "path": "pkg/cmd/server/server_test.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage server\n\nimport (\n\t\"errors\"\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\tkubefake \"k8s.io/client-go/kubernetes/fake\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/server/config\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\tdiscovery_mocks \"github.com/vmware-tanzu/velero/pkg/discovery/mocks\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n)\n\nfunc TestVeleroResourcesExist(t *testing.T) {\n\tvar (\n\t\tfakeDiscoveryHelper = &velerotest.FakeDiscoveryHelper{}\n\t\tserver = &server{\n\t\t\tlogger: velerotest.NewLogger(),\n\t\t\tdiscoveryHelper: fakeDiscoveryHelper,\n\t\t}\n\t)\n\n\t// Velero API group doesn't exist in discovery: should error\n\tfakeDiscoveryHelper.ResourceList = []*metav1.APIResourceList{\n\t\t{\n\t\t\tGroupVersion: \"foo/v1\",\n\t\t\tAPIResources: []metav1.APIResource{\n\t\t\t\t{\n\t\t\t\t\tName: \"Backups\",\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\trequire.Error(t, server.veleroResourcesExist())\n\n\t// Velero v1 API group doesn't contain any custom resources: should error\n\tveleroAPIResourceListVelerov1 := &metav1.APIResourceList{\n\t\tGroupVersion: velerov1api.SchemeGroupVersion.String(),\n\t}\n\n\tfakeDiscoveryHelper.ResourceList = append(fakeDiscoveryHelper.ResourceList, veleroAPIResourceListVelerov1)\n\trequire.Error(t, server.veleroResourcesExist())\n\n\t// Velero v2alpha1 API group doesn't contain any custom resources: should error\n\tveleroAPIResourceListVeleroV2alpha1 := &metav1.APIResourceList{\n\t\tGroupVersion: velerov2alpha1api.SchemeGroupVersion.String(),\n\t}\n\n\tfakeDiscoveryHelper.ResourceList = append(fakeDiscoveryHelper.ResourceList, veleroAPIResourceListVeleroV2alpha1)\n\trequire.Error(t, server.veleroResourcesExist())\n\n\t// Velero v1 API group contains all custom resources, but v2alpha1 doesn't contain any custom resources: should error\n\tfor kind := range velerov1api.CustomResources() {\n\t\tveleroAPIResourceListVelerov1.APIResources = append(veleroAPIResourceListVelerov1.APIResources, metav1.APIResource{\n\t\t\tKind: kind,\n\t\t})\n\t}\n\trequire.Error(t, server.veleroResourcesExist())\n\n\t// Velero v1 and v2alpha1 API group contain all custom resources: should not error\n\tfor kind := range velerov2alpha1api.CustomResources() {\n\t\tveleroAPIResourceListVeleroV2alpha1.APIResources = append(veleroAPIResourceListVeleroV2alpha1.APIResources, metav1.APIResource{\n\t\t\tKind: kind,\n\t\t})\n\t}\n\trequire.NoError(t, server.veleroResourcesExist())\n\n\t// Velero API group contains some but not all custom resources: should error\n\tveleroAPIResourceListVelerov1.APIResources = veleroAPIResourceListVelerov1.APIResources[:3]\n\tassert.Error(t, server.veleroResourcesExist())\n}\n\nfunc TestRemoveControllers(t *testing.T) {\n\tlogger := velerotest.NewLogger()\n\n\ttests := []struct {\n\t\tname string\n\t\tdisabledControllers []string\n\t\terrorExpected bool\n\t}{\n\t\t{\n\t\t\tname: \"Remove one disable controller\",\n\t\t\tdisabledControllers: []string{\n\t\t\t\tconstant.ControllerBackup,\n\t\t\t},\n\t\t\terrorExpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Remove all disable controllers\",\n\t\t\tdisabledControllers: []string{\n\t\t\t\tconstant.ControllerBackupOperations,\n\t\t\t\tconstant.ControllerBackup,\n\t\t\t\tconstant.ControllerBackupDeletion,\n\t\t\t\tconstant.ControllerBackupSync,\n\t\t\t\tconstant.ControllerDownloadRequest,\n\t\t\t\tconstant.ControllerGarbageCollection,\n\t\t\t\tconstant.ControllerBackupRepo,\n\t\t\t\tconstant.ControllerRestore,\n\t\t\t\tconstant.ControllerSchedule,\n\t\t\t\tconstant.ControllerServerStatusRequest,\n\t\t\t},\n\t\t\terrorExpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Remove with a non-disable controller included\",\n\t\t\tdisabledControllers: []string{\n\t\t\t\tconstant.ControllerBackup,\n\t\t\t\tconstant.ControllerBackupStorageLocation,\n\t\t\t},\n\t\t\terrorExpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Remove with a misspelled/non-existing controller name\",\n\t\t\tdisabledControllers: []string{\n\t\t\t\t\"go\",\n\t\t\t},\n\t\t\terrorExpected: true,\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tenabledRuntimeControllers := map[string]struct{}{\n\t\t\t\tconstant.ControllerBackupSync: {},\n\t\t\t\tconstant.ControllerBackup: {},\n\t\t\t\tconstant.ControllerGarbageCollection: {},\n\t\t\t\tconstant.ControllerRestore: {},\n\t\t\t\tconstant.ControllerServerStatusRequest: {},\n\t\t\t\tconstant.ControllerSchedule: {},\n\t\t\t\tconstant.ControllerBackupDeletion: {},\n\t\t\t\tconstant.ControllerBackupRepo: {},\n\t\t\t\tconstant.ControllerDownloadRequest: {},\n\t\t\t\tconstant.ControllerBackupOperations: {},\n\t\t\t}\n\n\t\t\ttotalNumOriginalControllers := len(enabledRuntimeControllers)\n\n\t\t\tif tt.errorExpected {\n\t\t\t\tassert.Error(t, removeControllers(tt.disabledControllers, enabledRuntimeControllers, logger))\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, removeControllers(tt.disabledControllers, enabledRuntimeControllers, logger))\n\n\t\t\t\ttotalNumEnabledControllers := len(enabledRuntimeControllers)\n\t\t\t\tassert.Equal(t, totalNumEnabledControllers, totalNumOriginalControllers-len(tt.disabledControllers))\n\n\t\t\t\tfor _, disabled := range tt.disabledControllers {\n\t\t\t\t\t_, ok := enabledRuntimeControllers[disabled]\n\t\t\t\t\tassert.False(t, ok)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestNewCommand(t *testing.T) {\n\tassert.NotNil(t, NewCommand(nil))\n}\n\nfunc Test_newServer(t *testing.T) {\n\tfactory := &mocks.Factory{}\n\tlogger := logrus.New()\n\n\t// invalid uploader type\n\t_, err := newServer(factory, &config.Config{\n\t\tUploaderType: \"invalid\",\n\t}, logger)\n\trequire.Error(t, err)\n\n\t// invalid clientQPS\n\t_, err = newServer(factory, &config.Config{\n\t\tUploaderType: uploader.KopiaType,\n\t\tClientQPS: -1,\n\t}, logger)\n\trequire.Error(t, err)\n\n\t// invalid clientQPS Restic uploader\n\t_, err = newServer(factory, &config.Config{\n\t\tUploaderType: uploader.ResticType,\n\t\tClientQPS: -1,\n\t}, logger)\n\trequire.Error(t, err)\n\n\t// invalid clientBurst\n\tfactory.On(\"SetClientQPS\", mock.Anything).Return()\n\t_, err = newServer(factory, &config.Config{\n\t\tUploaderType: uploader.KopiaType,\n\t\tClientQPS: 1,\n\t\tClientBurst: -1,\n\t}, logger)\n\trequire.Error(t, err)\n\n\t// invalid clientBclientPageSizeurst\n\tfactory.On(\"SetClientQPS\", mock.Anything).Return().\n\t\tOn(\"SetClientBurst\", mock.Anything).Return()\n\t_, err = newServer(factory, &config.Config{\n\t\tUploaderType: uploader.KopiaType,\n\t\tClientQPS: 1,\n\t\tClientBurst: 1,\n\t\tClientPageSize: -1,\n\t}, logger)\n\trequire.Error(t, err)\n\n\t// got error when creating client\n\tfactory.On(\"SetClientQPS\", mock.Anything).Return().\n\t\tOn(\"SetClientBurst\", mock.Anything).Return().\n\t\tOn(\"KubeClient\").Return(nil, nil).\n\t\tOn(\"Client\").Return(nil, nil).\n\t\tOn(\"DynamicClient\").Return(nil, errors.New(\"error\"))\n\t_, err = newServer(factory, &config.Config{\n\t\tUploaderType: uploader.KopiaType,\n\t\tClientQPS: 1,\n\t\tClientBurst: 1,\n\t\tClientPageSize: 100,\n\t}, logger)\n\trequire.Error(t, err)\n\n\tinvalidCM := builder.ForConfigMap(\"velero\", \"invalid\").Data(\"invalid\", \"{\\\"a\\\": \\\"b}\").Result()\n\tcrClient := velerotest.NewFakeControllerRuntimeClient(t, invalidCM)\n\n\tfactory.On(\"KubeClient\").Return(crClient, nil).\n\t\tOn(\"Client\").Return(nil, nil).\n\t\tOn(\"DynamicClient\").Return(nil, errors.New(\"error\"))\n\t_, err = newServer(factory, &config.Config{\n\t\tUploaderType: uploader.KopiaType,\n\t\tBackupRepoConfig: \"invalid\",\n\t}, logger)\n\trequire.Error(t, err)\n\n\tfactory.On(\"KubeClient\").Return(crClient, nil).\n\t\tOn(\"Client\").Return(nil, nil).\n\t\tOn(\"DynamicClient\").Return(nil, errors.New(\"error\"))\n\t_, err = newServer(factory, &config.Config{\n\t\tUploaderType: uploader.KopiaType,\n\t\tRepoMaintenanceJobConfig: \"invalid\",\n\t}, logger)\n\trequire.Error(t, err)\n}\n\nfunc Test_namespaceExists(t *testing.T) {\n\tclient := kubefake.NewSimpleClientset(&corev1api.Namespace{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"velero\",\n\t\t},\n\t})\n\tserver := &server{\n\t\tkubeClient: client,\n\t\tlogger: logrus.New(),\n\t}\n\n\t// namespace doesn't exist\n\trequire.Error(t, server.namespaceExists(\"not-exist\"))\n\n\t// namespace exists\n\tassert.NoError(t, server.namespaceExists(\"velero\"))\n}\n\nfunc Test_veleroResourcesExist(t *testing.T) {\n\thelper := &discovery_mocks.Helper{}\n\tserver := &server{\n\t\tdiscoveryHelper: helper,\n\t\tlogger: logrus.New(),\n\t}\n\n\t// velero resources don't exist\n\thelper.On(\"Resources\").Return(nil)\n\trequire.Error(t, server.veleroResourcesExist())\n\n\t// velero resources exist\n\thelper.On(\"Resources\").Unset()\n\thelper.On(\"Resources\").Return([]*metav1.APIResourceList{\n\t\t{\n\t\t\tGroupVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\tAPIResources: []metav1.APIResource{\n\t\t\t\t{Kind: \"Backup\"},\n\t\t\t\t{Kind: \"Restore\"},\n\t\t\t\t{Kind: \"Schedule\"},\n\t\t\t\t{Kind: \"DownloadRequest\"},\n\t\t\t\t{Kind: \"DeleteBackupRequest\"},\n\t\t\t\t{Kind: \"PodVolumeBackup\"},\n\t\t\t\t{Kind: \"PodVolumeRestore\"},\n\t\t\t\t{Kind: \"BackupRepository\"},\n\t\t\t\t{Kind: \"BackupStorageLocation\"},\n\t\t\t\t{Kind: \"VolumeSnapshotLocation\"},\n\t\t\t\t{Kind: \"ServerStatusRequest\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tGroupVersion: velerov2alpha1api.SchemeGroupVersion.String(),\n\t\t\tAPIResources: []metav1.APIResource{\n\t\t\t\t{Kind: \"DataUpload\"},\n\t\t\t\t{Kind: \"DataDownload\"},\n\t\t\t},\n\t\t},\n\t})\n\tassert.NoError(t, server.veleroResourcesExist())\n}\n\nfunc Test_markInProgressBackupsFailed(t *testing.T) {\n\tscheme := runtime.NewScheme()\n\tvelerov1api.AddToScheme(scheme)\n\n\tc := fake.NewClientBuilder().\n\t\tWithScheme(scheme).\n\t\tWithLists(&velerov1api.BackupList{\n\t\t\tItems: []velerov1api.Backup{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\tName: \"backup01\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\t\tPhase: velerov1api.BackupPhaseInProgress,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\tName: \"backup02\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\t\tPhase: velerov1api.BackupPhaseCompleted,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t}).\n\t\tBuild()\n\tmarkInProgressBackupsFailed(t.Context(), c, \"velero\", logrus.New())\n\n\tbackup01 := &velerov1api.Backup{}\n\trequire.NoError(t, c.Get(t.Context(), client.ObjectKey{Namespace: \"velero\", Name: \"backup01\"}, backup01))\n\tassert.Equal(t, velerov1api.BackupPhaseFailed, backup01.Status.Phase)\n\n\tbackup02 := &velerov1api.Backup{}\n\trequire.NoError(t, c.Get(t.Context(), client.ObjectKey{Namespace: \"velero\", Name: \"backup02\"}, backup02))\n\tassert.Equal(t, velerov1api.BackupPhaseCompleted, backup02.Status.Phase)\n}\n\nfunc Test_markInProgressRestoresFailed(t *testing.T) {\n\tscheme := runtime.NewScheme()\n\tvelerov1api.AddToScheme(scheme)\n\n\tc := fake.NewClientBuilder().\n\t\tWithScheme(scheme).\n\t\tWithLists(&velerov1api.RestoreList{\n\t\t\tItems: []velerov1api.Restore{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\tName: \"restore01\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: velerov1api.RestoreStatus{\n\t\t\t\t\t\tPhase: velerov1api.RestorePhaseInProgress,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\tName: \"restore02\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: velerov1api.RestoreStatus{\n\t\t\t\t\t\tPhase: velerov1api.RestorePhaseCompleted,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t}).\n\t\tBuild()\n\tmarkInProgressRestoresFailed(t.Context(), c, \"velero\", logrus.New())\n\n\trestore01 := &velerov1api.Restore{}\n\trequire.NoError(t, c.Get(t.Context(), client.ObjectKey{Namespace: \"velero\", Name: \"restore01\"}, restore01))\n\tassert.Equal(t, velerov1api.RestorePhaseFailed, restore01.Status.Phase)\n\n\trestore02 := &velerov1api.Restore{}\n\trequire.NoError(t, c.Get(t.Context(), client.ObjectKey{Namespace: \"velero\", Name: \"restore02\"}, restore02))\n\tassert.Equal(t, velerov1api.RestorePhaseCompleted, restore02.Status.Phase)\n}\n\nfunc Test_setDefaultBackupLocation(t *testing.T) {\n\tscheme := runtime.NewScheme()\n\tvelerov1api.AddToScheme(scheme)\n\n\tc := fake.NewClientBuilder().\n\t\tWithScheme(scheme).\n\t\tWithLists(&velerov1api.BackupStorageLocationList{\n\t\t\tItems: []velerov1api.BackupStorageLocation{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\tName: \"default\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\tName: \"non-default\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t}).\n\t\tBuild()\n\tsetDefaultBackupLocation(t.Context(), c, \"velero\", \"default\", logrus.New())\n\n\tdefaultLocation := &velerov1api.BackupStorageLocation{}\n\trequire.NoError(t, c.Get(t.Context(), client.ObjectKey{Namespace: \"velero\", Name: \"default\"}, defaultLocation))\n\tassert.True(t, defaultLocation.Spec.Default)\n\n\tnonDefaultLocation := &velerov1api.BackupStorageLocation{}\n\trequire.NoError(t, c.Get(t.Context(), client.ObjectKey{Namespace: \"velero\", Name: \"non-default\"}, nonDefaultLocation))\n\tassert.False(t, nonDefaultLocation.Spec.Default)\n\n\t// no default location specified\n\tc = fake.NewClientBuilder().WithScheme(scheme).Build()\n\terr := setDefaultBackupLocation(t.Context(), c, \"velero\", \"\", logrus.New())\n\trequire.NoError(t, err)\n\n\t// no default location created\n\terr = setDefaultBackupLocation(t.Context(), c, \"velero\", \"default\", logrus.New())\n\tassert.NoError(t, err)\n}\n" }, { "path": "pkg/cmd/test/const.go", "content": "package test\n\nvar VeleroNameSpace = \"velero-test\"\nvar CaptureFlag = \"CAPTRUE-OUTPUT\"\n" }, { "path": "pkg/cmd/util/cacert/bsl_cacert.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage cacert\n\nimport (\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// GetCACertFromBackup fetches the BackupStorageLocation for a backup and returns its cacert\nfunc GetCACertFromBackup(ctx context.Context, client kbclient.Client, namespace string, backup *velerov1api.Backup) (string, error) {\n\treturn GetCACertFromBSL(ctx, client, namespace, backup.Spec.StorageLocation)\n}\n\n// GetCACertFromRestore fetches the BackupStorageLocation for a restore's backup and returns its cacert\nfunc GetCACertFromRestore(ctx context.Context, client kbclient.Client, namespace string, restore *velerov1api.Restore) (string, error) {\n\t// First get the backup that this restore references\n\tbackup := &velerov1api.Backup{}\n\tkey := kbclient.ObjectKey{\n\t\tNamespace: namespace,\n\t\tName: restore.Spec.BackupName,\n\t}\n\n\tif err := client.Get(ctx, key, backup); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\t// Backup not found is not a fatal error for cacert retrieval\n\t\t\treturn \"\", nil\n\t\t}\n\t\treturn \"\", errors.Wrapf(err, \"error getting backup %s\", restore.Spec.BackupName)\n\t}\n\n\treturn GetCACertFromBackup(ctx, client, namespace, backup)\n}\n\n// GetCACertFromBSL fetches a BackupStorageLocation directly and returns its cacert\n// Priority order: caCertRef (from Secret) > caCert (inline, deprecated)\nfunc GetCACertFromBSL(ctx context.Context, client kbclient.Client, namespace, bslName string) (string, error) {\n\tif bslName == \"\" {\n\t\treturn \"\", nil\n\t}\n\n\tbsl := &velerov1api.BackupStorageLocation{}\n\tkey := kbclient.ObjectKey{\n\t\tNamespace: namespace,\n\t\tName: bslName,\n\t}\n\n\tif err := client.Get(ctx, key, bsl); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\t// BSL not found is not a fatal error, just means no cacert\n\t\t\treturn \"\", nil\n\t\t}\n\t\treturn \"\", errors.Wrapf(err, \"error getting backup storage location %s\", bslName)\n\t}\n\n\tif bsl.Spec.ObjectStorage == nil {\n\t\treturn \"\", nil\n\t}\n\n\t// Prefer caCertRef over inline caCert\n\tif bsl.Spec.ObjectStorage.CACertRef != nil {\n\t\t// Fetch certificate from Secret\n\t\tsecret := &corev1api.Secret{}\n\t\tsecretKey := types.NamespacedName{\n\t\t\tName: bsl.Spec.ObjectStorage.CACertRef.Name,\n\t\t\tNamespace: namespace,\n\t\t}\n\n\t\tif err := client.Get(ctx, secretKey, secret); err != nil {\n\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\treturn \"\", errors.Errorf(\"certificate secret %s not found in namespace %s\",\n\t\t\t\t\tbsl.Spec.ObjectStorage.CACertRef.Name, namespace)\n\t\t\t}\n\t\t\treturn \"\", errors.Wrapf(err, \"error getting certificate secret %s\",\n\t\t\t\tbsl.Spec.ObjectStorage.CACertRef.Name)\n\t\t}\n\n\t\tkeyName := bsl.Spec.ObjectStorage.CACertRef.Key\n\t\tif keyName == \"\" {\n\t\t\treturn \"\", errors.New(\"caCertRef key is empty\")\n\t\t}\n\n\t\tcertData, ok := secret.Data[keyName]\n\t\tif !ok {\n\t\t\treturn \"\", errors.Errorf(\"key %s not found in secret %s\",\n\t\t\t\tkeyName, bsl.Spec.ObjectStorage.CACertRef.Name)\n\t\t}\n\n\t\treturn string(certData), nil\n\t}\n\n\t// Fall back to inline caCert (deprecated)\n\tif len(bsl.Spec.ObjectStorage.CACert) > 0 {\n\t\treturn string(bsl.Spec.ObjectStorage.CACert), nil\n\t}\n\n\treturn \"\", nil\n}\n" }, { "path": "pkg/cmd/util/cacert/bsl_cacert_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage cacert\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n)\n\nfunc TestGetCACertFromBackup(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\tbsl *velerov1api.BackupStorageLocation\n\t\texpectedCACert string\n\t\texpectedError bool\n\t}{\n\t\t{\n\t\t\tname: \"backup with BSL containing cacert\",\n\t\t\tbackup: builder.ForBackup(\"test-ns\", \"test-backup\").\n\t\t\t\tStorageLocation(\"test-bsl\").\n\t\t\t\tResult(),\n\t\t\tbsl: builder.ForBackupStorageLocation(\"test-ns\", \"test-bsl\").\n\t\t\t\tProvider(\"aws\").\n\t\t\t\tBucket(\"test-bucket\").\n\t\t\t\tCACert([]byte(\"test-cacert-content\")).\n\t\t\t\tResult(),\n\t\t\texpectedCACert: \"test-cacert-content\",\n\t\t\texpectedError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"backup with BSL without cacert\",\n\t\t\tbackup: builder.ForBackup(\"test-ns\", \"test-backup\").\n\t\t\t\tStorageLocation(\"test-bsl\").\n\t\t\t\tResult(),\n\t\t\tbsl: builder.ForBackupStorageLocation(\"test-ns\", \"test-bsl\").\n\t\t\t\tProvider(\"aws\").\n\t\t\t\tBucket(\"test-bucket\").\n\t\t\t\tResult(),\n\t\t\texpectedCACert: \"\",\n\t\t\texpectedError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"backup without storage location\",\n\t\t\tbackup: builder.ForBackup(\"test-ns\", \"test-backup\").\n\t\t\t\tResult(),\n\t\t\tbsl: nil,\n\t\t\texpectedCACert: \"\",\n\t\t\texpectedError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"BSL not found\",\n\t\t\tbackup: builder.ForBackup(\"test-ns\", \"test-backup\").\n\t\t\t\tStorageLocation(\"missing-bsl\").\n\t\t\t\tResult(),\n\t\t\tbsl: nil,\n\t\t\texpectedCACert: \"\",\n\t\t\texpectedError: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar objs []runtime.Object\n\t\t\tobjs = append(objs, tc.backup)\n\t\t\tif tc.bsl != nil {\n\t\t\t\tobjs = append(objs, tc.bsl)\n\t\t\t}\n\n\t\t\tfakeClient := fake.NewClientBuilder().\n\t\t\t\tWithScheme(util.VeleroScheme).\n\t\t\t\tWithRuntimeObjects(objs...).\n\t\t\t\tBuild()\n\n\t\t\tcacert, err := GetCACertFromBackup(t.Context(), fakeClient, \"test-ns\", tc.backup)\n\n\t\t\tif tc.expectedError {\n\t\t\t\tassert.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, tc.expectedCACert, cacert)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetCACertFromRestore(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\trestore *velerov1api.Restore\n\t\tbackup *velerov1api.Backup\n\t\tbsl *velerov1api.BackupStorageLocation\n\t\texpectedCACert string\n\t\texpectedError bool\n\t}{\n\t\t{\n\t\t\tname: \"restore with backup having BSL containing cacert\",\n\t\t\trestore: builder.ForRestore(\"test-ns\", \"test-restore\").\n\t\t\t\tBackup(\"test-backup\").\n\t\t\t\tResult(),\n\t\t\tbackup: builder.ForBackup(\"test-ns\", \"test-backup\").\n\t\t\t\tStorageLocation(\"test-bsl\").\n\t\t\t\tResult(),\n\t\t\tbsl: builder.ForBackupStorageLocation(\"test-ns\", \"test-bsl\").\n\t\t\t\tProvider(\"aws\").\n\t\t\t\tBucket(\"test-bucket\").\n\t\t\t\tCACert([]byte(\"test-cacert-content\")).\n\t\t\t\tResult(),\n\t\t\texpectedCACert: \"test-cacert-content\",\n\t\t\texpectedError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"restore with backup not found\",\n\t\t\trestore: builder.ForRestore(\"test-ns\", \"test-restore\").\n\t\t\t\tBackup(\"missing-backup\").\n\t\t\t\tResult(),\n\t\t\tbackup: nil,\n\t\t\tbsl: nil,\n\t\t\texpectedCACert: \"\",\n\t\t\texpectedError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"restore with backup having BSL without cacert\",\n\t\t\trestore: builder.ForRestore(\"test-ns\", \"test-restore\").\n\t\t\t\tBackup(\"test-backup\").\n\t\t\t\tResult(),\n\t\t\tbackup: builder.ForBackup(\"test-ns\", \"test-backup\").\n\t\t\t\tStorageLocation(\"test-bsl\").\n\t\t\t\tResult(),\n\t\t\tbsl: builder.ForBackupStorageLocation(\"test-ns\", \"test-bsl\").\n\t\t\t\tProvider(\"aws\").\n\t\t\t\tBucket(\"test-bucket\").\n\t\t\t\tResult(),\n\t\t\texpectedCACert: \"\",\n\t\t\texpectedError: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar objs []runtime.Object\n\t\t\tobjs = append(objs, tc.restore)\n\t\t\tif tc.backup != nil {\n\t\t\t\tobjs = append(objs, tc.backup)\n\t\t\t}\n\t\t\tif tc.bsl != nil {\n\t\t\t\tobjs = append(objs, tc.bsl)\n\t\t\t}\n\n\t\t\tfakeClient := fake.NewClientBuilder().\n\t\t\t\tWithScheme(util.VeleroScheme).\n\t\t\t\tWithRuntimeObjects(objs...).\n\t\t\t\tBuild()\n\n\t\t\tcacert, err := GetCACertFromRestore(t.Context(), fakeClient, \"test-ns\", tc.restore)\n\n\t\t\tif tc.expectedError {\n\t\t\t\tassert.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, tc.expectedCACert, cacert)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetCACertFromBSL(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tbslName string\n\t\tbsl *velerov1api.BackupStorageLocation\n\t\texpectedCACert string\n\t\texpectedError bool\n\t}{\n\t\t{\n\t\t\tname: \"BSL with cacert\",\n\t\t\tbslName: \"test-bsl\",\n\t\t\tbsl: builder.ForBackupStorageLocation(\"test-ns\", \"test-bsl\").\n\t\t\t\tProvider(\"aws\").\n\t\t\t\tBucket(\"test-bucket\").\n\t\t\t\tCACert([]byte(\"test-cacert-content\")).\n\t\t\t\tResult(),\n\t\t\texpectedCACert: \"test-cacert-content\",\n\t\t\texpectedError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"BSL without cacert\",\n\t\t\tbslName: \"test-bsl\",\n\t\t\tbsl: builder.ForBackupStorageLocation(\"test-ns\", \"test-bsl\").\n\t\t\t\tProvider(\"aws\").\n\t\t\t\tBucket(\"test-bucket\").\n\t\t\t\tResult(),\n\t\t\texpectedCACert: \"\",\n\t\t\texpectedError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"empty BSL name\",\n\t\t\tbslName: \"\",\n\t\t\tbsl: nil,\n\t\t\texpectedCACert: \"\",\n\t\t\texpectedError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"BSL not found\",\n\t\t\tbslName: \"missing-bsl\",\n\t\t\tbsl: nil,\n\t\t\texpectedCACert: \"\",\n\t\t\texpectedError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"BSL with invalid CA cert format\",\n\t\t\tbslName: \"test-bsl\",\n\t\t\tbsl: builder.ForBackupStorageLocation(\"test-ns\", \"test-bsl\").\n\t\t\t\tProvider(\"aws\").\n\t\t\t\tBucket(\"test-bucket\").\n\t\t\t\tCACert([]byte(\"INVALID CERT DATA WITHOUT PEM HEADERS\")).\n\t\t\t\tResult(),\n\t\t\texpectedCACert: \"INVALID CERT DATA WITHOUT PEM HEADERS\", // We still return it, validation happens during TLS handshake\n\t\t\texpectedError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"BSL with malformed PEM certificate\",\n\t\t\tbslName: \"test-bsl\",\n\t\t\tbsl: builder.ForBackupStorageLocation(\"test-ns\", \"test-bsl\").\n\t\t\t\tProvider(\"aws\").\n\t\t\t\tBucket(\"test-bucket\").\n\t\t\t\tCACert([]byte(\"-----BEGIN CERTIFICATE-----\\nINVALID BASE64 DATA!!!\\n-----END CERTIFICATE-----\\n\")).\n\t\t\t\tResult(),\n\t\t\texpectedCACert: \"-----BEGIN CERTIFICATE-----\\nINVALID BASE64 DATA!!!\\n-----END CERTIFICATE-----\\n\",\n\t\t\texpectedError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"BSL with nil config\",\n\t\t\tbslName: \"test-bsl\",\n\t\t\tbsl: &velerov1api.BackupStorageLocation{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"test-ns\",\n\t\t\t\t\tName: \"test-bsl\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"aws\",\n\t\t\t\t\tConfig: nil,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedCACert: \"\",\n\t\t\texpectedError: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar objs []runtime.Object\n\t\t\tif tc.bsl != nil {\n\t\t\t\tobjs = append(objs, tc.bsl)\n\t\t\t}\n\n\t\t\tfakeClient := fake.NewClientBuilder().\n\t\t\t\tWithScheme(util.VeleroScheme).\n\t\t\t\tWithRuntimeObjects(objs...).\n\t\t\t\tBuild()\n\n\t\t\tcacert, err := GetCACertFromBSL(t.Context(), fakeClient, \"test-ns\", tc.bslName)\n\n\t\t\tif tc.expectedError {\n\t\t\t\tassert.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, tc.expectedCACert, cacert)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestGetCACertFromBSL_WithCACertRef tests the new caCertRef functionality\nfunc TestGetCACertFromBSL_WithCACertRef(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tbslName string\n\t\tbsl *velerov1api.BackupStorageLocation\n\t\tsecret *corev1api.Secret\n\t\texpectedCACert string\n\t\texpectedError bool\n\t\terrorContains string\n\t}{\n\t\t{\n\t\t\tname: \"BSL with caCertRef pointing to valid secret\",\n\t\t\tbslName: \"test-bsl\",\n\t\t\tbsl: &velerov1api.BackupStorageLocation{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"test-ns\",\n\t\t\t\t\tName: \"test-bsl\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"aws\",\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"test-bucket\",\n\t\t\t\t\t\t\tCACertRef: &corev1api.SecretKeySelector{\n\t\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\t\tName: \"test-secret\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\tKey: \"ca-bundle.crt\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsecret: &corev1api.Secret{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"test-ns\",\n\t\t\t\t\tName: \"test-secret\",\n\t\t\t\t},\n\t\t\t\tData: map[string][]byte{\n\t\t\t\t\t\"ca-bundle.crt\": []byte(\"test-cacert-from-secret\"),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedCACert: \"test-cacert-from-secret\",\n\t\t\texpectedError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"BSL with both caCertRef and caCert - caCertRef takes precedence\",\n\t\t\tbslName: \"test-bsl\",\n\t\t\tbsl: &velerov1api.BackupStorageLocation{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"test-ns\",\n\t\t\t\t\tName: \"test-bsl\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"aws\",\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"test-bucket\",\n\t\t\t\t\t\t\tCACert: []byte(\"inline-cacert-deprecated\"),\n\t\t\t\t\t\t\tCACertRef: &corev1api.SecretKeySelector{\n\t\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\t\tName: \"test-secret\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\tKey: \"ca-bundle.crt\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsecret: &corev1api.Secret{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"test-ns\",\n\t\t\t\t\tName: \"test-secret\",\n\t\t\t\t},\n\t\t\t\tData: map[string][]byte{\n\t\t\t\t\t\"ca-bundle.crt\": []byte(\"cacert-from-secret-takes-precedence\"),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedCACert: \"cacert-from-secret-takes-precedence\",\n\t\t\texpectedError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"BSL with caCertRef but secret not found\",\n\t\t\tbslName: \"test-bsl\",\n\t\t\tbsl: &velerov1api.BackupStorageLocation{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"test-ns\",\n\t\t\t\t\tName: \"test-bsl\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"aws\",\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"test-bucket\",\n\t\t\t\t\t\t\tCACertRef: &corev1api.SecretKeySelector{\n\t\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\t\tName: \"missing-secret\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\tKey: \"ca-bundle.crt\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsecret: nil,\n\t\t\texpectedCACert: \"\",\n\t\t\texpectedError: true,\n\t\t\terrorContains: \"certificate secret missing-secret not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"BSL with caCertRef but key not found in secret\",\n\t\t\tbslName: \"test-bsl\",\n\t\t\tbsl: &velerov1api.BackupStorageLocation{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"test-ns\",\n\t\t\t\t\tName: \"test-bsl\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"aws\",\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"test-bucket\",\n\t\t\t\t\t\t\tCACertRef: &corev1api.SecretKeySelector{\n\t\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\t\tName: \"test-secret\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\tKey: \"missing-key\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsecret: &corev1api.Secret{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"test-ns\",\n\t\t\t\t\tName: \"test-secret\",\n\t\t\t\t},\n\t\t\t\tData: map[string][]byte{\n\t\t\t\t\t\"ca-bundle.crt\": []byte(\"test-cacert\"),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedCACert: \"\",\n\t\t\texpectedError: true,\n\t\t\terrorContains: \"key missing-key not found in secret test-secret\",\n\t\t},\n\t\t{\n\t\t\tname: \"BSL with caCertRef but empty key\",\n\t\t\tbslName: \"test-bsl\",\n\t\t\tbsl: &velerov1api.BackupStorageLocation{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"test-ns\",\n\t\t\t\t\tName: \"test-bsl\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"aws\",\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"test-bucket\",\n\t\t\t\t\t\t\tCACertRef: &corev1api.SecretKeySelector{\n\t\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\t\tName: \"test-secret\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\tKey: \"\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsecret: &corev1api.Secret{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"test-ns\",\n\t\t\t\t\tName: \"test-secret\",\n\t\t\t\t},\n\t\t\t\tData: map[string][]byte{\n\t\t\t\t\t\"ca-bundle.crt\": []byte(\"test-cacert\"),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedCACert: \"\",\n\t\t\texpectedError: true,\n\t\t\terrorContains: \"caCertRef key is empty\",\n\t\t},\n\t\t{\n\t\t\tname: \"BSL with caCertRef containing multi-line PEM certificate\",\n\t\t\tbslName: \"test-bsl\",\n\t\t\tbsl: &velerov1api.BackupStorageLocation{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"test-ns\",\n\t\t\t\t\tName: \"test-bsl\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"aws\",\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"test-bucket\",\n\t\t\t\t\t\t\tCACertRef: &corev1api.SecretKeySelector{\n\t\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\t\tName: \"test-secret\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\tKey: \"ca.pem\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsecret: &corev1api.Secret{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"test-ns\",\n\t\t\t\t\tName: \"test-secret\",\n\t\t\t\t},\n\t\t\t\tData: map[string][]byte{\n\t\t\t\t\t\"ca.pem\": []byte(\"-----BEGIN CERTIFICATE-----\\nMIIDETC...\\n-----END CERTIFICATE-----\\n\"),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedCACert: \"-----BEGIN CERTIFICATE-----\\nMIIDETC...\\n-----END CERTIFICATE-----\\n\",\n\t\t\texpectedError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"BSL falls back to inline caCert when caCertRef is nil\",\n\t\t\tbslName: \"test-bsl\",\n\t\t\tbsl: builder.ForBackupStorageLocation(\"test-ns\", \"test-bsl\").\n\t\t\t\tProvider(\"aws\").\n\t\t\t\tBucket(\"test-bucket\").\n\t\t\t\tCACert([]byte(\"fallback-inline-cacert\")).\n\t\t\t\tResult(),\n\t\t\tsecret: nil,\n\t\t\texpectedCACert: \"fallback-inline-cacert\",\n\t\t\texpectedError: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar objs []runtime.Object\n\t\t\tif tc.bsl != nil {\n\t\t\t\tobjs = append(objs, tc.bsl)\n\t\t\t}\n\t\t\tif tc.secret != nil {\n\t\t\t\tobjs = append(objs, tc.secret)\n\t\t\t}\n\n\t\t\tscheme := runtime.NewScheme()\n\t\t\t_ = velerov1api.AddToScheme(scheme)\n\t\t\t_ = corev1api.AddToScheme(scheme)\n\n\t\t\tfakeClient := fake.NewClientBuilder().\n\t\t\t\tWithScheme(scheme).\n\t\t\t\tWithRuntimeObjects(objs...).\n\t\t\t\tBuild()\n\n\t\t\tcacert, err := GetCACertFromBSL(t.Context(), fakeClient, \"test-ns\", tc.bslName)\n\n\t\t\tif tc.expectedError {\n\t\t\t\trequire.Error(t, err)\n\t\t\t\tif tc.errorContains != \"\" {\n\t\t\t\t\tassert.Contains(t, err.Error(), tc.errorContains)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, tc.expectedCACert, cacert)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestGetCACertFromBackup_ClientError tests error scenarios where client.Get returns non-NotFound errors\nfunc TestGetCACertFromBackup_ClientError(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\tbsl *velerov1api.BackupStorageLocation\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"client error getting BSL\",\n\t\t\tbackup: builder.ForBackup(\"test-ns\", \"test-backup\").\n\t\t\t\tStorageLocation(\"test-bsl\").\n\t\t\t\tResult(),\n\t\t\tbsl: builder.ForBackupStorageLocation(\"different-ns\", \"test-bsl\"). // Different namespace to trigger error\n\t\t\t\t\t\t\t\t\t\t\t\tProvider(\"aws\").\n\t\t\t\t\t\t\t\t\t\t\t\tBucket(\"test-bucket\").\n\t\t\t\t\t\t\t\t\t\t\t\tCACert([]byte(\"test-cacert-content\")).\n\t\t\t\t\t\t\t\t\t\t\t\tResult(),\n\t\t\texpectedError: \"not found\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar objs []runtime.Object\n\t\t\tobjs = append(objs, tc.backup)\n\t\t\tif tc.bsl != nil {\n\t\t\t\tobjs = append(objs, tc.bsl)\n\t\t\t}\n\n\t\t\tfakeClient := fake.NewClientBuilder().\n\t\t\t\tWithScheme(util.VeleroScheme).\n\t\t\t\tWithRuntimeObjects(objs...).\n\t\t\t\tBuild()\n\n\t\t\t// Try to get BSL from wrong namespace to simulate error\n\t\t\t_, err := GetCACertFromBSL(t.Context(), fakeClient, \"wrong-ns\", tc.backup.Spec.StorageLocation)\n\n\t\t\trequire.NoError(t, err) // Not found errors are handled gracefully\n\t\t})\n\t}\n}\n\n// TestGetCACertFromRestore_ClientError tests error scenarios for GetCACertFromRestore\nfunc TestGetCACertFromRestore_ClientError(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\trestore *velerov1api.Restore\n\t\tbackup *velerov1api.Backup\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"backup in different namespace\",\n\t\t\trestore: builder.ForRestore(\"test-ns\", \"test-restore\").\n\t\t\t\tBackup(\"test-backup\").\n\t\t\t\tResult(),\n\t\t\tbackup: builder.ForBackup(\"different-ns\", \"test-backup\"). // Different namespace\n\t\t\t\t\t\t\t\t\t\t\tStorageLocation(\"test-bsl\").\n\t\t\t\t\t\t\t\t\t\t\tResult(),\n\t\t\texpectedError: \"not found\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar objs []runtime.Object\n\t\t\tobjs = append(objs, tc.restore)\n\t\t\tif tc.backup != nil {\n\t\t\t\tobjs = append(objs, tc.backup)\n\t\t\t}\n\n\t\t\tfakeClient := fake.NewClientBuilder().\n\t\t\t\tWithScheme(util.VeleroScheme).\n\t\t\t\tWithRuntimeObjects(objs...).\n\t\t\t\tBuild()\n\n\t\t\t// This should not find the backup in the wrong namespace\n\t\t\tcacert, err := GetCACertFromRestore(t.Context(), fakeClient, \"test-ns\", tc.restore)\n\n\t\t\trequire.NoError(t, err) // Not found errors are handled gracefully, returning empty string\n\t\t\tassert.Empty(t, cacert)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/cmd/util/confirm/confirm.go", "content": "package confirm\n\nimport (\n\t\"bufio\"\n\t\"fmt\"\n\t\"os\"\n\t\"strings\"\n\n\t\"github.com/spf13/pflag\"\n)\n\ntype ConfirmOptions struct {\n\tConfirm bool\n\tflagDescription string\n}\n\nfunc NewConfirmOptions() *ConfirmOptions {\n\treturn &ConfirmOptions{flagDescription: \"Confirm action\"}\n}\n\nfunc NewConfirmOptionsWithDescription(desc string) *ConfirmOptions {\n\treturn &ConfirmOptions{flagDescription: desc}\n}\n\n// Bind confirm flags.\nfunc (o *ConfirmOptions) BindFlags(flags *pflag.FlagSet) {\n\tflags.BoolVar(&o.Confirm, \"confirm\", o.Confirm, o.flagDescription)\n}\n\n// GetConfirmation ensures that the user confirms the action before proceeding.\nfunc GetConfirmation(prompts ...string) bool {\n\treader := bufio.NewReader(os.Stdin)\n\n\tfor {\n\t\tfor i := range prompts {\n\t\t\tfmt.Println(prompts[i])\n\t\t}\n\t\tfmt.Printf(\"Are you sure you want to continue (Y/N)? \")\n\n\t\tconfirmation, err := reader.ReadString('\\n')\n\t\tif err != nil {\n\t\t\tfmt.Fprintf(os.Stderr, \"error reading user input: %v\\n\", err)\n\t\t\treturn false\n\t\t}\n\t\tconfirmation = strings.TrimSpace(confirmation)\n\t\tif len(confirmation) != 1 {\n\t\t\tcontinue\n\t\t}\n\n\t\tswitch strings.ToLower(confirmation) {\n\t\tcase \"y\":\n\t\t\treturn true\n\t\tcase \"n\":\n\t\t\treturn false\n\t\t}\n\t}\n}\n" }, { "path": "pkg/cmd/util/downloadrequest/downloadrequest.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage downloadrequest\n\nimport (\n\t\"compress/gzip\"\n\t\"context\"\n\t\"crypto/tls\"\n\t\"crypto/x509\"\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\t\"net/url\"\n\t\"os\"\n\t\"time\"\n\n\t\"github.com/google/uuid\"\n\t\"github.com/pkg/errors\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tveleroV1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n)\n\n// ErrNotFound is exported for external packages to check for when a file is\n// not found\nvar ErrNotFound = errors.New(\"file not found\")\nvar ErrDownloadRequestDownloadURLTimeout = errors.New(\"download request download url timeout, check velero server logs for errors. backup storage location may not be available\")\n\nfunc Stream(\n\tctx context.Context,\n\tkbClient kbclient.Client,\n\tnamespace, name string,\n\tkind veleroV1api.DownloadTargetKind,\n\tw io.Writer,\n\ttimeout time.Duration,\n\tinsecureSkipTLSVerify bool,\n\tcaCertFile string,\n) error {\n\treturn StreamWithBSLCACert(ctx, kbClient, namespace, name, kind, w, timeout, insecureSkipTLSVerify, caCertFile, \"\")\n}\n\n// StreamWithBSLCACert is like Stream but accepts an additional bslCACert parameter\n// that contains the cacert from the BackupStorageLocation config\nfunc StreamWithBSLCACert(\n\tctx context.Context,\n\tkbClient kbclient.Client,\n\tnamespace, name string,\n\tkind veleroV1api.DownloadTargetKind,\n\tw io.Writer,\n\ttimeout time.Duration,\n\tinsecureSkipTLSVerify bool,\n\tcaCertFile string,\n\tbslCACert string,\n) error {\n\tctx, cancel := context.WithTimeout(ctx, timeout)\n\tdefer cancel()\n\n\tdownloadURL, err := getDownloadURL(ctx, kbClient, namespace, name, kind)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif err := download(ctx, downloadURL, kind, w, insecureSkipTLSVerify, caCertFile, bslCACert); err != nil {\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\nfunc getDownloadURL(\n\tctx context.Context,\n\tkbClient kbclient.Client,\n\tnamespace, name string,\n\tkind veleroV1api.DownloadTargetKind,\n) (string, error) {\n\tuuid, err := uuid.NewRandom()\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\treqName := fmt.Sprintf(\"%s-%s\", name, uuid.String())\n\tcreated := builder.ForDownloadRequest(namespace, reqName).Target(kind, name).Result()\n\n\tif err := kbClient.Create(ctx, created, &kbclient.CreateOptions{}); err != nil {\n\t\treturn \"\", errors.WithStack(err)\n\t}\n\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\treturn \"\", ErrDownloadRequestDownloadURLTimeout\n\n\t\tcase <-time.After(25 * time.Millisecond):\n\t\t\tupdated := &veleroV1api.DownloadRequest{}\n\t\t\tif err := kbClient.Get(ctx, kbclient.ObjectKey{Name: created.Name, Namespace: namespace}, updated); err != nil {\n\t\t\t\treturn \"\", errors.WithStack(err)\n\t\t\t}\n\n\t\t\tif updated.Status.DownloadURL != \"\" {\n\t\t\t\treturn updated.Status.DownloadURL, nil\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc download(\n\tctx context.Context,\n\tdownloadURL string,\n\tkind veleroV1api.DownloadTargetKind,\n\tw io.Writer,\n\tinsecureSkipTLSVerify bool,\n\tcaCertFile string,\n\tcaCertByteString string,\n) error {\n\tvar caPool *x509.CertPool\n\tvar err error\n\n\t// Initialize caPool once\n\tcaPool, err = x509.SystemCertPool()\n\tif err != nil {\n\t\tcaPool = x509.NewCertPool()\n\t}\n\n\t// Try to load CA cert from file first\n\tif len(caCertFile) > 0 {\n\t\tcaCert, err := os.ReadFile(caCertFile)\n\t\tif err != nil {\n\t\t\t// If caCertFile fails and BSL cert is available, fall back to it\n\t\t\tif len(caCertByteString) > 0 {\n\t\t\t\tfmt.Fprintf(os.Stderr, \"Warning: Failed to open CA certificate file %s: %v. Using CA certificate from backup storage location instead.\\n\", caCertFile, err)\n\t\t\t\tcaPool.AppendCertsFromPEM([]byte(caCertByteString))\n\t\t\t} else {\n\t\t\t\t// If no BSL cert available, return the original error\n\t\t\t\treturn errors.Wrapf(err, \"couldn't open cacert\")\n\t\t\t}\n\t\t} else {\n\t\t\tcaPool.AppendCertsFromPEM(caCert)\n\t\t}\n\t} else if len(caCertByteString) > 0 {\n\t\t// If no caCertFile specified, use BSL cert if available\n\t\tcaPool.AppendCertsFromPEM([]byte(caCertByteString))\n\t}\n\n\tdefaultTransport := http.DefaultTransport.(*http.Transport)\n\t// same settings as the default transport\n\t// aside from TLSClientConfig\n\thttpClient := new(http.Client)\n\thttpClient.Transport = &http.Transport{\n\t\tTLSClientConfig: &tls.Config{\n\t\t\tInsecureSkipVerify: insecureSkipTLSVerify, //nolint:gosec // This parameter is useful for some scenarios.\n\t\t\tRootCAs: caPool,\n\t\t},\n\t\tDialContext: defaultTransport.DialContext,\n\t\tForceAttemptHTTP2: defaultTransport.ForceAttemptHTTP2,\n\t\tMaxIdleConns: defaultTransport.MaxIdleConns,\n\t\tProxy: defaultTransport.Proxy,\n\t\tTLSHandshakeTimeout: defaultTransport.TLSHandshakeTimeout,\n\t\tExpectContinueTimeout: defaultTransport.ExpectContinueTimeout,\n\t}\n\n\thttpReq, err := http.NewRequestWithContext(ctx, http.MethodGet, downloadURL, nil)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tresp, err := httpClient.Do(httpReq)\n\tif err != nil {\n\t\tif urlErr, ok := err.(*url.Error); ok {\n\t\t\tif _, ok := urlErr.Err.(x509.UnknownAuthorityError); ok {\n\t\t\t\treturn fmt.Errorf(\"%s\\n\\nThe --insecure-skip-tls-verify flag can also be used to accept any TLS certificate for the download, but it is susceptible to man-in-the-middle attacks\", err.Error())\n\t\t\t}\n\t\t}\n\t\treturn err\n\t}\n\tdefer resp.Body.Close()\n\n\tif resp.StatusCode != http.StatusOK {\n\t\tbody, err := io.ReadAll(resp.Body)\n\t\tif err != nil {\n\t\t\treturn errors.Wrapf(err, \"request failed: unable to decode response body\")\n\t\t}\n\n\t\tif resp.StatusCode == http.StatusNotFound {\n\t\t\treturn ErrNotFound\n\t\t}\n\n\t\treturn errors.Errorf(\"request failed: %v\", string(body))\n\t}\n\n\treader := resp.Body\n\tif kind != veleroV1api.DownloadTargetKindBackupContents {\n\t\t// need to decompress logs\n\t\tgzipReader, err := gzip.NewReader(resp.Body)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer gzipReader.Close()\n\t\treader = gzipReader\n\t}\n\n\t_, err = io.Copy(w, reader)\n\treturn err\n}\n" }, { "path": "pkg/cmd/util/downloadrequest/downloadrequest_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage downloadrequest\n\nimport (\n\t\"bytes\"\n\t\"compress/gzip\"\n\t\"crypto/rand\"\n\t\"crypto/rsa\"\n\t\"crypto/tls\"\n\t\"crypto/x509\"\n\t\"crypto/x509/pkix\"\n\t\"encoding/pem\"\n\t\"math/big\"\n\t\"net\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"os\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/cacert\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n)\n\n// createSelfSignedCertificate creates a self-signed certificate for testing.\n// This allows us to test the BSL CA certificate functionality by ensuring\n// that the client properly validates server certificates against the CA cert\n// provided in the BackupStorageLocation configuration.\nfunc createSelfSignedCertificate(t *testing.T) (tls.Certificate, []byte) {\n\tt.Helper()\n\n\t// Generate a private key\n\tpriv, err := rsa.GenerateKey(rand.Reader, 2048)\n\trequire.NoError(t, err)\n\n\t// Create certificate template for a self-signed certificate\n\ttemplate := x509.Certificate{\n\t\tSerialNumber: big.NewInt(1),\n\t\tSubject: pkix.Name{\n\t\t\tOrganization: []string{\"Test Org\"},\n\t\t\tCountry: []string{\"US\"},\n\t\t\tProvince: []string{\"\"},\n\t\t\tLocality: []string{\"Test City\"},\n\t\t\tStreetAddress: []string{\"\"},\n\t\t\tPostalCode: []string{\"\"},\n\t\t},\n\t\tNotBefore: time.Now(),\n\t\tNotAfter: time.Now().Add(365 * 24 * time.Hour),\n\t\tKeyUsage: x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature,\n\t\tExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},\n\t\tBasicConstraintsValid: true,\n\t\tIPAddresses: []net.IP{net.IPv4(127, 0, 0, 1)},\n\t\tDNSNames: []string{\"localhost\"},\n\t}\n\n\t// Create the certificate\n\tcertDER, err := x509.CreateCertificate(rand.Reader, &template, &template, &priv.PublicKey, priv)\n\trequire.NoError(t, err)\n\n\t// Encode certificate and key to PEM\n\tcertPEM := pem.EncodeToMemory(&pem.Block{Type: \"CERTIFICATE\", Bytes: certDER})\n\tkeyPEM := pem.EncodeToMemory(&pem.Block{Type: \"RSA PRIVATE KEY\", Bytes: x509.MarshalPKCS1PrivateKey(priv)})\n\n\t// Create tls.Certificate\n\ttlsCert, err := tls.X509KeyPair(certPEM, keyPEM)\n\trequire.NoError(t, err)\n\n\treturn tlsCert, certPEM\n}\n\nfunc TestStream(t *testing.T) {\n\t// Create a test server that returns download content\n\ttestContent := \"test backup content\"\n\tdownloadServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.WriteHeader(http.StatusOK)\n\t\tif strings.Contains(r.URL.Path, \"log\") {\n\t\t\t// For logs, return gzipped content\n\t\t\tgzipWriter := gzip.NewWriter(w)\n\t\t\tgzipWriter.Write([]byte(testContent))\n\t\t\tgzipWriter.Close()\n\t\t} else {\n\t\t\tw.Write([]byte(testContent))\n\t\t}\n\t}))\n\tdefer downloadServer.Close()\n\n\ttestCases := []struct {\n\t\tname string\n\t\ttarget velerov1api.DownloadTargetKind\n\t\ttimeout time.Duration\n\t\tsetupClient func(*testing.T, kbclient.WithWatch)\n\t\texpectedError bool\n\t\texpectedErrMessage string\n\t\tvalidateContent func(*testing.T, *bytes.Buffer)\n\t}{\n\t\t{\n\t\t\tname: \"successful backup log download\",\n\t\t\ttarget: velerov1api.DownloadTargetKindBackupLog,\n\t\t\ttimeout: 5 * time.Second,\n\t\t\tsetupClient: func(t *testing.T, client kbclient.WithWatch) {\n\t\t\t\tt.Helper()\n\t\t\t\t// Simulate controller updating the DownloadRequest with URL\n\t\t\t\tgo func() {\n\t\t\t\t\ttime.Sleep(10 * time.Millisecond)\n\t\t\t\t\tlist := &velerov1api.DownloadRequestList{}\n\t\t\t\t\terr := client.List(t.Context(), list)\n\t\t\t\t\tassert.NoError(t, err)\n\n\t\t\t\t\tfor _, dr := range list.Items {\n\t\t\t\t\t\tdr.Status.DownloadURL = downloadServer.URL + \"/log\"\n\t\t\t\t\t\tdr.Status.Phase = velerov1api.DownloadRequestPhaseProcessed\n\t\t\t\t\t\terr := client.Update(t.Context(), &dr)\n\t\t\t\t\t\tassert.NoError(t, err)\n\t\t\t\t\t}\n\t\t\t\t}()\n\t\t\t},\n\t\t\texpectedError: false,\n\t\t\tvalidateContent: func(t *testing.T, buf *bytes.Buffer) {\n\t\t\t\tt.Helper()\n\t\t\t\t// Logs should be decompressed\n\t\t\t\tassert.Equal(t, testContent, buf.String())\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"successful backup contents download\",\n\t\t\ttarget: velerov1api.DownloadTargetKindBackupContents,\n\t\t\ttimeout: 5 * time.Second,\n\t\t\tsetupClient: func(t *testing.T, client kbclient.WithWatch) {\n\t\t\t\tt.Helper()\n\t\t\t\t// Simulate controller updating the DownloadRequest with URL\n\t\t\t\tgo func() {\n\t\t\t\t\ttime.Sleep(10 * time.Millisecond)\n\t\t\t\t\tlist := &velerov1api.DownloadRequestList{}\n\t\t\t\t\terr := client.List(t.Context(), list)\n\t\t\t\t\tassert.NoError(t, err)\n\n\t\t\t\t\tfor _, dr := range list.Items {\n\t\t\t\t\t\tdr.Status.DownloadURL = downloadServer.URL + \"/contents\"\n\t\t\t\t\t\tdr.Status.Phase = velerov1api.DownloadRequestPhaseProcessed\n\t\t\t\t\t\terr := client.Update(t.Context(), &dr)\n\t\t\t\t\t\tassert.NoError(t, err)\n\t\t\t\t\t}\n\t\t\t\t}()\n\t\t\t},\n\t\t\texpectedError: false,\n\t\t\tvalidateContent: func(t *testing.T, buf *bytes.Buffer) {\n\t\t\t\tt.Helper()\n\t\t\t\tassert.Equal(t, testContent, buf.String())\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"timeout waiting for download URL\",\n\t\t\ttarget: velerov1api.DownloadTargetKindBackupLog,\n\t\t\ttimeout: 50 * time.Millisecond,\n\t\t\tsetupClient: func(t *testing.T, client kbclient.WithWatch) {\n\t\t\t\tt.Helper()\n\t\t\t},\n\t\t\texpectedError: true,\n\t\t\texpectedErrMessage: \"download request download url timeout\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfakeClient := fake.NewClientBuilder().\n\t\t\t\tWithScheme(util.VeleroScheme).\n\t\t\t\tBuild()\n\n\t\t\tif tc.setupClient != nil {\n\t\t\t\ttc.setupClient(t, fakeClient)\n\t\t\t}\n\n\t\t\tvar buf bytes.Buffer\n\t\t\tctx := t.Context()\n\n\t\t\terr := Stream(ctx, fakeClient, \"test-ns\", \"test-backup\", tc.target, &buf, tc.timeout, false, \"\")\n\n\t\t\tif tc.expectedError {\n\t\t\t\trequire.Error(t, err)\n\t\t\t\tif tc.expectedErrMessage != \"\" {\n\t\t\t\t\tassert.Contains(t, err.Error(), tc.expectedErrMessage)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tif tc.validateContent != nil {\n\t\t\t\t\ttc.validateContent(t, &buf)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestStreamWithBSLCACert(t *testing.T) {\n\t// Create a test server that returns download content\n\ttestContent := \"test backup content with BSL CA cert\"\n\n\t// Create self-signed certificate for TLS testing\n\ttlsCert, serverCACertPEM := createSelfSignedCertificate(t)\n\n\t// Create TLS test server for testing CA certificate functionality\n\ttlsServer := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.WriteHeader(http.StatusOK)\n\t\tif strings.Contains(r.URL.Path, \"log\") {\n\t\t\t// For logs, return gzipped content\n\t\t\tgzipWriter := gzip.NewWriter(w)\n\t\t\tgzipWriter.Write([]byte(testContent))\n\t\t\tgzipWriter.Close()\n\t\t} else {\n\t\t\tw.Write([]byte(testContent))\n\t\t}\n\t}))\n\ttlsServer.TLS = &tls.Config{\n\t\tCertificates: []tls.Certificate{tlsCert},\n\t}\n\ttlsServer.StartTLS()\n\tdefer tlsServer.Close()\n\n\t// Also create a regular HTTP server for non-TLS tests\n\thttpServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.WriteHeader(http.StatusOK)\n\t\tif strings.Contains(r.URL.Path, \"log\") {\n\t\t\t// For logs, return gzipped content\n\t\t\tgzipWriter := gzip.NewWriter(w)\n\t\t\tgzipWriter.Write([]byte(testContent))\n\t\t\tgzipWriter.Close()\n\t\t} else {\n\t\t\tw.Write([]byte(testContent))\n\t\t}\n\t}))\n\tdefer httpServer.Close()\n\n\ttestCases := []struct {\n\t\tname string\n\t\ttarget velerov1api.DownloadTargetKind\n\t\tbslCACert string\n\t\ttimeout time.Duration\n\t\tsetupClient func(*testing.T, kbclient.WithWatch)\n\t\tuseTLS bool\n\t\texpectedError bool\n\t\texpectedErrMessage string\n\t\tvalidateContent func(*testing.T, *bytes.Buffer)\n\t}{\n\t\t{\n\t\t\tname: \"successful TLS backup log download with correct BSL CA cert\",\n\t\t\ttarget: velerov1api.DownloadTargetKindBackupLog,\n\t\t\tbslCACert: string(serverCACertPEM),\n\t\t\ttimeout: 5 * time.Second,\n\t\t\tuseTLS: true,\n\t\t\tsetupClient: func(t *testing.T, client kbclient.WithWatch) {\n\t\t\t\tt.Helper()\n\t\t\t\t// Simulate controller updating the DownloadRequest with URL\n\t\t\t\tgo func() {\n\t\t\t\t\ttime.Sleep(10 * time.Millisecond)\n\t\t\t\t\tlist := &velerov1api.DownloadRequestList{}\n\t\t\t\t\terr := client.List(t.Context(), list)\n\t\t\t\t\tassert.NoError(t, err)\n\n\t\t\t\t\tfor _, dr := range list.Items {\n\t\t\t\t\t\tdr.Status.DownloadURL = tlsServer.URL + \"/log\"\n\t\t\t\t\t\tdr.Status.Phase = velerov1api.DownloadRequestPhaseProcessed\n\t\t\t\t\t\terr := client.Update(t.Context(), &dr)\n\t\t\t\t\t\tassert.NoError(t, err)\n\t\t\t\t\t}\n\t\t\t\t}()\n\t\t\t},\n\t\t\texpectedError: false,\n\t\t\tvalidateContent: func(t *testing.T, buf *bytes.Buffer) {\n\t\t\t\tt.Helper()\n\t\t\t\t// Logs should be decompressed\n\t\t\t\tassert.Equal(t, testContent, buf.String())\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"successful TLS backup contents download with correct BSL CA cert\",\n\t\t\ttarget: velerov1api.DownloadTargetKindBackupContents,\n\t\t\tbslCACert: string(serverCACertPEM),\n\t\t\ttimeout: 5 * time.Second,\n\t\t\tuseTLS: true,\n\t\t\tsetupClient: func(t *testing.T, client kbclient.WithWatch) {\n\t\t\t\tt.Helper()\n\t\t\t\t// Simulate controller updating the DownloadRequest with URL\n\t\t\t\tgo func() {\n\t\t\t\t\ttime.Sleep(10 * time.Millisecond)\n\t\t\t\t\tlist := &velerov1api.DownloadRequestList{}\n\t\t\t\t\terr := client.List(t.Context(), list)\n\t\t\t\t\tassert.NoError(t, err)\n\n\t\t\t\t\tfor _, dr := range list.Items {\n\t\t\t\t\t\tdr.Status.DownloadURL = tlsServer.URL + \"/contents\"\n\t\t\t\t\t\tdr.Status.Phase = velerov1api.DownloadRequestPhaseProcessed\n\t\t\t\t\t\terr := client.Update(t.Context(), &dr)\n\t\t\t\t\t\tassert.NoError(t, err)\n\t\t\t\t\t}\n\t\t\t\t}()\n\t\t\t},\n\t\t\texpectedError: false,\n\t\t\tvalidateContent: func(t *testing.T, buf *bytes.Buffer) {\n\t\t\t\tt.Helper()\n\t\t\t\tassert.Equal(t, testContent, buf.String())\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"failed TLS download with wrong BSL CA cert\",\n\t\t\ttarget: velerov1api.DownloadTargetKindBackupContents,\n\t\t\tbslCACert: \"-----BEGIN CERTIFICATE-----\\nMIIBkTCB+wIJAKHHIgKwERfFMA0GCSqGSIb3DQEBCwUAMBkxFzAVBgNVBAMTDmRp\\nZmZlcmVudC1jZXJ0MB4XDTE5MDQwMTAwMDAwMFoXDTI5MDQwMTAwMDAwMFowGTEX\\nMBUGA1UEAxMOZGlmZmVyZW50LWNlcnQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJ\\nAoGBAOO4V+XrhVEGbTqnO2FM5eVFaM3KMKc3M9/C1aeg3vvY+Th3OhqJBxEYFxXL\\nZoSqkwL/E6BjQb0NdSyJY9wdM4Ie3gElcZBKYVpHXYYAVhrepRCRVJEIHdBN8ybr\\nFoBBDjd/ID1qy8Gdp3RihPFNvCNx0RWWqPAJtNXWJvCiNRCDAgMBAAEwDQYJKoZI\\nhvcNAQELBQADgYEAGEwwGz7HAmH0J3pAJzQKPCb8HJG8hTjD6qkMon3Bp6gZ\\n-----END CERTIFICATE-----\\n\",\n\t\t\ttimeout: 5 * time.Second,\n\t\t\tuseTLS: true,\n\t\t\tsetupClient: func(t *testing.T, client kbclient.WithWatch) {\n\t\t\t\tt.Helper()\n\t\t\t\t// Simulate controller updating the DownloadRequest with URL\n\t\t\t\tgo func() {\n\t\t\t\t\ttime.Sleep(10 * time.Millisecond)\n\t\t\t\t\tlist := &velerov1api.DownloadRequestList{}\n\t\t\t\t\terr := client.List(t.Context(), list)\n\t\t\t\t\tassert.NoError(t, err)\n\n\t\t\t\t\tfor _, dr := range list.Items {\n\t\t\t\t\t\tdr.Status.DownloadURL = tlsServer.URL + \"/contents\"\n\t\t\t\t\t\tdr.Status.Phase = velerov1api.DownloadRequestPhaseProcessed\n\t\t\t\t\t\terr := client.Update(t.Context(), &dr)\n\t\t\t\t\t\tassert.NoError(t, err)\n\t\t\t\t\t}\n\t\t\t\t}()\n\t\t\t},\n\t\t\texpectedError: true,\n\t\t\texpectedErrMessage: \"x509\",\n\t\t},\n\t\t{\n\t\t\tname: \"successful HTTP download with empty BSL CA cert\",\n\t\t\ttarget: velerov1api.DownloadTargetKindBackupContents,\n\t\t\tbslCACert: \"\",\n\t\t\ttimeout: 5 * time.Second,\n\t\t\tuseTLS: false,\n\t\t\tsetupClient: func(t *testing.T, client kbclient.WithWatch) {\n\t\t\t\tt.Helper()\n\t\t\t\t// Simulate controller updating the DownloadRequest with URL\n\t\t\t\tgo func() {\n\t\t\t\t\ttime.Sleep(10 * time.Millisecond)\n\t\t\t\t\tlist := &velerov1api.DownloadRequestList{}\n\t\t\t\t\terr := client.List(t.Context(), list)\n\t\t\t\t\tassert.NoError(t, err)\n\n\t\t\t\t\tfor _, dr := range list.Items {\n\t\t\t\t\t\tdr.Status.DownloadURL = httpServer.URL + \"/contents\"\n\t\t\t\t\t\tdr.Status.Phase = velerov1api.DownloadRequestPhaseProcessed\n\t\t\t\t\t\terr := client.Update(t.Context(), &dr)\n\t\t\t\t\t\tassert.NoError(t, err)\n\t\t\t\t\t}\n\t\t\t\t}()\n\t\t\t},\n\t\t\texpectedError: false,\n\t\t\tvalidateContent: func(t *testing.T, buf *bytes.Buffer) {\n\t\t\t\tt.Helper()\n\t\t\t\tassert.Equal(t, testContent, buf.String())\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"timeout waiting for download URL with BSL CA cert\",\n\t\t\ttarget: velerov1api.DownloadTargetKindBackupLog,\n\t\t\tbslCACert: \"test-ca-cert-content\",\n\t\t\ttimeout: 50 * time.Millisecond,\n\t\t\tsetupClient: func(t *testing.T, client kbclient.WithWatch) {\n\t\t\t\tt.Helper()\n\t\t\t},\n\t\t\texpectedError: true,\n\t\t\texpectedErrMessage: \"download request download url timeout\",\n\t\t},\n\t\t{\n\t\t\tname: \"failed TLS download with malformed BSL CA cert\",\n\t\t\ttarget: velerov1api.DownloadTargetKindBackupLog,\n\t\t\tbslCACert: \"-----BEGIN CERTIFICATE-----\\nINVALID CERT DATA\\n-----END CERTIFICATE-----\\n\",\n\t\t\ttimeout: 5 * time.Second,\n\t\t\tuseTLS: true,\n\t\t\tsetupClient: func(t *testing.T, client kbclient.WithWatch) {\n\t\t\t\tt.Helper()\n\t\t\t\t// Simulate controller updating the DownloadRequest with URL\n\t\t\t\tgo func() {\n\t\t\t\t\ttime.Sleep(10 * time.Millisecond)\n\t\t\t\t\tlist := &velerov1api.DownloadRequestList{}\n\t\t\t\t\terr := client.List(t.Context(), list)\n\t\t\t\t\tassert.NoError(t, err)\n\n\t\t\t\t\tfor _, dr := range list.Items {\n\t\t\t\t\t\tdr.Status.DownloadURL = tlsServer.URL + \"/log\"\n\t\t\t\t\t\tdr.Status.Phase = velerov1api.DownloadRequestPhaseProcessed\n\t\t\t\t\t\terr := client.Update(t.Context(), &dr)\n\t\t\t\t\t\tassert.NoError(t, err)\n\t\t\t\t\t}\n\t\t\t\t}()\n\t\t\t},\n\t\t\texpectedError: true,\n\t\t\texpectedErrMessage: \"x509\", // Should fail due to malformed cert not being added to pool\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfakeClient := fake.NewClientBuilder().\n\t\t\t\tWithScheme(util.VeleroScheme).\n\t\t\t\tBuild()\n\n\t\t\tif tc.setupClient != nil {\n\t\t\t\ttc.setupClient(t, fakeClient)\n\t\t\t}\n\n\t\t\tvar buf bytes.Buffer\n\t\t\tctx := t.Context()\n\n\t\t\terr := StreamWithBSLCACert(ctx, fakeClient, \"test-ns\", \"test-backup\", tc.target, &buf, tc.timeout, false, \"\", tc.bslCACert)\n\n\t\t\tif tc.expectedError {\n\t\t\t\trequire.Error(t, err)\n\t\t\t\tif tc.expectedErrMessage != \"\" {\n\t\t\t\t\tassert.Contains(t, err.Error(), tc.expectedErrMessage)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tif tc.validateContent != nil {\n\t\t\t\t\ttc.validateContent(t, &buf)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestDownload(t *testing.T) {\n\ttestContent := \"test content for download\"\n\tcompressedContent := new(bytes.Buffer)\n\tgzipWriter := gzip.NewWriter(compressedContent)\n\tgzipWriter.Write([]byte(testContent))\n\tgzipWriter.Close()\n\n\ttestCases := []struct {\n\t\tname string\n\t\tserverHandler http.HandlerFunc\n\t\ttarget velerov1api.DownloadTargetKind\n\t\tinsecureSkipTLSVerify bool\n\t\tcaCertFile string\n\t\tbslCACert string\n\t\texpectedContent string\n\t\texpectedError bool\n\t\terrorType error\n\t}{\n\t\t{\n\t\t\tname: \"successful download with gzip for logs\",\n\t\t\tserverHandler: func(w http.ResponseWriter, r *http.Request) {\n\t\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\t\tw.Write(compressedContent.Bytes())\n\t\t\t},\n\t\t\ttarget: velerov1api.DownloadTargetKindBackupLog,\n\t\t\texpectedContent: testContent,\n\t\t\texpectedError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"successful download without gzip for backup contents\",\n\t\t\tserverHandler: func(w http.ResponseWriter, r *http.Request) {\n\t\t\t\tw.WriteHeader(http.StatusOK)\n\t\t\t\tw.Write([]byte(testContent))\n\t\t\t},\n\t\t\ttarget: velerov1api.DownloadTargetKindBackupContents,\n\t\t\texpectedContent: testContent,\n\t\t\texpectedError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"404 not found error\",\n\t\t\tserverHandler: func(w http.ResponseWriter, r *http.Request) {\n\t\t\t\tw.WriteHeader(http.StatusNotFound)\n\t\t\t},\n\t\t\ttarget: velerov1api.DownloadTargetKindBackupLog,\n\t\t\texpectedError: true,\n\t\t\terrorType: ErrNotFound,\n\t\t},\n\t\t{\n\t\t\tname: \"500 internal server error\",\n\t\t\tserverHandler: func(w http.ResponseWriter, r *http.Request) {\n\t\t\t\tw.WriteHeader(http.StatusInternalServerError)\n\t\t\t\tw.Write([]byte(\"internal server error\"))\n\t\t\t},\n\t\t\ttarget: velerov1api.DownloadTargetKindBackupLog,\n\t\t\texpectedError: true,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tserver := httptest.NewServer(tc.serverHandler)\n\t\t\tdefer server.Close()\n\n\t\t\tvar buf bytes.Buffer\n\t\t\terr := download(\n\t\t\t\tt.Context(),\n\t\t\t\tserver.URL,\n\t\t\t\ttc.target,\n\t\t\t\t&buf,\n\t\t\t\ttc.insecureSkipTLSVerify,\n\t\t\t\ttc.caCertFile,\n\t\t\t\ttc.bslCACert,\n\t\t\t)\n\n\t\t\tif tc.expectedError {\n\t\t\t\trequire.Error(t, err)\n\t\t\t\tif tc.errorType != nil {\n\t\t\t\t\tassert.Equal(t, tc.errorType, err)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, tc.expectedContent, buf.String())\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestStreamWithBSLCACertEndToEnd tests the complete flow from BSL to download with CA cert\nfunc TestStreamWithBSLCACertEndToEnd(t *testing.T) {\n\ttestContent := \"end-to-end test content\"\n\n\t// Create self-signed certificate for TLS testing\n\ttlsCert, serverCACertPEM := createSelfSignedCertificate(t)\n\n\t// Create TLS test server\n\ttlsServer := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.WriteHeader(http.StatusOK)\n\t\tif strings.Contains(r.URL.Path, \"log\") {\n\t\t\t// For logs, return gzipped content\n\t\t\tgzipWriter := gzip.NewWriter(w)\n\t\t\tgzipWriter.Write([]byte(testContent))\n\t\t\tgzipWriter.Close()\n\t\t} else {\n\t\t\tw.Write([]byte(testContent))\n\t\t}\n\t}))\n\ttlsServer.TLS = &tls.Config{\n\t\tCertificates: []tls.Certificate{tlsCert},\n\t}\n\ttlsServer.StartTLS()\n\tdefer tlsServer.Close()\n\n\t// Create BSL with CA cert\n\tbsl := builder.ForBackupStorageLocation(\"test-ns\", \"test-bsl\").\n\t\tProvider(\"aws\").\n\t\tBucket(\"test-bucket\").\n\t\tCACert(serverCACertPEM).\n\t\tResult()\n\n\t// Create backup that references the BSL\n\tbackup := builder.ForBackup(\"test-ns\", \"test-backup\").\n\t\tStorageLocation(\"test-bsl\").\n\t\tResult()\n\n\t// Setup fake client with BSL and backup\n\tfakeClient := fake.NewClientBuilder().\n\t\tWithScheme(util.VeleroScheme).\n\t\tWithRuntimeObjects(bsl, backup).\n\t\tBuild()\n\n\t// Helper function to simulate controller updating the DownloadRequest\n\tsimulateControllerUpdate := func() {\n\t\tgo func() {\n\t\t\ttime.Sleep(10 * time.Millisecond)\n\t\t\tlist := &velerov1api.DownloadRequestList{}\n\t\t\terr := fakeClient.List(t.Context(), list)\n\t\t\tassert.NoError(t, err)\n\n\t\t\tfor _, dr := range list.Items {\n\t\t\t\tdr.Status.DownloadURL = tlsServer.URL + \"/log\"\n\t\t\t\tdr.Status.Phase = velerov1api.DownloadRequestPhaseProcessed\n\t\t\t\terr := fakeClient.Update(t.Context(), &dr)\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t}()\n\t}\n\n\t// Test the complete flow\n\tctx := t.Context()\n\n\t// First, try to download WITHOUT the CA cert - this should fail\n\tsimulateControllerUpdate()\n\tvar bufFail bytes.Buffer\n\terr := StreamWithBSLCACert(ctx, fakeClient, \"test-ns\", \"test-backup\", velerov1api.DownloadTargetKindBackupLog, &bufFail, 5*time.Second, false, \"\", \"\")\n\trequire.Error(t, err)\n\tassert.Contains(t, err.Error(), \"x509\") // Should fail with certificate validation error\n\n\t// Now get CA cert from BSL through backup\n\tcacertStr, err := cacert.GetCACertFromBackup(ctx, fakeClient, \"test-ns\", backup)\n\trequire.NoError(t, err)\n\trequire.Equal(t, string(serverCACertPEM), cacertStr)\n\n\t// Try again with the CA cert - this should succeed\n\tsimulateControllerUpdate()\n\tvar bufSuccess bytes.Buffer\n\terr = StreamWithBSLCACert(ctx, fakeClient, \"test-ns\", \"test-backup\", velerov1api.DownloadTargetKindBackupLog, &bufSuccess, 5*time.Second, false, \"\", cacertStr)\n\trequire.NoError(t, err)\n\n\t// Verify content was downloaded and decompressed correctly\n\tassert.Equal(t, testContent, bufSuccess.String())\n}\n\n// TestBackwardCompatibilityWithoutBSLCACert tests that old download requests work without BSL CA cert\nfunc TestBackwardCompatibilityWithoutBSLCACert(t *testing.T) {\n\ttestContent := \"backward compatibility test content\"\n\n\t// Create HTTP (not HTTPS) server to simulate old behavior where TLS wasn't required\n\thttpServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.WriteHeader(http.StatusOK)\n\t\tif strings.Contains(r.URL.Path, \"log\") {\n\t\t\t// For logs, return gzipped content\n\t\t\tgzipWriter := gzip.NewWriter(w)\n\t\t\tgzipWriter.Write([]byte(testContent))\n\t\t\tgzipWriter.Close()\n\t\t} else {\n\t\t\tw.Write([]byte(testContent))\n\t\t}\n\t}))\n\tdefer httpServer.Close()\n\n\t// Create BSL without CA cert (simulating old configuration)\n\tbsl := builder.ForBackupStorageLocation(\"test-ns\", \"test-bsl\").\n\t\tProvider(\"aws\").\n\t\tBucket(\"test-bucket\").\n\t\t// No CACert() call - simulating pre-CA cert support\n\t\tResult()\n\n\t// Create backup that references the BSL\n\tbackup := builder.ForBackup(\"test-ns\", \"test-backup\").\n\t\tStorageLocation(\"test-bsl\").\n\t\tResult()\n\n\t// Setup fake client with BSL and backup\n\tfakeClient := fake.NewClientBuilder().\n\t\tWithScheme(util.VeleroScheme).\n\t\tWithRuntimeObjects(bsl, backup).\n\t\tBuild()\n\n\t// Simulate controller updating the DownloadRequest with HTTP URL (old behavior)\n\tgo func() {\n\t\ttime.Sleep(10 * time.Millisecond)\n\t\tlist := &velerov1api.DownloadRequestList{}\n\t\terr := fakeClient.List(t.Context(), list)\n\t\tassert.NoError(t, err)\n\n\t\tfor _, dr := range list.Items {\n\t\t\tdr.Status.DownloadURL = httpServer.URL + \"/log\"\n\t\t\tdr.Status.Phase = velerov1api.DownloadRequestPhaseProcessed\n\t\t\terr := fakeClient.Update(t.Context(), &dr)\n\t\t\tassert.NoError(t, err)\n\t\t}\n\t}()\n\n\tctx := t.Context()\n\n\t// Test 1: Stream function (without BSL CA cert parameter) should work\n\tvar buf1 bytes.Buffer\n\terr := Stream(ctx, fakeClient, \"test-ns\", \"test-backup\", velerov1api.DownloadTargetKindBackupLog, &buf1, 5*time.Second, false, \"\")\n\trequire.NoError(t, err)\n\tassert.Equal(t, testContent, buf1.String())\n\n\t// Test 2: StreamWithBSLCACert with empty BSL CA cert should also work\n\tgo func() {\n\t\ttime.Sleep(10 * time.Millisecond)\n\t\tlist := &velerov1api.DownloadRequestList{}\n\t\terr := fakeClient.List(t.Context(), list)\n\t\tassert.NoError(t, err)\n\n\t\tfor _, dr := range list.Items {\n\t\t\tdr.Status.DownloadURL = httpServer.URL + \"/contents\"\n\t\t\tdr.Status.Phase = velerov1api.DownloadRequestPhaseProcessed\n\t\t\terr := fakeClient.Update(t.Context(), &dr)\n\t\t\tassert.NoError(t, err)\n\t\t}\n\t}()\n\n\tvar buf2 bytes.Buffer\n\terr = StreamWithBSLCACert(ctx, fakeClient, \"test-ns\", \"test-backup\", velerov1api.DownloadTargetKindBackupContents, &buf2, 5*time.Second, false, \"\", \"\")\n\trequire.NoError(t, err)\n\tassert.Equal(t, testContent, buf2.String())\n\n\t// Test 3: Getting CA cert from BSL should return empty string (not error)\n\tcacert, err := cacert.GetCACertFromBackup(ctx, fakeClient, \"test-ns\", backup)\n\trequire.NoError(t, err)\n\tassert.Empty(t, cacert)\n}\n\n// TestMixedEnvironmentHTTPAndHTTPS tests environment with both HTTP and HTTPS endpoints\nfunc TestMixedEnvironmentHTTPAndHTTPS(t *testing.T) {\n\ttestContentHTTP := \"http content\"\n\ttestContentHTTPS := \"https content\"\n\n\t// Create HTTP server\n\thttpServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.WriteHeader(http.StatusOK)\n\t\tw.Write([]byte(testContentHTTP))\n\t}))\n\tdefer httpServer.Close()\n\n\t// Create HTTPS server with self-signed cert\n\ttlsCert, serverCACertPEM := createSelfSignedCertificate(t)\n\thttpsServer := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.WriteHeader(http.StatusOK)\n\t\tw.Write([]byte(testContentHTTPS))\n\t}))\n\thttpsServer.TLS = &tls.Config{\n\t\tCertificates: []tls.Certificate{tlsCert},\n\t}\n\thttpsServer.StartTLS()\n\tdefer httpsServer.Close()\n\n\t// Create two BSLs - one for HTTP (old) and one for HTTPS (new)\n\tbslHTTP := builder.ForBackupStorageLocation(\"test-ns\", \"bsl-http\").\n\t\tProvider(\"aws\").\n\t\tBucket(\"http-bucket\").\n\t\t// No CA cert for HTTP\n\t\tResult()\n\n\tbslHTTPS := builder.ForBackupStorageLocation(\"test-ns\", \"bsl-https\").\n\t\tProvider(\"aws\").\n\t\tBucket(\"https-bucket\").\n\t\tCACert(serverCACertPEM).\n\t\tResult()\n\n\t// Create backups for each BSL\n\tbackupHTTP := builder.ForBackup(\"test-ns\", \"backup-http\").\n\t\tStorageLocation(\"bsl-http\").\n\t\tResult()\n\n\tbackupHTTPS := builder.ForBackup(\"test-ns\", \"backup-https\").\n\t\tStorageLocation(\"bsl-https\").\n\t\tResult()\n\n\t// Setup fake client\n\tfakeClient := fake.NewClientBuilder().\n\t\tWithScheme(util.VeleroScheme).\n\t\tWithRuntimeObjects(bslHTTP, bslHTTPS, backupHTTP, backupHTTPS).\n\t\tBuild()\n\n\tctx := t.Context()\n\n\t// Test HTTP backup download (backward compatible)\n\tgo func() {\n\t\ttime.Sleep(10 * time.Millisecond)\n\t\tlist := &velerov1api.DownloadRequestList{}\n\t\terr := fakeClient.List(t.Context(), list)\n\t\tassert.NoError(t, err)\n\n\t\tfor _, dr := range list.Items {\n\t\t\tif strings.Contains(dr.Name, \"backup-http\") {\n\t\t\t\tdr.Status.DownloadURL = httpServer.URL\n\t\t\t\tdr.Status.Phase = velerov1api.DownloadRequestPhaseProcessed\n\t\t\t\terr := fakeClient.Update(t.Context(), &dr)\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t}\n\t}()\n\n\tvar bufHTTP bytes.Buffer\n\terr := Stream(ctx, fakeClient, \"test-ns\", \"backup-http\", velerov1api.DownloadTargetKindBackupContents, &bufHTTP, 5*time.Second, false, \"\")\n\trequire.NoError(t, err)\n\tassert.Equal(t, testContentHTTP, bufHTTP.String())\n\n\t// Test HTTPS backup download (requires CA cert)\n\tgo func() {\n\t\ttime.Sleep(10 * time.Millisecond)\n\t\tlist := &velerov1api.DownloadRequestList{}\n\t\terr := fakeClient.List(t.Context(), list)\n\t\tassert.NoError(t, err)\n\n\t\tfor _, dr := range list.Items {\n\t\t\tif strings.Contains(dr.Name, \"backup-https\") {\n\t\t\t\tdr.Status.DownloadURL = httpsServer.URL\n\t\t\t\tdr.Status.Phase = velerov1api.DownloadRequestPhaseProcessed\n\t\t\t\terr := fakeClient.Update(t.Context(), &dr)\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t}\n\t}()\n\n\t// First try without CA cert - should fail\n\tvar bufHTTPSFail bytes.Buffer\n\terr = Stream(ctx, fakeClient, \"test-ns\", \"backup-https\", velerov1api.DownloadTargetKindBackupContents, &bufHTTPSFail, 5*time.Second, false, \"\")\n\trequire.Error(t, err)\n\tassert.Contains(t, err.Error(), \"x509\")\n\n\t// Get CA cert from HTTPS BSL\n\tcacertStr, err := cacert.GetCACertFromBackup(ctx, fakeClient, \"test-ns\", backupHTTPS)\n\trequire.NoError(t, err)\n\trequire.Equal(t, string(serverCACertPEM), cacertStr)\n\n\t// Try again with CA cert - should succeed\n\tgo func() {\n\t\ttime.Sleep(10 * time.Millisecond)\n\t\tlist := &velerov1api.DownloadRequestList{}\n\t\terr := fakeClient.List(t.Context(), list)\n\t\tassert.NoError(t, err)\n\n\t\tfor _, dr := range list.Items {\n\t\t\tif strings.Contains(dr.Name, \"backup-https\") {\n\t\t\t\tdr.Status.DownloadURL = httpsServer.URL\n\t\t\t\tdr.Status.Phase = velerov1api.DownloadRequestPhaseProcessed\n\t\t\t\terr := fakeClient.Update(t.Context(), &dr)\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t}\n\t}()\n\n\tvar bufHTTPSSuccess bytes.Buffer\n\terr = StreamWithBSLCACert(ctx, fakeClient, \"test-ns\", \"backup-https\", velerov1api.DownloadTargetKindBackupContents, &bufHTTPSSuccess, 5*time.Second, false, \"\", cacertStr)\n\trequire.NoError(t, err)\n\tassert.Equal(t, testContentHTTPS, bufHTTPSSuccess.String())\n}\n\n// TestBSLUpgradeScenario tests the scenario where a BSL is upgraded to include CA cert\nfunc TestBSLUpgradeScenario(t *testing.T) {\n\ttestContent := \"bsl upgrade test content\"\n\n\t// Create self-signed certificate for TLS testing\n\ttlsCert, serverCACertPEM := createSelfSignedCertificate(t)\n\n\t// Create HTTPS server\n\thttpsServer := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.WriteHeader(http.StatusOK)\n\t\tif strings.Contains(r.URL.Path, \"log\") {\n\t\t\t// For logs, return gzipped content\n\t\t\tgzipWriter := gzip.NewWriter(w)\n\t\t\tgzipWriter.Write([]byte(testContent))\n\t\t\tgzipWriter.Close()\n\t\t} else {\n\t\t\tw.Write([]byte(testContent))\n\t\t}\n\t}))\n\thttpsServer.TLS = &tls.Config{\n\t\tCertificates: []tls.Certificate{tlsCert},\n\t}\n\thttpsServer.StartTLS()\n\tdefer httpsServer.Close()\n\n\t// Create BSL initially without CA cert (pre-upgrade state)\n\tbsl := builder.ForBackupStorageLocation(\"test-ns\", \"test-bsl\").\n\t\tProvider(\"aws\").\n\t\tBucket(\"test-bucket\").\n\t\t// Initially no CA cert\n\t\tResult()\n\n\t// Create an old backup that references this BSL\n\toldBackup := builder.ForBackup(\"test-ns\", \"old-backup\").\n\t\tStorageLocation(\"test-bsl\").\n\t\tResult()\n\n\t// Setup fake client\n\tfakeClient := fake.NewClientBuilder().\n\t\tWithScheme(util.VeleroScheme).\n\t\tWithRuntimeObjects(bsl, oldBackup).\n\t\tBuild()\n\n\tctx := t.Context()\n\n\t// Test 1: Old backup with HTTPS URL should fail without CA cert\n\tgo func() {\n\t\ttime.Sleep(10 * time.Millisecond)\n\t\tlist := &velerov1api.DownloadRequestList{}\n\t\terr := fakeClient.List(t.Context(), list)\n\t\tassert.NoError(t, err)\n\n\t\tfor _, dr := range list.Items {\n\t\t\tdr.Status.DownloadURL = httpsServer.URL + \"/log\"\n\t\t\tdr.Status.Phase = velerov1api.DownloadRequestPhaseProcessed\n\t\t\terr := fakeClient.Update(t.Context(), &dr)\n\t\t\tassert.NoError(t, err)\n\t\t}\n\t}()\n\n\tvar bufFail bytes.Buffer\n\terr := Stream(ctx, fakeClient, \"test-ns\", \"old-backup\", velerov1api.DownloadTargetKindBackupLog, &bufFail, 5*time.Second, false, \"\")\n\trequire.Error(t, err)\n\tassert.Contains(t, err.Error(), \"x509\")\n\n\t// Simulate BSL upgrade - get current BSL and update it\n\tcurrentBSL := &velerov1api.BackupStorageLocation{}\n\terr = fakeClient.Get(ctx, kbclient.ObjectKey{Namespace: \"test-ns\", Name: \"test-bsl\"}, currentBSL)\n\trequire.NoError(t, err)\n\n\t// Update the BSL to include CA cert\n\t// Ensure ObjectStorage is initialized\n\tif currentBSL.Spec.ObjectStorage == nil {\n\t\tcurrentBSL.Spec.ObjectStorage = &velerov1api.ObjectStorageLocation{}\n\t}\n\tcurrentBSL.Spec.ObjectStorage.CACert = serverCACertPEM // CA cert added after upgrade\n\n\terr = fakeClient.Update(ctx, currentBSL)\n\trequire.NoError(t, err)\n\n\t// Test 2: After BSL upgrade, old backup should work with new CA cert\n\tcacertStr, err := cacert.GetCACertFromBackup(ctx, fakeClient, \"test-ns\", oldBackup)\n\trequire.NoError(t, err)\n\trequire.Equal(t, string(serverCACertPEM), cacertStr)\n\n\tgo func() {\n\t\ttime.Sleep(10 * time.Millisecond)\n\t\tlist := &velerov1api.DownloadRequestList{}\n\t\terr := fakeClient.List(t.Context(), list)\n\t\tassert.NoError(t, err)\n\n\t\tfor _, dr := range list.Items {\n\t\t\tdr.Status.DownloadURL = httpsServer.URL + \"/log\"\n\t\t\tdr.Status.Phase = velerov1api.DownloadRequestPhaseProcessed\n\t\t\terr := fakeClient.Update(t.Context(), &dr)\n\t\t\tassert.NoError(t, err)\n\t\t}\n\t}()\n\n\tvar bufSuccess bytes.Buffer\n\terr = StreamWithBSLCACert(ctx, fakeClient, \"test-ns\", \"old-backup\", velerov1api.DownloadTargetKindBackupLog, &bufSuccess, 5*time.Second, false, \"\", cacertStr)\n\trequire.NoError(t, err)\n\tassert.Equal(t, testContent, bufSuccess.String())\n\n\t// Test 3: New backup created after upgrade should also work\n\tnewBackup := builder.ForBackup(\"test-ns\", \"new-backup\").\n\t\tStorageLocation(\"test-bsl\").\n\t\tResult()\n\n\terr = fakeClient.Create(ctx, newBackup)\n\trequire.NoError(t, err)\n\n\tcacertStr2, err := cacert.GetCACertFromBackup(ctx, fakeClient, \"test-ns\", newBackup)\n\trequire.NoError(t, err)\n\trequire.Equal(t, string(serverCACertPEM), cacertStr2)\n}\n\n// TestConcurrentDownloadsWithBSLCACert tests multiple concurrent downloads with CA cert\nfunc TestConcurrentDownloadsWithBSLCACert(t *testing.T) {\n\ttestContent := \"concurrent test content\"\n\n\t// Create self-signed certificate for TLS testing\n\ttlsCert, serverCACertPEM := createSelfSignedCertificate(t)\n\n\t// Create TLS test server\n\ttlsServer := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t// Simulate some processing time\n\t\ttime.Sleep(10 * time.Millisecond)\n\t\tw.WriteHeader(http.StatusOK)\n\t\tw.Write([]byte(testContent))\n\t}))\n\ttlsServer.TLS = &tls.Config{\n\t\tCertificates: []tls.Certificate{tlsCert},\n\t}\n\ttlsServer.StartTLS()\n\tdefer tlsServer.Close()\n\n\t// Run multiple concurrent downloads\n\tnumConcurrent := 5\n\terrors := make(chan error, numConcurrent)\n\tresults := make(chan string, numConcurrent)\n\n\tfor i := range numConcurrent {\n\t\tgo func(idx int) {\n\t\t\tvar buf bytes.Buffer\n\t\t\terr := download(\n\t\t\t\tt.Context(),\n\t\t\t\ttlsServer.URL,\n\t\t\t\tvelerov1api.DownloadTargetKindBackupContents,\n\t\t\t\t&buf,\n\t\t\t\tfalse,\n\t\t\t\t\"\",\n\t\t\t\tstring(serverCACertPEM),\n\t\t\t)\n\t\t\tif err != nil {\n\t\t\t\terrors <- err\n\t\t\t\treturn\n\t\t\t}\n\t\t\tresults <- buf.String()\n\t\t}(i)\n\t}\n\n\t// Collect results\n\tfor i := range numConcurrent {\n\t\tselect {\n\t\tcase err := <-errors:\n\t\t\tt.Fatalf(\"Concurrent download %d failed: %v\", i, err)\n\t\tcase result := <-results:\n\t\t\tassert.Equal(t, testContent, result)\n\t\tcase <-time.After(5 * time.Second):\n\t\t\tt.Fatal(\"Timeout waiting for concurrent downloads\")\n\t\t}\n\t}\n}\n\nfunc TestDownloadWithBSLCACert(t *testing.T) {\n\ttestContent := \"test content with BSL CA cert\"\n\n\t// Create self-signed certificate for TLS testing\n\ttlsCert, serverCACertPEM := createSelfSignedCertificate(t)\n\n\t// Create TLS test server with self-signed cert\n\ttlsServer := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.WriteHeader(http.StatusOK)\n\t\tw.Write([]byte(testContent))\n\t}))\n\ttlsServer.TLS = &tls.Config{\n\t\tCertificates: []tls.Certificate{tlsCert},\n\t}\n\ttlsServer.StartTLS()\n\tdefer tlsServer.Close()\n\n\t// Create HTTP test server for non-TLS tests\n\thttpServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.WriteHeader(http.StatusOK)\n\t\tw.Write([]byte(testContent))\n\t}))\n\tdefer httpServer.Close()\n\n\ttestCases := []struct {\n\t\tname string\n\t\turl string\n\t\ttarget velerov1api.DownloadTargetKind\n\t\tinsecureSkipTLSVerify bool\n\t\tbslCACert string\n\t\texpectedContent string\n\t\texpectedError bool\n\t\texpectedErrContains string\n\t}{\n\t\t{\n\t\t\tname: \"successful TLS download with correct BSL CA cert\",\n\t\t\turl: tlsServer.URL,\n\t\t\ttarget: velerov1api.DownloadTargetKindBackupContents,\n\t\t\tbslCACert: string(serverCACertPEM),\n\t\t\texpectedContent: testContent,\n\t\t\texpectedError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"successful TLS download with insecureSkipTLSVerify\",\n\t\t\turl: tlsServer.URL,\n\t\t\ttarget: velerov1api.DownloadTargetKindBackupContents,\n\t\t\tinsecureSkipTLSVerify: true,\n\t\t\tbslCACert: \"\",\n\t\t\texpectedContent: testContent,\n\t\t\texpectedError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"failed TLS download without CA cert or insecureSkipTLSVerify\",\n\t\t\turl: tlsServer.URL,\n\t\t\ttarget: velerov1api.DownloadTargetKindBackupContents,\n\t\t\tbslCACert: \"\",\n\t\t\texpectedError: true,\n\t\t\texpectedErrContains: \"x509\",\n\t\t},\n\t\t{\n\t\t\tname: \"failed TLS download with wrong BSL CA cert\",\n\t\t\turl: tlsServer.URL,\n\t\t\ttarget: velerov1api.DownloadTargetKindBackupContents,\n\t\t\tbslCACert: \"-----BEGIN CERTIFICATE-----\\nMIIBkTCB+wIJAKHHIgKwERfFMA0GCSqGSIb3DQEBCwUAMBkxFzAVBgNVBAMTDmRp\\nZmZlcmVudC1jZXJ0MB4XDTE5MDQwMTAwMDAwMFoXDTI5MDQwMTAwMDAwMFowGTEX\\nMBUGA1UEAxMOZGlmZmVyZW50LWNlcnQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJ\\nAoGBAOO4V+XrhVEGbTqnO2FM5eVFaM3KMKc3M9/C1aeg3vvY+Th3OhqJBxEYFxXL\\nZoSqkwL/E6BjQb0NdSyJY9wdM4Ie3gElcZBKYVpHXYYAVhrepRCRVJEIHdBN8ybr\\nFoBBDjd/ID1qy8Gdp3RihPFNvCNx0RWWqPAJtNXWJvCiNRCDAgMBAAEwDQYJKoZI\\nhvcNAQELBQADgYEAGEwwGz7HAmH0J3pAJzQKPCb8HJG8hTjD6qkMon3Bp6gZ\\n-----END CERTIFICATE-----\\n\",\n\t\t\texpectedError: true,\n\t\t\texpectedErrContains: \"x509\",\n\t\t},\n\t\t{\n\t\t\tname: \"successful HTTP download with empty BSL CA cert\",\n\t\t\turl: httpServer.URL,\n\t\t\ttarget: velerov1api.DownloadTargetKindBackupContents,\n\t\t\tbslCACert: \"\",\n\t\t\texpectedContent: testContent,\n\t\t\texpectedError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"successful TLS download with multiple CA certs in PEM block\",\n\t\t\turl: tlsServer.URL,\n\t\t\ttarget: velerov1api.DownloadTargetKindBackupContents,\n\t\t\tbslCACert: \"-----BEGIN CERTIFICATE-----\\nMIIBkTCB+wIJAKHHIgKwERfFMA0GCSqGSIb3DQEBCwUAMBkxFzAVBgNVBAMTDmRp\\nZmZlcmVudC1jZXJ0MB4XDTE5MDQwMTAwMDAwMFoXDTI5MDQwMTAwMDAwMFowGTEX\\nMBUGA1UEAxMOZGlmZmVyZW50LWNlcnQwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJ\\nAoGBAOO4V+XrhVEGbTqnO2FM5eVFaM3KMKc3M9/C1aeg3vvY+Th3OhqJBxEYFxXL\\nZoSqkwL/E6BjQb0NdSyJY9wdM4Ie3gElcZBKYVpHXYYAVhrepRCRVJEIHdBN8ybr\\nFoBBDjd/ID1qy8Gdp3RihPFNvCNx0RWWqPAJtNXWJvCiNRCDAgMBAAEwDQYJKoZI\\nhvcNAQELBQADgYEAGEwwGz7HAmH0J3pAJzQKPCb8HJG8hTjD6qkMon3Bp6gZ\\n-----END CERTIFICATE-----\\n\" + string(serverCACertPEM), // First cert is wrong, but second is correct\n\t\t\texpectedContent: testContent,\n\t\t\texpectedError: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar buf bytes.Buffer\n\t\t\terr := download(\n\t\t\t\tt.Context(),\n\t\t\t\ttc.url,\n\t\t\t\ttc.target,\n\t\t\t\t&buf,\n\t\t\t\ttc.insecureSkipTLSVerify,\n\t\t\t\t\"\",\n\t\t\t\ttc.bslCACert,\n\t\t\t)\n\n\t\t\tif tc.expectedError {\n\t\t\t\trequire.Error(t, err)\n\t\t\t\tif tc.expectedErrContains != \"\" {\n\t\t\t\t\tassert.Contains(t, err.Error(), tc.expectedErrContains)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, tc.expectedContent, buf.String())\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestCACertFallback tests the fallback behavior when caCertFile fails\nfunc TestCACertFallback(t *testing.T) {\n\ttestContent := \"test content for CA cert fallback\"\n\n\t// Create self-signed certificate for TLS testing\n\ttlsCert, serverCACertPEM := createSelfSignedCertificate(t)\n\n\t// Create TLS test server\n\ttlsServer := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tw.WriteHeader(http.StatusOK)\n\t\tw.Write([]byte(testContent))\n\t}))\n\ttlsServer.TLS = &tls.Config{\n\t\tCertificates: []tls.Certificate{tlsCert},\n\t}\n\ttlsServer.StartTLS()\n\tdefer tlsServer.Close()\n\n\t// Create a temporary file path that doesn't exist\n\tnonExistentFile := \"/tmp/non-existent-ca-cert-file.pem\"\n\n\ttestCases := []struct {\n\t\tname string\n\t\tcaCertFile string\n\t\tbslCACert string\n\t\texpectedError bool\n\t\texpectedErrContains string\n\t\texpectedContent string\n\t}{\n\t\t{\n\t\t\tname: \"successful download with BSL cert when caCertFile fails\",\n\t\t\tcaCertFile: nonExistentFile,\n\t\t\tbslCACert: string(serverCACertPEM),\n\t\t\texpectedError: false,\n\t\t\texpectedContent: testContent,\n\t\t},\n\t\t{\n\t\t\tname: \"failed download when both caCertFile and BSL cert are invalid\",\n\t\t\tcaCertFile: nonExistentFile,\n\t\t\tbslCACert: \"\",\n\t\t\texpectedError: true,\n\t\t\texpectedErrContains: \"couldn't open cacert\",\n\t\t},\n\t\t{\n\t\t\tname: \"BSL cert used when caCertFile is empty\",\n\t\t\tcaCertFile: \"\",\n\t\t\tbslCACert: string(serverCACertPEM),\n\t\t\texpectedError: false,\n\t\t\texpectedContent: testContent,\n\t\t},\n\t\t{\n\t\t\tname: \"successful download with valid caCertFile (BSL cert ignored)\",\n\t\t\tcaCertFile: \"\", // Will be set to a valid temp file in test\n\t\t\tbslCACert: \"invalid cert that should be ignored\",\n\t\t\texpectedError: false,\n\t\t\texpectedContent: testContent,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tcaCertFile := tc.caCertFile\n\n\t\t\t// For the last test case, create a valid temp file\n\t\t\tif tc.name == \"successful download with valid caCertFile (BSL cert ignored)\" {\n\t\t\t\ttmpFile, err := os.CreateTemp(t.TempDir(), \"test-ca-cert-*.pem\")\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tdefer os.Remove(tmpFile.Name())\n\n\t\t\t\t_, err = tmpFile.Write(serverCACertPEM)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\ttmpFile.Close()\n\n\t\t\t\tcaCertFile = tmpFile.Name()\n\t\t\t}\n\n\t\t\tvar buf bytes.Buffer\n\t\t\terr := download(\n\t\t\t\tt.Context(),\n\t\t\t\ttlsServer.URL,\n\t\t\t\tvelerov1api.DownloadTargetKindBackupContents,\n\t\t\t\t&buf,\n\t\t\t\tfalse,\n\t\t\t\tcaCertFile,\n\t\t\t\ttc.bslCACert,\n\t\t\t)\n\n\t\t\tif tc.expectedError {\n\t\t\t\trequire.Error(t, err)\n\t\t\t\tif tc.expectedErrContains != \"\" {\n\t\t\t\t\tassert.Contains(t, err.Error(), tc.expectedErrContains)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, tc.expectedContent, buf.String())\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/cmd/util/flag/accessors.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage flag\n\nimport (\n\t\"github.com/spf13/cobra\"\n)\n\n// GetOptionalStringFlag returns the value of the specified flag from a\n// cobra command, or the zero value (\"\") if the flag was not specified.\nfunc GetOptionalStringFlag(cmd *cobra.Command, flagName string) string {\n\ts, _ := cmd.Flags().GetString(flagName)\n\treturn s\n}\n\n// GetOptionalBoolFlag returns the value of the specified flag from a\n// cobra command, or the zero value (false) if the flag was not specified.\nfunc GetOptionalBoolFlag(cmd *cobra.Command, flagName string) bool {\n\tb, _ := cmd.Flags().GetBool(flagName)\n\treturn b\n}\n\n// GetOptionalStringArrayFlag returns the value of the specified flag from a\n// cobra command, or the zero value if the flag was not specified.\nfunc GetOptionalStringArrayFlag(cmd *cobra.Command, flagName string) []string {\n\tf := cmd.Flag(flagName)\n\tif f == nil {\n\t\treturn []string{}\n\t}\n\tv := f.Value.(*StringArray)\n\treturn *v\n}\n" }, { "path": "pkg/cmd/util/flag/accessors_test.go", "content": "package flag\n\nimport (\n\t\"testing\"\n\n\t\"github.com/spf13/cobra\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestGetOptionalStringFlag(t *testing.T) {\n\tflagName := \"flag\"\n\n\t// not specified\n\tcmd := &cobra.Command{}\n\tassert.Empty(t, GetOptionalStringFlag(cmd, flagName))\n\n\t// specified\n\tcmd.Flags().String(flagName, \"value\", \"\")\n\tassert.Equal(t, \"value\", GetOptionalStringFlag(cmd, flagName))\n}\n\nfunc TestGetOptionalBoolFlag(t *testing.T) {\n\tflagName := \"flag\"\n\n\t// not specified\n\tcmd := &cobra.Command{}\n\tassert.False(t, GetOptionalBoolFlag(cmd, flagName))\n\n\t// specified\n\tcmd.Flags().Bool(flagName, true, \"\")\n\tassert.True(t, GetOptionalBoolFlag(cmd, flagName))\n}\n\nfunc TestGetOptionalStringArrayFlag(t *testing.T) {\n\tflagName := \"flag\"\n\n\t// not specified\n\tcmd := &cobra.Command{}\n\tassert.Equal(t, []string{}, GetOptionalStringArrayFlag(cmd, flagName))\n\n\t// specified\n\tvalues := NewStringArray(\"value\")\n\tcmd.Flags().Var(&values, flagName, \"\")\n\tassert.Equal(t, []string{\"value\"}, GetOptionalStringArrayFlag(cmd, flagName))\n}\n" }, { "path": "pkg/cmd/util/flag/array.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage flag\n\nimport (\n\t\"strings\"\n)\n\n// StringArray is a Cobra-compatible named type for defining a\n// string slice flag.\ntype StringArray []string\n\n// NewStringArray returns a StringArray for a provided\n// slice of values.\nfunc NewStringArray(initial ...string) StringArray {\n\treturn StringArray(initial)\n}\n\n// String returns a comma-separated list of the items\n// in the string array.\nfunc (sa *StringArray) String() string {\n\treturn strings.Join(*sa, \",\")\n}\n\n// Set comma-splits the provided string and assigns\n// the results to the receiver. It returns an error if\n// the string is not parseable.\nfunc (sa *StringArray) Set(s string) error {\n\t*sa = strings.Split(s, \",\")\n\treturn nil\n}\n\n// Type returns a string representation of the\n// StringArray type.\nfunc (sa *StringArray) Type() string {\n\treturn \"stringArray\"\n}\n" }, { "path": "pkg/cmd/util/flag/array_test.go", "content": "package flag\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestStringOfStringArray(t *testing.T) {\n\tarray := NewStringArray(\"a\", \"b\")\n\tassert.Equal(t, \"a,b\", array.String())\n}\n\nfunc TestSetOfStringArray(t *testing.T) {\n\tarray := NewStringArray()\n\trequire.NoError(t, array.Set(\"a,b\"))\n\tassert.Equal(t, \"a,b\", array.String())\n}\n\nfunc TestTypeOfStringArray(t *testing.T) {\n\tarray := NewStringArray()\n\tassert.Equal(t, \"stringArray\", array.Type())\n}\n" }, { "path": "pkg/cmd/util/flag/enum.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage flag\n\nimport (\n\t\"github.com/pkg/errors\"\n)\n\n// Enum is a Cobra-compatible wrapper for defining\n// a string flag that can be one of a specified set\n// of values.\ntype Enum struct {\n\tallowedValues []string\n\tvalue string\n}\n\n// NewEnum returns a new enum flag with the specified list\n// of allowed values, and the specified default value if\n// none is set.\nfunc NewEnum(defaultValue string, allowedValues ...string) *Enum {\n\treturn &Enum{\n\t\tallowedValues: allowedValues,\n\t\tvalue: defaultValue,\n\t}\n}\n\n// String returns a string representation of the\n// enum flag.\nfunc (e *Enum) String() string {\n\treturn e.value\n}\n\n// Set assigns the provided string to the enum\n// receiver. It returns an error if the string\n// is not an allowed value.\nfunc (e *Enum) Set(s string) error {\n\tfor _, val := range e.allowedValues {\n\t\tif val == s {\n\t\t\te.value = s\n\t\t\treturn nil\n\t\t}\n\t}\n\n\treturn errors.Errorf(\"invalid value: %q\", s)\n}\n\n// Type returns a string representation of the\n// Enum type.\nfunc (e *Enum) Type() string {\n\t// we don't want the help text to display anything regarding\n\t// the type because the usage text for the flag should capture\n\t// the possible options.\n\treturn \"\"\n}\n\n// AllowedValues returns a slice of the flag's valid\n// values.\nfunc (e *Enum) AllowedValues() []string {\n\treturn e.allowedValues\n}\n" }, { "path": "pkg/cmd/util/flag/enum_test.go", "content": "package flag\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestStringOfEnum(t *testing.T) {\n\tenum := NewEnum(\"a\", \"a\", \"b\", \"c\")\n\tassert.Equal(t, \"a\", enum.String())\n}\n\nfunc TestSetOfEnum(t *testing.T) {\n\tenum := NewEnum(\"a\", \"a\", \"b\", \"c\")\n\trequire.Error(t, enum.Set(\"d\"))\n\n\trequire.NoError(t, enum.Set(\"b\"))\n\tassert.Equal(t, \"b\", enum.String())\n}\n\nfunc TestTypeOfEnum(t *testing.T) {\n\tenum := NewEnum(\"a\", \"a\", \"b\", \"c\")\n\tassert.Empty(t, enum.Type())\n}\n\nfunc TestAllowedValuesOfEnum(t *testing.T) {\n\tenum := NewEnum(\"a\", \"a\", \"b\", \"c\")\n\tassert.Equal(t, []string{\"a\", \"b\", \"c\"}, enum.AllowedValues())\n}\n" }, { "path": "pkg/cmd/util/flag/label_selector_test.go", "content": "package flag\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\nfunc TestStringOfLabelSelector(t *testing.T) {\n\tls, err := metav1.ParseToLabelSelector(\"k1=v1,k2=v2\")\n\trequire.NoError(t, err)\n\tselector := &LabelSelector{\n\t\tLabelSelector: ls,\n\t}\n\tassert.Equal(t, \"k1=v1,k2=v2\", selector.String())\n}\n\nfunc TestSetOfLabelSelector(t *testing.T) {\n\tselector := &LabelSelector{}\n\trequire.NoError(t, selector.Set(\"k1=v1,k2=v2\"))\n\tstr := selector.String()\n\tassert.True(t, str == \"k1=v1,k2=v2\" || str == \"k2=v2,k2=v2\")\n}\n\nfunc TestTypeOfLabelSelector(t *testing.T) {\n\tselector := &LabelSelector{}\n\tassert.Equal(t, \"labelSelector\", selector.Type())\n}\n" }, { "path": "pkg/cmd/util/flag/labelselector.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage flag\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// LabelSelector is a Cobra-compatible wrapper for defining\n// a Kubernetes label-selector flag.\ntype LabelSelector struct {\n\tLabelSelector *metav1.LabelSelector\n}\n\n// String returns a string representation of the label\n// selector flag.\nfunc (ls *LabelSelector) String() string {\n\treturn metav1.FormatLabelSelector(ls.LabelSelector)\n}\n\n// Set parses the provided string and assigns the result\n// to the label-selector receiver. It returns an error if\n// the string is not parseable.\nfunc (ls *LabelSelector) Set(s string) error {\n\tparsed, err := metav1.ParseToLabelSelector(s)\n\tif err != nil {\n\t\treturn err\n\t}\n\tls.LabelSelector = parsed\n\treturn nil\n}\n\n// Type returns a string representation of the\n// LabelSelector type.\nfunc (ls *LabelSelector) Type() string {\n\treturn \"labelSelector\"\n}\n" }, { "path": "pkg/cmd/util/flag/map.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage flag\n\nimport (\n\t\"encoding/csv\"\n\t\"fmt\"\n\t\"strings\"\n\n\t\"github.com/pkg/errors\"\n)\n\n// Map is a Cobra-compatible wrapper for defining a flag containing\n// map data (i.e. a collection of key-value pairs).\ntype Map struct {\n\tdata map[string]string\n\tentryDelimiter rune\n\tkeyValueDelimiter rune\n}\n\n// NewMap returns a Map using the default delimiters ('=' between keys and\n// values, and ',' between map entries, e.g. k1=v1,k2=v2)\nfunc NewMap() Map {\n\tm := Map{\n\t\tdata: make(map[string]string),\n\t}\n\n\treturn m.WithEntryDelimiter(',').WithKeyValueDelimiter('=')\n}\n\n// WithEntryDelimiter sets the delimiter to be used between map\n// entries.\n//\n// For example, in \"k1=v1&k2=v2\", the entry delimiter is '&'\nfunc (m Map) WithEntryDelimiter(delimiter rune) Map {\n\tm.entryDelimiter = delimiter\n\treturn m\n}\n\n// WithKeyValueDelimiter sets the delimiter to be used between\n// keys and values.\n//\n// For example, in \"k1=v1&k2=v2\", the key-value delimiter is '='\nfunc (m Map) WithKeyValueDelimiter(delimiter rune) Map {\n\tm.keyValueDelimiter = delimiter\n\treturn m\n}\n\n// String returns a string representation of the Map flag.\nfunc (m *Map) String() string {\n\tvar a []string\n\tfor k, v := range m.data {\n\t\ta = append(a, fmt.Sprintf(\"%s%s%s\", k, string(m.keyValueDelimiter), v))\n\t}\n\treturn strings.Join(a, string(m.entryDelimiter))\n}\n\n// Set parses the provided string according to the delimiters and\n// assigns the result to the Map receiver. It returns an error if\n// the string is not parseable.\nfunc (m *Map) Set(s string) error {\n\t// use csv.Reader to support parsing input string contains entry delimiters.\n\t// e.g. `\"k1=a=b,c=d\",k2=v2` will be parsed into two parts: `k1=a=b,c=d` and `k2=v2`\n\tr := csv.NewReader(strings.NewReader(s))\n\tr.Comma = m.entryDelimiter\n\tparts, err := r.Read()\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error parsing %q\", s)\n\t}\n\n\tfor _, part := range parts {\n\t\tkvs := strings.SplitN(part, string(m.keyValueDelimiter), 2)\n\t\tif len(kvs) != 2 {\n\t\t\treturn errors.Errorf(\"error parsing %q\", part)\n\t\t}\n\t\tm.data[kvs[0]] = kvs[1]\n\t}\n\treturn nil\n}\n\n// Type returns a string representation of the\n// Map type.\nfunc (m *Map) Type() string {\n\treturn \"mapStringString\"\n}\n\n// Data returns the underlying golang map storing\n// the flag data.\nfunc (m *Map) Data() map[string]string {\n\treturn m.data\n}\n" }, { "path": "pkg/cmd/util/flag/map_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage flag\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestSetOfMap(t *testing.T) {\n\tcases := []struct {\n\t\tname string\n\t\tinput string\n\t\terror bool\n\t\texpected map[string]string\n\t}{\n\t\t{\n\t\t\tname: \"invalid_input_missing_quote\",\n\t\t\tinput: `\"k=v`,\n\t\t\terror: true,\n\t\t},\n\t\t{\n\t\t\tname: \"invalid_input_contains_no_key_value_delimiter\",\n\t\t\tinput: `k`,\n\t\t\terror: true,\n\t\t},\n\t\t{\n\t\t\tname: \"valid input\",\n\t\t\tinput: `k1=v1,k2=v2`,\n\t\t\terror: false,\n\t\t\texpected: map[string]string{\n\t\t\t\t\"k1\": \"v1\",\n\t\t\t\t\"k2\": \"v2\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"valid input whose value contains entry delimiter\",\n\t\t\tinput: `k1=v1,\"k2=a=b,c=d\"`,\n\t\t\terror: false,\n\t\t\texpected: map[string]string{\n\t\t\t\t\"k1\": \"v1\",\n\t\t\t\t\"k2\": \"a=b,c=d\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, c := range cases {\n\t\tt.Run(c.name, func(t *testing.T) {\n\t\t\tm := NewMap()\n\t\t\terr := m.Set(c.input)\n\t\t\tif c.error {\n\t\t\t\trequire.Error(t, err)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tassert.Equal(t, c.expected, m.Data())\n\t\t})\n\t}\n}\n\nfunc TestStringOfMap(t *testing.T) {\n\tm := NewMap()\n\trequire.NoError(t, m.Set(\"k1=v1,k2=v2\"))\n\tstr := m.String()\n\tassert.True(t, str == \"k1=v1,k2=v2\" || str == \"k2=v2,k1=v1\")\n}\n\nfunc TestTypeOfMap(t *testing.T) {\n\tm := NewMap()\n\tassert.Equal(t, \"mapStringString\", m.Type())\n}\n" }, { "path": "pkg/cmd/util/flag/optional_bool.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage flag\n\nimport \"strconv\"\n\ntype OptionalBool struct {\n\tValue *bool\n}\n\nfunc NewOptionalBool(defaultValue *bool) OptionalBool {\n\treturn OptionalBool{\n\t\tValue: defaultValue,\n\t}\n}\n\n// String returns a string representation of the\n// enum flag.\nfunc (f *OptionalBool) String() string {\n\tswitch f.Value {\n\tcase nil:\n\t\treturn \"\"\n\tdefault:\n\t\treturn strconv.FormatBool(*f.Value)\n\t}\n}\n\nfunc (f *OptionalBool) Set(val string) error {\n\tif val == \"\" {\n\t\tf.Value = nil\n\t\treturn nil\n\t}\n\n\tparsed, err := strconv.ParseBool(val)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tf.Value = &parsed\n\n\treturn nil\n}\n\nfunc (f *OptionalBool) Type() string {\n\treturn \"optionalBool\"\n}\n" }, { "path": "pkg/cmd/util/flag/optional_bool_test.go", "content": "package flag\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestStringOfOptionalBool(t *testing.T) {\n\t// nil\n\tob := NewOptionalBool(nil)\n\tassert.Equal(t, \"\", ob.String())\n\n\t// true\n\tb := true\n\tob = NewOptionalBool(&b)\n\tassert.Equal(t, \"true\", ob.String())\n\n\t// false\n\tb = false\n\tob = NewOptionalBool(&b)\n\tassert.Equal(t, \"false\", ob.String())\n}\n\nfunc TestSetOfOptionalBool(t *testing.T) {\n\t// error\n\tob := NewOptionalBool(nil)\n\trequire.Error(t, ob.Set(\"invalid\"))\n\n\t// nil\n\tob = NewOptionalBool(nil)\n\trequire.NoError(t, ob.Set(\"\"))\n\tassert.Nil(t, ob.Value)\n\n\t// true\n\tob = NewOptionalBool(nil)\n\trequire.NoError(t, ob.Set(\"true\"))\n\tassert.True(t, *ob.Value)\n}\n\nfunc TestTypeOfOptionalBool(t *testing.T) {\n\tob := NewOptionalBool(nil)\n\tassert.Equal(t, \"optionalBool\", ob.Type())\n}\n" }, { "path": "pkg/cmd/util/flag/orlabelselector.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage flag\n\nimport (\n\t\"strings\"\n\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// OrLabelSelector is a Cobra-compatible wrapper for defining\n// a Kubernetes or-label-selector flag.\ntype OrLabelSelector struct {\n\tOrLabelSelectors []*metav1.LabelSelector\n}\n\n// String returns a string representation of the or-label\n// selector flag.\nfunc (ls *OrLabelSelector) String() string {\n\torLabels := []string{}\n\tfor _, v := range ls.OrLabelSelectors {\n\t\torLabels = append(orLabels, metav1.FormatLabelSelector(v))\n\t}\n\treturn strings.Join(orLabels, \" or \")\n}\n\n// Set parses the provided string and assigns the result\n// to the or-label-selector receiver. It returns an error if\n// the string is not parseable.\nfunc (ls *OrLabelSelector) Set(s string) error {\n\torItems := strings.Split(s, \" or \")\n\tls.OrLabelSelectors = make([]*metav1.LabelSelector, 0)\n\tfor _, orItem := range orItems {\n\t\tparsed, err := metav1.ParseToLabelSelector(orItem)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tls.OrLabelSelectors = append(ls.OrLabelSelectors, parsed)\n\t}\n\treturn nil\n}\n\n// Type returns a string representation of the\n// OrLabelSelector type.\nfunc (ls *OrLabelSelector) Type() string {\n\treturn \"orLabelSelector\"\n}\n" }, { "path": "pkg/cmd/util/flag/orlabelselector_test.go", "content": "package flag\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\nfunc TestStringOfOrLabelSelector(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\torLabelSelector *OrLabelSelector\n\t\texpectedStr string\n\t}{\n\t\t{\n\t\t\tname: \"or between two labels\",\n\t\t\torLabelSelector: &OrLabelSelector{\n\t\t\t\tOrLabelSelectors: []*metav1.LabelSelector{\n\t\t\t\t\t{\n\t\t\t\t\t\tMatchLabels: map[string]string{\"k1\": \"v1\"},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tMatchLabels: map[string]string{\"k2\": \"v2\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedStr: \"k1=v1 or k2=v2\",\n\t\t},\n\t\t{\n\t\t\tname: \"or between two label groups\",\n\t\t\torLabelSelector: &OrLabelSelector{\n\t\t\t\tOrLabelSelectors: []*metav1.LabelSelector{\n\t\t\t\t\t{\n\t\t\t\t\t\tMatchLabels: map[string]string{\"k1\": \"v1\", \"k2\": \"v2\"},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tMatchLabels: map[string]string{\"a1\": \"b1\", \"a2\": \"b2\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedStr: \"k1=v1,k2=v2 or a1=b1,a2=b2\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tassert.Equal(t, test.expectedStr, test.orLabelSelector.String())\n\t\t})\n\t}\n}\n\nfunc TestSetOfOrLabelSelector(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tinputStr string\n\t\texpectedSelector *OrLabelSelector\n\t}{\n\t\t{\n\t\t\tname: \"or between two labels\",\n\t\t\tinputStr: \"k1=v1 or k2=v2\",\n\t\t\texpectedSelector: &OrLabelSelector{\n\t\t\t\tOrLabelSelectors: []*metav1.LabelSelector{\n\t\t\t\t\t{\n\t\t\t\t\t\tMatchLabels: map[string]string{\"k1\": \"v1\"},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tMatchLabels: map[string]string{\"k2\": \"v2\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"or between two label groups\",\n\t\t\tinputStr: \"k1=v1,k2=v2 or a1=b1,a2=b2\",\n\t\t\texpectedSelector: &OrLabelSelector{\n\t\t\t\tOrLabelSelectors: []*metav1.LabelSelector{\n\t\t\t\t\t{\n\t\t\t\t\t\tMatchLabels: map[string]string{\"k1\": \"v1\", \"k2\": \"v2\"},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tMatchLabels: map[string]string{\"a1\": \"b1\", \"a2\": \"b2\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tselector := &OrLabelSelector{}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\trequire.NoError(t, selector.Set(test.inputStr))\n\t\t\tassert.Len(t, selector.OrLabelSelectors, len(test.expectedSelector.OrLabelSelectors))\n\t\t\tassert.Equal(t, test.expectedSelector.String(), selector.String())\n\t\t})\n\t}\n}\n\nfunc TestTypeOfOrLabelSelector(t *testing.T) {\n\tselector := &OrLabelSelector{}\n\tassert.Equal(t, \"orLabelSelector\", selector.Type())\n}\n" }, { "path": "pkg/cmd/util/output/backup_describer.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage output\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"sort\"\n\t\"strconv\"\n\t\"strings\"\n\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\n\t\"github.com/fatih/color\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tveleroapishared \"github.com/vmware-tanzu/velero/pkg/apis/velero/shared\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/cacert\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/downloadrequest\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/collections\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/results\"\n)\n\n// DescribeBackup describes a backup in human-readable format.\nfunc DescribeBackup(\n\tctx context.Context,\n\tkbClient kbclient.Client,\n\tbackup *velerov1api.Backup,\n\tdeleteRequests []velerov1api.DeleteBackupRequest,\n\tpodVolumeBackups []velerov1api.PodVolumeBackup,\n\tdetails bool,\n\tinsecureSkipTLSVerify bool,\n\tcaCertFile string,\n) string {\n\treturn Describe(func(d *Describer) {\n\t\td.DescribeMetadata(backup.ObjectMeta)\n\n\t\td.Println()\n\t\tphase := backup.Status.Phase\n\t\tif phase == \"\" {\n\t\t\tphase = velerov1api.BackupPhaseNew\n\t\t}\n\t\tphaseString := string(phase)\n\t\tswitch phase {\n\t\tcase velerov1api.BackupPhaseFailedValidation, velerov1api.BackupPhasePartiallyFailed, velerov1api.BackupPhaseFailed:\n\t\t\tphaseString = color.RedString(phaseString)\n\t\tcase velerov1api.BackupPhaseCompleted:\n\t\t\tphaseString = color.GreenString(phaseString)\n\t\tcase velerov1api.BackupPhaseDeleting:\n\t\tcase velerov1api.BackupPhaseWaitingForPluginOperations, velerov1api.BackupPhaseWaitingForPluginOperationsPartiallyFailed:\n\t\tcase velerov1api.BackupPhaseFinalizing, velerov1api.BackupPhaseFinalizingPartiallyFailed:\n\t\tcase velerov1api.BackupPhaseInProgress:\n\t\tcase velerov1api.BackupPhaseNew:\n\t\tcase velerov1api.BackupPhaseQueued, velerov1api.BackupPhaseReadyToStart:\n\t\t}\n\n\t\tlogsNote := \"\"\n\t\tif backup.Status.Phase == velerov1api.BackupPhaseFailed || backup.Status.Phase == velerov1api.BackupPhasePartiallyFailed {\n\t\t\tlogsNote = fmt.Sprintf(\" (run `velero backup logs %s` for more information)\", backup.Name)\n\t\t}\n\n\t\td.Printf(\"Phase:\\t%s%s\\n\", phaseString, logsNote)\n\t\tif phase == velerov1api.BackupPhaseQueued {\n\t\t\td.Printf(\"Queue position:\\t%v\\n\", backup.Status.QueuePosition)\n\t\t}\n\n\t\tif backup.Spec.ResourcePolicy != nil {\n\t\t\td.Println()\n\t\t\tDescribeResourcePolicies(d, backup.Spec.ResourcePolicy)\n\t\t}\n\n\t\tif backup.Spec.UploaderConfig != nil && backup.Spec.UploaderConfig.ParallelFilesUpload > 0 {\n\t\t\td.Println()\n\t\t\tDescribeUploaderConfigForBackup(d, backup.Spec)\n\t\t}\n\n\t\tstatus := backup.Status\n\t\tif len(status.ValidationErrors) > 0 {\n\t\t\td.Println()\n\t\t\td.Printf(\"Validation errors:\")\n\t\t\tfor _, ve := range status.ValidationErrors {\n\t\t\t\td.Printf(\"\\t%s\\n\", color.RedString(ve))\n\t\t\t}\n\t\t}\n\n\t\td.Println()\n\t\tDescribeBackupResults(ctx, kbClient, d, backup, insecureSkipTLSVerify, caCertFile)\n\n\t\td.Println()\n\t\tDescribeBackupSpec(d, backup.Spec)\n\n\t\td.Println()\n\t\tDescribeBackupStatus(ctx, kbClient, d, backup, details, insecureSkipTLSVerify, caCertFile, podVolumeBackups)\n\n\t\tif len(deleteRequests) > 0 {\n\t\t\td.Println()\n\t\t\tDescribeDeleteBackupRequests(d, deleteRequests)\n\t\t}\n\t})\n}\n\n// DescribeResourcePolicies describes resource policies in human-readable format\nfunc DescribeResourcePolicies(d *Describer, resPolicies *corev1api.TypedLocalObjectReference) {\n\td.Printf(\"Resource policies:\\n\")\n\td.Printf(\"\\tType:\\t%s\\n\", resPolicies.Kind)\n\td.Printf(\"\\tName:\\t%s\\n\", resPolicies.Name)\n}\n\n// DescribeUploaderConfigForBackup describes uploader config in human-readable format\nfunc DescribeUploaderConfigForBackup(d *Describer, spec velerov1api.BackupSpec) {\n\td.Printf(\"Uploader config:\\n\")\n\td.Printf(\"\\tParallel files upload:\\t%d\\n\", spec.UploaderConfig.ParallelFilesUpload)\n}\n\n// DescribeBackupSpec describes a backup spec in human-readable format.\nfunc DescribeBackupSpec(d *Describer, spec velerov1api.BackupSpec) {\n\t// TODO make a helper for this and use it in all the describers.\n\td.Printf(\"Namespaces:\\n\")\n\tvar s string\n\tif len(spec.IncludedNamespaces) == 0 {\n\t\ts = \"*\"\n\t} else {\n\t\ts = strings.Join(spec.IncludedNamespaces, \", \")\n\t}\n\td.Printf(\"\\tIncluded:\\t%s\\n\", s)\n\tif len(spec.ExcludedNamespaces) == 0 {\n\t\ts = emptyDisplay\n\t} else {\n\t\ts = strings.Join(spec.ExcludedNamespaces, \", \")\n\t}\n\td.Printf(\"\\tExcluded:\\t%s\\n\", s)\n\n\td.Println()\n\td.Printf(\"Resources:\\n\")\n\tif collections.UseOldResourceFilters(spec) {\n\t\tif len(spec.IncludedResources) == 0 {\n\t\t\ts = \"*\"\n\t\t} else {\n\t\t\ts = strings.Join(spec.IncludedResources, \", \")\n\t\t}\n\t\td.Printf(\"\\tIncluded:\\t%s\\n\", s)\n\t\tif len(spec.ExcludedResources) == 0 {\n\t\t\ts = emptyDisplay\n\t\t} else {\n\t\t\ts = strings.Join(spec.ExcludedResources, \", \")\n\t\t}\n\t\td.Printf(\"\\tExcluded:\\t%s\\n\", s)\n\t\td.Printf(\"\\tCluster-scoped:\\t%s\\n\", BoolPointerString(spec.IncludeClusterResources, \"excluded\", \"included\", \"auto\"))\n\t} else {\n\t\tif len(spec.IncludedClusterScopedResources) == 0 {\n\t\t\ts = emptyDisplay\n\t\t} else {\n\t\t\ts = strings.Join(spec.IncludedClusterScopedResources, \", \")\n\t\t}\n\t\td.Printf(\"\\tIncluded cluster-scoped:\\t%s\\n\", s)\n\t\tif len(spec.ExcludedClusterScopedResources) == 0 {\n\t\t\ts = emptyDisplay\n\t\t} else {\n\t\t\ts = strings.Join(spec.ExcludedClusterScopedResources, \", \")\n\t\t}\n\t\td.Printf(\"\\tExcluded cluster-scoped:\\t%s\\n\", s)\n\n\t\tif len(spec.IncludedNamespaceScopedResources) == 0 {\n\t\t\ts = \"*\"\n\t\t} else {\n\t\t\ts = strings.Join(spec.IncludedNamespaceScopedResources, \", \")\n\t\t}\n\t\td.Printf(\"\\tIncluded namespace-scoped:\\t%s\\n\", s)\n\t\tif len(spec.ExcludedNamespaceScopedResources) == 0 {\n\t\t\ts = emptyDisplay\n\t\t} else {\n\t\t\ts = strings.Join(spec.ExcludedNamespaceScopedResources, \", \")\n\t\t}\n\t\td.Printf(\"\\tExcluded namespace-scoped:\\t%s\\n\", s)\n\t}\n\n\td.Println()\n\ts = emptyDisplay\n\tif spec.LabelSelector != nil {\n\t\ts = metav1.FormatLabelSelector(spec.LabelSelector)\n\t}\n\td.Printf(\"Label selector:\\t%s\\n\", s)\n\n\td.Println()\n\tif len(spec.OrLabelSelectors) == 0 {\n\t\ts = emptyDisplay\n\t} else {\n\t\torLabelSelectors := []string{}\n\t\tfor _, v := range spec.OrLabelSelectors {\n\t\t\torLabelSelectors = append(orLabelSelectors, metav1.FormatLabelSelector(v))\n\t\t}\n\t\ts = strings.Join(orLabelSelectors, \" or \")\n\t}\n\td.Printf(\"Or label selector:\\t%s\\n\", s)\n\n\td.Println()\n\td.Printf(\"Storage Location:\\t%s\\n\", spec.StorageLocation)\n\n\td.Println()\n\td.Printf(\"Velero-Native Snapshot PVs:\\t%s\\n\", BoolPointerString(spec.SnapshotVolumes, \"false\", \"true\", \"auto\"))\n\tif spec.DefaultVolumesToFsBackup != nil {\n\t\td.Printf(\"File System Backup (Default):\\t%s\\n\", BoolPointerString(spec.DefaultVolumesToFsBackup, \"false\", \"true\", \"\"))\n\t}\n\td.Printf(\"Snapshot Move Data:\\t%s\\n\", BoolPointerString(spec.SnapshotMoveData, \"false\", \"true\", \"auto\"))\n\tif len(spec.DataMover) == 0 {\n\t\ts = defaultDataMover\n\t} else {\n\t\ts = spec.DataMover\n\t}\n\td.Printf(\"Data Mover:\\t%s\\n\", s)\n\n\td.Println()\n\td.Printf(\"TTL:\\t%s\\n\", spec.TTL.Duration)\n\n\td.Println()\n\td.Printf(\"CSISnapshotTimeout:\\t%s\\n\", spec.CSISnapshotTimeout.Duration)\n\td.Printf(\"ItemOperationTimeout:\\t%s\\n\", spec.ItemOperationTimeout.Duration)\n\n\td.Println()\n\tif len(spec.Hooks.Resources) == 0 {\n\t\td.Printf(\"Hooks:\\t\" + emptyDisplay + \"\\n\")\n\t} else {\n\t\td.Printf(\"Hooks:\\n\")\n\t\td.Printf(\"\\tResources:\\n\")\n\t\tfor _, backupResourceHookSpec := range spec.Hooks.Resources {\n\t\t\td.Printf(\"\\t\\t%s:\\n\", backupResourceHookSpec.Name)\n\t\t\td.Printf(\"\\t\\t\\tNamespaces:\\n\")\n\t\t\tvar s string\n\t\t\tif len(backupResourceHookSpec.IncludedNamespaces) == 0 {\n\t\t\t\ts = \"*\"\n\t\t\t} else {\n\t\t\t\ts = strings.Join(backupResourceHookSpec.IncludedNamespaces, \", \")\n\t\t\t}\n\t\t\td.Printf(\"\\t\\t\\t\\tIncluded:\\t%s\\n\", s)\n\t\t\tif len(backupResourceHookSpec.ExcludedNamespaces) == 0 {\n\t\t\t\ts = emptyDisplay\n\t\t\t} else {\n\t\t\t\ts = strings.Join(backupResourceHookSpec.ExcludedNamespaces, \", \")\n\t\t\t}\n\t\t\td.Printf(\"\\t\\t\\t\\tExcluded:\\t%s\\n\", s)\n\n\t\t\td.Println()\n\t\t\td.Printf(\"\\t\\t\\tResources:\\n\")\n\t\t\tif len(backupResourceHookSpec.IncludedResources) == 0 {\n\t\t\t\ts = \"*\"\n\t\t\t} else {\n\t\t\t\ts = strings.Join(backupResourceHookSpec.IncludedResources, \", \")\n\t\t\t}\n\t\t\td.Printf(\"\\t\\t\\t\\tIncluded:\\t%s\\n\", s)\n\t\t\tif len(backupResourceHookSpec.ExcludedResources) == 0 {\n\t\t\t\ts = emptyDisplay\n\t\t\t} else {\n\t\t\t\ts = strings.Join(backupResourceHookSpec.ExcludedResources, \", \")\n\t\t\t}\n\t\t\td.Printf(\"\\t\\t\\t\\tExcluded:\\t%s\\n\", s)\n\n\t\t\td.Println()\n\t\t\ts = emptyDisplay\n\t\t\tif backupResourceHookSpec.LabelSelector != nil {\n\t\t\t\ts = metav1.FormatLabelSelector(backupResourceHookSpec.LabelSelector)\n\t\t\t}\n\t\t\td.Printf(\"\\t\\t\\tLabel selector:\\t%s\\n\", s)\n\n\t\t\tfor _, hook := range backupResourceHookSpec.PreHooks {\n\t\t\t\tif hook.Exec != nil {\n\t\t\t\t\td.Println()\n\t\t\t\t\td.Printf(\"\\t\\t\\tPre Exec Hook:\\n\")\n\t\t\t\t\td.Printf(\"\\t\\t\\t\\tContainer:\\t%s\\n\", hook.Exec.Container)\n\t\t\t\t\td.Printf(\"\\t\\t\\t\\tCommand:\\t%s\\n\", strings.Join(hook.Exec.Command, \" \"))\n\t\t\t\t\td.Printf(\"\\t\\t\\t\\tOn Error:\\t%s\\n\", hook.Exec.OnError)\n\t\t\t\t\td.Printf(\"\\t\\t\\t\\tTimeout:\\t%s\\n\", hook.Exec.Timeout.Duration)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tfor _, hook := range backupResourceHookSpec.PostHooks {\n\t\t\t\tif hook.Exec != nil {\n\t\t\t\t\td.Println()\n\t\t\t\t\td.Printf(\"\\t\\t\\tPost Exec Hook:\\n\")\n\t\t\t\t\td.Printf(\"\\t\\t\\t\\tContainer:\\t%s\\n\", hook.Exec.Container)\n\t\t\t\t\td.Printf(\"\\t\\t\\t\\tCommand:\\t%s\\n\", strings.Join(hook.Exec.Command, \" \"))\n\t\t\t\t\td.Printf(\"\\t\\t\\t\\tOn Error:\\t%s\\n\", hook.Exec.OnError)\n\t\t\t\t\td.Printf(\"\\t\\t\\t\\tTimeout:\\t%s\\n\", hook.Exec.Timeout.Duration)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\tif spec.OrderedResources != nil {\n\t\td.Println()\n\t\td.Printf(\"OrderedResources:\\n\")\n\t\tfor key, value := range spec.OrderedResources {\n\t\t\td.Printf(\"\\t%s: %s\\n\", key, value)\n\t\t}\n\t}\n}\n\n// DescribeBackupStatus describes a backup status in human-readable format.\nfunc DescribeBackupStatus(ctx context.Context,\n\tkbClient kbclient.Client,\n\td *Describer,\n\tbackup *velerov1api.Backup,\n\tdetails bool,\n\tinsecureSkipTLSVerify bool,\n\tcaCertPath string,\n\tpodVolumeBackups []velerov1api.PodVolumeBackup) {\n\tstatus := backup.Status\n\n\t// Status.Version has been deprecated, use Status.FormatVersion\n\td.Printf(\"Backup Format Version:\\t%s\\n\", status.FormatVersion)\n\n\td.Println()\n\t// \"\" output should only be applicable for backups that failed validation\n\tif status.StartTimestamp == nil || status.StartTimestamp.Time.IsZero() {\n\t\td.Printf(\"Started:\\t%s\\n\", \"\")\n\t} else {\n\t\td.Printf(\"Started:\\t%s\\n\", status.StartTimestamp.Time)\n\t}\n\tif status.CompletionTimestamp == nil || status.CompletionTimestamp.Time.IsZero() {\n\t\td.Printf(\"Completed:\\t%s\\n\", \"\")\n\t} else {\n\t\td.Printf(\"Completed:\\t%s\\n\", status.CompletionTimestamp.Time)\n\t}\n\n\td.Println()\n\t// Expiration can't be 0, it is always set to a 30-day default. It can be nil\n\t// if the controller hasn't processed this Backup yet, in which case this will\n\t// just display ``, though this should be temporary.\n\td.Printf(\"Expiration:\\t%s\\n\", status.Expiration)\n\td.Println()\n\n\tif backup.Status.Progress != nil {\n\t\tif backup.Status.Phase == velerov1api.BackupPhaseInProgress {\n\t\t\td.Printf(\"Estimated total items to be backed up:\\t%d\\n\", backup.Status.Progress.TotalItems)\n\t\t\td.Printf(\"Items backed up so far:\\t%d\\n\", backup.Status.Progress.ItemsBackedUp)\n\t\t} else {\n\t\t\td.Printf(\"Total items to be backed up:\\t%d\\n\", backup.Status.Progress.TotalItems)\n\t\t\td.Printf(\"Items backed up:\\t%d\\n\", backup.Status.Progress.ItemsBackedUp)\n\t\t}\n\n\t\td.Println()\n\t}\n\n\tdescribeBackupItemOperations(ctx, kbClient, d, backup, details, insecureSkipTLSVerify, caCertPath)\n\n\tif details {\n\t\tdescribeBackupResourceList(ctx, kbClient, d, backup, insecureSkipTLSVerify, caCertPath)\n\t\td.Println()\n\t}\n\n\tdescribeBackupVolumes(ctx, kbClient, d, backup, details, insecureSkipTLSVerify, caCertPath, podVolumeBackups)\n\n\tif status.HookStatus != nil {\n\t\td.Println()\n\t\td.Printf(\"HooksAttempted:\\t%d\\n\", status.HookStatus.HooksAttempted)\n\t\td.Printf(\"HooksFailed:\\t%d\\n\", status.HookStatus.HooksFailed)\n\t}\n}\n\nfunc describeBackupItemOperations(ctx context.Context, kbClient kbclient.Client, d *Describer, backup *velerov1api.Backup, details bool, insecureSkipTLSVerify bool, caCertPath string) {\n\tstatus := backup.Status\n\tif status.BackupItemOperationsAttempted > 0 {\n\t\tif !details {\n\t\t\td.Printf(\"Backup Item Operations:\\t%d of %d completed successfully, %d failed (specify --details for more information)\\n\", status.BackupItemOperationsCompleted, status.BackupItemOperationsAttempted, status.BackupItemOperationsFailed)\n\t\t\treturn\n\t\t}\n\n\t\t// Get BSL cacert if available\n\t\tbslCACert, err := cacert.GetCACertFromBackup(ctx, kbClient, backup.Namespace, backup)\n\t\tif err != nil {\n\t\t\t// Log the error but don't fail - we can still try to download without the BSL cacert\n\t\t\td.Printf(\"WARNING: Error getting cacert from BSL: %v\\n\", err)\n\t\t\tbslCACert = \"\"\n\t\t}\n\n\t\tbuf := new(bytes.Buffer)\n\t\tif err := downloadrequest.StreamWithBSLCACert(ctx, kbClient, backup.Namespace, backup.Name, velerov1api.DownloadTargetKindBackupItemOperations, buf, downloadRequestTimeout, insecureSkipTLSVerify, caCertPath, bslCACert); err != nil {\n\t\t\td.Printf(\"Backup Item Operations:\\t\\n\", err)\n\t\t\treturn\n\t\t}\n\n\t\tvar operations []*itemoperation.BackupOperation\n\t\tif err := json.NewDecoder(buf).Decode(&operations); err != nil {\n\t\t\td.Printf(\"Backup Item Operations:\\t\\n\", err)\n\t\t\treturn\n\t\t}\n\n\t\td.Printf(\"Backup Item Operations:\\n\")\n\t\tfor _, operation := range operations {\n\t\t\tdescribeBackupItemOperation(d, operation)\n\t\t}\n\t}\n}\n\nfunc describeBackupResourceList(ctx context.Context, kbClient kbclient.Client, d *Describer, backup *velerov1api.Backup, insecureSkipTLSVerify bool, caCertPath string) {\n\t// Get BSL cacert if available\n\tbslCACert, err := cacert.GetCACertFromBackup(ctx, kbClient, backup.Namespace, backup)\n\tif err != nil {\n\t\t// Log the error but don't fail - we can still try to download without the BSL cacert\n\t\td.Printf(\"WARNING: Error getting cacert from BSL: %v\\n\", err)\n\t\tbslCACert = \"\"\n\t}\n\n\tbuf := new(bytes.Buffer)\n\tif err := downloadrequest.StreamWithBSLCACert(ctx, kbClient, backup.Namespace, backup.Name, velerov1api.DownloadTargetKindBackupResourceList, buf, downloadRequestTimeout, insecureSkipTLSVerify, caCertPath, bslCACert); err != nil {\n\t\tif err == downloadrequest.ErrNotFound {\n\t\t\t// the backup resource list could be missing if (other reasons may exist as well):\n\t\t\t//\t- the backup was taken prior to v1.1; or\n\t\t\t//\t- the backup hasn't completed yet; or\n\t\t\t//\t- there was an error uploading the file; or\n\t\t\t//\t- the file was manually deleted after upload\n\t\t\td.Println(\"Resource List:\\t\")\n\t\t} else {\n\t\t\td.Printf(\"Resource List:\\t\\n\", err)\n\t\t}\n\t\treturn\n\t}\n\n\tvar resourceList map[string][]string\n\tif err := json.NewDecoder(buf).Decode(&resourceList); err != nil {\n\t\td.Printf(\"Resource List:\\t\\n\", err)\n\t\treturn\n\t}\n\n\td.Println(\"Resource List:\")\n\n\t// Sort GVKs in output\n\tgvks := make([]string, 0, len(resourceList))\n\tfor gvk := range resourceList {\n\t\tgvks = append(gvks, gvk)\n\t}\n\tsort.Strings(gvks)\n\n\tfor _, gvk := range gvks {\n\t\td.Printf(\"\\t%s:\\n\\t\\t- %s\\n\", gvk, strings.Join(resourceList[gvk], \"\\n\\t\\t- \"))\n\t}\n}\n\nfunc describeBackupVolumes(\n\tctx context.Context,\n\tkbClient kbclient.Client,\n\td *Describer,\n\tbackup *velerov1api.Backup,\n\tdetails bool,\n\tinsecureSkipTLSVerify bool,\n\tcaCertPath string,\n\tpodVolumeBackupCRs []velerov1api.PodVolumeBackup,\n) {\n\td.Println(\"Backup Volumes:\")\n\n\t// Get BSL cacert if available\n\tbslCACert, err := cacert.GetCACertFromBackup(ctx, kbClient, backup.Namespace, backup)\n\tif err != nil {\n\t\t// Log the error but don't fail - we can still try to download without the BSL cacert\n\t\td.Printf(\"WARNING: Error getting cacert from BSL: %v\\n\", err)\n\t\tbslCACert = \"\"\n\t}\n\n\tnativeSnapshots := []*volume.BackupVolumeInfo{}\n\tcsiSnapshots := []*volume.BackupVolumeInfo{}\n\tlegacyInfoSource := false\n\n\tbuf := new(bytes.Buffer)\n\terr = downloadrequest.StreamWithBSLCACert(ctx, kbClient, backup.Namespace, backup.Name, velerov1api.DownloadTargetKindBackupVolumeInfos, buf, downloadRequestTimeout, insecureSkipTLSVerify, caCertPath, bslCACert)\n\tif err == downloadrequest.ErrNotFound {\n\t\tnativeSnapshots, err = retrieveNativeSnapshotLegacy(ctx, kbClient, backup, insecureSkipTLSVerify, caCertPath, bslCACert)\n\t\tif err != nil {\n\t\t\td.Printf(\"\\t\\n\", err)\n\t\t\treturn\n\t\t}\n\n\t\tcsiSnapshots, err = retrieveCSISnapshotLegacy(ctx, kbClient, backup, insecureSkipTLSVerify, caCertPath, bslCACert)\n\t\tif err != nil {\n\t\t\td.Printf(\"\\t\\n\", err)\n\t\t\treturn\n\t\t}\n\n\t\tlegacyInfoSource = true\n\t} else if err != nil {\n\t\td.Printf(\"\\t\\n\", err)\n\t\treturn\n\t} else {\n\t\tvar volumeInfos []volume.BackupVolumeInfo\n\t\tif err := json.NewDecoder(buf).Decode(&volumeInfos); err != nil {\n\t\t\td.Printf(\"\\t\\n\", err)\n\t\t\treturn\n\t\t}\n\n\t\tfor i := range volumeInfos {\n\t\t\tswitch volumeInfos[i].BackupMethod {\n\t\t\tcase volume.NativeSnapshot:\n\t\t\t\tnativeSnapshots = append(nativeSnapshots, &volumeInfos[i])\n\t\t\tcase volume.CSISnapshot:\n\t\t\t\tcsiSnapshots = append(csiSnapshots, &volumeInfos[i])\n\t\t\t}\n\t\t}\n\t}\n\n\tdescribeNativeSnapshots(d, details, nativeSnapshots)\n\td.Println()\n\n\tdescribeCSISnapshots(d, details, csiSnapshots, legacyInfoSource)\n\td.Println()\n\n\tdescribePodVolumeBackups(d, details, podVolumeBackupCRs)\n}\n\nfunc retrieveNativeSnapshotLegacy(ctx context.Context, kbClient kbclient.Client, backup *velerov1api.Backup, insecureSkipTLSVerify bool, caCertPath string, bslCACert string) ([]*volume.BackupVolumeInfo, error) {\n\tstatus := backup.Status\n\tnativeSnapshots := []*volume.BackupVolumeInfo{}\n\n\tif status.VolumeSnapshotsAttempted == 0 {\n\t\treturn nativeSnapshots, nil\n\t}\n\n\tbuf := new(bytes.Buffer)\n\tif err := downloadrequest.StreamWithBSLCACert(ctx, kbClient, backup.Namespace, backup.Name, velerov1api.DownloadTargetKindBackupVolumeSnapshots, buf, downloadRequestTimeout, insecureSkipTLSVerify, caCertPath, bslCACert); err != nil {\n\t\treturn nativeSnapshots, errors.Wrapf(err, \"error to download native snapshot info\")\n\t}\n\n\tvar snapshots []*volume.Snapshot\n\tif err := json.NewDecoder(buf).Decode(&snapshots); err != nil {\n\t\treturn nativeSnapshots, errors.Wrapf(err, \"error to decode native snapshot info\")\n\t}\n\n\tfor _, snap := range snapshots {\n\t\tvolumeInfo := volume.BackupVolumeInfo{\n\t\t\tPVName: snap.Spec.PersistentVolumeName,\n\t\t\tNativeSnapshotInfo: &volume.NativeSnapshotInfo{\n\t\t\t\tSnapshotHandle: snap.Status.ProviderSnapshotID,\n\t\t\t\tVolumeType: snap.Spec.VolumeType,\n\t\t\t\tVolumeAZ: snap.Spec.VolumeAZ,\n\t\t\t},\n\t\t}\n\n\t\tif snap.Spec.VolumeIOPS != nil {\n\t\t\tvolumeInfo.NativeSnapshotInfo.IOPS = strconv.FormatInt(*snap.Spec.VolumeIOPS, 10)\n\t\t}\n\n\t\tnativeSnapshots = append(nativeSnapshots, &volumeInfo)\n\t}\n\n\treturn nativeSnapshots, nil\n}\n\nfunc retrieveCSISnapshotLegacy(ctx context.Context, kbClient kbclient.Client, backup *velerov1api.Backup, insecureSkipTLSVerify bool, caCertPath string, bslCACert string) ([]*volume.BackupVolumeInfo, error) {\n\tstatus := backup.Status\n\tcsiSnapshots := []*volume.BackupVolumeInfo{}\n\n\tif status.CSIVolumeSnapshotsAttempted == 0 {\n\t\treturn csiSnapshots, nil\n\t}\n\n\tvsBuf := new(bytes.Buffer)\n\terr := downloadrequest.StreamWithBSLCACert(ctx, kbClient, backup.Namespace, backup.Name, velerov1api.DownloadTargetKindCSIBackupVolumeSnapshots, vsBuf, downloadRequestTimeout, insecureSkipTLSVerify, caCertPath, bslCACert)\n\tif err != nil {\n\t\treturn csiSnapshots, errors.Wrapf(err, \"error to download vs list\")\n\t}\n\n\tvar vsList []snapshotv1api.VolumeSnapshot\n\tif err := json.NewDecoder(vsBuf).Decode(&vsList); err != nil {\n\t\treturn csiSnapshots, errors.Wrapf(err, \"error to decode vs list\")\n\t}\n\n\tvscBuf := new(bytes.Buffer)\n\terr = downloadrequest.StreamWithBSLCACert(ctx, kbClient, backup.Namespace, backup.Name, velerov1api.DownloadTargetKindCSIBackupVolumeSnapshotContents, vscBuf, downloadRequestTimeout, insecureSkipTLSVerify, caCertPath, bslCACert)\n\tif err != nil {\n\t\treturn csiSnapshots, errors.Wrapf(err, \"error to download vsc list\")\n\t}\n\n\tvar vscList []snapshotv1api.VolumeSnapshotContent\n\tif err := json.NewDecoder(vscBuf).Decode(&vscList); err != nil {\n\t\treturn csiSnapshots, errors.Wrapf(err, \"error to decode vsc list\")\n\t}\n\n\tfor _, vsc := range vscList {\n\t\tvolInfo := volume.BackupVolumeInfo{\n\t\t\tPreserveLocalSnapshot: true,\n\t\t\tCSISnapshotInfo: &volume.CSISnapshotInfo{\n\t\t\t\tVSCName: vsc.Name,\n\t\t\t\tDriver: vsc.Spec.Driver,\n\t\t\t},\n\t\t}\n\n\t\tif vsc.Status != nil && vsc.Status.SnapshotHandle != nil {\n\t\t\tvolInfo.CSISnapshotInfo.SnapshotHandle = *vsc.Status.SnapshotHandle\n\t\t}\n\n\t\tif vsc.Status != nil && vsc.Status.RestoreSize != nil {\n\t\t\tvolInfo.CSISnapshotInfo.Size = *vsc.Status.RestoreSize\n\t\t}\n\n\t\tfor _, vs := range vsList {\n\t\t\tif vs.Status.BoundVolumeSnapshotContentName == nil {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tif vs.Spec.Source.PersistentVolumeClaimName == nil {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tif *vs.Status.BoundVolumeSnapshotContentName == vsc.Name {\n\t\t\t\tvolInfo.PVCName = *vs.Spec.Source.PersistentVolumeClaimName\n\t\t\t\tvolInfo.PVCNamespace = vs.Namespace\n\t\t\t}\n\t\t}\n\n\t\tif volInfo.PVCName == \"\" {\n\t\t\tvolInfo.PVCName = \"\"\n\t\t}\n\n\t\tcsiSnapshots = append(csiSnapshots, &volInfo)\n\t}\n\n\treturn csiSnapshots, nil\n}\n\nfunc describeNativeSnapshots(d *Describer, details bool, infos []*volume.BackupVolumeInfo) {\n\tif len(infos) == 0 {\n\t\td.Printf(\"\\tVelero-Native Snapshots: \\n\")\n\t\treturn\n\t}\n\n\td.Println(\"\\tVelero-Native Snapshots:\")\n\tfor _, info := range infos {\n\t\tdescribNativeSnapshot(d, details, info)\n\t}\n}\n\nfunc describNativeSnapshot(d *Describer, details bool, info *volume.BackupVolumeInfo) {\n\tif details {\n\t\td.Printf(\"\\t\\t%s:\\n\", info.PVName)\n\t\td.Printf(\"\\t\\t\\tSnapshot ID:\\t%s\\n\", info.NativeSnapshotInfo.SnapshotHandle)\n\t\td.Printf(\"\\t\\t\\tType:\\t%s\\n\", info.NativeSnapshotInfo.VolumeType)\n\t\td.Printf(\"\\t\\t\\tAvailability Zone:\\t%s\\n\", info.NativeSnapshotInfo.VolumeAZ)\n\t\td.Printf(\"\\t\\t\\tIOPS:\\t%s\\n\", info.NativeSnapshotInfo.IOPS)\n\t\td.Printf(\"\\t\\t\\tResult:\\t%s\\n\", info.Result)\n\t} else {\n\t\td.Printf(\"\\t\\t%s: specify --details for more information\\n\", info.PVName)\n\t}\n}\n\nfunc describeCSISnapshots(d *Describer, details bool, infos []*volume.BackupVolumeInfo, legacyInfoSource bool) {\n\tif len(infos) == 0 {\n\t\tif legacyInfoSource {\n\t\t\td.Printf(\"\\tCSI Snapshots: \\n\")\n\t\t} else {\n\t\t\td.Printf(\"\\tCSI Snapshots: \\n\")\n\t\t}\n\t\treturn\n\t}\n\n\td.Println(\"\\tCSI Snapshots:\")\n\tfor _, info := range infos {\n\t\tdescribeCSISnapshot(d, details, info)\n\t}\n}\n\nfunc describeCSISnapshot(d *Describer, details bool, info *volume.BackupVolumeInfo) {\n\td.Printf(\"\\t\\t%s:\\n\", fmt.Sprintf(\"%s/%s\", info.PVCNamespace, info.PVCName))\n\n\tdescribeLocalSnapshot(d, details, info)\n\tdescribeDataMovement(d, details, info)\n}\n\nfunc describeLocalSnapshot(d *Describer, details bool, info *volume.BackupVolumeInfo) {\n\tif !info.PreserveLocalSnapshot {\n\t\treturn\n\t}\n\n\tif details {\n\t\td.Printf(\"\\t\\t\\tSnapshot:\\n\")\n\t\tif !info.SnapshotDataMoved && info.CSISnapshotInfo.OperationID != \"\" {\n\t\t\td.Printf(\"\\t\\t\\t\\tOperation ID: %s\\n\", info.CSISnapshotInfo.OperationID)\n\t\t}\n\n\t\td.Printf(\"\\t\\t\\t\\tSnapshot Content Name: %s\\n\", info.CSISnapshotInfo.VSCName)\n\t\td.Printf(\"\\t\\t\\t\\tStorage Snapshot ID: %s\\n\", info.CSISnapshotInfo.SnapshotHandle)\n\t\td.Printf(\"\\t\\t\\t\\tSnapshot Size (bytes): %d\\n\", info.CSISnapshotInfo.Size)\n\t\td.Printf(\"\\t\\t\\t\\tCSI Driver: %s\\n\", info.CSISnapshotInfo.Driver)\n\t\td.Printf(\"\\t\\t\\t\\tResult: %s\\n\", info.Result)\n\t} else {\n\t\td.Printf(\"\\t\\t\\tSnapshot: %s\\n\", \"included, specify --details for more information\")\n\t}\n}\n\nfunc describeDataMovement(d *Describer, details bool, info *volume.BackupVolumeInfo) {\n\tif !info.SnapshotDataMoved {\n\t\treturn\n\t}\n\n\tif details {\n\t\td.Printf(\"\\t\\t\\tData Movement:\\n\")\n\t\td.Printf(\"\\t\\t\\t\\tOperation ID: %s\\n\", info.SnapshotDataMovementInfo.OperationID)\n\n\t\tdataMover := \"velero\"\n\t\tif info.SnapshotDataMovementInfo.DataMover != \"\" {\n\t\t\tdataMover = info.SnapshotDataMovementInfo.DataMover\n\t\t}\n\t\td.Printf(\"\\t\\t\\t\\tData Mover: %s\\n\", dataMover)\n\t\td.Printf(\"\\t\\t\\t\\tUploader Type: %s\\n\", info.SnapshotDataMovementInfo.UploaderType)\n\t\td.Printf(\"\\t\\t\\t\\tMoved data Size (bytes): %d\\n\", info.SnapshotDataMovementInfo.Size)\n\t\tif info.SnapshotDataMovementInfo.IncrementalSize > 0 {\n\t\t\td.Printf(\"\\t\\t\\t\\tIncremental data Size (bytes): %d\\n\", info.SnapshotDataMovementInfo.IncrementalSize)\n\t\t}\n\t\td.Printf(\"\\t\\t\\t\\tResult: %s\\n\", info.Result)\n\t} else {\n\t\td.Printf(\"\\t\\t\\tData Movement: %s\\n\", \"included, specify --details for more information\")\n\t}\n}\n\nfunc describeBackupItemOperation(d *Describer, operation *itemoperation.BackupOperation) {\n\td.Printf(\"\\tOperation for %s %s/%s:\\n\", operation.Spec.ResourceIdentifier, operation.Spec.ResourceIdentifier.Namespace, operation.Spec.ResourceIdentifier.Name)\n\td.Printf(\"\\t\\tBackup Item Action Plugin:\\t%s\\n\", operation.Spec.BackupItemAction)\n\td.Printf(\"\\t\\tOperation ID:\\t%s\\n\", operation.Spec.OperationID)\n\tif len(operation.Spec.PostOperationItems) > 0 {\n\t\td.Printf(\"\\t\\tItems to Update:\\n\")\n\t}\n\tfor _, item := range operation.Spec.PostOperationItems {\n\t\td.Printf(\"\\t\\t\\t%s %s/%s\\n\", item, item.Namespace, item.Name)\n\t}\n\td.Printf(\"\\t\\tPhase:\\t%s\\n\", operation.Status.Phase)\n\tif operation.Status.Error != \"\" {\n\t\td.Printf(\"\\t\\tOperation Error:\\t%s\\n\", operation.Status.Error)\n\t}\n\tif operation.Status.NTotal > 0 || operation.Status.NCompleted > 0 {\n\t\td.Printf(\"\\t\\tProgress:\\t%v of %v complete (%s)\\n\",\n\t\t\toperation.Status.NCompleted,\n\t\t\toperation.Status.NTotal,\n\t\t\toperation.Status.OperationUnits)\n\t}\n\tif operation.Status.Description != \"\" {\n\t\td.Printf(\"\\t\\tProgress description:\\t%s\\n\", operation.Status.Description)\n\t}\n\tif operation.Status.Created != nil {\n\t\td.Printf(\"\\t\\tCreated:\\t%s\\n\", operation.Status.Created.String())\n\t}\n\tif operation.Status.Started != nil {\n\t\td.Printf(\"\\t\\tStarted:\\t%s\\n\", operation.Status.Started.String())\n\t}\n\tif operation.Status.Updated != nil {\n\t\td.Printf(\"\\t\\tUpdated:\\t%s\\n\", operation.Status.Updated.String())\n\t}\n}\n\n// DescribeDeleteBackupRequests describes delete backup requests in human-readable format.\nfunc DescribeDeleteBackupRequests(d *Describer, requests []velerov1api.DeleteBackupRequest) {\n\td.Printf(\"Deletion Attempts\")\n\tif count := failedDeletionCount(requests); count > 0 {\n\t\td.Printf(\" (%d failed)\", count)\n\t}\n\td.Println(\":\")\n\n\tstarted := false\n\tfor _, req := range requests {\n\t\tif !started {\n\t\t\tstarted = true\n\t\t} else {\n\t\t\td.Println()\n\t\t}\n\n\t\td.Printf(\"\\t%s: %s\\n\", req.CreationTimestamp.String(), req.Status.Phase)\n\t\tif len(req.Status.Errors) > 0 {\n\t\t\td.Printf(\"\\tErrors:\\n\")\n\t\t\tfor _, err := range req.Status.Errors {\n\t\t\t\td.Printf(\"\\t\\t%s\\n\", err)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc failedDeletionCount(requests []velerov1api.DeleteBackupRequest) int {\n\tvar count int\n\tfor _, req := range requests {\n\t\tif req.Status.Phase == velerov1api.DeleteBackupRequestPhaseProcessed && len(req.Status.Errors) > 0 {\n\t\t\tcount++\n\t\t}\n\t}\n\treturn count\n}\n\n// describePodVolumeBackups describes pod volume backups in human-readable format.\nfunc describePodVolumeBackups(d *Describer, details bool, podVolumeBackups []velerov1api.PodVolumeBackup) {\n\t// Get the type of pod volume uploader. Since the uploader only comes from a single source, we can\n\t// take the uploader type from the first element of the array.\n\tvar uploaderType string\n\tif len(podVolumeBackups) > 0 {\n\t\tuploaderType = podVolumeBackups[0].Spec.UploaderType\n\t} else {\n\t\td.Printf(\"\\tPod Volume Backups: \\n\")\n\t\treturn\n\t}\n\n\tif details {\n\t\td.Printf(\"\\tPod Volume Backups - %s:\\n\", uploaderType)\n\t} else {\n\t\td.Printf(\"\\tPod Volume Backups - %s (specify --details for more information):\\n\", uploaderType)\n\t}\n\n\t// separate backups by phase (combining and New into a single group)\n\tbackupsByPhase := groupByPhase(podVolumeBackups)\n\n\t// go through phases in a specific order\n\tfor _, phase := range []string{\n\t\tstring(velerov1api.PodVolumeBackupPhaseCompleted),\n\t\tstring(velerov1api.PodVolumeBackupPhaseFailed),\n\t\tstring(velerov1api.PodVolumeBackupPhaseCanceled),\n\t\t\"In Progress\",\n\t\tstring(velerov1api.PodVolumeBackupPhaseCanceling),\n\t\tstring(velerov1api.PodVolumeBackupPhasePrepared),\n\t\tstring(velerov1api.PodVolumeBackupPhaseAccepted),\n\t\tstring(velerov1api.PodVolumeBackupPhaseNew),\n\t} {\n\t\tif len(backupsByPhase[phase]) == 0 {\n\t\t\tcontinue\n\t\t}\n\n\t\t// if we're not printing details, just report the phase and count\n\t\tif !details {\n\t\t\td.Printf(\"\\t\\t%s:\\t%d\\n\", phase, len(backupsByPhase[phase]))\n\t\t\tcontinue\n\t\t}\n\n\t\t// group the backups in the current phase by pod (i.e. \"ns/name\")\n\t\tbackupsByPod := new(volumesByPod)\n\n\t\tfor _, backup := range backupsByPhase[phase] {\n\t\t\tbackupsByPod.Add(backup.Spec.Pod.Namespace, backup.Spec.Pod.Name, backup.Spec.Volume, phase, backup.Status.Progress, backup.Status.IncrementalBytes)\n\t\t}\n\n\t\td.Printf(\"\\t\\t%s:\\n\", phase)\n\t\tfor _, backupGroup := range backupsByPod.Sorted() {\n\t\t\tsort.Strings(backupGroup.volumes)\n\n\t\t\t// print volumes backed up for this pod\n\t\t\td.Printf(\"\\t\\t\\t%s: %s\\n\", backupGroup.label, strings.Join(backupGroup.volumes, \", \"))\n\t\t}\n\t}\n}\n\nfunc groupByPhase(backups []velerov1api.PodVolumeBackup) map[string][]velerov1api.PodVolumeBackup {\n\tbackupsByPhase := make(map[string][]velerov1api.PodVolumeBackup)\n\n\tphaseToGroup := map[velerov1api.PodVolumeBackupPhase]string{\n\t\tvelerov1api.PodVolumeBackupPhaseCompleted: string(velerov1api.PodVolumeBackupPhaseCompleted),\n\t\tvelerov1api.PodVolumeBackupPhaseFailed: string(velerov1api.PodVolumeBackupPhaseFailed),\n\t\tvelerov1api.PodVolumeBackupPhaseCanceled: string(velerov1api.PodVolumeBackupPhaseCanceled),\n\t\tvelerov1api.PodVolumeBackupPhaseInProgress: \"In Progress\",\n\t\tvelerov1api.PodVolumeBackupPhaseCanceling: string(velerov1api.PodVolumeBackupPhaseCanceling),\n\t\tvelerov1api.PodVolumeBackupPhasePrepared: string(velerov1api.PodVolumeBackupPhasePrepared),\n\t\tvelerov1api.PodVolumeBackupPhaseAccepted: string(velerov1api.PodVolumeBackupPhaseAccepted),\n\t\tvelerov1api.PodVolumeBackupPhaseNew: string(velerov1api.PodVolumeBackupPhaseNew),\n\t\t\"\": string(velerov1api.PodVolumeBackupPhaseNew),\n\t}\n\n\tfor _, backup := range backups {\n\t\tgroup := phaseToGroup[backup.Status.Phase]\n\t\tbackupsByPhase[group] = append(backupsByPhase[group], backup)\n\t}\n\n\treturn backupsByPhase\n}\n\ntype podVolumeGroup struct {\n\tlabel string\n\tvolumes []string\n}\n\n// volumesByPod stores podVolumeGroups, where the grouping\n// label is \"namespace/name\".\ntype volumesByPod struct {\n\tvolumesByPodMap map[string]*podVolumeGroup\n\tvolumesByPodSlice []*podVolumeGroup\n}\n\n// Add adds a pod volume with the specified pod namespace, name\n// and volume to the appropriate group.\n// Used for both backup and restore\nfunc (v *volumesByPod) Add(namespace, name, volume, phase string, progress veleroapishared.DataMoveOperationProgress, incrementalBytes int64) {\n\tif v.volumesByPodMap == nil {\n\t\tv.volumesByPodMap = make(map[string]*podVolumeGroup)\n\t}\n\n\tkey := fmt.Sprintf(\"%s/%s\", namespace, name)\n\n\t// append backup progress percentage if backup is in progress\n\tif phase == \"In Progress\" && progress.TotalBytes != 0 {\n\t\tvolume = fmt.Sprintf(\"%s (%.2f%%)\", volume, float64(progress.BytesDone)/float64(progress.TotalBytes)*100)\n\t} else if phase == string(velerov1api.PodVolumeBackupPhaseCompleted) && incrementalBytes > 0 {\n\t\tvolume = fmt.Sprintf(\"%s (size: %v, incremental size: %v)\", volume, progress.TotalBytes, incrementalBytes)\n\t} else if (phase == string(velerov1api.PodVolumeBackupPhaseCompleted) ||\n\t\tphase == string(velerov1api.PodVolumeRestorePhaseCompleted)) &&\n\t\tprogress.TotalBytes > 0 {\n\t\tvolume = fmt.Sprintf(\"%s (size: %v)\", volume, progress.TotalBytes)\n\t}\n\n\tif group, ok := v.volumesByPodMap[key]; !ok {\n\t\tgroup := &podVolumeGroup{\n\t\t\tlabel: key,\n\t\t\tvolumes: []string{volume},\n\t\t}\n\n\t\tv.volumesByPodMap[key] = group\n\t\tv.volumesByPodSlice = append(v.volumesByPodSlice, group)\n\t} else {\n\t\tgroup.volumes = append(group.volumes, volume)\n\t}\n}\n\n// Sorted returns a slice of all pod volume groups, ordered by\n// label.\nfunc (v *volumesByPod) Sorted() []*podVolumeGroup {\n\tsort.Slice(v.volumesByPodSlice, func(i, j int) bool {\n\t\treturn v.volumesByPodSlice[i].label <= v.volumesByPodSlice[j].label\n\t})\n\n\treturn v.volumesByPodSlice\n}\n\n// DescribeBackupResults describes errors and warnings in human-readable format.\nfunc DescribeBackupResults(ctx context.Context, kbClient kbclient.Client, d *Describer, backup *velerov1api.Backup, insecureSkipTLSVerify bool, caCertPath string) {\n\tif backup.Status.Warnings == 0 && backup.Status.Errors == 0 {\n\t\treturn\n\t}\n\n\t// Get BSL cacert if available\n\tbslCACert, err := cacert.GetCACertFromBackup(ctx, kbClient, backup.Namespace, backup)\n\tif err != nil {\n\t\t// Log the error but don't fail - we can still try to download without the BSL cacert\n\t\td.Printf(\"WARNING: Error getting cacert from BSL: %v\\n\", err)\n\t\tbslCACert = \"\"\n\t}\n\n\tvar buf bytes.Buffer\n\tvar resultMap map[string]results.Result\n\n\t// If err 'ErrNotFound' occurs, it means the backup bundle in the bucket has already been there before the backup-result file is introduced.\n\t// We only display the count of errors and warnings in this case.\n\terr = downloadrequest.StreamWithBSLCACert(ctx, kbClient, backup.Namespace, backup.Name, velerov1api.DownloadTargetKindBackupResults, &buf, downloadRequestTimeout, insecureSkipTLSVerify, caCertPath, bslCACert)\n\tif err == downloadrequest.ErrNotFound {\n\t\td.Printf(\"Errors:\\t%d\\n\", backup.Status.Errors)\n\t\td.Printf(\"Warnings:\\t%d\\n\", backup.Status.Warnings)\n\t\treturn\n\t} else if err != nil {\n\t\td.Printf(\"Warnings:\\t\\n\\nErrors:\\t\\n\", err, err)\n\t\treturn\n\t}\n\n\tif err := json.NewDecoder(&buf).Decode(&resultMap); err != nil {\n\t\td.Printf(\"Warnings:\\t\\n\\nErrors:\\t\\n\", err, err)\n\t\treturn\n\t}\n\n\tif backup.Status.Warnings > 0 {\n\t\td.Println()\n\t\tdescribeResult(d, \"Warnings\", resultMap[\"warnings\"])\n\t}\n\tif backup.Status.Errors > 0 {\n\t\td.Println()\n\t\tdescribeResult(d, \"Errors\", resultMap[\"errors\"])\n\t}\n}\n" }, { "path": "pkg/cmd/util/output/backup_describer_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage output\n\nimport (\n\t\"bytes\"\n\t\"testing\"\n\t\"text/tabwriter\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n)\n\nfunc TestDescribeUploaderConfig(t *testing.T) {\n\tinput := builder.ForBackup(\"test-ns\", \"test-backup-1\").ParallelFilesUpload(10).Result().Spec\n\td := &Describer{\n\t\tPrefix: \"\",\n\t\tout: &tabwriter.Writer{},\n\t\tbuf: &bytes.Buffer{},\n\t}\n\td.out.Init(d.buf, 0, 8, 2, ' ', 0)\n\tDescribeUploaderConfigForBackup(d, input)\n\td.out.Flush()\n\texpect := `Uploader config:\n Parallel files upload: 10\n`\n\tassert.Equal(t, expect, d.buf.String())\n}\n\nfunc TestDescribeResourcePolicies(t *testing.T) {\n\tinput := &corev1api.TypedLocalObjectReference{\n\t\tKind: \"configmap\",\n\t\tName: \"test-resource-policy\",\n\t}\n\td := &Describer{\n\t\tPrefix: \"\",\n\t\tout: &tabwriter.Writer{},\n\t\tbuf: &bytes.Buffer{},\n\t}\n\td.out.Init(d.buf, 0, 8, 2, ' ', 0)\n\tDescribeResourcePolicies(d, input)\n\td.out.Flush()\n\texpect := `Resource policies:\n Type: configmap\n Name: test-resource-policy\n`\n\tassert.Equal(t, expect, d.buf.String())\n}\n\nfunc TestDescribeBackupSpec(t *testing.T) {\n\tinput1 := builder.ForBackup(\"test-ns\", \"test-backup-1\").\n\t\tIncludedNamespaces(\"inc-ns-1\", \"inc-ns-2\").\n\t\tExcludedNamespaces(\"exc-ns-1\", \"exc-ns-2\").\n\t\tIncludedResources(\"inc-res-1\", \"inc-res-2\").\n\t\tExcludedResources(\"exc-res-1\", \"exc-res-2\").\n\t\tStorageLocation(\"backup-location\").\n\t\tTTL(72 * time.Hour).\n\t\tCSISnapshotTimeout(10 * time.Minute).\n\t\tDataMover(\"mover\").\n\t\tHooks(velerov1api.BackupHooks{\n\t\t\tResources: []velerov1api.BackupResourceHookSpec{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook-1\",\n\t\t\t\t\tPreHooks: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\tContainer: \"hook-container-1\",\n\t\t\t\t\t\t\t\tCommand: []string{\"pre\"},\n\t\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tPostHooks: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\tContainer: \"hook-container-1\",\n\t\t\t\t\t\t\t\tCommand: []string{\"post\"},\n\t\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tIncludedNamespaces: []string{\"hook-inc-ns-1\", \"hook-inc-ns-2\"},\n\t\t\t\t\tExcludedNamespaces: []string{\"hook-exc-ns-1\", \"hook-exc-ns-2\"},\n\t\t\t\t\tIncludedResources: []string{\"hook-inc-res-1\", \"hook-inc-res-2\"},\n\t\t\t\t\tExcludedResources: []string{\"hook-exc-res-1\", \"hook-exc-res-2\"},\n\t\t\t\t},\n\t\t\t},\n\t\t}).Result().Spec\n\n\texpect1 := `Namespaces:\n Included: inc-ns-1, inc-ns-2\n Excluded: exc-ns-1, exc-ns-2\n\nResources:\n Included: inc-res-1, inc-res-2\n Excluded: exc-res-1, exc-res-2\n Cluster-scoped: auto\n\nLabel selector: \n\nOr label selector: \n\nStorage Location: backup-location\n\nVelero-Native Snapshot PVs: auto\nSnapshot Move Data: auto\nData Mover: mover\n\nTTL: 72h0m0s\n\nCSISnapshotTimeout: 10m0s\nItemOperationTimeout: 0s\n\nHooks:\n Resources:\n hook-1:\n Namespaces:\n Included: hook-inc-ns-1, hook-inc-ns-2\n Excluded: hook-exc-ns-1, hook-exc-ns-2\n\n Resources:\n Included: hook-inc-res-1, hook-inc-res-2\n Excluded: hook-exc-res-1, hook-exc-res-2\n\n Label selector: \n\n Pre Exec Hook:\n Container: hook-container-1\n Command: pre\n On Error: Continue\n Timeout: 0s\n\n Post Exec Hook:\n Container: hook-container-1\n Command: post\n On Error: Continue\n Timeout: 0s\n`\n\n\tinput2 := builder.ForBackup(\"test-ns\", \"test-backup-2\").\n\t\tIncludedNamespaces(\"inc-ns-1\", \"inc-ns-2\").\n\t\tExcludedNamespaces(\"exc-ns-1\", \"exc-ns-2\").\n\t\tIncludedClusterScopedResources(\"inc-cluster-res-1\", \"inc-cluster-res-2\").\n\t\tIncludedNamespaceScopedResources(\"inc-ns-res-1\", \"inc-ns-res-2\").\n\t\tExcludedClusterScopedResources(\"exc-cluster-res-1\", \"exc-cluster-res-2\").\n\t\tExcludedNamespaceScopedResources(\"exc-ns-res-1\", \"exc-ns-res-2\").\n\t\tStorageLocation(\"backup-location\").\n\t\tTTL(72 * time.Hour).\n\t\tCSISnapshotTimeout(10 * time.Minute).\n\t\tDataMover(\"mover\").\n\t\tResult().Spec\n\n\texpect2 := `Namespaces:\n Included: inc-ns-1, inc-ns-2\n Excluded: exc-ns-1, exc-ns-2\n\nResources:\n Included cluster-scoped: inc-cluster-res-1, inc-cluster-res-2\n Excluded cluster-scoped: exc-cluster-res-1, exc-cluster-res-2\n Included namespace-scoped: inc-ns-res-1, inc-ns-res-2\n Excluded namespace-scoped: exc-ns-res-1, exc-ns-res-2\n\nLabel selector: \n\nOr label selector: \n\nStorage Location: backup-location\n\nVelero-Native Snapshot PVs: auto\nSnapshot Move Data: auto\nData Mover: mover\n\nTTL: 72h0m0s\n\nCSISnapshotTimeout: 10m0s\nItemOperationTimeout: 0s\n\nHooks: \n`\n\n\tinput3 := builder.ForBackup(\"test-ns\", \"test-backup-3\").\n\t\tStorageLocation(\"backup-location\").\n\t\tOrderedResources(map[string]string{\n\t\t\t\"kind1\": \"rs1-1, rs1-2\",\n\t\t}).Hooks(velerov1api.BackupHooks{\n\t\tResources: []velerov1api.BackupResourceHookSpec{\n\t\t\t{\n\t\t\t\tName: \"hook-1\",\n\t\t\t\tPreHooks: []velerov1api.BackupResourceHook{\n\t\t\t\t\t{\n\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\tContainer: \"hook-container-1\",\n\t\t\t\t\t\t\tCommand: []string{\"pre\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tPostHooks: []velerov1api.BackupResourceHook{\n\t\t\t\t\t{\n\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\tContainer: \"hook-container-1\",\n\t\t\t\t\t\t\tCommand: []string{\"post\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}).Result().Spec\n\n\texpect3 := `Namespaces:\n Included: *\n Excluded: \n\nResources:\n Included cluster-scoped: \n Excluded cluster-scoped: \n Included namespace-scoped: *\n Excluded namespace-scoped: \n\nLabel selector: \n\nOr label selector: \n\nStorage Location: backup-location\n\nVelero-Native Snapshot PVs: auto\nSnapshot Move Data: auto\nData Mover: velero\n\nTTL: 0s\n\nCSISnapshotTimeout: 0s\nItemOperationTimeout: 0s\n\nHooks:\n Resources:\n hook-1:\n Namespaces:\n Included: *\n Excluded: \n\n Resources:\n Included: *\n Excluded: \n\n Label selector: \n\n Pre Exec Hook:\n Container: hook-container-1\n Command: pre\n On Error: Continue\n Timeout: 0s\n\n Post Exec Hook:\n Container: hook-container-1\n Command: post\n On Error: Continue\n Timeout: 0s\n\nOrderedResources:\n kind1: rs1-1, rs1-2\n`\n\tinput4 := builder.ForBackup(\"test-ns\", \"test-backup-4\").\n\t\tDefaultVolumesToFsBackup(true).\n\t\tStorageLocation(\"backup-location\").\n\t\tResult().Spec\n\n\texpect4 := `Namespaces:\n Included: *\n Excluded: \n\nResources:\n Included cluster-scoped: \n Excluded cluster-scoped: \n Included namespace-scoped: *\n Excluded namespace-scoped: \n\nLabel selector: \n\nOr label selector: \n\nStorage Location: backup-location\n\nVelero-Native Snapshot PVs: auto\nFile System Backup (Default): true\nSnapshot Move Data: auto\nData Mover: velero\n\nTTL: 0s\n\nCSISnapshotTimeout: 0s\nItemOperationTimeout: 0s\n\nHooks: \n`\n\n\tinput5 := builder.ForBackup(\"test-ns\", \"test-backup-5\").\n\t\tDefaultVolumesToFsBackup(false).\n\t\tStorageLocation(\"backup-location\").\n\t\tResult().Spec\n\n\texpect5 := `Namespaces:\n Included: *\n Excluded: \n\nResources:\n Included cluster-scoped: \n Excluded cluster-scoped: \n Included namespace-scoped: *\n Excluded namespace-scoped: \n\nLabel selector: \n\nOr label selector: \n\nStorage Location: backup-location\n\nVelero-Native Snapshot PVs: auto\nFile System Backup (Default): false\nSnapshot Move Data: auto\nData Mover: velero\n\nTTL: 0s\n\nCSISnapshotTimeout: 0s\nItemOperationTimeout: 0s\n\nHooks: \n`\n\n\ttestcases := []struct {\n\t\tname string\n\t\tinput velerov1api.BackupSpec\n\t\texpect string\n\t}{\n\t\t{\n\t\t\tname: \"old resource filter with hooks\",\n\t\t\tinput: input1,\n\t\t\texpect: expect1,\n\t\t},\n\t\t{\n\t\t\tname: \"new resource filter\",\n\t\t\tinput: input2,\n\t\t\texpect: expect2,\n\t\t},\n\t\t{\n\t\t\tname: \"old resource filter with hooks and ordered resources\",\n\t\t\tinput: input3,\n\t\t\texpect: expect3,\n\t\t},\n\t\t{\n\t\t\tname: \"DefaultVolumesToFsBackup is true\",\n\t\t\tinput: input4,\n\t\t\texpect: expect4,\n\t\t},\n\t\t{\n\t\t\tname: \"DefaultVolumesToFsBackup is false\",\n\t\t\tinput: input5,\n\t\t\texpect: expect5,\n\t\t},\n\t}\n\n\tfor _, tc := range testcases {\n\t\tt.Run(tc.name, func(tt *testing.T) {\n\t\t\td := &Describer{\n\t\t\t\tPrefix: \"\",\n\t\t\t\tout: &tabwriter.Writer{},\n\t\t\t\tbuf: &bytes.Buffer{},\n\t\t\t}\n\t\t\td.out.Init(d.buf, 0, 8, 2, ' ', 0)\n\t\t\tDescribeBackupSpec(d, tc.input)\n\t\t\td.out.Flush()\n\t\t\tassert.Equal(tt, tc.expect, d.buf.String())\n\t\t})\n\t}\n}\n\nfunc TestDescribeNativeSnapshots(t *testing.T) {\n\ttestcases := []struct {\n\t\tname string\n\t\tvolumeInfo []*volume.BackupVolumeInfo\n\t\tinputDetails bool\n\t\texpect string\n\t}{\n\t\t{\n\t\t\tname: \"no details\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tBackupMethod: volume.NativeSnapshot,\n\t\t\t\t\tPVName: \"pv-1\",\n\t\t\t\t\tNativeSnapshotInfo: &volume.NativeSnapshotInfo{\n\t\t\t\t\t\tSnapshotHandle: \"snapshot-1\",\n\t\t\t\t\t\tVolumeType: \"ebs\",\n\t\t\t\t\t\tVolumeAZ: \"us-east-2\",\n\t\t\t\t\t\tIOPS: \"1000 mbps\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpect: ` Velero-Native Snapshots:\n pv-1: specify --details for more information\n`,\n\t\t},\n\t\t{\n\t\t\tname: \"details\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tBackupMethod: volume.NativeSnapshot,\n\t\t\t\t\tPVName: \"pv-1\",\n\t\t\t\t\tResult: volume.VolumeResultSucceeded,\n\t\t\t\t\tNativeSnapshotInfo: &volume.NativeSnapshotInfo{\n\t\t\t\t\t\tSnapshotHandle: \"snapshot-1\",\n\t\t\t\t\t\tVolumeType: \"ebs\",\n\t\t\t\t\t\tVolumeAZ: \"us-east-2\",\n\t\t\t\t\t\tIOPS: \"1000 mbps\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tinputDetails: true,\n\t\t\texpect: ` Velero-Native Snapshots:\n pv-1:\n Snapshot ID: snapshot-1\n Type: ebs\n Availability Zone: us-east-2\n IOPS: 1000 mbps\n Result: succeeded\n`,\n\t\t},\n\t}\n\n\tfor _, tc := range testcases {\n\t\tt.Run(tc.name, func(tt *testing.T) {\n\t\t\td := &Describer{\n\t\t\t\tPrefix: \"\",\n\t\t\t\tout: &tabwriter.Writer{},\n\t\t\t\tbuf: &bytes.Buffer{},\n\t\t\t}\n\t\t\td.out.Init(d.buf, 0, 8, 2, ' ', 0)\n\t\t\tdescribeNativeSnapshots(d, tc.inputDetails, tc.volumeInfo)\n\t\t\td.out.Flush()\n\t\t\tassert.Equal(t, tc.expect, d.buf.String())\n\t\t})\n\t}\n}\n\nfunc TestCSISnapshots(t *testing.T) {\n\ttestcases := []struct {\n\t\tname string\n\t\tvolumeInfo []*volume.BackupVolumeInfo\n\t\tinputDetails bool\n\t\texpect string\n\t\tlegacyInfoSource bool\n\t}{\n\t\t{\n\t\t\tname: \"empty info, not legacy\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{},\n\t\t\texpect: ` CSI Snapshots: \n`,\n\t\t},\n\t\t{\n\t\t\tname: \"empty info, legacy\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{},\n\t\t\tlegacyInfoSource: true,\n\t\t\texpect: ` CSI Snapshots: \n`,\n\t\t},\n\t\t{\n\t\t\tname: \"no details, local snapshot\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tBackupMethod: volume.CSISnapshot,\n\t\t\t\t\tPVCNamespace: \"pvc-ns-1\",\n\t\t\t\t\tPVCName: \"pvc-1\",\n\t\t\t\t\tPreserveLocalSnapshot: true,\n\t\t\t\t\tCSISnapshotInfo: &volume.CSISnapshotInfo{\n\t\t\t\t\t\tSnapshotHandle: \"snapshot-1\",\n\t\t\t\t\t\tSize: 1024,\n\t\t\t\t\t\tDriver: \"fake-driver\",\n\t\t\t\t\t\tVSCName: \"vsc-1\",\n\t\t\t\t\t\tOperationID: \"fake-operation-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpect: ` CSI Snapshots:\n pvc-ns-1/pvc-1:\n Snapshot: included, specify --details for more information\n`,\n\t\t},\n\t\t{\n\t\t\tname: \"details, local snapshot\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tBackupMethod: volume.CSISnapshot,\n\t\t\t\t\tPVCNamespace: \"pvc-ns-2\",\n\t\t\t\t\tPVCName: \"pvc-2\",\n\t\t\t\t\tPreserveLocalSnapshot: true,\n\t\t\t\t\tResult: volume.VolumeResultSucceeded,\n\t\t\t\t\tCSISnapshotInfo: &volume.CSISnapshotInfo{\n\t\t\t\t\t\tSnapshotHandle: \"snapshot-2\",\n\t\t\t\t\t\tSize: 1024,\n\t\t\t\t\t\tDriver: \"fake-driver\",\n\t\t\t\t\t\tVSCName: \"vsc-2\",\n\t\t\t\t\t\tOperationID: \"fake-operation-2\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tinputDetails: true,\n\t\t\texpect: ` CSI Snapshots:\n pvc-ns-2/pvc-2:\n Snapshot:\n Operation ID: fake-operation-2\n Snapshot Content Name: vsc-2\n Storage Snapshot ID: snapshot-2\n Snapshot Size (bytes): 1024\n CSI Driver: fake-driver\n Result: succeeded\n`,\n\t\t},\n\t\t{\n\t\t\tname: \"no details, data movement\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tBackupMethod: volume.CSISnapshot,\n\t\t\t\t\tPVCNamespace: \"pvc-ns-3\",\n\t\t\t\t\tPVCName: \"pvc-3\",\n\t\t\t\t\tSnapshotDataMoved: true,\n\t\t\t\t\tSnapshotDataMovementInfo: &volume.SnapshotDataMovementInfo{\n\t\t\t\t\t\tDataMover: \"velero\",\n\t\t\t\t\t\tUploaderType: \"fake-uploader\",\n\t\t\t\t\t\tSnapshotHandle: \"fake-repo-id-3\",\n\t\t\t\t\t\tOperationID: \"fake-operation-3\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpect: ` CSI Snapshots:\n pvc-ns-3/pvc-3:\n Data Movement: included, specify --details for more information\n`,\n\t\t},\n\t\t{\n\t\t\tname: \"details, data movement\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tBackupMethod: volume.CSISnapshot,\n\t\t\t\t\tPVCNamespace: \"pvc-ns-4\",\n\t\t\t\t\tPVCName: \"pvc-4\",\n\t\t\t\t\tSnapshotDataMoved: true,\n\t\t\t\t\tResult: volume.VolumeResultSucceeded,\n\t\t\t\t\tSnapshotDataMovementInfo: &volume.SnapshotDataMovementInfo{\n\t\t\t\t\t\tDataMover: \"velero\",\n\t\t\t\t\t\tUploaderType: \"fake-uploader\",\n\t\t\t\t\t\tSnapshotHandle: \"fake-repo-id-4\",\n\t\t\t\t\t\tOperationID: \"fake-operation-4\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tinputDetails: true,\n\t\t\texpect: ` CSI Snapshots:\n pvc-ns-4/pvc-4:\n Data Movement:\n Operation ID: fake-operation-4\n Data Mover: velero\n Uploader Type: fake-uploader\n Moved data Size (bytes): 0\n Result: succeeded\n`,\n\t\t},\n\t\t{\n\t\t\tname: \"details, data movement, data mover is empty\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tBackupMethod: volume.CSISnapshot,\n\t\t\t\t\tPVCNamespace: \"pvc-ns-5\",\n\t\t\t\t\tPVCName: \"pvc-5\",\n\t\t\t\t\tResult: volume.VolumeResultFailed,\n\t\t\t\t\tSnapshotDataMoved: true,\n\t\t\t\t\tSnapshotDataMovementInfo: &volume.SnapshotDataMovementInfo{\n\t\t\t\t\t\tUploaderType: \"fake-uploader\",\n\t\t\t\t\t\tSnapshotHandle: \"fake-repo-id-5\",\n\t\t\t\t\t\tOperationID: \"fake-operation-5\",\n\t\t\t\t\t\tSize: 100,\n\t\t\t\t\t\tIncrementalSize: 50,\n\t\t\t\t\t\tPhase: velerov2alpha1.DataUploadPhaseFailed,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tinputDetails: true,\n\t\t\texpect: ` CSI Snapshots:\n pvc-ns-5/pvc-5:\n Data Movement:\n Operation ID: fake-operation-5\n Data Mover: velero\n Uploader Type: fake-uploader\n Moved data Size (bytes): 100\n Incremental data Size (bytes): 50\n Result: failed\n`,\n\t\t},\n\t}\n\n\tfor _, tc := range testcases {\n\t\tt.Run(tc.name, func(tt *testing.T) {\n\t\t\td := &Describer{\n\t\t\t\tPrefix: \"\",\n\t\t\t\tout: &tabwriter.Writer{},\n\t\t\t\tbuf: &bytes.Buffer{},\n\t\t\t}\n\t\t\td.out.Init(d.buf, 0, 8, 2, ' ', 0)\n\t\t\tdescribeCSISnapshots(d, tc.inputDetails, tc.volumeInfo, tc.legacyInfoSource)\n\t\t\td.out.Flush()\n\t\t\tassert.Equal(t, tc.expect, d.buf.String())\n\t\t})\n\t}\n}\n\nfunc TestDescribePodVolumeBackups(t *testing.T) {\n\tpvb1 := builder.ForPodVolumeBackup(\"test-ns\", \"test-pvb1\").\n\t\tUploaderType(\"kopia\").\n\t\tPhase(velerov1api.PodVolumeBackupPhaseCompleted).\n\t\tBackupStorageLocation(\"bsl-1\").\n\t\tVolume(\"vol-1\").\n\t\tPodName(\"pod-1\").\n\t\tPodNamespace(\"pod-ns-1\").\n\t\tSnapshotID(\"snap-1\").Result()\n\tpvb2 := builder.ForPodVolumeBackup(\"test-ns1\", \"test-pvb2\").\n\t\tUploaderType(\"kopia\").\n\t\tPhase(velerov1api.PodVolumeBackupPhaseCompleted).\n\t\tBackupStorageLocation(\"bsl-1\").\n\t\tVolume(\"vol-2\").\n\t\tPodName(\"pod-2\").\n\t\tPodNamespace(\"pod-ns-1\").\n\t\tSnapshotID(\"snap-2\").Result()\n\tpvb3 := builder.ForPodVolumeBackup(\"test-ns1\", \"test-pvb3\").\n\t\tUploaderType(\"kopia\").\n\t\tPhase(velerov1api.PodVolumeBackupPhaseFailed).\n\t\tBackupStorageLocation(\"bsl-1\").\n\t\tVolume(\"vol-3\").\n\t\tPodName(\"pod-3\").\n\t\tPodNamespace(\"pod-ns-1\").\n\t\tSnapshotID(\"snap-3\").Result()\n\tpvb4 := builder.ForPodVolumeBackup(\"test-ns1\", \"test-pvb4\").\n\t\tUploaderType(\"kopia\").\n\t\tPhase(velerov1api.PodVolumeBackupPhaseCanceled).\n\t\tBackupStorageLocation(\"bsl-1\").\n\t\tVolume(\"vol-4\").\n\t\tPodName(\"pod-4\").\n\t\tPodNamespace(\"pod-ns-1\").\n\t\tSnapshotID(\"snap-4\").Result()\n\tpvb5 := builder.ForPodVolumeBackup(\"test-ns1\", \"test-pvb5\").\n\t\tUploaderType(\"kopia\").\n\t\tPhase(velerov1api.PodVolumeBackupPhaseInProgress).\n\t\tBackupStorageLocation(\"bsl-1\").\n\t\tVolume(\"vol-5\").\n\t\tPodName(\"pod-5\").\n\t\tPodNamespace(\"pod-ns-1\").\n\t\tSnapshotID(\"snap-5\").Result()\n\tpvb6 := builder.ForPodVolumeBackup(\"test-ns1\", \"test-pvb6\").\n\t\tUploaderType(\"kopia\").\n\t\tPhase(velerov1api.PodVolumeBackupPhaseCanceling).\n\t\tBackupStorageLocation(\"bsl-1\").\n\t\tVolume(\"vol-6\").\n\t\tPodName(\"pod-6\").\n\t\tPodNamespace(\"pod-ns-1\").\n\t\tSnapshotID(\"snap-6\").Result()\n\tpvb7 := builder.ForPodVolumeBackup(\"test-ns1\", \"test-pvb7\").\n\t\tUploaderType(\"kopia\").\n\t\tPhase(velerov1api.PodVolumeBackupPhasePrepared).\n\t\tBackupStorageLocation(\"bsl-1\").\n\t\tVolume(\"vol-7\").\n\t\tPodName(\"pod-7\").\n\t\tPodNamespace(\"pod-ns-1\").\n\t\tSnapshotID(\"snap-7\").Result()\n\tpvb8 := builder.ForPodVolumeBackup(\"test-ns1\", \"test-pvb6\").\n\t\tUploaderType(\"kopia\").\n\t\tPhase(velerov1api.PodVolumeBackupPhaseAccepted).\n\t\tBackupStorageLocation(\"bsl-1\").\n\t\tVolume(\"vol-8\").\n\t\tPodName(\"pod-8\").\n\t\tPodNamespace(\"pod-ns-1\").\n\t\tSnapshotID(\"snap-8\").Result()\n\n\ttestcases := []struct {\n\t\tname string\n\t\tinputPVBList []velerov1api.PodVolumeBackup\n\t\tinputDetails bool\n\t\texpect string\n\t}{\n\t\t{\n\t\t\tname: \"empty list\",\n\t\t\tinputPVBList: []velerov1api.PodVolumeBackup{},\n\t\t\tinputDetails: true,\n\t\t\texpect: ` Pod Volume Backups: \n`,\n\t\t},\n\t\t{\n\t\t\tname: \"2 completed pvbs no details\",\n\t\t\tinputPVBList: []velerov1api.PodVolumeBackup{*pvb1, *pvb2},\n\t\t\tinputDetails: false,\n\t\t\texpect: ` Pod Volume Backups - kopia (specify --details for more information):\n Completed: 2\n`,\n\t\t},\n\t\t{\n\t\t\tname: \"2 completed pvbs with details\",\n\t\t\tinputPVBList: []velerov1api.PodVolumeBackup{*pvb1, *pvb2},\n\t\t\tinputDetails: true,\n\t\t\texpect: ` Pod Volume Backups - kopia:\n Completed:\n pod-ns-1/pod-1: vol-1\n pod-ns-1/pod-2: vol-2\n`,\n\t\t},\n\t\t{\n\t\t\tname: \"all phases with details\",\n\t\t\tinputPVBList: []velerov1api.PodVolumeBackup{*pvb1, *pvb2, *pvb3, *pvb4, *pvb5, *pvb6, *pvb7, *pvb8},\n\t\t\tinputDetails: true,\n\t\t\texpect: ` Pod Volume Backups - kopia:\n Completed:\n pod-ns-1/pod-1: vol-1\n pod-ns-1/pod-2: vol-2\n Failed:\n pod-ns-1/pod-3: vol-3\n Canceled:\n pod-ns-1/pod-4: vol-4\n In Progress:\n pod-ns-1/pod-5: vol-5\n Canceling:\n pod-ns-1/pod-6: vol-6\n Prepared:\n pod-ns-1/pod-7: vol-7\n Accepted:\n pod-ns-1/pod-8: vol-8\n`,\n\t\t},\n\t}\n\n\tfor _, tc := range testcases {\n\t\tt.Run(tc.name, func(tt *testing.T) {\n\t\t\td := &Describer{\n\t\t\t\tPrefix: \"\",\n\t\t\t\tout: &tabwriter.Writer{},\n\t\t\t\tbuf: &bytes.Buffer{},\n\t\t\t}\n\t\t\td.out.Init(d.buf, 0, 8, 2, ' ', 0)\n\t\t\tdescribePodVolumeBackups(d, tc.inputDetails, tc.inputPVBList)\n\t\t\td.out.Flush()\n\t\t\tassert.Equal(tt, tc.expect, d.buf.String())\n\t\t})\n\t}\n}\n\nfunc TestDescribeDeleteBackupRequests(t *testing.T) {\n\tt1, err1 := time.Parse(\"2006-Jan-02\", \"2023-Jun-26\")\n\trequire.NoError(t, err1)\n\tdbr1 := builder.ForDeleteBackupRequest(\"velero\", \"dbr1\").\n\t\tObjectMeta(builder.WithCreationTimestamp(t1)).\n\t\tBackupName(\"bak-1\").\n\t\tPhase(velerov1api.DeleteBackupRequestPhaseProcessed).\n\t\tErrors(\"some error\").Result()\n\tt2, err2 := time.Parse(\"2006-Jan-02\", \"2023-Jun-25\")\n\trequire.NoError(t, err2)\n\tdbr2 := builder.ForDeleteBackupRequest(\"velero\", \"dbr2\").\n\t\tObjectMeta(builder.WithCreationTimestamp(t2)).\n\t\tBackupName(\"bak-2\").\n\t\tPhase(velerov1api.DeleteBackupRequestPhaseInProgress).Result()\n\n\ttestcases := []struct {\n\t\tname string\n\t\tinput []velerov1api.DeleteBackupRequest\n\t\texpect string\n\t}{\n\t\t{\n\t\t\tname: \"empty list\",\n\t\t\tinput: []velerov1api.DeleteBackupRequest{},\n\t\t\texpect: `Deletion Attempts:\n`,\n\t\t},\n\t\t{\n\t\t\tname: \"list with one failed and one in-progress request\",\n\t\t\tinput: []velerov1api.DeleteBackupRequest{*dbr1, *dbr2},\n\t\t\texpect: `Deletion Attempts (1 failed):\n 2023-06-26 00:00:00 +0000 UTC: Processed\n Errors:\n some error\n\n 2023-06-25 00:00:00 +0000 UTC: InProgress\n`,\n\t\t},\n\t}\n\tfor _, tc := range testcases {\n\t\tt.Run(tc.name, func(tt *testing.T) {\n\t\t\td := &Describer{\n\t\t\t\tPrefix: \"\",\n\t\t\t\tout: &tabwriter.Writer{},\n\t\t\t\tbuf: &bytes.Buffer{},\n\t\t\t}\n\t\t\td.out.Init(d.buf, 0, 8, 2, ' ', 0)\n\t\t\tDescribeDeleteBackupRequests(d, tc.input)\n\t\t\td.out.Flush()\n\t\t\tassert.Equal(tt, tc.expect, d.buf.String())\n\t\t})\n\t}\n}\n\nfunc TestDescribeBackupItemOperation(t *testing.T) {\n\tt1, err1 := time.Parse(\"2006-Jan-02\", \"2023-Jun-26\")\n\trequire.NoError(t, err1)\n\tt2, err2 := time.Parse(\"2006-Jan-02\", \"2023-Jun-25\")\n\trequire.NoError(t, err2)\n\tt3, err3 := time.Parse(\"2006-Jan-02\", \"2023-Jun-24\")\n\trequire.NoError(t, err3)\n\tinput := builder.ForBackupOperation().\n\t\tBackupName(\"backup-1\").\n\t\tOperationID(\"op-1\").\n\t\tBackupItemAction(\"action-1\").\n\t\tResourceIdentifier(\"group\", \"rs-type\", \"ns\", \"rs-name\").\n\t\tStatus(*builder.ForOperationStatus().\n\t\t\tPhase(itemoperation.OperationPhaseFailed).\n\t\t\tError(\"operation error\").\n\t\t\tProgress(50, 100, \"bytes\").\n\t\t\tDescription(\"operation description\").\n\t\t\tCreated(t3).\n\t\t\tStarted(t2).\n\t\t\tUpdated(t1).\n\t\t\tResult()).Result()\n\texpected := ` Operation for rs-type.group ns/rs-name:\n Backup Item Action Plugin: action-1\n Operation ID: op-1\n Phase: Failed\n Operation Error: operation error\n Progress: 50 of 100 complete (bytes)\n Progress description: operation description\n Created: 2023-06-24 00:00:00 +0000 UTC\n Started: 2023-06-25 00:00:00 +0000 UTC\n Updated: 2023-06-26 00:00:00 +0000 UTC\n`\n\td := &Describer{\n\t\tPrefix: \"\",\n\t\tout: &tabwriter.Writer{},\n\t\tbuf: &bytes.Buffer{},\n\t}\n\td.out.Init(d.buf, 0, 8, 2, ' ', 0)\n\tdescribeBackupItemOperation(d, input)\n\td.out.Flush()\n\tassert.Equal(t, expected, d.buf.String())\n}\n" }, { "path": "pkg/cmd/util/output/backup_printer.go", "content": "/*\nCopyright 2017, 2019, 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage output\n\nimport (\n\t\"fmt\"\n\t\"regexp\"\n\t\"sort\"\n\t\"strconv\"\n\t\"time\"\n\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/util/duration\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\nvar (\n\tbackupColumns = []metav1.TableColumnDefinition{\n\t\t// name needs Type and Format defined for the decorator to identify it:\n\t\t// https://github.com/kubernetes/kubernetes/blob/v1.15.3/pkg/printers/tableprinter.go#L204\n\t\t{Name: \"Name\", Type: \"string\", Format: \"name\"},\n\t\t{Name: \"Status\"},\n\t\t{Name: \"Errors\"},\n\t\t{Name: \"Warnings\"},\n\t\t{Name: \"Created\"},\n\t\t{Name: \"Expires\"},\n\t\t{Name: \"Storage Location\"},\n\t\t{Name: \"Queue Position\"},\n\t\t{Name: \"Selector\"},\n\t}\n)\n\nfunc printBackupList(list *velerov1api.BackupList) []metav1.TableRow {\n\tsortBackupsByPrefixAndTimestamp(list)\n\trows := make([]metav1.TableRow, 0, len(list.Items))\n\n\tfor i := range list.Items {\n\t\trows = append(rows, printBackup(&list.Items[i])...)\n\t}\n\treturn rows\n}\n\n// sort by default alphabetically, but if backups stem from a common schedule\n// (detected by the presence of a 14-digit timestamp suffix), then within that\n// group, sort by newest to oldest (i.e. prefix ASC, suffix DESC)\nvar timestampSuffix = regexp.MustCompile(\"-[0-9]{14}$\")\n\nfunc sortBackupsByPrefixAndTimestamp(list *velerov1api.BackupList) {\n\tsort.Slice(list.Items, func(i, j int) bool {\n\t\tiSuffixIndex := timestampSuffix.FindStringIndex(list.Items[i].Name)\n\t\tjSuffixIndex := timestampSuffix.FindStringIndex(list.Items[j].Name)\n\n\t\t// one/both don't have a timestamp suffix, so sort alphabetically\n\t\tif iSuffixIndex == nil || jSuffixIndex == nil {\n\t\t\treturn list.Items[i].Name < list.Items[j].Name\n\t\t}\n\n\t\t// different prefixes, so sort alphabetically\n\t\tif list.Items[i].Name[0:iSuffixIndex[0]] != list.Items[j].Name[0:jSuffixIndex[0]] {\n\t\t\treturn list.Items[i].Name < list.Items[j].Name\n\t\t}\n\n\t\t// same prefixes, so sort based on suffix (desc)\n\t\treturn list.Items[i].Name[iSuffixIndex[0]:] >= list.Items[j].Name[jSuffixIndex[0]:]\n\t})\n}\n\nfunc printBackup(backup *velerov1api.Backup) []metav1.TableRow {\n\trow := metav1.TableRow{\n\t\tObject: runtime.RawExtension{Object: backup},\n\t}\n\n\tvar expiration time.Time\n\tif backup.Status.Expiration != nil {\n\t\texpiration = backup.Status.Expiration.Time\n\t}\n\tif expiration.IsZero() && backup.Spec.TTL.Duration > 0 {\n\t\texpiration = backup.CreationTimestamp.Add(backup.Spec.TTL.Duration)\n\t}\n\n\tstatus := string(backup.Status.Phase)\n\tif status == \"\" {\n\t\tstatus = string(velerov1api.BackupPhaseNew)\n\t}\n\tif backup.DeletionTimestamp != nil && !backup.DeletionTimestamp.Time.IsZero() {\n\t\tstatus = \"Deleting\"\n\t}\n\n\trow.Cells = append(row.Cells,\n\t\tbackup.Name,\n\t\tstatus,\n\t\tbackup.Status.Errors,\n\t\tbackup.Status.Warnings,\n\t\tbackup.Status.StartTimestamp,\n\t\thumanReadableTimeFromNow(expiration),\n\t\tbackup.Spec.StorageLocation,\n\t\tqueuePosition(backup.Status.QueuePosition),\n\t\tmetav1.FormatLabelSelector(backup.Spec.LabelSelector),\n\t)\n\n\treturn []metav1.TableRow{row}\n}\n\nfunc humanReadableTimeFromNow(when time.Time) string {\n\tif when.IsZero() {\n\t\treturn \"n/a\"\n\t}\n\n\tnow := time.Now()\n\tswitch {\n\tcase when == now || when.After(now):\n\t\treturn duration.ShortHumanDuration(when.Sub(now))\n\tdefault:\n\t\treturn fmt.Sprintf(\"%s ago\", duration.ShortHumanDuration(now.Sub(when)))\n\t}\n}\n\nfunc queuePosition(pos int) string {\n\tif pos == 0 {\n\t\treturn \"\"\n\t} else {\n\t\treturn strconv.Itoa(pos)\n\t}\n}\n" }, { "path": "pkg/cmd/util/output/backup_printer_test.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage output\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\nfunc TestSortBackups(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackupList *v1.BackupList\n\t\texpected []v1.Backup\n\t}{\n\t\t{\n\t\t\tname: \"non-timestamped backups\",\n\t\t\tbackupList: &v1.BackupList{Items: []v1.Backup{\n\t\t\t\t{ObjectMeta: metav1.ObjectMeta{Name: \"a\"}},\n\t\t\t\t{ObjectMeta: metav1.ObjectMeta{Name: \"c\"}},\n\t\t\t\t{ObjectMeta: metav1.ObjectMeta{Name: \"b\"}},\n\t\t\t}},\n\t\t\texpected: []v1.Backup{\n\t\t\t\t{ObjectMeta: metav1.ObjectMeta{Name: \"a\"}},\n\t\t\t\t{ObjectMeta: metav1.ObjectMeta{Name: \"b\"}},\n\t\t\t\t{ObjectMeta: metav1.ObjectMeta{Name: \"c\"}},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"timestamped backups\",\n\t\t\tbackupList: &v1.BackupList{Items: []v1.Backup{\n\t\t\t\t{ObjectMeta: metav1.ObjectMeta{Name: \"schedule-20170102030405\"}},\n\t\t\t\t{ObjectMeta: metav1.ObjectMeta{Name: \"schedule-20170102030406\"}},\n\t\t\t\t{ObjectMeta: metav1.ObjectMeta{Name: \"schedule-20170102030407\"}},\n\t\t\t}},\n\t\t\texpected: []v1.Backup{\n\t\t\t\t{ObjectMeta: metav1.ObjectMeta{Name: \"schedule-20170102030407\"}},\n\t\t\t\t{ObjectMeta: metav1.ObjectMeta{Name: \"schedule-20170102030406\"}},\n\t\t\t\t{ObjectMeta: metav1.ObjectMeta{Name: \"schedule-20170102030405\"}},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"non-timestamped and timestamped backups\",\n\t\t\tbackupList: &v1.BackupList{Items: []v1.Backup{\n\t\t\t\t{ObjectMeta: metav1.ObjectMeta{Name: \"schedule-20170102030405\"}},\n\t\t\t\t{ObjectMeta: metav1.ObjectMeta{Name: \"schedule-20170102030406\"}},\n\t\t\t\t{ObjectMeta: metav1.ObjectMeta{Name: \"a\"}},\n\t\t\t\t{ObjectMeta: metav1.ObjectMeta{Name: \"schedule-20170102030407\"}},\n\t\t\t}},\n\t\t\texpected: []v1.Backup{\n\t\t\t\t{ObjectMeta: metav1.ObjectMeta{Name: \"a\"}},\n\t\t\t\t{ObjectMeta: metav1.ObjectMeta{Name: \"schedule-20170102030407\"}},\n\t\t\t\t{ObjectMeta: metav1.ObjectMeta{Name: \"schedule-20170102030406\"}},\n\t\t\t\t{ObjectMeta: metav1.ObjectMeta{Name: \"schedule-20170102030405\"}},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tsortBackupsByPrefixAndTimestamp(test.backupList)\n\n\t\t\tif assert.Len(t, test.backupList.Items, len(test.expected)) {\n\t\t\t\tfor i := range test.expected {\n\t\t\t\t\tassert.Equal(t, test.expected[i].Name, test.backupList.Items[i].Name)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/cmd/util/output/backup_repo_printer.go", "content": "/*\nCopyright 2018, 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage output\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\nvar (\n\tbackupRepoColumns = []metav1.TableColumnDefinition{\n\t\t// name needs Type and Format defined for the decorator to identify it:\n\t\t// https://github.com/kubernetes/kubernetes/blob/v1.15.3/pkg/printers/tableprinter.go#L204\n\t\t{Name: \"Name\", Type: \"string\", Format: \"name\"},\n\t\t{Name: \"Status\"},\n\t\t{Name: \"Last Maintenance\"},\n\t}\n)\n\nfunc printBackupRepoList(list *v1.BackupRepositoryList) []metav1.TableRow {\n\trows := make([]metav1.TableRow, 0, len(list.Items))\n\n\tfor i := range list.Items {\n\t\trows = append(rows, printBackupRepo(&list.Items[i])...)\n\t}\n\treturn rows\n}\n\nfunc printBackupRepo(repo *v1.BackupRepository) []metav1.TableRow {\n\trow := metav1.TableRow{\n\t\tObject: runtime.RawExtension{Object: repo},\n\t}\n\n\tstatus := repo.Status.Phase\n\tif status == \"\" {\n\t\tstatus = v1.BackupRepositoryPhaseNew\n\t}\n\n\tvar lastMaintenance string\n\tif repo.Status.LastMaintenanceTime == nil || repo.Status.LastMaintenanceTime.IsZero() {\n\t\tlastMaintenance = \"\"\n\t} else {\n\t\tlastMaintenance = repo.Status.LastMaintenanceTime.String()\n\t}\n\n\trow.Cells = append(row.Cells,\n\t\trepo.Name,\n\t\tstatus,\n\t\tlastMaintenance,\n\t)\n\n\treturn []metav1.TableRow{row}\n}\n" }, { "path": "pkg/cmd/util/output/backup_storage_location_printer.go", "content": "/*\nCopyright 2018, 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage output\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd\"\n)\n\nvar (\n\tbackupStorageLocationColumns = []metav1.TableColumnDefinition{\n\t\t// name needs Type and Format defined for the decorator to identify it:\n\t\t// https://github.com/kubernetes/kubernetes/blob/v1.15.3/pkg/printers/tableprinter.go#L204\n\t\t{Name: \"Name\", Type: \"string\", Format: \"name\"},\n\t\t{Name: \"Provider\"},\n\t\t{Name: \"Bucket/Prefix\"},\n\t\t{Name: \"Phase\"},\n\t\t{Name: \"Last Validated\"},\n\t\t{Name: \"Access Mode\"},\n\t\t{Name: \"Default\"},\n\t}\n)\n\nfunc printBackupStorageLocationList(list *velerov1api.BackupStorageLocationList) []metav1.TableRow {\n\trows := make([]metav1.TableRow, 0, len(list.Items))\n\n\tfor i := range list.Items {\n\t\trows = append(rows, printBackupStorageLocation(&list.Items[i])...)\n\t}\n\treturn rows\n}\n\nfunc printBackupStorageLocation(location *velerov1api.BackupStorageLocation) []metav1.TableRow {\n\trow := metav1.TableRow{\n\t\tObject: runtime.RawExtension{Object: location},\n\t}\n\n\tisDefault := \"\"\n\tif location.Spec.Default {\n\t\tisDefault = cmd.TRUE\n\t}\n\n\tbucketAndPrefix := location.Spec.ObjectStorage.Bucket\n\tif location.Spec.ObjectStorage.Prefix != \"\" {\n\t\tbucketAndPrefix += \"/\" + location.Spec.ObjectStorage.Prefix\n\t}\n\n\taccessMode := location.Spec.AccessMode\n\tif accessMode == \"\" {\n\t\taccessMode = velerov1api.BackupStorageLocationAccessModeReadWrite\n\t}\n\n\tstatus := location.Status.Phase\n\tif status == \"\" {\n\t\tstatus = \"Unknown\"\n\t}\n\n\tlastValidated := location.Status.LastValidationTime\n\tLastValidatedStr := \"Unknown\"\n\tif lastValidated != nil {\n\t\tLastValidatedStr = lastValidated.String()\n\t}\n\n\trow.Cells = append(row.Cells,\n\t\tlocation.Name,\n\t\tlocation.Spec.Provider,\n\t\tbucketAndPrefix,\n\t\tstatus,\n\t\tLastValidatedStr,\n\t\taccessMode,\n\t\tisDefault,\n\t)\n\n\treturn []metav1.TableRow{row}\n}\n" }, { "path": "pkg/cmd/util/output/backup_structured_describer.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage output\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"strings\"\n\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/cacert\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/downloadrequest\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/results\"\n)\n\n// DescribeBackupInSF describes a backup in structured format.\nfunc DescribeBackupInSF(\n\tctx context.Context,\n\tkbClient kbclient.Client,\n\tbackup *velerov1api.Backup,\n\tdeleteRequests []velerov1api.DeleteBackupRequest,\n\tpodVolumeBackups []velerov1api.PodVolumeBackup,\n\tdetails bool,\n\tinsecureSkipTLSVerify bool,\n\tcaCertFile string,\n\toutputFormat string,\n) string {\n\treturn DescribeInSF(func(d *StructuredDescriber) {\n\t\td.DescribeMetadata(backup.ObjectMeta)\n\n\t\td.Describe(\"phase\", backup.Status.Phase)\n\n\t\tif backup.Spec.ResourcePolicy != nil {\n\t\t\tDescribeResourcePoliciesInSF(d, backup.Spec.ResourcePolicy)\n\t\t}\n\n\t\tstatus := backup.Status\n\t\tif len(status.ValidationErrors) > 0 {\n\t\t\td.Describe(\"validationErrors\", status.ValidationErrors)\n\t\t}\n\n\t\tDescribeBackupResultsInSF(ctx, kbClient, d, backup, insecureSkipTLSVerify, caCertFile)\n\n\t\tDescribeBackupSpecInSF(d, backup.Spec)\n\n\t\tDescribeBackupStatusInSF(ctx, kbClient, d, backup, details, insecureSkipTLSVerify, caCertFile, podVolumeBackups)\n\n\t\tif len(deleteRequests) > 0 {\n\t\t\tDescribeDeleteBackupRequestsInSF(d, deleteRequests)\n\t\t}\n\t}, outputFormat)\n}\n\n// DescribeBackupSpecInSF describes a backup spec in structured format.\nfunc DescribeBackupSpecInSF(d *StructuredDescriber, spec velerov1api.BackupSpec) {\n\tbackupSpecInfo := make(map[string]any)\n\tvar s string\n\n\t// describe namespaces\n\tnamespaceInfo := make(map[string]any)\n\tif len(spec.IncludedNamespaces) == 0 {\n\t\ts = \"*\"\n\t} else {\n\t\ts = strings.Join(spec.IncludedNamespaces, \", \")\n\t}\n\tnamespaceInfo[\"included\"] = s\n\tif len(spec.ExcludedNamespaces) == 0 {\n\t\ts = emptyDisplay\n\t} else {\n\t\ts = strings.Join(spec.ExcludedNamespaces, \", \")\n\t}\n\tnamespaceInfo[\"excluded\"] = s\n\tbackupSpecInfo[\"namespaces\"] = namespaceInfo\n\n\t// describe resources\n\tresourcesInfo := make(map[string]string)\n\tif len(spec.IncludedResources) == 0 {\n\t\ts = \"*\"\n\t} else {\n\t\ts = strings.Join(spec.IncludedResources, \", \")\n\t}\n\tresourcesInfo[\"included\"] = s\n\n\tif len(spec.ExcludedResources) == 0 {\n\t\ts = emptyDisplay\n\t} else {\n\t\ts = strings.Join(spec.ExcludedResources, \", \")\n\t}\n\tresourcesInfo[\"excluded\"] = s\n\tresourcesInfo[\"clusterScoped\"] = BoolPointerString(spec.IncludeClusterResources, \"excluded\", \"included\", \"auto\")\n\tbackupSpecInfo[\"resources\"] = resourcesInfo\n\n\t// describe label selector\n\ts = emptyDisplay\n\tif spec.LabelSelector != nil {\n\t\ts = metav1.FormatLabelSelector(spec.LabelSelector)\n\t}\n\tbackupSpecInfo[\"labelSelector\"] = s\n\n\t// describe storage location\n\tbackupSpecInfo[\"storageLocation\"] = spec.StorageLocation\n\n\t// describe snapshot volumes\n\tbackupSpecInfo[\"veleroNativeSnapshotPVs\"] = BoolPointerString(spec.SnapshotVolumes, \"false\", \"true\", \"auto\")\n\t// describe snapshot move data\n\tbackupSpecInfo[\"veleroSnapshotMoveData\"] = BoolPointerString(spec.SnapshotMoveData, \"false\", \"true\", \"auto\")\n\t// describe data mover\n\tif len(spec.DataMover) == 0 {\n\t\ts = emptyDisplay\n\t} else {\n\t\ts = spec.DataMover\n\t}\n\tbackupSpecInfo[\"dataMover\"] = s\n\n\t// describe TTL\n\tbackupSpecInfo[\"TTL\"] = spec.TTL.Duration.String()\n\n\t// describe CSI snapshot timeout\n\tbackupSpecInfo[\"CSISnapshotTimeout\"] = spec.CSISnapshotTimeout.Duration.String()\n\n\t// describe hooks\n\thooksInfo := make(map[string]any)\n\thooksResources := make(map[string]any)\n\tfor _, backupResourceHookSpec := range spec.Hooks.Resources {\n\t\tResourceDetails := make(map[string]any)\n\t\tvar s string\n\t\tnamespaceInfo := make(map[string]string)\n\t\tif len(backupResourceHookSpec.IncludedNamespaces) == 0 {\n\t\t\ts = \"*\"\n\t\t} else {\n\t\t\ts = strings.Join(backupResourceHookSpec.IncludedNamespaces, \", \")\n\t\t}\n\t\tnamespaceInfo[\"included\"] = s\n\t\tif len(backupResourceHookSpec.ExcludedNamespaces) == 0 {\n\t\t\ts = emptyDisplay\n\t\t} else {\n\t\t\ts = strings.Join(backupResourceHookSpec.ExcludedNamespaces, \", \")\n\t\t}\n\t\tnamespaceInfo[\"excluded\"] = s\n\t\tResourceDetails[\"namespaces\"] = namespaceInfo\n\n\t\tresourcesInfo := make(map[string]string)\n\t\tif len(backupResourceHookSpec.IncludedResources) == 0 {\n\t\t\ts = \"*\"\n\t\t} else {\n\t\t\ts = strings.Join(backupResourceHookSpec.IncludedResources, \", \")\n\t\t}\n\t\tresourcesInfo[\"included\"] = s\n\t\tif len(backupResourceHookSpec.ExcludedResources) == 0 {\n\t\t\ts = emptyDisplay\n\t\t} else {\n\t\t\ts = strings.Join(backupResourceHookSpec.ExcludedResources, \", \")\n\t\t}\n\t\tresourcesInfo[\"excluded\"] = s\n\t\tResourceDetails[\"resources\"] = resourcesInfo\n\n\t\ts = emptyDisplay\n\t\tif backupResourceHookSpec.LabelSelector != nil {\n\t\t\ts = metav1.FormatLabelSelector(backupResourceHookSpec.LabelSelector)\n\t\t}\n\t\tResourceDetails[\"labelSelector\"] = s\n\n\t\tpreHooks := make([]map[string]any, 0)\n\t\tfor _, hook := range backupResourceHookSpec.PreHooks {\n\t\t\tif hook.Exec != nil {\n\t\t\t\tpreExecHook := make(map[string]any)\n\t\t\t\tpreExecHook[\"container\"] = hook.Exec.Container\n\t\t\t\tpreExecHook[\"command\"] = strings.Join(hook.Exec.Command, \" \")\n\t\t\t\tpreExecHook[\"onError:\"] = hook.Exec.OnError\n\t\t\t\tpreExecHook[\"timeout\"] = hook.Exec.Timeout.Duration.String()\n\t\t\t\tpreHooks = append(preHooks, preExecHook)\n\t\t\t}\n\t\t}\n\t\tResourceDetails[\"preExecHook\"] = preHooks\n\n\t\tpostHooks := make([]map[string]any, 0)\n\t\tfor _, hook := range backupResourceHookSpec.PostHooks {\n\t\t\tif hook.Exec != nil {\n\t\t\t\tpostExecHook := make(map[string]any)\n\t\t\t\tpostExecHook[\"container\"] = hook.Exec.Container\n\t\t\t\tpostExecHook[\"command\"] = strings.Join(hook.Exec.Command, \" \")\n\t\t\t\tpostExecHook[\"onError:\"] = hook.Exec.OnError\n\t\t\t\tpostExecHook[\"timeout\"] = hook.Exec.Timeout.Duration.String()\n\t\t\t\tpostHooks = append(postHooks, postExecHook)\n\t\t\t}\n\t\t}\n\t\tResourceDetails[\"postExecHook\"] = postHooks\n\t\thooksResources[backupResourceHookSpec.Name] = ResourceDetails\n\t}\n\tif len(spec.Hooks.Resources) > 0 {\n\t\thooksInfo[\"resources\"] = hooksResources\n\t\tbackupSpecInfo[\"hooks\"] = hooksInfo\n\t}\n\n\t// desrcibe ordered resources\n\tif spec.OrderedResources != nil {\n\t\tbackupSpecInfo[\"orderedResources\"] = spec.OrderedResources\n\t}\n\n\td.Describe(\"spec\", backupSpecInfo)\n}\n\n// DescribeBackupStatusInSF describes a backup status in structured format.\nfunc DescribeBackupStatusInSF(ctx context.Context, kbClient kbclient.Client, d *StructuredDescriber, backup *velerov1api.Backup, details bool,\n\tinsecureSkipTLSVerify bool, caCertPath string, podVolumeBackups []velerov1api.PodVolumeBackup) {\n\tstatus := backup.Status\n\tbackupStatusInfo := make(map[string]any)\n\n\t// Status.Version has been deprecated, use Status.FormatVersion\n\tbackupStatusInfo[\"backupFormatVersion\"] = status.FormatVersion\n\n\t// \"\" output should only be applicable for backups that failed validation\n\tif status.StartTimestamp == nil || status.StartTimestamp.Time.IsZero() {\n\t\tbackupStatusInfo[\"started\"] = \"\"\n\t} else {\n\t\tbackupStatusInfo[\"started\"] = status.StartTimestamp.Time.String()\n\t}\n\tif status.CompletionTimestamp == nil || status.CompletionTimestamp.Time.IsZero() {\n\t\tbackupStatusInfo[\"completed\"] = \"\"\n\t} else {\n\t\tbackupStatusInfo[\"completed\"] = status.CompletionTimestamp.Time.String()\n\t}\n\n\t// Expiration can't be 0, it is always set to a 30-day default. It can be nil\n\t// if the controller hasn't processed this Backup yet, in which case this will\n\t// just display ``, though this should be temporary.\n\tbackupStatusInfo[\"expiration\"] = status.Expiration.String()\n\n\tdefer d.Describe(\"status\", backupStatusInfo)\n\n\tif backup.Status.Progress != nil {\n\t\tif backup.Status.Phase == velerov1api.BackupPhaseInProgress {\n\t\t\tbackupStatusInfo[\"estimatedTotalItemsToBeBackedUp\"] = backup.Status.Progress.TotalItems\n\t\t\tbackupStatusInfo[\"itemsBackedUpSoFar\"] = backup.Status.Progress.ItemsBackedUp\n\t\t} else {\n\t\t\tbackupStatusInfo[\"totalItemsToBeBackedUp\"] = backup.Status.Progress.TotalItems\n\t\t\tbackupStatusInfo[\"itemsBackedUp\"] = backup.Status.Progress.ItemsBackedUp\n\t\t}\n\t}\n\n\tif details {\n\t\tdescribeBackupResourceListInSF(ctx, kbClient, backupStatusInfo, backup, insecureSkipTLSVerify, caCertPath)\n\t}\n\n\tdescribeBackupVolumesInSF(ctx, kbClient, backup, details, insecureSkipTLSVerify, caCertPath, podVolumeBackups, backupStatusInfo)\n\n\tif status.HookStatus != nil {\n\t\tbackupStatusInfo[\"hooksAttempted\"] = status.HookStatus.HooksAttempted\n\t\tbackupStatusInfo[\"hooksFailed\"] = status.HookStatus.HooksFailed\n\t}\n}\n\nfunc describeBackupResourceListInSF(ctx context.Context, kbClient kbclient.Client, backupStatusInfo map[string]any, backup *velerov1api.Backup, insecureSkipTLSVerify bool, caCertPath string) {\n\t// Get BSL cacert if available\n\tbslCACert, err := cacert.GetCACertFromBackup(ctx, kbClient, backup.Namespace, backup)\n\tif err != nil {\n\t\t// Log the error but don't fail - we can still try to download without the BSL cacert\n\t\tbackupStatusInfo[\"warningGettingBSLCACert\"] = fmt.Sprintf(\"Warning: Error getting cacert from BSL: %v\", err)\n\t\tbslCACert = \"\"\n\t}\n\n\t// In consideration of decoding structured output conveniently, the two separate fields were created here(in func describeBackupResourceList, there is only one field describing either error message or resource list)\n\t// the field of 'errorGettingResourceList' gives specific error message when it fails to get resources list\n\t// the field of 'resourceList' lists the rearranged resources\n\tbuf := new(bytes.Buffer)\n\tif err := downloadrequest.StreamWithBSLCACert(ctx, kbClient, backup.Namespace, backup.Name, velerov1api.DownloadTargetKindBackupResourceList, buf, downloadRequestTimeout, insecureSkipTLSVerify, caCertPath, bslCACert); err != nil {\n\t\tif err == downloadrequest.ErrNotFound {\n\t\t\t// the backup resource list could be missing if (other reasons may exist as well):\n\t\t\t//\t- the backup was taken prior to v1.1; or\n\t\t\t//\t- the backup hasn't completed yet; or\n\t\t\t//\t- there was an error uploading the file; or\n\t\t\t//\t- the file was manually deleted after upload\n\t\t\tbackupStatusInfo[\"errorGettingResourceList\"] = \"\"\n\t\t} else {\n\t\t\tbackupStatusInfo[\"errorGettingResourceList\"] = fmt.Sprintf(\"\", err)\n\t\t}\n\t\treturn\n\t}\n\n\tvar resourceList map[string][]string\n\tif err := json.NewDecoder(buf).Decode(&resourceList); err != nil {\n\t\tbackupStatusInfo[\"errorGettingResourceList\"] = fmt.Sprintf(\"\\n\", err)\n\t\treturn\n\t}\n\tbackupStatusInfo[\"resourceList\"] = resourceList\n}\n\nfunc describeBackupVolumesInSF(ctx context.Context, kbClient kbclient.Client, backup *velerov1api.Backup, details bool,\n\tinsecureSkipTLSVerify bool, caCertPath string, podVolumeBackupCRs []velerov1api.PodVolumeBackup, backupStatusInfo map[string]any) {\n\tbackupVolumes := make(map[string]any)\n\n\t// Get BSL cacert if available\n\tbslCACert, err := cacert.GetCACertFromBackup(ctx, kbClient, backup.Namespace, backup)\n\tif err != nil {\n\t\t// Log the error but don't fail - we can still try to download without the BSL cacert\n\t\tbackupVolumes[\"warningGettingBSLCACert\"] = fmt.Sprintf(\"Warning: Error getting cacert from BSL: %v\", err)\n\t\tbslCACert = \"\"\n\t}\n\n\tnativeSnapshots := []*volume.BackupVolumeInfo{}\n\tcsiSnapshots := []*volume.BackupVolumeInfo{}\n\tlegacyInfoSource := false\n\n\tbuf := new(bytes.Buffer)\n\terr = downloadrequest.StreamWithBSLCACert(ctx, kbClient, backup.Namespace, backup.Name, velerov1api.DownloadTargetKindBackupVolumeInfos, buf, downloadRequestTimeout, insecureSkipTLSVerify, caCertPath, bslCACert)\n\tif err == downloadrequest.ErrNotFound {\n\t\tnativeSnapshots, err = retrieveNativeSnapshotLegacy(ctx, kbClient, backup, insecureSkipTLSVerify, caCertPath, bslCACert)\n\t\tif err != nil {\n\t\t\tbackupVolumes[\"errorConcludeNativeSnapshot\"] = fmt.Sprintf(\"error concluding native snapshot info: %v\", err)\n\t\t\treturn\n\t\t}\n\n\t\tcsiSnapshots, err = retrieveCSISnapshotLegacy(ctx, kbClient, backup, insecureSkipTLSVerify, caCertPath, bslCACert)\n\t\tif err != nil {\n\t\t\tbackupVolumes[\"errorConcludeCSISnapshot\"] = fmt.Sprintf(\"error concluding CSI snapshot info: %v\", err)\n\t\t\treturn\n\t\t}\n\n\t\tlegacyInfoSource = true\n\t} else if err != nil {\n\t\tbackupVolumes[\"errorGetBackupVolumeInfo\"] = fmt.Sprintf(\"error getting backup volume info: %v\", err)\n\t\treturn\n\t} else {\n\t\tvar volumeInfos []volume.BackupVolumeInfo\n\t\tif err := json.NewDecoder(buf).Decode(&volumeInfos); err != nil {\n\t\t\tbackupVolumes[\"errorReadBackupVolumeInfo\"] = fmt.Sprintf(\"error reading backup volume info: %v\", err)\n\t\t\treturn\n\t\t}\n\n\t\tfor i := range volumeInfos {\n\t\t\tswitch volumeInfos[i].BackupMethod {\n\t\t\tcase volume.NativeSnapshot:\n\t\t\t\tnativeSnapshots = append(nativeSnapshots, &volumeInfos[i])\n\t\t\tcase volume.CSISnapshot:\n\t\t\t\tcsiSnapshots = append(csiSnapshots, &volumeInfos[i])\n\t\t\t}\n\t\t}\n\t}\n\n\tdescribeNativeSnapshotsInSF(details, nativeSnapshots, backupVolumes)\n\n\tdescribeCSISnapshotsInSF(details, csiSnapshots, backupVolumes, legacyInfoSource)\n\n\tdescribePodVolumeBackupsInSF(podVolumeBackupCRs, details, backupVolumes)\n\n\tbackupStatusInfo[\"backupVolumes\"] = backupVolumes\n}\n\nfunc describeNativeSnapshotsInSF(details bool, infos []*volume.BackupVolumeInfo, backupVolumes map[string]any) {\n\tif len(infos) == 0 {\n\t\tbackupVolumes[\"nativeSnapshots\"] = \"\"\n\t\treturn\n\t}\n\n\tsnapshotDetails := make(map[string]any)\n\tfor _, info := range infos {\n\t\tdescribNativeSnapshotInSF(details, info, snapshotDetails)\n\t}\n\tbackupVolumes[\"nativeSnapshots\"] = snapshotDetails\n}\n\nfunc describNativeSnapshotInSF(details bool, info *volume.BackupVolumeInfo, snapshotDetails map[string]any) {\n\tif details {\n\t\tsnapshotInfo := make(map[string]string)\n\t\tsnapshotInfo[\"snapshotID\"] = info.NativeSnapshotInfo.SnapshotHandle\n\t\tsnapshotInfo[\"type\"] = info.NativeSnapshotInfo.VolumeType\n\t\tsnapshotInfo[\"availabilityZone\"] = info.NativeSnapshotInfo.VolumeAZ\n\t\tsnapshotInfo[\"IOPS\"] = info.NativeSnapshotInfo.IOPS\n\t\tsnapshotInfo[\"result\"] = string(info.Result)\n\n\t\tsnapshotDetails[info.PVName] = snapshotInfo\n\t} else {\n\t\tsnapshotDetails[info.PVName] = \"specify --details for more information\"\n\t}\n}\n\nfunc describeCSISnapshotsInSF(details bool, infos []*volume.BackupVolumeInfo, backupVolumes map[string]any, legacyInfoSource bool) {\n\tif len(infos) == 0 {\n\t\tif legacyInfoSource {\n\t\t\tbackupVolumes[\"csiSnapshots\"] = \"\"\n\t\t} else {\n\t\t\tbackupVolumes[\"csiSnapshots\"] = \"\"\n\t\t}\n\t\treturn\n\t}\n\n\tsnapshotDetails := make(map[string]any)\n\tfor _, info := range infos {\n\t\tdescribeCSISnapshotInSF(details, info, snapshotDetails)\n\t}\n\tbackupVolumes[\"csiSnapshots\"] = snapshotDetails\n}\n\nfunc describeCSISnapshotInSF(details bool, info *volume.BackupVolumeInfo, snapshotDetails map[string]any) {\n\tsnapshotDetail := make(map[string]any)\n\n\tdescribeLocalSnapshotInSF(details, info, snapshotDetail)\n\tdescribeDataMovementInSF(details, info, snapshotDetail)\n\n\tsnapshotDetails[fmt.Sprintf(\"%s/%s\", info.PVCNamespace, info.PVCName)] = snapshotDetail\n}\n\n// describeLocalSnapshotInSF describes CSI volume snapshot contents in structured format.\nfunc describeLocalSnapshotInSF(details bool, info *volume.BackupVolumeInfo, snapshotDetail map[string]any) {\n\tif !info.PreserveLocalSnapshot {\n\t\treturn\n\t}\n\n\tif details {\n\t\tlocalSnapshot := make(map[string]any)\n\n\t\tif !info.SnapshotDataMoved {\n\t\t\tlocalSnapshot[\"operationID\"] = info.CSISnapshotInfo.OperationID\n\t\t}\n\n\t\tlocalSnapshot[\"snapshotContentName\"] = info.CSISnapshotInfo.VSCName\n\t\tlocalSnapshot[\"storageSnapshotID\"] = info.CSISnapshotInfo.SnapshotHandle\n\t\tlocalSnapshot[\"snapshotSize(bytes)\"] = info.CSISnapshotInfo.Size\n\t\tlocalSnapshot[\"csiDriver\"] = info.CSISnapshotInfo.Driver\n\t\tlocalSnapshot[\"result\"] = string(info.Result)\n\n\t\tsnapshotDetail[\"snapshot\"] = localSnapshot\n\t} else {\n\t\tsnapshotDetail[\"snapshot\"] = \"included, specify --details for more information\"\n\t}\n}\n\nfunc describeDataMovementInSF(details bool, info *volume.BackupVolumeInfo, snapshotDetail map[string]any) {\n\tif !info.SnapshotDataMoved {\n\t\treturn\n\t}\n\n\tif details {\n\t\tdataMovement := make(map[string]any)\n\t\tdataMovement[\"operationID\"] = info.SnapshotDataMovementInfo.OperationID\n\n\t\tdataMover := \"velero\"\n\t\tif info.SnapshotDataMovementInfo.DataMover != \"\" {\n\t\t\tdataMover = info.SnapshotDataMovementInfo.DataMover\n\t\t}\n\t\tdataMovement[\"dataMover\"] = dataMover\n\n\t\tdataMovement[\"uploaderType\"] = info.SnapshotDataMovementInfo.UploaderType\n\t\tdataMovement[\"result\"] = string(info.Result)\n\t\tif info.SnapshotDataMovementInfo.Size > 0 || info.SnapshotDataMovementInfo.IncrementalSize > 0 {\n\t\t\tdataMovement[\"size\"] = info.SnapshotDataMovementInfo.Size\n\t\t\tdataMovement[\"incrementalSize\"] = info.SnapshotDataMovementInfo.IncrementalSize\n\t\t}\n\n\t\tsnapshotDetail[\"dataMovement\"] = dataMovement\n\t} else {\n\t\tsnapshotDetail[\"dataMovement\"] = \"included, specify --details for more information\"\n\t}\n}\n\n// DescribeDeleteBackupRequestsInSF describes delete backup requests in structured format.\nfunc DescribeDeleteBackupRequestsInSF(d *StructuredDescriber, requests []velerov1api.DeleteBackupRequest) {\n\tdeletionAttempts := make(map[string]any)\n\tif count := failedDeletionCount(requests); count > 0 {\n\t\tdeletionAttempts[\"failed\"] = count\n\t}\n\n\tdeletionRequests := make([]map[string]any, 0)\n\tfor _, req := range requests {\n\t\tdeletionReq := make(map[string]any)\n\t\tdeletionReq[\"creationTimestamp\"] = req.CreationTimestamp.String()\n\t\tdeletionReq[\"phase\"] = req.Status.Phase\n\n\t\tif len(req.Status.Errors) > 0 {\n\t\t\tdeletionReq[\"errors\"] = req.Status.Errors\n\t\t}\n\t\tdeletionRequests = append(deletionRequests, deletionReq)\n\t}\n\tdeletionAttempts[\"deleteBackupRequests\"] = deletionRequests\n\td.Describe(\"deletionAttempts\", deletionAttempts)\n}\n\n// describePodVolumeBackupsInSF describes pod volume backups in structured format.\nfunc describePodVolumeBackupsInSF(backups []velerov1api.PodVolumeBackup, details bool, backupVolumes map[string]any) {\n\tpodVolumeBackupsInfo := make(map[string]any)\n\t// Get the type of pod volume uploader. Since the uploader only comes from a single source, we can\n\t// take the uploader type from the first element of the array.\n\tvar uploaderType string\n\tif len(backups) > 0 {\n\t\tuploaderType = backups[0].Spec.UploaderType\n\t} else {\n\t\tbackupVolumes[\"podVolumeBackups\"] = \"\"\n\t\treturn\n\t}\n\t// type display the type of pod volume backups\n\tpodVolumeBackupsInfo[\"uploderType\"] = uploaderType\n\n\tpodVolumeBackupsDetails := make(map[string]any)\n\t// separate backups by phase (combining and New into a single group)\n\tbackupsByPhase := groupByPhase(backups)\n\n\t// go through phases in a specific order\n\tfor _, phase := range []string{\n\t\tstring(velerov1api.PodVolumeBackupPhaseCompleted),\n\t\tstring(velerov1api.PodVolumeBackupPhaseFailed),\n\t\tstring(velerov1api.PodVolumeBackupPhaseCanceled),\n\t\t\"In Progress\",\n\t\tstring(velerov1api.PodVolumeBackupPhaseCanceling),\n\t\tstring(velerov1api.PodVolumeBackupPhasePrepared),\n\t\tstring(velerov1api.PodVolumeBackupPhaseAccepted),\n\t\tstring(velerov1api.PodVolumeBackupPhaseNew),\n\t} {\n\t\tif len(backupsByPhase[phase]) == 0 {\n\t\t\tcontinue\n\t\t}\n\t\t// if we're not printing details, just report the phase and count\n\t\tif !details {\n\t\t\tpodVolumeBackupsDetails[phase] = len(backupsByPhase[phase])\n\t\t\tcontinue\n\t\t}\n\t\t// group the backups in the current phase by pod (i.e. \"ns/name\")\n\t\tbackupsByPod := new(volumesByPod)\n\t\tfor _, backup := range backupsByPhase[phase] {\n\t\t\tbackupsByPod.Add(backup.Spec.Pod.Namespace, backup.Spec.Pod.Name, backup.Spec.Volume, phase, backup.Status.Progress, backup.Status.IncrementalBytes)\n\t\t}\n\n\t\tbackupsByPods := make([]map[string]string, 0)\n\t\tfor _, backupGroup := range backupsByPod.volumesByPodSlice {\n\t\t\t// print volumes backed up for this pod\n\t\t\tbackupsByPods = append(backupsByPods, map[string]string{backupGroup.label: strings.Join(backupGroup.volumes, \", \")})\n\t\t}\n\t\tpodVolumeBackupsDetails[phase] = backupsByPods\n\t}\n\t// Pod Volume Backups Details display the detailed pod volume backups info\n\tpodVolumeBackupsInfo[\"podVolumeBackupsDetails\"] = podVolumeBackupsDetails\n\tbackupVolumes[\"podVolumeBackups\"] = podVolumeBackupsInfo\n}\n\n// DescribeBackupResultsInSF describes errors and warnings in structured format.\nfunc DescribeBackupResultsInSF(ctx context.Context, kbClient kbclient.Client, d *StructuredDescriber, backup *velerov1api.Backup, insecureSkipTLSVerify bool, caCertPath string) {\n\tif backup.Status.Warnings == 0 && backup.Status.Errors == 0 {\n\t\treturn\n\t}\n\n\t// Get BSL cacert if available\n\tbslCACert, err := cacert.GetCACertFromBackup(ctx, kbClient, backup.Namespace, backup)\n\tif err != nil {\n\t\t// Log the error but don't fail - we can still try to download without the BSL cacert\n\t\twarnings := make(map[string]any)\n\t\twarnings[\"warningGettingBSLCACert\"] = fmt.Sprintf(\"Warning: Error getting cacert from BSL: %v\", err)\n\t\td.Describe(\"warningsGettingBSLCACert\", warnings)\n\t\tbslCACert = \"\"\n\t}\n\n\tvar buf bytes.Buffer\n\tvar resultMap map[string]results.Result\n\n\terrors, warnings := make(map[string]any), make(map[string]any)\n\tdefer func() {\n\t\td.Describe(\"errors\", errors)\n\t\td.Describe(\"warnings\", warnings)\n\t}()\n\n\t// If 'ErrNotFound' occurs, it means the backup bundle in the bucket has already been there before the backup-result file is introduced.\n\t// We only display the count of errors and warnings in this case.\n\terr = downloadrequest.StreamWithBSLCACert(ctx, kbClient, backup.Namespace, backup.Name, velerov1api.DownloadTargetKindBackupResults, &buf, downloadRequestTimeout, insecureSkipTLSVerify, caCertPath, bslCACert)\n\tif err == downloadrequest.ErrNotFound {\n\t\terrors[\"count\"] = backup.Status.Errors\n\t\twarnings[\"count\"] = backup.Status.Warnings\n\t\treturn\n\t} else if err != nil {\n\t\terrors[\"errorGettingErrors\"] = fmt.Errorf(\"\", err)\n\t\twarnings[\"errorGettingWarnings\"] = fmt.Errorf(\"\", err)\n\t\treturn\n\t}\n\n\tif err := json.NewDecoder(&buf).Decode(&resultMap); err != nil {\n\t\terrors[\"errorGettingErrors\"] = fmt.Errorf(\"\", err)\n\t\twarnings[\"errorGettingWarnings\"] = fmt.Errorf(\"\", err)\n\t\treturn\n\t}\n\n\tif backup.Status.Warnings > 0 {\n\t\tdescribeResultInSF(warnings, resultMap[\"warnings\"])\n\t}\n\tif backup.Status.Errors > 0 {\n\t\tdescribeResultInSF(errors, resultMap[\"errors\"])\n\t}\n}\n\n// DescribeResourcePoliciesInSF describes resource policies in structured format.\nfunc DescribeResourcePoliciesInSF(d *StructuredDescriber, resPolicies *corev1api.TypedLocalObjectReference) {\n\tpoliciesInfo := make(map[string]any)\n\tpoliciesInfo[\"type\"] = resPolicies.Kind\n\tpoliciesInfo[\"name\"] = resPolicies.Name\n\td.Describe(\"resourcePolicies\", policiesInfo)\n}\n\nfunc describeResultInSF(m map[string]any, result results.Result) {\n\tm[\"velero\"], m[\"cluster\"], m[\"namespace\"] = []string{}, []string{}, []string{}\n\n\tif len(result.Velero) > 0 {\n\t\tm[\"velero\"] = result.Velero\n\t}\n\tif len(result.Cluster) > 0 {\n\t\tm[\"cluster\"] = result.Cluster\n\t}\n\tif len(result.Namespaces) > 0 {\n\t\tm[\"namespace\"] = result.Namespaces\n\t}\n}\n" }, { "path": "pkg/cmd/util/output/backup_structured_describer_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage output\n\nimport (\n\t\"reflect\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/results\"\n)\n\nfunc TestDescribeBackupInSF(t *testing.T) {\n\tsd := &StructuredDescriber{\n\t\toutput: make(map[string]any),\n\t\tformat: \"\",\n\t}\n\tbackupBuilder1 := builder.ForBackup(\"test-ns\", \"test-backup\")\n\tbackupBuilder1.IncludedNamespaces(\"inc-ns-1\", \"inc-ns-2\").\n\t\tExcludedNamespaces(\"exc-ns-1\", \"exc-ns-2\").\n\t\tIncludedResources(\"inc-res-1\", \"inc-res-2\").\n\t\tExcludedResources(\"exc-res-1\", \"exc-res-2\").\n\t\tStorageLocation(\"backup-location\").\n\t\tTTL(72 * time.Hour).\n\t\tCSISnapshotTimeout(10 * time.Minute).\n\t\tDataMover(\"mover\").\n\t\tHooks(velerov1api.BackupHooks{\n\t\t\tResources: []velerov1api.BackupResourceHookSpec{\n\t\t\t\t{\n\t\t\t\t\tName: \"hook-1\",\n\t\t\t\t\tPreHooks: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\tContainer: \"hook-container-1\",\n\t\t\t\t\t\t\t\tCommand: []string{\"pre\"},\n\t\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tPostHooks: []velerov1api.BackupResourceHook{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\t\tContainer: \"hook-container-1\",\n\t\t\t\t\t\t\t\tCommand: []string{\"post\"},\n\t\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tIncludedNamespaces: []string{\"hook-inc-ns-1\", \"hook-inc-ns-2\"},\n\t\t\t\t\tExcludedNamespaces: []string{\"hook-exc-ns-1\", \"hook-exc-ns-2\"},\n\t\t\t\t\tIncludedResources: []string{\"hook-inc-res-1\", \"hook-inc-res-2\"},\n\t\t\t\t\tExcludedResources: []string{\"hook-exc-res-1\", \"hook-exc-res-2\"},\n\t\t\t\t},\n\t\t\t},\n\t\t})\n\n\texpect1 := map[string]any{\n\t\t\"spec\": map[string]any{\n\t\t\t\"namespaces\": map[string]any{\n\t\t\t\t\"included\": \"inc-ns-1, inc-ns-2\",\n\t\t\t\t\"excluded\": \"exc-ns-1, exc-ns-2\",\n\t\t\t},\n\t\t\t\"resources\": map[string]string{\n\t\t\t\t\"included\": \"inc-res-1, inc-res-2\",\n\t\t\t\t\"excluded\": \"exc-res-1, exc-res-2\",\n\t\t\t\t\"clusterScoped\": \"auto\",\n\t\t\t},\n\t\t\t\"dataMover\": \"mover\",\n\t\t\t\"labelSelector\": emptyDisplay,\n\t\t\t\"storageLocation\": \"backup-location\",\n\t\t\t\"veleroNativeSnapshotPVs\": \"auto\",\n\t\t\t\"TTL\": \"72h0m0s\",\n\t\t\t\"CSISnapshotTimeout\": \"10m0s\",\n\t\t\t\"veleroSnapshotMoveData\": \"auto\",\n\t\t\t\"hooks\": map[string]any{\n\t\t\t\t\"resources\": map[string]any{\n\t\t\t\t\t\"hook-1\": map[string]any{\n\t\t\t\t\t\t\"labelSelector\": emptyDisplay,\n\t\t\t\t\t\t\"namespaces\": map[string]string{\n\t\t\t\t\t\t\t\"included\": \"hook-inc-ns-1, hook-inc-ns-2\",\n\t\t\t\t\t\t\t\"excluded\": \"hook-exc-ns-1, hook-exc-ns-2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t\"preExecHook\": []map[string]any{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\"container\": \"hook-container-1\",\n\t\t\t\t\t\t\t\t\"command\": \"pre\",\n\t\t\t\t\t\t\t\t\"onError:\": velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\t\t\"timeout\": \"0s\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t\"postExecHook\": []map[string]any{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\"container\": \"hook-container-1\",\n\t\t\t\t\t\t\t\t\"command\": \"post\",\n\t\t\t\t\t\t\t\t\"onError:\": velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\t\t\"timeout\": \"0s\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t\"resources\": map[string]string{\n\t\t\t\t\t\t\t\"included\": \"hook-inc-res-1, hook-inc-res-2\",\n\t\t\t\t\t\t\t\"excluded\": \"hook-exc-res-1, hook-exc-res-2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tDescribeBackupSpecInSF(sd, backupBuilder1.Result().Spec)\n\tassert.True(t, reflect.DeepEqual(sd.output, expect1))\n\n\tbackupBuilder2 := builder.ForBackup(\"test-ns-2\", \"test-backup-2\").\n\t\tStorageLocation(\"backup-location\").\n\t\tOrderedResources(map[string]string{\n\t\t\t\"kind1\": \"rs1-1, rs1-2\",\n\t\t\t\"kind2\": \"rs2-1, rs2-2\",\n\t\t}).Hooks(velerov1api.BackupHooks{\n\t\tResources: []velerov1api.BackupResourceHookSpec{\n\t\t\t{\n\t\t\t\tName: \"hook-1\",\n\t\t\t\tPreHooks: []velerov1api.BackupResourceHook{\n\t\t\t\t\t{\n\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\tContainer: \"hook-container-1\",\n\t\t\t\t\t\t\tCommand: []string{\"pre\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tPostHooks: []velerov1api.BackupResourceHook{\n\t\t\t\t\t{\n\t\t\t\t\t\tExec: &velerov1api.ExecHook{\n\t\t\t\t\t\t\tContainer: \"hook-container-1\",\n\t\t\t\t\t\t\tCommand: []string{\"post\"},\n\t\t\t\t\t\t\tOnError: velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t})\n\n\texpect2 := map[string]any{\n\t\t\"spec\": map[string]any{\n\t\t\t\"namespaces\": map[string]any{\n\t\t\t\t\"included\": \"*\",\n\t\t\t\t\"excluded\": emptyDisplay,\n\t\t\t},\n\t\t\t\"resources\": map[string]string{\n\t\t\t\t\"included\": \"*\",\n\t\t\t\t\"excluded\": emptyDisplay,\n\t\t\t\t\"clusterScoped\": \"auto\",\n\t\t\t},\n\t\t\t\"dataMover\": emptyDisplay,\n\t\t\t\"labelSelector\": emptyDisplay,\n\t\t\t\"storageLocation\": \"backup-location\",\n\t\t\t\"veleroNativeSnapshotPVs\": \"auto\",\n\t\t\t\"TTL\": \"0s\",\n\t\t\t\"CSISnapshotTimeout\": \"0s\",\n\t\t\t\"veleroSnapshotMoveData\": \"auto\",\n\t\t\t\"hooks\": map[string]any{\n\t\t\t\t\"resources\": map[string]any{\n\t\t\t\t\t\"hook-1\": map[string]any{\n\t\t\t\t\t\t\"labelSelector\": emptyDisplay,\n\t\t\t\t\t\t\"namespaces\": map[string]string{\n\t\t\t\t\t\t\t\"included\": \"*\",\n\t\t\t\t\t\t\t\"excluded\": emptyDisplay,\n\t\t\t\t\t\t},\n\t\t\t\t\t\t\"preExecHook\": []map[string]any{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\"container\": \"hook-container-1\",\n\t\t\t\t\t\t\t\t\"command\": \"pre\",\n\t\t\t\t\t\t\t\t\"onError:\": velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\t\t\"timeout\": \"0s\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t\"postExecHook\": []map[string]any{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\"container\": \"hook-container-1\",\n\t\t\t\t\t\t\t\t\"command\": \"post\",\n\t\t\t\t\t\t\t\t\"onError:\": velerov1api.HookErrorModeContinue,\n\t\t\t\t\t\t\t\t\"timeout\": \"0s\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t\"resources\": map[string]string{\n\t\t\t\t\t\t\t\"included\": \"*\",\n\t\t\t\t\t\t\t\"excluded\": emptyDisplay,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\t\"orderedResources\": map[string]string{\n\t\t\t\t\"kind1\": \"rs1-1, rs1-2\",\n\t\t\t\t\"kind2\": \"rs2-1, rs2-2\",\n\t\t\t},\n\t\t},\n\t}\n\tDescribeBackupSpecInSF(sd, backupBuilder2.Result().Spec)\n\tassert.True(t, reflect.DeepEqual(sd.output, expect2))\n}\n\nfunc TestDescribePodVolumeBackupsInSF(t *testing.T) {\n\tpvbBuilder1 := builder.ForPodVolumeBackup(\"test-ns1\", \"test-pvb1\")\n\tpvb1 := pvbBuilder1.BackupStorageLocation(\"backup-location\").\n\t\tUploaderType(\"kopia\").\n\t\tPhase(velerov1api.PodVolumeBackupPhaseCompleted).\n\t\tBackupStorageLocation(\"bsl-1\").\n\t\tVolume(\"vol-1\").\n\t\tPodName(\"pod-1\").\n\t\tPodNamespace(\"pod-ns-1\").\n\t\tSnapshotID(\"snap-1\").Result()\n\n\tpvbBuilder2 := builder.ForPodVolumeBackup(\"test-ns1\", \"test-pvb2\")\n\tpvb2 := pvbBuilder2.BackupStorageLocation(\"backup-location\").\n\t\tUploaderType(\"kopia\").\n\t\tPhase(velerov1api.PodVolumeBackupPhaseCompleted).\n\t\tBackupStorageLocation(\"bsl-1\").\n\t\tVolume(\"vol-2\").\n\t\tPodName(\"pod-2\").\n\t\tPodNamespace(\"pod-ns-1\").\n\t\tSnapshotID(\"snap-2\").Result()\n\n\tpvb3 := builder.ForPodVolumeBackup(\"test-ns1\", \"test-pvb3\").\n\t\tUploaderType(\"kopia\").\n\t\tPhase(velerov1api.PodVolumeBackupPhaseFailed).\n\t\tBackupStorageLocation(\"bsl-1\").\n\t\tVolume(\"vol-3\").\n\t\tPodName(\"pod-3\").\n\t\tPodNamespace(\"pod-ns-1\").\n\t\tSnapshotID(\"snap-3\").Result()\n\tpvb4 := builder.ForPodVolumeBackup(\"test-ns1\", \"test-pvb4\").\n\t\tUploaderType(\"kopia\").\n\t\tPhase(velerov1api.PodVolumeBackupPhaseCanceled).\n\t\tBackupStorageLocation(\"bsl-1\").\n\t\tVolume(\"vol-4\").\n\t\tPodName(\"pod-4\").\n\t\tPodNamespace(\"pod-ns-1\").\n\t\tSnapshotID(\"snap-4\").Result()\n\tpvb5 := builder.ForPodVolumeBackup(\"test-ns1\", \"test-pvb5\").\n\t\tUploaderType(\"kopia\").\n\t\tPhase(velerov1api.PodVolumeBackupPhaseInProgress).\n\t\tBackupStorageLocation(\"bsl-1\").\n\t\tVolume(\"vol-5\").\n\t\tPodName(\"pod-5\").\n\t\tPodNamespace(\"pod-ns-1\").\n\t\tSnapshotID(\"snap-5\").Result()\n\tpvb6 := builder.ForPodVolumeBackup(\"test-ns1\", \"test-pvb6\").\n\t\tUploaderType(\"kopia\").\n\t\tPhase(velerov1api.PodVolumeBackupPhaseCanceling).\n\t\tBackupStorageLocation(\"bsl-1\").\n\t\tVolume(\"vol-6\").\n\t\tPodName(\"pod-6\").\n\t\tPodNamespace(\"pod-ns-1\").\n\t\tSnapshotID(\"snap-6\").Result()\n\tpvb7 := builder.ForPodVolumeBackup(\"test-ns1\", \"test-pvb7\").\n\t\tUploaderType(\"kopia\").\n\t\tPhase(velerov1api.PodVolumeBackupPhasePrepared).\n\t\tBackupStorageLocation(\"bsl-1\").\n\t\tVolume(\"vol-7\").\n\t\tPodName(\"pod-7\").\n\t\tPodNamespace(\"pod-ns-1\").\n\t\tSnapshotID(\"snap-7\").Result()\n\tpvb8 := builder.ForPodVolumeBackup(\"test-ns1\", \"test-pvb6\").\n\t\tUploaderType(\"kopia\").\n\t\tPhase(velerov1api.PodVolumeBackupPhaseAccepted).\n\t\tBackupStorageLocation(\"bsl-1\").\n\t\tVolume(\"vol-8\").\n\t\tPodName(\"pod-8\").\n\t\tPodNamespace(\"pod-ns-1\").\n\t\tSnapshotID(\"snap-8\").Result()\n\n\ttestcases := []struct {\n\t\tname string\n\t\tinputPVBList []velerov1api.PodVolumeBackup\n\t\tinputDetails bool\n\t\texpect map[string]any\n\t}{\n\t\t{\n\t\t\tname: \"empty list\",\n\t\t\tinputPVBList: []velerov1api.PodVolumeBackup{},\n\t\t\tinputDetails: false,\n\t\t\texpect: map[string]any{\"podVolumeBackups\": \"\"},\n\t\t},\n\t\t{\n\t\t\tname: \"2 completed pvbs\",\n\t\t\tinputPVBList: []velerov1api.PodVolumeBackup{*pvb1, *pvb2},\n\t\t\tinputDetails: true,\n\t\t\texpect: map[string]any{\n\t\t\t\t\"podVolumeBackups\": map[string]any{\n\t\t\t\t\t\"podVolumeBackupsDetails\": map[string]any{\n\t\t\t\t\t\t\"Completed\": []map[string]string{\n\t\t\t\t\t\t\t{\"pod-ns-1/pod-1\": \"vol-1\"},\n\t\t\t\t\t\t\t{\"pod-ns-1/pod-2\": \"vol-2\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t\"uploderType\": \"kopia\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"all phases\",\n\t\t\tinputPVBList: []velerov1api.PodVolumeBackup{*pvb1, *pvb2, *pvb3, *pvb4, *pvb5, *pvb6, *pvb7, *pvb8},\n\t\t\tinputDetails: true,\n\t\t\texpect: map[string]any{\n\t\t\t\t\"podVolumeBackups\": map[string]any{\n\t\t\t\t\t\"podVolumeBackupsDetails\": map[string]any{\n\t\t\t\t\t\t\"Completed\": []map[string]string{\n\t\t\t\t\t\t\t{\"pod-ns-1/pod-1\": \"vol-1\"},\n\t\t\t\t\t\t\t{\"pod-ns-1/pod-2\": \"vol-2\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t\"Failed\": []map[string]string{\n\t\t\t\t\t\t\t{\"pod-ns-1/pod-3\": \"vol-3\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t\"Canceled\": []map[string]string{\n\t\t\t\t\t\t\t{\"pod-ns-1/pod-4\": \"vol-4\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t\"In Progress\": []map[string]string{\n\t\t\t\t\t\t\t{\"pod-ns-1/pod-5\": \"vol-5\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t\"Canceling\": []map[string]string{\n\t\t\t\t\t\t\t{\"pod-ns-1/pod-6\": \"vol-6\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t\"Prepared\": []map[string]string{\n\t\t\t\t\t\t\t{\"pod-ns-1/pod-7\": \"vol-7\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t\"Accepted\": []map[string]string{\n\t\t\t\t\t\t\t{\"pod-ns-1/pod-8\": \"vol-8\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t\"uploderType\": \"kopia\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tfor _, tc := range testcases {\n\t\tt.Run(tc.name, func(tt *testing.T) {\n\t\t\toutput := make(map[string]any)\n\t\t\tdescribePodVolumeBackupsInSF(tc.inputPVBList, tc.inputDetails, output)\n\t\t\tassert.True(tt, reflect.DeepEqual(output, tc.expect))\n\t\t})\n\t}\n}\n\nfunc TestDescribeNativeSnapshotsInSF(t *testing.T) {\n\ttestcases := []struct {\n\t\tname string\n\t\tvolumeInfo []*volume.BackupVolumeInfo\n\t\tinputDetails bool\n\t\texpect map[string]any\n\t}{\n\t\t{\n\t\t\tname: \"no details\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tBackupMethod: volume.NativeSnapshot,\n\t\t\t\t\tPVName: \"pv-1\",\n\t\t\t\t\tNativeSnapshotInfo: &volume.NativeSnapshotInfo{\n\t\t\t\t\t\tSnapshotHandle: \"snapshot-1\",\n\t\t\t\t\t\tVolumeType: \"ebs\",\n\t\t\t\t\t\tVolumeAZ: \"us-east-2\",\n\t\t\t\t\t\tIOPS: \"1000 mbps\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpect: map[string]any{\n\t\t\t\t\"nativeSnapshots\": map[string]any{\n\t\t\t\t\t\"pv-1\": \"specify --details for more information\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"details\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tBackupMethod: volume.NativeSnapshot,\n\t\t\t\t\tPVName: \"pv-1\",\n\t\t\t\t\tResult: volume.VolumeResultSucceeded,\n\t\t\t\t\tNativeSnapshotInfo: &volume.NativeSnapshotInfo{\n\t\t\t\t\t\tSnapshotHandle: \"snapshot-1\",\n\t\t\t\t\t\tVolumeType: \"ebs\",\n\t\t\t\t\t\tVolumeAZ: \"us-east-2\",\n\t\t\t\t\t\tIOPS: \"1000 mbps\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tinputDetails: true,\n\t\t\texpect: map[string]any{\n\t\t\t\t\"nativeSnapshots\": map[string]any{\n\t\t\t\t\t\"pv-1\": map[string]string{\n\t\t\t\t\t\t\"snapshotID\": \"snapshot-1\",\n\t\t\t\t\t\t\"type\": \"ebs\",\n\t\t\t\t\t\t\"availabilityZone\": \"us-east-2\",\n\t\t\t\t\t\t\"IOPS\": \"1000 mbps\",\n\t\t\t\t\t\t\"result\": \"succeeded\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testcases {\n\t\tt.Run(tc.name, func(tt *testing.T) {\n\t\t\toutput := make(map[string]any)\n\t\t\tdescribeNativeSnapshotsInSF(tc.inputDetails, tc.volumeInfo, output)\n\t\t\tassert.True(tt, reflect.DeepEqual(output, tc.expect))\n\t\t})\n\t}\n}\n\nfunc TestDescribeCSISnapshotsInSF(t *testing.T) {\n\ttestcases := []struct {\n\t\tname string\n\t\tvolumeInfo []*volume.BackupVolumeInfo\n\t\tinputDetails bool\n\t\texpect map[string]any\n\t\tlegacyInfoSource bool\n\t}{\n\t\t{\n\t\t\tname: \"empty info, not legacy\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{},\n\t\t\texpect: map[string]any{\n\t\t\t\t\"csiSnapshots\": \"\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"empty info, legacy\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{},\n\t\t\tlegacyInfoSource: true,\n\t\t\texpect: map[string]any{\n\t\t\t\t\"csiSnapshots\": \"\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"no details, local snapshot\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tBackupMethod: volume.CSISnapshot,\n\t\t\t\t\tPVCNamespace: \"pvc-ns-1\",\n\t\t\t\t\tPVCName: \"pvc-1\",\n\t\t\t\t\tPreserveLocalSnapshot: true,\n\t\t\t\t\tCSISnapshotInfo: &volume.CSISnapshotInfo{\n\t\t\t\t\t\tSnapshotHandle: \"snapshot-1\",\n\t\t\t\t\t\tSize: 1024,\n\t\t\t\t\t\tDriver: \"fake-driver\",\n\t\t\t\t\t\tVSCName: \"vsc-1\",\n\t\t\t\t\t\tOperationID: \"fake-operation-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpect: map[string]any{\n\t\t\t\t\"csiSnapshots\": map[string]any{\n\t\t\t\t\t\"pvc-ns-1/pvc-1\": map[string]any{\n\t\t\t\t\t\t\"snapshot\": \"included, specify --details for more information\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"details, local snapshot\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tBackupMethod: volume.CSISnapshot,\n\t\t\t\t\tPVCNamespace: \"pvc-ns-2\",\n\t\t\t\t\tPVCName: \"pvc-2\",\n\t\t\t\t\tPreserveLocalSnapshot: true,\n\t\t\t\t\tResult: volume.VolumeResultSucceeded,\n\t\t\t\t\tCSISnapshotInfo: &volume.CSISnapshotInfo{\n\t\t\t\t\t\tSnapshotHandle: \"snapshot-2\",\n\t\t\t\t\t\tSize: 1024,\n\t\t\t\t\t\tDriver: \"fake-driver\",\n\t\t\t\t\t\tVSCName: \"vsc-2\",\n\t\t\t\t\t\tOperationID: \"fake-operation-2\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tinputDetails: true,\n\t\t\texpect: map[string]any{\n\t\t\t\t\"csiSnapshots\": map[string]any{\n\t\t\t\t\t\"pvc-ns-2/pvc-2\": map[string]any{\n\t\t\t\t\t\t\"snapshot\": map[string]any{\n\t\t\t\t\t\t\t\"operationID\": \"fake-operation-2\",\n\t\t\t\t\t\t\t\"snapshotContentName\": \"vsc-2\",\n\t\t\t\t\t\t\t\"storageSnapshotID\": \"snapshot-2\",\n\t\t\t\t\t\t\t\"snapshotSize(bytes)\": int64(1024),\n\t\t\t\t\t\t\t\"csiDriver\": \"fake-driver\",\n\t\t\t\t\t\t\t\"result\": \"succeeded\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"no details, data movement\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tBackupMethod: volume.CSISnapshot,\n\t\t\t\t\tPVCNamespace: \"pvc-ns-3\",\n\t\t\t\t\tPVCName: \"pvc-3\",\n\t\t\t\t\tSnapshotDataMoved: true,\n\t\t\t\t\tSnapshotDataMovementInfo: &volume.SnapshotDataMovementInfo{\n\t\t\t\t\t\tDataMover: \"velero\",\n\t\t\t\t\t\tUploaderType: \"fake-uploader\",\n\t\t\t\t\t\tSnapshotHandle: \"fake-repo-id-3\",\n\t\t\t\t\t\tOperationID: \"fake-operation-3\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpect: map[string]any{\n\t\t\t\t\"csiSnapshots\": map[string]any{\n\t\t\t\t\t\"pvc-ns-3/pvc-3\": map[string]any{\n\t\t\t\t\t\t\"dataMovement\": \"included, specify --details for more information\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"details, data movement\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tBackupMethod: volume.CSISnapshot,\n\t\t\t\t\tPVCNamespace: \"pvc-ns-4\",\n\t\t\t\t\tPVCName: \"pvc-4\",\n\t\t\t\t\tSnapshotDataMoved: true,\n\t\t\t\t\tResult: volume.VolumeResultSucceeded,\n\t\t\t\t\tSnapshotDataMovementInfo: &volume.SnapshotDataMovementInfo{\n\t\t\t\t\t\tDataMover: \"velero\",\n\t\t\t\t\t\tUploaderType: \"fake-uploader\",\n\t\t\t\t\t\tSnapshotHandle: \"fake-repo-id-4\",\n\t\t\t\t\t\tOperationID: \"fake-operation-4\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tinputDetails: true,\n\t\t\texpect: map[string]any{\n\t\t\t\t\"csiSnapshots\": map[string]any{\n\t\t\t\t\t\"pvc-ns-4/pvc-4\": map[string]any{\n\t\t\t\t\t\t\"dataMovement\": map[string]any{\n\t\t\t\t\t\t\t\"operationID\": \"fake-operation-4\",\n\t\t\t\t\t\t\t\"dataMover\": \"velero\",\n\t\t\t\t\t\t\t\"uploaderType\": \"fake-uploader\",\n\t\t\t\t\t\t\t\"result\": \"succeeded\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"details, data movement, data mover is empty\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tBackupMethod: volume.CSISnapshot,\n\t\t\t\t\tPVCNamespace: \"pvc-ns-4\",\n\t\t\t\t\tResult: volume.VolumeResultFailed,\n\t\t\t\t\tPVCName: \"pvc-4\",\n\t\t\t\t\tSnapshotDataMoved: true,\n\t\t\t\t\tSnapshotDataMovementInfo: &volume.SnapshotDataMovementInfo{\n\t\t\t\t\t\tUploaderType: \"fake-uploader\",\n\t\t\t\t\t\tSnapshotHandle: \"fake-repo-id-4\",\n\t\t\t\t\t\tOperationID: \"fake-operation-4\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tinputDetails: true,\n\t\t\texpect: map[string]any{\n\t\t\t\t\"csiSnapshots\": map[string]any{\n\t\t\t\t\t\"pvc-ns-4/pvc-4\": map[string]any{\n\t\t\t\t\t\t\"dataMovement\": map[string]any{\n\t\t\t\t\t\t\t\"operationID\": \"fake-operation-4\",\n\t\t\t\t\t\t\t\"dataMover\": \"velero\",\n\t\t\t\t\t\t\t\"uploaderType\": \"fake-uploader\",\n\t\t\t\t\t\t\t\"result\": \"failed\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testcases {\n\t\tt.Run(tc.name, func(tt *testing.T) {\n\t\t\toutput := make(map[string]any)\n\t\t\tdescribeCSISnapshotsInSF(tc.inputDetails, tc.volumeInfo, output, tc.legacyInfoSource)\n\t\t\tassert.True(tt, reflect.DeepEqual(output, tc.expect))\n\t\t})\n\t}\n}\n\nfunc TestDescribeResourcePoliciesInSF(t *testing.T) {\n\tinput := &corev1api.TypedLocalObjectReference{\n\t\tKind: \"configmap\",\n\t\tName: \"resource-policy-1\",\n\t}\n\texpect := map[string]any{\n\t\t\"resourcePolicies\": map[string]any{\n\t\t\t\"type\": \"configmap\",\n\t\t\t\"name\": \"resource-policy-1\",\n\t\t},\n\t}\n\tsd := &StructuredDescriber{\n\t\toutput: make(map[string]any),\n\t\tformat: \"\",\n\t}\n\tDescribeResourcePoliciesInSF(sd, input)\n\tassert.True(t, reflect.DeepEqual(sd.output, expect))\n}\n\nfunc TestDescribeBackupResultInSF(t *testing.T) {\n\tinput := results.Result{\n\t\tVelero: []string{\"msg-1\", \"msg-2\"},\n\t\tCluster: []string{\"cluster-1\", \"cluster-2\"},\n\t\tNamespaces: map[string][]string{\n\t\t\t\"ns-1\": {\"ns-1-msg-1\", \"ns-1-msg-2\"},\n\t\t},\n\t}\n\tgot := map[string]any{}\n\texpect := map[string]any{\n\t\t\"velero\": []string{\"msg-1\", \"msg-2\"},\n\t\t\"cluster\": []string{\"cluster-1\", \"cluster-2\"},\n\t\t\"namespace\": map[string][]string{\n\t\t\t\"ns-1\": {\"ns-1-msg-1\", \"ns-1-msg-2\"},\n\t\t},\n\t}\n\tdescribeResultInSF(got, input)\n\tassert.True(t, reflect.DeepEqual(got, expect))\n}\n\nfunc TestDescribeDeleteBackupRequestsInSF(t *testing.T) {\n\tt1, err1 := time.Parse(\"2006-Jan-02\", \"2023-Jun-26\")\n\trequire.NoError(t, err1)\n\tdbr1 := builder.ForDeleteBackupRequest(\"velero\", \"dbr1\").\n\t\tObjectMeta(builder.WithCreationTimestamp(t1)).\n\t\tBackupName(\"bak-1\").\n\t\tPhase(velerov1api.DeleteBackupRequestPhaseProcessed).\n\t\tErrors(\"some error\").Result()\n\tt2, err2 := time.Parse(\"2006-Jan-02\", \"2023-Jun-25\")\n\trequire.NoError(t, err2)\n\tdbr2 := builder.ForDeleteBackupRequest(\"velero\", \"dbr2\").\n\t\tObjectMeta(builder.WithCreationTimestamp(t2)).\n\t\tBackupName(\"bak-2\").\n\t\tPhase(velerov1api.DeleteBackupRequestPhaseInProgress).Result()\n\n\ttestcases := []struct {\n\t\tname string\n\t\tinput []velerov1api.DeleteBackupRequest\n\t\texpect map[string]any\n\t}{\n\t\t{\n\t\t\tname: \"empty list\",\n\t\t\tinput: []velerov1api.DeleteBackupRequest{},\n\t\t\texpect: map[string]any{\n\t\t\t\t\"deletionAttempts\": map[string]any{\n\t\t\t\t\t\"deleteBackupRequests\": []map[string]any{},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"list with one failed and one in-progress request\",\n\t\t\tinput: []velerov1api.DeleteBackupRequest{*dbr1, *dbr2},\n\t\t\texpect: map[string]any{\n\t\t\t\t\"deletionAttempts\": map[string]any{\n\t\t\t\t\t\"failed\": int(1),\n\t\t\t\t\t\"deleteBackupRequests\": []map[string]any{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\"creationTimestamp\": t1.String(),\n\t\t\t\t\t\t\t\"phase\": velerov1api.DeleteBackupRequestPhaseProcessed,\n\t\t\t\t\t\t\t\"errors\": []string{\n\t\t\t\t\t\t\t\t\"some error\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\"creationTimestamp\": t2.String(),\n\t\t\t\t\t\t\t\"phase\": velerov1api.DeleteBackupRequestPhaseInProgress,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tfor _, tc := range testcases {\n\t\tt.Run(tc.name, func(tt *testing.T) {\n\t\t\tsd := &StructuredDescriber{\n\t\t\t\toutput: make(map[string]any),\n\t\t\t\tformat: \"\",\n\t\t\t}\n\t\t\tDescribeDeleteBackupRequestsInSF(sd, tc.input)\n\t\t\tassert.True(tt, reflect.DeepEqual(sd.output, tc.expect))\n\t\t})\n\t}\n}\n" }, { "path": "pkg/cmd/util/output/describe.go", "content": "/*\nCopyright 2021 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage output\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"sort\"\n\t\"strings\"\n\t\"text/tabwriter\"\n\n\t\"github.com/fatih/color\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\ntype Describer struct {\n\tPrefix string\n\tout *tabwriter.Writer\n\tbuf *bytes.Buffer\n}\n\nfunc Describe(fn func(d *Describer)) string {\n\td := Describer{\n\t\tout: new(tabwriter.Writer),\n\t\tbuf: new(bytes.Buffer),\n\t}\n\td.out.Init(d.buf, 0, 8, 2, ' ', 0)\n\n\tfn(&d)\n\n\td.out.Flush()\n\treturn d.buf.String()\n}\n\nfunc (d *Describer) Printf(msg string, args ...any) {\n\tfmt.Fprint(d.out, d.Prefix)\n\tfmt.Fprintf(d.out, msg, args...)\n}\n\nfunc (d *Describer) Println(args ...any) {\n\tfmt.Fprint(d.out, d.Prefix)\n\tfmt.Fprintln(d.out, args...)\n}\n\n// DescribeMetadata describes standard object metadata in a consistent manner.\nfunc (d *Describer) DescribeMetadata(metadata metav1.ObjectMeta) {\n\td.Printf(\"Name:\\t%s\\n\", color.New(color.Bold).SprintFunc()(metadata.Name))\n\td.Printf(\"Namespace:\\t%s\\n\", metadata.Namespace)\n\td.DescribeMap(\"Labels\", metadata.Labels)\n\td.DescribeMap(\"Annotations\", metadata.Annotations)\n}\n\n// DescribeMap describes a map of key-value pairs using name as the heading.\nfunc (d *Describer) DescribeMap(name string, m map[string]string) {\n\td.Printf(\"%s:\\t\", name)\n\n\tfirst := true\n\tprefix := \"\"\n\tif len(m) > 0 {\n\t\tkeys := make([]string, 0, len(m))\n\t\tfor key := range m {\n\t\t\tkeys = append(keys, key)\n\t\t}\n\t\tsort.Strings(keys)\n\t\tfor _, key := range keys {\n\t\t\td.Printf(\"%s%s=%s\\n\", prefix, key, m[key])\n\t\t\tif first {\n\t\t\t\tfirst = false\n\t\t\t\tprefix = \"\\t\"\n\t\t\t}\n\t\t}\n\t} else {\n\t\td.Printf(\"\\n\")\n\t}\n}\n\n// DescribeSlice describes a slice of strings using name as the heading. The output is prefixed by\n// \"preindent\" number of tabs.\nfunc (d *Describer) DescribeSlice(preindent int, name string, s []string) {\n\tpretab := strings.Repeat(\"\\t\", preindent)\n\td.Printf(\"%s%s:\\t\", pretab, name)\n\n\tfirst := true\n\tprefix := \"\"\n\tif len(s) > 0 {\n\t\tfor _, x := range s {\n\t\t\td.Printf(\"%s%s\\n\", prefix, x)\n\t\t\tif first {\n\t\t\t\tfirst = false\n\t\t\t\tprefix = pretab + \"\\t\"\n\t\t\t}\n\t\t}\n\t} else {\n\t\td.Printf(\"%s\\n\", pretab)\n\t}\n}\n\n// BoolPointerString returns the appropriate string based on the bool pointer's value.\nfunc BoolPointerString(b *bool, falseString, trueString, nilString string) string {\n\tif b == nil {\n\t\treturn nilString\n\t}\n\tif *b {\n\t\treturn trueString\n\t}\n\treturn falseString\n}\n\ntype StructuredDescriber struct {\n\toutput map[string]any\n\tformat string\n}\n\n// NewStructuredDescriber creates a StructuredDescriber.\nfunc NewStructuredDescriber(format string) *StructuredDescriber {\n\treturn &StructuredDescriber{\n\t\toutput: make(map[string]any),\n\t\tformat: format,\n\t}\n}\n\n// DescribeInSF returns the structured output based on the func\n// that applies StructuredDescriber to collect outputs.\n// This function takes arg 'format' for future format extension.\nfunc DescribeInSF(fn func(d *StructuredDescriber), format string) string {\n\td := NewStructuredDescriber(format)\n\tfn(d)\n\treturn d.JSONEncode()\n}\n\n// Describe adds all types of argument to d.output.\nfunc (d *StructuredDescriber) Describe(name string, arg any) {\n\td.output[name] = arg\n}\n\n// DescribeMetadata describes standard object metadata.\nfunc (d *StructuredDescriber) DescribeMetadata(metadata metav1.ObjectMeta) {\n\tmetadataInfo := make(map[string]any)\n\tmetadataInfo[\"name\"] = metadata.Name\n\tmetadataInfo[\"namespace\"] = metadata.Namespace\n\tmetadataInfo[\"labels\"] = metadata.Labels\n\tmetadataInfo[\"annotations\"] = metadata.Annotations\n\td.Describe(\"metadata\", metadataInfo)\n}\n\n// JSONEncode encodes d.output to json\nfunc (d *StructuredDescriber) JSONEncode() string {\n\tbyteBuffer := &bytes.Buffer{}\n\tencoder := json.NewEncoder(byteBuffer)\n\tencoder.SetEscapeHTML(false)\n\tencoder.SetIndent(\"\", \" \")\n\n\terr := encoder.Encode(d.output)\n\tif err != nil {\n\t\tfmt.Printf(\"fail to encode %s\", err.Error())\n\t\treturn \"\"\n\t}\n\treturn byteBuffer.String()\n}\n" }, { "path": "pkg/cmd/util/output/describe_test.go", "content": "package output\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"reflect\"\n\t\"testing\"\n\t\"text/tabwriter\"\n\n\t\"github.com/fatih/color\"\n\t\"github.com/stretchr/testify/assert\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\nfunc TestBoolPointerString(t *testing.T) {\n\ttrueStr := \"true\"\n\tfalseStr := \"FALSE\"\n\tnilStr := \"nul\"\n\ttruee := true\n\tfalsee := false\n\ttestcases := []struct {\n\t\tname string\n\t\tinput *bool\n\t\texpect string\n\t}{\n\t\t{\n\t\t\tname: \"nil\",\n\t\t\tinput: nil,\n\t\t\texpect: nilStr,\n\t\t},\n\t\t{\n\t\t\tname: \"true\",\n\t\t\tinput: &truee,\n\t\t\texpect: trueStr,\n\t\t},\n\t\t{\n\t\t\tname: \"false\",\n\t\t\tinput: &falsee,\n\t\t\texpect: falseStr,\n\t\t},\n\t}\n\tfor _, tc := range testcases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tgot := BoolPointerString(tc.input, falseStr, trueStr, nilStr)\n\t\t\tassert.Equal(t, tc.expect, got)\n\t\t})\n\t}\n}\n\nfunc TestDescriber_DescribeMetadata(t *testing.T) {\n\tinput := metav1.ObjectMeta{\n\t\tName: \"test\",\n\t\tNamespace: \"test-ns\",\n\t\tLabels: map[string]string{\n\t\t\t\"test-key2\": \"v2\",\n\t\t\t\"test-key0\": \"v0\",\n\t\t},\n\t\tAnnotations: nil,\n\t}\n\texpect := new(bytes.Buffer)\n\td := &Describer{\n\t\tPrefix: \"pref-\",\n\t\tout: &tabwriter.Writer{},\n\t\tbuf: &bytes.Buffer{},\n\t}\n\td.out.Init(d.buf, 0, 8, 0, ' ', 0)\n\tfmt.Fprintf(expect, \"pref-Name: %s\\n\", color.New(color.Bold).SprintFunc()(\"test\"))\n\tfmt.Fprintf(expect, \"pref-Namespace: %s\\n\", \"test-ns\")\n\tfmt.Fprintf(expect, \"pref-Labels: %s\\n\", \"pref-test-key0=v0\")\n\tfmt.Fprintf(expect, \"pref- test-key2=v2\\n\")\n\tfmt.Fprintf(expect, \"pref-Annotations:%s\\n\", \"pref-\")\n\td.DescribeMetadata(input)\n\td.out.Flush()\n\tassert.Equal(t, expect.String(), d.buf.String())\n}\n\nfunc TestDescriber_DescribeSlice(t *testing.T) {\n\tinput := []string{\"a\", \"b\", \"c\"}\n\texpect := new(bytes.Buffer)\n\td := &Describer{\n\t\tPrefix: \"pref-\",\n\t\tout: &tabwriter.Writer{},\n\t\tbuf: &bytes.Buffer{},\n\t}\n\td.out.Init(d.buf, 0, 8, 0, ' ', 0)\n\tfmt.Fprintf(expect, \"pref-test:pref-a\\n\")\n\tfmt.Fprintf(expect, \"pref- b\\n\")\n\tfmt.Fprintf(expect, \"pref- c\\n\")\n\td.DescribeSlice(4, \"test\", input)\n\td.out.Flush()\n\tassert.Equal(t, expect.String(), d.buf.String())\n\tvar input2 []string\n\texpect2 := new(bytes.Buffer)\n\td2 := &Describer{\n\t\tPrefix: \"pref-\",\n\t\tout: &tabwriter.Writer{},\n\t\tbuf: &bytes.Buffer{},\n\t}\n\td2.out.Init(d2.buf, 0, 4, 0, ' ', 0)\n\tfmt.Fprintf(expect2, \"pref-test:pref-\\n\")\n\td2.DescribeSlice(4, \"test\", input2)\n\td2.out.Flush()\n\tassert.Equal(t, expect2.String(), d2.buf.String())\n}\n\nfunc TestStructuredDescriber_JSONEncode(t *testing.T) {\n\ttestcases := []struct {\n\t\tname string\n\t\tinputMap map[string]any\n\t\texpect string\n\t}{\n\t\t{\n\t\t\tname: \"invalid json\",\n\t\t\tinputMap: map[string]any{},\n\t\t\texpect: \"{}\\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"valid json\",\n\t\t\tinputMap: map[string]any{\"k1\": \"v1\"},\n\t\t\texpect: `{\n \"k1\": \"v1\"\n}\n`,\n\t\t},\n\t}\n\tfor _, tc := range testcases {\n\t\tt.Run(tc.name, func(tt *testing.T) {\n\t\t\td := &StructuredDescriber{\n\t\t\t\toutput: tc.inputMap,\n\t\t\t}\n\t\t\tgot := d.JSONEncode()\n\t\t\tassert.Equal(tt, tc.expect, got)\n\t\t})\n\t}\n}\n\nfunc TestStructuredDescriber_DescribeMetadata(t *testing.T) {\n\td := NewStructuredDescriber(\"\")\n\tinput := metav1.ObjectMeta{\n\t\tName: \"test\",\n\t\tNamespace: \"test-ns\",\n\t\tLabels: map[string]string{\n\t\t\t\"label-1\": \"v1\",\n\t\t\t\"label-2\": \"v2\",\n\t\t},\n\t\tAnnotations: map[string]string{\n\t\t\t\"annotation-1\": \"v1\",\n\t\t\t\"annotation-2\": \"v2\",\n\t\t},\n\t}\n\texpect := map[string]any{\n\t\t\"metadata\": map[string]any{\n\t\t\t\"name\": \"test\",\n\t\t\t\"namespace\": \"test-ns\",\n\t\t\t\"labels\": map[string]string{\n\t\t\t\t\"label-1\": \"v1\",\n\t\t\t\t\"label-2\": \"v2\",\n\t\t\t},\n\t\t\t\"annotations\": map[string]string{\n\t\t\t\t\"annotation-1\": \"v1\",\n\t\t\t\t\"annotation-2\": \"v2\",\n\t\t\t},\n\t\t},\n\t}\n\td.DescribeMetadata(input)\n\n\tassert.True(t, reflect.DeepEqual(expect, d.output))\n}\n" }, { "path": "pkg/cmd/util/output/output.go", "content": "/*\nCopyright 2017, 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage output\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/spf13/cobra\"\n\t\"github.com/spf13/pflag\"\n\t\"k8s.io/apimachinery/pkg/api/meta\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/cli-runtime/pkg/printers\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/flag\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/encode\"\n)\n\nconst (\n\tdownloadRequestTimeout = 30 * time.Second\n\temptyDisplay = \"\"\n\tdefaultDataMover = \"velero\"\n)\n\n// BindFlags defines a set of output-specific flags within the provided\n// FlagSet.\nfunc BindFlags(flags *pflag.FlagSet) {\n\tflags.StringP(\"output\", \"o\", \"table\", \"Output display format. For create commands, display the object but do not send it to the server. Valid formats are 'table', 'json', and 'yaml'. 'table' is not valid for the install command.\")\n\tlabelColumns := flag.NewStringArray()\n\tflags.VarP(&labelColumns, \"label-columns\", \"L\", \"Accepts a comma separated list of labels that are going to be presented as columns. Names are case-sensitive. You can also use multiple flag options like -L label1 -L label2...\")\n\tflags.Bool(\"show-labels\", false, \"Show labels in the last column\")\n}\n\n// BindFlagsSimple defines the output format flag only.\nfunc BindFlagsSimple(flags *pflag.FlagSet) {\n\tflags.StringP(\"output\", \"o\", \"table\", \"Output display format. For create commands, display the object but do not send it to the server. Valid formats are 'table', 'json', and 'yaml'. 'table' is not valid for the install command.\")\n}\n\n// ClearOutputFlagDefault sets the current and default value\n// of the \"output\" flag to the empty string.\nfunc ClearOutputFlagDefault(cmd *cobra.Command) {\n\tf := cmd.Flag(\"output\")\n\tif f == nil {\n\t\treturn\n\t}\n\tf.DefValue = \"\"\n\tif err := f.Value.Set(\"\"); err != nil {\n\t\tfmt.Printf(\"error clear the default value of output flag: %s\\n\", err.Error())\n\t}\n}\n\n// GetOutputFlagValue returns the value of the \"output\" flag\n// in the provided command, or the zero value if not present.\nfunc GetOutputFlagValue(cmd *cobra.Command) string {\n\treturn flag.GetOptionalStringFlag(cmd, \"output\")\n}\n\n// GetLabelColumnsValues returns the value of the \"label-columns\" flag\n// in the provided command, or the zero value if not present.\nfunc GetLabelColumnsValues(cmd *cobra.Command) []string {\n\treturn flag.GetOptionalStringArrayFlag(cmd, \"label-columns\")\n}\n\n// GetShowLabelsValue returns the value of the \"show-labels\" flag\n// in the provided command, or the zero value if not present.\nfunc GetShowLabelsValue(cmd *cobra.Command) bool {\n\treturn flag.GetOptionalBoolFlag(cmd, \"show-labels\")\n}\n\n// ValidateFlags returns an error if any of the output-related flags\n// were specified with invalid values, or nil otherwise.\nfunc ValidateFlags(cmd *cobra.Command) error {\n\tif err := validateOutputFlag(cmd); err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n\nfunc validateOutputFlag(cmd *cobra.Command) error {\n\toutput := GetOutputFlagValue(cmd)\n\tswitch output {\n\tcase \"\", \"json\", \"yaml\":\n\tcase \"table\":\n\t\tif cmd.Name() == \"install\" {\n\t\t\treturn errors.New(\"'table' format is not supported with 'install' command\")\n\t\t}\n\tdefault:\n\t\treturn errors.Errorf(\"invalid output format %q - valid values are 'table', 'json', and 'yaml'\", output)\n\t}\n\treturn nil\n}\n\n// PrintWithFormat prints the provided object in the format specified by\n// the command's flags.\nfunc PrintWithFormat(c *cobra.Command, obj runtime.Object) (bool, error) {\n\tformat := GetOutputFlagValue(c)\n\tif format == \"\" {\n\t\treturn false, nil\n\t}\n\n\tswitch format {\n\tcase \"table\":\n\t\treturn printTable(c, obj)\n\tcase \"json\", \"yaml\":\n\t\treturn printEncoded(obj, format)\n\t}\n\n\treturn false, errors.Errorf(\"unsupported output format %q; valid values are 'table', 'json', and 'yaml'\", format)\n}\n\nfunc printEncoded(obj runtime.Object, format string) (bool, error) {\n\t// assume we're printing obj\n\ttoPrint := obj\n\n\tif meta.IsListType(obj) {\n\t\tlist, _ := meta.ExtractList(obj)\n\t\tif len(list) == 1 {\n\t\t\t// if obj was a list and there was only 1 item, just print that 1 instead of a list\n\t\t\ttoPrint = list[0]\n\t\t}\n\t}\n\n\tencoded, err := encode.Encode(toPrint, format)\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\tfmt.Println(string(encoded))\n\n\treturn true, nil\n}\n\nfunc printTable(cmd *cobra.Command, obj runtime.Object) (bool, error) {\n\t// 1. generate table\n\tvar table *metav1.Table\n\n\tswitch objType := obj.(type) {\n\tcase *velerov1api.Backup:\n\t\ttable = &metav1.Table{\n\t\t\tColumnDefinitions: backupColumns,\n\t\t\tRows: printBackup(objType),\n\t\t}\n\tcase *velerov1api.BackupList:\n\t\ttable = &metav1.Table{\n\t\t\tColumnDefinitions: backupColumns,\n\t\t\tRows: printBackupList(objType),\n\t\t}\n\tcase *velerov1api.Restore:\n\t\ttable = &metav1.Table{\n\t\t\tColumnDefinitions: restoreColumns,\n\t\t\tRows: printRestore(objType),\n\t\t}\n\tcase *velerov1api.RestoreList:\n\t\ttable = &metav1.Table{\n\t\t\tColumnDefinitions: restoreColumns,\n\t\t\tRows: printRestoreList(objType),\n\t\t}\n\tcase *velerov1api.Schedule:\n\t\ttable = &metav1.Table{\n\t\t\tColumnDefinitions: scheduleColumns,\n\t\t\tRows: printSchedule(objType),\n\t\t}\n\tcase *velerov1api.ScheduleList:\n\t\ttable = &metav1.Table{\n\t\t\tColumnDefinitions: scheduleColumns,\n\t\t\tRows: printScheduleList(objType),\n\t\t}\n\tcase *velerov1api.BackupRepository:\n\t\ttable = &metav1.Table{\n\t\t\tColumnDefinitions: backupRepoColumns,\n\t\t\tRows: printBackupRepo(objType),\n\t\t}\n\tcase *velerov1api.BackupRepositoryList:\n\t\ttable = &metav1.Table{\n\t\t\tColumnDefinitions: backupRepoColumns,\n\t\t\tRows: printBackupRepoList(objType),\n\t\t}\n\tcase *velerov1api.BackupStorageLocation:\n\t\ttable = &metav1.Table{\n\t\t\tColumnDefinitions: backupStorageLocationColumns,\n\t\t\tRows: printBackupStorageLocation(objType),\n\t\t}\n\tcase *velerov1api.BackupStorageLocationList:\n\t\ttable = &metav1.Table{\n\t\t\tColumnDefinitions: backupStorageLocationColumns,\n\t\t\tRows: printBackupStorageLocationList(objType),\n\t\t}\n\tcase *velerov1api.VolumeSnapshotLocation:\n\t\ttable = &metav1.Table{\n\t\t\tColumnDefinitions: volumeSnapshotLocationColumns,\n\t\t\tRows: printVolumeSnapshotLocation(objType),\n\t\t}\n\tcase *velerov1api.VolumeSnapshotLocationList:\n\t\ttable = &metav1.Table{\n\t\t\tColumnDefinitions: volumeSnapshotLocationColumns,\n\t\t\tRows: printVolumeSnapshotLocationList(objType),\n\t\t}\n\tcase *velerov1api.ServerStatusRequest:\n\t\ttable = &metav1.Table{\n\t\t\tColumnDefinitions: pluginColumns,\n\t\t\tRows: printPluginList(objType),\n\t\t}\n\tdefault:\n\t\treturn false, errors.Errorf(\"type %T is not supported\", obj)\n\t}\n\n\t// 2. print table\n\ttablePrinter, err := NewPrinter(cmd)\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\terr = tablePrinter.PrintObj(table, os.Stdout)\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\treturn true, nil\n}\n\n// NewPrinter returns a printer for doing human-readable table printing of\n// Velero objects.\nfunc NewPrinter(cmd *cobra.Command) (printers.ResourcePrinter, error) {\n\toptions := printers.PrintOptions{\n\t\tShowLabels: GetShowLabelsValue(cmd),\n\t\tColumnLabels: GetLabelColumnsValues(cmd),\n\t}\n\n\tprinter := printers.NewTablePrinter(options)\n\n\treturn printer, nil\n}\n" }, { "path": "pkg/cmd/util/output/output_test.go", "content": "package output\n\nimport (\n\t\"github.com/spf13/cobra\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tvelerov1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\n\t\"testing\"\n)\n\nfunc TestBindFlags(t *testing.T) {\n\tcmd := &cobra.Command{}\n\tBindFlags(cmd.Flags())\n\tassert.NotNil(t, cmd.Flags().Lookup(\"output\"))\n\tassert.NotNil(t, cmd.Flags().Lookup(\"label-columns\"))\n\tassert.NotNil(t, cmd.Flags().Lookup(\"show-labels\"))\n\tassert.Nil(t, cmd.Flags().Lookup(\"not-exist\"))\n}\n\nfunc TestBindFlagsSimple(t *testing.T) {\n\tcmd := &cobra.Command{}\n\tBindFlagsSimple(cmd.Flags())\n\tassert.NotNil(t, cmd.Flags().Lookup(\"output\"))\n\tassert.Nil(t, cmd.Flags().Lookup(\"label-columns\"))\n\tassert.Nil(t, cmd.Flags().Lookup(\"show-labels\"))\n}\n\nfunc TestClearOutputFlagDefault(t *testing.T) {\n\tcmd := &cobra.Command{}\n\tClearOutputFlagDefault(cmd)\n\tassert.Nil(t, cmd.Flags().Lookup(\"output\"))\n\tBindFlags(cmd.Flags())\n\tcmd.Flags().Set(\"output\", \"json\")\n\tClearOutputFlagDefault(cmd)\n\tassert.Empty(t, cmd.Flags().Lookup(\"output\").Value.String())\n}\n\nfunc cmdWithFormat(use string, format string) *cobra.Command {\n\tcmd := &cobra.Command{\n\t\tUse: use,\n\t}\n\tBindFlags(cmd.Flags())\n\tcmd.Flags().Set(\"output\", format)\n\treturn cmd\n}\n\nfunc TestValidateFlags(t *testing.T) {\n\ttestcases := []struct {\n\t\tname string\n\t\tinput *cobra.Command\n\t\thasErr bool\n\t}{\n\t\t{\n\t\t\tname: \"unknown format\",\n\t\t\tinput: cmdWithFormat(\"whatever\", \"unknown\"),\n\t\t\thasErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"json format\",\n\t\t\tinput: cmdWithFormat(\"whatever\", \"json\"),\n\t\t\thasErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"yaml format\",\n\t\t\tinput: cmdWithFormat(\"whatever\", \"yaml\"),\n\t\t\thasErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"empty format\",\n\t\t\tinput: cmdWithFormat(\"whatever\", \"\"),\n\t\t\thasErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"install with table format\",\n\t\t\tinput: cmdWithFormat(\"install\", \"table\"),\n\t\t\thasErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"other with table format\",\n\t\t\tinput: cmdWithFormat(\"other\", \"table\"),\n\t\t\thasErr: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testcases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\terr := ValidateFlags(tc.input)\n\t\t\tif tc.hasErr {\n\t\t\t\tassert.Error(t, err)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestPrintWithFormat(t *testing.T) {\n\ttestcases := []struct {\n\t\tname string\n\t\tinput struct {\n\t\t\tcmd *cobra.Command\n\t\t\tobj runtime.Object\n\t\t}\n\t\thasErr bool\n\t\tprinted bool\n\t}{\n\t\t{\n\t\t\tname: \"empty format\",\n\t\t\tinput: struct {\n\t\t\t\tcmd *cobra.Command\n\t\t\t\tobj runtime.Object\n\t\t\t}{\n\t\t\t\tcmd: cmdWithFormat(\"describe\", \"\"),\n\t\t\t},\n\t\t\thasErr: false,\n\t\t\tprinted: false,\n\t\t},\n\t\t{\n\t\t\tname: \"json format backup\",\n\t\t\tinput: struct {\n\t\t\t\tcmd *cobra.Command\n\t\t\t\tobj runtime.Object\n\t\t\t}{\n\t\t\t\tcmd: cmdWithFormat(\"describe\", \"json\"),\n\t\t\t\tobj: &velerov1.Backup{},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t\tprinted: true,\n\t\t},\n\t\t{\n\t\t\tname: \"table format backup\",\n\t\t\tinput: struct {\n\t\t\t\tcmd *cobra.Command\n\t\t\t\tobj runtime.Object\n\t\t\t}{\n\t\t\t\tcmd: cmdWithFormat(\"describe\", \"table\"),\n\t\t\t\tobj: &velerov1.Backup{},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t\tprinted: true,\n\t\t},\n\t\t{\n\t\t\tname: \"json format backup list\",\n\t\t\tinput: struct {\n\t\t\t\tcmd *cobra.Command\n\t\t\t\tobj runtime.Object\n\t\t\t}{\n\t\t\t\tcmd: cmdWithFormat(\"describe\", \"json\"),\n\t\t\t\tobj: &velerov1.BackupList{\n\t\t\t\t\tItems: []velerov1.Backup{\n\t\t\t\t\t\t{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t\tprinted: true,\n\t\t},\n\t\t{\n\t\t\tname: \"table format backup list\",\n\t\t\tinput: struct {\n\t\t\t\tcmd *cobra.Command\n\t\t\t\tobj runtime.Object\n\t\t\t}{\n\t\t\t\tcmd: cmdWithFormat(\"describe\", \"table\"),\n\t\t\t\tobj: &velerov1.BackupList{\n\t\t\t\t\tItems: []velerov1.Backup{\n\t\t\t\t\t\t{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t\tprinted: true,\n\t\t},\n\t\t{\n\t\t\tname: \"table format backup\",\n\t\t\tinput: struct {\n\t\t\t\tcmd *cobra.Command\n\t\t\t\tobj runtime.Object\n\t\t\t}{\n\t\t\t\tcmd: cmdWithFormat(\"describe\", \"table\"),\n\t\t\t\tobj: &velerov1.Backup{},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t\tprinted: true,\n\t\t},\n\t\t{\n\t\t\tname: \"table format restore list\",\n\t\t\tinput: struct {\n\t\t\t\tcmd *cobra.Command\n\t\t\t\tobj runtime.Object\n\t\t\t}{\n\t\t\t\tcmd: cmdWithFormat(\"describe\", \"table\"),\n\t\t\t\tobj: &velerov1.RestoreList{\n\t\t\t\t\tItems: []velerov1.Restore{\n\t\t\t\t\t\t{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t\tprinted: true,\n\t\t},\n\t\t{\n\t\t\tname: \"table format restore\",\n\t\t\tinput: struct {\n\t\t\t\tcmd *cobra.Command\n\t\t\t\tobj runtime.Object\n\t\t\t}{\n\t\t\t\tcmd: cmdWithFormat(\"describe\", \"table\"),\n\t\t\t\tobj: &velerov1.Restore{},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t\tprinted: true,\n\t\t},\n\t\t{\n\t\t\tname: \"table format schedule list\",\n\t\t\tinput: struct {\n\t\t\t\tcmd *cobra.Command\n\t\t\t\tobj runtime.Object\n\t\t\t}{\n\t\t\t\tcmd: cmdWithFormat(\"describe\", \"table\"),\n\t\t\t\tobj: &velerov1.ScheduleList{\n\t\t\t\t\tItems: []velerov1.Schedule{\n\t\t\t\t\t\t{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t\tprinted: true,\n\t\t},\n\t\t{\n\t\t\tname: \"table format schedule\",\n\t\t\tinput: struct {\n\t\t\t\tcmd *cobra.Command\n\t\t\t\tobj runtime.Object\n\t\t\t}{\n\t\t\t\tcmd: cmdWithFormat(\"describe\", \"table\"),\n\t\t\t\tobj: &velerov1.Schedule{},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t\tprinted: true,\n\t\t},\n\t\t{\n\t\t\tname: \"table format backup repository list\",\n\t\t\tinput: struct {\n\t\t\t\tcmd *cobra.Command\n\t\t\t\tobj runtime.Object\n\t\t\t}{\n\t\t\t\tcmd: cmdWithFormat(\"describe\", \"table\"),\n\t\t\t\tobj: &velerov1.BackupRepositoryList{\n\t\t\t\t\tItems: []velerov1.BackupRepository{\n\t\t\t\t\t\t{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t\tprinted: true,\n\t\t},\n\t\t{\n\t\t\tname: \"table format backup repository\",\n\t\t\tinput: struct {\n\t\t\t\tcmd *cobra.Command\n\t\t\t\tobj runtime.Object\n\t\t\t}{\n\t\t\t\tcmd: cmdWithFormat(\"describe\", \"table\"),\n\t\t\t\tobj: &velerov1.BackupRepository{},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t\tprinted: true,\n\t\t},\n\t\t{\n\t\t\tname: \"table format backup location list\",\n\t\t\tinput: struct {\n\t\t\t\tcmd *cobra.Command\n\t\t\t\tobj runtime.Object\n\t\t\t}{\n\t\t\t\tcmd: cmdWithFormat(\"describe\", \"table\"),\n\t\t\t\tobj: &velerov1.BackupStorageLocationList{\n\t\t\t\t\tItems: []velerov1.BackupStorageLocation{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tSpec: velerov1.BackupStorageLocationSpec{\n\t\t\t\t\t\t\t\tProvider: \"aws\",\n\t\t\t\t\t\t\t\tStorageType: velerov1.StorageType{\n\t\t\t\t\t\t\t\t\tObjectStorage: &velerov1.ObjectStorageLocation{\n\t\t\t\t\t\t\t\t\t\tBucket: \"bucket\",\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t\tprinted: true,\n\t\t},\n\t\t{\n\t\t\tname: \"table format backup location\",\n\t\t\tinput: struct {\n\t\t\t\tcmd *cobra.Command\n\t\t\t\tobj runtime.Object\n\t\t\t}{\n\t\t\t\tcmd: cmdWithFormat(\"describe\", \"table\"),\n\t\t\t\tobj: &velerov1.BackupStorageLocation{\n\t\t\t\t\tSpec: velerov1.BackupStorageLocationSpec{\n\t\t\t\t\t\tProvider: \"aws\",\n\t\t\t\t\t\tStorageType: velerov1.StorageType{\n\t\t\t\t\t\t\tObjectStorage: &velerov1.ObjectStorageLocation{\n\t\t\t\t\t\t\t\tBucket: \"bucket\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t\tprinted: true,\n\t\t},\n\t\t{\n\t\t\tname: \"table format volume snapshot location list\",\n\t\t\tinput: struct {\n\t\t\t\tcmd *cobra.Command\n\t\t\t\tobj runtime.Object\n\t\t\t}{\n\t\t\t\tcmd: cmdWithFormat(\"describe\", \"table\"),\n\t\t\t\tobj: &velerov1.VolumeSnapshotLocationList{\n\t\t\t\t\tItems: []velerov1.VolumeSnapshotLocation{\n\t\t\t\t\t\t{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t\tprinted: true,\n\t\t},\n\t\t{\n\t\t\tname: \"table format volume snapshot location\",\n\t\t\tinput: struct {\n\t\t\t\tcmd *cobra.Command\n\t\t\t\tobj runtime.Object\n\t\t\t}{\n\t\t\t\tcmd: cmdWithFormat(\"describe\", \"table\"),\n\t\t\t\tobj: &velerov1.VolumeSnapshotLocation{},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t\tprinted: true,\n\t\t},\n\t\t{\n\t\t\tname: \"table format volume snapshot location\",\n\t\t\tinput: struct {\n\t\t\t\tcmd *cobra.Command\n\t\t\t\tobj runtime.Object\n\t\t\t}{\n\t\t\t\tcmd: cmdWithFormat(\"describe\", \"table\"),\n\t\t\t\tobj: &velerov1.VolumeSnapshotLocation{},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t\tprinted: true,\n\t\t},\n\t\t{\n\t\t\tname: \"table format plugin list via server status\",\n\t\t\tinput: struct {\n\t\t\t\tcmd *cobra.Command\n\t\t\t\tobj runtime.Object\n\t\t\t}{\n\t\t\t\tcmd: cmdWithFormat(\"describe\", \"table\"),\n\t\t\t\tobj: &velerov1.ServerStatusRequest{},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t\tprinted: true,\n\t\t},\n\t}\n\tfor _, tc := range testcases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tp, err := PrintWithFormat(tc.input.cmd, tc.input.obj)\n\t\t\tif tc.hasErr {\n\t\t\t\trequire.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\t\t\tassert.Equal(t, tc.printed, p)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/cmd/util/output/plugin_printer.go", "content": "/*\nCopyright 2019, 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage output\n\nimport (\n\t\"sort\"\n\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\nvar (\n\tpluginColumns = []metav1.TableColumnDefinition{\n\t\t// name needs Type and Format defined for the decorator to identify it:\n\t\t// https://github.com/kubernetes/kubernetes/blob/v1.15.3/pkg/printers/tableprinter.go#L204\n\t\t{Name: \"Name\", Type: \"string\", Format: \"name\"},\n\t\t{Name: \"Kind\"},\n\t}\n)\n\nfunc printPluginList(list *velerov1api.ServerStatusRequest) []metav1.TableRow {\n\tplugins := list.Status.Plugins\n\tsortByKindAndName(plugins)\n\n\trows := make([]metav1.TableRow, 0, len(plugins))\n\n\tfor _, plugin := range plugins {\n\t\trows = append(rows, printPlugin(plugin)...)\n\t}\n\treturn rows\n}\n\nfunc sortByKindAndName(plugins []velerov1api.PluginInfo) {\n\tsort.Slice(plugins, func(i, j int) bool {\n\t\tif plugins[i].Kind != plugins[j].Kind {\n\t\t\treturn plugins[i].Kind < plugins[j].Kind\n\t\t}\n\t\treturn plugins[i].Name < plugins[j].Name\n\t})\n}\n\nfunc printPlugin(plugin velerov1api.PluginInfo) []metav1.TableRow {\n\trow := metav1.TableRow{}\n\n\trow.Cells = append(row.Cells, plugin.Name, plugin.Kind)\n\n\treturn []metav1.TableRow{row}\n}\n" }, { "path": "pkg/cmd/util/output/restore_describer.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage output\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"sort\"\n\t\"strings\"\n\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/fatih/color\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/cacert\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/downloadrequest\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/results\"\n)\n\nfunc DescribeRestore(\n\tctx context.Context,\n\tkbClient kbclient.Client,\n\trestore *velerov1api.Restore,\n\tpodVolumeRestores []velerov1api.PodVolumeRestore,\n\tdetails bool,\n\tinsecureSkipTLSVerify bool,\n\tcaCertFile string,\n) string {\n\treturn Describe(func(d *Describer) {\n\t\td.DescribeMetadata(restore.ObjectMeta)\n\n\t\td.Println()\n\t\tphase := restore.Status.Phase\n\t\tif phase == \"\" {\n\t\t\tphase = velerov1api.RestorePhaseNew\n\t\t}\n\t\tphaseString := string(phase)\n\n\t\t// Append \"Deleting\" to phaseString if deletionTimestamp is marked.\n\t\tif !restore.DeletionTimestamp.IsZero() {\n\t\t\tphaseString += \" (Deleting)\"\n\t\t}\n\n\t\tswitch phase {\n\t\tcase velerov1api.RestorePhaseCompleted:\n\t\t\tphaseString = color.GreenString(phaseString)\n\t\tcase velerov1api.RestorePhaseFailedValidation, velerov1api.RestorePhasePartiallyFailed, velerov1api.RestorePhaseFailed:\n\t\t\tphaseString = color.RedString(phaseString)\n\t\t}\n\n\t\tresultsNote := \"\"\n\t\tif phase == velerov1api.RestorePhaseFailed || phase == velerov1api.RestorePhasePartiallyFailed {\n\t\t\tresultsNote = fmt.Sprintf(\" (run 'velero restore logs %s' for more information)\", restore.Name)\n\t\t}\n\n\t\td.Printf(\"Phase:\\t%s%s\\n\", phaseString, resultsNote)\n\t\tif restore.Status.Progress != nil {\n\t\t\tif restore.Status.Phase == velerov1api.RestorePhaseInProgress {\n\t\t\t\td.Printf(\"Estimated total items to be restored:\\t%d\\n\", restore.Status.Progress.TotalItems)\n\t\t\t\td.Printf(\"Items restored so far:\\t%d\\n\", restore.Status.Progress.ItemsRestored)\n\t\t\t} else {\n\t\t\t\td.Printf(\"Total items to be restored:\\t%d\\n\", restore.Status.Progress.TotalItems)\n\t\t\t\td.Printf(\"Items restored:\\t%d\\n\", restore.Status.Progress.ItemsRestored)\n\t\t\t}\n\t\t}\n\n\t\td.Println()\n\t\t// \"\" output should only be applicable for restore that failed validation\n\t\tif restore.Status.StartTimestamp == nil || restore.Status.StartTimestamp.IsZero() {\n\t\t\td.Printf(\"Started:\\t%s\\n\", \"\")\n\t\t} else {\n\t\t\td.Printf(\"Started:\\t%s\\n\", restore.Status.StartTimestamp)\n\t\t}\n\n\t\tif restore.Status.CompletionTimestamp == nil || restore.Status.CompletionTimestamp.IsZero() {\n\t\t\td.Printf(\"Completed:\\t%s\\n\", \"\")\n\t\t} else {\n\t\t\td.Printf(\"Completed:\\t%s\\n\", restore.Status.CompletionTimestamp)\n\t\t}\n\n\t\tif len(restore.Status.ValidationErrors) > 0 {\n\t\t\td.Println()\n\t\t\td.Printf(\"Validation errors:\")\n\t\t\tfor _, ve := range restore.Status.ValidationErrors {\n\t\t\t\td.Printf(\"\\t%s\\n\", color.RedString(ve))\n\t\t\t}\n\t\t}\n\n\t\tdescribeRestoreResults(ctx, kbClient, d, restore, insecureSkipTLSVerify, caCertFile)\n\n\t\td.Println()\n\t\td.Printf(\"Backup:\\t%s\\n\", restore.Spec.BackupName)\n\n\t\td.Println()\n\t\td.Printf(\"Namespaces:\\n\")\n\t\tvar s string\n\t\tif len(restore.Spec.IncludedNamespaces) == 0 {\n\t\t\ts = \"all namespaces found in the backup\"\n\t\t} else if len(restore.Spec.IncludedNamespaces) == 1 && restore.Spec.IncludedNamespaces[0] == \"*\" {\n\t\t\ts = \"all namespaces found in the backup\"\n\t\t} else {\n\t\t\ts = strings.Join(restore.Spec.IncludedNamespaces, \", \")\n\t\t}\n\t\td.Printf(\"\\tIncluded:\\t%s\\n\", s)\n\t\tif len(restore.Spec.ExcludedNamespaces) == 0 {\n\t\t\ts = emptyDisplay\n\t\t} else {\n\t\t\ts = strings.Join(restore.Spec.ExcludedNamespaces, \", \")\n\t\t}\n\t\td.Printf(\"\\tExcluded:\\t%s\\n\", s)\n\n\t\td.Println()\n\t\td.Printf(\"Resources:\\n\")\n\t\tif len(restore.Spec.IncludedResources) == 0 {\n\t\t\ts = \"*\"\n\t\t} else {\n\t\t\ts = strings.Join(restore.Spec.IncludedResources, \", \")\n\t\t}\n\t\td.Printf(\"\\tIncluded:\\t%s\\n\", s)\n\t\tif len(restore.Spec.ExcludedResources) == 0 {\n\t\t\ts = emptyDisplay\n\t\t} else {\n\t\t\ts = strings.Join(restore.Spec.ExcludedResources, \", \")\n\t\t}\n\t\td.Printf(\"\\tExcluded:\\t%s\\n\", s)\n\n\t\td.Printf(\"\\tCluster-scoped:\\t%s\\n\", BoolPointerString(restore.Spec.IncludeClusterResources, \"excluded\", \"included\", \"auto\"))\n\n\t\td.Println()\n\t\td.DescribeMap(\"Namespace mappings\", restore.Spec.NamespaceMapping)\n\n\t\td.Println()\n\t\ts = emptyDisplay\n\t\tif restore.Spec.LabelSelector != nil {\n\t\t\ts = metav1.FormatLabelSelector(restore.Spec.LabelSelector)\n\t\t}\n\t\td.Printf(\"Label selector:\\t%s\\n\", s)\n\n\t\td.Println()\n\t\tif len(restore.Spec.OrLabelSelectors) == 0 {\n\t\t\ts = emptyDisplay\n\t\t} else {\n\t\t\torLabelSelectors := []string{}\n\t\t\tfor _, v := range restore.Spec.OrLabelSelectors {\n\t\t\t\torLabelSelectors = append(orLabelSelectors, metav1.FormatLabelSelector(v))\n\t\t\t}\n\t\t\ts = strings.Join(orLabelSelectors, \" or \")\n\t\t}\n\t\td.Printf(\"Or label selector:\\t%s\\n\", s)\n\n\t\td.Println()\n\t\td.Printf(\"Restore PVs:\\t%s\\n\", BoolPointerString(restore.Spec.RestorePVs, \"false\", \"true\", \"auto\"))\n\n\t\tif len(podVolumeRestores) > 0 {\n\t\t\td.Println()\n\t\t\tdescribePodVolumeRestores(d, podVolumeRestores, details)\n\t\t}\n\n\t\t// Get BSL cacert if available\n\t\tbslCACert, err := cacert.GetCACertFromRestore(ctx, kbClient, restore.Namespace, restore)\n\t\tif err != nil {\n\t\t\t// Log the error but don't fail - we can still try to download without the BSL cacert\n\t\t\td.Printf(\"WARNING: Error getting cacert from BSL: %v\\n\", err)\n\t\t\tbslCACert = \"\"\n\t\t}\n\n\t\tbuf := new(bytes.Buffer)\n\t\tif err := downloadrequest.StreamWithBSLCACert(ctx, kbClient, restore.Namespace, restore.Name, velerov1api.DownloadTargetKindRestoreVolumeInfo,\n\t\t\tbuf, downloadRequestTimeout, insecureSkipTLSVerify, caCertFile, bslCACert); err == nil {\n\t\t\tvar restoreVolInfo []volume.RestoreVolumeInfo\n\t\t\tif err := json.NewDecoder(buf).Decode(&restoreVolInfo); err != nil {\n\t\t\t\td.Printf(\"\\t\\n\", err)\n\t\t\t} else {\n\t\t\t\tdescribeCSISnapshotsRestores(d, restoreVolInfo, details)\n\t\t\t}\n\t\t} else if err != nil && !errors.Is(err, downloadrequest.ErrNotFound) {\n\t\t\t// For the restores by older versions of velero, it will see NotFound Error when downloading the volume info.\n\t\t\t// In that case, no errors will be printed.\n\t\t\td.Printf(\"\\t\\n\", err)\n\t\t}\n\n\t\td.Println()\n\t\ts = emptyDisplay\n\t\tif restore.Spec.ExistingResourcePolicy != \"\" {\n\t\t\ts = string(restore.Spec.ExistingResourcePolicy)\n\t\t}\n\t\td.Printf(\"Existing Resource Policy: \\t%s\\n\", s)\n\t\td.Printf(\"ItemOperationTimeout:\\t%s\\n\", restore.Spec.ItemOperationTimeout.Duration)\n\n\t\td.Println()\n\t\td.Printf(\"Preserve Service NodePorts:\\t%s\\n\", BoolPointerString(restore.Spec.PreserveNodePorts, \"false\", \"true\", \"auto\"))\n\n\t\tif restore.Spec.ResourceModifier != nil {\n\t\t\td.Println()\n\t\t\tDescribeResourceModifier(d, restore.Spec.ResourceModifier)\n\t\t}\n\n\t\tdescribeUploaderConfigForRestore(d, restore.Spec)\n\n\t\td.Println()\n\t\tdescribeRestoreItemOperations(ctx, kbClient, d, restore, details, insecureSkipTLSVerify, caCertFile)\n\n\t\tif restore.Status.HookStatus != nil {\n\t\t\td.Println()\n\t\t\td.Printf(\"HooksAttempted: \\t%d\\n\", restore.Status.HookStatus.HooksAttempted)\n\t\t\td.Printf(\"HooksFailed: \\t%d\\n\", restore.Status.HookStatus.HooksFailed)\n\t\t}\n\n\t\tif details {\n\t\t\td.Println()\n\t\t\tdescribeRestoreResourceList(ctx, kbClient, d, restore, insecureSkipTLSVerify, caCertFile)\n\t\t}\n\t})\n}\n\n// describeUploaderConfigForRestore describes uploader config in human-readable format\nfunc describeUploaderConfigForRestore(d *Describer, spec velerov1api.RestoreSpec) {\n\tif spec.UploaderConfig != nil {\n\t\td.Println()\n\t\td.Printf(\"Uploader config:\\n\")\n\t\tif boolptr.IsSetToTrue(spec.UploaderConfig.WriteSparseFiles) {\n\t\t\td.Printf(\"\\tWrite Sparse Files:\\t%v\\n\", boolptr.IsSetToTrue(spec.UploaderConfig.WriteSparseFiles))\n\t\t}\n\t\tif spec.UploaderConfig.ParallelFilesDownload > 0 {\n\t\t\td.Printf(\"\\tParallel Restore:\\t%d\\n\", spec.UploaderConfig.ParallelFilesDownload)\n\t\t}\n\t}\n}\n\nfunc describeRestoreItemOperations(ctx context.Context, kbClient kbclient.Client, d *Describer, restore *velerov1api.Restore, details bool, insecureSkipTLSVerify bool, caCertPath string) {\n\tstatus := restore.Status\n\tif status.RestoreItemOperationsAttempted > 0 {\n\t\tif !details {\n\t\t\td.Printf(\"Restore Item Operations:\\t%d of %d completed successfully, %d failed (specify --details for more information)\\n\", status.RestoreItemOperationsCompleted, status.RestoreItemOperationsAttempted, status.RestoreItemOperationsFailed)\n\t\t\treturn\n\t\t}\n\n\t\t// Get BSL cacert if available\n\t\tbslCACert, err := cacert.GetCACertFromRestore(ctx, kbClient, restore.Namespace, restore)\n\t\tif err != nil {\n\t\t\t// Log the error but don't fail - we can still try to download without the BSL cacert\n\t\t\td.Printf(\"WARNING: Error getting cacert from BSL: %v\\n\", err)\n\t\t\tbslCACert = \"\"\n\t\t}\n\n\t\tbuf := new(bytes.Buffer)\n\t\tif err := downloadrequest.StreamWithBSLCACert(ctx, kbClient, restore.Namespace, restore.Name, velerov1api.DownloadTargetKindRestoreItemOperations, buf, downloadRequestTimeout, insecureSkipTLSVerify, caCertPath, bslCACert); err != nil {\n\t\t\td.Printf(\"Restore Item Operations:\\t\\n\", err)\n\t\t\treturn\n\t\t}\n\n\t\tvar operations []*itemoperation.RestoreOperation\n\t\tif err := json.NewDecoder(buf).Decode(&operations); err != nil {\n\t\t\td.Printf(\"Restore Item Operations:\\t\\n\", err)\n\t\t\treturn\n\t\t}\n\n\t\td.Printf(\"Restore Item Operations:\\n\")\n\t\tfor _, operation := range operations {\n\t\t\tdescribeRestoreItemOperation(d, operation)\n\t\t}\n\t}\n}\n\nfunc describeRestoreResults(ctx context.Context, kbClient kbclient.Client, d *Describer, restore *velerov1api.Restore, insecureSkipTLSVerify bool, caCertPath string) {\n\tif restore.Status.Warnings == 0 && restore.Status.Errors == 0 {\n\t\treturn\n\t}\n\n\t// Get BSL cacert if available\n\tbslCACert, err := cacert.GetCACertFromRestore(ctx, kbClient, restore.Namespace, restore)\n\tif err != nil {\n\t\t// Log the error but don't fail - we can still try to download without the BSL cacert\n\t\td.Printf(\"WARNING: Error getting cacert from BSL: %v\\n\", err)\n\t\tbslCACert = \"\"\n\t}\n\n\tvar buf bytes.Buffer\n\tvar resultMap map[string]results.Result\n\n\tif err := downloadrequest.StreamWithBSLCACert(ctx, kbClient, restore.Namespace, restore.Name, velerov1api.DownloadTargetKindRestoreResults, &buf, downloadRequestTimeout, insecureSkipTLSVerify, caCertPath, bslCACert); err != nil {\n\t\td.Printf(\"Warnings:\\t\\n\\nErrors:\\t\\n\", err, err)\n\t\treturn\n\t}\n\n\tif err := json.NewDecoder(&buf).Decode(&resultMap); err != nil {\n\t\td.Printf(\"Warnings:\\t\\n\\nErrors:\\t\\n\", err, err)\n\t\treturn\n\t}\n\n\tif restore.Status.Warnings > 0 {\n\t\td.Println()\n\t\tdescribeResult(d, \"Warnings\", resultMap[\"warnings\"])\n\t}\n\tif restore.Status.Errors > 0 {\n\t\td.Println()\n\t\tdescribeResult(d, \"Errors\", resultMap[\"errors\"])\n\t}\n}\n\nfunc describeResult(d *Describer, name string, result results.Result) {\n\td.Printf(\"%s:\\n\", name)\n\td.DescribeSlice(1, \"Velero\", result.Velero)\n\td.DescribeSlice(1, \"Cluster\", result.Cluster)\n\tif len(result.Namespaces) == 0 {\n\t\td.Printf(\"\\tNamespaces: \\n\")\n\t} else {\n\t\td.Printf(\"\\tNamespaces:\\n\")\n\t\tfor ns, warnings := range result.Namespaces {\n\t\t\td.DescribeSlice(2, ns, warnings)\n\t\t}\n\t}\n}\n\nfunc describeRestoreItemOperation(d *Describer, operation *itemoperation.RestoreOperation) {\n\td.Printf(\"\\tOperation for %s %s/%s:\\n\", operation.Spec.ResourceIdentifier, operation.Spec.ResourceIdentifier.Namespace, operation.Spec.ResourceIdentifier.Name)\n\td.Printf(\"\\t\\tRestore Item Action Plugin:\\t%s\\n\", operation.Spec.RestoreItemAction)\n\td.Printf(\"\\t\\tOperation ID:\\t%s\\n\", operation.Spec.OperationID)\n\td.Printf(\"\\t\\tPhase:\\t%s\\n\", operation.Status.Phase)\n\tif operation.Status.Error != \"\" {\n\t\td.Printf(\"\\t\\tOperation Error:\\t%s\\n\", operation.Status.Error)\n\t}\n\tif operation.Status.NTotal > 0 || operation.Status.NCompleted > 0 {\n\t\td.Printf(\"\\t\\tProgress:\\t%v of %v complete (%s)\\n\",\n\t\t\toperation.Status.NCompleted,\n\t\t\toperation.Status.NTotal,\n\t\t\toperation.Status.OperationUnits)\n\t}\n\tif operation.Status.Description != \"\" {\n\t\td.Printf(\"\\t\\tProgress description:\\t%s\\n\", operation.Status.Description)\n\t}\n\tif operation.Status.Created != nil {\n\t\td.Printf(\"\\t\\tCreated:\\t%s\\n\", operation.Status.Created.String())\n\t}\n\tif operation.Status.Started != nil {\n\t\td.Printf(\"\\t\\tStarted:\\t%s\\n\", operation.Status.Started.String())\n\t}\n\tif operation.Status.Updated != nil {\n\t\td.Printf(\"\\t\\tUpdated:\\t%s\\n\", operation.Status.Updated.String())\n\t}\n}\n\n// describePodVolumeRestores describes pod volume restores in human-readable format.\nfunc describePodVolumeRestores(d *Describer, restores []velerov1api.PodVolumeRestore, details bool) {\n\t// Get the type of pod volume uploader. Since the uploader only comes from a single source, we can\n\t// take the uploader type from the first element of the array.\n\tvar uploaderType string\n\tif len(restores) > 0 {\n\t\tuploaderType = restores[0].Spec.UploaderType\n\t} else {\n\t\treturn\n\t}\n\n\tif details {\n\t\td.Printf(\"%s Restores:\\n\", uploaderType)\n\t} else {\n\t\td.Printf(\"%s Restores (specify --details for more information):\\n\", uploaderType)\n\t}\n\n\t// separate restores by phase (combining and New into a single group)\n\trestoresByPhase := groupRestoresByPhase(restores)\n\n\t// go through phases in a specific order\n\tfor _, phase := range []string{\n\t\tstring(velerov1api.PodVolumeRestorePhaseCompleted),\n\t\tstring(velerov1api.PodVolumeRestorePhaseCanceled),\n\t\tstring(velerov1api.PodVolumeRestorePhaseFailed),\n\t\t\"In Progress\",\n\t\tstring(velerov1api.PodVolumeRestorePhasePrepared),\n\t\tstring(velerov1api.PodVolumeRestorePhaseAccepted),\n\t\tstring(velerov1api.PodVolumeRestorePhaseNew),\n\t} {\n\t\tif len(restoresByPhase[phase]) == 0 {\n\t\t\tcontinue\n\t\t}\n\n\t\t// if we're not printing details, just report the phase and count\n\t\tif !details {\n\t\t\td.Printf(\"\\t%s:\\t%d\\n\", phase, len(restoresByPhase[phase]))\n\t\t\tcontinue\n\t\t}\n\n\t\t// group the restores in the current phase by pod (i.e. \"ns/name\")\n\t\trestoresByPod := new(volumesByPod)\n\n\t\tfor _, restore := range restoresByPhase[phase] {\n\t\t\trestoresByPod.Add(restore.Spec.Pod.Namespace, restore.Spec.Pod.Name, restore.Spec.Volume, phase, restore.Status.Progress, 0)\n\t\t}\n\n\t\td.Printf(\"\\t%s:\\n\", phase)\n\t\tfor _, restoreGroup := range restoresByPod.Sorted() {\n\t\t\tsort.Strings(restoreGroup.volumes)\n\n\t\t\t// print volumes restored up for this pod\n\t\t\td.Printf(\"\\t\\t%s: %s\\n\", restoreGroup.label, strings.Join(restoreGroup.volumes, \", \"))\n\t\t}\n\t}\n}\n\n// describeCSISnapshotsRestores describes PVC restored via CSISnapshots, incl. data-movement, in human-readable format.\nfunc describeCSISnapshotsRestores(d *Describer, restoreVolInfo []volume.RestoreVolumeInfo, details bool) {\n\td.Println()\n\tvar nonDMInfoList, dmInfoList []volume.RestoreVolumeInfo\n\tfor _, info := range restoreVolInfo {\n\t\tif info.RestoreMethod != volume.CSISnapshot {\n\t\t\tcontinue\n\t\t}\n\t\tif info.SnapshotDataMoved {\n\t\t\tdmInfoList = append(dmInfoList, info)\n\t\t} else {\n\t\t\tnonDMInfoList = append(nonDMInfoList, info)\n\t\t}\n\t}\n\tif len(nonDMInfoList) == 0 && len(dmInfoList) == 0 {\n\t\td.Printf(\"CSI Snapshot Restores: \\n\")\n\t\treturn\n\t}\n\td.Printf(\"CSI Snapshot Restores:\\n\")\n\tfor _, info := range nonDMInfoList {\n\t\t// All CSI snapshots are restored via PVC\n\t\td.Printf(\"\\t%s/%s:\\n\", info.PVCNamespace, info.PVCName)\n\t\tif details {\n\t\t\td.Printf(\"\\t\\tSnapshot:\\n\")\n\t\t\td.Printf(\"\\t\\t\\tSnapshot Content Name: %s\\n\", info.CSISnapshotInfo.VSCName)\n\t\t\td.Printf(\"\\t\\t\\tStorage Snapshot ID: %s\\n\", info.CSISnapshotInfo.SnapshotHandle)\n\t\t\td.Printf(\"\\t\\t\\tCSI Driver: %s\\n\", info.CSISnapshotInfo.Driver)\n\t\t} else {\n\t\t\td.Printf(\"\\t\\tSnapshot: specify --details for more information\\n\")\n\t\t}\n\t}\n\tfor _, info := range dmInfoList {\n\t\td.Printf(\"\\t%s/%s:\\n\", info.PVCNamespace, info.PVCName)\n\t\tif details {\n\t\t\td.Printf(\"\\t\\tData Movement:\\n\")\n\t\t\td.Printf(\"\\t\\t\\tOperation ID: %s\\n\", info.SnapshotDataMovementInfo.OperationID)\n\t\t\td.Printf(\"\\t\\t\\tData Mover: %s\\n\", info.SnapshotDataMovementInfo.DataMover)\n\t\t\td.Printf(\"\\t\\t\\tUploader Type: %s\\n\", info.SnapshotDataMovementInfo.UploaderType)\n\t\t} else {\n\t\t\td.Printf(\"\\t\\tData Movement: specify --details for more information\\n\")\n\t\t}\n\t}\n}\n\nfunc groupRestoresByPhase(restores []velerov1api.PodVolumeRestore) map[string][]velerov1api.PodVolumeRestore {\n\trestoresByPhase := make(map[string][]velerov1api.PodVolumeRestore)\n\n\tphaseToGroup := map[velerov1api.PodVolumeRestorePhase]string{\n\t\tvelerov1api.PodVolumeRestorePhaseCompleted: string(velerov1api.PodVolumeRestorePhaseCompleted),\n\t\tvelerov1api.PodVolumeRestorePhaseCanceled: string(velerov1api.PodVolumeRestorePhaseCanceled),\n\t\tvelerov1api.PodVolumeRestorePhaseFailed: string(velerov1api.PodVolumeRestorePhaseFailed),\n\t\tvelerov1api.PodVolumeRestorePhaseInProgress: \"In Progress\",\n\t\tvelerov1api.PodVolumeRestorePhasePrepared: string(velerov1api.PodVolumeRestorePhasePrepared),\n\t\tvelerov1api.PodVolumeRestorePhaseAccepted: string(velerov1api.PodVolumeRestorePhaseAccepted),\n\t\tvelerov1api.PodVolumeRestorePhaseNew: string(velerov1api.PodVolumeRestorePhaseNew),\n\t\t\"\": string(velerov1api.PodVolumeRestorePhaseNew),\n\t}\n\n\tfor _, restore := range restores {\n\t\tgroup := phaseToGroup[restore.Status.Phase]\n\t\trestoresByPhase[group] = append(restoresByPhase[group], restore)\n\t}\n\n\treturn restoresByPhase\n}\n\nfunc describeRestoreResourceList(ctx context.Context, kbClient kbclient.Client, d *Describer, restore *velerov1api.Restore, insecureSkipTLSVerify bool, caCertPath string) {\n\t// Get BSL cacert if available\n\tbslCACert, err := cacert.GetCACertFromRestore(ctx, kbClient, restore.Namespace, restore)\n\tif err != nil {\n\t\t// Log the error but don't fail - we can still try to download without the BSL cacert\n\t\td.Printf(\"WARNING: Error getting cacert from BSL: %v\\n\", err)\n\t\tbslCACert = \"\"\n\t}\n\n\tbuf := new(bytes.Buffer)\n\tif err := downloadrequest.StreamWithBSLCACert(ctx, kbClient, restore.Namespace, restore.Name, velerov1api.DownloadTargetKindRestoreResourceList, buf, downloadRequestTimeout, insecureSkipTLSVerify, caCertPath, bslCACert); err != nil {\n\t\tif err == downloadrequest.ErrNotFound {\n\t\t\td.Println(\"Resource List:\\t\")\n\t\t} else {\n\t\t\td.Printf(\"Resource List:\\t\\n\", err)\n\t\t}\n\t\treturn\n\t}\n\n\tvar resourceList map[string][]string\n\tif err := json.NewDecoder(buf).Decode(&resourceList); err != nil {\n\t\td.Printf(\"Resource List:\\t\\n\", err)\n\t\treturn\n\t}\n\n\td.Println(\"Resource List:\")\n\n\t// Sort GVKs in output\n\tgvks := make([]string, 0, len(resourceList))\n\tfor gvk := range resourceList {\n\t\tgvks = append(gvks, gvk)\n\t}\n\tsort.Strings(gvks)\n\n\tfor _, gvk := range gvks {\n\t\td.Printf(\"\\t%s:\\n\\t\\t- %s\\n\", gvk, strings.Join(resourceList[gvk], \"\\n\\t\\t- \"))\n\t}\n}\n\n// DescribeResourceModifier describes resource policies in human-readable format\nfunc DescribeResourceModifier(d *Describer, resModifier *corev1api.TypedLocalObjectReference) {\n\td.Printf(\"Resource modifier:\\n\")\n\td.Printf(\"\\tType:\\t%s\\n\", resModifier.Kind)\n\td.Printf(\"\\tName:\\t%s\\n\", resModifier.Name)\n}\n" }, { "path": "pkg/cmd/util/output/restore_describer_test.go", "content": "package output\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"testing\"\n\t\"text/tabwriter\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/results\"\n)\n\nfunc TestDescribeResult(t *testing.T) {\n\ttestcases := []struct {\n\t\tname string\n\t\tinputName string\n\t\tinputResult results.Result\n\t\texpect string\n\t}{\n\t\t{\n\t\t\tname: \"result without ns warns\",\n\t\t\tinputName: \"restore-1\",\n\t\t\tinputResult: results.Result{\n\t\t\t\tVelero: []string{\"velero-msg-1\", \"velero-msg-2\"},\n\t\t\t\tCluster: []string{\"cluster-msg-1\", \"cluster-msg-2\"},\n\t\t\t\tNamespaces: map[string][]string{},\n\t\t\t},\n\t\t\texpect: `restore-1:\n Velero: velero-msg-1\n velero-msg-2\n Cluster: cluster-msg-1\n cluster-msg-2\n Namespaces: \n`,\n\t\t},\n\t\t{\n\t\t\tname: \"result with ns warns\",\n\t\t\tinputName: \"restore-2\",\n\t\t\tinputResult: results.Result{\n\t\t\t\tVelero: []string{\"velero-msg-1\", \"velero-msg-2\"},\n\t\t\t\tCluster: []string{\"cluster-msg-1\", \"cluster-msg-2\"},\n\t\t\t\tNamespaces: map[string][]string{\n\t\t\t\t\t\"ns-1\": {\"ns-1-warn-1\", \"ns-1-warn-2\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpect: `restore-2:\n Velero: velero-msg-1\n velero-msg-2\n Cluster: cluster-msg-1\n cluster-msg-2\n Namespaces:\n ns-1: ns-1-warn-1\n ns-1-warn-2\n`,\n\t\t},\n\t}\n\tfor _, tc := range testcases {\n\t\tt.Run(tc.name, func(tt *testing.T) {\n\t\t\td := &Describer{\n\t\t\t\tPrefix: \"\",\n\t\t\t\tout: &tabwriter.Writer{},\n\t\t\t\tbuf: &bytes.Buffer{},\n\t\t\t}\n\t\t\td.out.Init(d.buf, 0, 8, 2, ' ', 0)\n\t\t\tdescribeResult(d, tc.inputName, tc.inputResult)\n\t\t\td.out.Flush()\n\t\t\tassert.Equal(tt, tc.expect, d.buf.String())\n\t\t})\n\t}\n}\n\nfunc TestDescribeRestoreItemOperation(t *testing.T) {\n\tt1, err1 := time.Parse(\"2006-Jan-02\", \"2023-Jun-26\")\n\trequire.NoError(t, err1)\n\tt2, err2 := time.Parse(\"2006-Jan-02\", \"2023-Jun-25\")\n\trequire.NoError(t, err2)\n\tt3, err3 := time.Parse(\"2006-Jan-02\", \"2023-Jun-24\")\n\trequire.NoError(t, err3)\n\tinput := builder.ForRestoreOperation().\n\t\tRestoreName(\"restore-1\").\n\t\tOperationID(\"op-1\").\n\t\tRestoreItemAction(\"action-1\").\n\t\tResourceIdentifier(\"group\", \"rs-type\", \"ns\", \"rs-name\").\n\t\tStatus(*builder.ForOperationStatus().\n\t\t\tPhase(itemoperation.OperationPhaseFailed).\n\t\t\tError(\"operation error\").\n\t\t\tProgress(50, 100, \"bytes\").\n\t\t\tDescription(\"operation description\").\n\t\t\tCreated(t3).\n\t\t\tStarted(t2).\n\t\t\tUpdated(t1).\n\t\t\tResult()).Result()\n\texpected := ` Operation for rs-type.group ns/rs-name:\n Restore Item Action Plugin: action-1\n Operation ID: op-1\n Phase: Failed\n Operation Error: operation error\n Progress: 50 of 100 complete (bytes)\n Progress description: operation description\n Created: 2023-06-24 00:00:00 +0000 UTC\n Started: 2023-06-25 00:00:00 +0000 UTC\n Updated: 2023-06-26 00:00:00 +0000 UTC\n`\n\td := &Describer{\n\t\tPrefix: \"\",\n\t\tout: &tabwriter.Writer{},\n\t\tbuf: &bytes.Buffer{},\n\t}\n\td.out.Init(d.buf, 0, 8, 2, ' ', 0)\n\tdescribeRestoreItemOperation(d, input)\n\td.out.Flush()\n\tassert.Equal(t, expected, d.buf.String())\n}\n\nfunc TestDescribePodVolumeRestores(t *testing.T) {\n\tpvr1 := builder.ForPodVolumeRestore(\"velero\", \"pvr-1\").\n\t\tUploaderType(\"kopia\").\n\t\tPhase(velerov1api.PodVolumeRestorePhaseCompleted).\n\t\tBackupStorageLocation(\"bsl-1\").\n\t\tVolume(\"vol-1\").\n\t\tPodName(\"pod-1\").\n\t\tPodNamespace(\"pod-ns-1\").\n\t\tSnapshotID(\"snap-1\").Result()\n\tpvr2 := builder.ForPodVolumeRestore(\"velero\", \"pvr-2\").\n\t\tUploaderType(\"kopia\").\n\t\tPhase(velerov1api.PodVolumeRestorePhaseCompleted).\n\t\tBackupStorageLocation(\"bsl-1\").\n\t\tVolume(\"vol-2\").\n\t\tPodName(\"pod-2\").\n\t\tPodNamespace(\"pod-ns-1\").\n\t\tSnapshotID(\"snap-2\").Result()\n\n\ttestcases := []struct {\n\t\tname string\n\t\tinputPVRList []velerov1api.PodVolumeRestore\n\t\tinputDetails bool\n\t\texpect string\n\t}{\n\t\t{\n\t\t\tname: \"empty list\",\n\t\t\tinputPVRList: []velerov1api.PodVolumeRestore{},\n\t\t\tinputDetails: true,\n\t\t\texpect: ``,\n\t\t},\n\t\t{\n\t\t\tname: \"2 completed pvrs no details\",\n\t\t\tinputPVRList: []velerov1api.PodVolumeRestore{*pvr1, *pvr2},\n\t\t\tinputDetails: false,\n\t\t\texpect: `kopia Restores (specify --details for more information):\n Completed: 2\n`,\n\t\t},\n\t\t{\n\t\t\tname: \"2 completed pvrs with details\",\n\t\t\tinputPVRList: []velerov1api.PodVolumeRestore{*pvr1, *pvr2},\n\t\t\tinputDetails: true,\n\t\t\texpect: `kopia Restores:\n Completed:\n pod-ns-1/pod-1: vol-1\n pod-ns-1/pod-2: vol-2\n`,\n\t\t},\n\t}\n\n\tfor _, tc := range testcases {\n\t\tt.Run(tc.name, func(tt *testing.T) {\n\t\t\td := &Describer{\n\t\t\t\tPrefix: \"\",\n\t\t\t\tout: &tabwriter.Writer{},\n\t\t\t\tbuf: &bytes.Buffer{},\n\t\t\t}\n\t\t\td.out.Init(d.buf, 0, 8, 2, ' ', 0)\n\t\t\tdescribePodVolumeRestores(d, tc.inputPVRList, tc.inputDetails)\n\t\t\td.out.Flush()\n\t\t\tassert.Equal(tt, tc.expect, d.buf.String())\n\t\t})\n\t}\n}\nfunc TestDescribeUploaderConfigForRestore(t *testing.T) {\n\tcases := []struct {\n\t\tname string\n\t\tspec velerov1api.RestoreSpec\n\t\texpected string\n\t}{\n\t\t{\n\t\t\tname: \"UploaderConfigNil\",\n\t\t\tspec: velerov1api.RestoreSpec{}, // Create a RestoreSpec with nil UploaderConfig\n\t\t\texpected: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"test\",\n\t\t\tspec: velerov1api.RestoreSpec{\n\t\t\t\tUploaderConfig: &velerov1api.UploaderConfigForRestore{\n\t\t\t\t\tWriteSparseFiles: boolptr.True(),\n\t\t\t\t\tParallelFilesDownload: 4,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: \"\\nUploader config:\\n Write Sparse Files: true\\n Parallel Restore: 4\\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"WriteSparseFiles test\",\n\t\t\tspec: velerov1api.RestoreSpec{\n\t\t\t\tUploaderConfig: &velerov1api.UploaderConfigForRestore{\n\t\t\t\t\tWriteSparseFiles: boolptr.True(),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: \"\\nUploader config:\\n Write Sparse Files: true\\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"ParallelFilesDownload test\",\n\t\t\tspec: velerov1api.RestoreSpec{\n\t\t\t\tUploaderConfig: &velerov1api.UploaderConfigForRestore{\n\t\t\t\t\tParallelFilesDownload: 4,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: \"\\nUploader config:\\n Parallel Restore: 4\\n\",\n\t\t},\n\t}\n\n\tfor _, tc := range cases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\td := &Describer{\n\t\t\t\tPrefix: \"\",\n\t\t\t\tout: &tabwriter.Writer{},\n\t\t\t\tbuf: &bytes.Buffer{},\n\t\t\t}\n\t\t\td.out.Init(d.buf, 0, 8, 2, ' ', 0)\n\t\t\tdescribeUploaderConfigForRestore(d, tc.spec)\n\t\t\td.out.Flush()\n\t\t\tassert.Equal(t, tc.expected, d.buf.String(), \"Output should match expected\")\n\t\t})\n\t}\n}\n\nfunc TestDescribeCSISnapshotsRestore(t *testing.T) {\n\tcases := []struct {\n\t\tname string\n\t\tinputVolInfoList []volume.RestoreVolumeInfo\n\t\tinputDetail bool\n\t\texpect string\n\t}{\n\t\t{\n\t\t\tname: \"empty list\",\n\t\t\tinputVolInfoList: []volume.RestoreVolumeInfo{},\n\t\t\tinputDetail: true,\n\t\t\texpect: `\nCSI Snapshot Restores: \n`,\n\t\t},\n\t\t{\n\t\t\tname: \"list with non CSI snapshot\",\n\t\t\tinputVolInfoList: []volume.RestoreVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"pvc-2\",\n\t\t\t\t\tPVCNamespace: \"ns-2\",\n\t\t\t\t\tPVName: \"pv-2\",\n\t\t\t\t\tRestoreMethod: volume.NativeSnapshot,\n\t\t\t\t\tNativeSnapshotInfo: &volume.NativeSnapshotInfo{\n\t\t\t\t\t\tSnapshotHandle: \"snap-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tinputDetail: true,\n\t\t\texpect: `\nCSI Snapshot Restores: \n`,\n\t\t},\n\t\t{\n\t\t\tname: \"CSI restore without data movement, detailed\",\n\t\t\tinputVolInfoList: []volume.RestoreVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"pvc-1\",\n\t\t\t\t\tPVCNamespace: \"ns-1\",\n\t\t\t\t\tPVName: \"pv-1\",\n\t\t\t\t\tRestoreMethod: volume.CSISnapshot,\n\t\t\t\t\tCSISnapshotInfo: &volume.CSISnapshotInfo{\n\t\t\t\t\t\tSnapshotHandle: \"snapshot-handle-1\",\n\t\t\t\t\t\tSize: 1234,\n\t\t\t\t\t\tDriver: \"csi.test.driver\",\n\t\t\t\t\t\tVSCName: \"content-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tinputDetail: true,\n\t\t\texpect: `\nCSI Snapshot Restores:\n ns-1/pvc-1:\n Snapshot:\n Snapshot Content Name: content-1\n Storage Snapshot ID: snapshot-handle-1\n CSI Driver: csi.test.driver\n`,\n\t\t},\n\t\t{\n\t\t\tname: \"CSI restore with data movement, detailed\",\n\t\t\tinputVolInfoList: []volume.RestoreVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"pvc-3\",\n\t\t\t\t\tPVCNamespace: \"ns-3\",\n\t\t\t\t\tPVName: \"pv-3\",\n\t\t\t\t\tRestoreMethod: volume.CSISnapshot,\n\t\t\t\t\tSnapshotDataMoved: true,\n\t\t\t\t\tSnapshotDataMovementInfo: &volume.SnapshotDataMovementInfo{\n\t\t\t\t\t\tOperationID: \"op-3\",\n\t\t\t\t\t\tDataMover: \"velero\",\n\t\t\t\t\t\tUploaderType: \"kopia\",\n\t\t\t\t\t\tSize: 1234,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tinputDetail: true,\n\t\t\texpect: `\nCSI Snapshot Restores:\n ns-3/pvc-3:\n Data Movement:\n Operation ID: op-3\n Data Mover: velero\n Uploader Type: kopia\n`,\n\t\t},\n\t\t{\n\t\t\tname: \"vol info with different entries, without details\",\n\t\t\tinputVolInfoList: []volume.RestoreVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"pvc-3\",\n\t\t\t\t\tPVCNamespace: \"ns-3\",\n\t\t\t\t\tPVName: \"pv-3\",\n\t\t\t\t\tRestoreMethod: volume.CSISnapshot,\n\t\t\t\t\tSnapshotDataMoved: true,\n\t\t\t\t\tSnapshotDataMovementInfo: &volume.SnapshotDataMovementInfo{\n\t\t\t\t\t\tOperationID: \"op-3\",\n\t\t\t\t\t\tDataMover: \"velero\",\n\t\t\t\t\t\tUploaderType: \"kopia\",\n\t\t\t\t\t\tSize: 1234,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"pvc-2\",\n\t\t\t\t\tPVCNamespace: \"ns-2\",\n\t\t\t\t\tPVName: \"pv-2\",\n\t\t\t\t\tRestoreMethod: volume.NativeSnapshot,\n\t\t\t\t\tNativeSnapshotInfo: &volume.NativeSnapshotInfo{\n\t\t\t\t\t\tSnapshotHandle: \"snap-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"pvc-1\",\n\t\t\t\t\tPVCNamespace: \"ns-1\",\n\t\t\t\t\tPVName: \"pv-1\",\n\t\t\t\t\tRestoreMethod: volume.CSISnapshot,\n\t\t\t\t\tCSISnapshotInfo: &volume.CSISnapshotInfo{\n\t\t\t\t\t\tSnapshotHandle: \"snapshot-handle-1\",\n\t\t\t\t\t\tSize: 1234,\n\t\t\t\t\t\tDriver: \"csi.test.driver\",\n\t\t\t\t\t\tVSCName: \"content-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tinputDetail: false,\n\t\t\texpect: `\nCSI Snapshot Restores:\n ns-1/pvc-1:\n Snapshot: specify --details for more information\n ns-3/pvc-3:\n Data Movement: specify --details for more information\n`,\n\t\t},\n\t}\n\n\tfor _, tc := range cases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\td := &Describer{\n\t\t\t\tPrefix: \"\",\n\t\t\t\tout: &tabwriter.Writer{},\n\t\t\t\tbuf: &bytes.Buffer{},\n\t\t\t}\n\t\t\td.out.Init(d.buf, 0, 8, 2, ' ', 0)\n\t\t\tdescribeCSISnapshotsRestores(d, tc.inputVolInfoList, tc.inputDetail)\n\t\t\td.out.Flush()\n\t\t\tassert.Equal(t, tc.expect, d.buf.String())\n\t\t})\n\t}\n}\n\nfunc TestDescribeResourceModifier(t *testing.T) {\n\td := &Describer{\n\t\tPrefix: \"\",\n\t\tout: &tabwriter.Writer{},\n\t\tbuf: &bytes.Buffer{},\n\t}\n\n\td.out.Init(d.buf, 0, 8, 2, ' ', 0)\n\n\tDescribeResourceModifier(d, &corev1api.TypedLocalObjectReference{\n\t\tAPIGroup: &corev1api.SchemeGroupVersion.Group,\n\t\tKind: \"ConfigMap\",\n\t\tName: \"resourceModifier\",\n\t})\n\td.out.Flush()\n\n\texpectOutput := `Resource modifier:\n Type: ConfigMap\n Name: resourceModifier\n`\n\n\tfmt.Println(d.buf.String())\n\trequire.Equal(t, expectOutput, d.buf.String())\n}\n" }, { "path": "pkg/cmd/util/output/restore_printer.go", "content": "/*\nCopyright 2017, 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage output\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\nvar (\n\trestoreColumns = []metav1.TableColumnDefinition{\n\t\t// name needs Type and Format defined for the decorator to identify it:\n\t\t// https://github.com/kubernetes/kubernetes/blob/v1.15.3/pkg/printers/tableprinter.go#L204\n\t\t{Name: \"Name\", Type: \"string\", Format: \"name\"},\n\t\t{Name: \"Backup\"},\n\t\t{Name: \"Status\"},\n\t\t{Name: \"Started\"},\n\t\t{Name: \"Completed\"},\n\t\t{Name: \"Errors\"},\n\t\t{Name: \"Warnings\"},\n\t\t{Name: \"Created\"},\n\t\t{Name: \"Selector\"},\n\t}\n)\n\nfunc printRestoreList(list *v1.RestoreList) []metav1.TableRow {\n\trows := make([]metav1.TableRow, 0, len(list.Items))\n\n\tfor i := range list.Items {\n\t\trows = append(rows, printRestore(&list.Items[i])...)\n\t}\n\treturn rows\n}\n\nfunc printRestore(restore *v1.Restore) []metav1.TableRow {\n\trow := metav1.TableRow{\n\t\tObject: runtime.RawExtension{Object: restore},\n\t}\n\n\tstatus := restore.Status.Phase\n\tif status == \"\" {\n\t\tstatus = v1.RestorePhaseNew\n\t}\n\n\trow.Cells = append(row.Cells,\n\t\trestore.Name,\n\t\trestore.Spec.BackupName,\n\t\tstatus,\n\t\trestore.Status.StartTimestamp,\n\t\trestore.Status.CompletionTimestamp,\n\t\trestore.Status.Errors,\n\t\trestore.Status.Warnings,\n\t\trestore.CreationTimestamp.Time,\n\t\tmetav1.FormatLabelSelector(restore.Spec.LabelSelector),\n\t)\n\n\treturn []metav1.TableRow{row}\n}\n" }, { "path": "pkg/cmd/util/output/schedule_describe_test.go", "content": "package output\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n)\n\nfunc TestDescribeSchedule(t *testing.T) {\n\tinput1 := builder.ForSchedule(\"velero\", \"schedule-1\").\n\t\tPhase(velerov1api.SchedulePhaseFailedValidation).\n\t\tValidationError(\"validation failed\").Result()\n\texpect1 := `Name: schedule-1\nNamespace: velero\nLabels: \nAnnotations: \n\nPhase: FailedValidation\n\nValidation errors: validation failed\n\nPaused: false\n\nSchedule: \n\nBackup Template:\n Namespaces:\n Included: *\n Excluded: \n \n Resources:\n Included cluster-scoped: \n Excluded cluster-scoped: \n Included namespace-scoped: *\n Excluded namespace-scoped: \n \n Label selector: \n \n Or label selector: \n \n Storage Location: \n \n Velero-Native Snapshot PVs: auto\n Snapshot Move Data: auto\n Data Mover: velero\n \n TTL: 0s\n \n CSISnapshotTimeout: 0s\n ItemOperationTimeout: 0s\n \n Hooks: \n\nLast Backup: \n`\n\n\tinput2 := builder.ForSchedule(\"velero\", \"schedule-2\").\n\t\tPhase(velerov1api.SchedulePhaseEnabled).\n\t\tCronSchedule(\"0 0 * * *\").\n\t\tTemplate(builder.ForBackup(\"velero\", \"backup-1\").ParallelFilesUpload(10).Result().Spec).\n\t\tLastBackupTime(\"2023-06-25 15:04:05\").Result()\n\texpect2 := `Name: schedule-2\nNamespace: velero\nLabels: \nAnnotations: \n\nPhase: Enabled\n\nUploader config:\n Parallel files upload: 10\n\nPaused: false\n\nSchedule: 0 0 * * *\n\nBackup Template:\n Namespaces:\n Included: *\n Excluded: \n \n Resources:\n Included cluster-scoped: \n Excluded cluster-scoped: \n Included namespace-scoped: *\n Excluded namespace-scoped: \n \n Label selector: \n \n Or label selector: \n \n Storage Location: \n \n Velero-Native Snapshot PVs: auto\n Snapshot Move Data: auto\n Data Mover: velero\n \n TTL: 0s\n \n CSISnapshotTimeout: 0s\n ItemOperationTimeout: 0s\n \n Hooks: \n\nLast Backup: 2023-06-25 15:04:05 +0000 UTC\n`\n\n\tinput3 := builder.ForSchedule(\"velero\", \"schedule-3\").\n\t\tPhase(velerov1api.SchedulePhaseEnabled).\n\t\tCronSchedule(\"0 0 * * *\").\n\t\tTemplate(builder.ForBackup(\"velero\", \"backup-1\").DefaultVolumesToFsBackup(true).Result().Spec).\n\t\tLastBackupTime(\"2023-06-25 15:04:05\").Result()\n\texpect3 := `Name: schedule-3\nNamespace: velero\nLabels: \nAnnotations: \n\nPhase: Enabled\n\nPaused: false\n\nSchedule: 0 0 * * *\n\nBackup Template:\n Namespaces:\n Included: *\n Excluded: \n \n Resources:\n Included cluster-scoped: \n Excluded cluster-scoped: \n Included namespace-scoped: *\n Excluded namespace-scoped: \n \n Label selector: \n \n Or label selector: \n \n Storage Location: \n \n Velero-Native Snapshot PVs: auto\n File System Backup (Default): true\n Snapshot Move Data: auto\n Data Mover: velero\n \n TTL: 0s\n \n CSISnapshotTimeout: 0s\n ItemOperationTimeout: 0s\n \n Hooks: \n\nLast Backup: 2023-06-25 15:04:05 +0000 UTC\n`\n\n\tinput4 := builder.ForSchedule(\"velero\", \"schedule-4\").\n\t\tPhase(velerov1api.SchedulePhaseEnabled).\n\t\tCronSchedule(\"0 0 * * *\").\n\t\tTemplate(builder.ForBackup(\"velero\", \"backup-1\").DefaultVolumesToFsBackup(false).Result().Spec).\n\t\tLastBackupTime(\"2023-06-25 15:04:05\").Result()\n\texpect4 := `Name: schedule-4\nNamespace: velero\nLabels: \nAnnotations: \n\nPhase: Enabled\n\nPaused: false\n\nSchedule: 0 0 * * *\n\nBackup Template:\n Namespaces:\n Included: *\n Excluded: \n \n Resources:\n Included cluster-scoped: \n Excluded cluster-scoped: \n Included namespace-scoped: *\n Excluded namespace-scoped: \n \n Label selector: \n \n Or label selector: \n \n Storage Location: \n \n Velero-Native Snapshot PVs: auto\n File System Backup (Default): false\n Snapshot Move Data: auto\n Data Mover: velero\n \n TTL: 0s\n \n CSISnapshotTimeout: 0s\n ItemOperationTimeout: 0s\n \n Hooks: \n\nLast Backup: 2023-06-25 15:04:05 +0000 UTC\n`\n\n\ttestcases := []struct {\n\t\tname string\n\t\tinput *velerov1api.Schedule\n\t\texpect string\n\t}{\n\t\t{\n\t\t\tname: \"schedule failed in validation\",\n\t\t\tinput: input1,\n\t\t\texpect: expect1,\n\t\t},\n\t\t{\n\t\t\tname: \"schedule enabled\",\n\t\t\tinput: input2,\n\t\t\texpect: expect2,\n\t\t},\n\t\t{\n\t\t\tname: \"schedule with DefaultVolumesToFsBackup is true\",\n\t\t\tinput: input3,\n\t\t\texpect: expect3,\n\t\t},\n\t\t{\n\t\t\tname: \"schedule with DefaultVolumesToFsBackup is false\",\n\t\t\tinput: input4,\n\t\t\texpect: expect4,\n\t\t},\n\t}\n\n\tfor _, tc := range testcases {\n\t\tt.Run(tc.name, func(tt *testing.T) {\n\t\t\tassert.Equal(tt, tc.expect, DescribeSchedule(tc.input))\n\t\t})\n\t}\n}\n" }, { "path": "pkg/cmd/util/output/schedule_describer.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage output\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/fatih/color\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\nfunc DescribeSchedule(schedule *v1.Schedule) string {\n\treturn Describe(func(d *Describer) {\n\t\td.DescribeMetadata(schedule.ObjectMeta)\n\n\t\td.Println()\n\t\tphase := schedule.Status.Phase\n\n\t\tif phase == \"\" {\n\t\t\tphase = v1.SchedulePhaseNew\n\t\t}\n\t\tphaseString := string(phase)\n\t\tswitch phase {\n\t\tcase v1.SchedulePhaseEnabled:\n\t\t\tphaseString = color.GreenString(phaseString)\n\t\tcase v1.SchedulePhaseFailedValidation:\n\t\t\tphaseString = color.RedString(phaseString)\n\t\t}\n\t\td.Printf(\"Phase:\\t%s\\n\", phaseString)\n\n\t\tif schedule.Spec.Template.ResourcePolicy != nil {\n\t\t\td.Println()\n\t\t\tDescribeResourcePolicies(d, schedule.Spec.Template.ResourcePolicy)\n\t\t}\n\n\t\tif schedule.Spec.Template.UploaderConfig != nil && schedule.Spec.Template.UploaderConfig.ParallelFilesUpload > 0 {\n\t\t\td.Println()\n\t\t\tDescribeUploaderConfigForBackup(d, schedule.Spec.Template)\n\t\t}\n\n\t\tstatus := schedule.Status\n\t\tif len(status.ValidationErrors) > 0 {\n\t\t\td.Println()\n\t\t\td.Printf(\"Validation errors:\")\n\t\t\tfor _, ve := range status.ValidationErrors {\n\t\t\t\td.Printf(\"\\t%s\\n\", color.RedString(ve))\n\t\t\t}\n\t\t}\n\n\t\td.Println()\n\t\td.Printf(\"Paused:\\t%t\\n\", schedule.Spec.Paused)\n\n\t\td.Println()\n\t\tDescribeScheduleSpec(d, schedule.Spec)\n\n\t\td.Println()\n\t\tDescribeScheduleStatus(d, schedule.Status)\n\t})\n}\n\nfunc DescribeScheduleSpec(d *Describer, spec v1.ScheduleSpec) {\n\td.Printf(\"Schedule:\\t%s\\n\", spec.Schedule)\n\n\td.Println()\n\td.Println(\"Backup Template:\")\n\td.Prefix = \"\\t\"\n\tDescribeBackupSpec(d, spec.Template)\n\td.Prefix = \"\"\n}\n\nfunc DescribeScheduleStatus(d *Describer, status v1.ScheduleStatus) {\n\tlastBackup := \"\"\n\tif status.LastBackup != nil && !status.LastBackup.Time.IsZero() {\n\t\tlastBackup = fmt.Sprintf(\"%v\", status.LastBackup.Time)\n\t}\n\td.Printf(\"Last Backup:\\t%s\\n\", lastBackup)\n}\n" }, { "path": "pkg/cmd/util/output/schedule_printer.go", "content": "/*\nCopyright 2017, 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage output\n\nimport (\n\t\"time\"\n\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\nvar (\n\tscheduleColumns = []metav1.TableColumnDefinition{\n\t\t// name needs Type and Format defined for the decorator to identify it:\n\t\t// https://github.com/kubernetes/kubernetes/blob/v1.15.3/pkg/printers/tableprinter.go#L204\n\t\t{Name: \"Name\", Type: \"string\", Format: \"name\"},\n\t\t{Name: \"Status\"},\n\t\t{Name: \"Created\"},\n\t\t{Name: \"Schedule\"},\n\t\t{Name: \"Backup TTL\"},\n\t\t{Name: \"Last Backup\"},\n\t\t{Name: \"Selector\"},\n\t\t{Name: \"Paused\"},\n\t}\n)\n\nfunc printScheduleList(list *v1.ScheduleList) []metav1.TableRow {\n\trows := make([]metav1.TableRow, 0, len(list.Items))\n\n\tfor i := range list.Items {\n\t\trows = append(rows, printSchedule(&list.Items[i])...)\n\t}\n\treturn rows\n}\n\nfunc printSchedule(schedule *v1.Schedule) []metav1.TableRow {\n\trow := metav1.TableRow{\n\t\tObject: runtime.RawExtension{Object: schedule},\n\t}\n\n\tstatus := schedule.Status.Phase\n\tif status == \"\" {\n\t\tstatus = v1.SchedulePhaseNew\n\t}\n\n\tvar lastBackupTime time.Time\n\tif schedule.Status.LastBackup != nil {\n\t\tlastBackupTime = schedule.Status.LastBackup.Time\n\t}\n\n\trow.Cells = append(row.Cells,\n\t\tschedule.Name,\n\t\tstatus,\n\t\tschedule.CreationTimestamp.Time,\n\t\tschedule.Spec.Schedule,\n\t\tschedule.Spec.Template.TTL.Duration,\n\t\thumanReadableTimeFromNow(lastBackupTime),\n\t\tmetav1.FormatLabelSelector(schedule.Spec.Template.LabelSelector),\n\t\tschedule.Spec.Paused,\n\t)\n\n\treturn []metav1.TableRow{row}\n}\n" }, { "path": "pkg/cmd/util/output/volume_snapshot_location_printer.go", "content": "/*\nCopyright 2018, 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage output\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\nvar (\n\tvolumeSnapshotLocationColumns = []metav1.TableColumnDefinition{\n\t\t// name needs Type and Format defined for the decorator to identify it:\n\t\t// https://github.com/kubernetes/kubernetes/blob/v1.15.3/pkg/printers/tableprinter.go#L204\n\t\t{Name: \"Name\", Type: \"string\", Format: \"name\"},\n\t\t{Name: \"Provider\"},\n\t}\n)\n\nfunc printVolumeSnapshotLocationList(list *v1.VolumeSnapshotLocationList) []metav1.TableRow {\n\trows := make([]metav1.TableRow, 0, len(list.Items))\n\n\tfor i := range list.Items {\n\t\trows = append(rows, printVolumeSnapshotLocation(&list.Items[i])...)\n\t}\n\treturn rows\n}\n\nfunc printVolumeSnapshotLocation(location *v1.VolumeSnapshotLocation) []metav1.TableRow {\n\trow := metav1.TableRow{\n\t\tObject: runtime.RawExtension{Object: location},\n\t}\n\n\trow.Cells = append(row.Cells,\n\t\tlocation.Name,\n\t\tlocation.Spec.Provider,\n\t)\n\n\treturn []metav1.TableRow{row}\n}\n" }, { "path": "pkg/cmd/util/signals/signals.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage signals\n\nimport (\n\t\"context\"\n\t\"os\"\n\t\"os/signal\"\n\t\"syscall\"\n\n\t\"github.com/sirupsen/logrus\"\n)\n\n// CancelOnShutdown starts a goroutine that will call cancelFunc when\n// either SIGINT or SIGTERM is received\nfunc CancelOnShutdown(cancelFunc context.CancelFunc, logger logrus.FieldLogger) {\n\tsigs := make(chan os.Signal, 1)\n\tsignal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM)\n\n\tgo func() {\n\t\tsig := <-sigs\n\t\tlogger.Infof(\"Received signal %s, shutting down\", sig)\n\t\tcancelFunc()\n\t}()\n}\n" }, { "path": "pkg/cmd/velero/velero.go", "content": "/*\nCopyright 2021 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage velero\n\nimport (\n\t\"flag\"\n\t\"fmt\"\n\t\"os\"\n\n\t\"github.com/fatih/color\"\n\t\"github.com/spf13/cobra\"\n\t\"k8s.io/klog/v2\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/debug\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/podvolume\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/repomantenance\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/backup\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/backuplocation\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/bug\"\n\tcliclient \"github.com/vmware-tanzu/velero/pkg/cmd/cli/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/completion\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/create\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/delete\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/describe\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/get\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/install\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/plugin\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/repo\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/restore\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/schedule\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/snapshotlocation\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/uninstall\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/version\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/server\"\n\trunplugin \"github.com/vmware-tanzu/velero/pkg/cmd/server/plugin\"\n\tveleroflag \"github.com/vmware-tanzu/velero/pkg/cmd/util/flag\"\n\t\"github.com/vmware-tanzu/velero/pkg/features\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/datamover\"\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/cli/nodeagent\"\n)\n\nfunc NewCommand(name string) *cobra.Command {\n\t// Load the config here so that we can extract features from it.\n\tconfig, err := client.LoadConfig()\n\tif err != nil {\n\t\tfmt.Fprintf(os.Stderr, \"WARNING: Error reading config file: %v\\n\", err)\n\t}\n\n\t// Declare cmdFeatures and cmdColorzied here so we can access them in the PreRun hooks\n\t// without doing a chain of calls into the command's FlagSet\n\tvar cmdFeatures veleroflag.StringArray\n\tvar cmdColorzied veleroflag.OptionalBool\n\n\tc := &cobra.Command{\n\t\tUse: name,\n\t\tShort: \"Back up and restore Kubernetes cluster resources.\",\n\t\tLong: `Velero is a tool for managing disaster recovery, specifically for Kubernetes\ncluster resources. It provides a simple, configurable, and operationally robust\nway to back up your application state and associated data.\n\nIf you're familiar with kubectl, Velero supports a similar model, allowing you to\nexecute commands such as 'velero get backup' and 'velero create schedule'. The same\noperations can also be performed as 'velero backup get' and 'velero schedule create'.`,\n\t\t// PersistentPreRun will run before all subcommands EXCEPT in the following conditions:\n\t\t// - a subcommand defines its own PersistentPreRun function\n\t\t// - the command is run without arguments or with --help and only prints the usage info\n\t\tPersistentPreRun: func(cmd *cobra.Command, args []string) {\n\t\t\tfeatures.Enable(config.Features()...)\n\t\t\tfeatures.Enable(cmdFeatures...)\n\n\t\t\tswitch {\n\t\t\tcase cmdColorzied.Value != nil:\n\t\t\t\tcolor.NoColor = !*cmdColorzied.Value\n\t\t\tdefault:\n\t\t\t\tcolor.NoColor = !config.Colorized()\n\t\t\t}\n\t\t},\n\t}\n\n\tf := client.NewFactory(name, config)\n\tf.BindFlags(c.PersistentFlags())\n\n\t// Bind features directly to the root command so it's available to all callers.\n\tc.PersistentFlags().Var(&cmdFeatures, \"features\", \"Comma-separated list of features to enable for this Velero process. Combines with values from $HOME/.config/velero/config.json if present\")\n\n\t// Color will be enabled or disabled for all subcommands\n\tc.PersistentFlags().Var(&cmdColorzied, \"colorized\", \"Show colored output in TTY. Overrides 'colorized' value from $HOME/.config/velero/config.json if present. Enabled by default\")\n\n\tc.AddCommand(\n\t\tbackup.NewCommand(f),\n\t\tschedule.NewCommand(f),\n\t\trestore.NewCommand(f),\n\t\tserver.NewCommand(f),\n\t\tnodeagent.NewCommand(f),\n\t\tversion.NewCommand(f),\n\t\tget.NewCommand(f),\n\t\tinstall.NewCommand(f),\n\t\tuninstall.NewCommand(f),\n\t\tdescribe.NewCommand(f),\n\t\tcreate.NewCommand(f),\n\t\trunplugin.NewCommand(f),\n\t\tplugin.NewCommand(f),\n\t\tdelete.NewCommand(f),\n\t\tcliclient.NewCommand(),\n\t\tcompletion.NewCommand(),\n\t\trepo.NewCommand(f),\n\t\tbug.NewCommand(),\n\t\tbackuplocation.NewCommand(f),\n\t\tsnapshotlocation.NewCommand(f),\n\t\tdebug.NewCommand(f),\n\t\trepomantenance.NewCommand(f),\n\t\tdatamover.NewCommand(f),\n\t\tpodvolume.NewCommand(f),\n\t)\n\n\t// init and add the klog flags\n\tklog.InitFlags(flag.CommandLine)\n\tc.PersistentFlags().AddGoFlagSet(flag.CommandLine)\n\n\treturn c\n}\n" }, { "path": "pkg/constant/constant.go", "content": "package constant\n\nconst (\n\tControllerBackupQueue = \"backup-queue\"\n\tControllerBackup = \"backup\"\n\tControllerBackupOperations = \"backup-operations\"\n\tControllerBackupDeletion = \"backup-deletion\"\n\tControllerBackupFinalizer = \"backup-finalizer\"\n\tControllerBackupRepo = \"backup-repo\"\n\tControllerBackupStorageLocation = \"backup-storage-location\"\n\tControllerBackupSync = \"backup-sync\"\n\tControllerDataDownload = \"data-download\"\n\tControllerDataUpload = \"data-upload\"\n\tControllerDownloadRequest = \"download-request\"\n\tControllerGarbageCollection = \"gc\"\n\tControllerPodVolumeBackup = \"pod-volume-backup\"\n\tControllerPodVolumeRestore = \"pod-volume-restore\"\n\tControllerRestore = \"restore\"\n\tControllerRestoreOperations = \"restore-operations\"\n\tControllerSchedule = \"schedule\"\n\tControllerServerStatusRequest = \"server-status-request\"\n\tControllerRestoreFinalizer = \"restore-finalizer\"\n\n\tPluginCSIPVCRestoreRIA = \"velero.io/csi-pvc-restorer\"\n\tPluginCsiVolumeSnapshotRestoreRIA = \"velero.io/csi-volumesnapshot-restorer\"\n\n\tDefaultEphemeralStorageRequest = \"0\"\n\tDefaultEphemeralStorageLimit = \"0\"\n)\n" }, { "path": "pkg/controller/backup_controller.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"slices\"\n\t\"time\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\tkerrors \"k8s.io/apimachinery/pkg/util/errors\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/builder\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/controller\"\n\t\"sigs.k8s.io/controller-runtime/pkg/event\"\n\t\"sigs.k8s.io/controller-runtime/pkg/predicate\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\t\"github.com/vmware-tanzu/velero/internal/resourcepolicies\"\n\t\"github.com/vmware-tanzu/velero/internal/storage\"\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tpkgbackup \"github.com/vmware-tanzu/velero/pkg/backup\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/discovery\"\n\t\"github.com/vmware-tanzu/velero/pkg/features\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\t\"github.com/vmware-tanzu/velero/pkg/persistence\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/collections\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/encode\"\n\tkubeutil \"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/results\"\n\tveleroutil \"github.com/vmware-tanzu/velero/pkg/util/velero\"\n)\n\nconst (\n\tbackupResyncPeriod = time.Minute\n)\n\nvar autoExcludeNamespaceScopedResources = []string{\n\t// CSI VolumeSnapshot and VolumeSnapshotContent are intermediate resources.\n\t// Velero only handle the VS and VSC created during backup,\n\t// not during resource collecting.\n\t\"volumesnapshots.snapshot.storage.k8s.io\",\n}\n\nvar autoExcludeClusterScopedResources = []string{\n\t// CSI VolumeSnapshot and VolumeSnapshotContent are intermediate resources.\n\t// Velero only handle the VS and VSC created during backup,\n\t// not during resource collecting.\n\t\"volumesnapshotcontents.snapshot.storage.k8s.io\",\n}\n\ntype backupReconciler struct {\n\tctx context.Context\n\tlogger logrus.FieldLogger\n\tdiscoveryHelper discovery.Helper\n\tbackupper pkgbackup.Backupper\n\tkbClient kbclient.Client\n\tclock clock.WithTickerAndDelayedExecution\n\tbackupLogLevel logrus.Level\n\tnewPluginManager func(logrus.FieldLogger) clientmgmt.Manager\n\tbackupTracker BackupTracker\n\tdefaultBackupLocation string\n\tdefaultVolumesToFsBackup bool\n\tdefaultBackupTTL time.Duration\n\tdefaultVGSLabelKey string\n\tdefaultCSISnapshotTimeout time.Duration\n\tresourceTimeout time.Duration\n\tdefaultItemOperationTimeout time.Duration\n\tdefaultSnapshotLocations map[string]string\n\tmetrics *metrics.ServerMetrics\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter\n\tformatFlag logging.Format\n\tcredentialFileStore credentials.FileStore\n\tmaxConcurrentK8SConnections int\n\tdefaultSnapshotMoveData bool\n\tglobalCRClient kbclient.Client\n\titemBlockWorkerCount int\n\tconcurrentBackups int\n}\n\nfunc NewBackupReconciler(\n\tctx context.Context,\n\tdiscoveryHelper discovery.Helper,\n\tbackupper pkgbackup.Backupper,\n\tlogger logrus.FieldLogger,\n\tbackupLogLevel logrus.Level,\n\tnewPluginManager func(logrus.FieldLogger) clientmgmt.Manager,\n\tbackupTracker BackupTracker,\n\tkbClient kbclient.Client,\n\tdefaultBackupLocation string,\n\tdefaultVolumesToFsBackup bool,\n\tdefaultBackupTTL time.Duration,\n\tdefaultVGSLabelKey string,\n\tdefaultCSISnapshotTimeout time.Duration,\n\tresourceTimeout time.Duration,\n\tdefaultItemOperationTimeout time.Duration,\n\tdefaultSnapshotLocations map[string]string,\n\tmetrics *metrics.ServerMetrics,\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter,\n\tformatFlag logging.Format,\n\tcredentialStore credentials.FileStore,\n\tmaxConcurrentK8SConnections int,\n\tdefaultSnapshotMoveData bool,\n\titemBlockWorkerCount int,\n\tconcurrentBackups int,\n\tglobalCRClient kbclient.Client,\n) *backupReconciler {\n\tb := &backupReconciler{\n\t\tctx: ctx,\n\t\tdiscoveryHelper: discoveryHelper,\n\t\tbackupper: backupper,\n\t\tclock: &clock.RealClock{},\n\t\tlogger: logger,\n\t\tbackupLogLevel: backupLogLevel,\n\t\tnewPluginManager: newPluginManager,\n\t\tbackupTracker: backupTracker,\n\t\tkbClient: kbClient,\n\t\tdefaultBackupLocation: defaultBackupLocation,\n\t\tdefaultVolumesToFsBackup: defaultVolumesToFsBackup,\n\t\tdefaultBackupTTL: defaultBackupTTL,\n\t\tdefaultVGSLabelKey: defaultVGSLabelKey,\n\t\tdefaultCSISnapshotTimeout: defaultCSISnapshotTimeout,\n\t\tresourceTimeout: resourceTimeout,\n\t\tdefaultItemOperationTimeout: defaultItemOperationTimeout,\n\t\tdefaultSnapshotLocations: defaultSnapshotLocations,\n\t\tmetrics: metrics,\n\t\tbackupStoreGetter: backupStoreGetter,\n\t\tformatFlag: formatFlag,\n\t\tcredentialFileStore: credentialStore,\n\t\tmaxConcurrentK8SConnections: maxConcurrentK8SConnections,\n\t\tdefaultSnapshotMoveData: defaultSnapshotMoveData,\n\t\titemBlockWorkerCount: itemBlockWorkerCount,\n\t\tconcurrentBackups: max(concurrentBackups, 1),\n\t\tglobalCRClient: globalCRClient,\n\t}\n\tb.updateTotalBackupMetric()\n\treturn b\n}\n\nfunc (b *backupReconciler) SetupWithManager(mgr ctrl.Manager) error {\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\tFor(&velerov1api.Backup{}, builder.WithPredicates(predicate.Funcs{\n\t\t\tUpdateFunc: func(ue event.UpdateEvent) bool {\n\t\t\t\tbackup := ue.ObjectNew.(*velerov1api.Backup)\n\t\t\t\treturn backup.Status.Phase == velerov1api.BackupPhaseReadyToStart\n\t\t\t},\n\t\t\tCreateFunc: func(ce event.CreateEvent) bool {\n\t\t\t\treturn false\n\t\t\t},\n\t\t\tDeleteFunc: func(de event.DeleteEvent) bool {\n\t\t\t\treturn false\n\t\t\t},\n\t\t\tGenericFunc: func(ge event.GenericEvent) bool {\n\t\t\t\treturn false\n\t\t\t},\n\t\t})).\n\t\tWithOptions(controller.Options{\n\t\t\tMaxConcurrentReconciles: b.concurrentBackups,\n\t\t}).\n\t\tNamed(constant.ControllerBackup).\n\t\tComplete(b)\n}\n\nfunc (b *backupReconciler) updateTotalBackupMetric() {\n\tgo func() {\n\t\t// Wait for 5 seconds to let controller-runtime to setup k8s clients.\n\t\ttime.Sleep(5 * time.Second)\n\n\t\twait.Until(\n\t\t\tfunc() {\n\t\t\t\t// recompute backup_total metric\n\t\t\t\tbackups := &velerov1api.BackupList{}\n\t\t\t\terr := b.kbClient.List(context.Background(), backups, &kbclient.ListOptions{LabelSelector: labels.Everything()})\n\t\t\t\tif err != nil {\n\t\t\t\t\tb.logger.Error(err, \"Error computing backup_total metric\")\n\t\t\t\t} else {\n\t\t\t\t\tb.metrics.SetBackupTotal(int64(len(backups.Items)))\n\t\t\t\t}\n\n\t\t\t\t// recompute backup_last_successful_timestamp metric for each\n\t\t\t\t// schedule (including the empty schedule, i.e. ad-hoc backups)\n\t\t\t\tfor schedule, timestamp := range getLastSuccessBySchedule(backups.Items) {\n\t\t\t\t\tb.metrics.SetBackupLastSuccessfulTimestamp(schedule, timestamp)\n\t\t\t\t}\n\t\t\t},\n\t\t\tbackupResyncPeriod,\n\t\t\tb.ctx.Done(),\n\t\t)\n\t}()\n}\n\n// getLastSuccessBySchedule finds the most recent completed backup for each schedule\n// and returns a map of schedule name -> completion time of the most recent completed\n// backup. This map includes an entry for ad-hoc/non-scheduled backups, where the key\n// is the empty string.\nfunc getLastSuccessBySchedule(backups []velerov1api.Backup) map[string]time.Time {\n\tlastSuccessBySchedule := map[string]time.Time{}\n\tfor _, backup := range backups {\n\t\tif backup.Status.Phase != velerov1api.BackupPhaseCompleted {\n\t\t\tcontinue\n\t\t}\n\t\tif backup.Status.CompletionTimestamp == nil {\n\t\t\tcontinue\n\t\t}\n\n\t\tschedule := backup.Labels[velerov1api.ScheduleNameLabel]\n\t\ttimestamp := backup.Status.CompletionTimestamp.Time\n\n\t\tif timestamp.After(lastSuccessBySchedule[schedule]) {\n\t\t\tlastSuccessBySchedule[schedule] = timestamp\n\t\t}\n\t}\n\n\treturn lastSuccessBySchedule\n}\n\nfunc (b *backupReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\tlog := b.logger.WithFields(logrus.Fields{\n\t\t\"controller\": constant.ControllerBackup,\n\t\t\"backuprequest\": req.String(),\n\t})\n\n\tlog.Debug(\"Getting backup\")\n\n\toriginal := &velerov1api.Backup{}\n\terr := b.kbClient.Get(ctx, req.NamespacedName, original)\n\tif err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.Debug(\"backup not found\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\tlog.WithError(err).Error(\"error getting backup\")\n\t\treturn ctrl.Result{}, err\n\t}\n\n\t// Double-check we have the correct phase. In the unlikely event that multiple controller\n\t// instances are running, it's possible for controller A to succeed in changing the phase to\n\t// InProgress, while controller B's attempt to patch the phase fails. When controller B\n\t// reprocesses the same backup, it will either show up as New (informer hasn't seen the update\n\t// yet) or as InProgress. In the former case, the patch attempt will fail again, until the\n\t// informer sees the update. In the latter case, after the informer has seen the update to\n\t// InProgress, we still need this check so we can return nil to indicate we've finished processing\n\t// this key (even though it was a no-op).\n\tswitch original.Status.Phase {\n\tcase velerov1api.BackupPhaseReadyToStart:\n\t\t// only process ReadytToStart backups\n\tdefault:\n\t\tb.logger.WithFields(logrus.Fields{\n\t\t\t\"backup\": kubeutil.NamespaceAndName(original),\n\t\t\t\"phase\": original.Status.Phase,\n\t\t}).Debug(\"Backup is not handled\")\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\tlog.Debug(\"Preparing backup request\")\n\trequest := b.prepareBackupRequest(ctx, original, log)\n\t// delete worker pool after reconcile\n\tdefer request.StopWorkerPool()\n\tif len(request.Status.ValidationErrors) > 0 {\n\t\trequest.Status.Phase = velerov1api.BackupPhaseFailedValidation\n\t} else {\n\t\trequest.Status.Phase = velerov1api.BackupPhaseInProgress\n\t\trequest.Status.StartTimestamp = &metav1.Time{Time: b.clock.Now()}\n\t}\n\n\t// update status to\n\t// BackupPhaseFailedValidation\n\t// BackupPhaseInProgress\n\t// if patch fail, backup can reconcile again as phase would still be \"\" or New\n\tif err := kubeutil.PatchResource(original, request.Backup, b.kbClient); err != nil {\n\t\treturn ctrl.Result{}, errors.Wrapf(err, \"error updating Backup status to %s\", request.Status.Phase)\n\t}\n\n\tbackupScheduleName := request.GetLabels()[velerov1api.ScheduleNameLabel]\n\n\tb.backupTracker.Add(request.Namespace, request.Name)\n\tdefer func() {\n\t\tswitch request.Status.Phase {\n\t\tcase velerov1api.BackupPhaseCompleted, velerov1api.BackupPhasePartiallyFailed, velerov1api.BackupPhaseFailed, velerov1api.BackupPhaseFailedValidation:\n\t\t\tb.backupTracker.Delete(request.Namespace, request.Name)\n\t\tcase velerov1api.BackupPhaseWaitingForPluginOperations, velerov1api.BackupPhaseWaitingForPluginOperationsPartiallyFailed, velerov1api.BackupPhaseFinalizing, velerov1api.BackupPhaseFinalizingPartiallyFailed:\n\t\t\tb.backupTracker.AddPostProcessing(request.Namespace, request.Name)\n\t\t}\n\t}()\n\n\tif request.Status.Phase == velerov1api.BackupPhaseFailedValidation {\n\t\tlog.Debug(\"failed to validate backup status\")\n\t\tb.metrics.RegisterBackupValidationFailure(backupScheduleName)\n\t\tb.metrics.RegisterBackupLastStatus(backupScheduleName, metrics.BackupLastStatusFailure)\n\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\t// store ref to just-updated item for creating patch\n\toriginal = request.Backup.DeepCopy()\n\n\tlog.Debug(\"Running backup\")\n\n\tb.metrics.RegisterBackupAttempt(backupScheduleName)\n\n\t// execution & upload of backup\n\tif err := b.runBackup(request); err != nil {\n\t\t// even though runBackup sets the backup's phase prior\n\t\t// to uploading artifacts to object storage, we have to\n\t\t// check for an error again here and update the phase if\n\t\t// one is found, because there could've been an error\n\t\t// while uploading artifacts to object storage, which would\n\t\t// result in the backup being Failed.\n\t\tlog.WithError(err).Error(\"backup failed\")\n\t\trequest.Status.Phase = velerov1api.BackupPhaseFailed\n\t\trequest.Status.FailureReason = err.Error()\n\t}\n\n\tswitch request.Status.Phase {\n\tcase velerov1api.BackupPhaseCompleted:\n\t\tb.metrics.RegisterBackupSuccess(backupScheduleName)\n\t\tb.metrics.RegisterBackupLastStatus(backupScheduleName, metrics.BackupLastStatusSucc)\n\tcase velerov1api.BackupPhasePartiallyFailed:\n\t\tb.metrics.RegisterBackupPartialFailure(backupScheduleName)\n\t\tb.metrics.RegisterBackupLastStatus(backupScheduleName, metrics.BackupLastStatusFailure)\n\tcase velerov1api.BackupPhaseFailed:\n\t\tb.metrics.RegisterBackupFailed(backupScheduleName)\n\t\tb.metrics.RegisterBackupLastStatus(backupScheduleName, metrics.BackupLastStatusFailure)\n\tcase velerov1api.BackupPhaseFailedValidation:\n\t\tb.metrics.RegisterBackupValidationFailure(backupScheduleName)\n\t\tb.metrics.RegisterBackupLastStatus(backupScheduleName, metrics.BackupLastStatusFailure)\n\t}\n\tlog.Info(\"Updating backup's status\")\n\t// Phases were updated in runBackup()\n\t// This patch with retry update Phase from InProgress to\n\t// BackupPhaseWaitingForPluginOperations -> backup_operations_controller.go will now reconcile\n\t// BackupPhaseWaitingForPluginOperationsPartiallyFailed -> backup_operations_controller.go will now reconcile\n\t// BackupPhaseFinalizing -> backup_finalizer_controller.go will now reconcile\n\t// BackupPhaseFinalizingPartiallyFailed -> backup_finalizer_controller.go will now reconcile\n\t// BackupPhaseFailed\n\tif err := kubeutil.PatchResourceWithRetriesOnErrors(b.resourceTimeout, original, request.Backup, b.kbClient); err != nil {\n\t\tlog.WithError(err).Errorf(\"error updating backup's status from %v to %v\", original.Status.Phase, request.Backup.Status.Phase)\n\t}\n\treturn ctrl.Result{}, nil\n}\n\nfunc (b *backupReconciler) prepareBackupRequest(ctx context.Context, backup *velerov1api.Backup, logger logrus.FieldLogger) *pkgbackup.Request {\n\trequest := &pkgbackup.Request{\n\t\tBackup: backup.DeepCopy(), // don't modify items in the cache\n\t\tSkippedPVTracker: pkgbackup.NewSkipPVTracker(),\n\t\tBackedUpItems: pkgbackup.NewBackedUpItemsMap(),\n\t\tWorkerPool: pkgbackup.StartItemBlockWorkerPool(ctx, b.itemBlockWorkerCount, logger),\n\t}\n\trequest.VolumesInformation.Init()\n\n\t// set backup major version - deprecated, use Status.FormatVersion\n\trequest.Status.Version = pkgbackup.BackupVersion\n\n\t// set backup major, minor, and patch version\n\trequest.Status.FormatVersion = pkgbackup.BackupFormatVersion\n\n\tif request.Spec.TTL.Duration == 0 {\n\t\t// set default backup TTL\n\t\trequest.Spec.TTL.Duration = b.defaultBackupTTL\n\t}\n\n\tif len(request.Spec.VolumeGroupSnapshotLabelKey) == 0 {\n\t\trequest.Spec.VolumeGroupSnapshotLabelKey = b.defaultVGSLabelKey\n\t}\n\n\tif request.Spec.CSISnapshotTimeout.Duration == 0 {\n\t\t// set default CSI VolumeSnapshot timeout\n\t\trequest.Spec.CSISnapshotTimeout.Duration = b.defaultCSISnapshotTimeout\n\t}\n\n\tif request.Spec.ItemOperationTimeout.Duration == 0 {\n\t\t// set default item operation timeout\n\t\trequest.Spec.ItemOperationTimeout.Duration = b.defaultItemOperationTimeout\n\t}\n\n\t// calculate expiration\n\trequest.Status.Expiration = &metav1.Time{Time: b.clock.Now().Add(request.Spec.TTL.Duration)}\n\n\t// TODO: After we drop the support for backup v1 CR. Remove this code block after DefaultVolumesToRestic is removed from CRD\n\t// For now, for CRs created by old versions, we need to respect the DefaultVolumesToRestic value if it is set true\n\tif boolptr.IsSetToTrue(request.Spec.DefaultVolumesToRestic) {\n\t\tlogger.Warn(\"DefaultVolumesToRestic field will be deprecated, use DefaultVolumesToFsBackup instead. Automatically remap it to DefaultVolumesToFsBackup\")\n\t\trequest.Spec.DefaultVolumesToFsBackup = request.Spec.DefaultVolumesToRestic\n\t}\n\n\tif request.Spec.DefaultVolumesToFsBackup == nil {\n\t\trequest.Spec.DefaultVolumesToFsBackup = &b.defaultVolumesToFsBackup\n\t}\n\n\tif request.Spec.SnapshotMoveData == nil {\n\t\trequest.Spec.SnapshotMoveData = &b.defaultSnapshotMoveData\n\t}\n\n\t// find which storage location to use\n\tvar serverSpecified bool\n\tif request.Spec.StorageLocation == \"\" {\n\t\t// when the user doesn't specify a location, use the server default unless there is an existing BSL marked as default\n\t\t// TODO(2.0) b.defaultBackupLocation will be deprecated\n\t\trequest.Spec.StorageLocation = b.defaultBackupLocation\n\n\t\tlocationList, err := storage.ListBackupStorageLocations(context.Background(), b.kbClient, request.Namespace)\n\t\tif err == nil {\n\t\t\tfor _, location := range locationList.Items {\n\t\t\t\tif location.Spec.Default {\n\t\t\t\t\trequest.Spec.StorageLocation = location.Name\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\tserverSpecified = true\n\t}\n\n\t// get the storage location, and store the BackupStorageLocation API obj on the request\n\tstorageLocation := &velerov1api.BackupStorageLocation{}\n\tif err := b.kbClient.Get(context.Background(), kbclient.ObjectKey{\n\t\tNamespace: request.Namespace,\n\t\tName: request.Spec.StorageLocation,\n\t}, storageLocation); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tif serverSpecified {\n\t\t\t\t// TODO(2.0) remove this. For now, without mentioning \"server default\" it could be confusing trying to grasp where the default came from.\n\t\t\t\trequest.Status.ValidationErrors = append(request.Status.ValidationErrors, fmt.Sprintf(\"an existing backup storage location was not specified at backup creation time and the server default %s does not exist. Please address this issue (see `velero backup-location -h` for options) and create a new backup. Error: %v\", request.Spec.StorageLocation, err))\n\t\t\t} else {\n\t\t\t\trequest.Status.ValidationErrors = append(request.Status.ValidationErrors, fmt.Sprintf(\"an existing backup storage location was not specified at backup creation time and the default %s was not found. Please address this issue (see `velero backup-location -h` for options) and create a new backup. Error: %v\", request.Spec.StorageLocation, err))\n\t\t\t}\n\t\t} else {\n\t\t\trequest.Status.ValidationErrors = append(request.Status.ValidationErrors, fmt.Sprintf(\"error getting backup storage location: %v\", err))\n\t\t}\n\t} else {\n\t\trequest.StorageLocation = storageLocation\n\n\t\tif request.StorageLocation.Spec.AccessMode == velerov1api.BackupStorageLocationAccessModeReadOnly {\n\t\t\trequest.Status.ValidationErrors = append(request.Status.ValidationErrors,\n\t\t\t\tfmt.Sprintf(\"backup can't be created because backup storage location %s is currently in read-only mode\", request.StorageLocation.Name))\n\t\t}\n\n\t\tif !veleroutil.BSLIsAvailable(*request.StorageLocation) {\n\t\t\trequest.Status.ValidationErrors = append(\n\t\t\t\trequest.Status.ValidationErrors,\n\t\t\t\tfmt.Sprintf(\"backup can't be created because BackupStorageLocation %s is in Unavailable status.\", request.StorageLocation.Name),\n\t\t\t)\n\t\t}\n\t}\n\n\t// add the storage location as a label for easy filtering later.\n\tif request.Labels == nil {\n\t\trequest.Labels = make(map[string]string)\n\t}\n\trequest.Labels[velerov1api.StorageLocationLabel] = label.GetValidName(request.Spec.StorageLocation)\n\n\t// validate and get the backup's VolumeSnapshotLocations, and store the\n\t// VolumeSnapshotLocation API objs on the request\n\tif locs, errs := b.validateAndGetSnapshotLocations(request.Backup); len(errs) > 0 {\n\t\trequest.Status.ValidationErrors = append(request.Status.ValidationErrors, errs...)\n\t} else {\n\t\trequest.Spec.VolumeSnapshotLocations = nil\n\t\tfor _, loc := range locs {\n\t\t\trequest.Spec.VolumeSnapshotLocations = append(request.Spec.VolumeSnapshotLocations, loc.Name)\n\t\t\trequest.SnapshotLocations = append(request.SnapshotLocations, loc)\n\t\t}\n\t}\n\n\t// Getting all information of cluster version - useful for future skip-level migration\n\tif request.Annotations == nil {\n\t\trequest.Annotations = make(map[string]string)\n\t}\n\trequest.Annotations[velerov1api.SourceClusterK8sGitVersionAnnotation] = b.discoveryHelper.ServerVersion().String()\n\trequest.Annotations[velerov1api.SourceClusterK8sMajorVersionAnnotation] = b.discoveryHelper.ServerVersion().Major\n\trequest.Annotations[velerov1api.SourceClusterK8sMinorVersionAnnotation] = b.discoveryHelper.ServerVersion().Minor\n\trequest.Annotations[velerov1api.ResourceTimeoutAnnotation] = b.resourceTimeout.String()\n\n\t// Add namespaces with label velero.io/exclude-from-backup=true into request.Spec.ExcludedNamespaces\n\t// Essentially, adding the label velero.io/exclude-from-backup=true to a namespace would be equivalent to setting spec.ExcludedNamespaces\n\tnamespaces := corev1api.NamespaceList{}\n\tif err := b.kbClient.List(context.Background(), &namespaces, kbclient.MatchingLabels{velerov1api.ExcludeFromBackupLabel: \"true\"}); err == nil {\n\t\tfor _, ns := range namespaces.Items {\n\t\t\trequest.Spec.ExcludedNamespaces = append(request.Spec.ExcludedNamespaces, ns.Name)\n\t\t}\n\t} else {\n\t\trequest.Status.ValidationErrors = append(request.Status.ValidationErrors, fmt.Sprintf(\"error getting namespace list: %v\", err))\n\t}\n\n\t// validate whether Included/Excluded resources and IncludedClusterResource are mixed with\n\t// Included/Excluded cluster-scoped/namespace-scoped resources.\n\tif oldAndNewFilterParametersUsedTogether(request.Spec) {\n\t\tvalidatedError := fmt.Sprintf(\"include-resources, exclude-resources and include-cluster-resources are old filter parameters.\\n\" +\n\t\t\t\"include-cluster-scoped-resources, exclude-cluster-scoped-resources, include-namespace-scoped-resources and exclude-namespace-scoped-resources are new filter parameters.\\n\" +\n\t\t\t\"They cannot be used together\")\n\t\trequest.Status.ValidationErrors = append(request.Status.ValidationErrors, validatedError)\n\t}\n\n\tif collections.UseOldResourceFilters(request.Spec) {\n\t\t// validate the included/excluded resources\n\t\tieErr := collections.ValidateIncludesExcludes(request.Spec.IncludedResources, request.Spec.ExcludedResources)\n\t\tif len(ieErr) > 0 {\n\t\t\tfor _, err := range ieErr {\n\t\t\t\trequest.Status.ValidationErrors = append(request.Status.ValidationErrors, fmt.Sprintf(\"Invalid included/excluded resource lists: %v\", err))\n\t\t\t}\n\t\t} else {\n\t\t\trequest.Spec.IncludedResources, request.Spec.ExcludedResources =\n\t\t\t\tmodifyResourceIncludeExclude(\n\t\t\t\t\trequest.Spec.IncludedResources,\n\t\t\t\t\trequest.Spec.ExcludedResources,\n\t\t\t\t\tappend(autoExcludeNamespaceScopedResources, autoExcludeClusterScopedResources...),\n\t\t\t\t)\n\t\t}\n\t} else {\n\t\t// validate the cluster-scoped included/excluded resources\n\t\tclusterErr := collections.ValidateScopedIncludesExcludes(request.Spec.IncludedClusterScopedResources, request.Spec.ExcludedClusterScopedResources)\n\t\tif len(clusterErr) > 0 {\n\t\t\tfor _, err := range clusterErr {\n\t\t\t\trequest.Status.ValidationErrors = append(request.Status.ValidationErrors, fmt.Sprintf(\"Invalid cluster-scoped included/excluded resource lists: %s\", err))\n\t\t\t}\n\t\t} else {\n\t\t\trequest.Spec.IncludedClusterScopedResources, request.Spec.ExcludedClusterScopedResources =\n\t\t\t\tmodifyResourceIncludeExclude(\n\t\t\t\t\trequest.Spec.IncludedClusterScopedResources,\n\t\t\t\t\trequest.Spec.ExcludedClusterScopedResources,\n\t\t\t\t\tautoExcludeClusterScopedResources,\n\t\t\t\t)\n\t\t}\n\n\t\t// validate the namespace-scoped included/excluded resources\n\t\tnamespaceErr := collections.ValidateScopedIncludesExcludes(request.Spec.IncludedNamespaceScopedResources, request.Spec.ExcludedNamespaceScopedResources)\n\t\tif len(namespaceErr) > 0 {\n\t\t\tfor _, err := range namespaceErr {\n\t\t\t\trequest.Status.ValidationErrors = append(request.Status.ValidationErrors, fmt.Sprintf(\"Invalid namespace-scoped included/excluded resource lists: %s\", err))\n\t\t\t}\n\t\t} else {\n\t\t\trequest.Spec.IncludedNamespaceScopedResources, request.Spec.ExcludedNamespaceScopedResources =\n\t\t\t\tmodifyResourceIncludeExclude(\n\t\t\t\t\trequest.Spec.IncludedNamespaceScopedResources,\n\t\t\t\t\trequest.Spec.ExcludedNamespaceScopedResources,\n\t\t\t\t\tautoExcludeNamespaceScopedResources,\n\t\t\t\t)\n\t\t}\n\t}\n\n\t// validate the included/excluded namespaces\n\tfor _, err := range collections.ValidateNamespaceIncludesExcludes(request.Spec.IncludedNamespaces, request.Spec.ExcludedNamespaces) {\n\t\trequest.Status.ValidationErrors = append(request.Status.ValidationErrors, fmt.Sprintf(\"Invalid included/excluded namespace lists: %v\", err))\n\t}\n\n\t// validate that only one exists orLabelSelector or just labelSelector (singular)\n\tif request.Spec.OrLabelSelectors != nil && request.Spec.LabelSelector != nil {\n\t\trequest.Status.ValidationErrors = append(request.Status.ValidationErrors, \"encountered labelSelector as well as orLabelSelectors in backup spec, only one can be specified\")\n\t}\n\n\tresourcePolicies, err := resourcepolicies.GetResourcePoliciesFromBackup(*request.Backup, b.kbClient, logger)\n\tif err != nil {\n\t\trequest.Status.ValidationErrors = append(request.Status.ValidationErrors, err.Error())\n\t}\n\tif resourcePolicies != nil && resourcePolicies.GetIncludeExcludePolicy() != nil && collections.UseOldResourceFilters(request.Spec) {\n\t\trequest.Status.ValidationErrors = append(request.Status.ValidationErrors, \"include-resources, exclude-resources and include-cluster-resources are old filter parameters.\\n\"+\n\t\t\t\"They cannot be used with include-exclude policies.\")\n\t}\n\trequest.ResPolicies = resourcePolicies\n\treturn request\n}\n\n// validateAndGetSnapshotLocations gets a collection of VolumeSnapshotLocation objects that\n// this backup will use (returned as a map of provider name -> VSL), and ensures:\n// - each location name in .spec.volumeSnapshotLocations exists as a location\n// - exactly 1 location per provider\n// - a given provider's default location name is added to .spec.volumeSnapshotLocations if one\n// is not explicitly specified for the provider (if there's only one location for the provider,\n// it will automatically be used)\n//\n// if backup has snapshotVolume disabled then it returns empty VSL\nfunc (b *backupReconciler) validateAndGetSnapshotLocations(backup *velerov1api.Backup) (map[string]*velerov1api.VolumeSnapshotLocation, []string) {\n\terrors := []string{}\n\tproviderLocations := make(map[string]*velerov1api.VolumeSnapshotLocation)\n\n\tfor _, locationName := range backup.Spec.VolumeSnapshotLocations {\n\t\t// validate each locationName exists as a VolumeSnapshotLocation\n\t\tlocation := &velerov1api.VolumeSnapshotLocation{}\n\t\tif err := b.kbClient.Get(context.Background(), kbclient.ObjectKey{Namespace: backup.Namespace, Name: locationName}, location); err != nil {\n\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\terrors = append(errors, fmt.Sprintf(\"a VolumeSnapshotLocation CRD for the location %s with the name specified in the backup spec needs to be created before this snapshot can be executed. Error: %v\", locationName, err))\n\t\t\t} else {\n\t\t\t\terrors = append(errors, fmt.Sprintf(\"error getting volume snapshot location named %s: %v\", locationName, err))\n\t\t\t}\n\t\t\tcontinue\n\t\t}\n\n\t\t// ensure we end up with exactly 1 location *per provider*\n\t\tif providerLocation, ok := providerLocations[location.Spec.Provider]; ok {\n\t\t\t// if > 1 location name per provider as in [\"aws-us-east-1\" | \"aws-us-west-1\"] (same provider, multiple names)\n\t\t\tif providerLocation.Name != locationName {\n\t\t\t\terrors = append(errors, fmt.Sprintf(\"more than one VolumeSnapshotLocation name specified for provider %s: %s; unexpected name was %s\", location.Spec.Provider, locationName, providerLocation.Name))\n\t\t\t\tcontinue\n\t\t\t}\n\t\t} else {\n\t\t\t// keep track of all valid existing locations, per provider\n\t\t\tproviderLocations[location.Spec.Provider] = location\n\t\t}\n\t}\n\n\tif len(errors) > 0 {\n\t\treturn nil, errors\n\t}\n\tvolumeSnapshotLocations := &velerov1api.VolumeSnapshotLocationList{}\n\terr := b.kbClient.List(context.Background(), volumeSnapshotLocations, &kbclient.ListOptions{Namespace: backup.Namespace, LabelSelector: labels.Everything()})\n\tif err != nil {\n\t\terrors = append(errors, fmt.Sprintf(\"error listing volume snapshot locations: %v\", err))\n\t\treturn nil, errors\n\t}\n\n\t// build a map of provider->list of all locations for the provider\n\tallProviderLocations := make(map[string][]*velerov1api.VolumeSnapshotLocation)\n\tfor i := range volumeSnapshotLocations.Items {\n\t\tloc := volumeSnapshotLocations.Items[i]\n\t\tallProviderLocations[loc.Spec.Provider] = append(allProviderLocations[loc.Spec.Provider], &loc)\n\t}\n\n\t// go through each provider and make sure we have/can get a VSL\n\t// for it\n\tfor provider, locations := range allProviderLocations {\n\t\tif _, ok := providerLocations[provider]; ok {\n\t\t\t// backup's spec had a location named for this provider\n\t\t\tcontinue\n\t\t}\n\n\t\tif len(locations) > 1 {\n\t\t\t// more than one possible location for the provider: check\n\t\t\t// the defaults\n\t\t\tdefaultLocation := b.defaultSnapshotLocations[provider]\n\t\t\tif defaultLocation == \"\" {\n\t\t\t\terrors = append(errors, fmt.Sprintf(\"provider %s has more than one possible volume snapshot location, and none were specified explicitly or as a default\", provider))\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tlocation := &velerov1api.VolumeSnapshotLocation{}\n\t\t\tif err := b.kbClient.Get(context.Background(), kbclient.ObjectKey{Namespace: backup.Namespace, Name: defaultLocation}, location); err != nil {\n\t\t\t\terrors = append(errors, fmt.Sprintf(\"error getting volume snapshot location named %s: %v\", defaultLocation, err))\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tproviderLocations[provider] = location\n\t\t\tcontinue\n\t\t}\n\n\t\t// exactly one location for the provider: use it\n\t\tproviderLocations[provider] = locations[0]\n\t}\n\n\tif len(errors) > 0 {\n\t\treturn nil, errors\n\t}\n\n\t// add credential to config for each location\n\tfor _, location := range providerLocations {\n\t\terr = volume.UpdateVolumeSnapshotLocationWithCredentialConfig(location, b.credentialFileStore)\n\t\tif err != nil {\n\t\t\terrors = append(errors, fmt.Sprintf(\"error adding credentials to volume snapshot location named %s: %v\", location.Name, err))\n\t\t\tcontinue\n\t\t}\n\t}\n\n\tif len(errors) > 0 {\n\t\treturn nil, errors\n\t}\n\n\treturn providerLocations, nil\n}\n\n// runBackup runs and uploads a validated backup. Any error returned from this function\n// causes the backup to be Failed; if no error is returned, the backup's status's Errors\n// field is checked to see if the backup was a partial failure.\n\nfunc (b *backupReconciler) runBackup(backup *pkgbackup.Request) error {\n\tb.logger.WithField(constant.ControllerBackup, kubeutil.NamespaceAndName(backup)).Info(\"Setting up backup log\")\n\n\t// Log the backup to both a backup log file and to stdout. This will help see what happened if the upload of the\n\t// backup log failed for whatever reason.\n\tlogCounter := logging.NewLogHook()\n\tbackupLog, err := logging.NewTempFileLogger(b.backupLogLevel, b.formatFlag, logCounter, logrus.Fields{constant.ControllerBackup: kubeutil.NamespaceAndName(backup)})\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error creating dual mode logger for backup\")\n\t}\n\tdefer backupLog.Dispose(b.logger.WithField(constant.ControllerBackup, kubeutil.NamespaceAndName(backup)))\n\n\tbackupLog.Info(\"Setting up backup temp file\")\n\tbackupFile, err := os.CreateTemp(\"\", \"\")\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error creating temp file for backup\")\n\t}\n\tdefer closeAndRemoveFile(backupFile, backupLog)\n\n\tbackupLog.Info(\"Setting up plugin manager\")\n\tpluginManager := b.newPluginManager(backupLog)\n\tdefer pluginManager.CleanupClients()\n\n\tbackupLog.Info(\"Getting backup item actions\")\n\tactions, err := pluginManager.GetBackupItemActionsV2()\n\tif err != nil {\n\t\treturn err\n\t}\n\tbackupLog.Info(\"Getting ItemBlock actions\")\n\tibActions, err := pluginManager.GetItemBlockActions()\n\tif err != nil {\n\t\treturn err\n\t}\n\tbackupLog.Info(\"Setting up backup store to check for backup existence\")\n\tbackupStore, err := b.backupStoreGetter.Get(backup.StorageLocation, pluginManager, backupLog)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\texists, err := backupStore.BackupExists(backup.StorageLocation.Spec.StorageType.ObjectStorage.Bucket, backup.Name)\n\tif exists || err != nil {\n\t\tbackup.Status.Phase = velerov1api.BackupPhaseFailed\n\t\tbackup.Status.CompletionTimestamp = &metav1.Time{Time: b.clock.Now()}\n\t\tif err != nil {\n\t\t\treturn errors.Wrapf(err, \"error checking if backup already exists in object storage\")\n\t\t}\n\t\treturn errors.Errorf(\"backup already exists in object storage\")\n\t}\n\n\tbackupItemActionsResolver := framework.NewBackupItemActionResolverV2(actions)\n\titemBlockActionResolver := framework.NewItemBlockActionResolver(ibActions)\n\n\tvar fatalErrs []error\n\tif err := b.backupper.BackupWithResolvers(backupLog, backup, backupFile, backupItemActionsResolver, itemBlockActionResolver, pluginManager); err != nil {\n\t\tfatalErrs = append(fatalErrs, err)\n\t}\n\n\t// native snapshots phase will either be failed or completed right away\n\t// https://github.com/vmware-tanzu/velero/blob/de3ea52f0cc478e99efa7b9524c7f353514261a4/pkg/backup/item_backupper.go#L632-L639\n\tbackup.Status.VolumeSnapshotsAttempted = len(backup.VolumeSnapshots.Get())\n\tfor _, snap := range backup.VolumeSnapshots.Get() {\n\t\tif snap.Status.Phase == volume.SnapshotPhaseCompleted {\n\t\t\tbackup.Status.VolumeSnapshotsCompleted++\n\t\t}\n\t}\n\tvolumeSnapshots, volumeSnapshotContents, volumeSnapshotClasses := pkgbackup.GetBackupCSIResources(b.kbClient, b.globalCRClient, backup.Backup, backupLog)\n\t// Update CSIVolumeSnapshotsAttempted\n\tbackup.Status.CSIVolumeSnapshotsAttempted = len(volumeSnapshots)\n\n\t// Iterate over backup item operations and update progress.\n\t// Any errors on operations at this point should be added to backup errors.\n\t// If any operations are still not complete, then back will not be set to\n\t// Completed yet.\n\tinProgressOperations, _, opsCompleted, opsFailed, errs := getBackupItemOperationProgress(backup.Backup, pluginManager, *backup.GetItemOperationsList())\n\tif len(errs) > 0 {\n\t\tfor _, err := range errs {\n\t\t\tbackupLog.Error(err)\n\t\t}\n\t}\n\n\tbackup.Status.BackupItemOperationsAttempted = len(*backup.GetItemOperationsList())\n\tbackup.Status.BackupItemOperationsCompleted = opsCompleted\n\tbackup.Status.BackupItemOperationsFailed = opsFailed\n\n\tbackup.Status.Warnings = logCounter.GetCount(logrus.WarnLevel)\n\tbackup.Status.Errors = logCounter.GetCount(logrus.ErrorLevel)\n\n\tbackupWarnings := logCounter.GetEntries(logrus.WarnLevel)\n\tbackupErrors := logCounter.GetEntries(logrus.ErrorLevel)\n\tresults := map[string]results.Result{\n\t\t\"warnings\": backupWarnings,\n\t\t\"errors\": backupErrors,\n\t}\n\n\tbackupLog.DoneForPersist(b.logger.WithField(constant.ControllerBackup, kubeutil.NamespaceAndName(backup)))\n\n\t// Assign finalize phase as close to end as possible so that any errors\n\t// logged to backupLog are captured. This is done before uploading the\n\t// artifacts to object storage so that the JSON representation of the\n\t// backup in object storage has the terminal phase set.\n\tswitch {\n\tcase len(fatalErrs) > 0:\n\t\tbackup.Status.Phase = velerov1api.BackupPhaseFailed\n\tcase logCounter.GetCount(logrus.ErrorLevel) > 0:\n\t\tif inProgressOperations {\n\t\t\tbackup.Status.Phase = velerov1api.BackupPhaseWaitingForPluginOperationsPartiallyFailed\n\t\t} else {\n\t\t\tbackup.Status.Phase = velerov1api.BackupPhaseFinalizingPartiallyFailed\n\t\t}\n\tdefault:\n\t\tif inProgressOperations {\n\t\t\tbackup.Status.Phase = velerov1api.BackupPhaseWaitingForPluginOperations\n\t\t} else {\n\t\t\tbackup.Status.Phase = velerov1api.BackupPhaseFinalizing\n\t\t}\n\t}\n\t// Mark completion timestamp before serializing and uploading.\n\t// Otherwise, the JSON file in object storage has a CompletionTimestamp of 'null'.\n\tif backup.Status.Phase == velerov1api.BackupPhaseFailed ||\n\t\tbackup.Status.Phase == velerov1api.BackupPhasePartiallyFailed ||\n\t\tbackup.Status.Phase == velerov1api.BackupPhaseCompleted {\n\t\tbackup.Status.CompletionTimestamp = &metav1.Time{Time: b.clock.Now()}\n\t}\n\trecordBackupMetrics(backupLog, backup.Backup, backupFile, b.metrics, false)\n\n\t// re-instantiate the backup store because credentials could have changed since the original\n\t// instantiation, if this was a long-running backup\n\tbackupLog.Info(\"Setting up backup store to persist the backup\")\n\tbackupStore, err = b.backupStoreGetter.Get(backup.StorageLocation, pluginManager, backupLog)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif logFile, err := backupLog.GetPersistFile(); err != nil {\n\t\tfatalErrs = append(fatalErrs, errors.Wrap(err, \"error getting backup log file\"))\n\t} else {\n\t\tif errs := persistBackup(backup, backupFile, logFile, backupStore, volumeSnapshots, volumeSnapshotContents, volumeSnapshotClasses, results, b.globalCRClient, backupLog); len(errs) > 0 {\n\t\t\tfatalErrs = append(fatalErrs, errs...)\n\t\t}\n\t}\n\tb.logger.WithField(constant.ControllerBackup, kubeutil.NamespaceAndName(backup)).Infof(\"Initial backup processing complete, moving to %s\", backup.Status.Phase)\n\n\t// if we return a non-nil error, the calling function will update\n\t// the backup's phase to Failed.\n\treturn kerrors.NewAggregate(fatalErrs)\n}\n\nfunc recordBackupMetrics(log logrus.FieldLogger, backup *velerov1api.Backup, backupFile *os.File, serverMetrics *metrics.ServerMetrics, finalize bool) {\n\tbackupScheduleName := backup.GetLabels()[velerov1api.ScheduleNameLabel]\n\n\tif backupFile != nil {\n\t\tvar backupSizeBytes int64\n\t\tif backupFileStat, err := backupFile.Stat(); err != nil {\n\t\t\tlog.WithError(errors.WithStack(err)).Error(\"Error getting backup file info\")\n\t\t} else {\n\t\t\tbackupSizeBytes = backupFileStat.Size()\n\t\t}\n\t\tserverMetrics.SetBackupTarballSizeBytesGauge(backupScheduleName, backupSizeBytes)\n\t}\n\n\tif backup.Status.CompletionTimestamp != nil {\n\t\tbackupDuration := backup.Status.CompletionTimestamp.Time.Sub(backup.Status.StartTimestamp.Time)\n\t\tbackupDurationSeconds := float64(backupDuration / time.Second)\n\t\tserverMetrics.RegisterBackupDuration(backupScheduleName, backupDurationSeconds)\n\t}\n\n\tif !finalize {\n\t\tserverMetrics.RegisterVolumeSnapshotAttempts(backupScheduleName, backup.Status.VolumeSnapshotsAttempted)\n\t\tserverMetrics.RegisterVolumeSnapshotSuccesses(backupScheduleName, backup.Status.VolumeSnapshotsCompleted)\n\t\tserverMetrics.RegisterVolumeSnapshotFailures(backupScheduleName, backup.Status.VolumeSnapshotsAttempted-backup.Status.VolumeSnapshotsCompleted)\n\n\t\tif features.IsEnabled(velerov1api.CSIFeatureFlag) {\n\t\t\tserverMetrics.RegisterCSISnapshotAttempts(backupScheduleName, backup.Name, backup.Status.CSIVolumeSnapshotsAttempted)\n\t\t}\n\n\t\tif backup.Status.Progress != nil {\n\t\t\tserverMetrics.RegisterBackupItemsTotalGauge(backupScheduleName, backup.Status.Progress.TotalItems)\n\t\t}\n\t\tserverMetrics.RegisterBackupItemsErrorsGauge(backupScheduleName, backup.Status.Errors)\n\n\t\tif backup.Status.Warnings > 0 {\n\t\t\tserverMetrics.RegisterBackupWarning(backupScheduleName)\n\t\t}\n\t} else if features.IsEnabled(velerov1api.CSIFeatureFlag) {\n\t\tserverMetrics.RegisterCSISnapshotSuccesses(backupScheduleName, backup.Name, backup.Status.CSIVolumeSnapshotsCompleted)\n\t\tserverMetrics.RegisterCSISnapshotFailures(backupScheduleName, backup.Name, backup.Status.CSIVolumeSnapshotsAttempted-backup.Status.CSIVolumeSnapshotsCompleted)\n\t}\n}\n\nfunc persistBackup(backup *pkgbackup.Request,\n\tbackupContents, backupLog *os.File,\n\tbackupStore persistence.BackupStore,\n\tcsiVolumeSnapshots []snapshotv1api.VolumeSnapshot,\n\tcsiVolumeSnapshotContents []snapshotv1api.VolumeSnapshotContent,\n\tcsiVolumeSnapshotClasses []snapshotv1api.VolumeSnapshotClass,\n\tresults map[string]results.Result,\n\tcrClient kbclient.Client,\n\tlogger logrus.FieldLogger,\n) []error {\n\tpersistErrs := []error{}\n\tbackupJSON := new(bytes.Buffer)\n\n\tif err := encode.To(backup.Backup, \"json\", backupJSON); err != nil {\n\t\tpersistErrs = append(persistErrs, errors.Wrap(err, \"error encoding backup\"))\n\t}\n\n\t// Velero-native volume snapshots (as opposed to CSI ones)\n\tnativeVolumeSnapshots, errs := encode.ToJSONGzip(backup.VolumeSnapshots.Get(), \"native volumesnapshots list\")\n\tif errs != nil {\n\t\tpersistErrs = append(persistErrs, errs...)\n\t}\n\n\tvar backupItemOperations *bytes.Buffer\n\tbackupItemOperations, errs = encode.ToJSONGzip(backup.GetItemOperationsList(), \"backup item operations list\")\n\tif errs != nil {\n\t\tpersistErrs = append(persistErrs, errs...)\n\t}\n\n\tpodVolumeBackups, errs := encode.ToJSONGzip(backup.PodVolumeBackups, \"pod volume backups list\")\n\tif errs != nil {\n\t\tpersistErrs = append(persistErrs, errs...)\n\t}\n\n\tcsiSnapshotClassesJSON, errs := encode.ToJSONGzip(csiVolumeSnapshotClasses, \"csi volume snapshot classes list\")\n\tif errs != nil {\n\t\tpersistErrs = append(persistErrs, errs...)\n\t}\n\n\tbackupResourceList, errs := encode.ToJSONGzip(backup.BackupResourceList(), \"backup resources list\")\n\tif errs != nil {\n\t\tpersistErrs = append(persistErrs, errs...)\n\t}\n\n\tbackupResult, errs := encode.ToJSONGzip(results, \"backup results\")\n\tif errs != nil {\n\t\tpersistErrs = append(persistErrs, errs...)\n\t}\n\n\tbackup.FillVolumesInformation()\n\n\tvolumeInfoJSON, errs := encode.ToJSONGzip(backup.VolumesInformation.Result(\n\t\tcsiVolumeSnapshots,\n\t\tcsiVolumeSnapshotContents,\n\t\tcsiVolumeSnapshotClasses,\n\t\tcrClient,\n\t\tlogger,\n\t), \"backup volumes information\")\n\tif errs != nil {\n\t\tpersistErrs = append(persistErrs, errs...)\n\t}\n\n\tif len(persistErrs) > 0 {\n\t\t// Don't upload the JSON files or backup tarball if encoding to json fails.\n\t\tbackupJSON = nil\n\t\tbackupContents = nil\n\t\tnativeVolumeSnapshots = nil\n\t\tbackupItemOperations = nil\n\t\tbackupResourceList = nil\n\t\tcsiSnapshotClassesJSON = nil\n\t\tbackupResult = nil\n\t\tvolumeInfoJSON = nil\n\t}\n\n\tbackupInfo := persistence.BackupInfo{\n\t\tName: backup.Name,\n\t\tMetadata: backupJSON,\n\t\tContents: backupContents,\n\t\tLog: backupLog,\n\t\tBackupResults: backupResult,\n\t\tPodVolumeBackups: podVolumeBackups,\n\t\tVolumeSnapshots: nativeVolumeSnapshots,\n\t\tBackupItemOperations: backupItemOperations,\n\t\tBackupResourceList: backupResourceList,\n\t\tCSIVolumeSnapshotClasses: csiSnapshotClassesJSON,\n\t\tBackupVolumeInfo: volumeInfoJSON,\n\t}\n\tif err := backupStore.PutBackup(backupInfo); err != nil {\n\t\tpersistErrs = append(persistErrs, err)\n\t}\n\n\treturn persistErrs\n}\n\nfunc closeAndRemoveFile(file *os.File, log logrus.FieldLogger) {\n\tif file == nil {\n\t\tlog.Debug(\"Skipping removal of file due to nil file pointer\")\n\t\treturn\n\t}\n\tif err := file.Close(); err != nil {\n\t\tlog.WithError(err).WithField(\"file\", file.Name()).Error(\"error closing file\")\n\t}\n\tif err := os.Remove(file.Name()); err != nil {\n\t\tlog.WithError(err).WithField(\"file\", file.Name()).Error(\"error removing file\")\n\t}\n}\n\nfunc oldAndNewFilterParametersUsedTogether(backupSpec velerov1api.BackupSpec) bool {\n\thaveOldResourceFilterParameters := len(backupSpec.IncludedResources) > 0 ||\n\t\t(len(backupSpec.ExcludedResources) > 0) ||\n\t\t(backupSpec.IncludeClusterResources != nil)\n\thaveNewResourceFilterParameters := len(backupSpec.IncludedClusterScopedResources) > 0 ||\n\t\t(len(backupSpec.ExcludedClusterScopedResources) > 0) ||\n\t\t(len(backupSpec.IncludedNamespaceScopedResources) > 0) ||\n\t\t(len(backupSpec.ExcludedNamespaceScopedResources) > 0)\n\n\treturn haveOldResourceFilterParameters && haveNewResourceFilterParameters\n}\n\nfunc modifyResourceIncludeExclude(include, exclude, addedExclude []string) (modifiedInclude, modifiedExclude []string) {\n\tmodifiedInclude = include\n\tmodifiedExclude = exclude\n\n\texcludeStrSet := sets.NewString(exclude...)\n\tfor _, ex := range addedExclude {\n\t\tif !excludeStrSet.Has(ex) {\n\t\t\tmodifiedExclude = append(modifiedExclude, ex)\n\t\t}\n\t}\n\n\tfor _, exElem := range modifiedExclude {\n\t\tfor inIndex, inElem := range modifiedInclude {\n\t\t\tif inElem == exElem {\n\t\t\t\tmodifiedInclude = slices.Delete(modifiedInclude, inIndex, inIndex+1)\n\t\t\t}\n\t\t}\n\t}\n\n\treturn modifiedInclude, modifiedExclude\n}\n" }, { "path": "pkg/controller/backup_controller_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"reflect\"\n\t\"sort\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/google/go-cmp/cmp\"\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/version\"\n\t\"k8s.io/utils/clock\"\n\ttestclocks \"k8s.io/utils/clock/testing\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\tfakeClient \"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tpkgbackup \"github.com/vmware-tanzu/velero/pkg/backup\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/discovery\"\n\t\"github.com/vmware-tanzu/velero/pkg/features\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\t\"github.com/vmware-tanzu/velero/pkg/persistence\"\n\tpersistencemocks \"github.com/vmware-tanzu/velero/pkg/persistence/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework\"\n\tpluginmocks \"github.com/vmware-tanzu/velero/pkg/plugin/mocks\"\n\tbiav2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/backupitemaction/v2\"\n\tibav1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/itemblockaction/v1\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\tkubeutil \"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n)\n\ntype fakeBackupper struct {\n\tmock.Mock\n}\n\nfunc (b *fakeBackupper) Backup(logger logrus.FieldLogger, backup *pkgbackup.Request, backupFile io.Writer, actions []biav2.BackupItemAction, itemBlockActions []ibav1.ItemBlockAction, volumeSnapshotterGetter pkgbackup.VolumeSnapshotterGetter) error {\n\targs := b.Called(logger, backup, backupFile, actions, itemBlockActions, volumeSnapshotterGetter)\n\treturn args.Error(0)\n}\n\nfunc (b *fakeBackupper) BackupWithResolvers(logger logrus.FieldLogger, backup *pkgbackup.Request, backupFile io.Writer,\n\tbackupItemActionResolver framework.BackupItemActionResolverV2,\n\titemBlockActionResolver framework.ItemBlockActionResolver,\n\tvolumeSnapshotterGetter pkgbackup.VolumeSnapshotterGetter,\n) error {\n\targs := b.Called(logger, backup, backupFile, backupItemActionResolver, volumeSnapshotterGetter)\n\treturn args.Error(0)\n}\n\nfunc (b *fakeBackupper) FinalizeBackup(\n\tlogger logrus.FieldLogger,\n\tbackup *pkgbackup.Request,\n\tinBackupFile io.Reader,\n\toutBackupFile io.Writer,\n\tbackupItemActionResolver framework.BackupItemActionResolverV2,\n\tasyncBIAOperations []*itemoperation.BackupOperation,\n\tbackupStore persistence.BackupStore,\n) error {\n\targs := b.Called(logger, backup, inBackupFile, outBackupFile, backupItemActionResolver, asyncBIAOperations)\n\treturn args.Error(0)\n}\n\nfunc defaultBackup() *builder.BackupBuilder {\n\treturn builder.ForBackup(velerov1api.DefaultNamespace, \"backup-1\").Phase(velerov1api.BackupPhaseReadyToStart)\n}\n\nfunc namedBackup(name string) *builder.BackupBuilder {\n\treturn builder.ForBackup(velerov1api.DefaultNamespace, name).Phase(velerov1api.BackupPhaseReadyToStart)\n}\n\nfunc TestProcessBackupNonProcessedItems(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tkey string\n\t\tbackup *velerov1api.Backup\n\t}{\n\t\t{\n\t\t\tname: \"New backup is not processed\",\n\t\t\tkey: \"velero/backup-1\",\n\t\t\tbackup: defaultBackup().Phase(velerov1api.BackupPhaseNew).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Queued backup is not processed\",\n\t\t\tkey: \"velero/backup-1\",\n\t\t\tbackup: defaultBackup().Phase(velerov1api.BackupPhaseQueued).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"FailedValidation backup is not processed\",\n\t\t\tkey: \"velero/backup-1\",\n\t\t\tbackup: defaultBackup().Phase(velerov1api.BackupPhaseFailedValidation).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"InProgress backup is not processed\",\n\t\t\tkey: \"velero/backup-1\",\n\t\t\tbackup: defaultBackup().Phase(velerov1api.BackupPhaseInProgress).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Completed backup is not processed\",\n\t\t\tkey: \"velero/backup-1\",\n\t\t\tbackup: defaultBackup().Phase(velerov1api.BackupPhaseCompleted).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Failed backup is not processed\",\n\t\t\tkey: \"velero/backup-1\",\n\t\t\tbackup: defaultBackup().Phase(velerov1api.BackupPhaseFailed).Result(),\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tformatFlag := logging.FormatText\n\t\t\tlogger := logging.DefaultLogger(logrus.DebugLevel, formatFlag)\n\n\t\t\tc := &backupReconciler{\n\t\t\t\tkbClient: velerotest.NewFakeControllerRuntimeClient(t),\n\t\t\t\tformatFlag: formatFlag,\n\t\t\t\tlogger: logger,\n\t\t\t}\n\t\t\tif test.backup != nil {\n\t\t\t\trequire.NoError(t, c.kbClient.Create(t.Context(), test.backup))\n\t\t\t}\n\t\t\tactualResult, err := c.Reconcile(ctx, ctrl.Request{NamespacedName: types.NamespacedName{Namespace: test.backup.Namespace, Name: test.backup.Name}})\n\t\t\tassert.Equal(t, ctrl.Result{}, actualResult)\n\t\t\tassert.NoError(t, err)\n\n\t\t\t// Any backup that would actually proceed to validation will cause a segfault because this\n\t\t\t// test hasn't set up the necessary controller dependencies for validation/etc. So the lack\n\t\t\t// of segfaults during test execution here imply that backups are not being processed, which\n\t\t\t// is what we expect.\n\t\t})\n\t}\n}\n\nfunc TestProcessBackupValidationFailures(t *testing.T) {\n\tdefaultBackupLocation := builder.ForBackupStorageLocation(\"velero\", \"loc-1\").Phase(velerov1api.BackupStorageLocationPhaseAvailable).Result()\n\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\tbackupLocation *velerov1api.BackupStorageLocation\n\t\texpectedErrs []string\n\t}{\n\t\t{\n\t\t\tname: \"invalid included/excluded resources fails validation\",\n\t\t\tbackup: defaultBackup().IncludedResources(\"foo\").ExcludedResources(\"foo\").Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\texpectedErrs: []string{\"Invalid included/excluded resource lists: excludes list cannot contain an item in the includes list: foo\"},\n\t\t},\n\t\t{\n\t\t\tname: \"invalid included/excluded namespaces fails validation\",\n\t\t\tbackup: defaultBackup().IncludedNamespaces(\"foo\").ExcludedNamespaces(\"foo\").Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\texpectedErrs: []string{\"Invalid included/excluded namespace lists: excludes list cannot contain an item in the includes list: foo\"},\n\t\t},\n\t\t{\n\t\t\tname: \"non-existent backup location fails validation\",\n\t\t\tbackup: defaultBackup().StorageLocation(\"nonexistent\").Result(),\n\t\t\texpectedErrs: []string{\"an existing backup storage location was not specified at backup creation time and the default nonexistent was not found. Please address this issue (see `velero backup-location -h` for options) and create a new backup. Error: backupstoragelocations.velero.io \\\"nonexistent\\\" not found\"},\n\t\t},\n\t\t{\n\t\t\tname: \"backup for read-only backup location fails validation\",\n\t\t\tbackup: defaultBackup().StorageLocation(\"read-only\").Result(),\n\t\t\tbackupLocation: builder.ForBackupStorageLocation(\"velero\", \"read-only\").AccessMode(velerov1api.BackupStorageLocationAccessModeReadOnly).Phase(velerov1api.BackupStorageLocationPhaseAvailable).Result(),\n\t\t\texpectedErrs: []string{\"backup can't be created because backup storage location read-only is currently in read-only mode\"},\n\t\t},\n\t\t{\n\t\t\tname: \"labelSelector as well as orLabelSelectors both are specified in backup request fails validation\",\n\t\t\tbackup: defaultBackup().LabelSelector(&metav1.LabelSelector{MatchLabels: map[string]string{\"a\": \"b\"}}).OrLabelSelector([]*metav1.LabelSelector{\n\t\t\t\t{MatchLabels: map[string]string{\"a1\": \"b1\"}},\n\t\t\t\t{MatchLabels: map[string]string{\"a2\": \"b2\"}},\n\t\t\t\t{MatchLabels: map[string]string{\"a3\": \"b3\"}},\n\t\t\t\t{MatchLabels: map[string]string{\"a4\": \"b4\"}},\n\t\t\t}).Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\texpectedErrs: []string{\"encountered labelSelector as well as orLabelSelectors in backup spec, only one can be specified\"},\n\t\t},\n\t\t{\n\t\t\tname: \"use old filter parameters and new filter parameters together\",\n\t\t\tbackup: defaultBackup().IncludeClusterResources(true).IncludedNamespaceScopedResources(\"Deployment\").IncludedNamespaces(\"default\").Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\texpectedErrs: []string{\"include-resources, exclude-resources and include-cluster-resources are old filter parameters.\\ninclude-cluster-scoped-resources, exclude-cluster-scoped-resources, include-namespace-scoped-resources and exclude-namespace-scoped-resources are new filter parameters.\\nThey cannot be used together\"},\n\t\t},\n\t\t{\n\t\t\tname: \"BSL in unavailable state\",\n\t\t\tbackup: defaultBackup().StorageLocation(\"unavailable\").Result(),\n\t\t\tbackupLocation: builder.ForBackupStorageLocation(\"velero\", \"unavailable\").Phase(velerov1api.BackupStorageLocationPhaseUnavailable).Result(),\n\t\t\texpectedErrs: []string{\"backup can't be created because BackupStorageLocation unavailable is in Unavailable status.\"},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tformatFlag := logging.FormatText\n\t\t\tlogger := logging.DefaultLogger(logrus.DebugLevel, formatFlag)\n\n\t\t\tapiServer := velerotest.NewAPIServer(t)\n\t\t\tdiscoveryHelper, err := discovery.NewHelper(apiServer.DiscoveryClient, logger)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tvar fakeClient kbclient.Client\n\t\t\tif test.backupLocation != nil {\n\t\t\t\tfakeClient = velerotest.NewFakeControllerRuntimeClient(t, test.backupLocation)\n\t\t\t} else {\n\t\t\t\tfakeClient = velerotest.NewFakeControllerRuntimeClient(t)\n\t\t\t}\n\n\t\t\tc := &backupReconciler{\n\t\t\t\tlogger: logger,\n\t\t\t\tdiscoveryHelper: discoveryHelper,\n\t\t\t\tkbClient: fakeClient,\n\t\t\t\tdefaultBackupLocation: defaultBackupLocation.Name,\n\t\t\t\tclock: &clock.RealClock{},\n\t\t\t\tformatFlag: formatFlag,\n\t\t\t\tmetrics: metrics.NewServerMetrics(),\n\t\t\t\tbackupTracker: NewBackupTracker(),\n\t\t\t}\n\n\t\t\trequire.NotNil(t, test.backup)\n\t\t\trequire.NoError(t, c.kbClient.Create(t.Context(), test.backup))\n\n\t\t\tactualResult, err := c.Reconcile(ctx, ctrl.Request{NamespacedName: types.NamespacedName{Namespace: test.backup.Namespace, Name: test.backup.Name}})\n\t\t\tassert.Equal(t, ctrl.Result{}, actualResult)\n\t\t\trequire.NoError(t, err)\n\t\t\tres := &velerov1api.Backup{}\n\t\t\terr = c.kbClient.Get(t.Context(), kbclient.ObjectKey{Namespace: test.backup.Namespace, Name: test.backup.Name}, res)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.Equal(t, velerov1api.BackupPhaseFailedValidation, res.Status.Phase)\n\t\t\tassert.Equal(t, test.expectedErrs, res.Status.ValidationErrors)\n\n\t\t\t// Any backup that would actually proceed to processing will cause a segfault because this\n\t\t\t// test hasn't set up the necessary controller dependencies for running backups. So the lack\n\t\t\t// of segfaults during test execution here imply that backups are not being processed, which\n\t\t\t// is what we expect.\n\t\t})\n\t}\n}\n\nfunc TestBackupLocationLabel(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\tbackupLocation *velerov1api.BackupStorageLocation\n\t\texpectedBackupLocation string\n\t}{\n\t\t{\n\t\t\tname: \"valid backup location name should be used as a label\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tbackupLocation: builder.ForBackupStorageLocation(\"velero\", \"loc-1\").Result(),\n\t\t\texpectedBackupLocation: \"loc-1\",\n\t\t},\n\t\t{\n\t\t\tname: \"invalid storage location name should be handled while creating label\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tbackupLocation: builder.ForBackupStorageLocation(\"velero\", \"defaultdefaultdefaultdefaultdefaultdefaultdefaultdefaultdefaultdefault\").Result(),\n\t\t\texpectedBackupLocation: \"defaultdefaultdefaultdefaultdefaultdefaultdefaultdefaultd58343f\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tformatFlag := logging.FormatText\n\n\t\t\tvar (\n\t\t\t\tlogger = logging.DefaultLogger(logrus.DebugLevel, formatFlag)\n\t\t\t\tfakeClient = velerotest.NewFakeControllerRuntimeClient(t)\n\t\t\t)\n\n\t\t\tapiServer := velerotest.NewAPIServer(t)\n\t\t\tdiscoveryHelper, err := discovery.NewHelper(apiServer.DiscoveryClient, logger)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tc := &backupReconciler{\n\t\t\t\tdiscoveryHelper: discoveryHelper,\n\t\t\t\tkbClient: fakeClient,\n\t\t\t\tdefaultBackupLocation: test.backupLocation.Name,\n\t\t\t\tclock: &clock.RealClock{},\n\t\t\t\tformatFlag: formatFlag,\n\t\t\t}\n\n\t\t\tres := c.prepareBackupRequest(ctx, test.backup, logger)\n\t\t\tdefer res.WorkerPool.Stop()\n\t\t\tassert.NotNil(t, res)\n\t\t\tassert.Equal(t, test.expectedBackupLocation, res.Labels[velerov1api.StorageLocationLabel])\n\t\t})\n\t}\n}\n\nfunc Test_prepareBackupRequest_BackupStorageLocation(t *testing.T) {\n\tvar (\n\t\tdefaultBackupTTL = metav1.Duration{Duration: 24 * 30 * time.Hour}\n\t\tdefaultBackupLocation = \"default-location\"\n\t)\n\n\tnow, err := time.Parse(time.RFC1123Z, time.RFC1123Z)\n\trequire.NoError(t, err)\n\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\tbackupLocationNameInBackup string\n\t\tbackupLocationInAPIServer *velerov1api.BackupStorageLocation\n\t\tdefaultBackupLocationInAPIServer *velerov1api.BackupStorageLocation\n\t\texpectedBackupLocation string\n\t\texpectedSuccess bool\n\t\texpectedValidationError string\n\t}{\n\t\t{\n\t\t\tname: \"BackupLocation is specified in backup CR'spec and it can be found in ApiServer\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tbackupLocationNameInBackup: \"test-backup-location\",\n\t\t\tbackupLocationInAPIServer: builder.ForBackupStorageLocation(\"velero\", \"test-backup-location\").Result(),\n\t\t\tdefaultBackupLocationInAPIServer: builder.ForBackupStorageLocation(\"velero\", \"default-location\").Result(),\n\t\t\texpectedBackupLocation: \"test-backup-location\",\n\t\t\texpectedSuccess: true,\n\t\t},\n\t\t{\n\t\t\tname: \"BackupLocation is specified in backup CR'spec and it can't be found in ApiServer\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tbackupLocationNameInBackup: \"test-backup-location\",\n\t\t\tbackupLocationInAPIServer: nil,\n\t\t\tdefaultBackupLocationInAPIServer: nil,\n\t\t\texpectedSuccess: false,\n\t\t\texpectedValidationError: \"an existing backup storage location was not specified at backup creation time and the default test-backup-location was not found. Please address this issue (see `velero backup-location -h` for options) and create a new backup. Error: backupstoragelocations.velero.io \\\"test-backup-location\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"Using default BackupLocation and it can be found in ApiServer\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tbackupLocationNameInBackup: \"\",\n\t\t\tbackupLocationInAPIServer: builder.ForBackupStorageLocation(\"velero\", \"test-backup-location\").Result(),\n\t\t\tdefaultBackupLocationInAPIServer: builder.ForBackupStorageLocation(\"velero\", \"default-location\").Result(),\n\t\t\texpectedBackupLocation: defaultBackupLocation,\n\t\t\texpectedSuccess: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Using default BackupLocation and it can't be found in ApiServer\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tbackupLocationNameInBackup: \"\",\n\t\t\tbackupLocationInAPIServer: nil,\n\t\t\tdefaultBackupLocationInAPIServer: nil,\n\t\t\texpectedSuccess: false,\n\t\t\texpectedValidationError: fmt.Sprintf(\"an existing backup storage location was not specified at backup creation time and the server default %s does not exist. Please address this issue (see `velero backup-location -h` for options) and create a new backup. Error: backupstoragelocations.velero.io \\\"%s\\\" not found\", defaultBackupLocation, defaultBackupLocation),\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\t// Arrange\n\t\t\tvar (\n\t\t\t\tformatFlag = logging.FormatText\n\t\t\t\tlogger = logging.DefaultLogger(logrus.DebugLevel, formatFlag)\n\t\t\t\tapiServer = velerotest.NewAPIServer(t)\n\t\t\t)\n\n\t\t\t// objects that should init with client\n\t\t\tobjects := make([]runtime.Object, 0)\n\t\t\tif test.backupLocationInAPIServer != nil {\n\t\t\t\tobjects = append(objects, test.backupLocationInAPIServer)\n\t\t\t}\n\t\t\tif test.defaultBackupLocationInAPIServer != nil {\n\t\t\t\tobjects = append(objects, test.defaultBackupLocationInAPIServer)\n\t\t\t}\n\t\t\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, objects...)\n\n\t\t\tdiscoveryHelper, err := discovery.NewHelper(apiServer.DiscoveryClient, logger)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tc := &backupReconciler{\n\t\t\t\tdiscoveryHelper: discoveryHelper,\n\t\t\t\tdefaultBackupLocation: defaultBackupLocation,\n\t\t\t\tkbClient: fakeClient,\n\t\t\t\tdefaultBackupTTL: defaultBackupTTL.Duration,\n\t\t\t\tclock: testclocks.NewFakeClock(now),\n\t\t\t\tformatFlag: formatFlag,\n\t\t\t}\n\n\t\t\ttest.backup.Spec.StorageLocation = test.backupLocationNameInBackup\n\n\t\t\t// Run\n\t\t\tres := c.prepareBackupRequest(ctx, test.backup, logger)\n\t\t\tdefer res.WorkerPool.Stop()\n\n\t\t\t// Assert\n\t\t\tif test.expectedSuccess {\n\t\t\t\tassert.Equal(t, test.expectedBackupLocation, res.Spec.StorageLocation)\n\t\t\t\tassert.NotNil(t, res)\n\t\t\t} else {\n\t\t\t\t// in every test case, we only trigger one error at once\n\t\t\t\tif len(res.Status.ValidationErrors) > 1 {\n\t\t\t\t\tassert.Fail(t, \"multi error found in request\")\n\t\t\t\t}\n\t\t\t\tassert.Equal(t, test.expectedValidationError, res.Status.ValidationErrors[0])\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestDefaultBackupTTL(t *testing.T) {\n\tdefaultBackupTTL := metav1.Duration{Duration: 24 * 30 * time.Hour}\n\n\tnow, err := time.Parse(time.RFC1123Z, time.RFC1123Z)\n\trequire.NoError(t, err)\n\tnow = now.Local()\n\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\tbackupLocation *velerov1api.BackupStorageLocation\n\t\texpectedTTL metav1.Duration\n\t\texpectedExpiration metav1.Time\n\t}{\n\t\t{\n\t\t\tname: \"backup with no TTL specified\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\texpectedTTL: defaultBackupTTL,\n\t\t\texpectedExpiration: metav1.NewTime(now.Add(defaultBackupTTL.Duration)),\n\t\t},\n\t\t{\n\t\t\tname: \"backup with TTL specified\",\n\t\t\tbackup: defaultBackup().TTL(time.Hour).Result(),\n\t\t\texpectedTTL: metav1.Duration{Duration: 1 * time.Hour},\n\t\t\texpectedExpiration: metav1.NewTime(now.Add(1 * time.Hour)),\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tformatFlag := logging.FormatText\n\t\tvar (\n\t\t\tfakeClient kbclient.Client\n\t\t\tlogger = logging.DefaultLogger(logrus.DebugLevel, formatFlag)\n\t\t)\n\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tapiServer := velerotest.NewAPIServer(t)\n\t\t\tdiscoveryHelper, err := discovery.NewHelper(apiServer.DiscoveryClient, logger)\n\t\t\trequire.NoError(t, err)\n\t\t\t// add the test's backup storage location if it's different than the default\n\t\t\tif test.backupLocation != nil {\n\t\t\t\tfakeClient = velerotest.NewFakeControllerRuntimeClient(t, test.backupLocation)\n\t\t\t} else {\n\t\t\t\tfakeClient = velerotest.NewFakeControllerRuntimeClient(t)\n\t\t\t}\n\t\t\tc := &backupReconciler{\n\t\t\t\tlogger: logger,\n\t\t\t\tdiscoveryHelper: discoveryHelper,\n\t\t\t\tkbClient: fakeClient,\n\t\t\t\tdefaultBackupTTL: defaultBackupTTL.Duration,\n\t\t\t\tclock: testclocks.NewFakeClock(now),\n\t\t\t\tformatFlag: formatFlag,\n\t\t\t}\n\n\t\t\tres := c.prepareBackupRequest(ctx, test.backup, logger)\n\t\t\tdefer res.WorkerPool.Stop()\n\t\t\tassert.NotNil(t, res)\n\t\t\tassert.Equal(t, test.expectedTTL, res.Spec.TTL)\n\t\t\tassert.Equal(t, test.expectedExpiration, *res.Status.Expiration)\n\t\t})\n\t}\n}\n\nfunc TestPrepareBackupRequest_SetsVGSLabelKey(t *testing.T) {\n\tnow, err := time.Parse(time.RFC1123Z, time.RFC1123Z)\n\trequire.NoError(t, err)\n\tnow = now.Local()\n\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\tserverFlagKey string\n\t\texpectedLabelKey string\n\t}{\n\t\t{\n\t\t\tname: \"backup with spec label key set\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tVolumeGroupSnapshotLabelKey(\"spec-key\").\n\t\t\t\tResult(),\n\t\t\tserverFlagKey: \"server-key\",\n\t\t\texpectedLabelKey: \"spec-key\",\n\t\t},\n\t\t{\n\t\t\tname: \"backup with no spec key, uses server flag\",\n\t\t\tbackup: namedBackup(\"backup-2\").Result(),\n\t\t\tserverFlagKey: \"server-key\",\n\t\t\texpectedLabelKey: \"server-key\",\n\t\t},\n\t\t{\n\t\t\tname: \"backup with no spec or server flag, uses default\",\n\t\t\tbackup: namedBackup(\"backup-3\").Result(),\n\t\t\tserverFlagKey: velerov1api.DefaultVGSLabelKey,\n\t\t\texpectedLabelKey: velerov1api.DefaultVGSLabelKey,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tformatFlag := logging.FormatText\n\t\tlogger := logging.DefaultLogger(logrus.DebugLevel, formatFlag)\n\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, test.backup)\n\t\t\tapiServer := velerotest.NewAPIServer(t)\n\t\t\tdiscoveryHelper, err := discovery.NewHelper(apiServer.DiscoveryClient, logger)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tc := &backupReconciler{\n\t\t\t\tlogger: logger,\n\t\t\t\tkbClient: fakeClient,\n\t\t\t\tdefaultVGSLabelKey: test.serverFlagKey,\n\t\t\t\tdiscoveryHelper: discoveryHelper,\n\t\t\t\tclock: testclocks.NewFakeClock(now),\n\t\t\t}\n\n\t\t\tres := c.prepareBackupRequest(ctx, test.backup, logger)\n\t\t\tdefer res.WorkerPool.Stop()\n\t\t\tassert.NotNil(t, res)\n\n\t\t\tassert.Equal(t, test.expectedLabelKey, res.Spec.VolumeGroupSnapshotLabelKey)\n\t\t})\n\t}\n}\n\nfunc TestDefaultVolumesToResticDeprecation(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\tglobalVal bool\n\t\texpectGlobal bool\n\t\texpectRemap bool\n\t\texpectVal bool\n\t}{\n\t\t{\n\t\t\tname: \"DefaultVolumesToRestic is not set, DefaultVolumesToFsBackup is not set\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tglobalVal: true,\n\t\t\texpectGlobal: true,\n\t\t\texpectVal: true,\n\t\t},\n\t\t{\n\t\t\tname: \"DefaultVolumesToRestic is not set, DefaultVolumesToFsBackup is set to false\",\n\t\t\tbackup: defaultBackup().DefaultVolumesToFsBackup(false).Result(),\n\t\t\tglobalVal: true,\n\t\t\texpectVal: false,\n\t\t},\n\t\t{\n\t\t\tname: \"DefaultVolumesToRestic is not set, DefaultVolumesToFsBackup is set to true\",\n\t\t\tbackup: defaultBackup().DefaultVolumesToFsBackup(true).Result(),\n\t\t\tglobalVal: false,\n\t\t\texpectVal: true,\n\t\t},\n\t\t{\n\t\t\tname: \"DefaultVolumesToRestic is set to false, DefaultVolumesToFsBackup is not set\",\n\t\t\tbackup: defaultBackup().DefaultVolumesToRestic(false).Result(),\n\t\t\tglobalVal: false,\n\t\t\texpectGlobal: true,\n\t\t\texpectVal: false,\n\t\t},\n\t\t{\n\t\t\tname: \"DefaultVolumesToRestic is set to false, DefaultVolumesToFsBackup is set to true\",\n\t\t\tbackup: defaultBackup().DefaultVolumesToRestic(false).DefaultVolumesToFsBackup(true).Result(),\n\t\t\tglobalVal: false,\n\t\t\texpectVal: true,\n\t\t},\n\t\t{\n\t\t\tname: \"DefaultVolumesToRestic is set to false, DefaultVolumesToFsBackup is set to false\",\n\t\t\tbackup: defaultBackup().DefaultVolumesToRestic(false).DefaultVolumesToFsBackup(false).Result(),\n\t\t\tglobalVal: true,\n\t\t\texpectVal: false,\n\t\t},\n\t\t{\n\t\t\tname: \"DefaultVolumesToRestic is set to true, DefaultVolumesToFsBackup is not set\",\n\t\t\tbackup: defaultBackup().DefaultVolumesToRestic(true).Result(),\n\t\t\tglobalVal: false,\n\t\t\texpectRemap: true,\n\t\t\texpectVal: true,\n\t\t},\n\t\t{\n\t\t\tname: \"DefaultVolumesToRestic is set to true, DefaultVolumesToFsBackup is set to false\",\n\t\t\tbackup: defaultBackup().DefaultVolumesToRestic(true).DefaultVolumesToFsBackup(false).Result(),\n\t\t\tglobalVal: false,\n\t\t\texpectRemap: true,\n\t\t\texpectVal: true,\n\t\t},\n\t\t{\n\t\t\tname: \"DefaultVolumesToRestic is set to true, DefaultVolumesToFsBackup is set to true\",\n\t\t\tbackup: defaultBackup().DefaultVolumesToRestic(true).DefaultVolumesToFsBackup(true).Result(),\n\t\t\tglobalVal: false,\n\t\t\texpectRemap: true,\n\t\t\texpectVal: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tformatFlag := logging.FormatText\n\n\t\t\tvar (\n\t\t\t\tlogger = logging.DefaultLogger(logrus.DebugLevel, formatFlag)\n\t\t\t\tfakeClient = velerotest.NewFakeControllerRuntimeClient(t)\n\t\t\t)\n\n\t\t\tapiServer := velerotest.NewAPIServer(t)\n\t\t\tdiscoveryHelper, err := discovery.NewHelper(apiServer.DiscoveryClient, logger)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tc := &backupReconciler{\n\t\t\t\tlogger: logger,\n\t\t\t\tdiscoveryHelper: discoveryHelper,\n\t\t\t\tkbClient: fakeClient,\n\t\t\t\tclock: &clock.RealClock{},\n\t\t\t\tformatFlag: formatFlag,\n\t\t\t\tdefaultVolumesToFsBackup: test.globalVal,\n\t\t\t}\n\n\t\t\tres := c.prepareBackupRequest(ctx, test.backup, logger)\n\t\t\tdefer res.WorkerPool.Stop()\n\t\t\tassert.NotNil(t, res)\n\t\t\tassert.NotNil(t, res.Spec.DefaultVolumesToFsBackup)\n\t\t\tif test.expectRemap {\n\t\t\t\tassert.Equal(t, res.Spec.DefaultVolumesToRestic, res.Spec.DefaultVolumesToFsBackup)\n\t\t\t} else if test.expectGlobal {\n\t\t\t\tassert.NotSame(t, res.Spec.DefaultVolumesToRestic, res.Spec.DefaultVolumesToFsBackup)\n\t\t\t\tassert.Equal(t, &c.defaultVolumesToFsBackup, res.Spec.DefaultVolumesToFsBackup)\n\t\t\t} else {\n\t\t\t\tassert.NotSame(t, res.Spec.DefaultVolumesToRestic, res.Spec.DefaultVolumesToFsBackup)\n\t\t\t\tassert.NotEqual(t, &c.defaultVolumesToFsBackup, res.Spec.DefaultVolumesToFsBackup)\n\t\t\t}\n\n\t\t\tassert.Equal(t, test.expectVal, *res.Spec.DefaultVolumesToFsBackup)\n\t\t})\n\t}\n}\n\nfunc TestProcessBackupCompletions(t *testing.T) {\n\tdefaultBackupLocation := builder.ForBackupStorageLocation(\"velero\", \"loc-1\").Default(true).Bucket(\"store-1\").Phase(velerov1api.BackupStorageLocationPhaseAvailable).Result()\n\n\tnow, err := time.Parse(time.RFC1123Z, time.RFC1123Z)\n\trequire.NoError(t, err)\n\tnow = now.Local()\n\ttimestamp := metav1.NewTime(now)\n\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\tbackupLocation *velerov1api.BackupStorageLocation\n\t\tdefaultVolumesToFsBackup bool\n\t\tdefaultSnapshotMoveData bool\n\t\tenableCSI bool\n\t\texpectedResult *velerov1api.Backup\n\t\tbackupExists bool\n\t\texistenceCheckError error\n\t\tvolumeSnapshot *snapshotv1api.VolumeSnapshot\n\t}{\n\t\t// Finalizing\n\t\t{\n\t\t\tname: \"backup with no backup location gets the default\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\tdefaultVolumesToFsBackup: true,\n\t\t\texpectedResult: &velerov1api.Backup{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tAPIVersion: \"velero.io/v1\",\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"backup-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-major-version\": \"1\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-minor-version\": \"16\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-gitversion\": \"v1.16.4\",\n\t\t\t\t\t\t\"velero.io/resource-timeout\": \"0s\",\n\t\t\t\t\t},\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/storage-location\": \"loc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tStorageLocation: defaultBackupLocation.Name,\n\t\t\t\t\tDefaultVolumesToFsBackup: boolptr.True(),\n\t\t\t\t\tSnapshotMoveData: boolptr.False(),\n\t\t\t\t\tExcludedClusterScopedResources: autoExcludeClusterScopedResources,\n\t\t\t\t\tExcludedNamespaceScopedResources: autoExcludeNamespaceScopedResources,\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\tPhase: velerov1api.BackupPhaseFinalizing,\n\t\t\t\t\tVersion: 1,\n\t\t\t\t\tFormatVersion: \"1.1.0\",\n\t\t\t\t\tStartTimestamp: ×tamp,\n\t\t\t\t\tExpiration: ×tamp,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"backup with a specific backup location keeps it\",\n\t\t\tbackup: defaultBackup().StorageLocation(\"alt-loc\").Result(),\n\t\t\tbackupLocation: builder.ForBackupStorageLocation(\"velero\", \"alt-loc\").Bucket(\"store-1\").Phase(velerov1api.BackupStorageLocationPhaseAvailable).Result(),\n\t\t\tdefaultVolumesToFsBackup: false,\n\t\t\texpectedResult: &velerov1api.Backup{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tAPIVersion: \"velero.io/v1\",\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"backup-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-major-version\": \"1\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-minor-version\": \"16\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-gitversion\": \"v1.16.4\",\n\t\t\t\t\t\t\"velero.io/resource-timeout\": \"0s\",\n\t\t\t\t\t},\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/storage-location\": \"alt-loc\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tStorageLocation: \"alt-loc\",\n\t\t\t\t\tDefaultVolumesToFsBackup: boolptr.False(),\n\t\t\t\t\tSnapshotMoveData: boolptr.False(),\n\t\t\t\t\tExcludedClusterScopedResources: autoExcludeClusterScopedResources,\n\t\t\t\t\tExcludedNamespaceScopedResources: autoExcludeNamespaceScopedResources,\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\tPhase: velerov1api.BackupPhaseFinalizing,\n\t\t\t\t\tVersion: 1,\n\t\t\t\t\tFormatVersion: \"1.1.0\",\n\t\t\t\t\tStartTimestamp: ×tamp,\n\t\t\t\t\tExpiration: ×tamp,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"backup for a location with ReadWrite access mode gets processed\",\n\t\t\tbackup: defaultBackup().StorageLocation(\"read-write\").Result(),\n\t\t\tbackupLocation: builder.ForBackupStorageLocation(\"velero\", \"read-write\").\n\t\t\t\tBucket(\"store-1\").\n\t\t\t\tAccessMode(velerov1api.BackupStorageLocationAccessModeReadWrite).\n\t\t\t\tPhase(velerov1api.BackupStorageLocationPhaseAvailable).\n\t\t\t\tResult(),\n\t\t\tdefaultVolumesToFsBackup: true,\n\t\t\texpectedResult: &velerov1api.Backup{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tAPIVersion: \"velero.io/v1\",\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"backup-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-major-version\": \"1\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-minor-version\": \"16\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-gitversion\": \"v1.16.4\",\n\t\t\t\t\t\t\"velero.io/resource-timeout\": \"0s\",\n\t\t\t\t\t},\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/storage-location\": \"read-write\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tStorageLocation: \"read-write\",\n\t\t\t\t\tDefaultVolumesToFsBackup: boolptr.True(),\n\t\t\t\t\tSnapshotMoveData: boolptr.False(),\n\t\t\t\t\tExcludedClusterScopedResources: autoExcludeClusterScopedResources,\n\t\t\t\t\tExcludedNamespaceScopedResources: autoExcludeNamespaceScopedResources,\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\tPhase: velerov1api.BackupPhaseFinalizing,\n\t\t\t\t\tVersion: 1,\n\t\t\t\t\tFormatVersion: \"1.1.0\",\n\t\t\t\t\tStartTimestamp: ×tamp,\n\t\t\t\t\tExpiration: ×tamp,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"backup with a TTL has expiration set\",\n\t\t\tbackup: defaultBackup().TTL(10 * time.Minute).Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\tdefaultVolumesToFsBackup: false,\n\t\t\texpectedResult: &velerov1api.Backup{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tAPIVersion: \"velero.io/v1\",\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"backup-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-major-version\": \"1\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-minor-version\": \"16\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-gitversion\": \"v1.16.4\",\n\t\t\t\t\t\t\"velero.io/resource-timeout\": \"0s\",\n\t\t\t\t\t},\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/storage-location\": \"loc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tTTL: metav1.Duration{Duration: 10 * time.Minute},\n\t\t\t\t\tStorageLocation: defaultBackupLocation.Name,\n\t\t\t\t\tDefaultVolumesToFsBackup: boolptr.False(),\n\t\t\t\t\tSnapshotMoveData: boolptr.False(),\n\t\t\t\t\tExcludedClusterScopedResources: autoExcludeClusterScopedResources,\n\t\t\t\t\tExcludedNamespaceScopedResources: autoExcludeNamespaceScopedResources,\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\tPhase: velerov1api.BackupPhaseFinalizing,\n\t\t\t\t\tVersion: 1,\n\t\t\t\t\tFormatVersion: \"1.1.0\",\n\t\t\t\t\tExpiration: &metav1.Time{Time: now.Add(10 * time.Minute)},\n\t\t\t\t\tStartTimestamp: ×tamp,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"backup without an existing backup will succeed\",\n\t\t\tbackupExists: false,\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\tdefaultVolumesToFsBackup: true,\n\t\t\texpectedResult: &velerov1api.Backup{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tAPIVersion: \"velero.io/v1\",\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"backup-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-major-version\": \"1\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-minor-version\": \"16\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-gitversion\": \"v1.16.4\",\n\t\t\t\t\t\t\"velero.io/resource-timeout\": \"0s\",\n\t\t\t\t\t},\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/storage-location\": \"loc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tStorageLocation: defaultBackupLocation.Name,\n\t\t\t\t\tDefaultVolumesToFsBackup: boolptr.True(),\n\t\t\t\t\tSnapshotMoveData: boolptr.False(),\n\t\t\t\t\tExcludedClusterScopedResources: autoExcludeClusterScopedResources,\n\t\t\t\t\tExcludedNamespaceScopedResources: autoExcludeNamespaceScopedResources,\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\tPhase: velerov1api.BackupPhaseFinalizing,\n\t\t\t\t\tVersion: 1,\n\t\t\t\t\tFormatVersion: \"1.1.0\",\n\t\t\t\t\tStartTimestamp: ×tamp,\n\t\t\t\t\tExpiration: ×tamp,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"backup specifying a false value for 'DefaultVolumesToFsBackup' keeps it\",\n\t\t\tbackupExists: false,\n\t\t\tbackup: defaultBackup().DefaultVolumesToFsBackup(false).Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\t// value set in the controller is different from that specified in the backup\n\t\t\tdefaultVolumesToFsBackup: true,\n\t\t\texpectedResult: &velerov1api.Backup{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tAPIVersion: \"velero.io/v1\",\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"backup-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-major-version\": \"1\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-minor-version\": \"16\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-gitversion\": \"v1.16.4\",\n\t\t\t\t\t\t\"velero.io/resource-timeout\": \"0s\",\n\t\t\t\t\t},\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/storage-location\": \"loc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tStorageLocation: defaultBackupLocation.Name,\n\t\t\t\t\tDefaultVolumesToFsBackup: boolptr.False(),\n\t\t\t\t\tSnapshotMoveData: boolptr.False(),\n\t\t\t\t\tExcludedClusterScopedResources: autoExcludeClusterScopedResources,\n\t\t\t\t\tExcludedNamespaceScopedResources: autoExcludeNamespaceScopedResources,\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\tPhase: velerov1api.BackupPhaseFinalizing,\n\t\t\t\t\tVersion: 1,\n\t\t\t\t\tFormatVersion: \"1.1.0\",\n\t\t\t\t\tStartTimestamp: ×tamp,\n\t\t\t\t\tExpiration: ×tamp,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"backup specifying a true value for 'DefaultVolumesToFsBackup' keeps it\",\n\t\t\tbackupExists: false,\n\t\t\tbackup: defaultBackup().DefaultVolumesToFsBackup(true).Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\t// value set in the controller is different from that specified in the backup\n\t\t\tdefaultVolumesToFsBackup: false,\n\t\t\texpectedResult: &velerov1api.Backup{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tAPIVersion: \"velero.io/v1\",\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"backup-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-major-version\": \"1\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-minor-version\": \"16\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-gitversion\": \"v1.16.4\",\n\t\t\t\t\t\t\"velero.io/resource-timeout\": \"0s\",\n\t\t\t\t\t},\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/storage-location\": \"loc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tStorageLocation: defaultBackupLocation.Name,\n\t\t\t\t\tDefaultVolumesToFsBackup: boolptr.True(),\n\t\t\t\t\tSnapshotMoveData: boolptr.False(),\n\t\t\t\t\tExcludedClusterScopedResources: autoExcludeClusterScopedResources,\n\t\t\t\t\tExcludedNamespaceScopedResources: autoExcludeNamespaceScopedResources,\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\tPhase: velerov1api.BackupPhaseFinalizing,\n\t\t\t\t\tVersion: 1,\n\t\t\t\t\tFormatVersion: \"1.1.0\",\n\t\t\t\t\tStartTimestamp: ×tamp,\n\t\t\t\t\tExpiration: ×tamp,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"backup specifying no value for 'DefaultVolumesToFsBackup' gets the default true value\",\n\t\t\tbackupExists: false,\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\t// value set in the controller is different from that specified in the backup\n\t\t\tdefaultVolumesToFsBackup: true,\n\t\t\texpectedResult: &velerov1api.Backup{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tAPIVersion: \"velero.io/v1\",\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"backup-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-major-version\": \"1\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-minor-version\": \"16\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-gitversion\": \"v1.16.4\",\n\t\t\t\t\t\t\"velero.io/resource-timeout\": \"0s\",\n\t\t\t\t\t},\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/storage-location\": \"loc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tStorageLocation: defaultBackupLocation.Name,\n\t\t\t\t\tDefaultVolumesToFsBackup: boolptr.True(),\n\t\t\t\t\tSnapshotMoveData: boolptr.False(),\n\t\t\t\t\tExcludedClusterScopedResources: autoExcludeClusterScopedResources,\n\t\t\t\t\tExcludedNamespaceScopedResources: autoExcludeNamespaceScopedResources,\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\tPhase: velerov1api.BackupPhaseFinalizing,\n\t\t\t\t\tVersion: 1,\n\t\t\t\t\tFormatVersion: \"1.1.0\",\n\t\t\t\t\tStartTimestamp: ×tamp,\n\t\t\t\t\tExpiration: ×tamp,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"backup specifying no value for 'DefaultVolumesToFsBackup' gets the default false value\",\n\t\t\tbackupExists: false,\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\t// value set in the controller is different from that specified in the backup\n\t\t\tdefaultVolumesToFsBackup: false,\n\t\t\texpectedResult: &velerov1api.Backup{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tAPIVersion: \"velero.io/v1\",\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"backup-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-major-version\": \"1\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-minor-version\": \"16\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-gitversion\": \"v1.16.4\",\n\t\t\t\t\t\t\"velero.io/resource-timeout\": \"0s\",\n\t\t\t\t\t},\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/storage-location\": \"loc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tStorageLocation: defaultBackupLocation.Name,\n\t\t\t\t\tDefaultVolumesToFsBackup: boolptr.False(),\n\t\t\t\t\tSnapshotMoveData: boolptr.False(),\n\t\t\t\t\tExcludedClusterScopedResources: autoExcludeClusterScopedResources,\n\t\t\t\t\tExcludedNamespaceScopedResources: autoExcludeNamespaceScopedResources,\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\tPhase: velerov1api.BackupPhaseFinalizing,\n\t\t\t\t\tVersion: 1,\n\t\t\t\t\tFormatVersion: \"1.1.0\",\n\t\t\t\t\tStartTimestamp: ×tamp,\n\t\t\t\t\tExpiration: ×tamp,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\n\t\t// Failed\n\t\t{\n\t\t\tname: \"backup with existing backup will fail\",\n\t\t\tbackupExists: true,\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\tdefaultVolumesToFsBackup: true,\n\t\t\texpectedResult: &velerov1api.Backup{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tAPIVersion: \"velero.io/v1\",\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"backup-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-major-version\": \"1\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-minor-version\": \"16\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-gitversion\": \"v1.16.4\",\n\t\t\t\t\t\t\"velero.io/resource-timeout\": \"0s\",\n\t\t\t\t\t},\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/storage-location\": \"loc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tStorageLocation: defaultBackupLocation.Name,\n\t\t\t\t\tDefaultVolumesToFsBackup: boolptr.True(),\n\t\t\t\t\tSnapshotMoveData: boolptr.False(),\n\t\t\t\t\tExcludedClusterScopedResources: autoExcludeClusterScopedResources,\n\t\t\t\t\tExcludedNamespaceScopedResources: autoExcludeNamespaceScopedResources,\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\tPhase: velerov1api.BackupPhaseFailed,\n\t\t\t\t\tFailureReason: \"backup already exists in object storage\",\n\t\t\t\t\tVersion: 1,\n\t\t\t\t\tFormatVersion: \"1.1.0\",\n\t\t\t\t\tStartTimestamp: ×tamp,\n\t\t\t\t\tCompletionTimestamp: ×tamp,\n\t\t\t\t\tExpiration: ×tamp,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"error when checking if backup exists will cause backup to fail\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\texistenceCheckError: errors.New(\"Backup already exists in object storage\"),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\tdefaultVolumesToFsBackup: true,\n\t\t\texpectedResult: &velerov1api.Backup{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tAPIVersion: \"velero.io/v1\",\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"backup-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-major-version\": \"1\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-minor-version\": \"16\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-gitversion\": \"v1.16.4\",\n\t\t\t\t\t\t\"velero.io/resource-timeout\": \"0s\",\n\t\t\t\t\t},\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/storage-location\": \"loc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tStorageLocation: defaultBackupLocation.Name,\n\t\t\t\t\tDefaultVolumesToFsBackup: boolptr.True(),\n\t\t\t\t\tSnapshotMoveData: boolptr.False(),\n\t\t\t\t\tExcludedClusterScopedResources: autoExcludeClusterScopedResources,\n\t\t\t\t\tExcludedNamespaceScopedResources: autoExcludeNamespaceScopedResources,\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\tPhase: velerov1api.BackupPhaseFailed,\n\t\t\t\t\tFailureReason: \"error checking if backup already exists in object storage: Backup already exists in object storage\",\n\t\t\t\t\tVersion: 1,\n\t\t\t\t\tFormatVersion: \"1.1.0\",\n\t\t\t\t\tStartTimestamp: ×tamp,\n\t\t\t\t\tCompletionTimestamp: ×tamp,\n\t\t\t\t\tExpiration: ×tamp,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"backup with snapshot data movement when CSI feature is enabled\",\n\t\t\tbackup: defaultBackup().SnapshotMoveData(true).Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\tdefaultVolumesToFsBackup: false,\n\t\t\tenableCSI: true,\n\t\t\texpectedResult: &velerov1api.Backup{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tAPIVersion: \"velero.io/v1\",\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"backup-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-major-version\": \"1\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-minor-version\": \"16\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-gitversion\": \"v1.16.4\",\n\t\t\t\t\t\t\"velero.io/resource-timeout\": \"0s\",\n\t\t\t\t\t},\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/storage-location\": \"loc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tStorageLocation: defaultBackupLocation.Name,\n\t\t\t\t\tDefaultVolumesToFsBackup: boolptr.False(),\n\t\t\t\t\tSnapshotMoveData: boolptr.True(),\n\t\t\t\t\tExcludedClusterScopedResources: autoExcludeClusterScopedResources,\n\t\t\t\t\tExcludedNamespaceScopedResources: autoExcludeNamespaceScopedResources,\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\tPhase: velerov1api.BackupPhaseFinalizing,\n\t\t\t\t\tVersion: 1,\n\t\t\t\t\tFormatVersion: \"1.1.0\",\n\t\t\t\t\tStartTimestamp: ×tamp,\n\t\t\t\t\tExpiration: ×tamp,\n\t\t\t\t\tCSIVolumeSnapshotsAttempted: 0,\n\t\t\t\t\tCSIVolumeSnapshotsCompleted: 0,\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshot: builder.ForVolumeSnapshot(\"velero\", \"testVS\").VolumeSnapshotClass(\"testClass\").Status().BoundVolumeSnapshotContentName(\"testVSC\").RestoreSize(\"10G\").SourcePVC(\"testPVC\").ObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, \"backup-1\")).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"backup with snapshot data movement set to false when CSI feature is enabled\",\n\t\t\tbackup: defaultBackup().SnapshotMoveData(false).Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\tdefaultVolumesToFsBackup: false,\n\t\t\tenableCSI: true,\n\t\t\texpectedResult: &velerov1api.Backup{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tAPIVersion: \"velero.io/v1\",\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"backup-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-major-version\": \"1\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-minor-version\": \"16\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-gitversion\": \"v1.16.4\",\n\t\t\t\t\t\t\"velero.io/resource-timeout\": \"0s\",\n\t\t\t\t\t},\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/storage-location\": \"loc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tStorageLocation: defaultBackupLocation.Name,\n\t\t\t\t\tDefaultVolumesToFsBackup: boolptr.False(),\n\t\t\t\t\tSnapshotMoveData: boolptr.False(),\n\t\t\t\t\tExcludedClusterScopedResources: autoExcludeClusterScopedResources,\n\t\t\t\t\tExcludedNamespaceScopedResources: autoExcludeNamespaceScopedResources,\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\tPhase: velerov1api.BackupPhaseFinalizing,\n\t\t\t\t\tVersion: 1,\n\t\t\t\t\tFormatVersion: \"1.1.0\",\n\t\t\t\t\tStartTimestamp: ×tamp,\n\t\t\t\t\tExpiration: ×tamp,\n\t\t\t\t\tCSIVolumeSnapshotsAttempted: 1,\n\t\t\t\t\tCSIVolumeSnapshotsCompleted: 0,\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshot: builder.ForVolumeSnapshot(\"velero\", \"testVS\").VolumeSnapshotClass(\"testClass\").Status().BoundVolumeSnapshotContentName(\"testVSC\").RestoreSize(\"10G\").SourcePVC(\"testPVC\").ObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, \"backup-1\")).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"backup with snapshot data movement not set when CSI feature is enabled\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\tdefaultVolumesToFsBackup: false,\n\t\t\tenableCSI: true,\n\t\t\texpectedResult: &velerov1api.Backup{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tAPIVersion: \"velero.io/v1\",\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"backup-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-major-version\": \"1\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-minor-version\": \"16\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-gitversion\": \"v1.16.4\",\n\t\t\t\t\t\t\"velero.io/resource-timeout\": \"0s\",\n\t\t\t\t\t},\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/storage-location\": \"loc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tStorageLocation: defaultBackupLocation.Name,\n\t\t\t\t\tDefaultVolumesToFsBackup: boolptr.False(),\n\t\t\t\t\tSnapshotMoveData: boolptr.False(),\n\t\t\t\t\tExcludedClusterScopedResources: autoExcludeClusterScopedResources,\n\t\t\t\t\tExcludedNamespaceScopedResources: autoExcludeNamespaceScopedResources,\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\tPhase: velerov1api.BackupPhaseFinalizing,\n\t\t\t\t\tVersion: 1,\n\t\t\t\t\tFormatVersion: \"1.1.0\",\n\t\t\t\t\tStartTimestamp: ×tamp,\n\t\t\t\t\tExpiration: ×tamp,\n\t\t\t\t\tCSIVolumeSnapshotsAttempted: 1,\n\t\t\t\t\tCSIVolumeSnapshotsCompleted: 0,\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshot: builder.ForVolumeSnapshot(\"velero\", \"testVS\").VolumeSnapshotClass(\"testClass\").Status().BoundVolumeSnapshotContentName(\"testVSC\").RestoreSize(\"10G\").SourcePVC(\"testPVC\").ObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, \"backup-1\")).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"backup with snapshot data movement set to true and defaultSnapshotMoveData set to false\",\n\t\t\tbackup: defaultBackup().SnapshotMoveData(true).Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\tdefaultVolumesToFsBackup: false,\n\t\t\tdefaultSnapshotMoveData: false,\n\t\t\texpectedResult: &velerov1api.Backup{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tAPIVersion: \"velero.io/v1\",\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"backup-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-major-version\": \"1\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-minor-version\": \"16\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-gitversion\": \"v1.16.4\",\n\t\t\t\t\t\t\"velero.io/resource-timeout\": \"0s\",\n\t\t\t\t\t},\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/storage-location\": \"loc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tStorageLocation: defaultBackupLocation.Name,\n\t\t\t\t\tDefaultVolumesToFsBackup: boolptr.False(),\n\t\t\t\t\tSnapshotMoveData: boolptr.True(),\n\t\t\t\t\tExcludedClusterScopedResources: autoExcludeClusterScopedResources,\n\t\t\t\t\tExcludedNamespaceScopedResources: autoExcludeNamespaceScopedResources,\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\tPhase: velerov1api.BackupPhaseFinalizing,\n\t\t\t\t\tVersion: 1,\n\t\t\t\t\tFormatVersion: \"1.1.0\",\n\t\t\t\t\tStartTimestamp: ×tamp,\n\t\t\t\t\tExpiration: ×tamp,\n\t\t\t\t\tCSIVolumeSnapshotsAttempted: 0,\n\t\t\t\t\tCSIVolumeSnapshotsCompleted: 0,\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshot: builder.ForVolumeSnapshot(\"velero\", \"testVS\").VolumeSnapshotClass(\"testClass\").Status().BoundVolumeSnapshotContentName(\"testVSC\").RestoreSize(\"10G\").SourcePVC(\"testPVC\").ObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, \"backup-1\")).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"backup with snapshot data movement set to false and defaultSnapshotMoveData set to true\",\n\t\t\tbackup: defaultBackup().SnapshotMoveData(false).Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\tdefaultVolumesToFsBackup: false,\n\t\t\tdefaultSnapshotMoveData: true,\n\t\t\tenableCSI: true,\n\t\t\texpectedResult: &velerov1api.Backup{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tAPIVersion: \"velero.io/v1\",\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"backup-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-major-version\": \"1\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-minor-version\": \"16\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-gitversion\": \"v1.16.4\",\n\t\t\t\t\t\t\"velero.io/resource-timeout\": \"0s\",\n\t\t\t\t\t},\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/storage-location\": \"loc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tStorageLocation: defaultBackupLocation.Name,\n\t\t\t\t\tDefaultVolumesToFsBackup: boolptr.False(),\n\t\t\t\t\tSnapshotMoveData: boolptr.False(),\n\t\t\t\t\tExcludedClusterScopedResources: autoExcludeClusterScopedResources,\n\t\t\t\t\tExcludedNamespaceScopedResources: autoExcludeNamespaceScopedResources,\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\tPhase: velerov1api.BackupPhaseFinalizing,\n\t\t\t\t\tVersion: 1,\n\t\t\t\t\tFormatVersion: \"1.1.0\",\n\t\t\t\t\tStartTimestamp: ×tamp,\n\t\t\t\t\tExpiration: ×tamp,\n\t\t\t\t\tCSIVolumeSnapshotsAttempted: 1,\n\t\t\t\t\tCSIVolumeSnapshotsCompleted: 0,\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshot: builder.ForVolumeSnapshot(\"velero\", \"testVS\").VolumeSnapshotClass(\"testClass\").Status().BoundVolumeSnapshotContentName(\"testVSC\").RestoreSize(\"10G\").SourcePVC(\"testPVC\").ObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, \"backup-1\")).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"backup with snapshot data movement not set and defaultSnapshotMoveData set to true\",\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\tdefaultVolumesToFsBackup: false,\n\t\t\tdefaultSnapshotMoveData: true,\n\t\t\texpectedResult: &velerov1api.Backup{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tAPIVersion: \"velero.io/v1\",\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"backup-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-major-version\": \"1\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-minor-version\": \"16\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-gitversion\": \"v1.16.4\",\n\t\t\t\t\t\t\"velero.io/resource-timeout\": \"0s\",\n\t\t\t\t\t},\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/storage-location\": \"loc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tStorageLocation: defaultBackupLocation.Name,\n\t\t\t\t\tDefaultVolumesToFsBackup: boolptr.False(),\n\t\t\t\t\tSnapshotMoveData: boolptr.True(),\n\t\t\t\t\tExcludedClusterScopedResources: autoExcludeClusterScopedResources,\n\t\t\t\t\tExcludedNamespaceScopedResources: autoExcludeNamespaceScopedResources,\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\tPhase: velerov1api.BackupPhaseFinalizing,\n\t\t\t\t\tVersion: 1,\n\t\t\t\t\tFormatVersion: \"1.1.0\",\n\t\t\t\t\tStartTimestamp: ×tamp,\n\t\t\t\t\tExpiration: ×tamp,\n\t\t\t\t\tCSIVolumeSnapshotsAttempted: 0,\n\t\t\t\t\tCSIVolumeSnapshotsCompleted: 0,\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshot: builder.ForVolumeSnapshot(\"velero\", \"testVS\").VolumeSnapshotClass(\"testClass\").Status().BoundVolumeSnapshotContentName(\"testVSC\").RestoreSize(\"10G\").SourcePVC(\"testPVC\").ObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, \"backup-1\")).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"backup with namespace-scoped and cluster-scoped resource filters\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tExcludedClusterScopedResources(\"clusterroles\").\n\t\t\t\tIncludedClusterScopedResources(\"storageclasses\").\n\t\t\t\tExcludedNamespaceScopedResources(\"secrets\").\n\t\t\t\tIncludedNamespaceScopedResources(\"pods\").Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\tdefaultVolumesToFsBackup: false,\n\t\t\tdefaultSnapshotMoveData: true,\n\t\t\texpectedResult: &velerov1api.Backup{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tAPIVersion: \"velero.io/v1\",\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"backup-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-major-version\": \"1\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-minor-version\": \"16\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-gitversion\": \"v1.16.4\",\n\t\t\t\t\t\t\"velero.io/resource-timeout\": \"0s\",\n\t\t\t\t\t},\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/storage-location\": \"loc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tStorageLocation: defaultBackupLocation.Name,\n\t\t\t\t\tDefaultVolumesToFsBackup: boolptr.False(),\n\t\t\t\t\tSnapshotMoveData: boolptr.True(),\n\t\t\t\t\tIncludedClusterScopedResources: []string{\"storageclasses\"},\n\t\t\t\t\tExcludedClusterScopedResources: append([]string{\"clusterroles\"}, autoExcludeClusterScopedResources...),\n\t\t\t\t\tIncludedNamespaceScopedResources: []string{\"pods\"},\n\t\t\t\t\tExcludedNamespaceScopedResources: append([]string{\"secrets\"}, autoExcludeNamespaceScopedResources...),\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\tPhase: velerov1api.BackupPhaseFinalizing,\n\t\t\t\t\tVersion: 1,\n\t\t\t\t\tFormatVersion: \"1.1.0\",\n\t\t\t\t\tStartTimestamp: ×tamp,\n\t\t\t\t\tExpiration: ×tamp,\n\t\t\t\t\tCSIVolumeSnapshotsAttempted: 0,\n\t\t\t\t\tCSIVolumeSnapshotsCompleted: 0,\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshot: builder.ForVolumeSnapshot(\"velero\", \"testVS\").VolumeSnapshotClass(\"testClass\").Status().BoundVolumeSnapshotContentName(\"testVSC\").RestoreSize(\"10G\").SourcePVC(\"testPVC\").ObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, \"backup-1\")).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"backup's include filter overlap with default exclude resources\",\n\t\t\tbackup: defaultBackup().\n\t\t\t\tExcludedClusterScopedResources(\"clusterroles\").\n\t\t\t\tIncludedClusterScopedResources(\"storageclasses\", \"volumesnapshotcontents.snapshot.storage.k8s.io\").\n\t\t\t\tExcludedNamespaceScopedResources(\"secrets\").\n\t\t\t\tIncludedNamespaceScopedResources(\"pods\", \"volumesnapshots.snapshot.storage.k8s.io\").Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\tdefaultVolumesToFsBackup: false,\n\t\t\tdefaultSnapshotMoveData: true,\n\t\t\texpectedResult: &velerov1api.Backup{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tAPIVersion: \"velero.io/v1\",\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"backup-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-major-version\": \"1\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-minor-version\": \"16\",\n\t\t\t\t\t\t\"velero.io/source-cluster-k8s-gitversion\": \"v1.16.4\",\n\t\t\t\t\t\t\"velero.io/resource-timeout\": \"0s\",\n\t\t\t\t\t},\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"velero.io/storage-location\": \"loc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tStorageLocation: defaultBackupLocation.Name,\n\t\t\t\t\tDefaultVolumesToFsBackup: boolptr.False(),\n\t\t\t\t\tSnapshotMoveData: boolptr.True(),\n\t\t\t\t\tIncludedClusterScopedResources: []string{\"storageclasses\"},\n\t\t\t\t\tExcludedClusterScopedResources: append([]string{\"clusterroles\"}, autoExcludeClusterScopedResources...),\n\t\t\t\t\tIncludedNamespaceScopedResources: []string{\"pods\"},\n\t\t\t\t\tExcludedNamespaceScopedResources: append([]string{\"secrets\"}, autoExcludeNamespaceScopedResources...),\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\t\tPhase: velerov1api.BackupPhaseFinalizing,\n\t\t\t\t\tVersion: 1,\n\t\t\t\t\tFormatVersion: \"1.1.0\",\n\t\t\t\t\tStartTimestamp: ×tamp,\n\t\t\t\t\tExpiration: ×tamp,\n\t\t\t\t\tCSIVolumeSnapshotsAttempted: 0,\n\t\t\t\t\tCSIVolumeSnapshotsCompleted: 0,\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshot: builder.ForVolumeSnapshot(\"velero\", \"testVS\").VolumeSnapshotClass(\"testClass\").Status().BoundVolumeSnapshotContentName(\"testVSC\").RestoreSize(\"10G\").SourcePVC(\"testPVC\").ObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, \"backup-1\")).Result(),\n\t\t},\n\t}\n\n\tsnapshotHandle := \"testSnapshotID\"\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tformatFlag := logging.FormatText\n\t\t\tvar (\n\t\t\t\tlogger = logging.DefaultLogger(logrus.DebugLevel, formatFlag)\n\t\t\t\tpluginManager = new(pluginmocks.Manager)\n\t\t\t\tbackupStore = new(persistencemocks.BackupStore)\n\t\t\t\tbackupper = new(fakeBackupper)\n\t\t\t\tfakeGlobalClient = velerotest.NewFakeControllerRuntimeClient(t)\n\t\t\t)\n\n\t\t\tvar fakeClient kbclient.Client\n\t\t\t// add the test's backup storage location if it's different than the default\n\t\t\tif test.backupLocation != nil && test.backupLocation != defaultBackupLocation {\n\t\t\t\tfakeClient = velerotest.NewFakeControllerRuntimeClient(t, test.backupLocation,\n\t\t\t\t\tbuilder.ForVolumeSnapshotClass(\"testClass\").Driver(\"testDriver\").Result(),\n\t\t\t\t\tbuilder.ForVolumeSnapshotContent(\"testVSC\").ObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, \"backup-1\")).VolumeSnapshotClassName(\"testClass\").Status(&snapshotv1api.VolumeSnapshotContentStatus{\n\t\t\t\t\t\tSnapshotHandle: &snapshotHandle,\n\t\t\t\t\t}).Result(),\n\t\t\t\t)\n\t\t\t} else {\n\t\t\t\tfakeClient = velerotest.NewFakeControllerRuntimeClient(t,\n\t\t\t\t\tbuilder.ForVolumeSnapshotClass(\"testClass\").Driver(\"testDriver\").Result(),\n\t\t\t\t\tbuilder.ForVolumeSnapshotContent(\"testVSC\").ObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, \"backup-1\")).VolumeSnapshotClassName(\"testClass\").Status(&snapshotv1api.VolumeSnapshotContentStatus{\n\t\t\t\t\t\tSnapshotHandle: &snapshotHandle,\n\t\t\t\t\t}).Result(),\n\t\t\t\t)\n\t\t\t}\n\n\t\t\tif test.volumeSnapshot != nil {\n\t\t\t\trequire.NoError(t, fakeGlobalClient.Create(t.Context(), test.volumeSnapshot))\n\t\t\t}\n\n\t\t\tapiServer := velerotest.NewAPIServer(t)\n\n\t\t\tapiServer.DiscoveryClient.FakedServerVersion = &version.Info{\n\t\t\t\tMajor: \"1\",\n\t\t\t\tMinor: \"16\",\n\t\t\t\tGitVersion: \"v1.16.4\",\n\t\t\t\tGitCommit: \"FakeTest\",\n\t\t\t\tGitTreeState: \"\",\n\t\t\t\tBuildDate: \"\",\n\t\t\t\tGoVersion: \"\",\n\t\t\t\tCompiler: \"\",\n\t\t\t\tPlatform: \"\",\n\t\t\t}\n\n\t\t\tdiscoveryHelper, err := discovery.NewHelper(apiServer.DiscoveryClient, logger)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tc := &backupReconciler{\n\t\t\t\tlogger: logger,\n\t\t\t\tdiscoveryHelper: discoveryHelper,\n\t\t\t\tkbClient: fakeClient,\n\t\t\t\tdefaultBackupLocation: defaultBackupLocation.Name,\n\t\t\t\tdefaultVolumesToFsBackup: test.defaultVolumesToFsBackup,\n\t\t\t\tdefaultSnapshotMoveData: test.defaultSnapshotMoveData,\n\t\t\t\tbackupTracker: NewBackupTracker(),\n\t\t\t\tmetrics: metrics.NewServerMetrics(),\n\t\t\t\tclock: testclocks.NewFakeClock(now),\n\t\t\t\tnewPluginManager: func(logrus.FieldLogger) clientmgmt.Manager { return pluginManager },\n\t\t\t\tbackupStoreGetter: NewFakeSingleObjectBackupStoreGetter(backupStore),\n\t\t\t\tbackupper: backupper,\n\t\t\t\tformatFlag: formatFlag,\n\t\t\t\tglobalCRClient: fakeGlobalClient,\n\t\t\t}\n\n\t\t\tpluginManager.On(\"GetBackupItemActionsV2\").Return(nil, nil)\n\t\t\tpluginManager.On(\"GetItemBlockActions\").Return(nil, nil)\n\t\t\tpluginManager.On(\"CleanupClients\").Return(nil)\n\t\t\tbackupper.On(\"Backup\", mock.Anything, mock.Anything, mock.Anything, []biav2.BackupItemAction(nil), pluginManager).Return(nil)\n\t\t\tbackupper.On(\"BackupWithResolvers\", mock.Anything, mock.Anything, mock.Anything, framework.BackupItemActionResolverV2{}, pluginManager).Return(nil)\n\t\t\tbackupStore.On(\"BackupExists\", test.backupLocation.Spec.StorageType.ObjectStorage.Bucket, test.backup.Name).Return(test.backupExists, test.existenceCheckError)\n\n\t\t\t// Ensure we have a CompletionTimestamp when uploading and that the backup name matches the backup in the object store.\n\t\t\t// Failures will display the bytes in buf.\n\t\t\thasNameAndCompletionTimestampIfCompleted := func(info persistence.BackupInfo) bool {\n\t\t\t\tbuf := new(bytes.Buffer)\n\t\t\t\tbuf.ReadFrom(info.Metadata)\n\t\t\t\treturn info.Name == test.backup.Name &&\n\t\t\t\t\t(!(strings.Contains(buf.String(), `\"phase\": \"Completed\"`) ||\n\t\t\t\t\t\tstrings.Contains(buf.String(), `\"phase\": \"Failed\"`) ||\n\t\t\t\t\t\tstrings.Contains(buf.String(), `\"phase\": \"PartiallyFailed\"`)) ||\n\t\t\t\t\t\tstrings.Contains(buf.String(), `\"completionTimestamp\": \"2006-01-02T22:04:05Z\"`))\n\t\t\t}\n\t\t\tbackupStore.On(\"PutBackup\", mock.MatchedBy(hasNameAndCompletionTimestampIfCompleted)).Return(nil)\n\n\t\t\t// add the test's backup to the informer/lister store\n\t\t\trequire.NotNil(t, test.backup)\n\n\t\t\trequire.NoError(t, c.kbClient.Create(t.Context(), test.backup))\n\n\t\t\t// add the default backup storage location to the clientset and the informer/lister store\n\t\t\trequire.NoError(t, fakeClient.Create(t.Context(), defaultBackupLocation))\n\n\t\t\t// Enable CSI feature flag for SnapshotDataMovement test.\n\t\t\tif test.enableCSI {\n\t\t\t\tfeatures.Enable(velerov1api.CSIFeatureFlag)\n\t\t\t}\n\n\t\t\tactualResult, err := c.Reconcile(ctx, ctrl.Request{NamespacedName: types.NamespacedName{Namespace: test.backup.Namespace, Name: test.backup.Name}})\n\t\t\tassert.Equal(t, ctrl.Result{}, actualResult)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Disable CSI feature to not impact other test cases.\n\t\t\tif test.enableCSI {\n\t\t\t\tfeatures.Disable(velerov1api.CSIFeatureFlag)\n\t\t\t}\n\n\t\t\tres := &velerov1api.Backup{}\n\t\t\terr = c.kbClient.Get(t.Context(), kbclient.ObjectKey{Namespace: test.backup.Namespace, Name: test.backup.Name}, res)\n\t\t\trequire.NoError(t, err)\n\t\t\tres.ResourceVersion = \"\"\n\t\t\tassert.Equal(t, test.expectedResult, res)\n\t\t\t// reset defaultBackupLocation resourceVersion\n\t\t\tdefaultBackupLocation.ObjectMeta.ResourceVersion = \"\"\n\t\t})\n\t}\n}\n\nfunc TestValidateAndGetSnapshotLocations(t *testing.T) {\n\tdefaultBSL := builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"bsl\").Phase(velerov1api.BackupStorageLocationPhaseAvailable).Result()\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\tlocations []*velerov1api.VolumeSnapshotLocation\n\t\tdefaultLocations map[string]string\n\t\tbsl velerov1api.BackupStorageLocation\n\t\texpectedVolumeSnapshotLocationNames []string // adding these in the expected order will allow to test with better msgs in case of a test failure\n\t\texpectedErrors string\n\t\texpectedSuccess bool\n\t}{\n\t\t{\n\t\t\tname: \"location name does not correspond to any existing location\",\n\t\t\tbackup: defaultBackup().Phase(velerov1api.BackupPhaseNew).VolumeSnapshotLocations(\"random-name\").Result(),\n\t\t\tlocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"aws-us-east-1\").Provider(\"aws\").Result(),\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"aws-us-west-1\").Provider(\"aws\").Result(),\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"some-name\").Provider(\"fake-provider\").Result(),\n\t\t\t},\n\t\t\texpectedErrors: \"a VolumeSnapshotLocation CRD for the location random-name with the name specified in the backup spec needs to be created before this snapshot can be executed. Error: volumesnapshotlocations.velero.io \\\"random-name\\\" not found\", expectedSuccess: false,\n\t\t\tbsl: *defaultBSL,\n\t\t},\n\t\t{\n\t\t\tname: \"duplicate locationName per provider: should filter out dups\",\n\t\t\tbackup: defaultBackup().Phase(velerov1api.BackupPhaseNew).VolumeSnapshotLocations(\"aws-us-west-1\", \"aws-us-west-1\").Result(),\n\t\t\tlocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"aws-us-east-1\").Provider(\"aws\").Result(),\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"aws-us-west-1\").Provider(\"aws\").Result(),\n\t\t\t},\n\t\t\texpectedVolumeSnapshotLocationNames: []string{\"aws-us-west-1\"},\n\t\t\texpectedSuccess: true,\n\t\t\tbsl: *defaultBSL,\n\t\t},\n\t\t{\n\t\t\tname: \"multiple non-dupe location names per provider should error\",\n\t\t\tbackup: defaultBackup().Phase(velerov1api.BackupPhaseNew).VolumeSnapshotLocations(\"aws-us-east-1\", \"aws-us-west-1\").Result(),\n\t\t\tlocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"aws-us-east-1\").Provider(\"aws\").Result(),\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"aws-us-west-1\").Provider(\"aws\").Result(),\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"some-name\").Provider(\"fake-provider\").Result(),\n\t\t\t},\n\t\t\texpectedErrors: \"more than one VolumeSnapshotLocation name specified for provider aws: aws-us-west-1; unexpected name was aws-us-east-1\",\n\t\t\texpectedSuccess: false,\n\t\t\tbsl: *defaultBSL,\n\t\t},\n\t\t{\n\t\t\tname: \"no location name for the provider exists, only one VSL for the provider: use it\",\n\t\t\tbackup: defaultBackup().Phase(velerov1api.BackupPhaseNew).Result(),\n\t\t\tlocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"aws-us-east-1\").Provider(\"aws\").Result(),\n\t\t\t},\n\t\t\texpectedVolumeSnapshotLocationNames: []string{\"aws-us-east-1\"},\n\t\t\texpectedSuccess: true,\n\t\t\tbsl: *defaultBSL,\n\t\t},\n\t\t{\n\t\t\tname: \"no location name for the provider exists, no default, more than one VSL for the provider: error\",\n\t\t\tbackup: defaultBackup().Phase(velerov1api.BackupPhaseNew).Result(),\n\t\t\tlocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"aws-us-east-1\").Provider(\"aws\").Result(),\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"aws-us-west-1\").Provider(\"aws\").Result(),\n\t\t\t},\n\t\t\texpectedErrors: \"provider aws has more than one possible volume snapshot location, and none were specified explicitly or as a default\",\n\t\t\tbsl: *defaultBSL,\n\t\t},\n\t\t{\n\t\t\tname: \"no location name for the provider exists, more than one VSL for the provider: the provider's default should be added\",\n\t\t\tbackup: defaultBackup().Phase(velerov1api.BackupPhaseNew).Result(),\n\t\t\tdefaultLocations: map[string]string{\"aws\": \"aws-us-east-1\"},\n\t\t\tlocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"aws-us-east-1\").Provider(\"aws\").Result(),\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"aws-us-west-1\").Provider(\"aws\").Result(),\n\t\t\t},\n\t\t\texpectedVolumeSnapshotLocationNames: []string{\"aws-us-east-1\"},\n\t\t\texpectedSuccess: true,\n\t\t\tbsl: *defaultBSL,\n\t\t},\n\t\t{\n\t\t\tname: \"no existing location name and no default location name given\",\n\t\t\tbackup: defaultBackup().Phase(velerov1api.BackupPhaseNew).Result(),\n\t\t\texpectedSuccess: true,\n\t\t\tbsl: *defaultBSL,\n\t\t},\n\t\t{\n\t\t\tname: \"multiple location names for a provider, default location name for another provider\",\n\t\t\tbackup: defaultBackup().Phase(velerov1api.BackupPhaseNew).VolumeSnapshotLocations(\"aws-us-west-1\", \"aws-us-west-1\").Result(),\n\t\t\tdefaultLocations: map[string]string{\"fake-provider\": \"some-name\"},\n\t\t\tlocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"aws-us-west-1\").Provider(\"aws\").Result(),\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"some-name\").Provider(\"fake-provider\").Result(),\n\t\t\t},\n\t\t\texpectedVolumeSnapshotLocationNames: []string{\"aws-us-west-1\", \"some-name\"},\n\t\t\texpectedSuccess: true,\n\t\t\tbsl: *defaultBSL,\n\t\t},\n\t\t{\n\t\t\tname: \"location name does not correspond to any existing location and snapshotvolume disabled; should return error\",\n\t\t\tbackup: defaultBackup().Phase(velerov1api.BackupPhaseNew).VolumeSnapshotLocations(\"random-name\").SnapshotVolumes(false).Result(),\n\t\t\tlocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"aws-us-east-1\").Provider(\"aws\").Result(),\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"aws-us-west-1\").Provider(\"aws\").Result(),\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"some-name\").Provider(\"fake-provider\").Result(),\n\t\t\t},\n\t\t\texpectedVolumeSnapshotLocationNames: nil,\n\t\t\texpectedErrors: \"a VolumeSnapshotLocation CRD for the location random-name with the name specified in the backup spec needs to be created before this snapshot can be executed. Error: volumesnapshotlocations.velero.io \\\"random-name\\\" not found\", expectedSuccess: false,\n\t\t\tbsl: *defaultBSL,\n\t\t},\n\t\t{\n\t\t\tname: \"duplicate locationName per provider and snapshotvolume disabled; should return only one BSL\",\n\t\t\tbackup: defaultBackup().Phase(velerov1api.BackupPhaseNew).VolumeSnapshotLocations(\"aws-us-west-1\", \"aws-us-west-1\").SnapshotVolumes(false).Result(),\n\t\t\tlocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"aws-us-east-1\").Provider(\"aws\").Result(),\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"aws-us-west-1\").Provider(\"aws\").Result(),\n\t\t\t},\n\t\t\texpectedVolumeSnapshotLocationNames: []string{\"aws-us-west-1\"},\n\t\t\texpectedSuccess: true,\n\t\t\tbsl: *defaultBSL,\n\t\t},\n\t\t{\n\t\t\tname: \"no location name for the provider exists, only one VSL created and snapshotvolume disabled; should return the VSL\",\n\t\t\tbackup: defaultBackup().Phase(velerov1api.BackupPhaseNew).SnapshotVolumes(false).Result(),\n\t\t\tlocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"aws-us-east-1\").Provider(\"aws\").Result(),\n\t\t\t},\n\t\t\texpectedVolumeSnapshotLocationNames: []string{\"aws-us-east-1\"},\n\t\t\texpectedSuccess: true,\n\t\t\tbsl: *defaultBSL,\n\t\t},\n\t\t{\n\t\t\tname: \"multiple location names for a provider, no default location and backup has no location defined, but snapshotvolume disabled, should return error\",\n\t\t\tbackup: defaultBackup().Phase(velerov1api.BackupPhaseNew).SnapshotVolumes(false).Result(),\n\t\t\tlocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"aws-us-west-1\").Provider(\"aws\").Result(),\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"aws-us-east-1\").Provider(\"aws\").Result(),\n\t\t\t},\n\t\t\texpectedVolumeSnapshotLocationNames: nil,\n\t\t\texpectedErrors: \"provider aws has more than one possible volume snapshot location, and none were specified explicitly or as a default\",\n\t\t\tbsl: *defaultBSL,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tformatFlag := logging.FormatText\n\t\t\tlogger := logging.DefaultLogger(logrus.DebugLevel, formatFlag)\n\n\t\t\tc := &backupReconciler{\n\t\t\t\tlogger: logger,\n\t\t\t\tdefaultSnapshotLocations: test.defaultLocations,\n\t\t\t\tkbClient: velerotest.NewFakeControllerRuntimeClient(t),\n\t\t\t}\n\n\t\t\t// set up a Backup object to represent what we expect to be passed to backupper.Backup()\n\t\t\tbackup := test.backup.DeepCopy()\n\t\t\tbackup.Spec.VolumeSnapshotLocations = test.backup.Spec.VolumeSnapshotLocations\n\t\t\tfor _, location := range test.locations {\n\t\t\t\trequire.NoError(t, c.kbClient.Create(t.Context(), location))\n\t\t\t}\n\n\t\t\tproviderLocations, errs := c.validateAndGetSnapshotLocations(backup)\n\t\t\tif test.expectedSuccess {\n\t\t\t\tfor _, err := range errs {\n\t\t\t\t\trequire.NoError(t, errors.New(err), \"validateAndGetSnapshotLocations unexpected error: %v\", err)\n\t\t\t\t}\n\n\t\t\t\tvar locations []string\n\t\t\t\tfor _, loc := range providerLocations {\n\t\t\t\t\tlocations = append(locations, loc.Name)\n\t\t\t\t}\n\n\t\t\t\tsort.Strings(test.expectedVolumeSnapshotLocationNames)\n\t\t\t\tsort.Strings(locations)\n\t\t\t\trequire.Equal(t, test.expectedVolumeSnapshotLocationNames, locations)\n\t\t\t} else {\n\t\t\t\trequire.NotEmpty(t, errs, \"validateAndGetSnapshotLocations expected error\")\n\t\t\t\trequire.Contains(t, errs, test.expectedErrors)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// Test_getLastSuccessBySchedule verifies that the getLastSuccessBySchedule helper function correctly returns\n// the completion timestamp of the most recent completed backup for each schedule, including an entry for ad-hoc\n// or non-scheduled backups.\nfunc Test_getLastSuccessBySchedule(t *testing.T) {\n\tbuildBackup := func(phase velerov1api.BackupPhase, completion time.Time, schedule string) velerov1api.Backup {\n\t\tb := builder.ForBackup(\"\", \"\").\n\t\t\tObjectMeta(builder.WithLabels(velerov1api.ScheduleNameLabel, schedule)).\n\t\t\tPhase(phase)\n\n\t\tif !completion.IsZero() {\n\t\t\tb.CompletionTimestamp(completion)\n\t\t}\n\n\t\treturn *b.Result()\n\t}\n\n\t// create a static \"base time\" that can be used to easily construct completion timestamps\n\t// by using the .Add(...) method.\n\tbaseTime, err := time.Parse(time.RFC1123, time.RFC1123)\n\trequire.NoError(t, err)\n\n\ttests := []struct {\n\t\tname string\n\t\tbackups []velerov1api.Backup\n\t\twant map[string]time.Time\n\t}{\n\t\t{\n\t\t\tname: \"when backups is nil, an empty map is returned\",\n\t\t\tbackups: nil,\n\t\t\twant: map[string]time.Time{},\n\t\t},\n\t\t{\n\t\t\tname: \"when backups is empty, an empty map is returned\",\n\t\t\tbackups: []velerov1api.Backup{},\n\t\t\twant: map[string]time.Time{},\n\t\t},\n\t\t{\n\t\t\tname: \"when multiple completed backups for a schedule exist, the latest one is returned\",\n\t\t\tbackups: []velerov1api.Backup{\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseCompleted, baseTime, \"schedule-1\"),\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseCompleted, baseTime.Add(time.Second), \"schedule-1\"),\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseCompleted, baseTime.Add(-time.Second), \"schedule-1\"),\n\t\t\t},\n\t\t\twant: map[string]time.Time{\n\t\t\t\t\"schedule-1\": baseTime.Add(time.Second),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when the most recent backup for a schedule is Failed, the timestamp of the most recent Completed one is returned\",\n\t\t\tbackups: []velerov1api.Backup{\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseCompleted, baseTime, \"schedule-1\"),\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseFailed, baseTime.Add(time.Second), \"schedule-1\"),\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseCompleted, baseTime.Add(-time.Second), \"schedule-1\"),\n\t\t\t},\n\t\t\twant: map[string]time.Time{\n\t\t\t\t\"schedule-1\": baseTime,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when there are no Completed backups for a schedule, it's not returned\",\n\t\t\tbackups: []velerov1api.Backup{\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseInProgress, baseTime, \"schedule-1\"),\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseFailed, baseTime.Add(time.Second), \"schedule-1\"),\n\t\t\t\tbuildBackup(velerov1api.BackupPhasePartiallyFailed, baseTime.Add(-time.Second), \"schedule-1\"),\n\t\t\t},\n\t\t\twant: map[string]time.Time{},\n\t\t},\n\t\t{\n\t\t\tname: \"when backups exist without a schedule, the most recent Completed one is returned\",\n\t\t\tbackups: []velerov1api.Backup{\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseCompleted, baseTime, \"\"),\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseFailed, baseTime.Add(time.Second), \"\"),\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseCompleted, baseTime.Add(-time.Second), \"\"),\n\t\t\t},\n\t\t\twant: map[string]time.Time{\n\t\t\t\t\"\": baseTime,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when backups exist for multiple schedules, the most recent Completed timestamp for each schedule is returned\",\n\t\t\tbackups: []velerov1api.Backup{\n\t\t\t\t// ad-hoc backups (no schedule)\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseCompleted, baseTime.Add(30*time.Minute), \"\"),\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseFailed, baseTime.Add(time.Hour), \"\"),\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseCompleted, baseTime.Add(-time.Second), \"\"),\n\n\t\t\t\t// schedule-1\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseCompleted, baseTime, \"schedule-1\"),\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseFailed, baseTime.Add(time.Second), \"schedule-1\"),\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseCompleted, baseTime.Add(-time.Second), \"schedule-1\"),\n\n\t\t\t\t// schedule-2\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseCompleted, baseTime.Add(24*time.Hour), \"schedule-2\"),\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseCompleted, baseTime.Add(48*time.Hour), \"schedule-2\"),\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseCompleted, baseTime.Add(72*time.Hour), \"schedule-2\"),\n\n\t\t\t\t// schedule-3\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseNew, baseTime, \"schedule-3\"),\n\t\t\t\tbuildBackup(velerov1api.BackupPhaseInProgress, baseTime.Add(time.Minute), \"schedule-3\"),\n\t\t\t\tbuildBackup(velerov1api.BackupPhasePartiallyFailed, baseTime.Add(2*time.Minute), \"schedule-3\"),\n\t\t\t},\n\t\t\twant: map[string]time.Time{\n\t\t\t\t\"\": baseTime.Add(30 * time.Minute),\n\t\t\t\t\"schedule-1\": baseTime,\n\t\t\t\t\"schedule-2\": baseTime.Add(72 * time.Hour),\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tassert.Equal(t, tc.want, getLastSuccessBySchedule(tc.backups))\n\t\t})\n\t}\n}\n\n// Unit tests to make sure that the backup's status is updated correctly during reconcile.\n// To clear up confusion whether status can be updated with Patch alone without status writer and not kbClient.Status().Patch()\nfunc TestPatchResourceWorksWithStatus(t *testing.T) {\n\ttype args struct {\n\t\toriginal *velerov1api.Backup\n\t\tupdated *velerov1api.Backup\n\t}\n\ttests := []struct {\n\t\tname string\n\t\targs args\n\t\twantErr bool\n\t}{\n\t\t{\n\t\t\tname: \"patch backup status\",\n\t\t\targs: args{\n\t\t\t\toriginal: defaultBackup().SnapshotMoveData(false).Result(),\n\t\t\t\tupdated: defaultBackup().SnapshotMoveData(false).WithStatus(velerov1api.BackupStatus{\n\t\t\t\t\tCSIVolumeSnapshotsCompleted: 1,\n\t\t\t\t}).Result(),\n\t\t\t},\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tscheme := runtime.NewScheme()\n\t\t\terror := velerov1api.AddToScheme(scheme)\n\t\t\tif error != nil {\n\t\t\t\tt.Errorf(\"PatchResource() error = %v\", error)\n\t\t\t}\n\t\t\tfakeClient := fakeClient.NewClientBuilder().WithScheme(scheme).WithObjects(tt.args.original).Build()\n\t\t\tfromCluster := &velerov1api.Backup{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: tt.args.original.Name,\n\t\t\t\t\tNamespace: tt.args.original.Namespace,\n\t\t\t\t},\n\t\t\t}\n\t\t\t// check original exists\n\t\t\tif err := fakeClient.Get(t.Context(), kbclient.ObjectKeyFromObject(tt.args.updated), fromCluster); err != nil {\n\t\t\t\tt.Errorf(\"PatchResource() error = %v\", err)\n\t\t\t}\n\t\t\t// ignore resourceVersion\n\t\t\ttt.args.updated.ResourceVersion = fromCluster.ResourceVersion\n\t\t\ttt.args.original.ResourceVersion = fromCluster.ResourceVersion\n\t\t\tif err := kubeutil.PatchResource(tt.args.original, tt.args.updated, fakeClient); (err != nil) != tt.wantErr {\n\t\t\t\tt.Errorf(\"PatchResource() error = %v, wantErr %v\", err, tt.wantErr)\n\t\t\t}\n\t\t\t// check updated exists\n\t\t\tif err := fakeClient.Get(t.Context(), kbclient.ObjectKeyFromObject(tt.args.updated), fromCluster); err != nil {\n\t\t\t\tt.Errorf(\"PatchResource() error = %v\", err)\n\t\t\t}\n\n\t\t\t// check fromCluster is equal to updated\n\t\t\tif !reflect.DeepEqual(fromCluster, tt.args.updated) {\n\t\t\t\tt.Error(cmp.Diff(fromCluster, tt.args.updated))\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/controller/backup_deletion_controller.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"time\"\n\n\tjsonpatch \"github.com/evanphx/json-patch/v5\"\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\tkubeerrs \"k8s.io/apimachinery/pkg/util/errors\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\t\"github.com/vmware-tanzu/velero/internal/delete\"\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/discovery\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\t\"github.com/vmware-tanzu/velero/pkg/persistence\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\tvsv1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/volumesnapshotter/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/podvolume\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\trepomanager \"github.com/vmware-tanzu/velero/pkg/repository/manager\"\n\trepotypes \"github.com/vmware-tanzu/velero/pkg/repository/types\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/csi\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\tveleroutil \"github.com/vmware-tanzu/velero/pkg/util/velero\"\n)\n\nconst (\n\tdeleteBackupRequestMaxAge = 24 * time.Hour\n)\n\ntype backupDeletionReconciler struct {\n\tclient.Client\n\tlogger logrus.FieldLogger\n\tbackupTracker BackupTracker\n\trepoMgr repomanager.Manager\n\tmetrics *metrics.ServerMetrics\n\tclock clock.Clock\n\tdiscoveryHelper discovery.Helper\n\tnewPluginManager func(logrus.FieldLogger) clientmgmt.Manager\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter\n\tcredentialStore credentials.FileStore\n\trepoEnsurer *repository.Ensurer\n}\n\n// NewBackupDeletionReconciler creates a new backup deletion reconciler.\nfunc NewBackupDeletionReconciler(\n\tlogger logrus.FieldLogger,\n\tclient client.Client,\n\tbackupTracker BackupTracker,\n\trepoMgr repomanager.Manager,\n\tmetrics *metrics.ServerMetrics,\n\thelper discovery.Helper,\n\tnewPluginManager func(logrus.FieldLogger) clientmgmt.Manager,\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter,\n\tcredentialStore credentials.FileStore,\n\trepoEnsurer *repository.Ensurer,\n) *backupDeletionReconciler {\n\treturn &backupDeletionReconciler{\n\t\tClient: client,\n\t\tlogger: logger,\n\t\tbackupTracker: backupTracker,\n\t\trepoMgr: repoMgr,\n\t\tmetrics: metrics,\n\t\tclock: clock.RealClock{},\n\t\tdiscoveryHelper: helper,\n\t\tnewPluginManager: newPluginManager,\n\t\tbackupStoreGetter: backupStoreGetter,\n\t\tcredentialStore: credentialStore,\n\t\trepoEnsurer: repoEnsurer,\n\t}\n}\n\nfunc (r *backupDeletionReconciler) SetupWithManager(mgr ctrl.Manager) error {\n\t// Make sure the expired requests can be deleted eventually\n\ts := kube.NewPeriodicalEnqueueSource(r.logger.WithField(\"controller\", constant.ControllerBackupDeletion), mgr.GetClient(), &velerov1api.DeleteBackupRequestList{}, time.Hour, kube.PeriodicalEnqueueSourceOption{})\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\tFor(&velerov1api.DeleteBackupRequest{}).\n\t\tWatchesRawSource(s).\n\t\tComplete(r)\n}\n\n// +kubebuilder:rbac:groups=velero.io,resources=deletebackuprequests,verbs=get;list;watch;create;update;patch;delete\n// +kubebuilder:rbac:groups=velero.io,resources=deletebackuprequests/status,verbs=get;update;patch\n// +kubebuilder:rbac:groups=velero.io,resources=backups,verbs=delete\n\nfunc (r *backupDeletionReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\tlog := r.logger.WithFields(logrus.Fields{\n\t\t\"controller\": constant.ControllerBackupDeletion,\n\t\t\"deletebackuprequest\": req.String(),\n\t})\n\tlog.Debug(\"Getting deletebackuprequest\")\n\tdbr := &velerov1api.DeleteBackupRequest{}\n\tif err := r.Get(ctx, req.NamespacedName, dbr); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.Debug(\"Unable to find the deletebackuprequest\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\tlog.WithError(err).Error(\"Error getting deletebackuprequest\")\n\t\treturn ctrl.Result{}, err\n\t}\n\n\t// Since we use the reconciler along with the PeriodicalEnqueueSource, there may be reconciliation triggered by\n\t// stale requests.\n\tif dbr.Status.Phase == velerov1api.DeleteBackupRequestPhaseProcessed ||\n\t\tdbr.Status.Phase == velerov1api.DeleteBackupRequestPhaseInProgress {\n\t\tage := r.clock.Now().Sub(dbr.CreationTimestamp.Time)\n\t\tif age >= deleteBackupRequestMaxAge { // delete the expired request\n\t\t\tlog.Debugf(\"The request is expired, status: %s, deleting it.\", dbr.Status.Phase)\n\t\t\tif err := r.Delete(ctx, dbr); err != nil {\n\t\t\t\tlog.WithError(err).Error(\"Error deleting DeleteBackupRequest\")\n\t\t\t}\n\t\t} else {\n\t\t\tlog.Infof(\"The request has status '%s', skip.\", dbr.Status.Phase)\n\t\t}\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\t// Make sure we have the backup name\n\tif dbr.Spec.BackupName == \"\" {\n\t\terr := r.patchDeleteBackupRequestWithError(ctx, dbr, errors.New(\"spec.backupName is required\"))\n\t\treturn ctrl.Result{}, err\n\t}\n\n\tlog = log.WithField(\"backup\", dbr.Spec.BackupName)\n\n\t// Remove any existing deletion requests for this backup so we only have\n\t// one at a time\n\tif errs := r.deleteExistingDeletionRequests(ctx, dbr, log); errs != nil {\n\t\treturn ctrl.Result{}, kubeerrs.NewAggregate(errs)\n\t}\n\n\t// Don't allow deleting an in-progress backup\n\tif r.backupTracker.Contains(dbr.Namespace, dbr.Spec.BackupName) {\n\t\terr := r.patchDeleteBackupRequestWithError(ctx, dbr, errors.New(\"backup is still in progress\"))\n\t\treturn ctrl.Result{}, err\n\t}\n\n\t// Get the backup we're trying to delete\n\tbackup := &velerov1api.Backup{}\n\tif err := r.Get(ctx, types.NamespacedName{\n\t\tNamespace: dbr.Namespace,\n\t\tName: dbr.Spec.BackupName,\n\t}, backup); apierrors.IsNotFound(err) {\n\t\t// Couldn't find backup - update status to Processed and record the not-found error\n\t\terr = r.patchDeleteBackupRequestWithError(ctx, dbr, errors.New(\"backup not found\"))\n\t\treturn ctrl.Result{}, err\n\t} else if err != nil {\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error getting backup\")\n\t}\n\n\t// Don't allow deleting backups in read-only storage locations\n\tlocation := &velerov1api.BackupStorageLocation{}\n\tif err := r.Get(context.Background(), client.ObjectKey{\n\t\tNamespace: backup.Namespace,\n\t\tName: backup.Spec.StorageLocation,\n\t}, location); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\terr := r.patchDeleteBackupRequestWithError(ctx, dbr, fmt.Errorf(\"backup storage location %s not found\", backup.Spec.StorageLocation))\n\t\t\treturn ctrl.Result{}, err\n\t\t}\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error getting backup storage location\")\n\t}\n\n\tif location.Spec.AccessMode == velerov1api.BackupStorageLocationAccessModeReadOnly {\n\t\terr := r.patchDeleteBackupRequestWithError(ctx, dbr, fmt.Errorf(\"cannot delete backup because backup storage location %s is currently in read-only mode\", location.Name))\n\t\treturn ctrl.Result{}, err\n\t}\n\n\tif !veleroutil.BSLIsAvailable(*location) {\n\t\terr := r.patchDeleteBackupRequestWithError(ctx, dbr, fmt.Errorf(\"cannot delete backup because backup storage location %s is currently in Unavailable state\", location.Name))\n\t\treturn ctrl.Result{}, err\n\t}\n\n\t// if the request object has no labels defined, initialize an empty map since\n\t// we will be updating labels\n\tif dbr.Labels == nil {\n\t\tdbr.Labels = map[string]string{}\n\t}\n\t// Update status to InProgress and set backup-name and backup-uid label if needed\n\tdbr, err := r.patchDeleteBackupRequest(ctx, dbr, func(r *velerov1api.DeleteBackupRequest) {\n\t\tr.Status.Phase = velerov1api.DeleteBackupRequestPhaseInProgress\n\n\t\tif r.Labels[velerov1api.BackupNameLabel] == \"\" {\n\t\t\tr.Labels[velerov1api.BackupNameLabel] = label.GetValidName(dbr.Spec.BackupName)\n\t\t}\n\n\t\tif r.Labels[velerov1api.BackupUIDLabel] == \"\" {\n\t\t\tr.Labels[velerov1api.BackupUIDLabel] = string(backup.UID)\n\t\t}\n\t})\n\tif err != nil {\n\t\treturn ctrl.Result{}, err\n\t}\n\n\t// Set backup status to Deleting\n\tbackup, err = r.patchBackup(ctx, backup, func(b *velerov1api.Backup) {\n\t\tb.Status.Phase = velerov1api.BackupPhaseDeleting\n\t})\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"Error setting backup phase to deleting\")\n\t\terr2 := r.patchDeleteBackupRequestWithError(ctx, dbr, errors.Wrap(err, \"error setting backup phase to deleting\"))\n\t\treturn ctrl.Result{}, err2\n\t}\n\n\tbackupScheduleName := backup.GetLabels()[velerov1api.ScheduleNameLabel]\n\tr.metrics.RegisterBackupDeletionAttempt(backupScheduleName)\n\n\tpluginManager := r.newPluginManager(log)\n\tdefer pluginManager.CleanupClients()\n\n\tbackupStore, err := r.backupStoreGetter.Get(location, pluginManager, log)\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"Error getting the backup store\")\n\t\terr2 := r.patchDeleteBackupRequestWithError(ctx, dbr, errors.Wrap(err, \"error getting the backup store\"))\n\t\treturn ctrl.Result{}, err2\n\t}\n\n\tactions, err := pluginManager.GetDeleteItemActions()\n\tlog.Debugf(\"%d actions before invoking actions\", len(actions))\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"Error getting delete item actions\")\n\t\terr2 := r.patchDeleteBackupRequestWithError(ctx, dbr, errors.New(\"error getting delete item actions\"))\n\t\treturn ctrl.Result{}, err2\n\t}\n\t// don't defer CleanupClients here, since it was already called above.\n\n\tvar errs []string\n\n\tif len(actions) > 0 {\n\t\t// Download the tarball\n\t\tbackupFile, err := downloadToTempFile(backup.Name, backupStore, log)\n\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Errorf(\"Unable to download tarball for backup %s, skipping associated DeleteItemAction plugins\", backup.Name)\n\t\t\tlog.Info(\"Cleaning up CSI volumesnapshots\")\n\t\t\tr.deleteCSIVolumeSnapshotsIfAny(ctx, backup, log)\n\t\t} else {\n\t\t\tdefer closeAndRemoveFile(backupFile, r.logger)\n\t\t\tdeleteCtx := &delete.Context{\n\t\t\t\tBackup: backup,\n\t\t\t\tBackupReader: backupFile,\n\t\t\t\tActions: actions,\n\t\t\t\tLog: r.logger,\n\t\t\t\tDiscoveryHelper: r.discoveryHelper,\n\t\t\t\tFilesystem: filesystem.NewFileSystem(),\n\t\t\t}\n\n\t\t\t// Optimization: wrap in a gofunc? Would be useful for large backups with lots of objects.\n\t\t\t// but what do we do with the error returned? We can't just swallow it as that may lead to dangling resources.\n\t\t\terr = delete.InvokeDeleteActions(deleteCtx)\n\t\t\tif err != nil {\n\t\t\t\tlog.WithError(err).Error(\"Error invoking delete item actions\")\n\t\t\t\terr2 := r.patchDeleteBackupRequestWithError(ctx, dbr, errors.New(\"error invoking delete item actions\"))\n\t\t\t\treturn ctrl.Result{}, err2\n\t\t\t}\n\t\t}\n\t}\n\n\tif backupStore != nil {\n\t\tlog.Info(\"Removing PV snapshots\")\n\n\t\tif snapshots, err := backupStore.GetBackupVolumeSnapshots(backup.Name); err != nil {\n\t\t\terrs = append(errs, errors.Wrap(err, \"error getting backup's volume snapshots\").Error())\n\t\t} else {\n\t\t\tvolumeSnapshotters := make(map[string]vsv1.VolumeSnapshotter)\n\n\t\t\tfor _, snapshot := range snapshots {\n\t\t\t\tlog.WithField(\"providerSnapshotID\", snapshot.Status.ProviderSnapshotID).Info(\"Removing snapshot associated with backup\")\n\n\t\t\t\tvolumeSnapshotter, ok := volumeSnapshotters[snapshot.Spec.Location]\n\t\t\t\tif !ok {\n\t\t\t\t\tif volumeSnapshotter, err = r.volumeSnapshottersForVSL(ctx, backup.Namespace, snapshot.Spec.Location, pluginManager); err != nil {\n\t\t\t\t\t\terrs = append(errs, err.Error())\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\t\t\t\t\tvolumeSnapshotters[snapshot.Spec.Location] = volumeSnapshotter\n\t\t\t\t}\n\n\t\t\t\tif err := volumeSnapshotter.DeleteSnapshot(snapshot.Status.ProviderSnapshotID); err != nil {\n\t\t\t\t\terrs = append(errs, errors.Wrapf(err, \"error deleting snapshot %s\", snapshot.Status.ProviderSnapshotID).Error())\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\tlog.Info(\"Removing pod volume snapshots\")\n\tif deleteErrs := r.deletePodVolumeSnapshots(ctx, backup); len(deleteErrs) > 0 {\n\t\tfor _, err := range deleteErrs {\n\t\t\terrs = append(errs, err.Error())\n\t\t}\n\t}\n\n\tif boolptr.IsSetToTrue(backup.Spec.SnapshotMoveData) {\n\t\tlog.Info(\"Removing snapshot data by data mover\")\n\t\tif deleteErrs := r.deleteMovedSnapshots(ctx, backup); len(deleteErrs) > 0 {\n\t\t\tfor _, err := range deleteErrs {\n\t\t\t\terrs = append(errs, err.Error())\n\t\t\t}\n\t\t}\n\t\tduList := &velerov2alpha1.DataUploadList{}\n\t\tlog.Info(\"Removing local datauploads\")\n\t\tif err := r.Client.List(ctx, duList, &client.ListOptions{\n\t\t\tNamespace: backup.Namespace,\n\t\t\tLabelSelector: labels.SelectorFromSet(map[string]string{\n\t\t\t\tvelerov1api.BackupNameLabel: label.GetValidName(backup.Name),\n\t\t\t}),\n\t\t}); err != nil {\n\t\t\tlog.WithError(err).Error(\"Error listing datauploads\")\n\t\t\terrs = append(errs, err.Error())\n\t\t} else {\n\t\t\tfor i := range duList.Items {\n\t\t\t\tdu := duList.Items[i]\n\t\t\t\tif err := r.Delete(ctx, &du); err != nil {\n\t\t\t\t\terrs = append(errs, err.Error())\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\tif backupStore != nil {\n\t\tlog.Info(\"Removing backup from backup storage\")\n\t\tif err := backupStore.DeleteBackup(backup.Name); err != nil {\n\t\t\terrs = append(errs, err.Error())\n\t\t}\n\t}\n\n\tlog.Info(\"Removing restores\")\n\trestoreList := &velerov1api.RestoreList{}\n\tselector := labels.Everything()\n\tif err := r.List(ctx, restoreList, &client.ListOptions{\n\t\tNamespace: backup.Namespace,\n\t\tLabelSelector: selector,\n\t}); err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Error(\"Error listing restore API objects\")\n\t} else {\n\t\t// Restore files in object storage will be handled by restore finalizer, so we simply need to initiate a delete request on restores here.\n\t\tfor i, restore := range restoreList.Items {\n\t\t\tif restore.Spec.BackupName != backup.Name {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\trestoreLog := log.WithField(\"restore\", kube.NamespaceAndName(&restoreList.Items[i]))\n\n\t\t\trestoreLog.Info(\"Deleting restore referencing backup\")\n\t\t\tif err := r.Delete(ctx, &restoreList.Items[i]); err != nil {\n\t\t\t\terrs = append(errs, errors.Wrapf(err, \"error deleting restore %s\", kube.NamespaceAndName(&restoreList.Items[i])).Error())\n\t\t\t}\n\t\t}\n\n\t\t// Wait for the deletion of restores within certain amount of time.\n\t\t// Notice that there could be potential errors during the finalization process, which may result in the failure to delete the restore.\n\t\t// Therefore, it is advisable to set a timeout period for waiting.\n\t\terr := wait.PollUntilContextTimeout(ctx, time.Second, time.Minute, true, func(ctx context.Context) (bool, error) {\n\t\t\trestoreList := &velerov1api.RestoreList{}\n\t\t\tif err := r.List(ctx, restoreList, &client.ListOptions{Namespace: backup.Namespace, LabelSelector: selector}); err != nil {\n\t\t\t\treturn false, err\n\t\t\t}\n\t\t\tcnt := 0\n\t\t\tfor _, restore := range restoreList.Items {\n\t\t\t\tif restore.Spec.BackupName != backup.Name {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tcnt++\n\t\t\t}\n\n\t\t\tif cnt > 0 {\n\t\t\t\treturn false, nil\n\t\t\t} else {\n\t\t\t\treturn true, nil\n\t\t\t}\n\t\t})\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Error(\"Error polling for deletion of restores\")\n\t\t\terrs = append(errs, errors.Wrapf(err, \"error deleting restore %s\", err).Error())\n\t\t}\n\t}\n\n\tif len(errs) == 0 {\n\t\t// Only try to delete the backup object from kube if everything preceding went smoothly\n\t\tif err := r.Delete(ctx, backup); err != nil {\n\t\t\terrs = append(errs, errors.Wrapf(err, \"error deleting backup %s\", kube.NamespaceAndName(backup)).Error())\n\t\t}\n\t}\n\n\tif len(errs) == 0 {\n\t\tr.metrics.RegisterBackupDeletionSuccess(backupScheduleName)\n\t} else {\n\t\tr.metrics.RegisterBackupDeletionFailed(backupScheduleName)\n\t}\n\n\t// Update status to processed and record errors\n\tif _, err := r.patchDeleteBackupRequest(ctx, dbr, func(r *velerov1api.DeleteBackupRequest) {\n\t\tr.Status.Phase = velerov1api.DeleteBackupRequestPhaseProcessed\n\t\tr.Status.Errors = errs\n\t}); err != nil {\n\t\treturn ctrl.Result{}, err\n\t}\n\t// Everything deleted correctly, so we can delete all DeleteBackupRequests for this backup\n\tif len(errs) == 0 {\n\t\tlabelSelector, err := labels.Parse(fmt.Sprintf(\"%s=%s,%s=%s\", velerov1api.BackupNameLabel, label.GetValidName(backup.Name), velerov1api.BackupUIDLabel, backup.UID))\n\t\tif err != nil {\n\t\t\t// Should not be here\n\t\t\tr.logger.WithError(err).WithField(\"backup\", kube.NamespaceAndName(backup)).Error(\"error creating label selector for the backup for deleting DeleteBackupRequests\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\talldbr := &velerov1api.DeleteBackupRequest{}\n\t\terr = r.DeleteAllOf(ctx, alldbr, client.MatchingLabelsSelector{\n\t\t\tSelector: labelSelector,\n\t\t}, client.InNamespace(dbr.Namespace))\n\t\tif err != nil {\n\t\t\t// If this errors, all we can do is log it.\n\t\t\tr.logger.WithError(err).WithField(\"backup\", kube.NamespaceAndName(backup)).Error(\"error deleting all associated DeleteBackupRequests after successfully deleting the backup\")\n\t\t}\n\t}\n\tlog.Infof(\"Reconciliation done\")\n\n\treturn ctrl.Result{}, nil\n}\n\nfunc (r *backupDeletionReconciler) volumeSnapshottersForVSL(\n\tctx context.Context,\n\tnamespace, vslName string,\n\tpluginManager clientmgmt.Manager,\n) (vsv1.VolumeSnapshotter, error) {\n\tvsl := &velerov1api.VolumeSnapshotLocation{}\n\tif err := r.Client.Get(ctx, types.NamespacedName{\n\t\tNamespace: namespace,\n\t\tName: vslName,\n\t}, vsl); err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error getting volume snapshot location %s\", vslName)\n\t}\n\n\t// add credential to config\n\terr := volume.UpdateVolumeSnapshotLocationWithCredentialConfig(vsl, r.credentialStore)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tvolumeSnapshotter, err := pluginManager.GetVolumeSnapshotter(vsl.Spec.Provider)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error getting volume snapshotter for provider %s\", vsl.Spec.Provider)\n\t}\n\n\tif err = volumeSnapshotter.Init(vsl.Spec.Config); err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error initializing volume snapshotter for volume snapshot location %s\", vslName)\n\t}\n\n\treturn volumeSnapshotter, nil\n}\n\nfunc (r *backupDeletionReconciler) deleteExistingDeletionRequests(ctx context.Context, req *velerov1api.DeleteBackupRequest, log logrus.FieldLogger) []error {\n\tlog.Info(\"Removing existing deletion requests for backup\")\n\tdbrList := &velerov1api.DeleteBackupRequestList{}\n\tselector := label.NewSelectorForBackup(req.Spec.BackupName)\n\tif err := r.List(ctx, dbrList, &client.ListOptions{\n\t\tNamespace: req.Namespace,\n\t\tLabelSelector: selector,\n\t}); err != nil {\n\t\treturn []error{errors.Wrap(err, \"error listing existing DeleteBackupRequests for backup\")}\n\t}\n\tvar errs []error\n\tfor i, dbr := range dbrList.Items {\n\t\tif dbr.Name == req.Name {\n\t\t\tcontinue\n\t\t}\n\t\tif err := r.Delete(ctx, &dbrList.Items[i]); err != nil {\n\t\t\terrs = append(errs, errors.WithStack(err))\n\t\t} else {\n\t\t\tlog.Infof(\"deletion request '%s' removed.\", dbr.Name)\n\t\t}\n\t}\n\treturn errs\n}\n\n// deleteCSIVolumeSnapshotsIfAny clean up the CSI snapshots created by the backup, this should be called when the backup is failed\n// when it's running, e.g. due to velero pod restart, and the backup.tar is failed to be downloaded from storage.\nfunc (r *backupDeletionReconciler) deleteCSIVolumeSnapshotsIfAny(ctx context.Context, backup *velerov1api.Backup, log logrus.FieldLogger) {\n\tvsList := snapshotv1api.VolumeSnapshotList{}\n\tif err := r.Client.List(ctx, &vsList, &client.ListOptions{\n\t\tLabelSelector: labels.SelectorFromSet(map[string]string{\n\t\t\tvelerov1api.BackupNameLabel: label.GetValidName(backup.Name),\n\t\t}),\n\t}); err != nil {\n\t\tlog.WithError(err).Warnf(\"Could not list volume snapshots, abort\")\n\t\treturn\n\t}\n\tfor _, item := range vsList.Items {\n\t\tvs := item\n\t\tcsi.CleanupVolumeSnapshot(&vs, r.Client, log)\n\t}\n}\n\nfunc (r *backupDeletionReconciler) deletePodVolumeSnapshots(ctx context.Context, backup *velerov1api.Backup) []error {\n\tif r.repoMgr == nil {\n\t\treturn nil\n\t}\n\n\tdirectSnapshots, err := getSnapshotsInBackup(ctx, backup, r.Client)\n\tif err != nil {\n\t\treturn []error{err}\n\t}\n\n\treturn batchDeleteSnapshots(ctx, r.repoEnsurer, r.repoMgr, directSnapshots, backup, r.logger)\n}\n\nvar batchDeleteSnapshotFunc = batchDeleteSnapshots\n\nfunc (r *backupDeletionReconciler) deleteMovedSnapshots(ctx context.Context, backup *velerov1api.Backup) []error {\n\tif r.repoMgr == nil {\n\t\treturn nil\n\t}\n\tlist := &corev1api.ConfigMapList{}\n\tif err := r.Client.List(ctx, list, &client.ListOptions{\n\t\tNamespace: backup.Namespace,\n\t\tLabelSelector: labels.SelectorFromSet(\n\t\t\tmap[string]string{\n\t\t\t\tvelerov1api.BackupNameLabel: label.GetValidName(backup.Name),\n\t\t\t\tvelerov1api.DataUploadSnapshotInfoLabel: \"true\",\n\t\t\t}),\n\t}); err != nil {\n\t\treturn []error{errors.Wrapf(err, \"failed to retrieve config for snapshot info\")}\n\t}\n\tvar errs []error\n\tdirectSnapshots := map[string][]repotypes.SnapshotIdentifier{}\n\tfor i := range list.Items {\n\t\tcm := list.Items[i]\n\t\tif len(cm.Data) == 0 {\n\t\t\terrs = append(errs, errors.New(\"no snapshot info in config\"))\n\t\t\tcontinue\n\t\t}\n\n\t\tb, err := json.Marshal(cm.Data)\n\t\tif err != nil {\n\t\t\terrs = append(errs, errors.Wrapf(err, \"fail to marshal the snapshot info into JSON\"))\n\t\t\tcontinue\n\t\t}\n\n\t\tsnapshot := repotypes.SnapshotIdentifier{}\n\t\tif err := json.Unmarshal(b, &snapshot); err != nil {\n\t\t\terrs = append(errs, errors.Wrapf(err, \"failed to unmarshal snapshot info\"))\n\t\t\tcontinue\n\t\t}\n\n\t\tif snapshot.SnapshotID == \"\" || snapshot.VolumeNamespace == \"\" || snapshot.RepositoryType == \"\" {\n\t\t\terrs = append(errs, errors.Errorf(\"invalid snapshot, ID %s, namespace %s, repository %s\", snapshot.SnapshotID, snapshot.VolumeNamespace, snapshot.RepositoryType))\n\t\t\tcontinue\n\t\t}\n\n\t\tif directSnapshots[snapshot.VolumeNamespace] == nil {\n\t\t\tdirectSnapshots[snapshot.VolumeNamespace] = []repotypes.SnapshotIdentifier{}\n\t\t}\n\n\t\tdirectSnapshots[snapshot.VolumeNamespace] = append(directSnapshots[snapshot.VolumeNamespace], snapshot)\n\n\t\tr.logger.Infof(\"Deleting snapshot %s, namespace: %s, repo type: %s\", snapshot.SnapshotID, snapshot.VolumeNamespace, snapshot.RepositoryType)\n\t}\n\n\tfor i := range list.Items {\n\t\tcm := list.Items[i]\n\t\tif err := r.Client.Delete(ctx, &cm); err != nil {\n\t\t\tr.logger.Warnf(\"Failed to delete snapshot info configmap %s/%s: %v\", cm.Namespace, cm.Name, err)\n\t\t}\n\t}\n\n\tif len(directSnapshots) > 0 {\n\t\tdeleteErrs := batchDeleteSnapshotFunc(ctx, r.repoEnsurer, r.repoMgr, directSnapshots, backup, r.logger)\n\t\terrs = append(errs, deleteErrs...)\n\t}\n\n\treturn errs\n}\n\nfunc (r *backupDeletionReconciler) patchDeleteBackupRequest(ctx context.Context, req *velerov1api.DeleteBackupRequest, mutate func(*velerov1api.DeleteBackupRequest)) (*velerov1api.DeleteBackupRequest, error) {\n\toriginal := req.DeepCopy()\n\tmutate(req)\n\tif err := r.Patch(ctx, req, client.MergeFrom(original)); err != nil {\n\t\treturn nil, errors.Wrap(err, \"error patching the deletebackuprquest\")\n\t}\n\treturn req, nil\n}\n\nfunc (r *backupDeletionReconciler) patchDeleteBackupRequestWithError(ctx context.Context, req *velerov1api.DeleteBackupRequest, err error) error {\n\t_, err = r.patchDeleteBackupRequest(ctx, req, func(r *velerov1api.DeleteBackupRequest) {\n\t\tr.Status.Phase = velerov1api.DeleteBackupRequestPhaseProcessed\n\t\tr.Status.Errors = []string{err.Error()}\n\t})\n\treturn err\n}\n\nfunc (r *backupDeletionReconciler) patchBackup(ctx context.Context, backup *velerov1api.Backup, mutate func(*velerov1api.Backup)) (*velerov1api.Backup, error) {\n\t//TODO: The patchHelper can't be used here because the `backup/xxx/status` does not exist, until the backup resource is refactored\n\n\t// Record original json\n\toldData, err := json.Marshal(backup)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error marshaling original Backup\")\n\t}\n\n\tnewBackup := backup.DeepCopy()\n\tmutate(newBackup)\n\tnewData, err := json.Marshal(newBackup)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error marshaling updated Backup\")\n\t}\n\tpatchBytes, err := jsonpatch.CreateMergePatch(oldData, newData)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error creating json merge patch for Backup\")\n\t}\n\n\tif err := r.Client.Patch(ctx, backup, client.RawPatch(types.MergePatchType, patchBytes)); err != nil {\n\t\treturn nil, errors.Wrap(err, \"error patching Backup\")\n\t}\n\treturn backup, nil\n}\n\n// getSnapshotsInBackup returns a list of all pod volume snapshot ids associated with\n// a given Velero backup.\nfunc getSnapshotsInBackup(ctx context.Context, backup *velerov1api.Backup, kbClient client.Client) (map[string][]repotypes.SnapshotIdentifier, error) {\n\tpodVolumeBackups := &velerov1api.PodVolumeBackupList{}\n\toptions := &client.ListOptions{\n\t\tLabelSelector: labels.Set(map[string]string{\n\t\t\tvelerov1api.BackupNameLabel: label.GetValidName(backup.Name),\n\t\t}).AsSelector(),\n\t}\n\n\terr := kbClient.List(ctx, podVolumeBackups, options)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\treturn podvolume.GetSnapshotIdentifier(podVolumeBackups), nil\n}\n\nfunc batchDeleteSnapshots(ctx context.Context, repoEnsurer *repository.Ensurer, repoMgr repomanager.Manager,\n\tdirectSnapshots map[string][]repotypes.SnapshotIdentifier, backup *velerov1api.Backup, logger logrus.FieldLogger) []error {\n\tvar errs []error\n\tfor volumeNamespace, snapshots := range directSnapshots {\n\t\tbatchForget := []string{}\n\t\tfor _, snapshot := range snapshots {\n\t\t\tbatchForget = append(batchForget, snapshot.SnapshotID)\n\t\t}\n\n\t\t// For volumes in one backup, the BSL and repositoryType should always be the same\n\t\trepoType := snapshots[0].RepositoryType\n\t\trepo, err := repoEnsurer.EnsureRepo(ctx, backup.Namespace, volumeNamespace, backup.Spec.StorageLocation, repoType)\n\t\tif err != nil {\n\t\t\terrs = append(errs, errors.Wrapf(err, \"error to ensure repo %s-%s-%s, skip deleting PVB snapshots %v\", backup.Spec.StorageLocation, volumeNamespace, repoType, batchForget))\n\t\t\tcontinue\n\t\t}\n\n\t\tif forgetErrs := repoMgr.BatchForget(ctx, repo, batchForget); len(forgetErrs) > 0 {\n\t\t\terrs = append(errs, forgetErrs...)\n\t\t\tcontinue\n\t\t}\n\n\t\tlogger.Infof(\"Batch deleted snapshots %v\", batchForget)\n\t}\n\n\treturn errs\n}\n" }, { "path": "pkg/controller/backup_deletion_controller_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"reflect\"\n\t\"time\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\n\t\"context\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tpkgbackup \"github.com/vmware-tanzu/velero/pkg/backup\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\tpersistencemocks \"github.com/vmware-tanzu/velero/pkg/persistence/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\tpluginmocks \"github.com/vmware-tanzu/velero/pkg/plugin/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\trepomanager \"github.com/vmware-tanzu/velero/pkg/repository/manager\"\n\trepomocks \"github.com/vmware-tanzu/velero/pkg/repository/mocks\"\n\trepotypes \"github.com/vmware-tanzu/velero/pkg/repository/types\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\ntype backupDeletionControllerTestData struct {\n\tfakeClient client.Client\n\tvolumeSnapshotter *velerotest.FakeVolumeSnapshotter\n\tbackupStore *persistencemocks.BackupStore\n\tcontroller *backupDeletionReconciler\n\treq ctrl.Request\n}\n\nfunc defaultTestDbr() *velerov1api.DeleteBackupRequest {\n\treq := pkgbackup.NewDeleteBackupRequest(\"foo\", \"uid\")\n\treq.Namespace = velerov1api.DefaultNamespace\n\treq.Name = \"foo-abcde\"\n\treturn req\n}\n\nfunc setupBackupDeletionControllerTest(t *testing.T, req *velerov1api.DeleteBackupRequest, objects ...runtime.Object) *backupDeletionControllerTestData {\n\tt.Helper()\n\tvar (\n\t\tfakeClient = velerotest.NewFakeControllerRuntimeClient(t, append(objects, req)...)\n\t\tvolumeSnapshotter = &velerotest.FakeVolumeSnapshotter{SnapshotsTaken: sets.NewString()}\n\t\tpluginManager = &pluginmocks.Manager{}\n\t\tbackupStore = &persistencemocks.BackupStore{}\n\t)\n\n\tdata := &backupDeletionControllerTestData{\n\t\tfakeClient: fakeClient,\n\t\tvolumeSnapshotter: volumeSnapshotter,\n\t\tbackupStore: backupStore,\n\t\tcontroller: NewBackupDeletionReconciler(\n\t\t\tvelerotest.NewLogger(),\n\t\t\tfakeClient,\n\t\t\tNewBackupTracker(),\n\t\t\tnil, // repository manager\n\t\t\tmetrics.NewServerMetrics(),\n\t\t\tnil, // discovery helper\n\t\t\tfunc(logrus.FieldLogger) clientmgmt.Manager { return pluginManager },\n\t\t\tNewFakeSingleObjectBackupStoreGetter(backupStore),\n\t\t\tvelerotest.NewFakeCredentialsFileStore(\"\", nil),\n\t\t\tnil,\n\t\t),\n\t\treq: ctrl.Request{NamespacedName: types.NamespacedName{Namespace: req.Namespace, Name: req.Name}},\n\t}\n\n\tpluginManager.On(\"CleanupClients\").Return(nil)\n\treturn data\n}\n\nfunc TestBackupDeletionControllerReconcile(t *testing.T) {\n\tt.Run(\"failed to get backup store\", func(t *testing.T) {\n\t\tbackup := builder.ForBackup(velerov1api.DefaultNamespace, \"foo\").StorageLocation(\"default\").Result()\n\t\tlocation := &velerov1api.BackupStorageLocation{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: backup.Namespace,\n\t\t\t\tName: backup.Spec.StorageLocation,\n\t\t\t},\n\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\tProvider: \"objStoreProvider\",\n\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\tBucket: \"bucket\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tStatus: velerov1api.BackupStorageLocationStatus{\n\t\t\t\tPhase: velerov1api.BackupStorageLocationPhaseAvailable,\n\t\t\t},\n\t\t}\n\t\tdbr := defaultTestDbr()\n\t\ttd := setupBackupDeletionControllerTest(t, dbr, location, backup)\n\t\ttd.controller.backupStoreGetter = &fakeErrorBackupStoreGetter{}\n\t\t_, err := td.controller.Reconcile(ctx, td.req)\n\t\trequire.NoError(t, err)\n\t\tres := &velerov1api.DeleteBackupRequest{}\n\t\ttd.fakeClient.Get(ctx, td.req.NamespacedName, res)\n\t\tassert.Equal(t, \"Processed\", string(res.Status.Phase))\n\t\tassert.Len(t, res.Status.Errors, 1)\n\t\tassert.True(t, strings.HasPrefix(res.Status.Errors[0], \"error getting the backup store\"))\n\t})\n\n\tt.Run(\"missing spec.backupName\", func(t *testing.T) {\n\t\tdbr := defaultTestDbr()\n\t\tdbr.Spec.BackupName = \"\"\n\t\ttd := setupBackupDeletionControllerTest(t, dbr)\n\n\t\t_, err := td.controller.Reconcile(ctx, td.req)\n\t\trequire.NoError(t, err)\n\n\t\tres := &velerov1api.DeleteBackupRequest{}\n\t\terr = td.fakeClient.Get(ctx, td.req.NamespacedName, res)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, \"Processed\", string(res.Status.Phase))\n\t\tassert.Len(t, res.Status.Errors, 1)\n\t\tassert.Equal(t, \"spec.backupName is required\", res.Status.Errors[0])\n\t})\n\n\tt.Run(\"existing deletion requests for the backup are deleted\", func(t *testing.T) {\n\t\tinput := defaultTestDbr()\n\t\ttd := setupBackupDeletionControllerTest(t, input)\n\n\t\t// add the backup to the tracker so the execution of reconcile doesn't progress\n\t\t// past checking for an in-progress backup. this makes validation easier.\n\t\ttd.controller.backupTracker.Add(td.req.Namespace, input.Spec.BackupName)\n\t\texisting := &velerov1api.DeleteBackupRequest{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: td.req.Namespace,\n\t\t\t\tName: \"bar\",\n\t\t\t\tLabels: map[string]string{\n\t\t\t\t\tvelerov1api.BackupNameLabel: input.Spec.BackupName,\n\t\t\t\t},\n\t\t\t},\n\t\t\tSpec: velerov1api.DeleteBackupRequestSpec{\n\t\t\t\tBackupName: input.Spec.BackupName,\n\t\t\t},\n\t\t}\n\t\terr := td.fakeClient.Create(t.Context(), existing)\n\t\trequire.NoError(t, err)\n\t\texisting2 :=\n\t\t\t&velerov1api.DeleteBackupRequest{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: td.req.Namespace,\n\t\t\t\t\tName: \"bar-2\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\tvelerov1api.BackupNameLabel: \"some-other-backup\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.DeleteBackupRequestSpec{\n\t\t\t\t\tBackupName: \"some-other-backup\",\n\t\t\t\t},\n\t\t\t}\n\t\terr = td.fakeClient.Create(t.Context(), existing2)\n\t\trequire.NoError(t, err)\n\t\t_, err = td.controller.Reconcile(t.Context(), td.req)\n\t\trequire.NoError(t, err)\n\t\t// verify \"existing\" is deleted\n\t\terr = td.fakeClient.Get(t.Context(), types.NamespacedName{\n\t\t\tNamespace: existing.Namespace,\n\t\t\tName: existing.Name,\n\t\t}, &velerov1api.DeleteBackupRequest{})\n\n\t\tassert.True(t, apierrors.IsNotFound(err), \"Expected not found error, but actual value of error: %v\", err)\n\n\t\t// verify \"existing2\" remains\n\t\tassert.NoError(t, td.fakeClient.Get(t.Context(), types.NamespacedName{\n\t\t\tNamespace: existing2.Namespace,\n\t\t\tName: existing2.Name,\n\t\t}, &velerov1api.DeleteBackupRequest{}))\n\t})\n\tt.Run(\"deleting an in progress backup isn't allowed\", func(t *testing.T) {\n\t\tdbr := defaultTestDbr()\n\t\ttd := setupBackupDeletionControllerTest(t, dbr)\n\n\t\ttd.controller.backupTracker.Add(td.req.Namespace, dbr.Spec.BackupName)\n\t\t_, err := td.controller.Reconcile(t.Context(), td.req)\n\t\trequire.NoError(t, err)\n\n\t\tres := &velerov1api.DeleteBackupRequest{}\n\t\terr = td.fakeClient.Get(ctx, td.req.NamespacedName, res)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, \"Processed\", string(res.Status.Phase))\n\t\tassert.Len(t, res.Status.Errors, 1)\n\t\tassert.Equal(t, \"backup is still in progress\", res.Status.Errors[0])\n\t})\n\n\tt.Run(\"unable to find backup\", func(t *testing.T) {\n\t\ttd := setupBackupDeletionControllerTest(t, defaultTestDbr())\n\n\t\t_, err := td.controller.Reconcile(t.Context(), td.req)\n\t\trequire.NoError(t, err)\n\n\t\tres := &velerov1api.DeleteBackupRequest{}\n\t\terr = td.fakeClient.Get(ctx, td.req.NamespacedName, res)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, \"Processed\", string(res.Status.Phase))\n\t\tassert.Len(t, res.Status.Errors, 1)\n\t\tassert.Equal(t, \"backup not found\", res.Status.Errors[0])\n\t})\n\tt.Run(\"unable to find backup storage location\", func(t *testing.T) {\n\t\tbackup := builder.ForBackup(velerov1api.DefaultNamespace, \"foo\").StorageLocation(\"default\").Result()\n\n\t\ttd := setupBackupDeletionControllerTest(t, defaultTestDbr(), backup)\n\n\t\t_, err := td.controller.Reconcile(t.Context(), td.req)\n\t\trequire.NoError(t, err)\n\n\t\tres := &velerov1api.DeleteBackupRequest{}\n\t\terr = td.fakeClient.Get(ctx, td.req.NamespacedName, res)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, \"Processed\", string(res.Status.Phase))\n\t\tassert.Len(t, res.Status.Errors, 1)\n\t\tassert.Equal(t, \"backup storage location default not found\", res.Status.Errors[0])\n\t})\n\n\tt.Run(\"backup storage location is in read-only mode\", func(t *testing.T) {\n\t\tbackup := builder.ForBackup(velerov1api.DefaultNamespace, \"foo\").StorageLocation(\"default\").Result()\n\t\tlocation := builder.ForBackupStorageLocation(\"velero\", \"default\").Phase(velerov1api.BackupStorageLocationPhaseAvailable).AccessMode(velerov1api.BackupStorageLocationAccessModeReadOnly).Result()\n\n\t\ttd := setupBackupDeletionControllerTest(t, defaultTestDbr(), location, backup)\n\n\t\t_, err := td.controller.Reconcile(t.Context(), td.req)\n\t\trequire.NoError(t, err)\n\n\t\tres := &velerov1api.DeleteBackupRequest{}\n\t\terr = td.fakeClient.Get(ctx, td.req.NamespacedName, res)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, \"Processed\", string(res.Status.Phase))\n\t\tassert.Len(t, res.Status.Errors, 1)\n\t\tassert.Equal(t, \"cannot delete backup because backup storage location default is currently in read-only mode\", res.Status.Errors[0])\n\t})\n\n\tt.Run(\"backup storage location is in unavailable state\", func(t *testing.T) {\n\t\tbackup := builder.ForBackup(velerov1api.DefaultNamespace, \"foo\").StorageLocation(\"default\").Result()\n\t\tlocation := builder.ForBackupStorageLocation(\"velero\", \"default\").Phase(velerov1api.BackupStorageLocationPhaseUnavailable).Result()\n\n\t\ttd := setupBackupDeletionControllerTest(t, defaultTestDbr(), location, backup)\n\n\t\t_, err := td.controller.Reconcile(t.Context(), td.req)\n\t\trequire.NoError(t, err)\n\n\t\tres := &velerov1api.DeleteBackupRequest{}\n\t\terr = td.fakeClient.Get(ctx, td.req.NamespacedName, res)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, \"Processed\", string(res.Status.Phase))\n\t\tassert.Len(t, res.Status.Errors, 1)\n\t\tassert.Equal(t, \"cannot delete backup because backup storage location default is currently in Unavailable state\", res.Status.Errors[0])\n\t})\n\n\tt.Run(\"full delete, no errors\", func(t *testing.T) {\n\t\tinput := defaultTestDbr()\n\n\t\t// Clear out resource labels to make sure the controller adds them and does not\n\t\t// panic when encountering a nil Labels map\n\t\t// (https://github.com/vmware-tanzu/velero/issues/1546)\n\t\tinput.Labels = nil\n\n\t\tbackup := builder.ForBackup(velerov1api.DefaultNamespace, \"foo\").Result()\n\t\tbackup.UID = \"uid\"\n\t\tbackup.Spec.StorageLocation = \"primary\"\n\n\t\trestore1 := builder.ForRestore(velerov1api.DefaultNamespace, \"restore-1\").Phase(velerov1api.RestorePhaseCompleted).Backup(\"foo\").Result()\n\t\trestore2 := builder.ForRestore(velerov1api.DefaultNamespace, \"restore-2\").Phase(velerov1api.RestorePhaseCompleted).Backup(\"foo\").Result()\n\t\trestore3 := builder.ForRestore(velerov1api.DefaultNamespace, \"restore-3\").Phase(velerov1api.RestorePhaseCompleted).Backup(\"some-other-backup\").Result()\n\n\t\tlocation := &velerov1api.BackupStorageLocation{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: backup.Namespace,\n\t\t\t\tName: backup.Spec.StorageLocation,\n\t\t\t},\n\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\tProvider: \"objStoreProvider\",\n\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\tBucket: \"bucket\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tStatus: velerov1api.BackupStorageLocationStatus{\n\t\t\t\tPhase: velerov1api.BackupStorageLocationPhaseAvailable,\n\t\t\t},\n\t\t}\n\n\t\tsnapshotLocation := &velerov1api.VolumeSnapshotLocation{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: backup.Namespace,\n\t\t\t\tName: \"vsl-1\",\n\t\t\t},\n\t\t\tSpec: velerov1api.VolumeSnapshotLocationSpec{\n\t\t\t\tProvider: \"provider-1\",\n\t\t\t},\n\t\t}\n\t\ttd := setupBackupDeletionControllerTest(t, input, backup, restore1, restore2, restore3, location, snapshotLocation)\n\n\t\ttd.volumeSnapshotter.SnapshotsTaken.Insert(\"snap-1\")\n\n\t\tsnapshots := []*volume.Snapshot{\n\t\t\t{\n\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\tLocation: \"vsl-1\",\n\t\t\t\t},\n\t\t\t\tStatus: volume.SnapshotStatus{\n\t\t\t\t\tProviderSnapshotID: \"snap-1\",\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\n\t\tpluginManager := &pluginmocks.Manager{}\n\t\tpluginManager.On(\"GetVolumeSnapshotter\", \"provider-1\").Return(td.volumeSnapshotter, nil)\n\t\tpluginManager.On(\"GetDeleteItemActions\").Return(nil, nil)\n\t\tpluginManager.On(\"CleanupClients\")\n\t\ttd.controller.newPluginManager = func(logrus.FieldLogger) clientmgmt.Manager { return pluginManager }\n\n\t\ttd.backupStore.On(\"GetBackupVolumeSnapshots\", input.Spec.BackupName).Return(snapshots, nil)\n\t\ttd.backupStore.On(\"GetBackupContents\", input.Spec.BackupName).Return(io.NopCloser(bytes.NewReader([]byte(\"hello world\"))), nil)\n\t\ttd.backupStore.On(\"DeleteBackup\", input.Spec.BackupName).Return(nil)\n\n\t\t_, err := td.controller.Reconcile(t.Context(), td.req)\n\t\trequire.NoError(t, err)\n\n\t\t// the dbr should be deleted\n\t\tres := &velerov1api.DeleteBackupRequest{}\n\t\terr = td.fakeClient.Get(ctx, td.req.NamespacedName, res)\n\t\tassert.True(t, apierrors.IsNotFound(err), \"Expected not found error, but actual value of error: %v\", err)\n\t\tif err == nil {\n\t\t\tt.Logf(\"status of the dbr: %s, errors in dbr: %v\", res.Status.Phase, res.Status.Errors)\n\t\t}\n\n\t\t// backup CR, restore CR restore-1 and restore-2 should be deleted\n\t\terr = td.fakeClient.Get(t.Context(), types.NamespacedName{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: backup.Name,\n\t\t}, &velerov1api.Backup{})\n\t\tassert.True(t, apierrors.IsNotFound(err), \"Expected not found error, but actual value of error: %v\", err)\n\n\t\terr = td.fakeClient.Get(t.Context(), types.NamespacedName{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: \"restore-1\",\n\t\t}, &velerov1api.Restore{})\n\t\tassert.True(t, apierrors.IsNotFound(err), \"Expected not found error, but actual value of error: %v\", err)\n\n\t\terr = td.fakeClient.Get(t.Context(), types.NamespacedName{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: \"restore-2\",\n\t\t}, &velerov1api.Restore{})\n\t\tassert.True(t, apierrors.IsNotFound(err), \"Expected not found error, but actual value of error: %v\", err)\n\n\t\t// restore-3 should remain\n\t\terr = td.fakeClient.Get(t.Context(), types.NamespacedName{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: \"restore-3\",\n\t\t}, &velerov1api.Restore{})\n\t\trequire.NoError(t, err)\n\n\t\ttd.backupStore.AssertCalled(t, \"DeleteBackup\", input.Spec.BackupName)\n\n\t\t// Make sure snapshot was deleted\n\t\tassert.Equal(t, 0, td.volumeSnapshotter.SnapshotsTaken.Len())\n\t})\n\tt.Run(\"full delete, no errors, with backup name greater than 63 chars\", func(t *testing.T) {\n\t\tbackup := defaultBackup().\n\t\t\tObjectMeta(\n\t\t\t\tbuilder.WithName(\"the-really-long-backup-name-that-is-much-more-than-63-characters\"),\n\t\t\t).\n\t\t\tResult()\n\t\tbackup.UID = \"uid\"\n\t\tbackup.Spec.StorageLocation = \"primary\"\n\n\t\trestore1 := builder.ForRestore(backup.Namespace, \"restore-1\").\n\t\t\tPhase(velerov1api.RestorePhaseCompleted).\n\t\t\tBackup(backup.Name).\n\t\t\tResult()\n\t\trestore2 := builder.ForRestore(backup.Namespace, \"restore-2\").\n\t\t\tPhase(velerov1api.RestorePhaseCompleted).\n\t\t\tBackup(backup.Name).\n\t\t\tResult()\n\t\trestore3 := builder.ForRestore(backup.Namespace, \"restore-3\").\n\t\t\tPhase(velerov1api.RestorePhaseCompleted).\n\t\t\tBackup(\"some-other-backup\").\n\t\t\tResult()\n\n\t\tdbr := pkgbackup.NewDeleteBackupRequest(backup.Name, \"uid\")\n\t\tdbr.Namespace = velerov1api.DefaultNamespace\n\t\tdbr.Name = \"foo-abcde\"\n\t\t// Clear out resource labels to make sure the controller adds them\n\t\tdbr.Labels = nil\n\n\t\tlocation := &velerov1api.BackupStorageLocation{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: backup.Namespace,\n\t\t\t\tName: backup.Spec.StorageLocation,\n\t\t\t},\n\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\tProvider: \"objStoreProvider\",\n\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\tBucket: \"bucket\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tStatus: velerov1api.BackupStorageLocationStatus{\n\t\t\t\tPhase: velerov1api.BackupStorageLocationPhaseAvailable,\n\t\t\t},\n\t\t}\n\n\t\tsnapshotLocation := &velerov1api.VolumeSnapshotLocation{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: backup.Namespace,\n\t\t\t\tName: \"vsl-1\",\n\t\t\t},\n\t\t\tSpec: velerov1api.VolumeSnapshotLocationSpec{\n\t\t\t\tProvider: \"provider-1\",\n\t\t\t},\n\t\t}\n\t\ttd := setupBackupDeletionControllerTest(t, dbr, backup, restore1, restore2, restore3, location, snapshotLocation)\n\n\t\tsnapshots := []*volume.Snapshot{\n\t\t\t{\n\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\tLocation: \"vsl-1\",\n\t\t\t\t},\n\t\t\t\tStatus: volume.SnapshotStatus{\n\t\t\t\t\tProviderSnapshotID: \"snap-1\",\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\n\t\tpluginManager := &pluginmocks.Manager{}\n\t\tpluginManager.On(\"GetVolumeSnapshotter\", \"provider-1\").Return(td.volumeSnapshotter, nil)\n\t\tpluginManager.On(\"GetDeleteItemActions\").Return(nil, nil)\n\t\tpluginManager.On(\"CleanupClients\")\n\t\ttd.controller.newPluginManager = func(logrus.FieldLogger) clientmgmt.Manager { return pluginManager }\n\n\t\ttd.backupStore.On(\"GetBackupVolumeSnapshots\", dbr.Spec.BackupName).Return(snapshots, nil)\n\t\ttd.backupStore.On(\"GetBackupContents\", dbr.Spec.BackupName).Return(io.NopCloser(bytes.NewReader([]byte(\"hello world\"))), nil)\n\t\ttd.backupStore.On(\"DeleteBackup\", dbr.Spec.BackupName).Return(nil)\n\t\ttd.backupStore.On(\"DeleteRestore\", \"restore-1\").Return(nil)\n\t\ttd.backupStore.On(\"DeleteRestore\", \"restore-2\").Return(nil)\n\n\t\ttd.volumeSnapshotter.SnapshotsTaken.Insert(\"snap-1\")\n\n\t\t_, err := td.controller.Reconcile(t.Context(), td.req)\n\t\trequire.NoError(t, err)\n\n\t\t// the dbr should be deleted\n\t\tres := &velerov1api.DeleteBackupRequest{}\n\t\terr = td.fakeClient.Get(ctx, td.req.NamespacedName, res)\n\t\tassert.True(t, apierrors.IsNotFound(err), \"Expected not found error, but actual value of error: %v\", err)\n\t\tif err == nil {\n\t\t\tt.Logf(\"status of the dbr: %s, errors in dbr: %v\", res.Status.Phase, res.Status.Errors)\n\t\t}\n\n\t\t// backup CR, restore CR restore-1 and restore-2 should be deleted\n\t\terr = td.fakeClient.Get(t.Context(), types.NamespacedName{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: backup.Name,\n\t\t}, &velerov1api.Backup{})\n\t\tassert.True(t, apierrors.IsNotFound(err), \"Expected not found error, but actual value of error: %v\", err)\n\n\t\terr = td.fakeClient.Get(t.Context(), types.NamespacedName{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: \"restore-1\",\n\t\t}, &velerov1api.Restore{})\n\t\tassert.True(t, apierrors.IsNotFound(err), \"Expected not found error, but actual value of error: %v\", err)\n\n\t\terr = td.fakeClient.Get(t.Context(), types.NamespacedName{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: \"restore-2\",\n\t\t}, &velerov1api.Restore{})\n\t\tassert.True(t, apierrors.IsNotFound(err), \"Expected not found error, but actual value of error: %v\", err)\n\n\t\t// restore-3 should remain\n\t\terr = td.fakeClient.Get(t.Context(), types.NamespacedName{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: \"restore-3\",\n\t\t}, &velerov1api.Restore{})\n\t\trequire.NoError(t, err)\n\n\t\t// Make sure snapshot was deleted\n\t\tassert.Equal(t, 0, td.volumeSnapshotter.SnapshotsTaken.Len())\n\t})\n\tt.Run(\"backup is not downloaded when there are no DeleteItemAction plugins\", func(t *testing.T) {\n\t\tbackup := builder.ForBackup(velerov1api.DefaultNamespace, \"foo\").Result()\n\t\tbackup.UID = \"uid\"\n\t\tbackup.Spec.StorageLocation = \"primary\"\n\n\t\tinput := defaultTestDbr()\n\t\t// Clear out resource labels to make sure the controller adds them and does not\n\t\t// panic when encountering a nil Labels map\n\t\t// (https://github.com/vmware-tanzu/velero/issues/1546)\n\t\tinput.Labels = nil\n\n\t\tlocation := &velerov1api.BackupStorageLocation{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: backup.Namespace,\n\t\t\t\tName: backup.Spec.StorageLocation,\n\t\t\t},\n\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\tProvider: \"objStoreProvider\",\n\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\tBucket: \"bucket\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tStatus: velerov1api.BackupStorageLocationStatus{\n\t\t\t\tPhase: velerov1api.BackupStorageLocationPhaseAvailable,\n\t\t\t},\n\t\t}\n\n\t\tsnapshotLocation := &velerov1api.VolumeSnapshotLocation{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: backup.Namespace,\n\t\t\t\tName: \"vsl-1\",\n\t\t\t},\n\t\t\tSpec: velerov1api.VolumeSnapshotLocationSpec{\n\t\t\t\tProvider: \"provider-1\",\n\t\t\t},\n\t\t}\n\t\ttd := setupBackupDeletionControllerTest(t, defaultTestDbr(), backup, location, snapshotLocation)\n\t\ttd.volumeSnapshotter.SnapshotsTaken.Insert(\"snap-1\")\n\n\t\tsnapshots := []*volume.Snapshot{\n\t\t\t{\n\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\tLocation: \"vsl-1\",\n\t\t\t\t},\n\t\t\t\tStatus: volume.SnapshotStatus{\n\t\t\t\t\tProviderSnapshotID: \"snap-1\",\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\n\t\tpluginManager := &pluginmocks.Manager{}\n\t\tpluginManager.On(\"GetVolumeSnapshotter\", \"provider-1\").Return(td.volumeSnapshotter, nil)\n\t\tpluginManager.On(\"GetDeleteItemActions\").Return([]velero.DeleteItemAction{}, nil)\n\t\tpluginManager.On(\"CleanupClients\")\n\t\ttd.controller.newPluginManager = func(logrus.FieldLogger) clientmgmt.Manager { return pluginManager }\n\n\t\ttd.backupStore.On(\"GetBackupVolumeSnapshots\", input.Spec.BackupName).Return(snapshots, nil)\n\t\ttd.backupStore.On(\"DeleteBackup\", input.Spec.BackupName).Return(nil)\n\n\t\t_, err := td.controller.Reconcile(t.Context(), td.req)\n\t\trequire.NoError(t, err)\n\n\t\ttd.backupStore.AssertNotCalled(t, \"GetBackupContents\", mock.Anything)\n\t\ttd.backupStore.AssertCalled(t, \"DeleteBackup\", input.Spec.BackupName)\n\n\t\t// the dbr should be deleted\n\t\tres := &velerov1api.DeleteBackupRequest{}\n\t\terr = td.fakeClient.Get(ctx, td.req.NamespacedName, res)\n\t\tassert.True(t, apierrors.IsNotFound(err), \"Expected not found error, but actual value of error: %v\", err)\n\t\tif err == nil {\n\t\t\tt.Logf(\"status of the dbr: %s, errors in dbr: %v\", res.Status.Phase, res.Status.Errors)\n\t\t}\n\n\t\t// backup CR should be deleted\n\t\terr = td.fakeClient.Get(t.Context(), types.NamespacedName{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: backup.Name,\n\t\t}, &velerov1api.Backup{})\n\t\tassert.True(t, apierrors.IsNotFound(err), \"Expected not found error, but actual value of error: %v\", err)\n\n\t\t// Make sure snapshot was deleted\n\t\tassert.Equal(t, 0, td.volumeSnapshotter.SnapshotsTaken.Len())\n\t})\n\tt.Run(\"backup is still deleted if downloading tarball fails for DeleteItemAction plugins\", func(t *testing.T) {\n\t\tbackup := builder.ForBackup(velerov1api.DefaultNamespace, \"foo\").Result()\n\t\tbackup.UID = \"uid\"\n\t\tbackup.Spec.StorageLocation = \"primary\"\n\n\t\tinput := defaultTestDbr()\n\t\t// Clear out resource labels to make sure the controller adds them and does not\n\t\t// panic when encountering a nil Labels map\n\t\t// (https://github.com/vmware-tanzu/velero/issues/1546)\n\t\tinput.Labels = nil\n\n\t\tlocation := &velerov1api.BackupStorageLocation{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: backup.Namespace,\n\t\t\t\tName: backup.Spec.StorageLocation,\n\t\t\t},\n\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\tProvider: \"objStoreProvider\",\n\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\tBucket: \"bucket\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tStatus: velerov1api.BackupStorageLocationStatus{\n\t\t\t\tPhase: velerov1api.BackupStorageLocationPhaseAvailable,\n\t\t\t},\n\t\t}\n\n\t\tsnapshotLocation := &velerov1api.VolumeSnapshotLocation{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: backup.Namespace,\n\t\t\t\tName: \"vsl-1\",\n\t\t\t},\n\t\t\tSpec: velerov1api.VolumeSnapshotLocationSpec{\n\t\t\t\tProvider: \"provider-1\",\n\t\t\t},\n\t\t}\n\n\t\tcsiSnapshot := builder.ForVolumeSnapshot(\"user-ns\", \"vs-1\").ObjectMeta(\n\t\t\tbuilder.WithLabelsMap(map[string]string{\n\t\t\t\t\"velero.io/backup-name\": \"foo\",\n\t\t\t})).SourcePVC(\"some-pvc\").Result()\n\n\t\ttd := setupBackupDeletionControllerTest(t, defaultTestDbr(), backup, location, snapshotLocation, csiSnapshot)\n\t\ttd.volumeSnapshotter.SnapshotsTaken.Insert(\"snap-1\")\n\n\t\tsnapshots := []*volume.Snapshot{\n\t\t\t{\n\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\tLocation: \"vsl-1\",\n\t\t\t\t},\n\t\t\t\tStatus: volume.SnapshotStatus{\n\t\t\t\t\tProviderSnapshotID: \"snap-1\",\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\n\t\tpluginManager := &pluginmocks.Manager{}\n\t\tpluginManager.On(\"GetVolumeSnapshotter\", \"provider-1\").Return(td.volumeSnapshotter, nil)\n\t\tpluginManager.On(\"GetDeleteItemActions\").Return([]velero.DeleteItemAction{new(mocks.DeleteItemAction)}, nil)\n\t\tpluginManager.On(\"CleanupClients\")\n\t\ttd.controller.newPluginManager = func(logrus.FieldLogger) clientmgmt.Manager { return pluginManager }\n\n\t\ttd.backupStore.On(\"GetBackupVolumeSnapshots\", input.Spec.BackupName).Return(snapshots, nil)\n\t\ttd.backupStore.On(\"GetBackupContents\", input.Spec.BackupName).Return(nil, fmt.Errorf(\"error downloading tarball\"))\n\t\ttd.backupStore.On(\"DeleteBackup\", input.Spec.BackupName).Return(nil)\n\n\t\t_, err := td.controller.Reconcile(t.Context(), td.req)\n\t\trequire.NoError(t, err)\n\n\t\ttd.backupStore.AssertCalled(t, \"GetBackupContents\", input.Spec.BackupName)\n\t\ttd.backupStore.AssertCalled(t, \"DeleteBackup\", input.Spec.BackupName)\n\n\t\t// the dbr should be deleted\n\t\tres := &velerov1api.DeleteBackupRequest{}\n\t\terr = td.fakeClient.Get(ctx, td.req.NamespacedName, res)\n\t\tassert.True(t, apierrors.IsNotFound(err), \"Expected not found error, but actual value of error: %v\", err)\n\t\tif err == nil {\n\t\t\tt.Logf(\"status of the dbr: %s, errors in dbr: %v\", res.Status.Phase, res.Status.Errors)\n\t\t}\n\n\t\t// backup CR should be deleted\n\t\terr = td.fakeClient.Get(t.Context(), types.NamespacedName{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: backup.Name,\n\t\t}, &velerov1api.Backup{})\n\t\tassert.True(t, apierrors.IsNotFound(err), \"Expected not found error, but actual value of error: %v\", err)\n\n\t\t// leaked CSI snapshot should be deleted\n\t\terr = td.fakeClient.Get(t.Context(), types.NamespacedName{\n\t\t\tNamespace: \"user-ns\",\n\t\t\tName: \"vs-1\",\n\t\t}, &snapshotv1api.VolumeSnapshot{})\n\t\tassert.True(t, apierrors.IsNotFound(err), \"Expected not found error for the leaked CSI snapshot, but actual value of error: %v\", err)\n\n\t\t// Make sure snapshot was deleted\n\t\tassert.Equal(t, 0, td.volumeSnapshotter.SnapshotsTaken.Len())\n\t})\n\tt.Run(\"Expired request will be deleted if the status is processed\", func(t *testing.T) {\n\t\texpired := time.Date(2018, 4, 3, 12, 0, 0, 0, time.UTC)\n\t\tinput := defaultTestDbr()\n\t\tinput.CreationTimestamp = metav1.Time{\n\t\t\tTime: expired,\n\t\t}\n\t\tinput.Status.Phase = velerov1api.DeleteBackupRequestPhaseProcessed\n\t\ttd := setupBackupDeletionControllerTest(t, input)\n\t\ttd.backupStore.On(\"DeleteBackup\", mock.Anything).Return(nil)\n\t\t_, err := td.controller.Reconcile(t.Context(), td.req)\n\t\trequire.NoError(t, err)\n\n\t\tres := &velerov1api.DeleteBackupRequest{}\n\t\terr = td.fakeClient.Get(ctx, td.req.NamespacedName, res)\n\t\tassert.True(t, apierrors.IsNotFound(err), \"Expected not found error, but actual value of error: %v\", err)\n\t\ttd.backupStore.AssertNotCalled(t, \"DeleteBackup\", mock.Anything)\n\t})\n\n\tt.Run(\"Expired request will not be deleted if the status is not processed\", func(t *testing.T) {\n\t\texpired := time.Date(2018, 4, 3, 12, 0, 0, 0, time.UTC)\n\t\tinput := defaultTestDbr()\n\t\tinput.CreationTimestamp = metav1.Time{\n\t\t\tTime: expired,\n\t\t}\n\t\tinput.Status.Phase = velerov1api.DeleteBackupRequestPhaseNew\n\t\ttd := setupBackupDeletionControllerTest(t, input)\n\t\ttd.backupStore.On(\"DeleteBackup\", mock.Anything).Return(nil)\n\n\t\t_, err := td.controller.Reconcile(t.Context(), td.req)\n\t\trequire.NoError(t, err)\n\n\t\tres := &velerov1api.DeleteBackupRequest{}\n\t\terr = td.fakeClient.Get(ctx, td.req.NamespacedName, res)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, \"Processed\", string(res.Status.Phase))\n\t\tassert.Len(t, res.Status.Errors, 1)\n\t\tassert.Equal(t, \"backup not found\", res.Status.Errors[0])\n\t})\n}\n\nfunc TestGetSnapshotsInBackup(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpodVolumeBackups []velerov1api.PodVolumeBackup\n\t\texpected map[string][]repotypes.SnapshotIdentifier\n\t\tlongBackupNameEnabled bool\n\t}{\n\t\t{\n\t\t\tname: \"no pod volume backups\",\n\t\t\tpodVolumeBackups: nil,\n\t\t\texpected: map[string][]repotypes.SnapshotIdentifier{},\n\t\t},\n\t\t{\n\t\t\tname: \"no pod volume backups with matching label\",\n\t\t\tpodVolumeBackups: []velerov1api.PodVolumeBackup{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"foo\", Labels: map[string]string{velerov1api.BackupNameLabel: \"non-matching-backup-1\"}},\n\t\t\t\t\tSpec: velerov1api.PodVolumeBackupSpec{\n\t\t\t\t\t\tPod: corev1api.ObjectReference{Name: \"pod-1\", Namespace: \"ns-1\"},\n\t\t\t\t\t},\n\t\t\t\t\tStatus: velerov1api.PodVolumeBackupStatus{SnapshotID: \"snap-1\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"bar\", Labels: map[string]string{velerov1api.BackupNameLabel: \"non-matching-backup-2\"}},\n\t\t\t\t\tSpec: velerov1api.PodVolumeBackupSpec{\n\t\t\t\t\t\tPod: corev1api.ObjectReference{Name: \"pod-2\", Namespace: \"ns-2\"},\n\t\t\t\t\t},\n\t\t\t\t\tStatus: velerov1api.PodVolumeBackupStatus{SnapshotID: \"snap-2\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: map[string][]repotypes.SnapshotIdentifier{},\n\t\t},\n\t\t{\n\t\t\tname: \"some pod volume backups with matching label\",\n\t\t\tpodVolumeBackups: []velerov1api.PodVolumeBackup{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"foo\", Labels: map[string]string{velerov1api.BackupNameLabel: \"non-matching-backup-1\"}},\n\t\t\t\t\tSpec: velerov1api.PodVolumeBackupSpec{\n\t\t\t\t\t\tPod: corev1api.ObjectReference{Name: \"pod-1\", Namespace: \"ns-1\"},\n\t\t\t\t\t},\n\t\t\t\t\tStatus: velerov1api.PodVolumeBackupStatus{SnapshotID: \"snap-1\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"bar\", Labels: map[string]string{velerov1api.BackupNameLabel: \"non-matching-backup-2\"}},\n\t\t\t\t\tSpec: velerov1api.PodVolumeBackupSpec{\n\t\t\t\t\t\tPod: corev1api.ObjectReference{Name: \"pod-2\", Namespace: \"ns-2\"},\n\t\t\t\t\t},\n\t\t\t\t\tStatus: velerov1api.PodVolumeBackupStatus{SnapshotID: \"snap-2\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"completed-pvb\", Labels: map[string]string{velerov1api.BackupNameLabel: \"backup-1\"}},\n\t\t\t\t\tSpec: velerov1api.PodVolumeBackupSpec{\n\t\t\t\t\t\tPod: corev1api.ObjectReference{Name: \"pod-1\", Namespace: \"ns-1\"},\n\t\t\t\t\t},\n\t\t\t\t\tStatus: velerov1api.PodVolumeBackupStatus{SnapshotID: \"snap-3\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"completed-pvb-2\", Labels: map[string]string{velerov1api.BackupNameLabel: \"backup-1\"}},\n\t\t\t\t\tSpec: velerov1api.PodVolumeBackupSpec{\n\t\t\t\t\t\tPod: corev1api.ObjectReference{Name: \"pod-1\", Namespace: \"ns-1\"},\n\t\t\t\t\t},\n\t\t\t\t\tStatus: velerov1api.PodVolumeBackupStatus{SnapshotID: \"snap-4\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"incomplete-or-failed-pvb\", Labels: map[string]string{velerov1api.BackupNameLabel: \"backup-1\"}},\n\t\t\t\t\tSpec: velerov1api.PodVolumeBackupSpec{\n\t\t\t\t\t\tPod: corev1api.ObjectReference{Name: \"pod-1\", Namespace: \"ns-2\"},\n\t\t\t\t\t},\n\t\t\t\t\tStatus: velerov1api.PodVolumeBackupStatus{SnapshotID: \"\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: map[string][]repotypes.SnapshotIdentifier{\n\t\t\t\t\"ns-1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tVolumeNamespace: \"ns-1\",\n\t\t\t\t\t\tSnapshotID: \"snap-3\",\n\t\t\t\t\t\tRepositoryType: \"restic\",\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tVolumeNamespace: \"ns-1\",\n\t\t\t\t\t\tSnapshotID: \"snap-4\",\n\t\t\t\t\t\tRepositoryType: \"restic\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"some pod volume backups with matching label and backup name greater than 63 chars\",\n\t\t\tlongBackupNameEnabled: true,\n\t\t\tpodVolumeBackups: []velerov1api.PodVolumeBackup{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"foo\", Labels: map[string]string{velerov1api.BackupNameLabel: \"non-matching-backup-1\"}},\n\t\t\t\t\tSpec: velerov1api.PodVolumeBackupSpec{\n\t\t\t\t\t\tPod: corev1api.ObjectReference{Name: \"pod-1\", Namespace: \"ns-1\"},\n\t\t\t\t\t},\n\t\t\t\t\tStatus: velerov1api.PodVolumeBackupStatus{SnapshotID: \"snap-1\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"bar\", Labels: map[string]string{velerov1api.BackupNameLabel: \"non-matching-backup-2\"}},\n\t\t\t\t\tSpec: velerov1api.PodVolumeBackupSpec{\n\t\t\t\t\t\tPod: corev1api.ObjectReference{Name: \"pod-2\", Namespace: \"ns-2\"},\n\t\t\t\t\t},\n\t\t\t\t\tStatus: velerov1api.PodVolumeBackupStatus{SnapshotID: \"snap-2\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"completed-pvb\", Labels: map[string]string{velerov1api.BackupNameLabel: \"the-really-long-backup-name-that-is-much-more-than-63-cha6ca4bc\"}},\n\t\t\t\t\tSpec: velerov1api.PodVolumeBackupSpec{\n\t\t\t\t\t\tPod: corev1api.ObjectReference{Name: \"pod-1\", Namespace: \"ns-1\"},\n\t\t\t\t\t},\n\t\t\t\t\tStatus: velerov1api.PodVolumeBackupStatus{SnapshotID: \"snap-3\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"completed-pvb-2\", Labels: map[string]string{velerov1api.BackupNameLabel: \"backup-1\"}},\n\t\t\t\t\tSpec: velerov1api.PodVolumeBackupSpec{\n\t\t\t\t\t\tPod: corev1api.ObjectReference{Name: \"pod-1\", Namespace: \"ns-1\"},\n\t\t\t\t\t},\n\t\t\t\t\tStatus: velerov1api.PodVolumeBackupStatus{SnapshotID: \"snap-4\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"incomplete-or-failed-pvb\", Labels: map[string]string{velerov1api.BackupNameLabel: \"backup-1\"}},\n\t\t\t\t\tSpec: velerov1api.PodVolumeBackupSpec{\n\t\t\t\t\t\tPod: corev1api.ObjectReference{Name: \"pod-1\", Namespace: \"ns-2\"},\n\t\t\t\t\t},\n\t\t\t\t\tStatus: velerov1api.PodVolumeBackupStatus{SnapshotID: \"\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: map[string][]repotypes.SnapshotIdentifier{\n\t\t\t\t\"ns-1\": {\n\t\t\t\t\t{\n\t\t\t\t\t\tVolumeNamespace: \"ns-1\",\n\t\t\t\t\t\tSnapshotID: \"snap-3\",\n\t\t\t\t\t\tRepositoryType: \"restic\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\tclientBuilder = velerotest.NewFakeControllerRuntimeClientBuilder(t)\n\t\t\t\tveleroBackup = &velerov1api.Backup{}\n\t\t\t)\n\n\t\t\tveleroBackup.Name = \"backup-1\"\n\n\t\t\tif test.longBackupNameEnabled {\n\t\t\t\tveleroBackup.Name = \"the-really-long-backup-name-that-is-much-more-than-63-characters\"\n\t\t\t}\n\t\t\tclientBuilder.WithLists(&velerov1api.PodVolumeBackupList{\n\t\t\t\tItems: test.podVolumeBackups,\n\t\t\t})\n\n\t\t\tres, err := getSnapshotsInBackup(t.Context(), veleroBackup, clientBuilder.Build())\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.True(t, reflect.DeepEqual(res, test.expected))\n\t\t})\n\t}\n}\n\nfunc batchDeleteSucceed(ctx context.Context, repoEnsurer *repository.Ensurer, repoMgr repomanager.Manager, directSnapshots map[string][]repotypes.SnapshotIdentifier, backup *velerov1api.Backup, logger logrus.FieldLogger) []error {\n\treturn nil\n}\n\nfunc batchDeleteFail(ctx context.Context, repoEnsurer *repository.Ensurer, repoMgr repomanager.Manager, directSnapshots map[string][]repotypes.SnapshotIdentifier, backup *velerov1api.Backup, logger logrus.FieldLogger) []error {\n\treturn []error{\n\t\terrors.New(\"fake-delete-1\"),\n\t\terrors.New(\"fake-delete-2\"),\n\t}\n}\n\nfunc generateSnapshotData(snapshot *repotypes.SnapshotIdentifier) (map[string]string, error) {\n\tif snapshot == nil {\n\t\treturn nil, nil\n\t}\n\n\tb, err := json.Marshal(snapshot)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdata := make(map[string]string)\n\tif err := json.Unmarshal(b, &data); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn data, nil\n}\n\nfunc TestDeleteMovedSnapshots(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\trepoMgr repomanager.Manager\n\t\tbatchDeleteSucceed bool\n\t\tbackupName string\n\t\tsnapshots []*repotypes.SnapshotIdentifier\n\t\texpected []string\n\t}{\n\t\t{\n\t\t\tname: \"repoMgr is nil\",\n\t\t},\n\t\t{\n\t\t\tname: \"no cm\",\n\t\t\trepoMgr: repomocks.NewManager(t),\n\t\t},\n\t\t{\n\t\t\tname: \"bad cm info\",\n\t\t\trepoMgr: repomocks.NewManager(t),\n\t\t\tbackupName: \"backup-01\",\n\t\t\tsnapshots: []*repotypes.SnapshotIdentifier{nil},\n\t\t\texpected: []string{\"no snapshot info in config\"},\n\t\t},\n\t\t{\n\t\t\tname: \"invalid snapshots\",\n\t\t\trepoMgr: repomocks.NewManager(t),\n\t\t\tbackupName: \"backup-01\",\n\t\t\tsnapshots: []*repotypes.SnapshotIdentifier{\n\t\t\t\t{\n\t\t\t\t\tRepositoryType: \"repo-1\",\n\t\t\t\t\tVolumeNamespace: \"ns-1\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tSnapshotID: \"snapshot-1\",\n\t\t\t\t\tVolumeNamespace: \"ns-1\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tSnapshotID: \"snapshot-1\",\n\t\t\t\t\tRepositoryType: \"repo-1\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tbatchDeleteSucceed: true,\n\t\t\texpected: []string{\n\t\t\t\t\"invalid snapshot, ID , namespace ns-1, repository repo-1\",\n\t\t\t\t\"invalid snapshot, ID snapshot-1, namespace ns-1, repository \",\n\t\t\t\t\"invalid snapshot, ID snapshot-1, namespace , repository repo-1\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"batch delete succeed\",\n\t\t\trepoMgr: repomocks.NewManager(t),\n\t\t\tbackupName: \"backup-01\",\n\t\t\tsnapshots: []*repotypes.SnapshotIdentifier{\n\n\t\t\t\t{\n\t\t\t\t\tSnapshotID: \"snapshot-1\",\n\t\t\t\t\tRepositoryType: \"repo-1\",\n\t\t\t\t\tVolumeNamespace: \"ns-1\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tbatchDeleteSucceed: true,\n\t\t\texpected: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"batch delete fail\",\n\t\t\trepoMgr: repomocks.NewManager(t),\n\t\t\tbackupName: \"backup-01\",\n\t\t\tsnapshots: []*repotypes.SnapshotIdentifier{\n\t\t\t\t{\n\t\t\t\t\tRepositoryType: \"repo-1\",\n\t\t\t\t\tVolumeNamespace: \"ns-1\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tSnapshotID: \"snapshot-1\",\n\t\t\t\t\tRepositoryType: \"repo-1\",\n\t\t\t\t\tVolumeNamespace: \"ns-1\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: []string{\"invalid snapshot, ID , namespace ns-1, repository repo-1\", \"fake-delete-1\", \"fake-delete-2\"},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tobjs := []runtime.Object{}\n\t\t\tfor i, snapshot := range test.snapshots {\n\t\t\t\tsnapshotData, err := generateSnapshotData(snapshot)\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\tcm := corev1api.ConfigMap{\n\t\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\t\tAPIVersion: corev1api.SchemeGroupVersion.String(),\n\t\t\t\t\t\tKind: \"ConfigMap\",\n\t\t\t\t\t},\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\tName: fmt.Sprintf(\"du-info-%d\", i),\n\t\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\tvelerov1api.BackupNameLabel: test.backupName,\n\t\t\t\t\t\t\tvelerov1api.DataUploadSnapshotInfoLabel: \"true\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tData: snapshotData,\n\t\t\t\t}\n\n\t\t\t\tobjs = append(objs, &cm)\n\t\t\t}\n\n\t\t\tveleroBackup := &velerov1api.Backup{}\n\t\t\tcontroller := NewBackupDeletionReconciler(\n\t\t\t\tvelerotest.NewLogger(),\n\t\t\t\tvelerotest.NewFakeControllerRuntimeClient(t, objs...),\n\t\t\t\tNewBackupTracker(),\n\t\t\t\ttest.repoMgr,\n\t\t\t\tmetrics.NewServerMetrics(),\n\t\t\t\tnil, // discovery helper\n\t\t\t\tfunc(logrus.FieldLogger) clientmgmt.Manager { return pluginManager },\n\t\t\t\tNewFakeSingleObjectBackupStoreGetter(backupStore),\n\t\t\t\tvelerotest.NewFakeCredentialsFileStore(\"\", nil),\n\t\t\t\tnil,\n\t\t\t)\n\n\t\t\tveleroBackup.Name = test.backupName\n\n\t\t\tif test.batchDeleteSucceed {\n\t\t\t\tbatchDeleteSnapshotFunc = batchDeleteSucceed\n\t\t\t} else {\n\t\t\t\tbatchDeleteSnapshotFunc = batchDeleteFail\n\t\t\t}\n\n\t\t\terrs := controller.deleteMovedSnapshots(t.Context(), veleroBackup)\n\t\t\tif test.expected == nil {\n\t\t\t\tassert.Nil(t, errs)\n\t\t\t} else {\n\t\t\t\tassert.Len(t, errs, len(test.expected))\n\t\t\t\tfor i := range test.expected {\n\t\t\t\t\tassert.EqualError(t, errs[i], test.expected[i])\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/controller/backup_finalizer_controller.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"os\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\tclocks \"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tpkgbackup \"github.com/vmware-tanzu/velero/pkg/backup\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\t\"github.com/vmware-tanzu/velero/pkg/persistence\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/encode\"\n)\n\n// backupFinalizerReconciler reconciles a Backup object\ntype backupFinalizerReconciler struct {\n\tclient kbclient.Client\n\tglobalCRClient kbclient.Client\n\tclock clocks.WithTickerAndDelayedExecution\n\tbackupper pkgbackup.Backupper\n\tnewPluginManager func(logrus.FieldLogger) clientmgmt.Manager\n\tbackupTracker BackupTracker\n\tmetrics *metrics.ServerMetrics\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter\n\tlog logrus.FieldLogger\n\tresourceTimeout time.Duration\n}\n\n// NewBackupFinalizerReconciler initializes and returns backupFinalizerReconciler struct.\nfunc NewBackupFinalizerReconciler(\n\tclient kbclient.Client,\n\tglobalCRClient kbclient.Client,\n\tclock clocks.WithTickerAndDelayedExecution,\n\tbackupper pkgbackup.Backupper,\n\tnewPluginManager func(logrus.FieldLogger) clientmgmt.Manager,\n\tbackupTracker BackupTracker,\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter,\n\tlog logrus.FieldLogger,\n\tmetrics *metrics.ServerMetrics,\n\tresourceTimeout time.Duration,\n) *backupFinalizerReconciler {\n\treturn &backupFinalizerReconciler{\n\t\tclient: client,\n\t\tglobalCRClient: globalCRClient,\n\t\tclock: clock,\n\t\tbackupper: backupper,\n\t\tnewPluginManager: newPluginManager,\n\t\tbackupTracker: backupTracker,\n\t\tbackupStoreGetter: backupStoreGetter,\n\t\tlog: log,\n\t\tmetrics: metrics,\n\t}\n}\n\n// +kubebuilder:rbac:groups=velero.io,resources=backups,verbs=get;list;watch;create;update;patch;delete\n// +kubebuilder:rbac:groups=velero.io,resources=backups/status,verbs=get;update;patch\n\nfunc (r *backupFinalizerReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\tlog := r.log.WithFields(logrus.Fields{\n\t\t\"controller\": \"backup-finalizer\",\n\t\t\"backup\": req.NamespacedName,\n\t})\n\n\t// Fetch the Backup instance.\n\tlog.Debug(\"Getting Backup\")\n\tbackup := &velerov1api.Backup{}\n\tif err := r.client.Get(ctx, req.NamespacedName, backup); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.Debug(\"Unable to find Backup\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tlog.WithError(err).Error(\"Error getting Backup\")\n\t\treturn ctrl.Result{}, errors.WithStack(err)\n\t}\n\n\tswitch backup.Status.Phase {\n\tcase velerov1api.BackupPhaseFinalizing, velerov1api.BackupPhaseFinalizingPartiallyFailed:\n\t\t// only process backups finalizing after plugin operations are complete\n\tdefault:\n\t\tlog.Debug(\"Backup is not awaiting finalizing, skipping\")\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\toriginal := backup.DeepCopy()\n\tdefer func() {\n\t\tswitch backup.Status.Phase {\n\t\tcase\n\t\t\tvelerov1api.BackupPhaseCompleted,\n\t\t\tvelerov1api.BackupPhasePartiallyFailed,\n\t\t\tvelerov1api.BackupPhaseFailed,\n\t\t\tvelerov1api.BackupPhaseFailedValidation:\n\t\t\tr.backupTracker.Delete(backup.Namespace, backup.Name)\n\t\t}\n\t\t// Always attempt to Patch the backup object and status after each reconciliation.\n\t\t//\n\t\t// if this patch fails, there may not be another opportunity to update the backup object without external update event.\n\t\t// so we retry\n\t\t// This retries updating Finalzing/FinalizingPartiallyFailed to Completed/PartiallyFailed\n\t\tif err := client.RetryOnErrorMaxBackOff(r.resourceTimeout, func() error { return r.client.Patch(ctx, backup, kbclient.MergeFrom(original)) }); err != nil {\n\t\t\tlog.WithError(err).Error(\"Error updating backup\")\n\t\t\treturn\n\t\t}\n\t}()\n\n\tlocation := &velerov1api.BackupStorageLocation{}\n\tif err := r.client.Get(ctx, kbclient.ObjectKey{\n\t\tNamespace: backup.Namespace,\n\t\tName: backup.Spec.StorageLocation,\n\t}, location); err != nil {\n\t\treturn ctrl.Result{}, errors.WithStack(err)\n\t}\n\tpluginManager := r.newPluginManager(log)\n\tdefer pluginManager.CleanupClients()\n\n\tbackupStore, err := r.backupStoreGetter.Get(location, pluginManager, log)\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"Error getting a backup store\")\n\t\treturn ctrl.Result{}, errors.WithStack(err)\n\t}\n\n\t// Download item operations list and backup contents\n\toperations, err := backupStore.GetBackupItemOperations(backup.Name)\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"Error getting backup item operations\")\n\t\treturn ctrl.Result{}, errors.WithStack(err)\n\t}\n\n\tbackupRequest := &pkgbackup.Request{\n\t\tBackup: backup,\n\t\tStorageLocation: location,\n\t\tSkippedPVTracker: pkgbackup.NewSkipPVTracker(),\n\t\tBackedUpItems: pkgbackup.NewBackedUpItemsMap(),\n\t}\n\tvar outBackupFile *os.File\n\tif len(operations) > 0 {\n\t\tlog.Info(\"Setting up finalized backup temp file\")\n\t\tinBackupFile, err := downloadToTempFile(backup.Name, backupStore, log)\n\t\tif err != nil {\n\t\t\treturn ctrl.Result{}, errors.Wrap(err, \"error downloading backup\")\n\t\t}\n\t\tdefer closeAndRemoveFile(inBackupFile, log)\n\t\toutBackupFile, err = os.CreateTemp(\"\", \"\")\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Error(\"error creating temp file for backup\")\n\t\t\treturn ctrl.Result{}, errors.WithStack(err)\n\t\t}\n\t\tdefer closeAndRemoveFile(outBackupFile, log)\n\n\t\tlog.Info(\"Getting backup item actions\")\n\t\tactions, err := pluginManager.GetBackupItemActionsV2()\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Error(\"error getting Backup Item Actions\")\n\t\t\treturn ctrl.Result{}, errors.WithStack(err)\n\t\t}\n\t\tbackupItemActionsResolver := framework.NewBackupItemActionResolverV2(actions)\n\n\t\t// Call itemBackupper.BackupItem for the list of items updated by async operations\n\t\terr = r.backupper.FinalizeBackup(\n\t\t\tlog,\n\t\t\tbackupRequest,\n\t\t\tinBackupFile,\n\t\t\toutBackupFile,\n\t\t\tbackupItemActionsResolver,\n\t\t\toperations,\n\t\t\tbackupStore,\n\t\t)\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Error(\"error finalizing Backup\")\n\t\t\treturn ctrl.Result{}, errors.WithStack(err)\n\t\t}\n\t}\n\tbackupScheduleName := backupRequest.GetLabels()[velerov1api.ScheduleNameLabel]\n\tswitch backup.Status.Phase {\n\tcase velerov1api.BackupPhaseFinalizing:\n\t\tbackup.Status.Phase = velerov1api.BackupPhaseCompleted\n\t\tr.metrics.RegisterBackupSuccess(backupScheduleName)\n\t\tr.metrics.RegisterBackupLastStatus(backupScheduleName, metrics.BackupLastStatusSucc)\n\tcase velerov1api.BackupPhaseFinalizingPartiallyFailed:\n\t\tbackup.Status.Phase = velerov1api.BackupPhasePartiallyFailed\n\t\tr.metrics.RegisterBackupPartialFailure(backupScheduleName)\n\t\tr.metrics.RegisterBackupLastStatus(backupScheduleName, metrics.BackupLastStatusFailure)\n\t}\n\n\tbackup.Status.CompletionTimestamp = &metav1.Time{Time: r.clock.Now()}\n\tbackup.Status.CSIVolumeSnapshotsCompleted = updateCSIVolumeSnapshotsCompleted(operations)\n\n\trecordBackupMetrics(log, backup, outBackupFile, r.metrics, true)\n\n\t// update backup metadata in object store\n\tbackupJSON := new(bytes.Buffer)\n\tif err := encode.To(backup, \"json\", backupJSON); err != nil {\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error encoding backup json\")\n\t}\n\terr = backupStore.PutBackupMetadata(backup.Name, backupJSON)\n\tif err != nil {\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error uploading backup json\")\n\t}\n\tif len(operations) > 0 {\n\t\terr = backupStore.PutBackupContents(backup.Name, outBackupFile)\n\t\tif err != nil {\n\t\t\treturn ctrl.Result{}, errors.Wrap(err, \"error uploading backup final contents\")\n\t\t}\n\t}\n\treturn ctrl.Result{}, nil\n}\n\nfunc (r *backupFinalizerReconciler) SetupWithManager(mgr ctrl.Manager) error {\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\tFor(&velerov1api.Backup{}).\n\t\tNamed(constant.ControllerBackupFinalizer).\n\t\tComplete(r)\n}\n\n// updateCSIVolumeSnapshotsCompleted calculate the completed VS number according to\n// the backup's async operation list.\nfunc updateCSIVolumeSnapshotsCompleted(\n\toperations []*itemoperation.BackupOperation) int {\n\tcompletedNum := 0\n\n\tfor index := range operations {\n\t\tif operations[index].Spec.ResourceIdentifier.String() == kuberesource.VolumeSnapshots.String() &&\n\t\t\toperations[index].Status.Phase == itemoperation.OperationPhaseCompleted {\n\t\t\tcompletedNum++\n\t\t}\n\t}\n\n\treturn completedNum\n}\n" }, { "path": "pkg/controller/backup_finalizer_controller_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"bytes\"\n\t\"io\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\ttestclocks \"k8s.io/utils/clock/testing\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/features\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc mockBackupFinalizerReconciler(fakeClient kbclient.Client, fakeGlobalClient kbclient.Client, fakeClock *testclocks.FakeClock) (*backupFinalizerReconciler, *fakeBackupper) {\n\tbackupper := new(fakeBackupper)\n\treturn NewBackupFinalizerReconciler(\n\t\tfakeClient,\n\t\tfakeGlobalClient,\n\t\tfakeClock,\n\t\tbackupper,\n\t\tfunc(logrus.FieldLogger) clientmgmt.Manager { return pluginManager },\n\t\tNewBackupTracker(),\n\t\tNewFakeSingleObjectBackupStoreGetter(backupStore),\n\t\tlogrus.StandardLogger(),\n\t\tmetrics.NewServerMetrics(),\n\t\t10*time.Minute,\n\t), backupper\n}\nfunc TestBackupFinalizerReconcile(t *testing.T) {\n\tfakeClock := testclocks.NewFakeClock(time.Now())\n\tmetav1Now := metav1.NewTime(fakeClock.Now())\n\n\tdefaultBackupLocation := builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"default\").Result()\n\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\tbackupOperations []*itemoperation.BackupOperation\n\t\tbackupLocation *velerov1api.BackupStorageLocation\n\t\tenableCSI bool\n\t\texpectError bool\n\t\texpectPhase velerov1api.BackupPhase\n\t\texpectedCompletedVS int\n\t}{\n\t\t{\n\t\t\tname: \"Finalizing backup is completed\",\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-1\").\n\t\t\t\tStorageLocation(\"default\").\n\t\t\t\tObjectMeta(builder.WithUID(\"foo\")).\n\t\t\t\tStartTimestamp(fakeClock.Now()).\n\t\t\t\tPhase(velerov1api.BackupPhaseFinalizing).Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\texpectPhase: velerov1api.BackupPhaseCompleted,\n\t\t\tbackupOperations: []*itemoperation.BackupOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\t\t\tBackupName: \"backup-1\",\n\t\t\t\t\t\tBackupUID: \"foo\",\n\t\t\t\t\t\tBackupItemAction: \"foo\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPostOperationItems: []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tGroupResource: kuberesource.Secrets,\n\t\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\t\tName: \"secret-1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tOperationID: \"operation-1\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: itemoperation.OperationPhaseCompleted,\n\t\t\t\t\t\tCreated: &metav1Now,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"FinalizingPartiallyFailed backup is partially failed\",\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-2\").\n\t\t\t\tStorageLocation(\"default\").\n\t\t\t\tObjectMeta(builder.WithUID(\"foo\")).\n\t\t\t\tStartTimestamp(fakeClock.Now()).\n\t\t\t\tPhase(velerov1api.BackupPhaseFinalizingPartiallyFailed).Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\texpectPhase: velerov1api.BackupPhasePartiallyFailed,\n\t\t\tbackupOperations: []*itemoperation.BackupOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\t\t\tBackupName: \"backup-2\",\n\t\t\t\t\t\tBackupUID: \"foo\",\n\t\t\t\t\t\tBackupItemAction: \"foo\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\t\t\tNamespace: \"ns-2\",\n\t\t\t\t\t\t\tName: \"pod-2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPostOperationItems: []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tGroupResource: kuberesource.Secrets,\n\t\t\t\t\t\t\t\tNamespace: \"ns-2\",\n\t\t\t\t\t\t\t\tName: \"secret-2\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tOperationID: \"operation-2\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: itemoperation.OperationPhaseCompleted,\n\t\t\t\t\t\tCreated: &metav1Now,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Test calculate backup.Status.BackupItemOperationsCompleted\",\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-3\").\n\t\t\t\tStorageLocation(\"default\").\n\t\t\t\tObjectMeta(builder.WithUID(\"foo\")).\n\t\t\t\tStartTimestamp(fakeClock.Now()).\n\t\t\t\tWithStatus(velerov1api.BackupStatus{\n\t\t\t\t\tStartTimestamp: &metav1Now,\n\t\t\t\t\tCompletionTimestamp: &metav1Now,\n\t\t\t\t\tCSIVolumeSnapshotsAttempted: 1,\n\t\t\t\t\tPhase: velerov1api.BackupPhaseFinalizing,\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\tenableCSI: true,\n\t\t\texpectPhase: velerov1api.BackupPhaseCompleted,\n\t\t\texpectedCompletedVS: 1,\n\t\t\tbackupOperations: []*itemoperation.BackupOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\t\t\tBackupName: \"backup-3\",\n\t\t\t\t\t\tBackupUID: \"foo\",\n\t\t\t\t\t\tBackupItemAction: \"foo\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: kuberesource.VolumeSnapshots,\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"vs-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tPostOperationItems: []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tGroupResource: kuberesource.Secrets,\n\t\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\t\tName: \"secret-1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tOperationID: \"operation-3\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: itemoperation.OperationPhaseCompleted,\n\t\t\t\t\t\tCreated: &metav1Now,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tif test.backup == nil {\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tinitObjs := []runtime.Object{}\n\t\t\tinitObjs = append(initObjs, test.backup)\n\n\t\t\tif test.backupLocation != nil {\n\t\t\t\tinitObjs = append(initObjs, test.backupLocation)\n\t\t\t}\n\n\t\t\tif test.enableCSI {\n\t\t\t\tfeatures.Enable(velerov1api.CSIFeatureFlag)\n\t\t\t\tdefer features.Enable()\n\t\t\t}\n\n\t\t\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, initObjs...)\n\n\t\t\tfakeGlobalClient := velerotest.NewFakeControllerRuntimeClient(t, initObjs...)\n\n\t\t\treconciler, backupper := mockBackupFinalizerReconciler(fakeClient, fakeGlobalClient, fakeClock)\n\t\t\tpluginManager.On(\"CleanupClients\").Return(nil)\n\t\t\tbackupStore.On(\"GetBackupItemOperations\", test.backup.Name).Return(test.backupOperations, nil)\n\t\t\tbackupStore.On(\"GetBackupContents\", mock.Anything).Return(io.NopCloser(bytes.NewReader([]byte(\"hello world\"))), nil)\n\t\t\tbackupStore.On(\"PutBackupContents\", mock.Anything, mock.Anything).Return(nil)\n\t\t\tbackupStore.On(\"PutBackupMetadata\", mock.Anything, mock.Anything).Return(nil)\n\t\t\tbackupStore.On(\"GetBackupVolumeInfos\", mock.Anything).Return(nil, nil)\n\t\t\tbackupStore.On(\"PutBackupVolumeInfos\", mock.Anything, mock.Anything).Return(nil)\n\t\t\tpluginManager.On(\"GetBackupItemActionsV2\").Return(nil, nil)\n\t\t\tbackupper.On(\"FinalizeBackup\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, framework.BackupItemActionResolverV2{}, mock.Anything, mock.Anything).Return(nil)\n\t\t\t_, err := reconciler.Reconcile(t.Context(), ctrl.Request{NamespacedName: types.NamespacedName{Namespace: test.backup.Namespace, Name: test.backup.Name}})\n\t\t\tgotErr := err != nil\n\t\t\tassert.Equal(t, test.expectError, gotErr)\n\n\t\t\tbackupAfter := velerov1api.Backup{}\n\t\t\terr = fakeClient.Get(t.Context(), types.NamespacedName{\n\t\t\t\tNamespace: test.backup.Namespace,\n\t\t\t\tName: test.backup.Name,\n\t\t\t}, &backupAfter)\n\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Equal(t, test.expectPhase, backupAfter.Status.Phase)\n\t\t\tassert.Equal(t, test.expectedCompletedVS, backupAfter.Status.CSIVolumeSnapshotsCompleted)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/controller/backup_operations_controller.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\tv2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/backupitemaction/v2\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\tclocks \"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/builder\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/predicate\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperationmap\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\t\"github.com/vmware-tanzu/velero/pkg/persistence\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/encode\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nconst (\n\tdefaultBackupOperationsFrequency = 10 * time.Second\n)\n\ntype backupOperationsReconciler struct {\n\tclient.Client\n\tlogger logrus.FieldLogger\n\tclock clocks.WithTickerAndDelayedExecution\n\tfrequency time.Duration\n\titemOperationsMap *itemoperationmap.BackupItemOperationsMap\n\tnewPluginManager func(logger logrus.FieldLogger) clientmgmt.Manager\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter\n\tmetrics *metrics.ServerMetrics\n}\n\nfunc NewBackupOperationsReconciler(\n\tlogger logrus.FieldLogger,\n\tclient client.Client,\n\tfrequency time.Duration,\n\tnewPluginManager func(logrus.FieldLogger) clientmgmt.Manager,\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter,\n\tmetrics *metrics.ServerMetrics,\n\titemOperationsMap *itemoperationmap.BackupItemOperationsMap,\n) *backupOperationsReconciler {\n\tabor := &backupOperationsReconciler{\n\t\tClient: client,\n\t\tlogger: logger,\n\t\tclock: clocks.RealClock{},\n\t\tfrequency: frequency,\n\t\titemOperationsMap: itemOperationsMap,\n\t\tnewPluginManager: newPluginManager,\n\t\tbackupStoreGetter: backupStoreGetter,\n\t\tmetrics: metrics,\n\t}\n\tif abor.frequency <= 0 {\n\t\tabor.frequency = defaultBackupOperationsFrequency\n\t}\n\treturn abor\n}\n\nfunc (c *backupOperationsReconciler) SetupWithManager(mgr ctrl.Manager) error {\n\tgp := kube.NewGenericEventPredicate(func(object client.Object) bool {\n\t\tbackup := object.(*velerov1api.Backup)\n\t\treturn (backup.Status.Phase == velerov1api.BackupPhaseWaitingForPluginOperations ||\n\t\t\tbackup.Status.Phase == velerov1api.BackupPhaseWaitingForPluginOperationsPartiallyFailed)\n\t})\n\ts := kube.NewPeriodicalEnqueueSource(c.logger.WithField(\"controller\", constant.ControllerBackupOperations), mgr.GetClient(), &velerov1api.BackupList{}, c.frequency, kube.PeriodicalEnqueueSourceOption{\n\t\tPredicates: []predicate.Predicate{gp},\n\t})\n\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\tFor(&velerov1api.Backup{}, builder.WithPredicates(kube.FalsePredicate{})).\n\t\tWatchesRawSource(s).\n\t\tNamed(constant.ControllerBackupOperations).\n\t\tComplete(c)\n}\n\n// +kubebuilder:rbac:groups=velero.io,resources=backups,verbs=get;list;watch;update\n// +kubebuilder:rbac:groups=velero.io,resources=backups/status,verbs=get\n// +kubebuilder:rbac:groups=velero.io,resources=backupstoragelocations,verbs=get\n\nfunc (c *backupOperationsReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\tlog := c.logger.WithField(\"backup operations for backup\", req.String())\n\tlog.Debug(\"backupOperationsReconciler getting backup\")\n\n\toriginal := &velerov1api.Backup{}\n\tif err := c.Get(ctx, req.NamespacedName, original); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.WithError(err).Error(\"backup not found\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\treturn ctrl.Result{}, errors.Wrapf(err, \"error getting backup %s\", req.String())\n\t}\n\tbackup := original.DeepCopy()\n\tlog.Debugf(\"backup: %s\", backup.Name)\n\n\tlog = c.logger.WithFields(\n\t\tlogrus.Fields{\n\t\t\t\"backup\": req.String(),\n\t\t},\n\t)\n\n\tswitch backup.Status.Phase {\n\tcase velerov1api.BackupPhaseWaitingForPluginOperations, velerov1api.BackupPhaseWaitingForPluginOperationsPartiallyFailed:\n\t\t// only process backups waiting for plugin operations to complete\n\tdefault:\n\t\tlog.Debug(\"Backup has no ongoing plugin operations, skipping\")\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\tloc := &velerov1api.BackupStorageLocation{}\n\tif err := c.Get(ctx, client.ObjectKey{\n\t\tNamespace: req.Namespace,\n\t\tName: backup.Spec.StorageLocation,\n\t}, loc); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.Warnf(\"Cannot check progress on Backup operations because backup storage location %s does not exist; marking backup PartiallyFailed\", backup.Spec.StorageLocation)\n\t\t\tbackup.Status.Phase = velerov1api.BackupPhasePartiallyFailed\n\t\t} else {\n\t\t\tlog.Warnf(\"Cannot check progress on Backup operations because backup storage location %s could not be retrieved: %s; marking backup PartiallyFailed\", backup.Spec.StorageLocation, err.Error())\n\t\t\tbackup.Status.Phase = velerov1api.BackupPhasePartiallyFailed\n\t\t}\n\t\terr2 := c.updateBackupAndOperationsJSON(ctx, original, backup, nil, &itemoperationmap.OperationsForBackup{ErrsSinceUpdate: []string{err.Error()}}, false, false)\n\t\tif err2 != nil {\n\t\t\tlog.WithError(err2).Error(\"error updating Backup\")\n\t\t}\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error getting backup storage location\")\n\t}\n\n\tif loc.Spec.AccessMode == velerov1api.BackupStorageLocationAccessModeReadOnly {\n\t\tlog.Infof(\"Cannot check progress on Backup operations because backup storage location %s is currently in read-only mode; marking backup PartiallyFailed\", loc.Name)\n\t\tbackup.Status.Phase = velerov1api.BackupPhasePartiallyFailed\n\n\t\terr := c.updateBackupAndOperationsJSON(ctx, original, backup, nil, &itemoperationmap.OperationsForBackup{ErrsSinceUpdate: []string{\"BSL is read-only\"}}, false, false)\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Error(\"error updating Backup\")\n\t\t}\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\tpluginManager := c.newPluginManager(c.logger)\n\tdefer pluginManager.CleanupClients()\n\tbackupStore, err := c.backupStoreGetter.Get(loc, pluginManager, c.logger)\n\tif err != nil {\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error getting backup store\")\n\t}\n\n\toperations, err := c.itemOperationsMap.GetOperationsForBackup(backupStore, backup.Name)\n\tif err != nil {\n\t\terr2 := c.updateBackupAndOperationsJSON(ctx, original, backup, backupStore, &itemoperationmap.OperationsForBackup{ErrsSinceUpdate: []string{err.Error()}}, false, false)\n\t\tif err2 != nil {\n\t\t\treturn ctrl.Result{}, errors.Wrap(err2, \"error updating Backup\")\n\t\t}\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error getting backup operations\")\n\t}\n\tstillInProgress, changes, opsCompleted, opsFailed, errs := getBackupItemOperationProgress(backup, pluginManager, operations.Operations)\n\t// if len(errs)>0, need to update backup errors and error log\n\toperations.ErrsSinceUpdate = append(operations.ErrsSinceUpdate, errs...)\n\tbackup.Status.Errors += len(operations.ErrsSinceUpdate)\n\tcompletionChanges := false\n\tif backup.Status.BackupItemOperationsCompleted != opsCompleted || backup.Status.BackupItemOperationsFailed != opsFailed {\n\t\tcompletionChanges = true\n\t\tbackup.Status.BackupItemOperationsCompleted = opsCompleted\n\t\tbackup.Status.BackupItemOperationsFailed = opsFailed\n\t}\n\tif changes {\n\t\toperations.ChangesSinceUpdate = true\n\t}\n\n\tif len(operations.ErrsSinceUpdate) > 0 {\n\t\tbackup.Status.Phase = velerov1api.BackupPhaseWaitingForPluginOperationsPartiallyFailed\n\t}\n\n\t// if stillInProgress is false, backup moves to finalize phase and needs update\n\t// if operations.ErrsSinceUpdate is not empty, then backup phase needs to change to\n\t// BackupPhaseWaitingForPluginOperationsPartiallyFailed and needs update\n\t// If the only changes are incremental progress, then no write is necessary, progress can remain in memory\n\tif !stillInProgress {\n\t\tif backup.Status.Phase == velerov1api.BackupPhaseWaitingForPluginOperations {\n\t\t\tlog.Infof(\"Marking backup %s Finalizing\", backup.Name)\n\t\t\tbackup.Status.Phase = velerov1api.BackupPhaseFinalizing\n\t\t} else {\n\t\t\tlog.Infof(\"Marking backup %s FinalizingPartiallyFailed\", backup.Name)\n\t\t\tbackup.Status.Phase = velerov1api.BackupPhaseFinalizingPartiallyFailed\n\t\t}\n\t}\n\terr = c.updateBackupAndOperationsJSON(ctx, original, backup, backupStore, operations, changes, completionChanges)\n\tif err != nil {\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error updating Backup\")\n\t}\n\treturn ctrl.Result{}, nil\n}\n\nfunc (c *backupOperationsReconciler) updateBackupAndOperationsJSON(\n\tctx context.Context,\n\toriginal, backup *velerov1api.Backup,\n\tbackupStore persistence.BackupStore,\n\toperations *itemoperationmap.OperationsForBackup,\n\tchanges bool,\n\tcompletionChanges bool) error {\n\tbackupScheduleName := backup.GetLabels()[velerov1api.ScheduleNameLabel]\n\n\tif len(operations.ErrsSinceUpdate) > 0 {\n\t\tc.metrics.RegisterBackupItemsErrorsGauge(backupScheduleName, backup.Status.Errors)\n\t\t// FIXME: download/upload results once https://github.com/vmware-tanzu/velero/pull/5576 is merged\n\t}\n\tremoveIfComplete := true\n\tdefer func() {\n\t\t// remove local operations list if complete\n\t\tif removeIfComplete && (backup.Status.Phase == velerov1api.BackupPhaseCompleted ||\n\t\t\tbackup.Status.Phase == velerov1api.BackupPhasePartiallyFailed ||\n\t\t\tbackup.Status.Phase == velerov1api.BackupPhaseFinalizing ||\n\t\t\tbackup.Status.Phase == velerov1api.BackupPhaseFinalizingPartiallyFailed) {\n\t\t\tc.itemOperationsMap.DeleteOperationsForBackup(backup.Name)\n\t\t} else if changes {\n\t\t\tc.itemOperationsMap.PutOperationsForBackup(operations, backup.Name)\n\t\t}\n\t}()\n\n\t// update backup and upload progress if errs or complete\n\tif len(operations.ErrsSinceUpdate) > 0 ||\n\t\tbackup.Status.Phase == velerov1api.BackupPhaseCompleted ||\n\t\tbackup.Status.Phase == velerov1api.BackupPhasePartiallyFailed ||\n\t\tbackup.Status.Phase == velerov1api.BackupPhaseFinalizing ||\n\t\tbackup.Status.Phase == velerov1api.BackupPhaseFinalizingPartiallyFailed {\n\t\t// update file store\n\t\tif backupStore != nil {\n\t\t\tbackupJSON := new(bytes.Buffer)\n\t\t\tif err := encode.To(backup, \"json\", backupJSON); err != nil {\n\t\t\t\tremoveIfComplete = false\n\t\t\t\treturn errors.Wrap(err, \"error encoding backup json\")\n\t\t\t}\n\t\t\terr := backupStore.PutBackupMetadata(backup.Name, backupJSON)\n\t\t\tif err != nil {\n\t\t\t\tremoveIfComplete = false\n\t\t\t\treturn errors.Wrap(err, \"error uploading backup json\")\n\t\t\t}\n\t\t\tif err := c.itemOperationsMap.UploadProgressAndPutOperationsForBackup(backupStore, operations, backup.Name); err != nil {\n\t\t\t\tremoveIfComplete = false\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t\t// update backup\n\t\terr := c.Client.Patch(ctx, backup, client.MergeFrom(original))\n\t\tif err != nil {\n\t\t\tremoveIfComplete = false\n\t\t\treturn errors.Wrapf(err, \"error updating Backup %s\", backup.Name)\n\t\t}\n\t} else if completionChanges {\n\t\t// If backup is still incomplete and no new errors are found but there are some new operations\n\t\t// completed, patch backup to reflect new completion numbers, but don't upload detailed json file\n\t\terr := c.Client.Patch(ctx, backup, client.MergeFrom(original))\n\t\tif err != nil {\n\t\t\treturn errors.Wrapf(err, \"error updating Backup %s\", backup.Name)\n\t\t}\n\t}\n\treturn nil\n}\n\n// check progress of backupItemOperations\n// return: inProgressOperations, changes, completedCount, failedCount, errs\nfunc getBackupItemOperationProgress(\n\tbackup *velerov1api.Backup,\n\tpluginManager clientmgmt.Manager,\n\toperationsList []*itemoperation.BackupOperation) (bool, bool, int, int, []string) {\n\tinProgressOperations := false\n\tchanges := false\n\tvar errs []string\n\tvar completedCount, failedCount int\n\n\tfor _, operation := range operationsList {\n\t\tif operation.Status.Phase == itemoperation.OperationPhaseNew ||\n\t\t\toperation.Status.Phase == itemoperation.OperationPhaseInProgress {\n\t\t\tbia, err := pluginManager.GetBackupItemActionV2(operation.Spec.BackupItemAction)\n\t\t\tif err != nil {\n\t\t\t\toperation.Status.Phase = itemoperation.OperationPhaseFailed\n\t\t\t\toperation.Status.Error = err.Error()\n\t\t\t\terrs = append(errs, wrapErrMsg(err.Error(), bia))\n\t\t\t\tchanges = true\n\t\t\t\tfailedCount++\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\toperationProgress, err := bia.Progress(operation.Spec.OperationID, backup)\n\t\t\tif err != nil {\n\t\t\t\toperation.Status.Phase = itemoperation.OperationPhaseFailed\n\t\t\t\toperation.Status.Error = err.Error()\n\t\t\t\terrs = append(errs, wrapErrMsg(err.Error(), bia))\n\t\t\t\tchanges = true\n\t\t\t\tfailedCount++\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif operation.Status.NCompleted != operationProgress.NCompleted {\n\t\t\t\toperation.Status.NCompleted = operationProgress.NCompleted\n\t\t\t\tchanges = true\n\t\t\t}\n\t\t\tif operation.Status.NTotal != operationProgress.NTotal {\n\t\t\t\toperation.Status.NTotal = operationProgress.NTotal\n\t\t\t\tchanges = true\n\t\t\t}\n\t\t\tif operation.Status.OperationUnits != operationProgress.OperationUnits {\n\t\t\t\toperation.Status.OperationUnits = operationProgress.OperationUnits\n\t\t\t\tchanges = true\n\t\t\t}\n\t\t\tif operation.Status.Description != operationProgress.Description {\n\t\t\t\toperation.Status.Description = operationProgress.Description\n\t\t\t\tchanges = true\n\t\t\t}\n\t\t\tstarted := metav1.NewTime(operationProgress.Started)\n\t\t\tif operation.Status.Started == nil && !operationProgress.Started.IsZero() ||\n\t\t\t\toperation.Status.Started != nil && *(operation.Status.Started) != started {\n\t\t\t\toperation.Status.Started = &started\n\t\t\t\tchanges = true\n\t\t\t}\n\t\t\tupdated := metav1.NewTime(operationProgress.Updated)\n\t\t\tif operation.Status.Updated == nil && !operationProgress.Updated.IsZero() ||\n\t\t\t\toperation.Status.Updated != nil && *(operation.Status.Updated) != updated {\n\t\t\t\toperation.Status.Updated = &updated\n\t\t\t\tchanges = true\n\t\t\t}\n\n\t\t\tif operationProgress.Completed {\n\t\t\t\tif operationProgress.Err != \"\" {\n\t\t\t\t\toperation.Status.Phase = itemoperation.OperationPhaseFailed\n\t\t\t\t\toperation.Status.Error = operationProgress.Err\n\t\t\t\t\terrs = append(errs, wrapErrMsg(operationProgress.Err, bia))\n\t\t\t\t\tchanges = true\n\t\t\t\t\tfailedCount++\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\toperation.Status.Phase = itemoperation.OperationPhaseCompleted\n\t\t\t\tchanges = true\n\t\t\t\tcompletedCount++\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\t// cancel operation if past timeout period\n\t\t\tif operation.Status.Created.Time.Add(backup.Spec.ItemOperationTimeout.Duration).Before(time.Now()) {\n\t\t\t\t_ = bia.Cancel(operation.Spec.OperationID, backup)\n\t\t\t\toperation.Status.Phase = itemoperation.OperationPhaseFailed\n\t\t\t\toperation.Status.Error = \"Asynchronous action timed out\"\n\t\t\t\terrs = append(errs, wrapErrMsg(operation.Status.Error, bia))\n\t\t\t\tchanges = true\n\t\t\t\tfailedCount++\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif operation.Status.Phase == itemoperation.OperationPhaseNew &&\n\t\t\t\toperation.Status.Started != nil {\n\t\t\t\toperation.Status.Phase = itemoperation.OperationPhaseInProgress\n\t\t\t\tchanges = true\n\t\t\t}\n\t\t\t// if we reach this point, the operation is still running\n\t\t\tinProgressOperations = true\n\t\t} else if operation.Status.Phase == itemoperation.OperationPhaseCompleted {\n\t\t\tcompletedCount++\n\t\t} else if operation.Status.Phase == itemoperation.OperationPhaseFailed {\n\t\t\tfailedCount++\n\t\t}\n\t}\n\treturn inProgressOperations, changes, completedCount, failedCount, errs\n}\n\n// wrap the error message to include the BIA name\nfunc wrapErrMsg(errMsg string, bia v2.BackupItemAction) string {\n\tplugin := \"unknown\"\n\tif bia != nil {\n\t\tplugin = bia.Name()\n\t}\n\tif len(errMsg) > 0 {\n\t\terrMsg += \", \"\n\t}\n\treturn fmt.Sprintf(\"%splugin: %s\", errMsg, plugin)\n}\n" }, { "path": "pkg/controller/backup_operations_controller_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\tv2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/backupitemaction/v2\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\ttestclocks \"k8s.io/utils/clock/testing\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperationmap\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\tpersistencemocks \"github.com/vmware-tanzu/velero/pkg/persistence/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\tpluginmocks \"github.com/vmware-tanzu/velero/pkg/plugin/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tbiav2mocks \"github.com/vmware-tanzu/velero/pkg/plugin/velero/mocks/backupitemaction/v2\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nvar (\n\tpluginManager = &pluginmocks.Manager{}\n\tbackupStore = &persistencemocks.BackupStore{}\n\tbia = &biav2mocks.BackupItemAction{}\n)\n\nfunc mockBackupOperationsReconciler(fakeClient kbclient.Client, fakeClock *testclocks.FakeClock, freq time.Duration) *backupOperationsReconciler {\n\tabor := NewBackupOperationsReconciler(\n\t\tlogrus.StandardLogger(),\n\t\tfakeClient,\n\t\tfreq,\n\t\tfunc(logrus.FieldLogger) clientmgmt.Manager { return pluginManager },\n\t\tNewFakeSingleObjectBackupStoreGetter(backupStore),\n\t\tmetrics.NewServerMetrics(),\n\t\titemoperationmap.NewBackupItemOperationsMap(),\n\t)\n\tabor.clock = fakeClock\n\treturn abor\n}\n\nfunc TestBackupOperationsReconcile(t *testing.T) {\n\tfakeClock := testclocks.NewFakeClock(time.Now())\n\tmetav1Now := metav1.NewTime(fakeClock.Now())\n\n\tdefaultBackupLocation := builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"default\").Result()\n\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\tbackupOperations []*itemoperation.BackupOperation\n\t\tbackupLocation *velerov1api.BackupStorageLocation\n\t\toperationComplete bool\n\t\toperationErr string\n\t\texpectError bool\n\t\texpectPhase velerov1api.BackupPhase\n\t}{\n\t\t{\n\t\t\tname: \"WaitingForPluginOperations backup with completed operations is Finalizing\",\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-11\").\n\t\t\t\tStorageLocation(\"default\").\n\t\t\t\tItemOperationTimeout(60 * time.Minute).\n\t\t\t\tObjectMeta(builder.WithUID(\"foo-11\")).\n\t\t\t\tPhase(velerov1api.BackupPhaseWaitingForPluginOperations).Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\toperationComplete: true,\n\t\t\texpectPhase: velerov1api.BackupPhaseFinalizing,\n\t\t\tbackupOperations: []*itemoperation.BackupOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\t\t\tBackupName: \"backup-11\",\n\t\t\t\t\t\tBackupUID: \"foo-11\",\n\t\t\t\t\t\tBackupItemAction: \"foo-11\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tOperationID: \"operation-11\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: itemoperation.OperationPhaseNew,\n\t\t\t\t\t\tCreated: &metav1Now,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"WaitingForPluginOperations backup with incomplete operations is still incomplete\",\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-12\").\n\t\t\t\tStorageLocation(\"default\").\n\t\t\t\tItemOperationTimeout(60 * time.Minute).\n\t\t\t\tObjectMeta(builder.WithUID(\"foo-12\")).\n\t\t\t\tPhase(velerov1api.BackupPhaseWaitingForPluginOperations).Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\toperationComplete: false,\n\t\t\texpectPhase: velerov1api.BackupPhaseWaitingForPluginOperations,\n\t\t\tbackupOperations: []*itemoperation.BackupOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\t\t\tBackupName: \"backup-12\",\n\t\t\t\t\t\tBackupUID: \"foo-12\",\n\t\t\t\t\t\tBackupItemAction: \"foo-12\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tOperationID: \"operation-12\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: itemoperation.OperationPhaseNew,\n\t\t\t\t\t\tCreated: &metav1Now,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"WaitingForPluginOperations backup with completed failed operations is FinalizingPartiallyFailed\",\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-13\").\n\t\t\t\tStorageLocation(\"default\").\n\t\t\t\tItemOperationTimeout(60 * time.Minute).\n\t\t\t\tObjectMeta(builder.WithUID(\"foo-13\")).\n\t\t\t\tPhase(velerov1api.BackupPhaseWaitingForPluginOperations).Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\toperationComplete: true,\n\t\t\toperationErr: \"failed\",\n\t\t\texpectPhase: velerov1api.BackupPhaseFinalizingPartiallyFailed,\n\t\t\tbackupOperations: []*itemoperation.BackupOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\t\t\tBackupName: \"backup-13\",\n\t\t\t\t\t\tBackupUID: \"foo-13\",\n\t\t\t\t\t\tBackupItemAction: \"foo-13\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tOperationID: \"operation-13\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: itemoperation.OperationPhaseNew,\n\t\t\t\t\t\tCreated: &metav1Now,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"WaitingForPluginOperationsPartiallyFailed backup with completed operations is FinalizingPartiallyFailed\",\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-14\").\n\t\t\t\tStorageLocation(\"default\").\n\t\t\t\tItemOperationTimeout(60 * time.Minute).\n\t\t\t\tObjectMeta(builder.WithUID(\"foo-14\")).\n\t\t\t\tPhase(velerov1api.BackupPhaseWaitingForPluginOperationsPartiallyFailed).Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\toperationComplete: true,\n\t\t\texpectPhase: velerov1api.BackupPhaseFinalizingPartiallyFailed,\n\t\t\tbackupOperations: []*itemoperation.BackupOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\t\t\tBackupName: \"backup-14\",\n\t\t\t\t\t\tBackupUID: \"foo-14\",\n\t\t\t\t\t\tBackupItemAction: \"foo-14\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tOperationID: \"operation-14\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: itemoperation.OperationPhaseNew,\n\t\t\t\t\t\tCreated: &metav1Now,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"WaitingForPluginOperationsPartiallyFailed backup with incomplete operations is still incomplete\",\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-15\").\n\t\t\t\tStorageLocation(\"default\").\n\t\t\t\tItemOperationTimeout(60 * time.Minute).\n\t\t\t\tObjectMeta(builder.WithUID(\"foo-15\")).\n\t\t\t\tPhase(velerov1api.BackupPhaseWaitingForPluginOperationsPartiallyFailed).Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\toperationComplete: false,\n\t\t\texpectPhase: velerov1api.BackupPhaseWaitingForPluginOperationsPartiallyFailed,\n\t\t\tbackupOperations: []*itemoperation.BackupOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\t\t\tBackupName: \"backup-15\",\n\t\t\t\t\t\tBackupUID: \"foo-15\",\n\t\t\t\t\t\tBackupItemAction: \"foo-15\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tOperationID: \"operation-15\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: itemoperation.OperationPhaseNew,\n\t\t\t\t\t\tCreated: &metav1Now,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"WaitingForPluginOperationsPartiallyFailed backup with completed failed operations is FinalizingPartiallyFailed\",\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-16\").\n\t\t\t\tStorageLocation(\"default\").\n\t\t\t\tItemOperationTimeout(60 * time.Minute).\n\t\t\t\tObjectMeta(builder.WithUID(\"foo-16\")).\n\t\t\t\tPhase(velerov1api.BackupPhaseWaitingForPluginOperationsPartiallyFailed).Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\toperationComplete: true,\n\t\t\toperationErr: \"failed\",\n\t\t\texpectPhase: velerov1api.BackupPhaseFinalizingPartiallyFailed,\n\t\t\tbackupOperations: []*itemoperation.BackupOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\t\t\tBackupName: \"backup-16\",\n\t\t\t\t\t\tBackupUID: \"foo-16\",\n\t\t\t\t\t\tBackupItemAction: \"foo-16\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tOperationID: \"operation-16\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: itemoperation.OperationPhaseNew,\n\t\t\t\t\t\tCreated: &metav1Now,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tif test.backup == nil {\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tinitObjs := []runtime.Object{}\n\t\t\tinitObjs = append(initObjs, test.backup)\n\n\t\t\tif test.backupLocation != nil {\n\t\t\t\tinitObjs = append(initObjs, test.backupLocation)\n\t\t\t}\n\n\t\t\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, initObjs...)\n\t\t\treconciler := mockBackupOperationsReconciler(fakeClient, fakeClock, defaultBackupOperationsFrequency)\n\t\t\tpluginManager.On(\"CleanupClients\").Return(nil)\n\t\t\tbackupStore.On(\"GetBackupItemOperations\", test.backup.Name).Return(test.backupOperations, nil)\n\t\t\tbackupStore.On(\"PutBackupItemOperations\", mock.Anything, mock.Anything).Return(nil)\n\t\t\tbackupStore.On(\"PutBackupMetadata\", mock.Anything, mock.Anything).Return(nil)\n\t\t\tfor _, operation := range test.backupOperations {\n\t\t\t\tbia.On(\"Name\").Return(\"test\")\n\t\t\t\tbia.On(\"Progress\", operation.Spec.OperationID, mock.Anything).\n\t\t\t\t\tReturn(velero.OperationProgress{\n\t\t\t\t\t\tCompleted: test.operationComplete,\n\t\t\t\t\t\tErr: test.operationErr,\n\t\t\t\t\t}, nil)\n\t\t\t\tpluginManager.On(\"GetBackupItemActionV2\", operation.Spec.BackupItemAction).Return(bia, nil)\n\t\t\t}\n\t\t\t_, err := reconciler.Reconcile(t.Context(), ctrl.Request{NamespacedName: types.NamespacedName{Namespace: test.backup.Namespace, Name: test.backup.Name}})\n\t\t\tgotErr := err != nil\n\t\t\tassert.Equal(t, test.expectError, gotErr)\n\n\t\t\tbackupAfter := velerov1api.Backup{}\n\t\t\terr = fakeClient.Get(t.Context(), types.NamespacedName{\n\t\t\t\tNamespace: test.backup.Namespace,\n\t\t\t\tName: test.backup.Name,\n\t\t\t}, &backupAfter)\n\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Equal(t, test.expectPhase, backupAfter.Status.Phase)\n\t\t})\n\t}\n}\n\nfunc TestWrapErrMsg(t *testing.T) {\n\tbia2 := &biav2mocks.BackupItemAction{}\n\tbia2.On(\"Name\").Return(\"test-bia\")\n\tcases := []struct {\n\t\tname string\n\t\tinputErr string\n\t\tplugin v2.BackupItemAction\n\t\texpect string\n\t}{\n\t\t{\n\t\t\tname: \"empty error message\",\n\t\t\tinputErr: \"\",\n\t\t\tplugin: bia2,\n\t\t\texpect: \"plugin: test-bia\",\n\t\t},\n\t\t{\n\t\t\tname: \"nil bia\",\n\t\t\tinputErr: \"some error happened\",\n\t\t\tplugin: nil,\n\t\t\texpect: \"some error happened, plugin: unknown\",\n\t\t},\n\t\t{\n\t\t\tname: \"regular error and bia\",\n\t\t\tinputErr: \"some error happened\",\n\t\t\tplugin: bia2,\n\t\t\texpect: \"some error happened, plugin: test-bia\",\n\t\t},\n\t}\n\n\tfor _, test := range cases {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tgot := wrapErrMsg(test.inputErr, test.plugin)\n\t\t\tassert.Equal(t, test.expect, got)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/controller/backup_queue_controller.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"slices\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/builder\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/event\"\n\t\"sigs.k8s.io/controller-runtime/pkg/handler\"\n\t\"sigs.k8s.io/controller-runtime/pkg/predicate\"\n\t\"sigs.k8s.io/controller-runtime/pkg/reconcile\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/collections\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\n// backupQueueReconciler reconciles a Backup object\ntype backupQueueReconciler struct {\n\tclient.Client\n\tScheme *runtime.Scheme\n\tlogger logrus.FieldLogger\n\tconcurrentBackups int\n\tbackupTracker BackupTracker\n\tfrequency time.Duration\n}\n\nconst (\n\tdefaultQueuedBackupRecheckFrequency = time.Minute\n)\n\n// NewBackupQueueReconciler returns a new backupQueueReconciler\nfunc NewBackupQueueReconciler(\n\tclient client.Client,\n\tscheme *runtime.Scheme,\n\tlogger logrus.FieldLogger,\n\tconcurrentBackups int,\n\tbackupTracker BackupTracker,\n) *backupQueueReconciler {\n\treturn &backupQueueReconciler{\n\t\tClient: client,\n\t\tScheme: scheme,\n\t\tlogger: logger,\n\t\tconcurrentBackups: max(concurrentBackups, 1),\n\t\tbackupTracker: backupTracker,\n\t\tfrequency: defaultQueuedBackupRecheckFrequency,\n\t}\n}\n\nfunc queuePositionOrderFunc(objList client.ObjectList) client.ObjectList {\n\tbackupList := objList.(*velerov1api.BackupList)\n\tslices.SortFunc(backupList.Items, func(backup1, backup2 velerov1api.Backup) int {\n\t\tif backup1.Status.QueuePosition < backup2.Status.QueuePosition {\n\t\t\treturn -1\n\t\t} else if backup1.Status.QueuePosition == backup2.Status.QueuePosition {\n\t\t\treturn 0\n\t\t} else {\n\t\t\treturn 1\n\t\t}\n\t})\n\treturn backupList\n}\n\n// SetupWithManager adds the reconciler to the manager\nfunc (r *backupQueueReconciler) SetupWithManager(mgr ctrl.Manager) error {\n\t// For periodic requeue, only consider Queued backups, order by QueuePosition\n\tgp := kube.NewGenericEventPredicate(func(object client.Object) bool {\n\t\tbackup := object.(*velerov1api.Backup)\n\t\treturn backup.Status.Phase == velerov1api.BackupPhaseQueued\n\t})\n\n\ts := kube.NewPeriodicalEnqueueSource(r.logger.WithField(\"controller\", constant.ControllerBackupQueue), mgr.GetClient(), &velerov1api.BackupList{}, r.frequency, kube.PeriodicalEnqueueSourceOption{\n\t\tPredicates: []predicate.Predicate{gp},\n\t\tOrderFunc: queuePositionOrderFunc,\n\t})\n\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\tFor(&velerov1api.Backup{}, builder.WithPredicates(predicate.Funcs{\n\t\t\tUpdateFunc: func(ue event.UpdateEvent) bool {\n\t\t\t\tbackup := ue.ObjectNew.(*velerov1api.Backup)\n\t\t\t\treturn backup.Status.Phase == \"\" || backup.Status.Phase == velerov1api.BackupPhaseNew\n\t\t\t},\n\t\t\tCreateFunc: func(ce event.CreateEvent) bool {\n\t\t\t\tbackup := ce.Object.(*velerov1api.Backup)\n\t\t\t\treturn backup.Status.Phase == \"\" || backup.Status.Phase == velerov1api.BackupPhaseNew\n\t\t\t},\n\t\t\tDeleteFunc: func(de event.DeleteEvent) bool {\n\t\t\t\treturn false\n\t\t\t},\n\t\t\tGenericFunc: func(ge event.GenericEvent) bool {\n\t\t\t\treturn false\n\t\t\t},\n\t\t})).\n\t\tWatches(\n\t\t\t&velerov1api.Backup{},\n\t\t\thandler.EnqueueRequestsFromMapFunc(r.findQueuedBackupsToRequeue),\n\t\t\tbuilder.WithPredicates(predicate.Funcs{\n\t\t\t\tUpdateFunc: func(ue event.UpdateEvent) bool {\n\t\t\t\t\toldBackup := ue.ObjectOld.(*velerov1api.Backup)\n\t\t\t\t\tnewBackup := ue.ObjectNew.(*velerov1api.Backup)\n\t\t\t\t\treturn oldBackup.Status.Phase == velerov1api.BackupPhaseInProgress &&\n\t\t\t\t\t\tnewBackup.Status.Phase != velerov1api.BackupPhaseInProgress ||\n\t\t\t\t\t\toldBackup.Status.Phase != velerov1api.BackupPhaseQueued &&\n\t\t\t\t\t\t\tnewBackup.Status.Phase == velerov1api.BackupPhaseQueued &&\n\t\t\t\t\t\t\tr.backupTracker.RunningCount() < r.concurrentBackups\n\t\t\t\t},\n\t\t\t\tCreateFunc: func(event.CreateEvent) bool {\n\t\t\t\t\treturn false\n\t\t\t\t},\n\t\t\t\tDeleteFunc: func(de event.DeleteEvent) bool {\n\t\t\t\t\treturn false\n\t\t\t\t},\n\t\t\t\tGenericFunc: func(ge event.GenericEvent) bool {\n\t\t\t\t\treturn false\n\t\t\t\t},\n\t\t\t})).\n\t\tWatchesRawSource(s).\n\t\tNamed(constant.ControllerBackupQueue).\n\t\tComplete(r)\n}\n\nfunc (r *backupQueueReconciler) detectNamespaceConflict(ctx context.Context, backup *velerov1api.Backup, earlierBackups []velerov1api.Backup) (bool, string, []string, error) {\n\tnsList := &corev1api.NamespaceList{}\n\tif err := r.Client.List(ctx, nsList); err != nil {\n\t\treturn false, \"\", nil, err\n\t}\n\tvar clusterNamespaces []string\n\tfor _, ns := range nsList.Items {\n\t\tclusterNamespaces = append(clusterNamespaces, ns.Name)\n\t}\n\tfoundConflict, conflictBackup := detectNSConflictsInternal(backup, earlierBackups, clusterNamespaces)\n\treturn foundConflict, conflictBackup, clusterNamespaces, nil\n}\n\nfunc detectNSConflictsInternal(backup *velerov1api.Backup, earlierBackups []velerov1api.Backup, clusterNamespaces []string) (bool, string) {\n\tbackupNamespaces := sets.NewString(namespacesForBackup(backup, clusterNamespaces)...)\n\tfor _, earlierBackup := range earlierBackups {\n\t\t// This will never be true for the primary backup, but for the secondary\n\t\t// runnability check for queued backups ahead of the current backup, we\n\t\t// only care about backups ahead of it.\n\t\t// Backup isn't earlier than this one, skip\n\t\tif earlierBackup.Status.Phase == velerov1api.BackupPhaseQueued &&\n\t\t\tearlierBackup.Status.QueuePosition >= backup.Status.QueuePosition {\n\t\t\tcontinue\n\t\t}\n\t\tif backupNamespaces.HasAny(namespacesForBackup(&earlierBackup, clusterNamespaces)...) {\n\t\t\treturn true, earlierBackup.Name\n\t\t}\n\t}\n\treturn false, \"\"\n}\n\n// Returns true if there are backups ahead of the current backup that are runnable\n// This could happen if velero just reconciled the one earlier in the queue and rejected it\n// due to too many running backups, but a backup completed in between that reconcile and this one\n// so exit, as the recent completion has triggered another reconcile of all queued backups\nfunc (r *backupQueueReconciler) checkForEarlierRunnableBackups(backup *velerov1api.Backup, earlierBackups []velerov1api.Backup, clusterNamespaces []string) (bool, string) {\n\tfor _, earlierBackup := range earlierBackups {\n\t\t// if this backup is queued and ahead of current backup, check for conflicts\n\t\tif earlierBackup.Status.Phase != velerov1api.BackupPhaseQueued ||\n\t\t\tearlierBackup.Status.QueuePosition >= backup.Status.QueuePosition {\n\t\t\tcontinue\n\t\t}\n\t\tconflict, _ := detectNSConflictsInternal(&earlierBackup, earlierBackups, clusterNamespaces)\n\t\t// !conflict means we've found an earlier backup that is currently runnable\n\t\t// so current reconcile should exit to run this one\n\t\tif !conflict {\n\t\t\treturn true, earlierBackup.Name\n\t\t}\n\t}\n\treturn false, \"\"\n}\n\nfunc namespacesForBackup(backup *velerov1api.Backup, clusterNamespaces []string) []string {\n\t// Ignore error here. If a backup has invalid namespace wildcards, the backup controller\n\t// will validate and fail it. Consider the ns list empty for conflict detection purposes.\n\tnsList, err := collections.NewNamespaceIncludesExcludes().Includes(backup.Spec.IncludedNamespaces...).Excludes(backup.Spec.ExcludedNamespaces...).ActiveNamespaces(clusterNamespaces).ResolveNamespaceList()\n\tif err != nil {\n\t\treturn []string{}\n\t}\n\treturn nsList\n}\nfunc (r *backupQueueReconciler) getMaxQueuePosition(lister *queuedBackupsLister) int {\n\tqueuedBackups := lister.orderedQueued()\n\tmaxPos := 0\n\tif len(queuedBackups) > 0 {\n\t\tmaxPos = queuedBackups[len(queuedBackups)-1].Status.QueuePosition\n\t}\n\treturn maxPos\n}\n\nfunc (r *backupQueueReconciler) findQueuedBackupsToRequeue(ctx context.Context, obj client.Object) []reconcile.Request {\n\tbackup := obj.(*velerov1api.Backup)\n\trequests := []reconcile.Request{}\n\tallBackups := &velerov1api.BackupList{}\n\tif err := r.Client.List(ctx, allBackups, &client.ListOptions{Namespace: backup.Namespace}); err != nil {\n\t\tr.logger.WithError(err).Error(\"error listing backups\")\n\t\treturn requests\n\t}\n\tbackups := r.newQueuedBackupsLister(allBackups).orderedQueued()\n\tfor _, item := range backups {\n\t\trequests = append(requests, reconcile.Request{\n\t\t\tNamespacedName: types.NamespacedName{\n\t\t\t\tNamespace: item.GetNamespace(),\n\t\t\t\tName: item.GetName(),\n\t\t\t},\n\t\t})\n\t}\n\treturn requests\n}\n\n// Reconcile reconciles a Backup object\nfunc (r *backupQueueReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\tlog := r.logger.WithField(\"backup\", req.NamespacedName.String())\n\n\tlog.Debug(\"Getting backup\")\n\tbackup := &velerov1api.Backup{}\n\tif err := r.Get(ctx, req.NamespacedName, backup); err != nil {\n\t\tlog.WithError(err).Error(\"unable to get backup\")\n\t\treturn ctrl.Result{}, client.IgnoreNotFound(err)\n\t}\n\tswitch backup.Status.Phase {\n\tcase \"\", velerov1api.BackupPhaseNew:\n\t\t// queue new backup\n\t\tallBackups := &velerov1api.BackupList{}\n\t\tif err := r.Client.List(ctx, allBackups, &client.ListOptions{Namespace: backup.Namespace}); err != nil {\n\t\t\tr.logger.WithError(err).Error(\"error listing backups\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\tlister := r.newQueuedBackupsLister(allBackups)\n\t\tmaxQueuePosition := r.getMaxQueuePosition(lister)\n\t\toriginal := backup.DeepCopy()\n\t\tbackup.Status.Phase = velerov1api.BackupPhaseQueued\n\t\tbackup.Status.QueuePosition = maxQueuePosition + 1\n\t\tlog.Infof(\"Queueing backup %v, queue position %v\", backup.Name, backup.Status.QueuePosition)\n\t\tif err := kube.PatchResource(original, backup, r.Client); err != nil {\n\t\t\treturn ctrl.Result{}, errors.Wrapf(err, \"error updating Backup status to %s\", backup.Status.Phase)\n\t\t}\n\tcase velerov1api.BackupPhaseQueued:\n\t\t// handle queued backup\n\t\t// Find backups ahead of this one (InProgress, ReadyToStart, or Queued with higher position)\n\t\tallBackups := &velerov1api.BackupList{}\n\t\tif err := r.Client.List(ctx, allBackups, &client.ListOptions{Namespace: backup.Namespace}); err != nil {\n\t\t\tr.logger.WithError(err).Error(\"error listing backups\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\tlister := r.newQueuedBackupsLister(allBackups)\n\t\tif r.backupTracker.RunningCount() >= r.concurrentBackups {\n\t\t\tlog.Debugf(\"%v concurrent backups are already running, leaving %v queued\", r.concurrentBackups, backup.Name)\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\tearlierBackups := lister.earlierThan(backup.Status.QueuePosition)\n\t\tfoundConflict, conflictBackup, clusterNamespaces, err := r.detectNamespaceConflict(ctx, backup, earlierBackups)\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Error(\"error listing namespaces\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\tif foundConflict {\n\t\t\tlog.Infof(\"Backup %v has namespace conflict with %v, leaving queued\", backup.Name, conflictBackup)\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\tfoundEarlierRunnable, earlierRunnable := r.checkForEarlierRunnableBackups(backup, earlierBackups, clusterNamespaces)\n\t\tif foundEarlierRunnable {\n\t\t\tlog.Infof(\"Earlier queued backup %v is runnable, leaving %v queued\", earlierRunnable, backup.Name)\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\tlog.Infof(\"Dequeueing backup %v, moving to ReadyToStart\", backup.Name)\n\t\toriginal := backup.DeepCopy()\n\t\tbackup.Status.Phase = velerov1api.BackupPhaseReadyToStart\n\t\tbackup.Status.QueuePosition = 0\n\t\tif err := kube.PatchResource(original, backup, r.Client); err != nil {\n\t\t\treturn ctrl.Result{}, errors.Wrapf(err, \"error updating Backup status to %s\", backup.Status.Phase)\n\t\t}\n\t\tr.backupTracker.AddReadyToStart(backup.Namespace, backup.Name)\n\t\tlog.Debug(\"Updating queuePosition for remaining queued backups\")\n\t\tqueuedBackups := lister.orderedQueued()\n\t\tnewQueuePos := 1\n\t\tfor _, queuedBackup := range queuedBackups {\n\t\t\tif queuedBackup.Name != backup.Name {\n\t\t\t\toriginal := queuedBackup.DeepCopy()\n\t\t\t\tqueuedBackup.Status.QueuePosition = newQueuePos\n\t\t\t\tif err := kube.PatchResource(original, &queuedBackup, r.Client); err != nil {\n\t\t\t\t\tlog.WithError(errors.Wrapf(err, \"error updating Backup %s queuePosition to %v\", queuedBackup.Name, newQueuePos))\n\t\t\t\t\treturn ctrl.Result{}, nil\n\t\t\t\t}\n\t\t\t\tnewQueuePos++\n\t\t\t}\n\t\t}\n\t\treturn ctrl.Result{}, nil\n\tdefault:\n\t\tlog.Debug(\"Backup is not New or Queued, skipping\")\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\treturn ctrl.Result{}, nil\n}\n\n// queuedBackupsLister manages a list of all backups Queued, ReadyToStart, or InProgress\n// with methods to return specific subsets as needed\ntype queuedBackupsLister struct {\n\tbackups *velerov1api.BackupList\n}\n\nfunc (r *backupQueueReconciler) newQueuedBackupsLister(backupList *velerov1api.BackupList) *queuedBackupsLister {\n\tbackups := []velerov1api.Backup{}\n\tfor _, backup := range backupList.Items {\n\t\tif backup.Status.Phase == velerov1api.BackupPhaseQueued ||\n\t\t\tbackup.Status.Phase == velerov1api.BackupPhaseInProgress ||\n\t\t\tbackup.Status.Phase == velerov1api.BackupPhaseReadyToStart {\n\t\t\tbackups = append(backups, backup)\n\t\t}\n\t}\n\tbackupList.Items = backups\n\treturn &queuedBackupsLister{backupList}\n}\n\nfunc (l *queuedBackupsLister) earlierThan(queuePos int) []velerov1api.Backup {\n\tbackups := []velerov1api.Backup{}\n\tfor _, backup := range l.backups.Items {\n\t\t// InProgress and ReadyToStart backups have QueuePosition==0\n\t\tif backup.Status.QueuePosition < queuePos {\n\t\t\tbackups = append(backups, backup)\n\t\t}\n\t}\n\treturn backups\n}\n\nfunc (l *queuedBackupsLister) orderedQueued() []velerov1api.Backup {\n\tvar returnList []velerov1api.Backup\n\torderedBackupList := queuePositionOrderFunc(l.backups).(*velerov1api.BackupList)\n\tfor _, item := range orderedBackupList.Items {\n\t\tif item.Status.Phase == velerov1api.BackupPhaseQueued {\n\t\t\treturnList = append(returnList, item)\n\t\t}\n\t}\n\treturn returnList\n}\n" }, { "path": "pkg/controller/backup_queue_controller_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\n\t//metav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\n\t//\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestBackupQueueReconciler(t *testing.T) {\n\tscheme := runtime.NewScheme()\n\tvelerov1api.AddToScheme(scheme)\n\n\ttests := []struct {\n\t\tname string\n\t\tpriorBackups []*velerov1api.Backup\n\t\tnamespaces []string\n\t\tbackup *velerov1api.Backup\n\t\tconcurrentBackups int\n\t\texpectError bool\n\t\texpectPhase velerov1api.BackupPhase\n\t\texpectQueuePosition int\n\t}{\n\t\t{\n\t\t\tname: \"New Backup gets queued\",\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-11\").Result(),\n\t\t\texpectPhase: velerov1api.BackupPhaseQueued,\n\t\t\texpectQueuePosition: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"InProgress Backup is ignored\",\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-11\").Phase(velerov1api.BackupPhaseInProgress).Result(),\n\t\t\texpectPhase: velerov1api.BackupPhaseInProgress,\n\t\t},\n\t\t{\n\t\t\tname: \"Second New Backup gets queued with queuePosition 2\",\n\t\t\tpriorBackups: []*velerov1api.Backup{\n\t\t\t\tbuilder.ForBackup(velerov1api.DefaultNamespace, \"backup-11\").Phase(velerov1api.BackupPhaseQueued).QueuePosition(1).Result(),\n\t\t\t},\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-12\").Result(),\n\t\t\texpectPhase: velerov1api.BackupPhaseQueued,\n\t\t\texpectQueuePosition: 2,\n\t\t},\n\t\t{\n\t\t\tname: \"Queued Backup moves to ReadyToStart if no others are running\",\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-11\").Phase(velerov1api.BackupPhaseQueued).Result(),\n\t\t\texpectPhase: velerov1api.BackupPhaseReadyToStart,\n\t\t},\n\t\t{\n\t\t\tname: \"Queued Backup remains queued if no spaces available\",\n\t\t\tpriorBackups: []*velerov1api.Backup{\n\t\t\t\tbuilder.ForBackup(velerov1api.DefaultNamespace, \"backup-11\").Phase(velerov1api.BackupPhaseInProgress).Result(),\n\t\t\t\tbuilder.ForBackup(velerov1api.DefaultNamespace, \"backup-12\").Phase(velerov1api.BackupPhaseInProgress).Result(),\n\t\t\t},\n\t\t\tconcurrentBackups: 2,\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-20\").Phase(velerov1api.BackupPhaseQueued).QueuePosition(1).Result(),\n\t\t\texpectPhase: velerov1api.BackupPhaseQueued,\n\t\t\texpectQueuePosition: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"Queued Backup remains queued if no spaces available including ReadyToStart\",\n\t\t\tpriorBackups: []*velerov1api.Backup{\n\t\t\t\tbuilder.ForBackup(velerov1api.DefaultNamespace, \"backup-11\").Phase(velerov1api.BackupPhaseInProgress).Result(),\n\t\t\t\tbuilder.ForBackup(velerov1api.DefaultNamespace, \"backup-12\").Phase(velerov1api.BackupPhaseReadyToStart).Result(),\n\t\t\t},\n\t\t\tconcurrentBackups: 2,\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-20\").Phase(velerov1api.BackupPhaseQueued).QueuePosition(1).Result(),\n\t\t\texpectPhase: velerov1api.BackupPhaseQueued,\n\t\t\texpectQueuePosition: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"Queued Backup remains queued if earlier runnable backup is also queued\",\n\t\t\tpriorBackups: []*velerov1api.Backup{\n\t\t\t\tbuilder.ForBackup(velerov1api.DefaultNamespace, \"backup-11\").Phase(velerov1api.BackupPhaseInProgress).Result(),\n\t\t\t\tbuilder.ForBackup(velerov1api.DefaultNamespace, \"backup-12\").Phase(velerov1api.BackupPhaseQueued).QueuePosition(1).Result(),\n\t\t\t},\n\t\t\tconcurrentBackups: 3,\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-20\").Phase(velerov1api.BackupPhaseQueued).QueuePosition(2).Result(),\n\t\t\texpectPhase: velerov1api.BackupPhaseQueued,\n\t\t\texpectQueuePosition: 2,\n\t\t},\n\t\t{\n\t\t\tname: \"Queued Backup remains queued if in conflict with running backup\",\n\t\t\tpriorBackups: []*velerov1api.Backup{\n\t\t\t\tbuilder.ForBackup(velerov1api.DefaultNamespace, \"backup-11\").Phase(velerov1api.BackupPhaseInProgress).IncludedNamespaces(\"foo\").Result(),\n\t\t\t},\n\t\t\tnamespaces: []string{\"foo\"},\n\t\t\tconcurrentBackups: 3,\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-20\").Phase(velerov1api.BackupPhaseQueued).QueuePosition(1).IncludedNamespaces(\"foo\").Result(),\n\t\t\texpectPhase: velerov1api.BackupPhaseQueued,\n\t\t\texpectQueuePosition: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"Queued Backup remains queued if in conflict with ReadyToStart backup\",\n\t\t\tpriorBackups: []*velerov1api.Backup{\n\t\t\t\tbuilder.ForBackup(velerov1api.DefaultNamespace, \"backup-11\").Phase(velerov1api.BackupPhaseReadyToStart).IncludedNamespaces(\"foo\").Result(),\n\t\t\t},\n\t\t\tnamespaces: []string{\"foo\"},\n\t\t\tconcurrentBackups: 3,\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-20\").Phase(velerov1api.BackupPhaseQueued).QueuePosition(1).IncludedNamespaces(\"foo\").Result(),\n\t\t\texpectPhase: velerov1api.BackupPhaseQueued,\n\t\t\texpectQueuePosition: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"Queued Backup remains queued if in conflict with earlier queued backup\",\n\t\t\tpriorBackups: []*velerov1api.Backup{\n\t\t\t\tbuilder.ForBackup(velerov1api.DefaultNamespace, \"backup-11\").Phase(velerov1api.BackupPhaseQueued).QueuePosition(1).IncludedNamespaces(\"foo\").Result(),\n\t\t\t},\n\t\t\tnamespaces: []string{\"foo\", \"bar\"},\n\t\t\tconcurrentBackups: 3,\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-20\").Phase(velerov1api.BackupPhaseQueued).QueuePosition(2).IncludedNamespaces(\"foo\", \"bar\").Result(),\n\t\t\texpectPhase: velerov1api.BackupPhaseQueued,\n\t\t\texpectQueuePosition: 2,\n\t\t},\n\t\t{\n\t\t\tname: \"Queued Backup remains queued if earlier non-ns-conflict backup exists\",\n\t\t\tpriorBackups: []*velerov1api.Backup{\n\t\t\t\tbuilder.ForBackup(velerov1api.DefaultNamespace, \"backup-11\").Phase(velerov1api.BackupPhaseInProgress).IncludedNamespaces(\"bar\").Result(),\n\t\t\t\tbuilder.ForBackup(velerov1api.DefaultNamespace, \"backup-12\").Phase(velerov1api.BackupPhaseQueued).QueuePosition(1).IncludedNamespaces(\"foo\").Result(),\n\t\t\t},\n\t\t\tnamespaces: []string{\"foo\", \"bar\", \"baz\"},\n\t\t\tconcurrentBackups: 3,\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-20\").Phase(velerov1api.BackupPhaseQueued).QueuePosition(2).IncludedNamespaces(\"baz\").Result(),\n\t\t\texpectPhase: velerov1api.BackupPhaseQueued,\n\t\t\texpectQueuePosition: 2,\n\t\t},\n\t\t{\n\t\t\tname: \"Running all-namespace backup conflicts with queued one-namespace backup \",\n\t\t\tpriorBackups: []*velerov1api.Backup{\n\t\t\t\tbuilder.ForBackup(velerov1api.DefaultNamespace, \"backup-11\").Phase(velerov1api.BackupPhaseInProgress).IncludedNamespaces(\"*\").Result(),\n\t\t\t},\n\t\t\tnamespaces: []string{\"foo\", \"bar\"},\n\t\t\tconcurrentBackups: 3,\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-20\").Phase(velerov1api.BackupPhaseQueued).QueuePosition(1).IncludedNamespaces(\"foo\").Result(),\n\t\t\texpectPhase: velerov1api.BackupPhaseQueued,\n\t\t\texpectQueuePosition: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"Running one-namespace backup conflicts with queued all-namespace backup \",\n\t\t\tpriorBackups: []*velerov1api.Backup{\n\t\t\t\tbuilder.ForBackup(velerov1api.DefaultNamespace, \"backup-11\").Phase(velerov1api.BackupPhaseInProgress).IncludedNamespaces(\"bar\").Result(),\n\t\t\t},\n\t\t\tnamespaces: []string{\"foo\", \"bar\"},\n\t\t\tconcurrentBackups: 3,\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-20\").Phase(velerov1api.BackupPhaseQueued).QueuePosition(1).IncludedNamespaces(\"*\").Result(),\n\t\t\texpectPhase: velerov1api.BackupPhaseQueued,\n\t\t\texpectQueuePosition: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"Queued Backup moves to ReadyToStart if running count < concurrentBackups\",\n\t\t\tpriorBackups: []*velerov1api.Backup{\n\t\t\t\tbuilder.ForBackup(velerov1api.DefaultNamespace, \"backup-11\").Phase(velerov1api.BackupPhaseInProgress).Result(),\n\t\t\t\tbuilder.ForBackup(velerov1api.DefaultNamespace, \"backup-12\").Phase(velerov1api.BackupPhaseInProgress).Result(),\n\t\t\t},\n\t\t\tconcurrentBackups: 3,\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-20\").Phase(velerov1api.BackupPhaseQueued).QueuePosition(1).Result(),\n\t\t\texpectPhase: velerov1api.BackupPhaseReadyToStart,\n\t\t},\n\t\t{\n\t\t\tname: \"Queued Backup moves to ReadyToStart if running count < concurrentBackups and no ns conflict found\",\n\t\t\tpriorBackups: []*velerov1api.Backup{\n\t\t\t\tbuilder.ForBackup(velerov1api.DefaultNamespace, \"backup-11\").Phase(velerov1api.BackupPhaseReadyToStart).IncludedNamespaces(\"foo\").Result(),\n\t\t\t},\n\t\t\tnamespaces: []string{\"foo\", \"bar\"},\n\t\t\tconcurrentBackups: 3,\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"backup-20\").Phase(velerov1api.BackupPhaseQueued).QueuePosition(1).IncludedNamespaces(\"bar\").Result(),\n\t\t\texpectPhase: velerov1api.BackupPhaseReadyToStart,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tif test.backup == nil {\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tbackupTracker := NewBackupTracker()\n\t\t\tinitObjs := []runtime.Object{}\n\t\t\tfor _, priorBackup := range test.priorBackups {\n\t\t\t\tinitObjs = append(initObjs, priorBackup)\n\t\t\t\tif priorBackup.Status.Phase == velerov1api.BackupPhaseReadyToStart {\n\t\t\t\t\tbackupTracker.AddReadyToStart(priorBackup.Namespace, priorBackup.Name)\n\t\t\t\t} else if priorBackup.Status.Phase == velerov1api.BackupPhaseInProgress {\n\t\t\t\t\tbackupTracker.Add(priorBackup.Namespace, priorBackup.Name)\n\t\t\t\t}\n\t\t\t}\n\t\t\tfor _, ns := range test.namespaces {\n\t\t\t\tinitObjs = append(initObjs, builder.ForNamespace(ns).Result())\n\t\t\t}\n\t\t\tinitObjs = append(initObjs, test.backup)\n\n\t\t\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, initObjs...)\n\t\t\tlogger := logrus.New()\n\t\t\tlog := logger.WithField(\"controller\", \"backup-queue-test\")\n\t\t\tr := NewBackupQueueReconciler(fakeClient, scheme, log, test.concurrentBackups, backupTracker)\n\t\t\treq := ctrl.Request{NamespacedName: types.NamespacedName{Namespace: test.backup.Namespace, Name: test.backup.Name}}\n\t\t\tres, err := r.Reconcile(t.Context(), req)\n\t\t\tgotErr := err != nil\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Equal(t, ctrl.Result{}, res)\n\t\t\tassert.Equal(t, test.expectError, gotErr)\n\t\t\tbackupAfter := velerov1api.Backup{}\n\t\t\terr = fakeClient.Get(t.Context(), types.NamespacedName{\n\t\t\t\tNamespace: test.backup.Namespace,\n\t\t\t\tName: test.backup.Name,\n\t\t\t}, &backupAfter)\n\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Equal(t, test.expectPhase, backupAfter.Status.Phase)\n\t\t\tassert.Equal(t, test.expectQueuePosition, backupAfter.Status.QueuePosition)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/controller/backup_repository_controller.go", "content": "/*\nCopyright 2018, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"reflect\"\n\t\"slices\"\n\t\"time\"\n\n\t\"github.com/petar/GoLLRB/llrb\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\tclocks \"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/builder\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/reconcile\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\trepoconfig \"github.com/vmware-tanzu/velero/pkg/repository/config\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/maintenance\"\n\trepomanager \"github.com/vmware-tanzu/velero/pkg/repository/manager\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n)\n\nconst (\n\trepoSyncPeriod = 5 * time.Minute\n\tdefaultMaintainFrequency = 7 * 24 * time.Hour\n\tdefaultMaintenanceStatusQueueLength = 3\n)\n\ntype BackupRepoReconciler struct {\n\tclient.Client\n\tnamespace string\n\tlogger logrus.FieldLogger\n\tclock clocks.WithTickerAndDelayedExecution\n\tmaintenanceFrequency time.Duration\n\tbackupRepoConfig string\n\trepositoryManager repomanager.Manager\n\trepoMaintenanceConfig string\n\tlogLevel logrus.Level\n\tlogFormat *logging.FormatFlag\n\tmetrics *metrics.ServerMetrics\n}\n\nfunc NewBackupRepoReconciler(\n\tnamespace string,\n\tlogger logrus.FieldLogger,\n\tclient client.Client,\n\trepositoryManager repomanager.Manager,\n\tmaintenanceFrequency time.Duration,\n\tbackupRepoConfig string,\n\trepoMaintenanceConfig string,\n\tlogLevel logrus.Level,\n\tlogFormat *logging.FormatFlag,\n\tmetrics *metrics.ServerMetrics,\n) *BackupRepoReconciler {\n\tc := &BackupRepoReconciler{\n\t\tclient,\n\t\tnamespace,\n\t\tlogger,\n\t\tclocks.RealClock{},\n\t\tmaintenanceFrequency,\n\t\tbackupRepoConfig,\n\t\trepositoryManager,\n\t\trepoMaintenanceConfig,\n\t\tlogLevel,\n\t\tlogFormat,\n\t\tmetrics,\n\t}\n\n\treturn c\n}\n\nfunc (r *BackupRepoReconciler) SetupWithManager(mgr ctrl.Manager) error {\n\ts := kube.NewPeriodicalEnqueueSource(\n\t\tr.logger.WithField(\"controller\", constant.ControllerBackupRepo),\n\t\tmgr.GetClient(),\n\t\t&velerov1api.BackupRepositoryList{},\n\t\trepoSyncPeriod,\n\t\tkube.PeriodicalEnqueueSourceOption{},\n\t)\n\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\tFor(&velerov1api.BackupRepository{}, builder.WithPredicates(kube.SpecChangePredicate{})).\n\t\tWatchesRawSource(s).\n\t\tWatches(\n\t\t\t// mark BackupRepository as invalid when BSL is created, updated or deleted.\n\t\t\t// BSL may be recreated after deleting, so also include the create event\n\t\t\t&velerov1api.BackupStorageLocation{},\n\t\t\tkube.EnqueueRequestsFromMapUpdateFunc(r.invalidateBackupReposForBSL),\n\t\t\tbuilder.WithPredicates(\n\t\t\t\t// Combine three predicates together to guarantee\n\t\t\t\t// only BSL's Delete Event and Update Event can enqueue.\n\t\t\t\t// We don't care about BSL's Generic Event and Create Event,\n\t\t\t\t// because BSL's periodical enqueue triggers Generic Event,\n\t\t\t\t// and the BackupRepository controller restart will triggers BSL create event.\n\t\t\t\tkube.NewUpdateEventPredicate(\n\t\t\t\t\tr.needInvalidBackupRepo,\n\t\t\t\t),\n\t\t\t\tkube.NewGenericEventPredicate(\n\t\t\t\t\tfunc(client.Object) bool { return false },\n\t\t\t\t),\n\t\t\t\tkube.NewCreateEventPredicate(\n\t\t\t\t\tfunc(client.Object) bool { return false },\n\t\t\t\t),\n\t\t\t),\n\t\t).\n\t\tComplete(r)\n}\n\nfunc (r *BackupRepoReconciler) invalidateBackupReposForBSL(ctx context.Context, bslObj client.Object) []reconcile.Request {\n\tbsl := bslObj.(*velerov1api.BackupStorageLocation)\n\n\tlist := &velerov1api.BackupRepositoryList{}\n\toptions := &client.ListOptions{\n\t\tLabelSelector: labels.Set(map[string]string{\n\t\t\tvelerov1api.StorageLocationLabel: label.GetValidName(bsl.Name),\n\t\t}).AsSelector(),\n\t}\n\tif err := r.List(context.TODO(), list, options); err != nil {\n\t\tr.logger.WithField(\"BSL\", bsl.Name).WithError(err).Error(\"unable to list BackupRepositories\")\n\t\treturn []reconcile.Request{}\n\t}\n\n\trequests := []reconcile.Request{}\n\tfor i := range list.Items {\n\t\tr.logger.WithField(\"BSL\", bsl.Name).Infof(\"Invalidating Backup Repository %s\", list.Items[i].Name)\n\t\tif err := r.patchBackupRepository(context.Background(), &list.Items[i], repoNotReady(\"re-establish on BSL change, create or delete\")); err != nil {\n\t\t\tr.logger.WithField(\"BSL\", bsl.Name).WithError(err).Errorf(\"fail to patch BackupRepository %s\", list.Items[i].Name)\n\t\t\tcontinue\n\t\t}\n\t\trequests = append(requests, reconcile.Request{NamespacedName: types.NamespacedName{Namespace: list.Items[i].Namespace, Name: list.Items[i].Name}})\n\t}\n\n\treturn requests\n}\n\n// needInvalidBackupRepo returns true if the BSL's storage type, bucket, prefix, CACert, or config has changed\nfunc (r *BackupRepoReconciler) needInvalidBackupRepo(oldObj client.Object, newObj client.Object) bool {\n\toldBSL := oldObj.(*velerov1api.BackupStorageLocation)\n\tnewBSL := newObj.(*velerov1api.BackupStorageLocation)\n\n\toldStorage := oldBSL.Spec.StorageType.ObjectStorage\n\tnewStorage := newBSL.Spec.StorageType.ObjectStorage\n\toldConfig := oldBSL.Spec.Config\n\tnewConfig := newBSL.Spec.Config\n\n\tif oldStorage == nil {\n\t\toldStorage = &velerov1api.ObjectStorageLocation{}\n\t}\n\n\tif newStorage == nil {\n\t\tnewStorage = &velerov1api.ObjectStorageLocation{}\n\t}\n\n\tlogger := r.logger.WithField(\"BSL\", newBSL.Name)\n\n\tif oldStorage.Bucket != newStorage.Bucket {\n\t\tlogger.WithFields(logrus.Fields{\n\t\t\t\"old bucket\": oldStorage.Bucket,\n\t\t\t\"new bucket\": newStorage.Bucket,\n\t\t}).Info(\"BSL's bucket has changed, invalid backup repositories\")\n\n\t\treturn true\n\t}\n\n\tif oldStorage.Prefix != newStorage.Prefix {\n\t\tlogger.WithFields(logrus.Fields{\n\t\t\t\"old prefix\": oldStorage.Prefix,\n\t\t\t\"new prefix\": newStorage.Prefix,\n\t\t}).Info(\"BSL's prefix has changed, invalid backup repositories\")\n\n\t\treturn true\n\t}\n\n\t// Check if either CACert or CACertRef has changed\n\tif !bytes.Equal(oldStorage.CACert, newStorage.CACert) {\n\t\tlogger.Info(\"BSL's CACert has changed, invalid backup repositories\")\n\t\treturn true\n\t}\n\n\t// Check if CACertRef has changed\n\tif (oldStorage.CACertRef == nil && newStorage.CACertRef != nil) ||\n\t\t(oldStorage.CACertRef != nil && newStorage.CACertRef == nil) ||\n\t\t(oldStorage.CACertRef != nil && newStorage.CACertRef != nil &&\n\t\t\t(oldStorage.CACertRef.Name != newStorage.CACertRef.Name ||\n\t\t\t\toldStorage.CACertRef.Key != newStorage.CACertRef.Key)) {\n\t\tlogger.Info(\"BSL's CACertRef has changed, invalid backup repositories\")\n\t\treturn true\n\t}\n\n\tif !reflect.DeepEqual(oldConfig, newConfig) {\n\t\tlogger.Info(\"BSL's storage config has changed, invalid backup repositories\")\n\n\t\treturn true\n\t}\n\n\treturn false\n}\n\nfunc (r *BackupRepoReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\tlog := r.logger.WithField(\"backupRepo\", req.String())\n\tbackupRepo := &velerov1api.BackupRepository{}\n\tif err := r.Get(ctx, req.NamespacedName, backupRepo); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.Warnf(\"backup repository %s in namespace %s is not found\", req.Name, req.Namespace)\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\tlog.WithError(err).Error(\"error getting backup repository\")\n\t\treturn ctrl.Result{}, err\n\t}\n\n\tbsl, bslErr := r.getBSL(ctx, backupRepo)\n\tif bslErr != nil {\n\t\tlog.WithError(bslErr).Error(\"Fail to get BSL for BackupRepository. Skip reconciling.\")\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\tif backupRepo.Status.Phase == \"\" || backupRepo.Status.Phase == velerov1api.BackupRepositoryPhaseNew {\n\t\tif err := r.initializeRepo(ctx, backupRepo, bsl, log); err != nil {\n\t\t\tlog.WithError(err).Error(\"error initialize repository\")\n\t\t\treturn ctrl.Result{}, errors.WithStack(err)\n\t\t}\n\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\t// If the repository is ready or not-ready, check it for stale locks, but if\n\t// this fails for any reason, it's non-critical so we still continue on to the\n\t// rest of the \"process\" logic.\n\tlog.Debug(\"Checking repository for stale locks\")\n\tif err := r.repositoryManager.UnlockRepo(backupRepo); err != nil {\n\t\tlog.WithError(err).Error(\"Error checking repository for stale locks\")\n\t}\n\n\tswitch backupRepo.Status.Phase {\n\tcase velerov1api.BackupRepositoryPhaseNotReady:\n\t\tready, err := r.checkNotReadyRepo(ctx, backupRepo, bsl, log)\n\t\tif err != nil {\n\t\t\treturn ctrl.Result{}, err\n\t\t} else if !ready {\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\tfallthrough\n\tcase velerov1api.BackupRepositoryPhaseReady:\n\t\tif bsl.Spec.AccessMode == velerov1api.BackupStorageLocationAccessModeReadOnly {\n\t\t\tlog.Debugf(\"Skip running maintenance for BackupRepository, because its BSL is in the ReadOnly mode.\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tif err := r.recallMaintenance(ctx, backupRepo, log); err != nil {\n\t\t\treturn ctrl.Result{}, errors.Wrap(err, \"error handling incomplete repo maintenance jobs\")\n\t\t}\n\n\t\tif err := r.runMaintenanceIfDue(ctx, backupRepo, log); err != nil {\n\t\t\treturn ctrl.Result{}, errors.Wrap(err, \"error check and run repo maintenance jobs\")\n\t\t}\n\n\t\t// Get the configured number of maintenance jobs to keep from ConfigMap\n\t\tkeepJobs, err := maintenance.GetKeepLatestMaintenanceJobs(ctx, r.Client, log, r.namespace, r.repoMaintenanceConfig, backupRepo)\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Warn(\"Failed to get keepLatestMaintenanceJobs from ConfigMap, using CLI parameter value\")\n\t\t}\n\n\t\tif err := maintenance.DeleteOldJobs(r.Client, *backupRepo, keepJobs, log); err != nil {\n\t\t\tlog.WithError(err).Warn(\"Failed to delete old maintenance jobs\")\n\t\t}\n\t}\n\n\treturn ctrl.Result{}, nil\n}\n\nfunc (r *BackupRepoReconciler) getBSL(ctx context.Context, req *velerov1api.BackupRepository) (*velerov1api.BackupStorageLocation, error) {\n\tloc := new(velerov1api.BackupStorageLocation)\n\n\tif err := r.Get(ctx, client.ObjectKey{\n\t\tNamespace: req.Namespace,\n\t\tName: req.Spec.BackupStorageLocation,\n\t}, loc); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn loc, nil\n}\n\nfunc (r *BackupRepoReconciler) getIdentifierByBSL(bsl *velerov1api.BackupStorageLocation, req *velerov1api.BackupRepository) (string, error) {\n\trepoIdentifier, err := repoconfig.GetRepoIdentifier(bsl, req.Spec.VolumeNamespace)\n\tif err != nil {\n\t\treturn \"\", errors.Wrapf(err, \"error to get identifier for repo %s\", req.Name)\n\t}\n\n\treturn repoIdentifier, nil\n}\n\nfunc (r *BackupRepoReconciler) initializeRepo(ctx context.Context, req *velerov1api.BackupRepository, bsl *velerov1api.BackupStorageLocation, log logrus.FieldLogger) error {\n\tlog.WithField(\"repoConfig\", r.backupRepoConfig).Info(\"Initializing backup repository\")\n\n\tvar repoIdentifier string\n\t// Only get restic identifier for restic repositories\n\tif req.Spec.RepositoryType == \"\" || req.Spec.RepositoryType == velerov1api.BackupRepositoryTypeRestic {\n\t\tvar err error\n\t\trepoIdentifier, err = r.getIdentifierByBSL(bsl, req)\n\t\tif err != nil {\n\t\t\treturn r.patchBackupRepository(ctx, req, func(rr *velerov1api.BackupRepository) {\n\t\t\t\trr.Status.Message = err.Error()\n\t\t\t\trr.Status.Phase = velerov1api.BackupRepositoryPhaseNotReady\n\n\t\t\t\tif rr.Spec.MaintenanceFrequency.Duration <= 0 {\n\t\t\t\t\trr.Spec.MaintenanceFrequency = metav1.Duration{Duration: r.getRepositoryMaintenanceFrequency(req)}\n\t\t\t\t}\n\t\t\t})\n\t\t}\n\t}\n\n\tconfig, err := getBackupRepositoryConfig(ctx, r, r.backupRepoConfig, r.namespace, req.Name, req.Spec.RepositoryType, log)\n\tif err != nil {\n\t\tlog.WithError(err).Warn(\"Failed to get repo config, repo config is ignored\")\n\t} else if config != nil {\n\t\tlog.Infof(\"Init repo with config %v\", config)\n\t}\n\n\t// defaulting - if the patch fails, return an error so the item is returned to the queue\n\tif err := r.patchBackupRepository(ctx, req, func(rr *velerov1api.BackupRepository) {\n\t\t// Only set ResticIdentifier for restic repositories\n\t\tif rr.Spec.RepositoryType == \"\" || rr.Spec.RepositoryType == velerov1api.BackupRepositoryTypeRestic {\n\t\t\trr.Spec.ResticIdentifier = repoIdentifier\n\t\t}\n\n\t\tif rr.Spec.MaintenanceFrequency.Duration <= 0 {\n\t\t\trr.Spec.MaintenanceFrequency = metav1.Duration{Duration: r.getRepositoryMaintenanceFrequency(req)}\n\t\t}\n\n\t\trr.Spec.RepositoryConfig = config\n\t}); err != nil {\n\t\treturn err\n\t}\n\n\tif err := ensureRepo(req, r.repositoryManager); err != nil {\n\t\treturn r.patchBackupRepository(ctx, req, repoNotReady(err.Error()))\n\t}\n\n\treturn r.patchBackupRepository(ctx, req, func(rr *velerov1api.BackupRepository) {\n\t\trr.Status.Phase = velerov1api.BackupRepositoryPhaseReady\n\t\trr.Status.LastMaintenanceTime = &metav1.Time{Time: time.Now()}\n\t})\n}\n\nfunc (r *BackupRepoReconciler) getRepositoryMaintenanceFrequency(req *velerov1api.BackupRepository) time.Duration {\n\tif r.maintenanceFrequency > 0 {\n\t\tr.logger.WithField(\"frequency\", r.maintenanceFrequency).Info(\"Set user defined maintenance frequency\")\n\t\treturn r.maintenanceFrequency\n\t}\n\n\tfrequency, err := r.repositoryManager.DefaultMaintenanceFrequency(req)\n\tif err != nil || frequency <= 0 {\n\t\tr.logger.WithError(err).WithField(\"returned frequency\", frequency).Warn(\"Failed to get maitanance frequency, use the default one\")\n\t\tfrequency = defaultMaintainFrequency\n\t} else {\n\t\tr.logger.WithField(\"frequency\", frequency).Info(\"Set maintenance according to repository suggestion\")\n\t}\n\n\treturn frequency\n}\n\n// ensureRepo calls repo manager's PrepareRepo to ensure the repo is ready for use.\n// An error is returned if the repository can't be connected to or initialized.\nfunc ensureRepo(repo *velerov1api.BackupRepository, repoManager repomanager.Manager) error {\n\treturn repoManager.PrepareRepo(repo)\n}\n\nfunc (r *BackupRepoReconciler) recallMaintenance(ctx context.Context, req *velerov1api.BackupRepository, log logrus.FieldLogger) error {\n\thistory, err := maintenance.WaitAllJobsComplete(ctx, r.Client, req, defaultMaintenanceStatusQueueLength, log)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error waiting incomplete repo maintenance job for repo %s\", req.Name)\n\t}\n\n\tconsolidated := consolidateHistory(history, req.Status.RecentMaintenance)\n\tif consolidated == nil {\n\t\treturn nil\n\t}\n\n\tlastMaintenanceTime := getLastMaintenanceTimeFromHistory(consolidated)\n\n\tlog.Warn(\"Updating backup repository because of unrecorded histories\")\n\n\treturn r.patchBackupRepository(ctx, req, func(rr *velerov1api.BackupRepository) {\n\t\tif lastMaintenanceTime != nil && (rr.Status.LastMaintenanceTime == nil || lastMaintenanceTime.After(rr.Status.LastMaintenanceTime.Time)) {\n\t\t\tif rr.Status.LastMaintenanceTime != nil {\n\t\t\t\tlog.Warnf(\"Updating backup repository last maintenance time (%v) from history (%v)\", rr.Status.LastMaintenanceTime.Time, lastMaintenanceTime.Time)\n\t\t\t} else {\n\t\t\t\tlog.Warnf(\"Setting backup repository last maintenance time from history (%v)\", lastMaintenanceTime.Time)\n\t\t\t}\n\t\t\trr.Status.LastMaintenanceTime = lastMaintenanceTime\n\t\t}\n\n\t\trr.Status.RecentMaintenance = consolidated\n\t})\n}\n\ntype maintenanceStatusWrapper struct {\n\tstatus *velerov1api.BackupRepositoryMaintenanceStatus\n}\n\nfunc (w maintenanceStatusWrapper) Less(other llrb.Item) bool {\n\treturn w.status.StartTimestamp.Before(other.(maintenanceStatusWrapper).status.StartTimestamp)\n}\n\nfunc consolidateHistory(coming, cur []velerov1api.BackupRepositoryMaintenanceStatus) []velerov1api.BackupRepositoryMaintenanceStatus {\n\tif len(coming) == 0 {\n\t\treturn nil\n\t}\n\n\tif slices.EqualFunc(cur, coming, func(a, b velerov1api.BackupRepositoryMaintenanceStatus) bool {\n\t\treturn a.StartTimestamp.Equal(b.StartTimestamp)\n\t}) {\n\t\treturn nil\n\t}\n\n\tconsolidator := llrb.New()\n\tfor i := range cur {\n\t\tconsolidator.ReplaceOrInsert(maintenanceStatusWrapper{&cur[i]})\n\t}\n\n\tfor i := range coming {\n\t\tconsolidator.ReplaceOrInsert(maintenanceStatusWrapper{&coming[i]})\n\t}\n\n\ttruncated := []velerov1api.BackupRepositoryMaintenanceStatus{}\n\tfor consolidator.Len() > 0 {\n\t\tif len(truncated) == defaultMaintenanceStatusQueueLength {\n\t\t\tbreak\n\t\t}\n\n\t\titem := consolidator.DeleteMax()\n\t\ttruncated = append(truncated, *item.(maintenanceStatusWrapper).status)\n\t}\n\n\tslices.Reverse(truncated)\n\n\tif slices.EqualFunc(cur, truncated, func(a, b velerov1api.BackupRepositoryMaintenanceStatus) bool {\n\t\treturn a.StartTimestamp.Equal(b.StartTimestamp)\n\t}) {\n\t\treturn nil\n\t}\n\n\treturn truncated\n}\n\nfunc getLastMaintenanceTimeFromHistory(history []velerov1api.BackupRepositoryMaintenanceStatus) *metav1.Time {\n\ttime := history[0].CompleteTimestamp\n\n\tfor i := range history {\n\t\tif history[i].CompleteTimestamp == nil {\n\t\t\tcontinue\n\t\t}\n\n\t\tif time == nil || time.Before(history[i].CompleteTimestamp) {\n\t\t\ttime = history[i].CompleteTimestamp\n\t\t}\n\t}\n\n\treturn time\n}\n\nvar funcStartMaintenanceJob = maintenance.StartNewJob\nvar funcWaitMaintenanceJobComplete = maintenance.WaitJobComplete\n\nfunc (r *BackupRepoReconciler) runMaintenanceIfDue(ctx context.Context, req *velerov1api.BackupRepository, log logrus.FieldLogger) error {\n\tstartTime := r.clock.Now()\n\n\tif !dueForMaintenance(req, startTime) {\n\t\tlog.Debug(\"not due for maintenance\")\n\t\treturn nil\n\t}\n\n\tlog.Info(\"Running maintenance on backup repository\")\n\n\tjob, err := funcStartMaintenanceJob(r.Client, ctx, req, r.repoMaintenanceConfig, r.logLevel, r.logFormat, log)\n\tif err != nil {\n\t\tlog.WithError(err).Warn(\"Starting repo maintenance failed\")\n\n\t\t// Record failure metric when job fails to start\n\t\tif r.metrics != nil {\n\t\t\tr.metrics.RegisterRepoMaintenanceFailure(req.Name)\n\t\t}\n\n\t\treturn r.patchBackupRepository(ctx, req, func(rr *velerov1api.BackupRepository) {\n\t\t\tupdateRepoMaintenanceHistory(rr, velerov1api.BackupRepositoryMaintenanceFailed, &metav1.Time{Time: startTime}, nil, fmt.Sprintf(\"Failed to start maintenance job, err: %v\", err))\n\t\t})\n\t}\n\n\t// when WaitMaintenanceJobComplete fails, the maintenance result will be left aside temporarily\n\t// If the maintenenance still completes later, recallMaintenance recalls the left once and update LastMaintenanceTime and history\n\tstatus, err := funcWaitMaintenanceJobComplete(r.Client, ctx, job, r.namespace, log)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error waiting repo maintenance completion status\")\n\t}\n\n\tif status.Result == velerov1api.BackupRepositoryMaintenanceFailed {\n\t\tlog.WithError(err).Warn(\"Pruning repository failed\")\n\n\t\t// Record failure metric\n\t\tif r.metrics != nil {\n\t\t\tr.metrics.RegisterRepoMaintenanceFailure(req.Name)\n\t\t\tif status.StartTimestamp != nil && status.CompleteTimestamp != nil {\n\t\t\t\tduration := status.CompleteTimestamp.Sub(status.StartTimestamp.Time).Seconds()\n\t\t\t\tr.metrics.ObserveRepoMaintenanceDuration(req.Name, duration)\n\t\t\t}\n\t\t}\n\n\t\treturn r.patchBackupRepository(ctx, req, func(rr *velerov1api.BackupRepository) {\n\t\t\tupdateRepoMaintenanceHistory(rr, velerov1api.BackupRepositoryMaintenanceFailed, status.StartTimestamp, status.CompleteTimestamp, status.Message)\n\t\t})\n\t}\n\n\t// Record success metric\n\tif r.metrics != nil {\n\t\tr.metrics.RegisterRepoMaintenanceSuccess(req.Name)\n\t\tif status.StartTimestamp != nil && status.CompleteTimestamp != nil {\n\t\t\tduration := status.CompleteTimestamp.Sub(status.StartTimestamp.Time).Seconds()\n\t\t\tr.metrics.ObserveRepoMaintenanceDuration(req.Name, duration)\n\t\t}\n\t}\n\n\treturn r.patchBackupRepository(ctx, req, func(rr *velerov1api.BackupRepository) {\n\t\trr.Status.LastMaintenanceTime = &metav1.Time{Time: status.CompleteTimestamp.Time}\n\t\tupdateRepoMaintenanceHistory(rr, velerov1api.BackupRepositoryMaintenanceSucceeded, status.StartTimestamp, status.CompleteTimestamp, status.Message)\n\t})\n}\n\nfunc updateRepoMaintenanceHistory(repo *velerov1api.BackupRepository, result velerov1api.BackupRepositoryMaintenanceResult, startTime, completionTime *metav1.Time, message string) {\n\tlatest := velerov1api.BackupRepositoryMaintenanceStatus{\n\t\tResult: result,\n\t\tStartTimestamp: startTime,\n\t\tCompleteTimestamp: completionTime,\n\t\tMessage: message,\n\t}\n\n\tstartingPos := 0\n\tif len(repo.Status.RecentMaintenance) >= defaultMaintenanceStatusQueueLength {\n\t\tstartingPos = len(repo.Status.RecentMaintenance) - defaultMaintenanceStatusQueueLength + 1\n\t}\n\n\trepo.Status.RecentMaintenance = append(repo.Status.RecentMaintenance[startingPos:], latest)\n}\n\nfunc dueForMaintenance(req *velerov1api.BackupRepository, now time.Time) bool {\n\treturn req.Status.LastMaintenanceTime == nil || req.Status.LastMaintenanceTime.Add(req.Spec.MaintenanceFrequency.Duration).Before(now)\n}\n\nfunc (r *BackupRepoReconciler) checkNotReadyRepo(ctx context.Context, req *velerov1api.BackupRepository, bsl *velerov1api.BackupStorageLocation, log logrus.FieldLogger) (bool, error) {\n\tlog.Info(\"Checking backup repository for readiness\")\n\n\t// Only check and update restic identifier for restic repositories\n\tif req.Spec.RepositoryType == \"\" || req.Spec.RepositoryType == velerov1api.BackupRepositoryTypeRestic {\n\t\trepoIdentifier, err := r.getIdentifierByBSL(bsl, req)\n\t\tif err != nil {\n\t\t\treturn false, r.patchBackupRepository(ctx, req, repoNotReady(err.Error()))\n\t\t}\n\n\t\tif repoIdentifier != req.Spec.ResticIdentifier {\n\t\t\tif err := r.patchBackupRepository(ctx, req, func(rr *velerov1api.BackupRepository) {\n\t\t\t\trr.Spec.ResticIdentifier = repoIdentifier\n\t\t\t}); err != nil {\n\t\t\t\treturn false, err\n\t\t\t}\n\t\t}\n\t}\n\n\t// we need to ensure it (first check, if check fails, attempt to init)\n\t// because we don't know if it's been successfully initialized yet.\n\tif err := ensureRepo(req, r.repositoryManager); err != nil {\n\t\treturn false, r.patchBackupRepository(ctx, req, repoNotReady(err.Error()))\n\t}\n\terr := r.patchBackupRepository(ctx, req, repoReady())\n\tif err != nil {\n\t\treturn false, err\n\t}\n\treturn true, nil\n}\n\nfunc repoNotReady(msg string) func(*velerov1api.BackupRepository) {\n\treturn func(r *velerov1api.BackupRepository) {\n\t\tr.Status.Phase = velerov1api.BackupRepositoryPhaseNotReady\n\t\tr.Status.Message = msg\n\t}\n}\n\nfunc repoReady() func(*velerov1api.BackupRepository) {\n\treturn func(r *velerov1api.BackupRepository) {\n\t\tr.Status.Phase = velerov1api.BackupRepositoryPhaseReady\n\t\tr.Status.Message = \"\"\n\t}\n}\n\n// patchBackupRepository mutates req with the provided mutate function, and patches it\n// through the Kube API. After executing this function, req will be updated with both\n// the mutation and the results of the Patch() API call.\nfunc (r *BackupRepoReconciler) patchBackupRepository(ctx context.Context, req *velerov1api.BackupRepository, mutate func(*velerov1api.BackupRepository)) error {\n\toriginal := req.DeepCopy()\n\tmutate(req)\n\tif err := r.Patch(ctx, req, client.MergeFrom(original)); err != nil {\n\t\treturn errors.Wrap(err, \"error patching BackupRepository\")\n\t}\n\treturn nil\n}\n\nfunc getBackupRepositoryConfig(ctx context.Context, ctrlClient client.Client, configName, namespace, repoName, repoType string, log logrus.FieldLogger) (map[string]string, error) {\n\tif configName == \"\" {\n\t\treturn nil, nil\n\t}\n\n\tloc := &corev1api.ConfigMap{}\n\tif err := ctrlClient.Get(ctx, client.ObjectKey{\n\t\tNamespace: namespace,\n\t\tName: configName,\n\t}, loc); err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error getting configMap %s\", configName)\n\t}\n\n\tjsonData, found := loc.Data[repoType]\n\tif !found {\n\t\tlog.Infof(\"No data for repo type %s in config map %s\", repoType, configName)\n\t\treturn nil, nil\n\t}\n\n\tvar unmarshalled map[string]any\n\tif err := json.Unmarshal([]byte(jsonData), &unmarshalled); err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error unmarshalling config data from %s for repo %s, repo type %s\", configName, repoName, repoType)\n\t}\n\n\tresult := map[string]string{}\n\tfor k, v := range unmarshalled {\n\t\tresult[k] = fmt.Sprintf(\"%v\", v)\n\t}\n\n\treturn result, nil\n}\n" }, { "path": "pkg/controller/backup_repository_controller_test.go", "content": "/*\nCopyright The Velero Contributors.\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n http://www.apache.org/licenses/LICENSE-2.0\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/prometheus/client_golang/prometheus\"\n\tdto \"github.com/prometheus/client_model/go\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/maintenance\"\n\trepomaintenance \"github.com/vmware-tanzu/velero/pkg/repository/maintenance\"\n\trepomanager \"github.com/vmware-tanzu/velero/pkg/repository/manager\"\n\trepomokes \"github.com/vmware-tanzu/velero/pkg/repository/mocks\"\n\trepotypes \"github.com/vmware-tanzu/velero/pkg/repository/types\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\tclientFake \"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tbatchv1api \"k8s.io/api/batch/v1\"\n)\n\nconst testMaintenanceFrequency = 10 * time.Minute\n\nfunc mockBackupRepoReconciler(t *testing.T, mockOn string, arg any, ret ...any) *BackupRepoReconciler {\n\tt.Helper()\n\tmgr := &repomokes.Manager{}\n\tif mockOn != \"\" {\n\t\tmgr.On(mockOn, arg).Return(ret...)\n\t}\n\treturn NewBackupRepoReconciler(\n\t\tvelerov1api.DefaultNamespace,\n\t\tvelerotest.NewLogger(),\n\t\tvelerotest.NewFakeControllerRuntimeClient(t),\n\t\tmgr,\n\t\ttestMaintenanceFrequency,\n\t\t\"fake-repo-config\",\n\t\t\"\",\n\t\tlogrus.InfoLevel,\n\t\tnil,\n\t\tnil,\n\t)\n}\n\nfunc mockBackupRepositoryCR() *velerov1api.BackupRepository {\n\treturn &velerov1api.BackupRepository{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: \"repo\",\n\t\t},\n\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\tMaintenanceFrequency: metav1.Duration{Duration: testMaintenanceFrequency},\n\t\t},\n\t}\n}\n\nfunc TestPatchBackupRepository(t *testing.T) {\n\trr := mockBackupRepositoryCR()\n\treconciler := mockBackupRepoReconciler(t, \"\", nil, nil)\n\terr := reconciler.Client.Create(t.Context(), rr)\n\trequire.NoError(t, err)\n\terr = reconciler.patchBackupRepository(t.Context(), rr, repoReady())\n\trequire.NoError(t, err)\n\tassert.Equal(t, velerov1api.BackupRepositoryPhaseReady, rr.Status.Phase)\n\terr = reconciler.patchBackupRepository(t.Context(), rr, repoNotReady(\"not ready\"))\n\trequire.NoError(t, err)\n\tassert.NotEqual(t, velerov1api.BackupRepositoryPhaseReady, rr.Status.Phase)\n}\n\nfunc TestCheckNotReadyRepo(t *testing.T) {\n\t// Test for restic repository\n\tt.Run(\"restic repository\", func(t *testing.T) {\n\t\trr := mockBackupRepositoryCR()\n\t\trr.Spec.BackupStorageLocation = \"default\"\n\t\trr.Spec.ResticIdentifier = \"fake-identifier\"\n\t\trr.Spec.VolumeNamespace = \"volume-ns-1\"\n\t\trr.Spec.RepositoryType = velerov1api.BackupRepositoryTypeRestic\n\t\treconciler := mockBackupRepoReconciler(t, \"PrepareRepo\", rr, nil)\n\t\terr := reconciler.Client.Create(t.Context(), rr)\n\t\trequire.NoError(t, err)\n\t\tlocation := velerov1api.BackupStorageLocation{\n\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\tConfig: map[string]string{\"resticRepoPrefix\": \"s3:test.amazonaws.com/bucket/restic\"},\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\tName: rr.Spec.BackupStorageLocation,\n\t\t\t},\n\t\t}\n\n\t\t_, err = reconciler.checkNotReadyRepo(t.Context(), rr, &location, reconciler.logger)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, velerov1api.BackupRepositoryPhaseReady, rr.Status.Phase)\n\t\tassert.Equal(t, \"s3:test.amazonaws.com/bucket/restic/volume-ns-1\", rr.Spec.ResticIdentifier)\n\t})\n\n\t// Test for kopia repository\n\tt.Run(\"kopia repository\", func(t *testing.T) {\n\t\trr := mockBackupRepositoryCR()\n\t\trr.Spec.BackupStorageLocation = \"default\"\n\t\trr.Spec.VolumeNamespace = \"volume-ns-1\"\n\t\trr.Spec.RepositoryType = velerov1api.BackupRepositoryTypeKopia\n\t\treconciler := mockBackupRepoReconciler(t, \"PrepareRepo\", rr, nil)\n\t\terr := reconciler.Client.Create(t.Context(), rr)\n\t\trequire.NoError(t, err)\n\t\tlocation := velerov1api.BackupStorageLocation{\n\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\tConfig: map[string]string{\"resticRepoPrefix\": \"s3:test.amazonaws.com/bucket/restic\"},\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\tName: rr.Spec.BackupStorageLocation,\n\t\t\t},\n\t\t}\n\n\t\t_, err = reconciler.checkNotReadyRepo(t.Context(), rr, &location, reconciler.logger)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, velerov1api.BackupRepositoryPhaseReady, rr.Status.Phase)\n\t\t// ResticIdentifier should remain empty for kopia\n\t\tassert.Empty(t, rr.Spec.ResticIdentifier)\n\t})\n\n\t// Test for empty repository type (defaults to restic)\n\tt.Run(\"empty repository type\", func(t *testing.T) {\n\t\trr := mockBackupRepositoryCR()\n\t\trr.Spec.BackupStorageLocation = \"default\"\n\t\trr.Spec.ResticIdentifier = \"fake-identifier\"\n\t\trr.Spec.VolumeNamespace = \"volume-ns-1\"\n\t\t// Deliberately leave RepositoryType empty\n\t\treconciler := mockBackupRepoReconciler(t, \"PrepareRepo\", rr, nil)\n\t\terr := reconciler.Client.Create(t.Context(), rr)\n\t\trequire.NoError(t, err)\n\t\tlocation := velerov1api.BackupStorageLocation{\n\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\tConfig: map[string]string{\"resticRepoPrefix\": \"s3:test.amazonaws.com/bucket/restic\"},\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\tName: rr.Spec.BackupStorageLocation,\n\t\t\t},\n\t\t}\n\n\t\t_, err = reconciler.checkNotReadyRepo(t.Context(), rr, &location, reconciler.logger)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, velerov1api.BackupRepositoryPhaseReady, rr.Status.Phase)\n\t\tassert.Equal(t, \"s3:test.amazonaws.com/bucket/restic/volume-ns-1\", rr.Spec.ResticIdentifier)\n\t})\n}\n\nfunc startMaintenanceJobFail(client.Client, context.Context, *velerov1api.BackupRepository, string, logrus.Level, *logging.FormatFlag, logrus.FieldLogger) (string, error) {\n\treturn \"\", errors.New(\"fake-start-error\")\n}\n\nfunc startMaintenanceJobSucceed(client.Client, context.Context, *velerov1api.BackupRepository, string, logrus.Level, *logging.FormatFlag, logrus.FieldLogger) (string, error) {\n\treturn \"fake-job-name\", nil\n}\n\nfunc waitMaintenanceJobCompleteFail(client.Client, context.Context, string, string, logrus.FieldLogger) (velerov1api.BackupRepositoryMaintenanceStatus, error) {\n\treturn velerov1api.BackupRepositoryMaintenanceStatus{}, errors.New(\"fake-wait-error\")\n}\n\nfunc waitMaintenanceJobCompleteFunc(now time.Time, result velerov1api.BackupRepositoryMaintenanceResult, message string) func(client.Client, context.Context, string, string, logrus.FieldLogger) (velerov1api.BackupRepositoryMaintenanceStatus, error) {\n\tcompletionTimeStamp := &metav1.Time{Time: now.Add(time.Hour)}\n\tif result == velerov1api.BackupRepositoryMaintenanceFailed {\n\t\tcompletionTimeStamp = nil\n\t}\n\n\treturn func(client.Client, context.Context, string, string, logrus.FieldLogger) (velerov1api.BackupRepositoryMaintenanceStatus, error) {\n\t\treturn velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\tStartTimestamp: &metav1.Time{Time: now},\n\t\t\tCompleteTimestamp: completionTimeStamp,\n\t\t\tResult: result,\n\t\t\tMessage: message,\n\t\t}, nil\n\t}\n}\n\ntype fakeClock struct {\n\tnow time.Time\n}\n\nfunc (f *fakeClock) After(time.Duration) <-chan time.Time {\n\treturn nil\n}\n\nfunc (f *fakeClock) NewTicker(time.Duration) clock.Ticker {\n\treturn nil\n}\nfunc (f *fakeClock) NewTimer(time.Duration) clock.Timer {\n\treturn nil\n}\n\nfunc (f *fakeClock) Now() time.Time {\n\treturn f.now\n}\n\nfunc (f *fakeClock) Since(time.Time) time.Duration {\n\treturn 0\n}\n\nfunc (f *fakeClock) Sleep(time.Duration) {}\n\nfunc (f *fakeClock) Tick(time.Duration) <-chan time.Time {\n\treturn nil\n}\n\nfunc (f *fakeClock) AfterFunc(time.Duration, func()) clock.Timer {\n\treturn nil\n}\n\nfunc TestRunMaintenanceIfDue(t *testing.T) {\n\tnow := time.Now().Round(time.Second)\n\n\ttests := []struct {\n\t\tname string\n\t\trepo *velerov1api.BackupRepository\n\t\tstartJobFunc func(client.Client, context.Context, *velerov1api.BackupRepository, string, logrus.Level, *logging.FormatFlag, logrus.FieldLogger) (string, error)\n\t\twaitJobFunc func(client.Client, context.Context, string, string, logrus.FieldLogger) (velerov1api.BackupRepositoryMaintenanceStatus, error)\n\t\texpectedMaintenanceTime time.Time\n\t\texpectedHistory []velerov1api.BackupRepositoryMaintenanceStatus\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"not due\",\n\t\t\trepo: &velerov1api.BackupRepository{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"repo\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\t\t\tMaintenanceFrequency: metav1.Duration{Duration: time.Hour},\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupRepositoryStatus{\n\t\t\t\t\tLastMaintenanceTime: &metav1.Time{Time: now},\n\t\t\t\t\tRecentMaintenance: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(-time.Hour)},\n\t\t\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now},\n\t\t\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedMaintenanceTime: now,\n\t\t\texpectedHistory: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(-time.Hour)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"start failed\",\n\t\t\trepo: &velerov1api.BackupRepository{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"repo\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\t\t\tMaintenanceFrequency: metav1.Duration{Duration: time.Hour},\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupRepositoryStatus{\n\t\t\t\t\tLastMaintenanceTime: &metav1.Time{Time: now.Add(-time.Hour - time.Minute)},\n\t\t\t\t\tRecentMaintenance: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(-time.Hour * 2)},\n\t\t\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(-time.Hour)},\n\t\t\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tstartJobFunc: startMaintenanceJobFail,\n\t\t\texpectedMaintenanceTime: now.Add(-time.Hour - time.Minute),\n\t\t\texpectedHistory: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(-time.Hour * 2)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(-time.Hour)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceFailed,\n\t\t\t\t\tMessage: \"Failed to start maintenance job, err: fake-start-error\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"wait failed\",\n\t\t\trepo: &velerov1api.BackupRepository{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"repo\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\t\t\tMaintenanceFrequency: metav1.Duration{Duration: time.Hour},\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupRepositoryStatus{\n\t\t\t\t\tLastMaintenanceTime: &metav1.Time{Time: now.Add(-time.Hour - time.Minute)},\n\t\t\t\t\tRecentMaintenance: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(-time.Hour * 2)},\n\t\t\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(-time.Hour)},\n\t\t\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tstartJobFunc: startMaintenanceJobSucceed,\n\t\t\twaitJobFunc: waitMaintenanceJobCompleteFail,\n\t\t\texpectedErr: \"error waiting repo maintenance completion status: fake-wait-error\",\n\t\t\texpectedMaintenanceTime: now.Add(-time.Hour - time.Minute),\n\t\t\texpectedHistory: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(-time.Hour * 2)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(-time.Hour)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"maintenance failed\",\n\t\t\trepo: &velerov1api.BackupRepository{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"repo\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\t\t\tMaintenanceFrequency: metav1.Duration{Duration: time.Hour},\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupRepositoryStatus{\n\t\t\t\t\tLastMaintenanceTime: &metav1.Time{Time: now.Add(-time.Hour - time.Minute)},\n\t\t\t\t\tRecentMaintenance: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(-time.Hour * 2)},\n\t\t\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(-time.Hour)},\n\t\t\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tstartJobFunc: startMaintenanceJobSucceed,\n\t\t\twaitJobFunc: waitMaintenanceJobCompleteFunc(now, velerov1api.BackupRepositoryMaintenanceFailed, \"fake-maintenance-message\"),\n\t\t\texpectedMaintenanceTime: now.Add(-time.Hour - time.Minute),\n\t\t\texpectedHistory: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(-time.Hour * 2)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(-time.Hour)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceFailed,\n\t\t\t\t\tMessage: \"fake-maintenance-message\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"maintenance succeeded\",\n\t\t\trepo: &velerov1api.BackupRepository{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"repo\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\t\t\tMaintenanceFrequency: metav1.Duration{Duration: time.Hour},\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupRepositoryStatus{\n\t\t\t\t\tLastMaintenanceTime: &metav1.Time{Time: now.Add(-time.Hour - time.Minute)},\n\t\t\t\t\tRecentMaintenance: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(-time.Hour * 2)},\n\t\t\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(-time.Hour)},\n\t\t\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tstartJobFunc: startMaintenanceJobSucceed,\n\t\t\twaitJobFunc: waitMaintenanceJobCompleteFunc(now, velerov1api.BackupRepositoryMaintenanceSucceeded, \"\"),\n\t\t\texpectedMaintenanceTime: now.Add(time.Hour),\n\t\t\texpectedHistory: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(-time.Hour * 2)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(-time.Hour)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\treconciler := mockBackupRepoReconciler(t, \"\", test.repo, nil)\n\t\t\treconciler.clock = &fakeClock{now}\n\t\t\terr := reconciler.Client.Create(t.Context(), test.repo)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tfuncStartMaintenanceJob = test.startJobFunc\n\t\t\tfuncWaitMaintenanceJobComplete = test.waitJobFunc\n\n\t\t\terr = reconciler.runMaintenanceIfDue(t.Context(), test.repo, velerotest.NewLogger())\n\t\t\tif test.expectedErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tassert.Equal(t, test.expectedMaintenanceTime, test.repo.Status.LastMaintenanceTime.Time)\n\t\t\tassert.Len(t, test.repo.Status.RecentMaintenance, len(test.expectedHistory))\n\n\t\t\tfor i := 0; i < len(test.expectedHistory); i++ {\n\t\t\t\tassert.Equal(t, test.expectedHistory[i].StartTimestamp.Time, test.repo.Status.RecentMaintenance[i].StartTimestamp.Time)\n\t\t\t\tif test.expectedHistory[i].CompleteTimestamp == nil {\n\t\t\t\t\tassert.Nil(t, test.repo.Status.RecentMaintenance[i].CompleteTimestamp)\n\t\t\t\t} else {\n\t\t\t\t\tassert.Equal(t, test.expectedHistory[i].CompleteTimestamp.Time, test.repo.Status.RecentMaintenance[i].CompleteTimestamp.Time)\n\t\t\t\t}\n\n\t\t\t\tassert.Equal(t, test.expectedHistory[i].Result, test.repo.Status.RecentMaintenance[i].Result)\n\t\t\t\tassert.Equal(t, test.expectedHistory[i].Message, test.repo.Status.RecentMaintenance[i].Message)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestInitializeRepo(t *testing.T) {\n\trr := mockBackupRepositoryCR()\n\trr.Spec.BackupStorageLocation = \"default\"\n\treconciler := mockBackupRepoReconciler(t, \"PrepareRepo\", rr, nil)\n\terr := reconciler.Client.Create(t.Context(), rr)\n\trequire.NoError(t, err)\n\tlocation := velerov1api.BackupStorageLocation{\n\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\tConfig: map[string]string{\"resticRepoPrefix\": \"s3:test.amazonaws.com/bucket/restic\"},\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: rr.Spec.BackupStorageLocation,\n\t\t},\n\t}\n\n\terr = reconciler.initializeRepo(t.Context(), rr, &location, reconciler.logger)\n\trequire.NoError(t, err)\n\tassert.Equal(t, velerov1api.BackupRepositoryPhaseReady, rr.Status.Phase)\n}\n\nfunc TestBackupRepoReconcile(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\trepo *velerov1api.BackupRepository\n\t\texpectNil bool\n\t}{\n\t\t{\n\t\t\tname: \"test on api server not found\",\n\t\t\trepo: &velerov1api.BackupRepository{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"unknown\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\t\t\tMaintenanceFrequency: metav1.Duration{Duration: testMaintenanceFrequency},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectNil: true,\n\t\t},\n\t\t{\n\t\t\tname: \"test on initialize repo\",\n\t\t\trepo: &velerov1api.BackupRepository{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"repo\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\t\t\tMaintenanceFrequency: metav1.Duration{Duration: testMaintenanceFrequency},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectNil: true,\n\t\t},\n\t\t{\n\t\t\tname: \"test on repo with new phase\",\n\t\t\trepo: &velerov1api.BackupRepository{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"repo\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\t\t\tMaintenanceFrequency: metav1.Duration{Duration: testMaintenanceFrequency},\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupRepositoryStatus{\n\t\t\t\t\tPhase: velerov1api.BackupRepositoryPhaseNew,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectNil: true,\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\treconciler := mockBackupRepoReconciler(t, \"\", test.repo, nil)\n\t\t\terr := reconciler.Client.Create(t.Context(), test.repo)\n\t\t\trequire.NoError(t, err)\n\t\t\t_, err = reconciler.Reconcile(t.Context(), ctrl.Request{NamespacedName: types.NamespacedName{Namespace: test.repo.Namespace, Name: \"repo\"}})\n\t\t\tif test.expectNil {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.Error(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetRepositoryMaintenanceFrequency(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tmgr repotypes.SnapshotIdentifier\n\t\trepo *velerov1api.BackupRepository\n\t\tfreqReturn time.Duration\n\t\tfreqError error\n\t\tuserDefinedFreq time.Duration\n\t\texpectFreq time.Duration\n\t}{\n\t\t{\n\t\t\tname: \"user defined valid\",\n\t\t\tuserDefinedFreq: time.Hour,\n\t\t\texpectFreq: time.Hour,\n\t\t},\n\t\t{\n\t\t\tname: \"repo return valid\",\n\t\t\tfreqReturn: time.Hour * 2,\n\t\t\texpectFreq: time.Hour * 2,\n\t\t},\n\t\t{\n\t\t\tname: \"fall to default\",\n\t\t\tuserDefinedFreq: -1,\n\t\t\tfreqError: errors.New(\"fake-error\"),\n\t\t\texpectFreq: defaultMaintainFrequency,\n\t\t},\n\t\t{\n\t\t\tname: \"fall to default, no freq error\",\n\t\t\tfreqReturn: -1,\n\t\t\texpectFreq: defaultMaintainFrequency,\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tmgr := repomokes.Manager{}\n\t\t\tmgr.On(\"DefaultMaintenanceFrequency\", mock.Anything).Return(test.freqReturn, test.freqError)\n\t\t\treconciler := NewBackupRepoReconciler(\n\t\t\t\tvelerov1api.DefaultNamespace,\n\t\t\t\tvelerotest.NewLogger(),\n\t\t\t\tvelerotest.NewFakeControllerRuntimeClient(t),\n\t\t\t\t&mgr,\n\t\t\t\ttest.userDefinedFreq,\n\t\t\t\t\"\",\n\t\t\t\t\"\",\n\t\t\t\tlogrus.InfoLevel,\n\t\t\t\tnil,\n\t\t\t\tnil,\n\t\t\t)\n\n\t\t\tfreq := reconciler.getRepositoryMaintenanceFrequency(test.repo)\n\t\t\tassert.Equal(t, test.expectFreq, freq)\n\t\t})\n\t}\n}\n\nfunc TestNeedInvalidBackupRepo(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\toldBSL *velerov1api.BackupStorageLocation\n\t\tnewBSL *velerov1api.BackupStorageLocation\n\t\texpect bool\n\t}{\n\t\t{\n\t\t\tname: \"no change\",\n\t\t\toldBSL: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"old-provider\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tnewBSL: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"new-provider\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpect: false,\n\t\t},\n\t\t{\n\t\t\tname: \"other part change\",\n\t\t\toldBSL: &velerov1api.BackupStorageLocation{},\n\t\t\tnewBSL: &velerov1api.BackupStorageLocation{},\n\t\t\texpect: false,\n\t\t},\n\t\t{\n\t\t\tname: \"bucket change\",\n\t\t\toldBSL: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"old-bucket\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tnewBSL: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"new-bucket\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpect: true,\n\t\t},\n\t\t{\n\t\t\tname: \"prefix change\",\n\t\t\toldBSL: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tPrefix: \"old-prefix\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tnewBSL: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tPrefix: \"new-prefix\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpect: true,\n\t\t},\n\t\t{\n\t\t\tname: \"CACert change\",\n\t\t\toldBSL: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tCACert: []byte{0x11, 0x12, 0x13},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tnewBSL: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tCACert: []byte{0x21, 0x22, 0x23},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpect: true,\n\t\t},\n\t\t{\n\t\t\tname: \"config change\",\n\t\t\toldBSL: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"key1\": \"value1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tnewBSL: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"key2\": \"value2\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpect: true,\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\treconciler := NewBackupRepoReconciler(\n\t\t\t\tvelerov1api.DefaultNamespace,\n\t\t\t\tvelerotest.NewLogger(),\n\t\t\t\tvelerotest.NewFakeControllerRuntimeClient(t),\n\t\t\t\tnil,\n\t\t\t\ttime.Duration(0),\n\t\t\t\t\"\",\n\t\t\t\t\"\",\n\t\t\t\tlogrus.InfoLevel,\n\t\t\t\tnil,\n\t\t\t\tnil,\n\t\t\t)\n\n\t\t\tneed := reconciler.needInvalidBackupRepo(test.oldBSL, test.newBSL)\n\t\t\tassert.Equal(t, test.expect, need)\n\t\t})\n\t}\n}\n\nfunc TestGetBackupRepositoryConfig(t *testing.T) {\n\tconfigWithNoData := &corev1api.ConfigMap{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"config-1\",\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t},\n\t}\n\n\tconfigWithWrongData := &corev1api.ConfigMap{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"config-1\",\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t},\n\t\tData: map[string]string{\n\t\t\t\"fake-repo-type\": \"\",\n\t\t},\n\t}\n\n\tconfigWithData := &corev1api.ConfigMap{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"config-1\",\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t},\n\t\tData: map[string]string{\n\t\t\t\"fake-repo-type\": \"{\\\"cacheLimitMB\\\": 1000, \\\"enableCompression\\\": true, \\\"fullMaintenanceInterval\\\": \\\"fastGC\\\"}\",\n\t\t\t\"fake-repo-type-1\": \"{\\\"cacheLimitMB\\\": 1, \\\"enableCompression\\\": false}\",\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tcongiName string\n\t\trepoName string\n\t\trepoType string\n\t\tkubeClientObj []runtime.Object\n\t\texpectedErr string\n\t\texpectedResult map[string]string\n\t}{\n\t\t{\n\t\t\tname: \"empty configName\",\n\t\t},\n\t\t{\n\t\t\tname: \"get error\",\n\t\t\tcongiName: \"config-1\",\n\t\t\texpectedErr: \"error getting configMap config-1: configmaps \\\"config-1\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"no config for repo\",\n\t\t\tcongiName: \"config-1\",\n\t\t\trepoName: \"fake-repo\",\n\t\t\trepoType: \"fake-repo-type\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tconfigWithNoData,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"unmarshall error\",\n\t\t\tcongiName: \"config-1\",\n\t\t\trepoName: \"fake-repo\",\n\t\t\trepoType: \"fake-repo-type\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tconfigWithWrongData,\n\t\t\t},\n\t\t\texpectedErr: \"error unmarshalling config data from config-1 for repo fake-repo, repo type fake-repo-type: unexpected end of JSON input\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tcongiName: \"config-1\",\n\t\t\trepoName: \"fake-repo\",\n\t\t\trepoType: \"fake-repo-type\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tconfigWithData,\n\t\t\t},\n\t\t\texpectedResult: map[string]string{\n\t\t\t\t\"cacheLimitMB\": \"1000\",\n\t\t\t\t\"enableCompression\": \"true\",\n\t\t\t\t\"fullMaintenanceInterval\": \"fastGC\",\n\t\t\t},\n\t\t},\n\t}\n\n\tscheme := runtime.NewScheme()\n\tcorev1api.AddToScheme(scheme)\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeClientBuilder := clientFake.NewClientBuilder()\n\t\t\tfakeClientBuilder = fakeClientBuilder.WithScheme(scheme)\n\n\t\t\tfakeClient := fakeClientBuilder.WithRuntimeObjects(test.kubeClientObj...).Build()\n\n\t\t\tresult, err := getBackupRepositoryConfig(t.Context(), fakeClient, test.congiName, velerov1api.DefaultNamespace, test.repoName, test.repoType, velerotest.NewLogger())\n\n\t\t\tif test.expectedErr != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.expectedErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, test.expectedResult, result)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestUpdateRepoMaintenanceHistory(t *testing.T) {\n\tstandardTime := time.Now()\n\n\tbackupRepoWithoutHistory := &velerov1api.BackupRepository{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: \"repo\",\n\t\t},\n\t}\n\n\tbackupRepoWithHistory := &velerov1api.BackupRepository{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: \"repo\",\n\t\t},\n\t\tStatus: velerov1api.BackupRepositoryStatus{\n\t\t\tRecentMaintenance: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 24)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 23)},\n\t\t\t\t\tMessage: \"fake-history-message-1\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tbackupRepoWithFullHistory := &velerov1api.BackupRepository{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: \"repo\",\n\t\t},\n\t\tStatus: velerov1api.BackupRepositoryStatus{\n\t\t\tRecentMaintenance: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 24)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 23)},\n\t\t\t\t\tMessage: \"fake-history-message-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 22)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 21)},\n\t\t\t\t\tMessage: \"fake-history-message-3\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 20)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 19)},\n\t\t\t\t\tMessage: \"fake-history-message-4\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tbackupRepoWithOverFullHistory := &velerov1api.BackupRepository{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: \"repo\",\n\t\t},\n\t\tStatus: velerov1api.BackupRepositoryStatus{\n\t\t\tRecentMaintenance: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 24)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 23)},\n\t\t\t\t\tMessage: \"fake-history-message-5\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 22)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 21)},\n\t\t\t\t\tMessage: \"fake-history-message-6\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 20)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 19)},\n\t\t\t\t\tMessage: \"fake-history-message-7\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 18)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 17)},\n\t\t\t\t\tMessage: \"fake-history-message-8\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tbackupRepo *velerov1api.BackupRepository\n\t\tresult velerov1api.BackupRepositoryMaintenanceResult\n\t\texpectedHistory []velerov1api.BackupRepositoryMaintenanceStatus\n\t}{\n\t\t{\n\t\t\tname: \"empty history\",\n\t\t\tbackupRepo: backupRepoWithoutHistory,\n\t\t\tresult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\texpectedHistory: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: standardTime},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: standardTime.Add(time.Hour)},\n\t\t\t\t\tMessage: \"fake-message-0\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"less than history queue length\",\n\t\t\tbackupRepo: backupRepoWithHistory,\n\t\t\tresult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\texpectedHistory: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 24)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 23)},\n\t\t\t\t\tMessage: \"fake-history-message-1\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: standardTime},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: standardTime.Add(time.Hour)},\n\t\t\t\t\tMessage: \"fake-message-0\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"full history\",\n\t\t\tbackupRepo: backupRepoWithFullHistory,\n\t\t\tresult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\texpectedHistory: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 22)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 21)},\n\t\t\t\t\tMessage: \"fake-history-message-3\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 20)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 19)},\n\t\t\t\t\tMessage: \"fake-history-message-4\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: standardTime},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: standardTime.Add(time.Hour)},\n\t\t\t\t\tMessage: \"fake-message-0\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"over full history\",\n\t\t\tbackupRepo: backupRepoWithOverFullHistory,\n\t\t\tresult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\texpectedHistory: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 20)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 19)},\n\t\t\t\t\tMessage: \"fake-history-message-7\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 18)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: standardTime.Add(-time.Hour * 17)},\n\t\t\t\t\tMessage: \"fake-history-message-8\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: standardTime},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: standardTime.Add(time.Hour)},\n\t\t\t\t\tMessage: \"fake-message-0\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tupdateRepoMaintenanceHistory(test.backupRepo, test.result, &metav1.Time{Time: standardTime}, &metav1.Time{Time: standardTime.Add(time.Hour)}, \"fake-message-0\")\n\n\t\t\tfor at := range test.backupRepo.Status.RecentMaintenance {\n\t\t\t\tassert.Equal(t, test.expectedHistory[at].StartTimestamp.Time, test.backupRepo.Status.RecentMaintenance[at].StartTimestamp.Time)\n\t\t\t\tassert.Equal(t, test.expectedHistory[at].CompleteTimestamp.Time, test.backupRepo.Status.RecentMaintenance[at].CompleteTimestamp.Time)\n\t\t\t\tassert.Equal(t, test.expectedHistory[at].Message, test.backupRepo.Status.RecentMaintenance[at].Message)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestRecallMaintenance(t *testing.T) {\n\tnow := time.Now().Round(time.Second)\n\n\tschemeFail := runtime.NewScheme()\n\tvelerov1api.AddToScheme(schemeFail)\n\n\tscheme := runtime.NewScheme()\n\tbatchv1api.AddToScheme(scheme)\n\tcorev1api.AddToScheme(scheme)\n\tvelerov1api.AddToScheme(scheme)\n\n\tjobSucceeded := &batchv1api.Job{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"job1\",\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tLabels: map[string]string{maintenance.RepositoryNameLabel: \"repo\"},\n\t\t\tCreationTimestamp: metav1.Time{Time: now.Add(time.Hour)},\n\t\t},\n\t\tStatus: batchv1api.JobStatus{\n\t\t\tStartTime: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\tCompletionTime: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\tSucceeded: 1,\n\t\t},\n\t}\n\n\tjobPodSucceeded := builder.ForPod(velerov1api.DefaultNamespace, \"job1\").Labels(map[string]string{\"job-name\": \"job1\"}).ContainerStatuses(&corev1api.ContainerStatus{\n\t\tState: corev1api.ContainerState{\n\t\t\tTerminated: &corev1api.ContainerStateTerminated{},\n\t\t},\n\t}).Result()\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\truntimeScheme *runtime.Scheme\n\t\trepoLastMatainTime metav1.Time\n\t\texpectNewHistory []velerov1api.BackupRepositoryMaintenanceStatus\n\t\texpectTimeUpdate *metav1.Time\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"wait completion error\",\n\t\t\truntimeScheme: schemeFail,\n\t\t\texpectedErr: \"error waiting incomplete repo maintenance job for repo repo: error listing maintenance job for repo repo: no kind is registered for the type v1.JobList in scheme\",\n\t\t},\n\t\t{\n\t\t\tname: \"no consolidate result\",\n\t\t\truntimeScheme: scheme,\n\t\t},\n\t\t{\n\t\t\tname: \"no update last time\",\n\t\t\truntimeScheme: scheme,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tjobSucceeded,\n\t\t\t\tjobPodSucceeded,\n\t\t\t},\n\t\t\trepoLastMatainTime: metav1.Time{Time: now.Add(time.Hour * 5)},\n\t\t\texpectNewHistory: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"update last time\",\n\t\t\truntimeScheme: scheme,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tjobSucceeded,\n\t\t\t\tjobPodSucceeded,\n\t\t\t},\n\t\t\texpectNewHistory: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectTimeUpdate: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tr := mockBackupRepoReconciler(t, \"\", nil, nil)\n\n\t\t\tbackupRepo := mockBackupRepositoryCR()\n\t\t\tbackupRepo.Status.LastMaintenanceTime = &test.repoLastMatainTime\n\n\t\t\ttest.kubeClientObj = append(test.kubeClientObj, backupRepo)\n\n\t\t\tfakeClientBuilder := clientFake.NewClientBuilder()\n\t\t\tfakeClientBuilder = fakeClientBuilder.WithScheme(test.runtimeScheme)\n\t\t\tfakeClient := fakeClientBuilder.WithRuntimeObjects(test.kubeClientObj...).Build()\n\t\t\tr.Client = fakeClient\n\n\t\t\tlastTm := backupRepo.Status.LastMaintenanceTime\n\n\t\t\terr := r.recallMaintenance(t.Context(), backupRepo, velerotest.NewLogger())\n\t\t\tif test.expectedErr != \"\" {\n\t\t\t\tassert.ErrorContains(t, err, test.expectedErr)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\n\t\t\t\tif test.expectNewHistory == nil {\n\t\t\t\t\tassert.Nil(t, backupRepo.Status.RecentMaintenance)\n\t\t\t\t} else {\n\t\t\t\t\tassert.Len(t, backupRepo.Status.RecentMaintenance, len(test.expectNewHistory))\n\t\t\t\t\tfor i := 0; i < len(test.expectNewHistory); i++ {\n\t\t\t\t\t\tassert.Equal(t, test.expectNewHistory[i].StartTimestamp.Time, backupRepo.Status.RecentMaintenance[i].StartTimestamp.Time)\n\t\t\t\t\t\tassert.Equal(t, test.expectNewHistory[i].CompleteTimestamp.Time, backupRepo.Status.RecentMaintenance[i].CompleteTimestamp.Time)\n\t\t\t\t\t\tassert.Equal(t, test.expectNewHistory[i].Result, backupRepo.Status.RecentMaintenance[i].Result)\n\t\t\t\t\t\tassert.Equal(t, test.expectNewHistory[i].Message, backupRepo.Status.RecentMaintenance[i].Message)\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\tif test.expectTimeUpdate != nil {\n\t\t\t\t\tassert.Equal(t, test.expectTimeUpdate.Time, backupRepo.Status.LastMaintenanceTime.Time)\n\t\t\t\t} else {\n\t\t\t\t\tassert.Equal(t, lastTm, backupRepo.Status.LastMaintenanceTime)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestConsolidateHistory(t *testing.T) {\n\tnow := time.Now()\n\n\ttests := []struct {\n\t\tname string\n\t\tcur []velerov1api.BackupRepositoryMaintenanceStatus\n\t\tcoming []velerov1api.BackupRepositoryMaintenanceStatus\n\t\texpected []velerov1api.BackupRepositoryMaintenanceStatus\n\t}{\n\t\t{\n\t\t\tname: \"zero coming\",\n\t\t\tcur: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"identical coming\",\n\t\t\tcur: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tcoming: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"less than limit\",\n\t\t\tcur: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-2\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tcoming: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-3\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-3\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"more than limit\",\n\t\t\tcur: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-2\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tcoming: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-3\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 4)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-4\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-3\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 4)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-4\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"more than limit 2\",\n\t\t\tcur: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-3\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tcoming: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-3\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 4)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-4\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-3\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 4)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-4\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"coming is not used\",\n\t\t\tcur: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-3\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 4)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-4\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tcoming: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: nil,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tconsolidated := consolidateHistory(test.coming, test.cur)\n\t\t\tif test.expected == nil {\n\t\t\t\tassert.Nil(t, consolidated)\n\t\t\t} else {\n\t\t\t\tassert.Len(t, consolidated, len(test.expected))\n\t\t\t\tfor i := 0; i < len(test.expected); i++ {\n\t\t\t\t\tassert.Equal(t, *test.expected[i].StartTimestamp, *consolidated[i].StartTimestamp)\n\t\t\t\t\tassert.Equal(t, *test.expected[i].CompleteTimestamp, *consolidated[i].CompleteTimestamp)\n\t\t\t\t\tassert.Equal(t, test.expected[i].Result, consolidated[i].Result)\n\t\t\t\t\tassert.Equal(t, test.expected[i].Message, consolidated[i].Message)\n\t\t\t\t}\n\n\t\t\t\tassert.Nil(t, consolidateHistory(test.coming, consolidated))\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetLastMaintenanceTimeFromHistory(t *testing.T) {\n\tnow := time.Now()\n\n\ttests := []struct {\n\t\tname string\n\t\thistory []velerov1api.BackupRepositoryMaintenanceStatus\n\t\texpected time.Time\n\t}{\n\t\t{\n\t\t\tname: \"first one is nil\",\n\t\t\thistory: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceFailed,\n\t\t\t\t\tMessage: \"fake-maintenance-message\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-3\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: now.Add(time.Hour * 3),\n\t\t},\n\t\t{\n\t\t\tname: \"another one is nil\",\n\t\t\thistory: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceFailed,\n\t\t\t\t\tMessage: \"fake-maintenance-message-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-3\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: now.Add(time.Hour * 3),\n\t\t},\n\t\t{\n\t\t\tname: \"disordered\",\n\t\t\thistory: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message-3\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tMessage: \"fake-maintenance-message\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceFailed,\n\t\t\t\t\tMessage: \"fake-maintenance-message-2\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: now.Add(time.Hour * 3),\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\ttime := getLastMaintenanceTimeFromHistory(test.history)\n\t\t\tassert.Equal(t, test.expected, time.Time)\n\t\t})\n\t}\n}\n\nfunc TestDeleteOldMaintenanceJobWithConfigMap(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\trepo *velerov1api.BackupRepository\n\t\texpectedKeptJobs int\n\t\tmaintenanceJobs []batchv1api.Job\n\t\tbsl *velerov1api.BackupStorageLocation\n\t\trepoMaintenanceJob *corev1api.ConfigMap\n\t}{\n\t\t{\n\t\t\tname: \"test with global config\",\n\t\t\trepo: &velerov1api.BackupRepository{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"repo\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\t\t\tMaintenanceFrequency: metav1.Duration{Duration: testMaintenanceFrequency},\n\t\t\t\t\tBackupStorageLocation: \"default\",\n\t\t\t\t\tVolumeNamespace: \"test-ns\",\n\t\t\t\t\tRepositoryType: \"restic\",\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupRepositoryStatus{\n\t\t\t\t\tPhase: velerov1api.BackupRepositoryPhaseReady,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedKeptJobs: 5,\n\t\t\tmaintenanceJobs: []batchv1api.Job{\n\t\t\t\t*builder.ForJob(\"velero\", \"job-01\").ObjectMeta(builder.WithLabels(repomaintenance.RepositoryNameLabel, \"repo\")).Succeeded(1).Result(),\n\t\t\t\t*builder.ForJob(\"velero\", \"job-02\").ObjectMeta(builder.WithLabels(repomaintenance.RepositoryNameLabel, \"repo\")).Succeeded(1).Result(),\n\t\t\t\t*builder.ForJob(\"velero\", \"job-03\").ObjectMeta(builder.WithLabels(repomaintenance.RepositoryNameLabel, \"repo\")).Succeeded(1).Result(),\n\t\t\t\t*builder.ForJob(\"velero\", \"job-04\").ObjectMeta(builder.WithLabels(repomaintenance.RepositoryNameLabel, \"repo\")).Succeeded(1).Result(),\n\t\t\t\t*builder.ForJob(\"velero\", \"job-05\").ObjectMeta(builder.WithLabels(repomaintenance.RepositoryNameLabel, \"repo\")).Succeeded(1).Result(),\n\t\t\t\t*builder.ForJob(\"velero\", \"job-06\").ObjectMeta(builder.WithLabels(repomaintenance.RepositoryNameLabel, \"repo\")).Succeeded(1).Result(),\n\t\t\t},\n\t\t\tbsl: builder.ForBackupStorageLocation(\"velero\", \"default\").Result(),\n\t\t\trepoMaintenanceJob: &corev1api.ConfigMap{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"repo-maintenance-job-config\",\n\t\t\t\t},\n\t\t\t\tData: map[string]string{\n\t\t\t\t\t\"global\": `{\"keepLatestMaintenanceJobs\": 5}`,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"test with specific repo config overriding global\",\n\t\t\trepo: &velerov1api.BackupRepository{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"repo\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\t\t\tMaintenanceFrequency: metav1.Duration{Duration: testMaintenanceFrequency},\n\t\t\t\t\tBackupStorageLocation: \"default\",\n\t\t\t\t\tVolumeNamespace: \"test-ns\",\n\t\t\t\t\tRepositoryType: \"restic\",\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupRepositoryStatus{\n\t\t\t\t\tPhase: velerov1api.BackupRepositoryPhaseReady,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedKeptJobs: 2,\n\t\t\tmaintenanceJobs: []batchv1api.Job{\n\t\t\t\t*builder.ForJob(\"velero\", \"job-01\").ObjectMeta(builder.WithLabels(repomaintenance.RepositoryNameLabel, \"repo\")).Succeeded(1).Result(),\n\t\t\t\t*builder.ForJob(\"velero\", \"job-02\").ObjectMeta(builder.WithLabels(repomaintenance.RepositoryNameLabel, \"repo\")).Succeeded(1).Result(),\n\t\t\t\t*builder.ForJob(\"velero\", \"job-03\").ObjectMeta(builder.WithLabels(repomaintenance.RepositoryNameLabel, \"repo\")).Succeeded(1).Result(),\n\t\t\t},\n\t\t\tbsl: builder.ForBackupStorageLocation(\"velero\", \"default\").Result(),\n\t\t\trepoMaintenanceJob: &corev1api.ConfigMap{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"repo-maintenance-job-config\",\n\t\t\t\t},\n\t\t\t\tData: map[string]string{\n\t\t\t\t\t\"global\": `{\"keepLatestMaintenanceJobs\": 5}`,\n\t\t\t\t\t\"test-ns-default-restic\": `{\"keepLatestMaintenanceJobs\": 2}`,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tobjects := []runtime.Object{test.repo, test.bsl}\n\t\t\tif test.repoMaintenanceJob != nil {\n\t\t\t\tobjects = append(objects, test.repoMaintenanceJob)\n\t\t\t}\n\t\t\tcrClient := velerotest.NewFakeControllerRuntimeClient(t, objects...)\n\n\t\t\tfor _, job := range test.maintenanceJobs {\n\t\t\t\trequire.NoError(t, crClient.Create(t.Context(), &job))\n\t\t\t}\n\n\t\t\trepoLocker := repository.NewRepoLocker()\n\t\t\tmgr := repomanager.NewManager(\"\", crClient, repoLocker, nil, nil, nil)\n\n\t\t\trepoMaintenanceConfigName := \"\"\n\t\t\tif test.repoMaintenanceJob != nil {\n\t\t\t\trepoMaintenanceConfigName = test.repoMaintenanceJob.Name\n\t\t\t}\n\n\t\t\treconciler := NewBackupRepoReconciler(\n\t\t\t\tvelerov1api.DefaultNamespace,\n\t\t\t\tvelerotest.NewLogger(),\n\t\t\t\tcrClient,\n\t\t\t\tmgr,\n\t\t\t\ttime.Duration(0),\n\t\t\t\t\"\",\n\t\t\t\trepoMaintenanceConfigName,\n\t\t\t\tlogrus.InfoLevel,\n\t\t\t\tnil,\n\t\t\t\tnil,\n\t\t\t)\n\n\t\t\t_, err := reconciler.Reconcile(t.Context(), ctrl.Request{NamespacedName: types.NamespacedName{Namespace: test.repo.Namespace, Name: \"repo\"}})\n\t\t\trequire.NoError(t, err)\n\n\t\t\tjobList := new(batchv1api.JobList)\n\t\t\trequire.NoError(t, reconciler.Client.List(t.Context(), jobList, &client.ListOptions{Namespace: \"velero\"}))\n\t\t\tassert.Len(t, jobList.Items, test.expectedKeptJobs, \"Expected %d jobs to be kept, but got %d\", test.expectedKeptJobs, len(jobList.Items))\n\t\t})\n\t}\n}\n\nfunc TestInitializeRepoWithRepositoryTypes(t *testing.T) {\n\tscheme := runtime.NewScheme()\n\tcorev1api.AddToScheme(scheme)\n\tvelerov1api.AddToScheme(scheme)\n\n\t// Test for restic repository\n\tt.Run(\"restic repository\", func(t *testing.T) {\n\t\trr := mockBackupRepositoryCR()\n\t\trr.Spec.BackupStorageLocation = \"default\"\n\t\trr.Spec.VolumeNamespace = \"volume-ns-1\"\n\t\trr.Spec.RepositoryType = velerov1api.BackupRepositoryTypeRestic\n\n\t\tlocation := &velerov1api.BackupStorageLocation{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\tName: \"default\",\n\t\t\t},\n\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\tProvider: \"aws\",\n\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\tBucket: \"test-bucket\",\n\t\t\t\t\t\tPrefix: \"test-prefix\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\"region\": \"us-east-1\",\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\n\t\tfakeClient := clientFake.NewClientBuilder().WithScheme(scheme).WithRuntimeObjects(rr, location).Build()\n\t\tmgr := &repomokes.Manager{}\n\t\tmgr.On(\"PrepareRepo\", rr).Return(nil)\n\n\t\treconciler := NewBackupRepoReconciler(\n\t\t\tvelerov1api.DefaultNamespace,\n\t\t\tvelerotest.NewLogger(),\n\t\t\tfakeClient,\n\t\t\tmgr,\n\t\t\ttestMaintenanceFrequency,\n\t\t\t\"\",\n\t\t\t\"\",\n\t\t\tlogrus.InfoLevel,\n\t\t\tnil,\n\t\t\tnil,\n\t\t)\n\n\t\terr := reconciler.initializeRepo(t.Context(), rr, location, reconciler.logger)\n\t\trequire.NoError(t, err)\n\n\t\t// Verify ResticIdentifier is set for restic\n\t\tassert.NotEmpty(t, rr.Spec.ResticIdentifier)\n\t\tassert.Contains(t, rr.Spec.ResticIdentifier, \"volume-ns-1\")\n\t\tassert.Equal(t, velerov1api.BackupRepositoryPhaseReady, rr.Status.Phase)\n\t})\n\n\t// Test for kopia repository\n\tt.Run(\"kopia repository\", func(t *testing.T) {\n\t\trr := mockBackupRepositoryCR()\n\t\trr.Spec.BackupStorageLocation = \"default\"\n\t\trr.Spec.VolumeNamespace = \"volume-ns-1\"\n\t\trr.Spec.RepositoryType = velerov1api.BackupRepositoryTypeKopia\n\n\t\tlocation := &velerov1api.BackupStorageLocation{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\tName: \"default\",\n\t\t\t},\n\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\tProvider: \"aws\",\n\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\tBucket: \"test-bucket\",\n\t\t\t\t\t\tPrefix: \"test-prefix\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\"region\": \"us-east-1\",\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\n\t\tfakeClient := clientFake.NewClientBuilder().WithScheme(scheme).WithRuntimeObjects(rr, location).Build()\n\t\tmgr := &repomokes.Manager{}\n\t\tmgr.On(\"PrepareRepo\", rr).Return(nil)\n\n\t\treconciler := NewBackupRepoReconciler(\n\t\t\tvelerov1api.DefaultNamespace,\n\t\t\tvelerotest.NewLogger(),\n\t\t\tfakeClient,\n\t\t\tmgr,\n\t\t\ttestMaintenanceFrequency,\n\t\t\t\"\",\n\t\t\t\"\",\n\t\t\tlogrus.InfoLevel,\n\t\t\tnil,\n\t\t\tnil,\n\t\t)\n\n\t\terr := reconciler.initializeRepo(t.Context(), rr, location, reconciler.logger)\n\t\trequire.NoError(t, err)\n\n\t\t// Verify ResticIdentifier is NOT set for kopia\n\t\tassert.Empty(t, rr.Spec.ResticIdentifier)\n\t\tassert.Equal(t, velerov1api.BackupRepositoryPhaseReady, rr.Status.Phase)\n\t})\n\n\t// Test for empty repository type (defaults to restic)\n\tt.Run(\"empty repository type\", func(t *testing.T) {\n\t\trr := mockBackupRepositoryCR()\n\t\trr.Spec.BackupStorageLocation = \"default\"\n\t\trr.Spec.VolumeNamespace = \"volume-ns-1\"\n\t\t// Leave RepositoryType empty\n\n\t\tlocation := &velerov1api.BackupStorageLocation{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\tName: \"default\",\n\t\t\t},\n\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\tProvider: \"aws\",\n\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\tBucket: \"test-bucket\",\n\t\t\t\t\t\tPrefix: \"test-prefix\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\"region\": \"us-east-1\",\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\n\t\tfakeClient := clientFake.NewClientBuilder().WithScheme(scheme).WithRuntimeObjects(rr, location).Build()\n\t\tmgr := &repomokes.Manager{}\n\t\tmgr.On(\"PrepareRepo\", rr).Return(nil)\n\n\t\treconciler := NewBackupRepoReconciler(\n\t\t\tvelerov1api.DefaultNamespace,\n\t\t\tvelerotest.NewLogger(),\n\t\t\tfakeClient,\n\t\t\tmgr,\n\t\t\ttestMaintenanceFrequency,\n\t\t\t\"\",\n\t\t\t\"\",\n\t\t\tlogrus.InfoLevel,\n\t\t\tnil,\n\t\t\tnil,\n\t\t)\n\n\t\terr := reconciler.initializeRepo(t.Context(), rr, location, reconciler.logger)\n\t\trequire.NoError(t, err)\n\n\t\t// Verify ResticIdentifier is set when type is empty (defaults to restic)\n\t\tassert.NotEmpty(t, rr.Spec.ResticIdentifier)\n\t\tassert.Contains(t, rr.Spec.ResticIdentifier, \"volume-ns-1\")\n\t\tassert.Equal(t, velerov1api.BackupRepositoryPhaseReady, rr.Status.Phase)\n\t})\n}\n\nfunc TestRepoMaintenanceMetricsRecording(t *testing.T) {\n\tnow := time.Now().Round(time.Second)\n\n\ttests := []struct {\n\t\tname string\n\t\trepo *velerov1api.BackupRepository\n\t\tstartJobFunc func(client.Client, context.Context, *velerov1api.BackupRepository, string, logrus.Level, *logging.FormatFlag, logrus.FieldLogger) (string, error)\n\t\twaitJobFunc func(client.Client, context.Context, string, string, logrus.FieldLogger) (velerov1api.BackupRepositoryMaintenanceStatus, error)\n\t\texpectSuccess bool\n\t\texpectFailure bool\n\t\texpectDuration bool\n\t}{\n\t\t{\n\t\t\tname: \"metrics recorded on successful maintenance\",\n\t\t\trepo: &velerov1api.BackupRepository{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"test-repo-success\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\t\t\tMaintenanceFrequency: metav1.Duration{Duration: time.Hour},\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupRepositoryStatus{\n\t\t\t\t\tLastMaintenanceTime: &metav1.Time{Time: now.Add(-2 * time.Hour)},\n\t\t\t\t},\n\t\t\t},\n\t\t\tstartJobFunc: startMaintenanceJobSucceed,\n\t\t\twaitJobFunc: waitMaintenanceJobCompleteFunc(now, velerov1api.BackupRepositoryMaintenanceSucceeded, \"\"),\n\t\t\texpectSuccess: true,\n\t\t\texpectFailure: false,\n\t\t\texpectDuration: true,\n\t\t},\n\t\t{\n\t\t\tname: \"metrics recorded on failed maintenance\",\n\t\t\trepo: &velerov1api.BackupRepository{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"test-repo-failure\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\t\t\tMaintenanceFrequency: metav1.Duration{Duration: time.Hour},\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupRepositoryStatus{\n\t\t\t\t\tLastMaintenanceTime: &metav1.Time{Time: now.Add(-2 * time.Hour)},\n\t\t\t\t},\n\t\t\t},\n\t\t\tstartJobFunc: startMaintenanceJobSucceed,\n\t\t\twaitJobFunc: func(client.Client, context.Context, string, string, logrus.FieldLogger) (velerov1api.BackupRepositoryMaintenanceStatus, error) {\n\t\t\t\treturn velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Minute)}, // Job ran for 1 minute then failed\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceFailed,\n\t\t\t\t\tMessage: \"test error\",\n\t\t\t\t}, nil\n\t\t\t},\n\t\t\texpectSuccess: false,\n\t\t\texpectFailure: true,\n\t\t\texpectDuration: true,\n\t\t},\n\t\t{\n\t\t\tname: \"metrics recorded on job start failure\",\n\t\t\trepo: &velerov1api.BackupRepository{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: \"test-repo-start-fail\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\t\t\tMaintenanceFrequency: metav1.Duration{Duration: time.Hour},\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.BackupRepositoryStatus{\n\t\t\t\t\tLastMaintenanceTime: &metav1.Time{Time: now.Add(-2 * time.Hour)},\n\t\t\t\t},\n\t\t\t},\n\t\t\tstartJobFunc: startMaintenanceJobFail,\n\t\t\texpectSuccess: false,\n\t\t\texpectFailure: true,\n\t\t\texpectDuration: false, // No duration when job fails to start\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\t// Create metrics instance\n\t\t\tm := metrics.NewServerMetrics()\n\n\t\t\t// Create reconciler with metrics\n\t\t\treconciler := mockBackupRepoReconciler(t, \"\", test.repo, nil)\n\t\t\treconciler.metrics = m\n\t\t\treconciler.clock = &fakeClock{now}\n\n\t\t\terr := reconciler.Client.Create(t.Context(), test.repo)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Set up job functions\n\t\t\tfuncStartMaintenanceJob = test.startJobFunc\n\t\t\tfuncWaitMaintenanceJobComplete = test.waitJobFunc\n\n\t\t\t// Run maintenance\n\t\t\t_ = reconciler.runMaintenanceIfDue(t.Context(), test.repo, velerotest.NewLogger())\n\n\t\t\t// Verify metrics were recorded\n\t\t\tsuccessCount := getMaintenanceMetricValue(t, m, \"repo_maintenance_success_total\", test.repo.Name)\n\t\t\tfailureCount := getMaintenanceMetricValue(t, m, \"repo_maintenance_failure_total\", test.repo.Name)\n\t\t\tdurationCount := getMaintenanceDurationCount(t, m, test.repo.Name)\n\n\t\t\tif test.expectSuccess {\n\t\t\t\tassert.Equal(t, float64(1), successCount, \"Success metric should be recorded\")\n\t\t\t} else {\n\t\t\t\tassert.Equal(t, float64(0), successCount, \"Success metric should not be recorded\")\n\t\t\t}\n\n\t\t\tif test.expectFailure {\n\t\t\t\tassert.Equal(t, float64(1), failureCount, \"Failure metric should be recorded\")\n\t\t\t} else {\n\t\t\t\tassert.Equal(t, float64(0), failureCount, \"Failure metric should not be recorded\")\n\t\t\t}\n\n\t\t\tif test.expectDuration {\n\t\t\t\tassert.Equal(t, uint64(1), durationCount, \"Duration metric should be recorded\")\n\t\t\t} else {\n\t\t\t\tassert.Equal(t, uint64(0), durationCount, \"Duration metric should not be recorded\")\n\t\t\t}\n\t\t})\n\t}\n}\n\n// Helper to get maintenance metric value from ServerMetrics\nfunc getMaintenanceMetricValue(t *testing.T, m *metrics.ServerMetrics, metricName, repoName string) float64 {\n\tt.Helper()\n\n\tmetricMap := m.Metrics()\n\tcollector, ok := metricMap[metricName]\n\tif !ok {\n\t\treturn 0\n\t}\n\n\tch := make(chan prometheus.Metric, 1)\n\tcollector.Collect(ch)\n\tclose(ch)\n\n\tfor metric := range ch {\n\t\tdto := &dto.Metric{}\n\t\terr := metric.Write(dto)\n\t\trequire.NoError(t, err)\n\n\t\tfor _, label := range dto.Label {\n\t\t\tif *label.Name == \"repository_name\" && *label.Value == repoName {\n\t\t\t\tif dto.Counter != nil {\n\t\t\t\t\treturn *dto.Counter.Value\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\treturn 0\n}\n\n// Helper to get maintenance duration histogram count\nfunc getMaintenanceDurationCount(t *testing.T, m *metrics.ServerMetrics, repoName string) uint64 {\n\tt.Helper()\n\n\tmetricMap := m.Metrics()\n\tcollector, ok := metricMap[\"repo_maintenance_duration_seconds\"]\n\tif !ok {\n\t\treturn 0\n\t}\n\n\tch := make(chan prometheus.Metric, 1)\n\tcollector.Collect(ch)\n\tclose(ch)\n\n\tfor metric := range ch {\n\t\tdto := &dto.Metric{}\n\t\terr := metric.Write(dto)\n\t\trequire.NoError(t, err)\n\n\t\tfor _, label := range dto.Label {\n\t\t\tif *label.Name == \"repository_name\" && *label.Value == repoName {\n\t\t\t\tif dto.Histogram != nil {\n\t\t\t\t\treturn *dto.Histogram.SampleCount\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\treturn 0\n}\n" }, { "path": "pkg/controller/backup_storage_location_controller.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"regexp\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/builder\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/predicate\"\n\n\t\"github.com/vmware-tanzu/velero/internal/storage\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/persistence\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nconst (\n\t// keep the enqueue period a smaller value to make sure the BSL can be validated as expected.\n\t// The BSL validation frequency is 1 minute by default, if we set the enqueue period as 1 minute,\n\t// this will cause the actual validation interval for each BSL to be 2 minutes\n\tbslValidationEnqueuePeriod = 10 * time.Second\n)\n\n// sanitizeStorageError cleans up verbose HTTP responses from cloud provider errors,\n// particularly Azure which includes full HTTP response details and XML in error messages.\n// It extracts the error code and message while removing HTTP headers and response bodies.\n// It also scrubs sensitive information like SAS tokens from URLs.\nfunc sanitizeStorageError(err error) string {\n\tif err == nil {\n\t\treturn \"\"\n\t}\n\n\terrMsg := err.Error()\n\n\t// Scrub sensitive information from URLs (SAS tokens, credentials, etc.)\n\t// Azure SAS token parameters: sig, se, st, sp, spr, sv, sr, sip, srt, ss\n\t// These appear as query parameters in URLs like: ?sig=value&se=value\n\tsasParamsRegex := regexp.MustCompile(`([?&])(sig|se|st|sp|spr|sv|sr|sip|srt|ss)=([^&\\s<>\\n]+)`)\n\terrMsg = sasParamsRegex.ReplaceAllString(errMsg, `${1}${2}=***REDACTED***`)\n\n\t// Check if this looks like an Azure HTTP response error\n\t// Azure errors contain patterns like \"RESPONSE 404:\" and \"ERROR CODE:\"\n\tif !strings.Contains(errMsg, \"RESPONSE\") || !strings.Contains(errMsg, \"ERROR CODE:\") {\n\t\t// Not an Azure-style error, return as-is\n\t\treturn errMsg\n\t}\n\n\t// Extract the error code (e.g., \"ContainerNotFound\", \"BlobNotFound\")\n\terrorCodeRegex := regexp.MustCompile(`ERROR CODE:\\s*(\\w+)`)\n\terrorCodeMatch := errorCodeRegex.FindStringSubmatch(errMsg)\n\tvar errorCode string\n\tif len(errorCodeMatch) > 1 {\n\t\terrorCode = errorCodeMatch[1]\n\t}\n\n\t// Extract the error message from the XML or plain text\n\t// Look for message between tags or after \"RESPONSE XXX:\"\n\tvar errorMessage string\n\n\t// Try to extract from XML first\n\tmessageRegex := regexp.MustCompile(`(.*?)`)\n\tmessageMatch := messageRegex.FindStringSubmatch(errMsg)\n\tif len(messageMatch) > 1 {\n\t\terrorMessage = messageMatch[1]\n\t\t// Remove RequestId and Time from the message\n\t\tif idx := strings.Index(errorMessage, \"\\nRequestId:\"); idx != -1 {\n\t\t\terrorMessage = errorMessage[:idx]\n\t\t}\n\t} else {\n\t\t// Try to extract from plain text response (e.g., \"RESPONSE 404: 404 The specified container does not exist.\")\n\t\tresponseRegex := regexp.MustCompile(`RESPONSE\\s+\\d+:\\s+\\d+\\s+([^\\n]+)`)\n\t\tresponseMatch := responseRegex.FindStringSubmatch(errMsg)\n\t\tif len(responseMatch) > 1 {\n\t\t\terrorMessage = strings.TrimSpace(responseMatch[1])\n\t\t}\n\t}\n\n\t// Build a clean error message\n\tvar cleanMsg string\n\tif errorCode != \"\" && errorMessage != \"\" {\n\t\tcleanMsg = errorCode + \": \" + errorMessage\n\t} else if errorCode != \"\" {\n\t\tcleanMsg = errorCode\n\t} else if errorMessage != \"\" {\n\t\tcleanMsg = errorMessage\n\t} else {\n\t\t// Fallback: try to extract the desc part from gRPC error\n\t\tdescRegex := regexp.MustCompile(`desc\\s*=\\s*(.+)`)\n\t\tdescMatch := descRegex.FindStringSubmatch(errMsg)\n\t\tif len(descMatch) > 1 {\n\t\t\t// Take everything up to the first newline or \"RESPONSE\" marker\n\t\t\tdesc := descMatch[1]\n\t\t\tif idx := strings.Index(desc, \"\\n\"); idx != -1 {\n\t\t\t\tdesc = desc[:idx]\n\t\t\t}\n\t\t\tif idx := strings.Index(desc, \"RESPONSE\"); idx != -1 {\n\t\t\t\tdesc = strings.TrimSpace(desc[:idx])\n\t\t\t}\n\t\t\tcleanMsg = desc\n\t\t} else {\n\t\t\t// Last resort: return first line\n\t\t\tif idx := strings.Index(errMsg, \"\\n\"); idx != -1 {\n\t\t\t\tcleanMsg = errMsg[:idx]\n\t\t\t} else {\n\t\t\t\tcleanMsg = errMsg\n\t\t\t}\n\t\t}\n\t}\n\n\t// Preserve the prefix part of the error (e.g., \"rpc error: code = Unknown desc = \")\n\t// but replace the verbose description with our clean message\n\tif strings.Contains(errMsg, \"desc = \") {\n\t\tparts := strings.SplitN(errMsg, \"desc = \", 2)\n\t\tif len(parts) == 2 {\n\t\t\treturn parts[0] + \"desc = \" + cleanMsg\n\t\t}\n\t}\n\n\treturn cleanMsg\n}\n\n// BackupStorageLocationReconciler reconciles a BackupStorageLocation object\ntype backupStorageLocationReconciler struct {\n\tctx context.Context\n\tclient client.Client\n\tdefaultBackupLocationInfo storage.DefaultBackupLocationInfo\n\t// use variables to refer to these functions so they can be\n\t// replaced with fakes for testing.\n\tnewPluginManager func(logrus.FieldLogger) clientmgmt.Manager\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter\n\tmetrics *metrics.ServerMetrics\n\tlog logrus.FieldLogger\n}\n\n// NewBackupStorageLocationReconciler initialize and return a backupStorageLocationReconciler struct\nfunc NewBackupStorageLocationReconciler(\n\tctx context.Context,\n\tclient client.Client,\n\tdefaultBackupLocationInfo storage.DefaultBackupLocationInfo,\n\tnewPluginManager func(logrus.FieldLogger) clientmgmt.Manager,\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter,\n\tmetrics *metrics.ServerMetrics,\n\tlog logrus.FieldLogger) *backupStorageLocationReconciler {\n\treturn &backupStorageLocationReconciler{\n\t\tctx: ctx,\n\t\tclient: client,\n\t\tdefaultBackupLocationInfo: defaultBackupLocationInfo,\n\t\tnewPluginManager: newPluginManager,\n\t\tbackupStoreGetter: backupStoreGetter,\n\t\tmetrics: metrics,\n\t\tlog: log,\n\t}\n}\n\n// +kubebuilder:rbac:groups=velero.io,resources=backupstoragelocations,verbs=get;list;watch;create;update;patch;delete\n// +kubebuilder:rbac:groups=velero.io,resources=backupstoragelocations/status,verbs=get;update;patch\n\nfunc (r *backupStorageLocationReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\tvar unavailableErrors []string\n\tvar location velerov1api.BackupStorageLocation\n\n\tlog := r.log.WithField(\"controller\", constant.ControllerBackupStorageLocation).WithField(constant.ControllerBackupStorageLocation, req.NamespacedName.String())\n\tlog.Debug(\"Validating availability of BackupStorageLocation\")\n\n\tlocationList, err := storage.ListBackupStorageLocations(r.ctx, r.client, req.Namespace)\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"No BackupStorageLocations found, at least one is required\")\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\tpluginManager := r.newPluginManager(log)\n\tdefer pluginManager.CleanupClients()\n\n\t// find the BSL that matches the request\n\tfor _, bsl := range locationList.Items {\n\t\tif bsl.Name == req.Name && bsl.Namespace == req.Namespace {\n\t\t\tlocation = bsl\n\t\t}\n\t}\n\n\tif location.Name == \"\" || location.Namespace == \"\" {\n\t\tlog.WithError(err).Error(\"BackupStorageLocation is not found\")\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\t// decide the default BSL\n\tdefaultFound, err := r.ensureSingleDefaultBSL(locationList)\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"failed to ensure single default bsl\")\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\tfunc() {\n\t\tvar err error\n\t\toriginal := location.DeepCopy()\n\t\tdefer func() {\n\t\t\tlocation.Status.LastValidationTime = &metav1.Time{Time: time.Now().UTC()}\n\t\t\tif err != nil {\n\t\t\t\tlog.Info(\"BackupStorageLocation is invalid, marking as unavailable\")\n\t\t\t\terr = errors.Wrapf(err, \"BackupStorageLocation %q is unavailable\", location.Name)\n\t\t\t\tunavailableErrors = append(unavailableErrors, sanitizeStorageError(err))\n\t\t\t\tlocation.Status.Phase = velerov1api.BackupStorageLocationPhaseUnavailable\n\t\t\t\tlocation.Status.Message = sanitizeStorageError(err)\n\t\t\t} else {\n\t\t\t\tlog.Info(\"BackupStorageLocations is valid, marking as available\")\n\t\t\t\tlocation.Status.Phase = velerov1api.BackupStorageLocationPhaseAvailable\n\t\t\t\tlocation.Status.Message = \"\"\n\t\t\t}\n\t\t\tif err := r.client.Patch(r.ctx, &location, client.MergeFrom(original)); err != nil {\n\t\t\t\tlog.WithError(err).Error(\"Error updating BackupStorageLocation phase\")\n\t\t\t}\n\t\t}()\n\n\t\t// Validate the BackupStorageLocation spec\n\t\tif err = location.Validate(); err != nil {\n\t\t\tlog.WithError(err).Error(\"BackupStorageLocation spec is invalid\")\n\t\t\treturn\n\t\t}\n\n\t\tbackupStore, err := r.backupStoreGetter.Get(&location, pluginManager, log)\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Error(\"Error getting a backup store\")\n\t\t\treturn\n\t\t}\n\n\t\tlog.Info(\"Validating BackupStorageLocation\")\n\t\terr = backupStore.IsValid()\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Error(\"fail to validate backup store\")\n\t\t\treturn\n\t\t}\n\t}()\n\n\tr.logReconciledPhase(defaultFound, locationList, unavailableErrors)\n\n\treturn ctrl.Result{}, nil\n}\n\nfunc (r *backupStorageLocationReconciler) logReconciledPhase(defaultFound bool, locationList velerov1api.BackupStorageLocationList, errs []string) {\n\tvar availableBSLs []*velerov1api.BackupStorageLocation\n\tvar unAvailableBSLs []*velerov1api.BackupStorageLocation\n\tvar unknownBSLs []*velerov1api.BackupStorageLocation\n\tlog := r.log.WithField(\"controller\", constant.ControllerBackupStorageLocation)\n\n\tfor i, location := range locationList.Items {\n\t\tphase := location.Status.Phase\n\t\tswitch phase {\n\t\tcase velerov1api.BackupStorageLocationPhaseAvailable:\n\t\t\tavailableBSLs = append(availableBSLs, &locationList.Items[i])\n\t\t\tr.metrics.RegisterBackupLocationAvailable(locationList.Items[i].Name)\n\t\tcase velerov1api.BackupStorageLocationPhaseUnavailable:\n\t\t\tunAvailableBSLs = append(unAvailableBSLs, &locationList.Items[i])\n\t\t\tr.metrics.RegisterBackupLocationUnavailable(locationList.Items[i].Name)\n\t\tdefault:\n\t\t\tunknownBSLs = append(unknownBSLs, &locationList.Items[i])\n\t\t}\n\t}\n\n\tnumAvailable := len(availableBSLs)\n\tnumUnavailable := len(unAvailableBSLs)\n\tnumUnknown := len(unknownBSLs)\n\n\tif numUnavailable+numUnknown == len(locationList.Items) { // no available BSL\n\t\tif len(errs) > 0 {\n\t\t\tlog.Errorf(\"Current BackupStorageLocations available/unavailable/unknown: %v/%v/%v, %s)\", numAvailable, numUnavailable, numUnknown, strings.Join(errs, \"; \"))\n\t\t} else {\n\t\t\tlog.Errorf(\"Current BackupStorageLocations available/unavailable/unknown: %v/%v/%v)\", numAvailable, numUnavailable, numUnknown)\n\t\t}\n\t} else if numUnavailable > 0 { // some but not all BSL unavailable\n\t\tlog.Warnf(\"Unavailable BackupStorageLocations detected: available/unavailable/unknown: %v/%v/%v, %s)\", numAvailable, numUnavailable, numUnknown, strings.Join(errs, \"; \"))\n\t}\n\n\tif !defaultFound {\n\t\tlog.Warn(\"There is no existing BackupStorageLocation set as default. Please see `velero backup-location -h` for options.\")\n\t}\n}\n\nfunc (r *backupStorageLocationReconciler) SetupWithManager(mgr ctrl.Manager) error {\n\tgp := kube.NewGenericEventPredicate(func(object client.Object) bool {\n\t\tlocation := object.(*velerov1api.BackupStorageLocation)\n\t\treturn storage.IsReadyToValidate(location.Spec.ValidationFrequency, location.Status.LastValidationTime, r.defaultBackupLocationInfo.ServerValidationFrequency, r.log.WithField(\"controller\", constant.ControllerBackupStorageLocation))\n\t})\n\tg := kube.NewPeriodicalEnqueueSource(\n\t\tr.log.WithField(\"controller\", constant.ControllerBackupStorageLocation),\n\t\tmgr.GetClient(),\n\t\t&velerov1api.BackupStorageLocationList{},\n\t\tbslValidationEnqueuePeriod,\n\t\tkube.PeriodicalEnqueueSourceOption{\n\t\t\tPredicates: []predicate.Predicate{gp},\n\t\t},\n\t)\n\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\t// As the \"status.LastValidationTime\" field is always updated, this triggers new reconciling process, skip the update event that include no spec change to avoid the reconcile loop\n\t\tFor(&velerov1api.BackupStorageLocation{}, builder.WithPredicates(kube.SpecChangePredicate{})).\n\t\tWatchesRawSource(g).\n\t\tNamed(constant.ControllerBackupStorageLocation).\n\t\tComplete(r)\n}\n\n// ensureSingleDefaultBSL ensures that there is only one default BSL in the namespace.\n// the default BSL priority is as follows:\n// 1. follow the user's setting (the most recent validation BSL is the default BSL)\n// 2. follow the server's setting (\"velero server --default-backup-storage-location\")\nfunc (r *backupStorageLocationReconciler) ensureSingleDefaultBSL(locationList velerov1api.BackupStorageLocationList) (bool, error) {\n\t// get all default BSLs\n\tvar defaultBSLs []*velerov1api.BackupStorageLocation\n\tvar defaultFound bool\n\tfor i, location := range locationList.Items {\n\t\tif location.Spec.Default {\n\t\t\tdefaultBSLs = append(defaultBSLs, &locationList.Items[i])\n\t\t}\n\t}\n\n\tif len(defaultBSLs) > 1 { // more than 1 default BSL\n\t\t// find the most recent updated default BSL\n\t\tvar mostRecentCreatedBSL *velerov1api.BackupStorageLocation\n\t\tdefaultFound = true\n\t\tfor _, bsl := range defaultBSLs {\n\t\t\tif mostRecentCreatedBSL == nil {\n\t\t\t\tmostRecentCreatedBSL = bsl\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\t// For lack of a better way to compare timestamps, we use the CreationTimestamp\n\t\t\t// it cloud not really find the most recent updated BSL, but it is good enough for now\n\t\t\tbslTimestamp := bsl.CreationTimestamp\n\t\t\tmostRecentTimestamp := mostRecentCreatedBSL.CreationTimestamp\n\t\t\tif mostRecentTimestamp.Before(&bslTimestamp) {\n\t\t\t\tmostRecentCreatedBSL = bsl\n\t\t\t}\n\t\t}\n\n\t\t// unset all other default BSLs\n\t\tfor _, bsl := range defaultBSLs {\n\t\t\tif bsl.Name != mostRecentCreatedBSL.Name {\n\t\t\t\tbsl.Spec.Default = false\n\t\t\t\tif err := r.client.Update(r.ctx, bsl); err != nil {\n\t\t\t\t\treturn defaultFound, errors.Wrapf(err, \"failed to unset default backup storage location %q\", bsl.Name)\n\t\t\t\t}\n\t\t\t\tr.log.Debugf(\"update default backup storage location %q to false\", bsl.Name)\n\t\t\t}\n\t\t}\n\t} else if len(defaultBSLs) == 0 { // no default BSL\n\t\tdefaultFound = false\n\t} else { // only 1 default BSL\n\t\tdefaultFound = true\n\t}\n\treturn defaultFound, nil\n}\n" }, { "path": "pkg/controller/backup_storage_location_controller_test.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\n\t. \"github.com/onsi/ginkgo/v2\"\n\t. \"github.com/onsi/gomega\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/client-go/kubernetes/scheme\"\n\n\t\"github.com/vmware-tanzu/velero/internal/storage\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tpersistencemocks \"github.com/vmware-tanzu/velero/pkg/persistence/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\tpluginmocks \"github.com/vmware-tanzu/velero/pkg/plugin/mocks\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nvar _ = Describe(\"Backup Storage Location Reconciler\", func() {\n\tIt(\"Should successfully patch a backup storage location object status phase according to whether its storage is valid or not\", func() {\n\t\ttests := []struct {\n\t\t\tbackupLocation *velerov1api.BackupStorageLocation\n\t\t\tisValidError error\n\t\t\texpectedIsDefault bool\n\t\t\texpectedPhase velerov1api.BackupStorageLocationPhase\n\t\t}{\n\t\t\t{\n\t\t\t\tbackupLocation: builder.ForBackupStorageLocation(\"ns-1\", \"location-1\").ValidationFrequency(1 * time.Second).Default(true).Result(),\n\t\t\t\tisValidError: nil,\n\t\t\t\texpectedIsDefault: true,\n\t\t\t\texpectedPhase: velerov1api.BackupStorageLocationPhaseAvailable,\n\t\t\t},\n\t\t\t{\n\t\t\t\tbackupLocation: builder.ForBackupStorageLocation(\"ns-1\", \"location-2\").ValidationFrequency(1 * time.Second).Result(),\n\t\t\t\tisValidError: errors.New(\"an error\"),\n\t\t\t\texpectedIsDefault: false,\n\t\t\t\texpectedPhase: velerov1api.BackupStorageLocationPhaseUnavailable,\n\t\t\t},\n\t\t}\n\n\t\t// Setup\n\t\tvar (\n\t\t\tpluginManager = &pluginmocks.Manager{}\n\t\t\tbackupStores = make(map[string]*persistencemocks.BackupStore)\n\t\t)\n\t\tpluginManager.On(\"CleanupClients\").Return(nil)\n\n\t\tlocations := new(velerov1api.BackupStorageLocationList)\n\t\tfor i, test := range tests {\n\t\t\tlocation := test.backupLocation\n\t\t\tlocations.Items = append(locations.Items, *location)\n\t\t\tbackupStores[location.Name] = &persistencemocks.BackupStore{}\n\t\t\tbackupStore := backupStores[location.Name]\n\t\t\tbackupStore.On(\"IsValid\").Return(tests[i].isValidError)\n\t\t}\n\n\t\t// Setup reconciler\n\t\tExpect(velerov1api.AddToScheme(scheme.Scheme)).To(Succeed())\n\t\tr := backupStorageLocationReconciler{\n\t\t\tctx: ctx,\n\t\t\tclient: fake.NewClientBuilder().WithScheme(scheme.Scheme).WithRuntimeObjects(locations).Build(),\n\t\t\tdefaultBackupLocationInfo: storage.DefaultBackupLocationInfo{\n\t\t\t\tStorageLocation: \"location-1\",\n\t\t\t\tServerValidationFrequency: 0,\n\t\t\t},\n\t\t\tnewPluginManager: func(logrus.FieldLogger) clientmgmt.Manager { return pluginManager },\n\t\t\tbackupStoreGetter: NewFakeObjectBackupStoreGetter(backupStores),\n\t\t\tmetrics: metrics.NewServerMetrics(),\n\t\t\tlog: velerotest.NewLogger(),\n\t\t}\n\n\t\t// Assertions\n\t\tfor i, location := range locations.Items {\n\t\t\tactualResult, err := r.Reconcile(ctx, ctrl.Request{\n\t\t\t\tNamespacedName: types.NamespacedName{Namespace: location.Namespace, Name: location.Name},\n\t\t\t})\n\t\t\tExpect(actualResult).To(BeEquivalentTo(ctrl.Result{}))\n\t\t\tExpect(err).ToNot(HaveOccurred())\n\n\t\t\tkey := client.ObjectKey{Name: location.Name, Namespace: location.Namespace}\n\t\t\tinstance := &velerov1api.BackupStorageLocation{}\n\t\t\terr = r.client.Get(ctx, key, instance)\n\t\t\tExpect(err).ToNot(HaveOccurred())\n\t\t\tExpect(instance.Spec.Default).To(BeIdenticalTo(tests[i].expectedIsDefault))\n\t\t\tExpect(instance.Status.Phase).To(BeIdenticalTo(tests[i].expectedPhase))\n\t\t}\n\t})\n\n\tIt(\"Should successfully patch a backup storage location object spec default if the BSL is the default one\", func() {\n\t\ttests := []struct {\n\t\t\tbackupLocation *velerov1api.BackupStorageLocation\n\t\t\tisValidError error\n\t\t\texpectedIsDefault bool\n\t\t}{\n\t\t\t{\n\t\t\t\tbackupLocation: builder.ForBackupStorageLocation(\"ns-1\", \"location-1\").ValidationFrequency(1 * time.Second).Default(false).Result(),\n\t\t\t\tisValidError: nil,\n\t\t\t\texpectedIsDefault: false,\n\t\t\t},\n\t\t\t{\n\t\t\t\tbackupLocation: builder.ForBackupStorageLocation(\"ns-1\", \"location-2\").ValidationFrequency(1 * time.Second).Default(true).Result(),\n\t\t\t\tisValidError: nil,\n\t\t\t\texpectedIsDefault: true,\n\t\t\t},\n\t\t}\n\n\t\t// Setup\n\t\tvar (\n\t\t\tpluginManager = &pluginmocks.Manager{}\n\t\t\tbackupStores = make(map[string]*persistencemocks.BackupStore)\n\t\t)\n\t\tpluginManager.On(\"CleanupClients\").Return(nil)\n\n\t\tlocations := new(velerov1api.BackupStorageLocationList)\n\t\tfor i, test := range tests {\n\t\t\tlocation := test.backupLocation\n\t\t\tlocations.Items = append(locations.Items, *location)\n\t\t\tbackupStores[location.Name] = &persistencemocks.BackupStore{}\n\t\t\tbackupStore := backupStores[location.Name]\n\t\t\tbackupStore.On(\"IsValid\").Return(tests[i].isValidError)\n\t\t}\n\n\t\t// Setup reconciler\n\t\tExpect(velerov1api.AddToScheme(scheme.Scheme)).To(Succeed())\n\t\tr := backupStorageLocationReconciler{\n\t\t\tctx: ctx,\n\t\t\tclient: fake.NewClientBuilder().WithScheme(scheme.Scheme).WithRuntimeObjects(locations).Build(),\n\t\t\tdefaultBackupLocationInfo: storage.DefaultBackupLocationInfo{\n\t\t\t\tStorageLocation: \"default\",\n\t\t\t\tServerValidationFrequency: 0,\n\t\t\t},\n\t\t\tnewPluginManager: func(logrus.FieldLogger) clientmgmt.Manager { return pluginManager },\n\t\t\tbackupStoreGetter: NewFakeObjectBackupStoreGetter(backupStores),\n\t\t\tmetrics: metrics.NewServerMetrics(),\n\t\t\tlog: velerotest.NewLogger(),\n\t\t}\n\n\t\t// Assertions\n\t\tfor i, location := range locations.Items {\n\t\t\tactualResult, err := r.Reconcile(ctx, ctrl.Request{\n\t\t\t\tNamespacedName: types.NamespacedName{Namespace: location.Namespace, Name: location.Name},\n\t\t\t})\n\t\t\tExpect(actualResult).To(BeEquivalentTo(ctrl.Result{}))\n\t\t\tExpect(err).ToNot(HaveOccurred())\n\n\t\t\tkey := client.ObjectKey{Name: location.Name, Namespace: location.Namespace}\n\t\t\tinstance := &velerov1api.BackupStorageLocation{}\n\t\t\terr = r.client.Get(ctx, key, instance)\n\t\t\tExpect(err).ToNot(HaveOccurred())\n\t\t\tExpect(instance.Spec.Default).To(BeIdenticalTo(tests[i].expectedIsDefault))\n\t\t}\n\t})\n})\n\nfunc TestEnsureSingleDefaultBSL(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tlocations velerov1api.BackupStorageLocationList\n\t\tdefaultBackupInfo storage.DefaultBackupLocationInfo\n\t\texpectedDefaultSet bool\n\t\texpectedError error\n\t}{\n\t\t{\n\t\t\tname: \"MultipleDefaults\",\n\t\t\tlocations: func() velerov1api.BackupStorageLocationList {\n\t\t\t\tvar locations velerov1api.BackupStorageLocationList\n\t\t\t\tlocations.Items = append(locations.Items, *builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"location-1\").LastValidationTime(time.Now()).Default(true).Result())\n\t\t\t\tlocations.Items = append(locations.Items, *builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"location-2\").LastValidationTime(time.Now().Add(-1 * time.Hour)).Default(true).Result())\n\t\t\t\treturn locations\n\t\t\t}(),\n\t\t\texpectedDefaultSet: true,\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"NoDefault with exist default bsl in defaultBackupInfo\",\n\t\t\tlocations: func() velerov1api.BackupStorageLocationList {\n\t\t\t\tvar locations velerov1api.BackupStorageLocationList\n\t\t\t\tlocations.Items = append(locations.Items, *builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"location-1\").Default(false).Result())\n\t\t\t\tlocations.Items = append(locations.Items, *builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"location-2\").Default(false).Result())\n\t\t\t\treturn locations\n\t\t\t}(),\n\t\t\tdefaultBackupInfo: storage.DefaultBackupLocationInfo{\n\t\t\t\tStorageLocation: \"location-2\",\n\t\t\t},\n\t\t\texpectedDefaultSet: false,\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"NoDefault with non-exist default bsl in defaultBackupInfo\",\n\t\t\tlocations: func() velerov1api.BackupStorageLocationList {\n\t\t\t\tvar locations velerov1api.BackupStorageLocationList\n\t\t\t\tlocations.Items = append(locations.Items, *builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"location-1\").Default(false).Result())\n\t\t\t\tlocations.Items = append(locations.Items, *builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"location-2\").Default(false).Result())\n\t\t\t\treturn locations\n\t\t\t}(),\n\t\t\tdefaultBackupInfo: storage.DefaultBackupLocationInfo{\n\t\t\t\tStorageLocation: \"location-3\",\n\t\t\t},\n\t\t\texpectedDefaultSet: false,\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"SingleDefault\",\n\t\t\tlocations: func() velerov1api.BackupStorageLocationList {\n\t\t\t\tvar locations velerov1api.BackupStorageLocationList\n\t\t\t\tlocations.Items = append(locations.Items, *builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"location-1\").Default(true).Result())\n\t\t\t\tlocations.Items = append(locations.Items, *builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"location-2\").Default(false).Result())\n\t\t\t\treturn locations\n\t\t\t}(),\n\t\t\texpectedDefaultSet: true,\n\t\t\texpectedError: nil,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\t// Setup reconciler\n\t\trequire.NoError(t, velerov1api.AddToScheme(scheme.Scheme))\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tr := &backupStorageLocationReconciler{\n\t\t\t\tctx: t.Context(),\n\t\t\t\tclient: fake.NewClientBuilder().WithScheme(scheme.Scheme).WithRuntimeObjects(&test.locations).Build(),\n\t\t\t\tdefaultBackupLocationInfo: test.defaultBackupInfo,\n\t\t\t\tmetrics: metrics.NewServerMetrics(),\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\t\t\tdefaultFound, err := r.ensureSingleDefaultBSL(test.locations)\n\n\t\t\tassert.Equal(t, test.expectedDefaultSet, defaultFound)\n\t\t\tassert.Equal(t, test.expectedError, err)\n\t\t})\n\t}\n}\n\nfunc TestBSLReconcile(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tlocationList velerov1api.BackupStorageLocationList\n\t\tdefaultFound bool\n\t\texpectedError error\n\t}{\n\t\t{\n\t\t\tname: \"NoBSL\",\n\t\t\tlocationList: velerov1api.BackupStorageLocationList{},\n\t\t\tdefaultFound: false,\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"BSLNotFound\",\n\t\t\tlocationList: func() velerov1api.BackupStorageLocationList {\n\t\t\t\tvar locations velerov1api.BackupStorageLocationList\n\t\t\t\tlocations.Items = append(locations.Items, *builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"location-2\").Result())\n\t\t\t\treturn locations\n\t\t\t}(),\n\t\t\tdefaultFound: false,\n\t\t\texpectedError: nil,\n\t\t},\n\t}\n\tpluginManager := &pluginmocks.Manager{}\n\tpluginManager.On(\"CleanupClients\").Return(nil)\n\tfor _, test := range tests {\n\t\t// Setup reconciler\n\t\trequire.NoError(t, velerov1api.AddToScheme(scheme.Scheme))\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tr := &backupStorageLocationReconciler{\n\t\t\t\tctx: t.Context(),\n\t\t\t\tclient: fake.NewClientBuilder().WithScheme(scheme.Scheme).WithRuntimeObjects(&test.locationList).Build(),\n\t\t\t\tnewPluginManager: func(logrus.FieldLogger) clientmgmt.Manager { return pluginManager },\n\t\t\t\tmetrics: metrics.NewServerMetrics(),\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tresult, err := r.Reconcile(t.Context(), ctrl.Request{NamespacedName: types.NamespacedName{Namespace: velerov1api.DefaultNamespace, Name: \"location-1\"}})\n\t\t\tassert.Equal(t, test.expectedError, err)\n\t\t\tassert.Equal(t, ctrl.Result{}, result)\n\t\t})\n\t}\n}\n\nfunc TestSanitizeStorageError(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tinput error\n\t\texpected string\n\t}{\n\t\t{\n\t\t\tname: \"Nil error\",\n\t\t\tinput: nil,\n\t\t\texpected: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"Simple error without Azure formatting\",\n\t\t\tinput: errors.New(\"simple error message\"),\n\t\t\texpected: \"simple error message\",\n\t\t},\n\t\t{\n\t\t\tname: \"AWS style error\",\n\t\t\tinput: errors.New(\"NoSuchBucket: The specified bucket does not exist\"),\n\t\t\texpected: \"NoSuchBucket: The specified bucket does not exist\",\n\t\t},\n\t\t{\n\t\t\tname: \"Azure container not found error with full HTTP response\",\n\t\t\tinput: errors.New(`rpc error: code = Unknown desc = GET https://oadp100711zl59k.blob.core.windows.net/oadp100711zl59k1\n--------------------------------------------------------------------------------\nRESPONSE 404: 404 The specified container does not exist.\nERROR CODE: ContainerNotFound\n--------------------------------------------------------------------------------\nContainerNotFoundThe specified container does not exist.\nRequestId:63cf34d8-801e-0078-09b4-2e4682000000\nTime:2024-11-04T12:23:04.5623627Z\n--------------------------------------------------------------------------------\n`),\n\t\t\texpected: \"rpc error: code = Unknown desc = ContainerNotFound: The specified container does not exist.\",\n\t\t},\n\t\t{\n\t\t\tname: \"Azure blob not found error\",\n\t\t\tinput: errors.New(`rpc error: code = Unknown desc = GET https://storage.blob.core.windows.net/container/blob\n--------------------------------------------------------------------------------\nRESPONSE 404: 404 The specified blob does not exist.\nERROR CODE: BlobNotFound\n--------------------------------------------------------------------------------\nBlobNotFoundThe specified blob does not exist.\nRequestId:12345678-1234-1234-1234-123456789012\nTime:2024-11-04T12:23:04.5623627Z\n--------------------------------------------------------------------------------\n`),\n\t\t\texpected: \"rpc error: code = Unknown desc = BlobNotFound: The specified blob does not exist.\",\n\t\t},\n\t\t{\n\t\t\tname: \"Azure error with plain text response (no XML)\",\n\t\t\tinput: errors.New(`rpc error: code = Unknown desc = GET https://storage.blob.core.windows.net/container\n--------------------------------------------------------------------------------\nRESPONSE 404: 404 The specified container does not exist.\nERROR CODE: ContainerNotFound\n--------------------------------------------------------------------------------\n`),\n\t\t\texpected: \"rpc error: code = Unknown desc = ContainerNotFound: The specified container does not exist.\",\n\t\t},\n\t\t{\n\t\t\tname: \"Azure error without XML message but with error code\",\n\t\t\tinput: errors.New(`rpc error: code = Unknown desc = operation failed\nRESPONSE 403: 403 Forbidden\nERROR CODE: AuthorizationFailure\n--------------------------------------------------------------------------------\n`),\n\t\t\texpected: \"rpc error: code = Unknown desc = AuthorizationFailure: Forbidden\",\n\t\t},\n\t\t{\n\t\t\tname: \"Error with Azure SAS token in URL\",\n\t\t\tinput: errors.New(`rpc error: code = Unknown desc = GET https://storage.blob.core.windows.net/backup?sv=2020-08-04&sig=abc123secrettoken&se=2024-12-31T23:59:59Z&sp=rwdl\n--------------------------------------------------------------------------------\nRESPONSE 404: 404 The specified container does not exist.\nERROR CODE: ContainerNotFound\n--------------------------------------------------------------------------------\n`),\n\t\t\texpected: \"rpc error: code = Unknown desc = ContainerNotFound: The specified container does not exist.\",\n\t\t},\n\t\t{\n\t\t\tname: \"Error with multiple SAS parameters\",\n\t\t\tinput: errors.New(`GET https://mystorageaccount.blob.core.windows.net/container?sv=2020-08-04&ss=b&srt=sco&sp=rwdlac&se=2024-12-31&st=2024-01-01&sip=168.1.5.60&spr=https&sig=SIGNATURE_HASH`),\n\t\t\texpected: \"GET https://mystorageaccount.blob.core.windows.net/container?sv=***REDACTED***&ss=***REDACTED***&srt=***REDACTED***&sp=***REDACTED***&se=***REDACTED***&st=***REDACTED***&sip=***REDACTED***&spr=***REDACTED***&sig=***REDACTED***\",\n\t\t},\n\t\t{\n\t\t\tname: \"Simple URL without SAS tokens unchanged\",\n\t\t\tinput: errors.New(\"GET https://storage.blob.core.windows.net/container/blob\"),\n\t\t\texpected: \"GET https://storage.blob.core.windows.net/container/blob\",\n\t\t},\n\t\t{\n\t\t\tname: \"Azure error with SAS token in full HTTP response\",\n\t\t\tinput: errors.New(`rpc error: code = Unknown desc = GET https://oadp100711zl59k.blob.core.windows.net/backup?sig=secretsignature123&se=2024-12-31\n--------------------------------------------------------------------------------\nRESPONSE 404: 404 The specified container does not exist.\nERROR CODE: ContainerNotFound\n--------------------------------------------------------------------------------\nContainerNotFoundThe specified container does not exist.\nRequestId:63cf34d8-801e-0078-09b4-2e4682000000\nTime:2024-11-04T12:23:04.5623627Z\n--------------------------------------------------------------------------------\n`),\n\t\t\texpected: \"rpc error: code = Unknown desc = ContainerNotFound: The specified container does not exist.\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tactual := sanitizeStorageError(test.input)\n\t\t\tassert.Equal(t, test.expected, actual)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/controller/backup_sync_controller.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\t\"k8s.io/apimachinery/pkg/api/meta\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\t\"sigs.k8s.io/controller-runtime/pkg/builder\"\n\t\"sigs.k8s.io/controller-runtime/pkg/predicate\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\t\"github.com/vmware-tanzu/velero/pkg/persistence\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\tveleroutil \"github.com/vmware-tanzu/velero/pkg/util/velero\"\n\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n)\n\nconst (\n\tbackupSyncReconcilePeriod = time.Minute\n)\n\ntype backupSyncReconciler struct {\n\tclient client.Client\n\tnamespace string\n\tdefaultBackupSyncPeriod time.Duration\n\tnewPluginManager func(logrus.FieldLogger) clientmgmt.Manager\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter\n\tlogger logrus.FieldLogger\n}\n\n// NewBackupSyncReconciler is used to generate BackupSync reconciler structure.\nfunc NewBackupSyncReconciler(\n\tclient client.Client,\n\tnamespace string,\n\tdefaultBackupSyncPeriod time.Duration,\n\tnewPluginManager func(logrus.FieldLogger) clientmgmt.Manager,\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter,\n\tlogger logrus.FieldLogger) *backupSyncReconciler {\n\treturn &backupSyncReconciler{\n\t\tclient: client,\n\t\tnamespace: namespace,\n\t\tdefaultBackupSyncPeriod: defaultBackupSyncPeriod,\n\t\tnewPluginManager: newPluginManager,\n\t\tbackupStoreGetter: backupStoreGetter,\n\t\tlogger: logger,\n\t}\n}\n\n// Reconcile syncs between the backups in cluster and backups metadata in object store.\nfunc (b *backupSyncReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\tlog := b.logger.WithField(\"controller\", constant.ControllerBackupSync)\n\tlog = log.WithField(\"backupLocation\", req.String())\n\tlog.Debug(\"Begin to sync between backups' metadata in BSL object storage and cluster's existing backups.\")\n\n\tlocation := &velerov1api.BackupStorageLocation{}\n\terr := b.client.Get(ctx, req.NamespacedName, location)\n\tif err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.Debug(\"BackupStorageLocation is not found\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\treturn ctrl.Result{}, errors.Wrapf(err, \"error getting BackupStorageLocation %s\", req.String())\n\t}\n\tif !veleroutil.BSLIsAvailable(*location) {\n\t\tlog.Errorf(\"BackupStorageLocation is in unavailable state, skip syncing backup from it.\")\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\tpluginManager := b.newPluginManager(log)\n\tdefer pluginManager.CleanupClients()\n\n\tlog.Debug(\"Checking backup location for backups to sync into cluster\")\n\n\tbackupStore, err := b.backupStoreGetter.Get(location, pluginManager, log)\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"Error getting backup store for this location\")\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\t// get a list of all the backups that are stored in the backup storage location\n\tres, err := backupStore.ListBackups()\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"Error listing backups in backup store\")\n\t\treturn ctrl.Result{}, nil\n\t}\n\tbackupStoreBackups := sets.New[string](res...)\n\tlog.WithField(\"backupCount\", len(backupStoreBackups)).Debug(\"Got backups from backup store\")\n\n\t// get a list of all the backups that exist as custom resources in the cluster\n\tvar clusterBackupList velerov1api.BackupList\n\tlistOption := client.ListOptions{\n\t\tLabelSelector: labels.Everything(),\n\t\tNamespace: b.namespace,\n\t}\n\n\terr = b.client.List(ctx, &clusterBackupList, &listOption)\n\tif err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Error(\"Error getting backups from cluster, proceeding with sync into cluster\")\n\t} else {\n\t\tlog.WithField(\"backupCount\", len(clusterBackupList.Items)).Debug(\"Got backups from cluster\")\n\t}\n\n\t// get a list of backups that *are* in the backup storage location and *aren't* in the cluster\n\tclusterBackupsSet := sets.New[string]()\n\tfor _, b := range clusterBackupList.Items {\n\t\tclusterBackupsSet.Insert(b.Name)\n\t}\n\tbackupsToSync := backupStoreBackups.Difference(clusterBackupsSet)\n\n\tif count := backupsToSync.Len(); count > 0 {\n\t\tlog.Infof(\"Found %v backups in the backup location that do not exist in the cluster and need to be synced\", count)\n\t} else {\n\t\tlog.Debug(\"No backups found in the backup location that need to be synced into the cluster\")\n\t}\n\n\t// sync each backup\n\tfor backupName := range backupsToSync {\n\t\tlog = log.WithField(\"backup\", backupName)\n\t\tlog.Info(\"Attempting to sync backup into cluster\")\n\n\t\texist, err := backupStore.BackupExists(location.Spec.ObjectStorage.Bucket, backupName)\n\t\tif err != nil {\n\t\t\tlog.WithError(errors.WithStack(err)).Error(\"Error checking backup exist from backup store\")\n\t\t\tcontinue\n\t\t}\n\t\tif !exist {\n\t\t\tlog.Debugf(\"backup %s doesn't exist in backup store, skip\", backupName)\n\t\t\tcontinue\n\t\t}\n\n\t\tbackup, err := backupStore.GetBackupMetadata(backupName)\n\t\tif err != nil {\n\t\t\tlog.WithError(errors.WithStack(err)).Error(\"Error getting backup metadata from backup store\")\n\t\t\tcontinue\n\t\t}\n\n\t\tif backup.Status.Phase == velerov1api.BackupPhaseWaitingForPluginOperations ||\n\t\t\tbackup.Status.Phase == velerov1api.BackupPhaseWaitingForPluginOperationsPartiallyFailed ||\n\t\t\tbackup.Status.Phase == velerov1api.BackupPhaseFinalizing ||\n\t\t\tbackup.Status.Phase == velerov1api.BackupPhaseFinalizingPartiallyFailed {\n\t\t\tif backup.Status.Expiration == nil || backup.Status.Expiration.After(time.Now()) {\n\t\t\t\tlog.Debugf(\"Skipping non-expired incomplete backup %v\", backup.Name)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tlog.Debugf(\"%v Backup is past expiration, syncing for garbage collection\", backup.Status.Phase)\n\t\t\tbackup.Status.Phase = velerov1api.BackupPhasePartiallyFailed\n\t\t}\n\t\tbackup.Namespace = b.namespace\n\t\tbackup.ResourceVersion = \"\"\n\n\t\t// update the StorageLocation field and label since the name of the location\n\t\t// may be different in this cluster than in the cluster that created the\n\t\t// backup.\n\t\tbackup.Spec.StorageLocation = location.Name\n\t\tif backup.Labels == nil {\n\t\t\tbackup.Labels = make(map[string]string)\n\t\t}\n\t\tbackup.Labels[velerov1api.StorageLocationLabel] = label.GetValidName(backup.Spec.StorageLocation)\n\n\t\t//check for the ownership references. If they do not exist, remove them.\n\t\tbackup.ObjectMeta.OwnerReferences = b.filterBackupOwnerReferences(ctx, backup, log)\n\n\t\t// attempt to create backup custom resource via API\n\t\terr = b.client.Create(ctx, backup, &client.CreateOptions{})\n\t\tswitch {\n\t\tcase err != nil && apierrors.IsAlreadyExists(err):\n\t\t\tlog.Debug(\"Backup already exists in cluster\")\n\t\t\tcontinue\n\t\tcase err != nil && !apierrors.IsAlreadyExists(err):\n\t\t\tlog.WithError(errors.WithStack(err)).Error(\"Error syncing backup into cluster\")\n\t\t\tcontinue\n\t\tdefault:\n\t\t\tlog.Info(\"Successfully synced backup into cluster\")\n\t\t}\n\n\t\t// process the pod volume backups from object store, if any\n\t\tpodVolumeBackups, err := backupStore.GetPodVolumeBackups(backupName)\n\t\tif err != nil {\n\t\t\tlog.WithError(errors.WithStack(err)).Error(\"Error getting pod volume backups for this backup from backup store\")\n\t\t\tcontinue\n\t\t}\n\n\t\tfor _, podVolumeBackup := range podVolumeBackups {\n\t\t\tlog := log.WithField(\"podVolumeBackup\", podVolumeBackup.Name)\n\t\t\tlog.Debug(\"Checking this pod volume backup to see if it needs to be synced into the cluster\")\n\n\t\t\tfor i, ownerRef := range podVolumeBackup.OwnerReferences {\n\t\t\t\tif ownerRef.APIVersion == velerov1api.SchemeGroupVersion.String() && ownerRef.Kind == \"Backup\" && ownerRef.Name == backup.Name {\n\t\t\t\t\tlog.WithField(\"uid\", backup.UID).Debugf(\"Updating pod volume backup's owner reference UID\")\n\t\t\t\t\tpodVolumeBackup.OwnerReferences[i].UID = backup.UID\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif _, ok := podVolumeBackup.Labels[velerov1api.BackupUIDLabel]; ok {\n\t\t\t\tpodVolumeBackup.Labels[velerov1api.BackupUIDLabel] = string(backup.UID)\n\t\t\t}\n\n\t\t\tpodVolumeBackup.Namespace = backup.Namespace\n\t\t\tpodVolumeBackup.ResourceVersion = \"\"\n\t\t\tpodVolumeBackup.Spec.BackupStorageLocation = location.Name\n\n\t\t\terr = b.client.Create(ctx, podVolumeBackup, &client.CreateOptions{})\n\t\t\tswitch {\n\t\t\tcase err != nil && apierrors.IsAlreadyExists(err):\n\t\t\t\tlog.Debug(\"Pod volume backup already exists in cluster\")\n\t\t\t\tcontinue\n\t\t\tcase err != nil && !apierrors.IsAlreadyExists(err):\n\t\t\t\tlog.WithError(errors.WithStack(err)).Error(\"Error syncing pod volume backup into cluster\")\n\t\t\t\tcontinue\n\t\t\tdefault:\n\t\t\t\tlog.Debug(\"Synced pod volume backup into cluster\")\n\t\t\t}\n\t\t}\n\t}\n\n\tb.deleteOrphanedBackups(ctx, location.Name, backupStoreBackups, log)\n\n\t// update the location's last-synced time field\n\tstatusPatch := client.MergeFrom(location.DeepCopy())\n\tlocation.Status.LastSyncedTime = &metav1.Time{Time: time.Now().UTC()}\n\tif err := b.client.Patch(ctx, location, statusPatch); err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Error(\"Error patching backup location's last-synced time\")\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\treturn ctrl.Result{}, nil\n}\n\nfunc (b *backupSyncReconciler) filterBackupOwnerReferences(ctx context.Context, backup *velerov1api.Backup, log logrus.FieldLogger) []metav1.OwnerReference {\n\tlistedReferences := backup.ObjectMeta.OwnerReferences\n\tfoundReferences := make([]metav1.OwnerReference, 0)\n\tfor _, v := range listedReferences {\n\t\tswitch v.Kind {\n\t\tcase \"Schedule\":\n\t\t\tschedule := new(velerov1api.Schedule)\n\t\t\terr := b.client.Get(ctx, types.NamespacedName{\n\t\t\t\tName: v.Name,\n\t\t\t\tNamespace: backup.Namespace,\n\t\t\t}, schedule)\n\t\t\tswitch {\n\t\t\tcase err != nil && apierrors.IsNotFound(err):\n\t\t\t\tlog.Warnf(\"Removing missing schedule ownership reference %s/%s from backup\", backup.Namespace, v.Name)\n\t\t\t\tcontinue\n\t\t\tcase schedule.UID != v.UID:\n\t\t\t\tlog.Warnf(\"Removing schedule ownership reference with mismatched UIDs. Expected %s, got %s\", v.UID, schedule.UID)\n\t\t\t\tcontinue\n\t\t\tcase err != nil && !apierrors.IsNotFound(err):\n\t\t\t\tlog.WithError(errors.WithStack(err)).Error(\"Error finding schedule ownership reference, keeping schedule on backup\")\n\t\t\t}\n\t\tdefault:\n\t\t\tlog.Warnf(\"Unable to check ownership reference for unknown kind, %s\", v.Kind)\n\t\t}\n\t\tfoundReferences = append(foundReferences, v)\n\t}\n\treturn foundReferences\n}\n\n// SetupWithManager is used to setup controller and its watching sources.\nfunc (b *backupSyncReconciler) SetupWithManager(mgr ctrl.Manager) error {\n\tgp := kube.NewGenericEventPredicate(func(object client.Object) bool {\n\t\tlocation := object.(*velerov1api.BackupStorageLocation)\n\t\treturn b.locationFilterFunc(location)\n\t})\n\tbackupSyncSource := kube.NewPeriodicalEnqueueSource(\n\t\tb.logger.WithField(\"controller\", constant.ControllerBackupSync),\n\t\tmgr.GetClient(),\n\t\t&velerov1api.BackupStorageLocationList{},\n\t\tbackupSyncReconcilePeriod,\n\t\tkube.PeriodicalEnqueueSourceOption{\n\t\t\tOrderFunc: backupSyncSourceOrderFunc,\n\t\t\tPredicates: []predicate.Predicate{gp},\n\t\t},\n\t)\n\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\t// Filter all BSL events, because this controller is supposed to run periodically, not by event.\n\t\tFor(&velerov1api.BackupStorageLocation{}, builder.WithPredicates(kube.FalsePredicate{})).\n\t\tWatchesRawSource(backupSyncSource).\n\t\tNamed(constant.ControllerBackupSync).\n\t\tComplete(b)\n}\n\n// deleteOrphanedBackups deletes backup objects (CRDs) from Kubernetes that have the specified location\n// and a phase of Completed, but no corresponding backup in object storage.\nfunc (b *backupSyncReconciler) deleteOrphanedBackups(ctx context.Context, locationName string, backupStoreBackups sets.Set[string], log logrus.FieldLogger) {\n\tvar backupList velerov1api.BackupList\n\tlistOption := client.ListOptions{\n\t\tLabelSelector: labels.Set(map[string]string{\n\t\t\tvelerov1api.StorageLocationLabel: label.GetValidName(locationName),\n\t\t}).AsSelector(),\n\t}\n\terr := b.client.List(ctx, &backupList, &listOption)\n\tif err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Error(\"Error listing backups from cluster\")\n\t\treturn\n\t}\n\n\tif len(backupList.Items) == 0 {\n\t\treturn\n\t}\n\n\tfor i, backup := range backupList.Items {\n\t\tlog = log.WithField(\"backup\", backup.Name)\n\t\tif !(backup.Status.Phase == velerov1api.BackupPhaseCompleted || backup.Status.Phase == velerov1api.BackupPhasePartiallyFailed) || backupStoreBackups.Has(backup.Name) {\n\t\t\tcontinue\n\t\t}\n\n\t\tif err := b.client.Delete(ctx, &backupList.Items[i], &client.DeleteOptions{}); err != nil {\n\t\t\tlog.WithError(errors.WithStack(err)).Error(\"Error deleting orphaned backup from cluster\")\n\t\t}\n\t}\n}\n\n// backupSyncSourceOrderFunc returns a new slice with the default backup location first (if it exists),\n// followed by the rest of the locations in no particular order.\nfunc backupSyncSourceOrderFunc(objList client.ObjectList) client.ObjectList {\n\tinputBSLList := objList.(*velerov1api.BackupStorageLocationList)\n\tresultBSLList := &velerov1api.BackupStorageLocationList{}\n\tbslArray := make([]runtime.Object, 0)\n\n\tif len(inputBSLList.Items) <= 0 {\n\t\treturn objList\n\t}\n\n\tfor i := range inputBSLList.Items {\n\t\tlocation := inputBSLList.Items[i]\n\n\t\t// sync the default backup storage location first, if it exists\n\t\tif location.Spec.Default {\n\t\t\t// put the default location first\n\t\t\tbslArray = append(bslArray, &inputBSLList.Items[i])\n\t\t\t// append everything before the default\n\t\t\tfor _, bsl := range inputBSLList.Items[:i] {\n\t\t\t\tcpBsl := bsl\n\t\t\t\tbslArray = append(bslArray, &cpBsl)\n\t\t\t}\n\t\t\t// append everything after the default\n\t\t\tfor _, bsl := range inputBSLList.Items[i+1:] {\n\t\t\t\tcpBsl := bsl\n\t\t\t\tbslArray = append(bslArray, &cpBsl)\n\t\t\t}\n\t\t\tif err := meta.SetList(resultBSLList, bslArray); err != nil {\n\t\t\t\tfmt.Printf(\"fail to sort BSL list: %s\", err.Error())\n\t\t\t\treturn &velerov1api.BackupStorageLocationList{}\n\t\t\t}\n\n\t\t\treturn resultBSLList\n\t\t}\n\t}\n\n\t// No default BSL found. Return the input.\n\treturn objList\n}\n\nfunc (b *backupSyncReconciler) locationFilterFunc(location *velerov1api.BackupStorageLocation) bool {\n\tsyncPeriod := b.defaultBackupSyncPeriod\n\tif location.Spec.BackupSyncPeriod != nil {\n\t\tsyncPeriod = location.Spec.BackupSyncPeriod.Duration\n\t\tif syncPeriod == 0 {\n\t\t\tb.logger.Debug(\"Backup sync period for this location is set to 0, skipping sync\")\n\t\t\treturn false\n\t\t}\n\n\t\tif syncPeriod < 0 {\n\t\t\tb.logger.Debug(\"Backup sync period must be non-negative\")\n\t\t\tsyncPeriod = b.defaultBackupSyncPeriod\n\t\t}\n\t}\n\n\tlastSync := location.Status.LastSyncedTime\n\tif lastSync != nil {\n\t\tb.logger.Debug(\"Checking if backups need to be synced at this time for this location\")\n\t\tnextSync := lastSync.Add(syncPeriod)\n\t\tif time.Now().UTC().Before(nextSync) {\n\t\t\treturn false\n\t\t}\n\t}\n\treturn true\n}\n" }, { "path": "pkg/controller/backup_sync_controller_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\t. \"github.com/onsi/ginkgo/v2\"\n\t. \"github.com/onsi/gomega\"\n\n\t\"github.com/sirupsen/logrus\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\t\"k8s.io/apimachinery/pkg/api/meta\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\t\"k8s.io/apimachinery/pkg/util/validation\"\n\ttestclocks \"k8s.io/utils/clock/testing\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\tctrlClient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\tctrlfake \"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\tpersistencemocks \"github.com/vmware-tanzu/velero/pkg/persistence/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\tpluginmocks \"github.com/vmware-tanzu/velero/pkg/plugin/mocks\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc defaultLocation(namespace string) *velerov1api.BackupStorageLocation {\n\treturn &velerov1api.BackupStorageLocation{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: namespace,\n\t\t\tName: \"location-1\",\n\t\t},\n\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\tProvider: \"objStoreProvider\",\n\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\tBucket: \"bucket-1\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tDefault: true,\n\t\t},\n\t\tStatus: velerov1api.BackupStorageLocationStatus{\n\t\t\tPhase: velerov1api.BackupStorageLocationPhaseAvailable,\n\t\t},\n\t}\n}\n\nfunc defaultLocationsList(namespace string) []*velerov1api.BackupStorageLocation {\n\treturn []*velerov1api.BackupStorageLocation{\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: namespace,\n\t\t\t\tName: \"location-0\",\n\t\t\t},\n\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\tProvider: \"objStoreProvider\",\n\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\tBucket: \"bucket-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: namespace,\n\t\t\t\tName: \"location-1\",\n\t\t\t},\n\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\tProvider: \"objStoreProvider\",\n\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\tBucket: \"bucket-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tDefault: true,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: namespace,\n\t\t\t\tName: \"location-2\",\n\t\t\t},\n\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\tProvider: \"objStoreProvider\",\n\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\tBucket: \"bucket-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: namespace,\n\t\t\t\tName: \"location-3\",\n\t\t\t},\n\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\tProvider: \"objStoreProvider\",\n\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\tBucket: \"bucket-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n}\n\nfunc defaultLocationWithLongerLocationName(namespace string) *velerov1api.BackupStorageLocation {\n\treturn &velerov1api.BackupStorageLocation{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: namespace,\n\t\t\tName: \"the-really-long-location-name-that-is-much-more-than-63-characters-1\",\n\t\t},\n\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\tProvider: \"objStoreProvider\",\n\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\tBucket: \"bucket-1\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tStatus: velerov1api.BackupStorageLocationStatus{\n\t\t\tPhase: velerov1api.BackupStorageLocationPhaseAvailable,\n\t\t},\n\t}\n}\n\nfunc numBackups(c ctrlClient.WithWatch) (int, error) {\n\tvar existingK8SBackups velerov1api.BackupList\n\terr := c.List(context.TODO(), &existingK8SBackups, &ctrlClient.ListOptions{})\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\treturn len(existingK8SBackups.Items), nil\n}\n\nvar _ = Describe(\"Backup Sync Reconciler\", func() {\n\tIt(\"Test Backup Sync Reconciler basic function\", func() {\n\t\tfakeClock := testclocks.NewFakeClock(time.Now())\n\t\ttype cloudBackupData struct {\n\t\t\tbackup *velerov1api.Backup\n\t\t\tpodVolumeBackups []*velerov1api.PodVolumeBackup\n\t\t\tbackupShouldSkipSync bool // backups waiting for plugin operations should not sync\n\t\t}\n\n\t\ttests := []struct {\n\t\t\tname string\n\t\t\tnamespace string\n\t\t\tlocation *velerov1api.BackupStorageLocation\n\t\t\tcloudBackups []*cloudBackupData\n\t\t\texistingBackups []*velerov1api.Backup\n\t\t\texistingPodVolumeBackups []*velerov1api.PodVolumeBackup\n\t\t\tlongLocationNameEnabled bool\n\t\t}{\n\t\t\t{\n\t\t\t\tname: \"no cloud backups\",\n\t\t\t\tnamespace: \"ns-1\",\n\t\t\t\tlocation: defaultLocation(\"ns-1\"),\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"unavailable BSL\",\n\t\t\t\tnamespace: \"ns-1\",\n\t\t\t\tlocation: builder.ForBackupStorageLocation(\"ns-1\", \"default\").Phase(velerov1api.BackupStorageLocationPhaseUnavailable).Result(),\n\t\t\t\tcloudBackups: []*cloudBackupData{\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-1\").Result(),\n\t\t\t\t\t\tbackupShouldSkipSync: true,\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-2\").Result(),\n\t\t\t\t\t\tbackupShouldSkipSync: true,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"normal case\",\n\t\t\t\tnamespace: \"ns-1\",\n\t\t\t\tlocation: defaultLocation(\"ns-1\"),\n\t\t\t\tcloudBackups: []*cloudBackupData{\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-1\").Result(),\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-2\").Result(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"backups waiting for plugin operations aren't synced\",\n\t\t\t\tnamespace: \"ns-1\",\n\t\t\t\tlocation: defaultLocation(\"ns-1\"),\n\t\t\t\tcloudBackups: []*cloudBackupData{\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-1\").\n\t\t\t\t\t\t\tPhase(velerov1api.BackupPhaseWaitingForPluginOperations).Result(),\n\t\t\t\t\t\tbackupShouldSkipSync: true,\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-2\").\n\t\t\t\t\t\t\tPhase(velerov1api.BackupPhaseWaitingForPluginOperationsPartiallyFailed).Result(),\n\t\t\t\t\t\tbackupShouldSkipSync: true,\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-3\").\n\t\t\t\t\t\t\tPhase(velerov1api.BackupPhaseWaitingForPluginOperations).Result(),\n\t\t\t\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\t\t\t\tbuilder.ForPodVolumeBackup(\"ns-1\", \"pvb-1\").Result(),\n\t\t\t\t\t\t},\n\t\t\t\t\t\tbackupShouldSkipSync: true,\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-4\").\n\t\t\t\t\t\t\tPhase(velerov1api.BackupPhaseFinalizing).Result(),\n\t\t\t\t\t\tbackupShouldSkipSync: true,\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-5\").\n\t\t\t\t\t\t\tPhase(velerov1api.BackupPhaseFinalizingPartiallyFailed).Result(),\n\t\t\t\t\t\tbackupShouldSkipSync: true,\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-6\").\n\t\t\t\t\t\t\tPhase(velerov1api.BackupPhaseFinalizing).Result(),\n\t\t\t\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\t\t\t\tbuilder.ForPodVolumeBackup(\"ns-1\", \"pvb-2\").Result(),\n\t\t\t\t\t\t},\n\t\t\t\t\t\tbackupShouldSkipSync: true,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"expired backups waiting for plugin operations are synced\",\n\t\t\t\tnamespace: \"ns-1\",\n\t\t\t\tlocation: defaultLocation(\"ns-1\"),\n\t\t\t\tcloudBackups: []*cloudBackupData{\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-1\").\n\t\t\t\t\t\t\tPhase(velerov1api.BackupPhaseWaitingForPluginOperations).\n\t\t\t\t\t\t\tExpiration(fakeClock.Now().Add(-time.Hour)).Result(),\n\t\t\t\t\t\tbackupShouldSkipSync: true,\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-2\").\n\t\t\t\t\t\t\tPhase(velerov1api.BackupPhaseWaitingForPluginOperationsPartiallyFailed).\n\t\t\t\t\t\t\tExpiration(fakeClock.Now().Add(-time.Hour)).Result(),\n\t\t\t\t\t\tbackupShouldSkipSync: true,\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-3\").\n\t\t\t\t\t\t\tPhase(velerov1api.BackupPhaseWaitingForPluginOperations).\n\t\t\t\t\t\t\tExpiration(fakeClock.Now().Add(-time.Hour)).Result(),\n\t\t\t\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\t\t\t\tbuilder.ForPodVolumeBackup(\"ns-1\", \"pvb-1\").Result(),\n\t\t\t\t\t\t},\n\t\t\t\t\t\tbackupShouldSkipSync: true,\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-4\").\n\t\t\t\t\t\t\tPhase(velerov1api.BackupPhaseFinalizing).\n\t\t\t\t\t\t\tExpiration(fakeClock.Now().Add(-time.Hour)).Result(),\n\t\t\t\t\t\tbackupShouldSkipSync: true,\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-5\").\n\t\t\t\t\t\t\tPhase(velerov1api.BackupPhaseFinalizingPartiallyFailed).\n\t\t\t\t\t\t\tExpiration(fakeClock.Now().Add(-time.Hour)).Result(),\n\t\t\t\t\t\tbackupShouldSkipSync: true,\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-6\").\n\t\t\t\t\t\t\tPhase(velerov1api.BackupPhaseFinalizing).\n\t\t\t\t\t\t\tExpiration(fakeClock.Now().Add(-time.Hour)).Result(),\n\t\t\t\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\t\t\t\tbuilder.ForPodVolumeBackup(\"ns-1\", \"pvb-2\").Result(),\n\t\t\t\t\t\t},\n\t\t\t\t\t\tbackupShouldSkipSync: true,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"all synced backups get created in Velero server's namespace\",\n\t\t\t\tnamespace: \"velero\",\n\t\t\t\tlocation: defaultLocation(\"velero\"),\n\t\t\t\tcloudBackups: []*cloudBackupData{\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-1\").Result(),\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-2\").Result(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"new backups get synced when some cloud backups already exist in the cluster\",\n\t\t\t\tnamespace: \"ns-1\",\n\t\t\t\tlocation: defaultLocation(\"ns-1\"),\n\t\t\t\tcloudBackups: []*cloudBackupData{\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-1\").Result(),\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-2\").Result(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\texistingBackups: []*velerov1api.Backup{\n\t\t\t\t\t// add a label to each existing backup so we can differentiate it from the cloud\n\t\t\t\t\t// backup during verification\n\t\t\t\t\tbuilder.ForBackup(\"ns-1\", \"backup-1\").StorageLocation(\"location-1\").ObjectMeta(builder.WithLabels(\"i-exist\", \"true\")).Result(),\n\t\t\t\t\tbuilder.ForBackup(\"ns-1\", \"backup-3\").StorageLocation(\"location-2\").ObjectMeta(builder.WithLabels(\"i-exist\", \"true\")).Result(),\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"existing backups without a StorageLocation get it filled in\",\n\t\t\t\tnamespace: \"ns-1\",\n\t\t\t\tlocation: defaultLocation(\"ns-1\"),\n\t\t\t\tcloudBackups: []*cloudBackupData{\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-1\").Result(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\texistingBackups: []*velerov1api.Backup{\n\t\t\t\t\t// add a label to each existing backup so we can differentiate it from the cloud\n\t\t\t\t\t// backup during verification\n\t\t\t\t\tbuilder.ForBackup(\"ns-1\", \"backup-1\").ObjectMeta(builder.WithLabels(\"i-exist\", \"true\")).StorageLocation(\"location-1\").Result(),\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"backup storage location names and labels get updated\",\n\t\t\t\tnamespace: \"ns-1\",\n\t\t\t\tlocation: defaultLocation(\"ns-1\"),\n\t\t\t\tcloudBackups: []*cloudBackupData{\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-1\").StorageLocation(\"foo\").ObjectMeta(builder.WithLabels(velerov1api.StorageLocationLabel, \"foo\")).Result(),\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-2\").Result(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"backup storage location names and labels get updated with location name greater than 63 chars\",\n\t\t\t\tnamespace: \"ns-1\",\n\t\t\t\tlocation: defaultLocationWithLongerLocationName(\"ns-1\"),\n\t\t\t\tlongLocationNameEnabled: true,\n\t\t\t\tcloudBackups: []*cloudBackupData{\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-1\").StorageLocation(\"foo\").ObjectMeta(builder.WithLabels(velerov1api.StorageLocationLabel, \"foo\")).Result(),\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-2\").Result(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"all synced backups and pod volume backups get created in Velero server's namespace\",\n\t\t\t\tnamespace: \"ns-1\",\n\t\t\t\tlocation: defaultLocation(\"ns-1\"),\n\t\t\t\tcloudBackups: []*cloudBackupData{\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-1\").Result(),\n\t\t\t\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\t\t\t\tbuilder.ForPodVolumeBackup(\"ns-1\", \"pvb-1\").Result(),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-2\").Result(),\n\t\t\t\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\t\t\t\tbuilder.ForPodVolumeBackup(\"ns-1\", \"pvb-2\").Result(),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"new pod volume backups get synched when some pod volume backups already exist in the cluster\",\n\t\t\t\tnamespace: \"ns-1\",\n\t\t\t\tlocation: defaultLocation(\"ns-1\"),\n\t\t\t\tcloudBackups: []*cloudBackupData{\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-1\").Result(),\n\t\t\t\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\t\t\t\tbuilder.ForPodVolumeBackup(\"ns-1\", \"pvb-1\").Result(),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tbackup: builder.ForBackup(\"ns-1\", \"backup-2\").Result(),\n\t\t\t\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\t\t\t\tbuilder.ForPodVolumeBackup(\"ns-1\", \"pvb-3\").Result(),\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\texistingPodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\t\tbuilder.ForPodVolumeBackup(\"ns-1\", \"pvb-1\").Result(),\n\t\t\t\t\tbuilder.ForPodVolumeBackup(\"ns-1\", \"pvb-2\").Result(),\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\n\t\tfor _, test := range tests {\n\t\t\tvar (\n\t\t\t\tclient = ctrlfake.NewClientBuilder().Build()\n\t\t\t\tpluginManager = &pluginmocks.Manager{}\n\t\t\t\tbackupStores = make(map[string]*persistencemocks.BackupStore)\n\t\t\t)\n\n\t\t\tpluginManager.On(\"CleanupClients\").Return(nil)\n\t\t\tr := backupSyncReconciler{\n\t\t\t\tclient: client,\n\t\t\t\tnamespace: test.namespace,\n\t\t\t\tdefaultBackupSyncPeriod: time.Second * 10,\n\t\t\t\tnewPluginManager: func(logrus.FieldLogger) clientmgmt.Manager { return pluginManager },\n\t\t\t\tbackupStoreGetter: NewFakeObjectBackupStoreGetter(backupStores),\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tif test.location != nil {\n\t\t\t\tExpect(r.client.Create(ctx, test.location)).ShouldNot(HaveOccurred())\n\t\t\t\tbackupStores[test.location.Name] = &persistencemocks.BackupStore{}\n\n\t\t\t\tbackupStore, ok := backupStores[test.location.Name]\n\t\t\t\tExpect(ok).To(BeTrue(), \"no mock backup store for location %s\", test.location.Name)\n\n\t\t\t\tvar backupNames []string\n\t\t\t\tfor _, backup := range test.cloudBackups {\n\t\t\t\t\tbackupNames = append(backupNames, backup.backup.Name)\n\t\t\t\t\tbackupStore.On(\"GetBackupMetadata\", backup.backup.Name).Return(backup.backup, nil)\n\t\t\t\t\tbackupStore.On(\"GetPodVolumeBackups\", backup.backup.Name).Return(backup.podVolumeBackups, nil)\n\t\t\t\t\tbackupStore.On(\"BackupExists\", \"bucket-1\", backup.backup.Name).Return(true, nil)\n\t\t\t\t}\n\t\t\t\tbackupStore.On(\"ListBackups\").Return(backupNames, nil)\n\t\t\t}\n\n\t\t\tfor _, existingBackup := range test.existingBackups {\n\t\t\t\terr := client.Create(context.TODO(), existingBackup, &ctrlClient.CreateOptions{})\n\t\t\t\tExpect(err).ShouldNot(HaveOccurred())\n\t\t\t}\n\n\t\t\tfor _, existingPodVolumeBackup := range test.existingPodVolumeBackups {\n\t\t\t\terr := client.Create(context.TODO(), existingPodVolumeBackup, &ctrlClient.CreateOptions{})\n\t\t\t\tExpect(err).ShouldNot(HaveOccurred())\n\t\t\t}\n\n\t\t\tactualResult, err := r.Reconcile(ctx, ctrl.Request{\n\t\t\t\tNamespacedName: types.NamespacedName{Namespace: test.location.Namespace, Name: test.location.Name},\n\t\t\t})\n\n\t\t\tExpect(actualResult).To(BeEquivalentTo(ctrl.Result{}))\n\t\t\tExpect(err).ToNot(HaveOccurred())\n\n\t\t\t// process the cloud backups\n\t\t\tfor _, cloudBackupData := range test.cloudBackups {\n\t\t\t\tobj := &velerov1api.Backup{}\n\t\t\t\terr := client.Get(\n\t\t\t\t\tcontext.TODO(),\n\t\t\t\t\ttypes.NamespacedName{\n\t\t\t\t\t\tNamespace: cloudBackupData.backup.Namespace,\n\t\t\t\t\t\tName: cloudBackupData.backup.Name},\n\t\t\t\t\tobj)\n\t\t\t\tif cloudBackupData.backupShouldSkipSync &&\n\t\t\t\t\t(cloudBackupData.backup.Status.Expiration == nil ||\n\t\t\t\t\t\tcloudBackupData.backup.Status.Expiration.After(fakeClock.Now())) {\n\t\t\t\t\tExpect(apierrors.IsNotFound(err)).To(BeTrue())\n\t\t\t\t} else {\n\t\t\t\t\tExpect(err).ToNot(HaveOccurred())\n\n\t\t\t\t\t// did this cloud backup already exist in the cluster?\n\t\t\t\t\tvar existing *velerov1api.Backup\n\t\t\t\t\tfor _, obj := range test.existingBackups {\n\t\t\t\t\t\tif obj.Name == cloudBackupData.backup.Name {\n\t\t\t\t\t\t\texisting = obj\n\t\t\t\t\t\t\tbreak\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\n\t\t\t\t\tif existing != nil {\n\t\t\t\t\t\t// if this cloud backup already exists in the cluster, make sure that what we get from the\n\t\t\t\t\t\t// client is the existing backup, not the cloud one.\n\n\t\t\t\t\t\t// verify that the in-cluster backup has its storage location populated, if it's not already.\n\t\t\t\t\t\texpected := existing.DeepCopy()\n\t\t\t\t\t\texpected.Spec.StorageLocation = test.location.Name\n\n\t\t\t\t\t\tExpect(expected).To(BeEquivalentTo(obj))\n\t\t\t\t\t} else {\n\t\t\t\t\t\t// verify that the storage location field and label are set properly\n\t\t\t\t\t\tExpect(test.location.Name).To(BeEquivalentTo(obj.Spec.StorageLocation))\n\n\t\t\t\t\t\tlocationName := test.location.Name\n\t\t\t\t\t\tif test.longLocationNameEnabled {\n\t\t\t\t\t\t\tlocationName = label.GetValidName(locationName)\n\t\t\t\t\t\t}\n\t\t\t\t\t\tExpect(locationName).To(BeEquivalentTo(obj.Labels[velerov1api.StorageLocationLabel]))\n\t\t\t\t\t\tExpect(len(obj.Labels[velerov1api.StorageLocationLabel])).To(BeNumerically(\"<=\", validation.DNS1035LabelMaxLength))\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\t// process the cloud pod volume backups for this backup, if any\n\t\t\t\tfor _, podVolumeBackup := range cloudBackupData.podVolumeBackups {\n\t\t\t\t\tobjPodVolumeBackup := &velerov1api.PodVolumeBackup{}\n\t\t\t\t\terr := client.Get(\n\t\t\t\t\t\tcontext.TODO(),\n\t\t\t\t\t\ttypes.NamespacedName{\n\t\t\t\t\t\t\tNamespace: podVolumeBackup.Namespace,\n\t\t\t\t\t\t\tName: podVolumeBackup.Name,\n\t\t\t\t\t\t},\n\t\t\t\t\t\tobjPodVolumeBackup)\n\t\t\t\t\tif cloudBackupData.backupShouldSkipSync &&\n\t\t\t\t\t\t(cloudBackupData.backup.Status.Expiration == nil ||\n\t\t\t\t\t\t\tcloudBackupData.backup.Status.Expiration.After(fakeClock.Now())) {\n\t\t\t\t\t\tExpect(apierrors.IsNotFound(err)).To(BeTrue())\n\t\t\t\t\t} else {\n\t\t\t\t\t\tExpect(err).ShouldNot(HaveOccurred())\n\n\t\t\t\t\t\t// did this cloud pod volume backup already exist in the cluster?\n\t\t\t\t\t\tvar existingPodVolumeBackup *velerov1api.PodVolumeBackup\n\t\t\t\t\t\tfor _, objPodVolumeBackup := range test.existingPodVolumeBackups {\n\t\t\t\t\t\t\tif objPodVolumeBackup.Name == podVolumeBackup.Name {\n\t\t\t\t\t\t\t\texistingPodVolumeBackup = objPodVolumeBackup\n\t\t\t\t\t\t\t\tbreak\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\tif existingPodVolumeBackup != nil {\n\t\t\t\t\t\t\t// if this cloud pod volume backup already exists in the cluster, make sure that what we get from the\n\t\t\t\t\t\t\t// client is the existing backup, not the cloud one.\n\t\t\t\t\t\t\texpected := existingPodVolumeBackup.DeepCopy()\n\t\t\t\t\t\t\tExpect(expected).To(BeEquivalentTo(objPodVolumeBackup))\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t})\n\n\tIt(\"Test deleting orphaned backups.\", func() {\n\t\tlongLabelName := \"the-really-long-location-name-that-is-much-more-than-63-characters\"\n\n\t\tbaseBuilder := func(name string) *builder.BackupBuilder {\n\t\t\treturn builder.ForBackup(\"ns-1\", name).ObjectMeta(builder.WithLabels(velerov1api.StorageLocationLabel, \"default\"))\n\t\t}\n\n\t\ttests := []struct {\n\t\t\tname string\n\t\t\tcloudBackups sets.Set[string]\n\t\t\tk8sBackups []*velerov1api.Backup\n\t\t\tnamespace string\n\t\t\texpectedDeletes sets.Set[string]\n\t\t\tuseLongBSLName bool\n\t\t}{\n\t\t\t{\n\t\t\t\tname: \"no overlapping backups\",\n\t\t\t\tnamespace: \"ns-1\",\n\t\t\t\tcloudBackups: sets.New[string](\"backup-1\", \"backup-2\", \"backup-3\"),\n\t\t\t\tk8sBackups: []*velerov1api.Backup{\n\t\t\t\t\tbaseBuilder(\"backupA\").Phase(velerov1api.BackupPhaseCompleted).Result(),\n\t\t\t\t\tbaseBuilder(\"backupB\").Phase(velerov1api.BackupPhaseCompleted).Result(),\n\t\t\t\t\tbaseBuilder(\"backupC\").Phase(velerov1api.BackupPhasePartiallyFailed).Result(),\n\t\t\t\t},\n\t\t\t\texpectedDeletes: sets.New[string](\"backupA\", \"backupB\", \"backupC\"),\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"some overlapping backups\",\n\t\t\t\tnamespace: \"ns-1\",\n\t\t\t\tcloudBackups: sets.New[string](\"backup-1\", \"backup-2\", \"backup-3\"),\n\t\t\t\tk8sBackups: []*velerov1api.Backup{\n\t\t\t\t\tbaseBuilder(\"backup-1\").Phase(velerov1api.BackupPhaseCompleted).Result(),\n\t\t\t\t\tbaseBuilder(\"backup-2\").Phase(velerov1api.BackupPhaseCompleted).Result(),\n\t\t\t\t\tbaseBuilder(\"backup-B\").Phase(velerov1api.BackupPhaseCompleted).Result(),\n\t\t\t\t\tbaseBuilder(\"backup-C\").Phase(velerov1api.BackupPhasePartiallyFailed).Result(),\n\t\t\t\t},\n\t\t\t\texpectedDeletes: sets.New[string](\"backup-B\", \"backup-C\"),\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"all overlapping backups\",\n\t\t\t\tnamespace: \"ns-1\",\n\t\t\t\tcloudBackups: sets.New[string](\"backup-1\", \"backup-2\", \"backup-3\"),\n\t\t\t\tk8sBackups: []*velerov1api.Backup{\n\t\t\t\t\tbaseBuilder(\"backup-1\").Phase(velerov1api.BackupPhaseCompleted).Result(),\n\t\t\t\t\tbaseBuilder(\"backup-2\").Phase(velerov1api.BackupPhaseCompleted).Result(),\n\t\t\t\t\tbaseBuilder(\"backup-3\").Phase(velerov1api.BackupPhasePartiallyFailed).Result(),\n\t\t\t\t},\n\t\t\t\texpectedDeletes: sets.New[string](),\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"no overlapping backups but including backups that are not complete\",\n\t\t\t\tnamespace: \"ns-1\",\n\t\t\t\tcloudBackups: sets.New[string](\"backup-1\", \"backup-2\", \"backup-3\"),\n\t\t\t\tk8sBackups: []*velerov1api.Backup{\n\t\t\t\t\tbaseBuilder(\"backupA\").Phase(velerov1api.BackupPhaseCompleted).Result(),\n\t\t\t\t\tbaseBuilder(\"backupB\").Phase(velerov1api.BackupPhasePartiallyFailed).Result(),\n\t\t\t\t\tbaseBuilder(\"Deleting\").Phase(velerov1api.BackupPhaseDeleting).Result(),\n\t\t\t\t\tbaseBuilder(\"Failed\").Phase(velerov1api.BackupPhaseFailed).Result(),\n\t\t\t\t\tbaseBuilder(\"FailedValidation\").Phase(velerov1api.BackupPhaseFailedValidation).Result(),\n\t\t\t\t\tbaseBuilder(\"InProgress\").Phase(velerov1api.BackupPhaseInProgress).Result(),\n\t\t\t\t\tbaseBuilder(\"New\").Phase(velerov1api.BackupPhaseNew).Result(),\n\t\t\t\t},\n\t\t\t\texpectedDeletes: sets.New[string](\"backupA\", \"backupB\"),\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"all overlapping backups and all backups that are not complete\",\n\t\t\t\tnamespace: \"ns-1\",\n\t\t\t\tcloudBackups: sets.New[string](\"backup-1\", \"backup-2\", \"backup-3\"),\n\t\t\t\tk8sBackups: []*velerov1api.Backup{\n\t\t\t\t\tbaseBuilder(\"backup-1\").Phase(velerov1api.BackupPhaseFailed).Result(),\n\t\t\t\t\tbaseBuilder(\"backup-2\").Phase(velerov1api.BackupPhaseFailedValidation).Result(),\n\t\t\t\t\tbaseBuilder(\"backup-3\").Phase(velerov1api.BackupPhaseInProgress).Result(),\n\t\t\t\t},\n\t\t\t\texpectedDeletes: sets.New[string](),\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"no completed backups in other locations are deleted\",\n\t\t\t\tnamespace: \"ns-1\",\n\t\t\t\tcloudBackups: sets.New[string](\"backup-1\", \"backup-2\", \"backup-3\"),\n\t\t\t\tk8sBackups: []*velerov1api.Backup{\n\t\t\t\t\tbaseBuilder(\"backup-1\").Phase(velerov1api.BackupPhaseCompleted).Result(),\n\t\t\t\t\tbaseBuilder(\"backup-2\").Phase(velerov1api.BackupPhaseCompleted).Result(),\n\t\t\t\t\tbaseBuilder(\"backup-C\").Phase(velerov1api.BackupPhaseCompleted).Result(),\n\t\t\t\t\tbaseBuilder(\"backup-D\").Phase(velerov1api.BackupPhasePartiallyFailed).Result(),\n\n\t\t\t\t\tbaseBuilder(\"backup-4\").ObjectMeta(builder.WithLabels(velerov1api.StorageLocationLabel, \"alternate\")).Phase(velerov1api.BackupPhaseCompleted).Result(),\n\t\t\t\t\tbaseBuilder(\"backup-5\").ObjectMeta(builder.WithLabels(velerov1api.StorageLocationLabel, \"alternate\")).Phase(velerov1api.BackupPhaseCompleted).Result(),\n\t\t\t\t\tbaseBuilder(\"backup-6\").ObjectMeta(builder.WithLabels(velerov1api.StorageLocationLabel, \"alternate\")).Phase(velerov1api.BackupPhasePartiallyFailed).Result(),\n\t\t\t\t},\n\t\t\t\texpectedDeletes: sets.New[string](\"backup-C\", \"backup-D\"),\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"some overlapping backups\",\n\t\t\t\tnamespace: \"ns-1\",\n\t\t\t\tcloudBackups: sets.New[string](\"backup-1\", \"backup-2\", \"backup-3\"),\n\t\t\t\tk8sBackups: []*velerov1api.Backup{\n\t\t\t\t\tbuilder.ForBackup(\"ns-1\", \"backup-1\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithLabels(velerov1api.StorageLocationLabel, \"the-really-long-location-name-that-is-much-more-than-63-c69e779\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tPhase(velerov1api.BackupPhaseCompleted).\n\t\t\t\t\t\tResult(),\n\t\t\t\t\tbuilder.ForBackup(\"ns-1\", \"backup-2\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithLabels(velerov1api.StorageLocationLabel, \"the-really-long-location-name-that-is-much-more-than-63-c69e779\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tPhase(velerov1api.BackupPhaseCompleted).\n\t\t\t\t\t\tResult(),\n\t\t\t\t\tbuilder.ForBackup(\"ns-1\", \"backup-C\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithLabels(velerov1api.StorageLocationLabel, \"the-really-long-location-name-that-is-much-more-than-63-c69e779\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tPhase(velerov1api.BackupPhaseCompleted).\n\t\t\t\t\t\tResult(),\n\t\t\t\t\tbuilder.ForBackup(\"ns-1\", \"backup-D\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithLabels(velerov1api.StorageLocationLabel, \"the-really-long-location-name-that-is-much-more-than-63-c69e779\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tPhase(velerov1api.BackupPhasePartiallyFailed).\n\t\t\t\t\t\tResult(),\n\t\t\t\t},\n\t\t\t\texpectedDeletes: sets.New[string](\"backup-C\", \"backup-D\"),\n\t\t\t\tuseLongBSLName: true,\n\t\t\t},\n\t\t}\n\n\t\tfor _, test := range tests {\n\t\t\tvar (\n\t\t\t\tclient = ctrlfake.NewClientBuilder().Build()\n\t\t\t\tpluginManager = &pluginmocks.Manager{}\n\t\t\t\tbackupStores = make(map[string]*persistencemocks.BackupStore)\n\t\t\t)\n\n\t\t\tr := backupSyncReconciler{\n\t\t\t\tclient: client,\n\t\t\t\tnamespace: test.namespace,\n\t\t\t\tdefaultBackupSyncPeriod: time.Second * 10,\n\t\t\t\tnewPluginManager: func(logrus.FieldLogger) clientmgmt.Manager { return pluginManager },\n\t\t\t\tbackupStoreGetter: NewFakeObjectBackupStoreGetter(backupStores),\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tfor _, backup := range test.k8sBackups {\n\t\t\t\t// add test backup to client\n\t\t\t\terr := client.Create(context.TODO(), backup, &ctrlClient.CreateOptions{})\n\t\t\t\tExpect(err).ShouldNot(HaveOccurred())\n\t\t\t}\n\n\t\t\tbslName := \"default\"\n\t\t\tif test.useLongBSLName {\n\t\t\t\tbslName = longLabelName\n\t\t\t}\n\t\t\tr.deleteOrphanedBackups(ctx, bslName, test.cloudBackups, velerotest.NewLogger())\n\n\t\t\tnumBackups, err := numBackups(client)\n\t\t\tExpect(err).ShouldNot(HaveOccurred())\n\n\t\t\tfmt.Println(\"\")\n\n\t\t\texpected := len(test.k8sBackups) - len(test.expectedDeletes)\n\t\t\tExpect(expected).To(BeEquivalentTo(numBackups))\n\t\t}\n\t})\n\n\tIt(\"Test moving default BSL at the head of BSL array.\", func() {\n\t\tlocationList := &velerov1api.BackupStorageLocationList{}\n\t\tobjArray := make([]runtime.Object, 0)\n\n\t\t// Generate BSL array.\n\t\tlocations := defaultLocationsList(\"velero\")\n\t\tfor _, bsl := range locations {\n\t\t\tobjArray = append(objArray, bsl)\n\t\t}\n\n\t\tmeta.SetList(locationList, objArray)\n\n\t\ttestObjList := backupSyncSourceOrderFunc(locationList)\n\t\ttestObjArray, err := meta.ExtractList(testObjList)\n\t\tExpect(err).ShouldNot(HaveOccurred())\n\n\t\texpectLocation := testObjArray[0].(*velerov1api.BackupStorageLocation)\n\t\tExpect(expectLocation.Spec.Default).To(BeEquivalentTo(true))\n\n\t\t// If BSL list without default BSL is passed in, the output should be same with input.\n\t\tlocationList.Items = testObjList.(*velerov1api.BackupStorageLocationList).Items[1:]\n\t\ttestObjList = backupSyncSourceOrderFunc(locationList)\n\t\tExpect(testObjList).To(BeEquivalentTo(locationList))\n\n\t})\n\n\tWhen(\"testing validateOwnerReferences\", func() {\n\n\t\ttestCases := []struct {\n\t\t\tname string\n\t\t\tbackup *velerov1api.Backup\n\t\t\ttoCreate []ctrlClient.Object\n\t\t\texpectedReferences []metav1.OwnerReference\n\t\t}{\n\t\t\t{\n\t\t\t\tname: \"handles empty owner references\",\n\t\t\t\tbackup: &velerov1api.Backup{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tOwnerReferences: []metav1.OwnerReference{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\texpectedReferences: []metav1.OwnerReference{},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"handles missing schedule\",\n\t\t\t\tbackup: &velerov1api.Backup{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKind: \"Schedule\",\n\t\t\t\t\t\t\t\tName: \"some name\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\texpectedReferences: []metav1.OwnerReference{},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"handles existing reference\",\n\t\t\t\tbackup: &velerov1api.Backup{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKind: \"Schedule\",\n\t\t\t\t\t\t\t\tName: \"existing-schedule\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tNamespace: \"test-namespace\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\ttoCreate: []ctrlClient.Object{\n\t\t\t\t\t&velerov1api.Schedule{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"existing-schedule\",\n\t\t\t\t\t\t\tNamespace: \"test-namespace\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\texpectedReferences: []metav1.OwnerReference{\n\t\t\t\t\t{\n\t\t\t\t\t\tKind: \"Schedule\",\n\t\t\t\t\t\tName: \"existing-schedule\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"handles existing mismatched UID\",\n\t\t\t\tbackup: &velerov1api.Backup{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKind: \"Schedule\",\n\t\t\t\t\t\t\t\tName: \"existing-schedule\",\n\t\t\t\t\t\t\t\tUID: \"backup-UID\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tNamespace: \"test-namespace\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\ttoCreate: []ctrlClient.Object{\n\t\t\t\t\t&velerov1api.Schedule{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"existing-schedule\",\n\t\t\t\t\t\t\tNamespace: \"test-namespace\",\n\t\t\t\t\t\t\tUID: \"schedule-UID\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\texpectedReferences: []metav1.OwnerReference{},\n\t\t\t},\n\t\t\t{\n\t\t\t\tname: \"handles multiple references\",\n\t\t\t\tbackup: &velerov1api.Backup{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKind: \"Schedule\",\n\t\t\t\t\t\t\t\tName: \"existing-schedule\",\n\t\t\t\t\t\t\t\tUID: \"1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKind: \"Schedule\",\n\t\t\t\t\t\t\t\tName: \"missing-schedule\",\n\t\t\t\t\t\t\t\tUID: \"2\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKind: \"Schedule\",\n\t\t\t\t\t\t\t\tName: \"mismatched-uid-schedule\",\n\t\t\t\t\t\t\t\tUID: \"3\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKind: \"Schedule\",\n\t\t\t\t\t\t\t\tName: \"another-existing-schedule\",\n\t\t\t\t\t\t\t\tUID: \"4\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tNamespace: \"test-namespace\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\ttoCreate: []ctrlClient.Object{\n\t\t\t\t\t&velerov1api.Schedule{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"existing-schedule\",\n\t\t\t\t\t\t\tNamespace: \"test-namespace\",\n\t\t\t\t\t\t\tUID: \"1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t&velerov1api.Schedule{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"mismatched-uid-schedule\",\n\t\t\t\t\t\t\tNamespace: \"test-namespace\",\n\t\t\t\t\t\t\tUID: \"not-3\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t&velerov1api.Schedule{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"another-existing-schedule\",\n\t\t\t\t\t\t\tNamespace: \"test-namespace\",\n\t\t\t\t\t\t\tUID: \"4\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\texpectedReferences: []metav1.OwnerReference{\n\t\t\t\t\t{\n\t\t\t\t\t\tKind: \"Schedule\",\n\t\t\t\t\t\tName: \"existing-schedule\",\n\t\t\t\t\t\tUID: \"1\",\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tKind: \"Schedule\",\n\t\t\t\t\t\tName: \"another-existing-schedule\",\n\t\t\t\t\t\tUID: \"4\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\t\tfor _, test := range testCases {\n\t\t\tIt(test.name, func() {\n\t\t\t\tlogger := velerotest.NewLogger()\n\t\t\t\tb := backupSyncReconciler{\n\t\t\t\t\tclient: ctrlfake.NewClientBuilder().Build(),\n\t\t\t\t}\n\n\t\t\t\t//create all required schedules as needed.\n\t\t\t\tfor _, creatable := range test.toCreate {\n\t\t\t\t\terr := b.client.Create(context.Background(), creatable)\n\t\t\t\t\tExpect(err).ShouldNot(HaveOccurred())\n\t\t\t\t}\n\n\t\t\t\treferences := b.filterBackupOwnerReferences(context.Background(), test.backup, logger)\n\t\t\t\tExpect(references).To(BeEquivalentTo(test.expectedReferences))\n\t\t\t})\n\t\t}\n\t})\n})\n" }, { "path": "pkg/controller/backup_tracker.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"fmt\"\n\t\"sync\"\n\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n)\n\n// BackupTracker keeps track of in-progress backups.\ntype BackupTracker interface {\n\t// Add informs the tracker that a backup is ReadyToStart.\n\tAddReadyToStart(ns, name string)\n\t// Add informs the tracker that a backup is in progress.\n\tAdd(ns, name string)\n\t// Add informs the tracker that a backup has moved beyond InProgress\n\tAddPostProcessing(ns, name string)\n\t// Delete informs the tracker that a backup has reached a terminal state.\n\tDelete(ns, name string)\n\t// Contains returns true if backup is InProgress or post-InProgress\n\tContains(ns, name string) bool\n\t// RunningCount returns the number of backups which are ReadyToStart or InProgress\n\tRunningCount() int\n}\n\ntype backupTracker struct {\n\tlock sync.RWMutex\n\treadyToStartBackups sets.Set[string]\n\tinProgressBackups sets.Set[string]\n\tpostProgressBackups sets.Set[string]\n}\n\n// NewBackupTracker returns a new BackupTracker.\nfunc NewBackupTracker() BackupTracker {\n\treturn &backupTracker{\n\t\treadyToStartBackups: sets.New[string](),\n\t\tinProgressBackups: sets.New[string](),\n\t\tpostProgressBackups: sets.New[string](),\n\t}\n}\n\nfunc (bt *backupTracker) AddReadyToStart(ns, name string) {\n\tbt.lock.Lock()\n\tdefer bt.lock.Unlock()\n\n\tbt.readyToStartBackups.Insert(backupTrackerKey(ns, name))\n}\n\nfunc (bt *backupTracker) Add(ns, name string) {\n\tbt.lock.Lock()\n\tdefer bt.lock.Unlock()\n\n\tkey := backupTrackerKey(ns, name)\n\tbt.readyToStartBackups.Delete(key)\n\tbt.inProgressBackups.Insert(key)\n}\n\nfunc (bt *backupTracker) AddPostProcessing(ns, name string) {\n\tbt.lock.Lock()\n\tdefer bt.lock.Unlock()\n\n\tkey := backupTrackerKey(ns, name)\n\tbt.readyToStartBackups.Delete(key)\n\tbt.inProgressBackups.Delete(key)\n\tbt.postProgressBackups.Insert(key)\n}\n\nfunc (bt *backupTracker) Delete(ns, name string) {\n\tbt.lock.Lock()\n\tdefer bt.lock.Unlock()\n\n\tkey := backupTrackerKey(ns, name)\n\tbt.readyToStartBackups.Delete(key)\n\tbt.inProgressBackups.Delete(key)\n\tbt.postProgressBackups.Delete(key)\n}\n\n// Contains returns true if backup is InProgress or post-InProgress\n// ignores ReadyToStart, since this is used to determine whether\n// a backup is in progress and thus not able to be deleted now.\nfunc (bt *backupTracker) Contains(ns, name string) bool {\n\tbt.lock.RLock()\n\tdefer bt.lock.RUnlock()\n\n\tkey := backupTrackerKey(ns, name)\n\treturn bt.inProgressBackups.Has(key) || bt.postProgressBackups.Has(key)\n}\n\n// RunningCount returns the number of backups which are ReadyToStart or InProgress\n// used by queue controller to determine whether a new backup can be started.\nfunc (bt *backupTracker) RunningCount() int {\n\tbt.lock.RLock()\n\tdefer bt.lock.RUnlock()\n\n\treturn bt.inProgressBackups.Len() + bt.readyToStartBackups.Len()\n}\n\nfunc backupTrackerKey(ns, name string) string {\n\treturn fmt.Sprintf(\"%s/%s\", ns, name)\n}\n" }, { "path": "pkg/controller/backup_tracker_test.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestBackupTracker(t *testing.T) {\n\tbt := NewBackupTracker()\n\n\tassert.False(t, bt.Contains(\"ns\", \"name\"))\n\n\tbt.Add(\"ns\", \"name\")\n\tassert.True(t, bt.Contains(\"ns\", \"name\"))\n\n\tbt.Add(\"ns2\", \"name2\")\n\tassert.True(t, bt.Contains(\"ns\", \"name\"))\n\tassert.True(t, bt.Contains(\"ns2\", \"name2\"))\n\n\tbt.Delete(\"ns\", \"name\")\n\tassert.False(t, bt.Contains(\"ns\", \"name\"))\n\tassert.True(t, bt.Contains(\"ns2\", \"name2\"))\n\n\tbt.Delete(\"ns2\", \"name2\")\n\tassert.False(t, bt.Contains(\"ns2\", \"name2\"))\n}\n" }, { "path": "pkg/controller/data_download_controller.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/builder\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil\"\n\t\"sigs.k8s.io/controller-runtime/pkg/event\"\n\t\"sigs.k8s.io/controller-runtime/pkg/manager\"\n\t\"sigs.k8s.io/controller-runtime/pkg/predicate\"\n\t\"sigs.k8s.io/controller-runtime/pkg/reconcile\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/apis/velero/shared\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\tdatamover \"github.com/vmware-tanzu/velero/pkg/datamover\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\t\"github.com/vmware-tanzu/velero/pkg/exposer\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\t\"github.com/vmware-tanzu/velero/pkg/nodeagent\"\n\trepository \"github.com/vmware-tanzu/velero/pkg/repository/manager\"\n\tvelerotypes \"github.com/vmware-tanzu/velero/pkg/types\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\n// DataDownloadReconciler reconciles a DataDownload object\ntype DataDownloadReconciler struct {\n\tclient client.Client\n\tkubeClient kubernetes.Interface\n\tmgr manager.Manager\n\tlogger logrus.FieldLogger\n\tClock clock.WithTickerAndDelayedExecution\n\trestoreExposer exposer.GenericRestoreExposer\n\tnodeName string\n\tdataPathMgr *datapath.Manager\n\tvgdpCounter *exposer.VgdpCounter\n\tloadAffinity []*kube.LoadAffinity\n\trestorePVCConfig velerotypes.RestorePVC\n\tbackupRepoConfigs map[string]string\n\tcacheVolumeConfigs *velerotypes.CachePVC\n\tpodResources corev1api.ResourceRequirements\n\tpreparingTimeout time.Duration\n\tmetrics *metrics.ServerMetrics\n\tcancelledDataDownload map[string]time.Time\n\tdataMovePriorityClass string\n\trepoConfigMgr repository.ConfigManager\n\tpodLabels map[string]string\n\tpodAnnotations map[string]string\n}\n\nfunc NewDataDownloadReconciler(\n\tclient client.Client,\n\tmgr manager.Manager,\n\tkubeClient kubernetes.Interface,\n\tdataPathMgr *datapath.Manager,\n\tcounter *exposer.VgdpCounter,\n\tloadAffinity []*kube.LoadAffinity,\n\trestorePVCConfig velerotypes.RestorePVC,\n\tbackupRepoConfigs map[string]string,\n\tcacheVolumeConfigs *velerotypes.CachePVC,\n\tpodResources corev1api.ResourceRequirements,\n\tnodeName string,\n\tpreparingTimeout time.Duration,\n\tlogger logrus.FieldLogger,\n\tmetrics *metrics.ServerMetrics,\n\tdataMovePriorityClass string,\n\trepoConfigMgr repository.ConfigManager,\n\tpodLabels map[string]string,\n\tpodAnnotations map[string]string,\n) *DataDownloadReconciler {\n\treturn &DataDownloadReconciler{\n\t\tclient: client,\n\t\tkubeClient: kubeClient,\n\t\tmgr: mgr,\n\t\tlogger: logger.WithField(\"controller\", \"DataDownload\"),\n\t\tClock: &clock.RealClock{},\n\t\tnodeName: nodeName,\n\t\trestoreExposer: exposer.NewGenericRestoreExposer(kubeClient, logger),\n\t\trestorePVCConfig: restorePVCConfig,\n\t\tbackupRepoConfigs: backupRepoConfigs,\n\t\tcacheVolumeConfigs: cacheVolumeConfigs,\n\t\tdataPathMgr: dataPathMgr,\n\t\tvgdpCounter: counter,\n\t\tloadAffinity: loadAffinity,\n\t\tpodResources: podResources,\n\t\tpreparingTimeout: preparingTimeout,\n\t\tmetrics: metrics,\n\t\tcancelledDataDownload: make(map[string]time.Time),\n\t\tdataMovePriorityClass: dataMovePriorityClass,\n\t\trepoConfigMgr: repoConfigMgr,\n\t\tpodLabels: podLabels,\n\t\tpodAnnotations: podAnnotations,\n\t}\n}\n\n// +kubebuilder:rbac:groups=velero.io,resources=datadownloads,verbs=get;list;watch;create;update;patch;delete\n// +kubebuilder:rbac:groups=velero.io,resources=datadownloads/status,verbs=get;update;patch\n// +kubebuilder:rbac:groups=\"\",resources=pods,verbs=get\n// +kubebuilder:rbac:groups=\"\",resources=persistentvolumes,verbs=get\n// +kubebuilder:rbac:groups=\"\",resources=persistentvolumerclaims,verbs=get\n\nfunc (r *DataDownloadReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\tlog := r.logger.WithFields(logrus.Fields{\n\t\t\"controller\": \"datadownload\",\n\t\t\"datadownload\": req.NamespacedName,\n\t})\n\n\tlog.Infof(\"Reconcile %s\", req.Name)\n\n\tdd := &velerov2alpha1api.DataDownload{}\n\tif err := r.client.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: req.Name}, dd); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.Warn(\"DataDownload not found, skip\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\tlog.WithError(err).Error(\"Unable to get the DataDownload\")\n\t\treturn ctrl.Result{}, err\n\t}\n\n\tif !datamover.IsBuiltInUploader(dd.Spec.DataMover) {\n\t\tlog.WithField(\"data mover\", dd.Spec.DataMover).Info(\"it is not one built-in data mover which is not supported by Velero\")\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\t// Logic for clear resources when datadownload been deleted\n\tif !isDataDownloadInFinalState(dd) {\n\t\tif !controllerutil.ContainsFinalizer(dd, DataUploadDownloadFinalizer) {\n\t\t\tif err := UpdateDataDownloadWithRetry(ctx, r.client, req.NamespacedName, log, func(dd *velerov2alpha1api.DataDownload) bool {\n\t\t\t\tif controllerutil.ContainsFinalizer(dd, DataUploadDownloadFinalizer) {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tcontrollerutil.AddFinalizer(dd, DataUploadDownloadFinalizer)\n\n\t\t\t\treturn true\n\t\t\t}); err != nil {\n\t\t\t\tlog.WithError(err).Errorf(\"failed to add finalizer for dd %s/%s\", dd.Namespace, dd.Name)\n\t\t\t\treturn ctrl.Result{}, err\n\t\t\t}\n\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tif !dd.DeletionTimestamp.IsZero() {\n\t\t\tif !dd.Spec.Cancel {\n\t\t\t\t// when delete cr we need to clear up internal resources created by Velero, here we use the cancel mechanism\n\t\t\t\t// to help clear up resources instead of clear them directly in case of some conflict with Expose action\n\t\t\t\tlog.Warnf(\"Cancel dd under phase %s because it is being deleted\", dd.Status.Phase)\n\n\t\t\t\tif err := UpdateDataDownloadWithRetry(ctx, r.client, req.NamespacedName, log, func(dataDownload *velerov2alpha1api.DataDownload) bool {\n\t\t\t\t\tif dataDownload.Spec.Cancel {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\n\t\t\t\t\tdataDownload.Spec.Cancel = true\n\t\t\t\t\tdataDownload.Status.Message = \"Cancel datadownload because it is being deleted\"\n\n\t\t\t\t\treturn true\n\t\t\t\t}); err != nil {\n\t\t\t\t\tlog.WithError(err).Errorf(\"failed to set cancel flag for dd %s/%s\", dd.Namespace, dd.Name)\n\t\t\t\t\treturn ctrl.Result{}, err\n\t\t\t\t}\n\n\t\t\t\treturn ctrl.Result{}, nil\n\t\t\t}\n\t\t}\n\t} else {\n\t\tdelete(r.cancelledDataDownload, dd.Name)\n\n\t\t// put the finalizer remove action here for all cr will goes to the final status, we could check finalizer and do remove action in final status\n\t\t// instead of intermediate state.\n\t\t// remove finalizer no matter whether the cr is being deleted or not for it is no longer needed when internal resources are all cleaned up\n\t\t// also in final status cr won't block the direct delete of the velero namespace\n\t\tif controllerutil.ContainsFinalizer(dd, DataUploadDownloadFinalizer) {\n\t\t\tif err := UpdateDataDownloadWithRetry(ctx, r.client, req.NamespacedName, log, func(dd *velerov2alpha1api.DataDownload) bool {\n\t\t\t\tif !controllerutil.ContainsFinalizer(dd, DataUploadDownloadFinalizer) {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tcontrollerutil.RemoveFinalizer(dd, DataUploadDownloadFinalizer)\n\n\t\t\t\treturn true\n\t\t\t}); err != nil {\n\t\t\t\tlog.WithError(err).Error(\"error to remove finalizer\")\n\t\t\t\treturn ctrl.Result{}, err\n\t\t\t}\n\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t}\n\n\tif dd.Spec.Cancel {\n\t\tif spotted, found := r.cancelledDataDownload[dd.Name]; !found {\n\t\t\tr.cancelledDataDownload[dd.Name] = r.Clock.Now()\n\t\t} else {\n\t\t\tdelay := cancelDelayOthers\n\t\t\tif dd.Status.Phase == velerov2alpha1api.DataDownloadPhaseInProgress {\n\t\t\t\tdelay = cancelDelayInProgress\n\t\t\t}\n\n\t\t\tif time.Since(spotted) > delay {\n\t\t\t\tlog.Infof(\"Data download %s is canceled in Phase %s but not handled in rasonable time\", dd.GetName(), dd.Status.Phase)\n\t\t\t\tif r.tryCancelDataDownload(ctx, dd, \"\") {\n\t\t\t\t\tdelete(r.cancelledDataDownload, dd.Name)\n\t\t\t\t}\n\n\t\t\t\treturn ctrl.Result{}, nil\n\t\t\t}\n\t\t}\n\t}\n\n\tif dd.Status.Phase == \"\" || dd.Status.Phase == velerov2alpha1api.DataDownloadPhaseNew {\n\t\tif dd.Spec.Cancel {\n\t\t\tlog.Debugf(\"Data download is canceled in Phase %s\", dd.Status.Phase)\n\n\t\t\tr.tryCancelDataDownload(ctx, dd, \"\")\n\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tif r.vgdpCounter != nil && r.vgdpCounter.IsConstrained(ctx, r.logger) {\n\t\t\tlog.Debug(\"Data path initiation is constrained, requeue later\")\n\t\t\treturn ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5}, nil\n\t\t}\n\n\t\tif _, err := r.getTargetPVC(ctx, dd); err != nil {\n\t\t\tlog.WithField(\"error\", err).Debugf(\"Cannot find target PVC for DataDownload yet. Retry later.\")\n\t\t\treturn ctrl.Result{Requeue: true}, nil\n\t\t}\n\n\t\tlog.Info(\"Data download starting\")\n\n\t\taccepted, err := r.acceptDataDownload(ctx, dd)\n\t\tif err != nil {\n\t\t\treturn ctrl.Result{}, errors.Wrapf(err, \"error accepting the data download %s\", dd.Name)\n\t\t}\n\n\t\tif !accepted {\n\t\t\tlog.Debug(\"Data download is not accepted\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tlog.Info(\"Data download is accepted\")\n\n\t\texposeParam, err := r.setupExposeParam(dd)\n\t\tif err != nil {\n\t\t\treturn r.errorOut(ctx, dd, err, \"failed to set exposer parameters\", log)\n\t\t}\n\n\t\t// Expose() will trigger to create one pod whose volume is restored by a given volume snapshot,\n\t\t// but the pod maybe is not in the same node of the current controller, so we need to return it here.\n\t\t// And then only the controller who is in the same node could do the rest work.\n\t\terr = r.restoreExposer.Expose(ctx, getDataDownloadOwnerObject(dd), exposeParam)\n\t\tif err != nil {\n\t\t\treturn r.errorOut(ctx, dd, err, \"error to expose snapshot\", log)\n\t\t}\n\t\tlog.Info(\"Restore is exposed\")\n\n\t\treturn ctrl.Result{}, nil\n\t} else if dd.Status.Phase == velerov2alpha1api.DataDownloadPhaseAccepted {\n\t\tif peekErr := r.restoreExposer.PeekExposed(ctx, getDataDownloadOwnerObject(dd)); peekErr != nil {\n\t\t\tlog.Errorf(\"Cancel dd %s/%s because of expose error %s\", dd.Namespace, dd.Name, peekErr)\n\n\t\t\tdiags := strings.Split(r.restoreExposer.DiagnoseExpose(ctx, getDataDownloadOwnerObject(dd)), \"\\n\")\n\t\t\tfor _, diag := range diags {\n\t\t\t\tlog.Warnf(\"[Diagnose DD expose]%s\", diag)\n\t\t\t}\n\n\t\t\tr.tryCancelDataDownload(ctx, dd, fmt.Sprintf(\"found a datadownload %s/%s with expose error: %s. mark it as cancel\", dd.Namespace, dd.Name, peekErr))\n\t\t} else if dd.Status.AcceptedTimestamp != nil {\n\t\t\tif time.Since(dd.Status.AcceptedTimestamp.Time) >= r.preparingTimeout {\n\t\t\t\tr.onPrepareTimeout(ctx, dd)\n\t\t\t}\n\t\t}\n\n\t\treturn ctrl.Result{}, nil\n\t} else if dd.Status.Phase == velerov2alpha1api.DataDownloadPhasePrepared {\n\t\tlog.Infof(\"Data download is prepared and should be processed by %s (%s)\", dd.Status.Node, r.nodeName)\n\n\t\tif dd.Status.Node != r.nodeName {\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tif dd.Spec.Cancel {\n\t\t\tlog.Debugf(\"Data download is been canceled %s in Phase %s\", dd.GetName(), dd.Status.Phase)\n\t\t\tr.OnDataDownloadCancelled(ctx, dd.GetNamespace(), dd.GetName())\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tasyncBR := r.dataPathMgr.GetAsyncBR(dd.Name)\n\n\t\tif asyncBR != nil {\n\t\t\tlog.Info(\"Cancellable data path is already started\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tresult, err := r.restoreExposer.GetExposed(ctx, getDataDownloadOwnerObject(dd), r.client, r.nodeName, dd.Spec.OperationTimeout.Duration)\n\t\tif err != nil {\n\t\t\treturn r.errorOut(ctx, dd, err, \"restore exposer is not ready\", log)\n\t\t} else if result == nil {\n\t\t\treturn r.errorOut(ctx, dd, errors.New(\"no expose result is available for the current node\"), \"exposed snapshot is not ready\", log)\n\t\t}\n\n\t\tlog.Info(\"Restore PVC is ready and creating data path routine\")\n\n\t\t// Need to first create file system BR and get data path instance then update data download status\n\t\tcallbacks := datapath.Callbacks{\n\t\t\tOnCompleted: r.OnDataDownloadCompleted,\n\t\t\tOnFailed: r.OnDataDownloadFailed,\n\t\t\tOnCancelled: r.OnDataDownloadCancelled,\n\t\t\tOnProgress: r.OnDataDownloadProgress,\n\t\t}\n\n\t\tasyncBR, err = r.dataPathMgr.CreateMicroServiceBRWatcher(ctx, r.client, r.kubeClient, r.mgr, datapath.TaskTypeRestore,\n\t\t\tdd.Name, dd.Namespace, result.ByPod.HostingPod.Name, result.ByPod.HostingContainer, dd.Name, callbacks, false, log)\n\t\tif err != nil {\n\t\t\tif err == datapath.ConcurrentLimitExceed {\n\t\t\t\tlog.Debug(\"Data path instance is concurrent limited requeue later\")\n\t\t\t\treturn ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5}, nil\n\t\t\t} else {\n\t\t\t\treturn r.errorOut(ctx, dd, err, \"error to create data path\", log)\n\t\t\t}\n\t\t}\n\n\t\tif err := r.initCancelableDataPath(ctx, asyncBR, result, log); err != nil {\n\t\t\tlog.WithError(err).Errorf(\"Failed to init cancelable data path for %s\", dd.Name)\n\n\t\t\tr.closeDataPath(ctx, dd.Name)\n\t\t\treturn r.errorOut(ctx, dd, err, \"error initializing data path\", log)\n\t\t}\n\n\t\t// Update status to InProgress\n\t\tterminated := false\n\t\tif err := UpdateDataDownloadWithRetry(ctx, r.client, types.NamespacedName{Namespace: dd.Namespace, Name: dd.Name}, log, func(dd *velerov2alpha1api.DataDownload) bool {\n\t\t\tif isDataDownloadInFinalState(dd) {\n\t\t\t\tterminated = true\n\t\t\t\treturn false\n\t\t\t}\n\n\t\t\tdd.Status.Phase = velerov2alpha1api.DataDownloadPhaseInProgress\n\t\t\tdd.Status.StartTimestamp = &metav1.Time{Time: r.Clock.Now()}\n\n\t\t\tdelete(dd.Labels, exposer.ExposeOnGoingLabel)\n\n\t\t\treturn true\n\t\t}); err != nil {\n\t\t\tlog.WithError(err).Warnf(\"Failed to update datadownload %s to InProgress, will data path close and retry\", dd.Name)\n\n\t\t\tr.closeDataPath(ctx, dd.Name)\n\t\t\treturn ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5}, nil\n\t\t}\n\n\t\tif terminated {\n\t\t\tlog.Warnf(\"datadownload %s is terminated during transition from prepared\", dd.Name)\n\t\t\tr.closeDataPath(ctx, dd.Name)\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tlog.Info(\"Data download is marked as in progress\")\n\n\t\tif err := r.startCancelableDataPath(asyncBR, dd, result, log); err != nil {\n\t\t\tlog.WithError(err).Errorf(\"Failed to start cancelable data path for %s\", dd.Name)\n\n\t\t\tr.closeDataPath(ctx, dd.Name)\n\t\t\treturn r.errorOut(ctx, dd, err, \"error starting data path\", log)\n\t\t}\n\n\t\treturn ctrl.Result{}, nil\n\t} else if dd.Status.Phase == velerov2alpha1api.DataDownloadPhaseInProgress {\n\t\tif dd.Spec.Cancel {\n\t\t\tif dd.Status.Node != r.nodeName {\n\t\t\t\treturn ctrl.Result{}, nil\n\t\t\t}\n\n\t\t\tlog.Info(\"In progress data download is being canceled\")\n\n\t\t\tasyncBR := r.dataPathMgr.GetAsyncBR(dd.Name)\n\t\t\tif asyncBR == nil {\n\t\t\t\tr.OnDataDownloadCancelled(ctx, dd.GetNamespace(), dd.GetName())\n\t\t\t\treturn ctrl.Result{}, nil\n\t\t\t}\n\n\t\t\t// Update status to Canceling.\n\t\t\tif err := UpdateDataDownloadWithRetry(ctx, r.client, types.NamespacedName{Namespace: dd.Namespace, Name: dd.Name}, log, func(dd *velerov2alpha1api.DataDownload) bool {\n\t\t\t\tif isDataDownloadInFinalState(dd) {\n\t\t\t\t\tlog.Warnf(\"datadownload %s is terminated, abort setting it to canceling\", dd.Name)\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tdd.Status.Phase = velerov2alpha1api.DataDownloadPhaseCanceling\n\t\t\t\treturn true\n\t\t\t}); err != nil {\n\t\t\t\tlog.WithError(err).Error(\"error updating data download into canceling status\")\n\t\t\t\treturn ctrl.Result{}, err\n\t\t\t}\n\n\t\t\tasyncBR.Cancel()\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\treturn ctrl.Result{}, nil\n}\n\nfunc (r *DataDownloadReconciler) initCancelableDataPath(ctx context.Context, asyncBR datapath.AsyncBR, res *exposer.ExposeResult, log logrus.FieldLogger) error {\n\tlog.Info(\"Init cancelable dataDownload\")\n\n\tif err := asyncBR.Init(ctx, nil); err != nil {\n\t\treturn errors.Wrap(err, \"error initializing asyncBR\")\n\t}\n\n\tlog.Infof(\"async restore init for pod %s, volume %s\", res.ByPod.HostingPod.Name, res.ByPod.VolumeName)\n\n\treturn nil\n}\n\nfunc (r *DataDownloadReconciler) startCancelableDataPath(asyncBR datapath.AsyncBR, dd *velerov2alpha1api.DataDownload, res *exposer.ExposeResult, log logrus.FieldLogger) error {\n\tlog.Info(\"Start cancelable dataDownload\")\n\n\tif err := asyncBR.StartRestore(dd.Spec.SnapshotID, datapath.AccessPoint{\n\t\tByPath: res.ByPod.VolumeName,\n\t}, dd.Spec.DataMoverConfig); err != nil {\n\t\treturn errors.Wrapf(err, \"error starting async restore for pod %s, volume %s\", res.ByPod.HostingPod.Name, res.ByPod.VolumeName)\n\t}\n\n\tlog.Infof(\"Async restore started for pod %s, volume %s\", res.ByPod.HostingPod.Name, res.ByPod.VolumeName)\n\treturn nil\n}\n\nfunc (r *DataDownloadReconciler) OnDataDownloadCompleted(ctx context.Context, namespace string, ddName string, result datapath.Result) {\n\tdefer r.dataPathMgr.RemoveAsyncBR(ddName)\n\n\tlog := r.logger.WithField(\"datadownload\", ddName)\n\tlog.Info(\"Async fs restore data path completed\")\n\n\tvar dd velerov2alpha1api.DataDownload\n\tif err := r.client.Get(ctx, types.NamespacedName{Name: ddName, Namespace: namespace}, &dd); err != nil {\n\t\tlog.WithError(err).Warn(\"Failed to get datadownload on completion\")\n\t\treturn\n\t}\n\n\tobjRef := getDataDownloadOwnerObject(&dd)\n\terr := r.restoreExposer.RebindVolume(ctx, objRef, dd.Spec.TargetVolume.PVC, dd.Spec.TargetVolume.Namespace, dd.Spec.OperationTimeout.Duration)\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"Failed to rebind PV to target PVC on completion\")\n\t\treturn\n\t}\n\n\tlog.Info(\"Cleaning up exposed environment\")\n\tr.restoreExposer.CleanUp(ctx, objRef)\n\n\tif err := UpdateDataDownloadWithRetry(ctx, r.client, types.NamespacedName{Namespace: dd.Namespace, Name: dd.Name}, log, func(dd *velerov2alpha1api.DataDownload) bool {\n\t\tif isDataDownloadInFinalState(dd) {\n\t\t\treturn false\n\t\t}\n\n\t\tdd.Status.Phase = velerov2alpha1api.DataDownloadPhaseCompleted\n\t\tdd.Status.CompletionTimestamp = &metav1.Time{Time: r.Clock.Now()}\n\n\t\tdelete(dd.Labels, exposer.ExposeOnGoingLabel)\n\n\t\treturn true\n\t}); err != nil {\n\t\tlog.WithError(err).Error(\"error updating data download status\")\n\t} else {\n\t\tlog.Infof(\"Data download is marked as %s\", dd.Status.Phase)\n\t\tr.metrics.RegisterDataDownloadSuccess(r.nodeName)\n\t}\n}\n\nfunc (r *DataDownloadReconciler) OnDataDownloadFailed(ctx context.Context, namespace string, ddName string, err error) {\n\tdefer r.dataPathMgr.RemoveAsyncBR(ddName)\n\n\tlog := r.logger.WithField(\"datadownload\", ddName)\n\n\tlog.WithError(err).Error(\"Async fs restore data path failed\")\n\n\tvar dd velerov2alpha1api.DataDownload\n\tif getErr := r.client.Get(ctx, types.NamespacedName{Name: ddName, Namespace: namespace}, &dd); getErr != nil {\n\t\tlog.WithError(getErr).Warn(\"Failed to get data download on failure\")\n\t} else {\n\t\t_, _ = r.errorOut(ctx, &dd, err, \"data path restore failed\", log)\n\t}\n}\n\nfunc (r *DataDownloadReconciler) OnDataDownloadCancelled(ctx context.Context, namespace string, ddName string) {\n\tdefer r.dataPathMgr.RemoveAsyncBR(ddName)\n\n\tlog := r.logger.WithField(\"datadownload\", ddName)\n\n\tlog.Warn(\"Async fs backup data path canceled\")\n\n\tvar dd velerov2alpha1api.DataDownload\n\tif getErr := r.client.Get(ctx, types.NamespacedName{Name: ddName, Namespace: namespace}, &dd); getErr != nil {\n\t\tlog.WithError(getErr).Warn(\"Failed to get datadownload on cancel\")\n\t\treturn\n\t}\n\t// cleans up any objects generated during the snapshot expose\n\tr.restoreExposer.CleanUp(ctx, getDataDownloadOwnerObject(&dd))\n\n\tif err := UpdateDataDownloadWithRetry(ctx, r.client, types.NamespacedName{Namespace: dd.Namespace, Name: dd.Name}, log, func(dd *velerov2alpha1api.DataDownload) bool {\n\t\tif isDataDownloadInFinalState(dd) {\n\t\t\treturn false\n\t\t}\n\n\t\tdd.Status.Phase = velerov2alpha1api.DataDownloadPhaseCanceled\n\t\tif dd.Status.StartTimestamp.IsZero() {\n\t\t\tdd.Status.StartTimestamp = &metav1.Time{Time: r.Clock.Now()}\n\t\t}\n\t\tdd.Status.CompletionTimestamp = &metav1.Time{Time: r.Clock.Now()}\n\n\t\tdelete(dd.Labels, exposer.ExposeOnGoingLabel)\n\n\t\treturn true\n\t}); err != nil {\n\t\tlog.WithError(err).Error(\"error updating data download status\")\n\t} else {\n\t\tr.metrics.RegisterDataDownloadCancel(r.nodeName)\n\t\tdelete(r.cancelledDataDownload, dd.Name)\n\t}\n}\n\nfunc (r *DataDownloadReconciler) tryCancelDataDownload(ctx context.Context, dd *velerov2alpha1api.DataDownload, message string) bool {\n\tlog := r.logger.WithField(\"datadownload\", dd.Name)\n\tsucceeded, err := funcExclusiveUpdateDataDownload(ctx, r.client, dd, func(dataDownload *velerov2alpha1api.DataDownload) {\n\t\tdataDownload.Status.Phase = velerov2alpha1api.DataDownloadPhaseCanceled\n\t\tif dataDownload.Status.StartTimestamp.IsZero() {\n\t\t\tdataDownload.Status.StartTimestamp = &metav1.Time{Time: r.Clock.Now()}\n\t\t}\n\t\tdataDownload.Status.CompletionTimestamp = &metav1.Time{Time: r.Clock.Now()}\n\n\t\tif message != \"\" {\n\t\t\tdataDownload.Status.Message = message\n\t\t}\n\n\t\tdelete(dataDownload.Labels, exposer.ExposeOnGoingLabel)\n\t})\n\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"error updating datadownload status\")\n\t\treturn false\n\t} else if !succeeded {\n\t\tlog.Warn(\"conflict in updating datadownload status and will try it again later\")\n\t\treturn false\n\t}\n\n\t// success update\n\tr.metrics.RegisterDataDownloadCancel(r.nodeName)\n\tr.restoreExposer.CleanUp(ctx, getDataDownloadOwnerObject(dd))\n\n\tlog.Warn(\"data download is canceled\")\n\n\treturn true\n}\n\nfunc (r *DataDownloadReconciler) OnDataDownloadProgress(ctx context.Context, namespace string, ddName string, progress *uploader.Progress) {\n\tlog := r.logger.WithField(\"datadownload\", ddName)\n\n\tif err := UpdateDataDownloadWithRetry(ctx, r.client, types.NamespacedName{Namespace: namespace, Name: ddName}, log, func(dd *velerov2alpha1api.DataDownload) bool {\n\t\tdd.Status.Progress = shared.DataMoveOperationProgress{TotalBytes: progress.TotalBytes, BytesDone: progress.BytesDone}\n\t\treturn true\n\t}); err != nil {\n\t\tlog.WithError(err).Error(\"Failed to update progress\")\n\t}\n}\n\n// SetupWithManager registers the DataDownload controller.\n// The fresh new DataDownload CR first created will trigger to create one pod (long time, maybe failure or unknown status) by one of the datadownload controllers\n// then the request will get out of the Reconcile queue immediately by not blocking others' CR handling, in order to finish the rest data download process we need to\n// re-enqueue the previous related request once the related pod is in running status to keep going on the rest logic. and below logic will avoid handling the unwanted\n// pod status and also avoid block others CR handling\nfunc (r *DataDownloadReconciler) SetupWithManager(mgr ctrl.Manager) error {\n\tgp := kube.NewGenericEventPredicate(func(object client.Object) bool {\n\t\tdd := object.(*velerov2alpha1api.DataDownload)\n\t\tif dd.Status.Phase == velerov2alpha1api.DataDownloadPhaseAccepted {\n\t\t\treturn true\n\t\t}\n\n\t\tif dd.Spec.Cancel && !isDataDownloadInFinalState(dd) {\n\t\t\treturn true\n\t\t}\n\n\t\tif isDataDownloadInFinalState(dd) && !dd.DeletionTimestamp.IsZero() {\n\t\t\treturn true\n\t\t}\n\n\t\treturn false\n\t})\n\ts := kube.NewPeriodicalEnqueueSource(r.logger.WithField(\"controller\", constant.ControllerDataDownload), r.client, &velerov2alpha1api.DataDownloadList{}, preparingMonitorFrequency, kube.PeriodicalEnqueueSourceOption{\n\t\tPredicates: []predicate.Predicate{gp},\n\t})\n\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\tFor(&velerov2alpha1api.DataDownload{}).\n\t\tWatchesRawSource(s).\n\t\tWatches(&corev1api.Pod{}, kube.EnqueueRequestsFromMapUpdateFunc(r.findSnapshotRestoreForPod),\n\t\t\tbuilder.WithPredicates(predicate.Funcs{\n\t\t\t\tUpdateFunc: func(ue event.UpdateEvent) bool {\n\t\t\t\t\tnewObj := ue.ObjectNew.(*corev1api.Pod)\n\n\t\t\t\t\tif _, ok := newObj.Labels[velerov1api.DataDownloadLabel]; !ok {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\n\t\t\t\t\tif newObj.Spec.NodeName == \"\" {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\n\t\t\t\t\treturn true\n\t\t\t\t},\n\t\t\t\tCreateFunc: func(event.CreateEvent) bool {\n\t\t\t\t\treturn false\n\t\t\t\t},\n\t\t\t\tDeleteFunc: func(de event.DeleteEvent) bool {\n\t\t\t\t\treturn false\n\t\t\t\t},\n\t\t\t\tGenericFunc: func(ge event.GenericEvent) bool {\n\t\t\t\t\treturn false\n\t\t\t\t},\n\t\t\t})).\n\t\tComplete(r)\n}\n\nfunc (r *DataDownloadReconciler) findSnapshotRestoreForPod(ctx context.Context, podObj client.Object) []reconcile.Request {\n\tpod := podObj.(*corev1api.Pod)\n\tdd, err := findDataDownloadByPod(r.client, *pod)\n\n\tlog := r.logger.WithField(\"pod\", pod.Name)\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"unable to get DataDownload\")\n\t\treturn []reconcile.Request{}\n\t} else if dd == nil {\n\t\tlog.Error(\"get empty DataDownload\")\n\t\treturn []reconcile.Request{}\n\t}\n\tlog = log.WithFields(logrus.Fields{\n\t\t\"Dataddownload\": dd.Name,\n\t})\n\n\tif dd.Status.Phase != velerov2alpha1api.DataDownloadPhaseAccepted {\n\t\treturn []reconcile.Request{}\n\t}\n\n\tif pod.Status.Phase == corev1api.PodRunning {\n\t\tlog.Info(\"Preparing data download\")\n\t\tif err = UpdateDataDownloadWithRetry(context.Background(), r.client, types.NamespacedName{Namespace: dd.Namespace, Name: dd.Name}, log,\n\t\t\tfunc(dd *velerov2alpha1api.DataDownload) bool {\n\t\t\t\tif isDataDownloadInFinalState(dd) {\n\t\t\t\t\tlog.Warnf(\"datadownload %s is terminated, abort setting it to prepared\", dd.Name)\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tr.prepareDataDownload(dd)\n\t\t\t\treturn true\n\t\t\t}); err != nil {\n\t\t\tlog.WithError(err).Warn(\"failed to update dataudownload, prepare will halt for this dataudownload\")\n\t\t\treturn []reconcile.Request{}\n\t\t}\n\t} else if unrecoverable, reason := kube.IsPodUnrecoverable(pod, log); unrecoverable {\n\t\terr := UpdateDataDownloadWithRetry(context.Background(), r.client, types.NamespacedName{Namespace: dd.Namespace, Name: dd.Name}, r.logger.WithField(\"datadownlad\", dd.Name),\n\t\t\tfunc(dataDownload *velerov2alpha1api.DataDownload) bool {\n\t\t\t\tif dataDownload.Spec.Cancel {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tdataDownload.Spec.Cancel = true\n\t\t\t\tdataDownload.Status.Message = fmt.Sprintf(\"Cancel datadownload because the exposing pod %s/%s is in abnormal status for reason %s\", pod.Namespace, pod.Name, reason)\n\n\t\t\t\treturn true\n\t\t\t})\n\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Warn(\"failed to cancel datadownload, and it will wait for prepare timeout\")\n\t\t\treturn []reconcile.Request{}\n\t\t}\n\t\tlog.Infof(\"Exposed pod is in abnormal status(reason %s) and datadownload is marked as cancel\", reason)\n\t} else {\n\t\treturn []reconcile.Request{}\n\t}\n\n\trequest := reconcile.Request{\n\t\tNamespacedName: types.NamespacedName{\n\t\t\tNamespace: dd.Namespace,\n\t\t\tName: dd.Name,\n\t\t},\n\t}\n\treturn []reconcile.Request{request}\n}\n\nfunc (r *DataDownloadReconciler) prepareDataDownload(ssb *velerov2alpha1api.DataDownload) {\n\tssb.Status.Phase = velerov2alpha1api.DataDownloadPhasePrepared\n\tssb.Status.Node = r.nodeName\n}\n\nfunc (r *DataDownloadReconciler) errorOut(ctx context.Context, dd *velerov2alpha1api.DataDownload, err error, msg string, log logrus.FieldLogger) (ctrl.Result, error) {\n\tif r.restoreExposer != nil {\n\t\tr.restoreExposer.CleanUp(ctx, getDataDownloadOwnerObject(dd))\n\t}\n\treturn ctrl.Result{}, r.updateStatusToFailed(ctx, dd, err, msg, log)\n}\n\nfunc (r *DataDownloadReconciler) updateStatusToFailed(ctx context.Context, dd *velerov2alpha1api.DataDownload, err error, msg string, log logrus.FieldLogger) error {\n\tlog.Info(\"update data download status to Failed\")\n\n\tif patchErr := UpdateDataDownloadWithRetry(ctx, r.client, types.NamespacedName{Namespace: dd.Namespace, Name: dd.Name}, log, func(dd *velerov2alpha1api.DataDownload) bool {\n\t\tif isDataDownloadInFinalState(dd) {\n\t\t\treturn false\n\t\t}\n\n\t\tdd.Status.Phase = velerov2alpha1api.DataDownloadPhaseFailed\n\t\tdd.Status.Message = errors.WithMessage(err, msg).Error()\n\t\tdd.Status.CompletionTimestamp = &metav1.Time{Time: r.Clock.Now()}\n\n\t\tdelete(dd.Labels, exposer.ExposeOnGoingLabel)\n\n\t\treturn true\n\t}); patchErr != nil {\n\t\tlog.WithError(patchErr).Error(\"error updating DataDownload status\")\n\t} else {\n\t\tr.metrics.RegisterDataDownloadFailure(r.nodeName)\n\t}\n\n\treturn err\n}\n\nfunc (r *DataDownloadReconciler) acceptDataDownload(ctx context.Context, dd *velerov2alpha1api.DataDownload) (bool, error) {\n\tr.logger.Infof(\"Accepting data download %s\", dd.Name)\n\n\t// For all data download controller in each node-agent will try to update download CR, and only one controller will success,\n\t// and the success one could handle later logic\n\n\tupdated := dd.DeepCopy()\n\n\tupdateFunc := func(datadownload *velerov2alpha1api.DataDownload) {\n\t\tdatadownload.Status.Phase = velerov2alpha1api.DataDownloadPhaseAccepted\n\t\tdatadownload.Status.AcceptedByNode = r.nodeName\n\t\tdatadownload.Status.AcceptedTimestamp = &metav1.Time{Time: r.Clock.Now()}\n\n\t\tif datadownload.Labels == nil {\n\t\t\tdatadownload.Labels = make(map[string]string)\n\t\t}\n\t\tdatadownload.Labels[exposer.ExposeOnGoingLabel] = \"true\"\n\t}\n\n\tsucceeded, err := funcExclusiveUpdateDataDownload(ctx, r.client, updated, updateFunc)\n\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\tif succeeded {\n\t\tupdateFunc(dd) // If update success, it's need to update du values in memory\n\t\tr.logger.WithField(\"DataDownload\", dd.Name).Infof(\"This datadownload has been accepted by %s\", r.nodeName)\n\t\treturn true, nil\n\t}\n\n\tr.logger.WithField(\"DataDownload\", dd.Name).Info(\"This datadownload has been accepted by others\")\n\treturn false, nil\n}\n\nfunc (r *DataDownloadReconciler) onPrepareTimeout(ctx context.Context, dd *velerov2alpha1api.DataDownload) {\n\tlog := r.logger.WithField(\"DataDownload\", dd.Name)\n\n\tlog.Info(\"Timeout happened for preparing datadownload\")\n\tsucceeded, err := funcExclusiveUpdateDataDownload(ctx, r.client, dd, func(dd *velerov2alpha1api.DataDownload) {\n\t\tdd.Status.Phase = velerov2alpha1api.DataDownloadPhaseFailed\n\t\tdd.Status.Message = \"timeout on preparing data download\"\n\n\t\tdelete(dd.Labels, exposer.ExposeOnGoingLabel)\n\t})\n\n\tif err != nil {\n\t\tlog.WithError(err).Warn(\"Failed to update datadownload\")\n\t\treturn\n\t}\n\n\tif !succeeded {\n\t\tlog.Warn(\"Datadownload has been updated by others\")\n\t\treturn\n\t}\n\n\tdiags := strings.Split(r.restoreExposer.DiagnoseExpose(ctx, getDataDownloadOwnerObject(dd)), \"\\n\")\n\tfor _, diag := range diags {\n\t\tlog.Warnf(\"[Diagnose DD expose]%s\", diag)\n\t}\n\n\tr.restoreExposer.CleanUp(ctx, getDataDownloadOwnerObject(dd))\n\n\tlog.Info(\"Datadownload has been cleaned up\")\n\n\tr.metrics.RegisterDataDownloadFailure(r.nodeName)\n}\n\nvar funcExclusiveUpdateDataDownload = exclusiveUpdateDataDownload\n\nfunc exclusiveUpdateDataDownload(ctx context.Context, cli client.Client, dd *velerov2alpha1api.DataDownload,\n\tupdateFunc func(*velerov2alpha1api.DataDownload)) (bool, error) {\n\tupdateFunc(dd)\n\n\terr := cli.Update(ctx, dd)\n\n\tif err == nil {\n\t\treturn true, nil\n\t}\n\t// it won't rollback dd in memory when error\n\tif apierrors.IsConflict(err) {\n\t\treturn false, nil\n\t} else {\n\t\treturn false, err\n\t}\n}\n\nfunc (r *DataDownloadReconciler) getTargetPVC(ctx context.Context, dd *velerov2alpha1api.DataDownload) (*corev1api.PersistentVolumeClaim, error) {\n\treturn r.kubeClient.CoreV1().PersistentVolumeClaims(dd.Spec.TargetVolume.Namespace).Get(ctx, dd.Spec.TargetVolume.PVC, metav1.GetOptions{})\n}\n\nfunc (r *DataDownloadReconciler) closeDataPath(ctx context.Context, ddName string) {\n\tasyncBR := r.dataPathMgr.GetAsyncBR(ddName)\n\tif asyncBR != nil {\n\t\tasyncBR.Close(ctx)\n\t}\n\n\tr.dataPathMgr.RemoveAsyncBR(ddName)\n}\n\nfunc (r *DataDownloadReconciler) setupExposeParam(dd *velerov2alpha1api.DataDownload) (exposer.GenericRestoreExposeParam, error) {\n\tlog := r.logger.WithField(\"datadownload\", dd.Name)\n\n\tnodeOS := string(dd.Spec.NodeOS)\n\tif nodeOS == \"\" {\n\t\tlog.Info(\"nodeOS is empty in DD, fallback to linux\")\n\t\tnodeOS = kube.NodeOSLinux\n\t}\n\n\tif err := kube.HasNodeWithOS(context.Background(), nodeOS, r.kubeClient.CoreV1()); err != nil {\n\t\treturn exposer.GenericRestoreExposeParam{}, errors.Wrapf(err, \"no appropriate node to run datadownload %s/%s\", dd.Namespace, dd.Name)\n\t}\n\n\thostingPodLabels := map[string]string{velerov1api.DataDownloadLabel: dd.Name}\n\tif len(r.podLabels) > 0 {\n\t\tfor k, v := range r.podLabels {\n\t\t\thostingPodLabels[k] = v\n\t\t}\n\t} else {\n\t\tfor _, k := range util.ThirdPartyLabels {\n\t\t\tif v, err := nodeagent.GetLabelValue(context.Background(), r.kubeClient, dd.Namespace, k, nodeOS); err != nil {\n\t\t\t\tif err != nodeagent.ErrNodeAgentLabelNotFound {\n\t\t\t\t\tlog.WithError(err).Warnf(\"Failed to check node-agent label, skip adding host pod label %s\", k)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\thostingPodLabels[k] = v\n\t\t\t}\n\t\t}\n\t}\n\n\thostingPodAnnotation := map[string]string{}\n\tif len(r.podAnnotations) > 0 {\n\t\tfor k, v := range r.podAnnotations {\n\t\t\thostingPodAnnotation[k] = v\n\t\t}\n\t} else {\n\t\tfor _, k := range util.ThirdPartyAnnotations {\n\t\t\tif v, err := nodeagent.GetAnnotationValue(context.Background(), r.kubeClient, dd.Namespace, k, nodeOS); err != nil {\n\t\t\t\tif err != nodeagent.ErrNodeAgentAnnotationNotFound {\n\t\t\t\t\tlog.WithError(err).Warnf(\"Failed to check node-agent annotation, skip adding host pod annotation %s\", k)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\thostingPodAnnotation[k] = v\n\t\t\t}\n\t\t}\n\t}\n\n\thostingPodTolerations := []corev1api.Toleration{}\n\tfor _, k := range util.ThirdPartyTolerations {\n\t\tif v, err := nodeagent.GetToleration(context.Background(), r.kubeClient, dd.Namespace, k, nodeOS); err != nil {\n\t\t\tif err != nodeagent.ErrNodeAgentTolerationNotFound {\n\t\t\t\tlog.WithError(err).Warnf(\"Failed to check node-agent toleration, skip adding host pod toleration %s\", k)\n\t\t\t}\n\t\t} else {\n\t\t\thostingPodTolerations = append(hostingPodTolerations, *v)\n\t\t}\n\t}\n\n\tvar cacheVolume *exposer.CacheConfigs\n\tif r.cacheVolumeConfigs != nil {\n\t\tif limit, err := r.repoConfigMgr.ClientSideCacheLimit(velerov1api.BackupRepositoryTypeKopia, r.backupRepoConfigs); err != nil {\n\t\t\tlog.WithError(err).Warnf(\"Failed to get client side cache limit for repo type %s from configs %v\", velerov1api.BackupRepositoryTypeKopia, r.backupRepoConfigs)\n\t\t} else {\n\t\t\tcacheVolume = &exposer.CacheConfigs{\n\t\t\t\tLimit: limit,\n\t\t\t\tStorageClass: r.cacheVolumeConfigs.StorageClass,\n\t\t\t\tResidentThreshold: r.cacheVolumeConfigs.ResidentThresholdInMB << 20,\n\t\t\t}\n\t\t}\n\t}\n\n\treturn exposer.GenericRestoreExposeParam{\n\t\tTargetPVCName: dd.Spec.TargetVolume.PVC,\n\t\tTargetNamespace: dd.Spec.TargetVolume.Namespace,\n\t\tHostingPodLabels: hostingPodLabels,\n\t\tHostingPodAnnotations: hostingPodAnnotation,\n\t\tHostingPodTolerations: hostingPodTolerations,\n\t\tResources: r.podResources,\n\t\tOperationTimeout: dd.Spec.OperationTimeout.Duration,\n\t\tExposeTimeout: r.preparingTimeout,\n\t\tNodeOS: nodeOS,\n\t\tRestorePVCConfig: r.restorePVCConfig,\n\t\tLoadAffinity: r.loadAffinity,\n\t\tPriorityClassName: r.dataMovePriorityClass,\n\t\tRestoreSize: dd.Spec.SnapshotSize,\n\t\tCacheVolume: cacheVolume,\n\t}, nil\n}\n\nfunc getDataDownloadOwnerObject(dd *velerov2alpha1api.DataDownload) corev1api.ObjectReference {\n\treturn corev1api.ObjectReference{\n\t\tKind: dd.Kind,\n\t\tNamespace: dd.Namespace,\n\t\tName: dd.Name,\n\t\tUID: dd.UID,\n\t\tAPIVersion: dd.APIVersion,\n\t}\n}\n\nfunc findDataDownloadByPod(client client.Client, pod corev1api.Pod) (*velerov2alpha1api.DataDownload, error) {\n\tif label, exist := pod.Labels[velerov1api.DataDownloadLabel]; exist {\n\t\tdd := &velerov2alpha1api.DataDownload{}\n\t\terr := client.Get(context.Background(), types.NamespacedName{\n\t\t\tNamespace: pod.Namespace,\n\t\t\tName: label,\n\t\t}, dd)\n\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"error to find DataDownload by pod %s/%s\", pod.Namespace, pod.Name)\n\t\t}\n\t\treturn dd, nil\n\t}\n\n\treturn nil, nil\n}\n\nfunc isDataDownloadInFinalState(dd *velerov2alpha1api.DataDownload) bool {\n\treturn dd.Status.Phase == velerov2alpha1api.DataDownloadPhaseFailed ||\n\t\tdd.Status.Phase == velerov2alpha1api.DataDownloadPhaseCanceled ||\n\t\tdd.Status.Phase == velerov2alpha1api.DataDownloadPhaseCompleted\n}\n\nfunc UpdateDataDownloadWithRetry(ctx context.Context, client client.Client, namespacedName types.NamespacedName, log logrus.FieldLogger, updateFunc func(*velerov2alpha1api.DataDownload) bool) error {\n\treturn wait.PollUntilContextCancel(ctx, time.Second, true, func(ctx context.Context) (bool, error) {\n\t\tdd := &velerov2alpha1api.DataDownload{}\n\t\tif err := client.Get(ctx, namespacedName, dd); err != nil {\n\t\t\treturn false, errors.Wrap(err, \"getting DataDownload\")\n\t\t}\n\n\t\tif updateFunc(dd) {\n\t\t\terr := client.Update(ctx, dd)\n\t\t\tif err != nil {\n\t\t\t\tif apierrors.IsConflict(err) {\n\t\t\t\t\tlog.Debugf(\"failed to update datadownload for %s/%s and will retry it\", dd.Namespace, dd.Name)\n\t\t\t\t\treturn false, nil\n\t\t\t\t} else {\n\t\t\t\t\treturn false, errors.Wrapf(err, \"error updating datadownload %s/%s\", dd.Namespace, dd.Name)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\treturn true, nil\n\t})\n}\n\nvar funcResumeCancellableDataRestore = (*DataDownloadReconciler).resumeCancellableDataPath\n\nfunc (r *DataDownloadReconciler) AttemptDataDownloadResume(ctx context.Context, logger *logrus.Entry, ns string) error {\n\tdataDownloads := &velerov2alpha1api.DataDownloadList{}\n\tif err := r.client.List(ctx, dataDownloads, &client.ListOptions{Namespace: ns}); err != nil {\n\t\tr.logger.WithError(errors.WithStack(err)).Error(\"failed to list datadownloads\")\n\t\treturn errors.Wrapf(err, \"error to list datadownloads\")\n\t}\n\n\tfor i := range dataDownloads.Items {\n\t\tdd := &dataDownloads.Items[i]\n\t\tif dd.Status.Phase == velerov2alpha1api.DataDownloadPhaseInProgress {\n\t\t\tif dd.Status.Node != r.nodeName {\n\t\t\t\tlogger.WithField(\"dd\", dd.Name).WithField(\"current node\", r.nodeName).Infof(\"DD should be resumed by another node %s\", dd.Status.Node)\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\terr := funcResumeCancellableDataRestore(r, ctx, dd, logger)\n\t\t\tif err == nil {\n\t\t\t\tlogger.WithField(\"dd\", dd.Name).WithField(\"current node\", r.nodeName).Info(\"Completed to resume in progress DD\")\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tlogger.WithField(\"datadownload\", dd.GetName()).WithError(err).Warn(\"Failed to resume data path for dd, have to cancel it\")\n\n\t\t\tresumeErr := err\n\t\t\terr = UpdateDataDownloadWithRetry(ctx, r.client, types.NamespacedName{Namespace: dd.Namespace, Name: dd.Name}, logger.WithField(\"datadownload\", dd.Name),\n\t\t\t\tfunc(dataDownload *velerov2alpha1api.DataDownload) bool {\n\t\t\t\t\tif dataDownload.Spec.Cancel {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\n\t\t\t\t\tdataDownload.Spec.Cancel = true\n\t\t\t\t\tdataDownload.Status.Message = fmt.Sprintf(\"Resume InProgress datadownload failed with error %v, mark it as cancel\", resumeErr)\n\n\t\t\t\t\treturn true\n\t\t\t\t})\n\t\t\tif err != nil {\n\t\t\t\tlogger.WithError(errors.WithStack(err)).WithError(errors.WithStack(err)).Error(\"Failed to trigger datadownload cancel\")\n\t\t\t}\n\t\t} else if !isDataDownloadInFinalState(dd) {\n\t\t\t// the Prepared CR could be still handled by datadownload controller after node-agent restart\n\t\t\t// the accepted CR may also suvived from node-agent restart as long as the intermediate objects are all done\n\t\t\tlogger.WithField(\"datadownload\", dd.GetName()).Infof(\"find a datadownload with status %s\", dd.Status.Phase)\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (r *DataDownloadReconciler) resumeCancellableDataPath(ctx context.Context, dd *velerov2alpha1api.DataDownload, log logrus.FieldLogger) error {\n\tlog.Info(\"Resume cancelable dataDownload\")\n\n\tres, err := r.restoreExposer.GetExposed(ctx, getDataDownloadOwnerObject(dd), r.client, r.nodeName, dd.Spec.OperationTimeout.Duration)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to get exposed volume for dd %s\", dd.Name)\n\t}\n\n\tif res == nil {\n\t\treturn errors.Errorf(\"expose info missed for dd %s\", dd.Name)\n\t}\n\n\tcallbacks := datapath.Callbacks{\n\t\tOnCompleted: r.OnDataDownloadCompleted,\n\t\tOnFailed: r.OnDataDownloadFailed,\n\t\tOnCancelled: r.OnDataDownloadCancelled,\n\t\tOnProgress: r.OnDataDownloadProgress,\n\t}\n\n\tasyncBR, err := r.dataPathMgr.CreateMicroServiceBRWatcher(ctx, r.client, r.kubeClient, r.mgr, datapath.TaskTypeRestore, dd.Name, dd.Namespace, res.ByPod.HostingPod.Name, res.ByPod.HostingContainer, dd.Name, callbacks, true, log)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to create asyncBR watcher for dd %s\", dd.Name)\n\t}\n\n\tresumeComplete := false\n\tdefer func() {\n\t\tif !resumeComplete {\n\t\t\tr.closeDataPath(ctx, dd.Name)\n\t\t}\n\t}()\n\n\tif err := asyncBR.Init(ctx, nil); err != nil {\n\t\treturn errors.Wrapf(err, \"error to init asyncBR watcher for dd %s\", dd.Name)\n\t}\n\n\tif err := asyncBR.StartRestore(dd.Spec.SnapshotID, datapath.AccessPoint{\n\t\tByPath: res.ByPod.VolumeName,\n\t}, nil); err != nil {\n\t\treturn errors.Wrapf(err, \"error to resume asyncBR watcher for dd %s\", dd.Name)\n\t}\n\n\tresumeComplete = true\n\n\tlog.Infof(\"asyncBR is resumed for dd %s\", dd.Name)\n\n\treturn nil\n}\n" }, { "path": "pkg/controller/data_download_controller_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/client-go/kubernetes\"\n\tclientgofake \"k8s.io/client-go/kubernetes/fake\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/manager\"\n\t\"sigs.k8s.io/controller-runtime/pkg/reconcile\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\tdatapathmockes \"github.com/vmware-tanzu/velero/pkg/datapath/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/exposer\"\n\texposermockes \"github.com/vmware-tanzu/velero/pkg/exposer/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\tvelerotypes \"github.com/vmware-tanzu/velero/pkg/types\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nconst dataDownloadName string = \"datadownload-1\"\n\nfunc dataDownloadBuilder() *builder.DataDownloadBuilder {\n\treturn builder.ForDataDownload(velerov1api.DefaultNamespace, dataDownloadName).\n\t\tBackupStorageLocation(\"bsl-loc\").\n\t\tDataMover(\"velero\").\n\t\tSnapshotID(\"test-snapshot-id\").TargetVolume(velerov2alpha1api.TargetVolumeSpec{\n\t\tPV: \"test-pv\",\n\t\tPVC: \"test-pvc\",\n\t\tNamespace: \"test-ns\",\n\t})\n}\n\nfunc initDataDownloadReconciler(t *testing.T, objects []any, needError ...bool) (*DataDownloadReconciler, error) {\n\tt.Helper()\n\n\tvar errs = make([]error, 6)\n\tfor k, isError := range needError {\n\t\tif k == 0 && isError {\n\t\t\terrs[0] = fmt.Errorf(\"Get error\")\n\t\t} else if k == 1 && isError {\n\t\t\terrs[1] = fmt.Errorf(\"Create error\")\n\t\t} else if k == 2 && isError {\n\t\t\terrs[2] = fmt.Errorf(\"Update error\")\n\t\t} else if k == 3 && isError {\n\t\t\terrs[3] = fmt.Errorf(\"Patch error\")\n\t\t} else if k == 4 && isError {\n\t\t\terrs[4] = apierrors.NewConflict(velerov2alpha1api.Resource(\"datadownload\"), dataDownloadName, errors.New(\"conflict\"))\n\t\t} else if k == 5 && isError {\n\t\t\terrs[5] = fmt.Errorf(\"List error\")\n\t\t}\n\t}\n\treturn initDataDownloadReconcilerWithError(t, objects, errs...)\n}\n\nfunc initDataDownloadReconcilerWithError(t *testing.T, objects []any, needError ...error) (*DataDownloadReconciler, error) {\n\tt.Helper()\n\n\truntimeObjects := make([]runtime.Object, 0)\n\n\tfor _, obj := range objects {\n\t\truntimeObjects = append(runtimeObjects, obj.(runtime.Object))\n\t}\n\n\tfakeClient := FakeClient{\n\t\tClient: velerotest.NewFakeControllerRuntimeClient(t, runtimeObjects...),\n\t}\n\n\tfakeKubeClient := clientgofake.NewSimpleClientset(runtimeObjects...)\n\n\tfor k := range needError {\n\t\tif k == 0 {\n\t\t\tfakeClient.getError = needError[0]\n\t\t} else if k == 1 {\n\t\t\tfakeClient.createError = needError[1]\n\t\t} else if k == 2 {\n\t\t\tfakeClient.updateError = needError[2]\n\t\t} else if k == 3 {\n\t\t\tfakeClient.patchError = needError[3]\n\t\t} else if k == 4 {\n\t\t\tfakeClient.updateConflict = needError[4]\n\t\t} else if k == 5 {\n\t\t\tfakeClient.listError = needError[5]\n\t\t}\n\t}\n\n\tfakeFS := velerotest.NewFakeFileSystem()\n\tpathGlob := fmt.Sprintf(\"/host_pods/%s/volumes/*/%s\", \"test-uid\", \"test-pvc\")\n\t_, err := fakeFS.Create(pathGlob)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdataPathMgr := datapath.NewManager(1)\n\n\treturn NewDataDownloadReconciler(\n\t\t&fakeClient,\n\t\tnil,\n\t\tfakeKubeClient,\n\t\tdataPathMgr,\n\t\tnil,\n\t\tnil,\n\t\tvelerotypes.RestorePVC{},\n\t\tnil,\n\t\tnil,\n\t\tcorev1api.ResourceRequirements{},\n\t\t\"test-node\",\n\t\ttime.Minute*5,\n\t\tvelerotest.NewLogger(),\n\t\tmetrics.NewServerMetrics(),\n\t\t\"\",\n\t\tnil,\n\t\tnil, // podLabels\n\t\tnil, // podAnnotations\n\t), nil\n}\n\nfunc TestDataDownloadReconcile(t *testing.T) {\n\tsc := builder.ForStorageClass(\"sc\").Result()\n\n\tdaemonSet := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"velero\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t\tAPIVersion: appsv1api.SchemeGroupVersion.String(),\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tImage: \"fake-image\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tnode := builder.ForNode(\"fake-node\").Labels(map[string]string{kube.NodeOSLabel: kube.NodeOSLinux}).Result()\n\n\ttests := []struct {\n\t\tname string\n\t\tdd *velerov2alpha1api.DataDownload\n\t\tnotCreateDD bool\n\t\ttargetPVC *corev1api.PersistentVolumeClaim\n\t\tdataMgr *datapath.Manager\n\t\tneedErrs []bool\n\t\tneedCreateFSBR bool\n\t\tneedDelete bool\n\t\tsportTime *metav1.Time\n\t\tisExposeErr bool\n\t\tisGetExposeErr bool\n\t\tisGetExposeNil bool\n\t\tisPeekExposeErr bool\n\t\tisNilExposer bool\n\t\tnotNilExpose bool\n\t\tnotMockCleanUp bool\n\t\tmockInit bool\n\t\tmockInitErr error\n\t\tmockStart bool\n\t\tmockStartErr error\n\t\tmockCancel bool\n\t\tmockClose bool\n\t\tneedExclusiveUpdateError error\n\t\tconstrained bool\n\t\texpected *velerov2alpha1api.DataDownload\n\t\texpectDeleted bool\n\t\texpectCancelRecord bool\n\t\texpectedResult *ctrl.Result\n\t\texpectedErr string\n\t\texpectDataPath bool\n\t}{\n\t\t{\n\t\t\tname: \"dd not found\",\n\t\t\tdd: dataDownloadBuilder().Result(),\n\t\t\tnotCreateDD: true,\n\t\t},\n\t\t{\n\t\t\tname: \"dd not created in velero default namespace\",\n\t\t\tdd: builder.ForDataDownload(\"test-ns\", dataDownloadName).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"get dd fail\",\n\t\t\tdd: dataDownloadBuilder().Result(),\n\t\t\tneedErrs: []bool{true, false, false, false},\n\t\t\texpectedErr: \"Get error\",\n\t\t},\n\t\t{\n\t\t\tname: \"dd is not for built-in dm\",\n\t\t\tdd: dataDownloadBuilder().DataMover(\"other\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"add finalizer to dd\",\n\t\t\tdd: dataDownloadBuilder().Result(),\n\t\t\texpected: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"add finalizer to dd failed\",\n\t\t\tdd: dataDownloadBuilder().Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\texpectedErr: \"error updating datadownload velero/datadownload-1: Update error\",\n\t\t},\n\t\t{\n\t\t\tname: \"dd is under deletion\",\n\t\t\tdd: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\tneedDelete: true,\n\t\t\texpected: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"dd is under deletion but cancel failed\",\n\t\t\tdd: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\tneedDelete: true,\n\t\t\texpectedErr: \"error updating datadownload velero/datadownload-1: Update error\",\n\t\t},\n\t\t{\n\t\t\tname: \"dd is under deletion and in terminal state\",\n\t\t\tdd: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Phase(velerov2alpha1api.DataDownloadPhaseFailed).Result(),\n\t\t\tsportTime: &metav1.Time{Time: time.Now()},\n\t\t\tneedDelete: true,\n\t\t\texpectDeleted: true,\n\t\t},\n\t\t{\n\t\t\tname: \"dd is under deletion and in terminal state, but remove finalizer failed\",\n\t\t\tdd: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Phase(velerov2alpha1api.DataDownloadPhaseFailed).Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\tneedDelete: true,\n\t\t\texpectedErr: \"error updating datadownload velero/datadownload-1: Update error\",\n\t\t},\n\t\t{\n\t\t\tname: \"delay cancel negative for others\",\n\t\t\tdd: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Phase(velerov2alpha1api.DataDownloadPhasePrepared).Result(),\n\t\t\tsportTime: &metav1.Time{Time: time.Now()},\n\t\t\texpectCancelRecord: true,\n\t\t},\n\t\t{\n\t\t\tname: \"delay cancel negative for inProgress\",\n\t\t\tdd: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Phase(velerov2alpha1api.DataDownloadPhaseInProgress).Result(),\n\t\t\tsportTime: &metav1.Time{Time: time.Now().Add(-time.Minute * 58)},\n\t\t\texpectCancelRecord: true,\n\t\t},\n\t\t{\n\t\t\tname: \"delay cancel affirmative for others\",\n\t\t\tdd: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Phase(velerov2alpha1api.DataDownloadPhasePrepared).Result(),\n\t\t\tsportTime: &metav1.Time{Time: time.Now().Add(-time.Minute * 5)},\n\t\t\texpected: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Phase(velerov2alpha1api.DataDownloadPhaseCanceled).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"delay cancel affirmative for inProgress\",\n\t\t\tdd: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Phase(velerov2alpha1api.DataDownloadPhaseInProgress).Result(),\n\t\t\tsportTime: &metav1.Time{Time: time.Now().Add(-time.Hour)},\n\t\t\texpected: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Phase(velerov2alpha1api.DataDownloadPhaseCanceled).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"delay cancel failed\",\n\t\t\tdd: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Phase(velerov2alpha1api.DataDownloadPhaseInProgress).Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\tsportTime: &metav1.Time{Time: time.Now().Add(-time.Hour)},\n\t\t\texpected: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Phase(velerov2alpha1api.DataDownloadPhaseInProgress).Result(),\n\t\t\texpectCancelRecord: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Unknown data download status\",\n\t\t\tdd: dataDownloadBuilder().Phase(\"Unknown\").Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"dd is cancel on new\",\n\t\t\tdd: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Result(),\n\t\t\ttargetPVC: builder.ForPersistentVolumeClaim(\"test-ns\", \"test-pvc\").Result(),\n\t\t\texpectCancelRecord: true,\n\t\t\texpected: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Phase(velerov2alpha1api.DataDownloadPhaseCanceled).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"new dd but constrained\",\n\t\t\tdd: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\tconstrained: true,\n\t\t\texpected: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\texpectedResult: &ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5},\n\t\t},\n\t\t{\n\t\t\tname: \"new dd but no target PVC\",\n\t\t\tdd: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\texpectedResult: &ctrl.Result{Requeue: true},\n\t\t},\n\t\t{\n\t\t\tname: \"new dd but accept failed\",\n\t\t\tdd: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\ttargetPVC: builder.ForPersistentVolumeClaim(\"test-ns\", \"test-pvc\").Result(),\n\t\t\tneedExclusiveUpdateError: errors.New(\"exclusive-update-error\"),\n\t\t\texpected: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\texpectedErr: \"error accepting the data download datadownload-1: exclusive-update-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"dd is accepted but setup expose param failed\",\n\t\t\tdd: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).NodeOS(\"xxx\").Result(),\n\t\t\ttargetPVC: builder.ForPersistentVolumeClaim(\"test-ns\", \"test-pvc\").Result(),\n\t\t\texpected: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).NodeOS(\"xxx\").Phase(velerov2alpha1api.DataDownloadPhaseFailed).Message(\"failed to set exposer parameters\").Result(),\n\t\t\texpectedErr: \"no appropriate node to run datadownload velero/datadownload-1: node with OS xxx doesn't exist\",\n\t\t},\n\t\t{\n\t\t\tname: \"dd expose failed\",\n\t\t\tdd: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\ttargetPVC: builder.ForPersistentVolumeClaim(\"test-ns\", \"test-pvc\").StorageClass(\"test-sc\").Result(),\n\t\t\tisExposeErr: true,\n\t\t\texpected: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Phase(velerov2alpha1api.DataDownloadPhaseFailed).Message(\"error to expose snapshot\").Result(),\n\t\t\texpectedErr: \"Error to expose restore exposer\",\n\t\t},\n\t\t{\n\t\t\tname: \"dd succeeds for accepted\",\n\t\t\tdd: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\ttargetPVC: builder.ForPersistentVolumeClaim(\"test-ns\", \"test-pvc\").StorageClass(\"sc\").Result(),\n\t\t\texpected: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Phase(velerov2alpha1api.DataDownloadPhaseAccepted).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"prepare timeout on accepted\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseAccepted).Finalizers([]string{DataUploadDownloadFinalizer}).AcceptedTimestamp(&metav1.Time{Time: time.Now().Add(-time.Minute * 30)}).Result(),\n\t\t\texpected: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseFailed).Finalizers([]string{DataUploadDownloadFinalizer}).Phase(velerov2alpha1api.DataDownloadPhaseFailed).Message(\"timeout on preparing data download\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"peek error on accepted\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseAccepted).Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\tisPeekExposeErr: true,\n\t\t\texpected: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseCanceled).Finalizers([]string{DataUploadDownloadFinalizer}).Phase(velerov2alpha1api.DataDownloadPhaseCanceled).Message(\"found a datadownload velero/datadownload-1 with expose error: fake-peek-error. mark it as cancel\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"cancel on prepared\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhasePrepared).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Cancel(true).Result(),\n\t\t\texpected: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseCanceled).Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Phase(velerov2alpha1api.DataDownloadPhaseCanceled).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Failed to get restore expose on prepared\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhasePrepared).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\ttargetPVC: builder.ForPersistentVolumeClaim(\"test-ns\", \"test-pvc\").Result(),\n\t\t\tisGetExposeErr: true,\n\t\t\texpected: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseFailed).Finalizers([]string{DataUploadDownloadFinalizer}).Message(\"restore exposer is not ready\").Result(),\n\t\t\texpectedErr: \"Error to get restore exposer\",\n\t\t},\n\t\t{\n\t\t\tname: \"Get nil restore expose on prepared\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhasePrepared).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\ttargetPVC: builder.ForPersistentVolumeClaim(\"test-ns\", \"test-pvc\").Result(),\n\t\t\tisGetExposeNil: true,\n\t\t\texpected: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseFailed).Finalizers([]string{DataUploadDownloadFinalizer}).Message(\"exposed snapshot is not ready\").Result(),\n\t\t\texpectedErr: \"no expose result is available for the current node\",\n\t\t},\n\t\t{\n\t\t\tname: \"Error in data path is concurrent limited\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhasePrepared).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\ttargetPVC: builder.ForPersistentVolumeClaim(\"test-ns\", \"test-pvc\").Result(),\n\t\t\tdataMgr: datapath.NewManager(0),\n\t\t\tnotNilExpose: true,\n\t\t\tnotMockCleanUp: true,\n\t\t\texpectedResult: &ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5},\n\t\t},\n\t\t{\n\t\t\tname: \"data path init error\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhasePrepared).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\ttargetPVC: builder.ForPersistentVolumeClaim(\"test-ns\", \"test-pvc\").Result(),\n\t\t\tmockInit: true,\n\t\t\tmockInitErr: errors.New(\"fake-data-path-init-error\"),\n\t\t\tmockClose: true,\n\t\t\tnotNilExpose: true,\n\t\t\texpected: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseFailed).Finalizers([]string{DataUploadDownloadFinalizer}).Message(\"error initializing data path\").Result(),\n\t\t\texpectedErr: \"error initializing asyncBR: fake-data-path-init-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"Unable to update status to in progress for data download\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhasePrepared).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\ttargetPVC: builder.ForPersistentVolumeClaim(\"test-ns\", \"test-pvc\").Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\tmockInit: true,\n\t\t\tmockClose: true,\n\t\t\tnotNilExpose: true,\n\t\t\tnotMockCleanUp: true,\n\t\t\texpected: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhasePrepared).Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"data path start error\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhasePrepared).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\ttargetPVC: builder.ForPersistentVolumeClaim(\"test-ns\", \"test-pvc\").Result(),\n\t\t\tmockInit: true,\n\t\t\tmockStart: true,\n\t\t\tmockStartErr: errors.New(\"fake-data-path-start-error\"),\n\t\t\tmockClose: true,\n\t\t\tnotNilExpose: true,\n\t\t\texpected: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseFailed).Finalizers([]string{DataUploadDownloadFinalizer}).Message(\"error starting data path\").Result(),\n\t\t\texpectedErr: \"error starting async restore for pod test-name, volume test-pvc: fake-data-path-start-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"Prepare succeeds\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhasePrepared).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\ttargetPVC: builder.ForPersistentVolumeClaim(\"test-ns\", \"test-pvc\").Result(),\n\t\t\tmockInit: true,\n\t\t\tmockStart: true,\n\t\t\tnotNilExpose: true,\n\t\t\tnotMockCleanUp: true,\n\t\t\texpectDataPath: true,\n\t\t\texpected: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseInProgress).Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"In progress dd is not handled by the current node\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseInProgress).Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\texpected: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseInProgress).Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"In progress dd is not set as cancel\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseInProgress).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\texpected: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseInProgress).Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Cancel data downloand in progress with empty FSBR\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseInProgress).Cancel(true).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\texpected: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseCanceled).Cancel(true).Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Cancel data downloand in progress and patch data download error\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseInProgress).Cancel(true).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\tneedCreateFSBR: true,\n\t\t\texpected: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseInProgress).Cancel(true).Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\texpectedErr: \"error updating datadownload velero/datadownload-1: Update error\",\n\t\t\texpectCancelRecord: true,\n\t\t\texpectDataPath: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Cancel data downloand in progress succeeds\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseInProgress).Cancel(true).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\tneedCreateFSBR: true,\n\t\t\tmockCancel: true,\n\t\t\texpected: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseCanceling).Cancel(true).Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\texpectDataPath: true,\n\t\t\texpectCancelRecord: true,\n\t\t},\n\t\t{\n\t\t\tname: \"pvc StorageClass is nil\",\n\t\t\tdd: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\ttargetPVC: builder.ForPersistentVolumeClaim(\"test-ns\", \"test-pvc\").Result(),\n\t\t\texpected: dataDownloadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Phase(velerov2alpha1api.DataDownloadPhaseAccepted).Result(),\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tobjects := []any{daemonSet, node, sc}\n\n\t\t\tif test.targetPVC != nil {\n\t\t\t\tobjects = append(objects, test.targetPVC)\n\t\t\t}\n\n\t\t\tr, err := initDataDownloadReconciler(t, objects, test.needErrs...)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tif !test.notCreateDD {\n\t\t\t\terr = r.client.Create(t.Context(), test.dd)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif test.needDelete {\n\t\t\t\terr = r.client.Delete(t.Context(), test.dd)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif test.dataMgr != nil {\n\t\t\t\tr.dataPathMgr = test.dataMgr\n\t\t\t} else {\n\t\t\t\tr.dataPathMgr = datapath.NewManager(1)\n\t\t\t}\n\n\t\t\tif test.sportTime != nil {\n\t\t\t\tr.cancelledDataDownload[test.dd.Name] = test.sportTime.Time\n\t\t\t}\n\n\t\t\tif test.constrained {\n\t\t\t\tr.vgdpCounter = &exposer.VgdpCounter{}\n\t\t\t}\n\n\t\t\tfuncExclusiveUpdateDataDownload = exclusiveUpdateDataDownload\n\t\t\tif test.needExclusiveUpdateError != nil {\n\t\t\t\tfuncExclusiveUpdateDataDownload = func(context.Context, kbclient.Client, *velerov2alpha1api.DataDownload, func(*velerov2alpha1api.DataDownload)) (bool, error) {\n\t\t\t\t\treturn false, test.needExclusiveUpdateError\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tdatapath.MicroServiceBRWatcherCreator = func(kbclient.Client, kubernetes.Interface, manager.Manager, string, string,\n\t\t\t\tstring, string, string, string, datapath.Callbacks, logrus.FieldLogger) datapath.AsyncBR {\n\t\t\t\tasyncBR := datapathmockes.NewAsyncBR(t)\n\t\t\t\tif test.mockInit {\n\t\t\t\t\tasyncBR.On(\"Init\", mock.Anything, mock.Anything).Return(test.mockInitErr)\n\t\t\t\t}\n\n\t\t\t\tif test.mockStart {\n\t\t\t\t\tasyncBR.On(\"StartRestore\", mock.Anything, mock.Anything, mock.Anything).Return(test.mockStartErr)\n\t\t\t\t}\n\n\t\t\t\tif test.mockCancel {\n\t\t\t\t\tasyncBR.On(\"Cancel\").Return()\n\t\t\t\t}\n\n\t\t\t\tif test.mockClose {\n\t\t\t\t\tasyncBR.On(\"Close\", mock.Anything).Return()\n\t\t\t\t}\n\n\t\t\t\treturn asyncBR\n\t\t\t}\n\n\t\t\tif test.isExposeErr || test.isGetExposeErr || test.isGetExposeNil || test.isPeekExposeErr || test.isNilExposer || test.notNilExpose {\n\t\t\t\tif test.isNilExposer {\n\t\t\t\t\tr.restoreExposer = nil\n\t\t\t\t} else {\n\t\t\t\t\tr.restoreExposer = func() exposer.GenericRestoreExposer {\n\t\t\t\t\t\tep := exposermockes.NewMockGenericRestoreExposer(t)\n\t\t\t\t\t\tif test.isExposeErr {\n\t\t\t\t\t\t\tep.On(\"Expose\", mock.Anything, mock.Anything, mock.Anything).Return(errors.New(\"Error to expose restore exposer\"))\n\t\t\t\t\t\t} else if test.notNilExpose {\n\t\t\t\t\t\t\thostingPod := builder.ForPod(\"test-ns\", \"test-name\").Volumes(&corev1api.Volume{Name: \"test-pvc\"}).Result()\n\t\t\t\t\t\t\thostingPod.ObjectMeta.SetUID(\"test-uid\")\n\t\t\t\t\t\t\tep.On(\"GetExposed\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(&exposer.ExposeResult{ByPod: exposer.ExposeByPod{HostingPod: hostingPod, VolumeName: \"test-pvc\"}}, nil)\n\t\t\t\t\t\t} else if test.isGetExposeErr {\n\t\t\t\t\t\t\tep.On(\"GetExposed\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(nil, errors.New(\"Error to get restore exposer\"))\n\t\t\t\t\t\t} else if test.isGetExposeNil {\n\t\t\t\t\t\t\tep.On(\"GetExposed\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(nil, nil)\n\t\t\t\t\t\t} else if test.isPeekExposeErr {\n\t\t\t\t\t\t\tep.On(\"PeekExposed\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(errors.New(\"fake-peek-error\"))\n\t\t\t\t\t\t\tep.On(\"DiagnoseExpose\", mock.Anything, mock.Anything).Return(\"\")\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\tif !test.notMockCleanUp {\n\t\t\t\t\t\t\tep.On(\"CleanUp\", mock.Anything, mock.Anything).Return()\n\t\t\t\t\t\t}\n\t\t\t\t\t\treturn ep\n\t\t\t\t\t}()\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif test.needCreateFSBR {\n\t\t\t\tif fsBR := r.dataPathMgr.GetAsyncBR(test.dd.Name); fsBR == nil {\n\t\t\t\t\t_, err := r.dataPathMgr.CreateMicroServiceBRWatcher(ctx, r.client, nil, nil, datapath.TaskTypeRestore, test.dd.Name, pVBRRequestor,\n\t\t\t\t\t\tvelerov1api.DefaultNamespace, \"\", \"\", datapath.Callbacks{OnCancelled: r.OnDataDownloadCancelled}, false, velerotest.NewLogger())\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tactualResult, err := r.Reconcile(ctx, ctrl.Request{\n\t\t\t\tNamespacedName: types.NamespacedName{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: test.dd.Name,\n\t\t\t\t},\n\t\t\t})\n\n\t\t\tif test.expectedErr != \"\" {\n\t\t\t\trequire.EqualError(t, err, test.expectedErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif test.expectedResult != nil {\n\t\t\t\tassert.Equal(t, test.expectedResult.Requeue, actualResult.Requeue)\n\t\t\t\tassert.Equal(t, test.expectedResult.RequeueAfter, actualResult.RequeueAfter)\n\t\t\t}\n\n\t\t\tdd := velerov2alpha1api.DataDownload{}\n\t\t\terr = r.client.Get(ctx, kbclient.ObjectKey{\n\t\t\t\tName: test.dd.Name,\n\t\t\t\tNamespace: test.dd.Namespace,\n\t\t\t}, &dd)\n\n\t\t\tif test.expected != nil || test.expectDeleted {\n\t\t\t\tif test.expectDeleted {\n\t\t\t\t\tassert.True(t, apierrors.IsNotFound(err))\n\t\t\t\t} else {\n\t\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\t\tassert.Equal(t, test.expected.Status.Phase, dd.Status.Phase)\n\t\t\t\t\tassert.Contains(t, dd.Status.Message, test.expected.Status.Message)\n\t\t\t\t\tassert.Equal(t, dd.Finalizers, test.expected.Finalizers)\n\t\t\t\t\tassert.Equal(t, dd.Spec.Cancel, test.expected.Spec.Cancel)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif !test.expectDataPath {\n\t\t\t\tassert.Nil(t, r.dataPathMgr.GetAsyncBR(test.dd.Name))\n\t\t\t} else {\n\t\t\t\tassert.NotNil(t, r.dataPathMgr.GetAsyncBR(test.dd.Name))\n\t\t\t}\n\n\t\t\tif test.expectCancelRecord {\n\t\t\t\tassert.Contains(t, r.cancelledDataDownload, test.dd.Name)\n\t\t\t} else {\n\t\t\t\tassert.Empty(t, r.cancelledDataDownload)\n\t\t\t}\n\n\t\t\tif isDataDownloadInFinalState(&dd) || dd.Status.Phase == velerov2alpha1api.DataDownloadPhaseInProgress {\n\t\t\t\tassert.NotContains(t, dd.Labels, exposer.ExposeOnGoingLabel)\n\t\t\t} else if dd.Status.Phase == velerov2alpha1api.DataDownloadPhaseAccepted {\n\t\t\t\tassert.Contains(t, dd.Labels, exposer.ExposeOnGoingLabel)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestOnDataDownloadFailed(t *testing.T) {\n\tfor _, getErr := range []bool{true, false} {\n\t\tctx := t.Context()\n\t\tneedErrs := []bool{getErr, false, false, false}\n\t\tr, err := initDataDownloadReconciler(t, nil, needErrs...)\n\t\trequire.NoError(t, err)\n\n\t\tdd := dataDownloadBuilder().Result()\n\t\tnamespace := dd.Namespace\n\t\tddName := dd.Name\n\t\t// Add the DataDownload object to the fake client\n\t\trequire.NoError(t, r.client.Create(ctx, dd))\n\t\tr.OnDataDownloadFailed(ctx, namespace, ddName, fmt.Errorf(\"Failed to handle %v\", ddName))\n\t\tupdatedDD := &velerov2alpha1api.DataDownload{}\n\t\tif getErr {\n\t\t\trequire.Error(t, r.client.Get(ctx, types.NamespacedName{Name: ddName, Namespace: namespace}, updatedDD))\n\t\t\tassert.NotEqual(t, velerov2alpha1api.DataDownloadPhaseFailed, updatedDD.Status.Phase)\n\t\t\tassert.True(t, updatedDD.Status.StartTimestamp.IsZero())\n\t\t} else {\n\t\t\trequire.NoError(t, r.client.Get(ctx, types.NamespacedName{Name: ddName, Namespace: namespace}, updatedDD))\n\t\t\tassert.Equal(t, velerov2alpha1api.DataDownloadPhaseFailed, updatedDD.Status.Phase)\n\t\t\tassert.True(t, updatedDD.Status.StartTimestamp.IsZero())\n\t\t}\n\t}\n}\n\nfunc TestOnDataDownloadCancelled(t *testing.T) {\n\tfor _, getErr := range []bool{true, false} {\n\t\tctx := t.Context()\n\t\tneedErrs := []bool{getErr, false, false, false}\n\t\tr, err := initDataDownloadReconciler(t, nil, needErrs...)\n\t\trequire.NoError(t, err)\n\n\t\tdd := dataDownloadBuilder().Result()\n\t\tnamespace := dd.Namespace\n\t\tddName := dd.Name\n\t\t// Add the DataDownload object to the fake client\n\t\trequire.NoError(t, r.client.Create(ctx, dd))\n\t\tr.OnDataDownloadCancelled(ctx, namespace, ddName)\n\t\tupdatedDD := &velerov2alpha1api.DataDownload{}\n\t\tif getErr {\n\t\t\trequire.Error(t, r.client.Get(ctx, types.NamespacedName{Name: ddName, Namespace: namespace}, updatedDD))\n\t\t\tassert.NotEqual(t, velerov2alpha1api.DataDownloadPhaseFailed, updatedDD.Status.Phase)\n\t\t\tassert.True(t, updatedDD.Status.StartTimestamp.IsZero())\n\t\t} else {\n\t\t\trequire.NoError(t, r.client.Get(ctx, types.NamespacedName{Name: ddName, Namespace: namespace}, updatedDD))\n\t\t\tassert.Equal(t, velerov2alpha1api.DataDownloadPhaseCanceled, updatedDD.Status.Phase)\n\t\t\tassert.False(t, updatedDD.Status.StartTimestamp.IsZero())\n\t\t\tassert.False(t, updatedDD.Status.CompletionTimestamp.IsZero())\n\t\t}\n\t}\n}\n\nfunc TestOnDataDownloadCompleted(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\temptyFSBR bool\n\t\tisGetErr bool\n\t\trebindVolumeErr bool\n\t}{\n\t\t{\n\t\t\tname: \"Data download complete\",\n\t\t\temptyFSBR: false,\n\t\t\tisGetErr: false,\n\t\t\trebindVolumeErr: false,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\t\t\tneedErrs := []bool{test.isGetErr, false, false, false}\n\t\t\tr, err := initDataDownloadReconciler(t, nil, needErrs...)\n\t\t\tr.restoreExposer = func() exposer.GenericRestoreExposer {\n\t\t\t\tep := exposermockes.NewMockGenericRestoreExposer(t)\n\t\t\t\tif test.rebindVolumeErr {\n\t\t\t\t\tep.On(\"RebindVolume\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(errors.New(\"Error to rebind volume\"))\n\t\t\t\t} else {\n\t\t\t\t\tep.On(\"RebindVolume\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(nil)\n\t\t\t\t}\n\t\t\t\tep.On(\"CleanUp\", mock.Anything, mock.Anything).Return()\n\t\t\t\treturn ep\n\t\t\t}()\n\n\t\t\trequire.NoError(t, err)\n\t\t\tdd := dataDownloadBuilder().Result()\n\t\t\tnamespace := dd.Namespace\n\t\t\tddName := dd.Name\n\t\t\t// Add the DataDownload object to the fake client\n\t\t\trequire.NoError(t, r.client.Create(ctx, dd))\n\t\t\tr.OnDataDownloadCompleted(ctx, namespace, ddName, datapath.Result{})\n\t\t\tupdatedDD := &velerov2alpha1api.DataDownload{}\n\t\t\tif test.isGetErr {\n\t\t\t\trequire.Error(t, r.client.Get(ctx, types.NamespacedName{Name: ddName, Namespace: namespace}, updatedDD))\n\t\t\t\tassert.Equal(t, velerov2alpha1api.DataDownloadPhase(\"\"), updatedDD.Status.Phase)\n\t\t\t\tassert.True(t, updatedDD.Status.CompletionTimestamp.IsZero())\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, r.client.Get(ctx, types.NamespacedName{Name: ddName, Namespace: namespace}, updatedDD))\n\t\t\t\tassert.Equal(t, velerov2alpha1api.DataDownloadPhaseCompleted, updatedDD.Status.Phase)\n\t\t\t\tassert.False(t, updatedDD.Status.CompletionTimestamp.IsZero())\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestOnDataDownloadProgress(t *testing.T) {\n\ttotalBytes := int64(1024)\n\tbytesDone := int64(512)\n\ttests := []struct {\n\t\tname string\n\t\tdd *velerov2alpha1api.DataDownload\n\t\tprogress uploader.Progress\n\t\tneedErrs []bool\n\t}{\n\t\t{\n\t\t\tname: \"patch in progress phase success\",\n\t\t\tdd: dataDownloadBuilder().Result(),\n\t\t\tprogress: uploader.Progress{\n\t\t\t\tTotalBytes: totalBytes,\n\t\t\t\tBytesDone: bytesDone,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"failed to get datadownload\",\n\t\t\tdd: dataDownloadBuilder().Result(),\n\t\t\tneedErrs: []bool{true, false, false, false},\n\t\t},\n\t\t{\n\t\t\tname: \"failed to patch datadownload\",\n\t\t\tdd: dataDownloadBuilder().Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\n\t\t\tr, err := initDataDownloadReconciler(t, nil, test.needErrs...)\n\t\t\trequire.NoError(t, err)\n\t\t\tdefer func() {\n\t\t\t\tr.client.Delete(ctx, test.dd, &kbclient.DeleteOptions{})\n\t\t\t}()\n\t\t\t// Create a DataDownload object\n\t\t\tdd := dataDownloadBuilder().Result()\n\t\t\tnamespace := dd.Namespace\n\t\t\tduName := dd.Name\n\t\t\t// Add the DataDownload object to the fake client\n\t\t\trequire.NoError(t, r.client.Create(t.Context(), dd))\n\n\t\t\t// Create a Progress object\n\t\t\tprogress := &uploader.Progress{\n\t\t\t\tTotalBytes: totalBytes,\n\t\t\t\tBytesDone: bytesDone,\n\t\t\t}\n\n\t\t\t// Call the OnDataDownloadProgress function\n\t\t\tr.OnDataDownloadProgress(ctx, namespace, duName, progress)\n\t\t\tif len(test.needErrs) != 0 && !test.needErrs[0] {\n\t\t\t\t// Get the updated DataDownload object from the fake client\n\t\t\t\tupdatedDu := &velerov2alpha1api.DataDownload{}\n\t\t\t\trequire.NoError(t, r.client.Get(ctx, types.NamespacedName{Name: duName, Namespace: namespace}, updatedDu))\n\t\t\t\t// Assert that the DataDownload object has been updated with the progress\n\t\t\t\tassert.Equal(t, test.progress.TotalBytes, updatedDu.Status.Progress.TotalBytes)\n\t\t\t\tassert.Equal(t, test.progress.BytesDone, updatedDu.Status.Progress.BytesDone)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestFindDataDownloadForPod(t *testing.T) {\n\tneedErrs := []bool{false, false, false, false}\n\tr, err := initDataDownloadReconciler(t, nil, needErrs...)\n\trequire.NoError(t, err)\n\ttests := []struct {\n\t\tname string\n\t\tdu *velerov2alpha1api.DataDownload\n\t\tpod *corev1api.Pod\n\t\tcheckFunc func(*velerov2alpha1api.DataDownload, []reconcile.Request)\n\t}{\n\t\t{\n\t\t\tname: \"find dataDownload for pod\",\n\t\t\tdu: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseAccepted).Result(),\n\t\t\tpod: builder.ForPod(velerov1api.DefaultNamespace, dataDownloadName).Labels(map[string]string{velerov1api.DataDownloadLabel: dataDownloadName}).Status(corev1api.PodStatus{Phase: corev1api.PodRunning}).Result(),\n\t\t\tcheckFunc: func(du *velerov2alpha1api.DataDownload, requests []reconcile.Request) {\n\t\t\t\t// Assert that the function returns a single request\n\t\t\t\tassert.Len(t, requests, 1)\n\t\t\t\t// Assert that the request contains the correct namespaced name\n\t\t\t\tassert.Equal(t, du.Namespace, requests[0].Namespace)\n\t\t\t\tassert.Equal(t, du.Name, requests[0].Name)\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"no selected label found for pod\",\n\t\t\tdu: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseAccepted).Result(),\n\t\t\tpod: builder.ForPod(velerov1api.DefaultNamespace, dataDownloadName).Result(),\n\t\t\tcheckFunc: func(du *velerov2alpha1api.DataDownload, requests []reconcile.Request) {\n\t\t\t\t// Assert that the function returns a single request\n\t\t\t\tassert.Empty(t, requests)\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"no matched pod\",\n\t\t\tdu: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseAccepted).Result(),\n\t\t\tpod: builder.ForPod(velerov1api.DefaultNamespace, dataDownloadName).Labels(map[string]string{velerov1api.DataDownloadLabel: \"non-existing-datadownload\"}).Result(),\n\t\t\tcheckFunc: func(du *velerov2alpha1api.DataDownload, requests []reconcile.Request) {\n\t\t\t\tassert.Empty(t, requests)\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"dataDownload not accept\",\n\t\t\tdu: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseInProgress).Result(),\n\t\t\tpod: builder.ForPod(velerov1api.DefaultNamespace, dataDownloadName).Labels(map[string]string{velerov1api.DataDownloadLabel: dataDownloadName}).Result(),\n\t\t\tcheckFunc: func(du *velerov2alpha1api.DataDownload, requests []reconcile.Request) {\n\t\t\t\tassert.Empty(t, requests)\n\t\t\t},\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tctx := t.Context()\n\t\tassert.NoError(t, r.client.Create(ctx, test.pod))\n\t\tassert.NoError(t, r.client.Create(ctx, test.du))\n\t\t// Call the findSnapshotRestoreForPod function\n\t\trequests := r.findSnapshotRestoreForPod(t.Context(), test.pod)\n\t\ttest.checkFunc(test.du, requests)\n\t\tr.client.Delete(ctx, test.du, &kbclient.DeleteOptions{})\n\t\tif test.pod != nil {\n\t\t\tr.client.Delete(ctx, test.pod, &kbclient.DeleteOptions{})\n\t\t}\n\t}\n}\n\nfunc TestAcceptDataDownload(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdd *velerov2alpha1api.DataDownload\n\t\tneedErrs []error\n\t\tsucceeded bool\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"update fail\",\n\t\t\tdd: dataDownloadBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, fmt.Errorf(\"fake-update-error\"), nil},\n\t\t\texpectedErr: \"fake-update-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"accepted by others\",\n\t\t\tdd: dataDownloadBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, &fakeAPIStatus{metav1.StatusReasonConflict}, nil},\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tdd: dataDownloadBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, nil, nil},\n\t\t\tsucceeded: true,\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tctx := t.Context()\n\t\tr, err := initDataDownloadReconcilerWithError(t, nil, test.needErrs...)\n\t\trequire.NoError(t, err)\n\n\t\terr = r.client.Create(ctx, test.dd)\n\t\trequire.NoError(t, err)\n\n\t\tsucceeded, err := r.acceptDataDownload(ctx, test.dd)\n\t\tassert.Equal(t, test.succeeded, succeeded)\n\t\tif test.expectedErr == \"\" {\n\t\t\tassert.NoError(t, err)\n\t\t} else {\n\t\t\tassert.EqualError(t, err, test.expectedErr)\n\t\t}\n\t}\n}\n\nfunc TestOnDdPrepareTimeout(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdd *velerov2alpha1api.DataDownload\n\t\tneedErrs []error\n\t\texpected *velerov2alpha1api.DataDownload\n\t}{\n\t\t{\n\t\t\tname: \"update fail\",\n\t\t\tdd: dataDownloadBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, fmt.Errorf(\"fake-update-error\"), nil},\n\t\t\texpected: dataDownloadBuilder().Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"update interrupted\",\n\t\t\tdd: dataDownloadBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, &fakeAPIStatus{metav1.StatusReasonConflict}, nil},\n\t\t\texpected: dataDownloadBuilder().Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tdd: dataDownloadBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, nil, nil},\n\t\t\texpected: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseFailed).Result(),\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tctx := t.Context()\n\t\tr, err := initDataDownloadReconcilerWithError(t, nil, test.needErrs...)\n\t\trequire.NoError(t, err)\n\n\t\terr = r.client.Create(ctx, test.dd)\n\t\trequire.NoError(t, err)\n\n\t\tr.onPrepareTimeout(ctx, test.dd)\n\n\t\tdd := velerov2alpha1api.DataDownload{}\n\t\t_ = r.client.Get(ctx, kbclient.ObjectKey{\n\t\t\tName: test.dd.Name,\n\t\t\tNamespace: test.dd.Namespace,\n\t\t}, &dd)\n\n\t\tassert.Equal(t, test.expected.Status.Phase, dd.Status.Phase)\n\t}\n}\n\nfunc TestTryCancelDataDownload(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdd *velerov2alpha1api.DataDownload\n\t\tneedErrs []error\n\t\tsucceeded bool\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"update fail\",\n\t\t\tdd: dataDownloadBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, fmt.Errorf(\"fake-update-error\"), nil},\n\t\t},\n\t\t{\n\t\t\tname: \"cancel by others\",\n\t\t\tdd: dataDownloadBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, &fakeAPIStatus{metav1.StatusReasonConflict}, nil},\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tdd: dataDownloadBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, nil, nil},\n\t\t\tsucceeded: true,\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tctx := t.Context()\n\t\tr, err := initDataDownloadReconcilerWithError(t, nil, test.needErrs...)\n\t\trequire.NoError(t, err)\n\n\t\terr = r.client.Create(ctx, test.dd)\n\t\trequire.NoError(t, err)\n\n\t\tr.tryCancelDataDownload(ctx, test.dd, \"\")\n\n\t\tif test.expectedErr == \"\" {\n\t\t\tassert.NoError(t, err)\n\t\t} else {\n\t\t\tassert.EqualError(t, err, test.expectedErr)\n\t\t}\n\t}\n}\n\nfunc TestUpdateDataDownloadWithRetry(t *testing.T) {\n\tnamespacedName := types.NamespacedName{\n\t\tName: dataDownloadName,\n\t\tNamespace: \"velero\",\n\t}\n\n\t// Define test cases\n\ttestCases := []struct {\n\t\tName string\n\t\tneedErrs []bool\n\t\tnoChange bool\n\t\tExpectErr bool\n\t}{\n\t\t{\n\t\t\tName: \"SuccessOnFirstAttempt\",\n\t\t},\n\t\t{\n\t\t\tName: \"Error get\",\n\t\t\tneedErrs: []bool{true, false, false, false, false},\n\t\t\tExpectErr: true,\n\t\t},\n\t\t{\n\t\t\tName: \"Error update\",\n\t\t\tneedErrs: []bool{false, false, true, false, false},\n\t\t\tExpectErr: true,\n\t\t},\n\t\t{\n\t\t\tName: \"no change\",\n\t\t\tnoChange: true,\n\t\t\tneedErrs: []bool{false, false, true, false, false},\n\t\t},\n\t\t{\n\t\t\tName: \"Conflict with error timeout\",\n\t\t\tneedErrs: []bool{false, false, false, false, true},\n\t\t\tExpectErr: true,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.Name, func(t *testing.T) {\n\t\t\tctx, cancelFunc := context.WithTimeout(t.Context(), time.Second*5)\n\t\t\tdefer cancelFunc()\n\t\t\tr, err := initDataDownloadReconciler(t, nil, tc.needErrs...)\n\t\t\trequire.NoError(t, err)\n\t\t\terr = r.client.Create(ctx, dataDownloadBuilder().Result())\n\t\t\trequire.NoError(t, err)\n\t\t\tupdateFunc := func(dataDownload *velerov2alpha1api.DataDownload) bool {\n\t\t\t\tif tc.noChange {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tdataDownload.Spec.Cancel = true\n\n\t\t\t\treturn true\n\t\t\t}\n\t\t\terr = UpdateDataDownloadWithRetry(ctx, r.client, namespacedName, velerotest.NewLogger().WithField(\"name\", tc.Name), updateFunc)\n\t\t\tif tc.ExpectErr {\n\t\t\t\tassert.Error(t, err)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\ntype ddResumeTestHelper struct {\n\tresumeErr error\n\tgetExposeErr error\n\texposeResult *exposer.ExposeResult\n\tasyncBR datapath.AsyncBR\n}\n\nfunc (dt *ddResumeTestHelper) resumeCancellableDataPath(_ *DataUploadReconciler, _ context.Context, _ *velerov2alpha1api.DataUpload, _ logrus.FieldLogger) error {\n\treturn dt.resumeErr\n}\n\nfunc (dt *ddResumeTestHelper) Expose(context.Context, corev1api.ObjectReference, exposer.GenericRestoreExposeParam) error {\n\treturn nil\n}\n\nfunc (dt *ddResumeTestHelper) GetExposed(context.Context, corev1api.ObjectReference, kbclient.Client, string, time.Duration) (*exposer.ExposeResult, error) {\n\treturn dt.exposeResult, dt.getExposeErr\n}\n\nfunc (dt *ddResumeTestHelper) PeekExposed(context.Context, corev1api.ObjectReference) error {\n\treturn nil\n}\n\nfunc (dt *ddResumeTestHelper) DiagnoseExpose(context.Context, corev1api.ObjectReference) string {\n\treturn \"\"\n}\n\nfunc (dt *ddResumeTestHelper) RebindVolume(context.Context, corev1api.ObjectReference, string, string, time.Duration) error {\n\treturn nil\n}\n\nfunc (dt *ddResumeTestHelper) CleanUp(context.Context, corev1api.ObjectReference) {}\n\nfunc (dt *ddResumeTestHelper) newMicroServiceBRWatcher(kbclient.Client, kubernetes.Interface, manager.Manager, string, string, string, string, string, string,\n\tdatapath.Callbacks, logrus.FieldLogger) datapath.AsyncBR {\n\treturn dt.asyncBR\n}\n\nfunc TestAttemptDataDownloadResume(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdataUploads []velerov2alpha1api.DataDownload\n\t\tdd *velerov2alpha1api.DataDownload\n\t\tneedErrs []bool\n\t\tresumeErr error\n\t\tacceptedDataDownloads []string\n\t\tprepareddDataDownloads []string\n\t\tcancelledDataDownloads []string\n\t\tinProgressDataDownloads []string\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"Other DataDownload\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhasePrepared).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Other DataDownload\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseAccepted).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"InProgress DataDownload, not the current node\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseInProgress).Result(),\n\t\t\tinProgressDataDownloads: []string{dataDownloadName},\n\t\t},\n\t\t{\n\t\t\tname: \"InProgress DataDownload, no resume error\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseInProgress).Node(\"node-1\").Result(),\n\t\t\tinProgressDataDownloads: []string{dataDownloadName},\n\t\t},\n\t\t{\n\t\t\tname: \"InProgress DataDownload, resume error, cancel error\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseInProgress).Node(\"node-1\").Result(),\n\t\t\tresumeErr: errors.New(\"fake-resume-error\"),\n\t\t\tneedErrs: []bool{false, false, true, false, false, false},\n\t\t\tinProgressDataDownloads: []string{dataDownloadName},\n\t\t},\n\t\t{\n\t\t\tname: \"InProgress DataDownload, resume error, cancel succeed\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseInProgress).Node(\"node-1\").Result(),\n\t\t\tresumeErr: errors.New(\"fake-resume-error\"),\n\t\t\tcancelledDataDownloads: []string{dataDownloadName},\n\t\t\tinProgressDataDownloads: []string{dataDownloadName},\n\t\t},\n\t\t{\n\t\t\tname: \"Error\",\n\t\t\tneedErrs: []bool{false, false, false, false, false, true},\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhasePrepared).Result(),\n\t\t\texpectedError: \"error to list datadownloads: List error\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\t\t\tr, err := initDataDownloadReconciler(t, nil, test.needErrs...)\n\t\t\tr.nodeName = \"node-1\"\n\t\t\trequire.NoError(t, err)\n\t\t\tdefer func() {\n\t\t\t\tr.client.Delete(ctx, test.dd, &kbclient.DeleteOptions{})\n\t\t\t}()\n\n\t\t\trequire.NoError(t, r.client.Create(ctx, test.dd))\n\n\t\t\tdt := &duResumeTestHelper{\n\t\t\t\tresumeErr: test.resumeErr,\n\t\t\t}\n\n\t\t\tfuncResumeCancellableDataBackup = dt.resumeCancellableDataPath\n\n\t\t\t// Run the test\n\t\t\terr = r.AttemptDataDownloadResume(ctx, r.logger.WithField(\"name\", test.name), test.dd.Namespace)\n\n\t\t\tif test.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.expectedError)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\n\t\t\t\t// Verify DataDownload marked as Canceled\n\t\t\t\tfor _, duName := range test.cancelledDataDownloads {\n\t\t\t\t\tdataDownload := &velerov2alpha1api.DataDownload{}\n\t\t\t\t\terr := r.client.Get(t.Context(), types.NamespacedName{Namespace: \"velero\", Name: duName}, dataDownload)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tassert.True(t, dataDownload.Spec.Cancel)\n\t\t\t\t}\n\t\t\t\t// Verify DataDownload marked as Accepted\n\t\t\t\tfor _, duName := range test.acceptedDataDownloads {\n\t\t\t\t\tdataUpload := &velerov2alpha1api.DataDownload{}\n\t\t\t\t\terr := r.client.Get(t.Context(), types.NamespacedName{Namespace: \"velero\", Name: duName}, dataUpload)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tassert.Equal(t, velerov2alpha1api.DataDownloadPhaseAccepted, dataUpload.Status.Phase)\n\t\t\t\t}\n\t\t\t\t// Verify DataDownload marked as Prepared\n\t\t\t\tfor _, duName := range test.prepareddDataDownloads {\n\t\t\t\t\tdataUpload := &velerov2alpha1api.DataDownload{}\n\t\t\t\t\terr := r.client.Get(t.Context(), types.NamespacedName{Namespace: \"velero\", Name: duName}, dataUpload)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tassert.Equal(t, velerov2alpha1api.DataDownloadPhasePrepared, dataUpload.Status.Phase)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestResumeCancellableRestore(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdataDownloads []velerov2alpha1api.DataDownload\n\t\tdd *velerov2alpha1api.DataDownload\n\t\tgetExposeErr error\n\t\texposeResult *exposer.ExposeResult\n\t\tcreateWatcherErr error\n\t\tinitWatcherErr error\n\t\tstartWatcherErr error\n\t\tmockInit bool\n\t\tmockStart bool\n\t\tmockClose bool\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"get expose failed\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseInProgress).Result(),\n\t\t\tgetExposeErr: errors.New(\"fake-expose-error\"),\n\t\t\texpectedError: fmt.Sprintf(\"error to get exposed volume for dd %s: fake-expose-error\", dataDownloadName),\n\t\t},\n\t\t{\n\t\t\tname: \"no expose\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseAccepted).Node(\"node-1\").Result(),\n\t\t\texpectedError: fmt.Sprintf(\"expose info missed for dd %s\", dataDownloadName),\n\t\t},\n\t\t{\n\t\t\tname: \"watcher init error\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseAccepted).Node(\"node-1\").Result(),\n\t\t\texposeResult: &exposer.ExposeResult{\n\t\t\t\tByPod: exposer.ExposeByPod{\n\t\t\t\t\tHostingPod: &corev1api.Pod{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tmockInit: true,\n\t\t\tmockClose: true,\n\t\t\tinitWatcherErr: errors.New(\"fake-init-watcher-error\"),\n\t\t\texpectedError: fmt.Sprintf(\"error to init asyncBR watcher for dd %s: fake-init-watcher-error\", dataDownloadName),\n\t\t},\n\t\t{\n\t\t\tname: \"start watcher error\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseAccepted).Node(\"node-1\").Result(),\n\t\t\texposeResult: &exposer.ExposeResult{\n\t\t\t\tByPod: exposer.ExposeByPod{\n\t\t\t\t\tHostingPod: &corev1api.Pod{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tmockInit: true,\n\t\t\tmockStart: true,\n\t\t\tmockClose: true,\n\t\t\tstartWatcherErr: errors.New(\"fake-start-watcher-error\"),\n\t\t\texpectedError: fmt.Sprintf(\"error to resume asyncBR watcher for dd %s: fake-start-watcher-error\", dataDownloadName),\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tdd: dataDownloadBuilder().Phase(velerov2alpha1api.DataDownloadPhaseAccepted).Node(\"node-1\").Result(),\n\t\t\texposeResult: &exposer.ExposeResult{\n\t\t\t\tByPod: exposer.ExposeByPod{\n\t\t\t\t\tHostingPod: &corev1api.Pod{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tmockInit: true,\n\t\t\tmockStart: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\t\t\tr, err := initDataDownloadReconciler(t, nil, false)\n\t\t\tr.nodeName = \"node-1\"\n\t\t\trequire.NoError(t, err)\n\n\t\t\tmockAsyncBR := datapathmockes.NewAsyncBR(t)\n\n\t\t\tif test.mockInit {\n\t\t\t\tmockAsyncBR.On(\"Init\", mock.Anything, mock.Anything).Return(test.initWatcherErr)\n\t\t\t}\n\n\t\t\tif test.mockStart {\n\t\t\t\tmockAsyncBR.On(\"StartRestore\", mock.Anything, mock.Anything, mock.Anything).Return(test.startWatcherErr)\n\t\t\t}\n\n\t\t\tif test.mockClose {\n\t\t\t\tmockAsyncBR.On(\"Close\", mock.Anything).Return()\n\t\t\t}\n\n\t\t\tdt := &ddResumeTestHelper{\n\t\t\t\tgetExposeErr: test.getExposeErr,\n\t\t\t\texposeResult: test.exposeResult,\n\t\t\t\tasyncBR: mockAsyncBR,\n\t\t\t}\n\n\t\t\tr.restoreExposer = dt\n\n\t\t\tdatapath.MicroServiceBRWatcherCreator = dt.newMicroServiceBRWatcher\n\n\t\t\terr = r.resumeCancellableDataPath(ctx, test.dd, velerotest.NewLogger())\n\t\t\tif test.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.expectedError)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestDataDownloadSetupExposeParam(t *testing.T) {\n\t// Common objects for all cases\n\tnode := builder.ForNode(\"worker-1\").Labels(map[string]string{kube.NodeOSLabel: kube.NodeOSLinux}).Result()\n\n\tbaseDataDownload := dataDownloadBuilder().Result()\n\tbaseDataDownload.Namespace = velerov1api.DefaultNamespace\n\tbaseDataDownload.Spec.OperationTimeout = metav1.Duration{Duration: time.Minute * 10}\n\tbaseDataDownload.Spec.SnapshotSize = 5368709120 // 5Gi\n\n\ttype args struct {\n\t\tcustomLabels map[string]string\n\t\tcustomAnnotations map[string]string\n\t}\n\ttype want struct {\n\t\tlabels map[string]string\n\t\tannotations map[string]string\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\targs args\n\t\twant want\n\t}{\n\t\t{\n\t\t\tname: \"label has customize values\",\n\t\t\targs: args{\n\t\t\t\tcustomLabels: map[string]string{\"custom-label\": \"label-value\"},\n\t\t\t\tcustomAnnotations: nil,\n\t\t\t},\n\t\t\twant: want{\n\t\t\t\tlabels: map[string]string{\n\t\t\t\t\tvelerov1api.DataDownloadLabel: baseDataDownload.Name,\n\t\t\t\t\t\"custom-label\": \"label-value\",\n\t\t\t\t},\n\t\t\t\tannotations: map[string]string{},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"label has no customize values\",\n\t\t\targs: args{\n\t\t\t\tcustomLabels: nil,\n\t\t\t\tcustomAnnotations: nil,\n\t\t\t},\n\t\t\twant: want{\n\t\t\t\tlabels: map[string]string{velerov1api.DataDownloadLabel: baseDataDownload.Name},\n\t\t\t\tannotations: map[string]string{},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"annotation has customize values\",\n\t\t\targs: args{\n\t\t\t\tcustomLabels: nil,\n\t\t\t\tcustomAnnotations: map[string]string{\"custom-annotation\": \"annotation-value\"},\n\t\t\t},\n\t\t\twant: want{\n\t\t\t\tlabels: map[string]string{velerov1api.DataDownloadLabel: baseDataDownload.Name},\n\t\t\t\tannotations: map[string]string{\"custom-annotation\": \"annotation-value\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"both label and annotation have customize values\",\n\t\t\targs: args{\n\t\t\t\tcustomLabels: map[string]string{\"custom-label\": \"label-value\"},\n\t\t\t\tcustomAnnotations: map[string]string{\"custom-annotation\": \"annotation-value\"},\n\t\t\t},\n\t\t\twant: want{\n\t\t\t\tlabels: map[string]string{\n\t\t\t\t\tvelerov1api.DataDownloadLabel: baseDataDownload.Name,\n\t\t\t\t\t\"custom-label\": \"label-value\",\n\t\t\t\t},\n\t\t\t\tannotations: map[string]string{\"custom-annotation\": \"annotation-value\"},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\t// Fake clients per case\n\t\t\tfakeClient := FakeClient{\n\t\t\t\tClient: velerotest.NewFakeControllerRuntimeClient(t, node, baseDataDownload.DeepCopy()),\n\t\t\t}\n\t\t\tfakeKubeClient := clientgofake.NewSimpleClientset(node)\n\n\t\t\t// Reconciler config per case\n\t\t\tpreparingTimeout := time.Minute * 3\n\t\t\tpodRes := corev1api.ResourceRequirements{}\n\t\t\tr := NewDataDownloadReconciler(\n\t\t\t\t&fakeClient,\n\t\t\t\tnil,\n\t\t\t\tfakeKubeClient,\n\t\t\t\tdatapath.NewManager(1),\n\t\t\t\tnil,\n\t\t\t\tnil,\n\t\t\t\tvelerotypes.RestorePVC{},\n\t\t\t\tnil,\n\t\t\t\tnil,\n\t\t\t\tpodRes,\n\t\t\t\t\"test-node\",\n\t\t\t\tpreparingTimeout,\n\t\t\t\tvelerotest.NewLogger(),\n\t\t\t\tmetrics.NewServerMetrics(),\n\t\t\t\t\"download-priority\",\n\t\t\t\tnil, // repoConfigMgr (unused when cacheVolumeConfigs is nil)\n\t\t\t\ttt.args.customLabels,\n\t\t\t\ttt.args.customAnnotations,\n\t\t\t)\n\n\t\t\t// Act\n\t\t\tgot, err := r.setupExposeParam(baseDataDownload)\n\n\t\t\t// Assert no error\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Core fields\n\t\t\tassert.Equal(t, baseDataDownload.Spec.TargetVolume.PVC, got.TargetPVCName)\n\t\t\tassert.Equal(t, baseDataDownload.Spec.TargetVolume.Namespace, got.TargetNamespace)\n\n\t\t\t// Labels and Annotations\n\t\t\tassert.Equal(t, tt.want.labels, got.HostingPodLabels)\n\t\t\tassert.Equal(t, tt.want.annotations, got.HostingPodAnnotations)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/controller/data_upload_controller.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strings\"\n\t\"time\"\n\n\tsnapshotter \"github.com/kubernetes-csi/external-snapshotter/client/v8/clientset/versioned/typed/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"k8s.io/client-go/kubernetes\"\n\tclocks \"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/builder\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil\"\n\t\"sigs.k8s.io/controller-runtime/pkg/event\"\n\t\"sigs.k8s.io/controller-runtime/pkg/manager\"\n\t\"sigs.k8s.io/controller-runtime/pkg/predicate\"\n\t\"sigs.k8s.io/controller-runtime/pkg/reconcile\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/apis/velero/shared\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/datamover\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\t\"github.com/vmware-tanzu/velero/pkg/exposer\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\t\"github.com/vmware-tanzu/velero/pkg/nodeagent\"\n\tvelerotypes \"github.com/vmware-tanzu/velero/pkg/types\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nconst (\n\tdataUploadDownloadRequestor = \"snapshot-data-upload-download\"\n\tDataUploadDownloadFinalizer = \"velero.io/data-upload-download-finalizer\"\n\tpreparingMonitorFrequency = time.Minute\n\tcancelDelayInProgress = time.Hour\n\tcancelDelayOthers = time.Minute * 5\n)\n\n// DataUploadReconciler reconciles a DataUpload object\ntype DataUploadReconciler struct {\n\tclient client.Client\n\tkubeClient kubernetes.Interface\n\tcsiSnapshotClient snapshotter.SnapshotV1Interface\n\tmgr manager.Manager\n\tClock clocks.WithTickerAndDelayedExecution\n\tnodeName string\n\tlogger logrus.FieldLogger\n\tsnapshotExposerList map[velerov2alpha1api.SnapshotType]exposer.SnapshotExposer\n\tdataPathMgr *datapath.Manager\n\tvgdpCounter *exposer.VgdpCounter\n\tloadAffinity []*kube.LoadAffinity\n\tbackupPVCConfig map[string]velerotypes.BackupPVC\n\tpodResources corev1api.ResourceRequirements\n\tpreparingTimeout time.Duration\n\tmetrics *metrics.ServerMetrics\n\tcancelledDataUpload map[string]time.Time\n\tdataMovePriorityClass string\n\tpodLabels map[string]string\n\tpodAnnotations map[string]string\n}\n\nfunc NewDataUploadReconciler(\n\tclient client.Client,\n\tmgr manager.Manager,\n\tkubeClient kubernetes.Interface,\n\tcsiSnapshotClient snapshotter.SnapshotV1Interface,\n\tdataPathMgr *datapath.Manager,\n\tcounter *exposer.VgdpCounter,\n\tloadAffinity []*kube.LoadAffinity,\n\tbackupPVCConfig map[string]velerotypes.BackupPVC,\n\tpodResources corev1api.ResourceRequirements,\n\tclock clocks.WithTickerAndDelayedExecution,\n\tnodeName string,\n\tpreparingTimeout time.Duration,\n\tlog logrus.FieldLogger,\n\tmetrics *metrics.ServerMetrics,\n\tdataMovePriorityClass string,\n\tpodLabels map[string]string,\n\tpodAnnotations map[string]string,\n) *DataUploadReconciler {\n\treturn &DataUploadReconciler{\n\t\tclient: client,\n\t\tmgr: mgr,\n\t\tkubeClient: kubeClient,\n\t\tcsiSnapshotClient: csiSnapshotClient,\n\t\tClock: clock,\n\t\tnodeName: nodeName,\n\t\tlogger: log,\n\t\tsnapshotExposerList: map[velerov2alpha1api.SnapshotType]exposer.SnapshotExposer{\n\t\t\tvelerov2alpha1api.SnapshotTypeCSI: exposer.NewCSISnapshotExposer(\n\t\t\t\tkubeClient,\n\t\t\t\tcsiSnapshotClient,\n\t\t\t\tlog,\n\t\t\t),\n\t\t},\n\t\tdataPathMgr: dataPathMgr,\n\t\tvgdpCounter: counter,\n\t\tloadAffinity: loadAffinity,\n\t\tbackupPVCConfig: backupPVCConfig,\n\t\tpodResources: podResources,\n\t\tpreparingTimeout: preparingTimeout,\n\t\tmetrics: metrics,\n\t\tcancelledDataUpload: make(map[string]time.Time),\n\t\tdataMovePriorityClass: dataMovePriorityClass,\n\t\tpodLabels: podLabels,\n\t\tpodAnnotations: podAnnotations,\n\t}\n}\n\n// +kubebuilder:rbac:groups=velero.io,resources=datauploads,verbs=get;list;watch;create;update;patch;delete\n// +kubebuilder:rbac:groups=velero.io,resources=datauploads/status,verbs=get;update;patch\n// +kubebuilder:rbac:groups=\"\",resources=pods,verbs=get\n// +kubebuilder:rbac:groups=\"\",resources=persistentvolumes,verbs=get\n// +kubebuilder:rbac:groups=\"\",resources=persistentvolumerclaims,verbs=get\n\nfunc (r *DataUploadReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\tlog := r.logger.WithFields(logrus.Fields{\n\t\t\"controller\": \"dataupload\",\n\t\t\"dataupload\": req.NamespacedName,\n\t})\n\tlog.Infof(\"Reconcile %s\", req.Name)\n\tdu := &velerov2alpha1api.DataUpload{}\n\tif err := r.client.Get(ctx, req.NamespacedName, du); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.Debug(\"Unable to find DataUpload\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"getting DataUpload\")\n\t}\n\n\tif !datamover.IsBuiltInUploader(du.Spec.DataMover) {\n\t\tlog.WithField(\"Data mover\", du.Spec.DataMover).Debug(\"it is not one built-in data mover which is not supported by Velero\")\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\t// Logic for clear resources when dataupload been deleted\n\tif !isDataUploadInFinalState(du) {\n\t\tif !controllerutil.ContainsFinalizer(du, DataUploadDownloadFinalizer) {\n\t\t\tif err := UpdateDataUploadWithRetry(ctx, r.client, req.NamespacedName, log, func(dataUpload *velerov2alpha1api.DataUpload) bool {\n\t\t\t\tif controllerutil.ContainsFinalizer(dataUpload, DataUploadDownloadFinalizer) {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tcontrollerutil.AddFinalizer(dataUpload, DataUploadDownloadFinalizer)\n\n\t\t\t\treturn true\n\t\t\t}); err != nil {\n\t\t\t\tlog.WithError(err).Errorf(\"failed to add finalizer for du %s/%s\", du.Namespace, du.Name)\n\t\t\t\treturn ctrl.Result{}, err\n\t\t\t}\n\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tif !du.DeletionTimestamp.IsZero() {\n\t\t\tif !du.Spec.Cancel {\n\t\t\t\t// when delete cr we need to clear up internal resources created by Velero, here we use the cancel mechanism\n\t\t\t\t// to help clear up resources instead of clear them directly in case of some conflict with Expose action\n\t\t\t\tlog.Warnf(\"Cancel du under phase %s because it is being deleted\", du.Status.Phase)\n\n\t\t\t\tif err := UpdateDataUploadWithRetry(ctx, r.client, req.NamespacedName, log, func(dataUpload *velerov2alpha1api.DataUpload) bool {\n\t\t\t\t\tif dataUpload.Spec.Cancel {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\n\t\t\t\t\tdataUpload.Spec.Cancel = true\n\t\t\t\t\tdataUpload.Status.Message = \"Cancel dataupload because it is being deleted\"\n\n\t\t\t\t\treturn true\n\t\t\t\t}); err != nil {\n\t\t\t\t\tlog.WithError(err).Errorf(\"failed to set cancel flag for du %s/%s\", du.Namespace, du.Name)\n\t\t\t\t\treturn ctrl.Result{}, err\n\t\t\t\t}\n\n\t\t\t\treturn ctrl.Result{}, nil\n\t\t\t}\n\t\t}\n\t} else {\n\t\tdelete(r.cancelledDataUpload, du.Name)\n\n\t\t// put the finalizer remove action here for all cr will goes to the final status, we could check finalizer and do remove action in final status\n\t\t// instead of intermediate state.\n\t\t// remove finalizer no matter whether the cr is being deleted or not for it is no longer needed when internal resources are all cleaned up\n\t\t// also in final status cr won't block the direct delete of the velero namespace\n\t\tif controllerutil.ContainsFinalizer(du, DataUploadDownloadFinalizer) {\n\t\t\tif err := UpdateDataUploadWithRetry(ctx, r.client, req.NamespacedName, log, func(du *velerov2alpha1api.DataUpload) bool {\n\t\t\t\tif !controllerutil.ContainsFinalizer(du, DataUploadDownloadFinalizer) {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tcontrollerutil.RemoveFinalizer(du, DataUploadDownloadFinalizer)\n\n\t\t\t\treturn true\n\t\t\t}); err != nil {\n\t\t\t\tlog.WithError(err).Error(\"error to remove finalizer\")\n\t\t\t\treturn ctrl.Result{}, err\n\t\t\t}\n\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t}\n\n\tif du.Spec.Cancel {\n\t\tif spotted, found := r.cancelledDataUpload[du.Name]; !found {\n\t\t\tr.cancelledDataUpload[du.Name] = r.Clock.Now()\n\t\t} else {\n\t\t\tdelay := cancelDelayOthers\n\t\t\tif du.Status.Phase == velerov2alpha1api.DataUploadPhaseInProgress {\n\t\t\t\tdelay = cancelDelayInProgress\n\t\t\t}\n\n\t\t\tif time.Since(spotted) > delay {\n\t\t\t\tlog.Infof(\"Data upload %s is canceled in Phase %s but not handled in reasonable time\", du.GetName(), du.Status.Phase)\n\t\t\t\tif r.tryCancelDataUpload(ctx, du, \"\") {\n\t\t\t\t\tdelete(r.cancelledDataUpload, du.Name)\n\t\t\t\t}\n\n\t\t\t\treturn ctrl.Result{}, nil\n\t\t\t}\n\t\t}\n\t}\n\n\tep, ok := r.snapshotExposerList[du.Spec.SnapshotType]\n\tif !ok {\n\t\treturn r.errorOut(ctx, du, errors.Errorf(\"%s type of snapshot exposer is not exist\", du.Spec.SnapshotType), \"not exist type of exposer\", log)\n\t}\n\n\tif du.Status.Phase == \"\" || du.Status.Phase == velerov2alpha1api.DataUploadPhaseNew {\n\t\tif du.Spec.Cancel {\n\t\t\tlog.Debugf(\"Data upload is canceled in Phase %s\", du.Status.Phase)\n\n\t\t\tr.tryCancelDataUpload(ctx, du, \"\")\n\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tif r.vgdpCounter != nil && r.vgdpCounter.IsConstrained(ctx, r.logger) {\n\t\t\tlog.Debug(\"Data path initiation is constrained, requeue later\")\n\t\t\treturn ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5}, nil\n\t\t}\n\n\t\tlog.Info(\"Data upload starting\")\n\n\t\taccepted, err := r.acceptDataUpload(ctx, du)\n\t\tif err != nil {\n\t\t\treturn ctrl.Result{}, errors.Wrapf(err, \"error accepting the data upload %s\", du.Name)\n\t\t}\n\n\t\tif !accepted {\n\t\t\tlog.Debug(\"Data upload is not accepted\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tlog.Info(\"Data upload is accepted\")\n\n\t\texposeParam, err := r.setupExposeParam(du)\n\t\tif err != nil {\n\t\t\treturn r.errorOut(ctx, du, err, \"failed to set exposer parameters\", log)\n\t\t}\n\n\t\t// Expose() will trigger to create one pod whose volume is restored by a given volume snapshot,\n\t\t// but the pod maybe is not in the same node of the current controller, so we need to return it here.\n\t\t// And then only the controller who is in the same node could do the rest work.\n\t\tif err := ep.Expose(ctx, getOwnerObject(du), exposeParam); err != nil {\n\t\t\treturn r.errorOut(ctx, du, err, \"error exposing snapshot\", log)\n\t\t}\n\n\t\tlog.Info(\"Snapshot is exposed\")\n\n\t\treturn ctrl.Result{}, nil\n\t} else if du.Status.Phase == velerov2alpha1api.DataUploadPhaseAccepted {\n\t\tif peekErr := ep.PeekExposed(ctx, getOwnerObject(du)); peekErr != nil {\n\t\t\tlog.Errorf(\"Cancel du %s/%s because of expose error %s\", du.Namespace, du.Name, peekErr)\n\n\t\t\tdiags := strings.Split(ep.DiagnoseExpose(ctx, getOwnerObject(du)), \"\\n\")\n\t\t\tfor _, diag := range diags {\n\t\t\t\tlog.Warnf(\"[Diagnose DU expose]%s\", diag)\n\t\t\t}\n\n\t\t\tr.tryCancelDataUpload(ctx, du, fmt.Sprintf(\"found a du %s/%s with expose error: %s. mark it as cancel\", du.Namespace, du.Name, peekErr))\n\t\t} else if du.Status.AcceptedTimestamp != nil {\n\t\t\tif time.Since(du.Status.AcceptedTimestamp.Time) >= r.preparingTimeout {\n\t\t\t\tr.onPrepareTimeout(ctx, du)\n\t\t\t}\n\t\t}\n\n\t\treturn ctrl.Result{}, nil\n\t} else if du.Status.Phase == velerov2alpha1api.DataUploadPhasePrepared {\n\t\tlog.Infof(\"Data upload is prepared and should be processed by %s (%s)\", du.Status.Node, r.nodeName)\n\n\t\tif du.Status.Node != r.nodeName {\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tif du.Spec.Cancel {\n\t\t\tlog.Info(\"Prepared data upload is being canceled\")\n\t\t\tr.OnDataUploadCancelled(ctx, du.GetNamespace(), du.GetName())\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tasyncBR := r.dataPathMgr.GetAsyncBR(du.Name)\n\t\tif asyncBR != nil {\n\t\t\tlog.Info(\"Cancellable data path is already started\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\twaitExposePara := r.setupWaitExposePara(du)\n\t\tres, err := ep.GetExposed(ctx, getOwnerObject(du), du.Spec.OperationTimeout.Duration, waitExposePara)\n\t\tif err != nil {\n\t\t\treturn r.errorOut(ctx, du, err, \"exposed snapshot is not ready\", log)\n\t\t} else if res == nil {\n\t\t\treturn r.errorOut(ctx, du, errors.New(\"no expose result is available for the current node\"), \"exposed snapshot is not ready\", log)\n\t\t}\n\n\t\tif res.ByPod.NodeOS == nil {\n\t\t\treturn r.errorOut(ctx, du, errors.New(\"unsupported ambiguous node OS\"), \"invalid expose result\", log)\n\t\t}\n\n\t\tlog.Info(\"Exposed snapshot is ready and creating data path routine\")\n\n\t\t// Need to first create file system BR and get data path instance then update data upload status\n\t\tcallbacks := datapath.Callbacks{\n\t\t\tOnCompleted: r.OnDataUploadCompleted,\n\t\t\tOnFailed: r.OnDataUploadFailed,\n\t\t\tOnCancelled: r.OnDataUploadCancelled,\n\t\t\tOnProgress: r.OnDataUploadProgress,\n\t\t}\n\n\t\tasyncBR, err = r.dataPathMgr.CreateMicroServiceBRWatcher(ctx, r.client, r.kubeClient, r.mgr, datapath.TaskTypeBackup,\n\t\t\tdu.Name, du.Namespace, res.ByPod.HostingPod.Name, res.ByPod.HostingContainer, du.Name, callbacks, false, log)\n\t\tif err != nil {\n\t\t\tif err == datapath.ConcurrentLimitExceed {\n\t\t\t\tlog.Debug(\"Data path instance is concurrent limited requeue later\")\n\t\t\t\treturn ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5}, nil\n\t\t\t} else {\n\t\t\t\treturn r.errorOut(ctx, du, err, \"error to create data path\", log)\n\t\t\t}\n\t\t}\n\n\t\tif err := r.initCancelableDataPath(ctx, asyncBR, res, log); err != nil {\n\t\t\tlog.WithError(err).Errorf(\"Failed to init cancelable data path for %s\", du.Name)\n\n\t\t\tr.closeDataPath(ctx, du.Name)\n\t\t\treturn r.errorOut(ctx, du, err, \"error initializing data path\", log)\n\t\t}\n\n\t\t// Update status to InProgress\n\t\tterminated := false\n\t\tif err := UpdateDataUploadWithRetry(ctx, r.client, types.NamespacedName{Namespace: du.Namespace, Name: du.Name}, log, func(du *velerov2alpha1api.DataUpload) bool {\n\t\t\tif isDataUploadInFinalState(du) {\n\t\t\t\tterminated = true\n\t\t\t\treturn false\n\t\t\t}\n\n\t\t\tdu.Status.Phase = velerov2alpha1api.DataUploadPhaseInProgress\n\t\t\tdu.Status.StartTimestamp = &metav1.Time{Time: r.Clock.Now()}\n\t\t\tdu.Status.NodeOS = velerov2alpha1api.NodeOS(*res.ByPod.NodeOS)\n\n\t\t\tdelete(du.Labels, exposer.ExposeOnGoingLabel)\n\n\t\t\treturn true\n\t\t}); err != nil {\n\t\t\tlog.WithError(err).Warnf(\"Failed to update dataupload %s to InProgress, will data path close and retry\", du.Name)\n\n\t\t\tr.closeDataPath(ctx, du.Name)\n\t\t\treturn ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5}, nil\n\t\t}\n\n\t\tif terminated {\n\t\t\tlog.Warnf(\"dataupload %s is terminated during transition from prepared\", du.Name)\n\t\t\tr.closeDataPath(ctx, du.Name)\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tlog.Info(\"Data upload is marked as in progress\")\n\n\t\tif err := r.startCancelableDataPath(asyncBR, du, res, log); err != nil {\n\t\t\tlog.WithError(err).Errorf(\"Failed to start cancelable data path for %s\", du.Name)\n\t\t\tr.closeDataPath(ctx, du.Name)\n\n\t\t\treturn r.errorOut(ctx, du, err, \"error starting data path\", log)\n\t\t}\n\n\t\treturn ctrl.Result{}, nil\n\t} else if du.Status.Phase == velerov2alpha1api.DataUploadPhaseInProgress {\n\t\tif du.Spec.Cancel {\n\t\t\tif du.Status.Node != r.nodeName {\n\t\t\t\treturn ctrl.Result{}, nil\n\t\t\t}\n\n\t\t\tlog.Info(\"In progress data upload is being canceled\")\n\n\t\t\tasyncBR := r.dataPathMgr.GetAsyncBR(du.Name)\n\t\t\tif asyncBR == nil {\n\t\t\t\tr.OnDataUploadCancelled(ctx, du.GetNamespace(), du.GetName())\n\t\t\t\treturn ctrl.Result{}, nil\n\t\t\t}\n\n\t\t\t// Update status to Canceling\n\t\t\tif err := UpdateDataUploadWithRetry(ctx, r.client, types.NamespacedName{Namespace: du.Namespace, Name: du.Name}, log, func(du *velerov2alpha1api.DataUpload) bool {\n\t\t\t\tif isDataUploadInFinalState(du) {\n\t\t\t\t\tlog.Warnf(\"dataupload %s is terminated, abort setting it to canceling\", du.Name)\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tdu.Status.Phase = velerov2alpha1api.DataUploadPhaseCanceling\n\t\t\t\treturn true\n\t\t\t}); err != nil {\n\t\t\t\tlog.WithError(err).Error(\"error updating data upload into canceling status\")\n\t\t\t\treturn ctrl.Result{}, err\n\t\t\t}\n\n\t\t\tasyncBR.Cancel()\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t}\n\n\treturn ctrl.Result{}, nil\n}\n\nfunc (r *DataUploadReconciler) initCancelableDataPath(ctx context.Context, asyncBR datapath.AsyncBR, res *exposer.ExposeResult, log logrus.FieldLogger) error {\n\tlog.Info(\"Init cancelable dataUpload\")\n\n\tif err := asyncBR.Init(ctx, nil); err != nil {\n\t\treturn errors.Wrap(err, \"error initializing asyncBR\")\n\t}\n\n\tlog.Infof(\"async backup init for pod %s, volume %s\", res.ByPod.HostingPod.Name, res.ByPod.VolumeName)\n\n\treturn nil\n}\n\nfunc (r *DataUploadReconciler) startCancelableDataPath(asyncBR datapath.AsyncBR, du *velerov2alpha1api.DataUpload, res *exposer.ExposeResult, log logrus.FieldLogger) error {\n\tlog.Info(\"Start cancelable dataUpload\")\n\n\tif err := asyncBR.StartBackup(datapath.AccessPoint{\n\t\tByPath: res.ByPod.VolumeName,\n\t}, du.Spec.DataMoverConfig, nil); err != nil {\n\t\treturn errors.Wrapf(err, \"error starting async backup for pod %s, volume %s\", res.ByPod.HostingPod.Name, res.ByPod.VolumeName)\n\t}\n\n\tlog.Infof(\"Async backup started for pod %s, volume %s\", res.ByPod.HostingPod.Name, res.ByPod.VolumeName)\n\treturn nil\n}\n\nfunc (r *DataUploadReconciler) OnDataUploadCompleted(ctx context.Context, namespace string, duName string, result datapath.Result) {\n\tdefer r.dataPathMgr.RemoveAsyncBR(duName)\n\n\tlog := r.logger.WithField(\"dataupload\", duName)\n\n\tlog.Info(\"Async fs backup data path completed\")\n\n\tvar du velerov2alpha1api.DataUpload\n\tif err := r.client.Get(ctx, types.NamespacedName{Name: duName, Namespace: namespace}, &du); err != nil {\n\t\tlog.WithError(err).Warn(\"Failed to get dataupload on completion\")\n\t\treturn\n\t}\n\n\t// cleans up any objects generated during the snapshot expose\n\tep, ok := r.snapshotExposerList[du.Spec.SnapshotType]\n\tif !ok {\n\t\tlog.WithError(fmt.Errorf(\"%v type of snapshot exposer is not exist\", du.Spec.SnapshotType)).\n\t\t\tWarn(\"Failed to clean up resources on completion\")\n\t} else {\n\t\tvar volumeSnapshotName string\n\t\tif du.Spec.SnapshotType == velerov2alpha1api.SnapshotTypeCSI { // Other exposer should have another condition\n\t\t\tvolumeSnapshotName = du.Spec.CSISnapshot.VolumeSnapshot\n\t\t}\n\t\tep.CleanUp(ctx, getOwnerObject(&du), volumeSnapshotName, du.Spec.SourceNamespace)\n\t}\n\n\t// Update status to Completed with path & snapshot ID.\n\tif err := UpdateDataUploadWithRetry(ctx, r.client, types.NamespacedName{Namespace: du.Namespace, Name: du.Name}, log, func(du *velerov2alpha1api.DataUpload) bool {\n\t\tif isDataUploadInFinalState(du) {\n\t\t\treturn false\n\t\t}\n\n\t\tdu.Status.Path = result.Backup.Source.ByPath\n\t\tdu.Status.Phase = velerov2alpha1api.DataUploadPhaseCompleted\n\t\tdu.Status.SnapshotID = result.Backup.SnapshotID\n\t\tdu.Status.IncrementalBytes = result.Backup.IncrementalBytes\n\t\tdu.Status.CompletionTimestamp = &metav1.Time{Time: r.Clock.Now()}\n\t\tif result.Backup.EmptySnapshot {\n\t\t\tdu.Status.Message = \"volume was empty so no data was upload\"\n\t\t}\n\n\t\tdelete(du.Labels, exposer.ExposeOnGoingLabel)\n\n\t\treturn true\n\t}); err != nil {\n\t\tlog.WithError(err).Error(\"error updating DataUpload status\")\n\t} else {\n\t\tlog.Info(\"Data upload completed\")\n\t\tr.metrics.RegisterDataUploadSuccess(r.nodeName)\n\t}\n}\n\nfunc (r *DataUploadReconciler) OnDataUploadFailed(ctx context.Context, namespace, duName string, err error) {\n\tdefer r.dataPathMgr.RemoveAsyncBR(duName)\n\n\tlog := r.logger.WithField(\"dataupload\", duName)\n\n\tlog.WithError(err).Error(\"Async fs backup data path failed\")\n\n\tvar du velerov2alpha1api.DataUpload\n\tif getErr := r.client.Get(ctx, types.NamespacedName{Name: duName, Namespace: namespace}, &du); getErr != nil {\n\t\tlog.WithError(getErr).Warn(\"Failed to get dataupload on failure\")\n\t} else {\n\t\t_, _ = r.errorOut(ctx, &du, err, \"data path backup failed\", log)\n\t}\n}\n\nfunc (r *DataUploadReconciler) OnDataUploadCancelled(ctx context.Context, namespace string, duName string) {\n\tdefer r.dataPathMgr.RemoveAsyncBR(duName)\n\n\tlog := r.logger.WithField(\"dataupload\", duName)\n\n\tlog.Warn(\"Async fs backup data path canceled\")\n\n\tdu := &velerov2alpha1api.DataUpload{}\n\tif getErr := r.client.Get(ctx, types.NamespacedName{Name: duName, Namespace: namespace}, du); getErr != nil {\n\t\tlog.WithError(getErr).Warn(\"Failed to get dataupload on cancel\")\n\t\treturn\n\t}\n\t// cleans up any objects generated during the snapshot expose\n\tr.cleanUp(ctx, du, log)\n\n\tif err := UpdateDataUploadWithRetry(ctx, r.client, types.NamespacedName{Namespace: du.Namespace, Name: du.Name}, log, func(du *velerov2alpha1api.DataUpload) bool {\n\t\tif isDataUploadInFinalState(du) {\n\t\t\treturn false\n\t\t}\n\n\t\tdu.Status.Phase = velerov2alpha1api.DataUploadPhaseCanceled\n\t\tif du.Status.StartTimestamp.IsZero() {\n\t\t\tdu.Status.StartTimestamp = &metav1.Time{Time: r.Clock.Now()}\n\t\t}\n\t\tdu.Status.CompletionTimestamp = &metav1.Time{Time: r.Clock.Now()}\n\n\t\tdelete(du.Labels, exposer.ExposeOnGoingLabel)\n\n\t\treturn true\n\t}); err != nil {\n\t\tlog.WithError(err).Error(\"error updating DataUpload status\")\n\t} else {\n\t\tr.metrics.RegisterDataUploadCancel(r.nodeName)\n\t\tdelete(r.cancelledDataUpload, du.Name)\n\t}\n}\n\nfunc (r *DataUploadReconciler) tryCancelDataUpload(ctx context.Context, du *velerov2alpha1api.DataUpload, message string) bool {\n\tlog := r.logger.WithField(\"dataupload\", du.Name)\n\tsucceeded, err := funcExclusiveUpdateDataUpload(ctx, r.client, du, func(dataUpload *velerov2alpha1api.DataUpload) {\n\t\tdataUpload.Status.Phase = velerov2alpha1api.DataUploadPhaseCanceled\n\t\tif dataUpload.Status.StartTimestamp.IsZero() {\n\t\t\tdataUpload.Status.StartTimestamp = &metav1.Time{Time: r.Clock.Now()}\n\t\t}\n\t\tdataUpload.Status.CompletionTimestamp = &metav1.Time{Time: r.Clock.Now()}\n\n\t\tif message != \"\" {\n\t\t\tdataUpload.Status.Message = message\n\t\t}\n\n\t\tdelete(dataUpload.Labels, exposer.ExposeOnGoingLabel)\n\t})\n\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"error updating dataupload status\")\n\t\treturn false\n\t} else if !succeeded {\n\t\tlog.Warn(\"conflict in updating dataupload status and will try it again later\")\n\t\treturn false\n\t}\n\n\t// success update\n\tr.metrics.RegisterDataUploadCancel(r.nodeName)\n\t// cleans up any objects generated during the snapshot expose\n\tr.cleanUp(ctx, du, log)\n\n\tlog.Warn(\"data upload is canceled\")\n\n\treturn true\n}\n\nfunc (r *DataUploadReconciler) cleanUp(ctx context.Context, du *velerov2alpha1api.DataUpload, log logrus.FieldLogger) {\n\tep, ok := r.snapshotExposerList[du.Spec.SnapshotType]\n\tif !ok {\n\t\tlog.WithError(fmt.Errorf(\"%v type of snapshot exposer is not exist\", du.Spec.SnapshotType)).\n\t\t\tWarn(\"Failed to clean up resources on canceled\")\n\t} else {\n\t\tvar volumeSnapshotName string\n\t\tif du.Spec.SnapshotType == velerov2alpha1api.SnapshotTypeCSI { // Other exposer should have another condition\n\t\t\tvolumeSnapshotName = du.Spec.CSISnapshot.VolumeSnapshot\n\t\t}\n\t\tep.CleanUp(ctx, getOwnerObject(du), volumeSnapshotName, du.Spec.SourceNamespace)\n\t}\n}\n\nfunc (r *DataUploadReconciler) OnDataUploadProgress(ctx context.Context, namespace string, duName string, progress *uploader.Progress) {\n\tlog := r.logger.WithField(\"dataupload\", duName)\n\n\tif err := UpdateDataUploadWithRetry(ctx, r.client, types.NamespacedName{Namespace: namespace, Name: duName}, log, func(du *velerov2alpha1api.DataUpload) bool {\n\t\tdu.Status.Progress = shared.DataMoveOperationProgress{TotalBytes: progress.TotalBytes, BytesDone: progress.BytesDone}\n\t\treturn true\n\t}); err != nil {\n\t\tlog.WithError(err).Error(\"Failed to update progress\")\n\t}\n}\n\n// SetupWithManager registers the DataUpload controller.\n// The fresh new DataUpload CR first created will trigger to create one pod (long time, maybe failure or unknown status) by one of the dataupload controllers\n// then the request will get out of the Reconcile queue immediately by not blocking others' CR handling, in order to finish the rest data upload process we need to\n// re-enqueue the previous related request once the related pod is in running status to keep going on the rest logic. and below logic will avoid handling the unwanted\n// pod status and also avoid block others CR handling\nfunc (r *DataUploadReconciler) SetupWithManager(mgr ctrl.Manager) error {\n\tgp := kube.NewGenericEventPredicate(func(object client.Object) bool {\n\t\tdu := object.(*velerov2alpha1api.DataUpload)\n\t\tif du.Status.Phase == velerov2alpha1api.DataUploadPhaseAccepted {\n\t\t\treturn true\n\t\t}\n\n\t\tif du.Spec.Cancel && !isDataUploadInFinalState(du) {\n\t\t\treturn true\n\t\t}\n\n\t\tif isDataUploadInFinalState(du) && !du.DeletionTimestamp.IsZero() {\n\t\t\treturn true\n\t\t}\n\n\t\treturn false\n\t})\n\ts := kube.NewPeriodicalEnqueueSource(r.logger.WithField(\"controller\", constant.ControllerDataUpload), r.client, &velerov2alpha1api.DataUploadList{}, preparingMonitorFrequency, kube.PeriodicalEnqueueSourceOption{\n\t\tPredicates: []predicate.Predicate{gp},\n\t})\n\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\tFor(&velerov2alpha1api.DataUpload{}).\n\t\tWatchesRawSource(s).\n\t\tWatches(&corev1api.Pod{}, kube.EnqueueRequestsFromMapUpdateFunc(r.findDataUploadForPod),\n\t\t\tbuilder.WithPredicates(predicate.Funcs{\n\t\t\t\tUpdateFunc: func(ue event.UpdateEvent) bool {\n\t\t\t\t\tnewObj := ue.ObjectNew.(*corev1api.Pod)\n\n\t\t\t\t\tif _, ok := newObj.Labels[velerov1api.DataUploadLabel]; !ok {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\n\t\t\t\t\tif newObj.Spec.NodeName != r.nodeName {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\n\t\t\t\t\treturn true\n\t\t\t\t},\n\t\t\t\tCreateFunc: func(event.CreateEvent) bool {\n\t\t\t\t\treturn false\n\t\t\t\t},\n\t\t\t\tDeleteFunc: func(de event.DeleteEvent) bool {\n\t\t\t\t\treturn false\n\t\t\t\t},\n\t\t\t\tGenericFunc: func(ge event.GenericEvent) bool {\n\t\t\t\t\treturn false\n\t\t\t\t},\n\t\t\t})).\n\t\tComplete(r)\n}\n\nfunc (r *DataUploadReconciler) findDataUploadForPod(ctx context.Context, podObj client.Object) []reconcile.Request {\n\tpod := podObj.(*corev1api.Pod)\n\tdu, err := findDataUploadByPod(r.client, *pod)\n\tlog := r.logger.WithFields(logrus.Fields{\n\t\t\"Backup pod\": pod.Name,\n\t})\n\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"unable to get dataupload\")\n\t\treturn []reconcile.Request{}\n\t} else if du == nil {\n\t\tlog.Error(\"get empty DataUpload\")\n\t\treturn []reconcile.Request{}\n\t}\n\tlog = log.WithFields(logrus.Fields{\n\t\t\"Datadupload\": du.Name,\n\t})\n\n\tif du.Status.Phase != velerov2alpha1api.DataUploadPhaseAccepted {\n\t\treturn []reconcile.Request{}\n\t}\n\n\tif pod.Status.Phase == corev1api.PodRunning {\n\t\tlog.Info(\"Preparing dataupload\")\n\t\tif err = UpdateDataUploadWithRetry(context.Background(), r.client, types.NamespacedName{Namespace: du.Namespace, Name: du.Name}, log,\n\t\t\tfunc(du *velerov2alpha1api.DataUpload) bool {\n\t\t\t\tif isDataUploadInFinalState(du) {\n\t\t\t\t\tlog.Warnf(\"dataupload %s is terminated, abort setting it to prepared\", du.Name)\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tr.prepareDataUpload(du)\n\t\t\t\treturn true\n\t\t\t}); err != nil {\n\t\t\tlog.WithError(err).Warn(\"failed to update dataupload, prepare will halt for this dataupload\")\n\t\t\treturn []reconcile.Request{}\n\t\t}\n\t} else if unrecoverable, reason := kube.IsPodUnrecoverable(pod, log); unrecoverable { // let the abnormal backup pod failed early\n\t\terr := UpdateDataUploadWithRetry(context.Background(), r.client, types.NamespacedName{Namespace: du.Namespace, Name: du.Name}, r.logger.WithField(\"dataupload\", du.Name),\n\t\t\tfunc(dataUpload *velerov2alpha1api.DataUpload) bool {\n\t\t\t\tif dataUpload.Spec.Cancel {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tdataUpload.Spec.Cancel = true\n\t\t\t\tdataUpload.Status.Message = fmt.Sprintf(\"Cancel dataupload because the exposing pod %s/%s is in abnormal status for reason %s\", pod.Namespace, pod.Name, reason)\n\n\t\t\t\treturn true\n\t\t\t})\n\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Warn(\"failed to cancel dataupload, and it will wait for prepare timeout\")\n\t\t\treturn []reconcile.Request{}\n\t\t}\n\t\tlog.Infof(\"Exposed pod is in abnormal status(reason %s) and dataupload is marked as cancel\", reason)\n\t} else {\n\t\treturn []reconcile.Request{}\n\t}\n\n\trequest := reconcile.Request{\n\t\tNamespacedName: types.NamespacedName{\n\t\t\tNamespace: du.Namespace,\n\t\t\tName: du.Name,\n\t\t},\n\t}\n\treturn []reconcile.Request{request}\n}\n\nfunc (r *DataUploadReconciler) prepareDataUpload(du *velerov2alpha1api.DataUpload) {\n\tdu.Status.Phase = velerov2alpha1api.DataUploadPhasePrepared\n\tdu.Status.Node = r.nodeName\n}\n\nfunc (r *DataUploadReconciler) errorOut(ctx context.Context, du *velerov2alpha1api.DataUpload, err error, msg string, log logrus.FieldLogger) (ctrl.Result, error) {\n\tif se, ok := r.snapshotExposerList[du.Spec.SnapshotType]; ok {\n\t\tvar volumeSnapshotName string\n\t\tif du.Spec.SnapshotType == velerov2alpha1api.SnapshotTypeCSI { // Other exposer should have another condition\n\t\t\tvolumeSnapshotName = du.Spec.CSISnapshot.VolumeSnapshot\n\t\t}\n\t\tse.CleanUp(ctx, getOwnerObject(du), volumeSnapshotName, du.Spec.SourceNamespace)\n\t} else {\n\t\tlog.Errorf(\"failed to clean up exposed snapshot could not find %s snapshot exposer\", du.Spec.SnapshotType)\n\t}\n\n\treturn ctrl.Result{}, r.updateStatusToFailed(ctx, du, err, msg, log)\n}\n\nfunc (r *DataUploadReconciler) updateStatusToFailed(ctx context.Context, du *velerov2alpha1api.DataUpload, err error, msg string, log logrus.FieldLogger) error {\n\tlog.Info(\"update data upload status to Failed\")\n\n\tif patchErr := UpdateDataUploadWithRetry(ctx, r.client, types.NamespacedName{Namespace: du.Namespace, Name: du.Name}, log, func(du *velerov2alpha1api.DataUpload) bool {\n\t\tif isDataUploadInFinalState(du) {\n\t\t\treturn false\n\t\t}\n\n\t\tdu.Status.Phase = velerov2alpha1api.DataUploadPhaseFailed\n\t\tdu.Status.Message = errors.WithMessage(err, msg).Error()\n\t\tif du.Status.StartTimestamp.IsZero() {\n\t\t\tdu.Status.StartTimestamp = &metav1.Time{Time: r.Clock.Now()}\n\t\t}\n\n\t\tif dataPathError, ok := err.(datapath.DataPathError); ok {\n\t\t\tdu.Status.SnapshotID = dataPathError.GetSnapshotID()\n\t\t}\n\t\tdu.Status.CompletionTimestamp = &metav1.Time{Time: r.Clock.Now()}\n\n\t\tdelete(du.Labels, exposer.ExposeOnGoingLabel)\n\n\t\treturn true\n\t}); patchErr != nil {\n\t\tlog.WithError(patchErr).Error(\"error updating DataUpload status\")\n\t} else {\n\t\tr.metrics.RegisterDataUploadFailure(r.nodeName)\n\t}\n\n\treturn err\n}\n\nfunc (r *DataUploadReconciler) acceptDataUpload(ctx context.Context, du *velerov2alpha1api.DataUpload) (bool, error) {\n\tr.logger.Infof(\"Accepting data upload %s\", du.Name)\n\n\t// For all data upload controller in each node-agent will try to update dataupload CR, and only one controller will success,\n\t// and the success one could handle later logic\n\tupdated := du.DeepCopy()\n\n\tupdateFunc := func(dataUpload *velerov2alpha1api.DataUpload) {\n\t\tdataUpload.Status.Phase = velerov2alpha1api.DataUploadPhaseAccepted\n\t\tdataUpload.Status.AcceptedByNode = r.nodeName\n\t\tdataUpload.Status.AcceptedTimestamp = &metav1.Time{Time: r.Clock.Now()}\n\n\t\tif dataUpload.Labels == nil {\n\t\t\tdataUpload.Labels = make(map[string]string)\n\t\t}\n\t\tdataUpload.Labels[exposer.ExposeOnGoingLabel] = \"true\"\n\t}\n\n\tsucceeded, err := funcExclusiveUpdateDataUpload(ctx, r.client, updated, updateFunc)\n\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\tif succeeded {\n\t\tupdateFunc(du) // If update success, it's need to update du values in memory\n\t\tr.logger.WithField(\"Dataupload\", du.Name).Infof(\"This datauplod has been accepted by %s\", r.nodeName)\n\t\treturn true, nil\n\t}\n\n\tr.logger.WithField(\"Dataupload\", du.Name).Info(\"This datauplod has been accepted by others\")\n\treturn false, nil\n}\n\nfunc (r *DataUploadReconciler) onPrepareTimeout(ctx context.Context, du *velerov2alpha1api.DataUpload) {\n\tlog := r.logger.WithField(\"Dataupload\", du.Name)\n\n\tlog.Info(\"Timeout happened for preparing dataupload\")\n\n\tsucceeded, err := funcExclusiveUpdateDataUpload(ctx, r.client, du, func(du *velerov2alpha1api.DataUpload) {\n\t\tdu.Status.Phase = velerov2alpha1api.DataUploadPhaseFailed\n\t\tdu.Status.Message = \"timeout on preparing data upload\"\n\n\t\tdelete(du.Labels, exposer.ExposeOnGoingLabel)\n\t})\n\n\tif err != nil {\n\t\tlog.WithError(err).Warn(\"Failed to update dataupload\")\n\t\treturn\n\t}\n\n\tif !succeeded {\n\t\tlog.Warn(\"Dataupload has been updated by others\")\n\t\treturn\n\t}\n\n\tep, ok := r.snapshotExposerList[du.Spec.SnapshotType]\n\tif !ok {\n\t\tlog.WithError(fmt.Errorf(\"%v type of snapshot exposer is not exist\", du.Spec.SnapshotType)).\n\t\t\tWarn(\"Failed to clean up resources on canceled\")\n\t} else {\n\t\tvar volumeSnapshotName string\n\t\tif du.Spec.SnapshotType == velerov2alpha1api.SnapshotTypeCSI { // Other exposer should have another condition\n\t\t\tvolumeSnapshotName = du.Spec.CSISnapshot.VolumeSnapshot\n\t\t}\n\n\t\tdiags := strings.Split(ep.DiagnoseExpose(ctx, getOwnerObject(du)), \"\\n\")\n\t\tfor _, diag := range diags {\n\t\t\tlog.Warnf(\"[Diagnose DU expose]%s\", diag)\n\t\t}\n\n\t\tep.CleanUp(ctx, getOwnerObject(du), volumeSnapshotName, du.Spec.SourceNamespace)\n\n\t\tlog.Info(\"Dataupload has been cleaned up\")\n\t}\n\n\tr.metrics.RegisterDataUploadFailure(r.nodeName)\n}\n\nvar funcExclusiveUpdateDataUpload = exclusiveUpdateDataUpload\n\nfunc exclusiveUpdateDataUpload(ctx context.Context, cli client.Client, du *velerov2alpha1api.DataUpload,\n\tupdateFunc func(*velerov2alpha1api.DataUpload)) (bool, error) {\n\tupdateFunc(du)\n\n\terr := cli.Update(ctx, du)\n\tif err == nil {\n\t\treturn true, nil\n\t}\n\n\t// warn we won't rollback du values in memory when error\n\tif apierrors.IsConflict(err) {\n\t\treturn false, nil\n\t} else {\n\t\treturn false, err\n\t}\n}\n\nfunc (r *DataUploadReconciler) closeDataPath(ctx context.Context, duName string) {\n\tasyncBR := r.dataPathMgr.GetAsyncBR(duName)\n\tif asyncBR != nil {\n\t\tasyncBR.Close(ctx)\n\t}\n\n\tr.dataPathMgr.RemoveAsyncBR(duName)\n}\n\nfunc (r *DataUploadReconciler) setupExposeParam(du *velerov2alpha1api.DataUpload) (any, error) {\n\tlog := r.logger.WithField(\"dataupload\", du.Name)\n\n\tif du.Spec.SnapshotType == velerov2alpha1api.SnapshotTypeCSI {\n\t\tpvc := &corev1api.PersistentVolumeClaim{}\n\t\terr := r.client.Get(context.Background(), types.NamespacedName{\n\t\t\tNamespace: du.Spec.SourceNamespace,\n\t\t\tName: du.Spec.SourcePVC,\n\t\t}, pvc)\n\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"failed to get PVC %s/%s\", du.Spec.SourceNamespace, du.Spec.SourcePVC)\n\t\t}\n\n\t\tpv := &corev1api.PersistentVolume{}\n\t\tif err := r.client.Get(context.Background(), types.NamespacedName{\n\t\t\tName: pvc.Spec.VolumeName,\n\t\t}, pv); err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"failed to get source PV %s\", pvc.Spec.VolumeName)\n\t\t}\n\n\t\tnodeOS := kube.GetPVCAttachingNodeOS(pvc, r.kubeClient.CoreV1(), r.kubeClient.StorageV1(), log)\n\n\t\tif err := kube.HasNodeWithOS(context.Background(), nodeOS, r.kubeClient.CoreV1()); err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"no appropriate node to run data upload for PVC %s/%s\", du.Spec.SourceNamespace, du.Spec.SourcePVC)\n\t\t}\n\n\t\taccessMode := exposer.AccessModeFileSystem\n\t\tif pvc.Spec.VolumeMode != nil && *pvc.Spec.VolumeMode == corev1api.PersistentVolumeBlock {\n\t\t\taccessMode = exposer.AccessModeBlock\n\t\t}\n\n\t\thostingPodLabels := map[string]string{velerov1api.DataUploadLabel: du.Name}\n\t\tif len(r.podLabels) > 0 {\n\t\t\tfor k, v := range r.podLabels {\n\t\t\t\thostingPodLabels[k] = v\n\t\t\t}\n\t\t} else {\n\t\t\tfor _, k := range util.ThirdPartyLabels {\n\t\t\t\tif v, err := nodeagent.GetLabelValue(context.Background(), r.kubeClient, du.Namespace, k, nodeOS); err != nil {\n\t\t\t\t\tif err != nodeagent.ErrNodeAgentLabelNotFound {\n\t\t\t\t\t\tlog.WithError(err).Warnf(\"Failed to check node-agent label, skip adding host pod label %s\", k)\n\t\t\t\t\t}\n\t\t\t\t} else {\n\t\t\t\t\thostingPodLabels[k] = v\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\thostingPodAnnotation := map[string]string{}\n\t\tif len(r.podAnnotations) > 0 {\n\t\t\tfor k, v := range r.podAnnotations {\n\t\t\t\thostingPodAnnotation[k] = v\n\t\t\t}\n\t\t} else {\n\t\t\tfor _, k := range util.ThirdPartyAnnotations {\n\t\t\t\tif v, err := nodeagent.GetAnnotationValue(context.Background(), r.kubeClient, du.Namespace, k, nodeOS); err != nil {\n\t\t\t\t\tif err != nodeagent.ErrNodeAgentAnnotationNotFound {\n\t\t\t\t\t\tlog.WithError(err).Warnf(\"Failed to check node-agent annotation, skip adding host pod annotation %s\", k)\n\t\t\t\t\t}\n\t\t\t\t} else {\n\t\t\t\t\thostingPodAnnotation[k] = v\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\thostingPodTolerations := []corev1api.Toleration{}\n\t\tfor _, k := range util.ThirdPartyTolerations {\n\t\t\tif v, err := nodeagent.GetToleration(context.Background(), r.kubeClient, du.Namespace, k, nodeOS); err != nil {\n\t\t\t\tif err != nodeagent.ErrNodeAgentTolerationNotFound {\n\t\t\t\t\tlog.WithError(err).Warnf(\"Failed to check node-agent toleration, skip adding host pod toleration %s\", k)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\thostingPodTolerations = append(hostingPodTolerations, *v)\n\t\t\t}\n\t\t}\n\n\t\treturn &exposer.CSISnapshotExposeParam{\n\t\t\tSnapshotName: du.Spec.CSISnapshot.VolumeSnapshot,\n\t\t\tSourceNamespace: du.Spec.SourceNamespace,\n\t\t\tSourcePVCName: pvc.Name,\n\t\t\tSourcePVName: pv.Name,\n\t\t\tStorageClass: du.Spec.CSISnapshot.StorageClass,\n\t\t\tHostingPodLabels: hostingPodLabels,\n\t\t\tHostingPodAnnotations: hostingPodAnnotation,\n\t\t\tHostingPodTolerations: hostingPodTolerations,\n\t\t\tAccessMode: accessMode,\n\t\t\tOperationTimeout: du.Spec.OperationTimeout.Duration,\n\t\t\tExposeTimeout: r.preparingTimeout,\n\t\t\tVolumeSize: pvc.Spec.Resources.Requests[corev1api.ResourceStorage],\n\t\t\tAffinity: r.loadAffinity,\n\t\t\tBackupPVCConfig: r.backupPVCConfig,\n\t\t\tResources: r.podResources,\n\t\t\tNodeOS: nodeOS,\n\t\t\tPriorityClassName: r.dataMovePriorityClass,\n\t\t}, nil\n\t}\n\n\treturn nil, nil\n}\n\nfunc (r *DataUploadReconciler) setupWaitExposePara(du *velerov2alpha1api.DataUpload) any {\n\tif du.Spec.SnapshotType == velerov2alpha1api.SnapshotTypeCSI {\n\t\treturn &exposer.CSISnapshotExposeWaitParam{\n\t\t\tNodeClient: r.client,\n\t\t\tNodeName: r.nodeName,\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc getOwnerObject(du *velerov2alpha1api.DataUpload) corev1api.ObjectReference {\n\treturn corev1api.ObjectReference{\n\t\tKind: du.Kind,\n\t\tNamespace: du.Namespace,\n\t\tName: du.Name,\n\t\tUID: du.UID,\n\t\tAPIVersion: du.APIVersion,\n\t}\n}\n\nfunc findDataUploadByPod(client client.Client, pod corev1api.Pod) (*velerov2alpha1api.DataUpload, error) {\n\tif label, exist := pod.Labels[velerov1api.DataUploadLabel]; exist {\n\t\tdu := &velerov2alpha1api.DataUpload{}\n\t\terr := client.Get(context.Background(), types.NamespacedName{\n\t\t\tNamespace: pod.Namespace,\n\t\t\tName: label,\n\t\t}, du)\n\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"error to find DataUpload by pod %s/%s\", pod.Namespace, pod.Name)\n\t\t}\n\t\treturn du, nil\n\t}\n\treturn nil, nil\n}\n\nfunc isDataUploadInFinalState(du *velerov2alpha1api.DataUpload) bool {\n\treturn du.Status.Phase == velerov2alpha1api.DataUploadPhaseFailed ||\n\t\tdu.Status.Phase == velerov2alpha1api.DataUploadPhaseCanceled ||\n\t\tdu.Status.Phase == velerov2alpha1api.DataUploadPhaseCompleted\n}\n\nfunc UpdateDataUploadWithRetry(ctx context.Context, client client.Client, namespacedName types.NamespacedName, log logrus.FieldLogger, updateFunc func(*velerov2alpha1api.DataUpload) bool) error {\n\treturn wait.PollUntilContextCancel(ctx, time.Second, true, func(ctx context.Context) (bool, error) {\n\t\tdu := &velerov2alpha1api.DataUpload{}\n\t\tif err := client.Get(ctx, namespacedName, du); err != nil {\n\t\t\treturn false, errors.Wrap(err, \"getting DataUpload\")\n\t\t}\n\n\t\tif updateFunc(du) {\n\t\t\terr := client.Update(ctx, du)\n\t\t\tif err != nil {\n\t\t\t\tif apierrors.IsConflict(err) {\n\t\t\t\t\tlog.Debugf(\"failed to update dataupload for %s/%s and will retry it\", du.Namespace, du.Name)\n\t\t\t\t\treturn false, nil\n\t\t\t\t} else {\n\t\t\t\t\treturn false, errors.Wrapf(err, \"error updating dataupload with error %s/%s\", du.Namespace, du.Name)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\treturn true, nil\n\t})\n}\n\nvar funcResumeCancellableDataBackup = (*DataUploadReconciler).resumeCancellableDataPath\n\nfunc (r *DataUploadReconciler) AttemptDataUploadResume(ctx context.Context, logger *logrus.Entry, ns string) error {\n\tdataUploads := &velerov2alpha1api.DataUploadList{}\n\tif err := r.client.List(ctx, dataUploads, &client.ListOptions{Namespace: ns}); err != nil {\n\t\tr.logger.WithError(errors.WithStack(err)).Error(\"failed to list datauploads\")\n\t\treturn errors.Wrapf(err, \"error to list datauploads\")\n\t}\n\n\tfor i := range dataUploads.Items {\n\t\tdu := &dataUploads.Items[i]\n\t\tif du.Status.Phase == velerov2alpha1api.DataUploadPhaseInProgress {\n\t\t\tif du.Status.Node != r.nodeName {\n\t\t\t\tlogger.WithField(\"du\", du.Name).WithField(\"current node\", r.nodeName).Infof(\"DU should be resumed by another node %s\", du.Status.Node)\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\terr := funcResumeCancellableDataBackup(r, ctx, du, logger)\n\t\t\tif err == nil {\n\t\t\t\tlogger.WithField(\"du\", du.Name).WithField(\"current node\", r.nodeName).Info(\"Completed to resume in progress DU\")\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tlogger.WithField(\"dataupload\", du.GetName()).WithError(err).Warn(\"Failed to resume data path for du, have to cancel it\")\n\n\t\t\tresumeErr := err\n\t\t\terr = UpdateDataUploadWithRetry(ctx, r.client, types.NamespacedName{Namespace: du.Namespace, Name: du.Name}, logger.WithField(\"dataupload\", du.Name),\n\t\t\t\tfunc(dataUpload *velerov2alpha1api.DataUpload) bool {\n\t\t\t\t\tif dataUpload.Spec.Cancel {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\n\t\t\t\t\tdataUpload.Spec.Cancel = true\n\t\t\t\t\tdataUpload.Status.Message = fmt.Sprintf(\"Resume InProgress dataupload failed with error %v, mark it as cancel\", resumeErr)\n\n\t\t\t\t\treturn true\n\t\t\t\t})\n\t\t\tif err != nil {\n\t\t\t\tlogger.WithField(\"dataupload\", du.GetName()).WithError(errors.WithStack(err)).Error(\"Failed to trigger dataupload cancel\")\n\t\t\t}\n\t\t} else if !isDataUploadInFinalState(du) {\n\t\t\t// the Prepared CR could be still handled by dataupload controller after node-agent restart\n\t\t\t// the accepted CR may also suvived from node-agent restart as long as the intermediate objects are all done\n\t\t\tlogger.WithField(\"dataupload\", du.GetName()).Infof(\"find a dataupload with status %s\", du.Status.Phase)\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (r *DataUploadReconciler) resumeCancellableDataPath(ctx context.Context, du *velerov2alpha1api.DataUpload, log logrus.FieldLogger) error {\n\tlog.Info(\"Resume cancelable dataUpload\")\n\n\tep, ok := r.snapshotExposerList[du.Spec.SnapshotType]\n\tif !ok {\n\t\treturn errors.Errorf(\"error to find exposer for du %s\", du.Name)\n\t}\n\n\twaitExposePara := r.setupWaitExposePara(du)\n\tres, err := ep.GetExposed(ctx, getOwnerObject(du), du.Spec.OperationTimeout.Duration, waitExposePara)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to get exposed snapshot for du %s\", du.Name)\n\t}\n\n\tif res == nil {\n\t\treturn errors.Errorf(\"expose info missed for du %s\", du.Name)\n\t}\n\n\tcallbacks := datapath.Callbacks{\n\t\tOnCompleted: r.OnDataUploadCompleted,\n\t\tOnFailed: r.OnDataUploadFailed,\n\t\tOnCancelled: r.OnDataUploadCancelled,\n\t\tOnProgress: r.OnDataUploadProgress,\n\t}\n\n\tasyncBR, err := r.dataPathMgr.CreateMicroServiceBRWatcher(ctx, r.client, r.kubeClient, r.mgr, datapath.TaskTypeBackup, du.Name, du.Namespace, res.ByPod.HostingPod.Name, res.ByPod.HostingContainer, du.Name, callbacks, true, log)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to create asyncBR watcher for du %s\", du.Name)\n\t}\n\n\tresumeComplete := false\n\tdefer func() {\n\t\tif !resumeComplete {\n\t\t\tr.closeDataPath(ctx, du.Name)\n\t\t}\n\t}()\n\n\tif err := asyncBR.Init(ctx, nil); err != nil {\n\t\treturn errors.Wrapf(err, \"error to init asyncBR watcher for du %s\", du.Name)\n\t}\n\n\tif err := asyncBR.StartBackup(datapath.AccessPoint{\n\t\tByPath: res.ByPod.VolumeName,\n\t}, du.Spec.DataMoverConfig, nil); err != nil {\n\t\treturn errors.Wrapf(err, \"error to resume asyncBR watcher for du %s\", du.Name)\n\t}\n\n\tresumeComplete = true\n\n\tlog.Infof(\"asyncBR is resumed for du %s\", du.Name)\n\n\treturn nil\n}\n" }, { "path": "pkg/controller/data_upload_controller_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"testing\"\n\t\"time\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\tsnapshotFake \"github.com/kubernetes-csi/external-snapshotter/client/v8/clientset/versioned/fake\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\t\"k8s.io/apimachinery/pkg/api/resource\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/client-go/kubernetes\"\n\tclientgofake \"k8s.io/client-go/kubernetes/fake\"\n\t\"k8s.io/utils/clock\"\n\ttestclocks \"k8s.io/utils/clock/testing\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\t\"sigs.k8s.io/controller-runtime/pkg/manager\"\n\t\"sigs.k8s.io/controller-runtime/pkg/reconcile\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\tdatapathmocks \"github.com/vmware-tanzu/velero/pkg/datapath/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/exposer\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\tvelerotypes \"github.com/vmware-tanzu/velero/pkg/types\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nconst dataUploadName = \"dataupload-1\"\n\nconst fakeSnapshotType velerov2alpha1api.SnapshotType = \"fake-snapshot\"\n\ntype FakeClient struct {\n\tkbclient.Client\n\tgetError error\n\tcreateError error\n\tupdateError error\n\tpatchError error\n\tupdateConflict error\n\tlistError error\n}\n\nfunc (c *FakeClient) Get(ctx context.Context, key kbclient.ObjectKey, obj kbclient.Object, opts ...kbclient.GetOption) error {\n\tif c.getError != nil {\n\t\treturn c.getError\n\t}\n\n\treturn c.Client.Get(ctx, key, obj)\n}\n\nfunc (c *FakeClient) Create(ctx context.Context, obj kbclient.Object, opts ...kbclient.CreateOption) error {\n\tif c.createError != nil {\n\t\treturn c.createError\n\t}\n\n\treturn c.Client.Create(ctx, obj, opts...)\n}\n\nfunc (c *FakeClient) Update(ctx context.Context, obj kbclient.Object, opts ...kbclient.UpdateOption) error {\n\tif c.updateError != nil {\n\t\treturn c.updateError\n\t}\n\n\tif c.updateConflict != nil {\n\t\treturn c.updateConflict\n\t}\n\n\treturn c.Client.Update(ctx, obj, opts...)\n}\n\nfunc (c *FakeClient) Patch(ctx context.Context, obj kbclient.Object, patch kbclient.Patch, opts ...kbclient.PatchOption) error {\n\tif c.patchError != nil {\n\t\treturn c.patchError\n\t}\n\n\treturn c.Client.Patch(ctx, obj, patch, opts...)\n}\n\nfunc (c *FakeClient) List(ctx context.Context, list kbclient.ObjectList, opts ...kbclient.ListOption) error {\n\tif c.listError != nil {\n\t\treturn c.listError\n\t}\n\n\treturn c.Client.List(ctx, list, opts...)\n}\n\nfunc initDataUploaderReconciler(needError ...bool) (*DataUploadReconciler, error) {\n\tvar errs = make([]error, 6)\n\tfor k, isError := range needError {\n\t\tif k == 0 && isError {\n\t\t\terrs[0] = fmt.Errorf(\"Get error\")\n\t\t} else if k == 1 && isError {\n\t\t\terrs[1] = fmt.Errorf(\"Create error\")\n\t\t} else if k == 2 && isError {\n\t\t\terrs[2] = fmt.Errorf(\"Update error\")\n\t\t} else if k == 3 && isError {\n\t\t\terrs[3] = fmt.Errorf(\"Patch error\")\n\t\t} else if k == 4 && isError {\n\t\t\terrs[4] = apierrors.NewConflict(velerov2alpha1api.Resource(\"dataupload\"), dataUploadName, errors.New(\"conflict\"))\n\t\t} else if k == 5 && isError {\n\t\t\terrs[5] = fmt.Errorf(\"List error\")\n\t\t}\n\t}\n\n\treturn initDataUploaderReconcilerWithError(errs...)\n}\n\nfunc initDataUploaderReconcilerWithError(needError ...error) (*DataUploadReconciler, error) {\n\tvscName := \"fake-vsc\"\n\tvsObject := &snapshotv1api.VolumeSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-volume-snapshot\",\n\t\t\tNamespace: \"fake-ns\",\n\t\t},\n\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\tBoundVolumeSnapshotContentName: &vscName,\n\t\t\tReadyToUse: boolptr.True(),\n\t\t\tRestoreSize: &resource.Quantity{},\n\t\t},\n\t}\n\tvar restoreSize int64\n\tvscObj := &snapshotv1api.VolumeSnapshotContent{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-vsc\",\n\t\t},\n\t\tSpec: snapshotv1api.VolumeSnapshotContentSpec{\n\t\t\tDeletionPolicy: snapshotv1api.VolumeSnapshotContentDelete,\n\t\t},\n\t\tStatus: &snapshotv1api.VolumeSnapshotContentStatus{\n\t\t\tRestoreSize: &restoreSize,\n\t\t},\n\t}\n\n\tdaemonSet := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"velero\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t\tAPIVersion: appsv1api.SchemeGroupVersion.String(),\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tImage: \"fake-image\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tnode := builder.ForNode(\"fake-node\").Labels(map[string]string{kube.NodeOSLabel: kube.NodeOSLinux}).Result()\n\n\tdataPathMgr := datapath.NewManager(1)\n\n\tnow, err := time.Parse(time.RFC1123, time.RFC1123)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tnow = now.Local()\n\tscheme := runtime.NewScheme()\n\terr = velerov1api.AddToScheme(scheme)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\terr = velerov2alpha1api.AddToScheme(scheme)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\terr = corev1api.AddToScheme(scheme)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfakeClient := &FakeClient{\n\t\tClient: fake.NewClientBuilder().WithScheme(scheme).Build(),\n\t}\n\n\tfor k := range needError {\n\t\tif k == 0 {\n\t\t\tfakeClient.getError = needError[0]\n\t\t} else if k == 1 {\n\t\t\tfakeClient.createError = needError[1]\n\t\t} else if k == 2 {\n\t\t\tfakeClient.updateError = needError[2]\n\t\t} else if k == 3 {\n\t\t\tfakeClient.patchError = needError[3]\n\t\t} else if k == 4 {\n\t\t\tfakeClient.updateConflict = needError[4]\n\t\t} else if k == 5 {\n\t\t\tfakeClient.listError = needError[5]\n\t\t}\n\t}\n\n\tfakeSnapshotClient := snapshotFake.NewSimpleClientset(vsObject, vscObj)\n\tfakeKubeClient := clientgofake.NewSimpleClientset(daemonSet, node)\n\n\treturn NewDataUploadReconciler(\n\t\tfakeClient,\n\t\tnil,\n\t\tfakeKubeClient,\n\t\tfakeSnapshotClient.SnapshotV1(),\n\t\tdataPathMgr,\n\t\tnil,\n\t\tnil,\n\t\tmap[string]velerotypes.BackupPVC{},\n\t\tcorev1api.ResourceRequirements{},\n\t\ttestclocks.NewFakeClock(now),\n\t\t\"test-node\",\n\t\ttime.Minute*5,\n\t\tvelerotest.NewLogger(),\n\t\tmetrics.NewServerMetrics(),\n\t\t\"\", // dataMovePriorityClass\n\t\tnil, // podLabels\n\t\tnil, // podAnnotations\n\t), nil\n}\n\nfunc dataUploadBuilder() *builder.DataUploadBuilder {\n\tcsi := &velerov2alpha1api.CSISnapshotSpec{\n\t\tSnapshotClass: \"csi-azuredisk-vsc\",\n\t\tStorageClass: \"default\",\n\t\tVolumeSnapshot: \"fake-volume-snapshot\",\n\t}\n\treturn builder.ForDataUpload(velerov1api.DefaultNamespace, dataUploadName).\n\t\tBackupStorageLocation(\"bsl-loc\").\n\t\tDataMover(\"velero\").\n\t\tSnapshotType(\"CSI\").SourceNamespace(\"fake-ns\").SourcePVC(\"test-pvc\").CSISnapshot(csi)\n}\n\ntype fakeSnapshotExposer struct {\n\tkubeClient kbclient.Client\n\tclock clock.WithTickerAndDelayedExecution\n\tambiguousNodeOS bool\n\tpeekErr error\n\texposeErr error\n\tgetErr error\n\tgetNil bool\n}\n\nfunc (f *fakeSnapshotExposer) Expose(ctx context.Context, ownerObject corev1api.ObjectReference, param any) error {\n\tif f.exposeErr != nil {\n\t\treturn f.exposeErr\n\t}\n\n\treturn nil\n}\n\nfunc (f *fakeSnapshotExposer) GetExposed(ctx context.Context, du corev1api.ObjectReference, tm time.Duration, para any) (*exposer.ExposeResult, error) {\n\tif f.getErr != nil {\n\t\treturn nil, f.getErr\n\t}\n\n\tif f.getNil {\n\t\treturn nil, nil\n\t}\n\n\tpod := &corev1api.Pod{}\n\n\tnodeOS := \"linux\"\n\tpNodeOS := &nodeOS\n\tif f.ambiguousNodeOS {\n\t\tpNodeOS = nil\n\t}\n\treturn &exposer.ExposeResult{ByPod: exposer.ExposeByPod{HostingPod: pod, VolumeName: dataUploadName, NodeOS: pNodeOS}}, nil\n}\n\nfunc (f *fakeSnapshotExposer) PeekExposed(ctx context.Context, ownerObject corev1api.ObjectReference) error {\n\treturn f.peekErr\n}\n\nfunc (f *fakeSnapshotExposer) DiagnoseExpose(context.Context, corev1api.ObjectReference) string {\n\treturn \"\"\n}\n\nfunc (f *fakeSnapshotExposer) CleanUp(context.Context, corev1api.ObjectReference, string, string) {\n}\n\ntype fakeFSBR struct {\n\tkubeClient kbclient.Client\n\tclock clock.WithTickerAndDelayedExecution\n\tinitErr error\n\tstartErr error\n}\n\nfunc (f *fakeFSBR) Init(ctx context.Context, param any) error {\n\treturn f.initErr\n}\n\nfunc (f *fakeFSBR) StartBackup(source datapath.AccessPoint, uploaderConfigs map[string]string, param any) error {\n\treturn f.startErr\n}\n\nfunc (f *fakeFSBR) StartRestore(snapshotID string, target datapath.AccessPoint, uploaderConfigs map[string]string) error {\n\treturn nil\n}\n\nfunc (b *fakeFSBR) Cancel() {\n}\n\nfunc (b *fakeFSBR) Close(ctx context.Context) {\n}\n\nfunc TestReconcile(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdu *velerov2alpha1api.DataUpload\n\t\tnotCreateDU bool\n\t\tneedDelete bool\n\t\tsportTime *metav1.Time\n\t\tpod *corev1api.Pod\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\tsnapshotExposerList map[velerov2alpha1api.SnapshotType]exposer.SnapshotExposer\n\t\tdataMgr *datapath.Manager\n\t\tneedCreateFSBR bool\n\t\tneedExclusiveUpdateError error\n\t\texpected *velerov2alpha1api.DataUpload\n\t\texpectDeleted bool\n\t\texpectCancelRecord bool\n\t\tneedErrs []bool\n\t\tambiguousNodeOS bool\n\t\tpeekErr error\n\t\texposeErr error\n\t\tgetExposeErr error\n\t\tgetExposeNil bool\n\t\tfsBRInitErr error\n\t\tfsBRStartErr error\n\t\tconstrained bool\n\t\texpectedErr string\n\t\texpectedResult *ctrl.Result\n\t\texpectDataPath bool\n\t}{\n\t\t{\n\t\t\tname: \"du not found\",\n\t\t\tdu: dataUploadBuilder().Result(),\n\t\t\tnotCreateDU: true,\n\t\t},\n\t\t{\n\t\t\tname: \"du not created in velero default namespace\",\n\t\t\tdu: builder.ForDataUpload(\"test-ns\", dataUploadName).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"get du fail\",\n\t\t\tdu: dataUploadBuilder().Result(),\n\t\t\tneedErrs: []bool{true, false, false, false},\n\t\t\texpectedErr: \"getting DataUpload: Get error\",\n\t\t},\n\t\t{\n\t\t\tname: \"du is not for built-in dm\",\n\t\t\tdu: dataUploadBuilder().DataMover(\"other\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"add finalizer to du\",\n\t\t\tdu: dataUploadBuilder().Result(),\n\t\t\texpected: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"add finalizer to du failed\",\n\t\t\tdu: dataUploadBuilder().Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\texpectedErr: \"error updating dataupload with error velero/dataupload-1: Update error\",\n\t\t},\n\t\t{\n\t\t\tname: \"du is under deletion\",\n\t\t\tdu: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\tneedDelete: true,\n\t\t\texpected: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"du is under deletion but cancel failed\",\n\t\t\tdu: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\tneedDelete: true,\n\t\t\texpectedErr: \"error updating dataupload with error velero/dataupload-1: Update error\",\n\t\t},\n\t\t{\n\t\t\tname: \"du is under deletion and in terminal state\",\n\t\t\tdu: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Phase(velerov2alpha1api.DataUploadPhaseFailed).Result(),\n\t\t\tsportTime: &metav1.Time{Time: time.Now()},\n\t\t\tneedDelete: true,\n\t\t\texpectDeleted: true,\n\t\t},\n\t\t{\n\t\t\tname: \"du is under deletion and in terminal state, but remove finalizer failed\",\n\t\t\tdu: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Phase(velerov2alpha1api.DataUploadPhaseFailed).Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\tneedDelete: true,\n\t\t\texpectedErr: \"error updating dataupload with error velero/dataupload-1: Update error\",\n\t\t},\n\t\t{\n\t\t\tname: \"delay cancel negative for others\",\n\t\t\tdu: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Phase(velerov2alpha1api.DataUploadPhasePrepared).Result(),\n\t\t\tsportTime: &metav1.Time{Time: time.Now()},\n\t\t\texpectCancelRecord: true,\n\t\t},\n\t\t{\n\t\t\tname: \"delay cancel negative for inProgress\",\n\t\t\tdu: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Phase(velerov2alpha1api.DataUploadPhaseInProgress).Result(),\n\t\t\tsportTime: &metav1.Time{Time: time.Now().Add(-time.Minute * 58)},\n\t\t\texpectCancelRecord: true,\n\t\t},\n\t\t{\n\t\t\tname: \"delay cancel affirmative for others\",\n\t\t\tdu: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Phase(velerov2alpha1api.DataUploadPhasePrepared).Result(),\n\t\t\tsportTime: &metav1.Time{Time: time.Now().Add(-time.Minute * 5)},\n\t\t\texpected: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Phase(velerov2alpha1api.DataUploadPhaseCanceled).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"delay cancel affirmative for inProgress\",\n\t\t\tdu: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Phase(velerov2alpha1api.DataUploadPhaseInProgress).Result(),\n\t\t\tsportTime: &metav1.Time{Time: time.Now().Add(-time.Hour)},\n\t\t\texpected: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Phase(velerov2alpha1api.DataUploadPhaseCanceled).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"delay cancel failed\",\n\t\t\tdu: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Phase(velerov2alpha1api.DataUploadPhaseInProgress).Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\tsportTime: &metav1.Time{Time: time.Now().Add(-time.Hour)},\n\t\t\texpected: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Phase(velerov2alpha1api.DataUploadPhaseInProgress).Result(),\n\t\t\texpectCancelRecord: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Unknown data upload status\",\n\t\t\tdu: dataUploadBuilder().Phase(\"Unknown\").Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Unknown type of snapshot exposer is not initialized\",\n\t\t\tdu: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).SnapshotType(\"unknown type\").Result(),\n\t\t\texpected: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Phase(velerov2alpha1api.DataUploadPhaseFailed).Result(),\n\t\t\texpectedErr: \"unknown type type of snapshot exposer is not exist\",\n\t\t},\n\t\t{\n\t\t\tname: \"du is cancel on new\",\n\t\t\tdu: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Result(),\n\t\t\texpectCancelRecord: true,\n\t\t\texpected: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Phase(velerov2alpha1api.DataUploadPhaseCanceled).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"new du but constrained\",\n\t\t\tdu: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\tconstrained: true,\n\t\t\texpected: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\texpectedResult: &ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5},\n\t\t},\n\t\t{\n\t\t\tname: \"new du but accept failed\",\n\t\t\tdu: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\tneedExclusiveUpdateError: errors.New(\"exclusive-update-error\"),\n\t\t\texpected: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\texpectedErr: \"error accepting the data upload dataupload-1: exclusive-update-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"du is accepted but setup expose param failed on getting PVC\",\n\t\t\tdu: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\texpected: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Phase(velerov2alpha1api.DataUploadPhaseFailed).Message(\"failed to set exposer parameters\").Result(),\n\t\t\texpectedErr: \"failed to get PVC fake-ns/test-pvc: persistentvolumeclaims \\\"test-pvc\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"du expose failed\",\n\t\t\tdu: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).SnapshotType(fakeSnapshotType).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"fake-ns\", \"test-pvc\").Result(),\n\t\t\texposeErr: errors.New(\"fake-expose-error\"),\n\t\t\texpected: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Phase(velerov2alpha1api.DataUploadPhaseFailed).Message(\"error exposing snapshot\").Result(),\n\t\t\texpectedErr: \"fake-expose-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"du succeeds for accepted\",\n\t\t\tdu: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).SnapshotType(fakeSnapshotType).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"fake-ns\", \"test-pvc\").Result(),\n\t\t\texpected: dataUploadBuilder().Finalizers([]string{DataUploadDownloadFinalizer}).Phase(velerov2alpha1api.DataUploadPhaseAccepted).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"prepare timeout on accepted\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseAccepted).Finalizers([]string{DataUploadDownloadFinalizer}).AcceptedTimestamp(&metav1.Time{Time: time.Now().Add(-time.Minute * 30)}).Result(),\n\t\t\texpected: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseFailed).Finalizers([]string{DataUploadDownloadFinalizer}).Phase(velerov2alpha1api.DataUploadPhaseFailed).Message(\"timeout on preparing data upload\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"peek error on accepted\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseAccepted).SnapshotType(fakeSnapshotType).Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\tpeekErr: errors.New(\"fake-peak-error\"),\n\t\t\texpected: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseCanceled).Finalizers([]string{DataUploadDownloadFinalizer}).Phase(velerov2alpha1api.DataUploadPhaseCanceled).Message(\"found a du velero/dataupload-1 with expose error: fake-peak-error. mark it as cancel\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"cancel on prepared\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhasePrepared).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Cancel(true).Result(),\n\t\t\texpected: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseCanceled).Finalizers([]string{DataUploadDownloadFinalizer}).Cancel(true).Phase(velerov2alpha1api.DataUploadPhaseCanceled).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Failed to get snapshot expose on prepared\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhasePrepared).SnapshotType(fakeSnapshotType).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\tgetExposeErr: errors.New(\"fake-get-error\"),\n\t\t\texpected: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseFailed).Finalizers([]string{DataUploadDownloadFinalizer}).Message(\"exposed snapshot is not ready: fake-get-error\").Result(),\n\t\t\texpectedErr: \"fake-get-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"Get nil restore expose on prepared\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhasePrepared).SnapshotType(fakeSnapshotType).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\tgetExposeNil: true,\n\t\t\texpected: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseFailed).Finalizers([]string{DataUploadDownloadFinalizer}).Message(\"exposed snapshot is not ready\").Result(),\n\t\t\texpectedErr: \"no expose result is available for the current node\",\n\t\t},\n\t\t{\n\t\t\tname: \"Dataupload should fail if expose returns ambiguous nodeOS\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhasePrepared).SnapshotType(fakeSnapshotType).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\tambiguousNodeOS: true,\n\t\t\texpected: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseFailed).Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\texpectedErr: \"unsupported ambiguous node OS\",\n\t\t},\n\t\t{\n\t\t\tname: \"Error in data path is concurrent limited\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhasePrepared).SnapshotType(fakeSnapshotType).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\tdataMgr: datapath.NewManager(0),\n\t\t\texpectedResult: &ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5},\n\t\t},\n\t\t{\n\t\t\tname: \"data path init error\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhasePrepared).SnapshotType(fakeSnapshotType).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\tfsBRInitErr: errors.New(\"fake-data-path-init-error\"),\n\t\t\texpected: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseFailed).Finalizers([]string{DataUploadDownloadFinalizer}).Message(\"error initializing data path\").Result(),\n\t\t\texpectedErr: \"error initializing asyncBR: fake-data-path-init-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"Unable to update status to in progress for data upload\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhasePrepared).SnapshotType(fakeSnapshotType).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\texpected: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhasePrepared).Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\texpectedResult: &ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5},\n\t\t},\n\t\t{\n\t\t\tname: \"data path start error\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhasePrepared).SnapshotType(fakeSnapshotType).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\tfsBRStartErr: errors.New(\"fake-data-path-start-error\"),\n\t\t\texpected: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseFailed).Finalizers([]string{DataUploadDownloadFinalizer}).Message(\"error starting data path\").Result(),\n\t\t\texpectedErr: \"error starting async backup for pod , volume dataupload-1: fake-data-path-start-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"Prepare succeeds\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhasePrepared).SnapshotType(fakeSnapshotType).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\texpected: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseInProgress).Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\texpectDataPath: true,\n\t\t},\n\t\t{\n\t\t\tname: \"In progress du is not handled by the current node\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseInProgress).Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\texpected: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseInProgress).Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"In progress du is not set as cancel\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseInProgress).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\texpected: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseInProgress).Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Cancel data upload in progress with empty FSBR\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseInProgress).Cancel(true).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\texpected: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseCanceled).Cancel(true).Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Cancel data upload in progress and patch data upload error\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseInProgress).Cancel(true).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\tneedCreateFSBR: true,\n\t\t\texpected: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseInProgress).Cancel(true).Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\texpectedErr: \"error updating dataupload with error velero/dataupload-1: Update error\",\n\t\t\texpectCancelRecord: true,\n\t\t\texpectDataPath: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Cancel data upload in progress succeeds\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseInProgress).Cancel(true).Finalizers([]string{DataUploadDownloadFinalizer}).Node(\"test-node\").Result(),\n\t\t\tneedCreateFSBR: true,\n\t\t\texpected: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseCanceling).Cancel(true).Finalizers([]string{DataUploadDownloadFinalizer}).Result(),\n\t\t\texpectDataPath: true,\n\t\t\texpectCancelRecord: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tr, err := initDataUploaderReconciler(test.needErrs...)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tif !test.notCreateDU {\n\t\t\t\terr = r.client.Create(t.Context(), test.du)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif test.needDelete {\n\t\t\t\terr = r.client.Delete(t.Context(), test.du)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif test.pod != nil {\n\t\t\t\terr = r.client.Create(ctx, test.pod)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif test.pvc != nil {\n\t\t\t\terr = r.client.Create(ctx, test.pvc)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif test.dataMgr != nil {\n\t\t\t\tr.dataPathMgr = test.dataMgr\n\t\t\t} else {\n\t\t\t\tr.dataPathMgr = datapath.NewManager(1)\n\t\t\t}\n\n\t\t\tif test.sportTime != nil {\n\t\t\t\tr.cancelledDataUpload[test.du.Name] = test.sportTime.Time\n\t\t\t}\n\n\t\t\tif test.constrained {\n\t\t\t\tr.vgdpCounter = &exposer.VgdpCounter{}\n\t\t\t}\n\n\t\t\tif test.du.Spec.SnapshotType == fakeSnapshotType {\n\t\t\t\tr.snapshotExposerList = map[velerov2alpha1api.SnapshotType]exposer.SnapshotExposer{fakeSnapshotType: &fakeSnapshotExposer{r.client, r.Clock, test.ambiguousNodeOS, test.peekErr, test.exposeErr, test.getExposeErr, test.getExposeNil}}\n\t\t\t} else if test.du.Spec.SnapshotType == velerov2alpha1api.SnapshotTypeCSI {\n\t\t\t\tr.snapshotExposerList = map[velerov2alpha1api.SnapshotType]exposer.SnapshotExposer{velerov2alpha1api.SnapshotTypeCSI: exposer.NewCSISnapshotExposer(r.kubeClient, r.csiSnapshotClient, velerotest.NewLogger())}\n\t\t\t}\n\n\t\t\tfuncExclusiveUpdateDataUpload = exclusiveUpdateDataUpload\n\t\t\tif test.needExclusiveUpdateError != nil {\n\t\t\t\tfuncExclusiveUpdateDataUpload = func(context.Context, kbclient.Client, *velerov2alpha1api.DataUpload, func(*velerov2alpha1api.DataUpload)) (bool, error) {\n\t\t\t\t\treturn false, test.needExclusiveUpdateError\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tdatapath.MicroServiceBRWatcherCreator = func(kbclient.Client, kubernetes.Interface, manager.Manager, string, string, string, string, string, string, datapath.Callbacks, logrus.FieldLogger) datapath.AsyncBR {\n\t\t\t\treturn &fakeFSBR{\n\t\t\t\t\tkubeClient: r.client,\n\t\t\t\t\tclock: r.Clock,\n\t\t\t\t\tinitErr: test.fsBRInitErr,\n\t\t\t\t\tstartErr: test.fsBRStartErr,\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif test.needCreateFSBR {\n\t\t\t\tif fsBR := r.dataPathMgr.GetAsyncBR(test.du.Name); fsBR == nil {\n\t\t\t\t\t_, err := r.dataPathMgr.CreateMicroServiceBRWatcher(ctx, r.client, nil, nil, datapath.TaskTypeBackup, test.du.Name, velerov1api.DefaultNamespace, \"\", \"\", \"\", datapath.Callbacks{OnCancelled: r.OnDataUploadCancelled}, false, velerotest.NewLogger())\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tactualResult, err := r.Reconcile(ctx, ctrl.Request{\n\t\t\t\tNamespacedName: types.NamespacedName{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: test.du.Name,\n\t\t\t\t},\n\t\t\t})\n\n\t\t\tif test.expectedErr != \"\" {\n\t\t\t\trequire.EqualError(t, err, test.expectedErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif test.expectedResult != nil {\n\t\t\t\tassert.Equal(t, test.expectedResult.Requeue, actualResult.Requeue)\n\t\t\t\tassert.Equal(t, test.expectedResult.RequeueAfter, actualResult.RequeueAfter)\n\t\t\t}\n\n\t\t\tdu := velerov2alpha1api.DataUpload{}\n\t\t\terr = r.client.Get(ctx, kbclient.ObjectKey{\n\t\t\t\tName: test.du.Name,\n\t\t\t\tNamespace: test.du.Namespace,\n\t\t\t}, &du)\n\n\t\t\tif test.expected != nil || test.expectDeleted {\n\t\t\t\tif test.expectDeleted {\n\t\t\t\t\tassert.True(t, apierrors.IsNotFound(err))\n\t\t\t\t} else {\n\t\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\t\tassert.Equal(t, test.expected.Status.Phase, du.Status.Phase)\n\t\t\t\t\tassert.Contains(t, du.Status.Message, test.expected.Status.Message)\n\t\t\t\t\tassert.Equal(t, du.Finalizers, test.expected.Finalizers)\n\t\t\t\t\tassert.Equal(t, du.Spec.Cancel, test.expected.Spec.Cancel)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif !test.expectDataPath {\n\t\t\t\tassert.Nil(t, r.dataPathMgr.GetAsyncBR(test.du.Name))\n\t\t\t} else {\n\t\t\t\tassert.NotNil(t, r.dataPathMgr.GetAsyncBR(test.du.Name))\n\t\t\t}\n\n\t\t\tif test.expectCancelRecord {\n\t\t\t\tassert.Contains(t, r.cancelledDataUpload, test.du.Name)\n\t\t\t} else {\n\t\t\t\tassert.Empty(t, r.cancelledDataUpload)\n\t\t\t}\n\n\t\t\tif isDataUploadInFinalState(&du) || du.Status.Phase == velerov2alpha1api.DataUploadPhaseInProgress {\n\t\t\t\tassert.NotContains(t, du.Labels, exposer.ExposeOnGoingLabel)\n\t\t\t} else if du.Status.Phase == velerov2alpha1api.DataUploadPhaseAccepted {\n\t\t\t\tassert.Contains(t, du.Labels, exposer.ExposeOnGoingLabel)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestOnDataUploadCancelled(t *testing.T) {\n\tctx := t.Context()\n\tr, err := initDataUploaderReconciler()\n\trequire.NoError(t, err)\n\t// Create a DataUpload object\n\tdu := dataUploadBuilder().Result()\n\tnamespace := du.Namespace\n\tduName := du.Name\n\t// Add the DataUpload object to the fake client\n\trequire.NoError(t, r.client.Create(ctx, du))\n\n\tr.OnDataUploadCancelled(ctx, namespace, duName)\n\tupdatedDu := &velerov2alpha1api.DataUpload{}\n\trequire.NoError(t, r.client.Get(ctx, types.NamespacedName{Name: duName, Namespace: namespace}, updatedDu))\n\tassert.Equal(t, velerov2alpha1api.DataUploadPhaseCanceled, updatedDu.Status.Phase)\n\tassert.False(t, updatedDu.Status.CompletionTimestamp.IsZero())\n\tassert.False(t, updatedDu.Status.StartTimestamp.IsZero())\n}\n\nfunc TestOnDataUploadProgress(t *testing.T) {\n\ttotalBytes := int64(1024)\n\tbytesDone := int64(512)\n\ttests := []struct {\n\t\tname string\n\t\tdu *velerov2alpha1api.DataUpload\n\t\tprogress uploader.Progress\n\t\tneedErrs []bool\n\t}{\n\t\t{\n\t\t\tname: \"patch in progress phase success\",\n\t\t\tdu: dataUploadBuilder().Result(),\n\t\t\tprogress: uploader.Progress{\n\t\t\t\tTotalBytes: totalBytes,\n\t\t\t\tBytesDone: bytesDone,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"failed to get dataupload\",\n\t\t\tdu: dataUploadBuilder().Result(),\n\t\t\tneedErrs: []bool{true, false, false, false},\n\t\t},\n\t\t{\n\t\t\tname: \"failed to patch dataupload\",\n\t\t\tdu: dataUploadBuilder().Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\n\t\t\tr, err := initDataUploaderReconciler(test.needErrs...)\n\t\t\trequire.NoError(t, err)\n\t\t\tdefer func() {\n\t\t\t\tr.client.Delete(ctx, test.du, &kbclient.DeleteOptions{})\n\t\t\t}()\n\t\t\t// Create a DataUpload object\n\t\t\tdu := dataUploadBuilder().Result()\n\t\t\tnamespace := du.Namespace\n\t\t\tduName := du.Name\n\t\t\t// Add the DataUpload object to the fake client\n\t\t\trequire.NoError(t, r.client.Create(t.Context(), du))\n\n\t\t\t// Create a Progress object\n\t\t\tprogress := &uploader.Progress{\n\t\t\t\tTotalBytes: totalBytes,\n\t\t\t\tBytesDone: bytesDone,\n\t\t\t}\n\n\t\t\t// Call the OnDataUploadProgress function\n\t\t\tr.OnDataUploadProgress(ctx, namespace, duName, progress)\n\t\t\tif len(test.needErrs) != 0 && !test.needErrs[0] {\n\t\t\t\t// Get the updated DataUpload object from the fake client\n\t\t\t\tupdatedDu := &velerov2alpha1api.DataUpload{}\n\t\t\t\trequire.NoError(t, r.client.Get(ctx, types.NamespacedName{Name: duName, Namespace: namespace}, updatedDu))\n\t\t\t\t// Assert that the DataUpload object has been updated with the progress\n\t\t\t\tassert.Equal(t, test.progress.TotalBytes, updatedDu.Status.Progress.TotalBytes)\n\t\t\t\tassert.Equal(t, test.progress.BytesDone, updatedDu.Status.Progress.BytesDone)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestOnDataUploadFailed(t *testing.T) {\n\tctx := t.Context()\n\tr, err := initDataUploaderReconciler()\n\trequire.NoError(t, err)\n\n\t// Create a DataUpload object\n\tdu := dataUploadBuilder().Result()\n\tnamespace := du.Namespace\n\tduName := du.Name\n\t// Add the DataUpload object to the fake client\n\trequire.NoError(t, r.client.Create(ctx, du))\n\tr.snapshotExposerList = map[velerov2alpha1api.SnapshotType]exposer.SnapshotExposer{velerov2alpha1api.SnapshotTypeCSI: exposer.NewCSISnapshotExposer(r.kubeClient, r.csiSnapshotClient, velerotest.NewLogger())}\n\tr.OnDataUploadFailed(ctx, namespace, duName, fmt.Errorf(\"Failed to handle %v\", duName))\n\tupdatedDu := &velerov2alpha1api.DataUpload{}\n\trequire.NoError(t, r.client.Get(ctx, types.NamespacedName{Name: duName, Namespace: namespace}, updatedDu))\n\tassert.Equal(t, velerov2alpha1api.DataUploadPhaseFailed, updatedDu.Status.Phase)\n\tassert.False(t, updatedDu.Status.CompletionTimestamp.IsZero())\n\tassert.False(t, updatedDu.Status.StartTimestamp.IsZero())\n}\n\nfunc TestOnDataUploadCompleted(t *testing.T) {\n\tctx := t.Context()\n\tr, err := initDataUploaderReconciler()\n\trequire.NoError(t, err)\n\t// Create a DataUpload object\n\tdu := dataUploadBuilder().Result()\n\tnamespace := du.Namespace\n\tduName := du.Name\n\t// Add the DataUpload object to the fake client\n\trequire.NoError(t, r.client.Create(ctx, du))\n\tr.snapshotExposerList = map[velerov2alpha1api.SnapshotType]exposer.SnapshotExposer{velerov2alpha1api.SnapshotTypeCSI: exposer.NewCSISnapshotExposer(r.kubeClient, r.csiSnapshotClient, velerotest.NewLogger())}\n\tr.OnDataUploadCompleted(ctx, namespace, duName, datapath.Result{\n\t\tBackup: datapath.BackupResult{\n\t\t\tSnapshotID: \"fake-id\",\n\t\t\tSource: datapath.AccessPoint{\n\t\t\t\tByPath: \"fake-path\",\n\t\t\t},\n\t\t},\n\t})\n\tupdatedDu := &velerov2alpha1api.DataUpload{}\n\trequire.NoError(t, r.client.Get(ctx, types.NamespacedName{Name: duName, Namespace: namespace}, updatedDu))\n\tassert.Equal(t, velerov2alpha1api.DataUploadPhaseCompleted, updatedDu.Status.Phase)\n\tassert.False(t, updatedDu.Status.CompletionTimestamp.IsZero())\n\tassert.Equal(t, \"fake-id\", updatedDu.Status.SnapshotID)\n\tassert.Equal(t, \"fake-path\", updatedDu.Status.Path)\n}\n\nfunc TestFindDataUploadForPod(t *testing.T) {\n\tr, err := initDataUploaderReconciler()\n\trequire.NoError(t, err)\n\ttests := []struct {\n\t\tname string\n\t\tdu *velerov2alpha1api.DataUpload\n\t\tpod *corev1api.Pod\n\t\tcheckFunc func(*velerov2alpha1api.DataUpload, []reconcile.Request)\n\t}{\n\t\t{\n\t\t\tname: \"find dataUpload for pod\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseAccepted).Result(),\n\t\t\tpod: builder.ForPod(velerov1api.DefaultNamespace, dataUploadName).Labels(map[string]string{velerov1api.DataUploadLabel: dataUploadName}).Status(corev1api.PodStatus{Phase: corev1api.PodRunning}).Result(),\n\t\t\tcheckFunc: func(du *velerov2alpha1api.DataUpload, requests []reconcile.Request) {\n\t\t\t\t// Assert that the function returns a single request\n\t\t\t\tassert.Len(t, requests, 1)\n\t\t\t\t// Assert that the request contains the correct namespaced name\n\t\t\t\tassert.Equal(t, du.Namespace, requests[0].Namespace)\n\t\t\t\tassert.Equal(t, du.Name, requests[0].Name)\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"no selected label found for pod\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseAccepted).Result(),\n\t\t\tpod: builder.ForPod(velerov1api.DefaultNamespace, dataUploadName).Result(),\n\t\t\tcheckFunc: func(du *velerov2alpha1api.DataUpload, requests []reconcile.Request) {\n\t\t\t\t// Assert that the function returns a single request\n\t\t\t\tassert.Empty(t, requests)\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"no matched pod\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseAccepted).Result(),\n\t\t\tpod: builder.ForPod(velerov1api.DefaultNamespace, dataUploadName).Labels(map[string]string{velerov1api.DataUploadLabel: \"non-existing-dataupload\"}).Result(),\n\t\t\tcheckFunc: func(du *velerov2alpha1api.DataUpload, requests []reconcile.Request) {\n\t\t\t\tassert.Empty(t, requests)\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"dataUpload not accepte\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseInProgress).Result(),\n\t\t\tpod: builder.ForPod(velerov1api.DefaultNamespace, dataUploadName).Labels(map[string]string{velerov1api.DataUploadLabel: dataUploadName}).Result(),\n\t\t\tcheckFunc: func(du *velerov2alpha1api.DataUpload, requests []reconcile.Request) {\n\t\t\t\tassert.Empty(t, requests)\n\t\t\t},\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tctx := t.Context()\n\t\tassert.NoError(t, r.client.Create(ctx, test.pod))\n\t\tassert.NoError(t, r.client.Create(ctx, test.du))\n\t\t// Call the findDataUploadForPod function\n\t\trequests := r.findDataUploadForPod(t.Context(), test.pod)\n\t\ttest.checkFunc(test.du, requests)\n\t\tr.client.Delete(ctx, test.du, &kbclient.DeleteOptions{})\n\t\tif test.pod != nil {\n\t\t\tr.client.Delete(ctx, test.pod, &kbclient.DeleteOptions{})\n\t\t}\n\t}\n}\n\ntype fakeAPIStatus struct {\n\treason metav1.StatusReason\n}\n\nfunc (f *fakeAPIStatus) Status() metav1.Status {\n\treturn metav1.Status{\n\t\tReason: f.reason,\n\t}\n}\n\nfunc (f *fakeAPIStatus) Error() string {\n\treturn string(f.reason)\n}\n\nfunc TestAcceptDataUpload(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdu *velerov2alpha1api.DataUpload\n\t\tneedErrs []error\n\t\tsucceeded bool\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"update fail\",\n\t\t\tdu: dataUploadBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, fmt.Errorf(\"fake-update-error\"), nil},\n\t\t\texpectedErr: \"fake-update-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"accepted by others\",\n\t\t\tdu: dataUploadBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, &fakeAPIStatus{metav1.StatusReasonConflict}, nil},\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tdu: dataUploadBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, nil, nil},\n\t\t\tsucceeded: true,\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tctx := t.Context()\n\t\tr, err := initDataUploaderReconcilerWithError(test.needErrs...)\n\t\trequire.NoError(t, err)\n\n\t\terr = r.client.Create(ctx, test.du)\n\t\trequire.NoError(t, err)\n\n\t\tsucceeded, err := r.acceptDataUpload(ctx, test.du)\n\t\tassert.Equal(t, test.succeeded, succeeded)\n\t\tif test.expectedErr == \"\" {\n\t\t\tassert.NoError(t, err)\n\t\t} else {\n\t\t\tassert.EqualError(t, err, test.expectedErr)\n\t\t}\n\t}\n}\n\nfunc TestOnDuPrepareTimeout(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdu *velerov2alpha1api.DataUpload\n\t\tneedErrs []error\n\t\texpected *velerov2alpha1api.DataUpload\n\t}{\n\t\t{\n\t\t\tname: \"update fail\",\n\t\t\tdu: dataUploadBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, fmt.Errorf(\"fake-update-error\"), nil},\n\t\t\texpected: dataUploadBuilder().Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"update interrupted\",\n\t\t\tdu: dataUploadBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, &fakeAPIStatus{metav1.StatusReasonConflict}, nil},\n\t\t\texpected: dataUploadBuilder().Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tdu: dataUploadBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, nil, nil},\n\t\t\texpected: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseFailed).Result(),\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tctx := t.Context()\n\t\tr, err := initDataUploaderReconcilerWithError(test.needErrs...)\n\t\trequire.NoError(t, err)\n\n\t\terr = r.client.Create(ctx, test.du)\n\t\trequire.NoError(t, err)\n\n\t\tr.onPrepareTimeout(ctx, test.du)\n\n\t\tdu := velerov2alpha1api.DataUpload{}\n\t\t_ = r.client.Get(ctx, kbclient.ObjectKey{\n\t\t\tName: test.du.Name,\n\t\t\tNamespace: test.du.Namespace,\n\t\t}, &du)\n\n\t\tassert.Equal(t, test.expected.Status.Phase, du.Status.Phase)\n\t}\n}\n\nfunc TestTryCancelDataUpload(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdd *velerov2alpha1api.DataUpload\n\t\tneedErrs []error\n\t\tsucceeded bool\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"update fail\",\n\t\t\tdd: dataUploadBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, fmt.Errorf(\"fake-update-error\"), nil},\n\t\t},\n\t\t{\n\t\t\tname: \"cancel by others\",\n\t\t\tdd: dataUploadBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, &fakeAPIStatus{metav1.StatusReasonConflict}, nil},\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tdd: dataUploadBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, nil, nil},\n\t\t\tsucceeded: true,\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tctx := t.Context()\n\t\tr, err := initDataUploaderReconcilerWithError(test.needErrs...)\n\t\trequire.NoError(t, err)\n\n\t\terr = r.client.Create(ctx, test.dd)\n\t\trequire.NoError(t, err)\n\n\t\tr.tryCancelDataUpload(ctx, test.dd, \"\")\n\n\t\tif test.expectedErr == \"\" {\n\t\t\tassert.NoError(t, err)\n\t\t} else {\n\t\t\tassert.EqualError(t, err, test.expectedErr)\n\t\t}\n\t}\n}\n\nfunc TestUpdateDataUploadWithRetry(t *testing.T) {\n\tnamespacedName := types.NamespacedName{\n\t\tName: dataUploadName,\n\t\tNamespace: \"velero\",\n\t}\n\n\t// Define test cases\n\ttestCases := []struct {\n\t\tName string\n\t\tneedErrs []bool\n\t\tnoChange bool\n\t\tExpectErr bool\n\t}{\n\t\t{\n\t\t\tName: \"SuccessOnFirstAttempt\",\n\t\t},\n\t\t{\n\t\t\tName: \"Error get\",\n\t\t\tneedErrs: []bool{true, false, false, false, false},\n\t\t\tExpectErr: true,\n\t\t},\n\t\t{\n\t\t\tName: \"Error update\",\n\t\t\tneedErrs: []bool{false, false, true, false, false},\n\t\t\tExpectErr: true,\n\t\t},\n\t\t{\n\t\t\tName: \"no change\",\n\t\t\tnoChange: true,\n\t\t\tneedErrs: []bool{false, false, true, false, false},\n\t\t},\n\t\t{\n\t\t\tName: \"Conflict with error timeout\",\n\t\t\tneedErrs: []bool{false, false, false, false, true},\n\t\t\tExpectErr: true,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.Name, func(t *testing.T) {\n\t\t\tctx, cancelFunc := context.WithTimeout(t.Context(), time.Second*5)\n\t\t\tdefer cancelFunc()\n\t\t\tr, err := initDataUploaderReconciler(tc.needErrs...)\n\t\t\trequire.NoError(t, err)\n\t\t\terr = r.client.Create(ctx, dataUploadBuilder().Result())\n\t\t\trequire.NoError(t, err)\n\t\t\tupdateFunc := func(dataDownload *velerov2alpha1api.DataUpload) bool {\n\t\t\t\tif tc.noChange {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tdataDownload.Spec.Cancel = true\n\t\t\t\treturn true\n\t\t\t}\n\t\t\terr = UpdateDataUploadWithRetry(ctx, r.client, namespacedName, velerotest.NewLogger().WithField(\"name\", tc.Name), updateFunc)\n\t\t\tif tc.ExpectErr {\n\t\t\t\tassert.Error(t, err)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\ntype duResumeTestHelper struct {\n\tresumeErr error\n\tgetExposeErr error\n\texposeResult *exposer.ExposeResult\n\tasyncBR datapath.AsyncBR\n}\n\nfunc (dt *duResumeTestHelper) resumeCancellableDataPath(_ *DataUploadReconciler, _ context.Context, _ *velerov2alpha1api.DataUpload, _ logrus.FieldLogger) error {\n\treturn dt.resumeErr\n}\n\nfunc (dt *duResumeTestHelper) Expose(context.Context, corev1api.ObjectReference, any) error {\n\treturn nil\n}\n\nfunc (dt *duResumeTestHelper) GetExposed(context.Context, corev1api.ObjectReference, time.Duration, any) (*exposer.ExposeResult, error) {\n\treturn dt.exposeResult, dt.getExposeErr\n}\n\nfunc (dt *duResumeTestHelper) PeekExposed(context.Context, corev1api.ObjectReference) error {\n\treturn nil\n}\n\nfunc (dt *duResumeTestHelper) DiagnoseExpose(context.Context, corev1api.ObjectReference) string {\n\treturn \"\"\n}\n\nfunc (dt *duResumeTestHelper) CleanUp(context.Context, corev1api.ObjectReference, string, string) {}\n\nfunc (dt *duResumeTestHelper) newMicroServiceBRWatcher(kbclient.Client, kubernetes.Interface, manager.Manager, string, string, string, string, string, string,\n\tdatapath.Callbacks, logrus.FieldLogger) datapath.AsyncBR {\n\treturn dt.asyncBR\n}\n\nfunc TestAttemptDataUploadResume(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdataUploads []velerov2alpha1api.DataUpload\n\t\tdu *velerov2alpha1api.DataUpload\n\t\tneedErrs []bool\n\t\tacceptedDataUploads []string\n\t\tprepareddDataUploads []string\n\t\tcancelledDataUploads []string\n\t\tinProgressDataUploads []string\n\t\tresumeErr error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"Other DataUpload\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhasePrepared).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"InProgress DataUpload, not the current node\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseInProgress).Result(),\n\t\t\tinProgressDataUploads: []string{dataUploadName},\n\t\t},\n\t\t{\n\t\t\tname: \"InProgress DataUpload, resume error and update error\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseInProgress).Node(\"node-1\").Result(),\n\t\t\tneedErrs: []bool{false, false, true, false, false, false},\n\t\t\tresumeErr: errors.New(\"fake-resume-error\"),\n\t\t\tinProgressDataUploads: []string{dataUploadName},\n\t\t},\n\t\t{\n\t\t\tname: \"InProgress DataUpload, resume error and update succeed\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseInProgress).Node(\"node-1\").Result(),\n\t\t\tresumeErr: errors.New(\"fake-resume-error\"),\n\t\t\tcancelledDataUploads: []string{dataUploadName},\n\t\t\tinProgressDataUploads: []string{dataUploadName},\n\t\t},\n\t\t{\n\t\t\tname: \"InProgress DataUpload and resume succeed\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseInProgress).Node(\"node-1\").Result(),\n\t\t\tinProgressDataUploads: []string{dataUploadName},\n\t\t},\n\t\t{\n\t\t\tname: \"Error\",\n\t\t\tneedErrs: []bool{false, false, false, false, false, true},\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhasePrepared).Result(),\n\t\t\texpectedError: \"error to list datauploads: List error\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\t\t\tr, err := initDataUploaderReconciler(test.needErrs...)\n\t\t\tr.nodeName = \"node-1\"\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.NoError(t, r.client.Create(ctx, test.du))\n\n\t\t\tdt := &duResumeTestHelper{\n\t\t\t\tresumeErr: test.resumeErr,\n\t\t\t}\n\n\t\t\tfuncResumeCancellableDataBackup = dt.resumeCancellableDataPath\n\n\t\t\t// Run the test\n\t\t\terr = r.AttemptDataUploadResume(ctx, r.logger.WithField(\"name\", test.name), test.du.Namespace)\n\n\t\t\tif test.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.expectedError)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\n\t\t\t\t// Verify DataUploads marked as Canceled\n\t\t\t\tfor _, duName := range test.cancelledDataUploads {\n\t\t\t\t\tdataUpload := &velerov2alpha1api.DataUpload{}\n\t\t\t\t\terr := r.client.Get(t.Context(), types.NamespacedName{Namespace: \"velero\", Name: duName}, dataUpload)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tassert.True(t, dataUpload.Spec.Cancel)\n\t\t\t\t}\n\t\t\t\t// Verify DataUploads marked as Accepted\n\t\t\t\tfor _, duName := range test.acceptedDataUploads {\n\t\t\t\t\tdataUpload := &velerov2alpha1api.DataUpload{}\n\t\t\t\t\terr := r.client.Get(t.Context(), types.NamespacedName{Namespace: \"velero\", Name: duName}, dataUpload)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tassert.Equal(t, velerov2alpha1api.DataUploadPhaseAccepted, dataUpload.Status.Phase)\n\t\t\t\t}\n\t\t\t\t// Verify DataUploads marked as Prepared\n\t\t\t\tfor _, duName := range test.prepareddDataUploads {\n\t\t\t\t\tdataUpload := &velerov2alpha1api.DataUpload{}\n\t\t\t\t\terr := r.client.Get(t.Context(), types.NamespacedName{Namespace: \"velero\", Name: duName}, dataUpload)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tassert.Equal(t, velerov2alpha1api.DataUploadPhasePrepared, dataUpload.Status.Phase)\n\t\t\t\t}\n\t\t\t\t// Verify DataUploads marked as InProgress\n\t\t\t\tfor _, duName := range test.inProgressDataUploads {\n\t\t\t\t\tdataUpload := &velerov2alpha1api.DataUpload{}\n\t\t\t\t\terr := r.client.Get(t.Context(), types.NamespacedName{Namespace: \"velero\", Name: duName}, dataUpload)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tassert.Equal(t, velerov2alpha1api.DataUploadPhaseInProgress, dataUpload.Status.Phase)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestResumeCancellableBackup(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdataUploads []velerov2alpha1api.DataUpload\n\t\tdu *velerov2alpha1api.DataUpload\n\t\tgetExposeErr error\n\t\texposeResult *exposer.ExposeResult\n\t\tcreateWatcherErr error\n\t\tinitWatcherErr error\n\t\tstartWatcherErr error\n\t\tmockInit bool\n\t\tmockStart bool\n\t\tmockClose bool\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"not find exposer\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseInProgress).SnapshotType(\"\").Result(),\n\t\t\texpectedError: fmt.Sprintf(\"error to find exposer for du %s\", dataUploadName),\n\t\t},\n\t\t{\n\t\t\tname: \"get expose failed\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseInProgress).SnapshotType(velerov2alpha1api.SnapshotTypeCSI).Result(),\n\t\t\tgetExposeErr: errors.New(\"fake-expose-error\"),\n\t\t\texpectedError: fmt.Sprintf(\"error to get exposed snapshot for du %s: fake-expose-error\", dataUploadName),\n\t\t},\n\t\t{\n\t\t\tname: \"no expose\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseAccepted).Node(\"node-1\").Result(),\n\t\t\texpectedError: fmt.Sprintf(\"expose info missed for du %s\", dataUploadName),\n\t\t},\n\t\t{\n\t\t\tname: \"watcher init error\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseAccepted).Node(\"node-1\").Result(),\n\t\t\texposeResult: &exposer.ExposeResult{\n\t\t\t\tByPod: exposer.ExposeByPod{\n\t\t\t\t\tHostingPod: &corev1api.Pod{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tmockInit: true,\n\t\t\tmockClose: true,\n\t\t\tinitWatcherErr: errors.New(\"fake-init-watcher-error\"),\n\t\t\texpectedError: fmt.Sprintf(\"error to init asyncBR watcher for du %s: fake-init-watcher-error\", dataUploadName),\n\t\t},\n\t\t{\n\t\t\tname: \"start watcher error\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseAccepted).Node(\"node-1\").Result(),\n\t\t\texposeResult: &exposer.ExposeResult{\n\t\t\t\tByPod: exposer.ExposeByPod{\n\t\t\t\t\tHostingPod: &corev1api.Pod{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tmockInit: true,\n\t\t\tmockStart: true,\n\t\t\tmockClose: true,\n\t\t\tstartWatcherErr: errors.New(\"fake-start-watcher-error\"),\n\t\t\texpectedError: fmt.Sprintf(\"error to resume asyncBR watcher for du %s: fake-start-watcher-error\", dataUploadName),\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tdu: dataUploadBuilder().Phase(velerov2alpha1api.DataUploadPhaseAccepted).Node(\"node-1\").Result(),\n\t\t\texposeResult: &exposer.ExposeResult{\n\t\t\t\tByPod: exposer.ExposeByPod{\n\t\t\t\t\tHostingPod: &corev1api.Pod{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tmockInit: true,\n\t\t\tmockStart: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\t\t\tr, err := initDataUploaderReconciler()\n\t\t\tr.nodeName = \"node-1\"\n\t\t\trequire.NoError(t, err)\n\n\t\t\tmockAsyncBR := datapathmocks.NewAsyncBR(t)\n\n\t\t\tif test.mockInit {\n\t\t\t\tmockAsyncBR.On(\"Init\", mock.Anything, mock.Anything).Return(test.initWatcherErr)\n\t\t\t}\n\n\t\t\tif test.mockStart {\n\t\t\t\tmockAsyncBR.On(\"StartBackup\", mock.Anything, mock.Anything, mock.Anything).Return(test.startWatcherErr)\n\t\t\t}\n\n\t\t\tif test.mockClose {\n\t\t\t\tmockAsyncBR.On(\"Close\", mock.Anything).Return()\n\t\t\t}\n\n\t\t\tdt := &duResumeTestHelper{\n\t\t\t\tgetExposeErr: test.getExposeErr,\n\t\t\t\texposeResult: test.exposeResult,\n\t\t\t\tasyncBR: mockAsyncBR,\n\t\t\t}\n\n\t\t\tr.snapshotExposerList[velerov2alpha1api.SnapshotTypeCSI] = dt\n\n\t\t\tdatapath.MicroServiceBRWatcherCreator = dt.newMicroServiceBRWatcher\n\n\t\t\terr = r.resumeCancellableDataPath(ctx, test.du, velerotest.NewLogger())\n\t\t\tif test.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.expectedError)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestDataUploadSetupExposeParam(t *testing.T) {\n\t// Common objects for all cases\n\tfileMode := corev1api.PersistentVolumeFilesystem\n\tnode := builder.ForNode(\"worker-1\").Labels(map[string]string{kube.NodeOSLabel: kube.NodeOSLinux}).Result()\n\n\tpvc := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"app-ns\",\n\t\t\tName: \"test-pvc\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tVolumeName: \"test-pv\",\n\t\t\tVolumeMode: &fileMode,\n\t\t\tResources: corev1api.VolumeResourceRequirements{\n\t\t\t\tRequests: corev1api.ResourceList{\n\t\t\t\t\tcorev1api.ResourceStorage: resource.MustParse(\"10Gi\"),\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tpv := &corev1api.PersistentVolume{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-pv\",\n\t\t},\n\t}\n\n\tbaseDataUpload := dataUploadBuilder().Result()\n\tbaseDataUpload.Spec.SourceNamespace = \"app-ns\"\n\tbaseDataUpload.Spec.SourcePVC = \"test-pvc\"\n\tbaseDataUpload.Namespace = velerov1api.DefaultNamespace\n\tbaseDataUpload.Spec.OperationTimeout = metav1.Duration{Duration: time.Minute * 10}\n\n\ttype args struct {\n\t\tcustomLabels map[string]string\n\t\tcustomAnnotations map[string]string\n\t}\n\ttype want struct {\n\t\tlabels map[string]string\n\t\tannotations map[string]string\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\targs args\n\t\twant want\n\t}{\n\t\t{\n\t\t\tname: \"label has customize values\",\n\t\t\targs: args{\n\t\t\t\tcustomLabels: map[string]string{\"custom-label\": \"label-value\"},\n\t\t\t\tcustomAnnotations: nil,\n\t\t\t},\n\t\t\twant: want{\n\t\t\t\tlabels: map[string]string{\n\t\t\t\t\tvelerov1api.DataUploadLabel: baseDataUpload.Name,\n\t\t\t\t\t\"custom-label\": \"label-value\",\n\t\t\t\t},\n\t\t\t\tannotations: map[string]string{},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"label has no customize values\",\n\t\t\targs: args{\n\t\t\t\tcustomLabels: nil,\n\t\t\t\tcustomAnnotations: nil,\n\t\t\t},\n\t\t\twant: want{\n\t\t\t\tlabels: map[string]string{velerov1api.DataUploadLabel: baseDataUpload.Name},\n\t\t\t\tannotations: map[string]string{},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"annotation has customize values\",\n\t\t\targs: args{\n\t\t\t\tcustomLabels: nil,\n\t\t\t\tcustomAnnotations: map[string]string{\"custom-annotation\": \"annotation-value\"},\n\t\t\t},\n\t\t\twant: want{\n\t\t\t\tlabels: map[string]string{velerov1api.DataUploadLabel: baseDataUpload.Name},\n\t\t\t\tannotations: map[string]string{\"custom-annotation\": \"annotation-value\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"both label and annotation have customize values\",\n\t\t\targs: args{\n\t\t\t\tcustomLabels: map[string]string{\"custom-label\": \"label-value\"},\n\t\t\t\tcustomAnnotations: map[string]string{\"custom-annotation\": \"annotation-value\"},\n\t\t\t},\n\t\t\twant: want{\n\t\t\t\tlabels: map[string]string{\n\t\t\t\t\tvelerov1api.DataUploadLabel: baseDataUpload.Name,\n\t\t\t\t\t\"custom-label\": \"label-value\",\n\t\t\t\t},\n\t\t\t\tannotations: map[string]string{\"custom-annotation\": \"annotation-value\"},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\t// Fake clients per case\n\t\t\tfakeCRClient := velerotest.NewFakeControllerRuntimeClient(t, pvc, pv, node, baseDataUpload.DeepCopy())\n\t\t\tfakeKubeClient := clientgofake.NewSimpleClientset(node)\n\n\t\t\t// Reconciler config per case\n\t\t\tpreparingTimeout := time.Minute * 3\n\t\t\tpodRes := corev1api.ResourceRequirements{}\n\t\t\tr := NewDataUploadReconciler(\n\t\t\t\tfakeCRClient,\n\t\t\t\tnil,\n\t\t\t\tfakeKubeClient,\n\t\t\t\tnil, // snapshotClient (unused in setupExposeParam)\n\t\t\t\tdatapath.NewManager(1),\n\t\t\t\tnil, // dataPathMgr\n\t\t\t\tnil, // exposer (unused in setupExposeParam)\n\t\t\t\tmap[string]velerotypes.BackupPVC{},\n\t\t\t\tpodRes,\n\t\t\t\ttestclocks.NewFakeClock(time.Now()),\n\t\t\t\t\"test-node\",\n\t\t\t\tpreparingTimeout,\n\t\t\t\tvelerotest.NewLogger(),\n\t\t\t\tmetrics.NewServerMetrics(),\n\t\t\t\t\"upload-priority\",\n\t\t\t\ttt.args.customLabels,\n\t\t\t\ttt.args.customAnnotations,\n\t\t\t)\n\n\t\t\t// Act\n\t\t\tgot, err := r.setupExposeParam(baseDataUpload)\n\n\t\t\t// Assert no error\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.NotNil(t, got)\n\n\t\t\t// Type assertion to CSISnapshotExposeParam\n\t\t\tcsiParam, ok := got.(*exposer.CSISnapshotExposeParam)\n\t\t\trequire.True(t, ok, \"expected CSISnapshotExposeParam type\")\n\n\t\t\t// Labels and Annotations\n\t\t\tassert.Equal(t, tt.want.labels, csiParam.HostingPodLabels)\n\t\t\tassert.Equal(t, tt.want.annotations, csiParam.HostingPodAnnotations)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/controller/download_request_controller.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\tclocks \"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/predicate\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperationmap\"\n\t\"github.com/vmware-tanzu/velero/pkg/persistence\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nconst (\n\tdefaultDownloadRequestSyncPeriod = time.Minute\n)\n\n// downloadRequestReconciler reconciles a DownloadRequest object\ntype downloadRequestReconciler struct {\n\tclient kbclient.Client\n\tclock clocks.Clock\n\t// use variables to refer to these functions so they can be\n\t// replaced with fakes for testing.\n\tnewPluginManager func(logrus.FieldLogger) clientmgmt.Manager\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter\n\n\t// used to force update of async backup item operations before processing download request\n\tbackupItemOperationsMap *itemoperationmap.BackupItemOperationsMap\n\t// used to force update of async restore item operations before processing download request\n\trestoreItemOperationsMap *itemoperationmap.RestoreItemOperationsMap\n\n\tlog logrus.FieldLogger\n}\n\n// NewDownloadRequestReconciler initializes and returns downloadRequestReconciler struct.\nfunc NewDownloadRequestReconciler(\n\tclient kbclient.Client,\n\tclock clocks.Clock,\n\tnewPluginManager func(logrus.FieldLogger) clientmgmt.Manager,\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter,\n\tlog logrus.FieldLogger,\n\tbackupItemOperationsMap *itemoperationmap.BackupItemOperationsMap,\n\trestoreItemOperationsMap *itemoperationmap.RestoreItemOperationsMap,\n) *downloadRequestReconciler {\n\treturn &downloadRequestReconciler{\n\t\tclient: client,\n\t\tclock: clock,\n\t\tnewPluginManager: newPluginManager,\n\t\tbackupStoreGetter: backupStoreGetter,\n\t\tbackupItemOperationsMap: backupItemOperationsMap,\n\t\trestoreItemOperationsMap: restoreItemOperationsMap,\n\t\tlog: log,\n\t}\n}\n\n// +kubebuilder:rbac:groups=velero.io,resources=downloadrequests,verbs=get;list;watch;create;update;patch;delete\n// +kubebuilder:rbac:groups=velero.io,resources=downloadrequests/status,verbs=get;update;patch\n\nfunc (r *downloadRequestReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\tlog := r.log.WithFields(logrus.Fields{\n\t\t\"controller\": \"download-request\",\n\t\t\"downloadRequest\": req.NamespacedName,\n\t})\n\n\t// Fetch the DownloadRequest instance.\n\tlog.Debug(\"Getting DownloadRequest\")\n\tdownloadRequest := &velerov1api.DownloadRequest{}\n\tif err := r.client.Get(ctx, req.NamespacedName, downloadRequest); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.Debug(\"Unable to find DownloadRequest\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tlog.WithError(err).Error(\"Error getting DownloadRequest\")\n\t\treturn ctrl.Result{}, errors.WithStack(err)\n\t}\n\n\tif downloadRequest.Status != (velerov1api.DownloadRequestStatus{}) && downloadRequest.Status.Expiration != nil {\n\t\tif downloadRequest.Status.Expiration.Time.Before(r.clock.Now()) {\n\t\t\t// Delete any request that is expired, regardless of the phase: it is not\n\t\t\t// worth proceeding and trying/retrying to find it.\n\t\t\tlog.Debug(\"DownloadRequest has expired - deleting\")\n\t\t\tif err := r.client.Delete(ctx, downloadRequest); err != nil {\n\t\t\t\tlog.WithError(err).Error(\"Error deleting an expired download request\")\n\t\t\t\treturn ctrl.Result{}, errors.WithStack(err)\n\t\t\t}\n\t\t\treturn ctrl.Result{}, nil\n\t\t} else if downloadRequest.Status.Phase == velerov1api.DownloadRequestPhaseProcessed {\n\t\t\tlog.Debug(\"DownloadRequest has not yet expired.\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t}\n\n\t// Process a brand new request.\n\tif downloadRequest.Status.Phase == \"\" || downloadRequest.Status.Phase == velerov1api.DownloadRequestPhaseNew {\n\t\tbackupName := downloadRequest.Spec.Target.Name\n\t\toriginal := downloadRequest.DeepCopy()\n\t\tdefer func() {\n\t\t\t// Always attempt to Patch the downloadRequest object and status for new DownloadRequest.\n\t\t\tif err := r.client.Patch(ctx, downloadRequest, kbclient.MergeFrom(original)); err != nil {\n\t\t\t\tlog.WithError(err).Error(\"Error updating download request\")\n\t\t\t\treturn\n\t\t\t}\n\t\t}()\n\n\t\t// Update the expiration.\n\t\tdownloadRequest.Status.Expiration = &metav1.Time{Time: r.clock.Now().Add(persistence.DownloadURLTTL)}\n\n\t\tif downloadRequest.Spec.Target.Kind == velerov1api.DownloadTargetKindRestoreLog ||\n\t\t\tdownloadRequest.Spec.Target.Kind == velerov1api.DownloadTargetKindRestoreResults ||\n\t\t\tdownloadRequest.Spec.Target.Kind == velerov1api.DownloadTargetKindRestoreResourceList ||\n\t\t\tdownloadRequest.Spec.Target.Kind == velerov1api.DownloadTargetKindRestoreItemOperations ||\n\t\t\tdownloadRequest.Spec.Target.Kind == velerov1api.DownloadTargetKindRestoreVolumeInfo {\n\t\t\trestore := &velerov1api.Restore{}\n\t\t\tif err := r.client.Get(ctx, kbclient.ObjectKey{\n\t\t\t\tNamespace: downloadRequest.Namespace,\n\t\t\t\tName: downloadRequest.Spec.Target.Name,\n\t\t\t}, restore); err != nil {\n\t\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\t\tlog.WithError(err).Error(\"fail to get restore for DownloadRequest\")\n\t\t\t\t\treturn ctrl.Result{}, nil\n\t\t\t\t}\n\t\t\t\tlog.Warnf(\"fail to get restore for DownloadRequest %s. Retry later.\", err.Error())\n\t\t\t\treturn ctrl.Result{}, errors.WithStack(err)\n\t\t\t}\n\t\t\tbackupName = restore.Spec.BackupName\n\t\t}\n\n\t\tbackup := &velerov1api.Backup{}\n\t\tif err := r.client.Get(ctx, kbclient.ObjectKey{\n\t\t\tNamespace: downloadRequest.Namespace,\n\t\t\tName: backupName,\n\t\t}, backup); err != nil {\n\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\tlog.WithError(err).Error(\"fail to get backup for DownloadRequest\")\n\t\t\t\treturn ctrl.Result{}, nil\n\t\t\t}\n\t\t\tlog.Warnf(\"fail to get backup for DownloadRequest %s. Retry later.\", err.Error())\n\t\t\treturn ctrl.Result{}, errors.WithStack(err)\n\t\t}\n\n\t\tlocation := &velerov1api.BackupStorageLocation{}\n\t\tif err := r.client.Get(ctx, kbclient.ObjectKey{\n\t\t\tNamespace: backup.Namespace,\n\t\t\tName: backup.Spec.StorageLocation,\n\t\t}, location); err != nil {\n\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\tlog.Errorf(\"BSL for DownloadRequest cannot be found\")\n\t\t\t\treturn ctrl.Result{}, nil\n\t\t\t}\n\t\t\tlog.Warnf(\"fail to get BSL for DownloadRequest: %s\", err.Error())\n\t\t\treturn ctrl.Result{}, errors.WithStack(err)\n\t\t}\n\n\t\tpluginManager := r.newPluginManager(log)\n\t\tdefer pluginManager.CleanupClients()\n\n\t\tbackupStore, err := r.backupStoreGetter.Get(location, pluginManager, log)\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Error(\"Error getting a backup store\")\n\t\t\t// Fail to get backup store is due to BSL setting issue or credential issue.\n\t\t\t// It cannot be recovered. No need to retry.\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\t// If this is a request for backup item operations, force upload of in-memory operations that\n\t\t// are not yet uploaded (if there are any)\n\t\tif downloadRequest.Spec.Target.Kind == velerov1api.DownloadTargetKindBackupItemOperations &&\n\t\t\tr.backupItemOperationsMap != nil {\n\t\t\t// ignore errors here. If we can't upload anything here, process the download as usual\n\t\t\t_ = r.backupItemOperationsMap.UpdateForBackup(backupStore, backupName)\n\t\t}\n\t\t// If this is a request for restore item operations, force upload of in-memory operations that\n\t\t// are not yet uploaded (if there are any)\n\t\tif downloadRequest.Spec.Target.Kind == velerov1api.DownloadTargetKindRestoreItemOperations &&\n\t\t\tr.restoreItemOperationsMap != nil {\n\t\t\t// ignore errors here. If we can't upload anything here, process the download as usual\n\t\t\t_ = r.restoreItemOperationsMap.UpdateForRestore(backupStore, downloadRequest.Spec.Target.Name)\n\t\t}\n\n\t\tif downloadRequest.Status.DownloadURL, err = backupStore.GetDownloadURL(downloadRequest.Spec.Target); err != nil {\n\t\t\tlog.Warnf(\"fail to get Backup metadata file's download URL %s, retry later: %s\", downloadRequest.Spec.Target, err)\n\t\t\treturn ctrl.Result{}, errors.WithStack(err)\n\t\t}\n\n\t\tdownloadRequest.Status.Phase = velerov1api.DownloadRequestPhaseProcessed\n\n\t\t// Update the expiration again to extend the time we wait (the TTL) to start after successfully processing the URL.\n\t\tdownloadRequest.Status.Expiration = &metav1.Time{Time: r.clock.Now().Add(persistence.DownloadURLTTL)}\n\t}\n\n\treturn ctrl.Result{}, nil\n}\n\nfunc (r *downloadRequestReconciler) SetupWithManager(mgr ctrl.Manager) error {\n\tdownloadRequestPredicate := kube.NewGenericEventPredicate(func(object kbclient.Object) bool {\n\t\tdownloadRequest := object.(*velerov1api.DownloadRequest)\n\t\tif downloadRequest.Status != (velerov1api.DownloadRequestStatus{}) && downloadRequest.Status.Expiration != nil {\n\t\t\treturn downloadRequest.Status.Expiration.Time.Before(r.clock.Now())\n\t\t}\n\t\treturn true\n\t})\n\tdownloadRequestSource := kube.NewPeriodicalEnqueueSource(r.log.WithField(\"controller\", constant.ControllerDownloadRequest), mgr.GetClient(),\n\t\t&velerov1api.DownloadRequestList{}, defaultDownloadRequestSyncPeriod, kube.PeriodicalEnqueueSourceOption{\n\t\t\tPredicates: []predicate.Predicate{downloadRequestPredicate},\n\t\t})\n\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\tFor(&velerov1api.DownloadRequest{}).\n\t\tWatchesRawSource(downloadRequestSource).\n\t\tComplete(r)\n}\n" }, { "path": "pkg/controller/download_request_controller_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\t. \"github.com/onsi/ginkgo/v2\"\n\t. \"github.com/onsi/gomega\"\n\t\"github.com/sirupsen/logrus\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/client-go/kubernetes/scheme\"\n\ttestclocks \"k8s.io/utils/clock/testing\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tpersistencemocks \"github.com/vmware-tanzu/velero/pkg/persistence/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\tpluginmocks \"github.com/vmware-tanzu/velero/pkg/plugin/mocks\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nvar _ = Describe(\"Download Request Reconciler\", func() {\n\ttype request struct {\n\t\tdownloadRequest *velerov1api.DownloadRequest\n\t\tbackup *velerov1api.Backup\n\t\trestore *velerov1api.Restore\n\t\tbackupLocation *velerov1api.BackupStorageLocation\n\t\texpired bool\n\t\texpectedReconcileErr string\n\t\texpectGetsURL bool\n\t\texpectedRequeue ctrl.Result\n\t}\n\n\tdefaultBackup := func() *velerov1api.Backup {\n\t\treturn builder.ForBackup(velerov1api.DefaultNamespace, \"a-backup\").StorageLocation(\"a-location\").Result()\n\t}\n\n\tDescribeTable(\"a Download request\",\n\t\tfunc(test request) {\n\t\t\t// now will be used to set the fake clock's time; capture\n\t\t\t// it here so it can be referenced in the test case defs.\n\t\t\tnow, err := time.Parse(time.RFC1123, time.RFC1123)\n\t\t\tExpect(err).ToNot(HaveOccurred())\n\t\t\tnow = now.Local()\n\n\t\t\trClock := testclocks.NewFakeClock(now)\n\n\t\t\tconst signedURLTTL = 10 * time.Minute\n\n\t\t\tvar (\n\t\t\t\tpluginManager = &pluginmocks.Manager{}\n\t\t\t\tbackupStores = make(map[string]*persistencemocks.BackupStore)\n\t\t\t)\n\t\t\tpluginManager.On(\"CleanupClients\").Return(nil)\n\n\t\t\tExpect(test.downloadRequest).ToNot(BeNil())\n\n\t\t\t// Set .status.expiration properly for all requests test cases that are\n\t\t\t// meant to be expired. Since \"expired\" is relative to the controller's\n\t\t\t// clock time, it's easier to do this here than as part of the test case definitions.\n\t\t\tif test.expired {\n\t\t\t\ttest.downloadRequest.Status.Expiration = &metav1.Time{Time: rClock.Now().Add(-1 * time.Minute)}\n\t\t\t}\n\n\t\t\tfakeClient := fake.NewClientBuilder().WithScheme(scheme.Scheme).Build()\n\t\t\terr = fakeClient.Create(context.TODO(), test.downloadRequest)\n\t\t\tExpect(err).ToNot(HaveOccurred())\n\n\t\t\tif test.backup != nil {\n\t\t\t\terr := fakeClient.Create(context.TODO(), test.backup)\n\t\t\t\tExpect(err).ToNot(HaveOccurred())\n\t\t\t}\n\n\t\t\tif test.backupLocation != nil {\n\t\t\t\terr := fakeClient.Create(context.TODO(), test.backupLocation)\n\t\t\t\tExpect(err).ToNot(HaveOccurred())\n\t\t\t\tbackupStores[test.backupLocation.Name] = &persistencemocks.BackupStore{}\n\t\t\t}\n\n\t\t\tif test.restore != nil {\n\t\t\t\terr := fakeClient.Create(context.TODO(), test.restore)\n\t\t\t\tExpect(err).ToNot(HaveOccurred())\n\t\t\t}\n\n\t\t\t// Setup reconciler\n\t\t\tExpect(velerov1api.AddToScheme(scheme.Scheme)).To(Succeed())\n\t\t\tr := NewDownloadRequestReconciler(\n\t\t\t\tfakeClient,\n\t\t\t\trClock,\n\t\t\t\tfunc(logrus.FieldLogger) clientmgmt.Manager { return pluginManager },\n\t\t\t\tNewFakeObjectBackupStoreGetter(backupStores),\n\t\t\t\tvelerotest.NewLogger(),\n\t\t\t\tnil,\n\t\t\t\tnil,\n\t\t\t)\n\n\t\t\tif test.backupLocation != nil && test.expectGetsURL {\n\t\t\t\tbackupStores[test.backupLocation.Name].On(\"GetDownloadURL\", test.downloadRequest.Spec.Target).Return(\"a-url\", nil)\n\t\t\t}\n\n\t\t\tactualResult, err := r.Reconcile(context.Background(), ctrl.Request{\n\t\t\t\tNamespacedName: types.NamespacedName{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: test.downloadRequest.Name,\n\t\t\t\t},\n\t\t\t})\n\n\t\t\tExpect(actualResult).To(BeEquivalentTo(test.expectedRequeue))\n\t\t\tif test.expectedReconcileErr == \"\" {\n\t\t\t\tExpect(err).ToNot(HaveOccurred())\n\t\t\t} else {\n\t\t\t\tExpect(err.Error()).To(Equal(test.expectedReconcileErr))\n\t\t\t}\n\n\t\t\tinstance := &velerov1api.DownloadRequest{}\n\t\t\terr = r.client.Get(ctx, kbclient.ObjectKey{Name: test.downloadRequest.Name, Namespace: test.downloadRequest.Namespace}, instance)\n\n\t\t\tif test.expired {\n\t\t\t\tExpect(instance).ToNot(Equal(test.downloadRequest))\n\t\t\t\tExpect(apierrors.IsNotFound(err)).To(BeTrue())\n\t\t\t} else {\n\t\t\t\tif test.downloadRequest.Status.Phase == velerov1api.DownloadRequestPhaseProcessed {\n\t\t\t\t\tExpect(instance.Status).To(Equal(test.downloadRequest.Status))\n\t\t\t\t} else {\n\t\t\t\t\tExpect(instance.Status).ToNot(Equal(test.downloadRequest.Status))\n\t\t\t\t}\n\t\t\t\tExpect(err).ToNot(HaveOccurred())\n\t\t\t}\n\n\t\t\tif test.expectGetsURL {\n\t\t\t\tExpect(string(instance.Status.Phase)).To(Equal(string(velerov1api.DownloadRequestPhaseProcessed)))\n\t\t\t\tExpect(instance.Status.DownloadURL).To(Equal(\"a-url\"))\n\t\t\t\tExpect(velerotest.TimesAreEqual(instance.Status.Expiration.Time, r.clock.Now().Add(signedURLTTL))).To(BeTrue())\n\t\t\t}\n\t\t},\n\n\t\tEntry(\"backup contents request for nonexistent backup returns nil\", request{\n\t\t\tdownloadRequest: builder.ForDownloadRequest(velerov1api.DefaultNamespace, \"a-download-request\").Phase(\"\").Target(velerov1api.DownloadTargetKindBackupContents, \"a1-backup\").Result(),\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"non-matching-backup\").StorageLocation(\"a-location\").Result(),\n\t\t\tbackupLocation: builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"a-location\").Provider(\"a-provider\").Bucket(\"a-bucket\").Result(),\n\t\t\texpectedReconcileErr: \"\",\n\t\t\texpectedRequeue: ctrl.Result{},\n\t\t}),\n\t\tEntry(\"restore log request for nonexistent restore returns nil\", request{\n\t\t\tdownloadRequest: builder.ForDownloadRequest(velerov1api.DefaultNamespace, \"a-download-request\").Phase(\"\").Target(velerov1api.DownloadTargetKindRestoreLog, \"a-backup-20170912150214\").Result(),\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"non-matching-restore\").Phase(velerov1api.RestorePhaseCompleted).Backup(\"a-backup\").Result(),\n\t\t\tbackup: defaultBackup(),\n\t\t\tbackupLocation: builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"a-location\").Provider(\"a-provider\").Bucket(\"a-bucket\").Result(),\n\t\t\texpectedReconcileErr: \"\",\n\t\t\texpectedRequeue: ctrl.Result{},\n\t\t}),\n\t\tEntry(\"backup contents request for backup with nonexistent location returns nil\", request{\n\t\t\tdownloadRequest: builder.ForDownloadRequest(velerov1api.DefaultNamespace, \"a-download-request\").Phase(\"\").Target(velerov1api.DownloadTargetKindBackupContents, \"a-backup\").Result(),\n\t\t\tbackup: defaultBackup(),\n\t\t\tbackupLocation: builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"non-matching-location\").Provider(\"a-provider\").Bucket(\"a-bucket\").Result(),\n\t\t\texpectedReconcileErr: \"\",\n\t\t\texpectedRequeue: ctrl.Result{},\n\t\t}),\n\t\tEntry(\"backup contents request with phase '' gets a url\", request{\n\t\t\tdownloadRequest: builder.ForDownloadRequest(velerov1api.DefaultNamespace, \"a-download-request\").Phase(\"\").Target(velerov1api.DownloadTargetKindBackupContents, \"a-backup\").Result(),\n\t\t\tbackup: defaultBackup(),\n\t\t\tbackupLocation: builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"a-location\").Provider(\"a-provider\").Bucket(\"a-bucket\").Result(),\n\t\t\texpectGetsURL: true,\n\t\t\texpectedRequeue: ctrl.Result{},\n\t\t}),\n\t\tEntry(\"backup contents request with phase 'New' gets a url\", request{\n\t\t\tdownloadRequest: builder.ForDownloadRequest(velerov1api.DefaultNamespace, \"a-download-request\").Phase(velerov1api.DownloadRequestPhaseNew).Target(velerov1api.DownloadTargetKindBackupContents, \"a-backup\").Result(),\n\t\t\tbackup: defaultBackup(),\n\t\t\tbackupLocation: builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"a-location\").Provider(\"a-provider\").Bucket(\"a-bucket\").Result(),\n\t\t\texpectGetsURL: true,\n\t\t\texpectedRequeue: ctrl.Result{},\n\t\t}),\n\t\tEntry(\"backup log request with phase '' gets a url\", request{\n\t\t\tdownloadRequest: builder.ForDownloadRequest(velerov1api.DefaultNamespace, \"a-download-request\").Phase(\"\").Target(velerov1api.DownloadTargetKindBackupLog, \"a-backup\").Result(),\n\t\t\tbackup: defaultBackup(),\n\t\t\tbackupLocation: builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"a-location\").Provider(\"a-provider\").Bucket(\"a-bucket\").Result(),\n\t\t\texpectGetsURL: true,\n\t\t\texpectedRequeue: ctrl.Result{},\n\t\t}),\n\t\tEntry(\"backup log request with phase 'New' gets a url\", request{\n\t\t\tdownloadRequest: builder.ForDownloadRequest(velerov1api.DefaultNamespace, \"a-download-request\").Phase(velerov1api.DownloadRequestPhaseNew).Target(velerov1api.DownloadTargetKindBackupLog, \"a-backup\").Result(),\n\t\t\tbackup: defaultBackup(),\n\t\t\tbackupLocation: builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"a-location\").Provider(\"a-provider\").Bucket(\"a-bucket\").Result(),\n\t\t\texpectGetsURL: true,\n\t\t\texpectedRequeue: ctrl.Result{},\n\t\t}),\n\t\tEntry(\"restore log request with phase '' gets a url\", request{\n\t\t\tdownloadRequest: builder.ForDownloadRequest(velerov1api.DefaultNamespace, \"a-download-request\").Phase(\"\").Target(velerov1api.DownloadTargetKindRestoreLog, \"a-backup-20170912150214\").Result(),\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"a-backup-20170912150214\").Phase(velerov1api.RestorePhaseCompleted).Backup(\"a-backup\").Result(),\n\t\t\tbackup: defaultBackup(),\n\t\t\tbackupLocation: builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"a-location\").Provider(\"a-provider\").Bucket(\"a-bucket\").Result(),\n\t\t\texpectGetsURL: true,\n\t\t\texpectedRequeue: ctrl.Result{},\n\t\t}),\n\t\tEntry(\"restore log request with phase 'New' gets a url\", request{\n\t\t\tdownloadRequest: builder.ForDownloadRequest(velerov1api.DefaultNamespace, \"a-download-request\").Phase(velerov1api.DownloadRequestPhaseNew).Target(velerov1api.DownloadTargetKindRestoreLog, \"a-backup-20170912150214\").Result(),\n\t\t\tbackup: defaultBackup(),\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"a-backup-20170912150214\").Phase(velerov1api.RestorePhaseCompleted).Backup(\"a-backup\").Result(),\n\t\t\tbackupLocation: builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"a-location\").Provider(\"a-provider\").Bucket(\"a-bucket\").Result(),\n\t\t\texpectGetsURL: true,\n\t\t\texpectedRequeue: ctrl.Result{},\n\t\t}),\n\t\tEntry(\"restore results request with phase '' gets a url\", request{\n\t\t\tdownloadRequest: builder.ForDownloadRequest(velerov1api.DefaultNamespace, \"a-download-request\").Phase(\"\").Target(velerov1api.DownloadTargetKindRestoreResults, \"a-backup-20170912150214\").Result(),\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"a-backup-20170912150214\").Phase(velerov1api.RestorePhaseCompleted).Backup(\"a-backup\").Result(),\n\t\t\tbackup: defaultBackup(),\n\t\t\tbackupLocation: builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"a-location\").Provider(\"a-provider\").Bucket(\"a-bucket\").Result(),\n\t\t\texpectGetsURL: true,\n\t\t\texpectedRequeue: ctrl.Result{},\n\t\t}),\n\t\tEntry(\"restore results request with phase 'New' gets a url\", request{\n\t\t\tdownloadRequest: builder.ForDownloadRequest(velerov1api.DefaultNamespace, \"a-download-request\").Phase(velerov1api.DownloadRequestPhaseNew).Target(velerov1api.DownloadTargetKindRestoreResults, \"a-backup-20170912150214\").Result(),\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"a-backup-20170912150214\").Phase(velerov1api.RestorePhaseCompleted).Backup(\"a-backup\").Result(),\n\t\t\tbackup: defaultBackup(),\n\t\t\tbackupLocation: builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"a-location\").Provider(\"a-provider\").Bucket(\"a-bucket\").Result(),\n\t\t\texpectGetsURL: true,\n\t\t\texpectedRequeue: ctrl.Result{},\n\t\t}),\n\t\tEntry(\"request with phase 'Processed' and not expired is not deleted\", request{\n\t\t\tdownloadRequest: builder.ForDownloadRequest(velerov1api.DefaultNamespace, \"a-download-request\").Phase(velerov1api.DownloadRequestPhaseProcessed).Target(velerov1api.DownloadTargetKindBackupLog, \"a-backup-20170912150214\").Result(),\n\t\t\tbackup: defaultBackup(),\n\t\t\texpectedRequeue: ctrl.Result{},\n\t\t}),\n\t\tEntry(\"request with phase 'Processed' and expired is deleted\", request{\n\t\t\tdownloadRequest: builder.ForDownloadRequest(velerov1api.DefaultNamespace, \"a-download-request\").Phase(velerov1api.DownloadRequestPhaseProcessed).Target(velerov1api.DownloadTargetKindBackupLog, \"a-backup-20170912150214\").Result(),\n\t\t\tbackup: defaultBackup(),\n\t\t\texpired: true,\n\t\t\texpectedRequeue: ctrl.Result{},\n\t\t}),\n\t\tEntry(\"request with phase '' and expired is deleted\", request{\n\t\t\tdownloadRequest: builder.ForDownloadRequest(velerov1api.DefaultNamespace, \"a-download-request\").Phase(\"\").Target(velerov1api.DownloadTargetKindBackupLog, \"a-backup-20170912150214\").Result(),\n\t\t\tbackup: defaultBackup(),\n\t\t\texpired: true,\n\t\t\texpectedRequeue: ctrl.Result{},\n\t\t}),\n\t\tEntry(\"request with phase 'New' and expired is deleted\", request{\n\t\t\tdownloadRequest: builder.ForDownloadRequest(velerov1api.DefaultNamespace, \"a-download-request\").Phase(velerov1api.DownloadRequestPhaseNew).Target(velerov1api.DownloadTargetKindBackupLog, \"a-backup-20170912150214\").Result(),\n\t\t\tbackup: defaultBackup(),\n\t\t\texpired: true,\n\t\t\texpectedRequeue: ctrl.Result{},\n\t\t}),\n\t)\n})\n" }, { "path": "pkg/controller/gc_controller.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tclocks \"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/builder\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/event\"\n\t\"sigs.k8s.io/controller-runtime/pkg/predicate\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tpkgbackup \"github.com/vmware-tanzu/velero/pkg/backup\"\n\tveleroclient \"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\tveleroutil \"github.com/vmware-tanzu/velero/pkg/util/velero\"\n)\n\nconst (\n\tdefaultGCFrequency = 60 * time.Minute\n\tgarbageCollectionFailure = \"velero.io/gc-failure\"\n\tgcFailureBSLNotFound = \"BSLNotFound\"\n\tgcFailureBSLCannotGet = \"BSLCannotGet\"\n\tgcFailureBSLReadOnly = \"BSLReadOnly\"\n\tgcFailureBSLUnavailable = \"BSLUnavailable\"\n)\n\n// gcReconciler creates DeleteBackupRequests for expired backups.\ntype gcReconciler struct {\n\tclient.Client\n\tlogger logrus.FieldLogger\n\tclock clocks.WithTickerAndDelayedExecution\n\tfrequency time.Duration\n}\n\n// NewGCReconciler constructs a new gcReconciler.\nfunc NewGCReconciler(\n\tlogger logrus.FieldLogger,\n\tclient client.Client,\n\tfrequency time.Duration,\n) *gcReconciler {\n\tgcr := &gcReconciler{\n\t\tClient: client,\n\t\tlogger: logger,\n\t\tclock: clocks.RealClock{},\n\t\tfrequency: frequency,\n\t}\n\tif gcr.frequency <= 0 {\n\t\tgcr.frequency = defaultGCFrequency\n\t}\n\treturn gcr\n}\n\n// GCController only watches on CreateEvent for ensuring every new backup will be taken care of.\n// Other Events will be filtered to decrease the number of reconcile call. Especially UpdateEvent must be filtered since we removed\n// the backup status as the sub-resource of backup in v1.9, every change on it will be treated as UpdateEvent and trigger reconcile call.\nfunc (c *gcReconciler) SetupWithManager(mgr ctrl.Manager) error {\n\ts := kube.NewPeriodicalEnqueueSource(c.logger.WithField(\"controller\", constant.ControllerGarbageCollection), mgr.GetClient(), &velerov1api.BackupList{}, c.frequency, kube.PeriodicalEnqueueSourceOption{})\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\tFor(&velerov1api.Backup{}, builder.WithPredicates(predicate.Funcs{\n\t\t\tUpdateFunc: func(ue event.UpdateEvent) bool {\n\t\t\t\treturn false\n\t\t\t},\n\t\t\tDeleteFunc: func(de event.DeleteEvent) bool {\n\t\t\t\treturn false\n\t\t\t},\n\t\t\tGenericFunc: func(ge event.GenericEvent) bool {\n\t\t\t\treturn false\n\t\t\t},\n\t\t})).\n\t\tWatchesRawSource(s).\n\t\tNamed(constant.ControllerGarbageCollection).\n\t\tComplete(c)\n}\n\n// +kubebuilder:rbac:groups=velero.io,resources=backups,verbs=get;list;watch;update\n// +kubebuilder:rbac:groups=velero.io,resources=backups/status,verbs=get\n// +kubebuilder:rbac:groups=velero.io,resources=deletebackuprequests,verbs=get;list;watch;create;\n// +kubebuilder:rbac:groups=velero.io,resources=deletebackuprequests/status,verbs=get\n// +kubebuilder:rbac:groups=velero.io,resources=backupstoragelocations,verbs=get\n\nfunc (c *gcReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\tlog := c.logger.WithField(\"gc backup\", req.String())\n\tlog.Debug(\"gcController getting backup\")\n\n\tbackup := &velerov1api.Backup{}\n\tif err := c.Get(ctx, req.NamespacedName, backup); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.WithError(err).Error(\"backup not found\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\treturn ctrl.Result{}, errors.Wrapf(err, \"error getting backup %s\", req.String())\n\t}\n\tlog.Debugf(\"backup: %s\", backup.Name)\n\n\tlog = c.logger.WithFields(\n\t\tlogrus.Fields{\n\t\t\t\"backup\": req.String(),\n\t\t\t\"expiration\": backup.Status.Expiration,\n\t\t},\n\t)\n\n\tnow := c.clock.Now()\n\tif backup.Status.Expiration == nil || backup.Status.Expiration.After(now) {\n\t\tlog.Debug(\"Backup has not expired yet, skipping\")\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\tlog.Infof(\"Backup:%s has expired\", backup.Name)\n\n\tif backup.Labels == nil {\n\t\tbackup.Labels = make(map[string]string)\n\t}\n\n\tloc := &velerov1api.BackupStorageLocation{}\n\tif err := c.Get(ctx, client.ObjectKey{\n\t\tNamespace: req.Namespace,\n\t\tName: backup.Spec.StorageLocation,\n\t}, loc); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.Warnf(\"Backup cannot be garbage-collected because backup storage location %s does not exist\", backup.Spec.StorageLocation)\n\t\t\tbackup.Labels[garbageCollectionFailure] = gcFailureBSLNotFound\n\t\t} else {\n\t\t\tbackup.Labels[garbageCollectionFailure] = gcFailureBSLCannotGet\n\t\t}\n\n\t\tif err := c.Update(ctx, backup); err != nil {\n\t\t\tlog.WithError(err).Error(\"error updating backup labels\")\n\t\t}\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error getting backup storage location\")\n\t}\n\n\tif !veleroutil.BSLIsAvailable(*loc) {\n\t\tlog.Infof(\"BSL %s is unavailable, cannot gc backup\", loc.Name)\n\t\treturn ctrl.Result{}, fmt.Errorf(\"bsl %s is unavailable, cannot gc backup\", loc.Name)\n\t}\n\n\tif loc.Spec.AccessMode == velerov1api.BackupStorageLocationAccessModeReadOnly {\n\t\tlog.Infof(\"Backup cannot be garbage-collected because backup storage location %s is currently in read-only mode\", loc.Name)\n\t\tbackup.Labels[garbageCollectionFailure] = gcFailureBSLReadOnly\n\t\tif err := c.Update(ctx, backup); err != nil {\n\t\t\tlog.WithError(err).Error(\"error updating backup labels\")\n\t\t}\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\t// remove gc fail error label after this point\n\tdelete(backup.Labels, garbageCollectionFailure)\n\tif err := c.Update(ctx, backup); err != nil {\n\t\tlog.WithError(err).Error(\"error updating backup labels\")\n\t}\n\n\tselector := client.MatchingLabels{\n\t\tvelerov1api.BackupNameLabel: label.GetValidName(backup.Name),\n\t\tvelerov1api.BackupUIDLabel: string(backup.UID),\n\t}\n\n\tdbrs := &velerov1api.DeleteBackupRequestList{}\n\tif err := c.List(ctx, dbrs, selector); err != nil {\n\t\tlog.WithError(err).Error(\"error listing DeleteBackupRequests\")\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error listing existing DeleteBackupRequests for backup\")\n\t}\n\tlog.Debugf(\"length of dbrs:%d\", len(dbrs.Items))\n\n\t// if there's an existing unprocessed deletion request for this backup, don't create\n\t// another one\n\tfor _, dbr := range dbrs.Items {\n\t\tswitch dbr.Status.Phase {\n\t\tcase \"\", velerov1api.DeleteBackupRequestPhaseNew, velerov1api.DeleteBackupRequestPhaseInProgress:\n\t\t\tlog.Info(\"Backup already has a pending deletion request\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t}\n\n\tlog.Info(\"Creating a new deletion request\")\n\tndbr := pkgbackup.NewDeleteBackupRequest(backup.Name, string(backup.UID))\n\tndbr.SetNamespace(backup.Namespace)\n\tif err := veleroclient.CreateRetryGenerateName(c, ctx, ndbr); err != nil {\n\t\tlog.WithError(err).Error(\"error creating DeleteBackupRequests\")\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error creating DeleteBackupRequest\")\n\t}\n\n\treturn ctrl.Result{}, nil\n}\n" }, { "path": "pkg/controller/gc_controller_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/assert\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\ttestclocks \"k8s.io/utils/clock/testing\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc mockGCReconciler(fakeClient kbclient.Client, fakeClock *testclocks.FakeClock, freq time.Duration) *gcReconciler {\n\tgcr := NewGCReconciler(\n\t\tvelerotest.NewLogger(),\n\t\tfakeClient,\n\t\tfreq,\n\t)\n\tgcr.clock = fakeClock\n\treturn gcr\n}\n\nfunc TestGCReconcile(t *testing.T) {\n\tfakeClock := testclocks.NewFakeClock(time.Now())\n\tdefaultBackupLocation := builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"default\").Phase(velerov1api.BackupStorageLocationPhaseAvailable).Result()\n\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\tdeleteBackupRequests []*velerov1api.DeleteBackupRequest\n\t\tbackupLocation *velerov1api.BackupStorageLocation\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"can't find backup - no error\",\n\t\t},\n\t\t{\n\t\t\tname: \"unexpired backup is not deleted\",\n\t\t\tbackup: defaultBackup().Expiration(fakeClock.Now().Add(time.Minute)).StorageLocation(\"default\").Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t},\n\t\t{\n\t\t\tname: \"expired backup in read-only storage location is not deleted\",\n\t\t\tbackup: defaultBackup().Expiration(fakeClock.Now().Add(-time.Minute)).StorageLocation(\"read-only\").Result(),\n\t\t\tbackupLocation: builder.ForBackupStorageLocation(\"velero\", \"read-only\").AccessMode(velerov1api.BackupStorageLocationAccessModeReadOnly).Phase(velerov1api.BackupStorageLocationPhaseAvailable).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"expired backup in read-write storage location is deleted\",\n\t\t\tbackup: defaultBackup().Expiration(fakeClock.Now().Add(-time.Minute)).StorageLocation(\"read-write\").Result(),\n\t\t\tbackupLocation: builder.ForBackupStorageLocation(\"velero\", \"read-write\").AccessMode(velerov1api.BackupStorageLocationAccessModeReadWrite).Phase(velerov1api.BackupStorageLocationPhaseAvailable).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"expired backup with no pending deletion requests is deleted\",\n\t\t\tbackup: defaultBackup().Expiration(fakeClock.Now().Add(-time.Second)).StorageLocation(\"default\").Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t},\n\t\t{\n\t\t\tname: \"expired backup with a pending deletion request is not deleted\",\n\t\t\tbackup: defaultBackup().Expiration(fakeClock.Now().Add(-time.Second)).StorageLocation(\"default\").Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\tdeleteBackupRequests: []*velerov1api.DeleteBackupRequest{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\t\tName: \"foo\",\n\t\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\tvelerov1api.BackupNameLabel: \"backup-1\",\n\t\t\t\t\t\t\tvelerov1api.BackupUIDLabel: \"\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tStatus: velerov1api.DeleteBackupRequestStatus{\n\t\t\t\t\t\tPhase: velerov1api.DeleteBackupRequestPhaseInProgress,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"expired backup with only processed deletion requests is deleted\",\n\t\t\tbackup: defaultBackup().Expiration(fakeClock.Now().Add(-time.Second)).StorageLocation(\"default\").Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\tdeleteBackupRequests: []*velerov1api.DeleteBackupRequest{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\t\tName: \"foo\",\n\t\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\tvelerov1api.BackupNameLabel: \"backup-1\",\n\t\t\t\t\t\t\tvelerov1api.BackupUIDLabel: \"\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tStatus: velerov1api.DeleteBackupRequestStatus{\n\t\t\t\t\t\tPhase: velerov1api.DeleteBackupRequestPhaseProcessed,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"BSL is unavailable\",\n\t\t\tbackup: defaultBackup().Expiration(fakeClock.Now().Add(-time.Second)).StorageLocation(\"default\").Result(),\n\t\t\tbackupLocation: builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"default\").Phase(velerov1api.BackupStorageLocationPhaseUnavailable).Result(),\n\t\t\texpectError: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tif test.backup == nil {\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tinitObjs := []runtime.Object{}\n\t\t\tinitObjs = append(initObjs, test.backup)\n\n\t\t\tif test.backupLocation != nil {\n\t\t\t\tinitObjs = append(initObjs, test.backupLocation)\n\t\t\t}\n\n\t\t\tfor _, dbr := range test.deleteBackupRequests {\n\t\t\t\tinitObjs = append(initObjs, dbr)\n\t\t\t}\n\n\t\t\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, initObjs...)\n\t\t\treconciler := mockGCReconciler(fakeClient, fakeClock, defaultGCFrequency)\n\t\t\t_, err := reconciler.Reconcile(t.Context(), ctrl.Request{NamespacedName: types.NamespacedName{Namespace: test.backup.Namespace, Name: test.backup.Name}})\n\t\t\tgotErr := err != nil\n\t\t\tassert.Equal(t, test.expectError, gotErr)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/controller/interface.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport \"context\"\n\n// Interface represents a runnable component.\ntype Interface interface {\n\t// Run runs the component.\n\tRun(ctx context.Context, workers int) error\n}\n" }, { "path": "pkg/controller/pod_volume_backup_controller.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"k8s.io/client-go/kubernetes\"\n\tclocks \"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/builder\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil\"\n\t\"sigs.k8s.io/controller-runtime/pkg/event\"\n\t\"sigs.k8s.io/controller-runtime/pkg/manager\"\n\t\"sigs.k8s.io/controller-runtime/pkg/predicate\"\n\t\"sigs.k8s.io/controller-runtime/pkg/reconcile\"\n\n\tveleroapishared \"github.com/vmware-tanzu/velero/pkg/apis/velero/shared\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\t\"github.com/vmware-tanzu/velero/pkg/exposer\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\t\"github.com/vmware-tanzu/velero/pkg/nodeagent\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nconst (\n\tpVBRRequestor = \"pod-volume-backup-restore\"\n\tPodVolumeFinalizer = \"velero.io/pod-volume-finalizer\"\n)\n\n// NewPodVolumeBackupReconciler creates the PodVolumeBackupReconciler instance\nfunc NewPodVolumeBackupReconciler(\n\tclient client.Client,\n\tmgr manager.Manager,\n\tkubeClient kubernetes.Interface,\n\tdataPathMgr *datapath.Manager,\n\tcounter *exposer.VgdpCounter,\n\tnodeName string,\n\tpreparingTimeout time.Duration,\n\tresourceTimeout time.Duration,\n\tpodResources corev1api.ResourceRequirements,\n\tmetrics *metrics.ServerMetrics,\n\tlogger logrus.FieldLogger,\n\tdataMovePriorityClass string,\n\tprivileged bool,\n\tpodLabels map[string]string,\n\tpodAnnotations map[string]string,\n) *PodVolumeBackupReconciler {\n\treturn &PodVolumeBackupReconciler{\n\t\tclient: client,\n\t\tmgr: mgr,\n\t\tkubeClient: kubeClient,\n\t\tlogger: logger.WithField(\"controller\", \"PodVolumeBackup\"),\n\t\tnodeName: nodeName,\n\t\tclock: &clocks.RealClock{},\n\t\tmetrics: metrics,\n\t\tpodResources: podResources,\n\t\tdataPathMgr: dataPathMgr,\n\t\tvgdpCounter: counter,\n\t\tpreparingTimeout: preparingTimeout,\n\t\tresourceTimeout: resourceTimeout,\n\t\texposer: exposer.NewPodVolumeExposer(kubeClient, logger),\n\t\tcancelledPVB: make(map[string]time.Time),\n\t\tdataMovePriorityClass: dataMovePriorityClass,\n\t\tprivileged: privileged,\n\t\tpodLabels: podLabels,\n\t\tpodAnnotations: podAnnotations,\n\t}\n}\n\n// PodVolumeBackupReconciler reconciles a PodVolumeBackup object\ntype PodVolumeBackupReconciler struct {\n\tclient client.Client\n\tmgr manager.Manager\n\tkubeClient kubernetes.Interface\n\tclock clocks.WithTickerAndDelayedExecution\n\texposer exposer.PodVolumeExposer\n\tmetrics *metrics.ServerMetrics\n\tnodeName string\n\tlogger logrus.FieldLogger\n\tpodResources corev1api.ResourceRequirements\n\tdataPathMgr *datapath.Manager\n\tvgdpCounter *exposer.VgdpCounter\n\tpreparingTimeout time.Duration\n\tresourceTimeout time.Duration\n\tcancelledPVB map[string]time.Time\n\tdataMovePriorityClass string\n\tprivileged bool\n\tpodLabels map[string]string\n\tpodAnnotations map[string]string\n}\n\n// +kubebuilder:rbac:groups=velero.io,resources=podvolumebackups,verbs=get;list;watch;create;update;patch;delete\n// +kubebuilder:rbac:groups=velero.io,resources=podvolumebackups/status,verbs=get;update;patch\n\nfunc (r *PodVolumeBackupReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\tlog := r.logger.WithFields(logrus.Fields{\n\t\t\"controller\": \"podvolumebackup\",\n\t\t\"podvolumebackup\": req.NamespacedName,\n\t})\n\n\tvar pvb = &velerov1api.PodVolumeBackup{}\n\tif err := r.client.Get(ctx, req.NamespacedName, pvb); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.Warn(\"Unable to find PVB, skip\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"getting PVB\")\n\t}\n\tif len(pvb.OwnerReferences) == 1 {\n\t\tlog = log.WithField(\n\t\t\t\"backup\",\n\t\t\tfmt.Sprintf(\"%s/%s\", req.Namespace, pvb.OwnerReferences[0].Name),\n\t\t)\n\t}\n\n\tif !isPVBInFinalState(pvb) {\n\t\tif !controllerutil.ContainsFinalizer(pvb, PodVolumeFinalizer) {\n\t\t\tif err := UpdatePVBWithRetry(ctx, r.client, req.NamespacedName, log, func(pvb *velerov1api.PodVolumeBackup) bool {\n\t\t\t\tif controllerutil.ContainsFinalizer(pvb, PodVolumeFinalizer) {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tcontrollerutil.AddFinalizer(pvb, PodVolumeFinalizer)\n\n\t\t\t\treturn true\n\t\t\t}); err != nil {\n\t\t\t\tlog.WithError(err).Errorf(\"Failed to add finalizer for PVB %s/%s\", pvb.Namespace, pvb.Name)\n\t\t\t\treturn ctrl.Result{}, err\n\t\t\t}\n\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tif !pvb.DeletionTimestamp.IsZero() {\n\t\t\tif !pvb.Spec.Cancel {\n\t\t\t\tlog.Warnf(\"Cancel PVB under phase %s because it is being deleted\", pvb.Status.Phase)\n\n\t\t\t\tif err := UpdatePVBWithRetry(ctx, r.client, req.NamespacedName, log, func(pvb *velerov1api.PodVolumeBackup) bool {\n\t\t\t\t\tif pvb.Spec.Cancel {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\n\t\t\t\t\tpvb.Spec.Cancel = true\n\t\t\t\t\tpvb.Status.Message = \"Cancel PVB because it is being deleted\"\n\n\t\t\t\t\treturn true\n\t\t\t\t}); err != nil {\n\t\t\t\t\tlog.WithError(err).Errorf(\"Failed to set cancel flag for PVB %s/%s\", pvb.Namespace, pvb.Name)\n\t\t\t\t\treturn ctrl.Result{}, err\n\t\t\t\t}\n\n\t\t\t\treturn ctrl.Result{}, nil\n\t\t\t}\n\t\t}\n\t} else {\n\t\tdelete(r.cancelledPVB, pvb.Name)\n\n\t\tif controllerutil.ContainsFinalizer(pvb, PodVolumeFinalizer) {\n\t\t\tif err := UpdatePVBWithRetry(ctx, r.client, req.NamespacedName, log, func(pvb *velerov1api.PodVolumeBackup) bool {\n\t\t\t\tif !controllerutil.ContainsFinalizer(pvb, PodVolumeFinalizer) {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tcontrollerutil.RemoveFinalizer(pvb, PodVolumeFinalizer)\n\n\t\t\t\treturn true\n\t\t\t}); err != nil {\n\t\t\t\tlog.WithError(err).Error(\"error to remove finalizer\")\n\t\t\t\treturn ctrl.Result{}, err\n\t\t\t}\n\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t}\n\n\tif pvb.Spec.Cancel {\n\t\tif spotted, found := r.cancelledPVB[pvb.Name]; !found {\n\t\t\tr.cancelledPVB[pvb.Name] = r.clock.Now()\n\t\t} else {\n\t\t\tdelay := cancelDelayOthers\n\t\t\tif pvb.Status.Phase == velerov1api.PodVolumeBackupPhaseInProgress {\n\t\t\t\tdelay = cancelDelayInProgress\n\t\t\t}\n\n\t\t\tif time.Since(spotted) > delay {\n\t\t\t\tlog.Infof(\"PVB %s is canceled in Phase %s but not handled in reasonable time\", pvb.GetName(), pvb.Status.Phase)\n\t\t\t\tif r.tryCancelPodVolumeBackup(ctx, pvb, \"\") {\n\t\t\t\t\tdelete(r.cancelledPVB, pvb.Name)\n\t\t\t\t}\n\n\t\t\t\treturn ctrl.Result{}, nil\n\t\t\t}\n\t\t}\n\t}\n\n\tif pvb.Status.Phase == \"\" || pvb.Status.Phase == velerov1api.PodVolumeBackupPhaseNew {\n\t\tif pvb.Spec.Cancel {\n\t\t\tlog.Infof(\"PVB %s is canceled in Phase %s\", pvb.GetName(), pvb.Status.Phase)\n\t\t\tr.tryCancelPodVolumeBackup(ctx, pvb, \"\")\n\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\t// Only process items for this node.\n\t\tif pvb.Spec.Node != r.nodeName {\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tif r.vgdpCounter != nil && r.vgdpCounter.IsConstrained(ctx, r.logger) {\n\t\t\tlog.Debug(\"Data path initiation is constrained, requeue later\")\n\t\t\treturn ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5}, nil\n\t\t}\n\n\t\tlog.Info(\"Accepting PVB\")\n\n\t\tif err := r.acceptPodVolumeBackup(ctx, pvb); err != nil {\n\t\t\treturn ctrl.Result{}, errors.Wrapf(err, \"error accepting PVB %s\", pvb.Name)\n\t\t}\n\n\t\tlog.Info(\"Exposing PVB\")\n\n\t\texposeParam := r.setupExposeParam(pvb)\n\t\tif err := r.exposer.Expose(ctx, getPVBOwnerObject(pvb), exposeParam); err != nil {\n\t\t\treturn r.errorOut(ctx, pvb, err, \"error to expose PVB\", log)\n\t\t}\n\n\t\tlog.Info(\"PVB is exposed\")\n\n\t\treturn ctrl.Result{}, nil\n\t} else if pvb.Status.Phase == velerov1api.PodVolumeBackupPhaseAccepted {\n\t\tif peekErr := r.exposer.PeekExposed(ctx, getPVBOwnerObject(pvb)); peekErr != nil {\n\t\t\tlog.Errorf(\"Cancel PVB %s/%s because of expose error %s\", pvb.Namespace, pvb.Name, peekErr)\n\n\t\t\tdiags := strings.Split(r.exposer.DiagnoseExpose(ctx, getPVBOwnerObject(pvb)), \"\\n\")\n\t\t\tfor _, diag := range diags {\n\t\t\t\tlog.Warnf(\"[Diagnose PVB expose]%s\", diag)\n\t\t\t}\n\n\t\t\tr.tryCancelPodVolumeBackup(ctx, pvb, fmt.Sprintf(\"found a PVB %s/%s with expose error: %s. mark it as cancel\", pvb.Namespace, pvb.Name, peekErr))\n\t\t} else if pvb.Status.AcceptedTimestamp != nil {\n\t\t\tif time.Since(pvb.Status.AcceptedTimestamp.Time) >= r.preparingTimeout {\n\t\t\t\tr.onPrepareTimeout(ctx, pvb)\n\t\t\t}\n\t\t}\n\n\t\treturn ctrl.Result{}, nil\n\t} else if pvb.Status.Phase == velerov1api.PodVolumeBackupPhasePrepared {\n\t\tlog.Infof(\"PVB is prepared and should be processed by %s (%s)\", pvb.Spec.Node, r.nodeName)\n\n\t\tif pvb.Spec.Node != r.nodeName {\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tif pvb.Spec.Cancel {\n\t\t\tlog.Info(\"Prepared PVB is being canceled\")\n\t\t\tr.OnDataPathCancelled(ctx, pvb.GetNamespace(), pvb.GetName())\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tasyncBR := r.dataPathMgr.GetAsyncBR(pvb.Name)\n\t\tif asyncBR != nil {\n\t\t\tlog.Info(\"Cancellable data path is already started\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tres, err := r.exposer.GetExposed(ctx, getPVBOwnerObject(pvb), r.client, r.nodeName, r.resourceTimeout)\n\t\tif err != nil {\n\t\t\treturn r.errorOut(ctx, pvb, err, \"exposed PVB is not ready\", log)\n\t\t} else if res == nil {\n\t\t\treturn r.errorOut(ctx, pvb, errors.New(\"no expose result is available for the current node\"), \"exposed PVB is not ready\", log)\n\t\t}\n\n\t\tlog.Info(\"Exposed PVB is ready and creating data path routine\")\n\n\t\tcallbacks := datapath.Callbacks{\n\t\t\tOnCompleted: r.OnDataPathCompleted,\n\t\t\tOnFailed: r.OnDataPathFailed,\n\t\t\tOnCancelled: r.OnDataPathCancelled,\n\t\t\tOnProgress: r.OnDataPathProgress,\n\t\t}\n\n\t\tasyncBR, err = r.dataPathMgr.CreateMicroServiceBRWatcher(ctx, r.client, r.kubeClient, r.mgr, datapath.TaskTypeBackup,\n\t\t\tpvb.Name, pvb.Namespace, res.ByPod.HostingPod.Name, res.ByPod.HostingContainer, pvb.Name, callbacks, false, log)\n\t\tif err != nil {\n\t\t\tif err == datapath.ConcurrentLimitExceed {\n\t\t\t\tlog.Debug(\"Data path instance is concurrent limited requeue later\")\n\t\t\t\treturn ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5}, nil\n\t\t\t} else {\n\t\t\t\treturn r.errorOut(ctx, pvb, err, \"error to create data path\", log)\n\t\t\t}\n\t\t}\n\n\t\tr.metrics.RegisterPodVolumeBackupEnqueue(r.nodeName)\n\n\t\tif err := r.initCancelableDataPath(ctx, asyncBR, res, log); err != nil {\n\t\t\tlog.WithError(err).Errorf(\"Failed to init cancelable data path for %s\", pvb.Name)\n\n\t\t\tr.closeDataPath(ctx, pvb.Name)\n\t\t\treturn r.errorOut(ctx, pvb, err, \"error initializing data path\", log)\n\t\t}\n\n\t\tterminated := false\n\t\tif err := UpdatePVBWithRetry(ctx, r.client, types.NamespacedName{Namespace: pvb.Namespace, Name: pvb.Name}, log, func(pvb *velerov1api.PodVolumeBackup) bool {\n\t\t\tif isPVBInFinalState(pvb) {\n\t\t\t\tterminated = true\n\t\t\t\treturn false\n\t\t\t}\n\n\t\t\tpvb.Status.Phase = velerov1api.PodVolumeBackupPhaseInProgress\n\t\t\tpvb.Status.StartTimestamp = &metav1.Time{Time: r.clock.Now()}\n\n\t\t\tdelete(pvb.Labels, exposer.ExposeOnGoingLabel)\n\n\t\t\treturn true\n\t\t}); err != nil {\n\t\t\tlog.WithError(err).Warnf(\"Failed to update PVB %s to InProgress, will data path close and retry\", pvb.Name)\n\n\t\t\tr.closeDataPath(ctx, pvb.Name)\n\t\t\treturn ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5}, nil\n\t\t}\n\n\t\tif terminated {\n\t\t\tlog.Warnf(\"PVB %s is terminated during transition from prepared\", pvb.Name)\n\t\t\tr.closeDataPath(ctx, pvb.Name)\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tlog.Info(\"PVB is marked as in progress\")\n\n\t\tif err := r.startCancelableDataPath(asyncBR, pvb, res, log); err != nil {\n\t\t\tlog.WithError(err).Errorf(\"Failed to start cancelable data path for %s\", pvb.Name)\n\t\t\tr.closeDataPath(ctx, pvb.Name)\n\n\t\t\treturn r.errorOut(ctx, pvb, err, \"error starting data path\", log)\n\t\t}\n\n\t\treturn ctrl.Result{}, nil\n\t} else if pvb.Status.Phase == velerov1api.PodVolumeBackupPhaseInProgress {\n\t\tif pvb.Spec.Cancel {\n\t\t\tif pvb.Spec.Node != r.nodeName {\n\t\t\t\treturn ctrl.Result{}, nil\n\t\t\t}\n\n\t\t\tlog.Info(\"In progress PVB is being canceled\")\n\n\t\t\tasyncBR := r.dataPathMgr.GetAsyncBR(pvb.Name)\n\t\t\tif asyncBR == nil {\n\t\t\t\tr.OnDataPathCancelled(ctx, pvb.GetNamespace(), pvb.GetName())\n\t\t\t\treturn ctrl.Result{}, nil\n\t\t\t}\n\n\t\t\t// Update status to Canceling\n\t\t\tif err := UpdatePVBWithRetry(ctx, r.client, types.NamespacedName{Namespace: pvb.Namespace, Name: pvb.Name}, log, func(pvb *velerov1api.PodVolumeBackup) bool {\n\t\t\t\tif isPVBInFinalState(pvb) {\n\t\t\t\t\tlog.Warnf(\"PVB %s is terminated, abort setting it to canceling\", pvb.Name)\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tpvb.Status.Phase = velerov1api.PodVolumeBackupPhaseCanceling\n\t\t\t\treturn true\n\t\t\t}); err != nil {\n\t\t\t\tlog.WithError(err).Error(\"error updating PVB into canceling status\")\n\t\t\t\treturn ctrl.Result{}, err\n\t\t\t}\n\n\t\t\tasyncBR.Cancel()\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\treturn ctrl.Result{}, nil\n}\n\nfunc (r *PodVolumeBackupReconciler) acceptPodVolumeBackup(ctx context.Context, pvb *velerov1api.PodVolumeBackup) error {\n\treturn UpdatePVBWithRetry(ctx, r.client, types.NamespacedName{Namespace: pvb.Namespace, Name: pvb.Name}, r.logger, func(pvb *velerov1api.PodVolumeBackup) bool {\n\t\tpvb.Status.AcceptedTimestamp = &metav1.Time{Time: r.clock.Now()}\n\t\tpvb.Status.Phase = velerov1api.PodVolumeBackupPhaseAccepted\n\n\t\tif pvb.Labels == nil {\n\t\t\tpvb.Labels = make(map[string]string)\n\t\t}\n\t\tpvb.Labels[exposer.ExposeOnGoingLabel] = \"true\"\n\n\t\treturn true\n\t})\n}\n\nfunc (r *PodVolumeBackupReconciler) tryCancelPodVolumeBackup(ctx context.Context, pvb *velerov1api.PodVolumeBackup, message string) bool {\n\tlog := r.logger.WithField(\"PVB\", pvb.Name)\n\tsucceeded, err := funcExclusiveUpdatePodVolumeBackup(ctx, r.client, pvb, func(pvb *velerov1api.PodVolumeBackup) {\n\t\tpvb.Status.Phase = velerov1api.PodVolumeBackupPhaseCanceled\n\t\tif pvb.Status.StartTimestamp.IsZero() {\n\t\t\tpvb.Status.StartTimestamp = &metav1.Time{Time: r.clock.Now()}\n\t\t}\n\t\tpvb.Status.CompletionTimestamp = &metav1.Time{Time: r.clock.Now()}\n\n\t\tif message != \"\" {\n\t\t\tpvb.Status.Message = message\n\t\t}\n\n\t\tdelete(pvb.Labels, exposer.ExposeOnGoingLabel)\n\t})\n\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"error updating PVB status\")\n\t\treturn false\n\t} else if !succeeded {\n\t\tlog.Warn(\"conflict in updating PVB status and will try it again later\")\n\t\treturn false\n\t}\n\n\tr.exposer.CleanUp(ctx, getPVBOwnerObject(pvb))\n\n\tlog.Warn(\"PVB is canceled\")\n\n\treturn true\n}\n\nvar funcExclusiveUpdatePodVolumeBackup = exclusiveUpdatePodVolumeBackup\n\nfunc exclusiveUpdatePodVolumeBackup(ctx context.Context, cli client.Client, pvb *velerov1api.PodVolumeBackup, updateFunc func(*velerov1api.PodVolumeBackup)) (bool, error) {\n\tupdateFunc(pvb)\n\n\terr := cli.Update(ctx, pvb)\n\tif err == nil {\n\t\treturn true, nil\n\t}\n\n\tif apierrors.IsConflict(err) {\n\t\treturn false, nil\n\t} else {\n\t\treturn false, err\n\t}\n}\n\nfunc (r *PodVolumeBackupReconciler) onPrepareTimeout(ctx context.Context, pvb *velerov1api.PodVolumeBackup) {\n\tlog := r.logger.WithField(\"PVB\", pvb.Name)\n\n\tlog.Info(\"Timeout happened for preparing PVB\")\n\n\tsucceeded, err := funcExclusiveUpdatePodVolumeBackup(ctx, r.client, pvb, func(pvb *velerov1api.PodVolumeBackup) {\n\t\tpvb.Status.Phase = velerov1api.PodVolumeBackupPhaseFailed\n\t\tpvb.Status.Message = \"timeout on preparing PVB\"\n\n\t\tdelete(pvb.Labels, exposer.ExposeOnGoingLabel)\n\t})\n\n\tif err != nil {\n\t\tlog.WithError(err).Warn(\"Failed to update PVB\")\n\t\treturn\n\t}\n\n\tif !succeeded {\n\t\tlog.Warn(\"PVB has been updated by others\")\n\t\treturn\n\t}\n\n\tdiags := strings.Split(r.exposer.DiagnoseExpose(ctx, getPVBOwnerObject(pvb)), \"\\n\")\n\tfor _, diag := range diags {\n\t\tlog.Warnf(\"[Diagnose PVB expose]%s\", diag)\n\t}\n\n\tr.exposer.CleanUp(ctx, getPVBOwnerObject(pvb))\n\n\tlog.Info(\"PVB has been cleaned up\")\n}\n\nfunc (r *PodVolumeBackupReconciler) initCancelableDataPath(ctx context.Context, asyncBR datapath.AsyncBR, res *exposer.ExposeResult, log logrus.FieldLogger) error {\n\tlog.Info(\"Init cancelable PVB\")\n\n\tif err := asyncBR.Init(ctx, nil); err != nil {\n\t\treturn errors.Wrap(err, \"error initializing asyncBR\")\n\t}\n\n\tlog.Infof(\"async data path init for pod %s, volume %s\", res.ByPod.HostingPod.Name, res.ByPod.VolumeName)\n\n\treturn nil\n}\n\nfunc (r *PodVolumeBackupReconciler) startCancelableDataPath(asyncBR datapath.AsyncBR, pvb *velerov1api.PodVolumeBackup, res *exposer.ExposeResult, log logrus.FieldLogger) error {\n\tlog.Info(\"Start cancelable PVB\")\n\n\tif err := asyncBR.StartBackup(datapath.AccessPoint{\n\t\tByPath: res.ByPod.VolumeName,\n\t}, pvb.Spec.UploaderSettings, nil); err != nil {\n\t\treturn errors.Wrapf(err, \"error starting async backup for pod %s, volume %s\", res.ByPod.HostingPod.Name, res.ByPod.VolumeName)\n\t}\n\n\tlog.Infof(\"Async backup started for pod %s, volume %s\", res.ByPod.HostingPod.Name, res.ByPod.VolumeName)\n\treturn nil\n}\n\nfunc (r *PodVolumeBackupReconciler) OnDataPathCompleted(ctx context.Context, namespace string, pvbName string, result datapath.Result) {\n\tdefer r.dataPathMgr.RemoveAsyncBR(pvbName)\n\n\tlog := r.logger.WithField(\"PVB\", pvbName)\n\n\tlog.WithField(\"PVB\", pvbName).Info(\"Async fs backup data path completed\")\n\n\tpvb := &velerov1api.PodVolumeBackup{}\n\tif err := r.client.Get(ctx, types.NamespacedName{Name: pvbName, Namespace: namespace}, pvb); err != nil {\n\t\tlog.WithError(err).Warn(\"Failed to get PVB on completion\")\n\t\treturn\n\t}\n\n\tlog.Info(\"Cleaning up exposed environment\")\n\tr.exposer.CleanUp(ctx, getPVBOwnerObject(pvb))\n\n\t// Update status to Completed with path & snapshot ID.\n\tvar completionTime metav1.Time\n\tif err := UpdatePVBWithRetry(ctx, r.client, types.NamespacedName{Namespace: pvb.Namespace, Name: pvb.Name}, log, func(pvb *velerov1api.PodVolumeBackup) bool {\n\t\tcompletionTime = metav1.Time{Time: r.clock.Now()}\n\n\t\tif isPVBInFinalState(pvb) {\n\t\t\treturn false\n\t\t}\n\n\t\tpvb.Status.Path = result.Backup.Source.ByPath\n\t\tpvb.Status.Phase = velerov1api.PodVolumeBackupPhaseCompleted\n\t\tpvb.Status.SnapshotID = result.Backup.SnapshotID\n\t\tpvb.Status.CompletionTimestamp = &completionTime\n\t\tpvb.Status.IncrementalBytes = result.Backup.IncrementalBytes\n\t\tif result.Backup.EmptySnapshot {\n\t\t\tpvb.Status.Message = \"volume was empty so no snapshot was taken\"\n\t\t}\n\n\t\tdelete(pvb.Labels, exposer.ExposeOnGoingLabel)\n\n\t\treturn true\n\t}); err != nil {\n\t\tlog.WithError(err).Error(\"error updating PVB status\")\n\t} else {\n\t\tlatencyDuration := completionTime.Time.Sub(pvb.Status.StartTimestamp.Time)\n\t\tlatencySeconds := float64(latencyDuration / time.Second)\n\t\tbackupName := fmt.Sprintf(\"%s/%s\", pvb.Namespace, pvb.OwnerReferences[0].Name)\n\t\tgenerateOpName := fmt.Sprintf(\"%s-%s-%s-%s-backup\", pvb.Name, pvb.Spec.BackupStorageLocation, pvb.Spec.Pod.Namespace, pvb.Spec.UploaderType)\n\t\tr.metrics.ObservePodVolumeOpLatency(r.nodeName, pvb.Name, generateOpName, backupName, latencySeconds)\n\t\tr.metrics.RegisterPodVolumeOpLatencyGauge(r.nodeName, pvb.Name, generateOpName, backupName, latencySeconds)\n\t\tr.metrics.RegisterPodVolumeBackupDequeue(r.nodeName)\n\n\t\tlog.Info(\"PVB completed\")\n\t}\n}\n\nfunc (r *PodVolumeBackupReconciler) OnDataPathFailed(ctx context.Context, namespace, pvbName string, err error) {\n\tdefer r.dataPathMgr.RemoveAsyncBR(pvbName)\n\n\tlog := r.logger.WithField(\"PVB\", pvbName)\n\n\tlog.WithError(err).Error(\"Async fs backup data path failed\")\n\n\tvar pvb velerov1api.PodVolumeBackup\n\tif getErr := r.client.Get(ctx, types.NamespacedName{Name: pvbName, Namespace: namespace}, &pvb); getErr != nil {\n\t\tlog.WithError(getErr).Warn(\"Failed to get PVB on failure\")\n\t} else {\n\t\t_, _ = r.errorOut(ctx, &pvb, err, \"data path backup failed\", log)\n\t}\n}\n\nfunc (r *PodVolumeBackupReconciler) OnDataPathCancelled(ctx context.Context, namespace string, pvbName string) {\n\tdefer r.dataPathMgr.RemoveAsyncBR(pvbName)\n\n\tlog := r.logger.WithField(\"PVB\", pvbName)\n\n\tlog.Warn(\"Async fs backup data path canceled\")\n\n\tvar pvb velerov1api.PodVolumeBackup\n\tif getErr := r.client.Get(ctx, types.NamespacedName{Name: pvbName, Namespace: namespace}, &pvb); getErr != nil {\n\t\tlog.WithError(getErr).Warn(\"Failed to get PVB on cancel\")\n\t\treturn\n\t}\n\t// cleans up any objects generated during the snapshot expose\n\tr.exposer.CleanUp(ctx, getPVBOwnerObject(&pvb))\n\n\tif err := UpdatePVBWithRetry(ctx, r.client, types.NamespacedName{Namespace: pvb.Namespace, Name: pvb.Name}, log, func(pvb *velerov1api.PodVolumeBackup) bool {\n\t\tif isPVBInFinalState(pvb) {\n\t\t\treturn false\n\t\t}\n\n\t\tpvb.Status.Phase = velerov1api.PodVolumeBackupPhaseCanceled\n\t\tif pvb.Status.StartTimestamp.IsZero() {\n\t\t\tpvb.Status.StartTimestamp = &metav1.Time{Time: r.clock.Now()}\n\t\t}\n\t\tpvb.Status.CompletionTimestamp = &metav1.Time{Time: r.clock.Now()}\n\n\t\tdelete(pvb.Labels, exposer.ExposeOnGoingLabel)\n\n\t\treturn true\n\t}); err != nil {\n\t\tlog.WithError(err).Error(\"error updating PVB status on cancel\")\n\t} else {\n\t\tdelete(r.cancelledPVB, pvb.Name)\n\t}\n}\n\nfunc (r *PodVolumeBackupReconciler) OnDataPathProgress(ctx context.Context, namespace string, pvbName string, progress *uploader.Progress) {\n\tlog := r.logger.WithField(\"pvb\", pvbName)\n\n\tif err := UpdatePVBWithRetry(ctx, r.client, types.NamespacedName{Namespace: namespace, Name: pvbName}, log, func(pvb *velerov1api.PodVolumeBackup) bool {\n\t\tpvb.Status.Progress = veleroapishared.DataMoveOperationProgress{TotalBytes: progress.TotalBytes, BytesDone: progress.BytesDone}\n\t\treturn true\n\t}); err != nil {\n\t\tlog.WithError(err).Error(\"Failed to update progress\")\n\t}\n}\n\n// SetupWithManager registers the PVB controller.\nfunc (r *PodVolumeBackupReconciler) SetupWithManager(mgr ctrl.Manager) error {\n\tgp := kube.NewGenericEventPredicate(func(object client.Object) bool {\n\t\tpvb := object.(*velerov1api.PodVolumeBackup)\n\n\t\tif _, err := uploader.ValidateUploaderType(pvb.Spec.UploaderType); err != nil {\n\t\t\treturn false\n\t\t}\n\n\t\tif pvb.Status.Phase == velerov1api.PodVolumeBackupPhaseAccepted {\n\t\t\treturn true\n\t\t}\n\n\t\tif pvb.Spec.Cancel && !isPVBInFinalState(pvb) {\n\t\t\treturn true\n\t\t}\n\n\t\tif isPVBInFinalState(pvb) && !pvb.DeletionTimestamp.IsZero() {\n\t\t\treturn true\n\t\t}\n\n\t\treturn false\n\t})\n\n\ts := kube.NewPeriodicalEnqueueSource(r.logger.WithField(\"controller\", constant.ControllerPodVolumeBackup), r.client, &velerov1api.PodVolumeBackupList{}, preparingMonitorFrequency, kube.PeriodicalEnqueueSourceOption{\n\t\tPredicates: []predicate.Predicate{gp},\n\t})\n\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\tFor(&velerov1api.PodVolumeBackup{}).\n\t\tWatchesRawSource(s).\n\t\tWatches(&corev1api.Pod{}, kube.EnqueueRequestsFromMapUpdateFunc(r.findPVBForPod),\n\t\t\tbuilder.WithPredicates(predicate.Funcs{\n\t\t\t\tUpdateFunc: func(ue event.UpdateEvent) bool {\n\t\t\t\t\tnewObj := ue.ObjectNew.(*corev1api.Pod)\n\n\t\t\t\t\tif _, ok := newObj.Labels[velerov1api.PVBLabel]; !ok {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\n\t\t\t\t\tif newObj.Spec.NodeName == \"\" {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\n\t\t\t\t\treturn true\n\t\t\t\t},\n\t\t\t\tCreateFunc: func(event.CreateEvent) bool {\n\t\t\t\t\treturn false\n\t\t\t\t},\n\t\t\t\tDeleteFunc: func(de event.DeleteEvent) bool {\n\t\t\t\t\treturn false\n\t\t\t\t},\n\t\t\t\tGenericFunc: func(ge event.GenericEvent) bool {\n\t\t\t\t\treturn false\n\t\t\t\t},\n\t\t\t})).\n\t\tComplete(r)\n}\n\nfunc (r *PodVolumeBackupReconciler) findPVBForPod(ctx context.Context, podObj client.Object) []reconcile.Request {\n\tpod := podObj.(*corev1api.Pod)\n\tpvb, err := findPVBByPod(r.client, *pod)\n\n\tlog := r.logger.WithField(\"pod\", pod.Name)\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"unable to get PVB\")\n\t\treturn []reconcile.Request{}\n\t} else if pvb == nil {\n\t\tlog.Error(\"get empty PVB\")\n\t\treturn []reconcile.Request{}\n\t}\n\tlog = log.WithFields(logrus.Fields{\n\t\t\"PVB\": pvb.Name,\n\t})\n\n\tif pvb.Status.Phase != velerov1api.PodVolumeBackupPhaseAccepted {\n\t\treturn []reconcile.Request{}\n\t}\n\n\tif pod.Status.Phase == corev1api.PodRunning {\n\t\tlog.Info(\"Preparing PVB\")\n\n\t\tif err = UpdatePVBWithRetry(context.Background(), r.client, types.NamespacedName{Namespace: pvb.Namespace, Name: pvb.Name}, log,\n\t\t\tfunc(pvb *velerov1api.PodVolumeBackup) bool {\n\t\t\t\tif isPVBInFinalState(pvb) {\n\t\t\t\t\tlog.Warnf(\"PVB %s is terminated, abort setting it to prepared\", pvb.Name)\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tpvb.Status.Phase = velerov1api.PodVolumeBackupPhasePrepared\n\t\t\t\treturn true\n\t\t\t}); err != nil {\n\t\t\tlog.WithError(err).Warn(\"Failed to update PVB, prepare will halt for this PVB\")\n\t\t\treturn []reconcile.Request{}\n\t\t}\n\t} else if unrecoverable, reason := kube.IsPodUnrecoverable(pod, log); unrecoverable {\n\t\terr := UpdatePVBWithRetry(context.Background(), r.client, types.NamespacedName{Namespace: pvb.Namespace, Name: pvb.Name}, log,\n\t\t\tfunc(pvb *velerov1api.PodVolumeBackup) bool {\n\t\t\t\tif pvb.Spec.Cancel {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tpvb.Spec.Cancel = true\n\t\t\t\tpvb.Status.Message = fmt.Sprintf(\"Cancel PVB because the exposing pod %s/%s is in abnormal status for reason %s\", pod.Namespace, pod.Name, reason)\n\n\t\t\t\treturn true\n\t\t\t})\n\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Warn(\"failed to cancel PVB, and it will wait for prepare timeout\")\n\t\t\treturn []reconcile.Request{}\n\t\t}\n\t\tlog.Infof(\"Exposed pod is in abnormal status(reason %s) and PVB is marked as cancel\", reason)\n\t} else {\n\t\treturn []reconcile.Request{}\n\t}\n\n\trequest := reconcile.Request{\n\t\tNamespacedName: types.NamespacedName{\n\t\t\tNamespace: pvb.Namespace,\n\t\t\tName: pvb.Name,\n\t\t},\n\t}\n\treturn []reconcile.Request{request}\n}\n\nfunc (r *PodVolumeBackupReconciler) errorOut(ctx context.Context, pvb *velerov1api.PodVolumeBackup, err error, msg string, log logrus.FieldLogger) (ctrl.Result, error) {\n\tr.exposer.CleanUp(ctx, getPVBOwnerObject(pvb))\n\n\t_ = UpdatePVBStatusToFailed(ctx, r.client, pvb, err, msg, r.clock.Now(), log)\n\n\treturn ctrl.Result{}, err\n}\n\nfunc UpdatePVBStatusToFailed(ctx context.Context, c client.Client, pvb *velerov1api.PodVolumeBackup, errOut error, msg string, time time.Time, log logrus.FieldLogger) error {\n\tlog.Info(\"update PVB status to Failed\")\n\n\tif patchErr := UpdatePVBWithRetry(context.Background(), c, types.NamespacedName{Namespace: pvb.Namespace, Name: pvb.Name}, log,\n\t\tfunc(pvb *velerov1api.PodVolumeBackup) bool {\n\t\t\tif isPVBInFinalState(pvb) {\n\t\t\t\treturn false\n\t\t\t}\n\n\t\t\tpvb.Status.Phase = velerov1api.PodVolumeBackupPhaseFailed\n\t\t\tpvb.Status.CompletionTimestamp = &metav1.Time{Time: time}\n\t\t\tif dataPathError, ok := errOut.(datapath.DataPathError); ok {\n\t\t\t\tpvb.Status.SnapshotID = dataPathError.GetSnapshotID()\n\t\t\t}\n\t\t\tif len(strings.TrimSpace(msg)) == 0 {\n\t\t\t\tpvb.Status.Message = errOut.Error()\n\t\t\t} else {\n\t\t\t\tpvb.Status.Message = errors.WithMessage(errOut, msg).Error()\n\t\t\t}\n\t\t\tif pvb.Status.StartTimestamp.IsZero() {\n\t\t\t\tpvb.Status.StartTimestamp = &metav1.Time{Time: time}\n\t\t\t}\n\n\t\t\tdelete(pvb.Labels, exposer.ExposeOnGoingLabel)\n\n\t\t\treturn true\n\t\t}); patchErr != nil {\n\t\tlog.WithError(patchErr).Warn(\"error updating PVB status\")\n\t}\n\n\treturn errOut\n}\n\nfunc (r *PodVolumeBackupReconciler) closeDataPath(ctx context.Context, pvbName string) {\n\tasyncBR := r.dataPathMgr.GetAsyncBR(pvbName)\n\tif asyncBR != nil {\n\t\tasyncBR.Close(ctx)\n\t}\n\n\tr.dataPathMgr.RemoveAsyncBR(pvbName)\n}\n\nfunc (r *PodVolumeBackupReconciler) setupExposeParam(pvb *velerov1api.PodVolumeBackup) exposer.PodVolumeExposeParam {\n\tlog := r.logger.WithField(\"PVB\", pvb.Name)\n\n\tnodeOS, err := kube.GetNodeOS(context.Background(), pvb.Spec.Node, r.kubeClient.CoreV1())\n\tif err != nil {\n\t\tlog.WithError(err).Warnf(\"Failed to get nodeOS for node %s, use linux node-agent for hosting pod labels, annotations and tolerations\", pvb.Spec.Node)\n\t}\n\n\thostingPodLabels := map[string]string{velerov1api.PVBLabel: pvb.Name}\n\tif len(r.podLabels) > 0 {\n\t\tfor k, v := range r.podLabels {\n\t\t\thostingPodLabels[k] = v\n\t\t}\n\t} else {\n\t\tfor _, k := range util.ThirdPartyLabels {\n\t\t\tif v, err := nodeagent.GetLabelValue(context.Background(), r.kubeClient, pvb.Namespace, k, nodeOS); err != nil {\n\t\t\t\tif err != nodeagent.ErrNodeAgentLabelNotFound {\n\t\t\t\t\tlog.WithError(err).Warnf(\"Failed to check node-agent label, skip adding host pod label %s\", k)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\thostingPodLabels[k] = v\n\t\t\t}\n\t\t}\n\t}\n\n\thostingPodAnnotation := map[string]string{}\n\tif len(r.podAnnotations) > 0 {\n\t\tfor k, v := range r.podAnnotations {\n\t\t\thostingPodAnnotation[k] = v\n\t\t}\n\t} else {\n\t\tfor _, k := range util.ThirdPartyAnnotations {\n\t\t\tif v, err := nodeagent.GetAnnotationValue(context.Background(), r.kubeClient, pvb.Namespace, k, nodeOS); err != nil {\n\t\t\t\tif err != nodeagent.ErrNodeAgentAnnotationNotFound {\n\t\t\t\t\tlog.WithError(err).Warnf(\"Failed to check node-agent annotation, skip adding host pod annotation %s\", k)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\thostingPodAnnotation[k] = v\n\t\t\t}\n\t\t}\n\t}\n\n\thostingPodTolerations := []corev1api.Toleration{}\n\tfor _, k := range util.ThirdPartyTolerations {\n\t\tif v, err := nodeagent.GetToleration(context.Background(), r.kubeClient, pvb.Namespace, k, nodeOS); err != nil {\n\t\t\tif err != nodeagent.ErrNodeAgentTolerationNotFound {\n\t\t\t\tlog.WithError(err).Warnf(\"Failed to check node-agent toleration, skip adding host pod toleration %s\", k)\n\t\t\t}\n\t\t} else {\n\t\t\thostingPodTolerations = append(hostingPodTolerations, *v)\n\t\t}\n\t}\n\n\treturn exposer.PodVolumeExposeParam{\n\t\tType: exposer.PodVolumeExposeTypeBackup,\n\t\tClientNamespace: pvb.Spec.Pod.Namespace,\n\t\tClientPodName: pvb.Spec.Pod.Name,\n\t\tClientPodVolume: pvb.Spec.Volume,\n\t\tHostingPodLabels: hostingPodLabels,\n\t\tHostingPodAnnotations: hostingPodAnnotation,\n\t\tHostingPodTolerations: hostingPodTolerations,\n\t\tOperationTimeout: r.resourceTimeout,\n\t\tResources: r.podResources,\n\t\t// Priority class name for the data mover pod, retrieved from node-agent-configmap\n\t\tPriorityClassName: r.dataMovePriorityClass,\n\t\tPrivileged: r.privileged,\n\t}\n}\n\nfunc getPVBOwnerObject(pvb *velerov1api.PodVolumeBackup) corev1api.ObjectReference {\n\treturn corev1api.ObjectReference{\n\t\tKind: pvb.Kind,\n\t\tNamespace: pvb.Namespace,\n\t\tName: pvb.Name,\n\t\tUID: pvb.UID,\n\t\tAPIVersion: pvb.APIVersion,\n\t}\n}\n\nfunc findPVBByPod(client client.Client, pod corev1api.Pod) (*velerov1api.PodVolumeBackup, error) {\n\tif label, exist := pod.Labels[velerov1api.PVBLabel]; exist {\n\t\tpvb := &velerov1api.PodVolumeBackup{}\n\t\terr := client.Get(context.Background(), types.NamespacedName{\n\t\t\tNamespace: pod.Namespace,\n\t\t\tName: label,\n\t\t}, pvb)\n\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"error to find PVB by pod %s/%s\", pod.Namespace, pod.Name)\n\t\t}\n\t\treturn pvb, nil\n\t}\n\treturn nil, nil\n}\n\nfunc isPVBInFinalState(pvb *velerov1api.PodVolumeBackup) bool {\n\treturn pvb.Status.Phase == velerov1api.PodVolumeBackupPhaseFailed ||\n\t\tpvb.Status.Phase == velerov1api.PodVolumeBackupPhaseCanceled ||\n\t\tpvb.Status.Phase == velerov1api.PodVolumeBackupPhaseCompleted\n}\n\nfunc UpdatePVBWithRetry(ctx context.Context, client client.Client, namespacedName types.NamespacedName, log logrus.FieldLogger, updateFunc func(*velerov1api.PodVolumeBackup) bool) error {\n\treturn wait.PollUntilContextCancel(ctx, time.Millisecond*100, true, func(ctx context.Context) (bool, error) {\n\t\tpvb := &velerov1api.PodVolumeBackup{}\n\t\tif err := client.Get(ctx, namespacedName, pvb); err != nil {\n\t\t\treturn false, errors.Wrap(err, \"getting PVB\")\n\t\t}\n\n\t\tif updateFunc(pvb) {\n\t\t\terr := client.Update(ctx, pvb)\n\t\t\tif err != nil {\n\t\t\t\tif apierrors.IsConflict(err) {\n\t\t\t\t\tlog.Debugf(\"failed to update PVB for %s/%s and will retry it\", pvb.Namespace, pvb.Name)\n\t\t\t\t\treturn false, nil\n\t\t\t\t} else {\n\t\t\t\t\treturn false, errors.Wrapf(err, \"error updating PVB with error %s/%s\", pvb.Namespace, pvb.Name)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\treturn true, nil\n\t})\n}\n\nvar funcResumeCancellablePVB = (*PodVolumeBackupReconciler).resumeCancellableDataPath\n\nfunc (r *PodVolumeBackupReconciler) AttemptPVBResume(ctx context.Context, logger *logrus.Entry, ns string) error {\n\tpvbs := &velerov1api.PodVolumeBackupList{}\n\tif err := r.client.List(ctx, pvbs, &client.ListOptions{Namespace: ns}); err != nil {\n\t\tr.logger.WithError(errors.WithStack(err)).Error(\"failed to list PVBs\")\n\t\treturn errors.Wrapf(err, \"error to list PVBs\")\n\t}\n\n\tfor i := range pvbs.Items {\n\t\tpvb := &pvbs.Items[i]\n\t\tif pvb.Status.Phase == velerov1api.PodVolumeBackupPhaseInProgress {\n\t\t\tif pvb.Spec.Node != r.nodeName {\n\t\t\t\tlogger.WithField(\"PVB\", pvb.Name).WithField(\"current node\", r.nodeName).Infof(\"PVB should be resumed by another node %s\", pvb.Spec.Node)\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\terr := funcResumeCancellablePVB(r, ctx, pvb, logger)\n\t\t\tif err == nil {\n\t\t\t\tlogger.WithField(\"PVB\", pvb.Name).WithField(\"current node\", r.nodeName).Info(\"Completed to resume in progress PVB\")\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tlogger.WithField(\"PVB\", pvb.GetName()).WithError(err).Warn(\"Failed to resume data path for PVB, have to cancel it\")\n\n\t\t\tresumeErr := err\n\t\t\terr = UpdatePVBWithRetry(ctx, r.client, types.NamespacedName{Namespace: pvb.Namespace, Name: pvb.Name}, logger.WithField(\"PVB\", pvb.Name),\n\t\t\t\tfunc(pvb *velerov1api.PodVolumeBackup) bool {\n\t\t\t\t\tif pvb.Spec.Cancel {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\n\t\t\t\t\tpvb.Spec.Cancel = true\n\t\t\t\t\tpvb.Status.Message = fmt.Sprintf(\"Resume InProgress PVB failed with error %v, mark it as cancel\", resumeErr)\n\n\t\t\t\t\treturn true\n\t\t\t\t})\n\t\t\tif err != nil {\n\t\t\t\tlogger.WithField(\"PVB\", pvb.GetName()).WithError(errors.WithStack(err)).Error(\"Failed to trigger PVB cancel\")\n\t\t\t}\n\t\t} else if !isPVBInFinalState(pvb) {\n\t\t\tlogger.WithField(\"PVB\", pvb.GetName()).Infof(\"find a PVB with status %s\", pvb.Status.Phase)\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (r *PodVolumeBackupReconciler) resumeCancellableDataPath(ctx context.Context, pvb *velerov1api.PodVolumeBackup, log logrus.FieldLogger) error {\n\tlog.Info(\"Resume cancelable PVB\")\n\n\tres, err := r.exposer.GetExposed(ctx, getPVBOwnerObject(pvb), r.client, r.nodeName, r.resourceTimeout)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to get exposed PVB %s\", pvb.Name)\n\t}\n\n\tif res == nil {\n\t\treturn errors.Errorf(\"no expose result is available for the current node for PVB %s\", pvb.Name)\n\t}\n\n\tcallbacks := datapath.Callbacks{\n\t\tOnCompleted: r.OnDataPathCompleted,\n\t\tOnFailed: r.OnDataPathFailed,\n\t\tOnCancelled: r.OnDataPathCancelled,\n\t\tOnProgress: r.OnDataPathProgress,\n\t}\n\n\tasyncBR, err := r.dataPathMgr.CreateMicroServiceBRWatcher(ctx, r.client, r.kubeClient, r.mgr, datapath.TaskTypeBackup, pvb.Name, pvb.Namespace, res.ByPod.HostingPod.Name, res.ByPod.HostingContainer, pvb.Name, callbacks, true, log)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to create asyncBR watcher for PVB %s\", pvb.Name)\n\t}\n\n\tresumeComplete := false\n\tdefer func() {\n\t\tif !resumeComplete {\n\t\t\tr.closeDataPath(ctx, pvb.Name)\n\t\t}\n\t}()\n\n\tif err := asyncBR.Init(ctx, nil); err != nil {\n\t\treturn errors.Wrapf(err, \"error to init asyncBR watcher for PVB %s\", pvb.Name)\n\t}\n\n\tif err := asyncBR.StartBackup(datapath.AccessPoint{\n\t\tByPath: res.ByPod.VolumeName,\n\t}, pvb.Spec.UploaderSettings, nil); err != nil {\n\t\treturn errors.Wrapf(err, \"error to resume asyncBR watcher for PVB %s\", pvb.Name)\n\t}\n\n\tresumeComplete = true\n\n\tlog.Infof(\"asyncBR is resumed for PVB %s\", pvb.Name)\n\n\treturn nil\n}\n" }, { "path": "pkg/controller/pod_volume_backup_controller_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/client-go/kubernetes\"\n\tclientgofake \"k8s.io/client-go/kubernetes/fake\"\n\t\"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\t\"sigs.k8s.io/controller-runtime/pkg/manager\"\n\t\"sigs.k8s.io/controller-runtime/pkg/reconcile\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\tdatapathmocks \"github.com/vmware-tanzu/velero/pkg/datapath/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/exposer\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nconst pvbName = \"pvb-1\"\n\nfunc initPVBReconciler(needError ...bool) (*PodVolumeBackupReconciler, error) {\n\tvar errs = make([]error, 6)\n\tfor k, isError := range needError {\n\t\tif k == 0 && isError {\n\t\t\terrs[0] = fmt.Errorf(\"Get error\")\n\t\t} else if k == 1 && isError {\n\t\t\terrs[1] = fmt.Errorf(\"Create error\")\n\t\t} else if k == 2 && isError {\n\t\t\terrs[2] = fmt.Errorf(\"Update error\")\n\t\t} else if k == 3 && isError {\n\t\t\terrs[3] = fmt.Errorf(\"Patch error\")\n\t\t} else if k == 4 && isError {\n\t\t\terrs[4] = apierrors.NewConflict(velerov1api.Resource(\"podvolumebackup\"), pvbName, errors.New(\"conflict\"))\n\t\t} else if k == 5 && isError {\n\t\t\terrs[5] = fmt.Errorf(\"List error\")\n\t\t}\n\t}\n\n\treturn initPVBReconcilerWithError(errs...)\n}\n\nfunc initPVBReconcilerWithError(needError ...error) (*PodVolumeBackupReconciler, error) {\n\tdaemonSet := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"velero\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t\tAPIVersion: appsv1api.SchemeGroupVersion.String(),\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tImage: \"fake-image\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tnode := builder.ForNode(\"fake-node\").Labels(map[string]string{kube.NodeOSLabel: kube.NodeOSLinux}).Result()\n\n\tdataPathMgr := datapath.NewManager(1)\n\n\tscheme := runtime.NewScheme()\n\terr := velerov1api.AddToScheme(scheme)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\terr = corev1api.AddToScheme(scheme)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfakeClient := &FakeClient{\n\t\tClient: fake.NewClientBuilder().WithScheme(scheme).Build(),\n\t}\n\n\tfor k := range needError {\n\t\tif k == 0 {\n\t\t\tfakeClient.getError = needError[0]\n\t\t} else if k == 1 {\n\t\t\tfakeClient.createError = needError[1]\n\t\t} else if k == 2 {\n\t\t\tfakeClient.updateError = needError[2]\n\t\t} else if k == 3 {\n\t\t\tfakeClient.patchError = needError[3]\n\t\t} else if k == 4 {\n\t\t\tfakeClient.updateConflict = needError[4]\n\t\t} else if k == 5 {\n\t\t\tfakeClient.listError = needError[5]\n\t\t}\n\t}\n\n\tfakeKubeClient := clientgofake.NewSimpleClientset(daemonSet, node)\n\n\treturn NewPodVolumeBackupReconciler(\n\t\tfakeClient,\n\t\tnil,\n\t\tfakeKubeClient,\n\t\tdataPathMgr,\n\t\tnil,\n\t\t\"test-node\",\n\t\ttime.Minute*5,\n\t\ttime.Minute,\n\t\tcorev1api.ResourceRequirements{},\n\t\tmetrics.NewServerMetrics(),\n\t\tvelerotest.NewLogger(),\n\t\t\"\", // dataMovePriorityClass\n\t\tfalse, // privileged\n\t\tnil, // podLabels\n\t\tnil, // podAnnotations\n\t), nil\n}\n\nfunc pvbBuilder() *builder.PodVolumeBackupBuilder {\n\treturn builder.ForPodVolumeBackup(velerov1api.DefaultNamespace, pvbName).BackupStorageLocation(\"bsl-loc\")\n}\n\ntype fakePvbExposer struct {\n\tkubeClient client.Client\n\tclock clock.WithTickerAndDelayedExecution\n\tpeekErr error\n\texposeErr error\n\tgetErr error\n\tgetNil bool\n}\n\nfunc (f *fakePvbExposer) Expose(ctx context.Context, ownerObject corev1api.ObjectReference, param exposer.PodVolumeExposeParam) error {\n\tif f.exposeErr != nil {\n\t\treturn f.exposeErr\n\t}\n\n\treturn nil\n}\n\nfunc (f *fakePvbExposer) GetExposed(context.Context, corev1api.ObjectReference, client.Client, string, time.Duration) (*exposer.ExposeResult, error) {\n\tif f.getErr != nil {\n\t\treturn nil, f.getErr\n\t}\n\n\tif f.getNil {\n\t\treturn nil, nil\n\t}\n\n\tpod := &corev1api.Pod{}\n\n\tnodeOS := \"linux\"\n\tpNodeOS := &nodeOS\n\n\treturn &exposer.ExposeResult{ByPod: exposer.ExposeByPod{HostingPod: pod, VolumeName: pvbName, NodeOS: pNodeOS}}, nil\n}\n\nfunc (f *fakePvbExposer) PeekExposed(ctx context.Context, ownerObject corev1api.ObjectReference) error {\n\treturn f.peekErr\n}\n\nfunc (f *fakePvbExposer) DiagnoseExpose(context.Context, corev1api.ObjectReference) string {\n\treturn \"\"\n}\n\nfunc (f *fakePvbExposer) CleanUp(context.Context, corev1api.ObjectReference) {\n}\n\nfunc TestPVBReconcile(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvb *velerov1api.PodVolumeBackup\n\t\tnotCreatePvb bool\n\t\tneedDelete bool\n\t\tsportTime *metav1.Time\n\t\tpod *corev1api.Pod\n\t\tdataMgr *datapath.Manager\n\t\tneedCreateFSBR bool\n\t\tneedExclusiveUpdateError error\n\t\tneedMockExposer bool\n\t\texpected *velerov1api.PodVolumeBackup\n\t\texpectDeleted bool\n\t\texpectCancelRecord bool\n\t\tneedErrs []bool\n\t\tpeekErr error\n\t\texposeErr error\n\t\tgetExposeErr error\n\t\tgetExposeNil bool\n\t\tfsBRInitErr error\n\t\tfsBRStartErr error\n\t\tconstrained bool\n\t\texpectedErr string\n\t\texpectedResult *ctrl.Result\n\t\texpectDataPath bool\n\t}{\n\t\t{\n\t\t\tname: \"pvb not found\",\n\t\t\tpvb: pvbBuilder().Result(),\n\t\t\tnotCreatePvb: true,\n\t\t},\n\t\t{\n\t\t\tname: \"pvb not created in velero default namespace\",\n\t\t\tpvb: builder.ForPodVolumeBackup(\"test-ns\", pvbName).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"get pvb fail\",\n\t\t\tpvb: pvbBuilder().Result(),\n\t\t\tneedErrs: []bool{true, false, false, false},\n\t\t\texpectedErr: \"getting PVB: Get error\",\n\t\t},\n\t\t{\n\t\t\tname: \"add finalizer to pvb\",\n\t\t\tpvb: pvbBuilder().Result(),\n\t\t\texpected: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"add finalizer to pvb failed\",\n\t\t\tpvb: pvbBuilder().Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\texpectedErr: \"error updating PVB with error velero/pvb-1: Update error\",\n\t\t},\n\t\t{\n\t\t\tname: \"pvb is under deletion\",\n\t\t\tpvb: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t\tneedDelete: true,\n\t\t\texpected: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"pvb is under deletion but cancel failed\",\n\t\t\tpvb: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\tneedDelete: true,\n\t\t\texpectedErr: \"error updating PVB with error velero/pvb-1: Update error\",\n\t\t},\n\t\t{\n\t\t\tname: \"pvb is under deletion and in terminal state\",\n\t\t\tpvb: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Phase(velerov1api.PodVolumeBackupPhaseFailed).Result(),\n\t\t\tsportTime: &metav1.Time{Time: time.Now()},\n\t\t\tneedDelete: true,\n\t\t\texpectDeleted: true,\n\t\t},\n\t\t{\n\t\t\tname: \"pvb is under deletion and in terminal state, but remove finalizer failed\",\n\t\t\tpvb: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Phase(velerov1api.PodVolumeBackupPhaseFailed).Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\tneedDelete: true,\n\t\t\texpectedErr: \"error updating PVB with error velero/pvb-1: Update error\",\n\t\t},\n\t\t{\n\t\t\tname: \"delay cancel negative for others\",\n\t\t\tpvb: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Phase(velerov1api.PodVolumeBackupPhasePrepared).Result(),\n\t\t\tsportTime: &metav1.Time{Time: time.Now()},\n\t\t\texpectCancelRecord: true,\n\t\t},\n\t\t{\n\t\t\tname: \"delay cancel negative for inProgress\",\n\t\t\tpvb: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Phase(velerov1api.PodVolumeBackupPhaseInProgress).Result(),\n\t\t\tsportTime: &metav1.Time{Time: time.Now().Add(-time.Minute * 58)},\n\t\t\texpectCancelRecord: true,\n\t\t},\n\t\t{\n\t\t\tname: \"delay cancel affirmative for others\",\n\t\t\tpvb: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Phase(velerov1api.PodVolumeBackupPhasePrepared).Result(),\n\t\t\tsportTime: &metav1.Time{Time: time.Now().Add(-time.Minute * 5)},\n\t\t\texpected: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Phase(velerov1api.PodVolumeBackupPhaseCanceled).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"delay cancel affirmative for inProgress\",\n\t\t\tpvb: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Phase(velerov1api.PodVolumeBackupPhaseInProgress).Result(),\n\t\t\tsportTime: &metav1.Time{Time: time.Now().Add(-time.Hour)},\n\t\t\texpected: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Phase(velerov1api.PodVolumeBackupPhaseCanceled).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"delay cancel failed\",\n\t\t\tpvb: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Phase(velerov1api.PodVolumeBackupPhaseInProgress).Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\tsportTime: &metav1.Time{Time: time.Now().Add(-time.Hour)},\n\t\t\texpected: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Phase(velerov1api.PodVolumeBackupPhaseInProgress).Result(),\n\t\t\texpectCancelRecord: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Unknown pvb status\",\n\t\t\tpvb: pvbBuilder().Phase(\"Unknown\").Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"new pvb but constrained\",\n\t\t\tpvb: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tconstrained: true,\n\t\t\texpected: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t\texpectedResult: &ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5},\n\t\t},\n\t\t{\n\t\t\tname: \"new pvb but accept failed\",\n\t\t\tpvb: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\texpected: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t\texpectedErr: \"error accepting PVB pvb-1: error updating PVB with error velero/pvb-1: Update error\",\n\t\t},\n\t\t{\n\t\t\tname: \"pvb is cancel on accepted\",\n\t\t\tpvb: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Cancel(true).Result(),\n\t\t\texpected: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Phase(velerov1api.PodVolumeBackupPhaseCanceled).Result(),\n\t\t\texpectCancelRecord: true,\n\t\t},\n\t\t{\n\t\t\tname: \"pvb expose failed\",\n\t\t\tpvb: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tneedMockExposer: true,\n\t\t\texposeErr: errors.New(\"fake-expose-error\"),\n\t\t\texpected: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Phase(velerov1api.PodVolumeBackupPhaseFailed).Message(\"error to expose PVB\").Result(),\n\t\t\texpectedErr: \"fake-expose-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"pvb succeeds for accepted\",\n\t\t\tpvb: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tneedMockExposer: true,\n\t\t\texpected: pvbBuilder().Finalizers([]string{PodVolumeFinalizer}).Phase(velerov1api.PodVolumeBackupPhaseAccepted).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"prepare timeout on accepted\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseAccepted).Finalizers([]string{PodVolumeFinalizer}).AcceptedTimestamp(&metav1.Time{Time: time.Now().Add(-time.Minute * 30)}).Result(),\n\t\t\texpected: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseFailed).Finalizers([]string{PodVolumeFinalizer}).Phase(velerov1api.PodVolumeBackupPhaseFailed).Message(\"timeout on preparing PVB\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"peek error on accepted\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseAccepted).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t\tneedMockExposer: true,\n\t\t\tpeekErr: errors.New(\"fake-peak-error\"),\n\t\t\texpected: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseCanceled).Finalizers([]string{PodVolumeFinalizer}).Phase(velerov1api.PodVolumeBackupPhaseCanceled).Message(\"found a PVB velero/pvb-1 with expose error: fake-peak-error. mark it as cancel\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"cancel on prepared\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhasePrepared).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Cancel(true).Result(),\n\t\t\texpected: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseCanceled).Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Phase(velerov1api.PodVolumeBackupPhaseCanceled).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Failed to get pvb expose on prepared\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhasePrepared).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tneedMockExposer: true,\n\t\t\tgetExposeErr: errors.New(\"fake-get-error\"),\n\t\t\texpected: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseFailed).Finalizers([]string{PodVolumeFinalizer}).Message(\"exposed PVB is not ready: fake-get-error\").Result(),\n\t\t\texpectedErr: \"fake-get-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"Get nil restore expose on prepared\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhasePrepared).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tneedMockExposer: true,\n\t\t\tgetExposeNil: true,\n\t\t\texpected: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseFailed).Finalizers([]string{PodVolumeFinalizer}).Message(\"exposed PVB is not ready\").Result(),\n\t\t\texpectedErr: \"no expose result is available for the current node\",\n\t\t},\n\t\t{\n\t\t\tname: \"Error in data path is concurrent limited\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhasePrepared).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tneedMockExposer: true,\n\t\t\tdataMgr: datapath.NewManager(0),\n\t\t\texpectedResult: &ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5},\n\t\t},\n\t\t{\n\t\t\tname: \"data path init error\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhasePrepared).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tneedMockExposer: true,\n\t\t\tfsBRInitErr: errors.New(\"fake-data-path-init-error\"),\n\t\t\texpected: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseFailed).Finalizers([]string{PodVolumeFinalizer}).Message(\"error initializing data path\").Result(),\n\t\t\texpectedErr: \"error initializing asyncBR: fake-data-path-init-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"Unable to update status to in progress for data upload\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhasePrepared).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tneedMockExposer: true,\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\texpected: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhasePrepared).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t\texpectedResult: &ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5},\n\t\t},\n\t\t{\n\t\t\tname: \"data path start error\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhasePrepared).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tneedMockExposer: true,\n\t\t\tfsBRStartErr: errors.New(\"fake-data-path-start-error\"),\n\t\t\texpected: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseFailed).Finalizers([]string{PodVolumeFinalizer}).Message(\"error starting data path\").Result(),\n\t\t\texpectedErr: \"error starting async backup for pod , volume pvb-1: fake-data-path-start-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"Prepare succeeds\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhasePrepared).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tneedMockExposer: true,\n\t\t\texpected: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseInProgress).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t\texpectDataPath: true,\n\t\t},\n\t\t{\n\t\t\tname: \"In progress pvb is not handled by the current node\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseInProgress).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t\texpected: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseInProgress).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"In progress pvb is not set as cancel\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseInProgress).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\texpected: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseInProgress).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Cancel pvb in progress with empty FSBR\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseInProgress).Cancel(true).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\texpected: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseCanceled).Cancel(true).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Cancel pvb in progress and patch pvb error\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseInProgress).Cancel(true).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\tneedCreateFSBR: true,\n\t\t\texpected: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseInProgress).Cancel(true).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t\texpectedErr: \"error updating PVB with error velero/pvb-1: Update error\",\n\t\t\texpectCancelRecord: true,\n\t\t\texpectDataPath: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Cancel pvb in progress succeeds\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseInProgress).Cancel(true).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tneedCreateFSBR: true,\n\t\t\texpected: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseCanceling).Cancel(true).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t\texpectDataPath: true,\n\t\t\texpectCancelRecord: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tr, err := initPVBReconciler(test.needErrs...)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tif !test.notCreatePvb {\n\t\t\t\terr = r.client.Create(t.Context(), test.pvb)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif test.needDelete {\n\t\t\t\terr = r.client.Delete(t.Context(), test.pvb)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif test.pod != nil {\n\t\t\t\terr = r.client.Create(ctx, test.pod)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif test.dataMgr != nil {\n\t\t\t\tr.dataPathMgr = test.dataMgr\n\t\t\t} else {\n\t\t\t\tr.dataPathMgr = datapath.NewManager(1)\n\t\t\t}\n\n\t\t\tif test.sportTime != nil {\n\t\t\t\tr.cancelledPVB[test.pvb.Name] = test.sportTime.Time\n\t\t\t}\n\n\t\t\tif test.constrained {\n\t\t\t\tr.vgdpCounter = &exposer.VgdpCounter{}\n\t\t\t}\n\n\t\t\tif test.needMockExposer {\n\t\t\t\tr.exposer = &fakePvbExposer{r.client, r.clock, test.peekErr, test.exposeErr, test.getExposeErr, test.getExposeNil}\n\t\t\t}\n\n\t\t\tfuncExclusiveUpdatePodVolumeBackup = exclusiveUpdatePodVolumeBackup\n\t\t\tif test.needExclusiveUpdateError != nil {\n\t\t\t\tfuncExclusiveUpdatePodVolumeBackup = func(context.Context, client.Client, *velerov1api.PodVolumeBackup, func(*velerov1api.PodVolumeBackup)) (bool, error) {\n\t\t\t\t\treturn false, test.needExclusiveUpdateError\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tdatapath.MicroServiceBRWatcherCreator = func(client.Client, kubernetes.Interface, manager.Manager, string, string, string, string, string, string, datapath.Callbacks, logrus.FieldLogger) datapath.AsyncBR {\n\t\t\t\treturn &fakeFSBR{\n\t\t\t\t\tkubeClient: r.client,\n\t\t\t\t\tclock: r.clock,\n\t\t\t\t\tinitErr: test.fsBRInitErr,\n\t\t\t\t\tstartErr: test.fsBRStartErr,\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif test.needCreateFSBR {\n\t\t\t\tif fsBR := r.dataPathMgr.GetAsyncBR(test.pvb.Name); fsBR == nil {\n\t\t\t\t\t_, err := r.dataPathMgr.CreateMicroServiceBRWatcher(ctx, r.client, nil, nil, datapath.TaskTypeBackup, test.pvb.Name, velerov1api.DefaultNamespace, \"\", \"\", \"\", datapath.Callbacks{OnCancelled: r.OnDataPathCancelled}, false, velerotest.NewLogger())\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tactualResult, err := r.Reconcile(ctx, ctrl.Request{\n\t\t\t\tNamespacedName: types.NamespacedName{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: test.pvb.Name,\n\t\t\t\t},\n\t\t\t})\n\n\t\t\tif test.expectedErr != \"\" {\n\t\t\t\trequire.EqualError(t, err, test.expectedErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif test.expectedResult != nil {\n\t\t\t\tassert.Equal(t, test.expectedResult.Requeue, actualResult.Requeue)\n\t\t\t\tassert.Equal(t, test.expectedResult.RequeueAfter, actualResult.RequeueAfter)\n\t\t\t}\n\n\t\t\tpvb := velerov1api.PodVolumeBackup{}\n\t\t\terr = r.client.Get(ctx, client.ObjectKey{\n\t\t\t\tName: test.pvb.Name,\n\t\t\t\tNamespace: test.pvb.Namespace,\n\t\t\t}, &pvb)\n\n\t\t\tif test.expected != nil || test.expectDeleted {\n\t\t\t\tif test.expectDeleted {\n\t\t\t\t\tassert.True(t, apierrors.IsNotFound(err))\n\t\t\t\t} else {\n\t\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\t\tassert.Equal(t, test.expected.Status.Phase, pvb.Status.Phase)\n\t\t\t\t\tassert.Contains(t, pvb.Status.Message, test.expected.Status.Message)\n\t\t\t\t\tassert.Equal(t, pvb.Finalizers, test.expected.Finalizers)\n\t\t\t\t\tassert.Equal(t, pvb.Spec.Cancel, test.expected.Spec.Cancel)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif !test.expectDataPath {\n\t\t\t\tassert.Nil(t, r.dataPathMgr.GetAsyncBR(test.pvb.Name))\n\t\t\t} else {\n\t\t\t\tassert.NotNil(t, r.dataPathMgr.GetAsyncBR(test.pvb.Name))\n\t\t\t}\n\n\t\t\tif test.expectCancelRecord {\n\t\t\t\tassert.Contains(t, r.cancelledPVB, test.pvb.Name)\n\t\t\t} else {\n\t\t\t\tassert.Empty(t, r.cancelledPVB)\n\t\t\t}\n\n\t\t\tif isPVBInFinalState(&pvb) || pvb.Status.Phase == velerov1api.PodVolumeBackupPhaseInProgress {\n\t\t\t\tassert.NotContains(t, pvb.Labels, exposer.ExposeOnGoingLabel)\n\t\t\t} else if pvb.Status.Phase == velerov1api.PodVolumeBackupPhaseAccepted {\n\t\t\t\tassert.Contains(t, pvb.Labels, exposer.ExposeOnGoingLabel)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestOnPVBCancelled(t *testing.T) {\n\tctx := t.Context()\n\tr, err := initPVBReconciler()\n\trequire.NoError(t, err)\n\tpvb := pvbBuilder().Result()\n\tnamespace := pvb.Namespace\n\tpvbName := pvb.Name\n\n\trequire.NoError(t, r.client.Create(ctx, pvb))\n\n\tr.OnDataPathCancelled(ctx, namespace, pvbName)\n\tupdatedPvb := &velerov1api.PodVolumeBackup{}\n\trequire.NoError(t, r.client.Get(ctx, types.NamespacedName{Name: pvbName, Namespace: namespace}, updatedPvb))\n\tassert.Equal(t, velerov1api.PodVolumeBackupPhaseCanceled, updatedPvb.Status.Phase)\n\tassert.False(t, updatedPvb.Status.CompletionTimestamp.IsZero())\n\tassert.False(t, updatedPvb.Status.StartTimestamp.IsZero())\n}\n\nfunc TestOnPVBProgress(t *testing.T) {\n\ttotalBytes := int64(1024)\n\tbytesDone := int64(512)\n\ttests := []struct {\n\t\tname string\n\t\tpvb *velerov1api.PodVolumeBackup\n\t\tprogress uploader.Progress\n\t\tneedErrs []bool\n\t}{\n\t\t{\n\t\t\tname: \"patch in progress phase success\",\n\t\t\tpvb: pvbBuilder().Result(),\n\t\t\tprogress: uploader.Progress{\n\t\t\t\tTotalBytes: totalBytes,\n\t\t\t\tBytesDone: bytesDone,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"failed to get pvb\",\n\t\t\tpvb: pvbBuilder().Result(),\n\t\t\tneedErrs: []bool{true, false, false, false},\n\t\t},\n\t\t{\n\t\t\tname: \"failed to patch pvb\",\n\t\t\tpvb: pvbBuilder().Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\n\t\t\tr, err := initPVBReconciler(test.needErrs...)\n\t\t\trequire.NoError(t, err)\n\t\t\tdefer func() {\n\t\t\t\tr.client.Delete(ctx, test.pvb, &client.DeleteOptions{})\n\t\t\t}()\n\n\t\t\tpvb := pvbBuilder().Result()\n\t\t\tnamespace := pvb.Namespace\n\t\t\tpvbName := pvb.Name\n\n\t\t\trequire.NoError(t, r.client.Create(t.Context(), pvb))\n\n\t\t\t// Create a Progress object\n\t\t\tprogress := &uploader.Progress{\n\t\t\t\tTotalBytes: totalBytes,\n\t\t\t\tBytesDone: bytesDone,\n\t\t\t}\n\n\t\t\tr.OnDataPathProgress(ctx, namespace, pvbName, progress)\n\t\t\tif len(test.needErrs) != 0 && !test.needErrs[0] {\n\t\t\t\tupdatedPvb := &velerov1api.PodVolumeBackup{}\n\t\t\t\trequire.NoError(t, r.client.Get(ctx, types.NamespacedName{Name: pvbName, Namespace: namespace}, updatedPvb))\n\t\t\t\tassert.Equal(t, test.progress.TotalBytes, updatedPvb.Status.Progress.TotalBytes)\n\t\t\t\tassert.Equal(t, test.progress.BytesDone, updatedPvb.Status.Progress.BytesDone)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestOnPvbFailed(t *testing.T) {\n\tctx := t.Context()\n\tr, err := initPVBReconciler()\n\trequire.NoError(t, err)\n\n\tpvb := pvbBuilder().Result()\n\tnamespace := pvb.Namespace\n\tpvbName := pvb.Name\n\n\trequire.NoError(t, r.client.Create(ctx, pvb))\n\n\tr.OnDataPathFailed(ctx, namespace, pvbName, fmt.Errorf(\"Failed to handle %v\", pvbName))\n\tupdatedPvb := &velerov1api.PodVolumeBackup{}\n\trequire.NoError(t, r.client.Get(ctx, types.NamespacedName{Name: pvbName, Namespace: namespace}, updatedPvb))\n\tassert.Equal(t, velerov1api.PodVolumeBackupPhaseFailed, updatedPvb.Status.Phase)\n\tassert.False(t, updatedPvb.Status.CompletionTimestamp.IsZero())\n\tassert.False(t, updatedPvb.Status.StartTimestamp.IsZero())\n}\n\nfunc TestOnPvbCompleted(t *testing.T) {\n\tctx := t.Context()\n\tr, err := initPVBReconciler()\n\trequire.NoError(t, err)\n\n\tnow := time.Now()\n\tpvb := pvbBuilder().StartTimestamp(&metav1.Time{Time: now.Add(-time.Minute)}).CompletionTimestamp(&metav1.Time{Time: now}).OwnerReference(metav1.OwnerReference{Name: \"test-backup\"}).Result()\n\tnamespace := pvb.Namespace\n\tpvbName := pvb.Name\n\n\trequire.NoError(t, r.client.Create(ctx, pvb))\n\n\tr.OnDataPathCompleted(ctx, namespace, pvbName, datapath.Result{})\n\tupdatedPvb := &velerov1api.PodVolumeBackup{}\n\trequire.NoError(t, r.client.Get(ctx, types.NamespacedName{Name: pvbName, Namespace: namespace}, updatedPvb))\n\tassert.Equal(t, velerov1api.PodVolumeBackupPhaseCompleted, updatedPvb.Status.Phase)\n\tassert.False(t, updatedPvb.Status.CompletionTimestamp.IsZero())\n}\n\nfunc TestFindPvbForPod(t *testing.T) {\n\tr, err := initPVBReconciler()\n\trequire.NoError(t, err)\n\ttests := []struct {\n\t\tname string\n\t\tpvb *velerov1api.PodVolumeBackup\n\t\tpod *corev1api.Pod\n\t\tcheckFunc func(*velerov1api.PodVolumeBackup, []reconcile.Request)\n\t}{\n\t\t{\n\t\t\tname: \"find pvb for pod\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseAccepted).Result(),\n\t\t\tpod: builder.ForPod(velerov1api.DefaultNamespace, pvbName).Labels(map[string]string{velerov1api.PVBLabel: pvbName}).Status(corev1api.PodStatus{Phase: corev1api.PodRunning}).Result(),\n\t\t\tcheckFunc: func(pvb *velerov1api.PodVolumeBackup, requests []reconcile.Request) {\n\t\t\t\t// Assert that the function returns a single request\n\t\t\t\tassert.Len(t, requests, 1)\n\t\t\t\t// Assert that the request contains the correct namespaced name\n\t\t\t\tassert.Equal(t, pvb.Namespace, requests[0].Namespace)\n\t\t\t\tassert.Equal(t, pvb.Name, requests[0].Name)\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"no selected label found for pod\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseAccepted).Result(),\n\t\t\tpod: builder.ForPod(velerov1api.DefaultNamespace, pvbName).Result(),\n\t\t\tcheckFunc: func(pvb *velerov1api.PodVolumeBackup, requests []reconcile.Request) {\n\t\t\t\t// Assert that the function returns a single request\n\t\t\t\tassert.Empty(t, requests)\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"no matched pod\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseAccepted).Result(),\n\t\t\tpod: builder.ForPod(velerov1api.DefaultNamespace, pvbName).Labels(map[string]string{velerov1api.PVBLabel: \"non-existing-pvb\"}).Result(),\n\t\t\tcheckFunc: func(pvb *velerov1api.PodVolumeBackup, requests []reconcile.Request) {\n\t\t\t\tassert.Empty(t, requests)\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"pvb not accepte\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseInProgress).Result(),\n\t\t\tpod: builder.ForPod(velerov1api.DefaultNamespace, pvbName).Labels(map[string]string{velerov1api.PVBLabel: pvbName}).Result(),\n\t\t\tcheckFunc: func(pvb *velerov1api.PodVolumeBackup, requests []reconcile.Request) {\n\t\t\t\tassert.Empty(t, requests)\n\t\t\t},\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tctx := t.Context()\n\t\tassert.NoError(t, r.client.Create(ctx, test.pod))\n\t\tassert.NoError(t, r.client.Create(ctx, test.pvb))\n\n\t\trequests := r.findPVBForPod(t.Context(), test.pod)\n\t\ttest.checkFunc(test.pvb, requests)\n\t\tr.client.Delete(ctx, test.pvb, &client.DeleteOptions{})\n\t\tif test.pod != nil {\n\t\t\tr.client.Delete(ctx, test.pod, &client.DeleteOptions{})\n\t\t}\n\t}\n}\n\nfunc TestAcceptPvb(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvb *velerov1api.PodVolumeBackup\n\t\tneedErrs []error\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"update fail\",\n\t\t\tpvb: pvbBuilder().Node(\"test-node\").Result(),\n\t\t\tneedErrs: []error{nil, nil, fmt.Errorf(\"fake-update-error\"), nil},\n\t\t\texpectedErr: \"error updating PVB with error velero/pvb-1: fake-update-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tpvb: pvbBuilder().Node(\"test-node\").Result(),\n\t\t\tneedErrs: []error{nil, nil, nil, nil},\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tctx := t.Context()\n\t\tr, err := initPVBReconcilerWithError(test.needErrs...)\n\t\trequire.NoError(t, err)\n\n\t\terr = r.client.Create(ctx, test.pvb)\n\t\trequire.NoError(t, err)\n\n\t\terr = r.acceptPodVolumeBackup(ctx, test.pvb)\n\t\tif test.expectedErr == \"\" {\n\t\t\tassert.NoError(t, err)\n\t\t} else {\n\t\t\tassert.EqualError(t, err, test.expectedErr)\n\t\t}\n\t}\n}\n\nfunc TestOnPvbPrepareTimeout(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvb *velerov1api.PodVolumeBackup\n\t\tneedErrs []error\n\t\texpected *velerov1api.PodVolumeBackup\n\t}{\n\t\t{\n\t\t\tname: \"update fail\",\n\t\t\tpvb: pvbBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, fmt.Errorf(\"fake-update-error\"), nil},\n\t\t\texpected: pvbBuilder().Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"update interrupted\",\n\t\t\tpvb: pvbBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, &fakeAPIStatus{metav1.StatusReasonConflict}, nil},\n\t\t\texpected: pvbBuilder().Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tpvb: pvbBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, nil, nil},\n\t\t\texpected: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseFailed).Result(),\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tctx := t.Context()\n\t\tr, err := initPVBReconcilerWithError(test.needErrs...)\n\t\trequire.NoError(t, err)\n\n\t\terr = r.client.Create(ctx, test.pvb)\n\t\trequire.NoError(t, err)\n\n\t\tr.onPrepareTimeout(ctx, test.pvb)\n\n\t\tpvb := velerov1api.PodVolumeBackup{}\n\t\t_ = r.client.Get(ctx, client.ObjectKey{\n\t\t\tName: test.pvb.Name,\n\t\t\tNamespace: test.pvb.Namespace,\n\t\t}, &pvb)\n\n\t\tassert.Equal(t, test.expected.Status.Phase, pvb.Status.Phase)\n\t}\n}\n\nfunc TestTryCancelPvb(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvb *velerov1api.PodVolumeBackup\n\t\tneedErrs []error\n\t\tsucceeded bool\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"update fail\",\n\t\t\tpvb: pvbBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, fmt.Errorf(\"fake-update-error\"), nil},\n\t\t},\n\t\t{\n\t\t\tname: \"cancel by others\",\n\t\t\tpvb: pvbBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, &fakeAPIStatus{metav1.StatusReasonConflict}, nil},\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tpvb: pvbBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, nil, nil},\n\t\t\tsucceeded: true,\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tctx := t.Context()\n\t\tr, err := initPVBReconcilerWithError(test.needErrs...)\n\t\trequire.NoError(t, err)\n\n\t\terr = r.client.Create(ctx, test.pvb)\n\t\trequire.NoError(t, err)\n\n\t\tr.tryCancelPodVolumeBackup(ctx, test.pvb, \"\")\n\n\t\tif test.expectedErr == \"\" {\n\t\t\tassert.NoError(t, err)\n\t\t} else {\n\t\t\tassert.EqualError(t, err, test.expectedErr)\n\t\t}\n\t}\n}\n\nfunc TestUpdatePvbWithRetry(t *testing.T) {\n\tnamespacedName := types.NamespacedName{\n\t\tName: pvbName,\n\t\tNamespace: \"velero\",\n\t}\n\n\t// Define test cases\n\ttestCases := []struct {\n\t\tName string\n\t\tneedErrs []bool\n\t\tnoChange bool\n\t\tExpectErr bool\n\t}{\n\t\t{\n\t\t\tName: \"SuccessOnFirstAttempt\",\n\t\t},\n\t\t{\n\t\t\tName: \"Error get\",\n\t\t\tneedErrs: []bool{true, false, false, false, false},\n\t\t\tExpectErr: true,\n\t\t},\n\t\t{\n\t\t\tName: \"Error update\",\n\t\t\tneedErrs: []bool{false, false, true, false, false},\n\t\t\tExpectErr: true,\n\t\t},\n\t\t{\n\t\t\tName: \"no change\",\n\t\t\tnoChange: true,\n\t\t\tneedErrs: []bool{false, false, true, false, false},\n\t\t},\n\t\t{\n\t\t\tName: \"Conflict with error timeout\",\n\t\t\tneedErrs: []bool{false, false, false, false, true},\n\t\t\tExpectErr: true,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.Name, func(t *testing.T) {\n\t\t\tctx, cancelFunc := context.WithTimeout(t.Context(), time.Second*5)\n\t\t\tdefer cancelFunc()\n\t\t\tr, err := initPVBReconciler(tc.needErrs...)\n\t\t\trequire.NoError(t, err)\n\t\t\terr = r.client.Create(ctx, pvbBuilder().Result())\n\t\t\trequire.NoError(t, err)\n\t\t\tupdateFunc := func(pvb *velerov1api.PodVolumeBackup) bool {\n\t\t\t\tif tc.noChange {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tpvb.Spec.Cancel = true\n\t\t\t\treturn true\n\t\t\t}\n\t\t\terr = UpdatePVBWithRetry(ctx, r.client, namespacedName, velerotest.NewLogger().WithField(\"name\", tc.Name), updateFunc)\n\t\t\tif tc.ExpectErr {\n\t\t\t\tassert.Error(t, err)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\ntype pvbResumeTestHelper struct {\n\tresumeErr error\n\tgetExposeErr error\n\texposeResult *exposer.ExposeResult\n\tasyncBR datapath.AsyncBR\n}\n\nfunc (dt *pvbResumeTestHelper) resumeCancellableDataPath(_ *DataUploadReconciler, _ context.Context, _ *velerov2alpha1api.DataUpload, _ logrus.FieldLogger) error {\n\treturn dt.resumeErr\n}\n\nfunc (dt *pvbResumeTestHelper) Expose(context.Context, corev1api.ObjectReference, exposer.PodVolumeExposeParam) error {\n\treturn nil\n}\n\nfunc (dt *pvbResumeTestHelper) GetExposed(context.Context, corev1api.ObjectReference, kbclient.Client, string, time.Duration) (*exposer.ExposeResult, error) {\n\treturn dt.exposeResult, dt.getExposeErr\n}\n\nfunc (dt *pvbResumeTestHelper) PeekExposed(context.Context, corev1api.ObjectReference) error {\n\treturn nil\n}\n\nfunc (dt *pvbResumeTestHelper) DiagnoseExpose(context.Context, corev1api.ObjectReference) string {\n\treturn \"\"\n}\n\nfunc (dt *pvbResumeTestHelper) CleanUp(context.Context, corev1api.ObjectReference) {}\n\nfunc (dt *pvbResumeTestHelper) newMicroServiceBRWatcher(kbclient.Client, kubernetes.Interface, manager.Manager, string, string, string, string, string, string,\n\tdatapath.Callbacks, logrus.FieldLogger) datapath.AsyncBR {\n\treturn dt.asyncBR\n}\n\nfunc TestAttemptPVBResume(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvbs []*velerov1api.PodVolumeBackup\n\t\tpvb *velerov1api.PodVolumeBackup\n\t\tneedErrs []bool\n\t\tacceptedPvbs []string\n\t\tpreparedPvbs []string\n\t\tcancelledPvbs []string\n\t\tinProgressPvbs []string\n\t\tresumeErr error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"Other pvb\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhasePrepared).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"InProgress pvb, not the current node\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseInProgress).Result(),\n\t\t\tinProgressPvbs: []string{pvbName},\n\t\t},\n\t\t{\n\t\t\tname: \"InProgress pvb, resume error and update error\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseInProgress).Node(\"node-1\").Result(),\n\t\t\tneedErrs: []bool{false, false, true, false, false, false},\n\t\t\tresumeErr: errors.New(\"fake-resume-error\"),\n\t\t\tinProgressPvbs: []string{pvbName},\n\t\t},\n\t\t{\n\t\t\tname: \"InProgress pvb, resume error and update succeed\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseInProgress).Node(\"node-1\").Result(),\n\t\t\tresumeErr: errors.New(\"fake-resume-error\"),\n\t\t\tcancelledPvbs: []string{pvbName},\n\t\t\tinProgressPvbs: []string{pvbName},\n\t\t},\n\t\t{\n\t\t\tname: \"InProgress pvb and resume succeed\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseInProgress).Node(\"node-1\").Result(),\n\t\t\tinProgressPvbs: []string{pvbName},\n\t\t},\n\t\t{\n\t\t\tname: \"Error\",\n\t\t\tneedErrs: []bool{false, false, false, false, false, true},\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhasePrepared).Result(),\n\t\t\texpectedError: \"error to list PVBs: List error\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\t\t\tr, err := initPVBReconciler(test.needErrs...)\n\t\t\tr.nodeName = \"node-1\"\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.NoError(t, r.client.Create(ctx, test.pvb))\n\n\t\t\tdt := &pvbResumeTestHelper{\n\t\t\t\tresumeErr: test.resumeErr,\n\t\t\t}\n\n\t\t\tfuncResumeCancellableDataBackup = dt.resumeCancellableDataPath\n\n\t\t\t// Run the test\n\t\t\terr = r.AttemptPVBResume(ctx, r.logger.WithField(\"name\", test.name), test.pvb.Namespace)\n\n\t\t\tif test.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.expectedError)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\n\t\t\t\tfor _, pvbName := range test.cancelledPvbs {\n\t\t\t\t\tpvb := &velerov1api.PodVolumeBackup{}\n\t\t\t\t\terr := r.client.Get(t.Context(), types.NamespacedName{Namespace: \"velero\", Name: pvbName}, pvb)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tassert.True(t, pvb.Spec.Cancel)\n\t\t\t\t}\n\n\t\t\t\tfor _, pvbName := range test.acceptedPvbs {\n\t\t\t\t\tpvb := &velerov1api.PodVolumeBackup{}\n\t\t\t\t\terr := r.client.Get(t.Context(), types.NamespacedName{Namespace: \"velero\", Name: pvbName}, pvb)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tassert.Equal(t, velerov1api.PodVolumeBackupPhaseAccepted, pvb.Status.Phase)\n\t\t\t\t}\n\n\t\t\t\tfor _, pvbName := range test.preparedPvbs {\n\t\t\t\t\tpvb := &velerov1api.PodVolumeBackup{}\n\t\t\t\t\terr := r.client.Get(t.Context(), types.NamespacedName{Namespace: \"velero\", Name: pvbName}, pvb)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tassert.Equal(t, velerov1api.PodVolumeBackupPhasePrepared, pvb.Status.Phase)\n\t\t\t\t}\n\n\t\t\t\tfor _, pvbName := range test.inProgressPvbs {\n\t\t\t\t\tpvb := &velerov1api.PodVolumeBackup{}\n\t\t\t\t\terr := r.client.Get(t.Context(), types.NamespacedName{Namespace: \"velero\", Name: pvbName}, pvb)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tassert.Equal(t, velerov1api.PodVolumeBackupPhaseInProgress, pvb.Status.Phase)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestResumeCancellablePodVolumeBackup(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvbs []velerov1api.PodVolumeBackup\n\t\tpvb *velerov1api.PodVolumeBackup\n\t\tgetExposeErr error\n\t\texposeResult *exposer.ExposeResult\n\t\tcreateWatcherErr error\n\t\tinitWatcherErr error\n\t\tstartWatcherErr error\n\t\tmockInit bool\n\t\tmockStart bool\n\t\tmockClose bool\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"get expose failed\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseInProgress).Result(),\n\t\t\tgetExposeErr: errors.New(\"fake-expose-error\"),\n\t\t\texpectedError: fmt.Sprintf(\"error to get exposed PVB %s: fake-expose-error\", pvbName),\n\t\t},\n\t\t{\n\t\t\tname: \"no expose\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseAccepted).Node(\"node-1\").Result(),\n\t\t\texpectedError: fmt.Sprintf(\"no expose result is available for the current node for PVB %s\", pvbName),\n\t\t},\n\t\t{\n\t\t\tname: \"watcher init error\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseAccepted).Node(\"node-1\").Result(),\n\t\t\texposeResult: &exposer.ExposeResult{\n\t\t\t\tByPod: exposer.ExposeByPod{\n\t\t\t\t\tHostingPod: &corev1api.Pod{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tmockInit: true,\n\t\t\tmockClose: true,\n\t\t\tinitWatcherErr: errors.New(\"fake-init-watcher-error\"),\n\t\t\texpectedError: fmt.Sprintf(\"error to init asyncBR watcher for PVB %s: fake-init-watcher-error\", pvbName),\n\t\t},\n\t\t{\n\t\t\tname: \"start watcher error\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseAccepted).Node(\"node-1\").Result(),\n\t\t\texposeResult: &exposer.ExposeResult{\n\t\t\t\tByPod: exposer.ExposeByPod{\n\t\t\t\t\tHostingPod: &corev1api.Pod{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tmockInit: true,\n\t\t\tmockStart: true,\n\t\t\tmockClose: true,\n\t\t\tstartWatcherErr: errors.New(\"fake-start-watcher-error\"),\n\t\t\texpectedError: fmt.Sprintf(\"error to resume asyncBR watcher for PVB %s: fake-start-watcher-error\", pvbName),\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tpvb: pvbBuilder().Phase(velerov1api.PodVolumeBackupPhaseAccepted).Node(\"node-1\").Result(),\n\t\t\texposeResult: &exposer.ExposeResult{\n\t\t\t\tByPod: exposer.ExposeByPod{\n\t\t\t\t\tHostingPod: &corev1api.Pod{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tmockInit: true,\n\t\t\tmockStart: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\t\t\tr, err := initPVBReconciler()\n\t\t\tr.nodeName = \"node-1\"\n\t\t\trequire.NoError(t, err)\n\n\t\t\tmockAsyncBR := datapathmocks.NewAsyncBR(t)\n\n\t\t\tif test.mockInit {\n\t\t\t\tmockAsyncBR.On(\"Init\", mock.Anything, mock.Anything).Return(test.initWatcherErr)\n\t\t\t}\n\n\t\t\tif test.mockStart {\n\t\t\t\tmockAsyncBR.On(\"StartBackup\", mock.Anything, mock.Anything, mock.Anything).Return(test.startWatcherErr)\n\t\t\t}\n\n\t\t\tif test.mockClose {\n\t\t\t\tmockAsyncBR.On(\"Close\", mock.Anything).Return()\n\t\t\t}\n\n\t\t\tdt := &pvbResumeTestHelper{\n\t\t\t\tgetExposeErr: test.getExposeErr,\n\t\t\t\texposeResult: test.exposeResult,\n\t\t\t\tasyncBR: mockAsyncBR,\n\t\t\t}\n\n\t\t\tr.exposer = dt\n\n\t\t\tdatapath.MicroServiceBRWatcherCreator = dt.newMicroServiceBRWatcher\n\n\t\t\terr = r.resumeCancellableDataPath(ctx, test.pvb, velerotest.NewLogger())\n\t\t\tif test.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.expectedError)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestPodVolumeBackupSetupExposeParam(t *testing.T) {\n\t// common objects for all cases\n\tnode := builder.ForNode(\"worker-1\").Labels(map[string]string{kube.NodeOSLabel: kube.NodeOSLinux}).Result()\n\n\tbasePVB := pvbBuilder().Result()\n\tbasePVB.Spec.Node = \"worker-1\"\n\tbasePVB.Spec.Pod.Namespace = \"app-ns\"\n\tbasePVB.Spec.Pod.Name = \"app-pod\"\n\tbasePVB.Spec.Volume = \"data-vol\"\n\n\ttype args struct {\n\t\tcustomLabels map[string]string\n\t\tcustomAnnotations map[string]string\n\t}\n\ttype want struct {\n\t\tlabels map[string]string\n\t\tannotations map[string]string\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\targs args\n\t\twant want\n\t}{\n\t\t{\n\t\t\tname: \"label has customize values\",\n\t\t\targs: args{\n\t\t\t\tcustomLabels: map[string]string{\"custom-label\": \"label-value\"},\n\t\t\t\tcustomAnnotations: nil,\n\t\t\t},\n\t\t\twant: want{\n\t\t\t\tlabels: map[string]string{\n\t\t\t\t\tvelerov1api.PVBLabel: basePVB.Name,\n\t\t\t\t\t\"custom-label\": \"label-value\",\n\t\t\t\t},\n\t\t\t\tannotations: map[string]string{},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"label has no customize values\",\n\t\t\targs: args{\n\t\t\t\tcustomLabels: nil,\n\t\t\t\tcustomAnnotations: nil,\n\t\t\t},\n\t\t\twant: want{\n\t\t\t\tlabels: map[string]string{velerov1api.PVBLabel: basePVB.Name},\n\t\t\t\tannotations: map[string]string{},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"annotation has customize values\",\n\t\t\targs: args{\n\t\t\t\tcustomLabels: nil,\n\t\t\t\tcustomAnnotations: map[string]string{\"custom-annotation\": \"annotation-value\"},\n\t\t\t},\n\t\t\twant: want{\n\t\t\t\tlabels: map[string]string{velerov1api.PVBLabel: basePVB.Name},\n\t\t\t\tannotations: map[string]string{\"custom-annotation\": \"annotation-value\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"annotation has no customize values\",\n\t\t\targs: args{\n\t\t\t\tcustomLabels: map[string]string{\"another-label\": \"lval\"},\n\t\t\t\tcustomAnnotations: nil,\n\t\t\t},\n\t\t\twant: want{\n\t\t\t\tlabels: map[string]string{\n\t\t\t\t\tvelerov1api.PVBLabel: basePVB.Name,\n\t\t\t\t\t\"another-label\": \"lval\",\n\t\t\t\t},\n\t\t\t\tannotations: map[string]string{},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\t// Fake clients per case\n\t\t\tfakeCRClient := velerotest.NewFakeControllerRuntimeClient(t, node, basePVB.DeepCopy())\n\t\t\tfakeKubeClient := clientgofake.NewSimpleClientset(node)\n\n\t\t\t// Reconciler config per case\n\t\t\tpreparingTimeout := time.Minute * 3\n\t\t\tresourceTimeout := time.Minute * 10\n\t\t\tpodRes := corev1api.ResourceRequirements{}\n\t\t\tr := NewPodVolumeBackupReconciler(\n\t\t\t\tfakeCRClient,\n\t\t\t\tnil,\n\t\t\t\tfakeKubeClient,\n\t\t\t\tdatapath.NewManager(1),\n\t\t\t\tnil,\n\t\t\t\t\"test-node\",\n\t\t\t\tpreparingTimeout,\n\t\t\t\tresourceTimeout,\n\t\t\t\tpodRes,\n\t\t\t\tmetrics.NewServerMetrics(),\n\t\t\t\tvelerotest.NewLogger(),\n\t\t\t\t\"backup-priority\",\n\t\t\t\ttrue,\n\t\t\t\ttt.args.customLabels,\n\t\t\t\ttt.args.customAnnotations,\n\t\t\t)\n\n\t\t\t// Act\n\t\t\tgot := r.setupExposeParam(basePVB)\n\n\t\t\t// Core fields\n\t\t\tassert.Equal(t, exposer.PodVolumeExposeTypeBackup, got.Type)\n\t\t\tassert.Equal(t, basePVB.Spec.Pod.Namespace, got.ClientNamespace)\n\t\t\tassert.Equal(t, basePVB.Spec.Pod.Name, got.ClientPodName)\n\t\t\tassert.Equal(t, basePVB.Spec.Volume, got.ClientPodVolume)\n\n\t\t\t// Labels/Annotations\n\t\t\tassert.Equal(t, tt.want.labels, got.HostingPodLabels)\n\t\t\tassert.Equal(t, tt.want.annotations, got.HostingPodAnnotations)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/controller/pod_volume_restore_controller.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"k8s.io/client-go/kubernetes\"\n\tclocks \"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/builder\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil\"\n\t\"sigs.k8s.io/controller-runtime/pkg/event\"\n\t\"sigs.k8s.io/controller-runtime/pkg/handler\"\n\t\"sigs.k8s.io/controller-runtime/pkg/manager\"\n\t\"sigs.k8s.io/controller-runtime/pkg/predicate\"\n\t\"sigs.k8s.io/controller-runtime/pkg/reconcile\"\n\n\tveleroapishared \"github.com/vmware-tanzu/velero/pkg/apis/velero/shared\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\t\"github.com/vmware-tanzu/velero/pkg/exposer\"\n\t\"github.com/vmware-tanzu/velero/pkg/nodeagent\"\n\trepository \"github.com/vmware-tanzu/velero/pkg/repository/manager\"\n\t\"github.com/vmware-tanzu/velero/pkg/restorehelper\"\n\tvelerotypes \"github.com/vmware-tanzu/velero/pkg/types\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nfunc NewPodVolumeRestoreReconciler(\n\tclient client.Client,\n\tmgr manager.Manager,\n\tkubeClient kubernetes.Interface,\n\tdataPathMgr *datapath.Manager,\n\tcounter *exposer.VgdpCounter,\n\tnodeName string,\n\tpreparingTimeout time.Duration,\n\tresourceTimeout time.Duration,\n\tbackupRepoConfigs map[string]string,\n\tcacheVolumeConfigs *velerotypes.CachePVC,\n\tpodResources corev1api.ResourceRequirements,\n\tlogger logrus.FieldLogger,\n\tdataMovePriorityClass string,\n\tprivileged bool,\n\trepoConfigMgr repository.ConfigManager,\n\tpodLabels map[string]string,\n\tpodAnnotations map[string]string,\n) *PodVolumeRestoreReconciler {\n\treturn &PodVolumeRestoreReconciler{\n\t\tclient: client,\n\t\tmgr: mgr,\n\t\tkubeClient: kubeClient,\n\t\tlogger: logger.WithField(\"controller\", \"PodVolumeRestore\"),\n\t\tnodeName: nodeName,\n\t\tclock: &clocks.RealClock{},\n\t\tpodResources: podResources,\n\t\tbackupRepoConfigs: backupRepoConfigs,\n\t\tcacheVolumeConfigs: cacheVolumeConfigs,\n\t\tdataPathMgr: dataPathMgr,\n\t\tvgdpCounter: counter,\n\t\tpreparingTimeout: preparingTimeout,\n\t\tresourceTimeout: resourceTimeout,\n\t\texposer: exposer.NewPodVolumeExposer(kubeClient, logger),\n\t\tcancelledPVR: make(map[string]time.Time),\n\t\tdataMovePriorityClass: dataMovePriorityClass,\n\t\tprivileged: privileged,\n\t\trepoConfigMgr: repoConfigMgr,\n\t\tpodLabels: podLabels,\n\t\tpodAnnotations: podAnnotations,\n\t}\n}\n\ntype PodVolumeRestoreReconciler struct {\n\tclient client.Client\n\tmgr manager.Manager\n\tkubeClient kubernetes.Interface\n\tlogger logrus.FieldLogger\n\tnodeName string\n\tclock clocks.WithTickerAndDelayedExecution\n\tpodResources corev1api.ResourceRequirements\n\tbackupRepoConfigs map[string]string\n\tcacheVolumeConfigs *velerotypes.CachePVC\n\texposer exposer.PodVolumeExposer\n\tdataPathMgr *datapath.Manager\n\tvgdpCounter *exposer.VgdpCounter\n\tpreparingTimeout time.Duration\n\tresourceTimeout time.Duration\n\tcancelledPVR map[string]time.Time\n\tdataMovePriorityClass string\n\tprivileged bool\n\trepoConfigMgr repository.ConfigManager\n\tpodLabels map[string]string\n\tpodAnnotations map[string]string\n}\n\n// +kubebuilder:rbac:groups=velero.io,resources=podvolumerestores,verbs=get;list;watch;create;update;patch;delete\n// +kubebuilder:rbac:groups=velero.io,resources=podvolumerestores/status,verbs=get;update;patch\n// +kubebuilder:rbac:groups=\"\",resources=pods,verbs=get\n// +kubebuilder:rbac:groups=\"\",resources=persistentvolumes,verbs=get\n// +kubebuilder:rbac:groups=\"\",resources=persistentvolumerclaims,verbs=get\n\nfunc (r *PodVolumeRestoreReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\tlog := r.logger.WithField(\"PodVolumeRestore\", req.NamespacedName.String())\n\tlog.Info(\"Reconciling PVR by advanced controller\")\n\n\tpvr := &velerov1api.PodVolumeRestore{}\n\tif err := r.client.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: req.Name}, pvr); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.Warn(\"PVR not found, skip\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\tlog.WithError(err).Error(\"Unable to get the PVR\")\n\t\treturn ctrl.Result{}, err\n\t}\n\n\tlog = log.WithField(\"pod\", fmt.Sprintf(\"%s/%s\", pvr.Spec.Pod.Namespace, pvr.Spec.Pod.Name))\n\tif len(pvr.OwnerReferences) == 1 {\n\t\tlog = log.WithField(\"restore\", fmt.Sprintf(\"%s/%s\", pvr.Namespace, pvr.OwnerReferences[0].Name))\n\t}\n\n\t// Logic for clear resources when pvr been deleted\n\tif !isPVRInFinalState(pvr) {\n\t\tif !controllerutil.ContainsFinalizer(pvr, PodVolumeFinalizer) {\n\t\t\tif err := UpdatePVRWithRetry(ctx, r.client, req.NamespacedName, log, func(pvr *velerov1api.PodVolumeRestore) bool {\n\t\t\t\tif controllerutil.ContainsFinalizer(pvr, PodVolumeFinalizer) {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tcontrollerutil.AddFinalizer(pvr, PodVolumeFinalizer)\n\n\t\t\t\treturn true\n\t\t\t}); err != nil {\n\t\t\t\tlog.WithError(err).Errorf(\"failed to add finalizer for PVR %s/%s\", pvr.Namespace, pvr.Name)\n\t\t\t\treturn ctrl.Result{}, err\n\t\t\t}\n\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tif !pvr.DeletionTimestamp.IsZero() {\n\t\t\tif !pvr.Spec.Cancel {\n\t\t\t\tlog.Warnf(\"Cancel PVR under phase %s because it is being deleted\", pvr.Status.Phase)\n\n\t\t\t\tif err := UpdatePVRWithRetry(ctx, r.client, req.NamespacedName, log, func(pvr *velerov1api.PodVolumeRestore) bool {\n\t\t\t\t\tif pvr.Spec.Cancel {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\n\t\t\t\t\tpvr.Spec.Cancel = true\n\t\t\t\t\tpvr.Status.Message = \"Cancel PVR because it is being deleted\"\n\n\t\t\t\t\treturn true\n\t\t\t\t}); err != nil {\n\t\t\t\t\tlog.WithError(err).Errorf(\"failed to set cancel flag for PVR %s/%s\", pvr.Namespace, pvr.Name)\n\t\t\t\t\treturn ctrl.Result{}, err\n\t\t\t\t}\n\n\t\t\t\treturn ctrl.Result{}, nil\n\t\t\t}\n\t\t}\n\t} else {\n\t\tdelete(r.cancelledPVR, pvr.Name)\n\n\t\tif controllerutil.ContainsFinalizer(pvr, PodVolumeFinalizer) {\n\t\t\tif err := UpdatePVRWithRetry(ctx, r.client, req.NamespacedName, log, func(pvr *velerov1api.PodVolumeRestore) bool {\n\t\t\t\tif !controllerutil.ContainsFinalizer(pvr, PodVolumeFinalizer) {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tcontrollerutil.RemoveFinalizer(pvr, PodVolumeFinalizer)\n\n\t\t\t\treturn true\n\t\t\t}); err != nil {\n\t\t\t\tlog.WithError(err).Error(\"error to remove finalizer\")\n\t\t\t\treturn ctrl.Result{}, err\n\t\t\t}\n\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t}\n\n\tif pvr.Spec.Cancel {\n\t\tif spotted, found := r.cancelledPVR[pvr.Name]; !found {\n\t\t\tr.cancelledPVR[pvr.Name] = r.clock.Now()\n\t\t} else {\n\t\t\tdelay := cancelDelayOthers\n\t\t\tif pvr.Status.Phase == velerov1api.PodVolumeRestorePhaseInProgress {\n\t\t\t\tdelay = cancelDelayInProgress\n\t\t\t}\n\n\t\t\tif time.Since(spotted) > delay {\n\t\t\t\tlog.Infof(\"PVR %s is canceled in Phase %s but not handled in rasonable time\", pvr.GetName(), pvr.Status.Phase)\n\t\t\t\tif r.tryCancelPodVolumeRestore(ctx, pvr, \"\") {\n\t\t\t\t\tdelete(r.cancelledPVR, pvr.Name)\n\t\t\t\t}\n\n\t\t\t\treturn ctrl.Result{}, nil\n\t\t\t}\n\t\t}\n\t}\n\n\tif pvr.Status.Phase == \"\" || pvr.Status.Phase == velerov1api.PodVolumeRestorePhaseNew {\n\t\tif pvr.Spec.Cancel {\n\t\t\tlog.Infof(\"PVR %s is canceled in Phase %s\", pvr.GetName(), pvr.Status.Phase)\n\t\t\t_ = r.tryCancelPodVolumeRestore(ctx, pvr, \"\")\n\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tshouldProcess, pod, err := shouldProcess(ctx, r.client, log, pvr)\n\t\tif err != nil {\n\t\t\treturn ctrl.Result{}, err\n\t\t}\n\t\tif !shouldProcess {\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tif r.vgdpCounter != nil && r.vgdpCounter.IsConstrained(ctx, r.logger) {\n\t\t\tlog.Debug(\"Data path initiation is constrained, requeue later\")\n\t\t\treturn ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5}, nil\n\t\t}\n\n\t\tlog.Info(\"Accepting PVR\")\n\n\t\tif err := r.acceptPodVolumeRestore(ctx, pvr); err != nil {\n\t\t\treturn ctrl.Result{}, errors.Wrapf(err, \"error accepting PVR %s\", pvr.Name)\n\t\t}\n\n\t\tinitContainerIndex := getInitContainerIndex(pod)\n\t\tif initContainerIndex > 0 {\n\t\t\tlog.Warnf(`Init containers before the %s container may cause issues\n\t\t\t\t\t if they interfere with volumes being restored: %s index %d`, restorehelper.WaitInitContainer, restorehelper.WaitInitContainer, initContainerIndex)\n\t\t}\n\n\t\tlog.Info(\"Exposing PVR\")\n\n\t\texposeParam := r.setupExposeParam(pvr)\n\t\tif err := r.exposer.Expose(ctx, getPVROwnerObject(pvr), exposeParam); err != nil {\n\t\t\treturn r.errorOut(ctx, pvr, err, \"error to expose PVR\", log)\n\t\t}\n\n\t\tlog.Info(\"PVR is exposed\")\n\n\t\treturn ctrl.Result{}, nil\n\t} else if pvr.Status.Phase == velerov1api.PodVolumeRestorePhaseAccepted {\n\t\tif peekErr := r.exposer.PeekExposed(ctx, getPVROwnerObject(pvr)); peekErr != nil {\n\t\t\tlog.Errorf(\"Cancel PVR %s/%s because of expose error %s\", pvr.Namespace, pvr.Name, peekErr)\n\n\t\t\tdiags := strings.Split(r.exposer.DiagnoseExpose(ctx, getPVROwnerObject(pvr)), \"\\n\")\n\t\t\tfor _, diag := range diags {\n\t\t\t\tlog.Warnf(\"[Diagnose PVR expose]%s\", diag)\n\t\t\t}\n\n\t\t\t_ = r.tryCancelPodVolumeRestore(ctx, pvr, fmt.Sprintf(\"found a PVR %s/%s with expose error: %s. mark it as cancel\", pvr.Namespace, pvr.Name, peekErr))\n\t\t} else if pvr.Status.AcceptedTimestamp != nil {\n\t\t\tif time.Since(pvr.Status.AcceptedTimestamp.Time) >= r.preparingTimeout {\n\t\t\t\tr.onPrepareTimeout(ctx, pvr)\n\t\t\t}\n\t\t}\n\n\t\treturn ctrl.Result{}, nil\n\t} else if pvr.Status.Phase == velerov1api.PodVolumeRestorePhasePrepared {\n\t\tlog.Infof(\"PVR is prepared and should be processed by %s (%s)\", pvr.Status.Node, r.nodeName)\n\n\t\tif pvr.Status.Node != r.nodeName {\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tif pvr.Spec.Cancel {\n\t\t\tlog.Info(\"Prepared PVR is being canceled\")\n\t\t\tr.OnDataPathCancelled(ctx, pvr.GetNamespace(), pvr.GetName())\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tasyncBR := r.dataPathMgr.GetAsyncBR(pvr.Name)\n\t\tif asyncBR != nil {\n\t\t\tlog.Info(\"Cancellable data path is already started\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tres, err := r.exposer.GetExposed(ctx, getPVROwnerObject(pvr), r.client, r.nodeName, r.resourceTimeout)\n\t\tif err != nil {\n\t\t\treturn r.errorOut(ctx, pvr, err, \"exposed PVR is not ready\", log)\n\t\t} else if res == nil {\n\t\t\treturn r.errorOut(ctx, pvr, errors.New(\"no expose result is available for the current node\"), \"exposed PVR is not ready\", log)\n\t\t}\n\n\t\tlog.Info(\"Exposed PVR is ready and creating data path routine\")\n\n\t\tcallbacks := datapath.Callbacks{\n\t\t\tOnCompleted: r.OnDataPathCompleted,\n\t\t\tOnFailed: r.OnDataPathFailed,\n\t\t\tOnCancelled: r.OnDataPathCancelled,\n\t\t\tOnProgress: r.OnDataPathProgress,\n\t\t}\n\n\t\tasyncBR, err = r.dataPathMgr.CreateMicroServiceBRWatcher(ctx, r.client, r.kubeClient, r.mgr, datapath.TaskTypeRestore,\n\t\t\tpvr.Name, pvr.Namespace, res.ByPod.HostingPod.Name, res.ByPod.HostingContainer, pvr.Name, callbacks, false, log)\n\t\tif err != nil {\n\t\t\tif err == datapath.ConcurrentLimitExceed {\n\t\t\t\tlog.Debug(\"Data path instance is concurrent limited requeue later\")\n\t\t\t\treturn ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5}, nil\n\t\t\t} else {\n\t\t\t\treturn r.errorOut(ctx, pvr, err, \"error to create data path\", log)\n\t\t\t}\n\t\t}\n\n\t\tif err := r.initCancelableDataPath(ctx, asyncBR, res, log); err != nil {\n\t\t\tlog.WithError(err).Errorf(\"Failed to init cancelable data path for %s\", pvr.Name)\n\n\t\t\tr.closeDataPath(ctx, pvr.Name)\n\t\t\treturn r.errorOut(ctx, pvr, err, \"error initializing data path\", log)\n\t\t}\n\n\t\tterminated := false\n\t\tif err := UpdatePVRWithRetry(ctx, r.client, types.NamespacedName{Namespace: pvr.Namespace, Name: pvr.Name}, log, func(pvr *velerov1api.PodVolumeRestore) bool {\n\t\t\tif isPVRInFinalState(pvr) {\n\t\t\t\tterminated = true\n\t\t\t\treturn false\n\t\t\t}\n\n\t\t\tpvr.Status.Phase = velerov1api.PodVolumeRestorePhaseInProgress\n\t\t\tpvr.Status.StartTimestamp = &metav1.Time{Time: r.clock.Now()}\n\n\t\t\tdelete(pvr.Labels, exposer.ExposeOnGoingLabel)\n\n\t\t\treturn true\n\t\t}); err != nil {\n\t\t\tlog.WithError(err).Warnf(\"Failed to update PVR %s to InProgress, will data path close and retry\", pvr.Name)\n\n\t\t\tr.closeDataPath(ctx, pvr.Name)\n\t\t\treturn ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5}, nil\n\t\t}\n\n\t\tif terminated {\n\t\t\tlog.Warnf(\"PVR %s is terminated during transition from prepared\", pvr.Name)\n\t\t\tr.closeDataPath(ctx, pvr.Name)\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tlog.Info(\"PVR is marked as in progress\")\n\n\t\tif err := r.startCancelableDataPath(asyncBR, pvr, res, log); err != nil {\n\t\t\tlog.WithError(err).Errorf(\"Failed to start cancelable data path for %s\", pvr.Name)\n\t\t\tr.closeDataPath(ctx, pvr.Name)\n\n\t\t\treturn r.errorOut(ctx, pvr, err, \"error starting data path\", log)\n\t\t}\n\n\t\treturn ctrl.Result{}, nil\n\t} else if pvr.Status.Phase == velerov1api.PodVolumeRestorePhaseInProgress {\n\t\tif pvr.Spec.Cancel {\n\t\t\tif pvr.Status.Node != r.nodeName {\n\t\t\t\treturn ctrl.Result{}, nil\n\t\t\t}\n\n\t\t\tlog.Info(\"PVR is being canceled\")\n\n\t\t\tasyncBR := r.dataPathMgr.GetAsyncBR(pvr.Name)\n\t\t\tif asyncBR == nil {\n\t\t\t\tr.OnDataPathCancelled(ctx, pvr.GetNamespace(), pvr.GetName())\n\t\t\t\treturn ctrl.Result{}, nil\n\t\t\t}\n\n\t\t\t// Update status to Canceling\n\t\t\tif err := UpdatePVRWithRetry(ctx, r.client, types.NamespacedName{Namespace: pvr.Namespace, Name: pvr.Name}, log, func(pvr *velerov1api.PodVolumeRestore) bool {\n\t\t\t\tif isPVRInFinalState(pvr) {\n\t\t\t\t\tlog.Warnf(\"PVR %s is terminated, abort setting it to canceling\", pvr.Name)\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tpvr.Status.Phase = velerov1api.PodVolumeRestorePhaseCanceling\n\t\t\t\treturn true\n\t\t\t}); err != nil {\n\t\t\t\tlog.WithError(err).Error(\"error updating PVR into canceling status\")\n\t\t\t\treturn ctrl.Result{}, err\n\t\t\t}\n\n\t\t\tasyncBR.Cancel()\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\treturn ctrl.Result{}, nil\n}\n\nfunc (r *PodVolumeRestoreReconciler) acceptPodVolumeRestore(ctx context.Context, pvr *velerov1api.PodVolumeRestore) error {\n\treturn UpdatePVRWithRetry(ctx, r.client, types.NamespacedName{Namespace: pvr.Namespace, Name: pvr.Name}, r.logger, func(pvr *velerov1api.PodVolumeRestore) bool {\n\t\tpvr.Status.AcceptedTimestamp = &metav1.Time{Time: r.clock.Now()}\n\t\tpvr.Status.Phase = velerov1api.PodVolumeRestorePhaseAccepted\n\t\tpvr.Status.Node = r.nodeName\n\n\t\tif pvr.Labels == nil {\n\t\t\tpvr.Labels = make(map[string]string)\n\t\t}\n\t\tpvr.Labels[exposer.ExposeOnGoingLabel] = \"true\"\n\n\t\treturn true\n\t})\n}\n\nfunc (r *PodVolumeRestoreReconciler) tryCancelPodVolumeRestore(ctx context.Context, pvr *velerov1api.PodVolumeRestore, message string) bool {\n\tlog := r.logger.WithField(\"PVR\", pvr.Name)\n\tsucceeded, err := funcExclusiveUpdatePodVolumeRestore(ctx, r.client, pvr, func(pvr *velerov1api.PodVolumeRestore) {\n\t\tpvr.Status.Phase = velerov1api.PodVolumeRestorePhaseCanceled\n\t\tif pvr.Status.StartTimestamp.IsZero() {\n\t\t\tpvr.Status.StartTimestamp = &metav1.Time{Time: r.clock.Now()}\n\t\t}\n\t\tpvr.Status.CompletionTimestamp = &metav1.Time{Time: r.clock.Now()}\n\n\t\tif message != \"\" {\n\t\t\tpvr.Status.Message = message\n\t\t}\n\n\t\tdelete(pvr.Labels, exposer.ExposeOnGoingLabel)\n\t})\n\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"error updating PVR status\")\n\t\treturn false\n\t} else if !succeeded {\n\t\tlog.Warn(\"conflict in updating PVR status and will try it again later\")\n\t\treturn false\n\t}\n\n\tr.exposer.CleanUp(ctx, getPVROwnerObject(pvr))\n\n\tlog.Warn(\"PVR is canceled\")\n\n\treturn true\n}\n\nvar funcExclusiveUpdatePodVolumeRestore = exclusiveUpdatePodVolumeRestore\n\nfunc exclusiveUpdatePodVolumeRestore(ctx context.Context, cli client.Client, pvr *velerov1api.PodVolumeRestore,\n\tupdateFunc func(*velerov1api.PodVolumeRestore)) (bool, error) {\n\tupdateFunc(pvr)\n\n\terr := cli.Update(ctx, pvr)\n\tif err == nil {\n\t\treturn true, nil\n\t}\n\n\t// warn we won't rollback pvr values in memory when error\n\tif apierrors.IsConflict(err) {\n\t\treturn false, nil\n\t} else {\n\t\treturn false, err\n\t}\n}\n\nfunc (r *PodVolumeRestoreReconciler) onPrepareTimeout(ctx context.Context, pvr *velerov1api.PodVolumeRestore) {\n\tlog := r.logger.WithField(\"PVR\", pvr.Name)\n\n\tlog.Info(\"Timeout happened for preparing PVR\")\n\n\tsucceeded, err := funcExclusiveUpdatePodVolumeRestore(ctx, r.client, pvr, func(pvr *velerov1api.PodVolumeRestore) {\n\t\tpvr.Status.Phase = velerov1api.PodVolumeRestorePhaseFailed\n\t\tpvr.Status.Message = \"timeout on preparing PVR\"\n\n\t\tdelete(pvr.Labels, exposer.ExposeOnGoingLabel)\n\t})\n\n\tif err != nil {\n\t\tlog.WithError(err).Warn(\"Failed to update PVR\")\n\t\treturn\n\t}\n\n\tif !succeeded {\n\t\tlog.Warn(\"PVR has been updated by others\")\n\t\treturn\n\t}\n\n\tdiags := strings.Split(r.exposer.DiagnoseExpose(ctx, getPVROwnerObject(pvr)), \"\\n\")\n\tfor _, diag := range diags {\n\t\tlog.Warnf(\"[Diagnose PVR expose]%s\", diag)\n\t}\n\n\tr.exposer.CleanUp(ctx, getPVROwnerObject(pvr))\n\n\tlog.Info(\"PVR has been cleaned up\")\n}\n\nfunc (r *PodVolumeRestoreReconciler) initCancelableDataPath(ctx context.Context, asyncBR datapath.AsyncBR, res *exposer.ExposeResult, log logrus.FieldLogger) error {\n\tlog.Info(\"Init cancelable PVR\")\n\n\tif err := asyncBR.Init(ctx, nil); err != nil {\n\t\treturn errors.Wrap(err, \"error initializing asyncBR\")\n\t}\n\n\tlog.Infof(\"async data path init for pod %s, volume %s\", res.ByPod.HostingPod.Name, res.ByPod.VolumeName)\n\n\treturn nil\n}\n\nfunc (r *PodVolumeRestoreReconciler) startCancelableDataPath(asyncBR datapath.AsyncBR, pvr *velerov1api.PodVolumeRestore, res *exposer.ExposeResult, log logrus.FieldLogger) error {\n\tlog.Info(\"Start cancelable PVR\")\n\n\tif err := asyncBR.StartRestore(pvr.Spec.SnapshotID, datapath.AccessPoint{\n\t\tByPath: res.ByPod.VolumeName,\n\t}, pvr.Spec.UploaderSettings); err != nil {\n\t\treturn errors.Wrapf(err, \"error starting async restore for pod %s, volume %s\", res.ByPod.HostingPod.Name, res.ByPod.VolumeName)\n\t}\n\n\tlog.Infof(\"Async restore started for pod %s, volume %s\", res.ByPod.HostingPod.Name, res.ByPod.VolumeName)\n\treturn nil\n}\n\nfunc (r *PodVolumeRestoreReconciler) errorOut(ctx context.Context, pvr *velerov1api.PodVolumeRestore, err error, msg string, log logrus.FieldLogger) (ctrl.Result, error) {\n\tr.exposer.CleanUp(ctx, getPVROwnerObject(pvr))\n\n\treturn ctrl.Result{}, UpdatePVRStatusToFailed(ctx, r.client, pvr, err, msg, r.clock.Now(), log)\n}\n\nfunc UpdatePVRStatusToFailed(ctx context.Context, c client.Client, pvr *velerov1api.PodVolumeRestore, err error, msg string, time time.Time, log logrus.FieldLogger) error {\n\tlog.Info(\"update PVR status to Failed\")\n\n\tif patchErr := UpdatePVRWithRetry(context.Background(), c, types.NamespacedName{Namespace: pvr.Namespace, Name: pvr.Name}, log,\n\t\tfunc(pvr *velerov1api.PodVolumeRestore) bool {\n\t\t\tif isPVRInFinalState(pvr) {\n\t\t\t\treturn false\n\t\t\t}\n\n\t\t\tpvr.Status.Phase = velerov1api.PodVolumeRestorePhaseFailed\n\t\t\tpvr.Status.Message = errors.WithMessage(err, msg).Error()\n\t\t\tpvr.Status.CompletionTimestamp = &metav1.Time{Time: time}\n\n\t\t\tdelete(pvr.Labels, exposer.ExposeOnGoingLabel)\n\n\t\t\treturn true\n\t\t}); patchErr != nil {\n\t\tlog.WithError(patchErr).Warn(\"error updating PVR status\")\n\t}\n\n\treturn err\n}\n\nfunc shouldProcess(ctx context.Context, client client.Client, log logrus.FieldLogger, pvr *velerov1api.PodVolumeRestore) (bool, *corev1api.Pod, error) {\n\tif !isPVRNew(pvr) {\n\t\tlog.Debug(\"PVR is not new, skip\")\n\t\treturn false, nil, nil\n\t}\n\n\t// we filter the pods during the initialization of cache, if we can get a pod here, the pod must be in the same node with the controller\n\t// so we don't need to compare the node anymore\n\tpod := &corev1api.Pod{}\n\tif err := client.Get(ctx, types.NamespacedName{Namespace: pvr.Spec.Pod.Namespace, Name: pvr.Spec.Pod.Name}, pod); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.WithError(err).Debug(\"Pod not found on this node, skip\")\n\t\t\treturn false, nil, nil\n\t\t}\n\t\tlog.WithError(err).Error(\"Unable to get pod\")\n\t\treturn false, nil, err\n\t}\n\n\tif !isInitContainerRunning(pod) {\n\t\tlog.Debug(\"Pod is not running restore-wait init container, skip\")\n\t\treturn false, nil, nil\n\t}\n\n\treturn true, pod, nil\n}\n\nfunc (r *PodVolumeRestoreReconciler) closeDataPath(ctx context.Context, pvrName string) {\n\tasyncBR := r.dataPathMgr.GetAsyncBR(pvrName)\n\tif asyncBR != nil {\n\t\tasyncBR.Close(ctx)\n\t}\n\n\tr.dataPathMgr.RemoveAsyncBR(pvrName)\n}\n\nfunc (r *PodVolumeRestoreReconciler) SetupWithManager(mgr ctrl.Manager) error {\n\tgp := kube.NewGenericEventPredicate(func(object client.Object) bool {\n\t\tpvr := object.(*velerov1api.PodVolumeRestore)\n\t\tif IsLegacyPVR(pvr) {\n\t\t\treturn false\n\t\t}\n\n\t\tif pvr.Status.Phase == velerov1api.PodVolumeRestorePhaseAccepted {\n\t\t\treturn true\n\t\t}\n\n\t\tif pvr.Spec.Cancel && !isPVRInFinalState(pvr) {\n\t\t\treturn true\n\t\t}\n\n\t\tif isPVRInFinalState(pvr) && !pvr.DeletionTimestamp.IsZero() {\n\t\t\treturn true\n\t\t}\n\n\t\treturn false\n\t})\n\n\ts := kube.NewPeriodicalEnqueueSource(r.logger.WithField(\"controller\", constant.ControllerPodVolumeRestore), r.client, &velerov1api.PodVolumeRestoreList{}, preparingMonitorFrequency, kube.PeriodicalEnqueueSourceOption{\n\t\tPredicates: []predicate.Predicate{gp},\n\t})\n\n\tpred := kube.NewAllEventPredicate(func(obj client.Object) bool {\n\t\tpvr := obj.(*velerov1api.PodVolumeRestore)\n\t\treturn !IsLegacyPVR(pvr)\n\t})\n\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\tFor(&velerov1api.PodVolumeRestore{}, builder.WithPredicates(pred)).\n\t\tWatchesRawSource(s).\n\t\tWatches(&corev1api.Pod{}, handler.EnqueueRequestsFromMapFunc(r.findPVRForTargetPod)).\n\t\tWatches(&corev1api.Pod{}, kube.EnqueueRequestsFromMapUpdateFunc(r.findPVRForRestorePod),\n\t\t\tbuilder.WithPredicates(predicate.Funcs{\n\t\t\t\tUpdateFunc: func(ue event.UpdateEvent) bool {\n\t\t\t\t\tnewObj := ue.ObjectNew.(*corev1api.Pod)\n\n\t\t\t\t\tif _, ok := newObj.Labels[velerov1api.PVRLabel]; !ok {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\n\t\t\t\t\tif newObj.Spec.NodeName == \"\" {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\n\t\t\t\t\treturn true\n\t\t\t\t},\n\t\t\t\tCreateFunc: func(event.CreateEvent) bool {\n\t\t\t\t\treturn false\n\t\t\t\t},\n\t\t\t\tDeleteFunc: func(de event.DeleteEvent) bool {\n\t\t\t\t\treturn false\n\t\t\t\t},\n\t\t\t\tGenericFunc: func(ge event.GenericEvent) bool {\n\t\t\t\t\treturn false\n\t\t\t\t},\n\t\t\t})).\n\t\tComplete(r)\n}\n\nfunc (r *PodVolumeRestoreReconciler) findPVRForTargetPod(ctx context.Context, pod client.Object) []reconcile.Request {\n\tlist := &velerov1api.PodVolumeRestoreList{}\n\toptions := &client.ListOptions{\n\t\tLabelSelector: labels.Set(map[string]string{\n\t\t\tvelerov1api.PodUIDLabel: string(pod.GetUID()),\n\t\t}).AsSelector(),\n\t}\n\tif err := r.client.List(context.TODO(), list, options); err != nil {\n\t\tr.logger.WithField(\"pod\", fmt.Sprintf(\"%s/%s\", pod.GetNamespace(), pod.GetName())).WithError(err).\n\t\t\tError(\"unable to list PodVolumeRestores\")\n\t\treturn []reconcile.Request{}\n\t}\n\n\trequests := []reconcile.Request{}\n\tfor _, item := range list.Items {\n\t\tif IsLegacyPVR(&item) {\n\t\t\tcontinue\n\t\t}\n\n\t\trequests = append(requests, reconcile.Request{\n\t\t\tNamespacedName: types.NamespacedName{\n\t\t\t\tNamespace: item.GetNamespace(),\n\t\t\t\tName: item.GetName(),\n\t\t\t},\n\t\t})\n\t}\n\treturn requests\n}\n\nfunc (r *PodVolumeRestoreReconciler) findPVRForRestorePod(ctx context.Context, podObj client.Object) []reconcile.Request {\n\tpod := podObj.(*corev1api.Pod)\n\tpvr, err := findPVRByRestorePod(r.client, *pod)\n\n\tlog := r.logger.WithField(\"pod\", pod.Name)\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"unable to get PVR\")\n\t\treturn []reconcile.Request{}\n\t} else if pvr == nil {\n\t\tlog.Error(\"get empty PVR\")\n\t\treturn []reconcile.Request{}\n\t}\n\tlog = log.WithFields(logrus.Fields{\n\t\t\"PVR\": pvr.Name,\n\t})\n\n\tif pvr.Status.Phase != velerov1api.PodVolumeRestorePhaseAccepted {\n\t\treturn []reconcile.Request{}\n\t}\n\n\tif pod.Status.Phase == corev1api.PodRunning {\n\t\tlog.Info(\"Preparing PVR\")\n\n\t\tif err = UpdatePVRWithRetry(context.Background(), r.client, types.NamespacedName{Namespace: pvr.Namespace, Name: pvr.Name}, log,\n\t\t\tfunc(pvr *velerov1api.PodVolumeRestore) bool {\n\t\t\t\tif isPVRInFinalState(pvr) {\n\t\t\t\t\tlog.Warnf(\"PVR %s is terminated, abort setting it to prepared\", pvr.Name)\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tpvr.Status.Phase = velerov1api.PodVolumeRestorePhasePrepared\n\t\t\t\treturn true\n\t\t\t}); err != nil {\n\t\t\tlog.WithError(err).Warn(\"failed to update PVR, prepare will halt for this PVR\")\n\t\t\treturn []reconcile.Request{}\n\t\t}\n\t} else if unrecoverable, reason := kube.IsPodUnrecoverable(pod, log); unrecoverable {\n\t\terr := UpdatePVRWithRetry(context.Background(), r.client, types.NamespacedName{Namespace: pvr.Namespace, Name: pvr.Name}, log,\n\t\t\tfunc(pvr *velerov1api.PodVolumeRestore) bool {\n\t\t\t\tif pvr.Spec.Cancel {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tpvr.Spec.Cancel = true\n\t\t\t\tpvr.Status.Message = fmt.Sprintf(\"Cancel PVR because the exposing pod %s/%s is in abnormal status for reason %s\", pod.Namespace, pod.Name, reason)\n\n\t\t\t\treturn true\n\t\t\t})\n\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Warn(\"failed to cancel PVR, and it will wait for prepare timeout\")\n\t\t\treturn []reconcile.Request{}\n\t\t}\n\n\t\tlog.Infof(\"Exposed pod is in abnormal status(reason %s) and PVR is marked as cancel\", reason)\n\t} else {\n\t\treturn []reconcile.Request{}\n\t}\n\n\trequest := reconcile.Request{\n\t\tNamespacedName: types.NamespacedName{\n\t\t\tNamespace: pvr.Namespace,\n\t\t\tName: pvr.Name,\n\t\t},\n\t}\n\treturn []reconcile.Request{request}\n}\n\nfunc isPVRNew(pvr *velerov1api.PodVolumeRestore) bool {\n\treturn pvr.Status.Phase == \"\" || pvr.Status.Phase == velerov1api.PodVolumeRestorePhaseNew\n}\n\nfunc isInitContainerRunning(pod *corev1api.Pod) bool {\n\t// Pod volume wait container can be anywhere in the list of init containers, but must be running.\n\ti := getInitContainerIndex(pod)\n\treturn i >= 0 &&\n\t\tlen(pod.Status.InitContainerStatuses)-1 >= i &&\n\t\tpod.Status.InitContainerStatuses[i].State.Running != nil\n}\n\nfunc getInitContainerIndex(pod *corev1api.Pod) int {\n\t// Pod volume wait container can be anywhere in the list of init containers so locate it.\n\tfor i, initContainer := range pod.Spec.InitContainers {\n\t\tif initContainer.Name == restorehelper.WaitInitContainer {\n\t\t\treturn i\n\t\t}\n\t}\n\n\treturn -1\n}\n\nfunc (r *PodVolumeRestoreReconciler) OnDataPathCompleted(ctx context.Context, namespace string, pvrName string, result datapath.Result) {\n\tdefer r.dataPathMgr.RemoveAsyncBR(pvrName)\n\n\tlog := r.logger.WithField(\"PVR\", pvrName)\n\n\tlog.WithField(\"PVR\", pvrName).WithField(\"result\", result.Restore).Info(\"Async fs restore data path completed\")\n\n\tvar pvr velerov1api.PodVolumeRestore\n\tif err := r.client.Get(ctx, types.NamespacedName{Name: pvrName, Namespace: namespace}, &pvr); err != nil {\n\t\tlog.WithError(err).Warn(\"Failed to get PVR on completion\")\n\t\treturn\n\t}\n\n\tlog.Info(\"Cleaning up exposed environment\")\n\tr.exposer.CleanUp(ctx, getPVROwnerObject(&pvr))\n\n\tif err := UpdatePVRWithRetry(ctx, r.client, types.NamespacedName{Namespace: pvr.Namespace, Name: pvr.Name}, log, func(pvr *velerov1api.PodVolumeRestore) bool {\n\t\tif isPVRInFinalState(pvr) {\n\t\t\treturn false\n\t\t}\n\n\t\tpvr.Status.Phase = velerov1api.PodVolumeRestorePhaseCompleted\n\t\tpvr.Status.CompletionTimestamp = &metav1.Time{Time: r.clock.Now()}\n\n\t\tdelete(pvr.Labels, exposer.ExposeOnGoingLabel)\n\n\t\treturn true\n\t}); err != nil {\n\t\tlog.WithError(err).Error(\"error updating PVR status\")\n\t} else {\n\t\tlog.Info(\"Restore completed\")\n\t}\n}\n\nfunc (r *PodVolumeRestoreReconciler) OnDataPathFailed(ctx context.Context, namespace string, pvrName string, err error) {\n\tdefer r.dataPathMgr.RemoveAsyncBR(pvrName)\n\n\tlog := r.logger.WithField(\"PVR\", pvrName)\n\n\tlog.WithError(err).Error(\"Async fs restore data path failed\")\n\n\tvar pvr velerov1api.PodVolumeRestore\n\tif getErr := r.client.Get(ctx, types.NamespacedName{Name: pvrName, Namespace: namespace}, &pvr); getErr != nil {\n\t\tlog.WithError(getErr).Warn(\"Failed to get PVR on failure\")\n\t} else {\n\t\t_, _ = r.errorOut(ctx, &pvr, err, \"data path restore failed\", log)\n\t}\n}\n\nfunc (r *PodVolumeRestoreReconciler) OnDataPathCancelled(ctx context.Context, namespace string, pvrName string) {\n\tdefer r.dataPathMgr.RemoveAsyncBR(pvrName)\n\n\tlog := r.logger.WithField(\"PVR\", pvrName)\n\n\tlog.Warn(\"Async fs restore data path canceled\")\n\n\tvar pvr velerov1api.PodVolumeRestore\n\tif getErr := r.client.Get(ctx, types.NamespacedName{Name: pvrName, Namespace: namespace}, &pvr); getErr != nil {\n\t\tlog.WithError(getErr).Warn(\"Failed to get PVR on cancel\")\n\t\treturn\n\t}\n\t// cleans up any objects generated during the snapshot expose\n\tr.exposer.CleanUp(ctx, getPVROwnerObject(&pvr))\n\n\tif err := UpdatePVRWithRetry(ctx, r.client, types.NamespacedName{Namespace: pvr.Namespace, Name: pvr.Name}, log, func(pvr *velerov1api.PodVolumeRestore) bool {\n\t\tif isPVRInFinalState(pvr) {\n\t\t\treturn false\n\t\t}\n\n\t\tpvr.Status.Phase = velerov1api.PodVolumeRestorePhaseCanceled\n\t\tif pvr.Status.StartTimestamp.IsZero() {\n\t\t\tpvr.Status.StartTimestamp = &metav1.Time{Time: r.clock.Now()}\n\t\t}\n\t\tpvr.Status.CompletionTimestamp = &metav1.Time{Time: r.clock.Now()}\n\n\t\tdelete(pvr.Labels, exposer.ExposeOnGoingLabel)\n\n\t\treturn true\n\t}); err != nil {\n\t\tlog.WithError(err).Error(\"error updating PVR status on cancel\")\n\t} else {\n\t\tdelete(r.cancelledPVR, pvr.Name)\n\t}\n}\n\nfunc (r *PodVolumeRestoreReconciler) OnDataPathProgress(ctx context.Context, namespace string, pvrName string, progress *uploader.Progress) {\n\tlog := r.logger.WithField(\"PVR\", pvrName)\n\n\tif err := UpdatePVRWithRetry(ctx, r.client, types.NamespacedName{Namespace: namespace, Name: pvrName}, log, func(pvr *velerov1api.PodVolumeRestore) bool {\n\t\tpvr.Status.Progress = veleroapishared.DataMoveOperationProgress{TotalBytes: progress.TotalBytes, BytesDone: progress.BytesDone}\n\t\treturn true\n\t}); err != nil {\n\t\tlog.WithError(err).Error(\"Failed to update progress\")\n\t}\n}\n\nfunc (r *PodVolumeRestoreReconciler) setupExposeParam(pvr *velerov1api.PodVolumeRestore) exposer.PodVolumeExposeParam {\n\tlog := r.logger.WithField(\"PVR\", pvr.Name)\n\n\tnodeOS, err := kube.GetNodeOS(context.Background(), pvr.Status.Node, r.kubeClient.CoreV1())\n\tif err != nil {\n\t\tlog.WithError(err).Warnf(\"Failed to get nodeOS for node %s, use linux node-agent for hosting pod labels, annotations and tolerations\", pvr.Status.Node)\n\t}\n\n\thostingPodLabels := map[string]string{velerov1api.PVRLabel: pvr.Name}\n\tif len(r.podLabels) > 0 {\n\t\tfor k, v := range r.podLabels {\n\t\t\thostingPodLabels[k] = v\n\t\t}\n\t} else {\n\t\tfor _, k := range util.ThirdPartyLabels {\n\t\t\tif v, err := nodeagent.GetLabelValue(context.Background(), r.kubeClient, pvr.Namespace, k, nodeOS); err != nil {\n\t\t\t\tif err != nodeagent.ErrNodeAgentLabelNotFound {\n\t\t\t\t\tlog.WithError(err).Warnf(\"Failed to check node-agent label, skip adding host pod label %s\", k)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\thostingPodLabels[k] = v\n\t\t\t}\n\t\t}\n\t}\n\n\thostingPodAnnotation := map[string]string{}\n\tif len(r.podAnnotations) > 0 {\n\t\tfor k, v := range r.podAnnotations {\n\t\t\thostingPodAnnotation[k] = v\n\t\t}\n\t} else {\n\t\tfor _, k := range util.ThirdPartyAnnotations {\n\t\t\tif v, err := nodeagent.GetAnnotationValue(context.Background(), r.kubeClient, pvr.Namespace, k, nodeOS); err != nil {\n\t\t\t\tif err != nodeagent.ErrNodeAgentAnnotationNotFound {\n\t\t\t\t\tlog.WithError(err).Warnf(\"Failed to check node-agent annotation, skip adding host pod annotation %s\", k)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\thostingPodAnnotation[k] = v\n\t\t\t}\n\t\t}\n\t}\n\n\thostingPodTolerations := []corev1api.Toleration{}\n\tfor _, k := range util.ThirdPartyTolerations {\n\t\tif v, err := nodeagent.GetToleration(context.Background(), r.kubeClient, pvr.Namespace, k, nodeOS); err != nil {\n\t\t\tif err != nodeagent.ErrNodeAgentTolerationNotFound {\n\t\t\t\tlog.WithError(err).Warnf(\"Failed to check node-agent toleration, skip adding host pod toleration %s\", k)\n\t\t\t}\n\t\t} else {\n\t\t\thostingPodTolerations = append(hostingPodTolerations, *v)\n\t\t}\n\t}\n\n\tvar cacheVolume *exposer.CacheConfigs\n\tif r.cacheVolumeConfigs != nil {\n\t\tif limit, err := r.repoConfigMgr.ClientSideCacheLimit(velerov1api.BackupRepositoryTypeKopia, r.backupRepoConfigs); err != nil {\n\t\t\tlog.WithError(err).Warnf(\"Failed to get client side cache limit for repo type %s from configs %v\", velerov1api.BackupRepositoryTypeKopia, r.backupRepoConfigs)\n\t\t} else {\n\t\t\tcacheVolume = &exposer.CacheConfigs{\n\t\t\t\tLimit: limit,\n\t\t\t\tStorageClass: r.cacheVolumeConfigs.StorageClass,\n\t\t\t\tResidentThreshold: r.cacheVolumeConfigs.ResidentThresholdInMB << 20,\n\t\t\t}\n\t\t}\n\t}\n\n\treturn exposer.PodVolumeExposeParam{\n\t\tType: exposer.PodVolumeExposeTypeRestore,\n\t\tClientNamespace: pvr.Spec.Pod.Namespace,\n\t\tClientPodName: pvr.Spec.Pod.Name,\n\t\tClientPodVolume: pvr.Spec.Volume,\n\t\tHostingPodLabels: hostingPodLabels,\n\t\tHostingPodAnnotations: hostingPodAnnotation,\n\t\tHostingPodTolerations: hostingPodTolerations,\n\t\tOperationTimeout: r.resourceTimeout,\n\t\tResources: r.podResources,\n\t\tRestoreSize: pvr.Spec.SnapshotSize,\n\t\tCacheVolume: cacheVolume,\n\t\t// Priority class name for the data mover pod, retrieved from node-agent-configmap\n\t\tPriorityClassName: r.dataMovePriorityClass,\n\t\tPrivileged: r.privileged,\n\t}\n}\n\nfunc getPVROwnerObject(pvr *velerov1api.PodVolumeRestore) corev1api.ObjectReference {\n\treturn corev1api.ObjectReference{\n\t\tKind: pvr.Kind,\n\t\tNamespace: pvr.Namespace,\n\t\tName: pvr.Name,\n\t\tUID: pvr.UID,\n\t\tAPIVersion: pvr.APIVersion,\n\t}\n}\n\nfunc findPVRByRestorePod(client client.Client, pod corev1api.Pod) (*velerov1api.PodVolumeRestore, error) {\n\tif label, exist := pod.Labels[velerov1api.PVRLabel]; exist {\n\t\tpvr := &velerov1api.PodVolumeRestore{}\n\t\terr := client.Get(context.Background(), types.NamespacedName{\n\t\t\tNamespace: pod.Namespace,\n\t\t\tName: label,\n\t\t}, pvr)\n\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"error to find PVR by pod %s/%s\", pod.Namespace, pod.Name)\n\t\t}\n\t\treturn pvr, nil\n\t}\n\treturn nil, nil\n}\n\nfunc isPVRInFinalState(pvr *velerov1api.PodVolumeRestore) bool {\n\treturn pvr.Status.Phase == velerov1api.PodVolumeRestorePhaseFailed ||\n\t\tpvr.Status.Phase == velerov1api.PodVolumeRestorePhaseCanceled ||\n\t\tpvr.Status.Phase == velerov1api.PodVolumeRestorePhaseCompleted\n}\n\nfunc UpdatePVRWithRetry(ctx context.Context, client client.Client, namespacedName types.NamespacedName, log logrus.FieldLogger, updateFunc func(*velerov1api.PodVolumeRestore) bool) error {\n\treturn wait.PollUntilContextCancel(ctx, time.Millisecond*100, true, func(ctx context.Context) (bool, error) {\n\t\tpvr := &velerov1api.PodVolumeRestore{}\n\t\tif err := client.Get(ctx, namespacedName, pvr); err != nil {\n\t\t\treturn false, errors.Wrap(err, \"getting PVR\")\n\t\t}\n\n\t\tif updateFunc(pvr) {\n\t\t\terr := client.Update(ctx, pvr)\n\t\t\tif err != nil {\n\t\t\t\tif apierrors.IsConflict(err) {\n\t\t\t\t\tlog.Debugf(\"failed to update PVR for %s/%s and will retry it\", pvr.Namespace, pvr.Name)\n\t\t\t\t\treturn false, nil\n\t\t\t\t} else {\n\t\t\t\t\treturn false, errors.Wrapf(err, \"error updating PVR %s/%s\", pvr.Namespace, pvr.Name)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\treturn true, nil\n\t})\n}\n\nvar funcResumeCancellablePVR = (*PodVolumeRestoreReconciler).resumeCancellableDataPath\n\nfunc (r *PodVolumeRestoreReconciler) AttemptPVRResume(ctx context.Context, logger *logrus.Entry, ns string) error {\n\tpvrs := &velerov1api.PodVolumeRestoreList{}\n\tif err := r.client.List(ctx, pvrs, &client.ListOptions{Namespace: ns}); err != nil {\n\t\tr.logger.WithError(errors.WithStack(err)).Error(\"failed to list PVRs\")\n\t\treturn errors.Wrapf(err, \"error to list PVRs\")\n\t}\n\n\tfor i := range pvrs.Items {\n\t\tpvr := &pvrs.Items[i]\n\t\tif IsLegacyPVR(pvr) {\n\t\t\tcontinue\n\t\t}\n\n\t\tif pvr.Status.Phase == velerov1api.PodVolumeRestorePhaseInProgress {\n\t\t\tif pvr.Status.Node != r.nodeName {\n\t\t\t\tlogger.WithField(\"PVR\", pvr.Name).WithField(\"current node\", r.nodeName).Infof(\"PVR should be resumed by another node %s\", pvr.Status.Node)\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\terr := funcResumeCancellablePVR(r, ctx, pvr, logger)\n\t\t\tif err == nil {\n\t\t\t\tlogger.WithField(\"PVR\", pvr.Name).WithField(\"current node\", r.nodeName).Info(\"Completed to resume in progress PVR\")\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tlogger.WithField(\"PVR\", pvr.GetName()).WithError(err).Warn(\"Failed to resume data path for PVR, have to cancel it\")\n\n\t\t\tresumeErr := err\n\t\t\terr = UpdatePVRWithRetry(ctx, r.client, types.NamespacedName{Namespace: pvr.Namespace, Name: pvr.Name}, logger.WithField(\"PVR\", pvr.Name),\n\t\t\t\tfunc(pvr *velerov1api.PodVolumeRestore) bool {\n\t\t\t\t\tif pvr.Spec.Cancel {\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\n\t\t\t\t\tpvr.Spec.Cancel = true\n\t\t\t\t\tpvr.Status.Message = fmt.Sprintf(\"Resume InProgress PVR failed with error %v, mark it as cancel\", resumeErr)\n\n\t\t\t\t\treturn true\n\t\t\t\t})\n\t\t\tif err != nil {\n\t\t\t\tlogger.WithField(\"PVR\", pvr.GetName()).WithError(errors.WithStack(err)).Error(\"Failed to trigger PVR cancel\")\n\t\t\t}\n\t\t} else if !isPVRInFinalState(pvr) {\n\t\t\tlogger.WithField(\"PVR\", pvr.GetName()).Infof(\"find a PVR with status %s\", pvr.Status.Phase)\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (r *PodVolumeRestoreReconciler) resumeCancellableDataPath(ctx context.Context, pvr *velerov1api.PodVolumeRestore, log logrus.FieldLogger) error {\n\tlog.Info(\"Resume cancelable PVR\")\n\n\tres, err := r.exposer.GetExposed(ctx, getPVROwnerObject(pvr), r.client, r.nodeName, r.resourceTimeout)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to get exposed PVR %s\", pvr.Name)\n\t}\n\n\tif res == nil {\n\t\treturn errors.Errorf(\"no expose result is available for the current node for PVR %s\", pvr.Name)\n\t}\n\n\tcallbacks := datapath.Callbacks{\n\t\tOnCompleted: r.OnDataPathCompleted,\n\t\tOnFailed: r.OnDataPathFailed,\n\t\tOnCancelled: r.OnDataPathCancelled,\n\t\tOnProgress: r.OnDataPathProgress,\n\t}\n\n\tasyncBR, err := r.dataPathMgr.CreateMicroServiceBRWatcher(ctx, r.client, r.kubeClient, r.mgr, datapath.TaskTypeRestore, pvr.Name, pvr.Namespace, res.ByPod.HostingPod.Name, res.ByPod.HostingContainer, pvr.Name, callbacks, true, log)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to create asyncBR watcher for PVR %s\", pvr.Name)\n\t}\n\n\tresumeComplete := false\n\tdefer func() {\n\t\tif !resumeComplete {\n\t\t\tr.closeDataPath(ctx, pvr.Name)\n\t\t}\n\t}()\n\n\tif err := asyncBR.Init(ctx, nil); err != nil {\n\t\treturn errors.Wrapf(err, \"error to init asyncBR watcher for PVR %s\", pvr.Name)\n\t}\n\n\tif err := asyncBR.StartRestore(pvr.Spec.SnapshotID, datapath.AccessPoint{\n\t\tByPath: res.ByPod.VolumeName,\n\t}, pvr.Spec.UploaderSettings); err != nil {\n\t\treturn errors.Wrapf(err, \"error to resume asyncBR watcher for PVR %s\", pvr.Name)\n\t}\n\n\tresumeComplete = true\n\n\tlog.Infof(\"asyncBR is resumed for PVR %s\", pvr.Name)\n\n\treturn nil\n}\n" }, { "path": "pkg/controller/pod_volume_restore_controller_legacy.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/client-go/kubernetes\"\n\tclocks \"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/builder\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/handler\"\n\t\"sigs.k8s.io/controller-runtime/pkg/manager\"\n\t\"sigs.k8s.io/controller-runtime/pkg/reconcile\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\tveleroapishared \"github.com/vmware-tanzu/velero/pkg/apis/velero/shared\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\t\"github.com/vmware-tanzu/velero/pkg/exposer\"\n\t\"github.com/vmware-tanzu/velero/pkg/podvolume\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\t\"github.com/vmware-tanzu/velero/pkg/restorehelper\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nfunc InitLegacyPodVolumeRestoreReconciler(client client.Client, mgr manager.Manager, kubeClient kubernetes.Interface, dataPathMgr *datapath.Manager, namespace string,\n\tresourceTimeout time.Duration, logger logrus.FieldLogger) error {\n\tlog := logger.WithField(\"controller\", \"PodVolumeRestoreLegacy\")\n\n\tcredentialFileStore, err := credentials.NewNamespacedFileStore(client, namespace, credentials.DefaultStoreDirectory(), filesystem.NewFileSystem())\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error creating credentials file store\")\n\t}\n\n\tcredSecretStore, err := credentials.NewNamespacedSecretStore(client, namespace)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error creating secret file store\")\n\t}\n\n\tcredentialGetter := &credentials.CredentialGetter{FromFile: credentialFileStore, FromSecret: credSecretStore}\n\tensurer := repository.NewEnsurer(client, log, resourceTimeout)\n\n\treconciler := &PodVolumeRestoreReconcilerLegacy{\n\t\tClient: client,\n\t\tkubeClient: kubeClient,\n\t\tlogger: log,\n\t\trepositoryEnsurer: ensurer,\n\t\tcredentialGetter: credentialGetter,\n\t\tfileSystem: filesystem.NewFileSystem(),\n\t\tclock: &clocks.RealClock{},\n\t\tdataPathMgr: dataPathMgr,\n\t}\n\n\tif err = reconciler.SetupWithManager(mgr); err != nil {\n\t\treturn errors.Wrapf(err, \"error setup controller manager\")\n\t}\n\n\treturn nil\n}\n\ntype PodVolumeRestoreReconcilerLegacy struct {\n\tclient.Client\n\tkubeClient kubernetes.Interface\n\tlogger logrus.FieldLogger\n\trepositoryEnsurer *repository.Ensurer\n\tcredentialGetter *credentials.CredentialGetter\n\tfileSystem filesystem.Interface\n\tclock clocks.WithTickerAndDelayedExecution\n\tdataPathMgr *datapath.Manager\n}\n\n// +kubebuilder:rbac:groups=velero.io,resources=podvolumerestores,verbs=get;list;watch;create;update;patch;delete\n// +kubebuilder:rbac:groups=velero.io,resources=podvolumerestores/status,verbs=get;update;patch\n// +kubebuilder:rbac:groups=\"\",resources=pods,verbs=get\n// +kubebuilder:rbac:groups=\"\",resources=persistentvolumes,verbs=get\n// +kubebuilder:rbac:groups=\"\",resources=persistentvolumerclaims,verbs=get\n\nfunc (c *PodVolumeRestoreReconcilerLegacy) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\tlog := c.logger.WithField(\"PodVolumeRestore\", req.NamespacedName.String())\n\tlog.Info(\"Reconciling PVR by legacy controller\")\n\n\tpvr := &velerov1api.PodVolumeRestore{}\n\tif err := c.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: req.Name}, pvr); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.Warn(\"PodVolumeRestore not found, skip\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\tlog.WithError(err).Error(\"Unable to get the PodVolumeRestore\")\n\t\treturn ctrl.Result{}, err\n\t}\n\n\tlog = log.WithField(\"pod\", fmt.Sprintf(\"%s/%s\", pvr.Spec.Pod.Namespace, pvr.Spec.Pod.Name))\n\tif len(pvr.OwnerReferences) == 1 {\n\t\tlog = log.WithField(\"restore\", fmt.Sprintf(\"%s/%s\", pvr.Namespace, pvr.OwnerReferences[0].Name))\n\t}\n\n\tshouldProcess, pod, err := shouldProcess(ctx, c.Client, log, pvr)\n\tif err != nil {\n\t\treturn ctrl.Result{}, err\n\t}\n\tif !shouldProcess {\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\tinitContainerIndex := getInitContainerIndex(pod)\n\tif initContainerIndex > 0 {\n\t\tlog.Warnf(`Init containers before the %s container may cause issues\n\t\t if they interfere with volumes being restored: %s index %d`, restorehelper.WaitInitContainer, restorehelper.WaitInitContainer, initContainerIndex)\n\t}\n\n\tlog.Info(\"Restore starting\")\n\n\tcallbacks := datapath.Callbacks{\n\t\tOnCompleted: c.OnDataPathCompleted,\n\t\tOnFailed: c.OnDataPathFailed,\n\t\tOnCancelled: c.OnDataPathCancelled,\n\t\tOnProgress: c.OnDataPathProgress,\n\t}\n\n\tfsRestore, err := c.dataPathMgr.CreateFileSystemBR(pvr.Name, pVBRRequestor, ctx, c.Client, pvr.Namespace, callbacks, log)\n\tif err != nil {\n\t\tif err == datapath.ConcurrentLimitExceed {\n\t\t\treturn ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5}, nil\n\t\t} else {\n\t\t\treturn c.errorOut(ctx, pvr, err, \"error to create data path\", log)\n\t\t}\n\t}\n\n\toriginal := pvr.DeepCopy()\n\tpvr.Status.Phase = velerov1api.PodVolumeRestorePhaseInProgress\n\tpvr.Status.StartTimestamp = &metav1.Time{Time: c.clock.Now()}\n\tif err = c.Patch(ctx, pvr, client.MergeFrom(original)); err != nil {\n\t\tc.closeDataPath(ctx, pvr.Name)\n\t\treturn c.errorOut(ctx, pvr, err, \"error to update status to in progress\", log)\n\t}\n\n\tvolumePath, err := exposer.GetPodVolumeHostPath(ctx, pod, pvr.Spec.Volume, c.kubeClient, c.fileSystem, log)\n\tif err != nil {\n\t\tc.closeDataPath(ctx, pvr.Name)\n\t\treturn c.errorOut(ctx, pvr, err, \"error exposing host path for pod volume\", log)\n\t}\n\n\tlog.WithField(\"path\", volumePath.ByPath).Debugf(\"Found host path\")\n\n\tif err := fsRestore.Init(ctx, &datapath.FSBRInitParam{\n\t\tBSLName: pvr.Spec.BackupStorageLocation,\n\t\tSourceNamespace: pvr.Spec.SourceNamespace,\n\t\tUploaderType: pvr.Spec.UploaderType,\n\t\tRepositoryType: podvolume.GetPvrRepositoryType(pvr),\n\t\tRepoIdentifier: pvr.Spec.RepoIdentifier,\n\t\tRepositoryEnsurer: c.repositoryEnsurer,\n\t\tCredentialGetter: c.credentialGetter,\n\t}); err != nil {\n\t\tc.closeDataPath(ctx, pvr.Name)\n\t\treturn c.errorOut(ctx, pvr, err, \"error to initialize data path\", log)\n\t}\n\n\tif err := fsRestore.StartRestore(pvr.Spec.SnapshotID, volumePath, pvr.Spec.UploaderSettings); err != nil {\n\t\tc.closeDataPath(ctx, pvr.Name)\n\t\treturn c.errorOut(ctx, pvr, err, \"error starting data path restore\", log)\n\t}\n\n\tlog.WithField(\"path\", volumePath.ByPath).Info(\"Async fs restore data path started\")\n\n\treturn ctrl.Result{}, nil\n}\n\nfunc (c *PodVolumeRestoreReconcilerLegacy) errorOut(ctx context.Context, pvr *velerov1api.PodVolumeRestore, err error, msg string, log logrus.FieldLogger) (ctrl.Result, error) {\n\t_ = UpdatePVRStatusToFailed(ctx, c.Client, pvr, err, msg, c.clock.Now(), log)\n\treturn ctrl.Result{}, err\n}\n\nfunc (c *PodVolumeRestoreReconcilerLegacy) SetupWithManager(mgr ctrl.Manager) error {\n\t// The pod may not being scheduled at the point when its PVRs are initially reconciled.\n\t// By watching the pods, we can trigger the PVR reconciliation again once the pod is finally scheduled on the node.\n\tpred := kube.NewAllEventPredicate(func(obj client.Object) bool {\n\t\tpvr := obj.(*velerov1api.PodVolumeRestore)\n\t\treturn IsLegacyPVR(pvr)\n\t})\n\n\treturn ctrl.NewControllerManagedBy(mgr).Named(\"podvolumerestorelegacy\").\n\t\tFor(&velerov1api.PodVolumeRestore{}, builder.WithPredicates(pred)).\n\t\tWatches(&corev1api.Pod{}, handler.EnqueueRequestsFromMapFunc(c.findVolumeRestoresForPod)).\n\t\tComplete(c)\n}\n\nfunc (c *PodVolumeRestoreReconcilerLegacy) findVolumeRestoresForPod(ctx context.Context, pod client.Object) []reconcile.Request {\n\tlist := &velerov1api.PodVolumeRestoreList{}\n\toptions := &client.ListOptions{\n\t\tLabelSelector: labels.Set(map[string]string{\n\t\t\tvelerov1api.PodUIDLabel: string(pod.GetUID()),\n\t\t}).AsSelector(),\n\t}\n\tif err := c.Client.List(context.TODO(), list, options); err != nil {\n\t\tc.logger.WithField(\"pod\", fmt.Sprintf(\"%s/%s\", pod.GetNamespace(), pod.GetName())).WithError(err).\n\t\t\tError(\"unable to list PodVolumeRestores\")\n\t\treturn []reconcile.Request{}\n\t}\n\n\trequests := []reconcile.Request{}\n\tfor _, item := range list.Items {\n\t\tif !IsLegacyPVR(&item) {\n\t\t\tcontinue\n\t\t}\n\n\t\trequests = append(requests, reconcile.Request{\n\t\t\tNamespacedName: types.NamespacedName{\n\t\t\t\tNamespace: item.GetNamespace(),\n\t\t\t\tName: item.GetName(),\n\t\t\t},\n\t\t})\n\t}\n\treturn requests\n}\n\nfunc (c *PodVolumeRestoreReconcilerLegacy) OnDataPathCompleted(ctx context.Context, namespace string, pvrName string, result datapath.Result) {\n\tdefer c.dataPathMgr.RemoveAsyncBR(pvrName)\n\n\tlog := c.logger.WithField(\"pvr\", pvrName)\n\n\tlog.WithField(\"PVR\", pvrName).Info(\"Async fs restore data path completed\")\n\n\tvar pvr velerov1api.PodVolumeRestore\n\tif err := c.Client.Get(ctx, types.NamespacedName{Name: pvrName, Namespace: namespace}, &pvr); err != nil {\n\t\tlog.WithError(err).Warn(\"Failed to get PVR on completion\")\n\t\treturn\n\t}\n\n\tvolumePath := result.Restore.Target.ByPath\n\tif volumePath == \"\" {\n\t\t_, _ = c.errorOut(ctx, &pvr, errors.New(\"path is empty\"), \"invalid restore target\", log)\n\t\treturn\n\t}\n\n\t// Remove the .velero directory from the restored volume (it may contain done files from previous restores\n\t// of this volume, which we don't want to carry over). If this fails for any reason, log and continue, since\n\t// this is non-essential cleanup (the done files are named based on restore UID and the init container looks\n\t// for the one specific to the restore being executed).\n\tif err := os.RemoveAll(filepath.Join(volumePath, \".velero\")); err != nil {\n\t\tlog.WithError(err).Warnf(\"error removing .velero directory from directory %s\", volumePath)\n\t}\n\n\tvar restoreUID types.UID\n\tfor _, owner := range pvr.OwnerReferences {\n\t\tif boolptr.IsSetToTrue(owner.Controller) {\n\t\t\trestoreUID = owner.UID\n\t\t\tbreak\n\t\t}\n\t}\n\n\t// Create the .velero directory within the volume dir so we can write a done file\n\t// for this restore.\n\tif err := os.MkdirAll(filepath.Join(volumePath, \".velero\"), 0755); err != nil {\n\t\t_, _ = c.errorOut(ctx, &pvr, err, \"error creating .velero directory for done file\", log)\n\t\treturn\n\t}\n\n\t// Write a done file with name= into the just-created .velero dir\n\t// within the volume. The velero init container on the pod is waiting\n\t// for this file to exist in each restored volume before completing.\n\tif err := os.WriteFile(filepath.Join(volumePath, \".velero\", string(restoreUID)), nil, 0644); err != nil { //nolint:gosec // Internal usage. No need to check.\n\t\t_, _ = c.errorOut(ctx, &pvr, err, \"error writing done file\", log)\n\t\treturn\n\t}\n\n\toriginal := pvr.DeepCopy()\n\tpvr.Status.Phase = velerov1api.PodVolumeRestorePhaseCompleted\n\tpvr.Status.CompletionTimestamp = &metav1.Time{Time: c.clock.Now()}\n\tif err := c.Patch(ctx, &pvr, client.MergeFrom(original)); err != nil {\n\t\tlog.WithError(err).Error(\"error updating PodVolumeRestore status\")\n\t}\n\n\tlog.Info(\"Restore completed\")\n}\n\nfunc (c *PodVolumeRestoreReconcilerLegacy) OnDataPathFailed(ctx context.Context, namespace string, pvrName string, err error) {\n\tdefer c.dataPathMgr.RemoveAsyncBR(pvrName)\n\n\tlog := c.logger.WithField(\"pvr\", pvrName)\n\n\tlog.WithError(err).Error(\"Async fs restore data path failed\")\n\n\tvar pvr velerov1api.PodVolumeRestore\n\tif getErr := c.Client.Get(ctx, types.NamespacedName{Name: pvrName, Namespace: namespace}, &pvr); getErr != nil {\n\t\tlog.WithError(getErr).Warn(\"Failed to get PVR on failure\")\n\t} else {\n\t\t_, _ = c.errorOut(ctx, &pvr, err, \"data path restore failed\", log)\n\t}\n}\n\nfunc (c *PodVolumeRestoreReconcilerLegacy) OnDataPathCancelled(ctx context.Context, namespace string, pvrName string) {\n\tdefer c.dataPathMgr.RemoveAsyncBR(pvrName)\n\n\tlog := c.logger.WithField(\"pvr\", pvrName)\n\n\tlog.Warn(\"Async fs restore data path canceled\")\n\n\tvar pvr velerov1api.PodVolumeRestore\n\tif getErr := c.Client.Get(ctx, types.NamespacedName{Name: pvrName, Namespace: namespace}, &pvr); getErr != nil {\n\t\tlog.WithError(getErr).Warn(\"Failed to get PVR on cancel\")\n\t} else {\n\t\t_, _ = c.errorOut(ctx, &pvr, errors.New(\"PVR is canceled\"), \"data path restore canceled\", log)\n\t}\n}\n\nfunc (c *PodVolumeRestoreReconcilerLegacy) OnDataPathProgress(ctx context.Context, namespace string, pvrName string, progress *uploader.Progress) {\n\tlog := c.logger.WithField(\"pvr\", pvrName)\n\n\tvar pvr velerov1api.PodVolumeRestore\n\tif err := c.Client.Get(ctx, types.NamespacedName{Name: pvrName, Namespace: namespace}, &pvr); err != nil {\n\t\tlog.WithError(err).Warn(\"Failed to get PVB on progress\")\n\t\treturn\n\t}\n\n\toriginal := pvr.DeepCopy()\n\tpvr.Status.Progress = veleroapishared.DataMoveOperationProgress{TotalBytes: progress.TotalBytes, BytesDone: progress.BytesDone}\n\n\tif err := c.Client.Patch(ctx, &pvr, client.MergeFrom(original)); err != nil {\n\t\tlog.WithError(err).Error(\"Failed to update progress\")\n\t}\n}\n\nfunc (c *PodVolumeRestoreReconcilerLegacy) closeDataPath(ctx context.Context, pvbName string) {\n\tfsRestore := c.dataPathMgr.GetAsyncBR(pvbName)\n\tif fsRestore != nil {\n\t\tfsRestore.Close(ctx)\n\t}\n\n\tc.dataPathMgr.RemoveAsyncBR(pvbName)\n}\n\nfunc IsLegacyPVR(pvr *velerov1api.PodVolumeRestore) bool {\n\treturn pvr.Spec.UploaderType == uploader.ResticType\n}\n" }, { "path": "pkg/controller/pod_volume_restore_controller_legacy_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\nfunc TestFindVolumeRestoresForPodLegacy(t *testing.T) {\n\tpod := &corev1api.Pod{}\n\tpod.UID = \"uid\"\n\n\tscheme := runtime.NewScheme()\n\tscheme.AddKnownTypes(velerov1api.SchemeGroupVersion, &velerov1api.PodVolumeRestore{}, &velerov1api.PodVolumeRestoreList{})\n\tclientBuilder := fake.NewClientBuilder().WithScheme(scheme)\n\n\t// no matching PVR\n\treconciler := &PodVolumeRestoreReconcilerLegacy{\n\t\tClient: clientBuilder.Build(),\n\t\tlogger: logrus.New(),\n\t}\n\trequests := reconciler.findVolumeRestoresForPod(t.Context(), pod)\n\tassert.Empty(t, requests)\n\n\t// contain one matching PVR\n\treconciler.Client = clientBuilder.WithLists(&velerov1api.PodVolumeRestoreList{\n\t\tItems: []velerov1api.PodVolumeRestore{\n\t\t\t{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"pvr1\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\tvelerov1api.PodUIDLabel: string(pod.GetUID()),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"pvr2\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\tvelerov1api.PodUIDLabel: \"non-matching-uid\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"pvr3\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\tvelerov1api.PodUIDLabel: string(pod.GetUID()),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.PodVolumeRestoreSpec{\n\t\t\t\t\tUploaderType: \"kopia\",\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"pvr4\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\tvelerov1api.PodUIDLabel: string(pod.GetUID()),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.PodVolumeRestoreSpec{\n\t\t\t\t\tUploaderType: \"restic\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}).Build()\n\trequests = reconciler.findVolumeRestoresForPod(t.Context(), pod)\n\tassert.Len(t, requests, 1)\n}\n" }, { "path": "pkg/controller/pod_volume_restore_controller_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/client-go/kubernetes\"\n\tclientgofake \"k8s.io/client-go/kubernetes/fake\"\n\tclocks \"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\t\"sigs.k8s.io/controller-runtime/pkg/manager\"\n\t\"sigs.k8s.io/controller-runtime/pkg/reconcile\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\tdatapathmockes \"github.com/vmware-tanzu/velero/pkg/datapath/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/exposer\"\n\texposermockes \"github.com/vmware-tanzu/velero/pkg/exposer/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/restorehelper\"\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nfunc TestShouldProcess(t *testing.T) {\n\tcontrollerNode := \"foo\"\n\n\ttests := []struct {\n\t\tname string\n\t\tobj *velerov1api.PodVolumeRestore\n\t\tpod *corev1api.Pod\n\t\tshouldProcessed bool\n\t}{\n\t\t{\n\t\t\tname: \"InProgress phase pvr should not be processed\",\n\t\t\tobj: &velerov1api.PodVolumeRestore{\n\t\t\t\tStatus: velerov1api.PodVolumeRestoreStatus{\n\t\t\t\t\tPhase: velerov1api.PodVolumeRestorePhaseInProgress,\n\t\t\t\t},\n\t\t\t},\n\t\t\tshouldProcessed: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Completed phase pvr should not be processed\",\n\t\t\tobj: &velerov1api.PodVolumeRestore{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\tName: \"pvr-1\",\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.PodVolumeRestoreStatus{\n\t\t\t\t\tPhase: velerov1api.PodVolumeRestorePhaseCompleted,\n\t\t\t\t},\n\t\t\t},\n\t\t\tshouldProcessed: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Failed phase pvr should not be processed\",\n\t\t\tobj: &velerov1api.PodVolumeRestore{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\tName: \"pvr-1\",\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.PodVolumeRestoreStatus{\n\t\t\t\t\tPhase: velerov1api.PodVolumeRestorePhaseFailed,\n\t\t\t\t},\n\t\t\t},\n\t\t\tshouldProcessed: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Unable to get pvr's pod should not be processed\",\n\t\t\tobj: &velerov1api.PodVolumeRestore{\n\t\t\t\tSpec: velerov1api.PodVolumeRestoreSpec{\n\t\t\t\t\tPod: corev1api.ObjectReference{\n\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.PodVolumeRestoreStatus{\n\t\t\t\t\tPhase: \"\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tshouldProcessed: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Empty phase pvr with pod on node not running init container should not be processed\",\n\t\t\tobj: &velerov1api.PodVolumeRestore{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\tName: \"pvr-1\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.PodVolumeRestoreSpec{\n\t\t\t\t\tPod: corev1api.ObjectReference{\n\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.PodVolumeRestoreStatus{\n\t\t\t\t\tPhase: \"\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tNodeName: controllerNode,\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: restorehelper.WaitInitContainer,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tInitContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tState: corev1api.ContainerState{},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tshouldProcessed: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Empty phase pvr with pod on node running init container should be enqueued\",\n\t\t\tobj: &velerov1api.PodVolumeRestore{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\tName: \"pvr-1\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.PodVolumeRestoreSpec{\n\t\t\t\t\tPod: corev1api.ObjectReference{\n\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStatus: velerov1api.PodVolumeRestoreStatus{\n\t\t\t\t\tPhase: \"\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tNodeName: controllerNode,\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: restorehelper.WaitInitContainer,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tInitContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{\n\t\t\t\t\t\t\t\t\tStartedAt: metav1.Time{Time: time.Now()},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tshouldProcessed: true,\n\t\t},\n\t}\n\n\tfor _, ts := range tests {\n\t\tt.Run(ts.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\n\t\t\tvar objs []runtime.Object\n\t\t\tif ts.obj != nil {\n\t\t\t\tobjs = append(objs, ts.obj)\n\t\t\t}\n\t\t\tif ts.pod != nil {\n\t\t\t\tobjs = append(objs, ts.pod)\n\t\t\t}\n\t\t\tcli := test.NewFakeControllerRuntimeClient(t, objs...)\n\n\t\t\tc := &PodVolumeRestoreReconciler{\n\t\t\t\tlogger: logrus.New(),\n\t\t\t\tclient: cli,\n\t\t\t\tclock: &clocks.RealClock{},\n\t\t\t}\n\n\t\t\tshouldProcess, _, _ := shouldProcess(ctx, c.client, c.logger, ts.obj)\n\t\t\trequire.Equal(t, ts.shouldProcessed, shouldProcess)\n\t\t})\n\t}\n}\n\nfunc TestIsInitContainerRunning(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpod *corev1api.Pod\n\t\texpected bool\n\t}{\n\t\t{\n\t\t\tname: \"pod with no init containers should return false\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"pod with running init container that's not restore init should return false\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"non-restore-init\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tInitContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{StartedAt: metav1.Time{Time: time.Now()}},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"pod with running init container that's not first should still work\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"non-restore-init\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: restorehelper.WaitInitContainer,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tInitContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{StartedAt: metav1.Time{Time: time.Now()}},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{StartedAt: metav1.Time{Time: time.Now()}},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"pod with init container as first initContainer that's not running should return false\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: restorehelper.WaitInitContainer,\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"non-restore-init\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tInitContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tState: corev1api.ContainerState{},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{StartedAt: metav1.Time{Time: time.Now()}},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"pod with running init container as first initContainer should return true\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: restorehelper.WaitInitContainer,\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"non-restore-init\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tInitContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{StartedAt: metav1.Time{Time: time.Now()}},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\t\t\t\tRunning: &corev1api.ContainerStateRunning{StartedAt: metav1.Time{Time: time.Now()}},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"pod with init container with empty InitContainerStatuses should return 0\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: restorehelper.WaitInitContainer,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tInitContainerStatuses: []corev1api.ContainerStatus{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tassert.Equal(t, test.expected, isInitContainerRunning(test.pod))\n\t\t})\n\t}\n}\n\nfunc TestGetInitContainerIndex(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpod *corev1api.Pod\n\t\texpected int\n\t}{\n\t\t{\n\t\t\tname: \"init container is not present return -1\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: -1,\n\t\t},\n\t\t{\n\t\t\tname: \"pod with no init container return -1\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"non-restore-init\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: -1,\n\t\t},\n\t\t{\n\t\t\tname: \"pod with container as second initContainern should return 1\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"non-restore-init\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: restorehelper.WaitInitContainer,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"pod with init container as first initContainer should return 0\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: restorehelper.WaitInitContainer,\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"non-restore-init\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"pod with init container as first initContainer should return 0\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: restorehelper.WaitInitContainer,\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"non-restore-init\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: 0,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tassert.Equal(t, test.expected, getInitContainerIndex(test.pod))\n\t\t})\n\t}\n}\n\nfunc TestFindPVRForTargetPod(t *testing.T) {\n\tpod := &corev1api.Pod{}\n\tpod.UID = \"uid\"\n\n\tscheme := runtime.NewScheme()\n\tscheme.AddKnownTypes(velerov1api.SchemeGroupVersion, &velerov1api.PodVolumeRestore{}, &velerov1api.PodVolumeRestoreList{})\n\tclientBuilder := fake.NewClientBuilder().WithScheme(scheme)\n\n\t// no matching PVR\n\treconciler := &PodVolumeRestoreReconciler{\n\t\tclient: clientBuilder.Build(),\n\t\tlogger: logrus.New(),\n\t}\n\trequests := reconciler.findPVRForTargetPod(t.Context(), pod)\n\tassert.Empty(t, requests)\n\n\t// contain one matching PVR\n\treconciler.client = clientBuilder.WithLists(&velerov1api.PodVolumeRestoreList{\n\t\tItems: []velerov1api.PodVolumeRestore{\n\t\t\t{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"pvr1\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\tvelerov1api.PodUIDLabel: string(pod.GetUID()),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"pvr2\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\tvelerov1api.PodUIDLabel: \"non-matching-uid\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}).Build()\n\trequests = reconciler.findPVRForTargetPod(t.Context(), pod)\n\tassert.Len(t, requests, 1)\n}\n\nconst pvrName string = \"pvr-1\"\n\nfunc pvrBuilder() *builder.PodVolumeRestoreBuilder {\n\treturn builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).\n\t\tBackupStorageLocation(\"bsl-loc\").\n\t\tSnapshotID(\"test-snapshot-id\")\n}\n\nfunc initPodVolumeRestoreReconciler(objects []runtime.Object, cliObj []client.Object, needError ...bool) (*PodVolumeRestoreReconciler, error) {\n\tvar errs = make([]error, 6)\n\tfor k, isError := range needError {\n\t\tif k == 0 && isError {\n\t\t\terrs[0] = fmt.Errorf(\"Get error\")\n\t\t} else if k == 1 && isError {\n\t\t\terrs[1] = fmt.Errorf(\"Create error\")\n\t\t} else if k == 2 && isError {\n\t\t\terrs[2] = fmt.Errorf(\"Update error\")\n\t\t} else if k == 3 && isError {\n\t\t\terrs[3] = fmt.Errorf(\"Patch error\")\n\t\t} else if k == 4 && isError {\n\t\t\terrs[4] = apierrors.NewConflict(velerov1api.Resource(\"podvolumerestore\"), pvrName, errors.New(\"conflict\"))\n\t\t} else if k == 5 && isError {\n\t\t\terrs[5] = fmt.Errorf(\"List error\")\n\t\t}\n\t}\n\treturn initPodVolumeRestoreReconcilerWithError(objects, cliObj, errs...)\n}\n\nfunc initPodVolumeRestoreReconcilerWithError(objects []runtime.Object, cliObj []client.Object, needError ...error) (*PodVolumeRestoreReconciler, error) {\n\tscheme := runtime.NewScheme()\n\terr := velerov1api.AddToScheme(scheme)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\terr = corev1api.AddToScheme(scheme)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfakeClient := &FakeClient{\n\t\tClient: fake.NewClientBuilder().WithScheme(scheme).WithObjects(cliObj...).Build(),\n\t}\n\n\tfor k := range needError {\n\t\tif k == 0 {\n\t\t\tfakeClient.getError = needError[0]\n\t\t} else if k == 1 {\n\t\t\tfakeClient.createError = needError[1]\n\t\t} else if k == 2 {\n\t\t\tfakeClient.updateError = needError[2]\n\t\t} else if k == 3 {\n\t\t\tfakeClient.patchError = needError[3]\n\t\t} else if k == 4 {\n\t\t\tfakeClient.updateConflict = needError[4]\n\t\t} else if k == 5 {\n\t\t\tfakeClient.listError = needError[5]\n\t\t}\n\t}\n\n\tvar fakeKubeClient *clientgofake.Clientset\n\tif len(objects) != 0 {\n\t\tfakeKubeClient = clientgofake.NewSimpleClientset(objects...)\n\t} else {\n\t\tfakeKubeClient = clientgofake.NewSimpleClientset()\n\t}\n\n\tfakeFS := velerotest.NewFakeFileSystem()\n\tpathGlob := fmt.Sprintf(\"/host_pods/%s/volumes/*/%s\", \"test-uid\", \"test-pvc\")\n\t_, err = fakeFS.Create(pathGlob)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdataPathMgr := datapath.NewManager(1)\n\n\treturn NewPodVolumeRestoreReconciler(\n\t\tfakeClient,\n\t\tnil,\n\t\tfakeKubeClient,\n\t\tdataPathMgr,\n\t\tnil,\n\t\t\"test-node\",\n\t\ttime.Minute*5,\n\t\ttime.Minute,\n\t\tnil,\n\t\tnil,\n\t\tcorev1api.ResourceRequirements{},\n\t\tvelerotest.NewLogger(),\n\t\t\"\",\n\t\tfalse,\n\t\tnil,\n\t\tnil, // podLabels\n\t\tnil, // podAnnotations\n\t), nil\n}\n\nfunc TestPodVolumeRestoreReconcile(t *testing.T) {\n\tdaemonSet := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"velero\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t\tAPIVersion: appsv1api.SchemeGroupVersion.String(),\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tImage: \"fake-image\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tnode := builder.ForNode(\"fake-node\").Labels(map[string]string{kube.NodeOSLabel: kube.NodeOSLinux}).Result()\n\n\ttests := []struct {\n\t\tname string\n\t\tpvr *velerov1api.PodVolumeRestore\n\t\tnotCreatePVR bool\n\t\ttargetPod *corev1api.Pod\n\t\tdataMgr *datapath.Manager\n\t\tneedErrs []bool\n\t\tneedCreateFSBR bool\n\t\tneedDelete bool\n\t\tsportTime *metav1.Time\n\t\tmockExposeErr *bool\n\t\tisGetExposeErr bool\n\t\tisGetExposeNil bool\n\t\tisPeekExposeErr bool\n\t\tisNilExposer bool\n\t\tnotNilExpose bool\n\t\tnotMockCleanUp bool\n\t\tmockInit bool\n\t\tmockInitErr error\n\t\tmockStart bool\n\t\tmockStartErr error\n\t\tmockCancel bool\n\t\tmockClose bool\n\t\tneedExclusiveUpdateError error\n\t\tconstrained bool\n\t\texpected *velerov1api.PodVolumeRestore\n\t\texpectDeleted bool\n\t\texpectCancelRecord bool\n\t\texpectedResult *ctrl.Result\n\t\texpectedErr string\n\t\texpectDataPath bool\n\t}{\n\t\t{\n\t\t\tname: \"pvr not found\",\n\t\t\tpvr: pvrBuilder().Result(),\n\t\t\tnotCreatePVR: true,\n\t\t},\n\t\t{\n\t\t\tname: \"pvr not created in velero default namespace\",\n\t\t\tpvr: builder.ForPodVolumeRestore(\"test-ns\", pvrName).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"get dd fail\",\n\t\t\tpvr: builder.ForPodVolumeRestore(\"test-ns\", pvrName).Result(),\n\t\t\tneedErrs: []bool{true, false, false, false},\n\t\t\texpectedErr: \"Get error\",\n\t\t},\n\t\t{\n\t\t\tname: \"add finalizer to pvr\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Result(),\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"add finalizer to pvr failed\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\texpectedErr: \"error updating PVR velero/pvr-1: Update error\",\n\t\t},\n\t\t{\n\t\t\tname: \"pvr is under deletion\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t\tneedDelete: true,\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"pvr is under deletion but cancel failed\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\tneedDelete: true,\n\t\t\texpectedErr: \"error updating PVR velero/pvr-1: Update error\",\n\t\t},\n\t\t{\n\t\t\tname: \"pvr is under deletion and in terminal state\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).Phase(velerov1api.PodVolumeRestorePhaseFailed).Result(),\n\t\t\tsportTime: &metav1.Time{Time: time.Now()},\n\t\t\tneedDelete: true,\n\t\t\texpectDeleted: true,\n\t\t},\n\t\t{\n\t\t\tname: \"pvr is under deletion and in terminal state, but remove finalizer failed\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).Phase(velerov1api.PodVolumeRestorePhaseFailed).Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\tneedDelete: true,\n\t\t\texpectedErr: \"error updating PVR velero/pvr-1: Update error\",\n\t\t},\n\t\t{\n\t\t\tname: \"delay cancel negative for others\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Phase(velerov1api.PodVolumeRestorePhasePrepared).Result(),\n\t\t\tsportTime: &metav1.Time{Time: time.Now()},\n\t\t\texpectCancelRecord: true,\n\t\t},\n\t\t{\n\t\t\tname: \"delay cancel negative for inProgress\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Phase(velerov1api.PodVolumeRestorePhaseInProgress).Result(),\n\t\t\tsportTime: &metav1.Time{Time: time.Now().Add(-time.Minute * 58)},\n\t\t\texpectCancelRecord: true,\n\t\t},\n\t\t{\n\t\t\tname: \"delay cancel affirmative for others\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Phase(velerov1api.PodVolumeRestorePhasePrepared).Result(),\n\t\t\tsportTime: &metav1.Time{Time: time.Now().Add(-time.Minute * 5)},\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Phase(velerov1api.PodVolumeRestorePhaseCanceled).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"delay cancel affirmative for inProgress\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Phase(velerov1api.PodVolumeRestorePhaseInProgress).Result(),\n\t\t\tsportTime: &metav1.Time{Time: time.Now().Add(-time.Hour)},\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Phase(velerov1api.PodVolumeRestorePhaseCanceled).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"delay cancel failed\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Phase(velerov1api.PodVolumeRestorePhaseInProgress).Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\tsportTime: &metav1.Time{Time: time.Now().Add(-time.Hour)},\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Phase(velerov1api.PodVolumeRestorePhaseInProgress).Result(),\n\t\t\texpectCancelRecord: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Unknown pvr status\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(\"Unknown\").Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"new pvb but constrained\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).PodNamespace(\"test-ns\").PodName(\"test-pod\").Result(),\n\t\t\ttargetPod: builder.ForPod(\"test-ns\", \"test-pod\").InitContainers(&corev1api.Container{Name: restorehelper.WaitInitContainer}).InitContainerState(corev1api.ContainerState{Running: &corev1api.ContainerStateRunning{}}).Result(),\n\t\t\tconstrained: true,\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t\texpectedResult: &ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5},\n\t\t},\n\t\t{\n\t\t\tname: \"new pvr but accept failed\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).PodNamespace(\"test-ns\").PodName(\"test-pod\").Result(),\n\t\t\ttargetPod: builder.ForPod(\"test-ns\", \"test-pod\").InitContainers(&corev1api.Container{Name: restorehelper.WaitInitContainer}).InitContainerState(corev1api.ContainerState{Running: &corev1api.ContainerStateRunning{}}).Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t\texpectedErr: \"error accepting PVR pvr-1: error updating PVR velero/pvr-1: Update error\",\n\t\t},\n\t\t{\n\t\t\tname: \"pvr is cancel on accepted\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Result(),\n\t\t\texpectCancelRecord: true,\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Phase(velerov1api.PodVolumeRestorePhaseCanceled).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"pvr expose failed\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).PodNamespace(\"test-ns\").PodName(\"test-pod\").Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t\ttargetPod: builder.ForPod(\"test-ns\", \"test-pod\").InitContainers(&corev1api.Container{Name: restorehelper.WaitInitContainer}).InitContainerState(corev1api.ContainerState{Running: &corev1api.ContainerStateRunning{}}).Result(),\n\t\t\tmockExposeErr: boolptr.True(),\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).Phase(velerov1api.PodVolumeRestorePhaseFailed).Message(\"error to expose PVR\").Result(),\n\t\t\texpectedErr: \"Error to expose restore exposer\",\n\t\t},\n\t\t{\n\t\t\tname: \"pvr succeeds for accepted\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).PodNamespace(\"test-ns\").PodName(\"test-pod\").Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t\tmockExposeErr: boolptr.False(),\n\t\t\tnotMockCleanUp: true,\n\t\t\ttargetPod: builder.ForPod(\"test-ns\", \"test-pod\").InitContainers(&corev1api.Container{Name: restorehelper.WaitInitContainer}).InitContainerState(corev1api.ContainerState{Running: &corev1api.ContainerStateRunning{}}).Result(),\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Finalizers([]string{PodVolumeFinalizer}).Phase(velerov1api.PodVolumeRestorePhaseAccepted).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"prepare timeout on accepted\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseAccepted).Finalizers([]string{PodVolumeFinalizer}).AcceptedTimestamp(&metav1.Time{Time: time.Now().Add(-time.Minute * 30)}).Result(),\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseFailed).Finalizers([]string{PodVolumeFinalizer}).Phase(velerov1api.PodVolumeRestorePhaseFailed).Message(\"timeout on preparing PVR\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"peek error on accepted\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseAccepted).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t\tisPeekExposeErr: true,\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseCanceled).Finalizers([]string{PodVolumeFinalizer}).Phase(velerov1api.PodVolumeRestorePhaseCanceled).Message(\"found a PVR velero/pvr-1 with expose error: fake-peek-error. mark it as cancel\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"cancel on pvr\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhasePrepared).Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Node(\"test-node\").Result(),\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseCanceled).Finalizers([]string{PodVolumeFinalizer}).Cancel(true).Phase(velerov1api.PodVolumeRestorePhaseCanceled).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Failed to get restore expose on prepared\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhasePrepared).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tisGetExposeErr: true,\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseFailed).Finalizers([]string{PodVolumeFinalizer}).Message(\"exposed PVR is not ready\").Result(),\n\t\t\texpectedErr: \"Error to get PVR exposer\",\n\t\t},\n\t\t{\n\t\t\tname: \"Get nil restore expose on prepared\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhasePrepared).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tisGetExposeNil: true,\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseFailed).Finalizers([]string{PodVolumeFinalizer}).Message(\"exposed PVR is not ready\").Result(),\n\t\t\texpectedErr: \"no expose result is available for the current node\",\n\t\t},\n\t\t{\n\t\t\tname: \"Error in data path is concurrent limited\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhasePrepared).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tdataMgr: datapath.NewManager(0),\n\t\t\tnotNilExpose: true,\n\t\t\tnotMockCleanUp: true,\n\t\t\texpectedResult: &ctrl.Result{Requeue: true, RequeueAfter: time.Second * 5},\n\t\t},\n\t\t{\n\t\t\tname: \"data path init error\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhasePrepared).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tmockInit: true,\n\t\t\tmockInitErr: errors.New(\"fake-data-path-init-error\"),\n\t\t\tmockClose: true,\n\t\t\tnotNilExpose: true,\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseFailed).Finalizers([]string{PodVolumeFinalizer}).Message(\"error initializing data path\").Result(),\n\t\t\texpectedErr: \"error initializing asyncBR: fake-data-path-init-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"Unable to update status to in progress for pvr\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhasePrepared).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\tmockInit: true,\n\t\t\tmockClose: true,\n\t\t\tnotNilExpose: true,\n\t\t\tnotMockCleanUp: true,\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhasePrepared).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"data path start error\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhasePrepared).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tmockInit: true,\n\t\t\tmockStart: true,\n\t\t\tmockStartErr: errors.New(\"fake-data-path-start-error\"),\n\t\t\tmockClose: true,\n\t\t\tnotNilExpose: true,\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseFailed).Finalizers([]string{PodVolumeFinalizer}).Message(\"error starting data path\").Result(),\n\t\t\texpectedErr: \"error starting async restore for pod test-name, volume test-pvc: fake-data-path-start-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"Prepare succeeds\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhasePrepared).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tmockInit: true,\n\t\t\tmockStart: true,\n\t\t\tnotNilExpose: true,\n\t\t\tnotMockCleanUp: true,\n\t\t\texpectDataPath: true,\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseInProgress).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"In progress pvr is not handled by the current node\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseInProgress).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseInProgress).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"In progress pvr is not set as cancel\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseInProgress).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseInProgress).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Cancel pvr in progress with empty FSBR\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseInProgress).Cancel(true).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseCanceled).Cancel(true).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Cancel pvr in progress and patch pvr error\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseInProgress).Cancel(true).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t\tneedCreateFSBR: true,\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseInProgress).Cancel(true).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t\texpectedErr: \"error updating PVR velero/pvr-1: Update error\",\n\t\t\texpectCancelRecord: true,\n\t\t\texpectDataPath: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Cancel pvr in progress succeeds\",\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseInProgress).Cancel(true).Finalizers([]string{PodVolumeFinalizer}).Node(\"test-node\").Result(),\n\t\t\tneedCreateFSBR: true,\n\t\t\tmockCancel: true,\n\t\t\texpected: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseCanceling).Cancel(true).Finalizers([]string{PodVolumeFinalizer}).Result(),\n\t\t\texpectDataPath: true,\n\t\t\texpectCancelRecord: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tobjs := []runtime.Object{daemonSet, node}\n\n\t\t\tctlObj := []client.Object{}\n\t\t\tif test.targetPod != nil {\n\t\t\t\tctlObj = append(ctlObj, test.targetPod)\n\t\t\t}\n\n\t\t\tr, err := initPodVolumeRestoreReconciler(objs, ctlObj, test.needErrs...)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tif !test.notCreatePVR {\n\t\t\t\terr = r.client.Create(t.Context(), test.pvr)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif test.needDelete {\n\t\t\t\terr = r.client.Delete(t.Context(), test.pvr)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif test.dataMgr != nil {\n\t\t\t\tr.dataPathMgr = test.dataMgr\n\t\t\t} else {\n\t\t\t\tr.dataPathMgr = datapath.NewManager(1)\n\t\t\t}\n\n\t\t\tif test.sportTime != nil {\n\t\t\t\tr.cancelledPVR[test.pvr.Name] = test.sportTime.Time\n\t\t\t}\n\n\t\t\tif test.constrained {\n\t\t\t\tr.vgdpCounter = &exposer.VgdpCounter{}\n\t\t\t}\n\n\t\t\tfuncExclusiveUpdatePodVolumeRestore = exclusiveUpdatePodVolumeRestore\n\t\t\tif test.needExclusiveUpdateError != nil {\n\t\t\t\tfuncExclusiveUpdatePodVolumeRestore = func(context.Context, kbclient.Client, *velerov1api.PodVolumeRestore, func(*velerov1api.PodVolumeRestore)) (bool, error) {\n\t\t\t\t\treturn false, test.needExclusiveUpdateError\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tdatapath.MicroServiceBRWatcherCreator = func(kbclient.Client, kubernetes.Interface, manager.Manager, string, string,\n\t\t\t\tstring, string, string, string, datapath.Callbacks, logrus.FieldLogger) datapath.AsyncBR {\n\t\t\t\tasyncBR := datapathmockes.NewAsyncBR(t)\n\t\t\t\tif test.mockInit {\n\t\t\t\t\tasyncBR.On(\"Init\", mock.Anything, mock.Anything).Return(test.mockInitErr)\n\t\t\t\t}\n\n\t\t\t\tif test.mockStart {\n\t\t\t\t\tasyncBR.On(\"StartRestore\", mock.Anything, mock.Anything, mock.Anything).Return(test.mockStartErr)\n\t\t\t\t}\n\n\t\t\t\tif test.mockCancel {\n\t\t\t\t\tasyncBR.On(\"Cancel\").Return()\n\t\t\t\t}\n\n\t\t\t\tif test.mockClose {\n\t\t\t\t\tasyncBR.On(\"Close\", mock.Anything).Return()\n\t\t\t\t}\n\n\t\t\t\treturn asyncBR\n\t\t\t}\n\n\t\t\tif test.mockExposeErr != nil || test.isGetExposeErr || test.isGetExposeNil || test.isPeekExposeErr || test.isNilExposer || test.notNilExpose {\n\t\t\t\tif test.isNilExposer {\n\t\t\t\t\tr.exposer = nil\n\t\t\t\t} else {\n\t\t\t\t\tr.exposer = func() exposer.PodVolumeExposer {\n\t\t\t\t\t\tep := exposermockes.NewMockPodVolumeExposer(t)\n\t\t\t\t\t\tif test.mockExposeErr != nil {\n\t\t\t\t\t\t\tif boolptr.IsSetToTrue(test.mockExposeErr) {\n\t\t\t\t\t\t\t\tep.On(\"Expose\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(errors.New(\"Error to expose restore exposer\"))\n\t\t\t\t\t\t\t} else {\n\t\t\t\t\t\t\t\tep.On(\"Expose\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(nil)\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t} else if test.notNilExpose {\n\t\t\t\t\t\t\thostingPod := builder.ForPod(\"test-ns\", \"test-name\").Volumes(&corev1api.Volume{Name: \"test-pvc\"}).Result()\n\t\t\t\t\t\t\thostingPod.ObjectMeta.SetUID(\"test-uid\")\n\t\t\t\t\t\t\tep.On(\"GetExposed\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(&exposer.ExposeResult{ByPod: exposer.ExposeByPod{HostingPod: hostingPod, VolumeName: \"test-pvc\"}}, nil)\n\t\t\t\t\t\t} else if test.isGetExposeErr {\n\t\t\t\t\t\t\tep.On(\"GetExposed\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(nil, errors.New(\"Error to get PVR exposer\"))\n\t\t\t\t\t\t} else if test.isGetExposeNil {\n\t\t\t\t\t\t\tep.On(\"GetExposed\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(nil, nil)\n\t\t\t\t\t\t} else if test.isPeekExposeErr {\n\t\t\t\t\t\t\tep.On(\"PeekExposed\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(errors.New(\"fake-peek-error\"))\n\t\t\t\t\t\t\tep.On(\"DiagnoseExpose\", mock.Anything, mock.Anything).Return(\"\")\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\tif !test.notMockCleanUp {\n\t\t\t\t\t\t\tep.On(\"CleanUp\", mock.Anything, mock.Anything).Return()\n\t\t\t\t\t\t}\n\t\t\t\t\t\treturn ep\n\t\t\t\t\t}()\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif test.needCreateFSBR {\n\t\t\t\tif fsBR := r.dataPathMgr.GetAsyncBR(test.pvr.Name); fsBR == nil {\n\t\t\t\t\t_, err := r.dataPathMgr.CreateMicroServiceBRWatcher(ctx, r.client, nil, nil, datapath.TaskTypeRestore, test.pvr.Name, pVBRRequestor,\n\t\t\t\t\t\tvelerov1api.DefaultNamespace, \"\", \"\", datapath.Callbacks{OnCancelled: r.OnDataPathCancelled}, false, velerotest.NewLogger())\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tactualResult, err := r.Reconcile(ctx, ctrl.Request{\n\t\t\t\tNamespacedName: types.NamespacedName{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: test.pvr.Name,\n\t\t\t\t},\n\t\t\t})\n\n\t\t\tif test.expectedErr != \"\" {\n\t\t\t\trequire.EqualError(t, err, test.expectedErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif test.expectedResult != nil {\n\t\t\t\tassert.Equal(t, test.expectedResult.Requeue, actualResult.Requeue)\n\t\t\t\tassert.Equal(t, test.expectedResult.RequeueAfter, actualResult.RequeueAfter)\n\t\t\t}\n\n\t\t\tpvr := velerov1api.PodVolumeRestore{}\n\t\t\terr = r.client.Get(ctx, kbclient.ObjectKey{\n\t\t\t\tName: test.pvr.Name,\n\t\t\t\tNamespace: test.pvr.Namespace,\n\t\t\t}, &pvr)\n\n\t\t\tif test.expected != nil || test.expectDeleted {\n\t\t\t\tif test.expectDeleted {\n\t\t\t\t\tassert.True(t, apierrors.IsNotFound(err))\n\t\t\t\t} else {\n\t\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\t\tassert.Equal(t, test.expected.Status.Phase, pvr.Status.Phase)\n\t\t\t\t\tassert.Contains(t, pvr.Status.Message, test.expected.Status.Message)\n\t\t\t\t\tassert.Equal(t, test.expected.Finalizers, pvr.Finalizers)\n\t\t\t\t\tassert.Equal(t, test.expected.Spec.Cancel, pvr.Spec.Cancel)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif !test.expectDataPath {\n\t\t\t\tassert.Nil(t, r.dataPathMgr.GetAsyncBR(test.pvr.Name))\n\t\t\t} else {\n\t\t\t\tassert.NotNil(t, r.dataPathMgr.GetAsyncBR(test.pvr.Name))\n\t\t\t}\n\n\t\t\tif test.expectCancelRecord {\n\t\t\t\tassert.Contains(t, r.cancelledPVR, test.pvr.Name)\n\t\t\t} else {\n\t\t\t\tassert.Empty(t, r.cancelledPVR)\n\t\t\t}\n\n\t\t\tif isPVRInFinalState(&pvr) || pvr.Status.Phase == velerov1api.PodVolumeRestorePhaseInProgress {\n\t\t\t\tassert.NotContains(t, pvr.Labels, exposer.ExposeOnGoingLabel)\n\t\t\t} else if pvr.Status.Phase == velerov1api.PodVolumeRestorePhaseAccepted {\n\t\t\t\tassert.Contains(t, pvr.Labels, exposer.ExposeOnGoingLabel)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestPodVolumeRestoreSetupExposeParam(t *testing.T) {\n\t// common objects for all cases\n\tnode := builder.ForNode(\"worker-1\").Labels(map[string]string{kube.NodeOSLabel: kube.NodeOSLinux}).Result()\n\n\tbasePVR := pvrBuilder().Result()\n\tbasePVR.Status.Node = \"worker-1\"\n\tbasePVR.Spec.Pod.Namespace = \"app-ns\"\n\tbasePVR.Spec.Pod.Name = \"app-pod\"\n\tbasePVR.Spec.Volume = \"data-vol\"\n\n\ttype args struct {\n\t\tcustomLabels map[string]string\n\t\tcustomAnnotations map[string]string\n\t}\n\ttype want struct {\n\t\tlabels map[string]string\n\t\tannotations map[string]string\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\targs args\n\t\twant want\n\t}{\n\t\t{\n\t\t\tname: \"label has customize values\",\n\t\t\targs: args{\n\t\t\t\tcustomLabels: map[string]string{\"custom-label\": \"label-value\"},\n\t\t\t\tcustomAnnotations: nil,\n\t\t\t},\n\t\t\twant: want{\n\t\t\t\tlabels: map[string]string{\n\t\t\t\t\tvelerov1api.PVRLabel: basePVR.Name,\n\t\t\t\t\t\"custom-label\": \"label-value\",\n\t\t\t\t},\n\t\t\t\tannotations: map[string]string{},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"label has no customize values\",\n\t\t\targs: args{\n\t\t\t\tcustomLabels: nil,\n\t\t\t\tcustomAnnotations: nil,\n\t\t\t},\n\t\t\twant: want{\n\t\t\t\tlabels: map[string]string{velerov1api.PVRLabel: basePVR.Name},\n\t\t\t\tannotations: map[string]string{},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"annotation has customize values\",\n\t\t\targs: args{\n\t\t\t\tcustomLabels: nil,\n\t\t\t\tcustomAnnotations: map[string]string{\"custom-annotation\": \"annotation-value\"},\n\t\t\t},\n\t\t\twant: want{\n\t\t\t\tlabels: map[string]string{velerov1api.PVRLabel: basePVR.Name},\n\t\t\t\tannotations: map[string]string{\"custom-annotation\": \"annotation-value\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"annotation has no customize values\",\n\t\t\targs: args{\n\t\t\t\tcustomLabels: map[string]string{\"another-label\": \"lval\"},\n\t\t\t\tcustomAnnotations: nil,\n\t\t\t},\n\t\t\twant: want{\n\t\t\t\tlabels: map[string]string{\n\t\t\t\t\tvelerov1api.PVRLabel: basePVR.Name,\n\t\t\t\t\t\"another-label\": \"lval\",\n\t\t\t\t},\n\t\t\t\tannotations: map[string]string{},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\t// Fake clients per case\n\t\t\tfakeCRClient := velerotest.NewFakeControllerRuntimeClient(t, node, basePVR.DeepCopy())\n\t\t\tfakeKubeClient := clientgofake.NewSimpleClientset(node)\n\n\t\t\t// Reconciler config per case\n\t\t\tpreparingTimeout := time.Minute * 3\n\t\t\tresourceTimeout := time.Minute * 10\n\t\t\tpodRes := corev1api.ResourceRequirements{}\n\t\t\tr := NewPodVolumeRestoreReconciler(\n\t\t\t\tfakeCRClient,\n\t\t\t\tnil,\n\t\t\t\tfakeKubeClient,\n\t\t\t\tdatapath.NewManager(1),\n\t\t\t\tnil,\n\t\t\t\t\"test-node\",\n\t\t\t\tpreparingTimeout,\n\t\t\t\tresourceTimeout,\n\t\t\t\tnil, // backupRepoConfigs\n\t\t\t\tnil, // cacheVolumeConfigs -> keep nil so CacheVolume is nil\n\t\t\t\tpodRes,\n\t\t\t\tvelerotest.NewLogger(),\n\t\t\t\t\"restore-priority\",\n\t\t\t\ttrue,\n\t\t\t\tnil, // repoConfigMgr (unused when cacheVolumeConfigs is nil)\n\t\t\t\ttt.args.customLabels,\n\t\t\t\ttt.args.customAnnotations,\n\t\t\t)\n\n\t\t\t// Act\n\t\t\tgot := r.setupExposeParam(basePVR)\n\n\t\t\t// Core fields\n\t\t\tassert.Equal(t, exposer.PodVolumeExposeTypeRestore, got.Type)\n\t\t\tassert.Equal(t, basePVR.Spec.Pod.Namespace, got.ClientNamespace)\n\t\t\tassert.Equal(t, basePVR.Spec.Pod.Name, got.ClientPodName)\n\t\t\tassert.Equal(t, basePVR.Spec.Volume, got.ClientPodVolume)\n\n\t\t\t// Labels/Annotations\n\t\t\tassert.Equal(t, tt.want.labels, got.HostingPodLabels)\n\t\t\tassert.Equal(t, tt.want.annotations, got.HostingPodAnnotations)\n\t\t})\n\t}\n}\n\nfunc TestOnPodVolumeRestoreFailed(t *testing.T) {\n\tfor _, getErr := range []bool{true, false} {\n\t\tctx := t.Context()\n\t\tneedErrs := []bool{getErr, false, false, false}\n\t\tr, err := initPodVolumeRestoreReconciler(nil, []client.Object{}, needErrs...)\n\t\trequire.NoError(t, err)\n\n\t\tpvr := pvrBuilder().Result()\n\t\tnamespace := pvr.Namespace\n\t\tpvrName := pvr.Name\n\n\t\trequire.NoError(t, r.client.Create(ctx, pvr))\n\t\tr.OnDataPathFailed(ctx, namespace, pvrName, fmt.Errorf(\"Failed to handle %v\", pvrName))\n\t\tupdatedPVR := &velerov1api.PodVolumeRestore{}\n\t\tif getErr {\n\t\t\trequire.Error(t, r.client.Get(ctx, types.NamespacedName{Name: pvrName, Namespace: namespace}, updatedPVR))\n\t\t\tassert.NotEqual(t, velerov1api.PodVolumeRestorePhaseFailed, updatedPVR.Status.Phase)\n\t\t\tassert.True(t, updatedPVR.Status.StartTimestamp.IsZero())\n\t\t} else {\n\t\t\trequire.NoError(t, r.client.Get(ctx, types.NamespacedName{Name: pvrName, Namespace: namespace}, updatedPVR))\n\t\t\tassert.Equal(t, velerov1api.PodVolumeRestorePhaseFailed, updatedPVR.Status.Phase)\n\t\t\tassert.True(t, updatedPVR.Status.StartTimestamp.IsZero())\n\t\t}\n\t}\n}\n\nfunc TestOnPodVolumeRestoreCancelled(t *testing.T) {\n\tfor _, getErr := range []bool{true, false} {\n\t\tctx := t.Context()\n\t\tneedErrs := []bool{getErr, false, false, false}\n\t\tr, err := initPodVolumeRestoreReconciler(nil, nil, needErrs...)\n\t\trequire.NoError(t, err)\n\n\t\tpvr := pvrBuilder().Result()\n\t\tnamespace := pvr.Namespace\n\t\tpvrName := pvr.Name\n\n\t\trequire.NoError(t, r.client.Create(ctx, pvr))\n\t\tr.OnDataPathCancelled(ctx, namespace, pvrName)\n\t\tupdatedPVR := &velerov1api.PodVolumeRestore{}\n\t\tif getErr {\n\t\t\trequire.Error(t, r.client.Get(ctx, types.NamespacedName{Name: pvrName, Namespace: namespace}, updatedPVR))\n\t\t\tassert.NotEqual(t, velerov1api.PodVolumeRestorePhaseFailed, updatedPVR.Status.Phase)\n\t\t\tassert.True(t, updatedPVR.Status.StartTimestamp.IsZero())\n\t\t} else {\n\t\t\trequire.NoError(t, r.client.Get(ctx, types.NamespacedName{Name: pvrName, Namespace: namespace}, updatedPVR))\n\t\t\tassert.Equal(t, velerov1api.PodVolumeRestorePhaseCanceled, updatedPVR.Status.Phase)\n\t\t\tassert.False(t, updatedPVR.Status.StartTimestamp.IsZero())\n\t\t\tassert.False(t, updatedPVR.Status.CompletionTimestamp.IsZero())\n\t\t}\n\t}\n}\n\nfunc TestOnPodVolumeRestoreCompleted(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\temptyFSBR bool\n\t\tisGetErr bool\n\t\trebindVolumeErr bool\n\t}{\n\t\t{\n\t\t\tname: \"PVR complete\",\n\t\t\temptyFSBR: false,\n\t\t\tisGetErr: false,\n\t\t\trebindVolumeErr: false,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\t\t\tneedErrs := []bool{test.isGetErr, false, false, false}\n\t\t\tr, err := initPodVolumeRestoreReconciler(nil, []client.Object{}, needErrs...)\n\t\t\tr.exposer = func() exposer.PodVolumeExposer {\n\t\t\t\tep := exposermockes.NewMockPodVolumeExposer(t)\n\t\t\t\tep.On(\"CleanUp\", mock.Anything, mock.Anything).Return()\n\t\t\t\treturn ep\n\t\t\t}()\n\n\t\t\trequire.NoError(t, err)\n\t\t\tpvr := builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Result()\n\t\t\tnamespace := pvr.Namespace\n\t\t\tddName := pvr.Name\n\n\t\t\trequire.NoError(t, r.client.Create(ctx, pvr))\n\t\t\tr.OnDataPathCompleted(ctx, namespace, ddName, datapath.Result{})\n\t\t\tupdatedDD := &velerov1api.PodVolumeRestore{}\n\t\t\tif test.isGetErr {\n\t\t\t\trequire.Error(t, r.client.Get(ctx, types.NamespacedName{Name: ddName, Namespace: namespace}, updatedDD))\n\t\t\t\tassert.Equal(t, velerov1api.PodVolumeRestorePhase(\"\"), updatedDD.Status.Phase)\n\t\t\t\tassert.True(t, updatedDD.Status.CompletionTimestamp.IsZero())\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, r.client.Get(ctx, types.NamespacedName{Name: ddName, Namespace: namespace}, updatedDD))\n\t\t\t\tassert.Equal(t, velerov1api.PodVolumeRestorePhaseCompleted, updatedDD.Status.Phase)\n\t\t\t\tassert.False(t, updatedDD.Status.CompletionTimestamp.IsZero())\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestOnPodVolumeRestoreProgress(t *testing.T) {\n\ttotalBytes := int64(1024)\n\tbytesDone := int64(512)\n\ttests := []struct {\n\t\tname string\n\t\tpvr *velerov1api.PodVolumeRestore\n\t\tprogress uploader.Progress\n\t\tneedErrs []bool\n\t}{\n\t\t{\n\t\t\tname: \"patch in progress phase success\",\n\t\t\tpvr: pvrBuilder().Result(),\n\t\t\tprogress: uploader.Progress{\n\t\t\t\tTotalBytes: totalBytes,\n\t\t\t\tBytesDone: bytesDone,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"failed to get pvr\",\n\t\t\tpvr: pvrBuilder().Result(),\n\t\t\tneedErrs: []bool{true, false, false, false},\n\t\t},\n\t\t{\n\t\t\tname: \"failed to patch pvr\",\n\t\t\tpvr: pvrBuilder().Result(),\n\t\t\tneedErrs: []bool{false, false, true, false},\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\n\t\t\tr, err := initPodVolumeRestoreReconciler(nil, []client.Object{}, test.needErrs...)\n\t\t\trequire.NoError(t, err)\n\t\t\tdefer func() {\n\t\t\t\tr.client.Delete(ctx, test.pvr, &kbclient.DeleteOptions{})\n\t\t\t}()\n\n\t\t\tpvr := pvrBuilder().Result()\n\t\t\tnamespace := pvr.Namespace\n\t\t\tpvrName := pvr.Name\n\n\t\t\trequire.NoError(t, r.client.Create(t.Context(), pvr))\n\n\t\t\t// Create a Progress object\n\t\t\tprogress := &uploader.Progress{\n\t\t\t\tTotalBytes: totalBytes,\n\t\t\t\tBytesDone: bytesDone,\n\t\t\t}\n\n\t\t\tr.OnDataPathProgress(ctx, namespace, pvrName, progress)\n\t\t\tif len(test.needErrs) != 0 && !test.needErrs[0] {\n\t\t\t\tupdatedPVR := &velerov1api.PodVolumeRestore{}\n\t\t\t\trequire.NoError(t, r.client.Get(ctx, types.NamespacedName{Name: pvrName, Namespace: namespace}, updatedPVR))\n\t\t\t\tassert.Equal(t, test.progress.TotalBytes, updatedPVR.Status.Progress.TotalBytes)\n\t\t\t\tassert.Equal(t, test.progress.BytesDone, updatedPVR.Status.Progress.BytesDone)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestFindPVBForRestorePod(t *testing.T) {\n\tneedErrs := []bool{false, false, false, false}\n\tr, err := initPodVolumeRestoreReconciler(nil, []client.Object{}, needErrs...)\n\trequire.NoError(t, err)\n\ttests := []struct {\n\t\tname string\n\t\tpvr *velerov1api.PodVolumeRestore\n\t\tpod *corev1api.Pod\n\t\tcheckFunc func(*velerov1api.PodVolumeRestore, []reconcile.Request)\n\t}{\n\t\t{\n\t\t\tname: \"find pvr for pod\",\n\t\t\tpvr: pvrBuilder().Phase(velerov1api.PodVolumeRestorePhaseAccepted).Result(),\n\t\t\tpod: builder.ForPod(velerov1api.DefaultNamespace, pvrName).Labels(map[string]string{velerov1api.PVRLabel: pvrName}).Status(corev1api.PodStatus{Phase: corev1api.PodRunning}).Result(),\n\t\t\tcheckFunc: func(pvr *velerov1api.PodVolumeRestore, requests []reconcile.Request) {\n\t\t\t\t// Assert that the function returns a single request\n\t\t\t\tassert.Len(t, requests, 1)\n\t\t\t\t// Assert that the request contains the correct namespaced name\n\t\t\t\tassert.Equal(t, pvr.Namespace, requests[0].Namespace)\n\t\t\t\tassert.Equal(t, pvr.Name, requests[0].Name)\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"no selected label found for pod\",\n\t\t\tpvr: pvrBuilder().Phase(velerov1api.PodVolumeRestorePhaseAccepted).Result(),\n\t\t\tpod: builder.ForPod(velerov1api.DefaultNamespace, pvrName).Result(),\n\t\t\tcheckFunc: func(pvr *velerov1api.PodVolumeRestore, requests []reconcile.Request) {\n\t\t\t\t// Assert that the function returns a single request\n\t\t\t\tassert.Empty(t, requests)\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"no matched pod\",\n\t\t\tpvr: pvrBuilder().Phase(velerov1api.PodVolumeRestorePhaseAccepted).Result(),\n\t\t\tpod: builder.ForPod(velerov1api.DefaultNamespace, pvrName).Labels(map[string]string{velerov1api.PVRLabel: \"non-existing-pvr\"}).Result(),\n\t\t\tcheckFunc: func(pvr *velerov1api.PodVolumeRestore, requests []reconcile.Request) {\n\t\t\t\tassert.Empty(t, requests)\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"pvr not accept\",\n\t\t\tpvr: pvrBuilder().Phase(velerov1api.PodVolumeRestorePhaseInProgress).Result(),\n\t\t\tpod: builder.ForPod(velerov1api.DefaultNamespace, pvrName).Labels(map[string]string{velerov1api.PVRLabel: pvrName}).Result(),\n\t\t\tcheckFunc: func(pvr *velerov1api.PodVolumeRestore, requests []reconcile.Request) {\n\t\t\t\tassert.Empty(t, requests)\n\t\t\t},\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tctx := t.Context()\n\t\tassert.NoError(t, r.client.Create(ctx, test.pod))\n\t\tassert.NoError(t, r.client.Create(ctx, test.pvr))\n\t\t// Call the findSnapshotRestoreForPod function\n\t\trequests := r.findPVRForRestorePod(t.Context(), test.pod)\n\t\ttest.checkFunc(test.pvr, requests)\n\t\tr.client.Delete(ctx, test.pvr, &kbclient.DeleteOptions{})\n\t\tif test.pod != nil {\n\t\t\tr.client.Delete(ctx, test.pod, &kbclient.DeleteOptions{})\n\t\t}\n\t}\n}\n\nfunc TestOnPVRPrepareTimeout(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvr *velerov1api.PodVolumeRestore\n\t\tneedErrs []error\n\t\texpected *velerov1api.PodVolumeRestore\n\t}{\n\t\t{\n\t\t\tname: \"update fail\",\n\t\t\tpvr: pvrBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, fmt.Errorf(\"fake-update-error\"), nil},\n\t\t\texpected: pvrBuilder().Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"update interrupted\",\n\t\t\tpvr: pvrBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, &fakeAPIStatus{metav1.StatusReasonConflict}, nil},\n\t\t\texpected: pvrBuilder().Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tpvr: pvrBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, nil, nil},\n\t\t\texpected: pvrBuilder().Phase(velerov1api.PodVolumeRestorePhaseFailed).Result(),\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tctx := t.Context()\n\t\tr, err := initPodVolumeRestoreReconcilerWithError(nil, []client.Object{}, test.needErrs...)\n\t\trequire.NoError(t, err)\n\n\t\terr = r.client.Create(ctx, test.pvr)\n\t\trequire.NoError(t, err)\n\n\t\tr.onPrepareTimeout(ctx, test.pvr)\n\n\t\tpvr := velerov1api.PodVolumeRestore{}\n\t\t_ = r.client.Get(ctx, kbclient.ObjectKey{\n\t\t\tName: test.pvr.Name,\n\t\t\tNamespace: test.pvr.Namespace,\n\t\t}, &pvr)\n\n\t\tassert.Equal(t, test.expected.Status.Phase, pvr.Status.Phase)\n\t}\n}\n\nfunc TestTryCancelPVR(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvr *velerov1api.PodVolumeRestore\n\t\tneedErrs []error\n\t\tsucceeded bool\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"update fail\",\n\t\t\tpvr: pvrBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, fmt.Errorf(\"fake-update-error\"), nil},\n\t\t},\n\t\t{\n\t\t\tname: \"cancel by others\",\n\t\t\tpvr: pvrBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, &fakeAPIStatus{metav1.StatusReasonConflict}, nil},\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tpvr: pvrBuilder().Result(),\n\t\t\tneedErrs: []error{nil, nil, nil, nil},\n\t\t\tsucceeded: true,\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tctx := t.Context()\n\t\tr, err := initPodVolumeRestoreReconcilerWithError(nil, []client.Object{}, test.needErrs...)\n\t\trequire.NoError(t, err)\n\n\t\terr = r.client.Create(ctx, test.pvr)\n\t\trequire.NoError(t, err)\n\n\t\tr.tryCancelPodVolumeRestore(ctx, test.pvr, \"\")\n\n\t\tif test.expectedErr == \"\" {\n\t\t\tassert.NoError(t, err)\n\t\t} else {\n\t\t\tassert.EqualError(t, err, test.expectedErr)\n\t\t}\n\t}\n}\n\nfunc TestUpdatePVRWithRetry(t *testing.T) {\n\tnamespacedName := types.NamespacedName{\n\t\tName: pvrName,\n\t\tNamespace: \"velero\",\n\t}\n\n\t// Define test cases\n\ttestCases := []struct {\n\t\tName string\n\t\tneedErrs []bool\n\t\tnoChange bool\n\t\tExpectErr bool\n\t}{\n\t\t{\n\t\t\tName: \"SuccessOnFirstAttempt\",\n\t\t},\n\t\t{\n\t\t\tName: \"Error get\",\n\t\t\tneedErrs: []bool{true, false, false, false, false},\n\t\t\tExpectErr: true,\n\t\t},\n\t\t{\n\t\t\tName: \"Error update\",\n\t\t\tneedErrs: []bool{false, false, true, false, false},\n\t\t\tExpectErr: true,\n\t\t},\n\t\t{\n\t\t\tName: \"no change\",\n\t\t\tnoChange: true,\n\t\t\tneedErrs: []bool{false, false, true, false, false},\n\t\t},\n\t\t{\n\t\t\tName: \"Conflict with error timeout\",\n\t\t\tneedErrs: []bool{false, false, false, false, true},\n\t\t\tExpectErr: true,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.Name, func(t *testing.T) {\n\t\t\tctx, cancelFunc := context.WithTimeout(t.Context(), time.Second*5)\n\t\t\tdefer cancelFunc()\n\t\t\tr, err := initPodVolumeRestoreReconciler(nil, []client.Object{}, tc.needErrs...)\n\t\t\trequire.NoError(t, err)\n\t\t\terr = r.client.Create(ctx, pvrBuilder().Result())\n\t\t\trequire.NoError(t, err)\n\t\t\tupdateFunc := func(pvr *velerov1api.PodVolumeRestore) bool {\n\t\t\t\tif tc.noChange {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\n\t\t\t\tpvr.Spec.Cancel = true\n\n\t\t\t\treturn true\n\t\t\t}\n\t\t\terr = UpdatePVRWithRetry(ctx, r.client, namespacedName, velerotest.NewLogger().WithField(\"name\", tc.Name), updateFunc)\n\t\t\tif tc.ExpectErr {\n\t\t\t\tassert.Error(t, err)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestAttemptPVRResume(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvrs []velerov1api.PodVolumeRestore\n\t\tpvr *velerov1api.PodVolumeRestore\n\t\tneedErrs []bool\n\t\tresumeErr error\n\t\tacceptedPvrs []string\n\t\tpreparedPvrs []string\n\t\tcancelledPvrs []string\n\t\tinProgressPvrs []string\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"Other pvr\",\n\t\t\tpvr: pvrBuilder().Phase(velerov1api.PodVolumeRestorePhasePrepared).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Other pvr\",\n\t\t\tpvr: pvrBuilder().Phase(velerov1api.PodVolumeRestorePhaseAccepted).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"InProgress pvr, not the current node\",\n\t\t\tpvr: pvrBuilder().Phase(velerov1api.PodVolumeRestorePhaseInProgress).Result(),\n\t\t\tinProgressPvrs: []string{pvrName},\n\t\t},\n\t\t{\n\t\t\tname: \"InProgress pvr, no resume error\",\n\t\t\tpvr: pvrBuilder().Phase(velerov1api.PodVolumeRestorePhaseInProgress).Node(\"node-1\").Result(),\n\t\t\tinProgressPvrs: []string{pvrName},\n\t\t},\n\t\t{\n\t\t\tname: \"InProgress pvr, resume error, cancel error\",\n\t\t\tpvr: pvrBuilder().Phase(velerov1api.PodVolumeRestorePhaseInProgress).Node(\"node-1\").Result(),\n\t\t\tresumeErr: errors.New(\"fake-resume-error\"),\n\t\t\tneedErrs: []bool{false, false, true, false, false, false},\n\t\t\tinProgressPvrs: []string{pvrName},\n\t\t},\n\t\t{\n\t\t\tname: \"InProgress pvr, resume error, cancel succeed\",\n\t\t\tpvr: pvrBuilder().Phase(velerov1api.PodVolumeRestorePhaseInProgress).Node(\"node-1\").Result(),\n\t\t\tresumeErr: errors.New(\"fake-resume-error\"),\n\t\t\tcancelledPvrs: []string{pvrName},\n\t\t\tinProgressPvrs: []string{pvrName},\n\t\t},\n\t\t{\n\t\t\tname: \"Error\",\n\t\t\tneedErrs: []bool{false, false, false, false, false, true},\n\t\t\tpvr: pvrBuilder().Phase(velerov1api.PodVolumeRestorePhasePrepared).Result(),\n\t\t\texpectedError: \"error to list PVRs: List error\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\t\t\tr, err := initPodVolumeRestoreReconciler(nil, []client.Object{}, test.needErrs...)\n\t\t\tr.nodeName = \"node-1\"\n\t\t\trequire.NoError(t, err)\n\t\t\tdefer func() {\n\t\t\t\tr.client.Delete(ctx, test.pvr, &kbclient.DeleteOptions{})\n\t\t\t}()\n\n\t\t\trequire.NoError(t, r.client.Create(ctx, test.pvr))\n\n\t\t\tdt := &pvbResumeTestHelper{\n\t\t\t\tresumeErr: test.resumeErr,\n\t\t\t}\n\n\t\t\tfuncResumeCancellableDataBackup = dt.resumeCancellableDataPath\n\n\t\t\t// Run the test\n\t\t\terr = r.AttemptPVRResume(ctx, r.logger.WithField(\"name\", test.name), test.pvr.Namespace)\n\n\t\t\tif test.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.expectedError)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\n\t\t\t\tfor _, pvrName := range test.cancelledPvrs {\n\t\t\t\t\tpvr := &velerov1api.PodVolumeRestore{}\n\t\t\t\t\terr := r.client.Get(t.Context(), types.NamespacedName{Namespace: \"velero\", Name: pvrName}, pvr)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tassert.True(t, pvr.Spec.Cancel)\n\t\t\t\t}\n\n\t\t\t\tfor _, pvrName := range test.acceptedPvrs {\n\t\t\t\t\tpvr := &velerov1api.PodVolumeRestore{}\n\t\t\t\t\terr := r.client.Get(t.Context(), types.NamespacedName{Namespace: \"velero\", Name: pvrName}, pvr)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tassert.Equal(t, velerov1api.PodVolumeRestorePhaseAccepted, pvr.Status.Phase)\n\t\t\t\t}\n\n\t\t\t\tfor _, pvrName := range test.preparedPvrs {\n\t\t\t\t\tpvr := &velerov1api.PodVolumeRestore{}\n\t\t\t\t\terr := r.client.Get(t.Context(), types.NamespacedName{Namespace: \"velero\", Name: pvrName}, pvr)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tassert.Equal(t, velerov1api.PodVolumeRestorePhasePrepared, pvr.Status.Phase)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestResumeCancellablePodVolumeRestore(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvrs []velerov1api.PodVolumeRestore\n\t\tpvr *velerov1api.PodVolumeRestore\n\t\tgetExposeErr error\n\t\texposeResult *exposer.ExposeResult\n\t\tcreateWatcherErr error\n\t\tinitWatcherErr error\n\t\tstartWatcherErr error\n\t\tmockInit bool\n\t\tmockStart bool\n\t\tmockClose bool\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"get expose failed\",\n\t\t\tpvr: pvrBuilder().Phase(velerov1api.PodVolumeRestorePhaseInProgress).Result(),\n\t\t\tgetExposeErr: errors.New(\"fake-expose-error\"),\n\t\t\texpectedError: fmt.Sprintf(\"error to get exposed PVR %s: fake-expose-error\", pvrName),\n\t\t},\n\t\t{\n\t\t\tname: \"no expose\",\n\t\t\tpvr: pvrBuilder().Phase(velerov1api.PodVolumeRestorePhaseAccepted).Node(\"node-1\").Result(),\n\t\t\texpectedError: fmt.Sprintf(\"no expose result is available for the current node for PVR %s\", pvrName),\n\t\t},\n\t\t{\n\t\t\tname: \"watcher init error\",\n\t\t\tpvr: pvrBuilder().Phase(velerov1api.PodVolumeRestorePhaseAccepted).Node(\"node-1\").Result(),\n\t\t\texposeResult: &exposer.ExposeResult{\n\t\t\t\tByPod: exposer.ExposeByPod{\n\t\t\t\t\tHostingPod: &corev1api.Pod{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tmockInit: true,\n\t\t\tmockClose: true,\n\t\t\tinitWatcherErr: errors.New(\"fake-init-watcher-error\"),\n\t\t\texpectedError: fmt.Sprintf(\"error to init asyncBR watcher for PVR %s: fake-init-watcher-error\", pvrName),\n\t\t},\n\t\t{\n\t\t\tname: \"start watcher error\",\n\t\t\tpvr: pvrBuilder().Phase(velerov1api.PodVolumeRestorePhaseAccepted).Node(\"node-1\").Result(),\n\t\t\texposeResult: &exposer.ExposeResult{\n\t\t\t\tByPod: exposer.ExposeByPod{\n\t\t\t\t\tHostingPod: &corev1api.Pod{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tmockInit: true,\n\t\t\tmockStart: true,\n\t\t\tmockClose: true,\n\t\t\tstartWatcherErr: errors.New(\"fake-start-watcher-error\"),\n\t\t\texpectedError: fmt.Sprintf(\"error to resume asyncBR watcher for PVR %s: fake-start-watcher-error\", pvrName),\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tpvr: pvrBuilder().Phase(velerov1api.PodVolumeRestorePhaseAccepted).Node(\"node-1\").Result(),\n\t\t\texposeResult: &exposer.ExposeResult{\n\t\t\t\tByPod: exposer.ExposeByPod{\n\t\t\t\t\tHostingPod: &corev1api.Pod{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tmockInit: true,\n\t\t\tmockStart: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\t\t\tr, err := initPodVolumeRestoreReconciler(nil, []client.Object{})\n\t\t\tr.nodeName = \"node-1\"\n\t\t\trequire.NoError(t, err)\n\n\t\t\tmockAsyncBR := datapathmockes.NewAsyncBR(t)\n\n\t\t\tif test.mockInit {\n\t\t\t\tmockAsyncBR.On(\"Init\", mock.Anything, mock.Anything).Return(test.initWatcherErr)\n\t\t\t}\n\n\t\t\tif test.mockStart {\n\t\t\t\tmockAsyncBR.On(\"StartRestore\", mock.Anything, mock.Anything, mock.Anything).Return(test.startWatcherErr)\n\t\t\t}\n\n\t\t\tif test.mockClose {\n\t\t\t\tmockAsyncBR.On(\"Close\", mock.Anything).Return()\n\t\t\t}\n\n\t\t\tdt := &pvbResumeTestHelper{\n\t\t\t\tgetExposeErr: test.getExposeErr,\n\t\t\t\texposeResult: test.exposeResult,\n\t\t\t\tasyncBR: mockAsyncBR,\n\t\t\t}\n\n\t\t\tr.exposer = dt\n\n\t\t\tdatapath.MicroServiceBRWatcherCreator = dt.newMicroServiceBRWatcher\n\n\t\t\terr = r.resumeCancellableDataPath(ctx, test.pvr, velerotest.NewLogger())\n\t\t\tif test.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.expectedError)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/controller/restore_controller.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"bytes\"\n\t\"compress/gzip\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"sort\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil\"\n\n\t\"github.com/vmware-tanzu/velero/internal/hook\"\n\t\"github.com/vmware-tanzu/velero/internal/resourcemodifiers\"\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\t\"github.com/vmware-tanzu/velero/pkg/persistence\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework\"\n\tpkgrestore \"github.com/vmware-tanzu/velero/pkg/restore\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/collections\"\n\tkubeutil \"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/results\"\n\tveleroutil \"github.com/vmware-tanzu/velero/pkg/util/velero\"\n\tpkgrestoreUtil \"github.com/vmware-tanzu/velero/pkg/util/velero/restore\"\n)\n\n// nonRestorableResources is an exclusion list for the restoration process. Any resources\n// included here are explicitly excluded from the restoration process.\nvar nonRestorableResources = []string{\n\t\"nodes\",\n\t\"events\",\n\t\"events.events.k8s.io\",\n\n\t// Don't ever restore backups - if appropriate, they'll be synced in from object storage.\n\t// https://github.com/vmware-tanzu/velero/issues/622\n\t\"backups.velero.io\",\n\n\t// Restores are cluster-specific, and don't have value moving across clusters.\n\t// https://github.com/vmware-tanzu/velero/issues/622\n\t\"restores.velero.io\",\n\n\t// TODO: Remove this in v1.11 or v1.12\n\t// Restic repositories are automatically managed by Velero and will be automatically\n\t// created as needed if they don't exist.\n\t// https://github.com/vmware-tanzu/velero/issues/1113\n\t\"resticrepositories.velero.io\",\n\n\t// CSINode delegates cluster node for CSI operation.\n\t// VolumeAttachement records PV mounts to which node.\n\t// https://github.com/vmware-tanzu/velero/issues/4823\n\t\"csinodes.storage.k8s.io\",\n\t\"volumeattachments.storage.k8s.io\",\n\n\t// Backup repositories were renamed from Restic repositories\n\t\"backuprepositories.velero.io\",\n}\n\nvar ExternalResourcesFinalizer = \"restores.velero.io/external-resources-finalizer\"\n\ntype restoreReconciler struct {\n\tctx context.Context\n\tnamespace string\n\trestorer pkgrestore.Restorer\n\tkbClient client.Client\n\trestoreLogLevel logrus.Level\n\tlogger logrus.FieldLogger\n\tmetrics *metrics.ServerMetrics\n\tlogFormat logging.Format\n\tclock clock.WithTickerAndDelayedExecution\n\tdefaultItemOperationTimeout time.Duration\n\tdisableInformerCache bool\n\n\tnewPluginManager func(logger logrus.FieldLogger) clientmgmt.Manager\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter\n\tglobalCrClient client.Client\n\tresourceTimeout time.Duration\n}\n\ntype backupInfo struct {\n\tbackup *api.Backup\n\tlocation *api.BackupStorageLocation\n}\n\nfunc NewRestoreReconciler(\n\tctx context.Context,\n\tnamespace string,\n\trestorer pkgrestore.Restorer,\n\tkbClient client.Client,\n\tlogger logrus.FieldLogger,\n\trestoreLogLevel logrus.Level,\n\tnewPluginManager func(logrus.FieldLogger) clientmgmt.Manager,\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter,\n\tmetrics *metrics.ServerMetrics,\n\tlogFormat logging.Format,\n\tdefaultItemOperationTimeout time.Duration,\n\tdisableInformerCache bool,\n\tglobalCrClient client.Client,\n\tresourceTimeout time.Duration,\n) *restoreReconciler {\n\tr := &restoreReconciler{\n\t\tctx: ctx,\n\t\tnamespace: namespace,\n\t\trestorer: restorer,\n\t\tkbClient: kbClient,\n\t\tlogger: logger,\n\t\trestoreLogLevel: restoreLogLevel,\n\t\tmetrics: metrics,\n\t\tlogFormat: logFormat,\n\t\tclock: &clock.RealClock{},\n\t\tdefaultItemOperationTimeout: defaultItemOperationTimeout,\n\t\tdisableInformerCache: disableInformerCache,\n\n\t\t// use variables to refer to these functions so they can be\n\t\t// replaced with fakes for testing.\n\t\tnewPluginManager: newPluginManager,\n\t\tbackupStoreGetter: backupStoreGetter,\n\n\t\tglobalCrClient: globalCrClient,\n\t\tresourceTimeout: resourceTimeout,\n\t}\n\n\t// Move the periodical backup and restore metrics computing logic from controllers to here.\n\t// This is due to, after controllers using controller-runtime, controllers doesn't have a\n\t// timer as the before generic-controller, and the backup and restore controller only have\n\t// one length queue, furthermore the backup and restore process could last for a long time.\n\t// Compute the metric here is a better choice.\n\tr.updateTotalRestoreMetric()\n\n\treturn r\n}\n\nfunc (r *restoreReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\t// Developer note: any error returned by this method will\n\t// cause the restore to be re-enqueued and re-processed by\n\t// the controller.\n\tlog := r.logger.WithField(\"Restore\", req.NamespacedName.String())\n\n\trestore := &api.Restore{}\n\terr := r.kbClient.Get(ctx, client.ObjectKey{Namespace: req.Namespace, Name: req.Name}, restore)\n\tif err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.Debugf(\"restore[%s] not found\", req.Name)\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tlog.Errorf(\"Fail to get restore %s: %s\", req.NamespacedName.String(), err.Error())\n\t\treturn ctrl.Result{}, err\n\t}\n\n\t// deal with finalizer\n\tif !restore.DeletionTimestamp.IsZero() {\n\t\t// check the finalizer and run clean-up\n\t\tif controllerutil.ContainsFinalizer(restore, ExternalResourcesFinalizer) {\n\t\t\tif err := r.deleteExternalResources(restore); err != nil {\n\t\t\t\tlog.Errorf(\"fail to delete external resources: %s\", err.Error())\n\t\t\t\treturn ctrl.Result{}, err\n\t\t\t}\n\t\t\t// once finish clean-up, remove the finalizer from the restore so that the restore will be unlocked and deleted.\n\t\t\toriginal := restore.DeepCopy()\n\t\t\tcontrollerutil.RemoveFinalizer(restore, ExternalResourcesFinalizer)\n\t\t\tif err := kubeutil.PatchResource(original, restore, r.kbClient); err != nil {\n\t\t\t\tlog.Errorf(\"fail to remove finalizer: %s\", err.Error())\n\t\t\t\treturn ctrl.Result{}, err\n\t\t\t}\n\t\t\treturn ctrl.Result{}, nil\n\t\t} else {\n\t\t\tlog.Error(\"DeletionTimestamp is marked but can't find the expected finalizer\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t}\n\n\t// add finalizer\n\tif restore.DeletionTimestamp.IsZero() && !controllerutil.ContainsFinalizer(restore, ExternalResourcesFinalizer) {\n\t\toriginal := restore.DeepCopy()\n\t\tcontrollerutil.AddFinalizer(restore, ExternalResourcesFinalizer)\n\t\tif err := kubeutil.PatchResource(original, restore, r.kbClient); err != nil {\n\t\t\tlog.Errorf(\"fail to add finalizer: %s\", err.Error())\n\t\t\treturn ctrl.Result{}, err\n\t\t}\n\t}\n\n\tswitch restore.Status.Phase {\n\tcase \"\", api.RestorePhaseNew:\n\t\t// only process new restores\n\tdefault:\n\t\tr.logger.WithFields(logrus.Fields{\n\t\t\t\"restore\": kubeutil.NamespaceAndName(restore),\n\t\t\t\"phase\": restore.Status.Phase,\n\t\t}).Debug(\"Restore is not handled\")\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\t// store a copy of the original restore for creating patch\n\toriginal := restore.DeepCopy()\n\n\t// Validate the restore and fetch the backup\n\tinfo, resourceModifiers := r.validateAndComplete(restore)\n\n\t// Register attempts after validation so we don't have to fetch the backup multiple times\n\tbackupScheduleName := restore.Spec.ScheduleName\n\tr.metrics.RegisterRestoreAttempt(backupScheduleName)\n\n\tif len(restore.Status.ValidationErrors) > 0 {\n\t\trestore.Status.Phase = api.RestorePhaseFailedValidation\n\t\tr.metrics.RegisterRestoreValidationFailed(backupScheduleName)\n\t} else {\n\t\trestore.Status.StartTimestamp = &metav1.Time{Time: r.clock.Now()}\n\t\trestore.Status.Phase = api.RestorePhaseInProgress\n\t}\n\tif restore.Spec.ItemOperationTimeout.Duration == 0 {\n\t\t// set default item operation timeout\n\t\trestore.Spec.ItemOperationTimeout.Duration = r.defaultItemOperationTimeout\n\t}\n\n\t// patch to update status and persist to API\n\t// This is patching from \"\" or New, no retry needed\n\terr = kubeutil.PatchResource(original, restore, r.kbClient)\n\tif err != nil {\n\t\t// return the error so the restore can be re-processed; it's currently\n\t\t// still in phase = New.\n\t\tlog.Errorf(\"fail to update restore %s status to %s: %s\",\n\t\t\treq.NamespacedName.String(), restore.Status.Phase, err.Error())\n\t\treturn ctrl.Result{}, errors.Wrapf(err, \"error updating Restore phase to %s\", restore.Status.Phase)\n\t}\n\t// store ref to just-updated item for creating patch\n\toriginal = restore.DeepCopy()\n\n\tif restore.Status.Phase == api.RestorePhaseFailedValidation {\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\tif err := r.runValidatedRestore(restore, info, resourceModifiers); err != nil {\n\t\tlog.WithError(err).Debug(\"Restore failed\")\n\t\trestore.Status.Phase = api.RestorePhaseFailed\n\t\trestore.Status.FailureReason = err.Error()\n\t\tr.metrics.RegisterRestoreFailed(backupScheduleName)\n\t}\n\n\t// mark completion if in terminal phase\n\tif restore.Status.Phase == api.RestorePhaseFailed ||\n\t\trestore.Status.Phase == api.RestorePhasePartiallyFailed ||\n\t\trestore.Status.Phase == api.RestorePhaseCompleted {\n\t\trestore.Status.CompletionTimestamp = &metav1.Time{Time: r.clock.Now()}\n\t}\n\tlog.Debug(\"Updating restore's status\")\n\t// Phases were updated in runValidatedRestore\n\t// This patch with retry update Phase from InProgress to\n\t// WaitingForPluginOperations\n\t// WaitingForPluginOperationsPartiallyFailed\n\t// Finalizing\n\t// FinalizingPartiallyFailed\n\tif err = kubeutil.PatchResourceWithRetriesOnErrors(r.resourceTimeout, original, restore, r.kbClient); err != nil {\n\t\tlog.WithError(errors.WithStack(err)).Infof(\"Error updating restore's status from %v to %v\", original.Status.Phase, restore.Status.Phase)\n\t\t// No need to re-enqueue here, because restore's already set to InProgress before.\n\t\t// Controller only handle New restore.\n\t}\n\n\treturn ctrl.Result{}, nil\n}\n\nfunc (r *restoreReconciler) SetupWithManager(mgr ctrl.Manager) error {\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\tFor(&api.Restore{}).\n\t\tNamed(constant.ControllerRestore).\n\t\tComplete(r)\n}\n\nfunc (r *restoreReconciler) validateAndComplete(restore *api.Restore) (backupInfo, *resourcemodifiers.ResourceModifiers) {\n\t// add non-restorable resources to restore's excluded resources\n\texcludedResources := sets.NewString(restore.Spec.ExcludedResources...)\n\tfor _, nonrestorable := range nonRestorableResources {\n\t\tif !excludedResources.Has(nonrestorable) {\n\t\t\trestore.Spec.ExcludedResources = append(restore.Spec.ExcludedResources, nonrestorable)\n\t\t}\n\t}\n\n\t// validate that included resources don't contain any non-restorable resources\n\tincludedResources := sets.NewString(restore.Spec.IncludedResources...)\n\tfor _, nonRestorableResource := range nonRestorableResources {\n\t\tif includedResources.Has(nonRestorableResource) {\n\t\t\trestore.Status.ValidationErrors = append(restore.Status.ValidationErrors, fmt.Sprintf(\"%v are non-restorable resources\", nonRestorableResource))\n\t\t}\n\t}\n\n\t// validate included/excluded resources\n\tfor _, err := range collections.ValidateIncludesExcludes(restore.Spec.IncludedResources, restore.Spec.ExcludedResources) {\n\t\trestore.Status.ValidationErrors = append(restore.Status.ValidationErrors, fmt.Sprintf(\"Invalid included/excluded resource lists: %v\", err))\n\t}\n\n\t// validate included/excluded namespaces\n\tfor _, err := range collections.ValidateIncludesExcludes(restore.Spec.IncludedNamespaces, restore.Spec.ExcludedNamespaces) {\n\t\trestore.Status.ValidationErrors = append(restore.Status.ValidationErrors, fmt.Sprintf(\"Invalid included/excluded namespace lists: %v\", err))\n\t}\n\n\t// validate that only one exists orLabelSelector or just labelSelector (singular)\n\tif restore.Spec.OrLabelSelectors != nil && restore.Spec.LabelSelector != nil {\n\t\trestore.Status.ValidationErrors = append(restore.Status.ValidationErrors, \"encountered labelSelector as well as orLabelSelectors in restore spec, only one can be specified\")\n\t}\n\n\t// validate that exactly one of BackupName and ScheduleName have been specified\n\tif !backupXorScheduleProvided(restore) {\n\t\trestore.Status.ValidationErrors = append(restore.Status.ValidationErrors, \"Either a backup or schedule must be specified as a source for the restore, but not both\")\n\t\treturn backupInfo{}, nil\n\t}\n\n\t// validate Restore Init Hook's InitContainers\n\trestoreHooks, err := hook.GetRestoreHooksFromSpec(&restore.Spec.Hooks)\n\tif err != nil {\n\t\trestore.Status.ValidationErrors = append(restore.Status.ValidationErrors, err.Error())\n\t}\n\tfor _, resource := range restoreHooks {\n\t\tfor _, h := range resource.RestoreHooks {\n\t\t\tif h.Init != nil {\n\t\t\t\tfor _, container := range h.Init.InitContainers {\n\t\t\t\t\terr = hook.ValidateContainer(container.Raw)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\trestore.Status.ValidationErrors = append(restore.Status.ValidationErrors, err.Error())\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\t// validate ExistingResourcePolicy\n\tif restore.Spec.ExistingResourcePolicy != \"\" && !pkgrestoreUtil.IsResourcePolicyValid(string(restore.Spec.ExistingResourcePolicy)) {\n\t\trestore.Status.ValidationErrors = append(restore.Status.ValidationErrors, fmt.Sprintf(\"Invalid ExistingResourcePolicy: %s\", restore.Spec.ExistingResourcePolicy))\n\t}\n\n\t// if ScheduleName is specified, fill in BackupName with the most recent successful backup from\n\t// the schedule\n\tif restore.Spec.ScheduleName != \"\" {\n\t\tselector := labels.SelectorFromSet(labels.Set(map[string]string{\n\t\t\tapi.ScheduleNameLabel: restore.Spec.ScheduleName,\n\t\t}))\n\n\t\tbackupList := &api.BackupList{}\n\t\tif err := r.kbClient.List(context.Background(), backupList, &client.ListOptions{LabelSelector: selector}); err != nil {\n\t\t\trestore.Status.ValidationErrors = append(restore.Status.ValidationErrors, \"Unable to list backups for schedule\")\n\t\t\treturn backupInfo{}, nil\n\t\t}\n\t\tif len(backupList.Items) == 0 {\n\t\t\trestore.Status.ValidationErrors = append(restore.Status.ValidationErrors, \"No backups found for schedule\")\n\t\t}\n\n\t\tif backup := mostRecentCompletedBackup(backupList.Items); backup.Name != \"\" {\n\t\t\trestore.Spec.BackupName = backup.Name\n\t\t} else {\n\t\t\trestore.Status.ValidationErrors = append(restore.Status.ValidationErrors, \"No completed backups found for schedule\")\n\t\t\treturn backupInfo{}, nil\n\t\t}\n\t}\n\n\tinfo, err := r.fetchBackupInfo(restore.Spec.BackupName)\n\tif err != nil {\n\t\trestore.Status.ValidationErrors = append(restore.Status.ValidationErrors, fmt.Sprintf(\"Error retrieving backup: %v\", err))\n\t\treturn backupInfo{}, nil\n\t}\n\n\tif !veleroutil.BSLIsAvailable(*info.location) {\n\t\trestore.Status.ValidationErrors = append(restore.Status.ValidationErrors, fmt.Sprintf(\"The BSL %s is unavailable, cannot retrieve the backup\", info.location.Name))\n\t\treturn backupInfo{}, nil\n\t}\n\n\t// Fill in the ScheduleName so it's easier to consume for metrics.\n\tif restore.Spec.ScheduleName == \"\" {\n\t\trestore.Spec.ScheduleName = info.backup.GetLabels()[api.ScheduleNameLabel]\n\t}\n\n\tvar resourceModifiers *resourcemodifiers.ResourceModifiers\n\tif restore.Spec.ResourceModifier != nil && strings.EqualFold(restore.Spec.ResourceModifier.Kind, resourcemodifiers.ConfigmapRefType) {\n\t\tResourceModifierConfigMap := &corev1api.ConfigMap{}\n\t\terr := r.kbClient.Get(context.Background(), client.ObjectKey{Namespace: restore.Namespace, Name: restore.Spec.ResourceModifier.Name}, ResourceModifierConfigMap)\n\t\tif err != nil {\n\t\t\trestore.Status.ValidationErrors = append(restore.Status.ValidationErrors, fmt.Sprintf(\"failed to get resource modifiers configmap %s/%s\", restore.Namespace, restore.Spec.ResourceModifier.Name))\n\t\t\treturn backupInfo{}, nil\n\t\t}\n\t\tresourceModifiers, err = resourcemodifiers.GetResourceModifiersFromConfig(ResourceModifierConfigMap)\n\t\tif err != nil {\n\t\t\trestore.Status.ValidationErrors = append(restore.Status.ValidationErrors, errors.Wrapf(err, \"Error in parsing resource modifiers provided in configmap %s/%s\", restore.Namespace, restore.Spec.ResourceModifier.Name).Error())\n\t\t\treturn backupInfo{}, nil\n\t\t} else if err = resourceModifiers.Validate(); err != nil {\n\t\t\trestore.Status.ValidationErrors = append(restore.Status.ValidationErrors, errors.Wrapf(err, \"Validation error in resource modifiers provided in configmap %s/%s\", restore.Namespace, restore.Spec.ResourceModifier.Name).Error())\n\t\t\treturn backupInfo{}, nil\n\t\t}\n\t\tr.logger.Infof(\"Retrieved Resource modifiers provided in configmap %s/%s\", restore.Namespace, restore.Spec.ResourceModifier.Name)\n\t}\n\n\treturn info, resourceModifiers\n}\n\n// backupXorScheduleProvided returns true if exactly one of BackupName and\n// ScheduleName are non-empty for the restore, or false otherwise.\nfunc backupXorScheduleProvided(restore *api.Restore) bool {\n\tif restore.Spec.BackupName != \"\" && restore.Spec.ScheduleName != \"\" {\n\t\treturn false\n\t}\n\n\tif restore.Spec.BackupName == \"\" && restore.Spec.ScheduleName == \"\" {\n\t\treturn false\n\t}\n\n\treturn true\n}\n\n// mostRecentCompletedBackup returns the most recent backup that's\n// completed from a list of backups.\nfunc mostRecentCompletedBackup(backups []api.Backup) api.Backup {\n\tsort.Slice(backups, func(i, j int) bool {\n\t\t// Use .After() because we want descending sort.\n\n\t\tvar iStartTime, jStartTime time.Time\n\t\tif backups[i].Status.StartTimestamp != nil {\n\t\t\tiStartTime = backups[i].Status.StartTimestamp.Time\n\t\t}\n\t\tif backups[j].Status.StartTimestamp != nil {\n\t\t\tjStartTime = backups[j].Status.StartTimestamp.Time\n\t\t}\n\t\treturn iStartTime.After(jStartTime)\n\t})\n\n\tfor _, backup := range backups {\n\t\tif backup.Status.Phase == api.BackupPhaseCompleted {\n\t\t\treturn backup\n\t\t}\n\t}\n\n\treturn api.Backup{}\n}\n\n// fetchBackupInfo checks the backup lister for a backup that matches the given name. If it doesn't\n// find it, it returns an error.\nfunc (r *restoreReconciler) fetchBackupInfo(backupName string) (backupInfo, error) {\n\treturn fetchBackupInfoInternal(r.kbClient, r.namespace, backupName)\n}\n\nfunc fetchBackupInfoInternal(kbClient client.Client, namespace, backupName string) (backupInfo, error) {\n\tbackup := &api.Backup{}\n\terr := kbClient.Get(context.Background(), types.NamespacedName{Namespace: namespace, Name: backupName}, backup)\n\tif err != nil {\n\t\treturn backupInfo{}, errors.Wrap(err, fmt.Sprintf(\"can't find backup %s/%s\", namespace, backupName))\n\t}\n\n\tlocation := &api.BackupStorageLocation{}\n\tif err := kbClient.Get(context.Background(), client.ObjectKey{\n\t\tNamespace: namespace,\n\t\tName: backup.Spec.StorageLocation,\n\t}, location); err != nil {\n\t\treturn backupInfo{}, errors.WithStack(err)\n\t}\n\n\treturn backupInfo{\n\t\tbackup: backup,\n\t\tlocation: location,\n\t}, nil\n}\n\n// runValidatedRestore takes a validated restore API object and executes the restore process.\n// The log and results files are uploaded to backup storage. Any error returned from this function\n// means that the restore failed. This function updates the restore API object with warning and error\n// counts, but *does not* update its phase or patch it via the API.\nfunc (r *restoreReconciler) runValidatedRestore(restore *api.Restore, info backupInfo, resourceModifiers *resourcemodifiers.ResourceModifiers) error {\n\t// instantiate the per-restore logger that will output both to a temp file\n\t// (for upload to object storage) and to stdout.\n\trestoreLog, err := logging.NewTempFileLogger(r.restoreLogLevel, r.logFormat, nil, logrus.Fields{\"restore\": kubeutil.NamespaceAndName(restore)})\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer restoreLog.Dispose(r.logger)\n\n\tpluginManager := r.newPluginManager(restoreLog)\n\tdefer pluginManager.CleanupClients()\n\n\tbackupStore, err := r.backupStoreGetter.Get(info.location, pluginManager, r.logger)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tactions, err := pluginManager.GetRestoreItemActionsV2()\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error getting restore item actions\")\n\t}\n\tactionsResolver := framework.NewRestoreItemActionResolverV2(actions)\n\n\tbackupFile, err := downloadToTempFile(restore.Spec.BackupName, backupStore, restoreLog)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error downloading backup\")\n\t}\n\tdefer closeAndRemoveFile(backupFile, r.logger)\n\n\tlistOpts := &client.ListOptions{\n\t\tLabelSelector: labels.Set(map[string]string{\n\t\t\tapi.BackupNameLabel: label.GetValidName(restore.Spec.BackupName),\n\t\t}).AsSelector(),\n\t}\n\n\tpodVolumeBackupList := &api.PodVolumeBackupList{}\n\terr = r.kbClient.List(context.TODO(), podVolumeBackupList, listOpts)\n\tif err != nil {\n\t\trestoreLog.Errorf(\"Fail to list PodVolumeBackup :%s\", err.Error())\n\t\treturn errors.WithStack(err)\n\t}\n\n\tvolumeSnapshots, err := backupStore.GetBackupVolumeSnapshots(restore.Spec.BackupName)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error fetching volume snapshots metadata\")\n\t}\n\n\tcsiVolumeSnapshots, err := backupStore.GetCSIVolumeSnapshots(restore.Spec.BackupName)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"fail to fetch CSI VolumeSnapshots metadata\")\n\t}\n\n\tbackupVolumeInfoMap := make(map[string]volume.BackupVolumeInfo)\n\tvolumeInfos, err := backupStore.GetBackupVolumeInfos(restore.Spec.BackupName)\n\tif err != nil {\n\t\trestoreLog.WithError(err).Errorf(\"fail to get VolumeInfos metadata file for backup %s\", restore.Spec.BackupName)\n\t\treturn errors.WithStack(err)\n\t} else {\n\t\tfor _, volumeInfo := range volumeInfos {\n\t\t\tbackupVolumeInfoMap[volumeInfo.PVName] = *volumeInfo\n\t\t}\n\t}\n\n\trestoreLog.Info(\"starting restore\")\n\n\tvar podVolumeBackups []*api.PodVolumeBackup\n\tfor i := range podVolumeBackupList.Items {\n\t\tpodVolumeBackups = append(podVolumeBackups, &podVolumeBackupList.Items[i])\n\t}\n\n\trestoreReq := &pkgrestore.Request{\n\t\tLog: restoreLog,\n\t\tRestore: restore,\n\t\tBackup: info.backup,\n\t\tPodVolumeBackups: podVolumeBackups,\n\t\tVolumeSnapshots: volumeSnapshots,\n\t\tBackupReader: backupFile,\n\t\tResourceModifiers: resourceModifiers,\n\t\tDisableInformerCache: r.disableInformerCache,\n\t\tCSIVolumeSnapshots: csiVolumeSnapshots,\n\t\tBackupVolumeInfoMap: backupVolumeInfoMap,\n\t\tRestoreVolumeInfoTracker: volume.NewRestoreVolInfoTracker(restore, restoreLog, r.globalCrClient),\n\t\tResourceDeletionStatusTracker: kubeutil.NewResourceDeletionStatusTracker(),\n\t}\n\trestoreWarnings, restoreErrors := r.restorer.RestoreWithResolvers(restoreReq, actionsResolver, pluginManager)\n\n\t// Iterate over restore item operations and update progress.\n\t// Any errors on operations at this point should be added to restore errors.\n\t// If any operations are still not complete, then restore will not be set to\n\t// Completed yet.\n\tinProgressOperations, _, opsCompleted, opsFailed, errs := getRestoreItemOperationProgress(restoreReq.Restore, pluginManager, *restoreReq.GetItemOperationsList())\n\tif len(errs) > 0 {\n\t\tfor _, err := range errs {\n\t\t\trestoreErrors.Velero = append(restoreErrors.Velero, fmt.Sprintf(\"error from restore item operation: %v\", err))\n\t\t}\n\t}\n\n\trestore.Status.RestoreItemOperationsAttempted = len(*restoreReq.GetItemOperationsList())\n\trestore.Status.RestoreItemOperationsCompleted = opsCompleted\n\trestore.Status.RestoreItemOperationsFailed = opsFailed\n\n\t// log errors and warnings to the restore log\n\tfor _, msg := range restoreErrors.Velero {\n\t\trestoreLog.Errorf(\"Velero restore error: %v\", msg)\n\t}\n\tfor _, msg := range restoreErrors.Cluster {\n\t\trestoreLog.Errorf(\"Cluster resource restore error: %v\", msg)\n\t}\n\tfor ns, errs := range restoreErrors.Namespaces {\n\t\tfor _, msg := range errs {\n\t\t\trestoreLog.Errorf(\"Namespace %v, resource restore error: %v\", ns, msg)\n\t\t}\n\t}\n\tfor _, msg := range restoreWarnings.Velero {\n\t\trestoreLog.Warnf(\"Velero restore warning: %v\", msg)\n\t}\n\tfor _, msg := range restoreWarnings.Cluster {\n\t\trestoreLog.Warnf(\"Cluster resource restore warning: %v\", msg)\n\t}\n\tfor ns, errs := range restoreWarnings.Namespaces {\n\t\tfor _, msg := range errs {\n\t\t\trestoreLog.Warnf(\"Namespace %v, resource restore warning: %v\", ns, msg)\n\t\t}\n\t}\n\trestoreLog.Info(\"restore completed\")\n\n\trestoreLog.DoneForPersist(r.logger)\n\n\t// re-instantiate the backup store because credentials could have changed since the original\n\t// instantiation, if this was a long-running restore\n\tbackupStore, err = r.backupStoreGetter.Get(info.location, pluginManager, r.logger)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error setting up backup store to persist log and results files\")\n\t}\n\n\tif logReader, err := restoreLog.GetPersistFile(); err != nil {\n\t\trestoreErrors.Velero = append(restoreErrors.Velero, fmt.Sprintf(\"error getting restore log reader: %v\", err))\n\t} else {\n\t\tif err := backupStore.PutRestoreLog(restore.Spec.BackupName, restore.Name, logReader); err != nil {\n\t\t\trestoreErrors.Velero = append(restoreErrors.Velero, fmt.Sprintf(\"error uploading log file to backup storage: %v\", err))\n\t\t}\n\t}\n\n\t// At this point, no further logs should be written to restoreLog since it's been uploaded\n\t// to object storage.\n\n\trestore.Status.Warnings = len(restoreWarnings.Velero) + len(restoreWarnings.Cluster)\n\tfor _, w := range restoreWarnings.Namespaces {\n\t\trestore.Status.Warnings += len(w)\n\t}\n\n\trestore.Status.Errors = len(restoreErrors.Velero) + len(restoreErrors.Cluster)\n\tfor _, e := range restoreErrors.Namespaces {\n\t\trestore.Status.Errors += len(e)\n\t}\n\n\tm := map[string]results.Result{\n\t\t\"warnings\": restoreWarnings,\n\t\t\"errors\": restoreErrors,\n\t}\n\n\tif err := putResults(restore, m, backupStore); err != nil {\n\t\tr.logger.WithError(err).Error(\"Error uploading restore results to backup storage\")\n\t}\n\n\tif err := putRestoredResourceList(restore, restoreReq.RestoredResourceList(), backupStore); err != nil {\n\t\tr.logger.WithError(err).Error(\"Error uploading restored resource list to backup storage\")\n\t}\n\n\tif err := putOperationsForRestore(restore, *restoreReq.GetItemOperationsList(), backupStore); err != nil {\n\t\tr.logger.WithError(err).Error(\"Error uploading restore item action operation resource list to backup storage\")\n\t}\n\n\trestoreReq.RestoreVolumeInfoTracker.Populate(context.TODO(), restoreReq.RestoredResourceList())\n\tif err := putRestoreVolumeInfoList(restore, restoreReq.RestoreVolumeInfoTracker.Result(), backupStore); err != nil {\n\t\tr.logger.WithError(err).Error(\"Error uploading restored volume info to backup storage\")\n\t}\n\n\tif restore.Status.Errors > 0 {\n\t\tif inProgressOperations {\n\t\t\tr.logger.Debug(\"Restore WaitingForPluginOperationsPartiallyFailed\")\n\t\t\trestore.Status.Phase = api.RestorePhaseWaitingForPluginOperationsPartiallyFailed\n\t\t} else {\n\t\t\tr.logger.Debug(\"Restore FinalizingPartiallyFailed\")\n\t\t\trestore.Status.Phase = api.RestorePhaseFinalizingPartiallyFailed\n\t\t}\n\t} else {\n\t\tif inProgressOperations {\n\t\t\tr.logger.Debug(\"Restore WaitingForPluginOperations\")\n\t\t\trestore.Status.Phase = api.RestorePhaseWaitingForPluginOperations\n\t\t} else {\n\t\t\tr.logger.Debug(\"Restore Finalizing\")\n\t\t\trestore.Status.Phase = api.RestorePhaseFinalizing\n\t\t}\n\t}\n\treturn nil\n}\n\n// updateTotalRestoreMetric update the velero_restore_total metric every minute.\nfunc (r *restoreReconciler) updateTotalRestoreMetric() {\n\tgo func() {\n\t\t// Wait for 5 seconds to let controller-runtime to setup k8s clients.\n\t\ttime.Sleep(5 * time.Second)\n\n\t\twait.Until(\n\t\t\tfunc() {\n\t\t\t\t// recompute restore_total metric\n\t\t\t\trestoreList := &api.RestoreList{}\n\t\t\t\terr := r.kbClient.List(context.Background(), restoreList, &client.ListOptions{})\n\t\t\t\tif err != nil {\n\t\t\t\t\tr.logger.Error(err, \"Error computing restore_total metric\")\n\t\t\t\t} else {\n\t\t\t\t\tr.metrics.SetRestoreTotal(int64(len(restoreList.Items)))\n\t\t\t\t}\n\t\t\t},\n\t\t\t1*time.Minute,\n\t\t\tr.ctx.Done(),\n\t\t)\n\t}()\n}\n\n// deleteExternalResources deletes all the external resources related to the restore\nfunc (r *restoreReconciler) deleteExternalResources(restore *api.Restore) error {\n\tr.logger.Infof(\"Finalizer is deleting external resources, backup: %s\", restore.Spec.BackupName)\n\n\tif restore.Spec.BackupName == \"\" {\n\t\treturn nil\n\t}\n\n\tbackupInfo, err := r.fetchBackupInfo(restore.Spec.BackupName)\n\tif err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tr.logger.Errorf(\"got not found error: %v, skip deleting the restore files in object storage\", err)\n\t\t\treturn nil\n\t\t}\n\t\treturn errors.Wrap(err, fmt.Sprintf(\"can't get backup info, backup: %s\", restore.Spec.BackupName))\n\t}\n\n\tif !veleroutil.BSLIsAvailable(*backupInfo.location) {\n\t\treturn fmt.Errorf(\"bsl %s is unavailable, cannot get the backup info\", backupInfo.location.Name)\n\t}\n\n\t// delete restore files in object storage\n\tpluginManager := r.newPluginManager(r.logger)\n\tdefer pluginManager.CleanupClients()\n\n\tbackupStore, err := r.backupStoreGetter.Get(backupInfo.location, pluginManager, r.logger)\n\tif err != nil {\n\t\treturn errors.Wrap(err, fmt.Sprintf(\"can't get backupStore, backup: %s\", restore.Spec.BackupName))\n\t}\n\n\tif err = backupStore.DeleteRestore(restore.Name); err != nil {\n\t\treturn errors.Wrap(err, fmt.Sprintf(\"can't delete restore files in object storage, backup: %s\", restore.Spec.BackupName))\n\t}\n\n\treturn nil\n}\n\nfunc putResults(restore *api.Restore, results map[string]results.Result, backupStore persistence.BackupStore) error {\n\tbuf := new(bytes.Buffer)\n\tgzw := gzip.NewWriter(buf)\n\tdefer gzw.Close()\n\n\tif err := json.NewEncoder(gzw).Encode(results); err != nil {\n\t\treturn errors.Wrap(err, \"error encoding restore results to JSON\")\n\t}\n\n\tif err := gzw.Close(); err != nil {\n\t\treturn errors.Wrap(err, \"error closing gzip writer\")\n\t}\n\n\tif err := backupStore.PutRestoreResults(restore.Spec.BackupName, restore.Name, buf); err != nil {\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\nfunc putRestoredResourceList(restore *api.Restore, list map[string][]string, backupStore persistence.BackupStore) error {\n\tbuf := new(bytes.Buffer)\n\tgzw := gzip.NewWriter(buf)\n\tdefer gzw.Close()\n\n\tif err := json.NewEncoder(gzw).Encode(list); err != nil {\n\t\treturn errors.Wrap(err, \"error encoding restored resource list to JSON\")\n\t}\n\n\tif err := gzw.Close(); err != nil {\n\t\treturn errors.Wrap(err, \"error closing gzip writer\")\n\t}\n\n\tif err := backupStore.PutRestoredResourceList(restore.Name, buf); err != nil {\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\nfunc putOperationsForRestore(restore *api.Restore, operations []*itemoperation.RestoreOperation, backupStore persistence.BackupStore) error {\n\tbuf := new(bytes.Buffer)\n\tgzw := gzip.NewWriter(buf)\n\tdefer gzw.Close()\n\n\tif err := json.NewEncoder(gzw).Encode(operations); err != nil {\n\t\treturn errors.Wrap(err, \"error encoding restore item operations list to JSON\")\n\t}\n\n\tif err := gzw.Close(); err != nil {\n\t\treturn errors.Wrap(err, \"error closing gzip writer\")\n\t}\n\n\tif err := backupStore.PutRestoreItemOperations(restore.Name, buf); err != nil {\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\nfunc putRestoreVolumeInfoList(restore *api.Restore, volInfoList []*volume.RestoreVolumeInfo, store persistence.BackupStore) error {\n\tbuf := new(bytes.Buffer)\n\tgzw := gzip.NewWriter(buf)\n\tdefer gzw.Close()\n\n\tif err := json.NewEncoder(gzw).Encode(volInfoList); err != nil {\n\t\treturn errors.Wrap(err, \"error encoding restore volume info list to JSON\")\n\t}\n\n\tif err := gzw.Close(); err != nil {\n\t\treturn errors.Wrap(err, \"error closing gzip writer\")\n\t}\n\n\treturn store.PutRestoreVolumeInfo(restore.Name, buf)\n}\n\nfunc downloadToTempFile(backupName string, backupStore persistence.BackupStore, logger logrus.FieldLogger) (*os.File, error) {\n\treadCloser, err := backupStore.GetBackupContents(backupName)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdefer readCloser.Close()\n\n\tfile, err := os.CreateTemp(\"\", backupName)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error creating Backup temp file\")\n\t}\n\n\tn, err := io.Copy(file, readCloser)\n\tif err != nil {\n\t\t// Temporary file has been created if we go here. And some problems occurs such as network interruption and\n\t\t// so on. So we close and remove temporary file first to prevent residual file.\n\t\tcloseAndRemoveFile(file, logger)\n\t\treturn nil, errors.Wrap(err, \"error copying Backup to temp file\")\n\t}\n\n\tlog := logger.WithField(\"backup\", backupName)\n\n\tlog.WithFields(logrus.Fields{\n\t\t\"fileName\": file.Name(),\n\t\t\"bytes\": n,\n\t}).Debug(\"Copied Backup to file\")\n\n\tif _, err := file.Seek(0, 0); err != nil {\n\t\tcloseAndRemoveFile(file, logger)\n\t\treturn nil, errors.Wrap(err, \"error resetting Backup file offset\")\n\t}\n\n\treturn file, nil\n}\n" }, { "path": "pkg/controller/restore_controller_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"bytes\"\n\t\"io\"\n\t\"testing\"\n\t\"time\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\tclocktesting \"k8s.io/utils/clock/testing\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/resourcemodifiers\"\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\tpersistencemocks \"github.com/vmware-tanzu/velero/pkg/persistence/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework\"\n\tpluginmocks \"github.com/vmware-tanzu/velero/pkg/plugin/mocks\"\n\triav2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/restoreitemaction/v2\"\n\tpkgrestore \"github.com/vmware-tanzu/velero/pkg/restore\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/results\"\n)\n\nfunc TestFetchBackupInfo(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackupName string\n\t\tinformerLocations []*velerov1api.BackupStorageLocation\n\t\tinformerBackups []*velerov1api.Backup\n\t\tbackupStoreBackup *velerov1api.Backup\n\t\tbackupStoreError error\n\t\texpectedRes *velerov1api.Backup\n\t\texpectedErr bool\n\t}{\n\t\t{\n\t\t\tname: \"lister has backup\",\n\t\t\tbackupName: \"backup-1\",\n\t\t\tinformerLocations: []*velerov1api.BackupStorageLocation{builder.ForBackupStorageLocation(\"velero\", \"default\").Provider(\"myCloud\").Bucket(\"bucket\").Phase(velerov1api.BackupStorageLocationPhaseAvailable).Result()},\n\t\t\tinformerBackups: []*velerov1api.Backup{defaultBackup().StorageLocation(\"default\").Result()},\n\t\t\texpectedRes: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"lister does not have a backup, but backupSvc does\",\n\t\t\tbackupName: \"backup-1\",\n\t\t\tbackupStoreBackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\tinformerLocations: []*velerov1api.BackupStorageLocation{builder.ForBackupStorageLocation(\"velero\", \"default\").Provider(\"myCloud\").Bucket(\"bucket\").Phase(velerov1api.BackupStorageLocationPhaseAvailable).Result()},\n\t\t\tinformerBackups: []*velerov1api.Backup{defaultBackup().StorageLocation(\"default\").Result()},\n\t\t\texpectedRes: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"no backup\",\n\t\t\tbackupName: \"backup-1\",\n\t\t\tbackupStoreError: errors.New(\"no backup here\"),\n\t\t\texpectedErr: true,\n\t\t},\n\t}\n\n\tformatFlag := logging.FormatText\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\tfakeClient = velerotest.NewFakeControllerRuntimeClient(t)\n\t\t\t\tfakeGlobalClient = velerotest.NewFakeControllerRuntimeClient(t)\n\t\t\t\trestorer = &fakeRestorer{kbClient: fakeClient}\n\t\t\t\tlogger = velerotest.NewLogger()\n\t\t\t\tpluginManager = &pluginmocks.Manager{}\n\t\t\t\tbackupStore = &persistencemocks.BackupStore{}\n\t\t\t)\n\n\t\t\tdefer restorer.AssertExpectations(t)\n\t\t\tdefer backupStore.AssertExpectations(t)\n\n\t\t\tr := NewRestoreReconciler(\n\t\t\t\tt.Context(),\n\t\t\t\tvelerov1api.DefaultNamespace,\n\t\t\t\trestorer,\n\t\t\t\tfakeClient,\n\t\t\t\tlogger,\n\t\t\t\tlogrus.InfoLevel,\n\t\t\t\tfunc(logrus.FieldLogger) clientmgmt.Manager { return pluginManager },\n\t\t\t\tNewFakeSingleObjectBackupStoreGetter(backupStore),\n\t\t\t\tmetrics.NewServerMetrics(),\n\t\t\t\tformatFlag,\n\t\t\t\t60*time.Minute,\n\t\t\t\tfalse,\n\t\t\t\tfakeGlobalClient,\n\t\t\t\t10*time.Minute,\n\t\t\t)\n\n\t\t\tif test.backupStoreError == nil {\n\t\t\t\tfor _, itm := range test.informerLocations {\n\t\t\t\t\trequire.NoError(t, r.kbClient.Create(t.Context(), itm))\n\t\t\t\t}\n\n\t\t\t\tfor _, itm := range test.informerBackups {\n\t\t\t\t\trequire.NoError(t, r.kbClient.Create(t.Context(), itm))\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif test.backupStoreBackup != nil && test.backupStoreError != nil {\n\t\t\t\tpanic(\"developer error - only one of backupStoreBackup, backupStoreError can be non-nil\")\n\t\t\t}\n\n\t\t\tif test.backupStoreError != nil {\n\t\t\t\t// TODO why do I need .Maybe() here?\n\t\t\t\tbackupStore.On(\"GetBackupMetadata\", test.backupName).Return(nil, test.backupStoreError).Maybe()\n\t\t\t}\n\t\t\tif test.backupStoreBackup != nil {\n\t\t\t\t// TODO why do I need .Maybe() here?\n\t\t\t\tbackupStore.On(\"GetBackupMetadata\", test.backupName).Return(test.backupStoreBackup, nil).Maybe()\n\t\t\t}\n\n\t\t\tinfo, err := r.fetchBackupInfo(test.backupName)\n\n\t\t\trequire.Equal(t, test.expectedErr, err != nil)\n\t\t\tif test.expectedRes != nil {\n\t\t\t\tassert.Equal(t, test.expectedRes.Spec, info.backup.Spec)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestProcessQueueItemSkips(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tnamespace string\n\t\trestoreName string\n\t\trestore *velerov1api.Restore\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"missing restore returns nil\",\n\t\t\tnamespace: \"foo\",\n\t\t\trestoreName: \"bar\",\n\t\t\texpectError: false,\n\t\t},\n\t}\n\n\tformatFlag := logging.FormatText\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\tfakeClient = velerotest.NewFakeControllerRuntimeClient(t)\n\t\t\t\tfakeGlobalClient = velerotest.NewFakeControllerRuntimeClient(t)\n\t\t\t\trestorer = &fakeRestorer{kbClient: fakeClient}\n\t\t\t\tlogger = velerotest.NewLogger()\n\t\t\t)\n\n\t\t\tif test.restore != nil {\n\t\t\t\trequire.NoError(t, fakeClient.Create(t.Context(), test.restore))\n\t\t\t}\n\n\t\t\tr := NewRestoreReconciler(\n\t\t\t\tt.Context(),\n\t\t\t\tvelerov1api.DefaultNamespace,\n\t\t\t\trestorer,\n\t\t\t\tfakeClient,\n\t\t\t\tlogger,\n\t\t\t\tlogrus.InfoLevel,\n\t\t\t\tnil,\n\t\t\t\tnil, // backupStoreGetter\n\t\t\t\tmetrics.NewServerMetrics(),\n\t\t\t\tformatFlag,\n\t\t\t\t60*time.Minute,\n\t\t\t\tfalse,\n\t\t\t\tfakeGlobalClient,\n\t\t\t\t10*time.Minute,\n\t\t\t)\n\n\t\t\t_, err := r.Reconcile(t.Context(), ctrl.Request{NamespacedName: types.NamespacedName{\n\t\t\t\tNamespace: test.namespace,\n\t\t\t\tName: test.restoreName,\n\t\t\t}})\n\n\t\t\tassert.Equal(t, test.expectError, err != nil)\n\t\t})\n\t}\n}\n\nfunc TestRestoreReconcile(t *testing.T) {\n\tdefaultStorageLocation := builder.ForBackupStorageLocation(\"velero\", \"default\").Provider(\"myCloud\").Bucket(\"bucket\").Phase(velerov1api.BackupStorageLocationPhaseAvailable).Result()\n\n\tnow, err := time.Parse(time.RFC1123Z, time.RFC1123Z)\n\trequire.NoError(t, err)\n\tnow = now.Local()\n\ttimestamp := metav1.NewTime(now)\n\tassert.NotNil(t, timestamp)\n\n\ttests := []struct {\n\t\tname string\n\t\trestoreKey string\n\t\tlocation *velerov1api.BackupStorageLocation\n\t\trestore *velerov1api.Restore\n\t\tbackup *velerov1api.Backup\n\t\trestorerError error\n\t\texpectedErr bool\n\t\texpectedPhase string\n\t\texpectedStartTime *metav1.Time\n\t\texpectedCompletedTime *metav1.Time\n\t\texpectedValidationErrors []string\n\t\texpectedRestoreErrors int\n\t\texpectedRestorerCall *velerov1api.Restore\n\t\tbackupStoreGetBackupMetadataErr error\n\t\tbackupStoreGetBackupContentsErr error\n\t\tputRestoreLogErr error\n\t\texpectedFinalPhase string\n\t\taddValidFinalizer bool\n\t\temptyVolumeInfo bool\n\t}{\n\t\t{\n\t\t\tname: \"restore with both namespace in both includedNamespaces and excludedNamespaces fails validation\",\n\t\t\tlocation: defaultStorageLocation,\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"backup-1\", \"another-1\", \"*\", velerov1api.RestorePhaseNew).ExcludedNamespaces(\"another-1\").Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\texpectedErr: false,\n\t\t\texpectedPhase: string(velerov1api.RestorePhaseFailedValidation),\n\t\t\texpectedValidationErrors: []string{\"Invalid included/excluded namespace lists: excludes list cannot contain an item in the includes list: another-1\"},\n\t\t},\n\t\t{\n\t\t\tname: \"restore with resource in both includedResources and excludedResources fails validation\",\n\t\t\tlocation: defaultStorageLocation,\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"backup-1\", \"*\", \"a-resource\", velerov1api.RestorePhaseNew).ExcludedResources(\"a-resource\").Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\texpectedErr: false,\n\t\t\texpectedPhase: string(velerov1api.RestorePhaseFailedValidation),\n\t\t\texpectedValidationErrors: []string{\"Invalid included/excluded resource lists: excludes list cannot contain an item in the includes list: a-resource\"},\n\t\t},\n\t\t{\n\t\t\tname: \"new restore with empty backup and schedule names fails validation\",\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"\", \"ns-1\", \"\", velerov1api.RestorePhaseNew).Result(),\n\t\t\texpectedErr: false,\n\t\t\texpectedPhase: string(velerov1api.RestorePhaseFailedValidation),\n\t\t\texpectedValidationErrors: []string{\"Either a backup or schedule must be specified as a source for the restore, but not both\"},\n\t\t},\n\t\t{\n\t\t\tname: \"new restore with backup and schedule names provided fails validation\",\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"\", velerov1api.RestorePhaseNew).Schedule(\"sched-1\").Result(),\n\t\t\texpectedErr: false,\n\t\t\texpectedPhase: string(velerov1api.RestorePhaseFailedValidation),\n\t\t\texpectedValidationErrors: []string{\"Either a backup or schedule must be specified as a source for the restore, but not both\"},\n\t\t},\n\t\t{\n\t\t\tname: \"new restore with labelSelector as well as orLabelSelector fails validation\",\n\t\t\tlocation: defaultStorageLocation,\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"\", velerov1api.RestorePhaseNew).LabelSelector(&metav1.LabelSelector{MatchLabels: map[string]string{\"a\": \"b\"}}).OrLabelSelector([]*metav1.LabelSelector{{MatchLabels: map[string]string{\"a1\": \"b1\"}}, {MatchLabels: map[string]string{\"a2\": \"b2\"}}, {MatchLabels: map[string]string{\"a3\": \"b3\"}}, {MatchLabels: map[string]string{\"a4\": \"b4\"}}}).Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\texpectedErr: false,\n\t\t\texpectedValidationErrors: []string{\"encountered labelSelector as well as orLabelSelectors in restore spec, only one can be specified\"},\n\t\t\texpectedPhase: string(velerov1api.RestorePhaseFailedValidation),\n\t\t},\n\t\t{\n\t\t\tname: \"valid restore with schedule name gets executed\",\n\t\t\tlocation: defaultStorageLocation,\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"\", \"ns-1\", \"\", velerov1api.RestorePhaseNew).Schedule(\"sched-1\").Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").ObjectMeta(builder.WithLabels(velerov1api.ScheduleNameLabel, \"sched-1\")).Phase(velerov1api.BackupPhaseCompleted).Result(),\n\t\t\texpectedErr: false,\n\t\t\texpectedPhase: string(velerov1api.RestorePhaseInProgress),\n\t\t\texpectedStartTime: ×tamp,\n\t\t\texpectedCompletedTime: ×tamp,\n\t\t\texpectedRestorerCall: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"\", velerov1api.RestorePhaseInProgress).Schedule(\"sched-1\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"restore with non-existent backup name fails\",\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"*\", velerov1api.RestorePhaseNew).Result(),\n\t\t\texpectedErr: false,\n\t\t\texpectedPhase: string(velerov1api.RestorePhaseFailedValidation),\n\t\t\texpectedValidationErrors: []string{\"Error retrieving backup: backup.velero.io \\\"backup-1\\\" not found\"},\n\t\t\tbackupStoreGetBackupMetadataErr: errors.New(\"no backup here\"),\n\t\t},\n\t\t{\n\t\t\tname: \"restorer throwing an error causes the restore to fail\",\n\t\t\tlocation: defaultStorageLocation,\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"\", velerov1api.RestorePhaseNew).Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\trestorerError: errors.New(\"blarg\"),\n\t\t\texpectedErr: false,\n\t\t\texpectedPhase: string(velerov1api.RestorePhaseInProgress),\n\t\t\texpectedFinalPhase: string(velerov1api.RestorePhasePartiallyFailed),\n\t\t\texpectedStartTime: ×tamp,\n\t\t\texpectedCompletedTime: ×tamp,\n\t\t\texpectedRestoreErrors: 1,\n\t\t\texpectedRestorerCall: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"\", velerov1api.RestorePhaseInProgress).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"valid restore with none existingresourcepolicy gets executed\",\n\t\t\tlocation: defaultStorageLocation,\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"\", velerov1api.RestorePhaseNew).ExistingResourcePolicy(\"none\").Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\texpectedErr: false,\n\t\t\texpectedPhase: string(velerov1api.RestorePhaseInProgress),\n\t\t\texpectedStartTime: ×tamp,\n\t\t\texpectedCompletedTime: ×tamp,\n\t\t\texpectedRestorerCall: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"\", velerov1api.RestorePhaseInProgress).ExistingResourcePolicy(\"none\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"valid restore with update existingresourcepolicy gets executed\",\n\t\t\tlocation: defaultStorageLocation,\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"\", velerov1api.RestorePhaseNew).ExistingResourcePolicy(\"update\").Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\texpectedErr: false,\n\t\t\texpectedPhase: string(velerov1api.RestorePhaseInProgress),\n\t\t\texpectedStartTime: ×tamp,\n\t\t\texpectedCompletedTime: ×tamp,\n\t\t\texpectedRestorerCall: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"\", velerov1api.RestorePhaseInProgress).ExistingResourcePolicy(\"update\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"invalid restore with invalid existingresourcepolicy errors\",\n\t\t\tlocation: defaultStorageLocation,\n\t\t\trestore: NewRestore(\"foo\", \"invalidexistingresourcepolicy\", \"backup-1\", \"ns-1\", \"\", velerov1api.RestorePhaseNew).ExistingResourcePolicy(\"invalid\").Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\texpectedErr: false,\n\t\t\texpectedPhase: string(velerov1api.RestorePhaseFailedValidation),\n\t\t\texpectedStartTime: ×tamp,\n\t\t\texpectedCompletedTime: ×tamp,\n\t\t\texpectedRestorerCall: nil, // this restore should fail validation and not be passed to the restorer\n\t\t},\n\t\t{\n\t\t\tname: \"valid restore gets executed\",\n\t\t\tlocation: defaultStorageLocation,\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"\", velerov1api.RestorePhaseNew).Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\texpectedErr: false,\n\t\t\texpectedPhase: string(velerov1api.RestorePhaseInProgress),\n\t\t\texpectedStartTime: ×tamp,\n\t\t\texpectedCompletedTime: ×tamp,\n\t\t\texpectedRestorerCall: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"\", velerov1api.RestorePhaseInProgress).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"restoration of nodes is not supported\",\n\t\t\tlocation: defaultStorageLocation,\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"nodes\", velerov1api.RestorePhaseNew).Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\texpectedErr: false,\n\t\t\texpectedPhase: string(velerov1api.RestorePhaseFailedValidation),\n\t\t\texpectedValidationErrors: []string{\n\t\t\t\t\"nodes are non-restorable resources\",\n\t\t\t\t\"Invalid included/excluded resource lists: excludes list cannot contain an item in the includes list: nodes\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"restoration of events is not supported\",\n\t\t\tlocation: defaultStorageLocation,\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"events\", velerov1api.RestorePhaseNew).Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\texpectedErr: false,\n\t\t\texpectedPhase: string(velerov1api.RestorePhaseFailedValidation),\n\t\t\texpectedValidationErrors: []string{\n\t\t\t\t\"events are non-restorable resources\",\n\t\t\t\t\"Invalid included/excluded resource lists: excludes list cannot contain an item in the includes list: events\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"restoration of events.events.k8s.io is not supported\",\n\t\t\tlocation: defaultStorageLocation,\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"events.events.k8s.io\", velerov1api.RestorePhaseNew).Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\texpectedErr: false,\n\t\t\texpectedPhase: string(velerov1api.RestorePhaseFailedValidation),\n\t\t\texpectedValidationErrors: []string{\n\t\t\t\t\"events.events.k8s.io are non-restorable resources\",\n\t\t\t\t\"Invalid included/excluded resource lists: excludes list cannot contain an item in the includes list: events.events.k8s.io\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"restoration of backups.velero.io is not supported\",\n\t\t\tlocation: defaultStorageLocation,\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"backups.velero.io\", velerov1api.RestorePhaseNew).Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\texpectedErr: false,\n\t\t\texpectedPhase: string(velerov1api.RestorePhaseFailedValidation),\n\t\t\texpectedValidationErrors: []string{\n\t\t\t\t\"backups.velero.io are non-restorable resources\",\n\t\t\t\t\"Invalid included/excluded resource lists: excludes list cannot contain an item in the includes list: backups.velero.io\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"restoration of restores.velero.io is not supported\",\n\t\t\tlocation: defaultStorageLocation,\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"restores.velero.io\", velerov1api.RestorePhaseNew).Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\texpectedErr: false,\n\t\t\texpectedPhase: string(velerov1api.RestorePhaseFailedValidation),\n\t\t\texpectedValidationErrors: []string{\n\t\t\t\t\"restores.velero.io are non-restorable resources\",\n\t\t\t\t\"Invalid included/excluded resource lists: excludes list cannot contain an item in the includes list: restores.velero.io\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"backup download error results in failed restore\",\n\t\t\tlocation: defaultStorageLocation,\n\t\t\trestore: NewRestore(velerov1api.DefaultNamespace, \"bar\", \"backup-1\", \"ns-1\", \"\", velerov1api.RestorePhaseNew).Result(),\n\t\t\texpectedPhase: string(velerov1api.RestorePhaseInProgress),\n\t\t\texpectedFinalPhase: string(velerov1api.RestorePhaseFailed),\n\t\t\texpectedStartTime: ×tamp,\n\t\t\texpectedCompletedTime: ×tamp,\n\t\t\tbackupStoreGetBackupContentsErr: errors.New(\"Couldn't download backup\"),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"restore attached with an expected finalizer gets cleaned up successfully\",\n\t\t\tlocation: defaultStorageLocation,\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"\", velerov1api.RestorePhaseCompleted).ObjectMeta(builder.WithFinalizers(ExternalResourcesFinalizer), builder.WithDeletionTimestamp(timestamp.Time)).Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\texpectedErr: false,\n\t\t\taddValidFinalizer: true,\n\t\t},\n\t\t{\n\t\t\tname: \"restore attached with an unknown finalizer will be skipped\",\n\t\t\tlocation: defaultStorageLocation,\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"\", velerov1api.RestorePhaseCompleted).ObjectMeta(builder.WithFinalizers(\"restores.velero.io/unknown-finalizer\"), builder.WithDeletionTimestamp(timestamp.Time)).Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\texpectedErr: false,\n\t\t\taddValidFinalizer: false,\n\t\t},\n\t\t{\n\t\t\tname: \"completed restore will be skipped\",\n\t\t\tlocation: defaultStorageLocation,\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"\", velerov1api.RestorePhaseCompleted).Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"valid restore with empty VolumeInfos\",\n\t\t\tlocation: defaultStorageLocation,\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"\", velerov1api.RestorePhaseNew).Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\temptyVolumeInfo: true,\n\t\t\texpectedErr: false,\n\t\t\texpectedPhase: string(velerov1api.RestorePhaseInProgress),\n\t\t\texpectedStartTime: ×tamp,\n\t\t\texpectedCompletedTime: ×tamp,\n\t\t\texpectedRestorerCall: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"\", velerov1api.RestorePhaseInProgress).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Restore creation is rejected when BSL is unavailable\",\n\t\t\tlocation: builder.ForBackupStorageLocation(\"velero\", \"default\").Provider(\"myCloud\").Bucket(\"bucket\").Phase(velerov1api.BackupStorageLocationPhaseUnavailable).Result(),\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"\", velerov1api.RestorePhaseNew).Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\texpectedErr: false,\n\t\t\texpectedPhase: string(velerov1api.RestorePhaseNew),\n\t\t\texpectedValidationErrors: []string{\"The BSL default is unavailable, cannot retrieve the backup\"},\n\t\t},\n\t\t{\n\t\t\tname: \"Restore deletion is rejected when BSL is unavailable.\",\n\t\t\tlocation: builder.ForBackupStorageLocation(\"velero\", \"default\").Provider(\"myCloud\").Bucket(\"bucket\").Phase(velerov1api.BackupStorageLocationPhaseUnavailable).Result(),\n\t\t\trestore: NewRestore(\"foo\", \"bar\", \"backup-1\", \"ns-1\", \"\", velerov1api.RestorePhaseCompleted).ObjectMeta(builder.WithFinalizers(ExternalResourcesFinalizer), builder.WithDeletionTimestamp(timestamp.Time)).Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\texpectedErr: true,\n\t\t},\n\t}\n\n\tformatFlag := logging.FormatText\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\tfakeClient = velerotest.NewFakeControllerRuntimeClientBuilder(t).Build()\n\t\t\t\tfakeGlobalClient = velerotest.NewFakeControllerRuntimeClient(t)\n\t\t\t\trestorer = &fakeRestorer{kbClient: fakeClient}\n\t\t\t\tlogger = velerotest.NewLogger()\n\t\t\t\tpluginManager = &pluginmocks.Manager{}\n\t\t\t\tbackupStore = &persistencemocks.BackupStore{}\n\t\t\t)\n\n\t\t\tdefer restorer.AssertExpectations(t)\n\t\t\tdefer backupStore.AssertExpectations(t)\n\t\t\tdefer func() {\n\t\t\t\t// reset defaultStorageLocation resourceVersion\n\t\t\t\tdefaultStorageLocation.ObjectMeta.ResourceVersion = \"\"\n\t\t\t}()\n\n\t\t\tr := NewRestoreReconciler(\n\t\t\t\tt.Context(),\n\t\t\t\tvelerov1api.DefaultNamespace,\n\t\t\t\trestorer,\n\t\t\t\tfakeClient,\n\t\t\t\tlogger,\n\t\t\t\tlogrus.InfoLevel,\n\t\t\t\tfunc(logrus.FieldLogger) clientmgmt.Manager { return pluginManager },\n\t\t\t\tNewFakeSingleObjectBackupStoreGetter(backupStore),\n\t\t\t\tmetrics.NewServerMetrics(),\n\t\t\t\tformatFlag,\n\t\t\t\t60*time.Minute,\n\t\t\t\tfalse,\n\t\t\t\tfakeGlobalClient,\n\t\t\t\t10*time.Minute,\n\t\t\t)\n\n\t\t\tr.clock = clocktesting.NewFakeClock(now)\n\t\t\tif test.location != nil {\n\t\t\t\trequire.NoError(t, r.kbClient.Create(t.Context(), test.location))\n\t\t\t}\n\t\t\tif test.backup != nil {\n\t\t\t\trequire.NoError(t, r.kbClient.Create(t.Context(), test.backup))\n\t\t\t}\n\n\t\t\tif test.restore != nil {\n\t\t\t\tisDeletionTimestampSet := test.restore.DeletionTimestamp != nil\n\t\t\t\trequire.NoError(t, r.kbClient.Create(t.Context(), test.restore))\n\t\t\t\t// because of the changes introduced by https://github.com/kubernetes-sigs/controller-runtime/commit/7a66d580c0c53504f5b509b45e9300cc18a1cc30\n\t\t\t\t// the fake client ignores the DeletionTimestamp when calling the Create(),\n\t\t\t\t// so call Delete() here\n\t\t\t\tif isDeletionTimestampSet {\n\t\t\t\t\terr = r.kbClient.Delete(ctx, test.restore)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tvar warnings, errors results.Result\n\t\t\tif test.restorerError != nil {\n\t\t\t\terrors.Namespaces = map[string][]string{\"ns-1\": {test.restorerError.Error()}}\n\t\t\t}\n\t\t\tif test.putRestoreLogErr != nil {\n\t\t\t\terrors.Velero = append(errors.Velero, \"error uploading log file to object storage: \"+test.putRestoreLogErr.Error())\n\t\t\t}\n\t\t\tif test.expectedRestorerCall != nil {\n\t\t\t\tbackupStore.On(\"GetBackupContents\", test.backup.Name).Return(io.NopCloser(bytes.NewReader([]byte(\"hello world\"))), nil)\n\t\t\t\tbackupStore.On(\"GetCSIVolumeSnapshots\", test.backup.Name).Return([]*snapshotv1api.VolumeSnapshot{}, nil)\n\n\t\t\t\trestorer.On(\"RestoreWithResolvers\", mock.Anything, mock.Anything, mock.Anything, mock.Anything,\n\t\t\t\t\tmock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(warnings, errors)\n\n\t\t\t\tbackupStore.On(\"PutRestoreLog\", test.backup.Name, test.restore.Name, mock.Anything).Return(test.putRestoreLogErr)\n\n\t\t\t\tbackupStore.On(\"PutRestoreResults\", test.backup.Name, test.restore.Name, mock.Anything).Return(nil)\n\t\t\t\tbackupStore.On(\"PutRestoredResourceList\", test.restore.Name, mock.Anything).Return(nil)\n\t\t\t\tbackupStore.On(\"PutRestoreItemOperations\", mock.Anything, mock.Anything).Return(nil)\n\t\t\t\tbackupStore.On(\"PutRestoreVolumeInfo\", test.restore.Name, mock.Anything).Return(nil)\n\t\t\t\tif test.emptyVolumeInfo == true {\n\t\t\t\t\tbackupStore.On(\"GetBackupVolumeInfos\", test.backup.Name).Return(nil, nil)\n\t\t\t\t} else {\n\t\t\t\t\tbackupStore.On(\"GetBackupVolumeInfos\", test.backup.Name).Return([]*volume.BackupVolumeInfo{}, nil)\n\t\t\t\t}\n\n\t\t\t\tvolumeSnapshots := []*volume.Snapshot{\n\t\t\t\t\t{\n\t\t\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\t\t\tPersistentVolumeName: \"test-pv\",\n\t\t\t\t\t\t\tBackupName: test.backup.Name,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t}\n\t\t\t\tbackupStore.On(\"GetBackupVolumeSnapshots\", test.backup.Name).Return(volumeSnapshots, nil)\n\t\t\t}\n\n\t\t\tif test.backupStoreGetBackupMetadataErr != nil {\n\t\t\t\t// TODO why do I need .Maybe() here?\n\t\t\t\tbackupStore.On(\"GetBackupMetadata\", test.restore.Spec.BackupName).Return(nil, test.backupStoreGetBackupMetadataErr).Maybe()\n\t\t\t}\n\n\t\t\tif test.backupStoreGetBackupContentsErr != nil {\n\t\t\t\t// TODO why do I need .Maybe() here?\n\t\t\t\tbackupStore.On(\"GetBackupContents\", test.restore.Spec.BackupName).Return(nil, test.backupStoreGetBackupContentsErr).Maybe()\n\t\t\t}\n\n\t\t\tif test.restore != nil {\n\t\t\t\tpluginManager.On(\"GetRestoreItemActionsV2\").Return(nil, nil)\n\t\t\t\tpluginManager.On(\"CleanupClients\")\n\t\t\t}\n\n\t\t\tif test.addValidFinalizer {\n\t\t\t\tbackupStore.On(\"DeleteRestore\", test.restore.Name).Return(nil)\n\t\t\t}\n\n\t\t\t//err = r.processQueueItem(key)\n\t\t\t_, err = r.Reconcile(t.Context(), ctrl.Request{NamespacedName: types.NamespacedName{\n\t\t\t\tNamespace: test.restore.Namespace,\n\t\t\t\tName: test.restore.Name,\n\t\t\t}})\n\n\t\t\tassert.Equal(t, test.expectedErr, err != nil, \"got error %v\", err)\n\n\t\t\tif test.expectedPhase == \"\" {\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\t// struct and func for decoding patch content\n\t\t\ttype SpecPatch struct {\n\t\t\t\tBackupName string `json:\"backupName\"`\n\t\t\t}\n\n\t\t\ttype StatusPatch struct {\n\t\t\t\tPhase velerov1api.RestorePhase `json:\"phase\"`\n\t\t\t\tValidationErrors []string `json:\"validationErrors\"`\n\t\t\t\tErrors int `json:\"errors\"`\n\t\t\t\tStartTimestamp *metav1.Time `json:\"startTimestamp\"`\n\t\t\t\tCompletionTimestamp *metav1.Time `json:\"completionTimestamp\"`\n\t\t\t}\n\n\t\t\ttype Patch struct {\n\t\t\t\tSpec SpecPatch `json:\"spec,omitempty\"`\n\t\t\t\tStatus StatusPatch `json:\"status\"`\n\t\t\t}\n\n\t\t\texpected := Patch{\n\t\t\t\tStatus: StatusPatch{\n\t\t\t\t\tPhase: velerov1api.RestorePhase(test.expectedPhase),\n\t\t\t\t\tValidationErrors: test.expectedValidationErrors,\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tif test.expectedStartTime != nil {\n\t\t\t\texpected.Status.StartTimestamp = test.expectedStartTime\n\t\t\t}\n\n\t\t\t// if we don't expect a restore, validate it wasn't called and exit the test\n\t\t\tif test.expectedRestorerCall == nil {\n\t\t\t\tassert.Empty(t, restorer.Calls)\n\t\t\t\tassert.Zero(t, restorer.calledWithArg)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif !test.addValidFinalizer {\n\t\t\t\tassert.Len(t, restorer.Calls, 1)\n\t\t\t}\n\n\t\t\t// validate Patch call 2 (setting phase)\n\n\t\t\texpected = Patch{\n\t\t\t\tStatus: StatusPatch{\n\t\t\t\t\tPhase: velerov1api.RestorePhaseCompleted,\n\t\t\t\t\tErrors: test.expectedRestoreErrors,\n\t\t\t\t\tCompletionTimestamp: test.expectedCompletedTime,\n\t\t\t\t},\n\t\t\t}\n\t\t\t// Override our default expectations if the case requires it\n\t\t\tif test.expectedFinalPhase != \"\" {\n\t\t\t\texpected = Patch{\n\t\t\t\t\tStatus: StatusPatch{\n\t\t\t\t\t\tPhase: velerov1api.RestorePhase(test.expectedFinalPhase),\n\t\t\t\t\t\tErrors: test.expectedRestoreErrors,\n\t\t\t\t\t\tCompletionTimestamp: test.expectedCompletedTime,\n\t\t\t\t\t},\n\t\t\t\t}\n\t\t\t}\n\n\t\t\t// explicitly capturing the argument passed to Restore myself because\n\t\t\t// I want to validate the called arg as of the time of calling, but\n\t\t\t// the mock stores the pointer, which gets modified after\n\t\t\tassert.Equal(t, test.expectedRestorerCall.Spec, restorer.calledWithArg.Spec)\n\t\t\tassert.Equal(t, test.expectedRestorerCall.Status.Phase, restorer.calledWithArg.Status.Phase)\n\t\t})\n\t}\n}\n\nfunc TestValidateAndCompleteWhenScheduleNameSpecified(t *testing.T) {\n\tformatFlag := logging.FormatText\n\n\tvar (\n\t\tlogger = velerotest.NewLogger()\n\t\tpluginManager = &pluginmocks.Manager{}\n\t\tfakeClient = velerotest.NewFakeControllerRuntimeClient(t)\n\t\tfakeGlobalClient = velerotest.NewFakeControllerRuntimeClient(t)\n\t\tbackupStore = &persistencemocks.BackupStore{}\n\t)\n\n\tr := NewRestoreReconciler(\n\t\tt.Context(),\n\t\tvelerov1api.DefaultNamespace,\n\t\tnil,\n\t\tfakeClient,\n\t\tlogger,\n\t\tlogrus.DebugLevel,\n\t\tfunc(logrus.FieldLogger) clientmgmt.Manager { return pluginManager },\n\t\tNewFakeSingleObjectBackupStoreGetter(backupStore),\n\t\tmetrics.NewServerMetrics(),\n\t\tformatFlag,\n\t\t60*time.Minute,\n\t\tfalse,\n\t\tfakeGlobalClient,\n\t\t10*time.Minute,\n\t)\n\n\trestore := &velerov1api.Restore{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: \"restore-1\",\n\t\t},\n\t\tSpec: velerov1api.RestoreSpec{\n\t\t\tScheduleName: \"schedule-1\",\n\t\t},\n\t}\n\n\t// no backups created from the schedule: fail validation\n\trequire.NoError(t, r.kbClient.Create(t.Context(), defaultBackup().\n\t\tObjectMeta(builder.WithLabels(velerov1api.ScheduleNameLabel, \"non-matching-schedule\")).\n\t\tPhase(velerov1api.BackupPhaseCompleted).\n\t\tResult()))\n\n\tr.validateAndComplete(restore)\n\tassert.Contains(t, restore.Status.ValidationErrors, \"No backups found for schedule\")\n\tassert.Empty(t, restore.Spec.BackupName)\n\n\t// no completed backups created from the schedule: fail validation\n\trequire.NoError(t, r.kbClient.Create(\n\t\tt.Context(),\n\t\tdefaultBackup().\n\t\t\tObjectMeta(\n\t\t\t\tbuilder.WithName(\"backup-2\"),\n\t\t\t\tbuilder.WithLabels(velerov1api.ScheduleNameLabel, \"schedule-1\"),\n\t\t\t).\n\t\t\tPhase(velerov1api.BackupPhaseInProgress).\n\t\t\tResult(),\n\t))\n\n\tr.validateAndComplete(restore)\n\tassert.Contains(t, restore.Status.ValidationErrors, \"No completed backups found for schedule\")\n\tassert.Empty(t, restore.Spec.BackupName)\n\n\t// multiple completed backups created from the schedule: use most recent\n\tnow := time.Now()\n\n\trequire.NoError(t, r.kbClient.Create(t.Context(),\n\t\tdefaultBackup().\n\t\t\tObjectMeta(\n\t\t\t\tbuilder.WithName(\"foo\"),\n\t\t\t\tbuilder.WithLabels(velerov1api.ScheduleNameLabel, \"schedule-1\"),\n\t\t\t).\n\t\t\tStorageLocation(\"default\").\n\t\t\tPhase(velerov1api.BackupPhaseCompleted).\n\t\t\tStartTimestamp(now).\n\t\t\tResult(),\n\t))\n\n\tlocation := builder.ForBackupStorageLocation(\"velero\", \"default\").Provider(\"myCloud\").Bucket(\"bucket\").Phase(velerov1api.BackupStorageLocationPhaseAvailable).Result()\n\trequire.NoError(t, r.kbClient.Create(t.Context(), location))\n\n\trestore = &velerov1api.Restore{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: \"restore-1\",\n\t\t},\n\t\tSpec: velerov1api.RestoreSpec{\n\t\t\tScheduleName: \"schedule-1\",\n\t\t},\n\t}\n\tr.validateAndComplete(restore)\n\tassert.Nil(t, restore.Status.ValidationErrors)\n\tassert.Equal(t, \"foo\", restore.Spec.BackupName)\n}\n\nfunc TestValidateAndCompleteWithResourceModifierSpecified(t *testing.T) {\n\tformatFlag := logging.FormatText\n\n\tvar (\n\t\tlogger = velerotest.NewLogger()\n\t\tpluginManager = &pluginmocks.Manager{}\n\t\tfakeClient = velerotest.NewFakeControllerRuntimeClient(t)\n\t\tfakeGlobalClient = velerotest.NewFakeControllerRuntimeClient(t)\n\t\tbackupStore = &persistencemocks.BackupStore{}\n\t)\n\n\tr := NewRestoreReconciler(\n\t\tt.Context(),\n\t\tvelerov1api.DefaultNamespace,\n\t\tnil,\n\t\tfakeClient,\n\t\tlogger,\n\t\tlogrus.DebugLevel,\n\t\tfunc(logrus.FieldLogger) clientmgmt.Manager { return pluginManager },\n\t\tNewFakeSingleObjectBackupStoreGetter(backupStore),\n\t\tmetrics.NewServerMetrics(),\n\t\tformatFlag,\n\t\t60*time.Minute,\n\t\tfalse,\n\t\tfakeGlobalClient,\n\t\t10*time.Minute,\n\t)\n\n\trestore := &velerov1api.Restore{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: \"restore-1\",\n\t\t},\n\t\tSpec: velerov1api.RestoreSpec{\n\t\t\tBackupName: \"backup-1\",\n\t\t\tResourceModifier: &corev1api.TypedLocalObjectReference{\n\t\t\t\tKind: resourcemodifiers.ConfigmapRefType,\n\t\t\t\tName: \"test-configmap\",\n\t\t\t},\n\t\t},\n\t}\n\n\tlocation := builder.ForBackupStorageLocation(\"velero\", \"default\").Provider(\"myCloud\").Bucket(\"bucket\").Phase(velerov1api.BackupStorageLocationPhaseAvailable).Result()\n\trequire.NoError(t, r.kbClient.Create(t.Context(), location))\n\n\trequire.NoError(t, r.kbClient.Create(\n\t\tt.Context(),\n\t\tdefaultBackup().\n\t\t\tObjectMeta(\n\t\t\t\tbuilder.WithName(\"backup-1\"),\n\t\t\t).StorageLocation(\"default\").\n\t\t\tPhase(velerov1api.BackupPhaseCompleted).\n\t\t\tResult(),\n\t))\n\n\tr.validateAndComplete(restore)\n\tassert.Contains(t, restore.Status.ValidationErrors[0], \"failed to get resource modifiers configmap\")\n\n\trestore1 := &velerov1api.Restore{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: \"restore-1\",\n\t\t},\n\t\tSpec: velerov1api.RestoreSpec{\n\t\t\tBackupName: \"backup-1\",\n\t\t\tResourceModifier: &corev1api.TypedLocalObjectReference{\n\t\t\t\tKind: resourcemodifiers.ConfigmapRefType,\n\t\t\t\tName: \"test-configmap\",\n\t\t\t},\n\t\t},\n\t}\n\n\tcm1 := &corev1api.ConfigMap{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-configmap\",\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t},\n\t\tData: map[string]string{\n\t\t\t\"sub.yml\": \"version: v1\\nresourceModifierRules:\\n- conditions:\\n groupResource: persistentvolumeclaims\\n resourceNameRegex: \\\".*\\\"\\n namespaces:\\n - bar\\n - foo\\n patches:\\n - operation: replace\\n path: \\\"/spec/storageClassName\\\"\\n value: \\\"premium\\\"\\n - operation: remove\\n path: \\\"/metadata/labels/test\\\"\\n\\n\\n\",\n\t\t},\n\t}\n\trequire.NoError(t, r.kbClient.Create(t.Context(), cm1))\n\n\tr.validateAndComplete(restore1)\n\tassert.Nil(t, restore1.Status.ValidationErrors)\n\n\trestore2 := &velerov1api.Restore{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: \"restore-1\",\n\t\t},\n\t\tSpec: velerov1api.RestoreSpec{\n\t\t\tBackupName: \"backup-1\",\n\t\t\tResourceModifier: &corev1api.TypedLocalObjectReference{\n\t\t\t\t// intentional to ensure case insensitivity works as expected\n\t\t\t\tKind: \"confIGMaP\",\n\t\t\t\tName: \"test-configmap-invalid\",\n\t\t\t},\n\t\t},\n\t}\n\n\tinvalidVersionCm := &corev1api.ConfigMap{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-configmap-invalid\",\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t},\n\t\tData: map[string]string{\n\t\t\t\"sub.yml\": \"version1: v1\\nresourceModifierRules:\\n- conditions:\\n groupResource: persistentvolumeclaims\\n resourceNameRegex: \\\".*\\\"\\n namespaces:\\n - bar\\n - foo\\n patches:\\n - operation: replace\\n path: \\\"/spec/storageClassName\\\"\\n value: \\\"premium\\\"\\n - operation: remove\\n path: \\\"/metadata/labels/test\\\"\\n\\n\\n\",\n\t\t},\n\t}\n\trequire.NoError(t, r.kbClient.Create(t.Context(), invalidVersionCm))\n\n\tr.validateAndComplete(restore2)\n\tassert.Contains(t, restore2.Status.ValidationErrors[0], \"Error in parsing resource modifiers provided in configmap\")\n\n\trestore3 := &velerov1api.Restore{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: \"restore-1\",\n\t\t},\n\t\tSpec: velerov1api.RestoreSpec{\n\t\t\tBackupName: \"backup-1\",\n\t\t\tResourceModifier: &corev1api.TypedLocalObjectReference{\n\t\t\t\tKind: resourcemodifiers.ConfigmapRefType,\n\t\t\t\tName: \"test-configmap-invalid-operator\",\n\t\t\t},\n\t\t},\n\t}\n\n\tinvalidOperatorCm := &corev1api.ConfigMap{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-configmap-invalid-operator\",\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t},\n\t\tData: map[string]string{\n\t\t\t\"sub.yml\": \"version: v1\\nresourceModifierRules:\\n- conditions:\\n groupResource: persistentvolumeclaims\\n resourceNameRegex: \\\".*\\\"\\n namespaces:\\n - bar\\n - foo\\n patches:\\n - operation: invalid\\n path: \\\"/spec/storageClassName\\\"\\n value: \\\"premium\\\"\\n - operation: remove\\n path: \\\"/metadata/labels/test\\\"\\n\\n\\n\",\n\t\t},\n\t}\n\trequire.NoError(t, r.kbClient.Create(t.Context(), invalidOperatorCm))\n\n\tr.validateAndComplete(restore3)\n\tassert.Contains(t, restore3.Status.ValidationErrors[0], \"Validation error in resource modifiers provided in configmap\")\n}\n\nfunc TestBackupXorScheduleProvided(t *testing.T) {\n\tr := &velerov1api.Restore{}\n\tassert.False(t, backupXorScheduleProvided(r))\n\n\tr.Spec.BackupName = \"backup-1\"\n\tr.Spec.ScheduleName = \"schedule-1\"\n\tassert.False(t, backupXorScheduleProvided(r))\n\n\tr.Spec.BackupName = \"backup-1\"\n\tr.Spec.ScheduleName = \"\"\n\tassert.True(t, backupXorScheduleProvided(r))\n\n\tr.Spec.BackupName = \"\"\n\tr.Spec.ScheduleName = \"schedule-1\"\n\tassert.True(t, backupXorScheduleProvided(r))\n}\n\nfunc TestMostRecentCompletedBackup(t *testing.T) {\n\tbackups := []velerov1api.Backup{\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"a\",\n\t\t\t},\n\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\tPhase: \"\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"b\",\n\t\t\t},\n\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\tPhase: velerov1api.BackupPhaseNew,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"c\",\n\t\t\t},\n\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\tPhase: velerov1api.BackupPhaseInProgress,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"d\",\n\t\t\t},\n\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\tPhase: velerov1api.BackupPhaseFailedValidation,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"e\",\n\t\t\t},\n\t\t\tStatus: velerov1api.BackupStatus{\n\t\t\t\tPhase: velerov1api.BackupPhaseFailed,\n\t\t\t},\n\t\t},\n\t}\n\n\tassert.Empty(t, mostRecentCompletedBackup(backups).Name)\n\n\tnow := time.Now()\n\n\tbackups = append(backups, velerov1api.Backup{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"foo\",\n\t\t},\n\t\tStatus: velerov1api.BackupStatus{\n\t\t\tPhase: velerov1api.BackupPhaseCompleted,\n\t\t\tStartTimestamp: &metav1.Time{Time: now},\n\t\t},\n\t})\n\n\texpected := velerov1api.Backup{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"bar\",\n\t\t},\n\t\tStatus: velerov1api.BackupStatus{\n\t\t\tPhase: velerov1api.BackupPhaseCompleted,\n\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Second)},\n\t\t},\n\t}\n\tbackups = append(backups, expected)\n\n\tassert.Equal(t, expected, mostRecentCompletedBackup(backups))\n}\n\nfunc NewRestore(ns, name, backup, includeNS, includeResource string, phase velerov1api.RestorePhase) *builder.RestoreBuilder {\n\trestore := builder.ForRestore(ns, name).Phase(phase).Backup(backup).ItemOperationTimeout(60 * time.Minute)\n\n\tif includeNS != \"\" {\n\t\trestore = restore.IncludedNamespaces(includeNS)\n\t}\n\n\tif includeResource != \"\" {\n\t\trestore = restore.IncludedResources(includeResource)\n\t}\n\n\trestore.ExcludedResources(nonRestorableResources...)\n\n\treturn restore\n}\n\ntype fakeRestorer struct {\n\tmock.Mock\n\tcalledWithArg velerov1api.Restore\n\tkbClient client.Client\n}\n\nfunc (r *fakeRestorer) Restore(\n\tinfo *pkgrestore.Request,\n\tactions []riav2.RestoreItemAction,\n\tvolumeSnapshotterGetter pkgrestore.VolumeSnapshotterGetter,\n) (results.Result, results.Result) {\n\tres := r.Called(info.Log, info.Restore, info.Backup, info.BackupReader, actions)\n\n\tr.calledWithArg = *info.Restore\n\n\treturn res.Get(0).(results.Result), res.Get(1).(results.Result)\n}\n\nfunc (r *fakeRestorer) RestoreWithResolvers(req *pkgrestore.Request,\n\tresolver framework.RestoreItemActionResolverV2,\n\tvolumeSnapshotterGetter pkgrestore.VolumeSnapshotterGetter,\n) (results.Result, results.Result) {\n\tres := r.Called(req.Log, req.Restore, req.Backup, req.BackupReader, resolver,\n\t\tr.kbClient, volumeSnapshotterGetter)\n\n\tr.calledWithArg = *req.Restore\n\n\treturn res.Get(0).(results.Result), res.Get(1).(results.Result)\n}\n" }, { "path": "pkg/controller/restore_finalizer_controller.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tstoragev1api \"k8s.io/api/storage/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/hook\"\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\t\"github.com/vmware-tanzu/velero/pkg/persistence\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tkubeutil \"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/results\"\n)\n\ntype restoreFinalizerReconciler struct {\n\tclient.Client\n\tnamespace string\n\tlogger logrus.FieldLogger\n\tnewPluginManager func(logger logrus.FieldLogger) clientmgmt.Manager\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter\n\tmetrics *metrics.ServerMetrics\n\tclock clock.WithTickerAndDelayedExecution\n\tcrClient client.Client\n\tmultiHookTracker *hook.MultiHookTracker\n\tresourceTimeout time.Duration\n}\n\nfunc NewRestoreFinalizerReconciler(\n\tlogger logrus.FieldLogger,\n\tnamespace string,\n\tclient client.Client,\n\tnewPluginManager func(logrus.FieldLogger) clientmgmt.Manager,\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter,\n\tmetrics *metrics.ServerMetrics,\n\tcrClient client.Client,\n\tmultiHookTracker *hook.MultiHookTracker,\n\tresourceTimeout time.Duration,\n) *restoreFinalizerReconciler {\n\treturn &restoreFinalizerReconciler{\n\t\tClient: client,\n\t\tlogger: logger,\n\t\tnamespace: namespace,\n\t\tnewPluginManager: newPluginManager,\n\t\tbackupStoreGetter: backupStoreGetter,\n\t\tmetrics: metrics,\n\t\tclock: &clock.RealClock{},\n\t\tcrClient: crClient,\n\t\tmultiHookTracker: multiHookTracker,\n\t\tresourceTimeout: resourceTimeout,\n\t}\n}\n\nfunc (r *restoreFinalizerReconciler) SetupWithManager(mgr ctrl.Manager) error {\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\tFor(&velerov1api.Restore{}).\n\t\tNamed(constant.ControllerRestoreFinalizer).\n\t\tComplete(r)\n}\n\n// +kubebuilder:rbac:groups=velero.io,resources=restores,verbs=get;list;watch;update\n// +kubebuilder:rbac:groups=velero.io,resources=restores/status,verbs=get\nfunc (r *restoreFinalizerReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\tlog := r.logger.WithField(\"restore finalizer\", req.String())\n\tlog.Debug(\"restoreFinalizerReconciler getting restore\")\n\n\toriginal := &velerov1api.Restore{}\n\tif err := r.Get(ctx, req.NamespacedName, original); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.WithError(err).Error(\"restore not found\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\treturn ctrl.Result{}, errors.Wrapf(err, \"error getting restore %s\", req.String())\n\t}\n\trestore := original.DeepCopy()\n\tlog.Debugf(\"restore: %s\", restore.Name)\n\n\tlog = r.logger.WithFields(\n\t\tlogrus.Fields{\n\t\t\t\"restore\": req.String(),\n\t\t},\n\t)\n\n\tswitch restore.Status.Phase {\n\tcase velerov1api.RestorePhaseFinalizing, velerov1api.RestorePhaseFinalizingPartiallyFailed:\n\tdefault:\n\t\tlog.Debug(\"Restore is not awaiting finalization, skipping\")\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\tinfo, err := fetchBackupInfoInternal(r.Client, r.namespace, restore.Spec.BackupName)\n\tif err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.WithError(err).Error(\"not found backup, skip\")\n\t\t\tif err2 := r.finishProcessing(velerov1api.RestorePhasePartiallyFailed, restore, original); err2 != nil {\n\t\t\t\tlog.WithError(err2).Error(\"error updating restore's final status\")\n\t\t\t\treturn ctrl.Result{}, errors.Wrap(err2, \"error updating restore's final status\")\n\t\t\t}\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\tlog.WithError(err).Error(\"error getting backup info\")\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error getting backup info\")\n\t}\n\n\tpluginManager := r.newPluginManager(r.logger)\n\tdefer pluginManager.CleanupClients()\n\tbackupStore, err := r.backupStoreGetter.Get(info.location, pluginManager, r.logger)\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"error getting backup store\")\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error getting backup store\")\n\t}\n\n\tvolumeInfo, err := backupStore.GetBackupVolumeInfos(restore.Spec.BackupName)\n\tif err != nil {\n\t\tlog.WithError(err).Errorf(\"error getting volumeInfo for backup %s\", restore.Spec.BackupName)\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error getting volumeInfo\")\n\t}\n\n\trestoredResourceList, err := backupStore.GetRestoredResourceList(restore.Name)\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"error getting restoredResourceList\")\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error getting restoredResourceList\")\n\t}\n\n\trestoredPVCList := volume.RestoredPVCFromRestoredResourceList(restoredResourceList)\n\n\trestoreItemOperations, err := backupStore.GetRestoreItemOperations(restore.Name)\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"error getting itemOperationList\")\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error getting itemOperationList\")\n\t}\n\n\tfinalizerCtx := &finalizerContext{\n\t\tlogger: log,\n\t\trestore: restore,\n\t\tcrClient: r.crClient,\n\t\tvolumeInfo: volumeInfo,\n\t\trestoredPVCList: restoredPVCList,\n\t\tmultiHookTracker: r.multiHookTracker,\n\t\tresourceTimeout: r.resourceTimeout,\n\t\trestoreItemOperationList: restoreItemOperationList{\n\t\t\titems: restoreItemOperations,\n\t\t},\n\t}\n\twarnings, errs := finalizerCtx.execute()\n\n\twarningCnt := len(warnings.Velero) + len(warnings.Cluster)\n\tfor _, w := range warnings.Namespaces {\n\t\twarningCnt += len(w)\n\t}\n\terrCnt := len(errs.Velero) + len(errs.Cluster)\n\tfor _, e := range errs.Namespaces {\n\t\terrCnt += len(e)\n\t}\n\trestore.Status.Warnings += warningCnt\n\trestore.Status.Errors += errCnt\n\n\tif !errs.IsEmpty() {\n\t\trestore.Status.Phase = velerov1api.RestorePhaseFinalizingPartiallyFailed\n\t}\n\n\tif warningCnt > 0 || errCnt > 0 {\n\t\terr := r.updateResults(backupStore, restore, &warnings, &errs)\n\t\tif err != nil {\n\t\t\tlog.WithError(err).Error(\"error updating results\")\n\t\t\treturn ctrl.Result{}, errors.Wrap(err, \"error updating results\")\n\t\t}\n\t}\n\n\tfinalPhase := velerov1api.RestorePhaseCompleted\n\tif restore.Status.Phase == velerov1api.RestorePhaseFinalizingPartiallyFailed {\n\t\tfinalPhase = velerov1api.RestorePhasePartiallyFailed\n\t}\n\tlog.Infof(\"Marking restore %s\", finalPhase)\n\n\tif err := r.finishProcessing(finalPhase, restore, original); err != nil {\n\t\tlog.WithError(err).Error(\"error updating restore's final status\")\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error updating restore's final status\")\n\t}\n\n\treturn ctrl.Result{}, nil\n}\n\nfunc (r *restoreFinalizerReconciler) updateResults(backupStore persistence.BackupStore, restore *velerov1api.Restore, newWarnings *results.Result, newErrs *results.Result) error {\n\toriginResults, err := backupStore.GetRestoreResults(restore.Name)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error getting restore results\")\n\t}\n\twarnings := originResults[\"warnings\"]\n\terrs := originResults[\"errors\"]\n\twarnings.Merge(newWarnings)\n\terrs.Merge(newErrs)\n\n\tm := map[string]results.Result{\n\t\t\"warnings\": warnings,\n\t\t\"errors\": errs,\n\t}\n\tif err := putResults(restore, m, backupStore); err != nil {\n\t\treturn errors.Wrap(err, \"error putting restore results\")\n\t}\n\n\treturn nil\n}\n\nfunc (r *restoreFinalizerReconciler) finishProcessing(restorePhase velerov1api.RestorePhase, restore *velerov1api.Restore, original *velerov1api.Restore) error {\n\tif restorePhase == velerov1api.RestorePhasePartiallyFailed {\n\t\trestore.Status.Phase = velerov1api.RestorePhasePartiallyFailed\n\t\tr.metrics.RegisterRestorePartialFailure(restore.Spec.ScheduleName)\n\t} else {\n\t\trestore.Status.Phase = velerov1api.RestorePhaseCompleted\n\t\tr.metrics.RegisterRestoreSuccess(restore.Spec.ScheduleName)\n\t}\n\trestore.Status.CompletionTimestamp = &metav1.Time{Time: r.clock.Now()}\n\t// retry `Finalizing`/`FinalizingPartiallyFailed` to\n\t// - `Completed`\n\t// - `PartiallyFailed`\n\treturn kubeutil.PatchResourceWithRetriesOnErrors(r.resourceTimeout, original, restore, r.Client)\n}\n\ntype restoreItemOperationList struct {\n\titems []*itemoperation.RestoreOperation\n}\n\nfunc (r *restoreItemOperationList) selectByResource(group, resource, ns, name string) []*itemoperation.RestoreOperation {\n\tvar res []*itemoperation.RestoreOperation\n\trid := velero.ResourceIdentifier{\n\t\tGroupResource: schema.GroupResource{\n\t\t\tGroup: group,\n\t\t\tResource: resource,\n\t\t},\n\t\tNamespace: ns,\n\t\tName: name,\n\t}\n\tfor _, item := range r.items {\n\t\tif item != nil && item.Spec.ResourceIdentifier == rid {\n\t\t\tres = append(res, item)\n\t\t}\n\t}\n\treturn res\n}\n\n// SelectByPVC filters the restore item operation list by PVC namespace and name.\nfunc (r *restoreItemOperationList) SelectByPVC(ns, name string) []*itemoperation.RestoreOperation {\n\treturn r.selectByResource(\"\", \"persistentvolumeclaims\", ns, name)\n}\n\n// finalizerContext includes all the dependencies required by finalization tasks and\n// a function execute() to orderly implement task logic.\ntype finalizerContext struct {\n\tlogger logrus.FieldLogger\n\trestore *velerov1api.Restore\n\tcrClient client.Client\n\tvolumeInfo []*volume.BackupVolumeInfo\n\trestoredPVCList map[string]struct{}\n\trestoreItemOperationList restoreItemOperationList\n\tmultiHookTracker *hook.MultiHookTracker\n\tresourceTimeout time.Duration\n}\n\nfunc (ctx *finalizerContext) execute() (results.Result, results.Result) { //nolint:unparam //temporarily ignore the lint report: result 0 is always nil (unparam)\n\twarnings, errs := results.Result{}, results.Result{}\n\n\t// implement finalization tasks\n\tpdpErrs := ctx.patchDynamicPVWithVolumeInfo()\n\terrs.Merge(&pdpErrs)\n\n\trehErrs := ctx.WaitRestoreExecHook()\n\terrs.Merge(&rehErrs)\n\n\treturn warnings, errs\n}\n\n// patchDynamicPV patches newly dynamically provisioned PV using volume info\n// in order to restore custom settings that would otherwise be lost during dynamic PV recreation.\nfunc (ctx *finalizerContext) patchDynamicPVWithVolumeInfo() (errs results.Result) {\n\tctx.logger.Info(\"patching newly dynamically provisioned PV starts\")\n\n\tvar pvWaitGroup sync.WaitGroup\n\tvar resultLock sync.Mutex\n\n\tmaxConcurrency := 3\n\tsemaphore := make(chan struct{}, maxConcurrency)\n\n\tfor _, volumeItem := range ctx.volumeInfo {\n\t\tif (volumeItem.BackupMethod == volume.PodVolumeBackup || volumeItem.BackupMethod == volume.CSISnapshot) && volumeItem.PVInfo != nil {\n\t\t\t// Determine restored PVC namespace\n\t\t\trestoredNamespace := volumeItem.PVCNamespace\n\t\t\tif remapped, ok := ctx.restore.Spec.NamespaceMapping[restoredNamespace]; ok {\n\t\t\t\trestoredNamespace = remapped\n\t\t\t}\n\n\t\t\t// Check if PVC was restored in previous phase\n\t\t\tpvcKey := fmt.Sprintf(\"%s/%s\", restoredNamespace, volumeItem.PVCName)\n\t\t\tif _, restored := ctx.restoredPVCList[pvcKey]; !restored {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tpvWaitGroup.Add(1)\n\t\t\tgo func(volInfo volume.BackupVolumeInfo, restoredNamespace string) {\n\t\t\t\tdefer pvWaitGroup.Done()\n\n\t\t\t\tsemaphore <- struct{}{}\n\n\t\t\t\tlog := ctx.logger.WithField(\"PVC\", volInfo.PVCName).WithField(\"PVCNamespace\", restoredNamespace)\n\t\t\t\tlog.Debug(\"patching dynamic PV is in progress\")\n\n\t\t\t\terr := wait.PollUntilContextTimeout(context.Background(), 10*time.Second, ctx.resourceTimeout, true, func(context.Context) (bool, error) {\n\t\t\t\t\t// wait for PVC to be bound\n\t\t\t\t\tpvc := &corev1api.PersistentVolumeClaim{}\n\t\t\t\t\terr := ctx.crClient.Get(context.Background(), client.ObjectKey{Name: volInfo.PVCName, Namespace: restoredNamespace}, pvc)\n\t\t\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\t\t\tlog.Debug(\"error not finding PVC\")\n\t\t\t\t\t\treturn false, nil\n\t\t\t\t\t}\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\treturn false, err\n\t\t\t\t\t}\n\n\t\t\t\t\t// Check whether the async operation to populate the PVC is successful. If it's not, will skip patching the PV, instead of waiting.\n\t\t\t\t\toperations := ctx.restoreItemOperationList.SelectByPVC(pvc.Namespace, pvc.Name)\n\t\t\t\t\tfor _, op := range operations {\n\t\t\t\t\t\tif op.Spec.RestoreItemAction == constant.PluginCSIPVCRestoreRIA &&\n\t\t\t\t\t\t\top.Status.Phase != itemoperation.OperationPhaseCompleted {\n\t\t\t\t\t\t\tlog.Warnf(\"skipping PV patch, because the operation to restore the PVC is not completed, \"+\n\t\t\t\t\t\t\t\t\"operation: %s, phase: %s\", op.Spec.OperationID, op.Status.Phase)\n\t\t\t\t\t\t\treturn true, nil\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\n\t\t\t\t\t// We are handling a common but specific scenario where a PVC is in a pending state and uses a storage class with\n\t\t\t\t\t// VolumeBindingMode set to WaitForFirstConsumer. In this case, the PV patch step is skipped to avoid\n\t\t\t\t\t// failures due to the PVC not being bound, which could cause a timeout and result in a failed restore.\n\t\t\t\t\tif pvc.Status.Phase == corev1api.ClaimPending {\n\t\t\t\t\t\t// check if storage class used has VolumeBindingMode as WaitForFirstConsumer\n\t\t\t\t\t\tscName := *pvc.Spec.StorageClassName\n\t\t\t\t\t\tsc := &storagev1api.StorageClass{}\n\t\t\t\t\t\terr = ctx.crClient.Get(context.Background(), client.ObjectKey{Name: scName}, sc)\n\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\terrs.Add(restoredNamespace, err)\n\t\t\t\t\t\t\treturn false, err\n\t\t\t\t\t\t}\n\t\t\t\t\t\t// skip PV patch step for this scenario\n\t\t\t\t\t\t// because pvc would not be bound and the PV patch step would fail due to timeout thus failing the restore\n\t\t\t\t\t\tif *sc.VolumeBindingMode == storagev1api.VolumeBindingWaitForFirstConsumer {\n\t\t\t\t\t\t\tlog.Warnf(\"skipping PV patch to restore custom reclaim policy, if any: StorageClass %s used by PVC %s has VolumeBindingMode set to WaitForFirstConsumer, and the PVC is also in a pending state\", scName, pvc.Name)\n\t\t\t\t\t\t\treturn true, nil\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\n\t\t\t\t\tif pvc.Status.Phase != corev1api.ClaimBound || pvc.Spec.VolumeName == \"\" {\n\t\t\t\t\t\tlog.Debugf(\"PVC: %s not ready\", pvc.Name)\n\t\t\t\t\t\treturn false, nil\n\t\t\t\t\t}\n\n\t\t\t\t\t// wait for PV to be bound\n\t\t\t\t\tpvName := pvc.Spec.VolumeName\n\t\t\t\t\tpv := &corev1api.PersistentVolume{}\n\t\t\t\t\terr = ctx.crClient.Get(context.Background(), client.ObjectKey{Name: pvName}, pv)\n\t\t\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\t\t\tlog.Debugf(\"error not finding PV: %s\", pvName)\n\t\t\t\t\t\treturn false, nil\n\t\t\t\t\t}\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\treturn false, err\n\t\t\t\t\t}\n\n\t\t\t\t\tif pv.Spec.ClaimRef == nil || pv.Status.Phase != corev1api.VolumeBound {\n\t\t\t\t\t\tlog.Debugf(\"PV: %s not ready\", pvName)\n\t\t\t\t\t\treturn false, nil\n\t\t\t\t\t}\n\n\t\t\t\t\t// validate PV\n\t\t\t\t\tif pv.Spec.ClaimRef.Name != pvc.Name || pv.Spec.ClaimRef.Namespace != restoredNamespace {\n\t\t\t\t\t\treturn false, fmt.Errorf(\"PV was bound by unexpected PVC, unexpected PVC: %s/%s, expected PVC: %s/%s\",\n\t\t\t\t\t\t\tpv.Spec.ClaimRef.Namespace, pv.Spec.ClaimRef.Name, restoredNamespace, pvc.Name)\n\t\t\t\t\t}\n\n\t\t\t\t\t// patch PV's reclaim policy and label using the corresponding data stored in volume info\n\t\t\t\t\tif needPatch(pv, volInfo.PVInfo) {\n\t\t\t\t\t\tupdatedPV := pv.DeepCopy()\n\t\t\t\t\t\tupdatedPV.Labels = volInfo.PVInfo.Labels\n\t\t\t\t\t\tupdatedPV.Spec.PersistentVolumeReclaimPolicy = corev1api.PersistentVolumeReclaimPolicy(volInfo.PVInfo.ReclaimPolicy)\n\t\t\t\t\t\tif err := kubeutil.PatchResource(pv, updatedPV, ctx.crClient); err != nil {\n\t\t\t\t\t\t\treturn false, err\n\t\t\t\t\t\t}\n\t\t\t\t\t\tlog.Infof(\"newly dynamically provisioned PV:%s has been patched using volume info\", pvName)\n\t\t\t\t\t}\n\n\t\t\t\t\treturn true, nil\n\t\t\t\t})\n\n\t\t\t\tif err != nil {\n\t\t\t\t\terr = fmt.Errorf(\"fail to patch dynamic PV, err: %s, PVC: %s, PV: %s\", err, volInfo.PVCName, volInfo.PVName)\n\t\t\t\t\tctx.logger.WithError(errors.WithStack((err))).Error(\"err patching dynamic PV using volume info\")\n\t\t\t\t\tresultLock.Lock()\n\t\t\t\t\tdefer resultLock.Unlock()\n\t\t\t\t\terrs.Add(restoredNamespace, err)\n\t\t\t\t}\n\n\t\t\t\t<-semaphore\n\t\t\t}(*volumeItem, restoredNamespace)\n\t\t}\n\t}\n\n\tpvWaitGroup.Wait()\n\tctx.logger.Info(\"patching newly dynamically provisioned PV ends\")\n\n\treturn errs\n}\n\nfunc needPatch(newPV *corev1api.PersistentVolume, pvInfo *volume.PVInfo) bool {\n\tif newPV.Spec.PersistentVolumeReclaimPolicy != corev1api.PersistentVolumeReclaimPolicy(pvInfo.ReclaimPolicy) {\n\t\treturn true\n\t}\n\n\tnewPVLabels, pvLabels := newPV.Labels, pvInfo.Labels\n\tfor k, v := range pvLabels {\n\t\tif _, ok := newPVLabels[k]; !ok {\n\t\t\treturn true\n\t\t}\n\t\tif newPVLabels[k] != v {\n\t\t\treturn true\n\t\t}\n\t}\n\n\treturn false\n}\n\n// WaitRestoreExecHook waits for restore exec hooks to finish then update the hook execution results\nfunc (ctx *finalizerContext) WaitRestoreExecHook() (errs results.Result) {\n\tlog := ctx.logger.WithField(\"restore\", ctx.restore.Name)\n\tlog.Info(\"Waiting for restore exec hooks starts\")\n\n\t// wait for restore exec hooks to finish\n\terr := wait.PollUntilContextCancel(context.Background(), 1*time.Second, true, func(context.Context) (bool, error) {\n\t\tlog.Debug(\"Checking the progress of hooks execution\")\n\t\tif ctx.multiHookTracker.IsComplete(ctx.restore.Name) {\n\t\t\treturn true, nil\n\t\t}\n\t\treturn false, nil\n\t})\n\tif err != nil {\n\t\terrs.Add(ctx.restore.Namespace, err)\n\t\treturn errs\n\t}\n\tlog.Info(\"Done waiting for restore exec hooks starts\")\n\n\tfor _, ei := range ctx.multiHookTracker.HookErrs(ctx.restore.Name) {\n\t\terrs.Add(ei.Namespace, ei.Err)\n\t}\n\n\t// update hooks execution status\n\tupdated := ctx.restore.DeepCopy()\n\tif updated.Status.HookStatus == nil {\n\t\tupdated.Status.HookStatus = &velerov1api.HookStatus{}\n\t}\n\tupdated.Status.HookStatus.HooksAttempted, updated.Status.HookStatus.HooksFailed = ctx.multiHookTracker.Stat(ctx.restore.Name)\n\tlog.Debugf(\"hookAttempted: %d, hookFailed: %d\", updated.Status.HookStatus.HooksAttempted, updated.Status.HookStatus.HooksFailed)\n\n\tif err := kubeutil.PatchResource(ctx.restore, updated, ctx.crClient); err != nil {\n\t\tlog.WithError(errors.WithStack((err))).Error(\"Updating restore status\")\n\t\terrs.Add(ctx.restore.Namespace, err)\n\t}\n\n\t// delete the hook data for this restore\n\tctx.multiHookTracker.Delete(ctx.restore.Name)\n\n\treturn errs\n}\n" }, { "path": "pkg/controller/restore_finalizer_controller_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"fmt\"\n\t\"syscall\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\ttestclocks \"k8s.io/utils/clock/testing\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\tcrclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/hook\"\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\tpersistencemocks \"github.com/vmware-tanzu/velero/pkg/persistence/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\tpluginmocks \"github.com/vmware-tanzu/velero/pkg/plugin/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\tpkgUtilKubeMocks \"github.com/vmware-tanzu/velero/pkg/util/kube/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/results\"\n)\n\nfunc TestRestoreFinalizerReconcile(t *testing.T) {\n\tdefaultStorageLocation := builder.ForBackupStorageLocation(\"velero\", \"default\").Provider(\"myCloud\").Bucket(\"bucket\").Result()\n\tnow, err := time.Parse(time.RFC1123Z, time.RFC1123Z)\n\trequire.NoError(t, err)\n\tnow = now.Local()\n\ttimestamp := metav1.NewTime(now)\n\tassert.NotNil(t, timestamp)\n\n\trfrTests := []struct {\n\t\tname string\n\t\trestore *velerov1api.Restore\n\t\tbackup *velerov1api.Backup\n\t\tlocation *velerov1api.BackupStorageLocation\n\t\texpectError bool\n\t\texpectPhase velerov1api.RestorePhase\n\t\texpectWarningsCnt int\n\t\texpectErrsCnt int\n\t\tstatusCompare bool\n\t\texpectedCompletedTime *metav1.Time\n\t}{\n\t\t{\n\t\t\tname: \"Restore is not awaiting finalization, skip\",\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"restore-1\").Phase(velerov1api.RestorePhaseInProgress).Result(),\n\t\t\texpectError: false,\n\t\t\texpectPhase: velerov1api.RestorePhaseInProgress,\n\t\t\tstatusCompare: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Upon completion of all finalization tasks in the 'FinalizingPartiallyFailed' phase, the restore process transit to the 'PartiallyFailed' phase.\",\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"restore-1\").Phase(velerov1api.RestorePhaseFinalizingPartiallyFailed).Backup(\"backup-1\").Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\tlocation: defaultStorageLocation,\n\t\t\texpectError: false,\n\t\t\texpectPhase: velerov1api.RestorePhasePartiallyFailed,\n\t\t\tstatusCompare: true,\n\t\t\texpectedCompletedTime: ×tamp,\n\t\t\texpectWarningsCnt: 0,\n\t\t\texpectErrsCnt: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"Upon completion of all finalization tasks in the 'Finalizing' phase, the restore process transit to the 'Completed' phase.\",\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"restore-1\").Phase(velerov1api.RestorePhaseFinalizing).Backup(\"backup-1\").Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\tlocation: defaultStorageLocation,\n\t\t\texpectError: false,\n\t\t\texpectPhase: velerov1api.RestorePhaseCompleted,\n\t\t\tstatusCompare: true,\n\t\t\texpectedCompletedTime: ×tamp,\n\t\t\texpectWarningsCnt: 0,\n\t\t\texpectErrsCnt: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"Backup not exist\",\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"restore-1\").Phase(velerov1api.RestorePhaseFinalizing).Backup(\"backup-2\").Result(),\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Restore not exist\",\n\t\t\trestore: builder.ForRestore(\"unknown\", \"restore-1\").Phase(velerov1api.RestorePhaseFinalizing).Result(),\n\t\t\texpectError: false,\n\t\t\tstatusCompare: false,\n\t\t},\n\t}\n\n\tfor _, test := range rfrTests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tif test.restore == nil {\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tvar (\n\t\t\t\tfakeClient = velerotest.NewFakeControllerRuntimeClientBuilder(t).Build()\n\t\t\t\tlogger = velerotest.NewLogger()\n\t\t\t\tpluginManager = &pluginmocks.Manager{}\n\t\t\t\tbackupStore = &persistencemocks.BackupStore{}\n\t\t\t)\n\n\t\t\tdefer func() {\n\t\t\t\t// reset defaultStorageLocation resourceVersion\n\t\t\t\tdefaultStorageLocation.ObjectMeta.ResourceVersion = \"\"\n\t\t\t}()\n\n\t\t\tr := NewRestoreFinalizerReconciler(\n\t\t\t\tlogger,\n\t\t\t\tvelerov1api.DefaultNamespace,\n\t\t\t\tfakeClient,\n\t\t\t\tfunc(logrus.FieldLogger) clientmgmt.Manager { return pluginManager },\n\t\t\t\tNewFakeSingleObjectBackupStoreGetter(backupStore),\n\t\t\t\tmetrics.NewServerMetrics(),\n\t\t\t\tfakeClient,\n\t\t\t\thook.NewMultiHookTracker(),\n\t\t\t\t10*time.Minute,\n\t\t\t)\n\t\t\tr.clock = testclocks.NewFakeClock(now)\n\n\t\t\tif test.restore != nil && test.restore.Namespace == velerov1api.DefaultNamespace {\n\t\t\t\trequire.NoError(t, r.Client.Create(t.Context(), test.restore))\n\t\t\t\tbackupStore.On(\"GetRestoredResourceList\", test.restore.Name).Return(map[string][]string{}, nil)\n\t\t\t\tbackupStore.On(\"GetRestoreItemOperations\", test.restore.Name).Return([]*itemoperation.RestoreOperation{}, nil)\n\t\t\t}\n\t\t\tif test.backup != nil {\n\t\t\t\trequire.NoError(t, r.Client.Create(t.Context(), test.backup))\n\t\t\t\tbackupStore.On(\"GetBackupVolumeInfos\", test.backup.Name).Return(nil, nil)\n\t\t\t\tpluginManager.On(\"GetRestoreItemActionsV2\").Return(nil, nil)\n\t\t\t\tpluginManager.On(\"CleanupClients\")\n\t\t\t}\n\t\t\tif test.location != nil {\n\t\t\t\trequire.NoError(t, r.Client.Create(t.Context(), test.location))\n\t\t\t}\n\n\t\t\t_, err = r.Reconcile(t.Context(), ctrl.Request{NamespacedName: types.NamespacedName{\n\t\t\t\tNamespace: test.restore.Namespace,\n\t\t\t\tName: test.restore.Name,\n\t\t\t}})\n\n\t\t\tassert.Equal(t, test.expectError, err != nil)\n\t\t\tif test.expectError {\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tif test.statusCompare {\n\t\t\t\trestoreAfter := velerov1api.Restore{}\n\t\t\t\terr = fakeClient.Get(t.Context(), types.NamespacedName{\n\t\t\t\t\tNamespace: test.restore.Namespace,\n\t\t\t\t\tName: test.restore.Name,\n\t\t\t\t}, &restoreAfter)\n\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\tassert.Equal(t, test.expectPhase, restoreAfter.Status.Phase)\n\t\t\t\tassert.Equal(t, test.expectErrsCnt, restoreAfter.Status.Errors)\n\t\t\t\tassert.Equal(t, test.expectWarningsCnt, restoreAfter.Status.Warnings)\n\t\t\t\trequire.True(t, test.expectedCompletedTime.Equal(restoreAfter.Status.CompletionTimestamp))\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestUpdateResult(t *testing.T) {\n\tvar (\n\t\tfakeClient = velerotest.NewFakeControllerRuntimeClientBuilder(t).Build()\n\t\tlogger = velerotest.NewLogger()\n\t\tpluginManager = &pluginmocks.Manager{}\n\t\tbackupStore = &persistencemocks.BackupStore{}\n\t)\n\n\tr := NewRestoreFinalizerReconciler(\n\t\tlogger,\n\t\tvelerov1api.DefaultNamespace,\n\t\tfakeClient,\n\t\tfunc(logrus.FieldLogger) clientmgmt.Manager { return pluginManager },\n\t\tNewFakeSingleObjectBackupStoreGetter(backupStore),\n\t\tmetrics.NewServerMetrics(),\n\t\tfakeClient,\n\t\thook.NewMultiHookTracker(),\n\t\t10*time.Minute,\n\t)\n\trestore := builder.ForRestore(velerov1api.DefaultNamespace, \"restore-1\").Result()\n\tres := map[string]results.Result{\"warnings\": {}, \"errors\": {}}\n\n\tbackupStore.On(\"GetRestoreResults\", restore.Name).Return(res, nil)\n\tbackupStore.On(\"PutRestoreResults\", mock.Anything, mock.Anything, mock.Anything).Return(nil)\n\n\terr := r.updateResults(backupStore, restore, &results.Result{}, &results.Result{})\n\trequire.NoError(t, err)\n}\n\nfunc TestPatchDynamicPVWithVolumeInfo(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tvolumeInfo []*volume.BackupVolumeInfo\n\t\trestoredPVCNames map[string]struct{}\n\t\trestore *velerov1api.Restore\n\t\trestoredPVC []*corev1api.PersistentVolumeClaim\n\t\trestoredPV []*corev1api.PersistentVolume\n\t\texpectedPatch map[string]volume.PVInfo\n\t\texpectedErrNum int\n\t}{\n\t\t{\n\t\t\tname: \"no applicable volumeInfo\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{{BackupMethod: \"VeleroNativeSnapshot\", PVCName: \"pvc1\"}},\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"restore\").Result(),\n\t\t\texpectedPatch: nil,\n\t\t\texpectedErrNum: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"no restored PVC\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{{BackupMethod: \"PodVolumeBackup\", PVCName: \"pvc1\"}},\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"restore\").Result(),\n\t\t\texpectedPatch: nil,\n\t\t\texpectedErrNum: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"no applicable pv patch\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{{\n\t\t\t\tBackupMethod: \"PodVolumeBackup\",\n\t\t\t\tPVCName: \"pvc1\",\n\t\t\t\tPVName: \"pv1\",\n\t\t\t\tPVCNamespace: \"ns1\",\n\t\t\t\tPVInfo: &volume.PVInfo{\n\t\t\t\t\tReclaimPolicy: string(corev1api.PersistentVolumeReclaimDelete),\n\t\t\t\t\tLabels: map[string]string{\"label1\": \"label1-val\"},\n\t\t\t\t},\n\t\t\t}},\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"restore\").Result(),\n\t\t\trestoredPVCNames: map[string]struct{}{\"ns1/pvc1\": {}},\n\t\t\trestoredPV: []*corev1api.PersistentVolume{\n\t\t\t\tbuilder.ForPersistentVolume(\"new-pv1\").ObjectMeta(builder.WithLabels(\"label1\", \"label1-val\")).ClaimRef(\"ns1\", \"pvc1\").Phase(corev1api.VolumeBound).ReclaimPolicy(corev1api.PersistentVolumeReclaimDelete).Result()},\n\t\t\trestoredPVC: []*corev1api.PersistentVolumeClaim{\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns1\", \"pvc1\").VolumeName(\"new-pv1\").Phase(corev1api.ClaimBound).Result(),\n\t\t\t},\n\t\t\texpectedPatch: nil,\n\t\t\texpectedErrNum: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"an applicable pv patch\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{{\n\t\t\t\tBackupMethod: \"PodVolumeBackup\",\n\t\t\t\tPVCName: \"pvc1\",\n\t\t\t\tPVName: \"pv1\",\n\t\t\t\tPVCNamespace: \"ns1\",\n\t\t\t\tPVInfo: &volume.PVInfo{\n\t\t\t\t\tReclaimPolicy: string(corev1api.PersistentVolumeReclaimDelete),\n\t\t\t\t\tLabels: map[string]string{\"label1\": \"label1-val\"},\n\t\t\t\t},\n\t\t\t}},\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"restore\").Result(),\n\t\t\trestoredPVCNames: map[string]struct{}{\"ns1/pvc1\": {}},\n\t\t\trestoredPV: []*corev1api.PersistentVolume{\n\t\t\t\tbuilder.ForPersistentVolume(\"new-pv1\").ClaimRef(\"ns1\", \"pvc1\").Phase(corev1api.VolumeBound).ReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).Result()},\n\t\t\trestoredPVC: []*corev1api.PersistentVolumeClaim{\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns1\", \"pvc1\").VolumeName(\"new-pv1\").Phase(corev1api.ClaimBound).Result(),\n\t\t\t},\n\t\t\texpectedPatch: map[string]volume.PVInfo{\"new-pv1\": {\n\t\t\t\tReclaimPolicy: string(corev1api.PersistentVolumeReclaimDelete),\n\t\t\t\tLabels: map[string]string{\"label1\": \"label1-val\"},\n\t\t\t}},\n\t\t\texpectedErrNum: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"a mapped namespace restore\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{{\n\t\t\t\tBackupMethod: \"PodVolumeBackup\",\n\t\t\t\tPVCName: \"pvc1\",\n\t\t\t\tPVName: \"pv1\",\n\t\t\t\tPVCNamespace: \"ns2\",\n\t\t\t\tPVInfo: &volume.PVInfo{\n\t\t\t\t\tReclaimPolicy: string(corev1api.PersistentVolumeReclaimDelete),\n\t\t\t\t\tLabels: map[string]string{\"label1\": \"label1-val\"},\n\t\t\t\t},\n\t\t\t}},\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"restore\").NamespaceMappings(\"ns2\", \"ns1\").Result(),\n\t\t\trestoredPVCNames: map[string]struct{}{\"ns1/pvc1\": {}},\n\t\t\trestoredPV: []*corev1api.PersistentVolume{\n\t\t\t\tbuilder.ForPersistentVolume(\"new-pv1\").ClaimRef(\"ns1\", \"pvc1\").Phase(corev1api.VolumeBound).ReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).Result()},\n\t\t\trestoredPVC: []*corev1api.PersistentVolumeClaim{\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns1\", \"pvc1\").VolumeName(\"new-pv1\").Phase(corev1api.ClaimBound).Result(),\n\t\t\t},\n\t\t\texpectedPatch: map[string]volume.PVInfo{\"new-pv1\": {\n\t\t\t\tReclaimPolicy: string(corev1api.PersistentVolumeReclaimDelete),\n\t\t\t\tLabels: map[string]string{\"label1\": \"label1-val\"},\n\t\t\t}},\n\t\t\texpectedErrNum: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"two applicable pv patches\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{{\n\t\t\t\tBackupMethod: \"PodVolumeBackup\",\n\t\t\t\tPVCName: \"pvc1\",\n\t\t\t\tPVName: \"pv1\",\n\t\t\t\tPVCNamespace: \"ns1\",\n\t\t\t\tPVInfo: &volume.PVInfo{\n\t\t\t\t\tReclaimPolicy: string(corev1api.PersistentVolumeReclaimDelete),\n\t\t\t\t\tLabels: map[string]string{\"label1\": \"label1-val\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tBackupMethod: \"CSISnapshot\",\n\t\t\t\t\tPVCName: \"pvc2\",\n\t\t\t\t\tPVName: \"pv2\",\n\t\t\t\t\tPVCNamespace: \"ns2\",\n\t\t\t\t\tPVInfo: &volume.PVInfo{\n\t\t\t\t\t\tReclaimPolicy: string(corev1api.PersistentVolumeReclaimDelete),\n\t\t\t\t\t\tLabels: map[string]string{\"label2\": \"label2-val\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"restore\").Result(),\n\t\t\trestoredPVCNames: map[string]struct{}{\n\t\t\t\t\"ns1/pvc1\": {},\n\t\t\t\t\"ns2/pvc2\": {},\n\t\t\t},\n\t\t\trestoredPV: []*corev1api.PersistentVolume{\n\t\t\t\tbuilder.ForPersistentVolume(\"new-pv1\").ClaimRef(\"ns1\", \"pvc1\").Phase(corev1api.VolumeBound).ReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).Result(),\n\t\t\t\tbuilder.ForPersistentVolume(\"new-pv2\").ClaimRef(\"ns2\", \"pvc2\").Phase(corev1api.VolumeBound).ReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).Result(),\n\t\t\t},\n\t\t\trestoredPVC: []*corev1api.PersistentVolumeClaim{\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns1\", \"pvc1\").VolumeName(\"new-pv1\").Phase(corev1api.ClaimBound).Result(),\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns2\", \"pvc2\").VolumeName(\"new-pv2\").Phase(corev1api.ClaimBound).Result(),\n\t\t\t},\n\t\t\texpectedPatch: map[string]volume.PVInfo{\n\t\t\t\t\"new-pv1\": {\n\t\t\t\t\tReclaimPolicy: string(corev1api.PersistentVolumeReclaimDelete),\n\t\t\t\t\tLabels: map[string]string{\"label1\": \"label1-val\"},\n\t\t\t\t},\n\t\t\t\t\"new-pv2\": {\n\t\t\t\t\tReclaimPolicy: string(corev1api.PersistentVolumeReclaimDelete),\n\t\t\t\t\tLabels: map[string]string{\"label2\": \"label2-val\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedErrNum: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"an applicable pv patch with bound error\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{{\n\t\t\t\tBackupMethod: \"PodVolumeBackup\",\n\t\t\t\tPVCName: \"pvc1\",\n\t\t\t\tPVName: \"pv1\",\n\t\t\t\tPVCNamespace: \"ns1\",\n\t\t\t\tPVInfo: &volume.PVInfo{\n\t\t\t\t\tReclaimPolicy: string(corev1api.PersistentVolumeReclaimDelete),\n\t\t\t\t\tLabels: map[string]string{\"label1\": \"label1-val\"},\n\t\t\t\t},\n\t\t\t}},\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"restore\").Result(),\n\t\t\trestoredPVCNames: map[string]struct{}{\"ns1/pvc1\": {}},\n\t\t\trestoredPV: []*corev1api.PersistentVolume{\n\t\t\t\tbuilder.ForPersistentVolume(\"new-pv1\").ClaimRef(\"ns2\", \"pvc2\").Phase(corev1api.VolumeBound).ReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).Result()},\n\t\t\trestoredPVC: []*corev1api.PersistentVolumeClaim{\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns1\", \"pvc1\").VolumeName(\"new-pv1\").Phase(corev1api.ClaimBound).Result(),\n\t\t\t},\n\t\t\texpectedErrNum: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"two applicable pv patches with an error\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{{\n\t\t\t\tBackupMethod: \"PodVolumeBackup\",\n\t\t\t\tPVCName: \"pvc1\",\n\t\t\t\tPVName: \"pv1\",\n\t\t\t\tPVCNamespace: \"ns1\",\n\t\t\t\tPVInfo: &volume.PVInfo{\n\t\t\t\t\tReclaimPolicy: string(corev1api.PersistentVolumeReclaimDelete),\n\t\t\t\t\tLabels: map[string]string{\"label1\": \"label1-val\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tBackupMethod: \"CSISnapshot\",\n\t\t\t\t\tPVCName: \"pvc2\",\n\t\t\t\t\tPVName: \"pv2\",\n\t\t\t\t\tPVCNamespace: \"ns2\",\n\t\t\t\t\tPVInfo: &volume.PVInfo{\n\t\t\t\t\t\tReclaimPolicy: string(corev1api.PersistentVolumeReclaimDelete),\n\t\t\t\t\t\tLabels: map[string]string{\"label2\": \"label2-val\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"restore\").Result(),\n\t\t\trestoredPVCNames: map[string]struct{}{\n\t\t\t\t\"ns1/pvc1\": {},\n\t\t\t\t\"ns2/pvc2\": {},\n\t\t\t},\n\t\t\trestoredPV: []*corev1api.PersistentVolume{\n\t\t\t\tbuilder.ForPersistentVolume(\"new-pv1\").ClaimRef(\"ns1\", \"pvc1\").Phase(corev1api.VolumeBound).ReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).Result(),\n\t\t\t\tbuilder.ForPersistentVolume(\"new-pv2\").ClaimRef(\"ns3\", \"pvc3\").Phase(corev1api.VolumeBound).ReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).Result(),\n\t\t\t},\n\t\t\trestoredPVC: []*corev1api.PersistentVolumeClaim{\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns1\", \"pvc1\").VolumeName(\"new-pv1\").Phase(corev1api.ClaimBound).Result(),\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns2\", \"pvc2\").VolumeName(\"new-pv2\").Phase(corev1api.ClaimBound).Result(),\n\t\t\t},\n\t\t\texpectedPatch: map[string]volume.PVInfo{\n\t\t\t\t\"new-pv1\": {\n\t\t\t\t\tReclaimPolicy: string(corev1api.PersistentVolumeReclaimDelete),\n\t\t\t\t\tLabels: map[string]string{\"label1\": \"label1-val\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedErrNum: 1,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tvar (\n\t\t\tfakeClient = velerotest.NewFakeControllerRuntimeClientBuilder(t).Build()\n\t\t\tlogger = velerotest.NewLogger()\n\t\t)\n\t\tctx := &finalizerContext{\n\t\t\tlogger: logger,\n\t\t\tcrClient: fakeClient,\n\t\t\trestore: tc.restore,\n\t\t\trestoredPVCList: tc.restoredPVCNames,\n\t\t\tvolumeInfo: tc.volumeInfo,\n\t\t}\n\n\t\tfor _, pv := range tc.restoredPV {\n\t\t\trequire.NoError(t, ctx.crClient.Create(t.Context(), pv))\n\t\t}\n\t\tfor _, pvc := range tc.restoredPVC {\n\t\t\trequire.NoError(t, ctx.crClient.Create(t.Context(), pvc))\n\t\t}\n\n\t\terrs := ctx.patchDynamicPVWithVolumeInfo()\n\t\tif tc.expectedErrNum > 0 {\n\t\t\tassert.Len(t, errs.Namespaces, tc.expectedErrNum)\n\t\t}\n\n\t\tfor pvName, expectedPVInfo := range tc.expectedPatch {\n\t\t\tpv := &corev1api.PersistentVolume{}\n\t\t\terr := ctx.crClient.Get(t.Context(), crclient.ObjectKey{Name: pvName}, pv)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.Equal(t, expectedPVInfo.ReclaimPolicy, string(pv.Spec.PersistentVolumeReclaimPolicy))\n\t\t\tassert.Equal(t, expectedPVInfo.Labels, pv.Labels)\n\t\t}\n\t}\n}\n\nfunc TestWaitRestoreExecHook(t *testing.T) {\n\thookTracker1 := hook.NewMultiHookTracker()\n\trestoreName1 := \"restore1\"\n\n\thookTracker2 := hook.NewMultiHookTracker()\n\trestoreName2 := \"restore2\"\n\thookTracker2.Add(restoreName2, \"ns\", \"pod\", \"con1\", \"s1\", \"h1\", \"\", 0)\n\thookTracker2.Record(restoreName2, \"ns\", \"pod\", \"con1\", \"s1\", \"h1\", \"\", 0, false, nil)\n\n\thookTracker3 := hook.NewMultiHookTracker()\n\trestoreName3 := \"restore3\"\n\tpodNs, podName, container, source, hookName := \"ns\", \"pod\", \"con1\", \"s1\", \"h1\"\n\thookFailed, hookErr := true, fmt.Errorf(\"hook failed\")\n\thookTracker3.Add(restoreName3, podNs, podName, container, source, hookName, hook.PhasePre, 0)\n\n\ttests := []struct {\n\t\tname string\n\t\thookTracker *hook.MultiHookTracker\n\t\trestore *velerov1api.Restore\n\t\texpectedHooksAttempted int\n\t\texpectedHooksFailed int\n\t\texpectedHookErrs int\n\t\twaitSec int\n\t\tpodName string\n\t\tpodNs string\n\t\tContainer string\n\t\tSource string\n\t\thookName string\n\t\thookFailed bool\n\t\thookErr error\n\t}{\n\t\t{\n\t\t\tname: \"no restore exec hooks\",\n\t\t\thookTracker: hookTracker1,\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, restoreName1).Result(),\n\t\t\texpectedHooksAttempted: 0,\n\t\t\texpectedHooksFailed: 0,\n\t\t\texpectedHookErrs: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"1 restore exec hook having been executed\",\n\t\t\thookTracker: hookTracker2,\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, restoreName2).Result(),\n\t\t\texpectedHooksAttempted: 1,\n\t\t\texpectedHooksFailed: 0,\n\t\t\texpectedHookErrs: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"1 restore exec hook to be executed\",\n\t\t\thookTracker: hookTracker3,\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, restoreName3).Result(),\n\t\t\twaitSec: 2,\n\t\t\texpectedHooksAttempted: 1,\n\t\t\texpectedHooksFailed: 1,\n\t\t\texpectedHookErrs: 1,\n\t\t\tpodName: podName,\n\t\t\tpodNs: podNs,\n\t\t\tContainer: container,\n\t\t\tSource: source,\n\t\t\thookName: hookName,\n\t\t\thookFailed: hookFailed,\n\t\t\thookErr: hookErr,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tvar (\n\t\t\tfakeClient = velerotest.NewFakeControllerRuntimeClientBuilder(t).Build()\n\t\t\tlogger = velerotest.NewLogger()\n\t\t)\n\t\tctx := &finalizerContext{\n\t\t\tlogger: logger,\n\t\t\tcrClient: fakeClient,\n\t\t\trestore: tc.restore,\n\t\t\tmultiHookTracker: tc.hookTracker,\n\t\t}\n\t\trequire.NoError(t, ctx.crClient.Create(t.Context(), tc.restore))\n\n\t\tif tc.waitSec > 0 {\n\t\t\tgo func() {\n\t\t\t\ttime.Sleep(time.Second * time.Duration(tc.waitSec))\n\t\t\t\ttc.hookTracker.Record(tc.restore.Name, tc.podNs, tc.podName, tc.Container, tc.Source, tc.hookName, hook.PhasePre, 0, tc.hookFailed, tc.hookErr)\n\t\t\t}()\n\t\t}\n\n\t\terrs := ctx.WaitRestoreExecHook()\n\t\tassert.Len(t, errs.Namespaces, tc.expectedHookErrs)\n\n\t\tupdated := &velerov1api.Restore{}\n\t\terr := ctx.crClient.Get(t.Context(), crclient.ObjectKey{Namespace: velerov1api.DefaultNamespace, Name: tc.restore.Name}, updated)\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, tc.expectedHooksAttempted, updated.Status.HookStatus.HooksAttempted)\n\t\tassert.Equal(t, tc.expectedHooksFailed, updated.Status.HookStatus.HooksFailed)\n\t}\n}\n\n// test finishprocessing with mocks of kube client to simulate connection refused\nfunc Test_restoreFinalizerReconciler_finishProcessing(t *testing.T) {\n\ttype args struct {\n\t\t// mockClientActions simulate different client errors\n\t\tmockClientActions func(*pkgUtilKubeMocks.Client)\n\t\t// return bool indicating if the client method was called as expected\n\t\tmockClientAsserts func(*pkgUtilKubeMocks.Client) bool\n\t}\n\ttests := []struct {\n\t\tname string\n\t\targs args\n\t\twantErr bool\n\t}{\n\t\t{\n\t\t\tname: \"restore failed to patch status, should retry on connection refused\",\n\t\t\targs: args{\n\t\t\t\tmockClientActions: func(client *pkgUtilKubeMocks.Client) {\n\t\t\t\t\tclient.On(\"Patch\", mock.Anything, mock.Anything, mock.Anything).Return(syscall.ECONNREFUSED).Once()\n\t\t\t\t\tclient.On(\"Patch\", mock.Anything, mock.Anything, mock.Anything).Return(nil)\n\t\t\t\t},\n\t\t\t\tmockClientAsserts: func(client *pkgUtilKubeMocks.Client) bool {\n\t\t\t\t\treturn client.AssertNumberOfCalls(t, \"Patch\", 2)\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"restore failed to patch status, retry on connection refused until max retries\",\n\t\t\targs: args{\n\t\t\t\tmockClientActions: func(client *pkgUtilKubeMocks.Client) {\n\t\t\t\t\tclient.On(\"Patch\", mock.Anything, mock.Anything, mock.Anything).Return(syscall.ECONNREFUSED)\n\t\t\t\t},\n\t\t\t\tmockClientAsserts: func(client *pkgUtilKubeMocks.Client) bool {\n\t\t\t\t\treturn len(client.Calls) > 2\n\t\t\t\t},\n\t\t\t},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"restore patch status ok, should not retry\",\n\t\t\targs: args{\n\t\t\t\tmockClientActions: func(client *pkgUtilKubeMocks.Client) {\n\t\t\t\t\tclient.On(\"Patch\", mock.Anything, mock.Anything, mock.Anything).Return(nil)\n\t\t\t\t},\n\t\t\t\tmockClientAsserts: func(client *pkgUtilKubeMocks.Client) bool {\n\t\t\t\t\treturn client.AssertNumberOfCalls(t, \"Patch\", 1)\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tclient := pkgUtilKubeMocks.NewClient(t)\n\t\t\t// mock client actions\n\t\t\ttt.args.mockClientActions(client)\n\t\t\tr := &restoreFinalizerReconciler{\n\t\t\t\tClient: client,\n\t\t\t\tmetrics: metrics.NewServerMetrics(),\n\t\t\t\tclock: testclocks.NewFakeClock(time.Now()),\n\t\t\t\tresourceTimeout: 1 * time.Second,\n\t\t\t}\n\t\t\trestore := builder.ForRestore(velerov1api.DefaultNamespace, \"restoreName\").Result()\n\t\t\tif err := r.finishProcessing(velerov1api.RestorePhaseInProgress, restore, restore); (err != nil) != tt.wantErr {\n\t\t\t\tt.Errorf(\"restoreFinalizerReconciler.finishProcessing() error = %v, wantErr %v\", err, tt.wantErr)\n\t\t\t}\n\t\t\tif !tt.args.mockClientAsserts(client) {\n\t\t\t\tt.Errorf(\"mockClientAsserts() failed\")\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestRestoreOperationList(t *testing.T) {\n\tvar empty []*itemoperation.RestoreOperation\n\ttests := []struct {\n\t\tname string\n\t\titems []*itemoperation.RestoreOperation\n\t\tinputPVCNS string\n\t\tinputPVCName string\n\t\texpected []*itemoperation.RestoreOperation\n\t}{\n\t\t{\n\t\t\tname: \"no restore operations\",\n\t\t\titems: []*itemoperation.RestoreOperation{},\n\t\t\tinputPVCNS: \"ns-1\",\n\t\t\tinputPVCName: \"pvc-1\",\n\t\t\texpected: empty,\n\t\t},\n\t\t{\n\t\t\tname: \"one operation with matched info and a nil element\",\n\t\t\titems: []*itemoperation.RestoreOperation{\n\t\t\t\tnil,\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.RestoreOperationSpec{\n\t\t\t\t\t\tRestoreName: \"restore-1\",\n\t\t\t\t\t\tRestoreUID: \"uid-1\",\n\t\t\t\t\t\tRestoreItemAction: \"velero.io/csi-pvc-restorer\",\n\t\t\t\t\t\tOperationID: \"dd-abbb048d-7036-4855-bf50-ebba978b59a6.2426dd0e-b863-4222b5b2b\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: schema.GroupResource{\n\t\t\t\t\t\t\t\tGroup: \"\",\n\t\t\t\t\t\t\t\tResource: \"persistentvolumeclaims\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pvc-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: itemoperation.OperationPhaseCompleted,\n\t\t\t\t\t\tOperationUnits: \"Byte\",\n\t\t\t\t\t\tDescription: \"Completed\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tinputPVCNS: \"ns-1\",\n\t\t\tinputPVCName: \"pvc-1\",\n\t\t\texpected: []*itemoperation.RestoreOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.RestoreOperationSpec{\n\t\t\t\t\t\tRestoreName: \"restore-1\",\n\t\t\t\t\t\tRestoreUID: \"uid-1\",\n\t\t\t\t\t\tRestoreItemAction: \"velero.io/csi-pvc-restorer\",\n\t\t\t\t\t\tOperationID: \"dd-abbb048d-7036-4855-bf50-ebba978b59a6.2426dd0e-b863-4222b5b2b\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: schema.GroupResource{\n\t\t\t\t\t\t\t\tGroup: \"\",\n\t\t\t\t\t\t\t\tResource: \"persistentvolumeclaims\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pvc-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: itemoperation.OperationPhaseCompleted,\n\t\t\t\t\t\tOperationUnits: \"Byte\",\n\t\t\t\t\t\tDescription: \"Completed\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"one operation with incorrect resource type\",\n\t\t\titems: []*itemoperation.RestoreOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.RestoreOperationSpec{\n\t\t\t\t\t\tRestoreName: \"restore-1\",\n\t\t\t\t\t\tRestoreUID: \"uid-1\",\n\t\t\t\t\t\tRestoreItemAction: \"velero.io/csi-pvc-restorer\",\n\t\t\t\t\t\tOperationID: \"dd-abbb048d-7036-4855-bf50-ebba978b59a6.2426dd0e-b863-4222b5b2b\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: schema.GroupResource{\n\t\t\t\t\t\t\t\tGroup: \"\",\n\t\t\t\t\t\t\t\tResource: \"configmaps\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pvc-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: itemoperation.OperationPhaseCompleted,\n\t\t\t\t\t\tOperationUnits: \"Byte\",\n\t\t\t\t\t\tDescription: \"Completed\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tinputPVCNS: \"ns-1\",\n\t\t\tinputPVCName: \"pvc-1\",\n\t\t\texpected: empty,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tl := restoreItemOperationList{\n\t\t\t\titems: tt.items,\n\t\t\t}\n\t\t\tassert.Equal(t, tt.expected, l.SelectByPVC(tt.inputPVCNS, tt.inputPVCName))\n\t\t})\n\t}\n}\n" }, { "path": "pkg/controller/restore_operations_controller.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\tclocks \"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/builder\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/predicate\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperationmap\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\t\"github.com/vmware-tanzu/velero/pkg/persistence\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nconst (\n\tdefaultRestoreOperationsFrequency = 10 * time.Second\n)\n\ntype restoreOperationsReconciler struct {\n\tclient.Client\n\tnamespace string\n\tlogger logrus.FieldLogger\n\tclock clocks.WithTickerAndDelayedExecution\n\tfrequency time.Duration\n\titemOperationsMap *itemoperationmap.RestoreItemOperationsMap\n\tnewPluginManager func(logger logrus.FieldLogger) clientmgmt.Manager\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter\n\tmetrics *metrics.ServerMetrics\n}\n\nfunc NewRestoreOperationsReconciler(\n\tlogger logrus.FieldLogger,\n\tnamespace string,\n\tclient client.Client,\n\tfrequency time.Duration,\n\tnewPluginManager func(logrus.FieldLogger) clientmgmt.Manager,\n\tbackupStoreGetter persistence.ObjectBackupStoreGetter,\n\tmetrics *metrics.ServerMetrics,\n\titemOperationsMap *itemoperationmap.RestoreItemOperationsMap,\n) *restoreOperationsReconciler {\n\tabor := &restoreOperationsReconciler{\n\t\tClient: client,\n\t\tlogger: logger,\n\t\tnamespace: namespace,\n\t\tclock: clocks.RealClock{},\n\t\tfrequency: frequency,\n\t\titemOperationsMap: itemOperationsMap,\n\t\tnewPluginManager: newPluginManager,\n\t\tbackupStoreGetter: backupStoreGetter,\n\t\tmetrics: metrics,\n\t}\n\tif abor.frequency <= 0 {\n\t\tabor.frequency = defaultRestoreOperationsFrequency\n\t}\n\treturn abor\n}\n\nfunc (r *restoreOperationsReconciler) SetupWithManager(mgr ctrl.Manager) error {\n\tgp := kube.NewGenericEventPredicate(func(object client.Object) bool {\n\t\trestore := object.(*velerov1api.Restore)\n\t\treturn (restore.Status.Phase == velerov1api.RestorePhaseWaitingForPluginOperations ||\n\t\t\trestore.Status.Phase == velerov1api.RestorePhaseWaitingForPluginOperationsPartiallyFailed)\n\t})\n\ts := kube.NewPeriodicalEnqueueSource(r.logger.WithField(\"controller\", constant.ControllerRestoreOperations), mgr.GetClient(), &velerov1api.RestoreList{}, r.frequency, kube.PeriodicalEnqueueSourceOption{\n\t\tPredicates: []predicate.Predicate{gp},\n\t})\n\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\tFor(&velerov1api.Restore{}, builder.WithPredicates(kube.FalsePredicate{})).\n\t\tWatchesRawSource(s).\n\t\tNamed(constant.ControllerRestoreOperations).\n\t\tComplete(r)\n}\n\n// +kubebuilder:rbac:groups=velero.io,resources=restores,verbs=get;list;watch;update\n// +kubebuilder:rbac:groups=velero.io,resources=restores/status,verbs=get\n\nfunc (r *restoreOperationsReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\tlog := r.logger.WithField(\"restore operations for restore\", req.String())\n\tlog.Debug(\"restoreOperationsReconciler getting restore\")\n\n\toriginal := &velerov1api.Restore{}\n\tif err := r.Get(ctx, req.NamespacedName, original); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.WithError(err).Error(\"restore not found\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\treturn ctrl.Result{}, errors.Wrapf(err, \"error getting restore %s\", req.String())\n\t}\n\trestore := original.DeepCopy()\n\tlog.Debugf(\"restore: %s\", restore.Name)\n\n\tlog = r.logger.WithFields(\n\t\tlogrus.Fields{\n\t\t\t\"restore\": req.String(),\n\t\t},\n\t)\n\n\tswitch restore.Status.Phase {\n\tcase velerov1api.RestorePhaseWaitingForPluginOperations, velerov1api.RestorePhaseWaitingForPluginOperationsPartiallyFailed:\n\t\t// only process restores waiting for plugin operations to complete\n\tdefault:\n\t\tlog.Debug(\"Restore has no ongoing plugin operations, skipping\")\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\tinfo, err := r.fetchBackupInfo(restore.Spec.BackupName)\n\tif err != nil {\n\t\tlog.Warnf(\"Cannot check progress on Restore operations because backup info is unavailable %s; marking restore FinalizingPartiallyFailed\", err.Error())\n\t\trestore.Status.Phase = velerov1api.RestorePhaseFinalizingPartiallyFailed\n\t\terr2 := r.updateRestoreAndOperationsJSON(ctx, original, restore, nil, &itemoperationmap.OperationsForRestore{ErrsSinceUpdate: []string{err.Error()}}, false, false)\n\t\tif err2 != nil {\n\t\t\tlog.WithError(err2).Error(\"error updating Restore\")\n\t\t}\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error getting backup info\")\n\t}\n\n\tpluginManager := r.newPluginManager(r.logger)\n\tdefer pluginManager.CleanupClients()\n\tbackupStore, err := r.backupStoreGetter.Get(info.location, pluginManager, r.logger)\n\tif err != nil {\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error getting backup store\")\n\t}\n\n\toperations, err := r.itemOperationsMap.GetOperationsForRestore(backupStore, restore.Name)\n\tif err != nil {\n\t\terr2 := r.updateRestoreAndOperationsJSON(ctx, original, restore, backupStore, &itemoperationmap.OperationsForRestore{ErrsSinceUpdate: []string{err.Error()}}, false, false)\n\t\tif err2 != nil {\n\t\t\treturn ctrl.Result{}, errors.Wrap(err2, \"error updating Restore\")\n\t\t}\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error getting restore operations\")\n\t}\n\tstillInProgress, changes, opsCompleted, opsFailed, errs := getRestoreItemOperationProgress(restore, pluginManager, operations.Operations)\n\t// if len(errs)>0, need to update restore errors and error log\n\toperations.ErrsSinceUpdate = append(operations.ErrsSinceUpdate, errs...)\n\trestore.Status.Errors += len(operations.ErrsSinceUpdate)\n\tcompletionChanges := false\n\tif restore.Status.RestoreItemOperationsCompleted != opsCompleted || restore.Status.RestoreItemOperationsFailed != opsFailed {\n\t\tcompletionChanges = true\n\t\trestore.Status.RestoreItemOperationsCompleted = opsCompleted\n\t\trestore.Status.RestoreItemOperationsFailed = opsFailed\n\t}\n\tif changes {\n\t\toperations.ChangesSinceUpdate = true\n\t}\n\n\tif len(operations.ErrsSinceUpdate) > 0 {\n\t\trestore.Status.Phase = velerov1api.RestorePhaseWaitingForPluginOperationsPartiallyFailed\n\t}\n\n\t// if stillInProgress is false, restore moves to terminal phase and needs update\n\t// if operations.ErrsSinceUpdate is not empty, then restore phase needs to change to\n\t// RestorePhaseWaitingForPluginOperationsPartiallyFailed and needs update\n\t// If the only changes are incremental progress, then no write is necessary, progress can remain in memory\n\tif !stillInProgress {\n\t\tif restore.Status.Phase == velerov1api.RestorePhaseWaitingForPluginOperations {\n\t\t\tlog.Infof(\"Marking restore %s Finalizing\", restore.Name)\n\t\t\trestore.Status.Phase = velerov1api.RestorePhaseFinalizing\n\t\t} else {\n\t\t\tlog.Infof(\"Marking restore %s FinalizingPartiallyFailed\", restore.Name)\n\t\t\trestore.Status.Phase = velerov1api.RestorePhaseFinalizingPartiallyFailed\n\t\t}\n\t}\n\terr = r.updateRestoreAndOperationsJSON(ctx, original, restore, backupStore, operations, changes, completionChanges)\n\tif err != nil {\n\t\treturn ctrl.Result{}, errors.Wrap(err, \"error updating Restore\")\n\t}\n\treturn ctrl.Result{}, nil\n}\n\n// fetchBackupInfo checks the backup lister for a backup that matches the given name. If it doesn't\n// find it, it returns an error.\nfunc (r *restoreOperationsReconciler) fetchBackupInfo(backupName string) (backupInfo, error) {\n\treturn fetchBackupInfoInternal(r.Client, r.namespace, backupName)\n}\n\nfunc (r *restoreOperationsReconciler) updateRestoreAndOperationsJSON(\n\tctx context.Context,\n\toriginal, restore *velerov1api.Restore,\n\tbackupStore persistence.BackupStore,\n\toperations *itemoperationmap.OperationsForRestore,\n\tchanges bool,\n\tcompletionChanges bool) error {\n\tif len(operations.ErrsSinceUpdate) > 0 {\n\t\t// FIXME: download/upload results\n\t\tr.logger.WithField(\"restore\", restore.Name).Infof(\"Restore has %d errors\", len(operations.ErrsSinceUpdate))\n\t}\n\tremoveIfComplete := true\n\tdefer func() {\n\t\t// remove local operations list if complete\n\t\tif removeIfComplete && (restore.Status.Phase == velerov1api.RestorePhaseFinalizing ||\n\t\t\trestore.Status.Phase == velerov1api.RestorePhaseFinalizingPartiallyFailed) {\n\t\t\tr.itemOperationsMap.DeleteOperationsForRestore(restore.Name)\n\t\t} else if changes {\n\t\t\tr.itemOperationsMap.PutOperationsForRestore(operations, restore.Name)\n\t\t}\n\t}()\n\n\t// update restore and upload progress if errs or complete\n\tif len(operations.ErrsSinceUpdate) > 0 ||\n\t\trestore.Status.Phase == velerov1api.RestorePhaseFinalizing ||\n\t\trestore.Status.Phase == velerov1api.RestorePhaseFinalizingPartiallyFailed {\n\t\t// update file store\n\t\tif backupStore != nil {\n\t\t\tif err := r.itemOperationsMap.UploadProgressAndPutOperationsForRestore(backupStore, operations, restore.Name); err != nil {\n\t\t\t\tremoveIfComplete = false\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t\t// update restore\n\t\terr := r.Client.Patch(ctx, restore, client.MergeFrom(original))\n\t\tif err != nil {\n\t\t\tremoveIfComplete = false\n\t\t\treturn errors.Wrapf(err, \"error updating Restore %s\", restore.Name)\n\t\t}\n\t} else if completionChanges {\n\t\t// If restore is still incomplete and no new errors are found but there are some new operations\n\t\t// completed, patch restore to reflect new completion numbers, but don't upload detailed json file\n\t\terr := r.Client.Patch(ctx, restore, client.MergeFrom(original))\n\t\tif err != nil {\n\t\t\treturn errors.Wrapf(err, \"error updating Restore %s\", restore.Name)\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc getRestoreItemOperationProgress(\n\trestore *velerov1api.Restore,\n\tpluginManager clientmgmt.Manager,\n\toperationsList []*itemoperation.RestoreOperation) (bool, bool, int, int, []string) {\n\tinProgressOperations := false\n\tchanges := false\n\tvar errs []string\n\tvar completedCount, failedCount int\n\n\tfor _, operation := range operationsList {\n\t\tif operation.Status.Phase == itemoperation.OperationPhaseNew ||\n\t\t\toperation.Status.Phase == itemoperation.OperationPhaseInProgress {\n\t\t\tria, err := pluginManager.GetRestoreItemActionV2(operation.Spec.RestoreItemAction)\n\t\t\tif err != nil {\n\t\t\t\toperation.Status.Phase = itemoperation.OperationPhaseFailed\n\t\t\t\toperation.Status.Error = err.Error()\n\t\t\t\terrs = append(errs, err.Error())\n\t\t\t\tchanges = true\n\t\t\t\tfailedCount++\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\toperationProgress, err := ria.Progress(operation.Spec.OperationID, restore)\n\t\t\tif err != nil {\n\t\t\t\toperation.Status.Phase = itemoperation.OperationPhaseFailed\n\t\t\t\toperation.Status.Error = err.Error()\n\t\t\t\terrs = append(errs, err.Error())\n\t\t\t\tchanges = true\n\t\t\t\tfailedCount++\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif operation.Status.NCompleted != operationProgress.NCompleted {\n\t\t\t\toperation.Status.NCompleted = operationProgress.NCompleted\n\t\t\t\tchanges = true\n\t\t\t}\n\t\t\tif operation.Status.NTotal != operationProgress.NTotal {\n\t\t\t\toperation.Status.NTotal = operationProgress.NTotal\n\t\t\t\tchanges = true\n\t\t\t}\n\t\t\tif operation.Status.OperationUnits != operationProgress.OperationUnits {\n\t\t\t\toperation.Status.OperationUnits = operationProgress.OperationUnits\n\t\t\t\tchanges = true\n\t\t\t}\n\t\t\tif operation.Status.Description != operationProgress.Description {\n\t\t\t\toperation.Status.Description = operationProgress.Description\n\t\t\t\tchanges = true\n\t\t\t}\n\t\t\tstarted := metav1.NewTime(operationProgress.Started)\n\t\t\tif operation.Status.Started == nil && !operationProgress.Started.IsZero() ||\n\t\t\t\toperation.Status.Started != nil && *(operation.Status.Started) != started {\n\t\t\t\toperation.Status.Started = &started\n\t\t\t\tchanges = true\n\t\t\t}\n\t\t\tupdated := metav1.NewTime(operationProgress.Updated)\n\t\t\tif operation.Status.Updated == nil && !operationProgress.Updated.IsZero() ||\n\t\t\t\toperation.Status.Updated != nil && *(operation.Status.Updated) != updated {\n\t\t\t\toperation.Status.Updated = &updated\n\t\t\t\tchanges = true\n\t\t\t}\n\n\t\t\tif operationProgress.Completed {\n\t\t\t\tif operationProgress.Err != \"\" {\n\t\t\t\t\toperation.Status.Phase = itemoperation.OperationPhaseFailed\n\t\t\t\t\toperation.Status.Error = operationProgress.Err\n\t\t\t\t\terrs = append(errs, operationProgress.Err)\n\t\t\t\t\tchanges = true\n\t\t\t\t\tfailedCount++\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\toperation.Status.Phase = itemoperation.OperationPhaseCompleted\n\t\t\t\tchanges = true\n\t\t\t\tcompletedCount++\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\t// cancel operation if past timeout period\n\t\t\tif operation.Status.Created.Time.Add(restore.Spec.ItemOperationTimeout.Duration).Before(time.Now()) {\n\t\t\t\t_ = ria.Cancel(operation.Spec.OperationID, restore)\n\t\t\t\toperation.Status.Phase = itemoperation.OperationPhaseFailed\n\t\t\t\toperation.Status.Error = \"Asynchronous action timed out\"\n\t\t\t\terrs = append(errs, operation.Status.Error)\n\t\t\t\tchanges = true\n\t\t\t\tfailedCount++\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif operation.Status.Phase == itemoperation.OperationPhaseNew &&\n\t\t\t\toperation.Status.Started != nil {\n\t\t\t\toperation.Status.Phase = itemoperation.OperationPhaseInProgress\n\t\t\t\tchanges = true\n\t\t\t}\n\t\t\t// if we reach this point, the operation is still running\n\t\t\tinProgressOperations = true\n\t\t} else if operation.Status.Phase == itemoperation.OperationPhaseCompleted {\n\t\t\tcompletedCount++\n\t\t} else if operation.Status.Phase == itemoperation.OperationPhaseFailed {\n\t\t\tfailedCount++\n\t\t}\n\t}\n\treturn inProgressOperations, changes, completedCount, failedCount, errs\n}\n" }, { "path": "pkg/controller/restore_operations_controller_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\ttestclocks \"k8s.io/utils/clock/testing\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperationmap\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\tpersistencemocks \"github.com/vmware-tanzu/velero/pkg/persistence/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt\"\n\tpluginmocks \"github.com/vmware-tanzu/velero/pkg/plugin/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\triav2mocks \"github.com/vmware-tanzu/velero/pkg/plugin/velero/mocks/restoreitemaction/v2\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nvar (\n\trestorePluginManager = &pluginmocks.Manager{}\n\trestoreBackupStore = &persistencemocks.BackupStore{}\n\tria = &riav2mocks.RestoreItemAction{}\n)\n\nfunc mockRestoreOperationsReconciler(fakeClient kbclient.Client, fakeClock *testclocks.FakeClock, freq time.Duration) *restoreOperationsReconciler {\n\tabor := NewRestoreOperationsReconciler(\n\t\tlogrus.StandardLogger(),\n\t\tvelerov1api.DefaultNamespace,\n\t\tfakeClient,\n\t\tfreq,\n\t\tfunc(logrus.FieldLogger) clientmgmt.Manager { return restorePluginManager },\n\t\tNewFakeSingleObjectBackupStoreGetter(restoreBackupStore),\n\t\tmetrics.NewServerMetrics(),\n\t\titemoperationmap.NewRestoreItemOperationsMap(),\n\t)\n\tabor.clock = fakeClock\n\treturn abor\n}\n\nfunc TestRestoreOperationsReconcile(t *testing.T) {\n\tfakeClock := testclocks.NewFakeClock(time.Now())\n\tmetav1Now := metav1.NewTime(fakeClock.Now())\n\n\tdefaultBackupLocation := builder.ForBackupStorageLocation(velerov1api.DefaultNamespace, \"default\").Result()\n\n\ttests := []struct {\n\t\tname string\n\t\trestore *velerov1api.Restore\n\t\trestoreOperations []*itemoperation.RestoreOperation\n\t\tbackup *velerov1api.Backup\n\t\tbackupLocation *velerov1api.BackupStorageLocation\n\t\toperationComplete bool\n\t\toperationErr string\n\t\texpectError bool\n\t\texpectPhase velerov1api.RestorePhase\n\t}{\n\t\t{\n\t\t\tname: \"WaitingForPluginOperations restore with completed operations is Completed\",\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"restore-11\").\n\t\t\t\tBackup(\"backup-1\").\n\t\t\t\tItemOperationTimeout(60 * time.Minute).\n\t\t\t\tObjectMeta(builder.WithUID(\"foo-11\")).\n\t\t\t\tPhase(velerov1api.RestorePhaseWaitingForPluginOperations).Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\toperationComplete: true,\n\t\t\texpectPhase: velerov1api.RestorePhaseFinalizing,\n\t\t\trestoreOperations: []*itemoperation.RestoreOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.RestoreOperationSpec{\n\t\t\t\t\t\tRestoreName: \"restore-11\",\n\t\t\t\t\t\tRestoreUID: \"foo-11\",\n\t\t\t\t\t\tRestoreItemAction: \"foo-11\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tOperationID: \"operation-11\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: itemoperation.OperationPhaseInProgress,\n\t\t\t\t\t\tCreated: &metav1Now,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"WaitingForPluginOperations restore with incomplete operations is still incomplete\",\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"restore-12\").\n\t\t\t\tBackup(\"backup-1\").\n\t\t\t\tItemOperationTimeout(60 * time.Minute).\n\t\t\t\tObjectMeta(builder.WithUID(\"foo-12\")).\n\t\t\t\tPhase(velerov1api.RestorePhaseWaitingForPluginOperations).Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\toperationComplete: false,\n\t\t\texpectPhase: velerov1api.RestorePhaseWaitingForPluginOperations,\n\t\t\trestoreOperations: []*itemoperation.RestoreOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.RestoreOperationSpec{\n\t\t\t\t\t\tRestoreName: \"restore-12\",\n\t\t\t\t\t\tRestoreUID: \"foo-12\",\n\t\t\t\t\t\tRestoreItemAction: \"foo-12\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tOperationID: \"operation-12\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: itemoperation.OperationPhaseInProgress,\n\t\t\t\t\t\tCreated: &metav1Now,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"WaitingForPluginOperations restore with completed failed operations is PartiallyFailed\",\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"restore-13\").\n\t\t\t\tBackup(\"backup-1\").\n\t\t\t\tItemOperationTimeout(60 * time.Minute).\n\t\t\t\tObjectMeta(builder.WithUID(\"foo-13\")).\n\t\t\t\tPhase(velerov1api.RestorePhaseWaitingForPluginOperations).Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\toperationComplete: true,\n\t\t\toperationErr: \"failed\",\n\t\t\texpectPhase: velerov1api.RestorePhaseFinalizingPartiallyFailed,\n\t\t\trestoreOperations: []*itemoperation.RestoreOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.RestoreOperationSpec{\n\t\t\t\t\t\tRestoreName: \"restore-13\",\n\t\t\t\t\t\tRestoreUID: \"foo-13\",\n\t\t\t\t\t\tRestoreItemAction: \"foo-13\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tOperationID: \"operation-13\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: itemoperation.OperationPhaseInProgress,\n\t\t\t\t\t\tCreated: &metav1Now,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"WaitingForPluginOperationsPartiallyFailed restore with completed operations is PartiallyFailed\",\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"restore-14\").\n\t\t\t\tBackup(\"backup-1\").\n\t\t\t\tItemOperationTimeout(60 * time.Minute).\n\t\t\t\tObjectMeta(builder.WithUID(\"foo-14\")).\n\t\t\t\tPhase(velerov1api.RestorePhaseWaitingForPluginOperationsPartiallyFailed).Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\toperationComplete: true,\n\t\t\texpectPhase: velerov1api.RestorePhaseFinalizingPartiallyFailed,\n\t\t\trestoreOperations: []*itemoperation.RestoreOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.RestoreOperationSpec{\n\t\t\t\t\t\tRestoreName: \"restore-14\",\n\t\t\t\t\t\tRestoreUID: \"foo-14\",\n\t\t\t\t\t\tRestoreItemAction: \"foo-14\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tOperationID: \"operation-14\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: itemoperation.OperationPhaseInProgress,\n\t\t\t\t\t\tCreated: &metav1Now,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"WaitingForPluginOperationsPartiallyFailed restore with incomplete operations is still incomplete\",\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"restore-15\").\n\t\t\t\tBackup(\"backup-1\").\n\t\t\t\tItemOperationTimeout(60 * time.Minute).\n\t\t\t\tObjectMeta(builder.WithUID(\"foo-15\")).\n\t\t\t\tPhase(velerov1api.RestorePhaseWaitingForPluginOperationsPartiallyFailed).Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\toperationComplete: false,\n\t\t\texpectPhase: velerov1api.RestorePhaseWaitingForPluginOperationsPartiallyFailed,\n\t\t\trestoreOperations: []*itemoperation.RestoreOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.RestoreOperationSpec{\n\t\t\t\t\t\tRestoreName: \"restore-15\",\n\t\t\t\t\t\tRestoreUID: \"foo-15\",\n\t\t\t\t\t\tRestoreItemAction: \"foo-15\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tOperationID: \"operation-15\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: itemoperation.OperationPhaseInProgress,\n\t\t\t\t\t\tCreated: &metav1Now,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"WaitingForPluginOperationsPartiallyFailed restore with completed failed operations is PartiallyFailed\",\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"restore-16\").\n\t\t\t\tBackup(\"backup-1\").\n\t\t\t\tItemOperationTimeout(60 * time.Minute).\n\t\t\t\tObjectMeta(builder.WithUID(\"foo-16\")).\n\t\t\t\tPhase(velerov1api.RestorePhaseWaitingForPluginOperationsPartiallyFailed).Result(),\n\t\t\tbackup: defaultBackup().StorageLocation(\"default\").Result(),\n\t\t\tbackupLocation: defaultBackupLocation,\n\t\t\toperationComplete: true,\n\t\t\toperationErr: \"failed\",\n\t\t\texpectPhase: velerov1api.RestorePhaseFinalizingPartiallyFailed,\n\t\t\trestoreOperations: []*itemoperation.RestoreOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.RestoreOperationSpec{\n\t\t\t\t\t\tRestoreName: \"restore-16\",\n\t\t\t\t\t\tRestoreUID: \"foo-16\",\n\t\t\t\t\t\tRestoreItemAction: \"foo-16\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tOperationID: \"operation-16\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: itemoperation.OperationPhaseInProgress,\n\t\t\t\t\t\tCreated: &metav1Now,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tif test.restore == nil {\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tinitObjs := []runtime.Object{}\n\t\t\tinitObjs = append(initObjs, test.restore)\n\t\t\tinitObjs = append(initObjs, test.backup)\n\n\t\t\tif test.backupLocation != nil {\n\t\t\t\tinitObjs = append(initObjs, test.backupLocation)\n\t\t\t}\n\n\t\t\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, initObjs...)\n\t\t\treconciler := mockRestoreOperationsReconciler(fakeClient, fakeClock, defaultRestoreOperationsFrequency)\n\t\t\trestorePluginManager.On(\"CleanupClients\").Return(nil)\n\t\t\trestoreBackupStore.On(\"GetRestoreItemOperations\", test.restore.Name).Return(test.restoreOperations, nil)\n\t\t\trestoreBackupStore.On(\"PutRestoreItemOperations\", mock.Anything, mock.Anything).Return(nil)\n\t\t\trestoreBackupStore.On(\"PutRestoreMetadata\", mock.Anything, mock.Anything).Return(nil)\n\t\t\tfor _, operation := range test.restoreOperations {\n\t\t\t\tria.On(\"Progress\", operation.Spec.OperationID, mock.Anything).\n\t\t\t\t\tReturn(velero.OperationProgress{\n\t\t\t\t\t\tCompleted: test.operationComplete,\n\t\t\t\t\t\tErr: test.operationErr,\n\t\t\t\t\t}, nil)\n\t\t\t\trestorePluginManager.On(\"GetRestoreItemActionV2\", operation.Spec.RestoreItemAction).Return(ria, nil)\n\t\t\t}\n\n\t\t\t_, err := reconciler.Reconcile(t.Context(), ctrl.Request{NamespacedName: types.NamespacedName{Namespace: test.restore.Namespace, Name: test.restore.Name}})\n\t\t\tgotErr := err != nil\n\t\t\tassert.Equal(t, test.expectError, gotErr)\n\n\t\t\trestoreAfter := velerov1api.Restore{}\n\t\t\terr = fakeClient.Get(t.Context(), types.NamespacedName{\n\t\t\t\tNamespace: test.restore.Namespace,\n\t\t\t\tName: test.restore.Name,\n\t\t\t}, &restoreAfter)\n\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Equal(t, test.expectPhase, restoreAfter.Status.Phase)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/controller/schedule_controller.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\tcron \"github.com/robfig/cron/v3\"\n\t\"github.com/sirupsen/logrus\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\tclocks \"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\tbld \"sigs.k8s.io/controller-runtime/pkg/builder\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/predicate\"\n\n\tvelerov1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nconst (\n\tscheduleSyncPeriod = time.Minute\n)\n\ntype scheduleReconciler struct {\n\tclient.Client\n\tnamespace string\n\tlogger logrus.FieldLogger\n\tclock clocks.WithTickerAndDelayedExecution\n\tmetrics *metrics.ServerMetrics\n\tskipImmediately bool\n}\n\nfunc NewScheduleReconciler(\n\tnamespace string,\n\tlogger logrus.FieldLogger,\n\tclient client.Client,\n\tmetrics *metrics.ServerMetrics,\n\tskipImmediately bool,\n) *scheduleReconciler {\n\treturn &scheduleReconciler{\n\t\tClient: client,\n\t\tnamespace: namespace,\n\t\tlogger: logger,\n\t\tclock: clocks.RealClock{},\n\t\tmetrics: metrics,\n\t\tskipImmediately: skipImmediately,\n\t}\n}\n\nfunc (c *scheduleReconciler) SetupWithManager(mgr ctrl.Manager) error {\n\tpred := kube.NewAllEventPredicate(func(obj client.Object) bool {\n\t\tschedule := obj.(*velerov1.Schedule)\n\t\tif pause := schedule.Spec.Paused; pause {\n\t\t\tc.logger.Infof(\"schedule %s is paused, skip\", schedule.Name)\n\t\t\treturn false\n\t\t}\n\t\treturn true\n\t})\n\ts := kube.NewPeriodicalEnqueueSource(c.logger.WithField(\"controller\", constant.ControllerSchedule), mgr.GetClient(), &velerov1.ScheduleList{}, scheduleSyncPeriod,\n\t\tkube.PeriodicalEnqueueSourceOption{\n\t\t\tPredicates: []predicate.Predicate{pred},\n\t\t})\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\tFor(&velerov1.Schedule{}, bld.WithPredicates(kube.SpecChangePredicate{}, pred)).\n\t\tWatchesRawSource(s).\n\t\tComplete(c)\n}\n\n// +kubebuilder:rbac:groups=velero.io,resources=schedules,verbs=get;list;watch;create;update;patch;delete\n// +kubebuilder:rbac:groups=velero.io,resources=schedules/status,verbs=get;update;patch\n// +kubebuilder:rbac:groups=velero.io,resources=backups,verbs=create\n\nfunc (c *scheduleReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\tlog := c.logger.WithField(\"schedule\", req.String())\n\n\tlog.Debug(\"Getting schedule\")\n\tschedule := &velerov1.Schedule{}\n\tif err := c.Get(ctx, req.NamespacedName, schedule); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.WithError(err).Error(\"schedule not found\")\n\t\t\tc.metrics.RemoveSchedule(req.Name)\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\t\treturn ctrl.Result{}, errors.Wrapf(err, \"error getting schedule %s\", req.String())\n\t}\n\tc.metrics.InitSchedule(schedule.Name)\n\n\toriginal := schedule.DeepCopy()\n\n\tif schedule.Spec.SkipImmediately == nil {\n\t\tschedule.Spec.SkipImmediately = &c.skipImmediately\n\t}\n\tif schedule.Spec.SkipImmediately != nil && *schedule.Spec.SkipImmediately {\n\t\t*schedule.Spec.SkipImmediately = false\n\t\tschedule.Status.LastSkipped = &metav1.Time{Time: c.clock.Now()}\n\t}\n\n\t// validation - even if the item is Enabled, we can't trust it\n\t// so re-validate\n\tcurrentPhase := schedule.Status.Phase\n\n\tcronSchedule, errs := parseCronSchedule(schedule, c.logger)\n\tif len(errs) > 0 {\n\t\tschedule.Status.Phase = velerov1.SchedulePhaseFailedValidation\n\t\tschedule.Status.ValidationErrors = errs\n\t} else {\n\t\tschedule.Status.Phase = velerov1.SchedulePhaseEnabled\n\t\tschedule.Status.ValidationErrors = nil\n\n\t\t// Compute expected interval between consecutive scheduled backup runs.\n\t\t// Only meaningful when the cron expression is valid.\n\t\tnow := c.clock.Now()\n\t\tnextRun := cronSchedule.Next(now)\n\t\tnextNextRun := cronSchedule.Next(nextRun)\n\t\tc.metrics.SetScheduleExpectedIntervalSeconds(schedule.Name, nextNextRun.Sub(nextRun).Seconds())\n\t}\n\n\tscheduleNeedsPatch := false\n\terrStringArr := make([]string, 0)\n\tif currentPhase != schedule.Status.Phase {\n\t\tscheduleNeedsPatch = true\n\t\terrStringArr = append(errStringArr, fmt.Sprintf(\"phase to %s\", schedule.Status.Phase))\n\t}\n\t// update spec.SkipImmediately if it's changed\n\tif original.Spec.SkipImmediately != schedule.Spec.SkipImmediately {\n\t\tscheduleNeedsPatch = true\n\t\terrStringArr = append(errStringArr, fmt.Sprintf(\"spec.skipImmediately to %v\", schedule.Spec.SkipImmediately))\n\t}\n\t// update status if it's changed\n\tif original.Status.LastSkipped != schedule.Status.LastSkipped {\n\t\tscheduleNeedsPatch = true\n\t\terrStringArr = append(errStringArr, fmt.Sprintf(\"last skipped to %v\", schedule.Status.LastSkipped))\n\t}\n\tif scheduleNeedsPatch {\n\t\tif err := c.Patch(ctx, schedule, client.MergeFrom(original)); err != nil {\n\t\t\treturn ctrl.Result{}, errors.Wrapf(err, \"error updating %v for schedule %s\", errStringArr, req.String())\n\t\t}\n\t}\n\n\tif schedule.Status.Phase != velerov1.SchedulePhaseEnabled {\n\t\tlog.Debugf(\"the schedule's phase is %s, isn't %s, skip\", schedule.Status.Phase, velerov1.SchedulePhaseEnabled)\n\t\treturn ctrl.Result{}, nil\n\t}\n\n\t// Check for the schedule being due to run.\n\t// If there are backup created by this schedule still in New or InProgress state,\n\t// skip current backup creation to avoid running overlap backups.\n\t// As the schedule must be validated before checking whether it's due, we cannot put the checking log in Predicate\n\tif c.ifDue(schedule, cronSchedule) && !c.checkIfBackupInNewOrProgress(schedule) {\n\t\tif err := c.submitBackup(ctx, schedule); err != nil {\n\t\t\treturn ctrl.Result{}, errors.Wrapf(err, \"error submit backup for schedule %s\", req.String())\n\t\t}\n\t}\n\n\treturn ctrl.Result{}, nil\n}\n\nfunc parseCronSchedule(itm *velerov1.Schedule, logger logrus.FieldLogger) (cron.Schedule, []string) {\n\tvar validationErrors []string\n\tvar schedule cron.Schedule\n\n\t// cron.Parse panics if schedule is empty\n\tif len(itm.Spec.Schedule) == 0 {\n\t\tvalidationErrors = append(validationErrors, \"Schedule must be a non-empty valid Cron expression\")\n\t\treturn nil, validationErrors\n\t}\n\n\tlog := logger.WithField(\"schedule\", kube.NamespaceAndName(itm))\n\n\t// adding a recover() around cron.Parse because it panics on empty string and is possible\n\t// that it panics under other scenarios as well.\n\tfunc() {\n\t\tdefer func() {\n\t\t\tif r := recover(); r != nil {\n\t\t\t\tlog.WithFields(logrus.Fields{\n\t\t\t\t\t\"schedule\": itm.Spec.Schedule,\n\t\t\t\t\t\"recover\": r,\n\t\t\t\t}).Debug(\"Panic parsing schedule\")\n\t\t\t\tvalidationErrors = append(validationErrors, fmt.Sprintf(\"invalid schedule: %v\", r))\n\t\t\t}\n\t\t}()\n\n\t\tif res, err := cron.ParseStandard(itm.Spec.Schedule); err != nil {\n\t\t\tlog.WithError(errors.WithStack(err)).WithField(\"schedule\", itm.Spec.Schedule).Debug(\"Error parsing schedule\")\n\t\t\tvalidationErrors = append(validationErrors, fmt.Sprintf(\"invalid schedule: %v\", err))\n\t\t} else {\n\t\t\tschedule = res\n\t\t}\n\t}()\n\n\tif len(validationErrors) > 0 {\n\t\treturn nil, validationErrors\n\t}\n\n\treturn schedule, nil\n}\n\n// checkIfBackupInNewOrProgress check whether there are backups created by this schedule still in New or InProgress state\nfunc (c *scheduleReconciler) checkIfBackupInNewOrProgress(schedule *velerov1.Schedule) bool {\n\tlog := c.logger.WithField(\"schedule\", kube.NamespaceAndName(schedule))\n\tbackupList := &velerov1.BackupList{}\n\toptions := &client.ListOptions{\n\t\tNamespace: schedule.Namespace,\n\t\tLabelSelector: labels.Set(map[string]string{\n\t\t\tvelerov1.ScheduleNameLabel: schedule.Name,\n\t\t}).AsSelector(),\n\t}\n\n\terr := c.List(context.Background(), backupList, options)\n\tif err != nil {\n\t\tlog.Errorf(\"fail to list backup for schedule %s/%s: %s\", schedule.Namespace, schedule.Name, err.Error())\n\t\treturn true\n\t}\n\n\tfor _, backup := range backupList.Items {\n\t\tif backup.Status.Phase == \"\" || backup.Status.Phase == velerov1.BackupPhaseNew || backup.Status.Phase == velerov1.BackupPhaseInProgress {\n\t\t\tlog.Debugf(\"%s/%s still has backups that are in InProgress or New...\", schedule.Namespace, schedule.Name)\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n\n// ifDue check whether schedule is due to create a new backup.\nfunc (c *scheduleReconciler) ifDue(schedule *velerov1.Schedule, cronSchedule cron.Schedule) bool {\n\tisDue, nextRunTime := getNextRunTime(schedule, cronSchedule, c.clock.Now())\n\tlog := c.logger.WithField(\"schedule\", kube.NamespaceAndName(schedule))\n\n\tif !isDue {\n\t\tlog.WithField(\"nextRunTime\", nextRunTime).Debug(\"Schedule is not due, skipping\")\n\t\treturn false\n\t}\n\n\treturn true\n}\n\n// submitBackup create a backup from schedule.\nfunc (c *scheduleReconciler) submitBackup(ctx context.Context, schedule *velerov1.Schedule) error {\n\tc.logger.WithField(\"schedule\", schedule.Namespace+\"/\"+schedule.Name).Info(\"Schedule is due, going to submit backup.\")\n\n\tnow := c.clock.Now()\n\t// Don't attempt to \"catch up\" if there are any missed or failed runs - simply\n\t// trigger a Backup if it's time.\n\tbackup := getBackup(schedule, now)\n\tif err := c.Create(ctx, backup); err != nil {\n\t\treturn errors.Wrap(err, \"error creating Backup\")\n\t}\n\n\toriginal := schedule.DeepCopy()\n\tschedule.Status.LastBackup = &metav1.Time{Time: now}\n\n\tif err := c.Patch(ctx, schedule, client.MergeFrom(original)); err != nil {\n\t\treturn errors.Wrapf(err, \"error updating Schedule's LastBackup time to %v\", schedule.Status.LastBackup)\n\t}\n\n\treturn nil\n}\n\nfunc getNextRunTime(schedule *velerov1.Schedule, cronSchedule cron.Schedule, asOf time.Time) (bool, time.Time) {\n\tvar lastBackupTime time.Time\n\tif schedule.Status.LastBackup != nil {\n\t\tlastBackupTime = schedule.Status.LastBackup.Time\n\t} else {\n\t\tlastBackupTime = schedule.CreationTimestamp.Time\n\t}\n\tif schedule.Status.LastSkipped != nil && schedule.Status.LastSkipped.After(lastBackupTime) {\n\t\tlastBackupTime = schedule.Status.LastSkipped.Time\n\t}\n\n\tnextRunTime := cronSchedule.Next(lastBackupTime)\n\n\treturn asOf.After(nextRunTime), nextRunTime\n}\n\nfunc getBackup(item *velerov1.Schedule, timestamp time.Time) *velerov1.Backup {\n\tname := item.TimestampedName(timestamp)\n\treturn builder.\n\t\tForBackup(item.Namespace, name).\n\t\tFromSchedule(item).\n\t\tResult()\n}\n" }, { "path": "pkg/controller/schedule_controller_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\tcron \"github.com/robfig/cron/v3\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/client-go/kubernetes/scheme\"\n\ttestclocks \"k8s.io/utils/clock/testing\"\n\t\"k8s.io/utils/pointer\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerov1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/metrics\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\n// Test reconcile function of schedule controller. Pause is not covered as event filter will not allow it through\nfunc TestReconcileOfSchedule(t *testing.T) {\n\trequire.NoError(t, velerov1.AddToScheme(scheme.Scheme))\n\n\tnewScheduleBuilder := func(phase velerov1.SchedulePhase) *builder.ScheduleBuilder {\n\t\treturn builder.ForSchedule(\"ns\", \"name\").Phase(phase)\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tscheduleKey string\n\t\tschedule *velerov1.Schedule\n\t\tfakeClockTime string\n\t\texpectedPhase string\n\t\texpectedValidationErrors []string\n\t\texpectedBackupCreate *velerov1.Backup\n\t\texpectedLastBackup string\n\t\texpectedLastSkipped string\n\t\tbackup *velerov1.Backup\n\t\treconcilerSkipImmediately bool\n\t}{\n\t\t{\n\t\t\tname: \"missing schedule triggers no backup\",\n\t\t\tscheduleKey: \"foo/bar\",\n\t\t},\n\t\t{\n\t\t\tname: \"schedule with phase FailedValidation triggers no backup\",\n\t\t\tschedule: newScheduleBuilder(velerov1.SchedulePhaseFailedValidation).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"schedule with phase New gets validated and failed if invalid\",\n\t\t\tschedule: newScheduleBuilder(velerov1.SchedulePhaseNew).Result(),\n\t\t\texpectedPhase: string(velerov1.SchedulePhaseFailedValidation),\n\t\t\texpectedValidationErrors: []string{\"Schedule must be a non-empty valid Cron expression\"},\n\t\t},\n\t\t{\n\t\t\tname: \"schedule with phase gets validated and failed if invalid\",\n\t\t\tschedule: newScheduleBuilder(velerov1.SchedulePhase(\"\")).Result(),\n\t\t\texpectedPhase: string(velerov1.SchedulePhaseFailedValidation),\n\t\t\texpectedValidationErrors: []string{\"Schedule must be a non-empty valid Cron expression\"},\n\t\t},\n\t\t{\n\t\t\tname: \"schedule with phase Enabled gets re-validated and failed if invalid\",\n\t\t\tschedule: newScheduleBuilder(velerov1.SchedulePhaseEnabled).Result(),\n\t\t\texpectedPhase: string(velerov1.SchedulePhaseFailedValidation),\n\t\t\texpectedValidationErrors: []string{\"Schedule must be a non-empty valid Cron expression\"},\n\t\t},\n\t\t{\n\t\t\tname: \"schedule with phase New gets validated and triggers a backup\",\n\t\t\tschedule: newScheduleBuilder(velerov1.SchedulePhaseNew).CronSchedule(\"@every 5m\").Result(),\n\t\t\tfakeClockTime: \"2017-01-01 12:00:00\",\n\t\t\texpectedPhase: string(velerov1.SchedulePhaseEnabled),\n\t\t\texpectedBackupCreate: builder.ForBackup(\"ns\", \"name-20170101120000\").ObjectMeta(builder.WithLabels(velerov1.ScheduleNameLabel, \"name\")).Result(),\n\t\t\texpectedLastBackup: \"2017-01-01 12:00:00\",\n\t\t},\n\t\t{\n\t\t\tname: \"schedule with phase New and SkipImmediately gets validated and does not trigger a backup\",\n\t\t\tschedule: newScheduleBuilder(velerov1.SchedulePhaseNew).CronSchedule(\"@every 5m\").SkipImmediately(pointer.Bool(true)).Result(),\n\t\t\tfakeClockTime: \"2017-01-01 12:00:00\",\n\t\t\texpectedPhase: string(velerov1.SchedulePhaseEnabled),\n\t\t\texpectedLastSkipped: \"2017-01-01 12:00:00\",\n\t\t},\n\t\t{\n\t\t\tname: \"schedule with phase Enabled gets re-validated and triggers a backup if valid\",\n\t\t\tschedule: newScheduleBuilder(velerov1.SchedulePhaseEnabled).CronSchedule(\"@every 5m\").Result(),\n\t\t\tfakeClockTime: \"2017-01-01 12:00:00\",\n\t\t\texpectedPhase: string(velerov1.SchedulePhaseEnabled),\n\t\t\texpectedBackupCreate: builder.ForBackup(\"ns\", \"name-20170101120000\").ObjectMeta(builder.WithLabels(velerov1.ScheduleNameLabel, \"name\")).Result(),\n\t\t\texpectedLastBackup: \"2017-01-01 12:00:00\",\n\t\t},\n\t\t{\n\t\t\tname: \"schedule that's already run gets LastBackup updated\",\n\t\t\tschedule: newScheduleBuilder(velerov1.SchedulePhaseEnabled).CronSchedule(\"@every 5m\").LastBackupTime(\"2000-01-01 00:00:00\").Result(),\n\t\t\tfakeClockTime: \"2017-01-01 12:00:00\",\n\t\t\texpectedBackupCreate: builder.ForBackup(\"ns\", \"name-20170101120000\").ObjectMeta(builder.WithLabels(velerov1.ScheduleNameLabel, \"name\")).Result(),\n\t\t\texpectedLastBackup: \"2017-01-01 12:00:00\",\n\t\t},\n\t\t{\n\t\t\tname: \"schedule that's already run but has SkippedImmediately=nil gets LastBackup updated\",\n\t\t\tschedule: newScheduleBuilder(velerov1.SchedulePhaseEnabled).CronSchedule(\"@every 5m\").LastBackupTime(\"2000-01-01 00:00:00\").SkipImmediately(nil).Result(),\n\t\t\tfakeClockTime: \"2017-01-01 12:00:00\",\n\t\t\texpectedBackupCreate: builder.ForBackup(\"ns\", \"name-20170101120000\").ObjectMeta(builder.WithLabels(velerov1.ScheduleNameLabel, \"name\")).Result(),\n\t\t\texpectedLastBackup: \"2017-01-01 12:00:00\",\n\t\t},\n\t\t{\n\t\t\tname: \"schedule that's already run but has SkippedImmediately=false gets LastBackup updated\",\n\t\t\tschedule: newScheduleBuilder(velerov1.SchedulePhaseEnabled).CronSchedule(\"@every 5m\").LastBackupTime(\"2000-01-01 00:00:00\").SkipImmediately(pointer.Bool(false)).Result(),\n\t\t\tfakeClockTime: \"2017-01-01 12:00:00\",\n\t\t\texpectedBackupCreate: builder.ForBackup(\"ns\", \"name-20170101120000\").ObjectMeta(builder.WithLabels(velerov1.ScheduleNameLabel, \"name\")).Result(),\n\t\t\texpectedLastBackup: \"2017-01-01 12:00:00\",\n\t\t},\n\t\t{\n\t\t\tname: \"schedule that's already run, server has skipImmediately set to true, and Schedule has SkippedImmediately=nil do not get LastBackup updated\",\n\t\t\tschedule: newScheduleBuilder(velerov1.SchedulePhaseEnabled).CronSchedule(\"@every 5m\").LastBackupTime(\"2000-01-01 00:00:00\").SkipImmediately(nil).Result(),\n\t\t\tfakeClockTime: \"2017-01-01 12:00:00\",\n\t\t\texpectedLastBackup: \"2000-01-01 00:00:00\",\n\t\t\texpectedLastSkipped: \"2017-01-01 12:00:00\",\n\t\t\treconcilerSkipImmediately: true,\n\t\t},\n\t\t{\n\t\t\tname: \"schedule that's already run but has SkippedImmediately=true do not get LastBackup updated\",\n\t\t\tschedule: newScheduleBuilder(velerov1.SchedulePhaseEnabled).CronSchedule(\"@every 5m\").LastBackupTime(\"2000-01-01 00:00:00\").SkipImmediately(pointer.Bool(true)).Result(),\n\t\t\tfakeClockTime: \"2017-01-01 12:00:00\",\n\t\t\texpectedLastBackup: \"2000-01-01 00:00:00\",\n\t\t\texpectedLastSkipped: \"2017-01-01 12:00:00\",\n\t\t},\n\t\t{\n\t\t\tname: \"schedule already has backup in New state.\",\n\t\t\tschedule: newScheduleBuilder(velerov1.SchedulePhaseEnabled).CronSchedule(\"@every 5m\").LastBackupTime(\"2000-01-01 00:00:00\").Result(),\n\t\t\texpectedPhase: string(velerov1.SchedulePhaseEnabled),\n\t\t\tbackup: builder.ForBackup(\"ns\", \"name-20220905120000\").ObjectMeta(builder.WithLabels(velerov1.ScheduleNameLabel, \"name\")).Phase(velerov1.BackupPhaseNew).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"schedule already has backup with empty phase (not yet reconciled).\",\n\t\t\tschedule: newScheduleBuilder(velerov1.SchedulePhaseEnabled).CronSchedule(\"@every 5m\").LastBackupTime(\"2000-01-01 00:00:00\").Result(),\n\t\t\tfakeClockTime: \"2017-01-01 12:00:00\",\n\t\t\texpectedPhase: string(velerov1.SchedulePhaseEnabled),\n\t\t\tbackup: builder.ForBackup(\"ns\", \"name-20220905120000\").ObjectMeta(builder.WithLabels(velerov1.ScheduleNameLabel, \"name\")).Phase(\"\").Result(),\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\tclient = (&fake.ClientBuilder{}).Build()\n\t\t\t\tlogger = velerotest.NewLogger()\n\t\t\t\ttestTime time.Time\n\t\t\t\terr error\n\t\t\t)\n\n\t\t\treconciler := NewScheduleReconciler(\"namespace\", logger, client, metrics.NewServerMetrics(), test.reconcilerSkipImmediately)\n\n\t\t\tif test.fakeClockTime != \"\" {\n\t\t\t\ttestTime, err = time.Parse(\"2006-01-02 15:04:05\", test.fakeClockTime)\n\t\t\t\trequire.NoError(t, err, \"unable to parse test.fakeClockTime: %v\", err)\n\t\t\t}\n\t\t\treconciler.clock = testclocks.NewFakeClock(testTime)\n\n\t\t\tif test.schedule != nil {\n\t\t\t\trequire.NoError(t, client.Create(ctx, test.schedule))\n\t\t\t}\n\n\t\t\tif test.backup != nil {\n\t\t\t\trequire.NoError(t, client.Create(ctx, test.backup))\n\t\t\t}\n\n\t\t\tscheduleb4reconcile := &velerov1.Schedule{}\n\t\t\terr = client.Get(ctx, types.NamespacedName{Namespace: \"ns\", Name: \"name\"}, scheduleb4reconcile)\n\t\t\tif test.schedule != nil {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\t_, err = reconciler.Reconcile(ctx, ctrl.Request{NamespacedName: types.NamespacedName{Namespace: \"ns\", Name: \"name\"}})\n\t\t\trequire.NoError(t, err)\n\n\t\t\tschedule := &velerov1.Schedule{}\n\t\t\terr = client.Get(ctx, types.NamespacedName{Namespace: \"ns\", Name: \"name\"}, schedule)\n\t\t\tif len(test.expectedPhase) > 0 {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, test.expectedPhase, string(schedule.Status.Phase))\n\t\t\t}\n\t\t\tif len(test.expectedValidationErrors) > 0 {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, test.expectedValidationErrors, schedule.Status.ValidationErrors)\n\t\t\t}\n\t\t\tif len(test.expectedLastBackup) > 0 {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\trequire.NotNil(t, schedule.Status.LastBackup)\n\t\t\t\tassert.Equal(t, parseTime(test.expectedLastBackup).Unix(), schedule.Status.LastBackup.Unix())\n\t\t\t}\n\t\t\tif len(test.expectedLastSkipped) > 0 {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\trequire.NotNil(t, schedule.Status.LastSkipped)\n\t\t\t\tassert.Equal(t, parseTime(test.expectedLastSkipped).Unix(), schedule.Status.LastSkipped.Unix())\n\t\t\t}\n\n\t\t\t// we expect reconcile to flip SkipImmediately to false if it's true or the server is configured to skip immediately and the schedule doesn't have it set\n\t\t\tif scheduleb4reconcile.Spec.SkipImmediately != nil && *scheduleb4reconcile.Spec.SkipImmediately ||\n\t\t\t\ttest.reconcilerSkipImmediately && scheduleb4reconcile.Spec.SkipImmediately == nil {\n\t\t\t\tassert.Equal(t, schedule.Spec.SkipImmediately, pointer.Bool(false))\n\t\t\t}\n\n\t\t\tbackups := &velerov1.BackupList{}\n\t\t\trequire.NoError(t, client.List(ctx, backups))\n\n\t\t\t// If backup associated with schedule's status is in New or InProgress or empty phase,\n\t\t\t// new backup shouldn't be submitted.\n\t\t\tif test.backup != nil &&\n\t\t\t\t(test.backup.Status.Phase == \"\" || test.backup.Status.Phase == velerov1.BackupPhaseNew || test.backup.Status.Phase == velerov1.BackupPhaseInProgress) {\n\t\t\t\tassert.Len(t, backups.Items, 1)\n\t\t\t\trequire.NoError(t, client.Delete(ctx, test.backup))\n\t\t\t}\n\n\t\t\trequire.NoError(t, client.List(ctx, backups))\n\n\t\t\tif test.expectedBackupCreate == nil {\n\t\t\t\tassert.Empty(t, backups.Items)\n\t\t\t} else {\n\t\t\t\tassert.Len(t, backups.Items, 1)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc parseTime(timeString string) time.Time {\n\tres, _ := time.Parse(\"2006-01-02 15:04:05\", timeString)\n\treturn res\n}\n\nfunc TestGetNextRunTime(t *testing.T) {\n\tdefaultSchedule := func() *velerov1.Schedule {\n\t\treturn builder.ForSchedule(\"velero\", \"schedule-1\").CronSchedule(\"@every 5m\").Result()\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tschedule *velerov1.Schedule\n\t\tlastRanOffset string\n\t\texpectedDue bool\n\t\texpectedNextRunTimeOffset string\n\t}{\n\t\t{\n\t\t\tname: \"first run\",\n\t\t\tschedule: defaultSchedule(),\n\t\t\texpectedDue: false,\n\t\t\texpectedNextRunTimeOffset: \"5m\",\n\t\t},\n\t\t{\n\t\t\tname: \"just ran\",\n\t\t\tschedule: defaultSchedule(),\n\t\t\tlastRanOffset: \"0s\",\n\t\t\texpectedDue: false,\n\t\t\texpectedNextRunTimeOffset: \"5m\",\n\t\t},\n\t\t{\n\t\t\tname: \"almost but not quite time to run\",\n\t\t\tschedule: defaultSchedule(),\n\t\t\tlastRanOffset: \"4m59s\",\n\t\t\texpectedDue: false,\n\t\t\texpectedNextRunTimeOffset: \"5m\",\n\t\t},\n\t\t{\n\t\t\tname: \"time to run again\",\n\t\t\tschedule: defaultSchedule(),\n\t\t\tlastRanOffset: \"5m\",\n\t\t\texpectedDue: true,\n\t\t\texpectedNextRunTimeOffset: \"5m\",\n\t\t},\n\t\t{\n\t\t\tname: \"several runs missed\",\n\t\t\tschedule: defaultSchedule(),\n\t\t\tlastRanOffset: \"5h\",\n\t\t\texpectedDue: true,\n\t\t\texpectedNextRunTimeOffset: \"5m\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tcronSchedule, err := cron.ParseStandard(test.schedule.Spec.Schedule)\n\t\t\trequire.NoError(t, err, \"unable to parse test.schedule.Spec.Schedule: %v\", err)\n\n\t\t\ttestClock := testclocks.NewFakeClock(time.Now())\n\n\t\t\tif test.lastRanOffset != \"\" {\n\t\t\t\toffsetDuration, err := time.ParseDuration(test.lastRanOffset)\n\t\t\t\trequire.NoError(t, err, \"unable to parse test.lastRanOffset: %v\", err)\n\n\t\t\t\ttest.schedule.Status.LastBackup = &metav1.Time{Time: testClock.Now().Add(-offsetDuration)}\n\t\t\t\ttest.schedule.CreationTimestamp = *test.schedule.Status.LastBackup\n\t\t\t} else {\n\t\t\t\ttest.schedule.CreationTimestamp = metav1.Time{Time: testClock.Now()}\n\t\t\t}\n\n\t\t\tnextRunTimeOffset, err := time.ParseDuration(test.expectedNextRunTimeOffset)\n\t\t\tif err != nil {\n\t\t\t\tpanic(err)\n\t\t\t}\n\n\t\t\tvar baseTime time.Time\n\t\t\tif test.lastRanOffset != \"\" {\n\t\t\t\tbaseTime = test.schedule.Status.LastBackup.Time\n\t\t\t} else {\n\t\t\t\tbaseTime = test.schedule.CreationTimestamp.Time\n\t\t\t}\n\t\t\texpectedNextRunTime := baseTime.Add(nextRunTimeOffset)\n\n\t\t\tdue, nextRunTime := getNextRunTime(test.schedule, cronSchedule, testClock.Now())\n\n\t\t\tassert.Equal(t, test.expectedDue, due)\n\t\t\t// ignore diffs of under a second. the cron library does some rounding.\n\t\t\tassert.WithinDuration(t, expectedNextRunTime, nextRunTime, time.Second)\n\t\t})\n\t}\n}\n\nfunc TestParseCronSchedule(t *testing.T) {\n\t// From https://github.com/vmware-tanzu/velero/issues/30, where we originally were using cron.Parse(),\n\t// which treats the first field as seconds, and not minutes. We want to use cron.ParseStandard()\n\t// instead, which has the first field as minutes.\n\n\tnow := time.Date(2017, 8, 10, 12, 27, 0, 0, time.UTC)\n\n\t// Start with a Schedule with:\n\t// - schedule: once a day at 9am\n\t// - last backup: 2017-08-10 12:27:00 (just happened)\n\ts := builder.ForSchedule(\"velero\", \"schedule-1\").CronSchedule(\"0 9 * * *\").LastBackupTime(now.Format(\"2006-01-02 15:04:05\")).Result()\n\n\tlogger := velerotest.NewLogger()\n\n\tc, errs := parseCronSchedule(s, logger)\n\trequire.Empty(t, errs)\n\n\t// make sure we're not due and next backup is tomorrow at 9am\n\tdue, next := getNextRunTime(s, c, now)\n\tassert.False(t, due)\n\tassert.Equal(t, time.Date(2017, 8, 11, 9, 0, 0, 0, time.UTC), next)\n\n\t// advance the clock a couple of hours and make sure nothing has changed\n\tnow = now.Add(2 * time.Hour)\n\tdue, next = getNextRunTime(s, c, now)\n\tassert.False(t, due)\n\tassert.Equal(t, time.Date(2017, 8, 11, 9, 0, 0, 0, time.UTC), next)\n\n\t// advance clock to 1 minute after due time, make sure due=true\n\tnow = time.Date(2017, 8, 11, 9, 1, 0, 0, time.UTC)\n\tdue, next = getNextRunTime(s, c, now)\n\tassert.True(t, due)\n\tassert.Equal(t, time.Date(2017, 8, 11, 9, 0, 0, 0, time.UTC), next)\n\n\t// record backup time\n\ts.Status.LastBackup = &metav1.Time{Time: now}\n\n\t// advance clock 1 minute, make sure we're not due and next backup is tomorrow at 9am\n\tnow = time.Date(2017, 8, 11, 9, 2, 0, 0, time.UTC)\n\tdue, next = getNextRunTime(s, c, now)\n\tassert.False(t, due)\n\tassert.Equal(t, time.Date(2017, 8, 12, 9, 0, 0, 0, time.UTC), next)\n}\n\nfunc TestGetBackup(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tschedule *velerov1.Schedule\n\t\ttestClockTime string\n\t\texpectedBackup *velerov1.Backup\n\t}{\n\t\t{\n\t\t\tname: \"ensure name is formatted correctly (AM time)\",\n\t\t\tschedule: builder.ForSchedule(\"foo\", \"bar\").Result(),\n\t\t\ttestClockTime: \"2017-07-25 09:15:00\",\n\t\t\texpectedBackup: builder.ForBackup(\"foo\", \"bar-20170725091500\").ObjectMeta(builder.WithLabels(velerov1.ScheduleNameLabel, \"bar\")).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"ensure name is formatted correctly (PM time)\",\n\t\t\tschedule: builder.ForSchedule(\"foo\", \"bar\").Result(),\n\t\t\ttestClockTime: \"2017-07-25 14:15:00\",\n\t\t\texpectedBackup: builder.ForBackup(\"foo\", \"bar-20170725141500\").ObjectMeta(builder.WithLabels(velerov1.ScheduleNameLabel, \"bar\")).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"ensure schedule backup template is copied\",\n\t\t\tschedule: builder.ForSchedule(\"foo\", \"bar\").\n\t\t\t\tTemplate(builder.ForBackup(\"\", \"\").\n\t\t\t\t\tIncludedNamespaces(\"ns-1\", \"ns-2\").\n\t\t\t\t\tExcludedNamespaces(\"ns-3\").\n\t\t\t\t\tIncludedResources(\"foo\", \"bar\").\n\t\t\t\t\tExcludedResources(\"baz\").\n\t\t\t\t\tLabelSelector(&metav1.LabelSelector{MatchLabels: map[string]string{\"label\": \"value\"}}).\n\t\t\t\t\tTTL(time.Duration(300)).\n\t\t\t\t\tResult().\n\t\t\t\t\tSpec).\n\t\t\t\tResult(),\n\t\t\ttestClockTime: \"2017-07-25 09:15:00\",\n\t\t\texpectedBackup: builder.ForBackup(\"foo\", \"bar-20170725091500\").\n\t\t\t\tObjectMeta(builder.WithLabels(velerov1.ScheduleNameLabel, \"bar\")).\n\t\t\t\tIncludedNamespaces(\"ns-1\", \"ns-2\").\n\t\t\t\tExcludedNamespaces(\"ns-3\").\n\t\t\t\tIncludedResources(\"foo\", \"bar\").\n\t\t\t\tExcludedResources(\"baz\").\n\t\t\t\tLabelSelector(&metav1.LabelSelector{MatchLabels: map[string]string{\"label\": \"value\"}}).\n\t\t\t\tTTL(time.Duration(300)).\n\t\t\t\tResult(),\n\t\t},\n\t\t{\n\t\t\tname: \"ensure schedule labels are copied\",\n\t\t\tschedule: builder.ForSchedule(\"foo\", \"bar\").ObjectMeta(builder.WithLabels(\"foo\", \"bar\", \"bar\", \"baz\")).Result(),\n\t\t\ttestClockTime: \"2017-07-25 14:15:00\",\n\t\t\texpectedBackup: builder.ForBackup(\"foo\", \"bar-20170725141500\").ObjectMeta(builder.WithLabels(velerov1.ScheduleNameLabel, \"bar\", \"bar\", \"baz\", \"foo\", \"bar\")).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"ensure schedule annotations are copied\",\n\t\t\tschedule: builder.ForSchedule(\"foo\", \"bar\").ObjectMeta(builder.WithAnnotations(\"foo\", \"bar\", \"bar\", \"baz\")).Result(),\n\t\t\ttestClockTime: \"2017-07-25 14:15:00\",\n\t\t\texpectedBackup: builder.ForBackup(\"foo\", \"bar-20170725141500\").ObjectMeta(builder.WithLabels(velerov1.ScheduleNameLabel, \"bar\"), builder.WithAnnotations(\"bar\", \"baz\", \"foo\", \"bar\")).Result(),\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\ttestTime, err := time.Parse(\"2006-01-02 15:04:05\", test.testClockTime)\n\t\t\trequire.NoError(t, err, \"unable to parse test.testClockTime: %v\", err)\n\n\t\t\tbackup := getBackup(test.schedule, testclocks.NewFakeClock(testTime).Now())\n\n\t\t\tassert.Equal(t, test.expectedBackup.Namespace, backup.Namespace)\n\t\t\tassert.Equal(t, test.expectedBackup.Name, backup.Name)\n\t\t\tassert.Equal(t, test.expectedBackup.Labels, backup.Labels)\n\t\t\tassert.Equal(t, test.expectedBackup.Annotations, backup.Annotations)\n\t\t\tassert.Equal(t, test.expectedBackup.Spec, backup.Spec)\n\t\t})\n\t}\n}\n\nfunc TestCheckIfBackupInNewOrProgress(t *testing.T) {\n\trequire.NoError(t, velerov1.AddToScheme(scheme.Scheme))\n\n\tclient := fake.NewClientBuilder().WithScheme(scheme.Scheme).Build()\n\tlogger := velerotest.NewLogger()\n\n\t// Create testing schedule\n\ttestSchedule := builder.ForSchedule(\"ns\", \"name\").Phase(velerov1.SchedulePhaseEnabled).Result()\n\terr := client.Create(ctx, testSchedule)\n\trequire.NoError(t, err, \"fail to create schedule in TestCheckIfBackupInNewOrProgress: %v\", err)\n\n\t// Create backup in New phase.\n\tnewBackup := builder.ForBackup(\"ns\", \"backup-1\").\n\t\tObjectMeta(builder.WithLabels(velerov1.ScheduleNameLabel, \"name\")).\n\t\tPhase(velerov1.BackupPhaseNew).Result()\n\terr = client.Create(ctx, newBackup)\n\trequire.NoError(t, err, \"fail to create backup in New phase in TestCheckIfBackupInNewOrProgress: %v\", err)\n\n\treconciler := NewScheduleReconciler(\"ns\", logger, client, metrics.NewServerMetrics(), false)\n\tresult := reconciler.checkIfBackupInNewOrProgress(testSchedule)\n\tassert.True(t, result)\n\n\t// Clean backup in New phase.\n\terr = client.Delete(ctx, newBackup)\n\trequire.NoError(t, err, \"fail to delete backup in New phase in TestCheckIfBackupInNewOrProgress: %v\", err)\n\n\t// Create backup in InProgress phase.\n\tinProgressBackup := builder.ForBackup(\"ns\", \"backup-2\").\n\t\tObjectMeta(builder.WithLabels(velerov1.ScheduleNameLabel, \"name\")).\n\t\tPhase(velerov1.BackupPhaseInProgress).Result()\n\terr = client.Create(ctx, inProgressBackup)\n\trequire.NoError(t, err, \"fail to create backup in InProgress phase in TestCheckIfBackupInNewOrProgress: %v\", err)\n\n\treconciler = NewScheduleReconciler(\"namespace\", logger, client, metrics.NewServerMetrics(), false)\n\tresult = reconciler.checkIfBackupInNewOrProgress(testSchedule)\n\tassert.True(t, result)\n\n\t// Clean backup in InProgress phase.\n\terr = client.Delete(ctx, inProgressBackup)\n\trequire.NoError(t, err, \"fail to delete backup in InProgress phase in TestCheckIfBackupInNewOrProgress: %v\", err)\n\n\t// Create backup with empty phase (not yet reconciled).\n\temptyPhaseBackup := builder.ForBackup(\"ns\", \"backup-3\").\n\t\tObjectMeta(builder.WithLabels(velerov1.ScheduleNameLabel, \"name\")).\n\t\tPhase(\"\").Result()\n\terr = client.Create(ctx, emptyPhaseBackup)\n\trequire.NoError(t, err, \"fail to create backup with empty phase in TestCheckIfBackupInNewOrProgress: %v\", err)\n\n\treconciler = NewScheduleReconciler(\"namespace\", logger, client, metrics.NewServerMetrics(), false)\n\tresult = reconciler.checkIfBackupInNewOrProgress(testSchedule)\n\tassert.True(t, result)\n}\n" }, { "path": "pkg/controller/server_status_request_controller.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\tclocks \"k8s.io/utils/clock\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/controller\"\n\n\t\"github.com/vmware-tanzu/velero/internal/velero\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/buildinfo\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n)\n\nconst (\n\tttl = time.Minute\n\tstatusRequestResyncPeriod = 5 * time.Minute\n)\n\ntype PluginLister interface {\n\t// List returns all PluginIdentifiers for kind.\n\tList(kind common.PluginKind) []framework.PluginIdentifier\n}\n\n// serverStatusRequestReconciler reconciles a ServerStatusRequest object\ntype serverStatusRequestReconciler struct {\n\tclient client.Client\n\tctx context.Context\n\tpluginRegistry PluginLister\n\tclock clocks.WithTickerAndDelayedExecution\n\n\tlog logrus.FieldLogger\n}\n\n// NewServerStatusRequestReconciler initializes and returns serverStatusRequestReconciler struct.\nfunc NewServerStatusRequestReconciler(\n\tctx context.Context,\n\tclient client.Client,\n\tpluginRegistry PluginLister,\n\tclock clocks.WithTickerAndDelayedExecution,\n\tlog logrus.FieldLogger) *serverStatusRequestReconciler {\n\treturn &serverStatusRequestReconciler{\n\t\tclient: client,\n\t\tctx: ctx,\n\t\tpluginRegistry: pluginRegistry,\n\t\tclock: clock,\n\t\tlog: log,\n\t}\n}\n\n// +kubebuilder:rbac:groups=velero.io,resources=serverstatusrequests,verbs=get;list;watch;create;update;patch;delete\n// +kubebuilder:rbac:groups=velero.io,resources=serverstatusrequests/status,verbs=get;update;patch\n\nfunc (r *serverStatusRequestReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {\n\tlog := r.log.WithFields(logrus.Fields{\n\t\t\"controller\": constant.ControllerServerStatusRequest,\n\t\t\"serverStatusRequest\": req.NamespacedName,\n\t})\n\n\t// Fetch the ServerStatusRequest instance.\n\tlog.Debug(\"Getting ServerStatusRequest\")\n\tstatusRequest := &velerov1api.ServerStatusRequest{}\n\tif err := r.client.Get(r.ctx, req.NamespacedName, statusRequest); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.Debug(\"Unable to find ServerStatusRequest\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\n\t\tlog.WithError(err).Error(\"Error getting ServerStatusRequest\")\n\t\treturn ctrl.Result{}, err\n\t}\n\n\tlog = r.log.WithFields(logrus.Fields{\n\t\t\"controller\": constant.ControllerServerStatusRequest,\n\t\t\"serverStatusRequest\": req.NamespacedName,\n\t\t\"phase\": statusRequest.Status.Phase,\n\t})\n\n\tswitch statusRequest.Status.Phase {\n\tcase \"\", velerov1api.ServerStatusRequestPhaseNew:\n\t\tlog.Info(\"Processing new ServerStatusRequest\")\n\t\toriginal := statusRequest.DeepCopy()\n\t\tstatusRequest.Status.ServerVersion = buildinfo.Version\n\t\tstatusRequest.Status.Phase = velerov1api.ServerStatusRequestPhaseProcessed\n\t\tstatusRequest.Status.ProcessedTimestamp = &metav1.Time{Time: r.clock.Now()}\n\t\tstatusRequest.Status.Plugins = velero.GetInstalledPluginInfo(r.pluginRegistry)\n\n\t\tif err := r.client.Patch(r.ctx, statusRequest, client.MergeFrom(original)); err != nil {\n\t\t\tlog.WithError(err).Error(\"Error updating ServerStatusRequest status\")\n\t\t\treturn ctrl.Result{RequeueAfter: statusRequestResyncPeriod}, err\n\t\t}\n\tcase velerov1api.ServerStatusRequestPhaseProcessed:\n\t\tlog.Debug(\"Checking whether ServerStatusRequest has expired\")\n\t\texpiration := statusRequest.Status.ProcessedTimestamp.Add(ttl)\n\t\tif expiration.After(r.clock.Now()) {\n\t\t\tlog.Debug(\"ServerStatusRequest has not expired\")\n\t\t\treturn ctrl.Result{RequeueAfter: statusRequestResyncPeriod}, nil\n\t\t}\n\n\t\tlog.Debug(\"ServerStatusRequest has expired, deleting it\")\n\t\tif err := r.client.Delete(r.ctx, statusRequest); err != nil {\n\t\t\tlog.WithError(err).Error(\"Unable to delete the request\")\n\t\t\treturn ctrl.Result{}, nil\n\t\t}\n\tdefault:\n\t\treturn ctrl.Result{}, errors.New(\"unexpected ServerStatusRequest phase\")\n\t}\n\n\t// Requeue is mostly to handle deleting any expired status requests that were not\n\t// deleted as part of the normal client flow for whatever reason.\n\treturn ctrl.Result{RequeueAfter: statusRequestResyncPeriod}, nil\n}\n\nfunc (r *serverStatusRequestReconciler) SetupWithManager(mgr ctrl.Manager) error {\n\treturn ctrl.NewControllerManagedBy(mgr).\n\t\tFor(&velerov1api.ServerStatusRequest{}).\n\t\tWithOptions(controller.Options{\n\t\t\tMaxConcurrentReconciles: 10,\n\t\t}).\n\t\tComplete(r)\n}\n" }, { "path": "pkg/controller/server_status_request_controller_test.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\t. \"github.com/onsi/ginkgo/v2\"\n\t. \"github.com/onsi/gomega\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/client-go/kubernetes/scheme\"\n\ttestclocks \"k8s.io/utils/clock/testing\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/buildinfo\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc statusRequestBuilder(resourceVersion string) *builder.ServerStatusRequestBuilder {\n\treturn builder.ForServerStatusRequest(velerov1api.DefaultNamespace, \"sr-1\", resourceVersion)\n}\n\nvar _ = Describe(\"Server Status Request Reconciler\", func() {\n\ttype request struct {\n\t\treq *velerov1api.ServerStatusRequest\n\t\treqPluginLister *fakePluginLister\n\t\texpected *velerov1api.ServerStatusRequest\n\t\texpectedRequeue ctrl.Result\n\t\texpectedErrMsg string\n\t}\n\n\t// `now` will be used to set the fake clock's time; capture\n\t// it here so it can be referenced in the test case defs.\n\tnow, err := time.Parse(time.RFC1123, time.RFC1123)\n\tExpect(err).ToNot(HaveOccurred())\n\tnow = now.Local()\n\n\tDescribeTable(\"a Server Status request\",\n\t\tfunc(test request) {\n\t\t\t// Setup reconciler\n\t\t\tExpect(velerov1api.AddToScheme(scheme.Scheme)).To(Succeed())\n\t\t\tr := NewServerStatusRequestReconciler(\n\t\t\t\tcontext.Background(),\n\t\t\t\tfake.NewClientBuilder().WithScheme(scheme.Scheme).WithObjects(test.req).Build(),\n\t\t\t\ttest.reqPluginLister,\n\t\t\t\ttestclocks.NewFakeClock(now),\n\t\t\t\tvelerotest.NewLogger(),\n\t\t\t)\n\n\t\t\tactualResult, err := r.Reconcile(r.ctx, ctrl.Request{\n\t\t\t\tNamespacedName: types.NamespacedName{\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\tName: test.req.Name,\n\t\t\t\t},\n\t\t\t})\n\n\t\t\tExpect(actualResult).To(BeEquivalentTo(test.expectedRequeue))\n\t\t\tif test.expectedErrMsg == \"\" {\n\t\t\t\tExpect(err).ToNot(HaveOccurred())\n\t\t\t} else {\n\t\t\t\tExpect(err.Error()).To(BeEquivalentTo(test.expectedErrMsg))\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tinstance := &velerov1api.ServerStatusRequest{}\n\t\t\terr = r.client.Get(ctx, kbclient.ObjectKey{Name: test.req.Name, Namespace: test.req.Namespace}, instance)\n\n\t\t\t// Assertions\n\t\t\tif test.expected == nil {\n\t\t\t\tExpect(apierrors.IsNotFound(err)).To(BeTrue())\n\t\t\t} else {\n\t\t\t\tExpect(err).ToNot(HaveOccurred())\n\t\t\t\tEventually(instance.Status.Phase == test.expected.Status.Phase, timeout).Should(BeTrue())\n\t\t\t}\n\t\t},\n\t\tEntry(\"with phase=empty will be processed and phased successfully patched\", request{\n\t\t\treq: statusRequestBuilder(\"1\").\n\t\t\t\tServerVersion(buildinfo.Version).\n\t\t\t\tProcessedTimestamp(now).\n\t\t\t\tPlugins([]velerov1api.PluginInfo{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"custom.io/myown\",\n\t\t\t\t\t\tKind: \"VolumeSnapshotter\",\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\treqPluginLister: &fakePluginLister{\n\t\t\t\tplugins: []framework.PluginIdentifier{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"custom.io/myown\",\n\t\t\t\t\t\tKind: \"VolumeSnapshotter\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: statusRequestBuilder(\"1\").\n\t\t\t\tServerVersion(buildinfo.Version).\n\t\t\t\tPhase(velerov1api.ServerStatusRequestPhaseProcessed).\n\t\t\t\tProcessedTimestamp(now).\n\t\t\t\tPlugins([]velerov1api.PluginInfo{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"custom.io/myown\",\n\t\t\t\t\t\tKind: \"VolumeSnapshotter\",\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\texpectedRequeue: ctrl.Result{Requeue: false, RequeueAfter: statusRequestResyncPeriod},\n\t\t}),\n\t\tEntry(\"with phase=new will be processed and phased successfully patched\", request{\n\t\t\treq: statusRequestBuilder(\"1\").\n\t\t\t\tServerVersion(buildinfo.Version).\n\t\t\t\tPhase(velerov1api.ServerStatusRequestPhaseNew).\n\t\t\t\tProcessedTimestamp(now).\n\t\t\t\tPlugins([]velerov1api.PluginInfo{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"custom.io/myown\",\n\t\t\t\t\t\tKind: \"VolumeSnapshotter\",\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\treqPluginLister: &fakePluginLister{\n\t\t\t\tplugins: []framework.PluginIdentifier{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"custom.io/myown\",\n\t\t\t\t\t\tKind: \"VolumeSnapshotter\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: statusRequestBuilder(\"1\").\n\t\t\t\tServerVersion(buildinfo.Version).\n\t\t\t\tPhase(velerov1api.ServerStatusRequestPhaseProcessed).\n\t\t\t\tProcessedTimestamp(now).\n\t\t\t\tPlugins([]velerov1api.PluginInfo{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"custom.io/myown\",\n\t\t\t\t\t\tKind: \"VolumeSnapshotter\",\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\texpectedRequeue: ctrl.Result{Requeue: false, RequeueAfter: statusRequestResyncPeriod},\n\t\t}),\n\t\tEntry(\"with phase=Processed does not get deleted if not expired\", request{\n\t\t\treq: statusRequestBuilder(\"1\").\n\t\t\t\tServerVersion(buildinfo.Version).\n\t\t\t\tPhase(velerov1api.ServerStatusRequestPhaseProcessed).\n\t\t\t\tProcessedTimestamp(now). // not yet expired\n\t\t\t\tPlugins([]velerov1api.PluginInfo{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"custom.io/myotherown\",\n\t\t\t\t\t\tKind: \"VolumeSnapshotter\",\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\treqPluginLister: &fakePluginLister{\n\t\t\t\tplugins: []framework.PluginIdentifier{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"custom.io/myotherown\",\n\t\t\t\t\t\tKind: \"VolumeSnapshotter\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: statusRequestBuilder(\"1\").\n\t\t\t\tServerVersion(buildinfo.Version).\n\t\t\t\tPhase(velerov1api.ServerStatusRequestPhaseProcessed).\n\t\t\t\tProcessedTimestamp(now).\n\t\t\t\tPlugins([]velerov1api.PluginInfo{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"custom.io/myown\",\n\t\t\t\t\t\tKind: \"VolumeSnapshotter\",\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\texpectedRequeue: ctrl.Result{Requeue: false, RequeueAfter: statusRequestResyncPeriod},\n\t\t}),\n\t\tEntry(\"with phase=Processed gets deleted if expired\", request{\n\t\t\treq: statusRequestBuilder(\"1\").\n\t\t\t\tServerVersion(buildinfo.Version).\n\t\t\t\tPhase(velerov1api.ServerStatusRequestPhaseProcessed).\n\t\t\t\tProcessedTimestamp(now.Add(-61 * time.Second)). // expired\n\t\t\t\tPlugins([]velerov1api.PluginInfo{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"custom.io/myotherown\",\n\t\t\t\t\t\tKind: \"VolumeSnapshotter\",\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\treqPluginLister: &fakePluginLister{\n\t\t\t\tplugins: []framework.PluginIdentifier{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"custom.io/myotherown\",\n\t\t\t\t\t\tKind: \"VolumeSnapshotter\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: nil,\n\t\t\texpectedRequeue: ctrl.Result{Requeue: false, RequeueAfter: statusRequestResyncPeriod},\n\t\t}),\n\t\tEntry(\"with invalid phase returns an error and does not requeue\", request{\n\t\t\treq: statusRequestBuilder(\"1\").\n\t\t\t\tServerVersion(buildinfo.Version).\n\t\t\t\tPhase(\"an-invalid-phase\").\n\t\t\t\tProcessedTimestamp(now).\n\t\t\t\tPlugins([]velerov1api.PluginInfo{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"custom.io/myown\",\n\t\t\t\t\t\tKind: \"VolumeSnapshotter\",\n\t\t\t\t\t},\n\t\t\t\t}).\n\t\t\t\tResult(),\n\t\t\treqPluginLister: &fakePluginLister{\n\t\t\t\tplugins: []framework.PluginIdentifier{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"custom.io/myown\",\n\t\t\t\t\t\tKind: \"VolumeSnapshotter\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedErrMsg: \"unexpected ServerStatusRequest phase\",\n\t\t\texpectedRequeue: ctrl.Result{Requeue: false, RequeueAfter: 0},\n\t\t}),\n\t)\n})\n\ntype fakePluginLister struct {\n\tplugins []framework.PluginIdentifier\n}\n\nfunc (l *fakePluginLister) List(kind common.PluginKind) []framework.PluginIdentifier {\n\tvar plugins []framework.PluginIdentifier\n\tfor _, plugin := range l.plugins {\n\t\tif plugin.Kind == kind {\n\t\t\tplugins = append(plugins, plugin)\n\t\t}\n\t}\n\n\treturn plugins\n}\n" }, { "path": "pkg/controller/suite_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage controller\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"path/filepath\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/persistence\"\n\tpersistencemocks \"github.com/vmware-tanzu/velero/pkg/persistence/mocks\"\n\n\t\"k8s.io/klog/v2\"\n\t\"sigs.k8s.io/controller-runtime/pkg/envtest\"\n\t\"sigs.k8s.io/controller-runtime/pkg/manager\"\n\n\t. \"github.com/onsi/ginkgo/v2\"\n\t. \"github.com/onsi/gomega\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\n\t\"k8s.io/client-go/kubernetes/scheme\"\n\t\"k8s.io/client-go/rest\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t// +kubebuilder:scaffold:imports\n)\n\n// These tests use Ginkgo (BDD-style Go testing framework). Refer to\n// http://onsi.github.io/ginkgo/ to learn more about Ginkgo.\n\nconst (\n\ttimeout = time.Second * 30\n)\n\nvar (\n\tenv *envtest.Environment\n\ttestEnv *testEnvironment\n\tctx, cancel = context.WithCancel(context.Background())\n)\n\nfunc TestAPIs(t *testing.T) {\n\tRegisterFailHandler(Fail)\n\tRunSpecs(t, \"Controller Suite\")\n}\n\nvar _ = BeforeSuite(func(done Done) {\n\tBy(\"bootstrapping test environment\")\n\ttestEnv = newTestEnvironment()\n\n\tBy(\"starting the manager\")\n\tgo func() {\n\t\tdefer GinkgoRecover()\n\t\tExpect(testEnv.startManager()).To(Succeed())\n\t}()\n\n\tclose(done)\n})\n\nvar _ = AfterSuite(func() {\n\tBy(\"tearing down the test environment\")\n\terr := testEnv.stop()\n\tExpect(err).ToNot(HaveOccurred())\n})\n\n// testEnvironment encapsulates a Kubernetes local test environment.\ntype testEnvironment struct {\n\tmanager.Manager\n\tclient.Client\n\tConfig *rest.Config\n\n\tdoneMgr context.Context\n}\n\n// newTestEnvironment creates a new environment spinning up a local api-server.\n//\n// This function should be called only once for each package you're running tests within,\n// usually the environment is initialized in a suite_test.go file within a `BeforeSuite` ginkgo block.\nfunc newTestEnvironment() *testEnvironment {\n\t// scheme.Scheme is initialized with all native Kubernetes types\n\terr := velerov1api.AddToScheme(scheme.Scheme)\n\tExpect(err).NotTo(HaveOccurred())\n\n\terr = velerov2alpha1api.AddToScheme(scheme.Scheme)\n\tExpect(err).NotTo(HaveOccurred())\n\n\tenv = &envtest.Environment{\n\t\tCRDDirectoryPaths: []string{filepath.Join(\"..\", \"config\", \"crd\", \"bases\")},\n\t}\n\n\tif _, err := env.Start(); err != nil {\n\t\tpanic(err)\n\t}\n\n\tmgr, err := manager.New(env.Config, manager.Options{\n\t\tScheme: scheme.Scheme,\n\t})\n\tif err != nil {\n\t\tklog.Fatalf(\"Failed to start testenv manager: %v\", err)\n\t}\n\n\treturn &testEnvironment{\n\t\tManager: mgr,\n\t\tClient: mgr.GetClient(),\n\t\tConfig: mgr.GetConfig(),\n\t\tdoneMgr: ctx,\n\t}\n}\n\nfunc (t *testEnvironment) startManager() error {\n\treturn t.Manager.Start(t.doneMgr)\n}\n\nfunc (t *testEnvironment) stop() error {\n\tcancel()\n\treturn env.Stop()\n}\n\ntype fakeErrorBackupStoreGetter struct {\n}\n\nfunc (f *fakeErrorBackupStoreGetter) Get(*velerov1api.BackupStorageLocation, persistence.ObjectStoreGetter, logrus.FieldLogger) (persistence.BackupStore, error) {\n\treturn nil, fmt.Errorf(\"some error\")\n}\n\ntype fakeSingleObjectBackupStoreGetter struct {\n\tstore persistence.BackupStore\n}\n\nfunc (f *fakeSingleObjectBackupStoreGetter) Get(*velerov1api.BackupStorageLocation, persistence.ObjectStoreGetter, logrus.FieldLogger) (persistence.BackupStore, error) {\n\treturn f.store, nil\n}\n\n// NewFakeSingleObjectBackupStoreGetter returns an ObjectBackupStoreGetter\n// that will return only the given BackupStore.\nfunc NewFakeSingleObjectBackupStoreGetter(store persistence.BackupStore) persistence.ObjectBackupStoreGetter {\n\treturn &fakeSingleObjectBackupStoreGetter{store: store}\n}\n\ntype fakeObjectBackupStoreGetter struct {\n\tstores map[string]*persistencemocks.BackupStore\n}\n\nfunc (f *fakeObjectBackupStoreGetter) Get(loc *velerov1api.BackupStorageLocation, _ persistence.ObjectStoreGetter, _ logrus.FieldLogger) (persistence.BackupStore, error) {\n\treturn f.stores[loc.Name], nil\n}\n\n// NewFakeObjectBackupStoreGetter returns an ObjectBackupStoreGetter that will\n// return the BackupStore for a given BackupStorageLocation name.\nfunc NewFakeObjectBackupStoreGetter(stores map[string]*persistencemocks.BackupStore) persistence.ObjectBackupStoreGetter {\n\treturn &fakeObjectBackupStoreGetter{stores: stores}\n}\n" }, { "path": "pkg/datamover/backup_micro_service.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage datamover\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tcachetool \"k8s.io/client-go/tools/cache\"\n\t\"sigs.k8s.io/controller-runtime/pkg/cache\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n)\n\nconst (\n\tdataUploadDownloadRequestor = \"snapshot-data-upload-download\"\n)\n\n// BackupMicroService process data mover backups inside the backup pod\ntype BackupMicroService struct {\n\tctx context.Context\n\tclient client.Client\n\tkubeClient kubernetes.Interface\n\trepoEnsurer *repository.Ensurer\n\tcredentialGetter *credentials.CredentialGetter\n\tlogger logrus.FieldLogger\n\tdataPathMgr *datapath.Manager\n\teventRecorder kube.EventRecorder\n\n\tnamespace string\n\tdataUploadName string\n\tdataUpload *velerov2alpha1api.DataUpload\n\tsourceTargetPath datapath.AccessPoint\n\n\tresultSignal chan dataPathResult\n\n\tduInformer cache.Informer\n\tduHandler cachetool.ResourceEventHandlerRegistration\n\tnodeName string\n}\n\ntype dataPathResult struct {\n\terr error\n\tresult string\n}\n\nfunc NewBackupMicroService(ctx context.Context, client client.Client, kubeClient kubernetes.Interface, dataUploadName string, namespace string, nodeName string,\n\tsourceTargetPath datapath.AccessPoint, dataPathMgr *datapath.Manager, repoEnsurer *repository.Ensurer, cred *credentials.CredentialGetter,\n\tduInformer cache.Informer, log logrus.FieldLogger) *BackupMicroService {\n\treturn &BackupMicroService{\n\t\tctx: ctx,\n\t\tclient: client,\n\t\tkubeClient: kubeClient,\n\t\tcredentialGetter: cred,\n\t\tlogger: log,\n\t\trepoEnsurer: repoEnsurer,\n\t\tdataPathMgr: dataPathMgr,\n\t\tnamespace: namespace,\n\t\tdataUploadName: dataUploadName,\n\t\tsourceTargetPath: sourceTargetPath,\n\t\tnodeName: nodeName,\n\t\tresultSignal: make(chan dataPathResult),\n\t\tduInformer: duInformer,\n\t}\n}\n\nfunc (r *BackupMicroService) Init() error {\n\tr.eventRecorder = kube.NewEventRecorder(r.kubeClient, r.client.Scheme(), r.dataUploadName, r.nodeName, r.logger)\n\n\thandler, err := r.duInformer.AddEventHandler(\n\t\tcachetool.ResourceEventHandlerFuncs{\n\t\t\tUpdateFunc: func(oldObj any, newObj any) {\n\t\t\t\toldDu := oldObj.(*velerov2alpha1api.DataUpload)\n\t\t\t\tnewDu := newObj.(*velerov2alpha1api.DataUpload)\n\n\t\t\t\tif newDu.Name != r.dataUploadName {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif newDu.Status.Phase != velerov2alpha1api.DataUploadPhaseInProgress {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif newDu.Spec.Cancel && !oldDu.Spec.Cancel {\n\t\t\t\t\tr.cancelDataUpload(newDu)\n\t\t\t\t}\n\t\t\t},\n\t\t},\n\t)\n\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error adding du handler\")\n\t}\n\n\tr.duHandler = handler\n\n\treturn err\n}\n\nfunc (r *BackupMicroService) RunCancelableDataPath(ctx context.Context) (string, error) {\n\tlog := r.logger.WithFields(logrus.Fields{\n\t\t\"dataupload\": r.dataUploadName,\n\t})\n\n\tdu := &velerov2alpha1api.DataUpload{}\n\terr := wait.PollUntilContextCancel(ctx, 500*time.Millisecond, true, func(ctx context.Context) (bool, error) {\n\t\terr := r.client.Get(ctx, types.NamespacedName{\n\t\t\tNamespace: r.namespace,\n\t\t\tName: r.dataUploadName,\n\t\t}, du)\n\n\t\tif apierrors.IsNotFound(err) {\n\t\t\treturn false, nil\n\t\t}\n\n\t\tif err != nil {\n\t\t\treturn true, errors.Wrapf(err, \"error to get du %s\", r.dataUploadName)\n\t\t}\n\n\t\tif du.Status.Phase == velerov2alpha1api.DataUploadPhaseInProgress {\n\t\t\treturn true, nil\n\t\t} else {\n\t\t\treturn false, nil\n\t\t}\n\t})\n\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"Failed to wait du\")\n\t\treturn \"\", errors.Wrap(err, \"error waiting for du\")\n\t}\n\n\tr.dataUpload = du\n\n\tlog.Info(\"Run cancelable dataUpload\")\n\n\tcallbacks := datapath.Callbacks{\n\t\tOnCompleted: r.OnDataUploadCompleted,\n\t\tOnFailed: r.OnDataUploadFailed,\n\t\tOnCancelled: r.OnDataUploadCancelled,\n\t\tOnProgress: r.OnDataUploadProgress,\n\t}\n\n\tfsBackup, err := r.dataPathMgr.CreateFileSystemBR(du.Name, dataUploadDownloadRequestor, ctx, r.client, du.Namespace, callbacks, log)\n\tif err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error to create data path\")\n\t}\n\n\tlog.Debug(\"Async fs br created\")\n\n\tif err := fsBackup.Init(ctx, &datapath.FSBRInitParam{\n\t\tBSLName: du.Spec.BackupStorageLocation,\n\t\tSourceNamespace: du.Spec.SourceNamespace,\n\t\tUploaderType: GetUploaderType(du.Spec.DataMover),\n\t\tRepositoryType: velerov1api.BackupRepositoryTypeKopia,\n\t\tRepoIdentifier: \"\",\n\t\tRepositoryEnsurer: r.repoEnsurer,\n\t\tCredentialGetter: r.credentialGetter,\n\t}); err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error to initialize data path\")\n\t}\n\n\tlog.Info(\"Async fs br init\")\n\n\ttags := map[string]string{\n\t\tvelerov1api.AsyncOperationIDLabel: du.Labels[velerov1api.AsyncOperationIDLabel],\n\t}\n\n\tif err := fsBackup.StartBackup(r.sourceTargetPath, du.Spec.DataMoverConfig, &datapath.FSBRStartParam{\n\t\tRealSource: GetRealSource(du.Spec.SourceNamespace, du.Spec.SourcePVC),\n\t\tParentSnapshot: \"\",\n\t\tForceFull: false,\n\t\tTags: tags,\n\t}); err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error starting data path backup\")\n\t}\n\n\tlog.Info(\"Async fs backup data path started\")\n\tr.eventRecorder.Event(du, false, datapath.EventReasonStarted, \"Data path for %s started\", du.Name)\n\n\tresult := \"\"\n\tselect {\n\tcase <-ctx.Done():\n\t\terr = errors.New(\"timed out waiting for fs backup to complete\")\n\t\tbreak\n\tcase res := <-r.resultSignal:\n\t\terr = res.err\n\t\tresult = res.result\n\t\tbreak\n\t}\n\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"Async fs backup was not completed\")\n\t}\n\n\tr.eventRecorder.EndingEvent(du, false, datapath.EventReasonStopped, \"Data path for %s stopped\", du.Name)\n\n\treturn result, err\n}\n\nfunc (r *BackupMicroService) Shutdown() {\n\tr.eventRecorder.Shutdown()\n\tr.closeDataPath(r.ctx, r.dataUploadName)\n\n\tif r.duHandler != nil {\n\t\tif err := r.duInformer.RemoveEventHandler(r.duHandler); err != nil {\n\t\t\tr.logger.WithError(err).Warn(\"Failed to remove pod handler\")\n\t\t}\n\t}\n}\n\nvar funcMarshal = json.Marshal\n\nfunc (r *BackupMicroService) OnDataUploadCompleted(ctx context.Context, namespace string, duName string, result datapath.Result) {\n\tlog := r.logger.WithField(\"dataupload\", duName)\n\n\tbackupBytes, err := funcMarshal(result.Backup)\n\tif err != nil {\n\t\tlog.WithError(err).Errorf(\"Failed to marshal backup result %v\", result.Backup)\n\t\tr.resultSignal <- dataPathResult{\n\t\t\terr: errors.Wrapf(err, \"Failed to marshal backup result %v\", result.Backup),\n\t\t}\n\t} else {\n\t\tr.eventRecorder.Event(r.dataUpload, false, datapath.EventReasonCompleted, string(backupBytes))\n\t\tr.resultSignal <- dataPathResult{\n\t\t\tresult: string(backupBytes),\n\t\t}\n\t}\n\n\tlog.Info(\"Async fs backup completed\")\n}\n\nfunc (r *BackupMicroService) OnDataUploadFailed(ctx context.Context, namespace string, duName string, err error) {\n\tlog := r.logger.WithField(\"dataupload\", duName)\n\tlog.WithError(err).Error(\"Async fs backup data path failed\")\n\n\tr.eventRecorder.Event(r.dataUpload, false, datapath.EventReasonFailed, \"Data path for data upload %s failed, error %v\", r.dataUploadName, err)\n\tr.resultSignal <- dataPathResult{\n\t\terr: errors.Wrapf(err, \"Data path for data upload %s failed\", r.dataUploadName),\n\t}\n}\n\nfunc (r *BackupMicroService) OnDataUploadCancelled(ctx context.Context, namespace string, duName string) {\n\tlog := r.logger.WithField(\"dataupload\", duName)\n\tlog.Warn(\"Async fs backup data path canceled\")\n\n\tr.eventRecorder.Event(r.dataUpload, false, datapath.EventReasonCancelled, \"Data path for data upload %s canceled\", duName)\n\tr.resultSignal <- dataPathResult{\n\t\terr: errors.New(datapath.ErrCancelled),\n\t}\n}\n\nfunc (r *BackupMicroService) OnDataUploadProgress(ctx context.Context, namespace string, duName string, progress *uploader.Progress) {\n\tlog := r.logger.WithFields(logrus.Fields{\n\t\t\"dataupload\": duName,\n\t})\n\n\tprogressBytes, err := funcMarshal(progress)\n\tif err != nil {\n\t\tlog.WithError(err).Errorf(\"Failed to marshal progress %v\", progress)\n\t\treturn\n\t}\n\n\tr.eventRecorder.Event(r.dataUpload, false, datapath.EventReasonProgress, string(progressBytes))\n}\n\nfunc (r *BackupMicroService) closeDataPath(ctx context.Context, duName string) {\n\tfsBackup := r.dataPathMgr.GetAsyncBR(duName)\n\tif fsBackup != nil {\n\t\tfsBackup.Close(ctx)\n\t}\n\n\tr.dataPathMgr.RemoveAsyncBR(duName)\n}\n\nfunc (r *BackupMicroService) cancelDataUpload(du *velerov2alpha1api.DataUpload) {\n\tr.logger.WithField(\"DataUpload\", du.Name).Info(\"Data upload is being canceled\")\n\n\tr.eventRecorder.Event(du, false, datapath.EventReasonCancelling, \"Canceling for data upload %s\", du.Name)\n\n\tfsBackup := r.dataPathMgr.GetAsyncBR(du.Name)\n\tif fsBackup == nil {\n\t\tr.OnDataUploadCancelled(r.ctx, du.GetNamespace(), du.GetName())\n\t} else {\n\t\tfsBackup.Cancel()\n\t}\n}\n" }, { "path": "pkg/datamover/backup_micro_service_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage datamover\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"sync\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\n\tclientFake \"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tdatapathmockes \"github.com/vmware-tanzu/velero/pkg/datapath/mocks\"\n)\n\ntype backupMsTestHelper struct {\n\teventReason string\n\teventMsg string\n\tmarshalErr error\n\tmarshalBytes []byte\n\twithEvent bool\n\teventLock sync.Mutex\n}\n\nfunc (bt *backupMsTestHelper) Event(_ runtime.Object, _ bool, reason string, message string, a ...any) {\n\tbt.eventLock.Lock()\n\tdefer bt.eventLock.Unlock()\n\n\tbt.withEvent = true\n\tbt.eventReason = reason\n\tbt.eventMsg = fmt.Sprintf(message, a...)\n}\n\nfunc (bt *backupMsTestHelper) EndingEvent(_ runtime.Object, _ bool, reason string, message string, a ...any) {\n\tbt.eventLock.Lock()\n\tdefer bt.eventLock.Unlock()\n\n\tbt.withEvent = true\n\tbt.eventReason = reason\n\tbt.eventMsg = fmt.Sprintf(message, a...)\n}\nfunc (bt *backupMsTestHelper) Shutdown() {}\n\nfunc (bt *backupMsTestHelper) Marshal(v any) ([]byte, error) {\n\tif bt.marshalErr != nil {\n\t\treturn nil, bt.marshalErr\n\t}\n\n\treturn bt.marshalBytes, nil\n}\n\nfunc (bt *backupMsTestHelper) EventReason() string {\n\tbt.eventLock.Lock()\n\tdefer bt.eventLock.Unlock()\n\n\treturn bt.eventReason\n}\n\nfunc (bt *backupMsTestHelper) EventMessage() string {\n\tbt.eventLock.Lock()\n\tdefer bt.eventLock.Unlock()\n\n\treturn bt.eventMsg\n}\n\nfunc TestOnDataUploadFailed(t *testing.T) {\n\tdataUploadName := \"fake-data-upload\"\n\tbt := &backupMsTestHelper{}\n\n\tbs := &BackupMicroService{\n\t\tdataUploadName: dataUploadName,\n\t\tdataPathMgr: datapath.NewManager(1),\n\t\teventRecorder: bt,\n\t\tresultSignal: make(chan dataPathResult),\n\t\tlogger: velerotest.NewLogger(),\n\t}\n\n\texpectedErr := \"Data path for data upload fake-data-upload failed: fake-error\"\n\texpectedEventReason := datapath.EventReasonFailed\n\texpectedEventMsg := \"Data path for data upload fake-data-upload failed, error fake-error\"\n\n\tgo bs.OnDataUploadFailed(t.Context(), velerov1api.DefaultNamespace, dataUploadName, errors.New(\"fake-error\"))\n\n\tresult := <-bs.resultSignal\n\trequire.EqualError(t, result.err, expectedErr)\n\tassert.Equal(t, expectedEventReason, bt.EventReason())\n\tassert.Equal(t, expectedEventMsg, bt.EventMessage())\n}\n\nfunc TestOnDataUploadCancelled(t *testing.T) {\n\tdataUploadName := \"fake-data-upload\"\n\tbt := &backupMsTestHelper{}\n\n\tbs := &BackupMicroService{\n\t\tdataUploadName: dataUploadName,\n\t\tdataPathMgr: datapath.NewManager(1),\n\t\teventRecorder: bt,\n\t\tresultSignal: make(chan dataPathResult),\n\t\tlogger: velerotest.NewLogger(),\n\t}\n\n\texpectedErr := datapath.ErrCancelled\n\texpectedEventReason := datapath.EventReasonCancelled\n\texpectedEventMsg := \"Data path for data upload fake-data-upload canceled\"\n\n\tgo bs.OnDataUploadCancelled(t.Context(), velerov1api.DefaultNamespace, dataUploadName)\n\n\tresult := <-bs.resultSignal\n\trequire.EqualError(t, result.err, expectedErr)\n\tassert.Equal(t, expectedEventReason, bt.EventReason())\n\tassert.Equal(t, expectedEventMsg, bt.EventMessage())\n}\n\nfunc TestOnDataUploadCompleted(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\texpectedErr string\n\t\texpectedEventReason string\n\t\texpectedEventMsg string\n\t\tmarshalErr error\n\t\tmarshallStr string\n\t}{\n\t\t{\n\t\t\tname: \"marshal fail\",\n\t\t\tmarshalErr: errors.New(\"fake-marshal-error\"),\n\t\t\texpectedErr: \"Failed to marshal backup result { false { } 0 0}: fake-marshal-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tmarshallStr: \"fake-complete-string\",\n\t\t\texpectedEventReason: datapath.EventReasonCompleted,\n\t\t\texpectedEventMsg: \"fake-complete-string\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tdataUploadName := \"fake-data-upload\"\n\n\t\t\tbt := &backupMsTestHelper{\n\t\t\t\tmarshalErr: test.marshalErr,\n\t\t\t\tmarshalBytes: []byte(test.marshallStr),\n\t\t\t}\n\n\t\t\tbs := &BackupMicroService{\n\t\t\t\tdataPathMgr: datapath.NewManager(1),\n\t\t\t\teventRecorder: bt,\n\t\t\t\tresultSignal: make(chan dataPathResult),\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tfuncMarshal = bt.Marshal\n\n\t\t\tgo bs.OnDataUploadCompleted(t.Context(), velerov1api.DefaultNamespace, dataUploadName, datapath.Result{})\n\n\t\t\tresult := <-bs.resultSignal\n\t\t\tif test.marshalErr != nil {\n\t\t\t\tassert.EqualError(t, result.err, test.expectedErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, result.err)\n\t\t\t\tassert.Equal(t, test.expectedEventReason, bt.EventReason())\n\t\t\t\tassert.Equal(t, test.expectedEventMsg, bt.EventMessage())\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestOnDataUploadProgress(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\texpectedErr string\n\t\texpectedEventReason string\n\t\texpectedEventMsg string\n\t\tmarshalErr error\n\t\tmarshallStr string\n\t}{\n\t\t{\n\t\t\tname: \"marshal fail\",\n\t\t\tmarshalErr: errors.New(\"fake-marshal-error\"),\n\t\t\texpectedErr: \"Failed to marshal backup result\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tmarshallStr: \"fake-progress-string\",\n\t\t\texpectedEventReason: datapath.EventReasonProgress,\n\t\t\texpectedEventMsg: \"fake-progress-string\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tdataUploadName := \"fake-data-upload\"\n\n\t\t\tbt := &backupMsTestHelper{\n\t\t\t\tmarshalErr: test.marshalErr,\n\t\t\t\tmarshalBytes: []byte(test.marshallStr),\n\t\t\t}\n\n\t\t\tbs := &BackupMicroService{\n\t\t\t\tdataPathMgr: datapath.NewManager(1),\n\t\t\t\teventRecorder: bt,\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tfuncMarshal = bt.Marshal\n\n\t\t\tbs.OnDataUploadProgress(t.Context(), velerov1api.DefaultNamespace, dataUploadName, &uploader.Progress{})\n\n\t\t\tif test.marshalErr != nil {\n\t\t\t\tassert.False(t, bt.withEvent)\n\t\t\t} else {\n\t\t\t\tassert.True(t, bt.withEvent)\n\t\t\t\tassert.Equal(t, test.expectedEventReason, bt.EventReason())\n\t\t\t\tassert.Equal(t, test.expectedEventMsg, bt.EventMessage())\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestCancelDataUpload(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\texpectedEventReason string\n\t\texpectedEventMsg string\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"no fs backup\",\n\t\t\texpectedEventReason: datapath.EventReasonCancelled,\n\t\t\texpectedEventMsg: \"Data path for data upload fake-data-upload canceled\",\n\t\t\texpectedErr: datapath.ErrCancelled,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tdataUploadName := \"fake-data-upload\"\n\t\t\tdu := builder.ForDataUpload(velerov1api.DefaultNamespace, dataUploadName).Result()\n\n\t\t\tbt := &backupMsTestHelper{}\n\n\t\t\tbs := &BackupMicroService{\n\t\t\t\tdataPathMgr: datapath.NewManager(1),\n\t\t\t\teventRecorder: bt,\n\t\t\t\tresultSignal: make(chan dataPathResult),\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tgo bs.cancelDataUpload(du)\n\n\t\t\tresult := <-bs.resultSignal\n\n\t\t\trequire.EqualError(t, result.err, test.expectedErr)\n\t\t\tassert.True(t, bt.withEvent)\n\t\t\tassert.Equal(t, test.expectedEventReason, bt.EventReason())\n\t\t\tassert.Equal(t, test.expectedEventMsg, bt.EventMessage())\n\t\t})\n\t}\n}\n\nfunc TestRunCancelableDataPath(t *testing.T) {\n\tdataUploadName := \"fake-data-upload\"\n\tdu := builder.ForDataUpload(velerov1api.DefaultNamespace, dataUploadName).Phase(velerov2alpha1api.DataUploadPhaseNew).Result()\n\tduInProgress := builder.ForDataUpload(velerov1api.DefaultNamespace, dataUploadName).Phase(velerov2alpha1api.DataUploadPhaseInProgress).Result()\n\tctxTimeout, cancel := context.WithTimeout(t.Context(), time.Second)\n\n\ttests := []struct {\n\t\tname string\n\t\tctx context.Context\n\t\tresult *dataPathResult\n\t\tdataPathMgr *datapath.Manager\n\t\tkubeClientObj []runtime.Object\n\t\tinitErr error\n\t\tstartErr error\n\t\tdataPathStarted bool\n\t\texpectedEventMsg string\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"no du\",\n\t\t\tctx: ctxTimeout,\n\t\t\texpectedErr: \"error waiting for du: context deadline exceeded\",\n\t\t},\n\t\t{\n\t\t\tname: \"du not in in-progress\",\n\t\t\tctx: ctxTimeout,\n\t\t\tkubeClientObj: []runtime.Object{du},\n\t\t\texpectedErr: \"error waiting for du: context deadline exceeded\",\n\t\t},\n\t\t{\n\t\t\tname: \"create data path fail\",\n\t\t\tctx: t.Context(),\n\t\t\tkubeClientObj: []runtime.Object{duInProgress},\n\t\t\tdataPathMgr: datapath.NewManager(0),\n\t\t\texpectedErr: \"error to create data path: Concurrent number exceeds\",\n\t\t},\n\t\t{\n\t\t\tname: \"init data path fail\",\n\t\t\tctx: t.Context(),\n\t\t\tkubeClientObj: []runtime.Object{duInProgress},\n\t\t\tinitErr: errors.New(\"fake-init-error\"),\n\t\t\texpectedErr: \"error to initialize data path: fake-init-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"start data path fail\",\n\t\t\tctx: t.Context(),\n\t\t\tkubeClientObj: []runtime.Object{duInProgress},\n\t\t\tstartErr: errors.New(\"fake-start-error\"),\n\t\t\texpectedErr: \"error starting data path backup: fake-start-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"data path timeout\",\n\t\t\tctx: ctxTimeout,\n\t\t\tkubeClientObj: []runtime.Object{duInProgress},\n\t\t\tdataPathStarted: true,\n\t\t\texpectedEventMsg: fmt.Sprintf(\"Data path for %s stopped\", dataUploadName),\n\t\t\texpectedErr: \"timed out waiting for fs backup to complete\",\n\t\t},\n\t\t{\n\t\t\tname: \"data path returns error\",\n\t\t\tctx: t.Context(),\n\t\t\tkubeClientObj: []runtime.Object{duInProgress},\n\t\t\tdataPathStarted: true,\n\t\t\tresult: &dataPathResult{\n\t\t\t\terr: errors.New(\"fake-data-path-error\"),\n\t\t\t},\n\t\t\texpectedEventMsg: fmt.Sprintf(\"Data path for %s stopped\", dataUploadName),\n\t\t\texpectedErr: \"fake-data-path-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tctx: t.Context(),\n\t\t\tkubeClientObj: []runtime.Object{duInProgress},\n\t\t\tdataPathStarted: true,\n\t\t\tresult: &dataPathResult{\n\t\t\t\tresult: \"fake-succeed-result\",\n\t\t\t},\n\t\t\texpectedEventMsg: fmt.Sprintf(\"Data path for %s stopped\", dataUploadName),\n\t\t},\n\t}\n\n\tscheme := runtime.NewScheme()\n\tvelerov2alpha1api.AddToScheme(scheme)\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeClientBuilder := clientFake.NewClientBuilder()\n\t\t\tfakeClientBuilder = fakeClientBuilder.WithScheme(scheme)\n\n\t\t\tfakeClient := fakeClientBuilder.WithRuntimeObjects(test.kubeClientObj...).Build()\n\n\t\t\tbt := &backupMsTestHelper{}\n\n\t\t\tbs := &BackupMicroService{\n\t\t\t\tnamespace: velerov1api.DefaultNamespace,\n\t\t\t\tdataUploadName: dataUploadName,\n\t\t\t\tctx: t.Context(),\n\t\t\t\tclient: fakeClient,\n\t\t\t\tdataPathMgr: datapath.NewManager(1),\n\t\t\t\teventRecorder: bt,\n\t\t\t\tresultSignal: make(chan dataPathResult),\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tif test.ctx != nil {\n\t\t\t\tbs.ctx = test.ctx\n\t\t\t}\n\n\t\t\tif test.dataPathMgr != nil {\n\t\t\t\tbs.dataPathMgr = test.dataPathMgr\n\t\t\t}\n\n\t\t\tdatapath.FSBRCreator = func(string, string, kbclient.Client, string, datapath.Callbacks, logrus.FieldLogger) datapath.AsyncBR {\n\t\t\t\tfsBR := datapathmockes.NewAsyncBR(t)\n\t\t\t\tif test.initErr != nil {\n\t\t\t\t\tfsBR.On(\"Init\", mock.Anything, mock.Anything).Return(test.initErr)\n\t\t\t\t}\n\n\t\t\t\tif test.startErr != nil {\n\t\t\t\t\tfsBR.On(\"Init\", mock.Anything, mock.Anything).Return(nil)\n\t\t\t\t\tfsBR.On(\"StartBackup\", mock.Anything, mock.Anything, mock.Anything).Return(test.startErr)\n\t\t\t\t}\n\n\t\t\t\tif test.dataPathStarted {\n\t\t\t\t\tfsBR.On(\"Init\", mock.Anything, mock.Anything).Return(nil)\n\t\t\t\t\tfsBR.On(\"StartBackup\", mock.Anything, mock.Anything, mock.Anything).Return(nil)\n\t\t\t\t}\n\n\t\t\t\treturn fsBR\n\t\t\t}\n\n\t\t\tif test.result != nil {\n\t\t\t\tgo func() {\n\t\t\t\t\ttime.Sleep(time.Millisecond * 500)\n\t\t\t\t\tbs.resultSignal <- *test.result\n\t\t\t\t}()\n\t\t\t}\n\n\t\t\tresult, err := bs.RunCancelableDataPath(test.ctx)\n\n\t\t\tif test.expectedErr != \"\" {\n\t\t\t\trequire.EqualError(t, err, test.expectedErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, test.result.result, result)\n\t\t\t}\n\n\t\t\tif test.expectedEventMsg != \"\" {\n\t\t\t\tassert.True(t, bt.withEvent)\n\t\t\t\tassert.Equal(t, test.expectedEventMsg, bt.EventMessage())\n\t\t\t}\n\t\t})\n\t}\n\n\tcancel()\n}\n" }, { "path": "pkg/datamover/dataupload_delete_action.go", "content": "package datamover\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\trepotypes \"github.com/vmware-tanzu/velero/pkg/repository/types\"\n)\n\ntype DataUploadDeleteAction struct {\n\tlogger logrus.FieldLogger\n\tclient client.Client\n}\n\nfunc (d *DataUploadDeleteAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"datauploads.velero.io\"},\n\t}, nil\n}\n\nfunc (d *DataUploadDeleteAction) Execute(input *velero.DeleteItemActionExecuteInput) error {\n\td.logger.Infof(\"Executing DataUploadDeleteAction\")\n\tdu := &velerov2alpha1.DataUpload{}\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(input.Item.UnstructuredContent(), &du); err != nil {\n\t\treturn errors.WithStack(errors.Wrapf(err, \"failed to convert input.Item from unstructured\"))\n\t}\n\tcm := genConfigmap(input.Backup, *du)\n\tif cm == nil {\n\t\t// will not fail the backup deletion\n\t\treturn nil\n\t}\n\terr := d.client.Create(context.Background(), cm)\n\tif err != nil {\n\t\treturn errors.WithStack(errors.Wrapf(err, \"failed to create the configmap for DataUpload %s/%s\", du.Namespace, du.Name))\n\t}\n\treturn nil\n}\n\n// generate the configmap which is to be created and used as a way to communicate the snapshot info to the backup deletion controller\nfunc genConfigmap(bak *velerov1.Backup, du velerov2alpha1.DataUpload) *corev1api.ConfigMap {\n\tif !IsBuiltInUploader(du.Spec.DataMover) || du.Status.SnapshotID == \"\" {\n\t\treturn nil\n\t}\n\tsnapshot := repotypes.SnapshotIdentifier{\n\t\tVolumeNamespace: du.Spec.SourceNamespace,\n\t\tBackupStorageLocation: bak.Spec.StorageLocation,\n\t\tSnapshotID: du.Status.SnapshotID,\n\t\tRepositoryType: velerov1.BackupRepositoryTypeKopia,\n\t\tUploaderType: GetUploaderType(du.Spec.DataMover),\n\t\tSource: GetRealSource(du.Spec.SourceNamespace, du.Spec.SourcePVC),\n\t}\n\tb, err := json.Marshal(snapshot)\n\tif err != nil {\n\t\treturn nil\n\t}\n\tdata := make(map[string]string)\n\tif err := json.Unmarshal(b, &data); err != nil {\n\t\treturn nil\n\t}\n\treturn &corev1api.ConfigMap{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: corev1api.SchemeGroupVersion.String(),\n\t\t\tKind: \"ConfigMap\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: bak.Namespace,\n\t\t\tName: fmt.Sprintf(\"%s-info\", du.Name),\n\t\t\tLabels: map[string]string{\n\t\t\t\tvelerov1.BackupNameLabel: bak.Name,\n\t\t\t\tvelerov1.DataUploadSnapshotInfoLabel: \"true\",\n\t\t\t},\n\t\t},\n\t\tData: data,\n\t}\n}\n\nfunc NewDataUploadDeleteAction(logger logrus.FieldLogger, client client.Client) *DataUploadDeleteAction {\n\treturn &DataUploadDeleteAction{\n\t\tlogger: logger,\n\t\tclient: client,\n\t}\n}\n" }, { "path": "pkg/datamover/restore_micro_service.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage datamover\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"sigs.k8s.io/controller-runtime/pkg/cache\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\n\tcachetool \"k8s.io/client-go/tools/cache\"\n)\n\n// RestoreMicroService process data mover restores inside the restore pod\ntype RestoreMicroService struct {\n\tctx context.Context\n\tclient client.Client\n\tkubeClient kubernetes.Interface\n\trepoEnsurer *repository.Ensurer\n\tcredentialGetter *credentials.CredentialGetter\n\tlogger logrus.FieldLogger\n\tdataPathMgr *datapath.Manager\n\teventRecorder kube.EventRecorder\n\n\tnamespace string\n\tdataDownloadName string\n\tdataDownload *velerov2alpha1api.DataDownload\n\tsourceTargetPath datapath.AccessPoint\n\n\tresultSignal chan dataPathResult\n\n\tddInformer cache.Informer\n\tddHandler cachetool.ResourceEventHandlerRegistration\n\tnodeName string\n\tcacheDir string\n}\n\nfunc NewRestoreMicroService(ctx context.Context, client client.Client, kubeClient kubernetes.Interface, dataDownloadName string, namespace string, nodeName string,\n\tsourceTargetPath datapath.AccessPoint, dataPathMgr *datapath.Manager, repoEnsurer *repository.Ensurer, cred *credentials.CredentialGetter,\n\tddInformer cache.Informer, cacheDir string, log logrus.FieldLogger) *RestoreMicroService {\n\treturn &RestoreMicroService{\n\t\tctx: ctx,\n\t\tclient: client,\n\t\tkubeClient: kubeClient,\n\t\tcredentialGetter: cred,\n\t\tlogger: log,\n\t\trepoEnsurer: repoEnsurer,\n\t\tdataPathMgr: dataPathMgr,\n\t\tnamespace: namespace,\n\t\tdataDownloadName: dataDownloadName,\n\t\tsourceTargetPath: sourceTargetPath,\n\t\tnodeName: nodeName,\n\t\tresultSignal: make(chan dataPathResult),\n\t\tddInformer: ddInformer,\n\t\tcacheDir: cacheDir,\n\t}\n}\n\nfunc (r *RestoreMicroService) Init() error {\n\tr.eventRecorder = kube.NewEventRecorder(r.kubeClient, r.client.Scheme(), r.dataDownloadName, r.nodeName, r.logger)\n\n\thandler, err := r.ddInformer.AddEventHandler(\n\t\tcachetool.ResourceEventHandlerFuncs{\n\t\t\tUpdateFunc: func(oldObj any, newObj any) {\n\t\t\t\toldDd := oldObj.(*velerov2alpha1api.DataDownload)\n\t\t\t\tnewDd := newObj.(*velerov2alpha1api.DataDownload)\n\n\t\t\t\tif newDd.Name != r.dataDownloadName {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif newDd.Status.Phase != velerov2alpha1api.DataDownloadPhaseInProgress {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif newDd.Spec.Cancel && !oldDd.Spec.Cancel {\n\t\t\t\t\tr.cancelDataDownload(newDd)\n\t\t\t\t}\n\t\t\t},\n\t\t},\n\t)\n\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error adding dd handler\")\n\t}\n\n\tr.ddHandler = handler\n\n\treturn err\n}\n\nfunc (r *RestoreMicroService) RunCancelableDataPath(ctx context.Context) (string, error) {\n\tlog := r.logger.WithFields(logrus.Fields{\n\t\t\"datadownload\": r.dataDownloadName,\n\t})\n\n\tdd := &velerov2alpha1api.DataDownload{}\n\terr := wait.PollUntilContextCancel(ctx, 500*time.Millisecond, true, func(ctx context.Context) (bool, error) {\n\t\terr := r.client.Get(ctx, types.NamespacedName{\n\t\t\tNamespace: r.namespace,\n\t\t\tName: r.dataDownloadName,\n\t\t}, dd)\n\t\tif apierrors.IsNotFound(err) {\n\t\t\treturn false, nil\n\t\t}\n\n\t\tif err != nil {\n\t\t\treturn true, errors.Wrapf(err, \"error to get dd %s\", r.dataDownloadName)\n\t\t}\n\n\t\tif dd.Status.Phase == velerov2alpha1api.DataDownloadPhaseInProgress {\n\t\t\treturn true, nil\n\t\t} else {\n\t\t\treturn false, nil\n\t\t}\n\t})\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"Failed to wait dd\")\n\t\treturn \"\", errors.Wrap(err, \"error waiting for dd\")\n\t}\n\n\tr.dataDownload = dd\n\n\tlog.Info(\"Run cancelable dataDownload\")\n\n\tcallbacks := datapath.Callbacks{\n\t\tOnCompleted: r.OnDataDownloadCompleted,\n\t\tOnFailed: r.OnDataDownloadFailed,\n\t\tOnCancelled: r.OnDataDownloadCancelled,\n\t\tOnProgress: r.OnDataDownloadProgress,\n\t}\n\n\tfsRestore, err := r.dataPathMgr.CreateFileSystemBR(dd.Name, dataUploadDownloadRequestor, ctx, r.client, dd.Namespace, callbacks, log)\n\tif err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error to create data path\")\n\t}\n\n\tlog.Debug(\"Found volume path\")\n\tif err := fsRestore.Init(ctx,\n\t\t&datapath.FSBRInitParam{\n\t\t\tBSLName: dd.Spec.BackupStorageLocation,\n\t\t\tSourceNamespace: dd.Spec.SourceNamespace,\n\t\t\tUploaderType: GetUploaderType(dd.Spec.DataMover),\n\t\t\tRepositoryType: velerov1api.BackupRepositoryTypeKopia,\n\t\t\tRepoIdentifier: \"\",\n\t\t\tRepositoryEnsurer: r.repoEnsurer,\n\t\t\tCredentialGetter: r.credentialGetter,\n\t\t\tCacheDir: r.cacheDir,\n\t\t}); err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error to initialize data path\")\n\t}\n\tlog.Info(\"fs init\")\n\n\tif err := fsRestore.StartRestore(dd.Spec.SnapshotID, r.sourceTargetPath, dd.Spec.DataMoverConfig); err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error starting data path restore\")\n\t}\n\n\tlog.Info(\"Async fs restore data path started\")\n\tr.eventRecorder.Event(dd, false, datapath.EventReasonStarted, \"Data path for %s started\", dd.Name)\n\n\tresult := \"\"\n\tselect {\n\tcase <-ctx.Done():\n\t\terr = errors.New(\"timed out waiting for fs restore to complete\")\n\t\tbreak\n\tcase res := <-r.resultSignal:\n\t\terr = res.err\n\t\tresult = res.result\n\t\tbreak\n\t}\n\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"Async fs restore was not completed\")\n\t}\n\n\tr.eventRecorder.EndingEvent(dd, false, datapath.EventReasonStopped, \"Data path for %s stopped\", dd.Name)\n\n\treturn result, err\n}\n\nfunc (r *RestoreMicroService) Shutdown() {\n\tr.eventRecorder.Shutdown()\n\tr.closeDataPath(r.ctx, r.dataDownloadName)\n\n\tif r.ddHandler != nil {\n\t\tif err := r.ddInformer.RemoveEventHandler(r.ddHandler); err != nil {\n\t\t\tr.logger.WithError(err).Warn(\"Failed to remove pod handler\")\n\t\t}\n\t}\n}\n\nfunc (r *RestoreMicroService) OnDataDownloadCompleted(ctx context.Context, namespace string, ddName string, result datapath.Result) {\n\tlog := r.logger.WithField(\"datadownload\", ddName)\n\n\trestoreBytes, err := funcMarshal(result.Restore)\n\tif err != nil {\n\t\tlog.WithError(err).Errorf(\"Failed to marshal restore result %v\", result.Restore)\n\t\tr.resultSignal <- dataPathResult{\n\t\t\terr: errors.Wrapf(err, \"Failed to marshal restore result %v\", result.Restore),\n\t\t}\n\t} else {\n\t\tr.eventRecorder.Event(r.dataDownload, false, datapath.EventReasonCompleted, string(restoreBytes))\n\t\tr.resultSignal <- dataPathResult{\n\t\t\tresult: string(restoreBytes),\n\t\t}\n\t}\n\n\tlog.Info(\"Async fs restore data path completed\")\n}\n\nfunc (r *RestoreMicroService) OnDataDownloadFailed(ctx context.Context, namespace string, ddName string, err error) {\n\tlog := r.logger.WithField(\"datadownload\", ddName)\n\tlog.WithError(err).Error(\"Async fs restore data path failed\")\n\n\tr.eventRecorder.Event(r.dataDownload, false, datapath.EventReasonFailed, \"Data path for data download %s failed, error %v\", r.dataDownloadName, err)\n\tr.resultSignal <- dataPathResult{\n\t\terr: errors.Wrapf(err, \"Data path for data download %s failed\", r.dataDownloadName),\n\t}\n}\n\nfunc (r *RestoreMicroService) OnDataDownloadCancelled(ctx context.Context, namespace string, ddName string) {\n\tlog := r.logger.WithField(\"datadownload\", ddName)\n\tlog.Warn(\"Async fs restore data path canceled\")\n\n\tr.eventRecorder.Event(r.dataDownload, false, datapath.EventReasonCancelled, \"Data path for data download %s canceled\", ddName)\n\tr.resultSignal <- dataPathResult{\n\t\terr: errors.New(datapath.ErrCancelled),\n\t}\n}\n\nfunc (r *RestoreMicroService) OnDataDownloadProgress(ctx context.Context, namespace string, ddName string, progress *uploader.Progress) {\n\tlog := r.logger.WithFields(logrus.Fields{\n\t\t\"datadownload\": ddName,\n\t})\n\n\tprogressBytes, err := funcMarshal(progress)\n\tif err != nil {\n\t\tlog.WithError(err).Errorf(\"Failed to marshal progress %v\", progress)\n\t\treturn\n\t}\n\n\tr.eventRecorder.Event(r.dataDownload, false, datapath.EventReasonProgress, string(progressBytes))\n}\n\nfunc (r *RestoreMicroService) closeDataPath(ctx context.Context, ddName string) {\n\tfsRestore := r.dataPathMgr.GetAsyncBR(ddName)\n\tif fsRestore != nil {\n\t\tfsRestore.Close(ctx)\n\t}\n\n\tr.dataPathMgr.RemoveAsyncBR(ddName)\n}\n\nfunc (r *RestoreMicroService) cancelDataDownload(dd *velerov2alpha1api.DataDownload) {\n\tr.logger.WithField(\"DataDownload\", dd.Name).Info(\"Data download is being canceled\")\n\n\tr.eventRecorder.Event(dd, false, datapath.EventReasonCancelling, \"Canceling for data download %s\", dd.Name)\n\n\tfsBackup := r.dataPathMgr.GetAsyncBR(dd.Name)\n\tif fsBackup == nil {\n\t\tr.OnDataDownloadCancelled(r.ctx, dd.GetNamespace(), dd.GetName())\n\t} else {\n\t\tfsBackup.Cancel()\n\t}\n}\n" }, { "path": "pkg/datamover/restore_micro_service_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage datamover\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\tclientFake \"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\tdatapathmockes \"github.com/vmware-tanzu/velero/pkg/datapath/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestOnDataDownloadFailed(t *testing.T) {\n\tdataDownloadName := \"fake-data-download\"\n\tbt := &backupMsTestHelper{}\n\n\tbs := &RestoreMicroService{\n\t\tdataDownloadName: dataDownloadName,\n\t\tdataPathMgr: datapath.NewManager(1),\n\t\teventRecorder: bt,\n\t\tresultSignal: make(chan dataPathResult),\n\t\tlogger: velerotest.NewLogger(),\n\t}\n\n\texpectedErr := \"Data path for data download fake-data-download failed: fake-error\"\n\texpectedEventReason := datapath.EventReasonFailed\n\texpectedEventMsg := \"Data path for data download fake-data-download failed, error fake-error\"\n\n\tgo bs.OnDataDownloadFailed(t.Context(), velerov1api.DefaultNamespace, dataDownloadName, errors.New(\"fake-error\"))\n\n\tresult := <-bs.resultSignal\n\trequire.EqualError(t, result.err, expectedErr)\n\tassert.Equal(t, expectedEventReason, bt.EventReason())\n\tassert.Equal(t, expectedEventMsg, bt.EventMessage())\n}\n\nfunc TestOnDataDownloadCancelled(t *testing.T) {\n\tdataDownloadName := \"fake-data-download\"\n\tbt := &backupMsTestHelper{}\n\n\tbs := &RestoreMicroService{\n\t\tdataDownloadName: dataDownloadName,\n\t\tdataPathMgr: datapath.NewManager(1),\n\t\teventRecorder: bt,\n\t\tresultSignal: make(chan dataPathResult),\n\t\tlogger: velerotest.NewLogger(),\n\t}\n\n\texpectedErr := datapath.ErrCancelled\n\texpectedEventReason := datapath.EventReasonCancelled\n\texpectedEventMsg := \"Data path for data download fake-data-download canceled\"\n\n\tgo bs.OnDataDownloadCancelled(t.Context(), velerov1api.DefaultNamespace, dataDownloadName)\n\n\tresult := <-bs.resultSignal\n\trequire.EqualError(t, result.err, expectedErr)\n\tassert.Equal(t, expectedEventReason, bt.EventReason())\n\tassert.Equal(t, expectedEventMsg, bt.EventMessage())\n}\n\nfunc TestOnDataDownloadCompleted(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\texpectedErr string\n\t\texpectedEventReason string\n\t\texpectedEventMsg string\n\t\tmarshalErr error\n\t\tmarshallStr string\n\t}{\n\t\t{\n\t\t\tname: \"marshal fail\",\n\t\t\tmarshalErr: errors.New(\"fake-marshal-error\"),\n\t\t\texpectedErr: \"Failed to marshal restore result {{ } 0}: fake-marshal-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tmarshallStr: \"fake-complete-string\",\n\t\t\texpectedEventReason: datapath.EventReasonCompleted,\n\t\t\texpectedEventMsg: \"fake-complete-string\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tdataDownloadName := \"fake-data-download\"\n\n\t\t\tbt := &backupMsTestHelper{\n\t\t\t\tmarshalErr: test.marshalErr,\n\t\t\t\tmarshalBytes: []byte(test.marshallStr),\n\t\t\t}\n\n\t\t\tbs := &RestoreMicroService{\n\t\t\t\tdataPathMgr: datapath.NewManager(1),\n\t\t\t\teventRecorder: bt,\n\t\t\t\tresultSignal: make(chan dataPathResult),\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tfuncMarshal = bt.Marshal\n\n\t\t\tgo bs.OnDataDownloadCompleted(t.Context(), velerov1api.DefaultNamespace, dataDownloadName, datapath.Result{})\n\n\t\t\tresult := <-bs.resultSignal\n\t\t\tif test.marshalErr != nil {\n\t\t\t\tassert.EqualError(t, result.err, test.expectedErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, result.err)\n\t\t\t\tassert.Equal(t, test.expectedEventReason, bt.EventReason())\n\t\t\t\tassert.Equal(t, test.expectedEventMsg, bt.EventMessage())\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestOnDataDownloadProgress(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\texpectedEventReason string\n\t\texpectedEventMsg string\n\t\tmarshalErr error\n\t\tmarshallStr string\n\t}{\n\t\t{\n\t\t\tname: \"marshal fail\",\n\t\t\tmarshalErr: errors.New(\"fake-marshal-error\"),\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tmarshallStr: \"fake-progress-string\",\n\t\t\texpectedEventReason: datapath.EventReasonProgress,\n\t\t\texpectedEventMsg: \"fake-progress-string\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tdataDownloadName := \"fake-data-download\"\n\n\t\t\tbt := &backupMsTestHelper{\n\t\t\t\tmarshalErr: test.marshalErr,\n\t\t\t\tmarshalBytes: []byte(test.marshallStr),\n\t\t\t}\n\n\t\t\tbs := &RestoreMicroService{\n\t\t\t\tdataPathMgr: datapath.NewManager(1),\n\t\t\t\teventRecorder: bt,\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tfuncMarshal = bt.Marshal\n\n\t\t\tbs.OnDataDownloadProgress(t.Context(), velerov1api.DefaultNamespace, dataDownloadName, &uploader.Progress{})\n\n\t\t\tif test.marshalErr != nil {\n\t\t\t\tassert.False(t, bt.withEvent)\n\t\t\t} else {\n\t\t\t\tassert.True(t, bt.withEvent)\n\t\t\t\tassert.Equal(t, test.expectedEventReason, bt.EventReason())\n\t\t\t\tassert.Equal(t, test.expectedEventMsg, bt.EventMessage())\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestCancelDataDownload(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\texpectedEventReason string\n\t\texpectedEventMsg string\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"no fs restore\",\n\t\t\texpectedEventReason: datapath.EventReasonCancelled,\n\t\t\texpectedEventMsg: \"Data path for data download fake-data-download canceled\",\n\t\t\texpectedErr: datapath.ErrCancelled,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tdataDownloadName := \"fake-data-download\"\n\t\t\tdd := builder.ForDataDownload(velerov1api.DefaultNamespace, dataDownloadName).Result()\n\n\t\t\tbt := &backupMsTestHelper{}\n\n\t\t\tbs := &RestoreMicroService{\n\t\t\t\tdataPathMgr: datapath.NewManager(1),\n\t\t\t\teventRecorder: bt,\n\t\t\t\tresultSignal: make(chan dataPathResult),\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tgo bs.cancelDataDownload(dd)\n\n\t\t\tresult := <-bs.resultSignal\n\n\t\t\trequire.EqualError(t, result.err, test.expectedErr)\n\t\t\tassert.True(t, bt.withEvent)\n\t\t\tassert.Equal(t, test.expectedEventReason, bt.EventReason())\n\t\t\tassert.Equal(t, test.expectedEventMsg, bt.EventMessage())\n\t\t})\n\t}\n}\n\nfunc TestRunCancelableRestore(t *testing.T) {\n\tdataDownloadName := \"fake-data-download\"\n\tdd := builder.ForDataDownload(velerov1api.DefaultNamespace, dataDownloadName).Phase(velerov2alpha1api.DataDownloadPhaseNew).Result()\n\tddInProgress := builder.ForDataDownload(velerov1api.DefaultNamespace, dataDownloadName).Phase(velerov2alpha1api.DataDownloadPhaseInProgress).Result()\n\tctxTimeout, cancel := context.WithTimeout(t.Context(), time.Second)\n\n\ttests := []struct {\n\t\tname string\n\t\tctx context.Context\n\t\tresult *dataPathResult\n\t\tdataPathMgr *datapath.Manager\n\t\tkubeClientObj []runtime.Object\n\t\tinitErr error\n\t\tstartErr error\n\t\tdataPathStarted bool\n\t\texpectedEventMsg string\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"no dd\",\n\t\t\tctx: ctxTimeout,\n\t\t\texpectedErr: \"error waiting for dd: context deadline exceeded\",\n\t\t},\n\t\t{\n\t\t\tname: \"dd not in in-progress\",\n\t\t\tctx: ctxTimeout,\n\t\t\tkubeClientObj: []runtime.Object{dd},\n\t\t\texpectedErr: \"error waiting for dd: context deadline exceeded\",\n\t\t},\n\t\t{\n\t\t\tname: \"create data path fail\",\n\t\t\tctx: t.Context(),\n\t\t\tkubeClientObj: []runtime.Object{ddInProgress},\n\t\t\tdataPathMgr: datapath.NewManager(0),\n\t\t\texpectedErr: \"error to create data path: Concurrent number exceeds\",\n\t\t},\n\t\t{\n\t\t\tname: \"init data path fail\",\n\t\t\tctx: t.Context(),\n\t\t\tkubeClientObj: []runtime.Object{ddInProgress},\n\t\t\tinitErr: errors.New(\"fake-init-error\"),\n\t\t\texpectedErr: \"error to initialize data path: fake-init-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"start data path fail\",\n\t\t\tctx: t.Context(),\n\t\t\tkubeClientObj: []runtime.Object{ddInProgress},\n\t\t\tstartErr: errors.New(\"fake-start-error\"),\n\t\t\texpectedErr: \"error starting data path restore: fake-start-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"data path timeout\",\n\t\t\tctx: ctxTimeout,\n\t\t\tkubeClientObj: []runtime.Object{ddInProgress},\n\t\t\tdataPathStarted: true,\n\t\t\texpectedEventMsg: fmt.Sprintf(\"Data path for %s stopped\", dataDownloadName),\n\t\t\texpectedErr: \"timed out waiting for fs restore to complete\",\n\t\t},\n\t\t{\n\t\t\tname: \"data path returns error\",\n\t\t\tctx: t.Context(),\n\t\t\tkubeClientObj: []runtime.Object{ddInProgress},\n\t\t\tdataPathStarted: true,\n\t\t\tresult: &dataPathResult{\n\t\t\t\terr: errors.New(\"fake-data-path-error\"),\n\t\t\t},\n\t\t\texpectedEventMsg: fmt.Sprintf(\"Data path for %s stopped\", dataDownloadName),\n\t\t\texpectedErr: \"fake-data-path-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tctx: t.Context(),\n\t\t\tkubeClientObj: []runtime.Object{ddInProgress},\n\t\t\tdataPathStarted: true,\n\t\t\tresult: &dataPathResult{\n\t\t\t\tresult: \"fake-succeed-result\",\n\t\t\t},\n\t\t\texpectedEventMsg: fmt.Sprintf(\"Data path for %s stopped\", dataDownloadName),\n\t\t},\n\t}\n\n\tscheme := runtime.NewScheme()\n\tvelerov2alpha1api.AddToScheme(scheme)\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeClientBuilder := clientFake.NewClientBuilder()\n\t\t\tfakeClientBuilder = fakeClientBuilder.WithScheme(scheme)\n\n\t\t\tfakeClient := fakeClientBuilder.WithRuntimeObjects(test.kubeClientObj...).Build()\n\n\t\t\tbt := &backupMsTestHelper{}\n\n\t\t\trs := &RestoreMicroService{\n\t\t\t\tnamespace: velerov1api.DefaultNamespace,\n\t\t\t\tdataDownloadName: dataDownloadName,\n\t\t\t\tctx: t.Context(),\n\t\t\t\tclient: fakeClient,\n\t\t\t\tdataPathMgr: datapath.NewManager(1),\n\t\t\t\teventRecorder: bt,\n\t\t\t\tresultSignal: make(chan dataPathResult),\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tif test.ctx != nil {\n\t\t\t\trs.ctx = test.ctx\n\t\t\t}\n\n\t\t\tif test.dataPathMgr != nil {\n\t\t\t\trs.dataPathMgr = test.dataPathMgr\n\t\t\t}\n\n\t\t\tdatapath.FSBRCreator = func(string, string, kbclient.Client, string, datapath.Callbacks, logrus.FieldLogger) datapath.AsyncBR {\n\t\t\t\tfsBR := datapathmockes.NewAsyncBR(t)\n\t\t\t\tif test.initErr != nil {\n\t\t\t\t\tfsBR.On(\"Init\", mock.Anything, mock.Anything).Return(test.initErr)\n\t\t\t\t}\n\n\t\t\t\tif test.startErr != nil {\n\t\t\t\t\tfsBR.On(\"Init\", mock.Anything, mock.Anything).Return(nil)\n\t\t\t\t\tfsBR.On(\"StartRestore\", mock.Anything, mock.Anything, mock.Anything).Return(test.startErr)\n\t\t\t\t}\n\n\t\t\t\tif test.dataPathStarted {\n\t\t\t\t\tfsBR.On(\"Init\", mock.Anything, mock.Anything).Return(nil)\n\t\t\t\t\tfsBR.On(\"StartRestore\", mock.Anything, mock.Anything, mock.Anything).Return(nil)\n\t\t\t\t}\n\n\t\t\t\treturn fsBR\n\t\t\t}\n\n\t\t\tif test.result != nil {\n\t\t\t\tgo func() {\n\t\t\t\t\ttime.Sleep(time.Millisecond * 500)\n\t\t\t\t\trs.resultSignal <- *test.result\n\t\t\t\t}()\n\t\t\t}\n\n\t\t\tresult, err := rs.RunCancelableDataPath(test.ctx)\n\n\t\t\tif test.expectedErr != \"\" {\n\t\t\t\trequire.EqualError(t, err, test.expectedErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, test.result.result, result)\n\t\t\t}\n\n\t\t\tif test.expectedEventMsg != \"\" {\n\t\t\t\tassert.True(t, bt.withEvent)\n\t\t\t\tassert.Equal(t, test.expectedEventMsg, bt.EventMessage())\n\t\t\t}\n\t\t})\n\t}\n\n\tcancel()\n}\n" }, { "path": "pkg/datamover/util.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage datamover\n\nimport \"fmt\"\n\nfunc GetUploaderType(dataMover string) string {\n\tif dataMover == \"\" || dataMover == \"velero\" {\n\t\treturn \"kopia\"\n\t} else {\n\t\treturn dataMover\n\t}\n}\n\nfunc IsBuiltInUploader(dataMover string) bool {\n\treturn dataMover == \"\" || dataMover == \"velero\"\n}\n\nfunc GetRealSource(sourceNamespace string, pvcName string) string {\n\treturn fmt.Sprintf(\"%s/%s\", sourceNamespace, pvcName)\n}\n" }, { "path": "pkg/datamover/util_test.go", "content": "package datamover\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestIsBuiltInUploader(t *testing.T) {\n\ttestcases := []struct {\n\t\tname string\n\t\tdataMover string\n\t\twant bool\n\t}{\n\t\t{\n\t\t\tname: \"empty dataMover is builtin\",\n\t\t\tdataMover: \"\",\n\t\t\twant: true,\n\t\t},\n\t\t{\n\t\t\tname: \"velero dataMover is builtin\",\n\t\t\tdataMover: \"velero\",\n\t\t\twant: true,\n\t\t},\n\t\t{\n\t\t\tname: \"kopia dataMover is not builtin\",\n\t\t\tdataMover: \"kopia\",\n\t\t\twant: false,\n\t\t},\n\t}\n\tfor _, tc := range testcases {\n\t\tt.Run(tc.name, func(tt *testing.T) {\n\t\t\tassert.Equal(tt, tc.want, IsBuiltInUploader(tc.dataMover))\n\t\t})\n\t}\n}\n\nfunc TestGetUploaderType(t *testing.T) {\n\ttestcases := []struct {\n\t\tname string\n\t\tinput string\n\t\twant string\n\t}{\n\t\t{\n\t\t\tname: \"empty dataMover is kopia\",\n\t\t\tinput: \"\",\n\t\t\twant: \"kopia\",\n\t\t},\n\t\t{\n\t\t\tname: \"velero dataMover is kopia\",\n\t\t\tinput: \"velero\",\n\t\t\twant: \"kopia\",\n\t\t},\n\t\t{\n\t\t\tname: \"kopia dataMover is kopia\",\n\t\t\tinput: \"kopia\",\n\t\t\twant: \"kopia\",\n\t\t},\n\t\t{\n\t\t\tname: \"restic dataMover is restic\",\n\t\t\tinput: \"restic\",\n\t\t\twant: \"restic\",\n\t\t},\n\t}\n\tfor _, tc := range testcases {\n\t\tt.Run(tc.name, func(tt *testing.T) {\n\t\t\tassert.Equal(tt, tc.want, GetUploaderType(tc.input))\n\t\t})\n\t}\n}\n" }, { "path": "pkg/datapath/error.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage datapath\n\n// DataPathError represents an error that occurred during a backup or restore operation\ntype DataPathError struct {\n\tsnapshotID string\n\terr error\n}\n\n// Error implements error.\nfunc (e DataPathError) Error() string {\n\treturn e.err.Error()\n}\n\n// GetSnapshotID returns the snapshot ID for the error.\nfunc (e DataPathError) GetSnapshotID() string {\n\treturn e.snapshotID\n}\n" }, { "path": "pkg/datapath/error_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage datapath\n\nimport (\n\t\"errors\"\n\t\"testing\"\n)\n\nfunc TestGetSnapshotID(t *testing.T) {\n\t// Create a DataPathError instance for testing\n\terr := DataPathError{snapshotID: \"123\", err: errors.New(\"example error\")}\n\t// Call the GetSnapshotID method to retrieve the snapshot ID\n\tsnapshotID := err.GetSnapshotID()\n\t// Check if the retrieved snapshot ID matches the expected value\n\tif snapshotID != \"123\" {\n\t\tt.Errorf(\"GetSnapshotID() returned unexpected snapshot ID: got %s, want %s\", snapshotID, \"123\")\n\t}\n}\n\nfunc TestError(t *testing.T) {\n\t// Create a DataPathError instance for testing\n\terr := DataPathError{snapshotID: \"123\", err: errors.New(\"example error\")}\n\t// Call the Error method to retrieve the error message\n\terrMsg := err.Error()\n\t// Check if the retrieved error message matches the expected value\n\texpectedErrMsg := \"example error\"\n\tif errMsg != expectedErrMsg {\n\t\tt.Errorf(\"Error() returned unexpected error message: got %s, want %s\", errMsg, expectedErrMsg)\n\t}\n}\n" }, { "path": "pkg/datapath/file_system.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage datapath\n\nimport (\n\t\"context\"\n\t\"sync\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\trepokey \"github.com/vmware-tanzu/velero/pkg/repository/keys\"\n\trepoProvider \"github.com/vmware-tanzu/velero/pkg/repository/provider\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader/provider\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n)\n\n// FSBRInitParam define the input param for FSBR init\ntype FSBRInitParam struct {\n\tBSLName string\n\tSourceNamespace string\n\tUploaderType string\n\tRepositoryType string\n\tRepoIdentifier string\n\tRepositoryEnsurer *repository.Ensurer\n\tCredentialGetter *credentials.CredentialGetter\n\tFilesystem filesystem.Interface\n\tCacheDir string\n}\n\n// FSBRStartParam define the input param for FSBR start\ntype FSBRStartParam struct {\n\tRealSource string\n\tParentSnapshot string\n\tForceFull bool\n\tTags map[string]string\n}\n\ntype fileSystemBR struct {\n\tctx context.Context\n\tcancel context.CancelFunc\n\tbackupRepo *velerov1api.BackupRepository\n\tuploaderProv provider.Provider\n\tlog logrus.FieldLogger\n\tclient client.Client\n\tbackupLocation *velerov1api.BackupStorageLocation\n\tnamespace string\n\tinitialized bool\n\tcallbacks Callbacks\n\tjobName string\n\trequestorType string\n\twgDataPath sync.WaitGroup\n\tdataPathLock sync.Mutex\n}\n\nfunc newFileSystemBR(jobName string, requestorType string, client client.Client, namespace string, callbacks Callbacks, log logrus.FieldLogger) AsyncBR {\n\tfs := &fileSystemBR{\n\t\tjobName: jobName,\n\t\trequestorType: requestorType,\n\t\tclient: client,\n\t\tnamespace: namespace,\n\t\tcallbacks: callbacks,\n\t\twgDataPath: sync.WaitGroup{},\n\t\tlog: log,\n\t}\n\n\treturn fs\n}\n\nfunc (fs *fileSystemBR) Init(ctx context.Context, param any) error {\n\tinitParam := param.(*FSBRInitParam)\n\n\tvar err error\n\tdefer func() {\n\t\tif err != nil {\n\t\t\tfs.Close(ctx)\n\t\t}\n\t}()\n\n\tfs.ctx, fs.cancel = context.WithCancel(ctx)\n\n\tbackupLocation := &velerov1api.BackupStorageLocation{}\n\tif err = fs.client.Get(ctx, client.ObjectKey{\n\t\tNamespace: fs.namespace,\n\t\tName: initParam.BSLName,\n\t}, backupLocation); err != nil {\n\t\treturn errors.Wrapf(err, \"error getting backup storage location %s\", initParam.BSLName)\n\t}\n\n\tfs.backupLocation = backupLocation\n\n\tfs.backupRepo, err = initParam.RepositoryEnsurer.EnsureRepo(ctx, fs.namespace, initParam.SourceNamespace, initParam.BSLName, initParam.RepositoryType)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to ensure backup repository %s-%s-%s\", initParam.BSLName, initParam.SourceNamespace, initParam.RepositoryType)\n\t}\n\n\terr = fs.boostRepoConnect(ctx, initParam.RepositoryType, initParam.CredentialGetter, initParam.CacheDir)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to boost backup repository connection %s-%s-%s\", initParam.BSLName, initParam.SourceNamespace, initParam.RepositoryType)\n\t}\n\n\tfs.uploaderProv, err = provider.NewUploaderProvider(ctx, fs.client, initParam.UploaderType, fs.requestorType, initParam.RepoIdentifier,\n\t\tfs.backupLocation, fs.backupRepo, initParam.CredentialGetter, repokey.RepoKeySelector(), fs.log)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error creating uploader %s\", initParam.UploaderType)\n\t}\n\n\tfs.initialized = true\n\n\tfs.log.WithFields(\n\t\tlogrus.Fields{\n\t\t\t\"jobName\": fs.jobName,\n\t\t\t\"bsl\": initParam.BSLName,\n\t\t\t\"source namespace\": initParam.SourceNamespace,\n\t\t\t\"uploader\": initParam.UploaderType,\n\t\t\t\"repository\": initParam.RepositoryType,\n\t\t}).Info(\"FileSystemBR is initialized\")\n\n\treturn nil\n}\n\nfunc (fs *fileSystemBR) Close(ctx context.Context) {\n\tif fs.cancel != nil {\n\t\tfs.cancel()\n\t}\n\n\tfs.log.WithField(\"user\", fs.jobName).Info(\"Closing FileSystemBR\")\n\n\tfs.wgDataPath.Wait()\n\n\tfs.close(ctx)\n\n\tfs.log.WithField(\"user\", fs.jobName).Info(\"FileSystemBR is closed\")\n}\n\nfunc (fs *fileSystemBR) close(ctx context.Context) {\n\tfs.dataPathLock.Lock()\n\tdefer fs.dataPathLock.Unlock()\n\n\tif fs.uploaderProv != nil {\n\t\tif err := fs.uploaderProv.Close(ctx); err != nil {\n\t\t\tfs.log.Errorf(\"failed to close uploader provider with error %v\", err)\n\t\t}\n\n\t\tfs.uploaderProv = nil\n\t}\n}\n\nfunc (fs *fileSystemBR) StartBackup(source AccessPoint, uploaderConfig map[string]string, param any) error {\n\tif !fs.initialized {\n\t\treturn errors.New(\"file system data path is not initialized\")\n\t}\n\n\tfs.wgDataPath.Add(1)\n\n\tbackupParam := param.(*FSBRStartParam)\n\n\tgo func() {\n\t\tfs.log.Info(\"Start data path backup\")\n\n\t\tdefer func() {\n\t\t\tfs.close(context.Background())\n\t\t\tfs.wgDataPath.Done()\n\t\t}()\n\n\t\tsnapshotID, emptySnapshot, totalBytes, incrementalBytes, err := fs.uploaderProv.RunBackup(fs.ctx, source.ByPath, backupParam.RealSource, backupParam.Tags, backupParam.ForceFull,\n\t\t\tbackupParam.ParentSnapshot, source.VolMode, uploaderConfig, fs)\n\n\t\tif err == provider.ErrorCanceled {\n\t\t\tfs.callbacks.OnCancelled(context.Background(), fs.namespace, fs.jobName)\n\t\t} else if err != nil {\n\t\t\tdataPathErr := DataPathError{\n\t\t\t\tsnapshotID: snapshotID,\n\t\t\t\terr: err,\n\t\t\t}\n\t\t\tfs.callbacks.OnFailed(context.Background(), fs.namespace, fs.jobName, dataPathErr)\n\t\t} else {\n\t\t\tfs.callbacks.OnCompleted(context.Background(), fs.namespace, fs.jobName, Result{Backup: BackupResult{snapshotID, emptySnapshot, source, totalBytes, incrementalBytes}})\n\t\t}\n\t}()\n\n\treturn nil\n}\n\nfunc (fs *fileSystemBR) StartRestore(snapshotID string, target AccessPoint, uploaderConfigs map[string]string) error {\n\tif !fs.initialized {\n\t\treturn errors.New(\"file system data path is not initialized\")\n\t}\n\n\tfs.wgDataPath.Add(1)\n\n\tgo func() {\n\t\tfs.log.Info(\"Start data path restore\")\n\n\t\tdefer func() {\n\t\t\tfs.close(context.Background())\n\t\t\tfs.wgDataPath.Done()\n\t\t}()\n\n\t\ttotalBytes, err := fs.uploaderProv.RunRestore(fs.ctx, snapshotID, target.ByPath, target.VolMode, uploaderConfigs, fs)\n\n\t\tif err == provider.ErrorCanceled {\n\t\t\tfs.callbacks.OnCancelled(context.Background(), fs.namespace, fs.jobName)\n\t\t} else if err != nil {\n\t\t\tdataPathErr := DataPathError{\n\t\t\t\tsnapshotID: snapshotID,\n\t\t\t\terr: err,\n\t\t\t}\n\t\t\tfs.callbacks.OnFailed(context.Background(), fs.namespace, fs.jobName, dataPathErr)\n\t\t} else {\n\t\t\tfs.callbacks.OnCompleted(context.Background(), fs.namespace, fs.jobName, Result{Restore: RestoreResult{Target: target, TotalBytes: totalBytes}})\n\t\t}\n\t}()\n\n\treturn nil\n}\n\n// UpdateProgress which implement ProgressUpdater interface to update progress status\nfunc (fs *fileSystemBR) UpdateProgress(p *uploader.Progress) {\n\tif fs.callbacks.OnProgress != nil {\n\t\tfs.callbacks.OnProgress(context.Background(), fs.namespace, fs.jobName, &uploader.Progress{TotalBytes: p.TotalBytes, BytesDone: p.BytesDone})\n\t}\n}\n\nfunc (fs *fileSystemBR) Cancel() {\n\tfs.cancel()\n\tfs.log.WithField(\"user\", fs.jobName).Info(\"FileSystemBR is canceled\")\n}\n\nfunc (fs *fileSystemBR) boostRepoConnect(ctx context.Context, repositoryType string, credentialGetter *credentials.CredentialGetter, cacheDir string) error {\n\tif repositoryType == velerov1api.BackupRepositoryTypeKopia {\n\t\tif err := repoProvider.NewUnifiedRepoProvider(*credentialGetter, repositoryType, fs.log).BoostRepoConnect(ctx, repoProvider.RepoParam{BackupLocation: fs.backupLocation, BackupRepo: fs.backupRepo, CacheDir: cacheDir}); err != nil {\n\t\t\treturn err\n\t\t}\n\t} else {\n\t\tif err := repoProvider.NewResticRepositoryProvider(*credentialGetter, filesystem.NewFileSystem(), fs.log).BoostRepoConnect(ctx, repoProvider.RepoParam{BackupLocation: fs.backupLocation, BackupRepo: fs.backupRepo}); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n" }, { "path": "pkg/datapath/file_system_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage datapath\n\nimport (\n\t\"context\"\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader/provider\"\n\tproviderMock \"github.com/vmware-tanzu/velero/pkg/uploader/provider/mocks\"\n)\n\nfunc TestAsyncBackup(t *testing.T) {\n\tvar asyncErr error\n\tvar asyncResult Result\n\tfinish := make(chan struct{})\n\tvar failErr = errors.New(\"fake-fail-error\")\n\ttests := []struct {\n\t\tname string\n\t\tuploaderProv provider.Provider\n\t\tcallbacks Callbacks\n\t\terr error\n\t\tresult Result\n\t\tpath string\n\t}{\n\t\t{\n\t\t\tname: \"async backup fail\",\n\t\t\tcallbacks: Callbacks{\n\t\t\t\tOnCompleted: nil,\n\t\t\t\tOnCancelled: nil,\n\t\t\t\tOnFailed: func(ctx context.Context, namespace string, job string, err error) {\n\t\t\t\t\tasyncErr = failErr\n\t\t\t\t\tasyncResult = Result{}\n\t\t\t\t\tfinish <- struct{}{}\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: failErr,\n\t\t},\n\t\t{\n\t\t\tname: \"async backup cancel\",\n\t\t\tcallbacks: Callbacks{\n\t\t\t\tOnCompleted: nil,\n\t\t\t\tOnFailed: nil,\n\t\t\t\tOnCancelled: func(ctx context.Context, namespace string, job string) {\n\t\t\t\t\tasyncErr = provider.ErrorCanceled\n\t\t\t\t\tasyncResult = Result{}\n\t\t\t\t\tfinish <- struct{}{}\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: provider.ErrorCanceled,\n\t\t},\n\t\t{\n\t\t\tname: \"async backup complete\",\n\t\t\tcallbacks: Callbacks{\n\t\t\t\tOnFailed: nil,\n\t\t\t\tOnCancelled: nil,\n\t\t\t\tOnCompleted: func(ctx context.Context, namespace string, job string, result Result) {\n\t\t\t\t\tasyncResult = result\n\t\t\t\t\tasyncErr = nil\n\t\t\t\t\tfinish <- struct{}{}\n\t\t\t\t},\n\t\t\t},\n\t\t\tresult: Result{\n\t\t\t\tBackup: BackupResult{\n\t\t\t\t\tSnapshotID: \"fake-snapshot\",\n\t\t\t\t\tEmptySnapshot: false,\n\t\t\t\t\tSource: AccessPoint{ByPath: \"fake-path\"},\n\t\t\t\t\tTotalBytes: 1000,\n\t\t\t\t},\n\t\t\t},\n\t\t\tpath: \"fake-path\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfs := newFileSystemBR(\"job-1\", \"test\", nil, \"velero\", Callbacks{}, velerotest.NewLogger()).(*fileSystemBR)\n\t\t\tmockProvider := providerMock.NewProvider(t)\n\t\t\tmockProvider.On(\"RunBackup\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(test.result.Backup.SnapshotID, test.result.Backup.EmptySnapshot, test.result.Backup.TotalBytes, test.result.Backup.IncrementalBytes, test.err)\n\t\t\tmockProvider.On(\"Close\", mock.Anything).Return(nil)\n\t\t\tfs.uploaderProv = mockProvider\n\t\t\tfs.initialized = true\n\t\t\tfs.callbacks = test.callbacks\n\n\t\t\terr := fs.StartBackup(AccessPoint{ByPath: test.path}, map[string]string{}, &FSBRStartParam{})\n\t\t\trequire.NoError(t, err)\n\n\t\t\t<-finish\n\n\t\t\t// Ensure the goroutine finishes so deferred fs.close executes, satisfying mock expectations.\n\t\t\tfs.wgDataPath.Wait()\n\n\t\t\tassert.Equal(t, test.err, asyncErr)\n\t\t\tassert.Equal(t, test.result, asyncResult)\n\t\t})\n\t}\n\n\tclose(finish)\n}\n\nfunc TestAsyncRestore(t *testing.T) {\n\tvar asyncErr error\n\tvar asyncResult Result\n\tfinish := make(chan struct{})\n\tvar failErr = errors.New(\"fake-fail-error\")\n\ttests := []struct {\n\t\tname string\n\t\tuploaderProv provider.Provider\n\t\tcallbacks Callbacks\n\t\terr error\n\t\tresult Result\n\t\tpath string\n\t\tsnapshot string\n\t}{\n\t\t{\n\t\t\tname: \"async restore fail\",\n\t\t\tcallbacks: Callbacks{\n\t\t\t\tOnCompleted: nil,\n\t\t\t\tOnCancelled: nil,\n\t\t\t\tOnFailed: func(ctx context.Context, namespace string, job string, err error) {\n\t\t\t\t\tasyncErr = failErr\n\t\t\t\t\tasyncResult = Result{}\n\t\t\t\t\tfinish <- struct{}{}\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: failErr,\n\t\t},\n\t\t{\n\t\t\tname: \"async restore cancel\",\n\t\t\tcallbacks: Callbacks{\n\t\t\t\tOnCompleted: nil,\n\t\t\t\tOnFailed: nil,\n\t\t\t\tOnCancelled: func(ctx context.Context, namespace string, job string) {\n\t\t\t\t\tasyncErr = provider.ErrorCanceled\n\t\t\t\t\tasyncResult = Result{}\n\t\t\t\t\tfinish <- struct{}{}\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: provider.ErrorCanceled,\n\t\t},\n\t\t{\n\t\t\tname: \"async restore complete\",\n\t\t\tcallbacks: Callbacks{\n\t\t\t\tOnFailed: nil,\n\t\t\t\tOnCancelled: nil,\n\t\t\t\tOnCompleted: func(ctx context.Context, namespace string, job string, result Result) {\n\t\t\t\t\tasyncResult = result\n\t\t\t\t\tasyncErr = nil\n\t\t\t\t\tfinish <- struct{}{}\n\t\t\t\t},\n\t\t\t},\n\t\t\tresult: Result{\n\t\t\t\tRestore: RestoreResult{\n\t\t\t\t\tTarget: AccessPoint{ByPath: \"fake-path\"},\n\t\t\t\t\tTotalBytes: 1000,\n\t\t\t\t},\n\t\t\t},\n\t\t\tpath: \"fake-path\",\n\t\t\tsnapshot: \"fake-snapshot\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfs := newFileSystemBR(\"job-1\", \"test\", nil, \"velero\", Callbacks{}, velerotest.NewLogger()).(*fileSystemBR)\n\t\t\tmockProvider := providerMock.NewProvider(t)\n\t\t\tmockProvider.On(\"RunRestore\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(test.result.Restore.TotalBytes, test.err)\n\t\t\tmockProvider.On(\"Close\", mock.Anything).Return(nil)\n\t\t\tfs.uploaderProv = mockProvider\n\t\t\tfs.initialized = true\n\t\t\tfs.callbacks = test.callbacks\n\n\t\t\terr := fs.StartRestore(test.snapshot, AccessPoint{ByPath: test.path}, map[string]string{})\n\t\t\trequire.NoError(t, err)\n\n\t\t\t<-finish\n\n\t\t\t// Ensure the goroutine finishes so deferred fs.close executes, satisfying mock expectations.\n\t\t\tfs.wgDataPath.Wait()\n\n\t\t\tassert.Equal(t, asyncErr, test.err)\n\t\t\tassert.Equal(t, asyncResult, test.result)\n\t\t})\n\t}\n\n\tclose(finish)\n}\n" }, { "path": "pkg/datapath/manager.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage datapath\n\nimport (\n\t\"context\"\n\t\"sync\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/manager\"\n)\n\nvar ConcurrentLimitExceed error = errors.New(\"Concurrent number exceeds\")\nvar FSBRCreator = newFileSystemBR\nvar MicroServiceBRWatcherCreator = newMicroServiceBRWatcher\n\ntype Manager struct {\n\tcocurrentNum int\n\ttrackerLock sync.Mutex\n\ttracker map[string]AsyncBR\n}\n\n// NewManager creates the data path manager to manage concurrent data path instances\nfunc NewManager(cocurrentNum int) *Manager {\n\treturn &Manager{\n\t\tcocurrentNum: cocurrentNum,\n\t\ttracker: map[string]AsyncBR{},\n\t}\n}\n\n// CreateFileSystemBR creates a new file system backup/restore data path instance\nfunc (m *Manager) CreateFileSystemBR(jobName string, requestorType string, ctx context.Context, client client.Client, namespace string, callbacks Callbacks, log logrus.FieldLogger) (AsyncBR, error) {\n\tm.trackerLock.Lock()\n\tdefer m.trackerLock.Unlock()\n\n\tif len(m.tracker) >= m.cocurrentNum {\n\t\treturn nil, ConcurrentLimitExceed\n\t}\n\n\tm.tracker[jobName] = FSBRCreator(jobName, requestorType, client, namespace, callbacks, log)\n\n\treturn m.tracker[jobName], nil\n}\n\n// CreateMicroServiceBRWatcher creates a new micro service watcher instance\nfunc (m *Manager) CreateMicroServiceBRWatcher(ctx context.Context, client client.Client, kubeClient kubernetes.Interface, mgr manager.Manager, taskType string,\n\ttaskName string, namespace string, podName string, containerName string, associatedObject string, callbacks Callbacks, resume bool, log logrus.FieldLogger) (AsyncBR, error) {\n\tm.trackerLock.Lock()\n\tdefer m.trackerLock.Unlock()\n\n\tif !resume {\n\t\tif len(m.tracker) >= m.cocurrentNum {\n\t\t\treturn nil, ConcurrentLimitExceed\n\t\t}\n\t}\n\n\tm.tracker[taskName] = MicroServiceBRWatcherCreator(client, kubeClient, mgr, taskType, taskName, namespace, podName, containerName, associatedObject, callbacks, log)\n\n\treturn m.tracker[taskName], nil\n}\n\n// RemoveAsyncBR removes a file system backup/restore data path instance\nfunc (m *Manager) RemoveAsyncBR(jobName string) {\n\tm.trackerLock.Lock()\n\tdefer m.trackerLock.Unlock()\n\n\tdelete(m.tracker, jobName)\n}\n\n// GetAsyncBR returns the file system backup/restore data path instance for the specified job name\nfunc (m *Manager) GetAsyncBR(jobName string) AsyncBR {\n\tm.trackerLock.Lock()\n\tdefer m.trackerLock.Unlock()\n\n\tif async, exist := m.tracker[jobName]; exist {\n\t\treturn async\n\t} else {\n\t\treturn nil\n\t}\n}\n" }, { "path": "pkg/datapath/manager_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage datapath\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestCreateFileSystemBR(t *testing.T) {\n\tm := NewManager(2)\n\n\tasync_job_1, err := m.CreateFileSystemBR(\"job-1\", \"test\", t.Context(), nil, \"velero\", Callbacks{}, nil)\n\trequire.NoError(t, err)\n\n\t_, err = m.CreateFileSystemBR(\"job-2\", \"test\", t.Context(), nil, \"velero\", Callbacks{}, nil)\n\trequire.NoError(t, err)\n\n\t_, err = m.CreateFileSystemBR(\"job-3\", \"test\", t.Context(), nil, \"velero\", Callbacks{}, nil)\n\tassert.Equal(t, ConcurrentLimitExceed, err)\n\n\tret := m.GetAsyncBR(\"job-0\")\n\tassert.Nil(t, ret)\n\n\tret = m.GetAsyncBR(\"job-1\")\n\tassert.Equal(t, async_job_1, ret)\n\n\tm.RemoveAsyncBR(\"job-0\")\n\tassert.Len(t, m.tracker, 2)\n\n\tm.RemoveAsyncBR(\"job-1\")\n\tassert.Len(t, m.tracker, 1)\n\n\tret = m.GetAsyncBR(\"job-1\")\n\tassert.Nil(t, ret)\n}\n\nfunc TestCreateMicroServiceBRWatcher(t *testing.T) {\n\tm := NewManager(2)\n\n\tasync_job_1, err := m.CreateMicroServiceBRWatcher(t.Context(), nil, nil, nil, \"test\", \"job-1\", \"velero\", \"pod-1\", \"container\", \"du-1\", Callbacks{}, false, nil)\n\trequire.NoError(t, err)\n\n\t_, err = m.CreateMicroServiceBRWatcher(t.Context(), nil, nil, nil, \"test\", \"job-2\", \"velero\", \"pod-2\", \"container\", \"du-2\", Callbacks{}, false, nil)\n\trequire.NoError(t, err)\n\n\t_, err = m.CreateMicroServiceBRWatcher(t.Context(), nil, nil, nil, \"test\", \"job-3\", \"velero\", \"pod-3\", \"container\", \"du-3\", Callbacks{}, false, nil)\n\tassert.Equal(t, ConcurrentLimitExceed, err)\n\n\tasync_job_4, err := m.CreateMicroServiceBRWatcher(t.Context(), nil, nil, nil, \"test\", \"job-4\", \"velero\", \"pod-4\", \"container\", \"du-4\", Callbacks{}, true, nil)\n\trequire.NoError(t, err)\n\n\tret := m.GetAsyncBR(\"job-0\")\n\tassert.Nil(t, ret)\n\n\tret = m.GetAsyncBR(\"job-1\")\n\tassert.Equal(t, async_job_1, ret)\n\n\tret = m.GetAsyncBR(\"job-4\")\n\tassert.Equal(t, async_job_4, ret)\n\n\tm.RemoveAsyncBR(\"job-0\")\n\tassert.Len(t, m.tracker, 3)\n\n\tm.RemoveAsyncBR(\"job-1\")\n\tassert.Len(t, m.tracker, 2)\n\n\tret = m.GetAsyncBR(\"job-1\")\n\tassert.Nil(t, ret)\n}\n" }, { "path": "pkg/datapath/micro_service_watcher.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage datapath\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"os\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"k8s.io/client-go/tools/cache\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\n\tctrlcache \"sigs.k8s.io/controller-runtime/pkg/cache\"\n\t\"sigs.k8s.io/controller-runtime/pkg/manager\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n)\n\nconst (\n\tTaskTypeBackup = \"backup\"\n\tTaskTypeRestore = \"restore\"\n\n\tErrCancelled = \"data path is canceled\"\n\n\tEventReasonStarted = \"Data-Path-Started\"\n\tEventReasonCompleted = \"Data-Path-Completed\"\n\tEventReasonFailed = \"Data-Path-Failed\"\n\tEventReasonCancelled = \"Data-Path-Canceled\"\n\tEventReasonProgress = \"Data-Path-Progress\"\n\tEventReasonCancelling = \"Data-Path-Canceling\"\n\tEventReasonStopped = \"Data-Path-Stopped\"\n)\n\ntype microServiceBRWatcher struct {\n\tctx context.Context\n\tcancel context.CancelFunc\n\tlog logrus.FieldLogger\n\tclient client.Client\n\tkubeClient kubernetes.Interface\n\tmgr manager.Manager\n\tnamespace string\n\tcallbacks Callbacks\n\ttaskName string\n\ttaskType string\n\tthisPod string\n\tthisContainer string\n\tassociatedObject string\n\teventCh chan *corev1api.Event\n\tpodCh chan *corev1api.Pod\n\tstartedFromEvent bool\n\tterminatedFromEvent bool\n\twgWatcher sync.WaitGroup\n\teventInformer ctrlcache.Informer\n\tpodInformer ctrlcache.Informer\n\teventHandler cache.ResourceEventHandlerRegistration\n\tpodHandler cache.ResourceEventHandlerRegistration\n\twatcherLock sync.Mutex\n}\n\nfunc newMicroServiceBRWatcher(client client.Client, kubeClient kubernetes.Interface, mgr manager.Manager, taskType string, taskName string, namespace string,\n\tpodName string, containerName string, associatedObject string, callbacks Callbacks, log logrus.FieldLogger) AsyncBR {\n\tms := µServiceBRWatcher{\n\t\tmgr: mgr,\n\t\tclient: client,\n\t\tkubeClient: kubeClient,\n\t\tnamespace: namespace,\n\t\tcallbacks: callbacks,\n\t\ttaskType: taskType,\n\t\ttaskName: taskName,\n\t\tthisPod: podName,\n\t\tthisContainer: containerName,\n\t\tassociatedObject: associatedObject,\n\t\teventCh: make(chan *corev1api.Event, 10),\n\t\tpodCh: make(chan *corev1api.Pod, 2),\n\t\twgWatcher: sync.WaitGroup{},\n\t\tlog: log,\n\t}\n\n\treturn ms\n}\n\nfunc (ms *microServiceBRWatcher) Init(ctx context.Context, param any) error {\n\teventInformer, err := ms.mgr.GetCache().GetInformer(ctx, &corev1api.Event{})\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error getting event informer\")\n\t}\n\n\tpodInformer, err := ms.mgr.GetCache().GetInformer(ctx, &corev1api.Pod{})\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error getting pod informer\")\n\t}\n\n\teventHandler, err := eventInformer.AddEventHandler(\n\t\tcache.ResourceEventHandlerFuncs{\n\t\t\tAddFunc: func(obj any) {\n\t\t\t\tevt := obj.(*corev1api.Event)\n\t\t\t\tif evt.InvolvedObject.Namespace != ms.namespace || evt.InvolvedObject.Name != ms.associatedObject {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tms.eventCh <- evt\n\t\t\t},\n\t\t\tUpdateFunc: func(_, obj any) {\n\t\t\t\tevt := obj.(*corev1api.Event)\n\t\t\t\tif evt.InvolvedObject.Namespace != ms.namespace || evt.InvolvedObject.Name != ms.associatedObject {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tms.eventCh <- evt\n\t\t\t},\n\t\t},\n\t)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error registering event handler\")\n\t}\n\n\tpodHandler, err := podInformer.AddEventHandler(\n\t\tcache.ResourceEventHandlerFuncs{\n\t\t\tUpdateFunc: func(_, obj any) {\n\t\t\t\tpod := obj.(*corev1api.Pod)\n\t\t\t\tif pod.Namespace != ms.namespace || pod.Name != ms.thisPod {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif pod.Status.Phase == corev1api.PodSucceeded || pod.Status.Phase == corev1api.PodFailed {\n\t\t\t\t\tms.podCh <- pod\n\t\t\t\t}\n\t\t\t},\n\t\t},\n\t)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error registering pod handler\")\n\t}\n\n\tif err := ms.reEnsureThisPod(ctx); err != nil {\n\t\treturn err\n\t}\n\n\tms.eventInformer = eventInformer\n\tms.podInformer = podInformer\n\tms.eventHandler = eventHandler\n\tms.podHandler = podHandler\n\n\tms.ctx, ms.cancel = context.WithCancel(ctx)\n\n\tms.log.WithFields(\n\t\tlogrus.Fields{\n\t\t\t\"taskType\": ms.taskType,\n\t\t\t\"taskName\": ms.taskName,\n\t\t\t\"thisPod\": ms.thisPod,\n\t\t}).Info(\"MicroServiceBR is initialized\")\n\n\treturn nil\n}\n\nfunc (ms *microServiceBRWatcher) Close(ctx context.Context) {\n\tif ms.cancel != nil {\n\t\tms.cancel()\n\t}\n\n\tms.log.WithField(\"taskType\", ms.taskType).WithField(\"taskName\", ms.taskName).Info(\"Closing MicroServiceBR\")\n\n\tms.wgWatcher.Wait()\n\n\tms.close()\n\n\tms.log.WithField(\"taskType\", ms.taskType).WithField(\"taskName\", ms.taskName).Info(\"MicroServiceBR is closed\")\n}\n\nfunc (ms *microServiceBRWatcher) close() {\n\tms.watcherLock.Lock()\n\tdefer ms.watcherLock.Unlock()\n\n\tif ms.eventHandler != nil {\n\t\tif err := ms.eventInformer.RemoveEventHandler(ms.eventHandler); err != nil {\n\t\t\tms.log.WithError(err).Warn(\"Failed to remove event handler\")\n\t\t}\n\n\t\tms.eventHandler = nil\n\t}\n\n\tif ms.podHandler != nil {\n\t\tif err := ms.podInformer.RemoveEventHandler(ms.podHandler); err != nil {\n\t\t\tms.log.WithError(err).Warn(\"Failed to remove pod handler\")\n\t\t}\n\n\t\tms.podHandler = nil\n\t}\n}\n\nfunc (ms *microServiceBRWatcher) StartBackup(source AccessPoint, uploaderConfig map[string]string, param any) error {\n\tms.log.Infof(\"Start watching backup ms for source %v\", source.ByPath)\n\n\tms.startWatch()\n\n\treturn nil\n}\n\nfunc (ms *microServiceBRWatcher) StartRestore(snapshotID string, target AccessPoint, uploaderConfigs map[string]string) error {\n\tms.log.Infof(\"Start watching restore ms to target %s, from snapshot %s\", target.ByPath, snapshotID)\n\n\tms.startWatch()\n\n\treturn nil\n}\n\nfunc (ms *microServiceBRWatcher) reEnsureThisPod(ctx context.Context) error {\n\tthisPod := &corev1api.Pod{}\n\tif err := ms.client.Get(ctx, types.NamespacedName{\n\t\tNamespace: ms.namespace,\n\t\tName: ms.thisPod,\n\t}, thisPod); err != nil {\n\t\treturn errors.Wrapf(err, \"error getting this pod %s\", ms.thisPod)\n\t}\n\n\tif thisPod.Status.Phase == corev1api.PodSucceeded || thisPod.Status.Phase == corev1api.PodFailed {\n\t\tms.podCh <- thisPod\n\t\tms.log.WithField(\"this pod\", ms.thisPod).Infof(\"This pod comes to terminital status %s before watch start\", thisPod.Status.Phase)\n\t}\n\n\treturn nil\n}\n\nvar funcGetPodTerminationMessage = kube.GetPodContainerTerminateMessage\nvar funcRedirectLog = redirectDataMoverLogs\nvar funcGetResultFromMessage = getResultFromMessage\nvar funcGetProgressFromMessage = getProgressFromMessage\n\nvar eventWaitTimeout = time.Minute\n\nfunc (ms *microServiceBRWatcher) startWatch() {\n\tms.wgWatcher.Add(1)\n\n\tgo func() {\n\t\tms.log.Info(\"Start watching data path pod\")\n\n\t\tdefer func() {\n\t\t\tms.close()\n\t\t\tms.wgWatcher.Done()\n\t\t}()\n\n\t\tvar lastPod *corev1api.Pod\n\n\twatchLoop:\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-ms.ctx.Done():\n\t\t\t\tbreak watchLoop\n\t\t\tcase pod := <-ms.podCh:\n\t\t\t\tlastPod = pod\n\t\t\t\tbreak watchLoop\n\t\t\tcase evt := <-ms.eventCh:\n\t\t\t\tms.onEvent(evt)\n\t\t\t}\n\t\t}\n\n\t\tif lastPod == nil {\n\t\t\tms.log.Warn(\"Watch loop is canceled on waiting data path pod\")\n\t\t\treturn\n\t\t}\n\n\tepilogLoop:\n\t\tfor !ms.startedFromEvent || !ms.terminatedFromEvent {\n\t\t\tselect {\n\t\t\tcase <-ms.ctx.Done():\n\t\t\t\tms.log.Warn(\"Watch loop is canceled on waiting final event\")\n\t\t\t\treturn\n\t\t\tcase <-time.After(eventWaitTimeout):\n\t\t\t\tbreak epilogLoop\n\t\t\tcase evt := <-ms.eventCh:\n\t\t\t\tms.onEvent(evt)\n\t\t\t}\n\t\t}\n\n\t\tterminateMessage := funcGetPodTerminationMessage(lastPod, ms.thisContainer)\n\n\t\tlogger := ms.log.WithField(\"data path pod\", lastPod.Name)\n\n\t\tlogger.Infof(\"Finish waiting data path pod, phase %s, message %s\", lastPod.Status.Phase, terminateMessage)\n\n\t\tif !ms.startedFromEvent {\n\t\t\tlogger.Warn(\"VGDP seems not started\")\n\t\t}\n\n\t\tif ms.startedFromEvent && !ms.terminatedFromEvent {\n\t\t\tlogger.Warn(\"VGDP started but termination event is not received\")\n\t\t}\n\n\t\tlogger.Info(\"Recording data path pod logs\")\n\n\t\tif err := funcRedirectLog(ms.ctx, ms.kubeClient, ms.namespace, lastPod.Name, ms.thisContainer, ms.log); err != nil {\n\t\t\tlogger.WithError(err).Warn(\"Failed to collect data mover logs\")\n\t\t}\n\n\t\tlogger.Info(\"Calling callback on data path pod termination\")\n\n\t\tif lastPod.Status.Phase == corev1api.PodSucceeded {\n\t\t\tresult := funcGetResultFromMessage(ms.taskType, terminateMessage, ms.log)\n\t\t\tms.callbacks.OnProgress(ms.ctx, ms.namespace, ms.taskName, getCompletionProgressFromResult(ms.taskType, result))\n\t\t\tms.callbacks.OnCompleted(ms.ctx, ms.namespace, ms.taskName, result)\n\t\t} else {\n\t\t\tif strings.HasSuffix(terminateMessage, ErrCancelled) {\n\t\t\t\tms.callbacks.OnCancelled(ms.ctx, ms.namespace, ms.taskName)\n\t\t\t} else {\n\t\t\t\tms.callbacks.OnFailed(ms.ctx, ms.namespace, ms.taskName, errors.New(terminateMessage))\n\t\t\t}\n\t\t}\n\n\t\tlogger.Info(\"Complete callback on data path pod termination\")\n\t}()\n}\n\nfunc (ms *microServiceBRWatcher) onEvent(evt *corev1api.Event) {\n\tswitch evt.Reason {\n\tcase EventReasonStarted:\n\t\tms.startedFromEvent = true\n\t\tms.log.Infof(\"Received data path start message: %s\", evt.Message)\n\tcase EventReasonProgress:\n\t\tms.callbacks.OnProgress(ms.ctx, ms.namespace, ms.taskName, funcGetProgressFromMessage(evt.Message, ms.log))\n\tcase EventReasonCompleted:\n\t\tms.log.Infof(\"Received data path completed message: %v\", funcGetResultFromMessage(ms.taskType, evt.Message, ms.log))\n\tcase EventReasonCancelled:\n\t\tms.log.Infof(\"Received data path canceled message: %s\", evt.Message)\n\tcase EventReasonFailed:\n\t\tms.log.Infof(\"Received data path failed message: %s\", evt.Message)\n\tcase EventReasonCancelling:\n\t\tms.log.Infof(\"Received data path canceling message: %s\", evt.Message)\n\tcase EventReasonStopped:\n\t\tms.terminatedFromEvent = true\n\t\tms.log.Infof(\"Received data path stop message: %s\", evt.Message)\n\tdefault:\n\t\tms.log.Infof(\"Received event for data path %s, reason: %s, message: %s\", ms.taskName, evt.Reason, evt.Message)\n\t}\n}\n\nfunc getResultFromMessage(taskType string, message string, logger logrus.FieldLogger) Result {\n\tresult := Result{}\n\n\tif taskType == TaskTypeBackup {\n\t\tbackupResult := BackupResult{}\n\t\terr := json.Unmarshal([]byte(message), &backupResult)\n\t\tif err != nil {\n\t\t\tlogger.WithError(err).Errorf(\"Failed to unmarshal result message %s\", message)\n\t\t} else {\n\t\t\tresult.Backup = backupResult\n\t\t}\n\t} else {\n\t\trestoreResult := RestoreResult{}\n\t\terr := json.Unmarshal([]byte(message), &restoreResult)\n\t\tif err != nil {\n\t\t\tlogger.WithError(err).Errorf(\"Failed to unmarshal result message %s\", message)\n\t\t} else {\n\t\t\tresult.Restore = restoreResult\n\t\t}\n\t}\n\n\treturn result\n}\n\nfunc getProgressFromMessage(message string, logger logrus.FieldLogger) *uploader.Progress {\n\tprogress := &uploader.Progress{}\n\terr := json.Unmarshal([]byte(message), progress)\n\tif err != nil {\n\t\tlogger.WithError(err).Debugf(\"Failed to unmarshal progress message %s\", message)\n\t}\n\n\treturn progress\n}\n\nfunc getCompletionProgressFromResult(taskType string, result Result) *uploader.Progress {\n\tprogress := &uploader.Progress{}\n\tif taskType == TaskTypeBackup {\n\t\tprogress.BytesDone = result.Backup.TotalBytes\n\t\tprogress.TotalBytes = result.Backup.TotalBytes\n\t} else {\n\t\tprogress.BytesDone = result.Restore.TotalBytes\n\t\tprogress.TotalBytes = result.Restore.TotalBytes\n\t}\n\n\treturn progress\n}\n\nfunc (ms *microServiceBRWatcher) Cancel() {\n\tms.log.WithField(\"taskType\", ms.taskType).WithField(\"taskName\", ms.taskName).Info(\"MicroServiceBR is canceled\")\n}\n\nvar funcCreateTemp = os.CreateTemp\nvar funcCollectPodLogs = kube.CollectPodLogs\n\nfunc redirectDataMoverLogs(ctx context.Context, kubeClient kubernetes.Interface, namespace string, thisPod string, thisContainer string, logger logrus.FieldLogger) error {\n\tlogger.Infof(\"Starting to collect data mover pod log for %s\", thisPod)\n\n\tlogFile, err := funcCreateTemp(\"\", \"\")\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to create temp file for data mover pod log\")\n\t}\n\n\tdefer logFile.Close()\n\n\tlogFileName := logFile.Name()\n\tlogger.Infof(\"Created log file %s\", logFileName)\n\n\terr = funcCollectPodLogs(ctx, kubeClient.CoreV1(), thisPod, namespace, thisContainer, logFile)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to collect logs to %s for data mover pod %s\", logFileName, thisPod)\n\t}\n\n\tlogFile.Close()\n\n\tlogger.Infof(\"Redirecting to log file %s\", logFileName)\n\n\thookLogger := logger.WithField(logging.LogSourceKey, logFileName)\n\thookLogger.Logln(logging.ListeningLevel, logging.ListeningMessage)\n\n\tlogger.Infof(\"Completed to collect data mover pod log for %s\", thisPod)\n\n\treturn nil\n}\n" }, { "path": "pkg/datapath/micro_service_watcher_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage datapath\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"path\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes\"\n\tkubeclientfake \"k8s.io/client-go/kubernetes/fake\"\n\tcorev1client \"k8s.io/client-go/kubernetes/typed/core/v1\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n)\n\nfunc TestReEnsureThisPod(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tnamespace string\n\t\tthisPod string\n\t\tkubeClientObj []runtime.Object\n\t\texpectChan bool\n\t\texpectErr string\n\t}{\n\t\t{\n\t\t\tname: \"get pod error\",\n\t\t\tthisPod: \"fak-pod-1\",\n\t\t\texpectErr: \"error getting this pod fak-pod-1: pods \\\"fak-pod-1\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"get pod not in terminated state\",\n\t\t\tnamespace: \"velero\",\n\t\t\tthisPod: \"fake-pod-1\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tbuilder.ForPod(\"velero\", \"fake-pod-1\").Phase(corev1api.PodRunning).Result(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"get pod succeed state\",\n\t\t\tnamespace: \"velero\",\n\t\t\tthisPod: \"fake-pod-1\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tbuilder.ForPod(\"velero\", \"fake-pod-1\").Phase(corev1api.PodSucceeded).Result(),\n\t\t\t},\n\t\t\texpectChan: true,\n\t\t},\n\t\t{\n\t\t\tname: \"get pod failed state\",\n\t\t\tnamespace: \"velero\",\n\t\t\tthisPod: \"fake-pod-1\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tbuilder.ForPod(\"velero\", \"fake-pod-1\").Phase(corev1api.PodFailed).Result(),\n\t\t\t},\n\t\t\texpectChan: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tscheme := runtime.NewScheme()\n\t\t\tcorev1api.AddToScheme(scheme)\n\t\t\tfakeClientBuilder := fake.NewClientBuilder()\n\t\t\tfakeClientBuilder = fakeClientBuilder.WithScheme(scheme)\n\n\t\t\tfakeClient := fakeClientBuilder.WithRuntimeObjects(test.kubeClientObj...).Build()\n\n\t\t\tms := µServiceBRWatcher{\n\t\t\t\tnamespace: test.namespace,\n\t\t\t\tthisPod: test.thisPod,\n\t\t\t\tclient: fakeClient,\n\t\t\t\tpodCh: make(chan *corev1api.Pod, 2),\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\terr := ms.reEnsureThisPod(t.Context())\n\t\t\tif test.expectErr != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.expectErr)\n\t\t\t} else {\n\t\t\t\tif test.expectChan {\n\t\t\t\t\tassert.Len(t, ms.podCh, 1)\n\t\t\t\t\tpod := <-ms.podCh\n\t\t\t\t\tassert.Equal(t, pod.Name, test.thisPod)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\ntype startWatchFake struct {\n\tterminationMessage string\n\tredirectErr error\n\tcomplete bool\n\tfailed bool\n\tcanceled bool\n\tprogress int\n}\n\nfunc (sw *startWatchFake) getPodContainerTerminateMessage(pod *corev1api.Pod, container string) string {\n\treturn sw.terminationMessage\n}\n\nfunc (sw *startWatchFake) redirectDataMoverLogs(ctx context.Context, kubeClient kubernetes.Interface, namespace string, thisPod string, thisContainer string, logger logrus.FieldLogger) error {\n\treturn sw.redirectErr\n}\n\nfunc (sw *startWatchFake) getResultFromMessage(_ string, _ string, _ logrus.FieldLogger) Result {\n\treturn Result{}\n}\n\nfunc (sw *startWatchFake) OnCompleted(ctx context.Context, namespace string, task string, result Result) {\n\tsw.complete = true\n}\n\nfunc (sw *startWatchFake) OnFailed(ctx context.Context, namespace string, task string, err error) {\n\tsw.failed = true\n}\n\nfunc (sw *startWatchFake) OnCancelled(ctx context.Context, namespace string, task string) {\n\tsw.canceled = true\n}\n\nfunc (sw *startWatchFake) OnProgress(ctx context.Context, namespace string, task string, progress *uploader.Progress) {\n\tsw.progress++\n}\n\ntype insertEvent struct {\n\tevent *corev1api.Event\n\tafter time.Duration\n\tdelay time.Duration\n}\n\nfunc TestStartWatch(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tnamespace string\n\t\tthisPod string\n\t\tthisContainer string\n\t\tterminationMessage string\n\t\tredirectLogErr error\n\t\tinsertPod *corev1api.Pod\n\t\tinsertEventsBefore []insertEvent\n\t\tinsertEventsAfter []insertEvent\n\t\tctxCancel bool\n\t\texpectStartEvent bool\n\t\texpectTerminateEvent bool\n\t\texpectComplete bool\n\t\texpectCancel bool\n\t\texpectFail bool\n\t\texpectProgress int\n\t}{\n\t\t{\n\t\t\tname: \"exit from ctx\",\n\t\t\tthisPod: \"fak-pod-1\",\n\t\t\tthisContainer: \"fake-container-1\",\n\t\t\tctxCancel: true,\n\t\t},\n\t\t{\n\t\t\tname: \"completed with rantional sequence\",\n\t\t\tthisPod: \"fak-pod-1\",\n\t\t\tthisContainer: \"fake-container-1\",\n\t\t\tinsertPod: builder.ForPod(\"velero\", \"fake-pod-1\").Phase(corev1api.PodSucceeded).Result(),\n\t\t\tinsertEventsBefore: []insertEvent{\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonStarted},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonCompleted},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonStopped},\n\t\t\t\t\tdelay: time.Second,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectStartEvent: true,\n\t\t\texpectTerminateEvent: true,\n\t\t\texpectComplete: true,\n\t\t\texpectProgress: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"completed\",\n\t\t\tthisPod: \"fak-pod-1\",\n\t\t\tthisContainer: \"fake-container-1\",\n\t\t\tinsertPod: builder.ForPod(\"velero\", \"fake-pod-1\").Phase(corev1api.PodSucceeded).Result(),\n\t\t\tinsertEventsBefore: []insertEvent{\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonStarted},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonCompleted},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonStopped},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectStartEvent: true,\n\t\t\texpectTerminateEvent: true,\n\t\t\texpectComplete: true,\n\t\t\texpectProgress: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"completed with redirect error\",\n\t\t\tthisPod: \"fak-pod-1\",\n\t\t\tthisContainer: \"fake-container-1\",\n\t\t\tinsertPod: builder.ForPod(\"velero\", \"fake-pod-1\").Phase(corev1api.PodSucceeded).Result(),\n\t\t\tinsertEventsBefore: []insertEvent{\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonStarted},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonCompleted},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonStopped},\n\t\t\t\t},\n\t\t\t},\n\t\t\tredirectLogErr: errors.New(\"fake-error\"),\n\t\t\texpectStartEvent: true,\n\t\t\texpectTerminateEvent: true,\n\t\t\texpectComplete: true,\n\t\t\texpectProgress: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"complete but terminated event not received in time\",\n\t\t\tthisPod: \"fak-pod-1\",\n\t\t\tthisContainer: \"fake-container-1\",\n\t\t\tinsertPod: builder.ForPod(\"velero\", \"fake-pod-1\").Phase(corev1api.PodSucceeded).Result(),\n\t\t\tinsertEventsBefore: []insertEvent{\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonStarted},\n\t\t\t\t},\n\t\t\t},\n\t\t\tinsertEventsAfter: []insertEvent{\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonStarted},\n\t\t\t\t\tafter: time.Second * 6,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectStartEvent: true,\n\t\t\texpectComplete: true,\n\t\t\texpectProgress: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"complete but terminated event not received immediately\",\n\t\t\tthisPod: \"fak-pod-1\",\n\t\t\tthisContainer: \"fake-container-1\",\n\t\t\tinsertPod: builder.ForPod(\"velero\", \"fake-pod-1\").Phase(corev1api.PodSucceeded).Result(),\n\t\t\tinsertEventsBefore: []insertEvent{\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonStarted},\n\t\t\t\t},\n\t\t\t},\n\t\t\tinsertEventsAfter: []insertEvent{\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonCompleted},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonStopped},\n\t\t\t\t\tdelay: time.Second,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectStartEvent: true,\n\t\t\texpectTerminateEvent: true,\n\t\t\texpectComplete: true,\n\t\t\texpectProgress: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"completed with progress\",\n\t\t\tthisPod: \"fak-pod-1\",\n\t\t\tthisContainer: \"fake-container-1\",\n\t\t\tinsertPod: builder.ForPod(\"velero\", \"fake-pod-1\").Phase(corev1api.PodSucceeded).Result(),\n\t\t\tinsertEventsBefore: []insertEvent{\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonStarted},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonProgress, Message: \"fake-progress-1\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonProgress, Message: \"fake-progress-2\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonCompleted},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonStopped},\n\t\t\t\t\tdelay: time.Second,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectStartEvent: true,\n\t\t\texpectTerminateEvent: true,\n\t\t\texpectComplete: true,\n\t\t\texpectProgress: 3,\n\t\t},\n\t\t{\n\t\t\tname: \"failed\",\n\t\t\tthisPod: \"fak-pod-1\",\n\t\t\tthisContainer: \"fake-container-1\",\n\t\t\tinsertPod: builder.ForPod(\"velero\", \"fake-pod-1\").Phase(corev1api.PodFailed).Result(),\n\t\t\tinsertEventsBefore: []insertEvent{\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonStarted},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonCancelled},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonStopped},\n\t\t\t\t},\n\t\t\t},\n\t\t\tterminationMessage: \"fake-termination-message-1\",\n\t\t\texpectStartEvent: true,\n\t\t\texpectTerminateEvent: true,\n\t\t\texpectFail: true,\n\t\t},\n\t\t{\n\t\t\tname: \"pod crash\",\n\t\t\tthisPod: \"fak-pod-1\",\n\t\t\tthisContainer: \"fake-container-1\",\n\t\t\tinsertPod: builder.ForPod(\"velero\", \"fake-pod-1\").Phase(corev1api.PodFailed).Result(),\n\t\t\tterminationMessage: \"fake-termination-message-2\",\n\t\t\texpectFail: true,\n\t\t},\n\t\t{\n\t\t\tname: \"canceled\",\n\t\t\tthisPod: \"fak-pod-1\",\n\t\t\tthisContainer: \"fake-container-1\",\n\t\t\tinsertPod: builder.ForPod(\"velero\", \"fake-pod-1\").Phase(corev1api.PodFailed).Result(),\n\t\t\tinsertEventsBefore: []insertEvent{\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonStarted},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonCancelled},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tevent: &corev1api.Event{Reason: EventReasonStopped},\n\t\t\t\t},\n\t\t\t},\n\t\t\tterminationMessage: fmt.Sprintf(\"Failed to init data path service for DataUpload %s: %v\", \"fake-du-name\", errors.New(ErrCancelled)),\n\t\t\texpectStartEvent: true,\n\t\t\texpectTerminateEvent: true,\n\t\t\texpectCancel: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tctx, cancel := context.WithCancel(t.Context())\n\t\t\teventWaitTimeout = time.Second * 5\n\n\t\t\tsw := startWatchFake{\n\t\t\t\tterminationMessage: test.terminationMessage,\n\t\t\t\tredirectErr: test.redirectLogErr,\n\t\t\t}\n\t\t\tfuncGetPodTerminationMessage = sw.getPodContainerTerminateMessage\n\t\t\tfuncRedirectLog = sw.redirectDataMoverLogs\n\t\t\tfuncGetResultFromMessage = sw.getResultFromMessage\n\n\t\t\tms := µServiceBRWatcher{\n\t\t\t\tctx: ctx,\n\t\t\t\tnamespace: test.namespace,\n\t\t\t\tthisPod: test.thisPod,\n\t\t\t\tthisContainer: test.thisContainer,\n\t\t\t\tpodCh: make(chan *corev1api.Pod, 2),\n\t\t\t\teventCh: make(chan *corev1api.Event, 10),\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t\tcallbacks: Callbacks{\n\t\t\t\t\tOnCompleted: sw.OnCompleted,\n\t\t\t\t\tOnFailed: sw.OnFailed,\n\t\t\t\t\tOnCancelled: sw.OnCancelled,\n\t\t\t\t\tOnProgress: sw.OnProgress,\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tms.startWatch()\n\n\t\t\tif test.ctxCancel {\n\t\t\t\tcancel()\n\t\t\t}\n\n\t\t\tfor _, ev := range test.insertEventsBefore {\n\t\t\t\tif ev.after != 0 {\n\t\t\t\t\ttime.Sleep(ev.after)\n\t\t\t\t}\n\n\t\t\t\tms.eventCh <- ev.event\n\n\t\t\t\tif ev.delay != 0 {\n\t\t\t\t\ttime.Sleep(ev.delay)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif test.insertPod != nil {\n\t\t\t\tms.podCh <- test.insertPod\n\t\t\t}\n\n\t\t\tfor _, ev := range test.insertEventsAfter {\n\t\t\t\tif ev.after != 0 {\n\t\t\t\t\ttime.Sleep(ev.after)\n\t\t\t\t}\n\n\t\t\t\tms.eventCh <- ev.event\n\n\t\t\t\tif ev.delay != 0 {\n\t\t\t\t\ttime.Sleep(ev.delay)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tms.wgWatcher.Wait()\n\n\t\t\tassert.Equal(t, test.expectStartEvent, ms.startedFromEvent)\n\t\t\tassert.Equal(t, test.expectTerminateEvent, ms.terminatedFromEvent)\n\t\t\tassert.Equal(t, test.expectComplete, sw.complete)\n\t\t\tassert.Equal(t, test.expectCancel, sw.canceled)\n\t\t\tassert.Equal(t, test.expectFail, sw.failed)\n\t\t\tassert.Equal(t, test.expectProgress, sw.progress)\n\n\t\t\tcancel()\n\t\t})\n\t}\n}\n\nfunc TestGetResultFromMessage(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\ttaskType string\n\t\tmessage string\n\t\texpectResult Result\n\t}{\n\t\t{\n\t\t\tname: \"error to unmarshall backup result\",\n\t\t\ttaskType: TaskTypeBackup,\n\t\t\tmessage: \"fake-message\",\n\t\t\texpectResult: Result{},\n\t\t},\n\t\t{\n\t\t\tname: \"error to unmarshall restore result\",\n\t\t\ttaskType: TaskTypeRestore,\n\t\t\tmessage: \"fake-message\",\n\t\t\texpectResult: Result{},\n\t\t},\n\t\t{\n\t\t\tname: \"succeed to unmarshall backup result\",\n\t\t\ttaskType: TaskTypeBackup,\n\t\t\tmessage: \"{\\\"snapshotID\\\":\\\"fake-snapshot-id\\\",\\\"emptySnapshot\\\":true,\\\"source\\\":{\\\"byPath\\\":\\\"fake-path-1\\\",\\\"volumeMode\\\":\\\"Block\\\"}}\",\n\t\t\texpectResult: Result{\n\t\t\t\tBackup: BackupResult{\n\t\t\t\t\tSnapshotID: \"fake-snapshot-id\",\n\t\t\t\t\tEmptySnapshot: true,\n\t\t\t\t\tSource: AccessPoint{\n\t\t\t\t\t\tByPath: \"fake-path-1\",\n\t\t\t\t\t\tVolMode: uploader.PersistentVolumeBlock,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"succeed to unmarshall restore result\",\n\t\t\ttaskType: TaskTypeRestore,\n\t\t\tmessage: \"{\\\"target\\\":{\\\"byPath\\\":\\\"fake-path-2\\\",\\\"volumeMode\\\":\\\"Filesystem\\\"}}\",\n\t\t\texpectResult: Result{\n\t\t\t\tRestore: RestoreResult{\n\t\t\t\t\tTarget: AccessPoint{\n\t\t\t\t\t\tByPath: \"fake-path-2\",\n\t\t\t\t\t\tVolMode: uploader.PersistentVolumeFilesystem,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tresult := getResultFromMessage(test.taskType, test.message, velerotest.NewLogger())\n\t\t\tassert.Equal(t, test.expectResult, result)\n\t\t})\n\t}\n}\n\nfunc TestGetProgressFromMessage(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tmessage string\n\t\texpectProgress uploader.Progress\n\t}{\n\t\t{\n\t\t\tname: \"error to unmarshall progress\",\n\t\t\tmessage: \"fake-message\",\n\t\t\texpectProgress: uploader.Progress{},\n\t\t},\n\t\t{\n\t\t\tname: \"succeed to unmarshall progress\",\n\t\t\tmessage: \"{\\\"totalBytes\\\":1000,\\\"doneBytes\\\":200}\",\n\t\t\texpectProgress: uploader.Progress{\n\t\t\t\tTotalBytes: 1000,\n\t\t\t\tBytesDone: 200,\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tprogress := getProgressFromMessage(test.message, velerotest.NewLogger())\n\t\t\tassert.Equal(t, test.expectProgress, *progress)\n\t\t})\n\t}\n}\n\ntype redirectFake struct {\n\tlogFile *os.File\n\tcreateTempErr error\n\tgetPodLogErr error\n\tlogMessage string\n}\n\nfunc (rf *redirectFake) fakeCreateTempFile(_ string, _ string) (*os.File, error) {\n\tif rf.createTempErr != nil {\n\t\treturn nil, rf.createTempErr\n\t}\n\n\treturn rf.logFile, nil\n}\n\nfunc (rf *redirectFake) fakeCollectPodLogs(_ context.Context, _ corev1client.CoreV1Interface, _ string, _ string, _ string, output io.Writer) error {\n\tif rf.getPodLogErr != nil {\n\t\treturn rf.getPodLogErr\n\t}\n\n\t_, err := output.Write([]byte(rf.logMessage))\n\n\treturn err\n}\n\nfunc TestRedirectDataMoverLogs(t *testing.T) {\n\tlogFileName := path.Join(os.TempDir(), \"test-logger-file.log\")\n\n\tvar buffer string\n\n\ttests := []struct {\n\t\tname string\n\t\tthisPod string\n\t\tlogMessage string\n\t\tlogger logrus.FieldLogger\n\t\tcreateTempErr error\n\t\tcollectLogErr error\n\t\texpectErr string\n\t}{\n\t\t{\n\t\t\tname: \"error to create temp file\",\n\t\t\tthisPod: \"fake-pod\",\n\t\t\tcreateTempErr: errors.New(\"fake-create-temp-error\"),\n\t\t\tlogger: velerotest.NewLogger(),\n\t\t\texpectErr: \"error to create temp file for data mover pod log: fake-create-temp-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"error to collect pod log\",\n\t\t\tthisPod: \"fake-pod\",\n\t\t\tcollectLogErr: errors.New(\"fake-collect-log-error\"),\n\t\t\tlogger: velerotest.NewLogger(),\n\t\t\texpectErr: fmt.Sprintf(\"error to collect logs to %s for data mover pod fake-pod: fake-collect-log-error\", logFileName),\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tthisPod: \"fake-pod\",\n\t\t\tlogMessage: \"fake-log-message-01\\nfake-log-message-02\\nfake-log-message-03\\n\",\n\t\t\tlogger: velerotest.NewSingleLoggerWithHooks(&buffer, logging.DefaultHooks(true)),\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tbuffer = \"\"\n\n\t\t\tlogFile, err := os.Create(logFileName)\n\t\t\trequire.NoError(t, err)\n\n\t\t\trf := redirectFake{\n\t\t\t\tlogFile: logFile,\n\t\t\t\tcreateTempErr: test.createTempErr,\n\t\t\t\tgetPodLogErr: test.collectLogErr,\n\t\t\t\tlogMessage: test.logMessage,\n\t\t\t}\n\n\t\t\tfuncCreateTemp = rf.fakeCreateTempFile\n\t\t\tfuncCollectPodLogs = rf.fakeCollectPodLogs\n\n\t\t\tfakeKubeClient := kubeclientfake.NewSimpleClientset()\n\n\t\t\terr = redirectDataMoverLogs(t.Context(), fakeKubeClient, \"\", test.thisPod, \"\", test.logger)\n\t\t\tif test.expectErr != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.expectErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\tassert.Contains(t, buffer, test.logMessage)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/datapath/mocks/asyncBR.go", "content": "// Code generated by mockery v2.39.1. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tcontext \"context\"\n\n\tmock \"github.com/stretchr/testify/mock\"\n\tdatapath \"github.com/vmware-tanzu/velero/pkg/datapath\"\n)\n\n// AsyncBR is an autogenerated mock type for the AsyncBR type\ntype AsyncBR struct {\n\tmock.Mock\n}\n\n// Cancel provides a mock function with given fields:\nfunc (_m *AsyncBR) Cancel() {\n\t_m.Called()\n}\n\n// Close provides a mock function with given fields: ctx\nfunc (_m *AsyncBR) Close(ctx context.Context) {\n\t_m.Called(ctx)\n}\n\n// Init provides a mock function with given fields: ctx, param\nfunc (_m *AsyncBR) Init(ctx context.Context, param interface{}) error {\n\tret := _m.Called(ctx, param)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Init\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, interface{}) error); ok {\n\t\tr0 = rf(ctx, param)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// StartBackup provides a mock function with given fields: source, dataMoverConfig, param\nfunc (_m *AsyncBR) StartBackup(source datapath.AccessPoint, dataMoverConfig map[string]string, param interface{}) error {\n\tret := _m.Called(source, dataMoverConfig, param)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for StartBackup\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(datapath.AccessPoint, map[string]string, interface{}) error); ok {\n\t\tr0 = rf(source, dataMoverConfig, param)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// StartRestore provides a mock function with given fields: snapshotID, target, dataMoverConfig\nfunc (_m *AsyncBR) StartRestore(snapshotID string, target datapath.AccessPoint, dataMoverConfig map[string]string) error {\n\tret := _m.Called(snapshotID, target, dataMoverConfig)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for StartRestore\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(string, datapath.AccessPoint, map[string]string) error); ok {\n\t\tr0 = rf(snapshotID, target, dataMoverConfig)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// NewAsyncBR creates a new instance of AsyncBR. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\n// The first argument is typically a *testing.T value.\nfunc NewAsyncBR(t interface {\n\tmock.TestingT\n\tCleanup(func())\n}) *AsyncBR {\n\tmock := &AsyncBR{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/datapath/types.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage datapath\n\nimport (\n\t\"context\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n)\n\n// Result represents the result of a backup/restore\ntype Result struct {\n\tBackup BackupResult\n\tRestore RestoreResult\n}\n\n// BackupResult represents the result of a backup\ntype BackupResult struct {\n\tSnapshotID string `json:\"snapshotID\"`\n\tEmptySnapshot bool `json:\"emptySnapshot\"`\n\tSource AccessPoint `json:\"source,omitempty\"`\n\tTotalBytes int64 `json:\"totalBytes,omitempty\"`\n\tIncrementalBytes int64 `json:\"incrementalBytes,omitempty\"`\n}\n\n// RestoreResult represents the result of a restore\ntype RestoreResult struct {\n\tTarget AccessPoint `json:\"target,omitempty\"`\n\tTotalBytes int64 `json:\"totalBytes,omitempty\"`\n}\n\n// Callbacks defines the collection of callbacks during backup/restore\ntype Callbacks struct {\n\tOnCompleted func(context.Context, string, string, Result)\n\tOnFailed func(context.Context, string, string, error)\n\tOnCancelled func(context.Context, string, string)\n\tOnProgress func(context.Context, string, string, *uploader.Progress)\n}\n\n// AccessPoint represents an access point that has been exposed to a data path instance\ntype AccessPoint struct {\n\tByPath string `json:\"byPath\"`\n\tVolMode uploader.PersistentVolumeMode `json:\"volumeMode\"`\n}\n\n// AsyncBR is the interface for asynchronous data path methods\ntype AsyncBR interface {\n\t// Init initializes an asynchronous data path instance\n\tInit(ctx context.Context, param any) error\n\n\t// StartBackup starts an asynchronous data path instance for backup\n\tStartBackup(source AccessPoint, dataMoverConfig map[string]string, param any) error\n\n\t// StartRestore starts an asynchronous data path instance for restore\n\tStartRestore(snapshotID string, target AccessPoint, dataMoverConfig map[string]string) error\n\n\t// Cancel cancels an asynchronous data path instance\n\tCancel()\n\n\t// Close closes an asynchronous data path instance\n\tClose(ctx context.Context)\n}\n" }, { "path": "pkg/discovery/helper.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage discovery\n\nimport (\n\t\"sort\"\n\t\"strings\"\n\t\"sync\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/api/meta\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/apimachinery/pkg/version\"\n\t\"k8s.io/client-go/discovery\"\n\t\"k8s.io/client-go/restmapper\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/features\"\n\tkcmdutil \"github.com/vmware-tanzu/velero/third_party/kubernetes/pkg/kubectl/cmd/util\"\n)\n\n//go:generate mockery --name Helper\n\n// Helper exposes functions for interacting with the Kubernetes discovery\n// API.\ntype Helper interface {\n\t// Resources gets the current set of resources retrieved from discovery\n\t// that are backuppable by Velero.\n\tResources() []*metav1.APIResourceList\n\n\t// ResourceFor gets a fully-resolved GroupVersionResource and an\n\t// APIResource for the provided partially-specified GroupVersionResource.\n\tResourceFor(input schema.GroupVersionResource) (schema.GroupVersionResource, metav1.APIResource, error)\n\n\t// KindFor gets a fully-resolved GroupVersionResource and an\n\t// APIResource for the provided partially-specified GroupVersionKind.\n\tKindFor(input schema.GroupVersionKind) (schema.GroupVersionResource, metav1.APIResource, error)\n\n\t// Refresh pulls an updated set of Velero-backuppable resources from the\n\t// discovery API.\n\tRefresh() error\n\n\t// APIGroups gets the current set of supported APIGroups\n\t// in the cluster.\n\tAPIGroups() []metav1.APIGroup\n\n\t// ServerVersion retrieves and parses the server's k8s version (git version)\n\t// in the cluster.\n\tServerVersion() *version.Info\n}\n\ntype serverResourcesInterface interface {\n\t// ServerPreferredResources() is used to populate Resources() with only Preferred Versions - this is the default\n\tServerPreferredResources() ([]*metav1.APIResourceList, error)\n\t// ServerGroupsAndResources returns supported groups and resources for *all* groups and versions\n\t// Used to populate Resources() if feature flag is passed\n\tServerGroupsAndResources() ([]*metav1.APIGroup, []*metav1.APIResourceList, error)\n}\n\ntype helper struct {\n\tdiscoveryClient discovery.AggregatedDiscoveryInterface\n\tlogger logrus.FieldLogger\n\n\t// lock guards mapper, resources and resourcesMap\n\tlock sync.RWMutex\n\tmapper meta.RESTMapper\n\tresources []*metav1.APIResourceList\n\tresourcesMap map[schema.GroupVersionResource]metav1.APIResource\n\tkindMap map[schema.GroupVersionKind]metav1.APIResource\n\tapiGroups []metav1.APIGroup\n\tserverVersion *version.Info\n}\n\nvar _ Helper = &helper{}\n\nfunc NewHelper(discoveryClient discovery.AggregatedDiscoveryInterface, logger logrus.FieldLogger) (Helper, error) {\n\th := &helper{\n\t\tdiscoveryClient: discoveryClient,\n\t\tlogger: logger,\n\t}\n\tif err := h.Refresh(); err != nil {\n\t\treturn nil, err\n\t}\n\treturn h, nil\n}\n\nfunc (h *helper) ResourceFor(input schema.GroupVersionResource) (schema.GroupVersionResource, metav1.APIResource, error) {\n\th.lock.RLock()\n\tdefer h.lock.RUnlock()\n\n\tgvr, err := h.mapper.ResourceFor(input)\n\tif err != nil {\n\t\treturn schema.GroupVersionResource{}, metav1.APIResource{}, err\n\t}\n\n\tapiResource, found := h.resourcesMap[gvr]\n\tif !found {\n\t\treturn schema.GroupVersionResource{}, metav1.APIResource{}, errors.Errorf(\"APIResource not found for GroupVersionResource %s\", gvr)\n\t}\n\n\treturn gvr, apiResource, nil\n}\n\nfunc (h *helper) KindFor(input schema.GroupVersionKind) (schema.GroupVersionResource, metav1.APIResource, error) {\n\th.lock.RLock()\n\tdefer h.lock.RUnlock()\n\n\tif resource, ok := h.kindMap[input]; ok {\n\t\treturn schema.GroupVersionResource{\n\t\t\tGroup: resource.Group,\n\t\t\tVersion: resource.Version,\n\t\t\tResource: resource.Name,\n\t\t}, resource, nil\n\t}\n\tm, err := h.mapper.RESTMapping(schema.GroupKind{Group: input.Group, Kind: input.Kind}, input.Version)\n\tif err != nil {\n\t\treturn schema.GroupVersionResource{}, metav1.APIResource{}, err\n\t}\n\tif resource, ok := h.kindMap[m.GroupVersionKind]; ok {\n\t\treturn schema.GroupVersionResource{\n\t\t\tGroup: resource.Group,\n\t\t\tVersion: resource.Version,\n\t\t\tResource: resource.Name,\n\t\t}, resource, nil\n\t}\n\treturn schema.GroupVersionResource{}, metav1.APIResource{}, errors.Errorf(\"APIResource not found for GroupVersionKind %v \", input)\n}\n\nfunc (h *helper) Refresh() error {\n\th.lock.Lock()\n\tdefer h.lock.Unlock()\n\n\tgroupResources, err := restmapper.GetAPIGroupResources(h.discoveryClient)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\tvar serverResources []*metav1.APIResourceList\n\n\tif features.IsEnabled(velerov1api.APIGroupVersionsFeatureFlag) {\n\t\t// ServerGroupsAndResources returns all APIGroup and APIResouceList - not only preferred versions\n\t\t_, serverAllResources, err := refreshServerGroupsAndResources(h.discoveryClient, h.logger)\n\t\tif err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\t\th.logger.Infof(\"The '%s' feature flag was specified, using all API group versions.\", velerov1api.APIGroupVersionsFeatureFlag)\n\t\tserverResources = serverAllResources\n\t} else {\n\t\t// ServerPreferredResources() returns only preferred APIGroup - this is the default since no feature flag has been passed\n\t\tserverPreferredResources, err := refreshServerPreferredResources(h.discoveryClient, h.logger)\n\t\tif err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\t\tserverResources = serverPreferredResources\n\t}\n\n\th.resources = discovery.FilteredBy(\n\t\tAnd(filterByVerbs, skipSubresource),\n\t\tserverResources,\n\t)\n\n\tsortResources(h.resources)\n\n\tshortcutExpander, err := kcmdutil.NewShortcutExpander(restmapper.NewDiscoveryRESTMapper(groupResources), h.resources, h.logger)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\th.mapper = shortcutExpander\n\n\th.resourcesMap = make(map[schema.GroupVersionResource]metav1.APIResource)\n\th.kindMap = make(map[schema.GroupVersionKind]metav1.APIResource)\n\tfor _, resourceGroup := range h.resources {\n\t\tgv, err := schema.ParseGroupVersion(resourceGroup.GroupVersion)\n\t\tif err != nil {\n\t\t\treturn errors.Wrapf(err, \"unable to parse GroupVersion %s\", resourceGroup.GroupVersion)\n\t\t}\n\n\t\tfor _, resource := range resourceGroup.APIResources {\n\t\t\tgvr := gv.WithResource(resource.Name)\n\t\t\tgvk := gv.WithKind(resource.Kind)\n\t\t\tresource.Group = gv.Group\n\t\t\tresource.Version = gv.Version\n\t\t\th.resourcesMap[gvr] = resource\n\t\t\th.kindMap[gvk] = resource\n\t\t}\n\t}\n\n\tapiGroupList, err := h.discoveryClient.ServerGroups()\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\th.apiGroups = apiGroupList.Groups\n\n\tserverVersion, err := h.discoveryClient.ServerVersion()\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\th.serverVersion = serverVersion\n\n\treturn nil\n}\n\nfunc refreshServerPreferredResources(discoveryClient serverResourcesInterface, logger logrus.FieldLogger) ([]*metav1.APIResourceList, error) {\n\tpreferredResources, err := discoveryClient.ServerPreferredResources()\n\tif err != nil {\n\t\tif discoveryErr, ok := err.(*discovery.ErrGroupDiscoveryFailed); ok {\n\t\t\tfor groupVersion, err := range discoveryErr.Groups {\n\t\t\t\tlogger.WithError(err).Warnf(\"Failed to discover group: %v\", groupVersion)\n\t\t\t}\n\t\t\treturn preferredResources, nil\n\t\t}\n\t}\n\treturn preferredResources, err\n}\n\nfunc refreshServerGroupsAndResources(discoveryClient serverResourcesInterface, logger logrus.FieldLogger) ([]*metav1.APIGroup, []*metav1.APIResourceList, error) {\n\tserverGroups, serverResources, err := discoveryClient.ServerGroupsAndResources()\n\tif err != nil {\n\t\tif discoveryErr, ok := err.(*discovery.ErrGroupDiscoveryFailed); ok {\n\t\t\tfor groupVersion, err := range discoveryErr.Groups {\n\t\t\t\tlogger.WithError(err).Warnf(\"Failed to discover group: %v\", groupVersion)\n\t\t\t}\n\t\t\treturn serverGroups, serverResources, nil\n\t\t}\n\t}\n\treturn serverGroups, serverResources, err\n}\n\n// And returns a composite predicate that implements a logical AND of the predicates passed to it.\nfunc And(predicates ...discovery.ResourcePredicateFunc) discovery.ResourcePredicate {\n\treturn and{predicates}\n}\n\ntype and struct {\n\tpredicates []discovery.ResourcePredicateFunc\n}\n\nfunc (a and) Match(groupVersion string, r *metav1.APIResource) bool {\n\tfor _, p := range a.predicates {\n\t\tif !p(groupVersion, r) {\n\t\t\treturn false\n\t\t}\n\t}\n\n\treturn true\n}\n\nfunc filterByVerbs(groupVersion string, r *metav1.APIResource) bool {\n\treturn discovery.SupportsAllVerbs{Verbs: []string{\"list\", \"create\", \"get\", \"delete\"}}.Match(groupVersion, r)\n}\n\nfunc skipSubresource(_ string, r *metav1.APIResource) bool {\n\t// if we have a slash, then this is a subresource and we shouldn't include it.\n\treturn !strings.Contains(r.Name, \"/\")\n}\n\n// sortResources sources resources by moving extensions to the end of the slice. The order of all\n// the other resources is preserved.\nfunc sortResources(resources []*metav1.APIResourceList) {\n\tsort.SliceStable(resources, func(i, j int) bool {\n\t\tleft := resources[i]\n\t\tleftGV, _ := schema.ParseGroupVersion(left.GroupVersion)\n\t\t// not checking error because it should be impossible to fail to parse data coming from the\n\t\t// apiserver\n\t\tif leftGV.Group == \"extensions\" {\n\t\t\t// always sort extensions at the bottom by saying left is \"greater\"\n\t\t\treturn false\n\t\t}\n\n\t\tright := resources[j]\n\t\trightGV, _ := schema.ParseGroupVersion(right.GroupVersion)\n\t\t// not checking error because it should be impossible to fail to parse data coming from the\n\t\t// apiserver\n\t\tif rightGV.Group == \"extensions\" {\n\t\t\t// always sort extensions at the bottom by saying left is \"less\"\n\t\t\treturn true\n\t\t}\n\n\t\treturn i < j\n\t})\n}\n\nfunc (h *helper) Resources() []*metav1.APIResourceList {\n\th.lock.RLock()\n\tdefer h.lock.RUnlock()\n\treturn h.resources\n}\n\nfunc (h *helper) APIGroups() []metav1.APIGroup {\n\th.lock.RLock()\n\tdefer h.lock.RUnlock()\n\treturn h.apiGroups\n}\n\nfunc (h *helper) ServerVersion() *version.Info {\n\th.lock.RLock()\n\tdefer h.lock.RUnlock()\n\treturn h.serverVersion\n}\n" }, { "path": "pkg/discovery/helper_test.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage discovery\n\nimport (\n\t\"errors\"\n\t\"sync\"\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/apimachinery/pkg/version\"\n\t\"k8s.io/client-go/discovery/fake\"\n\tclientgotesting \"k8s.io/client-go/testing\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/features\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n)\n\nfunc TestSortResources(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tresources []*metav1.APIResourceList\n\t\texpected []*metav1.APIResourceList\n\t}{\n\t\t{\n\t\t\tname: \"no resources\",\n\t\t},\n\t\t{\n\t\t\tname: \"no extensions, order is preserved\",\n\t\t\tresources: []*metav1.APIResourceList{\n\t\t\t\t{GroupVersion: \"v1\"},\n\t\t\t\t{GroupVersion: \"groupC/v1\"},\n\t\t\t\t{GroupVersion: \"groupA/v1\"},\n\t\t\t\t{GroupVersion: \"groupB/v1\"},\n\t\t\t},\n\t\t\texpected: []*metav1.APIResourceList{\n\t\t\t\t{GroupVersion: \"v1\"},\n\t\t\t\t{GroupVersion: \"groupC/v1\"},\n\t\t\t\t{GroupVersion: \"groupA/v1\"},\n\t\t\t\t{GroupVersion: \"groupB/v1\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"extensions moves to end, order is preserved\",\n\t\t\tresources: []*metav1.APIResourceList{\n\t\t\t\t{GroupVersion: \"extensions/v1beta1\"},\n\t\t\t\t{GroupVersion: \"v1\"},\n\t\t\t\t{GroupVersion: \"groupC/v1\"},\n\t\t\t\t{GroupVersion: \"groupA/v1\"},\n\t\t\t\t{GroupVersion: \"groupB/v1\"},\n\t\t\t\t{GroupVersion: \"apps/v1beta1\"},\n\t\t\t},\n\t\t\texpected: []*metav1.APIResourceList{\n\t\t\t\t{GroupVersion: \"v1\"},\n\t\t\t\t{GroupVersion: \"groupC/v1\"},\n\t\t\t\t{GroupVersion: \"groupA/v1\"},\n\t\t\t\t{GroupVersion: \"groupB/v1\"},\n\t\t\t\t{GroupVersion: \"apps/v1beta1\"},\n\t\t\t\t{GroupVersion: \"extensions/v1beta1\"},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tt.Log(\"before\")\n\t\t\tfor _, r := range test.resources {\n\t\t\t\tt.Log(r.GroupVersion)\n\t\t\t}\n\t\t\tsortResources(test.resources)\n\t\t\tt.Logf(\"after\")\n\t\t\tfor _, r := range test.resources {\n\t\t\t\tt.Log(r.GroupVersion)\n\t\t\t}\n\t\t\tassert.Equal(t, test.expected, test.resources)\n\t\t})\n\t}\n}\n\nfunc TestFilteringByVerbs(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tgroupVersion string\n\t\tres *metav1.APIResource\n\t\texpected bool\n\t}{\n\t\t{\n\t\t\tname: \"resource that supports list, create, get, delete\",\n\t\t\tgroupVersion: \"v1\",\n\t\t\tres: &metav1.APIResource{\n\t\t\t\tVerbs: metav1.Verbs{\"list\", \"create\", \"get\", \"delete\"},\n\t\t\t},\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"resource that supports list, create, get, delete in a different order\",\n\t\t\tgroupVersion: \"v1\",\n\t\t\tres: &metav1.APIResource{\n\t\t\t\tVerbs: metav1.Verbs{\"delete\", \"get\", \"create\", \"list\"},\n\t\t\t},\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"resource that supports list, create, get, delete, and more\",\n\t\t\tgroupVersion: \"v1\",\n\t\t\tres: &metav1.APIResource{\n\t\t\t\tVerbs: metav1.Verbs{\"list\", \"create\", \"get\", \"delete\", \"update\", \"patch\", \"deletecollection\"},\n\t\t\t},\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"resource that supports only list and create\",\n\t\t\tgroupVersion: \"v1\",\n\t\t\tres: &metav1.APIResource{\n\t\t\t\tVerbs: metav1.Verbs{\"list\", \"create\"},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"resource that supports only get and delete\",\n\t\t\tgroupVersion: \"v1\",\n\t\t\tres: &metav1.APIResource{\n\t\t\t\tVerbs: metav1.Verbs{\"get\", \"delete\"},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tout := filterByVerbs(test.groupVersion, test.res)\n\t\t\tassert.Equal(t, test.expected, out)\n\t\t})\n\t}\n}\n\nfunc TestRefreshServerPreferredResources(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tresourceList []*metav1.APIResourceList\n\t\tapiGroup []*metav1.APIGroup\n\t\tfailedGroups map[schema.GroupVersion]error\n\t\treturnError error\n\t}{\n\t\t{\n\t\t\tname: \"all groups discovered, no error is returned\",\n\t\t\tresourceList: []*metav1.APIResourceList{\n\t\t\t\t{GroupVersion: \"groupB/v1\"},\n\t\t\t\t{GroupVersion: \"apps/v1beta1\"},\n\t\t\t\t{GroupVersion: \"extensions/v1beta1\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"failed to discover some groups, no error is returned\",\n\t\t\tresourceList: []*metav1.APIResourceList{\n\t\t\t\t{GroupVersion: \"groupB/v1\"},\n\t\t\t\t{GroupVersion: \"apps/v1beta1\"},\n\t\t\t\t{GroupVersion: \"extensions/v1beta1\"},\n\t\t\t},\n\t\t\tfailedGroups: map[schema.GroupVersion]error{\n\t\t\t\t{Group: \"groupA\", Version: \"v1\"}: errors.New(\"Fake error\"),\n\t\t\t\t{Group: \"groupC\", Version: \"v2\"}: errors.New(\"Fake error\"),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"non ErrGroupDiscoveryFailed error, returns error\",\n\t\t\treturnError: errors.New(\"Generic error\"),\n\t\t},\n\t}\n\n\tformatFlag := logging.FormatText\n\n\tfor _, test := range tests {\n\t\tfakeServer := velerotest.NewFakeServerResourcesInterface(test.resourceList, test.apiGroup, test.failedGroups, test.returnError)\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tresources, err := refreshServerPreferredResources(fakeServer, logging.DefaultLogger(logrus.DebugLevel, formatFlag))\n\t\t\tif test.returnError != nil {\n\t\t\t\trequire.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, test.returnError, err)\n\t\t\t}\n\n\t\t\tassert.Equal(t, test.resourceList, resources)\n\t\t})\n\t}\n}\n\nfunc TestHelper_ResourceFor(t *testing.T) {\n\tfakeDiscoveryClient := &fake.FakeDiscovery{\n\t\tFake: &clientgotesting.Fake{},\n\t}\n\tfakeDiscoveryClient.Resources = []*metav1.APIResourceList{\n\t\t{\n\t\t\tGroupVersion: \"v1\",\n\t\t\tAPIResources: []metav1.APIResource{\n\t\t\t\t{\n\t\t\t\t\tName: \"pods\",\n\t\t\t\t\tKind: \"Pod\",\n\t\t\t\t\tGroup: \"\",\n\t\t\t\t\tVersion: \"v1\",\n\t\t\t\t\tVerbs: []string{\"create\", \"get\", \"list\"},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\th := &helper{\n\t\tdiscoveryClient: &velerotest.DiscoveryClient{\n\t\t\tFakeDiscovery: fakeDiscoveryClient,\n\t\t},\n\t\tlock: sync.RWMutex{},\n\t\tmapper: nil,\n\t\tresources: fakeDiscoveryClient.Resources,\n\t\tresourcesMap: make(map[schema.GroupVersionResource]metav1.APIResource),\n\t\tserverVersion: &version.Info{Major: \"1\", Minor: \"22\", GitVersion: \"v1.22.1\"},\n\t}\n\n\tfor _, resourceList := range h.resources {\n\t\tfor _, resource := range resourceList.APIResources {\n\t\t\tgvr := schema.GroupVersionResource{\n\t\t\t\tGroup: resource.Group,\n\t\t\t\tVersion: resource.Version,\n\t\t\t\tResource: resource.Name,\n\t\t\t}\n\t\t\th.resourcesMap[gvr] = resource\n\t\t}\n\t}\n\tpvGVR := schema.GroupVersionResource{\n\t\tGroup: \"\",\n\t\tVersion: \"v1\",\n\t\tResource: \"pods\",\n\t}\n\th.mapper = &velerotest.FakeMapper{Resources: map[schema.GroupVersionResource]schema.GroupVersionResource{pvGVR: pvGVR}}\n\n\ttests := []struct {\n\t\tname string\n\t\terr string\n\t\tinput *schema.GroupVersionResource\n\t\tisNotFoundRes bool\n\t\texpectedGVR *schema.GroupVersionResource\n\t\texpectedAPIResource *metav1.APIResource\n\t}{\n\t\t{\n\t\t\tname: \"Found resource\",\n\t\t\tinput: &schema.GroupVersionResource{\n\t\t\t\tGroup: \"\",\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResource: \"pods\",\n\t\t\t},\n\t\t\texpectedAPIResource: &metav1.APIResource{\n\t\t\t\tName: \"pods\",\n\t\t\t\tKind: \"Pod\",\n\t\t\t\tGroup: \"\",\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVerbs: []string{\"create\", \"get\", \"list\"},\n\t\t\t},\n\t\t\texpectedGVR: &schema.GroupVersionResource{\n\t\t\t\tGroup: \"\",\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResource: \"pods\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Error to found resource\",\n\t\t\tinput: &schema.GroupVersionResource{\n\t\t\t\tGroup: \"\",\n\t\t\t\tVersion: \"v2\",\n\t\t\t\tResource: \"pods\",\n\t\t\t},\n\t\t\terr: \"invalid resource\",\n\t\t\texpectedGVR: &schema.GroupVersionResource{},\n\t\t\texpectedAPIResource: &metav1.APIResource{},\n\t\t},\n\t\t{\n\t\t\tname: \"Error to found api resource\",\n\t\t\tinput: &schema.GroupVersionResource{\n\t\t\t\tGroup: \"\",\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResource: \"pods\",\n\t\t\t},\n\t\t\tisNotFoundRes: true,\n\t\t\terr: \"APIResource not found for GroupVersionResource\",\n\t\t\texpectedGVR: &schema.GroupVersionResource{},\n\t\t\texpectedAPIResource: &metav1.APIResource{},\n\t\t},\n\t}\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tif tc.isNotFoundRes {\n\t\t\t\th.resourcesMap = nil\n\t\t\t}\n\t\t\tgvr, apiResource, err := h.ResourceFor(*tc.input)\n\t\t\tif tc.err == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t} else {\n\t\t\t\trequire.ErrorContains(t, err, tc.err)\n\t\t\t}\n\t\t\tassert.Equal(t, *tc.expectedGVR, gvr)\n\t\t\tassert.Equal(t, *tc.expectedAPIResource, apiResource)\n\t\t})\n\t}\n}\n\nfunc TestHelper_KindFor(t *testing.T) {\n\tfakeDiscoveryClient := &fake.FakeDiscovery{\n\t\tFake: &clientgotesting.Fake{},\n\t}\n\tfakeDiscoveryClient.Resources = []*metav1.APIResourceList{\n\t\t{\n\t\t\tGroupVersion: \"v1\",\n\t\t\tAPIResources: []metav1.APIResource{\n\t\t\t\t{\n\t\t\t\t\tName: \"pods\",\n\t\t\t\t\tKind: \"Pod\",\n\t\t\t\t\tGroup: \"\",\n\t\t\t\t\tVersion: \"v1\",\n\t\t\t\t\tVerbs: []string{\"create\", \"get\", \"list\"},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tpvGVK := schema.GroupVersionKind{\n\t\tGroup: \"\",\n\t\tVersion: \"v1\",\n\t\tKind: \"Deployment\",\n\t}\n\tpvAPIRes := metav1.APIResource{\n\t\tName: \"deployments\",\n\t\tKind: \"Deployment\",\n\t\tGroup: \"apps\",\n\t\tVersion: \"v1\",\n\t\tVerbs: []string{\"create\", \"get\", \"list\"},\n\t}\n\n\th := &helper{\n\t\tdiscoveryClient: &velerotest.DiscoveryClient{\n\t\t\tFakeDiscovery: fakeDiscoveryClient,\n\t\t},\n\t\tlock: sync.RWMutex{},\n\t\tresources: fakeDiscoveryClient.Resources,\n\t\tresourcesMap: make(map[schema.GroupVersionResource]metav1.APIResource),\n\t\tserverVersion: &version.Info{Major: \"1\", Minor: \"22\", GitVersion: \"v1.22.1\"},\n\t}\n\n\th.kindMap = map[schema.GroupVersionKind]metav1.APIResource{pvGVK: pvAPIRes}\n\th.mapper = &velerotest.FakeMapper{KindToPluralResource: map[schema.GroupVersionKind]schema.GroupVersionResource{}}\n\tfor _, resourceList := range h.resources {\n\t\tfor _, resource := range resourceList.APIResources {\n\t\t\tgvr := schema.GroupVersionResource{\n\t\t\t\tGroup: resource.Group,\n\t\t\t\tVersion: resource.Version,\n\t\t\t\tResource: resource.Name,\n\t\t\t}\n\t\t\th.resourcesMap[gvr] = resource\n\t\t}\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\terr string\n\t\tinput *schema.GroupVersionKind\n\t\tisNotFoundRes bool\n\t\texpectedGVR *schema.GroupVersionResource\n\t\texpectedAPIResource *metav1.APIResource\n\t}{\n\t\t{\n\t\t\tname: \"Found resource\",\n\t\t\tinput: &schema.GroupVersionKind{\n\t\t\t\tGroup: \"\",\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tKind: \"Deployment\",\n\t\t\t},\n\t\t\texpectedAPIResource: &metav1.APIResource{\n\t\t\t\tName: \"deployments\",\n\t\t\t\tKind: \"Deployment\",\n\t\t\t\tGroup: \"apps\",\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVerbs: []string{\"create\", \"get\", \"list\"},\n\t\t\t},\n\t\t\texpectedGVR: &schema.GroupVersionResource{\n\t\t\t\tGroup: \"apps\",\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tResource: \"deployments\",\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"Not found resource\",\n\t\t\tinput: &schema.GroupVersionKind{\n\t\t\t\tGroup: \"\",\n\t\t\t\tVersion: \"v2\",\n\t\t\t\tKind: \"Deployment\",\n\t\t\t},\n\t\t\texpectedAPIResource: &metav1.APIResource{},\n\t\t\texpectedGVR: &schema.GroupVersionResource{},\n\t\t\terr: \"no matches for kind\",\n\t\t},\n\t}\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tgvr, apiResource, err := h.KindFor(*tc.input)\n\t\t\tif tc.err == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t} else {\n\t\t\t\trequire.ErrorContains(t, err, tc.err)\n\t\t\t}\n\t\t\tassert.Equal(t, *tc.expectedGVR, gvr)\n\t\t\tassert.Equal(t, *tc.expectedAPIResource, apiResource)\n\t\t})\n\t}\n}\n\nfunc TestHelper_Refresh(t *testing.T) {\n\ttestCases := []struct {\n\t\tdescription string\n\t\tfeatures string\n\t\tgroupResources []*metav1.APIResourceList\n\t\tserverGroups []*metav1.APIGroup\n\t\texpectedErr error\n\t\texpectedResource metav1.APIResource\n\t}{\n\t\t{\n\t\t\tdescription: \"Default case - Resource found\",\n\t\t\tgroupResources: []*metav1.APIResourceList{\n\t\t\t\t{\n\t\t\t\t\tGroupVersion: \"v1\",\n\t\t\t\t\tAPIResources: []metav1.APIResource{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"pods\",\n\t\t\t\t\t\t\tKind: \"Pod\",\n\t\t\t\t\t\t\tGroup: \"\",\n\t\t\t\t\t\t\tVersion: \"v1\",\n\t\t\t\t\t\t\tVerbs: []string{\"get\", \"list\", \"create\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tserverGroups: []*metav1.APIGroup{\n\t\t\t\t{\n\t\t\t\t\tName: \"group1\",\n\t\t\t\t\tVersions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tGroupVersion: \"v1\",\n\t\t\t\t\t\t\tVersion: \"v1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedErr: nil,\n\t\t\texpectedResource: metav1.APIResource{\n\t\t\t\tName: \"pods\",\n\t\t\t\tKind: \"Pod\",\n\t\t\t\tGroup: \"\",\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVerbs: []string{\"get\", \"list\", \"create\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tdescription: \"Feature flag enabled - ServerGroupsAndResources\",\n\t\t\tfeatures: velerov1api.APIGroupVersionsFeatureFlag,\n\t\t\tgroupResources: []*metav1.APIResourceList{},\n\t\t\tserverGroups: []*metav1.APIGroup{\n\t\t\t\t{\n\t\t\t\t\tName: \"group1\",\n\t\t\t\t\tVersions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tGroupVersion: \"v1\",\n\t\t\t\t\t\t\tVersion: \"v1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedErr: nil,\n\t\t\texpectedResource: metav1.APIResource{\n\t\t\t\tName: \"pods\",\n\t\t\t\tKind: \"Pod\",\n\t\t\t\tGroup: \"\",\n\t\t\t\tVersion: \"v1\",\n\t\t\t\tVerbs: []string{\"get\", \"list\", \"create\"},\n\t\t\t},\n\t\t},\n\t}\n\n\tfakeDiscoveryClient := &fake.FakeDiscovery{\n\t\tFake: &clientgotesting.Fake{},\n\t}\n\tfakeDiscoveryClient.Resources = []*metav1.APIResourceList{\n\t\t{\n\t\t\tGroupVersion: \"v1\",\n\t\t\tAPIResources: []metav1.APIResource{\n\t\t\t\t{\n\t\t\t\t\tName: \"pods\",\n\t\t\t\t\tKind: \"Pod\",\n\t\t\t\t\tGroup: \"\",\n\t\t\t\t\tVersion: \"v1\",\n\t\t\t\t\tVerbs: []string{\"create\", \"get\", \"list\"},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, testCase := range testCases {\n\t\tt.Run(testCase.description, func(t *testing.T) {\n\t\t\th := &helper{\n\t\t\t\tlock: sync.RWMutex{},\n\t\t\t\tdiscoveryClient: &velerotest.DiscoveryClient{\n\t\t\t\t\tFakeDiscovery: fakeDiscoveryClient,\n\t\t\t\t},\n\t\t\t\tlogger: logrus.New(),\n\t\t\t}\n\t\t\t// Set feature flags\n\t\t\tif testCase.features != \"\" {\n\t\t\t\tfeatures.Enable(testCase.features)\n\t\t\t}\n\t\t\terr := h.Refresh()\n\t\t\tassert.Equal(t, testCase.expectedErr, err)\n\t\t})\n\t}\n}\n\nfunc TestHelper_refreshServerPreferredResources(t *testing.T) {\n\tapiList := []*metav1.APIResourceList{\n\t\t{\n\t\t\tGroupVersion: \"v1\",\n\t\t\tAPIResources: []metav1.APIResource{\n\t\t\t\t{\n\t\t\t\t\tName: \"pods\",\n\t\t\t\t\tKind: \"Pod\",\n\t\t\t\t\tGroup: \"\",\n\t\t\t\t\tVersion: \"v1\",\n\t\t\t\t\tVerbs: []string{\"create\", \"get\", \"list\"},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\texpectedErr error\n\t}{\n\t\t{\n\t\t\tname: \"success get preferred resources\",\n\t\t\texpectedErr: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"failed to get preferred resources\",\n\t\t\texpectedErr: errors.New(\"Failed to discover preferred resources\"),\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfakeClient := velerotest.NewFakeServerResourcesInterface(apiList, []*metav1.APIGroup{}, map[schema.GroupVersion]error{}, tc.expectedErr)\n\n\t\t\tresources, err := refreshServerPreferredResources(fakeClient, logrus.New())\n\n\t\t\tif tc.expectedErr != nil {\n\t\t\t\tassert.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.NotNil(t, resources)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestHelper_refreshServerGroupsAndResources(t *testing.T) {\n\tapiList := []*metav1.APIResourceList{\n\t\t{\n\t\t\tGroupVersion: \"v1\",\n\t\t\tAPIResources: []metav1.APIResource{\n\t\t\t\t{\n\t\t\t\t\tName: \"pods\",\n\t\t\t\t\tKind: \"Pod\",\n\t\t\t\t\tGroup: \"\",\n\t\t\t\t\tVersion: \"v1\",\n\t\t\t\t\tVerbs: []string{\"create\", \"get\", \"list\"},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tapiGroup := []*metav1.APIGroup{\n\t\t{\n\t\t\tName: \"group1\",\n\t\t\tVersions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t{\n\t\t\t\t\tGroupVersion: \"v1\",\n\t\t\t\t\tVersion: \"v1\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\ttests := []struct {\n\t\tname string\n\t\texpectedErr error\n\t}{\n\t\t{\n\t\t\tname: \"success get service groups and resouorces\",\n\t\t},\n\t\t{\n\t\t\tname: \"failed to service groups and resouorces\",\n\t\t\texpectedErr: errors.New(\"Failed to discover service groups and resouorces\"),\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfakeClient := velerotest.NewFakeServerResourcesInterface(apiList, apiGroup, map[schema.GroupVersion]error{}, tc.expectedErr)\n\n\t\t\tserverGroups, serverResources, err := refreshServerGroupsAndResources(fakeClient, logrus.New())\n\n\t\t\tif tc.expectedErr != nil {\n\t\t\t\tassert.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.NotNil(t, serverGroups)\n\t\t\t\tassert.NotNil(t, serverResources)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestHelper(t *testing.T) {\n\tfakeDiscoveryClient := &fake.FakeDiscovery{\n\t\tFake: &clientgotesting.Fake{},\n\t}\n\th, err := NewHelper(&velerotest.DiscoveryClient{\n\t\tFakeDiscovery: fakeDiscoveryClient,\n\t}, logrus.New())\n\trequire.NoError(t, err)\n\t// All below calls put together for the implementation are empty or just very simple, and just want to cover testing\n\t// If wanting to write unit tests for some functions could remove it and with writing new function alone\n\th.Resources()\n\th.APIGroups()\n\th.ServerVersion()\n}\n" }, { "path": "pkg/discovery/mocks/Helper.go", "content": "// Code generated by mockery v2.20.0. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tmock \"github.com/stretchr/testify/mock\"\n\tschema \"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tversion \"k8s.io/apimachinery/pkg/version\"\n)\n\n// Helper is an autogenerated mock type for the Helper type\ntype Helper struct {\n\tmock.Mock\n}\n\n// APIGroups provides a mock function with given fields:\nfunc (_m *Helper) APIGroups() []metav1.APIGroup {\n\tret := _m.Called()\n\n\tvar r0 []metav1.APIGroup\n\tif rf, ok := ret.Get(0).(func() []metav1.APIGroup); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]metav1.APIGroup)\n\t\t}\n\t}\n\n\treturn r0\n}\n\n// KindFor provides a mock function with given fields: input\nfunc (_m *Helper) KindFor(input schema.GroupVersionKind) (schema.GroupVersionResource, metav1.APIResource, error) {\n\tret := _m.Called(input)\n\n\tvar r0 schema.GroupVersionResource\n\tvar r1 metav1.APIResource\n\tvar r2 error\n\tif rf, ok := ret.Get(0).(func(schema.GroupVersionKind) (schema.GroupVersionResource, metav1.APIResource, error)); ok {\n\t\treturn rf(input)\n\t}\n\tif rf, ok := ret.Get(0).(func(schema.GroupVersionKind) schema.GroupVersionResource); ok {\n\t\tr0 = rf(input)\n\t} else {\n\t\tr0 = ret.Get(0).(schema.GroupVersionResource)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(schema.GroupVersionKind) metav1.APIResource); ok {\n\t\tr1 = rf(input)\n\t} else {\n\t\tr1 = ret.Get(1).(metav1.APIResource)\n\t}\n\n\tif rf, ok := ret.Get(2).(func(schema.GroupVersionKind) error); ok {\n\t\tr2 = rf(input)\n\t} else {\n\t\tr2 = ret.Error(2)\n\t}\n\n\treturn r0, r1, r2\n}\n\n// Refresh provides a mock function with given fields:\nfunc (_m *Helper) Refresh() error {\n\tret := _m.Called()\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func() error); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// ResourceFor provides a mock function with given fields: input\nfunc (_m *Helper) ResourceFor(input schema.GroupVersionResource) (schema.GroupVersionResource, metav1.APIResource, error) {\n\tret := _m.Called(input)\n\n\tvar r0 schema.GroupVersionResource\n\tvar r1 metav1.APIResource\n\tvar r2 error\n\tif rf, ok := ret.Get(0).(func(schema.GroupVersionResource) (schema.GroupVersionResource, metav1.APIResource, error)); ok {\n\t\treturn rf(input)\n\t}\n\tif rf, ok := ret.Get(0).(func(schema.GroupVersionResource) schema.GroupVersionResource); ok {\n\t\tr0 = rf(input)\n\t} else {\n\t\tr0 = ret.Get(0).(schema.GroupVersionResource)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(schema.GroupVersionResource) metav1.APIResource); ok {\n\t\tr1 = rf(input)\n\t} else {\n\t\tr1 = ret.Get(1).(metav1.APIResource)\n\t}\n\n\tif rf, ok := ret.Get(2).(func(schema.GroupVersionResource) error); ok {\n\t\tr2 = rf(input)\n\t} else {\n\t\tr2 = ret.Error(2)\n\t}\n\n\treturn r0, r1, r2\n}\n\n// Resources provides a mock function with given fields:\nfunc (_m *Helper) Resources() []*metav1.APIResourceList {\n\tret := _m.Called()\n\n\tvar r0 []*metav1.APIResourceList\n\tif rf, ok := ret.Get(0).(func() []*metav1.APIResourceList); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]*metav1.APIResourceList)\n\t\t}\n\t}\n\n\treturn r0\n}\n\n// ServerVersion provides a mock function with given fields:\nfunc (_m *Helper) ServerVersion() *version.Info {\n\tret := _m.Called()\n\n\tvar r0 *version.Info\n\tif rf, ok := ret.Get(0).(func() *version.Info); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(*version.Info)\n\t\t}\n\t}\n\n\treturn r0\n}\n\ntype mockConstructorTestingTNewHelper interface {\n\tmock.TestingT\n\tCleanup(func())\n}\n\n// NewHelper creates a new instance of Helper. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\nfunc NewHelper(t mockConstructorTestingTNewHelper) *Helper {\n\tmock := &Helper{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/exposer/cache_volume.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage exposer\n\nimport (\n\t\"context\"\n\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/api/resource\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\tcorev1client \"k8s.io/client-go/kubernetes/typed/core/v1\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\ntype CacheConfigs struct {\n\tLimit int64\n\tStorageClass string\n\tResidentThreshold int64\n}\n\nconst (\n\tcacheVolumeName = \"cachedir\"\n\tcacheVolumeDirSuffix = \"-cache\"\n)\n\nfunc createCachePVC(ctx context.Context, pvcClient corev1client.CoreV1Interface, ownerObject corev1api.ObjectReference, sc string, size int64, selectedNode string) (*corev1api.PersistentVolumeClaim, error) {\n\tcachePVCName := getCachePVCName(ownerObject)\n\n\tvolumeMode := corev1api.PersistentVolumeFilesystem\n\n\tpvcObj := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: ownerObject.Namespace,\n\t\t\tName: cachePVCName,\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: ownerObject.APIVersion,\n\t\t\t\t\tKind: ownerObject.Kind,\n\t\t\t\t\tName: ownerObject.Name,\n\t\t\t\t\tUID: ownerObject.UID,\n\t\t\t\t\tController: boolptr.True(),\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tAccessModes: []corev1api.PersistentVolumeAccessMode{corev1api.ReadWriteOnce},\n\t\t\tStorageClassName: &sc,\n\t\t\tVolumeMode: &volumeMode,\n\t\t\tResources: corev1api.VolumeResourceRequirements{\n\t\t\t\tRequests: corev1api.ResourceList{\n\t\t\t\t\tcorev1api.ResourceStorage: *resource.NewQuantity(size, resource.BinarySI),\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tif selectedNode != \"\" {\n\t\tpvcObj.Annotations = map[string]string{\n\t\t\tkube.KubeAnnSelectedNode: selectedNode,\n\t\t}\n\t}\n\n\treturn pvcClient.PersistentVolumeClaims(pvcObj.Namespace).Create(ctx, pvcObj, metav1.CreateOptions{})\n}\n\nfunc getCachePVCName(ownerObject corev1api.ObjectReference) string {\n\treturn ownerObject.Name + cacheVolumeDirSuffix\n}\n\nfunc getCacheVolumeSize(dataSize int64, info *CacheConfigs) int64 {\n\tif info == nil {\n\t\treturn 0\n\t}\n\n\tif dataSize != 0 && dataSize < info.ResidentThreshold {\n\t\treturn 0\n\t}\n\n\t// 20% inflate and round up to GB\n\tvolumeSize := (info.Limit*12/10 + (1 << 30) - 1) / (1 << 30) * (1 << 30)\n\n\treturn volumeSize\n}\n" }, { "path": "pkg/exposer/cache_volume_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage exposer\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestGetCacheVolumeSize(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdataSize int64\n\t\tinfo *CacheConfigs\n\t\texpected int64\n\t}{\n\t\t{\n\t\t\tname: \"nil info\",\n\t\t\tdataSize: 1024,\n\t\t\texpected: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"0 data size\",\n\t\t\tinfo: &CacheConfigs{Limit: 1 << 30, ResidentThreshold: 5120},\n\t\t\texpected: 2 << 30,\n\t\t},\n\t\t{\n\t\t\tname: \"0 threshold\",\n\t\t\tdataSize: 2048,\n\t\t\tinfo: &CacheConfigs{Limit: 1 << 30},\n\t\t\texpected: 2 << 30,\n\t\t},\n\t\t{\n\t\t\tname: \"data size is smaller\",\n\t\t\tdataSize: 2048,\n\t\t\tinfo: &CacheConfigs{Limit: 1 << 30, ResidentThreshold: 5120},\n\t\t\texpected: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"data size is lager\",\n\t\t\tdataSize: 2048,\n\t\t\tinfo: &CacheConfigs{Limit: 1 << 30, ResidentThreshold: 1024},\n\t\t\texpected: 2 << 30,\n\t\t},\n\t\t{\n\t\t\tname: \"limit smaller than 1G\",\n\t\t\tdataSize: 2048,\n\t\t\tinfo: &CacheConfigs{Limit: 5120, ResidentThreshold: 1024},\n\t\t\texpected: 1 << 30,\n\t\t},\n\t\t{\n\t\t\tname: \"larger than 1G after inflate\",\n\t\t\tdataSize: 2048,\n\t\t\tinfo: &CacheConfigs{Limit: (1 << 30) - 1024, ResidentThreshold: 1024},\n\t\t\texpected: 2 << 30,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tsize := getCacheVolumeSize(test.dataSize, test.info)\n\t\t\trequire.Equal(t, test.expected, size)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/exposer/csi_snapshot.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage exposer\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\tsnapshotter \"github.com/kubernetes-csi/external-snapshotter/client/v8/clientset/versioned/typed/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\t\"k8s.io/apimachinery/pkg/api/resource\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/nodeagent\"\n\tvelerotypes \"github.com/vmware-tanzu/velero/pkg/types\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/csi\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\n// CSISnapshotExposeParam define the input param for Expose of CSI snapshots\ntype CSISnapshotExposeParam struct {\n\t// SnapshotName is the original volume snapshot name\n\tSnapshotName string\n\n\t// SourceNamespace is the original namespace of the volume that the snapshot is taken for\n\tSourceNamespace string\n\n\t// SourcePVCName is the original name of the PVC that the snapshot is taken for\n\tSourcePVCName string\n\n\t// SourcePVName is the name of PV for SourcePVC\n\tSourcePVName string\n\n\t// AccessMode defines the mode to access the snapshot\n\tAccessMode string\n\n\t// StorageClass is the storage class of the volume that the snapshot is taken for\n\tStorageClass string\n\n\t// HostingPodLabels is the labels that are going to apply to the hosting pod\n\tHostingPodLabels map[string]string\n\n\t// HostingPodAnnotations is the annotations that are going to apply to the hosting pod\n\tHostingPodAnnotations map[string]string\n\n\t// HostingPodTolerations is the tolerations that are going to apply to the hosting pod\n\tHostingPodTolerations []corev1api.Toleration\n\n\t// OperationTimeout specifies the time wait for resources operations in Expose\n\tOperationTimeout time.Duration\n\n\t// ExposeTimeout specifies the timeout for the entire expose process\n\tExposeTimeout time.Duration\n\n\t// VolumeSize specifies the size of the source volume\n\tVolumeSize resource.Quantity\n\n\t// Affinity specifies the node affinity of the backup pod\n\tAffinity []*kube.LoadAffinity\n\n\t// BackupPVCConfig is the config for backupPVC (intermediate PVC) of snapshot data movement\n\tBackupPVCConfig map[string]velerotypes.BackupPVC\n\n\t// Resources defines the resource requirements of the hosting pod\n\tResources corev1api.ResourceRequirements\n\n\t// NodeOS specifies the OS of node that the source volume is attaching\n\tNodeOS string\n\n\t// PriorityClassName is the priority class name for the data mover pod\n\tPriorityClassName string\n}\n\n// CSISnapshotExposeWaitParam define the input param for WaitExposed of CSI snapshots\ntype CSISnapshotExposeWaitParam struct {\n\t// NodeClient is the client that is used to find the hosting pod\n\tNodeClient client.Client\n\tNodeName string\n}\n\n// NewCSISnapshotExposer create a new instance of CSI snapshot exposer\nfunc NewCSISnapshotExposer(kubeClient kubernetes.Interface, csiSnapshotClient snapshotter.SnapshotV1Interface, log logrus.FieldLogger) SnapshotExposer {\n\treturn &csiSnapshotExposer{\n\t\tkubeClient: kubeClient,\n\t\tcsiSnapshotClient: csiSnapshotClient,\n\t\tlog: log,\n\t}\n}\n\ntype csiSnapshotExposer struct {\n\tkubeClient kubernetes.Interface\n\tcsiSnapshotClient snapshotter.SnapshotV1Interface\n\tlog logrus.FieldLogger\n}\n\nfunc (e *csiSnapshotExposer) Expose(ctx context.Context, ownerObject corev1api.ObjectReference, param any) error {\n\tcsiExposeParam := param.(*CSISnapshotExposeParam)\n\n\tcurLog := e.log.WithFields(logrus.Fields{\n\t\t\"owner\": ownerObject.Name,\n\t})\n\n\tvolumeTopology, err := kube.GetVolumeTopology(ctx, e.kubeClient.CoreV1(), e.kubeClient.StorageV1(), csiExposeParam.SourcePVName, csiExposeParam.StorageClass)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error getting volume topology for PV %s, storage class %s\", csiExposeParam.SourcePVName, csiExposeParam.StorageClass)\n\t}\n\n\tif volumeTopology != nil {\n\t\tcurLog.Infof(\"Using volume topology %v\", volumeTopology)\n\t}\n\n\tcurLog.Info(\"Exposing CSI snapshot\")\n\n\tvolumeSnapshot, err := csi.WaitVolumeSnapshotReady(ctx, e.csiSnapshotClient, csiExposeParam.SnapshotName, csiExposeParam.SourceNamespace, csiExposeParam.ExposeTimeout, curLog)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error wait volume snapshot ready\")\n\t}\n\n\tcurLog.Info(\"Volumesnapshot is ready\")\n\n\tvsc, err := csi.GetVolumeSnapshotContentForVolumeSnapshot(volumeSnapshot, e.csiSnapshotClient)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to get volume snapshot content\")\n\t}\n\n\tcurLog.WithField(\"vsc name\", vsc.Name).WithField(\"vs name\", volumeSnapshot.Name).Infof(\"Got VSC from VS in namespace %s\", volumeSnapshot.Namespace)\n\n\tbackupVS, err := e.createBackupVS(ctx, ownerObject, volumeSnapshot)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to create backup volume snapshot\")\n\t}\n\n\tcurLog.WithField(\"vs name\", backupVS.Name).Infof(\"Backup VS is created from %s/%s\", volumeSnapshot.Namespace, volumeSnapshot.Name)\n\n\tdefer func() {\n\t\tif err != nil {\n\t\t\tcsi.DeleteVolumeSnapshotIfAny(ctx, e.csiSnapshotClient, backupVS.Name, backupVS.Namespace, curLog)\n\t\t}\n\t}()\n\n\tbackupVSC, err := e.createBackupVSC(ctx, ownerObject, vsc, backupVS)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to create backup volume snapshot content\")\n\t}\n\n\tcurLog.WithField(\"vsc name\", backupVSC.Name).Infof(\"Backup VSC is created from %s\", vsc.Name)\n\n\tretained, err := csi.RetainVSC(ctx, e.csiSnapshotClient, vsc)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to retain volume snapshot content\")\n\t}\n\n\tcurLog.WithField(\"vsc name\", vsc.Name).WithField(\"retained\", (retained != nil)).Info(\"Finished to retain VSC\")\n\n\terr = csi.EnsureDeleteVS(ctx, e.csiSnapshotClient, volumeSnapshot.Name, volumeSnapshot.Namespace, csiExposeParam.OperationTimeout)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to delete volume snapshot\")\n\t}\n\n\tcurLog.WithField(\"vs name\", volumeSnapshot.Name).Infof(\"VS is deleted in namespace %s\", volumeSnapshot.Namespace)\n\n\terr = csi.EnsureDeleteVSC(ctx, e.csiSnapshotClient, vsc.Name, csiExposeParam.OperationTimeout)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to delete volume snapshot content\")\n\t}\n\n\tcurLog.WithField(\"vsc name\", vsc.Name).Infof(\"VSC is deleted\")\n\n\tvar volumeSize resource.Quantity\n\tif volumeSnapshot.Status.RestoreSize != nil && !volumeSnapshot.Status.RestoreSize.IsZero() {\n\t\tvolumeSize = *volumeSnapshot.Status.RestoreSize\n\t} else {\n\t\tvolumeSize = csiExposeParam.VolumeSize\n\t\tcurLog.WithField(\"vs name\", volumeSnapshot.Name).Warnf(\"The snapshot doesn't contain a valid restore size, use source volume's size %v\", volumeSize)\n\t}\n\n\t// check if there is a mapping for source pvc storage class in backupPVC config\n\t// if the mapping exists then use the values(storage class, readOnly accessMode)\n\t// for backupPVC (intermediate PVC in snapshot data movement) object creation\n\tbackupPVCStorageClass := csiExposeParam.StorageClass\n\tbackupPVCReadOnly := false\n\tspcNoRelabeling := false\n\tbackupPVCAnnotations := map[string]string{}\n\tintoleratableNodes := []string{}\n\tif value, exists := csiExposeParam.BackupPVCConfig[csiExposeParam.StorageClass]; exists {\n\t\tif value.StorageClass != \"\" {\n\t\t\tbackupPVCStorageClass = value.StorageClass\n\t\t}\n\n\t\tbackupPVCReadOnly = value.ReadOnly\n\t\tif value.SPCNoRelabeling {\n\t\t\tif backupPVCReadOnly {\n\t\t\t\tspcNoRelabeling = true\n\t\t\t} else {\n\t\t\t\tcurLog.WithField(\"vs name\", volumeSnapshot.Name).Warn(\"Ignoring spcNoRelabling for read-write volume\")\n\t\t\t}\n\t\t}\n\n\t\tif len(value.Annotations) > 0 {\n\t\t\tbackupPVCAnnotations = value.Annotations\n\t\t}\n\n\t\tif _, found := backupPVCAnnotations[util.VSphereCNSFastCloneAnno]; found {\n\t\t\tif n, err := kube.GetPVAttachedNodes(ctx, csiExposeParam.SourcePVName, e.kubeClient.StorageV1()); err != nil {\n\t\t\t\tcurLog.WithField(\"source PV\", csiExposeParam.SourcePVName).WithError(err).Warnf(\"Failed to get attached node for source PV, ignore %s annotation\", util.VSphereCNSFastCloneAnno)\n\t\t\t\tdelete(backupPVCAnnotations, util.VSphereCNSFastCloneAnno)\n\t\t\t} else {\n\t\t\t\tintoleratableNodes = n\n\t\t\t}\n\t\t}\n\t}\n\n\tbackupPVC, err := e.createBackupPVC(ctx, ownerObject, backupVS.Name, backupPVCStorageClass, csiExposeParam.AccessMode, volumeSize, backupPVCReadOnly, backupPVCAnnotations)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to create backup pvc\")\n\t}\n\n\tcurLog.WithField(\"pvc name\", backupPVC.Name).Info(\"Backup PVC is created\")\n\tdefer func() {\n\t\tif err != nil {\n\t\t\tkube.DeletePVAndPVCIfAny(ctx, e.kubeClient.CoreV1(), backupPVC.Name, backupPVC.Namespace, 0, curLog)\n\t\t}\n\t}()\n\n\taffinity := kube.GetLoadAffinityByStorageClass(csiExposeParam.Affinity, backupPVCStorageClass, curLog)\n\n\tbackupPod, err := e.createBackupPod(\n\t\tctx,\n\t\townerObject,\n\t\tbackupPVC,\n\t\tcsiExposeParam.OperationTimeout,\n\t\tcsiExposeParam.HostingPodLabels,\n\t\tcsiExposeParam.HostingPodAnnotations,\n\t\tcsiExposeParam.HostingPodTolerations,\n\t\taffinity,\n\t\tcsiExposeParam.Resources,\n\t\tbackupPVCReadOnly,\n\t\tspcNoRelabeling,\n\t\tcsiExposeParam.NodeOS,\n\t\tcsiExposeParam.PriorityClassName,\n\t\tintoleratableNodes,\n\t\tvolumeTopology,\n\t)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to create backup pod\")\n\t}\n\n\tcurLog.WithField(\"pod name\", backupPod.Name).WithField(\"affinity\", affinity).Info(\"Backup pod is created\")\n\n\tdefer func() {\n\t\tif err != nil {\n\t\t\tkube.DeletePodIfAny(ctx, e.kubeClient.CoreV1(), backupPod.Name, backupPod.Namespace, curLog)\n\t\t}\n\t}()\n\n\treturn nil\n}\n\nfunc (e *csiSnapshotExposer) GetExposed(ctx context.Context, ownerObject corev1api.ObjectReference, timeout time.Duration, param any) (*ExposeResult, error) {\n\texposeWaitParam := param.(*CSISnapshotExposeWaitParam)\n\n\tbackupPodName := ownerObject.Name\n\tbackupPVCName := ownerObject.Name\n\n\tcontainerName := string(ownerObject.UID)\n\tvolumeName := string(ownerObject.UID)\n\n\tcurLog := e.log.WithFields(logrus.Fields{\n\t\t\"owner\": ownerObject.Name,\n\t})\n\n\tpod := &corev1api.Pod{}\n\terr := exposeWaitParam.NodeClient.Get(ctx, types.NamespacedName{\n\t\tNamespace: ownerObject.Namespace,\n\t\tName: backupPodName,\n\t}, pod)\n\tif err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tcurLog.WithField(\"backup pod\", backupPodName).Debugf(\"Backup pod is not running in the current node %s\", exposeWaitParam.NodeName)\n\t\t\treturn nil, nil\n\t\t} else {\n\t\t\treturn nil, errors.Wrapf(err, \"error to get backup pod %s\", backupPodName)\n\t\t}\n\t}\n\n\tcurLog.WithField(\"pod\", pod.Name).Infof(\"Backup pod is in running state in node %s\", pod.Spec.NodeName)\n\n\t_, err = kube.WaitPVCBound(ctx, e.kubeClient.CoreV1(), e.kubeClient.CoreV1(), backupPVCName, ownerObject.Namespace, timeout)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error to wait backup PVC bound, %s\", backupPVCName)\n\t}\n\n\tcurLog.WithField(\"backup pvc\", backupPVCName).Info(\"Backup PVC is bound\")\n\n\ti := 0\n\tfor i = 0; i < len(pod.Spec.Volumes); i++ {\n\t\tif pod.Spec.Volumes[i].Name == volumeName {\n\t\t\tbreak\n\t\t}\n\t}\n\n\tif i == len(pod.Spec.Volumes) {\n\t\treturn nil, errors.Errorf(\"backup pod %s doesn't have the expected backup volume\", pod.Name)\n\t}\n\n\tcurLog.WithField(\"pod\", pod.Name).Infof(\"Backup volume is found in pod at index %v\", i)\n\n\tvar nodeOS *string\n\tif pod.Spec.OS != nil {\n\t\tos := string(pod.Spec.OS.Name)\n\t\tnodeOS = &os\n\t}\n\n\treturn &ExposeResult{ByPod: ExposeByPod{\n\t\tHostingPod: pod,\n\t\tHostingContainer: containerName,\n\t\tVolumeName: volumeName,\n\t\tNodeOS: nodeOS,\n\t}}, nil\n}\n\nfunc (e *csiSnapshotExposer) PeekExposed(ctx context.Context, ownerObject corev1api.ObjectReference) error {\n\tbackupPodName := ownerObject.Name\n\n\tcurLog := e.log.WithFields(logrus.Fields{\n\t\t\"owner\": ownerObject.Name,\n\t})\n\n\tpod, err := e.kubeClient.CoreV1().Pods(ownerObject.Namespace).Get(ctx, backupPodName, metav1.GetOptions{})\n\tif apierrors.IsNotFound(err) {\n\t\treturn nil\n\t}\n\n\tif err != nil {\n\t\tcurLog.WithError(err).Warnf(\"error to peek backup pod %s\", backupPodName)\n\t\treturn nil\n\t}\n\n\tif podFailed, message := kube.IsPodUnrecoverable(pod, curLog); podFailed {\n\t\treturn errors.New(message)\n\t}\n\n\treturn nil\n}\n\nfunc (e *csiSnapshotExposer) DiagnoseExpose(ctx context.Context, ownerObject corev1api.ObjectReference) string {\n\tbackupPodName := ownerObject.Name\n\tbackupPVCName := ownerObject.Name\n\tbackupVSName := ownerObject.Name\n\n\tdiag := \"begin diagnose CSI exposer\\n\"\n\n\tpod, err := e.kubeClient.CoreV1().Pods(ownerObject.Namespace).Get(ctx, backupPodName, metav1.GetOptions{})\n\tif err != nil {\n\t\tpod = nil\n\t\tdiag += fmt.Sprintf(\"error getting backup pod %s, err: %v\\n\", backupPodName, err)\n\t}\n\n\tpvc, err := e.kubeClient.CoreV1().PersistentVolumeClaims(ownerObject.Namespace).Get(ctx, backupPVCName, metav1.GetOptions{})\n\tif err != nil {\n\t\tpvc = nil\n\t\tdiag += fmt.Sprintf(\"error getting backup pvc %s, err: %v\\n\", backupPVCName, err)\n\t}\n\n\tvs, err := e.csiSnapshotClient.VolumeSnapshots(ownerObject.Namespace).Get(ctx, backupVSName, metav1.GetOptions{})\n\tif err != nil {\n\t\tvs = nil\n\t\tdiag += fmt.Sprintf(\"error getting backup vs %s, err: %v\\n\", backupVSName, err)\n\t}\n\n\tevents, err := e.kubeClient.CoreV1().Events(ownerObject.Namespace).List(ctx, metav1.ListOptions{})\n\tif err != nil {\n\t\tdiag += fmt.Sprintf(\"error listing events, err: %v\\n\", err)\n\t}\n\n\tif pod != nil {\n\t\tdiag += kube.DiagnosePod(pod, events)\n\n\t\tif pod.Spec.NodeName != \"\" {\n\t\t\tif err := nodeagent.KbClientIsRunningInNode(ctx, ownerObject.Namespace, pod.Spec.NodeName, e.kubeClient); err != nil {\n\t\t\t\tdiag += fmt.Sprintf(\"node-agent is not running in node %s, err: %v\\n\", pod.Spec.NodeName, err)\n\t\t\t}\n\t\t}\n\t}\n\n\tif pvc != nil {\n\t\tdiag += kube.DiagnosePVC(pvc, events)\n\n\t\tif pvc.Spec.VolumeName != \"\" {\n\t\t\tif pv, err := e.kubeClient.CoreV1().PersistentVolumes().Get(ctx, pvc.Spec.VolumeName, metav1.GetOptions{}); err != nil {\n\t\t\t\tdiag += fmt.Sprintf(\"error getting backup pv %s, err: %v\\n\", pvc.Spec.VolumeName, err)\n\t\t\t} else {\n\t\t\t\tdiag += kube.DiagnosePV(pv)\n\t\t\t}\n\t\t}\n\t}\n\n\tif vs != nil {\n\t\tdiag += csi.DiagnoseVS(vs, events)\n\n\t\tif vs.Status != nil && vs.Status.BoundVolumeSnapshotContentName != nil && *vs.Status.BoundVolumeSnapshotContentName != \"\" {\n\t\t\tif vsc, err := e.csiSnapshotClient.VolumeSnapshotContents().Get(ctx, *vs.Status.BoundVolumeSnapshotContentName, metav1.GetOptions{}); err != nil {\n\t\t\t\tdiag += fmt.Sprintf(\"error getting backup vsc %s, err: %v\\n\", *vs.Status.BoundVolumeSnapshotContentName, err)\n\t\t\t} else {\n\t\t\t\tdiag += csi.DiagnoseVSC(vsc)\n\t\t\t}\n\t\t}\n\t}\n\n\tdiag += \"end diagnose CSI exposer\"\n\n\treturn diag\n}\n\nconst cleanUpTimeout = time.Minute\n\nfunc (e *csiSnapshotExposer) CleanUp(ctx context.Context, ownerObject corev1api.ObjectReference, vsName string, sourceNamespace string) {\n\tbackupPodName := ownerObject.Name\n\tbackupPVCName := ownerObject.Name\n\tbackupVSName := ownerObject.Name\n\n\tkube.DeletePodIfAny(ctx, e.kubeClient.CoreV1(), backupPodName, ownerObject.Namespace, e.log)\n\tkube.DeletePVAndPVCIfAny(ctx, e.kubeClient.CoreV1(), backupPVCName, ownerObject.Namespace, cleanUpTimeout, e.log)\n\n\tcsi.DeleteVolumeSnapshotIfAny(ctx, e.csiSnapshotClient, backupVSName, ownerObject.Namespace, e.log)\n\tcsi.DeleteVolumeSnapshotIfAny(ctx, e.csiSnapshotClient, vsName, sourceNamespace, e.log)\n}\n\nfunc getVolumeModeByAccessMode(accessMode string) (corev1api.PersistentVolumeMode, error) {\n\tswitch accessMode {\n\tcase AccessModeFileSystem:\n\t\treturn corev1api.PersistentVolumeFilesystem, nil\n\tcase AccessModeBlock:\n\t\treturn corev1api.PersistentVolumeBlock, nil\n\tdefault:\n\t\treturn \"\", errors.Errorf(\"unsupported access mode %s\", accessMode)\n\t}\n}\n\nfunc (e *csiSnapshotExposer) createBackupVS(ctx context.Context, ownerObject corev1api.ObjectReference, snapshotVS *snapshotv1api.VolumeSnapshot) (*snapshotv1api.VolumeSnapshot, error) {\n\tbackupVSName := ownerObject.Name\n\tbackupVSCName := ownerObject.Name\n\n\tvs := &snapshotv1api.VolumeSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: backupVSName,\n\t\t\tNamespace: ownerObject.Namespace,\n\t\t\tAnnotations: snapshotVS.Annotations,\n\t\t\t// Don't add ownerReference to SnapshotBackup.\n\t\t\t// The backupPVC should be deleted before backupVS, otherwise, the deletion of backupVS will fail since\n\t\t\t// backupPVC has its dataSource referring to it\n\t\t},\n\t\tSpec: snapshotv1api.VolumeSnapshotSpec{\n\t\t\tSource: snapshotv1api.VolumeSnapshotSource{\n\t\t\t\tVolumeSnapshotContentName: &backupVSCName,\n\t\t\t},\n\t\t\tVolumeSnapshotClassName: snapshotVS.Spec.VolumeSnapshotClassName,\n\t\t},\n\t}\n\n\treturn e.csiSnapshotClient.VolumeSnapshots(vs.Namespace).Create(ctx, vs, metav1.CreateOptions{})\n}\n\nfunc (e *csiSnapshotExposer) createBackupVSC(ctx context.Context, ownerObject corev1api.ObjectReference, snapshotVSC *snapshotv1api.VolumeSnapshotContent, vs *snapshotv1api.VolumeSnapshot) (*snapshotv1api.VolumeSnapshotContent, error) {\n\tbackupVSCName := ownerObject.Name\n\n\tvsc := &snapshotv1api.VolumeSnapshotContent{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: backupVSCName,\n\t\t\tAnnotations: snapshotVSC.Annotations,\n\t\t\tLabels: map[string]string{},\n\t\t},\n\t\tSpec: snapshotv1api.VolumeSnapshotContentSpec{\n\t\t\tVolumeSnapshotRef: corev1api.ObjectReference{\n\t\t\t\tName: vs.Name,\n\t\t\t\tNamespace: vs.Namespace,\n\t\t\t\tUID: vs.UID,\n\t\t\t\tResourceVersion: vs.ResourceVersion,\n\t\t\t},\n\t\t\tSource: snapshotv1api.VolumeSnapshotContentSource{\n\t\t\t\tSnapshotHandle: snapshotVSC.Status.SnapshotHandle,\n\t\t\t},\n\t\t\tDeletionPolicy: snapshotv1api.VolumeSnapshotContentDelete,\n\t\t\tDriver: snapshotVSC.Spec.Driver,\n\t\t\tVolumeSnapshotClassName: snapshotVSC.Spec.VolumeSnapshotClassName,\n\t\t},\n\t}\n\n\t/*\n\t\tWe need to keep the label of the managing node for distributed snapshots.\n\t\tThe external snapshot manager will only manage snapshots matching it's node if that feature is enabled.\n\n\t\thttps://github.com/kubernetes-csi/external-snapshotter/tree/4cedb3f45790ac593ebfa3324c490abedf739477?tab=readme-ov-file#distributed-snapshotting\n\t\thttps://github.com/kubernetes-csi/external-snapshotter/blob/4cedb3f45790ac593ebfa3324c490abedf739477/pkg/utils/util.go#L158\n\t*/\n\tif manager, ok := snapshotVSC.Labels[kube.VolumeSnapshotContentManagedByLabel]; ok {\n\t\tvsc.ObjectMeta.Labels[kube.VolumeSnapshotContentManagedByLabel] = manager\n\t}\n\n\treturn e.csiSnapshotClient.VolumeSnapshotContents().Create(ctx, vsc, metav1.CreateOptions{})\n}\n\nfunc (e *csiSnapshotExposer) createBackupPVC(ctx context.Context, ownerObject corev1api.ObjectReference, backupVS, storageClass, accessMode string, resource resource.Quantity, readOnly bool, annotations map[string]string) (*corev1api.PersistentVolumeClaim, error) {\n\tbackupPVCName := ownerObject.Name\n\n\tvolumeMode, err := getVolumeModeByAccessMode(accessMode)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tpvcAccessMode := corev1api.ReadWriteOnce\n\n\tif readOnly {\n\t\tpvcAccessMode = corev1api.ReadOnlyMany\n\t}\n\n\tdataSource := &corev1api.TypedLocalObjectReference{\n\t\tAPIGroup: &snapshotv1api.SchemeGroupVersion.Group,\n\t\tKind: \"VolumeSnapshot\",\n\t\tName: backupVS,\n\t}\n\n\tpvc := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: ownerObject.Namespace,\n\t\t\tName: backupPVCName,\n\t\t\tAnnotations: annotations,\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: ownerObject.APIVersion,\n\t\t\t\t\tKind: ownerObject.Kind,\n\t\t\t\t\tName: ownerObject.Name,\n\t\t\t\t\tUID: ownerObject.UID,\n\t\t\t\t\tController: boolptr.True(),\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tAccessModes: []corev1api.PersistentVolumeAccessMode{\n\t\t\t\tpvcAccessMode,\n\t\t\t},\n\t\t\tStorageClassName: &storageClass,\n\t\t\tVolumeMode: &volumeMode,\n\t\t\tDataSource: dataSource,\n\t\t\tDataSourceRef: nil,\n\n\t\t\tResources: corev1api.VolumeResourceRequirements{\n\t\t\t\tRequests: corev1api.ResourceList{\n\t\t\t\t\tcorev1api.ResourceStorage: resource,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tcreated, err := e.kubeClient.CoreV1().PersistentVolumeClaims(pvc.Namespace).Create(ctx, pvc, metav1.CreateOptions{})\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error to create pvc\")\n\t}\n\n\treturn created, err\n}\n\nfunc (e *csiSnapshotExposer) createBackupPod(\n\tctx context.Context,\n\townerObject corev1api.ObjectReference,\n\tbackupPVC *corev1api.PersistentVolumeClaim,\n\toperationTimeout time.Duration,\n\tlabel map[string]string,\n\tannotation map[string]string,\n\ttoleration []corev1api.Toleration,\n\taffinity *kube.LoadAffinity,\n\tresources corev1api.ResourceRequirements,\n\tbackupPVCReadOnly bool,\n\tspcNoRelabeling bool,\n\tnodeOS string,\n\tpriorityClassName string,\n\tintoleratableNodes []string,\n\tvolumeTopology *corev1api.NodeSelector,\n) (*corev1api.Pod, error) {\n\tpodName := ownerObject.Name\n\n\tcontainerName := string(ownerObject.UID)\n\tvolumeName := string(ownerObject.UID)\n\n\tpodInfo, err := getInheritedPodInfo(ctx, e.kubeClient, ownerObject.Namespace, nodeOS)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error to get inherited pod info from node-agent\")\n\t}\n\n\t// Log the priority class if it's set\n\tif priorityClassName != \"\" {\n\t\te.log.Debugf(\"Setting priority class %q for data mover pod %s\", priorityClassName, podName)\n\t}\n\n\tvar gracePeriod int64\n\tvolumeMounts, volumeDevices, volumePath := kube.MakePodPVCAttachment(volumeName, backupPVC.Spec.VolumeMode, backupPVCReadOnly)\n\tvolumeMounts = append(volumeMounts, podInfo.volumeMounts...)\n\n\tvolumes := []corev1api.Volume{{\n\t\tName: volumeName,\n\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\tClaimName: backupPVC.Name,\n\t\t\t},\n\t\t},\n\t}}\n\n\tif backupPVCReadOnly {\n\t\tvolumes[0].VolumeSource.PersistentVolumeClaim.ReadOnly = true\n\t}\n\n\tvolumes = append(volumes, podInfo.volumes...)\n\n\tif label == nil {\n\t\tlabel = make(map[string]string)\n\t}\n\tlabel[podGroupLabel] = podGroupSnapshot\n\n\tvolumeMode := corev1api.PersistentVolumeFilesystem\n\tif backupPVC.Spec.VolumeMode != nil {\n\t\tvolumeMode = *backupPVC.Spec.VolumeMode\n\t}\n\n\targs := []string{\n\t\tfmt.Sprintf(\"--volume-path=%s\", volumePath),\n\t\tfmt.Sprintf(\"--volume-mode=%s\", volumeMode),\n\t\tfmt.Sprintf(\"--data-upload=%s\", ownerObject.Name),\n\t\tfmt.Sprintf(\"--resource-timeout=%s\", operationTimeout.String()),\n\t}\n\n\targs = append(args, podInfo.logFormatArgs...)\n\targs = append(args, podInfo.logLevelArgs...)\n\n\tif affinity == nil {\n\t\taffinity = &kube.LoadAffinity{}\n\t}\n\n\tvar securityCtx *corev1api.PodSecurityContext\n\tnodeSelector := map[string]string{}\n\tpodOS := corev1api.PodOS{}\n\tif nodeOS == kube.NodeOSWindows {\n\t\tuserID := \"ContainerAdministrator\"\n\t\tsecurityCtx = &corev1api.PodSecurityContext{\n\t\t\tWindowsOptions: &corev1api.WindowsSecurityContextOptions{\n\t\t\t\tRunAsUserName: &userID,\n\t\t\t},\n\t\t}\n\n\t\tpodOS.Name = kube.NodeOSWindows\n\n\t\taffinity.NodeSelector.MatchExpressions = append(affinity.NodeSelector.MatchExpressions, metav1.LabelSelectorRequirement{\n\t\t\tKey: kube.NodeOSLabel,\n\t\t\tValues: []string{kube.NodeOSWindows},\n\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t})\n\n\t\ttoleration = append(toleration, []corev1api.Toleration{\n\t\t\t{\n\t\t\t\tKey: \"os\",\n\t\t\t\tOperator: \"Equal\",\n\t\t\t\tEffect: \"NoSchedule\",\n\t\t\t\tValue: \"windows\",\n\t\t\t},\n\t\t\t{\n\t\t\t\tKey: \"os\",\n\t\t\t\tOperator: \"Equal\",\n\t\t\t\tEffect: \"NoExecute\",\n\t\t\t\tValue: \"windows\",\n\t\t\t},\n\t\t}...)\n\t} else {\n\t\tuserID := int64(0)\n\t\tsecurityCtx = &corev1api.PodSecurityContext{\n\t\t\tRunAsUser: &userID,\n\t\t}\n\n\t\tif spcNoRelabeling {\n\t\t\tsecurityCtx.SELinuxOptions = &corev1api.SELinuxOptions{\n\t\t\t\tType: \"spc_t\",\n\t\t\t}\n\t\t}\n\n\t\tpodOS.Name = kube.NodeOSLinux\n\n\t\taffinity.NodeSelector.MatchExpressions = append(affinity.NodeSelector.MatchExpressions, metav1.LabelSelectorRequirement{\n\t\t\tKey: kube.NodeOSLabel,\n\t\t\tValues: []string{kube.NodeOSWindows},\n\t\t\tOperator: metav1.LabelSelectorOpNotIn,\n\t\t})\n\t}\n\n\tif len(intoleratableNodes) > 0 {\n\t\tif affinity == nil {\n\t\t\taffinity = &kube.LoadAffinity{}\n\t\t}\n\n\t\taffinity.NodeSelector.MatchExpressions = append(affinity.NodeSelector.MatchExpressions, metav1.LabelSelectorRequirement{\n\t\t\tKey: \"kubernetes.io/hostname\",\n\t\t\tValues: intoleratableNodes,\n\t\t\tOperator: metav1.LabelSelectorOpNotIn,\n\t\t})\n\t}\n\n\tpodAffinity := kube.ToSystemAffinity(affinity, volumeTopology)\n\n\tpod := &corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: podName,\n\t\t\tNamespace: ownerObject.Namespace,\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: ownerObject.APIVersion,\n\t\t\t\t\tKind: ownerObject.Kind,\n\t\t\t\t\tName: ownerObject.Name,\n\t\t\t\t\tUID: ownerObject.UID,\n\t\t\t\t\tController: boolptr.True(),\n\t\t\t\t},\n\t\t\t},\n\t\t\tLabels: label,\n\t\t\tAnnotations: annotation,\n\t\t},\n\t\tSpec: corev1api.PodSpec{\n\t\t\tTopologySpreadConstraints: []corev1api.TopologySpreadConstraint{\n\t\t\t\t{\n\t\t\t\t\tMaxSkew: 1,\n\t\t\t\t\tTopologyKey: \"kubernetes.io/hostname\",\n\t\t\t\t\tWhenUnsatisfiable: corev1api.ScheduleAnyway,\n\t\t\t\t\tLabelSelector: &metav1.LabelSelector{\n\t\t\t\t\t\tMatchLabels: map[string]string{\n\t\t\t\t\t\t\tpodGroupLabel: podGroupSnapshot,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tNodeSelector: nodeSelector,\n\t\t\tOS: &podOS,\n\t\t\tAffinity: podAffinity,\n\t\t\tContainers: []corev1api.Container{\n\t\t\t\t{\n\t\t\t\t\tName: containerName,\n\t\t\t\t\tImage: podInfo.image,\n\t\t\t\t\tImagePullPolicy: corev1api.PullNever,\n\t\t\t\t\tCommand: []string{\n\t\t\t\t\t\t\"/velero\",\n\t\t\t\t\t\t\"data-mover\",\n\t\t\t\t\t\t\"backup\",\n\t\t\t\t\t},\n\t\t\t\t\tArgs: args,\n\t\t\t\t\tVolumeMounts: volumeMounts,\n\t\t\t\t\tVolumeDevices: volumeDevices,\n\t\t\t\t\tEnv: podInfo.env,\n\t\t\t\t\tEnvFrom: podInfo.envFrom,\n\t\t\t\t\tResources: resources,\n\t\t\t\t},\n\t\t\t},\n\t\t\tPriorityClassName: priorityClassName,\n\t\t\tServiceAccountName: podInfo.serviceAccount,\n\t\t\tTerminationGracePeriodSeconds: &gracePeriod,\n\t\t\tVolumes: volumes,\n\t\t\tRestartPolicy: corev1api.RestartPolicyNever,\n\t\t\tSecurityContext: securityCtx,\n\t\t\tTolerations: toleration,\n\t\t\tDNSPolicy: podInfo.dnsPolicy,\n\t\t\tDNSConfig: podInfo.dnsConfig,\n\t\t\tImagePullSecrets: podInfo.imagePullSecrets,\n\t\t},\n\t}\n\n\treturn e.kubeClient.CoreV1().Pods(ownerObject.Namespace).Create(ctx, pod, metav1.CreateOptions{})\n}\n" }, { "path": "pkg/exposer/csi_snapshot_priority_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage exposer\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\tsnapshotFake \"github.com/kubernetes-csi/external-snapshotter/client/v8/clientset/versioned/fake\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/client-go/kubernetes/fake\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nfunc TestCreateBackupPodWithPriorityClass(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tnodeAgentConfigMapData string\n\t\texpectedPriorityClass string\n\t\tdescription string\n\t}{\n\t\t{\n\t\t\tname: \"with priority class in config map\",\n\t\t\tnodeAgentConfigMapData: `{\n\t\t\t\t\"priorityClassName\": \"high-priority\"\n\t\t\t}`,\n\t\t\texpectedPriorityClass: \"high-priority\",\n\t\t\tdescription: \"Should set priority class from node-agent-configmap\",\n\t\t},\n\t\t{\n\t\t\tname: \"without priority class in config map\",\n\t\t\tnodeAgentConfigMapData: `{\n\t\t\t\t\"loadAffinity\": []\n\t\t\t}`,\n\t\t\texpectedPriorityClass: \"\",\n\t\t\tdescription: \"Should have empty priority class when not specified\",\n\t\t},\n\t\t{\n\t\t\tname: \"empty config map\",\n\t\t\tnodeAgentConfigMapData: `{}`,\n\t\t\texpectedPriorityClass: \"\",\n\t\t\tdescription: \"Should handle empty config map gracefully\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\n\t\t\t// Create fake Kubernetes client\n\t\t\tkubeClient := fake.NewSimpleClientset()\n\n\t\t\t// Create node-agent daemonset (required for getInheritedPodInfo)\n\t\t\tdaemonSet := &appsv1api.DaemonSet{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"node-agent\",\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t},\n\t\t\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"node-agent\",\n\t\t\t\t\t\t\t\t\tImage: \"velero/velero:latest\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t}\n\t\t\t_, err := kubeClient.AppsV1().DaemonSets(velerov1api.DefaultNamespace).Create(ctx, daemonSet, metav1.CreateOptions{})\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Create node-agent config map\n\t\t\tconfigMap := &corev1api.ConfigMap{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"node-agent-config\",\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t},\n\t\t\t\tData: map[string]string{\n\t\t\t\t\t\"config\": tc.nodeAgentConfigMapData,\n\t\t\t\t},\n\t\t\t}\n\t\t\t_, err = kubeClient.CoreV1().ConfigMaps(velerov1api.DefaultNamespace).Create(ctx, configMap, metav1.CreateOptions{})\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Create owner object for the backup pod\n\t\t\townerObject := corev1api.ObjectReference{\n\t\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"DataUpload\",\n\t\t\t\tName: \"test-dataupload\",\n\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\tUID: \"test-uid\",\n\t\t\t}\n\n\t\t\t// Create a backup PVC\n\t\t\tbackupPVC := &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-backup-pvc\",\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\tAccessModes: []corev1api.PersistentVolumeAccessMode{\n\t\t\t\t\t\tcorev1api.ReadWriteOnce,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t}\n\n\t\t\t// Create fake snapshot client\n\t\t\tfakeSnapshotClient := snapshotFake.NewSimpleClientset()\n\n\t\t\t// Create CSI snapshot exposer\n\t\t\texposer := &csiSnapshotExposer{\n\t\t\t\tkubeClient: kubeClient,\n\t\t\t\tcsiSnapshotClient: fakeSnapshotClient.SnapshotV1(),\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\t// Call createBackupPod\n\t\t\tpod, err := exposer.createBackupPod(\n\t\t\t\tctx,\n\t\t\t\townerObject,\n\t\t\t\tbackupPVC,\n\t\t\t\ttime.Minute*5,\n\t\t\t\tnil, // labels\n\t\t\t\tnil, // annotations\n\t\t\t\tnil, // tolerations\n\t\t\t\tnil, // affinity\n\t\t\t\tcorev1api.ResourceRequirements{},\n\t\t\t\tfalse, // backupPVCReadOnly\n\t\t\t\tfalse, // spcNoRelabeling\n\t\t\t\tkube.NodeOSLinux,\n\t\t\t\ttc.expectedPriorityClass,\n\t\t\t\tnil,\n\t\t\t\tnil,\n\t\t\t)\n\n\t\t\trequire.NoError(t, err, tc.description)\n\t\t\tassert.NotNil(t, pod)\n\t\t\tassert.Equal(t, tc.expectedPriorityClass, pod.Spec.PriorityClassName, tc.description)\n\t\t})\n\t}\n}\n\nfunc TestCreateBackupPodWithMissingConfigMap(t *testing.T) {\n\tctx := t.Context()\n\n\t// Create fake Kubernetes client without config map\n\tkubeClient := fake.NewSimpleClientset()\n\n\t// Create node-agent daemonset (required for getInheritedPodInfo)\n\tdaemonSet := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"node-agent\",\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"node-agent\",\n\t\t\t\t\t\t\tImage: \"velero/velero:latest\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\t_, err := kubeClient.AppsV1().DaemonSets(velerov1api.DefaultNamespace).Create(ctx, daemonSet, metav1.CreateOptions{})\n\trequire.NoError(t, err)\n\n\t// Create owner object for the backup pod\n\townerObject := corev1api.ObjectReference{\n\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\tKind: \"DataUpload\",\n\t\tName: \"test-dataupload\",\n\t\tNamespace: velerov1api.DefaultNamespace,\n\t\tUID: \"test-uid\",\n\t}\n\n\t// Create a backup PVC\n\tbackupPVC := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-backup-pvc\",\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tAccessModes: []corev1api.PersistentVolumeAccessMode{\n\t\t\t\tcorev1api.ReadWriteOnce,\n\t\t\t},\n\t\t},\n\t}\n\n\t// Create fake snapshot client\n\tfakeSnapshotClient := snapshotFake.NewSimpleClientset()\n\n\t// Create CSI snapshot exposer\n\texposer := &csiSnapshotExposer{\n\t\tkubeClient: kubeClient,\n\t\tcsiSnapshotClient: fakeSnapshotClient.SnapshotV1(),\n\t\tlog: velerotest.NewLogger(),\n\t}\n\n\t// Call createBackupPod\n\tpod, err := exposer.createBackupPod(\n\t\tctx,\n\t\townerObject,\n\t\tbackupPVC,\n\t\ttime.Minute*5,\n\t\tnil, // labels\n\t\tnil, // annotations\n\t\tnil, // tolerations\n\t\tnil, // affinity\n\t\tcorev1api.ResourceRequirements{},\n\t\tfalse, // backupPVCReadOnly\n\t\tfalse, // spcNoRelabeling\n\t\tkube.NodeOSLinux,\n\t\t\"\", // empty priority class since config map is missing\n\t\tnil,\n\t\tnil,\n\t)\n\n\t// Should succeed even when config map is missing\n\trequire.NoError(t, err, \"Should succeed even when config map is missing\")\n\tassert.NotNil(t, pod)\n\tassert.Empty(t, pod.Spec.PriorityClassName, \"Should have empty priority class when config map is missing\")\n}\n" }, { "path": "pkg/exposer/csi_snapshot_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage exposer\n\nimport (\n\t\"fmt\"\n\t\"testing\"\n\t\"time\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\tsnapshotFake \"github.com/kubernetes-csi/external-snapshotter/client/v8/clientset/versioned/fake\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/api/resource\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes/fake\"\n\tclientTesting \"k8s.io/client-go/testing\"\n\t\"k8s.io/utils/pointer\"\n\tclientFake \"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerov1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\tvelerotypes \"github.com/vmware-tanzu/velero/pkg/types\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\n\tstoragev1api \"k8s.io/api/storage/v1\"\n)\n\ntype reactor struct {\n\tverb string\n\tresource string\n\treactorFunc clientTesting.ReactionFunc\n}\n\nfunc TestExpose(t *testing.T) {\n\tvscName := \"fake-vsc\"\n\tbackup := &velerov1.Backup{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: velerov1.SchemeGroupVersion.String(),\n\t\t\tKind: \"Backup\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup\",\n\t\t\tUID: \"fake-uid\",\n\t\t},\n\t}\n\n\tvar restoreSize int64 = 123456\n\n\tscObj := &storagev1api.StorageClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-sc\",\n\t\t},\n\t}\n\n\tsnapshotClass := \"fake-snapshot-class\"\n\tvsObject := &snapshotv1api.VolumeSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-vs\",\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tAnnotations: map[string]string{\n\t\t\t\t\"fake-key-1\": \"fake-value-1\",\n\t\t\t\t\"fake-key-2\": \"fake-value-2\",\n\t\t\t},\n\t\t},\n\t\tSpec: snapshotv1api.VolumeSnapshotSpec{\n\t\t\tSource: snapshotv1api.VolumeSnapshotSource{\n\t\t\t\tVolumeSnapshotContentName: &vscName,\n\t\t\t},\n\t\t\tVolumeSnapshotClassName: &snapshotClass,\n\t\t},\n\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\tBoundVolumeSnapshotContentName: &vscName,\n\t\t\tReadyToUse: boolptr.True(),\n\t\t\tRestoreSize: resource.NewQuantity(restoreSize, \"\"),\n\t\t},\n\t}\n\n\tvsObjectWithoutRestoreSize := &snapshotv1api.VolumeSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-vs\",\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tAnnotations: map[string]string{\n\t\t\t\t\"fake-key-1\": \"fake-value-1\",\n\t\t\t\t\"fake-key-2\": \"fake-value-2\",\n\t\t\t},\n\t\t},\n\t\tSpec: snapshotv1api.VolumeSnapshotSpec{\n\t\t\tSource: snapshotv1api.VolumeSnapshotSource{\n\t\t\t\tVolumeSnapshotContentName: &vscName,\n\t\t\t},\n\t\t\tVolumeSnapshotClassName: &snapshotClass,\n\t\t},\n\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\tBoundVolumeSnapshotContentName: &vscName,\n\t\t\tReadyToUse: boolptr.True(),\n\t\t},\n\t}\n\n\tsnapshotHandle := \"fake-handle\"\n\tvscObj := &snapshotv1api.VolumeSnapshotContent{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: vscName,\n\t\t\tAnnotations: map[string]string{\n\t\t\t\t\"fake-key-3\": \"fake-value-3\",\n\t\t\t\t\"fake-key-4\": \"fake-value-4\",\n\t\t\t},\n\t\t},\n\t\tSpec: snapshotv1api.VolumeSnapshotContentSpec{\n\t\t\tDeletionPolicy: snapshotv1api.VolumeSnapshotContentDelete,\n\t\t\tDriver: \"fake-driver\",\n\t\t\tVolumeSnapshotClassName: &snapshotClass,\n\t\t},\n\t\tStatus: &snapshotv1api.VolumeSnapshotContentStatus{\n\t\t\tRestoreSize: &restoreSize,\n\t\t\tSnapshotHandle: &snapshotHandle,\n\t\t},\n\t}\n\n\tvscObjWithLabels := vscObj\n\tvscObjWithLabels.Labels = map[string]string{\n\t\t\"snapshot.storage.kubernetes.io/managed-by\": \"worker\",\n\t}\n\n\tdaemonSet := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"velero\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t\tAPIVersion: appsv1api.SchemeGroupVersion.String(),\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"node-agent\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tpvName := \"pv-1\"\n\tvolumeAttachement1 := &storagev1api.VolumeAttachment{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"va1\",\n\t\t},\n\t\tSpec: storagev1api.VolumeAttachmentSpec{\n\t\t\tSource: storagev1api.VolumeAttachmentSource{\n\t\t\t\tPersistentVolumeName: &pvName,\n\t\t\t},\n\t\t\tNodeName: \"node-1\",\n\t\t},\n\t}\n\n\tvolumeAttachement2 := &storagev1api.VolumeAttachment{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"va2\",\n\t\t},\n\t\tSpec: storagev1api.VolumeAttachmentSpec{\n\t\t\tSource: storagev1api.VolumeAttachmentSource{\n\t\t\t\tPersistentVolumeName: &pvName,\n\t\t\t},\n\t\t\tNodeName: \"node-2\",\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tsnapshotClientObj []runtime.Object\n\t\tkubeClientObj []runtime.Object\n\t\townerBackup *velerov1.Backup\n\t\texposeParam CSISnapshotExposeParam\n\t\tsnapReactors []reactor\n\t\tkubeReactors []reactor\n\t\terr string\n\t\texpectedVolumeSize *resource.Quantity\n\t\texpectedReadOnlyPVC bool\n\t\texpectedBackupPVCStorageClass string\n\t\texpectedAffinity *corev1api.Affinity\n\t\texpectedPVCAnnotation map[string]string\n\t}{\n\t\t{\n\t\t\tname: \"get volume topology fail\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tSourcePVName: \"fake-pv\",\n\t\t\t},\n\t\t\terr: \"error getting volume topology for PV fake-pv, storage class fake-sc: error getting storage class fake-sc: storageclasses.storage.k8s.io \\\"fake-sc\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"wait vs ready fail\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tSourcePVName: \"fake-pv\",\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\terr: \"error wait volume snapshot ready: error to get VolumeSnapshot /fake-vs: volumesnapshots.snapshot.storage.k8s.io \\\"fake-vs\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"get vsc fail\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tSourceNamespace: \"fake-ns\",\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tSourcePVName: \"fake-pv\",\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\tvsObject,\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\terr: \"error to get volume snapshot content: error getting volume snapshot content from API: volumesnapshotcontents.snapshot.storage.k8s.io \\\"fake-vsc\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"delete vs fail\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tSourceNamespace: \"fake-ns\",\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tSourcePVName: \"fake-pv\",\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\tvsObject,\n\t\t\t\tvscObj,\n\t\t\t},\n\t\t\tsnapReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"volumesnapshots\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-delete-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\terr: \"error to delete volume snapshot: error to delete volume snapshot: fake-delete-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"delete vsc fail\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tSourceNamespace: \"fake-ns\",\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tSourcePVName: \"fake-pv\",\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\tvsObject,\n\t\t\t\tvscObj,\n\t\t\t},\n\t\t\tsnapReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"volumesnapshotcontents\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-delete-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\terr: \"error to delete volume snapshot content: error to delete volume snapshot content: fake-delete-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"create backup vs fail\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tSourceNamespace: \"fake-ns\",\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tSourcePVName: \"fake-pv\",\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\tvsObject,\n\t\t\t\tvscObj,\n\t\t\t},\n\t\t\tsnapReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"create\",\n\t\t\t\t\tresource: \"volumesnapshots\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-create-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\terr: \"error to create backup volume snapshot: fake-create-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"create backup vsc fail\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tSourceNamespace: \"fake-ns\",\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tSourcePVName: \"fake-pv\",\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\tvsObject,\n\t\t\t\tvscObj,\n\t\t\t},\n\t\t\tsnapReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"create\",\n\t\t\t\t\tresource: \"volumesnapshotcontents\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-create-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\terr: \"error to create backup volume snapshot content: fake-create-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"create backup pvc fail, invalid access mode\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tSourceNamespace: \"fake-ns\",\n\t\t\t\tAccessMode: \"fake-mode\",\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tSourcePVName: \"fake-pv\",\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\tvsObject,\n\t\t\t\tvscObj,\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\terr: \"error to create backup pvc: unsupported access mode fake-mode\",\n\t\t},\n\t\t{\n\t\t\tname: \"create backup pvc fail\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tSourceNamespace: \"fake-ns\",\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tAccessMode: AccessModeFileSystem,\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tSourcePVName: \"fake-pv\",\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\tvsObject,\n\t\t\t\tvscObj,\n\t\t\t},\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"create\",\n\t\t\t\t\tresource: \"persistentvolumeclaims\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-create-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\terr: \"error to create backup pvc: error to create pvc: fake-create-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"create backup pod fail\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tSourceNamespace: \"fake-ns\",\n\t\t\t\tAccessMode: AccessModeFileSystem,\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tSourcePVName: \"fake-pv\",\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\tvsObject,\n\t\t\t\tvscObj,\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSet,\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"create\",\n\t\t\t\t\tresource: \"pods\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-create-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to create backup pod: fake-create-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"success\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tSourceNamespace: \"fake-ns\",\n\t\t\t\tAccessMode: AccessModeFileSystem,\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tSourcePVName: \"fake-pv\",\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\tvsObject,\n\t\t\t\tvscObj,\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSet,\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\texpectedAffinity: &corev1api.Affinity{\n\t\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"windows\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"success-with-labels\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tSourceNamespace: \"fake-ns\",\n\t\t\t\tAccessMode: AccessModeFileSystem,\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tSourcePVName: \"fake-pv\",\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\tvsObject,\n\t\t\t\tvscObjWithLabels,\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSet,\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\texpectedAffinity: &corev1api.Affinity{\n\t\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"windows\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"restore size from exposeParam\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tSourceNamespace: \"fake-ns\",\n\t\t\t\tAccessMode: AccessModeFileSystem,\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tVolumeSize: *resource.NewQuantity(567890, \"\"),\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tSourcePVName: \"fake-pv\",\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\tvsObjectWithoutRestoreSize,\n\t\t\t\tvscObj,\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSet,\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\texpectedVolumeSize: resource.NewQuantity(567890, \"\"),\n\t\t\texpectedAffinity: &corev1api.Affinity{\n\t\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"windows\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"backupPod mounts read only backupPVC\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tSourceNamespace: \"fake-ns\",\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tSourcePVName: \"fake-pv\",\n\t\t\t\tAccessMode: AccessModeFileSystem,\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tBackupPVCConfig: map[string]velerotypes.BackupPVC{\n\t\t\t\t\t\"fake-sc\": {\n\t\t\t\t\t\tStorageClass: \"fake-sc-read-only\",\n\t\t\t\t\t\tReadOnly: true,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\tvsObject,\n\t\t\t\tvscObj,\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSet,\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\texpectedReadOnlyPVC: true,\n\t\t\texpectedAffinity: &corev1api.Affinity{\n\t\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"windows\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"backupPod mounts read only backupPVC and storageClass specified in backupPVC config\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tSourceNamespace: \"fake-ns\",\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tSourcePVName: \"fake-pv\",\n\t\t\t\tAccessMode: AccessModeFileSystem,\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tBackupPVCConfig: map[string]velerotypes.BackupPVC{\n\t\t\t\t\t\"fake-sc\": {\n\t\t\t\t\t\tStorageClass: \"fake-sc-read-only\",\n\t\t\t\t\t\tReadOnly: true,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\tvsObject,\n\t\t\t\tvscObj,\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSet,\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\texpectedReadOnlyPVC: true,\n\t\t\texpectedBackupPVCStorageClass: \"fake-sc-read-only\",\n\t\t\texpectedAffinity: &corev1api.Affinity{\n\t\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"windows\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"backupPod mounts backupPVC with storageClass specified in backupPVC config\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tSourceNamespace: \"fake-ns\",\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tSourcePVName: \"fake-pv\",\n\t\t\t\tAccessMode: AccessModeFileSystem,\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tBackupPVCConfig: map[string]velerotypes.BackupPVC{\n\t\t\t\t\t\"fake-sc\": {\n\t\t\t\t\t\tStorageClass: \"fake-sc-read-only\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\tvsObject,\n\t\t\t\tvscObj,\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSet,\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\texpectedBackupPVCStorageClass: \"fake-sc-read-only\",\n\t\t\texpectedAffinity: &corev1api.Affinity{\n\t\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"windows\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Affinity per StorageClass\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tSourceNamespace: \"fake-ns\",\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tSourcePVName: \"fake-pv\",\n\t\t\t\tAccessMode: AccessModeFileSystem,\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tAffinity: []*kube.LoadAffinity{\n\t\t\t\t\t{\n\t\t\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\t\t\t\tValues: []string{\"Linux\"},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\tvsObject,\n\t\t\t\tvscObj,\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSet,\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\texpectedAffinity: &corev1api.Affinity{\n\t\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpIn,\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"Linux\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"windows\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Affinity per StorageClass with expectedBackupPVCStorageClass\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tSourceNamespace: \"fake-ns\",\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tSourcePVName: \"fake-pv\",\n\t\t\t\tAccessMode: AccessModeFileSystem,\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tBackupPVCConfig: map[string]velerotypes.BackupPVC{\n\t\t\t\t\t\"fake-sc\": {\n\t\t\t\t\t\tStorageClass: \"fake-sc-read-only\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tAffinity: []*kube.LoadAffinity{\n\t\t\t\t\t{\n\t\t\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tKey: \"kubernetes.io/arch\",\n\t\t\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\t\t\t\tValues: []string{\"amd64\"},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\tStorageClass: \"fake-sc-read-only\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\tvsObject,\n\t\t\t\tvscObj,\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSet,\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\texpectedBackupPVCStorageClass: \"fake-sc-read-only\",\n\t\t\texpectedAffinity: &corev1api.Affinity{\n\t\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"kubernetes.io/arch\",\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpIn,\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"amd64\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"windows\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Affinity in exposeParam is nil\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tSourceNamespace: \"fake-ns\",\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tSourcePVName: \"fake-pv\",\n\t\t\t\tAccessMode: AccessModeFileSystem,\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tBackupPVCConfig: map[string]velerotypes.BackupPVC{\n\t\t\t\t\t\"fake-sc\": {\n\t\t\t\t\t\tStorageClass: \"fake-sc-read-only\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tAffinity: nil,\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\tvsObject,\n\t\t\t\tvscObj,\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSet,\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\texpectedBackupPVCStorageClass: \"fake-sc-read-only\",\n\t\t\texpectedAffinity: &corev1api.Affinity{\n\t\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"windows\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"IntolerateSourceNode, get source node fail\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tSourceNamespace: \"fake-ns\",\n\t\t\t\tSourcePVName: pvName,\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tAccessMode: AccessModeFileSystem,\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tBackupPVCConfig: map[string]velerotypes.BackupPVC{\n\t\t\t\t\t\"fake-sc\": {\n\t\t\t\t\t\tAnnotations: map[string]string{util.VSphereCNSFastCloneAnno: \"true\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tAffinity: nil,\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\tvsObject,\n\t\t\t\tvscObj,\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSet,\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"list\",\n\t\t\t\t\tresource: \"volumeattachments\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-create-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedAffinity: &corev1api.Affinity{\n\t\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"windows\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedPVCAnnotation: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"IntolerateSourceNode, get empty source node\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tSourceNamespace: \"fake-ns\",\n\t\t\t\tSourcePVName: pvName,\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tAccessMode: AccessModeFileSystem,\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tBackupPVCConfig: map[string]velerotypes.BackupPVC{\n\t\t\t\t\t\"fake-sc\": {\n\t\t\t\t\t\tAnnotations: map[string]string{util.VSphereCNSFastCloneAnno: \"true\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tAffinity: nil,\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\tvsObject,\n\t\t\t\tvscObj,\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSet,\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\texpectedAffinity: &corev1api.Affinity{\n\t\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"windows\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedPVCAnnotation: map[string]string{util.VSphereCNSFastCloneAnno: \"true\"},\n\t\t},\n\t\t{\n\t\t\tname: \"IntolerateSourceNode, get source nodes\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: CSISnapshotExposeParam{\n\t\t\t\tSnapshotName: \"fake-vs\",\n\t\t\t\tSourceNamespace: \"fake-ns\",\n\t\t\t\tSourcePVName: pvName,\n\t\t\t\tStorageClass: \"fake-sc\",\n\t\t\t\tAccessMode: AccessModeFileSystem,\n\t\t\t\tOperationTimeout: time.Millisecond,\n\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\tBackupPVCConfig: map[string]velerotypes.BackupPVC{\n\t\t\t\t\t\"fake-sc\": {\n\t\t\t\t\t\tAnnotations: map[string]string{util.VSphereCNSFastCloneAnno: \"true\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tAffinity: nil,\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\tvsObject,\n\t\t\t\tvscObj,\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSet,\n\t\t\t\tvolumeAttachement1,\n\t\t\t\tvolumeAttachement2,\n\t\t\t\tscObj,\n\t\t\t},\n\t\t\texpectedAffinity: &corev1api.Affinity{\n\t\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"windows\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"kubernetes.io/hostname\",\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"node-1\", \"node-2\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedPVCAnnotation: map[string]string{util.VSphereCNSFastCloneAnno: \"true\"},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeSnapshotClient := snapshotFake.NewSimpleClientset(test.snapshotClientObj...)\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tfor _, reactor := range test.snapReactors {\n\t\t\t\tfakeSnapshotClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\tfor _, reactor := range test.kubeReactors {\n\t\t\t\tfakeKubeClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\texposer := csiSnapshotExposer{\n\t\t\t\tkubeClient: fakeKubeClient,\n\t\t\t\tcsiSnapshotClient: fakeSnapshotClient.SnapshotV1(),\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tvar ownerObject corev1api.ObjectReference\n\t\t\tif test.ownerBackup != nil {\n\t\t\t\townerObject = corev1api.ObjectReference{\n\t\t\t\t\tKind: test.ownerBackup.Kind,\n\t\t\t\t\tNamespace: test.ownerBackup.Namespace,\n\t\t\t\t\tName: test.ownerBackup.Name,\n\t\t\t\t\tUID: test.ownerBackup.UID,\n\t\t\t\t\tAPIVersion: test.ownerBackup.APIVersion,\n\t\t\t\t}\n\t\t\t}\n\n\t\t\terr := exposer.Expose(t.Context(), ownerObject, &test.exposeParam)\n\t\t\tif err == nil {\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\tbackupPod, err := exposer.kubeClient.CoreV1().Pods(ownerObject.Namespace).Get(t.Context(), ownerObject.Name, metav1.GetOptions{})\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\tbackupPVC, err := exposer.kubeClient.CoreV1().PersistentVolumeClaims(ownerObject.Namespace).Get(t.Context(), ownerObject.Name, metav1.GetOptions{})\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\texpectedVS, err := exposer.csiSnapshotClient.VolumeSnapshots(ownerObject.Namespace).Get(t.Context(), ownerObject.Name, metav1.GetOptions{})\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\texpectedVSC, err := exposer.csiSnapshotClient.VolumeSnapshotContents().Get(t.Context(), ownerObject.Name, metav1.GetOptions{})\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\tassert.Equal(t, expectedVS.Annotations, vsObject.Annotations)\n\t\t\t\tassert.Equal(t, *expectedVS.Spec.VolumeSnapshotClassName, *vsObject.Spec.VolumeSnapshotClassName)\n\t\t\t\tassert.Equal(t, expectedVSC.Name, *expectedVS.Spec.Source.VolumeSnapshotContentName)\n\n\t\t\t\tassert.Equal(t, expectedVSC.Annotations, vscObj.Annotations)\n\t\t\t\tassert.Equal(t, expectedVSC.Labels, vscObj.Labels)\n\t\t\t\tassert.Equal(t, expectedVSC.Spec.DeletionPolicy, vscObj.Spec.DeletionPolicy)\n\t\t\t\tassert.Equal(t, expectedVSC.Spec.Driver, vscObj.Spec.Driver)\n\t\t\t\tassert.Equal(t, *expectedVSC.Spec.VolumeSnapshotClassName, *vscObj.Spec.VolumeSnapshotClassName)\n\n\t\t\t\tif test.expectedVolumeSize != nil {\n\t\t\t\t\tassert.Equal(t, *test.expectedVolumeSize, backupPVC.Spec.Resources.Requests[corev1api.ResourceStorage])\n\t\t\t\t} else {\n\t\t\t\t\tassert.Equal(t, *resource.NewQuantity(restoreSize, \"\"), backupPVC.Spec.Resources.Requests[corev1api.ResourceStorage])\n\t\t\t\t}\n\n\t\t\t\tif test.expectedReadOnlyPVC {\n\t\t\t\t\tgotReadOnlyAccessMode := false\n\t\t\t\t\tfor _, accessMode := range backupPVC.Spec.AccessModes {\n\t\t\t\t\t\tif accessMode == corev1api.ReadOnlyMany {\n\t\t\t\t\t\t\tgotReadOnlyAccessMode = true\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t\tassert.Equal(t, test.expectedReadOnlyPVC, gotReadOnlyAccessMode)\n\t\t\t\t}\n\n\t\t\t\tif test.expectedBackupPVCStorageClass != \"\" {\n\t\t\t\t\tassert.Equal(t, test.expectedBackupPVCStorageClass, *backupPVC.Spec.StorageClassName)\n\t\t\t\t}\n\n\t\t\t\tif test.expectedAffinity != nil {\n\t\t\t\t\tassert.Equal(t, test.expectedAffinity, backupPod.Spec.Affinity)\n\t\t\t\t} else {\n\t\t\t\t\tassert.Nil(t, backupPod.Spec.Affinity)\n\t\t\t\t}\n\n\t\t\t\tif test.expectedPVCAnnotation != nil {\n\t\t\t\t\tassert.Equal(t, test.expectedPVCAnnotation, backupPVC.Annotations)\n\t\t\t\t} else {\n\t\t\t\t\tassert.Empty(t, backupPVC.Annotations)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, test.err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetExpose(t *testing.T) {\n\tbackup := &velerov1.Backup{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: velerov1.SchemeGroupVersion.String(),\n\t\t\tKind: \"Backup\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup\",\n\t\t\tUID: \"fake-uid\",\n\t\t},\n\t}\n\n\tbackupPod := &corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: backup.Namespace,\n\t\t\tName: backup.Name,\n\t\t},\n\t\tSpec: corev1api.PodSpec{\n\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t{\n\t\t\t\t\tName: \"fake-volume\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"fake-volume-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: string(backup.UID),\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tbackupPodWithoutVolume := &corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: backup.Namespace,\n\t\t\tName: backup.Name,\n\t\t},\n\t\tSpec: corev1api.PodSpec{\n\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t{\n\t\t\t\t\tName: \"fake-volume-1\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"fake-volume-2\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tbackupPVC := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: backup.Namespace,\n\t\t\tName: backup.Name,\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tVolumeName: \"fake-pv-name\",\n\t\t},\n\t}\n\n\tbackupPV := &corev1api.PersistentVolume{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-pv-name\",\n\t\t},\n\t}\n\n\tscheme := runtime.NewScheme()\n\tcorev1api.AddToScheme(scheme)\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\townerBackup *velerov1.Backup\n\t\texposeWaitParam CSISnapshotExposeWaitParam\n\t\tTimeout time.Duration\n\t\terr string\n\t\texpectedResult *ExposeResult\n\t}{\n\t\t{\n\t\t\tname: \"backup pod is not found\",\n\t\t\townerBackup: backup,\n\t\t\texposeWaitParam: CSISnapshotExposeWaitParam{\n\t\t\t\tNodeName: \"fake-node\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"wait pvc bound fail\",\n\t\t\townerBackup: backup,\n\t\t\texposeWaitParam: CSISnapshotExposeWaitParam{\n\t\t\t\tNodeName: \"fake-node\",\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tbackupPod,\n\t\t\t},\n\t\t\tTimeout: time.Second,\n\t\t\terr: \"error to wait backup PVC bound, fake-backup: error to wait for rediness of PVC: error to get pvc velero/fake-backup: persistentvolumeclaims \\\"fake-backup\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"backup volume not found in pod\",\n\t\t\townerBackup: backup,\n\t\t\texposeWaitParam: CSISnapshotExposeWaitParam{\n\t\t\t\tNodeName: \"fake-node\",\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tbackupPodWithoutVolume,\n\t\t\t\tbackupPVC,\n\t\t\t\tbackupPV,\n\t\t\t},\n\t\t\tTimeout: time.Second,\n\t\t\terr: \"backup pod fake-backup doesn't have the expected backup volume\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\townerBackup: backup,\n\t\t\texposeWaitParam: CSISnapshotExposeWaitParam{\n\t\t\t\tNodeName: \"fake-node\",\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tbackupPod,\n\t\t\t\tbackupPVC,\n\t\t\t\tbackupPV,\n\t\t\t},\n\t\t\tTimeout: time.Second,\n\t\t\texpectedResult: &ExposeResult{\n\t\t\t\tByPod: ExposeByPod{\n\t\t\t\t\tHostingPod: backupPod,\n\t\t\t\t\tVolumeName: string(backup.UID),\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tfakeClientBuilder := clientFake.NewClientBuilder()\n\t\t\tfakeClientBuilder = fakeClientBuilder.WithScheme(scheme)\n\n\t\t\tfakeClient := fakeClientBuilder.WithRuntimeObjects(test.kubeClientObj...).Build()\n\n\t\t\texposer := csiSnapshotExposer{\n\t\t\t\tkubeClient: fakeKubeClient,\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tvar ownerObject corev1api.ObjectReference\n\t\t\tif test.ownerBackup != nil {\n\t\t\t\townerObject = corev1api.ObjectReference{\n\t\t\t\t\tKind: test.ownerBackup.Kind,\n\t\t\t\t\tNamespace: test.ownerBackup.Namespace,\n\t\t\t\t\tName: test.ownerBackup.Name,\n\t\t\t\t\tUID: test.ownerBackup.UID,\n\t\t\t\t\tAPIVersion: test.ownerBackup.APIVersion,\n\t\t\t\t}\n\t\t\t}\n\n\t\t\ttest.exposeWaitParam.NodeClient = fakeClient\n\n\t\t\tresult, err := exposer.GetExposed(t.Context(), ownerObject, test.Timeout, &test.exposeWaitParam)\n\t\t\tif test.err == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\tif test.expectedResult == nil {\n\t\t\t\t\tassert.Nil(t, result)\n\t\t\t\t} else {\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tassert.Equal(t, test.expectedResult.ByPod.VolumeName, result.ByPod.VolumeName)\n\t\t\t\t\tassert.Equal(t, test.expectedResult.ByPod.HostingPod.Name, result.ByPod.HostingPod.Name)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, test.err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestPeekExpose(t *testing.T) {\n\tbackup := &velerov1.Backup{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: velerov1.SchemeGroupVersion.String(),\n\t\t\tKind: \"Backup\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup\",\n\t\t\tUID: \"fake-uid\",\n\t\t},\n\t}\n\n\tbackupPodUrecoverable := &corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: backup.Namespace,\n\t\t\tName: backup.Name,\n\t\t},\n\t\tStatus: corev1api.PodStatus{\n\t\t\tPhase: corev1api.PodFailed,\n\t\t},\n\t}\n\n\tbackupPod := &corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: backup.Namespace,\n\t\t\tName: backup.Name,\n\t\t},\n\t}\n\n\tscheme := runtime.NewScheme()\n\tcorev1api.AddToScheme(scheme)\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\townerBackup *velerov1.Backup\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"backup pod is not found\",\n\t\t\townerBackup: backup,\n\t\t},\n\t\t{\n\t\t\tname: \"pod is unrecoverable\",\n\t\t\townerBackup: backup,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tbackupPodUrecoverable,\n\t\t\t},\n\t\t\terr: \"Pod is in abnormal state [Failed], message []\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\townerBackup: backup,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tbackupPod,\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\texposer := csiSnapshotExposer{\n\t\t\t\tkubeClient: fakeKubeClient,\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tvar ownerObject corev1api.ObjectReference\n\t\t\tif test.ownerBackup != nil {\n\t\t\t\townerObject = corev1api.ObjectReference{\n\t\t\t\t\tKind: test.ownerBackup.Kind,\n\t\t\t\t\tNamespace: test.ownerBackup.Namespace,\n\t\t\t\t\tName: test.ownerBackup.Name,\n\t\t\t\t\tUID: test.ownerBackup.UID,\n\t\t\t\t\tAPIVersion: test.ownerBackup.APIVersion,\n\t\t\t\t}\n\t\t\t}\n\n\t\t\terr := exposer.PeekExposed(t.Context(), ownerObject)\n\t\t\tif test.err == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, test.err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc Test_csiSnapshotExposer_createBackupPVC(t *testing.T) {\n\tbackup := &velerov1.Backup{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: velerov1.SchemeGroupVersion.String(),\n\t\t\tKind: \"Backup\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup\",\n\t\t\tUID: \"fake-uid\",\n\t\t},\n\t}\n\n\tdataSource := &corev1api.TypedLocalObjectReference{\n\t\tAPIGroup: &snapshotv1api.SchemeGroupVersion.Group,\n\t\tKind: \"VolumeSnapshot\",\n\t\tName: \"fake-snapshot\",\n\t}\n\tvolumeMode := corev1api.PersistentVolumeFilesystem\n\n\tbackupPVC := corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup\",\n\t\t\tAnnotations: map[string]string{},\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: backup.APIVersion,\n\t\t\t\t\tKind: backup.Kind,\n\t\t\t\t\tName: backup.Name,\n\t\t\t\t\tUID: backup.UID,\n\t\t\t\t\tController: pointer.BoolPtr(true),\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tAccessModes: []corev1api.PersistentVolumeAccessMode{\n\t\t\t\tcorev1api.ReadWriteOnce,\n\t\t\t},\n\t\t\tVolumeMode: &volumeMode,\n\t\t\tDataSource: dataSource,\n\t\t\tDataSourceRef: nil,\n\t\t\tStorageClassName: pointer.String(\"fake-storage-class\"),\n\t\t\tResources: corev1api.VolumeResourceRequirements{\n\t\t\t\tRequests: corev1api.ResourceList{\n\t\t\t\t\tcorev1api.ResourceStorage: resource.MustParse(\"1Gi\"),\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tbackupPVCReadOnly := corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup\",\n\t\t\tAnnotations: map[string]string{},\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: backup.APIVersion,\n\t\t\t\t\tKind: backup.Kind,\n\t\t\t\t\tName: backup.Name,\n\t\t\t\t\tUID: backup.UID,\n\t\t\t\t\tController: pointer.BoolPtr(true),\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tAccessModes: []corev1api.PersistentVolumeAccessMode{\n\t\t\t\tcorev1api.ReadOnlyMany,\n\t\t\t},\n\t\t\tVolumeMode: &volumeMode,\n\t\t\tDataSource: dataSource,\n\t\t\tDataSourceRef: nil,\n\t\t\tStorageClassName: pointer.String(\"fake-storage-class\"),\n\t\t\tResources: corev1api.VolumeResourceRequirements{\n\t\t\t\tRequests: corev1api.ResourceList{\n\t\t\t\t\tcorev1api.ResourceStorage: resource.MustParse(\"1Gi\"),\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\townerBackup *velerov1.Backup\n\t\tbackupVS string\n\t\tstorageClass string\n\t\taccessMode string\n\t\tresource resource.Quantity\n\t\treadOnly bool\n\t\tkubeClientObj []runtime.Object\n\t\tsnapshotClientObj []runtime.Object\n\t\twant *corev1api.PersistentVolumeClaim\n\t\twantErr assert.ErrorAssertionFunc\n\t}{\n\t\t{\n\t\t\tname: \"backupPVC gets created successfully with parameters from source PVC\",\n\t\t\townerBackup: backup,\n\t\t\tbackupVS: \"fake-snapshot\",\n\t\t\tstorageClass: \"fake-storage-class\",\n\t\t\taccessMode: AccessModeFileSystem,\n\t\t\tresource: resource.MustParse(\"1Gi\"),\n\t\t\treadOnly: false,\n\t\t\twant: &backupPVC,\n\t\t\twantErr: assert.NoError,\n\t\t},\n\t\t{\n\t\t\tname: \"backupPVC gets created successfully with parameters from source PVC but accessMode from backupPVC Config as read only\",\n\t\t\townerBackup: backup,\n\t\t\tbackupVS: \"fake-snapshot\",\n\t\t\tstorageClass: \"fake-storage-class\",\n\t\t\taccessMode: AccessModeFileSystem,\n\t\t\tresource: resource.MustParse(\"1Gi\"),\n\t\t\treadOnly: true,\n\t\t\twant: &backupPVCReadOnly,\n\t\t\twantErr: assert.NoError,\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(tt.kubeClientObj...)\n\t\t\tfakeSnapshotClient := snapshotFake.NewSimpleClientset(tt.snapshotClientObj...)\n\t\t\te := &csiSnapshotExposer{\n\t\t\t\tkubeClient: fakeKubeClient,\n\t\t\t\tcsiSnapshotClient: fakeSnapshotClient.SnapshotV1(),\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\t\t\tvar ownerObject corev1api.ObjectReference\n\t\t\tif tt.ownerBackup != nil {\n\t\t\t\townerObject = corev1api.ObjectReference{\n\t\t\t\t\tKind: tt.ownerBackup.Kind,\n\t\t\t\t\tNamespace: tt.ownerBackup.Namespace,\n\t\t\t\t\tName: tt.ownerBackup.Name,\n\t\t\t\t\tUID: tt.ownerBackup.UID,\n\t\t\t\t\tAPIVersion: tt.ownerBackup.APIVersion,\n\t\t\t\t}\n\t\t\t}\n\t\t\tgot, err := e.createBackupPVC(t.Context(), ownerObject, tt.backupVS, tt.storageClass, tt.accessMode, tt.resource, tt.readOnly, map[string]string{})\n\t\t\tif !tt.wantErr(t, err, fmt.Sprintf(\"createBackupPVC(%v, %v, %v, %v, %v, %v)\", ownerObject, tt.backupVS, tt.storageClass, tt.accessMode, tt.resource, tt.readOnly)) {\n\t\t\t\treturn\n\t\t\t}\n\t\t\tassert.Equalf(t, tt.want, got, \"createBackupPVC(%v, %v, %v, %v, %v, %v)\", ownerObject, tt.backupVS, tt.storageClass, tt.accessMode, tt.resource, tt.readOnly)\n\t\t})\n\t}\n}\n\nfunc Test_csiSnapshotExposer_DiagnoseExpose(t *testing.T) {\n\tbackup := &velerov1.Backup{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: velerov1.SchemeGroupVersion.String(),\n\t\t\tKind: \"Backup\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup\",\n\t\t\tUID: \"fake-uid\",\n\t\t},\n\t}\n\n\tbackupPodWithoutNodeName := corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup\",\n\t\t\tUID: \"fake-pod-uid\",\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: backup.APIVersion,\n\t\t\t\t\tKind: backup.Kind,\n\t\t\t\t\tName: backup.Name,\n\t\t\t\t\tUID: backup.UID,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tStatus: corev1api.PodStatus{\n\t\t\tPhase: corev1api.PodPending,\n\t\t\tConditions: []corev1api.PodCondition{\n\t\t\t\t{\n\t\t\t\t\tType: corev1api.PodInitialized,\n\t\t\t\t\tStatus: corev1api.ConditionTrue,\n\t\t\t\t\tMessage: \"fake-pod-message\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tMessage: \"fake-pod-message-1\",\n\t\t},\n\t}\n\n\tbackupPodWithNodeName := corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup\",\n\t\t\tUID: \"fake-pod-uid\",\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: backup.APIVersion,\n\t\t\t\t\tKind: backup.Kind,\n\t\t\t\t\tName: backup.Name,\n\t\t\t\t\tUID: backup.UID,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tSpec: corev1api.PodSpec{\n\t\t\tNodeName: \"fake-node\",\n\t\t},\n\t\tStatus: corev1api.PodStatus{\n\t\t\tPhase: corev1api.PodPending,\n\t\t\tConditions: []corev1api.PodCondition{\n\t\t\t\t{\n\t\t\t\t\tType: corev1api.PodInitialized,\n\t\t\t\t\tStatus: corev1api.ConditionTrue,\n\t\t\t\t\tMessage: \"fake-pod-message\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tbackupPVCWithoutVolumeName := corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup\",\n\t\t\tUID: \"fake-pvc-uid\",\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: backup.APIVersion,\n\t\t\t\t\tKind: backup.Kind,\n\t\t\t\t\tName: backup.Name,\n\t\t\t\t\tUID: backup.UID,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\tPhase: corev1api.ClaimPending,\n\t\t},\n\t}\n\n\tbackupPVCWithVolumeName := corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup\",\n\t\t\tUID: \"fake-pvc-uid\",\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: backup.APIVersion,\n\t\t\t\t\tKind: backup.Kind,\n\t\t\t\t\tName: backup.Name,\n\t\t\t\t\tUID: backup.UID,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tVolumeName: \"fake-pv\",\n\t\t},\n\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\tPhase: corev1api.ClaimPending,\n\t\t},\n\t}\n\n\tbackupPV := corev1api.PersistentVolume{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-pv\",\n\t\t},\n\t\tStatus: corev1api.PersistentVolumeStatus{\n\t\t\tPhase: corev1api.VolumePending,\n\t\t\tMessage: \"fake-pv-message\",\n\t\t},\n\t}\n\n\treadyToUse := false\n\tvscMessage := \"fake-vsc-message\"\n\tbackupVSC := snapshotv1api.VolumeSnapshotContent{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-vsc\",\n\t\t},\n\t\tStatus: &snapshotv1api.VolumeSnapshotContentStatus{\n\t\t\tReadyToUse: &readyToUse,\n\t\t\tError: &snapshotv1api.VolumeSnapshotError{\n\t\t\t\tMessage: &vscMessage,\n\t\t\t},\n\t\t},\n\t}\n\n\tbackupVSWithoutStatus := snapshotv1api.VolumeSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup\",\n\t\t\tUID: \"fake-vs-uid\",\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: backup.APIVersion,\n\t\t\t\t\tKind: backup.Kind,\n\t\t\t\t\tName: backup.Name,\n\t\t\t\t\tUID: backup.UID,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tbackupVSWithoutVSC := snapshotv1api.VolumeSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup\",\n\t\t\tUID: \"fake-vs-uid\",\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: backup.APIVersion,\n\t\t\t\t\tKind: backup.Kind,\n\t\t\t\t\tName: backup.Name,\n\t\t\t\t\tUID: backup.UID,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{},\n\t}\n\n\tvsMessage := \"fake-vs-message\"\n\tbackupVSWithVSC := snapshotv1api.VolumeSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup\",\n\t\t\tUID: \"fake-vs-uid\",\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: backup.APIVersion,\n\t\t\t\t\tKind: backup.Kind,\n\t\t\t\t\tName: backup.Name,\n\t\t\t\t\tUID: backup.UID,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\tBoundVolumeSnapshotContentName: &backupVSC.Name,\n\t\t\tError: &snapshotv1api.VolumeSnapshotError{\n\t\t\t\tMessage: &vsMessage,\n\t\t\t},\n\t\t},\n\t}\n\n\tnodeAgentPod := corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"node-agent-pod-1\",\n\t\t\tLabels: map[string]string{\"role\": \"node-agent\"},\n\t\t},\n\t\tSpec: corev1api.PodSpec{\n\t\t\tNodeName: \"fake-node\",\n\t\t},\n\t\tStatus: corev1api.PodStatus{\n\t\t\tPhase: corev1api.PodRunning,\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\townerBackup *velerov1.Backup\n\t\tkubeClientObj []runtime.Object\n\t\tsnapshotClientObj []runtime.Object\n\t\texpected string\n\t}{\n\t\t{\n\t\t\tname: \"no pod, pvc, vs\",\n\t\t\townerBackup: backup,\n\t\t\texpected: `begin diagnose CSI exposer\nerror getting backup pod fake-backup, err: pods \"fake-backup\" not found\nerror getting backup pvc fake-backup, err: persistentvolumeclaims \"fake-backup\" not found\nerror getting backup vs fake-backup, err: volumesnapshots.snapshot.storage.k8s.io \"fake-backup\" not found\nend diagnose CSI exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"pod without node name, pvc without volume name, vs without status\",\n\t\t\townerBackup: backup,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&backupPodWithoutNodeName,\n\t\t\t\t&backupPVCWithoutVolumeName,\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\t&backupVSWithoutStatus,\n\t\t\t},\n\t\t\texpected: `begin diagnose CSI exposer\nPod velero/fake-backup, phase Pending, node name , message fake-pod-message-1\nPod condition Initialized, status True, reason , message fake-pod-message\nPVC velero/fake-backup, phase Pending, binding to \nVS velero/fake-backup, bind to , readyToUse false, errMessage \nend diagnose CSI exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"pod without node name, pvc without volume name, vs without VSC\",\n\t\t\townerBackup: backup,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&backupPodWithoutNodeName,\n\t\t\t\t&backupPVCWithoutVolumeName,\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\t&backupVSWithoutVSC,\n\t\t\t},\n\t\t\texpected: `begin diagnose CSI exposer\nPod velero/fake-backup, phase Pending, node name , message fake-pod-message-1\nPod condition Initialized, status True, reason , message fake-pod-message\nPVC velero/fake-backup, phase Pending, binding to \nVS velero/fake-backup, bind to , readyToUse false, errMessage \nend diagnose CSI exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"pod with node name, no node agent\",\n\t\t\townerBackup: backup,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&backupPodWithNodeName,\n\t\t\t\t&backupPVCWithoutVolumeName,\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\t&backupVSWithoutVSC,\n\t\t\t},\n\t\t\texpected: `begin diagnose CSI exposer\nPod velero/fake-backup, phase Pending, node name fake-node, message \nPod condition Initialized, status True, reason , message fake-pod-message\nnode-agent is not running in node fake-node, err: daemonset pod not found in running state in node fake-node\nPVC velero/fake-backup, phase Pending, binding to \nVS velero/fake-backup, bind to , readyToUse false, errMessage \nend diagnose CSI exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"pod with node name, node agent is running\",\n\t\t\townerBackup: backup,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&backupPodWithNodeName,\n\t\t\t\t&backupPVCWithoutVolumeName,\n\t\t\t\t&nodeAgentPod,\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\t&backupVSWithoutVSC,\n\t\t\t},\n\t\t\texpected: `begin diagnose CSI exposer\nPod velero/fake-backup, phase Pending, node name fake-node, message \nPod condition Initialized, status True, reason , message fake-pod-message\nPVC velero/fake-backup, phase Pending, binding to \nVS velero/fake-backup, bind to , readyToUse false, errMessage \nend diagnose CSI exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"pvc with volume name, no pv\",\n\t\t\townerBackup: backup,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&backupPodWithNodeName,\n\t\t\t\t&backupPVCWithVolumeName,\n\t\t\t\t&nodeAgentPod,\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\t&backupVSWithoutVSC,\n\t\t\t},\n\t\t\texpected: `begin diagnose CSI exposer\nPod velero/fake-backup, phase Pending, node name fake-node, message \nPod condition Initialized, status True, reason , message fake-pod-message\nPVC velero/fake-backup, phase Pending, binding to fake-pv\nerror getting backup pv fake-pv, err: persistentvolumes \"fake-pv\" not found\nVS velero/fake-backup, bind to , readyToUse false, errMessage \nend diagnose CSI exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"pvc with volume name, pv exists\",\n\t\t\townerBackup: backup,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&backupPodWithNodeName,\n\t\t\t\t&backupPVCWithVolumeName,\n\t\t\t\t&backupPV,\n\t\t\t\t&nodeAgentPod,\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\t&backupVSWithoutVSC,\n\t\t\t},\n\t\t\texpected: `begin diagnose CSI exposer\nPod velero/fake-backup, phase Pending, node name fake-node, message \nPod condition Initialized, status True, reason , message fake-pod-message\nPVC velero/fake-backup, phase Pending, binding to fake-pv\nPV fake-pv, phase Pending, reason , message fake-pv-message\nVS velero/fake-backup, bind to , readyToUse false, errMessage \nend diagnose CSI exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"vs with vsc, vsc doesn't exist\",\n\t\t\townerBackup: backup,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&backupPodWithNodeName,\n\t\t\t\t&backupPVCWithVolumeName,\n\t\t\t\t&backupPV,\n\t\t\t\t&nodeAgentPod,\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\t&backupVSWithVSC,\n\t\t\t},\n\t\t\texpected: `begin diagnose CSI exposer\nPod velero/fake-backup, phase Pending, node name fake-node, message \nPod condition Initialized, status True, reason , message fake-pod-message\nPVC velero/fake-backup, phase Pending, binding to fake-pv\nPV fake-pv, phase Pending, reason , message fake-pv-message\nVS velero/fake-backup, bind to fake-vsc, readyToUse false, errMessage fake-vs-message\nerror getting backup vsc fake-vsc, err: volumesnapshotcontents.snapshot.storage.k8s.io \"fake-vsc\" not found\nend diagnose CSI exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"vs with vsc, vsc exists\",\n\t\t\townerBackup: backup,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&backupPodWithNodeName,\n\t\t\t\t&backupPVCWithVolumeName,\n\t\t\t\t&backupPV,\n\t\t\t\t&nodeAgentPod,\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\t&backupVSWithVSC,\n\t\t\t\t&backupVSC,\n\t\t\t},\n\t\t\texpected: `begin diagnose CSI exposer\nPod velero/fake-backup, phase Pending, node name fake-node, message \nPod condition Initialized, status True, reason , message fake-pod-message\nPVC velero/fake-backup, phase Pending, binding to fake-pv\nPV fake-pv, phase Pending, reason , message fake-pv-message\nVS velero/fake-backup, bind to fake-vsc, readyToUse false, errMessage fake-vs-message\nVSC fake-vsc, readyToUse false, errMessage fake-vsc-message, handle \nend diagnose CSI exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"with events\",\n\t\t\townerBackup: backup,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&backupPodWithNodeName,\n\t\t\t\t&backupPVCWithVolumeName,\n\t\t\t\t&backupPV,\n\t\t\t\t&nodeAgentPod,\n\t\t\t\t&corev1api.Event{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Namespace: velerov1.DefaultNamespace, Name: \"event-1\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-uid-1\"},\n\t\t\t\t\tReason: \"reason-1\",\n\t\t\t\t\tMessage: \"message-1\",\n\t\t\t\t},\n\t\t\t\t&corev1api.Event{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Namespace: velerov1.DefaultNamespace, Name: \"event-2\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-pod-uid\"},\n\t\t\t\t\tReason: \"reason-2\",\n\t\t\t\t\tMessage: \"message-2\",\n\t\t\t\t},\n\t\t\t\t&corev1api.Event{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Namespace: velerov1.DefaultNamespace, Name: \"event-3\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-pvc-uid\"},\n\t\t\t\t\tReason: \"reason-3\",\n\t\t\t\t\tMessage: \"message-3\",\n\t\t\t\t},\n\t\t\t\t&corev1api.Event{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Namespace: velerov1.DefaultNamespace, Name: \"event-4\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-vs-uid\"},\n\t\t\t\t\tReason: \"reason-4\",\n\t\t\t\t\tMessage: \"message-4\",\n\t\t\t\t},\n\t\t\t\t&corev1api.Event{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Namespace: \"other-namespace\", Name: \"event-5\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-pod-uid\"},\n\t\t\t\t\tReason: \"reason-5\",\n\t\t\t\t\tMessage: \"message-5\",\n\t\t\t\t},\n\t\t\t\t&corev1api.Event{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Namespace: velerov1.DefaultNamespace, Name: \"event-6\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-pod-uid\"},\n\t\t\t\t\tReason: \"reason-6\",\n\t\t\t\t\tMessage: \"message-6\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tsnapshotClientObj: []runtime.Object{\n\t\t\t\t&backupVSWithVSC,\n\t\t\t\t&backupVSC,\n\t\t\t},\n\t\t\texpected: `begin diagnose CSI exposer\nPod velero/fake-backup, phase Pending, node name fake-node, message \nPod condition Initialized, status True, reason , message fake-pod-message\nPod event reason reason-2, message message-2\nPod event reason reason-6, message message-6\nPVC velero/fake-backup, phase Pending, binding to fake-pv\nPVC event reason reason-3, message message-3\nPV fake-pv, phase Pending, reason , message fake-pv-message\nVS velero/fake-backup, bind to fake-vsc, readyToUse false, errMessage fake-vs-message\nVS event reason reason-4, message message-4\nVSC fake-vsc, readyToUse false, errMessage fake-vsc-message, handle \nend diagnose CSI exposer`,\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(tt.kubeClientObj...)\n\t\t\tfakeSnapshotClient := snapshotFake.NewSimpleClientset(tt.snapshotClientObj...)\n\t\t\te := &csiSnapshotExposer{\n\t\t\t\tkubeClient: fakeKubeClient,\n\t\t\t\tcsiSnapshotClient: fakeSnapshotClient.SnapshotV1(),\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\t\t\tvar ownerObject corev1api.ObjectReference\n\t\t\tif tt.ownerBackup != nil {\n\t\t\t\townerObject = corev1api.ObjectReference{\n\t\t\t\t\tKind: tt.ownerBackup.Kind,\n\t\t\t\t\tNamespace: tt.ownerBackup.Namespace,\n\t\t\t\t\tName: tt.ownerBackup.Name,\n\t\t\t\t\tUID: tt.ownerBackup.UID,\n\t\t\t\t\tAPIVersion: tt.ownerBackup.APIVersion,\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tdiag := e.DiagnoseExpose(t.Context(), ownerObject)\n\t\t\tassert.Equal(t, tt.expected, diag)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/exposer/generic_restore.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage exposer\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/nodeagent\"\n\tvelerotypes \"github.com/vmware-tanzu/velero/pkg/types\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\n// GenericRestoreExposeParam define the input param for Generic Restore Expose\ntype GenericRestoreExposeParam struct {\n\t// TargetPVCName is the target volume name to be restored\n\tTargetPVCName string\n\n\t// TargetNamespace is the namespace of the volume to be restored\n\tTargetNamespace string\n\n\t// HostingPodLabels is the labels that are going to apply to the hosting pod\n\tHostingPodLabels map[string]string\n\n\t// HostingPodAnnotations is the annotations that are going to apply to the hosting pod\n\tHostingPodAnnotations map[string]string\n\n\t// HostingPodTolerations is the tolerations that are going to apply to the hosting pod\n\tHostingPodTolerations []corev1api.Toleration\n\n\t// Resources defines the resource requirements of the hosting pod\n\tResources corev1api.ResourceRequirements\n\n\t// ExposeTimeout specifies the timeout for the entire expose process\n\tExposeTimeout time.Duration\n\n\t// OperationTimeout specifies the time wait for resources operations in Expose\n\tOperationTimeout time.Duration\n\n\t// NodeOS specifies the OS of node that the volume should be attached\n\tNodeOS string\n\n\t// RestorePVCConfig is the config for restorePVC (intermediate PVC) of generic restore\n\tRestorePVCConfig velerotypes.RestorePVC\n\n\t// LoadAffinity specifies the node affinity of the backup pod\n\tLoadAffinity []*kube.LoadAffinity\n\n\t// PriorityClassName is the priority class name for the data mover pod\n\tPriorityClassName string\n\n\t// RestoreSize specifies the data size for the volume to be restored\n\tRestoreSize int64\n\n\t// CacheVolume specifies the info for cache volumes\n\tCacheVolume *CacheConfigs\n}\n\n// GenericRestoreExposer is the interfaces for a generic restore exposer\ntype GenericRestoreExposer interface {\n\t// Expose starts the process to a restore expose, the expose process may take long time\n\tExpose(context.Context, corev1api.ObjectReference, GenericRestoreExposeParam) error\n\n\t// GetExposed polls the status of the expose.\n\t// If the expose is accessible by the current caller, it waits the expose ready and returns the expose result.\n\t// Otherwise, it returns nil as the expose result without an error.\n\tGetExposed(context.Context, corev1api.ObjectReference, client.Client, string, time.Duration) (*ExposeResult, error)\n\n\t// PeekExposed tests the status of the expose.\n\t// If the expose is incomplete but not recoverable, it returns an error.\n\t// Otherwise, it returns nil immediately.\n\tPeekExposed(context.Context, corev1api.ObjectReference) error\n\n\t// DiagnoseExpose generate the diagnostic info when the expose is not finished for a long time.\n\t// If it finds any problem, it returns an string about the problem.\n\tDiagnoseExpose(context.Context, corev1api.ObjectReference) string\n\n\t// RebindVolume unexposes the restored PV and rebind it to the target PVC\n\tRebindVolume(context.Context, corev1api.ObjectReference, string, string, time.Duration) error\n\n\t// CleanUp cleans up any objects generated during the restore expose\n\tCleanUp(context.Context, corev1api.ObjectReference)\n}\n\n// NewGenericRestoreExposer creates a new instance of generic restore exposer\nfunc NewGenericRestoreExposer(kubeClient kubernetes.Interface, log logrus.FieldLogger) GenericRestoreExposer {\n\treturn &genericRestoreExposer{\n\t\tkubeClient: kubeClient,\n\t\tlog: log,\n\t}\n}\n\ntype genericRestoreExposer struct {\n\tkubeClient kubernetes.Interface\n\tlog logrus.FieldLogger\n}\n\nfunc (e *genericRestoreExposer) Expose(ctx context.Context, ownerObject corev1api.ObjectReference, param GenericRestoreExposeParam) error {\n\tcurLog := e.log.WithFields(logrus.Fields{\n\t\t\"owner\": ownerObject.Name,\n\t\t\"target PVC\": param.TargetPVCName,\n\t\t\"target namespace\": param.TargetNamespace,\n\t})\n\n\tselectedNode, targetPVC, err := kube.WaitPVCConsumed(\n\t\tctx,\n\t\te.kubeClient.CoreV1(),\n\t\tparam.TargetPVCName,\n\t\tparam.TargetNamespace,\n\t\te.kubeClient.StorageV1(),\n\t\tparam.ExposeTimeout,\n\t\tparam.RestorePVCConfig.IgnoreDelayBinding,\n\t)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to wait target PVC consumed, %s/%s\", param.TargetNamespace, param.TargetPVCName)\n\t}\n\n\tcurLog.WithField(\"target PVC\", param.TargetPVCName).WithField(\"selected node\", selectedNode).Info(\"Target PVC is consumed\")\n\n\tif kube.IsPVCBound(targetPVC) {\n\t\treturn errors.Errorf(\"Target PVC %s/%s has already been bound, abort\", param.TargetNamespace, param.TargetPVCName)\n\t}\n\n\t// Data mover allows the StorageClass name not set for PVC.\n\tstorageClassName := \"\"\n\tif targetPVC.Spec.StorageClassName != nil {\n\t\tstorageClassName = *targetPVC.Spec.StorageClassName\n\t}\n\n\taffinity := kube.GetLoadAffinityByStorageClass(param.LoadAffinity, storageClassName, curLog)\n\n\tvar cachePVC *corev1api.PersistentVolumeClaim\n\tif param.CacheVolume != nil {\n\t\tcacheVolumeSize := getCacheVolumeSize(param.RestoreSize, param.CacheVolume)\n\t\tif cacheVolumeSize > 0 {\n\t\t\tcurLog.Infof(\"Creating cache PVC with size %v\", cacheVolumeSize)\n\n\t\t\tif pvc, err := createCachePVC(ctx, e.kubeClient.CoreV1(), ownerObject, param.CacheVolume.StorageClass, cacheVolumeSize, selectedNode); err != nil {\n\t\t\t\treturn errors.Wrap(err, \"error to create cache pvc\")\n\t\t\t} else {\n\t\t\t\tcachePVC = pvc\n\t\t\t}\n\n\t\t\tdefer func() {\n\t\t\t\tif err != nil {\n\t\t\t\t\tkube.DeletePVAndPVCIfAny(ctx, e.kubeClient.CoreV1(), cachePVC.Name, cachePVC.Namespace, 0, curLog)\n\t\t\t\t}\n\t\t\t}()\n\t\t} else {\n\t\t\tcurLog.Infof(\"Don't need to create cache volume, restore size %v, cache info %v\", param.RestoreSize, param.CacheVolume)\n\t\t}\n\t}\n\n\trestorePod, err := e.createRestorePod(\n\t\tctx,\n\t\townerObject,\n\t\ttargetPVC,\n\t\tparam.OperationTimeout,\n\t\tparam.HostingPodLabels,\n\t\tparam.HostingPodAnnotations,\n\t\tparam.HostingPodTolerations,\n\t\tselectedNode,\n\t\tparam.Resources,\n\t\tparam.NodeOS,\n\t\taffinity,\n\t\tparam.PriorityClassName,\n\t\tcachePVC,\n\t)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to create restore pod\")\n\t}\n\n\tcurLog.WithField(\"pod name\", restorePod.Name).Info(\"Restore pod is created\")\n\n\tdefer func() {\n\t\tif err != nil {\n\t\t\tkube.DeletePodIfAny(ctx, e.kubeClient.CoreV1(), restorePod.Name, restorePod.Namespace, curLog)\n\t\t}\n\t}()\n\n\trestorePVC, err := e.createRestorePVC(ctx, ownerObject, targetPVC, selectedNode)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to create restore pvc\")\n\t}\n\n\tcurLog.WithField(\"pvc name\", restorePVC.Name).Info(\"Restore PVC is created\")\n\n\tdefer func() {\n\t\tif err != nil {\n\t\t\tkube.DeletePVAndPVCIfAny(ctx, e.kubeClient.CoreV1(), restorePVC.Name, restorePVC.Namespace, 0, curLog)\n\t\t}\n\t}()\n\n\treturn nil\n}\n\nfunc (e *genericRestoreExposer) GetExposed(ctx context.Context, ownerObject corev1api.ObjectReference, nodeClient client.Client, nodeName string, timeout time.Duration) (*ExposeResult, error) {\n\trestorePodName := ownerObject.Name\n\trestorePVCName := ownerObject.Name\n\n\tcontainerName := string(ownerObject.UID)\n\tvolumeName := string(ownerObject.UID)\n\n\tcurLog := e.log.WithFields(logrus.Fields{\n\t\t\"owner\": ownerObject.Name,\n\t\t\"node\": nodeName,\n\t})\n\n\tpod := &corev1api.Pod{}\n\terr := nodeClient.Get(ctx, types.NamespacedName{\n\t\tNamespace: ownerObject.Namespace,\n\t\tName: restorePodName,\n\t}, pod)\n\tif err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tcurLog.WithField(\"restore pod\", restorePodName).Debug(\"Restore pod is not running in the current node\")\n\t\t\treturn nil, nil\n\t\t} else {\n\t\t\treturn nil, errors.Wrapf(err, \"error to get restore pod %s\", restorePodName)\n\t\t}\n\t}\n\n\tcurLog.WithField(\"pod\", pod.Name).Infof(\"Restore pod is in running state in node %s\", pod.Spec.NodeName)\n\n\t_, err = kube.WaitPVCBound(ctx, e.kubeClient.CoreV1(), e.kubeClient.CoreV1(), restorePVCName, ownerObject.Namespace, timeout)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error to wait restore PVC bound, %s\", restorePVCName)\n\t}\n\n\tcurLog.WithField(\"restore pvc\", restorePVCName).Info(\"Restore PVC is bound\")\n\n\ti := 0\n\tfor i = 0; i < len(pod.Spec.Volumes); i++ {\n\t\tif pod.Spec.Volumes[i].Name == volumeName {\n\t\t\tbreak\n\t\t}\n\t}\n\n\tif i == len(pod.Spec.Volumes) {\n\t\treturn nil, errors.Errorf(\"restore pod %s doesn't have the expected restore volume\", pod.Name)\n\t}\n\n\tcurLog.WithField(\"pod\", pod.Name).Infof(\"Restore volume is found in pod at index %v\", i)\n\n\treturn &ExposeResult{ByPod: ExposeByPod{\n\t\tHostingPod: pod,\n\t\tHostingContainer: containerName,\n\t\tVolumeName: volumeName,\n\t}}, nil\n}\n\nfunc (e *genericRestoreExposer) PeekExposed(ctx context.Context, ownerObject corev1api.ObjectReference) error {\n\trestorePodName := ownerObject.Name\n\n\tcurLog := e.log.WithFields(logrus.Fields{\n\t\t\"owner\": ownerObject.Name,\n\t})\n\n\tpod, err := e.kubeClient.CoreV1().Pods(ownerObject.Namespace).Get(ctx, restorePodName, metav1.GetOptions{})\n\tif apierrors.IsNotFound(err) {\n\t\treturn nil\n\t}\n\n\tif err != nil {\n\t\tcurLog.WithError(err).Warnf(\"error to peek restore pod %s\", restorePodName)\n\t\treturn nil\n\t}\n\n\tif podFailed, message := kube.IsPodUnrecoverable(pod, curLog); podFailed {\n\t\treturn errors.New(message)\n\t}\n\n\treturn nil\n}\n\nfunc (e *genericRestoreExposer) DiagnoseExpose(ctx context.Context, ownerObject corev1api.ObjectReference) string {\n\trestorePodName := ownerObject.Name\n\trestorePVCName := ownerObject.Name\n\n\tdiag := \"begin diagnose restore exposer\\n\"\n\n\tpod, err := e.kubeClient.CoreV1().Pods(ownerObject.Namespace).Get(ctx, restorePodName, metav1.GetOptions{})\n\tif err != nil {\n\t\tpod = nil\n\t\tdiag += fmt.Sprintf(\"error getting restore pod %s, err: %v\\n\", restorePodName, err)\n\t}\n\n\tpvc, err := e.kubeClient.CoreV1().PersistentVolumeClaims(ownerObject.Namespace).Get(ctx, restorePVCName, metav1.GetOptions{})\n\tif err != nil {\n\t\tpvc = nil\n\t\tdiag += fmt.Sprintf(\"error getting restore pvc %s, err: %v\\n\", restorePVCName, err)\n\t}\n\n\tcachePVC, err := e.kubeClient.CoreV1().PersistentVolumeClaims(ownerObject.Namespace).Get(ctx, getCachePVCName(ownerObject), metav1.GetOptions{})\n\tif err != nil {\n\t\tcachePVC = nil\n\n\t\tif !apierrors.IsNotFound(err) {\n\t\t\tdiag += fmt.Sprintf(\"error getting cache pvc %s, err: %v\\n\", getCachePVCName(ownerObject), err)\n\t\t}\n\t}\n\n\tevents, err := e.kubeClient.CoreV1().Events(ownerObject.Namespace).List(ctx, metav1.ListOptions{})\n\tif err != nil {\n\t\tdiag += fmt.Sprintf(\"error listing events, err: %v\\n\", err)\n\t}\n\n\tif pod != nil {\n\t\tdiag += kube.DiagnosePod(pod, events)\n\n\t\tif pod.Spec.NodeName != \"\" {\n\t\t\tif err := nodeagent.KbClientIsRunningInNode(ctx, ownerObject.Namespace, pod.Spec.NodeName, e.kubeClient); err != nil {\n\t\t\t\tdiag += fmt.Sprintf(\"node-agent is not running in node %s, err: %v\\n\", pod.Spec.NodeName, err)\n\t\t\t}\n\t\t}\n\t}\n\n\tif pvc != nil {\n\t\tdiag += kube.DiagnosePVC(pvc, events)\n\n\t\tif pvc.Spec.VolumeName != \"\" {\n\t\t\tif pv, err := e.kubeClient.CoreV1().PersistentVolumes().Get(ctx, pvc.Spec.VolumeName, metav1.GetOptions{}); err != nil {\n\t\t\t\tdiag += fmt.Sprintf(\"error getting restore pv %s, err: %v\\n\", pvc.Spec.VolumeName, err)\n\t\t\t} else {\n\t\t\t\tdiag += kube.DiagnosePV(pv)\n\t\t\t}\n\t\t}\n\t}\n\n\tif cachePVC != nil {\n\t\tdiag += kube.DiagnosePVC(cachePVC, events)\n\n\t\tif cachePVC.Spec.VolumeName != \"\" {\n\t\t\tif pv, err := e.kubeClient.CoreV1().PersistentVolumes().Get(ctx, cachePVC.Spec.VolumeName, metav1.GetOptions{}); err != nil {\n\t\t\t\tdiag += fmt.Sprintf(\"error getting cache pv %s, err: %v\\n\", cachePVC.Spec.VolumeName, err)\n\t\t\t} else {\n\t\t\t\tdiag += kube.DiagnosePV(pv)\n\t\t\t}\n\t\t}\n\t}\n\n\tdiag += \"end diagnose restore exposer\"\n\n\treturn diag\n}\n\nfunc (e *genericRestoreExposer) CleanUp(ctx context.Context, ownerObject corev1api.ObjectReference) {\n\trestorePodName := ownerObject.Name\n\trestorePVCName := ownerObject.Name\n\tcachePVCName := getCachePVCName(ownerObject)\n\n\tkube.DeletePodIfAny(ctx, e.kubeClient.CoreV1(), restorePodName, ownerObject.Namespace, e.log)\n\tkube.DeletePVAndPVCIfAny(ctx, e.kubeClient.CoreV1(), restorePVCName, ownerObject.Namespace, 0, e.log)\n\tkube.DeletePVAndPVCIfAny(ctx, e.kubeClient.CoreV1(), cachePVCName, ownerObject.Namespace, 0, e.log)\n}\n\nfunc (e *genericRestoreExposer) RebindVolume(ctx context.Context, ownerObject corev1api.ObjectReference, targetPVCName string, targetNamespace string, timeout time.Duration) error {\n\trestorePodName := ownerObject.Name\n\trestorePVCName := ownerObject.Name\n\n\tcurLog := e.log.WithFields(logrus.Fields{\n\t\t\"owner\": ownerObject.Name,\n\t\t\"target PVC\": targetPVCName,\n\t\t\"target namespace\": targetNamespace,\n\t})\n\n\ttargetPVC, err := e.kubeClient.CoreV1().PersistentVolumeClaims(targetNamespace).Get(ctx, targetPVCName, metav1.GetOptions{})\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to get target PVC %s/%s\", targetNamespace, targetPVCName)\n\t}\n\n\trestorePV, err := kube.WaitPVCBound(ctx, e.kubeClient.CoreV1(), e.kubeClient.CoreV1(), restorePVCName, ownerObject.Namespace, timeout)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to get PV from restore PVC %s\", restorePVCName)\n\t}\n\n\torgReclaim := restorePV.Spec.PersistentVolumeReclaimPolicy\n\n\tcurLog.WithField(\"restore PV\", restorePV.Name).Info(\"Restore PV is retrieved\")\n\n\tretained, err := kube.SetPVReclaimPolicy(ctx, e.kubeClient.CoreV1(), restorePV, corev1api.PersistentVolumeReclaimRetain)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to retain PV %s\", restorePV.Name)\n\t}\n\n\tcurLog.WithField(\"restore PV\", restorePV.Name).WithField(\"retained\", (retained != nil)).Info(\"Restore PV is retained\")\n\n\tdefer func() {\n\t\tif retained != nil {\n\t\t\tcurLog.WithField(\"retained PV\", retained.Name).Info(\"Deleting retained PV on error\")\n\t\t\tkube.DeletePVIfAny(ctx, e.kubeClient.CoreV1(), retained.Name, curLog)\n\t\t}\n\t}()\n\n\tif retained != nil {\n\t\trestorePV = retained\n\t}\n\n\terr = kube.EnsureDeletePod(ctx, e.kubeClient.CoreV1(), restorePodName, ownerObject.Namespace, timeout)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to delete restore pod %s\", restorePodName)\n\t}\n\n\terr = kube.EnsureDeletePVC(ctx, e.kubeClient.CoreV1(), restorePVCName, ownerObject.Namespace, timeout)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to delete restore PVC %s\", restorePVCName)\n\t}\n\n\tcurLog.WithField(\"restore PVC\", restorePVCName).Info(\"Restore PVC is deleted\")\n\n\t_, err = kube.RebindPVC(ctx, e.kubeClient.CoreV1(), targetPVC, restorePV.Name)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to rebind target PVC %s/%s to %s\", targetPVC.Namespace, targetPVC.Name, restorePV.Name)\n\t}\n\n\tcurLog.WithField(\"tartet PVC\", fmt.Sprintf(\"%s/%s\", targetPVC.Namespace, targetPVC.Name)).WithField(\"restore PV\", restorePV.Name).Info(\"Target PVC is rebound to restore PV\")\n\n\tvar matchLabel map[string]string\n\tif targetPVC.Spec.Selector != nil {\n\t\tmatchLabel = targetPVC.Spec.Selector.MatchLabels\n\t}\n\n\trestorePVName := restorePV.Name\n\trestorePV, err = kube.ResetPVBinding(ctx, e.kubeClient.CoreV1(), restorePV, matchLabel, targetPVC)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to reset binding info for restore PV %s\", restorePVName)\n\t}\n\n\tcurLog.WithField(\"restore PV\", restorePV.Name).Info(\"Restore PV is rebound\")\n\n\trestorePV, err = kube.WaitPVBound(ctx, e.kubeClient.CoreV1(), restorePV.Name, targetPVC.Name, targetPVC.Namespace, timeout)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to wait restore PV bound, restore PV %s\", restorePVName)\n\t}\n\n\tcurLog.WithField(\"restore PV\", restorePV.Name).Info(\"Restore PV is ready\")\n\n\tretained = nil\n\n\t_, err = kube.SetPVReclaimPolicy(ctx, e.kubeClient.CoreV1(), restorePV, orgReclaim)\n\tif err != nil {\n\t\tcurLog.WithField(\"restore PV\", restorePV.Name).WithError(err).Warn(\"Restore PV's reclaim policy is not restored\")\n\t} else {\n\t\tcurLog.WithField(\"restore PV\", restorePV.Name).Info(\"Restore PV's reclaim policy is restored\")\n\t}\n\n\treturn nil\n}\n\nfunc (e *genericRestoreExposer) createRestorePod(\n\tctx context.Context,\n\townerObject corev1api.ObjectReference,\n\ttargetPVC *corev1api.PersistentVolumeClaim,\n\toperationTimeout time.Duration,\n\tlabel map[string]string,\n\tannotation map[string]string,\n\ttoleration []corev1api.Toleration,\n\tselectedNode string,\n\tresources corev1api.ResourceRequirements,\n\tnodeOS string,\n\taffinity *kube.LoadAffinity,\n\tpriorityClassName string,\n\tcachePVC *corev1api.PersistentVolumeClaim,\n) (*corev1api.Pod, error) {\n\trestorePodName := ownerObject.Name\n\trestorePVCName := ownerObject.Name\n\n\tcontainerName := string(ownerObject.UID)\n\tvolumeName := string(ownerObject.UID)\n\n\tnodeSelector := map[string]string{}\n\tif selectedNode != \"\" {\n\t\taffinity = nil\n\t\tnodeSelector[\"kubernetes.io/hostname\"] = selectedNode\n\t\te.log.Infof(\"Selected node for restore pod. Ignore affinity from the node-agent config.\")\n\t}\n\n\tif affinity == nil {\n\t\taffinity = &kube.LoadAffinity{}\n\t}\n\n\tpodInfo, err := getInheritedPodInfo(ctx, e.kubeClient, ownerObject.Namespace, nodeOS)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error to get inherited pod info from node-agent\")\n\t}\n\n\t// Log the priority class if it's set\n\tif priorityClassName != \"\" {\n\t\te.log.Debugf(\"Setting priority class %q for data mover pod %s\", priorityClassName, restorePodName)\n\t}\n\n\tvar gracePeriod int64\n\tvolumeMounts, volumeDevices, volumePath := kube.MakePodPVCAttachment(volumeName, targetPVC.Spec.VolumeMode, false)\n\n\tvolumes := []corev1api.Volume{{\n\t\tName: volumeName,\n\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\tClaimName: restorePVCName,\n\t\t\t},\n\t\t},\n\t}}\n\n\tcacheVolumePath := \"\"\n\tif cachePVC != nil {\n\t\tmnt, _, path := kube.MakePodPVCAttachment(cacheVolumeName, nil, false)\n\t\tvolumeMounts = append(volumeMounts, mnt...)\n\n\t\tvolumes = append(volumes, corev1api.Volume{\n\t\t\tName: cacheVolumeName,\n\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\tClaimName: cachePVC.Name,\n\t\t\t\t},\n\t\t\t},\n\t\t})\n\n\t\tcacheVolumePath = path\n\t}\n\n\tvolumeMounts = append(volumeMounts, podInfo.volumeMounts...)\n\tvolumes = append(volumes, podInfo.volumes...)\n\n\tif label == nil {\n\t\tlabel = make(map[string]string)\n\t}\n\tlabel[podGroupLabel] = podGroupGenericRestore\n\n\tvolumeMode := corev1api.PersistentVolumeFilesystem\n\tif targetPVC.Spec.VolumeMode != nil {\n\t\tvolumeMode = *targetPVC.Spec.VolumeMode\n\t}\n\n\targs := []string{\n\t\tfmt.Sprintf(\"--volume-path=%s\", volumePath),\n\t\tfmt.Sprintf(\"--volume-mode=%s\", volumeMode),\n\t\tfmt.Sprintf(\"--data-download=%s\", ownerObject.Name),\n\t\tfmt.Sprintf(\"--resource-timeout=%s\", operationTimeout.String()),\n\t\tfmt.Sprintf(\"--cache-volume-path=%s\", cacheVolumePath),\n\t}\n\n\targs = append(args, podInfo.logFormatArgs...)\n\targs = append(args, podInfo.logLevelArgs...)\n\n\tvar securityCtx *corev1api.PodSecurityContext\n\tpodOS := corev1api.PodOS{}\n\tif nodeOS == kube.NodeOSWindows {\n\t\tuserID := \"ContainerAdministrator\"\n\t\tsecurityCtx = &corev1api.PodSecurityContext{\n\t\t\tWindowsOptions: &corev1api.WindowsSecurityContextOptions{\n\t\t\t\tRunAsUserName: &userID,\n\t\t\t},\n\t\t}\n\n\t\tpodOS.Name = kube.NodeOSWindows\n\n\t\taffinity.NodeSelector.MatchExpressions = append(affinity.NodeSelector.MatchExpressions, metav1.LabelSelectorRequirement{\n\t\t\tKey: kube.NodeOSLabel,\n\t\t\tValues: []string{kube.NodeOSWindows},\n\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t})\n\n\t\ttoleration = append(toleration, []corev1api.Toleration{\n\t\t\t{\n\t\t\t\tKey: \"os\",\n\t\t\t\tOperator: \"Equal\",\n\t\t\t\tEffect: \"NoSchedule\",\n\t\t\t\tValue: \"windows\",\n\t\t\t},\n\t\t\t{\n\t\t\t\tKey: \"os\",\n\t\t\t\tOperator: \"Equal\",\n\t\t\t\tEffect: \"NoExecute\",\n\t\t\t\tValue: \"windows\",\n\t\t\t},\n\t\t}...)\n\t} else {\n\t\tuserID := int64(0)\n\t\tsecurityCtx = &corev1api.PodSecurityContext{\n\t\t\tRunAsUser: &userID,\n\t\t}\n\n\t\tpodOS.Name = kube.NodeOSLinux\n\n\t\taffinity.NodeSelector.MatchExpressions = append(affinity.NodeSelector.MatchExpressions, metav1.LabelSelectorRequirement{\n\t\t\tKey: kube.NodeOSLabel,\n\t\t\tValues: []string{kube.NodeOSWindows},\n\t\t\tOperator: metav1.LabelSelectorOpNotIn,\n\t\t})\n\t}\n\n\tpodAffinity := kube.ToSystemAffinity(affinity, nil)\n\n\tpod := &corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: restorePodName,\n\t\t\tNamespace: ownerObject.Namespace,\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: ownerObject.APIVersion,\n\t\t\t\t\tKind: ownerObject.Kind,\n\t\t\t\t\tName: ownerObject.Name,\n\t\t\t\t\tUID: ownerObject.UID,\n\t\t\t\t\tController: boolptr.True(),\n\t\t\t\t},\n\t\t\t},\n\t\t\tLabels: label,\n\t\t\tAnnotations: annotation,\n\t\t},\n\t\tSpec: corev1api.PodSpec{\n\t\t\tTopologySpreadConstraints: []corev1api.TopologySpreadConstraint{\n\t\t\t\t{\n\t\t\t\t\tMaxSkew: 1,\n\t\t\t\t\tTopologyKey: \"kubernetes.io/hostname\",\n\t\t\t\t\tWhenUnsatisfiable: corev1api.ScheduleAnyway,\n\t\t\t\t\tLabelSelector: &metav1.LabelSelector{\n\t\t\t\t\t\tMatchLabels: map[string]string{\n\t\t\t\t\t\t\tpodGroupLabel: podGroupGenericRestore,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tNodeSelector: nodeSelector,\n\t\t\tOS: &podOS,\n\t\t\tContainers: []corev1api.Container{\n\t\t\t\t{\n\t\t\t\t\tName: containerName,\n\t\t\t\t\tImage: podInfo.image,\n\t\t\t\t\tImagePullPolicy: corev1api.PullNever,\n\t\t\t\t\tCommand: []string{\n\t\t\t\t\t\t\"/velero\",\n\t\t\t\t\t\t\"data-mover\",\n\t\t\t\t\t\t\"restore\",\n\t\t\t\t\t},\n\t\t\t\t\tArgs: args,\n\t\t\t\t\tVolumeMounts: volumeMounts,\n\t\t\t\t\tVolumeDevices: volumeDevices,\n\t\t\t\t\tEnv: podInfo.env,\n\t\t\t\t\tEnvFrom: podInfo.envFrom,\n\t\t\t\t\tResources: resources,\n\t\t\t\t},\n\t\t\t},\n\t\t\tPriorityClassName: priorityClassName,\n\t\t\tServiceAccountName: podInfo.serviceAccount,\n\t\t\tTerminationGracePeriodSeconds: &gracePeriod,\n\t\t\tVolumes: volumes,\n\t\t\tRestartPolicy: corev1api.RestartPolicyNever,\n\t\t\tSecurityContext: securityCtx,\n\t\t\tTolerations: toleration,\n\t\t\tDNSPolicy: podInfo.dnsPolicy,\n\t\t\tDNSConfig: podInfo.dnsConfig,\n\t\t\tAffinity: podAffinity,\n\t\t\tImagePullSecrets: podInfo.imagePullSecrets,\n\t\t},\n\t}\n\n\treturn e.kubeClient.CoreV1().Pods(ownerObject.Namespace).Create(ctx, pod, metav1.CreateOptions{})\n}\n\nfunc (e *genericRestoreExposer) createRestorePVC(ctx context.Context, ownerObject corev1api.ObjectReference, targetPVC *corev1api.PersistentVolumeClaim, selectedNode string) (*corev1api.PersistentVolumeClaim, error) {\n\trestorePVCName := ownerObject.Name\n\n\tpvcObj := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: ownerObject.Namespace,\n\t\t\tName: restorePVCName,\n\t\t\tLabels: targetPVC.Labels,\n\t\t\tAnnotations: targetPVC.Annotations,\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: ownerObject.APIVersion,\n\t\t\t\t\tKind: ownerObject.Kind,\n\t\t\t\t\tName: ownerObject.Name,\n\t\t\t\t\tUID: ownerObject.UID,\n\t\t\t\t\tController: boolptr.True(),\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tAccessModes: targetPVC.Spec.AccessModes,\n\t\t\tStorageClassName: targetPVC.Spec.StorageClassName,\n\t\t\tVolumeMode: targetPVC.Spec.VolumeMode,\n\t\t\tResources: targetPVC.Spec.Resources,\n\t\t},\n\t}\n\n\tif selectedNode != \"\" {\n\t\tpvcObj.Annotations = map[string]string{\n\t\t\tkube.KubeAnnSelectedNode: selectedNode,\n\t\t}\n\t}\n\n\treturn e.kubeClient.CoreV1().PersistentVolumeClaims(pvcObj.Namespace).Create(ctx, pvcObj, metav1.CreateOptions{})\n}\n" }, { "path": "pkg/exposer/generic_restore_priority_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage exposer\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/client-go/kubernetes/fake\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\n// TestCreateRestorePodWithPriorityClass verifies that the priority class name is properly set in the restore pod\nfunc TestCreateRestorePodWithPriorityClass(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tnodeAgentConfigMapData string\n\t\texpectedPriorityClass string\n\t\tdescription string\n\t}{\n\t\t{\n\t\t\tname: \"with priority class in config map\",\n\t\t\tnodeAgentConfigMapData: `{\n\t\t\t\t\"priorityClassName\": \"low-priority\"\n\t\t\t}`,\n\t\t\texpectedPriorityClass: \"low-priority\",\n\t\t\tdescription: \"Should set priority class from node-agent-configmap\",\n\t\t},\n\t\t{\n\t\t\tname: \"without priority class in config map\",\n\t\t\tnodeAgentConfigMapData: `{\n\t\t\t\t\"loadAffinity\": []\n\t\t\t}`,\n\t\t\texpectedPriorityClass: \"\",\n\t\t\tdescription: \"Should have empty priority class when not specified\",\n\t\t},\n\t\t{\n\t\t\tname: \"empty config map\",\n\t\t\tnodeAgentConfigMapData: `{}`,\n\t\t\texpectedPriorityClass: \"\",\n\t\t\tdescription: \"Should handle empty config map gracefully\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\n\t\t\t// Create fake Kubernetes client\n\t\t\tkubeClient := fake.NewSimpleClientset()\n\n\t\t\t// Create node-agent daemonset (required for getInheritedPodInfo)\n\t\t\tdaemonSet := &appsv1api.DaemonSet{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"node-agent\",\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t},\n\t\t\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"node-agent\",\n\t\t\t\t\t\t\t\t\tImage: \"velero/velero:latest\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t}\n\t\t\t_, err := kubeClient.AppsV1().DaemonSets(velerov1api.DefaultNamespace).Create(ctx, daemonSet, metav1.CreateOptions{})\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Create node-agent config map\n\t\t\tconfigMap := &corev1api.ConfigMap{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"node-agent-config\",\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t},\n\t\t\t\tData: map[string]string{\n\t\t\t\t\t\"config\": tc.nodeAgentConfigMapData,\n\t\t\t\t},\n\t\t\t}\n\t\t\t_, err = kubeClient.CoreV1().ConfigMaps(velerov1api.DefaultNamespace).Create(ctx, configMap, metav1.CreateOptions{})\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Create owner object for the restore pod\n\t\t\townerObject := corev1api.ObjectReference{\n\t\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\t\tKind: \"DataDownload\",\n\t\t\t\tName: \"test-datadownload\",\n\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\tUID: \"test-uid\",\n\t\t\t}\n\n\t\t\t// Create a target PVC\n\t\t\ttargetPVC := &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-target-pvc\",\n\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\tAccessModes: []corev1api.PersistentVolumeAccessMode{\n\t\t\t\t\t\tcorev1api.ReadWriteOnce,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t}\n\n\t\t\t// Create generic restore exposer\n\t\t\texposer := &genericRestoreExposer{\n\t\t\t\tkubeClient: kubeClient,\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\t// Call createRestorePod\n\t\t\tpod, err := exposer.createRestorePod(\n\t\t\t\tctx,\n\t\t\t\townerObject,\n\t\t\t\ttargetPVC,\n\t\t\t\ttime.Minute*5,\n\t\t\t\tnil, // labels\n\t\t\t\tnil, // annotations\n\t\t\t\tnil, // tolerations\n\t\t\t\t\"\", // selectedNode\n\t\t\t\tcorev1api.ResourceRequirements{},\n\t\t\t\tkube.NodeOSLinux,\n\t\t\t\tnil, // affinity\n\t\t\t\ttc.expectedPriorityClass,\n\t\t\t\tnil,\n\t\t\t)\n\n\t\t\trequire.NoError(t, err, tc.description)\n\t\t\tassert.NotNil(t, pod)\n\t\t\tassert.Equal(t, tc.expectedPriorityClass, pod.Spec.PriorityClassName, tc.description)\n\t\t})\n\t}\n}\n\nfunc TestCreateRestorePodWithMissingConfigMap(t *testing.T) {\n\tctx := t.Context()\n\n\t// Create fake Kubernetes client without config map\n\tkubeClient := fake.NewSimpleClientset()\n\n\t// Create node-agent daemonset (required for getInheritedPodInfo)\n\tdaemonSet := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"node-agent\",\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"node-agent\",\n\t\t\t\t\t\t\tImage: \"velero/velero:latest\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\t_, err := kubeClient.AppsV1().DaemonSets(velerov1api.DefaultNamespace).Create(ctx, daemonSet, metav1.CreateOptions{})\n\trequire.NoError(t, err)\n\n\t// Create owner object for the restore pod\n\townerObject := corev1api.ObjectReference{\n\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\tKind: \"DataDownload\",\n\t\tName: \"test-datadownload\",\n\t\tNamespace: velerov1api.DefaultNamespace,\n\t\tUID: \"test-uid\",\n\t}\n\n\t// Create a target PVC\n\ttargetPVC := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-target-pvc\",\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tAccessModes: []corev1api.PersistentVolumeAccessMode{\n\t\t\t\tcorev1api.ReadWriteOnce,\n\t\t\t},\n\t\t},\n\t}\n\n\t// Create generic restore exposer\n\texposer := &genericRestoreExposer{\n\t\tkubeClient: kubeClient,\n\t\tlog: velerotest.NewLogger(),\n\t}\n\n\t// Call createRestorePod\n\tpod, err := exposer.createRestorePod(\n\t\tctx,\n\t\townerObject,\n\t\ttargetPVC,\n\t\ttime.Minute*5,\n\t\tnil, // labels\n\t\tnil, // annotations\n\t\tnil, // tolerations\n\t\t\"\", // selectedNode\n\t\tcorev1api.ResourceRequirements{},\n\t\tkube.NodeOSLinux,\n\t\tnil, // affinity\n\t\t\"\", // empty priority class since config map is missing\n\t\tnil,\n\t)\n\n\t// Should succeed even when config map is missing\n\trequire.NoError(t, err, \"Should succeed even when config map is missing\")\n\tassert.NotNil(t, pod)\n\tassert.Empty(t, pod.Spec.PriorityClassName, \"Should have empty priority class when config map is missing\")\n}\n" }, { "path": "pkg/exposer/generic_restore_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage exposer\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tstoragev1api \"k8s.io/api/storage/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes/fake\"\n\tclientTesting \"k8s.io/client-go/testing\"\n\n\tvelerov1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nfunc TestRestoreExpose(t *testing.T) {\n\tscName := \"fake-sc\"\n\trestore := &velerov1.Restore{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: velerov1.SchemeGroupVersion.String(),\n\t\t\tKind: \"Restore\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-restore\",\n\t\t\tUID: \"fake-uid\",\n\t\t},\n\t}\n\n\ttargetPVCObj := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"fake-target-pvc\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tStorageClassName: &scName,\n\t\t},\n\t}\n\n\tstorageClass := &storagev1api.StorageClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-sc\",\n\t\t},\n\t}\n\n\ttargetPVCObjBound := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"fake-target-pvc\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tVolumeName: \"fake-pv\",\n\t\t},\n\t}\n\n\tdaemonSet := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"velero\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t\tAPIVersion: appsv1api.SchemeGroupVersion.String(),\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tImage: \"fake-image\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\townerRestore *velerov1.Restore\n\t\ttargetPVCName string\n\t\ttargetNamespace string\n\t\tkubeReactors []reactor\n\t\tcacheVolume *CacheConfigs\n\t\texpectBackupPod bool\n\t\texpectBackupPVC bool\n\t\texpectCachePVC bool\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"wait target pvc consumed fail\",\n\t\t\ttargetPVCName: \"fake-target-pvc\",\n\t\t\ttargetNamespace: \"fake-ns\",\n\t\t\townerRestore: restore,\n\t\t\terr: \"error to wait target PVC consumed, fake-ns/fake-target-pvc: error to wait for PVC: error to get pvc fake-ns/fake-target-pvc: persistentvolumeclaims \\\"fake-target-pvc\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"target pvc is already bound\",\n\t\t\ttargetPVCName: \"fake-target-pvc\",\n\t\t\ttargetNamespace: \"fake-ns\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\ttargetPVCObjBound,\n\t\t\t\tstorageClass,\n\t\t\t},\n\t\t\terr: \"Target PVC fake-ns/fake-target-pvc has already been bound, abort\",\n\t\t},\n\t\t{\n\t\t\tname: \"create restore pod fail\",\n\t\t\ttargetPVCName: \"fake-target-pvc\",\n\t\t\ttargetNamespace: \"fake-ns\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\ttargetPVCObj,\n\t\t\t\tdaemonSet,\n\t\t\t\tstorageClass,\n\t\t\t},\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"create\",\n\t\t\t\t\tresource: \"pods\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-create-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to create restore pod: fake-create-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"create restore pvc fail\",\n\t\t\ttargetPVCName: \"fake-target-pvc\",\n\t\t\ttargetNamespace: \"fake-ns\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\ttargetPVCObj,\n\t\t\t\tdaemonSet,\n\t\t\t\tstorageClass,\n\t\t\t},\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"create\",\n\t\t\t\t\tresource: \"persistentvolumeclaims\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-create-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to create restore pvc: fake-create-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\ttargetPVCName: \"fake-target-pvc\",\n\t\t\ttargetNamespace: \"fake-ns\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\ttargetPVCObj,\n\t\t\t\tdaemonSet,\n\t\t\t\tstorageClass,\n\t\t\t},\n\t\t\texpectBackupPod: true,\n\t\t\texpectBackupPVC: true,\n\t\t},\n\t\t{\n\t\t\tname: \"succeed, cache config, no cache volume\",\n\t\t\ttargetPVCName: \"fake-target-pvc\",\n\t\t\ttargetNamespace: \"fake-ns\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\ttargetPVCObj,\n\t\t\t\tdaemonSet,\n\t\t\t\tstorageClass,\n\t\t\t},\n\t\t\tcacheVolume: &CacheConfigs{},\n\t\t\texpectBackupPod: true,\n\t\t\texpectBackupPVC: true,\n\t\t},\n\t\t{\n\t\t\tname: \"create cache volume fail\",\n\t\t\ttargetPVCName: \"fake-target-pvc\",\n\t\t\ttargetNamespace: \"fake-ns\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\ttargetPVCObj,\n\t\t\t\tdaemonSet,\n\t\t\t\tstorageClass,\n\t\t\t},\n\t\t\tcacheVolume: &CacheConfigs{Limit: 1024},\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"create\",\n\t\t\t\t\tresource: \"persistentvolumeclaims\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-create-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to create cache pvc: fake-create-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed with cache volume\",\n\t\t\ttargetPVCName: \"fake-target-pvc\",\n\t\t\ttargetNamespace: \"fake-ns\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\ttargetPVCObj,\n\t\t\t\tdaemonSet,\n\t\t\t\tstorageClass,\n\t\t\t},\n\t\t\tcacheVolume: &CacheConfigs{Limit: 1024},\n\t\t\texpectBackupPod: true,\n\t\t\texpectBackupPVC: true,\n\t\t\texpectCachePVC: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tfor _, reactor := range test.kubeReactors {\n\t\t\t\tfakeKubeClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\texposer := genericRestoreExposer{\n\t\t\t\tkubeClient: fakeKubeClient,\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tvar ownerObject corev1api.ObjectReference\n\t\t\tif test.ownerRestore != nil {\n\t\t\t\townerObject = corev1api.ObjectReference{\n\t\t\t\t\tKind: test.ownerRestore.Kind,\n\t\t\t\t\tNamespace: test.ownerRestore.Namespace,\n\t\t\t\t\tName: test.ownerRestore.Name,\n\t\t\t\t\tUID: test.ownerRestore.UID,\n\t\t\t\t\tAPIVersion: test.ownerRestore.APIVersion,\n\t\t\t\t}\n\t\t\t}\n\n\t\t\terr := exposer.Expose(\n\t\t\t\tt.Context(),\n\t\t\t\townerObject,\n\t\t\t\tGenericRestoreExposeParam{\n\t\t\t\t\tTargetPVCName: test.targetPVCName,\n\t\t\t\t\tTargetNamespace: test.targetNamespace,\n\t\t\t\t\tHostingPodLabels: map[string]string{},\n\t\t\t\t\tResources: corev1api.ResourceRequirements{},\n\t\t\t\t\tExposeTimeout: time.Millisecond,\n\t\t\t\t\tLoadAffinity: nil,\n\t\t\t\t\tCacheVolume: test.cacheVolume,\n\t\t\t\t},\n\t\t\t)\n\n\t\t\tif test.err != \"\" {\n\t\t\t\trequire.EqualError(t, err, test.err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\t_, err = exposer.kubeClient.CoreV1().Pods(ownerObject.Namespace).Get(t.Context(), ownerObject.Name, metav1.GetOptions{})\n\t\t\tif test.expectBackupPod {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t} else {\n\t\t\t\trequire.True(t, apierrors.IsNotFound(err))\n\t\t\t}\n\n\t\t\t_, err = exposer.kubeClient.CoreV1().PersistentVolumeClaims(ownerObject.Namespace).Get(t.Context(), ownerObject.Name, metav1.GetOptions{})\n\t\t\tif test.expectBackupPVC {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t} else {\n\t\t\t\trequire.True(t, apierrors.IsNotFound(err))\n\t\t\t}\n\n\t\t\t_, err = exposer.kubeClient.CoreV1().PersistentVolumeClaims(ownerObject.Namespace).Get(t.Context(), getCachePVCName(ownerObject), metav1.GetOptions{})\n\t\t\tif test.expectCachePVC {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t} else {\n\t\t\t\trequire.True(t, apierrors.IsNotFound(err))\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestRebindVolume(t *testing.T) {\n\trestore := &velerov1.Restore{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: velerov1.SchemeGroupVersion.String(),\n\t\t\tKind: \"Restore\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-restore\",\n\t\t\tUID: \"fake-uid\",\n\t\t},\n\t}\n\n\ttargetPVCObj := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"fake-target-pvc\",\n\t\t},\n\t}\n\n\trestorePVCObj := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-restore\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tVolumeName: \"fake-restore-pv\",\n\t\t},\n\t}\n\n\trestorePVObj := &corev1api.PersistentVolume{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-restore-pv\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\tPersistentVolumeReclaimPolicy: corev1api.PersistentVolumeReclaimDelete,\n\t\t},\n\t}\n\n\trestorePod := &corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-restore\",\n\t\t},\n\t}\n\n\thookCount := 0\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\townerRestore *velerov1.Restore\n\t\ttargetPVCName string\n\t\ttargetNamespace string\n\t\tkubeReactors []reactor\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"get target pvc fail\",\n\t\t\ttargetPVCName: \"fake-target-pvc\",\n\t\t\ttargetNamespace: \"fake-ns\",\n\t\t\townerRestore: restore,\n\t\t\terr: \"error to get target PVC fake-ns/fake-target-pvc: persistentvolumeclaims \\\"fake-target-pvc\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"wait restore pvc bound fail\",\n\t\t\ttargetPVCName: \"fake-target-pvc\",\n\t\t\ttargetNamespace: \"fake-ns\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\ttargetPVCObj,\n\t\t\t},\n\t\t\terr: \"error to get PV from restore PVC fake-restore: error to wait for rediness of PVC: error to get pvc velero/fake-restore: persistentvolumeclaims \\\"fake-restore\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"retain target pv fail\",\n\t\t\ttargetPVCName: \"fake-target-pvc\",\n\t\t\ttargetNamespace: \"fake-ns\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\ttargetPVCObj,\n\t\t\t\trestorePVCObj,\n\t\t\t\trestorePVObj,\n\t\t\t},\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"patch\",\n\t\t\t\t\tresource: \"persistentvolumes\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-patch-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to retain PV fake-restore-pv: error patching PV: fake-patch-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"delete restore pod fail\",\n\t\t\ttargetPVCName: \"fake-target-pvc\",\n\t\t\ttargetNamespace: \"fake-ns\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\ttargetPVCObj,\n\t\t\t\trestorePVCObj,\n\t\t\t\trestorePVObj,\n\t\t\t\trestorePod,\n\t\t\t},\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"pods\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-delete-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to delete restore pod fake-restore: error to delete pod fake-restore: fake-delete-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"delete restore pvc fail\",\n\t\t\ttargetPVCName: \"fake-target-pvc\",\n\t\t\ttargetNamespace: \"fake-ns\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\ttargetPVCObj,\n\t\t\t\trestorePVCObj,\n\t\t\t\trestorePVObj,\n\t\t\t\trestorePod,\n\t\t\t},\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"persistentvolumeclaims\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-delete-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to delete restore PVC fake-restore: error to delete pvc fake-restore: fake-delete-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"rebind target pvc fail\",\n\t\t\ttargetPVCName: \"fake-target-pvc\",\n\t\t\ttargetNamespace: \"fake-ns\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\ttargetPVCObj,\n\t\t\t\trestorePVCObj,\n\t\t\t\trestorePVObj,\n\t\t\t\trestorePod,\n\t\t\t},\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"patch\",\n\t\t\t\t\tresource: \"persistentvolumeclaims\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-patch-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to rebind target PVC fake-ns/fake-target-pvc to fake-restore-pv: error patching PVC: fake-patch-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"reset pv binding fail\",\n\t\t\ttargetPVCName: \"fake-target-pvc\",\n\t\t\ttargetNamespace: \"fake-ns\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\ttargetPVCObj,\n\t\t\t\trestorePVCObj,\n\t\t\t\trestorePVObj,\n\t\t\t\trestorePod,\n\t\t\t},\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"patch\",\n\t\t\t\t\tresource: \"persistentvolumes\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\tif hookCount == 0 {\n\t\t\t\t\t\t\thookCount++\n\t\t\t\t\t\t\treturn false, nil, nil\n\t\t\t\t\t\t} else {\n\t\t\t\t\t\t\treturn true, nil, errors.New(\"fake-patch-error\")\n\t\t\t\t\t\t}\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to reset binding info for restore PV fake-restore-pv: error patching PV: fake-patch-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"wait restore PV bound fail\",\n\t\t\ttargetPVCName: \"fake-target-pvc\",\n\t\t\ttargetNamespace: \"fake-ns\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\ttargetPVCObj,\n\t\t\t\trestorePVCObj,\n\t\t\t\trestorePVObj,\n\t\t\t\trestorePod,\n\t\t\t},\n\t\t\terr: \"error to wait restore PV bound, restore PV fake-restore-pv: error to wait for bound of PV: context deadline exceeded\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tfor _, reactor := range test.kubeReactors {\n\t\t\t\tfakeKubeClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\texposer := genericRestoreExposer{\n\t\t\t\tkubeClient: fakeKubeClient,\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tvar ownerObject corev1api.ObjectReference\n\t\t\tif test.ownerRestore != nil {\n\t\t\t\townerObject = corev1api.ObjectReference{\n\t\t\t\t\tKind: test.ownerRestore.Kind,\n\t\t\t\t\tNamespace: test.ownerRestore.Namespace,\n\t\t\t\t\tName: test.ownerRestore.Name,\n\t\t\t\t\tUID: test.ownerRestore.UID,\n\t\t\t\t\tAPIVersion: test.ownerRestore.APIVersion,\n\t\t\t\t}\n\t\t\t}\n\n\t\t\thookCount = 0\n\n\t\t\terr := exposer.RebindVolume(t.Context(), ownerObject, test.targetPVCName, test.targetNamespace, time.Millisecond)\n\t\t\tassert.EqualError(t, err, test.err)\n\t\t})\n\t}\n}\n\nfunc TestRestorePeekExpose(t *testing.T) {\n\trestore := &velerov1.Restore{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: velerov1.SchemeGroupVersion.String(),\n\t\t\tKind: \"Restore\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-restore\",\n\t\t\tUID: \"fake-uid\",\n\t\t},\n\t}\n\n\trestorePodUrecoverable := &corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: restore.Namespace,\n\t\t\tName: restore.Name,\n\t\t},\n\t\tStatus: corev1api.PodStatus{\n\t\t\tPhase: corev1api.PodFailed,\n\t\t},\n\t}\n\n\trestorePod := &corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: restore.Namespace,\n\t\t\tName: restore.Name,\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\townerRestore *velerov1.Restore\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"restore pod is not found\",\n\t\t\townerRestore: restore,\n\t\t},\n\t\t{\n\t\t\tname: \"pod is unrecoverable\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\trestorePodUrecoverable,\n\t\t\t},\n\t\t\terr: \"Pod is in abnormal state [Failed], message []\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\trestorePod,\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\texposer := genericRestoreExposer{\n\t\t\t\tkubeClient: fakeKubeClient,\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tvar ownerObject corev1api.ObjectReference\n\t\t\tif test.ownerRestore != nil {\n\t\t\t\townerObject = corev1api.ObjectReference{\n\t\t\t\t\tKind: test.ownerRestore.Kind,\n\t\t\t\t\tNamespace: test.ownerRestore.Namespace,\n\t\t\t\t\tName: test.ownerRestore.Name,\n\t\t\t\t\tUID: test.ownerRestore.UID,\n\t\t\t\t\tAPIVersion: test.ownerRestore.APIVersion,\n\t\t\t\t}\n\t\t\t}\n\n\t\t\terr := exposer.PeekExposed(t.Context(), ownerObject)\n\t\t\tif test.err == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, test.err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc Test_ReastoreDiagnoseExpose(t *testing.T) {\n\trestore := &velerov1.Restore{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: velerov1.SchemeGroupVersion.String(),\n\t\t\tKind: \"Restore\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-restore\",\n\t\t\tUID: \"fake-uid\",\n\t\t},\n\t}\n\n\trestorePodWithoutNodeName := corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-restore\",\n\t\t\tUID: \"fake-pod-uid\",\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: restore.APIVersion,\n\t\t\t\t\tKind: restore.Kind,\n\t\t\t\t\tName: restore.Name,\n\t\t\t\t\tUID: restore.UID,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tStatus: corev1api.PodStatus{\n\t\t\tPhase: corev1api.PodPending,\n\t\t\tConditions: []corev1api.PodCondition{\n\t\t\t\t{\n\t\t\t\t\tType: corev1api.PodInitialized,\n\t\t\t\t\tStatus: corev1api.ConditionTrue,\n\t\t\t\t\tMessage: \"fake-pod-message\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tMessage: \"fake-pod-message-1\",\n\t\t},\n\t}\n\n\trestorePodWithNodeName := corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-restore\",\n\t\t\tUID: \"fake-pod-uid\",\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: restore.APIVersion,\n\t\t\t\t\tKind: restore.Kind,\n\t\t\t\t\tName: restore.Name,\n\t\t\t\t\tUID: restore.UID,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tSpec: corev1api.PodSpec{\n\t\t\tNodeName: \"fake-node\",\n\t\t},\n\t\tStatus: corev1api.PodStatus{\n\t\t\tPhase: corev1api.PodPending,\n\t\t\tConditions: []corev1api.PodCondition{\n\t\t\t\t{\n\t\t\t\t\tType: corev1api.PodInitialized,\n\t\t\t\t\tStatus: corev1api.ConditionTrue,\n\t\t\t\t\tMessage: \"fake-pod-message\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\trestorePVCWithoutVolumeName := corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-restore\",\n\t\t\tUID: \"fake-pvc-uid\",\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: restore.APIVersion,\n\t\t\t\t\tKind: restore.Kind,\n\t\t\t\t\tName: restore.Name,\n\t\t\t\t\tUID: restore.UID,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\tPhase: corev1api.ClaimPending,\n\t\t},\n\t}\n\n\trestorePVCWithVolumeName := corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-restore\",\n\t\t\tUID: \"fake-pvc-uid\",\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: restore.APIVersion,\n\t\t\t\t\tKind: restore.Kind,\n\t\t\t\t\tName: restore.Name,\n\t\t\t\t\tUID: restore.UID,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tVolumeName: \"fake-pv\",\n\t\t},\n\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\tPhase: corev1api.ClaimPending,\n\t\t},\n\t}\n\n\trestorePV := corev1api.PersistentVolume{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-pv\",\n\t\t},\n\t\tStatus: corev1api.PersistentVolumeStatus{\n\t\t\tPhase: corev1api.VolumePending,\n\t\t\tMessage: \"fake-pv-message\",\n\t\t},\n\t}\n\n\tcachePVCWithVolumeName := corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-restore-cache\",\n\t\t\tUID: \"fake-cache-pvc-uid\",\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: restore.APIVersion,\n\t\t\t\t\tKind: restore.Kind,\n\t\t\t\t\tName: restore.Name,\n\t\t\t\t\tUID: restore.UID,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tVolumeName: \"fake-pv-cache\",\n\t\t},\n\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\tPhase: corev1api.ClaimPending,\n\t\t},\n\t}\n\n\tcachePV := corev1api.PersistentVolume{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-pv-cache\",\n\t\t},\n\t\tStatus: corev1api.PersistentVolumeStatus{\n\t\t\tPhase: corev1api.VolumePending,\n\t\t\tMessage: \"fake-pv-message\",\n\t\t},\n\t}\n\n\tnodeAgentPod := corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"node-agent-pod-1\",\n\t\t\tLabels: map[string]string{\"role\": \"node-agent\"},\n\t\t},\n\t\tSpec: corev1api.PodSpec{\n\t\t\tNodeName: \"fake-node\",\n\t\t},\n\t\tStatus: corev1api.PodStatus{\n\t\t\tPhase: corev1api.PodRunning,\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\townerRestore *velerov1.Restore\n\t\tkubeClientObj []runtime.Object\n\t\texpected string\n\t}{\n\t\t{\n\t\t\tname: \"no pod, pvc\",\n\t\t\townerRestore: restore,\n\t\t\texpected: `begin diagnose restore exposer\nerror getting restore pod fake-restore, err: pods \"fake-restore\" not found\nerror getting restore pvc fake-restore, err: persistentvolumeclaims \"fake-restore\" not found\nend diagnose restore exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"pod without node name, pvc without volume name, vs without status\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&restorePodWithoutNodeName,\n\t\t\t\t&restorePVCWithoutVolumeName,\n\t\t\t},\n\t\t\texpected: `begin diagnose restore exposer\nPod velero/fake-restore, phase Pending, node name , message fake-pod-message-1\nPod condition Initialized, status True, reason , message fake-pod-message\nPVC velero/fake-restore, phase Pending, binding to \nend diagnose restore exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"pod without node name, pvc without volume name\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&restorePodWithoutNodeName,\n\t\t\t\t&restorePVCWithoutVolumeName,\n\t\t\t},\n\t\t\texpected: `begin diagnose restore exposer\nPod velero/fake-restore, phase Pending, node name , message fake-pod-message-1\nPod condition Initialized, status True, reason , message fake-pod-message\nPVC velero/fake-restore, phase Pending, binding to \nend diagnose restore exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"pod with node name, no node agent\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&restorePodWithNodeName,\n\t\t\t\t&restorePVCWithoutVolumeName,\n\t\t\t},\n\t\t\texpected: `begin diagnose restore exposer\nPod velero/fake-restore, phase Pending, node name fake-node, message \nPod condition Initialized, status True, reason , message fake-pod-message\nnode-agent is not running in node fake-node, err: daemonset pod not found in running state in node fake-node\nPVC velero/fake-restore, phase Pending, binding to \nend diagnose restore exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"pod with node name, node agent is running\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&restorePodWithNodeName,\n\t\t\t\t&restorePVCWithoutVolumeName,\n\t\t\t\t&nodeAgentPod,\n\t\t\t},\n\t\t\texpected: `begin diagnose restore exposer\nPod velero/fake-restore, phase Pending, node name fake-node, message \nPod condition Initialized, status True, reason , message fake-pod-message\nPVC velero/fake-restore, phase Pending, binding to \nend diagnose restore exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"pvc with volume name, no pv\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&restorePodWithNodeName,\n\t\t\t\t&restorePVCWithVolumeName,\n\t\t\t\t&nodeAgentPod,\n\t\t\t},\n\t\t\texpected: `begin diagnose restore exposer\nPod velero/fake-restore, phase Pending, node name fake-node, message \nPod condition Initialized, status True, reason , message fake-pod-message\nPVC velero/fake-restore, phase Pending, binding to fake-pv\nerror getting restore pv fake-pv, err: persistentvolumes \"fake-pv\" not found\nend diagnose restore exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"pvc with volume name, pv exists\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&restorePodWithNodeName,\n\t\t\t\t&restorePVCWithVolumeName,\n\t\t\t\t&restorePV,\n\t\t\t\t&nodeAgentPod,\n\t\t\t},\n\t\t\texpected: `begin diagnose restore exposer\nPod velero/fake-restore, phase Pending, node name fake-node, message \nPod condition Initialized, status True, reason , message fake-pod-message\nPVC velero/fake-restore, phase Pending, binding to fake-pv\nPV fake-pv, phase Pending, reason , message fake-pv-message\nend diagnose restore exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"cache pvc with volume name, no pv\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&restorePodWithNodeName,\n\t\t\t\t&restorePVCWithVolumeName,\n\t\t\t\t&cachePVCWithVolumeName,\n\t\t\t\t&nodeAgentPod,\n\t\t\t},\n\t\t\texpected: `begin diagnose restore exposer\nPod velero/fake-restore, phase Pending, node name fake-node, message \nPod condition Initialized, status True, reason , message fake-pod-message\nPVC velero/fake-restore, phase Pending, binding to fake-pv\nerror getting restore pv fake-pv, err: persistentvolumes \"fake-pv\" not found\nPVC velero/fake-restore-cache, phase Pending, binding to fake-pv-cache\nerror getting cache pv fake-pv-cache, err: persistentvolumes \"fake-pv-cache\" not found\nend diagnose restore exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"cache pvc with volume name, pv exists\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&restorePodWithNodeName,\n\t\t\t\t&restorePVCWithVolumeName,\n\t\t\t\t&cachePVCWithVolumeName,\n\t\t\t\t&restorePV,\n\t\t\t\t&cachePV,\n\t\t\t\t&nodeAgentPod,\n\t\t\t},\n\t\t\texpected: `begin diagnose restore exposer\nPod velero/fake-restore, phase Pending, node name fake-node, message \nPod condition Initialized, status True, reason , message fake-pod-message\nPVC velero/fake-restore, phase Pending, binding to fake-pv\nPV fake-pv, phase Pending, reason , message fake-pv-message\nPVC velero/fake-restore-cache, phase Pending, binding to fake-pv-cache\nPV fake-pv-cache, phase Pending, reason , message fake-pv-message\nend diagnose restore exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"with events\",\n\t\t\townerRestore: restore,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&restorePodWithNodeName,\n\t\t\t\t&restorePVCWithVolumeName,\n\t\t\t\t&restorePV,\n\t\t\t\t&nodeAgentPod,\n\t\t\t\t&corev1api.Event{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Namespace: velerov1.DefaultNamespace, Name: \"event-1\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-uid-1\"},\n\t\t\t\t\tReason: \"reason-1\",\n\t\t\t\t\tMessage: \"message-1\",\n\t\t\t\t},\n\t\t\t\t&corev1api.Event{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Namespace: velerov1.DefaultNamespace, Name: \"event-2\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-pod-uid\"},\n\t\t\t\t\tReason: \"reason-2\",\n\t\t\t\t\tMessage: \"message-2\",\n\t\t\t\t},\n\t\t\t\t&corev1api.Event{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Namespace: velerov1.DefaultNamespace, Name: \"event-3\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-pvc-uid\"},\n\t\t\t\t\tReason: \"reason-3\",\n\t\t\t\t\tMessage: \"message-3\",\n\t\t\t\t},\n\t\t\t\t&corev1api.Event{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Namespace: \"other-namespace\", Name: \"event-4\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-pod-uid\"},\n\t\t\t\t\tReason: \"reason-4\",\n\t\t\t\t\tMessage: \"message-4\",\n\t\t\t\t},\n\t\t\t\t&corev1api.Event{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Namespace: velerov1.DefaultNamespace, Name: \"event-5\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-pod-uid\"},\n\t\t\t\t\tReason: \"reason-5\",\n\t\t\t\t\tMessage: \"message-5\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: `begin diagnose restore exposer\nPod velero/fake-restore, phase Pending, node name fake-node, message \nPod condition Initialized, status True, reason , message fake-pod-message\nPod event reason reason-2, message message-2\nPod event reason reason-5, message message-5\nPVC velero/fake-restore, phase Pending, binding to fake-pv\nPVC event reason reason-3, message message-3\nPV fake-pv, phase Pending, reason , message fake-pv-message\nend diagnose restore exposer`,\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\te := genericRestoreExposer{\n\t\t\t\tkubeClient: fakeKubeClient,\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tvar ownerObject corev1api.ObjectReference\n\t\t\tif test.ownerRestore != nil {\n\t\t\t\townerObject = corev1api.ObjectReference{\n\t\t\t\t\tKind: test.ownerRestore.Kind,\n\t\t\t\t\tNamespace: test.ownerRestore.Namespace,\n\t\t\t\t\tName: test.ownerRestore.Name,\n\t\t\t\t\tUID: test.ownerRestore.UID,\n\t\t\t\t\tAPIVersion: test.ownerRestore.APIVersion,\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tdiag := e.DiagnoseExpose(t.Context(), ownerObject)\n\t\t\tassert.Equal(t, test.expected, diag)\n\t\t})\n\t}\n}\n\nfunc TestCreateRestorePod(t *testing.T) {\n\tscName := \"storage-class-01\"\n\n\tdaemonSet := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"velero\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t\tAPIVersion: appsv1api.SchemeGroupVersion.String(),\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tImage: \"fake-image\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tdaemonSetWin := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"velero\",\n\t\t\tName: \"node-agent-windows\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t\tAPIVersion: appsv1api.SchemeGroupVersion.String(),\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tImage: \"fake-image\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\ttargetPVCObj := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"fake-target-pvc\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tStorageClassName: &scName,\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\tselectedNode string\n\t\taffinity *kube.LoadAffinity\n\t\tnodeOS string\n\t\texpectedPod *corev1api.Pod\n\t}{\n\t\t{\n\t\t\tname: \"linux\",\n\t\t\tkubeClientObj: []runtime.Object{daemonSet, daemonSetWin, targetPVCObj},\n\t\t\tselectedNode: \"\",\n\t\t\taffinity: &kube.LoadAffinity{\n\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\t\tValues: []string{\"linux\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStorageClass: scName,\n\t\t\t},\n\t\t\tnodeOS: \"linux\",\n\t\t},\n\t\t{\n\t\t\tname: \"windows\",\n\t\t\tkubeClientObj: []runtime.Object{daemonSet, daemonSetWin, targetPVCObj},\n\t\t\tselectedNode: \"\",\n\t\t\taffinity: &kube.LoadAffinity{\n\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\t\tValues: []string{\"windows\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStorageClass: scName,\n\t\t\t},\n\t\t\tnodeOS: \"windows\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\texposer := genericRestoreExposer{\n\t\t\t\tkubeClient: fakeKubeClient,\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tpod, err := exposer.createRestorePod(\n\t\t\t\tt.Context(),\n\t\t\t\tcorev1api.ObjectReference{\n\t\t\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\t\t\tName: \"data-download\",\n\t\t\t\t},\n\t\t\t\ttargetPVCObj,\n\t\t\t\ttime.Second*3,\n\t\t\t\tnil,\n\t\t\t\tnil,\n\t\t\t\tnil,\n\t\t\t\ttest.selectedNode,\n\t\t\t\tcorev1api.ResourceRequirements{},\n\t\t\t\ttest.nodeOS,\n\t\t\t\ttest.affinity,\n\t\t\t\t\"\", // priority class name\n\t\t\t\tnil,\n\t\t\t)\n\n\t\t\trequire.NoError(t, err)\n\t\t\tif test.expectedPod != nil {\n\t\t\t\tassert.Equal(t, test.expectedPod, pod)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/exposer/host_path.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage exposer\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strings\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/client-go/kubernetes\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\t\"github.com/vmware-tanzu/velero/pkg/nodeagent\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nvar getVolumeDirectory = kube.GetVolumeDirectory\nvar getVolumeMode = kube.GetVolumeMode\nvar singlePathMatch = kube.SinglePathMatch\n\n// GetPodVolumeHostPath returns a path that can be accessed from the host for a given volume of a pod\nfunc GetPodVolumeHostPath(ctx context.Context, pod *corev1api.Pod, volumeName string,\n\tkubeClient kubernetes.Interface, fs filesystem.Interface, log logrus.FieldLogger) (datapath.AccessPoint, error) {\n\tlogger := log.WithField(\"pod name\", pod.Name).WithField(\"pod UID\", pod.GetUID()).WithField(\"volume\", volumeName)\n\n\tvolDir, err := getVolumeDirectory(ctx, logger, pod, volumeName, kubeClient)\n\tif err != nil {\n\t\treturn datapath.AccessPoint{}, errors.Wrapf(err, \"error getting volume directory name for volume %s in pod %s\", volumeName, pod.Name)\n\t}\n\n\tlogger.WithField(\"volDir\", volDir).Info(\"Got volume dir\")\n\n\tvolMode, err := getVolumeMode(ctx, logger, pod, volumeName, kubeClient)\n\tif err != nil {\n\t\treturn datapath.AccessPoint{}, errors.Wrapf(err, \"error getting volume mode for volume %s in pod %s\", volumeName, pod.Name)\n\t}\n\n\tvolSubDir := \"volumes\"\n\tif volMode == uploader.PersistentVolumeBlock {\n\t\tvolSubDir = \"volumeDevices\"\n\t}\n\n\tpathGlob := fmt.Sprintf(\"%s/%s/%s/*/%s\", nodeagent.HostPodVolumeMountPath(), string(pod.GetUID()), volSubDir, volDir)\n\tlogger.WithField(\"pathGlob\", pathGlob).Debug(\"Looking for path matching glob\")\n\n\tpath, err := singlePathMatch(pathGlob, fs, logger)\n\tif err != nil {\n\t\treturn datapath.AccessPoint{}, errors.Wrapf(err, \"error identifying unique volume path on host for volume %s in pod %s\", volumeName, pod.Name)\n\t}\n\n\tlogger.WithField(\"path\", path).Info(\"Found path matching glob\")\n\n\treturn datapath.AccessPoint{\n\t\tByPath: path,\n\t\tVolMode: volMode,\n\t}, nil\n}\n\nvar getHostPodPath = nodeagent.GetHostPodPath\n\nfunc ExtractPodVolumeHostPath(ctx context.Context, path string, kubeClient kubernetes.Interface, veleroNamespace string, osType string) (string, error) {\n\tpodPath, err := getHostPodPath(ctx, kubeClient, veleroNamespace, osType)\n\tif err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error getting host pod path from node-agent\")\n\t}\n\n\tif osType == kube.NodeOSWindows {\n\t\tpodPath = strings.Replace(podPath, \"/\", \"\\\\\", -1)\n\t}\n\n\tif osType == kube.NodeOSWindows {\n\t\treturn strings.Replace(path, nodeagent.HostPodVolumeMountPathWin(), podPath, 1), nil\n\t} else {\n\t\treturn strings.Replace(path, nodeagent.HostPodVolumeMountPath(), podPath, 1), nil\n\t}\n}\n" }, { "path": "pkg/exposer/host_path_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage exposer\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/client-go/kubernetes\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/nodeagent\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nfunc TestGetPodVolumeHostPath(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tgetVolumeDirFunc func(context.Context, logrus.FieldLogger, *corev1api.Pod, string, kubernetes.Interface) (string, error)\n\t\tgetVolumeModeFunc func(context.Context, logrus.FieldLogger, *corev1api.Pod, string, kubernetes.Interface) (uploader.PersistentVolumeMode, error)\n\t\tpathMatchFunc func(string, filesystem.Interface, logrus.FieldLogger) (string, error)\n\t\tpod *corev1api.Pod\n\t\tpvc string\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"get volume dir fail\",\n\t\t\tgetVolumeDirFunc: func(context.Context, logrus.FieldLogger, *corev1api.Pod, string, kubernetes.Interface) (string, error) {\n\t\t\t\treturn \"\", errors.New(\"fake-error-1\")\n\t\t\t},\n\t\t\tpod: builder.ForPod(velerov1api.DefaultNamespace, \"fake-pod-1\").Result(),\n\t\t\tpvc: \"fake-pvc-1\",\n\t\t\terr: \"error getting volume directory name for volume fake-pvc-1 in pod fake-pod-1: fake-error-1\",\n\t\t},\n\t\t{\n\t\t\tname: \"single path match fail\",\n\t\t\tgetVolumeDirFunc: func(context.Context, logrus.FieldLogger, *corev1api.Pod, string, kubernetes.Interface) (string, error) {\n\t\t\t\treturn \"\", nil\n\t\t\t},\n\t\t\tgetVolumeModeFunc: func(context.Context, logrus.FieldLogger, *corev1api.Pod, string, kubernetes.Interface) (uploader.PersistentVolumeMode, error) {\n\t\t\t\treturn uploader.PersistentVolumeFilesystem, nil\n\t\t\t},\n\t\t\tpathMatchFunc: func(string, filesystem.Interface, logrus.FieldLogger) (string, error) {\n\t\t\t\treturn \"\", errors.New(\"fake-error-2\")\n\t\t\t},\n\t\t\tpod: builder.ForPod(velerov1api.DefaultNamespace, \"fake-pod-2\").Result(),\n\t\t\tpvc: \"fake-pvc-1\",\n\t\t\terr: \"error identifying unique volume path on host for volume fake-pvc-1 in pod fake-pod-2: fake-error-2\",\n\t\t},\n\t\t{\n\t\t\tname: \"get block volume dir success\",\n\t\t\tgetVolumeDirFunc: func(context.Context, logrus.FieldLogger, *corev1api.Pod, string, kubernetes.Interface) (\n\t\t\t\tstring, error) {\n\t\t\t\treturn \"fake-pvc-1\", nil\n\t\t\t},\n\t\t\tpathMatchFunc: func(string, filesystem.Interface, logrus.FieldLogger) (string, error) {\n\t\t\t\treturn \"/host_pods/fake-pod-1-id/volumeDevices/kubernetes.io~csi/fake-pvc-1-id\", nil\n\t\t\t},\n\t\t\tpod: builder.ForPod(velerov1api.DefaultNamespace, \"fake-pod-1\").Result(),\n\t\t\tpvc: \"fake-pvc-1\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tif test.getVolumeDirFunc != nil {\n\t\t\t\tgetVolumeDirectory = test.getVolumeDirFunc\n\t\t\t}\n\n\t\t\tif test.getVolumeModeFunc != nil {\n\t\t\t\tgetVolumeMode = test.getVolumeModeFunc\n\t\t\t}\n\n\t\t\tif test.pathMatchFunc != nil {\n\t\t\t\tsinglePathMatch = test.pathMatchFunc\n\t\t\t}\n\n\t\t\t_, err := GetPodVolumeHostPath(t.Context(), test.pod, test.pvc, nil, nil, velerotest.NewLogger())\n\t\t\tif test.err != \"\" || err != nil {\n\t\t\t\tassert.EqualError(t, err, test.err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestExtractPodVolumeHostPath(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tgetHostPodPathFunc func(context.Context, kubernetes.Interface, string, string) (string, error)\n\t\tpath string\n\t\tosType string\n\t\texpectedErr string\n\t\texpected string\n\t}{\n\t\t{\n\t\t\tname: \"get host pod path error\",\n\t\t\tgetHostPodPathFunc: func(context.Context, kubernetes.Interface, string, string) (string, error) {\n\t\t\t\treturn \"\", errors.New(\"fake-error-1\")\n\t\t\t},\n\n\t\t\texpectedErr: \"error getting host pod path from node-agent: fake-error-1\",\n\t\t},\n\t\t{\n\t\t\tname: \"Windows os\",\n\t\t\tgetHostPodPathFunc: func(context.Context, kubernetes.Interface, string, string) (string, error) {\n\t\t\t\treturn \"/var/lib/kubelet/pods\", nil\n\t\t\t},\n\t\t\tpath: fmt.Sprintf(\"\\\\%s\\\\pod-id-xxx\\\\volumes\\\\kubernetes.io~csi\\\\pvc-id-xxx\\\\mount\", nodeagent.HostPodVolumeMountPoint),\n\t\t\tosType: kube.NodeOSWindows,\n\t\t\texpected: \"\\\\var\\\\lib\\\\kubelet\\\\pods\\\\pod-id-xxx\\\\volumes\\\\kubernetes.io~csi\\\\pvc-id-xxx\\\\mount\",\n\t\t},\n\t\t{\n\t\t\tname: \"linux OS\",\n\t\t\tgetHostPodPathFunc: func(context.Context, kubernetes.Interface, string, string) (string, error) {\n\t\t\t\treturn \"/var/lib/kubelet/pods\", nil\n\t\t\t},\n\t\t\tpath: fmt.Sprintf(\"/%s/pod-id-xxx/volumes/kubernetes.io~csi/pvc-id-xxx/mount\", nodeagent.HostPodVolumeMountPoint),\n\t\t\tosType: kube.NodeOSLinux,\n\t\t\texpected: \"/var/lib/kubelet/pods/pod-id-xxx/volumes/kubernetes.io~csi/pvc-id-xxx/mount\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tif test.getHostPodPathFunc != nil {\n\t\t\t\tgetHostPodPath = test.getHostPodPathFunc\n\t\t\t}\n\n\t\t\tpath, err := ExtractPodVolumeHostPath(t.Context(), test.path, nil, \"\", test.osType)\n\n\t\t\tif test.expectedErr != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.expectedErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, test.expected, path)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/exposer/image.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage exposer\n\nimport (\n\t\"context\"\n\t\"strings\"\n\n\t\"github.com/pkg/errors\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/client-go/kubernetes\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/nodeagent\"\n)\n\ntype inheritedPodInfo struct {\n\timage string\n\tserviceAccount string\n\tenv []corev1api.EnvVar\n\tenvFrom []corev1api.EnvFromSource\n\tvolumeMounts []corev1api.VolumeMount\n\tvolumes []corev1api.Volume\n\tlogLevelArgs []string\n\tlogFormatArgs []string\n\tdnsPolicy corev1api.DNSPolicy\n\tdnsConfig *corev1api.PodDNSConfig\n\timagePullSecrets []corev1api.LocalObjectReference\n}\n\nfunc getInheritedPodInfo(ctx context.Context, client kubernetes.Interface, veleroNamespace string, osType string) (inheritedPodInfo, error) {\n\tpodInfo := inheritedPodInfo{}\n\n\tpodSpec, err := nodeagent.GetPodSpec(ctx, client, veleroNamespace, osType)\n\tif err != nil {\n\t\treturn podInfo, errors.Wrap(err, \"error to get node-agent pod template\")\n\t}\n\n\tif len(podSpec.Containers) != 1 {\n\t\treturn podInfo, errors.New(\"unexpected pod template from node-agent\")\n\t}\n\n\tpodInfo.image = podSpec.Containers[0].Image\n\tpodInfo.serviceAccount = podSpec.ServiceAccountName\n\n\tpodInfo.env = podSpec.Containers[0].Env\n\tpodInfo.envFrom = podSpec.Containers[0].EnvFrom\n\tpodInfo.volumeMounts = podSpec.Containers[0].VolumeMounts\n\tpodInfo.volumes = podSpec.Volumes\n\n\tpodInfo.dnsPolicy = podSpec.DNSPolicy\n\tpodInfo.dnsConfig = podSpec.DNSConfig\n\n\targs := podSpec.Containers[0].Args\n\tfor i, arg := range args {\n\t\tif arg == \"--log-format\" {\n\t\t\tpodInfo.logFormatArgs = append(podInfo.logFormatArgs, args[i:i+2]...)\n\t\t} else if strings.HasPrefix(arg, \"--log-format\") {\n\t\t\tpodInfo.logFormatArgs = append(podInfo.logFormatArgs, arg)\n\t\t} else if arg == \"--log-level\" {\n\t\t\tpodInfo.logLevelArgs = append(podInfo.logLevelArgs, args[i:i+2]...)\n\t\t} else if strings.HasPrefix(arg, \"--log-level\") {\n\t\t\tpodInfo.logLevelArgs = append(podInfo.logLevelArgs, arg)\n\t\t}\n\t}\n\n\tpodInfo.imagePullSecrets = podSpec.ImagePullSecrets\n\n\treturn podInfo, nil\n}\n" }, { "path": "pkg/exposer/image_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage exposer\n\nimport (\n\t\"reflect\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/client-go/kubernetes/fake\"\n)\n\nfunc TestGetInheritedPodInfo(t *testing.T) {\n\tdaemonSet := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t},\n\t}\n\n\tdaemonSetWithNoLog := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"container-1\",\n\t\t\t\t\t\t\tImage: \"image-1\",\n\t\t\t\t\t\t\tEnv: []corev1api.EnvVar{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"env-1\",\n\t\t\t\t\t\t\t\t\tValue: \"value-1\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"env-2\",\n\t\t\t\t\t\t\t\t\tValue: \"value-2\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tEnvFrom: []corev1api.EnvFromSource{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tConfigMapRef: &corev1api.ConfigMapEnvSource{\n\t\t\t\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\t\t\t\tName: \"test-configmap\",\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tSecretRef: &corev1api.SecretEnvSource{\n\t\t\t\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\t\t\t\tName: \"test-secret\",\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tVolumeMounts: []corev1api.VolumeMount{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"volume-1\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"volume-2\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"volume-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"volume-2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tServiceAccountName: \"sa-1\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tdaemonSetWithLog := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"container-1\",\n\t\t\t\t\t\t\tImage: \"image-1\",\n\t\t\t\t\t\t\tEnv: []corev1api.EnvVar{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"env-1\",\n\t\t\t\t\t\t\t\t\tValue: \"value-1\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"env-2\",\n\t\t\t\t\t\t\t\t\tValue: \"value-2\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tEnvFrom: []corev1api.EnvFromSource{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tConfigMapRef: &corev1api.ConfigMapEnvSource{\n\t\t\t\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\t\t\t\tName: \"test-configmap\",\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tSecretRef: &corev1api.SecretEnvSource{\n\t\t\t\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\t\t\t\tName: \"test-secret\",\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tVolumeMounts: []corev1api.VolumeMount{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"volume-1\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"volume-2\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tArgs: []string{\n\t\t\t\t\t\t\t\t\"--log-format=json\",\n\t\t\t\t\t\t\t\t\"--log-level\",\n\t\t\t\t\t\t\t\t\"debug\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tCommand: []string{\n\t\t\t\t\t\t\t\t\"command-1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"volume-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"volume-2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tServiceAccountName: \"sa-1\",\n\t\t\t\t\tImagePullSecrets: []corev1api.LocalObjectReference{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"imagePullSecret1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tscheme := runtime.NewScheme()\n\tappsv1api.AddToScheme(scheme)\n\n\ttests := []struct {\n\t\tname string\n\t\tnamespace string\n\t\tclient kubernetes.Interface\n\t\tkubeClientObj []runtime.Object\n\t\tresult inheritedPodInfo\n\t\texpectErr string\n\t}{\n\t\t{\n\t\t\tname: \"ds is not found\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\texpectErr: \"error to get node-agent pod template: error to get node-agent daemonset: daemonsets.apps \\\"node-agent\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"ds pod container number is invalidate\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSet,\n\t\t\t},\n\t\t\texpectErr: \"unexpected pod template from node-agent\",\n\t\t},\n\t\t{\n\t\t\tname: \"no log info\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSetWithNoLog,\n\t\t\t},\n\t\t\tresult: inheritedPodInfo{\n\t\t\t\timage: \"image-1\",\n\t\t\t\tserviceAccount: \"sa-1\",\n\t\t\t\tenv: []corev1api.EnvVar{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"env-1\",\n\t\t\t\t\t\tValue: \"value-1\",\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"env-2\",\n\t\t\t\t\t\tValue: \"value-2\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tenvFrom: []corev1api.EnvFromSource{\n\t\t\t\t\t{\n\t\t\t\t\t\tConfigMapRef: &corev1api.ConfigMapEnvSource{\n\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\tName: \"test-configmap\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tSecretRef: &corev1api.SecretEnvSource{\n\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\tName: \"test-secret\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tvolumeMounts: []corev1api.VolumeMount{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"volume-1\",\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"volume-2\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tvolumes: []corev1api.Volume{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"volume-1\",\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"volume-2\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with log info\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSetWithLog,\n\t\t\t},\n\t\t\tresult: inheritedPodInfo{\n\t\t\t\timage: \"image-1\",\n\t\t\t\tserviceAccount: \"sa-1\",\n\t\t\t\tenv: []corev1api.EnvVar{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"env-1\",\n\t\t\t\t\t\tValue: \"value-1\",\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"env-2\",\n\t\t\t\t\t\tValue: \"value-2\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tenvFrom: []corev1api.EnvFromSource{\n\t\t\t\t\t{\n\t\t\t\t\t\tConfigMapRef: &corev1api.ConfigMapEnvSource{\n\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\tName: \"test-configmap\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tSecretRef: &corev1api.SecretEnvSource{\n\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\tName: \"test-secret\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tvolumeMounts: []corev1api.VolumeMount{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"volume-1\",\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"volume-2\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tvolumes: []corev1api.Volume{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"volume-1\",\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"volume-2\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tlogFormatArgs: []string{\n\t\t\t\t\t\"--log-format=json\",\n\t\t\t\t},\n\t\t\t\tlogLevelArgs: []string{\n\t\t\t\t\t\"--log-level\",\n\t\t\t\t\t\"debug\",\n\t\t\t\t},\n\t\t\t\timagePullSecrets: []corev1api.LocalObjectReference{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"imagePullSecret1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\t\t\tinfo, err := getInheritedPodInfo(t.Context(), fakeKubeClient, test.namespace, kube.NodeOSLinux)\n\n\t\t\tif test.expectErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.True(t, reflect.DeepEqual(info, test.result))\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, test.expectErr)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/exposer/mocks/GenericRestoreExposer.go", "content": "// Code generated by mockery; DO NOT EDIT.\n// github.com/vektra/mockery\n// template: testify\n\npackage mocks\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\tmock \"github.com/stretchr/testify/mock\"\n\t\"github.com/vmware-tanzu/velero/pkg/exposer\"\n\t\"k8s.io/api/core/v1\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n)\n\n// NewMockGenericRestoreExposer creates a new instance of MockGenericRestoreExposer. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\n// The first argument is typically a *testing.T value.\nfunc NewMockGenericRestoreExposer(t interface {\n\tmock.TestingT\n\tCleanup(func())\n}) *MockGenericRestoreExposer {\n\tmock := &MockGenericRestoreExposer{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n\n// MockGenericRestoreExposer is an autogenerated mock type for the GenericRestoreExposer type\ntype MockGenericRestoreExposer struct {\n\tmock.Mock\n}\n\ntype MockGenericRestoreExposer_Expecter struct {\n\tmock *mock.Mock\n}\n\nfunc (_m *MockGenericRestoreExposer) EXPECT() *MockGenericRestoreExposer_Expecter {\n\treturn &MockGenericRestoreExposer_Expecter{mock: &_m.Mock}\n}\n\n// CleanUp provides a mock function for the type MockGenericRestoreExposer\nfunc (_mock *MockGenericRestoreExposer) CleanUp(context1 context.Context, objectReference v1.ObjectReference) {\n\t_mock.Called(context1, objectReference)\n\treturn\n}\n\n// MockGenericRestoreExposer_CleanUp_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'CleanUp'\ntype MockGenericRestoreExposer_CleanUp_Call struct {\n\t*mock.Call\n}\n\n// CleanUp is a helper method to define mock.On call\n// - context1 context.Context\n// - objectReference v1.ObjectReference\nfunc (_e *MockGenericRestoreExposer_Expecter) CleanUp(context1 interface{}, objectReference interface{}) *MockGenericRestoreExposer_CleanUp_Call {\n\treturn &MockGenericRestoreExposer_CleanUp_Call{Call: _e.mock.On(\"CleanUp\", context1, objectReference)}\n}\n\nfunc (_c *MockGenericRestoreExposer_CleanUp_Call) Run(run func(context1 context.Context, objectReference v1.ObjectReference)) *MockGenericRestoreExposer_CleanUp_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 v1.ObjectReference\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(v1.ObjectReference)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockGenericRestoreExposer_CleanUp_Call) Return() *MockGenericRestoreExposer_CleanUp_Call {\n\t_c.Call.Return()\n\treturn _c\n}\n\nfunc (_c *MockGenericRestoreExposer_CleanUp_Call) RunAndReturn(run func(context1 context.Context, objectReference v1.ObjectReference)) *MockGenericRestoreExposer_CleanUp_Call {\n\t_c.Run(run)\n\treturn _c\n}\n\n// DiagnoseExpose provides a mock function for the type MockGenericRestoreExposer\nfunc (_mock *MockGenericRestoreExposer) DiagnoseExpose(context1 context.Context, objectReference v1.ObjectReference) string {\n\tret := _mock.Called(context1, objectReference)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for DiagnoseExpose\")\n\t}\n\n\tvar r0 string\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, v1.ObjectReference) string); ok {\n\t\tr0 = returnFunc(context1, objectReference)\n\t} else {\n\t\tr0 = ret.Get(0).(string)\n\t}\n\treturn r0\n}\n\n// MockGenericRestoreExposer_DiagnoseExpose_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'DiagnoseExpose'\ntype MockGenericRestoreExposer_DiagnoseExpose_Call struct {\n\t*mock.Call\n}\n\n// DiagnoseExpose is a helper method to define mock.On call\n// - context1 context.Context\n// - objectReference v1.ObjectReference\nfunc (_e *MockGenericRestoreExposer_Expecter) DiagnoseExpose(context1 interface{}, objectReference interface{}) *MockGenericRestoreExposer_DiagnoseExpose_Call {\n\treturn &MockGenericRestoreExposer_DiagnoseExpose_Call{Call: _e.mock.On(\"DiagnoseExpose\", context1, objectReference)}\n}\n\nfunc (_c *MockGenericRestoreExposer_DiagnoseExpose_Call) Run(run func(context1 context.Context, objectReference v1.ObjectReference)) *MockGenericRestoreExposer_DiagnoseExpose_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 v1.ObjectReference\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(v1.ObjectReference)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockGenericRestoreExposer_DiagnoseExpose_Call) Return(s string) *MockGenericRestoreExposer_DiagnoseExpose_Call {\n\t_c.Call.Return(s)\n\treturn _c\n}\n\nfunc (_c *MockGenericRestoreExposer_DiagnoseExpose_Call) RunAndReturn(run func(context1 context.Context, objectReference v1.ObjectReference) string) *MockGenericRestoreExposer_DiagnoseExpose_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// Expose provides a mock function for the type MockGenericRestoreExposer\nfunc (_mock *MockGenericRestoreExposer) Expose(context1 context.Context, objectReference v1.ObjectReference, genericRestoreExposeParam exposer.GenericRestoreExposeParam) error {\n\tret := _mock.Called(context1, objectReference, genericRestoreExposeParam)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Expose\")\n\t}\n\n\tvar r0 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, v1.ObjectReference, exposer.GenericRestoreExposeParam) error); ok {\n\t\tr0 = returnFunc(context1, objectReference, genericRestoreExposeParam)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\treturn r0\n}\n\n// MockGenericRestoreExposer_Expose_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Expose'\ntype MockGenericRestoreExposer_Expose_Call struct {\n\t*mock.Call\n}\n\n// Expose is a helper method to define mock.On call\n// - context1 context.Context\n// - objectReference v1.ObjectReference\n// - genericRestoreExposeParam exposer.GenericRestoreExposeParam\nfunc (_e *MockGenericRestoreExposer_Expecter) Expose(context1 interface{}, objectReference interface{}, genericRestoreExposeParam interface{}) *MockGenericRestoreExposer_Expose_Call {\n\treturn &MockGenericRestoreExposer_Expose_Call{Call: _e.mock.On(\"Expose\", context1, objectReference, genericRestoreExposeParam)}\n}\n\nfunc (_c *MockGenericRestoreExposer_Expose_Call) Run(run func(context1 context.Context, objectReference v1.ObjectReference, genericRestoreExposeParam exposer.GenericRestoreExposeParam)) *MockGenericRestoreExposer_Expose_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 v1.ObjectReference\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(v1.ObjectReference)\n\t\t}\n\t\tvar arg2 exposer.GenericRestoreExposeParam\n\t\tif args[2] != nil {\n\t\t\targ2 = args[2].(exposer.GenericRestoreExposeParam)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t\targ2,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockGenericRestoreExposer_Expose_Call) Return(err error) *MockGenericRestoreExposer_Expose_Call {\n\t_c.Call.Return(err)\n\treturn _c\n}\n\nfunc (_c *MockGenericRestoreExposer_Expose_Call) RunAndReturn(run func(context1 context.Context, objectReference v1.ObjectReference, genericRestoreExposeParam exposer.GenericRestoreExposeParam) error) *MockGenericRestoreExposer_Expose_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// GetExposed provides a mock function for the type MockGenericRestoreExposer\nfunc (_mock *MockGenericRestoreExposer) GetExposed(context1 context.Context, objectReference v1.ObjectReference, client1 client.Client, s string, duration time.Duration) (*exposer.ExposeResult, error) {\n\tret := _mock.Called(context1, objectReference, client1, s, duration)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetExposed\")\n\t}\n\n\tvar r0 *exposer.ExposeResult\n\tvar r1 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, v1.ObjectReference, client.Client, string, time.Duration) (*exposer.ExposeResult, error)); ok {\n\t\treturn returnFunc(context1, objectReference, client1, s, duration)\n\t}\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, v1.ObjectReference, client.Client, string, time.Duration) *exposer.ExposeResult); ok {\n\t\tr0 = returnFunc(context1, objectReference, client1, s, duration)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(*exposer.ExposeResult)\n\t\t}\n\t}\n\tif returnFunc, ok := ret.Get(1).(func(context.Context, v1.ObjectReference, client.Client, string, time.Duration) error); ok {\n\t\tr1 = returnFunc(context1, objectReference, client1, s, duration)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\treturn r0, r1\n}\n\n// MockGenericRestoreExposer_GetExposed_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'GetExposed'\ntype MockGenericRestoreExposer_GetExposed_Call struct {\n\t*mock.Call\n}\n\n// GetExposed is a helper method to define mock.On call\n// - context1 context.Context\n// - objectReference v1.ObjectReference\n// - client1 client.Client\n// - s string\n// - duration time.Duration\nfunc (_e *MockGenericRestoreExposer_Expecter) GetExposed(context1 interface{}, objectReference interface{}, client1 interface{}, s interface{}, duration interface{}) *MockGenericRestoreExposer_GetExposed_Call {\n\treturn &MockGenericRestoreExposer_GetExposed_Call{Call: _e.mock.On(\"GetExposed\", context1, objectReference, client1, s, duration)}\n}\n\nfunc (_c *MockGenericRestoreExposer_GetExposed_Call) Run(run func(context1 context.Context, objectReference v1.ObjectReference, client1 client.Client, s string, duration time.Duration)) *MockGenericRestoreExposer_GetExposed_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 v1.ObjectReference\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(v1.ObjectReference)\n\t\t}\n\t\tvar arg2 client.Client\n\t\tif args[2] != nil {\n\t\t\targ2 = args[2].(client.Client)\n\t\t}\n\t\tvar arg3 string\n\t\tif args[3] != nil {\n\t\t\targ3 = args[3].(string)\n\t\t}\n\t\tvar arg4 time.Duration\n\t\tif args[4] != nil {\n\t\t\targ4 = args[4].(time.Duration)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t\targ2,\n\t\t\targ3,\n\t\t\targ4,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockGenericRestoreExposer_GetExposed_Call) Return(exposeResult *exposer.ExposeResult, err error) *MockGenericRestoreExposer_GetExposed_Call {\n\t_c.Call.Return(exposeResult, err)\n\treturn _c\n}\n\nfunc (_c *MockGenericRestoreExposer_GetExposed_Call) RunAndReturn(run func(context1 context.Context, objectReference v1.ObjectReference, client1 client.Client, s string, duration time.Duration) (*exposer.ExposeResult, error)) *MockGenericRestoreExposer_GetExposed_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// PeekExposed provides a mock function for the type MockGenericRestoreExposer\nfunc (_mock *MockGenericRestoreExposer) PeekExposed(context1 context.Context, objectReference v1.ObjectReference) error {\n\tret := _mock.Called(context1, objectReference)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PeekExposed\")\n\t}\n\n\tvar r0 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, v1.ObjectReference) error); ok {\n\t\tr0 = returnFunc(context1, objectReference)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\treturn r0\n}\n\n// MockGenericRestoreExposer_PeekExposed_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'PeekExposed'\ntype MockGenericRestoreExposer_PeekExposed_Call struct {\n\t*mock.Call\n}\n\n// PeekExposed is a helper method to define mock.On call\n// - context1 context.Context\n// - objectReference v1.ObjectReference\nfunc (_e *MockGenericRestoreExposer_Expecter) PeekExposed(context1 interface{}, objectReference interface{}) *MockGenericRestoreExposer_PeekExposed_Call {\n\treturn &MockGenericRestoreExposer_PeekExposed_Call{Call: _e.mock.On(\"PeekExposed\", context1, objectReference)}\n}\n\nfunc (_c *MockGenericRestoreExposer_PeekExposed_Call) Run(run func(context1 context.Context, objectReference v1.ObjectReference)) *MockGenericRestoreExposer_PeekExposed_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 v1.ObjectReference\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(v1.ObjectReference)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockGenericRestoreExposer_PeekExposed_Call) Return(err error) *MockGenericRestoreExposer_PeekExposed_Call {\n\t_c.Call.Return(err)\n\treturn _c\n}\n\nfunc (_c *MockGenericRestoreExposer_PeekExposed_Call) RunAndReturn(run func(context1 context.Context, objectReference v1.ObjectReference) error) *MockGenericRestoreExposer_PeekExposed_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// RebindVolume provides a mock function for the type MockGenericRestoreExposer\nfunc (_mock *MockGenericRestoreExposer) RebindVolume(context1 context.Context, objectReference v1.ObjectReference, s string, s1 string, duration time.Duration) error {\n\tret := _mock.Called(context1, objectReference, s, s1, duration)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for RebindVolume\")\n\t}\n\n\tvar r0 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, v1.ObjectReference, string, string, time.Duration) error); ok {\n\t\tr0 = returnFunc(context1, objectReference, s, s1, duration)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\treturn r0\n}\n\n// MockGenericRestoreExposer_RebindVolume_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'RebindVolume'\ntype MockGenericRestoreExposer_RebindVolume_Call struct {\n\t*mock.Call\n}\n\n// RebindVolume is a helper method to define mock.On call\n// - context1 context.Context\n// - objectReference v1.ObjectReference\n// - s string\n// - s1 string\n// - duration time.Duration\nfunc (_e *MockGenericRestoreExposer_Expecter) RebindVolume(context1 interface{}, objectReference interface{}, s interface{}, s1 interface{}, duration interface{}) *MockGenericRestoreExposer_RebindVolume_Call {\n\treturn &MockGenericRestoreExposer_RebindVolume_Call{Call: _e.mock.On(\"RebindVolume\", context1, objectReference, s, s1, duration)}\n}\n\nfunc (_c *MockGenericRestoreExposer_RebindVolume_Call) Run(run func(context1 context.Context, objectReference v1.ObjectReference, s string, s1 string, duration time.Duration)) *MockGenericRestoreExposer_RebindVolume_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 v1.ObjectReference\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(v1.ObjectReference)\n\t\t}\n\t\tvar arg2 string\n\t\tif args[2] != nil {\n\t\t\targ2 = args[2].(string)\n\t\t}\n\t\tvar arg3 string\n\t\tif args[3] != nil {\n\t\t\targ3 = args[3].(string)\n\t\t}\n\t\tvar arg4 time.Duration\n\t\tif args[4] != nil {\n\t\t\targ4 = args[4].(time.Duration)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t\targ2,\n\t\t\targ3,\n\t\t\targ4,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockGenericRestoreExposer_RebindVolume_Call) Return(err error) *MockGenericRestoreExposer_RebindVolume_Call {\n\t_c.Call.Return(err)\n\treturn _c\n}\n\nfunc (_c *MockGenericRestoreExposer_RebindVolume_Call) RunAndReturn(run func(context1 context.Context, objectReference v1.ObjectReference, s string, s1 string, duration time.Duration) error) *MockGenericRestoreExposer_RebindVolume_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n" }, { "path": "pkg/exposer/mocks/PodVolumeExposer.go", "content": "// Code generated by mockery; DO NOT EDIT.\n// github.com/vektra/mockery\n// template: testify\n\npackage mocks\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\tmock \"github.com/stretchr/testify/mock\"\n\t\"github.com/vmware-tanzu/velero/pkg/exposer\"\n\t\"k8s.io/api/core/v1\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n)\n\n// NewMockPodVolumeExposer creates a new instance of MockPodVolumeExposer. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\n// The first argument is typically a *testing.T value.\nfunc NewMockPodVolumeExposer(t interface {\n\tmock.TestingT\n\tCleanup(func())\n}) *MockPodVolumeExposer {\n\tmock := &MockPodVolumeExposer{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n\n// MockPodVolumeExposer is an autogenerated mock type for the PodVolumeExposer type\ntype MockPodVolumeExposer struct {\n\tmock.Mock\n}\n\ntype MockPodVolumeExposer_Expecter struct {\n\tmock *mock.Mock\n}\n\nfunc (_m *MockPodVolumeExposer) EXPECT() *MockPodVolumeExposer_Expecter {\n\treturn &MockPodVolumeExposer_Expecter{mock: &_m.Mock}\n}\n\n// CleanUp provides a mock function for the type MockPodVolumeExposer\nfunc (_mock *MockPodVolumeExposer) CleanUp(context1 context.Context, objectReference v1.ObjectReference) {\n\t_mock.Called(context1, objectReference)\n\treturn\n}\n\n// MockPodVolumeExposer_CleanUp_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'CleanUp'\ntype MockPodVolumeExposer_CleanUp_Call struct {\n\t*mock.Call\n}\n\n// CleanUp is a helper method to define mock.On call\n// - context1 context.Context\n// - objectReference v1.ObjectReference\nfunc (_e *MockPodVolumeExposer_Expecter) CleanUp(context1 interface{}, objectReference interface{}) *MockPodVolumeExposer_CleanUp_Call {\n\treturn &MockPodVolumeExposer_CleanUp_Call{Call: _e.mock.On(\"CleanUp\", context1, objectReference)}\n}\n\nfunc (_c *MockPodVolumeExposer_CleanUp_Call) Run(run func(context1 context.Context, objectReference v1.ObjectReference)) *MockPodVolumeExposer_CleanUp_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 v1.ObjectReference\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(v1.ObjectReference)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockPodVolumeExposer_CleanUp_Call) Return() *MockPodVolumeExposer_CleanUp_Call {\n\t_c.Call.Return()\n\treturn _c\n}\n\nfunc (_c *MockPodVolumeExposer_CleanUp_Call) RunAndReturn(run func(context1 context.Context, objectReference v1.ObjectReference)) *MockPodVolumeExposer_CleanUp_Call {\n\t_c.Run(run)\n\treturn _c\n}\n\n// DiagnoseExpose provides a mock function for the type MockPodVolumeExposer\nfunc (_mock *MockPodVolumeExposer) DiagnoseExpose(context1 context.Context, objectReference v1.ObjectReference) string {\n\tret := _mock.Called(context1, objectReference)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for DiagnoseExpose\")\n\t}\n\n\tvar r0 string\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, v1.ObjectReference) string); ok {\n\t\tr0 = returnFunc(context1, objectReference)\n\t} else {\n\t\tr0 = ret.Get(0).(string)\n\t}\n\treturn r0\n}\n\n// MockPodVolumeExposer_DiagnoseExpose_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'DiagnoseExpose'\ntype MockPodVolumeExposer_DiagnoseExpose_Call struct {\n\t*mock.Call\n}\n\n// DiagnoseExpose is a helper method to define mock.On call\n// - context1 context.Context\n// - objectReference v1.ObjectReference\nfunc (_e *MockPodVolumeExposer_Expecter) DiagnoseExpose(context1 interface{}, objectReference interface{}) *MockPodVolumeExposer_DiagnoseExpose_Call {\n\treturn &MockPodVolumeExposer_DiagnoseExpose_Call{Call: _e.mock.On(\"DiagnoseExpose\", context1, objectReference)}\n}\n\nfunc (_c *MockPodVolumeExposer_DiagnoseExpose_Call) Run(run func(context1 context.Context, objectReference v1.ObjectReference)) *MockPodVolumeExposer_DiagnoseExpose_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 v1.ObjectReference\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(v1.ObjectReference)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockPodVolumeExposer_DiagnoseExpose_Call) Return(s string) *MockPodVolumeExposer_DiagnoseExpose_Call {\n\t_c.Call.Return(s)\n\treturn _c\n}\n\nfunc (_c *MockPodVolumeExposer_DiagnoseExpose_Call) RunAndReturn(run func(context1 context.Context, objectReference v1.ObjectReference) string) *MockPodVolumeExposer_DiagnoseExpose_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// Expose provides a mock function for the type MockPodVolumeExposer\nfunc (_mock *MockPodVolumeExposer) Expose(context1 context.Context, objectReference v1.ObjectReference, podVolumeExposeParam exposer.PodVolumeExposeParam) error {\n\tret := _mock.Called(context1, objectReference, podVolumeExposeParam)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Expose\")\n\t}\n\n\tvar r0 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, v1.ObjectReference, exposer.PodVolumeExposeParam) error); ok {\n\t\tr0 = returnFunc(context1, objectReference, podVolumeExposeParam)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\treturn r0\n}\n\n// MockPodVolumeExposer_Expose_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Expose'\ntype MockPodVolumeExposer_Expose_Call struct {\n\t*mock.Call\n}\n\n// Expose is a helper method to define mock.On call\n// - context1 context.Context\n// - objectReference v1.ObjectReference\n// - podVolumeExposeParam exposer.PodVolumeExposeParam\nfunc (_e *MockPodVolumeExposer_Expecter) Expose(context1 interface{}, objectReference interface{}, podVolumeExposeParam interface{}) *MockPodVolumeExposer_Expose_Call {\n\treturn &MockPodVolumeExposer_Expose_Call{Call: _e.mock.On(\"Expose\", context1, objectReference, podVolumeExposeParam)}\n}\n\nfunc (_c *MockPodVolumeExposer_Expose_Call) Run(run func(context1 context.Context, objectReference v1.ObjectReference, podVolumeExposeParam exposer.PodVolumeExposeParam)) *MockPodVolumeExposer_Expose_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 v1.ObjectReference\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(v1.ObjectReference)\n\t\t}\n\t\tvar arg2 exposer.PodVolumeExposeParam\n\t\tif args[2] != nil {\n\t\t\targ2 = args[2].(exposer.PodVolumeExposeParam)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t\targ2,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockPodVolumeExposer_Expose_Call) Return(err error) *MockPodVolumeExposer_Expose_Call {\n\t_c.Call.Return(err)\n\treturn _c\n}\n\nfunc (_c *MockPodVolumeExposer_Expose_Call) RunAndReturn(run func(context1 context.Context, objectReference v1.ObjectReference, podVolumeExposeParam exposer.PodVolumeExposeParam) error) *MockPodVolumeExposer_Expose_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// GetExposed provides a mock function for the type MockPodVolumeExposer\nfunc (_mock *MockPodVolumeExposer) GetExposed(context1 context.Context, objectReference v1.ObjectReference, client1 client.Client, s string, duration time.Duration) (*exposer.ExposeResult, error) {\n\tret := _mock.Called(context1, objectReference, client1, s, duration)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetExposed\")\n\t}\n\n\tvar r0 *exposer.ExposeResult\n\tvar r1 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, v1.ObjectReference, client.Client, string, time.Duration) (*exposer.ExposeResult, error)); ok {\n\t\treturn returnFunc(context1, objectReference, client1, s, duration)\n\t}\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, v1.ObjectReference, client.Client, string, time.Duration) *exposer.ExposeResult); ok {\n\t\tr0 = returnFunc(context1, objectReference, client1, s, duration)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(*exposer.ExposeResult)\n\t\t}\n\t}\n\tif returnFunc, ok := ret.Get(1).(func(context.Context, v1.ObjectReference, client.Client, string, time.Duration) error); ok {\n\t\tr1 = returnFunc(context1, objectReference, client1, s, duration)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\treturn r0, r1\n}\n\n// MockPodVolumeExposer_GetExposed_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'GetExposed'\ntype MockPodVolumeExposer_GetExposed_Call struct {\n\t*mock.Call\n}\n\n// GetExposed is a helper method to define mock.On call\n// - context1 context.Context\n// - objectReference v1.ObjectReference\n// - client1 client.Client\n// - s string\n// - duration time.Duration\nfunc (_e *MockPodVolumeExposer_Expecter) GetExposed(context1 interface{}, objectReference interface{}, client1 interface{}, s interface{}, duration interface{}) *MockPodVolumeExposer_GetExposed_Call {\n\treturn &MockPodVolumeExposer_GetExposed_Call{Call: _e.mock.On(\"GetExposed\", context1, objectReference, client1, s, duration)}\n}\n\nfunc (_c *MockPodVolumeExposer_GetExposed_Call) Run(run func(context1 context.Context, objectReference v1.ObjectReference, client1 client.Client, s string, duration time.Duration)) *MockPodVolumeExposer_GetExposed_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 v1.ObjectReference\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(v1.ObjectReference)\n\t\t}\n\t\tvar arg2 client.Client\n\t\tif args[2] != nil {\n\t\t\targ2 = args[2].(client.Client)\n\t\t}\n\t\tvar arg3 string\n\t\tif args[3] != nil {\n\t\t\targ3 = args[3].(string)\n\t\t}\n\t\tvar arg4 time.Duration\n\t\tif args[4] != nil {\n\t\t\targ4 = args[4].(time.Duration)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t\targ2,\n\t\t\targ3,\n\t\t\targ4,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockPodVolumeExposer_GetExposed_Call) Return(exposeResult *exposer.ExposeResult, err error) *MockPodVolumeExposer_GetExposed_Call {\n\t_c.Call.Return(exposeResult, err)\n\treturn _c\n}\n\nfunc (_c *MockPodVolumeExposer_GetExposed_Call) RunAndReturn(run func(context1 context.Context, objectReference v1.ObjectReference, client1 client.Client, s string, duration time.Duration) (*exposer.ExposeResult, error)) *MockPodVolumeExposer_GetExposed_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// PeekExposed provides a mock function for the type MockPodVolumeExposer\nfunc (_mock *MockPodVolumeExposer) PeekExposed(context1 context.Context, objectReference v1.ObjectReference) error {\n\tret := _mock.Called(context1, objectReference)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PeekExposed\")\n\t}\n\n\tvar r0 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, v1.ObjectReference) error); ok {\n\t\tr0 = returnFunc(context1, objectReference)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\treturn r0\n}\n\n// MockPodVolumeExposer_PeekExposed_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'PeekExposed'\ntype MockPodVolumeExposer_PeekExposed_Call struct {\n\t*mock.Call\n}\n\n// PeekExposed is a helper method to define mock.On call\n// - context1 context.Context\n// - objectReference v1.ObjectReference\nfunc (_e *MockPodVolumeExposer_Expecter) PeekExposed(context1 interface{}, objectReference interface{}) *MockPodVolumeExposer_PeekExposed_Call {\n\treturn &MockPodVolumeExposer_PeekExposed_Call{Call: _e.mock.On(\"PeekExposed\", context1, objectReference)}\n}\n\nfunc (_c *MockPodVolumeExposer_PeekExposed_Call) Run(run func(context1 context.Context, objectReference v1.ObjectReference)) *MockPodVolumeExposer_PeekExposed_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 v1.ObjectReference\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(v1.ObjectReference)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockPodVolumeExposer_PeekExposed_Call) Return(err error) *MockPodVolumeExposer_PeekExposed_Call {\n\t_c.Call.Return(err)\n\treturn _c\n}\n\nfunc (_c *MockPodVolumeExposer_PeekExposed_Call) RunAndReturn(run func(context1 context.Context, objectReference v1.ObjectReference) error) *MockPodVolumeExposer_PeekExposed_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n" }, { "path": "pkg/exposer/pod_volume.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage exposer\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/nodeagent\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nconst (\n\tPodVolumeExposeTypeBackup = \"pod-volume-backup\"\n\tPodVolumeExposeTypeRestore = \"pod-volume-restore\"\n)\n\n// PodVolumeExposeParam define the input param for pod volume Expose\ntype PodVolumeExposeParam struct {\n\t// ClientPodName is the name of pod to be backed up or restored\n\tClientPodName string\n\n\t// ClientNamespace is the namespace to be backed up or restored\n\tClientNamespace string\n\n\t// ClientNamespace is the pod volume for the client PVC\n\tClientPodVolume string\n\n\t// HostingPodLabels is the labels that are going to apply to the hosting pod\n\tHostingPodLabels map[string]string\n\n\t// HostingPodAnnotations is the annotations that are going to apply to the hosting pod\n\tHostingPodAnnotations map[string]string\n\n\t// HostingPodTolerations is the tolerations that are going to apply to the hosting pod\n\tHostingPodTolerations []corev1api.Toleration\n\n\t// Resources defines the resource requirements of the hosting pod\n\tResources corev1api.ResourceRequirements\n\n\t// OperationTimeout specifies the time wait for resources operations in Expose\n\tOperationTimeout time.Duration\n\n\t// Type specifies the type of the expose, either backup or erstore\n\tType string\n\n\t// PriorityClassName is the priority class name for the data mover pod\n\tPriorityClassName string\n\n\t// Privileged indicates whether to create the pod with a privileged container\n\tPrivileged bool\n\n\t// RestoreSize specifies the data size for the volume to be restored, for restore only\n\tRestoreSize int64\n\n\t// CacheVolume specifies the info for cache volumes, for restore only\n\tCacheVolume *CacheConfigs\n}\n\n// PodVolumeExposer is the interfaces for a pod volume exposer\ntype PodVolumeExposer interface {\n\t// Expose starts the process to a pod volume expose, the expose process may take long time\n\tExpose(context.Context, corev1api.ObjectReference, PodVolumeExposeParam) error\n\n\t// GetExposed polls the status of the expose.\n\t// If the expose is accessible by the current caller, it waits the expose ready and returns the expose result.\n\t// Otherwise, it returns nil as the expose result without an error.\n\tGetExposed(context.Context, corev1api.ObjectReference, client.Client, string, time.Duration) (*ExposeResult, error)\n\n\t// PeekExposed tests the status of the expose.\n\t// If the expose is incomplete but not recoverable, it returns an error.\n\t// Otherwise, it returns nil immediately.\n\tPeekExposed(context.Context, corev1api.ObjectReference) error\n\n\t// DiagnoseExpose generate the diagnostic info when the expose is not finished for a long time.\n\t// If it finds any problem, it returns an string about the problem.\n\tDiagnoseExpose(context.Context, corev1api.ObjectReference) string\n\n\t// CleanUp cleans up any objects generated during the restore expose\n\tCleanUp(context.Context, corev1api.ObjectReference)\n}\n\n// NewPodVolumeExposer creates a new instance of pod volume exposer\nfunc NewPodVolumeExposer(kubeClient kubernetes.Interface, log logrus.FieldLogger) PodVolumeExposer {\n\treturn &podVolumeExposer{\n\t\tkubeClient: kubeClient,\n\t\tfs: filesystem.NewFileSystem(),\n\t\tlog: log,\n\t}\n}\n\ntype podVolumeExposer struct {\n\tkubeClient kubernetes.Interface\n\tfs filesystem.Interface\n\tlog logrus.FieldLogger\n}\n\nvar getPodVolumeHostPath = GetPodVolumeHostPath\nvar extractPodVolumeHostPath = ExtractPodVolumeHostPath\n\nfunc (e *podVolumeExposer) Expose(ctx context.Context, ownerObject corev1api.ObjectReference, param PodVolumeExposeParam) error {\n\tcurLog := e.log.WithFields(logrus.Fields{\n\t\t\"owner\": ownerObject.Name,\n\t\t\"client pod\": param.ClientPodName,\n\t\t\"client pod volume\": param.ClientPodVolume,\n\t\t\"client namespace\": param.ClientNamespace,\n\t\t\"type\": param.Type,\n\t})\n\n\tpod, err := e.kubeClient.CoreV1().Pods(param.ClientNamespace).Get(ctx, param.ClientPodName, metav1.GetOptions{})\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error getting client pod %s\", param.ClientPodName)\n\t}\n\n\tif pod.Spec.NodeName == \"\" {\n\t\treturn errors.Errorf(\"client pod %s doesn't have a node name\", pod.Name)\n\t}\n\n\tnodeOS, err := kube.GetNodeOS(ctx, pod.Spec.NodeName, e.kubeClient.CoreV1())\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error getting OS for node %s\", pod.Spec.NodeName)\n\t}\n\n\tcurLog.Infof(\"Client pod is running in node %s, os %s\", pod.Spec.NodeName, nodeOS)\n\n\tpath, err := getPodVolumeHostPath(ctx, pod, param.ClientPodVolume, e.kubeClient, e.fs, e.log)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to get pod volume path\")\n\t}\n\n\tpath.ByPath, err = extractPodVolumeHostPath(ctx, path.ByPath, e.kubeClient, ownerObject.Namespace, nodeOS)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to extract pod volume path\")\n\t}\n\n\tcurLog.WithField(\"path\", path).Infof(\"Host path is retrieved for pod %s, volume %s\", param.ClientPodName, param.ClientPodVolume)\n\n\tvar cachePVC *corev1api.PersistentVolumeClaim\n\tif param.CacheVolume != nil {\n\t\tcacheVolumeSize := getCacheVolumeSize(param.RestoreSize, param.CacheVolume)\n\t\tif cacheVolumeSize > 0 {\n\t\t\tcurLog.Infof(\"Creating cache PVC with size %v\", cacheVolumeSize)\n\n\t\t\tif pvc, err := createCachePVC(ctx, e.kubeClient.CoreV1(), ownerObject, param.CacheVolume.StorageClass, cacheVolumeSize, pod.Spec.NodeName); err != nil {\n\t\t\t\treturn errors.Wrap(err, \"error to create cache pvc\")\n\t\t\t} else {\n\t\t\t\tcachePVC = pvc\n\t\t\t}\n\n\t\t\tdefer func() {\n\t\t\t\tif err != nil {\n\t\t\t\t\tkube.DeletePVAndPVCIfAny(ctx, e.kubeClient.CoreV1(), cachePVC.Name, cachePVC.Namespace, 0, curLog)\n\t\t\t\t}\n\t\t\t}()\n\t\t} else {\n\t\t\tcurLog.Infof(\"Don't need to create cache volume, restore size %v, cache info %v\", param.RestoreSize, param.CacheVolume)\n\t\t}\n\t}\n\n\thostingPod, err := e.createHostingPod(\n\t\tctx,\n\t\townerObject,\n\t\tparam.Type,\n\t\tpath.ByPath,\n\t\tparam.OperationTimeout,\n\t\tparam.HostingPodLabels,\n\t\tparam.HostingPodAnnotations,\n\t\tparam.HostingPodTolerations,\n\t\tpod.Spec.NodeName,\n\t\tparam.Resources,\n\t\tnodeOS,\n\t\tparam.PriorityClassName,\n\t\tparam.Privileged,\n\t\tcachePVC,\n\t)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to create hosting pod\")\n\t}\n\n\tcurLog.WithField(\"pod name\", hostingPod.Name).Info(\"Hosting pod is created\")\n\n\treturn nil\n}\n\nfunc (e *podVolumeExposer) GetExposed(ctx context.Context, ownerObject corev1api.ObjectReference, nodeClient client.Client, nodeName string, timeout time.Duration) (*ExposeResult, error) {\n\thostingPodName := ownerObject.Name\n\n\tcontainerName := string(ownerObject.UID)\n\tvolumeName := string(ownerObject.UID)\n\n\tcurLog := e.log.WithFields(logrus.Fields{\n\t\t\"owner\": ownerObject.Name,\n\t\t\"node\": nodeName,\n\t})\n\n\tvar updated *corev1api.Pod\n\terr := wait.PollUntilContextTimeout(ctx, 2*time.Second, timeout, true, func(ctx context.Context) (bool, error) {\n\t\tpod := &corev1api.Pod{}\n\t\terr := nodeClient.Get(ctx, types.NamespacedName{\n\t\t\tNamespace: ownerObject.Namespace,\n\t\t\tName: hostingPodName,\n\t\t}, pod)\n\n\t\tif err != nil {\n\t\t\treturn false, errors.Wrapf(err, \"error to get pod %s/%s\", ownerObject.Namespace, hostingPodName)\n\t\t}\n\n\t\tif pod.Status.Phase != corev1api.PodRunning {\n\t\t\treturn false, nil\n\t\t}\n\n\t\tupdated = pod\n\n\t\treturn true, nil\n\t})\n\n\tif err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tcurLog.WithField(\"hosting pod\", hostingPodName).Debug(\"Hosting pod is not running in the current node\")\n\t\t\treturn nil, nil\n\t\t} else {\n\t\t\treturn nil, errors.Wrapf(err, \"error to wait for rediness of pod %s\", hostingPodName)\n\t\t}\n\t}\n\n\tcurLog.WithField(\"pod\", updated.Name).Infof(\"Hosting pod is in running state in node %s\", updated.Spec.NodeName)\n\n\treturn &ExposeResult{ByPod: ExposeByPod{\n\t\tHostingPod: updated,\n\t\tHostingContainer: containerName,\n\t\tVolumeName: volumeName,\n\t}}, nil\n}\n\nfunc (e *podVolumeExposer) PeekExposed(ctx context.Context, ownerObject corev1api.ObjectReference) error {\n\thostingPodName := ownerObject.Name\n\n\tcurLog := e.log.WithFields(logrus.Fields{\n\t\t\"owner\": ownerObject.Name,\n\t})\n\n\tpod, err := e.kubeClient.CoreV1().Pods(ownerObject.Namespace).Get(ctx, hostingPodName, metav1.GetOptions{})\n\tif apierrors.IsNotFound(err) {\n\t\treturn nil\n\t}\n\n\tif err != nil {\n\t\tcurLog.WithError(err).Warnf(\"error to peek hosting pod %s\", hostingPodName)\n\t\treturn nil\n\t}\n\n\tif podFailed, message := kube.IsPodUnrecoverable(pod, curLog); podFailed {\n\t\treturn errors.New(message)\n\t}\n\n\treturn nil\n}\n\nfunc (e *podVolumeExposer) DiagnoseExpose(ctx context.Context, ownerObject corev1api.ObjectReference) string {\n\thostingPodName := ownerObject.Name\n\n\tdiag := \"begin diagnose pod volume exposer\\n\"\n\n\tpod, err := e.kubeClient.CoreV1().Pods(ownerObject.Namespace).Get(ctx, hostingPodName, metav1.GetOptions{})\n\tif err != nil {\n\t\tpod = nil\n\t\tdiag += fmt.Sprintf(\"error getting hosting pod %s, err: %v\\n\", hostingPodName, err)\n\t}\n\n\tcachePVC, err := e.kubeClient.CoreV1().PersistentVolumeClaims(ownerObject.Namespace).Get(ctx, getCachePVCName(ownerObject), metav1.GetOptions{})\n\tif err != nil {\n\t\tcachePVC = nil\n\n\t\tif !apierrors.IsNotFound(err) {\n\t\t\tdiag += fmt.Sprintf(\"error getting cache pvc %s, err: %v\\n\", getCachePVCName(ownerObject), err)\n\t\t}\n\t}\n\n\tevents, err := e.kubeClient.CoreV1().Events(ownerObject.Namespace).List(ctx, metav1.ListOptions{})\n\tif err != nil {\n\t\tdiag += fmt.Sprintf(\"error listing events, err: %v\\n\", err)\n\t}\n\n\tif pod != nil {\n\t\tdiag += kube.DiagnosePod(pod, events)\n\n\t\tif pod.Spec.NodeName != \"\" {\n\t\t\tif err := nodeagent.KbClientIsRunningInNode(ctx, ownerObject.Namespace, pod.Spec.NodeName, e.kubeClient); err != nil {\n\t\t\t\tdiag += fmt.Sprintf(\"node-agent is not running in node %s, err: %v\\n\", pod.Spec.NodeName, err)\n\t\t\t}\n\t\t}\n\t}\n\n\tif cachePVC != nil {\n\t\tdiag += kube.DiagnosePVC(cachePVC, events)\n\n\t\tif cachePVC.Spec.VolumeName != \"\" {\n\t\t\tif pv, err := e.kubeClient.CoreV1().PersistentVolumes().Get(ctx, cachePVC.Spec.VolumeName, metav1.GetOptions{}); err != nil {\n\t\t\t\tdiag += fmt.Sprintf(\"error getting cache pv %s, err: %v\\n\", cachePVC.Spec.VolumeName, err)\n\t\t\t} else {\n\t\t\t\tdiag += kube.DiagnosePV(pv)\n\t\t\t}\n\t\t}\n\t}\n\n\tdiag += \"end diagnose pod volume exposer\"\n\n\treturn diag\n}\n\nfunc (e *podVolumeExposer) CleanUp(ctx context.Context, ownerObject corev1api.ObjectReference) {\n\trestorePodName := ownerObject.Name\n\tcachePVCName := getCachePVCName(ownerObject)\n\n\tkube.DeletePodIfAny(ctx, e.kubeClient.CoreV1(), restorePodName, ownerObject.Namespace, e.log)\n\tkube.DeletePVAndPVCIfAny(ctx, e.kubeClient.CoreV1(), cachePVCName, ownerObject.Namespace, 0, e.log)\n}\n\nfunc (e *podVolumeExposer) createHostingPod(\n\tctx context.Context,\n\townerObject corev1api.ObjectReference,\n\texposeType string,\n\thostPath string,\n\toperationTimeout time.Duration,\n\tlabel map[string]string,\n\tannotation map[string]string,\n\ttoleration []corev1api.Toleration,\n\tselectedNode string,\n\tresources corev1api.ResourceRequirements,\n\tnodeOS string,\n\tpriorityClassName string,\n\tprivileged bool,\n\tcachePVC *corev1api.PersistentVolumeClaim,\n) (*corev1api.Pod, error) {\n\thostingPodName := ownerObject.Name\n\n\tcontainerName := string(ownerObject.UID)\n\tclientVolumeName := string(ownerObject.UID)\n\tclientVolumePath := \"/\" + clientVolumeName\n\n\tpodInfo, err := getInheritedPodInfo(ctx, e.kubeClient, ownerObject.Namespace, nodeOS)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error to get inherited pod info from node-agent\")\n\t}\n\n\t// Log the priority class if it's set\n\tif priorityClassName != \"\" {\n\t\te.log.Debugf(\"Setting priority class %q for data mover pod %s\", priorityClassName, hostingPodName)\n\t}\n\n\tvar gracePeriod int64\n\tmountPropagation := corev1api.MountPropagationHostToContainer\n\tvolumeMounts := []corev1api.VolumeMount{{\n\t\tName: clientVolumeName,\n\t\tMountPath: clientVolumePath,\n\t\tMountPropagation: &mountPropagation,\n\t}}\n\n\tvolumes := []corev1api.Volume{{\n\t\tName: clientVolumeName,\n\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\tHostPath: &corev1api.HostPathVolumeSource{\n\t\t\t\tPath: hostPath,\n\t\t\t},\n\t\t},\n\t}}\n\n\tcacheVolumePath := \"\"\n\tif cachePVC != nil {\n\t\tmnt, _, path := kube.MakePodPVCAttachment(cacheVolumeName, nil, false)\n\t\tvolumeMounts = append(volumeMounts, mnt...)\n\n\t\tvolumes = append(volumes, corev1api.Volume{\n\t\t\tName: cacheVolumeName,\n\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\tClaimName: cachePVC.Name,\n\t\t\t\t},\n\t\t\t},\n\t\t})\n\n\t\tcacheVolumePath = path\n\t}\n\n\tvolumeMounts = append(volumeMounts, podInfo.volumeMounts...)\n\tvolumes = append(volumes, podInfo.volumes...)\n\n\targs := []string{\n\t\tfmt.Sprintf(\"--volume-path=%s\", clientVolumePath),\n\t\tfmt.Sprintf(\"--resource-timeout=%s\", operationTimeout.String()),\n\t}\n\n\tcommand := []string{\n\t\t\"/velero\",\n\t\t\"pod-volume\",\n\t}\n\n\tif exposeType == PodVolumeExposeTypeBackup {\n\t\targs = append(args, fmt.Sprintf(\"--pod-volume-backup=%s\", ownerObject.Name))\n\t\tcommand = append(command, \"backup\")\n\t} else {\n\t\targs = append(args, fmt.Sprintf(\"--pod-volume-restore=%s\", ownerObject.Name))\n\t\targs = append(args, fmt.Sprintf(\"--cache-volume-path=%s\", cacheVolumePath))\n\t\tcommand = append(command, \"restore\")\n\t}\n\n\targs = append(args, podInfo.logFormatArgs...)\n\targs = append(args, podInfo.logLevelArgs...)\n\n\taffinity := &kube.LoadAffinity{}\n\n\tvar securityCtx *corev1api.PodSecurityContext\n\tvar containerSecurityCtx *corev1api.SecurityContext\n\tnodeSelector := map[string]string{}\n\tpodOS := corev1api.PodOS{}\n\tif nodeOS == kube.NodeOSWindows {\n\t\tuserID := \"ContainerAdministrator\"\n\t\tsecurityCtx = &corev1api.PodSecurityContext{\n\t\t\tWindowsOptions: &corev1api.WindowsSecurityContextOptions{\n\t\t\t\tRunAsUserName: &userID,\n\t\t\t},\n\t\t}\n\n\t\tpodOS.Name = kube.NodeOSWindows\n\n\t\taffinity.NodeSelector.MatchExpressions = append(affinity.NodeSelector.MatchExpressions, metav1.LabelSelectorRequirement{\n\t\t\tKey: kube.NodeOSLabel,\n\t\t\tValues: []string{kube.NodeOSWindows},\n\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t})\n\n\t\ttoleration = append(toleration, []corev1api.Toleration{\n\t\t\t{\n\t\t\t\tKey: \"os\",\n\t\t\t\tOperator: \"Equal\",\n\t\t\t\tEffect: \"NoSchedule\",\n\t\t\t\tValue: \"windows\",\n\t\t\t},\n\t\t\t{\n\t\t\t\tKey: \"os\",\n\t\t\t\tOperator: \"Equal\",\n\t\t\t\tEffect: \"NoExecute\",\n\t\t\t\tValue: \"windows\",\n\t\t\t},\n\t\t}...)\n\t} else {\n\t\tuserID := int64(0)\n\t\tsecurityCtx = &corev1api.PodSecurityContext{\n\t\t\tRunAsUser: &userID,\n\t\t}\n\t\tcontainerSecurityCtx = &corev1api.SecurityContext{\n\t\t\tPrivileged: &privileged,\n\t\t}\n\n\t\tpodOS.Name = kube.NodeOSLinux\n\n\t\taffinity.NodeSelector.MatchExpressions = append(affinity.NodeSelector.MatchExpressions, metav1.LabelSelectorRequirement{\n\t\t\tKey: kube.NodeOSLabel,\n\t\t\tValues: []string{kube.NodeOSWindows},\n\t\t\tOperator: metav1.LabelSelectorOpNotIn,\n\t\t})\n\t}\n\n\tpodAffinity := kube.ToSystemAffinity(affinity, nil)\n\n\tpod := &corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: hostingPodName,\n\t\t\tNamespace: ownerObject.Namespace,\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: ownerObject.APIVersion,\n\t\t\t\t\tKind: ownerObject.Kind,\n\t\t\t\t\tName: ownerObject.Name,\n\t\t\t\t\tUID: ownerObject.UID,\n\t\t\t\t\tController: boolptr.True(),\n\t\t\t\t},\n\t\t\t},\n\t\t\tLabels: label,\n\t\t\tAnnotations: annotation,\n\t\t},\n\t\tSpec: corev1api.PodSpec{\n\t\t\tNodeSelector: nodeSelector,\n\t\t\tOS: &podOS,\n\t\t\tAffinity: podAffinity,\n\t\t\tContainers: []corev1api.Container{\n\t\t\t\t{\n\t\t\t\t\tName: containerName,\n\t\t\t\t\tImage: podInfo.image,\n\t\t\t\t\tImagePullPolicy: corev1api.PullNever,\n\t\t\t\t\tCommand: command,\n\t\t\t\t\tArgs: args,\n\t\t\t\t\tVolumeMounts: volumeMounts,\n\t\t\t\t\tEnv: podInfo.env,\n\t\t\t\t\tEnvFrom: podInfo.envFrom,\n\t\t\t\t\tResources: resources,\n\t\t\t\t\tSecurityContext: containerSecurityCtx,\n\t\t\t\t},\n\t\t\t},\n\t\t\tPriorityClassName: priorityClassName,\n\t\t\tServiceAccountName: podInfo.serviceAccount,\n\t\t\tTerminationGracePeriodSeconds: &gracePeriod,\n\t\t\tVolumes: volumes,\n\t\t\tNodeName: selectedNode,\n\t\t\tRestartPolicy: corev1api.RestartPolicyNever,\n\t\t\tSecurityContext: securityCtx,\n\t\t\tTolerations: toleration,\n\t\t\tDNSPolicy: podInfo.dnsPolicy,\n\t\t\tDNSConfig: podInfo.dnsConfig,\n\t\t\tImagePullSecrets: podInfo.imagePullSecrets,\n\t\t},\n\t}\n\n\treturn e.kubeClient.CoreV1().Pods(ownerObject.Namespace).Create(ctx, pod, metav1.CreateOptions{})\n}\n" }, { "path": "pkg/exposer/pod_volume_test.go", "content": "package exposer\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"k8s.io/client-go/kubernetes/fake\"\n\tclientTesting \"k8s.io/client-go/testing\"\n\tclientFake \"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerov1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n)\n\nfunc TestPodVolumeExpose(t *testing.T) {\n\tbackup := &velerov1.Backup{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: velerov1.SchemeGroupVersion.String(),\n\t\t\tKind: \"Backup\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup\",\n\t\t\tUID: \"fake-uid\",\n\t\t},\n\t}\n\n\tpodWithNoNode := builder.ForPod(\"fake-ns\", \"fake-client-pod\").Result()\n\tpodWithNode := builder.ForPod(\"fake-ns\", \"fake-client-pod\").NodeName(\"fake-node\").Result()\n\n\tnode := builder.ForNode(\"fake-node\").Result()\n\n\tdaemonSet := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"velero\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t\tAPIVersion: appsv1api.SchemeGroupVersion.String(),\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"node-agent\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tsnapshotClientObj []runtime.Object\n\t\tkubeClientObj []runtime.Object\n\t\townerBackup *velerov1.Backup\n\t\texposeParam PodVolumeExposeParam\n\t\tfuncGetPodVolumeHostPath func(context.Context, *corev1api.Pod, string, kubernetes.Interface, filesystem.Interface, logrus.FieldLogger) (datapath.AccessPoint, error)\n\t\tfuncExtractPodVolumeHostPath func(context.Context, string, kubernetes.Interface, string, string) (string, error)\n\t\tkubeReactors []reactor\n\t\texpectBackupPod bool\n\t\texpectCachePVC bool\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"get client pod fail\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: PodVolumeExposeParam{\n\t\t\t\tClientNamespace: \"fake-ns\",\n\t\t\t\tClientPodName: \"fake-client-pod\",\n\t\t\t},\n\t\t\terr: \"error getting client pod fake-client-pod: pods \\\"fake-client-pod\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"client pod with no node name\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: PodVolumeExposeParam{\n\t\t\t\tClientNamespace: \"fake-ns\",\n\t\t\t\tClientPodName: \"fake-client-pod\",\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpodWithNoNode,\n\t\t\t},\n\t\t\terr: \"client pod fake-client-pod doesn't have a node name\",\n\t\t},\n\t\t{\n\t\t\tname: \"get node os fail\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: PodVolumeExposeParam{\n\t\t\t\tClientNamespace: \"fake-ns\",\n\t\t\t\tClientPodName: \"fake-client-pod\",\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpodWithNode,\n\t\t\t},\n\t\t\terr: \"error getting OS for node fake-node: error getting node fake-node: nodes \\\"fake-node\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"get pod volume path fail\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: PodVolumeExposeParam{\n\t\t\t\tClientNamespace: \"fake-ns\",\n\t\t\t\tClientPodName: \"fake-client-pod\",\n\t\t\t\tClientPodVolume: \"fake-client-volume\",\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpodWithNode,\n\t\t\t\tnode,\n\t\t\t},\n\t\t\tfuncGetPodVolumeHostPath: func(context.Context, *corev1api.Pod, string, kubernetes.Interface, filesystem.Interface, logrus.FieldLogger) (datapath.AccessPoint, error) {\n\t\t\t\treturn datapath.AccessPoint{}, errors.New(\"fake-get-pod-volume-path-error\")\n\t\t\t},\n\t\t\terr: \"error to get pod volume path: fake-get-pod-volume-path-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"extract pod volume path fail\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: PodVolumeExposeParam{\n\t\t\t\tClientNamespace: \"fake-ns\",\n\t\t\t\tClientPodName: \"fake-client-pod\",\n\t\t\t\tClientPodVolume: \"fake-client-volume\",\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpodWithNode,\n\t\t\t\tnode,\n\t\t\t},\n\t\t\tfuncGetPodVolumeHostPath: func(context.Context, *corev1api.Pod, string, kubernetes.Interface, filesystem.Interface, logrus.FieldLogger) (datapath.AccessPoint, error) {\n\t\t\t\treturn datapath.AccessPoint{\n\t\t\t\t\tByPath: \"/var/lib/kubelet/pods/pod-id-xxx/volumes/kubernetes.io~csi/pvc-id-xxx/mount\",\n\t\t\t\t}, nil\n\t\t\t},\n\t\t\tfuncExtractPodVolumeHostPath: func(context.Context, string, kubernetes.Interface, string, string) (string, error) {\n\t\t\t\treturn \"\", errors.New(\"fake-extract-error\")\n\t\t\t},\n\t\t\terr: \"error to extract pod volume path: fake-extract-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"create hosting pod fail\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: PodVolumeExposeParam{\n\t\t\t\tClientNamespace: \"fake-ns\",\n\t\t\t\tClientPodName: \"fake-client-pod\",\n\t\t\t\tClientPodVolume: \"fake-client-volume\",\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpodWithNode,\n\t\t\t\tnode,\n\t\t\t},\n\t\t\tfuncGetPodVolumeHostPath: func(context.Context, *corev1api.Pod, string, kubernetes.Interface, filesystem.Interface, logrus.FieldLogger) (datapath.AccessPoint, error) {\n\t\t\t\treturn datapath.AccessPoint{\n\t\t\t\t\tByPath: \"/host_pods/pod-id-xxx/volumes/kubernetes.io~csi/pvc-id-xxx/mount\",\n\t\t\t\t}, nil\n\t\t\t},\n\t\t\tfuncExtractPodVolumeHostPath: func(context.Context, string, kubernetes.Interface, string, string) (string, error) {\n\t\t\t\treturn \"/var/lib/kubelet/pods/pod-id-xxx/volumes/kubernetes.io~csi/pvc-id-xxx/mount\", nil\n\t\t\t},\n\t\t\terr: \"error to create hosting pod: error to get inherited pod info from node-agent: error to get node-agent pod template: error to get node-agent daemonset: daemonsets.apps \\\"node-agent\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: PodVolumeExposeParam{\n\t\t\t\tClientNamespace: \"fake-ns\",\n\t\t\t\tClientPodName: \"fake-client-pod\",\n\t\t\t\tClientPodVolume: \"fake-client-volume\",\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpodWithNode,\n\t\t\t\tnode,\n\t\t\t\tdaemonSet,\n\t\t\t},\n\t\t\tfuncGetPodVolumeHostPath: func(context.Context, *corev1api.Pod, string, kubernetes.Interface, filesystem.Interface, logrus.FieldLogger) (datapath.AccessPoint, error) {\n\t\t\t\treturn datapath.AccessPoint{\n\t\t\t\t\tByPath: \"/host_pods/pod-id-xxx/volumes/kubernetes.io~csi/pvc-id-xxx/mount\",\n\t\t\t\t}, nil\n\t\t\t},\n\t\t\tfuncExtractPodVolumeHostPath: func(context.Context, string, kubernetes.Interface, string, string) (string, error) {\n\t\t\t\treturn \"/var/lib/kubelet/pods/pod-id-xxx/volumes/kubernetes.io~csi/pvc-id-xxx/mount\", nil\n\t\t\t},\n\t\t\texpectBackupPod: true,\n\t\t},\n\t\t{\n\t\t\tname: \"succeed with privileged pod\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: PodVolumeExposeParam{\n\t\t\t\tClientNamespace: \"fake-ns\",\n\t\t\t\tClientPodName: \"fake-client-pod\",\n\t\t\t\tClientPodVolume: \"fake-client-volume\",\n\t\t\t\tPrivileged: true,\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpodWithNode,\n\t\t\t\tnode,\n\t\t\t\tdaemonSet,\n\t\t\t},\n\t\t\tfuncGetPodVolumeHostPath: func(context.Context, *corev1api.Pod, string, kubernetes.Interface, filesystem.Interface, logrus.FieldLogger) (datapath.AccessPoint, error) {\n\t\t\t\treturn datapath.AccessPoint{\n\t\t\t\t\tByPath: \"/host_pods/pod-id-xxx/volumes/kubernetes.io~csi/pvc-id-xxx/mount\",\n\t\t\t\t}, nil\n\t\t\t},\n\t\t\tfuncExtractPodVolumeHostPath: func(context.Context, string, kubernetes.Interface, string, string) (string, error) {\n\t\t\t\treturn \"/var/lib/kubelet/pods/pod-id-xxx/volumes/kubernetes.io~csi/pvc-id-xxx/mount\", nil\n\t\t\t},\n\t\t\texpectBackupPod: true,\n\t\t},\n\t\t{\n\t\t\tname: \"succeed, cache config, no cache volume\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: PodVolumeExposeParam{\n\t\t\t\tClientNamespace: \"fake-ns\",\n\t\t\t\tClientPodName: \"fake-client-pod\",\n\t\t\t\tClientPodVolume: \"fake-client-volume\",\n\t\t\t\tCacheVolume: &CacheConfigs{},\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpodWithNode,\n\t\t\t\tnode,\n\t\t\t\tdaemonSet,\n\t\t\t},\n\t\t\tfuncGetPodVolumeHostPath: func(context.Context, *corev1api.Pod, string, kubernetes.Interface, filesystem.Interface, logrus.FieldLogger) (datapath.AccessPoint, error) {\n\t\t\t\treturn datapath.AccessPoint{\n\t\t\t\t\tByPath: \"/host_pods/pod-id-xxx/volumes/kubernetes.io~csi/pvc-id-xxx/mount\",\n\t\t\t\t}, nil\n\t\t\t},\n\t\t\tfuncExtractPodVolumeHostPath: func(context.Context, string, kubernetes.Interface, string, string) (string, error) {\n\t\t\t\treturn \"/var/lib/kubelet/pods/pod-id-xxx/volumes/kubernetes.io~csi/pvc-id-xxx/mount\", nil\n\t\t\t},\n\t\t\texpectBackupPod: true,\n\t\t},\n\t\t{\n\t\t\tname: \"create cache volume fail\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: PodVolumeExposeParam{\n\t\t\t\tClientNamespace: \"fake-ns\",\n\t\t\t\tClientPodName: \"fake-client-pod\",\n\t\t\t\tClientPodVolume: \"fake-client-volume\",\n\t\t\t\tCacheVolume: &CacheConfigs{Limit: 1024},\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpodWithNode,\n\t\t\t\tnode,\n\t\t\t\tdaemonSet,\n\t\t\t},\n\t\t\tfuncGetPodVolumeHostPath: func(context.Context, *corev1api.Pod, string, kubernetes.Interface, filesystem.Interface, logrus.FieldLogger) (datapath.AccessPoint, error) {\n\t\t\t\treturn datapath.AccessPoint{\n\t\t\t\t\tByPath: \"/host_pods/pod-id-xxx/volumes/kubernetes.io~csi/pvc-id-xxx/mount\",\n\t\t\t\t}, nil\n\t\t\t},\n\t\t\tfuncExtractPodVolumeHostPath: func(context.Context, string, kubernetes.Interface, string, string) (string, error) {\n\t\t\t\treturn \"/var/lib/kubelet/pods/pod-id-xxx/volumes/kubernetes.io~csi/pvc-id-xxx/mount\", nil\n\t\t\t},\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"create\",\n\t\t\t\t\tresource: \"persistentvolumeclaims\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-create-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to create cache pvc: fake-create-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed with cache volume\",\n\t\t\townerBackup: backup,\n\t\t\texposeParam: PodVolumeExposeParam{\n\t\t\t\tClientNamespace: \"fake-ns\",\n\t\t\t\tClientPodName: \"fake-client-pod\",\n\t\t\t\tClientPodVolume: \"fake-client-volume\",\n\t\t\t\tCacheVolume: &CacheConfigs{Limit: 1024},\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpodWithNode,\n\t\t\t\tnode,\n\t\t\t\tdaemonSet,\n\t\t\t},\n\t\t\tfuncGetPodVolumeHostPath: func(context.Context, *corev1api.Pod, string, kubernetes.Interface, filesystem.Interface, logrus.FieldLogger) (datapath.AccessPoint, error) {\n\t\t\t\treturn datapath.AccessPoint{\n\t\t\t\t\tByPath: \"/host_pods/pod-id-xxx/volumes/kubernetes.io~csi/pvc-id-xxx/mount\",\n\t\t\t\t}, nil\n\t\t\t},\n\t\t\tfuncExtractPodVolumeHostPath: func(context.Context, string, kubernetes.Interface, string, string) (string, error) {\n\t\t\t\treturn \"/var/lib/kubelet/pods/pod-id-xxx/volumes/kubernetes.io~csi/pvc-id-xxx/mount\", nil\n\t\t\t},\n\t\t\texpectBackupPod: true,\n\t\t\texpectCachePVC: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tfor _, reactor := range test.kubeReactors {\n\t\t\t\tfakeKubeClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\texposer := podVolumeExposer{\n\t\t\t\tkubeClient: fakeKubeClient,\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tvar ownerObject corev1api.ObjectReference\n\t\t\tif test.ownerBackup != nil {\n\t\t\t\townerObject = corev1api.ObjectReference{\n\t\t\t\t\tKind: test.ownerBackup.Kind,\n\t\t\t\t\tNamespace: test.ownerBackup.Namespace,\n\t\t\t\t\tName: test.ownerBackup.Name,\n\t\t\t\t\tUID: test.ownerBackup.UID,\n\t\t\t\t\tAPIVersion: test.ownerBackup.APIVersion,\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif test.funcGetPodVolumeHostPath != nil {\n\t\t\t\tgetPodVolumeHostPath = test.funcGetPodVolumeHostPath\n\t\t\t}\n\n\t\t\tif test.funcExtractPodVolumeHostPath != nil {\n\t\t\t\textractPodVolumeHostPath = test.funcExtractPodVolumeHostPath\n\t\t\t}\n\n\t\t\terr := exposer.Expose(t.Context(), ownerObject, test.exposeParam)\n\t\t\tif err == nil {\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\t_, err = exposer.kubeClient.CoreV1().Pods(ownerObject.Namespace).Get(t.Context(), ownerObject.Name, metav1.GetOptions{})\n\t\t\t\trequire.NoError(t, err)\n\t\t\t} else {\n\t\t\t\trequire.EqualError(t, err, test.err)\n\t\t\t}\n\n\t\t\t_, err = exposer.kubeClient.CoreV1().Pods(ownerObject.Namespace).Get(t.Context(), ownerObject.Name, metav1.GetOptions{})\n\t\t\tif test.expectBackupPod {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t} else {\n\t\t\t\trequire.True(t, apierrors.IsNotFound(err))\n\t\t\t}\n\n\t\t\t_, err = exposer.kubeClient.CoreV1().PersistentVolumeClaims(ownerObject.Namespace).Get(t.Context(), getCachePVCName(ownerObject), metav1.GetOptions{})\n\t\t\tif test.expectCachePVC {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t} else {\n\t\t\t\trequire.True(t, apierrors.IsNotFound(err))\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetPodVolumeExpose(t *testing.T) {\n\tbackup := &velerov1.Backup{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: velerov1.SchemeGroupVersion.String(),\n\t\t\tKind: \"Backup\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup\",\n\t\t\tUID: \"fake-uid\",\n\t\t},\n\t}\n\n\tbackupPodNotRunning := builder.ForPod(backup.Namespace, backup.Name).Result()\n\tbackupPodRunning := builder.ForPod(backup.Namespace, backup.Name).Phase(corev1api.PodRunning).Result()\n\n\tscheme := runtime.NewScheme()\n\tcorev1api.AddToScheme(scheme)\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\townerBackup *velerov1.Backup\n\t\tnodeName string\n\t\tTimeout time.Duration\n\t\terr string\n\t\texpectedResult *ExposeResult\n\t}{\n\t\t{\n\t\t\tname: \"backup pod is not found\",\n\t\t\townerBackup: backup,\n\t\t\tnodeName: \"fake-node\",\n\t\t},\n\t\t{\n\t\t\tname: \"wait backup pod running fail\",\n\t\t\townerBackup: backup,\n\t\t\tnodeName: \"fake-node\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tbackupPodNotRunning,\n\t\t\t},\n\t\t\tTimeout: time.Second,\n\t\t\terr: \"error to wait for rediness of pod fake-backup: context deadline exceeded\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\townerBackup: backup,\n\t\t\tnodeName: \"fake-node\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tbackupPodRunning,\n\t\t\t},\n\t\t\tTimeout: time.Second,\n\t\t\texpectedResult: &ExposeResult{\n\t\t\t\tByPod: ExposeByPod{\n\t\t\t\t\tHostingPod: backupPodRunning,\n\t\t\t\t\tVolumeName: string(backup.UID),\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tfakeClientBuilder := clientFake.NewClientBuilder()\n\t\t\tfakeClientBuilder = fakeClientBuilder.WithScheme(scheme)\n\n\t\t\tfakeClient := fakeClientBuilder.WithRuntimeObjects(test.kubeClientObj...).Build()\n\n\t\t\texposer := podVolumeExposer{\n\t\t\t\tkubeClient: fakeKubeClient,\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tvar ownerObject corev1api.ObjectReference\n\t\t\tif test.ownerBackup != nil {\n\t\t\t\townerObject = corev1api.ObjectReference{\n\t\t\t\t\tKind: test.ownerBackup.Kind,\n\t\t\t\t\tNamespace: test.ownerBackup.Namespace,\n\t\t\t\t\tName: test.ownerBackup.Name,\n\t\t\t\t\tUID: test.ownerBackup.UID,\n\t\t\t\t\tAPIVersion: test.ownerBackup.APIVersion,\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tresult, err := exposer.GetExposed(t.Context(), ownerObject, fakeClient, test.nodeName, test.Timeout)\n\t\t\tif test.err == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\tif test.expectedResult == nil {\n\t\t\t\t\tassert.Nil(t, result)\n\t\t\t\t} else {\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tassert.Equal(t, test.expectedResult.ByPod.VolumeName, result.ByPod.VolumeName)\n\t\t\t\t\tassert.Equal(t, test.expectedResult.ByPod.HostingPod.Name, result.ByPod.HostingPod.Name)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, test.err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestPodVolumePeekExpose(t *testing.T) {\n\tbackup := &velerov1.Backup{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: velerov1.SchemeGroupVersion.String(),\n\t\t\tKind: \"Backup\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup\",\n\t\t\tUID: \"fake-uid\",\n\t\t},\n\t}\n\n\tbackupPodUrecoverable := &corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: backup.Namespace,\n\t\t\tName: backup.Name,\n\t\t},\n\t\tStatus: corev1api.PodStatus{\n\t\t\tPhase: corev1api.PodFailed,\n\t\t},\n\t}\n\n\tbackupPod := &corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: backup.Namespace,\n\t\t\tName: backup.Name,\n\t\t},\n\t}\n\n\tscheme := runtime.NewScheme()\n\tcorev1api.AddToScheme(scheme)\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\townerBackup *velerov1.Backup\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"backup pod is not found\",\n\t\t\townerBackup: backup,\n\t\t},\n\t\t{\n\t\t\tname: \"pod is unrecoverable\",\n\t\t\townerBackup: backup,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tbackupPodUrecoverable,\n\t\t\t},\n\t\t\terr: \"Pod is in abnormal state [Failed], message []\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\townerBackup: backup,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tbackupPod,\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\texposer := podVolumeExposer{\n\t\t\t\tkubeClient: fakeKubeClient,\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tvar ownerObject corev1api.ObjectReference\n\t\t\tif test.ownerBackup != nil {\n\t\t\t\townerObject = corev1api.ObjectReference{\n\t\t\t\t\tKind: test.ownerBackup.Kind,\n\t\t\t\t\tNamespace: test.ownerBackup.Namespace,\n\t\t\t\t\tName: test.ownerBackup.Name,\n\t\t\t\t\tUID: test.ownerBackup.UID,\n\t\t\t\t\tAPIVersion: test.ownerBackup.APIVersion,\n\t\t\t\t}\n\t\t\t}\n\n\t\t\terr := exposer.PeekExposed(t.Context(), ownerObject)\n\t\t\tif test.err == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, test.err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestPodVolumeDiagnoseExpose(t *testing.T) {\n\tbackup := &velerov1.Backup{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: velerov1.SchemeGroupVersion.String(),\n\t\t\tKind: \"Backup\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup\",\n\t\t\tUID: \"fake-uid\",\n\t\t},\n\t}\n\n\tbackupPodWithoutNodeName := corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup\",\n\t\t\tUID: \"fake-pod-uid\",\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: backup.APIVersion,\n\t\t\t\t\tKind: backup.Kind,\n\t\t\t\t\tName: backup.Name,\n\t\t\t\t\tUID: backup.UID,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tStatus: corev1api.PodStatus{\n\t\t\tPhase: corev1api.PodPending,\n\t\t\tConditions: []corev1api.PodCondition{\n\t\t\t\t{\n\t\t\t\t\tType: corev1api.PodInitialized,\n\t\t\t\t\tStatus: corev1api.ConditionTrue,\n\t\t\t\t\tMessage: \"fake-pod-message\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tMessage: \"fake-pod-message-1\",\n\t\t},\n\t}\n\n\tbackupPodWithNodeName := corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup\",\n\t\t\tUID: \"fake-pod-uid\",\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: backup.APIVersion,\n\t\t\t\t\tKind: backup.Kind,\n\t\t\t\t\tName: backup.Name,\n\t\t\t\t\tUID: backup.UID,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tSpec: corev1api.PodSpec{\n\t\t\tNodeName: \"fake-node\",\n\t\t},\n\t\tStatus: corev1api.PodStatus{\n\t\t\tPhase: corev1api.PodPending,\n\t\t\tConditions: []corev1api.PodCondition{\n\t\t\t\t{\n\t\t\t\t\tType: corev1api.PodInitialized,\n\t\t\t\t\tStatus: corev1api.ConditionTrue,\n\t\t\t\t\tMessage: \"fake-pod-message\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tcachePVCWithVolumeName := corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"fake-backup-cache\",\n\t\t\tUID: \"fake-cache-pvc-uid\",\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: backup.APIVersion,\n\t\t\t\t\tKind: backup.Kind,\n\t\t\t\t\tName: backup.Name,\n\t\t\t\t\tUID: backup.UID,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tVolumeName: \"fake-pv-cache\",\n\t\t},\n\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\tPhase: corev1api.ClaimPending,\n\t\t},\n\t}\n\n\tcachePV := corev1api.PersistentVolume{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-pv-cache\",\n\t\t},\n\t\tStatus: corev1api.PersistentVolumeStatus{\n\t\t\tPhase: corev1api.VolumePending,\n\t\t\tMessage: \"fake-pv-message\",\n\t\t},\n\t}\n\n\tnodeAgentPod := corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"node-agent-pod-1\",\n\t\t\tLabels: map[string]string{\"role\": \"node-agent\"},\n\t\t},\n\t\tSpec: corev1api.PodSpec{\n\t\t\tNodeName: \"fake-node\",\n\t\t},\n\t\tStatus: corev1api.PodStatus{\n\t\t\tPhase: corev1api.PodRunning,\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\townerBackup *velerov1.Backup\n\t\tkubeClientObj []runtime.Object\n\t\tsnapshotClientObj []runtime.Object\n\t\texpected string\n\t}{\n\t\t{\n\t\t\tname: \"no pod\",\n\t\t\townerBackup: backup,\n\t\t\texpected: `begin diagnose pod volume exposer\nerror getting hosting pod fake-backup, err: pods \"fake-backup\" not found\nend diagnose pod volume exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"pod without node name, pvc without volume name, vs without status\",\n\t\t\townerBackup: backup,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&backupPodWithoutNodeName,\n\t\t\t},\n\t\t\texpected: `begin diagnose pod volume exposer\nPod velero/fake-backup, phase Pending, node name , message fake-pod-message-1\nPod condition Initialized, status True, reason , message fake-pod-message\nend diagnose pod volume exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"pod without node name\",\n\t\t\townerBackup: backup,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&backupPodWithoutNodeName,\n\t\t\t},\n\t\t\texpected: `begin diagnose pod volume exposer\nPod velero/fake-backup, phase Pending, node name , message fake-pod-message-1\nPod condition Initialized, status True, reason , message fake-pod-message\nend diagnose pod volume exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"pod with node name, no node agent\",\n\t\t\townerBackup: backup,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&backupPodWithNodeName,\n\t\t\t},\n\t\t\texpected: `begin diagnose pod volume exposer\nPod velero/fake-backup, phase Pending, node name fake-node, message \nPod condition Initialized, status True, reason , message fake-pod-message\nnode-agent is not running in node fake-node, err: daemonset pod not found in running state in node fake-node\nend diagnose pod volume exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"pod with node name, node agent is running\",\n\t\t\townerBackup: backup,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&backupPodWithNodeName,\n\t\t\t\t&nodeAgentPod,\n\t\t\t},\n\t\t\texpected: `begin diagnose pod volume exposer\nPod velero/fake-backup, phase Pending, node name fake-node, message \nPod condition Initialized, status True, reason , message fake-pod-message\nend diagnose pod volume exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"cache pvc with volume name, no pv\",\n\t\t\townerBackup: backup,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&backupPodWithNodeName,\n\t\t\t\t&cachePVCWithVolumeName,\n\t\t\t\t&nodeAgentPod,\n\t\t\t},\n\t\t\texpected: `begin diagnose pod volume exposer\nPod velero/fake-backup, phase Pending, node name fake-node, message \nPod condition Initialized, status True, reason , message fake-pod-message\nPVC velero/fake-backup-cache, phase Pending, binding to fake-pv-cache\nerror getting cache pv fake-pv-cache, err: persistentvolumes \"fake-pv-cache\" not found\nend diagnose pod volume exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"cache pvc with volume name, pv exists\",\n\t\t\townerBackup: backup,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&backupPodWithNodeName,\n\t\t\t\t&cachePVCWithVolumeName,\n\t\t\t\t&cachePV,\n\t\t\t\t&nodeAgentPod,\n\t\t\t},\n\t\t\texpected: `begin diagnose pod volume exposer\nPod velero/fake-backup, phase Pending, node name fake-node, message \nPod condition Initialized, status True, reason , message fake-pod-message\nPVC velero/fake-backup-cache, phase Pending, binding to fake-pv-cache\nPV fake-pv-cache, phase Pending, reason , message fake-pv-message\nend diagnose pod volume exposer`,\n\t\t},\n\t\t{\n\t\t\tname: \"with events\",\n\t\t\townerBackup: backup,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&backupPodWithNodeName,\n\t\t\t\t&nodeAgentPod,\n\t\t\t\t&corev1api.Event{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Namespace: velerov1.DefaultNamespace, Name: \"event-1\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-uid-1\"},\n\t\t\t\t\tReason: \"reason-1\",\n\t\t\t\t\tMessage: \"message-1\",\n\t\t\t\t},\n\t\t\t\t&corev1api.Event{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Namespace: velerov1.DefaultNamespace, Name: \"event-2\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-pod-uid\"},\n\t\t\t\t\tReason: \"reason-2\",\n\t\t\t\t\tMessage: \"message-2\",\n\t\t\t\t},\n\t\t\t\t&corev1api.Event{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Namespace: \"other-namespace\", Name: \"event-3\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-pod-uid\"},\n\t\t\t\t\tReason: \"reason-3\",\n\t\t\t\t\tMessage: \"message-3\",\n\t\t\t\t},\n\t\t\t\t&corev1api.Event{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Namespace: velerov1.DefaultNamespace, Name: \"event-4\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-pod-uid\"},\n\t\t\t\t\tReason: \"reason-4\",\n\t\t\t\t\tMessage: \"message-4\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: `begin diagnose pod volume exposer\nPod velero/fake-backup, phase Pending, node name fake-node, message \nPod condition Initialized, status True, reason , message fake-pod-message\nPod event reason reason-2, message message-2\nPod event reason reason-4, message message-4\nend diagnose pod volume exposer`,\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(tt.kubeClientObj...)\n\t\t\te := &podVolumeExposer{\n\t\t\t\tkubeClient: fakeKubeClient,\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\t\t\tvar ownerObject corev1api.ObjectReference\n\t\t\tif tt.ownerBackup != nil {\n\t\t\t\townerObject = corev1api.ObjectReference{\n\t\t\t\t\tKind: tt.ownerBackup.Kind,\n\t\t\t\t\tNamespace: tt.ownerBackup.Namespace,\n\t\t\t\t\tName: tt.ownerBackup.Name,\n\t\t\t\t\tUID: tt.ownerBackup.UID,\n\t\t\t\t\tAPIVersion: tt.ownerBackup.APIVersion,\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tdiag := e.DiagnoseExpose(t.Context(), ownerObject)\n\t\t\tassert.Equal(t, tt.expected, diag)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/exposer/snapshot.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage exposer\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\tcorev1api \"k8s.io/api/core/v1\"\n)\n\n// SnapshotExposer is the interfaces for a snapshot exposer\ntype SnapshotExposer interface {\n\t// Expose starts the process to expose a snapshot, the expose process may take long time\n\tExpose(context.Context, corev1api.ObjectReference, any) error\n\t// GetExposed polls the status of the expose.\n\t// If the expose is accessible by the current caller, it waits the expose ready and returns the expose result.\n\t// Otherwise, it returns nil as the expose result without an error.\n\tGetExposed(context.Context, corev1api.ObjectReference, time.Duration, any) (*ExposeResult, error)\n\n\t// PeekExposed tests the status of the expose.\n\t// If the expose is incomplete but not recoverable, it returns an error.\n\t// Otherwise, it returns nil immediately.\n\tPeekExposed(context.Context, corev1api.ObjectReference) error\n\n\t// DiagnoseExpose generate the diagnostic info when the expose is not finished for a long time.\n\t// If it finds any problem, it returns an string about the problem.\n\tDiagnoseExpose(context.Context, corev1api.ObjectReference) string\n\n\t// CleanUp cleans up any objects generated during the snapshot expose\n\tCleanUp(context.Context, corev1api.ObjectReference, string, string)\n}\n" }, { "path": "pkg/exposer/types.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage exposer\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n)\n\nconst (\n\tAccessModeFileSystem = \"by-file-system\"\n\tAccessModeBlock = \"by-block-device\"\n\tpodGroupLabel = \"velero.io/exposer-pod-group\"\n\tpodGroupSnapshot = \"snapshot-exposer\"\n\tpodGroupGenericRestore = \"generic-restore-exposer\"\n\tExposeOnGoingLabel = \"velero.io/expose-on-going\"\n)\n\n// ExposeResult defines the result of expose.\n// Varying from the type of the expose, the result may be different.\ntype ExposeResult struct {\n\tByPod ExposeByPod\n}\n\n// ExposeByPod defines the result for the expose method that a hosting pod is created\ntype ExposeByPod struct {\n\tHostingPod *corev1api.Pod\n\tHostingContainer string\n\tVolumeName string\n\tNodeOS *string\n}\n" }, { "path": "pkg/exposer/vgdp_counter.go", "content": "package exposer\n\nimport (\n\t\"context\"\n\t\"sync/atomic\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/client-go/tools/cache\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\n\t\"sigs.k8s.io/controller-runtime/pkg/manager\"\n\n\tctlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n)\n\ntype dynamicQueueLength struct {\n\tqueueLength int\n\tchangeID uint64\n}\n\ntype VgdpCounter struct {\n\tclient ctlclient.Client\n\tallowedQueueLength int\n\n\tduState dynamicQueueLength\n\tddState dynamicQueueLength\n\tpvbState dynamicQueueLength\n\tpvrState dynamicQueueLength\n\n\tduCacheState dynamicQueueLength\n\tddCacheState dynamicQueueLength\n\tpvbCacheState dynamicQueueLength\n\tpvrCacheState dynamicQueueLength\n}\n\nfunc StartVgdpCounter(ctx context.Context, mgr manager.Manager, queueLength int) (*VgdpCounter, error) {\n\tcounter := &VgdpCounter{\n\t\tclient: mgr.GetClient(),\n\t\tallowedQueueLength: queueLength,\n\t}\n\n\tatomic.StoreUint64(&counter.duState.changeID, 1)\n\tatomic.StoreUint64(&counter.ddState.changeID, 1)\n\tatomic.StoreUint64(&counter.pvbState.changeID, 1)\n\tatomic.StoreUint64(&counter.pvrState.changeID, 1)\n\n\tif err := counter.initListeners(ctx, mgr); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn counter, nil\n}\n\nfunc (w *VgdpCounter) initListeners(ctx context.Context, mgr manager.Manager) error {\n\tduInformer, err := mgr.GetCache().GetInformer(ctx, &velerov2alpha1api.DataUpload{})\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error getting du informer\")\n\t}\n\n\tif _, err := duInformer.AddEventHandler(\n\t\tcache.ResourceEventHandlerFuncs{\n\t\t\tUpdateFunc: func(oldObj, newObj any) {\n\t\t\t\toldDu := oldObj.(*velerov2alpha1api.DataUpload)\n\t\t\t\tnewDu := newObj.(*velerov2alpha1api.DataUpload)\n\n\t\t\t\tif oldDu.Status.Phase == newDu.Status.Phase {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif newDu.Status.Phase == velerov2alpha1api.DataUploadPhaseAccepted ||\n\t\t\t\t\toldDu.Status.Phase == velerov2alpha1api.DataUploadPhasePrepared ||\n\t\t\t\t\toldDu.Status.Phase == velerov2alpha1api.DataUploadPhaseAccepted && newDu.Status.Phase != velerov2alpha1api.DataUploadPhasePrepared {\n\t\t\t\t\tatomic.AddUint64(&w.duState.changeID, 1)\n\t\t\t\t}\n\t\t\t},\n\t\t},\n\t); err != nil {\n\t\treturn errors.Wrap(err, \"error registering du handler\")\n\t}\n\n\tddInformer, err := mgr.GetCache().GetInformer(ctx, &velerov2alpha1api.DataDownload{})\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error getting dd informer\")\n\t}\n\n\tif _, err := ddInformer.AddEventHandler(\n\t\tcache.ResourceEventHandlerFuncs{\n\t\t\tUpdateFunc: func(oldObj, newObj any) {\n\t\t\t\toldDd := oldObj.(*velerov2alpha1api.DataDownload)\n\t\t\t\tnewDd := newObj.(*velerov2alpha1api.DataDownload)\n\n\t\t\t\tif oldDd.Status.Phase == newDd.Status.Phase {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif newDd.Status.Phase == velerov2alpha1api.DataDownloadPhaseAccepted ||\n\t\t\t\t\toldDd.Status.Phase == velerov2alpha1api.DataDownloadPhasePrepared ||\n\t\t\t\t\toldDd.Status.Phase == velerov2alpha1api.DataDownloadPhaseAccepted && newDd.Status.Phase != velerov2alpha1api.DataDownloadPhasePrepared {\n\t\t\t\t\tatomic.AddUint64(&w.ddState.changeID, 1)\n\t\t\t\t}\n\t\t\t},\n\t\t},\n\t); err != nil {\n\t\treturn errors.Wrap(err, \"error registering dd handler\")\n\t}\n\n\tpvbInformer, err := mgr.GetCache().GetInformer(ctx, &velerov1api.PodVolumeBackup{})\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error getting PVB informer\")\n\t}\n\n\tif _, err := pvbInformer.AddEventHandler(\n\t\tcache.ResourceEventHandlerFuncs{\n\t\t\tUpdateFunc: func(oldObj, newObj any) {\n\t\t\t\toldPvb := oldObj.(*velerov1api.PodVolumeBackup)\n\t\t\t\tnewPvb := newObj.(*velerov1api.PodVolumeBackup)\n\n\t\t\t\tif oldPvb.Status.Phase == newPvb.Status.Phase {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif newPvb.Status.Phase == velerov1api.PodVolumeBackupPhaseAccepted ||\n\t\t\t\t\toldPvb.Status.Phase == velerov1api.PodVolumeBackupPhasePrepared ||\n\t\t\t\t\toldPvb.Status.Phase == velerov1api.PodVolumeBackupPhaseAccepted && newPvb.Status.Phase != velerov1api.PodVolumeBackupPhasePrepared {\n\t\t\t\t\tatomic.AddUint64(&w.pvbState.changeID, 1)\n\t\t\t\t}\n\t\t\t},\n\t\t},\n\t); err != nil {\n\t\treturn errors.Wrap(err, \"error registering PVB handler\")\n\t}\n\n\tpvrInformer, err := mgr.GetCache().GetInformer(ctx, &velerov1api.PodVolumeRestore{})\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error getting PVR informer\")\n\t}\n\n\tif _, err := pvrInformer.AddEventHandler(\n\t\tcache.ResourceEventHandlerFuncs{\n\t\t\tUpdateFunc: func(oldObj, newObj any) {\n\t\t\t\toldPvr := oldObj.(*velerov1api.PodVolumeRestore)\n\t\t\t\tnewPvr := newObj.(*velerov1api.PodVolumeRestore)\n\n\t\t\t\tif oldPvr.Status.Phase == newPvr.Status.Phase {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif newPvr.Status.Phase == velerov1api.PodVolumeRestorePhaseAccepted ||\n\t\t\t\t\toldPvr.Status.Phase == velerov1api.PodVolumeRestorePhasePrepared ||\n\t\t\t\t\toldPvr.Status.Phase == velerov1api.PodVolumeRestorePhaseAccepted && newPvr.Status.Phase != velerov1api.PodVolumeRestorePhasePrepared {\n\t\t\t\t\tatomic.AddUint64(&w.pvrState.changeID, 1)\n\t\t\t\t}\n\t\t\t},\n\t\t},\n\t); err != nil {\n\t\treturn errors.Wrap(err, \"error registering PVR handler\")\n\t}\n\n\treturn nil\n}\n\nfunc (w *VgdpCounter) IsConstrained(ctx context.Context, log logrus.FieldLogger) bool {\n\tid := atomic.LoadUint64(&w.duState.changeID)\n\tif id != w.duCacheState.changeID {\n\t\tduList := &velerov2alpha1api.DataUploadList{}\n\t\tif err := w.client.List(ctx, duList, &ctlclient.ListOptions{LabelSelector: labels.SelectorFromSet(labels.Set(map[string]string{ExposeOnGoingLabel: \"true\"}))}); err != nil {\n\t\t\tlog.WithError(err).Warn(\"Failed to list data uploads, skip counting\")\n\t\t} else {\n\t\t\tw.duCacheState.queueLength = len(duList.Items)\n\t\t\tw.duCacheState.changeID = id\n\n\t\t\tlog.Infof(\"Query queue length for du %d\", w.duCacheState.queueLength)\n\t\t}\n\t}\n\n\tid = atomic.LoadUint64(&w.ddState.changeID)\n\tif id != w.ddCacheState.changeID {\n\t\tddList := &velerov2alpha1api.DataDownloadList{}\n\t\tif err := w.client.List(ctx, ddList, &ctlclient.ListOptions{LabelSelector: labels.SelectorFromSet(labels.Set(map[string]string{ExposeOnGoingLabel: \"true\"}))}); err != nil {\n\t\t\tlog.WithError(err).Warn(\"Failed to list data downloads, skip counting\")\n\t\t} else {\n\t\t\tw.ddCacheState.queueLength = len(ddList.Items)\n\t\t\tw.ddCacheState.changeID = id\n\n\t\t\tlog.Infof(\"Query queue length for dd %d\", w.ddCacheState.queueLength)\n\t\t}\n\t}\n\n\tid = atomic.LoadUint64(&w.pvbState.changeID)\n\tif id != w.pvbCacheState.changeID {\n\t\tpvbList := &velerov1api.PodVolumeBackupList{}\n\t\tif err := w.client.List(ctx, pvbList, &ctlclient.ListOptions{LabelSelector: labels.SelectorFromSet(labels.Set(map[string]string{ExposeOnGoingLabel: \"true\"}))}); err != nil {\n\t\t\tlog.WithError(err).Warn(\"Failed to list PVB, skip counting\")\n\t\t} else {\n\t\t\tw.pvbCacheState.queueLength = len(pvbList.Items)\n\t\t\tw.pvbCacheState.changeID = id\n\n\t\t\tlog.Infof(\"Query queue length for pvb %d\", w.pvbCacheState.queueLength)\n\t\t}\n\t}\n\n\tid = atomic.LoadUint64(&w.pvrState.changeID)\n\tif id != w.pvrCacheState.changeID {\n\t\tpvrList := &velerov1api.PodVolumeRestoreList{}\n\t\tif err := w.client.List(ctx, pvrList, &ctlclient.ListOptions{LabelSelector: labels.SelectorFromSet(labels.Set(map[string]string{ExposeOnGoingLabel: \"true\"}))}); err != nil {\n\t\t\tlog.WithError(err).Warn(\"Failed to list PVR, skip counting\")\n\t\t} else {\n\t\t\tw.pvrCacheState.queueLength = len(pvrList.Items)\n\t\t\tw.pvrCacheState.changeID = id\n\n\t\t\tlog.Infof(\"Query queue length for pvr %d\", w.pvrCacheState.queueLength)\n\t\t}\n\t}\n\n\texisting := w.duCacheState.queueLength + w.ddCacheState.queueLength + w.pvbCacheState.queueLength + w.pvrCacheState.queueLength\n\tconstrained := existing >= w.allowedQueueLength\n\n\treturn constrained\n}\n" }, { "path": "pkg/exposer/vgdp_counter_test.go", "content": "package exposer\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n)\n\nfunc TestIsConstrained(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tcounter VgdpCounter\n\t\tkubeClientObj []client.Object\n\t\tgetErr bool\n\t\texpected bool\n\t}{\n\t\t{\n\t\t\tname: \"no change, constrained\",\n\t\t\tcounter: VgdpCounter{},\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"no change, not constrained\",\n\t\t\tcounter: VgdpCounter{allowedQueueLength: 1},\n\t\t},\n\t\t{\n\t\t\tname: \"change in du, get failed\",\n\t\t\tcounter: VgdpCounter{\n\t\t\t\tallowedQueueLength: 1,\n\t\t\t\tduState: dynamicQueueLength{0, 1},\n\t\t\t},\n\t\t\tgetErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"change in du, constrained\",\n\t\t\tcounter: VgdpCounter{\n\t\t\t\tallowedQueueLength: 1,\n\t\t\t\tduState: dynamicQueueLength{0, 1},\n\t\t\t},\n\t\t\tkubeClientObj: []client.Object{\n\t\t\t\tbuilder.ForDataUpload(\"velero\", \"test-1\").Labels(map[string]string{ExposeOnGoingLabel: \"true\"}).Result(),\n\t\t\t},\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"change in dd, get failed\",\n\t\t\tcounter: VgdpCounter{\n\t\t\t\tallowedQueueLength: 1,\n\t\t\t\tddState: dynamicQueueLength{0, 1},\n\t\t\t},\n\t\t\tgetErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"change in dd, constrained\",\n\t\t\tcounter: VgdpCounter{\n\t\t\t\tallowedQueueLength: 1,\n\t\t\t\tddState: dynamicQueueLength{0, 1},\n\t\t\t},\n\t\t\tkubeClientObj: []client.Object{\n\t\t\t\tbuilder.ForDataDownload(\"velero\", \"test-1\").Labels(map[string]string{ExposeOnGoingLabel: \"true\"}).Result(),\n\t\t\t},\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"change in pvb, get failed\",\n\t\t\tcounter: VgdpCounter{\n\t\t\t\tallowedQueueLength: 1,\n\t\t\t\tpvbState: dynamicQueueLength{0, 1},\n\t\t\t},\n\t\t\tgetErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"change in pvb, constrained\",\n\t\t\tcounter: VgdpCounter{\n\t\t\t\tallowedQueueLength: 1,\n\t\t\t\tpvbState: dynamicQueueLength{0, 1},\n\t\t\t},\n\t\t\tkubeClientObj: []client.Object{\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"test-1\").Labels(map[string]string{ExposeOnGoingLabel: \"true\"}).Result(),\n\t\t\t},\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"change in pvr, get failed\",\n\t\t\tcounter: VgdpCounter{\n\t\t\t\tallowedQueueLength: 1,\n\t\t\t\tpvrState: dynamicQueueLength{0, 1},\n\t\t\t},\n\t\t\tgetErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"change in pvr, constrained\",\n\t\t\tcounter: VgdpCounter{\n\t\t\t\tallowedQueueLength: 1,\n\t\t\t\tpvrState: dynamicQueueLength{0, 1},\n\t\t\t},\n\t\t\tkubeClientObj: []client.Object{\n\t\t\t\tbuilder.ForPodVolumeRestore(\"velero\", \"test-1\").Labels(map[string]string{ExposeOnGoingLabel: \"true\"}).Result(),\n\t\t\t},\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"change in du, pvb, not constrained\",\n\t\t\tcounter: VgdpCounter{\n\t\t\t\tallowedQueueLength: 3,\n\t\t\t\tduState: dynamicQueueLength{0, 1},\n\t\t\t\tpvbState: dynamicQueueLength{0, 1},\n\t\t\t},\n\t\t\tkubeClientObj: []client.Object{\n\t\t\t\tbuilder.ForDataUpload(\"velero\", \"test-1\").Labels(map[string]string{ExposeOnGoingLabel: \"true\"}).Result(),\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"test-1\").Labels(map[string]string{ExposeOnGoingLabel: \"true\"}).Result(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"change in dd, pvr, constrained\",\n\t\t\tcounter: VgdpCounter{\n\t\t\t\tallowedQueueLength: 1,\n\t\t\t\tddState: dynamicQueueLength{0, 1},\n\t\t\t\tpvrState: dynamicQueueLength{0, 1},\n\t\t\t},\n\t\t\tkubeClientObj: []client.Object{\n\t\t\t\tbuilder.ForDataDownload(\"velero\", \"test-1\").Labels(map[string]string{ExposeOnGoingLabel: \"true\"}).Result(),\n\t\t\t\tbuilder.ForPodVolumeRestore(\"velero\", \"test-1\").Labels(map[string]string{ExposeOnGoingLabel: \"true\"}).Result(),\n\t\t\t},\n\t\t\texpected: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tscheme := runtime.NewScheme()\n\n\t\t\tif !test.getErr {\n\t\t\t\terr := velerov1api.AddToScheme(scheme)\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\terr = velerov2alpha1api.AddToScheme(scheme)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\ttest.counter.client = fake.NewClientBuilder().WithScheme(scheme).WithObjects(test.kubeClientObj...).Build()\n\n\t\t\tresult := test.counter.IsConstrained(t.Context(), velerotest.NewLogger())\n\n\t\t\tassert.Equal(t, test.expected, result)\n\n\t\t\tif !test.getErr {\n\t\t\t\tassert.Equal(t, test.counter.duState.changeID, test.counter.duCacheState.changeID)\n\t\t\t\tassert.Equal(t, test.counter.ddState.changeID, test.counter.ddCacheState.changeID)\n\t\t\t\tassert.Equal(t, test.counter.pvbState.changeID, test.counter.pvbCacheState.changeID)\n\t\t\t\tassert.Equal(t, test.counter.pvrState.changeID, test.counter.pvrCacheState.changeID)\n\t\t\t} else {\n\t\t\t\tor := test.counter.duState.changeID != test.counter.duCacheState.changeID\n\t\t\t\tif !or {\n\t\t\t\t\tor = test.counter.ddState.changeID != test.counter.ddCacheState.changeID\n\t\t\t\t}\n\n\t\t\t\tif !or {\n\t\t\t\t\tor = test.counter.pvbState.changeID != test.counter.pvbCacheState.changeID\n\t\t\t\t}\n\n\t\t\t\tif !or {\n\t\t\t\t\tor = test.counter.pvrState.changeID != test.counter.pvrCacheState.changeID\n\t\t\t\t}\n\n\t\t\t\tassert.True(t, or)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/features/feature_flags.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage features\n\nimport (\n\t\"strings\"\n\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n)\n\ntype featureFlagSet struct {\n\tset sets.Set[string]\n}\n\n// featureFlags will store all the flags for this process until NewFeatureFlagSet is called.\nvar featureFlags featureFlagSet\n\n// IsEnabled returns True if a specified flag is enabled.\nfunc IsEnabled(name string) bool {\n\treturn featureFlags.set.Has(name)\n}\n\n// Enable adds a given slice of feature names to the current feature list.\nfunc Enable(names ...string) {\n\t// Initialize the flag set so that users don't have to\n\tif featureFlags.set == nil {\n\t\tNewFeatureFlagSet()\n\t}\n\n\tfeatureFlags.set.Insert(names...)\n}\n\n// Disable removes all feature flags in a given slice from the current feature list.\nfunc Disable(names ...string) {\n\tfeatureFlags.set.Delete(names...)\n}\n\n// All returns enabled features as a slice of strings.\nfunc All() []string {\n\treturn sets.List[string](featureFlags.set)\n}\n\n// Serialize returns all features as a comma-separated string.\nfunc Serialize() string {\n\treturn strings.Join(All(), \",\")\n}\n\n// NewFeatureFlagSet initializes and populates a new FeatureFlagSet.\n// This must be called to properly initialize the set for tracking flags.\n// It is also useful for selectively controlling flags during tests.\nfunc NewFeatureFlagSet(flags ...string) {\n\tfeatureFlags = featureFlagSet{\n\t\tset: sets.New[string](flags...),\n\t}\n}\n" }, { "path": "pkg/features/feature_flags_test.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage features\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestFeatureFlags(t *testing.T) {\n\t// Prepare a new flag set\n\tNewFeatureFlagSet(\"feature1\", \"feature2\")\n\n\tassert.True(t, IsEnabled(\"feature1\"))\n\tassert.False(t, IsEnabled(\"feature3\"))\n\tassert.Equal(t, []string{\"feature1\", \"feature2\"}, All())\n\n\tEnable(\"feature3\")\n\tassert.True(t, IsEnabled(\"feature3\"))\n\tassert.Equal(t, []string{\"feature1\", \"feature2\", \"feature3\"}, All())\n\n\tDisable(\"feature3\")\n\tassert.Equal(t, []string{\"feature1\", \"feature2\"}, All())\n\n\tassert.Equal(t, \"feature1,feature2\", Serialize())\n\n\t// Calling NewFeatureFlagSet re-initializes the set of flags\n\tNewFeatureFlagSet()\n\tassert.Empty(t, All())\n}\n" }, { "path": "pkg/install/daemonset.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage install\n\nimport (\n\t\"fmt\"\n\t\"path/filepath\"\n\t\"strings\"\n\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/utils/ptr\"\n\n\t\"github.com/vmware-tanzu/velero/internal/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/nodeagent\"\n)\n\nfunc DaemonSet(namespace string, opts ...podTemplateOption) *appsv1api.DaemonSet {\n\tc := &podTemplateConfig{\n\t\timage: velero.DefaultVeleroImage(),\n\t}\n\n\tfor _, opt := range opts {\n\t\topt(c)\n\t}\n\n\tpullPolicy := corev1api.PullAlways\n\timageParts := strings.Split(c.image, \":\")\n\tif len(imageParts) == 2 && imageParts[1] != \"latest\" {\n\t\tpullPolicy = corev1api.PullIfNotPresent\n\t}\n\n\tdaemonSetArgs := []string{\n\t\t\"node-agent\",\n\t\t\"server\",\n\t}\n\tif len(c.features) > 0 {\n\t\tdaemonSetArgs = append(daemonSetArgs, fmt.Sprintf(\"--features=%s\", strings.Join(c.features, \",\")))\n\t}\n\n\tif len(c.nodeAgentConfigMap) > 0 {\n\t\tdaemonSetArgs = append(daemonSetArgs, fmt.Sprintf(\"--node-agent-configmap=%s\", c.nodeAgentConfigMap))\n\t}\n\n\tif len(c.backupRepoConfigMap) > 0 {\n\t\tdaemonSetArgs = append(daemonSetArgs, fmt.Sprintf(\"--backup-repository-configmap=%s\", c.backupRepoConfigMap))\n\t}\n\n\tuserID := int64(0)\n\tmountPropagationMode := corev1api.MountPropagationHostToContainer\n\n\tdsName := \"node-agent\"\n\tif c.forWindows {\n\t\tdsName = \"node-agent-windows\"\n\t}\n\thostPodsVolumePath := filepath.Join(c.kubeletRootDir, \"pods\")\n\thostPluginsVolumePath := filepath.Join(c.kubeletRootDir, \"plugins\")\n\tvolumes := []corev1api.Volume{}\n\tvolumeMounts := []corev1api.VolumeMount{}\n\tif !c.nodeAgentDisableHostPath {\n\t\tvolumes = append(volumes, []corev1api.Volume{\n\t\t\t{\n\t\t\t\tName: \"host-pods\",\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tHostPath: &corev1api.HostPathVolumeSource{\n\t\t\t\t\t\tPath: hostPodsVolumePath,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tName: \"host-plugins\",\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tHostPath: &corev1api.HostPathVolumeSource{\n\t\t\t\t\t\tPath: hostPluginsVolumePath,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t}...)\n\n\t\tvolumeMounts = append(volumeMounts, []corev1api.VolumeMount{\n\t\t\t{\n\t\t\t\tName: nodeagent.HostPodVolumeMount,\n\t\t\t\tMountPath: nodeagent.HostPodVolumeMountPath(),\n\t\t\t\tMountPropagation: &mountPropagationMode,\n\t\t\t},\n\t\t\t{\n\t\t\t\tName: \"host-plugins\",\n\t\t\t\tMountPath: \"/var/lib/kubelet/plugins\",\n\t\t\t\tMountPropagation: &mountPropagationMode,\n\t\t\t},\n\t\t}...)\n\t}\n\n\tvolumes = append(volumes, corev1api.Volume{\n\t\tName: \"scratch\",\n\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\tEmptyDir: new(corev1api.EmptyDirVolumeSource),\n\t\t},\n\t})\n\n\tvolumeMounts = append(volumeMounts, corev1api.VolumeMount{\n\t\tName: \"scratch\",\n\t\tMountPath: \"/scratch\",\n\t})\n\n\tdaemonSet := &appsv1api.DaemonSet{\n\t\tObjectMeta: objectMeta(namespace, dsName),\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t\tAPIVersion: appsv1api.SchemeGroupVersion.String(),\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tSelector: &metav1.LabelSelector{\n\t\t\t\tMatchLabels: map[string]string{\n\t\t\t\t\t\"name\": dsName,\n\t\t\t\t},\n\t\t\t},\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tLabels: podLabels(c.labels, map[string]string{\n\t\t\t\t\t\t\"name\": dsName,\n\t\t\t\t\t\t\"role\": \"node-agent\",\n\t\t\t\t\t}),\n\t\t\t\t\tAnnotations: c.annotations,\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tServiceAccountName: c.serviceAccountName,\n\t\t\t\t\tSecurityContext: &corev1api.PodSecurityContext{\n\t\t\t\t\t\tRunAsUser: &userID,\n\t\t\t\t\t},\n\t\t\t\t\tVolumes: volumes,\n\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: dsName,\n\t\t\t\t\t\t\tImage: c.image,\n\t\t\t\t\t\t\tPorts: containerPorts(),\n\t\t\t\t\t\t\tImagePullPolicy: pullPolicy,\n\t\t\t\t\t\t\tCommand: []string{\n\t\t\t\t\t\t\t\t\"/velero\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tArgs: daemonSetArgs,\n\t\t\t\t\t\t\tSecurityContext: &corev1api.SecurityContext{\n\t\t\t\t\t\t\t\tPrivileged: &c.privilegedNodeAgent,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tVolumeMounts: volumeMounts,\n\t\t\t\t\t\t\tEnv: []corev1api.EnvVar{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"NODE_NAME\",\n\t\t\t\t\t\t\t\t\tValueFrom: &corev1api.EnvVarSource{\n\t\t\t\t\t\t\t\t\t\tFieldRef: &corev1api.ObjectFieldSelector{\n\t\t\t\t\t\t\t\t\t\t\tFieldPath: \"spec.nodeName\",\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"VELERO_NAMESPACE\",\n\t\t\t\t\t\t\t\t\tValueFrom: &corev1api.EnvVarSource{\n\t\t\t\t\t\t\t\t\t\tFieldRef: &corev1api.ObjectFieldSelector{\n\t\t\t\t\t\t\t\t\t\t\tFieldPath: \"metadata.namespace\",\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"VELERO_SCRATCH_DIR\",\n\t\t\t\t\t\t\t\t\tValue: \"/scratch\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tResources: c.resources,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tPriorityClassName: c.priorityClassName,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tif c.withSecret {\n\t\tdaemonSet.Spec.Template.Spec.Volumes = append(\n\t\t\tdaemonSet.Spec.Template.Spec.Volumes,\n\t\t\tcorev1api.Volume{\n\t\t\t\tName: \"cloud-credentials\",\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tSecret: &corev1api.SecretVolumeSource{\n\t\t\t\t\t\t// read-only for Owner, Group, Public\n\t\t\t\t\t\tDefaultMode: ptr.To(int32(0444)),\n\t\t\t\t\t\tSecretName: \"cloud-credentials\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t)\n\n\t\tdaemonSet.Spec.Template.Spec.Containers[0].VolumeMounts = append(\n\t\t\tdaemonSet.Spec.Template.Spec.Containers[0].VolumeMounts,\n\t\t\tcorev1api.VolumeMount{\n\t\t\t\tName: \"cloud-credentials\",\n\t\t\t\tMountPath: \"/credentials\",\n\t\t\t},\n\t\t)\n\n\t\tdaemonSet.Spec.Template.Spec.Containers[0].Env = append(daemonSet.Spec.Template.Spec.Containers[0].Env, []corev1api.EnvVar{\n\t\t\t{\n\t\t\t\tName: \"GOOGLE_APPLICATION_CREDENTIALS\",\n\t\t\t\tValue: \"/credentials/cloud\",\n\t\t\t},\n\t\t\t{\n\t\t\t\tName: \"AWS_SHARED_CREDENTIALS_FILE\",\n\t\t\t\tValue: \"/credentials/cloud\",\n\t\t\t},\n\t\t\t{\n\t\t\t\tName: \"AZURE_CREDENTIALS_FILE\",\n\t\t\t\tValue: \"/credentials/cloud\",\n\t\t\t},\n\t\t\t{\n\t\t\t\tName: \"ALIBABA_CLOUD_CREDENTIALS_FILE\",\n\t\t\t\tValue: \"/credentials/cloud\",\n\t\t\t},\n\t\t}...)\n\t}\n\n\tif c.forWindows {\n\t\tdaemonSet.Spec.Template.Spec.SecurityContext = nil\n\t\tdaemonSet.Spec.Template.Spec.Containers[0].SecurityContext = nil\n\t\tdaemonSet.Spec.Template.Spec.OS = &corev1api.PodOS{\n\t\t\tName: \"windows\",\n\t\t}\n\n\t\tdaemonSet.Spec.Template.Spec.Affinity = &corev1api.Affinity{\n\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\t\tValues: []string{\"windows\"},\n\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpIn,\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\n\t\tdaemonSet.Spec.Template.Spec.Tolerations = []corev1api.Toleration{\n\t\t\t{\n\t\t\t\tKey: \"os\",\n\t\t\t\tOperator: \"Equal\",\n\t\t\t\tEffect: \"NoSchedule\",\n\t\t\t\tValue: \"windows\",\n\t\t\t},\n\t\t\t{\n\t\t\t\tKey: \"os\",\n\t\t\t\tOperator: \"Equal\",\n\t\t\t\tEffect: \"NoExecute\",\n\t\t\t\tValue: \"windows\",\n\t\t\t},\n\t\t}\n\t} else {\n\t\tdaemonSet.Spec.Template.Spec.Affinity = &corev1api.Affinity{\n\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\t\tValues: []string{\"windows\"},\n\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t}\n\t}\n\n\tdaemonSet.Spec.Template.Spec.Containers[0].Env = append(daemonSet.Spec.Template.Spec.Containers[0].Env, c.envVars...)\n\n\treturn daemonSet\n}\n" }, { "path": "pkg/install/daemonset_test.go", "content": "/*\nCopyright 2019, 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage install\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\tcorev1api \"k8s.io/api/core/v1\"\n)\n\nfunc TestDaemonSet(t *testing.T) {\n\tuserID := int64(0)\n\tboolFalse := false\n\tboolTrue := true\n\n\tds := DaemonSet(\"velero\")\n\n\tassert.Equal(t, \"node-agent\", ds.Spec.Template.Spec.Containers[0].Name)\n\tassert.Equal(t, \"velero\", ds.ObjectMeta.Namespace)\n\tassert.Equal(t, \"node-agent\", ds.Spec.Template.ObjectMeta.Labels[\"name\"])\n\tassert.Equal(t, \"node-agent\", ds.Spec.Template.ObjectMeta.Labels[\"role\"])\n\tassert.Equal(t, &corev1api.Affinity{\n\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t{\n\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\tValues: []string{\"windows\"},\n\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}, ds.Spec.Template.Spec.Affinity)\n\tassert.Equal(t, corev1api.PodSecurityContext{RunAsUser: &userID}, *ds.Spec.Template.Spec.SecurityContext)\n\tassert.Equal(t, corev1api.SecurityContext{Privileged: &boolFalse}, *ds.Spec.Template.Spec.Containers[0].SecurityContext)\n\tassert.Len(t, ds.Spec.Template.Spec.Volumes, 3)\n\tassert.Len(t, ds.Spec.Template.Spec.Containers[0].VolumeMounts, 3)\n\n\tds = DaemonSet(\"velero\", WithPrivilegedNodeAgent(true))\n\tassert.Equal(t, corev1api.SecurityContext{Privileged: &boolTrue}, *ds.Spec.Template.Spec.Containers[0].SecurityContext)\n\n\tds = DaemonSet(\"velero\", WithImage(\"velero/velero:v0.11\"))\n\tassert.Equal(t, \"velero/velero:v0.11\", ds.Spec.Template.Spec.Containers[0].Image)\n\tassert.Equal(t, corev1api.PullIfNotPresent, ds.Spec.Template.Spec.Containers[0].ImagePullPolicy)\n\n\tds = DaemonSet(\"velero\", WithSecret(true))\n\tassert.Len(t, ds.Spec.Template.Spec.Containers[0].Env, 7)\n\tassert.Len(t, ds.Spec.Template.Spec.Volumes, 4)\n\n\tds = DaemonSet(\"velero\", WithFeatures([]string{\"foo,bar,baz\"}))\n\tassert.Len(t, ds.Spec.Template.Spec.Containers[0].Args, 3)\n\tassert.Equal(t, \"--features=foo,bar,baz\", ds.Spec.Template.Spec.Containers[0].Args[2])\n\n\tds = DaemonSet(\"velero\", WithNodeAgentConfigMap(\"node-agent-config-map\"))\n\tassert.Len(t, ds.Spec.Template.Spec.Containers[0].Args, 3)\n\tassert.Equal(t, \"--node-agent-configmap=node-agent-config-map\", ds.Spec.Template.Spec.Containers[0].Args[2])\n\n\tds = DaemonSet(\"velero\", WithBackupRepoConfigMap(\"backup-repo-config-map\"))\n\tassert.Len(t, ds.Spec.Template.Spec.Containers[0].Args, 3)\n\tassert.Equal(t, \"--backup-repository-configmap=backup-repo-config-map\", ds.Spec.Template.Spec.Containers[0].Args[2])\n\n\tds = DaemonSet(\"velero\", WithServiceAccountName(\"test-sa\"))\n\tassert.Equal(t, \"test-sa\", ds.Spec.Template.Spec.ServiceAccountName)\n\n\tds = DaemonSet(\"velero\", WithKubeletRootDir(\"/data/test/kubelet\"))\n\tassert.Equal(t, \"/data/test/kubelet/pods\", ds.Spec.Template.Spec.Volumes[0].HostPath.Path)\n\tassert.Equal(t, \"/data/test/kubelet/plugins\", ds.Spec.Template.Spec.Volumes[1].HostPath.Path)\n\n\tds = DaemonSet(\"velero\", WithNodeAgentDisableHostPath(true))\n\tassert.Len(t, ds.Spec.Template.Spec.Volumes, 1)\n\tassert.Len(t, ds.Spec.Template.Spec.Containers[0].VolumeMounts, 1)\n\n\tds = DaemonSet(\"velero\", WithForWindows())\n\tassert.Equal(t, \"node-agent-windows\", ds.Spec.Template.Spec.Containers[0].Name)\n\tassert.Equal(t, \"velero\", ds.ObjectMeta.Namespace)\n\tassert.Equal(t, \"node-agent-windows\", ds.Spec.Template.ObjectMeta.Labels[\"name\"])\n\tassert.Equal(t, \"node-agent\", ds.Spec.Template.ObjectMeta.Labels[\"role\"])\n\tassert.Equal(t, \"windows\", string(ds.Spec.Template.Spec.OS.Name))\n\tassert.Equal(t, &corev1api.Affinity{\n\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t{\n\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\tValues: []string{\"windows\"},\n\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpIn,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}, ds.Spec.Template.Spec.Affinity)\n\tassert.Equal(t, (*corev1api.PodSecurityContext)(nil), ds.Spec.Template.Spec.SecurityContext)\n\tassert.Equal(t, (*corev1api.SecurityContext)(nil), ds.Spec.Template.Spec.Containers[0].SecurityContext)\n}\n\nfunc TestDaemonSetWithPriorityClassName(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tpriorityClassName string\n\t\texpectedValue string\n\t}{\n\t\t{\n\t\t\tname: \"with priority class name\",\n\t\t\tpriorityClassName: \"high-priority\",\n\t\t\texpectedValue: \"high-priority\",\n\t\t},\n\t\t{\n\t\t\tname: \"without priority class name\",\n\t\t\tpriorityClassName: \"\",\n\t\t\texpectedValue: \"\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t// Create a daemonset with the priority class name option\n\t\t\tvar opts []podTemplateOption\n\t\t\tif tc.priorityClassName != \"\" {\n\t\t\t\topts = append(opts, WithPriorityClassName(tc.priorityClassName))\n\t\t\t}\n\n\t\t\tdaemonset := DaemonSet(\"velero\", opts...)\n\n\t\t\t// Verify the priority class name is set correctly\n\t\t\tassert.Equal(t, tc.expectedValue, daemonset.Spec.Template.Spec.PriorityClassName)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/install/deployment.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage install\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n\t\"time\"\n\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/utils/ptr\"\n\n\t\"github.com/vmware-tanzu/velero/internal/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\ntype podTemplateOption func(*podTemplateConfig)\n\ntype podTemplateConfig struct {\n\timage string\n\tenvVars []corev1api.EnvVar\n\trestoreOnly bool\n\tannotations map[string]string\n\tlabels map[string]string\n\tresources corev1api.ResourceRequirements\n\twithSecret bool\n\tdefaultRepoMaintenanceFrequency time.Duration\n\tgarbageCollectionFrequency time.Duration\n\tpodVolumeOperationTimeout time.Duration\n\tplugins []string\n\tfeatures []string\n\tdefaultVolumesToFsBackup bool\n\tserviceAccountName string\n\tuploaderType string\n\tdefaultSnapshotMoveData bool\n\tprivilegedNodeAgent bool\n\tdisableInformerCache bool\n\tscheduleSkipImmediately bool\n\tpodResources kube.PodResources\n\tkeepLatestMaintenanceJobs int\n\tbackupRepoConfigMap string\n\trepoMaintenanceJobConfigMap string\n\tnodeAgentConfigMap string\n\titemBlockWorkerCount int\n\tconcurrentBackups int\n\tforWindows bool\n\tkubeletRootDir string\n\tnodeAgentDisableHostPath bool\n\tpriorityClassName string\n}\n\nfunc WithImage(image string) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.image = image\n\t}\n}\n\nfunc WithAnnotations(annotations map[string]string) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.annotations = annotations\n\t}\n}\n\nfunc WithLabels(labels map[string]string) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.labels = labels\n\t}\n}\n\nfunc WithEnvFromSecretKey(varName, secret, key string) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.envVars = append(c.envVars, corev1api.EnvVar{\n\t\t\tName: varName,\n\t\t\tValueFrom: &corev1api.EnvVarSource{\n\t\t\t\tSecretKeyRef: &corev1api.SecretKeySelector{\n\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\tName: secret,\n\t\t\t\t\t},\n\t\t\t\t\tKey: key,\n\t\t\t\t},\n\t\t\t},\n\t\t})\n\t}\n}\n\nfunc WithSecret(secretPresent bool) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.withSecret = secretPresent\n\t}\n}\n\nfunc WithRestoreOnly(b bool) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.restoreOnly = b\n\t}\n}\n\nfunc WithResources(resources corev1api.ResourceRequirements) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.resources = resources\n\t}\n}\n\nfunc WithDefaultRepoMaintenanceFrequency(val time.Duration) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.defaultRepoMaintenanceFrequency = val\n\t}\n}\n\nfunc WithGarbageCollectionFrequency(val time.Duration) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.garbageCollectionFrequency = val\n\t}\n}\n\nfunc WithPodVolumeOperationTimeout(val time.Duration) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.podVolumeOperationTimeout = val\n\t}\n}\n\nfunc WithPlugins(plugins []string) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.plugins = plugins\n\t}\n}\n\nfunc WithFeatures(features []string) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.features = features\n\t}\n}\n\nfunc WithUploaderType(t string) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.uploaderType = t\n\t}\n}\n\nfunc WithDefaultVolumesToFsBackup(b bool) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.defaultVolumesToFsBackup = b\n\t}\n}\n\nfunc WithDefaultSnapshotMoveData(b bool) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.defaultSnapshotMoveData = b\n\t}\n}\n\nfunc WithDisableInformerCache(b bool) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.disableInformerCache = b\n\t}\n}\n\nfunc WithServiceAccountName(sa string) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.serviceAccountName = sa\n\t}\n}\n\nfunc WithPrivilegedNodeAgent(b bool) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.privilegedNodeAgent = b\n\t}\n}\n\nfunc WithNodeAgentConfigMap(nodeAgentConfigMap string) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.nodeAgentConfigMap = nodeAgentConfigMap\n\t}\n}\n\nfunc WithScheduleSkipImmediately(b bool) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.scheduleSkipImmediately = b\n\t}\n}\n\nfunc WithPodResources(podResources kube.PodResources) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.podResources = podResources\n\t}\n}\n\nfunc WithKeepLatestMaintenanceJobs(keepLatestMaintenanceJobs int) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.keepLatestMaintenanceJobs = keepLatestMaintenanceJobs\n\t}\n}\n\nfunc WithBackupRepoConfigMap(backupRepoConfigMap string) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.backupRepoConfigMap = backupRepoConfigMap\n\t}\n}\nfunc WithRepoMaintenanceJobConfigMap(repoMaintenanceJobConfigMap string) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.repoMaintenanceJobConfigMap = repoMaintenanceJobConfigMap\n\t}\n}\n\nfunc WithItemBlockWorkerCount(itemBlockWorkerCount int) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.itemBlockWorkerCount = itemBlockWorkerCount\n\t}\n}\n\nfunc WithConcurrentBackups(concurrentBackups int) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.concurrentBackups = concurrentBackups\n\t}\n}\n\nfunc WithPriorityClassName(priorityClassName string) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.priorityClassName = priorityClassName\n\t}\n}\n\nfunc WithForWindows() podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.forWindows = true\n\t}\n}\n\nfunc WithKubeletRootDir(kubeletRootDir string) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.kubeletRootDir = kubeletRootDir\n\t}\n}\n\nfunc WithNodeAgentDisableHostPath(disable bool) podTemplateOption {\n\treturn func(c *podTemplateConfig) {\n\t\tc.nodeAgentDisableHostPath = disable\n\t}\n}\n\nfunc Deployment(namespace string, opts ...podTemplateOption) *appsv1api.Deployment {\n\t// TODO: Add support for server args\n\tc := &podTemplateConfig{\n\t\timage: velero.DefaultVeleroImage(),\n\t}\n\n\tfor _, opt := range opts {\n\t\topt(c)\n\t}\n\n\tpullPolicy := corev1api.PullAlways\n\timageParts := strings.Split(c.image, \":\")\n\tif len(imageParts) == 2 && imageParts[1] != \"latest\" {\n\t\tpullPolicy = corev1api.PullIfNotPresent\n\t}\n\n\targs := []string{\"server\"}\n\tif len(c.features) > 0 {\n\t\targs = append(args, fmt.Sprintf(\"--features=%s\", strings.Join(c.features, \",\")))\n\t}\n\n\tif c.defaultVolumesToFsBackup {\n\t\targs = append(args, \"--default-volumes-to-fs-backup=true\")\n\t}\n\n\tif c.defaultSnapshotMoveData {\n\t\targs = append(args, \"--default-snapshot-move-data=true\")\n\t}\n\n\tif c.disableInformerCache {\n\t\targs = append(args, \"--disable-informer-cache=true\")\n\t}\n\n\tif c.scheduleSkipImmediately {\n\t\targs = append(args, \"--schedule-skip-immediately=true\")\n\t}\n\n\tif len(c.uploaderType) > 0 {\n\t\targs = append(args, fmt.Sprintf(\"--uploader-type=%s\", c.uploaderType))\n\t}\n\n\tif c.restoreOnly {\n\t\targs = append(args, \"--restore-only\")\n\t}\n\n\tif c.defaultRepoMaintenanceFrequency > 0 {\n\t\targs = append(args, fmt.Sprintf(\"--default-repo-maintain-frequency=%v\", c.defaultRepoMaintenanceFrequency))\n\t}\n\n\tif c.garbageCollectionFrequency > 0 {\n\t\targs = append(args, fmt.Sprintf(\"--garbage-collection-frequency=%v\", c.garbageCollectionFrequency))\n\t}\n\n\tif c.podVolumeOperationTimeout > 0 {\n\t\targs = append(args, fmt.Sprintf(\"--fs-backup-timeout=%v\", c.podVolumeOperationTimeout))\n\t}\n\n\tif c.keepLatestMaintenanceJobs > 0 {\n\t\targs = append(args, fmt.Sprintf(\"--keep-latest-maintenance-jobs=%d\", c.keepLatestMaintenanceJobs))\n\t}\n\n\tif len(c.podResources.CPULimit) > 0 {\n\t\targs = append(args, fmt.Sprintf(\"--maintenance-job-cpu-limit=%s\", c.podResources.CPULimit))\n\t}\n\n\tif len(c.podResources.CPURequest) > 0 {\n\t\targs = append(args, fmt.Sprintf(\"--maintenance-job-cpu-request=%s\", c.podResources.CPURequest))\n\t}\n\n\tif len(c.podResources.MemoryLimit) > 0 {\n\t\targs = append(args, fmt.Sprintf(\"--maintenance-job-mem-limit=%s\", c.podResources.MemoryLimit))\n\t}\n\n\tif len(c.podResources.MemoryRequest) > 0 {\n\t\targs = append(args, fmt.Sprintf(\"--maintenance-job-mem-request=%s\", c.podResources.MemoryRequest))\n\t}\n\n\tif len(c.backupRepoConfigMap) > 0 {\n\t\targs = append(args, fmt.Sprintf(\"--backup-repository-configmap=%s\", c.backupRepoConfigMap))\n\t}\n\n\tif len(c.repoMaintenanceJobConfigMap) > 0 {\n\t\targs = append(args, fmt.Sprintf(\"--repo-maintenance-job-configmap=%s\", c.repoMaintenanceJobConfigMap))\n\t}\n\n\tif c.itemBlockWorkerCount > 0 {\n\t\targs = append(args, fmt.Sprintf(\"--item-block-worker-count=%d\", c.itemBlockWorkerCount))\n\t}\n\n\tif c.concurrentBackups > 0 {\n\t\targs = append(args, fmt.Sprintf(\"--concurrent-backups=%d\", c.concurrentBackups))\n\t}\n\n\tdeployment := &appsv1api.Deployment{\n\t\tObjectMeta: objectMeta(namespace, \"velero\"),\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"Deployment\",\n\t\t\tAPIVersion: appsv1api.SchemeGroupVersion.String(),\n\t\t},\n\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\tSelector: &metav1.LabelSelector{MatchLabels: map[string]string{\"deploy\": \"velero\"}},\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tLabels: podLabels(c.labels, map[string]string{\"deploy\": \"velero\"}),\n\t\t\t\t\tAnnotations: podAnnotations(c.annotations),\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tRestartPolicy: corev1api.RestartPolicyAlways,\n\t\t\t\t\tServiceAccountName: c.serviceAccountName,\n\t\t\t\t\tOS: &corev1api.PodOS{\n\t\t\t\t\t\tName: \"linux\",\n\t\t\t\t\t},\n\t\t\t\t\tAffinity: &corev1api.Affinity{\n\t\t\t\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\t\t\t\t\tValues: []string{\"windows\"},\n\t\t\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t\tImage: c.image,\n\t\t\t\t\t\t\tPorts: containerPorts(),\n\t\t\t\t\t\t\tImagePullPolicy: pullPolicy,\n\t\t\t\t\t\t\tCommand: []string{\n\t\t\t\t\t\t\t\t\"/velero\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tArgs: args,\n\t\t\t\t\t\t\tVolumeMounts: []corev1api.VolumeMount{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"plugins\",\n\t\t\t\t\t\t\t\t\tMountPath: \"/plugins\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"scratch\",\n\t\t\t\t\t\t\t\t\tMountPath: \"/scratch\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tEnv: []corev1api.EnvVar{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"VELERO_SCRATCH_DIR\",\n\t\t\t\t\t\t\t\t\tValue: \"/scratch\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"VELERO_NAMESPACE\",\n\t\t\t\t\t\t\t\t\tValueFrom: &corev1api.EnvVarSource{\n\t\t\t\t\t\t\t\t\t\tFieldRef: &corev1api.ObjectFieldSelector{\n\t\t\t\t\t\t\t\t\t\t\tFieldPath: \"metadata.namespace\",\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"LD_LIBRARY_PATH\",\n\t\t\t\t\t\t\t\t\tValue: \"/plugins\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tResources: c.resources,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"plugins\",\n\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\tEmptyDir: &corev1api.EmptyDirVolumeSource{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"scratch\",\n\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\tEmptyDir: new(corev1api.EmptyDirVolumeSource),\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tPriorityClassName: c.priorityClassName,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tif c.withSecret {\n\t\tdeployment.Spec.Template.Spec.Volumes = append(\n\t\t\tdeployment.Spec.Template.Spec.Volumes,\n\t\t\tcorev1api.Volume{\n\t\t\t\tName: \"cloud-credentials\",\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tSecret: &corev1api.SecretVolumeSource{\n\t\t\t\t\t\t// read-only for Owner, Group, Public\n\t\t\t\t\t\tDefaultMode: ptr.To(int32(0444)),\n\t\t\t\t\t\tSecretName: \"cloud-credentials\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t)\n\n\t\tdeployment.Spec.Template.Spec.Containers[0].VolumeMounts = append(\n\t\t\tdeployment.Spec.Template.Spec.Containers[0].VolumeMounts,\n\t\t\tcorev1api.VolumeMount{\n\t\t\t\tName: \"cloud-credentials\",\n\t\t\t\tMountPath: \"/credentials\",\n\t\t\t},\n\t\t)\n\n\t\tdeployment.Spec.Template.Spec.Containers[0].Env = append(deployment.Spec.Template.Spec.Containers[0].Env, []corev1api.EnvVar{\n\t\t\t{\n\t\t\t\tName: \"GOOGLE_APPLICATION_CREDENTIALS\",\n\t\t\t\tValue: \"/credentials/cloud\",\n\t\t\t},\n\t\t\t{\n\t\t\t\tName: \"AWS_SHARED_CREDENTIALS_FILE\",\n\t\t\t\tValue: \"/credentials/cloud\",\n\t\t\t},\n\t\t\t{\n\t\t\t\tName: \"AZURE_CREDENTIALS_FILE\",\n\t\t\t\tValue: \"/credentials/cloud\",\n\t\t\t},\n\t\t\t{\n\t\t\t\tName: \"ALIBABA_CLOUD_CREDENTIALS_FILE\",\n\t\t\t\tValue: \"/credentials/cloud\",\n\t\t\t},\n\t\t}...)\n\t}\n\n\tdeployment.Spec.Template.Spec.Containers[0].Env = append(deployment.Spec.Template.Spec.Containers[0].Env, c.envVars...)\n\n\tif len(c.plugins) > 0 {\n\t\tfor _, image := range c.plugins {\n\t\t\tcontainer := *builder.ForPluginContainer(image, pullPolicy).Result()\n\t\t\tdeployment.Spec.Template.Spec.InitContainers = append(deployment.Spec.Template.Spec.InitContainers, container)\n\t\t}\n\t}\n\n\treturn deployment\n}\n" }, { "path": "pkg/install/deployment_test.go", "content": "/*\nCopyright 2019, 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage install\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/assert\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nfunc TestDeployment(t *testing.T) {\n\tdeploy := Deployment(\"velero\")\n\n\tassert.Equal(t, \"velero\", deploy.ObjectMeta.Namespace)\n\n\tdeploy = Deployment(\"velero\", WithRestoreOnly(true))\n\tassert.Equal(t, \"--restore-only\", deploy.Spec.Template.Spec.Containers[0].Args[1])\n\n\tdeploy = Deployment(\"velero\", WithEnvFromSecretKey(\"my-var\", \"my-secret\", \"my-key\"))\n\tenvSecret := deploy.Spec.Template.Spec.Containers[0].Env[3]\n\tassert.Equal(t, \"my-var\", envSecret.Name)\n\tassert.Equal(t, \"my-secret\", envSecret.ValueFrom.SecretKeyRef.LocalObjectReference.Name)\n\tassert.Equal(t, \"my-key\", envSecret.ValueFrom.SecretKeyRef.Key)\n\n\tdeploy = Deployment(\"velero\", WithImage(\"velero/velero:v0.11\"))\n\tassert.Equal(t, \"velero/velero:v0.11\", deploy.Spec.Template.Spec.Containers[0].Image)\n\tassert.Equal(t, corev1api.PullIfNotPresent, deploy.Spec.Template.Spec.Containers[0].ImagePullPolicy)\n\n\tdeploy = Deployment(\"velero\", WithSecret(true))\n\tassert.Len(t, deploy.Spec.Template.Spec.Containers[0].Env, 7)\n\tassert.Len(t, deploy.Spec.Template.Spec.Volumes, 3)\n\n\tdeploy = Deployment(\"velero\", WithDefaultRepoMaintenanceFrequency(24*time.Hour))\n\tassert.Len(t, deploy.Spec.Template.Spec.Containers[0].Args, 2)\n\tassert.Equal(t, \"--default-repo-maintain-frequency=24h0m0s\", deploy.Spec.Template.Spec.Containers[0].Args[1])\n\n\tdeploy = Deployment(\"velero\", WithGarbageCollectionFrequency(24*time.Hour))\n\tassert.Len(t, deploy.Spec.Template.Spec.Containers[0].Args, 2)\n\tassert.Equal(t, \"--garbage-collection-frequency=24h0m0s\", deploy.Spec.Template.Spec.Containers[0].Args[1])\n\n\tdeploy = Deployment(\"velero\", WithFeatures([]string{\"EnableCSI\", \"foo\", \"bar\", \"baz\"}))\n\tassert.Len(t, deploy.Spec.Template.Spec.Containers[0].Args, 2)\n\tassert.Equal(t, \"--features=EnableCSI,foo,bar,baz\", deploy.Spec.Template.Spec.Containers[0].Args[1])\n\n\tdeploy = Deployment(\"velero\", WithUploaderType(\"kopia\"))\n\tassert.Len(t, deploy.Spec.Template.Spec.Containers[0].Args, 2)\n\tassert.Equal(t, \"--uploader-type=kopia\", deploy.Spec.Template.Spec.Containers[0].Args[1])\n\n\tdeploy = Deployment(\"velero\", WithServiceAccountName(\"test-sa\"))\n\tassert.Equal(t, \"test-sa\", deploy.Spec.Template.Spec.ServiceAccountName)\n\n\tdeploy = Deployment(\"velero\", WithDisableInformerCache(true))\n\tassert.Len(t, deploy.Spec.Template.Spec.Containers[0].Args, 2)\n\tassert.Equal(t, \"--disable-informer-cache=true\", deploy.Spec.Template.Spec.Containers[0].Args[1])\n\n\tdeploy = Deployment(\"velero\", WithKeepLatestMaintenanceJobs(3))\n\tassert.Len(t, deploy.Spec.Template.Spec.Containers[0].Args, 2)\n\tassert.Equal(t, \"--keep-latest-maintenance-jobs=3\", deploy.Spec.Template.Spec.Containers[0].Args[1])\n\n\tdeploy = Deployment(\n\t\t\"velero\",\n\t\tWithPodResources(\n\t\t\tkube.PodResources{\n\t\t\t\tCPURequest: \"100m\",\n\t\t\t\tMemoryRequest: \"256Mi\",\n\t\t\t\tCPULimit: \"200m\",\n\t\t\t\tMemoryLimit: \"512Mi\",\n\t\t\t},\n\t\t),\n\t)\n\tassert.Len(t, deploy.Spec.Template.Spec.Containers[0].Args, 5)\n\tassert.Equal(t, \"--maintenance-job-cpu-limit=200m\", deploy.Spec.Template.Spec.Containers[0].Args[1])\n\tassert.Equal(t, \"--maintenance-job-cpu-request=100m\", deploy.Spec.Template.Spec.Containers[0].Args[2])\n\tassert.Equal(t, \"--maintenance-job-mem-limit=512Mi\", deploy.Spec.Template.Spec.Containers[0].Args[3])\n\tassert.Equal(t, \"--maintenance-job-mem-request=256Mi\", deploy.Spec.Template.Spec.Containers[0].Args[4])\n\n\tdeploy = Deployment(\"velero\", WithBackupRepoConfigMap(\"test-backup-repo-config\"))\n\tassert.Len(t, deploy.Spec.Template.Spec.Containers[0].Args, 2)\n\tassert.Equal(t, \"--backup-repository-configmap=test-backup-repo-config\", deploy.Spec.Template.Spec.Containers[0].Args[1])\n\n\tdeploy = Deployment(\"velero\", WithRepoMaintenanceJobConfigMap(\"test-repo-maintenance-config\"))\n\tassert.Len(t, deploy.Spec.Template.Spec.Containers[0].Args, 2)\n\tassert.Equal(t, \"--repo-maintenance-job-configmap=test-repo-maintenance-config\", deploy.Spec.Template.Spec.Containers[0].Args[1])\n\n\tassert.Equal(t, &corev1api.Affinity{\n\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t{\n\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\tValues: []string{\"windows\"},\n\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}, deploy.Spec.Template.Spec.Affinity)\n}\n\nfunc TestDeploymentWithPriorityClassName(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tpriorityClassName string\n\t\texpectedValue string\n\t}{\n\t\t{\n\t\t\tname: \"with priority class name\",\n\t\t\tpriorityClassName: \"high-priority\",\n\t\t\texpectedValue: \"high-priority\",\n\t\t},\n\t\t{\n\t\t\tname: \"without priority class name\",\n\t\t\tpriorityClassName: \"\",\n\t\t\texpectedValue: \"\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t// Create a deployment with the priority class name option\n\t\t\tvar opts []podTemplateOption\n\t\t\tif tc.priorityClassName != \"\" {\n\t\t\t\topts = append(opts, WithPriorityClassName(tc.priorityClassName))\n\t\t\t}\n\n\t\t\tdeployment := Deployment(\"velero\", opts...)\n\n\t\t\t// Verify the priority class name is set correctly\n\t\t\tassert.Equal(t, tc.expectedValue, deployment.Spec.Template.Spec.PriorityClassName)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/install/doc.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\n// Package install provides public functions for easily creating and installing\n// resources necessary for Velero to operate. Some default settings are assumed with these functions.\npackage install\n" }, { "path": "pkg/install/import_test.go", "content": "package install\n\nimport (\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"regexp\"\n\t\"runtime\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\n// test that this package do not import cloud provider\n\n// Prevent https://github.com/vmware-tanzu/velero/issues/8207 and https://github.com/vmware-tanzu/velero/issues/8157\nfunc TestPkgImportNoCloudProvider(t *testing.T) {\n\t_, filename, _, ok := runtime.Caller(0)\n\tif !ok {\n\t\tt.Fatalf(\"No caller information\")\n\t}\n\tt.Logf(\"Current test file path: %s\", filename)\n\tt.Logf(\"Current test directory: %s\", filepath.Dir(filename)) // should be this package name\n\t// go list -f {{.Deps}} ./\n\tcmd := exec.CommandContext(\n\t\tt.Context(),\n\t\t\"go\",\n\t\t\"list\",\n\t\t\"-f\",\n\t\t\"{{.Deps}}\",\n\t\t\".\",\n\t)\n\t// set cmd.Dir to this package even if executed from different dir\n\tcmd.Dir = filepath.Dir(filename)\n\toutput, err := cmd.Output()\n\trequire.NoError(t, err)\n\t// split dep by line, replace space with newline\n\tdeps := strings.ReplaceAll(string(output), \" \", \"\\n\")\n\trequire.NotEmpty(t, deps)\n\t// ignore k8s.io\n\tk8sio, err := regexp.Compile(\"^k8s.io\")\n\trequire.NoError(t, err)\n\tcloudProvider, err := regexp.Compile(\"aws|cloud.google.com|azure\")\n\trequire.NoError(t, err)\n\tcloudProviderDeps := []string{}\n\tfor _, dep := range strings.Split(deps, \"\\n\") {\n\t\tif !k8sio.MatchString(dep) {\n\t\t\tif cloudProvider.MatchString(dep) {\n\t\t\t\tcloudProviderDeps = append(cloudProviderDeps, dep)\n\t\t\t}\n\t\t}\n\t}\n\trequire.Empty(t, cloudProviderDeps)\n}\n" }, { "path": "pkg/install/install.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage install\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapiextv1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1\"\n\tapiextv1beta1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\n// kindToResource translates a Kind (mixed case, singular) to a Resource (lowercase, plural) string.\n// This is to accommodate the dynamic client's need for an APIResource, as the Unstructured objects do not have easy helpers for this information.\nvar kindToResource = map[string]string{\n\t\"CustomResourceDefinition\": \"customresourcedefinitions\",\n\t\"Namespace\": \"namespaces\",\n\t\"ClusterRoleBinding\": \"clusterrolebindings\",\n\t\"ServiceAccount\": \"serviceaccounts\",\n\t\"Deployment\": \"deployments\",\n\t\"DaemonSet\": \"daemonsets\",\n\t\"Secret\": \"secrets\",\n\t\"ConfigMap\": \"configmaps\",\n\t\"BackupStorageLocation\": \"backupstoragelocations\",\n\t\"VolumeSnapshotLocation\": \"volumesnapshotlocations\",\n}\n\n// ResourceGroup represents a collection of Kubernetes objects with a common ready condition\ntype ResourceGroup struct {\n\tCRDResources []*unstructured.Unstructured\n\tOtherResources []*unstructured.Unstructured\n}\n\n// crdV1Beta1ReadinessFn returns a function that can be used for polling to check\n// if the provided unstructured v1beta1 CRDs are ready for use in the cluster.\nfunc crdV1Beta1ReadinessFn(kbClient kbclient.Client, unstructuredCrds []*unstructured.Unstructured) func(context.Context) (bool, error) {\n\t// Track all the CRDs that have been found and in ready state.\n\t// len should be equal to len(unstructuredCrds) in the happy path.\n\treturn func(ctx context.Context) (bool, error) {\n\t\tfoundCRDs := make([]*apiextv1beta1.CustomResourceDefinition, 0)\n\t\tfor _, unstructuredCrd := range unstructuredCrds {\n\t\t\tcrd := &apiextv1beta1.CustomResourceDefinition{}\n\t\t\tkey := kbclient.ObjectKey{Name: unstructuredCrd.GetName()}\n\t\t\terr := kbClient.Get(ctx, key, crd)\n\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\treturn false, nil\n\t\t\t} else if err != nil {\n\t\t\t\treturn false, errors.Wrapf(err, \"error waiting for %s to be ready\", crd.GetName())\n\t\t\t}\n\t\t\tfoundCRDs = append(foundCRDs, crd)\n\t\t}\n\n\t\tif len(foundCRDs) != len(unstructuredCrds) {\n\t\t\treturn false, nil\n\t\t}\n\n\t\tfor _, crd := range foundCRDs {\n\t\t\tready := kube.IsV1Beta1CRDReady(crd)\n\t\t\tif !ready {\n\t\t\t\treturn false, nil\n\t\t\t}\n\t\t}\n\t\treturn true, nil\n\t}\n}\n\n// crdV1ReadinessFn returns a function that can be used for polling to check\n// if the provided unstructured v1 CRDs are ready for use in the cluster.\nfunc crdV1ReadinessFn(kbClient kbclient.Client, unstructuredCrds []*unstructured.Unstructured) func(context.Context) (bool, error) {\n\treturn func(ctx context.Context) (bool, error) {\n\t\tfoundCRDs := make([]*apiextv1.CustomResourceDefinition, 0)\n\t\tfor _, unstructuredCrd := range unstructuredCrds {\n\t\t\tcrd := &apiextv1.CustomResourceDefinition{}\n\t\t\tkey := kbclient.ObjectKey{Name: unstructuredCrd.GetName()}\n\t\t\terr := kbClient.Get(ctx, key, crd)\n\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\treturn false, nil\n\t\t\t} else if err != nil {\n\t\t\t\treturn false, errors.Wrapf(err, \"error waiting for %s to be ready\", crd.GetName())\n\t\t\t}\n\t\t\tfoundCRDs = append(foundCRDs, crd)\n\t\t}\n\n\t\tif len(foundCRDs) != len(unstructuredCrds) {\n\t\t\treturn false, nil\n\t\t}\n\n\t\tfor _, crd := range foundCRDs {\n\t\t\tready := kube.IsV1CRDReady(crd)\n\t\t\tif !ready {\n\t\t\t\treturn false, nil\n\t\t\t}\n\t\t}\n\t\treturn true, nil\n\t}\n}\n\n// crdsAreReady polls the API server to see if the Velero CRDs are ready to create objects.\nfunc crdsAreReady(kbClient kbclient.Client, crds []*unstructured.Unstructured) (bool, error) {\n\tif len(crds) == 0 {\n\t\t// no CRDs to check so return\n\t\treturn true, nil\n\t}\n\n\t// We assume that all Velero CRDs have the same GVK so we can use the GVK of the\n\t// first CRD to determine whether to use the v1beta1 or v1 API during polling.\n\tgvk := crds[0].GroupVersionKind()\n\n\tvar crdReadinessFn func(context.Context) (bool, error)\n\tif gvk.Version == \"v1beta1\" {\n\t\tcrdReadinessFn = crdV1Beta1ReadinessFn(kbClient, crds)\n\t} else if gvk.Version == \"v1\" {\n\t\tcrdReadinessFn = crdV1ReadinessFn(kbClient, crds)\n\t} else {\n\t\treturn false, fmt.Errorf(\"unsupported CRD version %q\", gvk.Version)\n\t}\n\n\terr := wait.PollUntilContextTimeout(context.Background(), time.Second, time.Minute, true, crdReadinessFn)\n\tif err != nil {\n\t\treturn false, errors.Wrap(err, \"Error polling for CRDs\")\n\t}\n\treturn true, nil\n}\n\nfunc isAvailable(c appsv1api.DeploymentCondition) bool {\n\t// Make sure that the deployment has been available for at least 10 seconds.\n\t// This is because the deployment can show as Ready momentarily before the pods fall into a CrashLoopBackOff.\n\t// See podutils.IsPodAvailable upstream for similar logic with pods\n\tif c.Type == appsv1api.DeploymentAvailable && c.Status == corev1api.ConditionTrue {\n\t\tif !c.LastTransitionTime.IsZero() && c.LastTransitionTime.Add(10*time.Second).Before(time.Now()) {\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n\n// DeploymentIsReady will poll the Kubernetes API server to see if the velero deployment is ready to service user requests.\nfunc DeploymentIsReady(factory client.DynamicFactory, namespace string) (bool, error) {\n\tgvk := schema.FromAPIVersionAndKind(appsv1api.SchemeGroupVersion.String(), \"Deployment\")\n\tapiResource := metav1.APIResource{\n\t\tName: \"deployments\",\n\t\tNamespaced: true,\n\t}\n\tc, err := factory.ClientForGroupVersionResource(gvk.GroupVersion(), apiResource, namespace)\n\tif err != nil {\n\t\treturn false, errors.Wrapf(err, \"Error creating client for deployment polling\")\n\t}\n\t// declare this variable out of scope so we can return it\n\tvar isReady bool\n\tvar readyObservations int32\n\terr = wait.PollUntilContextTimeout(context.Background(), time.Second, 3*time.Minute, true, func(ctx context.Context) (bool, error) {\n\t\tunstructuredDeployment, err := c.Get(\"velero\", metav1.GetOptions{})\n\t\tif apierrors.IsNotFound(err) {\n\t\t\treturn false, nil\n\t\t} else if err != nil {\n\t\t\treturn false, errors.Wrap(err, \"error waiting for deployment to be ready\")\n\t\t}\n\n\t\tdeploy := new(appsv1api.Deployment)\n\t\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(unstructuredDeployment.Object, deploy); err != nil {\n\t\t\treturn false, errors.Wrap(err, \"error converting deployment from unstructured\")\n\t\t}\n\n\t\tfor _, cond := range deploy.Status.Conditions {\n\t\t\tif isAvailable(cond) {\n\t\t\t\treadyObservations++\n\t\t\t}\n\t\t}\n\t\t// Make sure we query the deployment enough times to see the state change, provided there is one.\n\t\tif readyObservations > 4 {\n\t\t\tisReady = true\n\t\t\treturn true, nil\n\t\t}\n\t\treturn false, nil\n\t})\n\treturn isReady, err\n}\n\n// NodeAgentIsReady will poll the Kubernetes API server to ensure the node-agent daemonset is ready, i.e. that\n// pods are scheduled and available on all of the desired nodes.\nfunc NodeAgentIsReady(factory client.DynamicFactory, namespace string) (bool, error) {\n\treturn daemonSetIsReady(factory, namespace, \"node-agent\")\n}\n\n// NodeAgentWindowsIsReady will poll the Kubernetes API server to ensure the node-agent-windows daemonset is ready, i.e. that\n// pods are scheduled and available on all of the desired nodes.\nfunc NodeAgentWindowsIsReady(factory client.DynamicFactory, namespace string) (bool, error) {\n\treturn daemonSetIsReady(factory, namespace, \"node-agent-windows\")\n}\n\nfunc daemonSetIsReady(factory client.DynamicFactory, namespace string, name string) (bool, error) {\n\tgvk := schema.FromAPIVersionAndKind(appsv1api.SchemeGroupVersion.String(), \"DaemonSet\")\n\tapiResource := metav1.APIResource{\n\t\tName: \"daemonsets\",\n\t\tNamespaced: true,\n\t}\n\n\tc, err := factory.ClientForGroupVersionResource(gvk.GroupVersion(), apiResource, namespace)\n\tif err != nil {\n\t\treturn false, errors.Wrapf(err, \"Error creating client for daemonset polling\")\n\t}\n\n\t// declare this variable out of scope so we can return it\n\tvar isReady bool\n\tvar readyObservations int32\n\n\terr = wait.PollUntilContextTimeout(context.Background(), time.Second, time.Minute, true, func(ctx context.Context) (bool, error) {\n\t\tunstructuredDaemonSet, err := c.Get(name, metav1.GetOptions{})\n\t\tif apierrors.IsNotFound(err) {\n\t\t\treturn false, nil\n\t\t} else if err != nil {\n\t\t\treturn false, errors.Wrap(err, \"error waiting for daemonset to be ready\")\n\t\t}\n\n\t\tdaemonSet := new(appsv1api.DaemonSet)\n\t\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(unstructuredDaemonSet.Object, daemonSet); err != nil {\n\t\t\treturn false, errors.Wrap(err, \"error converting daemonset from unstructured\")\n\t\t}\n\n\t\tif daemonSet.Status.NumberAvailable == daemonSet.Status.DesiredNumberScheduled {\n\t\t\treadyObservations++\n\t\t}\n\n\t\t// Wait for 5 observations of the daemonset being \"ready\" to be consistent with our check for\n\t\t// the deployment being ready.\n\t\tif readyObservations > 4 {\n\t\t\tisReady = true\n\t\t\treturn true, nil\n\t\t}\n\t\treturn false, nil\n\t})\n\treturn isReady, err\n}\n\n// GroupResources groups resources based on whether the resources are CustomResourceDefinitions or other types of Kubernetes objects\n// This is useful to wait for readiness before creating CRD objects\nfunc GroupResources(resources *unstructured.UnstructuredList) *ResourceGroup {\n\trg := new(ResourceGroup)\n\n\tfor i, r := range resources.Items {\n\t\tif r.GetKind() == \"CustomResourceDefinition\" {\n\t\t\trg.CRDResources = append(rg.CRDResources, &resources.Items[i])\n\t\t\tcontinue\n\t\t}\n\t\trg.OtherResources = append(rg.OtherResources, &resources.Items[i])\n\t}\n\n\treturn rg\n}\n\n// createOrApplyResource attempts to create or apply a resource in the cluster.\n// If apply is true, it uses server-side apply to update existing resources.\n// If apply is false and the resource already exists in the cluster, it's merely logged.\nfunc createOrApplyResource(r *unstructured.Unstructured, factory client.DynamicFactory, w io.Writer, apply bool) error {\n\tid := fmt.Sprintf(\"%s/%s\", r.GetKind(), r.GetName())\n\n\t// Helper to reduce boilerplate message about the same object\n\tlog := func(f string) {\n\t\tfmt.Fprintf(w, \"%s: %s\\n\", id, f)\n\t}\n\n\tc, err := CreateClient(r, factory, w)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif apply {\n\t\tlog(\"attempting to apply resource\")\n\t\t// Set field manager for server-side apply and force to override conflicts\n\t\tapplyOpts := metav1.ApplyOptions{\n\t\t\tFieldManager: \"velero-cli\",\n\t\t\tForce: true,\n\t\t}\n\n\t\tif _, err := c.Apply(r.GetName(), r, applyOpts); err != nil {\n\t\t\treturn errors.Wrapf(err, \"Error applying resource %s\", id)\n\t\t}\n\t\tlog(\"applied\")\n\t} else {\n\t\tlog(\"attempting to create resource\")\n\t\tif _, err := c.Create(r); apierrors.IsAlreadyExists(err) {\n\t\t\tlog(\"already exists, proceeding\")\n\t\t} else if err != nil {\n\t\t\treturn errors.Wrapf(err, \"Error creating resource %s\", id)\n\t\t} else {\n\t\t\tlog(\"created\")\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// CreateClient creates a client for an unstructured resource\nfunc CreateClient(r *unstructured.Unstructured, factory client.DynamicFactory, w io.Writer) (client.Dynamic, error) {\n\tid := fmt.Sprintf(\"%s/%s\", r.GetKind(), r.GetName())\n\n\t// Helper to reduce boilerplate message about the same object\n\tlog := func(f string, a ...any) {\n\t\tformat := strings.Join([]string{id, \": \", f, \"\\n\"}, \"\")\n\t\tfmt.Fprintf(w, format, a...)\n\t}\n\tlog(\"attempting to create resource client\")\n\n\tgvk := schema.FromAPIVersionAndKind(r.GetAPIVersion(), r.GetKind())\n\n\tapiResource := metav1.APIResource{\n\t\tName: kindToResource[r.GetKind()],\n\t\tNamespaced: (r.GetNamespace() != \"\"),\n\t}\n\n\tc, err := factory.ClientForGroupVersionResource(gvk.GroupVersion(), apiResource, r.GetNamespace())\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"Error creating client for resource %s\", id)\n\t}\n\n\treturn c, nil\n}\n\n// Install creates resources on the Kubernetes cluster.\n// An unstructured list of resources is sent, one at a time, to the server. These are assumed to be in the preferred order already.\n// Resources will be sorted into CustomResourceDefinitions and any other resource type, and the function will wait up to 1 minute\n// for CRDs to be ready before proceeding.\n// If apply is true, it uses server-side apply to update existing resources.\n// An io.Writer can be used to output to a log or the console.\nfunc Install(dynamicFactory client.DynamicFactory, kbClient kbclient.Client, resources *unstructured.UnstructuredList, w io.Writer, apply bool) error {\n\trg := GroupResources(resources)\n\n\t//Install CRDs first\n\tfor _, r := range rg.CRDResources {\n\t\tif err := createOrApplyResource(r, dynamicFactory, w, apply); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\t// Wait for CRDs to be ready before proceeding\n\tfmt.Fprint(w, \"Waiting for resources to be ready in cluster...\\n\")\n\t_, err := crdsAreReady(kbClient, rg.CRDResources)\n\tif wait.Interrupted(err) {\n\t\treturn errors.Errorf(\"timeout reached, CRDs not ready\")\n\t} else if err != nil {\n\t\treturn err\n\t}\n\n\t// Install all other resources\n\tfor _, r := range rg.OtherResources {\n\t\tif err = createOrApplyResource(r, dynamicFactory, w, apply); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn nil\n}\n" }, { "path": "pkg/install/install_test.go", "content": "package install\n\nimport (\n\t\"bytes\"\n\t\"errors\"\n\t\"os\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapiextv1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1\"\n\tapiextv1beta1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tv1crds \"github.com/vmware-tanzu/velero/config/crd/v1/crds\"\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestInstall(t *testing.T) {\n\tdc := &test.FakeDynamicClient{}\n\tdc.On(\"Create\", mock.Anything).Return(&unstructured.Unstructured{}, nil)\n\n\tfactory := &test.FakeDynamicFactory{}\n\tfactory.On(\"ClientForGroupVersionResource\", mock.Anything, mock.Anything, mock.Anything).Return(dc, nil)\n\n\tc := fake.NewClientBuilder().WithObjects(\n\t\t&apiextv1.CustomResourceDefinition{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"backuprepositories.velero.io\",\n\t\t\t},\n\n\t\t\tStatus: apiextv1.CustomResourceDefinitionStatus{\n\t\t\t\tConditions: []apiextv1.CustomResourceDefinitionCondition{\n\t\t\t\t\t{\n\t\t\t\t\t\tType: apiextv1.Established,\n\t\t\t\t\t\tStatus: apiextv1.ConditionTrue,\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tType: apiextv1.NamesAccepted,\n\t\t\t\t\t\tStatus: apiextv1.ConditionTrue,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t).Build()\n\n\tresources := &unstructured.UnstructuredList{}\n\trequire.NoError(t, appendUnstructured(resources, v1crds.CRDs[0]))\n\trequire.NoError(t, appendUnstructured(resources, Namespace(\"velero\")))\n\n\tassert.NoError(t, Install(factory, c, resources, os.Stdout, false))\n}\n\nfunc Test_crdsAreReady(t *testing.T) {\n\tc := fake.NewClientBuilder().WithObjects(\n\t\t&apiextv1beta1.CustomResourceDefinition{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"backuprepositories.velero.io\",\n\t\t\t},\n\n\t\t\tStatus: apiextv1beta1.CustomResourceDefinitionStatus{\n\t\t\t\tConditions: []apiextv1beta1.CustomResourceDefinitionCondition{\n\t\t\t\t\t{\n\t\t\t\t\t\tType: apiextv1beta1.Established,\n\t\t\t\t\t\tStatus: apiextv1beta1.ConditionTrue,\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tType: apiextv1beta1.NamesAccepted,\n\t\t\t\t\t\tStatus: apiextv1beta1.ConditionTrue,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t).Build()\n\n\tcrd := &apiextv1beta1.CustomResourceDefinition{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"CustomResourceDefinition\",\n\t\t\tAPIVersion: \"v1beta1\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"backuprepositories.velero.io\",\n\t\t},\n\t}\n\tobj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(crd)\n\trequire.NoError(t, err)\n\n\tcrds := []*unstructured.Unstructured{\n\t\t{\n\t\t\tObject: obj,\n\t\t},\n\t}\n\n\tready, err := crdsAreReady(c, crds)\n\trequire.NoError(t, err)\n\tassert.True(t, ready)\n}\n\nfunc TestDeploymentIsReady(t *testing.T) {\n\tdeployment := &appsv1api.Deployment{\n\t\tStatus: appsv1api.DeploymentStatus{\n\t\t\tConditions: []appsv1api.DeploymentCondition{\n\t\t\t\t{\n\t\t\t\t\tType: appsv1api.DeploymentAvailable,\n\t\t\t\t\tStatus: corev1api.ConditionTrue,\n\t\t\t\t\tLastTransitionTime: metav1.NewTime(time.Now().Add(-15 * time.Second)),\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tobj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(deployment)\n\trequire.NoError(t, err)\n\n\tdc := &test.FakeDynamicClient{}\n\tdc.On(\"Get\", mock.Anything, mock.Anything).Return(&unstructured.Unstructured{Object: obj}, nil)\n\n\tfactory := &test.FakeDynamicFactory{}\n\tfactory.On(\"ClientForGroupVersionResource\", mock.Anything, mock.Anything, mock.Anything).Return(dc, nil)\n\n\tready, err := DeploymentIsReady(factory, \"velero\")\n\trequire.NoError(t, err)\n\tassert.True(t, ready)\n}\n\nfunc TestNodeAgentIsReady(t *testing.T) {\n\tdaemonset := &appsv1api.DaemonSet{\n\t\tStatus: appsv1api.DaemonSetStatus{\n\t\t\tNumberAvailable: 1,\n\t\t\tDesiredNumberScheduled: 1,\n\t\t},\n\t}\n\tobj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(daemonset)\n\trequire.NoError(t, err)\n\n\tdc := &test.FakeDynamicClient{}\n\tdc.On(\"Get\", mock.Anything, mock.Anything).Return(&unstructured.Unstructured{Object: obj}, nil)\n\n\tfactory := &test.FakeDynamicFactory{}\n\tfactory.On(\"ClientForGroupVersionResource\", mock.Anything, mock.Anything, mock.Anything).Return(dc, nil)\n\n\tready, err := NodeAgentIsReady(factory, \"velero\")\n\trequire.NoError(t, err)\n\tassert.True(t, ready)\n}\n\nfunc TestNodeAgentWindowsIsReady(t *testing.T) {\n\tdaemonset := &appsv1api.DaemonSet{\n\t\tStatus: appsv1api.DaemonSetStatus{\n\t\t\tNumberAvailable: 0,\n\t\t\tDesiredNumberScheduled: 0,\n\t\t},\n\t}\n\tobj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(daemonset)\n\trequire.NoError(t, err)\n\n\tdc := &test.FakeDynamicClient{}\n\tdc.On(\"Get\", mock.Anything, mock.Anything).Return(&unstructured.Unstructured{Object: obj}, nil)\n\n\tfactory := &test.FakeDynamicFactory{}\n\tfactory.On(\"ClientForGroupVersionResource\", mock.Anything, mock.Anything, mock.Anything).Return(dc, nil)\n\n\tready, err := NodeAgentWindowsIsReady(factory, \"velero\")\n\trequire.NoError(t, err)\n\tassert.True(t, ready)\n}\n\nfunc TestCreateOrApplyResourceError(t *testing.T) {\n\tr := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"ConfigMap\",\n\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\"name\": \"test-configmap\",\n\t\t\t\t\"namespace\": \"velero\",\n\t\t\t},\n\t\t},\n\t}\n\n\tdc := &test.FakeDynamicClient{}\n\texpectedErr := errors.New(\"create error\")\n\tdc.On(\"Create\", mock.Anything).Return(&unstructured.Unstructured{}, expectedErr)\n\n\tfactory := &test.FakeDynamicFactory{}\n\tfactory.On(\"ClientForGroupVersionResource\", mock.Anything, mock.Anything, mock.Anything).Return(dc, nil)\n\n\tvar buf bytes.Buffer\n\terr := createOrApplyResource(r, factory, &buf, false)\n\n\trequire.Error(t, err)\n\trequire.Contains(t, err.Error(), expectedErr.Error())\n}\n\nfunc TestCreateOrApplyResourceAlreadyExists(t *testing.T) {\n\tr := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"ConfigMap\",\n\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\"name\": \"test-configmap\",\n\t\t\t\t\"namespace\": \"velero\",\n\t\t\t},\n\t\t},\n\t}\n\n\tdc := &test.FakeDynamicClient{}\n\talreadyExistsErr := apierrors.NewAlreadyExists(schema.GroupResource{Resource: \"configmaps\"}, \"test-configmap\")\n\t// We need to return a non-nil unstructured object even though it's not used\n\tdc.On(\"Create\", mock.Anything).Return(&unstructured.Unstructured{}, alreadyExistsErr)\n\n\tfactory := &test.FakeDynamicFactory{}\n\tfactory.On(\"ClientForGroupVersionResource\", mock.Anything, mock.Anything, mock.Anything).Return(dc, nil)\n\n\tvar buf bytes.Buffer\n\terr := createOrApplyResource(r, factory, &buf, false)\n\n\trequire.NoError(t, err)\n}\n\nfunc TestCreateOrApplyResourceClientError(t *testing.T) {\n\tr := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"ConfigMap\",\n\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\"name\": \"test-configmap\",\n\t\t\t\t\"namespace\": \"velero\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfactory := &test.FakeDynamicFactory{}\n\texpectedErr := errors.New(\"client creation error\")\n\t// Return error from ClientForGroupVersionResource\n\tfactory.On(\"ClientForGroupVersionResource\", mock.Anything, mock.Anything, mock.Anything).Return(&test.FakeDynamicClient{}, expectedErr)\n\n\tvar buf bytes.Buffer\n\terr := createOrApplyResource(r, factory, &buf, false)\n\n\trequire.Error(t, err)\n\trequire.Contains(t, err.Error(), expectedErr.Error())\n}\n\nfunc TestCreateOrApplyResourceApplyError(t *testing.T) {\n\tr := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"ConfigMap\",\n\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\"name\": \"test-configmap\",\n\t\t\t\t\"namespace\": \"velero\",\n\t\t\t},\n\t\t},\n\t}\n\n\tdc := &test.FakeDynamicClient{}\n\texpectedErr := errors.New(\"apply error\")\n\t// Mock Apply to return an error\n\tdc.On(\"Apply\", mock.Anything, mock.Anything, mock.Anything).Return(&unstructured.Unstructured{}, expectedErr)\n\n\tfactory := &test.FakeDynamicFactory{}\n\tfactory.On(\"ClientForGroupVersionResource\", mock.Anything, mock.Anything, mock.Anything).Return(dc, nil)\n\n\tvar buf bytes.Buffer\n\terr := createOrApplyResource(r, factory, &buf, true) // true for apply flag to use Apply\n\n\trequire.Error(t, err)\n\trequire.Contains(t, err.Error(), expectedErr.Error())\n}\n\nfunc TestInstallErrorAfterCreateClient(t *testing.T) {\n\t// Create a test non-CRD resource\n\tnonCRDResource := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"ConfigMap\",\n\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\"name\": \"test-configmap\",\n\t\t\t},\n\t\t},\n\t}\n\n\tresources := &unstructured.UnstructuredList{\n\t\tItems: []unstructured.Unstructured{*nonCRDResource},\n\t}\n\n\t// Mock the factory to return a client that will succeed on ClientForGroupVersionResource\n\t// but fail on Create\n\tdc := &test.FakeDynamicClient{}\n\texpectedErr := errors.New(\"create error after successful client creation\")\n\tdc.On(\"Create\", mock.Anything).Return(&unstructured.Unstructured{}, expectedErr)\n\n\tfactory := &test.FakeDynamicFactory{}\n\tfactory.On(\"ClientForGroupVersionResource\", mock.Anything, mock.Anything, mock.Anything).Return(dc, nil)\n\n\tc := fake.NewClientBuilder().Build()\n\n\tvar buf bytes.Buffer\n\terr := Install(factory, c, resources, &buf, false)\n\n\trequire.Error(t, err)\n\trequire.Contains(t, err.Error(), expectedErr.Error())\n}\n\nfunc TestInstallErrorOnCRDResource(t *testing.T) {\n\tcrdResource := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"apiVersion\": \"apiextensions.k8s.io/v1\",\n\t\t\t\"kind\": \"CustomResourceDefinition\",\n\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\"name\": \"test-crd\",\n\t\t\t},\n\t\t},\n\t}\n\n\tresources := &unstructured.UnstructuredList{\n\t\tItems: []unstructured.Unstructured{*crdResource},\n\t}\n\n\tdc := &test.FakeDynamicClient{}\n\texpectedErr := errors.New(\"error creating CRD resource\")\n\t// We need to return a non-nil unstructured object even though it's not used\n\tdc.On(\"Create\", mock.Anything).Return(&unstructured.Unstructured{}, expectedErr)\n\n\tfactory := &test.FakeDynamicFactory{}\n\tfactory.On(\"ClientForGroupVersionResource\", mock.Anything, mock.Anything, mock.Anything).Return(dc, nil)\n\n\tc := fake.NewClientBuilder().Build()\n\n\tvar buf bytes.Buffer\n\terr := Install(factory, c, resources, &buf, false)\n\n\trequire.Error(t, err)\n\trequire.Contains(t, err.Error(), expectedErr.Error())\n}\n\nfunc TestInstallWithApplyFlag(t *testing.T) {\n\t// Create a test resource\n\ttestResource := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"ConfigMap\",\n\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\"name\": \"test-configmap\",\n\t\t\t\t\"namespace\": \"velero\",\n\t\t\t},\n\t\t\t\"data\": map[string]any{\n\t\t\t\t\"key1\": \"value1\",\n\t\t\t},\n\t\t},\n\t}\n\n\tresources := &unstructured.UnstructuredList{\n\t\tItems: []unstructured.Unstructured{*testResource},\n\t}\n\n\t// Test case 1: Without apply flag (create)\n\t{\n\t\tdc := &test.FakeDynamicClient{}\n\t\t// Expect Create to be called\n\t\tdc.On(\"Create\", mock.Anything).Return(testResource, nil)\n\t\t// Apply should not be called\n\n\t\tfactory := &test.FakeDynamicFactory{}\n\t\tfactory.On(\"ClientForGroupVersionResource\", mock.Anything, mock.Anything, mock.Anything).Return(dc, nil)\n\n\t\tc := fake.NewClientBuilder().Build()\n\n\t\terr := Install(factory, c, resources, os.Stdout, false)\n\t\trequire.NoError(t, err)\n\n\t\t// Verify that Create was called and Apply was not\n\t\tdc.AssertCalled(t, \"Create\", mock.Anything)\n\t\tdc.AssertNotCalled(t, \"Apply\", mock.Anything, mock.Anything, mock.Anything)\n\t}\n\n\t// Test case 2: With apply flag\n\t{\n\t\tdc := &test.FakeDynamicClient{}\n\t\t// Create should not be called\n\t\t// Expect Apply to be called\n\t\tdc.On(\"Apply\", mock.Anything, mock.Anything, mock.Anything).Return(testResource, nil)\n\n\t\tfactory := &test.FakeDynamicFactory{}\n\t\tfactory.On(\"ClientForGroupVersionResource\", mock.Anything, mock.Anything, mock.Anything).Return(dc, nil)\n\n\t\tc := fake.NewClientBuilder().Build()\n\n\t\terr := Install(factory, c, resources, os.Stdout, true)\n\t\trequire.NoError(t, err)\n\n\t\t// Verify that Apply was called and Create was not\n\t\tdc.AssertCalled(t, \"Apply\", mock.Anything, mock.Anything, mock.Anything)\n\t\tdc.AssertNotCalled(t, \"Create\", mock.Anything)\n\t}\n}\n" }, { "path": "pkg/install/resources.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage install\n\nimport (\n\t\"fmt\"\n\t\"time\"\n\n\tcorev1api \"k8s.io/api/core/v1\"\n\trbacv1 \"k8s.io/api/rbac/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\tv1crds \"github.com/vmware-tanzu/velero/config/crd/v1/crds\"\n\tv2alpha1crds \"github.com/vmware-tanzu/velero/config/crd/v2alpha1/crds\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nconst (\n\tdefaultServiceAccountName = \"velero\"\n\tpodSecurityLevel = \"privileged\"\n\tpodSecurityVersion = \"latest\"\n)\n\nvar (\n\t// default values for Velero server pod resource request/limit\n\tDefaultVeleroPodCPURequest = \"500m\"\n\tDefaultVeleroPodMemRequest = \"128Mi\"\n\tDefaultVeleroPodCPULimit = \"1000m\"\n\tDefaultVeleroPodMemLimit = \"512Mi\"\n\n\t// default values for node-agent pod resource request/limit,\n\t// \"0\" means no request/limit is set, so as to make the QoS as BestEffort\n\tDefaultNodeAgentPodCPURequest = \"0\"\n\tDefaultNodeAgentPodMemRequest = \"0\"\n\tDefaultNodeAgentPodCPULimit = \"0\"\n\tDefaultNodeAgentPodMemLimit = \"0\"\n\n\tDefaultVeleroNamespace = \"velero\"\n\n\tDefaultKubeletRootDir = \"/var/lib/kubelet\"\n)\n\nfunc Labels() map[string]string {\n\treturn map[string]string{\n\t\t\"component\": \"velero\",\n\t}\n}\n\nfunc podLabels(userLabels ...map[string]string) map[string]string {\n\t// Use the default labels as a starting point\n\tbase := Labels()\n\n\t// Merge base labels with user labels to enforce CLI precedence\n\tfor _, labels := range userLabels {\n\t\tfor k, v := range labels {\n\t\t\tbase[k] = v\n\t\t}\n\t}\n\n\treturn base\n}\n\nfunc podAnnotations(userAnnotations map[string]string) map[string]string {\n\t// Use the default annotations as a starting point\n\tbase := map[string]string{\n\t\t\"prometheus.io/scrape\": \"true\",\n\t\t\"prometheus.io/port\": \"8085\",\n\t\t\"prometheus.io/path\": \"/metrics\",\n\t}\n\n\t// Merge base annotations with user annotations to enforce CLI precedence\n\tfor k, v := range userAnnotations {\n\t\tbase[k] = v\n\t}\n\n\treturn base\n}\n\nfunc containerPorts() []corev1api.ContainerPort {\n\treturn []corev1api.ContainerPort{\n\t\t{\n\t\t\tName: \"metrics\",\n\t\t\tContainerPort: 8085,\n\t\t},\n\t}\n}\n\nfunc objectMeta(namespace, name string) metav1.ObjectMeta {\n\treturn metav1.ObjectMeta{\n\t\tName: name,\n\t\tNamespace: namespace,\n\t\tLabels: Labels(),\n\t}\n}\n\nfunc ServiceAccount(namespace string, annotations map[string]string) *corev1api.ServiceAccount {\n\tobjMeta := objectMeta(namespace, defaultServiceAccountName)\n\tobjMeta.Annotations = annotations\n\treturn &corev1api.ServiceAccount{\n\t\tObjectMeta: objMeta,\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"ServiceAccount\",\n\t\t\tAPIVersion: corev1api.SchemeGroupVersion.String(),\n\t\t},\n\t}\n}\n\nfunc ClusterRoleBinding(namespace string) *rbacv1.ClusterRoleBinding {\n\tcrbName := \"velero\"\n\tif namespace != DefaultVeleroNamespace {\n\t\tcrbName = \"velero-\" + namespace\n\t}\n\tcrb := &rbacv1.ClusterRoleBinding{\n\t\tObjectMeta: objectMeta(\"\", crbName),\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"ClusterRoleBinding\",\n\t\t\tAPIVersion: rbacv1.SchemeGroupVersion.String(),\n\t\t},\n\t\tSubjects: []rbacv1.Subject{\n\t\t\t{\n\t\t\t\tKind: \"ServiceAccount\",\n\t\t\t\tNamespace: namespace,\n\t\t\t\tName: \"velero\",\n\t\t\t},\n\t\t},\n\t\tRoleRef: rbacv1.RoleRef{\n\t\t\tKind: \"ClusterRole\",\n\t\t\tName: \"cluster-admin\",\n\t\t\tAPIGroup: \"rbac.authorization.k8s.io\",\n\t\t},\n\t}\n\n\treturn crb\n}\n\nfunc Namespace(namespace string) *corev1api.Namespace {\n\tns := &corev1api.Namespace{\n\t\tObjectMeta: objectMeta(\"\", namespace),\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"Namespace\",\n\t\t\tAPIVersion: corev1api.SchemeGroupVersion.String(),\n\t\t},\n\t}\n\n\tns.Labels[\"pod-security.kubernetes.io/enforce\"] = podSecurityLevel\n\tns.Labels[\"pod-security.kubernetes.io/enforce-version\"] = podSecurityVersion\n\tns.Labels[\"pod-security.kubernetes.io/audit\"] = podSecurityLevel\n\tns.Labels[\"pod-security.kubernetes.io/audit-version\"] = podSecurityVersion\n\tns.Labels[\"pod-security.kubernetes.io/warn\"] = podSecurityLevel\n\tns.Labels[\"pod-security.kubernetes.io/warn-version\"] = podSecurityVersion\n\n\treturn ns\n}\n\nfunc BackupStorageLocation(namespace, provider, bucket, prefix string, config map[string]string, caCert []byte) *velerov1api.BackupStorageLocation {\n\treturn &velerov1api.BackupStorageLocation{\n\t\tObjectMeta: objectMeta(namespace, \"default\"),\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"BackupStorageLocation\",\n\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t},\n\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\tProvider: provider,\n\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\tBucket: bucket,\n\t\t\t\t\tPrefix: prefix,\n\t\t\t\t\tCACert: caCert,\n\t\t\t\t},\n\t\t\t},\n\t\t\tConfig: config,\n\t\t\tDefault: true,\n\t\t},\n\t}\n}\n\nfunc VolumeSnapshotLocation(namespace, provider string, config map[string]string) *velerov1api.VolumeSnapshotLocation {\n\treturn &velerov1api.VolumeSnapshotLocation{\n\t\tObjectMeta: objectMeta(namespace, \"default\"),\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"VolumeSnapshotLocation\",\n\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t},\n\t\tSpec: velerov1api.VolumeSnapshotLocationSpec{\n\t\t\tProvider: provider,\n\t\t\tConfig: config,\n\t\t},\n\t}\n}\n\nfunc Secret(namespace string, data []byte) *corev1api.Secret {\n\treturn &corev1api.Secret{\n\t\tObjectMeta: objectMeta(namespace, \"cloud-credentials\"),\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"Secret\",\n\t\t\tAPIVersion: corev1api.SchemeGroupVersion.String(),\n\t\t},\n\t\tData: map[string][]byte{\n\t\t\t\"cloud\": data,\n\t\t},\n\t\tType: corev1api.SecretTypeOpaque,\n\t}\n}\n\nfunc appendUnstructured(list *unstructured.UnstructuredList, obj runtime.Object) error {\n\tu, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&obj)\n\n\t// Remove the status field so we're not sending blank data to the server.\n\t// On CRDs, having an empty status is actually a validation error.\n\tdelete(u, \"status\")\n\tif err != nil {\n\t\treturn err\n\t}\n\tlist.Items = append(list.Items, unstructured.Unstructured{Object: u})\n\treturn nil\n}\n\ntype VeleroOptions struct {\n\tNamespace string\n\tImage string\n\tProviderName string\n\tBucket string\n\tPrefix string\n\tPodAnnotations map[string]string\n\tPodLabels map[string]string\n\tServiceAccountAnnotations map[string]string\n\tServiceAccountName string\n\tVeleroPodResources corev1api.ResourceRequirements\n\tNodeAgentPodResources corev1api.ResourceRequirements\n\tSecretData []byte\n\tRestoreOnly bool\n\tUseNodeAgent bool\n\tUseNodeAgentWindows bool\n\tPrivilegedNodeAgent bool\n\tUseVolumeSnapshots bool\n\tBSLConfig map[string]string\n\tVSLConfig map[string]string\n\tDefaultRepoMaintenanceFrequency time.Duration\n\tGarbageCollectionFrequency time.Duration\n\tPodVolumeOperationTimeout time.Duration\n\tPlugins []string\n\tNoDefaultBackupLocation bool\n\tCACertData []byte\n\tFeatures []string\n\tDefaultVolumesToFsBackup bool\n\tUploaderType string\n\tDefaultSnapshotMoveData bool\n\tDisableInformerCache bool\n\tScheduleSkipImmediately bool\n\tPodResources kube.PodResources\n\tKeepLatestMaintenanceJobs int\n\tBackupRepoConfigMap string\n\tRepoMaintenanceJobConfigMap string\n\tNodeAgentConfigMap string\n\tItemBlockWorkerCount int\n\tConcurrentBackups int\n\tKubeletRootDir string\n\tNodeAgentDisableHostPath bool\n\tServerPriorityClassName string\n\tNodeAgentPriorityClassName string\n}\n\nfunc AllCRDs() *unstructured.UnstructuredList {\n\tresources := new(unstructured.UnstructuredList)\n\t// Set the GVK so that the serialization framework outputs the list properly\n\tresources.SetGroupVersionKind(schema.GroupVersionKind{Group: \"\", Version: \"v1\", Kind: \"List\"})\n\n\tfor _, crd := range v1crds.CRDs {\n\t\tcrd.SetLabels(Labels())\n\t\tif err := appendUnstructured(resources, crd); err != nil {\n\t\t\tfmt.Printf(\"error appending v1 CRD %s: %s\\n\", crd.GetName(), err.Error())\n\t\t}\n\t}\n\n\tfor _, crd := range v2alpha1crds.CRDs {\n\t\tcrd.SetLabels(Labels())\n\t\tif err := appendUnstructured(resources, crd); err != nil {\n\t\t\tfmt.Printf(\"error appending v2alpha1 CRD %s: %s\\n\", crd.GetName(), err.Error())\n\t\t}\n\t}\n\n\treturn resources\n}\n\n// AllResources returns a list of all resources necessary to install Velero, in the appropriate order, into a Kubernetes cluster.\n// Items are unstructured, since there are different data types returned.\nfunc AllResources(o *VeleroOptions) *unstructured.UnstructuredList {\n\tresources := AllCRDs()\n\n\tns := Namespace(o.Namespace)\n\tif err := appendUnstructured(resources, ns); err != nil {\n\t\tfmt.Printf(\"error appending Namespace %s: %s\\n\", ns.GetName(), err.Error())\n\t}\n\n\tserviceAccountName := defaultServiceAccountName\n\tif o.ServiceAccountName == \"\" {\n\t\tcrb := ClusterRoleBinding(o.Namespace)\n\t\tif err := appendUnstructured(resources, crb); err != nil {\n\t\t\tfmt.Printf(\"error appending ClusterRoleBinding %s: %s\\n\", crb.GetName(), err.Error())\n\t\t}\n\t\tsa := ServiceAccount(o.Namespace, o.ServiceAccountAnnotations)\n\t\tif err := appendUnstructured(resources, sa); err != nil {\n\t\t\tfmt.Printf(\"error appending ServiceAccount %s: %s\\n\", sa.GetName(), err.Error())\n\t\t}\n\t} else {\n\t\tserviceAccountName = o.ServiceAccountName\n\t}\n\n\tif o.SecretData != nil {\n\t\tsec := Secret(o.Namespace, o.SecretData)\n\t\tif err := appendUnstructured(resources, sec); err != nil {\n\t\t\tfmt.Printf(\"error appending Secret %s: %s\\n\", sec.GetName(), err.Error())\n\t\t}\n\t}\n\n\tif !o.NoDefaultBackupLocation {\n\t\tbsl := BackupStorageLocation(o.Namespace, o.ProviderName, o.Bucket, o.Prefix, o.BSLConfig, o.CACertData)\n\t\tif err := appendUnstructured(resources, bsl); err != nil {\n\t\t\tfmt.Printf(\"error appending BackupStorageLocation %s: %s\\n\", bsl.GetName(), err.Error())\n\t\t}\n\t}\n\n\t// A snapshot location may not be desirable for users relying on pod volume backup/restore\n\tif o.UseVolumeSnapshots {\n\t\tvsl := VolumeSnapshotLocation(o.Namespace, o.ProviderName, o.VSLConfig)\n\t\tif err := appendUnstructured(resources, vsl); err != nil {\n\t\t\tfmt.Printf(\"error appending VolumeSnapshotLocation %s: %s\\n\", vsl.GetName(), err.Error())\n\t\t}\n\t}\n\n\tsecretPresent := o.SecretData != nil\n\n\tdeployOpts := []podTemplateOption{\n\t\tWithAnnotations(o.PodAnnotations),\n\t\tWithLabels(o.PodLabels),\n\t\tWithImage(o.Image),\n\t\tWithResources(o.VeleroPodResources),\n\t\tWithSecret(secretPresent),\n\t\tWithDefaultRepoMaintenanceFrequency(o.DefaultRepoMaintenanceFrequency),\n\t\tWithServiceAccountName(serviceAccountName),\n\t\tWithGarbageCollectionFrequency(o.GarbageCollectionFrequency),\n\t\tWithPodVolumeOperationTimeout(o.PodVolumeOperationTimeout),\n\t\tWithUploaderType(o.UploaderType),\n\t\tWithScheduleSkipImmediately(o.ScheduleSkipImmediately),\n\t\tWithPodResources(o.PodResources),\n\t\tWithKeepLatestMaintenanceJobs(o.KeepLatestMaintenanceJobs),\n\t\tWithItemBlockWorkerCount(o.ItemBlockWorkerCount),\n\t\tWithConcurrentBackups(o.ConcurrentBackups),\n\t}\n\n\tif o.ServerPriorityClassName != \"\" {\n\t\tdeployOpts = append(deployOpts, WithPriorityClassName(o.ServerPriorityClassName))\n\t}\n\n\tif len(o.Features) > 0 {\n\t\tdeployOpts = append(deployOpts, WithFeatures(o.Features))\n\t}\n\n\tif o.RestoreOnly {\n\t\tdeployOpts = append(deployOpts, WithRestoreOnly(true))\n\t}\n\n\tif len(o.Plugins) > 0 {\n\t\tdeployOpts = append(deployOpts, WithPlugins(o.Plugins))\n\t}\n\n\tif o.DefaultVolumesToFsBackup {\n\t\tdeployOpts = append(deployOpts, WithDefaultVolumesToFsBackup(true))\n\t}\n\n\tif o.DefaultSnapshotMoveData {\n\t\tdeployOpts = append(deployOpts, WithDefaultSnapshotMoveData(true))\n\t}\n\n\tif o.DisableInformerCache {\n\t\tdeployOpts = append(deployOpts, WithDisableInformerCache(true))\n\t}\n\n\tif len(o.BackupRepoConfigMap) > 0 {\n\t\tdeployOpts = append(deployOpts, WithBackupRepoConfigMap(o.BackupRepoConfigMap))\n\t}\n\n\tif len(o.RepoMaintenanceJobConfigMap) > 0 {\n\t\tdeployOpts = append(deployOpts, WithRepoMaintenanceJobConfigMap(o.RepoMaintenanceJobConfigMap))\n\t}\n\n\tdeploy := Deployment(o.Namespace, deployOpts...)\n\n\tif err := appendUnstructured(resources, deploy); err != nil {\n\t\tfmt.Printf(\"error appending Deployment %s: %s\\n\", deploy.GetName(), err.Error())\n\t}\n\n\tif o.UseNodeAgent || o.UseNodeAgentWindows {\n\t\tdsOpts := []podTemplateOption{\n\t\t\tWithAnnotations(o.PodAnnotations),\n\t\t\tWithLabels(o.PodLabels),\n\t\t\tWithImage(o.Image),\n\t\t\tWithResources(o.NodeAgentPodResources),\n\t\t\tWithSecret(secretPresent),\n\t\t\tWithServiceAccountName(serviceAccountName),\n\t\t\tWithNodeAgentDisableHostPath(o.NodeAgentDisableHostPath),\n\t\t}\n\t\tif len(o.Features) > 0 {\n\t\t\tdsOpts = append(dsOpts, WithFeatures(o.Features))\n\t\t}\n\t\tif o.PrivilegedNodeAgent {\n\t\t\tdsOpts = append(dsOpts, WithPrivilegedNodeAgent(true))\n\t\t}\n\t\tif len(o.NodeAgentConfigMap) > 0 {\n\t\t\tdsOpts = append(dsOpts, WithNodeAgentConfigMap(o.NodeAgentConfigMap))\n\t\t}\n\n\t\tif len(o.BackupRepoConfigMap) > 0 {\n\t\t\tdsOpts = append(dsOpts, WithBackupRepoConfigMap(o.BackupRepoConfigMap))\n\t\t}\n\n\t\tif len(o.KubeletRootDir) > 0 {\n\t\t\tdsOpts = append(dsOpts, WithKubeletRootDir(o.KubeletRootDir))\n\t\t}\n\n\t\tif o.NodeAgentPriorityClassName != \"\" {\n\t\t\tdsOpts = append(dsOpts, WithPriorityClassName(o.NodeAgentPriorityClassName))\n\t\t}\n\n\t\tif o.UseNodeAgent {\n\t\t\tds := DaemonSet(o.Namespace, dsOpts...)\n\t\t\tif err := appendUnstructured(resources, ds); err != nil {\n\t\t\t\tfmt.Printf(\"error appending DaemonSet %s: %s\\n\", ds.GetName(), err.Error())\n\t\t\t}\n\t\t}\n\n\t\tif o.UseNodeAgentWindows {\n\t\t\tdsOpts = append(dsOpts, WithForWindows())\n\n\t\t\tdsWin := DaemonSet(o.Namespace, dsOpts...)\n\t\t\tif err := appendUnstructured(resources, dsWin); err != nil {\n\t\t\t\tfmt.Printf(\"error appending DaemonSet %s: %s\\n\", dsWin.GetName(), err.Error())\n\t\t\t}\n\t\t}\n\t}\n\n\treturn resources\n}\n" }, { "path": "pkg/install/resources_test.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage install\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n)\n\nfunc TestResources(t *testing.T) {\n\tbsl := BackupStorageLocation(DefaultVeleroNamespace, \"test\", \"test\", \"\", make(map[string]string), []byte(\"test\"))\n\n\tassert.Equal(t, \"velero\", bsl.ObjectMeta.Namespace)\n\tassert.Equal(t, \"test\", bsl.Spec.Provider)\n\tassert.Equal(t, \"test\", bsl.Spec.StorageType.ObjectStorage.Bucket)\n\tassert.Equal(t, make(map[string]string), bsl.Spec.Config)\n\tassert.Equal(t, []byte(\"test\"), bsl.Spec.ObjectStorage.CACert)\n\n\tvsl := VolumeSnapshotLocation(DefaultVeleroNamespace, \"test\", make(map[string]string))\n\n\tassert.Equal(t, \"velero\", vsl.ObjectMeta.Namespace)\n\tassert.Equal(t, \"test\", vsl.Spec.Provider)\n\tassert.Equal(t, make(map[string]string), vsl.Spec.Config)\n\n\tns := Namespace(\"velero\")\n\n\tassert.Equal(t, \"velero\", ns.Name)\n\t// For k8s version v1.25 and later, need to add the following labels to make\n\t// velero installation namespace has privileged version to work with\n\t// PSA(Pod Security Admission) and PSS(Pod Security Standards).\n\tassert.Equal(t, \"privileged\", ns.Labels[\"pod-security.kubernetes.io/enforce\"])\n\tassert.Equal(t, \"latest\", ns.Labels[\"pod-security.kubernetes.io/enforce-version\"])\n\tassert.Equal(t, \"privileged\", ns.Labels[\"pod-security.kubernetes.io/audit\"])\n\tassert.Equal(t, \"latest\", ns.Labels[\"pod-security.kubernetes.io/audit-version\"])\n\tassert.Equal(t, \"privileged\", ns.Labels[\"pod-security.kubernetes.io/warn\"])\n\tassert.Equal(t, \"latest\", ns.Labels[\"pod-security.kubernetes.io/warn-version\"])\n\n\tcrb := ClusterRoleBinding(DefaultVeleroNamespace)\n\t// The CRB is a cluster-scoped resource\n\tassert.Empty(t, crb.ObjectMeta.Namespace)\n\tassert.Equal(t, \"velero\", crb.ObjectMeta.Name)\n\tassert.Equal(t, \"velero\", crb.Subjects[0].Namespace)\n\n\tcustomNamespaceCRB := ClusterRoleBinding(\"foo\")\n\t// The CRB is a cluster-scoped resource\n\tassert.Empty(t, customNamespaceCRB.ObjectMeta.Namespace)\n\tassert.Equal(t, \"velero-foo\", customNamespaceCRB.ObjectMeta.Name)\n\tassert.Equal(t, \"foo\", customNamespaceCRB.Subjects[0].Namespace)\n\n\tsa := ServiceAccount(DefaultVeleroNamespace, map[string]string{\"abcd\": \"cbd\"})\n\tassert.Equal(t, \"velero\", sa.ObjectMeta.Namespace)\n\tassert.Equal(t, \"cbd\", sa.ObjectMeta.Annotations[\"abcd\"])\n}\n\nfunc TestAllCRDs(t *testing.T) {\n\tlist := AllCRDs()\n\tassert.Len(t, list.Items, 13)\n\tassert.Equal(t, Labels(), list.Items[0].GetLabels())\n}\n\nfunc TestAllResources(t *testing.T) {\n\toption := &VeleroOptions{\n\t\tNamespace: \"velero\",\n\t\tSecretData: []byte{'a'},\n\t\tUseVolumeSnapshots: true,\n\t\tUseNodeAgent: true,\n\t\tUseNodeAgentWindows: true,\n\t}\n\tlist := AllResources(option)\n\n\tobjects := map[string][]unstructured.Unstructured{}\n\tfor _, item := range list.Items {\n\t\tobjects[item.GetKind()] = append(objects[item.GetKind()], item)\n\t}\n\n\tns, exist := objects[\"Namespace\"]\n\trequire.True(t, exist)\n\tassert.Equal(t, \"velero\", ns[0].GetName())\n\n\t_, exist = objects[\"ClusterRoleBinding\"]\n\tassert.True(t, exist)\n\n\t_, exist = objects[\"ServiceAccount\"]\n\tassert.True(t, exist)\n\n\t_, exist = objects[\"Secret\"]\n\tassert.True(t, exist)\n\n\t_, exist = objects[\"BackupStorageLocation\"]\n\tassert.True(t, exist)\n\n\t_, exist = objects[\"VolumeSnapshotLocation\"]\n\tassert.True(t, exist)\n\n\t_, exist = objects[\"Deployment\"]\n\tassert.True(t, exist)\n\n\tds, exist := objects[\"DaemonSet\"]\n\tassert.True(t, exist)\n\n\tassert.Len(t, ds, 2)\n}\n\nfunc TestAllResourcesWithPriorityClassName(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tserverPriorityClassName string\n\t\tnodeAgentPriorityClassName string\n\t\tuseNodeAgent bool\n\t}{\n\t\t{\n\t\t\tname: \"with same priority class for server and node agent\",\n\t\t\tserverPriorityClassName: \"high-priority\",\n\t\t\tnodeAgentPriorityClassName: \"high-priority\",\n\t\t\tuseNodeAgent: true,\n\t\t},\n\t\t{\n\t\t\tname: \"with different priority classes for server and node agent\",\n\t\t\tserverPriorityClassName: \"high-priority\",\n\t\t\tnodeAgentPriorityClassName: \"medium-priority\",\n\t\t\tuseNodeAgent: true,\n\t\t},\n\t\t{\n\t\t\tname: \"with only server priority class\",\n\t\t\tserverPriorityClassName: \"high-priority\",\n\t\t\tnodeAgentPriorityClassName: \"\",\n\t\t\tuseNodeAgent: true,\n\t\t},\n\t\t{\n\t\t\tname: \"with only node agent priority class\",\n\t\t\tserverPriorityClassName: \"\",\n\t\t\tnodeAgentPriorityClassName: \"medium-priority\",\n\t\t\tuseNodeAgent: true,\n\t\t},\n\t\t{\n\t\t\tname: \"with priority class name without node agent\",\n\t\t\tserverPriorityClassName: \"high-priority\",\n\t\t\tnodeAgentPriorityClassName: \"medium-priority\",\n\t\t\tuseNodeAgent: false,\n\t\t},\n\t\t{\n\t\t\tname: \"without priority class name with node agent\",\n\t\t\tserverPriorityClassName: \"\",\n\t\t\tnodeAgentPriorityClassName: \"\",\n\t\t\tuseNodeAgent: true,\n\t\t},\n\t\t{\n\t\t\tname: \"without priority class name without node agent\",\n\t\t\tserverPriorityClassName: \"\",\n\t\t\tnodeAgentPriorityClassName: \"\",\n\t\t\tuseNodeAgent: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t// Create VeleroOptions with the priority class names\n\t\t\toptions := &VeleroOptions{\n\t\t\t\tNamespace: \"velero\",\n\t\t\t\tUseNodeAgent: tc.useNodeAgent,\n\t\t\t\tServerPriorityClassName: tc.serverPriorityClassName,\n\t\t\t\tNodeAgentPriorityClassName: tc.nodeAgentPriorityClassName,\n\t\t\t}\n\n\t\t\t// Generate all resources\n\t\t\tresources := AllResources(options)\n\n\t\t\t// Find the deployment and verify priority class name\n\t\t\tdeploymentFound := false\n\t\t\tdaemonsetFound := false\n\n\t\t\tfor i := range resources.Items {\n\t\t\t\titem := resources.Items[i]\n\n\t\t\t\t// Check deployment\n\t\t\t\tif item.GetKind() == \"Deployment\" && item.GetName() == \"velero\" {\n\t\t\t\t\tdeploymentFound = true\n\n\t\t\t\t\t// Extract priority class name from the unstructured object\n\t\t\t\t\tpriorityClassName, found, err := unstructured.NestedString(\n\t\t\t\t\t\titem.Object,\n\t\t\t\t\t\t\"spec\", \"template\", \"spec\", \"priorityClassName\",\n\t\t\t\t\t)\n\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tif tc.serverPriorityClassName != \"\" {\n\t\t\t\t\t\tassert.True(t, found, \"Server priorityClassName should be set\")\n\t\t\t\t\t\tassert.Equal(t, tc.serverPriorityClassName, priorityClassName,\n\t\t\t\t\t\t\t\"Server deployment should have the correct priority class\")\n\t\t\t\t\t} else {\n\t\t\t\t\t\t// If no priority class name was provided, it might not be set at all\n\t\t\t\t\t\tif found {\n\t\t\t\t\t\t\tassert.Empty(t, priorityClassName)\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\t// Check daemonset if node agent is enabled\n\t\t\t\tif tc.useNodeAgent && item.GetKind() == \"DaemonSet\" && item.GetName() == \"node-agent\" {\n\t\t\t\t\tdaemonsetFound = true\n\n\t\t\t\t\t// Extract priority class name from the unstructured object\n\t\t\t\t\tpriorityClassName, found, err := unstructured.NestedString(\n\t\t\t\t\t\titem.Object,\n\t\t\t\t\t\t\"spec\", \"template\", \"spec\", \"priorityClassName\",\n\t\t\t\t\t)\n\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tif tc.nodeAgentPriorityClassName != \"\" {\n\t\t\t\t\t\tassert.True(t, found, \"Node agent priorityClassName should be set\")\n\t\t\t\t\t\tassert.Equal(t, tc.nodeAgentPriorityClassName, priorityClassName,\n\t\t\t\t\t\t\t\"Node agent daemonset should have the correct priority class\")\n\t\t\t\t\t} else {\n\t\t\t\t\t\t// If no priority class name was provided, it might not be set at all\n\t\t\t\t\t\tif found {\n\t\t\t\t\t\t\tassert.Empty(t, priorityClassName)\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\n\t\t\t// Verify we found the deployment\n\t\t\tassert.True(t, deploymentFound, \"Deployment should be present in resources\")\n\n\t\t\t// Verify we found the daemonset if node agent is enabled\n\t\t\tif tc.useNodeAgent {\n\t\t\t\tassert.True(t, daemonsetFound, \"DaemonSet should be present when UseNodeAgent is true\")\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/itemblock/actions/pod_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/actionhelpers\"\n)\n\n// PodAction implements ItemBlockAction.\ntype PodAction struct {\n\tlog logrus.FieldLogger\n}\n\n// NewPodAction creates a new ItemBlockAction for pods.\nfunc NewPodAction(logger logrus.FieldLogger) *PodAction {\n\treturn &PodAction{log: logger}\n}\n\n// AppliesTo returns a ResourceSelector that applies only to pods.\nfunc (a *PodAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"pods\"},\n\t}, nil\n}\n\n// GetRelatedItems scans the pod's spec.volumes for persistentVolumeClaim volumes and returns a\n// ResourceIdentifier list containing references to all of the persistentVolumeClaim volumes used by\n// the pod. This ensures that when a pod is backed up, all referenced PVCs are backed up along with the pod.\nfunc (a *PodAction) GetRelatedItems(item runtime.Unstructured, backup *v1.Backup) ([]velero.ResourceIdentifier, error) {\n\ta.log.Info(\"Executing pod ItemBlockAction\")\n\tdefer a.log.Info(\"Done executing pod ItemBlockAction\")\n\n\tpod := new(corev1api.Pod)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(item.UnstructuredContent(), pod); err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\treturn actionhelpers.RelatedItemsForPod(pod, a.log), nil\n}\n\nfunc (a *PodAction) Name() string {\n\treturn \"PodItemBlockAction\"\n}\n" }, { "path": "pkg/itemblock/actions/pod_action_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestPodActionAppliesTo(t *testing.T) {\n\ta := NewPodAction(velerotest.NewLogger())\n\n\tactual, err := a.AppliesTo()\n\trequire.NoError(t, err)\n\n\texpected := velero.ResourceSelector{\n\t\tIncludedResources: []string{\"pods\"},\n\t}\n\tassert.Equal(t, expected, actual)\n}\n\nfunc TestPodActionGetRelatedItems(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpod runtime.Unstructured\n\t\texpected []velero.ResourceIdentifier\n\t}{\n\t\t{\n\t\t\tname: \"no spec.volumes\",\n\t\t\tpod: velerotest.UnstructuredOrDie(`\n\t\t\t{\n\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\"kind\": \"Pod\",\n\t\t\t\t\"metadata\": {\n\t\t\t\t\t\"namespace\": \"foo\",\n\t\t\t\t\t\"name\": \"bar\"\n\t\t\t\t}\n\t\t\t}\n\t\t\t`),\n\t\t},\n\t\t{\n\t\t\tname: \"persistentVolumeClaim without claimName\",\n\t\t\tpod: velerotest.UnstructuredOrDie(`\n\t\t\t{\n\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\"kind\": \"Pod\",\n\t\t\t\t\"metadata\": {\n\t\t\t\t\t\"namespace\": \"foo\",\n\t\t\t\t\t\"name\": \"bar\"\n\t\t\t\t},\n\t\t\t\t\"spec\": {\n\t\t\t\t\t\"volumes\": [\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\"persistentVolumeClaim\": {}\n\t\t\t\t\t\t}\n\t\t\t\t\t]\n\t\t\t\t}\n\t\t\t}\n\t\t\t`),\n\t\t},\n\t\t{\n\t\t\tname: \"full test, mix of volume types\",\n\t\t\tpod: velerotest.UnstructuredOrDie(`\n\t\t\t{\n\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\"kind\": \"Pod\",\n\t\t\t\t\"metadata\": {\n\t\t\t\t\t\"namespace\": \"foo\",\n\t\t\t\t\t\"name\": \"bar\"\n\t\t\t\t},\n\t\t\t\t\"spec\": {\n\t\t\t\t\t\"volumes\": [\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\"persistentVolumeClaim\": {}\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\"emptyDir\": {}\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\"persistentVolumeClaim\": {\"claimName\": \"claim1\"}\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\"emptyDir\": {}\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\"persistentVolumeClaim\": {\"claimName\": \"claim2\"}\n\t\t\t\t\t\t}\n\t\t\t\t\t]\n\t\t\t\t}\n\t\t\t}\n\t\t\t`),\n\t\t\texpected: []velero.ResourceIdentifier{\n\t\t\t\t{GroupResource: kuberesource.PersistentVolumeClaims, Namespace: \"foo\", Name: \"claim1\"},\n\t\t\t\t{GroupResource: kuberesource.PersistentVolumeClaims, Namespace: \"foo\", Name: \"claim2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"test priority class\",\n\t\t\tpod: velerotest.UnstructuredOrDie(`\n\t\t\t{\n\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\"kind\": \"Pod\",\n\t\t\t\t\"metadata\": {\n\t\t\t\t\t\"namespace\": \"foo\",\n\t\t\t\t\t\"name\": \"bar\"\n\t\t\t\t},\n\t\t\t\t\"spec\": {\n\t\t\t\t\t\"priorityClassName\": \"testPriorityClass\"\n\t\t\t\t}\n\t\t\t}\n\t\t\t`),\n\t\t\texpected: []velero.ResourceIdentifier{\n\t\t\t\t{GroupResource: kuberesource.PriorityClasses, Name: \"testPriorityClass\"},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\ta := NewPodAction(velerotest.NewLogger())\n\n\t\t\trelatedItems, err := a.GetRelatedItems(test.pod, nil)\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Equal(t, test.expected, relatedItems)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/itemblock/actions/pvc_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\tcrclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\tplugincommon \"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/actionhelpers\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\n// PVCAction inspects a PersistentVolumeClaim for the PersistentVolume\n// that it references and backs it up\ntype PVCAction struct {\n\tlog logrus.FieldLogger\n\tcrClient crclient.Client\n\t// map[namespace]->[map[pvcVolumes]->[]podName]\n\tnsPVCs map[string]map[string][]string\n}\n\nfunc NewPVCAction(f client.Factory) plugincommon.HandlerInitializer {\n\treturn func(logger logrus.FieldLogger) (any, error) {\n\t\tcrClient, err := f.KubebuilderClient()\n\t\tif err != nil {\n\t\t\treturn nil, errors.WithStack(err)\n\t\t}\n\n\t\treturn &PVCAction{\n\t\t\tlog: logger,\n\t\t\tcrClient: crClient,\n\t\t}, nil\n\t}\n}\n\nfunc (a *PVCAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"persistentvolumeclaims\"},\n\t}, nil\n}\n\nfunc (a *PVCAction) GetRelatedItems(item runtime.Unstructured, backup *v1.Backup) ([]velero.ResourceIdentifier, error) {\n\ta.log.Info(\"Executing PVC ItemBlockAction\")\n\tdefer a.log.Info(\"Done executing PVC ItemBlockAction\")\n\n\tpvc := new(corev1api.PersistentVolumeClaim)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(item.UnstructuredContent(), &pvc); err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to convert unstructured item to persistent volume claim\")\n\t}\n\n\tif pvc.Status.Phase != corev1api.ClaimBound || pvc.Spec.VolumeName == \"\" {\n\t\treturn nil, nil\n\t}\n\t// returns the PV for the PVC (shared with BIA additionalItems)\n\trelatedItems := actionhelpers.RelatedItemsForPVC(pvc, a.log)\n\n\t// Adds pods mounting this PVC to ensure that multiple pods mounting the same RWX\n\t// volume get backed up together.\n\tpvcs, err := a.getPVCList(pvc.Namespace)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfor _, pod := range pvcs[pvc.Name] {\n\t\ta.log.Infof(\"Adding related Pod %s to PVC %s\", pod, pvc.Name)\n\t\trelatedItems = append(relatedItems, velero.ResourceIdentifier{\n\t\t\tGroupResource: kuberesource.Pods,\n\t\t\tNamespace: pvc.Namespace,\n\t\t\tName: pod,\n\t\t})\n\t}\n\n\t// Gather groupedPVCs based on VGS label provided in the backup\n\tgroupedPVCs, err := a.getGroupedPVCs(context.Background(), pvc, backup)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Add the groupedPVCs to relatedItems so that they processed in a single item block\n\trelatedItems = append(relatedItems, groupedPVCs...)\n\n\treturn relatedItems, nil\n}\n\nfunc (a *PVCAction) getPVCList(ns string) (map[string][]string, error) {\n\tif a.nsPVCs == nil {\n\t\ta.nsPVCs = make(map[string]map[string][]string)\n\t}\n\tpvcList, ok := a.nsPVCs[ns]\n\tif ok {\n\t\treturn pvcList, nil\n\t}\n\tpvcList = make(map[string][]string)\n\tpods := new(corev1api.PodList)\n\terr := a.crClient.List(context.Background(), pods, crclient.InNamespace(ns))\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to list pods\")\n\t}\n\tfor i := range pods.Items {\n\t\tif kube.IsPodRunning(&pods.Items[i]) != nil {\n\t\t\ta.log.Debugf(\"Pod %s is not running, not adding to Pod list for PVC IBA plugin\", pods.Items[i].Name)\n\t\t\tcontinue\n\t\t}\n\t\tfor _, volume := range pods.Items[i].Spec.Volumes {\n\t\t\tif volume.VolumeSource.PersistentVolumeClaim != nil {\n\t\t\t\tpvcList[volume.VolumeSource.PersistentVolumeClaim.ClaimName] = append(pvcList[volume.VolumeSource.PersistentVolumeClaim.ClaimName], pods.Items[i].Name)\n\t\t\t}\n\t\t}\n\t}\n\ta.nsPVCs[ns] = pvcList\n\treturn pvcList, nil\n}\n\nfunc (a *PVCAction) Name() string {\n\treturn \"PVCItemBlockAction\"\n}\n\n// getGroupedPVCs returns other PVCs in the same group based on the VGS label key in the Backup spec.\nfunc (a *PVCAction) getGroupedPVCs(ctx context.Context, pvc *corev1api.PersistentVolumeClaim, backup *v1.Backup) ([]velero.ResourceIdentifier, error) {\n\tvar related []velero.ResourceIdentifier\n\n\tvgsLabelKey := backup.Spec.VolumeGroupSnapshotLabelKey\n\tif vgsLabelKey == \"\" {\n\t\ta.log.Debug(\"No VolumeGroupSnapshotLabelKey provided in backup spec; skipping PVC grouping\")\n\t\treturn nil, nil\n\t}\n\n\tgroupID, ok := pvc.Labels[vgsLabelKey]\n\tif !ok || groupID == \"\" {\n\t\t// PVC does not belong to any VGS group or groupID has empty value\n\t\ta.log.Debug(\"PVC does not belong to any PVC group or group label value is empty; skipping PVC grouping\")\n\t\treturn nil, nil\n\t}\n\n\tpvcList := new(corev1api.PersistentVolumeClaimList)\n\tif err := a.crClient.List(\n\t\tctx,\n\t\tpvcList,\n\t\tcrclient.InNamespace(pvc.Namespace),\n\t\tcrclient.MatchingLabels{vgsLabelKey: groupID},\n\t); err != nil {\n\t\treturn nil, errors.Wrapf(err, \"failed to list PVCs for VGS grouping with label %s=%s in namespace %s\", vgsLabelKey, groupID, pvc.Namespace)\n\t}\n\n\tif len(pvcList.Items) <= 1 {\n\t\t// Only the current PVC exists in this group\n\t\treturn nil, nil\n\t}\n\n\tfor _, groupPVC := range pvcList.Items {\n\t\tif groupPVC.Name == pvc.Name {\n\t\t\tcontinue\n\t\t}\n\n\t\ta.log.Infof(\"Adding grouped PVC %s (group %s) to relatedItems for PVC %s\", groupPVC.Name, groupID, pvc.Name)\n\n\t\trelated = append(related, velero.ResourceIdentifier{\n\t\t\tGroupResource: kuberesource.PersistentVolumeClaims,\n\t\t\tNamespace: groupPVC.Namespace,\n\t\t\tName: groupPVC.Name,\n\t\t})\n\t}\n\n\treturn related, nil\n}\n" }, { "path": "pkg/itemblock/actions/pvc_action_test.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"fmt\"\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestBackupPVAction(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\tpods []*corev1api.Pod\n\t\texpectedErr error\n\t\texpectedRelated []velero.ResourceIdentifier\n\t}{\n\t\t{\n\t\t\tname: \"Test no volumeName\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").Phase(corev1api.ClaimBound).Result(),\n\t\t\texpectedErr: nil,\n\t\t\texpectedRelated: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"Test empty volumeName\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").VolumeName(\"\").Phase(corev1api.ClaimBound).Result(),\n\t\t\texpectedErr: nil,\n\t\t\texpectedRelated: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"Test no status phase\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").VolumeName(\"testPV\").Result(),\n\t\t\texpectedErr: nil,\n\t\t\texpectedRelated: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"Test pending status phase\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").VolumeName(\"testPV\").Phase(corev1api.ClaimPending).Result(),\n\t\t\texpectedErr: nil,\n\t\t\texpectedRelated: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"Test lost status phase\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").VolumeName(\"testPV\").Phase(corev1api.ClaimLost).Result(),\n\t\t\texpectedErr: nil,\n\t\t\texpectedRelated: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"Test with volume\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").VolumeName(\"testPV\").Phase(corev1api.ClaimBound).Result(),\n\t\t\texpectedErr: nil,\n\t\t\texpectedRelated: []velero.ResourceIdentifier{\n\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"testPV\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Test with volume and one running pod\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").VolumeName(\"testPV\").Phase(corev1api.ClaimBound).Result(),\n\t\t\tpods: []*corev1api.Pod{\n\t\t\t\tbuilder.ForPod(\"velero\", \"testPod1\").Volumes(builder.ForVolume(\"testPVC\").PersistentVolumeClaimSource(\"testPVC\").Result()).NodeName(\"velero\").Phase(corev1api.PodRunning).Result(),\n\t\t\t},\n\t\t\texpectedErr: nil,\n\t\t\texpectedRelated: []velero.ResourceIdentifier{\n\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"testPV\"},\n\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"velero\", Name: \"testPod1\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Test with volume and multiple running pods\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").VolumeName(\"testPV\").Phase(corev1api.ClaimBound).Result(),\n\t\t\tpods: []*corev1api.Pod{\n\t\t\t\tbuilder.ForPod(\"velero\", \"testPod1\").Volumes(builder.ForVolume(\"testPVC\").PersistentVolumeClaimSource(\"testPVC\").Result()).NodeName(\"velero\").Phase(corev1api.PodRunning).Result(),\n\t\t\t\tbuilder.ForPod(\"velero\", \"testPod2\").Volumes(builder.ForVolume(\"testPVC\").PersistentVolumeClaimSource(\"testPVC\").Result()).NodeName(\"velero\").Phase(corev1api.PodRunning).Result(),\n\t\t\t\tbuilder.ForPod(\"velero\", \"testPod3\").Volumes(builder.ForVolume(\"testPVC\").PersistentVolumeClaimSource(\"testPVC\").Result()).NodeName(\"velero\").Phase(corev1api.PodRunning).Result(),\n\t\t\t},\n\t\t\texpectedErr: nil,\n\t\t\texpectedRelated: []velero.ResourceIdentifier{\n\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"testPV\"},\n\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"velero\", Name: \"testPod1\"},\n\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"velero\", Name: \"testPod2\"},\n\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"velero\", Name: \"testPod3\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Test with volume and multiple running pods, some not running\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").VolumeName(\"testPV\").Phase(corev1api.ClaimBound).Result(),\n\t\t\tpods: []*corev1api.Pod{\n\t\t\t\tbuilder.ForPod(\"velero\", \"testPod1\").Volumes(builder.ForVolume(\"testPVC\").PersistentVolumeClaimSource(\"testPVC\").Result()).NodeName(\"velero\").Phase(corev1api.PodSucceeded).Result(),\n\t\t\t\tbuilder.ForPod(\"velero\", \"testPod2\").Volumes(builder.ForVolume(\"testPVC\").PersistentVolumeClaimSource(\"testPVC\").Result()).NodeName(\"velero\").Phase(corev1api.PodRunning).Result(),\n\t\t\t\tbuilder.ForPod(\"velero\", \"testPod3\").Volumes(builder.ForVolume(\"testPVC\").PersistentVolumeClaimSource(\"testPVC\").Result()).Phase(corev1api.PodRunning).Result(),\n\t\t\t},\n\t\t\texpectedErr: nil,\n\t\t\texpectedRelated: []velero.ResourceIdentifier{\n\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"testPV\"},\n\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"velero\", Name: \"testPod2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Test with PVC grouping via VGS label\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC-1\").ObjectMeta(builder.WithLabels(\"velero.io/group\", \"db\")).VolumeName(\"testPV-1\").Phase(corev1api.ClaimBound).Result(),\n\t\t\tpods: []*corev1api.Pod{\n\t\t\t\tbuilder.ForPod(\"velero\", \"testPod-1\").\n\t\t\t\t\tVolumes(builder.ForVolume(\"testPV-1\").PersistentVolumeClaimSource(\"testPVC-1\").Result()).\n\t\t\t\t\tNodeName(\"node\").\n\t\t\t\t\tPhase(corev1api.PodRunning).Result(),\n\t\t\t},\n\t\t\texpectedErr: nil,\n\t\t\texpectedRelated: []velero.ResourceIdentifier{\n\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"testPV-1\"},\n\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"velero\", Name: \"testPod-1\"},\n\t\t\t\t{GroupResource: kuberesource.PersistentVolumeClaims, Namespace: \"velero\", Name: \"groupedPVC\"},\n\t\t\t},\n\t\t},\n\t}\n\n\tbackup := &v1.Backup{}\n\tlogger := logrus.New()\n\n\tf := &factorymocks.Factory{}\n\tf.On(\"KubebuilderClient\").Return(nil, fmt.Errorf(\"\"))\n\tplugin := NewPVCAction(f)\n\t_, err := plugin(logger)\n\trequire.Error(t, err)\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(*testing.T) {\n\t\t\tcrClient := velerotest.NewFakeControllerRuntimeClient(t)\n\t\t\tf := &factorymocks.Factory{}\n\t\t\tf.On(\"KubebuilderClient\").Return(crClient, nil)\n\t\t\tplugin := NewPVCAction(f)\n\t\t\ti, err := plugin(logger)\n\t\t\trequire.NoError(t, err)\n\t\t\ta := i.(*PVCAction)\n\n\t\t\tif tc.pvc != nil {\n\t\t\t\trequire.NoError(t, crClient.Create(t.Context(), tc.pvc))\n\t\t\t}\n\t\t\tfor _, pod := range tc.pods {\n\t\t\t\trequire.NoError(t, crClient.Create(t.Context(), pod))\n\t\t\t}\n\n\t\t\tif tc.name == \"Test with PVC grouping via VGS label\" {\n\t\t\t\tgroupedPVC := builder.ForPersistentVolumeClaim(\"velero\", \"groupedPVC\").ObjectMeta(builder.WithLabels(\"velero.io/group\", \"db\")).VolumeName(\"groupedPV\").Phase(corev1api.ClaimBound).Result()\n\t\t\t\trequire.NoError(t, crClient.Create(t.Context(), groupedPVC))\n\t\t\t\tbackup.Spec.VolumeGroupSnapshotLabelKey = \"velero.io/group\"\n\t\t\t}\n\n\t\t\tpvcMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&tc.pvc)\n\t\t\trequire.NoError(t, err)\n\n\t\t\trelatedItems, err := a.GetRelatedItems(&unstructured.Unstructured{Object: pvcMap}, backup)\n\t\t\tif tc.expectedErr != nil {\n\t\t\t\trequire.EqualError(t, err, tc.expectedErr.Error())\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\t\t\tassert.Equal(t, tc.expectedRelated, relatedItems)\n\t\t})\n\t}\n}\n\n// Test_getGroupedPVCs verifies the PVC grouping logic for VolumeGroupSnapshots.\n// This ensures only same-namespace PVCs with the same label key and value are included.\nfunc Test_getGroupedPVCs(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tlabelKey string\n\t\tgroupValue string\n\t\texistingPVCs []*corev1api.PersistentVolumeClaim\n\t\ttargetPVC *corev1api.PersistentVolumeClaim\n\t\texpectedRelated []velero.ResourceIdentifier\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"No label key in spec\",\n\t\t\tlabelKey: \"\",\n\t\t\ttargetPVC: builder.ForPersistentVolumeClaim(\"ns\", \"pvc-1\").Result(),\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"No group value\",\n\t\t\tlabelKey: \"velero.io/group\",\n\t\t\tgroupValue: \"\",\n\t\t\ttargetPVC: builder.ForPersistentVolumeClaim(\"ns\", \"pvc-1\").Result(),\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Target PVC does not have the label\",\n\t\t\tlabelKey: \"velero.io/group\",\n\t\t\ttargetPVC: builder.ForPersistentVolumeClaim(\"ns\", \"pvc-1\").Result(),\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Target PVC has label, but no group matches\",\n\t\t\tlabelKey: \"velero.io/group\",\n\t\t\tgroupValue: \"group-1\",\n\t\t\ttargetPVC: builder.ForPersistentVolumeClaim(\"ns\", \"pvc-1\").ObjectMeta(builder.WithLabels(\"velero.io/group\", \"group-1\")).Result(),\n\t\t\texistingPVCs: []*corev1api.PersistentVolumeClaim{\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns\", \"pvc-1\").ObjectMeta(builder.WithLabels(\"velero.io/group\", \"group-1\")).Result(),\n\t\t\t},\n\t\t\texpectError: false,\n\t\t\texpectedRelated: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"Multiple PVCs in the same group\",\n\t\t\tlabelKey: \"velero.io/group\",\n\t\t\tgroupValue: \"group-1\",\n\t\t\ttargetPVC: builder.ForPersistentVolumeClaim(\"ns\", \"pvc-1\").ObjectMeta(builder.WithLabels(\"velero.io/group\", \"group-1\")).Result(),\n\t\t\texistingPVCs: []*corev1api.PersistentVolumeClaim{\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns\", \"pvc-1\").ObjectMeta(builder.WithLabels(\"velero.io/group\", \"group-1\")).Result(),\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns\", \"pvc-2\").ObjectMeta(builder.WithLabels(\"velero.io/group\", \"group-1\")).Result(),\n\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns\", \"pvc-3\").ObjectMeta(builder.WithLabels(\"velero.io/group\", \"group-1\")).Result(),\n\t\t\t},\n\t\t\texpectError: false,\n\t\t\texpectedRelated: []velero.ResourceIdentifier{\n\t\t\t\t{GroupResource: kuberesource.PersistentVolumeClaims, Namespace: \"ns\", Name: \"pvc-2\"},\n\t\t\t\t{GroupResource: kuberesource.PersistentVolumeClaims, Namespace: \"ns\", Name: \"pvc-3\"},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tcrClient := velerotest.NewFakeControllerRuntimeClient(t)\n\t\t\tfor _, pvc := range tc.existingPVCs {\n\t\t\t\trequire.NoError(t, crClient.Create(t.Context(), pvc))\n\t\t\t}\n\n\t\t\tlogger := logrus.New()\n\t\t\ta := &PVCAction{\n\t\t\t\tlog: logger,\n\t\t\t\tcrClient: crClient,\n\t\t\t}\n\n\t\t\tbackup := builder.ForBackup(\"ns\", \"bkp\").VolumeGroupSnapshotLabelKey(tc.labelKey).Result()\n\n\t\t\trelated, err := a.getGroupedPVCs(t.Context(), tc.targetPVC, backup)\n\t\t\tif tc.expectError {\n\t\t\t\trequire.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tassert.ElementsMatch(t, tc.expectedRelated, related)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/itemblock/actions/service_account_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/api/meta\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerodiscovery \"github.com/vmware-tanzu/velero/pkg/discovery\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/actionhelpers\"\n)\n\n// ServiceAccountAction implements ItemBlockAction.\ntype ServiceAccountAction struct {\n\tlog logrus.FieldLogger\n\tclusterRoleBindings []actionhelpers.ClusterRoleBinding\n}\n\n// NewServiceAccountAction creates a new ItemBlockAction for service accounts.\nfunc NewServiceAccountAction(logger logrus.FieldLogger, clusterRoleBindingListers map[string]actionhelpers.ClusterRoleBindingLister, discoveryHelper velerodiscovery.Helper) (*ServiceAccountAction, error) {\n\tcrbs, err := actionhelpers.ClusterRoleBindingsForAction(clusterRoleBindingListers, discoveryHelper)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &ServiceAccountAction{\n\t\tlog: logger,\n\t\tclusterRoleBindings: crbs,\n\t}, nil\n}\n\n// AppliesTo returns a ResourceSelector that applies only to service accounts.\nfunc (a *ServiceAccountAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"serviceaccounts\"},\n\t}, nil\n}\n\n// GetRelatedItems checks for any ClusterRoleBindings that have this service account as a subject, and\n// returns the ClusterRoleBinding and associated ClusterRole.\nfunc (a *ServiceAccountAction) GetRelatedItems(item runtime.Unstructured, backup *v1.Backup) ([]velero.ResourceIdentifier, error) {\n\ta.log.Info(\"Running ServiceAccount ItemBlockAction\")\n\tdefer a.log.Info(\"Done running ServiceAccount ItemBlockAction\")\n\n\tobjectMeta, err := meta.Accessor(item)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\treturn actionhelpers.RelatedItemsForServiceAccount(objectMeta, a.clusterRoleBindings, a.log), nil\n}\n\nfunc (a *ServiceAccountAction) Name() string {\n\treturn \"ServiceAccountItemBlockAction\"\n}\n" }, { "path": "pkg/itemblock/actions/service_account_action_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"fmt\"\n\t\"sort\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\trbacv1 \"k8s.io/api/rbac/v1\"\n\trbacbeta \"k8s.io/api/rbac/v1beta1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/actionhelpers\"\n)\n\nfunc newV1ClusterRoleBindingList(rbacCRBList []rbacv1.ClusterRoleBinding) []actionhelpers.ClusterRoleBinding {\n\tvar crbs []actionhelpers.ClusterRoleBinding\n\tfor _, c := range rbacCRBList {\n\t\tcrbs = append(crbs, actionhelpers.V1ClusterRoleBinding{Crb: c})\n\t}\n\n\treturn crbs\n}\n\nfunc newV1beta1ClusterRoleBindingList(rbacCRBList []rbacbeta.ClusterRoleBinding) []actionhelpers.ClusterRoleBinding {\n\tvar crbs []actionhelpers.ClusterRoleBinding\n\tfor _, c := range rbacCRBList {\n\t\tcrbs = append(crbs, actionhelpers.V1beta1ClusterRoleBinding{Crb: c})\n\t}\n\n\treturn crbs\n}\n\ntype FakeV1ClusterRoleBindingLister struct {\n\tv1crbs []rbacv1.ClusterRoleBinding\n}\n\nfunc (f FakeV1ClusterRoleBindingLister) List() ([]actionhelpers.ClusterRoleBinding, error) {\n\tvar crbs []actionhelpers.ClusterRoleBinding\n\tfor _, c := range f.v1crbs {\n\t\tcrbs = append(crbs, actionhelpers.V1ClusterRoleBinding{Crb: c})\n\t}\n\treturn crbs, nil\n}\n\ntype FakeV1beta1ClusterRoleBindingLister struct {\n\tv1beta1crbs []rbacbeta.ClusterRoleBinding\n}\n\nfunc (f FakeV1beta1ClusterRoleBindingLister) List() ([]actionhelpers.ClusterRoleBinding, error) {\n\tvar crbs []actionhelpers.ClusterRoleBinding\n\tfor _, c := range f.v1beta1crbs {\n\t\tcrbs = append(crbs, actionhelpers.V1beta1ClusterRoleBinding{Crb: c})\n\t}\n\treturn crbs, nil\n}\n\nfunc TestServiceAccountActionAppliesTo(t *testing.T) {\n\t// Instantiating the struct directly since using\n\t// NewServiceAccountAction requires a full Kubernetes clientset\n\ta := &ServiceAccountAction{}\n\n\tactual, err := a.AppliesTo()\n\trequire.NoError(t, err)\n\n\texpected := velero.ResourceSelector{\n\t\tIncludedResources: []string{\"serviceaccounts\"},\n\t}\n\tassert.Equal(t, expected, actual)\n}\n\nfunc TestNewServiceAccountAction(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tversion string\n\t\texpectedCRBs []actionhelpers.ClusterRoleBinding\n\t}{\n\t\t{\n\t\t\tname: \"rbac v1 API instantiates an saAction\",\n\t\t\tversion: rbacv1.SchemeGroupVersion.Version,\n\t\t\texpectedCRBs: []actionhelpers.ClusterRoleBinding{\n\t\t\t\tactionhelpers.V1ClusterRoleBinding{\n\t\t\t\t\tCrb: rbacv1.ClusterRoleBinding{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"v1crb-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tactionhelpers.V1ClusterRoleBinding{\n\t\t\t\t\tCrb: rbacv1.ClusterRoleBinding{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"v1crb-2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"rbac v1beta1 API instantiates an saAction\",\n\t\t\tversion: rbacbeta.SchemeGroupVersion.Version,\n\t\t\texpectedCRBs: []actionhelpers.ClusterRoleBinding{\n\t\t\t\tactionhelpers.V1beta1ClusterRoleBinding{\n\t\t\t\t\tCrb: rbacbeta.ClusterRoleBinding{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"v1beta1crb-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tactionhelpers.V1beta1ClusterRoleBinding{\n\t\t\t\t\tCrb: rbacbeta.ClusterRoleBinding{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"v1beta1crb-2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"no RBAC API instantiates an saAction with empty slice\",\n\t\t\tversion: \"\",\n\t\t\texpectedCRBs: []actionhelpers.ClusterRoleBinding{},\n\t\t},\n\t}\n\t// Set up all of our fakes outside the test loop\n\tdiscoveryHelper := velerotest.FakeDiscoveryHelper{}\n\tlogger := velerotest.NewLogger()\n\n\tv1crbs := []rbacv1.ClusterRoleBinding{\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"v1crb-1\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"v1crb-2\",\n\t\t\t},\n\t\t},\n\t}\n\n\tv1beta1crbs := []rbacbeta.ClusterRoleBinding{\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"v1beta1crb-1\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"v1beta1crb-2\",\n\t\t\t},\n\t\t},\n\t}\n\n\tclusterRoleBindingListers := map[string]actionhelpers.ClusterRoleBindingLister{\n\t\trbacv1.SchemeGroupVersion.Version: FakeV1ClusterRoleBindingLister{v1crbs: v1crbs},\n\t\trbacbeta.SchemeGroupVersion.Version: FakeV1beta1ClusterRoleBindingLister{v1beta1crbs: v1beta1crbs},\n\t\t\"\": actionhelpers.NoopClusterRoleBindingLister{},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\t// We only care about the preferred version, nothing else in the list\n\t\t\tdiscoveryHelper.APIGroupsList = []metav1.APIGroup{\n\t\t\t\t{\n\t\t\t\t\tName: rbacv1.GroupName,\n\t\t\t\t\tPreferredVersion: metav1.GroupVersionForDiscovery{\n\t\t\t\t\t\tVersion: test.version,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t}\n\t\t\taction, err := NewServiceAccountAction(logger, clusterRoleBindingListers, &discoveryHelper)\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Equal(t, test.expectedCRBs, action.clusterRoleBindings)\n\t\t})\n\t}\n}\n\nfunc TestServiceAccountActionExecute(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tserviceAccount runtime.Unstructured\n\t\tcrbs []rbacv1.ClusterRoleBinding\n\t\texpectedAdditionalItems []velero.ResourceIdentifier\n\t}{\n\t\t{\n\t\t\tname: \"no crbs\",\n\t\t\tserviceAccount: velerotest.UnstructuredOrDie(`\n\t\t\t{\n\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\"metadata\": {\n\t\t\t\t\t\"namespace\": \"velero\",\n\t\t\t\t\t\"name\": \"velero\"\n\t\t\t\t}\n\t\t\t}\n\t\t\t`),\n\t\t\tcrbs: nil,\n\t\t\texpectedAdditionalItems: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"no matching crbs\",\n\t\t\tserviceAccount: velerotest.UnstructuredOrDie(`\n\t\t\t{\n\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\"metadata\": {\n\t\t\t\t\t\"namespace\": \"velero\",\n\t\t\t\t\t\"name\": \"velero\"\n\t\t\t\t}\n\t\t\t}\n\t\t\t`),\n\t\t\tcrbs: []rbacv1.ClusterRoleBinding{\n\t\t\t\t{\n\t\t\t\t\tSubjects: []rbacv1.Subject{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: \"non-matching-kind\",\n\t\t\t\t\t\t\tNamespace: \"non-matching-ns\",\n\t\t\t\t\t\t\tName: \"non-matching-name\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: \"non-matching-kind\",\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: rbacv1.ServiceAccountKind,\n\t\t\t\t\t\t\tNamespace: \"non-matching-ns\",\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: rbacv1.ServiceAccountKind,\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"non-matching-name\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tRoleRef: rbacv1.RoleRef{\n\t\t\t\t\t\tName: \"role\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedAdditionalItems: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"some matching crbs\",\n\t\t\tserviceAccount: velerotest.UnstructuredOrDie(`\n\t\t\t{\n\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\"metadata\": {\n\t\t\t\t\t\"namespace\": \"velero\",\n\t\t\t\t\t\"name\": \"velero\"\n\t\t\t\t}\n\t\t\t}\n\t\t\t`),\n\t\t\tcrbs: []rbacv1.ClusterRoleBinding{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"crb-1\",\n\t\t\t\t\t},\n\t\t\t\t\tSubjects: []rbacv1.Subject{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: \"non-matching-kind\",\n\t\t\t\t\t\t\tNamespace: \"non-matching-ns\",\n\t\t\t\t\t\t\tName: \"non-matching-name\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tRoleRef: rbacv1.RoleRef{\n\t\t\t\t\t\tName: \"role-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"crb-2\",\n\t\t\t\t\t},\n\t\t\t\t\tSubjects: []rbacv1.Subject{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: \"non-matching-kind\",\n\t\t\t\t\t\t\tNamespace: \"non-matching-ns\",\n\t\t\t\t\t\t\tName: \"non-matching-name\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: rbacv1.ServiceAccountKind,\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tRoleRef: rbacv1.RoleRef{\n\t\t\t\t\t\tName: \"role-2\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"crb-3\",\n\t\t\t\t\t},\n\t\t\t\t\tSubjects: []rbacv1.Subject{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: rbacv1.ServiceAccountKind,\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tRoleRef: rbacv1.RoleRef{\n\t\t\t\t\t\tName: \"role-3\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"crb-4\",\n\t\t\t\t\t},\n\t\t\t\t\tSubjects: []rbacv1.Subject{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: rbacv1.ServiceAccountKind,\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: \"non-matching-kind\",\n\t\t\t\t\t\t\tNamespace: \"non-matching-ns\",\n\t\t\t\t\t\t\tName: \"non-matching-name\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tRoleRef: rbacv1.RoleRef{\n\t\t\t\t\t\tName: \"role-4\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedAdditionalItems: []velero.ResourceIdentifier{\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoleBindings,\n\t\t\t\t\tName: \"crb-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoleBindings,\n\t\t\t\t\tName: \"crb-3\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoleBindings,\n\t\t\t\t\tName: \"crb-4\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoles,\n\t\t\t\t\tName: \"role-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoles,\n\t\t\t\t\tName: \"role-3\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoles,\n\t\t\t\t\tName: \"role-4\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\t// Create the action struct directly so we don't need to mock a clientset\n\t\t\taction := &ServiceAccountAction{\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t\tclusterRoleBindings: newV1ClusterRoleBindingList(test.crbs),\n\t\t\t}\n\n\t\t\tadditional, err := action.GetRelatedItems(test.serviceAccount, nil)\n\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// ensure slices are ordered for valid comparison\n\t\t\tsort.Slice(test.expectedAdditionalItems, func(i, j int) bool {\n\t\t\t\treturn fmt.Sprintf(\"%s.%s\", test.expectedAdditionalItems[i].GroupResource.String(), test.expectedAdditionalItems[i].Name) <\n\t\t\t\t\tfmt.Sprintf(\"%s.%s\", test.expectedAdditionalItems[j].GroupResource.String(), test.expectedAdditionalItems[j].Name)\n\t\t\t})\n\n\t\t\tsort.Slice(additional, func(i, j int) bool {\n\t\t\t\treturn fmt.Sprintf(\"%s.%s\", additional[i].GroupResource.String(), additional[i].Name) <\n\t\t\t\t\tfmt.Sprintf(\"%s.%s\", additional[j].GroupResource.String(), additional[j].Name)\n\t\t\t})\n\n\t\t\tassert.Equal(t, test.expectedAdditionalItems, additional)\n\t\t})\n\t}\n}\n\nfunc TestServiceAccountActionExecuteOnBeta1(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tserviceAccount runtime.Unstructured\n\t\tcrbs []rbacbeta.ClusterRoleBinding\n\t\texpectedAdditionalItems []velero.ResourceIdentifier\n\t}{\n\t\t{\n\t\t\tname: \"no crbs\",\n\t\t\tserviceAccount: velerotest.UnstructuredOrDie(`\n\t\t\t{\n\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\"metadata\": {\n\t\t\t\t\t\"namespace\": \"velero\",\n\t\t\t\t\t\"name\": \"velero\"\n\t\t\t\t}\n\t\t\t}\n\t\t\t`),\n\t\t\tcrbs: nil,\n\t\t\texpectedAdditionalItems: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"no matching crbs\",\n\t\t\tserviceAccount: velerotest.UnstructuredOrDie(`\n\t\t\t{\n\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\"metadata\": {\n\t\t\t\t\t\"namespace\": \"velero\",\n\t\t\t\t\t\"name\": \"velero\"\n\t\t\t\t}\n\t\t\t}\n\t\t\t`),\n\t\t\tcrbs: []rbacbeta.ClusterRoleBinding{\n\t\t\t\t{\n\t\t\t\t\tSubjects: []rbacbeta.Subject{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: \"non-matching-kind\",\n\t\t\t\t\t\t\tNamespace: \"non-matching-ns\",\n\t\t\t\t\t\t\tName: \"non-matching-name\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: \"non-matching-kind\",\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: rbacbeta.ServiceAccountKind,\n\t\t\t\t\t\t\tNamespace: \"non-matching-ns\",\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: rbacbeta.ServiceAccountKind,\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"non-matching-name\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tRoleRef: rbacbeta.RoleRef{\n\t\t\t\t\t\tName: \"role\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedAdditionalItems: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"some matching crbs\",\n\t\t\tserviceAccount: velerotest.UnstructuredOrDie(`\n\t\t\t{\n\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\"metadata\": {\n\t\t\t\t\t\"namespace\": \"velero\",\n\t\t\t\t\t\"name\": \"velero\"\n\t\t\t\t}\n\t\t\t}\n\t\t\t`),\n\t\t\tcrbs: []rbacbeta.ClusterRoleBinding{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"crb-1\",\n\t\t\t\t\t},\n\t\t\t\t\tSubjects: []rbacbeta.Subject{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: \"non-matching-kind\",\n\t\t\t\t\t\t\tNamespace: \"non-matching-ns\",\n\t\t\t\t\t\t\tName: \"non-matching-name\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tRoleRef: rbacbeta.RoleRef{\n\t\t\t\t\t\tName: \"role-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"crb-2\",\n\t\t\t\t\t},\n\t\t\t\t\tSubjects: []rbacbeta.Subject{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: \"non-matching-kind\",\n\t\t\t\t\t\t\tNamespace: \"non-matching-ns\",\n\t\t\t\t\t\t\tName: \"non-matching-name\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: rbacbeta.ServiceAccountKind,\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tRoleRef: rbacbeta.RoleRef{\n\t\t\t\t\t\tName: \"role-2\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"crb-3\",\n\t\t\t\t\t},\n\t\t\t\t\tSubjects: []rbacbeta.Subject{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: rbacbeta.ServiceAccountKind,\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tRoleRef: rbacbeta.RoleRef{\n\t\t\t\t\t\tName: \"role-3\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"crb-4\",\n\t\t\t\t\t},\n\t\t\t\t\tSubjects: []rbacbeta.Subject{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: rbacbeta.ServiceAccountKind,\n\t\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKind: \"non-matching-kind\",\n\t\t\t\t\t\t\tNamespace: \"non-matching-ns\",\n\t\t\t\t\t\t\tName: \"non-matching-name\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tRoleRef: rbacbeta.RoleRef{\n\t\t\t\t\t\tName: \"role-4\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedAdditionalItems: []velero.ResourceIdentifier{\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoleBindings,\n\t\t\t\t\tName: \"crb-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoleBindings,\n\t\t\t\t\tName: \"crb-3\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoleBindings,\n\t\t\t\t\tName: \"crb-4\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoles,\n\t\t\t\t\tName: \"role-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoles,\n\t\t\t\t\tName: \"role-3\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.ClusterRoles,\n\t\t\t\t\tName: \"role-4\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\t// Create the action struct directly so we don't need to mock a clientset\n\t\t\taction := &ServiceAccountAction{\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t\tclusterRoleBindings: newV1beta1ClusterRoleBindingList(test.crbs),\n\t\t\t}\n\n\t\t\tadditional, err := action.GetRelatedItems(test.serviceAccount, nil)\n\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// ensure slices are ordered for valid comparison\n\t\t\tsort.Slice(test.expectedAdditionalItems, func(i, j int) bool {\n\t\t\t\treturn fmt.Sprintf(\"%s.%s\", test.expectedAdditionalItems[i].GroupResource.String(), test.expectedAdditionalItems[i].Name) <\n\t\t\t\t\tfmt.Sprintf(\"%s.%s\", test.expectedAdditionalItems[j].GroupResource.String(), test.expectedAdditionalItems[j].Name)\n\t\t\t})\n\n\t\t\tsort.Slice(additional, func(i, j int) bool {\n\t\t\t\treturn fmt.Sprintf(\"%s.%s\", additional[i].GroupResource.String(), additional[i].Name) <\n\t\t\t\t\tfmt.Sprintf(\"%s.%s\", additional[j].GroupResource.String(), additional[j].Name)\n\t\t\t})\n\n\t\t\tassert.Equal(t, test.expectedAdditionalItems, additional)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/itemblock/itemblock.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage itemblock\n\nimport (\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n)\n\ntype ItemBlock struct {\n\tLog logrus.FieldLogger\n\tItems []ItemBlockItem\n}\n\ntype ItemBlockItem struct {\n\tGr schema.GroupResource\n\tItem *unstructured.Unstructured\n\tPreferredGVR schema.GroupVersionResource\n}\n\nfunc (ib *ItemBlock) AddUnstructured(gr schema.GroupResource, item *unstructured.Unstructured, preferredGVR schema.GroupVersionResource) {\n\tib.Items = append(ib.Items, ItemBlockItem{\n\t\tGr: gr,\n\t\tItem: item,\n\t\tPreferredGVR: preferredGVR,\n\t})\n}\n\n// Could return multiple items if EnableAPIGroupVersions is set. The item matching the preferredGVR is returned first\nfunc (ib *ItemBlock) FindItem(gr schema.GroupResource, namespace, name string) []ItemBlockItem {\n\tvar itemList []ItemBlockItem\n\tvar returnList []ItemBlockItem\n\n\tfor _, item := range ib.Items {\n\t\tif item.Gr == gr && item.Item != nil && item.Item.GetName() == name && item.Item.GetNamespace() == namespace {\n\t\t\titemGV, err := schema.ParseGroupVersion(item.Item.GetAPIVersion())\n\t\t\tif err == nil && item.PreferredGVR.GroupVersion() == itemGV {\n\t\t\t\treturnList = append(returnList, item)\n\t\t\t} else {\n\t\t\t\titemList = append(itemList, item)\n\t\t\t}\n\t\t}\n\t}\n\treturn append(returnList, itemList...)\n}\n" }, { "path": "pkg/itemoperation/backup_operation.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage itemoperation\n\nimport (\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// BackupOperation stores information about an async item operation\n// started by a BackupItemAction plugin (v2 or later)\ntype BackupOperation struct {\n\tSpec BackupOperationSpec `json:\"spec\"`\n\n\tStatus OperationStatus `json:\"status\"`\n}\n\nfunc (in *BackupOperation) DeepCopy() *BackupOperation {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(BackupOperation)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\nfunc (in *BackupOperation) DeepCopyInto(out *BackupOperation) {\n\t*out = *in\n\tin.Spec.DeepCopyInto(&out.Spec)\n\tin.Status.DeepCopyInto(&out.Status)\n}\n\ntype BackupOperationSpec struct {\n\t// BackupName is the name of the Velero backup this item operation\n\t// is associated with.\n\tBackupName string `json:\"backupName\"`\n\n\t// BackupUID is the UID of the Velero backup this item operation\n\t// is associated with.\n\tBackupUID string `json:\"backupUID\"`\n\n\t// BackupItemAction is the name of the BackupItemAction plugin that started the operation\n\tBackupItemAction string `json:\"backupItemAction\"`\n\n\t// Kubernetes resource identifier for the item\n\tResourceIdentifier velero.ResourceIdentifier `json:\"resourceIdentifier\"`\n\n\t// OperationID returned by the BIA plugin\n\tOperationID string `json:\"operationID\"`\n\n\t// Items needing to be added to the backup after all async operations have completed\n\tPostOperationItems []velero.ResourceIdentifier `json:\"postOperationItems\"`\n}\n\nfunc (in *BackupOperationSpec) DeepCopy() *BackupOperationSpec {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(BackupOperationSpec)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\nfunc (in *BackupOperationSpec) DeepCopyInto(out *BackupOperationSpec) {\n\t*out = *in\n\tin.ResourceIdentifier.DeepCopyInto(&out.ResourceIdentifier)\n\tif in.PostOperationItems != nil {\n\t\tin, out := &in.PostOperationItems, &out.PostOperationItems\n\t\t*out = make([]velero.ResourceIdentifier, len(*in))\n\t\tfor i := range *in {\n\t\t\t(*in)[i].DeepCopyInto(&(*out)[i])\n\t\t}\n\t}\n}\n" }, { "path": "pkg/itemoperation/restore_operation.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage itemoperation\n\nimport (\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// RestoreOperation stores information about an async item operation\n// started by a RestoreItemAction plugin (v2 or later)\ntype RestoreOperation struct {\n\tSpec RestoreOperationSpec `json:\"spec\"`\n\n\tStatus OperationStatus `json:\"status\"`\n}\n\nfunc (in *RestoreOperation) DeepCopy() *RestoreOperation {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(RestoreOperation)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\nfunc (in *RestoreOperation) DeepCopyInto(out *RestoreOperation) {\n\t*out = *in\n\tin.Spec.DeepCopyInto(&out.Spec)\n\tin.Status.DeepCopyInto(&out.Status)\n}\n\ntype RestoreOperationSpec struct {\n\t// RestoreName is the name of the Velero restore this item operation\n\t// is associated with.\n\tRestoreName string `json:\"restoreName\"`\n\n\t// RestoreUID is the UID of the Velero restore this item operation\n\t// is associated with.\n\tRestoreUID string `json:\"restoreUID\"`\n\n\t// RestoreItemAction is the name of the RestoreItemAction plugin that started the operation\n\tRestoreItemAction string `json:\"restoreItemAction\"`\n\n\t// Kubernetes resource identifier for the item\n\tResourceIdentifier velero.ResourceIdentifier `json:\"resourceIdentifier\"`\n\n\t// OperationID returned by the RIA plugin\n\tOperationID string `json:\"operationID\"`\n}\n\nfunc (in *RestoreOperationSpec) DeepCopy() *RestoreOperationSpec {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(RestoreOperationSpec)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\nfunc (in *RestoreOperationSpec) DeepCopyInto(out *RestoreOperationSpec) {\n\t*out = *in\n\tin.ResourceIdentifier.DeepCopyInto(&out.ResourceIdentifier)\n}\n" }, { "path": "pkg/itemoperation/shared.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage itemoperation\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\n// OperationPhase is the lifecycle phase of a Velero item operation\ntype OperationPhase string\n\ntype OperationStatus struct {\n\t// Phase is the current state of the item operation.\n\tPhase OperationPhase `json:\"phase,omitempty\"`\n\n\t// Error displays the reason for a failed operation\n\tError string `json:\"error,omitempty\"`\n\n\t// Amount of operation completed (measured in OperationUnits)\n\t// i.e. number of bytes transferred for a volume\n\tNCompleted int64 `json:\"nCompleted,omitempty\"`\n\n\t// Total Amount of operation (measured in OperationUnits)\n\t// i.e. volume size in bytes\n\tNTotal int64 `json:\"nTotal,omitempty\"`\n\n\t// Units that NCompleted,NTotal are measured in\n\t// i.e. \"bytes\"\n\tOperationUnits string `json:\"operationUnits,omitempty\"`\n\n\t// Description of progress made\n\t// i.e. \"processing\", \"Current phase: Running\", etc.\n\tDescription string `json:\"description,omitempty\"`\n\n\t// Created records the time the item operation was created\n\tCreated *metav1.Time `json:\"created,omitempty\"`\n\n\t// Started records the time the item operation was started, if known\n\t// +optional\n\t// +nullable\n\tStarted *metav1.Time `json:\"started,omitempty\"`\n\n\t// Updated records the time the item operation was updated, if known.\n\t// +optional\n\t// +nullable\n\tUpdated *metav1.Time `json:\"updated,omitempty\"`\n}\n\nfunc (in *OperationStatus) DeepCopy() *OperationStatus {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(OperationStatus)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\nfunc (in *OperationStatus) DeepCopyInto(out *OperationStatus) {\n\t*out = *in\n\tif in.Created != nil {\n\t\tin, out := &in.Created, &out.Created\n\t\t*out = (*in).DeepCopy()\n\t}\n\tif in.Started != nil {\n\t\tin, out := &in.Started, &out.Started\n\t\t*out = (*in).DeepCopy()\n\t}\n\tif in.Updated != nil {\n\t\tin, out := &in.Updated, &out.Updated\n\t\t*out = (*in).DeepCopy()\n\t}\n}\n\nconst (\n\t// OperationPhaseNew means the item operation has been created but not started\n\t// by the plugin\n\tOperationPhaseNew OperationPhase = \"New\"\n\n\t// OperationPhaseInProgress means the item operation has been created and started\n\t// by the plugin\n\tOperationPhaseInProgress OperationPhase = \"InProgress\"\n\n\t// OperationPhaseCompleted means the item operation was successfully completed\n\t// and can be used for restore.\n\tOperationPhaseCompleted OperationPhase = \"Completed\"\n\n\t// OperationPhaseFailed means the item operation ended with an error.\n\tOperationPhaseFailed OperationPhase = \"Failed\"\n)\n" }, { "path": "pkg/itemoperationmap/backup_operation_map.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage itemoperationmap\n\nimport (\n\t\"bytes\"\n\t\"sync\"\n\n\t\"github.com/pkg/errors\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/persistence\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/encode\"\n)\n\ntype BackupItemOperationsMap struct {\n\topsMap map[string]*OperationsForBackup\n\topsLock sync.Mutex\n}\n\n// Returns a pointer to a new BackupItemOperationsMap\nfunc NewBackupItemOperationsMap() *BackupItemOperationsMap {\n\treturn &BackupItemOperationsMap{opsMap: make(map[string]*OperationsForBackup)}\n}\n\n// returns a deep copy so we can minimize the time the map is locked\nfunc (m *BackupItemOperationsMap) GetOperationsForBackup(\n\tbackupStore persistence.BackupStore,\n\tbackupName string) (*OperationsForBackup, error) {\n\tvar err error\n\t// lock operations map\n\tm.opsLock.Lock()\n\tdefer m.opsLock.Unlock()\n\n\toperations, ok := m.opsMap[backupName]\n\tif !ok || len(operations.Operations) == 0 {\n\t\toperations = &OperationsForBackup{}\n\t\toperations.Operations, err = backupStore.GetBackupItemOperations(backupName)\n\t\tif err == nil {\n\t\t\tm.opsMap[backupName] = operations\n\t\t}\n\t}\n\treturn operations.DeepCopy(), err\n}\n\nfunc (m *BackupItemOperationsMap) PutOperationsForBackup(\n\toperations *OperationsForBackup,\n\tbackupName string) {\n\t// lock operations map\n\tm.opsLock.Lock()\n\tdefer m.opsLock.Unlock()\n\tif operations != nil {\n\t\tm.opsMap[backupName] = operations\n\t}\n}\n\nfunc (m *BackupItemOperationsMap) DeleteOperationsForBackup(backupName string) {\n\t// lock operations map\n\tm.opsLock.Lock()\n\tdefer m.opsLock.Unlock()\n\tdelete(m.opsMap, backupName)\n}\n\n// UploadProgressAndPutOperationsForBackup will upload the item operations for this backup to\n// the object store and update the map for this backup with the modified operations\nfunc (m *BackupItemOperationsMap) UploadProgressAndPutOperationsForBackup(\n\tbackupStore persistence.BackupStore,\n\toperations *OperationsForBackup,\n\tbackupName string) error {\n\tm.opsLock.Lock()\n\tdefer m.opsLock.Unlock()\n\n\tif operations == nil {\n\t\treturn errors.New(\"nil operations passed in\")\n\t}\n\tif err := operations.uploadProgress(backupStore, backupName); err != nil {\n\t\treturn err\n\t}\n\tm.opsMap[backupName] = operations\n\treturn nil\n}\n\n// UpdateForBackup will upload the item operations for this backup to\n// the object store, if it has changes not yet uploaded\nfunc (m *BackupItemOperationsMap) UpdateForBackup(backupStore persistence.BackupStore, backupName string) error {\n\t// lock operations map\n\tm.opsLock.Lock()\n\tdefer m.opsLock.Unlock()\n\n\toperations, ok := m.opsMap[backupName]\n\t// if operations for this backup aren't found, or if there are no changes\n\t// or errors since last update, do nothing\n\tif !ok || (!operations.ChangesSinceUpdate && len(operations.ErrsSinceUpdate) == 0) {\n\t\treturn nil\n\t}\n\tif err := operations.uploadProgress(backupStore, backupName); err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n\ntype OperationsForBackup struct {\n\tOperations []*itemoperation.BackupOperation\n\tChangesSinceUpdate bool\n\tErrsSinceUpdate []string\n}\n\nfunc (m *OperationsForBackup) DeepCopy() *OperationsForBackup {\n\tif m == nil {\n\t\treturn nil\n\t}\n\tout := new(OperationsForBackup)\n\tm.DeepCopyInto(out)\n\treturn out\n}\n\nfunc (m *OperationsForBackup) DeepCopyInto(out *OperationsForBackup) {\n\t*out = *m\n\tif m.Operations != nil {\n\t\tin, out := &m.Operations, &out.Operations\n\t\t*out = make([]*itemoperation.BackupOperation, len(*in))\n\t\tfor i := range *in {\n\t\t\tif (*in)[i] != nil {\n\t\t\t\tin, out := &(*in)[i], &(*out)[i]\n\t\t\t\t*out = new(itemoperation.BackupOperation)\n\t\t\t\t(*in).DeepCopyInto(*out)\n\t\t\t}\n\t\t}\n\t}\n\tif m.ErrsSinceUpdate != nil {\n\t\tin, out := &m.ErrsSinceUpdate, &out.ErrsSinceUpdate\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n}\n\nfunc (m *OperationsForBackup) uploadProgress(backupStore persistence.BackupStore, backupName string) error {\n\tif len(m.Operations) > 0 {\n\t\tvar backupItemOperations *bytes.Buffer\n\t\tbackupItemOperations, errs := encode.ToJSONGzip(m.Operations, \"backup item operations list\")\n\t\tif errs != nil {\n\t\t\treturn errors.Wrap(errs[0], \"error encoding item operations json\")\n\t\t}\n\t\terr := backupStore.PutBackupItemOperations(backupName, backupItemOperations)\n\t\tif err != nil {\n\t\t\treturn errors.Wrap(err, \"error uploading item operations json\")\n\t\t}\n\t}\n\tm.ChangesSinceUpdate = false\n\tm.ErrsSinceUpdate = nil\n\treturn nil\n}\n" }, { "path": "pkg/itemoperationmap/restore_operation_map.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage itemoperationmap\n\nimport (\n\t\"bytes\"\n\t\"sync\"\n\n\t\"github.com/pkg/errors\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/persistence\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/encode\"\n)\n\ntype RestoreItemOperationsMap struct {\n\topsMap map[string]*OperationsForRestore\n\topsLock sync.Mutex\n}\n\n// Returns a pointer to a new RestoreItemOperationsMap\nfunc NewRestoreItemOperationsMap() *RestoreItemOperationsMap {\n\treturn &RestoreItemOperationsMap{opsMap: make(map[string]*OperationsForRestore)}\n}\n\n// returns a deep copy so we can minimize the time the map is locked\nfunc (m *RestoreItemOperationsMap) GetOperationsForRestore(\n\tbackupStore persistence.BackupStore,\n\trestoreName string) (*OperationsForRestore, error) {\n\tvar err error\n\t// lock operations map\n\tm.opsLock.Lock()\n\tdefer m.opsLock.Unlock()\n\n\toperations, ok := m.opsMap[restoreName]\n\tif !ok || len(operations.Operations) == 0 {\n\t\toperations = &OperationsForRestore{}\n\t\toperations.Operations, err = backupStore.GetRestoreItemOperations(restoreName)\n\t\tif err == nil {\n\t\t\tm.opsMap[restoreName] = operations\n\t\t}\n\t}\n\treturn operations.DeepCopy(), err\n}\n\nfunc (m *RestoreItemOperationsMap) PutOperationsForRestore(\n\toperations *OperationsForRestore,\n\trestoreName string) {\n\t// lock operations map\n\tm.opsLock.Lock()\n\tdefer m.opsLock.Unlock()\n\tif operations != nil {\n\t\tm.opsMap[restoreName] = operations\n\t}\n}\n\nfunc (m *RestoreItemOperationsMap) DeleteOperationsForRestore(restoreName string) {\n\t// lock operations map\n\tm.opsLock.Lock()\n\tdefer m.opsLock.Unlock()\n\tdelete(m.opsMap, restoreName)\n}\n\n// UploadProgressAndPutOperationsForRestore will upload the item operations for this restore to\n// the object store and update the map for this restore with the modified operations\nfunc (m *RestoreItemOperationsMap) UploadProgressAndPutOperationsForRestore(\n\tbackupStore persistence.BackupStore,\n\toperations *OperationsForRestore,\n\trestoreName string) error {\n\tm.opsLock.Lock()\n\tdefer m.opsLock.Unlock()\n\n\tif operations == nil {\n\t\treturn errors.New(\"nil operations passed in\")\n\t}\n\tif err := operations.uploadProgress(backupStore, restoreName); err != nil {\n\t\treturn err\n\t}\n\tm.opsMap[restoreName] = operations\n\treturn nil\n}\n\n// UpdateForRestore will upload the item operations for this restore to\n// the object store, if it has changes not yet uploaded\nfunc (m *RestoreItemOperationsMap) UpdateForRestore(backupStore persistence.BackupStore, restoreName string) error {\n\t// lock operations map\n\tm.opsLock.Lock()\n\tdefer m.opsLock.Unlock()\n\n\toperations, ok := m.opsMap[restoreName]\n\t// if operations for this restore aren't found, or if there are no changes\n\t// or errors since last update, do nothing\n\tif !ok || (!operations.ChangesSinceUpdate && len(operations.ErrsSinceUpdate) == 0) {\n\t\treturn nil\n\t}\n\tif err := operations.uploadProgress(backupStore, restoreName); err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n\ntype OperationsForRestore struct {\n\tOperations []*itemoperation.RestoreOperation\n\tChangesSinceUpdate bool\n\tErrsSinceUpdate []string\n}\n\nfunc (m *OperationsForRestore) DeepCopy() *OperationsForRestore {\n\tif m == nil {\n\t\treturn nil\n\t}\n\tout := new(OperationsForRestore)\n\tm.DeepCopyInto(out)\n\treturn out\n}\n\nfunc (m *OperationsForRestore) DeepCopyInto(out *OperationsForRestore) {\n\t*out = *m\n\tif m.Operations != nil {\n\t\tin, out := &m.Operations, &out.Operations\n\t\t*out = make([]*itemoperation.RestoreOperation, len(*in))\n\t\tfor i := range *in {\n\t\t\tif (*in)[i] != nil {\n\t\t\t\tin, out := &(*in)[i], &(*out)[i]\n\t\t\t\t*out = new(itemoperation.RestoreOperation)\n\t\t\t\t(*in).DeepCopyInto(*out)\n\t\t\t}\n\t\t}\n\t}\n\tif m.ErrsSinceUpdate != nil {\n\t\tin, out := &m.ErrsSinceUpdate, &out.ErrsSinceUpdate\n\t\t*out = make([]string, len(*in))\n\t\tcopy(*out, *in)\n\t}\n}\n\nfunc (m *OperationsForRestore) uploadProgress(backupStore persistence.BackupStore, restoreName string) error {\n\tif len(m.Operations) > 0 {\n\t\tvar restoreItemOperations *bytes.Buffer\n\t\trestoreItemOperations, errs := encode.ToJSONGzip(m.Operations, \"restore item operations list\")\n\t\tif errs != nil {\n\t\t\treturn errors.Wrap(errs[0], \"error encoding item operations json\")\n\t\t}\n\t\terr := backupStore.PutRestoreItemOperations(restoreName, restoreItemOperations)\n\t\tif err != nil {\n\t\t\treturn errors.Wrap(err, \"error uploading item operations json\")\n\t\t}\n\t}\n\tm.ChangesSinceUpdate = false\n\tm.ErrsSinceUpdate = nil\n\treturn nil\n}\n" }, { "path": "pkg/kopia/kopia_log.go", "content": "package kopia\n\n/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\nimport (\n\t\"context\"\n\t\"io\"\n\n\t\"github.com/kopia/kopia/repo/logging\"\n\t\"github.com/sirupsen/logrus\"\n\t\"go.uber.org/zap\"\n\t\"go.uber.org/zap/zapcore\"\n)\n\ntype kopiaLog struct {\n\tmodule string\n\tlogger logrus.FieldLogger\n}\n\ntype repoLog struct {\n\tlogger logrus.FieldLogger\n}\n\n// SetupKopiaLog sets the Kopia log handler to the specific context, Kopia modules\n// call the logger in the context to write logs\nfunc SetupKopiaLog(ctx context.Context, logger logrus.FieldLogger) context.Context {\n\treturn logging.WithLogger(ctx, func(module string) logging.Logger {\n\t\tkpLog := &kopiaLog{module, logger}\n\t\treturn zap.New(kpLog).Sugar()\n\t})\n}\n\nfunc RepositoryLogger(logger logrus.FieldLogger) io.Writer {\n\treturn &repoLog{logger: logger}\n}\n\n// Enabled decides whether a given logging level is enabled when logging a message\nfunc (kl *kopiaLog) Enabled(level zapcore.Level) bool {\n\tentry := kl.logger.WithField(\"null\", \"null\")\n\tswitch level {\n\tcase zapcore.DebugLevel:\n\t\treturn (entry.Logger.GetLevel() >= logrus.DebugLevel)\n\tcase zapcore.InfoLevel:\n\t\treturn (entry.Logger.GetLevel() >= logrus.InfoLevel)\n\tcase zapcore.WarnLevel:\n\t\treturn (entry.Logger.GetLevel() >= logrus.WarnLevel)\n\tcase zapcore.ErrorLevel:\n\t\treturn (entry.Logger.GetLevel() >= logrus.ErrorLevel)\n\tcase zapcore.DPanicLevel:\n\t\treturn (entry.Logger.GetLevel() >= logrus.PanicLevel)\n\tcase zapcore.PanicLevel:\n\t\treturn (entry.Logger.GetLevel() >= logrus.PanicLevel)\n\tcase zapcore.FatalLevel:\n\t\treturn (entry.Logger.GetLevel() >= logrus.FatalLevel)\n\tdefault:\n\t\treturn false\n\t}\n}\n\n// With adds structured context to the Core.\nfunc (kl *kopiaLog) With(fields []zapcore.Field) zapcore.Core {\n\tcopied := kl.logrusFields(fields)\n\n\treturn &kopiaLog{\n\t\tmodule: kl.module,\n\t\tlogger: kl.logger.WithFields(copied),\n\t}\n}\n\n// Check determines whether the supplied Entry should be logged. If the entry\n// should be logged, the Core adds itself to the CheckedEntry and returns the result.\nfunc (kl *kopiaLog) Check(ent zapcore.Entry, ce *zapcore.CheckedEntry) *zapcore.CheckedEntry {\n\tif kl.Enabled(ent.Level) {\n\t\treturn ce.AddCore(ent, kl)\n\t}\n\n\treturn ce\n}\n\n// Write serializes the Entry and any Fields supplied at the log site and writes them to their destination.\nfunc (kl *kopiaLog) Write(ent zapcore.Entry, fields []zapcore.Field) error {\n\tcopied := kl.logrusFieldsForWrite(ent, fields)\n\tlogger := kl.logger.WithFields(copied)\n\n\tswitch ent.Level {\n\tcase zapcore.DebugLevel:\n\t\tlogger.Debug(ent.Message)\n\tcase zapcore.InfoLevel:\n\t\tlogger.Info(ent.Message)\n\tcase zapcore.WarnLevel:\n\t\tlogger.Warn(ent.Message)\n\tcase zapcore.ErrorLevel:\n\t\t// We see Kopia generates error logs for some normal cases or non-critical\n\t\t// cases. So Kopia's error logs are regarded as warning logs so that they don't\n\t\t// affect Velero's workflow.\n\t\tlogger.Warn(ent.Message)\n\tcase zapcore.DPanicLevel:\n\t\tlogger.Panic(ent.Message)\n\tcase zapcore.PanicLevel:\n\t\tlogger.Panic(ent.Message)\n\tcase zapcore.FatalLevel:\n\t\tlogger.Fatal(ent.Message)\n\t}\n\n\treturn nil\n}\n\n// Sync flushes buffered logs (if any).\nfunc (kl *kopiaLog) Sync() error {\n\treturn nil\n}\n\nfunc (kl *kopiaLog) logrusFields(fields []zapcore.Field) logrus.Fields {\n\tif fields == nil {\n\t\treturn logrus.Fields{}\n\t}\n\n\tm := zapcore.NewMapObjectEncoder()\n\tfor _, field := range fields {\n\t\tfield.AddTo(m)\n\t}\n\n\treturn m.Fields\n}\n\nfunc (kl *kopiaLog) getLogModule() string {\n\treturn \"kopia/\" + kl.module\n}\n\nfunc (kl *kopiaLog) logrusFieldsForWrite(ent zapcore.Entry, fields []zapcore.Field) logrus.Fields {\n\tcopied := kl.logrusFields(fields)\n\n\tcopied[\"logModule\"] = kl.getLogModule()\n\n\tif ent.Caller.Function != \"\" {\n\t\tcopied[\"function\"] = ent.Caller.Function\n\t}\n\n\tpath := ent.Caller.FullPath()\n\tif path != \"undefined\" {\n\t\tcopied[\"path\"] = path\n\t}\n\n\tif ent.LoggerName != \"\" {\n\t\tcopied[\"logger name\"] = ent.LoggerName\n\t}\n\n\tif ent.Stack != \"\" {\n\t\tcopied[\"stack\"] = ent.Stack\n\t}\n\n\tif ent.Level == zap.ErrorLevel {\n\t\tcopied[\"sublevel\"] = \"error\"\n\t}\n\n\treturn copied\n}\n\nfunc (rl *repoLog) Write(p []byte) (int, error) {\n\trl.logger.Debug(string(p))\n\n\treturn len(p), nil\n}\n" }, { "path": "pkg/kopia/kopia_log_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kopia\n\nimport (\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"go.uber.org/zap/zapcore\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestEnabled(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tlevel logrus.Level\n\t\tzapLevel zapcore.Level\n\t\texpected bool\n\t}{\n\t\t{\n\t\t\tname: \"check debug again debug\",\n\t\t\tlevel: logrus.DebugLevel,\n\t\t\tzapLevel: zapcore.DebugLevel,\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"check debug again info\",\n\t\t\tlevel: logrus.InfoLevel,\n\t\t\tzapLevel: zapcore.DebugLevel,\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"check info again debug\",\n\t\t\tlevel: logrus.DebugLevel,\n\t\t\tzapLevel: zapcore.InfoLevel,\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"check info again info\",\n\t\t\tlevel: logrus.InfoLevel,\n\t\t\tzapLevel: zapcore.InfoLevel,\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"check warn again warn\",\n\t\t\tlevel: logrus.WarnLevel,\n\t\t\tzapLevel: zapcore.WarnLevel,\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"check info again error\",\n\t\t\tlevel: logrus.ErrorLevel,\n\t\t\tzapLevel: zapcore.InfoLevel,\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"check error again error\",\n\t\t\tlevel: logrus.ErrorLevel,\n\t\t\tzapLevel: zapcore.ErrorLevel,\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"check dppanic again panic\",\n\t\t\tlevel: logrus.PanicLevel,\n\t\t\tzapLevel: zapcore.DPanicLevel,\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"check panic again error\",\n\t\t\tlevel: logrus.ErrorLevel,\n\t\t\tzapLevel: zapcore.PanicLevel,\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"check fatal again fatal\",\n\t\t\tlevel: logrus.FatalLevel,\n\t\t\tzapLevel: zapcore.FatalLevel,\n\t\t\texpected: true,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlog := kopiaLog{\n\t\t\t\tlogger: test.NewLoggerWithLevel(tc.level),\n\t\t\t}\n\t\t\tm := log.Enabled(tc.zapLevel)\n\n\t\t\trequire.Equal(t, tc.expected, m)\n\t\t})\n\t}\n}\n\nfunc TestLogrusFieldsForWrite(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tmodule string\n\t\tzapEntry zapcore.Entry\n\t\tzapFields []zapcore.Field\n\t\texpected logrus.Fields\n\t}{\n\t\t{\n\t\t\tname: \"debug with nil fields\",\n\t\t\tmodule: \"module-01\",\n\t\t\tzapEntry: zapcore.Entry{\n\t\t\t\tLevel: zapcore.DebugLevel,\n\t\t\t},\n\t\t\tzapFields: nil,\n\t\t\texpected: logrus.Fields{\n\t\t\t\t\"logModule\": \"kopia/module-01\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"error with nil fields\",\n\t\t\tmodule: \"module-02\",\n\t\t\tzapEntry: zapcore.Entry{\n\t\t\t\tLevel: zapcore.ErrorLevel,\n\t\t\t},\n\t\t\tzapFields: nil,\n\t\t\texpected: logrus.Fields{\n\t\t\t\t\"logModule\": \"kopia/module-02\",\n\t\t\t\t\"sublevel\": \"error\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"info with nil string filed\",\n\t\t\tmodule: \"module-03\",\n\t\t\tzapEntry: zapcore.Entry{\n\t\t\t\tLevel: zapcore.InfoLevel,\n\t\t\t},\n\t\t\tzapFields: []zapcore.Field{\n\t\t\t\t{\n\t\t\t\t\tKey: \"key-01\",\n\t\t\t\t\tType: zapcore.StringType,\n\t\t\t\t\tString: \"value-01\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: logrus.Fields{\n\t\t\t\t\"logModule\": \"kopia/module-03\",\n\t\t\t\t\"key-01\": \"value-01\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"info with logger name\",\n\t\t\tmodule: \"module-04\",\n\t\t\tzapEntry: zapcore.Entry{\n\t\t\t\tLevel: zapcore.InfoLevel,\n\t\t\t\tLoggerName: \"logger-name-01\",\n\t\t\t},\n\t\t\tzapFields: nil,\n\t\t\texpected: logrus.Fields{\n\t\t\t\t\"logModule\": \"kopia/module-04\",\n\t\t\t\t\"logger name\": \"logger-name-01\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"info with function name\",\n\t\t\tmodule: \"module-05\",\n\t\t\tzapEntry: zapcore.Entry{\n\t\t\t\tLevel: zapcore.InfoLevel,\n\t\t\t\tCaller: zapcore.EntryCaller{\n\t\t\t\t\tFunction: \"function-name-01\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tzapFields: nil,\n\t\t\texpected: logrus.Fields{\n\t\t\t\t\"logModule\": \"kopia/module-05\",\n\t\t\t\t\"function\": \"function-name-01\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"info with undefined path\",\n\t\t\tmodule: \"module-06\",\n\t\t\tzapEntry: zapcore.Entry{\n\t\t\t\tLevel: zapcore.InfoLevel,\n\t\t\t\tCaller: zapcore.EntryCaller{\n\t\t\t\t\tDefined: false,\n\t\t\t\t},\n\t\t\t},\n\t\t\tzapFields: nil,\n\t\t\texpected: logrus.Fields{\n\t\t\t\t\"logModule\": \"kopia/module-06\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"info with defined path\",\n\t\t\tmodule: \"module-06\",\n\t\t\tzapEntry: zapcore.Entry{\n\t\t\t\tLevel: zapcore.InfoLevel,\n\t\t\t\tCaller: zapcore.EntryCaller{\n\t\t\t\t\tDefined: true,\n\t\t\t\t\tFile: \"file-name-01\",\n\t\t\t\t\tLine: 100,\n\t\t\t\t},\n\t\t\t},\n\t\t\tzapFields: nil,\n\t\t\texpected: logrus.Fields{\n\t\t\t\t\"logModule\": \"kopia/module-06\",\n\t\t\t\t\"path\": \"file-name-01:100\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"info with stack\",\n\t\t\tmodule: \"module-07\",\n\t\t\tzapEntry: zapcore.Entry{\n\t\t\t\tLevel: zapcore.InfoLevel,\n\t\t\t\tStack: \"fake-stack\",\n\t\t\t},\n\t\t\tzapFields: nil,\n\t\t\texpected: logrus.Fields{\n\t\t\t\t\"logModule\": \"kopia/module-07\",\n\t\t\t\t\"stack\": \"fake-stack\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlog := kopiaLog{\n\t\t\t\tmodule: tc.module,\n\t\t\t\tlogger: test.NewLogger(),\n\t\t\t}\n\t\t\tm := log.logrusFieldsForWrite(tc.zapEntry, tc.zapFields)\n\n\t\t\trequire.Equal(t, tc.expected, m)\n\t\t})\n\t}\n}\n\nfunc TestWrite(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tent zapcore.Entry\n\t\tlogMessage string\n\t\tlogLevel string\n\t\tshouldPanic bool\n\t}{\n\t\t{\n\t\t\tname: \"write debug\",\n\t\t\tent: zapcore.Entry{\n\t\t\t\tLevel: zapcore.DebugLevel,\n\t\t\t\tMessage: \"fake-message\",\n\t\t\t},\n\t\t\tlogMessage: \"fake-message\",\n\t\t\tlogLevel: \"level=debug\",\n\t\t},\n\t\t{\n\t\t\tname: \"write info\",\n\t\t\tent: zapcore.Entry{\n\t\t\t\tLevel: zapcore.InfoLevel,\n\t\t\t\tMessage: \"fake-message\",\n\t\t\t},\n\t\t\tlogMessage: \"fake-message\",\n\t\t\tlogLevel: \"level=info\",\n\t\t},\n\t\t{\n\t\t\tname: \"write warn\",\n\t\t\tent: zapcore.Entry{\n\t\t\t\tLevel: zapcore.WarnLevel,\n\t\t\t\tMessage: \"fake-message\",\n\t\t\t},\n\t\t\tlogMessage: \"fake-message\",\n\t\t\tlogLevel: \"level=warn\",\n\t\t},\n\t\t{\n\t\t\tname: \"write error\",\n\t\t\tent: zapcore.Entry{\n\t\t\t\tLevel: zapcore.ErrorLevel,\n\t\t\t\tMessage: \"fake-message\",\n\t\t\t},\n\t\t\tlogMessage: \"fake-message\",\n\t\t\tlogLevel: \"level=warn\",\n\t\t},\n\t\t{\n\t\t\tname: \"write DPanic\",\n\t\t\tent: zapcore.Entry{\n\t\t\t\tLevel: zapcore.DPanicLevel,\n\t\t\t\tMessage: \"fake-message\",\n\t\t\t},\n\t\t\tlogMessage: \"fake-message\",\n\t\t\tlogLevel: \"level=panic\",\n\t\t\tshouldPanic: true,\n\t\t},\n\t\t{\n\t\t\tname: \"write panic\",\n\t\t\tent: zapcore.Entry{\n\t\t\t\tLevel: zapcore.PanicLevel,\n\t\t\t\tMessage: \"fake-message\",\n\t\t\t},\n\t\t\tlogMessage: \"fake-message\",\n\t\t\tlogLevel: \"level=panic\",\n\t\t\tshouldPanic: true,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlogMessage := \"\"\n\n\t\t\tlog := kopiaLog{\n\t\t\t\tlogger: test.NewSingleLogger(&logMessage),\n\t\t\t}\n\n\t\t\tif tc.shouldPanic {\n\t\t\t\tdefer func() {\n\t\t\t\t\tr := recover()\n\t\t\t\t\tassert.NotNil(t, r)\n\n\t\t\t\t\tif len(tc.logMessage) > 0 {\n\t\t\t\t\t\tassert.Contains(t, logMessage, tc.logMessage)\n\t\t\t\t\t}\n\n\t\t\t\t\tif len(tc.logLevel) > 0 {\n\t\t\t\t\t\tassert.Contains(t, logMessage, tc.logLevel)\n\t\t\t\t\t}\n\t\t\t\t}()\n\t\t\t}\n\n\t\t\terr := log.Write(tc.ent, nil)\n\n\t\t\trequire.NoError(t, err)\n\n\t\t\tif len(tc.logMessage) > 0 {\n\t\t\t\tassert.Contains(t, logMessage, tc.logMessage)\n\t\t\t}\n\n\t\t\tif len(tc.logLevel) > 0 {\n\t\t\t\tassert.Contains(t, logMessage, tc.logLevel)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/kuberesource/kuberesource.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kuberesource\n\nimport (\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n)\n\nvar (\n\tClusterRoleBindings = schema.GroupResource{Group: \"rbac.authorization.k8s.io\", Resource: \"clusterrolebindings\"}\n\tClusterRoles = schema.GroupResource{Group: \"rbac.authorization.k8s.io\", Resource: \"clusterroles\"}\n\tCustomResourceDefinitions = schema.GroupResource{Group: \"apiextensions.k8s.io\", Resource: \"customresourcedefinitions\"}\n\tJobs = schema.GroupResource{Group: \"batch\", Resource: \"jobs\"}\n\tNamespaces = schema.GroupResource{Group: \"\", Resource: \"namespaces\"}\n\tPersistentVolumeClaims = schema.GroupResource{Group: \"\", Resource: \"persistentvolumeclaims\"}\n\tPersistentVolumes = schema.GroupResource{Group: \"\", Resource: \"persistentvolumes\"}\n\tPods = schema.GroupResource{Group: \"\", Resource: \"pods\"}\n\tServiceAccounts = schema.GroupResource{Group: \"\", Resource: \"serviceaccounts\"}\n\tSecrets = schema.GroupResource{Group: \"\", Resource: \"secrets\"}\n\tVolumeSnapshotClasses = schema.GroupResource{Group: \"snapshot.storage.k8s.io\", Resource: \"volumesnapshotclasses\"}\n\tVolumeSnapshots = schema.GroupResource{Group: \"snapshot.storage.k8s.io\", Resource: \"volumesnapshots\"}\n\tVolumeGroupSnapshots = schema.GroupResource{Group: \"snapshot.storage.k8s.io\", Resource: \"volumegroupsnapshots\"}\n\tVolumeSnapshotContents = schema.GroupResource{Group: \"snapshot.storage.k8s.io\", Resource: \"volumesnapshotcontents\"}\n\tPriorityClasses = schema.GroupResource{Group: \"scheduling.k8s.io\", Resource: \"priorityclasses\"}\n\tDataUploads = schema.GroupResource{Group: \"velero.io\", Resource: \"datauploads\"}\n\tVGSKind = \"VolumeGroupSnapshot\"\n)\n" }, { "path": "pkg/label/label.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage label\n\nimport (\n\t\"crypto/sha256\"\n\t\"crypto/sha3\"\n\t\"encoding/hex\"\n\t\"fmt\"\n\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/util/validation\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// GetValidName converts an input string to valid Kubernetes label string in accordance to rfc1035 DNS Label spec\n// (https://github.com/kubernetes/community/blob/master/contributors/design-proposals/architecture/identifiers.md)\n// Length of the label is adjusted basis the DNS1035LabelMaxLength (defined at k8s.io/apimachinery/pkg/util/validation)\n// If length exceeds, we trim the label name to contain only max allowed characters\n// Additionally, the last 6 characters of the label name are replaced by the first 6 characters of the sha256 of original label\nfunc GetValidName(label string) string {\n\tif len(label) <= validation.DNS1035LabelMaxLength {\n\t\treturn label\n\t}\n\n\tsha := sha256.Sum256([]byte(label))\n\tstrSha := hex.EncodeToString(sha[:])\n\tcharsFromLabel := validation.DNS1035LabelMaxLength - 6\n\tif charsFromLabel < 0 {\n\t\t// Derive the label name from sha hash in case the DNS1035LabelMaxLength is less than 6\n\t\treturn string(strSha[validation.DNS1035LabelMaxLength])\n\t}\n\n\treturn label[:charsFromLabel] + strSha[:6]\n}\n\n// ReturnNameOrHash returns the original name if it is within the DNS1035LabelMaxLength limit,\n// otherwise it returns the sha3 Sum224 hash(length is 56) of the name.\nfunc ReturnNameOrHash(name string) string {\n\tif len(name) <= validation.DNS1035LabelMaxLength {\n\t\treturn name\n\t}\n\n\thash := sha3.Sum224([]byte(name))\n\treturn hex.EncodeToString(hash[:])\n}\n\n// NewSelectorForBackup returns a Selector based on the backup name.\n// This is useful for interacting with Listers that need a Selector.\nfunc NewSelectorForBackup(name string) labels.Selector {\n\treturn labels.SelectorFromSet(map[string]string{velerov1api.BackupNameLabel: GetValidName(name)})\n}\n\n// NewListOptionsForBackup returns a ListOptions based on the backup name.\n// This is useful for interacting with client-go clients that needs a ListOptions.\nfunc NewListOptionsForBackup(name string) metav1.ListOptions {\n\treturn metav1.ListOptions{\n\t\tLabelSelector: fmt.Sprintf(\"%s=%s\", velerov1api.BackupNameLabel, GetValidName(name)),\n\t}\n}\n\n// NewSelectorForRestore returns a Selector based on the restore name.\n// This is useful for interacting with Listers that need a Selector.\nfunc NewSelectorForRestore(name string) labels.Selector {\n\treturn labels.SelectorFromSet(map[string]string{velerov1api.RestoreNameLabel: GetValidName(name)})\n}\n" }, { "path": "pkg/label/label_test.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage label\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestGetValidLabelName(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tlabel string\n\t\texpectedLabel string\n\t}{\n\t\t{\n\t\t\tname: \"valid label name should not be modified\",\n\t\t\tlabel: \"short label value\",\n\t\t\texpectedLabel: \"short label value\",\n\t\t},\n\t\t{\n\t\t\tname: \"label with more than 63 characters should be modified\",\n\t\t\tlabel: \"this_is_a_very_long_label_value_that_will_be_rejected_by_Kubernetes\",\n\t\t\texpectedLabel: \"this_is_a_very_long_label_value_that_will_be_rejected_by_8d0722\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tlabelVal := GetValidName(test.label)\n\t\t\tassert.Equal(t, test.expectedLabel, labelVal)\n\t\t})\n\t}\n}\n\nfunc TestReturnNameOrHash(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tlabel string\n\t\texpectedLabel string\n\t}{\n\t\t{\n\t\t\tname: \"valid label name should not be modified\",\n\t\t\tlabel: \"short label value\",\n\t\t\texpectedLabel: \"short label value\",\n\t\t},\n\t\t{\n\t\t\tname: \"label with more than 63 characters should be modified\",\n\t\t\tlabel: \"this_is_a_very_long_label_value_that_will_be_rejected_by_Kubernetes\",\n\t\t\texpectedLabel: \"1a7399f2d00e268fc12daf431d6667319d1461e2609981070bb7e85c\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tlabelVal := ReturnNameOrHash(test.label)\n\t\t\tassert.Equal(t, test.expectedLabel, labelVal)\n\t\t})\n\t}\n}\n\nfunc TestNewSelectorForBackup(t *testing.T) {\n\tselector := NewSelectorForBackup(\"my-backup\")\n\tassert.Equal(t, \"velero.io/backup-name=my-backup\", selector.String())\n}\n\nfunc TestNewListOptionsForBackup(t *testing.T) {\n\toption := NewListOptionsForBackup(\"my-backup\")\n\tassert.Equal(t, \"velero.io/backup-name=my-backup\", option.LabelSelector)\n}\n" }, { "path": "pkg/metrics/metrics.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage metrics\n\nimport (\n\t\"time\"\n\n\t\"github.com/prometheus/client_golang/prometheus\"\n)\n\n// ServerMetrics contains Prometheus metrics for the Velero server.\ntype ServerMetrics struct {\n\tmetrics map[string]prometheus.Collector\n}\n\n// Metrics returns the metrics map for testing purposes.\nfunc (m *ServerMetrics) Metrics() map[string]prometheus.Collector {\n\treturn m.metrics\n}\n\nconst (\n\tmetricNamespace = \"velero\"\n\tpodVolumeMetricsNamespace = \"podVolume\"\n\t//Velero metrics\n\tbackupTarballSizeBytesGauge = \"backup_tarball_size_bytes\"\n\tbackupTotal = \"backup_total\"\n\tbackupAttemptTotal = \"backup_attempt_total\"\n\tbackupSuccessTotal = \"backup_success_total\"\n\tbackupPartialFailureTotal = \"backup_partial_failure_total\"\n\tbackupFailureTotal = \"backup_failure_total\"\n\tbackupValidationFailureTotal = \"backup_validation_failure_total\"\n\tbackupDurationSeconds = \"backup_duration_seconds\"\n\tbackupDeletionAttemptTotal = \"backup_deletion_attempt_total\"\n\tbackupDeletionSuccessTotal = \"backup_deletion_success_total\"\n\tbackupDeletionFailureTotal = \"backup_deletion_failure_total\"\n\tbackupLastSuccessfulTimestamp = \"backup_last_successful_timestamp\"\n\tbackupItemsTotalGauge = \"backup_items_total\"\n\tbackupItemsErrorsGauge = \"backup_items_errors\"\n\tbackupWarningTotal = \"backup_warning_total\"\n\tbackupLastStatus = \"backup_last_status\"\n\tbackupLocationStatus = \"backup_location_status_gauge\"\n\trestoreTotal = \"restore_total\"\n\trestoreAttemptTotal = \"restore_attempt_total\"\n\trestoreValidationFailedTotal = \"restore_validation_failed_total\"\n\trestoreSuccessTotal = \"restore_success_total\"\n\trestorePartialFailureTotal = \"restore_partial_failure_total\"\n\trestoreFailedTotal = \"restore_failed_total\"\n\tvolumeSnapshotAttemptTotal = \"volume_snapshot_attempt_total\"\n\tvolumeSnapshotSuccessTotal = \"volume_snapshot_success_total\"\n\tvolumeSnapshotFailureTotal = \"volume_snapshot_failure_total\"\n\tcsiSnapshotAttemptTotal = \"csi_snapshot_attempt_total\"\n\tcsiSnapshotSuccessTotal = \"csi_snapshot_success_total\"\n\tcsiSnapshotFailureTotal = \"csi_snapshot_failure_total\"\n\n\t// pod volume metrics\n\tpodVolumeBackupEnqueueTotal = \"pod_volume_backup_enqueue_count\"\n\tpodVolumeBackupDequeueTotal = \"pod_volume_backup_dequeue_count\"\n\tpodVolumeOperationLatencySeconds = \"pod_volume_operation_latency_seconds\"\n\tpodVolumeOperationLatencyGaugeSeconds = \"pod_volume_operation_latency_seconds_gauge\"\n\n\t// data mover metrics\n\tDataUploadSuccessTotal = \"data_upload_success_total\"\n\tDataUploadFailureTotal = \"data_upload_failure_total\"\n\tDataUploadCancelTotal = \"data_upload_cancel_total\"\n\tDataDownloadSuccessTotal = \"data_download_success_total\"\n\tDataDownloadFailureTotal = \"data_download_failure_total\"\n\tDataDownloadCancelTotal = \"data_download_cancel_total\"\n\n\t// schedule metrics\n\tscheduleExpectedIntervalSeconds = \"schedule_expected_interval_seconds\"\n\n\t// repo maintenance metrics\n\trepoMaintenanceSuccessTotal = \"repo_maintenance_success_total\"\n\trepoMaintenanceFailureTotal = \"repo_maintenance_failure_total\"\n\t// repoMaintenanceDurationSeconds tracks the distribution of maintenance job durations.\n\t// Each completed job's duration is recorded in the appropriate bucket, allowing\n\t// analysis of individual job performance and trending over time.\n\trepoMaintenanceDurationSeconds = \"repo_maintenance_duration_seconds\"\n\n\t// Labels\n\tnodeMetricLabel = \"node\"\n\tpodVolumeOperationLabel = \"operation\"\n\tbslNameLabel = \"backup_location_name\"\n\tpvbNameLabel = \"pod_volume_backup\"\n\tscheduleLabel = \"schedule\"\n\tbackupNameLabel = \"backupName\"\n\trepositoryNameLabel = \"repository_name\"\n\n\t// metrics values\n\tBackupLastStatusSucc int64 = 1\n\tBackupLastStatusFailure int64 = 0\n)\n\n// NewServerMetrics returns new ServerMetrics\nfunc NewServerMetrics() *ServerMetrics {\n\treturn &ServerMetrics{\n\t\tmetrics: map[string]prometheus.Collector{\n\t\t\tbackupTarballSizeBytesGauge: prometheus.NewGaugeVec(\n\t\t\t\tprometheus.GaugeOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: backupTarballSizeBytesGauge,\n\t\t\t\t\tHelp: \"Size, in bytes, of a backup\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\tbackupLastSuccessfulTimestamp: prometheus.NewGaugeVec(\n\t\t\t\tprometheus.GaugeOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: backupLastSuccessfulTimestamp,\n\t\t\t\t\tHelp: \"Last time a backup ran successfully, Unix timestamp in seconds\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\tbackupTotal: prometheus.NewGauge(\n\t\t\t\tprometheus.GaugeOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: backupTotal,\n\t\t\t\t\tHelp: \"Current number of existent backups\",\n\t\t\t\t},\n\t\t\t),\n\t\t\tbackupAttemptTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: backupAttemptTotal,\n\t\t\t\t\tHelp: \"Total number of attempted backups\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\tbackupSuccessTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: backupSuccessTotal,\n\t\t\t\t\tHelp: \"Total number of successful backups\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\tbackupPartialFailureTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: backupPartialFailureTotal,\n\t\t\t\t\tHelp: \"Total number of partially failed backups\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\tbackupFailureTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: backupFailureTotal,\n\t\t\t\t\tHelp: \"Total number of failed backups\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\tbackupValidationFailureTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: backupValidationFailureTotal,\n\t\t\t\t\tHelp: \"Total number of validation failed backups\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\tbackupDeletionAttemptTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: backupDeletionAttemptTotal,\n\t\t\t\t\tHelp: \"Total number of attempted backup deletions\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\tbackupDeletionSuccessTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: backupDeletionSuccessTotal,\n\t\t\t\t\tHelp: \"Total number of successful backup deletions\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\tbackupDeletionFailureTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: backupDeletionFailureTotal,\n\t\t\t\t\tHelp: \"Total number of failed backup deletions\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\tbackupDurationSeconds: prometheus.NewHistogramVec(\n\t\t\t\tprometheus.HistogramOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: backupDurationSeconds,\n\t\t\t\t\tHelp: \"Time taken to complete backup, in seconds\",\n\t\t\t\t\tBuckets: []float64{\n\t\t\t\t\t\ttoSeconds(1 * time.Minute),\n\t\t\t\t\t\ttoSeconds(5 * time.Minute),\n\t\t\t\t\t\ttoSeconds(10 * time.Minute),\n\t\t\t\t\t\ttoSeconds(15 * time.Minute),\n\t\t\t\t\t\ttoSeconds(30 * time.Minute),\n\t\t\t\t\t\ttoSeconds(1 * time.Hour),\n\t\t\t\t\t\ttoSeconds(2 * time.Hour),\n\t\t\t\t\t\ttoSeconds(3 * time.Hour),\n\t\t\t\t\t\ttoSeconds(4 * time.Hour),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\tbackupItemsTotalGauge: prometheus.NewGaugeVec(\n\t\t\t\tprometheus.GaugeOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: backupItemsTotalGauge,\n\t\t\t\t\tHelp: \"Total number of items backed up\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\tbackupItemsErrorsGauge: prometheus.NewGaugeVec(\n\t\t\t\tprometheus.GaugeOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: backupItemsErrorsGauge,\n\t\t\t\t\tHelp: \"Total number of errors encountered during backup\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\tbackupWarningTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: backupWarningTotal,\n\t\t\t\t\tHelp: \"Total number of warned backups\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\tbackupLastStatus: prometheus.NewGaugeVec(\n\t\t\t\tprometheus.GaugeOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: backupLastStatus,\n\t\t\t\t\tHelp: \"Last status of the backup. A value of 1 is success, 0 is failure\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\tbackupLocationStatus: prometheus.NewGaugeVec(\n\t\t\t\tprometheus.GaugeOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: backupLocationStatus,\n\t\t\t\t\tHelp: \"The status of backup location. A value of 1 is available, 0 is unavailable\",\n\t\t\t\t},\n\t\t\t\t[]string{bslNameLabel},\n\t\t\t),\n\t\t\trestoreTotal: prometheus.NewGauge(\n\t\t\t\tprometheus.GaugeOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: restoreTotal,\n\t\t\t\t\tHelp: \"Current number of existent restores\",\n\t\t\t\t},\n\t\t\t),\n\t\t\trestoreAttemptTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: restoreAttemptTotal,\n\t\t\t\t\tHelp: \"Total number of attempted restores\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\trestoreSuccessTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: restoreSuccessTotal,\n\t\t\t\t\tHelp: \"Total number of successful restores\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\trestorePartialFailureTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: restorePartialFailureTotal,\n\t\t\t\t\tHelp: \"Total number of partially failed restores\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\trestoreFailedTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: restoreFailedTotal,\n\t\t\t\t\tHelp: \"Total number of failed restores\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\trestoreValidationFailedTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: restoreValidationFailedTotal,\n\t\t\t\t\tHelp: \"Total number of failed restores failing validations\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\tvolumeSnapshotAttemptTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: volumeSnapshotAttemptTotal,\n\t\t\t\t\tHelp: \"Total number of attempted volume snapshots\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\tvolumeSnapshotSuccessTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: volumeSnapshotSuccessTotal,\n\t\t\t\t\tHelp: \"Total number of successful volume snapshots\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\tvolumeSnapshotFailureTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: volumeSnapshotFailureTotal,\n\t\t\t\t\tHelp: \"Total number of failed volume snapshots\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\tcsiSnapshotAttemptTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: csiSnapshotAttemptTotal,\n\t\t\t\t\tHelp: \"Total number of CSI attempted volume snapshots\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel, backupNameLabel},\n\t\t\t),\n\t\t\tcsiSnapshotSuccessTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: csiSnapshotSuccessTotal,\n\t\t\t\t\tHelp: \"Total number of CSI successful volume snapshots\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel, backupNameLabel},\n\t\t\t),\n\t\t\tcsiSnapshotFailureTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: csiSnapshotFailureTotal,\n\t\t\t\t\tHelp: \"Total number of CSI failed volume snapshots\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel, backupNameLabel},\n\t\t\t),\n\t\t\tscheduleExpectedIntervalSeconds: prometheus.NewGaugeVec(\n\t\t\t\tprometheus.GaugeOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: scheduleExpectedIntervalSeconds,\n\t\t\t\t\tHelp: \"Expected interval between consecutive scheduled backups, in seconds\",\n\t\t\t\t},\n\t\t\t\t[]string{scheduleLabel},\n\t\t\t),\n\t\t\trepoMaintenanceSuccessTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: repoMaintenanceSuccessTotal,\n\t\t\t\t\tHelp: \"Total number of successful repo maintenance jobs\",\n\t\t\t\t},\n\t\t\t\t[]string{repositoryNameLabel},\n\t\t\t),\n\t\t\trepoMaintenanceFailureTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: repoMaintenanceFailureTotal,\n\t\t\t\t\tHelp: \"Total number of failed repo maintenance jobs\",\n\t\t\t\t},\n\t\t\t\t[]string{repositoryNameLabel},\n\t\t\t),\n\t\t\trepoMaintenanceDurationSeconds: prometheus.NewHistogramVec(\n\t\t\t\tprometheus.HistogramOpts{\n\t\t\t\t\tNamespace: metricNamespace,\n\t\t\t\t\tName: repoMaintenanceDurationSeconds,\n\t\t\t\t\tHelp: \"Time taken to complete repo maintenance jobs, in seconds\",\n\t\t\t\t\tBuckets: []float64{\n\t\t\t\t\t\ttoSeconds(1 * time.Minute),\n\t\t\t\t\t\ttoSeconds(5 * time.Minute),\n\t\t\t\t\t\ttoSeconds(10 * time.Minute),\n\t\t\t\t\t\ttoSeconds(15 * time.Minute),\n\t\t\t\t\t\ttoSeconds(30 * time.Minute),\n\t\t\t\t\t\ttoSeconds(1 * time.Hour),\n\t\t\t\t\t\ttoSeconds(2 * time.Hour),\n\t\t\t\t\t\ttoSeconds(3 * time.Hour),\n\t\t\t\t\t\ttoSeconds(4 * time.Hour),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t[]string{repositoryNameLabel},\n\t\t\t),\n\t\t},\n\t}\n}\n\nfunc NewNodeMetrics() *ServerMetrics {\n\treturn &ServerMetrics{\n\t\tmetrics: map[string]prometheus.Collector{\n\t\t\tpodVolumeBackupEnqueueTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: podVolumeMetricsNamespace,\n\t\t\t\t\tName: podVolumeBackupEnqueueTotal,\n\t\t\t\t\tHelp: \"Total number of pod_volume_backup objects enqueued\",\n\t\t\t\t},\n\t\t\t\t[]string{nodeMetricLabel},\n\t\t\t),\n\t\t\tpodVolumeBackupDequeueTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: podVolumeMetricsNamespace,\n\t\t\t\t\tName: podVolumeBackupDequeueTotal,\n\t\t\t\t\tHelp: \"Total number of pod_volume_backup objects dequeued\",\n\t\t\t\t},\n\t\t\t\t[]string{nodeMetricLabel},\n\t\t\t),\n\t\t\tpodVolumeOperationLatencyGaugeSeconds: prometheus.NewGaugeVec(\n\t\t\t\tprometheus.GaugeOpts{\n\t\t\t\t\tNamespace: podVolumeMetricsNamespace,\n\t\t\t\t\tName: podVolumeOperationLatencyGaugeSeconds,\n\t\t\t\t\tHelp: \"Gauge metric indicating time taken, in seconds, to perform pod volume operations\",\n\t\t\t\t},\n\t\t\t\t[]string{nodeMetricLabel, podVolumeOperationLabel, backupNameLabel, pvbNameLabel},\n\t\t\t),\n\t\t\tpodVolumeOperationLatencySeconds: prometheus.NewHistogramVec(\n\t\t\t\tprometheus.HistogramOpts{\n\t\t\t\t\tNamespace: podVolumeMetricsNamespace,\n\t\t\t\t\tName: podVolumeOperationLatencySeconds,\n\t\t\t\t\tHelp: \"Time taken to complete pod volume operations, in seconds\",\n\t\t\t\t\tBuckets: []float64{\n\t\t\t\t\t\ttoSeconds(1 * time.Minute),\n\t\t\t\t\t\ttoSeconds(5 * time.Minute),\n\t\t\t\t\t\ttoSeconds(10 * time.Minute),\n\t\t\t\t\t\ttoSeconds(15 * time.Minute),\n\t\t\t\t\t\ttoSeconds(30 * time.Minute),\n\t\t\t\t\t\ttoSeconds(1 * time.Hour),\n\t\t\t\t\t\ttoSeconds(2 * time.Hour),\n\t\t\t\t\t\ttoSeconds(3 * time.Hour),\n\t\t\t\t\t\ttoSeconds(4 * time.Hour),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t[]string{nodeMetricLabel, podVolumeOperationLabel, backupNameLabel, pvbNameLabel},\n\t\t\t),\n\t\t\tDataUploadSuccessTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: podVolumeMetricsNamespace,\n\t\t\t\t\tName: DataUploadSuccessTotal,\n\t\t\t\t\tHelp: \"Total number of successful uploaded snapshots\",\n\t\t\t\t},\n\t\t\t\t[]string{nodeMetricLabel},\n\t\t\t),\n\t\t\tDataUploadFailureTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: podVolumeMetricsNamespace,\n\t\t\t\t\tName: DataUploadFailureTotal,\n\t\t\t\t\tHelp: \"Total number of failed uploaded snapshots\",\n\t\t\t\t},\n\t\t\t\t[]string{nodeMetricLabel},\n\t\t\t),\n\t\t\tDataUploadCancelTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: podVolumeMetricsNamespace,\n\t\t\t\t\tName: DataUploadCancelTotal,\n\t\t\t\t\tHelp: \"Total number of canceled uploaded snapshots\",\n\t\t\t\t},\n\t\t\t\t[]string{nodeMetricLabel},\n\t\t\t),\n\t\t\tDataDownloadSuccessTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: podVolumeMetricsNamespace,\n\t\t\t\t\tName: DataDownloadSuccessTotal,\n\t\t\t\t\tHelp: \"Total number of successful downloaded snapshots\",\n\t\t\t\t},\n\t\t\t\t[]string{nodeMetricLabel},\n\t\t\t),\n\t\t\tDataDownloadFailureTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: podVolumeMetricsNamespace,\n\t\t\t\t\tName: DataDownloadFailureTotal,\n\t\t\t\t\tHelp: \"Total number of failed downloaded snapshots\",\n\t\t\t\t},\n\t\t\t\t[]string{nodeMetricLabel},\n\t\t\t),\n\t\t\tDataDownloadCancelTotal: prometheus.NewCounterVec(\n\t\t\t\tprometheus.CounterOpts{\n\t\t\t\t\tNamespace: podVolumeMetricsNamespace,\n\t\t\t\t\tName: DataDownloadCancelTotal,\n\t\t\t\t\tHelp: \"Total number of canceled downloaded snapshots\",\n\t\t\t\t},\n\t\t\t\t[]string{nodeMetricLabel},\n\t\t\t),\n\t\t},\n\t}\n}\n\n// RegisterAllMetrics registers all prometheus metrics.\nfunc (m *ServerMetrics) RegisterAllMetrics() {\n\tfor _, pm := range m.metrics {\n\t\tprometheus.MustRegister(pm)\n\t}\n}\n\n// InitSchedule initializes counter metrics of a schedule.\nfunc (m *ServerMetrics) InitSchedule(scheduleName string) {\n\tif c, ok := m.metrics[backupAttemptTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(scheduleName).Add(0)\n\t}\n\tif c, ok := m.metrics[backupSuccessTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(scheduleName).Add(0)\n\t}\n\tif c, ok := m.metrics[backupPartialFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(scheduleName).Add(0)\n\t}\n\tif c, ok := m.metrics[backupFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(scheduleName).Add(0)\n\t}\n\tif c, ok := m.metrics[backupValidationFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(scheduleName).Add(0)\n\t}\n\tif c, ok := m.metrics[backupDeletionAttemptTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(scheduleName).Add(0)\n\t}\n\tif c, ok := m.metrics[backupDeletionSuccessTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(scheduleName).Add(0)\n\t}\n\tif c, ok := m.metrics[backupDeletionFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(scheduleName).Add(0)\n\t}\n\tif c, ok := m.metrics[backupItemsTotalGauge].(*prometheus.GaugeVec); ok {\n\t\tc.WithLabelValues(scheduleName).Add(0)\n\t}\n\tif c, ok := m.metrics[backupItemsErrorsGauge].(*prometheus.GaugeVec); ok {\n\t\tc.WithLabelValues(scheduleName).Add(0)\n\t}\n\tif c, ok := m.metrics[backupWarningTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(scheduleName).Add(0)\n\t}\n\tif c, ok := m.metrics[backupLastStatus].(*prometheus.GaugeVec); ok {\n\t\tc.WithLabelValues(scheduleName).Set(float64(1))\n\t}\n\tif c, ok := m.metrics[restoreAttemptTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(scheduleName).Add(0)\n\t}\n\tif c, ok := m.metrics[restorePartialFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(scheduleName).Add(0)\n\t}\n\tif c, ok := m.metrics[restoreFailedTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(scheduleName).Add(0)\n\t}\n\tif c, ok := m.metrics[restoreSuccessTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(scheduleName).Add(0)\n\t}\n\tif c, ok := m.metrics[restoreValidationFailedTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(scheduleName).Add(0)\n\t}\n\tif c, ok := m.metrics[volumeSnapshotSuccessTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(scheduleName).Add(0)\n\t}\n\tif c, ok := m.metrics[volumeSnapshotAttemptTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(scheduleName).Add(0)\n\t}\n\tif c, ok := m.metrics[volumeSnapshotFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(scheduleName).Add(0)\n\t}\n\tif c, ok := m.metrics[csiSnapshotAttemptTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(scheduleName, \"\").Add(0)\n\t}\n\tif c, ok := m.metrics[csiSnapshotSuccessTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(scheduleName, \"\").Add(0)\n\t}\n\tif c, ok := m.metrics[csiSnapshotFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(scheduleName, \"\").Add(0)\n\t}\n}\n\n// RemoveSchedule removes metrics associated with a specified schedule.\nfunc (m *ServerMetrics) RemoveSchedule(scheduleName string) {\n\tif g, ok := m.metrics[backupTarballSizeBytesGauge].(*prometheus.GaugeVec); ok {\n\t\tg.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[backupAttemptTotal].(*prometheus.CounterVec); ok {\n\t\tc.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[backupSuccessTotal].(*prometheus.CounterVec); ok {\n\t\tc.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[backupPartialFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[backupFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[backupValidationFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.DeleteLabelValues(scheduleName)\n\t}\n\tif h, ok := m.metrics[backupDurationSeconds].(*prometheus.HistogramVec); ok {\n\t\th.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[backupDeletionAttemptTotal].(*prometheus.CounterVec); ok {\n\t\tc.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[backupDeletionSuccessTotal].(*prometheus.CounterVec); ok {\n\t\tc.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[backupDeletionFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.DeleteLabelValues(scheduleName)\n\t}\n\tif g, ok := m.metrics[backupLastSuccessfulTimestamp].(*prometheus.GaugeVec); ok {\n\t\tg.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[backupItemsTotalGauge].(*prometheus.GaugeVec); ok {\n\t\tc.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[backupItemsErrorsGauge].(*prometheus.GaugeVec); ok {\n\t\tc.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[backupWarningTotal].(*prometheus.CounterVec); ok {\n\t\tc.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[backupLastStatus].(*prometheus.GaugeVec); ok {\n\t\tc.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[restoreAttemptTotal].(*prometheus.CounterVec); ok {\n\t\tc.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[restorePartialFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[restoreFailedTotal].(*prometheus.CounterVec); ok {\n\t\tc.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[restoreSuccessTotal].(*prometheus.CounterVec); ok {\n\t\tc.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[restoreValidationFailedTotal].(*prometheus.CounterVec); ok {\n\t\tc.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[volumeSnapshotSuccessTotal].(*prometheus.CounterVec); ok {\n\t\tc.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[volumeSnapshotAttemptTotal].(*prometheus.CounterVec); ok {\n\t\tc.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[volumeSnapshotFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.DeleteLabelValues(scheduleName)\n\t}\n\tif c, ok := m.metrics[csiSnapshotAttemptTotal].(*prometheus.CounterVec); ok {\n\t\tc.DeleteLabelValues(scheduleName, \"\")\n\t}\n\tif c, ok := m.metrics[csiSnapshotSuccessTotal].(*prometheus.CounterVec); ok {\n\t\tc.DeleteLabelValues(scheduleName, \"\")\n\t}\n\tif c, ok := m.metrics[csiSnapshotFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.DeleteLabelValues(scheduleName, \"\")\n\t}\n\tif g, ok := m.metrics[scheduleExpectedIntervalSeconds].(*prometheus.GaugeVec); ok {\n\t\tg.DeleteLabelValues(scheduleName)\n\t}\n}\n\n// InitMetricsForNode initializes counter metrics for a node.\nfunc (m *ServerMetrics) InitMetricsForNode(node string) {\n\tif c, ok := m.metrics[podVolumeBackupEnqueueTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(node).Add(0)\n\t}\n\tif c, ok := m.metrics[podVolumeBackupDequeueTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(node).Add(0)\n\t}\n\tif c, ok := m.metrics[DataUploadSuccessTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(node).Add(0)\n\t}\n\tif c, ok := m.metrics[DataUploadFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(node).Add(0)\n\t}\n\tif c, ok := m.metrics[DataUploadCancelTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(node).Add(0)\n\t}\n\tif c, ok := m.metrics[DataDownloadSuccessTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(node).Add(0)\n\t}\n\tif c, ok := m.metrics[DataDownloadFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(node).Add(0)\n\t}\n\tif c, ok := m.metrics[DataDownloadCancelTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(node).Add(0)\n\t}\n}\n\n// RegisterPodVolumeBackupEnqueue records enqueuing of a PodVolumeBackup object.\nfunc (m *ServerMetrics) RegisterPodVolumeBackupEnqueue(node string) {\n\tif c, ok := m.metrics[podVolumeBackupEnqueueTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(node).Inc()\n\t}\n}\n\n// RegisterPodVolumeBackupDequeue records dequeuing of a PodVolumeBackup object.\nfunc (m *ServerMetrics) RegisterPodVolumeBackupDequeue(node string) {\n\tif c, ok := m.metrics[podVolumeBackupDequeueTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(node).Inc()\n\t}\n}\n\n// RegisterDataUploadSuccess records successful uploaded snapshots.\nfunc (m *ServerMetrics) RegisterDataUploadSuccess(node string) {\n\tif c, ok := m.metrics[DataUploadSuccessTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(node).Inc()\n\t}\n}\n\n// RegisterDataUploadFailure records failed uploaded snapshots.\nfunc (m *ServerMetrics) RegisterDataUploadFailure(node string) {\n\tif c, ok := m.metrics[DataUploadFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(node).Inc()\n\t}\n}\n\n// RegisterDataUploadCancel records canceled uploaded snapshots.\nfunc (m *ServerMetrics) RegisterDataUploadCancel(node string) {\n\tif c, ok := m.metrics[DataUploadCancelTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(node).Inc()\n\t}\n}\n\n// RegisterDataDownloadSuccess records successful downloaded snapshots.\nfunc (m *ServerMetrics) RegisterDataDownloadSuccess(node string) {\n\tif c, ok := m.metrics[DataDownloadSuccessTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(node).Inc()\n\t}\n}\n\n// RegisterDataDownloadFailure records failed downloaded snapshots.\nfunc (m *ServerMetrics) RegisterDataDownloadFailure(node string) {\n\tif c, ok := m.metrics[DataDownloadFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(node).Inc()\n\t}\n}\n\n// RegisterDataDownloadCancel records canceled downloaded snapshots.\nfunc (m *ServerMetrics) RegisterDataDownloadCancel(node string) {\n\tif c, ok := m.metrics[DataDownloadCancelTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(node).Inc()\n\t}\n}\n\n// ObservePodVolumeOpLatency records the number of seconds a pod volume operation took.\nfunc (m *ServerMetrics) ObservePodVolumeOpLatency(node, pvbName, opName, backupName string, seconds float64) {\n\tif h, ok := m.metrics[podVolumeOperationLatencySeconds].(*prometheus.HistogramVec); ok {\n\t\th.WithLabelValues(node, opName, backupName, pvbName).Observe(seconds)\n\t}\n}\n\n// RegisterPodVolumeOpLatencyGauge registers the pod volume operation latency as a gauge metric.\nfunc (m *ServerMetrics) RegisterPodVolumeOpLatencyGauge(node, pvbName, opName, backupName string, seconds float64) {\n\tif g, ok := m.metrics[podVolumeOperationLatencyGaugeSeconds].(*prometheus.GaugeVec); ok {\n\t\tg.WithLabelValues(node, opName, backupName, pvbName).Set(seconds)\n\t}\n}\n\n// SetBackupTarballSizeBytesGauge records the size, in bytes, of a backup tarball.\nfunc (m *ServerMetrics) SetBackupTarballSizeBytesGauge(backupSchedule string, size int64) {\n\tif g, ok := m.metrics[backupTarballSizeBytesGauge].(*prometheus.GaugeVec); ok {\n\t\tg.WithLabelValues(backupSchedule).Set(float64(size))\n\t}\n}\n\n// SetBackupLastSuccessfulTimestamp records the last time a backup ran successfully, Unix timestamp in seconds\nfunc (m *ServerMetrics) SetBackupLastSuccessfulTimestamp(backupSchedule string, time time.Time) {\n\tif g, ok := m.metrics[backupLastSuccessfulTimestamp].(*prometheus.GaugeVec); ok {\n\t\tg.WithLabelValues(backupSchedule).Set(float64(time.Unix()))\n\t}\n}\n\n// SetScheduleExpectedIntervalSeconds records the expected interval in seconds,\n// between consecutive backups for a schedule.\nfunc (m *ServerMetrics) SetScheduleExpectedIntervalSeconds(scheduleName string, seconds float64) {\n\tif g, ok := m.metrics[scheduleExpectedIntervalSeconds].(*prometheus.GaugeVec); ok {\n\t\tg.WithLabelValues(scheduleName).Set(seconds)\n\t}\n}\n\n// SetBackupTotal records the current number of existent backups.\nfunc (m *ServerMetrics) SetBackupTotal(numberOfBackups int64) {\n\tif g, ok := m.metrics[backupTotal].(prometheus.Gauge); ok {\n\t\tg.Set(float64(numberOfBackups))\n\t}\n}\n\n// RegisterBackupAttempt records an backup attempt.\nfunc (m *ServerMetrics) RegisterBackupAttempt(backupSchedule string) {\n\tif c, ok := m.metrics[backupAttemptTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(backupSchedule).Inc()\n\t}\n}\n\n// RegisterBackupSuccess records a successful completion of a backup.\nfunc (m *ServerMetrics) RegisterBackupSuccess(backupSchedule string) {\n\tif c, ok := m.metrics[backupSuccessTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(backupSchedule).Inc()\n\t}\n\tm.SetBackupLastSuccessfulTimestamp(backupSchedule, time.Now())\n}\n\n// RegisterBackupPartialFailure records a partially failed backup.\nfunc (m *ServerMetrics) RegisterBackupPartialFailure(backupSchedule string) {\n\tif c, ok := m.metrics[backupPartialFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(backupSchedule).Inc()\n\t}\n}\n\n// RegisterBackupFailed records a failed backup.\nfunc (m *ServerMetrics) RegisterBackupFailed(backupSchedule string) {\n\tif c, ok := m.metrics[backupFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(backupSchedule).Inc()\n\t}\n}\n\n// RegisterBackupValidationFailure records a validation failed backup.\nfunc (m *ServerMetrics) RegisterBackupValidationFailure(backupSchedule string) {\n\tif c, ok := m.metrics[backupValidationFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(backupSchedule).Inc()\n\t}\n}\n\n// RegisterBackupDuration records the number of seconds a backup took.\nfunc (m *ServerMetrics) RegisterBackupDuration(backupSchedule string, seconds float64) {\n\tif c, ok := m.metrics[backupDurationSeconds].(*prometheus.HistogramVec); ok {\n\t\tc.WithLabelValues(backupSchedule).Observe(seconds)\n\t}\n}\n\n// RegisterBackupDeletionAttempt records the number of attempted backup deletions\nfunc (m *ServerMetrics) RegisterBackupDeletionAttempt(backupSchedule string) {\n\tif c, ok := m.metrics[backupDeletionAttemptTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(backupSchedule).Inc()\n\t}\n}\n\n// RegisterBackupDeletionFailed records the number of failed backup deletions\nfunc (m *ServerMetrics) RegisterBackupDeletionFailed(backupSchedule string) {\n\tif c, ok := m.metrics[backupDeletionFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(backupSchedule).Inc()\n\t}\n}\n\n// RegisterBackupDeletionSuccess records the number of successful backup deletions\nfunc (m *ServerMetrics) RegisterBackupDeletionSuccess(backupSchedule string) {\n\tif c, ok := m.metrics[backupDeletionSuccessTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(backupSchedule).Inc()\n\t}\n}\n\n// RegisterBackupItemsTotalGauge records the number of items to be backed up.\nfunc (m *ServerMetrics) RegisterBackupItemsTotalGauge(backupSchedule string, items int) {\n\tif c, ok := m.metrics[backupItemsTotalGauge].(*prometheus.GaugeVec); ok {\n\t\tc.WithLabelValues(backupSchedule).Set(float64(items))\n\t}\n}\n\n// RegisterBackupItemsErrorsGauge records the number of all error messages that were generated during\n// execution of the backup.\nfunc (m *ServerMetrics) RegisterBackupItemsErrorsGauge(backupSchedule string, items int) {\n\tif c, ok := m.metrics[backupItemsErrorsGauge].(*prometheus.GaugeVec); ok {\n\t\tc.WithLabelValues(backupSchedule).Set(float64(items))\n\t}\n}\n\n// RegisterBackupWarning records a warned backup.\nfunc (m *ServerMetrics) RegisterBackupWarning(backupSchedule string) {\n\tif c, ok := m.metrics[backupWarningTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(backupSchedule).Inc()\n\t}\n}\n\n// RegisterBackupLastStatus records the last status of the backup.\nfunc (m *ServerMetrics) RegisterBackupLastStatus(backupSchedule string, lastStatus int64) {\n\tif g, ok := m.metrics[backupLastStatus].(*prometheus.GaugeVec); ok {\n\t\tg.WithLabelValues(backupSchedule).Set(float64(lastStatus))\n\t}\n}\n\n// toSeconds translates a time.Duration value into a float64\n// representing the number of seconds in that duration.\nfunc toSeconds(d time.Duration) float64 {\n\treturn float64(d / time.Second)\n}\n\n// SetRestoreTotal records the current number of existent restores.\nfunc (m *ServerMetrics) SetRestoreTotal(numberOfRestores int64) {\n\tif g, ok := m.metrics[restoreTotal].(prometheus.Gauge); ok {\n\t\tg.Set(float64(numberOfRestores))\n\t}\n}\n\n// RegisterRestoreAttempt records an attempt to restore a backup.\nfunc (m *ServerMetrics) RegisterRestoreAttempt(backupSchedule string) {\n\tif c, ok := m.metrics[restoreAttemptTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(backupSchedule).Inc()\n\t}\n}\n\n// RegisterRestoreSuccess records a successful (maybe partial) completion of a restore.\nfunc (m *ServerMetrics) RegisterRestoreSuccess(backupSchedule string) {\n\tif c, ok := m.metrics[restoreSuccessTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(backupSchedule).Inc()\n\t}\n}\n\n// RegisterRestorePartialFailure records a restore that partially failed.\nfunc (m *ServerMetrics) RegisterRestorePartialFailure(backupSchedule string) {\n\tif c, ok := m.metrics[restorePartialFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(backupSchedule).Inc()\n\t}\n}\n\n// RegisterRestoreFailed records a restore that failed.\nfunc (m *ServerMetrics) RegisterRestoreFailed(backupSchedule string) {\n\tif c, ok := m.metrics[restoreFailedTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(backupSchedule).Inc()\n\t}\n}\n\n// RegisterRestoreValidationFailed records a restore that failed validation.\nfunc (m *ServerMetrics) RegisterRestoreValidationFailed(backupSchedule string) {\n\tif c, ok := m.metrics[restoreValidationFailedTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(backupSchedule).Inc()\n\t}\n}\n\n// RegisterVolumeSnapshotAttempts records an attempt to snapshot a volume.\nfunc (m *ServerMetrics) RegisterVolumeSnapshotAttempts(backupSchedule string, volumeSnapshotsAttempted int) {\n\tif c, ok := m.metrics[volumeSnapshotAttemptTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(backupSchedule).Add(float64(volumeSnapshotsAttempted))\n\t}\n}\n\n// RegisterVolumeSnapshotSuccesses records a completed volume snapshot.\nfunc (m *ServerMetrics) RegisterVolumeSnapshotSuccesses(backupSchedule string, volumeSnapshotsCompleted int) {\n\tif c, ok := m.metrics[volumeSnapshotSuccessTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(backupSchedule).Add(float64(volumeSnapshotsCompleted))\n\t}\n}\n\n// RegisterVolumeSnapshotFailures records a failed volume snapshot.\nfunc (m *ServerMetrics) RegisterVolumeSnapshotFailures(backupSchedule string, volumeSnapshotsFailed int) {\n\tif c, ok := m.metrics[volumeSnapshotFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(backupSchedule).Add(float64(volumeSnapshotsFailed))\n\t}\n}\n\n// RegisterCSISnapshotAttempts records an attempt to snapshot a volume by CSI plugin.\nfunc (m *ServerMetrics) RegisterCSISnapshotAttempts(backupSchedule, backupName string, csiSnapshotsAttempted int) {\n\tif c, ok := m.metrics[csiSnapshotAttemptTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(backupSchedule, backupName).Add(float64(csiSnapshotsAttempted))\n\t}\n}\n\n// RegisterCSISnapshotSuccesses records a completed volume snapshot by CSI plugin.\nfunc (m *ServerMetrics) RegisterCSISnapshotSuccesses(backupSchedule, backupName string, csiSnapshotCompleted int) {\n\tif c, ok := m.metrics[csiSnapshotSuccessTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(backupSchedule, backupName).Add(float64(csiSnapshotCompleted))\n\t}\n}\n\n// RegisterCSISnapshotFailures records a failed volume snapshot by CSI plugin.\nfunc (m *ServerMetrics) RegisterCSISnapshotFailures(backupSchedule, backupName string, csiSnapshotsFailed int) {\n\tif c, ok := m.metrics[csiSnapshotFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(backupSchedule, backupName).Add(float64(csiSnapshotsFailed))\n\t}\n}\n\n// RegisterBackupLocationAvailable records the availability of a backup location.\nfunc (m *ServerMetrics) RegisterBackupLocationAvailable(backupLocationName string) {\n\tif g, ok := m.metrics[backupLocationStatus].(*prometheus.GaugeVec); ok {\n\t\tg.WithLabelValues(backupLocationName).Set(float64(1))\n\t}\n}\n\n// RegisterBackupLocationUnavailable records the availability of a backup location.\nfunc (m *ServerMetrics) RegisterBackupLocationUnavailable(backupLocationName string) {\n\tif g, ok := m.metrics[backupLocationStatus].(*prometheus.GaugeVec); ok {\n\t\tg.WithLabelValues(backupLocationName).Set(float64(0))\n\t}\n}\n\n// RegisterRepoMaintenanceSuccess records a successful repo maintenance job.\nfunc (m *ServerMetrics) RegisterRepoMaintenanceSuccess(repositoryName string) {\n\tif c, ok := m.metrics[repoMaintenanceSuccessTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(repositoryName).Inc()\n\t}\n}\n\n// RegisterRepoMaintenanceFailure records a failed repo maintenance job.\nfunc (m *ServerMetrics) RegisterRepoMaintenanceFailure(repositoryName string) {\n\tif c, ok := m.metrics[repoMaintenanceFailureTotal].(*prometheus.CounterVec); ok {\n\t\tc.WithLabelValues(repositoryName).Inc()\n\t}\n}\n\n// ObserveRepoMaintenanceDuration records the number of seconds a repo maintenance job took.\nfunc (m *ServerMetrics) ObserveRepoMaintenanceDuration(repositoryName string, seconds float64) {\n\tif h, ok := m.metrics[repoMaintenanceDurationSeconds].(*prometheus.HistogramVec); ok {\n\t\th.WithLabelValues(repositoryName).Observe(seconds)\n\t}\n}\n" }, { "path": "pkg/metrics/metrics_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage metrics\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/prometheus/client_golang/prometheus\"\n\tdto \"github.com/prometheus/client_model/go\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\n// TestBackupMetricsWithAdhocBackups verifies that metrics are properly recorded\n// for both scheduled and adhoc (non-scheduled) backups.\nfunc TestBackupMetricsWithAdhocBackups(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tscheduleName string\n\t\texpectedLabel string\n\t\tdescription string\n\t}{\n\t\t{\n\t\t\tname: \"scheduled backup metrics\",\n\t\t\tscheduleName: \"daily-backup\",\n\t\t\texpectedLabel: \"daily-backup\",\n\t\t\tdescription: \"Metrics should be recorded with the schedule name label\",\n\t\t},\n\t\t{\n\t\t\tname: \"adhoc backup metrics with empty schedule\",\n\t\t\tscheduleName: \"\",\n\t\t\texpectedLabel: \"\",\n\t\t\tdescription: \"Metrics should be recorded with empty schedule label for adhoc backups\",\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t// Create a new metrics instance\n\t\t\tm := NewServerMetrics()\n\n\t\t\t// Test backup attempt metric\n\t\t\tt.Run(\"RegisterBackupAttempt\", func(t *testing.T) {\n\t\t\t\tm.RegisterBackupAttempt(tc.scheduleName)\n\n\t\t\t\tmetric := getMetricValue(t, m.metrics[backupAttemptTotal].(*prometheus.CounterVec), tc.expectedLabel)\n\t\t\t\tassert.Equal(t, float64(1), metric, tc.description)\n\t\t\t})\n\n\t\t\t// Test backup success metric\n\t\t\tt.Run(\"RegisterBackupSuccess\", func(t *testing.T) {\n\t\t\t\tm.RegisterBackupSuccess(tc.scheduleName)\n\n\t\t\t\tmetric := getMetricValue(t, m.metrics[backupSuccessTotal].(*prometheus.CounterVec), tc.expectedLabel)\n\t\t\t\tassert.Equal(t, float64(1), metric, tc.description)\n\t\t\t})\n\n\t\t\t// Test backup failure metric\n\t\t\tt.Run(\"RegisterBackupFailed\", func(t *testing.T) {\n\t\t\t\tm.RegisterBackupFailed(tc.scheduleName)\n\n\t\t\t\tmetric := getMetricValue(t, m.metrics[backupFailureTotal].(*prometheus.CounterVec), tc.expectedLabel)\n\t\t\t\tassert.Equal(t, float64(1), metric, tc.description)\n\t\t\t})\n\n\t\t\t// Test backup partial failure metric\n\t\t\tt.Run(\"RegisterBackupPartialFailure\", func(t *testing.T) {\n\t\t\t\tm.RegisterBackupPartialFailure(tc.scheduleName)\n\n\t\t\t\tmetric := getMetricValue(t, m.metrics[backupPartialFailureTotal].(*prometheus.CounterVec), tc.expectedLabel)\n\t\t\t\tassert.Equal(t, float64(1), metric, tc.description)\n\t\t\t})\n\n\t\t\t// Test backup validation failure metric\n\t\t\tt.Run(\"RegisterBackupValidationFailure\", func(t *testing.T) {\n\t\t\t\tm.RegisterBackupValidationFailure(tc.scheduleName)\n\n\t\t\t\tmetric := getMetricValue(t, m.metrics[backupValidationFailureTotal].(*prometheus.CounterVec), tc.expectedLabel)\n\t\t\t\tassert.Equal(t, float64(1), metric, tc.description)\n\t\t\t})\n\n\t\t\t// Test backup warning metric\n\t\t\tt.Run(\"RegisterBackupWarning\", func(t *testing.T) {\n\t\t\t\tm.RegisterBackupWarning(tc.scheduleName)\n\n\t\t\t\tmetric := getMetricValue(t, m.metrics[backupWarningTotal].(*prometheus.CounterVec), tc.expectedLabel)\n\t\t\t\tassert.Equal(t, float64(1), metric, tc.description)\n\t\t\t})\n\n\t\t\t// Test backup items total gauge\n\t\t\tt.Run(\"RegisterBackupItemsTotalGauge\", func(t *testing.T) {\n\t\t\t\tm.RegisterBackupItemsTotalGauge(tc.scheduleName, 100)\n\n\t\t\t\tmetric := getMetricValue(t, m.metrics[backupItemsTotalGauge].(*prometheus.GaugeVec), tc.expectedLabel)\n\t\t\t\tassert.Equal(t, float64(100), metric, tc.description)\n\t\t\t})\n\n\t\t\t// Test backup items errors gauge\n\t\t\tt.Run(\"RegisterBackupItemsErrorsGauge\", func(t *testing.T) {\n\t\t\t\tm.RegisterBackupItemsErrorsGauge(tc.scheduleName, 5)\n\n\t\t\t\tmetric := getMetricValue(t, m.metrics[backupItemsErrorsGauge].(*prometheus.GaugeVec), tc.expectedLabel)\n\t\t\t\tassert.Equal(t, float64(5), metric, tc.description)\n\t\t\t})\n\n\t\t\t// Test backup duration metric\n\t\t\tt.Run(\"RegisterBackupDuration\", func(t *testing.T) {\n\t\t\t\tm.RegisterBackupDuration(tc.scheduleName, 120.5)\n\n\t\t\t\t// For histogram, we check the count\n\t\t\t\tmetric := getHistogramCount(t, m.metrics[backupDurationSeconds].(*prometheus.HistogramVec), tc.expectedLabel)\n\t\t\t\tassert.Equal(t, uint64(1), metric, tc.description)\n\t\t\t})\n\n\t\t\t// Test backup last status metric\n\t\t\tt.Run(\"RegisterBackupLastStatus\", func(t *testing.T) {\n\t\t\t\tm.RegisterBackupLastStatus(tc.scheduleName, BackupLastStatusSucc)\n\n\t\t\t\tmetric := getMetricValue(t, m.metrics[backupLastStatus].(*prometheus.GaugeVec), tc.expectedLabel)\n\t\t\t\tassert.Equal(t, float64(BackupLastStatusSucc), metric, tc.description)\n\t\t\t})\n\n\t\t\t// Test backup tarball size metric\n\t\t\tt.Run(\"SetBackupTarballSizeBytesGauge\", func(t *testing.T) {\n\t\t\t\tm.SetBackupTarballSizeBytesGauge(tc.scheduleName, 1024*1024)\n\n\t\t\t\tmetric := getMetricValue(t, m.metrics[backupTarballSizeBytesGauge].(*prometheus.GaugeVec), tc.expectedLabel)\n\t\t\t\tassert.Equal(t, float64(1024*1024), metric, tc.description)\n\t\t\t})\n\n\t\t\t// Test backup last successful timestamp\n\t\t\tt.Run(\"SetBackupLastSuccessfulTimestamp\", func(t *testing.T) {\n\t\t\t\ttestTime := time.Now()\n\t\t\t\tm.SetBackupLastSuccessfulTimestamp(tc.scheduleName, testTime)\n\n\t\t\t\tmetric := getMetricValue(t, m.metrics[backupLastSuccessfulTimestamp].(*prometheus.GaugeVec), tc.expectedLabel)\n\t\t\t\tassert.Equal(t, float64(testTime.Unix()), metric, tc.description)\n\t\t\t})\n\t\t})\n\t}\n}\n\n// TestRestoreMetricsWithAdhocBackups verifies that restore metrics are properly recorded\n// for restores from both scheduled and adhoc backups.\nfunc TestRestoreMetricsWithAdhocBackups(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tscheduleName string\n\t\texpectedLabel string\n\t\tdescription string\n\t}{\n\t\t{\n\t\t\tname: \"restore from scheduled backup\",\n\t\t\tscheduleName: \"daily-backup\",\n\t\t\texpectedLabel: \"daily-backup\",\n\t\t\tdescription: \"Restore metrics should use the backup's schedule name\",\n\t\t},\n\t\t{\n\t\t\tname: \"restore from adhoc backup\",\n\t\t\tscheduleName: \"\",\n\t\t\texpectedLabel: \"\",\n\t\t\tdescription: \"Restore metrics should have empty schedule label for adhoc backup restores\",\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tm := NewServerMetrics()\n\n\t\t\t// Test restore attempt metric\n\t\t\tt.Run(\"RegisterRestoreAttempt\", func(t *testing.T) {\n\t\t\t\tm.RegisterRestoreAttempt(tc.scheduleName)\n\n\t\t\t\tmetric := getMetricValue(t, m.metrics[restoreAttemptTotal].(*prometheus.CounterVec), tc.expectedLabel)\n\t\t\t\tassert.Equal(t, float64(1), metric, tc.description)\n\t\t\t})\n\n\t\t\t// Test restore success metric\n\t\t\tt.Run(\"RegisterRestoreSuccess\", func(t *testing.T) {\n\t\t\t\tm.RegisterRestoreSuccess(tc.scheduleName)\n\n\t\t\t\tmetric := getMetricValue(t, m.metrics[restoreSuccessTotal].(*prometheus.CounterVec), tc.expectedLabel)\n\t\t\t\tassert.Equal(t, float64(1), metric, tc.description)\n\t\t\t})\n\n\t\t\t// Test restore failed metric\n\t\t\tt.Run(\"RegisterRestoreFailed\", func(t *testing.T) {\n\t\t\t\tm.RegisterRestoreFailed(tc.scheduleName)\n\n\t\t\t\tmetric := getMetricValue(t, m.metrics[restoreFailedTotal].(*prometheus.CounterVec), tc.expectedLabel)\n\t\t\t\tassert.Equal(t, float64(1), metric, tc.description)\n\t\t\t})\n\n\t\t\t// Test restore partial failure metric\n\t\t\tt.Run(\"RegisterRestorePartialFailure\", func(t *testing.T) {\n\t\t\t\tm.RegisterRestorePartialFailure(tc.scheduleName)\n\n\t\t\t\tmetric := getMetricValue(t, m.metrics[restorePartialFailureTotal].(*prometheus.CounterVec), tc.expectedLabel)\n\t\t\t\tassert.Equal(t, float64(1), metric, tc.description)\n\t\t\t})\n\n\t\t\t// Test restore validation failed metric\n\t\t\tt.Run(\"RegisterRestoreValidationFailed\", func(t *testing.T) {\n\t\t\t\tm.RegisterRestoreValidationFailed(tc.scheduleName)\n\n\t\t\t\tmetric := getMetricValue(t, m.metrics[restoreValidationFailedTotal].(*prometheus.CounterVec), tc.expectedLabel)\n\t\t\t\tassert.Equal(t, float64(1), metric, tc.description)\n\t\t\t})\n\t\t})\n\t}\n}\n\n// TestMultipleAdhocBackupsShareMetrics verifies that multiple adhoc backups\n// accumulate metrics under the same empty schedule label.\nfunc TestMultipleAdhocBackupsShareMetrics(t *testing.T) {\n\tm := NewServerMetrics()\n\n\t// Simulate multiple adhoc backup attempts\n\tfor i := 0; i < 5; i++ {\n\t\tm.RegisterBackupAttempt(\"\")\n\t}\n\n\t// Simulate some successes and failures\n\tm.RegisterBackupSuccess(\"\")\n\tm.RegisterBackupSuccess(\"\")\n\tm.RegisterBackupFailed(\"\")\n\tm.RegisterBackupPartialFailure(\"\")\n\tm.RegisterBackupValidationFailure(\"\")\n\n\t// Verify accumulated metrics\n\tattemptMetric := getMetricValue(t, m.metrics[backupAttemptTotal].(*prometheus.CounterVec), \"\")\n\tassert.Equal(t, float64(5), attemptMetric, \"All adhoc backup attempts should be counted together\")\n\n\tsuccessMetric := getMetricValue(t, m.metrics[backupSuccessTotal].(*prometheus.CounterVec), \"\")\n\tassert.Equal(t, float64(2), successMetric, \"All adhoc backup successes should be counted together\")\n\n\tfailureMetric := getMetricValue(t, m.metrics[backupFailureTotal].(*prometheus.CounterVec), \"\")\n\tassert.Equal(t, float64(1), failureMetric, \"All adhoc backup failures should be counted together\")\n\n\tpartialFailureMetric := getMetricValue(t, m.metrics[backupPartialFailureTotal].(*prometheus.CounterVec), \"\")\n\tassert.Equal(t, float64(1), partialFailureMetric, \"All adhoc partial failures should be counted together\")\n\n\tvalidationFailureMetric := getMetricValue(t, m.metrics[backupValidationFailureTotal].(*prometheus.CounterVec), \"\")\n\tassert.Equal(t, float64(1), validationFailureMetric, \"All adhoc validation failures should be counted together\")\n}\n\n// TestSetScheduleExpectedIntervalSeconds verifies that the expected interval metric\n// is properly recorded for schedules.\nfunc TestSetScheduleExpectedIntervalSeconds(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tscheduleName string\n\t\tintervalSeconds float64\n\t\tdescription string\n\t}{\n\t\t{\n\t\t\tname: \"every 5 minutes schedule\",\n\t\t\tscheduleName: \"frequent-backup\",\n\t\t\tintervalSeconds: 300,\n\t\t\tdescription: \"Expected interval should be 5m in seconds\",\n\t\t},\n\t\t{\n\t\t\tname: \"daily schedule\",\n\t\t\tscheduleName: \"daily-backup\",\n\t\t\tintervalSeconds: 86400,\n\t\t\tdescription: \"Expected interval should be 24h in seconds\",\n\t\t},\n\t\t{\n\t\t\tname: \"monthly schedule\",\n\t\t\tscheduleName: \"monthly-backup\",\n\t\t\tintervalSeconds: 2678400, // 31 days in seconds\n\t\t\tdescription: \"Expected interval should be 31 days in seconds\",\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tm := NewServerMetrics()\n\t\t\tm.SetScheduleExpectedIntervalSeconds(tc.scheduleName, tc.intervalSeconds)\n\n\t\t\tmetric := getMetricValue(t, m.metrics[scheduleExpectedIntervalSeconds].(*prometheus.GaugeVec), tc.scheduleName)\n\t\t\tassert.Equal(t, tc.intervalSeconds, metric, tc.description)\n\t\t})\n\t}\n}\n\n// TestScheduleExpectedIntervalNotInitializedByDefault verifies that the expected\n// interval metric is not initialized by InitSchedule, so it only appears for\n// schedules with a valid cron expression.\nfunc TestScheduleExpectedIntervalNotInitializedByDefault(t *testing.T) {\n\tm := NewServerMetrics()\n\tm.InitSchedule(\"test-schedule\")\n\n\t// The metric should not have any values after InitSchedule\n\tch := make(chan prometheus.Metric, 1)\n\tm.metrics[scheduleExpectedIntervalSeconds].(*prometheus.GaugeVec).Collect(ch)\n\tclose(ch)\n\n\tcount := 0\n\tfor range ch {\n\t\tcount++\n\t}\n\tassert.Equal(t, 0, count, \"scheduleExpectedIntervalSeconds should not be initialized by InitSchedule\")\n}\n\n// TestRemoveScheduleCleansUpExpectedInterval verifies that RemoveSchedule\n// cleans up the expected interval metric.\nfunc TestRemoveScheduleCleansUpExpectedInterval(t *testing.T) {\n\tm := NewServerMetrics()\n\tm.InitSchedule(\"test-schedule\")\n\tm.SetScheduleExpectedIntervalSeconds(\"test-schedule\", 3600)\n\n\t// Verify metric exists\n\tmetric := getMetricValue(t, m.metrics[scheduleExpectedIntervalSeconds].(*prometheus.GaugeVec), \"test-schedule\")\n\tassert.Equal(t, float64(3600), metric)\n\n\t// Remove schedule and verify metric is cleaned up\n\tm.RemoveSchedule(\"test-schedule\")\n\n\tch := make(chan prometheus.Metric, 1)\n\tm.metrics[scheduleExpectedIntervalSeconds].(*prometheus.GaugeVec).Collect(ch)\n\tclose(ch)\n\n\tcount := 0\n\tfor range ch {\n\t\tcount++\n\t}\n\tassert.Equal(t, 0, count, \"scheduleExpectedIntervalSeconds should be removed after RemoveSchedule\")\n}\n\n// TestInitScheduleWithEmptyName verifies that InitSchedule works correctly\n// with an empty schedule name (for adhoc backups).\nfunc TestInitScheduleWithEmptyName(t *testing.T) {\n\tm := NewServerMetrics()\n\n\t// Initialize metrics for empty schedule (adhoc backups)\n\tm.InitSchedule(\"\")\n\n\t// Verify all metrics are initialized with 0\n\tmetrics := []string{\n\t\tbackupAttemptTotal,\n\t\tbackupSuccessTotal,\n\t\tbackupPartialFailureTotal,\n\t\tbackupFailureTotal,\n\t\tbackupValidationFailureTotal,\n\t\tbackupDeletionAttemptTotal,\n\t\tbackupDeletionSuccessTotal,\n\t\tbackupDeletionFailureTotal,\n\t\tbackupItemsTotalGauge,\n\t\tbackupItemsErrorsGauge,\n\t\tbackupWarningTotal,\n\t\trestoreAttemptTotal,\n\t\trestorePartialFailureTotal,\n\t\trestoreFailedTotal,\n\t\trestoreSuccessTotal,\n\t\trestoreValidationFailedTotal,\n\t\tvolumeSnapshotSuccessTotal,\n\t\tvolumeSnapshotAttemptTotal,\n\t\tvolumeSnapshotFailureTotal,\n\t}\n\n\tfor _, metricName := range metrics {\n\t\tt.Run(metricName, func(t *testing.T) {\n\t\t\tvar value float64\n\t\t\tswitch vec := m.metrics[metricName].(type) {\n\t\t\tcase *prometheus.CounterVec:\n\t\t\t\tvalue = getMetricValue(t, vec, \"\")\n\t\t\tcase *prometheus.GaugeVec:\n\t\t\t\tvalue = getMetricValue(t, vec, \"\")\n\t\t\t}\n\t\t\tassert.Equal(t, float64(0), value, \"Metric %s should be initialized to 0 for empty schedule\", metricName)\n\t\t})\n\t}\n\n\t// Special case: backupLastStatus should be initialized to 1 (success)\n\tlastStatusValue := getMetricValue(t, m.metrics[backupLastStatus].(*prometheus.GaugeVec), \"\")\n\tassert.Equal(t, float64(1), lastStatusValue, \"backupLastStatus should be initialized to 1 for empty schedule\")\n}\n\n// Helper function to get metric value from a CounterVec or GaugeVec\nfunc getMetricValue(t *testing.T, vec prometheus.Collector, scheduleLabel string) float64 {\n\tt.Helper()\n\tch := make(chan prometheus.Metric, 1)\n\tvec.Collect(ch)\n\tclose(ch)\n\n\tfor metric := range ch {\n\t\tdto := &dto.Metric{}\n\t\terr := metric.Write(dto)\n\t\trequire.NoError(t, err)\n\n\t\t// Check if this metric has the expected schedule label\n\t\thasCorrectLabel := false\n\t\tfor _, label := range dto.Label {\n\t\t\tif *label.Name == \"schedule\" && *label.Value == scheduleLabel {\n\t\t\t\thasCorrectLabel = true\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\n\t\tif hasCorrectLabel {\n\t\t\tif dto.Counter != nil {\n\t\t\t\treturn *dto.Counter.Value\n\t\t\t}\n\t\t\tif dto.Gauge != nil {\n\t\t\t\treturn *dto.Gauge.Value\n\t\t\t}\n\t\t}\n\t}\n\n\tt.Fatalf(\"Metric with schedule label '%s' not found\", scheduleLabel)\n\treturn 0\n}\n\n// Helper function to get histogram count\nfunc getHistogramCount(t *testing.T, vec *prometheus.HistogramVec, scheduleLabel string) uint64 {\n\tt.Helper()\n\tch := make(chan prometheus.Metric, 1)\n\tvec.Collect(ch)\n\tclose(ch)\n\n\tfor metric := range ch {\n\t\tdto := &dto.Metric{}\n\t\terr := metric.Write(dto)\n\t\trequire.NoError(t, err)\n\n\t\t// Check if this metric has the expected schedule label\n\t\thasCorrectLabel := false\n\t\tfor _, label := range dto.Label {\n\t\t\tif *label.Name == \"schedule\" && *label.Value == scheduleLabel {\n\t\t\t\thasCorrectLabel = true\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\n\t\tif hasCorrectLabel && dto.Histogram != nil {\n\t\t\treturn *dto.Histogram.SampleCount\n\t\t}\n\t}\n\n\tt.Fatalf(\"Histogram with schedule label '%s' not found\", scheduleLabel)\n\treturn 0\n}\n\n// TestRepoMaintenanceMetrics verifies that repo maintenance metrics are properly recorded.\nfunc TestRepoMaintenanceMetrics(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\trepositoryName string\n\t\tdescription string\n\t}{\n\t\t{\n\t\t\tname: \"maintenance job metrics for repository\",\n\t\t\trepositoryName: \"default-restic-abcd\",\n\t\t\tdescription: \"Metrics should be recorded with the repository name label\",\n\t\t},\n\t\t{\n\t\t\tname: \"maintenance job metrics for different repository\",\n\t\t\trepositoryName: \"velero-backup-repo-xyz\",\n\t\t\tdescription: \"Metrics should be recorded with different repository name\",\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tm := NewServerMetrics()\n\n\t\t\t// Test repo maintenance success metric\n\t\t\tt.Run(\"RegisterRepoMaintenanceSuccess\", func(t *testing.T) {\n\t\t\t\tm.RegisterRepoMaintenanceSuccess(tc.repositoryName)\n\n\t\t\t\tmetric := getMaintenanceMetricValue(t, m.metrics[repoMaintenanceSuccessTotal].(*prometheus.CounterVec), tc.repositoryName)\n\t\t\t\tassert.Equal(t, float64(1), metric, tc.description)\n\t\t\t})\n\n\t\t\t// Test repo maintenance failure metric\n\t\t\tt.Run(\"RegisterRepoMaintenanceFailure\", func(t *testing.T) {\n\t\t\t\tm.RegisterRepoMaintenanceFailure(tc.repositoryName)\n\n\t\t\t\tmetric := getMaintenanceMetricValue(t, m.metrics[repoMaintenanceFailureTotal].(*prometheus.CounterVec), tc.repositoryName)\n\t\t\t\tassert.Equal(t, float64(1), metric, tc.description)\n\t\t\t})\n\n\t\t\t// Test repo maintenance duration metric\n\t\t\tt.Run(\"ObserveRepoMaintenanceDuration\", func(t *testing.T) {\n\t\t\t\tm.ObserveRepoMaintenanceDuration(tc.repositoryName, 300.5)\n\n\t\t\t\t// For histogram, we check the count\n\t\t\t\tmetric := getMaintenanceHistogramCount(t, m.metrics[repoMaintenanceDurationSeconds].(*prometheus.HistogramVec), tc.repositoryName)\n\t\t\t\tassert.Equal(t, uint64(1), metric, tc.description)\n\t\t\t})\n\t\t})\n\t}\n}\n\n// TestMultipleRepoMaintenanceJobsAccumulate verifies that multiple repo maintenance jobs\n// accumulate metrics under the same repository label.\nfunc TestMultipleRepoMaintenanceJobsAccumulate(t *testing.T) {\n\tm := NewServerMetrics()\n\trepoName := \"default-restic-test\"\n\n\t// Simulate multiple repo maintenance job executions\n\tm.RegisterRepoMaintenanceSuccess(repoName)\n\tm.RegisterRepoMaintenanceSuccess(repoName)\n\tm.RegisterRepoMaintenanceSuccess(repoName)\n\tm.RegisterRepoMaintenanceFailure(repoName)\n\tm.RegisterRepoMaintenanceFailure(repoName)\n\n\t// Record multiple durations\n\tm.ObserveRepoMaintenanceDuration(repoName, 120.5)\n\tm.ObserveRepoMaintenanceDuration(repoName, 180.3)\n\tm.ObserveRepoMaintenanceDuration(repoName, 90.7)\n\n\t// Verify accumulated metrics\n\tsuccessMetric := getMaintenanceMetricValue(t, m.metrics[repoMaintenanceSuccessTotal].(*prometheus.CounterVec), repoName)\n\tassert.Equal(t, float64(3), successMetric, \"All repo maintenance successes should be counted\")\n\n\tfailureMetric := getMaintenanceMetricValue(t, m.metrics[repoMaintenanceFailureTotal].(*prometheus.CounterVec), repoName)\n\tassert.Equal(t, float64(2), failureMetric, \"All repo maintenance failures should be counted\")\n\n\tdurationCount := getMaintenanceHistogramCount(t, m.metrics[repoMaintenanceDurationSeconds].(*prometheus.HistogramVec), repoName)\n\tassert.Equal(t, uint64(3), durationCount, \"All repo maintenance durations should be observed\")\n}\n\n// Helper function to get metric value from a CounterVec with repository_name label\nfunc getMaintenanceMetricValue(t *testing.T, vec prometheus.Collector, repositoryName string) float64 {\n\tt.Helper()\n\tch := make(chan prometheus.Metric, 1)\n\tvec.Collect(ch)\n\tclose(ch)\n\n\tfor metric := range ch {\n\t\tdto := &dto.Metric{}\n\t\terr := metric.Write(dto)\n\t\trequire.NoError(t, err)\n\n\t\t// Check if this metric has the expected repository_name label\n\t\thasCorrectLabel := false\n\t\tfor _, label := range dto.Label {\n\t\t\tif *label.Name == \"repository_name\" && *label.Value == repositoryName {\n\t\t\t\thasCorrectLabel = true\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\n\t\tif hasCorrectLabel {\n\t\t\tif dto.Counter != nil {\n\t\t\t\treturn *dto.Counter.Value\n\t\t\t}\n\t\t\tif dto.Gauge != nil {\n\t\t\t\treturn *dto.Gauge.Value\n\t\t\t}\n\t\t}\n\t}\n\n\tt.Fatalf(\"Metric with repository_name label '%s' not found\", repositoryName)\n\treturn 0\n}\n\n// Helper function to get histogram count with repository_name label\nfunc getMaintenanceHistogramCount(t *testing.T, vec *prometheus.HistogramVec, repositoryName string) uint64 {\n\tt.Helper()\n\tch := make(chan prometheus.Metric, 1)\n\tvec.Collect(ch)\n\tclose(ch)\n\n\tfor metric := range ch {\n\t\tdto := &dto.Metric{}\n\t\terr := metric.Write(dto)\n\t\trequire.NoError(t, err)\n\n\t\t// Check if this metric has the expected repository_name label\n\t\thasCorrectLabel := false\n\t\tfor _, label := range dto.Label {\n\t\t\tif *label.Name == \"repository_name\" && *label.Value == repositoryName {\n\t\t\t\thasCorrectLabel = true\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\n\t\tif hasCorrectLabel && dto.Histogram != nil {\n\t\t\treturn *dto.Histogram.SampleCount\n\t\t}\n\t}\n\n\tt.Fatalf(\"Histogram with repository_name label '%s' not found\", repositoryName)\n\treturn 0\n}\n" }, { "path": "pkg/nodeagent/node_agent.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage nodeagent\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\n\t\"github.com/pkg/errors\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/client-go/kubernetes\"\n\tctrlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerotypes \"github.com/vmware-tanzu/velero/pkg/types\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nconst (\n\t// daemonSet is the name of the Velero node agent daemonset on linux nodes.\n\tdaemonSet = \"node-agent\"\n\n\t// daemonsetWindows is the name of the Velero node agent daemonset on Windows nodes.\n\tdaemonsetWindows = \"node-agent-windows\"\n\n\t// nodeAgentRole marks pods with node-agent role on all nodes.\n\tnodeAgentRole = \"node-agent\"\n\n\t// HostPodVolumeMount is the name of the volume in node-agent for host-pod mount\n\tHostPodVolumeMount = \"host-pods\"\n\n\t// HostPodVolumeMountPoint is the mount point of the volume in node-agent for host-pod mount\n\tHostPodVolumeMountPoint = \"host_pods\"\n)\n\nvar (\n\tErrDaemonSetNotFound = errors.New(\"daemonset not found\")\n\tErrNodeAgentLabelNotFound = errors.New(\"node-agent label not found\")\n\tErrNodeAgentAnnotationNotFound = errors.New(\"node-agent annotation not found\")\n\tErrNodeAgentTolerationNotFound = errors.New(\"node-agent toleration not found\")\n)\n\nfunc IsRunningOnLinux(ctx context.Context, kubeClient kubernetes.Interface, namespace string) error {\n\treturn isRunning(ctx, kubeClient, namespace, daemonSet)\n}\n\nfunc IsRunningOnWindows(ctx context.Context, kubeClient kubernetes.Interface, namespace string) error {\n\treturn isRunning(ctx, kubeClient, namespace, daemonsetWindows)\n}\n\nfunc isRunning(ctx context.Context, kubeClient kubernetes.Interface, namespace string, daemonset string) error {\n\tif _, err := kubeClient.AppsV1().DaemonSets(namespace).Get(ctx, daemonset, metav1.GetOptions{}); apierrors.IsNotFound(err) {\n\t\treturn ErrDaemonSetNotFound\n\t} else if err != nil {\n\t\treturn err\n\t} else {\n\t\treturn nil\n\t}\n}\n\n// KbClientIsRunningInNode checks if the node agent pod is running properly in a specified node through kube client. If not, return the error found\nfunc KbClientIsRunningInNode(ctx context.Context, namespace string, nodeName string, kubeClient kubernetes.Interface) error {\n\treturn isRunningInNode(ctx, namespace, nodeName, nil, kubeClient)\n}\n\n// IsRunningInNode checks if the node agent pod is running properly in a specified node through controller client. If not, return the error found\nfunc IsRunningInNode(ctx context.Context, namespace string, nodeName string, crClient ctrlclient.Client) error {\n\treturn isRunningInNode(ctx, namespace, nodeName, crClient, nil)\n}\n\nfunc isRunningInNode(ctx context.Context, namespace string, nodeName string, crClient ctrlclient.Client, kubeClient kubernetes.Interface) error {\n\tif nodeName == \"\" {\n\t\treturn errors.New(\"node name is empty\")\n\t}\n\n\tpods := new(corev1api.PodList)\n\tparsedSelector, err := labels.Parse(fmt.Sprintf(\"role=%s\", nodeAgentRole))\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"fail to parse selector\")\n\t}\n\n\tif crClient != nil {\n\t\terr = crClient.List(ctx, pods, &ctrlclient.ListOptions{LabelSelector: parsedSelector})\n\t} else {\n\t\tpods, err = kubeClient.CoreV1().Pods(namespace).List(ctx, metav1.ListOptions{LabelSelector: parsedSelector.String()})\n\t}\n\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"failed to list node-agent pods\")\n\t}\n\n\tfor i := range pods.Items {\n\t\tif kube.IsPodRunning(&pods.Items[i]) != nil {\n\t\t\tcontinue\n\t\t}\n\n\t\tif pods.Items[i].Spec.NodeName == nodeName {\n\t\t\treturn nil\n\t\t}\n\t}\n\n\treturn errors.Errorf(\"daemonset pod not found in running state in node %s\", nodeName)\n}\n\nfunc GetPodSpec(ctx context.Context, kubeClient kubernetes.Interface, namespace string, osType string) (*corev1api.PodSpec, error) {\n\tdsName := daemonSet\n\tif osType == kube.NodeOSWindows {\n\t\tdsName = daemonsetWindows\n\t}\n\n\tds, err := kubeClient.AppsV1().DaemonSets(namespace).Get(ctx, dsName, metav1.GetOptions{})\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error to get %s daemonset\", dsName)\n\t}\n\n\treturn &ds.Spec.Template.Spec, nil\n}\n\nfunc GetConfigs(ctx context.Context, namespace string, kubeClient kubernetes.Interface, configName string) (*velerotypes.NodeAgentConfigs, error) {\n\tcm, err := kubeClient.CoreV1().ConfigMaps(namespace).Get(ctx, configName, metav1.GetOptions{})\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error to get node agent configs %s\", configName)\n\t}\n\n\tif cm.Data == nil {\n\t\treturn nil, errors.Errorf(\"data is not available in config map %s\", configName)\n\t}\n\n\tif len(cm.Data) > 1 {\n\t\treturn nil, errors.Errorf(\"more than one keys are found in ConfigMap %s's data. only expect one\", configName)\n\t}\n\n\tjsonString := \"\"\n\tfor _, v := range cm.Data {\n\t\tjsonString = v\n\t}\n\n\tconfigs := &velerotypes.NodeAgentConfigs{}\n\terr = json.Unmarshal([]byte(jsonString), configs)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error to unmarshall configs from %s\", configName)\n\t}\n\n\treturn configs, nil\n}\n\nfunc GetLabelValue(ctx context.Context, kubeClient kubernetes.Interface, namespace string, key string, osType string) (string, error) {\n\tdsName := daemonSet\n\tif osType == kube.NodeOSWindows {\n\t\tdsName = daemonsetWindows\n\t}\n\n\tds, err := kubeClient.AppsV1().DaemonSets(namespace).Get(ctx, dsName, metav1.GetOptions{})\n\tif err != nil {\n\t\treturn \"\", errors.Wrapf(err, \"error getting %s daemonset\", dsName)\n\t}\n\n\tif ds.Spec.Template.Labels == nil {\n\t\treturn \"\", ErrNodeAgentLabelNotFound\n\t}\n\n\tval, found := ds.Spec.Template.Labels[key]\n\tif !found {\n\t\treturn \"\", ErrNodeAgentLabelNotFound\n\t}\n\n\treturn val, nil\n}\n\nfunc GetAnnotationValue(ctx context.Context, kubeClient kubernetes.Interface, namespace string, key string, osType string) (string, error) {\n\tdsName := daemonSet\n\tif osType == kube.NodeOSWindows {\n\t\tdsName = daemonsetWindows\n\t}\n\n\tds, err := kubeClient.AppsV1().DaemonSets(namespace).Get(ctx, dsName, metav1.GetOptions{})\n\tif err != nil {\n\t\treturn \"\", errors.Wrapf(err, \"error getting %s daemonset\", dsName)\n\t}\n\n\tif ds.Spec.Template.Annotations == nil {\n\t\treturn \"\", ErrNodeAgentAnnotationNotFound\n\t}\n\n\tval, found := ds.Spec.Template.Annotations[key]\n\tif !found {\n\t\treturn \"\", ErrNodeAgentAnnotationNotFound\n\t}\n\n\treturn val, nil\n}\n\nfunc GetToleration(ctx context.Context, kubeClient kubernetes.Interface, namespace string, key string, osType string) (*corev1api.Toleration, error) {\n\tdsName := daemonSet\n\tif osType == kube.NodeOSWindows {\n\t\tdsName = daemonsetWindows\n\t}\n\n\tds, err := kubeClient.AppsV1().DaemonSets(namespace).Get(ctx, dsName, metav1.GetOptions{})\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error getting %s daemonset\", dsName)\n\t}\n\n\tfor i, t := range ds.Spec.Template.Spec.Tolerations {\n\t\tif t.Key == key {\n\t\t\treturn &ds.Spec.Template.Spec.Tolerations[i], nil\n\t\t}\n\t}\n\n\treturn nil, ErrNodeAgentTolerationNotFound\n}\n\nfunc GetHostPodPath(ctx context.Context, kubeClient kubernetes.Interface, namespace string, osType string) (string, error) {\n\tdsName := daemonSet\n\tif osType == kube.NodeOSWindows {\n\t\tdsName = daemonsetWindows\n\t}\n\n\tds, err := kubeClient.AppsV1().DaemonSets(namespace).Get(ctx, dsName, metav1.GetOptions{})\n\tif err != nil {\n\t\treturn \"\", errors.Wrapf(err, \"error getting daemonset %s\", dsName)\n\t}\n\n\tvar volume *corev1api.Volume\n\tfor _, v := range ds.Spec.Template.Spec.Volumes {\n\t\tif v.Name == HostPodVolumeMount {\n\t\t\tvolume = &v\n\t\t\tbreak\n\t\t}\n\t}\n\n\tif volume == nil {\n\t\treturn \"\", errors.New(\"host pod volume is not found\")\n\t}\n\n\tif volume.HostPath == nil {\n\t\treturn \"\", errors.New(\"host pod volume is not a host path volume\")\n\t}\n\n\tif volume.HostPath.Path == \"\" {\n\t\treturn \"\", errors.New(\"host pod volume path is empty\")\n\t}\n\n\treturn volume.HostPath.Path, nil\n}\n\nfunc HostPodVolumeMountPath() string {\n\treturn \"/\" + HostPodVolumeMountPoint\n}\n\nfunc HostPodVolumeMountPathWin() string {\n\treturn \"\\\\\" + HostPodVolumeMountPoint\n}\n" }, { "path": "pkg/nodeagent/node_agent_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage nodeagent\n\nimport (\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes/fake\"\n\tclientTesting \"k8s.io/client-go/testing\"\n\tclientFake \"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tvelerotypes \"github.com/vmware-tanzu/velero/pkg/types\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\ntype reactor struct {\n\tverb string\n\tresource string\n\treactorFunc clientTesting.ReactionFunc\n}\n\nfunc TestIsRunning(t *testing.T) {\n\tds := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\tnamespace string\n\t\tkubeReactors []reactor\n\t\texpectErr string\n\t}{\n\t\t{\n\t\t\tname: \"ds is not found\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\texpectErr: \"daemonset not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"ds get error\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"get\",\n\t\t\t\t\tresource: \"daemonsets\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-get-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectErr: \"fake-get-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tds,\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tfor _, reactor := range test.kubeReactors {\n\t\t\t\tfakeKubeClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\terr := isRunning(t.Context(), fakeKubeClient, test.namespace, daemonSet)\n\t\t\tif test.expectErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, test.expectErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestIsRunningInNode(t *testing.T) {\n\tscheme := runtime.NewScheme()\n\tcorev1api.AddToScheme(scheme)\n\n\tnonNodeAgentPod := builder.ForPod(\"fake-ns\", \"fake-pod\").Result()\n\tnodeAgentPodNotRunning := builder.ForPod(\"fake-ns\", \"fake-pod\").Labels(map[string]string{\"role\": \"node-agent\"}).Result()\n\tnodeAgentPodRunning1 := builder.ForPod(\"fake-ns\", \"fake-pod-1\").Labels(map[string]string{\"role\": \"node-agent\"}).Phase(corev1api.PodRunning).Result()\n\tnodeAgentPodRunning2 := builder.ForPod(\"fake-ns\", \"fake-pod-2\").Labels(map[string]string{\"role\": \"node-agent\"}).Phase(corev1api.PodRunning).Result()\n\tnodeAgentPodRunning3 := builder.ForPod(\"fake-ns\", \"fake-pod-3\").\n\t\tLabels(map[string]string{\"role\": \"node-agent\"}).\n\t\tPhase(corev1api.PodRunning).\n\t\tNodeName(\"fake-node\").\n\t\tResult()\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\tnodeName string\n\t\texpectErr string\n\t}{\n\t\t{\n\t\t\tname: \"node name is empty\",\n\t\t\texpectErr: \"node name is empty\",\n\t\t},\n\t\t{\n\t\t\tname: \"ds pod not found\",\n\t\t\tnodeName: \"fake-node\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tnonNodeAgentPod,\n\t\t\t},\n\t\t\texpectErr: \"daemonset pod not found in running state in node fake-node\",\n\t\t},\n\t\t{\n\t\t\tname: \"ds po are not all running\",\n\t\t\tnodeName: \"fake-node\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tnodeAgentPodNotRunning,\n\t\t\t\tnodeAgentPodRunning1,\n\t\t\t},\n\t\t\texpectErr: \"daemonset pod not found in running state in node fake-node\",\n\t\t},\n\t\t{\n\t\t\tname: \"ds pods wrong node name\",\n\t\t\tnodeName: \"fake-node\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tnodeAgentPodNotRunning,\n\t\t\t\tnodeAgentPodRunning1,\n\t\t\t\tnodeAgentPodRunning2,\n\t\t\t},\n\t\t\texpectErr: \"daemonset pod not found in running state in node fake-node\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tnodeName: \"fake-node\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tnodeAgentPodNotRunning,\n\t\t\t\tnodeAgentPodRunning1,\n\t\t\t\tnodeAgentPodRunning2,\n\t\t\t\tnodeAgentPodRunning3,\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeClientBuilder := clientFake.NewClientBuilder()\n\t\t\tfakeClientBuilder = fakeClientBuilder.WithScheme(scheme)\n\n\t\t\tfakeClient := fakeClientBuilder.WithRuntimeObjects(test.kubeClientObj...).Build()\n\n\t\t\terr := IsRunningInNode(t.Context(), \"\", test.nodeName, fakeClient)\n\t\t\tif test.expectErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, test.expectErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetPodSpec(t *testing.T) {\n\tpodSpec := corev1api.PodSpec{\n\t\tNodeName: \"fake-node\",\n\t}\n\n\tdaemonSet := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: podSpec,\n\t\t\t},\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\tnamespace string\n\t\texpectErr string\n\t\texpectSpec corev1api.PodSpec\n\t}{\n\t\t{\n\t\t\tname: \"ds is not found\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\texpectErr: \"error to get node-agent daemonset: daemonsets.apps \\\"node-agent\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSet,\n\t\t\t},\n\t\t\texpectSpec: podSpec,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tspec, err := GetPodSpec(t.Context(), fakeKubeClient, test.namespace, kube.NodeOSLinux)\n\t\t\tif test.expectErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, *spec, test.expectSpec)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, test.expectErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetConfigs(t *testing.T) {\n\tcm := builder.ForConfigMap(\"fake-ns\", \"node-agent-config\").Result()\n\tcmWithInvalidDataFormat := builder.ForConfigMap(\"fake-ns\", \"node-agent-config\").Data(\"fake-key\", \"wrong\").Result()\n\tcmWithoutCocurrentData := builder.ForConfigMap(\"fake-ns\", \"node-agent-config\").Data(\"fake-key\", \"{\\\"someothers\\\":{\\\"someother\\\": 10}}\").Result()\n\tcmWithValidData := builder.ForConfigMap(\"fake-ns\", \"node-agent-config\").Data(\"fake-key\", \"{\\\"loadConcurrency\\\":{\\\"globalConfig\\\": 5}}\").Result()\n\tcmWithPriorityClass := builder.ForConfigMap(\"fake-ns\", \"node-agent-config\").Data(\"fake-key\", \"{\\\"priorityClassName\\\": \\\"high-priority\\\"}\").Result()\n\tcmWithPriorityClassAndOther := builder.ForConfigMap(\"fake-ns\", \"node-agent-config\").Data(\"fake-key\", \"{\\\"priorityClassName\\\": \\\"low-priority\\\", \\\"loadConcurrency\\\":{\\\"globalConfig\\\": 3}}\").Result()\n\tcmWithMultipleKeysInData := builder.ForConfigMap(\"fake-ns\", \"node-agent-config\").Data(\"fake-key-1\", \"{}\", \"fake-key-2\", \"{}\").Result()\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\tnamespace string\n\t\tkubeReactors []reactor\n\t\texpectResult *velerotypes.NodeAgentConfigs\n\t\texpectErr string\n\t}{\n\t\t{\n\t\t\tname: \"cm get error\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"get\",\n\t\t\t\t\tresource: \"configmaps\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-get-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectErr: \"error to get node agent configs node-agent-config: fake-get-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"cm's data is nil\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcm,\n\t\t\t},\n\t\t\texpectErr: \"data is not available in config map node-agent-config\",\n\t\t},\n\t\t{\n\t\t\tname: \"cm's data is with invalid format\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcmWithInvalidDataFormat,\n\t\t\t},\n\t\t\texpectErr: \"error to unmarshall configs from node-agent-config: invalid character 'w' looking for beginning of value\",\n\t\t},\n\t\t{\n\t\t\tname: \"concurrency configs are not found\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcmWithoutCocurrentData,\n\t\t\t},\n\t\t\texpectResult: &velerotypes.NodeAgentConfigs{},\n\t\t},\n\t\t{\n\t\t\tname: \"success\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcmWithValidData,\n\t\t\t},\n\t\t\texpectResult: &velerotypes.NodeAgentConfigs{\n\t\t\t\tLoadConcurrency: &velerotypes.LoadConcurrency{\n\t\t\t\t\tGlobalConfig: 5,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"configmap with priority class name\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcmWithPriorityClass,\n\t\t\t},\n\t\t\texpectResult: &velerotypes.NodeAgentConfigs{\n\t\t\t\tPriorityClassName: \"high-priority\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"configmap with priority class and other configs\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcmWithPriorityClassAndOther,\n\t\t\t},\n\t\t\texpectResult: &velerotypes.NodeAgentConfigs{\n\t\t\t\tPriorityClassName: \"low-priority\",\n\t\t\t\tLoadConcurrency: &velerotypes.LoadConcurrency{\n\t\t\t\t\tGlobalConfig: 3,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"ConfigMap's Data has more than one key\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcmWithMultipleKeysInData,\n\t\t\t},\n\t\t\texpectErr: \"more than one keys are found in ConfigMap node-agent-config's data. only expect one\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tfor _, reactor := range test.kubeReactors {\n\t\t\t\tfakeKubeClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\tresult, err := GetConfigs(t.Context(), test.namespace, fakeKubeClient, \"node-agent-config\")\n\t\t\tif test.expectErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\tif test.expectResult == nil {\n\t\t\t\t\tassert.Nil(t, result)\n\t\t\t\t} else {\n\t\t\t\t\t// Check PriorityClassName\n\t\t\t\t\tassert.Equal(t, test.expectResult.PriorityClassName, result.PriorityClassName)\n\n\t\t\t\t\t// Check LoadConcurrency\n\t\t\t\t\tif test.expectResult.LoadConcurrency == nil {\n\t\t\t\t\t\tassert.Nil(t, result.LoadConcurrency)\n\t\t\t\t\t} else {\n\t\t\t\t\t\tassert.Equal(t, *test.expectResult.LoadConcurrency, *result.LoadConcurrency)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, test.expectErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetLabelValue(t *testing.T) {\n\tdaemonSet := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t},\n\t}\n\n\tdaemonSetWithOtherLabel := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"fake-other-label\": \"fake-value-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tdaemonSetWithLabel := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"fake-label\": \"fake-value-2\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tdaemonSetWithEmptyLabel := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"fake-label\": \"\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\tnamespace string\n\t\texpectedValue string\n\t\texpectErr string\n\t}{\n\t\t{\n\t\t\tname: \"ds get error\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\texpectErr: \"error getting node-agent daemonset: daemonsets.apps \\\"node-agent\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"no label\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSet,\n\t\t\t},\n\t\t\texpectErr: ErrNodeAgentLabelNotFound.Error(),\n\t\t},\n\t\t{\n\t\t\tname: \"no expecting label\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSetWithOtherLabel,\n\t\t\t},\n\t\t\texpectErr: ErrNodeAgentLabelNotFound.Error(),\n\t\t},\n\t\t{\n\t\t\tname: \"expecting label\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSetWithLabel,\n\t\t\t},\n\t\t\texpectedValue: \"fake-value-2\",\n\t\t},\n\t\t{\n\t\t\tname: \"expecting empty label\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSetWithEmptyLabel,\n\t\t\t},\n\t\t\texpectedValue: \"\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tvalue, err := GetLabelValue(t.Context(), fakeKubeClient, test.namespace, \"fake-label\", kube.NodeOSLinux)\n\t\t\tif test.expectErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, test.expectedValue, value)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, test.expectErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetAnnotationValue(t *testing.T) {\n\tdaemonSet := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t},\n\t}\n\n\tdaemonSetWithOtherAnnotation := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"fake-other-annotation\": \"fake-value-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tdaemonSetWithAnnotation := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"fake-annotation\": \"fake-value-2\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tdaemonSetWithEmptyAnnotation := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"fake-annotation\": \"\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\tnamespace string\n\t\texpectedValue string\n\t\texpectErr string\n\t}{\n\t\t{\n\t\t\tname: \"ds get error\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\texpectErr: \"error getting node-agent daemonset: daemonsets.apps \\\"node-agent\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"no annotation\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSet,\n\t\t\t},\n\t\t\texpectErr: ErrNodeAgentAnnotationNotFound.Error(),\n\t\t},\n\t\t{\n\t\t\tname: \"no expecting annotation\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSetWithOtherAnnotation,\n\t\t\t},\n\t\t\texpectErr: ErrNodeAgentAnnotationNotFound.Error(),\n\t\t},\n\t\t{\n\t\t\tname: \"expecting annotation\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSetWithAnnotation,\n\t\t\t},\n\t\t\texpectedValue: \"fake-value-2\",\n\t\t},\n\t\t{\n\t\t\tname: \"expecting empty annotation\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSetWithEmptyAnnotation,\n\t\t\t},\n\t\t\texpectedValue: \"\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tvalue, err := GetAnnotationValue(t.Context(), fakeKubeClient, test.namespace, \"fake-annotation\", kube.NodeOSLinux)\n\t\t\tif test.expectErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, test.expectedValue, value)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, test.expectErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetToleration(t *testing.T) {\n\tdaemonSet := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t},\n\t}\n\n\tdaemonSetWithOtherToleration := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tTolerations: []corev1api.Toleration{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKey: \"other-toleration-key\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tdaemonSetWithToleration := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tTolerations: []corev1api.Toleration{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKey: \"fake-toleration\",\n\t\t\t\t\t\t\tValue: \"true\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\tnamespace string\n\t\texpectedValue corev1api.Toleration\n\t\texpectErr string\n\t}{\n\t\t// {\n\t\t// \tname: \"ds get error\",\n\t\t// \tnamespace: \"fake-ns\",\n\t\t// \texpectErr: \"error getting node-agent daemonset: daemonsets.apps \\\"node-agent\\\" not found\",\n\t\t// },\n\t\t{\n\t\t\tname: \"no toleration\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSet,\n\t\t\t},\n\t\t\texpectErr: ErrNodeAgentTolerationNotFound.Error(),\n\t\t},\n\t\t{\n\t\t\tname: \"no expecting toleration\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSetWithOtherToleration,\n\t\t\t},\n\t\t\texpectErr: ErrNodeAgentTolerationNotFound.Error(),\n\t\t},\n\t\t{\n\t\t\tname: \"expecting toleration\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSetWithToleration,\n\t\t\t},\n\t\t\texpectedValue: corev1api.Toleration{\n\t\t\t\tKey: \"fake-toleration\",\n\t\t\t\tValue: \"true\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tvalue, err := GetToleration(t.Context(), fakeKubeClient, test.namespace, \"fake-toleration\", kube.NodeOSLinux)\n\t\t\tif test.expectErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, test.expectedValue, *value)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, test.expectErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetHostPodPath(t *testing.T) {\n\tdaemonSet := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t},\n\t}\n\n\tdaemonSetWithHostPodVolume := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: HostPodVolumeMount,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tdaemonSetWithHostPodVolumeAndEmptyPath := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: HostPodVolumeMount,\n\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\tHostPath: &corev1api.HostPathVolumeSource{},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tdaemonSetWithHostPodVolumeAndValidPath := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"node-agent\",\n\t\t},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"DaemonSet\",\n\t\t},\n\t\tSpec: appsv1api.DaemonSetSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: HostPodVolumeMount,\n\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\tHostPath: &corev1api.HostPathVolumeSource{\n\t\t\t\t\t\t\t\t\tPath: \"/var/lib/kubelet/pods\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\tnamespace string\n\t\tosType string\n\t\texpectedValue string\n\t\texpectErr string\n\t}{\n\t\t{\n\t\t\tname: \"ds get error\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tosType: kube.NodeOSWindows,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSet,\n\t\t\t},\n\t\t\texpectErr: \"error getting daemonset node-agent-windows: daemonsets.apps \\\"node-agent-windows\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"no host pod volume\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tosType: kube.NodeOSLinux,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSet,\n\t\t\t},\n\t\t\texpectErr: \"host pod volume is not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"no host pod volume path\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tosType: kube.NodeOSLinux,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSetWithHostPodVolume,\n\t\t\t},\n\t\t\texpectErr: \"host pod volume is not a host path volume\",\n\t\t},\n\t\t{\n\t\t\tname: \"empty host pod volume path\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tosType: kube.NodeOSLinux,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSetWithHostPodVolumeAndEmptyPath,\n\t\t\t},\n\t\t\texpectErr: \"host pod volume path is empty\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tosType: kube.NodeOSLinux,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSetWithHostPodVolumeAndValidPath,\n\t\t\t},\n\t\t\texpectedValue: \"/var/lib/kubelet/pods\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed on empty os type\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tdaemonSetWithHostPodVolumeAndValidPath,\n\t\t\t},\n\t\t\texpectedValue: \"/var/lib/kubelet/pods\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tpath, err := GetHostPodPath(t.Context(), fakeKubeClient, test.namespace, test.osType)\n\n\t\t\tif test.expectErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, test.expectedValue, path)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, test.expectErr)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/persistence/in_memory_object_store.go", "content": "/*\n\nCopyright the Velero contributors.\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\n\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage persistence\n\nimport (\n\t\"bytes\"\n\t\"errors\"\n\t\"io\"\n\t\"strings\"\n\t\"time\"\n)\n\ntype BucketData map[string][]byte\n\n// inMemoryObjectStore is a simple implementation of the ObjectStore interface\n// that stores its data in-memory/in-proc. This is mainly intended to be used\n// as a test fake.\ntype inMemoryObjectStore struct {\n\tData map[string]BucketData\n\tConfig map[string]string\n}\n\nfunc newInMemoryObjectStore(buckets ...string) *inMemoryObjectStore {\n\to := &inMemoryObjectStore{\n\t\tData: make(map[string]BucketData),\n\t}\n\n\tfor _, bucket := range buckets {\n\t\to.Data[bucket] = make(map[string][]byte)\n\t}\n\n\treturn o\n}\n\n//\n// Interface Implementation\n//\n\nfunc (o *inMemoryObjectStore) Init(config map[string]string) error {\n\to.Config = config\n\treturn nil\n}\n\nfunc (o *inMemoryObjectStore) PutObject(bucket, key string, body io.Reader) error {\n\tbucketData, ok := o.Data[bucket]\n\tif !ok {\n\t\treturn errors.New(\"bucket not found\")\n\t}\n\n\tobj, err := io.ReadAll(body)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tbucketData[key] = obj\n\n\treturn nil\n}\n\nfunc (o *inMemoryObjectStore) ObjectExists(bucket, key string) (bool, error) {\n\tbucketData, ok := o.Data[bucket]\n\tif !ok {\n\t\treturn false, errors.New(\"bucket not found\")\n\t}\n\n\t_, ok = bucketData[key]\n\treturn ok, nil\n}\n\nfunc (o *inMemoryObjectStore) GetObject(bucket, key string) (io.ReadCloser, error) {\n\tbucketData, ok := o.Data[bucket]\n\tif !ok {\n\t\treturn nil, errors.New(\"bucket not found\")\n\t}\n\n\tobj, ok := bucketData[key]\n\tif !ok {\n\t\treturn nil, errors.New(\"key not found\")\n\t}\n\n\treturn io.NopCloser(bytes.NewReader(obj)), nil\n}\n\nfunc (o *inMemoryObjectStore) ListCommonPrefixes(bucket, prefix, delimiter string) ([]string, error) {\n\tkeys, err := o.ListObjects(bucket, prefix)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// For each key, check if it has an instance of the delimiter *after* the prefix.\n\t// If not, skip it; if so, return the prefix of the key up to/including the delimiter.\n\n\tvar prefixes []string\n\tfor _, key := range keys {\n\t\t// everything after 'prefix'\n\t\tafterPrefix := key[len(prefix):]\n\n\t\t// index of the *start* of 'delimiter' in 'afterPrefix'\n\t\tdelimiterStart := strings.Index(afterPrefix, delimiter)\n\t\tif delimiterStart == -1 {\n\t\t\tcontinue\n\t\t}\n\n\t\t// return the prefix, plus everything after the prefix and before\n\t\t// the delimiter, plus the delimiter\n\t\tfullPrefix := prefix + afterPrefix[0:delimiterStart] + delimiter\n\n\t\tprefixes = append(prefixes, fullPrefix)\n\t}\n\n\treturn prefixes, nil\n}\n\nfunc (o *inMemoryObjectStore) ListObjects(bucket, prefix string) ([]string, error) {\n\tbucketData, ok := o.Data[bucket]\n\tif !ok {\n\t\treturn nil, errors.New(\"bucket not found\")\n\t}\n\n\tvar objs []string\n\tfor key := range bucketData {\n\t\tif strings.HasPrefix(key, prefix) {\n\t\t\tobjs = append(objs, key)\n\t\t}\n\t}\n\n\treturn objs, nil\n}\n\nfunc (o *inMemoryObjectStore) DeleteObject(bucket, key string) error {\n\tbucketData, ok := o.Data[bucket]\n\tif !ok {\n\t\treturn errors.New(\"bucket not found\")\n\t}\n\n\tdelete(bucketData, key)\n\n\treturn nil\n}\n\nfunc (o *inMemoryObjectStore) CreateSignedURL(bucket, key string, ttl time.Duration) (string, error) {\n\tbucketData, ok := o.Data[bucket]\n\tif !ok {\n\t\treturn \"\", errors.New(\"bucket not found\")\n\t}\n\n\t_, ok = bucketData[key]\n\tif !ok {\n\t\treturn \"\", errors.New(\"key not found\")\n\t}\n\n\treturn \"a-url\", nil\n}\n\n//\n// Test Helper Methods\n//\n\nfunc (o *inMemoryObjectStore) ClearBucket(bucket string) {\n\tif _, ok := o.Data[bucket]; !ok {\n\t\treturn\n\t}\n\n\to.Data[bucket] = make(map[string][]byte)\n}\n" }, { "path": "pkg/persistence/mocks/backup_store.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n// Code generated by mockery v2.42.2. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tio \"io\"\n\n\tmock \"github.com/stretchr/testify/mock\"\n\titemoperation \"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\n\tpersistence \"github.com/vmware-tanzu/velero/pkg/persistence\"\n\n\tresults \"github.com/vmware-tanzu/velero/pkg/util/results\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\n\tvolume \"github.com/vmware-tanzu/velero/internal/volume\"\n\n\tvolumesnapshotv1 \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n)\n\n// BackupStore is an autogenerated mock type for the BackupStore type\ntype BackupStore struct {\n\tmock.Mock\n}\n\n// BackupExists provides a mock function with given fields: bucket, backupName\nfunc (_m *BackupStore) BackupExists(bucket string, backupName string) (bool, error) {\n\tret := _m.Called(bucket, backupName)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for BackupExists\")\n\t}\n\n\tvar r0 bool\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string, string) (bool, error)); ok {\n\t\treturn rf(bucket, backupName)\n\t}\n\tif rf, ok := ret.Get(0).(func(string, string) bool); ok {\n\t\tr0 = rf(bucket, backupName)\n\t} else {\n\t\tr0 = ret.Get(0).(bool)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string, string) error); ok {\n\t\tr1 = rf(bucket, backupName)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// DeleteBackup provides a mock function with given fields: name\nfunc (_m *BackupStore) DeleteBackup(name string) error {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for DeleteBackup\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(string) error); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// DeleteRestore provides a mock function with given fields: name\nfunc (_m *BackupStore) DeleteRestore(name string) error {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for DeleteRestore\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(string) error); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// GetBackupContents provides a mock function with given fields: name\nfunc (_m *BackupStore) GetBackupContents(name string) (io.ReadCloser, error) {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetBackupContents\")\n\t}\n\n\tvar r0 io.ReadCloser\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) (io.ReadCloser, error)); ok {\n\t\treturn rf(name)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) io.ReadCloser); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(io.ReadCloser)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(name)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetBackupItemOperations provides a mock function with given fields: name\nfunc (_m *BackupStore) GetBackupItemOperations(name string) ([]*itemoperation.BackupOperation, error) {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetBackupItemOperations\")\n\t}\n\n\tvar r0 []*itemoperation.BackupOperation\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) ([]*itemoperation.BackupOperation, error)); ok {\n\t\treturn rf(name)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) []*itemoperation.BackupOperation); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]*itemoperation.BackupOperation)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(name)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetBackupMetadata provides a mock function with given fields: name\nfunc (_m *BackupStore) GetBackupMetadata(name string) (*v1.Backup, error) {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetBackupMetadata\")\n\t}\n\n\tvar r0 *v1.Backup\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) (*v1.Backup, error)); ok {\n\t\treturn rf(name)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) *v1.Backup); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(*v1.Backup)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(name)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetBackupVolumeInfos provides a mock function with given fields: name\nfunc (_m *BackupStore) GetBackupVolumeInfos(name string) ([]*volume.BackupVolumeInfo, error) {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetBackupVolumeInfos\")\n\t}\n\n\tvar r0 []*volume.BackupVolumeInfo\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) ([]*volume.BackupVolumeInfo, error)); ok {\n\t\treturn rf(name)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) []*volume.BackupVolumeInfo); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]*volume.BackupVolumeInfo)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(name)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetBackupVolumeSnapshots provides a mock function with given fields: name\nfunc (_m *BackupStore) GetBackupVolumeSnapshots(name string) ([]*volume.Snapshot, error) {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetBackupVolumeSnapshots\")\n\t}\n\n\tvar r0 []*volume.Snapshot\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) ([]*volume.Snapshot, error)); ok {\n\t\treturn rf(name)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) []*volume.Snapshot); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]*volume.Snapshot)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(name)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetCSIVolumeSnapshotClasses provides a mock function with given fields: name\nfunc (_m *BackupStore) GetCSIVolumeSnapshotClasses(name string) ([]*volumesnapshotv1.VolumeSnapshotClass, error) {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetCSIVolumeSnapshotClasses\")\n\t}\n\n\tvar r0 []*volumesnapshotv1.VolumeSnapshotClass\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) ([]*volumesnapshotv1.VolumeSnapshotClass, error)); ok {\n\t\treturn rf(name)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) []*volumesnapshotv1.VolumeSnapshotClass); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]*volumesnapshotv1.VolumeSnapshotClass)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(name)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetCSIVolumeSnapshotContents provides a mock function with given fields: name\nfunc (_m *BackupStore) GetCSIVolumeSnapshotContents(name string) ([]*volumesnapshotv1.VolumeSnapshotContent, error) {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetCSIVolumeSnapshotContents\")\n\t}\n\n\tvar r0 []*volumesnapshotv1.VolumeSnapshotContent\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) ([]*volumesnapshotv1.VolumeSnapshotContent, error)); ok {\n\t\treturn rf(name)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) []*volumesnapshotv1.VolumeSnapshotContent); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]*volumesnapshotv1.VolumeSnapshotContent)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(name)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetCSIVolumeSnapshots provides a mock function with given fields: name\nfunc (_m *BackupStore) GetCSIVolumeSnapshots(name string) ([]*volumesnapshotv1.VolumeSnapshot, error) {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetCSIVolumeSnapshots\")\n\t}\n\n\tvar r0 []*volumesnapshotv1.VolumeSnapshot\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) ([]*volumesnapshotv1.VolumeSnapshot, error)); ok {\n\t\treturn rf(name)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) []*volumesnapshotv1.VolumeSnapshot); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]*volumesnapshotv1.VolumeSnapshot)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(name)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetDownloadURL provides a mock function with given fields: target\nfunc (_m *BackupStore) GetDownloadURL(target v1.DownloadTarget) (string, error) {\n\tret := _m.Called(target)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetDownloadURL\")\n\t}\n\n\tvar r0 string\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(v1.DownloadTarget) (string, error)); ok {\n\t\treturn rf(target)\n\t}\n\tif rf, ok := ret.Get(0).(func(v1.DownloadTarget) string); ok {\n\t\tr0 = rf(target)\n\t} else {\n\t\tr0 = ret.Get(0).(string)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(v1.DownloadTarget) error); ok {\n\t\tr1 = rf(target)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetPodVolumeBackups provides a mock function with given fields: name\nfunc (_m *BackupStore) GetPodVolumeBackups(name string) ([]*v1.PodVolumeBackup, error) {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetPodVolumeBackups\")\n\t}\n\n\tvar r0 []*v1.PodVolumeBackup\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) ([]*v1.PodVolumeBackup, error)); ok {\n\t\treturn rf(name)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) []*v1.PodVolumeBackup); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]*v1.PodVolumeBackup)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(name)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetRestoreItemOperations provides a mock function with given fields: name\nfunc (_m *BackupStore) GetRestoreItemOperations(name string) ([]*itemoperation.RestoreOperation, error) {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetRestoreItemOperations\")\n\t}\n\n\tvar r0 []*itemoperation.RestoreOperation\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) ([]*itemoperation.RestoreOperation, error)); ok {\n\t\treturn rf(name)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) []*itemoperation.RestoreOperation); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]*itemoperation.RestoreOperation)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(name)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetRestoreResults provides a mock function with given fields: name\nfunc (_m *BackupStore) GetRestoreResults(name string) (map[string]results.Result, error) {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetRestoreResults\")\n\t}\n\n\tvar r0 map[string]results.Result\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) (map[string]results.Result, error)); ok {\n\t\treturn rf(name)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) map[string]results.Result); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(map[string]results.Result)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(name)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetRestoredResourceList provides a mock function with given fields: name\nfunc (_m *BackupStore) GetRestoredResourceList(name string) (map[string][]string, error) {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetRestoredResourceList\")\n\t}\n\n\tvar r0 map[string][]string\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) (map[string][]string, error)); ok {\n\t\treturn rf(name)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) map[string][]string); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(map[string][]string)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(name)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// IsValid provides a mock function with given fields:\nfunc (_m *BackupStore) IsValid() error {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for IsValid\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func() error); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// ListBackups provides a mock function with given fields:\nfunc (_m *BackupStore) ListBackups() ([]string, error) {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for ListBackups\")\n\t}\n\n\tvar r0 []string\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func() ([]string, error)); ok {\n\t\treturn rf()\n\t}\n\tif rf, ok := ret.Get(0).(func() []string); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]string)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// PutBackup provides a mock function with given fields: info\nfunc (_m *BackupStore) PutBackup(info persistence.BackupInfo) error {\n\tret := _m.Called(info)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PutBackup\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(persistence.BackupInfo) error); ok {\n\t\tr0 = rf(info)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// PutBackupContents provides a mock function with given fields: backup, backupContents\nfunc (_m *BackupStore) PutBackupContents(backup string, backupContents io.Reader) error {\n\tret := _m.Called(backup, backupContents)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PutBackupContents\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(string, io.Reader) error); ok {\n\t\tr0 = rf(backup, backupContents)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// PutBackupItemOperations provides a mock function with given fields: backup, backupItemOperations\nfunc (_m *BackupStore) PutBackupItemOperations(backup string, backupItemOperations io.Reader) error {\n\tret := _m.Called(backup, backupItemOperations)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PutBackupItemOperations\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(string, io.Reader) error); ok {\n\t\tr0 = rf(backup, backupItemOperations)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// PutBackupMetadata provides a mock function with given fields: backup, backupMetadata\nfunc (_m *BackupStore) PutBackupMetadata(backup string, backupMetadata io.Reader) error {\n\tret := _m.Called(backup, backupMetadata)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PutBackupMetadata\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(string, io.Reader) error); ok {\n\t\tr0 = rf(backup, backupMetadata)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// PutBackupVolumeInfos provides a mock function with given fields: name, volumeInfo\nfunc (_m *BackupStore) PutBackupVolumeInfos(name string, volumeInfo io.Reader) error {\n\tret := _m.Called(name, volumeInfo)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PutBackupVolumeInfos\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(string, io.Reader) error); ok {\n\t\tr0 = rf(name, volumeInfo)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// PutRestoreItemOperations provides a mock function with given fields: restore, restoreItemOperations\nfunc (_m *BackupStore) PutRestoreItemOperations(restore string, restoreItemOperations io.Reader) error {\n\tret := _m.Called(restore, restoreItemOperations)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PutRestoreItemOperations\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(string, io.Reader) error); ok {\n\t\tr0 = rf(restore, restoreItemOperations)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// PutRestoreLog provides a mock function with given fields: backup, restore, log\nfunc (_m *BackupStore) PutRestoreLog(backup string, restore string, log io.Reader) error {\n\tret := _m.Called(backup, restore, log)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PutRestoreLog\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(string, string, io.Reader) error); ok {\n\t\tr0 = rf(backup, restore, log)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// PutRestoreResults provides a mock function with given fields: backup, restore, _a2\nfunc (_m *BackupStore) PutRestoreResults(backup string, restore string, _a2 io.Reader) error {\n\tret := _m.Called(backup, restore, _a2)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PutRestoreResults\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(string, string, io.Reader) error); ok {\n\t\tr0 = rf(backup, restore, _a2)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// PutRestoreVolumeInfo provides a mock function with given fields: restore, volumeInfo\nfunc (_m *BackupStore) PutRestoreVolumeInfo(restore string, volumeInfo io.Reader) error {\n\tret := _m.Called(restore, volumeInfo)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PutRestoreVolumeInfo\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(string, io.Reader) error); ok {\n\t\tr0 = rf(restore, volumeInfo)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// PutRestoredResourceList provides a mock function with given fields: restore, _a1\nfunc (_m *BackupStore) PutRestoredResourceList(restore string, _a1 io.Reader) error {\n\tret := _m.Called(restore, _a1)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PutRestoredResourceList\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(string, io.Reader) error); ok {\n\t\tr0 = rf(restore, _a1)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// NewBackupStore creates a new instance of BackupStore. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\n// The first argument is typically a *testing.T value.\nfunc NewBackupStore(t interface {\n\tmock.TestingT\n\tCleanup(func())\n}) *BackupStore {\n\tmock := &BackupStore{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/persistence/mocks/object_store.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\n// Code generated by mockery v1.0.0. DO NOT EDIT.\npackage mocks\n\nimport io \"io\"\nimport mock \"github.com/stretchr/testify/mock\"\nimport time \"time\"\n\n// ObjectStore is an autogenerated mock type for the ObjectStore type\ntype ObjectStore struct {\n\tmock.Mock\n}\n\n// CreateSignedURL provides a mock function with given fields: bucket, key, ttl\nfunc (_m *ObjectStore) CreateSignedURL(bucket string, key string, ttl time.Duration) (string, error) {\n\tret := _m.Called(bucket, key, ttl)\n\n\tvar r0 string\n\tif rf, ok := ret.Get(0).(func(string, string, time.Duration) string); ok {\n\t\tr0 = rf(bucket, key, ttl)\n\t} else {\n\t\tr0 = ret.Get(0).(string)\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(string, string, time.Duration) error); ok {\n\t\tr1 = rf(bucket, key, ttl)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// DeleteObject provides a mock function with given fields: bucket, key\nfunc (_m *ObjectStore) DeleteObject(bucket string, key string) error {\n\tret := _m.Called(bucket, key)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(string, string) error); ok {\n\t\tr0 = rf(bucket, key)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// GetObject provides a mock function with given fields: bucket, key\nfunc (_m *ObjectStore) GetObject(bucket string, key string) (io.ReadCloser, error) {\n\tret := _m.Called(bucket, key)\n\n\tvar r0 io.ReadCloser\n\tif rf, ok := ret.Get(0).(func(string, string) io.ReadCloser); ok {\n\t\tr0 = rf(bucket, key)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(io.ReadCloser)\n\t\t}\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(string, string) error); ok {\n\t\tr1 = rf(bucket, key)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Init provides a mock function with given fields: config\nfunc (_m *ObjectStore) Init(config map[string]string) error {\n\tret := _m.Called(config)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(map[string]string) error); ok {\n\t\tr0 = rf(config)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// ListCommonPrefixes provides a mock function with given fields: bucket, prefix, delimiter\nfunc (_m *ObjectStore) ListCommonPrefixes(bucket string, prefix string, delimiter string) ([]string, error) {\n\tret := _m.Called(bucket, prefix, delimiter)\n\n\tvar r0 []string\n\tif rf, ok := ret.Get(0).(func(string, string, string) []string); ok {\n\t\tr0 = rf(bucket, prefix, delimiter)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]string)\n\t\t}\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(string, string, string) error); ok {\n\t\tr1 = rf(bucket, prefix, delimiter)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// ListObjects provides a mock function with given fields: bucket, prefix\nfunc (_m *ObjectStore) ListObjects(bucket string, prefix string) ([]string, error) {\n\tret := _m.Called(bucket, prefix)\n\n\tvar r0 []string\n\tif rf, ok := ret.Get(0).(func(string, string) []string); ok {\n\t\tr0 = rf(bucket, prefix)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]string)\n\t\t}\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(string, string) error); ok {\n\t\tr1 = rf(bucket, prefix)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// ObjectExists provides a mock function with given fields: bucket, key\nfunc (_m *ObjectStore) ObjectExists(bucket string, key string) (bool, error) {\n\tret := _m.Called(bucket, key)\n\n\tvar r0 bool\n\tif rf, ok := ret.Get(0).(func(string, string) bool); ok {\n\t\tr0 = rf(bucket, key)\n\t} else {\n\t\tr0 = ret.Get(0).(bool)\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(string, string) error); ok {\n\t\tr1 = rf(bucket, key)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// PutObject provides a mock function with given fields: bucket, key, body\nfunc (_m *ObjectStore) PutObject(bucket string, key string, body io.Reader) error {\n\tret := _m.Called(bucket, key, body)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(string, string, io.Reader) error); ok {\n\t\tr0 = rf(bucket, key, body)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n" }, { "path": "pkg/persistence/object_store.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage persistence\n\nimport (\n\t\"compress/gzip\"\n\t\"encoding/json\"\n\t\"io\"\n\t\"strings\"\n\t\"time\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/runtime/serializer\"\n\tkerrors \"k8s.io/apimachinery/pkg/util/errors\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/results\"\n)\n\ntype BackupInfo struct {\n\tName string\n\tMetadata,\n\tContents,\n\tLog,\n\tBackupResults,\n\tPodVolumeBackups,\n\tVolumeSnapshots,\n\tBackupItemOperations,\n\tBackupResourceList,\n\tCSIVolumeSnapshotClasses,\n\tBackupVolumeInfo io.Reader\n}\n\n// BackupStore defines operations for creating, retrieving, and deleting\n// Velero backup and restore data in/from a persistent backup store.\ntype BackupStore interface {\n\tIsValid() error\n\n\tListBackups() ([]string, error)\n\n\tPutBackup(info BackupInfo) error\n\tPutBackupMetadata(backup string, backupMetadata io.Reader) error\n\tPutBackupItemOperations(backup string, backupItemOperations io.Reader) error\n\tPutBackupContents(backup string, backupContents io.Reader) error\n\tGetBackupMetadata(name string) (*velerov1api.Backup, error)\n\tGetBackupItemOperations(name string) ([]*itemoperation.BackupOperation, error)\n\tGetBackupVolumeSnapshots(name string) ([]*volume.Snapshot, error)\n\tGetPodVolumeBackups(name string) ([]*velerov1api.PodVolumeBackup, error)\n\tGetBackupContents(name string) (io.ReadCloser, error)\n\tGetCSIVolumeSnapshots(name string) ([]*snapshotv1api.VolumeSnapshot, error)\n\tGetCSIVolumeSnapshotClasses(name string) ([]*snapshotv1api.VolumeSnapshotClass, error)\n\tPutBackupVolumeInfos(name string, volumeInfo io.Reader) error\n\tGetBackupVolumeInfos(name string) ([]*volume.BackupVolumeInfo, error)\n\tGetRestoreResults(name string) (map[string]results.Result, error)\n\n\t// BackupExists checks if the backup metadata file exists in object storage.\n\tBackupExists(bucket, backupName string) (bool, error)\n\n\tDeleteBackup(name string) error\n\n\tPutRestoreLog(backup, restore string, log io.Reader) error\n\tPutRestoreResults(backup, restore string, results io.Reader) error\n\tPutRestoredResourceList(restore string, results io.Reader) error\n\tPutRestoreItemOperations(restore string, restoreItemOperations io.Reader) error\n\tGetRestoreItemOperations(name string) ([]*itemoperation.RestoreOperation, error)\n\tPutRestoreVolumeInfo(restore string, volumeInfo io.Reader) error\n\tDeleteRestore(name string) error\n\tGetRestoredResourceList(name string) (map[string][]string, error)\n\n\tGetDownloadURL(target velerov1api.DownloadTarget) (string, error)\n}\n\n// DownloadURLTTL is how long a download URL is valid for.\nconst DownloadURLTTL = 10 * time.Minute\n\ntype objectBackupStore struct {\n\tobjectStore velero.ObjectStore\n\tbucket string\n\tlayout *ObjectStoreLayout\n\tlogger logrus.FieldLogger\n}\n\n// ObjectStoreGetter is a type that can get a velero.ObjectStore\n// from a provider name.\ntype ObjectStoreGetter interface {\n\tGetObjectStore(provider string) (velero.ObjectStore, error)\n}\n\n// ObjectBackupStoreGetter is a type that can get a velero.BackupStore for a\n// given BackupStorageLocation and ObjectStore.\ntype ObjectBackupStoreGetter interface {\n\tGet(location *velerov1api.BackupStorageLocation, objectStoreGetter ObjectStoreGetter, logger logrus.FieldLogger) (BackupStore, error)\n}\n\ntype objectBackupStoreGetter struct {\n\tcredentialStore credentials.FileStore\n\tsecretStore credentials.SecretStore\n}\n\n// NewObjectBackupStoreGetter returns a ObjectBackupStoreGetter that can get a velero.BackupStore.\nfunc NewObjectBackupStoreGetter(credentialStore credentials.FileStore) ObjectBackupStoreGetter {\n\treturn &objectBackupStoreGetter{credentialStore: credentialStore}\n}\n\n// NewObjectBackupStoreGetterWithSecretStore returns an ObjectBackupStoreGetter with SecretStore\n// support for resolving caCertRef from Kubernetes Secrets.\nfunc NewObjectBackupStoreGetterWithSecretStore(credentialStore credentials.FileStore, secretStore credentials.SecretStore) ObjectBackupStoreGetter {\n\treturn &objectBackupStoreGetter{\n\t\tcredentialStore: credentialStore,\n\t\tsecretStore: secretStore,\n\t}\n}\n\nfunc (b *objectBackupStoreGetter) Get(location *velerov1api.BackupStorageLocation, objectStoreGetter ObjectStoreGetter, logger logrus.FieldLogger) (BackupStore, error) {\n\tif location.Spec.ObjectStorage == nil {\n\t\treturn nil, errors.New(\"backup storage location does not use object storage\")\n\t}\n\n\tif location.Spec.Provider == \"\" {\n\t\treturn nil, errors.New(\"object storage provider name must not be empty\")\n\t}\n\n\t// trim off any leading/trailing slashes\n\tbucket := strings.Trim(location.Spec.ObjectStorage.Bucket, \"/\")\n\tprefix := strings.Trim(location.Spec.ObjectStorage.Prefix, \"/\")\n\n\t// if there are any slashes in the middle of 'bucket', the user\n\t// probably put / in the bucket field, which we\n\t// don't support.\n\t// Exception: MRAP ARNs (arn:aws:s3::...) legitimately contain slashes.\n\tif strings.Contains(bucket, \"/\") && !strings.HasPrefix(bucket, \"arn:aws:s3:\") {\n\t\treturn nil, errors.Errorf(\"backup storage location's bucket name %q must not contain a '/' (if using a prefix, put it in the 'Prefix' field instead)\", location.Spec.ObjectStorage.Bucket)\n\t}\n\n\t// Pass a new map into the object store rather than modifying the passed-in\n\t// location. This prevents Velero controllers from accidentally modifying\n\t// the in-cluster BSL with data which doesn't belong in Spec.Config\n\tobjectStoreConfig := make(map[string]string)\n\tif location.Spec.Config != nil {\n\t\tfor key, val := range location.Spec.Config {\n\t\t\tobjectStoreConfig[key] = val\n\t\t}\n\t}\n\n\t// add the bucket name and prefix to the config map so that object stores\n\t// can use them when initializing. The AWS object store uses the bucket\n\t// name to determine the bucket's region when setting up its client.\n\tobjectStoreConfig[\"bucket\"] = bucket\n\tobjectStoreConfig[\"prefix\"] = prefix\n\n\t// Only include a CACert if it's specified in order to maintain compatibility with plugins that don't expect it.\n\t// Prefer caCertRef (from Secret) over inline caCert (deprecated).\n\tif location.Spec.ObjectStorage.CACertRef != nil {\n\t\tif b.secretStore != nil {\n\t\t\tcaCertString, err := b.secretStore.Get(location.Spec.ObjectStorage.CACertRef)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, errors.Wrap(err, \"error getting CA certificate from secret\")\n\t\t\t}\n\t\t\tobjectStoreConfig[\"caCert\"] = caCertString\n\t\t}\n\t} else if location.Spec.ObjectStorage.CACert != nil {\n\t\tobjectStoreConfig[\"caCert\"] = string(location.Spec.ObjectStorage.CACert)\n\t}\n\n\t// If the BSL specifies a credential, fetch its path on disk and pass to\n\t// plugin via the config.\n\tif location.Spec.Credential != nil {\n\t\tcredsFile, err := b.credentialStore.Path(location.Spec.Credential)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrap(err, \"unable to get credentials\")\n\t\t}\n\n\t\tobjectStoreConfig[\"credentialsFile\"] = credsFile\n\t}\n\n\tobjectStore, err := objectStoreGetter.GetObjectStore(location.Spec.Provider)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif err := objectStore.Init(objectStoreConfig); err != nil {\n\t\treturn nil, err\n\t}\n\n\tlog := logger.WithFields(logrus.Fields(map[string]any{\n\t\t\"bucket\": bucket,\n\t\t\"prefix\": prefix,\n\t}))\n\n\treturn &objectBackupStore{\n\t\tobjectStore: objectStore,\n\t\tbucket: bucket,\n\t\tlayout: NewObjectStoreLayout(prefix),\n\t\tlogger: log,\n\t}, nil\n}\n\nfunc (s *objectBackupStore) IsValid() error {\n\tdirs, err := s.objectStore.ListCommonPrefixes(s.bucket, s.layout.rootPrefix, \"/\")\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\tvar invalid []string\n\tfor _, dir := range dirs {\n\t\tsubdir := strings.TrimSuffix(strings.TrimPrefix(dir, s.layout.rootPrefix), \"/\")\n\t\tif !s.layout.isValidSubdir(subdir) {\n\t\t\tinvalid = append(invalid, subdir)\n\t\t}\n\t}\n\n\tif len(invalid) > 0 {\n\t\t// don't include more than 3 invalid dirs in the error message\n\t\tif len(invalid) > 3 {\n\t\t\treturn errors.Errorf(\"Backup store contains %d invalid top-level directories: %v\", len(invalid), append(invalid[:3], \"...\"))\n\t\t}\n\t\treturn errors.Errorf(\"Backup store contains invalid top-level directories: %v\", invalid)\n\t}\n\n\treturn nil\n}\n\nfunc (s *objectBackupStore) ListBackups() ([]string, error) {\n\tprefixes, err := s.objectStore.ListCommonPrefixes(s.bucket, s.layout.subdirs[\"backups\"], \"/\")\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif len(prefixes) == 0 {\n\t\treturn []string{}, nil\n\t}\n\n\toutput := make([]string, 0, len(prefixes))\n\n\tfor _, prefix := range prefixes {\n\t\t// values returned from a call to ObjectStore's\n\t\t// ListCommonPrefixes method return the *full* prefix, inclusive\n\t\t// of s.backupsPrefix, and include the delimiter (\"/\") as a suffix. Trim\n\t\t// each of those off to get the backup name.\n\t\tbackupName := strings.TrimSuffix(strings.TrimPrefix(prefix, s.layout.subdirs[\"backups\"]), \"/\")\n\n\t\toutput = append(output, backupName)\n\t}\n\n\treturn output, nil\n}\n\nfunc (s *objectBackupStore) PutBackup(info BackupInfo) error {\n\tif err := seekAndPutObject(s.objectStore, s.bucket, s.layout.getBackupLogKey(info.Name), info.Log); err != nil {\n\t\t// Uploading the log file is best-effort; if it fails, we log the error but it doesn't impact the\n\t\t// backup's status.\n\t\ts.logger.WithError(err).WithField(\"backup\", info.Name).Error(\"Error uploading log file\")\n\t}\n\n\tif err := seekAndPutObject(s.objectStore, s.bucket, s.layout.getBackupMetadataKey(info.Name), info.Metadata); err != nil {\n\t\t// failure to upload metadata file is a hard-stop\n\t\treturn err\n\t}\n\n\tif err := seekAndPutObject(s.objectStore, s.bucket, s.layout.getBackupContentsKey(info.Name), info.Contents); err != nil {\n\t\tdeleteErr := s.objectStore.DeleteObject(s.bucket, s.layout.getBackupMetadataKey(info.Name))\n\t\treturn kerrors.NewAggregate([]error{err, deleteErr})\n\t}\n\n\t// Since the logic for all of these files is the exact same except for the name and the contents,\n\t// use a map literal to iterate through them and write them to the bucket.\n\tvar backupObjs = map[string]io.Reader{\n\t\ts.layout.getPodVolumeBackupsKey(info.Name): info.PodVolumeBackups,\n\t\ts.layout.getBackupVolumeSnapshotsKey(info.Name): info.VolumeSnapshots,\n\t\ts.layout.getBackupItemOperationsKey(info.Name): info.BackupItemOperations,\n\t\ts.layout.getBackupResourceListKey(info.Name): info.BackupResourceList,\n\t\ts.layout.getBackupResultsKey(info.Name): info.BackupResults,\n\t\ts.layout.getBackupVolumeInfoKey(info.Name): info.BackupVolumeInfo,\n\t}\n\n\tfor key, reader := range backupObjs {\n\t\tif err := seekAndPutObject(s.objectStore, s.bucket, key, reader); err != nil {\n\t\t\terrs := []error{err}\n\n\t\t\t// attempt to clean up the backup contents and metadata if we fail to upload and of the extra files.\n\t\t\tdeleteErr := s.objectStore.DeleteObject(s.bucket, s.layout.getBackupContentsKey(info.Name))\n\t\t\terrs = append(errs, deleteErr)\n\n\t\t\tdeleteErr = s.objectStore.DeleteObject(s.bucket, s.layout.getBackupMetadataKey(info.Name))\n\t\t\terrs = append(errs, deleteErr)\n\t\t\treturn kerrors.NewAggregate(errs)\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (s *objectBackupStore) GetBackupMetadata(name string) (*velerov1api.Backup, error) {\n\tmetadataKey := s.layout.getBackupMetadataKey(name)\n\n\tres, err := s.objectStore.GetObject(s.bucket, metadataKey)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdefer res.Close()\n\n\tdata, err := io.ReadAll(res)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tcodecFactory := serializer.NewCodecFactory(util.VeleroScheme)\n\n\tdecoder := codecFactory.UniversalDecoder(velerov1api.SchemeGroupVersion)\n\tobj, _, err := decoder.Decode(data, nil, nil)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tbackupObj, ok := obj.(*velerov1api.Backup)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"unexpected type for %s/%s: %T\", s.bucket, metadataKey, obj)\n\t}\n\n\treturn backupObj, nil\n}\n\nfunc (s *objectBackupStore) PutBackupMetadata(backup string, backupMetadata io.Reader) error {\n\treturn seekAndPutObject(s.objectStore, s.bucket, s.layout.getBackupMetadataKey(backup), backupMetadata)\n}\n\nfunc (s *objectBackupStore) GetBackupVolumeSnapshots(name string) ([]*volume.Snapshot, error) {\n\t// if the volumesnapshots file doesn't exist, we don't want to return an error, since\n\t// a legacy backup or a backup with no snapshots would not have this file, so check for\n\t// its existence before attempting to get its contents.\n\tres, err := tryGet(s.objectStore, s.bucket, s.layout.getBackupVolumeSnapshotsKey(name))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif res == nil {\n\t\treturn nil, nil\n\t}\n\tdefer res.Close()\n\n\tvar volumeSnapshots []*volume.Snapshot\n\tif err := decode(res, &volumeSnapshots); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn volumeSnapshots, nil\n}\n\nfunc (s *objectBackupStore) GetBackupItemOperations(name string) ([]*itemoperation.BackupOperation, error) {\n\t// if the itemoperations file doesn't exist, we don't want to return an error, since\n\t// a legacy backup or a backup with no async operations would not have this file, so check for\n\t// its existence before attempting to get its contents.\n\tres, err := tryGet(s.objectStore, s.bucket, s.layout.getBackupItemOperationsKey(name))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif res == nil {\n\t\treturn nil, nil\n\t}\n\tdefer res.Close()\n\n\tvar backupItemOperations []*itemoperation.BackupOperation\n\tif err := decode(res, &backupItemOperations); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn backupItemOperations, nil\n}\n\nfunc (s *objectBackupStore) GetRestoreItemOperations(name string) ([]*itemoperation.RestoreOperation, error) {\n\t// if the itemoperations file doesn't exist, we don't want to return an error, since\n\t// a legacy restore or a restore with no async operations would not have this file, so check for\n\t// its existence before attempting to get its contents.\n\tres, err := tryGet(s.objectStore, s.bucket, s.layout.getRestoreItemOperationsKey(name))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif res == nil {\n\t\treturn nil, nil\n\t}\n\tdefer res.Close()\n\n\tvar restoreItemOperations []*itemoperation.RestoreOperation\n\tif err := decode(res, &restoreItemOperations); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn restoreItemOperations, nil\n}\n\n// tryGet returns the object with the given key if it exists, nil if it does not exist,\n// or an error if it was unable to check existence or get the object.\nfunc tryGet(objectStore velero.ObjectStore, bucket, key string) (io.ReadCloser, error) {\n\texists, err := objectStore.ObjectExists(bucket, key)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\tif !exists {\n\t\treturn nil, nil\n\t}\n\n\treturn objectStore.GetObject(bucket, key)\n}\n\n// decode extracts a .json.gz file reader into the object pointed to\n// by 'into'.\nfunc decode(jsongzReader io.Reader, into any) error {\n\tgzr, err := gzip.NewReader(jsongzReader)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\tdefer gzr.Close()\n\n\tif err := json.NewDecoder(gzr).Decode(into); err != nil {\n\t\treturn errors.Wrap(err, \"error decoding object data\")\n\t}\n\n\treturn nil\n}\n\nfunc (s *objectBackupStore) GetCSIVolumeSnapshotClasses(name string) ([]*snapshotv1api.VolumeSnapshotClass, error) {\n\tres, err := tryGet(s.objectStore, s.bucket, s.layout.getCSIVolumeSnapshotClassesKey(name))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif res == nil {\n\t\t// this indicates that the no CSI volumesnapshots were prensent in the backup\n\t\treturn nil, nil\n\t}\n\tdefer res.Close()\n\n\tvar csiVSClasses []*snapshotv1api.VolumeSnapshotClass\n\tif err := decode(res, &csiVSClasses); err != nil {\n\t\treturn nil, err\n\t}\n\treturn csiVSClasses, nil\n}\n\nfunc (s *objectBackupStore) GetCSIVolumeSnapshots(name string) ([]*snapshotv1api.VolumeSnapshot, error) {\n\tres, err := tryGet(s.objectStore, s.bucket, s.layout.getCSIVolumeSnapshotKey(name))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif res == nil {\n\t\t// this indicates that the no CSI volumesnapshots were prensent in the backup\n\t\treturn nil, nil\n\t}\n\tdefer res.Close()\n\n\tvar csiSnaps []*snapshotv1api.VolumeSnapshot\n\tif err := decode(res, &csiSnaps); err != nil {\n\t\treturn nil, err\n\t}\n\treturn csiSnaps, nil\n}\n\nfunc (s *objectBackupStore) GetCSIVolumeSnapshotContents(name string) ([]*snapshotv1api.VolumeSnapshotContent, error) {\n\tres, err := tryGet(s.objectStore, s.bucket, s.layout.getCSIVolumeSnapshotContentsKey(name))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif res == nil {\n\t\t// this indicates that the no CSI volumesnapshotcontents were prensent in the backup\n\t\treturn nil, nil\n\t}\n\tdefer res.Close()\n\n\tvar snapConts []*snapshotv1api.VolumeSnapshotContent\n\tif err := decode(res, &snapConts); err != nil {\n\t\treturn nil, err\n\t}\n\treturn snapConts, nil\n}\n\nfunc (s *objectBackupStore) GetPodVolumeBackups(name string) ([]*velerov1api.PodVolumeBackup, error) {\n\t// if the podvolumebackups file doesn't exist, we don't want to return an error, since\n\t// a legacy backup or a backup with no pod volume backups would not have this file, so\n\t// check for its existence before attempting to get its contents.\n\tres, err := tryGet(s.objectStore, s.bucket, s.layout.getPodVolumeBackupsKey(name))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif res == nil {\n\t\treturn nil, nil\n\t}\n\tdefer res.Close()\n\n\tvar podVolumeBackups []*velerov1api.PodVolumeBackup\n\tif err := decode(res, &podVolumeBackups); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn podVolumeBackups, nil\n}\n\nfunc (s *objectBackupStore) GetBackupVolumeInfos(name string) ([]*volume.BackupVolumeInfo, error) {\n\tvolumeInfos := make([]*volume.BackupVolumeInfo, 0)\n\n\tres, err := tryGet(s.objectStore, s.bucket, s.layout.getBackupVolumeInfoKey(name))\n\tif err != nil {\n\t\treturn volumeInfos, err\n\t}\n\tif res == nil {\n\t\treturn volumeInfos, nil\n\t}\n\tdefer res.Close()\n\n\tif err := decode(res, &volumeInfos); err != nil {\n\t\treturn volumeInfos, err\n\t}\n\n\treturn volumeInfos, nil\n}\n\nfunc (s *objectBackupStore) PutBackupVolumeInfos(name string, volumeInfo io.Reader) error {\n\treturn s.objectStore.PutObject(s.bucket, s.layout.getBackupVolumeInfoKey(name), volumeInfo)\n}\n\nfunc (s *objectBackupStore) GetRestoreResults(name string) (map[string]results.Result, error) {\n\tresults := make(map[string]results.Result)\n\n\tres, err := tryGet(s.objectStore, s.bucket, s.layout.getRestoreResultsKey(name))\n\tif err != nil {\n\t\treturn results, err\n\t}\n\tif res == nil {\n\t\treturn results, nil\n\t}\n\tdefer res.Close()\n\n\tif err := decode(res, &results); err != nil {\n\t\treturn results, err\n\t}\n\n\treturn results, nil\n}\n\nfunc (s *objectBackupStore) GetBackupContents(name string) (io.ReadCloser, error) {\n\treturn s.objectStore.GetObject(s.bucket, s.layout.getBackupContentsKey(name))\n}\n\nfunc (s *objectBackupStore) BackupExists(bucket, backupName string) (bool, error) {\n\treturn s.objectStore.ObjectExists(bucket, s.layout.getBackupMetadataKey(backupName))\n}\n\nfunc (s *objectBackupStore) DeleteBackup(name string) error {\n\tobjects, err := s.objectStore.ListObjects(s.bucket, s.layout.getBackupDir(name))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tvar errs []error\n\tfor _, key := range objects {\n\t\ts.logger.WithFields(logrus.Fields{\n\t\t\t\"key\": key,\n\t\t}).Debug(\"Trying to delete object\")\n\t\tif err := s.objectStore.DeleteObject(s.bucket, key); err != nil {\n\t\t\terrs = append(errs, err)\n\t\t}\n\t}\n\n\treturn errors.WithStack(kerrors.NewAggregate(errs))\n}\n\nfunc (s *objectBackupStore) DeleteRestore(name string) error {\n\tobjects, err := s.objectStore.ListObjects(s.bucket, s.layout.getRestoreDir(name))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tvar errs []error\n\tfor _, key := range objects {\n\t\ts.logger.WithFields(logrus.Fields{\n\t\t\t\"key\": key,\n\t\t}).Debug(\"Trying to delete object\")\n\t\tif err := s.objectStore.DeleteObject(s.bucket, key); err != nil {\n\t\t\terrs = append(errs, err)\n\t\t}\n\t}\n\n\treturn errors.WithStack(kerrors.NewAggregate(errs))\n}\n\nfunc (s *objectBackupStore) PutRestoreLog(backup string, restore string, log io.Reader) error {\n\treturn s.objectStore.PutObject(s.bucket, s.layout.getRestoreLogKey(restore), log)\n}\n\nfunc (s *objectBackupStore) PutRestoreResults(backup string, restore string, results io.Reader) error {\n\treturn s.objectStore.PutObject(s.bucket, s.layout.getRestoreResultsKey(restore), results)\n}\n\nfunc (s *objectBackupStore) PutRestoredResourceList(restore string, list io.Reader) error {\n\treturn s.objectStore.PutObject(s.bucket, s.layout.getRestoreResourceListKey(restore), list)\n}\n\nfunc (s *objectBackupStore) PutRestoreItemOperations(restore string, restoreItemOperations io.Reader) error {\n\treturn seekAndPutObject(s.objectStore, s.bucket, s.layout.getRestoreItemOperationsKey(restore), restoreItemOperations)\n}\n\nfunc (s *objectBackupStore) PutRestoreVolumeInfo(restore string, volumeInfo io.Reader) error {\n\treturn seekAndPutObject(s.objectStore, s.bucket, s.layout.getRestoreVolumeInfoKey(restore), volumeInfo)\n}\n\nfunc (s *objectBackupStore) PutBackupItemOperations(backup string, backupItemOperations io.Reader) error {\n\treturn seekAndPutObject(s.objectStore, s.bucket, s.layout.getBackupItemOperationsKey(backup), backupItemOperations)\n}\n\nfunc (s *objectBackupStore) PutBackupContents(backup string, backupContents io.Reader) error {\n\treturn seekAndPutObject(s.objectStore, s.bucket, s.layout.getBackupContentsKey(backup), backupContents)\n}\n\nfunc (s *objectBackupStore) GetDownloadURL(target velerov1api.DownloadTarget) (string, error) {\n\tswitch target.Kind {\n\tcase velerov1api.DownloadTargetKindBackupContents:\n\t\treturn s.objectStore.CreateSignedURL(s.bucket, s.layout.getBackupContentsKey(target.Name), DownloadURLTTL)\n\tcase velerov1api.DownloadTargetKindBackupLog:\n\t\treturn s.objectStore.CreateSignedURL(s.bucket, s.layout.getBackupLogKey(target.Name), DownloadURLTTL)\n\tcase velerov1api.DownloadTargetKindBackupVolumeSnapshots:\n\t\treturn s.objectStore.CreateSignedURL(s.bucket, s.layout.getBackupVolumeSnapshotsKey(target.Name), DownloadURLTTL)\n\tcase velerov1api.DownloadTargetKindBackupItemOperations:\n\t\treturn s.objectStore.CreateSignedURL(s.bucket, s.layout.getBackupItemOperationsKey(target.Name), DownloadURLTTL)\n\tcase velerov1api.DownloadTargetKindRestoreItemOperations:\n\t\treturn s.objectStore.CreateSignedURL(s.bucket, s.layout.getRestoreItemOperationsKey(target.Name), DownloadURLTTL)\n\tcase velerov1api.DownloadTargetKindBackupResourceList:\n\t\treturn s.objectStore.CreateSignedURL(s.bucket, s.layout.getBackupResourceListKey(target.Name), DownloadURLTTL)\n\tcase velerov1api.DownloadTargetKindRestoreLog:\n\t\treturn s.objectStore.CreateSignedURL(s.bucket, s.layout.getRestoreLogKey(target.Name), DownloadURLTTL)\n\tcase velerov1api.DownloadTargetKindRestoreResults:\n\t\treturn s.objectStore.CreateSignedURL(s.bucket, s.layout.getRestoreResultsKey(target.Name), DownloadURLTTL)\n\tcase velerov1api.DownloadTargetKindRestoreResourceList:\n\t\treturn s.objectStore.CreateSignedURL(s.bucket, s.layout.getRestoreResourceListKey(target.Name), DownloadURLTTL)\n\tcase velerov1api.DownloadTargetKindCSIBackupVolumeSnapshots:\n\t\treturn s.objectStore.CreateSignedURL(s.bucket, s.layout.getCSIVolumeSnapshotKey(target.Name), DownloadURLTTL)\n\tcase velerov1api.DownloadTargetKindCSIBackupVolumeSnapshotContents:\n\t\treturn s.objectStore.CreateSignedURL(s.bucket, s.layout.getCSIVolumeSnapshotContentsKey(target.Name), DownloadURLTTL)\n\tcase velerov1api.DownloadTargetKindBackupResults:\n\t\treturn s.objectStore.CreateSignedURL(s.bucket, s.layout.getBackupResultsKey(target.Name), DownloadURLTTL)\n\tcase velerov1api.DownloadTargetKindBackupVolumeInfos:\n\t\treturn s.objectStore.CreateSignedURL(s.bucket, s.layout.getBackupVolumeInfoKey(target.Name), DownloadURLTTL)\n\tcase velerov1api.DownloadTargetKindRestoreVolumeInfo:\n\t\treturn s.objectStore.CreateSignedURL(s.bucket, s.layout.getRestoreVolumeInfoKey(target.Name), DownloadURLTTL)\n\tdefault:\n\t\treturn \"\", errors.Errorf(\"unsupported download target kind %q\", target.Kind)\n\t}\n}\n\nfunc (s *objectBackupStore) GetRestoredResourceList(name string) (map[string][]string, error) {\n\tlist := make(map[string][]string)\n\n\tres, err := tryGet(s.objectStore, s.bucket, s.layout.getRestoreResourceListKey(name))\n\tif err != nil {\n\t\treturn list, err\n\t}\n\tif res == nil {\n\t\treturn list, nil\n\t}\n\tdefer res.Close()\n\n\tif err := decode(res, &list); err != nil {\n\t\treturn list, err\n\t}\n\n\treturn list, nil\n}\n\nfunc seekToBeginning(r io.Reader) error {\n\tseeker, ok := r.(io.Seeker)\n\tif !ok {\n\t\treturn nil\n\t}\n\n\t_, err := seeker.Seek(0, 0)\n\treturn err\n}\n\nfunc seekAndPutObject(objectStore velero.ObjectStore, bucket, key string, file io.Reader) error {\n\tif file == nil {\n\t\treturn nil\n\t}\n\n\tif err := seekToBeginning(file); err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\treturn objectStore.PutObject(bucket, key, file)\n}\n" }, { "path": "pkg/persistence/object_store_layout.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage persistence\n\nimport (\n\t\"fmt\"\n\t\"path\"\n\t\"strings\"\n)\n\n// ObjectStoreLayout defines how Velero's persisted files map to\n// keys in an object storage bucket.\ntype ObjectStoreLayout struct {\n\trootPrefix string\n\tsubdirs map[string]string\n}\n\nfunc NewObjectStoreLayout(prefix string) *ObjectStoreLayout {\n\tif prefix != \"\" && !strings.HasSuffix(prefix, \"/\") {\n\t\tprefix = prefix + \"/\"\n\t}\n\n\tsubdirs := map[string]string{\n\t\t\"backups\": path.Join(prefix, \"backups\") + \"/\",\n\t\t\"restores\": path.Join(prefix, \"restores\") + \"/\",\n\t\t\"restic\": path.Join(prefix, \"restic\") + \"/\",\n\t\t\"metadata\": path.Join(prefix, \"metadata\") + \"/\",\n\t\t\"plugins\": path.Join(prefix, \"plugins\") + \"/\",\n\t\t\"kopia\": path.Join(prefix, \"kopia\") + \"/\",\n\t}\n\n\treturn &ObjectStoreLayout{\n\t\trootPrefix: prefix,\n\t\tsubdirs: subdirs,\n\t}\n}\n\n// GetResticDir returns the full prefix representing the restic\n// directory within an object storage bucket containing a backup\n// store.\nfunc (l *ObjectStoreLayout) GetResticDir() string {\n\treturn l.subdirs[\"restic\"]\n}\n\nfunc (l *ObjectStoreLayout) isValidSubdir(name string) bool {\n\t_, ok := l.subdirs[name]\n\treturn ok\n}\n\nfunc (l *ObjectStoreLayout) getBackupDir(backup string) string {\n\treturn path.Join(l.subdirs[\"backups\"], backup) + \"/\"\n}\n\nfunc (l *ObjectStoreLayout) getRestoreDir(restore string) string {\n\treturn path.Join(l.subdirs[\"restores\"], restore) + \"/\"\n}\n\nfunc (l *ObjectStoreLayout) getBackupMetadataKey(backup string) string {\n\treturn path.Join(l.subdirs[\"backups\"], backup, \"velero-backup.json\")\n}\n\nfunc (l *ObjectStoreLayout) getBackupContentsKey(backup string) string {\n\treturn path.Join(l.subdirs[\"backups\"], backup, fmt.Sprintf(\"%s.tar.gz\", backup))\n}\n\nfunc (l *ObjectStoreLayout) getBackupLogKey(backup string) string {\n\treturn path.Join(l.subdirs[\"backups\"], backup, fmt.Sprintf(\"%s-logs.gz\", backup))\n}\n\nfunc (l *ObjectStoreLayout) getPodVolumeBackupsKey(backup string) string {\n\treturn path.Join(l.subdirs[\"backups\"], backup, fmt.Sprintf(\"%s-podvolumebackups.json.gz\", backup))\n}\n\nfunc (l *ObjectStoreLayout) getBackupVolumeSnapshotsKey(backup string) string {\n\treturn path.Join(l.subdirs[\"backups\"], backup, fmt.Sprintf(\"%s-volumesnapshots.json.gz\", backup))\n}\n\nfunc (l *ObjectStoreLayout) getBackupItemOperationsKey(backup string) string {\n\treturn path.Join(l.subdirs[\"backups\"], backup, fmt.Sprintf(\"%s-itemoperations.json.gz\", backup))\n}\n\nfunc (l *ObjectStoreLayout) getBackupResourceListKey(backup string) string {\n\treturn path.Join(l.subdirs[\"backups\"], backup, fmt.Sprintf(\"%s-resource-list.json.gz\", backup))\n}\n\nfunc (l *ObjectStoreLayout) getRestoreLogKey(restore string) string {\n\treturn path.Join(l.subdirs[\"restores\"], restore, fmt.Sprintf(\"restore-%s-logs.gz\", restore))\n}\n\nfunc (l *ObjectStoreLayout) getRestoreResultsKey(restore string) string {\n\treturn path.Join(l.subdirs[\"restores\"], restore, fmt.Sprintf(\"restore-%s-results.gz\", restore))\n}\n\nfunc (l *ObjectStoreLayout) getRestoreResourceListKey(restore string) string {\n\treturn path.Join(l.subdirs[\"restores\"], restore, fmt.Sprintf(\"restore-%s-resource-list.json.gz\", restore))\n}\n\nfunc (l *ObjectStoreLayout) getRestoreItemOperationsKey(restore string) string {\n\treturn path.Join(l.subdirs[\"restores\"], restore, fmt.Sprintf(\"restore-%s-itemoperations.json.gz\", restore))\n}\n\nfunc (l *ObjectStoreLayout) getCSIVolumeSnapshotKey(backup string) string {\n\treturn path.Join(l.subdirs[\"backups\"], backup, fmt.Sprintf(\"%s-csi-volumesnapshots.json.gz\", backup))\n}\n\nfunc (l *ObjectStoreLayout) getCSIVolumeSnapshotContentsKey(backup string) string {\n\treturn path.Join(l.subdirs[\"backups\"], backup, fmt.Sprintf(\"%s-csi-volumesnapshotcontents.json.gz\", backup))\n}\n\nfunc (l *ObjectStoreLayout) getCSIVolumeSnapshotClassesKey(backup string) string {\n\treturn path.Join(l.subdirs[\"backups\"], backup, fmt.Sprintf(\"%s-csi-volumesnapshotclasses.json.gz\", backup))\n}\n\nfunc (l *ObjectStoreLayout) getBackupResultsKey(backup string) string {\n\treturn path.Join(l.subdirs[\"backups\"], backup, fmt.Sprintf(\"%s-results.gz\", backup))\n}\n\nfunc (l *ObjectStoreLayout) getBackupVolumeInfoKey(backup string) string {\n\treturn path.Join(l.subdirs[\"backups\"], backup, fmt.Sprintf(\"%s-volumeinfo.json.gz\", backup))\n}\n\nfunc (l *ObjectStoreLayout) getRestoreVolumeInfoKey(restore string) string {\n\treturn path.Join(l.subdirs[\"restores\"], restore, fmt.Sprintf(\"%s-volumeinfo.json.gz\", restore))\n}\n" }, { "path": "pkg/persistence/object_store_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage persistence\n\nimport (\n\t\"bytes\"\n\t\"compress/gzip\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"sort\"\n\t\"strings\"\n\t\"testing\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tprovidermocks \"github.com/vmware-tanzu/velero/pkg/plugin/velero/mocks\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/encode\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/results\"\n)\n\ntype objectBackupStoreTestHarness struct {\n\t// embedded to reduce verbosity when calling methods\n\t*objectBackupStore\n\n\tobjectStore *inMemoryObjectStore\n\tbucket, prefix string\n}\n\nfunc newObjectBackupStoreTestHarness(bucket, prefix string) *objectBackupStoreTestHarness {\n\tobjectStore := newInMemoryObjectStore(bucket)\n\n\treturn &objectBackupStoreTestHarness{\n\t\tobjectBackupStore: &objectBackupStore{\n\t\t\tobjectStore: objectStore,\n\t\t\tbucket: bucket,\n\t\t\tlayout: NewObjectStoreLayout(prefix),\n\t\t\tlogger: velerotest.NewLogger(),\n\t\t},\n\t\tobjectStore: objectStore,\n\t\tbucket: bucket,\n\t\tprefix: prefix,\n\t}\n}\n\nfunc TestIsValid(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tprefix string\n\t\tstorageData BucketData\n\t\texpectErr bool\n\t}{\n\t\t{\n\t\t\tname: \"empty backup store with no prefix is valid\",\n\t\t\texpectErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"empty backup store with a prefix is valid\",\n\t\t\tprefix: \"bar\",\n\t\t\texpectErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"backup store with no prefix and only unsupported directories is invalid\",\n\t\t\tstorageData: map[string][]byte{\n\t\t\t\t\"backup-1/velero-backup.json\": {},\n\t\t\t\t\"backup-2/velero-backup.json\": {},\n\t\t\t},\n\t\t\texpectErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"backup store with a prefix and only unsupported directories is invalid\",\n\t\t\tprefix: \"backups\",\n\t\t\tstorageData: map[string][]byte{\n\t\t\t\t\"backups/backup-1/velero-backup.json\": {},\n\t\t\t\t\"backups/backup-2/velero-backup.json\": {},\n\t\t\t},\n\t\t\texpectErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"backup store with no prefix and both supported and unsupported directories is invalid\",\n\t\t\tstorageData: map[string][]byte{\n\t\t\t\t\"backups/backup-1/velero-backup.json\": {},\n\t\t\t\t\"backups/backup-2/velero-backup.json\": {},\n\t\t\t\t\"restores/restore-1/foo\": {},\n\t\t\t\t\"unsupported-dir/foo\": {},\n\t\t\t},\n\t\t\texpectErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"backup store with a prefix and both supported and unsupported directories is invalid\",\n\t\t\tprefix: \"cluster-1\",\n\t\t\tstorageData: map[string][]byte{\n\t\t\t\t\"cluster-1/backups/backup-1/velero-backup.json\": {},\n\t\t\t\t\"cluster-1/backups/backup-2/velero-backup.json\": {},\n\t\t\t\t\"cluster-1/restores/restore-1/foo\": {},\n\t\t\t\t\"cluster-1/unsupported-dir/foo\": {},\n\t\t\t},\n\t\t\texpectErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"backup store with no prefix and only supported directories is valid\",\n\t\t\tstorageData: map[string][]byte{\n\t\t\t\t\"backups/backup-1/velero-backup.json\": {},\n\t\t\t\t\"backups/backup-2/velero-backup.json\": {},\n\t\t\t\t\"restores/restore-1/foo\": {},\n\t\t\t},\n\t\t\texpectErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"backup store with a prefix and only supported directories is valid\",\n\t\t\tprefix: \"cluster-1\",\n\t\t\tstorageData: map[string][]byte{\n\t\t\t\t\"cluster-1/backups/backup-1/velero-backup.json\": {},\n\t\t\t\t\"cluster-1/backups/backup-2/velero-backup.json\": {},\n\t\t\t\t\"cluster-1/restores/restore-1/foo\": {},\n\t\t\t},\n\t\t\texpectErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"backup store with plugins directory is valid\",\n\t\t\tstorageData: map[string][]byte{\n\t\t\t\t\"plugins/vsphere/foo\": {},\n\t\t\t},\n\t\t\texpectErr: false,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tharness := newObjectBackupStoreTestHarness(\"foo\", tc.prefix)\n\n\t\t\tfor key, obj := range tc.storageData {\n\t\t\t\trequire.NoError(t, harness.objectStore.PutObject(harness.bucket, key, bytes.NewReader(obj)))\n\t\t\t}\n\n\t\t\terr := harness.IsValid()\n\t\t\tif tc.expectErr {\n\t\t\t\tassert.Error(t, err)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestListBackups(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tprefix string\n\t\tstorageData BucketData\n\t\texpectedRes []string\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"normal case\",\n\t\t\tstorageData: map[string][]byte{\n\t\t\t\t\"backups/backup-1/velero-backup.json\": encodeToBytes(builder.ForBackup(\"\", \"backup-1\").Result()),\n\t\t\t\t\"backups/backup-2/velero-backup.json\": encodeToBytes(builder.ForBackup(\"\", \"backup-2\").Result()),\n\t\t\t},\n\t\t\texpectedRes: []string{\"backup-1\", \"backup-2\"},\n\t\t},\n\t\t{\n\t\t\tname: \"normal case with backup store prefix\",\n\t\t\tprefix: \"velero-backups/\",\n\t\t\tstorageData: map[string][]byte{\n\t\t\t\t\"velero-backups/backups/backup-1/velero-backup.json\": encodeToBytes(builder.ForBackup(\"\", \"backup-1\").Result()),\n\t\t\t\t\"velero-backups/backups/backup-2/velero-backup.json\": encodeToBytes(builder.ForBackup(\"\", \"backup-2\").Result()),\n\t\t\t},\n\t\t\texpectedRes: []string{\"backup-1\", \"backup-2\"},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tharness := newObjectBackupStoreTestHarness(\"foo\", tc.prefix)\n\n\t\t\tfor key, obj := range tc.storageData {\n\t\t\t\trequire.NoError(t, harness.objectStore.PutObject(harness.bucket, key, bytes.NewReader(obj)))\n\t\t\t}\n\n\t\t\tres, err := harness.ListBackups()\n\n\t\t\tvelerotest.AssertErrorMatches(t, tc.expectedErr, err)\n\n\t\t\tsort.Strings(tc.expectedRes)\n\t\t\tsort.Strings(res)\n\n\t\t\tassert.Equal(t, tc.expectedRes, res)\n\t\t})\n\t}\n}\n\nfunc TestPutBackup(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tprefix string\n\t\tmetadata io.Reader\n\t\tcontents io.Reader\n\t\tlog io.Reader\n\t\tpodVolumeBackup io.Reader\n\t\tsnapshots io.Reader\n\t\tbackupItemOperations io.Reader\n\t\tresourceList io.Reader\n\t\tbackupVolumeInfo io.Reader\n\t\texpectedErr string\n\t\texpectedKeys []string\n\t}{\n\t\t{\n\t\t\tname: \"normal case\",\n\t\t\tmetadata: newStringReadSeeker(\"metadata\"),\n\t\t\tcontents: newStringReadSeeker(\"contents\"),\n\t\t\tlog: newStringReadSeeker(\"log\"),\n\t\t\tpodVolumeBackup: newStringReadSeeker(\"podVolumeBackup\"),\n\t\t\tsnapshots: newStringReadSeeker(\"snapshots\"),\n\t\t\tbackupItemOperations: newStringReadSeeker(\"backupItemOperations\"),\n\t\t\tresourceList: newStringReadSeeker(\"resourceList\"),\n\t\t\tbackupVolumeInfo: newStringReadSeeker(\"backupVolumeInfo\"),\n\t\t\texpectedErr: \"\",\n\t\t\texpectedKeys: []string{\n\t\t\t\t\"backups/backup-1/velero-backup.json\",\n\t\t\t\t\"backups/backup-1/backup-1.tar.gz\",\n\t\t\t\t\"backups/backup-1/backup-1-logs.gz\",\n\t\t\t\t\"backups/backup-1/backup-1-podvolumebackups.json.gz\",\n\t\t\t\t\"backups/backup-1/backup-1-volumesnapshots.json.gz\",\n\t\t\t\t\"backups/backup-1/backup-1-itemoperations.json.gz\",\n\t\t\t\t\"backups/backup-1/backup-1-resource-list.json.gz\",\n\t\t\t\t\"backups/backup-1/backup-1-volumeinfo.json.gz\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"normal case with backup store prefix\",\n\t\t\tprefix: \"prefix-1/\",\n\t\t\tmetadata: newStringReadSeeker(\"metadata\"),\n\t\t\tcontents: newStringReadSeeker(\"contents\"),\n\t\t\tlog: newStringReadSeeker(\"log\"),\n\t\t\tpodVolumeBackup: newStringReadSeeker(\"podVolumeBackup\"),\n\t\t\tsnapshots: newStringReadSeeker(\"snapshots\"),\n\t\t\tbackupItemOperations: newStringReadSeeker(\"backupItemOperations\"),\n\t\t\tresourceList: newStringReadSeeker(\"resourceList\"),\n\t\t\tbackupVolumeInfo: newStringReadSeeker(\"backupVolumeInfo\"),\n\t\t\texpectedErr: \"\",\n\t\t\texpectedKeys: []string{\n\t\t\t\t\"prefix-1/backups/backup-1/velero-backup.json\",\n\t\t\t\t\"prefix-1/backups/backup-1/backup-1.tar.gz\",\n\t\t\t\t\"prefix-1/backups/backup-1/backup-1-logs.gz\",\n\t\t\t\t\"prefix-1/backups/backup-1/backup-1-podvolumebackups.json.gz\",\n\t\t\t\t\"prefix-1/backups/backup-1/backup-1-volumesnapshots.json.gz\",\n\t\t\t\t\"prefix-1/backups/backup-1/backup-1-itemoperations.json.gz\",\n\t\t\t\t\"prefix-1/backups/backup-1/backup-1-resource-list.json.gz\",\n\t\t\t\t\"prefix-1/backups/backup-1/backup-1-volumeinfo.json.gz\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"error on metadata upload does not upload data\",\n\t\t\tmetadata: new(errorReader),\n\t\t\tcontents: newStringReadSeeker(\"contents\"),\n\t\t\tlog: newStringReadSeeker(\"log\"),\n\t\t\tpodVolumeBackup: newStringReadSeeker(\"podVolumeBackup\"),\n\t\t\tsnapshots: newStringReadSeeker(\"snapshots\"),\n\t\t\tbackupItemOperations: newStringReadSeeker(\"backupItemOperations\"),\n\t\t\tresourceList: newStringReadSeeker(\"resourceList\"),\n\t\t\tbackupVolumeInfo: newStringReadSeeker(\"backupVolumeInfo\"),\n\t\t\texpectedErr: \"error readers return errors\",\n\t\t\texpectedKeys: []string{\"backups/backup-1/backup-1-logs.gz\"},\n\t\t},\n\t\t{\n\t\t\tname: \"error on data upload deletes metadata\",\n\t\t\tmetadata: newStringReadSeeker(\"metadata\"),\n\t\t\tcontents: new(errorReader),\n\t\t\tlog: newStringReadSeeker(\"log\"),\n\t\t\tsnapshots: newStringReadSeeker(\"snapshots\"),\n\t\t\tbackupItemOperations: newStringReadSeeker(\"backupItemOperations\"),\n\t\t\tresourceList: newStringReadSeeker(\"resourceList\"),\n\t\t\tbackupVolumeInfo: newStringReadSeeker(\"backupVolumeInfo\"),\n\t\t\texpectedErr: \"error readers return errors\",\n\t\t\texpectedKeys: []string{\"backups/backup-1/backup-1-logs.gz\"},\n\t\t},\n\t\t{\n\t\t\tname: \"error on log upload is ok\",\n\t\t\tmetadata: newStringReadSeeker(\"foo\"),\n\t\t\tcontents: newStringReadSeeker(\"bar\"),\n\t\t\tlog: new(errorReader),\n\t\t\tpodVolumeBackup: newStringReadSeeker(\"podVolumeBackup\"),\n\t\t\tsnapshots: newStringReadSeeker(\"snapshots\"),\n\t\t\tbackupItemOperations: newStringReadSeeker(\"backupItemOperations\"),\n\t\t\tresourceList: newStringReadSeeker(\"resourceList\"),\n\t\t\tbackupVolumeInfo: newStringReadSeeker(\"backupVolumeInfo\"),\n\t\t\texpectedErr: \"\",\n\t\t\texpectedKeys: []string{\n\t\t\t\t\"backups/backup-1/velero-backup.json\",\n\t\t\t\t\"backups/backup-1/backup-1.tar.gz\",\n\t\t\t\t\"backups/backup-1/backup-1-podvolumebackups.json.gz\",\n\t\t\t\t\"backups/backup-1/backup-1-volumesnapshots.json.gz\",\n\t\t\t\t\"backups/backup-1/backup-1-itemoperations.json.gz\",\n\t\t\t\t\"backups/backup-1/backup-1-resource-list.json.gz\",\n\t\t\t\t\"backups/backup-1/backup-1-volumeinfo.json.gz\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"data should be uploaded even when metadata is nil\",\n\t\t\tmetadata: nil,\n\t\t\tcontents: newStringReadSeeker(\"contents\"),\n\t\t\tlog: newStringReadSeeker(\"log\"),\n\t\t\tpodVolumeBackup: newStringReadSeeker(\"podVolumeBackup\"),\n\t\t\tsnapshots: newStringReadSeeker(\"snapshots\"),\n\t\t\tresourceList: newStringReadSeeker(\"resourceList\"),\n\t\t\tbackupVolumeInfo: newStringReadSeeker(\"backupVolumeInfo\"),\n\t\t\texpectedErr: \"\",\n\t\t\texpectedKeys: []string{\n\t\t\t\t\"backups/backup-1/backup-1.tar.gz\",\n\t\t\t\t\"backups/backup-1/backup-1-logs.gz\",\n\t\t\t\t\"backups/backup-1/backup-1-podvolumebackups.json.gz\",\n\t\t\t\t\"backups/backup-1/backup-1-volumesnapshots.json.gz\",\n\t\t\t\t\"backups/backup-1/backup-1-resource-list.json.gz\",\n\t\t\t\t\"backups/backup-1/backup-1-volumeinfo.json.gz\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tharness := newObjectBackupStoreTestHarness(\"foo\", tc.prefix)\n\n\t\t\tbackupInfo := BackupInfo{\n\t\t\t\tName: \"backup-1\",\n\t\t\t\tMetadata: tc.metadata,\n\t\t\t\tContents: tc.contents,\n\t\t\t\tLog: tc.log,\n\t\t\t\tPodVolumeBackups: tc.podVolumeBackup,\n\t\t\t\tVolumeSnapshots: tc.snapshots,\n\t\t\t\tBackupItemOperations: tc.backupItemOperations,\n\t\t\t\tBackupResourceList: tc.resourceList,\n\t\t\t\tBackupVolumeInfo: tc.backupVolumeInfo,\n\t\t\t}\n\t\t\terr := harness.PutBackup(backupInfo)\n\n\t\t\tvelerotest.AssertErrorMatches(t, tc.expectedErr, err)\n\t\t\tassert.Len(t, harness.objectStore.Data[harness.bucket], len(tc.expectedKeys))\n\t\t\tfor _, key := range tc.expectedKeys {\n\t\t\t\tassert.Contains(t, harness.objectStore.Data[harness.bucket], key)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetBackupMetadata(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackupName string\n\t\tkey string\n\t\tobj metav1.Object\n\t\twantErr error\n\t}{\n\t\t{\n\t\t\tname: \"metadata file returns correctly\",\n\t\t\tbackupName: \"foo\",\n\t\t\tkey: \"backups/foo/velero-backup.json\",\n\t\t\tobj: builder.ForBackup(velerov1api.DefaultNamespace, \"foo\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"no metadata file returns an error\",\n\t\t\tbackupName: \"foo\",\n\t\t\twantErr: errors.New(\"key not found\"),\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tharness := newObjectBackupStoreTestHarness(\"test-bucket\", \"\")\n\n\t\t\tif tc.obj != nil {\n\t\t\t\tjsonBytes, err := json.Marshal(tc.obj)\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\trequire.NoError(t, harness.objectStore.PutObject(harness.bucket, tc.key, bytes.NewReader(jsonBytes)))\n\t\t\t}\n\n\t\t\tres, err := harness.GetBackupMetadata(tc.backupName)\n\t\t\tif tc.wantErr != nil {\n\t\t\t\tassert.Equal(t, tc.wantErr, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\tassert.Equal(t, tc.obj.GetNamespace(), res.Namespace)\n\t\t\t\tassert.Equal(t, tc.obj.GetName(), res.Name)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetBackupVolumeSnapshots(t *testing.T) {\n\tharness := newObjectBackupStoreTestHarness(\"test-bucket\", \"\")\n\n\t// volumesnapshots file not found should not error\n\tharness.objectStore.PutObject(harness.bucket, \"backups/test-backup/velero-backup.json\", newStringReadSeeker(\"foo\"))\n\tres, err := harness.GetBackupVolumeSnapshots(\"test-backup\")\n\trequire.NoError(t, err)\n\tassert.Nil(t, res)\n\n\t// volumesnapshots file containing invalid data should error\n\tharness.objectStore.PutObject(harness.bucket, \"backups/test-backup/test-backup-volumesnapshots.json.gz\", newStringReadSeeker(\"foo\"))\n\t_, err = harness.GetBackupVolumeSnapshots(\"test-backup\")\n\trequire.Error(t, err)\n\n\t// volumesnapshots file containing gzipped json data should return correctly\n\tsnapshots := []*volume.Snapshot{\n\t\t{\n\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\tBackupName: \"test-backup\",\n\t\t\t\tPersistentVolumeName: \"pv-1\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\tBackupName: \"test-backup\",\n\t\t\t\tPersistentVolumeName: \"pv-2\",\n\t\t\t},\n\t\t},\n\t}\n\n\tobj := new(bytes.Buffer)\n\tgzw := gzip.NewWriter(obj)\n\n\trequire.NoError(t, json.NewEncoder(gzw).Encode(snapshots))\n\trequire.NoError(t, gzw.Close())\n\trequire.NoError(t, harness.objectStore.PutObject(harness.bucket, \"backups/test-backup/test-backup-volumesnapshots.json.gz\", obj))\n\n\tres, err = harness.GetBackupVolumeSnapshots(\"test-backup\")\n\trequire.NoError(t, err)\n\tassert.Equal(t, snapshots, res)\n}\n\nfunc TestGetBackupItemOperations(t *testing.T) {\n\tharness := newObjectBackupStoreTestHarness(\"test-bucket\", \"\")\n\n\t// itemoperations file not found should not error\n\tharness.objectStore.PutObject(harness.bucket, \"backups/test-backup/velero-backup.json\", newStringReadSeeker(\"foo\"))\n\tres, err := harness.GetBackupItemOperations(\"test-backup\")\n\trequire.NoError(t, err)\n\tassert.Nil(t, res)\n\n\t// itemoperations file containing invalid data should error\n\tharness.objectStore.PutObject(harness.bucket, \"backups/test-backup/test-backup-itemoperations.json.gz\", newStringReadSeeker(\"foo\"))\n\t_, err = harness.GetBackupItemOperations(\"test-backup\")\n\trequire.Error(t, err)\n\n\t// itemoperations file containing gzipped json data should return correctly\n\toperations := []*itemoperation.BackupOperation{\n\t\t{\n\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\tBackupName: \"test-backup\",\n\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\tNamespace: \"ns\",\n\t\t\t\t\tName: \"item-1\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tSpec: itemoperation.BackupOperationSpec{\n\t\t\t\tBackupName: \"test-backup\",\n\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\tNamespace: \"ns\",\n\t\t\t\t\tName: \"item-2\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tobj := new(bytes.Buffer)\n\tgzw := gzip.NewWriter(obj)\n\n\trequire.NoError(t, json.NewEncoder(gzw).Encode(operations))\n\trequire.NoError(t, gzw.Close())\n\trequire.NoError(t, harness.objectStore.PutObject(harness.bucket, \"backups/test-backup/test-backup-itemoperations.json.gz\", obj))\n\n\tres, err = harness.GetBackupItemOperations(\"test-backup\")\n\trequire.NoError(t, err)\n\tassert.Equal(t, operations, res)\n}\n\nfunc TestGetRestoreItemOperations(t *testing.T) {\n\tharness := newObjectBackupStoreTestHarness(\"test-bucket\", \"\")\n\n\t// itemoperations file not found should not error\n\tres, err := harness.GetRestoreItemOperations(\"test-restore\")\n\trequire.NoError(t, err)\n\tassert.Nil(t, res)\n\n\t// itemoperations file containing invalid data should error\n\tharness.objectStore.PutObject(harness.bucket, \"restores/test-restore/restore-test-restore-itemoperations.json.gz\", newStringReadSeeker(\"foo\"))\n\t_, err = harness.GetRestoreItemOperations(\"test-restore\")\n\trequire.Error(t, err)\n\n\t// itemoperations file containing gzipped json data should return correctly\n\toperations := []*itemoperation.RestoreOperation{\n\t\t{\n\t\t\tSpec: itemoperation.RestoreOperationSpec{\n\t\t\t\tRestoreName: \"test-restore\",\n\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\tNamespace: \"ns\",\n\t\t\t\t\tName: \"item-1\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tSpec: itemoperation.RestoreOperationSpec{\n\t\t\t\tRestoreName: \"test-restore\",\n\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\tNamespace: \"ns\",\n\t\t\t\t\tName: \"item-2\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tobj := new(bytes.Buffer)\n\tgzw := gzip.NewWriter(obj)\n\n\trequire.NoError(t, json.NewEncoder(gzw).Encode(operations))\n\trequire.NoError(t, gzw.Close())\n\trequire.NoError(t, harness.objectStore.PutObject(harness.bucket, \"restores/test-restore/restore-test-restore-itemoperations.json.gz\", obj))\n\n\tres, err = harness.GetRestoreItemOperations(\"test-restore\")\n\trequire.NoError(t, err)\n\tassert.Equal(t, operations, res)\n}\n\nfunc TestGetBackupContents(t *testing.T) {\n\tharness := newObjectBackupStoreTestHarness(\"test-bucket\", \"\")\n\n\tharness.objectStore.PutObject(harness.bucket, \"backups/test-backup/test-backup.tar.gz\", newStringReadSeeker(\"foo\"))\n\n\trc, err := harness.GetBackupContents(\"test-backup\")\n\trequire.NoError(t, err)\n\trequire.NotNil(t, rc)\n\n\tdata, err := io.ReadAll(rc)\n\trequire.NoError(t, err)\n\tassert.Equal(t, \"foo\", string(data))\n}\n\nfunc TestDeleteBackup(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tprefix string\n\t\tlistObjectsError error\n\t\tdeleteErrors []error\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"normal case\",\n\t\t},\n\t\t{\n\t\t\tname: \"normal case with backup store prefix\",\n\t\t\tprefix: \"velero-backups/\",\n\t\t},\n\t\t{\n\t\t\tname: \"some delete errors, do as much as we can\",\n\t\t\tdeleteErrors: []error{errors.New(\"a\"), nil, errors.New(\"c\")},\n\t\t\texpectedErr: \"[a, c]\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tobjectStore := new(providermocks.ObjectStore)\n\t\t\tbackupStore := &objectBackupStore{\n\t\t\t\tobjectStore: objectStore,\n\t\t\t\tbucket: \"test-bucket\",\n\t\t\t\tlayout: NewObjectStoreLayout(test.prefix),\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t}\n\t\t\tdefer objectStore.AssertExpectations(t)\n\n\t\t\tobjects := []string{test.prefix + \"backups/bak/velero-backup.json\", test.prefix + \"backups/bak/bak.tar.gz\", test.prefix + \"backups/bak/bak.log.gz\"}\n\n\t\t\tobjectStore.On(\"ListObjects\", backupStore.bucket, test.prefix+\"backups/bak/\").Return(objects, test.listObjectsError)\n\t\t\tfor i, obj := range objects {\n\t\t\t\tvar err error\n\t\t\t\tif i < len(test.deleteErrors) {\n\t\t\t\t\terr = test.deleteErrors[i]\n\t\t\t\t}\n\n\t\t\t\tobjectStore.On(\"DeleteObject\", backupStore.bucket, obj).Return(err)\n\t\t\t}\n\n\t\t\terr := backupStore.DeleteBackup(\"bak\")\n\n\t\t\tvelerotest.AssertErrorMatches(t, test.expectedErr, err)\n\t\t})\n\t}\n}\n\nfunc TestDeleteRestore(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tprefix string\n\t\tlistObjectsError error\n\t\tdeleteErrors []error\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"normal case\",\n\t\t},\n\t\t{\n\t\t\tname: \"normal case with backup store prefix\",\n\t\t\tprefix: \"velero-backups/\",\n\t\t},\n\t\t{\n\t\t\tname: \"some delete errors, do as much as we can\",\n\t\t\tdeleteErrors: []error{errors.New(\"a\"), nil, errors.New(\"c\")},\n\t\t\texpectedErr: \"[a, c]\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tobjectStore := new(providermocks.ObjectStore)\n\t\t\tbackupStore := &objectBackupStore{\n\t\t\t\tobjectStore: objectStore,\n\t\t\t\tbucket: \"test-bucket\",\n\t\t\t\tlayout: NewObjectStoreLayout(test.prefix),\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t}\n\t\t\tdefer objectStore.AssertExpectations(t)\n\n\t\t\tobjects := []string{test.prefix + \"restores/bak/velero-restore.json\", test.prefix + \"restores/bak/bak.tar.gz\", test.prefix + \"restores/bak/bak.log.gz\"}\n\n\t\t\tobjectStore.On(\"ListObjects\", backupStore.bucket, test.prefix+\"restores/bak/\").Return(objects, test.listObjectsError)\n\t\t\tfor i, obj := range objects {\n\t\t\t\tvar err error\n\t\t\t\tif i < len(test.deleteErrors) {\n\t\t\t\t\terr = test.deleteErrors[i]\n\t\t\t\t}\n\n\t\t\t\tobjectStore.On(\"DeleteObject\", backupStore.bucket, obj).Return(err)\n\t\t\t}\n\n\t\t\terr := backupStore.DeleteRestore(\"bak\")\n\n\t\t\tvelerotest.AssertErrorMatches(t, test.expectedErr, err)\n\t\t})\n\t}\n}\n\nfunc TestGetDownloadURL(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\ttargetName string\n\t\texpectedKeyByKind map[velerov1api.DownloadTargetKind]string\n\t\tprefix string\n\t}{\n\t\t{\n\t\t\tname: \"backup\",\n\t\t\ttargetName: \"my-backup\",\n\t\t\texpectedKeyByKind: map[velerov1api.DownloadTargetKind]string{\n\t\t\t\tvelerov1api.DownloadTargetKindBackupContents: \"backups/my-backup/my-backup.tar.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindBackupLog: \"backups/my-backup/my-backup-logs.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindBackupVolumeSnapshots: \"backups/my-backup/my-backup-volumesnapshots.json.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindBackupItemOperations: \"backups/my-backup/my-backup-itemoperations.json.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindBackupResourceList: \"backups/my-backup/my-backup-resource-list.json.gz\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"backup with prefix\",\n\t\t\ttargetName: \"my-backup\",\n\t\t\tprefix: \"velero-backups/\",\n\t\t\texpectedKeyByKind: map[velerov1api.DownloadTargetKind]string{\n\t\t\t\tvelerov1api.DownloadTargetKindBackupContents: \"velero-backups/backups/my-backup/my-backup.tar.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindBackupLog: \"velero-backups/backups/my-backup/my-backup-logs.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindBackupVolumeSnapshots: \"velero-backups/backups/my-backup/my-backup-volumesnapshots.json.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindBackupItemOperations: \"velero-backups/backups/my-backup/my-backup-itemoperations.json.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindBackupResourceList: \"velero-backups/backups/my-backup/my-backup-resource-list.json.gz\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"backup with multiple dashes\",\n\t\t\ttargetName: \"b-cool-20170913154901-20170913154902\",\n\t\t\texpectedKeyByKind: map[velerov1api.DownloadTargetKind]string{\n\t\t\t\tvelerov1api.DownloadTargetKindBackupContents: \"backups/b-cool-20170913154901-20170913154902/b-cool-20170913154901-20170913154902.tar.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindBackupLog: \"backups/b-cool-20170913154901-20170913154902/b-cool-20170913154901-20170913154902-logs.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindBackupVolumeSnapshots: \"backups/b-cool-20170913154901-20170913154902/b-cool-20170913154901-20170913154902-volumesnapshots.json.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindBackupItemOperations: \"backups/b-cool-20170913154901-20170913154902/b-cool-20170913154901-20170913154902-itemoperations.json.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindBackupResourceList: \"backups/b-cool-20170913154901-20170913154902/b-cool-20170913154901-20170913154902-resource-list.json.gz\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"scheduled backup\",\n\t\t\ttargetName: \"my-backup-20170913154901\",\n\t\t\texpectedKeyByKind: map[velerov1api.DownloadTargetKind]string{\n\t\t\t\tvelerov1api.DownloadTargetKindBackupContents: \"backups/my-backup-20170913154901/my-backup-20170913154901.tar.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindBackupLog: \"backups/my-backup-20170913154901/my-backup-20170913154901-logs.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindBackupVolumeSnapshots: \"backups/my-backup-20170913154901/my-backup-20170913154901-volumesnapshots.json.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindBackupItemOperations: \"backups/my-backup-20170913154901/my-backup-20170913154901-itemoperations.json.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindBackupResourceList: \"backups/my-backup-20170913154901/my-backup-20170913154901-resource-list.json.gz\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"scheduled backup with prefix\",\n\t\t\ttargetName: \"my-backup-20170913154901\",\n\t\t\tprefix: \"velero-backups/\",\n\t\t\texpectedKeyByKind: map[velerov1api.DownloadTargetKind]string{\n\t\t\t\tvelerov1api.DownloadTargetKindBackupContents: \"velero-backups/backups/my-backup-20170913154901/my-backup-20170913154901.tar.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindBackupLog: \"velero-backups/backups/my-backup-20170913154901/my-backup-20170913154901-logs.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindBackupVolumeSnapshots: \"velero-backups/backups/my-backup-20170913154901/my-backup-20170913154901-volumesnapshots.json.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindBackupItemOperations: \"velero-backups/backups/my-backup-20170913154901/my-backup-20170913154901-itemoperations.json.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindBackupResourceList: \"velero-backups/backups/my-backup-20170913154901/my-backup-20170913154901-resource-list.json.gz\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"restore\",\n\t\t\ttargetName: \"my-backup\",\n\t\t\texpectedKeyByKind: map[velerov1api.DownloadTargetKind]string{\n\t\t\t\tvelerov1api.DownloadTargetKindRestoreLog: \"restores/my-backup/restore-my-backup-logs.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindRestoreResults: \"restores/my-backup/restore-my-backup-results.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindRestoreItemOperations: \"restores/my-backup/restore-my-backup-itemoperations.json.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindRestoreResourceList: \"restores/my-backup/restore-my-backup-resource-list.json.gz\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"restore with prefix\",\n\t\t\ttargetName: \"my-backup\",\n\t\t\tprefix: \"velero-backups/\",\n\t\t\texpectedKeyByKind: map[velerov1api.DownloadTargetKind]string{\n\t\t\t\tvelerov1api.DownloadTargetKindRestoreLog: \"velero-backups/restores/my-backup/restore-my-backup-logs.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindRestoreResults: \"velero-backups/restores/my-backup/restore-my-backup-results.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindRestoreItemOperations: \"velero-backups/restores/my-backup/restore-my-backup-itemoperations.json.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindRestoreResourceList: \"velero-backups/restores/my-backup/restore-my-backup-resource-list.json.gz\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"restore with multiple dashes\",\n\t\t\ttargetName: \"b-cool-20170913154901-20170913154902\",\n\t\t\texpectedKeyByKind: map[velerov1api.DownloadTargetKind]string{\n\t\t\t\tvelerov1api.DownloadTargetKindRestoreLog: \"restores/b-cool-20170913154901-20170913154902/restore-b-cool-20170913154901-20170913154902-logs.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindRestoreResults: \"restores/b-cool-20170913154901-20170913154902/restore-b-cool-20170913154901-20170913154902-results.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindRestoreItemOperations: \"restores/b-cool-20170913154901-20170913154902/restore-b-cool-20170913154901-20170913154902-itemoperations.json.gz\",\n\t\t\t\tvelerov1api.DownloadTargetKindRestoreResourceList: \"restores/b-cool-20170913154901-20170913154902/restore-b-cool-20170913154901-20170913154902-resource-list.json.gz\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"\",\n\t\t\ttargetName: \"my-backup\",\n\t\t\texpectedKeyByKind: map[velerov1api.DownloadTargetKind]string{\n\t\t\t\tvelerov1api.DownloadTargetKindBackupVolumeInfos: \"backups/my-backup/my-backup-volumeinfo.json.gz\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tharness := newObjectBackupStoreTestHarness(\"test-bucket\", test.prefix)\n\n\t\t\tfor kind, expectedKey := range test.expectedKeyByKind {\n\t\t\t\tt.Run(string(kind), func(t *testing.T) {\n\t\t\t\t\trequire.NoError(t, harness.objectStore.PutObject(\"test-bucket\", expectedKey, newStringReadSeeker(\"foo\")))\n\n\t\t\t\t\turl, err := harness.GetDownloadURL(velerov1api.DownloadTarget{Kind: kind, Name: test.targetName})\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\tassert.Equal(t, \"a-url\", url)\n\t\t\t\t})\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetCSIVolumeSnapshotClasses(t *testing.T) {\n\tharness := newObjectBackupStoreTestHarness(\"test-bucket\", \"\")\n\n\t// file not found should not error\n\tres, err := harness.GetCSIVolumeSnapshotClasses(\"test-backup\")\n\trequire.NoError(t, err)\n\tassert.Nil(t, res)\n\n\t// file containing invalid data should error\n\tharness.objectStore.PutObject(harness.bucket, \"backups/test-backup/test-backup-csi-volumesnapshotclasses.json.gz\", newStringReadSeeker(\"foo\"))\n\t_, err = harness.GetCSIVolumeSnapshotClasses(\"test-backup\")\n\trequire.Error(t, err)\n\n\t// file containing gzipped json data should return correctly\n\tclasses := []*snapshotv1api.VolumeSnapshotClass{\n\t\t{\n\t\t\tDriver: \"driver\",\n\t\t},\n\t}\n\n\tobj := new(bytes.Buffer)\n\tgzw := gzip.NewWriter(obj)\n\n\trequire.NoError(t, json.NewEncoder(gzw).Encode(classes))\n\trequire.NoError(t, gzw.Close())\n\trequire.NoError(t, harness.objectStore.PutObject(harness.bucket, \"backups/test-backup/test-backup-csi-volumesnapshotclasses.json.gz\", obj))\n\n\tres, err = harness.GetCSIVolumeSnapshotClasses(\"test-backup\")\n\trequire.NoError(t, err)\n\tassert.Equal(t, classes, res)\n}\n\nfunc TestGetCSIVolumeSnapshots(t *testing.T) {\n\tharness := newObjectBackupStoreTestHarness(\"test-bucket\", \"\")\n\n\t// file not found should not error\n\tres, err := harness.GetCSIVolumeSnapshots(\"test-backup\")\n\trequire.NoError(t, err)\n\tassert.Nil(t, res)\n\n\t// file containing invalid data should error\n\tharness.objectStore.PutObject(harness.bucket, \"backups/test-backup/test-backup-csi-volumesnapshots.json.gz\", newStringReadSeeker(\"foo\"))\n\t_, err = harness.GetCSIVolumeSnapshots(\"test-backup\")\n\trequire.Error(t, err)\n\n\t// file containing gzipped json data should return correctly\n\tsnapshots := []*snapshotv1api.VolumeSnapshot{\n\t\t{\n\t\t\tSpec: snapshotv1api.VolumeSnapshotSpec{\n\t\t\t\tSource: snapshotv1api.VolumeSnapshotSource{\n\t\t\t\t\tVolumeSnapshotContentName: nil,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tobj := new(bytes.Buffer)\n\tgzw := gzip.NewWriter(obj)\n\n\trequire.NoError(t, json.NewEncoder(gzw).Encode(snapshots))\n\trequire.NoError(t, gzw.Close())\n\trequire.NoError(t, harness.objectStore.PutObject(harness.bucket, \"backups/test-backup/test-backup-csi-volumesnapshots.json.gz\", obj))\n\n\tres, err = harness.GetCSIVolumeSnapshots(\"test-backup\")\n\trequire.NoError(t, err)\n\tassert.Equal(t, snapshots, res)\n}\n\ntype objectStoreGetter map[string]velero.ObjectStore\n\nfunc (osg objectStoreGetter) GetObjectStore(provider string) (velero.ObjectStore, error) {\n\tres, ok := osg[provider]\n\tif !ok {\n\t\treturn nil, errors.New(\"object store not found\")\n\t}\n\n\treturn res, nil\n}\n\n// TestNewObjectBackupStore runs the NewObjectBackupStoreGetter constructor and ensures\n// that it provides a BackupStore with a correctly constructed ObjectBackupStore or\n// that an appropriate error is returned.\nfunc TestNewObjectBackupStoreGetter(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tlocation *velerov1api.BackupStorageLocation\n\t\tobjectStoreGetter objectStoreGetter\n\t\tcredFileStore credentials.FileStore\n\t\tfileStoreErr error\n\t\twantBucket string\n\t\twantPrefix string\n\t\twantErr string\n\t}{\n\t\t{\n\t\t\tname: \"when location does not use object storage, a backup store can't be retrieved\",\n\t\t\tlocation: new(velerov1api.BackupStorageLocation),\n\t\t\tcredFileStore: velerotest.NewFakeCredentialsFileStore(\"\", nil),\n\t\t\twantErr: \"backup storage location does not use object storage\",\n\t\t},\n\t\t{\n\t\t\tname: \"when object storage does not specify a provider, a backup store can't be retrieved\",\n\t\t\tlocation: builder.ForBackupStorageLocation(\"\", \"\").Bucket(\"\").Result(),\n\t\t\tcredFileStore: velerotest.NewFakeCredentialsFileStore(\"\", nil),\n\t\t\twantErr: \"object storage provider name must not be empty\",\n\t\t},\n\t\t{\n\t\t\tname: \"when the Bucket field has a '/' in the middle, a backup store can't be retrieved\",\n\t\t\tlocation: builder.ForBackupStorageLocation(\"\", \"\").Provider(\"provider-1\").Bucket(\"invalid/bucket\").Result(),\n\t\t\tcredFileStore: velerotest.NewFakeCredentialsFileStore(\"\", nil),\n\t\t\twantErr: \"backup storage location's bucket name \\\"invalid/bucket\\\" must not contain a '/' (if using a prefix, put it in the 'Prefix' field instead)\",\n\t\t},\n\t\t{\n\t\t\tname: \"when the credential selector is invalid, a backup store can't be retrieved\",\n\t\t\tlocation: builder.ForBackupStorageLocation(\"\", \"\").Provider(\"provider-1\").Bucket(\"bucket\").Credential(\n\t\t\t\tbuilder.ForSecretKeySelector(\"does-not-exist\", \"does-not-exist\").Result(),\n\t\t\t).Result(),\n\t\t\tcredFileStore: velerotest.NewFakeCredentialsFileStore(\"\", fmt.Errorf(\"secret does not exist\")),\n\t\t\twantErr: \"unable to get credentials: secret does not exist\",\n\t\t},\n\t\t{\n\t\t\tname: \"when Bucket has a leading and trailing slash, they are both stripped\",\n\t\t\tlocation: builder.ForBackupStorageLocation(\"\", \"\").Provider(\"provider-1\").Bucket(\"/bucket/\").Result(),\n\t\t\tobjectStoreGetter: objectStoreGetter{\n\t\t\t\t\"provider-1\": newInMemoryObjectStore(\"bucket\"),\n\t\t\t},\n\t\t\tcredFileStore: velerotest.NewFakeCredentialsFileStore(\"\", nil),\n\t\t\twantBucket: \"bucket\",\n\t\t},\n\t\t{\n\t\t\tname: \"when Prefix has a leading and trailing slash, the leading slash is stripped and the trailing slash is left\",\n\t\t\tlocation: builder.ForBackupStorageLocation(\"\", \"\").Provider(\"provider-1\").Bucket(\"bucket\").Prefix(\"/prefix/\").Result(),\n\t\t\tobjectStoreGetter: objectStoreGetter{\n\t\t\t\t\"provider-1\": newInMemoryObjectStore(\"bucket\"),\n\t\t\t},\n\t\t\tcredFileStore: velerotest.NewFakeCredentialsFileStore(\"\", nil),\n\t\t\twantBucket: \"bucket\",\n\t\t\twantPrefix: \"prefix/\",\n\t\t},\n\t\t{\n\t\t\tname: \"when Prefix has no leading or trailing slash, a trailing slash is added\",\n\t\t\tlocation: builder.ForBackupStorageLocation(\"\", \"\").Provider(\"provider-1\").Bucket(\"bucket\").Prefix(\"prefix\").Result(),\n\t\t\tobjectStoreGetter: objectStoreGetter{\n\t\t\t\t\"provider-1\": newInMemoryObjectStore(\"bucket\"),\n\t\t\t},\n\t\t\tcredFileStore: velerotest.NewFakeCredentialsFileStore(\"\", nil),\n\t\t\twantBucket: \"bucket\",\n\t\t\twantPrefix: \"prefix/\",\n\t\t},\n\t\t{\n\t\t\tname: \"when the Bucket field is an MRAP ARN, it should be valid\",\n\t\t\tlocation: builder.ForBackupStorageLocation(\"\", \"\").Provider(\"provider-1\").Bucket(\"arn:aws:s3::123456789012:accesspoint/abcdef0123456.mrap\").Result(),\n\t\t\tobjectStoreGetter: objectStoreGetter{\n\t\t\t\t\"provider-1\": newInMemoryObjectStore(\"arn:aws:s3::123456789012:accesspoint/abcdef0123456.mrap\"),\n\t\t\t},\n\t\t\tcredFileStore: velerotest.NewFakeCredentialsFileStore(\"\", nil),\n\t\t\twantBucket: \"arn:aws:s3::123456789012:accesspoint/abcdef0123456.mrap\",\n\t\t},\n\t\t{\n\t\t\tname: \"when the Bucket field is an MRAP ARN with trailing slash, it should be valid and trimmed\",\n\t\t\tlocation: builder.ForBackupStorageLocation(\"\", \"\").Provider(\"provider-1\").Bucket(\"arn:aws:s3::123456789012:accesspoint/abcdef0123456.mrap/\").Result(),\n\t\t\tobjectStoreGetter: objectStoreGetter{\n\t\t\t\t\"provider-1\": newInMemoryObjectStore(\"arn:aws:s3::123456789012:accesspoint/abcdef0123456.mrap\"),\n\t\t\t},\n\t\t\tcredFileStore: velerotest.NewFakeCredentialsFileStore(\"\", nil),\n\t\t\twantBucket: \"arn:aws:s3::123456789012:accesspoint/abcdef0123456.mrap\",\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tgetter := NewObjectBackupStoreGetter(tc.credFileStore)\n\t\t\tres, err := getter.Get(tc.location, tc.objectStoreGetter, velerotest.NewLogger())\n\t\t\tif tc.wantErr != \"\" {\n\t\t\t\trequire.EqualError(t, err, tc.wantErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\tstore, ok := res.(*objectBackupStore)\n\t\t\t\trequire.True(t, ok)\n\n\t\t\t\tassert.Equal(t, tc.wantBucket, store.bucket)\n\t\t\t\tassert.Equal(t, tc.wantPrefix, store.layout.rootPrefix)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestNewObjectBackupStoreGetterConfig runs the NewObjectBackupStoreGetter constructor and ensures\n// that it initializes the ObjectBackupStore with the correct config.\nfunc TestNewObjectBackupStoreGetterConfig(t *testing.T) {\n\tprovider := \"provider\"\n\tbucket := \"bucket\"\n\n\ttests := []struct {\n\t\tname string\n\t\tlocation *velerov1api.BackupStorageLocation\n\t\tgetter ObjectBackupStoreGetter\n\t\tcredentialPath string\n\t\twantConfig map[string]string\n\t}{\n\t\t{\n\t\t\tname: \"location with bucket but no prefix has config initialized with bucket and empty prefix\",\n\t\t\tlocation: builder.ForBackupStorageLocation(\"\", \"\").Provider(provider).Bucket(bucket).Result(),\n\t\t\tgetter: NewObjectBackupStoreGetter(velerotest.NewFakeCredentialsFileStore(\"\", nil)),\n\t\t\twantConfig: map[string]string{\n\t\t\t\t\"bucket\": \"bucket\",\n\t\t\t\t\"prefix\": \"\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"location with bucket and prefix has config initialized with bucket and prefix\",\n\t\t\tlocation: builder.ForBackupStorageLocation(\"\", \"\").Provider(provider).Bucket(bucket).Prefix(\"prefix\").Result(),\n\t\t\tgetter: NewObjectBackupStoreGetter(velerotest.NewFakeCredentialsFileStore(\"\", nil)),\n\t\t\twantConfig: map[string]string{\n\t\t\t\t\"bucket\": \"bucket\",\n\t\t\t\t\"prefix\": \"prefix\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"location with CACert is initialized with caCert\",\n\t\t\tlocation: builder.ForBackupStorageLocation(\"\", \"\").Provider(provider).Bucket(bucket).CACert([]byte(\"cacert-data\")).Result(),\n\t\t\tgetter: NewObjectBackupStoreGetter(velerotest.NewFakeCredentialsFileStore(\"\", nil)),\n\t\t\twantConfig: map[string]string{\n\t\t\t\t\"bucket\": \"bucket\",\n\t\t\t\t\"prefix\": \"\",\n\t\t\t\t\"caCert\": \"cacert-data\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"location with Credential is initialized with path of serialized secret\",\n\t\t\tlocation: builder.ForBackupStorageLocation(\"\", \"\").Provider(provider).Bucket(bucket).Credential(\n\t\t\t\tbuilder.ForSecretKeySelector(\"does-not-exist\", \"does-not-exist\").Result(),\n\t\t\t).Result(),\n\t\t\tgetter: NewObjectBackupStoreGetter(velerotest.NewFakeCredentialsFileStore(\"/tmp/credentials/secret-file\", nil)),\n\t\t\twantConfig: map[string]string{\n\t\t\t\t\"bucket\": \"bucket\",\n\t\t\t\t\"prefix\": \"\",\n\t\t\t\t\"credentialsFile\": \"/tmp/credentials/secret-file\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"location with CACertRef is initialized with caCert from secret\",\n\t\t\tlocation: builder.ForBackupStorageLocation(\"\", \"\").Provider(provider).Bucket(bucket).CACertRef(\n\t\t\t\tbuilder.ForSecretKeySelector(\"cacert-secret\", \"ca.crt\").Result(),\n\t\t\t).Result(),\n\t\t\tgetter: NewObjectBackupStoreGetterWithSecretStore(\n\t\t\t\tvelerotest.NewFakeCredentialsFileStore(\"\", nil),\n\t\t\t\tvelerotest.NewFakeCredentialsSecretStore(\"cacert-from-secret\", nil),\n\t\t\t),\n\t\t\twantConfig: map[string]string{\n\t\t\t\t\"bucket\": \"bucket\",\n\t\t\t\t\"prefix\": \"\",\n\t\t\t\t\"caCert\": \"cacert-from-secret\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"location with CACertRef and no SecretStore uses no caCert\",\n\t\t\tlocation: builder.ForBackupStorageLocation(\"\", \"\").Provider(provider).Bucket(bucket).CACertRef(\n\t\t\t\tbuilder.ForSecretKeySelector(\"cacert-secret\", \"ca.crt\").Result(),\n\t\t\t).Result(),\n\t\t\tgetter: NewObjectBackupStoreGetter(velerotest.NewFakeCredentialsFileStore(\"\", nil)),\n\t\t\twantConfig: map[string]string{\n\t\t\t\t\"bucket\": \"bucket\",\n\t\t\t\t\"prefix\": \"\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tobjStore := newInMemoryObjectStore(bucket)\n\t\t\tobjStoreGetter := &objectStoreGetter{provider: objStore}\n\n\t\t\t_, err := tc.getter.Get(tc.location, objStoreGetter, velerotest.NewLogger())\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.Equal(t, tc.wantConfig, objStore.Config)\n\t\t})\n\t}\n}\n\nfunc TestGetBackupVolumeInfos(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tvolumeInfo []*volume.BackupVolumeInfo\n\t\tvolumeInfoStr string\n\t\texpectedErr string\n\t\texpectedResult []*volume.BackupVolumeInfo\n\t}{\n\t\t{\n\t\t\tname: \"No VolumeInfos, expect no error.\",\n\t\t},\n\t\t{\n\t\t\tname: \"Valid BackupVolumeInfo, should pass.\",\n\t\t\tvolumeInfo: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"pvcName\",\n\t\t\t\t\tPVName: \"pvName\",\n\t\t\t\t\tSkipped: true,\n\t\t\t\t\tSnapshotDataMoved: false,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedResult: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"pvcName\",\n\t\t\t\t\tPVName: \"pvName\",\n\t\t\t\t\tSkipped: true,\n\t\t\t\t\tSnapshotDataMoved: false,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Invalid BackupVolumeInfo string, should also pass.\",\n\t\t\tvolumeInfoStr: `[{\"abc\": \"123\", \"def\": \"456\", \"pvcName\": \"pvcName\"}]`,\n\t\t\texpectedResult: []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"pvcName\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tharness := newObjectBackupStoreTestHarness(\"test-bucket\", \"\")\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tif tc.volumeInfo != nil {\n\t\t\t\tobj := new(bytes.Buffer)\n\t\t\t\tgzw := gzip.NewWriter(obj)\n\n\t\t\t\trequire.NoError(t, json.NewEncoder(gzw).Encode(tc.volumeInfo))\n\t\t\t\trequire.NoError(t, gzw.Close())\n\t\t\t\tharness.objectStore.PutObject(harness.bucket, \"backups/test-backup/test-backup-volumeinfo.json.gz\", obj)\n\t\t\t}\n\n\t\t\tif tc.volumeInfoStr != \"\" {\n\t\t\t\tobj := new(bytes.Buffer)\n\t\t\t\tgzw := gzip.NewWriter(obj)\n\t\t\t\t_, err := gzw.Write([]byte(tc.volumeInfoStr))\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\trequire.NoError(t, gzw.Close())\n\t\t\t\tharness.objectStore.PutObject(harness.bucket, \"backups/test-backup/test-backup-volumeinfo.json.gz\", obj)\n\t\t\t}\n\n\t\t\tresult, err := harness.GetBackupVolumeInfos(\"test-backup\")\n\t\t\tif tc.expectedErr != \"\" {\n\t\t\t\trequire.Equal(t, tc.expectedErr, err.Error())\n\t\t\t} else {\n\t\t\t\tif err != nil {\n\t\t\t\t\tfmt.Println(err.Error())\n\t\t\t\t}\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif len(tc.expectedResult) > 0 {\n\t\t\t\trequire.Equal(t, tc.expectedResult, result)\n\t\t\t}\n\t\t})\n\t}\n}\nfunc TestGetRestoreResults(t *testing.T) {\n\tharness := newObjectBackupStoreTestHarness(\"test-bucket\", \"\")\n\n\t// file not found should not error\n\t_, err := harness.GetRestoreResults(\"test-restore\")\n\trequire.NoError(t, err)\n\n\t// file containing invalid data should error\n\tharness.objectStore.PutObject(harness.bucket, \"restores/test-restore/restore-test-restore-results.gz\", newStringReadSeeker(\"foo\"))\n\t_, err = harness.GetRestoreResults(\"test-restore\")\n\trequire.Error(t, err)\n\n\t// file containing gzipped json data should return correctly\n\tcontents := map[string]results.Result{\n\t\t\"warnings\": {Cluster: []string{\"cluster warning\"}},\n\t\t\"errors\": {Namespaces: map[string][]string{\"test-ns\": {\"namespace error\"}}},\n\t}\n\tobj := new(bytes.Buffer)\n\tgzw := gzip.NewWriter(obj)\n\n\trequire.NoError(t, json.NewEncoder(gzw).Encode(contents))\n\trequire.NoError(t, gzw.Close())\n\trequire.NoError(t, harness.objectStore.PutObject(harness.bucket, \"restores/test-restore/restore-test-restore-results.gz\", obj))\n\tres, err := harness.GetRestoreResults(\"test-restore\")\n\n\trequire.NoError(t, err)\n\tassert.Equal(t, contents[\"warnings\"], res[\"warnings\"])\n\tassert.Equal(t, contents[\"errors\"], res[\"errors\"])\n}\n\nfunc TestGetRestoredResourceList(t *testing.T) {\n\tharness := newObjectBackupStoreTestHarness(\"test-bucket\", \"\")\n\n\t// file not found should not error\n\t_, err := harness.GetRestoredResourceList(\"test-restore\")\n\trequire.NoError(t, err)\n\n\t// file containing invalid data should error\n\tharness.objectStore.PutObject(harness.bucket, \"restores/test-restore/restore-test-restore-resource-list.json.gz\", newStringReadSeeker(\"foo\"))\n\t_, err = harness.GetRestoredResourceList(\"test-restore\")\n\trequire.Error(t, err)\n\n\t// file containing gzipped json data should return correctly\n\tlist := map[string][]string{\n\t\t\"pod\": {\"test-ns/pod1(created)\", \"test-ns/pod2(skipped)\"},\n\t}\n\tobj := new(bytes.Buffer)\n\tgzw := gzip.NewWriter(obj)\n\n\trequire.NoError(t, json.NewEncoder(gzw).Encode(list))\n\trequire.NoError(t, gzw.Close())\n\trequire.NoError(t, harness.objectStore.PutObject(harness.bucket, \"restores/test-restore/restore-test-restore-resource-list.json.gz\", obj))\n\tres, err := harness.GetRestoredResourceList(\"test-restore\")\n\n\trequire.NoError(t, err)\n\tassert.Equal(t, list[\"pod\"], res[\"pod\"])\n}\n\nfunc TestPutBackupVolumeInfos(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tprefix string\n\t\texpectedErr string\n\t\texpectedKeys []string\n\t}{\n\t\t{\n\t\t\tname: \"normal case\",\n\t\t\texpectedErr: \"\",\n\t\t\texpectedKeys: []string{\n\t\t\t\t\"backups/backup-1/backup-1-volumeinfo.json.gz\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tharness := newObjectBackupStoreTestHarness(\"foo\", tc.prefix)\n\n\t\t\tvolumeInfos := []*volume.BackupVolumeInfo{\n\t\t\t\t{\n\t\t\t\t\tPVCName: \"test\",\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tbuf := new(bytes.Buffer)\n\t\t\tgzw := gzip.NewWriter(buf)\n\t\t\tdefer gzw.Close()\n\n\t\t\trequire.NoError(t, json.NewEncoder(gzw).Encode(volumeInfos))\n\t\t\tbufferContent := buf.Bytes()\n\n\t\t\terr := harness.PutBackupVolumeInfos(\"backup-1\", buf)\n\n\t\t\tvelerotest.AssertErrorMatches(t, tc.expectedErr, err)\n\t\t\tassert.Len(t, harness.objectStore.Data[harness.bucket], len(tc.expectedKeys))\n\t\t\tfor _, key := range tc.expectedKeys {\n\t\t\t\tassert.Contains(t, harness.objectStore.Data[harness.bucket], key)\n\t\t\t\tassert.Equal(t, harness.objectStore.Data[harness.bucket][key], bufferContent)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc encodeToBytes(obj runtime.Object) []byte {\n\tres, err := encode.Encode(obj, \"json\")\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn res\n}\n\ntype stringReadSeeker struct {\n\t*strings.Reader\n}\n\nfunc newStringReadSeeker(s string) *stringReadSeeker {\n\treturn &stringReadSeeker{\n\t\tReader: strings.NewReader(s),\n\t}\n}\n\nfunc (srs *stringReadSeeker) Seek(offset int64, whence int) (int64, error) {\n\treturn 0, nil\n}\n\ntype errorReader struct{}\n\nfunc (r *errorReader) Read([]byte) (int, error) {\n\treturn 0, errors.New(\"error readers return errors\")\n}\n" }, { "path": "pkg/plugin/clientmgmt/backupitemaction/v1/restartable_backup_item_action.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\t\"github.com/pkg/errors\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tbiav1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/backupitemaction/v1\"\n)\n\n// AdaptedBackupItemAction is a backup item action adapted to the v1 BackupItemAction API\ntype AdaptedBackupItemAction struct {\n\tKind common.PluginKind\n\n\t// Get returns a restartable BackupItemAction for the given name and process, wrapping if necessary\n\tGetRestartable func(name string, restartableProcess process.RestartableProcess) biav1.BackupItemAction\n}\n\nfunc AdaptedBackupItemActions() []AdaptedBackupItemAction {\n\treturn []AdaptedBackupItemAction{\n\t\t{\n\t\t\tKind: common.PluginKindBackupItemAction,\n\t\t\tGetRestartable: func(name string, restartableProcess process.RestartableProcess) biav1.BackupItemAction {\n\t\t\t\treturn NewRestartableBackupItemAction(name, restartableProcess)\n\t\t\t},\n\t\t},\n\t}\n}\n\n// RestartableBackupItemAction is a backup item action for a given implementation (such as \"pod\"). It is associated with\n// a restartableProcess, which may be shared and used to run multiple plugins. At the beginning of each method\n// call, the restartableBackupItemAction asks its restartableProcess to restart itself if needed (e.g. if the\n// process terminated for any reason), then it proceeds with the actual call.\ntype RestartableBackupItemAction struct {\n\tKey process.KindAndName\n\tSharedPluginProcess process.RestartableProcess\n}\n\n// NewRestartableBackupItemAction returns a new RestartableBackupItemAction.\nfunc NewRestartableBackupItemAction(name string, sharedPluginProcess process.RestartableProcess) *RestartableBackupItemAction {\n\tr := &RestartableBackupItemAction{\n\t\tKey: process.KindAndName{Kind: common.PluginKindBackupItemAction, Name: name},\n\t\tSharedPluginProcess: sharedPluginProcess,\n\t}\n\treturn r\n}\n\n// getBackupItemAction returns the backup item action for this restartableBackupItemAction. It does *not* restart the\n// plugin process.\nfunc (r *RestartableBackupItemAction) getBackupItemAction() (biav1.BackupItemAction, error) {\n\tplugin, err := r.SharedPluginProcess.GetByKindAndName(r.Key)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tbackupItemAction, ok := plugin.(biav1.BackupItemAction)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"plugin %T is not a BackupItemAction\", plugin)\n\t}\n\n\treturn backupItemAction, nil\n}\n\n// getDelegate restarts the plugin process (if needed) and returns the backup item action for this restartableBackupItemAction.\nfunc (r *RestartableBackupItemAction) getDelegate() (biav1.BackupItemAction, error) {\n\tif err := r.SharedPluginProcess.ResetIfNeeded(); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn r.getBackupItemAction()\n}\n\n// AppliesTo restarts the plugin's process if needed, then delegates the call.\nfunc (r *RestartableBackupItemAction) AppliesTo() (velero.ResourceSelector, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn velero.ResourceSelector{}, err\n\t}\n\n\treturn delegate.AppliesTo()\n}\n\n// Execute restarts the plugin's process if needed, then delegates the call.\nfunc (r *RestartableBackupItemAction) Execute(item runtime.Unstructured, backup *api.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\treturn delegate.Execute(item, backup)\n}\n" }, { "path": "pkg/plugin/clientmgmt/backupitemaction/v1/restartable_backup_item_action_test.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\t\"github.com/vmware-tanzu/velero/internal/restartabletest\"\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tmocks \"github.com/vmware-tanzu/velero/pkg/plugin/velero/mocks/backupitemaction/v1\"\n)\n\nfunc TestRestartableGetBackupItemAction(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tplugin any\n\t\tgetError error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"error getting by kind and name\",\n\t\t\tgetError: errors.Errorf(\"get error\"),\n\t\t\texpectedError: \"get error\",\n\t\t},\n\t\t{\n\t\t\tname: \"wrong type\",\n\t\t\tplugin: 3,\n\t\t\texpectedError: \"plugin int is not a BackupItemAction\",\n\t\t},\n\t\t{\n\t\t\tname: \"happy path\",\n\t\t\tplugin: new(mocks.BackupItemAction),\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tp := new(restartabletest.MockRestartableProcess)\n\t\t\tdefer p.AssertExpectations(t)\n\n\t\t\tname := \"pod\"\n\t\t\tkey := process.KindAndName{Kind: common.PluginKindBackupItemAction, Name: name}\n\t\t\tp.On(\"GetByKindAndName\", key).Return(tc.plugin, tc.getError)\n\n\t\t\tr := NewRestartableBackupItemAction(name, p)\n\t\t\ta, err := r.getBackupItemAction()\n\t\t\tif tc.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, tc.expectedError)\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.Equal(t, tc.plugin, a)\n\t\t})\n\t}\n}\n\nfunc TestRestartableBackupItemActionGetDelegate(t *testing.T) {\n\tp := new(restartabletest.MockRestartableProcess)\n\tdefer p.AssertExpectations(t)\n\n\t// Reset error\n\tp.On(\"ResetIfNeeded\").Return(errors.Errorf(\"reset error\")).Once()\n\tname := \"pod\"\n\tr := NewRestartableBackupItemAction(name, p)\n\ta, err := r.getDelegate()\n\tassert.Nil(t, a)\n\trequire.EqualError(t, err, \"reset error\")\n\n\t// Happy path\n\tp.On(\"ResetIfNeeded\").Return(nil)\n\texpected := new(mocks.BackupItemAction)\n\tkey := process.KindAndName{Kind: common.PluginKindBackupItemAction, Name: name}\n\tp.On(\"GetByKindAndName\", key).Return(expected, nil)\n\n\ta, err = r.getDelegate()\n\trequire.NoError(t, err)\n\tassert.Equal(t, expected, a)\n}\n\nfunc TestRestartableBackupItemActionDelegatedFunctions(t *testing.T) {\n\tb := new(v1.Backup)\n\n\tpv := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"color\": \"blue\",\n\t\t},\n\t}\n\n\tpvToReturn := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"color\": \"green\",\n\t\t},\n\t}\n\n\tadditionalItems := []velero.ResourceIdentifier{\n\t\t{\n\t\t\tGroupResource: schema.GroupResource{Group: \"velero.io\", Resource: \"backups\"},\n\t\t},\n\t}\n\n\trestartabletest.RunRestartableDelegateTests(\n\t\tt,\n\t\tcommon.PluginKindBackupItemAction,\n\t\tfunc(key process.KindAndName, p process.RestartableProcess) any {\n\t\t\treturn &RestartableBackupItemAction{\n\t\t\t\tKey: key,\n\t\t\t\tSharedPluginProcess: p,\n\t\t\t}\n\t\t},\n\t\tfunc() restartabletest.Mockable {\n\t\t\treturn new(mocks.BackupItemAction)\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"AppliesTo\",\n\t\t\tInputs: []any{},\n\t\t\tExpectedErrorOutputs: []any{velero.ResourceSelector{}, errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{velero.ResourceSelector{IncludedNamespaces: []string{\"a\"}}, errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"Execute\",\n\t\t\tInputs: []any{pv, b},\n\t\t\tExpectedErrorOutputs: []any{nil, ([]velero.ResourceIdentifier)(nil), errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{pvToReturn, additionalItems, errors.Errorf(\"delegate error\")},\n\t\t},\n\t)\n}\n" }, { "path": "pkg/plugin/clientmgmt/backupitemaction/v2/restartable_backup_item_action.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v2\n\nimport (\n\t\"github.com/pkg/errors\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tbiav1cli \"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/backupitemaction/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tbiav2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/backupitemaction/v2\"\n)\n\n// AdaptedBackupItemAction is a v1 BackupItemAction adapted to implement the v2 API\ntype AdaptedBackupItemAction struct {\n\tKind common.PluginKind\n\n\t// Get returns a restartable BackupItemAction for the given name and process, wrapping if necessary\n\tGetRestartable func(name string, restartableProcess process.RestartableProcess) biav2.BackupItemAction\n}\n\nfunc AdaptedBackupItemActions() []AdaptedBackupItemAction {\n\treturn []AdaptedBackupItemAction{\n\t\t{\n\t\t\tKind: common.PluginKindBackupItemActionV2,\n\t\t\tGetRestartable: func(name string, restartableProcess process.RestartableProcess) biav2.BackupItemAction {\n\t\t\t\treturn NewRestartableBackupItemAction(name, restartableProcess)\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tKind: common.PluginKindBackupItemAction,\n\t\t\tGetRestartable: func(name string, restartableProcess process.RestartableProcess) biav2.BackupItemAction {\n\t\t\t\treturn NewAdaptedV1RestartableBackupItemAction(biav1cli.NewRestartableBackupItemAction(name, restartableProcess))\n\t\t\t},\n\t\t},\n\t}\n}\n\n// restartableBackupItemAction is a backup item action for a given implementation (such as \"pod\"). It is associated with\n// a restartableProcess, which may be shared and used to run multiple plugins. At the beginning of each method\n// call, the restartableBackupItemAction asks its restartableProcess to restart itself if needed (e.g. if the\n// process terminated for any reason), then it proceeds with the actual call.\ntype RestartableBackupItemAction struct {\n\tKey process.KindAndName\n\tSharedPluginProcess process.RestartableProcess\n}\n\n// NewRestartableBackupItemAction returns a new RestartableBackupItemAction.\nfunc NewRestartableBackupItemAction(name string, sharedPluginProcess process.RestartableProcess) *RestartableBackupItemAction {\n\tr := &RestartableBackupItemAction{\n\t\tKey: process.KindAndName{Kind: common.PluginKindBackupItemActionV2, Name: name},\n\t\tSharedPluginProcess: sharedPluginProcess,\n\t}\n\treturn r\n}\n\n// getBackupItemAction returns the backup item action for this restartableBackupItemAction. It does *not* restart the\n// plugin process.\nfunc (r *RestartableBackupItemAction) getBackupItemAction() (biav2.BackupItemAction, error) {\n\tplugin, err := r.SharedPluginProcess.GetByKindAndName(r.Key)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tbackupItemAction, ok := plugin.(biav2.BackupItemAction)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"plugin %T (returned for %v) is not a BackupItemActionV2\", plugin, r.Key)\n\t}\n\n\treturn backupItemAction, nil\n}\n\n// getDelegate restarts the plugin process (if needed) and returns the backup item action for this restartableBackupItemAction.\nfunc (r *RestartableBackupItemAction) getDelegate() (biav2.BackupItemAction, error) {\n\tif err := r.SharedPluginProcess.ResetIfNeeded(); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn r.getBackupItemAction()\n}\n\n// Name returns the plugin's name.\nfunc (r *RestartableBackupItemAction) Name() string {\n\treturn r.Key.Name\n}\n\n// AppliesTo restarts the plugin's process if needed, then delegates the call.\nfunc (r *RestartableBackupItemAction) AppliesTo() (velero.ResourceSelector, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn velero.ResourceSelector{}, err\n\t}\n\n\treturn delegate.AppliesTo()\n}\n\n// Execute restarts the plugin's process if needed, then delegates the call.\nfunc (r *RestartableBackupItemAction) Execute(item runtime.Unstructured, backup *api.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn nil, nil, \"\", nil, err\n\t}\n\n\treturn delegate.Execute(item, backup)\n}\n\n// Progress restarts the plugin's process if needed, then delegates the call.\nfunc (r *RestartableBackupItemAction) Progress(operationID string, backup *api.Backup) (velero.OperationProgress, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn velero.OperationProgress{}, err\n\t}\n\n\treturn delegate.Progress(operationID, backup)\n}\n\n// Cancel restarts the plugin's process if needed, then delegates the call.\nfunc (r *RestartableBackupItemAction) Cancel(operationID string, backup *api.Backup) error {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn delegate.Cancel(operationID, backup)\n}\n\ntype AdaptedV1RestartableBackupItemAction struct {\n\tV1Restartable *biav1cli.RestartableBackupItemAction\n}\n\n// NewAdaptedV1RestartableBackupItemAction returns a new v1 RestartableBackupItemAction adapted to v2\nfunc NewAdaptedV1RestartableBackupItemAction(v1Restartable *biav1cli.RestartableBackupItemAction) *AdaptedV1RestartableBackupItemAction {\n\tr := &AdaptedV1RestartableBackupItemAction{\n\t\tV1Restartable: v1Restartable,\n\t}\n\treturn r\n}\n\n// Name restarts the plugin's name.\nfunc (r *AdaptedV1RestartableBackupItemAction) Name() string {\n\treturn r.V1Restartable.Key.Name\n}\n\n// AppliesTo delegates to the v1 AppliesTo call.\nfunc (r *AdaptedV1RestartableBackupItemAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn r.V1Restartable.AppliesTo()\n}\n\n// Execute delegates to the v1 Execute call, returning an empty operationID.\nfunc (r *AdaptedV1RestartableBackupItemAction) Execute(item runtime.Unstructured, backup *api.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\tupdatedItem, additionalItems, err := r.V1Restartable.Execute(item, backup)\n\treturn updatedItem, additionalItems, \"\", nil, err\n}\n\n// Progress returns with an error since v1 plugins will never return an operationID, which means that\n// any operationID passed in here will be invalid.\nfunc (r *AdaptedV1RestartableBackupItemAction) Progress(operationID string, backup *api.Backup) (velero.OperationProgress, error) {\n\treturn velero.OperationProgress{}, biav2.AsyncOperationsNotSupportedError()\n}\n\n// Cancel just returns without error since v1 plugins don't implement it.\nfunc (r *AdaptedV1RestartableBackupItemAction) Cancel(operationID string, backup *api.Backup) error {\n\treturn nil\n}\n" }, { "path": "pkg/plugin/clientmgmt/backupitemaction/v2/restartable_backup_item_action_test.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v2\n\nimport (\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\t\"github.com/vmware-tanzu/velero/internal/restartabletest\"\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tmocksv2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/mocks/backupitemaction/v2\"\n)\n\nfunc TestRestartableGetBackupItemAction(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tplugin any\n\t\tgetError error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"error getting by kind and name\",\n\t\t\tgetError: errors.Errorf(\"get error\"),\n\t\t\texpectedError: \"get error\",\n\t\t},\n\t\t{\n\t\t\tname: \"wrong type\",\n\t\t\tplugin: 3,\n\t\t\texpectedError: \"plugin int (returned for {BackupItemActionV2 pod}) is not a BackupItemActionV2\",\n\t\t},\n\t\t{\n\t\t\tname: \"happy path\",\n\t\t\tplugin: new(mocksv2.BackupItemAction),\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tp := new(restartabletest.MockRestartableProcess)\n\t\t\tdefer p.AssertExpectations(t)\n\n\t\t\tname := \"pod\"\n\t\t\tkey := process.KindAndName{Kind: common.PluginKindBackupItemActionV2, Name: name}\n\t\t\tp.On(\"GetByKindAndName\", key).Return(tc.plugin, tc.getError)\n\n\t\t\tr := NewRestartableBackupItemAction(name, p)\n\t\t\ta, err := r.getBackupItemAction()\n\t\t\tif tc.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, tc.expectedError)\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.Equal(t, tc.plugin, a)\n\t\t})\n\t}\n}\n\nfunc TestRestartableBackupItemActionGetDelegate(t *testing.T) {\n\tp := new(restartabletest.MockRestartableProcess)\n\tdefer p.AssertExpectations(t)\n\n\t// Reset error\n\tp.On(\"ResetIfNeeded\").Return(errors.Errorf(\"reset error\")).Once()\n\tname := \"pod\"\n\tr := NewRestartableBackupItemAction(name, p)\n\ta, err := r.getDelegate()\n\tassert.Nil(t, a)\n\trequire.EqualError(t, err, \"reset error\")\n\n\t// Happy path\n\tp.On(\"ResetIfNeeded\").Return(nil)\n\texpected := new(mocksv2.BackupItemAction)\n\tkey := process.KindAndName{Kind: common.PluginKindBackupItemActionV2, Name: name}\n\tp.On(\"GetByKindAndName\", key).Return(expected, nil)\n\n\ta, err = r.getDelegate()\n\trequire.NoError(t, err)\n\tassert.Equal(t, expected, a)\n}\n\nfunc TestRestartableBackupItemActionDelegatedFunctions(t *testing.T) {\n\tb := new(v1.Backup)\n\n\tpv := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"color\": \"blue\",\n\t\t},\n\t}\n\n\toid := \"operation1\"\n\n\tpvToReturn := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"color\": \"green\",\n\t\t},\n\t}\n\n\tadditionalItems := []velero.ResourceIdentifier{\n\t\t{\n\t\t\tGroupResource: schema.GroupResource{Group: \"velero.io\", Resource: \"backups\"},\n\t\t},\n\t}\n\n\trestartabletest.RunRestartableDelegateTests(\n\t\tt,\n\t\tcommon.PluginKindBackupItemAction,\n\t\tfunc(key process.KindAndName, p process.RestartableProcess) any {\n\t\t\treturn &RestartableBackupItemAction{\n\t\t\t\tKey: key,\n\t\t\t\tSharedPluginProcess: p,\n\t\t\t}\n\t\t},\n\t\tfunc() restartabletest.Mockable {\n\t\t\treturn new(mocksv2.BackupItemAction)\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"AppliesTo\",\n\t\t\tInputs: []any{},\n\t\t\tExpectedErrorOutputs: []any{velero.ResourceSelector{}, errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{velero.ResourceSelector{IncludedNamespaces: []string{\"a\"}}, errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"Execute\",\n\t\t\tInputs: []any{pv, b},\n\t\t\tExpectedErrorOutputs: []any{nil, ([]velero.ResourceIdentifier)(nil), \"\", ([]velero.ResourceIdentifier)(nil), errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{pvToReturn, additionalItems, \"\", ([]velero.ResourceIdentifier)(nil), errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"Progress\",\n\t\t\tInputs: []any{oid, b},\n\t\t\tExpectedErrorOutputs: []any{velero.OperationProgress{}, errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{velero.OperationProgress{}, errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"Cancel\",\n\t\t\tInputs: []any{oid, b},\n\t\t\tExpectedErrorOutputs: []any{errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{errors.Errorf(\"delegate error\")},\n\t\t},\n\t)\n}\n" }, { "path": "pkg/plugin/clientmgmt/itemblockaction/v1/restartable_item_block_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\t\"github.com/pkg/errors\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tibav1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/itemblockaction/v1\"\n)\n\n// AdaptedItemBlockAction is an ItemBlock action adapted to the v1 ItemBlockAction API\ntype AdaptedItemBlockAction struct {\n\tKind common.PluginKind\n\n\t// Get returns a restartable ItemBlockAction for the given name and process, wrapping if necessary\n\tGetRestartable func(name string, restartableProcess process.RestartableProcess) ibav1.ItemBlockAction\n}\n\nfunc AdaptedItemBlockActions() []AdaptedItemBlockAction {\n\treturn []AdaptedItemBlockAction{\n\t\t{\n\t\t\tKind: common.PluginKindItemBlockAction,\n\t\t\tGetRestartable: func(name string, restartableProcess process.RestartableProcess) ibav1.ItemBlockAction {\n\t\t\t\treturn NewRestartableItemBlockAction(name, restartableProcess)\n\t\t\t},\n\t\t},\n\t}\n}\n\n// RestartableItemBlockAction is an ItemBlock action for a given implementation (such as \"pod\"). It is associated with\n// a restartableProcess, which may be shared and used to run multiple plugins. At the beginning of each method\n// call, the restartableItemBlockAction asks its restartableProcess to restart itself if needed (e.g. if the\n// process terminated for any reason), then it proceeds with the actual call.\ntype RestartableItemBlockAction struct {\n\tKey process.KindAndName\n\tSharedPluginProcess process.RestartableProcess\n}\n\n// NewRestartableItemBlockAction returns a new RestartableItemBlockAction.\nfunc NewRestartableItemBlockAction(name string, sharedPluginProcess process.RestartableProcess) *RestartableItemBlockAction {\n\tr := &RestartableItemBlockAction{\n\t\tKey: process.KindAndName{Kind: common.PluginKindItemBlockAction, Name: name},\n\t\tSharedPluginProcess: sharedPluginProcess,\n\t}\n\treturn r\n}\n\n// getItemBlockAction returns the ItemBlock action for this restartableItemBlockAction. It does *not* restart the\n// plugin process.\nfunc (r *RestartableItemBlockAction) getItemBlockAction() (ibav1.ItemBlockAction, error) {\n\tplugin, err := r.SharedPluginProcess.GetByKindAndName(r.Key)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\titemBlockAction, ok := plugin.(ibav1.ItemBlockAction)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"plugin %T is not an ItemBlockAction\", plugin)\n\t}\n\n\treturn itemBlockAction, nil\n}\n\n// getDelegate restarts the plugin process (if needed) and returns the ItemBlock action for this restartableItemBlockAction.\nfunc (r *RestartableItemBlockAction) getDelegate() (ibav1.ItemBlockAction, error) {\n\tif err := r.SharedPluginProcess.ResetIfNeeded(); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn r.getItemBlockAction()\n}\n\n// Name returns the plugin's name.\nfunc (r *RestartableItemBlockAction) Name() string {\n\treturn r.Key.Name\n}\n\n// AppliesTo restarts the plugin's process if needed, then delegates the call.\nfunc (r *RestartableItemBlockAction) AppliesTo() (velero.ResourceSelector, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn velero.ResourceSelector{}, err\n\t}\n\n\treturn delegate.AppliesTo()\n}\n\n// GetRelatedItems restarts the plugin's process if needed, then delegates the call.\nfunc (r *RestartableItemBlockAction) GetRelatedItems(item runtime.Unstructured, backup *api.Backup) ([]velero.ResourceIdentifier, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn delegate.GetRelatedItems(item, backup)\n}\n" }, { "path": "pkg/plugin/clientmgmt/itemblockaction/v1/restartable_item_block_action_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\t\"github.com/vmware-tanzu/velero/internal/restartabletest\"\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tmocks \"github.com/vmware-tanzu/velero/pkg/plugin/velero/mocks/itemblockaction/v1\"\n)\n\nfunc TestRestartableGetItemBlockAction(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tplugin any\n\t\tgetError error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"error getting by kind and name\",\n\t\t\tgetError: errors.Errorf(\"get error\"),\n\t\t\texpectedError: \"get error\",\n\t\t},\n\t\t{\n\t\t\tname: \"wrong type\",\n\t\t\tplugin: 3,\n\t\t\texpectedError: \"plugin int is not an ItemBlockAction\",\n\t\t},\n\t\t{\n\t\t\tname: \"happy path\",\n\t\t\tplugin: new(mocks.ItemBlockAction),\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tp := new(restartabletest.MockRestartableProcess)\n\t\t\tdefer p.AssertExpectations(t)\n\n\t\t\tname := \"pod\"\n\t\t\tkey := process.KindAndName{Kind: common.PluginKindItemBlockAction, Name: name}\n\t\t\tp.On(\"GetByKindAndName\", key).Return(tc.plugin, tc.getError)\n\n\t\t\tr := NewRestartableItemBlockAction(name, p)\n\t\t\ta, err := r.getItemBlockAction()\n\t\t\tif tc.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, tc.expectedError)\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.Equal(t, tc.plugin, a)\n\t\t})\n\t}\n}\n\nfunc TestRestartableItemBlockActionGetDelegate(t *testing.T) {\n\tp := new(restartabletest.MockRestartableProcess)\n\tdefer p.AssertExpectations(t)\n\n\t// Reset error\n\tp.On(\"ResetIfNeeded\").Return(errors.Errorf(\"reset error\")).Once()\n\tname := \"pod\"\n\tr := NewRestartableItemBlockAction(name, p)\n\ta, err := r.getDelegate()\n\tassert.Nil(t, a)\n\trequire.EqualError(t, err, \"reset error\")\n\n\t// Happy path\n\tp.On(\"ResetIfNeeded\").Return(nil)\n\texpected := new(mocks.ItemBlockAction)\n\tkey := process.KindAndName{Kind: common.PluginKindItemBlockAction, Name: name}\n\tp.On(\"GetByKindAndName\", key).Return(expected, nil)\n\n\ta, err = r.getDelegate()\n\trequire.NoError(t, err)\n\tassert.Equal(t, expected, a)\n}\n\nfunc TestRestartableItemBlockActionDelegatedFunctions(t *testing.T) {\n\tb := new(v1.Backup)\n\n\tpv := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"color\": \"blue\",\n\t\t},\n\t}\n\n\trelatedItems := []velero.ResourceIdentifier{\n\t\t{\n\t\t\tGroupResource: schema.GroupResource{Group: \"velero.io\", Resource: \"backups\"},\n\t\t},\n\t}\n\n\trestartabletest.RunRestartableDelegateTests(\n\t\tt,\n\t\tcommon.PluginKindItemBlockAction,\n\t\tfunc(key process.KindAndName, p process.RestartableProcess) any {\n\t\t\treturn &RestartableItemBlockAction{\n\t\t\t\tKey: key,\n\t\t\t\tSharedPluginProcess: p,\n\t\t\t}\n\t\t},\n\t\tfunc() restartabletest.Mockable {\n\t\t\treturn new(mocks.ItemBlockAction)\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"AppliesTo\",\n\t\t\tInputs: []any{},\n\t\t\tExpectedErrorOutputs: []any{velero.ResourceSelector{}, errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{velero.ResourceSelector{IncludedNamespaces: []string{\"a\"}}, errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"GetRelatedItems\",\n\t\t\tInputs: []any{pv, b},\n\t\t\tExpectedErrorOutputs: []any{([]velero.ResourceIdentifier)(nil), errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{relatedItems, errors.Errorf(\"delegate error\")},\n\t\t},\n\t)\n}\n" }, { "path": "pkg/plugin/clientmgmt/manager.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage clientmgmt\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"strings\"\n\t\"sync\"\n\n\t\"github.com/sirupsen/logrus\"\n\n\tbiav1cli \"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/backupitemaction/v1\"\n\tbiav2cli \"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/backupitemaction/v2\"\n\tibav1cli \"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/itemblockaction/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n\triav1cli \"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/restoreitemaction/v1\"\n\triav2cli \"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/restoreitemaction/v2\"\n\tvsv1cli \"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/volumesnapshotter/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tbiav1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/backupitemaction/v1\"\n\tbiav2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/backupitemaction/v2\"\n\tibav1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/itemblockaction/v1\"\n\triav1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/restoreitemaction/v1\"\n\triav2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/restoreitemaction/v2\"\n\tvsv1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/volumesnapshotter/v1\"\n)\n\n// Manager manages the lifecycles of plugins.\ntype Manager interface {\n\t// GetObjectStore returns the ObjectStore plugin for name.\n\tGetObjectStore(name string) (velero.ObjectStore, error)\n\n\t// GetVolumeSnapshotter returns the VolumeSnapshotter plugin for name.\n\tGetVolumeSnapshotter(name string) (vsv1.VolumeSnapshotter, error)\n\n\t// GetBackupItemActions returns all v1 backup item action plugins.\n\tGetBackupItemActions() ([]biav1.BackupItemAction, error)\n\n\t// GetBackupItemAction returns the backup item action plugin for name.\n\tGetBackupItemAction(name string) (biav1.BackupItemAction, error)\n\n\t// GetBackupItemActionsV2 returns all v2 backup item action plugins (including those adapted from v1).\n\tGetBackupItemActionsV2() ([]biav2.BackupItemAction, error)\n\n\t// GetBackupItemActionV2 returns the backup item action plugin for name.\n\tGetBackupItemActionV2(name string) (biav2.BackupItemAction, error)\n\n\t// GetRestoreItemActions returns all restore item action plugins.\n\tGetRestoreItemActions() ([]riav1.RestoreItemAction, error)\n\n\t// GetRestoreItemAction returns the restore item action plugin for name.\n\tGetRestoreItemAction(name string) (riav1.RestoreItemAction, error)\n\n\t// GetRestoreItemActionsV2 returns all v2 restore item action plugins.\n\tGetRestoreItemActionsV2() ([]riav2.RestoreItemAction, error)\n\n\t// GetRestoreItemActionV2 returns the restore item action plugin for name.\n\tGetRestoreItemActionV2(name string) (riav2.RestoreItemAction, error)\n\n\t// GetDeleteItemActions returns all delete item action plugins.\n\tGetDeleteItemActions() ([]velero.DeleteItemAction, error)\n\n\t// GetDeleteItemAction returns the delete item action plugin for name.\n\tGetDeleteItemAction(name string) (velero.DeleteItemAction, error)\n\n\t// GetItemBlockActions returns all v1 ItemBlock action plugins.\n\tGetItemBlockActions() ([]ibav1.ItemBlockAction, error)\n\n\t// GetItemBlockAction returns the ItemBlock action plugin for name.\n\tGetItemBlockAction(name string) (ibav1.ItemBlockAction, error)\n\n\t// CleanupClients terminates all of the Manager's running plugin processes.\n\tCleanupClients()\n}\n\n// Used checking for adapted plugin versions\nvar pluginNotFoundErrType = &process.PluginNotFoundError{}\n\n// manager implements Manager.\ntype manager struct {\n\tlogger logrus.FieldLogger\n\tlogLevel logrus.Level\n\tregistry process.Registry\n\n\trestartableProcessFactory process.RestartableProcessFactory\n\n\t// lock guards restartableProcesses\n\tlock sync.Mutex\n\trestartableProcesses map[string]process.RestartableProcess\n}\n\n// NewManager constructs a manager for getting plugins.\nfunc NewManager(logger logrus.FieldLogger, level logrus.Level, registry process.Registry) Manager {\n\treturn &manager{\n\t\tlogger: logger,\n\t\tlogLevel: level,\n\t\tregistry: registry,\n\n\t\trestartableProcessFactory: process.NewRestartableProcessFactory(),\n\n\t\trestartableProcesses: make(map[string]process.RestartableProcess),\n\t}\n}\n\nfunc (m *manager) CleanupClients() {\n\tm.lock.Lock()\n\n\tfor _, restartableProcess := range m.restartableProcesses {\n\t\trestartableProcess.Stop()\n\t}\n\n\tm.lock.Unlock()\n}\n\n// getRestartableProcess returns a restartableProcess for a plugin identified by kind and name, creating a\n// restartableProcess if it is the first time it has been requested.\nfunc (m *manager) getRestartableProcess(kind common.PluginKind, name string) (process.RestartableProcess, error) {\n\tm.lock.Lock()\n\tdefer m.lock.Unlock()\n\n\tlogger := m.logger.WithFields(logrus.Fields{\n\t\t\"kind\": kind.String(),\n\t\t\"name\": name,\n\t})\n\tlogger.Debug(\"looking for plugin in registry\")\n\n\tinfo, err := m.registry.Get(kind, name)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tlogger = logger.WithField(\"command\", info.Command)\n\n\trestartableProcess, found := m.restartableProcesses[info.Command]\n\tif found {\n\t\tlogger.Debug(\"found preexisting restartable plugin process\")\n\t\treturn restartableProcess, nil\n\t}\n\n\tlogger.Debug(\"creating new restartable plugin process\")\n\n\trestartableProcess, err = m.restartableProcessFactory.NewRestartableProcess(info.Command, m.logger, m.logLevel)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tm.restartableProcesses[info.Command] = restartableProcess\n\n\treturn restartableProcess, nil\n}\n\n// GetObjectStore returns a restartableObjectStore for name.\nfunc (m *manager) GetObjectStore(name string) (velero.ObjectStore, error) {\n\tname = sanitizeName(name)\n\n\trestartableProcess, err := m.getRestartableProcess(common.PluginKindObjectStore, name)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tr := NewRestartableObjectStore(name, restartableProcess)\n\n\treturn r, nil\n}\n\n// GetVolumeSnapshotter returns a restartableVolumeSnapshotter for name.\nfunc (m *manager) GetVolumeSnapshotter(name string) (vsv1.VolumeSnapshotter, error) {\n\tname = sanitizeName(name)\n\n\tfor _, adaptedVolumeSnapshotter := range vsv1cli.AdaptedVolumeSnapshotters() {\n\t\trestartableProcess, err := m.getRestartableProcess(adaptedVolumeSnapshotter.Kind, name)\n\t\t// Check if plugin was not found\n\t\tif errors.As(err, &pluginNotFoundErrType) {\n\t\t\tcontinue\n\t\t}\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\treturn adaptedVolumeSnapshotter.GetRestartable(name, restartableProcess), nil\n\t}\n\treturn nil, fmt.Errorf(\"unable to get valid VolumeSnapshotter for %q\", name)\n}\n\n// GetBackupItemActions returns all backup item actions as restartableBackupItemActions.\nfunc (m *manager) GetBackupItemActions() ([]biav1.BackupItemAction, error) {\n\tlist := m.registry.List(common.PluginKindBackupItemAction)\n\n\tactions := make([]biav1.BackupItemAction, 0, len(list))\n\n\tfor i := range list {\n\t\tid := list[i]\n\n\t\tr, err := m.GetBackupItemAction(id.Name)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tactions = append(actions, r)\n\t}\n\n\treturn actions, nil\n}\n\n// GetBackupItemAction returns a restartableBackupItemAction for name.\nfunc (m *manager) GetBackupItemAction(name string) (biav1.BackupItemAction, error) {\n\tname = sanitizeName(name)\n\n\tfor _, adaptedBackupItemAction := range biav1cli.AdaptedBackupItemActions() {\n\t\trestartableProcess, err := m.getRestartableProcess(adaptedBackupItemAction.Kind, name)\n\t\t// Check if plugin was not found\n\t\tif errors.As(err, &pluginNotFoundErrType) {\n\t\t\tcontinue\n\t\t}\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\treturn adaptedBackupItemAction.GetRestartable(name, restartableProcess), nil\n\t}\n\treturn nil, fmt.Errorf(\"unable to get valid BackupItemAction for %q\", name)\n}\n\n// GetBackupItemActionsV2 returns all v2 backup item actions as RestartableBackupItemActions.\nfunc (m *manager) GetBackupItemActionsV2() ([]biav2.BackupItemAction, error) {\n\tlist := m.registry.List(common.PluginKindBackupItemActionV2)\n\n\tactions := make([]biav2.BackupItemAction, 0, len(list))\n\n\tfor i := range list {\n\t\tid := list[i]\n\n\t\tr, err := m.GetBackupItemActionV2(id.Name)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tactions = append(actions, r)\n\t}\n\n\treturn actions, nil\n}\n\n// GetBackupItemActionV2 returns a v2 restartableBackupItemAction for name.\nfunc (m *manager) GetBackupItemActionV2(name string) (biav2.BackupItemAction, error) {\n\tname = sanitizeName(name)\n\n\tfor _, adaptedBackupItemAction := range biav2cli.AdaptedBackupItemActions() {\n\t\trestartableProcess, err := m.getRestartableProcess(adaptedBackupItemAction.Kind, name)\n\t\t// Check if plugin was not found\n\t\tif errors.As(err, &pluginNotFoundErrType) {\n\t\t\tcontinue\n\t\t}\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\treturn adaptedBackupItemAction.GetRestartable(name, restartableProcess), nil\n\t}\n\treturn nil, fmt.Errorf(\"unable to get valid BackupItemActionV2 for %q\", name)\n}\n\n// GetRestoreItemActions returns all restore item actions as restartableRestoreItemActions.\nfunc (m *manager) GetRestoreItemActions() ([]riav1.RestoreItemAction, error) {\n\tlist := m.registry.List(common.PluginKindRestoreItemAction)\n\n\tactions := make([]riav1.RestoreItemAction, 0, len(list))\n\n\tfor i := range list {\n\t\tid := list[i]\n\n\t\tr, err := m.GetRestoreItemAction(id.Name)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tactions = append(actions, r)\n\t}\n\n\treturn actions, nil\n}\n\n// GetRestoreItemAction returns a restartableRestoreItemAction for name.\nfunc (m *manager) GetRestoreItemAction(name string) (riav1.RestoreItemAction, error) {\n\tname = sanitizeName(name)\n\n\tfor _, adaptedRestoreItemAction := range riav1cli.AdaptedRestoreItemActions() {\n\t\trestartableProcess, err := m.getRestartableProcess(adaptedRestoreItemAction.Kind, name)\n\t\t// Check if plugin was not found\n\t\tif errors.As(err, &pluginNotFoundErrType) {\n\t\t\tcontinue\n\t\t}\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\treturn adaptedRestoreItemAction.GetRestartable(name, restartableProcess), nil\n\t}\n\treturn nil, fmt.Errorf(\"unable to get valid RestoreItemAction for %q\", name)\n}\n\n// GetRestoreItemActionsV2 returns all v2 restore item actions as restartableRestoreItemActions.\nfunc (m *manager) GetRestoreItemActionsV2() ([]riav2.RestoreItemAction, error) {\n\tlist := m.registry.List(common.PluginKindRestoreItemActionV2)\n\n\tactions := make([]riav2.RestoreItemAction, 0, len(list))\n\n\tfor i := range list {\n\t\tid := list[i]\n\n\t\tr, err := m.GetRestoreItemActionV2(id.Name)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tactions = append(actions, r)\n\t}\n\n\treturn actions, nil\n}\n\n// GetRestoreItemActionV2 returns a v2 restartableRestoreItemAction for name.\nfunc (m *manager) GetRestoreItemActionV2(name string) (riav2.RestoreItemAction, error) {\n\tname = sanitizeName(name)\n\n\tfor _, adaptedRestoreItemAction := range riav2cli.AdaptedRestoreItemActions() {\n\t\trestartableProcess, err := m.getRestartableProcess(adaptedRestoreItemAction.Kind, name)\n\t\t// Check if plugin was not found\n\t\tif errors.As(err, &pluginNotFoundErrType) {\n\t\t\tcontinue\n\t\t}\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\treturn adaptedRestoreItemAction.GetRestartable(name, restartableProcess), nil\n\t}\n\treturn nil, fmt.Errorf(\"unable to get valid RestoreItemActionV2 for %q\", name)\n}\n\n// GetDeleteItemActions returns all delete item actions as restartableDeleteItemActions.\nfunc (m *manager) GetDeleteItemActions() ([]velero.DeleteItemAction, error) {\n\tlist := m.registry.List(common.PluginKindDeleteItemAction)\n\n\tactions := make([]velero.DeleteItemAction, 0, len(list))\n\n\tfor i := range list {\n\t\tid := list[i]\n\n\t\tr, err := m.GetDeleteItemAction(id.Name)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tactions = append(actions, r)\n\t}\n\n\treturn actions, nil\n}\n\n// GetDeleteItemAction returns a restartableDeleteItemAction for name.\nfunc (m *manager) GetDeleteItemAction(name string) (velero.DeleteItemAction, error) {\n\tname = sanitizeName(name)\n\n\trestartableProcess, err := m.getRestartableProcess(common.PluginKindDeleteItemAction, name)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tr := NewRestartableDeleteItemAction(name, restartableProcess)\n\treturn r, nil\n}\n\n// GetItemBlockActions returns all ItemBlock actions as restartableItemBlockActions.\nfunc (m *manager) GetItemBlockActions() ([]ibav1.ItemBlockAction, error) {\n\tlist := m.registry.List(common.PluginKindItemBlockAction)\n\n\tactions := make([]ibav1.ItemBlockAction, 0, len(list))\n\n\tfor i := range list {\n\t\tid := list[i]\n\n\t\tr, err := m.GetItemBlockAction(id.Name)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tactions = append(actions, r)\n\t}\n\n\treturn actions, nil\n}\n\n// GetItemBlockAction returns a restartableItemBlockAction for name.\nfunc (m *manager) GetItemBlockAction(name string) (ibav1.ItemBlockAction, error) {\n\tname = sanitizeName(name)\n\n\tfor _, adaptedItemBlockAction := range ibav1cli.AdaptedItemBlockActions() {\n\t\trestartableProcess, err := m.getRestartableProcess(adaptedItemBlockAction.Kind, name)\n\t\t// Check if plugin was not found\n\t\tif errors.As(err, &pluginNotFoundErrType) {\n\t\t\tcontinue\n\t\t}\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\treturn adaptedItemBlockAction.GetRestartable(name, restartableProcess), nil\n\t}\n\treturn nil, fmt.Errorf(\"unable to get valid ItemBlockAction for %q\", name)\n}\n\n// sanitizeName adds \"velero.io\" to legacy plugins that weren't namespaced.\nfunc sanitizeName(name string) string {\n\t// Backwards compatibility with non-namespaced Velero plugins, following principle of least surprise\n\t// since DeleteItemActions were not bundled with Velero when plugins were non-namespaced.\n\tif !strings.Contains(name, \"/\") {\n\t\tname = \"velero.io/\" + name\n\t}\n\n\treturn name\n}\n" }, { "path": "pkg/plugin/clientmgmt/manager_test.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage clientmgmt\n\nimport (\n\t\"fmt\"\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/vmware-tanzu/velero/internal/restartabletest\"\n\tbiav1cli \"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/backupitemaction/v1\"\n\tbiav2cli \"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/backupitemaction/v2\"\n\tibav1cli \"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/itemblockaction/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n\triav1cli \"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/restoreitemaction/v1\"\n\triav2cli \"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/restoreitemaction/v2\"\n\tvsv1cli \"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/volumesnapshotter/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\ntype mockRegistry struct {\n\tmock.Mock\n}\n\nfunc (r *mockRegistry) DiscoverPlugins() error {\n\targs := r.Called()\n\treturn args.Error(0)\n}\n\nfunc (r *mockRegistry) List(kind common.PluginKind) []framework.PluginIdentifier {\n\targs := r.Called(kind)\n\treturn args.Get(0).([]framework.PluginIdentifier)\n}\n\nfunc (r *mockRegistry) Get(kind common.PluginKind, name string) (framework.PluginIdentifier, error) {\n\targs := r.Called(kind, name)\n\tvar id framework.PluginIdentifier\n\tif args.Get(0) != nil {\n\t\tid = args.Get(0).(framework.PluginIdentifier)\n\t}\n\treturn id, args.Error(1)\n}\n\nfunc TestNewManager(t *testing.T) {\n\tlogger := test.NewLogger()\n\tlogLevel := logrus.InfoLevel\n\n\tregistry := &mockRegistry{}\n\tdefer registry.AssertExpectations(t)\n\n\tm := NewManager(logger, logLevel, registry).(*manager)\n\tassert.Equal(t, logger, m.logger)\n\tassert.Equal(t, logLevel, m.logLevel)\n\tassert.Equal(t, registry, m.registry)\n\tassert.NotNil(t, m.restartableProcesses)\n\tassert.Empty(t, m.restartableProcesses)\n}\n\ntype mockRestartableProcessFactory struct {\n\tmock.Mock\n}\n\nfunc (f *mockRestartableProcessFactory) NewRestartableProcess(command string, logger logrus.FieldLogger, logLevel logrus.Level) (process.RestartableProcess, error) {\n\targs := f.Called(command, logger, logLevel)\n\tvar rp process.RestartableProcess\n\tif args.Get(0) != nil {\n\t\trp = args.Get(0).(process.RestartableProcess)\n\t}\n\treturn rp, args.Error(1)\n}\n\nfunc TestGetRestartableProcess(t *testing.T) {\n\tlogger := test.NewLogger()\n\tlogLevel := logrus.InfoLevel\n\n\tregistry := &mockRegistry{}\n\tdefer registry.AssertExpectations(t)\n\n\tm := NewManager(logger, logLevel, registry).(*manager)\n\tfactory := &mockRestartableProcessFactory{}\n\tdefer factory.AssertExpectations(t)\n\tm.restartableProcessFactory = factory\n\n\t// Test 1: registry error\n\tpluginKind := common.PluginKindBackupItemAction\n\tpluginName := \"pod\"\n\tregistry.On(\"Get\", pluginKind, pluginName).Return(nil, errors.Errorf(\"registry\")).Once()\n\trp, err := m.getRestartableProcess(pluginKind, pluginName)\n\tassert.Nil(t, rp)\n\trequire.EqualError(t, err, \"registry\")\n\n\t// Test 2: registry ok, factory error\n\tpodID := framework.PluginIdentifier{\n\t\tCommand: \"/command\",\n\t\tKind: pluginKind,\n\t\tName: pluginName,\n\t}\n\tregistry.On(\"Get\", pluginKind, pluginName).Return(podID, nil)\n\tfactory.On(\"NewRestartableProcess\", podID.Command, logger, logLevel).Return(nil, errors.Errorf(\"factory\")).Once()\n\trp, err = m.getRestartableProcess(pluginKind, pluginName)\n\tassert.Nil(t, rp)\n\trequire.EqualError(t, err, \"factory\")\n\n\t// Test 3: registry ok, factory ok\n\trestartableProcess := &restartabletest.MockRestartableProcess{}\n\tdefer restartableProcess.AssertExpectations(t)\n\tfactory.On(\"NewRestartableProcess\", podID.Command, logger, logLevel).Return(restartableProcess, nil).Once()\n\trp, err = m.getRestartableProcess(pluginKind, pluginName)\n\trequire.NoError(t, err)\n\tassert.Equal(t, restartableProcess, rp)\n\n\t// Test 4: retrieve from cache\n\trp, err = m.getRestartableProcess(pluginKind, pluginName)\n\trequire.NoError(t, err)\n\tassert.Equal(t, restartableProcess, rp)\n}\n\nfunc TestCleanupClients(t *testing.T) {\n\tlogger := test.NewLogger()\n\tlogLevel := logrus.InfoLevel\n\n\tregistry := &mockRegistry{}\n\tdefer registry.AssertExpectations(t)\n\n\tm := NewManager(logger, logLevel, registry).(*manager)\n\n\tfor i := 0; i < 5; i++ {\n\t\trp := &restartabletest.MockRestartableProcess{}\n\t\tdefer rp.AssertExpectations(t)\n\t\trp.On(\"Stop\")\n\t\tm.restartableProcesses[fmt.Sprintf(\"rp%d\", i)] = rp\n\t}\n\n\tm.CleanupClients()\n}\n\nfunc TestGetObjectStore(t *testing.T) {\n\tgetPluginTest(t,\n\t\tcommon.PluginKindObjectStore,\n\t\t\"velero.io/aws\",\n\t\tfunc(m Manager, name string) (any, error) {\n\t\t\treturn m.GetObjectStore(name)\n\t\t},\n\t\tfunc(name string, sharedPluginProcess process.RestartableProcess) any {\n\t\t\treturn &restartableObjectStore{\n\t\t\t\tkey: process.KindAndName{Kind: common.PluginKindObjectStore, Name: name},\n\t\t\t\tsharedPluginProcess: sharedPluginProcess,\n\t\t\t}\n\t\t},\n\t\ttrue,\n\t)\n}\n\nfunc TestGetVolumeSnapshotter(t *testing.T) {\n\tgetPluginTest(t,\n\t\tcommon.PluginKindVolumeSnapshotter,\n\t\t\"velero.io/aws\",\n\t\tfunc(m Manager, name string) (any, error) {\n\t\t\treturn m.GetVolumeSnapshotter(name)\n\t\t},\n\t\tfunc(name string, sharedPluginProcess process.RestartableProcess) any {\n\t\t\treturn &vsv1cli.RestartableVolumeSnapshotter{\n\t\t\t\tKey: process.KindAndName{Kind: common.PluginKindVolumeSnapshotter, Name: name},\n\t\t\t\tSharedPluginProcess: sharedPluginProcess,\n\t\t\t}\n\t\t},\n\t\ttrue,\n\t)\n}\n\nfunc TestGetBackupItemAction(t *testing.T) {\n\tgetPluginTest(t,\n\t\tcommon.PluginKindBackupItemAction,\n\t\t\"velero.io/pod\",\n\t\tfunc(m Manager, name string) (any, error) {\n\t\t\treturn m.GetBackupItemAction(name)\n\t\t},\n\t\tfunc(name string, sharedPluginProcess process.RestartableProcess) any {\n\t\t\treturn &biav1cli.RestartableBackupItemAction{\n\t\t\t\tKey: process.KindAndName{Kind: common.PluginKindBackupItemAction, Name: name},\n\t\t\t\tSharedPluginProcess: sharedPluginProcess,\n\t\t\t}\n\t\t},\n\t\tfalse,\n\t)\n}\n\nfunc TestGetBackupItemActionV2(t *testing.T) {\n\tgetPluginTest(t,\n\t\tcommon.PluginKindBackupItemActionV2,\n\t\t\"velero.io/pod\",\n\t\tfunc(m Manager, name string) (any, error) {\n\t\t\treturn m.GetBackupItemActionV2(name)\n\t\t},\n\t\tfunc(name string, sharedPluginProcess process.RestartableProcess) any {\n\t\t\treturn &biav2cli.RestartableBackupItemAction{\n\t\t\t\tKey: process.KindAndName{Kind: common.PluginKindBackupItemActionV2, Name: name},\n\t\t\t\tSharedPluginProcess: sharedPluginProcess,\n\t\t\t}\n\t\t},\n\t\tfalse,\n\t)\n}\n\nfunc TestGetRestoreItemAction(t *testing.T) {\n\tgetPluginTest(t,\n\t\tcommon.PluginKindRestoreItemAction,\n\t\t\"velero.io/pod\",\n\t\tfunc(m Manager, name string) (any, error) {\n\t\t\treturn m.GetRestoreItemAction(name)\n\t\t},\n\t\tfunc(name string, sharedPluginProcess process.RestartableProcess) any {\n\t\t\treturn &riav1cli.RestartableRestoreItemAction{\n\t\t\t\tKey: process.KindAndName{Kind: common.PluginKindRestoreItemAction, Name: name},\n\t\t\t\tSharedPluginProcess: sharedPluginProcess,\n\t\t\t}\n\t\t},\n\t\tfalse,\n\t)\n}\n\nfunc TestGetRestoreItemActionV2(t *testing.T) {\n\tgetPluginTest(t,\n\t\tcommon.PluginKindRestoreItemActionV2,\n\t\t\"velero.io/pod\",\n\t\tfunc(m Manager, name string) (any, error) {\n\t\t\treturn m.GetRestoreItemActionV2(name)\n\t\t},\n\t\tfunc(name string, sharedPluginProcess process.RestartableProcess) any {\n\t\t\treturn &riav2cli.RestartableRestoreItemAction{\n\t\t\t\tKey: process.KindAndName{Kind: common.PluginKindRestoreItemActionV2, Name: name},\n\t\t\t\tSharedPluginProcess: sharedPluginProcess,\n\t\t\t}\n\t\t},\n\t\tfalse,\n\t)\n}\n\nfunc TestGetItemBlockAction(t *testing.T) {\n\tgetPluginTest(t,\n\t\tcommon.PluginKindItemBlockAction,\n\t\t\"velero.io/pod\",\n\t\tfunc(m Manager, name string) (any, error) {\n\t\t\treturn m.GetItemBlockAction(name)\n\t\t},\n\t\tfunc(name string, sharedPluginProcess process.RestartableProcess) any {\n\t\t\treturn &ibav1cli.RestartableItemBlockAction{\n\t\t\t\tKey: process.KindAndName{Kind: common.PluginKindItemBlockAction, Name: name},\n\t\t\t\tSharedPluginProcess: sharedPluginProcess,\n\t\t\t}\n\t\t},\n\t\tfalse,\n\t)\n}\n\nfunc getPluginTest(\n\tt *testing.T,\n\tkind common.PluginKind,\n\tname string,\n\tgetPluginFunc func(m Manager, name string) (any, error),\n\texpectedResultFunc func(name string, sharedPluginProcess process.RestartableProcess) any,\n\treinitializable bool,\n) {\n\tt.Helper()\n\tlogger := test.NewLogger()\n\tlogLevel := logrus.InfoLevel\n\n\tregistry := &mockRegistry{}\n\tdefer registry.AssertExpectations(t)\n\n\tm := NewManager(logger, logLevel, registry).(*manager)\n\tfactory := &mockRestartableProcessFactory{}\n\tdefer factory.AssertExpectations(t)\n\tm.restartableProcessFactory = factory\n\n\tpluginKind := kind\n\tpluginName := name\n\tpluginID := framework.PluginIdentifier{\n\t\tCommand: \"/command\",\n\t\tKind: pluginKind,\n\t\tName: pluginName,\n\t}\n\tregistry.On(\"Get\", pluginKind, pluginName).Return(pluginID, nil)\n\n\trestartableProcess := &restartabletest.MockRestartableProcess{}\n\tdefer restartableProcess.AssertExpectations(t)\n\n\t// Test 1: error getting restartable process\n\tfactory.On(\"NewRestartableProcess\", pluginID.Command, logger, logLevel).Return(nil, errors.Errorf(\"NewRestartableProcess\")).Once()\n\tactual, err := getPluginFunc(m, pluginName)\n\tassert.Nil(t, actual)\n\trequire.EqualError(t, err, \"NewRestartableProcess\")\n\n\t// Test 2: happy path\n\tfactory.On(\"NewRestartableProcess\", pluginID.Command, logger, logLevel).Return(restartableProcess, nil).Once()\n\n\texpected := expectedResultFunc(name, restartableProcess)\n\tif reinitializable {\n\t\tkey := process.KindAndName{Kind: pluginID.Kind, Name: pluginID.Name}\n\t\trestartableProcess.On(\"AddReinitializer\", key, expected)\n\t}\n\n\tactual, err = getPluginFunc(m, pluginName)\n\trequire.NoError(t, err)\n\tassert.Equal(t, expected, actual)\n}\n\nfunc TestGetBackupItemActions(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tnames []string\n\t\tnewRestartableProcessError error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"No items\",\n\t\t\tnames: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"Error getting restartable process\",\n\t\t\tnames: []string{\"velero.io/a\", \"velero.io/b\", \"velero.io/c\"},\n\t\t\tnewRestartableProcessError: errors.Errorf(\"NewRestartableProcess\"),\n\t\t\texpectedError: \"NewRestartableProcess\",\n\t\t},\n\t\t{\n\t\t\tname: \"Happy path\",\n\t\t\tnames: []string{\"velero.io/a\", \"velero.io/b\", \"velero.io/c\"},\n\t\t},\n\t}\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlogger := test.NewLogger()\n\t\t\tlogLevel := logrus.InfoLevel\n\n\t\t\tregistry := &mockRegistry{}\n\t\t\tdefer registry.AssertExpectations(t)\n\n\t\t\tm := NewManager(logger, logLevel, registry).(*manager)\n\t\t\tfactory := &mockRestartableProcessFactory{}\n\t\t\tdefer factory.AssertExpectations(t)\n\t\t\tm.restartableProcessFactory = factory\n\n\t\t\tpluginKind := common.PluginKindBackupItemAction\n\t\t\tvar pluginIDs []framework.PluginIdentifier\n\t\t\tfor i := range tc.names {\n\t\t\t\tpluginID := framework.PluginIdentifier{\n\t\t\t\t\tCommand: \"/command\",\n\t\t\t\t\tKind: pluginKind,\n\t\t\t\t\tName: tc.names[i],\n\t\t\t\t}\n\t\t\t\tpluginIDs = append(pluginIDs, pluginID)\n\t\t\t}\n\t\t\tregistry.On(\"List\", pluginKind).Return(pluginIDs)\n\n\t\t\tvar expectedActions []any\n\t\t\tfor i := range pluginIDs {\n\t\t\t\tpluginID := pluginIDs[i]\n\t\t\t\tpluginName := pluginID.Name\n\n\t\t\t\tregistry.On(\"Get\", pluginKind, pluginName).Return(pluginID, nil)\n\n\t\t\t\trestartableProcess := &restartabletest.MockRestartableProcess{}\n\t\t\t\tdefer restartableProcess.AssertExpectations(t)\n\n\t\t\t\texpected := &biav1cli.RestartableBackupItemAction{\n\t\t\t\t\tKey: process.KindAndName{Kind: pluginKind, Name: pluginName},\n\t\t\t\t\tSharedPluginProcess: restartableProcess,\n\t\t\t\t}\n\n\t\t\t\tif tc.newRestartableProcessError != nil {\n\t\t\t\t\t// Test 1: error getting restartable process\n\t\t\t\t\tfactory.On(\"NewRestartableProcess\", pluginID.Command, logger, logLevel).Return(nil, errors.Errorf(\"NewRestartableProcess\")).Once()\n\t\t\t\t\tbreak\n\t\t\t\t}\n\n\t\t\t\t// Test 2: happy path\n\t\t\t\tif i == 0 {\n\t\t\t\t\tfactory.On(\"NewRestartableProcess\", pluginID.Command, logger, logLevel).Return(restartableProcess, nil).Once()\n\t\t\t\t}\n\n\t\t\t\texpectedActions = append(expectedActions, expected)\n\t\t\t}\n\n\t\t\tbackupItemActions, err := m.GetBackupItemActions()\n\t\t\tif tc.newRestartableProcessError != nil {\n\t\t\t\tassert.Nil(t, backupItemActions)\n\t\t\t\tassert.EqualError(t, err, \"NewRestartableProcess\")\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tvar actual []any\n\t\t\t\tfor i := range backupItemActions {\n\t\t\t\t\tactual = append(actual, backupItemActions[i])\n\t\t\t\t}\n\t\t\t\tassert.Equal(t, expectedActions, actual)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetBackupItemActionsV2(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tnames []string\n\t\tnewRestartableProcessError error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"No items\",\n\t\t\tnames: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"Error getting restartable process\",\n\t\t\tnames: []string{\"velero.io/a\", \"velero.io/b\", \"velero.io/c\"},\n\t\t\tnewRestartableProcessError: errors.Errorf(\"NewRestartableProcess\"),\n\t\t\texpectedError: \"NewRestartableProcess\",\n\t\t},\n\t\t{\n\t\t\tname: \"Happy path\",\n\t\t\tnames: []string{\"velero.io/a\", \"velero.io/b\", \"velero.io/c\"},\n\t\t},\n\t}\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlogger := test.NewLogger()\n\t\t\tlogLevel := logrus.InfoLevel\n\n\t\t\tregistry := &mockRegistry{}\n\t\t\tdefer registry.AssertExpectations(t)\n\n\t\t\tm := NewManager(logger, logLevel, registry).(*manager)\n\t\t\tfactory := &mockRestartableProcessFactory{}\n\t\t\tdefer factory.AssertExpectations(t)\n\t\t\tm.restartableProcessFactory = factory\n\n\t\t\tpluginKind := common.PluginKindBackupItemActionV2\n\t\t\tvar pluginIDs []framework.PluginIdentifier\n\t\t\tfor i := range tc.names {\n\t\t\t\tpluginID := framework.PluginIdentifier{\n\t\t\t\t\tCommand: \"/command\",\n\t\t\t\t\tKind: pluginKind,\n\t\t\t\t\tName: tc.names[i],\n\t\t\t\t}\n\t\t\t\tpluginIDs = append(pluginIDs, pluginID)\n\t\t\t}\n\t\t\tregistry.On(\"List\", pluginKind).Return(pluginIDs)\n\n\t\t\tvar expectedActions []any\n\t\t\tfor i := range pluginIDs {\n\t\t\t\tpluginID := pluginIDs[i]\n\t\t\t\tpluginName := pluginID.Name\n\n\t\t\t\tregistry.On(\"Get\", pluginKind, pluginName).Return(pluginID, nil)\n\n\t\t\t\trestartableProcess := &restartabletest.MockRestartableProcess{}\n\t\t\t\tdefer restartableProcess.AssertExpectations(t)\n\n\t\t\t\texpected := &biav2cli.RestartableBackupItemAction{\n\t\t\t\t\tKey: process.KindAndName{Kind: pluginKind, Name: pluginName},\n\t\t\t\t\tSharedPluginProcess: restartableProcess,\n\t\t\t\t}\n\n\t\t\t\tif tc.newRestartableProcessError != nil {\n\t\t\t\t\t// Test 1: error getting restartable process\n\t\t\t\t\tfactory.On(\"NewRestartableProcess\", pluginID.Command, logger, logLevel).Return(nil, errors.Errorf(\"NewRestartableProcess\")).Once()\n\t\t\t\t\tbreak\n\t\t\t\t}\n\n\t\t\t\t// Test 2: happy path\n\t\t\t\tif i == 0 {\n\t\t\t\t\tfactory.On(\"NewRestartableProcess\", pluginID.Command, logger, logLevel).Return(restartableProcess, nil).Once()\n\t\t\t\t}\n\n\t\t\t\texpectedActions = append(expectedActions, expected)\n\t\t\t}\n\n\t\t\tbackupItemActions, err := m.GetBackupItemActionsV2()\n\t\t\tif tc.newRestartableProcessError != nil {\n\t\t\t\tassert.Nil(t, backupItemActions)\n\t\t\t\tassert.EqualError(t, err, \"NewRestartableProcess\")\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tvar actual []any\n\t\t\t\tfor i := range backupItemActions {\n\t\t\t\t\tactual = append(actual, backupItemActions[i])\n\t\t\t\t}\n\t\t\t\tassert.Equal(t, expectedActions, actual)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetRestoreItemActions(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tnames []string\n\t\tnewRestartableProcessError error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"No items\",\n\t\t\tnames: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"Error getting restartable process\",\n\t\t\tnames: []string{\"velero.io/a\", \"velero.io/b\", \"velero.io/c\"},\n\t\t\tnewRestartableProcessError: errors.Errorf(\"NewRestartableProcess\"),\n\t\t\texpectedError: \"NewRestartableProcess\",\n\t\t},\n\t\t{\n\t\t\tname: \"Happy path\",\n\t\t\tnames: []string{\"velero.io/a\", \"velero.io/b\", \"velero.io/c\"},\n\t\t},\n\t}\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlogger := test.NewLogger()\n\t\t\tlogLevel := logrus.InfoLevel\n\n\t\t\tregistry := &mockRegistry{}\n\t\t\tdefer registry.AssertExpectations(t)\n\n\t\t\tm := NewManager(logger, logLevel, registry).(*manager)\n\t\t\tfactory := &mockRestartableProcessFactory{}\n\t\t\tdefer factory.AssertExpectations(t)\n\t\t\tm.restartableProcessFactory = factory\n\n\t\t\tpluginKind := common.PluginKindRestoreItemAction\n\t\t\tvar pluginIDs []framework.PluginIdentifier\n\t\t\tfor i := range tc.names {\n\t\t\t\tpluginID := framework.PluginIdentifier{\n\t\t\t\t\tCommand: \"/command\",\n\t\t\t\t\tKind: pluginKind,\n\t\t\t\t\tName: tc.names[i],\n\t\t\t\t}\n\t\t\t\tpluginIDs = append(pluginIDs, pluginID)\n\t\t\t}\n\t\t\tregistry.On(\"List\", pluginKind).Return(pluginIDs)\n\n\t\t\tvar expectedActions []any\n\t\t\tfor i := range pluginIDs {\n\t\t\t\tpluginID := pluginIDs[i]\n\t\t\t\tpluginName := pluginID.Name\n\n\t\t\t\tregistry.On(\"Get\", pluginKind, pluginName).Return(pluginID, nil)\n\n\t\t\t\trestartableProcess := &restartabletest.MockRestartableProcess{}\n\t\t\t\tdefer restartableProcess.AssertExpectations(t)\n\n\t\t\t\texpected := &riav1cli.RestartableRestoreItemAction{\n\t\t\t\t\tKey: process.KindAndName{Kind: pluginKind, Name: pluginName},\n\t\t\t\t\tSharedPluginProcess: restartableProcess,\n\t\t\t\t}\n\n\t\t\t\tif tc.newRestartableProcessError != nil {\n\t\t\t\t\t// Test 1: error getting restartable process\n\t\t\t\t\tfactory.On(\"NewRestartableProcess\", pluginID.Command, logger, logLevel).Return(nil, errors.Errorf(\"NewRestartableProcess\")).Once()\n\t\t\t\t\tbreak\n\t\t\t\t}\n\n\t\t\t\t// Test 2: happy path\n\t\t\t\tif i == 0 {\n\t\t\t\t\tfactory.On(\"NewRestartableProcess\", pluginID.Command, logger, logLevel).Return(restartableProcess, nil).Once()\n\t\t\t\t}\n\n\t\t\t\texpectedActions = append(expectedActions, expected)\n\t\t\t}\n\n\t\t\trestoreItemActions, err := m.GetRestoreItemActions()\n\t\t\tif tc.newRestartableProcessError != nil {\n\t\t\t\tassert.Nil(t, restoreItemActions)\n\t\t\t\tassert.EqualError(t, err, \"NewRestartableProcess\")\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tvar actual []any\n\t\t\t\tfor i := range restoreItemActions {\n\t\t\t\t\tactual = append(actual, restoreItemActions[i])\n\t\t\t\t}\n\t\t\t\tassert.Equal(t, expectedActions, actual)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetRestoreItemActionsV2(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tnames []string\n\t\tnewRestartableProcessError error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"No items\",\n\t\t\tnames: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"Error getting restartable process\",\n\t\t\tnames: []string{\"velero.io/a\", \"velero.io/b\", \"velero.io/c\"},\n\t\t\tnewRestartableProcessError: errors.Errorf(\"NewRestartableProcess\"),\n\t\t\texpectedError: \"NewRestartableProcess\",\n\t\t},\n\t\t{\n\t\t\tname: \"Happy path\",\n\t\t\tnames: []string{\"velero.io/a\", \"velero.io/b\", \"velero.io/c\"},\n\t\t},\n\t}\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlogger := test.NewLogger()\n\t\t\tlogLevel := logrus.InfoLevel\n\n\t\t\tregistry := &mockRegistry{}\n\t\t\tdefer registry.AssertExpectations(t)\n\n\t\t\tm := NewManager(logger, logLevel, registry).(*manager)\n\t\t\tfactory := &mockRestartableProcessFactory{}\n\t\t\tdefer factory.AssertExpectations(t)\n\t\t\tm.restartableProcessFactory = factory\n\n\t\t\tpluginKind := common.PluginKindRestoreItemActionV2\n\t\t\tvar pluginIDs []framework.PluginIdentifier\n\t\t\tfor i := range tc.names {\n\t\t\t\tpluginID := framework.PluginIdentifier{\n\t\t\t\t\tCommand: \"/command\",\n\t\t\t\t\tKind: pluginKind,\n\t\t\t\t\tName: tc.names[i],\n\t\t\t\t}\n\t\t\t\tpluginIDs = append(pluginIDs, pluginID)\n\t\t\t}\n\t\t\tregistry.On(\"List\", pluginKind).Return(pluginIDs)\n\n\t\t\tvar expectedActions []any\n\t\t\tfor i := range pluginIDs {\n\t\t\t\tpluginID := pluginIDs[i]\n\t\t\t\tpluginName := pluginID.Name\n\n\t\t\t\tregistry.On(\"Get\", pluginKind, pluginName).Return(pluginID, nil)\n\n\t\t\t\trestartableProcess := &restartabletest.MockRestartableProcess{}\n\t\t\t\tdefer restartableProcess.AssertExpectations(t)\n\n\t\t\t\texpected := &riav2cli.RestartableRestoreItemAction{\n\t\t\t\t\tKey: process.KindAndName{Kind: pluginKind, Name: pluginName},\n\t\t\t\t\tSharedPluginProcess: restartableProcess,\n\t\t\t\t}\n\n\t\t\t\tif tc.newRestartableProcessError != nil {\n\t\t\t\t\t// Test 1: error getting restartable process\n\t\t\t\t\tfactory.On(\"NewRestartableProcess\", pluginID.Command, logger, logLevel).Return(nil, errors.Errorf(\"NewRestartableProcess\")).Once()\n\t\t\t\t\tbreak\n\t\t\t\t}\n\n\t\t\t\t// Test 2: happy path\n\t\t\t\tif i == 0 {\n\t\t\t\t\tfactory.On(\"NewRestartableProcess\", pluginID.Command, logger, logLevel).Return(restartableProcess, nil).Once()\n\t\t\t\t}\n\n\t\t\t\texpectedActions = append(expectedActions, expected)\n\t\t\t}\n\n\t\t\trestoreItemActions, err := m.GetRestoreItemActionsV2()\n\t\t\tif tc.newRestartableProcessError != nil {\n\t\t\t\tassert.Nil(t, restoreItemActions)\n\t\t\t\tassert.EqualError(t, err, \"NewRestartableProcess\")\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tvar actual []any\n\t\t\t\tfor i := range restoreItemActions {\n\t\t\t\t\tactual = append(actual, restoreItemActions[i])\n\t\t\t\t}\n\t\t\t\tassert.Equal(t, expectedActions, actual)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetDeleteItemAction(t *testing.T) {\n\tgetPluginTest(t,\n\t\tcommon.PluginKindDeleteItemAction,\n\t\t\"velero.io/deleter\",\n\t\tfunc(m Manager, name string) (any, error) {\n\t\t\treturn m.GetDeleteItemAction(name)\n\t\t},\n\t\tfunc(name string, sharedPluginProcess process.RestartableProcess) any {\n\t\t\treturn &restartableDeleteItemAction{\n\t\t\t\tkey: process.KindAndName{Kind: common.PluginKindDeleteItemAction, Name: name},\n\t\t\t\tsharedPluginProcess: sharedPluginProcess,\n\t\t\t}\n\t\t},\n\t\tfalse,\n\t)\n}\n\nfunc TestGetDeleteItemActions(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tnames []string\n\t\tnewRestartableProcessError error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"No items\",\n\t\t\tnames: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"Error getting restartable process\",\n\t\t\tnames: []string{\"velero.io/a\", \"velero.io/b\", \"velero.io/c\"},\n\t\t\tnewRestartableProcessError: errors.Errorf(\"NewRestartableProcess\"),\n\t\t\texpectedError: \"NewRestartableProcess\",\n\t\t},\n\t\t{\n\t\t\tname: \"Happy path\",\n\t\t\tnames: []string{\"velero.io/a\", \"velero.io/b\", \"velero.io/c\"},\n\t\t},\n\t}\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlogger := test.NewLogger()\n\t\t\tlogLevel := logrus.InfoLevel\n\n\t\t\tregistry := &mockRegistry{}\n\t\t\tdefer registry.AssertExpectations(t)\n\n\t\t\tm := NewManager(logger, logLevel, registry).(*manager)\n\t\t\tfactory := &mockRestartableProcessFactory{}\n\t\t\tdefer factory.AssertExpectations(t)\n\t\t\tm.restartableProcessFactory = factory\n\n\t\t\tpluginKind := common.PluginKindDeleteItemAction\n\t\t\tvar pluginIDs []framework.PluginIdentifier\n\t\t\tfor i := range tc.names {\n\t\t\t\tpluginID := framework.PluginIdentifier{\n\t\t\t\t\tCommand: \"/command\",\n\t\t\t\t\tKind: pluginKind,\n\t\t\t\t\tName: tc.names[i],\n\t\t\t\t}\n\t\t\t\tpluginIDs = append(pluginIDs, pluginID)\n\t\t\t}\n\t\t\tregistry.On(\"List\", pluginKind).Return(pluginIDs)\n\n\t\t\tvar expectedActions []any\n\t\t\tfor i := range pluginIDs {\n\t\t\t\tpluginID := pluginIDs[i]\n\t\t\t\tpluginName := pluginID.Name\n\n\t\t\t\tregistry.On(\"Get\", pluginKind, pluginName).Return(pluginID, nil)\n\n\t\t\t\trestartableProcess := &restartabletest.MockRestartableProcess{}\n\t\t\t\tdefer restartableProcess.AssertExpectations(t)\n\n\t\t\t\texpected := &restartableDeleteItemAction{\n\t\t\t\t\tkey: process.KindAndName{Kind: pluginKind, Name: pluginName},\n\t\t\t\t\tsharedPluginProcess: restartableProcess,\n\t\t\t\t}\n\n\t\t\t\tif tc.newRestartableProcessError != nil {\n\t\t\t\t\t// Test 1: error getting restartable process\n\t\t\t\t\tfactory.On(\"NewRestartableProcess\", pluginID.Command, logger, logLevel).Return(nil, errors.Errorf(\"NewRestartableProcess\")).Once()\n\t\t\t\t\tbreak\n\t\t\t\t}\n\n\t\t\t\t// Test 2: happy path\n\t\t\t\tif i == 0 {\n\t\t\t\t\tfactory.On(\"NewRestartableProcess\", pluginID.Command, logger, logLevel).Return(restartableProcess, nil).Once()\n\t\t\t\t}\n\n\t\t\t\texpectedActions = append(expectedActions, expected)\n\t\t\t}\n\n\t\t\tdeleteItemActions, err := m.GetDeleteItemActions()\n\t\t\tif tc.newRestartableProcessError != nil {\n\t\t\t\tassert.Nil(t, deleteItemActions)\n\t\t\t\tassert.EqualError(t, err, \"NewRestartableProcess\")\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tvar actual []any\n\t\t\t\tfor i := range deleteItemActions {\n\t\t\t\t\tactual = append(actual, deleteItemActions[i])\n\t\t\t\t}\n\t\t\t\tassert.Equal(t, expectedActions, actual)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetItemBlockActions(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tnames []string\n\t\tnewRestartableProcessError error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"No items\",\n\t\t\tnames: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"Error getting restartable process\",\n\t\t\tnames: []string{\"velero.io/a\", \"velero.io/b\", \"velero.io/c\"},\n\t\t\tnewRestartableProcessError: errors.Errorf(\"NewRestartableProcess\"),\n\t\t\texpectedError: \"NewRestartableProcess\",\n\t\t},\n\t\t{\n\t\t\tname: \"Happy path\",\n\t\t\tnames: []string{\"velero.io/a\", \"velero.io/b\", \"velero.io/c\"},\n\t\t},\n\t}\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlogger := test.NewLogger()\n\t\t\tlogLevel := logrus.InfoLevel\n\n\t\t\tregistry := &mockRegistry{}\n\t\t\tdefer registry.AssertExpectations(t)\n\n\t\t\tm := NewManager(logger, logLevel, registry).(*manager)\n\t\t\tfactory := &mockRestartableProcessFactory{}\n\t\t\tdefer factory.AssertExpectations(t)\n\t\t\tm.restartableProcessFactory = factory\n\n\t\t\tpluginKind := common.PluginKindItemBlockAction\n\t\t\tvar pluginIDs []framework.PluginIdentifier\n\t\t\tfor i := range tc.names {\n\t\t\t\tpluginID := framework.PluginIdentifier{\n\t\t\t\t\tCommand: \"/command\",\n\t\t\t\t\tKind: pluginKind,\n\t\t\t\t\tName: tc.names[i],\n\t\t\t\t}\n\t\t\t\tpluginIDs = append(pluginIDs, pluginID)\n\t\t\t}\n\t\t\tregistry.On(\"List\", pluginKind).Return(pluginIDs)\n\n\t\t\tvar expectedActions []any\n\t\t\tfor i := range pluginIDs {\n\t\t\t\tpluginID := pluginIDs[i]\n\t\t\t\tpluginName := pluginID.Name\n\n\t\t\t\tregistry.On(\"Get\", pluginKind, pluginName).Return(pluginID, nil)\n\n\t\t\t\trestartableProcess := &restartabletest.MockRestartableProcess{}\n\t\t\t\tdefer restartableProcess.AssertExpectations(t)\n\n\t\t\t\texpected := &ibav1cli.RestartableItemBlockAction{\n\t\t\t\t\tKey: process.KindAndName{Kind: pluginKind, Name: pluginName},\n\t\t\t\t\tSharedPluginProcess: restartableProcess,\n\t\t\t\t}\n\n\t\t\t\tif tc.newRestartableProcessError != nil {\n\t\t\t\t\t// Test 1: error getting restartable process\n\t\t\t\t\tfactory.On(\"NewRestartableProcess\", pluginID.Command, logger, logLevel).Return(nil, errors.Errorf(\"NewRestartableProcess\")).Once()\n\t\t\t\t\tbreak\n\t\t\t\t}\n\n\t\t\t\t// Test 2: happy path\n\t\t\t\tif i == 0 {\n\t\t\t\t\tfactory.On(\"NewRestartableProcess\", pluginID.Command, logger, logLevel).Return(restartableProcess, nil).Once()\n\t\t\t\t}\n\n\t\t\t\texpectedActions = append(expectedActions, expected)\n\t\t\t}\n\n\t\t\titemBlockActions, err := m.GetItemBlockActions()\n\t\t\tif tc.newRestartableProcessError != nil {\n\t\t\t\tassert.Nil(t, itemBlockActions)\n\t\t\t\tassert.EqualError(t, err, \"NewRestartableProcess\")\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tvar actual []any\n\t\t\t\tfor i := range itemBlockActions {\n\t\t\t\t\tactual = append(actual, itemBlockActions[i])\n\t\t\t\t}\n\t\t\t\tassert.Equal(t, expectedActions, actual)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestSanitizeName(t *testing.T) {\n\ttests := []struct {\n\t\tname, pluginName, expectedName string\n\t}{\n\t\t{\n\t\t\tname: \"Legacy, non-namespaced plugin\",\n\t\t\tpluginName: \"aws\",\n\t\t\texpectedName: \"velero.io/aws\",\n\t\t},\n\t\t{\n\t\t\tname: \"A Velero plugin\",\n\t\t\tpluginName: \"velero.io/aws\",\n\t\t\texpectedName: \"velero.io/aws\",\n\t\t},\n\t\t{\n\t\t\tname: \"A non-Velero plugin with a Velero namespace\",\n\t\t\tpluginName: \"velero.io/plugin-for-velero\",\n\t\t\texpectedName: \"velero.io/plugin-for-velero\",\n\t\t},\n\t\t{\n\t\t\tname: \"A non-Velero plugin with a non-Velero namespace\",\n\t\t\tpluginName: \"digitalocean.com/velero\",\n\t\t\texpectedName: \"digitalocean.com/velero\",\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tsanitizedName := sanitizeName(tc.pluginName)\n\t\t\tassert.Equal(t, tc.expectedName, sanitizedName)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/plugin/clientmgmt/process/client_builder.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\n// Package clientmgmt contains the plugin client for Velero.\npackage process\n\nimport (\n\t\"context\"\n\t\"os\"\n\t\"os/exec\"\n\n\thclog \"github.com/hashicorp/go-hclog\"\n\thcplugin \"github.com/hashicorp/go-plugin\"\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework\"\n\tbiav2 \"github.com/vmware-tanzu/velero/pkg/plugin/framework/backupitemaction/v2\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tibav1 \"github.com/vmware-tanzu/velero/pkg/plugin/framework/itemblockaction/v1\"\n\triav2 \"github.com/vmware-tanzu/velero/pkg/plugin/framework/restoreitemaction/v2\"\n)\n\n// clientBuilder builds go-plugin Clients.\ntype clientBuilder struct {\n\tcommandName string\n\tcommandArgs []string\n\tclientLogger logrus.FieldLogger\n\tpluginLogger hclog.Logger\n}\n\n// newClientBuilder returns a new clientBuilder with commandName to name. If the command matches the currently running\n// process (i.e. velero), this also sets commandArgs to the internal Velero command to run plugins.\nfunc newClientBuilder(command string, logger logrus.FieldLogger, logLevel logrus.Level) *clientBuilder {\n\tb := &clientBuilder{\n\t\tcommandName: command,\n\t\tclientLogger: logger,\n\t\tpluginLogger: newLogrusAdapter(logger, logLevel),\n\t}\n\tif command == os.Args[0] {\n\t\t// For plugins compiled into the velero executable, we need to run \"velero run-plugins\"\n\t\tb.commandArgs = []string{\"run-plugins\"}\n\t}\n\t// exclude \"velero\" and \"server\" from \"velero server --flags ...\"\n\tb.commandArgs = append(b.commandArgs, os.Args[2:]...)\n\n\treturn b\n}\n\nfunc newLogrusAdapter(pluginLogger logrus.FieldLogger, logLevel logrus.Level) *logrusAdapter {\n\treturn &logrusAdapter{impl: pluginLogger, level: logLevel}\n}\n\nfunc (b *clientBuilder) clientConfig() *hcplugin.ClientConfig {\n\treturn &hcplugin.ClientConfig{\n\t\tHandshakeConfig: framework.Handshake(),\n\t\tAllowedProtocols: []hcplugin.Protocol{hcplugin.ProtocolGRPC},\n\t\tPlugins: map[string]hcplugin.Plugin{\n\t\t\tstring(common.PluginKindBackupItemAction): framework.NewBackupItemActionPlugin(common.ClientLogger(b.clientLogger)),\n\t\t\tstring(common.PluginKindBackupItemActionV2): biav2.NewBackupItemActionPlugin(common.ClientLogger(b.clientLogger)),\n\t\t\tstring(common.PluginKindVolumeSnapshotter): framework.NewVolumeSnapshotterPlugin(common.ClientLogger(b.clientLogger)),\n\t\t\tstring(common.PluginKindObjectStore): framework.NewObjectStorePlugin(common.ClientLogger(b.clientLogger)),\n\t\t\tstring(common.PluginKindPluginLister): &framework.PluginListerPlugin{},\n\t\t\tstring(common.PluginKindRestoreItemAction): framework.NewRestoreItemActionPlugin(common.ClientLogger(b.clientLogger)),\n\t\t\tstring(common.PluginKindRestoreItemActionV2): riav2.NewRestoreItemActionPlugin(common.ClientLogger(b.clientLogger)),\n\t\t\tstring(common.PluginKindDeleteItemAction): framework.NewDeleteItemActionPlugin(common.ClientLogger(b.clientLogger)),\n\t\t\tstring(common.PluginKindItemBlockAction): ibav1.NewItemBlockActionPlugin(common.ClientLogger(b.clientLogger)),\n\t\t},\n\t\tLogger: b.pluginLogger,\n\t\tCmd: exec.CommandContext(context.Background(), b.commandName, b.commandArgs...), //nolint:gosec // Internal call. No need to check the command line.\n\t}\n}\n\n// client creates a new go-plugin Client with support for all of Velero's plugin kinds (BackupItemAction, VolumeSnapshotter,\n// ObjectStore, PluginLister, RestoreItemAction).\nfunc (b *clientBuilder) client() *hcplugin.Client {\n\treturn hcplugin.NewClient(b.clientConfig())\n}\n" }, { "path": "pkg/plugin/clientmgmt/process/client_builder_test.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage process\n\nimport (\n\t\"os\"\n\t\"os/exec\"\n\t\"testing\"\n\n\thcplugin \"github.com/hashicorp/go-plugin\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework\"\n\tbiav2 \"github.com/vmware-tanzu/velero/pkg/plugin/framework/backupitemaction/v2\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tibav1 \"github.com/vmware-tanzu/velero/pkg/plugin/framework/itemblockaction/v1\"\n\triav2 \"github.com/vmware-tanzu/velero/pkg/plugin/framework/restoreitemaction/v2\"\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestNewClientBuilder(t *testing.T) {\n\tlogger := test.NewLogger()\n\tlogLevel := logrus.InfoLevel\n\tcb := newClientBuilder(\"velero\", logger, logLevel)\n\tassert.Equal(t, \"velero\", cb.commandName)\n\tassert.Equal(t, newLogrusAdapter(logger, logLevel), cb.pluginLogger)\n\n\tcb = newClientBuilder(os.Args[0], logger, logLevel)\n\tassert.Equal(t, cb.commandName, os.Args[0])\n\tassert.Equal(t, newLogrusAdapter(logger, logLevel), cb.pluginLogger)\n}\n\nfunc TestClientConfig(t *testing.T) {\n\tlogger := test.NewLogger()\n\tlogLevel := logrus.InfoLevel\n\tcb := newClientBuilder(\"velero\", logger, logLevel)\n\n\texpected := &hcplugin.ClientConfig{\n\t\tHandshakeConfig: framework.Handshake(),\n\t\tAllowedProtocols: []hcplugin.Protocol{hcplugin.ProtocolGRPC},\n\t\tPlugins: map[string]hcplugin.Plugin{\n\t\t\tstring(common.PluginKindBackupItemAction): framework.NewBackupItemActionPlugin(common.ClientLogger(logger)),\n\t\t\tstring(common.PluginKindBackupItemActionV2): biav2.NewBackupItemActionPlugin(common.ClientLogger(logger)),\n\t\t\tstring(common.PluginKindVolumeSnapshotter): framework.NewVolumeSnapshotterPlugin(common.ClientLogger(logger)),\n\t\t\tstring(common.PluginKindObjectStore): framework.NewObjectStorePlugin(common.ClientLogger(logger)),\n\t\t\tstring(common.PluginKindPluginLister): &framework.PluginListerPlugin{},\n\t\t\tstring(common.PluginKindRestoreItemAction): framework.NewRestoreItemActionPlugin(common.ClientLogger(logger)),\n\t\t\tstring(common.PluginKindRestoreItemActionV2): riav2.NewRestoreItemActionPlugin(common.ClientLogger(logger)),\n\t\t\tstring(common.PluginKindDeleteItemAction): framework.NewDeleteItemActionPlugin(common.ClientLogger(logger)),\n\t\t\tstring(common.PluginKindItemBlockAction): ibav1.NewItemBlockActionPlugin(common.ClientLogger(logger)),\n\t\t},\n\t\tLogger: cb.pluginLogger,\n\t\tCmd: exec.CommandContext(t.Context(), cb.commandName, cb.commandArgs...),\n\t}\n\n\tcc := cb.clientConfig()\n\tassert.Equal(t, expected.HandshakeConfig, cc.HandshakeConfig)\n\tassert.Equal(t, expected.AllowedProtocols, cc.AllowedProtocols)\n\tassert.Equal(t, expected.Plugins, cc.Plugins)\n}\n" }, { "path": "pkg/plugin/clientmgmt/process/logrus_adapter.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage process\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"log\"\n\n\thclog \"github.com/hashicorp/go-hclog\"\n\t\"github.com/sirupsen/logrus\"\n)\n\nconst pluginNameField = \"pluginName\"\n\n// logrusAdapter implements the hclog.Logger interface and\n// delegates all calls to a logrus logger.\ntype logrusAdapter struct {\n\timpl logrus.FieldLogger\n\tlevel logrus.Level\n\tname string\n}\n\n// args are alternating key, value pairs, where the keys\n// are expected to be strings, and values can be any type.\nfunc argsToFields(args ...any) logrus.Fields {\n\tfields := make(map[string]any)\n\n\tfor i := 0; i < len(args); i += 2 {\n\t\tswitch args[i] {\n\t\tcase \"time\", \"timestamp\", \"level\":\n\t\t\t// remove `time` & `timestamp` because this info will be added\n\t\t\t// by the Velero logger and we don't want to have duplicated\n\t\t\t// fields.\n\t\t\t//\n\t\t\t// remove `level` because it'll be added by the Velero logger based\n\t\t\t// on the call we make (and go-plugin is determining which level\n\t\t\t// to log at based on the hclog-compatible `@level` field which\n\t\t\t// we're adding via HcLogLevelHook).\n\t\tdefault:\n\t\t\tvar val any\n\t\t\tif i+1 < len(args) {\n\t\t\t\tval = args[i+1]\n\t\t\t}\n\n\t\t\tfields[fmt.Sprintf(\"%v\", args[i])] = val\n\t\t}\n\t}\n\n\treturn logrus.Fields(fields)\n}\n\n// Trace emits a message and key/value pairs at the DEBUG level\n// (logrus doesn't have a TRACE level)\nfunc (l *logrusAdapter) Trace(msg string, args ...any) {\n\tl.Debug(msg, args...)\n}\n\n// Debug emits a message and key/value pairs at the DEBUG level\nfunc (l *logrusAdapter) Debug(msg string, args ...any) {\n\tl.impl.WithFields(argsToFields(args...)).Debug(msg)\n}\n\n// Info emits a message and key/value pairs at the INFO level\nfunc (l *logrusAdapter) Info(msg string, args ...any) {\n\tl.impl.WithFields(argsToFields(args...)).Info(msg)\n}\n\n// Warn emits a message and key/value pairs at the WARN level\nfunc (l *logrusAdapter) Warn(msg string, args ...any) {\n\tl.impl.WithFields(argsToFields(args...)).Warn(msg)\n}\n\n// Error emits a message and key/value pairs at the ERROR level\nfunc (l *logrusAdapter) Error(msg string, args ...any) {\n\tl.impl.WithFields(argsToFields(args...)).Error(msg)\n}\n\n// IsTrace indicates if TRACE logs would be emitted. This and the other Is* guards\n// are used to elide expensive logging code based on the current level.\nfunc (l *logrusAdapter) IsTrace() bool {\n\treturn l.IsDebug()\n}\n\n// IsDebug indicates if DEBUG logs would be emitted. This and the other Is* guards\n// are used to elide expensive logging code based on the current level.\nfunc (l *logrusAdapter) IsDebug() bool {\n\treturn l.level <= logrus.DebugLevel\n}\n\n// IsInfo indicates if INFO logs would be emitted. This and the other Is* guards\n// are used to elide expensive logging code based on the current level.\nfunc (l *logrusAdapter) IsInfo() bool {\n\treturn l.level <= logrus.InfoLevel\n}\n\n// IsWarn indicates if WARN logs would be emitted. This and the other Is* guards\n// are used to elide expensive logging code based on the current level.\nfunc (l *logrusAdapter) IsWarn() bool {\n\treturn l.level <= logrus.WarnLevel\n}\n\n// IsError indicates if ERROR logs would be emitted. This and the other Is* guards\n// are used to elide expensive logging code based on the current level.\nfunc (l *logrusAdapter) IsError() bool {\n\treturn l.level <= logrus.ErrorLevel\n}\n\n// With creates a sublogger that will always have the given key/value pairs\nfunc (l *logrusAdapter) With(args ...any) hclog.Logger {\n\treturn &logrusAdapter{\n\t\timpl: l.impl.WithFields(argsToFields(args...)),\n\t\tlevel: l.level,\n\t}\n}\n\n// Named creates a logger that will add a `pluginName` field with the name string\n// as the value. If the logger already has a name, the new value will be appended\n// to the current name.\nfunc (l *logrusAdapter) Named(name string) hclog.Logger {\n\tvar newName string\n\tif l.name == \"\" {\n\t\tnewName = name\n\t} else {\n\t\tnewName = l.name + \".\" + name\n\t}\n\n\treturn l.ResetNamed(newName)\n}\n\n// ResetNamed creates a logger that will add a `pluginName` field with the name string\n// as the value. This sets the name of the logger to the value directly, unlike `Named`\n// which appends the given value to the current name.\nfunc (l *logrusAdapter) ResetNamed(name string) hclog.Logger {\n\treturn &logrusAdapter{\n\t\timpl: l.impl.WithField(pluginNameField, name),\n\t\tlevel: l.level,\n\t\tname: name,\n\t}\n}\n\n// StandardLogger returns a value that conforms to the stdlib log.Logger interface\nfunc (l *logrusAdapter) StandardLogger(opts *hclog.StandardLoggerOptions) *log.Logger {\n\tpanic(\"not implemented\")\n}\n\n// Updates the level. This should affect all sub-loggers as well. If an\n// implementation cannot update the level on the fly, it should no-op.\nfunc (l *logrusAdapter) SetLevel(_ hclog.Level) {\n}\n\n// ImpliedArgs returns With key/value pairs\nfunc (l *logrusAdapter) ImpliedArgs() []any {\n\tpanic(\"not implemented\")\n}\n\n// Args are alternating key, val pairs\n// keys must be strings\n// vals can be any type, but display is implementation specific\n// Emit a message and key/value pairs at a provided log level\nfunc (l *logrusAdapter) Log(level hclog.Level, msg string, args ...any) {\n\tswitch level {\n\tcase hclog.Trace:\n\t\tl.Trace(msg, args...)\n\tcase hclog.Debug:\n\t\tl.Debug(msg, args...)\n\tcase hclog.Info:\n\t\tl.Info(msg, args...)\n\tcase hclog.Warn:\n\t\tl.Warn(msg, args...)\n\tcase hclog.Error:\n\t\tl.Error(msg, args...)\n\t}\n}\n\n// Returns the Name of the logger\nfunc (l *logrusAdapter) Name() string {\n\treturn l.name\n}\n\n// Return a value that conforms to io.Writer, which can be passed into log.SetOutput()\nfunc (l *logrusAdapter) StandardWriter(opts *hclog.StandardLoggerOptions) io.Writer {\n\tpanic(\"not implemented\")\n}\n" }, { "path": "pkg/plugin/clientmgmt/process/logrus_adapter_test.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage process\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestArgsToFields(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\targs []any\n\t\texpectedFields logrus.Fields\n\t}{\n\t\t{\n\t\t\tname: \"empty args results in empty map of fields\",\n\t\t\targs: []any{},\n\t\t\texpectedFields: logrus.Fields(map[string]any{}),\n\t\t},\n\t\t{\n\t\t\tname: \"matching string keys/values are correctly set as fields\",\n\t\t\targs: []any{\"key-1\", \"value-1\", \"key-2\", \"value-2\"},\n\t\t\texpectedFields: logrus.Fields(map[string]any{\n\t\t\t\t\"key-1\": \"value-1\",\n\t\t\t\t\"key-2\": \"value-2\",\n\t\t\t}),\n\t\t},\n\t\t{\n\t\t\tname: \"time/timestamp/level entries are removed\",\n\t\t\targs: []any{\"time\", time.Now(), \"key-1\", \"value-1\", \"timestamp\", time.Now(), \"key-2\", \"value-2\", \"level\", \"WARN\"},\n\t\t\texpectedFields: logrus.Fields(map[string]any{\n\t\t\t\t\"key-1\": \"value-1\",\n\t\t\t\t\"key-2\": \"value-2\",\n\t\t\t}),\n\t\t},\n\t\t{\n\t\t\tname: \"odd number of args adds the last arg as a field with a nil value\",\n\t\t\targs: []any{\"key-1\", \"value-1\", \"key-2\", \"value-2\", \"key-3\"},\n\t\t\texpectedFields: logrus.Fields(map[string]any{\n\t\t\t\t\"key-1\": \"value-1\",\n\t\t\t\t\"key-2\": \"value-2\",\n\t\t\t\t\"key-3\": nil,\n\t\t\t}),\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tassert.Equal(t, test.expectedFields, argsToFields(test.args...))\n\t\t})\n\t}\n}\n" }, { "path": "pkg/plugin/clientmgmt/process/process.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage process\n\nimport (\n\tplugin \"github.com/hashicorp/go-plugin\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n)\n\ntype Factory interface {\n\tnewProcess(command string, logger logrus.FieldLogger, logLevel logrus.Level) (Process, error)\n}\n\ntype processFactory struct {\n}\n\nfunc newProcessFactory() Factory {\n\treturn &processFactory{}\n}\n\nfunc (pf *processFactory) newProcess(command string, logger logrus.FieldLogger, logLevel logrus.Level) (Process, error) {\n\treturn newProcess(command, logger, logLevel)\n}\n\ntype Process interface {\n\tdispense(key KindAndName) (any, error)\n\texited() bool\n\tkill()\n}\n\ntype process struct {\n\tclient *plugin.Client\n\tprotocolClient plugin.ClientProtocol\n}\n\nfunc newProcess(command string, logger logrus.FieldLogger, logLevel logrus.Level) (Process, error) {\n\tbuilder := newClientBuilder(command, logger.WithField(\"cmd\", command), logLevel)\n\n\t// This creates a new go-plugin Client that has its own unique exec.Cmd for launching the plugin process.\n\tclient := builder.client()\n\n\t// This launches the plugin process.\n\tprotocolClient, err := client.Client()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tp := &process{\n\t\tclient: client,\n\t\tprotocolClient: protocolClient,\n\t}\n\n\treturn p, nil\n}\n\n// removeFeaturesFlag looks for and removes the '--features' arg\n// as well as the arg immediately following it (the flag value).\nfunc removeFeaturesFlag(args []string) []string {\n\tvar commandArgs []string\n\tvar featureFlag bool\n\n\tfor _, arg := range args {\n\t\t// if this arg is the flag name, skip it\n\t\tif arg == \"--features\" {\n\t\t\tfeatureFlag = true\n\t\t\tcontinue\n\t\t}\n\n\t\t// if the last arg we saw was the flag name, then\n\t\t// this arg is the value for the flag, so skip it\n\t\tif featureFlag {\n\t\t\tfeatureFlag = false\n\t\t\tcontinue\n\t\t}\n\n\t\t// otherwise, keep the arg\n\t\tcommandArgs = append(commandArgs, arg)\n\t}\n\n\treturn commandArgs\n}\n\nfunc (r *process) dispense(key KindAndName) (any, error) {\n\t// This calls GRPCClient(clientConn) on the plugin instance registered for key.name.\n\tdispensed, err := r.protocolClient.Dispense(key.Kind.String())\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\t// Currently all plugins except for PluginLister dispense clientDispenser instances.\n\tif clientDispenser, ok := dispensed.(common.ClientDispenser); ok {\n\t\tif key.Name == \"\" {\n\t\t\treturn nil, errors.Errorf(\"%s plugin requested but name is missing\", key.Kind.String())\n\t\t}\n\t\t// Get the instance that implements our plugin interface (e.g. ObjectStore) that is a gRPC-based\n\t\t// client\n\t\tdispensed = clientDispenser.ClientFor(key.Name)\n\t}\n\n\treturn dispensed, nil\n}\n\nfunc (r *process) exited() bool {\n\treturn r.client.Exited()\n}\n\nfunc (r *process) kill() {\n\tr.client.Kill()\n}\n" }, { "path": "pkg/plugin/clientmgmt/process/process_test.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage process\n\nimport (\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n)\n\ntype mockClientProtocol struct {\n\tmock.Mock\n}\n\nfunc (cp *mockClientProtocol) Close() error {\n\targs := cp.Called()\n\treturn args.Error(0)\n}\n\nfunc (cp *mockClientProtocol) Dispense(name string) (any, error) {\n\targs := cp.Called(name)\n\treturn args.Get(0), args.Error(1)\n}\n\nfunc (cp *mockClientProtocol) Ping() error {\n\targs := cp.Called()\n\treturn args.Error(0)\n}\n\ntype mockClientDispenser struct {\n\tmock.Mock\n}\n\nfunc (cd *mockClientDispenser) ClientFor(name string) any {\n\targs := cd.Called(name)\n\treturn args.Get(0)\n}\n\nfunc TestDispense(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tmissingKeyName bool\n\t\tdispenseError error\n\t\tclientDispenser bool\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"protocol client dispense error\",\n\t\t\tdispenseError: errors.Errorf(\"protocol client dispense\"),\n\t\t\texpectedError: \"protocol client dispense\",\n\t\t},\n\t\t{\n\t\t\tname: \"plugin lister, no error\",\n\t\t},\n\t\t{\n\t\t\tname: \"client dispenser, missing key name\",\n\t\t\tclientDispenser: true,\n\t\t\tmissingKeyName: true,\n\t\t\texpectedError: \"ObjectStore plugin requested but name is missing\",\n\t\t},\n\t\t{\n\t\t\tname: \"client dispenser, have key name\",\n\t\t\tclientDispenser: true,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tp := new(process)\n\t\t\tprotocolClient := new(mockClientProtocol)\n\t\t\tdefer protocolClient.AssertExpectations(t)\n\t\t\tp.protocolClient = protocolClient\n\n\t\t\tclientDispenser := new(mockClientDispenser)\n\t\t\tdefer clientDispenser.AssertExpectations(t)\n\n\t\t\tvar client any\n\n\t\t\tkey := KindAndName{}\n\t\t\tif tc.clientDispenser {\n\t\t\t\tkey.Kind = common.PluginKindObjectStore\n\t\t\t\tprotocolClient.On(\"Dispense\", key.Kind.String()).Return(clientDispenser, tc.dispenseError)\n\n\t\t\t\tif !tc.missingKeyName {\n\t\t\t\t\tkey.Name = \"aws\"\n\t\t\t\t\tclient = &framework.BackupItemActionGRPCClient{}\n\t\t\t\t\tclientDispenser.On(\"ClientFor\", key.Name).Return(client)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tkey.Kind = common.PluginKindPluginLister\n\t\t\t\tclient = &framework.PluginListerGRPCClient{}\n\t\t\t\tprotocolClient.On(\"Dispense\", key.Kind.String()).Return(client, tc.dispenseError)\n\t\t\t}\n\n\t\t\tdispensed, err := p.dispense(key)\n\n\t\t\tif tc.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, tc.expectedError)\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Equal(t, client, dispensed)\n\t\t})\n\t}\n}\n\nfunc Test_removeFeaturesFlag(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tcommandArgs []string\n\t\twant []string\n\t}{\n\t\t{\n\t\t\tname: \"when commandArgs is nil, a nil slice is returned\",\n\t\t\tcommandArgs: nil,\n\t\t\twant: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"when commandArgs is empty, a nil slice is returned\",\n\t\t\tcommandArgs: []string{},\n\t\t\twant: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"when commandArgs does not contain --features, it is returned as-is\",\n\t\t\tcommandArgs: []string{\"--log-level\", \"debug\", \"--another-flag\", \"foo\"},\n\t\t\twant: []string{\"--log-level\", \"debug\", \"--another-flag\", \"foo\"},\n\t\t},\n\t\t{\n\t\t\tname: \"when --features is the only flag, a nil slice is returned\",\n\t\t\tcommandArgs: []string{\"--features\", \"EnableCSI\"},\n\t\t\twant: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"when --features is the first flag, it's properly removed\",\n\t\t\tcommandArgs: []string{\"--features\", \"EnableCSI\", \"--log-level\", \"debug\", \"--another-flag\", \"foo\"},\n\t\t\twant: []string{\"--log-level\", \"debug\", \"--another-flag\", \"foo\"},\n\t\t},\n\t\t{\n\t\t\tname: \"when --features is the last flag, it's properly removed\",\n\t\t\tcommandArgs: []string{\"--log-level\", \"debug\", \"--another-flag\", \"foo\", \"--features\", \"EnableCSI\"},\n\t\t\twant: []string{\"--log-level\", \"debug\", \"--another-flag\", \"foo\"},\n\t\t},\n\t\t{\n\t\t\tname: \"when --features is neither the first nor last flag, it's properly removed\",\n\t\t\tcommandArgs: []string{\"--log-level\", \"debug\", \"--features\", \"EnableCSI\", \"--another-flag\", \"foo\"},\n\t\t\twant: []string{\"--log-level\", \"debug\", \"--another-flag\", \"foo\"},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tassert.Equal(t, tc.want, removeFeaturesFlag(tc.commandArgs))\n\t\t})\n\t}\n}\n" }, { "path": "pkg/plugin/clientmgmt/process/registry.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage process\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n)\n\n// Registry manages information about available plugins.\ntype Registry interface {\n\t// DiscoverPlugins discovers all available plugins.\n\tDiscoverPlugins() error\n\t// List returns all PluginIdentifiers for kind.\n\tList(kind common.PluginKind) []framework.PluginIdentifier\n\t// Get returns the PluginIdentifier for kind and name.\n\tGet(kind common.PluginKind, name string) (framework.PluginIdentifier, error)\n}\n\n// KindAndName is a convenience struct that combines a PluginKind and a name.\ntype KindAndName struct {\n\tKind common.PluginKind\n\tName string\n}\n\n// registry implements Registry.\ntype registry struct {\n\t// dir is the directory to search for plugins.\n\tdir string\n\tlogger logrus.FieldLogger\n\tlogLevel logrus.Level\n\n\tprocessFactory Factory\n\tfs filesystem.Interface\n\tpluginsByID map[KindAndName]framework.PluginIdentifier\n\tpluginsByKind map[common.PluginKind][]framework.PluginIdentifier\n}\n\n// NewRegistry returns a new registry.\nfunc NewRegistry(dir string, logger logrus.FieldLogger, logLevel logrus.Level) Registry {\n\treturn ®istry{\n\t\tdir: dir,\n\t\tlogger: logger,\n\t\tlogLevel: logLevel,\n\n\t\tprocessFactory: newProcessFactory(),\n\t\tfs: filesystem.NewFileSystem(),\n\t\tpluginsByID: make(map[KindAndName]framework.PluginIdentifier),\n\t\tpluginsByKind: make(map[common.PluginKind][]framework.PluginIdentifier),\n\t}\n}\n\nfunc (r *registry) DiscoverPlugins() error {\n\tplugins, err := r.readPluginsDir(r.dir)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Start by adding velero's internal plugins\n\tcommands := []string{os.Args[0]}\n\t// Then add the discovered plugin executables\n\tcommands = append(commands, plugins...)\n\n\treturn r.discoverPlugins(commands)\n}\n\nfunc (r *registry) discoverPlugins(commands []string) error {\n\tfor _, command := range commands {\n\t\tplugins, err := r.listPlugins(command)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tfor _, plugin := range plugins {\n\t\t\tr.logger.WithFields(logrus.Fields{\n\t\t\t\t\"kind\": plugin.Kind,\n\t\t\t\t\"name\": plugin.Name,\n\t\t\t\t\"command\": command,\n\t\t\t}).Info(\"registering plugin\")\n\n\t\t\tif err := r.register(plugin); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// List returns info about all plugin binaries that implement the given\n// PluginKind.\nfunc (r *registry) List(kind common.PluginKind) []framework.PluginIdentifier {\n\treturn r.pluginsByKind[kind]\n}\n\n// Get returns info about a plugin with the given name and kind, or an\n// error if one cannot be found.\nfunc (r *registry) Get(kind common.PluginKind, name string) (framework.PluginIdentifier, error) {\n\tp, found := r.pluginsByID[KindAndName{Kind: kind, Name: name}]\n\tif !found {\n\t\treturn framework.PluginIdentifier{}, newPluginNotFoundError(kind, name)\n\t}\n\treturn p, nil\n}\n\n// readPluginsDir recursively reads dir looking for plugins.\nfunc (r *registry) readPluginsDir(dir string) ([]string, error) {\n\tif _, err := r.fs.Stat(dir); err != nil {\n\t\tif os.IsNotExist(err) {\n\t\t\treturn []string{}, nil\n\t\t}\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tfiles, err := r.fs.ReadDir(dir)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tfullPaths := make([]string, 0, len(files))\n\tfor _, file := range files {\n\t\tfullPath := filepath.Join(dir, file.Name())\n\n\t\tif file.IsDir() {\n\t\t\tsubDirPaths, err := r.readPluginsDir(fullPath)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t\tfullPaths = append(fullPaths, subDirPaths...)\n\t\t\tcontinue\n\t\t}\n\n\t\tif !executable(file) {\n\t\t\tr.logger.Warnf(\"Searching plugin skip file %s, not executable, mode %v, ext %s\", file.Name(), file.Mode(), strings.ToLower(filepath.Ext(file.Name())))\n\t\t\tcontinue\n\t\t}\n\n\t\tfullPaths = append(fullPaths, fullPath)\n\t}\n\treturn fullPaths, nil\n}\n\n// executable determines if a file is executable.\nfunc executable(info os.FileInfo) bool {\n\treturn executableLinux(info) || executableWindows(info)\n}\n\nfunc executableWindows(info os.FileInfo) bool {\n\text := strings.ToLower(filepath.Ext(info.Name()))\n\treturn (ext == \".exe\")\n}\n\nfunc executableLinux(info os.FileInfo) bool {\n\t/*\n\t\tWhen we AND the mode with 0111:\n\n\t\t- 0100 (user executable)\n\t\t- 0010 (group executable)\n\t\t- 0001 (other executable)\n\n\t\tthe result will be 0 if and only if none of the executable bits is set.\n\t*/\n\treturn (info.Mode() & 0111) != 0\n}\n\n// listPlugins executes command, queries it for registered plugins, and returns the list of PluginIdentifiers.\nfunc (r *registry) listPlugins(command string) ([]framework.PluginIdentifier, error) {\n\tprocess, err := r.processFactory.newProcess(command, r.logger, r.logLevel)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdefer process.kill()\n\n\tplugin, err := process.dispense(KindAndName{Kind: common.PluginKindPluginLister})\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tlister, ok := plugin.(framework.PluginLister)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"%T is not a PluginLister\", plugin)\n\t}\n\n\treturn lister.ListPlugins()\n}\n\n// register registers a PluginIdentifier with the registry.\nfunc (r *registry) register(id framework.PluginIdentifier) error {\n\tkey := KindAndName{Kind: id.Kind, Name: id.Name}\n\tif existing, found := r.pluginsByID[key]; found {\n\t\treturn newDuplicatePluginRegistrationError(existing, id)\n\t}\n\n\t// no need to pass list of existing plugins since the check if this exists was done above\n\tif err := common.ValidatePluginName(id.Name, nil); err != nil {\n\t\treturn errors.Errorf(\"invalid plugin name %q: %s\", id.Name, err)\n\t}\n\n\tr.pluginsByID[key] = id\n\tr.pluginsByKind[id.Kind] = append(r.pluginsByKind[id.Kind], id)\n\n\t// if id.Kind is adaptable to newer plugin versions, list it under the other versions as well\n\t// If BackupItemAction is adaptable to BackupItemActionV2, then it would be listed under both\n\t// kinds\n\tif kinds, ok := common.PluginKindsAdaptableTo[id.Kind]; ok {\n\t\tfor _, kind := range kinds {\n\t\t\tr.pluginsByKind[kind] = append(r.pluginsByKind[kind], id)\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// pluginNotFoundError indicates a plugin could not be located for kind and name.\ntype PluginNotFoundError struct {\n\tkind common.PluginKind\n\tname string\n}\n\n// newPluginNotFoundError returns a new pluginNotFoundError for kind and name.\nfunc newPluginNotFoundError(kind common.PluginKind, name string) *PluginNotFoundError {\n\treturn &PluginNotFoundError{\n\t\tkind: kind,\n\t\tname: name,\n\t}\n}\n\nfunc (e *PluginNotFoundError) Error() string {\n\treturn fmt.Sprintf(\"unable to locate %v plugin named %s\", e.kind, e.name)\n}\n\ntype duplicatePluginRegistrationError struct {\n\texisting framework.PluginIdentifier\n\tduplicate framework.PluginIdentifier\n}\n\nfunc newDuplicatePluginRegistrationError(existing, duplicate framework.PluginIdentifier) *duplicatePluginRegistrationError {\n\treturn &duplicatePluginRegistrationError{\n\t\texisting: existing,\n\t\tduplicate: duplicate,\n\t}\n}\n\nfunc (e *duplicatePluginRegistrationError) Error() string {\n\treturn fmt.Sprintf(\n\t\t\"unable to register plugin (kind=%s, name=%s, command=%s) because another plugin is already registered for this kind and name (command=%s)\",\n\t\tstring(e.duplicate.Kind),\n\t\te.duplicate.Name,\n\t\te.duplicate.Command,\n\t\te.existing.Command,\n\t)\n}\n" }, { "path": "pkg/plugin/clientmgmt/process/registry_test.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\thttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage process\n\nimport (\n\t\"io/fs\"\n\t\"os\"\n\t\"sort\"\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestNewRegistry(t *testing.T) {\n\tlogger := test.NewLogger()\n\tlogLevel := logrus.InfoLevel\n\tdir := \"/plugins\"\n\n\tr := NewRegistry(dir, logger, logLevel).(*registry)\n\tassert.Equal(t, dir, r.dir)\n\tassert.Equal(t, logger, r.logger)\n\tassert.Equal(t, logLevel, r.logLevel)\n\tassert.NotNil(t, r.pluginsByID)\n\tassert.Empty(t, r.pluginsByID)\n\tassert.NotNil(t, r.pluginsByKind)\n\tassert.Empty(t, r.pluginsByKind)\n}\n\ntype fakeFileInfo struct {\n\tfs.FileInfo\n\tname string\n\tmode os.FileMode\n}\n\nfunc (f *fakeFileInfo) Mode() os.FileMode {\n\treturn f.mode\n}\n\nfunc (f *fakeFileInfo) Name() string {\n\treturn f.name\n}\n\nfunc TestExecutable(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tfileName string\n\t\tmode uint32\n\t\texpectExecutable bool\n\t}{\n\t\t{\n\t\t\tname: \"no perms\",\n\t\t\tmode: 0000,\n\t\t},\n\t\t{\n\t\t\tname: \"r--r--r--\",\n\t\t\tmode: 0444,\n\t\t},\n\t\t{\n\t\t\tname: \"rw-rw-rw-\",\n\t\t\tmode: 0666,\n\t\t},\n\t\t{\n\t\t\tname: \"--x------\",\n\t\t\tmode: 0100,\n\t\t\texpectExecutable: true,\n\t\t},\n\t\t{\n\t\t\tname: \"-----x---\",\n\t\t\tmode: 0010,\n\t\t\texpectExecutable: true,\n\t\t},\n\t\t{\n\t\t\tname: \"--------x\",\n\t\t\tmode: 0001,\n\t\t\texpectExecutable: true,\n\t\t},\n\t\t{\n\t\t\tname: \"rwxrwxrwx\",\n\t\t\tmode: 0777,\n\t\t\texpectExecutable: true,\n\t\t},\n\t\t{\n\t\t\tname: \"windows lower case\",\n\t\t\tfileName: \"test.exe\",\n\t\t\tmode: 0,\n\t\t\texpectExecutable: true,\n\t\t},\n\t\t{\n\t\t\tname: \"windows upper case\",\n\t\t\tfileName: \"test.EXE\",\n\t\t\tmode: 0,\n\t\t\texpectExecutable: true,\n\t\t},\n\t\t{\n\t\t\tname: \"windows wrong ext\",\n\t\t\tfileName: \"test.EXE1\",\n\t\t\tmode: 0,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tinfo := &fakeFileInfo{\n\t\t\t\tname: test.fileName,\n\t\t\t\tmode: os.FileMode(test.mode),\n\t\t\t}\n\n\t\t\tassert.Equal(t, test.expectExecutable, executable(info))\n\t\t})\n\t}\n}\n\nfunc TestReadPluginsDir(t *testing.T) {\n\tlogger := test.NewLogger()\n\tlogLevel := logrus.InfoLevel\n\tdir := \"/plugins\"\n\n\tr := NewRegistry(dir, logger, logLevel).(*registry)\n\tr.fs = test.NewFakeFileSystem().\n\t\tWithFileAndMode(\"/plugins/executable1\", []byte(\"plugin1\"), 0755).\n\t\tWithFileAndMode(\"/plugins/nonexecutable2\", []byte(\"plugin2\"), 0644).\n\t\tWithFileAndMode(\"/plugins/executable3\", []byte(\"plugin3\"), 0755).\n\t\tWithFileAndMode(\"/plugins/nested/executable4\", []byte(\"plugin4\"), 0755).\n\t\tWithFileAndMode(\"/plugins/nested/nonexecutable5\", []byte(\"plugin4\"), 0644).\n\t\tWithFileAndMode(\"/plugins/nested/win-exe1.exe\", []byte(\"plugin4\"), 0600).\n\t\tWithFileAndMode(\"/plugins/nested/WIN-EXE2.EXE\", []byte(\"plugin4\"), 0600)\n\n\tplugins, err := r.readPluginsDir(dir)\n\trequire.NoError(t, err)\n\n\texpected := []string{\n\t\t\"/plugins/executable1\",\n\t\t\"/plugins/executable3\",\n\t\t\"/plugins/nested/executable4\",\n\t\t\"/plugins/nested/win-exe1.exe\",\n\t\t\"/plugins/nested/WIN-EXE2.EXE\",\n\t}\n\n\tsort.Strings(plugins)\n\tsort.Strings(expected)\n\tassert.Equal(t, expected, plugins)\n}\n" }, { "path": "pkg/plugin/clientmgmt/process/restartable_process.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage process\n\nimport (\n\t\"sync\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n)\n\ntype RestartableProcessFactory interface {\n\tNewRestartableProcess(command string, logger logrus.FieldLogger, logLevel logrus.Level) (RestartableProcess, error)\n}\n\ntype restartableProcessFactory struct {\n}\n\nfunc NewRestartableProcessFactory() RestartableProcessFactory {\n\treturn &restartableProcessFactory{}\n}\n\nfunc (rpf *restartableProcessFactory) NewRestartableProcess(command string, logger logrus.FieldLogger, logLevel logrus.Level) (RestartableProcess, error) {\n\treturn newRestartableProcess(command, logger, logLevel)\n}\n\ntype RestartableProcess interface {\n\tAddReinitializer(key KindAndName, r Reinitializer)\n\tReset() error\n\tResetIfNeeded() error\n\tGetByKindAndName(key KindAndName) (any, error)\n\tStop()\n}\n\n// restartableProcess encapsulates the lifecycle for all plugins contained in a single executable file. It is able\n// to restart a plugin process if it is terminated for any reason. If this happens, all plugins are reinitialized using\n// the original configuration data.\ntype restartableProcess struct {\n\tcommand string\n\tlogger logrus.FieldLogger\n\tlogLevel logrus.Level\n\n\t// lock guards all of the fields below\n\tlock sync.RWMutex\n\tprocess Process\n\tplugins map[KindAndName]any\n\treinitializers map[KindAndName]Reinitializer\n\tresetFailures int\n}\n\n// reinitializer is capable of reinitializing a restartable plugin instance using the newly dispensed plugin.\ntype Reinitializer interface {\n\t// reinitialize reinitializes a restartable plugin instance using the newly dispensed plugin.\n\tReinitialize(dispensed any) error\n}\n\n// newRestartableProcess creates a new restartableProcess for the given command and options.\nfunc newRestartableProcess(command string, logger logrus.FieldLogger, logLevel logrus.Level) (RestartableProcess, error) {\n\tp := &restartableProcess{\n\t\tcommand: command,\n\t\tlogger: logger,\n\t\tlogLevel: logLevel,\n\t\tplugins: make(map[KindAndName]any),\n\t\treinitializers: make(map[KindAndName]Reinitializer),\n\t}\n\n\t// This launches the process\n\terr := p.Reset()\n\n\treturn p, err\n}\n\n// AddReinitializer registers the reinitializer r for key.\nfunc (p *restartableProcess) AddReinitializer(key KindAndName, r Reinitializer) {\n\tp.lock.Lock()\n\tdefer p.lock.Unlock()\n\n\tp.reinitializers[key] = r\n}\n\n// Reset acquires the lock and calls resetLH.\nfunc (p *restartableProcess) Reset() error {\n\tp.lock.Lock()\n\tdefer p.lock.Unlock()\n\n\treturn p.resetLH()\n}\n\n// resetLH (re)launches the plugin process. It redispenses all previously dispensed plugins and reinitializes all the\n// registered reinitializers using the newly dispensed plugins.\n//\n// Callers of resetLH *must* acquire the lock before calling it.\nfunc (p *restartableProcess) resetLH() error {\n\tif p.resetFailures > 10 {\n\t\treturn errors.Errorf(\"unable to restart plugin process: exceeded maximum number of reset failures\")\n\t}\n\n\tprocess, err := newProcess(p.command, p.logger, p.logLevel)\n\tif err != nil {\n\t\tp.resetFailures++\n\t\treturn err\n\t}\n\tp.process = process\n\n\t// Redispense any previously dispensed plugins, reinitializing if necessary.\n\t// Start by creating a new map to hold the newly dispensed plugins.\n\tnewPlugins := make(map[KindAndName]any)\n\tfor key := range p.plugins {\n\t\t// Re-dispense\n\t\tdispensed, err := p.process.dispense(key)\n\t\tif err != nil {\n\t\t\tp.resetFailures++\n\t\t\treturn err\n\t\t}\n\t\t// Store in the new map\n\t\tnewPlugins[key] = dispensed\n\n\t\t// Reinitialize\n\t\tif r, found := p.reinitializers[key]; found {\n\t\t\tif err := r.Reinitialize(dispensed); err != nil {\n\t\t\t\tp.resetFailures++\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t}\n\n\t// Make sure we update p's plugins!\n\tp.plugins = newPlugins\n\n\tp.resetFailures = 0\n\n\treturn nil\n}\n\n// ResetIfNeeded checks if the plugin process has exited and resets p if it has.\nfunc (p *restartableProcess) ResetIfNeeded() error {\n\tp.lock.Lock()\n\tdefer p.lock.Unlock()\n\n\tif p.process.exited() {\n\t\tp.logger.Info(\"Plugin process exited - restarting.\")\n\t\treturn p.resetLH()\n\t}\n\n\treturn nil\n}\n\n// GetByKindAndName acquires the lock and calls getByKindAndNameLH.\nfunc (p *restartableProcess) GetByKindAndName(key KindAndName) (any, error) {\n\tp.lock.Lock()\n\tdefer p.lock.Unlock()\n\n\treturn p.getByKindAndNameLH(key)\n}\n\n// getByKindAndNameLH returns the dispensed plugin for key. If the plugin hasn't been dispensed before, it dispenses a\n// new one.\nfunc (p *restartableProcess) getByKindAndNameLH(key KindAndName) (any, error) {\n\tdispensed, found := p.plugins[key]\n\tif found {\n\t\treturn dispensed, nil\n\t}\n\n\tdispensed, err := p.process.dispense(key)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tp.plugins[key] = dispensed\n\treturn p.plugins[key], nil\n}\n\n// stop terminates the plugin process.\nfunc (p *restartableProcess) Stop() {\n\tp.lock.Lock()\n\tp.process.kill()\n\tp.lock.Unlock()\n}\n" }, { "path": "pkg/plugin/clientmgmt/restartable_delete_item_action.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage clientmgmt\n\nimport (\n\t\"github.com/pkg/errors\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// restartableDeleteItemAction is a delete item action for a given implementation (such as \"pod\"). It is associated with\n// a restartableProcess, which may be shared and used to run multiple plugins. At the beginning of each method\n// call, the restartableDeleteItemAction asks its restartableProcess to restart itself if needed (e.g. if the\n// process terminated for any reason), then it proceeds with the actual call.\ntype restartableDeleteItemAction struct {\n\tkey process.KindAndName\n\tsharedPluginProcess process.RestartableProcess\n}\n\n// NewRestartableDeleteItemAction returns a new restartableDeleteItemAction.\nfunc NewRestartableDeleteItemAction(name string, sharedPluginProcess process.RestartableProcess) *restartableDeleteItemAction {\n\tr := &restartableDeleteItemAction{\n\t\tkey: process.KindAndName{Kind: common.PluginKindDeleteItemAction, Name: name},\n\t\tsharedPluginProcess: sharedPluginProcess,\n\t}\n\treturn r\n}\n\n// getDeleteItemAction returns the delete item action for this restartableDeleteItemAction. It does *not* restart the\n// plugin process.\nfunc (r *restartableDeleteItemAction) getDeleteItemAction() (velero.DeleteItemAction, error) {\n\tplugin, err := r.sharedPluginProcess.GetByKindAndName(r.key)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdeleteItemAction, ok := plugin.(velero.DeleteItemAction)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"plugin %T is not a DeleteItemAction\", plugin)\n\t}\n\n\treturn deleteItemAction, nil\n}\n\n// getDelegate restarts the plugin process (if needed) and returns the delete item action for this restartableDeleteItemAction.\nfunc (r *restartableDeleteItemAction) getDelegate() (velero.DeleteItemAction, error) {\n\tif err := r.sharedPluginProcess.ResetIfNeeded(); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn r.getDeleteItemAction()\n}\n\n// AppliesTo restarts the plugin's process if needed, then delegates the call.\nfunc (r *restartableDeleteItemAction) AppliesTo() (velero.ResourceSelector, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn velero.ResourceSelector{}, err\n\t}\n\n\treturn delegate.AppliesTo()\n}\n\n// Execute restarts the plugin's process if needed, then delegates the call.\nfunc (r *restartableDeleteItemAction) Execute(input *velero.DeleteItemActionExecuteInput) error {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn delegate.Execute(input)\n}\n" }, { "path": "pkg/plugin/clientmgmt/restartable_delete_item_action_test.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage clientmgmt\n\nimport (\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\n\t\"github.com/vmware-tanzu/velero/internal/restartabletest\"\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero/mocks\"\n)\n\nfunc TestRestartableGetDeleteItemAction(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tplugin any\n\t\tgetError error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"error getting by kind and name\",\n\t\t\tgetError: errors.Errorf(\"get error\"),\n\t\t\texpectedError: \"get error\",\n\t\t},\n\t\t{\n\t\t\tname: \"wrong type\",\n\t\t\tplugin: 3,\n\t\t\texpectedError: \"plugin int is not a DeleteItemAction\",\n\t\t},\n\t\t{\n\t\t\tname: \"happy path\",\n\t\t\tplugin: new(mocks.DeleteItemAction),\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tp := new(restartabletest.MockRestartableProcess)\n\t\t\tdefer p.AssertExpectations(t)\n\n\t\t\tname := \"pod\"\n\t\t\tkey := process.KindAndName{Kind: common.PluginKindDeleteItemAction, Name: name}\n\t\t\tp.On(\"GetByKindAndName\", key).Return(tc.plugin, tc.getError)\n\n\t\t\tr := NewRestartableDeleteItemAction(name, p)\n\t\t\ta, err := r.getDeleteItemAction()\n\t\t\tif tc.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, tc.expectedError)\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.Equal(t, tc.plugin, a)\n\t\t})\n\t}\n}\n\nfunc TestRestartableDeleteItemActionGetDelegate(t *testing.T) {\n\tp := new(restartabletest.MockRestartableProcess)\n\tdefer p.AssertExpectations(t)\n\n\t// Reset error\n\tp.On(\"ResetIfNeeded\").Return(errors.Errorf(\"reset error\")).Once()\n\tname := \"pod\"\n\tr := NewRestartableDeleteItemAction(name, p)\n\ta, err := r.getDelegate()\n\tassert.Nil(t, a)\n\trequire.EqualError(t, err, \"reset error\")\n\n\t// Happy path\n\t// Currently broken since this mocks out the restore item action interface\n\tp.On(\"ResetIfNeeded\").Return(nil)\n\texpected := new(mocks.DeleteItemAction)\n\tkey := process.KindAndName{Kind: common.PluginKindDeleteItemAction, Name: name}\n\tp.On(\"GetByKindAndName\", key).Return(expected, nil)\n\n\ta, err = r.getDelegate()\n\trequire.NoError(t, err)\n\tassert.Equal(t, expected, a)\n}\n\nfunc TestRestartableDeleteItemActionDelegatedFunctions(t *testing.T) {\n\tpv := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"color\": \"blue\",\n\t\t},\n\t}\n\n\tbackup := &api.Backup{}\n\n\tinput := &velero.DeleteItemActionExecuteInput{\n\t\tItem: pv,\n\t\tBackup: backup,\n\t}\n\n\trestartabletest.RunRestartableDelegateTests(\n\t\tt,\n\t\tcommon.PluginKindDeleteItemAction,\n\t\tfunc(key process.KindAndName, p process.RestartableProcess) any {\n\t\t\treturn &restartableDeleteItemAction{\n\t\t\t\tkey: key,\n\t\t\t\tsharedPluginProcess: p,\n\t\t\t}\n\t\t},\n\t\tfunc() restartabletest.Mockable {\n\t\t\t// Currently broken because this mocks the restore item action interface\n\t\t\treturn new(mocks.DeleteItemAction)\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"AppliesTo\",\n\t\t\tInputs: []any{},\n\t\t\tExpectedErrorOutputs: []any{velero.ResourceSelector{}, errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{velero.ResourceSelector{IncludedNamespaces: []string{\"a\"}}, errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"Execute\",\n\t\t\tInputs: []any{input},\n\t\t\tExpectedErrorOutputs: []any{errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{errors.Errorf(\"delegate error\")},\n\t\t},\n\t)\n}\n" }, { "path": "pkg/plugin/clientmgmt/restartable_object_store.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage clientmgmt\n\nimport (\n\t\"io\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// restartableObjectStore is an object store for a given implementation (such as \"aws\"). It is associated with\n// a restartableProcess, which may be shared and used to run multiple plugins. At the beginning of each method\n// call, the restartableObjectStore asks its restartableProcess to restart itself if needed (e.g. if the\n// process terminated for any reason), then it proceeds with the actual call.\ntype restartableObjectStore struct {\n\tkey process.KindAndName\n\tsharedPluginProcess process.RestartableProcess\n\t// config contains the data used to initialize the plugin. It is used to reinitialize the plugin in the event its\n\t// sharedPluginProcess gets restarted.\n\tconfig map[string]string\n}\n\n// NewRestartableObjectStore returns a new restartableObjectStore.\nfunc NewRestartableObjectStore(name string, sharedPluginProcess process.RestartableProcess) *restartableObjectStore {\n\tkey := process.KindAndName{Kind: common.PluginKindObjectStore, Name: name}\n\tr := &restartableObjectStore{\n\t\tkey: key,\n\t\tsharedPluginProcess: sharedPluginProcess,\n\t}\n\n\t// Register our reinitializer so we can reinitialize after a restart with r.config.\n\tsharedPluginProcess.AddReinitializer(key, r)\n\n\treturn r\n}\n\n// reinitialize reinitializes a re-dispensed plugin using the initial data passed to Init().\nfunc (r *restartableObjectStore) Reinitialize(dispensed any) error {\n\tobjectStore, ok := dispensed.(velero.ObjectStore)\n\tif !ok {\n\t\treturn errors.Errorf(\"plugin %T is not a ObjectStore\", dispensed)\n\t}\n\n\treturn r.init(objectStore, r.config)\n}\n\n// getObjectStore returns the object store for this restartableObjectStore. It does *not* restart the\n// plugin process.\nfunc (r *restartableObjectStore) getObjectStore() (velero.ObjectStore, error) {\n\tplugin, err := r.sharedPluginProcess.GetByKindAndName(r.key)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tobjectStore, ok := plugin.(velero.ObjectStore)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"plugin %T is not a ObjectStore\", plugin)\n\t}\n\n\treturn objectStore, nil\n}\n\n// getDelegate restarts the plugin process (if needed) and returns the object store for this restartableObjectStore.\nfunc (r *restartableObjectStore) getDelegate() (velero.ObjectStore, error) {\n\tif err := r.sharedPluginProcess.ResetIfNeeded(); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn r.getObjectStore()\n}\n\n// Init initializes the object store instance using config. If this is the first invocation, r stores config for future\n// reinitialization needs. Init does NOT restart the shared plugin process. Init may only be called once.\nfunc (r *restartableObjectStore) Init(config map[string]string) error {\n\tif r.config != nil {\n\t\treturn errors.Errorf(\"already initialized\")\n\t}\n\n\t// Not using getDelegate() to avoid possible infinite recursion\n\tdelegate, err := r.getObjectStore()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tr.config = config\n\n\treturn r.init(delegate, config)\n}\n\n// init calls Init on objectStore with config. This is split out from Init() so that both Init() and reinitialize() may\n// call it using a specific ObjectStore.\nfunc (r *restartableObjectStore) init(objectStore velero.ObjectStore, config map[string]string) error {\n\treturn objectStore.Init(config)\n}\n\n// PutObject restarts the plugin's process if needed, then delegates the call.\nfunc (r *restartableObjectStore) PutObject(bucket string, key string, body io.Reader) error {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn err\n\t}\n\treturn delegate.PutObject(bucket, key, body)\n}\n\n// ObjectExists restarts the plugin's process if needed, then delegates the call.\nfunc (r *restartableObjectStore) ObjectExists(bucket, key string) (bool, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn false, err\n\t}\n\treturn delegate.ObjectExists(bucket, key)\n}\n\n// GetObject restarts the plugin's process if needed, then delegates the call.\nfunc (r *restartableObjectStore) GetObject(bucket string, key string) (io.ReadCloser, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn delegate.GetObject(bucket, key)\n}\n\n// ListCommonPrefixes restarts the plugin's process if needed, then delegates the call.\nfunc (r *restartableObjectStore) ListCommonPrefixes(bucket string, prefix string, delimiter string) ([]string, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn delegate.ListCommonPrefixes(bucket, prefix, delimiter)\n}\n\n// ListObjects restarts the plugin's process if needed, then delegates the call.\nfunc (r *restartableObjectStore) ListObjects(bucket string, prefix string) ([]string, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn delegate.ListObjects(bucket, prefix)\n}\n\n// DeleteObject restarts the plugin's process if needed, then delegates the call.\nfunc (r *restartableObjectStore) DeleteObject(bucket string, key string) error {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn err\n\t}\n\treturn delegate.DeleteObject(bucket, key)\n}\n\n// CreateSignedURL restarts the plugin's process if needed, then delegates the call.\nfunc (r *restartableObjectStore) CreateSignedURL(bucket string, key string, ttl time.Duration) (string, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\treturn delegate.CreateSignedURL(bucket, key, ttl)\n}\n" }, { "path": "pkg/plugin/clientmgmt/restartable_object_store_test.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage clientmgmt\n\nimport (\n\t\"io\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/vmware-tanzu/velero/internal/restartabletest\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tprovidermocks \"github.com/vmware-tanzu/velero/pkg/plugin/velero/mocks\"\n)\n\nfunc TestRestartableGetObjectStore(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tplugin any\n\t\tgetError error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"error getting by kind and name\",\n\t\t\tgetError: errors.Errorf(\"get error\"),\n\t\t\texpectedError: \"get error\",\n\t\t},\n\t\t{\n\t\t\tname: \"wrong type\",\n\t\t\tplugin: 3,\n\t\t\texpectedError: \"plugin int is not a ObjectStore\",\n\t\t},\n\t\t{\n\t\t\tname: \"happy path\",\n\t\t\tplugin: new(providermocks.ObjectStore),\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tp := new(restartabletest.MockRestartableProcess)\n\t\t\tp.Test(t)\n\t\t\tdefer p.AssertExpectations(t)\n\n\t\t\tname := \"aws\"\n\t\t\tkey := process.KindAndName{Kind: common.PluginKindObjectStore, Name: name}\n\t\t\tp.On(\"GetByKindAndName\", key).Return(tc.plugin, tc.getError)\n\n\t\t\tr := &restartableObjectStore{\n\t\t\t\tkey: key,\n\t\t\t\tsharedPluginProcess: p,\n\t\t\t}\n\t\t\ta, err := r.getObjectStore()\n\t\t\tif tc.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, tc.expectedError)\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.Equal(t, tc.plugin, a)\n\t\t})\n\t}\n}\n\nfunc TestRestartableObjectStoreReinitialize(t *testing.T) {\n\tp := new(restartabletest.MockRestartableProcess)\n\tp.Test(t)\n\tdefer p.AssertExpectations(t)\n\n\tname := \"aws\"\n\tkey := process.KindAndName{Kind: common.PluginKindObjectStore, Name: name}\n\tr := &restartableObjectStore{\n\t\tkey: key,\n\t\tsharedPluginProcess: p,\n\t\tconfig: map[string]string{\n\t\t\t\"color\": \"blue\",\n\t\t},\n\t}\n\n\terr := r.Reinitialize(3)\n\trequire.EqualError(t, err, \"plugin int is not a ObjectStore\")\n\n\tobjectStore := new(providermocks.ObjectStore)\n\tobjectStore.Test(t)\n\tdefer objectStore.AssertExpectations(t)\n\n\tobjectStore.On(\"Init\", r.config).Return(errors.Errorf(\"init error\")).Once()\n\terr = r.Reinitialize(objectStore)\n\trequire.EqualError(t, err, \"init error\")\n\n\tobjectStore.On(\"Init\", r.config).Return(nil)\n\terr = r.Reinitialize(objectStore)\n\tassert.NoError(t, err)\n}\n\nfunc TestRestartableObjectStoreGetDelegate(t *testing.T) {\n\tp := new(restartabletest.MockRestartableProcess)\n\tp.Test(t)\n\tdefer p.AssertExpectations(t)\n\n\t// Reset error\n\tp.On(\"ResetIfNeeded\").Return(errors.Errorf(\"reset error\")).Once()\n\tname := \"aws\"\n\tkey := process.KindAndName{Kind: common.PluginKindObjectStore, Name: name}\n\tr := &restartableObjectStore{\n\t\tkey: key,\n\t\tsharedPluginProcess: p,\n\t}\n\ta, err := r.getDelegate()\n\tassert.Nil(t, a)\n\trequire.EqualError(t, err, \"reset error\")\n\n\t// Happy path\n\tp.On(\"ResetIfNeeded\").Return(nil)\n\tobjectStore := new(providermocks.ObjectStore)\n\tobjectStore.Test(t)\n\tdefer objectStore.AssertExpectations(t)\n\tp.On(\"GetByKindAndName\", key).Return(objectStore, nil)\n\n\ta, err = r.getDelegate()\n\trequire.NoError(t, err)\n\tassert.Equal(t, objectStore, a)\n}\n\nfunc TestRestartableObjectStoreInit(t *testing.T) {\n\tp := new(restartabletest.MockRestartableProcess)\n\tp.Test(t)\n\tdefer p.AssertExpectations(t)\n\n\t// getObjectStore error\n\tname := \"aws\"\n\tkey := process.KindAndName{Kind: common.PluginKindObjectStore, Name: name}\n\tr := &restartableObjectStore{\n\t\tkey: key,\n\t\tsharedPluginProcess: p,\n\t}\n\tp.On(\"GetByKindAndName\", key).Return(nil, errors.Errorf(\"GetByKindAndName error\")).Once()\n\n\tconfig := map[string]string{\n\t\t\"color\": \"blue\",\n\t}\n\terr := r.Init(config)\n\trequire.EqualError(t, err, \"GetByKindAndName error\")\n\n\t// Delegate returns error\n\tobjectStore := new(providermocks.ObjectStore)\n\tobjectStore.Test(t)\n\tdefer objectStore.AssertExpectations(t)\n\tp.On(\"GetByKindAndName\", key).Return(objectStore, nil)\n\tobjectStore.On(\"Init\", config).Return(errors.Errorf(\"Init error\")).Once()\n\n\terr = r.Init(config)\n\trequire.EqualError(t, err, \"Init error\")\n\n\t// wipe this out because the previous failed Init call set it\n\tr.config = nil\n\n\t// Happy path\n\tobjectStore.On(\"Init\", config).Return(nil)\n\terr = r.Init(config)\n\trequire.NoError(t, err)\n\tassert.Equal(t, config, r.config)\n\n\t// Calling Init twice is forbidden\n\terr = r.Init(config)\n\tassert.EqualError(t, err, \"already initialized\")\n}\n\nfunc TestRestartableObjectStoreDelegatedFunctions(t *testing.T) {\n\trestartabletest.RunRestartableDelegateTests(\n\t\tt,\n\t\tcommon.PluginKindObjectStore,\n\t\tfunc(key process.KindAndName, p process.RestartableProcess) any {\n\t\t\treturn &restartableObjectStore{\n\t\t\t\tkey: key,\n\t\t\t\tsharedPluginProcess: p,\n\t\t\t}\n\t\t},\n\t\tfunc() restartabletest.Mockable {\n\t\t\treturn new(providermocks.ObjectStore)\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"PutObject\",\n\t\t\tInputs: []any{\"bucket\", \"key\", strings.NewReader(\"body\")},\n\t\t\tExpectedErrorOutputs: []any{errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"GetObject\",\n\t\t\tInputs: []any{\"bucket\", \"key\"},\n\t\t\tExpectedErrorOutputs: []any{nil, errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{io.NopCloser(strings.NewReader(\"object\")), errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"ListCommonPrefixes\",\n\t\t\tInputs: []any{\"bucket\", \"prefix\", \"delimiter\"},\n\t\t\tExpectedErrorOutputs: []any{([]string)(nil), errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{[]string{\"a\", \"b\"}, errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"ListObjects\",\n\t\t\tInputs: []any{\"bucket\", \"prefix\"},\n\t\t\tExpectedErrorOutputs: []any{([]string)(nil), errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{[]string{\"a\", \"b\"}, errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"DeleteObject\",\n\t\t\tInputs: []any{\"bucket\", \"key\"},\n\t\t\tExpectedErrorOutputs: []any{errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"CreateSignedURL\",\n\t\t\tInputs: []any{\"bucket\", \"key\", 30 * time.Minute},\n\t\t\tExpectedErrorOutputs: []any{\"\", errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{\"signedURL\", errors.Errorf(\"delegate error\")},\n\t\t},\n\t)\n}\n" }, { "path": "pkg/plugin/clientmgmt/restoreitemaction/v1/restartable_restore_item_action.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\t\"github.com/pkg/errors\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\triav1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/restoreitemaction/v1\"\n)\n\n// AdaptedRestoreItemAction is a restore item action adapted to the v1 RestoreItemAction API\ntype AdaptedRestoreItemAction struct {\n\tKind common.PluginKind\n\n\t// Get returns a restartable RestoreItemAction for the given name and process, wrapping if necessary\n\tGetRestartable func(name string, restartableProcess process.RestartableProcess) riav1.RestoreItemAction\n}\n\nfunc AdaptedRestoreItemActions() []AdaptedRestoreItemAction {\n\treturn []AdaptedRestoreItemAction{\n\t\t{\n\t\t\tKind: common.PluginKindRestoreItemAction,\n\t\t\tGetRestartable: func(name string, restartableProcess process.RestartableProcess) riav1.RestoreItemAction {\n\t\t\t\treturn NewRestartableRestoreItemAction(name, restartableProcess)\n\t\t\t},\n\t\t},\n\t}\n}\n\n// RestartableRestoreItemAction is a restore item action for a given implementation (such as \"pod\"). It is associated with\n// a restartableProcess, which may be shared and used to run multiple plugins. At the beginning of each method\n// call, the RestartableRestoreItemAction asks its restartableProcess to restart itself if needed (e.g. if the\n// process terminated for any reason), then it proceeds with the actual call.\ntype RestartableRestoreItemAction struct {\n\tKey process.KindAndName\n\tSharedPluginProcess process.RestartableProcess\n}\n\n// NewRestartableRestoreItemAction returns a new RestartableRestoreItemAction.\nfunc NewRestartableRestoreItemAction(name string, sharedPluginProcess process.RestartableProcess) *RestartableRestoreItemAction {\n\tr := &RestartableRestoreItemAction{\n\t\tKey: process.KindAndName{Kind: common.PluginKindRestoreItemAction, Name: name},\n\t\tSharedPluginProcess: sharedPluginProcess,\n\t}\n\treturn r\n}\n\n// getRestoreItemAction returns the restore item action for this RestartableRestoreItemAction. It does *not* restart the\n// plugin process.\nfunc (r *RestartableRestoreItemAction) getRestoreItemAction() (riav1.RestoreItemAction, error) {\n\tplugin, err := r.SharedPluginProcess.GetByKindAndName(r.Key)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\trestoreItemAction, ok := plugin.(riav1.RestoreItemAction)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"plugin %T is not a RestoreItemAction\", plugin)\n\t}\n\n\treturn restoreItemAction, nil\n}\n\n// getDelegate restarts the plugin process (if needed) and returns the restore item action for this RestartableRestoreItemAction.\nfunc (r *RestartableRestoreItemAction) getDelegate() (riav1.RestoreItemAction, error) {\n\tif err := r.SharedPluginProcess.ResetIfNeeded(); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn r.getRestoreItemAction()\n}\n\n// AppliesTo restarts the plugin's process if needed, then delegates the call.\nfunc (r RestartableRestoreItemAction) AppliesTo() (velero.ResourceSelector, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn velero.ResourceSelector{}, err\n\t}\n\n\treturn delegate.AppliesTo()\n}\n\n// Execute restarts the plugin's process if needed, then delegates the call.\nfunc (r *RestartableRestoreItemAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn delegate.Execute(input)\n}\n" }, { "path": "pkg/plugin/clientmgmt/restoreitemaction/v1/restartable_restore_item_action_test.go", "content": "/*\nCopyright 2018, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\n\t\"github.com/vmware-tanzu/velero/internal/restartabletest\"\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tmocks \"github.com/vmware-tanzu/velero/pkg/plugin/velero/mocks/restoreitemaction/v1\"\n)\n\nfunc TestRestartableGetRestoreItemAction(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tplugin any\n\t\tgetError error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"error getting by kind and name\",\n\t\t\tgetError: errors.Errorf(\"get error\"),\n\t\t\texpectedError: \"get error\",\n\t\t},\n\t\t{\n\t\t\tname: \"wrong type\",\n\t\t\tplugin: 3,\n\t\t\texpectedError: \"plugin int is not a RestoreItemAction\",\n\t\t},\n\t\t{\n\t\t\tname: \"happy path\",\n\t\t\tplugin: new(mocks.RestoreItemAction),\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tp := new(restartabletest.MockRestartableProcess)\n\t\t\tdefer p.AssertExpectations(t)\n\n\t\t\tname := \"pod\"\n\t\t\tkey := process.KindAndName{Kind: common.PluginKindRestoreItemAction, Name: name}\n\t\t\tp.On(\"GetByKindAndName\", key).Return(tc.plugin, tc.getError)\n\n\t\t\tr := NewRestartableRestoreItemAction(name, p)\n\t\t\ta, err := r.getRestoreItemAction()\n\t\t\tif tc.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, tc.expectedError)\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.Equal(t, tc.plugin, a)\n\t\t})\n\t}\n}\n\nfunc TestRestartableRestoreItemActionGetDelegate(t *testing.T) {\n\tp := new(restartabletest.MockRestartableProcess)\n\tdefer p.AssertExpectations(t)\n\n\t// Reset error\n\tp.On(\"ResetIfNeeded\").Return(errors.Errorf(\"reset error\")).Once()\n\tname := \"pod\"\n\tr := NewRestartableRestoreItemAction(name, p)\n\ta, err := r.getDelegate()\n\tassert.Nil(t, a)\n\trequire.EqualError(t, err, \"reset error\")\n\n\t// Happy path\n\tp.On(\"ResetIfNeeded\").Return(nil)\n\texpected := new(mocks.RestoreItemAction)\n\tkey := process.KindAndName{Kind: common.PluginKindRestoreItemAction, Name: name}\n\tp.On(\"GetByKindAndName\", key).Return(expected, nil)\n\n\ta, err = r.getDelegate()\n\trequire.NoError(t, err)\n\tassert.Equal(t, expected, a)\n}\n\nfunc TestRestartableRestoreItemActionDelegatedFunctions(t *testing.T) {\n\tpv := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"color\": \"blue\",\n\t\t},\n\t}\n\n\tinput := &velero.RestoreItemActionExecuteInput{\n\t\tItem: pv,\n\t\tItemFromBackup: pv,\n\t\tRestore: new(v1.Restore),\n\t}\n\n\toutput := &velero.RestoreItemActionExecuteOutput{\n\t\tUpdatedItem: &unstructured.Unstructured{\n\t\t\tObject: map[string]any{\n\t\t\t\t\"color\": \"green\",\n\t\t\t},\n\t\t},\n\t}\n\n\trestartabletest.RunRestartableDelegateTests(\n\t\tt,\n\t\tcommon.PluginKindRestoreItemAction,\n\t\tfunc(key process.KindAndName, p process.RestartableProcess) any {\n\t\t\treturn &RestartableRestoreItemAction{\n\t\t\t\tKey: key,\n\t\t\t\tSharedPluginProcess: p,\n\t\t\t}\n\t\t},\n\t\tfunc() restartabletest.Mockable {\n\t\t\treturn new(mocks.RestoreItemAction)\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"AppliesTo\",\n\t\t\tInputs: []any{},\n\t\t\tExpectedErrorOutputs: []any{velero.ResourceSelector{}, errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{velero.ResourceSelector{IncludedNamespaces: []string{\"a\"}}, errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"Execute\",\n\t\t\tInputs: []any{input},\n\t\t\tExpectedErrorOutputs: []any{nil, errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{output, errors.Errorf(\"delegate error\")},\n\t\t},\n\t)\n}\n" }, { "path": "pkg/plugin/clientmgmt/restoreitemaction/v2/restartable_restore_item_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v2\n\nimport (\n\t\"github.com/pkg/errors\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n\triav1cli \"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/restoreitemaction/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\triav2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/restoreitemaction/v2\"\n)\n\n// AdaptedRestoreItemAction is a v1 RestoreItemAction adapted to implement the v2 API\ntype AdaptedRestoreItemAction struct {\n\tKind common.PluginKind\n\n\t// Get returns a restartable RestoreItemAction for the given name and process, wrapping if necessary\n\tGetRestartable func(name string, restartableProcess process.RestartableProcess) riav2.RestoreItemAction\n}\n\nfunc AdaptedRestoreItemActions() []AdaptedRestoreItemAction {\n\treturn []AdaptedRestoreItemAction{\n\t\t{\n\t\t\tKind: common.PluginKindRestoreItemActionV2,\n\t\t\tGetRestartable: func(name string, restartableProcess process.RestartableProcess) riav2.RestoreItemAction {\n\t\t\t\treturn NewRestartableRestoreItemAction(name, restartableProcess)\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tKind: common.PluginKindRestoreItemAction,\n\t\t\tGetRestartable: func(name string, restartableProcess process.RestartableProcess) riav2.RestoreItemAction {\n\t\t\t\treturn NewAdaptedV1RestartableRestoreItemAction(riav1cli.NewRestartableRestoreItemAction(name, restartableProcess))\n\t\t\t},\n\t\t},\n\t}\n}\n\n// RestartableRestoreItemAction is a restore item action for a given implementation (such as \"pod\"). It is associated with\n// a restartableProcess, which may be shared and used to run multiple plugins. At the beginning of each method\n// call, the RestartableRestoreItemAction asks its restartableProcess to restart itself if needed (e.g. if the\n// process terminated for any reason), then it proceeds with the actual call.\ntype RestartableRestoreItemAction struct {\n\tKey process.KindAndName\n\tSharedPluginProcess process.RestartableProcess\n}\n\n// NewRestartableRestoreItemAction returns a new RestartableRestoreItemAction.\nfunc NewRestartableRestoreItemAction(name string, sharedPluginProcess process.RestartableProcess) *RestartableRestoreItemAction {\n\tr := &RestartableRestoreItemAction{\n\t\tKey: process.KindAndName{Kind: common.PluginKindRestoreItemActionV2, Name: name},\n\t\tSharedPluginProcess: sharedPluginProcess,\n\t}\n\treturn r\n}\n\n// getRestoreItemAction returns the restore item action for this RestartableRestoreItemAction. It does *not* restart the\n// plugin process.\nfunc (r *RestartableRestoreItemAction) getRestoreItemAction() (riav2.RestoreItemAction, error) {\n\tplugin, err := r.SharedPluginProcess.GetByKindAndName(r.Key)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\trestoreItemAction, ok := plugin.(riav2.RestoreItemAction)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"plugin %T is not a RestoreItemActionV2\", plugin)\n\t}\n\n\treturn restoreItemAction, nil\n}\n\n// getDelegate restarts the plugin process (if needed) and returns the restore item action for this RestartableRestoreItemAction.\nfunc (r *RestartableRestoreItemAction) getDelegate() (riav2.RestoreItemAction, error) {\n\tif err := r.SharedPluginProcess.ResetIfNeeded(); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn r.getRestoreItemAction()\n}\n\n// Name returns the plugin's name.\nfunc (r *RestartableRestoreItemAction) Name() string {\n\treturn r.Key.Name\n}\n\n// AppliesTo restarts the plugin's process if needed, then delegates the call.\nfunc (r RestartableRestoreItemAction) AppliesTo() (velero.ResourceSelector, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn velero.ResourceSelector{}, err\n\t}\n\n\treturn delegate.AppliesTo()\n}\n\n// Execute restarts the plugin's process if needed, then delegates the call.\nfunc (r *RestartableRestoreItemAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn delegate.Execute(input)\n}\n\n// Progress restarts the plugin's process if needed, then delegates the call.\nfunc (r *RestartableRestoreItemAction) Progress(operationID string, restore *api.Restore) (velero.OperationProgress, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn velero.OperationProgress{}, err\n\t}\n\n\treturn delegate.Progress(operationID, restore)\n}\n\n// Cancel restarts the plugin's process if needed, then delegates the call.\nfunc (r *RestartableRestoreItemAction) Cancel(operationID string, restore *api.Restore) error {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn delegate.Cancel(operationID, restore)\n}\n\n// AreAdditionalItemsReady restarts the plugin's process if needed, then delegates the call.\nfunc (r *RestartableRestoreItemAction) AreAdditionalItemsReady(additionalItems []velero.ResourceIdentifier, restore *api.Restore) (bool, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\treturn delegate.AreAdditionalItemsReady(additionalItems, restore)\n}\n\ntype AdaptedV1RestartableRestoreItemAction struct {\n\tV1Restartable *riav1cli.RestartableRestoreItemAction\n}\n\n// NewAdaptedV1RestartableRestoreItemAction returns a new v1 RestartableRestoreItemAction adapted to v2\nfunc NewAdaptedV1RestartableRestoreItemAction(v1Restartable *riav1cli.RestartableRestoreItemAction) *AdaptedV1RestartableRestoreItemAction {\n\tr := &AdaptedV1RestartableRestoreItemAction{\n\t\tV1Restartable: v1Restartable,\n\t}\n\treturn r\n}\n\n// Name restarts the plugin's name.\nfunc (r *AdaptedV1RestartableRestoreItemAction) Name() string {\n\treturn r.V1Restartable.Key.Name\n}\n\n// AppliesTo delegates to the v1 AppliesTo call.\nfunc (r *AdaptedV1RestartableRestoreItemAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn r.V1Restartable.AppliesTo()\n}\n\n// Execute delegates to the v1 Execute call, returning an empty operationID.\nfunc (r *AdaptedV1RestartableRestoreItemAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\treturn r.V1Restartable.Execute(input)\n}\n\n// Progress returns with an error since v1 plugins will never return an operationID, which means that\n// any operationID passed in here will be invalid.\nfunc (r *AdaptedV1RestartableRestoreItemAction) Progress(operationID string, restore *api.Restore) (velero.OperationProgress, error) {\n\treturn velero.OperationProgress{}, riav2.AsyncOperationsNotSupportedError()\n}\n\n// Cancel just returns without error since v1 plugins don't implement it.\nfunc (r *AdaptedV1RestartableRestoreItemAction) Cancel(operationID string, restore *api.Restore) error {\n\treturn nil\n}\n\n// AreAdditionalItemsReady just returns true since v1 plugins don't wait for items.\nfunc (r *AdaptedV1RestartableRestoreItemAction) AreAdditionalItemsReady(additionalItems []velero.ResourceIdentifier, restore *api.Restore) (bool, error) {\n\treturn true, nil\n}\n" }, { "path": "pkg/plugin/clientmgmt/restoreitemaction/v2/restartable_restore_item_action_test.go", "content": "/*\nCopyright 2018, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v2\n\nimport (\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\n\t\"github.com/vmware-tanzu/velero/internal/restartabletest\"\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tmocks \"github.com/vmware-tanzu/velero/pkg/plugin/velero/mocks/restoreitemaction/v2\"\n)\n\nfunc TestRestartableGetRestoreItemAction(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tplugin any\n\t\tgetError error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"error getting by kind and name\",\n\t\t\tgetError: errors.Errorf(\"get error\"),\n\t\t\texpectedError: \"get error\",\n\t\t},\n\t\t{\n\t\t\tname: \"wrong type\",\n\t\t\tplugin: 3,\n\t\t\texpectedError: \"plugin int is not a RestoreItemActionV2\",\n\t\t},\n\t\t{\n\t\t\tname: \"happy path\",\n\t\t\tplugin: new(mocks.RestoreItemAction),\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tp := new(restartabletest.MockRestartableProcess)\n\t\t\tdefer p.AssertExpectations(t)\n\n\t\t\tname := \"pod\"\n\t\t\tkey := process.KindAndName{Kind: common.PluginKindRestoreItemActionV2, Name: name}\n\t\t\tp.On(\"GetByKindAndName\", key).Return(tc.plugin, tc.getError)\n\n\t\t\tr := NewRestartableRestoreItemAction(name, p)\n\t\t\ta, err := r.getRestoreItemAction()\n\t\t\tif tc.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, tc.expectedError)\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.Equal(t, tc.plugin, a)\n\t\t})\n\t}\n}\n\nfunc TestRestartableRestoreItemActionGetDelegate(t *testing.T) {\n\tp := new(restartabletest.MockRestartableProcess)\n\tdefer p.AssertExpectations(t)\n\n\t// Reset error\n\tp.On(\"ResetIfNeeded\").Return(errors.Errorf(\"reset error\")).Once()\n\tname := \"pod\"\n\tr := NewRestartableRestoreItemAction(name, p)\n\ta, err := r.getDelegate()\n\tassert.Nil(t, a)\n\trequire.EqualError(t, err, \"reset error\")\n\n\t// Happy path\n\tp.On(\"ResetIfNeeded\").Return(nil)\n\texpected := new(mocks.RestoreItemAction)\n\tkey := process.KindAndName{Kind: common.PluginKindRestoreItemActionV2, Name: name}\n\tp.On(\"GetByKindAndName\", key).Return(expected, nil)\n\n\ta, err = r.getDelegate()\n\trequire.NoError(t, err)\n\tassert.Equal(t, expected, a)\n}\n\nfunc TestRestartableRestoreItemActionDelegatedFunctions(t *testing.T) {\n\tpv := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"color\": \"blue\",\n\t\t},\n\t}\n\n\tinput := &velero.RestoreItemActionExecuteInput{\n\t\tItem: pv,\n\t\tItemFromBackup: pv,\n\t\tRestore: new(v1.Restore),\n\t}\n\n\toutput := &velero.RestoreItemActionExecuteOutput{\n\t\tUpdatedItem: &unstructured.Unstructured{\n\t\t\tObject: map[string]any{\n\t\t\t\t\"color\": \"green\",\n\t\t\t},\n\t\t},\n\t}\n\n\tr := new(v1.Restore)\n\toid := \"operation1\"\n\tadditionalItems := []velero.ResourceIdentifier{}\n\trestartabletest.RunRestartableDelegateTests(\n\t\tt,\n\t\tcommon.PluginKindRestoreItemActionV2,\n\t\tfunc(key process.KindAndName, p process.RestartableProcess) any {\n\t\t\treturn &RestartableRestoreItemAction{\n\t\t\t\tKey: key,\n\t\t\t\tSharedPluginProcess: p,\n\t\t\t}\n\t\t},\n\t\tfunc() restartabletest.Mockable {\n\t\t\treturn new(mocks.RestoreItemAction)\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"AppliesTo\",\n\t\t\tInputs: []any{},\n\t\t\tExpectedErrorOutputs: []any{velero.ResourceSelector{}, errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{velero.ResourceSelector{IncludedNamespaces: []string{\"a\"}}, errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"Execute\",\n\t\t\tInputs: []any{input},\n\t\t\tExpectedErrorOutputs: []any{nil, errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{output, errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"Progress\",\n\t\t\tInputs: []any{oid, r},\n\t\t\tExpectedErrorOutputs: []any{velero.OperationProgress{}, errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{velero.OperationProgress{}, errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"Cancel\",\n\t\t\tInputs: []any{oid, r},\n\t\t\tExpectedErrorOutputs: []any{errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"AreAdditionalItemsReady\",\n\t\t\tInputs: []any{additionalItems, r},\n\t\t\tExpectedErrorOutputs: []any{false, errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{true, errors.Errorf(\"delegate error\")},\n\t\t},\n\t)\n}\n" }, { "path": "pkg/plugin/clientmgmt/volumesnapshotter/v1/restartable_volume_snapshotter.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\t\"github.com/pkg/errors\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tvsv1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/volumesnapshotter/v1\"\n)\n\n// AdaptedVolumeSnapshotter is a volume snapshotter adapted to the v1 VolumeSnapshotter API\ntype AdaptedVolumeSnapshotter struct {\n\tKind common.PluginKind\n\n\t// Get returns a restartable VolumeSnapshotter for the given name and process, wrapping if necessary\n\tGetRestartable func(name string, restartableProcess process.RestartableProcess) vsv1.VolumeSnapshotter\n}\n\nfunc AdaptedVolumeSnapshotters() []AdaptedVolumeSnapshotter {\n\treturn []AdaptedVolumeSnapshotter{\n\t\t{\n\t\t\tKind: common.PluginKindVolumeSnapshotter,\n\t\t\tGetRestartable: func(name string, restartableProcess process.RestartableProcess) vsv1.VolumeSnapshotter {\n\t\t\t\treturn NewRestartableVolumeSnapshotter(name, restartableProcess)\n\t\t\t},\n\t\t},\n\t}\n}\n\n// RestartableVolumeSnapshotter is a volume snapshotter for a given implementation (such as \"aws\"). It is associated with\n// a restartableProcess, which may be shared and used to run multiple plugins. At the beginning of each method\n// call, the restartableVolumeSnapshotter asks its restartableProcess to restart itself if needed (e.g. if the\n// process terminated for any reason), then it proceeds with the actual call.\ntype RestartableVolumeSnapshotter struct {\n\tKey process.KindAndName\n\tSharedPluginProcess process.RestartableProcess\n\tconfig map[string]string\n}\n\n// NewRestartableVolumeSnapshotter returns a new restartableVolumeSnapshotter.\nfunc NewRestartableVolumeSnapshotter(name string, sharedPluginProcess process.RestartableProcess) *RestartableVolumeSnapshotter {\n\tkey := process.KindAndName{Kind: common.PluginKindVolumeSnapshotter, Name: name}\n\tr := &RestartableVolumeSnapshotter{\n\t\tKey: key,\n\t\tSharedPluginProcess: sharedPluginProcess,\n\t}\n\n\t// Register our reinitializer so we can reinitialize after a restart with r.config.\n\tsharedPluginProcess.AddReinitializer(key, r)\n\n\treturn r\n}\n\n// reinitialize reinitializes a re-dispensed plugin using the initial data passed to Init().\nfunc (r *RestartableVolumeSnapshotter) Reinitialize(dispensed any) error {\n\tvolumeSnapshotter, ok := dispensed.(vsv1.VolumeSnapshotter)\n\tif !ok {\n\t\treturn errors.Errorf(\"plugin %T is not a VolumeSnapshotter\", dispensed)\n\t}\n\treturn r.init(volumeSnapshotter, r.config)\n}\n\n// getVolumeSnapshotter returns the volume snapshotter for this restartableVolumeSnapshotter. It does *not* restart the\n// plugin process.\nfunc (r *RestartableVolumeSnapshotter) getVolumeSnapshotter() (vsv1.VolumeSnapshotter, error) {\n\tplugin, err := r.SharedPluginProcess.GetByKindAndName(r.Key)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tvolumeSnapshotter, ok := plugin.(vsv1.VolumeSnapshotter)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"plugin %T is not a VolumeSnapshotter\", plugin)\n\t}\n\n\treturn volumeSnapshotter, nil\n}\n\n// getDelegate restarts the plugin process (if needed) and returns the volume snapshotter for this RestartableVolumeSnapshotter.\nfunc (r *RestartableVolumeSnapshotter) getDelegate() (vsv1.VolumeSnapshotter, error) {\n\tif err := r.SharedPluginProcess.ResetIfNeeded(); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn r.getVolumeSnapshotter()\n}\n\n// Init initializes the volume snapshotter instance using config. If this is the first invocation, r stores config for future\n// reinitialization needs. Init does NOT restart the shared plugin process. Init may only be called once.\nfunc (r *RestartableVolumeSnapshotter) Init(config map[string]string) error {\n\tif r.config != nil {\n\t\treturn errors.Errorf(\"already initialized\")\n\t}\n\n\t// Not using getDelegate() to avoid possible infinite recursion\n\tdelegate, err := r.getVolumeSnapshotter()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tr.config = config\n\n\treturn r.init(delegate, config)\n}\n\n// init calls Init on volumeSnapshotter with config. This is split out from Init() so that both Init() and reinitialize() may\n// call it using a specific VolumeSnapshotter.\nfunc (r *RestartableVolumeSnapshotter) init(volumeSnapshotter vsv1.VolumeSnapshotter, config map[string]string) error {\n\treturn volumeSnapshotter.Init(config)\n}\n\n// CreateVolumeFromSnapshot restarts the plugin's process if needed, then delegates the call.\nfunc (r *RestartableVolumeSnapshotter) CreateVolumeFromSnapshot(snapshotID string, volumeType string, volumeAZ string, iops *int64) (volumeID string, err error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\treturn delegate.CreateVolumeFromSnapshot(snapshotID, volumeType, volumeAZ, iops)\n}\n\n// GetVolumeID restarts the plugin's process if needed, then delegates the call.\nfunc (r *RestartableVolumeSnapshotter) GetVolumeID(pv runtime.Unstructured) (string, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\treturn delegate.GetVolumeID(pv)\n}\n\n// SetVolumeID restarts the plugin's process if needed, then delegates the call.\nfunc (r *RestartableVolumeSnapshotter) SetVolumeID(pv runtime.Unstructured, volumeID string) (runtime.Unstructured, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn delegate.SetVolumeID(pv, volumeID)\n}\n\n// GetVolumeInfo restarts the plugin's process if needed, then delegates the call.\nfunc (r *RestartableVolumeSnapshotter) GetVolumeInfo(volumeID string, volumeAZ string) (string, *int64, error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn \"\", nil, err\n\t}\n\treturn delegate.GetVolumeInfo(volumeID, volumeAZ)\n}\n\n// CreateSnapshot restarts the plugin's process if needed, then delegates the call.\nfunc (r *RestartableVolumeSnapshotter) CreateSnapshot(volumeID string, volumeAZ string, tags map[string]string) (snapshotID string, err error) {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\treturn delegate.CreateSnapshot(volumeID, volumeAZ, tags)\n}\n\n// DeleteSnapshot restarts the plugin's process if needed, then delegates the call.\nfunc (r *RestartableVolumeSnapshotter) DeleteSnapshot(snapshotID string) error {\n\tdelegate, err := r.getDelegate()\n\tif err != nil {\n\t\treturn err\n\t}\n\treturn delegate.DeleteSnapshot(snapshotID)\n}\n" }, { "path": "pkg/plugin/clientmgmt/volumesnapshotter/v1/restartable_volume_snapshotter_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\t\"testing\"\n\n\t\"github.com/Azure/azure-sdk-for-go/sdk/azcore/to\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\n\t\"github.com/vmware-tanzu/velero/internal/restartabletest\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tprovidermocks \"github.com/vmware-tanzu/velero/pkg/plugin/velero/mocks/volumesnapshotter/v1\"\n)\n\nfunc TestRestartableGetVolumeSnapshotter(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tplugin any\n\t\tgetError error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"error getting by kind and name\",\n\t\t\tgetError: errors.Errorf(\"get error\"),\n\t\t\texpectedError: \"get error\",\n\t\t},\n\t\t{\n\t\t\tname: \"wrong type\",\n\t\t\tplugin: 3,\n\t\t\texpectedError: \"plugin int is not a VolumeSnapshotter\",\n\t\t},\n\t\t{\n\t\t\tname: \"happy path\",\n\t\t\tplugin: new(providermocks.VolumeSnapshotter),\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tp := new(restartabletest.MockRestartableProcess)\n\t\t\tp.Test(t)\n\t\t\tdefer p.AssertExpectations(t)\n\n\t\t\tname := \"aws\"\n\t\t\tkey := process.KindAndName{Kind: common.PluginKindVolumeSnapshotter, Name: name}\n\t\t\tp.On(\"GetByKindAndName\", key).Return(tc.plugin, tc.getError)\n\n\t\t\tr := &RestartableVolumeSnapshotter{\n\t\t\t\tKey: key,\n\t\t\t\tSharedPluginProcess: p,\n\t\t\t}\n\t\t\ta, err := r.getVolumeSnapshotter()\n\t\t\tif tc.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, tc.expectedError)\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.Equal(t, tc.plugin, a)\n\t\t})\n\t}\n}\n\nfunc TestRestartableVolumeSnapshotterReinitialize(t *testing.T) {\n\tp := new(restartabletest.MockRestartableProcess)\n\tp.Test(t)\n\tdefer p.AssertExpectations(t)\n\n\tname := \"aws\"\n\tkey := process.KindAndName{Kind: common.PluginKindVolumeSnapshotter, Name: name}\n\tr := &RestartableVolumeSnapshotter{\n\t\tKey: key,\n\t\tSharedPluginProcess: p,\n\t\tconfig: map[string]string{\n\t\t\t\"color\": \"blue\",\n\t\t},\n\t}\n\n\terr := r.Reinitialize(3)\n\trequire.EqualError(t, err, \"plugin int is not a VolumeSnapshotter\")\n\n\tvolumeSnapshotter := new(providermocks.VolumeSnapshotter)\n\tvolumeSnapshotter.Test(t)\n\tdefer volumeSnapshotter.AssertExpectations(t)\n\n\tvolumeSnapshotter.On(\"Init\", r.config).Return(errors.Errorf(\"init error\")).Once()\n\terr = r.Reinitialize(volumeSnapshotter)\n\trequire.EqualError(t, err, \"init error\")\n\n\tvolumeSnapshotter.On(\"Init\", r.config).Return(nil)\n\terr = r.Reinitialize(volumeSnapshotter)\n\tassert.NoError(t, err)\n}\n\nfunc TestRestartableVolumeSnapshotterGetDelegate(t *testing.T) {\n\tp := new(restartabletest.MockRestartableProcess)\n\tp.Test(t)\n\tdefer p.AssertExpectations(t)\n\n\t// Reset error\n\tp.On(\"ResetIfNeeded\").Return(errors.Errorf(\"reset error\")).Once()\n\tname := \"aws\"\n\tkey := process.KindAndName{Kind: common.PluginKindVolumeSnapshotter, Name: name}\n\tr := &RestartableVolumeSnapshotter{\n\t\tKey: key,\n\t\tSharedPluginProcess: p,\n\t}\n\ta, err := r.getDelegate()\n\tassert.Nil(t, a)\n\trequire.EqualError(t, err, \"reset error\")\n\n\t// Happy path\n\tp.On(\"ResetIfNeeded\").Return(nil)\n\tvolumeSnapshotter := new(providermocks.VolumeSnapshotter)\n\tvolumeSnapshotter.Test(t)\n\tdefer volumeSnapshotter.AssertExpectations(t)\n\tp.On(\"GetByKindAndName\", key).Return(volumeSnapshotter, nil)\n\n\ta, err = r.getDelegate()\n\trequire.NoError(t, err)\n\tassert.Equal(t, volumeSnapshotter, a)\n}\n\nfunc TestRestartableVolumeSnapshotterInit(t *testing.T) {\n\tp := new(restartabletest.MockRestartableProcess)\n\tp.Test(t)\n\tdefer p.AssertExpectations(t)\n\n\t// getVolumeSnapshottererror\n\tname := \"aws\"\n\tkey := process.KindAndName{Kind: common.PluginKindVolumeSnapshotter, Name: name}\n\tr := &RestartableVolumeSnapshotter{\n\t\tKey: key,\n\t\tSharedPluginProcess: p,\n\t}\n\tp.On(\"GetByKindAndName\", key).Return(nil, errors.Errorf(\"GetByKindAndName error\")).Once()\n\n\tconfig := map[string]string{\n\t\t\"color\": \"blue\",\n\t}\n\terr := r.Init(config)\n\trequire.EqualError(t, err, \"GetByKindAndName error\")\n\n\t// Delegate returns error\n\tvolumeSnapshotter := new(providermocks.VolumeSnapshotter)\n\tvolumeSnapshotter.Test(t)\n\tdefer volumeSnapshotter.AssertExpectations(t)\n\tp.On(\"GetByKindAndName\", key).Return(volumeSnapshotter, nil)\n\tvolumeSnapshotter.On(\"Init\", config).Return(errors.Errorf(\"Init error\")).Once()\n\n\terr = r.Init(config)\n\trequire.EqualError(t, err, \"Init error\")\n\n\t// wipe this out because the previous failed Init call set it\n\tr.config = nil\n\n\t// Happy path\n\tvolumeSnapshotter.On(\"Init\", config).Return(nil)\n\terr = r.Init(config)\n\trequire.NoError(t, err)\n\tassert.Equal(t, config, r.config)\n\n\t// Calling Init twice is forbidden\n\terr = r.Init(config)\n\tassert.EqualError(t, err, \"already initialized\")\n}\n\nfunc TestRestartableVolumeSnapshotterDelegatedFunctions(t *testing.T) {\n\tpv := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"color\": \"blue\",\n\t\t},\n\t}\n\n\tpvToReturn := &unstructured.Unstructured{\n\t\tObject: map[string]any{\n\t\t\t\"color\": \"green\",\n\t\t},\n\t}\n\n\trestartabletest.RunRestartableDelegateTests(\n\t\tt,\n\t\tcommon.PluginKindVolumeSnapshotter,\n\t\tfunc(key process.KindAndName, p process.RestartableProcess) any {\n\t\t\treturn &RestartableVolumeSnapshotter{\n\t\t\t\tKey: key,\n\t\t\t\tSharedPluginProcess: p,\n\t\t\t}\n\t\t},\n\t\tfunc() restartabletest.Mockable {\n\t\t\treturn new(providermocks.VolumeSnapshotter)\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"CreateVolumeFromSnapshot\",\n\t\t\tInputs: []any{\"snapshotID\", \"volumeID\", \"volumeAZ\", to.Ptr(int64(10000))},\n\t\t\tExpectedErrorOutputs: []any{\"\", errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{\"volumeID\", errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"GetVolumeID\",\n\t\t\tInputs: []any{pv},\n\t\t\tExpectedErrorOutputs: []any{\"\", errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{\"volumeID\", errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"SetVolumeID\",\n\t\t\tInputs: []any{pv, \"volumeID\"},\n\t\t\tExpectedErrorOutputs: []any{nil, errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{pvToReturn, errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"GetVolumeInfo\",\n\t\t\tInputs: []any{\"volumeID\", \"volumeAZ\"},\n\t\t\tExpectedErrorOutputs: []any{\"\", (*int64)(nil), errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{\"volumeType\", to.Ptr(int64(10000)), errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"CreateSnapshot\",\n\t\t\tInputs: []any{\"volumeID\", \"volumeAZ\", map[string]string{\"a\": \"b\"}},\n\t\t\tExpectedErrorOutputs: []any{\"\", errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{\"snapshotID\", errors.Errorf(\"delegate error\")},\n\t\t},\n\t\trestartabletest.RestartableDelegateTest{\n\t\t\tFunction: \"DeleteSnapshot\",\n\t\t\tInputs: []any{\"snapshotID\"},\n\t\t\tExpectedErrorOutputs: []any{errors.Errorf(\"reset error\")},\n\t\t\tExpectedDelegateOutputs: []any{errors.Errorf(\"delegate error\")},\n\t\t},\n\t)\n}\n" }, { "path": "pkg/plugin/framework/action_resolver.go", "content": "/*\nCopyright the Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/discovery\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tbiav1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/backupitemaction/v1\"\n\tbiav2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/backupitemaction/v2\"\n\tibav1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/itemblockaction/v1\"\n\triav1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/restoreitemaction/v1\"\n\triav2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/restoreitemaction/v2\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/collections\"\n)\n\n/*\nVelero has a variety of Actions that can be executed on Kubernetes resources. The Actions (BackupItemAction, RestoreItemAction\nand others) implement the Applicable interface which returns a ResourceSelector for the Action. The ResourceSelector\ncan specify namespaces, resource names and labels to include or exclude. The ResourceSelector is resolved into lists\nof namespaces and resources present in the backup to be matched against. These lists and the label selector are then used to\ndecide whether or not the ResolvedAction should be used for a particular resource.\n*/\n\n// ResolvedAction is an action that has had the namespaces, resources names and labels to include or exclude resolved\ntype ResolvedAction interface {\n\t// ShouldUse returns true if the resolved namespaces, resource names and labels match those passed in the parameters.\n\t// metadata is optional and may be nil\n\tShouldUse(groupResource schema.GroupResource, namespace string, metadata metav1.Object,\n\t\tlog logrus.FieldLogger) bool\n}\n\n// resolvedAction is a core struct that holds the resolved namespaces, resource names and labels\ntype resolvedAction struct {\n\tResourceIncludesExcludes *collections.IncludesExcludes\n\tNamespaceIncludesExcludes *collections.IncludesExcludes\n\tSelector labels.Selector\n}\n\nfunc (recv resolvedAction) ShouldUse(groupResource schema.GroupResource, namespace string, metadata metav1.Object,\n\tlog logrus.FieldLogger) bool {\n\tif !recv.ResourceIncludesExcludes.ShouldInclude(groupResource.String()) {\n\t\tlog.Debug(\"Skipping action because it does not apply to this resource\")\n\t\treturn false\n\t}\n\n\tif namespace != \"\" && !recv.NamespaceIncludesExcludes.ShouldInclude(namespace) {\n\t\tlog.Debug(\"Skipping action because it does not apply to this namespace\")\n\t\treturn false\n\t}\n\n\tif namespace == \"\" && !recv.NamespaceIncludesExcludes.IncludeEverything() {\n\t\tlog.Debug(\"Skipping action because resource is cluster-scoped and action only applies to specific namespaces\")\n\t\treturn false\n\t}\n\n\tif metadata != nil && !recv.Selector.Matches(labels.Set(metadata.GetLabels())) {\n\t\tlog.Debug(\"Skipping action because label selector does not match\")\n\t\treturn false\n\t}\n\treturn true\n}\n\n// resolveAction resolves the resources, namespaces and selector into fully-qualified versions\nfunc resolveAction(helper discovery.Helper, action velero.Applicable) (resources *collections.IncludesExcludes,\n\tnamespaces *collections.IncludesExcludes, selector labels.Selector, err error) {\n\tresourceSelector, err := action.AppliesTo()\n\tif err != nil {\n\t\treturn nil, nil, nil, err\n\t}\n\n\tresources = collections.GetResourceIncludesExcludes(helper, resourceSelector.IncludedResources, resourceSelector.ExcludedResources)\n\tnamespaces = collections.NewIncludesExcludes().Includes(resourceSelector.IncludedNamespaces...).Excludes(resourceSelector.ExcludedNamespaces...)\n\n\tselector = labels.Everything()\n\tif resourceSelector.LabelSelector != \"\" {\n\t\tif selector, err = labels.Parse(resourceSelector.LabelSelector); err != nil {\n\t\t\treturn nil, nil, nil, err\n\t\t}\n\t}\n\n\treturn\n}\n\ntype BackupItemResolvedAction struct {\n\tbiav1.BackupItemAction\n\tresolvedAction\n}\n\nfunc NewBackupItemActionResolver(actions []biav1.BackupItemAction) BackupItemActionResolver {\n\treturn BackupItemActionResolver{\n\t\tactions: actions,\n\t}\n}\n\ntype BackupItemResolvedActionV2 struct {\n\tbiav2.BackupItemAction\n\tresolvedAction\n}\n\nfunc NewBackupItemActionResolverV2(actions []biav2.BackupItemAction) BackupItemActionResolverV2 {\n\treturn BackupItemActionResolverV2{\n\t\tactions: actions,\n\t}\n}\n\nfunc NewRestoreItemActionResolver(actions []riav1.RestoreItemAction) RestoreItemActionResolver {\n\treturn RestoreItemActionResolver{\n\t\tactions: actions,\n\t}\n}\n\nfunc NewRestoreItemActionResolverV2(actions []riav2.RestoreItemAction) RestoreItemActionResolverV2 {\n\treturn RestoreItemActionResolverV2{\n\t\tactions: actions,\n\t}\n}\n\nfunc NewDeleteItemActionResolver(actions []velero.DeleteItemAction) DeleteItemActionResolver {\n\treturn DeleteItemActionResolver{\n\t\tactions: actions,\n\t}\n}\n\ntype ActionResolver interface {\n\tResolveAction(helper discovery.Helper, action velero.Applicable, log logrus.FieldLogger) (ResolvedAction, error)\n}\n\ntype BackupItemActionResolver struct {\n\tactions []biav1.BackupItemAction\n}\n\nfunc (recv BackupItemActionResolver) ResolveActions(helper discovery.Helper, log logrus.FieldLogger) ([]BackupItemResolvedAction, error) {\n\tvar resolved []BackupItemResolvedAction\n\tfor _, action := range recv.actions {\n\t\tresources, namespaces, selector, err := resolveAction(helper, action)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tres := BackupItemResolvedAction{\n\t\t\tBackupItemAction: action,\n\t\t\tresolvedAction: resolvedAction{\n\t\t\t\tResourceIncludesExcludes: resources,\n\t\t\t\tNamespaceIncludesExcludes: namespaces,\n\t\t\t\tSelector: selector,\n\t\t\t},\n\t\t}\n\t\tresolved = append(resolved, res)\n\t}\n\treturn resolved, nil\n}\n\ntype BackupItemActionResolverV2 struct {\n\tactions []biav2.BackupItemAction\n}\n\nfunc (recv BackupItemActionResolverV2) ResolveActions(helper discovery.Helper, log logrus.FieldLogger) ([]BackupItemResolvedActionV2, error) {\n\tvar resolved []BackupItemResolvedActionV2\n\tfor _, action := range recv.actions {\n\t\tlog.Debugf(\"resolving BackupItemAction for: %v\", action)\n\t\tresources, namespaces, selector, err := resolveAction(helper, action)\n\t\tif err != nil {\n\t\t\tlog.WithError(errors.WithStack(err)).Debugf(\"resolveAction error, action: %v\", action)\n\t\t\treturn nil, err\n\t\t}\n\t\tres := BackupItemResolvedActionV2{\n\t\t\tBackupItemAction: action,\n\t\t\tresolvedAction: resolvedAction{\n\t\t\t\tResourceIncludesExcludes: resources,\n\t\t\t\tNamespaceIncludesExcludes: namespaces,\n\t\t\t\tSelector: selector,\n\t\t\t},\n\t\t}\n\t\tresolved = append(resolved, res)\n\t}\n\treturn resolved, nil\n}\n\ntype RestoreItemResolvedAction struct {\n\triav1.RestoreItemAction\n\tresolvedAction\n}\n\ntype RestoreItemResolvedActionV2 struct {\n\triav2.RestoreItemAction\n\tresolvedAction\n}\n\ntype RestoreItemActionResolver struct {\n\tactions []riav1.RestoreItemAction\n}\n\nfunc (recv RestoreItemActionResolver) ResolveActions(helper discovery.Helper, log logrus.FieldLogger) ([]RestoreItemResolvedAction, error) {\n\tvar resolved []RestoreItemResolvedAction\n\tfor _, action := range recv.actions {\n\t\tresources, namespaces, selector, err := resolveAction(helper, action)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tres := RestoreItemResolvedAction{\n\t\t\tRestoreItemAction: action,\n\t\t\tresolvedAction: resolvedAction{\n\t\t\t\tResourceIncludesExcludes: resources,\n\t\t\t\tNamespaceIncludesExcludes: namespaces,\n\t\t\t\tSelector: selector,\n\t\t\t},\n\t\t}\n\t\tresolved = append(resolved, res)\n\t}\n\treturn resolved, nil\n}\n\ntype RestoreItemActionResolverV2 struct {\n\tactions []riav2.RestoreItemAction\n}\n\nfunc (recv RestoreItemActionResolverV2) ResolveActions(helper discovery.Helper, log logrus.FieldLogger) ([]RestoreItemResolvedActionV2, error) {\n\tvar resolved []RestoreItemResolvedActionV2\n\tfor _, action := range recv.actions {\n\t\tresources, namespaces, selector, err := resolveAction(helper, action)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tres := RestoreItemResolvedActionV2{\n\t\t\tRestoreItemAction: action,\n\t\t\tresolvedAction: resolvedAction{\n\t\t\t\tResourceIncludesExcludes: resources,\n\t\t\t\tNamespaceIncludesExcludes: namespaces,\n\t\t\t\tSelector: selector,\n\t\t\t},\n\t\t}\n\t\tresolved = append(resolved, res)\n\t}\n\treturn resolved, nil\n}\n\ntype DeleteItemResolvedAction struct {\n\tvelero.DeleteItemAction\n\tresolvedAction\n}\n\ntype DeleteItemActionResolver struct {\n\tactions []velero.DeleteItemAction\n}\n\nfunc (recv DeleteItemActionResolver) ResolveActions(helper discovery.Helper, log logrus.FieldLogger) ([]DeleteItemResolvedAction, error) {\n\tvar resolved []DeleteItemResolvedAction\n\tfor _, action := range recv.actions {\n\t\tresources, namespaces, selector, err := resolveAction(helper, action)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tres := DeleteItemResolvedAction{\n\t\t\tDeleteItemAction: action,\n\t\t\tresolvedAction: resolvedAction{\n\t\t\t\tResourceIncludesExcludes: resources,\n\t\t\t\tNamespaceIncludesExcludes: namespaces,\n\t\t\t\tSelector: selector,\n\t\t\t},\n\t\t}\n\t\tresolved = append(resolved, res)\n\t}\n\treturn resolved, nil\n}\n\ntype ItemBlockResolvedAction struct {\n\tibav1.ItemBlockAction\n\tresolvedAction\n}\n\ntype ItemBlockActionResolver struct {\n\tactions []ibav1.ItemBlockAction\n}\n\nfunc NewItemBlockActionResolver(actions []ibav1.ItemBlockAction) ItemBlockActionResolver {\n\treturn ItemBlockActionResolver{\n\t\tactions: actions,\n\t}\n}\n\nfunc (recv ItemBlockActionResolver) ResolveActions(helper discovery.Helper, log logrus.FieldLogger) ([]ItemBlockResolvedAction, error) {\n\tvar resolved []ItemBlockResolvedAction\n\tfor _, action := range recv.actions {\n\t\tresources, namespaces, selector, err := resolveAction(helper, action)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tres := ItemBlockResolvedAction{\n\t\t\tItemBlockAction: action,\n\t\t\tresolvedAction: resolvedAction{\n\t\t\t\tResourceIncludesExcludes: resources,\n\t\t\t\tNamespaceIncludesExcludes: namespaces,\n\t\t\t\tSelector: selector,\n\t\t\t},\n\t\t}\n\t\tresolved = append(resolved, res)\n\t}\n\treturn resolved, nil\n}\n" }, { "path": "pkg/plugin/framework/action_resolver_test.go", "content": "/*\nCopyright the Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"testing\"\n\n\t\"k8s.io/apimachinery/pkg/labels\"\n\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\ntype mockApplicable struct {\n\tselector velero.ResourceSelector\n}\n\nfunc (recv mockApplicable) AppliesTo() (velero.ResourceSelector, error) {\n\treturn recv.selector, nil\n}\n\nfunc TestActionResolverNamespace(t *testing.T) {\n\tdiscoveryHelper := velerotest.NewFakeDiscoveryHelper(false, map[schema.GroupVersionResource]schema.GroupVersionResource{})\n\tnamespaceMatchApplicable := mockApplicable{\n\t\tselector: velero.ResourceSelector{\n\t\t\tIncludedNamespaces: []string{\"default\"},\n\t\t},\n\t}\n\tresources, namespaces, selector, err := resolveAction(discoveryHelper, namespaceMatchApplicable)\n\trequire.NoError(t, err)\n\trequire.Equal(t, []string{\"default\"}, namespaces.GetIncludes())\n\trequire.Empty(t, namespaces.GetExcludes())\n\trequire.Empty(t, resources.GetIncludes())\n\trequire.Empty(t, resources.GetExcludes())\n\trequire.True(t, selector.Empty())\n}\n\nfunc TestActionResolverResource(t *testing.T) {\n\tpvGVR := schema.GroupVersionResource{\n\t\tGroup: \"\",\n\t\tVersion: \"v1\",\n\t\tResource: \"persistentvolumes\",\n\t}\n\tdiscoveryHelper := velerotest.NewFakeDiscoveryHelper(false, map[schema.GroupVersionResource]schema.GroupVersionResource{pvGVR: pvGVR})\n\tnamespaceMatchApplicable := mockApplicable{\n\t\tselector: velero.ResourceSelector{\n\t\t\tIncludedResources: []string{\"persistentvolumes\"},\n\t\t},\n\t}\n\tresources, namespaces, selector, err := resolveAction(discoveryHelper, namespaceMatchApplicable)\n\trequire.NoError(t, err)\n\trequire.Empty(t, namespaces.GetIncludes())\n\trequire.Empty(t, namespaces.GetExcludes())\n\trequire.True(t, resources.ShouldInclude(\"persistentvolumes\"))\n\trequire.Empty(t, resources.GetExcludes())\n\trequire.True(t, selector.Empty())\n}\n\nfunc TestActionResolverLabel(t *testing.T) {\n\tdiscoveryHelper := velerotest.NewFakeDiscoveryHelper(false, map[schema.GroupVersionResource]schema.GroupVersionResource{})\n\tnamespaceMatchApplicable := mockApplicable{\n\t\tselector: velero.ResourceSelector{\n\t\t\tLabelSelector: \"myLabel=true\",\n\t\t},\n\t}\n\tcheckLabel, err := labels.ConvertSelectorToLabelsMap(\"myLabel=true\")\n\trequire.NoError(t, err)\n\n\tresources, namespaces, selector, err := resolveAction(discoveryHelper, namespaceMatchApplicable)\n\trequire.NoError(t, err)\n\trequire.Empty(t, namespaces.GetIncludes())\n\trequire.Empty(t, namespaces.GetExcludes())\n\trequire.Empty(t, resources.GetIncludes())\n\trequire.Empty(t, resources.GetExcludes())\n\trequire.True(t, selector.Matches(checkLabel))\n}\n" }, { "path": "pkg/plugin/framework/backup_item_action.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"context\"\n\n\tplugin \"github.com/hashicorp/go-plugin\"\n\t\"google.golang.org/grpc\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tprotobiav1 \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n)\n\n// BackupItemActionPlugin is an implementation of go-plugin's Plugin\n// interface with support for gRPC for the backup/ItemAction\n// interface.\ntype BackupItemActionPlugin struct {\n\tplugin.NetRPCUnsupportedPlugin\n\t*common.PluginBase\n}\n\n// GRPCClient returns a clientDispenser for BackupItemAction gRPC clients.\nfunc (p *BackupItemActionPlugin) GRPCClient(_ context.Context, _ *plugin.GRPCBroker, clientConn *grpc.ClientConn) (any, error) {\n\treturn common.NewClientDispenser(p.ClientLogger, clientConn, newBackupItemActionGRPCClient), nil\n}\n\n// GRPCServer registers a BackupItemAction gRPC server.\nfunc (p *BackupItemActionPlugin) GRPCServer(_ *plugin.GRPCBroker, server *grpc.Server) error {\n\tprotobiav1.RegisterBackupItemActionServer(server, &BackupItemActionGRPCServer{mux: p.ServerMux})\n\treturn nil\n}\n" }, { "path": "pkg/plugin/framework/backup_item_action_client.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"encoding/json\"\n\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"google.golang.org/grpc\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tprotobiav1 \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// NewBackupItemActionPlugin constructs a BackupItemActionPlugin.\nfunc NewBackupItemActionPlugin(options ...common.PluginOption) *BackupItemActionPlugin {\n\treturn &BackupItemActionPlugin{\n\t\tPluginBase: common.NewPluginBase(options...),\n\t}\n}\n\n// BackupItemActionGRPCClient implements the backup/ItemAction interface and uses a\n// gRPC client to make calls to the plugin server.\ntype BackupItemActionGRPCClient struct {\n\t*common.ClientBase\n\tgrpcClient protobiav1.BackupItemActionClient\n}\n\nfunc newBackupItemActionGRPCClient(base *common.ClientBase, clientConn *grpc.ClientConn) any {\n\treturn &BackupItemActionGRPCClient{\n\t\tClientBase: base,\n\t\tgrpcClient: protobiav1.NewBackupItemActionClient(clientConn),\n\t}\n}\n\nfunc (c *BackupItemActionGRPCClient) AppliesTo() (velero.ResourceSelector, error) {\n\treq := &protobiav1.BackupItemActionAppliesToRequest{\n\t\tPlugin: c.Plugin,\n\t}\n\n\tres, err := c.grpcClient.AppliesTo(context.Background(), req)\n\tif err != nil {\n\t\treturn velero.ResourceSelector{}, common.FromGRPCError(err)\n\t}\n\n\tif res.ResourceSelector == nil {\n\t\treturn velero.ResourceSelector{}, nil\n\t}\n\n\treturn velero.ResourceSelector{\n\t\tIncludedNamespaces: res.ResourceSelector.IncludedNamespaces,\n\t\tExcludedNamespaces: res.ResourceSelector.ExcludedNamespaces,\n\t\tIncludedResources: res.ResourceSelector.IncludedResources,\n\t\tExcludedResources: res.ResourceSelector.ExcludedResources,\n\t\tLabelSelector: res.ResourceSelector.Selector,\n\t}, nil\n}\n\nfunc (c *BackupItemActionGRPCClient) Execute(item runtime.Unstructured, backup *api.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, error) {\n\titemJSON, err := json.Marshal(item.UnstructuredContent())\n\tif err != nil {\n\t\treturn nil, nil, errors.WithStack(err)\n\t}\n\n\tbackupJSON, err := json.Marshal(backup)\n\tif err != nil {\n\t\treturn nil, nil, errors.WithStack(err)\n\t}\n\n\treq := &protobiav1.ExecuteRequest{\n\t\tPlugin: c.Plugin,\n\t\tItem: itemJSON,\n\t\tBackup: backupJSON,\n\t}\n\n\tres, err := c.grpcClient.Execute(context.Background(), req)\n\tif err != nil {\n\t\treturn nil, nil, common.FromGRPCError(err)\n\t}\n\n\tvar updatedItem unstructured.Unstructured\n\tif err := json.Unmarshal(res.Item, &updatedItem); err != nil {\n\t\treturn nil, nil, errors.WithStack(err)\n\t}\n\n\tvar additionalItems []velero.ResourceIdentifier\n\n\tfor _, itm := range res.AdditionalItems {\n\t\tnewItem := velero.ResourceIdentifier{\n\t\t\tGroupResource: schema.GroupResource{\n\t\t\t\tGroup: itm.Group,\n\t\t\t\tResource: itm.Resource,\n\t\t\t},\n\t\t\tNamespace: itm.Namespace,\n\t\t\tName: itm.Name,\n\t\t}\n\n\t\tadditionalItems = append(additionalItems, newItem)\n\t}\n\n\treturn &updatedItem, additionalItems, nil\n}\n" }, { "path": "pkg/plugin/framework/backup_item_action_server.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"encoding/json\"\n\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tbiav1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/backupitemaction/v1\"\n)\n\n// BackupItemActionGRPCServer implements the proto-generated BackupItemAction interface, and accepts\n// gRPC calls and forwards them to an implementation of the pluggable interface.\ntype BackupItemActionGRPCServer struct {\n\tmux *common.ServerMux\n}\n\nfunc (s *BackupItemActionGRPCServer) getImpl(name string) (biav1.BackupItemAction, error) {\n\timpl, err := s.mux.GetHandler(name)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\titemAction, ok := impl.(biav1.BackupItemAction)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"%T is not a backup item action\", impl)\n\t}\n\n\treturn itemAction, nil\n}\n\nfunc (s *BackupItemActionGRPCServer) AppliesTo(\n\tctx context.Context, req *proto.BackupItemActionAppliesToRequest) (\n\tresponse *proto.BackupItemActionAppliesToResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tresourceSelector, err := impl.AppliesTo()\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &proto.BackupItemActionAppliesToResponse{\n\t\tResourceSelector: &proto.ResourceSelector{\n\t\t\tIncludedNamespaces: resourceSelector.IncludedNamespaces,\n\t\t\tExcludedNamespaces: resourceSelector.ExcludedNamespaces,\n\t\t\tIncludedResources: resourceSelector.IncludedResources,\n\t\t\tExcludedResources: resourceSelector.ExcludedResources,\n\t\t\tSelector: resourceSelector.LabelSelector,\n\t\t},\n\t}, nil\n}\n\nfunc (s *BackupItemActionGRPCServer) Execute(\n\tctx context.Context, req *proto.ExecuteRequest) (response *proto.ExecuteResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tvar item unstructured.Unstructured\n\tvar backup api.Backup\n\n\tif err := json.Unmarshal(req.Item, &item); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\tif err := json.Unmarshal(req.Backup, &backup); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\n\tupdatedItem, additionalItems, err := impl.Execute(&item, &backup)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\t// If the plugin implementation returned a nil updatedItem (meaning no modifications), reset updatedItem to the\n\t// original item.\n\tvar updatedItemJSON []byte\n\tif updatedItem == nil {\n\t\tupdatedItemJSON = req.Item\n\t} else {\n\t\tupdatedItemJSON, err = json.Marshal(updatedItem.UnstructuredContent())\n\t\tif err != nil {\n\t\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t\t}\n\t}\n\n\tres := &proto.ExecuteResponse{\n\t\tItem: updatedItemJSON,\n\t}\n\n\tfor _, item := range additionalItems {\n\t\tres.AdditionalItems = append(res.AdditionalItems, backupResourceIdentifierToProto(item))\n\t}\n\n\treturn res, nil\n}\n\nfunc backupResourceIdentifierToProto(id velero.ResourceIdentifier) *proto.ResourceIdentifier {\n\treturn &proto.ResourceIdentifier{\n\t\tGroup: id.Group,\n\t\tResource: id.Resource,\n\t\tNamespace: id.Namespace,\n\t\tName: id.Name,\n\t}\n}\n" }, { "path": "pkg/plugin/framework/backup_item_action_test.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"encoding/json\"\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tmocks \"github.com/vmware-tanzu/velero/pkg/plugin/velero/mocks/backupitemaction/v1\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestBackupItemActionGRPCServerExecute(t *testing.T) {\n\tinvalidItem := []byte(\"this is gibberish json\")\n\tvalidItem := []byte(`\n\t{\n\t\t\"apiVersion\": \"v1\",\n\t\t\"kind\": \"ConfigMap\",\n\t\t\"metadata\": {\n\t\t\t\"namespace\": \"myns\",\n\t\t\t\"name\": \"myconfigmap\"\n\t\t},\n\t\t\"data\": {\n\t\t\t\"key\": \"value\"\n\t\t}\n\t}`)\n\tvar validItemObject unstructured.Unstructured\n\terr := json.Unmarshal(validItem, &validItemObject)\n\trequire.NoError(t, err)\n\n\tupdatedItem := []byte(`\n\t\t{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"ConfigMap\",\n\t\t\t\"metadata\": {\n\t\t\t\t\"namespace\": \"myns\",\n\t\t\t\t\"name\": \"myconfigmap\"\n\t\t\t},\n\t\t\t\"data\": {\n\t\t\t\t\"key\": \"changed!\"\n\t\t\t}\n\t\t}`)\n\tvar updatedItemObject unstructured.Unstructured\n\terr = json.Unmarshal(updatedItem, &updatedItemObject)\n\trequire.NoError(t, err)\n\n\tinvalidBackup := []byte(\"this is gibberish json\")\n\tvalidBackup := []byte(`\n\t{\n\t\t\"apiVersion\": \"velero.io/v1\",\n\t\t\"kind\": \"Backup\",\n\t\t\"metadata\": {\n\t\t\t\"namespace\": \"myns\",\n\t\t\t\"name\": \"mybackup\"\n\t\t},\n\t\t\"spec\": {\n\t\t\t\"includedNamespaces\": [\"*\"],\n\t\t\t\"includedResources\": [\"*\"],\n\t\t\t\"ttl\": \"60m\"\n\t\t}\n\t}`)\n\tvar validBackupObject v1.Backup\n\terr = json.Unmarshal(validBackup, &validBackupObject)\n\trequire.NoError(t, err)\n\n\ttests := []struct {\n\t\tname string\n\t\tbackup []byte\n\t\titem []byte\n\t\timplUpdatedItem runtime.Unstructured\n\t\timplAdditionalItems []velero.ResourceIdentifier\n\t\timplError error\n\t\texpectError bool\n\t\tskipMock bool\n\t}{\n\t\t{\n\t\t\tname: \"error unmarshaling item\",\n\t\t\titem: invalidItem,\n\t\t\tbackup: validBackup,\n\t\t\texpectError: true,\n\t\t\tskipMock: true,\n\t\t},\n\t\t{\n\t\t\tname: \"error unmarshaling backup\",\n\t\t\titem: validItem,\n\t\t\tbackup: invalidBackup,\n\t\t\texpectError: true,\n\t\t\tskipMock: true,\n\t\t},\n\t\t{\n\t\t\tname: \"error running impl\",\n\t\t\titem: validItem,\n\t\t\tbackup: validBackup,\n\t\t\timplError: errors.New(\"impl error\"),\n\t\t\texpectError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"nil updatedItem / no additionalItems\",\n\t\t\titem: validItem,\n\t\t\tbackup: validBackup,\n\t\t},\n\t\t{\n\t\t\tname: \"same updatedItem / some additionalItems\",\n\t\t\titem: validItem,\n\t\t\tbackup: validBackup,\n\t\t\timplUpdatedItem: &validItemObject,\n\t\t\timplAdditionalItems: []velero.ResourceIdentifier{\n\t\t\t\t{\n\t\t\t\t\tGroupResource: schema.GroupResource{Group: \"v1\", Resource: \"pods\"},\n\t\t\t\t\tNamespace: \"myns\",\n\t\t\t\t\tName: \"mypod\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"different updatedItem\",\n\t\t\titem: validItem,\n\t\t\tbackup: validBackup,\n\t\t\timplUpdatedItem: &updatedItemObject,\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\titemAction := &mocks.BackupItemAction{}\n\t\t\tdefer itemAction.AssertExpectations(t)\n\n\t\t\tif !test.skipMock {\n\t\t\t\titemAction.On(\"Execute\", &validItemObject, &validBackupObject).Return(test.implUpdatedItem, test.implAdditionalItems, test.implError)\n\t\t\t}\n\n\t\t\ts := &BackupItemActionGRPCServer{mux: &common.ServerMux{\n\t\t\t\tServerLog: velerotest.NewLogger(),\n\t\t\t\tHandlers: map[string]any{\n\t\t\t\t\t\"xyz\": itemAction,\n\t\t\t\t},\n\t\t\t}}\n\n\t\t\treq := &proto.ExecuteRequest{\n\t\t\t\tPlugin: \"xyz\",\n\t\t\t\tItem: test.item,\n\t\t\t\tBackup: test.backup,\n\t\t\t}\n\n\t\t\tresp, err := s.Execute(t.Context(), req)\n\n\t\t\t// Verify error\n\t\t\tassert.Equal(t, test.expectError, err != nil)\n\t\t\tif err != nil {\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NotNil(t, resp)\n\n\t\t\t// Verify updated item\n\t\t\tupdatedItem := test.implUpdatedItem\n\t\t\tif updatedItem == nil {\n\t\t\t\t// If the impl returned nil for its updatedItem, we should expect the plugin to return the original item\n\t\t\t\tupdatedItem = &validItemObject\n\t\t\t}\n\n\t\t\tvar respItem unstructured.Unstructured\n\t\t\terr = json.Unmarshal(resp.Item, &respItem)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.Equal(t, updatedItem, &respItem)\n\n\t\t\t// Verify additional items\n\t\t\tvar expectedAdditionalItems []*proto.ResourceIdentifier\n\t\t\tfor _, item := range test.implAdditionalItems {\n\t\t\t\texpectedAdditionalItems = append(expectedAdditionalItems, backupResourceIdentifierToProto(item))\n\t\t\t}\n\t\t\tassert.Equal(t, expectedAdditionalItems, resp.AdditionalItems)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/plugin/framework/backupitemaction/v2/backup_item_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v2\n\nimport (\n\t\"context\"\n\n\tplugin \"github.com/hashicorp/go-plugin\"\n\t\"google.golang.org/grpc\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tprotobiav2 \"github.com/vmware-tanzu/velero/pkg/plugin/generated/backupitemaction/v2\"\n)\n\n// BackupItemActionPlugin is an implementation of go-plugin's Plugin\n// interface with support for gRPC for the backup/ItemAction\n// interface.\ntype BackupItemActionPlugin struct {\n\tplugin.NetRPCUnsupportedPlugin\n\t*common.PluginBase\n}\n\n// GRPCClient returns a clientDispenser for BackupItemAction gRPC clients.\nfunc (p *BackupItemActionPlugin) GRPCClient(_ context.Context, _ *plugin.GRPCBroker, clientConn *grpc.ClientConn) (any, error) {\n\treturn common.NewClientDispenser(p.ClientLogger, clientConn, newBackupItemActionGRPCClient), nil\n}\n\n// GRPCServer registers a BackupItemAction gRPC server.\nfunc (p *BackupItemActionPlugin) GRPCServer(_ *plugin.GRPCBroker, server *grpc.Server) error {\n\tprotobiav2.RegisterBackupItemActionServer(server, &BackupItemActionGRPCServer{mux: p.ServerMux})\n\treturn nil\n}\n" }, { "path": "pkg/plugin/framework/backupitemaction/v2/backup_item_action_client.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v2\n\nimport (\n\t\"encoding/json\"\n\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"google.golang.org/grpc\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tprotobiav2 \"github.com/vmware-tanzu/velero/pkg/plugin/generated/backupitemaction/v2\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// NewBackupItemActionPlugin constructs a BackupItemActionPlugin.\nfunc NewBackupItemActionPlugin(options ...common.PluginOption) *BackupItemActionPlugin {\n\treturn &BackupItemActionPlugin{\n\t\tPluginBase: common.NewPluginBase(options...),\n\t}\n}\n\n// BackupItemActionGRPCClient implements the backup/ItemAction interface and uses a\n// gRPC client to make calls to the plugin server.\ntype BackupItemActionGRPCClient struct {\n\t*common.ClientBase\n\tgrpcClient protobiav2.BackupItemActionClient\n}\n\nfunc newBackupItemActionGRPCClient(base *common.ClientBase, clientConn *grpc.ClientConn) any {\n\treturn &BackupItemActionGRPCClient{\n\t\tClientBase: base,\n\t\tgrpcClient: protobiav2.NewBackupItemActionClient(clientConn),\n\t}\n}\n\nfunc (c *BackupItemActionGRPCClient) AppliesTo() (velero.ResourceSelector, error) {\n\treq := &protobiav2.BackupItemActionAppliesToRequest{\n\t\tPlugin: c.Plugin,\n\t}\n\n\tres, err := c.grpcClient.AppliesTo(context.Background(), req)\n\tif err != nil {\n\t\treturn velero.ResourceSelector{}, common.FromGRPCError(err)\n\t}\n\n\tif res.ResourceSelector == nil {\n\t\treturn velero.ResourceSelector{}, nil\n\t}\n\n\treturn velero.ResourceSelector{\n\t\tIncludedNamespaces: res.ResourceSelector.IncludedNamespaces,\n\t\tExcludedNamespaces: res.ResourceSelector.ExcludedNamespaces,\n\t\tIncludedResources: res.ResourceSelector.IncludedResources,\n\t\tExcludedResources: res.ResourceSelector.ExcludedResources,\n\t\tLabelSelector: res.ResourceSelector.Selector,\n\t}, nil\n}\n\nfunc (c *BackupItemActionGRPCClient) Execute(item runtime.Unstructured, backup *api.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\titemJSON, err := json.Marshal(item.UnstructuredContent())\n\tif err != nil {\n\t\treturn nil, nil, \"\", nil, errors.WithStack(err)\n\t}\n\n\tbackupJSON, err := json.Marshal(backup)\n\tif err != nil {\n\t\treturn nil, nil, \"\", nil, errors.WithStack(err)\n\t}\n\n\treq := &protobiav2.ExecuteRequest{\n\t\tPlugin: c.Plugin,\n\t\tItem: itemJSON,\n\t\tBackup: backupJSON,\n\t}\n\n\tres, err := c.grpcClient.Execute(context.Background(), req)\n\tif err != nil {\n\t\treturn nil, nil, \"\", nil, common.FromGRPCError(err)\n\t}\n\n\tvar updatedItem unstructured.Unstructured\n\tif err := json.Unmarshal(res.Item, &updatedItem); err != nil {\n\t\treturn nil, nil, \"\", nil, errors.WithStack(err)\n\t}\n\n\tvar additionalItems []velero.ResourceIdentifier\n\n\tfor _, itm := range res.AdditionalItems {\n\t\tnewItem := velero.ResourceIdentifier{\n\t\t\tGroupResource: schema.GroupResource{\n\t\t\t\tGroup: itm.Group,\n\t\t\t\tResource: itm.Resource,\n\t\t\t},\n\t\t\tNamespace: itm.Namespace,\n\t\t\tName: itm.Name,\n\t\t}\n\n\t\tadditionalItems = append(additionalItems, newItem)\n\t}\n\n\tvar postOperationItems []velero.ResourceIdentifier\n\n\tfor _, itm := range res.PostOperationItems {\n\t\tnewItem := velero.ResourceIdentifier{\n\t\t\tGroupResource: schema.GroupResource{\n\t\t\t\tGroup: itm.Group,\n\t\t\t\tResource: itm.Resource,\n\t\t\t},\n\t\t\tNamespace: itm.Namespace,\n\t\t\tName: itm.Name,\n\t\t}\n\n\t\tpostOperationItems = append(postOperationItems, newItem)\n\t}\n\n\treturn &updatedItem, additionalItems, res.OperationID, postOperationItems, nil\n}\n\nfunc (c *BackupItemActionGRPCClient) Progress(operationID string, backup *api.Backup) (velero.OperationProgress, error) {\n\tbackupJSON, err := json.Marshal(backup)\n\tif err != nil {\n\t\treturn velero.OperationProgress{}, errors.WithStack(err)\n\t}\n\treq := &protobiav2.BackupItemActionProgressRequest{\n\t\tPlugin: c.Plugin,\n\t\tOperationID: operationID,\n\t\tBackup: backupJSON,\n\t}\n\n\tres, err := c.grpcClient.Progress(context.Background(), req)\n\tif err != nil {\n\t\treturn velero.OperationProgress{}, common.FromGRPCError(err)\n\t}\n\n\treturn velero.OperationProgress{\n\t\tCompleted: res.Progress.Completed,\n\t\tErr: res.Progress.Err,\n\t\tNCompleted: res.Progress.NCompleted,\n\t\tNTotal: res.Progress.NTotal,\n\t\tOperationUnits: res.Progress.OperationUnits,\n\t\tDescription: res.Progress.Description,\n\t\tStarted: res.Progress.Started.AsTime(),\n\t\tUpdated: res.Progress.Updated.AsTime(),\n\t}, nil\n}\n\nfunc (c *BackupItemActionGRPCClient) Cancel(operationID string, backup *api.Backup) error {\n\tbackupJSON, err := json.Marshal(backup)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\treq := &protobiav2.BackupItemActionCancelRequest{\n\t\tPlugin: c.Plugin,\n\t\tOperationID: operationID,\n\t\tBackup: backupJSON,\n\t}\n\n\t_, err = c.grpcClient.Cancel(context.Background(), req)\n\tif err != nil {\n\t\treturn common.FromGRPCError(err)\n\t}\n\n\treturn nil\n}\n\n// This shouldn't be called on the GRPC client since the RestartableBackupItemAction won't delegate\n// this method\nfunc (c *BackupItemActionGRPCClient) Name() string {\n\treturn \"\"\n}\n" }, { "path": "pkg/plugin/framework/backupitemaction/v2/backup_item_action_server.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v2\n\nimport (\n\t\"encoding/json\"\n\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"google.golang.org/protobuf/types/known/emptypb\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\n\ttimestamppb \"google.golang.org/protobuf/types/known/timestamppb\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n\tprotobiav2 \"github.com/vmware-tanzu/velero/pkg/plugin/generated/backupitemaction/v2\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tbiav2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/backupitemaction/v2\"\n)\n\n// BackupItemActionGRPCServer implements the proto-generated BackupItemAction interface, and accepts\n// gRPC calls and forwards them to an implementation of the pluggable interface.\ntype BackupItemActionGRPCServer struct {\n\tmux *common.ServerMux\n}\n\nfunc (s *BackupItemActionGRPCServer) getImpl(name string) (biav2.BackupItemAction, error) {\n\timpl, err := s.mux.GetHandler(name)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\titemAction, ok := impl.(biav2.BackupItemAction)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"%T is not a backup item action\", impl)\n\t}\n\n\treturn itemAction, nil\n}\n\nfunc (s *BackupItemActionGRPCServer) AppliesTo(\n\tctx context.Context, req *protobiav2.BackupItemActionAppliesToRequest) (\n\tresponse *protobiav2.BackupItemActionAppliesToResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tresourceSelector, err := impl.AppliesTo()\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &protobiav2.BackupItemActionAppliesToResponse{\n\t\tResourceSelector: &proto.ResourceSelector{\n\t\t\tIncludedNamespaces: resourceSelector.IncludedNamespaces,\n\t\t\tExcludedNamespaces: resourceSelector.ExcludedNamespaces,\n\t\t\tIncludedResources: resourceSelector.IncludedResources,\n\t\t\tExcludedResources: resourceSelector.ExcludedResources,\n\t\t\tSelector: resourceSelector.LabelSelector,\n\t\t},\n\t}, nil\n}\n\nfunc (s *BackupItemActionGRPCServer) Execute(\n\tctx context.Context, req *protobiav2.ExecuteRequest) (response *protobiav2.ExecuteResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tvar item unstructured.Unstructured\n\tvar backup api.Backup\n\n\tif err := json.Unmarshal(req.Item, &item); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\tif err := json.Unmarshal(req.Backup, &backup); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\n\tupdatedItem, additionalItems, operationID, postOperationItems, err := impl.Execute(&item, &backup)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\t// If the plugin implementation returned a nil updatedItem (meaning no modifications), reset updatedItem to the\n\t// original item.\n\tvar updatedItemJSON []byte\n\tif updatedItem == nil {\n\t\tupdatedItemJSON = req.Item\n\t} else {\n\t\tupdatedItemJSON, err = json.Marshal(updatedItem.UnstructuredContent())\n\t\tif err != nil {\n\t\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t\t}\n\t}\n\n\tres := &protobiav2.ExecuteResponse{\n\t\tItem: updatedItemJSON,\n\t\tOperationID: operationID,\n\t}\n\n\tfor _, item := range additionalItems {\n\t\tres.AdditionalItems = append(res.AdditionalItems, backupResourceIdentifierToProto(item))\n\t}\n\tfor _, item := range postOperationItems {\n\t\tres.PostOperationItems = append(res.PostOperationItems, backupResourceIdentifierToProto(item))\n\t}\n\n\treturn res, nil\n}\n\nfunc (s *BackupItemActionGRPCServer) Progress(\n\tctx context.Context, req *protobiav2.BackupItemActionProgressRequest) (\n\tresponse *protobiav2.BackupItemActionProgressResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tvar backup api.Backup\n\tif err := json.Unmarshal(req.Backup, &backup); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\n\tprogress, err := impl.Progress(req.OperationID, &backup)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tres := &protobiav2.BackupItemActionProgressResponse{\n\t\tProgress: &proto.OperationProgress{\n\t\t\tCompleted: progress.Completed,\n\t\t\tErr: progress.Err,\n\t\t\tNCompleted: progress.NCompleted,\n\t\t\tNTotal: progress.NTotal,\n\t\t\tOperationUnits: progress.OperationUnits,\n\t\t\tDescription: progress.Description,\n\t\t\tStarted: timestamppb.New(progress.Started),\n\t\t\tUpdated: timestamppb.New(progress.Updated),\n\t\t},\n\t}\n\treturn res, nil\n}\n\nfunc (s *BackupItemActionGRPCServer) Cancel(\n\tctx context.Context, req *protobiav2.BackupItemActionCancelRequest) (\n\tresponse *emptypb.Empty, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tvar backup api.Backup\n\tif err := json.Unmarshal(req.Backup, &backup); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\n\terr = impl.Cancel(req.OperationID, &backup)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &emptypb.Empty{}, nil\n}\n\nfunc backupResourceIdentifierToProto(id velero.ResourceIdentifier) *proto.ResourceIdentifier {\n\treturn &proto.ResourceIdentifier{\n\t\tGroup: id.Group,\n\t\tResource: id.Resource,\n\t\tNamespace: id.Namespace,\n\t\tName: id.Name,\n\t}\n}\n\n// This shouldn't be called on the GRPC server since the server won't ever receive this request, as\n// the RestartableBackupItemAction in Velero won't delegate this to the server\nfunc (s *BackupItemActionGRPCServer) Name() string {\n\treturn \"\"\n}\n" }, { "path": "pkg/plugin/framework/backupitemaction/v2/backup_item_action_test.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v2\n\nimport (\n\t\"encoding/json\"\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n\tprotobiav2 \"github.com/vmware-tanzu/velero/pkg/plugin/generated/backupitemaction/v2\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tmocks \"github.com/vmware-tanzu/velero/pkg/plugin/velero/mocks/backupitemaction/v2\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestBackupItemActionGRPCServerExecute(t *testing.T) {\n\tinvalidItem := []byte(\"this is gibberish json\")\n\tvalidItem := []byte(`\n\t{\n\t\t\"apiVersion\": \"v1\",\n\t\t\"kind\": \"ConfigMap\",\n\t\t\"metadata\": {\n\t\t\t\"namespace\": \"myns\",\n\t\t\t\"name\": \"myconfigmap\"\n\t\t},\n\t\t\"data\": {\n\t\t\t\"key\": \"value\"\n\t\t}\n\t}`)\n\tvar validItemObject unstructured.Unstructured\n\terr := json.Unmarshal(validItem, &validItemObject)\n\trequire.NoError(t, err)\n\n\tupdatedItem := []byte(`\n\t\t{\n\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\"kind\": \"ConfigMap\",\n\t\t\t\"metadata\": {\n\t\t\t\t\"namespace\": \"myns\",\n\t\t\t\t\"name\": \"myconfigmap\"\n\t\t\t},\n\t\t\t\"data\": {\n\t\t\t\t\"key\": \"changed!\"\n\t\t\t}\n\t\t}`)\n\tvar updatedItemObject unstructured.Unstructured\n\terr = json.Unmarshal(updatedItem, &updatedItemObject)\n\trequire.NoError(t, err)\n\n\tinvalidBackup := []byte(\"this is gibberish json\")\n\tvalidBackup := []byte(`\n\t{\n\t\t\"apiVersion\": \"velero.io/v1\",\n\t\t\"kind\": \"Backup\",\n\t\t\"metadata\": {\n\t\t\t\"namespace\": \"myns\",\n\t\t\t\"name\": \"mybackup\"\n\t\t},\n\t\t\"spec\": {\n\t\t\t\"includedNamespaces\": [\"*\"],\n\t\t\t\"includedResources\": [\"*\"],\n\t\t\t\"ttl\": \"60m\"\n\t\t}\n\t}`)\n\tvar validBackupObject v1.Backup\n\terr = json.Unmarshal(validBackup, &validBackupObject)\n\trequire.NoError(t, err)\n\n\ttests := []struct {\n\t\tname string\n\t\tbackup []byte\n\t\titem []byte\n\t\timplUpdatedItem runtime.Unstructured\n\t\timplAdditionalItems []velero.ResourceIdentifier\n\t\timplOperationID string\n\t\timplPostOperationItems []velero.ResourceIdentifier\n\t\timplError error\n\t\texpectError bool\n\t\tskipMock bool\n\t}{\n\t\t{\n\t\t\tname: \"error unmarshaling item\",\n\t\t\titem: invalidItem,\n\t\t\tbackup: validBackup,\n\t\t\texpectError: true,\n\t\t\tskipMock: true,\n\t\t},\n\t\t{\n\t\t\tname: \"error unmarshaling backup\",\n\t\t\titem: validItem,\n\t\t\tbackup: invalidBackup,\n\t\t\texpectError: true,\n\t\t\tskipMock: true,\n\t\t},\n\t\t{\n\t\t\tname: \"error running impl\",\n\t\t\titem: validItem,\n\t\t\tbackup: validBackup,\n\t\t\timplError: errors.New(\"impl error\"),\n\t\t\texpectError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"nil updatedItem / no additionalItems\",\n\t\t\titem: validItem,\n\t\t\tbackup: validBackup,\n\t\t},\n\t\t{\n\t\t\tname: \"same updatedItem / some additionalItems\",\n\t\t\titem: validItem,\n\t\t\tbackup: validBackup,\n\t\t\timplUpdatedItem: &validItemObject,\n\t\t\timplAdditionalItems: []velero.ResourceIdentifier{\n\t\t\t\t{\n\t\t\t\t\tGroupResource: schema.GroupResource{Group: \"v1\", Resource: \"pods\"},\n\t\t\t\t\tNamespace: \"myns\",\n\t\t\t\t\tName: \"mypod\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"different updatedItem\",\n\t\t\titem: validItem,\n\t\t\tbackup: validBackup,\n\t\t\timplUpdatedItem: &updatedItemObject,\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\titemAction := &mocks.BackupItemAction{}\n\t\t\tdefer itemAction.AssertExpectations(t)\n\n\t\t\tif !test.skipMock {\n\t\t\t\titemAction.On(\"Execute\", &validItemObject, &validBackupObject).Return(test.implUpdatedItem, test.implAdditionalItems, test.implOperationID, test.implPostOperationItems, test.implError)\n\t\t\t}\n\n\t\t\ts := &BackupItemActionGRPCServer{mux: &common.ServerMux{\n\t\t\t\tServerLog: velerotest.NewLogger(),\n\t\t\t\tHandlers: map[string]any{\n\t\t\t\t\t\"xyz\": itemAction,\n\t\t\t\t},\n\t\t\t}}\n\n\t\t\treq := &protobiav2.ExecuteRequest{\n\t\t\t\tPlugin: \"xyz\",\n\t\t\t\tItem: test.item,\n\t\t\t\tBackup: test.backup,\n\t\t\t}\n\n\t\t\tresp, err := s.Execute(t.Context(), req)\n\n\t\t\t// Verify error\n\t\t\tassert.Equal(t, test.expectError, err != nil)\n\t\t\tif err != nil {\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NotNil(t, resp)\n\n\t\t\t// Verify updated item\n\t\t\tupdatedItem := test.implUpdatedItem\n\t\t\tif updatedItem == nil {\n\t\t\t\t// If the impl returned nil for its updatedItem, we should expect the plugin to return the original item\n\t\t\t\tupdatedItem = &validItemObject\n\t\t\t}\n\n\t\t\tvar respItem unstructured.Unstructured\n\t\t\terr = json.Unmarshal(resp.Item, &respItem)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.Equal(t, updatedItem, &respItem)\n\n\t\t\t// Verify additional items\n\t\t\tvar expectedAdditionalItems []*proto.ResourceIdentifier\n\t\t\tfor _, item := range test.implAdditionalItems {\n\t\t\t\texpectedAdditionalItems = append(expectedAdditionalItems, backupResourceIdentifierToProto(item))\n\t\t\t}\n\t\t\tassert.Equal(t, expectedAdditionalItems, resp.AdditionalItems)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/plugin/framework/common/client_dispenser.go", "content": "/*\nCopyright 2018, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage common\n\nimport (\n\t\"github.com/sirupsen/logrus\"\n\t\"google.golang.org/grpc\"\n)\n\n// ClientBase implements client and contains shared fields common to all clients.\ntype ClientBase struct {\n\tPlugin string\n\tLogger logrus.FieldLogger\n}\n\ntype ClientDispenser interface {\n\tClientFor(name string) any\n}\n\n// clientDispenser supports the initialization and retrieval of multiple implementations for a single plugin kind, such as\n// \"aws\" and \"azure\" implementations of the object store plugin.\ntype clientDispenser struct {\n\t// logger is the log the plugin should use.\n\tlogger logrus.FieldLogger\n\t// clienConn is shared among all implementations for this client.\n\tclientConn *grpc.ClientConn\n\t// initFunc returns a client that implements a plugin interface, such as ObjectStore.\n\tinitFunc clientInitFunc\n\t// clients keeps track of all the initialized implementations.\n\tclients map[string]any\n}\n\ntype clientInitFunc func(base *ClientBase, clientConn *grpc.ClientConn) any\n\n// newClientDispenser creates a new clientDispenser.\nfunc NewClientDispenser(logger logrus.FieldLogger, clientConn *grpc.ClientConn, initFunc clientInitFunc) *clientDispenser {\n\treturn &clientDispenser{\n\t\tclientConn: clientConn,\n\t\tlogger: logger,\n\t\tinitFunc: initFunc,\n\t\tclients: make(map[string]any),\n\t}\n}\n\n// ClientFor returns a gRPC client stub for the implementation of a plugin named name. If the client stub does not\n// currently exist, clientFor creates it.\nfunc (cd *clientDispenser) ClientFor(name string) any {\n\tif client, found := cd.clients[name]; found {\n\t\treturn client\n\t}\n\n\tbase := &ClientBase{\n\t\tPlugin: name,\n\t\tLogger: cd.logger,\n\t}\n\t// Initialize the plugin (e.g. newBackupItemActionGRPCClient())\n\tclient := cd.initFunc(base, cd.clientConn)\n\tcd.clients[name] = client\n\n\treturn client\n}\n" }, { "path": "pkg/plugin/framework/common/client_dispenser_test.go", "content": "/*\nCopyright 2018, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage common\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"google.golang.org/grpc\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\ntype fakeClient struct {\n\tbase *ClientBase\n\tclientConn *grpc.ClientConn\n}\n\nfunc TestNewClientDispenser(t *testing.T) {\n\tlogger := test.NewLogger()\n\n\tclientConn := new(grpc.ClientConn)\n\n\tc := 3\n\tinitFunc := func(base *ClientBase, clientConn *grpc.ClientConn) any {\n\t\treturn c\n\t}\n\n\tcd := NewClientDispenser(logger, clientConn, initFunc)\n\tassert.Equal(t, clientConn, cd.clientConn)\n\tassert.NotNil(t, cd.clients)\n\tassert.Empty(t, cd.clients)\n}\n\nfunc TestClientFor(t *testing.T) {\n\tlogger := test.NewLogger()\n\tclientConn := new(grpc.ClientConn)\n\n\tc := new(fakeClient)\n\tcount := 0\n\tinitFunc := func(base *ClientBase, clientConn *grpc.ClientConn) any {\n\t\tc.base = base\n\t\tc.clientConn = clientConn\n\t\tcount++\n\t\treturn c\n\t}\n\n\tcd := NewClientDispenser(logger, clientConn, initFunc)\n\n\tactual := cd.ClientFor(\"pod\")\n\trequire.IsType(t, &fakeClient{}, actual)\n\ttyped := actual.(*fakeClient)\n\tassert.Equal(t, 1, count)\n\tassert.Equal(t, &typed, &c)\n\texpectedBase := &ClientBase{\n\t\tPlugin: \"pod\",\n\t\tLogger: logger,\n\t}\n\tassert.Equal(t, expectedBase, typed.base)\n\tassert.Equal(t, clientConn, typed.clientConn)\n\n\t// Make sure we reuse a previous client\n\tactual = cd.ClientFor(\"pod\")\n\trequire.IsType(t, &fakeClient{}, actual)\n\ttyped = actual.(*fakeClient)\n\tassert.Equal(t, 1, count)\n}\n" }, { "path": "pkg/plugin/framework/common/client_errors.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage common\n\nimport (\n\t\"google.golang.org/grpc/status\"\n\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n)\n\n// FromGRPCError takes a gRPC status error, extracts a stack trace\n// from the details if it exists, and returns an error that can\n// provide information about where it was created.\n//\n// This function should be used in the internal plugin client code to convert\n// all errors returned from the plugin server before they're passed back to\n// the rest of the Velero codebase. This will enable them to display location\n// information when they're logged.\nfunc FromGRPCError(err error) error {\n\tstatusErr, ok := status.FromError(err)\n\tif !ok {\n\t\treturn statusErr.Err()\n\t}\n\n\tfor _, detail := range statusErr.Details() {\n\t\tif t, ok := detail.(*proto.Stack); ok {\n\t\t\treturn &ProtoStackError{\n\t\t\t\terror: err,\n\t\t\t\tstack: t,\n\t\t\t}\n\t\t}\n\t}\n\n\treturn err\n}\n\ntype ProtoStackError struct {\n\terror\n\tstack *proto.Stack\n}\n\nfunc (e *ProtoStackError) File() string {\n\tif e.stack == nil || len(e.stack.Frames) < 1 {\n\t\treturn \"\"\n\t}\n\n\treturn e.stack.Frames[0].File\n}\n\nfunc (e *ProtoStackError) Line() int32 {\n\tif e.stack == nil || len(e.stack.Frames) < 1 {\n\t\treturn 0\n\t}\n\n\treturn e.stack.Frames[0].Line\n}\n\nfunc (e *ProtoStackError) Function() string {\n\tif e.stack == nil || len(e.stack.Frames) < 1 {\n\t\treturn \"\"\n\t}\n\n\treturn e.stack.Frames[0].Function\n}\n" }, { "path": "pkg/plugin/framework/common/handle_panic.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage common\n\nimport (\n\t\"runtime/debug\"\n\n\t\"github.com/pkg/errors\"\n\t\"google.golang.org/grpc/codes\"\n)\n\n// HandlePanic is a panic handler for the server half of velero plugins.\nfunc HandlePanic(p any) error {\n\tif p == nil {\n\t\treturn nil\n\t}\n\n\t// If p is an error with a stack trace, we want to retain\n\t// it to preserve the stack trace. Otherwise, create a new\n\t// error here.\n\tvar err error\n\n\tif panicErr, ok := p.(error); !ok {\n\t\terr = errors.Errorf(\"plugin panicked: %v\", p)\n\t} else {\n\t\tif _, ok := panicErr.(StackTracer); ok {\n\t\t\terr = panicErr\n\t\t} else {\n\t\t\terrWithStacktrace := errors.Errorf(\"%v, stack trace: %s\", panicErr, debug.Stack())\n\t\t\terr = errors.Wrap(errWithStacktrace, \"plugin panicked\")\n\t\t}\n\t}\n\n\treturn NewGRPCErrorWithCode(err, codes.Aborted)\n}\n" }, { "path": "pkg/plugin/framework/common/plugin_base.go", "content": "/*\nCopyright 2018, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage common\n\nimport (\n\t\"github.com/sirupsen/logrus\"\n)\n\ntype PluginBase struct {\n\tClientLogger logrus.FieldLogger\n\t*ServerMux\n}\n\nfunc NewPluginBase(options ...PluginOption) *PluginBase {\n\tbase := new(PluginBase)\n\tfor _, option := range options {\n\t\toption(base)\n\t}\n\treturn base\n}\n\ntype PluginOption func(base *PluginBase)\n\nfunc ClientLogger(logger logrus.FieldLogger) PluginOption {\n\treturn func(base *PluginBase) {\n\t\tbase.ClientLogger = logger\n\t}\n}\n\nfunc ServerLogger(logger logrus.FieldLogger) PluginOption {\n\treturn func(base *PluginBase) {\n\t\tbase.ServerMux = NewServerMux(logger)\n\t}\n}\n" }, { "path": "pkg/plugin/framework/common/plugin_base_test.go", "content": "/*\nCopyright 2018, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage common\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestClientLogger(t *testing.T) {\n\tbase := &PluginBase{}\n\tlogger := test.NewLogger()\n\tf := ClientLogger(logger)\n\tf(base)\n\tassert.Equal(t, logger, base.ClientLogger)\n}\n\nfunc TestServerLogger(t *testing.T) {\n\tbase := &PluginBase{}\n\tlogger := test.NewLogger()\n\tf := ServerLogger(logger)\n\tf(base)\n\tassert.Equal(t, NewServerMux(logger), base.ServerMux)\n}\n" }, { "path": "pkg/plugin/framework/common/plugin_config.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage common\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\t\"github.com/pkg/errors\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\tcorev1client \"k8s.io/client-go/kubernetes/typed/core/v1\"\n)\n\nfunc PluginConfigLabelSelector(kind PluginKind, name string) string {\n\treturn fmt.Sprintf(\"velero.io/plugin-config,%s=%s\", name, kind)\n}\n\nfunc GetPluginConfig(kind PluginKind, name string, client corev1client.ConfigMapInterface) (*corev1api.ConfigMap, error) {\n\topts := metav1.ListOptions{\n\t\t// velero.io/plugin-config: true\n\t\t// velero.io/pod-volume-restore: RestoreItemAction\n\t\tLabelSelector: PluginConfigLabelSelector(kind, name),\n\t}\n\n\tlist, err := client.List(context.Background(), opts)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tif len(list.Items) == 0 {\n\t\treturn nil, nil\n\t}\n\n\tif len(list.Items) > 1 {\n\t\tvar items []string\n\t\tfor _, item := range list.Items {\n\t\t\titems = append(items, item.Name)\n\t\t}\n\t\treturn nil, errors.Errorf(\"found more than one ConfigMap matching label selector %q: %v\", opts.LabelSelector, items)\n\t}\n\n\treturn &list.Items[0], nil\n}\n" }, { "path": "pkg/plugin/framework/common/plugin_config_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage common\n\nimport (\n\t\"reflect\"\n\t\"testing\"\n\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes/fake\"\n\n\tvelerov1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\nfunc TestGetPluginConfig(t *testing.T) {\n\ttype args struct {\n\t\tkind PluginKind\n\t\tname string\n\t\tobjects []runtime.Object\n\t}\n\tpluginLabelsMap := map[string]string{\"velero.io/plugin-config\": \"\", \"foo\": \"RestoreItemAction\"}\n\ttestConfigMap := &corev1api.ConfigMap{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"ConfigMap\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"foo-config\",\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tLabels: pluginLabelsMap,\n\t\t},\n\t}\n\ttests := []struct {\n\t\tname string\n\t\targs args\n\t\twant *corev1api.ConfigMap\n\t\twantErr bool\n\t}{\n\t\t{\n\t\t\tname: \"should return nil if no config map found\",\n\t\t\targs: args{\n\t\t\t\tkind: PluginKindRestoreItemAction,\n\t\t\t\tname: \"foo\",\n\t\t\t\tobjects: []runtime.Object{},\n\t\t\t},\n\t\t\twant: nil,\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should return error if more than one config map found\",\n\t\t\targs: args{\n\t\t\t\tkind: PluginKindRestoreItemAction,\n\t\t\t\tname: \"foo\",\n\t\t\t\tobjects: []runtime.Object{\n\t\t\t\t\t&corev1api.ConfigMap{\n\t\t\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\t\t\tKind: \"ConfigMap\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"foo-config\",\n\t\t\t\t\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\t\t\t\t\tLabels: pluginLabelsMap,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t&corev1api.ConfigMap{\n\t\t\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\t\t\tKind: \"ConfigMap\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tName: \"foo-config-duplicate\",\n\t\t\t\t\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\t\t\t\t\tLabels: pluginLabelsMap,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: nil,\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"should return pointer to configmap if only one config map with label found\",\n\t\t\targs: args{\n\t\t\t\tkind: PluginKindRestoreItemAction,\n\t\t\t\tname: \"foo\",\n\t\t\t\tobjects: []runtime.Object{\n\t\t\t\t\ttestConfigMap,\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: testConfigMap,\n\t\t\twantErr: false,\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tfakeClient := fake.NewSimpleClientset(tt.args.objects...)\n\t\t\tgot, err := GetPluginConfig(tt.args.kind, tt.args.name, fakeClient.CoreV1().ConfigMaps(velerov1.DefaultNamespace))\n\t\t\tif (err != nil) != tt.wantErr {\n\t\t\t\tt.Errorf(\"GetPluginConfig() error = %v, wantErr %v\", err, tt.wantErr)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif !reflect.DeepEqual(got, tt.want) {\n\t\t\t\tt.Errorf(\"GetPluginConfig() = %v, want %v\", got, tt.want)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/plugin/framework/common/plugin_kinds.go", "content": "/*\nCopyright 2018, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage common\n\n// PluginKind is a type alias for a string that describes\n// the kind of a Velero-supported plugin.\ntype PluginKind string\n\n// String returns the string for k.\nfunc (k PluginKind) String() string {\n\treturn string(k)\n}\n\nconst (\n\t// PluginKindObjectStore represents an object store plugin.\n\tPluginKindObjectStore PluginKind = \"ObjectStore\"\n\n\t// PluginKindVolumeSnapshotter represents a volume snapshotter plugin.\n\tPluginKindVolumeSnapshotter PluginKind = \"VolumeSnapshotter\"\n\n\t// PluginKindBackupItemAction represents a backup item action plugin.\n\tPluginKindBackupItemAction PluginKind = \"BackupItemAction\"\n\n\t// PluginKindBackupItemActionV2 represents a v2 backup item action plugin.\n\tPluginKindBackupItemActionV2 PluginKind = \"BackupItemActionV2\"\n\n\t// PluginKindRestoreItemAction represents a restore item action plugin.\n\tPluginKindRestoreItemAction PluginKind = \"RestoreItemAction\"\n\n\t// PluginKindRestoreItemActionV2 represents a v2 restore item action plugin.\n\tPluginKindRestoreItemActionV2 PluginKind = \"RestoreItemActionV2\"\n\n\t// PluginKindDeleteItemAction represents a delete item action plugin.\n\tPluginKindDeleteItemAction PluginKind = \"DeleteItemAction\"\n\n\t// PluginKindItemBlockAction represents a v1 ItemBlock action plugin.\n\tPluginKindItemBlockAction PluginKind = \"ItemBlockAction\"\n\n\t// PluginKindPluginLister represents a plugin lister plugin.\n\tPluginKindPluginLister PluginKind = \"PluginLister\"\n)\n\n// PluginKindsAdaptableTo if there are plugin kinds that are adaptable to newer API versions, list them here.\n// The older (adaptable) version is the key, and the value is the full list of newer\n// plugin kinds that are capable of adapting it.\nvar PluginKindsAdaptableTo = map[PluginKind][]PluginKind{\n\tPluginKindBackupItemAction: {PluginKindBackupItemActionV2},\n\tPluginKindRestoreItemAction: {PluginKindRestoreItemActionV2},\n}\n\n// AllPluginKinds contains all the valid plugin kinds that Velero supports, excluding PluginLister because that is not a\n// kind that a developer would ever need to implement (it's handled by Velero and the Velero plugin library code).\nfunc AllPluginKinds() map[string]PluginKind {\n\tallPluginKinds := make(map[string]PluginKind)\n\tallPluginKinds[PluginKindObjectStore.String()] = PluginKindObjectStore\n\tallPluginKinds[PluginKindVolumeSnapshotter.String()] = PluginKindVolumeSnapshotter\n\tallPluginKinds[PluginKindBackupItemAction.String()] = PluginKindBackupItemAction\n\tallPluginKinds[PluginKindBackupItemActionV2.String()] = PluginKindBackupItemActionV2\n\tallPluginKinds[PluginKindRestoreItemAction.String()] = PluginKindRestoreItemAction\n\tallPluginKinds[PluginKindRestoreItemActionV2.String()] = PluginKindRestoreItemActionV2\n\tallPluginKinds[PluginKindDeleteItemAction.String()] = PluginKindDeleteItemAction\n\tallPluginKinds[PluginKindItemBlockAction.String()] = PluginKindItemBlockAction\n\treturn allPluginKinds\n}\n" }, { "path": "pkg/plugin/framework/common/server_errors.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage common\n\nimport (\n\t\"github.com/pkg/errors\"\n\t\"google.golang.org/grpc/codes\"\n\t\"google.golang.org/grpc/status\"\n\t\"google.golang.org/protobuf/protoadapt\"\n\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n)\n\n// NewGRPCErrorWithCode wraps err in a gRPC status error with the error's stack trace\n// included in the details if it exists. This provides an easy way to send\n// stack traces from plugin servers across the wire to the plugin client.\n//\n// This function should be used in the internal plugin server code to wrap\n// all errors before they're returned.\nfunc NewGRPCErrorWithCode(err error, code codes.Code, details ...protoadapt.MessageV1) error {\n\t// if it's already a gRPC status error, use it; otherwise, create a new one\n\tstatusErr, ok := status.FromError(err)\n\tif !ok {\n\t\tstatusErr = status.New(code, err.Error())\n\t}\n\n\t// get a Stack for the error and add it to details\n\tif stack := ErrorStack(err); stack != nil {\n\t\tdetails = append(details, stack)\n\t}\n\n\tstatusErr, err = statusErr.WithDetails(details...)\n\tif err != nil {\n\t\treturn status.Errorf(codes.Unknown, \"error adding details to the gRPC error: %v\", err)\n\t}\n\n\treturn statusErr.Err()\n}\n\n// NewGRPCError is a convenience function for creating a new gRPC error\n// with code = codes.Unknown\nfunc NewGRPCError(err error, details ...protoadapt.MessageV1) error {\n\treturn NewGRPCErrorWithCode(err, codes.Unknown, details...)\n}\n\n// ErrorStack gets a stack trace, if it exists, from the provided error, and\n// returns it as a *proto.Stack.\nfunc ErrorStack(err error) *proto.Stack {\n\tstackTracer, ok := err.(StackTracer)\n\tif !ok {\n\t\treturn nil\n\t}\n\n\tstackTrace := new(proto.Stack)\n\tfor _, frame := range stackTracer.StackTrace() {\n\t\tlocation := logging.GetFrameLocationInfo(frame)\n\n\t\tstackTrace.Frames = append(stackTrace.Frames, &proto.StackFrame{\n\t\t\tFile: location.File,\n\t\t\tLine: int32(location.Line),\n\t\t\tFunction: location.Function,\n\t\t})\n\t}\n\n\treturn stackTrace\n}\n\ntype StackTracer interface {\n\tStackTrace() errors.StackTrace\n}\n" }, { "path": "pkg/plugin/framework/common/server_mux.go", "content": "/*\nCopyright 2018, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage common\n\nimport (\n\t\"strings\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\t\"k8s.io/apimachinery/pkg/util/validation\"\n)\n\n// HandlerInitializer is a function that initializes and returns a new instance of one of Velero's plugin interfaces\n// (ObjectStore, VolumeSnapshotter, BackupItemAction, RestoreItemAction).\ntype HandlerInitializer func(logger logrus.FieldLogger) (any, error)\n\n// ServerMux manages multiple implementations of a single plugin kind, such as pod and pvc BackupItemActions.\ntype ServerMux struct {\n\tkind PluginKind\n\tinitializers map[string]HandlerInitializer\n\tHandlers map[string]any\n\tServerLog logrus.FieldLogger\n}\n\n// NewServerMux returns a new ServerMux.\nfunc NewServerMux(logger logrus.FieldLogger) *ServerMux {\n\treturn &ServerMux{\n\t\tinitializers: make(map[string]HandlerInitializer),\n\t\tHandlers: make(map[string]any),\n\t\tServerLog: logger,\n\t}\n}\n\n// register validates the plugin name and registers the\n// initializer for the given name.\nfunc (m *ServerMux) Register(name string, f HandlerInitializer) {\n\tif err := ValidatePluginName(name, m.Names()); err != nil {\n\t\tm.ServerLog.Errorf(\"invalid plugin name %q: %s\", name, err)\n\t\treturn\n\t}\n\tm.initializers[name] = f\n}\n\n// names returns a list of all registered implementations.\nfunc (m *ServerMux) Names() []string {\n\treturn sets.StringKeySet(m.initializers).List()\n}\n\n// GetHandler returns the instance for a plugin with the given name. If an instance has already been initialized,\n// that is returned. Otherwise, the instance is initialized by calling its initialization function.\nfunc (m *ServerMux) GetHandler(name string) (any, error) {\n\tif instance, found := m.Handlers[name]; found {\n\t\treturn instance, nil\n\t}\n\n\tinitializer, found := m.initializers[name]\n\tif !found {\n\t\treturn nil, errors.Errorf(\"%v plugin: %s was not found or has an invalid name format\", m.kind, name)\n\t}\n\n\tinstance, err := initializer(m.ServerLog)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tm.Handlers[name] = instance\n\n\treturn m.Handlers[name], nil\n}\n\n// ValidatePluginName checks if the given name:\n// - the plugin name has two parts separated by '/'\n// - non of the above parts is empty\n// - the prefix is a valid DNS subdomain name\n// - a plugin with the same name does not already exist (if list of existing names is passed in)\nfunc ValidatePluginName(name string, existingNames []string) error {\n\t// validate there is one \"/\" and two parts\n\tparts := strings.Split(name, \"/\")\n\tif len(parts) != 2 {\n\t\treturn errors.Errorf(\"plugin name must have exactly two parts separated by a `/`. Accepted format: /. %s is invalid\", name)\n\t}\n\n\t// validate both prefix and name are non-empty\n\tif parts[0] == \"\" || parts[1] == \"\" {\n\t\treturn errors.Errorf(\"both parts of the plugin name must be non-empty. Accepted format: /. %s is invalid\", name)\n\t}\n\n\t// validate that the prefix is a DNS subdomain\n\tif errs := validation.IsDNS1123Subdomain(parts[0]); len(errs) != 0 {\n\t\treturn errors.Errorf(\"first part of the plugin name must be a valid DNS subdomain. Accepted format: /. first part %q is invalid: %s\", parts[0], strings.Join(errs, \"; \"))\n\t}\n\n\tfor _, existingName := range existingNames {\n\t\tif strings.Compare(name, existingName) == 0 {\n\t\t\treturn errors.New(\"plugin name \" + existingName + \" already exists\")\n\t\t}\n\t}\n\n\treturn nil\n}\n" }, { "path": "pkg/plugin/framework/common/server_mux_test.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage common\n\nimport (\n\t\"strings\"\n\t\"testing\"\n)\n\nfunc TestValidPluginName(t *testing.T) {\n\tsuccessCases := []struct {\n\t\tpluginName string\n\t\texistingNames []string\n\t}{\n\t\t{\"example.io/azure\", []string{\"velero.io/aws\"}},\n\t\t{\"with-dashes/name\", []string{\"velero.io/aws\"}},\n\t\t{\"prefix/Uppercase_Is_OK_123\", []string{\"velero.io/aws\"}},\n\t\t{\"example-with-dash.io/azure\", []string{\"velero.io/aws\"}},\n\t\t{\"1.2.3.4/5678\", []string{\"velero.io/aws\"}},\n\t\t{\"example.io/azure\", []string{\"velero.io/aws\"}},\n\t\t{\"example.io/azure\", []string{\"\"}},\n\t\t{\"example.io/azure\", nil},\n\t\t{strings.Repeat(\"a\", 253) + \"/name\", []string{\"velero.io/aws\"}},\n\t}\n\tfor i, tt := range successCases {\n\t\tt.Run(tt.pluginName, func(t *testing.T) {\n\t\t\tif err := ValidatePluginName(tt.pluginName, tt.existingNames); err != nil {\n\t\t\t\tt.Errorf(\"case[%d]: %q: expected success: %v\", i, successCases[i], err)\n\t\t\t}\n\t\t})\n\t}\n\n\terrorCases := []struct {\n\t\tpluginName string\n\t\texistingNames []string\n\t}{\n\t\t{\"\", []string{\"velero.io/aws\"}},\n\t\t{\"single\", []string{\"velero.io/aws\"}},\n\t\t{\"/\", []string{\"velero.io/aws\"}},\n\t\t{\"//\", []string{\"velero.io/aws\"}},\n\t\t{\"///\", []string{\"velero.io/aws\"}},\n\t\t{\"a/\", []string{\"velero.io/aws\"}},\n\t\t{\"/a\", []string{\"velero.io/aws\"}},\n\t\t{\"velero.io/aws\", []string{\"velero.io/aws\"}},\n\t\t{\"Uppercase_Is_OK_123/name\", []string{\"velero.io/aws\"}},\n\t\t{strings.Repeat(\"a\", 254) + \"/name\", []string{\"velero.io/aws\"}},\n\t\t{\"ospecialchars%^=@\", []string{\"velero.io/aws\"}},\n\t}\n\n\tfor i, tt := range errorCases {\n\t\tt.Run(tt.pluginName, func(t *testing.T) {\n\t\t\tif err := ValidatePluginName(tt.pluginName, tt.existingNames); err == nil {\n\t\t\t\tt.Errorf(\"case[%d]: %q: expected failure.\", i, errorCases[i])\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/plugin/framework/delete_item_action.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"context\"\n\n\tplugin \"github.com/hashicorp/go-plugin\"\n\t\"google.golang.org/grpc\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n)\n\n// DeleteItemActionPlugin is an implementation of go-plugin's Plugin\n// interface with support for gRPC for the restore/ItemAction\n// interface.\ntype DeleteItemActionPlugin struct {\n\tplugin.NetRPCUnsupportedPlugin\n\t*common.PluginBase\n}\n\n// GRPCClient returns a DeleteItemAction gRPC client.\nfunc (p *DeleteItemActionPlugin) GRPCClient(_ context.Context, _ *plugin.GRPCBroker, clientConn *grpc.ClientConn) (any, error) {\n\treturn common.NewClientDispenser(p.ClientLogger, clientConn, newDeleteItemActionGRPCClient), nil\n}\n\n// GRPCServer registers a DeleteItemAction gRPC server.\nfunc (p *DeleteItemActionPlugin) GRPCServer(_ *plugin.GRPCBroker, server *grpc.Server) error {\n\tproto.RegisterDeleteItemActionServer(server, &DeleteItemActionGRPCServer{mux: p.ServerMux})\n\treturn nil\n}\n" }, { "path": "pkg/plugin/framework/delete_item_action_client.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"encoding/json\"\n\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"google.golang.org/grpc\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\nvar _ velero.DeleteItemAction = &DeleteItemActionGRPCClient{}\n\n// NewDeleteItemActionPlugin constructs a DeleteItemActionPlugin.\nfunc NewDeleteItemActionPlugin(options ...common.PluginOption) *DeleteItemActionPlugin {\n\treturn &DeleteItemActionPlugin{\n\t\tPluginBase: common.NewPluginBase(options...),\n\t}\n}\n\n// DeleteItemActionGRPCClient implements the DeleteItemAction interface and uses a\n// gRPC client to make calls to the plugin server.\ntype DeleteItemActionGRPCClient struct {\n\t*common.ClientBase\n\tgrpcClient proto.DeleteItemActionClient\n}\n\nfunc newDeleteItemActionGRPCClient(base *common.ClientBase, clientConn *grpc.ClientConn) any {\n\treturn &DeleteItemActionGRPCClient{\n\t\tClientBase: base,\n\t\tgrpcClient: proto.NewDeleteItemActionClient(clientConn),\n\t}\n}\n\nfunc (c *DeleteItemActionGRPCClient) AppliesTo() (velero.ResourceSelector, error) {\n\tres, err := c.grpcClient.AppliesTo(context.Background(), &proto.DeleteItemActionAppliesToRequest{Plugin: c.Plugin})\n\tif err != nil {\n\t\treturn velero.ResourceSelector{}, common.FromGRPCError(err)\n\t}\n\n\tif res.ResourceSelector == nil {\n\t\treturn velero.ResourceSelector{}, nil\n\t}\n\n\treturn velero.ResourceSelector{\n\t\tIncludedNamespaces: res.ResourceSelector.IncludedNamespaces,\n\t\tExcludedNamespaces: res.ResourceSelector.ExcludedNamespaces,\n\t\tIncludedResources: res.ResourceSelector.IncludedResources,\n\t\tExcludedResources: res.ResourceSelector.ExcludedResources,\n\t\tLabelSelector: res.ResourceSelector.Selector,\n\t}, nil\n}\n\nfunc (c *DeleteItemActionGRPCClient) Execute(input *velero.DeleteItemActionExecuteInput) error {\n\titemJSON, err := json.Marshal(input.Item.UnstructuredContent())\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\tbackupJSON, err := json.Marshal(input.Backup)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\treq := &proto.DeleteItemActionExecuteRequest{\n\t\tPlugin: c.Plugin,\n\t\tItem: itemJSON,\n\t\tBackup: backupJSON,\n\t}\n\n\t// First return item is just an empty struct no matter what.\n\tif _, err = c.grpcClient.Execute(context.Background(), req); err != nil {\n\t\treturn common.FromGRPCError(err)\n\t}\n\n\treturn nil\n}\n" }, { "path": "pkg/plugin/framework/delete_item_action_server.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"encoding/json\"\n\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// DeleteItemActionGRPCServer implements the proto-generated DeleteItemActionServer interface, and accepts\n// gRPC calls and forwards them to an implementation of the pluggable interface.\ntype DeleteItemActionGRPCServer struct {\n\tmux *common.ServerMux\n}\n\nfunc (s *DeleteItemActionGRPCServer) getImpl(name string) (velero.DeleteItemAction, error) {\n\timpl, err := s.mux.GetHandler(name)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\titemAction, ok := impl.(velero.DeleteItemAction)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"%T is not a delete item action\", impl)\n\t}\n\n\treturn itemAction, nil\n}\n\nfunc (s *DeleteItemActionGRPCServer) AppliesTo(ctx context.Context, req *proto.DeleteItemActionAppliesToRequest) (response *proto.DeleteItemActionAppliesToResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tresourceSelector, err := impl.AppliesTo()\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &proto.DeleteItemActionAppliesToResponse{\n\t\tResourceSelector: &proto.ResourceSelector{\n\t\t\tIncludedNamespaces: resourceSelector.IncludedNamespaces,\n\t\t\tExcludedNamespaces: resourceSelector.ExcludedNamespaces,\n\t\t\tIncludedResources: resourceSelector.IncludedResources,\n\t\t\tExcludedResources: resourceSelector.ExcludedResources,\n\t\t\tSelector: resourceSelector.LabelSelector,\n\t\t},\n\t}, nil\n}\n\nfunc (s *DeleteItemActionGRPCServer) Execute(ctx context.Context, req *proto.DeleteItemActionExecuteRequest) (_ *proto.Empty, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tvar (\n\t\titem unstructured.Unstructured\n\t\tbackup api.Backup\n\t)\n\n\tif err := json.Unmarshal(req.Item, &item); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\n\tif err = json.Unmarshal(req.Backup, &backup); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\n\tif err := impl.Execute(&velero.DeleteItemActionExecuteInput{\n\t\tItem: &item,\n\t\tBackup: &backup,\n\t}); err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &proto.Empty{}, nil\n}\n" }, { "path": "pkg/plugin/framework/doc.go", "content": "// Package framework is the common package that any plugin client\n// will need to import, for example, both plugin authors and Velero core.\npackage framework\n" }, { "path": "pkg/plugin/framework/examples_test.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\nfunc ExampleNewServer_volumeSnapshotter() {\n\tNewServer(). // call the server\n\t\t\tRegisterVolumeSnapshotter(\"example.io/volumesnapshotter\", newVolumeSnapshotter). // register the plugin with a valid name\n\t\t\tRegisterDeleteItemAction(\"example.io/delete-item-action\", newDeleteItemAction).\n\t\t\tServe() // serve the plugin\n}\n\nfunc newVolumeSnapshotter(logger logrus.FieldLogger) (any, error) {\n\treturn &VolumeSnapshotter{FieldLogger: logger}, nil\n}\n\ntype VolumeSnapshotter struct {\n\tFieldLogger logrus.FieldLogger\n}\n\n// Implement all methods for the VolumeSnapshotter interface...\nfunc (b *VolumeSnapshotter) Init(config map[string]string) error {\n\tb.FieldLogger.Infof(\"VolumeSnapshotter.Init called\")\n\n\t// ...\n\n\treturn nil\n}\n\nfunc (b *VolumeSnapshotter) CreateVolumeFromSnapshot(snapshotID, volumeType, volumeAZ string, iops *int64) (volumeID string, err error) {\n\tb.FieldLogger.Infof(\"CreateVolumeFromSnapshot called\")\n\n\t// ...\n\n\treturn \"volumeID\", nil\n}\n\nfunc (b *VolumeSnapshotter) GetVolumeID(pv runtime.Unstructured) (string, error) {\n\tb.FieldLogger.Infof(\"GetVolumeID called\")\n\n\t// ...\n\n\treturn \"volumeID\", nil\n}\n\nfunc (b *VolumeSnapshotter) SetVolumeID(pv runtime.Unstructured, volumeID string) (runtime.Unstructured, error) {\n\tb.FieldLogger.Infof(\"SetVolumeID called\")\n\n\t// ...\n\n\treturn nil, nil\n}\n\nfunc (b *VolumeSnapshotter) GetVolumeInfo(volumeID, volumeAZ string) (string, *int64, error) {\n\tb.FieldLogger.Infof(\"GetVolumeInfo called\")\n\n\t// ...\n\n\treturn \"volumeFilesystemType\", nil, nil\n}\n\nfunc (b *VolumeSnapshotter) CreateSnapshot(volumeID, volumeAZ string, tags map[string]string) (snapshotID string, err error) {\n\tb.FieldLogger.Infof(\"CreateSnapshot called\")\n\n\t// ...\n\n\treturn \"snapshotID\", nil\n}\n\nfunc (b *VolumeSnapshotter) DeleteSnapshot(snapshotID string) error {\n\tb.FieldLogger.Infof(\"DeleteSnapshot called\")\n\n\t// ...\n\n\treturn nil\n}\n\n// Implement all methods for the DeleteItemAction interface\n\nfunc newDeleteItemAction(logger logrus.FieldLogger) (any, error) {\n\treturn DeleteItemAction{FieldLogger: logger}, nil\n}\n\ntype DeleteItemAction struct {\n\tFieldLogger logrus.FieldLogger\n}\n\nfunc (d *DeleteItemAction) AppliesTo() (velero.ResourceSelector, error) {\n\td.FieldLogger.Infof(\"AppliesTo called\")\n\n\t// ...\n\n\treturn velero.ResourceSelector{}, nil\n}\n\nfunc (d *DeleteItemAction) Execute(input *velero.DeleteItemActionExecuteInput) error {\n\td.FieldLogger.Infof(\"Execute called\")\n\n\t// ...\n\n\treturn nil\n}\n" }, { "path": "pkg/plugin/framework/handshake.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport plugin \"github.com/hashicorp/go-plugin\"\n\n// Handshake returns the configuration information that allows go-plugin clients and servers to perform a handshake.\nfunc Handshake() plugin.HandshakeConfig {\n\treturn plugin.HandshakeConfig{\n\t\t// The ProtocolVersion is the version that must match between Velero framework\n\t\t// and Velero client plugins. This should be bumped whenever a change happens in\n\t\t// one or the other that makes it so that they can't safely communicate.\n\t\tProtocolVersion: 2,\n\n\t\tMagicCookieKey: \"VELERO_PLUGIN\",\n\t\tMagicCookieValue: \"hello\",\n\t}\n}\n" }, { "path": "pkg/plugin/framework/import_test.go", "content": "package framework\n\nimport (\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"regexp\"\n\t\"runtime\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\n// test that this package do not import cloud provider\n\n// Prevent https://github.com/vmware-tanzu/velero/issues/8207 and https://github.com/vmware-tanzu/velero/issues/8157\nfunc TestPkgImportNoCloudProvider(t *testing.T) {\n\t_, filename, _, ok := runtime.Caller(0)\n\tif !ok {\n\t\tt.Fatalf(\"No caller information\")\n\t}\n\tt.Logf(\"Current test file path: %s\", filename)\n\tt.Logf(\"Current test directory: %s\", filepath.Dir(filename)) // should be this package name\n\t// go list -f {{.Deps}} ./\n\tcmd := exec.CommandContext(\n\t\tt.Context(),\n\t\t\"go\",\n\t\t\"list\",\n\t\t\"-f\",\n\t\t\"{{.Deps}}\",\n\t\t\".\",\n\t)\n\t// set cmd.Dir to this package even if executed from different dir\n\tcmd.Dir = filepath.Dir(filename)\n\toutput, err := cmd.Output()\n\trequire.NoError(t, err)\n\t// split dep by line, replace space with newline\n\tdeps := strings.ReplaceAll(string(output), \" \", \"\\n\")\n\trequire.NotEmpty(t, deps)\n\t// ignore k8s.io\n\tk8sio, err := regexp.Compile(\"^k8s.io\")\n\trequire.NoError(t, err)\n\tcloudProvider, err := regexp.Compile(\"aws|cloud.google.com|azure\")\n\trequire.NoError(t, err)\n\tcloudProviderDeps := []string{}\n\tfor _, dep := range strings.Split(deps, \"\\n\") {\n\t\tif !k8sio.MatchString(dep) {\n\t\t\tif cloudProvider.MatchString(dep) {\n\t\t\t\tcloudProviderDeps = append(cloudProviderDeps, dep)\n\t\t\t}\n\t\t}\n\t}\n\trequire.Empty(t, cloudProviderDeps)\n}\n" }, { "path": "pkg/plugin/framework/interface.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport plugin \"github.com/hashicorp/go-plugin\"\n\n// Interface represents a Velero plugin.\ntype Interface interface {\n\tplugin.Plugin\n\n\t// names returns a list of all the registered implementations for this plugin (such as \"pod\" and \"pvc\" for\n\t// BackupItemAction).\n\tNames() []string\n}\n" }, { "path": "pkg/plugin/framework/itemblockaction/v1/item_block_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\t\"context\"\n\n\tplugin \"github.com/hashicorp/go-plugin\"\n\t\"google.golang.org/grpc\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tprotoibav1 \"github.com/vmware-tanzu/velero/pkg/plugin/generated/itemblockaction/v1\"\n)\n\n// ItemBlockActionPlugin is an implementation of go-plugin's Plugin\n// interface with support for gRPC for the backup/ItemAction\n// interface.\ntype ItemBlockActionPlugin struct {\n\tplugin.NetRPCUnsupportedPlugin\n\t*common.PluginBase\n}\n\n// GRPCClient returns a clientDispenser for ItemBlockAction gRPC clients.\nfunc (p *ItemBlockActionPlugin) GRPCClient(_ context.Context, _ *plugin.GRPCBroker, clientConn *grpc.ClientConn) (any, error) {\n\treturn common.NewClientDispenser(p.ClientLogger, clientConn, newItemBlockActionGRPCClient), nil\n}\n\n// GRPCServer registers a ItemBlockAction gRPC server.\nfunc (p *ItemBlockActionPlugin) GRPCServer(_ *plugin.GRPCBroker, server *grpc.Server) error {\n\tprotoibav1.RegisterItemBlockActionServer(server, &ItemBlockActionGRPCServer{mux: p.ServerMux})\n\treturn nil\n}\n" }, { "path": "pkg/plugin/framework/itemblockaction/v1/item_block_action_client.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\t\"encoding/json\"\n\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"google.golang.org/grpc\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tprotoibav1 \"github.com/vmware-tanzu/velero/pkg/plugin/generated/itemblockaction/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// NewItemBlockActionPlugin constructs a ItemBlockActionPlugin.\nfunc NewItemBlockActionPlugin(options ...common.PluginOption) *ItemBlockActionPlugin {\n\treturn &ItemBlockActionPlugin{\n\t\tPluginBase: common.NewPluginBase(options...),\n\t}\n}\n\n// ItemBlockActionGRPCClient implements the backup/ItemAction interface and uses a\n// gRPC client to make calls to the plugin server.\ntype ItemBlockActionGRPCClient struct {\n\t*common.ClientBase\n\tgrpcClient protoibav1.ItemBlockActionClient\n}\n\nfunc newItemBlockActionGRPCClient(base *common.ClientBase, clientConn *grpc.ClientConn) any {\n\treturn &ItemBlockActionGRPCClient{\n\t\tClientBase: base,\n\t\tgrpcClient: protoibav1.NewItemBlockActionClient(clientConn),\n\t}\n}\n\nfunc (c *ItemBlockActionGRPCClient) AppliesTo() (velero.ResourceSelector, error) {\n\treq := &protoibav1.ItemBlockActionAppliesToRequest{\n\t\tPlugin: c.Plugin,\n\t}\n\n\tres, err := c.grpcClient.AppliesTo(context.Background(), req)\n\tif err != nil {\n\t\treturn velero.ResourceSelector{}, common.FromGRPCError(err)\n\t}\n\n\tif res.ResourceSelector == nil {\n\t\treturn velero.ResourceSelector{}, nil\n\t}\n\n\treturn velero.ResourceSelector{\n\t\tIncludedNamespaces: res.ResourceSelector.IncludedNamespaces,\n\t\tExcludedNamespaces: res.ResourceSelector.ExcludedNamespaces,\n\t\tIncludedResources: res.ResourceSelector.IncludedResources,\n\t\tExcludedResources: res.ResourceSelector.ExcludedResources,\n\t\tLabelSelector: res.ResourceSelector.Selector,\n\t}, nil\n}\n\nfunc (c *ItemBlockActionGRPCClient) GetRelatedItems(item runtime.Unstructured, backup *api.Backup) ([]velero.ResourceIdentifier, error) {\n\titemJSON, err := json.Marshal(item.UnstructuredContent())\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tbackupJSON, err := json.Marshal(backup)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\treq := &protoibav1.ItemBlockActionGetRelatedItemsRequest{\n\t\tPlugin: c.Plugin,\n\t\tItem: itemJSON,\n\t\tBackup: backupJSON,\n\t}\n\n\tres, err := c.grpcClient.GetRelatedItems(context.Background(), req)\n\tif err != nil {\n\t\treturn nil, common.FromGRPCError(err)\n\t}\n\n\tvar relatedItems []velero.ResourceIdentifier\n\n\tfor _, itm := range res.RelatedItems {\n\t\tnewItem := velero.ResourceIdentifier{\n\t\t\tGroupResource: schema.GroupResource{\n\t\t\t\tGroup: itm.Group,\n\t\t\t\tResource: itm.Resource,\n\t\t\t},\n\t\t\tNamespace: itm.Namespace,\n\t\t\tName: itm.Name,\n\t\t}\n\n\t\trelatedItems = append(relatedItems, newItem)\n\t}\n\n\treturn relatedItems, nil\n}\n\n// This shouldn't be called on the GRPC client since the RestartableItemBlockAction won't delegate\n// this method\nfunc (c *ItemBlockActionGRPCClient) Name() string {\n\treturn \"\"\n}\n" }, { "path": "pkg/plugin/framework/itemblockaction/v1/item_block_action_server.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\t\"encoding/json\"\n\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n\tprotoibav1 \"github.com/vmware-tanzu/velero/pkg/plugin/generated/itemblockaction/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tibav1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/itemblockaction/v1\"\n)\n\n// ItemBlockActionGRPCServer implements the proto-generated ItemBlockAction interface, and accepts\n// gRPC calls and forwards them to an implementation of the pluggable interface.\ntype ItemBlockActionGRPCServer struct {\n\tmux *common.ServerMux\n}\n\nfunc (s *ItemBlockActionGRPCServer) getImpl(name string) (ibav1.ItemBlockAction, error) {\n\timpl, err := s.mux.GetHandler(name)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\titemAction, ok := impl.(ibav1.ItemBlockAction)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"%T is not an ItemBlock action\", impl)\n\t}\n\n\treturn itemAction, nil\n}\n\nfunc (s *ItemBlockActionGRPCServer) AppliesTo(\n\tctx context.Context, req *protoibav1.ItemBlockActionAppliesToRequest) (\n\tresponse *protoibav1.ItemBlockActionAppliesToResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tresourceSelector, err := impl.AppliesTo()\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &protoibav1.ItemBlockActionAppliesToResponse{\n\t\tResourceSelector: &proto.ResourceSelector{\n\t\t\tIncludedNamespaces: resourceSelector.IncludedNamespaces,\n\t\t\tExcludedNamespaces: resourceSelector.ExcludedNamespaces,\n\t\t\tIncludedResources: resourceSelector.IncludedResources,\n\t\t\tExcludedResources: resourceSelector.ExcludedResources,\n\t\t\tSelector: resourceSelector.LabelSelector,\n\t\t},\n\t}, nil\n}\n\nfunc (s *ItemBlockActionGRPCServer) GetRelatedItems(\n\tctx context.Context, req *protoibav1.ItemBlockActionGetRelatedItemsRequest) (response *protoibav1.ItemBlockActionGetRelatedItemsResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tvar item unstructured.Unstructured\n\tvar backup api.Backup\n\n\tif err := json.Unmarshal(req.Item, &item); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\tif err := json.Unmarshal(req.Backup, &backup); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\n\trelatedItems, err := impl.GetRelatedItems(&item, &backup)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tres := &protoibav1.ItemBlockActionGetRelatedItemsResponse{}\n\n\tfor _, item := range relatedItems {\n\t\tres.RelatedItems = append(res.RelatedItems, backupResourceIdentifierToProto(item))\n\t}\n\n\treturn res, nil\n}\n\nfunc backupResourceIdentifierToProto(id velero.ResourceIdentifier) *proto.ResourceIdentifier {\n\treturn &proto.ResourceIdentifier{\n\t\tGroup: id.Group,\n\t\tResource: id.Resource,\n\t\tNamespace: id.Namespace,\n\t\tName: id.Name,\n\t}\n}\n\n// This shouldn't be called on the GRPC server since the server won't ever receive this request, as\n// the RestartableItemBlockAction in Velero won't delegate this to the server\nfunc (s *ItemBlockActionGRPCServer) Name() string {\n\treturn \"\"\n}\n" }, { "path": "pkg/plugin/framework/itemblockaction/v1/item_block_action_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\t\"encoding/json\"\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n\tprotoibav1 \"github.com/vmware-tanzu/velero/pkg/plugin/generated/itemblockaction/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tmocks \"github.com/vmware-tanzu/velero/pkg/plugin/velero/mocks/itemblockaction/v1\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestItemBlockActionGRPCServerGetRelatedItems(t *testing.T) {\n\tinvalidItem := []byte(\"this is gibberish json\")\n\tvalidItem := []byte(`\n\t{\n\t\t\"apiVersion\": \"v1\",\n\t\t\"kind\": \"ConfigMap\",\n\t\t\"metadata\": {\n\t\t\t\"namespace\": \"myns\",\n\t\t\t\"name\": \"myconfigmap\"\n\t\t},\n\t\t\"data\": {\n\t\t\t\"key\": \"value\"\n\t\t}\n\t}`)\n\tvar validItemObject unstructured.Unstructured\n\terr := json.Unmarshal(validItem, &validItemObject)\n\trequire.NoError(t, err)\n\n\tinvalidBackup := []byte(\"this is gibberish json\")\n\tvalidBackup := []byte(`\n\t{\n\t\t\"apiVersion\": \"velero.io/v1\",\n\t\t\"kind\": \"Backup\",\n\t\t\"metadata\": {\n\t\t\t\"namespace\": \"myns\",\n\t\t\t\"name\": \"mybackup\"\n\t\t},\n\t\t\"spec\": {\n\t\t\t\"includedNamespaces\": [\"*\"],\n\t\t\t\"includedResources\": [\"*\"],\n\t\t\t\"ttl\": \"60m\"\n\t\t}\n\t}`)\n\tvar validBackupObject v1.Backup\n\terr = json.Unmarshal(validBackup, &validBackupObject)\n\trequire.NoError(t, err)\n\n\ttests := []struct {\n\t\tname string\n\t\tbackup []byte\n\t\titem []byte\n\t\timplRelatedItems []velero.ResourceIdentifier\n\t\timplError error\n\t\texpectError bool\n\t\tskipMock bool\n\t}{\n\t\t{\n\t\t\tname: \"error unmarshaling item\",\n\t\t\titem: invalidItem,\n\t\t\tbackup: validBackup,\n\t\t\texpectError: true,\n\t\t\tskipMock: true,\n\t\t},\n\t\t{\n\t\t\tname: \"error unmarshaling backup\",\n\t\t\titem: validItem,\n\t\t\tbackup: invalidBackup,\n\t\t\texpectError: true,\n\t\t\tskipMock: true,\n\t\t},\n\t\t{\n\t\t\tname: \"error running impl\",\n\t\t\titem: validItem,\n\t\t\tbackup: validBackup,\n\t\t\timplError: errors.New(\"impl error\"),\n\t\t\texpectError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"no relatedItems\",\n\t\t\titem: validItem,\n\t\t\tbackup: validBackup,\n\t\t},\n\t\t{\n\t\t\tname: \"some relatedItems\",\n\t\t\titem: validItem,\n\t\t\tbackup: validBackup,\n\t\t\timplRelatedItems: []velero.ResourceIdentifier{\n\t\t\t\t{\n\t\t\t\t\tGroupResource: schema.GroupResource{Group: \"v1\", Resource: \"pods\"},\n\t\t\t\t\tNamespace: \"myns\",\n\t\t\t\t\tName: \"mypod\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\titemAction := &mocks.ItemBlockAction{}\n\t\t\tdefer itemAction.AssertExpectations(t)\n\n\t\t\tif !test.skipMock {\n\t\t\t\titemAction.On(\"GetRelatedItems\", &validItemObject, &validBackupObject).Return(test.implRelatedItems, test.implError)\n\t\t\t}\n\n\t\t\ts := &ItemBlockActionGRPCServer{mux: &common.ServerMux{\n\t\t\t\tServerLog: velerotest.NewLogger(),\n\t\t\t\tHandlers: map[string]any{\n\t\t\t\t\t\"xyz\": itemAction,\n\t\t\t\t},\n\t\t\t}}\n\n\t\t\treq := &protoibav1.ItemBlockActionGetRelatedItemsRequest{\n\t\t\t\tPlugin: \"xyz\",\n\t\t\t\tItem: test.item,\n\t\t\t\tBackup: test.backup,\n\t\t\t}\n\n\t\t\tresp, err := s.GetRelatedItems(t.Context(), req)\n\n\t\t\t// Verify error\n\t\t\tassert.Equal(t, test.expectError, err != nil)\n\t\t\tif err != nil {\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NotNil(t, resp)\n\n\t\t\t// Verify related items\n\t\t\tvar expectedRelatedItems []*proto.ResourceIdentifier\n\t\t\tfor _, item := range test.implRelatedItems {\n\t\t\t\texpectedRelatedItems = append(expectedRelatedItems, backupResourceIdentifierToProto(item))\n\t\t\t}\n\t\t\tassert.Equal(t, expectedRelatedItems, resp.RelatedItems)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/plugin/framework/logger.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n)\n\n// newLogger returns a logger that is suitable for use within an\n// Velero plugin.\nfunc newLogger() *logrus.Logger {\n\tlogger := logrus.New()\n\t/*\n\t\t!!!DO NOT SET THE OUTPUT TO STDOUT!!!\n\n\t\tgo-plugin uses stdout for a communications protocol between client and server.\n\n\t\tstderr is used for log messages from server to client. The velero server makes sure they are logged to stdout.\n\t*/\n\n\t// we use the JSON formatter because go-plugin will parse incoming\n\t// JSON on stderr and use it to create structured log entries.\n\tlogger.Formatter = &logrus.JSONFormatter{\n\t\tFieldMap: logrus.FieldMap{\n\t\t\t// this is the hclog-compatible message field\n\t\t\tlogrus.FieldKeyMsg: \"@message\",\n\t\t},\n\t\t// Velero server already adds timestamps when emitting logs, so\n\t\t// don't do it within the plugin.\n\t\tDisableTimestamp: true,\n\t}\n\n\t// set a logger name for the location hook which will signal to the Velero\n\t// server logger that the location has been set within a hook.\n\tlogger.Hooks.Add((&logging.LogLocationHook{}).WithLoggerName(\"plugin\"))\n\n\t// make sure we attempt to record the error location\n\tlogger.Hooks.Add(&logging.ErrorLocationHook{})\n\n\t// this hook adjusts the string representation of WarnLevel to \"warn\"\n\t// rather than \"warning\" to make it parseable by go-plugin within the\n\t// Velero server code\n\tlogger.Hooks.Add(&logging.HcLogLevelHook{})\n\n\treturn logger\n}\n" }, { "path": "pkg/plugin/framework/logger_test.go", "content": "/*\nCopyright 2018, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage framework\n\nimport (\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n)\n\nfunc TestNewLogger(t *testing.T) {\n\tl := newLogger()\n\n\texpectedFormatter := &logrus.JSONFormatter{\n\t\tFieldMap: logrus.FieldMap{\n\t\t\tlogrus.FieldKeyMsg: \"@message\",\n\t\t},\n\t\tDisableTimestamp: true,\n\t}\n\tassert.Equal(t, expectedFormatter, l.Formatter)\n\n\texpectedHooks := []logrus.Hook{\n\t\t(&logging.LogLocationHook{}).WithLoggerName(\"plugin\"),\n\t\t&logging.ErrorLocationHook{},\n\t\t&logging.HcLogLevelHook{},\n\t}\n\n\tfor _, level := range logrus.AllLevels {\n\t\tassert.Equal(t, expectedHooks, l.Hooks[level])\n\t}\n}\n" }, { "path": "pkg/plugin/framework/object_store.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"context\"\n\n\tplugin \"github.com/hashicorp/go-plugin\"\n\t\"google.golang.org/grpc\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n)\n\n// ObjectStorePlugin is an implementation of go-plugin's Plugin\n// interface with support for gRPC for the cloudprovider/ObjectStore\n// interface.\ntype ObjectStorePlugin struct {\n\tplugin.NetRPCUnsupportedPlugin\n\t*common.PluginBase\n}\n\n// GRPCClient returns an ObjectStore gRPC client.\nfunc (p *ObjectStorePlugin) GRPCClient(_ context.Context, _ *plugin.GRPCBroker, clientConn *grpc.ClientConn) (any, error) {\n\treturn common.NewClientDispenser(p.ClientLogger, clientConn, newObjectStoreGRPCClient), nil\n}\n\n// GRPCServer registers an ObjectStore gRPC server.\nfunc (p *ObjectStorePlugin) GRPCServer(_ *plugin.GRPCBroker, server *grpc.Server) error {\n\tproto.RegisterObjectStoreServer(server, &ObjectStoreGRPCServer{mux: p.ServerMux})\n\treturn nil\n}\n" }, { "path": "pkg/plugin/framework/object_store_client.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"io\"\n\t\"time\"\n\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"google.golang.org/grpc\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n)\n\nconst byteChunkSize = 16384\n\n// NewObjectStorePlugin construct an ObjectStorePlugin.\nfunc NewObjectStorePlugin(options ...common.PluginOption) *ObjectStorePlugin {\n\treturn &ObjectStorePlugin{\n\t\tPluginBase: common.NewPluginBase(options...),\n\t}\n}\n\n// ObjectStoreGRPCClient implements the ObjectStore interface and uses a\n// gRPC client to make calls to the plugin server.\ntype ObjectStoreGRPCClient struct {\n\t*common.ClientBase\n\tgrpcClient proto.ObjectStoreClient\n}\n\nfunc newObjectStoreGRPCClient(base *common.ClientBase, clientConn *grpc.ClientConn) any {\n\treturn &ObjectStoreGRPCClient{\n\t\tClientBase: base,\n\t\tgrpcClient: proto.NewObjectStoreClient(clientConn),\n\t}\n}\n\n// Init prepares the ObjectStore for usage using the provided map of\n// configuration key-value pairs. It returns an error if the ObjectStore\n// cannot be initialized from the provided config.\nfunc (c *ObjectStoreGRPCClient) Init(config map[string]string) error {\n\treq := &proto.ObjectStoreInitRequest{\n\t\tPlugin: c.Plugin,\n\t\tConfig: config,\n\t}\n\n\tif _, err := c.grpcClient.Init(context.Background(), req); err != nil {\n\t\treturn common.FromGRPCError(err)\n\t}\n\n\treturn nil\n}\n\n// PutObject creates a new object using the data in body within the specified\n// object storage bucket with the given key.\nfunc (c *ObjectStoreGRPCClient) PutObject(bucket, key string, body io.Reader) error {\n\tstream, err := c.grpcClient.PutObject(context.Background())\n\tif err != nil {\n\t\treturn common.FromGRPCError(err)\n\t}\n\n\t// read from the provider io.Reader into chunks, and send each one over\n\t// the gRPC stream\n\tchunk := make([]byte, byteChunkSize)\n\tfor {\n\t\tn, err := body.Read(chunk)\n\t\tif err == io.EOF {\n\t\t\tif _, resErr := stream.CloseAndRecv(); resErr != nil {\n\t\t\t\treturn common.FromGRPCError(resErr)\n\t\t\t}\n\t\t\treturn nil\n\t\t}\n\t\tif err != nil {\n\t\t\tif err := stream.CloseSend(); err != nil {\n\t\t\t\treturn common.FromGRPCError(err)\n\t\t\t}\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\n\t\tif err := stream.Send(&proto.PutObjectRequest{Plugin: c.Plugin, Bucket: bucket, Key: key, Body: chunk[0:n]}); err != nil {\n\t\t\treturn common.FromGRPCError(err)\n\t\t}\n\t}\n}\n\n// ObjectExists checks if there is an object with the given key in the object storage bucket.\nfunc (c *ObjectStoreGRPCClient) ObjectExists(bucket, key string) (bool, error) {\n\treq := &proto.ObjectExistsRequest{\n\t\tPlugin: c.Plugin,\n\t\tBucket: bucket,\n\t\tKey: key,\n\t}\n\n\tres, err := c.grpcClient.ObjectExists(context.Background(), req)\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\treturn res.Exists, nil\n}\n\n// GetObject retrieves the object with the given key from the specified\n// bucket in object storage.\nfunc (c *ObjectStoreGRPCClient) GetObject(bucket, key string) (io.ReadCloser, error) {\n\treq := &proto.GetObjectRequest{\n\t\tPlugin: c.Plugin,\n\t\tBucket: bucket,\n\t\tKey: key,\n\t}\n\n\tstream, err := c.grpcClient.GetObject(context.Background(), req)\n\tif err != nil {\n\t\treturn nil, common.FromGRPCError(err)\n\t}\n\n\treceive := func() ([]byte, error) {\n\t\tdata, err := stream.Recv()\n\t\tif err == io.EOF {\n\t\t\t// we need to return io.EOF errors unwrapped so that\n\t\t\t// calling code sees them as io.EOF and knows to stop\n\t\t\t// reading.\n\t\t\treturn nil, err\n\t\t}\n\t\tif err != nil {\n\t\t\treturn nil, common.FromGRPCError(err)\n\t\t}\n\n\t\treturn data.Data, nil\n\t}\n\n\tclose := func() error {\n\t\tif err := stream.CloseSend(); err != nil {\n\t\t\treturn common.FromGRPCError(err)\n\t\t}\n\t\treturn nil\n\t}\n\n\treturn &StreamReadCloser{receive: receive, close: close}, nil\n}\n\n// ListCommonPrefixes gets a list of all object key prefixes that come\n// after the provided prefix and before the provided delimiter (this is\n// often used to simulate a directory hierarchy in object storage).\nfunc (c *ObjectStoreGRPCClient) ListCommonPrefixes(bucket, prefix, delimiter string) ([]string, error) {\n\treq := &proto.ListCommonPrefixesRequest{\n\t\tPlugin: c.Plugin,\n\t\tBucket: bucket,\n\t\tPrefix: prefix,\n\t\tDelimiter: delimiter,\n\t}\n\n\tres, err := c.grpcClient.ListCommonPrefixes(context.Background(), req)\n\tif err != nil {\n\t\treturn nil, common.FromGRPCError(err)\n\t}\n\n\treturn res.Prefixes, nil\n}\n\n// ListObjects gets a list of all objects in bucket that have the same prefix.\nfunc (c *ObjectStoreGRPCClient) ListObjects(bucket, prefix string) ([]string, error) {\n\treq := &proto.ListObjectsRequest{\n\t\tPlugin: c.Plugin,\n\t\tBucket: bucket,\n\t\tPrefix: prefix,\n\t}\n\n\tres, err := c.grpcClient.ListObjects(context.Background(), req)\n\tif err != nil {\n\t\treturn nil, common.FromGRPCError(err)\n\t}\n\n\treturn res.Keys, nil\n}\n\n// DeleteObject removes object with the specified key from the given\n// bucket.\nfunc (c *ObjectStoreGRPCClient) DeleteObject(bucket, key string) error {\n\treq := &proto.DeleteObjectRequest{\n\t\tPlugin: c.Plugin,\n\t\tBucket: bucket,\n\t\tKey: key,\n\t}\n\n\tif _, err := c.grpcClient.DeleteObject(context.Background(), req); err != nil {\n\t\treturn common.FromGRPCError(err)\n\t}\n\n\treturn nil\n}\n\n// CreateSignedURL creates a pre-signed URL for the given bucket and key that expires after ttl.\nfunc (c *ObjectStoreGRPCClient) CreateSignedURL(bucket, key string, ttl time.Duration) (string, error) {\n\treq := &proto.CreateSignedURLRequest{\n\t\tPlugin: c.Plugin,\n\t\tBucket: bucket,\n\t\tKey: key,\n\t\tTtl: int64(ttl),\n\t}\n\n\tres, err := c.grpcClient.CreateSignedURL(context.Background(), req)\n\tif err != nil {\n\t\treturn \"\", common.FromGRPCError(err)\n\t}\n\n\treturn res.Url, nil\n}\n" }, { "path": "pkg/plugin/framework/object_store_server.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"io\"\n\t\"time\"\n\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// ObjectStoreGRPCServer implements the proto-generated ObjectStoreServer interface, and accepts\n// gRPC calls and forwards them to an implementation of the pluggable interface.\ntype ObjectStoreGRPCServer struct {\n\tmux *common.ServerMux\n}\n\nfunc (s *ObjectStoreGRPCServer) getImpl(name string) (velero.ObjectStore, error) {\n\timpl, err := s.mux.GetHandler(name)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\titemAction, ok := impl.(velero.ObjectStore)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"%T is not an object store\", impl)\n\t}\n\n\treturn itemAction, nil\n}\n\n// Init prepares the ObjectStore for usage using the provided map of\n// configuration key-value pairs. It returns an error if the ObjectStore\n// cannot be initialized from the provided config.\nfunc (s *ObjectStoreGRPCServer) Init(ctx context.Context, req *proto.ObjectStoreInitRequest) (response *proto.Empty, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tif err := impl.Init(req.Config); err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &proto.Empty{}, nil\n}\n\n// PutObject creates a new object using the data in body within the specified\n// object storage bucket with the given key.\nfunc (s *ObjectStoreGRPCServer) PutObject(stream proto.ObjectStore_PutObjectServer) (err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\t// we need to read the first chunk ahead of time to get the bucket and key;\n\t// in our receive method, we'll use `first` on the first call\n\tfirstChunk, err := stream.Recv()\n\tif err != nil {\n\t\treturn common.NewGRPCError(errors.WithStack(err))\n\t}\n\n\timpl, err := s.getImpl(firstChunk.Plugin)\n\tif err != nil {\n\t\treturn common.NewGRPCError(err)\n\t}\n\n\tbucket := firstChunk.Bucket\n\tkey := firstChunk.Key\n\n\treceive := func() ([]byte, error) {\n\t\tif firstChunk != nil {\n\t\t\tres := firstChunk.Body\n\t\t\tfirstChunk = nil\n\t\t\treturn res, nil\n\t\t}\n\n\t\tdata, err := stream.Recv()\n\t\tif err == io.EOF {\n\t\t\t// we need to return io.EOF errors unwrapped so that\n\t\t\t// calling code sees them as io.EOF and knows to stop\n\t\t\t// reading.\n\t\t\treturn nil, err\n\t\t}\n\t\tif err != nil {\n\t\t\treturn nil, errors.WithStack(err)\n\t\t}\n\t\treturn data.Body, nil\n\t}\n\n\tclose := func() error {\n\t\treturn nil\n\t}\n\n\tif err := impl.PutObject(bucket, key, &StreamReadCloser{receive: receive, close: close}); err != nil {\n\t\treturn common.NewGRPCError(err)\n\t}\n\n\tif err := stream.SendAndClose(&proto.Empty{}); err != nil {\n\t\treturn common.NewGRPCError(errors.WithStack(err))\n\t}\n\n\treturn nil\n}\n\n// ObjectExists checks if there is an object with the given key in the object storage bucket.\nfunc (s *ObjectStoreGRPCServer) ObjectExists(ctx context.Context, req *proto.ObjectExistsRequest) (response *proto.ObjectExistsResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\texists, err := impl.ObjectExists(req.Bucket, req.Key)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &proto.ObjectExistsResponse{Exists: exists}, nil\n}\n\n// GetObject retrieves the object with the given key from the specified\n// bucket in object storage.\nfunc (s *ObjectStoreGRPCServer) GetObject(req *proto.GetObjectRequest, stream proto.ObjectStore_GetObjectServer) (err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn common.NewGRPCError(err)\n\t}\n\n\trdr, err := impl.GetObject(req.Bucket, req.Key)\n\tif err != nil {\n\t\treturn common.NewGRPCError(err)\n\t}\n\tdefer rdr.Close()\n\n\tchunk := make([]byte, byteChunkSize)\n\tfor {\n\t\tn, err := rdr.Read(chunk)\n\t\tif err != nil && err != io.EOF {\n\t\t\treturn common.NewGRPCError(errors.WithStack(err))\n\t\t}\n\t\tif n == 0 {\n\t\t\treturn nil\n\t\t}\n\n\t\tif err := stream.Send(&proto.Bytes{Data: chunk[0:n]}); err != nil {\n\t\t\treturn common.NewGRPCError(errors.WithStack(err))\n\t\t}\n\t}\n}\n\n// ListCommonPrefixes gets a list of all object key prefixes that start with\n// the specified prefix and stop at the next instance of the provided delimiter\n// (this is often used to simulate a directory hierarchy in object storage).\nfunc (s *ObjectStoreGRPCServer) ListCommonPrefixes(ctx context.Context, req *proto.ListCommonPrefixesRequest) (response *proto.ListCommonPrefixesResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tprefixes, err := impl.ListCommonPrefixes(req.Bucket, req.Prefix, req.Delimiter)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &proto.ListCommonPrefixesResponse{Prefixes: prefixes}, nil\n}\n\n// ListObjects gets a list of all objects in bucket that have the same prefix.\nfunc (s *ObjectStoreGRPCServer) ListObjects(ctx context.Context, req *proto.ListObjectsRequest) (response *proto.ListObjectsResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tkeys, err := impl.ListObjects(req.Bucket, req.Prefix)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &proto.ListObjectsResponse{Keys: keys}, nil\n}\n\n// DeleteObject removes object with the specified key from the given\n// bucket.\nfunc (s *ObjectStoreGRPCServer) DeleteObject(ctx context.Context, req *proto.DeleteObjectRequest) (response *proto.Empty, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tif err := impl.DeleteObject(req.Bucket, req.Key); err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &proto.Empty{}, nil\n}\n\n// CreateSignedURL creates a pre-signed URL for the given bucket and key that expires after ttl.\nfunc (s *ObjectStoreGRPCServer) CreateSignedURL(ctx context.Context, req *proto.CreateSignedURLRequest) (response *proto.CreateSignedURLResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\turl, err := impl.CreateSignedURL(req.Bucket, req.Key, time.Duration(req.Ttl))\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &proto.CreateSignedURLResponse{Url: url}, nil\n}\n" }, { "path": "pkg/plugin/framework/plugin_lister.go", "content": "/*\nCopyright 2018, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"context\"\n\n\tplugin \"github.com/hashicorp/go-plugin\"\n\t\"github.com/pkg/errors\"\n\t\"google.golang.org/grpc\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n)\n\n// PluginIdentifier uniquely identifies a plugin by command, kind, and name.\ntype PluginIdentifier struct {\n\tCommand string\n\tKind common.PluginKind\n\tName string\n}\n\n// PluginLister lists plugins.\ntype PluginLister interface {\n\tListPlugins() ([]PluginIdentifier, error)\n}\n\n// pluginLister implements PluginLister.\ntype pluginLister struct {\n\tplugins []PluginIdentifier\n}\n\n// NewPluginLister returns a new PluginLister for plugins.\nfunc NewPluginLister(plugins ...PluginIdentifier) PluginLister {\n\treturn &pluginLister{plugins: plugins}\n}\n\n// ListPlugins returns the pluginLister's plugins.\nfunc (pl *pluginLister) ListPlugins() ([]PluginIdentifier, error) {\n\treturn pl.plugins, nil\n}\n\n// PluginListerPlugin is a go-plugin Plugin for a PluginLister.\ntype PluginListerPlugin struct {\n\tplugin.NetRPCUnsupportedPlugin\n\timpl PluginLister\n}\n\n// NewPluginListerPlugin creates a new PluginListerPlugin with impl as the server-side implementation.\nfunc NewPluginListerPlugin(impl PluginLister) *PluginListerPlugin {\n\treturn &PluginListerPlugin{impl: impl}\n}\n\n//////////////////////////////////////////////////////////////////////////////\n// client code\n//////////////////////////////////////////////////////////////////////////////\n\n// GRPCClient returns a PluginLister gRPC client.\nfunc (p *PluginListerPlugin) GRPCClient(_ context.Context, _ *plugin.GRPCBroker, clientConn *grpc.ClientConn) (any, error) {\n\treturn &PluginListerGRPCClient{grpcClient: proto.NewPluginListerClient(clientConn)}, nil\n}\n\n// PluginListerGRPCClient implements PluginLister and uses a gRPC client to make calls to the plugin server.\ntype PluginListerGRPCClient struct {\n\tgrpcClient proto.PluginListerClient\n}\n\n// ListPlugins uses the gRPC client to request the list of plugins from the server. It translates the protobuf response\n// to []PluginIdentifier.\nfunc (c *PluginListerGRPCClient) ListPlugins() ([]PluginIdentifier, error) {\n\tresp, err := c.grpcClient.ListPlugins(context.Background(), &proto.Empty{})\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tret := make([]PluginIdentifier, len(resp.Plugins))\n\tfor i, id := range resp.Plugins {\n\t\tif _, ok := common.AllPluginKinds()[id.Kind]; !ok {\n\t\t\treturn nil, errors.Errorf(\"invalid plugin kind: %s\", id.Kind)\n\t\t}\n\n\t\tret[i] = PluginIdentifier{\n\t\t\tCommand: id.Command,\n\t\t\tKind: common.PluginKind(id.Kind),\n\t\t\tName: id.Name,\n\t\t}\n\t}\n\n\treturn ret, nil\n}\n\n//////////////////////////////////////////////////////////////////////////////\n// server code\n//////////////////////////////////////////////////////////////////////////////\n\n// GRPCServer registers a PluginLister gRPC server.\nfunc (p *PluginListerPlugin) GRPCServer(_ *plugin.GRPCBroker, server *grpc.Server) error {\n\tproto.RegisterPluginListerServer(server, &PluginListerGRPCServer{impl: p.impl})\n\treturn nil\n}\n\n// PluginListerGRPCServer implements the proto-generated PluginLister gRPC service interface. It accepts gRPC calls,\n// forwards them to impl, and translates the responses to protobuf.\ntype PluginListerGRPCServer struct {\n\timpl PluginLister\n}\n\n// ListPlugins returns a list of registered plugins, delegating to s.impl to perform the listing.\nfunc (s *PluginListerGRPCServer) ListPlugins(ctx context.Context, req *proto.Empty) (*proto.ListPluginsResponse, error) {\n\tlist, err := s.impl.ListPlugins()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tplugins := make([]*proto.PluginIdentifier, len(list))\n\tfor i, id := range list {\n\t\tif _, ok := common.AllPluginKinds()[id.Kind.String()]; !ok {\n\t\t\treturn nil, errors.Errorf(\"invalid plugin kind: %s\", id.Kind)\n\t\t}\n\n\t\tplugins[i] = &proto.PluginIdentifier{\n\t\t\tCommand: id.Command,\n\t\t\tKind: id.Kind.String(),\n\t\t\tName: id.Name,\n\t\t}\n\t}\n\tret := &proto.ListPluginsResponse{\n\t\tPlugins: plugins,\n\t}\n\treturn ret, nil\n}\n" }, { "path": "pkg/plugin/framework/plugin_types_test.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"testing\"\n\n\tplugin \"github.com/hashicorp/go-plugin\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestPluginImplementationsAreGRPCPlugins(t *testing.T) {\n\tpluginImpls := []any{\n\t\tnew(VolumeSnapshotterPlugin),\n\t\tnew(BackupItemActionPlugin),\n\t\tnew(ObjectStorePlugin),\n\t\tnew(PluginListerPlugin),\n\t\tnew(RestoreItemActionPlugin),\n\t}\n\n\tfor _, impl := range pluginImpls {\n\t\t_, ok := impl.(plugin.GRPCPlugin)\n\t\tassert.True(t, ok, \"plugin implementation %T does not implement the go-plugin.GRPCPlugin interface\", impl)\n\t}\n}\n" }, { "path": "pkg/plugin/framework/restore_item_action.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"context\"\n\n\tplugin \"github.com/hashicorp/go-plugin\"\n\t\"google.golang.org/grpc\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n)\n\n// RestoreItemActionPlugin is an implementation of go-plugin's Plugin\n// interface with support for gRPC for the restore/ItemAction\n// interface.\ntype RestoreItemActionPlugin struct {\n\tplugin.NetRPCUnsupportedPlugin\n\t*common.PluginBase\n}\n\n// GRPCClient returns a RestoreItemAction gRPC client.\nfunc (p *RestoreItemActionPlugin) GRPCClient(_ context.Context, _ *plugin.GRPCBroker, clientConn *grpc.ClientConn) (any, error) {\n\treturn common.NewClientDispenser(p.ClientLogger, clientConn, newRestoreItemActionGRPCClient), nil\n}\n\n// GRPCServer registers a RestoreItemAction gRPC server.\nfunc (p *RestoreItemActionPlugin) GRPCServer(_ *plugin.GRPCBroker, server *grpc.Server) error {\n\tproto.RegisterRestoreItemActionServer(server, &RestoreItemActionGRPCServer{mux: p.ServerMux})\n\treturn nil\n}\n" }, { "path": "pkg/plugin/framework/restore_item_action_client.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"encoding/json\"\n\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"google.golang.org/grpc\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\triav1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/restoreitemaction/v1\"\n)\n\nvar _ riav1.RestoreItemAction = &RestoreItemActionGRPCClient{}\n\n// NewRestoreItemActionPlugin constructs a RestoreItemActionPlugin.\nfunc NewRestoreItemActionPlugin(options ...common.PluginOption) *RestoreItemActionPlugin {\n\treturn &RestoreItemActionPlugin{\n\t\tPluginBase: common.NewPluginBase(options...),\n\t}\n}\n\n// RestoreItemActionGRPCClient implements the backup/ItemAction interface and uses a\n// gRPC client to make calls to the plugin server.\ntype RestoreItemActionGRPCClient struct {\n\t*common.ClientBase\n\tgrpcClient proto.RestoreItemActionClient\n}\n\nfunc newRestoreItemActionGRPCClient(base *common.ClientBase, clientConn *grpc.ClientConn) any {\n\treturn &RestoreItemActionGRPCClient{\n\t\tClientBase: base,\n\t\tgrpcClient: proto.NewRestoreItemActionClient(clientConn),\n\t}\n}\n\nfunc (c *RestoreItemActionGRPCClient) AppliesTo() (velero.ResourceSelector, error) {\n\tres, err := c.grpcClient.AppliesTo(context.Background(), &proto.RestoreItemActionAppliesToRequest{Plugin: c.Plugin})\n\tif err != nil {\n\t\treturn velero.ResourceSelector{}, common.FromGRPCError(err)\n\t}\n\n\tif res.ResourceSelector == nil {\n\t\treturn velero.ResourceSelector{}, nil\n\t}\n\n\treturn velero.ResourceSelector{\n\t\tIncludedNamespaces: res.ResourceSelector.IncludedNamespaces,\n\t\tExcludedNamespaces: res.ResourceSelector.ExcludedNamespaces,\n\t\tIncludedResources: res.ResourceSelector.IncludedResources,\n\t\tExcludedResources: res.ResourceSelector.ExcludedResources,\n\t\tLabelSelector: res.ResourceSelector.Selector,\n\t}, nil\n}\n\nfunc (c *RestoreItemActionGRPCClient) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\titemJSON, err := json.Marshal(input.Item.UnstructuredContent())\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\titemFromBackupJSON, err := json.Marshal(input.ItemFromBackup.UnstructuredContent())\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\trestoreJSON, err := json.Marshal(input.Restore)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\treq := &proto.RestoreItemActionExecuteRequest{\n\t\tPlugin: c.Plugin,\n\t\tItem: itemJSON,\n\t\tItemFromBackup: itemFromBackupJSON,\n\t\tRestore: restoreJSON,\n\t}\n\n\tres, err := c.grpcClient.Execute(context.Background(), req)\n\tif err != nil {\n\t\treturn nil, common.FromGRPCError(err)\n\t}\n\n\tvar updatedItem unstructured.Unstructured\n\tif err := json.Unmarshal(res.Item, &updatedItem); err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tvar additionalItems []velero.ResourceIdentifier\n\tfor _, itm := range res.AdditionalItems {\n\t\tnewItem := velero.ResourceIdentifier{\n\t\t\tGroupResource: schema.GroupResource{\n\t\t\t\tGroup: itm.Group,\n\t\t\t\tResource: itm.Resource,\n\t\t\t},\n\t\t\tNamespace: itm.Namespace,\n\t\t\tName: itm.Name,\n\t\t}\n\n\t\tadditionalItems = append(additionalItems, newItem)\n\t}\n\n\treturn &velero.RestoreItemActionExecuteOutput{\n\t\tUpdatedItem: &updatedItem,\n\t\tAdditionalItems: additionalItems,\n\t\tSkipRestore: res.SkipRestore,\n\t}, nil\n}\n" }, { "path": "pkg/plugin/framework/restore_item_action_server.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"encoding/json\"\n\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\triav1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/restoreitemaction/v1\"\n)\n\n// RestoreItemActionGRPCServer implements the proto-generated RestoreItemActionServer interface, and accepts\n// gRPC calls and forwards them to an implementation of the pluggable interface.\ntype RestoreItemActionGRPCServer struct {\n\tmux *common.ServerMux\n}\n\nfunc (s *RestoreItemActionGRPCServer) getImpl(name string) (riav1.RestoreItemAction, error) {\n\timpl, err := s.mux.GetHandler(name)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\titemAction, ok := impl.(riav1.RestoreItemAction)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"%T is not a restore item action\", impl)\n\t}\n\n\treturn itemAction, nil\n}\n\nfunc (s *RestoreItemActionGRPCServer) AppliesTo(ctx context.Context, req *proto.RestoreItemActionAppliesToRequest) (response *proto.RestoreItemActionAppliesToResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tresourceSelector, err := impl.AppliesTo()\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &proto.RestoreItemActionAppliesToResponse{\n\t\tResourceSelector: &proto.ResourceSelector{\n\t\t\tIncludedNamespaces: resourceSelector.IncludedNamespaces,\n\t\t\tExcludedNamespaces: resourceSelector.ExcludedNamespaces,\n\t\t\tIncludedResources: resourceSelector.IncludedResources,\n\t\t\tExcludedResources: resourceSelector.ExcludedResources,\n\t\t\tSelector: resourceSelector.LabelSelector,\n\t\t},\n\t}, nil\n}\n\nfunc (s *RestoreItemActionGRPCServer) Execute(ctx context.Context, req *proto.RestoreItemActionExecuteRequest) (response *proto.RestoreItemActionExecuteResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tvar (\n\t\titem unstructured.Unstructured\n\t\titemFromBackup unstructured.Unstructured\n\t\trestoreObj api.Restore\n\t)\n\n\tif err := json.Unmarshal(req.Item, &item); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\n\tif err := json.Unmarshal(req.ItemFromBackup, &itemFromBackup); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\n\tif err := json.Unmarshal(req.Restore, &restoreObj); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\n\texecuteOutput, err := impl.Execute(&velero.RestoreItemActionExecuteInput{\n\t\tItem: &item,\n\t\tItemFromBackup: &itemFromBackup,\n\t\tRestore: &restoreObj,\n\t})\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\t// If the plugin implementation returned a nil updateItem (meaning no modifications), reset updatedItem to the\n\t// original item.\n\tvar updatedItemJSON []byte\n\tif executeOutput.UpdatedItem == nil {\n\t\tupdatedItemJSON = req.Item\n\t} else {\n\t\tupdatedItemJSON, err = json.Marshal(executeOutput.UpdatedItem.UnstructuredContent())\n\t\tif err != nil {\n\t\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t\t}\n\t}\n\n\tres := &proto.RestoreItemActionExecuteResponse{\n\t\tItem: updatedItemJSON,\n\t\tSkipRestore: executeOutput.SkipRestore,\n\t}\n\n\tfor _, item := range executeOutput.AdditionalItems {\n\t\tres.AdditionalItems = append(res.AdditionalItems, restoreResourceIdentifierToProto(item))\n\t}\n\n\treturn res, nil\n}\n\nfunc restoreResourceIdentifierToProto(id velero.ResourceIdentifier) *proto.ResourceIdentifier {\n\treturn &proto.ResourceIdentifier{\n\t\tGroup: id.Group,\n\t\tResource: id.Resource,\n\t\tNamespace: id.Namespace,\n\t\tName: id.Name,\n\t}\n}\n" }, { "path": "pkg/plugin/framework/restoreitemaction/v2/restore_item_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v2\n\nimport (\n\t\"context\"\n\n\tplugin \"github.com/hashicorp/go-plugin\"\n\t\"google.golang.org/grpc\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tprotoriav2 \"github.com/vmware-tanzu/velero/pkg/plugin/generated/restoreitemaction/v2\"\n)\n\n// RestoreItemActionPlugin is an implementation of go-plugin's Plugin\n// interface with support for gRPC for the restore/ItemAction\n// interface.\ntype RestoreItemActionPlugin struct {\n\tplugin.NetRPCUnsupportedPlugin\n\t*common.PluginBase\n}\n\n// GRPCClient returns a RestoreItemAction gRPC client.\nfunc (p *RestoreItemActionPlugin) GRPCClient(_ context.Context, _ *plugin.GRPCBroker, clientConn *grpc.ClientConn) (any, error) {\n\treturn common.NewClientDispenser(p.ClientLogger, clientConn, newRestoreItemActionGRPCClient), nil\n}\n\n// GRPCServer registers a RestoreItemAction gRPC server.\nfunc (p *RestoreItemActionPlugin) GRPCServer(_ *plugin.GRPCBroker, server *grpc.Server) error {\n\tprotoriav2.RegisterRestoreItemActionServer(server, &RestoreItemActionGRPCServer{mux: p.ServerMux})\n\treturn nil\n}\n" }, { "path": "pkg/plugin/framework/restoreitemaction/v2/restore_item_action_client.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v2\n\nimport (\n\t\"encoding/json\"\n\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"google.golang.org/grpc\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tprotoriav2 \"github.com/vmware-tanzu/velero/pkg/plugin/generated/restoreitemaction/v2\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\triav2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/restoreitemaction/v2\"\n)\n\nvar _ riav2.RestoreItemAction = &RestoreItemActionGRPCClient{}\n\n// NewRestoreItemActionPlugin constructs a RestoreItemActionPlugin.\nfunc NewRestoreItemActionPlugin(options ...common.PluginOption) *RestoreItemActionPlugin {\n\treturn &RestoreItemActionPlugin{\n\t\tPluginBase: common.NewPluginBase(options...),\n\t}\n}\n\n// RestoreItemActionGRPCClient implements the backup/ItemAction interface and uses a\n// gRPC client to make calls to the plugin server.\ntype RestoreItemActionGRPCClient struct {\n\t*common.ClientBase\n\tgrpcClient protoriav2.RestoreItemActionClient\n}\n\nfunc newRestoreItemActionGRPCClient(base *common.ClientBase, clientConn *grpc.ClientConn) any {\n\treturn &RestoreItemActionGRPCClient{\n\t\tClientBase: base,\n\t\tgrpcClient: protoriav2.NewRestoreItemActionClient(clientConn),\n\t}\n}\n\nfunc (c *RestoreItemActionGRPCClient) AppliesTo() (velero.ResourceSelector, error) {\n\tres, err := c.grpcClient.AppliesTo(context.Background(), &protoriav2.RestoreItemActionAppliesToRequest{Plugin: c.Plugin})\n\tif err != nil {\n\t\treturn velero.ResourceSelector{}, common.FromGRPCError(err)\n\t}\n\n\tif res.ResourceSelector == nil {\n\t\treturn velero.ResourceSelector{}, nil\n\t}\n\n\treturn velero.ResourceSelector{\n\t\tIncludedNamespaces: res.ResourceSelector.IncludedNamespaces,\n\t\tExcludedNamespaces: res.ResourceSelector.ExcludedNamespaces,\n\t\tIncludedResources: res.ResourceSelector.IncludedResources,\n\t\tExcludedResources: res.ResourceSelector.ExcludedResources,\n\t\tLabelSelector: res.ResourceSelector.Selector,\n\t}, nil\n}\n\nfunc (c *RestoreItemActionGRPCClient) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\titemJSON, err := json.Marshal(input.Item.UnstructuredContent())\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\titemFromBackupJSON, err := json.Marshal(input.ItemFromBackup.UnstructuredContent())\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\trestoreJSON, err := json.Marshal(input.Restore)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\treq := &protoriav2.RestoreItemActionExecuteRequest{\n\t\tPlugin: c.Plugin,\n\t\tItem: itemJSON,\n\t\tItemFromBackup: itemFromBackupJSON,\n\t\tRestore: restoreJSON,\n\t}\n\n\tres, err := c.grpcClient.Execute(context.Background(), req)\n\tif err != nil {\n\t\treturn nil, common.FromGRPCError(err)\n\t}\n\n\tvar updatedItem unstructured.Unstructured\n\tif err := json.Unmarshal(res.Item, &updatedItem); err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tvar additionalItems []velero.ResourceIdentifier\n\tfor _, itm := range res.AdditionalItems {\n\t\tnewItem := velero.ResourceIdentifier{\n\t\t\tGroupResource: schema.GroupResource{\n\t\t\t\tGroup: itm.Group,\n\t\t\t\tResource: itm.Resource,\n\t\t\t},\n\t\t\tNamespace: itm.Namespace,\n\t\t\tName: itm.Name,\n\t\t}\n\n\t\tadditionalItems = append(additionalItems, newItem)\n\t}\n\n\treturn &velero.RestoreItemActionExecuteOutput{\n\t\tUpdatedItem: &updatedItem,\n\t\tAdditionalItems: additionalItems,\n\t\tSkipRestore: res.SkipRestore,\n\t\tOperationID: res.OperationID,\n\t\tWaitForAdditionalItems: res.WaitForAdditionalItems,\n\t\tAdditionalItemsReadyTimeout: res.AdditionalItemsReadyTimeout.AsDuration(),\n\t}, nil\n}\n\nfunc (c *RestoreItemActionGRPCClient) Progress(operationID string, restore *api.Restore) (velero.OperationProgress, error) {\n\trestoreJSON, err := json.Marshal(restore)\n\tif err != nil {\n\t\treturn velero.OperationProgress{}, errors.WithStack(err)\n\t}\n\treq := &protoriav2.RestoreItemActionProgressRequest{\n\t\tPlugin: c.Plugin,\n\t\tOperationID: operationID,\n\t\tRestore: restoreJSON,\n\t}\n\n\tres, err := c.grpcClient.Progress(context.Background(), req)\n\tif err != nil {\n\t\treturn velero.OperationProgress{}, common.FromGRPCError(err)\n\t}\n\n\treturn velero.OperationProgress{\n\t\tCompleted: res.Progress.Completed,\n\t\tErr: res.Progress.Err,\n\t\tNCompleted: res.Progress.NCompleted,\n\t\tNTotal: res.Progress.NTotal,\n\t\tOperationUnits: res.Progress.OperationUnits,\n\t\tDescription: res.Progress.Description,\n\t\tStarted: res.Progress.Started.AsTime(),\n\t\tUpdated: res.Progress.Updated.AsTime(),\n\t}, nil\n}\n\nfunc (c *RestoreItemActionGRPCClient) Cancel(operationID string, restore *api.Restore) error {\n\trestoreJSON, err := json.Marshal(restore)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\treq := &protoriav2.RestoreItemActionCancelRequest{\n\t\tPlugin: c.Plugin,\n\t\tOperationID: operationID,\n\t\tRestore: restoreJSON,\n\t}\n\n\t_, err = c.grpcClient.Cancel(context.Background(), req)\n\tif err != nil {\n\t\treturn common.FromGRPCError(err)\n\t}\n\n\treturn nil\n}\n\nfunc (c *RestoreItemActionGRPCClient) AreAdditionalItemsReady(additionalItems []velero.ResourceIdentifier, restore *api.Restore) (bool, error) {\n\trestoreJSON, err := json.Marshal(restore)\n\tif err != nil {\n\t\treturn false, errors.WithStack(err)\n\t}\n\n\treq := &protoriav2.RestoreItemActionItemsReadyRequest{\n\t\tPlugin: c.Plugin,\n\t\tRestore: restoreJSON,\n\t}\n\tfor _, item := range additionalItems {\n\t\treq.AdditionalItems = append(req.AdditionalItems, restoreResourceIdentifierToProto(item))\n\t}\n\n\tres, err := c.grpcClient.AreAdditionalItemsReady(context.Background(), req)\n\tif err != nil {\n\t\treturn false, common.FromGRPCError(err)\n\t}\n\n\treturn res.Ready, nil\n}\n\n// This shouldn't be called on the GRPC client since the RestartableRestoreItemAction won't delegate\n// this method\nfunc (c *RestoreItemActionGRPCClient) Name() string {\n\treturn \"\"\n}\n" }, { "path": "pkg/plugin/framework/restoreitemaction/v2/restore_item_action_server.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v2\n\nimport (\n\t\"encoding/json\"\n\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"google.golang.org/protobuf/types/known/durationpb\"\n\t\"google.golang.org/protobuf/types/known/emptypb\"\n\t\"google.golang.org/protobuf/types/known/timestamppb\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n\tprotoriav2 \"github.com/vmware-tanzu/velero/pkg/plugin/generated/restoreitemaction/v2\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\triav2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/restoreitemaction/v2\"\n)\n\n// RestoreItemActionGRPCServer implements the proto-generated RestoreItemActionServer interface, and accepts\n// gRPC calls and forwards them to an implementation of the pluggable interface.\ntype RestoreItemActionGRPCServer struct {\n\tmux *common.ServerMux\n}\n\nfunc (s *RestoreItemActionGRPCServer) getImpl(name string) (riav2.RestoreItemAction, error) {\n\timpl, err := s.mux.GetHandler(name)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\titemAction, ok := impl.(riav2.RestoreItemAction)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"%T is not a restore item action (v2)\", impl)\n\t}\n\n\treturn itemAction, nil\n}\n\nfunc (s *RestoreItemActionGRPCServer) AppliesTo(ctx context.Context, req *protoriav2.RestoreItemActionAppliesToRequest) (response *protoriav2.RestoreItemActionAppliesToResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tresourceSelector, err := impl.AppliesTo()\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &protoriav2.RestoreItemActionAppliesToResponse{\n\t\tResourceSelector: &proto.ResourceSelector{\n\t\t\tIncludedNamespaces: resourceSelector.IncludedNamespaces,\n\t\t\tExcludedNamespaces: resourceSelector.ExcludedNamespaces,\n\t\t\tIncludedResources: resourceSelector.IncludedResources,\n\t\t\tExcludedResources: resourceSelector.ExcludedResources,\n\t\t\tSelector: resourceSelector.LabelSelector,\n\t\t},\n\t}, nil\n}\n\nfunc (s *RestoreItemActionGRPCServer) Execute(ctx context.Context, req *protoriav2.RestoreItemActionExecuteRequest) (response *protoriav2.RestoreItemActionExecuteResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tvar (\n\t\titem unstructured.Unstructured\n\t\titemFromBackup unstructured.Unstructured\n\t\trestoreObj api.Restore\n\t)\n\n\tif err := json.Unmarshal(req.Item, &item); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\n\tif err := json.Unmarshal(req.ItemFromBackup, &itemFromBackup); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\n\tif err := json.Unmarshal(req.Restore, &restoreObj); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\n\texecuteOutput, err := impl.Execute(&velero.RestoreItemActionExecuteInput{\n\t\tItem: &item,\n\t\tItemFromBackup: &itemFromBackup,\n\t\tRestore: &restoreObj,\n\t})\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\t// If the plugin implementation returned a nil updateItem (meaning no modifications), reset updatedItem to the\n\t// original item.\n\tvar updatedItemJSON []byte\n\tif executeOutput.UpdatedItem == nil {\n\t\tupdatedItemJSON = req.Item\n\t} else {\n\t\tupdatedItemJSON, err = json.Marshal(executeOutput.UpdatedItem.UnstructuredContent())\n\t\tif err != nil {\n\t\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t\t}\n\t}\n\n\tres := &protoriav2.RestoreItemActionExecuteResponse{\n\t\tItem: updatedItemJSON,\n\t\tSkipRestore: executeOutput.SkipRestore,\n\t\tOperationID: executeOutput.OperationID,\n\t\tWaitForAdditionalItems: executeOutput.WaitForAdditionalItems,\n\t\tAdditionalItemsReadyTimeout: durationpb.New(executeOutput.AdditionalItemsReadyTimeout),\n\t}\n\n\tfor _, item := range executeOutput.AdditionalItems {\n\t\tres.AdditionalItems = append(res.AdditionalItems, restoreResourceIdentifierToProto(item))\n\t}\n\n\treturn res, nil\n}\n\nfunc (s *RestoreItemActionGRPCServer) Progress(ctx context.Context, req *protoriav2.RestoreItemActionProgressRequest) (\n\tresponse *protoriav2.RestoreItemActionProgressResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tvar restore api.Restore\n\tif err := json.Unmarshal(req.Restore, &restore); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\n\tprogress, err := impl.Progress(req.OperationID, &restore)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tres := &protoriav2.RestoreItemActionProgressResponse{\n\t\tProgress: &proto.OperationProgress{\n\t\t\tCompleted: progress.Completed,\n\t\t\tErr: progress.Err,\n\t\t\tNCompleted: progress.NCompleted,\n\t\t\tNTotal: progress.NTotal,\n\t\t\tOperationUnits: progress.OperationUnits,\n\t\t\tDescription: progress.Description,\n\t\t\tStarted: timestamppb.New(progress.Started),\n\t\t\tUpdated: timestamppb.New(progress.Updated),\n\t\t},\n\t}\n\treturn res, nil\n}\n\nfunc (s *RestoreItemActionGRPCServer) Cancel(\n\tctx context.Context, req *protoriav2.RestoreItemActionCancelRequest) (\n\tresponse *emptypb.Empty, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tvar restore api.Restore\n\tif err := json.Unmarshal(req.Restore, &restore); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\n\terr = impl.Cancel(req.OperationID, &restore)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &emptypb.Empty{}, nil\n}\n\nfunc (s *RestoreItemActionGRPCServer) AreAdditionalItemsReady(ctx context.Context, req *protoriav2.RestoreItemActionItemsReadyRequest) (\n\tresponse *protoriav2.RestoreItemActionItemsReadyResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tvar restore api.Restore\n\tif err := json.Unmarshal(req.Restore, &restore); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\tvar additionalItems []velero.ResourceIdentifier\n\tfor _, itm := range req.AdditionalItems {\n\t\tnewItem := velero.ResourceIdentifier{\n\t\t\tGroupResource: schema.GroupResource{\n\t\t\t\tGroup: itm.Group,\n\t\t\t\tResource: itm.Resource,\n\t\t\t},\n\t\t\tNamespace: itm.Namespace,\n\t\t\tName: itm.Name,\n\t\t}\n\n\t\tadditionalItems = append(additionalItems, newItem)\n\t}\n\tready, err := impl.AreAdditionalItemsReady(additionalItems, &restore)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tres := &protoriav2.RestoreItemActionItemsReadyResponse{\n\t\tReady: ready,\n\t}\n\treturn res, nil\n}\n\nfunc restoreResourceIdentifierToProto(id velero.ResourceIdentifier) *proto.ResourceIdentifier {\n\treturn &proto.ResourceIdentifier{\n\t\tGroup: id.Group,\n\t\tResource: id.Resource,\n\t\tNamespace: id.Namespace,\n\t\tName: id.Name,\n\t}\n}\n\n// This shouldn't be called on the GRPC server since the server won't ever receive this request, as\n// the RestartableRestoreItemAction in Velero won't delegate this to the server\nfunc (s *RestoreItemActionGRPCServer) Name() string {\n\treturn \"\"\n}\n" }, { "path": "pkg/plugin/framework/server.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"os\"\n\t\"strings\"\n\n\tplugin \"github.com/hashicorp/go-plugin\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/spf13/pflag\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/server/config\"\n\tbiav2 \"github.com/vmware-tanzu/velero/pkg/plugin/framework/backupitemaction/v2\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tibav1 \"github.com/vmware-tanzu/velero/pkg/plugin/framework/itemblockaction/v1\"\n\triav2 \"github.com/vmware-tanzu/velero/pkg/plugin/framework/restoreitemaction/v2\"\n)\n\n// Server serves registered plugin implementations.\ntype Server interface {\n\t// BindFlags defines the plugin server's command-line flags\n\t// on the provided FlagSet. If you're not sure what flag set\n\t// to use, pflag.CommandLine is the default set of command-line\n\t// flags.\n\t//\n\t// This method must be called prior to calling .Serve().\n\tBindFlags(flags *pflag.FlagSet) Server\n\n\t// GetConfig return the config parsed from the flags\n\tGetConfig() *config.Config\n\n\t// RegisterBackupItemAction registers a backup item action. Accepted format\n\t// for the plugin name is /.\n\tRegisterBackupItemAction(pluginName string, initializer common.HandlerInitializer) Server\n\n\t// RegisterBackupItemActions registers multiple backup item actions.\n\tRegisterBackupItemActions(map[string]common.HandlerInitializer) Server\n\n\t// RegisterBackupItemActionV2 registers a v2 backup item action. Accepted format\n\t// for the plugin name is /.\n\tRegisterBackupItemActionV2(pluginName string, initializer common.HandlerInitializer) Server\n\n\t// RegisterBackupItemActionsV2 registers multiple v2 backup item actions.\n\tRegisterBackupItemActionsV2(map[string]common.HandlerInitializer) Server\n\n\t// RegisterVolumeSnapshotter registers a volume snapshotter. Accepted format\n\t// for the plugin name is /.\n\tRegisterVolumeSnapshotter(pluginName string, initializer common.HandlerInitializer) Server\n\n\t// RegisterVolumeSnapshotters registers multiple volume snapshotters.\n\tRegisterVolumeSnapshotters(map[string]common.HandlerInitializer) Server\n\n\t// RegisterObjectStore registers an object store. Accepted format\n\t// for the plugin name is /.\n\tRegisterObjectStore(pluginName string, initializer common.HandlerInitializer) Server\n\n\t// RegisterObjectStores registers multiple object stores.\n\tRegisterObjectStores(map[string]common.HandlerInitializer) Server\n\n\t// RegisterRestoreItemAction registers a restore item action. Accepted format\n\t// for the plugin name is /.\n\tRegisterRestoreItemAction(pluginName string, initializer common.HandlerInitializer) Server\n\n\t// RegisterRestoreItemActions registers multiple restore item actions.\n\tRegisterRestoreItemActions(map[string]common.HandlerInitializer) Server\n\n\t// RegisterRestoreItemActionV2 registers a v2 restore item action. Accepted format\n\t// for the plugin name is /.\n\tRegisterRestoreItemActionV2(pluginName string, initializer common.HandlerInitializer) Server\n\n\t// RegisterRestoreItemActionsV2 registers multiple v2 restore item actions.\n\tRegisterRestoreItemActionsV2(map[string]common.HandlerInitializer) Server\n\n\t// RegisterDeleteItemAction registers a delete item action. Accepted format\n\t// for the plugin name is /.\n\tRegisterDeleteItemAction(pluginName string, initializer common.HandlerInitializer) Server\n\n\t// RegisterDeleteItemActions registers multiple Delete item actions.\n\tRegisterDeleteItemActions(map[string]common.HandlerInitializer) Server\n\n\t// RegisterItemBlockAction registers a ItemBlock action. Accepted format\n\t// for the plugin name is /.\n\tRegisterItemBlockAction(pluginName string, initializer common.HandlerInitializer) Server\n\n\t// RegisterItemBlockActions registers multiple ItemBlock actions.\n\tRegisterItemBlockActions(map[string]common.HandlerInitializer) Server\n\n\t// Server runs the plugin server.\n\tServe()\n}\n\n// server implements Server.\ntype server struct {\n\tconfig *config.Config\n\tlog *logrus.Logger\n\tflagSet *pflag.FlagSet\n\tbackupItemAction *BackupItemActionPlugin\n\tbackupItemActionV2 *biav2.BackupItemActionPlugin\n\tvolumeSnapshotter *VolumeSnapshotterPlugin\n\tobjectStore *ObjectStorePlugin\n\trestoreItemAction *RestoreItemActionPlugin\n\trestoreItemActionV2 *riav2.RestoreItemActionPlugin\n\tdeleteItemAction *DeleteItemActionPlugin\n\titemBlockAction *ibav1.ItemBlockActionPlugin\n}\n\n// NewServer returns a new Server\nfunc NewServer() Server {\n\tlog := newLogger()\n\n\treturn &server{\n\t\tconfig: config.GetDefaultConfig(),\n\t\tlog: log,\n\t\tbackupItemAction: NewBackupItemActionPlugin(common.ServerLogger(log)),\n\t\tbackupItemActionV2: biav2.NewBackupItemActionPlugin(common.ServerLogger(log)),\n\t\tvolumeSnapshotter: NewVolumeSnapshotterPlugin(common.ServerLogger(log)),\n\t\tobjectStore: NewObjectStorePlugin(common.ServerLogger(log)),\n\t\trestoreItemAction: NewRestoreItemActionPlugin(common.ServerLogger(log)),\n\t\trestoreItemActionV2: riav2.NewRestoreItemActionPlugin(common.ServerLogger(log)),\n\t\tdeleteItemAction: NewDeleteItemActionPlugin(common.ServerLogger(log)),\n\t\titemBlockAction: ibav1.NewItemBlockActionPlugin(common.ServerLogger(log)),\n\t}\n}\n\nfunc (s *server) BindFlags(flags *pflag.FlagSet) Server {\n\ts.flagSet = flags\n\ts.config.BindFlags(flags)\n\ts.flagSet.ParseErrorsWhitelist.UnknownFlags = true // Velero.io word list : ignore\n\treturn s\n}\n\nfunc (s *server) GetConfig() *config.Config {\n\treturn s.config\n}\n\nfunc (s *server) RegisterBackupItemAction(name string, initializer common.HandlerInitializer) Server {\n\ts.backupItemAction.Register(name, initializer)\n\treturn s\n}\n\nfunc (s *server) RegisterBackupItemActions(m map[string]common.HandlerInitializer) Server {\n\tfor name := range m {\n\t\ts.RegisterBackupItemAction(name, m[name])\n\t}\n\treturn s\n}\n\nfunc (s *server) RegisterBackupItemActionV2(name string, initializer common.HandlerInitializer) Server {\n\ts.backupItemActionV2.Register(name, initializer)\n\treturn s\n}\n\nfunc (s *server) RegisterBackupItemActionsV2(m map[string]common.HandlerInitializer) Server {\n\tfor name := range m {\n\t\ts.RegisterBackupItemActionV2(name, m[name])\n\t}\n\treturn s\n}\n\nfunc (s *server) RegisterVolumeSnapshotter(name string, initializer common.HandlerInitializer) Server {\n\ts.volumeSnapshotter.Register(name, initializer)\n\treturn s\n}\n\nfunc (s *server) RegisterVolumeSnapshotters(m map[string]common.HandlerInitializer) Server {\n\tfor name := range m {\n\t\ts.RegisterVolumeSnapshotter(name, m[name])\n\t}\n\treturn s\n}\n\nfunc (s *server) RegisterObjectStore(name string, initializer common.HandlerInitializer) Server {\n\ts.objectStore.Register(name, initializer)\n\treturn s\n}\n\nfunc (s *server) RegisterObjectStores(m map[string]common.HandlerInitializer) Server {\n\tfor name := range m {\n\t\ts.RegisterObjectStore(name, m[name])\n\t}\n\treturn s\n}\n\nfunc (s *server) RegisterRestoreItemAction(name string, initializer common.HandlerInitializer) Server {\n\ts.restoreItemAction.Register(name, initializer)\n\treturn s\n}\n\nfunc (s *server) RegisterRestoreItemActions(m map[string]common.HandlerInitializer) Server {\n\tfor name := range m {\n\t\ts.RegisterRestoreItemAction(name, m[name])\n\t}\n\treturn s\n}\n\nfunc (s *server) RegisterRestoreItemActionV2(name string, initializer common.HandlerInitializer) Server {\n\ts.restoreItemActionV2.Register(name, initializer)\n\treturn s\n}\n\nfunc (s *server) RegisterRestoreItemActionsV2(m map[string]common.HandlerInitializer) Server {\n\tfor name := range m {\n\t\ts.RegisterRestoreItemActionV2(name, m[name])\n\t}\n\treturn s\n}\n\nfunc (s *server) RegisterDeleteItemAction(name string, initializer common.HandlerInitializer) Server {\n\ts.deleteItemAction.Register(name, initializer)\n\treturn s\n}\n\nfunc (s *server) RegisterDeleteItemActions(m map[string]common.HandlerInitializer) Server {\n\tfor name := range m {\n\t\ts.RegisterDeleteItemAction(name, m[name])\n\t}\n\treturn s\n}\n\nfunc (s *server) RegisterItemBlockAction(name string, initializer common.HandlerInitializer) Server {\n\ts.itemBlockAction.Register(name, initializer)\n\treturn s\n}\n\nfunc (s *server) RegisterItemBlockActions(m map[string]common.HandlerInitializer) Server {\n\tfor name := range m {\n\t\ts.RegisterItemBlockAction(name, m[name])\n\t}\n\treturn s\n}\n\n// getNames returns a list of PluginIdentifiers registered with plugin.\nfunc getNames(command string, kind common.PluginKind, plugin Interface) []PluginIdentifier {\n\tvar pluginIdentifiers []PluginIdentifier\n\n\tfor _, name := range plugin.Names() {\n\t\tid := PluginIdentifier{Command: command, Kind: kind, Name: name}\n\t\tpluginIdentifiers = append(pluginIdentifiers, id)\n\t}\n\n\treturn pluginIdentifiers\n}\n\nfunc (s *server) Serve() {\n\tif s.flagSet != nil && !s.flagSet.Parsed() {\n\t\ts.log.Debugf(\"Parsing flags\")\n\t\tif err := s.flagSet.Parse(os.Args[1:]); err != nil {\n\t\t\ts.log.Errorf(\"fail to parse the flags: %s\", err.Error())\n\t\t\treturn\n\t\t}\n\t}\n\n\ts.log.Level = s.config.LogLevel.Parse()\n\ts.log.Debugf(\"Setting log level to %s\", strings.ToUpper(s.log.Level.String()))\n\n\tcommand := os.Args[0]\n\n\tvar pluginIdentifiers []PluginIdentifier\n\tpluginIdentifiers = append(pluginIdentifiers, getNames(command, common.PluginKindBackupItemAction, s.backupItemAction)...)\n\tpluginIdentifiers = append(pluginIdentifiers, getNames(command, common.PluginKindBackupItemActionV2, s.backupItemActionV2)...)\n\tpluginIdentifiers = append(pluginIdentifiers, getNames(command, common.PluginKindVolumeSnapshotter, s.volumeSnapshotter)...)\n\tpluginIdentifiers = append(pluginIdentifiers, getNames(command, common.PluginKindObjectStore, s.objectStore)...)\n\tpluginIdentifiers = append(pluginIdentifiers, getNames(command, common.PluginKindRestoreItemAction, s.restoreItemAction)...)\n\tpluginIdentifiers = append(pluginIdentifiers, getNames(command, common.PluginKindRestoreItemActionV2, s.restoreItemActionV2)...)\n\tpluginIdentifiers = append(pluginIdentifiers, getNames(command, common.PluginKindDeleteItemAction, s.deleteItemAction)...)\n\tpluginIdentifiers = append(pluginIdentifiers, getNames(command, common.PluginKindItemBlockAction, s.itemBlockAction)...)\n\n\tpluginLister := NewPluginLister(pluginIdentifiers...)\n\n\tplugin.Serve(&plugin.ServeConfig{\n\t\tHandshakeConfig: Handshake(),\n\t\tPlugins: map[string]plugin.Plugin{\n\t\t\tstring(common.PluginKindBackupItemAction): s.backupItemAction,\n\t\t\tstring(common.PluginKindBackupItemActionV2): s.backupItemActionV2,\n\t\t\tstring(common.PluginKindVolumeSnapshotter): s.volumeSnapshotter,\n\t\t\tstring(common.PluginKindObjectStore): s.objectStore,\n\t\t\tstring(common.PluginKindPluginLister): NewPluginListerPlugin(pluginLister),\n\t\t\tstring(common.PluginKindRestoreItemAction): s.restoreItemAction,\n\t\t\tstring(common.PluginKindRestoreItemActionV2): s.restoreItemActionV2,\n\t\t\tstring(common.PluginKindDeleteItemAction): s.deleteItemAction,\n\t\t\tstring(common.PluginKindItemBlockAction): s.itemBlockAction,\n\t\t},\n\t\tGRPCServer: plugin.DefaultGRPCServer,\n\t})\n}\n" }, { "path": "pkg/plugin/framework/stream_reader.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"bytes\"\n\t\"io\"\n)\n\n// ReceiveFunc is a function that either returns a slice\n// of an arbitrary number of bytes OR an error. Returning\n// an io.EOF means there is no more data to be read; any\n// other error is considered an actual error.\ntype ReceiveFunc func() ([]byte, error)\n\n// CloseFunc is used to signal to the source of data that\n// the StreamReadCloser has been closed.\ntype CloseFunc func() error\n\n// StreamReadCloser wraps a ReceiveFunc and a CloseSendFunc\n// to implement io.ReadCloser.\ntype StreamReadCloser struct {\n\tbuf *bytes.Buffer\n\treceive ReceiveFunc\n\tclose CloseFunc\n}\n\nfunc (s *StreamReadCloser) Read(p []byte) (n int, err error) {\n\tfor {\n\t\t// if buf exists and holds at least as much as we're trying to read,\n\t\t// read from the buffer\n\t\tif s.buf != nil && s.buf.Len() >= len(p) {\n\t\t\treturn s.buf.Read(p)\n\t\t}\n\n\t\t// if buf is nil, create it\n\t\tif s.buf == nil {\n\t\t\ts.buf = new(bytes.Buffer)\n\t\t}\n\n\t\t// buf exists but doesn't hold enough data to fill p, so\n\t\t// receive again. If we get an EOF, return what's in the\n\t\t// buffer; else, write the new data to the buffer and\n\t\t// try another read.\n\t\tdata, err := s.receive()\n\t\tif err == io.EOF {\n\t\t\treturn s.buf.Read(p)\n\t\t}\n\t\tif err != nil {\n\t\t\treturn 0, err\n\t\t}\n\n\t\tif _, err := s.buf.Write(data); err != nil {\n\t\t\treturn 0, err\n\t\t}\n\t}\n}\n\nfunc (s *StreamReadCloser) Close() error {\n\treturn s.close()\n}\n" }, { "path": "pkg/plugin/framework/stream_reader_test.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"bytes\"\n\t\"io\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\ntype stringByteReceiver struct {\n\tbuf *bytes.Buffer\n\tchunkSize int\n}\n\nfunc (r *stringByteReceiver) Receive() ([]byte, error) {\n\tchunk := make([]byte, r.chunkSize)\n\n\tn, err := r.buf.Read(chunk)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn chunk[0:n], nil\n}\n\nfunc (r *stringByteReceiver) CloseSend() error {\n\tr.buf = nil\n\treturn nil\n}\n\nfunc TestStreamReader(t *testing.T) {\n\ts := \"hello world, it's me, streamreader!!!!!\"\n\n\trdr := &stringByteReceiver{\n\t\tbuf: bytes.NewBufferString(s),\n\t\tchunkSize: 3,\n\t}\n\n\tsr := &StreamReadCloser{\n\t\treceive: rdr.Receive,\n\t\tclose: rdr.CloseSend,\n\t}\n\n\tres, err := io.ReadAll(sr)\n\n\trequire.NoError(t, err)\n\tassert.Equal(t, s, string(res))\n}\n" }, { "path": "pkg/plugin/framework/validation.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"github.com/pkg/errors\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n)\n\n// ValidateObjectStoreConfigKeys ensures that an object store's config\n// is valid by making sure each `config` key is in the `validKeys` list.\n// The special keys \"bucket\" and \"prefix\" are always considered valid.\nfunc ValidateObjectStoreConfigKeys(config map[string]string, validKeys ...string) error {\n\t// `bucket` and `prefix` are automatically added to all object\n\t// store config by velero, so add them as valid keys.\n\treturn validateConfigKeys(config, append(validKeys, \"bucket\", \"prefix\", \"caCert\")...)\n}\n\n// ValidateVolumeSnapshotterConfigKeys ensures that a volume snapshotter's\n// config is valid by making sure each `config` key is in the `validKeys` list.\nfunc ValidateVolumeSnapshotterConfigKeys(config map[string]string, validKeys ...string) error {\n\treturn validateConfigKeys(config, validKeys...)\n}\n\nfunc validateConfigKeys(config map[string]string, validKeys ...string) error {\n\tvalidKeysSet := sets.NewString(validKeys...)\n\n\tvar invalidKeys []string\n\tfor k := range config {\n\t\tif !validKeysSet.Has(k) {\n\t\t\tinvalidKeys = append(invalidKeys, k)\n\t\t}\n\t}\n\n\tif len(invalidKeys) > 0 {\n\t\treturn errors.Errorf(\"config has invalid keys %v; valid keys are %v\", invalidKeys, validKeys)\n\t}\n\n\treturn nil\n}\n" }, { "path": "pkg/plugin/framework/validation_test.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestValidateConfigKeys(t *testing.T) {\n\trequire.NoError(t, validateConfigKeys(nil))\n\trequire.NoError(t, validateConfigKeys(map[string]string{}))\n\trequire.NoError(t, validateConfigKeys(map[string]string{\"foo\": \"bar\"}, \"foo\"))\n\trequire.NoError(t, validateConfigKeys(map[string]string{\"foo\": \"bar\", \"bar\": \"baz\"}, \"foo\", \"bar\"))\n\n\trequire.Error(t, validateConfigKeys(map[string]string{\"foo\": \"bar\"}))\n\trequire.Error(t, validateConfigKeys(map[string]string{\"foo\": \"bar\"}, \"Foo\"))\n\trequire.Error(t, validateConfigKeys(map[string]string{\"foo\": \"bar\", \"boo\": \"\"}, \"foo\"))\n\n\trequire.NoError(t, ValidateObjectStoreConfigKeys(map[string]string{\"bucket\": \"foo\"}))\n\tassert.Error(t, ValidateVolumeSnapshotterConfigKeys(map[string]string{\"bucket\": \"foo\"}))\n}\n" }, { "path": "pkg/plugin/framework/volume_snapshotter.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"context\"\n\n\tplugin \"github.com/hashicorp/go-plugin\"\n\t\"google.golang.org/grpc\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n)\n\n// VolumeSnapshotterPlugin is an implementation of go-plugin's Plugin\n// interface with support for gRPC for the cloudprovider/VolumeSnapshotter\n// interface.\ntype VolumeSnapshotterPlugin struct {\n\tplugin.NetRPCUnsupportedPlugin\n\t*common.PluginBase\n}\n\n// GRPCClient returns a VolumeSnapshotter gRPC client.\nfunc (p *VolumeSnapshotterPlugin) GRPCClient(_ context.Context, _ *plugin.GRPCBroker, clientConn *grpc.ClientConn) (any, error) {\n\treturn common.NewClientDispenser(p.ClientLogger, clientConn, newVolumeSnapshotterGRPCClient), nil\n}\n\n// GRPCServer registers a VolumeSnapshotter gRPC server.\nfunc (p *VolumeSnapshotterPlugin) GRPCServer(_ *plugin.GRPCBroker, server *grpc.Server) error {\n\tproto.RegisterVolumeSnapshotterServer(server, &VolumeSnapshotterGRPCServer{mux: p.ServerMux})\n\treturn nil\n}\n" }, { "path": "pkg/plugin/framework/volume_snapshotter_client.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"encoding/json\"\n\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"google.golang.org/grpc\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n)\n\n// NewVolumeSnapshotterPlugin constructs a VolumeSnapshotterPlugin.\nfunc NewVolumeSnapshotterPlugin(options ...common.PluginOption) *VolumeSnapshotterPlugin {\n\treturn &VolumeSnapshotterPlugin{\n\t\tPluginBase: common.NewPluginBase(options...),\n\t}\n}\n\n// VolumeSnapshotterGRPCClient implements the cloudprovider.VolumeSnapshotter interface and uses a\n// gRPC client to make calls to the plugin server.\ntype VolumeSnapshotterGRPCClient struct {\n\t*common.ClientBase\n\tgrpcClient proto.VolumeSnapshotterClient\n}\n\nfunc newVolumeSnapshotterGRPCClient(base *common.ClientBase, clientConn *grpc.ClientConn) any {\n\treturn &VolumeSnapshotterGRPCClient{\n\t\tClientBase: base,\n\t\tgrpcClient: proto.NewVolumeSnapshotterClient(clientConn),\n\t}\n}\n\n// Init prepares the VolumeSnapshotter for usage using the provided map of\n// configuration key-value pairs. It returns an error if the VolumeSnapshotter\n// cannot be initialized from the provided config.\nfunc (c *VolumeSnapshotterGRPCClient) Init(config map[string]string) error {\n\treq := &proto.VolumeSnapshotterInitRequest{\n\t\tPlugin: c.Plugin,\n\t\tConfig: config,\n\t}\n\n\tif _, err := c.grpcClient.Init(context.Background(), req); err != nil {\n\t\treturn common.FromGRPCError(err)\n\t}\n\n\treturn nil\n}\n\n// CreateVolumeFromSnapshot creates a new block volume, initialized from the provided snapshot,\n// and with the specified type and IOPS (if using provisioned IOPS).\nfunc (c *VolumeSnapshotterGRPCClient) CreateVolumeFromSnapshot(snapshotID, volumeType, volumeAZ string, iops *int64) (string, error) {\n\treq := &proto.CreateVolumeRequest{\n\t\tPlugin: c.Plugin,\n\t\tSnapshotID: snapshotID,\n\t\tVolumeType: volumeType,\n\t\tVolumeAZ: volumeAZ,\n\t}\n\n\tif iops == nil {\n\t\treq.Iops = 0\n\t} else {\n\t\treq.Iops = *iops\n\t}\n\n\tres, err := c.grpcClient.CreateVolumeFromSnapshot(context.Background(), req)\n\tif err != nil {\n\t\treturn \"\", common.FromGRPCError(err)\n\t}\n\n\treturn res.VolumeID, nil\n}\n\n// GetVolumeInfo returns the type and IOPS (if using provisioned IOPS) for a specified block\n// volume.\nfunc (c *VolumeSnapshotterGRPCClient) GetVolumeInfo(volumeID, volumeAZ string) (string, *int64, error) {\n\treq := &proto.GetVolumeInfoRequest{\n\t\tPlugin: c.Plugin,\n\t\tVolumeID: volumeID,\n\t\tVolumeAZ: volumeAZ,\n\t}\n\n\tres, err := c.grpcClient.GetVolumeInfo(context.Background(), req)\n\tif err != nil {\n\t\treturn \"\", nil, common.FromGRPCError(err)\n\t}\n\n\tvar iops *int64\n\tif res.Iops != 0 {\n\t\tiops = &res.Iops\n\t}\n\n\treturn res.VolumeType, iops, nil\n}\n\n// CreateSnapshot creates a snapshot of the specified block volume, and applies the provided\n// set of tags to the snapshot.\nfunc (c *VolumeSnapshotterGRPCClient) CreateSnapshot(volumeID, volumeAZ string, tags map[string]string) (string, error) {\n\treq := &proto.CreateSnapshotRequest{\n\t\tPlugin: c.Plugin,\n\t\tVolumeID: volumeID,\n\t\tVolumeAZ: volumeAZ,\n\t\tTags: tags,\n\t}\n\n\tres, err := c.grpcClient.CreateSnapshot(context.Background(), req)\n\tif err != nil {\n\t\treturn \"\", common.FromGRPCError(err)\n\t}\n\n\treturn res.SnapshotID, nil\n}\n\n// DeleteSnapshot deletes the specified volume snapshot.\nfunc (c *VolumeSnapshotterGRPCClient) DeleteSnapshot(snapshotID string) error {\n\treq := &proto.DeleteSnapshotRequest{\n\t\tPlugin: c.Plugin,\n\t\tSnapshotID: snapshotID,\n\t}\n\n\tif _, err := c.grpcClient.DeleteSnapshot(context.Background(), req); err != nil {\n\t\treturn common.FromGRPCError(err)\n\t}\n\n\treturn nil\n}\n\nfunc (c *VolumeSnapshotterGRPCClient) GetVolumeID(pv runtime.Unstructured) (string, error) {\n\tencodedPV, err := json.Marshal(pv.UnstructuredContent())\n\tif err != nil {\n\t\treturn \"\", errors.WithStack(err)\n\t}\n\n\treq := &proto.GetVolumeIDRequest{\n\t\tPlugin: c.Plugin,\n\t\tPersistentVolume: encodedPV,\n\t}\n\n\tresp, err := c.grpcClient.GetVolumeID(context.Background(), req)\n\tif err != nil {\n\t\treturn \"\", common.FromGRPCError(err)\n\t}\n\n\treturn resp.VolumeID, nil\n}\n\nfunc (c *VolumeSnapshotterGRPCClient) SetVolumeID(pv runtime.Unstructured, volumeID string) (runtime.Unstructured, error) {\n\tencodedPV, err := json.Marshal(pv.UnstructuredContent())\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\treq := &proto.SetVolumeIDRequest{\n\t\tPlugin: c.Plugin,\n\t\tPersistentVolume: encodedPV,\n\t\tVolumeID: volumeID,\n\t}\n\n\tresp, err := c.grpcClient.SetVolumeID(context.Background(), req)\n\tif err != nil {\n\t\treturn nil, common.FromGRPCError(err)\n\t}\n\n\tvar updatedPV unstructured.Unstructured\n\tif err := json.Unmarshal(resp.PersistentVolume, &updatedPV); err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\treturn &updatedPV, nil\n}\n" }, { "path": "pkg/plugin/framework/volume_snapshotter_server.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage framework\n\nimport (\n\t\"encoding/json\"\n\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\tproto \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n\tvsv1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/volumesnapshotter/v1\"\n)\n\n// VolumeSnapshotterGRPCServer implements the proto-generated VolumeSnapshotterServer interface, and accepts\n// gRPC calls and forwards them to an implementation of the pluggable interface.\ntype VolumeSnapshotterGRPCServer struct {\n\tmux *common.ServerMux\n}\n\nfunc (s *VolumeSnapshotterGRPCServer) getImpl(name string) (vsv1.VolumeSnapshotter, error) {\n\timpl, err := s.mux.GetHandler(name)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tvolumeSnapshotter, ok := impl.(vsv1.VolumeSnapshotter)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"%T is not a volume snapshotter\", impl)\n\t}\n\n\treturn volumeSnapshotter, nil\n}\n\n// Init prepares the VolumeSnapshotter for usage using the provided map of\n// configuration key-value pairs. It returns an error if the VolumeSnapshotter\n// cannot be initialized from the provided config.\nfunc (s *VolumeSnapshotterGRPCServer) Init(ctx context.Context, req *proto.VolumeSnapshotterInitRequest) (response *proto.Empty, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tif err := impl.Init(req.Config); err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &proto.Empty{}, nil\n}\n\n// CreateVolumeFromSnapshot creates a new block volume, initialized from the provided snapshot,\n// and with the specified type and IOPS (if using provisioned IOPS).\nfunc (s *VolumeSnapshotterGRPCServer) CreateVolumeFromSnapshot(ctx context.Context, req *proto.CreateVolumeRequest) (response *proto.CreateVolumeResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tsnapshotID := req.SnapshotID\n\tvolumeType := req.VolumeType\n\tvolumeAZ := req.VolumeAZ\n\tvar iops *int64\n\n\tif req.Iops != 0 {\n\t\tiops = &req.Iops\n\t}\n\n\tvolumeID, err := impl.CreateVolumeFromSnapshot(snapshotID, volumeType, volumeAZ, iops)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &proto.CreateVolumeResponse{VolumeID: volumeID}, nil\n}\n\n// GetVolumeInfo returns the type and IOPS (if using provisioned IOPS) for a specified block\n// volume.\nfunc (s *VolumeSnapshotterGRPCServer) GetVolumeInfo(ctx context.Context, req *proto.GetVolumeInfoRequest) (response *proto.GetVolumeInfoResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tvolumeType, iops, err := impl.GetVolumeInfo(req.VolumeID, req.VolumeAZ)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tres := &proto.GetVolumeInfoResponse{\n\t\tVolumeType: volumeType,\n\t}\n\n\tif iops != nil {\n\t\tres.Iops = *iops\n\t}\n\n\treturn res, nil\n}\n\n// CreateSnapshot creates a snapshot of the specified block volume, and applies the provided\n// set of tags to the snapshot.\nfunc (s *VolumeSnapshotterGRPCServer) CreateSnapshot(ctx context.Context, req *proto.CreateSnapshotRequest) (response *proto.CreateSnapshotResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tsnapshotID, err := impl.CreateSnapshot(req.VolumeID, req.VolumeAZ, req.Tags)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &proto.CreateSnapshotResponse{SnapshotID: snapshotID}, nil\n}\n\n// DeleteSnapshot deletes the specified volume snapshot.\nfunc (s *VolumeSnapshotterGRPCServer) DeleteSnapshot(ctx context.Context, req *proto.DeleteSnapshotRequest) (response *proto.Empty, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tif err := impl.DeleteSnapshot(req.SnapshotID); err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &proto.Empty{}, nil\n}\n\nfunc (s *VolumeSnapshotterGRPCServer) GetVolumeID(ctx context.Context, req *proto.GetVolumeIDRequest) (response *proto.GetVolumeIDResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tvar pv unstructured.Unstructured\n\n\tif err := json.Unmarshal(req.PersistentVolume, &pv); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\n\tvolumeID, err := impl.GetVolumeID(&pv)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &proto.GetVolumeIDResponse{VolumeID: volumeID}, nil\n}\n\nfunc (s *VolumeSnapshotterGRPCServer) SetVolumeID(ctx context.Context, req *proto.SetVolumeIDRequest) (response *proto.SetVolumeIDResponse, err error) {\n\tdefer func() {\n\t\tif recoveredErr := common.HandlePanic(recover()); recoveredErr != nil {\n\t\t\terr = recoveredErr\n\t\t}\n\t}()\n\n\timpl, err := s.getImpl(req.Plugin)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tvar pv unstructured.Unstructured\n\tif err := json.Unmarshal(req.PersistentVolume, &pv); err != nil {\n\t\treturn nil, common.NewGRPCError(errors.WithStack(err))\n\t}\n\n\tupdatedPV, err := impl.SetVolumeID(&pv, req.VolumeID)\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\tupdatedPVBytes, err := json.Marshal(updatedPV.UnstructuredContent())\n\tif err != nil {\n\t\treturn nil, common.NewGRPCError(err)\n\t}\n\n\treturn &proto.SetVolumeIDResponse{PersistentVolume: updatedPVBytes}, nil\n}\n" }, { "path": "pkg/plugin/generated/BackupItemAction.pb.go", "content": "// Code generated by protoc-gen-go. DO NOT EDIT.\n// versions:\n// \tprotoc-gen-go v1.33.0\n// \tprotoc v4.25.2\n// source: BackupItemAction.proto\n\npackage generated\n\nimport (\n\tprotoreflect \"google.golang.org/protobuf/reflect/protoreflect\"\n\tprotoimpl \"google.golang.org/protobuf/runtime/protoimpl\"\n\treflect \"reflect\"\n\tsync \"sync\"\n)\n\nconst (\n\t// Verify that this generated code is sufficiently up-to-date.\n\t_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)\n\t// Verify that runtime/protoimpl is sufficiently up-to-date.\n\t_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)\n)\n\ntype ExecuteRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tItem []byte `protobuf:\"bytes,2,opt,name=item,proto3\" json:\"item,omitempty\"`\n\tBackup []byte `protobuf:\"bytes,3,opt,name=backup,proto3\" json:\"backup,omitempty\"`\n}\n\nfunc (x *ExecuteRequest) Reset() {\n\t*x = ExecuteRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_BackupItemAction_proto_msgTypes[0]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *ExecuteRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*ExecuteRequest) ProtoMessage() {}\n\nfunc (x *ExecuteRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_BackupItemAction_proto_msgTypes[0]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use ExecuteRequest.ProtoReflect.Descriptor instead.\nfunc (*ExecuteRequest) Descriptor() ([]byte, []int) {\n\treturn file_BackupItemAction_proto_rawDescGZIP(), []int{0}\n}\n\nfunc (x *ExecuteRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *ExecuteRequest) GetItem() []byte {\n\tif x != nil {\n\t\treturn x.Item\n\t}\n\treturn nil\n}\n\nfunc (x *ExecuteRequest) GetBackup() []byte {\n\tif x != nil {\n\t\treturn x.Backup\n\t}\n\treturn nil\n}\n\ntype ExecuteResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tItem []byte `protobuf:\"bytes,1,opt,name=item,proto3\" json:\"item,omitempty\"`\n\tAdditionalItems []*ResourceIdentifier `protobuf:\"bytes,2,rep,name=additionalItems,proto3\" json:\"additionalItems,omitempty\"`\n}\n\nfunc (x *ExecuteResponse) Reset() {\n\t*x = ExecuteResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_BackupItemAction_proto_msgTypes[1]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *ExecuteResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*ExecuteResponse) ProtoMessage() {}\n\nfunc (x *ExecuteResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_BackupItemAction_proto_msgTypes[1]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use ExecuteResponse.ProtoReflect.Descriptor instead.\nfunc (*ExecuteResponse) Descriptor() ([]byte, []int) {\n\treturn file_BackupItemAction_proto_rawDescGZIP(), []int{1}\n}\n\nfunc (x *ExecuteResponse) GetItem() []byte {\n\tif x != nil {\n\t\treturn x.Item\n\t}\n\treturn nil\n}\n\nfunc (x *ExecuteResponse) GetAdditionalItems() []*ResourceIdentifier {\n\tif x != nil {\n\t\treturn x.AdditionalItems\n\t}\n\treturn nil\n}\n\ntype BackupItemActionAppliesToRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n}\n\nfunc (x *BackupItemActionAppliesToRequest) Reset() {\n\t*x = BackupItemActionAppliesToRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_BackupItemAction_proto_msgTypes[2]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *BackupItemActionAppliesToRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*BackupItemActionAppliesToRequest) ProtoMessage() {}\n\nfunc (x *BackupItemActionAppliesToRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_BackupItemAction_proto_msgTypes[2]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use BackupItemActionAppliesToRequest.ProtoReflect.Descriptor instead.\nfunc (*BackupItemActionAppliesToRequest) Descriptor() ([]byte, []int) {\n\treturn file_BackupItemAction_proto_rawDescGZIP(), []int{2}\n}\n\nfunc (x *BackupItemActionAppliesToRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\ntype BackupItemActionAppliesToResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tResourceSelector *ResourceSelector `protobuf:\"bytes,1,opt,name=ResourceSelector,proto3\" json:\"ResourceSelector,omitempty\"`\n}\n\nfunc (x *BackupItemActionAppliesToResponse) Reset() {\n\t*x = BackupItemActionAppliesToResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_BackupItemAction_proto_msgTypes[3]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *BackupItemActionAppliesToResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*BackupItemActionAppliesToResponse) ProtoMessage() {}\n\nfunc (x *BackupItemActionAppliesToResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_BackupItemAction_proto_msgTypes[3]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use BackupItemActionAppliesToResponse.ProtoReflect.Descriptor instead.\nfunc (*BackupItemActionAppliesToResponse) Descriptor() ([]byte, []int) {\n\treturn file_BackupItemAction_proto_rawDescGZIP(), []int{3}\n}\n\nfunc (x *BackupItemActionAppliesToResponse) GetResourceSelector() *ResourceSelector {\n\tif x != nil {\n\t\treturn x.ResourceSelector\n\t}\n\treturn nil\n}\n\nvar File_BackupItemAction_proto protoreflect.FileDescriptor\n\nvar file_BackupItemAction_proto_rawDesc = []byte{\n\t0x0a, 0x16, 0x42, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69,\n\t0x6f, 0x6e, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x09, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61,\n\t0x74, 0x65, 0x64, 0x1a, 0x0c, 0x53, 0x68, 0x61, 0x72, 0x65, 0x64, 0x2e, 0x70, 0x72, 0x6f, 0x74,\n\t0x6f, 0x22, 0x54, 0x0a, 0x0e, 0x45, 0x78, 0x65, 0x63, 0x75, 0x74, 0x65, 0x52, 0x65, 0x71, 0x75,\n\t0x65, 0x73, 0x74, 0x12, 0x16, 0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20,\n\t0x01, 0x28, 0x09, 0x52, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x12, 0x12, 0x0a, 0x04, 0x69,\n\t0x74, 0x65, 0x6d, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x04, 0x69, 0x74, 0x65, 0x6d, 0x12,\n\t0x16, 0x0a, 0x06, 0x62, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0c, 0x52,\n\t0x06, 0x62, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x22, 0x6e, 0x0a, 0x0f, 0x45, 0x78, 0x65, 0x63, 0x75,\n\t0x74, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x69, 0x74,\n\t0x65, 0x6d, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x04, 0x69, 0x74, 0x65, 0x6d, 0x12, 0x47,\n\t0x0a, 0x0f, 0x61, 0x64, 0x64, 0x69, 0x74, 0x69, 0x6f, 0x6e, 0x61, 0x6c, 0x49, 0x74, 0x65, 0x6d,\n\t0x73, 0x18, 0x02, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x1d, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61,\n\t0x74, 0x65, 0x64, 0x2e, 0x52, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x49, 0x64, 0x65, 0x6e,\n\t0x74, 0x69, 0x66, 0x69, 0x65, 0x72, 0x52, 0x0f, 0x61, 0x64, 0x64, 0x69, 0x74, 0x69, 0x6f, 0x6e,\n\t0x61, 0x6c, 0x49, 0x74, 0x65, 0x6d, 0x73, 0x22, 0x3a, 0x0a, 0x20, 0x42, 0x61, 0x63, 0x6b, 0x75,\n\t0x70, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x41, 0x70, 0x70, 0x6c, 0x69,\n\t0x65, 0x73, 0x54, 0x6f, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x16, 0x0a, 0x06, 0x70,\n\t0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x70, 0x6c, 0x75,\n\t0x67, 0x69, 0x6e, 0x22, 0x6c, 0x0a, 0x21, 0x42, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x49, 0x74, 0x65,\n\t0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x41, 0x70, 0x70, 0x6c, 0x69, 0x65, 0x73, 0x54, 0x6f,\n\t0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x47, 0x0a, 0x10, 0x52, 0x65, 0x73, 0x6f,\n\t0x75, 0x72, 0x63, 0x65, 0x53, 0x65, 0x6c, 0x65, 0x63, 0x74, 0x6f, 0x72, 0x18, 0x01, 0x20, 0x01,\n\t0x28, 0x0b, 0x32, 0x1b, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x52,\n\t0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x53, 0x65, 0x6c, 0x65, 0x63, 0x74, 0x6f, 0x72, 0x52,\n\t0x10, 0x52, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x53, 0x65, 0x6c, 0x65, 0x63, 0x74, 0x6f,\n\t0x72, 0x32, 0xbc, 0x01, 0x0a, 0x10, 0x42, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x49, 0x74, 0x65, 0x6d,\n\t0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x66, 0x0a, 0x09, 0x41, 0x70, 0x70, 0x6c, 0x69, 0x65,\n\t0x73, 0x54, 0x6f, 0x12, 0x2b, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e,\n\t0x42, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e,\n\t0x41, 0x70, 0x70, 0x6c, 0x69, 0x65, 0x73, 0x54, 0x6f, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74,\n\t0x1a, 0x2c, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x42, 0x61, 0x63,\n\t0x6b, 0x75, 0x70, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x41, 0x70, 0x70,\n\t0x6c, 0x69, 0x65, 0x73, 0x54, 0x6f, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x40,\n\t0x0a, 0x07, 0x45, 0x78, 0x65, 0x63, 0x75, 0x74, 0x65, 0x12, 0x19, 0x2e, 0x67, 0x65, 0x6e, 0x65,\n\t0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x45, 0x78, 0x65, 0x63, 0x75, 0x74, 0x65, 0x52, 0x65, 0x71,\n\t0x75, 0x65, 0x73, 0x74, 0x1a, 0x1a, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64,\n\t0x2e, 0x45, 0x78, 0x65, 0x63, 0x75, 0x74, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65,\n\t0x42, 0x35, 0x5a, 0x33, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x76,\n\t0x6d, 0x77, 0x61, 0x72, 0x65, 0x2d, 0x74, 0x61, 0x6e, 0x7a, 0x75, 0x2f, 0x76, 0x65, 0x6c, 0x65,\n\t0x72, 0x6f, 0x2f, 0x70, 0x6b, 0x67, 0x2f, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x2f, 0x67, 0x65,\n\t0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,\n}\n\nvar (\n\tfile_BackupItemAction_proto_rawDescOnce sync.Once\n\tfile_BackupItemAction_proto_rawDescData = file_BackupItemAction_proto_rawDesc\n)\n\nfunc file_BackupItemAction_proto_rawDescGZIP() []byte {\n\tfile_BackupItemAction_proto_rawDescOnce.Do(func() {\n\t\tfile_BackupItemAction_proto_rawDescData = protoimpl.X.CompressGZIP(file_BackupItemAction_proto_rawDescData)\n\t})\n\treturn file_BackupItemAction_proto_rawDescData\n}\n\nvar file_BackupItemAction_proto_msgTypes = make([]protoimpl.MessageInfo, 4)\nvar file_BackupItemAction_proto_goTypes = []interface{}{\n\t(*ExecuteRequest)(nil), // 0: generated.ExecuteRequest\n\t(*ExecuteResponse)(nil), // 1: generated.ExecuteResponse\n\t(*BackupItemActionAppliesToRequest)(nil), // 2: generated.BackupItemActionAppliesToRequest\n\t(*BackupItemActionAppliesToResponse)(nil), // 3: generated.BackupItemActionAppliesToResponse\n\t(*ResourceIdentifier)(nil), // 4: generated.ResourceIdentifier\n\t(*ResourceSelector)(nil), // 5: generated.ResourceSelector\n}\nvar file_BackupItemAction_proto_depIdxs = []int32{\n\t4, // 0: generated.ExecuteResponse.additionalItems:type_name -> generated.ResourceIdentifier\n\t5, // 1: generated.BackupItemActionAppliesToResponse.ResourceSelector:type_name -> generated.ResourceSelector\n\t2, // 2: generated.BackupItemAction.AppliesTo:input_type -> generated.BackupItemActionAppliesToRequest\n\t0, // 3: generated.BackupItemAction.Execute:input_type -> generated.ExecuteRequest\n\t3, // 4: generated.BackupItemAction.AppliesTo:output_type -> generated.BackupItemActionAppliesToResponse\n\t1, // 5: generated.BackupItemAction.Execute:output_type -> generated.ExecuteResponse\n\t4, // [4:6] is the sub-list for method output_type\n\t2, // [2:4] is the sub-list for method input_type\n\t2, // [2:2] is the sub-list for extension type_name\n\t2, // [2:2] is the sub-list for extension extendee\n\t0, // [0:2] is the sub-list for field type_name\n}\n\nfunc init() { file_BackupItemAction_proto_init() }\nfunc file_BackupItemAction_proto_init() {\n\tif File_BackupItemAction_proto != nil {\n\t\treturn\n\t}\n\tfile_Shared_proto_init()\n\tif !protoimpl.UnsafeEnabled {\n\t\tfile_BackupItemAction_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*ExecuteRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_BackupItemAction_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*ExecuteResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_BackupItemAction_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*BackupItemActionAppliesToRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_BackupItemAction_proto_msgTypes[3].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*BackupItemActionAppliesToResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t}\n\ttype x struct{}\n\tout := protoimpl.TypeBuilder{\n\t\tFile: protoimpl.DescBuilder{\n\t\t\tGoPackagePath: reflect.TypeOf(x{}).PkgPath(),\n\t\t\tRawDescriptor: file_BackupItemAction_proto_rawDesc,\n\t\t\tNumEnums: 0,\n\t\t\tNumMessages: 4,\n\t\t\tNumExtensions: 0,\n\t\t\tNumServices: 1,\n\t\t},\n\t\tGoTypes: file_BackupItemAction_proto_goTypes,\n\t\tDependencyIndexes: file_BackupItemAction_proto_depIdxs,\n\t\tMessageInfos: file_BackupItemAction_proto_msgTypes,\n\t}.Build()\n\tFile_BackupItemAction_proto = out.File\n\tfile_BackupItemAction_proto_rawDesc = nil\n\tfile_BackupItemAction_proto_goTypes = nil\n\tfile_BackupItemAction_proto_depIdxs = nil\n}\n" }, { "path": "pkg/plugin/generated/BackupItemAction_grpc.pb.go", "content": "// Code generated by protoc-gen-go-grpc. DO NOT EDIT.\n// versions:\n// - protoc-gen-go-grpc v1.3.0\n// - protoc v4.25.2\n// source: BackupItemAction.proto\n\npackage generated\n\nimport (\n\tcontext \"context\"\n\tgrpc \"google.golang.org/grpc\"\n\tcodes \"google.golang.org/grpc/codes\"\n\tstatus \"google.golang.org/grpc/status\"\n)\n\n// This is a compile-time assertion to ensure that this generated file\n// is compatible with the grpc package it is being compiled against.\n// Requires gRPC-Go v1.32.0 or later.\nconst _ = grpc.SupportPackageIsVersion7\n\nconst (\n\tBackupItemAction_AppliesTo_FullMethodName = \"/generated.BackupItemAction/AppliesTo\"\n\tBackupItemAction_Execute_FullMethodName = \"/generated.BackupItemAction/Execute\"\n)\n\n// BackupItemActionClient is the client API for BackupItemAction service.\n//\n// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.\ntype BackupItemActionClient interface {\n\tAppliesTo(ctx context.Context, in *BackupItemActionAppliesToRequest, opts ...grpc.CallOption) (*BackupItemActionAppliesToResponse, error)\n\tExecute(ctx context.Context, in *ExecuteRequest, opts ...grpc.CallOption) (*ExecuteResponse, error)\n}\n\ntype backupItemActionClient struct {\n\tcc grpc.ClientConnInterface\n}\n\nfunc NewBackupItemActionClient(cc grpc.ClientConnInterface) BackupItemActionClient {\n\treturn &backupItemActionClient{cc}\n}\n\nfunc (c *backupItemActionClient) AppliesTo(ctx context.Context, in *BackupItemActionAppliesToRequest, opts ...grpc.CallOption) (*BackupItemActionAppliesToResponse, error) {\n\tout := new(BackupItemActionAppliesToResponse)\n\terr := c.cc.Invoke(ctx, BackupItemAction_AppliesTo_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *backupItemActionClient) Execute(ctx context.Context, in *ExecuteRequest, opts ...grpc.CallOption) (*ExecuteResponse, error) {\n\tout := new(ExecuteResponse)\n\terr := c.cc.Invoke(ctx, BackupItemAction_Execute_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\n// BackupItemActionServer is the server API for BackupItemAction service.\n// All implementations should embed UnimplementedBackupItemActionServer\n// for forward compatibility\ntype BackupItemActionServer interface {\n\tAppliesTo(context.Context, *BackupItemActionAppliesToRequest) (*BackupItemActionAppliesToResponse, error)\n\tExecute(context.Context, *ExecuteRequest) (*ExecuteResponse, error)\n}\n\n// UnimplementedBackupItemActionServer should be embedded to have forward compatible implementations.\ntype UnimplementedBackupItemActionServer struct {\n}\n\nfunc (UnimplementedBackupItemActionServer) AppliesTo(context.Context, *BackupItemActionAppliesToRequest) (*BackupItemActionAppliesToResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method AppliesTo not implemented\")\n}\nfunc (UnimplementedBackupItemActionServer) Execute(context.Context, *ExecuteRequest) (*ExecuteResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method Execute not implemented\")\n}\n\n// UnsafeBackupItemActionServer may be embedded to opt out of forward compatibility for this service.\n// Use of this interface is not recommended, as added methods to BackupItemActionServer will\n// result in compilation errors.\ntype UnsafeBackupItemActionServer interface {\n\tmustEmbedUnimplementedBackupItemActionServer()\n}\n\nfunc RegisterBackupItemActionServer(s grpc.ServiceRegistrar, srv BackupItemActionServer) {\n\ts.RegisterService(&BackupItemAction_ServiceDesc, srv)\n}\n\nfunc _BackupItemAction_AppliesTo_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(BackupItemActionAppliesToRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(BackupItemActionServer).AppliesTo(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: BackupItemAction_AppliesTo_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(BackupItemActionServer).AppliesTo(ctx, req.(*BackupItemActionAppliesToRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _BackupItemAction_Execute_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(ExecuteRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(BackupItemActionServer).Execute(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: BackupItemAction_Execute_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(BackupItemActionServer).Execute(ctx, req.(*ExecuteRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\n// BackupItemAction_ServiceDesc is the grpc.ServiceDesc for BackupItemAction service.\n// It's only intended for direct use with grpc.RegisterService,\n// and not to be introspected or modified (even as a copy)\nvar BackupItemAction_ServiceDesc = grpc.ServiceDesc{\n\tServiceName: \"generated.BackupItemAction\",\n\tHandlerType: (*BackupItemActionServer)(nil),\n\tMethods: []grpc.MethodDesc{\n\t\t{\n\t\t\tMethodName: \"AppliesTo\",\n\t\t\tHandler: _BackupItemAction_AppliesTo_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"Execute\",\n\t\t\tHandler: _BackupItemAction_Execute_Handler,\n\t\t},\n\t},\n\tStreams: []grpc.StreamDesc{},\n\tMetadata: \"BackupItemAction.proto\",\n}\n" }, { "path": "pkg/plugin/generated/DeleteItemAction.pb.go", "content": "// Code generated by protoc-gen-go. DO NOT EDIT.\n// versions:\n// \tprotoc-gen-go v1.33.0\n// \tprotoc v4.25.2\n// source: DeleteItemAction.proto\n\npackage generated\n\nimport (\n\tprotoreflect \"google.golang.org/protobuf/reflect/protoreflect\"\n\tprotoimpl \"google.golang.org/protobuf/runtime/protoimpl\"\n\treflect \"reflect\"\n\tsync \"sync\"\n)\n\nconst (\n\t// Verify that this generated code is sufficiently up-to-date.\n\t_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)\n\t// Verify that runtime/protoimpl is sufficiently up-to-date.\n\t_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)\n)\n\ntype DeleteItemActionExecuteRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tItem []byte `protobuf:\"bytes,2,opt,name=item,proto3\" json:\"item,omitempty\"`\n\tBackup []byte `protobuf:\"bytes,3,opt,name=backup,proto3\" json:\"backup,omitempty\"`\n}\n\nfunc (x *DeleteItemActionExecuteRequest) Reset() {\n\t*x = DeleteItemActionExecuteRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_DeleteItemAction_proto_msgTypes[0]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *DeleteItemActionExecuteRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*DeleteItemActionExecuteRequest) ProtoMessage() {}\n\nfunc (x *DeleteItemActionExecuteRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_DeleteItemAction_proto_msgTypes[0]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use DeleteItemActionExecuteRequest.ProtoReflect.Descriptor instead.\nfunc (*DeleteItemActionExecuteRequest) Descriptor() ([]byte, []int) {\n\treturn file_DeleteItemAction_proto_rawDescGZIP(), []int{0}\n}\n\nfunc (x *DeleteItemActionExecuteRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *DeleteItemActionExecuteRequest) GetItem() []byte {\n\tif x != nil {\n\t\treturn x.Item\n\t}\n\treturn nil\n}\n\nfunc (x *DeleteItemActionExecuteRequest) GetBackup() []byte {\n\tif x != nil {\n\t\treturn x.Backup\n\t}\n\treturn nil\n}\n\ntype DeleteItemActionAppliesToRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n}\n\nfunc (x *DeleteItemActionAppliesToRequest) Reset() {\n\t*x = DeleteItemActionAppliesToRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_DeleteItemAction_proto_msgTypes[1]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *DeleteItemActionAppliesToRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*DeleteItemActionAppliesToRequest) ProtoMessage() {}\n\nfunc (x *DeleteItemActionAppliesToRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_DeleteItemAction_proto_msgTypes[1]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use DeleteItemActionAppliesToRequest.ProtoReflect.Descriptor instead.\nfunc (*DeleteItemActionAppliesToRequest) Descriptor() ([]byte, []int) {\n\treturn file_DeleteItemAction_proto_rawDescGZIP(), []int{1}\n}\n\nfunc (x *DeleteItemActionAppliesToRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\ntype DeleteItemActionAppliesToResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tResourceSelector *ResourceSelector `protobuf:\"bytes,1,opt,name=ResourceSelector,proto3\" json:\"ResourceSelector,omitempty\"`\n}\n\nfunc (x *DeleteItemActionAppliesToResponse) Reset() {\n\t*x = DeleteItemActionAppliesToResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_DeleteItemAction_proto_msgTypes[2]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *DeleteItemActionAppliesToResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*DeleteItemActionAppliesToResponse) ProtoMessage() {}\n\nfunc (x *DeleteItemActionAppliesToResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_DeleteItemAction_proto_msgTypes[2]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use DeleteItemActionAppliesToResponse.ProtoReflect.Descriptor instead.\nfunc (*DeleteItemActionAppliesToResponse) Descriptor() ([]byte, []int) {\n\treturn file_DeleteItemAction_proto_rawDescGZIP(), []int{2}\n}\n\nfunc (x *DeleteItemActionAppliesToResponse) GetResourceSelector() *ResourceSelector {\n\tif x != nil {\n\t\treturn x.ResourceSelector\n\t}\n\treturn nil\n}\n\nvar File_DeleteItemAction_proto protoreflect.FileDescriptor\n\nvar file_DeleteItemAction_proto_rawDesc = []byte{\n\t0x0a, 0x16, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69,\n\t0x6f, 0x6e, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x09, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61,\n\t0x74, 0x65, 0x64, 0x1a, 0x0c, 0x53, 0x68, 0x61, 0x72, 0x65, 0x64, 0x2e, 0x70, 0x72, 0x6f, 0x74,\n\t0x6f, 0x22, 0x64, 0x0a, 0x1e, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41,\n\t0x63, 0x74, 0x69, 0x6f, 0x6e, 0x45, 0x78, 0x65, 0x63, 0x75, 0x74, 0x65, 0x52, 0x65, 0x71, 0x75,\n\t0x65, 0x73, 0x74, 0x12, 0x16, 0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20,\n\t0x01, 0x28, 0x09, 0x52, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x12, 0x12, 0x0a, 0x04, 0x69,\n\t0x74, 0x65, 0x6d, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x04, 0x69, 0x74, 0x65, 0x6d, 0x12,\n\t0x16, 0x0a, 0x06, 0x62, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0c, 0x52,\n\t0x06, 0x62, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x22, 0x3a, 0x0a, 0x20, 0x44, 0x65, 0x6c, 0x65, 0x74,\n\t0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x41, 0x70, 0x70, 0x6c, 0x69,\n\t0x65, 0x73, 0x54, 0x6f, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x16, 0x0a, 0x06, 0x70,\n\t0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x70, 0x6c, 0x75,\n\t0x67, 0x69, 0x6e, 0x22, 0x6c, 0x0a, 0x21, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x49, 0x74, 0x65,\n\t0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x41, 0x70, 0x70, 0x6c, 0x69, 0x65, 0x73, 0x54, 0x6f,\n\t0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x47, 0x0a, 0x10, 0x52, 0x65, 0x73, 0x6f,\n\t0x75, 0x72, 0x63, 0x65, 0x53, 0x65, 0x6c, 0x65, 0x63, 0x74, 0x6f, 0x72, 0x18, 0x01, 0x20, 0x01,\n\t0x28, 0x0b, 0x32, 0x1b, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x52,\n\t0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x53, 0x65, 0x6c, 0x65, 0x63, 0x74, 0x6f, 0x72, 0x52,\n\t0x10, 0x52, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x53, 0x65, 0x6c, 0x65, 0x63, 0x74, 0x6f,\n\t0x72, 0x32, 0xc2, 0x01, 0x0a, 0x10, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x49, 0x74, 0x65, 0x6d,\n\t0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x66, 0x0a, 0x09, 0x41, 0x70, 0x70, 0x6c, 0x69, 0x65,\n\t0x73, 0x54, 0x6f, 0x12, 0x2b, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e,\n\t0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e,\n\t0x41, 0x70, 0x70, 0x6c, 0x69, 0x65, 0x73, 0x54, 0x6f, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74,\n\t0x1a, 0x2c, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x44, 0x65, 0x6c,\n\t0x65, 0x74, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x41, 0x70, 0x70,\n\t0x6c, 0x69, 0x65, 0x73, 0x54, 0x6f, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x46,\n\t0x0a, 0x07, 0x45, 0x78, 0x65, 0x63, 0x75, 0x74, 0x65, 0x12, 0x29, 0x2e, 0x67, 0x65, 0x6e, 0x65,\n\t0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x49, 0x74, 0x65, 0x6d,\n\t0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x45, 0x78, 0x65, 0x63, 0x75, 0x74, 0x65, 0x52, 0x65, 0x71,\n\t0x75, 0x65, 0x73, 0x74, 0x1a, 0x10, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64,\n\t0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x42, 0x35, 0x5a, 0x33, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62,\n\t0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x76, 0x6d, 0x77, 0x61, 0x72, 0x65, 0x2d, 0x74, 0x61, 0x6e, 0x7a,\n\t0x75, 0x2f, 0x76, 0x65, 0x6c, 0x65, 0x72, 0x6f, 0x2f, 0x70, 0x6b, 0x67, 0x2f, 0x70, 0x6c, 0x75,\n\t0x67, 0x69, 0x6e, 0x2f, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x62, 0x06, 0x70,\n\t0x72, 0x6f, 0x74, 0x6f, 0x33,\n}\n\nvar (\n\tfile_DeleteItemAction_proto_rawDescOnce sync.Once\n\tfile_DeleteItemAction_proto_rawDescData = file_DeleteItemAction_proto_rawDesc\n)\n\nfunc file_DeleteItemAction_proto_rawDescGZIP() []byte {\n\tfile_DeleteItemAction_proto_rawDescOnce.Do(func() {\n\t\tfile_DeleteItemAction_proto_rawDescData = protoimpl.X.CompressGZIP(file_DeleteItemAction_proto_rawDescData)\n\t})\n\treturn file_DeleteItemAction_proto_rawDescData\n}\n\nvar file_DeleteItemAction_proto_msgTypes = make([]protoimpl.MessageInfo, 3)\nvar file_DeleteItemAction_proto_goTypes = []interface{}{\n\t(*DeleteItemActionExecuteRequest)(nil), // 0: generated.DeleteItemActionExecuteRequest\n\t(*DeleteItemActionAppliesToRequest)(nil), // 1: generated.DeleteItemActionAppliesToRequest\n\t(*DeleteItemActionAppliesToResponse)(nil), // 2: generated.DeleteItemActionAppliesToResponse\n\t(*ResourceSelector)(nil), // 3: generated.ResourceSelector\n\t(*Empty)(nil), // 4: generated.Empty\n}\nvar file_DeleteItemAction_proto_depIdxs = []int32{\n\t3, // 0: generated.DeleteItemActionAppliesToResponse.ResourceSelector:type_name -> generated.ResourceSelector\n\t1, // 1: generated.DeleteItemAction.AppliesTo:input_type -> generated.DeleteItemActionAppliesToRequest\n\t0, // 2: generated.DeleteItemAction.Execute:input_type -> generated.DeleteItemActionExecuteRequest\n\t2, // 3: generated.DeleteItemAction.AppliesTo:output_type -> generated.DeleteItemActionAppliesToResponse\n\t4, // 4: generated.DeleteItemAction.Execute:output_type -> generated.Empty\n\t3, // [3:5] is the sub-list for method output_type\n\t1, // [1:3] is the sub-list for method input_type\n\t1, // [1:1] is the sub-list for extension type_name\n\t1, // [1:1] is the sub-list for extension extendee\n\t0, // [0:1] is the sub-list for field type_name\n}\n\nfunc init() { file_DeleteItemAction_proto_init() }\nfunc file_DeleteItemAction_proto_init() {\n\tif File_DeleteItemAction_proto != nil {\n\t\treturn\n\t}\n\tfile_Shared_proto_init()\n\tif !protoimpl.UnsafeEnabled {\n\t\tfile_DeleteItemAction_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*DeleteItemActionExecuteRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_DeleteItemAction_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*DeleteItemActionAppliesToRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_DeleteItemAction_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*DeleteItemActionAppliesToResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t}\n\ttype x struct{}\n\tout := protoimpl.TypeBuilder{\n\t\tFile: protoimpl.DescBuilder{\n\t\t\tGoPackagePath: reflect.TypeOf(x{}).PkgPath(),\n\t\t\tRawDescriptor: file_DeleteItemAction_proto_rawDesc,\n\t\t\tNumEnums: 0,\n\t\t\tNumMessages: 3,\n\t\t\tNumExtensions: 0,\n\t\t\tNumServices: 1,\n\t\t},\n\t\tGoTypes: file_DeleteItemAction_proto_goTypes,\n\t\tDependencyIndexes: file_DeleteItemAction_proto_depIdxs,\n\t\tMessageInfos: file_DeleteItemAction_proto_msgTypes,\n\t}.Build()\n\tFile_DeleteItemAction_proto = out.File\n\tfile_DeleteItemAction_proto_rawDesc = nil\n\tfile_DeleteItemAction_proto_goTypes = nil\n\tfile_DeleteItemAction_proto_depIdxs = nil\n}\n" }, { "path": "pkg/plugin/generated/DeleteItemAction_grpc.pb.go", "content": "// Code generated by protoc-gen-go-grpc. DO NOT EDIT.\n// versions:\n// - protoc-gen-go-grpc v1.3.0\n// - protoc v4.25.2\n// source: DeleteItemAction.proto\n\npackage generated\n\nimport (\n\tcontext \"context\"\n\tgrpc \"google.golang.org/grpc\"\n\tcodes \"google.golang.org/grpc/codes\"\n\tstatus \"google.golang.org/grpc/status\"\n)\n\n// This is a compile-time assertion to ensure that this generated file\n// is compatible with the grpc package it is being compiled against.\n// Requires gRPC-Go v1.32.0 or later.\nconst _ = grpc.SupportPackageIsVersion7\n\nconst (\n\tDeleteItemAction_AppliesTo_FullMethodName = \"/generated.DeleteItemAction/AppliesTo\"\n\tDeleteItemAction_Execute_FullMethodName = \"/generated.DeleteItemAction/Execute\"\n)\n\n// DeleteItemActionClient is the client API for DeleteItemAction service.\n//\n// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.\ntype DeleteItemActionClient interface {\n\tAppliesTo(ctx context.Context, in *DeleteItemActionAppliesToRequest, opts ...grpc.CallOption) (*DeleteItemActionAppliesToResponse, error)\n\tExecute(ctx context.Context, in *DeleteItemActionExecuteRequest, opts ...grpc.CallOption) (*Empty, error)\n}\n\ntype deleteItemActionClient struct {\n\tcc grpc.ClientConnInterface\n}\n\nfunc NewDeleteItemActionClient(cc grpc.ClientConnInterface) DeleteItemActionClient {\n\treturn &deleteItemActionClient{cc}\n}\n\nfunc (c *deleteItemActionClient) AppliesTo(ctx context.Context, in *DeleteItemActionAppliesToRequest, opts ...grpc.CallOption) (*DeleteItemActionAppliesToResponse, error) {\n\tout := new(DeleteItemActionAppliesToResponse)\n\terr := c.cc.Invoke(ctx, DeleteItemAction_AppliesTo_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *deleteItemActionClient) Execute(ctx context.Context, in *DeleteItemActionExecuteRequest, opts ...grpc.CallOption) (*Empty, error) {\n\tout := new(Empty)\n\terr := c.cc.Invoke(ctx, DeleteItemAction_Execute_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\n// DeleteItemActionServer is the server API for DeleteItemAction service.\n// All implementations should embed UnimplementedDeleteItemActionServer\n// for forward compatibility\ntype DeleteItemActionServer interface {\n\tAppliesTo(context.Context, *DeleteItemActionAppliesToRequest) (*DeleteItemActionAppliesToResponse, error)\n\tExecute(context.Context, *DeleteItemActionExecuteRequest) (*Empty, error)\n}\n\n// UnimplementedDeleteItemActionServer should be embedded to have forward compatible implementations.\ntype UnimplementedDeleteItemActionServer struct {\n}\n\nfunc (UnimplementedDeleteItemActionServer) AppliesTo(context.Context, *DeleteItemActionAppliesToRequest) (*DeleteItemActionAppliesToResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method AppliesTo not implemented\")\n}\nfunc (UnimplementedDeleteItemActionServer) Execute(context.Context, *DeleteItemActionExecuteRequest) (*Empty, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method Execute not implemented\")\n}\n\n// UnsafeDeleteItemActionServer may be embedded to opt out of forward compatibility for this service.\n// Use of this interface is not recommended, as added methods to DeleteItemActionServer will\n// result in compilation errors.\ntype UnsafeDeleteItemActionServer interface {\n\tmustEmbedUnimplementedDeleteItemActionServer()\n}\n\nfunc RegisterDeleteItemActionServer(s grpc.ServiceRegistrar, srv DeleteItemActionServer) {\n\ts.RegisterService(&DeleteItemAction_ServiceDesc, srv)\n}\n\nfunc _DeleteItemAction_AppliesTo_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(DeleteItemActionAppliesToRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(DeleteItemActionServer).AppliesTo(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: DeleteItemAction_AppliesTo_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(DeleteItemActionServer).AppliesTo(ctx, req.(*DeleteItemActionAppliesToRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _DeleteItemAction_Execute_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(DeleteItemActionExecuteRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(DeleteItemActionServer).Execute(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: DeleteItemAction_Execute_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(DeleteItemActionServer).Execute(ctx, req.(*DeleteItemActionExecuteRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\n// DeleteItemAction_ServiceDesc is the grpc.ServiceDesc for DeleteItemAction service.\n// It's only intended for direct use with grpc.RegisterService,\n// and not to be introspected or modified (even as a copy)\nvar DeleteItemAction_ServiceDesc = grpc.ServiceDesc{\n\tServiceName: \"generated.DeleteItemAction\",\n\tHandlerType: (*DeleteItemActionServer)(nil),\n\tMethods: []grpc.MethodDesc{\n\t\t{\n\t\t\tMethodName: \"AppliesTo\",\n\t\t\tHandler: _DeleteItemAction_AppliesTo_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"Execute\",\n\t\t\tHandler: _DeleteItemAction_Execute_Handler,\n\t\t},\n\t},\n\tStreams: []grpc.StreamDesc{},\n\tMetadata: \"DeleteItemAction.proto\",\n}\n" }, { "path": "pkg/plugin/generated/ObjectStore.pb.go", "content": "// Code generated by protoc-gen-go. DO NOT EDIT.\n// versions:\n// \tprotoc-gen-go v1.33.0\n// \tprotoc v4.25.2\n// source: ObjectStore.proto\n\npackage generated\n\nimport (\n\tprotoreflect \"google.golang.org/protobuf/reflect/protoreflect\"\n\tprotoimpl \"google.golang.org/protobuf/runtime/protoimpl\"\n\treflect \"reflect\"\n\tsync \"sync\"\n)\n\nconst (\n\t// Verify that this generated code is sufficiently up-to-date.\n\t_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)\n\t// Verify that runtime/protoimpl is sufficiently up-to-date.\n\t_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)\n)\n\ntype PutObjectRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tBucket string `protobuf:\"bytes,2,opt,name=bucket,proto3\" json:\"bucket,omitempty\"`\n\tKey string `protobuf:\"bytes,3,opt,name=key,proto3\" json:\"key,omitempty\"`\n\tBody []byte `protobuf:\"bytes,4,opt,name=body,proto3\" json:\"body,omitempty\"`\n}\n\nfunc (x *PutObjectRequest) Reset() {\n\t*x = PutObjectRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_ObjectStore_proto_msgTypes[0]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *PutObjectRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*PutObjectRequest) ProtoMessage() {}\n\nfunc (x *PutObjectRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_ObjectStore_proto_msgTypes[0]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use PutObjectRequest.ProtoReflect.Descriptor instead.\nfunc (*PutObjectRequest) Descriptor() ([]byte, []int) {\n\treturn file_ObjectStore_proto_rawDescGZIP(), []int{0}\n}\n\nfunc (x *PutObjectRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *PutObjectRequest) GetBucket() string {\n\tif x != nil {\n\t\treturn x.Bucket\n\t}\n\treturn \"\"\n}\n\nfunc (x *PutObjectRequest) GetKey() string {\n\tif x != nil {\n\t\treturn x.Key\n\t}\n\treturn \"\"\n}\n\nfunc (x *PutObjectRequest) GetBody() []byte {\n\tif x != nil {\n\t\treturn x.Body\n\t}\n\treturn nil\n}\n\ntype ObjectExistsRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tBucket string `protobuf:\"bytes,2,opt,name=bucket,proto3\" json:\"bucket,omitempty\"`\n\tKey string `protobuf:\"bytes,3,opt,name=key,proto3\" json:\"key,omitempty\"`\n}\n\nfunc (x *ObjectExistsRequest) Reset() {\n\t*x = ObjectExistsRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_ObjectStore_proto_msgTypes[1]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *ObjectExistsRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*ObjectExistsRequest) ProtoMessage() {}\n\nfunc (x *ObjectExistsRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_ObjectStore_proto_msgTypes[1]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use ObjectExistsRequest.ProtoReflect.Descriptor instead.\nfunc (*ObjectExistsRequest) Descriptor() ([]byte, []int) {\n\treturn file_ObjectStore_proto_rawDescGZIP(), []int{1}\n}\n\nfunc (x *ObjectExistsRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *ObjectExistsRequest) GetBucket() string {\n\tif x != nil {\n\t\treturn x.Bucket\n\t}\n\treturn \"\"\n}\n\nfunc (x *ObjectExistsRequest) GetKey() string {\n\tif x != nil {\n\t\treturn x.Key\n\t}\n\treturn \"\"\n}\n\ntype ObjectExistsResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tExists bool `protobuf:\"varint,1,opt,name=exists,proto3\" json:\"exists,omitempty\"`\n}\n\nfunc (x *ObjectExistsResponse) Reset() {\n\t*x = ObjectExistsResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_ObjectStore_proto_msgTypes[2]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *ObjectExistsResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*ObjectExistsResponse) ProtoMessage() {}\n\nfunc (x *ObjectExistsResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_ObjectStore_proto_msgTypes[2]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use ObjectExistsResponse.ProtoReflect.Descriptor instead.\nfunc (*ObjectExistsResponse) Descriptor() ([]byte, []int) {\n\treturn file_ObjectStore_proto_rawDescGZIP(), []int{2}\n}\n\nfunc (x *ObjectExistsResponse) GetExists() bool {\n\tif x != nil {\n\t\treturn x.Exists\n\t}\n\treturn false\n}\n\ntype GetObjectRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tBucket string `protobuf:\"bytes,2,opt,name=bucket,proto3\" json:\"bucket,omitempty\"`\n\tKey string `protobuf:\"bytes,3,opt,name=key,proto3\" json:\"key,omitempty\"`\n}\n\nfunc (x *GetObjectRequest) Reset() {\n\t*x = GetObjectRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_ObjectStore_proto_msgTypes[3]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *GetObjectRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*GetObjectRequest) ProtoMessage() {}\n\nfunc (x *GetObjectRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_ObjectStore_proto_msgTypes[3]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use GetObjectRequest.ProtoReflect.Descriptor instead.\nfunc (*GetObjectRequest) Descriptor() ([]byte, []int) {\n\treturn file_ObjectStore_proto_rawDescGZIP(), []int{3}\n}\n\nfunc (x *GetObjectRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *GetObjectRequest) GetBucket() string {\n\tif x != nil {\n\t\treturn x.Bucket\n\t}\n\treturn \"\"\n}\n\nfunc (x *GetObjectRequest) GetKey() string {\n\tif x != nil {\n\t\treturn x.Key\n\t}\n\treturn \"\"\n}\n\ntype Bytes struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tData []byte `protobuf:\"bytes,1,opt,name=data,proto3\" json:\"data,omitempty\"`\n}\n\nfunc (x *Bytes) Reset() {\n\t*x = Bytes{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_ObjectStore_proto_msgTypes[4]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *Bytes) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*Bytes) ProtoMessage() {}\n\nfunc (x *Bytes) ProtoReflect() protoreflect.Message {\n\tmi := &file_ObjectStore_proto_msgTypes[4]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use Bytes.ProtoReflect.Descriptor instead.\nfunc (*Bytes) Descriptor() ([]byte, []int) {\n\treturn file_ObjectStore_proto_rawDescGZIP(), []int{4}\n}\n\nfunc (x *Bytes) GetData() []byte {\n\tif x != nil {\n\t\treturn x.Data\n\t}\n\treturn nil\n}\n\ntype ListCommonPrefixesRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tBucket string `protobuf:\"bytes,2,opt,name=bucket,proto3\" json:\"bucket,omitempty\"`\n\tDelimiter string `protobuf:\"bytes,3,opt,name=delimiter,proto3\" json:\"delimiter,omitempty\"`\n\tPrefix string `protobuf:\"bytes,4,opt,name=prefix,proto3\" json:\"prefix,omitempty\"`\n}\n\nfunc (x *ListCommonPrefixesRequest) Reset() {\n\t*x = ListCommonPrefixesRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_ObjectStore_proto_msgTypes[5]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *ListCommonPrefixesRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*ListCommonPrefixesRequest) ProtoMessage() {}\n\nfunc (x *ListCommonPrefixesRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_ObjectStore_proto_msgTypes[5]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use ListCommonPrefixesRequest.ProtoReflect.Descriptor instead.\nfunc (*ListCommonPrefixesRequest) Descriptor() ([]byte, []int) {\n\treturn file_ObjectStore_proto_rawDescGZIP(), []int{5}\n}\n\nfunc (x *ListCommonPrefixesRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *ListCommonPrefixesRequest) GetBucket() string {\n\tif x != nil {\n\t\treturn x.Bucket\n\t}\n\treturn \"\"\n}\n\nfunc (x *ListCommonPrefixesRequest) GetDelimiter() string {\n\tif x != nil {\n\t\treturn x.Delimiter\n\t}\n\treturn \"\"\n}\n\nfunc (x *ListCommonPrefixesRequest) GetPrefix() string {\n\tif x != nil {\n\t\treturn x.Prefix\n\t}\n\treturn \"\"\n}\n\ntype ListCommonPrefixesResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPrefixes []string `protobuf:\"bytes,1,rep,name=prefixes,proto3\" json:\"prefixes,omitempty\"`\n}\n\nfunc (x *ListCommonPrefixesResponse) Reset() {\n\t*x = ListCommonPrefixesResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_ObjectStore_proto_msgTypes[6]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *ListCommonPrefixesResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*ListCommonPrefixesResponse) ProtoMessage() {}\n\nfunc (x *ListCommonPrefixesResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_ObjectStore_proto_msgTypes[6]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use ListCommonPrefixesResponse.ProtoReflect.Descriptor instead.\nfunc (*ListCommonPrefixesResponse) Descriptor() ([]byte, []int) {\n\treturn file_ObjectStore_proto_rawDescGZIP(), []int{6}\n}\n\nfunc (x *ListCommonPrefixesResponse) GetPrefixes() []string {\n\tif x != nil {\n\t\treturn x.Prefixes\n\t}\n\treturn nil\n}\n\ntype ListObjectsRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tBucket string `protobuf:\"bytes,2,opt,name=bucket,proto3\" json:\"bucket,omitempty\"`\n\tPrefix string `protobuf:\"bytes,3,opt,name=prefix,proto3\" json:\"prefix,omitempty\"`\n}\n\nfunc (x *ListObjectsRequest) Reset() {\n\t*x = ListObjectsRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_ObjectStore_proto_msgTypes[7]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *ListObjectsRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*ListObjectsRequest) ProtoMessage() {}\n\nfunc (x *ListObjectsRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_ObjectStore_proto_msgTypes[7]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use ListObjectsRequest.ProtoReflect.Descriptor instead.\nfunc (*ListObjectsRequest) Descriptor() ([]byte, []int) {\n\treturn file_ObjectStore_proto_rawDescGZIP(), []int{7}\n}\n\nfunc (x *ListObjectsRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *ListObjectsRequest) GetBucket() string {\n\tif x != nil {\n\t\treturn x.Bucket\n\t}\n\treturn \"\"\n}\n\nfunc (x *ListObjectsRequest) GetPrefix() string {\n\tif x != nil {\n\t\treturn x.Prefix\n\t}\n\treturn \"\"\n}\n\ntype ListObjectsResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tKeys []string `protobuf:\"bytes,1,rep,name=keys,proto3\" json:\"keys,omitempty\"`\n}\n\nfunc (x *ListObjectsResponse) Reset() {\n\t*x = ListObjectsResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_ObjectStore_proto_msgTypes[8]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *ListObjectsResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*ListObjectsResponse) ProtoMessage() {}\n\nfunc (x *ListObjectsResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_ObjectStore_proto_msgTypes[8]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use ListObjectsResponse.ProtoReflect.Descriptor instead.\nfunc (*ListObjectsResponse) Descriptor() ([]byte, []int) {\n\treturn file_ObjectStore_proto_rawDescGZIP(), []int{8}\n}\n\nfunc (x *ListObjectsResponse) GetKeys() []string {\n\tif x != nil {\n\t\treturn x.Keys\n\t}\n\treturn nil\n}\n\ntype DeleteObjectRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tBucket string `protobuf:\"bytes,2,opt,name=bucket,proto3\" json:\"bucket,omitempty\"`\n\tKey string `protobuf:\"bytes,3,opt,name=key,proto3\" json:\"key,omitempty\"`\n}\n\nfunc (x *DeleteObjectRequest) Reset() {\n\t*x = DeleteObjectRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_ObjectStore_proto_msgTypes[9]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *DeleteObjectRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*DeleteObjectRequest) ProtoMessage() {}\n\nfunc (x *DeleteObjectRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_ObjectStore_proto_msgTypes[9]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use DeleteObjectRequest.ProtoReflect.Descriptor instead.\nfunc (*DeleteObjectRequest) Descriptor() ([]byte, []int) {\n\treturn file_ObjectStore_proto_rawDescGZIP(), []int{9}\n}\n\nfunc (x *DeleteObjectRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *DeleteObjectRequest) GetBucket() string {\n\tif x != nil {\n\t\treturn x.Bucket\n\t}\n\treturn \"\"\n}\n\nfunc (x *DeleteObjectRequest) GetKey() string {\n\tif x != nil {\n\t\treturn x.Key\n\t}\n\treturn \"\"\n}\n\ntype CreateSignedURLRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tBucket string `protobuf:\"bytes,2,opt,name=bucket,proto3\" json:\"bucket,omitempty\"`\n\tKey string `protobuf:\"bytes,3,opt,name=key,proto3\" json:\"key,omitempty\"`\n\tTtl int64 `protobuf:\"varint,4,opt,name=ttl,proto3\" json:\"ttl,omitempty\"`\n}\n\nfunc (x *CreateSignedURLRequest) Reset() {\n\t*x = CreateSignedURLRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_ObjectStore_proto_msgTypes[10]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *CreateSignedURLRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*CreateSignedURLRequest) ProtoMessage() {}\n\nfunc (x *CreateSignedURLRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_ObjectStore_proto_msgTypes[10]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use CreateSignedURLRequest.ProtoReflect.Descriptor instead.\nfunc (*CreateSignedURLRequest) Descriptor() ([]byte, []int) {\n\treturn file_ObjectStore_proto_rawDescGZIP(), []int{10}\n}\n\nfunc (x *CreateSignedURLRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *CreateSignedURLRequest) GetBucket() string {\n\tif x != nil {\n\t\treturn x.Bucket\n\t}\n\treturn \"\"\n}\n\nfunc (x *CreateSignedURLRequest) GetKey() string {\n\tif x != nil {\n\t\treturn x.Key\n\t}\n\treturn \"\"\n}\n\nfunc (x *CreateSignedURLRequest) GetTtl() int64 {\n\tif x != nil {\n\t\treturn x.Ttl\n\t}\n\treturn 0\n}\n\ntype CreateSignedURLResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tUrl string `protobuf:\"bytes,1,opt,name=url,proto3\" json:\"url,omitempty\"`\n}\n\nfunc (x *CreateSignedURLResponse) Reset() {\n\t*x = CreateSignedURLResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_ObjectStore_proto_msgTypes[11]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *CreateSignedURLResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*CreateSignedURLResponse) ProtoMessage() {}\n\nfunc (x *CreateSignedURLResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_ObjectStore_proto_msgTypes[11]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use CreateSignedURLResponse.ProtoReflect.Descriptor instead.\nfunc (*CreateSignedURLResponse) Descriptor() ([]byte, []int) {\n\treturn file_ObjectStore_proto_rawDescGZIP(), []int{11}\n}\n\nfunc (x *CreateSignedURLResponse) GetUrl() string {\n\tif x != nil {\n\t\treturn x.Url\n\t}\n\treturn \"\"\n}\n\ntype ObjectStoreInitRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tConfig map[string]string `protobuf:\"bytes,2,rep,name=config,proto3\" json:\"config,omitempty\" protobuf_key:\"bytes,1,opt,name=key,proto3\" protobuf_val:\"bytes,2,opt,name=value,proto3\"`\n}\n\nfunc (x *ObjectStoreInitRequest) Reset() {\n\t*x = ObjectStoreInitRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_ObjectStore_proto_msgTypes[12]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *ObjectStoreInitRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*ObjectStoreInitRequest) ProtoMessage() {}\n\nfunc (x *ObjectStoreInitRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_ObjectStore_proto_msgTypes[12]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use ObjectStoreInitRequest.ProtoReflect.Descriptor instead.\nfunc (*ObjectStoreInitRequest) Descriptor() ([]byte, []int) {\n\treturn file_ObjectStore_proto_rawDescGZIP(), []int{12}\n}\n\nfunc (x *ObjectStoreInitRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *ObjectStoreInitRequest) GetConfig() map[string]string {\n\tif x != nil {\n\t\treturn x.Config\n\t}\n\treturn nil\n}\n\nvar File_ObjectStore_proto protoreflect.FileDescriptor\n\nvar file_ObjectStore_proto_rawDesc = []byte{\n\t0x0a, 0x11, 0x4f, 0x62, 0x6a, 0x65, 0x63, 0x74, 0x53, 0x74, 0x6f, 0x72, 0x65, 0x2e, 0x70, 0x72,\n\t0x6f, 0x74, 0x6f, 0x12, 0x09, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x1a, 0x0c,\n\t0x53, 0x68, 0x61, 0x72, 0x65, 0x64, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, 0x68, 0x0a, 0x10,\n\t0x50, 0x75, 0x74, 0x4f, 0x62, 0x6a, 0x65, 0x63, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74,\n\t0x12, 0x16, 0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09,\n\t0x52, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x12, 0x16, 0x0a, 0x06, 0x62, 0x75, 0x63, 0x6b,\n\t0x65, 0x74, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x62, 0x75, 0x63, 0x6b, 0x65, 0x74,\n\t0x12, 0x10, 0x0a, 0x03, 0x6b, 0x65, 0x79, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x6b,\n\t0x65, 0x79, 0x12, 0x12, 0x0a, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x18, 0x04, 0x20, 0x01, 0x28, 0x0c,\n\t0x52, 0x04, 0x62, 0x6f, 0x64, 0x79, 0x22, 0x57, 0x0a, 0x13, 0x4f, 0x62, 0x6a, 0x65, 0x63, 0x74,\n\t0x45, 0x78, 0x69, 0x73, 0x74, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x16, 0x0a,\n\t0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x70,\n\t0x6c, 0x75, 0x67, 0x69, 0x6e, 0x12, 0x16, 0x0a, 0x06, 0x62, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x18,\n\t0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x62, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x12, 0x10, 0x0a,\n\t0x03, 0x6b, 0x65, 0x79, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x6b, 0x65, 0x79, 0x22,\n\t0x2e, 0x0a, 0x14, 0x4f, 0x62, 0x6a, 0x65, 0x63, 0x74, 0x45, 0x78, 0x69, 0x73, 0x74, 0x73, 0x52,\n\t0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x16, 0x0a, 0x06, 0x65, 0x78, 0x69, 0x73, 0x74,\n\t0x73, 0x18, 0x01, 0x20, 0x01, 0x28, 0x08, 0x52, 0x06, 0x65, 0x78, 0x69, 0x73, 0x74, 0x73, 0x22,\n\t0x54, 0x0a, 0x10, 0x47, 0x65, 0x74, 0x4f, 0x62, 0x6a, 0x65, 0x63, 0x74, 0x52, 0x65, 0x71, 0x75,\n\t0x65, 0x73, 0x74, 0x12, 0x16, 0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20,\n\t0x01, 0x28, 0x09, 0x52, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x12, 0x16, 0x0a, 0x06, 0x62,\n\t0x75, 0x63, 0x6b, 0x65, 0x74, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x62, 0x75, 0x63,\n\t0x6b, 0x65, 0x74, 0x12, 0x10, 0x0a, 0x03, 0x6b, 0x65, 0x79, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09,\n\t0x52, 0x03, 0x6b, 0x65, 0x79, 0x22, 0x1b, 0x0a, 0x05, 0x42, 0x79, 0x74, 0x65, 0x73, 0x12, 0x12,\n\t0x0a, 0x04, 0x64, 0x61, 0x74, 0x61, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x04, 0x64, 0x61,\n\t0x74, 0x61, 0x22, 0x81, 0x01, 0x0a, 0x19, 0x4c, 0x69, 0x73, 0x74, 0x43, 0x6f, 0x6d, 0x6d, 0x6f,\n\t0x6e, 0x50, 0x72, 0x65, 0x66, 0x69, 0x78, 0x65, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74,\n\t0x12, 0x16, 0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09,\n\t0x52, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x12, 0x16, 0x0a, 0x06, 0x62, 0x75, 0x63, 0x6b,\n\t0x65, 0x74, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x62, 0x75, 0x63, 0x6b, 0x65, 0x74,\n\t0x12, 0x1c, 0x0a, 0x09, 0x64, 0x65, 0x6c, 0x69, 0x6d, 0x69, 0x74, 0x65, 0x72, 0x18, 0x03, 0x20,\n\t0x01, 0x28, 0x09, 0x52, 0x09, 0x64, 0x65, 0x6c, 0x69, 0x6d, 0x69, 0x74, 0x65, 0x72, 0x12, 0x16,\n\t0x0a, 0x06, 0x70, 0x72, 0x65, 0x66, 0x69, 0x78, 0x18, 0x04, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06,\n\t0x70, 0x72, 0x65, 0x66, 0x69, 0x78, 0x22, 0x38, 0x0a, 0x1a, 0x4c, 0x69, 0x73, 0x74, 0x43, 0x6f,\n\t0x6d, 0x6d, 0x6f, 0x6e, 0x50, 0x72, 0x65, 0x66, 0x69, 0x78, 0x65, 0x73, 0x52, 0x65, 0x73, 0x70,\n\t0x6f, 0x6e, 0x73, 0x65, 0x12, 0x1a, 0x0a, 0x08, 0x70, 0x72, 0x65, 0x66, 0x69, 0x78, 0x65, 0x73,\n\t0x18, 0x01, 0x20, 0x03, 0x28, 0x09, 0x52, 0x08, 0x70, 0x72, 0x65, 0x66, 0x69, 0x78, 0x65, 0x73,\n\t0x22, 0x5c, 0x0a, 0x12, 0x4c, 0x69, 0x73, 0x74, 0x4f, 0x62, 0x6a, 0x65, 0x63, 0x74, 0x73, 0x52,\n\t0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x16, 0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e,\n\t0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x12, 0x16,\n\t0x0a, 0x06, 0x62, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06,\n\t0x62, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x12, 0x16, 0x0a, 0x06, 0x70, 0x72, 0x65, 0x66, 0x69, 0x78,\n\t0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x70, 0x72, 0x65, 0x66, 0x69, 0x78, 0x22, 0x29,\n\t0x0a, 0x13, 0x4c, 0x69, 0x73, 0x74, 0x4f, 0x62, 0x6a, 0x65, 0x63, 0x74, 0x73, 0x52, 0x65, 0x73,\n\t0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x6b, 0x65, 0x79, 0x73, 0x18, 0x01, 0x20,\n\t0x03, 0x28, 0x09, 0x52, 0x04, 0x6b, 0x65, 0x79, 0x73, 0x22, 0x57, 0x0a, 0x13, 0x44, 0x65, 0x6c,\n\t0x65, 0x74, 0x65, 0x4f, 0x62, 0x6a, 0x65, 0x63, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74,\n\t0x12, 0x16, 0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09,\n\t0x52, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x12, 0x16, 0x0a, 0x06, 0x62, 0x75, 0x63, 0x6b,\n\t0x65, 0x74, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x62, 0x75, 0x63, 0x6b, 0x65, 0x74,\n\t0x12, 0x10, 0x0a, 0x03, 0x6b, 0x65, 0x79, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x6b,\n\t0x65, 0x79, 0x22, 0x6c, 0x0a, 0x16, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x53, 0x69, 0x67, 0x6e,\n\t0x65, 0x64, 0x55, 0x52, 0x4c, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x16, 0x0a, 0x06,\n\t0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x70, 0x6c,\n\t0x75, 0x67, 0x69, 0x6e, 0x12, 0x16, 0x0a, 0x06, 0x62, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x18, 0x02,\n\t0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x62, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x12, 0x10, 0x0a, 0x03,\n\t0x6b, 0x65, 0x79, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x6b, 0x65, 0x79, 0x12, 0x10,\n\t0x0a, 0x03, 0x74, 0x74, 0x6c, 0x18, 0x04, 0x20, 0x01, 0x28, 0x03, 0x52, 0x03, 0x74, 0x74, 0x6c,\n\t0x22, 0x2b, 0x0a, 0x17, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x53, 0x69, 0x67, 0x6e, 0x65, 0x64,\n\t0x55, 0x52, 0x4c, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x10, 0x0a, 0x03, 0x75,\n\t0x72, 0x6c, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x75, 0x72, 0x6c, 0x22, 0xb2, 0x01,\n\t0x0a, 0x16, 0x4f, 0x62, 0x6a, 0x65, 0x63, 0x74, 0x53, 0x74, 0x6f, 0x72, 0x65, 0x49, 0x6e, 0x69,\n\t0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x16, 0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67,\n\t0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e,\n\t0x12, 0x45, 0x0a, 0x06, 0x63, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x18, 0x02, 0x20, 0x03, 0x28, 0x0b,\n\t0x32, 0x2d, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x4f, 0x62, 0x6a,\n\t0x65, 0x63, 0x74, 0x53, 0x74, 0x6f, 0x72, 0x65, 0x49, 0x6e, 0x69, 0x74, 0x52, 0x65, 0x71, 0x75,\n\t0x65, 0x73, 0x74, 0x2e, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x52,\n\t0x06, 0x63, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x1a, 0x39, 0x0a, 0x0b, 0x43, 0x6f, 0x6e, 0x66, 0x69,\n\t0x67, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x12, 0x10, 0x0a, 0x03, 0x6b, 0x65, 0x79, 0x18, 0x01, 0x20,\n\t0x01, 0x28, 0x09, 0x52, 0x03, 0x6b, 0x65, 0x79, 0x12, 0x14, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75,\n\t0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x3a, 0x02,\n\t0x38, 0x01, 0x32, 0xe4, 0x04, 0x0a, 0x0b, 0x4f, 0x62, 0x6a, 0x65, 0x63, 0x74, 0x53, 0x74, 0x6f,\n\t0x72, 0x65, 0x12, 0x3b, 0x0a, 0x04, 0x49, 0x6e, 0x69, 0x74, 0x12, 0x21, 0x2e, 0x67, 0x65, 0x6e,\n\t0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x4f, 0x62, 0x6a, 0x65, 0x63, 0x74, 0x53, 0x74, 0x6f,\n\t0x72, 0x65, 0x49, 0x6e, 0x69, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x10, 0x2e,\n\t0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x12,\n\t0x3c, 0x0a, 0x09, 0x50, 0x75, 0x74, 0x4f, 0x62, 0x6a, 0x65, 0x63, 0x74, 0x12, 0x1b, 0x2e, 0x67,\n\t0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x50, 0x75, 0x74, 0x4f, 0x62, 0x6a, 0x65,\n\t0x63, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x10, 0x2e, 0x67, 0x65, 0x6e, 0x65,\n\t0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x28, 0x01, 0x12, 0x4f, 0x0a,\n\t0x0c, 0x4f, 0x62, 0x6a, 0x65, 0x63, 0x74, 0x45, 0x78, 0x69, 0x73, 0x74, 0x73, 0x12, 0x1e, 0x2e,\n\t0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x4f, 0x62, 0x6a, 0x65, 0x63, 0x74,\n\t0x45, 0x78, 0x69, 0x73, 0x74, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1f, 0x2e,\n\t0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x4f, 0x62, 0x6a, 0x65, 0x63, 0x74,\n\t0x45, 0x78, 0x69, 0x73, 0x74, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x3c,\n\t0x0a, 0x09, 0x47, 0x65, 0x74, 0x4f, 0x62, 0x6a, 0x65, 0x63, 0x74, 0x12, 0x1b, 0x2e, 0x67, 0x65,\n\t0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x47, 0x65, 0x74, 0x4f, 0x62, 0x6a, 0x65, 0x63,\n\t0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x10, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72,\n\t0x61, 0x74, 0x65, 0x64, 0x2e, 0x42, 0x79, 0x74, 0x65, 0x73, 0x30, 0x01, 0x12, 0x61, 0x0a, 0x12,\n\t0x4c, 0x69, 0x73, 0x74, 0x43, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x50, 0x72, 0x65, 0x66, 0x69, 0x78,\n\t0x65, 0x73, 0x12, 0x24, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x4c,\n\t0x69, 0x73, 0x74, 0x43, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x50, 0x72, 0x65, 0x66, 0x69, 0x78, 0x65,\n\t0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x25, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72,\n\t0x61, 0x74, 0x65, 0x64, 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x43, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x50,\n\t0x72, 0x65, 0x66, 0x69, 0x78, 0x65, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12,\n\t0x4c, 0x0a, 0x0b, 0x4c, 0x69, 0x73, 0x74, 0x4f, 0x62, 0x6a, 0x65, 0x63, 0x74, 0x73, 0x12, 0x1d,\n\t0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x4f,\n\t0x62, 0x6a, 0x65, 0x63, 0x74, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1e, 0x2e,\n\t0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x4f, 0x62,\n\t0x6a, 0x65, 0x63, 0x74, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x40, 0x0a,\n\t0x0c, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x4f, 0x62, 0x6a, 0x65, 0x63, 0x74, 0x12, 0x1e, 0x2e,\n\t0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65,\n\t0x4f, 0x62, 0x6a, 0x65, 0x63, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x10, 0x2e,\n\t0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x12,\n\t0x58, 0x0a, 0x0f, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x53, 0x69, 0x67, 0x6e, 0x65, 0x64, 0x55,\n\t0x52, 0x4c, 0x12, 0x21, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x43,\n\t0x72, 0x65, 0x61, 0x74, 0x65, 0x53, 0x69, 0x67, 0x6e, 0x65, 0x64, 0x55, 0x52, 0x4c, 0x52, 0x65,\n\t0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x22, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65,\n\t0x64, 0x2e, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x53, 0x69, 0x67, 0x6e, 0x65, 0x64, 0x55, 0x52,\n\t0x4c, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x42, 0x35, 0x5a, 0x33, 0x67, 0x69, 0x74,\n\t0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x76, 0x6d, 0x77, 0x61, 0x72, 0x65, 0x2d, 0x74,\n\t0x61, 0x6e, 0x7a, 0x75, 0x2f, 0x76, 0x65, 0x6c, 0x65, 0x72, 0x6f, 0x2f, 0x70, 0x6b, 0x67, 0x2f,\n\t0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x2f, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64,\n\t0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,\n}\n\nvar (\n\tfile_ObjectStore_proto_rawDescOnce sync.Once\n\tfile_ObjectStore_proto_rawDescData = file_ObjectStore_proto_rawDesc\n)\n\nfunc file_ObjectStore_proto_rawDescGZIP() []byte {\n\tfile_ObjectStore_proto_rawDescOnce.Do(func() {\n\t\tfile_ObjectStore_proto_rawDescData = protoimpl.X.CompressGZIP(file_ObjectStore_proto_rawDescData)\n\t})\n\treturn file_ObjectStore_proto_rawDescData\n}\n\nvar file_ObjectStore_proto_msgTypes = make([]protoimpl.MessageInfo, 14)\nvar file_ObjectStore_proto_goTypes = []interface{}{\n\t(*PutObjectRequest)(nil), // 0: generated.PutObjectRequest\n\t(*ObjectExistsRequest)(nil), // 1: generated.ObjectExistsRequest\n\t(*ObjectExistsResponse)(nil), // 2: generated.ObjectExistsResponse\n\t(*GetObjectRequest)(nil), // 3: generated.GetObjectRequest\n\t(*Bytes)(nil), // 4: generated.Bytes\n\t(*ListCommonPrefixesRequest)(nil), // 5: generated.ListCommonPrefixesRequest\n\t(*ListCommonPrefixesResponse)(nil), // 6: generated.ListCommonPrefixesResponse\n\t(*ListObjectsRequest)(nil), // 7: generated.ListObjectsRequest\n\t(*ListObjectsResponse)(nil), // 8: generated.ListObjectsResponse\n\t(*DeleteObjectRequest)(nil), // 9: generated.DeleteObjectRequest\n\t(*CreateSignedURLRequest)(nil), // 10: generated.CreateSignedURLRequest\n\t(*CreateSignedURLResponse)(nil), // 11: generated.CreateSignedURLResponse\n\t(*ObjectStoreInitRequest)(nil), // 12: generated.ObjectStoreInitRequest\n\tnil, // 13: generated.ObjectStoreInitRequest.ConfigEntry\n\t(*Empty)(nil), // 14: generated.Empty\n}\nvar file_ObjectStore_proto_depIdxs = []int32{\n\t13, // 0: generated.ObjectStoreInitRequest.config:type_name -> generated.ObjectStoreInitRequest.ConfigEntry\n\t12, // 1: generated.ObjectStore.Init:input_type -> generated.ObjectStoreInitRequest\n\t0, // 2: generated.ObjectStore.PutObject:input_type -> generated.PutObjectRequest\n\t1, // 3: generated.ObjectStore.ObjectExists:input_type -> generated.ObjectExistsRequest\n\t3, // 4: generated.ObjectStore.GetObject:input_type -> generated.GetObjectRequest\n\t5, // 5: generated.ObjectStore.ListCommonPrefixes:input_type -> generated.ListCommonPrefixesRequest\n\t7, // 6: generated.ObjectStore.ListObjects:input_type -> generated.ListObjectsRequest\n\t9, // 7: generated.ObjectStore.DeleteObject:input_type -> generated.DeleteObjectRequest\n\t10, // 8: generated.ObjectStore.CreateSignedURL:input_type -> generated.CreateSignedURLRequest\n\t14, // 9: generated.ObjectStore.Init:output_type -> generated.Empty\n\t14, // 10: generated.ObjectStore.PutObject:output_type -> generated.Empty\n\t2, // 11: generated.ObjectStore.ObjectExists:output_type -> generated.ObjectExistsResponse\n\t4, // 12: generated.ObjectStore.GetObject:output_type -> generated.Bytes\n\t6, // 13: generated.ObjectStore.ListCommonPrefixes:output_type -> generated.ListCommonPrefixesResponse\n\t8, // 14: generated.ObjectStore.ListObjects:output_type -> generated.ListObjectsResponse\n\t14, // 15: generated.ObjectStore.DeleteObject:output_type -> generated.Empty\n\t11, // 16: generated.ObjectStore.CreateSignedURL:output_type -> generated.CreateSignedURLResponse\n\t9, // [9:17] is the sub-list for method output_type\n\t1, // [1:9] is the sub-list for method input_type\n\t1, // [1:1] is the sub-list for extension type_name\n\t1, // [1:1] is the sub-list for extension extendee\n\t0, // [0:1] is the sub-list for field type_name\n}\n\nfunc init() { file_ObjectStore_proto_init() }\nfunc file_ObjectStore_proto_init() {\n\tif File_ObjectStore_proto != nil {\n\t\treturn\n\t}\n\tfile_Shared_proto_init()\n\tif !protoimpl.UnsafeEnabled {\n\t\tfile_ObjectStore_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*PutObjectRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_ObjectStore_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*ObjectExistsRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_ObjectStore_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*ObjectExistsResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_ObjectStore_proto_msgTypes[3].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*GetObjectRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_ObjectStore_proto_msgTypes[4].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*Bytes); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_ObjectStore_proto_msgTypes[5].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*ListCommonPrefixesRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_ObjectStore_proto_msgTypes[6].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*ListCommonPrefixesResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_ObjectStore_proto_msgTypes[7].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*ListObjectsRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_ObjectStore_proto_msgTypes[8].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*ListObjectsResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_ObjectStore_proto_msgTypes[9].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*DeleteObjectRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_ObjectStore_proto_msgTypes[10].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*CreateSignedURLRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_ObjectStore_proto_msgTypes[11].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*CreateSignedURLResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_ObjectStore_proto_msgTypes[12].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*ObjectStoreInitRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t}\n\ttype x struct{}\n\tout := protoimpl.TypeBuilder{\n\t\tFile: protoimpl.DescBuilder{\n\t\t\tGoPackagePath: reflect.TypeOf(x{}).PkgPath(),\n\t\t\tRawDescriptor: file_ObjectStore_proto_rawDesc,\n\t\t\tNumEnums: 0,\n\t\t\tNumMessages: 14,\n\t\t\tNumExtensions: 0,\n\t\t\tNumServices: 1,\n\t\t},\n\t\tGoTypes: file_ObjectStore_proto_goTypes,\n\t\tDependencyIndexes: file_ObjectStore_proto_depIdxs,\n\t\tMessageInfos: file_ObjectStore_proto_msgTypes,\n\t}.Build()\n\tFile_ObjectStore_proto = out.File\n\tfile_ObjectStore_proto_rawDesc = nil\n\tfile_ObjectStore_proto_goTypes = nil\n\tfile_ObjectStore_proto_depIdxs = nil\n}\n" }, { "path": "pkg/plugin/generated/ObjectStore_grpc.pb.go", "content": "// Code generated by protoc-gen-go-grpc. DO NOT EDIT.\n// versions:\n// - protoc-gen-go-grpc v1.3.0\n// - protoc v4.25.2\n// source: ObjectStore.proto\n\npackage generated\n\nimport (\n\tcontext \"context\"\n\tgrpc \"google.golang.org/grpc\"\n\tcodes \"google.golang.org/grpc/codes\"\n\tstatus \"google.golang.org/grpc/status\"\n)\n\n// This is a compile-time assertion to ensure that this generated file\n// is compatible with the grpc package it is being compiled against.\n// Requires gRPC-Go v1.32.0 or later.\nconst _ = grpc.SupportPackageIsVersion7\n\nconst (\n\tObjectStore_Init_FullMethodName = \"/generated.ObjectStore/Init\"\n\tObjectStore_PutObject_FullMethodName = \"/generated.ObjectStore/PutObject\"\n\tObjectStore_ObjectExists_FullMethodName = \"/generated.ObjectStore/ObjectExists\"\n\tObjectStore_GetObject_FullMethodName = \"/generated.ObjectStore/GetObject\"\n\tObjectStore_ListCommonPrefixes_FullMethodName = \"/generated.ObjectStore/ListCommonPrefixes\"\n\tObjectStore_ListObjects_FullMethodName = \"/generated.ObjectStore/ListObjects\"\n\tObjectStore_DeleteObject_FullMethodName = \"/generated.ObjectStore/DeleteObject\"\n\tObjectStore_CreateSignedURL_FullMethodName = \"/generated.ObjectStore/CreateSignedURL\"\n)\n\n// ObjectStoreClient is the client API for ObjectStore service.\n//\n// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.\ntype ObjectStoreClient interface {\n\tInit(ctx context.Context, in *ObjectStoreInitRequest, opts ...grpc.CallOption) (*Empty, error)\n\tPutObject(ctx context.Context, opts ...grpc.CallOption) (ObjectStore_PutObjectClient, error)\n\tObjectExists(ctx context.Context, in *ObjectExistsRequest, opts ...grpc.CallOption) (*ObjectExistsResponse, error)\n\tGetObject(ctx context.Context, in *GetObjectRequest, opts ...grpc.CallOption) (ObjectStore_GetObjectClient, error)\n\tListCommonPrefixes(ctx context.Context, in *ListCommonPrefixesRequest, opts ...grpc.CallOption) (*ListCommonPrefixesResponse, error)\n\tListObjects(ctx context.Context, in *ListObjectsRequest, opts ...grpc.CallOption) (*ListObjectsResponse, error)\n\tDeleteObject(ctx context.Context, in *DeleteObjectRequest, opts ...grpc.CallOption) (*Empty, error)\n\tCreateSignedURL(ctx context.Context, in *CreateSignedURLRequest, opts ...grpc.CallOption) (*CreateSignedURLResponse, error)\n}\n\ntype objectStoreClient struct {\n\tcc grpc.ClientConnInterface\n}\n\nfunc NewObjectStoreClient(cc grpc.ClientConnInterface) ObjectStoreClient {\n\treturn &objectStoreClient{cc}\n}\n\nfunc (c *objectStoreClient) Init(ctx context.Context, in *ObjectStoreInitRequest, opts ...grpc.CallOption) (*Empty, error) {\n\tout := new(Empty)\n\terr := c.cc.Invoke(ctx, ObjectStore_Init_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *objectStoreClient) PutObject(ctx context.Context, opts ...grpc.CallOption) (ObjectStore_PutObjectClient, error) {\n\tstream, err := c.cc.NewStream(ctx, &ObjectStore_ServiceDesc.Streams[0], ObjectStore_PutObject_FullMethodName, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tx := &objectStorePutObjectClient{stream}\n\treturn x, nil\n}\n\ntype ObjectStore_PutObjectClient interface {\n\tSend(*PutObjectRequest) error\n\tCloseAndRecv() (*Empty, error)\n\tgrpc.ClientStream\n}\n\ntype objectStorePutObjectClient struct {\n\tgrpc.ClientStream\n}\n\nfunc (x *objectStorePutObjectClient) Send(m *PutObjectRequest) error {\n\treturn x.ClientStream.SendMsg(m)\n}\n\nfunc (x *objectStorePutObjectClient) CloseAndRecv() (*Empty, error) {\n\tif err := x.ClientStream.CloseSend(); err != nil {\n\t\treturn nil, err\n\t}\n\tm := new(Empty)\n\tif err := x.ClientStream.RecvMsg(m); err != nil {\n\t\treturn nil, err\n\t}\n\treturn m, nil\n}\n\nfunc (c *objectStoreClient) ObjectExists(ctx context.Context, in *ObjectExistsRequest, opts ...grpc.CallOption) (*ObjectExistsResponse, error) {\n\tout := new(ObjectExistsResponse)\n\terr := c.cc.Invoke(ctx, ObjectStore_ObjectExists_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *objectStoreClient) GetObject(ctx context.Context, in *GetObjectRequest, opts ...grpc.CallOption) (ObjectStore_GetObjectClient, error) {\n\tstream, err := c.cc.NewStream(ctx, &ObjectStore_ServiceDesc.Streams[1], ObjectStore_GetObject_FullMethodName, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tx := &objectStoreGetObjectClient{stream}\n\tif err := x.ClientStream.SendMsg(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := x.ClientStream.CloseSend(); err != nil {\n\t\treturn nil, err\n\t}\n\treturn x, nil\n}\n\ntype ObjectStore_GetObjectClient interface {\n\tRecv() (*Bytes, error)\n\tgrpc.ClientStream\n}\n\ntype objectStoreGetObjectClient struct {\n\tgrpc.ClientStream\n}\n\nfunc (x *objectStoreGetObjectClient) Recv() (*Bytes, error) {\n\tm := new(Bytes)\n\tif err := x.ClientStream.RecvMsg(m); err != nil {\n\t\treturn nil, err\n\t}\n\treturn m, nil\n}\n\nfunc (c *objectStoreClient) ListCommonPrefixes(ctx context.Context, in *ListCommonPrefixesRequest, opts ...grpc.CallOption) (*ListCommonPrefixesResponse, error) {\n\tout := new(ListCommonPrefixesResponse)\n\terr := c.cc.Invoke(ctx, ObjectStore_ListCommonPrefixes_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *objectStoreClient) ListObjects(ctx context.Context, in *ListObjectsRequest, opts ...grpc.CallOption) (*ListObjectsResponse, error) {\n\tout := new(ListObjectsResponse)\n\terr := c.cc.Invoke(ctx, ObjectStore_ListObjects_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *objectStoreClient) DeleteObject(ctx context.Context, in *DeleteObjectRequest, opts ...grpc.CallOption) (*Empty, error) {\n\tout := new(Empty)\n\terr := c.cc.Invoke(ctx, ObjectStore_DeleteObject_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *objectStoreClient) CreateSignedURL(ctx context.Context, in *CreateSignedURLRequest, opts ...grpc.CallOption) (*CreateSignedURLResponse, error) {\n\tout := new(CreateSignedURLResponse)\n\terr := c.cc.Invoke(ctx, ObjectStore_CreateSignedURL_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\n// ObjectStoreServer is the server API for ObjectStore service.\n// All implementations should embed UnimplementedObjectStoreServer\n// for forward compatibility\ntype ObjectStoreServer interface {\n\tInit(context.Context, *ObjectStoreInitRequest) (*Empty, error)\n\tPutObject(ObjectStore_PutObjectServer) error\n\tObjectExists(context.Context, *ObjectExistsRequest) (*ObjectExistsResponse, error)\n\tGetObject(*GetObjectRequest, ObjectStore_GetObjectServer) error\n\tListCommonPrefixes(context.Context, *ListCommonPrefixesRequest) (*ListCommonPrefixesResponse, error)\n\tListObjects(context.Context, *ListObjectsRequest) (*ListObjectsResponse, error)\n\tDeleteObject(context.Context, *DeleteObjectRequest) (*Empty, error)\n\tCreateSignedURL(context.Context, *CreateSignedURLRequest) (*CreateSignedURLResponse, error)\n}\n\n// UnimplementedObjectStoreServer should be embedded to have forward compatible implementations.\ntype UnimplementedObjectStoreServer struct {\n}\n\nfunc (UnimplementedObjectStoreServer) Init(context.Context, *ObjectStoreInitRequest) (*Empty, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method Init not implemented\")\n}\nfunc (UnimplementedObjectStoreServer) PutObject(ObjectStore_PutObjectServer) error {\n\treturn status.Errorf(codes.Unimplemented, \"method PutObject not implemented\")\n}\nfunc (UnimplementedObjectStoreServer) ObjectExists(context.Context, *ObjectExistsRequest) (*ObjectExistsResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method ObjectExists not implemented\")\n}\nfunc (UnimplementedObjectStoreServer) GetObject(*GetObjectRequest, ObjectStore_GetObjectServer) error {\n\treturn status.Errorf(codes.Unimplemented, \"method GetObject not implemented\")\n}\nfunc (UnimplementedObjectStoreServer) ListCommonPrefixes(context.Context, *ListCommonPrefixesRequest) (*ListCommonPrefixesResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method ListCommonPrefixes not implemented\")\n}\nfunc (UnimplementedObjectStoreServer) ListObjects(context.Context, *ListObjectsRequest) (*ListObjectsResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method ListObjects not implemented\")\n}\nfunc (UnimplementedObjectStoreServer) DeleteObject(context.Context, *DeleteObjectRequest) (*Empty, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method DeleteObject not implemented\")\n}\nfunc (UnimplementedObjectStoreServer) CreateSignedURL(context.Context, *CreateSignedURLRequest) (*CreateSignedURLResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method CreateSignedURL not implemented\")\n}\n\n// UnsafeObjectStoreServer may be embedded to opt out of forward compatibility for this service.\n// Use of this interface is not recommended, as added methods to ObjectStoreServer will\n// result in compilation errors.\ntype UnsafeObjectStoreServer interface {\n\tmustEmbedUnimplementedObjectStoreServer()\n}\n\nfunc RegisterObjectStoreServer(s grpc.ServiceRegistrar, srv ObjectStoreServer) {\n\ts.RegisterService(&ObjectStore_ServiceDesc, srv)\n}\n\nfunc _ObjectStore_Init_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(ObjectStoreInitRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(ObjectStoreServer).Init(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: ObjectStore_Init_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(ObjectStoreServer).Init(ctx, req.(*ObjectStoreInitRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _ObjectStore_PutObject_Handler(srv interface{}, stream grpc.ServerStream) error {\n\treturn srv.(ObjectStoreServer).PutObject(&objectStorePutObjectServer{stream})\n}\n\ntype ObjectStore_PutObjectServer interface {\n\tSendAndClose(*Empty) error\n\tRecv() (*PutObjectRequest, error)\n\tgrpc.ServerStream\n}\n\ntype objectStorePutObjectServer struct {\n\tgrpc.ServerStream\n}\n\nfunc (x *objectStorePutObjectServer) SendAndClose(m *Empty) error {\n\treturn x.ServerStream.SendMsg(m)\n}\n\nfunc (x *objectStorePutObjectServer) Recv() (*PutObjectRequest, error) {\n\tm := new(PutObjectRequest)\n\tif err := x.ServerStream.RecvMsg(m); err != nil {\n\t\treturn nil, err\n\t}\n\treturn m, nil\n}\n\nfunc _ObjectStore_ObjectExists_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(ObjectExistsRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(ObjectStoreServer).ObjectExists(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: ObjectStore_ObjectExists_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(ObjectStoreServer).ObjectExists(ctx, req.(*ObjectExistsRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _ObjectStore_GetObject_Handler(srv interface{}, stream grpc.ServerStream) error {\n\tm := new(GetObjectRequest)\n\tif err := stream.RecvMsg(m); err != nil {\n\t\treturn err\n\t}\n\treturn srv.(ObjectStoreServer).GetObject(m, &objectStoreGetObjectServer{stream})\n}\n\ntype ObjectStore_GetObjectServer interface {\n\tSend(*Bytes) error\n\tgrpc.ServerStream\n}\n\ntype objectStoreGetObjectServer struct {\n\tgrpc.ServerStream\n}\n\nfunc (x *objectStoreGetObjectServer) Send(m *Bytes) error {\n\treturn x.ServerStream.SendMsg(m)\n}\n\nfunc _ObjectStore_ListCommonPrefixes_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(ListCommonPrefixesRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(ObjectStoreServer).ListCommonPrefixes(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: ObjectStore_ListCommonPrefixes_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(ObjectStoreServer).ListCommonPrefixes(ctx, req.(*ListCommonPrefixesRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _ObjectStore_ListObjects_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(ListObjectsRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(ObjectStoreServer).ListObjects(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: ObjectStore_ListObjects_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(ObjectStoreServer).ListObjects(ctx, req.(*ListObjectsRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _ObjectStore_DeleteObject_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(DeleteObjectRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(ObjectStoreServer).DeleteObject(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: ObjectStore_DeleteObject_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(ObjectStoreServer).DeleteObject(ctx, req.(*DeleteObjectRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _ObjectStore_CreateSignedURL_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(CreateSignedURLRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(ObjectStoreServer).CreateSignedURL(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: ObjectStore_CreateSignedURL_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(ObjectStoreServer).CreateSignedURL(ctx, req.(*CreateSignedURLRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\n// ObjectStore_ServiceDesc is the grpc.ServiceDesc for ObjectStore service.\n// It's only intended for direct use with grpc.RegisterService,\n// and not to be introspected or modified (even as a copy)\nvar ObjectStore_ServiceDesc = grpc.ServiceDesc{\n\tServiceName: \"generated.ObjectStore\",\n\tHandlerType: (*ObjectStoreServer)(nil),\n\tMethods: []grpc.MethodDesc{\n\t\t{\n\t\t\tMethodName: \"Init\",\n\t\t\tHandler: _ObjectStore_Init_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"ObjectExists\",\n\t\t\tHandler: _ObjectStore_ObjectExists_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"ListCommonPrefixes\",\n\t\t\tHandler: _ObjectStore_ListCommonPrefixes_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"ListObjects\",\n\t\t\tHandler: _ObjectStore_ListObjects_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"DeleteObject\",\n\t\t\tHandler: _ObjectStore_DeleteObject_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"CreateSignedURL\",\n\t\t\tHandler: _ObjectStore_CreateSignedURL_Handler,\n\t\t},\n\t},\n\tStreams: []grpc.StreamDesc{\n\t\t{\n\t\t\tStreamName: \"PutObject\",\n\t\t\tHandler: _ObjectStore_PutObject_Handler,\n\t\t\tClientStreams: true,\n\t\t},\n\t\t{\n\t\t\tStreamName: \"GetObject\",\n\t\t\tHandler: _ObjectStore_GetObject_Handler,\n\t\t\tServerStreams: true,\n\t\t},\n\t},\n\tMetadata: \"ObjectStore.proto\",\n}\n" }, { "path": "pkg/plugin/generated/PluginLister.pb.go", "content": "// Code generated by protoc-gen-go. DO NOT EDIT.\n// versions:\n// \tprotoc-gen-go v1.33.0\n// \tprotoc v4.25.2\n// source: PluginLister.proto\n\npackage generated\n\nimport (\n\tprotoreflect \"google.golang.org/protobuf/reflect/protoreflect\"\n\tprotoimpl \"google.golang.org/protobuf/runtime/protoimpl\"\n\treflect \"reflect\"\n\tsync \"sync\"\n)\n\nconst (\n\t// Verify that this generated code is sufficiently up-to-date.\n\t_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)\n\t// Verify that runtime/protoimpl is sufficiently up-to-date.\n\t_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)\n)\n\ntype PluginIdentifier struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tCommand string `protobuf:\"bytes,1,opt,name=command,proto3\" json:\"command,omitempty\"`\n\tKind string `protobuf:\"bytes,2,opt,name=kind,proto3\" json:\"kind,omitempty\"`\n\tName string `protobuf:\"bytes,3,opt,name=name,proto3\" json:\"name,omitempty\"`\n}\n\nfunc (x *PluginIdentifier) Reset() {\n\t*x = PluginIdentifier{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_PluginLister_proto_msgTypes[0]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *PluginIdentifier) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*PluginIdentifier) ProtoMessage() {}\n\nfunc (x *PluginIdentifier) ProtoReflect() protoreflect.Message {\n\tmi := &file_PluginLister_proto_msgTypes[0]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use PluginIdentifier.ProtoReflect.Descriptor instead.\nfunc (*PluginIdentifier) Descriptor() ([]byte, []int) {\n\treturn file_PluginLister_proto_rawDescGZIP(), []int{0}\n}\n\nfunc (x *PluginIdentifier) GetCommand() string {\n\tif x != nil {\n\t\treturn x.Command\n\t}\n\treturn \"\"\n}\n\nfunc (x *PluginIdentifier) GetKind() string {\n\tif x != nil {\n\t\treturn x.Kind\n\t}\n\treturn \"\"\n}\n\nfunc (x *PluginIdentifier) GetName() string {\n\tif x != nil {\n\t\treturn x.Name\n\t}\n\treturn \"\"\n}\n\ntype ListPluginsResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugins []*PluginIdentifier `protobuf:\"bytes,1,rep,name=plugins,proto3\" json:\"plugins,omitempty\"`\n}\n\nfunc (x *ListPluginsResponse) Reset() {\n\t*x = ListPluginsResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_PluginLister_proto_msgTypes[1]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *ListPluginsResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*ListPluginsResponse) ProtoMessage() {}\n\nfunc (x *ListPluginsResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_PluginLister_proto_msgTypes[1]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use ListPluginsResponse.ProtoReflect.Descriptor instead.\nfunc (*ListPluginsResponse) Descriptor() ([]byte, []int) {\n\treturn file_PluginLister_proto_rawDescGZIP(), []int{1}\n}\n\nfunc (x *ListPluginsResponse) GetPlugins() []*PluginIdentifier {\n\tif x != nil {\n\t\treturn x.Plugins\n\t}\n\treturn nil\n}\n\nvar File_PluginLister_proto protoreflect.FileDescriptor\n\nvar file_PluginLister_proto_rawDesc = []byte{\n\t0x0a, 0x12, 0x50, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x4c, 0x69, 0x73, 0x74, 0x65, 0x72, 0x2e, 0x70,\n\t0x72, 0x6f, 0x74, 0x6f, 0x12, 0x09, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x1a,\n\t0x0c, 0x53, 0x68, 0x61, 0x72, 0x65, 0x64, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, 0x54, 0x0a,\n\t0x10, 0x50, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x49, 0x64, 0x65, 0x6e, 0x74, 0x69, 0x66, 0x69, 0x65,\n\t0x72, 0x12, 0x18, 0x0a, 0x07, 0x63, 0x6f, 0x6d, 0x6d, 0x61, 0x6e, 0x64, 0x18, 0x01, 0x20, 0x01,\n\t0x28, 0x09, 0x52, 0x07, 0x63, 0x6f, 0x6d, 0x6d, 0x61, 0x6e, 0x64, 0x12, 0x12, 0x0a, 0x04, 0x6b,\n\t0x69, 0x6e, 0x64, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6b, 0x69, 0x6e, 0x64, 0x12,\n\t0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e,\n\t0x61, 0x6d, 0x65, 0x22, 0x4c, 0x0a, 0x13, 0x4c, 0x69, 0x73, 0x74, 0x50, 0x6c, 0x75, 0x67, 0x69,\n\t0x6e, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x35, 0x0a, 0x07, 0x70, 0x6c,\n\t0x75, 0x67, 0x69, 0x6e, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x1b, 0x2e, 0x67, 0x65,\n\t0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x50, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x49, 0x64,\n\t0x65, 0x6e, 0x74, 0x69, 0x66, 0x69, 0x65, 0x72, 0x52, 0x07, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e,\n\t0x73, 0x32, 0x4f, 0x0a, 0x0c, 0x50, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x4c, 0x69, 0x73, 0x74, 0x65,\n\t0x72, 0x12, 0x3f, 0x0a, 0x0b, 0x4c, 0x69, 0x73, 0x74, 0x50, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x73,\n\t0x12, 0x10, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x45, 0x6d, 0x70,\n\t0x74, 0x79, 0x1a, 0x1e, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x4c,\n\t0x69, 0x73, 0x74, 0x50, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e,\n\t0x73, 0x65, 0x42, 0x35, 0x5a, 0x33, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d,\n\t0x2f, 0x76, 0x6d, 0x77, 0x61, 0x72, 0x65, 0x2d, 0x74, 0x61, 0x6e, 0x7a, 0x75, 0x2f, 0x76, 0x65,\n\t0x6c, 0x65, 0x72, 0x6f, 0x2f, 0x70, 0x6b, 0x67, 0x2f, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x2f,\n\t0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f,\n\t0x33,\n}\n\nvar (\n\tfile_PluginLister_proto_rawDescOnce sync.Once\n\tfile_PluginLister_proto_rawDescData = file_PluginLister_proto_rawDesc\n)\n\nfunc file_PluginLister_proto_rawDescGZIP() []byte {\n\tfile_PluginLister_proto_rawDescOnce.Do(func() {\n\t\tfile_PluginLister_proto_rawDescData = protoimpl.X.CompressGZIP(file_PluginLister_proto_rawDescData)\n\t})\n\treturn file_PluginLister_proto_rawDescData\n}\n\nvar file_PluginLister_proto_msgTypes = make([]protoimpl.MessageInfo, 2)\nvar file_PluginLister_proto_goTypes = []interface{}{\n\t(*PluginIdentifier)(nil), // 0: generated.PluginIdentifier\n\t(*ListPluginsResponse)(nil), // 1: generated.ListPluginsResponse\n\t(*Empty)(nil), // 2: generated.Empty\n}\nvar file_PluginLister_proto_depIdxs = []int32{\n\t0, // 0: generated.ListPluginsResponse.plugins:type_name -> generated.PluginIdentifier\n\t2, // 1: generated.PluginLister.ListPlugins:input_type -> generated.Empty\n\t1, // 2: generated.PluginLister.ListPlugins:output_type -> generated.ListPluginsResponse\n\t2, // [2:3] is the sub-list for method output_type\n\t1, // [1:2] is the sub-list for method input_type\n\t1, // [1:1] is the sub-list for extension type_name\n\t1, // [1:1] is the sub-list for extension extendee\n\t0, // [0:1] is the sub-list for field type_name\n}\n\nfunc init() { file_PluginLister_proto_init() }\nfunc file_PluginLister_proto_init() {\n\tif File_PluginLister_proto != nil {\n\t\treturn\n\t}\n\tfile_Shared_proto_init()\n\tif !protoimpl.UnsafeEnabled {\n\t\tfile_PluginLister_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*PluginIdentifier); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_PluginLister_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*ListPluginsResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t}\n\ttype x struct{}\n\tout := protoimpl.TypeBuilder{\n\t\tFile: protoimpl.DescBuilder{\n\t\t\tGoPackagePath: reflect.TypeOf(x{}).PkgPath(),\n\t\t\tRawDescriptor: file_PluginLister_proto_rawDesc,\n\t\t\tNumEnums: 0,\n\t\t\tNumMessages: 2,\n\t\t\tNumExtensions: 0,\n\t\t\tNumServices: 1,\n\t\t},\n\t\tGoTypes: file_PluginLister_proto_goTypes,\n\t\tDependencyIndexes: file_PluginLister_proto_depIdxs,\n\t\tMessageInfos: file_PluginLister_proto_msgTypes,\n\t}.Build()\n\tFile_PluginLister_proto = out.File\n\tfile_PluginLister_proto_rawDesc = nil\n\tfile_PluginLister_proto_goTypes = nil\n\tfile_PluginLister_proto_depIdxs = nil\n}\n" }, { "path": "pkg/plugin/generated/PluginLister_grpc.pb.go", "content": "// Code generated by protoc-gen-go-grpc. DO NOT EDIT.\n// versions:\n// - protoc-gen-go-grpc v1.3.0\n// - protoc v4.25.2\n// source: PluginLister.proto\n\npackage generated\n\nimport (\n\tcontext \"context\"\n\tgrpc \"google.golang.org/grpc\"\n\tcodes \"google.golang.org/grpc/codes\"\n\tstatus \"google.golang.org/grpc/status\"\n)\n\n// This is a compile-time assertion to ensure that this generated file\n// is compatible with the grpc package it is being compiled against.\n// Requires gRPC-Go v1.32.0 or later.\nconst _ = grpc.SupportPackageIsVersion7\n\nconst (\n\tPluginLister_ListPlugins_FullMethodName = \"/generated.PluginLister/ListPlugins\"\n)\n\n// PluginListerClient is the client API for PluginLister service.\n//\n// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.\ntype PluginListerClient interface {\n\tListPlugins(ctx context.Context, in *Empty, opts ...grpc.CallOption) (*ListPluginsResponse, error)\n}\n\ntype pluginListerClient struct {\n\tcc grpc.ClientConnInterface\n}\n\nfunc NewPluginListerClient(cc grpc.ClientConnInterface) PluginListerClient {\n\treturn &pluginListerClient{cc}\n}\n\nfunc (c *pluginListerClient) ListPlugins(ctx context.Context, in *Empty, opts ...grpc.CallOption) (*ListPluginsResponse, error) {\n\tout := new(ListPluginsResponse)\n\terr := c.cc.Invoke(ctx, PluginLister_ListPlugins_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\n// PluginListerServer is the server API for PluginLister service.\n// All implementations should embed UnimplementedPluginListerServer\n// for forward compatibility\ntype PluginListerServer interface {\n\tListPlugins(context.Context, *Empty) (*ListPluginsResponse, error)\n}\n\n// UnimplementedPluginListerServer should be embedded to have forward compatible implementations.\ntype UnimplementedPluginListerServer struct {\n}\n\nfunc (UnimplementedPluginListerServer) ListPlugins(context.Context, *Empty) (*ListPluginsResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method ListPlugins not implemented\")\n}\n\n// UnsafePluginListerServer may be embedded to opt out of forward compatibility for this service.\n// Use of this interface is not recommended, as added methods to PluginListerServer will\n// result in compilation errors.\ntype UnsafePluginListerServer interface {\n\tmustEmbedUnimplementedPluginListerServer()\n}\n\nfunc RegisterPluginListerServer(s grpc.ServiceRegistrar, srv PluginListerServer) {\n\ts.RegisterService(&PluginLister_ServiceDesc, srv)\n}\n\nfunc _PluginLister_ListPlugins_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(Empty)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(PluginListerServer).ListPlugins(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: PluginLister_ListPlugins_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(PluginListerServer).ListPlugins(ctx, req.(*Empty))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\n// PluginLister_ServiceDesc is the grpc.ServiceDesc for PluginLister service.\n// It's only intended for direct use with grpc.RegisterService,\n// and not to be introspected or modified (even as a copy)\nvar PluginLister_ServiceDesc = grpc.ServiceDesc{\n\tServiceName: \"generated.PluginLister\",\n\tHandlerType: (*PluginListerServer)(nil),\n\tMethods: []grpc.MethodDesc{\n\t\t{\n\t\t\tMethodName: \"ListPlugins\",\n\t\t\tHandler: _PluginLister_ListPlugins_Handler,\n\t\t},\n\t},\n\tStreams: []grpc.StreamDesc{},\n\tMetadata: \"PluginLister.proto\",\n}\n" }, { "path": "pkg/plugin/generated/RestoreItemAction.pb.go", "content": "// Code generated by protoc-gen-go. DO NOT EDIT.\n// versions:\n// \tprotoc-gen-go v1.33.0\n// \tprotoc v4.25.2\n// source: RestoreItemAction.proto\n\npackage generated\n\nimport (\n\tprotoreflect \"google.golang.org/protobuf/reflect/protoreflect\"\n\tprotoimpl \"google.golang.org/protobuf/runtime/protoimpl\"\n\treflect \"reflect\"\n\tsync \"sync\"\n)\n\nconst (\n\t// Verify that this generated code is sufficiently up-to-date.\n\t_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)\n\t// Verify that runtime/protoimpl is sufficiently up-to-date.\n\t_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)\n)\n\ntype RestoreItemActionExecuteRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tItem []byte `protobuf:\"bytes,2,opt,name=item,proto3\" json:\"item,omitempty\"`\n\tRestore []byte `protobuf:\"bytes,3,opt,name=restore,proto3\" json:\"restore,omitempty\"`\n\tItemFromBackup []byte `protobuf:\"bytes,4,opt,name=itemFromBackup,proto3\" json:\"itemFromBackup,omitempty\"`\n}\n\nfunc (x *RestoreItemActionExecuteRequest) Reset() {\n\t*x = RestoreItemActionExecuteRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_RestoreItemAction_proto_msgTypes[0]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *RestoreItemActionExecuteRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*RestoreItemActionExecuteRequest) ProtoMessage() {}\n\nfunc (x *RestoreItemActionExecuteRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_RestoreItemAction_proto_msgTypes[0]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use RestoreItemActionExecuteRequest.ProtoReflect.Descriptor instead.\nfunc (*RestoreItemActionExecuteRequest) Descriptor() ([]byte, []int) {\n\treturn file_RestoreItemAction_proto_rawDescGZIP(), []int{0}\n}\n\nfunc (x *RestoreItemActionExecuteRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *RestoreItemActionExecuteRequest) GetItem() []byte {\n\tif x != nil {\n\t\treturn x.Item\n\t}\n\treturn nil\n}\n\nfunc (x *RestoreItemActionExecuteRequest) GetRestore() []byte {\n\tif x != nil {\n\t\treturn x.Restore\n\t}\n\treturn nil\n}\n\nfunc (x *RestoreItemActionExecuteRequest) GetItemFromBackup() []byte {\n\tif x != nil {\n\t\treturn x.ItemFromBackup\n\t}\n\treturn nil\n}\n\ntype RestoreItemActionExecuteResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tItem []byte `protobuf:\"bytes,1,opt,name=item,proto3\" json:\"item,omitempty\"`\n\tAdditionalItems []*ResourceIdentifier `protobuf:\"bytes,2,rep,name=additionalItems,proto3\" json:\"additionalItems,omitempty\"`\n\tSkipRestore bool `protobuf:\"varint,3,opt,name=skipRestore,proto3\" json:\"skipRestore,omitempty\"`\n}\n\nfunc (x *RestoreItemActionExecuteResponse) Reset() {\n\t*x = RestoreItemActionExecuteResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_RestoreItemAction_proto_msgTypes[1]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *RestoreItemActionExecuteResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*RestoreItemActionExecuteResponse) ProtoMessage() {}\n\nfunc (x *RestoreItemActionExecuteResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_RestoreItemAction_proto_msgTypes[1]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use RestoreItemActionExecuteResponse.ProtoReflect.Descriptor instead.\nfunc (*RestoreItemActionExecuteResponse) Descriptor() ([]byte, []int) {\n\treturn file_RestoreItemAction_proto_rawDescGZIP(), []int{1}\n}\n\nfunc (x *RestoreItemActionExecuteResponse) GetItem() []byte {\n\tif x != nil {\n\t\treturn x.Item\n\t}\n\treturn nil\n}\n\nfunc (x *RestoreItemActionExecuteResponse) GetAdditionalItems() []*ResourceIdentifier {\n\tif x != nil {\n\t\treturn x.AdditionalItems\n\t}\n\treturn nil\n}\n\nfunc (x *RestoreItemActionExecuteResponse) GetSkipRestore() bool {\n\tif x != nil {\n\t\treturn x.SkipRestore\n\t}\n\treturn false\n}\n\ntype RestoreItemActionAppliesToRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n}\n\nfunc (x *RestoreItemActionAppliesToRequest) Reset() {\n\t*x = RestoreItemActionAppliesToRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_RestoreItemAction_proto_msgTypes[2]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *RestoreItemActionAppliesToRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*RestoreItemActionAppliesToRequest) ProtoMessage() {}\n\nfunc (x *RestoreItemActionAppliesToRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_RestoreItemAction_proto_msgTypes[2]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use RestoreItemActionAppliesToRequest.ProtoReflect.Descriptor instead.\nfunc (*RestoreItemActionAppliesToRequest) Descriptor() ([]byte, []int) {\n\treturn file_RestoreItemAction_proto_rawDescGZIP(), []int{2}\n}\n\nfunc (x *RestoreItemActionAppliesToRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\ntype RestoreItemActionAppliesToResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tResourceSelector *ResourceSelector `protobuf:\"bytes,1,opt,name=ResourceSelector,proto3\" json:\"ResourceSelector,omitempty\"`\n}\n\nfunc (x *RestoreItemActionAppliesToResponse) Reset() {\n\t*x = RestoreItemActionAppliesToResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_RestoreItemAction_proto_msgTypes[3]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *RestoreItemActionAppliesToResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*RestoreItemActionAppliesToResponse) ProtoMessage() {}\n\nfunc (x *RestoreItemActionAppliesToResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_RestoreItemAction_proto_msgTypes[3]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use RestoreItemActionAppliesToResponse.ProtoReflect.Descriptor instead.\nfunc (*RestoreItemActionAppliesToResponse) Descriptor() ([]byte, []int) {\n\treturn file_RestoreItemAction_proto_rawDescGZIP(), []int{3}\n}\n\nfunc (x *RestoreItemActionAppliesToResponse) GetResourceSelector() *ResourceSelector {\n\tif x != nil {\n\t\treturn x.ResourceSelector\n\t}\n\treturn nil\n}\n\nvar File_RestoreItemAction_proto protoreflect.FileDescriptor\n\nvar file_RestoreItemAction_proto_rawDesc = []byte{\n\t0x0a, 0x17, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74,\n\t0x69, 0x6f, 0x6e, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x09, 0x67, 0x65, 0x6e, 0x65, 0x72,\n\t0x61, 0x74, 0x65, 0x64, 0x1a, 0x0c, 0x53, 0x68, 0x61, 0x72, 0x65, 0x64, 0x2e, 0x70, 0x72, 0x6f,\n\t0x74, 0x6f, 0x22, 0x8f, 0x01, 0x0a, 0x1f, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x49, 0x74,\n\t0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x45, 0x78, 0x65, 0x63, 0x75, 0x74, 0x65, 0x52,\n\t0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x16, 0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e,\n\t0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x12, 0x12,\n\t0x0a, 0x04, 0x69, 0x74, 0x65, 0x6d, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x04, 0x69, 0x74,\n\t0x65, 0x6d, 0x12, 0x18, 0x0a, 0x07, 0x72, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x18, 0x03, 0x20,\n\t0x01, 0x28, 0x0c, 0x52, 0x07, 0x72, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x12, 0x26, 0x0a, 0x0e,\n\t0x69, 0x74, 0x65, 0x6d, 0x46, 0x72, 0x6f, 0x6d, 0x42, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x18, 0x04,\n\t0x20, 0x01, 0x28, 0x0c, 0x52, 0x0e, 0x69, 0x74, 0x65, 0x6d, 0x46, 0x72, 0x6f, 0x6d, 0x42, 0x61,\n\t0x63, 0x6b, 0x75, 0x70, 0x22, 0xa1, 0x01, 0x0a, 0x20, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65,\n\t0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x45, 0x78, 0x65, 0x63, 0x75, 0x74,\n\t0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x69, 0x74, 0x65,\n\t0x6d, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x04, 0x69, 0x74, 0x65, 0x6d, 0x12, 0x47, 0x0a,\n\t0x0f, 0x61, 0x64, 0x64, 0x69, 0x74, 0x69, 0x6f, 0x6e, 0x61, 0x6c, 0x49, 0x74, 0x65, 0x6d, 0x73,\n\t0x18, 0x02, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x1d, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74,\n\t0x65, 0x64, 0x2e, 0x52, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x49, 0x64, 0x65, 0x6e, 0x74,\n\t0x69, 0x66, 0x69, 0x65, 0x72, 0x52, 0x0f, 0x61, 0x64, 0x64, 0x69, 0x74, 0x69, 0x6f, 0x6e, 0x61,\n\t0x6c, 0x49, 0x74, 0x65, 0x6d, 0x73, 0x12, 0x20, 0x0a, 0x0b, 0x73, 0x6b, 0x69, 0x70, 0x52, 0x65,\n\t0x73, 0x74, 0x6f, 0x72, 0x65, 0x18, 0x03, 0x20, 0x01, 0x28, 0x08, 0x52, 0x0b, 0x73, 0x6b, 0x69,\n\t0x70, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x22, 0x3b, 0x0a, 0x21, 0x52, 0x65, 0x73, 0x74,\n\t0x6f, 0x72, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x41, 0x70, 0x70,\n\t0x6c, 0x69, 0x65, 0x73, 0x54, 0x6f, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x16, 0x0a,\n\t0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x70,\n\t0x6c, 0x75, 0x67, 0x69, 0x6e, 0x22, 0x6d, 0x0a, 0x22, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65,\n\t0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x41, 0x70, 0x70, 0x6c, 0x69, 0x65,\n\t0x73, 0x54, 0x6f, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x47, 0x0a, 0x10, 0x52,\n\t0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x53, 0x65, 0x6c, 0x65, 0x63, 0x74, 0x6f, 0x72, 0x18,\n\t0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1b, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65,\n\t0x64, 0x2e, 0x52, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x53, 0x65, 0x6c, 0x65, 0x63, 0x74,\n\t0x6f, 0x72, 0x52, 0x10, 0x52, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x53, 0x65, 0x6c, 0x65,\n\t0x63, 0x74, 0x6f, 0x72, 0x32, 0xe1, 0x01, 0x0a, 0x11, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65,\n\t0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x68, 0x0a, 0x09, 0x41, 0x70,\n\t0x70, 0x6c, 0x69, 0x65, 0x73, 0x54, 0x6f, 0x12, 0x2c, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61,\n\t0x74, 0x65, 0x64, 0x2e, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41,\n\t0x63, 0x74, 0x69, 0x6f, 0x6e, 0x41, 0x70, 0x70, 0x6c, 0x69, 0x65, 0x73, 0x54, 0x6f, 0x52, 0x65,\n\t0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2d, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65,\n\t0x64, 0x2e, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74,\n\t0x69, 0x6f, 0x6e, 0x41, 0x70, 0x70, 0x6c, 0x69, 0x65, 0x73, 0x54, 0x6f, 0x52, 0x65, 0x73, 0x70,\n\t0x6f, 0x6e, 0x73, 0x65, 0x12, 0x62, 0x0a, 0x07, 0x45, 0x78, 0x65, 0x63, 0x75, 0x74, 0x65, 0x12,\n\t0x2a, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x52, 0x65, 0x73, 0x74,\n\t0x6f, 0x72, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x45, 0x78, 0x65,\n\t0x63, 0x75, 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2b, 0x2e, 0x67, 0x65,\n\t0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x49,\n\t0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x45, 0x78, 0x65, 0x63, 0x75, 0x74, 0x65,\n\t0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x42, 0x35, 0x5a, 0x33, 0x67, 0x69, 0x74, 0x68,\n\t0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x76, 0x6d, 0x77, 0x61, 0x72, 0x65, 0x2d, 0x74, 0x61,\n\t0x6e, 0x7a, 0x75, 0x2f, 0x76, 0x65, 0x6c, 0x65, 0x72, 0x6f, 0x2f, 0x70, 0x6b, 0x67, 0x2f, 0x70,\n\t0x6c, 0x75, 0x67, 0x69, 0x6e, 0x2f, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x62,\n\t0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,\n}\n\nvar (\n\tfile_RestoreItemAction_proto_rawDescOnce sync.Once\n\tfile_RestoreItemAction_proto_rawDescData = file_RestoreItemAction_proto_rawDesc\n)\n\nfunc file_RestoreItemAction_proto_rawDescGZIP() []byte {\n\tfile_RestoreItemAction_proto_rawDescOnce.Do(func() {\n\t\tfile_RestoreItemAction_proto_rawDescData = protoimpl.X.CompressGZIP(file_RestoreItemAction_proto_rawDescData)\n\t})\n\treturn file_RestoreItemAction_proto_rawDescData\n}\n\nvar file_RestoreItemAction_proto_msgTypes = make([]protoimpl.MessageInfo, 4)\nvar file_RestoreItemAction_proto_goTypes = []interface{}{\n\t(*RestoreItemActionExecuteRequest)(nil), // 0: generated.RestoreItemActionExecuteRequest\n\t(*RestoreItemActionExecuteResponse)(nil), // 1: generated.RestoreItemActionExecuteResponse\n\t(*RestoreItemActionAppliesToRequest)(nil), // 2: generated.RestoreItemActionAppliesToRequest\n\t(*RestoreItemActionAppliesToResponse)(nil), // 3: generated.RestoreItemActionAppliesToResponse\n\t(*ResourceIdentifier)(nil), // 4: generated.ResourceIdentifier\n\t(*ResourceSelector)(nil), // 5: generated.ResourceSelector\n}\nvar file_RestoreItemAction_proto_depIdxs = []int32{\n\t4, // 0: generated.RestoreItemActionExecuteResponse.additionalItems:type_name -> generated.ResourceIdentifier\n\t5, // 1: generated.RestoreItemActionAppliesToResponse.ResourceSelector:type_name -> generated.ResourceSelector\n\t2, // 2: generated.RestoreItemAction.AppliesTo:input_type -> generated.RestoreItemActionAppliesToRequest\n\t0, // 3: generated.RestoreItemAction.Execute:input_type -> generated.RestoreItemActionExecuteRequest\n\t3, // 4: generated.RestoreItemAction.AppliesTo:output_type -> generated.RestoreItemActionAppliesToResponse\n\t1, // 5: generated.RestoreItemAction.Execute:output_type -> generated.RestoreItemActionExecuteResponse\n\t4, // [4:6] is the sub-list for method output_type\n\t2, // [2:4] is the sub-list for method input_type\n\t2, // [2:2] is the sub-list for extension type_name\n\t2, // [2:2] is the sub-list for extension extendee\n\t0, // [0:2] is the sub-list for field type_name\n}\n\nfunc init() { file_RestoreItemAction_proto_init() }\nfunc file_RestoreItemAction_proto_init() {\n\tif File_RestoreItemAction_proto != nil {\n\t\treturn\n\t}\n\tfile_Shared_proto_init()\n\tif !protoimpl.UnsafeEnabled {\n\t\tfile_RestoreItemAction_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*RestoreItemActionExecuteRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_RestoreItemAction_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*RestoreItemActionExecuteResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_RestoreItemAction_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*RestoreItemActionAppliesToRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_RestoreItemAction_proto_msgTypes[3].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*RestoreItemActionAppliesToResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t}\n\ttype x struct{}\n\tout := protoimpl.TypeBuilder{\n\t\tFile: protoimpl.DescBuilder{\n\t\t\tGoPackagePath: reflect.TypeOf(x{}).PkgPath(),\n\t\t\tRawDescriptor: file_RestoreItemAction_proto_rawDesc,\n\t\t\tNumEnums: 0,\n\t\t\tNumMessages: 4,\n\t\t\tNumExtensions: 0,\n\t\t\tNumServices: 1,\n\t\t},\n\t\tGoTypes: file_RestoreItemAction_proto_goTypes,\n\t\tDependencyIndexes: file_RestoreItemAction_proto_depIdxs,\n\t\tMessageInfos: file_RestoreItemAction_proto_msgTypes,\n\t}.Build()\n\tFile_RestoreItemAction_proto = out.File\n\tfile_RestoreItemAction_proto_rawDesc = nil\n\tfile_RestoreItemAction_proto_goTypes = nil\n\tfile_RestoreItemAction_proto_depIdxs = nil\n}\n" }, { "path": "pkg/plugin/generated/RestoreItemAction_grpc.pb.go", "content": "// Code generated by protoc-gen-go-grpc. DO NOT EDIT.\n// versions:\n// - protoc-gen-go-grpc v1.3.0\n// - protoc v4.25.2\n// source: RestoreItemAction.proto\n\npackage generated\n\nimport (\n\tcontext \"context\"\n\tgrpc \"google.golang.org/grpc\"\n\tcodes \"google.golang.org/grpc/codes\"\n\tstatus \"google.golang.org/grpc/status\"\n)\n\n// This is a compile-time assertion to ensure that this generated file\n// is compatible with the grpc package it is being compiled against.\n// Requires gRPC-Go v1.32.0 or later.\nconst _ = grpc.SupportPackageIsVersion7\n\nconst (\n\tRestoreItemAction_AppliesTo_FullMethodName = \"/generated.RestoreItemAction/AppliesTo\"\n\tRestoreItemAction_Execute_FullMethodName = \"/generated.RestoreItemAction/Execute\"\n)\n\n// RestoreItemActionClient is the client API for RestoreItemAction service.\n//\n// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.\ntype RestoreItemActionClient interface {\n\tAppliesTo(ctx context.Context, in *RestoreItemActionAppliesToRequest, opts ...grpc.CallOption) (*RestoreItemActionAppliesToResponse, error)\n\tExecute(ctx context.Context, in *RestoreItemActionExecuteRequest, opts ...grpc.CallOption) (*RestoreItemActionExecuteResponse, error)\n}\n\ntype restoreItemActionClient struct {\n\tcc grpc.ClientConnInterface\n}\n\nfunc NewRestoreItemActionClient(cc grpc.ClientConnInterface) RestoreItemActionClient {\n\treturn &restoreItemActionClient{cc}\n}\n\nfunc (c *restoreItemActionClient) AppliesTo(ctx context.Context, in *RestoreItemActionAppliesToRequest, opts ...grpc.CallOption) (*RestoreItemActionAppliesToResponse, error) {\n\tout := new(RestoreItemActionAppliesToResponse)\n\terr := c.cc.Invoke(ctx, RestoreItemAction_AppliesTo_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *restoreItemActionClient) Execute(ctx context.Context, in *RestoreItemActionExecuteRequest, opts ...grpc.CallOption) (*RestoreItemActionExecuteResponse, error) {\n\tout := new(RestoreItemActionExecuteResponse)\n\terr := c.cc.Invoke(ctx, RestoreItemAction_Execute_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\n// RestoreItemActionServer is the server API for RestoreItemAction service.\n// All implementations should embed UnimplementedRestoreItemActionServer\n// for forward compatibility\ntype RestoreItemActionServer interface {\n\tAppliesTo(context.Context, *RestoreItemActionAppliesToRequest) (*RestoreItemActionAppliesToResponse, error)\n\tExecute(context.Context, *RestoreItemActionExecuteRequest) (*RestoreItemActionExecuteResponse, error)\n}\n\n// UnimplementedRestoreItemActionServer should be embedded to have forward compatible implementations.\ntype UnimplementedRestoreItemActionServer struct {\n}\n\nfunc (UnimplementedRestoreItemActionServer) AppliesTo(context.Context, *RestoreItemActionAppliesToRequest) (*RestoreItemActionAppliesToResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method AppliesTo not implemented\")\n}\nfunc (UnimplementedRestoreItemActionServer) Execute(context.Context, *RestoreItemActionExecuteRequest) (*RestoreItemActionExecuteResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method Execute not implemented\")\n}\n\n// UnsafeRestoreItemActionServer may be embedded to opt out of forward compatibility for this service.\n// Use of this interface is not recommended, as added methods to RestoreItemActionServer will\n// result in compilation errors.\ntype UnsafeRestoreItemActionServer interface {\n\tmustEmbedUnimplementedRestoreItemActionServer()\n}\n\nfunc RegisterRestoreItemActionServer(s grpc.ServiceRegistrar, srv RestoreItemActionServer) {\n\ts.RegisterService(&RestoreItemAction_ServiceDesc, srv)\n}\n\nfunc _RestoreItemAction_AppliesTo_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(RestoreItemActionAppliesToRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(RestoreItemActionServer).AppliesTo(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: RestoreItemAction_AppliesTo_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(RestoreItemActionServer).AppliesTo(ctx, req.(*RestoreItemActionAppliesToRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _RestoreItemAction_Execute_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(RestoreItemActionExecuteRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(RestoreItemActionServer).Execute(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: RestoreItemAction_Execute_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(RestoreItemActionServer).Execute(ctx, req.(*RestoreItemActionExecuteRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\n// RestoreItemAction_ServiceDesc is the grpc.ServiceDesc for RestoreItemAction service.\n// It's only intended for direct use with grpc.RegisterService,\n// and not to be introspected or modified (even as a copy)\nvar RestoreItemAction_ServiceDesc = grpc.ServiceDesc{\n\tServiceName: \"generated.RestoreItemAction\",\n\tHandlerType: (*RestoreItemActionServer)(nil),\n\tMethods: []grpc.MethodDesc{\n\t\t{\n\t\t\tMethodName: \"AppliesTo\",\n\t\t\tHandler: _RestoreItemAction_AppliesTo_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"Execute\",\n\t\t\tHandler: _RestoreItemAction_Execute_Handler,\n\t\t},\n\t},\n\tStreams: []grpc.StreamDesc{},\n\tMetadata: \"RestoreItemAction.proto\",\n}\n" }, { "path": "pkg/plugin/generated/Shared.pb.go", "content": "// Code generated by protoc-gen-go. DO NOT EDIT.\n// versions:\n// \tprotoc-gen-go v1.33.0\n// \tprotoc v4.25.2\n// source: Shared.proto\n\npackage generated\n\nimport (\n\tprotoreflect \"google.golang.org/protobuf/reflect/protoreflect\"\n\tprotoimpl \"google.golang.org/protobuf/runtime/protoimpl\"\n\ttimestamppb \"google.golang.org/protobuf/types/known/timestamppb\"\n\treflect \"reflect\"\n\tsync \"sync\"\n)\n\nconst (\n\t// Verify that this generated code is sufficiently up-to-date.\n\t_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)\n\t// Verify that runtime/protoimpl is sufficiently up-to-date.\n\t_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)\n)\n\ntype Empty struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n}\n\nfunc (x *Empty) Reset() {\n\t*x = Empty{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_Shared_proto_msgTypes[0]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *Empty) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*Empty) ProtoMessage() {}\n\nfunc (x *Empty) ProtoReflect() protoreflect.Message {\n\tmi := &file_Shared_proto_msgTypes[0]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use Empty.ProtoReflect.Descriptor instead.\nfunc (*Empty) Descriptor() ([]byte, []int) {\n\treturn file_Shared_proto_rawDescGZIP(), []int{0}\n}\n\ntype Stack struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tFrames []*StackFrame `protobuf:\"bytes,1,rep,name=frames,proto3\" json:\"frames,omitempty\"`\n}\n\nfunc (x *Stack) Reset() {\n\t*x = Stack{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_Shared_proto_msgTypes[1]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *Stack) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*Stack) ProtoMessage() {}\n\nfunc (x *Stack) ProtoReflect() protoreflect.Message {\n\tmi := &file_Shared_proto_msgTypes[1]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use Stack.ProtoReflect.Descriptor instead.\nfunc (*Stack) Descriptor() ([]byte, []int) {\n\treturn file_Shared_proto_rawDescGZIP(), []int{1}\n}\n\nfunc (x *Stack) GetFrames() []*StackFrame {\n\tif x != nil {\n\t\treturn x.Frames\n\t}\n\treturn nil\n}\n\ntype StackFrame struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tFile string `protobuf:\"bytes,1,opt,name=file,proto3\" json:\"file,omitempty\"`\n\tLine int32 `protobuf:\"varint,2,opt,name=line,proto3\" json:\"line,omitempty\"`\n\tFunction string `protobuf:\"bytes,3,opt,name=function,proto3\" json:\"function,omitempty\"`\n}\n\nfunc (x *StackFrame) Reset() {\n\t*x = StackFrame{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_Shared_proto_msgTypes[2]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *StackFrame) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*StackFrame) ProtoMessage() {}\n\nfunc (x *StackFrame) ProtoReflect() protoreflect.Message {\n\tmi := &file_Shared_proto_msgTypes[2]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use StackFrame.ProtoReflect.Descriptor instead.\nfunc (*StackFrame) Descriptor() ([]byte, []int) {\n\treturn file_Shared_proto_rawDescGZIP(), []int{2}\n}\n\nfunc (x *StackFrame) GetFile() string {\n\tif x != nil {\n\t\treturn x.File\n\t}\n\treturn \"\"\n}\n\nfunc (x *StackFrame) GetLine() int32 {\n\tif x != nil {\n\t\treturn x.Line\n\t}\n\treturn 0\n}\n\nfunc (x *StackFrame) GetFunction() string {\n\tif x != nil {\n\t\treturn x.Function\n\t}\n\treturn \"\"\n}\n\ntype ResourceIdentifier struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tGroup string `protobuf:\"bytes,1,opt,name=group,proto3\" json:\"group,omitempty\"`\n\tResource string `protobuf:\"bytes,2,opt,name=resource,proto3\" json:\"resource,omitempty\"`\n\tNamespace string `protobuf:\"bytes,3,opt,name=namespace,proto3\" json:\"namespace,omitempty\"`\n\tName string `protobuf:\"bytes,4,opt,name=name,proto3\" json:\"name,omitempty\"`\n}\n\nfunc (x *ResourceIdentifier) Reset() {\n\t*x = ResourceIdentifier{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_Shared_proto_msgTypes[3]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *ResourceIdentifier) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*ResourceIdentifier) ProtoMessage() {}\n\nfunc (x *ResourceIdentifier) ProtoReflect() protoreflect.Message {\n\tmi := &file_Shared_proto_msgTypes[3]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use ResourceIdentifier.ProtoReflect.Descriptor instead.\nfunc (*ResourceIdentifier) Descriptor() ([]byte, []int) {\n\treturn file_Shared_proto_rawDescGZIP(), []int{3}\n}\n\nfunc (x *ResourceIdentifier) GetGroup() string {\n\tif x != nil {\n\t\treturn x.Group\n\t}\n\treturn \"\"\n}\n\nfunc (x *ResourceIdentifier) GetResource() string {\n\tif x != nil {\n\t\treturn x.Resource\n\t}\n\treturn \"\"\n}\n\nfunc (x *ResourceIdentifier) GetNamespace() string {\n\tif x != nil {\n\t\treturn x.Namespace\n\t}\n\treturn \"\"\n}\n\nfunc (x *ResourceIdentifier) GetName() string {\n\tif x != nil {\n\t\treturn x.Name\n\t}\n\treturn \"\"\n}\n\ntype ResourceSelector struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tIncludedNamespaces []string `protobuf:\"bytes,1,rep,name=includedNamespaces,proto3\" json:\"includedNamespaces,omitempty\"`\n\tExcludedNamespaces []string `protobuf:\"bytes,2,rep,name=excludedNamespaces,proto3\" json:\"excludedNamespaces,omitempty\"`\n\tIncludedResources []string `protobuf:\"bytes,3,rep,name=includedResources,proto3\" json:\"includedResources,omitempty\"`\n\tExcludedResources []string `protobuf:\"bytes,4,rep,name=excludedResources,proto3\" json:\"excludedResources,omitempty\"`\n\tSelector string `protobuf:\"bytes,5,opt,name=selector,proto3\" json:\"selector,omitempty\"`\n}\n\nfunc (x *ResourceSelector) Reset() {\n\t*x = ResourceSelector{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_Shared_proto_msgTypes[4]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *ResourceSelector) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*ResourceSelector) ProtoMessage() {}\n\nfunc (x *ResourceSelector) ProtoReflect() protoreflect.Message {\n\tmi := &file_Shared_proto_msgTypes[4]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use ResourceSelector.ProtoReflect.Descriptor instead.\nfunc (*ResourceSelector) Descriptor() ([]byte, []int) {\n\treturn file_Shared_proto_rawDescGZIP(), []int{4}\n}\n\nfunc (x *ResourceSelector) GetIncludedNamespaces() []string {\n\tif x != nil {\n\t\treturn x.IncludedNamespaces\n\t}\n\treturn nil\n}\n\nfunc (x *ResourceSelector) GetExcludedNamespaces() []string {\n\tif x != nil {\n\t\treturn x.ExcludedNamespaces\n\t}\n\treturn nil\n}\n\nfunc (x *ResourceSelector) GetIncludedResources() []string {\n\tif x != nil {\n\t\treturn x.IncludedResources\n\t}\n\treturn nil\n}\n\nfunc (x *ResourceSelector) GetExcludedResources() []string {\n\tif x != nil {\n\t\treturn x.ExcludedResources\n\t}\n\treturn nil\n}\n\nfunc (x *ResourceSelector) GetSelector() string {\n\tif x != nil {\n\t\treturn x.Selector\n\t}\n\treturn \"\"\n}\n\ntype OperationProgress struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tCompleted bool `protobuf:\"varint,1,opt,name=completed,proto3\" json:\"completed,omitempty\"`\n\tErr string `protobuf:\"bytes,2,opt,name=err,proto3\" json:\"err,omitempty\"`\n\tNCompleted int64 `protobuf:\"varint,3,opt,name=nCompleted,proto3\" json:\"nCompleted,omitempty\"`\n\tNTotal int64 `protobuf:\"varint,4,opt,name=nTotal,proto3\" json:\"nTotal,omitempty\"`\n\tOperationUnits string `protobuf:\"bytes,5,opt,name=operationUnits,proto3\" json:\"operationUnits,omitempty\"`\n\tDescription string `protobuf:\"bytes,6,opt,name=description,proto3\" json:\"description,omitempty\"`\n\tStarted *timestamppb.Timestamp `protobuf:\"bytes,7,opt,name=started,proto3\" json:\"started,omitempty\"`\n\tUpdated *timestamppb.Timestamp `protobuf:\"bytes,8,opt,name=updated,proto3\" json:\"updated,omitempty\"`\n}\n\nfunc (x *OperationProgress) Reset() {\n\t*x = OperationProgress{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_Shared_proto_msgTypes[5]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *OperationProgress) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*OperationProgress) ProtoMessage() {}\n\nfunc (x *OperationProgress) ProtoReflect() protoreflect.Message {\n\tmi := &file_Shared_proto_msgTypes[5]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use OperationProgress.ProtoReflect.Descriptor instead.\nfunc (*OperationProgress) Descriptor() ([]byte, []int) {\n\treturn file_Shared_proto_rawDescGZIP(), []int{5}\n}\n\nfunc (x *OperationProgress) GetCompleted() bool {\n\tif x != nil {\n\t\treturn x.Completed\n\t}\n\treturn false\n}\n\nfunc (x *OperationProgress) GetErr() string {\n\tif x != nil {\n\t\treturn x.Err\n\t}\n\treturn \"\"\n}\n\nfunc (x *OperationProgress) GetNCompleted() int64 {\n\tif x != nil {\n\t\treturn x.NCompleted\n\t}\n\treturn 0\n}\n\nfunc (x *OperationProgress) GetNTotal() int64 {\n\tif x != nil {\n\t\treturn x.NTotal\n\t}\n\treturn 0\n}\n\nfunc (x *OperationProgress) GetOperationUnits() string {\n\tif x != nil {\n\t\treturn x.OperationUnits\n\t}\n\treturn \"\"\n}\n\nfunc (x *OperationProgress) GetDescription() string {\n\tif x != nil {\n\t\treturn x.Description\n\t}\n\treturn \"\"\n}\n\nfunc (x *OperationProgress) GetStarted() *timestamppb.Timestamp {\n\tif x != nil {\n\t\treturn x.Started\n\t}\n\treturn nil\n}\n\nfunc (x *OperationProgress) GetUpdated() *timestamppb.Timestamp {\n\tif x != nil {\n\t\treturn x.Updated\n\t}\n\treturn nil\n}\n\nvar File_Shared_proto protoreflect.FileDescriptor\n\nvar file_Shared_proto_rawDesc = []byte{\n\t0x0a, 0x0c, 0x53, 0x68, 0x61, 0x72, 0x65, 0x64, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x09,\n\t0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x1a, 0x1f, 0x67, 0x6f, 0x6f, 0x67, 0x6c,\n\t0x65, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2f, 0x74, 0x69, 0x6d, 0x65, 0x73,\n\t0x74, 0x61, 0x6d, 0x70, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, 0x07, 0x0a, 0x05, 0x45, 0x6d,\n\t0x70, 0x74, 0x79, 0x22, 0x36, 0x0a, 0x05, 0x53, 0x74, 0x61, 0x63, 0x6b, 0x12, 0x2d, 0x0a, 0x06,\n\t0x66, 0x72, 0x61, 0x6d, 0x65, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x15, 0x2e, 0x67,\n\t0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x53, 0x74, 0x61, 0x63, 0x6b, 0x46, 0x72,\n\t0x61, 0x6d, 0x65, 0x52, 0x06, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x73, 0x22, 0x50, 0x0a, 0x0a, 0x53,\n\t0x74, 0x61, 0x63, 0x6b, 0x46, 0x72, 0x61, 0x6d, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x66, 0x69, 0x6c,\n\t0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x66, 0x69, 0x6c, 0x65, 0x12, 0x12, 0x0a,\n\t0x04, 0x6c, 0x69, 0x6e, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x05, 0x52, 0x04, 0x6c, 0x69, 0x6e,\n\t0x65, 0x12, 0x1a, 0x0a, 0x08, 0x66, 0x75, 0x6e, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x18, 0x03, 0x20,\n\t0x01, 0x28, 0x09, 0x52, 0x08, 0x66, 0x75, 0x6e, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x22, 0x78, 0x0a,\n\t0x12, 0x52, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x49, 0x64, 0x65, 0x6e, 0x74, 0x69, 0x66,\n\t0x69, 0x65, 0x72, 0x12, 0x14, 0x0a, 0x05, 0x67, 0x72, 0x6f, 0x75, 0x70, 0x18, 0x01, 0x20, 0x01,\n\t0x28, 0x09, 0x52, 0x05, 0x67, 0x72, 0x6f, 0x75, 0x70, 0x12, 0x1a, 0x0a, 0x08, 0x72, 0x65, 0x73,\n\t0x6f, 0x75, 0x72, 0x63, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x72, 0x65, 0x73,\n\t0x6f, 0x75, 0x72, 0x63, 0x65, 0x12, 0x1c, 0x0a, 0x09, 0x6e, 0x61, 0x6d, 0x65, 0x73, 0x70, 0x61,\n\t0x63, 0x65, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0x6e, 0x61, 0x6d, 0x65, 0x73, 0x70,\n\t0x61, 0x63, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x04, 0x20, 0x01, 0x28,\n\t0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x22, 0xea, 0x01, 0x0a, 0x10, 0x52, 0x65, 0x73, 0x6f,\n\t0x75, 0x72, 0x63, 0x65, 0x53, 0x65, 0x6c, 0x65, 0x63, 0x74, 0x6f, 0x72, 0x12, 0x2e, 0x0a, 0x12,\n\t0x69, 0x6e, 0x63, 0x6c, 0x75, 0x64, 0x65, 0x64, 0x4e, 0x61, 0x6d, 0x65, 0x73, 0x70, 0x61, 0x63,\n\t0x65, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x09, 0x52, 0x12, 0x69, 0x6e, 0x63, 0x6c, 0x75, 0x64,\n\t0x65, 0x64, 0x4e, 0x61, 0x6d, 0x65, 0x73, 0x70, 0x61, 0x63, 0x65, 0x73, 0x12, 0x2e, 0x0a, 0x12,\n\t0x65, 0x78, 0x63, 0x6c, 0x75, 0x64, 0x65, 0x64, 0x4e, 0x61, 0x6d, 0x65, 0x73, 0x70, 0x61, 0x63,\n\t0x65, 0x73, 0x18, 0x02, 0x20, 0x03, 0x28, 0x09, 0x52, 0x12, 0x65, 0x78, 0x63, 0x6c, 0x75, 0x64,\n\t0x65, 0x64, 0x4e, 0x61, 0x6d, 0x65, 0x73, 0x70, 0x61, 0x63, 0x65, 0x73, 0x12, 0x2c, 0x0a, 0x11,\n\t0x69, 0x6e, 0x63, 0x6c, 0x75, 0x64, 0x65, 0x64, 0x52, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65,\n\t0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x09, 0x52, 0x11, 0x69, 0x6e, 0x63, 0x6c, 0x75, 0x64, 0x65,\n\t0x64, 0x52, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x73, 0x12, 0x2c, 0x0a, 0x11, 0x65, 0x78,\n\t0x63, 0x6c, 0x75, 0x64, 0x65, 0x64, 0x52, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x73, 0x18,\n\t0x04, 0x20, 0x03, 0x28, 0x09, 0x52, 0x11, 0x65, 0x78, 0x63, 0x6c, 0x75, 0x64, 0x65, 0x64, 0x52,\n\t0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x73, 0x12, 0x1a, 0x0a, 0x08, 0x73, 0x65, 0x6c, 0x65,\n\t0x63, 0x74, 0x6f, 0x72, 0x18, 0x05, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x73, 0x65, 0x6c, 0x65,\n\t0x63, 0x74, 0x6f, 0x72, 0x22, 0xb1, 0x02, 0x0a, 0x11, 0x4f, 0x70, 0x65, 0x72, 0x61, 0x74, 0x69,\n\t0x6f, 0x6e, 0x50, 0x72, 0x6f, 0x67, 0x72, 0x65, 0x73, 0x73, 0x12, 0x1c, 0x0a, 0x09, 0x63, 0x6f,\n\t0x6d, 0x70, 0x6c, 0x65, 0x74, 0x65, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x08, 0x52, 0x09, 0x63,\n\t0x6f, 0x6d, 0x70, 0x6c, 0x65, 0x74, 0x65, 0x64, 0x12, 0x10, 0x0a, 0x03, 0x65, 0x72, 0x72, 0x18,\n\t0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x65, 0x72, 0x72, 0x12, 0x1e, 0x0a, 0x0a, 0x6e, 0x43,\n\t0x6f, 0x6d, 0x70, 0x6c, 0x65, 0x74, 0x65, 0x64, 0x18, 0x03, 0x20, 0x01, 0x28, 0x03, 0x52, 0x0a,\n\t0x6e, 0x43, 0x6f, 0x6d, 0x70, 0x6c, 0x65, 0x74, 0x65, 0x64, 0x12, 0x16, 0x0a, 0x06, 0x6e, 0x54,\n\t0x6f, 0x74, 0x61, 0x6c, 0x18, 0x04, 0x20, 0x01, 0x28, 0x03, 0x52, 0x06, 0x6e, 0x54, 0x6f, 0x74,\n\t0x61, 0x6c, 0x12, 0x26, 0x0a, 0x0e, 0x6f, 0x70, 0x65, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x55,\n\t0x6e, 0x69, 0x74, 0x73, 0x18, 0x05, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0e, 0x6f, 0x70, 0x65, 0x72,\n\t0x61, 0x74, 0x69, 0x6f, 0x6e, 0x55, 0x6e, 0x69, 0x74, 0x73, 0x12, 0x20, 0x0a, 0x0b, 0x64, 0x65,\n\t0x73, 0x63, 0x72, 0x69, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x18, 0x06, 0x20, 0x01, 0x28, 0x09, 0x52,\n\t0x0b, 0x64, 0x65, 0x73, 0x63, 0x72, 0x69, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x34, 0x0a, 0x07,\n\t0x73, 0x74, 0x61, 0x72, 0x74, 0x65, 0x64, 0x18, 0x07, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1a, 0x2e,\n\t0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e,\n\t0x54, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x52, 0x07, 0x73, 0x74, 0x61, 0x72, 0x74,\n\t0x65, 0x64, 0x12, 0x34, 0x0a, 0x07, 0x75, 0x70, 0x64, 0x61, 0x74, 0x65, 0x64, 0x18, 0x08, 0x20,\n\t0x01, 0x28, 0x0b, 0x32, 0x1a, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f,\n\t0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x54, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x52,\n\t0x07, 0x75, 0x70, 0x64, 0x61, 0x74, 0x65, 0x64, 0x42, 0x35, 0x5a, 0x33, 0x67, 0x69, 0x74, 0x68,\n\t0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x76, 0x6d, 0x77, 0x61, 0x72, 0x65, 0x2d, 0x74, 0x61,\n\t0x6e, 0x7a, 0x75, 0x2f, 0x76, 0x65, 0x6c, 0x65, 0x72, 0x6f, 0x2f, 0x70, 0x6b, 0x67, 0x2f, 0x70,\n\t0x6c, 0x75, 0x67, 0x69, 0x6e, 0x2f, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x62,\n\t0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,\n}\n\nvar (\n\tfile_Shared_proto_rawDescOnce sync.Once\n\tfile_Shared_proto_rawDescData = file_Shared_proto_rawDesc\n)\n\nfunc file_Shared_proto_rawDescGZIP() []byte {\n\tfile_Shared_proto_rawDescOnce.Do(func() {\n\t\tfile_Shared_proto_rawDescData = protoimpl.X.CompressGZIP(file_Shared_proto_rawDescData)\n\t})\n\treturn file_Shared_proto_rawDescData\n}\n\nvar file_Shared_proto_msgTypes = make([]protoimpl.MessageInfo, 6)\nvar file_Shared_proto_goTypes = []interface{}{\n\t(*Empty)(nil), // 0: generated.Empty\n\t(*Stack)(nil), // 1: generated.Stack\n\t(*StackFrame)(nil), // 2: generated.StackFrame\n\t(*ResourceIdentifier)(nil), // 3: generated.ResourceIdentifier\n\t(*ResourceSelector)(nil), // 4: generated.ResourceSelector\n\t(*OperationProgress)(nil), // 5: generated.OperationProgress\n\t(*timestamppb.Timestamp)(nil), // 6: google.protobuf.Timestamp\n}\nvar file_Shared_proto_depIdxs = []int32{\n\t2, // 0: generated.Stack.frames:type_name -> generated.StackFrame\n\t6, // 1: generated.OperationProgress.started:type_name -> google.protobuf.Timestamp\n\t6, // 2: generated.OperationProgress.updated:type_name -> google.protobuf.Timestamp\n\t3, // [3:3] is the sub-list for method output_type\n\t3, // [3:3] is the sub-list for method input_type\n\t3, // [3:3] is the sub-list for extension type_name\n\t3, // [3:3] is the sub-list for extension extendee\n\t0, // [0:3] is the sub-list for field type_name\n}\n\nfunc init() { file_Shared_proto_init() }\nfunc file_Shared_proto_init() {\n\tif File_Shared_proto != nil {\n\t\treturn\n\t}\n\tif !protoimpl.UnsafeEnabled {\n\t\tfile_Shared_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*Empty); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_Shared_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*Stack); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_Shared_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*StackFrame); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_Shared_proto_msgTypes[3].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*ResourceIdentifier); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_Shared_proto_msgTypes[4].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*ResourceSelector); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_Shared_proto_msgTypes[5].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*OperationProgress); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t}\n\ttype x struct{}\n\tout := protoimpl.TypeBuilder{\n\t\tFile: protoimpl.DescBuilder{\n\t\t\tGoPackagePath: reflect.TypeOf(x{}).PkgPath(),\n\t\t\tRawDescriptor: file_Shared_proto_rawDesc,\n\t\t\tNumEnums: 0,\n\t\t\tNumMessages: 6,\n\t\t\tNumExtensions: 0,\n\t\t\tNumServices: 0,\n\t\t},\n\t\tGoTypes: file_Shared_proto_goTypes,\n\t\tDependencyIndexes: file_Shared_proto_depIdxs,\n\t\tMessageInfos: file_Shared_proto_msgTypes,\n\t}.Build()\n\tFile_Shared_proto = out.File\n\tfile_Shared_proto_rawDesc = nil\n\tfile_Shared_proto_goTypes = nil\n\tfile_Shared_proto_depIdxs = nil\n}\n" }, { "path": "pkg/plugin/generated/VolumeSnapshotter.pb.go", "content": "// Code generated by protoc-gen-go. DO NOT EDIT.\n// versions:\n// \tprotoc-gen-go v1.33.0\n// \tprotoc v4.25.2\n// source: VolumeSnapshotter.proto\n\npackage generated\n\nimport (\n\tprotoreflect \"google.golang.org/protobuf/reflect/protoreflect\"\n\tprotoimpl \"google.golang.org/protobuf/runtime/protoimpl\"\n\treflect \"reflect\"\n\tsync \"sync\"\n)\n\nconst (\n\t// Verify that this generated code is sufficiently up-to-date.\n\t_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)\n\t// Verify that runtime/protoimpl is sufficiently up-to-date.\n\t_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)\n)\n\ntype CreateVolumeRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tSnapshotID string `protobuf:\"bytes,2,opt,name=snapshotID,proto3\" json:\"snapshotID,omitempty\"`\n\tVolumeType string `protobuf:\"bytes,3,opt,name=volumeType,proto3\" json:\"volumeType,omitempty\"`\n\tVolumeAZ string `protobuf:\"bytes,4,opt,name=volumeAZ,proto3\" json:\"volumeAZ,omitempty\"`\n\tIops int64 `protobuf:\"varint,5,opt,name=iops,proto3\" json:\"iops,omitempty\"`\n}\n\nfunc (x *CreateVolumeRequest) Reset() {\n\t*x = CreateVolumeRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_VolumeSnapshotter_proto_msgTypes[0]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *CreateVolumeRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*CreateVolumeRequest) ProtoMessage() {}\n\nfunc (x *CreateVolumeRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_VolumeSnapshotter_proto_msgTypes[0]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use CreateVolumeRequest.ProtoReflect.Descriptor instead.\nfunc (*CreateVolumeRequest) Descriptor() ([]byte, []int) {\n\treturn file_VolumeSnapshotter_proto_rawDescGZIP(), []int{0}\n}\n\nfunc (x *CreateVolumeRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *CreateVolumeRequest) GetSnapshotID() string {\n\tif x != nil {\n\t\treturn x.SnapshotID\n\t}\n\treturn \"\"\n}\n\nfunc (x *CreateVolumeRequest) GetVolumeType() string {\n\tif x != nil {\n\t\treturn x.VolumeType\n\t}\n\treturn \"\"\n}\n\nfunc (x *CreateVolumeRequest) GetVolumeAZ() string {\n\tif x != nil {\n\t\treturn x.VolumeAZ\n\t}\n\treturn \"\"\n}\n\nfunc (x *CreateVolumeRequest) GetIops() int64 {\n\tif x != nil {\n\t\treturn x.Iops\n\t}\n\treturn 0\n}\n\ntype CreateVolumeResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tVolumeID string `protobuf:\"bytes,1,opt,name=volumeID,proto3\" json:\"volumeID,omitempty\"`\n}\n\nfunc (x *CreateVolumeResponse) Reset() {\n\t*x = CreateVolumeResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_VolumeSnapshotter_proto_msgTypes[1]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *CreateVolumeResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*CreateVolumeResponse) ProtoMessage() {}\n\nfunc (x *CreateVolumeResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_VolumeSnapshotter_proto_msgTypes[1]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use CreateVolumeResponse.ProtoReflect.Descriptor instead.\nfunc (*CreateVolumeResponse) Descriptor() ([]byte, []int) {\n\treturn file_VolumeSnapshotter_proto_rawDescGZIP(), []int{1}\n}\n\nfunc (x *CreateVolumeResponse) GetVolumeID() string {\n\tif x != nil {\n\t\treturn x.VolumeID\n\t}\n\treturn \"\"\n}\n\ntype GetVolumeInfoRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tVolumeID string `protobuf:\"bytes,2,opt,name=volumeID,proto3\" json:\"volumeID,omitempty\"`\n\tVolumeAZ string `protobuf:\"bytes,3,opt,name=volumeAZ,proto3\" json:\"volumeAZ,omitempty\"`\n}\n\nfunc (x *GetVolumeInfoRequest) Reset() {\n\t*x = GetVolumeInfoRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_VolumeSnapshotter_proto_msgTypes[2]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *GetVolumeInfoRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*GetVolumeInfoRequest) ProtoMessage() {}\n\nfunc (x *GetVolumeInfoRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_VolumeSnapshotter_proto_msgTypes[2]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use GetVolumeInfoRequest.ProtoReflect.Descriptor instead.\nfunc (*GetVolumeInfoRequest) Descriptor() ([]byte, []int) {\n\treturn file_VolumeSnapshotter_proto_rawDescGZIP(), []int{2}\n}\n\nfunc (x *GetVolumeInfoRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *GetVolumeInfoRequest) GetVolumeID() string {\n\tif x != nil {\n\t\treturn x.VolumeID\n\t}\n\treturn \"\"\n}\n\nfunc (x *GetVolumeInfoRequest) GetVolumeAZ() string {\n\tif x != nil {\n\t\treturn x.VolumeAZ\n\t}\n\treturn \"\"\n}\n\ntype GetVolumeInfoResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tVolumeType string `protobuf:\"bytes,1,opt,name=volumeType,proto3\" json:\"volumeType,omitempty\"`\n\tIops int64 `protobuf:\"varint,2,opt,name=iops,proto3\" json:\"iops,omitempty\"`\n}\n\nfunc (x *GetVolumeInfoResponse) Reset() {\n\t*x = GetVolumeInfoResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_VolumeSnapshotter_proto_msgTypes[3]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *GetVolumeInfoResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*GetVolumeInfoResponse) ProtoMessage() {}\n\nfunc (x *GetVolumeInfoResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_VolumeSnapshotter_proto_msgTypes[3]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use GetVolumeInfoResponse.ProtoReflect.Descriptor instead.\nfunc (*GetVolumeInfoResponse) Descriptor() ([]byte, []int) {\n\treturn file_VolumeSnapshotter_proto_rawDescGZIP(), []int{3}\n}\n\nfunc (x *GetVolumeInfoResponse) GetVolumeType() string {\n\tif x != nil {\n\t\treturn x.VolumeType\n\t}\n\treturn \"\"\n}\n\nfunc (x *GetVolumeInfoResponse) GetIops() int64 {\n\tif x != nil {\n\t\treturn x.Iops\n\t}\n\treturn 0\n}\n\ntype CreateSnapshotRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tVolumeID string `protobuf:\"bytes,2,opt,name=volumeID,proto3\" json:\"volumeID,omitempty\"`\n\tVolumeAZ string `protobuf:\"bytes,3,opt,name=volumeAZ,proto3\" json:\"volumeAZ,omitempty\"`\n\tTags map[string]string `protobuf:\"bytes,4,rep,name=tags,proto3\" json:\"tags,omitempty\" protobuf_key:\"bytes,1,opt,name=key,proto3\" protobuf_val:\"bytes,2,opt,name=value,proto3\"`\n}\n\nfunc (x *CreateSnapshotRequest) Reset() {\n\t*x = CreateSnapshotRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_VolumeSnapshotter_proto_msgTypes[4]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *CreateSnapshotRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*CreateSnapshotRequest) ProtoMessage() {}\n\nfunc (x *CreateSnapshotRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_VolumeSnapshotter_proto_msgTypes[4]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use CreateSnapshotRequest.ProtoReflect.Descriptor instead.\nfunc (*CreateSnapshotRequest) Descriptor() ([]byte, []int) {\n\treturn file_VolumeSnapshotter_proto_rawDescGZIP(), []int{4}\n}\n\nfunc (x *CreateSnapshotRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *CreateSnapshotRequest) GetVolumeID() string {\n\tif x != nil {\n\t\treturn x.VolumeID\n\t}\n\treturn \"\"\n}\n\nfunc (x *CreateSnapshotRequest) GetVolumeAZ() string {\n\tif x != nil {\n\t\treturn x.VolumeAZ\n\t}\n\treturn \"\"\n}\n\nfunc (x *CreateSnapshotRequest) GetTags() map[string]string {\n\tif x != nil {\n\t\treturn x.Tags\n\t}\n\treturn nil\n}\n\ntype CreateSnapshotResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tSnapshotID string `protobuf:\"bytes,1,opt,name=snapshotID,proto3\" json:\"snapshotID,omitempty\"`\n}\n\nfunc (x *CreateSnapshotResponse) Reset() {\n\t*x = CreateSnapshotResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_VolumeSnapshotter_proto_msgTypes[5]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *CreateSnapshotResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*CreateSnapshotResponse) ProtoMessage() {}\n\nfunc (x *CreateSnapshotResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_VolumeSnapshotter_proto_msgTypes[5]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use CreateSnapshotResponse.ProtoReflect.Descriptor instead.\nfunc (*CreateSnapshotResponse) Descriptor() ([]byte, []int) {\n\treturn file_VolumeSnapshotter_proto_rawDescGZIP(), []int{5}\n}\n\nfunc (x *CreateSnapshotResponse) GetSnapshotID() string {\n\tif x != nil {\n\t\treturn x.SnapshotID\n\t}\n\treturn \"\"\n}\n\ntype DeleteSnapshotRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tSnapshotID string `protobuf:\"bytes,2,opt,name=snapshotID,proto3\" json:\"snapshotID,omitempty\"`\n}\n\nfunc (x *DeleteSnapshotRequest) Reset() {\n\t*x = DeleteSnapshotRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_VolumeSnapshotter_proto_msgTypes[6]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *DeleteSnapshotRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*DeleteSnapshotRequest) ProtoMessage() {}\n\nfunc (x *DeleteSnapshotRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_VolumeSnapshotter_proto_msgTypes[6]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use DeleteSnapshotRequest.ProtoReflect.Descriptor instead.\nfunc (*DeleteSnapshotRequest) Descriptor() ([]byte, []int) {\n\treturn file_VolumeSnapshotter_proto_rawDescGZIP(), []int{6}\n}\n\nfunc (x *DeleteSnapshotRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *DeleteSnapshotRequest) GetSnapshotID() string {\n\tif x != nil {\n\t\treturn x.SnapshotID\n\t}\n\treturn \"\"\n}\n\ntype GetVolumeIDRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tPersistentVolume []byte `protobuf:\"bytes,2,opt,name=persistentVolume,proto3\" json:\"persistentVolume,omitempty\"`\n}\n\nfunc (x *GetVolumeIDRequest) Reset() {\n\t*x = GetVolumeIDRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_VolumeSnapshotter_proto_msgTypes[7]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *GetVolumeIDRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*GetVolumeIDRequest) ProtoMessage() {}\n\nfunc (x *GetVolumeIDRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_VolumeSnapshotter_proto_msgTypes[7]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use GetVolumeIDRequest.ProtoReflect.Descriptor instead.\nfunc (*GetVolumeIDRequest) Descriptor() ([]byte, []int) {\n\treturn file_VolumeSnapshotter_proto_rawDescGZIP(), []int{7}\n}\n\nfunc (x *GetVolumeIDRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *GetVolumeIDRequest) GetPersistentVolume() []byte {\n\tif x != nil {\n\t\treturn x.PersistentVolume\n\t}\n\treturn nil\n}\n\ntype GetVolumeIDResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tVolumeID string `protobuf:\"bytes,1,opt,name=volumeID,proto3\" json:\"volumeID,omitempty\"`\n}\n\nfunc (x *GetVolumeIDResponse) Reset() {\n\t*x = GetVolumeIDResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_VolumeSnapshotter_proto_msgTypes[8]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *GetVolumeIDResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*GetVolumeIDResponse) ProtoMessage() {}\n\nfunc (x *GetVolumeIDResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_VolumeSnapshotter_proto_msgTypes[8]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use GetVolumeIDResponse.ProtoReflect.Descriptor instead.\nfunc (*GetVolumeIDResponse) Descriptor() ([]byte, []int) {\n\treturn file_VolumeSnapshotter_proto_rawDescGZIP(), []int{8}\n}\n\nfunc (x *GetVolumeIDResponse) GetVolumeID() string {\n\tif x != nil {\n\t\treturn x.VolumeID\n\t}\n\treturn \"\"\n}\n\ntype SetVolumeIDRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tPersistentVolume []byte `protobuf:\"bytes,2,opt,name=persistentVolume,proto3\" json:\"persistentVolume,omitempty\"`\n\tVolumeID string `protobuf:\"bytes,3,opt,name=volumeID,proto3\" json:\"volumeID,omitempty\"`\n}\n\nfunc (x *SetVolumeIDRequest) Reset() {\n\t*x = SetVolumeIDRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_VolumeSnapshotter_proto_msgTypes[9]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *SetVolumeIDRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*SetVolumeIDRequest) ProtoMessage() {}\n\nfunc (x *SetVolumeIDRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_VolumeSnapshotter_proto_msgTypes[9]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use SetVolumeIDRequest.ProtoReflect.Descriptor instead.\nfunc (*SetVolumeIDRequest) Descriptor() ([]byte, []int) {\n\treturn file_VolumeSnapshotter_proto_rawDescGZIP(), []int{9}\n}\n\nfunc (x *SetVolumeIDRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *SetVolumeIDRequest) GetPersistentVolume() []byte {\n\tif x != nil {\n\t\treturn x.PersistentVolume\n\t}\n\treturn nil\n}\n\nfunc (x *SetVolumeIDRequest) GetVolumeID() string {\n\tif x != nil {\n\t\treturn x.VolumeID\n\t}\n\treturn \"\"\n}\n\ntype SetVolumeIDResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPersistentVolume []byte `protobuf:\"bytes,1,opt,name=persistentVolume,proto3\" json:\"persistentVolume,omitempty\"`\n}\n\nfunc (x *SetVolumeIDResponse) Reset() {\n\t*x = SetVolumeIDResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_VolumeSnapshotter_proto_msgTypes[10]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *SetVolumeIDResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*SetVolumeIDResponse) ProtoMessage() {}\n\nfunc (x *SetVolumeIDResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_VolumeSnapshotter_proto_msgTypes[10]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use SetVolumeIDResponse.ProtoReflect.Descriptor instead.\nfunc (*SetVolumeIDResponse) Descriptor() ([]byte, []int) {\n\treturn file_VolumeSnapshotter_proto_rawDescGZIP(), []int{10}\n}\n\nfunc (x *SetVolumeIDResponse) GetPersistentVolume() []byte {\n\tif x != nil {\n\t\treturn x.PersistentVolume\n\t}\n\treturn nil\n}\n\ntype VolumeSnapshotterInitRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tConfig map[string]string `protobuf:\"bytes,2,rep,name=config,proto3\" json:\"config,omitempty\" protobuf_key:\"bytes,1,opt,name=key,proto3\" protobuf_val:\"bytes,2,opt,name=value,proto3\"`\n}\n\nfunc (x *VolumeSnapshotterInitRequest) Reset() {\n\t*x = VolumeSnapshotterInitRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_VolumeSnapshotter_proto_msgTypes[11]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *VolumeSnapshotterInitRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*VolumeSnapshotterInitRequest) ProtoMessage() {}\n\nfunc (x *VolumeSnapshotterInitRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_VolumeSnapshotter_proto_msgTypes[11]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use VolumeSnapshotterInitRequest.ProtoReflect.Descriptor instead.\nfunc (*VolumeSnapshotterInitRequest) Descriptor() ([]byte, []int) {\n\treturn file_VolumeSnapshotter_proto_rawDescGZIP(), []int{11}\n}\n\nfunc (x *VolumeSnapshotterInitRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *VolumeSnapshotterInitRequest) GetConfig() map[string]string {\n\tif x != nil {\n\t\treturn x.Config\n\t}\n\treturn nil\n}\n\nvar File_VolumeSnapshotter_proto protoreflect.FileDescriptor\n\nvar file_VolumeSnapshotter_proto_rawDesc = []byte{\n\t0x0a, 0x17, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x53, 0x6e, 0x61, 0x70, 0x73, 0x68, 0x6f, 0x74,\n\t0x74, 0x65, 0x72, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x09, 0x67, 0x65, 0x6e, 0x65, 0x72,\n\t0x61, 0x74, 0x65, 0x64, 0x1a, 0x0c, 0x53, 0x68, 0x61, 0x72, 0x65, 0x64, 0x2e, 0x70, 0x72, 0x6f,\n\t0x74, 0x6f, 0x22, 0x9d, 0x01, 0x0a, 0x13, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x56, 0x6f, 0x6c,\n\t0x75, 0x6d, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x16, 0x0a, 0x06, 0x70, 0x6c,\n\t0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x70, 0x6c, 0x75, 0x67,\n\t0x69, 0x6e, 0x12, 0x1e, 0x0a, 0x0a, 0x73, 0x6e, 0x61, 0x70, 0x73, 0x68, 0x6f, 0x74, 0x49, 0x44,\n\t0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0a, 0x73, 0x6e, 0x61, 0x70, 0x73, 0x68, 0x6f, 0x74,\n\t0x49, 0x44, 0x12, 0x1e, 0x0a, 0x0a, 0x76, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x54, 0x79, 0x70, 0x65,\n\t0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0a, 0x76, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x54, 0x79,\n\t0x70, 0x65, 0x12, 0x1a, 0x0a, 0x08, 0x76, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x41, 0x5a, 0x18, 0x04,\n\t0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x76, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x41, 0x5a, 0x12, 0x12,\n\t0x0a, 0x04, 0x69, 0x6f, 0x70, 0x73, 0x18, 0x05, 0x20, 0x01, 0x28, 0x03, 0x52, 0x04, 0x69, 0x6f,\n\t0x70, 0x73, 0x22, 0x32, 0x0a, 0x14, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x56, 0x6f, 0x6c, 0x75,\n\t0x6d, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x1a, 0x0a, 0x08, 0x76, 0x6f,\n\t0x6c, 0x75, 0x6d, 0x65, 0x49, 0x44, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x76, 0x6f,\n\t0x6c, 0x75, 0x6d, 0x65, 0x49, 0x44, 0x22, 0x66, 0x0a, 0x14, 0x47, 0x65, 0x74, 0x56, 0x6f, 0x6c,\n\t0x75, 0x6d, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x16,\n\t0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06,\n\t0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x12, 0x1a, 0x0a, 0x08, 0x76, 0x6f, 0x6c, 0x75, 0x6d, 0x65,\n\t0x49, 0x44, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x76, 0x6f, 0x6c, 0x75, 0x6d, 0x65,\n\t0x49, 0x44, 0x12, 0x1a, 0x0a, 0x08, 0x76, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x41, 0x5a, 0x18, 0x03,\n\t0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x76, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x41, 0x5a, 0x22, 0x4b,\n\t0x0a, 0x15, 0x47, 0x65, 0x74, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x52,\n\t0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x1e, 0x0a, 0x0a, 0x76, 0x6f, 0x6c, 0x75, 0x6d,\n\t0x65, 0x54, 0x79, 0x70, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0a, 0x76, 0x6f, 0x6c,\n\t0x75, 0x6d, 0x65, 0x54, 0x79, 0x70, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x69, 0x6f, 0x70, 0x73, 0x18,\n\t0x02, 0x20, 0x01, 0x28, 0x03, 0x52, 0x04, 0x69, 0x6f, 0x70, 0x73, 0x22, 0xe0, 0x01, 0x0a, 0x15,\n\t0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x53, 0x6e, 0x61, 0x70, 0x73, 0x68, 0x6f, 0x74, 0x52, 0x65,\n\t0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x16, 0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18,\n\t0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x12, 0x1a, 0x0a,\n\t0x08, 0x76, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x49, 0x44, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52,\n\t0x08, 0x76, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x49, 0x44, 0x12, 0x1a, 0x0a, 0x08, 0x76, 0x6f, 0x6c,\n\t0x75, 0x6d, 0x65, 0x41, 0x5a, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x76, 0x6f, 0x6c,\n\t0x75, 0x6d, 0x65, 0x41, 0x5a, 0x12, 0x3e, 0x0a, 0x04, 0x74, 0x61, 0x67, 0x73, 0x18, 0x04, 0x20,\n\t0x03, 0x28, 0x0b, 0x32, 0x2a, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e,\n\t0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x53, 0x6e, 0x61, 0x70, 0x73, 0x68, 0x6f, 0x74, 0x52, 0x65,\n\t0x71, 0x75, 0x65, 0x73, 0x74, 0x2e, 0x54, 0x61, 0x67, 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x52,\n\t0x04, 0x74, 0x61, 0x67, 0x73, 0x1a, 0x37, 0x0a, 0x09, 0x54, 0x61, 0x67, 0x73, 0x45, 0x6e, 0x74,\n\t0x72, 0x79, 0x12, 0x10, 0x0a, 0x03, 0x6b, 0x65, 0x79, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52,\n\t0x03, 0x6b, 0x65, 0x79, 0x12, 0x14, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x02, 0x20,\n\t0x01, 0x28, 0x09, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x3a, 0x02, 0x38, 0x01, 0x22, 0x38,\n\t0x0a, 0x16, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x53, 0x6e, 0x61, 0x70, 0x73, 0x68, 0x6f, 0x74,\n\t0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x1e, 0x0a, 0x0a, 0x73, 0x6e, 0x61, 0x70,\n\t0x73, 0x68, 0x6f, 0x74, 0x49, 0x44, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0a, 0x73, 0x6e,\n\t0x61, 0x70, 0x73, 0x68, 0x6f, 0x74, 0x49, 0x44, 0x22, 0x4f, 0x0a, 0x15, 0x44, 0x65, 0x6c, 0x65,\n\t0x74, 0x65, 0x53, 0x6e, 0x61, 0x70, 0x73, 0x68, 0x6f, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73,\n\t0x74, 0x12, 0x16, 0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28,\n\t0x09, 0x52, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x12, 0x1e, 0x0a, 0x0a, 0x73, 0x6e, 0x61,\n\t0x70, 0x73, 0x68, 0x6f, 0x74, 0x49, 0x44, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0a, 0x73,\n\t0x6e, 0x61, 0x70, 0x73, 0x68, 0x6f, 0x74, 0x49, 0x44, 0x22, 0x58, 0x0a, 0x12, 0x47, 0x65, 0x74,\n\t0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x49, 0x44, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12,\n\t0x16, 0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52,\n\t0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x12, 0x2a, 0x0a, 0x10, 0x70, 0x65, 0x72, 0x73, 0x69,\n\t0x73, 0x74, 0x65, 0x6e, 0x74, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28,\n\t0x0c, 0x52, 0x10, 0x70, 0x65, 0x72, 0x73, 0x69, 0x73, 0x74, 0x65, 0x6e, 0x74, 0x56, 0x6f, 0x6c,\n\t0x75, 0x6d, 0x65, 0x22, 0x31, 0x0a, 0x13, 0x47, 0x65, 0x74, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65,\n\t0x49, 0x44, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x1a, 0x0a, 0x08, 0x76, 0x6f,\n\t0x6c, 0x75, 0x6d, 0x65, 0x49, 0x44, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x76, 0x6f,\n\t0x6c, 0x75, 0x6d, 0x65, 0x49, 0x44, 0x22, 0x74, 0x0a, 0x12, 0x53, 0x65, 0x74, 0x56, 0x6f, 0x6c,\n\t0x75, 0x6d, 0x65, 0x49, 0x44, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x16, 0x0a, 0x06,\n\t0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x70, 0x6c,\n\t0x75, 0x67, 0x69, 0x6e, 0x12, 0x2a, 0x0a, 0x10, 0x70, 0x65, 0x72, 0x73, 0x69, 0x73, 0x74, 0x65,\n\t0x6e, 0x74, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x10,\n\t0x70, 0x65, 0x72, 0x73, 0x69, 0x73, 0x74, 0x65, 0x6e, 0x74, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65,\n\t0x12, 0x1a, 0x0a, 0x08, 0x76, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x49, 0x44, 0x18, 0x03, 0x20, 0x01,\n\t0x28, 0x09, 0x52, 0x08, 0x76, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x49, 0x44, 0x22, 0x41, 0x0a, 0x13,\n\t0x53, 0x65, 0x74, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x49, 0x44, 0x52, 0x65, 0x73, 0x70, 0x6f,\n\t0x6e, 0x73, 0x65, 0x12, 0x2a, 0x0a, 0x10, 0x70, 0x65, 0x72, 0x73, 0x69, 0x73, 0x74, 0x65, 0x6e,\n\t0x74, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x10, 0x70,\n\t0x65, 0x72, 0x73, 0x69, 0x73, 0x74, 0x65, 0x6e, 0x74, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x22,\n\t0xbe, 0x01, 0x0a, 0x1c, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x53, 0x6e, 0x61, 0x70, 0x73, 0x68,\n\t0x6f, 0x74, 0x74, 0x65, 0x72, 0x49, 0x6e, 0x69, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74,\n\t0x12, 0x16, 0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09,\n\t0x52, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x12, 0x4b, 0x0a, 0x06, 0x63, 0x6f, 0x6e, 0x66,\n\t0x69, 0x67, 0x18, 0x02, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x33, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72,\n\t0x61, 0x74, 0x65, 0x64, 0x2e, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x53, 0x6e, 0x61, 0x70, 0x73,\n\t0x68, 0x6f, 0x74, 0x74, 0x65, 0x72, 0x49, 0x6e, 0x69, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73,\n\t0x74, 0x2e, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x06, 0x63,\n\t0x6f, 0x6e, 0x66, 0x69, 0x67, 0x1a, 0x39, 0x0a, 0x0b, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x45,\n\t0x6e, 0x74, 0x72, 0x79, 0x12, 0x10, 0x0a, 0x03, 0x6b, 0x65, 0x79, 0x18, 0x01, 0x20, 0x01, 0x28,\n\t0x09, 0x52, 0x03, 0x6b, 0x65, 0x79, 0x12, 0x14, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18,\n\t0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x3a, 0x02, 0x38, 0x01,\n\t0x32, 0xc0, 0x04, 0x0a, 0x11, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x53, 0x6e, 0x61, 0x70, 0x73,\n\t0x68, 0x6f, 0x74, 0x74, 0x65, 0x72, 0x12, 0x41, 0x0a, 0x04, 0x49, 0x6e, 0x69, 0x74, 0x12, 0x27,\n\t0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x56, 0x6f, 0x6c, 0x75, 0x6d,\n\t0x65, 0x53, 0x6e, 0x61, 0x70, 0x73, 0x68, 0x6f, 0x74, 0x74, 0x65, 0x72, 0x49, 0x6e, 0x69, 0x74,\n\t0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x10, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61,\n\t0x74, 0x65, 0x64, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x12, 0x5b, 0x0a, 0x18, 0x43, 0x72, 0x65,\n\t0x61, 0x74, 0x65, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x46, 0x72, 0x6f, 0x6d, 0x53, 0x6e, 0x61,\n\t0x70, 0x73, 0x68, 0x6f, 0x74, 0x12, 0x1e, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65,\n\t0x64, 0x2e, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x52, 0x65,\n\t0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1f, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65,\n\t0x64, 0x2e, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x52, 0x65,\n\t0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x52, 0x0a, 0x0d, 0x47, 0x65, 0x74, 0x56, 0x6f, 0x6c,\n\t0x75, 0x6d, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x12, 0x1f, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61,\n\t0x74, 0x65, 0x64, 0x2e, 0x47, 0x65, 0x74, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x49, 0x6e, 0x66,\n\t0x6f, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x20, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72,\n\t0x61, 0x74, 0x65, 0x64, 0x2e, 0x47, 0x65, 0x74, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x49, 0x6e,\n\t0x66, 0x6f, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x55, 0x0a, 0x0e, 0x43, 0x72,\n\t0x65, 0x61, 0x74, 0x65, 0x53, 0x6e, 0x61, 0x70, 0x73, 0x68, 0x6f, 0x74, 0x12, 0x20, 0x2e, 0x67,\n\t0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x53,\n\t0x6e, 0x61, 0x70, 0x73, 0x68, 0x6f, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x21,\n\t0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x43, 0x72, 0x65, 0x61, 0x74,\n\t0x65, 0x53, 0x6e, 0x61, 0x70, 0x73, 0x68, 0x6f, 0x74, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73,\n\t0x65, 0x12, 0x44, 0x0a, 0x0e, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x53, 0x6e, 0x61, 0x70, 0x73,\n\t0x68, 0x6f, 0x74, 0x12, 0x20, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e,\n\t0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x53, 0x6e, 0x61, 0x70, 0x73, 0x68, 0x6f, 0x74, 0x52, 0x65,\n\t0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x10, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65,\n\t0x64, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x12, 0x4c, 0x0a, 0x0b, 0x47, 0x65, 0x74, 0x56, 0x6f,\n\t0x6c, 0x75, 0x6d, 0x65, 0x49, 0x44, 0x12, 0x1d, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74,\n\t0x65, 0x64, 0x2e, 0x47, 0x65, 0x74, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x49, 0x44, 0x52, 0x65,\n\t0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1e, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65,\n\t0x64, 0x2e, 0x47, 0x65, 0x74, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x49, 0x44, 0x52, 0x65, 0x73,\n\t0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x4c, 0x0a, 0x0b, 0x53, 0x65, 0x74, 0x56, 0x6f, 0x6c, 0x75,\n\t0x6d, 0x65, 0x49, 0x44, 0x12, 0x1d, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64,\n\t0x2e, 0x53, 0x65, 0x74, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x49, 0x44, 0x52, 0x65, 0x71, 0x75,\n\t0x65, 0x73, 0x74, 0x1a, 0x1e, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e,\n\t0x53, 0x65, 0x74, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x49, 0x44, 0x52, 0x65, 0x73, 0x70, 0x6f,\n\t0x6e, 0x73, 0x65, 0x42, 0x35, 0x5a, 0x33, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f,\n\t0x6d, 0x2f, 0x76, 0x6d, 0x77, 0x61, 0x72, 0x65, 0x2d, 0x74, 0x61, 0x6e, 0x7a, 0x75, 0x2f, 0x76,\n\t0x65, 0x6c, 0x65, 0x72, 0x6f, 0x2f, 0x70, 0x6b, 0x67, 0x2f, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e,\n\t0x2f, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74,\n\t0x6f, 0x33,\n}\n\nvar (\n\tfile_VolumeSnapshotter_proto_rawDescOnce sync.Once\n\tfile_VolumeSnapshotter_proto_rawDescData = file_VolumeSnapshotter_proto_rawDesc\n)\n\nfunc file_VolumeSnapshotter_proto_rawDescGZIP() []byte {\n\tfile_VolumeSnapshotter_proto_rawDescOnce.Do(func() {\n\t\tfile_VolumeSnapshotter_proto_rawDescData = protoimpl.X.CompressGZIP(file_VolumeSnapshotter_proto_rawDescData)\n\t})\n\treturn file_VolumeSnapshotter_proto_rawDescData\n}\n\nvar file_VolumeSnapshotter_proto_msgTypes = make([]protoimpl.MessageInfo, 14)\nvar file_VolumeSnapshotter_proto_goTypes = []interface{}{\n\t(*CreateVolumeRequest)(nil), // 0: generated.CreateVolumeRequest\n\t(*CreateVolumeResponse)(nil), // 1: generated.CreateVolumeResponse\n\t(*GetVolumeInfoRequest)(nil), // 2: generated.GetVolumeInfoRequest\n\t(*GetVolumeInfoResponse)(nil), // 3: generated.GetVolumeInfoResponse\n\t(*CreateSnapshotRequest)(nil), // 4: generated.CreateSnapshotRequest\n\t(*CreateSnapshotResponse)(nil), // 5: generated.CreateSnapshotResponse\n\t(*DeleteSnapshotRequest)(nil), // 6: generated.DeleteSnapshotRequest\n\t(*GetVolumeIDRequest)(nil), // 7: generated.GetVolumeIDRequest\n\t(*GetVolumeIDResponse)(nil), // 8: generated.GetVolumeIDResponse\n\t(*SetVolumeIDRequest)(nil), // 9: generated.SetVolumeIDRequest\n\t(*SetVolumeIDResponse)(nil), // 10: generated.SetVolumeIDResponse\n\t(*VolumeSnapshotterInitRequest)(nil), // 11: generated.VolumeSnapshotterInitRequest\n\tnil, // 12: generated.CreateSnapshotRequest.TagsEntry\n\tnil, // 13: generated.VolumeSnapshotterInitRequest.ConfigEntry\n\t(*Empty)(nil), // 14: generated.Empty\n}\nvar file_VolumeSnapshotter_proto_depIdxs = []int32{\n\t12, // 0: generated.CreateSnapshotRequest.tags:type_name -> generated.CreateSnapshotRequest.TagsEntry\n\t13, // 1: generated.VolumeSnapshotterInitRequest.config:type_name -> generated.VolumeSnapshotterInitRequest.ConfigEntry\n\t11, // 2: generated.VolumeSnapshotter.Init:input_type -> generated.VolumeSnapshotterInitRequest\n\t0, // 3: generated.VolumeSnapshotter.CreateVolumeFromSnapshot:input_type -> generated.CreateVolumeRequest\n\t2, // 4: generated.VolumeSnapshotter.GetVolumeInfo:input_type -> generated.GetVolumeInfoRequest\n\t4, // 5: generated.VolumeSnapshotter.CreateSnapshot:input_type -> generated.CreateSnapshotRequest\n\t6, // 6: generated.VolumeSnapshotter.DeleteSnapshot:input_type -> generated.DeleteSnapshotRequest\n\t7, // 7: generated.VolumeSnapshotter.GetVolumeID:input_type -> generated.GetVolumeIDRequest\n\t9, // 8: generated.VolumeSnapshotter.SetVolumeID:input_type -> generated.SetVolumeIDRequest\n\t14, // 9: generated.VolumeSnapshotter.Init:output_type -> generated.Empty\n\t1, // 10: generated.VolumeSnapshotter.CreateVolumeFromSnapshot:output_type -> generated.CreateVolumeResponse\n\t3, // 11: generated.VolumeSnapshotter.GetVolumeInfo:output_type -> generated.GetVolumeInfoResponse\n\t5, // 12: generated.VolumeSnapshotter.CreateSnapshot:output_type -> generated.CreateSnapshotResponse\n\t14, // 13: generated.VolumeSnapshotter.DeleteSnapshot:output_type -> generated.Empty\n\t8, // 14: generated.VolumeSnapshotter.GetVolumeID:output_type -> generated.GetVolumeIDResponse\n\t10, // 15: generated.VolumeSnapshotter.SetVolumeID:output_type -> generated.SetVolumeIDResponse\n\t9, // [9:16] is the sub-list for method output_type\n\t2, // [2:9] is the sub-list for method input_type\n\t2, // [2:2] is the sub-list for extension type_name\n\t2, // [2:2] is the sub-list for extension extendee\n\t0, // [0:2] is the sub-list for field type_name\n}\n\nfunc init() { file_VolumeSnapshotter_proto_init() }\nfunc file_VolumeSnapshotter_proto_init() {\n\tif File_VolumeSnapshotter_proto != nil {\n\t\treturn\n\t}\n\tfile_Shared_proto_init()\n\tif !protoimpl.UnsafeEnabled {\n\t\tfile_VolumeSnapshotter_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*CreateVolumeRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_VolumeSnapshotter_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*CreateVolumeResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_VolumeSnapshotter_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*GetVolumeInfoRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_VolumeSnapshotter_proto_msgTypes[3].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*GetVolumeInfoResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_VolumeSnapshotter_proto_msgTypes[4].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*CreateSnapshotRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_VolumeSnapshotter_proto_msgTypes[5].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*CreateSnapshotResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_VolumeSnapshotter_proto_msgTypes[6].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*DeleteSnapshotRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_VolumeSnapshotter_proto_msgTypes[7].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*GetVolumeIDRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_VolumeSnapshotter_proto_msgTypes[8].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*GetVolumeIDResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_VolumeSnapshotter_proto_msgTypes[9].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*SetVolumeIDRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_VolumeSnapshotter_proto_msgTypes[10].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*SetVolumeIDResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_VolumeSnapshotter_proto_msgTypes[11].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*VolumeSnapshotterInitRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t}\n\ttype x struct{}\n\tout := protoimpl.TypeBuilder{\n\t\tFile: protoimpl.DescBuilder{\n\t\t\tGoPackagePath: reflect.TypeOf(x{}).PkgPath(),\n\t\t\tRawDescriptor: file_VolumeSnapshotter_proto_rawDesc,\n\t\t\tNumEnums: 0,\n\t\t\tNumMessages: 14,\n\t\t\tNumExtensions: 0,\n\t\t\tNumServices: 1,\n\t\t},\n\t\tGoTypes: file_VolumeSnapshotter_proto_goTypes,\n\t\tDependencyIndexes: file_VolumeSnapshotter_proto_depIdxs,\n\t\tMessageInfos: file_VolumeSnapshotter_proto_msgTypes,\n\t}.Build()\n\tFile_VolumeSnapshotter_proto = out.File\n\tfile_VolumeSnapshotter_proto_rawDesc = nil\n\tfile_VolumeSnapshotter_proto_goTypes = nil\n\tfile_VolumeSnapshotter_proto_depIdxs = nil\n}\n" }, { "path": "pkg/plugin/generated/VolumeSnapshotter_grpc.pb.go", "content": "// Code generated by protoc-gen-go-grpc. DO NOT EDIT.\n// versions:\n// - protoc-gen-go-grpc v1.3.0\n// - protoc v4.25.2\n// source: VolumeSnapshotter.proto\n\npackage generated\n\nimport (\n\tcontext \"context\"\n\tgrpc \"google.golang.org/grpc\"\n\tcodes \"google.golang.org/grpc/codes\"\n\tstatus \"google.golang.org/grpc/status\"\n)\n\n// This is a compile-time assertion to ensure that this generated file\n// is compatible with the grpc package it is being compiled against.\n// Requires gRPC-Go v1.32.0 or later.\nconst _ = grpc.SupportPackageIsVersion7\n\nconst (\n\tVolumeSnapshotter_Init_FullMethodName = \"/generated.VolumeSnapshotter/Init\"\n\tVolumeSnapshotter_CreateVolumeFromSnapshot_FullMethodName = \"/generated.VolumeSnapshotter/CreateVolumeFromSnapshot\"\n\tVolumeSnapshotter_GetVolumeInfo_FullMethodName = \"/generated.VolumeSnapshotter/GetVolumeInfo\"\n\tVolumeSnapshotter_CreateSnapshot_FullMethodName = \"/generated.VolumeSnapshotter/CreateSnapshot\"\n\tVolumeSnapshotter_DeleteSnapshot_FullMethodName = \"/generated.VolumeSnapshotter/DeleteSnapshot\"\n\tVolumeSnapshotter_GetVolumeID_FullMethodName = \"/generated.VolumeSnapshotter/GetVolumeID\"\n\tVolumeSnapshotter_SetVolumeID_FullMethodName = \"/generated.VolumeSnapshotter/SetVolumeID\"\n)\n\n// VolumeSnapshotterClient is the client API for VolumeSnapshotter service.\n//\n// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.\ntype VolumeSnapshotterClient interface {\n\tInit(ctx context.Context, in *VolumeSnapshotterInitRequest, opts ...grpc.CallOption) (*Empty, error)\n\tCreateVolumeFromSnapshot(ctx context.Context, in *CreateVolumeRequest, opts ...grpc.CallOption) (*CreateVolumeResponse, error)\n\tGetVolumeInfo(ctx context.Context, in *GetVolumeInfoRequest, opts ...grpc.CallOption) (*GetVolumeInfoResponse, error)\n\tCreateSnapshot(ctx context.Context, in *CreateSnapshotRequest, opts ...grpc.CallOption) (*CreateSnapshotResponse, error)\n\tDeleteSnapshot(ctx context.Context, in *DeleteSnapshotRequest, opts ...grpc.CallOption) (*Empty, error)\n\tGetVolumeID(ctx context.Context, in *GetVolumeIDRequest, opts ...grpc.CallOption) (*GetVolumeIDResponse, error)\n\tSetVolumeID(ctx context.Context, in *SetVolumeIDRequest, opts ...grpc.CallOption) (*SetVolumeIDResponse, error)\n}\n\ntype volumeSnapshotterClient struct {\n\tcc grpc.ClientConnInterface\n}\n\nfunc NewVolumeSnapshotterClient(cc grpc.ClientConnInterface) VolumeSnapshotterClient {\n\treturn &volumeSnapshotterClient{cc}\n}\n\nfunc (c *volumeSnapshotterClient) Init(ctx context.Context, in *VolumeSnapshotterInitRequest, opts ...grpc.CallOption) (*Empty, error) {\n\tout := new(Empty)\n\terr := c.cc.Invoke(ctx, VolumeSnapshotter_Init_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *volumeSnapshotterClient) CreateVolumeFromSnapshot(ctx context.Context, in *CreateVolumeRequest, opts ...grpc.CallOption) (*CreateVolumeResponse, error) {\n\tout := new(CreateVolumeResponse)\n\terr := c.cc.Invoke(ctx, VolumeSnapshotter_CreateVolumeFromSnapshot_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *volumeSnapshotterClient) GetVolumeInfo(ctx context.Context, in *GetVolumeInfoRequest, opts ...grpc.CallOption) (*GetVolumeInfoResponse, error) {\n\tout := new(GetVolumeInfoResponse)\n\terr := c.cc.Invoke(ctx, VolumeSnapshotter_GetVolumeInfo_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *volumeSnapshotterClient) CreateSnapshot(ctx context.Context, in *CreateSnapshotRequest, opts ...grpc.CallOption) (*CreateSnapshotResponse, error) {\n\tout := new(CreateSnapshotResponse)\n\terr := c.cc.Invoke(ctx, VolumeSnapshotter_CreateSnapshot_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *volumeSnapshotterClient) DeleteSnapshot(ctx context.Context, in *DeleteSnapshotRequest, opts ...grpc.CallOption) (*Empty, error) {\n\tout := new(Empty)\n\terr := c.cc.Invoke(ctx, VolumeSnapshotter_DeleteSnapshot_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *volumeSnapshotterClient) GetVolumeID(ctx context.Context, in *GetVolumeIDRequest, opts ...grpc.CallOption) (*GetVolumeIDResponse, error) {\n\tout := new(GetVolumeIDResponse)\n\terr := c.cc.Invoke(ctx, VolumeSnapshotter_GetVolumeID_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *volumeSnapshotterClient) SetVolumeID(ctx context.Context, in *SetVolumeIDRequest, opts ...grpc.CallOption) (*SetVolumeIDResponse, error) {\n\tout := new(SetVolumeIDResponse)\n\terr := c.cc.Invoke(ctx, VolumeSnapshotter_SetVolumeID_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\n// VolumeSnapshotterServer is the server API for VolumeSnapshotter service.\n// All implementations should embed UnimplementedVolumeSnapshotterServer\n// for forward compatibility\ntype VolumeSnapshotterServer interface {\n\tInit(context.Context, *VolumeSnapshotterInitRequest) (*Empty, error)\n\tCreateVolumeFromSnapshot(context.Context, *CreateVolumeRequest) (*CreateVolumeResponse, error)\n\tGetVolumeInfo(context.Context, *GetVolumeInfoRequest) (*GetVolumeInfoResponse, error)\n\tCreateSnapshot(context.Context, *CreateSnapshotRequest) (*CreateSnapshotResponse, error)\n\tDeleteSnapshot(context.Context, *DeleteSnapshotRequest) (*Empty, error)\n\tGetVolumeID(context.Context, *GetVolumeIDRequest) (*GetVolumeIDResponse, error)\n\tSetVolumeID(context.Context, *SetVolumeIDRequest) (*SetVolumeIDResponse, error)\n}\n\n// UnimplementedVolumeSnapshotterServer should be embedded to have forward compatible implementations.\ntype UnimplementedVolumeSnapshotterServer struct {\n}\n\nfunc (UnimplementedVolumeSnapshotterServer) Init(context.Context, *VolumeSnapshotterInitRequest) (*Empty, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method Init not implemented\")\n}\nfunc (UnimplementedVolumeSnapshotterServer) CreateVolumeFromSnapshot(context.Context, *CreateVolumeRequest) (*CreateVolumeResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method CreateVolumeFromSnapshot not implemented\")\n}\nfunc (UnimplementedVolumeSnapshotterServer) GetVolumeInfo(context.Context, *GetVolumeInfoRequest) (*GetVolumeInfoResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method GetVolumeInfo not implemented\")\n}\nfunc (UnimplementedVolumeSnapshotterServer) CreateSnapshot(context.Context, *CreateSnapshotRequest) (*CreateSnapshotResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method CreateSnapshot not implemented\")\n}\nfunc (UnimplementedVolumeSnapshotterServer) DeleteSnapshot(context.Context, *DeleteSnapshotRequest) (*Empty, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method DeleteSnapshot not implemented\")\n}\nfunc (UnimplementedVolumeSnapshotterServer) GetVolumeID(context.Context, *GetVolumeIDRequest) (*GetVolumeIDResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method GetVolumeID not implemented\")\n}\nfunc (UnimplementedVolumeSnapshotterServer) SetVolumeID(context.Context, *SetVolumeIDRequest) (*SetVolumeIDResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method SetVolumeID not implemented\")\n}\n\n// UnsafeVolumeSnapshotterServer may be embedded to opt out of forward compatibility for this service.\n// Use of this interface is not recommended, as added methods to VolumeSnapshotterServer will\n// result in compilation errors.\ntype UnsafeVolumeSnapshotterServer interface {\n\tmustEmbedUnimplementedVolumeSnapshotterServer()\n}\n\nfunc RegisterVolumeSnapshotterServer(s grpc.ServiceRegistrar, srv VolumeSnapshotterServer) {\n\ts.RegisterService(&VolumeSnapshotter_ServiceDesc, srv)\n}\n\nfunc _VolumeSnapshotter_Init_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(VolumeSnapshotterInitRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(VolumeSnapshotterServer).Init(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: VolumeSnapshotter_Init_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(VolumeSnapshotterServer).Init(ctx, req.(*VolumeSnapshotterInitRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _VolumeSnapshotter_CreateVolumeFromSnapshot_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(CreateVolumeRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(VolumeSnapshotterServer).CreateVolumeFromSnapshot(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: VolumeSnapshotter_CreateVolumeFromSnapshot_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(VolumeSnapshotterServer).CreateVolumeFromSnapshot(ctx, req.(*CreateVolumeRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _VolumeSnapshotter_GetVolumeInfo_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(GetVolumeInfoRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(VolumeSnapshotterServer).GetVolumeInfo(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: VolumeSnapshotter_GetVolumeInfo_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(VolumeSnapshotterServer).GetVolumeInfo(ctx, req.(*GetVolumeInfoRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _VolumeSnapshotter_CreateSnapshot_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(CreateSnapshotRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(VolumeSnapshotterServer).CreateSnapshot(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: VolumeSnapshotter_CreateSnapshot_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(VolumeSnapshotterServer).CreateSnapshot(ctx, req.(*CreateSnapshotRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _VolumeSnapshotter_DeleteSnapshot_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(DeleteSnapshotRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(VolumeSnapshotterServer).DeleteSnapshot(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: VolumeSnapshotter_DeleteSnapshot_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(VolumeSnapshotterServer).DeleteSnapshot(ctx, req.(*DeleteSnapshotRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _VolumeSnapshotter_GetVolumeID_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(GetVolumeIDRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(VolumeSnapshotterServer).GetVolumeID(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: VolumeSnapshotter_GetVolumeID_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(VolumeSnapshotterServer).GetVolumeID(ctx, req.(*GetVolumeIDRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _VolumeSnapshotter_SetVolumeID_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(SetVolumeIDRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(VolumeSnapshotterServer).SetVolumeID(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: VolumeSnapshotter_SetVolumeID_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(VolumeSnapshotterServer).SetVolumeID(ctx, req.(*SetVolumeIDRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\n// VolumeSnapshotter_ServiceDesc is the grpc.ServiceDesc for VolumeSnapshotter service.\n// It's only intended for direct use with grpc.RegisterService,\n// and not to be introspected or modified (even as a copy)\nvar VolumeSnapshotter_ServiceDesc = grpc.ServiceDesc{\n\tServiceName: \"generated.VolumeSnapshotter\",\n\tHandlerType: (*VolumeSnapshotterServer)(nil),\n\tMethods: []grpc.MethodDesc{\n\t\t{\n\t\t\tMethodName: \"Init\",\n\t\t\tHandler: _VolumeSnapshotter_Init_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"CreateVolumeFromSnapshot\",\n\t\t\tHandler: _VolumeSnapshotter_CreateVolumeFromSnapshot_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"GetVolumeInfo\",\n\t\t\tHandler: _VolumeSnapshotter_GetVolumeInfo_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"CreateSnapshot\",\n\t\t\tHandler: _VolumeSnapshotter_CreateSnapshot_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"DeleteSnapshot\",\n\t\t\tHandler: _VolumeSnapshotter_DeleteSnapshot_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"GetVolumeID\",\n\t\t\tHandler: _VolumeSnapshotter_GetVolumeID_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"SetVolumeID\",\n\t\t\tHandler: _VolumeSnapshotter_SetVolumeID_Handler,\n\t\t},\n\t},\n\tStreams: []grpc.StreamDesc{},\n\tMetadata: \"VolumeSnapshotter.proto\",\n}\n" }, { "path": "pkg/plugin/generated/backupitemaction/v2/BackupItemAction.pb.go", "content": "// Code generated by protoc-gen-go. DO NOT EDIT.\n// versions:\n// \tprotoc-gen-go v1.33.0\n// \tprotoc v4.25.2\n// source: backupitemaction/v2/BackupItemAction.proto\n\npackage v2\n\nimport (\n\tgenerated \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n\tprotoreflect \"google.golang.org/protobuf/reflect/protoreflect\"\n\tprotoimpl \"google.golang.org/protobuf/runtime/protoimpl\"\n\temptypb \"google.golang.org/protobuf/types/known/emptypb\"\n\treflect \"reflect\"\n\tsync \"sync\"\n)\n\nconst (\n\t// Verify that this generated code is sufficiently up-to-date.\n\t_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)\n\t// Verify that runtime/protoimpl is sufficiently up-to-date.\n\t_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)\n)\n\ntype ExecuteRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tItem []byte `protobuf:\"bytes,2,opt,name=item,proto3\" json:\"item,omitempty\"`\n\tBackup []byte `protobuf:\"bytes,3,opt,name=backup,proto3\" json:\"backup,omitempty\"`\n}\n\nfunc (x *ExecuteRequest) Reset() {\n\t*x = ExecuteRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_backupitemaction_v2_BackupItemAction_proto_msgTypes[0]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *ExecuteRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*ExecuteRequest) ProtoMessage() {}\n\nfunc (x *ExecuteRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_backupitemaction_v2_BackupItemAction_proto_msgTypes[0]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use ExecuteRequest.ProtoReflect.Descriptor instead.\nfunc (*ExecuteRequest) Descriptor() ([]byte, []int) {\n\treturn file_backupitemaction_v2_BackupItemAction_proto_rawDescGZIP(), []int{0}\n}\n\nfunc (x *ExecuteRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *ExecuteRequest) GetItem() []byte {\n\tif x != nil {\n\t\treturn x.Item\n\t}\n\treturn nil\n}\n\nfunc (x *ExecuteRequest) GetBackup() []byte {\n\tif x != nil {\n\t\treturn x.Backup\n\t}\n\treturn nil\n}\n\ntype ExecuteResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tItem []byte `protobuf:\"bytes,1,opt,name=item,proto3\" json:\"item,omitempty\"`\n\tAdditionalItems []*generated.ResourceIdentifier `protobuf:\"bytes,2,rep,name=additionalItems,proto3\" json:\"additionalItems,omitempty\"`\n\tOperationID string `protobuf:\"bytes,3,opt,name=operationID,proto3\" json:\"operationID,omitempty\"`\n\tPostOperationItems []*generated.ResourceIdentifier `protobuf:\"bytes,4,rep,name=postOperationItems,proto3\" json:\"postOperationItems,omitempty\"`\n}\n\nfunc (x *ExecuteResponse) Reset() {\n\t*x = ExecuteResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_backupitemaction_v2_BackupItemAction_proto_msgTypes[1]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *ExecuteResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*ExecuteResponse) ProtoMessage() {}\n\nfunc (x *ExecuteResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_backupitemaction_v2_BackupItemAction_proto_msgTypes[1]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use ExecuteResponse.ProtoReflect.Descriptor instead.\nfunc (*ExecuteResponse) Descriptor() ([]byte, []int) {\n\treturn file_backupitemaction_v2_BackupItemAction_proto_rawDescGZIP(), []int{1}\n}\n\nfunc (x *ExecuteResponse) GetItem() []byte {\n\tif x != nil {\n\t\treturn x.Item\n\t}\n\treturn nil\n}\n\nfunc (x *ExecuteResponse) GetAdditionalItems() []*generated.ResourceIdentifier {\n\tif x != nil {\n\t\treturn x.AdditionalItems\n\t}\n\treturn nil\n}\n\nfunc (x *ExecuteResponse) GetOperationID() string {\n\tif x != nil {\n\t\treturn x.OperationID\n\t}\n\treturn \"\"\n}\n\nfunc (x *ExecuteResponse) GetPostOperationItems() []*generated.ResourceIdentifier {\n\tif x != nil {\n\t\treturn x.PostOperationItems\n\t}\n\treturn nil\n}\n\ntype BackupItemActionAppliesToRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n}\n\nfunc (x *BackupItemActionAppliesToRequest) Reset() {\n\t*x = BackupItemActionAppliesToRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_backupitemaction_v2_BackupItemAction_proto_msgTypes[2]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *BackupItemActionAppliesToRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*BackupItemActionAppliesToRequest) ProtoMessage() {}\n\nfunc (x *BackupItemActionAppliesToRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_backupitemaction_v2_BackupItemAction_proto_msgTypes[2]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use BackupItemActionAppliesToRequest.ProtoReflect.Descriptor instead.\nfunc (*BackupItemActionAppliesToRequest) Descriptor() ([]byte, []int) {\n\treturn file_backupitemaction_v2_BackupItemAction_proto_rawDescGZIP(), []int{2}\n}\n\nfunc (x *BackupItemActionAppliesToRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\ntype BackupItemActionAppliesToResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tResourceSelector *generated.ResourceSelector `protobuf:\"bytes,1,opt,name=ResourceSelector,proto3\" json:\"ResourceSelector,omitempty\"`\n}\n\nfunc (x *BackupItemActionAppliesToResponse) Reset() {\n\t*x = BackupItemActionAppliesToResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_backupitemaction_v2_BackupItemAction_proto_msgTypes[3]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *BackupItemActionAppliesToResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*BackupItemActionAppliesToResponse) ProtoMessage() {}\n\nfunc (x *BackupItemActionAppliesToResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_backupitemaction_v2_BackupItemAction_proto_msgTypes[3]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use BackupItemActionAppliesToResponse.ProtoReflect.Descriptor instead.\nfunc (*BackupItemActionAppliesToResponse) Descriptor() ([]byte, []int) {\n\treturn file_backupitemaction_v2_BackupItemAction_proto_rawDescGZIP(), []int{3}\n}\n\nfunc (x *BackupItemActionAppliesToResponse) GetResourceSelector() *generated.ResourceSelector {\n\tif x != nil {\n\t\treturn x.ResourceSelector\n\t}\n\treturn nil\n}\n\ntype BackupItemActionProgressRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tOperationID string `protobuf:\"bytes,2,opt,name=operationID,proto3\" json:\"operationID,omitempty\"`\n\tBackup []byte `protobuf:\"bytes,3,opt,name=backup,proto3\" json:\"backup,omitempty\"`\n}\n\nfunc (x *BackupItemActionProgressRequest) Reset() {\n\t*x = BackupItemActionProgressRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_backupitemaction_v2_BackupItemAction_proto_msgTypes[4]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *BackupItemActionProgressRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*BackupItemActionProgressRequest) ProtoMessage() {}\n\nfunc (x *BackupItemActionProgressRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_backupitemaction_v2_BackupItemAction_proto_msgTypes[4]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use BackupItemActionProgressRequest.ProtoReflect.Descriptor instead.\nfunc (*BackupItemActionProgressRequest) Descriptor() ([]byte, []int) {\n\treturn file_backupitemaction_v2_BackupItemAction_proto_rawDescGZIP(), []int{4}\n}\n\nfunc (x *BackupItemActionProgressRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *BackupItemActionProgressRequest) GetOperationID() string {\n\tif x != nil {\n\t\treturn x.OperationID\n\t}\n\treturn \"\"\n}\n\nfunc (x *BackupItemActionProgressRequest) GetBackup() []byte {\n\tif x != nil {\n\t\treturn x.Backup\n\t}\n\treturn nil\n}\n\ntype BackupItemActionProgressResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tProgress *generated.OperationProgress `protobuf:\"bytes,1,opt,name=progress,proto3\" json:\"progress,omitempty\"`\n}\n\nfunc (x *BackupItemActionProgressResponse) Reset() {\n\t*x = BackupItemActionProgressResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_backupitemaction_v2_BackupItemAction_proto_msgTypes[5]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *BackupItemActionProgressResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*BackupItemActionProgressResponse) ProtoMessage() {}\n\nfunc (x *BackupItemActionProgressResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_backupitemaction_v2_BackupItemAction_proto_msgTypes[5]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use BackupItemActionProgressResponse.ProtoReflect.Descriptor instead.\nfunc (*BackupItemActionProgressResponse) Descriptor() ([]byte, []int) {\n\treturn file_backupitemaction_v2_BackupItemAction_proto_rawDescGZIP(), []int{5}\n}\n\nfunc (x *BackupItemActionProgressResponse) GetProgress() *generated.OperationProgress {\n\tif x != nil {\n\t\treturn x.Progress\n\t}\n\treturn nil\n}\n\ntype BackupItemActionCancelRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tOperationID string `protobuf:\"bytes,2,opt,name=operationID,proto3\" json:\"operationID,omitempty\"`\n\tBackup []byte `protobuf:\"bytes,3,opt,name=backup,proto3\" json:\"backup,omitempty\"`\n}\n\nfunc (x *BackupItemActionCancelRequest) Reset() {\n\t*x = BackupItemActionCancelRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_backupitemaction_v2_BackupItemAction_proto_msgTypes[6]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *BackupItemActionCancelRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*BackupItemActionCancelRequest) ProtoMessage() {}\n\nfunc (x *BackupItemActionCancelRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_backupitemaction_v2_BackupItemAction_proto_msgTypes[6]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use BackupItemActionCancelRequest.ProtoReflect.Descriptor instead.\nfunc (*BackupItemActionCancelRequest) Descriptor() ([]byte, []int) {\n\treturn file_backupitemaction_v2_BackupItemAction_proto_rawDescGZIP(), []int{6}\n}\n\nfunc (x *BackupItemActionCancelRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *BackupItemActionCancelRequest) GetOperationID() string {\n\tif x != nil {\n\t\treturn x.OperationID\n\t}\n\treturn \"\"\n}\n\nfunc (x *BackupItemActionCancelRequest) GetBackup() []byte {\n\tif x != nil {\n\t\treturn x.Backup\n\t}\n\treturn nil\n}\n\nvar File_backupitemaction_v2_BackupItemAction_proto protoreflect.FileDescriptor\n\nvar file_backupitemaction_v2_BackupItemAction_proto_rawDesc = []byte{\n\t0x0a, 0x2a, 0x62, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x69, 0x74, 0x65, 0x6d, 0x61, 0x63, 0x74, 0x69,\n\t0x6f, 0x6e, 0x2f, 0x76, 0x32, 0x2f, 0x42, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x49, 0x74, 0x65, 0x6d,\n\t0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x02, 0x76, 0x32,\n\t0x1a, 0x0c, 0x53, 0x68, 0x61, 0x72, 0x65, 0x64, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x1a, 0x1b,\n\t0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2f,\n\t0x65, 0x6d, 0x70, 0x74, 0x79, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, 0x54, 0x0a, 0x0e, 0x45,\n\t0x78, 0x65, 0x63, 0x75, 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x16, 0x0a,\n\t0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x70,\n\t0x6c, 0x75, 0x67, 0x69, 0x6e, 0x12, 0x12, 0x0a, 0x04, 0x69, 0x74, 0x65, 0x6d, 0x18, 0x02, 0x20,\n\t0x01, 0x28, 0x0c, 0x52, 0x04, 0x69, 0x74, 0x65, 0x6d, 0x12, 0x16, 0x0a, 0x06, 0x62, 0x61, 0x63,\n\t0x6b, 0x75, 0x70, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x06, 0x62, 0x61, 0x63, 0x6b, 0x75,\n\t0x70, 0x22, 0xdf, 0x01, 0x0a, 0x0f, 0x45, 0x78, 0x65, 0x63, 0x75, 0x74, 0x65, 0x52, 0x65, 0x73,\n\t0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x69, 0x74, 0x65, 0x6d, 0x18, 0x01, 0x20,\n\t0x01, 0x28, 0x0c, 0x52, 0x04, 0x69, 0x74, 0x65, 0x6d, 0x12, 0x47, 0x0a, 0x0f, 0x61, 0x64, 0x64,\n\t0x69, 0x74, 0x69, 0x6f, 0x6e, 0x61, 0x6c, 0x49, 0x74, 0x65, 0x6d, 0x73, 0x18, 0x02, 0x20, 0x03,\n\t0x28, 0x0b, 0x32, 0x1d, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x52,\n\t0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x49, 0x64, 0x65, 0x6e, 0x74, 0x69, 0x66, 0x69, 0x65,\n\t0x72, 0x52, 0x0f, 0x61, 0x64, 0x64, 0x69, 0x74, 0x69, 0x6f, 0x6e, 0x61, 0x6c, 0x49, 0x74, 0x65,\n\t0x6d, 0x73, 0x12, 0x20, 0x0a, 0x0b, 0x6f, 0x70, 0x65, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x49,\n\t0x44, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0b, 0x6f, 0x70, 0x65, 0x72, 0x61, 0x74, 0x69,\n\t0x6f, 0x6e, 0x49, 0x44, 0x12, 0x4d, 0x0a, 0x12, 0x70, 0x6f, 0x73, 0x74, 0x4f, 0x70, 0x65, 0x72,\n\t0x61, 0x74, 0x69, 0x6f, 0x6e, 0x49, 0x74, 0x65, 0x6d, 0x73, 0x18, 0x04, 0x20, 0x03, 0x28, 0x0b,\n\t0x32, 0x1d, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x52, 0x65, 0x73,\n\t0x6f, 0x75, 0x72, 0x63, 0x65, 0x49, 0x64, 0x65, 0x6e, 0x74, 0x69, 0x66, 0x69, 0x65, 0x72, 0x52,\n\t0x12, 0x70, 0x6f, 0x73, 0x74, 0x4f, 0x70, 0x65, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x49, 0x74,\n\t0x65, 0x6d, 0x73, 0x22, 0x3a, 0x0a, 0x20, 0x42, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x49, 0x74, 0x65,\n\t0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x41, 0x70, 0x70, 0x6c, 0x69, 0x65, 0x73, 0x54, 0x6f,\n\t0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x16, 0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69,\n\t0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x22,\n\t0x6c, 0x0a, 0x21, 0x42, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74,\n\t0x69, 0x6f, 0x6e, 0x41, 0x70, 0x70, 0x6c, 0x69, 0x65, 0x73, 0x54, 0x6f, 0x52, 0x65, 0x73, 0x70,\n\t0x6f, 0x6e, 0x73, 0x65, 0x12, 0x47, 0x0a, 0x10, 0x52, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65,\n\t0x53, 0x65, 0x6c, 0x65, 0x63, 0x74, 0x6f, 0x72, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1b,\n\t0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x52, 0x65, 0x73, 0x6f, 0x75,\n\t0x72, 0x63, 0x65, 0x53, 0x65, 0x6c, 0x65, 0x63, 0x74, 0x6f, 0x72, 0x52, 0x10, 0x52, 0x65, 0x73,\n\t0x6f, 0x75, 0x72, 0x63, 0x65, 0x53, 0x65, 0x6c, 0x65, 0x63, 0x74, 0x6f, 0x72, 0x22, 0x73, 0x0a,\n\t0x1f, 0x42, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f,\n\t0x6e, 0x50, 0x72, 0x6f, 0x67, 0x72, 0x65, 0x73, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74,\n\t0x12, 0x16, 0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09,\n\t0x52, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x12, 0x20, 0x0a, 0x0b, 0x6f, 0x70, 0x65, 0x72,\n\t0x61, 0x74, 0x69, 0x6f, 0x6e, 0x49, 0x44, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0b, 0x6f,\n\t0x70, 0x65, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x49, 0x44, 0x12, 0x16, 0x0a, 0x06, 0x62, 0x61,\n\t0x63, 0x6b, 0x75, 0x70, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x06, 0x62, 0x61, 0x63, 0x6b,\n\t0x75, 0x70, 0x22, 0x5c, 0x0a, 0x20, 0x42, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x49, 0x74, 0x65, 0x6d,\n\t0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x50, 0x72, 0x6f, 0x67, 0x72, 0x65, 0x73, 0x73, 0x52, 0x65,\n\t0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x38, 0x0a, 0x08, 0x70, 0x72, 0x6f, 0x67, 0x72, 0x65,\n\t0x73, 0x73, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1c, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72,\n\t0x61, 0x74, 0x65, 0x64, 0x2e, 0x4f, 0x70, 0x65, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x50, 0x72,\n\t0x6f, 0x67, 0x72, 0x65, 0x73, 0x73, 0x52, 0x08, 0x70, 0x72, 0x6f, 0x67, 0x72, 0x65, 0x73, 0x73,\n\t0x22, 0x71, 0x0a, 0x1d, 0x42, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63,\n\t0x74, 0x69, 0x6f, 0x6e, 0x43, 0x61, 0x6e, 0x63, 0x65, 0x6c, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73,\n\t0x74, 0x12, 0x16, 0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28,\n\t0x09, 0x52, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x12, 0x20, 0x0a, 0x0b, 0x6f, 0x70, 0x65,\n\t0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x49, 0x44, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0b,\n\t0x6f, 0x70, 0x65, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x49, 0x44, 0x12, 0x16, 0x0a, 0x06, 0x62,\n\t0x61, 0x63, 0x6b, 0x75, 0x70, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x06, 0x62, 0x61, 0x63,\n\t0x6b, 0x75, 0x70, 0x32, 0xbc, 0x02, 0x0a, 0x10, 0x42, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x49, 0x74,\n\t0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x58, 0x0a, 0x09, 0x41, 0x70, 0x70, 0x6c,\n\t0x69, 0x65, 0x73, 0x54, 0x6f, 0x12, 0x24, 0x2e, 0x76, 0x32, 0x2e, 0x42, 0x61, 0x63, 0x6b, 0x75,\n\t0x70, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x41, 0x70, 0x70, 0x6c, 0x69,\n\t0x65, 0x73, 0x54, 0x6f, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x25, 0x2e, 0x76, 0x32,\n\t0x2e, 0x42, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f,\n\t0x6e, 0x41, 0x70, 0x70, 0x6c, 0x69, 0x65, 0x73, 0x54, 0x6f, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e,\n\t0x73, 0x65, 0x12, 0x32, 0x0a, 0x07, 0x45, 0x78, 0x65, 0x63, 0x75, 0x74, 0x65, 0x12, 0x12, 0x2e,\n\t0x76, 0x32, 0x2e, 0x45, 0x78, 0x65, 0x63, 0x75, 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73,\n\t0x74, 0x1a, 0x13, 0x2e, 0x76, 0x32, 0x2e, 0x45, 0x78, 0x65, 0x63, 0x75, 0x74, 0x65, 0x52, 0x65,\n\t0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x55, 0x0a, 0x08, 0x50, 0x72, 0x6f, 0x67, 0x72, 0x65,\n\t0x73, 0x73, 0x12, 0x23, 0x2e, 0x76, 0x32, 0x2e, 0x42, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x49, 0x74,\n\t0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x50, 0x72, 0x6f, 0x67, 0x72, 0x65, 0x73, 0x73,\n\t0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x24, 0x2e, 0x76, 0x32, 0x2e, 0x42, 0x61, 0x63,\n\t0x6b, 0x75, 0x70, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x50, 0x72, 0x6f,\n\t0x67, 0x72, 0x65, 0x73, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x43, 0x0a,\n\t0x06, 0x43, 0x61, 0x6e, 0x63, 0x65, 0x6c, 0x12, 0x21, 0x2e, 0x76, 0x32, 0x2e, 0x42, 0x61, 0x63,\n\t0x6b, 0x75, 0x70, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x43, 0x61, 0x6e,\n\t0x63, 0x65, 0x6c, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x16, 0x2e, 0x67, 0x6f, 0x6f,\n\t0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70,\n\t0x74, 0x79, 0x42, 0x49, 0x5a, 0x47, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d,\n\t0x2f, 0x76, 0x6d, 0x77, 0x61, 0x72, 0x65, 0x2d, 0x74, 0x61, 0x6e, 0x7a, 0x75, 0x2f, 0x76, 0x65,\n\t0x6c, 0x65, 0x72, 0x6f, 0x2f, 0x70, 0x6b, 0x67, 0x2f, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x2f,\n\t0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2f, 0x62, 0x61, 0x63, 0x6b, 0x75, 0x70,\n\t0x69, 0x74, 0x65, 0x6d, 0x61, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x2f, 0x76, 0x32, 0x62, 0x06, 0x70,\n\t0x72, 0x6f, 0x74, 0x6f, 0x33,\n}\n\nvar (\n\tfile_backupitemaction_v2_BackupItemAction_proto_rawDescOnce sync.Once\n\tfile_backupitemaction_v2_BackupItemAction_proto_rawDescData = file_backupitemaction_v2_BackupItemAction_proto_rawDesc\n)\n\nfunc file_backupitemaction_v2_BackupItemAction_proto_rawDescGZIP() []byte {\n\tfile_backupitemaction_v2_BackupItemAction_proto_rawDescOnce.Do(func() {\n\t\tfile_backupitemaction_v2_BackupItemAction_proto_rawDescData = protoimpl.X.CompressGZIP(file_backupitemaction_v2_BackupItemAction_proto_rawDescData)\n\t})\n\treturn file_backupitemaction_v2_BackupItemAction_proto_rawDescData\n}\n\nvar file_backupitemaction_v2_BackupItemAction_proto_msgTypes = make([]protoimpl.MessageInfo, 7)\nvar file_backupitemaction_v2_BackupItemAction_proto_goTypes = []interface{}{\n\t(*ExecuteRequest)(nil), // 0: v2.ExecuteRequest\n\t(*ExecuteResponse)(nil), // 1: v2.ExecuteResponse\n\t(*BackupItemActionAppliesToRequest)(nil), // 2: v2.BackupItemActionAppliesToRequest\n\t(*BackupItemActionAppliesToResponse)(nil), // 3: v2.BackupItemActionAppliesToResponse\n\t(*BackupItemActionProgressRequest)(nil), // 4: v2.BackupItemActionProgressRequest\n\t(*BackupItemActionProgressResponse)(nil), // 5: v2.BackupItemActionProgressResponse\n\t(*BackupItemActionCancelRequest)(nil), // 6: v2.BackupItemActionCancelRequest\n\t(*generated.ResourceIdentifier)(nil), // 7: generated.ResourceIdentifier\n\t(*generated.ResourceSelector)(nil), // 8: generated.ResourceSelector\n\t(*generated.OperationProgress)(nil), // 9: generated.OperationProgress\n\t(*emptypb.Empty)(nil), // 10: google.protobuf.Empty\n}\nvar file_backupitemaction_v2_BackupItemAction_proto_depIdxs = []int32{\n\t7, // 0: v2.ExecuteResponse.additionalItems:type_name -> generated.ResourceIdentifier\n\t7, // 1: v2.ExecuteResponse.postOperationItems:type_name -> generated.ResourceIdentifier\n\t8, // 2: v2.BackupItemActionAppliesToResponse.ResourceSelector:type_name -> generated.ResourceSelector\n\t9, // 3: v2.BackupItemActionProgressResponse.progress:type_name -> generated.OperationProgress\n\t2, // 4: v2.BackupItemAction.AppliesTo:input_type -> v2.BackupItemActionAppliesToRequest\n\t0, // 5: v2.BackupItemAction.Execute:input_type -> v2.ExecuteRequest\n\t4, // 6: v2.BackupItemAction.Progress:input_type -> v2.BackupItemActionProgressRequest\n\t6, // 7: v2.BackupItemAction.Cancel:input_type -> v2.BackupItemActionCancelRequest\n\t3, // 8: v2.BackupItemAction.AppliesTo:output_type -> v2.BackupItemActionAppliesToResponse\n\t1, // 9: v2.BackupItemAction.Execute:output_type -> v2.ExecuteResponse\n\t5, // 10: v2.BackupItemAction.Progress:output_type -> v2.BackupItemActionProgressResponse\n\t10, // 11: v2.BackupItemAction.Cancel:output_type -> google.protobuf.Empty\n\t8, // [8:12] is the sub-list for method output_type\n\t4, // [4:8] is the sub-list for method input_type\n\t4, // [4:4] is the sub-list for extension type_name\n\t4, // [4:4] is the sub-list for extension extendee\n\t0, // [0:4] is the sub-list for field type_name\n}\n\nfunc init() { file_backupitemaction_v2_BackupItemAction_proto_init() }\nfunc file_backupitemaction_v2_BackupItemAction_proto_init() {\n\tif File_backupitemaction_v2_BackupItemAction_proto != nil {\n\t\treturn\n\t}\n\tif !protoimpl.UnsafeEnabled {\n\t\tfile_backupitemaction_v2_BackupItemAction_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*ExecuteRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_backupitemaction_v2_BackupItemAction_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*ExecuteResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_backupitemaction_v2_BackupItemAction_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*BackupItemActionAppliesToRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_backupitemaction_v2_BackupItemAction_proto_msgTypes[3].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*BackupItemActionAppliesToResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_backupitemaction_v2_BackupItemAction_proto_msgTypes[4].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*BackupItemActionProgressRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_backupitemaction_v2_BackupItemAction_proto_msgTypes[5].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*BackupItemActionProgressResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_backupitemaction_v2_BackupItemAction_proto_msgTypes[6].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*BackupItemActionCancelRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t}\n\ttype x struct{}\n\tout := protoimpl.TypeBuilder{\n\t\tFile: protoimpl.DescBuilder{\n\t\t\tGoPackagePath: reflect.TypeOf(x{}).PkgPath(),\n\t\t\tRawDescriptor: file_backupitemaction_v2_BackupItemAction_proto_rawDesc,\n\t\t\tNumEnums: 0,\n\t\t\tNumMessages: 7,\n\t\t\tNumExtensions: 0,\n\t\t\tNumServices: 1,\n\t\t},\n\t\tGoTypes: file_backupitemaction_v2_BackupItemAction_proto_goTypes,\n\t\tDependencyIndexes: file_backupitemaction_v2_BackupItemAction_proto_depIdxs,\n\t\tMessageInfos: file_backupitemaction_v2_BackupItemAction_proto_msgTypes,\n\t}.Build()\n\tFile_backupitemaction_v2_BackupItemAction_proto = out.File\n\tfile_backupitemaction_v2_BackupItemAction_proto_rawDesc = nil\n\tfile_backupitemaction_v2_BackupItemAction_proto_goTypes = nil\n\tfile_backupitemaction_v2_BackupItemAction_proto_depIdxs = nil\n}\n" }, { "path": "pkg/plugin/generated/backupitemaction/v2/BackupItemAction_grpc.pb.go", "content": "// Code generated by protoc-gen-go-grpc. DO NOT EDIT.\n// versions:\n// - protoc-gen-go-grpc v1.3.0\n// - protoc v4.25.2\n// source: backupitemaction/v2/BackupItemAction.proto\n\npackage v2\n\nimport (\n\tcontext \"context\"\n\tgrpc \"google.golang.org/grpc\"\n\tcodes \"google.golang.org/grpc/codes\"\n\tstatus \"google.golang.org/grpc/status\"\n\temptypb \"google.golang.org/protobuf/types/known/emptypb\"\n)\n\n// This is a compile-time assertion to ensure that this generated file\n// is compatible with the grpc package it is being compiled against.\n// Requires gRPC-Go v1.32.0 or later.\nconst _ = grpc.SupportPackageIsVersion7\n\nconst (\n\tBackupItemAction_AppliesTo_FullMethodName = \"/v2.BackupItemAction/AppliesTo\"\n\tBackupItemAction_Execute_FullMethodName = \"/v2.BackupItemAction/Execute\"\n\tBackupItemAction_Progress_FullMethodName = \"/v2.BackupItemAction/Progress\"\n\tBackupItemAction_Cancel_FullMethodName = \"/v2.BackupItemAction/Cancel\"\n)\n\n// BackupItemActionClient is the client API for BackupItemAction service.\n//\n// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.\ntype BackupItemActionClient interface {\n\tAppliesTo(ctx context.Context, in *BackupItemActionAppliesToRequest, opts ...grpc.CallOption) (*BackupItemActionAppliesToResponse, error)\n\tExecute(ctx context.Context, in *ExecuteRequest, opts ...grpc.CallOption) (*ExecuteResponse, error)\n\tProgress(ctx context.Context, in *BackupItemActionProgressRequest, opts ...grpc.CallOption) (*BackupItemActionProgressResponse, error)\n\tCancel(ctx context.Context, in *BackupItemActionCancelRequest, opts ...grpc.CallOption) (*emptypb.Empty, error)\n}\n\ntype backupItemActionClient struct {\n\tcc grpc.ClientConnInterface\n}\n\nfunc NewBackupItemActionClient(cc grpc.ClientConnInterface) BackupItemActionClient {\n\treturn &backupItemActionClient{cc}\n}\n\nfunc (c *backupItemActionClient) AppliesTo(ctx context.Context, in *BackupItemActionAppliesToRequest, opts ...grpc.CallOption) (*BackupItemActionAppliesToResponse, error) {\n\tout := new(BackupItemActionAppliesToResponse)\n\terr := c.cc.Invoke(ctx, BackupItemAction_AppliesTo_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *backupItemActionClient) Execute(ctx context.Context, in *ExecuteRequest, opts ...grpc.CallOption) (*ExecuteResponse, error) {\n\tout := new(ExecuteResponse)\n\terr := c.cc.Invoke(ctx, BackupItemAction_Execute_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *backupItemActionClient) Progress(ctx context.Context, in *BackupItemActionProgressRequest, opts ...grpc.CallOption) (*BackupItemActionProgressResponse, error) {\n\tout := new(BackupItemActionProgressResponse)\n\terr := c.cc.Invoke(ctx, BackupItemAction_Progress_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *backupItemActionClient) Cancel(ctx context.Context, in *BackupItemActionCancelRequest, opts ...grpc.CallOption) (*emptypb.Empty, error) {\n\tout := new(emptypb.Empty)\n\terr := c.cc.Invoke(ctx, BackupItemAction_Cancel_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\n// BackupItemActionServer is the server API for BackupItemAction service.\n// All implementations should embed UnimplementedBackupItemActionServer\n// for forward compatibility\ntype BackupItemActionServer interface {\n\tAppliesTo(context.Context, *BackupItemActionAppliesToRequest) (*BackupItemActionAppliesToResponse, error)\n\tExecute(context.Context, *ExecuteRequest) (*ExecuteResponse, error)\n\tProgress(context.Context, *BackupItemActionProgressRequest) (*BackupItemActionProgressResponse, error)\n\tCancel(context.Context, *BackupItemActionCancelRequest) (*emptypb.Empty, error)\n}\n\n// UnimplementedBackupItemActionServer should be embedded to have forward compatible implementations.\ntype UnimplementedBackupItemActionServer struct {\n}\n\nfunc (UnimplementedBackupItemActionServer) AppliesTo(context.Context, *BackupItemActionAppliesToRequest) (*BackupItemActionAppliesToResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method AppliesTo not implemented\")\n}\nfunc (UnimplementedBackupItemActionServer) Execute(context.Context, *ExecuteRequest) (*ExecuteResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method Execute not implemented\")\n}\nfunc (UnimplementedBackupItemActionServer) Progress(context.Context, *BackupItemActionProgressRequest) (*BackupItemActionProgressResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method Progress not implemented\")\n}\nfunc (UnimplementedBackupItemActionServer) Cancel(context.Context, *BackupItemActionCancelRequest) (*emptypb.Empty, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method Cancel not implemented\")\n}\n\n// UnsafeBackupItemActionServer may be embedded to opt out of forward compatibility for this service.\n// Use of this interface is not recommended, as added methods to BackupItemActionServer will\n// result in compilation errors.\ntype UnsafeBackupItemActionServer interface {\n\tmustEmbedUnimplementedBackupItemActionServer()\n}\n\nfunc RegisterBackupItemActionServer(s grpc.ServiceRegistrar, srv BackupItemActionServer) {\n\ts.RegisterService(&BackupItemAction_ServiceDesc, srv)\n}\n\nfunc _BackupItemAction_AppliesTo_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(BackupItemActionAppliesToRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(BackupItemActionServer).AppliesTo(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: BackupItemAction_AppliesTo_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(BackupItemActionServer).AppliesTo(ctx, req.(*BackupItemActionAppliesToRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _BackupItemAction_Execute_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(ExecuteRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(BackupItemActionServer).Execute(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: BackupItemAction_Execute_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(BackupItemActionServer).Execute(ctx, req.(*ExecuteRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _BackupItemAction_Progress_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(BackupItemActionProgressRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(BackupItemActionServer).Progress(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: BackupItemAction_Progress_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(BackupItemActionServer).Progress(ctx, req.(*BackupItemActionProgressRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _BackupItemAction_Cancel_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(BackupItemActionCancelRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(BackupItemActionServer).Cancel(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: BackupItemAction_Cancel_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(BackupItemActionServer).Cancel(ctx, req.(*BackupItemActionCancelRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\n// BackupItemAction_ServiceDesc is the grpc.ServiceDesc for BackupItemAction service.\n// It's only intended for direct use with grpc.RegisterService,\n// and not to be introspected or modified (even as a copy)\nvar BackupItemAction_ServiceDesc = grpc.ServiceDesc{\n\tServiceName: \"v2.BackupItemAction\",\n\tHandlerType: (*BackupItemActionServer)(nil),\n\tMethods: []grpc.MethodDesc{\n\t\t{\n\t\t\tMethodName: \"AppliesTo\",\n\t\t\tHandler: _BackupItemAction_AppliesTo_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"Execute\",\n\t\t\tHandler: _BackupItemAction_Execute_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"Progress\",\n\t\t\tHandler: _BackupItemAction_Progress_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"Cancel\",\n\t\t\tHandler: _BackupItemAction_Cancel_Handler,\n\t\t},\n\t},\n\tStreams: []grpc.StreamDesc{},\n\tMetadata: \"backupitemaction/v2/BackupItemAction.proto\",\n}\n" }, { "path": "pkg/plugin/generated/itemblockaction/v1/ItemBlockAction.pb.go", "content": "// Code generated by protoc-gen-go. DO NOT EDIT.\n// versions:\n// \tprotoc-gen-go v1.33.0\n// \tprotoc v4.25.2\n// source: itemblockaction/v1/ItemBlockAction.proto\n\npackage v1\n\nimport (\n\tgenerated \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n\tprotoreflect \"google.golang.org/protobuf/reflect/protoreflect\"\n\tprotoimpl \"google.golang.org/protobuf/runtime/protoimpl\"\n\treflect \"reflect\"\n\tsync \"sync\"\n)\n\nconst (\n\t// Verify that this generated code is sufficiently up-to-date.\n\t_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)\n\t// Verify that runtime/protoimpl is sufficiently up-to-date.\n\t_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)\n)\n\ntype ItemBlockActionAppliesToRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n}\n\nfunc (x *ItemBlockActionAppliesToRequest) Reset() {\n\t*x = ItemBlockActionAppliesToRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_itemblockaction_v1_ItemBlockAction_proto_msgTypes[0]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *ItemBlockActionAppliesToRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*ItemBlockActionAppliesToRequest) ProtoMessage() {}\n\nfunc (x *ItemBlockActionAppliesToRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_itemblockaction_v1_ItemBlockAction_proto_msgTypes[0]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use ItemBlockActionAppliesToRequest.ProtoReflect.Descriptor instead.\nfunc (*ItemBlockActionAppliesToRequest) Descriptor() ([]byte, []int) {\n\treturn file_itemblockaction_v1_ItemBlockAction_proto_rawDescGZIP(), []int{0}\n}\n\nfunc (x *ItemBlockActionAppliesToRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\ntype ItemBlockActionAppliesToResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tResourceSelector *generated.ResourceSelector `protobuf:\"bytes,1,opt,name=ResourceSelector,proto3\" json:\"ResourceSelector,omitempty\"`\n}\n\nfunc (x *ItemBlockActionAppliesToResponse) Reset() {\n\t*x = ItemBlockActionAppliesToResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_itemblockaction_v1_ItemBlockAction_proto_msgTypes[1]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *ItemBlockActionAppliesToResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*ItemBlockActionAppliesToResponse) ProtoMessage() {}\n\nfunc (x *ItemBlockActionAppliesToResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_itemblockaction_v1_ItemBlockAction_proto_msgTypes[1]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use ItemBlockActionAppliesToResponse.ProtoReflect.Descriptor instead.\nfunc (*ItemBlockActionAppliesToResponse) Descriptor() ([]byte, []int) {\n\treturn file_itemblockaction_v1_ItemBlockAction_proto_rawDescGZIP(), []int{1}\n}\n\nfunc (x *ItemBlockActionAppliesToResponse) GetResourceSelector() *generated.ResourceSelector {\n\tif x != nil {\n\t\treturn x.ResourceSelector\n\t}\n\treturn nil\n}\n\ntype ItemBlockActionGetRelatedItemsRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tItem []byte `protobuf:\"bytes,2,opt,name=item,proto3\" json:\"item,omitempty\"`\n\tBackup []byte `protobuf:\"bytes,3,opt,name=backup,proto3\" json:\"backup,omitempty\"`\n}\n\nfunc (x *ItemBlockActionGetRelatedItemsRequest) Reset() {\n\t*x = ItemBlockActionGetRelatedItemsRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_itemblockaction_v1_ItemBlockAction_proto_msgTypes[2]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *ItemBlockActionGetRelatedItemsRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*ItemBlockActionGetRelatedItemsRequest) ProtoMessage() {}\n\nfunc (x *ItemBlockActionGetRelatedItemsRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_itemblockaction_v1_ItemBlockAction_proto_msgTypes[2]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use ItemBlockActionGetRelatedItemsRequest.ProtoReflect.Descriptor instead.\nfunc (*ItemBlockActionGetRelatedItemsRequest) Descriptor() ([]byte, []int) {\n\treturn file_itemblockaction_v1_ItemBlockAction_proto_rawDescGZIP(), []int{2}\n}\n\nfunc (x *ItemBlockActionGetRelatedItemsRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *ItemBlockActionGetRelatedItemsRequest) GetItem() []byte {\n\tif x != nil {\n\t\treturn x.Item\n\t}\n\treturn nil\n}\n\nfunc (x *ItemBlockActionGetRelatedItemsRequest) GetBackup() []byte {\n\tif x != nil {\n\t\treturn x.Backup\n\t}\n\treturn nil\n}\n\ntype ItemBlockActionGetRelatedItemsResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tRelatedItems []*generated.ResourceIdentifier `protobuf:\"bytes,1,rep,name=relatedItems,proto3\" json:\"relatedItems,omitempty\"`\n}\n\nfunc (x *ItemBlockActionGetRelatedItemsResponse) Reset() {\n\t*x = ItemBlockActionGetRelatedItemsResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_itemblockaction_v1_ItemBlockAction_proto_msgTypes[3]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *ItemBlockActionGetRelatedItemsResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*ItemBlockActionGetRelatedItemsResponse) ProtoMessage() {}\n\nfunc (x *ItemBlockActionGetRelatedItemsResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_itemblockaction_v1_ItemBlockAction_proto_msgTypes[3]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use ItemBlockActionGetRelatedItemsResponse.ProtoReflect.Descriptor instead.\nfunc (*ItemBlockActionGetRelatedItemsResponse) Descriptor() ([]byte, []int) {\n\treturn file_itemblockaction_v1_ItemBlockAction_proto_rawDescGZIP(), []int{3}\n}\n\nfunc (x *ItemBlockActionGetRelatedItemsResponse) GetRelatedItems() []*generated.ResourceIdentifier {\n\tif x != nil {\n\t\treturn x.RelatedItems\n\t}\n\treturn nil\n}\n\nvar File_itemblockaction_v1_ItemBlockAction_proto protoreflect.FileDescriptor\n\nvar file_itemblockaction_v1_ItemBlockAction_proto_rawDesc = []byte{\n\t0x0a, 0x28, 0x69, 0x74, 0x65, 0x6d, 0x62, 0x6c, 0x6f, 0x63, 0x6b, 0x61, 0x63, 0x74, 0x69, 0x6f,\n\t0x6e, 0x2f, 0x76, 0x31, 0x2f, 0x49, 0x74, 0x65, 0x6d, 0x42, 0x6c, 0x6f, 0x63, 0x6b, 0x41, 0x63,\n\t0x74, 0x69, 0x6f, 0x6e, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x02, 0x76, 0x31, 0x1a, 0x0c,\n\t0x53, 0x68, 0x61, 0x72, 0x65, 0x64, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, 0x39, 0x0a, 0x1f,\n\t0x49, 0x74, 0x65, 0x6d, 0x42, 0x6c, 0x6f, 0x63, 0x6b, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x41,\n\t0x70, 0x70, 0x6c, 0x69, 0x65, 0x73, 0x54, 0x6f, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12,\n\t0x16, 0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52,\n\t0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x22, 0x6b, 0x0a, 0x20, 0x49, 0x74, 0x65, 0x6d, 0x42,\n\t0x6c, 0x6f, 0x63, 0x6b, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x41, 0x70, 0x70, 0x6c, 0x69, 0x65,\n\t0x73, 0x54, 0x6f, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x47, 0x0a, 0x10, 0x52,\n\t0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x53, 0x65, 0x6c, 0x65, 0x63, 0x74, 0x6f, 0x72, 0x18,\n\t0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1b, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65,\n\t0x64, 0x2e, 0x52, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x53, 0x65, 0x6c, 0x65, 0x63, 0x74,\n\t0x6f, 0x72, 0x52, 0x10, 0x52, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x53, 0x65, 0x6c, 0x65,\n\t0x63, 0x74, 0x6f, 0x72, 0x22, 0x6b, 0x0a, 0x25, 0x49, 0x74, 0x65, 0x6d, 0x42, 0x6c, 0x6f, 0x63,\n\t0x6b, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x47, 0x65, 0x74, 0x52, 0x65, 0x6c, 0x61, 0x74, 0x65,\n\t0x64, 0x49, 0x74, 0x65, 0x6d, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x16, 0x0a,\n\t0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x70,\n\t0x6c, 0x75, 0x67, 0x69, 0x6e, 0x12, 0x12, 0x0a, 0x04, 0x69, 0x74, 0x65, 0x6d, 0x18, 0x02, 0x20,\n\t0x01, 0x28, 0x0c, 0x52, 0x04, 0x69, 0x74, 0x65, 0x6d, 0x12, 0x16, 0x0a, 0x06, 0x62, 0x61, 0x63,\n\t0x6b, 0x75, 0x70, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x06, 0x62, 0x61, 0x63, 0x6b, 0x75,\n\t0x70, 0x22, 0x6b, 0x0a, 0x26, 0x49, 0x74, 0x65, 0x6d, 0x42, 0x6c, 0x6f, 0x63, 0x6b, 0x41, 0x63,\n\t0x74, 0x69, 0x6f, 0x6e, 0x47, 0x65, 0x74, 0x52, 0x65, 0x6c, 0x61, 0x74, 0x65, 0x64, 0x49, 0x74,\n\t0x65, 0x6d, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x41, 0x0a, 0x0c, 0x72,\n\t0x65, 0x6c, 0x61, 0x74, 0x65, 0x64, 0x49, 0x74, 0x65, 0x6d, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28,\n\t0x0b, 0x32, 0x1d, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x52, 0x65,\n\t0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x49, 0x64, 0x65, 0x6e, 0x74, 0x69, 0x66, 0x69, 0x65, 0x72,\n\t0x52, 0x0c, 0x72, 0x65, 0x6c, 0x61, 0x74, 0x65, 0x64, 0x49, 0x74, 0x65, 0x6d, 0x73, 0x32, 0xd3,\n\t0x01, 0x0a, 0x0f, 0x49, 0x74, 0x65, 0x6d, 0x42, 0x6c, 0x6f, 0x63, 0x6b, 0x41, 0x63, 0x74, 0x69,\n\t0x6f, 0x6e, 0x12, 0x56, 0x0a, 0x09, 0x41, 0x70, 0x70, 0x6c, 0x69, 0x65, 0x73, 0x54, 0x6f, 0x12,\n\t0x23, 0x2e, 0x76, 0x31, 0x2e, 0x49, 0x74, 0x65, 0x6d, 0x42, 0x6c, 0x6f, 0x63, 0x6b, 0x41, 0x63,\n\t0x74, 0x69, 0x6f, 0x6e, 0x41, 0x70, 0x70, 0x6c, 0x69, 0x65, 0x73, 0x54, 0x6f, 0x52, 0x65, 0x71,\n\t0x75, 0x65, 0x73, 0x74, 0x1a, 0x24, 0x2e, 0x76, 0x31, 0x2e, 0x49, 0x74, 0x65, 0x6d, 0x42, 0x6c,\n\t0x6f, 0x63, 0x6b, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x41, 0x70, 0x70, 0x6c, 0x69, 0x65, 0x73,\n\t0x54, 0x6f, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x68, 0x0a, 0x0f, 0x47, 0x65,\n\t0x74, 0x52, 0x65, 0x6c, 0x61, 0x74, 0x65, 0x64, 0x49, 0x74, 0x65, 0x6d, 0x73, 0x12, 0x29, 0x2e,\n\t0x76, 0x31, 0x2e, 0x49, 0x74, 0x65, 0x6d, 0x42, 0x6c, 0x6f, 0x63, 0x6b, 0x41, 0x63, 0x74, 0x69,\n\t0x6f, 0x6e, 0x47, 0x65, 0x74, 0x52, 0x65, 0x6c, 0x61, 0x74, 0x65, 0x64, 0x49, 0x74, 0x65, 0x6d,\n\t0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2a, 0x2e, 0x76, 0x31, 0x2e, 0x49, 0x74,\n\t0x65, 0x6d, 0x42, 0x6c, 0x6f, 0x63, 0x6b, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x47, 0x65, 0x74,\n\t0x52, 0x65, 0x6c, 0x61, 0x74, 0x65, 0x64, 0x49, 0x74, 0x65, 0x6d, 0x73, 0x52, 0x65, 0x73, 0x70,\n\t0x6f, 0x6e, 0x73, 0x65, 0x42, 0x48, 0x5a, 0x46, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63,\n\t0x6f, 0x6d, 0x2f, 0x76, 0x6d, 0x77, 0x61, 0x72, 0x65, 0x2d, 0x74, 0x61, 0x6e, 0x7a, 0x75, 0x2f,\n\t0x76, 0x65, 0x6c, 0x65, 0x72, 0x6f, 0x2f, 0x70, 0x6b, 0x67, 0x2f, 0x70, 0x6c, 0x75, 0x67, 0x69,\n\t0x6e, 0x2f, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2f, 0x69, 0x74, 0x65, 0x6d,\n\t0x62, 0x6c, 0x6f, 0x63, 0x6b, 0x61, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x2f, 0x76, 0x31, 0x62, 0x06,\n\t0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,\n}\n\nvar (\n\tfile_itemblockaction_v1_ItemBlockAction_proto_rawDescOnce sync.Once\n\tfile_itemblockaction_v1_ItemBlockAction_proto_rawDescData = file_itemblockaction_v1_ItemBlockAction_proto_rawDesc\n)\n\nfunc file_itemblockaction_v1_ItemBlockAction_proto_rawDescGZIP() []byte {\n\tfile_itemblockaction_v1_ItemBlockAction_proto_rawDescOnce.Do(func() {\n\t\tfile_itemblockaction_v1_ItemBlockAction_proto_rawDescData = protoimpl.X.CompressGZIP(file_itemblockaction_v1_ItemBlockAction_proto_rawDescData)\n\t})\n\treturn file_itemblockaction_v1_ItemBlockAction_proto_rawDescData\n}\n\nvar file_itemblockaction_v1_ItemBlockAction_proto_msgTypes = make([]protoimpl.MessageInfo, 4)\nvar file_itemblockaction_v1_ItemBlockAction_proto_goTypes = []interface{}{\n\t(*ItemBlockActionAppliesToRequest)(nil), // 0: v1.ItemBlockActionAppliesToRequest\n\t(*ItemBlockActionAppliesToResponse)(nil), // 1: v1.ItemBlockActionAppliesToResponse\n\t(*ItemBlockActionGetRelatedItemsRequest)(nil), // 2: v1.ItemBlockActionGetRelatedItemsRequest\n\t(*ItemBlockActionGetRelatedItemsResponse)(nil), // 3: v1.ItemBlockActionGetRelatedItemsResponse\n\t(*generated.ResourceSelector)(nil), // 4: generated.ResourceSelector\n\t(*generated.ResourceIdentifier)(nil), // 5: generated.ResourceIdentifier\n}\nvar file_itemblockaction_v1_ItemBlockAction_proto_depIdxs = []int32{\n\t4, // 0: v1.ItemBlockActionAppliesToResponse.ResourceSelector:type_name -> generated.ResourceSelector\n\t5, // 1: v1.ItemBlockActionGetRelatedItemsResponse.relatedItems:type_name -> generated.ResourceIdentifier\n\t0, // 2: v1.ItemBlockAction.AppliesTo:input_type -> v1.ItemBlockActionAppliesToRequest\n\t2, // 3: v1.ItemBlockAction.GetRelatedItems:input_type -> v1.ItemBlockActionGetRelatedItemsRequest\n\t1, // 4: v1.ItemBlockAction.AppliesTo:output_type -> v1.ItemBlockActionAppliesToResponse\n\t3, // 5: v1.ItemBlockAction.GetRelatedItems:output_type -> v1.ItemBlockActionGetRelatedItemsResponse\n\t4, // [4:6] is the sub-list for method output_type\n\t2, // [2:4] is the sub-list for method input_type\n\t2, // [2:2] is the sub-list for extension type_name\n\t2, // [2:2] is the sub-list for extension extendee\n\t0, // [0:2] is the sub-list for field type_name\n}\n\nfunc init() { file_itemblockaction_v1_ItemBlockAction_proto_init() }\nfunc file_itemblockaction_v1_ItemBlockAction_proto_init() {\n\tif File_itemblockaction_v1_ItemBlockAction_proto != nil {\n\t\treturn\n\t}\n\tif !protoimpl.UnsafeEnabled {\n\t\tfile_itemblockaction_v1_ItemBlockAction_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*ItemBlockActionAppliesToRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_itemblockaction_v1_ItemBlockAction_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*ItemBlockActionAppliesToResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_itemblockaction_v1_ItemBlockAction_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*ItemBlockActionGetRelatedItemsRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_itemblockaction_v1_ItemBlockAction_proto_msgTypes[3].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*ItemBlockActionGetRelatedItemsResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t}\n\ttype x struct{}\n\tout := protoimpl.TypeBuilder{\n\t\tFile: protoimpl.DescBuilder{\n\t\t\tGoPackagePath: reflect.TypeOf(x{}).PkgPath(),\n\t\t\tRawDescriptor: file_itemblockaction_v1_ItemBlockAction_proto_rawDesc,\n\t\t\tNumEnums: 0,\n\t\t\tNumMessages: 4,\n\t\t\tNumExtensions: 0,\n\t\t\tNumServices: 1,\n\t\t},\n\t\tGoTypes: file_itemblockaction_v1_ItemBlockAction_proto_goTypes,\n\t\tDependencyIndexes: file_itemblockaction_v1_ItemBlockAction_proto_depIdxs,\n\t\tMessageInfos: file_itemblockaction_v1_ItemBlockAction_proto_msgTypes,\n\t}.Build()\n\tFile_itemblockaction_v1_ItemBlockAction_proto = out.File\n\tfile_itemblockaction_v1_ItemBlockAction_proto_rawDesc = nil\n\tfile_itemblockaction_v1_ItemBlockAction_proto_goTypes = nil\n\tfile_itemblockaction_v1_ItemBlockAction_proto_depIdxs = nil\n}\n" }, { "path": "pkg/plugin/generated/itemblockaction/v1/ItemBlockAction_grpc.pb.go", "content": "// Code generated by protoc-gen-go-grpc. DO NOT EDIT.\n// versions:\n// - protoc-gen-go-grpc v1.3.0\n// - protoc v4.25.2\n// source: itemblockaction/v1/ItemBlockAction.proto\n\npackage v1\n\nimport (\n\tcontext \"context\"\n\tgrpc \"google.golang.org/grpc\"\n\tcodes \"google.golang.org/grpc/codes\"\n\tstatus \"google.golang.org/grpc/status\"\n)\n\n// This is a compile-time assertion to ensure that this generated file\n// is compatible with the grpc package it is being compiled against.\n// Requires gRPC-Go v1.32.0 or later.\nconst _ = grpc.SupportPackageIsVersion7\n\nconst (\n\tItemBlockAction_AppliesTo_FullMethodName = \"/v1.ItemBlockAction/AppliesTo\"\n\tItemBlockAction_GetRelatedItems_FullMethodName = \"/v1.ItemBlockAction/GetRelatedItems\"\n)\n\n// ItemBlockActionClient is the client API for ItemBlockAction service.\n//\n// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.\ntype ItemBlockActionClient interface {\n\tAppliesTo(ctx context.Context, in *ItemBlockActionAppliesToRequest, opts ...grpc.CallOption) (*ItemBlockActionAppliesToResponse, error)\n\tGetRelatedItems(ctx context.Context, in *ItemBlockActionGetRelatedItemsRequest, opts ...grpc.CallOption) (*ItemBlockActionGetRelatedItemsResponse, error)\n}\n\ntype itemBlockActionClient struct {\n\tcc grpc.ClientConnInterface\n}\n\nfunc NewItemBlockActionClient(cc grpc.ClientConnInterface) ItemBlockActionClient {\n\treturn &itemBlockActionClient{cc}\n}\n\nfunc (c *itemBlockActionClient) AppliesTo(ctx context.Context, in *ItemBlockActionAppliesToRequest, opts ...grpc.CallOption) (*ItemBlockActionAppliesToResponse, error) {\n\tout := new(ItemBlockActionAppliesToResponse)\n\terr := c.cc.Invoke(ctx, ItemBlockAction_AppliesTo_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *itemBlockActionClient) GetRelatedItems(ctx context.Context, in *ItemBlockActionGetRelatedItemsRequest, opts ...grpc.CallOption) (*ItemBlockActionGetRelatedItemsResponse, error) {\n\tout := new(ItemBlockActionGetRelatedItemsResponse)\n\terr := c.cc.Invoke(ctx, ItemBlockAction_GetRelatedItems_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\n// ItemBlockActionServer is the server API for ItemBlockAction service.\n// All implementations should embed UnimplementedItemBlockActionServer\n// for forward compatibility\ntype ItemBlockActionServer interface {\n\tAppliesTo(context.Context, *ItemBlockActionAppliesToRequest) (*ItemBlockActionAppliesToResponse, error)\n\tGetRelatedItems(context.Context, *ItemBlockActionGetRelatedItemsRequest) (*ItemBlockActionGetRelatedItemsResponse, error)\n}\n\n// UnimplementedItemBlockActionServer should be embedded to have forward compatible implementations.\ntype UnimplementedItemBlockActionServer struct {\n}\n\nfunc (UnimplementedItemBlockActionServer) AppliesTo(context.Context, *ItemBlockActionAppliesToRequest) (*ItemBlockActionAppliesToResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method AppliesTo not implemented\")\n}\nfunc (UnimplementedItemBlockActionServer) GetRelatedItems(context.Context, *ItemBlockActionGetRelatedItemsRequest) (*ItemBlockActionGetRelatedItemsResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method GetRelatedItems not implemented\")\n}\n\n// UnsafeItemBlockActionServer may be embedded to opt out of forward compatibility for this service.\n// Use of this interface is not recommended, as added methods to ItemBlockActionServer will\n// result in compilation errors.\ntype UnsafeItemBlockActionServer interface {\n\tmustEmbedUnimplementedItemBlockActionServer()\n}\n\nfunc RegisterItemBlockActionServer(s grpc.ServiceRegistrar, srv ItemBlockActionServer) {\n\ts.RegisterService(&ItemBlockAction_ServiceDesc, srv)\n}\n\nfunc _ItemBlockAction_AppliesTo_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(ItemBlockActionAppliesToRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(ItemBlockActionServer).AppliesTo(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: ItemBlockAction_AppliesTo_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(ItemBlockActionServer).AppliesTo(ctx, req.(*ItemBlockActionAppliesToRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _ItemBlockAction_GetRelatedItems_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(ItemBlockActionGetRelatedItemsRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(ItemBlockActionServer).GetRelatedItems(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: ItemBlockAction_GetRelatedItems_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(ItemBlockActionServer).GetRelatedItems(ctx, req.(*ItemBlockActionGetRelatedItemsRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\n// ItemBlockAction_ServiceDesc is the grpc.ServiceDesc for ItemBlockAction service.\n// It's only intended for direct use with grpc.RegisterService,\n// and not to be introspected or modified (even as a copy)\nvar ItemBlockAction_ServiceDesc = grpc.ServiceDesc{\n\tServiceName: \"v1.ItemBlockAction\",\n\tHandlerType: (*ItemBlockActionServer)(nil),\n\tMethods: []grpc.MethodDesc{\n\t\t{\n\t\t\tMethodName: \"AppliesTo\",\n\t\t\tHandler: _ItemBlockAction_AppliesTo_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"GetRelatedItems\",\n\t\t\tHandler: _ItemBlockAction_GetRelatedItems_Handler,\n\t\t},\n\t},\n\tStreams: []grpc.StreamDesc{},\n\tMetadata: \"itemblockaction/v1/ItemBlockAction.proto\",\n}\n" }, { "path": "pkg/plugin/generated/restoreitemaction/v2/RestoreItemAction.pb.go", "content": "// Code generated by protoc-gen-go. DO NOT EDIT.\n// versions:\n// \tprotoc-gen-go v1.33.0\n// \tprotoc v4.25.2\n// source: restoreitemaction/v2/RestoreItemAction.proto\n\npackage v2\n\nimport (\n\tgenerated \"github.com/vmware-tanzu/velero/pkg/plugin/generated\"\n\tprotoreflect \"google.golang.org/protobuf/reflect/protoreflect\"\n\tprotoimpl \"google.golang.org/protobuf/runtime/protoimpl\"\n\tdurationpb \"google.golang.org/protobuf/types/known/durationpb\"\n\temptypb \"google.golang.org/protobuf/types/known/emptypb\"\n\treflect \"reflect\"\n\tsync \"sync\"\n)\n\nconst (\n\t// Verify that this generated code is sufficiently up-to-date.\n\t_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)\n\t// Verify that runtime/protoimpl is sufficiently up-to-date.\n\t_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)\n)\n\ntype RestoreItemActionExecuteRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tItem []byte `protobuf:\"bytes,2,opt,name=item,proto3\" json:\"item,omitempty\"`\n\tRestore []byte `protobuf:\"bytes,3,opt,name=restore,proto3\" json:\"restore,omitempty\"`\n\tItemFromBackup []byte `protobuf:\"bytes,4,opt,name=itemFromBackup,proto3\" json:\"itemFromBackup,omitempty\"`\n}\n\nfunc (x *RestoreItemActionExecuteRequest) Reset() {\n\t*x = RestoreItemActionExecuteRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[0]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *RestoreItemActionExecuteRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*RestoreItemActionExecuteRequest) ProtoMessage() {}\n\nfunc (x *RestoreItemActionExecuteRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[0]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use RestoreItemActionExecuteRequest.ProtoReflect.Descriptor instead.\nfunc (*RestoreItemActionExecuteRequest) Descriptor() ([]byte, []int) {\n\treturn file_restoreitemaction_v2_RestoreItemAction_proto_rawDescGZIP(), []int{0}\n}\n\nfunc (x *RestoreItemActionExecuteRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *RestoreItemActionExecuteRequest) GetItem() []byte {\n\tif x != nil {\n\t\treturn x.Item\n\t}\n\treturn nil\n}\n\nfunc (x *RestoreItemActionExecuteRequest) GetRestore() []byte {\n\tif x != nil {\n\t\treturn x.Restore\n\t}\n\treturn nil\n}\n\nfunc (x *RestoreItemActionExecuteRequest) GetItemFromBackup() []byte {\n\tif x != nil {\n\t\treturn x.ItemFromBackup\n\t}\n\treturn nil\n}\n\ntype RestoreItemActionExecuteResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tItem []byte `protobuf:\"bytes,1,opt,name=item,proto3\" json:\"item,omitempty\"`\n\tAdditionalItems []*generated.ResourceIdentifier `protobuf:\"bytes,2,rep,name=additionalItems,proto3\" json:\"additionalItems,omitempty\"`\n\tSkipRestore bool `protobuf:\"varint,3,opt,name=skipRestore,proto3\" json:\"skipRestore,omitempty\"`\n\tOperationID string `protobuf:\"bytes,4,opt,name=operationID,proto3\" json:\"operationID,omitempty\"`\n\tWaitForAdditionalItems bool `protobuf:\"varint,5,opt,name=waitForAdditionalItems,proto3\" json:\"waitForAdditionalItems,omitempty\"`\n\tAdditionalItemsReadyTimeout *durationpb.Duration `protobuf:\"bytes,6,opt,name=additionalItemsReadyTimeout,proto3\" json:\"additionalItemsReadyTimeout,omitempty\"`\n}\n\nfunc (x *RestoreItemActionExecuteResponse) Reset() {\n\t*x = RestoreItemActionExecuteResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[1]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *RestoreItemActionExecuteResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*RestoreItemActionExecuteResponse) ProtoMessage() {}\n\nfunc (x *RestoreItemActionExecuteResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[1]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use RestoreItemActionExecuteResponse.ProtoReflect.Descriptor instead.\nfunc (*RestoreItemActionExecuteResponse) Descriptor() ([]byte, []int) {\n\treturn file_restoreitemaction_v2_RestoreItemAction_proto_rawDescGZIP(), []int{1}\n}\n\nfunc (x *RestoreItemActionExecuteResponse) GetItem() []byte {\n\tif x != nil {\n\t\treturn x.Item\n\t}\n\treturn nil\n}\n\nfunc (x *RestoreItemActionExecuteResponse) GetAdditionalItems() []*generated.ResourceIdentifier {\n\tif x != nil {\n\t\treturn x.AdditionalItems\n\t}\n\treturn nil\n}\n\nfunc (x *RestoreItemActionExecuteResponse) GetSkipRestore() bool {\n\tif x != nil {\n\t\treturn x.SkipRestore\n\t}\n\treturn false\n}\n\nfunc (x *RestoreItemActionExecuteResponse) GetOperationID() string {\n\tif x != nil {\n\t\treturn x.OperationID\n\t}\n\treturn \"\"\n}\n\nfunc (x *RestoreItemActionExecuteResponse) GetWaitForAdditionalItems() bool {\n\tif x != nil {\n\t\treturn x.WaitForAdditionalItems\n\t}\n\treturn false\n}\n\nfunc (x *RestoreItemActionExecuteResponse) GetAdditionalItemsReadyTimeout() *durationpb.Duration {\n\tif x != nil {\n\t\treturn x.AdditionalItemsReadyTimeout\n\t}\n\treturn nil\n}\n\ntype RestoreItemActionAppliesToRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n}\n\nfunc (x *RestoreItemActionAppliesToRequest) Reset() {\n\t*x = RestoreItemActionAppliesToRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[2]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *RestoreItemActionAppliesToRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*RestoreItemActionAppliesToRequest) ProtoMessage() {}\n\nfunc (x *RestoreItemActionAppliesToRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[2]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use RestoreItemActionAppliesToRequest.ProtoReflect.Descriptor instead.\nfunc (*RestoreItemActionAppliesToRequest) Descriptor() ([]byte, []int) {\n\treturn file_restoreitemaction_v2_RestoreItemAction_proto_rawDescGZIP(), []int{2}\n}\n\nfunc (x *RestoreItemActionAppliesToRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\ntype RestoreItemActionAppliesToResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tResourceSelector *generated.ResourceSelector `protobuf:\"bytes,1,opt,name=ResourceSelector,proto3\" json:\"ResourceSelector,omitempty\"`\n}\n\nfunc (x *RestoreItemActionAppliesToResponse) Reset() {\n\t*x = RestoreItemActionAppliesToResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[3]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *RestoreItemActionAppliesToResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*RestoreItemActionAppliesToResponse) ProtoMessage() {}\n\nfunc (x *RestoreItemActionAppliesToResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[3]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use RestoreItemActionAppliesToResponse.ProtoReflect.Descriptor instead.\nfunc (*RestoreItemActionAppliesToResponse) Descriptor() ([]byte, []int) {\n\treturn file_restoreitemaction_v2_RestoreItemAction_proto_rawDescGZIP(), []int{3}\n}\n\nfunc (x *RestoreItemActionAppliesToResponse) GetResourceSelector() *generated.ResourceSelector {\n\tif x != nil {\n\t\treturn x.ResourceSelector\n\t}\n\treturn nil\n}\n\ntype RestoreItemActionProgressRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tOperationID string `protobuf:\"bytes,2,opt,name=operationID,proto3\" json:\"operationID,omitempty\"`\n\tRestore []byte `protobuf:\"bytes,3,opt,name=restore,proto3\" json:\"restore,omitempty\"`\n}\n\nfunc (x *RestoreItemActionProgressRequest) Reset() {\n\t*x = RestoreItemActionProgressRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[4]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *RestoreItemActionProgressRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*RestoreItemActionProgressRequest) ProtoMessage() {}\n\nfunc (x *RestoreItemActionProgressRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[4]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use RestoreItemActionProgressRequest.ProtoReflect.Descriptor instead.\nfunc (*RestoreItemActionProgressRequest) Descriptor() ([]byte, []int) {\n\treturn file_restoreitemaction_v2_RestoreItemAction_proto_rawDescGZIP(), []int{4}\n}\n\nfunc (x *RestoreItemActionProgressRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *RestoreItemActionProgressRequest) GetOperationID() string {\n\tif x != nil {\n\t\treturn x.OperationID\n\t}\n\treturn \"\"\n}\n\nfunc (x *RestoreItemActionProgressRequest) GetRestore() []byte {\n\tif x != nil {\n\t\treturn x.Restore\n\t}\n\treturn nil\n}\n\ntype RestoreItemActionProgressResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tProgress *generated.OperationProgress `protobuf:\"bytes,1,opt,name=progress,proto3\" json:\"progress,omitempty\"`\n}\n\nfunc (x *RestoreItemActionProgressResponse) Reset() {\n\t*x = RestoreItemActionProgressResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[5]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *RestoreItemActionProgressResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*RestoreItemActionProgressResponse) ProtoMessage() {}\n\nfunc (x *RestoreItemActionProgressResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[5]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use RestoreItemActionProgressResponse.ProtoReflect.Descriptor instead.\nfunc (*RestoreItemActionProgressResponse) Descriptor() ([]byte, []int) {\n\treturn file_restoreitemaction_v2_RestoreItemAction_proto_rawDescGZIP(), []int{5}\n}\n\nfunc (x *RestoreItemActionProgressResponse) GetProgress() *generated.OperationProgress {\n\tif x != nil {\n\t\treturn x.Progress\n\t}\n\treturn nil\n}\n\ntype RestoreItemActionCancelRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tOperationID string `protobuf:\"bytes,2,opt,name=operationID,proto3\" json:\"operationID,omitempty\"`\n\tRestore []byte `protobuf:\"bytes,3,opt,name=restore,proto3\" json:\"restore,omitempty\"`\n}\n\nfunc (x *RestoreItemActionCancelRequest) Reset() {\n\t*x = RestoreItemActionCancelRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[6]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *RestoreItemActionCancelRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*RestoreItemActionCancelRequest) ProtoMessage() {}\n\nfunc (x *RestoreItemActionCancelRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[6]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use RestoreItemActionCancelRequest.ProtoReflect.Descriptor instead.\nfunc (*RestoreItemActionCancelRequest) Descriptor() ([]byte, []int) {\n\treturn file_restoreitemaction_v2_RestoreItemAction_proto_rawDescGZIP(), []int{6}\n}\n\nfunc (x *RestoreItemActionCancelRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *RestoreItemActionCancelRequest) GetOperationID() string {\n\tif x != nil {\n\t\treturn x.OperationID\n\t}\n\treturn \"\"\n}\n\nfunc (x *RestoreItemActionCancelRequest) GetRestore() []byte {\n\tif x != nil {\n\t\treturn x.Restore\n\t}\n\treturn nil\n}\n\ntype RestoreItemActionItemsReadyRequest struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tPlugin string `protobuf:\"bytes,1,opt,name=plugin,proto3\" json:\"plugin,omitempty\"`\n\tRestore []byte `protobuf:\"bytes,2,opt,name=restore,proto3\" json:\"restore,omitempty\"`\n\tAdditionalItems []*generated.ResourceIdentifier `protobuf:\"bytes,3,rep,name=additionalItems,proto3\" json:\"additionalItems,omitempty\"`\n}\n\nfunc (x *RestoreItemActionItemsReadyRequest) Reset() {\n\t*x = RestoreItemActionItemsReadyRequest{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[7]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *RestoreItemActionItemsReadyRequest) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*RestoreItemActionItemsReadyRequest) ProtoMessage() {}\n\nfunc (x *RestoreItemActionItemsReadyRequest) ProtoReflect() protoreflect.Message {\n\tmi := &file_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[7]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use RestoreItemActionItemsReadyRequest.ProtoReflect.Descriptor instead.\nfunc (*RestoreItemActionItemsReadyRequest) Descriptor() ([]byte, []int) {\n\treturn file_restoreitemaction_v2_RestoreItemAction_proto_rawDescGZIP(), []int{7}\n}\n\nfunc (x *RestoreItemActionItemsReadyRequest) GetPlugin() string {\n\tif x != nil {\n\t\treturn x.Plugin\n\t}\n\treturn \"\"\n}\n\nfunc (x *RestoreItemActionItemsReadyRequest) GetRestore() []byte {\n\tif x != nil {\n\t\treturn x.Restore\n\t}\n\treturn nil\n}\n\nfunc (x *RestoreItemActionItemsReadyRequest) GetAdditionalItems() []*generated.ResourceIdentifier {\n\tif x != nil {\n\t\treturn x.AdditionalItems\n\t}\n\treturn nil\n}\n\ntype RestoreItemActionItemsReadyResponse struct {\n\tstate protoimpl.MessageState\n\tsizeCache protoimpl.SizeCache\n\tunknownFields protoimpl.UnknownFields\n\n\tReady bool `protobuf:\"varint,1,opt,name=ready,proto3\" json:\"ready,omitempty\"`\n}\n\nfunc (x *RestoreItemActionItemsReadyResponse) Reset() {\n\t*x = RestoreItemActionItemsReadyResponse{}\n\tif protoimpl.UnsafeEnabled {\n\t\tmi := &file_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[8]\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tms.StoreMessageInfo(mi)\n\t}\n}\n\nfunc (x *RestoreItemActionItemsReadyResponse) String() string {\n\treturn protoimpl.X.MessageStringOf(x)\n}\n\nfunc (*RestoreItemActionItemsReadyResponse) ProtoMessage() {}\n\nfunc (x *RestoreItemActionItemsReadyResponse) ProtoReflect() protoreflect.Message {\n\tmi := &file_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[8]\n\tif protoimpl.UnsafeEnabled && x != nil {\n\t\tms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))\n\t\tif ms.LoadMessageInfo() == nil {\n\t\t\tms.StoreMessageInfo(mi)\n\t\t}\n\t\treturn ms\n\t}\n\treturn mi.MessageOf(x)\n}\n\n// Deprecated: Use RestoreItemActionItemsReadyResponse.ProtoReflect.Descriptor instead.\nfunc (*RestoreItemActionItemsReadyResponse) Descriptor() ([]byte, []int) {\n\treturn file_restoreitemaction_v2_RestoreItemAction_proto_rawDescGZIP(), []int{8}\n}\n\nfunc (x *RestoreItemActionItemsReadyResponse) GetReady() bool {\n\tif x != nil {\n\t\treturn x.Ready\n\t}\n\treturn false\n}\n\nvar File_restoreitemaction_v2_RestoreItemAction_proto protoreflect.FileDescriptor\n\nvar file_restoreitemaction_v2_RestoreItemAction_proto_rawDesc = []byte{\n\t0x0a, 0x2c, 0x72, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x69, 0x74, 0x65, 0x6d, 0x61, 0x63, 0x74,\n\t0x69, 0x6f, 0x6e, 0x2f, 0x76, 0x32, 0x2f, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x49, 0x74,\n\t0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x02,\n\t0x76, 0x32, 0x1a, 0x0c, 0x53, 0x68, 0x61, 0x72, 0x65, 0x64, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f,\n\t0x1a, 0x1b, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75,\n\t0x66, 0x2f, 0x65, 0x6d, 0x70, 0x74, 0x79, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x1a, 0x1e, 0x67,\n\t0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2f, 0x64,\n\t0x75, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, 0x8f, 0x01,\n\t0x0a, 0x1f, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74,\n\t0x69, 0x6f, 0x6e, 0x45, 0x78, 0x65, 0x63, 0x75, 0x74, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73,\n\t0x74, 0x12, 0x16, 0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28,\n\t0x09, 0x52, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x12, 0x12, 0x0a, 0x04, 0x69, 0x74, 0x65,\n\t0x6d, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x04, 0x69, 0x74, 0x65, 0x6d, 0x12, 0x18, 0x0a,\n\t0x07, 0x72, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x07,\n\t0x72, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x12, 0x26, 0x0a, 0x0e, 0x69, 0x74, 0x65, 0x6d, 0x46,\n\t0x72, 0x6f, 0x6d, 0x42, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x18, 0x04, 0x20, 0x01, 0x28, 0x0c, 0x52,\n\t0x0e, 0x69, 0x74, 0x65, 0x6d, 0x46, 0x72, 0x6f, 0x6d, 0x42, 0x61, 0x63, 0x6b, 0x75, 0x70, 0x22,\n\t0xd8, 0x02, 0x0a, 0x20, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41,\n\t0x63, 0x74, 0x69, 0x6f, 0x6e, 0x45, 0x78, 0x65, 0x63, 0x75, 0x74, 0x65, 0x52, 0x65, 0x73, 0x70,\n\t0x6f, 0x6e, 0x73, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x69, 0x74, 0x65, 0x6d, 0x18, 0x01, 0x20, 0x01,\n\t0x28, 0x0c, 0x52, 0x04, 0x69, 0x74, 0x65, 0x6d, 0x12, 0x47, 0x0a, 0x0f, 0x61, 0x64, 0x64, 0x69,\n\t0x74, 0x69, 0x6f, 0x6e, 0x61, 0x6c, 0x49, 0x74, 0x65, 0x6d, 0x73, 0x18, 0x02, 0x20, 0x03, 0x28,\n\t0x0b, 0x32, 0x1d, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2e, 0x52, 0x65,\n\t0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x49, 0x64, 0x65, 0x6e, 0x74, 0x69, 0x66, 0x69, 0x65, 0x72,\n\t0x52, 0x0f, 0x61, 0x64, 0x64, 0x69, 0x74, 0x69, 0x6f, 0x6e, 0x61, 0x6c, 0x49, 0x74, 0x65, 0x6d,\n\t0x73, 0x12, 0x20, 0x0a, 0x0b, 0x73, 0x6b, 0x69, 0x70, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65,\n\t0x18, 0x03, 0x20, 0x01, 0x28, 0x08, 0x52, 0x0b, 0x73, 0x6b, 0x69, 0x70, 0x52, 0x65, 0x73, 0x74,\n\t0x6f, 0x72, 0x65, 0x12, 0x20, 0x0a, 0x0b, 0x6f, 0x70, 0x65, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e,\n\t0x49, 0x44, 0x18, 0x04, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0b, 0x6f, 0x70, 0x65, 0x72, 0x61, 0x74,\n\t0x69, 0x6f, 0x6e, 0x49, 0x44, 0x12, 0x36, 0x0a, 0x16, 0x77, 0x61, 0x69, 0x74, 0x46, 0x6f, 0x72,\n\t0x41, 0x64, 0x64, 0x69, 0x74, 0x69, 0x6f, 0x6e, 0x61, 0x6c, 0x49, 0x74, 0x65, 0x6d, 0x73, 0x18,\n\t0x05, 0x20, 0x01, 0x28, 0x08, 0x52, 0x16, 0x77, 0x61, 0x69, 0x74, 0x46, 0x6f, 0x72, 0x41, 0x64,\n\t0x64, 0x69, 0x74, 0x69, 0x6f, 0x6e, 0x61, 0x6c, 0x49, 0x74, 0x65, 0x6d, 0x73, 0x12, 0x5b, 0x0a,\n\t0x1b, 0x61, 0x64, 0x64, 0x69, 0x74, 0x69, 0x6f, 0x6e, 0x61, 0x6c, 0x49, 0x74, 0x65, 0x6d, 0x73,\n\t0x52, 0x65, 0x61, 0x64, 0x79, 0x54, 0x69, 0x6d, 0x65, 0x6f, 0x75, 0x74, 0x18, 0x06, 0x20, 0x01,\n\t0x28, 0x0b, 0x32, 0x19, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74,\n\t0x6f, 0x62, 0x75, 0x66, 0x2e, 0x44, 0x75, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x1b, 0x61,\n\t0x64, 0x64, 0x69, 0x74, 0x69, 0x6f, 0x6e, 0x61, 0x6c, 0x49, 0x74, 0x65, 0x6d, 0x73, 0x52, 0x65,\n\t0x61, 0x64, 0x79, 0x54, 0x69, 0x6d, 0x65, 0x6f, 0x75, 0x74, 0x22, 0x3b, 0x0a, 0x21, 0x52, 0x65,\n\t0x73, 0x74, 0x6f, 0x72, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x41,\n\t0x70, 0x70, 0x6c, 0x69, 0x65, 0x73, 0x54, 0x6f, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12,\n\t0x16, 0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52,\n\t0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x22, 0x6d, 0x0a, 0x22, 0x52, 0x65, 0x73, 0x74, 0x6f,\n\t0x72, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x41, 0x70, 0x70, 0x6c,\n\t0x69, 0x65, 0x73, 0x54, 0x6f, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x47, 0x0a,\n\t0x10, 0x52, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x53, 0x65, 0x6c, 0x65, 0x63, 0x74, 0x6f,\n\t0x72, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1b, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61,\n\t0x74, 0x65, 0x64, 0x2e, 0x52, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x53, 0x65, 0x6c, 0x65,\n\t0x63, 0x74, 0x6f, 0x72, 0x52, 0x10, 0x52, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x53, 0x65,\n\t0x6c, 0x65, 0x63, 0x74, 0x6f, 0x72, 0x22, 0x76, 0x0a, 0x20, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72,\n\t0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x50, 0x72, 0x6f, 0x67, 0x72,\n\t0x65, 0x73, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x16, 0x0a, 0x06, 0x70, 0x6c,\n\t0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x70, 0x6c, 0x75, 0x67,\n\t0x69, 0x6e, 0x12, 0x20, 0x0a, 0x0b, 0x6f, 0x70, 0x65, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x49,\n\t0x44, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0b, 0x6f, 0x70, 0x65, 0x72, 0x61, 0x74, 0x69,\n\t0x6f, 0x6e, 0x49, 0x44, 0x12, 0x18, 0x0a, 0x07, 0x72, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x18,\n\t0x03, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x07, 0x72, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x22, 0x5d,\n\t0x0a, 0x21, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74,\n\t0x69, 0x6f, 0x6e, 0x50, 0x72, 0x6f, 0x67, 0x72, 0x65, 0x73, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f,\n\t0x6e, 0x73, 0x65, 0x12, 0x38, 0x0a, 0x08, 0x70, 0x72, 0x6f, 0x67, 0x72, 0x65, 0x73, 0x73, 0x18,\n\t0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1c, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65,\n\t0x64, 0x2e, 0x4f, 0x70, 0x65, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x50, 0x72, 0x6f, 0x67, 0x72,\n\t0x65, 0x73, 0x73, 0x52, 0x08, 0x70, 0x72, 0x6f, 0x67, 0x72, 0x65, 0x73, 0x73, 0x22, 0x74, 0x0a,\n\t0x1e, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69,\n\t0x6f, 0x6e, 0x43, 0x61, 0x6e, 0x63, 0x65, 0x6c, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12,\n\t0x16, 0x0a, 0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52,\n\t0x06, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x12, 0x20, 0x0a, 0x0b, 0x6f, 0x70, 0x65, 0x72, 0x61,\n\t0x74, 0x69, 0x6f, 0x6e, 0x49, 0x44, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0b, 0x6f, 0x70,\n\t0x65, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x49, 0x44, 0x12, 0x18, 0x0a, 0x07, 0x72, 0x65, 0x73,\n\t0x74, 0x6f, 0x72, 0x65, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x07, 0x72, 0x65, 0x73, 0x74,\n\t0x6f, 0x72, 0x65, 0x22, 0x9f, 0x01, 0x0a, 0x22, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x49,\n\t0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x49, 0x74, 0x65, 0x6d, 0x73, 0x52, 0x65,\n\t0x61, 0x64, 0x79, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x16, 0x0a, 0x06, 0x70, 0x6c,\n\t0x75, 0x67, 0x69, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x70, 0x6c, 0x75, 0x67,\n\t0x69, 0x6e, 0x12, 0x18, 0x0a, 0x07, 0x72, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x18, 0x02, 0x20,\n\t0x01, 0x28, 0x0c, 0x52, 0x07, 0x72, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x12, 0x47, 0x0a, 0x0f,\n\t0x61, 0x64, 0x64, 0x69, 0x74, 0x69, 0x6f, 0x6e, 0x61, 0x6c, 0x49, 0x74, 0x65, 0x6d, 0x73, 0x18,\n\t0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x1d, 0x2e, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65,\n\t0x64, 0x2e, 0x52, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x49, 0x64, 0x65, 0x6e, 0x74, 0x69,\n\t0x66, 0x69, 0x65, 0x72, 0x52, 0x0f, 0x61, 0x64, 0x64, 0x69, 0x74, 0x69, 0x6f, 0x6e, 0x61, 0x6c,\n\t0x49, 0x74, 0x65, 0x6d, 0x73, 0x22, 0x3b, 0x0a, 0x23, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65,\n\t0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x49, 0x74, 0x65, 0x6d, 0x73, 0x52,\n\t0x65, 0x61, 0x64, 0x79, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x14, 0x0a, 0x05,\n\t0x72, 0x65, 0x61, 0x64, 0x79, 0x18, 0x01, 0x20, 0x01, 0x28, 0x08, 0x52, 0x05, 0x72, 0x65, 0x61,\n\t0x64, 0x79, 0x32, 0xd0, 0x03, 0x0a, 0x11, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x49, 0x74,\n\t0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x5a, 0x0a, 0x09, 0x41, 0x70, 0x70, 0x6c,\n\t0x69, 0x65, 0x73, 0x54, 0x6f, 0x12, 0x25, 0x2e, 0x76, 0x32, 0x2e, 0x52, 0x65, 0x73, 0x74, 0x6f,\n\t0x72, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x41, 0x70, 0x70, 0x6c,\n\t0x69, 0x65, 0x73, 0x54, 0x6f, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x26, 0x2e, 0x76,\n\t0x32, 0x2e, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74,\n\t0x69, 0x6f, 0x6e, 0x41, 0x70, 0x70, 0x6c, 0x69, 0x65, 0x73, 0x54, 0x6f, 0x52, 0x65, 0x73, 0x70,\n\t0x6f, 0x6e, 0x73, 0x65, 0x12, 0x54, 0x0a, 0x07, 0x45, 0x78, 0x65, 0x63, 0x75, 0x74, 0x65, 0x12,\n\t0x23, 0x2e, 0x76, 0x32, 0x2e, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x49, 0x74, 0x65, 0x6d,\n\t0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x45, 0x78, 0x65, 0x63, 0x75, 0x74, 0x65, 0x52, 0x65, 0x71,\n\t0x75, 0x65, 0x73, 0x74, 0x1a, 0x24, 0x2e, 0x76, 0x32, 0x2e, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72,\n\t0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x45, 0x78, 0x65, 0x63, 0x75,\n\t0x74, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x57, 0x0a, 0x08, 0x50, 0x72,\n\t0x6f, 0x67, 0x72, 0x65, 0x73, 0x73, 0x12, 0x24, 0x2e, 0x76, 0x32, 0x2e, 0x52, 0x65, 0x73, 0x74,\n\t0x6f, 0x72, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x50, 0x72, 0x6f,\n\t0x67, 0x72, 0x65, 0x73, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x25, 0x2e, 0x76,\n\t0x32, 0x2e, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74,\n\t0x69, 0x6f, 0x6e, 0x50, 0x72, 0x6f, 0x67, 0x72, 0x65, 0x73, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f,\n\t0x6e, 0x73, 0x65, 0x12, 0x44, 0x0a, 0x06, 0x43, 0x61, 0x6e, 0x63, 0x65, 0x6c, 0x12, 0x22, 0x2e,\n\t0x76, 0x32, 0x2e, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63,\n\t0x74, 0x69, 0x6f, 0x6e, 0x43, 0x61, 0x6e, 0x63, 0x65, 0x6c, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73,\n\t0x74, 0x1a, 0x16, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f,\n\t0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x12, 0x6a, 0x0a, 0x17, 0x41, 0x72, 0x65,\n\t0x41, 0x64, 0x64, 0x69, 0x74, 0x69, 0x6f, 0x6e, 0x61, 0x6c, 0x49, 0x74, 0x65, 0x6d, 0x73, 0x52,\n\t0x65, 0x61, 0x64, 0x79, 0x12, 0x26, 0x2e, 0x76, 0x32, 0x2e, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72,\n\t0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x49, 0x74, 0x65, 0x6d, 0x73,\n\t0x52, 0x65, 0x61, 0x64, 0x79, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x27, 0x2e, 0x76,\n\t0x32, 0x2e, 0x52, 0x65, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x49, 0x74, 0x65, 0x6d, 0x41, 0x63, 0x74,\n\t0x69, 0x6f, 0x6e, 0x49, 0x74, 0x65, 0x6d, 0x73, 0x52, 0x65, 0x61, 0x64, 0x79, 0x52, 0x65, 0x73,\n\t0x70, 0x6f, 0x6e, 0x73, 0x65, 0x42, 0x4a, 0x5a, 0x48, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e,\n\t0x63, 0x6f, 0x6d, 0x2f, 0x76, 0x6d, 0x77, 0x61, 0x72, 0x65, 0x2d, 0x74, 0x61, 0x6e, 0x7a, 0x75,\n\t0x2f, 0x76, 0x65, 0x6c, 0x65, 0x72, 0x6f, 0x2f, 0x70, 0x6b, 0x67, 0x2f, 0x70, 0x6c, 0x75, 0x67,\n\t0x69, 0x6e, 0x2f, 0x67, 0x65, 0x6e, 0x65, 0x72, 0x61, 0x74, 0x65, 0x64, 0x2f, 0x72, 0x65, 0x73,\n\t0x74, 0x6f, 0x72, 0x65, 0x69, 0x74, 0x65, 0x6d, 0x61, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x2f, 0x76,\n\t0x32, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,\n}\n\nvar (\n\tfile_restoreitemaction_v2_RestoreItemAction_proto_rawDescOnce sync.Once\n\tfile_restoreitemaction_v2_RestoreItemAction_proto_rawDescData = file_restoreitemaction_v2_RestoreItemAction_proto_rawDesc\n)\n\nfunc file_restoreitemaction_v2_RestoreItemAction_proto_rawDescGZIP() []byte {\n\tfile_restoreitemaction_v2_RestoreItemAction_proto_rawDescOnce.Do(func() {\n\t\tfile_restoreitemaction_v2_RestoreItemAction_proto_rawDescData = protoimpl.X.CompressGZIP(file_restoreitemaction_v2_RestoreItemAction_proto_rawDescData)\n\t})\n\treturn file_restoreitemaction_v2_RestoreItemAction_proto_rawDescData\n}\n\nvar file_restoreitemaction_v2_RestoreItemAction_proto_msgTypes = make([]protoimpl.MessageInfo, 9)\nvar file_restoreitemaction_v2_RestoreItemAction_proto_goTypes = []interface{}{\n\t(*RestoreItemActionExecuteRequest)(nil), // 0: v2.RestoreItemActionExecuteRequest\n\t(*RestoreItemActionExecuteResponse)(nil), // 1: v2.RestoreItemActionExecuteResponse\n\t(*RestoreItemActionAppliesToRequest)(nil), // 2: v2.RestoreItemActionAppliesToRequest\n\t(*RestoreItemActionAppliesToResponse)(nil), // 3: v2.RestoreItemActionAppliesToResponse\n\t(*RestoreItemActionProgressRequest)(nil), // 4: v2.RestoreItemActionProgressRequest\n\t(*RestoreItemActionProgressResponse)(nil), // 5: v2.RestoreItemActionProgressResponse\n\t(*RestoreItemActionCancelRequest)(nil), // 6: v2.RestoreItemActionCancelRequest\n\t(*RestoreItemActionItemsReadyRequest)(nil), // 7: v2.RestoreItemActionItemsReadyRequest\n\t(*RestoreItemActionItemsReadyResponse)(nil), // 8: v2.RestoreItemActionItemsReadyResponse\n\t(*generated.ResourceIdentifier)(nil), // 9: generated.ResourceIdentifier\n\t(*durationpb.Duration)(nil), // 10: google.protobuf.Duration\n\t(*generated.ResourceSelector)(nil), // 11: generated.ResourceSelector\n\t(*generated.OperationProgress)(nil), // 12: generated.OperationProgress\n\t(*emptypb.Empty)(nil), // 13: google.protobuf.Empty\n}\nvar file_restoreitemaction_v2_RestoreItemAction_proto_depIdxs = []int32{\n\t9, // 0: v2.RestoreItemActionExecuteResponse.additionalItems:type_name -> generated.ResourceIdentifier\n\t10, // 1: v2.RestoreItemActionExecuteResponse.additionalItemsReadyTimeout:type_name -> google.protobuf.Duration\n\t11, // 2: v2.RestoreItemActionAppliesToResponse.ResourceSelector:type_name -> generated.ResourceSelector\n\t12, // 3: v2.RestoreItemActionProgressResponse.progress:type_name -> generated.OperationProgress\n\t9, // 4: v2.RestoreItemActionItemsReadyRequest.additionalItems:type_name -> generated.ResourceIdentifier\n\t2, // 5: v2.RestoreItemAction.AppliesTo:input_type -> v2.RestoreItemActionAppliesToRequest\n\t0, // 6: v2.RestoreItemAction.Execute:input_type -> v2.RestoreItemActionExecuteRequest\n\t4, // 7: v2.RestoreItemAction.Progress:input_type -> v2.RestoreItemActionProgressRequest\n\t6, // 8: v2.RestoreItemAction.Cancel:input_type -> v2.RestoreItemActionCancelRequest\n\t7, // 9: v2.RestoreItemAction.AreAdditionalItemsReady:input_type -> v2.RestoreItemActionItemsReadyRequest\n\t3, // 10: v2.RestoreItemAction.AppliesTo:output_type -> v2.RestoreItemActionAppliesToResponse\n\t1, // 11: v2.RestoreItemAction.Execute:output_type -> v2.RestoreItemActionExecuteResponse\n\t5, // 12: v2.RestoreItemAction.Progress:output_type -> v2.RestoreItemActionProgressResponse\n\t13, // 13: v2.RestoreItemAction.Cancel:output_type -> google.protobuf.Empty\n\t8, // 14: v2.RestoreItemAction.AreAdditionalItemsReady:output_type -> v2.RestoreItemActionItemsReadyResponse\n\t10, // [10:15] is the sub-list for method output_type\n\t5, // [5:10] is the sub-list for method input_type\n\t5, // [5:5] is the sub-list for extension type_name\n\t5, // [5:5] is the sub-list for extension extendee\n\t0, // [0:5] is the sub-list for field type_name\n}\n\nfunc init() { file_restoreitemaction_v2_RestoreItemAction_proto_init() }\nfunc file_restoreitemaction_v2_RestoreItemAction_proto_init() {\n\tif File_restoreitemaction_v2_RestoreItemAction_proto != nil {\n\t\treturn\n\t}\n\tif !protoimpl.UnsafeEnabled {\n\t\tfile_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*RestoreItemActionExecuteRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*RestoreItemActionExecuteResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*RestoreItemActionAppliesToRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[3].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*RestoreItemActionAppliesToResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[4].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*RestoreItemActionProgressRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[5].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*RestoreItemActionProgressResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[6].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*RestoreItemActionCancelRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[7].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*RestoreItemActionItemsReadyRequest); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\tfile_restoreitemaction_v2_RestoreItemAction_proto_msgTypes[8].Exporter = func(v interface{}, i int) interface{} {\n\t\t\tswitch v := v.(*RestoreItemActionItemsReadyResponse); i {\n\t\t\tcase 0:\n\t\t\t\treturn &v.state\n\t\t\tcase 1:\n\t\t\t\treturn &v.sizeCache\n\t\t\tcase 2:\n\t\t\t\treturn &v.unknownFields\n\t\t\tdefault:\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t}\n\ttype x struct{}\n\tout := protoimpl.TypeBuilder{\n\t\tFile: protoimpl.DescBuilder{\n\t\t\tGoPackagePath: reflect.TypeOf(x{}).PkgPath(),\n\t\t\tRawDescriptor: file_restoreitemaction_v2_RestoreItemAction_proto_rawDesc,\n\t\t\tNumEnums: 0,\n\t\t\tNumMessages: 9,\n\t\t\tNumExtensions: 0,\n\t\t\tNumServices: 1,\n\t\t},\n\t\tGoTypes: file_restoreitemaction_v2_RestoreItemAction_proto_goTypes,\n\t\tDependencyIndexes: file_restoreitemaction_v2_RestoreItemAction_proto_depIdxs,\n\t\tMessageInfos: file_restoreitemaction_v2_RestoreItemAction_proto_msgTypes,\n\t}.Build()\n\tFile_restoreitemaction_v2_RestoreItemAction_proto = out.File\n\tfile_restoreitemaction_v2_RestoreItemAction_proto_rawDesc = nil\n\tfile_restoreitemaction_v2_RestoreItemAction_proto_goTypes = nil\n\tfile_restoreitemaction_v2_RestoreItemAction_proto_depIdxs = nil\n}\n" }, { "path": "pkg/plugin/generated/restoreitemaction/v2/RestoreItemAction_grpc.pb.go", "content": "// Code generated by protoc-gen-go-grpc. DO NOT EDIT.\n// versions:\n// - protoc-gen-go-grpc v1.3.0\n// - protoc v4.25.2\n// source: restoreitemaction/v2/RestoreItemAction.proto\n\npackage v2\n\nimport (\n\tcontext \"context\"\n\tgrpc \"google.golang.org/grpc\"\n\tcodes \"google.golang.org/grpc/codes\"\n\tstatus \"google.golang.org/grpc/status\"\n\temptypb \"google.golang.org/protobuf/types/known/emptypb\"\n)\n\n// This is a compile-time assertion to ensure that this generated file\n// is compatible with the grpc package it is being compiled against.\n// Requires gRPC-Go v1.32.0 or later.\nconst _ = grpc.SupportPackageIsVersion7\n\nconst (\n\tRestoreItemAction_AppliesTo_FullMethodName = \"/v2.RestoreItemAction/AppliesTo\"\n\tRestoreItemAction_Execute_FullMethodName = \"/v2.RestoreItemAction/Execute\"\n\tRestoreItemAction_Progress_FullMethodName = \"/v2.RestoreItemAction/Progress\"\n\tRestoreItemAction_Cancel_FullMethodName = \"/v2.RestoreItemAction/Cancel\"\n\tRestoreItemAction_AreAdditionalItemsReady_FullMethodName = \"/v2.RestoreItemAction/AreAdditionalItemsReady\"\n)\n\n// RestoreItemActionClient is the client API for RestoreItemAction service.\n//\n// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.\ntype RestoreItemActionClient interface {\n\tAppliesTo(ctx context.Context, in *RestoreItemActionAppliesToRequest, opts ...grpc.CallOption) (*RestoreItemActionAppliesToResponse, error)\n\tExecute(ctx context.Context, in *RestoreItemActionExecuteRequest, opts ...grpc.CallOption) (*RestoreItemActionExecuteResponse, error)\n\tProgress(ctx context.Context, in *RestoreItemActionProgressRequest, opts ...grpc.CallOption) (*RestoreItemActionProgressResponse, error)\n\tCancel(ctx context.Context, in *RestoreItemActionCancelRequest, opts ...grpc.CallOption) (*emptypb.Empty, error)\n\tAreAdditionalItemsReady(ctx context.Context, in *RestoreItemActionItemsReadyRequest, opts ...grpc.CallOption) (*RestoreItemActionItemsReadyResponse, error)\n}\n\ntype restoreItemActionClient struct {\n\tcc grpc.ClientConnInterface\n}\n\nfunc NewRestoreItemActionClient(cc grpc.ClientConnInterface) RestoreItemActionClient {\n\treturn &restoreItemActionClient{cc}\n}\n\nfunc (c *restoreItemActionClient) AppliesTo(ctx context.Context, in *RestoreItemActionAppliesToRequest, opts ...grpc.CallOption) (*RestoreItemActionAppliesToResponse, error) {\n\tout := new(RestoreItemActionAppliesToResponse)\n\terr := c.cc.Invoke(ctx, RestoreItemAction_AppliesTo_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *restoreItemActionClient) Execute(ctx context.Context, in *RestoreItemActionExecuteRequest, opts ...grpc.CallOption) (*RestoreItemActionExecuteResponse, error) {\n\tout := new(RestoreItemActionExecuteResponse)\n\terr := c.cc.Invoke(ctx, RestoreItemAction_Execute_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *restoreItemActionClient) Progress(ctx context.Context, in *RestoreItemActionProgressRequest, opts ...grpc.CallOption) (*RestoreItemActionProgressResponse, error) {\n\tout := new(RestoreItemActionProgressResponse)\n\terr := c.cc.Invoke(ctx, RestoreItemAction_Progress_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *restoreItemActionClient) Cancel(ctx context.Context, in *RestoreItemActionCancelRequest, opts ...grpc.CallOption) (*emptypb.Empty, error) {\n\tout := new(emptypb.Empty)\n\terr := c.cc.Invoke(ctx, RestoreItemAction_Cancel_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\nfunc (c *restoreItemActionClient) AreAdditionalItemsReady(ctx context.Context, in *RestoreItemActionItemsReadyRequest, opts ...grpc.CallOption) (*RestoreItemActionItemsReadyResponse, error) {\n\tout := new(RestoreItemActionItemsReadyResponse)\n\terr := c.cc.Invoke(ctx, RestoreItemAction_AreAdditionalItemsReady_FullMethodName, in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\n// RestoreItemActionServer is the server API for RestoreItemAction service.\n// All implementations should embed UnimplementedRestoreItemActionServer\n// for forward compatibility\ntype RestoreItemActionServer interface {\n\tAppliesTo(context.Context, *RestoreItemActionAppliesToRequest) (*RestoreItemActionAppliesToResponse, error)\n\tExecute(context.Context, *RestoreItemActionExecuteRequest) (*RestoreItemActionExecuteResponse, error)\n\tProgress(context.Context, *RestoreItemActionProgressRequest) (*RestoreItemActionProgressResponse, error)\n\tCancel(context.Context, *RestoreItemActionCancelRequest) (*emptypb.Empty, error)\n\tAreAdditionalItemsReady(context.Context, *RestoreItemActionItemsReadyRequest) (*RestoreItemActionItemsReadyResponse, error)\n}\n\n// UnimplementedRestoreItemActionServer should be embedded to have forward compatible implementations.\ntype UnimplementedRestoreItemActionServer struct {\n}\n\nfunc (UnimplementedRestoreItemActionServer) AppliesTo(context.Context, *RestoreItemActionAppliesToRequest) (*RestoreItemActionAppliesToResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method AppliesTo not implemented\")\n}\nfunc (UnimplementedRestoreItemActionServer) Execute(context.Context, *RestoreItemActionExecuteRequest) (*RestoreItemActionExecuteResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method Execute not implemented\")\n}\nfunc (UnimplementedRestoreItemActionServer) Progress(context.Context, *RestoreItemActionProgressRequest) (*RestoreItemActionProgressResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method Progress not implemented\")\n}\nfunc (UnimplementedRestoreItemActionServer) Cancel(context.Context, *RestoreItemActionCancelRequest) (*emptypb.Empty, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method Cancel not implemented\")\n}\nfunc (UnimplementedRestoreItemActionServer) AreAdditionalItemsReady(context.Context, *RestoreItemActionItemsReadyRequest) (*RestoreItemActionItemsReadyResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method AreAdditionalItemsReady not implemented\")\n}\n\n// UnsafeRestoreItemActionServer may be embedded to opt out of forward compatibility for this service.\n// Use of this interface is not recommended, as added methods to RestoreItemActionServer will\n// result in compilation errors.\ntype UnsafeRestoreItemActionServer interface {\n\tmustEmbedUnimplementedRestoreItemActionServer()\n}\n\nfunc RegisterRestoreItemActionServer(s grpc.ServiceRegistrar, srv RestoreItemActionServer) {\n\ts.RegisterService(&RestoreItemAction_ServiceDesc, srv)\n}\n\nfunc _RestoreItemAction_AppliesTo_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(RestoreItemActionAppliesToRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(RestoreItemActionServer).AppliesTo(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: RestoreItemAction_AppliesTo_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(RestoreItemActionServer).AppliesTo(ctx, req.(*RestoreItemActionAppliesToRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _RestoreItemAction_Execute_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(RestoreItemActionExecuteRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(RestoreItemActionServer).Execute(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: RestoreItemAction_Execute_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(RestoreItemActionServer).Execute(ctx, req.(*RestoreItemActionExecuteRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _RestoreItemAction_Progress_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(RestoreItemActionProgressRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(RestoreItemActionServer).Progress(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: RestoreItemAction_Progress_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(RestoreItemActionServer).Progress(ctx, req.(*RestoreItemActionProgressRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _RestoreItemAction_Cancel_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(RestoreItemActionCancelRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(RestoreItemActionServer).Cancel(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: RestoreItemAction_Cancel_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(RestoreItemActionServer).Cancel(ctx, req.(*RestoreItemActionCancelRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nfunc _RestoreItemAction_AreAdditionalItemsReady_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(RestoreItemActionItemsReadyRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(RestoreItemActionServer).AreAdditionalItemsReady(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: RestoreItemAction_AreAdditionalItemsReady_FullMethodName,\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(RestoreItemActionServer).AreAdditionalItemsReady(ctx, req.(*RestoreItemActionItemsReadyRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\n// RestoreItemAction_ServiceDesc is the grpc.ServiceDesc for RestoreItemAction service.\n// It's only intended for direct use with grpc.RegisterService,\n// and not to be introspected or modified (even as a copy)\nvar RestoreItemAction_ServiceDesc = grpc.ServiceDesc{\n\tServiceName: \"v2.RestoreItemAction\",\n\tHandlerType: (*RestoreItemActionServer)(nil),\n\tMethods: []grpc.MethodDesc{\n\t\t{\n\t\t\tMethodName: \"AppliesTo\",\n\t\t\tHandler: _RestoreItemAction_AppliesTo_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"Execute\",\n\t\t\tHandler: _RestoreItemAction_Execute_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"Progress\",\n\t\t\tHandler: _RestoreItemAction_Progress_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"Cancel\",\n\t\t\tHandler: _RestoreItemAction_Cancel_Handler,\n\t\t},\n\t\t{\n\t\t\tMethodName: \"AreAdditionalItemsReady\",\n\t\t\tHandler: _RestoreItemAction_AreAdditionalItemsReady_Handler,\n\t\t},\n\t},\n\tStreams: []grpc.StreamDesc{},\n\tMetadata: \"restoreitemaction/v2/RestoreItemAction.proto\",\n}\n" }, { "path": "pkg/plugin/mocks/manager.go", "content": "// Code generated by mockery v2.43.2. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tmock \"github.com/stretchr/testify/mock\"\n\titemblockactionv1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/itemblockaction/v1\"\n\n\trestoreitemactionv1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/restoreitemaction/v1\"\n\n\trestoreitemactionv2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/restoreitemaction/v2\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/backupitemaction/v1\"\n\n\tv2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/backupitemaction/v2\"\n\n\tvelero \"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\n\tvolumesnapshotterv1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/volumesnapshotter/v1\"\n)\n\n// Manager is an autogenerated mock type for the Manager type\ntype Manager struct {\n\tmock.Mock\n}\n\n// CleanupClients provides a mock function with given fields:\nfunc (_m *Manager) CleanupClients() {\n\t_m.Called()\n}\n\n// GetBackupItemAction provides a mock function with given fields: name\nfunc (_m *Manager) GetBackupItemAction(name string) (v1.BackupItemAction, error) {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetBackupItemAction\")\n\t}\n\n\tvar r0 v1.BackupItemAction\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) (v1.BackupItemAction, error)); ok {\n\t\treturn rf(name)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) v1.BackupItemAction); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(v1.BackupItemAction)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(name)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetBackupItemActionV2 provides a mock function with given fields: name\nfunc (_m *Manager) GetBackupItemActionV2(name string) (v2.BackupItemAction, error) {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetBackupItemActionV2\")\n\t}\n\n\tvar r0 v2.BackupItemAction\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) (v2.BackupItemAction, error)); ok {\n\t\treturn rf(name)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) v2.BackupItemAction); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(v2.BackupItemAction)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(name)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetBackupItemActions provides a mock function with given fields:\nfunc (_m *Manager) GetBackupItemActions() ([]v1.BackupItemAction, error) {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetBackupItemActions\")\n\t}\n\n\tvar r0 []v1.BackupItemAction\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func() ([]v1.BackupItemAction, error)); ok {\n\t\treturn rf()\n\t}\n\tif rf, ok := ret.Get(0).(func() []v1.BackupItemAction); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]v1.BackupItemAction)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetBackupItemActionsV2 provides a mock function with given fields:\nfunc (_m *Manager) GetBackupItemActionsV2() ([]v2.BackupItemAction, error) {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetBackupItemActionsV2\")\n\t}\n\n\tvar r0 []v2.BackupItemAction\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func() ([]v2.BackupItemAction, error)); ok {\n\t\treturn rf()\n\t}\n\tif rf, ok := ret.Get(0).(func() []v2.BackupItemAction); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]v2.BackupItemAction)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetDeleteItemAction provides a mock function with given fields: name\nfunc (_m *Manager) GetDeleteItemAction(name string) (velero.DeleteItemAction, error) {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetDeleteItemAction\")\n\t}\n\n\tvar r0 velero.DeleteItemAction\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) (velero.DeleteItemAction, error)); ok {\n\t\treturn rf(name)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) velero.DeleteItemAction); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(velero.DeleteItemAction)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(name)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetDeleteItemActions provides a mock function with given fields:\nfunc (_m *Manager) GetDeleteItemActions() ([]velero.DeleteItemAction, error) {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetDeleteItemActions\")\n\t}\n\n\tvar r0 []velero.DeleteItemAction\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func() ([]velero.DeleteItemAction, error)); ok {\n\t\treturn rf()\n\t}\n\tif rf, ok := ret.Get(0).(func() []velero.DeleteItemAction); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]velero.DeleteItemAction)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetItemBlockAction provides a mock function with given fields: name\nfunc (_m *Manager) GetItemBlockAction(name string) (itemblockactionv1.ItemBlockAction, error) {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetItemBlockAction\")\n\t}\n\n\tvar r0 itemblockactionv1.ItemBlockAction\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) (itemblockactionv1.ItemBlockAction, error)); ok {\n\t\treturn rf(name)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) itemblockactionv1.ItemBlockAction); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(itemblockactionv1.ItemBlockAction)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(name)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetItemBlockActions provides a mock function with given fields:\nfunc (_m *Manager) GetItemBlockActions() ([]itemblockactionv1.ItemBlockAction, error) {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetItemBlockActions\")\n\t}\n\n\tvar r0 []itemblockactionv1.ItemBlockAction\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func() ([]itemblockactionv1.ItemBlockAction, error)); ok {\n\t\treturn rf()\n\t}\n\tif rf, ok := ret.Get(0).(func() []itemblockactionv1.ItemBlockAction); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]itemblockactionv1.ItemBlockAction)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetObjectStore provides a mock function with given fields: name\nfunc (_m *Manager) GetObjectStore(name string) (velero.ObjectStore, error) {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetObjectStore\")\n\t}\n\n\tvar r0 velero.ObjectStore\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) (velero.ObjectStore, error)); ok {\n\t\treturn rf(name)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) velero.ObjectStore); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(velero.ObjectStore)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(name)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetRestoreItemAction provides a mock function with given fields: name\nfunc (_m *Manager) GetRestoreItemAction(name string) (restoreitemactionv1.RestoreItemAction, error) {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetRestoreItemAction\")\n\t}\n\n\tvar r0 restoreitemactionv1.RestoreItemAction\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) (restoreitemactionv1.RestoreItemAction, error)); ok {\n\t\treturn rf(name)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) restoreitemactionv1.RestoreItemAction); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(restoreitemactionv1.RestoreItemAction)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(name)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetRestoreItemActionV2 provides a mock function with given fields: name\nfunc (_m *Manager) GetRestoreItemActionV2(name string) (restoreitemactionv2.RestoreItemAction, error) {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetRestoreItemActionV2\")\n\t}\n\n\tvar r0 restoreitemactionv2.RestoreItemAction\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) (restoreitemactionv2.RestoreItemAction, error)); ok {\n\t\treturn rf(name)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) restoreitemactionv2.RestoreItemAction); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(restoreitemactionv2.RestoreItemAction)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(name)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetRestoreItemActions provides a mock function with given fields:\nfunc (_m *Manager) GetRestoreItemActions() ([]restoreitemactionv1.RestoreItemAction, error) {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetRestoreItemActions\")\n\t}\n\n\tvar r0 []restoreitemactionv1.RestoreItemAction\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func() ([]restoreitemactionv1.RestoreItemAction, error)); ok {\n\t\treturn rf()\n\t}\n\tif rf, ok := ret.Get(0).(func() []restoreitemactionv1.RestoreItemAction); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]restoreitemactionv1.RestoreItemAction)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetRestoreItemActionsV2 provides a mock function with given fields:\nfunc (_m *Manager) GetRestoreItemActionsV2() ([]restoreitemactionv2.RestoreItemAction, error) {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetRestoreItemActionsV2\")\n\t}\n\n\tvar r0 []restoreitemactionv2.RestoreItemAction\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func() ([]restoreitemactionv2.RestoreItemAction, error)); ok {\n\t\treturn rf()\n\t}\n\tif rf, ok := ret.Get(0).(func() []restoreitemactionv2.RestoreItemAction); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]restoreitemactionv2.RestoreItemAction)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetVolumeSnapshotter provides a mock function with given fields: name\nfunc (_m *Manager) GetVolumeSnapshotter(name string) (volumesnapshotterv1.VolumeSnapshotter, error) {\n\tret := _m.Called(name)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetVolumeSnapshotter\")\n\t}\n\n\tvar r0 volumesnapshotterv1.VolumeSnapshotter\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) (volumesnapshotterv1.VolumeSnapshotter, error)); ok {\n\t\treturn rf(name)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) volumesnapshotterv1.VolumeSnapshotter); ok {\n\t\tr0 = rf(name)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(volumesnapshotterv1.VolumeSnapshotter)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(name)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// NewManager creates a new instance of Manager. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\n// The first argument is typically a *testing.T value.\nfunc NewManager(t interface {\n\tmock.TestingT\n\tCleanup(func())\n}) *Manager {\n\tmock := &Manager{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/plugin/mocks/process_factory.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n// Code generated by mockery v1.0.0. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tlogrus \"github.com/sirupsen/logrus\"\n\tmock \"github.com/stretchr/testify/mock\"\n\n\tprocess \"github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt/process\"\n)\n\n// ProcessFactory is an autogenerated mock type for the ProcessFactory type\ntype ProcessFactory struct {\n\tmock.Mock\n}\n\n// newProcess provides a mock function with given fields: command, logger, logLevel\nfunc (_m *ProcessFactory) newProcess(command string, logger logrus.FieldLogger, logLevel logrus.Level) (process.Process, error) {\n\tret := _m.Called(command, logger, logLevel)\n\n\tvar r0 process.Process\n\tif rf, ok := ret.Get(0).(func(string, logrus.FieldLogger, logrus.Level) process.Process); ok {\n\t\tr0 = rf(command, logger, logLevel)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(process.Process)\n\t\t}\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(string, logrus.FieldLogger, logrus.Level) error); ok {\n\t\tr1 = rf(command, logger, logLevel)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n" }, { "path": "pkg/plugin/proto/BackupItemAction.proto", "content": "syntax = \"proto3\";\npackage generated;\noption go_package = \"github.com/vmware-tanzu/velero/pkg/plugin/generated\";\n\nimport \"Shared.proto\";\n\nmessage ExecuteRequest {\n string plugin = 1;\n bytes item = 2;\n bytes backup = 3;\n}\n\nmessage ExecuteResponse {\n bytes item = 1;\n repeated ResourceIdentifier additionalItems = 2;\n}\n\nservice BackupItemAction {\n rpc AppliesTo(BackupItemActionAppliesToRequest) returns (BackupItemActionAppliesToResponse);\n rpc Execute(ExecuteRequest) returns (ExecuteResponse);\n}\n\nmessage BackupItemActionAppliesToRequest {\n string plugin = 1;\n}\n\nmessage BackupItemActionAppliesToResponse {\n ResourceSelector ResourceSelector = 1;\n}" }, { "path": "pkg/plugin/proto/DeleteItemAction.proto", "content": "syntax = \"proto3\";\npackage generated;\noption go_package = \"github.com/vmware-tanzu/velero/pkg/plugin/generated\";\n\nimport \"Shared.proto\";\n\nmessage DeleteItemActionExecuteRequest {\n string plugin = 1;\n bytes item = 2;\n bytes backup = 3;\n}\n\nservice DeleteItemAction {\n rpc AppliesTo(DeleteItemActionAppliesToRequest) returns (DeleteItemActionAppliesToResponse);\n rpc Execute(DeleteItemActionExecuteRequest) returns (Empty);\n}\n\nmessage DeleteItemActionAppliesToRequest {\n string plugin = 1;\n}\n\nmessage DeleteItemActionAppliesToResponse {\n ResourceSelector ResourceSelector = 1;\n}\n" }, { "path": "pkg/plugin/proto/ObjectStore.proto", "content": "syntax = \"proto3\";\npackage generated;\noption go_package = \"github.com/vmware-tanzu/velero/pkg/plugin/generated\";\n\nimport \"Shared.proto\";\n\nmessage PutObjectRequest {\n string plugin = 1;\n string bucket = 2;\n string key = 3;\n bytes body = 4;\n}\n\nmessage ObjectExistsRequest {\n string plugin = 1;\n string bucket = 2;\n string key = 3;\n}\n\nmessage ObjectExistsResponse {\n bool exists = 1;\n}\n\nmessage GetObjectRequest {\n string plugin = 1;\n string bucket = 2;\n string key = 3;\n}\n\nmessage Bytes {\n bytes data = 1;\n}\n\nmessage ListCommonPrefixesRequest {\n string plugin = 1;\n string bucket = 2;\n string delimiter = 3;\n string prefix = 4;\n}\n\nmessage ListCommonPrefixesResponse {\n repeated string prefixes = 1;\n}\n\nmessage ListObjectsRequest {\n string plugin = 1;\n string bucket = 2;\n string prefix = 3;\n}\n\nmessage ListObjectsResponse {\n repeated string keys = 1;\n}\n\nmessage DeleteObjectRequest {\n string plugin = 1;\n string bucket = 2; \n string key = 3;\n}\n\n\nmessage CreateSignedURLRequest {\n string plugin = 1;\n string bucket = 2;\n string key = 3;\n int64 ttl = 4;\n}\n\nmessage CreateSignedURLResponse {\n string url = 1;\n}\n\nmessage ObjectStoreInitRequest {\n string plugin = 1;\n map config = 2;\n}\n\nservice ObjectStore {\n rpc Init(ObjectStoreInitRequest) returns (Empty);\n rpc PutObject(stream PutObjectRequest) returns (Empty);\n rpc ObjectExists(ObjectExistsRequest) returns (ObjectExistsResponse);\n rpc GetObject(GetObjectRequest) returns (stream Bytes);\n rpc ListCommonPrefixes(ListCommonPrefixesRequest) returns (ListCommonPrefixesResponse);\n rpc ListObjects(ListObjectsRequest) returns (ListObjectsResponse);\n rpc DeleteObject(DeleteObjectRequest) returns (Empty);\n rpc CreateSignedURL(CreateSignedURLRequest) returns (CreateSignedURLResponse);\n}\n" }, { "path": "pkg/plugin/proto/PluginLister.proto", "content": "syntax = \"proto3\";\npackage generated;\noption go_package = \"github.com/vmware-tanzu/velero/pkg/plugin/generated\";\n\nimport \"Shared.proto\";\n\nmessage PluginIdentifier {\n string command = 1;\n string kind = 2;\n string name = 3;\n}\n\nmessage ListPluginsResponse {\n repeated PluginIdentifier plugins = 1;\n}\n\nservice PluginLister {\n rpc ListPlugins(Empty) returns (ListPluginsResponse);\n}\n" }, { "path": "pkg/plugin/proto/RestoreItemAction.proto", "content": "syntax = \"proto3\";\npackage generated;\noption go_package = \"github.com/vmware-tanzu/velero/pkg/plugin/generated\";\n\nimport \"Shared.proto\";\n\nmessage RestoreItemActionExecuteRequest {\n string plugin = 1;\n bytes item = 2;\n bytes restore = 3;\n bytes itemFromBackup = 4;\n}\n\nmessage RestoreItemActionExecuteResponse {\n bytes item = 1;\n repeated ResourceIdentifier additionalItems = 2;\n bool skipRestore = 3;\n}\n\nservice RestoreItemAction {\n rpc AppliesTo(RestoreItemActionAppliesToRequest) returns (RestoreItemActionAppliesToResponse);\n rpc Execute(RestoreItemActionExecuteRequest) returns (RestoreItemActionExecuteResponse);\n}\n\nmessage RestoreItemActionAppliesToRequest {\n string plugin = 1;\n}\n\nmessage RestoreItemActionAppliesToResponse {\n ResourceSelector ResourceSelector = 1;\n}\n" }, { "path": "pkg/plugin/proto/Shared.proto", "content": "syntax = \"proto3\";\npackage generated;\noption go_package = \"github.com/vmware-tanzu/velero/pkg/plugin/generated\";\n\nimport \"google/protobuf/timestamp.proto\";\n\nmessage Empty {}\n\nmessage Stack {\n repeated StackFrame frames = 1;\n}\n\nmessage StackFrame {\n string file = 1;\n int32 line = 2;\n string function = 3;\n}\n\nmessage ResourceIdentifier {\n string group = 1;\n string resource = 2;\n string namespace = 3;\n string name = 4;\n}\n\nmessage ResourceSelector {\n repeated string includedNamespaces = 1;\n repeated string excludedNamespaces = 2;\n repeated string includedResources = 3;\n repeated string excludedResources = 4;\n string selector = 5;\n}\n\nmessage OperationProgress {\n bool completed = 1;\n string err = 2;\n int64 nCompleted = 3;\n int64 nTotal = 4;\n string operationUnits = 5;\n string description = 6;\n google.protobuf.Timestamp started = 7;\n google.protobuf.Timestamp updated = 8;\n}\n" }, { "path": "pkg/plugin/proto/VolumeSnapshotter.proto", "content": "syntax = \"proto3\";\npackage generated;\noption go_package = \"github.com/vmware-tanzu/velero/pkg/plugin/generated\";\n\nimport \"Shared.proto\";\n\nmessage CreateVolumeRequest {\n string plugin = 1;\n string snapshotID = 2;\n string volumeType = 3;\n string volumeAZ = 4;\n int64 iops = 5;\n}\n\nmessage CreateVolumeResponse {\n string volumeID = 1;\n}\n\nmessage GetVolumeInfoRequest {\n string plugin = 1;\n string volumeID = 2;\n string volumeAZ = 3;\n}\n\nmessage GetVolumeInfoResponse {\n string volumeType = 1;\n int64 iops = 2;\n}\n\nmessage CreateSnapshotRequest {\n string plugin = 1;\n string volumeID = 2;\n string volumeAZ = 3;\n map tags = 4;\n}\n\nmessage CreateSnapshotResponse {\n string snapshotID = 1;\n}\n\nmessage DeleteSnapshotRequest {\n string plugin = 1;\n string snapshotID = 2;\n}\n\nmessage GetVolumeIDRequest {\n string plugin = 1;\n bytes persistentVolume = 2;\n}\n\nmessage GetVolumeIDResponse {\n string volumeID = 1;\n}\n\nmessage SetVolumeIDRequest {\n string plugin = 1;\n bytes persistentVolume = 2;\n string volumeID = 3;\n}\n\nmessage SetVolumeIDResponse {\n bytes persistentVolume = 1;\n}\n\nmessage VolumeSnapshotterInitRequest {\n string plugin = 1;\n map config = 2;\n}\n\nservice VolumeSnapshotter {\n rpc Init(VolumeSnapshotterInitRequest) returns (Empty);\n rpc CreateVolumeFromSnapshot(CreateVolumeRequest) returns (CreateVolumeResponse);\n rpc GetVolumeInfo(GetVolumeInfoRequest) returns (GetVolumeInfoResponse);\n rpc CreateSnapshot(CreateSnapshotRequest) returns (CreateSnapshotResponse);\n rpc DeleteSnapshot(DeleteSnapshotRequest) returns (Empty);\n rpc GetVolumeID(GetVolumeIDRequest) returns (GetVolumeIDResponse);\n rpc SetVolumeID(SetVolumeIDRequest) returns (SetVolumeIDResponse);\n}\n" }, { "path": "pkg/plugin/proto/backupitemaction/v2/BackupItemAction.proto", "content": "syntax = \"proto3\";\npackage v2;\noption go_package = \"github.com/vmware-tanzu/velero/pkg/plugin/generated/backupitemaction/v2\";\n\nimport \"Shared.proto\";\nimport \"google/protobuf/empty.proto\";\n\n\nmessage ExecuteRequest {\n string plugin = 1;\n bytes item = 2;\n bytes backup = 3;\n}\n\nmessage ExecuteResponse {\n bytes item = 1;\n repeated generated.ResourceIdentifier additionalItems = 2;\n string operationID = 3;\n repeated generated.ResourceIdentifier postOperationItems = 4;\n}\n\nservice BackupItemAction {\n rpc AppliesTo(BackupItemActionAppliesToRequest) returns (BackupItemActionAppliesToResponse);\n rpc Execute(ExecuteRequest) returns (ExecuteResponse);\n rpc Progress(BackupItemActionProgressRequest) returns (BackupItemActionProgressResponse);\n rpc Cancel(BackupItemActionCancelRequest) returns (google.protobuf.Empty);\n}\n\nmessage BackupItemActionAppliesToRequest {\n string plugin = 1;\n}\n\nmessage BackupItemActionAppliesToResponse {\n generated.ResourceSelector ResourceSelector = 1;\n}\n\nmessage BackupItemActionProgressRequest {\n string plugin = 1;\n string operationID = 2;\n bytes backup = 3;\n}\n\nmessage BackupItemActionProgressResponse {\n generated.OperationProgress progress = 1;\n}\n\nmessage BackupItemActionCancelRequest {\n string plugin = 1;\n string operationID = 2;\n bytes backup = 3;\n}\n" }, { "path": "pkg/plugin/proto/itemblockaction/v1/ItemBlockAction.proto", "content": "syntax = \"proto3\";\npackage v1;\noption go_package = \"github.com/vmware-tanzu/velero/pkg/plugin/generated/itemblockaction/v1\";\n\nimport \"Shared.proto\";\n\n\nservice ItemBlockAction {\n rpc AppliesTo(ItemBlockActionAppliesToRequest) returns (ItemBlockActionAppliesToResponse);\n rpc GetRelatedItems(ItemBlockActionGetRelatedItemsRequest) returns (ItemBlockActionGetRelatedItemsResponse);\n}\n\nmessage ItemBlockActionAppliesToRequest {\n string plugin = 1;\n}\n\nmessage ItemBlockActionAppliesToResponse {\n generated.ResourceSelector ResourceSelector = 1;\n}\n\nmessage ItemBlockActionGetRelatedItemsRequest {\n string plugin = 1;\n bytes item = 2;\n bytes backup = 3;\n}\n\nmessage ItemBlockActionGetRelatedItemsResponse {\n repeated generated.ResourceIdentifier relatedItems = 1;\n}\n" }, { "path": "pkg/plugin/proto/restoreitemaction/v2/RestoreItemAction.proto", "content": "syntax = \"proto3\";\npackage v2;\noption go_package = \"github.com/vmware-tanzu/velero/pkg/plugin/generated/restoreitemaction/v2\";\n\nimport \"Shared.proto\";\nimport \"google/protobuf/empty.proto\";\nimport \"google/protobuf/duration.proto\";\n\nmessage RestoreItemActionExecuteRequest {\n string plugin = 1;\n bytes item = 2;\n bytes restore = 3;\n bytes itemFromBackup = 4;\n}\n\nmessage RestoreItemActionExecuteResponse {\n bytes item = 1;\n repeated generated.ResourceIdentifier additionalItems = 2;\n bool skipRestore = 3;\n string operationID = 4;\n bool waitForAdditionalItems = 5;\n google.protobuf.Duration additionalItemsReadyTimeout = 6;\n}\n\nservice RestoreItemAction {\n rpc AppliesTo(RestoreItemActionAppliesToRequest) returns (RestoreItemActionAppliesToResponse);\n rpc Execute(RestoreItemActionExecuteRequest) returns (RestoreItemActionExecuteResponse);\n rpc Progress(RestoreItemActionProgressRequest) returns (RestoreItemActionProgressResponse);\n rpc Cancel(RestoreItemActionCancelRequest) returns (google.protobuf.Empty);\n rpc AreAdditionalItemsReady(RestoreItemActionItemsReadyRequest) returns (RestoreItemActionItemsReadyResponse);\n}\n\nmessage RestoreItemActionAppliesToRequest {\n string plugin = 1;\n}\n\nmessage RestoreItemActionAppliesToResponse {\n generated.ResourceSelector ResourceSelector = 1;\n}\n\nmessage RestoreItemActionProgressRequest {\n string plugin = 1;\n string operationID = 2;\n bytes restore = 3;\n}\nmessage RestoreItemActionProgressResponse {\n generated.OperationProgress progress = 1;\n}\nmessage RestoreItemActionCancelRequest {\n string plugin = 1;\n string operationID = 2;\n bytes restore = 3;\n}\nmessage RestoreItemActionItemsReadyRequest {\n string plugin = 1;\n bytes restore = 2;\n repeated generated.ResourceIdentifier additionalItems = 3;\n}\nmessage RestoreItemActionItemsReadyResponse {\n bool ready = 1;\n}\n" }, { "path": "pkg/plugin/utils/volumehelper/volume_policy_helper.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage volumehelper\n\nimport (\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\tcrclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/resourcepolicies\"\n\t\"github.com/vmware-tanzu/velero/internal/volumehelper\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n)\n\n// ShouldPerformSnapshotWithBackup is used for third-party plugins.\n// It supports to check whether the PVC or PodVolume should be backed\n// up on demand. On the other hand, the volumeHelperImpl assume there\n// is a VolumeHelper instance initialized before calling the\n// ShouldPerformXXX functions.\n//\n// Deprecated: Use ShouldPerformSnapshotWithVolumeHelper instead for better performance.\n// ShouldPerformSnapshotWithVolumeHelper allows passing a pre-created VolumeHelper with\n// an internal PVC-to-Pod cache, which avoids O(N*M) complexity when there are many PVCs and pods.\n// See issue #9179 for details.\nfunc ShouldPerformSnapshotWithBackup(\n\tunstructured runtime.Unstructured,\n\tgroupResource schema.GroupResource,\n\tbackup velerov1api.Backup,\n\tcrClient crclient.Client,\n\tlogger logrus.FieldLogger,\n) (bool, error) {\n\treturn ShouldPerformSnapshotWithVolumeHelper(\n\t\tunstructured,\n\t\tgroupResource,\n\t\tbackup,\n\t\tcrClient,\n\t\tlogger,\n\t\tnil, // no cached VolumeHelper, will create one\n\t)\n}\n\n// ShouldPerformSnapshotWithVolumeHelper is like ShouldPerformSnapshotWithBackup\n// but accepts an optional VolumeHelper. If vh is non-nil, it will be used directly,\n// avoiding the overhead of creating a new VolumeHelper on each call.\n// This is useful for BIA plugins that process multiple PVCs during a single backup\n// and want to reuse the same VolumeHelper (with its internal cache) across calls.\nfunc ShouldPerformSnapshotWithVolumeHelper(\n\tunstructured runtime.Unstructured,\n\tgroupResource schema.GroupResource,\n\tbackup velerov1api.Backup,\n\tcrClient crclient.Client,\n\tlogger logrus.FieldLogger,\n\tvh volumehelper.VolumeHelper,\n) (bool, error) {\n\t// If a VolumeHelper is provided, use it directly\n\tif vh != nil {\n\t\treturn vh.ShouldPerformSnapshot(unstructured, groupResource)\n\t}\n\n\t// Otherwise, create a new VolumeHelper (original behavior for third-party plugins)\n\tresourcePolicies, err := resourcepolicies.GetResourcePoliciesFromBackup(\n\t\tbackup,\n\t\tcrClient,\n\t\tlogger,\n\t)\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\t//nolint:staticcheck // Intentional use of deprecated function for backwards compatibility\n\tvolumeHelperImpl := volumehelper.NewVolumeHelperImpl(\n\t\tresourcePolicies,\n\t\tbackup.Spec.SnapshotVolumes,\n\t\tlogger,\n\t\tcrClient,\n\t\tboolptr.IsSetToTrue(backup.Spec.DefaultVolumesToFsBackup),\n\t\ttrue,\n\t)\n\n\treturn volumeHelperImpl.ShouldPerformSnapshot(unstructured, groupResource)\n}\n" }, { "path": "pkg/plugin/utils/volumehelper/volume_policy_helper_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage volumehelper\n\nimport (\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/internal/volumehelper\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestShouldPerformSnapshotWithBackup(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\tpv *corev1api.PersistentVolume\n\t\tbackup *velerov1api.Backup\n\t\twantSnapshot bool\n\t\twantError bool\n\t}{\n\t\t{\n\t\t\tname: \"Returns true when snapshotVolumes not set\",\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-pvc\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\tVolumeName: \"test-pv\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\t\t\tPhase: corev1api.ClaimBound,\n\t\t\t\t},\n\t\t\t},\n\t\t\tpv: &corev1api.PersistentVolume{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-pv\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{\n\t\t\t\t\t\t\tDriver: \"test-driver\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tClaimRef: &corev1api.ObjectReference{\n\t\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\t\tName: \"test-pvc\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tbackup: &velerov1api.Backup{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-backup\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t},\n\t\t\twantSnapshot: true,\n\t\t\twantError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Returns false when snapshotVolumes is false\",\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-pvc\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\tVolumeName: \"test-pv\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\t\t\tPhase: corev1api.ClaimBound,\n\t\t\t\t},\n\t\t\t},\n\t\t\tpv: &corev1api.PersistentVolume{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-pv\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{\n\t\t\t\t\t\t\tDriver: \"test-driver\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tClaimRef: &corev1api.ObjectReference{\n\t\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\t\tName: \"test-pvc\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tbackup: &velerov1api.Backup{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-backup\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupSpec{\n\t\t\t\t\tSnapshotVolumes: boolPtr(false),\n\t\t\t\t},\n\t\t\t},\n\t\t\twantSnapshot: false,\n\t\t\twantError: false,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\t// Create fake client with PV and PVC\n\t\t\tclient := velerotest.NewFakeControllerRuntimeClient(t, tt.pv, tt.pvc)\n\n\t\t\t// Convert PVC to unstructured\n\t\t\tpvcMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tt.pvc)\n\t\t\trequire.NoError(t, err)\n\t\t\tunstructuredPVC := &unstructured.Unstructured{Object: pvcMap}\n\n\t\t\tlogger := logrus.New()\n\n\t\t\t// Call the function under test - this is the wrapper for third-party plugins\n\t\t\tresult, err := ShouldPerformSnapshotWithBackup(\n\t\t\t\tunstructuredPVC,\n\t\t\t\tkuberesource.PersistentVolumeClaims,\n\t\t\t\t*tt.backup,\n\t\t\t\tclient,\n\t\t\t\tlogger,\n\t\t\t)\n\n\t\t\tif tt.wantError {\n\t\t\t\trequire.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\trequire.Equal(t, tt.wantSnapshot, result)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc boolPtr(b bool) *bool {\n\treturn &b\n}\n\nfunc TestShouldPerformSnapshotWithVolumeHelper(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\tpv *corev1api.PersistentVolume\n\t\tbackup *velerov1api.Backup\n\t\twantSnapshot bool\n\t\twantError bool\n\t}{\n\t\t{\n\t\t\tname: \"Returns true with nil VolumeHelper when snapshotVolumes not set\",\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-pvc\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\tVolumeName: \"test-pv\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\t\t\tPhase: corev1api.ClaimBound,\n\t\t\t\t},\n\t\t\t},\n\t\t\tpv: &corev1api.PersistentVolume{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-pv\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{\n\t\t\t\t\t\t\tDriver: \"test-driver\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tClaimRef: &corev1api.ObjectReference{\n\t\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t\t\tName: \"test-pvc\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tbackup: &velerov1api.Backup{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-backup\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t},\n\t\t\twantSnapshot: true,\n\t\t\twantError: false,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\t// Create fake client with PV\n\t\t\tclient := velerotest.NewFakeControllerRuntimeClient(t, tt.pv, tt.pvc)\n\n\t\t\t// Convert PVC to unstructured\n\t\t\tpvcMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tt.pvc)\n\t\t\trequire.NoError(t, err)\n\t\t\tunstructuredPVC := &unstructured.Unstructured{Object: pvcMap}\n\n\t\t\tlogger := logrus.New()\n\n\t\t\t// Call the function under test with nil VolumeHelper\n\t\t\t// This exercises the fallback path that creates a new VolumeHelper per call\n\t\t\tresult, err := ShouldPerformSnapshotWithVolumeHelper(\n\t\t\t\tunstructuredPVC,\n\t\t\t\tkuberesource.PersistentVolumeClaims,\n\t\t\t\t*tt.backup,\n\t\t\t\tclient,\n\t\t\t\tlogger,\n\t\t\t\tnil, // Pass nil for VolumeHelper - exercises fallback path\n\t\t\t)\n\n\t\t\tif tt.wantError {\n\t\t\t\trequire.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\trequire.Equal(t, tt.wantSnapshot, result)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// TestShouldPerformSnapshotWithNonNilVolumeHelper tests the ShouldPerformSnapshotWithVolumeHelper\n// function when a pre-created VolumeHelper is passed. This exercises the cached path used\n// by BIA plugins for better performance.\nfunc TestShouldPerformSnapshotWithNonNilVolumeHelper(t *testing.T) {\n\tpvc := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-pvc\",\n\t\t\tNamespace: \"default\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tVolumeName: \"test-pv\",\n\t\t},\n\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\tPhase: corev1api.ClaimBound,\n\t\t},\n\t}\n\n\tpv := &corev1api.PersistentVolume{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-pv\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{\n\t\t\t\t\tDriver: \"test-driver\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tClaimRef: &corev1api.ObjectReference{\n\t\t\t\tNamespace: \"default\",\n\t\t\t\tName: \"test-pvc\",\n\t\t\t},\n\t\t},\n\t}\n\n\tbackup := &velerov1api.Backup{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-backup\",\n\t\t\tNamespace: \"velero\",\n\t\t},\n\t\tSpec: velerov1api.BackupSpec{\n\t\t\tIncludedNamespaces: []string{\"default\"},\n\t\t},\n\t}\n\n\t// Create fake client with PV and PVC\n\tclient := velerotest.NewFakeControllerRuntimeClient(t, pv, pvc)\n\n\tlogger := logrus.New()\n\n\t// Create VolumeHelper using the internal function with namespace caching\n\tvh, err := volumehelper.NewVolumeHelperImplWithNamespaces(\n\t\tnil, // no resource policies for this test\n\t\tnil, // snapshotVolumes not set\n\t\tlogger,\n\t\tclient,\n\t\tfalse, // defaultVolumesToFSBackup\n\t\ttrue, // backupExcludePVC\n\t\t[]string{\"default\"},\n\t)\n\trequire.NoError(t, err)\n\trequire.NotNil(t, vh)\n\n\t// Convert PVC to unstructured\n\tpvcMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(pvc)\n\trequire.NoError(t, err)\n\tunstructuredPVC := &unstructured.Unstructured{Object: pvcMap}\n\n\t// Call with non-nil VolumeHelper - exercises the cached path\n\tresult, err := ShouldPerformSnapshotWithVolumeHelper(\n\t\tunstructuredPVC,\n\t\tkuberesource.PersistentVolumeClaims,\n\t\t*backup,\n\t\tclient,\n\t\tlogger,\n\t\tvh, // Pass non-nil VolumeHelper - exercises cached path\n\t)\n\n\trequire.NoError(t, err)\n\trequire.True(t, result, \"Should return true for snapshot when snapshotVolumes not set\")\n}\n" }, { "path": "pkg/plugin/velero/backupitemaction/v1/backup_item_action.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// BackupItemAction is an actor that performs an operation on an individual item being backed up.\ntype BackupItemAction interface {\n\t// AppliesTo returns information about which resources this action should be invoked for.\n\t// A BackupItemAction's Execute function will only be invoked on items that match the returned\n\t// selector. A zero-valued ResourceSelector matches all resources.\n\tAppliesTo() (velero.ResourceSelector, error)\n\n\t// Execute allows the ItemAction to perform arbitrary logic with the item being backed up,\n\t// including mutating the item itself prior to backup. The item (unmodified or modified)\n\t// should be returned, along with an optional slice of ResourceIdentifiers specifying\n\t// additional related items that should be backed up.\n\tExecute(item runtime.Unstructured, backup *api.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, error)\n}\n" }, { "path": "pkg/plugin/velero/backupitemaction/v2/backup_item_action.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v2\n\nimport (\n\t\"fmt\"\n\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/pkg/errors\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// BackupItemAction is an actor that performs an operation on an individual item being backed up.\ntype BackupItemAction interface {\n\t// Name returns the name of this BIA. Plugins which implement this interface must define Name,\n\t// but its content is unimportant, as it won't actually be called via RPC. Velero's plugin infrastructure\n\t// will implement this directly rather than delegating to the RPC plugin in order to return the name\n\t// that the plugin was registered under. The plugins must implement the method to complete the interface.\n\tName() string\n\n\t// AppliesTo returns information about which resources this action should be invoked for.\n\t// A BackupItemAction's Execute function will only be invoked on items that match the returned\n\t// selector. A zero-valued ResourceSelector matches all resources.\n\tAppliesTo() (velero.ResourceSelector, error)\n\n\t// Execute allows the BackupItemAction to perform arbitrary logic with the item being backed up,\n\t// including mutating the item itself prior to backup. The item (unmodified or modified)\n\t// should be returned, along with an optional slice of ResourceIdentifiers specifying\n\t// additional related items that should be backed up now, an optional operationID for actions which\n\t// initiate (asynchronous) operations, and a second slice of ResourceIdentifiers specifying related items\n\t// which should be backed up after all operations have completed. This last field will be\n\t// ignored if operationID is empty, and should not be filled in unless the resource must be updated in the\n\t// backup after operations complete (i.e. some of the item's Kubernetes metadata will be updated\n\t// during the operation which will be required during restore)\n\t// Note that (async) operations are not supported for items being backed up during Finalize phases,\n\t// so a plugin should not return an OperationID if the backup phase is \"Finalizing\"\n\t// or \"FinalizingPartiallyFailed\". The plugin should check the incoming\n\t// backup.Status.Phase before initiating operations, since the backup has already passed the waiting\n\t// for plugin operations phase. Plugins being called during Finalize will only be called for resources\n\t// that were returned as postOperationItems.\n\tExecute(item runtime.Unstructured, backup *api.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error)\n\n\t// Progress allows the BackupItemAction to report on progress of an asynchronous action.\n\t// For the passed-in operation, the plugin will return an OperationProgress struct, indicating\n\t// whether the operation has completed, whether there were any errors, a plugin-specific\n\t// indication of how much of the operation is done (items completed out of items-to-complete),\n\t// and started/updated timestamps\n\tProgress(operationID string, backup *api.Backup) (velero.OperationProgress, error)\n\n\t// Cancel allows the BackupItemAction to cancel an asynchronous action (if possible).\n\t// Velero will call this if the wait timeout for asynchronous actions has been reached.\n\t// If operation cancel is not supported, then the plugin just needs to return. No error\n\t// return is expected in this case, since cancellation is optional here.\n\tCancel(operationID string, backup *api.Backup) error\n}\n\nfunc AsyncOperationsNotSupportedError() error {\n\treturn errors.New(\"Plugin does not support asynchronous operations\")\n}\n\nfunc InvalidOperationIDError(operationID string) error {\n\treturn fmt.Errorf(\"operation ID %v is invalid\", operationID)\n}\n" }, { "path": "pkg/plugin/velero/delete_item_action.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage velero\n\nimport (\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// DeleteItemAction is an actor that performs an operation on an individual item being restored.\ntype DeleteItemAction interface {\n\t// AppliesTo returns information about which resources this action should be invoked for.\n\t// A DeleteItemAction's Execute function will only be invoked on items that match the returned\n\t// selector. A zero-valued ResourceSelector matches all resources.\n\tAppliesTo() (ResourceSelector, error)\n\n\t// Execute allows the ItemAction to perform arbitrary logic with the item being deleted.\n\t// An error should be returned if there were problems with the deletion process, but the\n\t// overall deletion process cannot be stopped.\n\t// Returned errors are logged.\n\tExecute(input *DeleteItemActionExecuteInput) error\n}\n\n// DeleteItemActionExecuteInput contains the input parameters for the ItemAction's Execute function.\ntype DeleteItemActionExecuteInput struct {\n\t// Item is the item taken from the pristine backed up version of resource.\n\tItem runtime.Unstructured\n\t// Backup is the representation of the restore resource processed by Velero.\n\tBackup *velerov1api.Backup\n}\n" }, { "path": "pkg/plugin/velero/itemblockaction/v1/item_block_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// ItemBlockAction is an action that returns a list of related items that must be backed up\n// along with the current item (and not in a separate parallel backup thread).\ntype ItemBlockAction interface {\n\t// Name returns the name of this IBA. Plugins which implement this interface must define Name,\n\t// but its content is unimportant, as it won't actually be called via RPC. Velero's plugin infrastructure\n\t// will implement this directly rather than delegating to the RPC plugin in order to return the name\n\t// that the plugin was registered under. The plugins must implement the method to complete the interface.\n\tName() string\n\n\t// AppliesTo returns information about which resources this action should be invoked for.\n\t// A ItemBlockAction's GetRelatedItems function will only be invoked on items that match the returned\n\t// selector. A zero-valued ResourceSelector matches all resources.\n\tAppliesTo() (velero.ResourceSelector, error)\n\n\t// GetRelatedItems allows the ItemBlockAction to identify related items which must be backed up\n\t// along with the current item. In many cases, these will be the same items that a related\n\t// BackupItemAction's Execute method will return as additionalItems, but there may be differences.\n\t// For example, items that are newly-created in the BIA Execute and don't yet exist at backup\n\t// start will *not* be returned here.\n\tGetRelatedItems(item runtime.Unstructured, backup *api.Backup) ([]velero.ResourceIdentifier, error)\n}\n" }, { "path": "pkg/plugin/velero/mocks/DeleteItemAction.go", "content": "// Code generated by mockery v2.1.0. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tmock \"github.com/stretchr/testify/mock\"\n\tvelero \"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// DeleteItemAction is an autogenerated mock type for the DeleteItemAction type\ntype DeleteItemAction struct {\n\tmock.Mock\n}\n\n// AppliesTo provides a mock function with given fields:\nfunc (_m *DeleteItemAction) AppliesTo() (velero.ResourceSelector, error) {\n\tret := _m.Called()\n\n\tvar r0 velero.ResourceSelector\n\tif rf, ok := ret.Get(0).(func() velero.ResourceSelector); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(velero.ResourceSelector)\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Execute provides a mock function with given fields: input\nfunc (_m *DeleteItemAction) Execute(input *velero.DeleteItemActionExecuteInput) error {\n\tret := _m.Called(input)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(*velero.DeleteItemActionExecuteInput) error); ok {\n\t\tr0 = rf(input)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n" }, { "path": "pkg/plugin/velero/mocks/backupitemaction/v1/BackupItemAction.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n// Code generated by mockery v1.0.0. DO NOT EDIT.\n\npackage v1\n\nimport (\n\tmock \"github.com/stretchr/testify/mock\"\n\truntime \"k8s.io/apimachinery/pkg/runtime\"\n\n\tvelero \"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\n\tvelerov1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// BackupItemAction is an autogenerated mock type for the BackupItemAction type\ntype BackupItemAction struct {\n\tmock.Mock\n}\n\n// AppliesTo provides a mock function with given fields:\nfunc (_m *BackupItemAction) AppliesTo() (velero.ResourceSelector, error) {\n\tret := _m.Called()\n\n\tvar r0 velero.ResourceSelector\n\tif rf, ok := ret.Get(0).(func() velero.ResourceSelector); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(velero.ResourceSelector)\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Execute provides a mock function with given fields: item, backup\nfunc (_m *BackupItemAction) Execute(item runtime.Unstructured, backup *velerov1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, error) {\n\tret := _m.Called(item, backup)\n\n\tvar r0 runtime.Unstructured\n\tif rf, ok := ret.Get(0).(func(runtime.Unstructured, *velerov1.Backup) runtime.Unstructured); ok {\n\t\tr0 = rf(item, backup)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(runtime.Unstructured)\n\t\t}\n\t}\n\n\tvar r1 []velero.ResourceIdentifier\n\tif rf, ok := ret.Get(1).(func(runtime.Unstructured, *velerov1.Backup) []velero.ResourceIdentifier); ok {\n\t\tr1 = rf(item, backup)\n\t} else {\n\t\tif ret.Get(1) != nil {\n\t\t\tr1 = ret.Get(1).([]velero.ResourceIdentifier)\n\t\t}\n\t}\n\n\tvar r2 error\n\tif rf, ok := ret.Get(2).(func(runtime.Unstructured, *velerov1.Backup) error); ok {\n\t\tr2 = rf(item, backup)\n\t} else {\n\t\tr2 = ret.Error(2)\n\t}\n\n\treturn r0, r1, r2\n}\n" }, { "path": "pkg/plugin/velero/mocks/backupitemaction/v2/BackupItemAction.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n// Code generated by mockery v2.16.0. DO NOT EDIT.\n\npackage v2\n\nimport (\n\tmock \"github.com/stretchr/testify/mock\"\n\truntime \"k8s.io/apimachinery/pkg/runtime\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\n\tvelero \"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// BackupItemAction is an autogenerated mock type for the BackupItemAction type\ntype BackupItemAction struct {\n\tmock.Mock\n}\n\n// AppliesTo provides a mock function with given fields:\nfunc (_m *BackupItemAction) AppliesTo() (velero.ResourceSelector, error) {\n\tret := _m.Called()\n\n\tvar r0 velero.ResourceSelector\n\tif rf, ok := ret.Get(0).(func() velero.ResourceSelector); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(velero.ResourceSelector)\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Cancel provides a mock function with given fields: operationID, backup\nfunc (_m *BackupItemAction) Cancel(operationID string, backup *v1.Backup) error {\n\tret := _m.Called(operationID, backup)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(string, *v1.Backup) error); ok {\n\t\tr0 = rf(operationID, backup)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// Execute provides a mock function with given fields: item, backup\nfunc (_m *BackupItemAction) Execute(item runtime.Unstructured, backup *v1.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error) {\n\tret := _m.Called(item, backup)\n\n\tvar r0 runtime.Unstructured\n\tif rf, ok := ret.Get(0).(func(runtime.Unstructured, *v1.Backup) runtime.Unstructured); ok {\n\t\tr0 = rf(item, backup)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(runtime.Unstructured)\n\t\t}\n\t}\n\n\tvar r1 []velero.ResourceIdentifier\n\tif rf, ok := ret.Get(1).(func(runtime.Unstructured, *v1.Backup) []velero.ResourceIdentifier); ok {\n\t\tr1 = rf(item, backup)\n\t} else {\n\t\tif ret.Get(1) != nil {\n\t\t\tr1 = ret.Get(1).([]velero.ResourceIdentifier)\n\t\t}\n\t}\n\n\tvar r2 string\n\tif rf, ok := ret.Get(2).(func(runtime.Unstructured, *v1.Backup) string); ok {\n\t\tr2 = rf(item, backup)\n\t} else {\n\t\tr2 = ret.Get(2).(string)\n\t}\n\n\tvar r3 []velero.ResourceIdentifier\n\tif rf, ok := ret.Get(3).(func(runtime.Unstructured, *v1.Backup) []velero.ResourceIdentifier); ok {\n\t\tr3 = rf(item, backup)\n\t} else {\n\t\tif ret.Get(3) != nil {\n\t\t\tr3 = ret.Get(3).([]velero.ResourceIdentifier)\n\t\t}\n\t}\n\n\tvar r4 error\n\tif rf, ok := ret.Get(4).(func(runtime.Unstructured, *v1.Backup) error); ok {\n\t\tr4 = rf(item, backup)\n\t} else {\n\t\tr4 = ret.Error(4)\n\t}\n\n\treturn r0, r1, r2, r3, r4\n}\n\n// Name provides a mock function with given fields:\nfunc (_m *BackupItemAction) Name() string {\n\tret := _m.Called()\n\n\tvar r0 string\n\tif rf, ok := ret.Get(0).(func() string); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(string)\n\t}\n\n\treturn r0\n}\n\n// Progress provides a mock function with given fields: operationID, backup\nfunc (_m *BackupItemAction) Progress(operationID string, backup *v1.Backup) (velero.OperationProgress, error) {\n\tret := _m.Called(operationID, backup)\n\n\tvar r0 velero.OperationProgress\n\tif rf, ok := ret.Get(0).(func(string, *v1.Backup) velero.OperationProgress); ok {\n\t\tr0 = rf(operationID, backup)\n\t} else {\n\t\tr0 = ret.Get(0).(velero.OperationProgress)\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(string, *v1.Backup) error); ok {\n\t\tr1 = rf(operationID, backup)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\ntype mockConstructorTestingTNewBackupItemAction interface {\n\tmock.TestingT\n\tCleanup(func())\n}\n\n// NewBackupItemAction creates a new instance of BackupItemAction. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\nfunc NewBackupItemAction(t mockConstructorTestingTNewBackupItemAction) *BackupItemAction {\n\tmock := &BackupItemAction{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/plugin/velero/mocks/itemblockaction/v1/ItemBlockAction.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n// Code generated by mockery v2.43.2. DO NOT EDIT.\n\npackage v1\n\nimport (\n\tmock \"github.com/stretchr/testify/mock\"\n\truntime \"k8s.io/apimachinery/pkg/runtime\"\n\n\tvelero \"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\n\tvelerov1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// ItemBlockAction is an autogenerated mock type for the ItemBlockAction type\ntype ItemBlockAction struct {\n\tmock.Mock\n}\n\n// AppliesTo provides a mock function with given fields:\nfunc (_m *ItemBlockAction) AppliesTo() (velero.ResourceSelector, error) {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for AppliesTo\")\n\t}\n\n\tvar r0 velero.ResourceSelector\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func() (velero.ResourceSelector, error)); ok {\n\t\treturn rf()\n\t}\n\tif rf, ok := ret.Get(0).(func() velero.ResourceSelector); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(velero.ResourceSelector)\n\t}\n\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetRelatedItems provides a mock function with given fields: item, backup\nfunc (_m *ItemBlockAction) GetRelatedItems(item runtime.Unstructured, backup *velerov1.Backup) ([]velero.ResourceIdentifier, error) {\n\tret := _m.Called(item, backup)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetRelatedItems\")\n\t}\n\n\tvar r0 []velero.ResourceIdentifier\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(runtime.Unstructured, *velerov1.Backup) ([]velero.ResourceIdentifier, error)); ok {\n\t\treturn rf(item, backup)\n\t}\n\tif rf, ok := ret.Get(0).(func(runtime.Unstructured, *velerov1.Backup) []velero.ResourceIdentifier); ok {\n\t\tr0 = rf(item, backup)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]velero.ResourceIdentifier)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(runtime.Unstructured, *velerov1.Backup) error); ok {\n\t\tr1 = rf(item, backup)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Name provides a mock function with given fields:\nfunc (_m *ItemBlockAction) Name() string {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Name\")\n\t}\n\n\tvar r0 string\n\tif rf, ok := ret.Get(0).(func() string); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(string)\n\t}\n\n\treturn r0\n}\n\n// NewItemBlockAction creates a new instance of ItemBlockAction. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\n// The first argument is typically a *testing.T value.\nfunc NewItemBlockAction(t interface {\n\tmock.TestingT\n\tCleanup(func())\n}) *ItemBlockAction {\n\tmock := &ItemBlockAction{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/plugin/velero/mocks/object_store.go", "content": "// Code generated by mockery v1.0.0. DO NOT EDIT.\n\npackage mocks\n\nimport io \"io\"\nimport mock \"github.com/stretchr/testify/mock\"\nimport time \"time\"\n\n// ObjectStore is an autogenerated mock type for the ObjectStore type\ntype ObjectStore struct {\n\tmock.Mock\n}\n\n// CreateSignedURL provides a mock function with given fields: bucket, key, ttl\nfunc (_m *ObjectStore) CreateSignedURL(bucket string, key string, ttl time.Duration) (string, error) {\n\tret := _m.Called(bucket, key, ttl)\n\n\tvar r0 string\n\tif rf, ok := ret.Get(0).(func(string, string, time.Duration) string); ok {\n\t\tr0 = rf(bucket, key, ttl)\n\t} else {\n\t\tr0 = ret.Get(0).(string)\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(string, string, time.Duration) error); ok {\n\t\tr1 = rf(bucket, key, ttl)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// DeleteObject provides a mock function with given fields: bucket, key\nfunc (_m *ObjectStore) DeleteObject(bucket string, key string) error {\n\tret := _m.Called(bucket, key)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(string, string) error); ok {\n\t\tr0 = rf(bucket, key)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// GetObject provides a mock function with given fields: bucket, key\nfunc (_m *ObjectStore) GetObject(bucket string, key string) (io.ReadCloser, error) {\n\tret := _m.Called(bucket, key)\n\n\tvar r0 io.ReadCloser\n\tif rf, ok := ret.Get(0).(func(string, string) io.ReadCloser); ok {\n\t\tr0 = rf(bucket, key)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(io.ReadCloser)\n\t\t}\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(string, string) error); ok {\n\t\tr1 = rf(bucket, key)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Init provides a mock function with given fields: config\nfunc (_m *ObjectStore) Init(config map[string]string) error {\n\tret := _m.Called(config)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(map[string]string) error); ok {\n\t\tr0 = rf(config)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// ListCommonPrefixes provides a mock function with given fields: bucket, prefix, delimiter\nfunc (_m *ObjectStore) ListCommonPrefixes(bucket string, prefix string, delimiter string) ([]string, error) {\n\tret := _m.Called(bucket, prefix, delimiter)\n\n\tvar r0 []string\n\tif rf, ok := ret.Get(0).(func(string, string, string) []string); ok {\n\t\tr0 = rf(bucket, prefix, delimiter)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]string)\n\t\t}\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(string, string, string) error); ok {\n\t\tr1 = rf(bucket, prefix, delimiter)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// ListObjects provides a mock function with given fields: bucket, prefix\nfunc (_m *ObjectStore) ListObjects(bucket string, prefix string) ([]string, error) {\n\tret := _m.Called(bucket, prefix)\n\n\tvar r0 []string\n\tif rf, ok := ret.Get(0).(func(string, string) []string); ok {\n\t\tr0 = rf(bucket, prefix)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]string)\n\t\t}\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(string, string) error); ok {\n\t\tr1 = rf(bucket, prefix)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// ObjectExists provides a mock function with given fields: bucket, key\nfunc (_m *ObjectStore) ObjectExists(bucket string, key string) (bool, error) {\n\tret := _m.Called(bucket, key)\n\n\tvar r0 bool\n\tif rf, ok := ret.Get(0).(func(string, string) bool); ok {\n\t\tr0 = rf(bucket, key)\n\t} else {\n\t\tr0 = ret.Get(0).(bool)\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(string, string) error); ok {\n\t\tr1 = rf(bucket, key)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// PutObject provides a mock function with given fields: bucket, key, body\nfunc (_m *ObjectStore) PutObject(bucket string, key string, body io.Reader) error {\n\tret := _m.Called(bucket, key, body)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(string, string, io.Reader) error); ok {\n\t\tr0 = rf(bucket, key, body)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n" }, { "path": "pkg/plugin/velero/mocks/restoreitemaction/v1/RestoreItemAction.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n// Code generated by mockery v1.0.0. DO NOT EDIT.\n\npackage v1\n\nimport (\n\tmock \"github.com/stretchr/testify/mock\"\n\tvelero \"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// RestoreItemAction is an autogenerated mock type for the RestoreItemAction type\ntype RestoreItemAction struct {\n\tmock.Mock\n}\n\n// AppliesTo provides a mock function with given fields:\nfunc (_m *RestoreItemAction) AppliesTo() (velero.ResourceSelector, error) {\n\tret := _m.Called()\n\n\tvar r0 velero.ResourceSelector\n\tif rf, ok := ret.Get(0).(func() velero.ResourceSelector); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(velero.ResourceSelector)\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Execute provides a mock function with given fields: input\nfunc (_m *RestoreItemAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\tret := _m.Called(input)\n\n\tvar r0 *velero.RestoreItemActionExecuteOutput\n\tif rf, ok := ret.Get(0).(func(*velero.RestoreItemActionExecuteInput) *velero.RestoreItemActionExecuteOutput); ok {\n\t\tr0 = rf(input)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(*velero.RestoreItemActionExecuteOutput)\n\t\t}\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(*velero.RestoreItemActionExecuteInput) error); ok {\n\t\tr1 = rf(input)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n" }, { "path": "pkg/plugin/velero/mocks/restoreitemaction/v2/RestoreItemAction.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n// Code generated by mockery v2.16.0. DO NOT EDIT.\n\npackage v2\n\nimport (\n\tmock \"github.com/stretchr/testify/mock\"\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\n\tvelero \"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// RestoreItemAction is an autogenerated mock type for the RestoreItemAction type\ntype RestoreItemAction struct {\n\tmock.Mock\n}\n\n// AppliesTo provides a mock function with given fields:\nfunc (_m *RestoreItemAction) AppliesTo() (velero.ResourceSelector, error) {\n\tret := _m.Called()\n\n\tvar r0 velero.ResourceSelector\n\tif rf, ok := ret.Get(0).(func() velero.ResourceSelector); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(velero.ResourceSelector)\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// AreAdditionalItemsReady provides a mock function with given fields: AdditionalItems, restore\nfunc (_m *RestoreItemAction) AreAdditionalItemsReady(additionalItems []velero.ResourceIdentifier, restore *v1.Restore) (bool, error) {\n\tret := _m.Called(additionalItems, restore)\n\n\tvar r0 bool\n\tif rf, ok := ret.Get(0).(func([]velero.ResourceIdentifier, *v1.Restore) bool); ok {\n\t\tr0 = rf(additionalItems, restore)\n\t} else {\n\t\tr0 = ret.Get(0).(bool)\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func([]velero.ResourceIdentifier, *v1.Restore) error); ok {\n\t\tr1 = rf(additionalItems, restore)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Cancel provides a mock function with given fields: operationID, restore\nfunc (_m *RestoreItemAction) Cancel(operationID string, restore *v1.Restore) error {\n\tret := _m.Called(operationID, restore)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(string, *v1.Restore) error); ok {\n\t\tr0 = rf(operationID, restore)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// Execute provides a mock function with given fields: input\nfunc (_m *RestoreItemAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\tret := _m.Called(input)\n\n\tvar r0 *velero.RestoreItemActionExecuteOutput\n\tif rf, ok := ret.Get(0).(func(*velero.RestoreItemActionExecuteInput) *velero.RestoreItemActionExecuteOutput); ok {\n\t\tr0 = rf(input)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(*velero.RestoreItemActionExecuteOutput)\n\t\t}\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(*velero.RestoreItemActionExecuteInput) error); ok {\n\t\tr1 = rf(input)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Name provides a mock function with given fields:\nfunc (_m *RestoreItemAction) Name() string {\n\tret := _m.Called()\n\n\tvar r0 string\n\tif rf, ok := ret.Get(0).(func() string); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(string)\n\t}\n\n\treturn r0\n}\n\n// Progress provides a mock function with given fields: operationID, restore\nfunc (_m *RestoreItemAction) Progress(operationID string, restore *v1.Restore) (velero.OperationProgress, error) {\n\tret := _m.Called(operationID, restore)\n\n\tvar r0 velero.OperationProgress\n\tif rf, ok := ret.Get(0).(func(string, *v1.Restore) velero.OperationProgress); ok {\n\t\tr0 = rf(operationID, restore)\n\t} else {\n\t\tr0 = ret.Get(0).(velero.OperationProgress)\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(string, *v1.Restore) error); ok {\n\t\tr1 = rf(operationID, restore)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\ntype mockConstructorTestingTNewRestoreItemAction interface {\n\tmock.TestingT\n\tCleanup(func())\n}\n\n// NewRestoreItemAction creates a new instance of RestoreItemAction. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\nfunc NewRestoreItemAction(t mockConstructorTestingTNewRestoreItemAction) *RestoreItemAction {\n\tmock := &RestoreItemAction{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/plugin/velero/mocks/volumesnapshotter/v1/VolumeSnapshotter.go", "content": "// Code generated by mockery v1.0.0. DO NOT EDIT.\n\npackage mocks\n\nimport mock \"github.com/stretchr/testify/mock\"\nimport runtime \"k8s.io/apimachinery/pkg/runtime\"\n\n// VolumeSnapshotter is an autogenerated mock type for the VolumeSnapshotter type\ntype VolumeSnapshotter struct {\n\tmock.Mock\n}\n\n// CreateSnapshot provides a mock function with given fields: volumeID, volumeAZ, tags\nfunc (_m *VolumeSnapshotter) CreateSnapshot(volumeID string, volumeAZ string, tags map[string]string) (string, error) {\n\tret := _m.Called(volumeID, volumeAZ, tags)\n\n\tvar r0 string\n\tif rf, ok := ret.Get(0).(func(string, string, map[string]string) string); ok {\n\t\tr0 = rf(volumeID, volumeAZ, tags)\n\t} else {\n\t\tr0 = ret.Get(0).(string)\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(string, string, map[string]string) error); ok {\n\t\tr1 = rf(volumeID, volumeAZ, tags)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// CreateVolumeFromSnapshot provides a mock function with given fields: snapshotID, volumeType, volumeAZ, iops\nfunc (_m *VolumeSnapshotter) CreateVolumeFromSnapshot(snapshotID string, volumeType string, volumeAZ string, iops *int64) (string, error) {\n\tret := _m.Called(snapshotID, volumeType, volumeAZ, iops)\n\n\tvar r0 string\n\tif rf, ok := ret.Get(0).(func(string, string, string, *int64) string); ok {\n\t\tr0 = rf(snapshotID, volumeType, volumeAZ, iops)\n\t} else {\n\t\tr0 = ret.Get(0).(string)\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(string, string, string, *int64) error); ok {\n\t\tr1 = rf(snapshotID, volumeType, volumeAZ, iops)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// DeleteSnapshot provides a mock function with given fields: snapshotID\nfunc (_m *VolumeSnapshotter) DeleteSnapshot(snapshotID string) error {\n\tret := _m.Called(snapshotID)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(string) error); ok {\n\t\tr0 = rf(snapshotID)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// GetVolumeID provides a mock function with given fields: pv\nfunc (_m *VolumeSnapshotter) GetVolumeID(pv runtime.Unstructured) (string, error) {\n\tret := _m.Called(pv)\n\n\tvar r0 string\n\tif rf, ok := ret.Get(0).(func(runtime.Unstructured) string); ok {\n\t\tr0 = rf(pv)\n\t} else {\n\t\tr0 = ret.Get(0).(string)\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(runtime.Unstructured) error); ok {\n\t\tr1 = rf(pv)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetVolumeInfo provides a mock function with given fields: volumeID, volumeAZ\nfunc (_m *VolumeSnapshotter) GetVolumeInfo(volumeID string, volumeAZ string) (string, *int64, error) {\n\tret := _m.Called(volumeID, volumeAZ)\n\n\tvar r0 string\n\tif rf, ok := ret.Get(0).(func(string, string) string); ok {\n\t\tr0 = rf(volumeID, volumeAZ)\n\t} else {\n\t\tr0 = ret.Get(0).(string)\n\t}\n\n\tvar r1 *int64\n\tif rf, ok := ret.Get(1).(func(string, string) *int64); ok {\n\t\tr1 = rf(volumeID, volumeAZ)\n\t} else {\n\t\tif ret.Get(1) != nil {\n\t\t\tr1 = ret.Get(1).(*int64)\n\t\t}\n\t}\n\n\tvar r2 error\n\tif rf, ok := ret.Get(2).(func(string, string) error); ok {\n\t\tr2 = rf(volumeID, volumeAZ)\n\t} else {\n\t\tr2 = ret.Error(2)\n\t}\n\n\treturn r0, r1, r2\n}\n\n// Init provides a mock function with given fields: config\nfunc (_m *VolumeSnapshotter) Init(config map[string]string) error {\n\tret := _m.Called(config)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(map[string]string) error); ok {\n\t\tr0 = rf(config)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// SetVolumeID provides a mock function with given fields: pv, volumeID\nfunc (_m *VolumeSnapshotter) SetVolumeID(pv runtime.Unstructured, volumeID string) (runtime.Unstructured, error) {\n\tret := _m.Called(pv, volumeID)\n\n\tvar r0 runtime.Unstructured\n\tif rf, ok := ret.Get(0).(func(runtime.Unstructured, string) runtime.Unstructured); ok {\n\t\tr0 = rf(pv, volumeID)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(runtime.Unstructured)\n\t\t}\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(runtime.Unstructured, string) error); ok {\n\t\tr1 = rf(pv, volumeID)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n" }, { "path": "pkg/plugin/velero/object_store.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage velero\n\nimport (\n\t\"io\"\n\t\"time\"\n)\n\n// ObjectStore exposes basic object-storage operations required\n// by Velero.\ntype ObjectStore interface {\n\t// Init prepares the ObjectStore for usage using the provided map of\n\t// configuration key-value pairs. It returns an error if the ObjectStore\n\t// cannot be initialized from the provided config.\n\tInit(config map[string]string) error\n\n\t// PutObject creates a new object using the data in body within the specified\n\t// object storage bucket with the given key.\n\tPutObject(bucket, key string, body io.Reader) error\n\n\t// ObjectExists checks if there is an object with the given key in the object storage bucket.\n\tObjectExists(bucket, key string) (bool, error)\n\n\t// GetObject retrieves the object with the given key from the specified\n\t// bucket in object storage.\n\tGetObject(bucket, key string) (io.ReadCloser, error)\n\n\t// ListCommonPrefixes gets a list of all object key prefixes that start with\n\t// the specified prefix and stop at the next instance of the provided delimiter.\n\t//\n\t// For example, if the bucket contains the following keys:\n\t//\t\ta-prefix/foo-1/bar\n\t// \t\ta-prefix/foo-1/baz\n\t//\t\ta-prefix/foo-2/baz\n\t// \t\tsome-other-prefix/foo-3/bar\n\t// and the provided prefix arg is \"a-prefix/\", and the delimiter is \"/\",\n\t// this will return the slice {\"a-prefix/foo-1/\", \"a-prefix/foo-2/\"}.\n\tListCommonPrefixes(bucket, prefix, delimiter string) ([]string, error)\n\n\t// ListObjects gets a list of all keys in the specified bucket\n\t// that have the given prefix.\n\tListObjects(bucket, prefix string) ([]string, error)\n\n\t// DeleteObject removes the object with the specified key from the given\n\t// bucket.\n\tDeleteObject(bucket, key string) error\n\n\t// CreateSignedURL creates a pre-signed URL for the given bucket and key that expires after ttl.\n\tCreateSignedURL(bucket, key string, ttl time.Duration) (string, error)\n}\n" }, { "path": "pkg/plugin/velero/restore_item_action_shared.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage velero\n\nimport (\n\t\"time\"\n\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// RestoreItemActionExecuteInput contains the input parameters for the ItemAction's Execute function.\ntype RestoreItemActionExecuteInput struct {\n\t// Item is the item being restored. It is likely different from the pristine backed up version\n\t// (metadata reset, changed by various restore item action plugins, etc.).\n\tItem runtime.Unstructured\n\t// ItemFromBackup is the item taken from the pristine backed up version of resource.\n\tItemFromBackup runtime.Unstructured\n\t// Restore is the representation of the restore resource processed by Velero.\n\tRestore *api.Restore\n}\n\n// RestoreItemActionExecuteOutput contains the output variables for the ItemAction's Execution function.\ntype RestoreItemActionExecuteOutput struct {\n\t// UpdatedItem is the item being restored mutated by ItemAction.\n\tUpdatedItem runtime.Unstructured\n\n\t// AdditionalItems is a list of additional related items that should\n\t// be restored.\n\tAdditionalItems []ResourceIdentifier\n\n\t// SkipRestore tells velero to stop executing further actions\n\t// on this item, and skip the restore step. When this field's\n\t// value is true, AdditionalItems will be ignored.\n\tSkipRestore bool\n\n\t// v2 and later\n\t// OperationID is an identifier which indicates an ongoing asynchronous action which Velero will\n\t// continue to monitor after restoring this item. If left blank, then there is no ongoing operation.\n\tOperationID string\n\n\t// v2 and later\n\t// WaitForAdditionalItems determines whether velero will wait\n\t// until AreAdditionalItemsReady returns true before restoring\n\t// this item. If this field's value is true, then after restoring\n\t// the returned AdditionalItems, velero will not restore this item\n\t// until AreAdditionalItemsReady returns true or the timeout is\n\t// reached. Otherwise, AreAdditionalItemsReady is not called.\n\tWaitForAdditionalItems bool\n\n\t// v2 and later\n\t// AdditionalItemsReadyTimeout will override serverConfig.additionalItemsReadyTimeout\n\t// if specified. This value specifies how long velero will wait\n\t// for additional items to be ready before moving on.\n\tAdditionalItemsReadyTimeout time.Duration\n}\n\n// NewRestoreItemActionExecuteOutput creates a new RestoreItemActionExecuteOutput\nfunc NewRestoreItemActionExecuteOutput(item runtime.Unstructured) *RestoreItemActionExecuteOutput {\n\treturn &RestoreItemActionExecuteOutput{\n\t\tUpdatedItem: item,\n\t}\n}\n\n// WithoutRestore returns SkipRestore for RestoreItemActionExecuteOutput\nfunc (r *RestoreItemActionExecuteOutput) WithoutRestore() *RestoreItemActionExecuteOutput {\n\tr.SkipRestore = true\n\treturn r\n}\n\n// WithOperationID returns RestoreItemActionExecuteOutput with OperationID set.\nfunc (r *RestoreItemActionExecuteOutput) WithOperationID(operationID string) *RestoreItemActionExecuteOutput {\n\tr.OperationID = operationID\n\treturn r\n}\n\n// WithItemsWait returns RestoreItemActionExecuteOutput with WaitForAdditionalItems set to true.\nfunc (r *RestoreItemActionExecuteOutput) WithItemsWait() *RestoreItemActionExecuteOutput {\n\tr.WaitForAdditionalItems = true\n\treturn r\n}\n" }, { "path": "pkg/plugin/velero/restoreitemaction/v1/restore_item_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// RestoreItemAction is an actor that performs an operation on an individual item being restored.\ntype RestoreItemAction interface {\n\t// AppliesTo returns information about which resources this action should be invoked for.\n\t// A RestoreItemAction's Execute function will only be invoked on items that match the returned\n\t// selector. A zero-valued ResourceSelector matches all resources.\n\tAppliesTo() (velero.ResourceSelector, error)\n\n\t// Execute allows the ItemAction to perform arbitrary logic with the item being restored,\n\t// including mutating the item itself prior to restore. The item (unmodified or modified)\n\t// should be returned, along with an optional slice of ResourceIdentifiers specifying additional\n\t// related items that should be restored, a warning (which will be logged but will not prevent\n\t// the item from being restored) or error (which will be logged and will prevent the item\n\t// from being restored) if applicable.\n\tExecute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error)\n}\n" }, { "path": "pkg/plugin/velero/restoreitemaction/v2/restore_item_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v2\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/pkg/errors\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// RestoreItemAction is an actor that performs an operation on an individual item being restored.\ntype RestoreItemAction interface {\n\t// Name returns the name of this RIA. Plugins which implement this interface must define Name,\n\t// but its content is unimportant, as it won't actually be called via RPC. Velero's plugin infrastructure\n\t// will implement this directly rather than delegating to the RPC plugin in order to return the name\n\t// that the plugin was registered under. The plugins must implement the method to complete the interface.\n\tName() string\n\n\t// AppliesTo returns information about which resources this action should be invoked for.\n\t// A RestoreItemAction's Execute function will only be invoked on items that match the returned\n\t// selector. A zero-valued ResourceSelector matches all resources.\n\tAppliesTo() (velero.ResourceSelector, error)\n\n\t// Execute allows the ItemAction to perform arbitrary logic with the item being restored,\n\t// including mutating the item itself prior to restore. The return struct includes:\n\t// The item (unmodified or modified), an optional slice of ResourceIdentifiers\n\t// specifying additional related items that should be restored, an optional OperationID,\n\t// a bool (waitForAdditionalItems) specifying whether Velero should wait until restored additional\n\t// items are ready before restoring this resource, and an optional timeout for the additional items\n\t// wait period. An error is returned if the action fails.\n\tExecute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error)\n\n\t// Progress allows the RestoreItemAction to report on progress of an asynchronous action.\n\t// For the passed-in operation, the plugin will return an OperationProgress struct, indicating\n\t// whether the operation has completed, whether there were any errors, a plugin-specific\n\t// indication of how much of the operation is done (items completed out of items-to-complete),\n\t// and started/updated timestamps\n\tProgress(operationID string, restore *api.Restore) (velero.OperationProgress, error)\n\n\t// Cancel allows the RestoreItemAction to cancel an asynchronous action (if possible).\n\t// Velero will call this if the wait timeout for asynchronous actions has been reached.\n\t// If operation cancel is not supported, then the plugin just needs to return. No error\n\t// return is expected in this case, since cancellation is optional here.\n\tCancel(operationID string, restore *api.Restore) error\n\n\t// AreAdditionalItemsReady allows the ItemAction to communicate whether the passed-in\n\t// slice of AdditionalItems (previously returned by Execute())\n\t// are ready. Returns true if all items are ready, and false\n\t// otherwise. The second return value is to report errors\n\tAreAdditionalItemsReady(additionalItems []velero.ResourceIdentifier, restore *api.Restore) (bool, error)\n}\n\nfunc AsyncOperationsNotSupportedError() error {\n\treturn errors.New(\"Plugin does not support asynchronous operations\")\n}\n\nfunc InvalidOperationIDError(operationID string) error {\n\treturn fmt.Errorf(\"operation ID %v is invalid\", operationID)\n}\n" }, { "path": "pkg/plugin/velero/shared.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\n// Package velero contains the interfaces necessary to implement\n// all of the Velero plugins. Users create their own binary containing\n// implementations of the plugin kinds in this package. Multiple\n// plugins of any type can be implemented.\npackage velero\n\nimport (\n\t\"time\"\n\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n)\n\n// ResourceSelector is a collection of included/excluded namespaces,\n// included/excluded resources, and a label-selector that can be used\n// to match a set of items from a cluster.\ntype ResourceSelector struct {\n\t// IncludedNamespaces is a slice of namespace names to match. All\n\t// namespaces in this slice, except those in ExcludedNamespaces,\n\t// will be matched. A nil/empty slice matches all namespaces.\n\tIncludedNamespaces []string\n\t// ExcludedNamespaces is a slice of namespace names to exclude.\n\t// All namespaces in IncludedNamespaces, *except* those in\n\t// this slice, will be matched.\n\tExcludedNamespaces []string\n\t// IncludedResources is a slice of resources to match. Resources may be specified\n\t// as full names (e.g. \"services\"), abbreviations (e.g. \"svc\"), or with the\n\t// groups they are in (e.g. \"ingresses.extensions\"). All resources in this slice,\n\t// except those in ExcludedResources, will be matched. A nil/empty slice matches\n\t// all resources.\n\tIncludedResources []string\n\t// ExcludedResources is a slice of resources to exclude. Resources may be specified\n\t// as full names (e.g. \"services\"), abbreviations (e.g. \"svc\"), or with the\n\t// groups they are in (e.g. \"ingresses.extensions\"). All resources in IncludedResources,\n\t// *except* those in this slice, will be matched.\n\tExcludedResources []string\n\t// LabelSelector is a string representation of a selector to apply\n\t// when matching resources. See \"k8s.io/apimachinery/pkg/labels\".Parse()\n\t// for details on syntax.\n\tLabelSelector string\n}\n\n// Applicable allows actions and plugins to specify which resources they should be invoked for\ntype Applicable interface {\n\t// AppliesTo returns information about which resources this Responder should be invoked for.\n\tAppliesTo() (ResourceSelector, error)\n}\n\n// ResourceIdentifier describes a single item by its group, resource, namespace, and name.\ntype ResourceIdentifier struct {\n\tschema.GroupResource\n\tNamespace string\n\tName string\n}\n\nfunc (in *ResourceIdentifier) DeepCopy() *ResourceIdentifier {\n\tif in == nil {\n\t\treturn nil\n\t}\n\tout := new(ResourceIdentifier)\n\tin.DeepCopyInto(out)\n\treturn out\n}\n\nfunc (in *ResourceIdentifier) DeepCopyInto(out *ResourceIdentifier) {\n\t*out = *in\n\tout.GroupResource = in.GroupResource\n}\n\n// OperationProgress describes progress of an asynchronous plugin operation.\ntype OperationProgress struct {\n\t// True when the operation has completed, either successfully or with a failure\n\tCompleted bool\n\t// Set when the operation has failed\n\tErr string\n\t// Quantity completed so far and the total quantity associated with the operation\n\t// in OperationUnits. For data mover and volume snapshotter use cases, this will\n\t// usually be in bytes. On successful completion, NCompleted and NTotal should be\n\t// the same\n\tNCompleted, NTotal int64\n\t// Units represented by NCompleted and NTotal -- for data mover and item\n\t// snapshotters, this will usually be bytes.\n\tOperationUnits string\n\t// Optional description of operation progress (i.e. \"Current phase: Running\")\n\tDescription string\n\t// When the operation was started and when the last update was seen. Not all\n\t// systems retain when the upload was begun, return Time 0 (time.Unix(0, 0))\n\t// if unknown.\n\tStarted, Updated time.Time\n}\n" }, { "path": "pkg/plugin/velero/volumesnapshotter/v1/volume_snapshotter.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage v1\n\nimport (\n\t\"k8s.io/apimachinery/pkg/runtime\"\n)\n\n// VolumeSnapshotter defines the operations needed by Velero to\n// take snapshots of persistent volumes during backup, and to restore\n// persistent volumes from snapshots during restore.\ntype VolumeSnapshotter interface {\n\t// Init prepares the VolumeSnapshotter for usage using the provided map of\n\t// configuration key-value pairs. It returns an error if the VolumeSnapshotter\n\t// cannot be initialized from the provided config.\n\tInit(config map[string]string) error\n\n\t// CreateVolumeFromSnapshot creates a new volume in the specified\n\t// availability zone, initialized from the provided snapshot,\n\t// and with the specified type and IOPS (if using provisioned IOPS).\n\tCreateVolumeFromSnapshot(snapshotID, volumeType, volumeAZ string, iops *int64) (volumeID string, err error)\n\n\t// GetVolumeID returns the cloud provider specific identifier for the PersistentVolume.\n\tGetVolumeID(pv runtime.Unstructured) (string, error)\n\n\t// SetVolumeID sets the cloud provider specific identifier for the PersistentVolume.\n\tSetVolumeID(pv runtime.Unstructured, volumeID string) (runtime.Unstructured, error)\n\n\t// GetVolumeInfo returns the type and IOPS (if using provisioned IOPS) for\n\t// the specified volume in the given availability zone.\n\tGetVolumeInfo(volumeID, volumeAZ string) (string, *int64, error)\n\n\t// CreateSnapshot creates a snapshot of the specified volume, and applies the provided\n\t// set of tags to the snapshot.\n\tCreateSnapshot(volumeID, volumeAZ string, tags map[string]string) (snapshotID string, err error)\n\n\t// DeleteSnapshot deletes the specified volume snapshot.\n\tDeleteSnapshot(snapshotID string) error\n}\n" }, { "path": "pkg/podexec/pod_command_executor.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podexec\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"net/url\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\tkscheme \"k8s.io/client-go/kubernetes/scheme\"\n\t\"k8s.io/client-go/rest\"\n\t\"k8s.io/client-go/tools/remotecommand\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\nconst defaultTimeout = 30 * time.Second\n\n// PodCommandExecutor is capable of executing a command in a container in a pod.\ntype PodCommandExecutor interface {\n\t// ExecutePodCommand executes a command in a container in a pod. If the command takes longer than\n\t// the specified timeout, an error is returned.\n\tExecutePodCommand(log logrus.FieldLogger, item map[string]any, namespace, name, hookName string, hook *api.ExecHook) error\n}\n\ntype poster interface {\n\tPost() *rest.Request\n}\n\ntype defaultPodCommandExecutor struct {\n\trestClientConfig *rest.Config\n\trestClient poster\n\n\tstreamExecutorFactory streamExecutorFactory\n}\n\n// NewPodCommandExecutor creates a new PodCommandExecutor.\nfunc NewPodCommandExecutor(restClientConfig *rest.Config, restClient poster) PodCommandExecutor {\n\treturn &defaultPodCommandExecutor{\n\t\trestClientConfig: restClientConfig,\n\t\trestClient: restClient,\n\n\t\tstreamExecutorFactory: &defaultStreamExecutorFactory{},\n\t}\n}\n\n// ExecutePodCommand uses the pod exec API to execute a command in a container in a pod. If the\n// command takes longer than the specified timeout, an error is returned (NOTE: it is not currently\n// possible to ensure the command is terminated when the timeout occurs, so it may continue to run\n// in the background).\nfunc (e *defaultPodCommandExecutor) ExecutePodCommand(log logrus.FieldLogger, item map[string]any, namespace, name, hookName string, hook *api.ExecHook) error {\n\tif item == nil {\n\t\treturn errors.New(\"item is required\")\n\t}\n\tif namespace == \"\" {\n\t\treturn errors.New(\"namespace is required\")\n\t}\n\tif name == \"\" {\n\t\treturn errors.New(\"name is required\")\n\t}\n\tif hookName == \"\" {\n\t\treturn errors.New(\"hookName is required\")\n\t}\n\tif hook == nil {\n\t\treturn errors.New(\"hook is required\")\n\t}\n\n\tlocalHook := *hook\n\n\tpod := new(corev1api.Pod)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(item, pod); err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\tif localHook.Container == \"\" {\n\t\tif err := setDefaultHookContainer(pod, &localHook); err != nil {\n\t\t\treturn err\n\t\t}\n\t} else if err := ensureContainerExists(pod, localHook.Container); err != nil {\n\t\treturn err\n\t}\n\n\tif len(localHook.Command) == 0 {\n\t\treturn errors.New(\"command is required\")\n\t}\n\n\tswitch localHook.OnError {\n\tcase api.HookErrorModeFail, api.HookErrorModeContinue:\n\t\t// use the specified value\n\tdefault:\n\t\t// default to fail\n\t\tlocalHook.OnError = api.HookErrorModeFail\n\t}\n\n\tif localHook.Timeout.Duration == 0 {\n\t\tlocalHook.Timeout.Duration = defaultTimeout\n\t}\n\n\thookLog := log.WithFields(\n\t\tlogrus.Fields{\n\t\t\t\"hookName\": hookName,\n\t\t\t\"hookContainer\": localHook.Container,\n\t\t\t\"hookCommand\": localHook.Command,\n\t\t\t\"hookOnError\": localHook.OnError,\n\t\t\t\"hookTimeout\": localHook.Timeout,\n\t\t},\n\t)\n\n\tif pod.Status.Phase == corev1api.PodSucceeded || pod.Status.Phase == corev1api.PodFailed {\n\t\thookLog.Infof(\"Pod entered phase %s before some post-backup exec hooks ran\", pod.Status.Phase)\n\t\treturn nil\n\t}\n\n\thookLog.Info(\"running exec hook\")\n\n\treq := e.restClient.Post().\n\t\tResource(\"pods\").\n\t\tNamespace(namespace).\n\t\tName(name).\n\t\tSubResource(\"exec\")\n\n\treq.VersionedParams(&corev1api.PodExecOptions{\n\t\tContainer: localHook.Container,\n\t\tCommand: localHook.Command,\n\t\tStdout: true,\n\t\tStderr: true,\n\t}, kscheme.ParameterCodec)\n\n\texecutor, err := e.streamExecutorFactory.NewSPDYExecutor(e.restClientConfig, \"POST\", req.URL())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tvar stdout, stderr bytes.Buffer\n\n\tstreamOptions := remotecommand.StreamOptions{\n\t\tStdout: &stdout,\n\t\tStderr: &stderr,\n\t}\n\n\terrCh := make(chan error)\n\n\tgo func() {\n\t\terr = executor.StreamWithContext(context.Background(), streamOptions)\n\t\terrCh <- err\n\t}()\n\n\tvar timeoutCh <-chan time.Time\n\tif localHook.Timeout.Duration > 0 {\n\t\ttimer := time.NewTimer(localHook.Timeout.Duration)\n\t\tdefer timer.Stop()\n\t\ttimeoutCh = timer.C\n\t}\n\n\tselect {\n\tcase err = <-errCh:\n\tcase <-timeoutCh:\n\t\treturn errors.Errorf(\"timed out after %v\", localHook.Timeout.Duration)\n\t}\n\n\thookLog.Infof(\"stdout: %s\", stdout.String())\n\thookLog.Infof(\"stderr: %s\", stderr.String())\n\n\treturn err\n}\n\nfunc ensureContainerExists(pod *corev1api.Pod, container string) error {\n\tfor _, c := range pod.Spec.Containers {\n\t\tif c.Name == container {\n\t\t\treturn nil\n\t\t}\n\t}\n\n\treturn errors.Errorf(\"no such container: %q\", container)\n}\n\nfunc setDefaultHookContainer(pod *corev1api.Pod, hook *api.ExecHook) error {\n\tif len(pod.Spec.Containers) < 1 {\n\t\treturn errors.New(\"need at least 1 container\")\n\t}\n\n\thook.Container = pod.Spec.Containers[0].Name\n\n\treturn nil\n}\n\ntype streamExecutorFactory interface {\n\tNewSPDYExecutor(config *rest.Config, method string, url *url.URL) (remotecommand.Executor, error)\n}\n\ntype defaultStreamExecutorFactory struct{}\n\nfunc (f *defaultStreamExecutorFactory) NewSPDYExecutor(config *rest.Config, method string, url *url.URL) (remotecommand.Executor, error) {\n\treturn remotecommand.NewSPDYExecutor(config, method, url)\n}\n" }, { "path": "pkg/podexec/pod_command_executor_test.go", "content": "/*\nCopyright 2017, 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podexec\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"net/url\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/client-go/rest\"\n\t\"k8s.io/client-go/tools/remotecommand\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestNewPodCommandExecutor(t *testing.T) {\n\trestClientConfig := &rest.Config{Host: \"foo\"}\n\tposter := &mockPoster{}\n\tpce := NewPodCommandExecutor(restClientConfig, poster).(*defaultPodCommandExecutor)\n\tassert.Equal(t, restClientConfig, pce.restClientConfig)\n\tassert.Equal(t, poster, pce.restClient)\n\tassert.Equal(t, &defaultStreamExecutorFactory{}, pce.streamExecutorFactory)\n}\n\nfunc TestExecutePodCommandMissingInputs(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\titem map[string]any\n\t\tpodNamespace string\n\t\tpodName string\n\t\thookName string\n\t\thook *v1.ExecHook\n\t}{\n\t\t{\n\t\t\tname: \"missing item\",\n\t\t},\n\t\t{\n\t\t\tname: \"missing pod namespace\",\n\t\t\titem: map[string]any{},\n\t\t},\n\t\t{\n\t\t\tname: \"missing pod name\",\n\t\t\titem: map[string]any{},\n\t\t\tpodNamespace: \"ns\",\n\t\t},\n\t\t{\n\t\t\tname: \"missing hookName\",\n\t\t\titem: map[string]any{},\n\t\t\tpodNamespace: \"ns\",\n\t\t\tpodName: \"pod\",\n\t\t},\n\t\t{\n\t\t\tname: \"missing hook\",\n\t\t\titem: map[string]any{},\n\t\t\tpodNamespace: \"ns\",\n\t\t\tpodName: \"pod\",\n\t\t\thookName: \"hook\",\n\t\t},\n\t\t{\n\t\t\tname: \"container not found\",\n\t\t\titem: velerotest.UnstructuredOrDie(`{\"kind\":\"Pod\",\"spec\":{\"containers\":[{\"name\":\"foo\"}]}}`).Object,\n\t\t\tpodNamespace: \"ns\",\n\t\t\tpodName: \"pod\",\n\t\t\thookName: \"hook\",\n\t\t\thook: &v1.ExecHook{\n\t\t\t\tContainer: \"missing\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"command missing\",\n\t\t\titem: velerotest.UnstructuredOrDie(`{\"kind\":\"Pod\",\"spec\":{\"containers\":[{\"name\":\"foo\"}]}}`).Object,\n\t\t\tpodNamespace: \"ns\",\n\t\t\tpodName: \"pod\",\n\t\t\thookName: \"hook\",\n\t\t\thook: &v1.ExecHook{\n\t\t\t\tContainer: \"foo\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"hook's container is not overwritten by pod\",\n\t\t\titem: velerotest.UnstructuredOrDie(`{\"kind\":\"Pod\",\"spec\":{\"containers\":[{\"name\":\"foo\"}]}}`).Object,\n\t\t\tpodNamespace: \"ns\",\n\t\t\tpodName: \"pod\",\n\t\t\thookName: \"hook\",\n\t\t\thook: &v1.ExecHook{\n\t\t\t\tContainer: \"\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tpod := new(corev1api.Pod)\n\t\t\thookPodContainerNotSame := false\n\t\t\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(test.item, pod); err != nil {\n\t\t\t\tassert.Error(t, err)\n\t\t\t}\n\t\t\tif (len(pod.Spec.Containers) > 0) && (pod.Spec.Containers[0].Name != test.hook.Container) {\n\t\t\t\thookPodContainerNotSame = true\n\t\t\t}\n\n\t\t\te := &defaultPodCommandExecutor{}\n\t\t\terr := e.ExecutePodCommand(velerotest.NewLogger(), test.item, test.podNamespace, test.podName, test.hookName, test.hook)\n\n\t\t\tif hookPodContainerNotSame && test.hook.Container == pod.Spec.Containers[0].Name {\n\t\t\t\trequire.Error(t, fmt.Errorf(\"hook exec container is overwritten\"))\n\t\t\t}\n\t\t\tassert.Error(t, err)\n\t\t})\n\t}\n}\n\nfunc TestExecutePodCommand(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tcontainerName string\n\t\texpectedContainerName string\n\t\tcommand []string\n\t\terrorMode v1.HookErrorMode\n\t\texpectedErrorMode v1.HookErrorMode\n\t\ttimeout time.Duration\n\t\texpectedTimeout time.Duration\n\t\thookError error\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"validate defaults\",\n\t\t\tcommand: []string{\"some\", \"command\"},\n\t\t\texpectedContainerName: \"foo\",\n\t\t\texpectedErrorMode: v1.HookErrorModeFail,\n\t\t\texpectedTimeout: 30 * time.Second,\n\t\t},\n\t\t{\n\t\t\tname: \"use specified values\",\n\t\t\tcommand: []string{\"some\", \"command\"},\n\t\t\tcontainerName: \"bar\",\n\t\t\texpectedContainerName: \"bar\",\n\t\t\terrorMode: v1.HookErrorModeContinue,\n\t\t\texpectedErrorMode: v1.HookErrorModeContinue,\n\t\t\ttimeout: 10 * time.Second,\n\t\t\texpectedTimeout: 10 * time.Second,\n\t\t},\n\t\t{\n\t\t\tname: \"hook error\",\n\t\t\tcommand: []string{\"some\", \"command\"},\n\t\t\texpectedContainerName: \"foo\",\n\t\t\texpectedErrorMode: v1.HookErrorModeFail,\n\t\t\texpectedTimeout: 30 * time.Second,\n\t\t\thookError: errors.New(\"hook error\"),\n\t\t\texpectedError: \"hook error\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\thook := v1.ExecHook{\n\t\t\t\tContainer: test.containerName,\n\t\t\t\tCommand: test.command,\n\t\t\t\tOnError: test.errorMode,\n\t\t\t\tTimeout: metav1.Duration{Duration: test.timeout},\n\t\t\t}\n\n\t\t\tpod, err := velerotest.GetAsMap(`\n{\n\t\"metadata\": {\n\t\t\"namespace\": \"namespace\",\n\t\t\"name\": \"name\"\n\t},\n\t\"spec\": {\n\t\t\"containers\": [\n\t\t\t{\"name\": \"foo\"},\n\t\t\t{\"name\": \"bar\"}\n\t\t]\n\t}\n}`)\n\n\t\t\trequire.NoError(t, err)\n\n\t\t\tclientConfig := &rest.Config{}\n\t\t\tposter := &mockPoster{}\n\t\t\tdefer poster.AssertExpectations(t)\n\t\t\tpodCommandExecutor := NewPodCommandExecutor(clientConfig, poster).(*defaultPodCommandExecutor)\n\n\t\t\tstreamExecutorFactory := &mockStreamExecutorFactory{}\n\t\t\tdefer streamExecutorFactory.AssertExpectations(t)\n\t\t\tpodCommandExecutor.streamExecutorFactory = streamExecutorFactory\n\n\t\t\tbaseURL, _ := url.Parse(\"https://some.server\")\n\t\t\tcontentConfig := rest.ClientContentConfig{\n\t\t\t\tGroupVersion: schema.GroupVersion{Group: \"\", Version: \"v1\"},\n\t\t\t}\n\t\t\tposter.On(\"Post\").Return(rest.NewRequestWithClient(baseURL, \"/api/v1\", contentConfig, nil))\n\n\t\t\tstreamExecutor := &mockStreamExecutor{}\n\t\t\tdefer streamExecutor.AssertExpectations(t)\n\n\t\t\texpectedCommand := strings.Join(test.command, \"&command=\")\n\t\t\texpectedURL, _ := url.Parse(\n\t\t\t\tfmt.Sprintf(\"https://some.server/api/v1/namespaces/namespace/pods/name/exec?command=%s&container=%s&stderr=true&stdout=true\", expectedCommand, test.expectedContainerName),\n\t\t\t)\n\t\t\tstreamExecutorFactory.On(\"NewSPDYExecutor\", clientConfig, \"POST\", expectedURL).Return(streamExecutor, nil)\n\n\t\t\tvar stdout, stderr bytes.Buffer\n\t\t\texpectedStreamOptions := remotecommand.StreamOptions{\n\t\t\t\tStdout: &stdout,\n\t\t\t\tStderr: &stderr,\n\t\t\t}\n\t\t\tstreamExecutor.On(\"StreamWithContext\", mock.Anything, expectedStreamOptions).Return(test.hookError)\n\n\t\t\terr = podCommandExecutor.ExecutePodCommand(velerotest.NewLogger(), pod, \"namespace\", \"name\", \"hookName\", &hook)\n\t\t\tif test.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.expectedError)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\trequire.NoError(t, err)\n\t\t})\n\t}\n}\n\nfunc TestEnsureContainerExists(t *testing.T) {\n\tpod := &corev1api.Pod{\n\t\tSpec: corev1api.PodSpec{\n\t\t\tContainers: []corev1api.Container{\n\t\t\t\t{\n\t\t\t\t\tName: \"foo\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\terr := ensureContainerExists(pod, \"bar\")\n\trequire.EqualError(t, err, `no such container: \"bar\"`)\n\n\terr = ensureContainerExists(pod, \"foo\")\n\tassert.NoError(t, err)\n}\n\nfunc TestPodCompeted(t *testing.T) {\n\tpod := &corev1api.Pod{\n\t\tSpec: corev1api.PodSpec{\n\t\t\tContainers: []corev1api.Container{\n\t\t\t\t{\n\t\t\t\t\tName: \"foo\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\tStatus: corev1api.PodStatus{\n\t\t\tPhase: corev1api.PodSucceeded,\n\t\t},\n\t}\n\n\tobj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(pod)\n\trequire.NoError(t, err)\n\n\tclientConfig := &rest.Config{}\n\tposter := &mockPoster{}\n\tdefer poster.AssertExpectations(t)\n\tpodCommandExecutor := NewPodCommandExecutor(clientConfig, poster).(*defaultPodCommandExecutor)\n\n\thook := v1.ExecHook{\n\t\tContainer: \"foo\",\n\t\tCommand: []string{\"some\", \"command\"},\n\t}\n\n\terr = podCommandExecutor.ExecutePodCommand(velerotest.NewLogger(), obj, \"namespace\", \"name\", \"hookName\", &hook)\n\trequire.NoError(t, err)\n}\n\ntype mockStreamExecutorFactory struct {\n\tmock.Mock\n}\n\nfunc (f *mockStreamExecutorFactory) NewSPDYExecutor(config *rest.Config, method string, url *url.URL) (remotecommand.Executor, error) {\n\targs := f.Called(config, method, url)\n\treturn args.Get(0).(remotecommand.Executor), args.Error(1)\n}\n\ntype mockStreamExecutor struct {\n\tmock.Mock\n\tremotecommand.Executor\n}\n\nfunc (e *mockStreamExecutor) StreamWithContext(ctx context.Context, options remotecommand.StreamOptions) error {\n\targs := e.Called(ctx, options)\n\treturn args.Error(0)\n}\n\ntype mockPoster struct {\n\tmock.Mock\n}\n\nfunc (p *mockPoster) Post() *rest.Request {\n\targs := p.Called()\n\treturn args.Get(0).(*rest.Request)\n}\n" }, { "path": "pkg/podvolume/backup_micro_service.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tcachetool \"k8s.io/client-go/tools/cache\"\n\t\"sigs.k8s.io/controller-runtime/pkg/cache\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n)\n\nconst (\n\tpodVolumeRequestor = \"snapshot-pod-volume\"\n)\n\n// BackupMicroService process data mover backups inside the backup pod\ntype BackupMicroService struct {\n\tctx context.Context\n\tclient client.Client\n\tkubeClient kubernetes.Interface\n\trepoEnsurer *repository.Ensurer\n\tcredentialGetter *credentials.CredentialGetter\n\tlogger logrus.FieldLogger\n\tdataPathMgr *datapath.Manager\n\teventRecorder kube.EventRecorder\n\n\tnamespace string\n\tpvbName string\n\tpvb *velerov1api.PodVolumeBackup\n\tsourceTargetPath datapath.AccessPoint\n\n\tresultSignal chan dataPathResult\n\n\tpvbInformer cache.Informer\n\tpvbHandler cachetool.ResourceEventHandlerRegistration\n\tnodeName string\n}\n\ntype dataPathResult struct {\n\terr error\n\tresult string\n}\n\nfunc NewBackupMicroService(ctx context.Context, client client.Client, kubeClient kubernetes.Interface, pvbName string, namespace string, nodeName string,\n\tsourceTargetPath datapath.AccessPoint, dataPathMgr *datapath.Manager, repoEnsurer *repository.Ensurer, cred *credentials.CredentialGetter,\n\tpvbInformer cache.Informer, log logrus.FieldLogger) *BackupMicroService {\n\treturn &BackupMicroService{\n\t\tctx: ctx,\n\t\tclient: client,\n\t\tkubeClient: kubeClient,\n\t\tcredentialGetter: cred,\n\t\tlogger: log,\n\t\trepoEnsurer: repoEnsurer,\n\t\tdataPathMgr: dataPathMgr,\n\t\tnamespace: namespace,\n\t\tpvbName: pvbName,\n\t\tsourceTargetPath: sourceTargetPath,\n\t\tnodeName: nodeName,\n\t\tresultSignal: make(chan dataPathResult),\n\t\tpvbInformer: pvbInformer,\n\t}\n}\n\nfunc (r *BackupMicroService) Init() error {\n\tr.eventRecorder = kube.NewEventRecorder(r.kubeClient, r.client.Scheme(), r.pvbName, r.nodeName, r.logger)\n\n\thandler, err := r.pvbInformer.AddEventHandler(\n\t\tcachetool.ResourceEventHandlerFuncs{\n\t\t\tUpdateFunc: func(oldObj any, newObj any) {\n\t\t\t\toldPvb := oldObj.(*velerov1api.PodVolumeBackup)\n\t\t\t\tnewPvb := newObj.(*velerov1api.PodVolumeBackup)\n\n\t\t\t\tif newPvb.Name != r.pvbName {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif newPvb.Status.Phase != velerov1api.PodVolumeBackupPhaseInProgress {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif newPvb.Spec.Cancel && !oldPvb.Spec.Cancel {\n\t\t\t\t\tr.cancelPodVolumeBackup(newPvb)\n\t\t\t\t}\n\t\t\t},\n\t\t},\n\t)\n\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error adding PVB handler\")\n\t}\n\n\tr.pvbHandler = handler\n\n\treturn err\n}\n\nfunc (r *BackupMicroService) RunCancelableDataPath(ctx context.Context) (string, error) {\n\tlog := r.logger.WithFields(logrus.Fields{\n\t\t\"PVB\": r.pvbName,\n\t})\n\n\tpvb := &velerov1api.PodVolumeBackup{}\n\terr := wait.PollUntilContextCancel(ctx, 500*time.Millisecond, true, func(ctx context.Context) (bool, error) {\n\t\terr := r.client.Get(ctx, types.NamespacedName{\n\t\t\tNamespace: r.namespace,\n\t\t\tName: r.pvbName,\n\t\t}, pvb)\n\n\t\tif apierrors.IsNotFound(err) {\n\t\t\treturn false, nil\n\t\t}\n\n\t\tif err != nil {\n\t\t\treturn true, errors.Wrapf(err, \"error to get PVB %s\", r.pvbName)\n\t\t}\n\n\t\tif pvb.Status.Phase == velerov1api.PodVolumeBackupPhaseInProgress {\n\t\t\treturn true, nil\n\t\t} else {\n\t\t\treturn false, nil\n\t\t}\n\t})\n\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"Failed to wait PVB\")\n\t\treturn \"\", errors.Wrap(err, \"error waiting for PVB\")\n\t}\n\n\tr.pvb = pvb\n\n\tlog.Info(\"Run cancelable PVB\")\n\n\tcallbacks := datapath.Callbacks{\n\t\tOnCompleted: r.OnDataPathCompleted,\n\t\tOnFailed: r.OnDataPathFailed,\n\t\tOnCancelled: r.OnDataPathCancelled,\n\t\tOnProgress: r.OnDataPathProgress,\n\t}\n\n\tfsBackup, err := r.dataPathMgr.CreateFileSystemBR(pvb.Name, podVolumeRequestor, ctx, r.client, pvb.Namespace, callbacks, log)\n\tif err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error to create data path\")\n\t}\n\n\tlog.Debug(\"Async fs br created\")\n\n\tif err := fsBackup.Init(ctx, &datapath.FSBRInitParam{\n\t\tBSLName: pvb.Spec.BackupStorageLocation,\n\t\tSourceNamespace: pvb.Spec.Pod.Namespace,\n\t\tUploaderType: pvb.Spec.UploaderType,\n\t\tRepositoryType: velerov1api.BackupRepositoryTypeKopia,\n\t\tRepoIdentifier: \"\",\n\t\tRepositoryEnsurer: r.repoEnsurer,\n\t\tCredentialGetter: r.credentialGetter,\n\t}); err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error to initialize data path\")\n\t}\n\n\tlog.Info(\"Async fs br init\")\n\n\ttags := map[string]string{}\n\n\tif err := fsBackup.StartBackup(r.sourceTargetPath, pvb.Spec.UploaderSettings, &datapath.FSBRStartParam{\n\t\tRealSource: GetRealSource(pvb),\n\t\tParentSnapshot: \"\",\n\t\tForceFull: false,\n\t\tTags: tags,\n\t}); err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error starting data path backup\")\n\t}\n\n\tlog.Info(\"Async fs backup data path started\")\n\tr.eventRecorder.Event(pvb, false, datapath.EventReasonStarted, \"Data path for %s started\", pvb.Name)\n\n\tresult := \"\"\n\tselect {\n\tcase <-ctx.Done():\n\t\terr = errors.New(\"timed out waiting for fs backup to complete\")\n\t\tbreak\n\tcase res := <-r.resultSignal:\n\t\terr = res.err\n\t\tresult = res.result\n\t\tbreak\n\t}\n\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"Async fs backup was not completed\")\n\t}\n\n\tr.eventRecorder.EndingEvent(pvb, false, datapath.EventReasonStopped, \"Data path for %s stopped\", pvb.Name)\n\n\treturn result, err\n}\n\nfunc (r *BackupMicroService) Shutdown() {\n\tr.eventRecorder.Shutdown()\n\tr.closeDataPath(r.ctx, r.pvbName)\n\n\tif r.pvbHandler != nil {\n\t\tif err := r.pvbInformer.RemoveEventHandler(r.pvbHandler); err != nil {\n\t\t\tr.logger.WithError(err).Warn(\"Failed to remove pod handler\")\n\t\t}\n\t}\n}\n\nvar funcMarshal = json.Marshal\n\nfunc (r *BackupMicroService) OnDataPathCompleted(ctx context.Context, namespace string, pvbName string, result datapath.Result) {\n\tlog := r.logger.WithField(\"PVB\", pvbName)\n\n\tbackupBytes, err := funcMarshal(result.Backup)\n\tif err != nil {\n\t\tlog.WithError(err).Errorf(\"Failed to marshal backup result %v\", result.Backup)\n\t\tr.resultSignal <- dataPathResult{\n\t\t\terr: errors.Wrapf(err, \"Failed to marshal backup result %v\", result.Backup),\n\t\t}\n\t} else {\n\t\tr.eventRecorder.Event(r.pvb, false, datapath.EventReasonCompleted, string(backupBytes))\n\t\tr.resultSignal <- dataPathResult{\n\t\t\tresult: string(backupBytes),\n\t\t}\n\t}\n\n\tlog.Info(\"Async fs backup completed\")\n}\n\nfunc (r *BackupMicroService) OnDataPathFailed(ctx context.Context, namespace string, pvbName string, err error) {\n\tlog := r.logger.WithField(\"PVB\", pvbName)\n\tlog.WithError(err).Error(\"Async fs backup data path failed\")\n\n\tr.eventRecorder.Event(r.pvb, false, datapath.EventReasonFailed, \"Data path for PVB %s failed, error %v\", r.pvbName, err)\n\tr.resultSignal <- dataPathResult{\n\t\terr: errors.Wrapf(err, \"Data path for PVB %s failed\", r.pvbName),\n\t}\n}\n\nfunc (r *BackupMicroService) OnDataPathCancelled(ctx context.Context, namespace string, pvbName string) {\n\tlog := r.logger.WithField(\"PVB\", pvbName)\n\tlog.Warn(\"Async fs backup data path canceled\")\n\n\tr.eventRecorder.Event(r.pvb, false, datapath.EventReasonCancelled, \"Data path for PVB %s canceled\", pvbName)\n\tr.resultSignal <- dataPathResult{\n\t\terr: errors.New(datapath.ErrCancelled),\n\t}\n}\n\nfunc (r *BackupMicroService) OnDataPathProgress(ctx context.Context, namespace string, pvbName string, progress *uploader.Progress) {\n\tlog := r.logger.WithFields(logrus.Fields{\n\t\t\"PVB\": pvbName,\n\t})\n\n\tprogressBytes, err := funcMarshal(progress)\n\tif err != nil {\n\t\tlog.WithError(err).Errorf(\"Failed to marshal progress %v\", progress)\n\t\treturn\n\t}\n\n\tr.eventRecorder.Event(r.pvb, false, datapath.EventReasonProgress, string(progressBytes))\n}\n\nfunc (r *BackupMicroService) closeDataPath(ctx context.Context, duName string) {\n\tfsBackup := r.dataPathMgr.GetAsyncBR(duName)\n\tif fsBackup != nil {\n\t\tfsBackup.Close(ctx)\n\t}\n\n\tr.dataPathMgr.RemoveAsyncBR(duName)\n}\n\nfunc (r *BackupMicroService) cancelPodVolumeBackup(pvb *velerov1api.PodVolumeBackup) {\n\tr.logger.WithField(\"PVB\", pvb.Name).Info(\"PVB is being canceled\")\n\n\tr.eventRecorder.Event(pvb, false, datapath.EventReasonCancelling, \"Canceling for PVB %s\", pvb.Name)\n\n\tfsBackup := r.dataPathMgr.GetAsyncBR(pvb.Name)\n\tif fsBackup == nil {\n\t\tr.OnDataPathCancelled(r.ctx, pvb.GetNamespace(), pvb.GetName())\n\t} else {\n\t\tfsBackup.Cancel()\n\t}\n}\n" }, { "path": "pkg/podvolume/backup_micro_service_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"sync\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\n\tclientFake \"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tdatapathmockes \"github.com/vmware-tanzu/velero/pkg/datapath/mocks\"\n)\n\ntype backupMsTestHelper struct {\n\teventReason string\n\teventMsg string\n\tmarshalErr error\n\tmarshalBytes []byte\n\twithEvent bool\n\teventLock sync.Mutex\n}\n\nfunc (bt *backupMsTestHelper) Event(_ runtime.Object, _ bool, reason string, message string, a ...any) {\n\tbt.eventLock.Lock()\n\tdefer bt.eventLock.Unlock()\n\n\tbt.withEvent = true\n\tbt.eventReason = reason\n\tbt.eventMsg = fmt.Sprintf(message, a...)\n}\n\nfunc (bt *backupMsTestHelper) EndingEvent(_ runtime.Object, _ bool, reason string, message string, a ...any) {\n\tbt.eventLock.Lock()\n\tdefer bt.eventLock.Unlock()\n\n\tbt.withEvent = true\n\tbt.eventReason = reason\n\tbt.eventMsg = fmt.Sprintf(message, a...)\n}\nfunc (bt *backupMsTestHelper) Shutdown() {}\n\nfunc (bt *backupMsTestHelper) Marshal(v any) ([]byte, error) {\n\tif bt.marshalErr != nil {\n\t\treturn nil, bt.marshalErr\n\t}\n\n\treturn bt.marshalBytes, nil\n}\n\nfunc (bt *backupMsTestHelper) EventReason() string {\n\tbt.eventLock.Lock()\n\tdefer bt.eventLock.Unlock()\n\n\treturn bt.eventReason\n}\n\nfunc (bt *backupMsTestHelper) EventMessage() string {\n\tbt.eventLock.Lock()\n\tdefer bt.eventLock.Unlock()\n\n\treturn bt.eventMsg\n}\n\nfunc TestOnDataPathFailed(t *testing.T) {\n\tpvbName := \"fake-pvb\"\n\tbt := &backupMsTestHelper{}\n\n\tbs := &BackupMicroService{\n\t\tpvbName: pvbName,\n\t\tdataPathMgr: datapath.NewManager(1),\n\t\teventRecorder: bt,\n\t\tresultSignal: make(chan dataPathResult),\n\t\tlogger: velerotest.NewLogger(),\n\t}\n\n\texpectedErr := \"Data path for PVB fake-pvb failed: fake-error\"\n\texpectedEventReason := datapath.EventReasonFailed\n\texpectedEventMsg := \"Data path for PVB fake-pvb failed, error fake-error\"\n\n\tgo bs.OnDataPathFailed(t.Context(), velerov1api.DefaultNamespace, pvbName, errors.New(\"fake-error\"))\n\n\tresult := <-bs.resultSignal\n\trequire.EqualError(t, result.err, expectedErr)\n\tassert.Equal(t, expectedEventReason, bt.EventReason())\n\tassert.Equal(t, expectedEventMsg, bt.EventMessage())\n}\n\nfunc TestOnDataPathCancelled(t *testing.T) {\n\tpvbName := \"fake-pvb\"\n\tbt := &backupMsTestHelper{}\n\n\tbs := &BackupMicroService{\n\t\tpvbName: pvbName,\n\t\tdataPathMgr: datapath.NewManager(1),\n\t\teventRecorder: bt,\n\t\tresultSignal: make(chan dataPathResult),\n\t\tlogger: velerotest.NewLogger(),\n\t}\n\n\texpectedErr := datapath.ErrCancelled\n\texpectedEventReason := datapath.EventReasonCancelled\n\texpectedEventMsg := \"Data path for PVB fake-pvb canceled\"\n\n\tgo bs.OnDataPathCancelled(t.Context(), velerov1api.DefaultNamespace, pvbName)\n\n\tresult := <-bs.resultSignal\n\trequire.EqualError(t, result.err, expectedErr)\n\tassert.Equal(t, expectedEventReason, bt.EventReason())\n\tassert.Equal(t, expectedEventMsg, bt.EventMessage())\n}\n\nfunc TestOnDataPathCompleted(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\texpectedErr string\n\t\texpectedEventReason string\n\t\texpectedEventMsg string\n\t\tmarshalErr error\n\t\tmarshallStr string\n\t}{\n\t\t{\n\t\t\tname: \"marshal fail\",\n\t\t\tmarshalErr: errors.New(\"fake-marshal-error\"),\n\t\t\texpectedErr: \"Failed to marshal backup result { false { } 0 0}: fake-marshal-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tmarshallStr: \"fake-complete-string\",\n\t\t\texpectedEventReason: datapath.EventReasonCompleted,\n\t\t\texpectedEventMsg: \"fake-complete-string\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tpvbName := \"fake-pvb\"\n\n\t\t\tbt := &backupMsTestHelper{\n\t\t\t\tmarshalErr: test.marshalErr,\n\t\t\t\tmarshalBytes: []byte(test.marshallStr),\n\t\t\t}\n\n\t\t\tbs := &BackupMicroService{\n\t\t\t\tdataPathMgr: datapath.NewManager(1),\n\t\t\t\teventRecorder: bt,\n\t\t\t\tresultSignal: make(chan dataPathResult),\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tfuncMarshal = bt.Marshal\n\n\t\t\tgo bs.OnDataPathCompleted(t.Context(), velerov1api.DefaultNamespace, pvbName, datapath.Result{})\n\n\t\t\tresult := <-bs.resultSignal\n\t\t\tif test.marshalErr != nil {\n\t\t\t\tassert.EqualError(t, result.err, test.expectedErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, result.err)\n\t\t\t\tassert.Equal(t, test.expectedEventReason, bt.EventReason())\n\t\t\t\tassert.Equal(t, test.expectedEventMsg, bt.EventMessage())\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestOnDataPathProgress(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\texpectedErr string\n\t\texpectedEventReason string\n\t\texpectedEventMsg string\n\t\tmarshalErr error\n\t\tmarshallStr string\n\t}{\n\t\t{\n\t\t\tname: \"marshal fail\",\n\t\t\tmarshalErr: errors.New(\"fake-marshal-error\"),\n\t\t\texpectedErr: \"Failed to marshal backup result\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tmarshallStr: \"fake-progress-string\",\n\t\t\texpectedEventReason: datapath.EventReasonProgress,\n\t\t\texpectedEventMsg: \"fake-progress-string\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tpvbName := \"fake-pvb\"\n\n\t\t\tbt := &backupMsTestHelper{\n\t\t\t\tmarshalErr: test.marshalErr,\n\t\t\t\tmarshalBytes: []byte(test.marshallStr),\n\t\t\t}\n\n\t\t\tbs := &BackupMicroService{\n\t\t\t\tdataPathMgr: datapath.NewManager(1),\n\t\t\t\teventRecorder: bt,\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tfuncMarshal = bt.Marshal\n\n\t\t\tbs.OnDataPathProgress(t.Context(), velerov1api.DefaultNamespace, pvbName, &uploader.Progress{})\n\n\t\t\tif test.marshalErr != nil {\n\t\t\t\tassert.False(t, bt.withEvent)\n\t\t\t} else {\n\t\t\t\tassert.True(t, bt.withEvent)\n\t\t\t\tassert.Equal(t, test.expectedEventReason, bt.EventReason())\n\t\t\t\tassert.Equal(t, test.expectedEventMsg, bt.EventMessage())\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestCancelPodVolumeBackup(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\texpectedEventReason string\n\t\texpectedEventMsg string\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"no fs backup\",\n\t\t\texpectedEventReason: datapath.EventReasonCancelled,\n\t\t\texpectedEventMsg: \"Data path for PVB fake-pvb canceled\",\n\t\t\texpectedErr: datapath.ErrCancelled,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tpvbName := \"fake-pvb\"\n\t\t\tpvb := builder.ForPodVolumeBackup(velerov1api.DefaultNamespace, pvbName).Result()\n\n\t\t\tbt := &backupMsTestHelper{}\n\n\t\t\tbs := &BackupMicroService{\n\t\t\t\tdataPathMgr: datapath.NewManager(1),\n\t\t\t\teventRecorder: bt,\n\t\t\t\tresultSignal: make(chan dataPathResult),\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tgo bs.cancelPodVolumeBackup(pvb)\n\n\t\t\tresult := <-bs.resultSignal\n\n\t\t\trequire.EqualError(t, result.err, test.expectedErr)\n\t\t\tassert.True(t, bt.withEvent)\n\t\t\tassert.Equal(t, test.expectedEventReason, bt.EventReason())\n\t\t\tassert.Equal(t, test.expectedEventMsg, bt.EventMessage())\n\t\t})\n\t}\n}\n\nfunc TestRunCancelableDataPath(t *testing.T) {\n\tpvbName := \"fake-pvb\"\n\tpvb := builder.ForPodVolumeBackup(velerov1api.DefaultNamespace, pvbName).Phase(velerov1api.PodVolumeBackupPhaseNew).Result()\n\tpvbInProgress := builder.ForPodVolumeBackup(velerov1api.DefaultNamespace, pvbName).Phase(velerov1api.PodVolumeBackupPhaseInProgress).Result()\n\tctxTimeout, cancel := context.WithTimeout(t.Context(), time.Second)\n\n\ttests := []struct {\n\t\tname string\n\t\tctx context.Context\n\t\tresult *dataPathResult\n\t\tdataPathMgr *datapath.Manager\n\t\tkubeClientObj []runtime.Object\n\t\tinitErr error\n\t\tstartErr error\n\t\tdataPathStarted bool\n\t\texpectedEventMsg string\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"no pvb\",\n\t\t\tctx: ctxTimeout,\n\t\t\texpectedErr: \"error waiting for PVB: context deadline exceeded\",\n\t\t},\n\t\t{\n\t\t\tname: \"pvb not in in-progress\",\n\t\t\tctx: ctxTimeout,\n\t\t\tkubeClientObj: []runtime.Object{pvb},\n\t\t\texpectedErr: \"error waiting for PVB: context deadline exceeded\",\n\t\t},\n\t\t{\n\t\t\tname: \"create data path fail\",\n\t\t\tctx: t.Context(),\n\t\t\tkubeClientObj: []runtime.Object{pvbInProgress},\n\t\t\tdataPathMgr: datapath.NewManager(0),\n\t\t\texpectedErr: \"error to create data path: Concurrent number exceeds\",\n\t\t},\n\t\t{\n\t\t\tname: \"init data path fail\",\n\t\t\tctx: t.Context(),\n\t\t\tkubeClientObj: []runtime.Object{pvbInProgress},\n\t\t\tinitErr: errors.New(\"fake-init-error\"),\n\t\t\texpectedErr: \"error to initialize data path: fake-init-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"start data path fail\",\n\t\t\tctx: t.Context(),\n\t\t\tkubeClientObj: []runtime.Object{pvbInProgress},\n\t\t\tstartErr: errors.New(\"fake-start-error\"),\n\t\t\texpectedErr: \"error starting data path backup: fake-start-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"data path timeout\",\n\t\t\tctx: ctxTimeout,\n\t\t\tkubeClientObj: []runtime.Object{pvbInProgress},\n\t\t\tdataPathStarted: true,\n\t\t\texpectedEventMsg: fmt.Sprintf(\"Data path for %s stopped\", pvbName),\n\t\t\texpectedErr: \"timed out waiting for fs backup to complete\",\n\t\t},\n\t\t{\n\t\t\tname: \"data path returns error\",\n\t\t\tctx: t.Context(),\n\t\t\tkubeClientObj: []runtime.Object{pvbInProgress},\n\t\t\tdataPathStarted: true,\n\t\t\tresult: &dataPathResult{\n\t\t\t\terr: errors.New(\"fake-data-path-error\"),\n\t\t\t},\n\t\t\texpectedEventMsg: fmt.Sprintf(\"Data path for %s stopped\", pvbName),\n\t\t\texpectedErr: \"fake-data-path-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tctx: t.Context(),\n\t\t\tkubeClientObj: []runtime.Object{pvbInProgress},\n\t\t\tdataPathStarted: true,\n\t\t\tresult: &dataPathResult{\n\t\t\t\tresult: \"fake-succeed-result\",\n\t\t\t},\n\t\t\texpectedEventMsg: fmt.Sprintf(\"Data path for %s stopped\", pvbName),\n\t\t},\n\t}\n\n\tscheme := runtime.NewScheme()\n\tvelerov1api.AddToScheme(scheme)\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeClientBuilder := clientFake.NewClientBuilder()\n\t\t\tfakeClientBuilder = fakeClientBuilder.WithScheme(scheme)\n\n\t\t\tfakeClient := fakeClientBuilder.WithRuntimeObjects(test.kubeClientObj...).Build()\n\n\t\t\tbt := &backupMsTestHelper{}\n\n\t\t\tbs := &BackupMicroService{\n\t\t\t\tnamespace: velerov1api.DefaultNamespace,\n\t\t\t\tpvbName: pvbName,\n\t\t\t\tctx: t.Context(),\n\t\t\t\tclient: fakeClient,\n\t\t\t\tdataPathMgr: datapath.NewManager(1),\n\t\t\t\teventRecorder: bt,\n\t\t\t\tresultSignal: make(chan dataPathResult),\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tif test.ctx != nil {\n\t\t\t\tbs.ctx = test.ctx\n\t\t\t}\n\n\t\t\tif test.dataPathMgr != nil {\n\t\t\t\tbs.dataPathMgr = test.dataPathMgr\n\t\t\t}\n\n\t\t\tdatapath.FSBRCreator = func(string, string, kbclient.Client, string, datapath.Callbacks, logrus.FieldLogger) datapath.AsyncBR {\n\t\t\t\tfsBR := datapathmockes.NewAsyncBR(t)\n\t\t\t\tif test.initErr != nil {\n\t\t\t\t\tfsBR.On(\"Init\", mock.Anything, mock.Anything).Return(test.initErr)\n\t\t\t\t}\n\n\t\t\t\tif test.startErr != nil {\n\t\t\t\t\tfsBR.On(\"Init\", mock.Anything, mock.Anything).Return(nil)\n\t\t\t\t\tfsBR.On(\"StartBackup\", mock.Anything, mock.Anything, mock.Anything).Return(test.startErr)\n\t\t\t\t}\n\n\t\t\t\tif test.dataPathStarted {\n\t\t\t\t\tfsBR.On(\"Init\", mock.Anything, mock.Anything).Return(nil)\n\t\t\t\t\tfsBR.On(\"StartBackup\", mock.Anything, mock.Anything, mock.Anything).Return(nil)\n\t\t\t\t}\n\n\t\t\t\treturn fsBR\n\t\t\t}\n\n\t\t\tif test.result != nil {\n\t\t\t\tgo func() {\n\t\t\t\t\ttime.Sleep(time.Millisecond * 500)\n\t\t\t\t\tbs.resultSignal <- *test.result\n\t\t\t\t}()\n\t\t\t}\n\n\t\t\tresult, err := bs.RunCancelableDataPath(test.ctx)\n\n\t\t\tif test.expectedErr != \"\" {\n\t\t\t\trequire.EqualError(t, err, test.expectedErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, test.result.result, result)\n\t\t\t}\n\n\t\t\tif test.expectedEventMsg != \"\" {\n\t\t\t\tassert.True(t, bt.withEvent)\n\t\t\t\tassert.Equal(t, test.expectedEventMsg, bt.EventMessage())\n\t\t\t}\n\t\t})\n\t}\n\n\tcancel()\n}\n" }, { "path": "pkg/podvolume/backupper.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"sync\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\t\"k8s.io/client-go/tools/cache\"\n\tctrlcache \"sigs.k8s.io/controller-runtime/pkg/cache\"\n\tctrlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/resourcepolicies\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tveleroclient \"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\t\"github.com/vmware-tanzu/velero/pkg/nodeagent\"\n\t\"github.com/vmware-tanzu/velero/pkg/podvolume/configs\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\tuploaderutil \"github.com/vmware-tanzu/velero/pkg/uploader/util\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nconst (\n\tindexNamePod = \"POD\"\n\tpvbKeyPattern = \"%s+%s+%s\"\n)\n\n// Backupper can execute pod volume backups of volumes in a pod.\ntype Backupper interface {\n\t// BackupPodVolumes backs up all specified volumes in a pod.\n\tBackupPodVolumes(backup *velerov1api.Backup, pod *corev1api.Pod, volumesToBackup []string, resPolicies *resourcepolicies.Policies, log logrus.FieldLogger) ([]*velerov1api.PodVolumeBackup, *PVCBackupSummary, []error)\n\tWaitAllPodVolumesProcessed(log logrus.FieldLogger) []*velerov1api.PodVolumeBackup\n\tGetPodVolumeBackupByPodAndVolume(podNamespace, podName, volume string) (*velerov1api.PodVolumeBackup, error)\n\tListPodVolumeBackupsByPod(podNamespace, podName string) ([]*velerov1api.PodVolumeBackup, error)\n}\n\ntype backupper struct {\n\tctx context.Context\n\trepoLocker *repository.RepoLocker\n\trepoEnsurer *repository.Ensurer\n\tcrClient ctrlclient.Client\n\tuploaderType string\n\tpvbInformer ctrlcache.Informer\n\thandlerRegistration cache.ResourceEventHandlerRegistration\n\twg sync.WaitGroup\n\t// pvbIndexer holds all PVBs created by this backuper and is capable to search\n\t// the PVBs based on specific properties quickly because of the embedded indexes.\n\t// The statuses of the PVBs are got updated when Informer receives update events.\n\tpvbIndexer cache.Indexer\n}\n\ntype skippedPVC struct {\n\tPVC *corev1api.PersistentVolumeClaim\n\tReason string\n}\n\n// PVCBackupSummary is a summary for which PVCs are skipped, which are backed up after each execution of the Backupper\n// The scope should be within one pod, so the volume name is the key for the maps\ntype PVCBackupSummary struct {\n\tBackedup map[string]*corev1api.PersistentVolumeClaim\n\tSkipped map[string]*skippedPVC\n\tpvcMap map[string]*corev1api.PersistentVolumeClaim\n}\n\nfunc NewPVCBackupSummary() *PVCBackupSummary {\n\treturn &PVCBackupSummary{\n\t\tBackedup: make(map[string]*corev1api.PersistentVolumeClaim),\n\t\tSkipped: make(map[string]*skippedPVC),\n\t\tpvcMap: make(map[string]*corev1api.PersistentVolumeClaim),\n\t}\n}\n\nfunc (pbs *PVCBackupSummary) addBackedup(volumeName string) {\n\tif pvc, ok := pbs.pvcMap[volumeName]; ok {\n\t\tpbs.Backedup[volumeName] = pvc\n\t\tdelete(pbs.Skipped, volumeName)\n\t}\n}\n\nfunc (pbs *PVCBackupSummary) addSkipped(volumeName string, reason string) {\n\tif pvc, ok := pbs.pvcMap[volumeName]; ok {\n\t\tif _, ok2 := pbs.Backedup[volumeName]; !ok2 { // if it's not backed up, add it to skipped\n\t\t\tpbs.Skipped[volumeName] = &skippedPVC{\n\t\t\t\tPVC: pvc,\n\t\t\t\tReason: reason,\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc podIndexFunc(obj any) ([]string, error) {\n\tpvb, ok := obj.(*velerov1api.PodVolumeBackup)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"expected PodVolumeBackup, but got %T\", obj)\n\t}\n\tif pvb == nil {\n\t\treturn nil, errors.New(\"PodVolumeBackup is nil\")\n\t}\n\treturn []string{cache.NewObjectName(pvb.Spec.Pod.Namespace, pvb.Spec.Pod.Name).String()}, nil\n}\n\n// the PVB's name is auto-generated when creating the PVB, we cannot get the name or uid before creating it.\n// So we cannot use namespace&name or uid as the key because we need to insert PVB into the indexer before creating it in API server\nfunc podVolumeBackupKey(obj any) (string, error) {\n\tpvb, ok := obj.(*velerov1api.PodVolumeBackup)\n\tif !ok {\n\t\treturn \"\", fmt.Errorf(\"expected PodVolumeBackup, but got %T\", obj)\n\t}\n\treturn fmt.Sprintf(pvbKeyPattern, pvb.Spec.Pod.Namespace, pvb.Spec.Pod.Name, pvb.Spec.Volume), nil\n}\n\nfunc newBackupper(\n\tctx context.Context,\n\tlog logrus.FieldLogger,\n\trepoLocker *repository.RepoLocker,\n\trepoEnsurer *repository.Ensurer,\n\tpvbInformer ctrlcache.Informer,\n\tcrClient ctrlclient.Client,\n\tuploaderType string,\n\tbackup *velerov1api.Backup,\n) *backupper {\n\tb := &backupper{\n\t\tctx: ctx,\n\t\trepoLocker: repoLocker,\n\t\trepoEnsurer: repoEnsurer,\n\t\tcrClient: crClient,\n\t\tuploaderType: uploaderType,\n\t\tpvbInformer: pvbInformer,\n\t\twg: sync.WaitGroup{},\n\t\tpvbIndexer: cache.NewIndexer(podVolumeBackupKey, cache.Indexers{\n\t\t\tindexNamePod: podIndexFunc,\n\t\t}),\n\t}\n\n\tb.handlerRegistration, _ = pvbInformer.AddEventHandler(\n\t\tcache.ResourceEventHandlerFuncs{\n\t\t\tUpdateFunc: func(_, obj any) {\n\t\t\t\tpvb, ok := obj.(*velerov1api.PodVolumeBackup)\n\t\t\t\tif !ok {\n\t\t\t\t\tlog.Errorf(\"expected PodVolumeBackup, but got %T\", obj)\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif pvb.GetLabels()[velerov1api.BackupUIDLabel] != string(backup.UID) {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif pvb.Status.Phase != velerov1api.PodVolumeBackupPhaseCompleted &&\n\t\t\t\t\tpvb.Status.Phase != velerov1api.PodVolumeBackupPhaseFailed &&\n\t\t\t\t\tpvb.Status.Phase != velerov1api.PodVolumeBackupPhaseCanceled {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tstatusChangedToFinal := true\n\t\t\t\texistObj, exist, err := b.pvbIndexer.Get(pvb)\n\t\t\t\tif err == nil && exist {\n\t\t\t\t\texistPVB, ok := existObj.(*velerov1api.PodVolumeBackup)\n\t\t\t\t\t// the PVB in the indexer is already in final status, no need to call WaitGroup.Done()\n\t\t\t\t\tif ok && (existPVB.Status.Phase == velerov1api.PodVolumeBackupPhaseCompleted ||\n\t\t\t\t\t\texistPVB.Status.Phase == velerov1api.PodVolumeBackupPhaseFailed ||\n\t\t\t\t\t\tpvb.Status.Phase == velerov1api.PodVolumeBackupPhaseCanceled) {\n\t\t\t\t\t\tstatusChangedToFinal = false\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\t// the Indexer inserts PVB directly if the PVB to be updated doesn't exist\n\t\t\t\tif err := b.pvbIndexer.Update(pvb); err != nil {\n\t\t\t\t\tlog.WithError(err).Errorf(\"failed to update PVB %s/%s in indexer\", pvb.Namespace, pvb.Name)\n\t\t\t\t}\n\n\t\t\t\t// call WaitGroup.Done() once only when the PVB changes to final status the first time.\n\t\t\t\t// This avoid the cases that the handler gets multiple update events whose PVBs are all in final status\n\t\t\t\t// which causes panic with \"negative WaitGroup counter\" error\n\t\t\t\tif statusChangedToFinal {\n\t\t\t\t\tb.wg.Done()\n\t\t\t\t}\n\t\t\t},\n\t\t},\n\t)\n\n\treturn b\n}\n\nfunc resultsKey(ns, name string) string {\n\treturn fmt.Sprintf(\"%s/%s\", ns, name)\n}\n\nfunc (b *backupper) getMatchAction(resPolicies *resourcepolicies.Policies, pvc *corev1api.PersistentVolumeClaim, volume *corev1api.Volume) (*resourcepolicies.Action, error) {\n\tif pvc != nil {\n\t\t// Ignore err, if the PV is not available (Pending/Lost PVC or PV fetch failed) - try matching with PVC only\n\t\t// GetPVForPVC returns nil for all error cases\n\t\tpv, _ := kube.GetPVForPVC(pvc, b.crClient)\n\t\tvfd := resourcepolicies.NewVolumeFilterData(pv, nil, pvc)\n\t\treturn resPolicies.GetMatchAction(vfd)\n\t}\n\n\tif volume != nil {\n\t\tvfd := resourcepolicies.NewVolumeFilterData(nil, volume, pvc)\n\t\treturn resPolicies.GetMatchAction(vfd)\n\t}\n\n\treturn nil, errors.Errorf(\"failed to check resource policies for empty volume\")\n}\n\nvar funcGetRepositoryType = getRepositoryType\n\nfunc (b *backupper) BackupPodVolumes(backup *velerov1api.Backup, pod *corev1api.Pod, volumesToBackup []string, resPolicies *resourcepolicies.Policies, log logrus.FieldLogger) ([]*velerov1api.PodVolumeBackup, *PVCBackupSummary, []error) {\n\tif len(volumesToBackup) == 0 {\n\t\treturn nil, nil, nil\n\t}\n\n\tlog.Infof(\"pod %s/%s has volumes to backup: %v\", pod.Namespace, pod.Name, volumesToBackup)\n\n\tvar (\n\t\tpvcSummary = NewPVCBackupSummary()\n\t\tpodVolumes = make(map[string]corev1api.Volume)\n\t\terrs = []error{}\n\t)\n\n\t// put the pod's volumes and the PVC associated in maps for efficient lookup below\n\tfor _, podVolume := range pod.Spec.Volumes {\n\t\tpodVolumes[podVolume.Name] = podVolume\n\t\tif podVolume.PersistentVolumeClaim != nil {\n\t\t\tpvc := new(corev1api.PersistentVolumeClaim)\n\t\t\terr := b.crClient.Get(context.TODO(), ctrlclient.ObjectKey{Namespace: pod.Namespace, Name: podVolume.PersistentVolumeClaim.ClaimName}, pvc)\n\t\t\tif err != nil {\n\t\t\t\terrs = append(errs, errors.Wrap(err, \"error getting persistent volume claim for volume\"))\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tpvcSummary.pvcMap[podVolume.Name] = pvc\n\t\t}\n\t}\n\n\tif msg, err := uploader.ValidateUploaderType(b.uploaderType); err != nil {\n\t\tskipAllPodVolumes(pod, volumesToBackup, err, pvcSummary, log)\n\t\treturn nil, pvcSummary, []error{err}\n\t} else if msg != \"\" {\n\t\tlog.Warn(msg)\n\t}\n\n\tif err := kube.IsPodRunning(pod); err != nil {\n\t\tskipAllPodVolumes(pod, volumesToBackup, err, pvcSummary, log)\n\t\treturn nil, pvcSummary, nil\n\t}\n\n\terr := nodeagent.IsRunningInNode(b.ctx, backup.Namespace, pod.Spec.NodeName, b.crClient)\n\tif err != nil {\n\t\tskipAllPodVolumes(pod, volumesToBackup, err, pvcSummary, log)\n\t\treturn nil, pvcSummary, []error{err}\n\t}\n\n\trepositoryType := funcGetRepositoryType(b.uploaderType)\n\tif repositoryType == \"\" {\n\t\terr := errors.Errorf(\"empty repository type, uploader %s\", b.uploaderType)\n\t\tskipAllPodVolumes(pod, volumesToBackup, err, pvcSummary, log)\n\t\treturn nil, pvcSummary, []error{err}\n\t}\n\n\trepo, err := b.repoEnsurer.EnsureRepo(b.ctx, backup.Namespace, pod.Namespace, backup.Spec.StorageLocation, repositoryType)\n\tif err != nil {\n\t\tskipAllPodVolumes(pod, volumesToBackup, err, pvcSummary, log)\n\t\treturn nil, pvcSummary, []error{err}\n\t}\n\n\t// get a single non-exclusive lock since we'll wait for all individual\n\t// backups to be complete before releasing it.\n\tb.repoLocker.Lock(repo.Name)\n\tdefer b.repoLocker.Unlock(repo.Name)\n\n\tvar (\n\t\tpodVolumeBackups []*velerov1api.PodVolumeBackup\n\t\tmountedPodVolumes = sets.Set[string]{}\n\t\tattachedPodDevices = sets.Set[string]{}\n\t)\n\n\tfor _, container := range pod.Spec.Containers {\n\t\tfor _, volumeMount := range container.VolumeMounts {\n\t\t\tmountedPodVolumes.Insert(volumeMount.Name)\n\t\t}\n\t\tfor _, volumeDevice := range container.VolumeDevices {\n\t\t\tattachedPodDevices.Insert(volumeDevice.Name)\n\t\t}\n\t}\n\n\trepoIdentifier := \"\"\n\tif repositoryType == velerov1api.BackupRepositoryTypeRestic {\n\t\trepoIdentifier = repo.Spec.ResticIdentifier\n\t}\n\n\tfor _, volumeName := range volumesToBackup {\n\t\tvolume, ok := podVolumes[volumeName]\n\t\tif !ok {\n\t\t\tlog.Warnf(\"No volume named %s found in pod %s/%s, skipping\", volumeName, pod.Namespace, pod.Name)\n\t\t\tcontinue\n\t\t}\n\t\tvar pvc *corev1api.PersistentVolumeClaim\n\t\tif volume.PersistentVolumeClaim != nil {\n\t\t\tpvc, ok = pvcSummary.pvcMap[volumeName]\n\t\t\tif !ok {\n\t\t\t\t// there should have been error happened retrieving the PVC and it's recorded already\n\t\t\t\tcontinue\n\t\t\t}\n\t\t}\n\n\t\tif resPolicies != nil {\n\t\t\tif action, err := b.getMatchAction(resPolicies, pvc, &volume); err != nil {\n\t\t\t\terrs = append(errs, errors.Wrapf(err, \"error getting pv for pvc %s\", pvc.Spec.VolumeName))\n\t\t\t\tcontinue\n\t\t\t} else if action != nil && action.Type == resourcepolicies.Skip {\n\t\t\t\tlog.Infof(\"skip backup of volume %s for the matched resource policies\", volumeName)\n\t\t\t\tpvcSummary.addSkipped(volumeName, \"matched action is 'skip' in chosen resource policies\")\n\t\t\t\tcontinue\n\t\t\t}\n\t\t}\n\n\t\t// hostPath volumes are not supported because they're not mounted into /var/lib/kubelet/pods, so our\n\t\t// daemonset pod has no way to access their data.\n\t\tisHostPath, err := isHostPathVolume(&volume, pvc, b.crClient)\n\t\tif err != nil {\n\t\t\terrs = append(errs, errors.Wrap(err, \"error checking if volume is a hostPath volume\"))\n\t\t\tcontinue\n\t\t}\n\t\tif isHostPath {\n\t\t\tlog.Warnf(\"Volume %s in pod %s/%s is a hostPath volume which is not supported for pod volume backup, skipping\", volumeName, pod.Namespace, pod.Name)\n\t\t\tcontinue\n\t\t}\n\n\t\t// check if volume is a block volume\n\t\tif attachedPodDevices.Has(volumeName) {\n\t\t\tmsg := fmt.Sprintf(\"volume %s declared in pod %s/%s is a block volume. Block volumes are not supported for fs backup, skipping\",\n\t\t\t\tvolumeName, pod.Namespace, pod.Name)\n\t\t\tlog.Warn(msg)\n\t\t\tpvcSummary.addSkipped(volumeName, msg)\n\t\t\tcontinue\n\t\t}\n\n\t\t// volumes that are not mounted by any container should not be backed up, because\n\t\t// its directory is not created\n\t\tif !mountedPodVolumes.Has(volumeName) {\n\t\t\tmsg := fmt.Sprintf(\"volume %s is declared in pod %s/%s but not mounted by any container, skipping\", volumeName, pod.Namespace, pod.Name)\n\t\t\tlog.Warn(msg)\n\t\t\tpvcSummary.addSkipped(volumeName, msg)\n\t\t\tcontinue\n\t\t}\n\n\t\tvolumeBackup := newPodVolumeBackup(backup, pod, volume, repoIdentifier, b.uploaderType, pvc)\n\t\t// the PVB must be added into the indexer before creating it in API server otherwise unexpected behavior may happen:\n\t\t// the PVB may be handled very quickly by the controller and the informer handler will insert the PVB before \"b.pvbIndexer.Add(volumeBackup)\" runs,\n\t\t// this causes the PVB inserted by \"b.pvbIndexer.Add(volumeBackup)\" overrides the PVB in the indexer while the PVB inserted by \"b.pvbIndexer.Add(volumeBackup)\"\n\t\t// contains empty \"Status\"\n\t\tif err := b.pvbIndexer.Add(volumeBackup); err != nil {\n\t\t\terrs = append(errs, errors.Wrapf(err, \"failed to add PodVolumeBackup %s/%s to indexer\", volumeBackup.Namespace, volumeBackup.Name))\n\t\t\tcontinue\n\t\t}\n\t\t// similar with above: the PVB may be handled very quickly by the controller and the informer handler will call \"b.wg.Done()\" before \"b.wg.Add(1)\" runs which causes panic\n\t\t// see https://github.com/vmware-tanzu/velero/issues/8657\n\t\tb.wg.Add(1)\n\t\tif err := veleroclient.CreateRetryGenerateName(b.crClient, b.ctx, volumeBackup); err != nil {\n\t\t\tb.wg.Done()\n\t\t\terrs = append(errs, err)\n\t\t\tcontinue\n\t\t}\n\n\t\tpodVolumeBackups = append(podVolumeBackups, volumeBackup)\n\t\tpvcSummary.addBackedup(volumeName)\n\t}\n\n\treturn podVolumeBackups, pvcSummary, errs\n}\n\nfunc (b *backupper) WaitAllPodVolumesProcessed(log logrus.FieldLogger) []*velerov1api.PodVolumeBackup {\n\tdefer func() {\n\t\tif err := b.pvbInformer.RemoveEventHandler(b.handlerRegistration); err != nil {\n\t\t\tlog.Debugf(\"failed to remove the event handler for PVB: %v\", err)\n\t\t}\n\t}()\n\n\tlog.Info(\"Waiting for completion of PVB\")\n\n\tvar podVolumeBackups []*velerov1api.PodVolumeBackup\n\t// if no pod volume backups are tracked, return directly to avoid issue mentioned in\n\t// https://github.com/vmware-tanzu/velero/issues/8723\n\tif len(b.pvbIndexer.List()) == 0 {\n\t\treturn podVolumeBackups\n\t}\n\n\tdone := make(chan struct{})\n\tgo func() {\n\t\tdefer close(done)\n\t\tb.wg.Wait()\n\t}()\n\n\tselect {\n\tcase <-b.ctx.Done():\n\t\tlog.Error(\"timed out waiting for all PodVolumeBackups to complete\")\n\tcase <-done:\n\t\tfor _, obj := range b.pvbIndexer.List() {\n\t\t\tpvb, ok := obj.(*velerov1api.PodVolumeBackup)\n\t\t\tif !ok {\n\t\t\t\tlog.Errorf(\"expected PodVolumeBackup, but got %T\", obj)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tpodVolumeBackups = append(podVolumeBackups, pvb)\n\t\t\tif pvb.Status.Phase == velerov1api.PodVolumeBackupPhaseFailed {\n\t\t\t\tlog.Errorf(\"pod volume backup failed: %s\", pvb.Status.Message)\n\t\t\t} else if pvb.Status.Phase == velerov1api.PodVolumeBackupPhaseCanceled {\n\t\t\t\tlog.Errorf(\"pod volume backup canceled: %s\", pvb.Status.Message)\n\t\t\t}\n\t\t}\n\t}\n\treturn podVolumeBackups\n}\n\nfunc (b *backupper) GetPodVolumeBackupByPodAndVolume(podNamespace, podName, volume string) (*velerov1api.PodVolumeBackup, error) {\n\tobj, exist, err := b.pvbIndexer.GetByKey(fmt.Sprintf(pvbKeyPattern, podNamespace, podName, volume))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif !exist {\n\t\treturn nil, nil\n\t}\n\tpvb, ok := obj.(*velerov1api.PodVolumeBackup)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"expected PodVolumeBackup, but got %T\", obj)\n\t}\n\treturn pvb, nil\n}\n\nfunc (b *backupper) ListPodVolumeBackupsByPod(podNamespace, podName string) ([]*velerov1api.PodVolumeBackup, error) {\n\tobjs, err := b.pvbIndexer.ByIndex(indexNamePod, cache.NewObjectName(podNamespace, podName).String())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tvar pvbs []*velerov1api.PodVolumeBackup\n\tfor _, obj := range objs {\n\t\tpvb, ok := obj.(*velerov1api.PodVolumeBackup)\n\t\tif !ok {\n\t\t\treturn nil, errors.Errorf(\"expected PodVolumeBackup, but got %T\", obj)\n\t\t}\n\t\tpvbs = append(pvbs, pvb)\n\t}\n\treturn pvbs, nil\n}\n\nfunc skipAllPodVolumes(pod *corev1api.Pod, volumesToBackup []string, err error, pvcSummary *PVCBackupSummary, log logrus.FieldLogger) {\n\tfor _, volumeName := range volumesToBackup {\n\t\tlog.WithError(err).Warnf(\"Skip pod volume %s\", volumeName)\n\t\tpvcSummary.addSkipped(volumeName, fmt.Sprintf(\"encountered a problem with backing up the PVC of pod %s/%s: %v\", pod.Namespace, pod.Name, err))\n\t}\n}\n\n// isHostPathVolume returns true if the volume is either a hostPath pod volume or a persistent\n// volume claim on a hostPath persistent volume, or false otherwise.\nfunc isHostPathVolume(volume *corev1api.Volume, pvc *corev1api.PersistentVolumeClaim, crClient ctrlclient.Client) (bool, error) {\n\tif volume.HostPath != nil {\n\t\treturn true, nil\n\t}\n\n\tif pvc == nil || pvc.Spec.VolumeName == \"\" {\n\t\treturn false, nil\n\t}\n\n\tpv := new(corev1api.PersistentVolume)\n\terr := crClient.Get(context.TODO(), ctrlclient.ObjectKey{Name: pvc.Spec.VolumeName}, pv)\n\tif err != nil {\n\t\treturn false, errors.WithStack(err)\n\t}\n\n\treturn pv.Spec.HostPath != nil, nil\n}\n\nfunc newPodVolumeBackup(backup *velerov1api.Backup, pod *corev1api.Pod, volume corev1api.Volume, repoIdentifier, uploaderType string, pvc *corev1api.PersistentVolumeClaim) *velerov1api.PodVolumeBackup {\n\tpvb := &velerov1api.PodVolumeBackup{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: backup.Namespace,\n\t\t\tGenerateName: backup.Name + \"-\",\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\t\t\tKind: \"Backup\",\n\t\t\t\t\tName: backup.Name,\n\t\t\t\t\tUID: backup.UID,\n\t\t\t\t\tController: boolptr.True(),\n\t\t\t\t},\n\t\t\t},\n\t\t\tLabels: map[string]string{\n\t\t\t\tvelerov1api.BackupNameLabel: label.GetValidName(backup.Name),\n\t\t\t\tvelerov1api.BackupUIDLabel: string(backup.UID),\n\t\t\t},\n\t\t},\n\t\tSpec: velerov1api.PodVolumeBackupSpec{\n\t\t\tNode: pod.Spec.NodeName,\n\t\t\tPod: corev1api.ObjectReference{\n\t\t\t\tKind: \"Pod\",\n\t\t\t\tNamespace: pod.Namespace,\n\t\t\t\tName: pod.Name,\n\t\t\t\tUID: pod.UID,\n\t\t\t},\n\t\t\tVolume: volume.Name,\n\t\t\tTags: map[string]string{\n\t\t\t\t\"backup\": backup.Name,\n\t\t\t\t\"backup-uid\": string(backup.UID),\n\t\t\t\t\"pod\": pod.Name,\n\t\t\t\t\"pod-uid\": string(pod.UID),\n\t\t\t\t\"ns\": pod.Namespace,\n\t\t\t\t\"volume\": volume.Name,\n\t\t\t},\n\t\t\tBackupStorageLocation: backup.Spec.StorageLocation,\n\t\t\tRepoIdentifier: repoIdentifier,\n\t\t\tUploaderType: uploaderType,\n\t\t},\n\t}\n\n\tif pvc != nil {\n\t\t// this annotation is used in pkg/restore to identify if a PVC\n\t\t// has a pod volume backup.\n\t\tpvb.Annotations = map[string]string{\n\t\t\tconfigs.PVCNameAnnotation: pvc.Name,\n\t\t}\n\n\t\t// this label is used by the pod volume backup controller to tell\n\t\t// if a pod volume backup is for a PVC.\n\t\tpvb.Labels[velerov1api.PVCUIDLabel] = string(pvc.UID)\n\n\t\t// this tag is not used by velero, but useful for debugging.\n\t\tpvb.Spec.Tags[\"pvc-uid\"] = string(pvc.UID)\n\t}\n\n\tif backup.Spec.UploaderConfig != nil {\n\t\tpvb.Spec.UploaderSettings = uploaderutil.StoreBackupConfig(backup.Spec.UploaderConfig)\n\t}\n\n\treturn pvb\n}\n" }, { "path": "pkg/podvolume/backupper_factory.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/client-go/tools/cache\"\n\tctrlcache \"sigs.k8s.io/controller-runtime/pkg/cache\"\n\tctrlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n)\n\n// BackupperFactory can construct pod volumes backuppers.\ntype BackupperFactory interface {\n\t// NewBackupper returns a pod volumes backupper for use during a single Velero backup.\n\tNewBackupper(context.Context, logrus.FieldLogger, *velerov1api.Backup, string) (Backupper, error)\n}\n\nfunc NewBackupperFactory(\n\trepoLocker *repository.RepoLocker,\n\trepoEnsurer *repository.Ensurer,\n\tcrClient ctrlclient.Client,\n\tpvbInformer ctrlcache.Informer,\n\tlog logrus.FieldLogger,\n) BackupperFactory {\n\treturn &backupperFactory{\n\t\trepoLocker: repoLocker,\n\t\trepoEnsurer: repoEnsurer,\n\t\tcrClient: crClient,\n\t\tpvbInformer: pvbInformer,\n\t\tlog: log,\n\t}\n}\n\ntype backupperFactory struct {\n\trepoLocker *repository.RepoLocker\n\trepoEnsurer *repository.Ensurer\n\tcrClient ctrlclient.Client\n\tpvbInformer ctrlcache.Informer\n\tlog logrus.FieldLogger\n}\n\nfunc (bf *backupperFactory) NewBackupper(ctx context.Context, log logrus.FieldLogger, backup *velerov1api.Backup, uploaderType string) (Backupper, error) {\n\tb := newBackupper(ctx, log, bf.repoLocker, bf.repoEnsurer, bf.pvbInformer, bf.crClient, uploaderType, backup)\n\n\tif !cache.WaitForCacheSync(ctx.Done(), bf.pvbInformer.HasSynced) {\n\t\treturn nil, errors.New(\"timed out waiting for caches to sync\")\n\t}\n\n\treturn b, nil\n}\n" }, { "path": "pkg/podvolume/backupper_test.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\tctrlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\tctrlfake \"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tclientTesting \"k8s.io/client-go/testing\"\n\t\"k8s.io/client-go/tools/cache\"\n\n\t\"github.com/vmware-tanzu/velero/internal/resourcepolicies\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nfunc TestIsHostPathVolume(t *testing.T) {\n\t// hostPath pod volume\n\tvol := &corev1api.Volume{\n\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\tHostPath: &corev1api.HostPathVolumeSource{},\n\t\t},\n\t}\n\tisHostPath, err := isHostPathVolume(vol, nil, nil)\n\trequire.NoError(t, err)\n\tassert.True(t, isHostPath)\n\n\t// non-hostPath pod volume\n\tvol = &corev1api.Volume{\n\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\tEmptyDir: &corev1api.EmptyDirVolumeSource{},\n\t\t},\n\t}\n\tisHostPath, err = isHostPathVolume(vol, nil, nil)\n\trequire.NoError(t, err)\n\tassert.False(t, isHostPath)\n\n\t// PVC that doesn't have a PV\n\tvol = &corev1api.Volume{\n\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\tClaimName: \"pvc-1\",\n\t\t\t},\n\t\t},\n\t}\n\tpvc := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"ns-1\",\n\t\t\tName: \"pvc-1\",\n\t\t},\n\t}\n\tisHostPath, err = isHostPathVolume(vol, pvc, nil)\n\trequire.NoError(t, err)\n\tassert.False(t, isHostPath)\n\n\t// PVC that claims a non-hostPath PV\n\tvol = &corev1api.Volume{\n\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\tClaimName: \"pvc-1\",\n\t\t\t},\n\t\t},\n\t}\n\tpvc = &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"ns-1\",\n\t\t\tName: \"pvc-1\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tVolumeName: \"pv-1\",\n\t\t},\n\t}\n\tpv := &corev1api.PersistentVolume{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"pv-1\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeSpec{},\n\t}\n\tcrClient1 := velerotest.NewFakeControllerRuntimeClient(t, pv)\n\tisHostPath, err = isHostPathVolume(vol, pvc, crClient1)\n\trequire.NoError(t, err)\n\tassert.False(t, isHostPath)\n\n\t// PVC that claims a hostPath PV\n\tvol = &corev1api.Volume{\n\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\tClaimName: \"pvc-1\",\n\t\t\t},\n\t\t},\n\t}\n\tpvc = &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"ns-1\",\n\t\t\tName: \"pvc-1\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tVolumeName: \"pv-1\",\n\t\t},\n\t}\n\tpv = &corev1api.PersistentVolume{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"pv-1\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\tHostPath: &corev1api.HostPathVolumeSource{},\n\t\t\t},\n\t\t},\n\t}\n\tcrClient2 := velerotest.NewFakeControllerRuntimeClient(t, pv)\n\n\tisHostPath, err = isHostPathVolume(vol, pvc, crClient2)\n\trequire.NoError(t, err)\n\tassert.True(t, isHostPath)\n}\n\nfunc Test_backupper_BackupPodVolumes_log_test(t *testing.T) {\n\ttype args struct {\n\t\tbackup *velerov1api.Backup\n\t\tpod *corev1api.Pod\n\t\tvolumesToBackup []string\n\t\tresPolicies *resourcepolicies.Policies\n\t}\n\ttests := []struct {\n\t\tname string\n\t\targs args\n\t\twantLog string\n\t}{\n\t\t{\n\t\t\tname: \"backup pod volumes should log volume names\",\n\t\t\targs: args{\n\t\t\t\tbackup: &velerov1api.Backup{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"backup-1\",\n\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tpod: &corev1api.Pod{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t},\n\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tName: \"vol-1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tName: \"vol-2\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tvolumesToBackup: []string{\"vol-1\", \"vol-2\"},\n\t\t\t\tresPolicies: nil,\n\t\t\t},\n\t\t\twantLog: \"pod ns-1/pod-1 has volumes to backup: [vol-1 vol-2]\",\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tb := &backupper{\n\t\t\t\tctx: t.Context(),\n\t\t\t}\n\t\t\tlogOutput := bytes.Buffer{}\n\t\t\tvar log = logrus.New()\n\t\t\tlog.SetOutput(&logOutput)\n\t\t\tb.BackupPodVolumes(tt.args.backup, tt.args.pod, tt.args.volumesToBackup, tt.args.resPolicies, log)\n\t\t\tfmt.Println(logOutput.String())\n\t\t\tassert.Contains(t, logOutput.String(), tt.wantLog)\n\t\t})\n\t}\n}\n\ntype reactor struct {\n\tverb string\n\tresource string\n\treactorFunc clientTesting.ReactionFunc\n}\n\nfunc createBackupRepoObj() *velerov1api.BackupRepository {\n\tbkRepoObj := repository.NewBackupRepository(velerov1api.DefaultNamespace, repository.BackupRepositoryKey{\n\t\tVolumeNamespace: \"fake-ns\",\n\t\tBackupLocation: \"fake-bsl\",\n\t\tRepositoryType: \"kopia\",\n\t})\n\n\tbkRepoObj.Status.Phase = velerov1api.BackupRepositoryPhaseReady\n\n\treturn bkRepoObj\n}\n\nfunc createPodObj(running bool, withVolume bool, withVolumeMounted bool, volumeNum int) *corev1api.Pod {\n\tpodObj := builder.ForPod(\"fake-ns\", \"fake-pod\").Result()\n\tpodObj.Spec.NodeName = \"fake-node-name\"\n\tif running {\n\t\tpodObj.Status.Phase = corev1api.PodRunning\n\t}\n\n\tif withVolume {\n\t\tfor i := 0; i < volumeNum; i++ {\n\t\t\tpodObj.Spec.Volumes = append(podObj.Spec.Volumes, corev1api.Volume{\n\t\t\t\tName: fmt.Sprintf(\"fake-volume-%d\", i+1),\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\tClaimName: fmt.Sprintf(\"fake-pvc-%d\", i+1),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t})\n\t\t}\n\n\t\tif withVolumeMounted {\n\t\t\tvolumeMount := []corev1api.VolumeMount{}\n\t\t\tfor i := 0; i < volumeNum; i++ {\n\t\t\t\tvolumeMount = append(volumeMount, corev1api.VolumeMount{\n\t\t\t\t\tName: fmt.Sprintf(\"fake-volume-%d\", i+1),\n\t\t\t\t})\n\t\t\t}\n\t\t\tpodObj.Spec.Containers = []corev1api.Container{\n\t\t\t\t{\n\t\t\t\t\tName: \"fake-container\",\n\t\t\t\t\tVolumeMounts: volumeMount,\n\t\t\t\t},\n\t\t\t}\n\t\t}\n\t}\n\n\treturn podObj\n}\n\nfunc createNodeAgentPodObj(running bool) *corev1api.Pod {\n\tpodObj := builder.ForPod(velerov1api.DefaultNamespace, \"fake-node-agent\").Result()\n\tpodObj.Labels = map[string]string{\"role\": \"node-agent\"}\n\n\tif running {\n\t\tpodObj.Status.Phase = corev1api.PodRunning\n\t\tpodObj.Spec.NodeName = \"fake-node-name\"\n\t}\n\n\treturn podObj\n}\n\nfunc createPVObj(index int, withHostPath bool) *corev1api.PersistentVolume {\n\tpvObj := builder.ForPersistentVolume(fmt.Sprintf(\"fake-pv-%d\", index)).Result()\n\tif withHostPath {\n\t\tpvObj.Spec.HostPath = &corev1api.HostPathVolumeSource{Path: \"fake-host-path\"}\n\t}\n\n\treturn pvObj\n}\n\nfunc createPVCObj(index int) *corev1api.PersistentVolumeClaim {\n\tpvcObj := builder.ForPersistentVolumeClaim(\"fake-ns\", fmt.Sprintf(\"fake-pvc-%d\", index)).VolumeName(fmt.Sprintf(\"fake-pv-%d\", index)).Result()\n\treturn pvcObj\n}\n\nfunc createPVBObj(fail bool, withSnapshot bool, index int, uploaderType string) *velerov1api.PodVolumeBackup {\n\tpvbObj := builder.ForPodVolumeBackup(velerov1api.DefaultNamespace, fmt.Sprintf(\"fake-pvb-%d\", index)).\n\t\tPodName(\"fake-pod\").PodNamespace(\"fake-ns\").Volume(fmt.Sprintf(\"fake-volume-%d\", index)).Result()\n\tif fail {\n\t\tpvbObj.Status.Phase = velerov1api.PodVolumeBackupPhaseFailed\n\t\tpvbObj.Status.Message = \"fake-message\"\n\t} else {\n\t\tpvbObj.Status.Phase = velerov1api.PodVolumeBackupPhaseCompleted\n\t}\n\n\tif withSnapshot {\n\t\tpvbObj.Status.SnapshotID = fmt.Sprintf(\"fake-snapshot-id-%d\", index)\n\t}\n\n\tpvbObj.Spec.UploaderType = uploaderType\n\n\treturn pvbObj\n}\n\nfunc createNodeObj() *corev1api.Node {\n\treturn builder.ForNode(\"fake-node-name\").Labels(map[string]string{\"kubernetes.io/os\": \"linux\"}).Result()\n}\n\nfunc TestBackupPodVolumes(t *testing.T) {\n\tscheme := runtime.NewScheme()\n\trequire.NoError(t, velerov1api.AddToScheme(scheme))\n\trequire.NoError(t, corev1api.AddToScheme(scheme))\n\tlog := logrus.New()\n\n\ttests := []struct {\n\t\tname string\n\t\tbsl string\n\t\tuploaderType string\n\t\tvolumes []string\n\t\tsourcePod *corev1api.Pod\n\t\tkubeClientObj []runtime.Object\n\t\tctlClientObj []runtime.Object\n\t\tveleroClientObj []runtime.Object\n\t\tveleroReactors []reactor\n\t\truntimeScheme *runtime.Scheme\n\t\tpvbs int\n\t\tmockGetRepositoryType bool\n\t\terrs []string\n\t}{\n\t\t{\n\t\t\tname: \"empty volume list\",\n\t\t},\n\t\t{\n\t\t\tname: \"wrong uploader type\",\n\t\t\tvolumes: []string{\n\t\t\t\t\"fake-volume-1\",\n\t\t\t\t\"fake-volume-2\",\n\t\t\t},\n\t\t\tsourcePod: createPodObj(true, false, false, 2),\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcreateNodeAgentPodObj(true),\n\t\t\t},\n\t\t\tuploaderType: \"fake-uploader-type\",\n\t\t\terrs: []string{\n\t\t\t\t\"invalid uploader type 'fake-uploader-type', valid type: 'kopia'\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"pod is not running\",\n\t\t\tvolumes: []string{\n\t\t\t\t\"fake-volume-1\",\n\t\t\t\t\"fake-volume-2\",\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcreateNodeAgentPodObj(true),\n\t\t\t},\n\t\t\tctlClientObj: []runtime.Object{\n\t\t\t\tcreateBackupRepoObj(),\n\t\t\t},\n\t\t\truntimeScheme: scheme,\n\t\t\tsourcePod: createPodObj(false, false, false, 2),\n\t\t\tuploaderType: \"kopia\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t},\n\t\t{\n\t\t\tname: \"node-agent pod is not running in node\",\n\t\t\tvolumes: []string{\n\t\t\t\t\"fake-volume-1\",\n\t\t\t\t\"fake-volume-2\",\n\t\t\t},\n\t\t\tsourcePod: createPodObj(true, false, false, 2),\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcreateNodeObj(),\n\t\t\t},\n\t\t\tuploaderType: \"kopia\",\n\t\t\terrs: []string{\n\t\t\t\t\"daemonset pod not found in running state in node fake-node-name\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"wrong repository type\",\n\t\t\tvolumes: []string{\n\t\t\t\t\"fake-volume-1\",\n\t\t\t\t\"fake-volume-2\",\n\t\t\t},\n\t\t\tsourcePod: createPodObj(true, false, false, 2),\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcreateNodeAgentPodObj(true),\n\t\t\t\tcreateNodeObj(),\n\t\t\t},\n\t\t\tuploaderType: \"kopia\",\n\t\t\tmockGetRepositoryType: true,\n\t\t\terrs: []string{\n\t\t\t\t\"empty repository type, uploader kopia\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"ensure repo fail\",\n\t\t\tvolumes: []string{\n\t\t\t\t\"fake-volume-1\",\n\t\t\t\t\"fake-volume-2\",\n\t\t\t},\n\t\t\tsourcePod: createPodObj(true, false, false, 2),\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcreateNodeAgentPodObj(true),\n\t\t\t\tcreateNodeObj(),\n\t\t\t},\n\t\t\tuploaderType: \"kopia\",\n\t\t\terrs: []string{\n\t\t\t\t\"wrong parameters, namespace \\\"fake-ns\\\", backup storage location \\\"\\\", repository type \\\"kopia\\\"\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"volume not found in pod\",\n\t\t\tvolumes: []string{\n\t\t\t\t\"fake-volume-1\",\n\t\t\t\t\"fake-volume-2\",\n\t\t\t},\n\t\t\tsourcePod: createPodObj(true, false, false, 2),\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcreateNodeAgentPodObj(true),\n\t\t\t\tcreateNodeObj(),\n\t\t\t},\n\t\t\tctlClientObj: []runtime.Object{\n\t\t\t\tcreateBackupRepoObj(),\n\t\t\t},\n\t\t\truntimeScheme: scheme,\n\t\t\tuploaderType: \"kopia\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t},\n\t\t{\n\t\t\tname: \"PVC not found\",\n\t\t\tvolumes: []string{\n\t\t\t\t\"fake-volume-1\",\n\t\t\t\t\"fake-volume-2\",\n\t\t\t},\n\t\t\tsourcePod: createPodObj(true, true, false, 2),\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcreateNodeAgentPodObj(true),\n\t\t\t\tcreateNodeObj(),\n\t\t\t},\n\t\t\tctlClientObj: []runtime.Object{\n\t\t\t\tcreateBackupRepoObj(),\n\t\t\t},\n\t\t\truntimeScheme: scheme,\n\t\t\tuploaderType: \"kopia\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\terrs: []string{\n\t\t\t\t\"error getting persistent volume claim for volume: persistentvolumeclaims \\\"fake-pvc-1\\\" not found\",\n\t\t\t\t\"error getting persistent volume claim for volume: persistentvolumeclaims \\\"fake-pvc-2\\\" not found\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"check host path fail\",\n\t\t\tvolumes: []string{\n\t\t\t\t\"fake-volume-1\",\n\t\t\t\t\"fake-volume-2\",\n\t\t\t},\n\t\t\tsourcePod: createPodObj(true, true, false, 2),\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcreateNodeAgentPodObj(true),\n\t\t\t\tcreateNodeObj(),\n\t\t\t\tcreatePVCObj(1),\n\t\t\t\tcreatePVCObj(2),\n\t\t\t},\n\t\t\tctlClientObj: []runtime.Object{\n\t\t\t\tcreateBackupRepoObj(),\n\t\t\t},\n\t\t\truntimeScheme: scheme,\n\t\t\tuploaderType: \"kopia\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\terrs: []string{\n\t\t\t\t\"error checking if volume is a hostPath volume: persistentvolumes \\\"fake-pv-1\\\" not found\",\n\t\t\t\t\"error checking if volume is a hostPath volume: persistentvolumes \\\"fake-pv-2\\\" not found\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"host path volume should be skipped\",\n\t\t\tvolumes: []string{\n\t\t\t\t\"fake-volume-1\",\n\t\t\t\t\"fake-volume-2\",\n\t\t\t},\n\t\t\tsourcePod: createPodObj(true, true, false, 2),\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcreateNodeAgentPodObj(true),\n\t\t\t\tcreateNodeObj(),\n\t\t\t\tcreatePVCObj(1),\n\t\t\t\tcreatePVCObj(2),\n\t\t\t\tcreatePVObj(1, true),\n\t\t\t\tcreatePVObj(2, true),\n\t\t\t},\n\t\t\tctlClientObj: []runtime.Object{\n\t\t\t\tcreateBackupRepoObj(),\n\t\t\t},\n\t\t\truntimeScheme: scheme,\n\t\t\tuploaderType: \"kopia\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\terrs: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"volume not mounted by pod should be skipped\",\n\t\t\tvolumes: []string{\n\t\t\t\t\"fake-volume-1\",\n\t\t\t\t\"fake-volume-2\",\n\t\t\t},\n\t\t\tsourcePod: createPodObj(true, true, false, 2),\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcreateNodeAgentPodObj(true),\n\t\t\t\tcreateNodeObj(),\n\t\t\t\tcreatePVCObj(1),\n\t\t\t\tcreatePVCObj(2),\n\t\t\t\tcreatePVObj(1, false),\n\t\t\t\tcreatePVObj(2, false),\n\t\t\t},\n\t\t\tctlClientObj: []runtime.Object{\n\t\t\t\tcreateBackupRepoObj(),\n\t\t\t},\n\t\t\truntimeScheme: scheme,\n\t\t\tuploaderType: \"kopia\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\terrs: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"return completed pvbs\",\n\t\t\tvolumes: []string{\n\t\t\t\t\"fake-volume-1\",\n\t\t\t},\n\t\t\tsourcePod: createPodObj(true, true, true, 1),\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcreateNodeAgentPodObj(true),\n\t\t\t\tcreateNodeObj(),\n\t\t\t\tcreatePVCObj(1),\n\t\t\t\tcreatePVObj(1, false),\n\t\t\t},\n\t\t\tctlClientObj: []runtime.Object{\n\t\t\t\tcreateBackupRepoObj(),\n\t\t\t},\n\t\t\truntimeScheme: scheme,\n\t\t\tuploaderType: \"kopia\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\tpvbs: 1,\n\t\t\terrs: []string{},\n\t\t},\n\t}\n\t// TODO add more verification around PVCBackupSummary returned by \"BackupPodVolumes\"\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\n\t\t\tfakeClientBuilder := ctrlfake.NewClientBuilder()\n\t\t\tif test.runtimeScheme != nil {\n\t\t\t\tfakeClientBuilder = fakeClientBuilder.WithScheme(test.runtimeScheme)\n\t\t\t}\n\n\t\t\tobjList := append(test.ctlClientObj, test.veleroClientObj...)\n\t\t\tobjList = append(objList, test.kubeClientObj...)\n\t\t\tfakeCtrlClient := fakeClientBuilder.WithRuntimeObjects(objList...).Build()\n\n\t\t\tfakeCRWatchClient := velerotest.NewFakeControllerRuntimeWatchClient(t, test.kubeClientObj...)\n\t\t\tlw := kube.InternalLW{\n\t\t\t\tClient: fakeCRWatchClient,\n\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\tObjectList: new(velerov1api.PodVolumeBackupList),\n\t\t\t}\n\n\t\t\tpvbInformer := cache.NewSharedIndexInformer(&lw, &velerov1api.PodVolumeBackup{}, 0, cache.Indexers{})\n\n\t\t\tgo pvbInformer.Run(ctx.Done())\n\t\t\trequire.True(t, cache.WaitForCacheSync(ctx.Done(), pvbInformer.HasSynced))\n\n\t\t\tensurer := repository.NewEnsurer(fakeCtrlClient, velerotest.NewLogger(), time.Millisecond)\n\n\t\t\tbackupObj := builder.ForBackup(velerov1api.DefaultNamespace, \"fake-backup\").Result()\n\t\t\tbackupObj.Spec.StorageLocation = test.bsl\n\n\t\t\tfactory := NewBackupperFactory(repository.NewRepoLocker(), ensurer, fakeCtrlClient, pvbInformer, velerotest.NewLogger())\n\t\t\tbp, err := factory.NewBackupper(ctx, log, backupObj, test.uploaderType)\n\n\t\t\trequire.NoError(t, err)\n\n\t\t\tif test.mockGetRepositoryType {\n\t\t\t\tfuncGetRepositoryType = func(string) string { return \"\" }\n\t\t\t} else {\n\t\t\t\tfuncGetRepositoryType = getRepositoryType\n\t\t\t}\n\n\t\t\tpvbs, _, errs := bp.BackupPodVolumes(backupObj, test.sourcePod, test.volumes, nil, velerotest.NewLogger())\n\n\t\t\tif test.errs == nil {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tfor i := 0; i < len(errs); i++ {\n\t\t\t\t\trequire.EqualError(t, errs[i], test.errs[i])\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tassert.Len(t, pvbs, test.pvbs)\n\t\t})\n\t}\n}\n\nfunc TestGetPodVolumeBackupByPodAndVolume(t *testing.T) {\n\tbackupper := &backupper{\n\t\tpvbIndexer: cache.NewIndexer(podVolumeBackupKey, cache.Indexers{\n\t\t\tindexNamePod: podIndexFunc,\n\t\t}),\n\t}\n\n\tobj := &velerov1api.PodVolumeBackup{\n\t\tSpec: velerov1api.PodVolumeBackupSpec{\n\t\t\tPod: corev1api.ObjectReference{\n\t\t\t\tKind: \"Pod\",\n\t\t\t\tNamespace: \"default\",\n\t\t\t\tName: \"pod\",\n\t\t\t},\n\t\t\tVolume: \"volume\",\n\t\t},\n\t}\n\n\terr := backupper.pvbIndexer.Add(obj)\n\trequire.NoError(t, err)\n\n\t// incorrect pod namespace\n\tpvb, err := backupper.GetPodVolumeBackupByPodAndVolume(\"invalid-namespace\", \"pod\", \"volume\")\n\trequire.NoError(t, err)\n\tassert.Nil(t, pvb)\n\n\t// incorrect pod name\n\tpvb, err = backupper.GetPodVolumeBackupByPodAndVolume(\"default\", \"invalid-pod\", \"volume\")\n\trequire.NoError(t, err)\n\tassert.Nil(t, pvb)\n\n\t// incorrect volume\n\tpvb, err = backupper.GetPodVolumeBackupByPodAndVolume(\"default\", \"pod\", \"invalid-volume\")\n\trequire.NoError(t, err)\n\tassert.Nil(t, pvb)\n\n\t// correct pod namespace, name and volume\n\tpvb, err = backupper.GetPodVolumeBackupByPodAndVolume(\"default\", \"pod\", \"volume\")\n\trequire.NoError(t, err)\n\tassert.NotNil(t, pvb)\n}\n\nfunc TestListPodVolumeBackupsByPodp(t *testing.T) {\n\tbackupper := &backupper{\n\t\tpvbIndexer: cache.NewIndexer(cache.MetaNamespaceKeyFunc, cache.Indexers{\n\t\t\tindexNamePod: podIndexFunc,\n\t\t}),\n\t}\n\n\tobj1 := &velerov1api.PodVolumeBackup{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"velero\",\n\t\t\tName: \"pvb1\",\n\t\t},\n\t\tSpec: velerov1api.PodVolumeBackupSpec{\n\t\t\tPod: corev1api.ObjectReference{\n\t\t\t\tKind: \"Pod\",\n\t\t\t\tNamespace: \"default\",\n\t\t\t\tName: \"pod\",\n\t\t\t},\n\t\t},\n\t}\n\tobj2 := &velerov1api.PodVolumeBackup{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"velero\",\n\t\t\tName: \"pvb2\",\n\t\t},\n\t\tSpec: velerov1api.PodVolumeBackupSpec{\n\t\t\tPod: corev1api.ObjectReference{\n\t\t\t\tKind: \"Pod\",\n\t\t\t\tNamespace: \"default\",\n\t\t\t\tName: \"pod\",\n\t\t\t},\n\t\t},\n\t}\n\n\terr := backupper.pvbIndexer.Add(obj1)\n\trequire.NoError(t, err)\n\terr = backupper.pvbIndexer.Add(obj2)\n\trequire.NoError(t, err)\n\n\t// not exist PVBs\n\tpvbs, err := backupper.ListPodVolumeBackupsByPod(\"invalid-namespace\", \"invalid-name\")\n\trequire.NoError(t, err)\n\tassert.Empty(t, pvbs)\n\n\t// exist PVBs\n\tpvbs, err = backupper.ListPodVolumeBackupsByPod(\"default\", \"pod\")\n\trequire.NoError(t, err)\n\tassert.Len(t, pvbs, 2)\n}\n\ntype logHook struct {\n\tentry *logrus.Entry\n}\n\nfunc (l *logHook) Levels() []logrus.Level {\n\treturn []logrus.Level{logrus.ErrorLevel}\n}\nfunc (l *logHook) Fire(entry *logrus.Entry) error {\n\tl.entry = entry\n\treturn nil\n}\n\nfunc TestWaitAllPodVolumesProcessed(t *testing.T) {\n\ttimeoutCtx, cancelFunc := context.WithCancel(t.Context())\n\tcancelFunc()\n\tlog := logrus.New()\n\tpvb := builder.ForPodVolumeBackup(velerov1api.DefaultNamespace, \"pvb\").\n\t\tPodNamespace(\"pod-namespace\").PodName(\"pod-name\").Volume(\"volume\").Result()\n\tcases := []struct {\n\t\tname string\n\t\tctx context.Context\n\t\tpvb *velerov1api.PodVolumeBackup\n\t\tstatusToBeUpdated *velerov1api.PodVolumeBackupStatus\n\t\texpectedErr string\n\t\texpectedPVBPhase velerov1api.PodVolumeBackupPhase\n\t}{\n\t\t{\n\t\t\tname: \"contains no pvb should report no error\",\n\t\t\tctx: timeoutCtx,\n\t\t},\n\t\t{\n\t\t\tname: \"context canceled\",\n\t\t\tctx: timeoutCtx,\n\t\t\tpvb: pvb,\n\t\t\texpectedErr: \"timed out waiting for all PodVolumeBackups to complete\",\n\t\t},\n\t\t{\n\t\t\tname: \"failed pvbs\",\n\t\t\tctx: t.Context(),\n\t\t\tpvb: pvb,\n\t\t\tstatusToBeUpdated: &velerov1api.PodVolumeBackupStatus{\n\t\t\t\tPhase: velerov1api.PodVolumeBackupPhaseFailed,\n\t\t\t\tMessage: \"failed\",\n\t\t\t},\n\t\t\texpectedPVBPhase: velerov1api.PodVolumeBackupPhaseFailed,\n\t\t\texpectedErr: \"pod volume backup failed: failed\",\n\t\t},\n\t\t{\n\t\t\tname: \"completed pvbs\",\n\t\t\tctx: t.Context(),\n\t\t\tpvb: pvb,\n\t\t\tstatusToBeUpdated: &velerov1api.PodVolumeBackupStatus{\n\t\t\t\tPhase: velerov1api.PodVolumeBackupPhaseCompleted,\n\t\t\t\tMessage: \"completed\",\n\t\t\t},\n\t\t\texpectedPVBPhase: velerov1api.PodVolumeBackupPhaseCompleted,\n\t\t},\n\t}\n\n\tfor _, c := range cases {\n\t\tvar objs []ctrlclient.Object\n\t\tif c.pvb != nil {\n\t\t\tobjs = append(objs, c.pvb)\n\t\t}\n\t\tscheme := runtime.NewScheme()\n\t\tvelerov1api.AddToScheme(scheme)\n\t\tclient := ctrlfake.NewClientBuilder().WithScheme(scheme).WithObjects(objs...).Build()\n\n\t\tlw := kube.InternalLW{\n\t\t\tClient: client,\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tObjectList: new(velerov1api.PodVolumeBackupList),\n\t\t}\n\n\t\tinformer := cache.NewSharedIndexInformer(&lw, &velerov1api.PodVolumeBackup{}, 0, cache.Indexers{})\n\n\t\tctx := t.Context()\n\t\tgo informer.Run(ctx.Done())\n\t\trequire.True(t, cache.WaitForCacheSync(ctx.Done(), informer.HasSynced))\n\n\t\tlogger := logrus.New()\n\t\tlogHook := &logHook{}\n\t\tlogger.Hooks.Add(logHook)\n\n\t\tbackuper := newBackupper(c.ctx, log, nil, nil, informer, nil, \"\", &velerov1api.Backup{})\n\t\tif c.pvb != nil {\n\t\t\trequire.NoError(t, backuper.pvbIndexer.Add(c.pvb))\n\t\t\tbackuper.wg.Add(1)\n\t\t}\n\n\t\tif c.statusToBeUpdated != nil {\n\t\t\tpvb := &velerov1api.PodVolumeBackup{}\n\t\t\terr := client.Get(t.Context(), ctrlclient.ObjectKey{Namespace: c.pvb.Namespace, Name: c.pvb.Name}, pvb)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tpvb.Status = *c.statusToBeUpdated\n\t\t\terr = client.Update(t.Context(), pvb)\n\t\t\trequire.NoError(t, err)\n\t\t}\n\n\t\tpvbs := backuper.WaitAllPodVolumesProcessed(logger)\n\n\t\tif c.expectedErr != \"\" {\n\t\t\tassert.Equal(t, c.expectedErr, logHook.entry.Message)\n\t\t} else {\n\t\t\tassert.Nil(t, logHook.entry)\n\t\t}\n\n\t\tif c.expectedPVBPhase != \"\" {\n\t\t\trequire.Len(t, pvbs, 1)\n\t\t\tassert.Equal(t, c.expectedPVBPhase, pvbs[0].Status.Phase)\n\t\t}\n\t}\n}\n\nfunc TestPVCBackupSummary(t *testing.T) {\n\tpbs := NewPVCBackupSummary()\n\tpbs.pvcMap[\"vol-1\"] = builder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").VolumeName(\"pv-1\").Result()\n\tpbs.pvcMap[\"vol-2\"] = builder.ForPersistentVolumeClaim(\"ns-2\", \"pvc-2\").VolumeName(\"pv-2\").Result()\n\n\t// it won't be added if the volme is not in the pvc map.\n\tpbs.addSkipped(\"vol-3\", \"whatever reason\")\n\tassert.Empty(t, pbs.Skipped)\n\tpbs.addBackedup(\"vol-3\")\n\tassert.Empty(t, pbs.Backedup)\n\n\t// only can be added as skipped when it's not in backedup set\n\tpbs.addBackedup(\"vol-1\")\n\tassert.Len(t, pbs.Backedup, 1)\n\tassert.Equal(t, \"pvc-1\", pbs.Backedup[\"vol-1\"].Name)\n\tpbs.addSkipped(\"vol-1\", \"whatever reason\")\n\tassert.Empty(t, pbs.Skipped)\n\tpbs.addSkipped(\"vol-2\", \"vol-2 has to be skipped\")\n\tassert.Len(t, pbs.Skipped, 1)\n\tassert.Equal(t, \"pvc-2\", pbs.Skipped[\"vol-2\"].PVC.Name)\n\n\t// adding a vol as backedup removes it from skipped set\n\tpbs.addBackedup(\"vol-2\")\n\tassert.Empty(t, pbs.Skipped)\n\tassert.Len(t, pbs.Backedup, 2)\n}\n\nfunc TestGetMatchAction_PendingPVC(t *testing.T) {\n\t// Create resource policies that skip Pending/Lost PVCs\n\tresPolicies := &resourcepolicies.ResourcePolicies{\n\t\tVersion: \"v1\",\n\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t{\n\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\"pvcPhase\": []string{\"Pending\", \"Lost\"},\n\t\t\t\t},\n\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\tType: resourcepolicies.Skip,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tpolicies := &resourcepolicies.Policies{}\n\terr := policies.BuildPolicy(resPolicies)\n\trequire.NoError(t, err)\n\n\ttestCases := []struct {\n\t\tname string\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\tvolume *corev1api.Volume\n\t\tpv *corev1api.PersistentVolume\n\t\texpectedAction *resourcepolicies.Action\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"Pending PVC with pvcPhase skip policy should return skip action\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns\", \"pending-pvc\").\n\t\t\t\tStorageClass(\"test-sc\").\n\t\t\t\tPhase(corev1api.ClaimPending).\n\t\t\t\tResult(),\n\t\t\tvolume: &corev1api.Volume{\n\t\t\t\tName: \"test-volume\",\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\tClaimName: \"pending-pvc\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpv: nil,\n\t\t\texpectedAction: &resourcepolicies.Action{Type: resourcepolicies.Skip},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Lost PVC with pvcPhase skip policy should return skip action\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns\", \"lost-pvc\").\n\t\t\t\tStorageClass(\"test-sc\").\n\t\t\t\tPhase(corev1api.ClaimLost).\n\t\t\t\tResult(),\n\t\t\tvolume: &corev1api.Volume{\n\t\t\t\tName: \"test-volume\",\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\tClaimName: \"lost-pvc\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpv: nil,\n\t\t\texpectedAction: &resourcepolicies.Action{Type: resourcepolicies.Skip},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Bound PVC with matching PV should not match pvcPhase policy\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns\", \"bound-pvc\").\n\t\t\t\tStorageClass(\"test-sc\").\n\t\t\t\tVolumeName(\"test-pv\").\n\t\t\t\tPhase(corev1api.ClaimBound).\n\t\t\t\tResult(),\n\t\t\tvolume: &corev1api.Volume{\n\t\t\t\tName: \"test-volume\",\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\tClaimName: \"bound-pvc\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpv: builder.ForPersistentVolume(\"test-pv\").StorageClass(\"test-sc\").Result(),\n\t\t\texpectedAction: nil,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Pending PVC with no matching policy should return nil action\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns\", \"pending-pvc-no-match\").\n\t\t\t\tStorageClass(\"test-sc\").\n\t\t\t\tPhase(corev1api.ClaimPending).\n\t\t\t\tResult(),\n\t\t\tvolume: &corev1api.Volume{\n\t\t\t\tName: \"test-volume\",\n\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\tClaimName: \"pending-pvc-no-match\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpv: nil,\n\t\t\texpectedAction: &resourcepolicies.Action{Type: resourcepolicies.Skip}, // Will match the pvcPhase policy\n\t\t\texpectError: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t// Build fake client with PV if present\n\t\t\tvar objs []runtime.Object\n\t\t\tif tc.pv != nil {\n\t\t\t\tobjs = append(objs, tc.pv)\n\t\t\t}\n\t\t\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, objs...)\n\n\t\t\tb := &backupper{\n\t\t\t\tcrClient: fakeClient,\n\t\t\t}\n\n\t\t\taction, err := b.getMatchAction(policies, tc.pvc, tc.volume)\n\t\t\tif tc.expectError {\n\t\t\t\trequire.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif tc.expectedAction == nil {\n\t\t\t\tassert.Nil(t, action)\n\t\t\t} else {\n\t\t\t\trequire.NotNil(t, action)\n\t\t\t\tassert.Equal(t, tc.expectedAction.Type, action.Type)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetMatchAction_PVCWithoutPVLookupError(t *testing.T) {\n\t// Test that when a PVC has a VolumeName but the PV doesn't exist,\n\t// the function ignores the error and tries to match with PVC only\n\tresPolicies := &resourcepolicies.ResourcePolicies{\n\t\tVersion: \"v1\",\n\t\tVolumePolicies: []resourcepolicies.VolumePolicy{\n\t\t\t{\n\t\t\t\tConditions: map[string]any{\n\t\t\t\t\t\"pvcPhase\": []string{\"Pending\"},\n\t\t\t\t},\n\t\t\t\tAction: resourcepolicies.Action{\n\t\t\t\t\tType: resourcepolicies.Skip,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tpolicies := &resourcepolicies.Policies{}\n\terr := policies.BuildPolicy(resPolicies)\n\trequire.NoError(t, err)\n\n\t// Pending PVC without a matching PV in the cluster\n\tpvc := builder.ForPersistentVolumeClaim(\"ns\", \"pending-pvc\").\n\t\tStorageClass(\"test-sc\").\n\t\tPhase(corev1api.ClaimPending).\n\t\tResult()\n\n\tvolume := &corev1api.Volume{\n\t\tName: \"test-volume\",\n\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\tClaimName: \"pending-pvc\",\n\t\t\t},\n\t\t},\n\t}\n\n\t// Empty client - no PV exists\n\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\tb := &backupper{\n\t\tcrClient: fakeClient,\n\t}\n\n\t// Should succeed even though PV lookup would fail\n\t// because the function ignores PV lookup errors and uses PVC-only matching\n\taction, err := b.getMatchAction(policies, pvc, volume)\n\trequire.NoError(t, err)\n\trequire.NotNil(t, action)\n\tassert.Equal(t, resourcepolicies.Skip, action.Type)\n}\n" }, { "path": "pkg/podvolume/configs/configs.go", "content": "package configs\n\nconst (\n\t// PVCNameAnnotation is the key for the annotation added to\n\t// pod volume backups when they're for a PVC.\n\tPVCNameAnnotation = \"velero.io/pvc-name\"\n\n\t// DefaultVolumesToFsBackup specifies whether pod volume backup should be used, by default, to\n\t// take backup of all pod volumes.\n\tDefaultVolumesToFsBackup = false\n)\n" }, { "path": "pkg/podvolume/mocks/restorer.go", "content": "// Code generated by mockery v2.42.2. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tmock \"github.com/stretchr/testify/mock\"\n\tpodvolume \"github.com/vmware-tanzu/velero/pkg/podvolume\"\n\n\tvolume \"github.com/vmware-tanzu/velero/internal/volume\"\n)\n\n// Restorer is an autogenerated mock type for the Restorer type\ntype Restorer struct {\n\tmock.Mock\n}\n\n// RestorePodVolumes provides a mock function with given fields: _a0, _a1\nfunc (_m *Restorer) RestorePodVolumes(_a0 podvolume.RestoreData, _a1 *volume.RestoreVolumeInfoTracker) []error {\n\tret := _m.Called(_a0, _a1)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for RestorePodVolumes\")\n\t}\n\n\tvar r0 []error\n\tif rf, ok := ret.Get(0).(func(podvolume.RestoreData, *volume.RestoreVolumeInfoTracker) []error); ok {\n\t\tr0 = rf(_a0, _a1)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]error)\n\t\t}\n\t}\n\n\treturn r0\n}\n\n// NewRestorer creates a new instance of Restorer. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\n// The first argument is typically a *testing.T value.\nfunc NewRestorer(t interface {\n\tmock.TestingT\n\tCleanup(func())\n}) *Restorer {\n\tmock := &Restorer{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/podvolume/restore_micro_service.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"sigs.k8s.io/controller-runtime/pkg/cache\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\n\tcachetool \"k8s.io/client-go/tools/cache\"\n)\n\n// RestoreMicroService process data mover restores inside the restore pod\ntype RestoreMicroService struct {\n\tctx context.Context\n\tclient client.Client\n\tkubeClient kubernetes.Interface\n\trepoEnsurer *repository.Ensurer\n\tcredentialGetter *credentials.CredentialGetter\n\tlogger logrus.FieldLogger\n\tdataPathMgr *datapath.Manager\n\teventRecorder kube.EventRecorder\n\n\tnamespace string\n\tpvrName string\n\tpvr *velerov1api.PodVolumeRestore\n\tsourceTargetPath datapath.AccessPoint\n\n\tresultSignal chan dataPathResult\n\n\tpvrInformer cache.Informer\n\tpvrHandler cachetool.ResourceEventHandlerRegistration\n\tnodeName string\n\tcacheDir string\n}\n\nfunc NewRestoreMicroService(ctx context.Context, client client.Client, kubeClient kubernetes.Interface, pvrName string, namespace string, nodeName string,\n\tsourceTargetPath datapath.AccessPoint, dataPathMgr *datapath.Manager, repoEnsurer *repository.Ensurer, cred *credentials.CredentialGetter,\n\tpvrInformer cache.Informer, cacheDir string, log logrus.FieldLogger) *RestoreMicroService {\n\treturn &RestoreMicroService{\n\t\tctx: ctx,\n\t\tclient: client,\n\t\tkubeClient: kubeClient,\n\t\tcredentialGetter: cred,\n\t\tlogger: log,\n\t\trepoEnsurer: repoEnsurer,\n\t\tdataPathMgr: dataPathMgr,\n\t\tnamespace: namespace,\n\t\tpvrName: pvrName,\n\t\tsourceTargetPath: sourceTargetPath,\n\t\tnodeName: nodeName,\n\t\tresultSignal: make(chan dataPathResult),\n\t\tpvrInformer: pvrInformer,\n\t\tcacheDir: cacheDir,\n\t}\n}\n\nfunc (r *RestoreMicroService) Init() error {\n\tr.eventRecorder = kube.NewEventRecorder(r.kubeClient, r.client.Scheme(), r.pvrName, r.nodeName, r.logger)\n\n\thandler, err := r.pvrInformer.AddEventHandler(\n\t\tcachetool.ResourceEventHandlerFuncs{\n\t\t\tUpdateFunc: func(oldObj any, newObj any) {\n\t\t\t\toldPvr := oldObj.(*velerov1api.PodVolumeRestore)\n\t\t\t\tnewPvr := newObj.(*velerov1api.PodVolumeRestore)\n\n\t\t\t\tif newPvr.Name != r.pvrName {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif newPvr.Status.Phase != velerov1api.PodVolumeRestorePhaseInProgress {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif newPvr.Spec.Cancel && !oldPvr.Spec.Cancel {\n\t\t\t\t\tr.cancelPodVolumeRestore(newPvr)\n\t\t\t\t}\n\t\t\t},\n\t\t},\n\t)\n\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error adding PVR handler\")\n\t}\n\n\tr.pvrHandler = handler\n\n\treturn err\n}\n\nfunc (r *RestoreMicroService) RunCancelableDataPath(ctx context.Context) (string, error) {\n\tlog := r.logger.WithFields(logrus.Fields{\n\t\t\"PVR\": r.pvrName,\n\t})\n\n\tpvr := &velerov1api.PodVolumeRestore{}\n\terr := wait.PollUntilContextCancel(ctx, 500*time.Millisecond, true, func(ctx context.Context) (bool, error) {\n\t\terr := r.client.Get(ctx, types.NamespacedName{\n\t\t\tNamespace: r.namespace,\n\t\t\tName: r.pvrName,\n\t\t}, pvr)\n\t\tif apierrors.IsNotFound(err) {\n\t\t\treturn false, nil\n\t\t}\n\n\t\tif err != nil {\n\t\t\treturn true, errors.Wrapf(err, \"error to get PVR %s\", r.pvrName)\n\t\t}\n\n\t\tif pvr.Status.Phase == velerov1api.PodVolumeRestorePhaseInProgress {\n\t\t\treturn true, nil\n\t\t} else {\n\t\t\treturn false, nil\n\t\t}\n\t})\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"Failed to wait PVR\")\n\t\treturn \"\", errors.Wrap(err, \"error waiting for PVR\")\n\t}\n\n\tr.pvr = pvr\n\n\tlog.Info(\"Run cancelable PVR\")\n\n\tcallbacks := datapath.Callbacks{\n\t\tOnCompleted: r.OnPvrCompleted,\n\t\tOnFailed: r.OnPvrFailed,\n\t\tOnCancelled: r.OnPvrCancelled,\n\t\tOnProgress: r.OnPvrProgress,\n\t}\n\n\tfsRestore, err := r.dataPathMgr.CreateFileSystemBR(pvr.Name, podVolumeRequestor, ctx, r.client, pvr.Namespace, callbacks, log)\n\tif err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error to create data path\")\n\t}\n\n\tlog.Debug(\"Async fs br created\")\n\n\tif err := fsRestore.Init(ctx,\n\t\t&datapath.FSBRInitParam{\n\t\t\tBSLName: pvr.Spec.BackupStorageLocation,\n\t\t\tSourceNamespace: pvr.Spec.SourceNamespace,\n\t\t\tUploaderType: pvr.Spec.UploaderType,\n\t\t\tRepositoryType: velerov1api.BackupRepositoryTypeKopia,\n\t\t\tRepoIdentifier: \"\",\n\t\t\tRepositoryEnsurer: r.repoEnsurer,\n\t\t\tCredentialGetter: r.credentialGetter,\n\t\t\tCacheDir: r.cacheDir,\n\t\t}); err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error to initialize data path\")\n\t}\n\n\tlog.Info(\"Async fs br init\")\n\n\tif err := fsRestore.StartRestore(pvr.Spec.SnapshotID, r.sourceTargetPath, pvr.Spec.UploaderSettings); err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error starting data path restore\")\n\t}\n\n\tlog.Info(\"Async fs restore data path started\")\n\tr.eventRecorder.Event(pvr, false, datapath.EventReasonStarted, \"Data path for %s started\", pvr.Name)\n\n\tresult := \"\"\n\tselect {\n\tcase <-ctx.Done():\n\t\terr = errors.New(\"timed out waiting for fs restore to complete\")\n\t\tbreak\n\tcase res := <-r.resultSignal:\n\t\terr = res.err\n\t\tresult = res.result\n\t\tbreak\n\t}\n\n\tif err != nil {\n\t\tlog.WithError(err).Error(\"Async fs restore was not completed\")\n\t}\n\n\tr.eventRecorder.EndingEvent(pvr, false, datapath.EventReasonStopped, \"Data path for %s stopped\", pvr.Name)\n\n\treturn result, err\n}\n\nfunc (r *RestoreMicroService) Shutdown() {\n\tr.eventRecorder.Shutdown()\n\tr.closeDataPath(r.ctx, r.pvrName)\n\n\tif r.pvrHandler != nil {\n\t\tif err := r.pvrInformer.RemoveEventHandler(r.pvrHandler); err != nil {\n\t\t\tr.logger.WithError(err).Warn(\"Failed to remove pod handler\")\n\t\t}\n\t}\n}\n\nvar funcWriteCompletionMark = writeCompletionMark\n\nfunc (r *RestoreMicroService) OnPvrCompleted(ctx context.Context, namespace string, pvrName string, result datapath.Result) {\n\tlog := r.logger.WithField(\"PVR\", pvrName)\n\n\terr := funcWriteCompletionMark(r.pvr, result.Restore, log)\n\tif err != nil {\n\t\tlog.WithError(err).Warnf(\"Failed to write completion mark, restored pod may failed to start\")\n\t}\n\n\trestoreBytes, err := funcMarshal(result.Restore)\n\tif err != nil {\n\t\tlog.WithError(err).Errorf(\"Failed to marshal restore result %v\", result.Restore)\n\t\tr.recordPvrFailed(fmt.Sprintf(\"error marshaling restore result %v\", result.Restore), err)\n\t} else {\n\t\tr.eventRecorder.Event(r.pvr, false, datapath.EventReasonCompleted, string(restoreBytes))\n\t\tr.resultSignal <- dataPathResult{\n\t\t\tresult: string(restoreBytes),\n\t\t}\n\t}\n\n\tlog.Info(\"Async fs restore data path completed\")\n}\n\nfunc (r *RestoreMicroService) recordPvrFailed(msg string, err error) {\n\tevtMsg := fmt.Sprintf(\"%s, error %v\", msg, err)\n\tr.eventRecorder.Event(r.pvr, false, datapath.EventReasonFailed, evtMsg)\n\tr.resultSignal <- dataPathResult{\n\t\terr: errors.Wrap(err, msg),\n\t}\n}\n\nfunc (r *RestoreMicroService) OnPvrFailed(ctx context.Context, namespace string, pvrName string, err error) {\n\tlog := r.logger.WithField(\"PVR\", pvrName)\n\tlog.WithError(err).Error(\"Async fs restore data path failed\")\n\n\tr.recordPvrFailed(fmt.Sprintf(\"Data path for PVR %s failed\", pvrName), err)\n}\n\nfunc (r *RestoreMicroService) OnPvrCancelled(ctx context.Context, namespace string, pvrName string) {\n\tlog := r.logger.WithField(\"PVR\", pvrName)\n\tlog.Warn(\"Async fs restore data path canceled\")\n\n\tr.eventRecorder.Event(r.pvr, false, datapath.EventReasonCancelled, \"Data path for PVR %s canceled\", pvrName)\n\tr.resultSignal <- dataPathResult{\n\t\terr: errors.New(datapath.ErrCancelled),\n\t}\n}\n\nfunc (r *RestoreMicroService) OnPvrProgress(ctx context.Context, namespace string, pvrName string, progress *uploader.Progress) {\n\tlog := r.logger.WithFields(logrus.Fields{\n\t\t\"PVR\": pvrName,\n\t})\n\n\tprogressBytes, err := funcMarshal(progress)\n\tif err != nil {\n\t\tlog.WithError(err).Errorf(\"Failed to marshal progress %v\", progress)\n\t\treturn\n\t}\n\n\tr.eventRecorder.Event(r.pvr, false, datapath.EventReasonProgress, string(progressBytes))\n}\n\nfunc (r *RestoreMicroService) closeDataPath(ctx context.Context, pvrName string) {\n\tfsRestore := r.dataPathMgr.GetAsyncBR(pvrName)\n\tif fsRestore != nil {\n\t\tfsRestore.Close(ctx)\n\t}\n\n\tr.dataPathMgr.RemoveAsyncBR(pvrName)\n}\n\nfunc (r *RestoreMicroService) cancelPodVolumeRestore(pvr *velerov1api.PodVolumeRestore) {\n\tr.logger.WithField(\"PVR\", pvr.Name).Info(\"PVR is being canceled\")\n\n\tr.eventRecorder.Event(pvr, false, datapath.EventReasonCancelling, \"Canceling for PVR %s\", pvr.Name)\n\n\tfsBackup := r.dataPathMgr.GetAsyncBR(pvr.Name)\n\tif fsBackup == nil {\n\t\tr.OnPvrCancelled(r.ctx, pvr.GetNamespace(), pvr.GetName())\n\t} else {\n\t\tfsBackup.Cancel()\n\t}\n}\n\nvar funcRemoveAll = os.RemoveAll\nvar funcMkdirAll = os.MkdirAll\nvar funcWriteFile = os.WriteFile\n\nfunc writeCompletionMark(pvr *velerov1api.PodVolumeRestore, result datapath.RestoreResult, log logrus.FieldLogger) error {\n\tvolumePath := result.Target.ByPath\n\tif volumePath == \"\" {\n\t\treturn errors.New(\"target volume is empty in restore result\")\n\t}\n\n\t// Remove the .velero directory from the restored volume (it may contain done files from previous restores\n\t// of this volume, which we don't want to carry over). If this fails for any reason, log and continue, since\n\t// this is non-essential cleanup (the done files are named based on restore UID and the init container looks\n\t// for the one specific to the restore being executed).\n\tif err := funcRemoveAll(filepath.Join(volumePath, \".velero\")); err != nil {\n\t\tlog.WithError(err).Warnf(\"Failed to remove .velero directory from directory %s\", volumePath)\n\t}\n\n\tif len(pvr.OwnerReferences) == 0 {\n\t\treturn errors.New(\"error finding restore UID\")\n\t}\n\n\trestoreUID := pvr.OwnerReferences[0].UID\n\n\t// Create the .velero directory within the volume dir so we can write a done file\n\t// for this restore.\n\tif err := funcMkdirAll(filepath.Join(volumePath, \".velero\"), 0755); err != nil {\n\t\treturn errors.Wrapf(err, \"error creating .velero directory for done file\")\n\t}\n\n\t// Write a done file with name= into the just-created .velero dir\n\t// within the volume. The velero init container on the pod is waiting\n\t// for this file to exist in each restored volume before completing.\n\tif err := funcWriteFile(filepath.Join(volumePath, \".velero\", string(restoreUID)), nil, 0644); err != nil {\n\t\treturn errors.Wrapf(err, \"error writing done file\")\n\t}\n\n\treturn nil\n}\n" }, { "path": "pkg/podvolume/restore_micro_service_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"sync\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/datapath\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\n\tclientFake \"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tdatapathmockes \"github.com/vmware-tanzu/velero/pkg/datapath/mocks\"\n\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\ntype restoreMsTestHelper struct {\n\teventReason string\n\teventMsg string\n\tmarshalErr error\n\tmarshalBytes []byte\n\twithEvent bool\n\teventLock sync.Mutex\n\twriteCompletionErr error\n}\n\nfunc (rt *restoreMsTestHelper) Event(_ runtime.Object, _ bool, reason string, message string, a ...any) {\n\trt.eventLock.Lock()\n\tdefer rt.eventLock.Unlock()\n\n\trt.withEvent = true\n\trt.eventReason = reason\n\trt.eventMsg = fmt.Sprintf(message, a...)\n}\n\nfunc (rt *restoreMsTestHelper) EndingEvent(_ runtime.Object, _ bool, reason string, message string, a ...any) {\n\trt.eventLock.Lock()\n\tdefer rt.eventLock.Unlock()\n\n\trt.withEvent = true\n\trt.eventReason = reason\n\trt.eventMsg = fmt.Sprintf(message, a...)\n}\nfunc (rt *restoreMsTestHelper) Shutdown() {}\n\nfunc (rt *restoreMsTestHelper) Marshal(v any) ([]byte, error) {\n\tif rt.marshalErr != nil {\n\t\treturn nil, rt.marshalErr\n\t}\n\n\treturn rt.marshalBytes, nil\n}\n\nfunc (rt *restoreMsTestHelper) EventReason() string {\n\trt.eventLock.Lock()\n\tdefer rt.eventLock.Unlock()\n\n\treturn rt.eventReason\n}\n\nfunc (rt *restoreMsTestHelper) EventMessage() string {\n\trt.eventLock.Lock()\n\tdefer rt.eventLock.Unlock()\n\n\treturn rt.eventMsg\n}\n\nfunc (rt *restoreMsTestHelper) WriteCompletionMark(*velerov1api.PodVolumeRestore, datapath.RestoreResult, logrus.FieldLogger) error {\n\treturn rt.writeCompletionErr\n}\n\nfunc TestOnPvrFailed(t *testing.T) {\n\tpvrName := \"fake-pvr\"\n\trt := &restoreMsTestHelper{}\n\n\trs := &RestoreMicroService{\n\t\tpvrName: pvrName,\n\t\tdataPathMgr: datapath.NewManager(1),\n\t\teventRecorder: rt,\n\t\tresultSignal: make(chan dataPathResult),\n\t\tlogger: velerotest.NewLogger(),\n\t}\n\n\texpectedErr := \"Data path for PVR fake-pvr failed: fake-error\"\n\texpectedEventReason := datapath.EventReasonFailed\n\texpectedEventMsg := \"Data path for PVR fake-pvr failed, error fake-error\"\n\n\tgo rs.OnPvrFailed(t.Context(), velerov1api.DefaultNamespace, pvrName, errors.New(\"fake-error\"))\n\n\tresult := <-rs.resultSignal\n\trequire.EqualError(t, result.err, expectedErr)\n\tassert.Equal(t, expectedEventReason, rt.EventReason())\n\tassert.Equal(t, expectedEventMsg, rt.EventMessage())\n}\n\nfunc TestPvrCancelled(t *testing.T) {\n\tpvrName := \"fake-pvr\"\n\trt := &restoreMsTestHelper{}\n\n\trs := RestoreMicroService{\n\t\tpvrName: pvrName,\n\t\tdataPathMgr: datapath.NewManager(1),\n\t\teventRecorder: rt,\n\t\tresultSignal: make(chan dataPathResult),\n\t\tlogger: velerotest.NewLogger(),\n\t}\n\n\texpectedErr := datapath.ErrCancelled\n\texpectedEventReason := datapath.EventReasonCancelled\n\texpectedEventMsg := \"Data path for PVR fake-pvr canceled\"\n\n\tgo rs.OnPvrCancelled(t.Context(), velerov1api.DefaultNamespace, pvrName)\n\n\tresult := <-rs.resultSignal\n\trequire.EqualError(t, result.err, expectedErr)\n\tassert.Equal(t, expectedEventReason, rt.EventReason())\n\tassert.Equal(t, expectedEventMsg, rt.EventMessage())\n}\n\nfunc TestOnPvrCompleted(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\texpectedErr string\n\t\texpectedEventReason string\n\t\texpectedEventMsg string\n\t\tmarshalErr error\n\t\tmarshallStr string\n\t\twriteCompletionErr error\n\t\texpectedLog string\n\t}{\n\t\t{\n\t\t\tname: \"marshal fail\",\n\t\t\tmarshalErr: errors.New(\"fake-marshal-error\"),\n\t\t\texpectedErr: \"error marshaling restore result {{ } 0}: fake-marshal-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tmarshallStr: \"fake-complete-string\",\n\t\t\texpectedEventReason: datapath.EventReasonCompleted,\n\t\t\texpectedEventMsg: \"fake-complete-string\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed but write completion mark fail\",\n\t\t\tmarshallStr: \"fake-complete-string\",\n\t\t\twriteCompletionErr: errors.New(\"fake-write-completion-error\"),\n\t\t\texpectedEventReason: datapath.EventReasonCompleted,\n\t\t\texpectedEventMsg: \"fake-complete-string\",\n\t\t\texpectedLog: \"Failed to write completion mark, restored pod may failed to start\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tpvrName := \"fake-pvr\"\n\n\t\t\trt := &restoreMsTestHelper{\n\t\t\t\tmarshalErr: test.marshalErr,\n\t\t\t\tmarshalBytes: []byte(test.marshallStr),\n\t\t\t\twriteCompletionErr: test.writeCompletionErr,\n\t\t\t}\n\n\t\t\tlogBuffer := []string{}\n\n\t\t\trs := &RestoreMicroService{\n\t\t\t\tdataPathMgr: datapath.NewManager(1),\n\t\t\t\teventRecorder: rt,\n\t\t\t\tresultSignal: make(chan dataPathResult),\n\t\t\t\tlogger: velerotest.NewMultipleLogger(&logBuffer),\n\t\t\t}\n\n\t\t\tfuncMarshal = rt.Marshal\n\t\t\tfuncWriteCompletionMark = rt.WriteCompletionMark\n\n\t\t\tgo rs.OnPvrCompleted(t.Context(), velerov1api.DefaultNamespace, pvrName, datapath.Result{})\n\n\t\t\tresult := <-rs.resultSignal\n\t\t\tif test.marshalErr != nil {\n\t\t\t\tassert.EqualError(t, result.err, test.expectedErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, result.err)\n\t\t\t\tassert.Equal(t, test.expectedEventReason, rt.EventReason())\n\t\t\t\tassert.Equal(t, test.expectedEventMsg, rt.EventMessage())\n\n\t\t\t\tif test.expectedLog != \"\" {\n\t\t\t\t\tassert.Contains(t, logBuffer[0], test.expectedLog)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestOnPvrProgress(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\texpectedErr string\n\t\texpectedEventReason string\n\t\texpectedEventMsg string\n\t\tmarshalErr error\n\t\tmarshallStr string\n\t}{\n\t\t{\n\t\t\tname: \"marshal fail\",\n\t\t\tmarshalErr: errors.New(\"fake-marshal-error\"),\n\t\t\texpectedErr: \"Failed to marshal restore result\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tmarshallStr: \"fake-progress-string\",\n\t\t\texpectedEventReason: datapath.EventReasonProgress,\n\t\t\texpectedEventMsg: \"fake-progress-string\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tpvrName := \"fake-pvr\"\n\n\t\t\trt := &restoreMsTestHelper{\n\t\t\t\tmarshalErr: test.marshalErr,\n\t\t\t\tmarshalBytes: []byte(test.marshallStr),\n\t\t\t}\n\n\t\t\trs := &RestoreMicroService{\n\t\t\t\tdataPathMgr: datapath.NewManager(1),\n\t\t\t\teventRecorder: rt,\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tfuncMarshal = rt.Marshal\n\n\t\t\trs.OnPvrProgress(t.Context(), velerov1api.DefaultNamespace, pvrName, &uploader.Progress{})\n\n\t\t\tif test.marshalErr != nil {\n\t\t\t\tassert.False(t, rt.withEvent)\n\t\t\t} else {\n\t\t\t\tassert.True(t, rt.withEvent)\n\t\t\t\tassert.Equal(t, test.expectedEventReason, rt.EventReason())\n\t\t\t\tassert.Equal(t, test.expectedEventMsg, rt.EventMessage())\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestCancelPodVolumeRestore(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\texpectedEventReason string\n\t\texpectedEventMsg string\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"no fs restore\",\n\t\t\texpectedEventReason: datapath.EventReasonCancelled,\n\t\t\texpectedEventMsg: \"Data path for PVR fake-pvr canceled\",\n\t\t\texpectedErr: datapath.ErrCancelled,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tpvrName := \"fake-pvr\"\n\t\t\tpvr := builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Result()\n\n\t\t\trt := &restoreMsTestHelper{}\n\n\t\t\trs := &RestoreMicroService{\n\t\t\t\tdataPathMgr: datapath.NewManager(1),\n\t\t\t\teventRecorder: rt,\n\t\t\t\tresultSignal: make(chan dataPathResult),\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tgo rs.cancelPodVolumeRestore(pvr)\n\n\t\t\tresult := <-rs.resultSignal\n\n\t\t\trequire.EqualError(t, result.err, test.expectedErr)\n\t\t\tassert.True(t, rt.withEvent)\n\t\t\tassert.Equal(t, test.expectedEventReason, rt.EventReason())\n\t\t\tassert.Equal(t, test.expectedEventMsg, rt.EventMessage())\n\t\t})\n\t}\n}\n\nfunc TestRunCancelableDataPathRestore(t *testing.T) {\n\tpvrName := \"fake-pvr\"\n\tpvr := builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseNew).Result()\n\tpvrInProgress := builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, pvrName).Phase(velerov1api.PodVolumeRestorePhaseInProgress).Result()\n\tctxTimeout, cancel := context.WithTimeout(t.Context(), time.Second)\n\n\ttests := []struct {\n\t\tname string\n\t\tctx context.Context\n\t\tresult *dataPathResult\n\t\tdataPathMgr *datapath.Manager\n\t\tkubeClientObj []runtime.Object\n\t\tinitErr error\n\t\tstartErr error\n\t\tdataPathStarted bool\n\t\texpectedEventMsg string\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"no pvr\",\n\t\t\tctx: ctxTimeout,\n\t\t\texpectedErr: \"error waiting for PVR: context deadline exceeded\",\n\t\t},\n\t\t{\n\t\t\tname: \"pvr not in in-progress\",\n\t\t\tctx: ctxTimeout,\n\t\t\tkubeClientObj: []runtime.Object{pvr},\n\t\t\texpectedErr: \"error waiting for PVR: context deadline exceeded\",\n\t\t},\n\t\t{\n\t\t\tname: \"create data path fail\",\n\t\t\tctx: t.Context(),\n\t\t\tkubeClientObj: []runtime.Object{pvrInProgress},\n\t\t\tdataPathMgr: datapath.NewManager(0),\n\t\t\texpectedErr: \"error to create data path: Concurrent number exceeds\",\n\t\t},\n\t\t{\n\t\t\tname: \"init data path fail\",\n\t\t\tctx: t.Context(),\n\t\t\tkubeClientObj: []runtime.Object{pvrInProgress},\n\t\t\tinitErr: errors.New(\"fake-init-error\"),\n\t\t\texpectedErr: \"error to initialize data path: fake-init-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"start data path fail\",\n\t\t\tctx: t.Context(),\n\t\t\tkubeClientObj: []runtime.Object{pvrInProgress},\n\t\t\tstartErr: errors.New(\"fake-start-error\"),\n\t\t\texpectedErr: \"error starting data path restore: fake-start-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"data path timeout\",\n\t\t\tctx: ctxTimeout,\n\t\t\tkubeClientObj: []runtime.Object{pvrInProgress},\n\t\t\tdataPathStarted: true,\n\t\t\texpectedEventMsg: fmt.Sprintf(\"Data path for %s stopped\", pvrName),\n\t\t\texpectedErr: \"timed out waiting for fs restore to complete\",\n\t\t},\n\t\t{\n\t\t\tname: \"data path returns error\",\n\t\t\tctx: t.Context(),\n\t\t\tkubeClientObj: []runtime.Object{pvrInProgress},\n\t\t\tdataPathStarted: true,\n\t\t\tresult: &dataPathResult{\n\t\t\t\terr: errors.New(\"fake-data-path-error\"),\n\t\t\t},\n\t\t\texpectedEventMsg: fmt.Sprintf(\"Data path for %s stopped\", pvrName),\n\t\t\texpectedErr: \"fake-data-path-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tctx: t.Context(),\n\t\t\tkubeClientObj: []runtime.Object{pvrInProgress},\n\t\t\tdataPathStarted: true,\n\t\t\tresult: &dataPathResult{\n\t\t\t\tresult: \"fake-succeed-result\",\n\t\t\t},\n\t\t\texpectedEventMsg: fmt.Sprintf(\"Data path for %s stopped\", pvrName),\n\t\t},\n\t}\n\n\tscheme := runtime.NewScheme()\n\tvelerov1api.AddToScheme(scheme)\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeClientBuilder := clientFake.NewClientBuilder()\n\t\t\tfakeClientBuilder = fakeClientBuilder.WithScheme(scheme)\n\n\t\t\tfakeClient := fakeClientBuilder.WithRuntimeObjects(test.kubeClientObj...).Build()\n\n\t\t\trt := &restoreMsTestHelper{}\n\n\t\t\trs := &RestoreMicroService{\n\t\t\t\tnamespace: velerov1api.DefaultNamespace,\n\t\t\t\tpvrName: pvrName,\n\t\t\t\tctx: t.Context(),\n\t\t\t\tclient: fakeClient,\n\t\t\t\tdataPathMgr: datapath.NewManager(1),\n\t\t\t\teventRecorder: rt,\n\t\t\t\tresultSignal: make(chan dataPathResult),\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tif test.ctx != nil {\n\t\t\t\trs.ctx = test.ctx\n\t\t\t}\n\n\t\t\tif test.dataPathMgr != nil {\n\t\t\t\trs.dataPathMgr = test.dataPathMgr\n\t\t\t}\n\n\t\t\tdatapath.FSBRCreator = func(string, string, kbclient.Client, string, datapath.Callbacks, logrus.FieldLogger) datapath.AsyncBR {\n\t\t\t\tfsBR := datapathmockes.NewAsyncBR(t)\n\t\t\t\tif test.initErr != nil {\n\t\t\t\t\tfsBR.On(\"Init\", mock.Anything, mock.Anything).Return(test.initErr)\n\t\t\t\t}\n\n\t\t\t\tif test.startErr != nil {\n\t\t\t\t\tfsBR.On(\"Init\", mock.Anything, mock.Anything).Return(nil)\n\t\t\t\t\tfsBR.On(\"StartRestore\", mock.Anything, mock.Anything, mock.Anything).Return(test.startErr)\n\t\t\t\t}\n\n\t\t\t\tif test.dataPathStarted {\n\t\t\t\t\tfsBR.On(\"Init\", mock.Anything, mock.Anything).Return(nil)\n\t\t\t\t\tfsBR.On(\"StartRestore\", mock.Anything, mock.Anything, mock.Anything).Return(nil)\n\t\t\t\t}\n\n\t\t\t\treturn fsBR\n\t\t\t}\n\n\t\t\tif test.result != nil {\n\t\t\t\tgo func() {\n\t\t\t\t\ttime.Sleep(time.Millisecond * 500)\n\t\t\t\t\trs.resultSignal <- *test.result\n\t\t\t\t}()\n\t\t\t}\n\n\t\t\tresult, err := rs.RunCancelableDataPath(test.ctx)\n\n\t\t\tif test.expectedErr != \"\" {\n\t\t\t\trequire.EqualError(t, err, test.expectedErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, test.result.result, result)\n\t\t\t}\n\n\t\t\tif test.expectedEventMsg != \"\" {\n\t\t\t\tassert.True(t, rt.withEvent)\n\t\t\t\tassert.Equal(t, test.expectedEventMsg, rt.EventMessage())\n\t\t\t}\n\t\t})\n\t}\n\n\tcancel()\n}\n\nfunc TestWriteCompletionMark(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvr *velerov1api.PodVolumeRestore\n\t\tresult datapath.RestoreResult\n\t\tfuncRemoveAll func(string) error\n\t\tfuncMkdirAll func(string, os.FileMode) error\n\t\tfuncWriteFile func(string, []byte, os.FileMode) error\n\t\texpectedErr string\n\t\texpectedLog string\n\t}{\n\t\t{\n\t\t\tname: \"no volume path\",\n\t\t\tresult: datapath.RestoreResult{},\n\t\t\texpectedErr: \"target volume is empty in restore result\",\n\t\t},\n\t\t{\n\t\t\tname: \"no owner reference\",\n\t\t\tresult: datapath.RestoreResult{\n\t\t\t\tTarget: datapath.AccessPoint{\n\t\t\t\t\tByPath: \"fake-volume-path\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, \"fake-pvr\").Result(),\n\t\t\tfuncRemoveAll: func(string) error {\n\t\t\t\treturn nil\n\t\t\t},\n\t\t\texpectedErr: \"error finding restore UID\",\n\t\t},\n\t\t{\n\t\t\tname: \"mkdir fail\",\n\t\t\tresult: datapath.RestoreResult{\n\t\t\t\tTarget: datapath.AccessPoint{\n\t\t\t\t\tByPath: \"fake-volume-path\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, \"fake-pvr\").OwnerReference([]metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tUID: \"fake-uid\",\n\t\t\t\t},\n\t\t\t}).Result(),\n\t\t\tfuncRemoveAll: func(string) error {\n\t\t\t\treturn nil\n\t\t\t},\n\t\t\tfuncMkdirAll: func(string, os.FileMode) error {\n\t\t\t\treturn errors.New(\"fake-mk-dir-error\")\n\t\t\t},\n\t\t\texpectedErr: \"error creating .velero directory for done file: fake-mk-dir-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"write file fail\",\n\t\t\tresult: datapath.RestoreResult{\n\t\t\t\tTarget: datapath.AccessPoint{\n\t\t\t\t\tByPath: \"fake-volume-path\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, \"fake-pvr\").OwnerReference([]metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tUID: \"fake-uid\",\n\t\t\t\t},\n\t\t\t}).Result(),\n\t\t\tfuncRemoveAll: func(string) error {\n\t\t\t\treturn nil\n\t\t\t},\n\t\t\tfuncMkdirAll: func(string, os.FileMode) error {\n\t\t\t\treturn nil\n\t\t\t},\n\t\t\tfuncWriteFile: func(string, []byte, os.FileMode) error {\n\t\t\t\treturn errors.New(\"fake-write-file-error\")\n\t\t\t},\n\t\t\texpectedErr: \"error writing done file: fake-write-file-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tresult: datapath.RestoreResult{\n\t\t\t\tTarget: datapath.AccessPoint{\n\t\t\t\t\tByPath: \"fake-volume-path\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, \"fake-pvr\").OwnerReference([]metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tUID: \"fake-uid\",\n\t\t\t\t},\n\t\t\t}).Result(),\n\t\t\tfuncRemoveAll: func(string) error {\n\t\t\t\treturn nil\n\t\t\t},\n\t\t\tfuncMkdirAll: func(string, os.FileMode) error {\n\t\t\t\treturn nil\n\t\t\t},\n\t\t\tfuncWriteFile: func(string, []byte, os.FileMode) error {\n\t\t\t\treturn nil\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"succeed but previous dir is not removed\",\n\t\t\tresult: datapath.RestoreResult{\n\t\t\t\tTarget: datapath.AccessPoint{\n\t\t\t\t\tByPath: \"fake-volume-path\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvr: builder.ForPodVolumeRestore(velerov1api.DefaultNamespace, \"fake-pvr\").OwnerReference([]metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tUID: \"fake-uid\",\n\t\t\t\t},\n\t\t\t}).Result(),\n\t\t\tfuncRemoveAll: func(string) error {\n\t\t\t\treturn errors.New(\"fake-remove-dir-error\")\n\t\t\t},\n\t\t\tfuncMkdirAll: func(string, os.FileMode) error {\n\t\t\t\treturn nil\n\t\t\t},\n\t\t\tfuncWriteFile: func(string, []byte, os.FileMode) error {\n\t\t\t\treturn nil\n\t\t\t},\n\t\t\texpectedLog: \"Failed to remove .velero directory from directory fake-volume-path\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tif test.funcRemoveAll != nil {\n\t\t\t\tfuncRemoveAll = test.funcRemoveAll\n\t\t\t}\n\n\t\t\tif test.funcMkdirAll != nil {\n\t\t\t\tfuncMkdirAll = test.funcMkdirAll\n\t\t\t}\n\n\t\t\tif test.funcWriteFile != nil {\n\t\t\t\tfuncWriteFile = test.funcWriteFile\n\t\t\t}\n\n\t\t\tlogBuffer := \"\"\n\t\t\terr := writeCompletionMark(test.pvr, test.result, velerotest.NewSingleLogger(&logBuffer))\n\n\t\t\tif test.expectedErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t} else {\n\t\t\t\trequire.EqualError(t, err, test.expectedErr)\n\t\t\t}\n\n\t\t\tif test.expectedLog != \"\" {\n\t\t\t\tassert.Contains(t, logBuffer, test.expectedLog)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/podvolume/restorer.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"context\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"k8s.io/client-go/tools/cache\"\n\tctrlcache \"sigs.k8s.io/controller-runtime/pkg/cache\"\n\tctrlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tveleroclient \"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\t\"github.com/vmware-tanzu/velero/pkg/nodeagent\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\tuploaderutil \"github.com/vmware-tanzu/velero/pkg/uploader/util\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\ntype RestoreData struct {\n\tRestore *velerov1api.Restore\n\tPod *corev1api.Pod\n\tPodVolumeBackups []*velerov1api.PodVolumeBackup\n\tSourceNamespace, BackupLocation string\n}\n\n// Restorer can execute pod volume restores of volumes in a pod.\ntype Restorer interface {\n\t// RestorePodVolumes restores all annotated volumes in a pod.\n\tRestorePodVolumes(RestoreData, *volume.RestoreVolumeInfoTracker) []error\n}\n\ntype restorer struct {\n\tctx context.Context\n\trepoLocker *repository.RepoLocker\n\trepoEnsurer *repository.Ensurer\n\tkubeClient kubernetes.Interface\n\tcrClient ctrlclient.Client\n\n\tresultsLock sync.Mutex\n\tresults map[string]chan *velerov1api.PodVolumeRestore\n\tnodeAgentCheck chan error\n\tlog logrus.FieldLogger\n}\n\nfunc newRestorer(\n\tctx context.Context,\n\trepoLocker *repository.RepoLocker,\n\trepoEnsurer *repository.Ensurer,\n\tpvrInformer ctrlcache.Informer,\n\tkubeClient kubernetes.Interface,\n\tcrClient ctrlclient.Client,\n\trestore *velerov1api.Restore,\n\tlog logrus.FieldLogger,\n) *restorer {\n\tr := &restorer{\n\t\tctx: ctx,\n\t\trepoLocker: repoLocker,\n\t\trepoEnsurer: repoEnsurer,\n\t\tkubeClient: kubeClient,\n\t\tcrClient: crClient,\n\n\t\tresults: make(map[string]chan *velerov1api.PodVolumeRestore),\n\t\tlog: log,\n\t}\n\n\t_, _ = pvrInformer.AddEventHandler(\n\t\tcache.ResourceEventHandlerFuncs{\n\t\t\tUpdateFunc: func(oldObj, newObj any) {\n\t\t\t\tpvr := newObj.(*velerov1api.PodVolumeRestore)\n\t\t\t\tpvrOld := oldObj.(*velerov1api.PodVolumeRestore)\n\n\t\t\t\tif pvr.GetLabels()[velerov1api.RestoreUIDLabel] != string(restore.UID) {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif pvr.Status.Phase == pvrOld.Status.Phase {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tif pvr.Status.Phase == velerov1api.PodVolumeRestorePhaseCompleted || pvr.Status.Phase == velerov1api.PodVolumeRestorePhaseFailed || pvr.Status.Phase == velerov1api.PodVolumeRestorePhaseCanceled {\n\t\t\t\t\tr.resultsLock.Lock()\n\t\t\t\t\tdefer r.resultsLock.Unlock()\n\n\t\t\t\t\tresChan, ok := r.results[resultsKey(pvr.Spec.Pod.Namespace, pvr.Spec.Pod.Name)]\n\t\t\t\t\tif !ok {\n\t\t\t\t\t\tlog.Errorf(\"No results channel found for pod %s/%s to send pod volume restore %s/%s on\", pvr.Spec.Pod.Namespace, pvr.Spec.Pod.Name, pvr.Namespace, pvr.Name)\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tresChan <- pvr\n\t\t\t\t}\n\t\t\t},\n\t\t},\n\t)\n\n\treturn r\n}\n\nfunc (r *restorer) RestorePodVolumes(data RestoreData, tracker *volume.RestoreVolumeInfoTracker) []error {\n\tvolumesToRestore := getVolumeBackupInfoForPod(data.PodVolumeBackups, data.Pod, data.SourceNamespace)\n\tif len(volumesToRestore) == 0 {\n\t\treturn nil\n\t}\n\n\tif err := nodeagent.IsRunningOnLinux(r.ctx, r.kubeClient, data.Restore.Namespace); err != nil {\n\t\treturn []error{errors.Wrapf(err, \"error to check node agent status\")}\n\t}\n\n\trepositoryType, err := getVolumesRepositoryType(volumesToRestore)\n\tif err != nil {\n\t\treturn []error{err}\n\t}\n\n\trepo, err := r.repoEnsurer.EnsureRepo(r.ctx, data.Restore.Namespace, data.SourceNamespace, data.BackupLocation, repositoryType)\n\tif err != nil {\n\t\treturn []error{err}\n\t}\n\n\t// get a single non-exclusive lock since we'll wait for all individual\n\t// restores to be complete before releasing it.\n\tr.repoLocker.Lock(repo.Name)\n\tdefer r.repoLocker.Unlock(repo.Name)\n\n\tresultsChan := make(chan *velerov1api.PodVolumeRestore)\n\n\tr.resultsLock.Lock()\n\tr.results[resultsKey(data.Pod.Namespace, data.Pod.Name)] = resultsChan\n\tr.resultsLock.Unlock()\n\n\tr.nodeAgentCheck = make(chan error)\n\n\tvar (\n\t\terrs []error\n\t\tnumRestores int\n\t\tpodVolumes = make(map[string]corev1api.Volume)\n\t)\n\n\t// put the pod's volumes in a map for efficient lookup below\n\tfor _, podVolume := range data.Pod.Spec.Volumes {\n\t\tpodVolumes[podVolume.Name] = podVolume\n\t}\n\n\trepoIdentifier := \"\"\n\tif repositoryType == velerov1api.BackupRepositoryTypeRestic {\n\t\trepoIdentifier = repo.Spec.ResticIdentifier\n\t}\n\n\tfor volume, backupInfo := range volumesToRestore {\n\t\tvolumeObj, ok := podVolumes[volume]\n\t\tvar pvc *corev1api.PersistentVolumeClaim\n\t\tif ok {\n\t\t\tif volumeObj.PersistentVolumeClaim != nil {\n\t\t\t\tpvc = new(corev1api.PersistentVolumeClaim)\n\t\t\t\terr := r.crClient.Get(context.TODO(), ctrlclient.ObjectKey{Namespace: data.Pod.Namespace, Name: volumeObj.PersistentVolumeClaim.ClaimName}, pvc)\n\t\t\t\tif err != nil {\n\t\t\t\t\terrs = append(errs, errors.Wrap(err, \"error getting persistent volume claim for volume\"))\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\tvolumeRestore := newPodVolumeRestore(data.Restore, data.Pod, data.BackupLocation, volume, backupInfo.snapshotID, backupInfo.snapshotSize, repoIdentifier, backupInfo.uploaderType, data.SourceNamespace, pvc)\n\t\tif err := veleroclient.CreateRetryGenerateName(r.crClient, r.ctx, volumeRestore); err != nil {\n\t\t\terrs = append(errs, errors.WithStack(err))\n\t\t\tcontinue\n\t\t}\n\t\tnumRestores++\n\t}\n\n\tcheckCtx, checkCancel := context.WithCancel(context.Background())\n\tgo func() {\n\t\tnodeName := \"\"\n\n\t\tcheckFunc := func(ctx context.Context) (bool, error) {\n\t\t\tnewObj, err := r.kubeClient.CoreV1().Pods(data.Pod.Namespace).Get(ctx, data.Pod.Name, metav1.GetOptions{})\n\t\t\tif err != nil {\n\t\t\t\treturn false, err\n\t\t\t}\n\n\t\t\tnodeName = newObj.Spec.NodeName\n\n\t\t\terr = kube.IsPodScheduled(newObj)\n\t\t\tif err != nil {\n\t\t\t\tr.log.WithField(\"error\", err).Debugf(\"Pod %s/%s is not scheduled yet\", newObj.GetNamespace(), newObj.GetName())\n\t\t\t\treturn false, nil\n\t\t\t}\n\t\t\treturn true, nil\n\t\t}\n\n\t\terr := wait.PollUntilContextTimeout(checkCtx, time.Millisecond*500, time.Minute*10, true, checkFunc)\n\t\tif wait.Interrupted(err) {\n\t\t\tr.log.WithError(err).Error(\"Restoring pod is not scheduled until timeout or cancel, disengage\")\n\t\t} else if err != nil {\n\t\t\tr.log.WithError(err).Error(\"Failed to check node-agent pod status, disengage\")\n\t\t} else {\n\t\t\terr = nodeagent.IsRunningInNode(checkCtx, data.Restore.Namespace, nodeName, r.crClient)\n\t\t\tif err != nil {\n\t\t\t\tr.log.WithField(\"node\", nodeName).WithError(err).Error(\"node-agent pod is not running in node, abort the restore\")\n\t\t\t\tr.nodeAgentCheck <- errors.Wrapf(err, \"node-agent pod is not running in node %s\", nodeName)\n\t\t\t}\n\t\t}\n\t}()\n\nForEachVolume:\n\tfor i := 0; i < numRestores; i++ {\n\t\tselect {\n\t\tcase <-r.ctx.Done():\n\t\t\terrs = append(errs, errors.New(\"timed out waiting for all PodVolumeRestores to complete\"))\n\t\t\tbreak ForEachVolume\n\t\tcase res := <-resultsChan:\n\t\t\tif res.Status.Phase == velerov1api.PodVolumeRestorePhaseFailed {\n\t\t\t\terrs = append(errs, errors.Errorf(\"pod volume restore failed: %s\", res.Status.Message))\n\t\t\t} else if res.Status.Phase == velerov1api.PodVolumeRestorePhaseCanceled {\n\t\t\t\terrs = append(errs, errors.Errorf(\"pod volume restore canceled: %s\", res.Status.Message))\n\t\t\t}\n\t\t\ttracker.TrackPodVolume(res)\n\t\tcase err := <-r.nodeAgentCheck:\n\t\t\terrs = append(errs, err)\n\t\t\tbreak ForEachVolume\n\t\t}\n\t}\n\n\t// This is to prevent the case that resultsChan is signaled before nodeAgentCheck though this is unlikely possible.\n\t// One possible case is that the CR is edited and set to an ending state manually, either completed or failed.\n\t// In this case, we must notify the check routine to stop.\n\tcheckCancel()\n\n\tr.resultsLock.Lock()\n\tdelete(r.results, resultsKey(data.Pod.Namespace, data.Pod.Name))\n\tr.resultsLock.Unlock()\n\n\treturn errs\n}\n\nfunc newPodVolumeRestore(restore *velerov1api.Restore, pod *corev1api.Pod, backupLocation, volume, snapshot string, size int64, repoIdentifier, uploaderType, sourceNamespace string, pvc *corev1api.PersistentVolumeClaim) *velerov1api.PodVolumeRestore {\n\tpvr := &velerov1api.PodVolumeRestore{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: restore.Namespace,\n\t\t\tGenerateName: restore.Name + \"-\",\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\t\t\tKind: \"Restore\",\n\t\t\t\t\tName: restore.Name,\n\t\t\t\t\tUID: restore.UID,\n\t\t\t\t\tController: boolptr.True(),\n\t\t\t\t},\n\t\t\t},\n\t\t\tLabels: map[string]string{\n\t\t\t\tvelerov1api.RestoreNameLabel: label.GetValidName(restore.Name),\n\t\t\t\tvelerov1api.RestoreUIDLabel: string(restore.UID),\n\t\t\t\tvelerov1api.PodUIDLabel: string(pod.UID),\n\t\t\t},\n\t\t},\n\t\tSpec: velerov1api.PodVolumeRestoreSpec{\n\t\t\tPod: corev1api.ObjectReference{\n\t\t\t\tKind: \"Pod\",\n\t\t\t\tNamespace: pod.Namespace,\n\t\t\t\tName: pod.Name,\n\t\t\t\tUID: pod.UID,\n\t\t\t},\n\t\t\tVolume: volume,\n\t\t\tSnapshotID: snapshot,\n\t\t\tSnapshotSize: size,\n\t\t\tBackupStorageLocation: backupLocation,\n\t\t\tRepoIdentifier: repoIdentifier,\n\t\t\tUploaderType: uploaderType,\n\t\t\tSourceNamespace: sourceNamespace,\n\t\t},\n\t}\n\tif pvc != nil {\n\t\t// this label is not used by velero, but useful for debugging.\n\t\tpvr.Labels[velerov1api.PVCUIDLabel] = string(pvc.UID)\n\t}\n\n\tif restore.Spec.UploaderConfig != nil {\n\t\tpvr.Spec.UploaderSettings = uploaderutil.StoreRestoreConfig(restore.Spec.UploaderConfig)\n\t}\n\n\treturn pvr\n}\n\nfunc getVolumesRepositoryType(volumes map[string]volumeBackupInfo) (string, error) {\n\tif len(volumes) == 0 {\n\t\treturn \"\", errors.New(\"empty volume list\")\n\t}\n\n\t// the podVolumeBackups list come from one backup. In one backup, it is impossible that volumes are\n\t// backed up by different uploaders or to different repositories. Asserting this ensures one repo only,\n\t// which will simplify the following logics\n\trepositoryType := \"\"\n\tfor _, backupInfo := range volumes {\n\t\tif backupInfo.repositoryType == \"\" {\n\t\t\treturn \"\", errors.Errorf(\"empty repository type found among volume snapshots, snapshot ID %s, uploader %s\",\n\t\t\t\tbackupInfo.snapshotID, backupInfo.uploaderType)\n\t\t}\n\n\t\tif repositoryType == \"\" {\n\t\t\trepositoryType = backupInfo.repositoryType\n\t\t} else if repositoryType != backupInfo.repositoryType {\n\t\t\treturn \"\", errors.Errorf(\"multiple repository type in one backup, current type %s, differential one [type %s, snapshot ID %s, uploader %s]\",\n\t\t\t\trepositoryType, backupInfo.repositoryType, backupInfo.snapshotID, backupInfo.uploaderType)\n\t\t}\n\t}\n\n\treturn repositoryType, nil\n}\n" }, { "path": "pkg/podvolume/restorer_factory.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"k8s.io/client-go/tools/cache\"\n\tctrlcache \"sigs.k8s.io/controller-runtime/pkg/cache\"\n\tctrlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n)\n\n// RestorerFactory can construct pod volumes restorers.\ntype RestorerFactory interface {\n\t// NewRestorer returns a pod volumes restorer for use during a single Velero restore.\n\tNewRestorer(context.Context, *velerov1api.Restore) (Restorer, error)\n}\n\nfunc NewRestorerFactory(repoLocker *repository.RepoLocker,\n\trepoEnsurer *repository.Ensurer,\n\tkubeClient kubernetes.Interface,\n\tcrClient ctrlclient.Client,\n\tpvrInformer ctrlcache.Informer,\n\tlog logrus.FieldLogger) RestorerFactory {\n\treturn &restorerFactory{\n\t\trepoLocker: repoLocker,\n\t\trepoEnsurer: repoEnsurer,\n\t\tkubeClient: kubeClient,\n\t\tcrClient: crClient,\n\t\tpvrInformer: pvrInformer,\n\t\tlog: log,\n\t}\n}\n\ntype restorerFactory struct {\n\trepoLocker *repository.RepoLocker\n\trepoEnsurer *repository.Ensurer\n\tkubeClient kubernetes.Interface\n\tcrClient ctrlclient.Client\n\tpvrInformer ctrlcache.Informer\n\tlog logrus.FieldLogger\n}\n\nfunc (rf *restorerFactory) NewRestorer(ctx context.Context, restore *velerov1api.Restore) (Restorer, error) {\n\tr := newRestorer(ctx, rf.repoLocker, rf.repoEnsurer, rf.pvrInformer, rf.kubeClient, rf.crClient, restore, rf.log)\n\n\tif !cache.WaitForCacheSync(ctx.Done(), rf.pvrInformer.HasSynced) {\n\t\treturn nil, errors.New(\"timed out waiting for cache to sync\")\n\t}\n\n\treturn r, nil\n}\n" }, { "path": "pkg/podvolume/restorer_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes\"\n\tkubefake \"k8s.io/client-go/kubernetes/fake\"\n\t\"k8s.io/client-go/tools/cache\"\n\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nfunc TestGetVolumesRepositoryType(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tvolumes map[string]volumeBackupInfo\n\t\texpected string\n\t\texpectedErr string\n\t\tprefixOnly bool\n\t}{\n\t\t{\n\t\t\tname: \"empty volume\",\n\t\t\texpectedErr: \"empty volume list\",\n\t\t},\n\t\t{\n\t\t\tname: \"empty repository type, first one\",\n\t\t\tvolumes: map[string]volumeBackupInfo{\n\t\t\t\t\"volume1\": {\"fake-snapshot-id-1\", 0, \"fake-uploader-1\", \"\"},\n\t\t\t\t\"volume2\": {\"\", 0, \"\", \"fake-type\"},\n\t\t\t},\n\t\t\texpectedErr: \"empty repository type found among volume snapshots, snapshot ID fake-snapshot-id-1, uploader fake-uploader-1\",\n\t\t},\n\t\t{\n\t\t\tname: \"empty repository type, last one\",\n\t\t\tvolumes: map[string]volumeBackupInfo{\n\t\t\t\t\"volume1\": {\"\", 0, \"\", \"fake-type\"},\n\t\t\t\t\"volume2\": {\"\", 0, \"\", \"fake-type\"},\n\t\t\t\t\"volume3\": {\"fake-snapshot-id-3\", 0, \"fake-uploader-3\", \"\"},\n\t\t\t},\n\t\t\texpectedErr: \"empty repository type found among volume snapshots, snapshot ID fake-snapshot-id-3, uploader fake-uploader-3\",\n\t\t},\n\t\t{\n\t\t\tname: \"empty repository type, middle one\",\n\t\t\tvolumes: map[string]volumeBackupInfo{\n\t\t\t\t\"volume1\": {\"\", 0, \"\", \"fake-type\"},\n\t\t\t\t\"volume2\": {\"fake-snapshot-id-2\", 0, \"fake-uploader-2\", \"\"},\n\t\t\t\t\"volume3\": {\"\", 0, \"\", \"fake-type\"},\n\t\t\t},\n\t\t\texpectedErr: \"empty repository type found among volume snapshots, snapshot ID fake-snapshot-id-2, uploader fake-uploader-2\",\n\t\t},\n\t\t{\n\t\t\tname: \"mismatch repository type\",\n\t\t\tvolumes: map[string]volumeBackupInfo{\n\t\t\t\t\"volume1\": {\"\", 0, \"\", \"fake-type1\"},\n\t\t\t\t\"volume2\": {\"fake-snapshot-id-2\", 0, \"fake-uploader-2\", \"fake-type2\"},\n\t\t\t},\n\t\t\tprefixOnly: true,\n\t\t\texpectedErr: \"multiple repository type in one backup\",\n\t\t},\n\t\t{\n\t\t\tname: \"success\",\n\t\t\tvolumes: map[string]volumeBackupInfo{\n\t\t\t\t\"volume1\": {\"\", 0, \"\", \"fake-type\"},\n\t\t\t\t\"volume2\": {\"\", 0, \"\", \"fake-type\"},\n\t\t\t\t\"volume3\": {\"\", 0, \"\", \"fake-type\"},\n\t\t\t},\n\t\t\texpected: \"fake-type\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactual, err := getVolumesRepositoryType(tc.volumes)\n\t\t\tassert.Equal(t, tc.expected, actual)\n\n\t\t\tif err != nil {\n\t\t\t\tif tc.prefixOnly {\n\t\t\t\t\terrMsg := err.Error()\n\t\t\t\t\tif len(errMsg) >= len(tc.expectedErr) {\n\t\t\t\t\t\terrMsg = errMsg[0:len(tc.expectedErr)]\n\t\t\t\t\t}\n\n\t\t\t\t\tassert.Equal(t, tc.expectedErr, errMsg)\n\t\t\t\t} else {\n\t\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc createNodeAgentDaemonset() *appsv1api.DaemonSet {\n\tds := &appsv1api.DaemonSet{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"node-agent\",\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t},\n\t}\n\n\treturn ds\n}\n\nfunc createPVRObj(fail bool, index int) *velerov1api.PodVolumeRestore {\n\tpvrObj := &velerov1api.PodVolumeRestore{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\tKind: \"PodVolumeRestore\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: fmt.Sprintf(\"fake-pvr-%d\", index),\n\t\t},\n\t}\n\n\tif fail {\n\t\tpvrObj.Status.Phase = velerov1api.PodVolumeRestorePhaseFailed\n\t\tpvrObj.Status.Message = \"fake-message\"\n\t} else {\n\t\tpvrObj.Status.Phase = velerov1api.PodVolumeRestorePhaseCompleted\n\t}\n\n\treturn pvrObj\n}\n\ntype expectError struct {\n\terr string\n\tprefixOnly bool\n}\n\nfunc TestRestorePodVolumes(t *testing.T) {\n\tscheme := runtime.NewScheme()\n\tvelerov1api.AddToScheme(scheme)\n\tcorev1api.AddToScheme(scheme)\n\n\tctxWithCancel, cancel := context.WithCancel(t.Context())\n\tdefer cancel()\n\n\tfailedPVR := createPVRObj(true, 1)\n\tcompletedPVR := createPVRObj(false, 1)\n\n\ttests := []struct {\n\t\tname string\n\t\tctx context.Context\n\t\tbsl string\n\t\tkubeClientObj []runtime.Object\n\t\tctlClientObj []runtime.Object\n\t\tveleroClientObj []runtime.Object\n\t\tveleroReactors []reactor\n\t\truntimeScheme *runtime.Scheme\n\t\tretPVRs []*velerov1api.PodVolumeRestore\n\t\tpvbs []*velerov1api.PodVolumeBackup\n\t\trestoredPod *corev1api.Pod\n\t\tsourceNamespace string\n\t\terrs []expectError\n\t}{\n\t\t{\n\t\t\tname: \"no volume to restore\",\n\t\t\tpvbs: []*velerov1api.PodVolumeBackup{},\n\t\t\trestoredPod: createPodObj(false, false, false, 1),\n\t\t},\n\t\t{\n\t\t\tname: \"node-agent is not running\",\n\t\t\tpvbs: []*velerov1api.PodVolumeBackup{\n\t\t\t\tcreatePVBObj(true, true, 1, \"kopia\"),\n\t\t\t\tcreatePVBObj(true, true, 2, \"kopia\"),\n\t\t\t},\n\t\t\trestoredPod: createPodObj(false, false, false, 2),\n\t\t\tsourceNamespace: \"fake-ns\",\n\t\t\terrs: []expectError{\n\t\t\t\t{\n\t\t\t\t\terr: \"error to check node agent status: daemonset not found\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"get repository type fail\",\n\t\t\tpvbs: []*velerov1api.PodVolumeBackup{\n\t\t\t\tcreatePVBObj(true, true, 1, \"restic\"),\n\t\t\t\tcreatePVBObj(true, true, 2, \"kopia\"),\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcreateNodeAgentDaemonset(),\n\t\t\t},\n\t\t\trestoredPod: createPodObj(false, false, false, 2),\n\t\t\tsourceNamespace: \"fake-ns\",\n\t\t\terrs: []expectError{\n\t\t\t\t{\n\t\t\t\t\terr: \"multiple repository type in one backup\",\n\t\t\t\t\tprefixOnly: true,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"ensure repo fail\",\n\t\t\tpvbs: []*velerov1api.PodVolumeBackup{\n\t\t\t\tcreatePVBObj(true, true, 1, \"kopia\"),\n\t\t\t\tcreatePVBObj(true, true, 2, \"kopia\"),\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcreateNodeAgentDaemonset(),\n\t\t\t},\n\t\t\trestoredPod: createPodObj(false, false, false, 2),\n\t\t\tsourceNamespace: \"fake-ns\",\n\t\t\truntimeScheme: scheme,\n\t\t\terrs: []expectError{\n\t\t\t\t{\n\t\t\t\t\terr: \"wrong parameters, namespace \\\"fake-ns\\\", backup storage location \\\"\\\", repository type \\\"kopia\\\"\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"get pvc fail\",\n\t\t\tpvbs: []*velerov1api.PodVolumeBackup{\n\t\t\t\tcreatePVBObj(true, true, 1, \"kopia\"),\n\t\t\t\tcreatePVBObj(true, true, 2, \"kopia\"),\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcreateNodeAgentDaemonset(),\n\t\t\t},\n\t\t\tctlClientObj: []runtime.Object{\n\t\t\t\tcreateBackupRepoObj(),\n\t\t\t},\n\t\t\trestoredPod: createPodObj(true, true, true, 2),\n\t\t\tsourceNamespace: \"fake-ns\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\truntimeScheme: scheme,\n\t\t\terrs: []expectError{\n\t\t\t\t{\n\t\t\t\t\terr: \"error getting persistent volume claim for volume: persistentvolumeclaims \\\"fake-pvc-1\\\" not found\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\terr: \"error getting persistent volume claim for volume: persistentvolumeclaims \\\"fake-pvc-2\\\" not found\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"create pvb fail\",\n\t\t\tctx: ctxWithCancel,\n\t\t\tpvbs: []*velerov1api.PodVolumeBackup{\n\t\t\t\tcreatePVBObj(true, true, 1, \"kopia\"),\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcreateNodeAgentDaemonset(),\n\t\t\t\tcreatePVCObj(1),\n\t\t\t},\n\t\t\tctlClientObj: []runtime.Object{\n\t\t\t\tcreateBackupRepoObj(),\n\t\t\t},\n\t\t\trestoredPod: createPodObj(true, true, true, 1),\n\t\t\tsourceNamespace: \"fake-ns\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\truntimeScheme: scheme,\n\t\t\terrs: []expectError{\n\t\t\t\t{\n\t\t\t\t\terr: \"timed out waiting for all PodVolumeRestores to complete\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"create pvb fail\",\n\t\t\tpvbs: []*velerov1api.PodVolumeBackup{\n\t\t\t\tcreatePVBObj(true, true, 1, \"kopia\"),\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcreateNodeAgentDaemonset(),\n\t\t\t\tcreatePVCObj(1),\n\t\t\t},\n\t\t\tctlClientObj: []runtime.Object{\n\t\t\t\tcreateBackupRepoObj(),\n\t\t\t},\n\t\t\trestoredPod: createPodObj(true, true, true, 1),\n\t\t\tsourceNamespace: \"fake-ns\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\truntimeScheme: scheme,\n\t\t\tretPVRs: []*velerov1api.PodVolumeRestore{\n\t\t\t\tfailedPVR,\n\t\t\t},\n\t\t\terrs: []expectError{\n\t\t\t\t{\n\t\t\t\t\terr: \"pod volume restore failed: fake-message\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"node-agent pod is not running\",\n\t\t\tpvbs: []*velerov1api.PodVolumeBackup{\n\t\t\t\tcreatePVBObj(true, true, 1, \"kopia\"),\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcreateNodeAgentDaemonset(),\n\t\t\t\tcreateNodeObj(),\n\t\t\t\tcreatePVCObj(1),\n\t\t\t\tcreatePodObj(true, true, true, 1),\n\t\t\t},\n\t\t\tctlClientObj: []runtime.Object{\n\t\t\t\tcreateBackupRepoObj(),\n\t\t\t},\n\t\t\trestoredPod: createPodObj(true, true, true, 1),\n\t\t\tsourceNamespace: \"fake-ns\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\truntimeScheme: scheme,\n\t\t\terrs: []expectError{\n\t\t\t\t{\n\t\t\t\t\terr: \"node-agent pod is not running in node fake-node-name: daemonset pod not found in running state in node fake-node-name\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"complete\",\n\t\t\tpvbs: []*velerov1api.PodVolumeBackup{\n\t\t\t\tcreatePVBObj(true, true, 1, \"kopia\"),\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tcreateNodeAgentDaemonset(),\n\t\t\t\tcreateNodeObj(),\n\t\t\t\tcreatePVCObj(1),\n\t\t\t\tcreatePodObj(true, true, true, 1),\n\t\t\t\tcreateNodeAgentPodObj(true),\n\t\t\t},\n\t\t\tctlClientObj: []runtime.Object{\n\t\t\t\tcreateBackupRepoObj(),\n\t\t\t},\n\t\t\trestoredPod: createPodObj(true, true, true, 1),\n\t\t\tsourceNamespace: \"fake-ns\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\truntimeScheme: scheme,\n\t\t\tretPVRs: []*velerov1api.PodVolumeRestore{\n\t\t\t\tcompletedPVR,\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\t\t\tif test.ctx != nil {\n\t\t\t\tctx = test.ctx\n\t\t\t}\n\n\t\t\tobjClient := append(test.ctlClientObj, test.kubeClientObj...)\n\t\t\tobjClient = append(objClient, test.veleroClientObj...)\n\n\t\t\tfakeCRClient := velerotest.NewFakeControllerRuntimeClient(t, objClient...)\n\n\t\t\tfakeKubeClient := kubefake.NewSimpleClientset(test.kubeClientObj...)\n\t\t\tvar kubeClient kubernetes.Interface = fakeKubeClient\n\n\t\t\tfakeCRWatchClient := velerotest.NewFakeControllerRuntimeWatchClient(t, test.kubeClientObj...)\n\t\t\tlw := kube.InternalLW{\n\t\t\t\tClient: fakeCRWatchClient,\n\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\tObjectList: new(velerov1api.PodVolumeRestoreList),\n\t\t\t}\n\n\t\t\tpvrInformer := cache.NewSharedIndexInformer(&lw, &velerov1api.PodVolumeBackup{}, 0, cache.Indexers{})\n\n\t\t\tgo pvrInformer.Run(ctx.Done())\n\t\t\trequire.True(t, cache.WaitForCacheSync(ctx.Done(), pvrInformer.HasSynced))\n\n\t\t\tensurer := repository.NewEnsurer(fakeCRClient, velerotest.NewLogger(), time.Millisecond)\n\n\t\t\trestoreObj := builder.ForRestore(velerov1api.DefaultNamespace, \"fake-restore\").Result()\n\n\t\t\tfactory := NewRestorerFactory(repository.NewRepoLocker(), ensurer, kubeClient,\n\t\t\t\tfakeCRClient, pvrInformer, velerotest.NewLogger())\n\t\t\trs, err := factory.NewRestorer(ctx, restoreObj)\n\n\t\t\trequire.NoError(t, err)\n\n\t\t\tgo func() {\n\t\t\t\tif test.ctx != nil {\n\t\t\t\t\ttime.Sleep(time.Second)\n\t\t\t\t\tcancel()\n\t\t\t\t} else if test.retPVRs != nil {\n\t\t\t\t\ttime.Sleep(time.Second)\n\t\t\t\t\tfor _, pvr := range test.retPVRs {\n\t\t\t\t\t\trs.(*restorer).results[resultsKey(test.restoredPod.Namespace, test.restoredPod.Name)] <- pvr\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}()\n\n\t\t\terrs := rs.RestorePodVolumes(RestoreData{\n\t\t\t\tRestore: restoreObj,\n\t\t\t\tPod: test.restoredPod,\n\t\t\t\tPodVolumeBackups: test.pvbs,\n\t\t\t\tSourceNamespace: test.sourceNamespace,\n\t\t\t\tBackupLocation: test.bsl,\n\t\t\t}, volume.NewRestoreVolInfoTracker(restoreObj, logrus.New(), fakeCRClient))\n\n\t\t\tif errs == nil {\n\t\t\t\tassert.Nil(t, test.errs)\n\t\t\t} else {\n\t\t\t\tfor i := 0; i < len(errs); i++ {\n\t\t\t\t\tif test.errs[i].prefixOnly {\n\t\t\t\t\t\terrMsg := errs[i].Error()\n\t\t\t\t\t\tif len(errMsg) >= len(test.errs[i].err) {\n\t\t\t\t\t\t\terrMsg = errMsg[0:len(test.errs[i].err)]\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\tassert.Equal(t, test.errs[i].err, errMsg)\n\t\t\t\t\t} else {\n\t\t\t\t\t\tfor i := 0; i < len(errs); i++ {\n\t\t\t\t\t\t\tj := 0\n\t\t\t\t\t\t\tfor ; j < len(test.errs); j++ {\n\t\t\t\t\t\t\t\terr := errs[i].Error()\n\t\t\t\t\t\t\t\tif err == test.errs[j].err {\n\t\t\t\t\t\t\t\t\tbreak\n\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\tassert.Less(t, j, len(test.errs))\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/podvolume/snaphost_tracker_test.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n)\n\nfunc TestOptoutVolume(t *testing.T) {\n\tpod := builder.ForPod(\"ns-1\", \"pod-1\").Volumes(\n\t\tbuilder.ForVolume(\"pod-vol-1\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\tbuilder.ForVolume(\"pod-vol-2\").PersistentVolumeClaimSource(\"pvc-2\").Result(),\n\t).Result()\n\ttracker := NewTracker()\n\ttracker.Optout(pod, \"pod-vol-1\")\n\tok, pn := tracker.OptedoutByPod(\"ns-1\", \"pvc-1\")\n\tassert.True(t, ok)\n\tassert.Equal(t, \"pod-1\", pn)\n\t// if a volume is tracked for opted out, it can't be tracked as \"tracked\" or \"taken\"\n\ttracker.Track(pod, \"pod-vol-1\")\n\ttracker.Track(pod, \"pod-vol-2\")\n\tassert.False(t, tracker.Has(\"ns-1\", \"pvc-1\"))\n\tassert.True(t, tracker.Has(\"ns-1\", \"pvc-2\"))\n\ttracker.Take(pod, \"pod-vol-1\")\n\ttracker.Take(pod, \"pod-vol-2\")\n\tok1, _ := tracker.TakenForPodVolume(pod, \"pod-vol-1\")\n\tassert.False(t, ok1)\n\tok2, _ := tracker.TakenForPodVolume(pod, \"pod-vol-2\")\n\tassert.True(t, ok2)\n}\n\nfunc TestABC(t *testing.T) {\n\ttracker := NewTracker()\n\tv1, v2 := tracker.OptedoutByPod(\"a\", \"b\")\n\tt.Logf(\"v1: %v, v2: %v\", v1, v2)\n}\n" }, { "path": "pkg/podvolume/snapshot_tracker.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"fmt\"\n\t\"sync\"\n\n\tcorev1api \"k8s.io/api/core/v1\"\n)\n\n// Tracker keeps track of persistent volume claims that have been handled\n// via pod volume backup.\ntype Tracker struct {\n\tpvcs map[string]pvcSnapshotStatus\n\tpvcPod map[string]string\n\t*sync.RWMutex\n}\n\ntype pvcSnapshotStatus int\n\nconst (\n\tpvcSnapshotStatusNotTracked pvcSnapshotStatus = -1\n\tpvcSnapshotStatusTracked pvcSnapshotStatus = iota\n\tpvcSnapshotStatusTaken\n\tpvcSnapshotStatusOptedout\n)\n\nfunc NewTracker() *Tracker {\n\treturn &Tracker{\n\t\tpvcs: make(map[string]pvcSnapshotStatus),\n\t\t// key: pvc ns/name, value: pod name\n\t\tpvcPod: make(map[string]string),\n\t\tRWMutex: &sync.RWMutex{},\n\t}\n}\n\n// Track indicates a volume from a pod should be snapshotted by pod volume backup.\nfunc (t *Tracker) Track(pod *corev1api.Pod, volumeName string) {\n\tt.recordStatus(pod, volumeName, pvcSnapshotStatusTracked, pvcSnapshotStatusNotTracked)\n}\n\n// Take indicates a volume from a pod has been taken by pod volume backup.\nfunc (t *Tracker) Take(pod *corev1api.Pod, volumeName string) {\n\tt.recordStatus(pod, volumeName, pvcSnapshotStatusTaken, pvcSnapshotStatusTracked)\n}\n\n// Optout indicates a volume from a pod has been opted out by pod's annotation\nfunc (t *Tracker) Optout(pod *corev1api.Pod, volumeName string) {\n\tt.recordStatus(pod, volumeName, pvcSnapshotStatusOptedout, pvcSnapshotStatusNotTracked)\n}\n\n// OptedoutByPod returns true if the PVC with the specified namespace and name has been opted out by the pod. The\n// second return value is the name of the pod which has the annotation that opted out the volume/pvc\nfunc (t *Tracker) OptedoutByPod(namespace, name string) (bool, string) {\n\tt.RLock()\n\tdefer t.RUnlock()\n\tstatus, found := t.pvcs[key(namespace, name)]\n\n\tif !found || status != pvcSnapshotStatusOptedout {\n\t\treturn false, \"\"\n\t}\n\treturn true, t.pvcPod[key(namespace, name)]\n}\n\n// if the volume is a PVC, record the status and the related pod\nfunc (t *Tracker) recordStatus(pod *corev1api.Pod, volumeName string, status pvcSnapshotStatus, preReqStatus pvcSnapshotStatus) {\n\tt.Lock()\n\tdefer t.Unlock()\n\tfor _, volume := range pod.Spec.Volumes {\n\t\tif volume.Name == volumeName {\n\t\t\tif volume.PersistentVolumeClaim != nil {\n\t\t\t\tt.pvcPod[key(pod.Namespace, volume.PersistentVolumeClaim.ClaimName)] = pod.Name\n\t\t\t\tcurrStatus, ok := t.pvcs[key(pod.Namespace, volume.PersistentVolumeClaim.ClaimName)]\n\t\t\t\tif !ok {\n\t\t\t\t\tcurrStatus = pvcSnapshotStatusNotTracked\n\t\t\t\t}\n\t\t\t\tif currStatus == preReqStatus {\n\t\t\t\t\tt.pvcs[key(pod.Namespace, volume.PersistentVolumeClaim.ClaimName)] = status\n\t\t\t\t}\n\t\t\t}\n\t\t\tbreak\n\t\t}\n\t}\n}\n\n// Has returns true if the PVC with the specified namespace and name has been tracked.\nfunc (t *Tracker) Has(namespace, name string) bool {\n\tt.RLock()\n\tdefer t.RUnlock()\n\tstatus, found := t.pvcs[key(namespace, name)]\n\treturn found && (status == pvcSnapshotStatusTracked || status == pvcSnapshotStatusTaken)\n}\n\n// TakenForPodVolume returns true and the PVC's name if the pod volume with the specified name uses a\n// PVC and that PVC has been taken by pod volume backup.\nfunc (t *Tracker) TakenForPodVolume(pod *corev1api.Pod, volume string) (bool, string) {\n\tt.RLock()\n\tdefer t.RUnlock()\n\tfor _, podVolume := range pod.Spec.Volumes {\n\t\tif podVolume.Name != volume {\n\t\t\tcontinue\n\t\t}\n\n\t\tif podVolume.PersistentVolumeClaim == nil {\n\t\t\treturn false, \"\"\n\t\t}\n\n\t\tstatus, found := t.pvcs[key(pod.Namespace, podVolume.PersistentVolumeClaim.ClaimName)]\n\t\tif !found {\n\t\t\treturn false, \"\"\n\t\t}\n\n\t\tif status != pvcSnapshotStatusTaken {\n\t\t\treturn false, \"\"\n\t\t}\n\n\t\treturn true, podVolume.PersistentVolumeClaim.ClaimName\n\t}\n\n\treturn false, \"\"\n}\n\nfunc key(namespace, name string) string {\n\treturn fmt.Sprintf(\"%s/%s\", namespace, name)\n}\n" }, { "path": "pkg/podvolume/util.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/podvolume/configs\"\n\trepotypes \"github.com/vmware-tanzu/velero/pkg/repository/types\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n)\n\nconst (\n\t// Deprecated.\n\t//\n\t// TODO(2.0): remove\n\tpodAnnotationPrefix = \"snapshot.velero.io/\"\n)\n\n// volumeBackupInfo describes the backup info of a volume backed up by PodVolumeBackups\ntype volumeBackupInfo struct {\n\tsnapshotID string\n\tsnapshotSize int64\n\tuploaderType string\n\trepositoryType string\n}\n\n// GetVolumeBackupsForPod returns a map, of volume name -> snapshot id,\n// of the PodVolumeBackups that exist for the provided pod.\nfunc GetVolumeBackupsForPod(podVolumeBackups []*velerov1api.PodVolumeBackup, pod *corev1api.Pod, sourcePodNs string) map[string]string {\n\tvolumeBkInfo := getVolumeBackupInfoForPod(podVolumeBackups, pod, sourcePodNs)\n\tif volumeBkInfo == nil {\n\t\treturn nil\n\t}\n\n\tvolumes := make(map[string]string)\n\tfor k, v := range volumeBkInfo {\n\t\tvolumes[k] = v.snapshotID\n\t}\n\n\treturn volumes\n}\n\n// GetPvbRepositoryType returns the repositoryType according to the PVB information\nfunc GetPvbRepositoryType(pvb *velerov1api.PodVolumeBackup) string {\n\treturn getRepositoryType(pvb.Spec.UploaderType)\n}\n\n// GetPvrRepositoryType returns the repositoryType according to the PVR information\nfunc GetPvrRepositoryType(pvr *velerov1api.PodVolumeRestore) string {\n\treturn getRepositoryType(pvr.Spec.UploaderType)\n}\n\n// getVolumeBackupInfoForPod returns a map, of volume name -> VolumeBackupInfo,\n// of the PodVolumeBackups that exist for the provided pod.\nfunc getVolumeBackupInfoForPod(podVolumeBackups []*velerov1api.PodVolumeBackup, pod *corev1api.Pod, sourcePodNs string) map[string]volumeBackupInfo {\n\tvolumes := make(map[string]volumeBackupInfo)\n\n\tfor _, pvb := range podVolumeBackups {\n\t\tif !isPVBMatchPod(pvb, pod.GetName(), sourcePodNs) {\n\t\t\tcontinue\n\t\t}\n\n\t\t// skip PVBs without a snapshot ID since there's nothing\n\t\t// to restore (they could be failed, or for empty volumes).\n\t\tif pvb.Status.SnapshotID == \"\" {\n\t\t\tcontinue\n\t\t}\n\n\t\t// If the volume came from a projected or DownwardAPI source, skip its restore.\n\t\t// This allows backups affected by https://github.com/vmware-tanzu/velero/issues/3863\n\t\t// or https://github.com/vmware-tanzu/velero/issues/4053 to be restored successfully.\n\t\tif volumeHasNonRestorableSource(pvb.Spec.Volume, pod.Spec.Volumes) {\n\t\t\tcontinue\n\t\t}\n\n\t\tvolumes[pvb.Spec.Volume] = volumeBackupInfo{\n\t\t\tsnapshotID: pvb.Status.SnapshotID,\n\t\t\tsnapshotSize: pvb.Status.Progress.TotalBytes,\n\t\t\tuploaderType: getUploaderTypeOrDefault(pvb.Spec.UploaderType),\n\t\t\trepositoryType: getRepositoryType(pvb.Spec.UploaderType),\n\t\t}\n\t}\n\n\tif len(volumes) > 0 {\n\t\treturn volumes\n\t}\n\n\tfromAnnntation := getPodSnapshotAnnotations(pod)\n\tif fromAnnntation == nil {\n\t\treturn nil\n\t}\n\n\tfor k, v := range fromAnnntation {\n\t\tvolumes[k] = volumeBackupInfo{v, 0, uploader.ResticType, velerov1api.BackupRepositoryTypeRestic}\n\t}\n\n\treturn volumes\n}\n\n// GetSnapshotIdentifier returns the snapshots represented by SnapshotIdentifier for the given PVBs\nfunc GetSnapshotIdentifier(podVolumeBackups *velerov1api.PodVolumeBackupList) map[string][]repotypes.SnapshotIdentifier {\n\tres := map[string][]repotypes.SnapshotIdentifier{}\n\tfor _, item := range podVolumeBackups.Items {\n\t\tif item.Status.SnapshotID == \"\" {\n\t\t\tcontinue\n\t\t}\n\n\t\tif res[item.Spec.Pod.Namespace] == nil {\n\t\t\tres[item.Spec.Pod.Namespace] = []repotypes.SnapshotIdentifier{}\n\t\t}\n\n\t\tsnapshots := res[item.Spec.Pod.Namespace]\n\n\t\tsnapshots = append(snapshots, repotypes.SnapshotIdentifier{\n\t\t\tVolumeNamespace: item.Spec.Pod.Namespace,\n\t\t\tBackupStorageLocation: item.Spec.BackupStorageLocation,\n\t\t\tSnapshotID: item.Status.SnapshotID,\n\t\t\tRepositoryType: getRepositoryType(item.Spec.UploaderType),\n\t\t\tUploaderType: item.Spec.UploaderType,\n\t\t\tSource: item.Status.Path,\n\t\t\tRepoIdentifier: item.Spec.RepoIdentifier,\n\t\t})\n\n\t\tres[item.Spec.Pod.Namespace] = snapshots\n\t}\n\n\treturn res\n}\n\nfunc GetRealSource(pvb *velerov1api.PodVolumeBackup) string {\n\tpvcName := \"\"\n\tif pvb.Annotations != nil {\n\t\tpvcName = pvb.Annotations[configs.PVCNameAnnotation]\n\t}\n\n\tif pvcName != \"\" {\n\t\treturn fmt.Sprintf(\"%s/%s/%s\", pvb.Spec.Pod.Namespace, pvb.Spec.Pod.Name, pvcName)\n\t} else {\n\t\treturn fmt.Sprintf(\"%s/%s/%s\", pvb.Spec.Pod.Namespace, pvb.Spec.Pod.Name, pvb.Spec.Volume)\n\t}\n}\n\nfunc getUploaderTypeOrDefault(uploaderType string) string {\n\tif uploaderType != \"\" {\n\t\treturn uploaderType\n\t}\n\treturn uploader.ResticType\n}\n\n// getRepositoryType returns the hardcode repositoryType for different backup methods - Restic or Kopia,uploaderType\n// indicates the method.\n// For Restic backup method, it is always hardcode to BackupRepositoryTypeRestic, never changed.\n// For Kopia backup method, this means we hardcode repositoryType as BackupRepositoryTypeKopia for Unified Repo,\n// at present (Kopia backup method is using Unified Repo). However, it doesn't mean we could deduce repositoryType\n// from uploaderType for Unified Repo.\n// TODO: post v1.10, refactor this function for Kopia backup method. In future, when we have multiple implementations of\n// Unified Repo (besides Kopia), we will add the repositoryType to BSL, because by then, we are not able to hardcode\n// the repositoryType to BackupRepositoryTypeKopia for Unified Repo.\nfunc getRepositoryType(uploaderType string) string {\n\tswitch uploaderType {\n\tcase \"\", uploader.ResticType:\n\t\treturn velerov1api.BackupRepositoryTypeRestic\n\tcase uploader.KopiaType:\n\t\treturn velerov1api.BackupRepositoryTypeKopia\n\tdefault:\n\t\treturn \"\"\n\t}\n}\n\nfunc isPVBMatchPod(pvb *velerov1api.PodVolumeBackup, podName string, namespace string) bool {\n\treturn podName == pvb.Spec.Pod.Name && namespace == pvb.Spec.Pod.Namespace\n}\n\n// volumeHasNonRestorableSource checks if the given volume exists in the list of podVolumes\n// and returns true if the volume's source is not restorable. This is true for volumes with\n// a Projected or DownwardAPI source.\nfunc volumeHasNonRestorableSource(volumeName string, podVolumes []corev1api.Volume) bool {\n\tvar volume corev1api.Volume\n\tfor _, v := range podVolumes {\n\t\tif v.Name == volumeName {\n\t\t\tvolume = v\n\t\t\tbreak\n\t\t}\n\t}\n\treturn volume.Projected != nil || volume.DownwardAPI != nil\n}\n\n// getPodSnapshotAnnotations returns a map, of volume name -> snapshot id,\n// of all snapshots for this pod.\n// TODO(2.0) to remove\n// Deprecated: we will stop using pod annotations to record pod volume snapshot IDs after they're taken,\n// therefore we won't need to check if these annotations exist.\nfunc getPodSnapshotAnnotations(obj metav1.Object) map[string]string {\n\tvar res map[string]string\n\n\tinsertSafe := func(k, v string) {\n\t\tif res == nil {\n\t\t\tres = make(map[string]string)\n\t\t}\n\t\tres[k] = v\n\t}\n\n\tfor k, v := range obj.GetAnnotations() {\n\t\tif strings.HasPrefix(k, podAnnotationPrefix) {\n\t\t\tinsertSafe(k[len(podAnnotationPrefix):], v)\n\t\t}\n\t}\n\n\treturn res\n}\n" }, { "path": "pkg/podvolume/util_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n)\n\nfunc TestGetVolumeBackupsForPod(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpodVolumeBackups []*velerov1api.PodVolumeBackup\n\t\tpodVolumes []corev1api.Volume\n\t\tpodAnnotations map[string]string\n\t\tpodName string\n\t\tsourcePodNs string\n\t\texpected map[string]string\n\t}{\n\t\t{\n\t\t\tname: \"nil annotations results in no volume backups returned\",\n\t\t\tpodAnnotations: nil,\n\t\t\texpected: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"empty annotations results in no volume backups returned\",\n\t\t\tpodAnnotations: make(map[string]string),\n\t\t\texpected: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"pod annotations with no snapshot annotation prefix results in no volume backups returned\",\n\t\t\tpodAnnotations: map[string]string{\"foo\": \"bar\"},\n\t\t\texpected: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"pod annotation with only snapshot annotation prefix, results in volume backup with empty volume key\",\n\t\t\tpodAnnotations: map[string]string{podAnnotationPrefix: \"snapshotID\"},\n\t\t\texpected: map[string]string{\"\": \"snapshotID\"},\n\t\t},\n\t\t{\n\t\t\tname: \"pod annotation with snapshot annotation prefix results in volume backup with volume name and snapshot ID\",\n\t\t\tpodAnnotations: map[string]string{podAnnotationPrefix + \"volume\": \"snapshotID\"},\n\t\t\texpected: map[string]string{\"volume\": \"snapshotID\"},\n\t\t},\n\t\t{\n\t\t\tname: \"only pod annotations with snapshot annotation prefix are considered\",\n\t\t\tpodAnnotations: map[string]string{\"x\": \"y\", podAnnotationPrefix + \"volume1\": \"snapshot1\", podAnnotationPrefix + \"volume2\": \"snapshot2\"},\n\t\t\texpected: map[string]string{\"volume1\": \"snapshot1\", \"volume2\": \"snapshot2\"},\n\t\t},\n\t\t{\n\t\t\tname: \"pod annotations are not considered if PVBs are provided\",\n\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-1\").PodName(\"TestPod\").PodNamespace(\"TestNS\").SnapshotID(\"snapshot1\").Volume(\"pvbtest1-foo\").Result(),\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-2\").PodName(\"TestPod\").PodNamespace(\"TestNS\").SnapshotID(\"snapshot2\").Volume(\"pvbtest2-abc\").Result(),\n\t\t\t},\n\t\t\tpodName: \"TestPod\",\n\t\t\tsourcePodNs: \"TestNS\",\n\t\t\tpodAnnotations: map[string]string{\"x\": \"y\", podAnnotationPrefix + \"foo\": \"bar\", podAnnotationPrefix + \"abc\": \"123\"},\n\t\t\texpected: map[string]string{\"pvbtest1-foo\": \"snapshot1\", \"pvbtest2-abc\": \"snapshot2\"},\n\t\t},\n\t\t{\n\t\t\tname: \"volume backups are returned even if no pod annotations are present\",\n\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-1\").PodName(\"TestPod\").PodNamespace(\"TestNS\").SnapshotID(\"snapshot1\").Volume(\"pvbtest1-foo\").Result(),\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-2\").PodName(\"TestPod\").PodNamespace(\"TestNS\").SnapshotID(\"snapshot2\").Volume(\"pvbtest2-abc\").Result(),\n\t\t\t},\n\t\t\tpodName: \"TestPod\",\n\t\t\tsourcePodNs: \"TestNS\",\n\t\t\texpected: map[string]string{\"pvbtest1-foo\": \"snapshot1\", \"pvbtest2-abc\": \"snapshot2\"},\n\t\t},\n\t\t{\n\t\t\tname: \"only volumes from PVBs with snapshot IDs are returned\",\n\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-1\").PodName(\"TestPod\").PodNamespace(\"TestNS\").SnapshotID(\"snapshot1\").Volume(\"pvbtest1-foo\").Result(),\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-2\").PodName(\"TestPod\").PodNamespace(\"TestNS\").SnapshotID(\"snapshot2\").Volume(\"pvbtest2-abc\").Result(),\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-3\").PodName(\"TestPod\").PodNamespace(\"TestNS\").Volume(\"pvbtest3-foo\").Result(),\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-4\").PodName(\"TestPod\").PodNamespace(\"TestNS\").Volume(\"pvbtest4-abc\").Result(),\n\t\t\t},\n\t\t\tpodName: \"TestPod\",\n\t\t\tsourcePodNs: \"TestNS\",\n\t\t\texpected: map[string]string{\"pvbtest1-foo\": \"snapshot1\", \"pvbtest2-abc\": \"snapshot2\"},\n\t\t},\n\t\t{\n\t\t\tname: \"only volumes from PVBs for the given pod are returned\",\n\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-1\").PodName(\"TestPod\").PodNamespace(\"TestNS\").SnapshotID(\"snapshot1\").Volume(\"pvbtest1-foo\").Result(),\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-2\").PodName(\"TestPod\").PodNamespace(\"TestNS\").SnapshotID(\"snapshot2\").Volume(\"pvbtest2-abc\").Result(),\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-3\").PodName(\"TestAnotherPod\").SnapshotID(\"snapshot3\").Volume(\"pvbtest3-xyz\").Result(),\n\t\t\t},\n\t\t\tpodName: \"TestPod\",\n\t\t\tsourcePodNs: \"TestNS\",\n\t\t\texpected: map[string]string{\"pvbtest1-foo\": \"snapshot1\", \"pvbtest2-abc\": \"snapshot2\"},\n\t\t},\n\t\t{\n\t\t\tname: \"only volumes from PVBs which match the pod name and source pod namespace are returned\",\n\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-1\").PodName(\"TestPod\").PodNamespace(\"TestNS\").SnapshotID(\"snapshot1\").Volume(\"pvbtest1-foo\").Result(),\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-2\").PodName(\"TestAnotherPod\").PodNamespace(\"TestNS\").SnapshotID(\"snapshot2\").Volume(\"pvbtest2-abc\").Result(),\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-3\").PodName(\"TestPod\").PodNamespace(\"TestAnotherNS\").SnapshotID(\"snapshot3\").Volume(\"pvbtest3-xyz\").Result(),\n\t\t\t},\n\t\t\tpodName: \"TestPod\",\n\t\t\tsourcePodNs: \"TestNS\",\n\t\t\texpected: map[string]string{\"pvbtest1-foo\": \"snapshot1\"},\n\t\t},\n\t\t{\n\t\t\tname: \"volumes from PVBs that correspond to a pod volume from a projected source are not returned\",\n\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-1\").PodName(\"TestPod\").PodNamespace(\"TestNS\").SnapshotID(\"snapshot1\").Volume(\"pvb-non-projected\").Result(),\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-1\").PodName(\"TestPod\").PodNamespace(\"TestNS\").SnapshotID(\"snapshot2\").Volume(\"pvb-projected\").Result(),\n\t\t\t},\n\t\t\tpodVolumes: []corev1api.Volume{\n\t\t\t\t{\n\t\t\t\t\tName: \"pvb-non-projected\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"pvb-projected\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tProjected: &corev1api.ProjectedVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpodName: \"TestPod\",\n\t\t\tsourcePodNs: \"TestNS\",\n\t\t\texpected: map[string]string{\"pvb-non-projected\": \"snapshot1\"},\n\t\t},\n\t\t{\n\t\t\tname: \"volumes from PVBs that correspond to a pod volume from a DownwardAPI source are not returned\",\n\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-1\").PodName(\"TestPod\").PodNamespace(\"TestNS\").SnapshotID(\"snapshot1\").Volume(\"pvb-non-downwardapi\").Result(),\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-1\").PodName(\"TestPod\").PodNamespace(\"TestNS\").SnapshotID(\"snapshot2\").Volume(\"pvb-downwardapi\").Result(),\n\t\t\t},\n\t\t\tpodVolumes: []corev1api.Volume{\n\t\t\t\t{\n\t\t\t\t\tName: \"pvb-non-downwardapi\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"pvb-downwardapi\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tDownwardAPI: &corev1api.DownwardAPIVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpodName: \"TestPod\",\n\t\t\tsourcePodNs: \"TestNS\",\n\t\t\texpected: map[string]string{\"pvb-non-downwardapi\": \"snapshot1\"},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tpod := &corev1api.Pod{}\n\t\t\tpod.Annotations = test.podAnnotations\n\t\t\tpod.Name = test.podName\n\t\t\tpod.Spec.Volumes = test.podVolumes\n\n\t\t\tres := GetVolumeBackupsForPod(test.podVolumeBackups, pod, test.sourcePodNs)\n\t\t\tassert.Equal(t, test.expected, res)\n\t\t})\n\t}\n}\n\nfunc TestVolumeHasNonRestorableSource(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tvolumeName string\n\t\tpodVolumes []corev1api.Volume\n\t\texpected bool\n\t}{\n\t\t{\n\t\t\tname: \"volume name not in list of volumes\",\n\t\t\tvolumeName: \"missing-volume\",\n\t\t\tpodVolumes: []corev1api.Volume{\n\t\t\t\t{\n\t\t\t\t\tName: \"restorable\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"projected\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tProjected: &corev1api.ProjectedVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"downwardapi\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tDownwardAPI: &corev1api.DownwardAPIVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"volume name in list of volumes but not projected or DownwardAPI\",\n\t\t\tvolumeName: \"restorable\",\n\t\t\tpodVolumes: []corev1api.Volume{\n\t\t\t\t{\n\t\t\t\t\tName: \"restorable\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"projected\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tProjected: &corev1api.ProjectedVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"downwardapi\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tDownwardAPI: &corev1api.DownwardAPIVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"volume name in list of volumes and projected\",\n\t\t\tvolumeName: \"projected\",\n\t\t\tpodVolumes: []corev1api.Volume{\n\t\t\t\t{\n\t\t\t\t\tName: \"restorable\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"projected\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tProjected: &corev1api.ProjectedVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"downwardapi\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tDownwardAPI: &corev1api.DownwardAPIVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"volume name in list of volumes and is a DownwardAPI volume\",\n\t\t\tvolumeName: \"downwardapi\",\n\t\t\tpodVolumes: []corev1api.Volume{\n\t\t\t\t{\n\t\t\t\t\tName: \"restorable\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"projected\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tProjected: &corev1api.ProjectedVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"downwardapi\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tDownwardAPI: &corev1api.DownwardAPIVolumeSource{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: true,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactual := volumeHasNonRestorableSource(tc.volumeName, tc.podVolumes)\n\t\t\tassert.Equal(t, tc.expected, actual)\n\t\t})\n\t}\n}\n\nfunc TestGetRealSource(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tpvb *velerov1api.PodVolumeBackup\n\t\texpected string\n\t}{\n\t\t{\n\t\t\tname: \"pvb with empty annotation\",\n\t\t\tpvb: builder.ForPodVolumeBackup(\"fake-ns\", \"fake-name\").PodNamespace(\"fake-pod-ns\").PodName(\"fake-pod-name\").Volume(\"fake-volume\").Result(),\n\t\t\texpected: \"fake-pod-ns/fake-pod-name/fake-volume\",\n\t\t},\n\t\t{\n\t\t\tname: \"pvb without pvc name annotation\",\n\t\t\tpvb: builder.ForPodVolumeBackup(\"fake-ns\", \"fake-name\").PodNamespace(\"fake-pod-ns\").PodName(\"fake-pod-name\").Volume(\"fake-volume\").Annotations(map[string]string{}).Result(),\n\t\t\texpected: \"fake-pod-ns/fake-pod-name/fake-volume\",\n\t\t},\n\t\t{\n\t\t\tname: \"pvb with pvc name annotation\",\n\t\t\tpvb: builder.ForPodVolumeBackup(\"fake-ns\", \"fake-name\").PodNamespace(\"fake-pod-ns\").PodName(\"fake-pod-name\").Volume(\"fake-volume\").Annotations(map[string]string{\"velero.io/pvc-name\": \"fake-pvc-name\"}).Result(),\n\t\t\texpected: \"fake-pod-ns/fake-pod-name/fake-pvc-name\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactual := GetRealSource(tc.pvb)\n\t\t\tassert.Equal(t, tc.expected, actual)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/repository/backup_repo_op.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage repository\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\t\"github.com/pkg/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n)\n\n// A BackupRepositoryKey uniquely identify a backup repository\ntype BackupRepositoryKey struct {\n\tVolumeNamespace string\n\tBackupLocation string\n\tRepositoryType string\n}\n\nvar (\n\terrBackupRepoNotFound = errors.New(\"backup repository not found\")\n\terrBackupRepoNotProvisioned = errors.New(\"backup repository not provisioned\")\n)\n\nfunc repoLabelsFromKey(key BackupRepositoryKey) labels.Set {\n\treturn map[string]string{\n\t\tvelerov1api.VolumeNamespaceLabel: label.GetValidName(key.VolumeNamespace),\n\t\tvelerov1api.StorageLocationLabel: label.GetValidName(key.BackupLocation),\n\t\tvelerov1api.RepositoryTypeLabel: label.GetValidName(key.RepositoryType),\n\t}\n}\n\n// GetBackupRepository gets a backup repository through BackupRepositoryKey and ensure ready if required.\nfunc GetBackupRepository(ctx context.Context, cli client.Client, namespace string, key BackupRepositoryKey, options ...bool) (*velerov1api.BackupRepository, error) {\n\tvar ensureReady = true\n\tif len(options) > 0 {\n\t\tensureReady = options[0]\n\t}\n\n\tselector := labels.SelectorFromSet(repoLabelsFromKey(key))\n\n\tbackupRepoList := &velerov1api.BackupRepositoryList{}\n\terr := cli.List(ctx, backupRepoList, &client.ListOptions{\n\t\tNamespace: namespace,\n\t\tLabelSelector: selector,\n\t})\n\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error getting backup repository list\")\n\t}\n\n\tif len(backupRepoList.Items) == 0 {\n\t\treturn nil, errBackupRepoNotFound\n\t}\n\n\tif len(backupRepoList.Items) > 1 {\n\t\treturn nil, errors.Errorf(\"more than one BackupRepository found for workload namespace %q, backup storage location %q, repository type %q\", key.VolumeNamespace, key.BackupLocation, key.RepositoryType)\n\t}\n\n\trepo := &backupRepoList.Items[0]\n\n\tif ensureReady {\n\t\tif repo.Status.Phase == velerov1api.BackupRepositoryPhaseNotReady {\n\t\t\treturn nil, errors.Errorf(\"backup repository is not ready: %s\", repo.Status.Message)\n\t\t}\n\n\t\tif repo.Status.Phase == \"\" || repo.Status.Phase == velerov1api.BackupRepositoryPhaseNew {\n\t\t\treturn nil, errBackupRepoNotProvisioned\n\t\t}\n\t}\n\n\treturn repo, nil\n}\n\nfunc NewBackupRepository(namespace string, key BackupRepositoryKey) *velerov1api.BackupRepository {\n\treturn &velerov1api.BackupRepository{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: namespace,\n\t\t\tName: fmt.Sprintf(\"%s-%s-%s\", key.VolumeNamespace, key.BackupLocation, key.RepositoryType),\n\t\t\tLabels: repoLabelsFromKey(key),\n\t\t},\n\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\tVolumeNamespace: key.VolumeNamespace,\n\t\t\tBackupStorageLocation: key.BackupLocation,\n\t\t\tRepositoryType: key.RepositoryType,\n\t\t},\n\t}\n}\n\nfunc isBackupRepositoryNotFoundError(err error) bool {\n\treturn err == errBackupRepoNotFound\n}\n\nfunc isBackupRepositoryNotProvisionedError(err error) bool {\n\treturn err == errBackupRepoNotProvisioned\n}\n" }, { "path": "pkg/repository/backup_repo_op_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage repository\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/stretchr/testify/assert\"\n\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc buildBackupRepo(key BackupRepositoryKey, phase velerov1api.BackupRepositoryPhase, seqNum string) velerov1api.BackupRepository {\n\treturn velerov1api.BackupRepository{\n\t\tSpec: velerov1api.BackupRepositorySpec{ResticIdentifier: \"\"},\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\tKind: \"BackupRepository\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tName: fmt.Sprintf(\"%s-%s-%s-%s\", key.VolumeNamespace, key.BackupLocation, key.RepositoryType, seqNum),\n\t\t\tLabels: map[string]string{\n\t\t\t\tvelerov1api.StorageLocationLabel: key.BackupLocation,\n\t\t\t\tvelerov1api.VolumeNamespaceLabel: key.VolumeNamespace,\n\t\t\t\tvelerov1api.RepositoryTypeLabel: key.RepositoryType,\n\t\t\t},\n\t\t},\n\t\tStatus: velerov1api.BackupRepositoryStatus{\n\t\t\tPhase: phase,\n\t\t},\n\t}\n}\n\nfunc buildBackupRepoPointer(key BackupRepositoryKey, phase velerov1api.BackupRepositoryPhase, seqNum string) *velerov1api.BackupRepository {\n\tvalue := buildBackupRepo(key, phase, seqNum)\n\treturn &value\n}\n\nfunc TestGetBackupRepository(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tbackupRepositories []velerov1api.BackupRepository\n\t\tensureReady bool\n\t\tbackupRepositoryKey BackupRepositoryKey\n\t\texpected *velerov1api.BackupRepository\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"repository not found\",\n\t\t\texpectedErr: \"backup repository not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"found more than one repository\",\n\t\t\tbackupRepositories: []velerov1api.BackupRepository{\n\t\t\t\tbuildBackupRepo(BackupRepositoryKey{\"fake-volume-ns\", \"fake-bsl\", \"fake-repository-type\"}, velerov1api.BackupRepositoryPhaseReady, \"01\"),\n\t\t\t\tbuildBackupRepo(BackupRepositoryKey{\"fake-volume-ns\", \"fake-bsl\", \"fake-repository-type\"}, velerov1api.BackupRepositoryPhaseReady, \"02\")},\n\t\t\tbackupRepositoryKey: BackupRepositoryKey{\"fake-volume-ns\", \"fake-bsl\", \"fake-repository-type\"},\n\t\t\texpectedErr: \"more than one BackupRepository found for workload namespace \\\"fake-volume-ns\\\", backup storage location \\\"fake-bsl\\\", repository type \\\"fake-repository-type\\\"\",\n\t\t},\n\t\t{\n\t\t\tname: \"repository not ready, not expect ready\",\n\t\t\tbackupRepositories: []velerov1api.BackupRepository{\n\t\t\t\tbuildBackupRepo(BackupRepositoryKey{\"fake-volume-ns-01\", \"fake-bsl-01\", \"fake-repository-type-01\"}, velerov1api.BackupRepositoryPhaseReady, \"01\"),\n\t\t\t\tbuildBackupRepo(BackupRepositoryKey{\"fake-volume-ns-02\", \"fake-bsl-02\", \"fake-repository-type-02\"}, velerov1api.BackupRepositoryPhaseNotReady, \"02\")},\n\t\t\tbackupRepositoryKey: BackupRepositoryKey{\"fake-volume-ns-02\", \"fake-bsl-02\", \"fake-repository-type-02\"},\n\t\t\texpected: buildBackupRepoPointer(BackupRepositoryKey{\"fake-volume-ns-02\", \"fake-bsl-02\", \"fake-repository-type-02\"}, velerov1api.BackupRepositoryPhaseNotReady, \"02\"),\n\t\t},\n\t\t{\n\t\t\tname: \"repository is new, not expect ready\",\n\t\t\tbackupRepositories: []velerov1api.BackupRepository{\n\t\t\t\tbuildBackupRepo(BackupRepositoryKey{\"fake-volume-ns-01\", \"fake-bsl-01\", \"fake-repository-type-01\"}, velerov1api.BackupRepositoryPhaseReady, \"01\"),\n\t\t\t\tbuildBackupRepo(BackupRepositoryKey{\"fake-volume-ns-02\", \"fake-bsl-02\", \"fake-repository-type-02\"}, velerov1api.BackupRepositoryPhaseNew, \"02\")},\n\t\t\tbackupRepositoryKey: BackupRepositoryKey{\"fake-volume-ns-02\", \"fake-bsl-02\", \"fake-repository-type-02\"},\n\t\t\texpected: buildBackupRepoPointer(BackupRepositoryKey{\"fake-volume-ns-02\", \"fake-bsl-02\", \"fake-repository-type-02\"}, velerov1api.BackupRepositoryPhaseNew, \"02\"),\n\t\t},\n\t\t{\n\t\t\tname: \"repository state is empty, not expect ready\",\n\t\t\tbackupRepositories: []velerov1api.BackupRepository{\n\t\t\t\tbuildBackupRepo(BackupRepositoryKey{\"fake-volume-ns-01\", \"fake-bsl-01\", \"fake-repository-type-01\"}, velerov1api.BackupRepositoryPhaseReady, \"01\"),\n\t\t\t\tbuildBackupRepo(BackupRepositoryKey{\"fake-volume-ns-02\", \"fake-bsl-02\", \"fake-repository-type-02\"}, \"\", \"02\")},\n\t\t\tbackupRepositoryKey: BackupRepositoryKey{\"fake-volume-ns-02\", \"fake-bsl-02\", \"fake-repository-type-02\"},\n\t\t\texpected: buildBackupRepoPointer(BackupRepositoryKey{\"fake-volume-ns-02\", \"fake-bsl-02\", \"fake-repository-type-02\"}, \"\", \"02\"),\n\t\t},\n\t\t{\n\t\t\tname: \"repository not ready, expect ready\",\n\t\t\tbackupRepositories: []velerov1api.BackupRepository{\n\t\t\t\tbuildBackupRepo(BackupRepositoryKey{\"fake-volume-ns-01\", \"fake-bsl-01\", \"fake-repository-type-01\"}, velerov1api.BackupRepositoryPhaseReady, \"01\"),\n\t\t\t\tbuildBackupRepo(BackupRepositoryKey{\"fake-volume-ns-02\", \"fake-bsl-02\", \"fake-repository-type-02\"}, velerov1api.BackupRepositoryPhaseNotReady, \"02\")},\n\t\t\tbackupRepositoryKey: BackupRepositoryKey{\"fake-volume-ns-02\", \"fake-bsl-02\", \"fake-repository-type-02\"},\n\t\t\tensureReady: true,\n\t\t\texpectedErr: \"backup repository is not ready: \",\n\t\t},\n\t\t{\n\t\t\tname: \"repository is new, expect ready\",\n\t\t\tbackupRepositories: []velerov1api.BackupRepository{\n\t\t\t\tbuildBackupRepo(BackupRepositoryKey{\"fake-volume-ns-01\", \"fake-bsl-01\", \"fake-repository-type-01\"}, velerov1api.BackupRepositoryPhaseReady, \"01\"),\n\t\t\t\tbuildBackupRepo(BackupRepositoryKey{\"fake-volume-ns-02\", \"fake-bsl-02\", \"fake-repository-type-02\"}, velerov1api.BackupRepositoryPhaseNew, \"02\")},\n\t\t\tbackupRepositoryKey: BackupRepositoryKey{\"fake-volume-ns-02\", \"fake-bsl-02\", \"fake-repository-type-02\"},\n\t\t\tensureReady: true,\n\t\t\texpectedErr: \"backup repository not provisioned\",\n\t\t},\n\t\t{\n\t\t\tname: \"repository state is empty, expect ready\",\n\t\t\tbackupRepositories: []velerov1api.BackupRepository{\n\t\t\t\tbuildBackupRepo(BackupRepositoryKey{\"fake-volume-ns-01\", \"fake-bsl-01\", \"fake-repository-type-01\"}, velerov1api.BackupRepositoryPhaseReady, \"01\"),\n\t\t\t\tbuildBackupRepo(BackupRepositoryKey{\"fake-volume-ns-02\", \"fake-bsl-02\", \"fake-repository-type-02\"}, \"\", \"02\")},\n\t\t\tbackupRepositoryKey: BackupRepositoryKey{\"fake-volume-ns-02\", \"fake-bsl-02\", \"fake-repository-type-02\"},\n\t\t\tensureReady: true,\n\t\t\texpectedErr: \"backup repository not provisioned\",\n\t\t},\n\t\t{\n\t\t\tname: \"repository ready, expect ready\",\n\t\t\tbackupRepositories: []velerov1api.BackupRepository{\n\t\t\t\tbuildBackupRepo(BackupRepositoryKey{\"fake-volume-ns-01\", \"fake-bsl-01\", \"fake-repository-type-01\"}, velerov1api.BackupRepositoryPhaseNotReady, \"01\"),\n\t\t\t\tbuildBackupRepo(BackupRepositoryKey{\"fake-volume-ns-02\", \"fake-bsl-02\", \"fake-repository-type-02\"}, velerov1api.BackupRepositoryPhaseReady, \"02\")},\n\t\t\tbackupRepositoryKey: BackupRepositoryKey{\"fake-volume-ns-02\", \"fake-bsl-02\", \"fake-repository-type-02\"},\n\t\t\tensureReady: true,\n\t\t\texpected: buildBackupRepoPointer(BackupRepositoryKey{\"fake-volume-ns-02\", \"fake-bsl-02\", \"fake-repository-type-02\"}, velerov1api.BackupRepositoryPhaseReady, \"02\"),\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tclientBuilder := velerotest.NewFakeControllerRuntimeClientBuilder(t)\n\t\t\tclientBuilder.WithLists(&velerov1api.BackupRepositoryList{\n\t\t\t\tItems: tc.backupRepositories,\n\t\t\t})\n\t\t\tfakeClient := clientBuilder.Build()\n\n\t\t\tbackupRepo, err := GetBackupRepository(t.Context(), fakeClient, velerov1api.DefaultNamespace, tc.backupRepositoryKey, tc.ensureReady)\n\n\t\t\tif backupRepo != nil && tc.expected != nil {\n\t\t\t\tbackupRepo.ResourceVersion = tc.expected.ResourceVersion\n\t\t\t\trequire.Equal(t, *tc.expected, *backupRepo)\n\t\t\t} else {\n\t\t\t\trequire.Equal(t, tc.expected, backupRepo)\n\t\t\t}\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/repository/config/aws.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\n//nolint:gosec // Internal usage. No need to check.\npackage config\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"time\"\n\n\t\"github.com/aws/aws-sdk-go-v2/credentials\"\n\t\"github.com/aws/aws-sdk-go-v2/credentials/stscreds\"\n\t\"github.com/aws/aws-sdk-go-v2/service/sts\"\n\n\t\"github.com/aws/aws-sdk-go-v2/aws\"\n\tawsconfig \"github.com/aws/aws-sdk-go-v2/config\"\n\ts3manager \"github.com/aws/aws-sdk-go-v2/feature/s3/manager\"\n\t\"github.com/aws/aws-sdk-go-v2/service/s3\"\n\t\"github.com/pkg/errors\"\n)\n\n// getS3CredentialsFunc is used to make testing more convenient\nvar getS3CredentialsFunc = GetS3Credentials\n\nconst (\n\t// AWS specific environment variable\n\tawsProfileEnvVar = \"AWS_PROFILE\"\n\tawsKeyIDEnvVar = \"AWS_ACCESS_KEY_ID\"\n\tawsSecretKeyEnvVar = \"AWS_SECRET_ACCESS_KEY\"\n\tawsSessTokenEnvVar = \"AWS_SESSION_TOKEN\"\n\tawsProfileKey = \"profile\"\n\tawsCredentialsFileEnvVar = \"AWS_SHARED_CREDENTIALS_FILE\"\n\tawsConfigFileEnvVar = \"AWS_CONFIG_FILE\"\n\tawsDefaultProfile = \"default\"\n)\n\n// GetS3ResticEnvVars gets the environment variables that restic\n// relies on (AWS_PROFILE) based on info in the provided object\n// storage location config map.\nfunc GetS3ResticEnvVars(config map[string]string) (map[string]string, error) {\n\tresult := make(map[string]string)\n\n\tif credentialsFile, ok := config[CredentialsFileKey]; ok {\n\t\tresult[awsCredentialsFileEnvVar] = credentialsFile\n\t}\n\n\tif profile, ok := config[awsProfileKey]; ok {\n\t\tresult[awsProfileEnvVar] = profile\n\t}\n\n\t// GetS3ResticEnvVars reads the AWS config, from files and envs\n\t// if needed assumes the role and returns the session credentials\n\t// setting these variables emulates what would happen for example when using kube2iam\n\tif creds, err := getS3CredentialsFunc(config); err == nil && creds != nil {\n\t\tresult[awsKeyIDEnvVar] = creds.AccessKeyID\n\t\tresult[awsSecretKeyEnvVar] = creds.SecretAccessKey\n\t\tresult[awsSessTokenEnvVar] = creds.SessionToken\n\t\tresult[awsCredentialsFileEnvVar] = \"\"\n\t\tresult[awsProfileEnvVar] = \"\" // profile is not needed since we have the credentials from profile via GetS3Credentials\n\t\tresult[awsConfigFileEnvVar] = \"\"\n\t}\n\n\treturn result, nil\n}\n\n// GetS3Credentials gets the S3 credential values according to the information\n// of the provided config or the system's environment variables\nfunc GetS3Credentials(config map[string]string) (*aws.Credentials, error) {\n\tvar opts []func(*awsconfig.LoadOptions) error\n\tcredentialsFile := config[CredentialsFileKey]\n\tif credentialsFile == \"\" {\n\t\tcredentialsFile = os.Getenv(awsCredentialsFileEnvVar)\n\t}\n\tif credentialsFile != \"\" {\n\t\topts = append(opts, awsconfig.WithSharedCredentialsFiles([]string{credentialsFile}),\n\t\t\t// To support the existing use case where config file is passed\n\t\t\t// as credentials of a BSL\n\t\t\tawsconfig.WithSharedConfigFiles([]string{credentialsFile}))\n\t}\n\topts = append(opts, awsconfig.WithSharedConfigProfile(config[awsProfileKey]))\n\n\tcfg, err := awsconfig.LoadDefaultConfig(context.Background(), opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif credentialsFile != \"\" && os.Getenv(\"AWS_WEB_IDENTITY_TOKEN_FILE\") != \"\" && os.Getenv(\"AWS_ROLE_ARN\") != \"\" {\n\t\t// Reset the config to use the credentials from the credentials/config file\n\t\tprofile := config[awsProfileKey]\n\t\tif profile == \"\" {\n\t\t\tprofile = awsDefaultProfile\n\t\t}\n\t\tsfp, err := awsconfig.LoadSharedConfigProfile(context.Background(), profile, func(o *awsconfig.LoadSharedConfigOptions) {\n\t\t\to.ConfigFiles = []string{credentialsFile}\n\t\t\to.CredentialsFiles = []string{credentialsFile}\n\t\t})\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"error loading config profile '%s': %v\", profile, err)\n\t\t}\n\t\tif err := resolveCredsFromProfile(&cfg, &sfp); err != nil {\n\t\t\treturn nil, fmt.Errorf(\"error resolving creds from profile '%s': %v\", profile, err)\n\t\t}\n\t}\n\n\tcreds, err := cfg.Credentials.Retrieve(context.Background())\n\n\treturn &creds, err\n}\n\n// GetAWSBucketRegion returns the AWS region that a bucket is in, or an error\n// if the region cannot be determined.\n// It will use us-east-1 as hinting server and requires config param to use as credentials\nfunc GetAWSBucketRegion(bucket string, config map[string]string) (string, error) {\n\tcfg, err := awsconfig.LoadDefaultConfig(context.Background(), awsconfig.WithCredentialsProvider(\n\t\taws.CredentialsProviderFunc(\n\t\t\tfunc(context.Context) (aws.Credentials, error) {\n\t\t\t\ts3creds, err := GetS3Credentials(config)\n\t\t\t\tif s3creds == nil {\n\t\t\t\t\treturn aws.Credentials{}, err\n\t\t\t\t}\n\t\t\t\treturn *s3creds, err\n\t\t\t},\n\t\t),\n\t))\n\tif err != nil {\n\t\treturn \"\", errors.WithStack(err)\n\t}\n\tclient := s3.NewFromConfig(cfg)\n\tregion, err := s3manager.GetBucketRegion(context.Background(), client, bucket, func(o *s3.Options) { o.Region = \"us-east-1\" })\n\tif err != nil {\n\t\treturn \"\", errors.WithStack(err)\n\t}\n\tif region == \"\" {\n\t\treturn \"\", errors.New(\"unable to determine bucket's region\")\n\t}\n\treturn region, nil\n}\n\nfunc resolveCredsFromProfile(cfg *aws.Config, sharedConfig *awsconfig.SharedConfig) error {\n\tvar err error\n\tswitch {\n\tcase sharedConfig.Source != nil:\n\t\t// Assume IAM role with credentials source from a different profile.\n\t\terr = resolveCredsFromProfile(cfg, sharedConfig.Source)\n\tcase sharedConfig.Credentials.HasKeys():\n\t\t// Static Credentials from Shared Config/Credentials file.\n\t\tcfg.Credentials = credentials.StaticCredentialsProvider{\n\t\t\tValue: sharedConfig.Credentials,\n\t\t}\n\t}\n\tif err != nil {\n\t\treturn err\n\t}\n\tif len(sharedConfig.RoleARN) > 0 {\n\t\tcredsFromAssumeRole(cfg, sharedConfig)\n\t}\n\treturn nil\n}\n\nfunc credsFromAssumeRole(cfg *aws.Config, sharedCfg *awsconfig.SharedConfig) {\n\toptFns := []func(*stscreds.AssumeRoleOptions){\n\t\tfunc(options *stscreds.AssumeRoleOptions) {\n\t\t\toptions.RoleSessionName = sharedCfg.RoleSessionName\n\t\t\tif sharedCfg.RoleDurationSeconds != nil {\n\t\t\t\tif *sharedCfg.RoleDurationSeconds/time.Minute > 15 {\n\t\t\t\t\toptions.Duration = *sharedCfg.RoleDurationSeconds\n\t\t\t\t}\n\t\t\t}\n\t\t\tif len(sharedCfg.ExternalID) > 0 {\n\t\t\t\toptions.ExternalID = aws.String(sharedCfg.ExternalID)\n\t\t\t}\n\t\t\tif len(sharedCfg.MFASerial) != 0 {\n\t\t\t\toptions.SerialNumber = aws.String(sharedCfg.MFASerial)\n\t\t\t}\n\t\t},\n\t}\n\tcfg.Credentials = stscreds.NewAssumeRoleProvider(sts.NewFromConfig(*cfg), sharedCfg.RoleARN, optFns...)\n}\n" }, { "path": "pkg/repository/config/aws_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage config\n\nimport (\n\t\"os\"\n\t\"reflect\"\n\t\"testing\"\n\n\t\"github.com/aws/aws-sdk-go-v2/aws\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestGetS3ResticEnvVars(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tconfig map[string]string\n\t\texpected map[string]string\n\t\tgetS3Credentials func(config map[string]string) (*aws.Credentials, error)\n\t}{\n\t\t{\n\t\t\tname: \"when config is empty, no env vars are returned\",\n\t\t\tconfig: map[string]string{},\n\t\t\texpected: map[string]string{},\n\t\t\tgetS3Credentials: func(config map[string]string) (*aws.Credentials, error) {\n\t\t\t\treturn nil, nil\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when config contains profile key, profile env var is set with profile value\",\n\t\t\tconfig: map[string]string{\n\t\t\t\t\"profile\": \"profile-value\",\n\t\t\t},\n\t\t\texpected: map[string]string{\n\t\t\t\t\"AWS_PROFILE\": \"profile-value\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when config contains credentials file key, credentials file env var is set with credentials file value\",\n\t\t\tconfig: map[string]string{\n\t\t\t\t\"credentialsFile\": \"/tmp/credentials/path/to/secret\",\n\t\t\t},\n\t\t\texpected: map[string]string{\n\t\t\t\t\"AWS_SHARED_CREDENTIALS_FILE\": \"/tmp/credentials/path/to/secret\",\n\t\t\t},\n\t\t\tgetS3Credentials: func(config map[string]string) (*aws.Credentials, error) {\n\t\t\t\treturn nil, nil\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t// Mock GetS3Credentials\n\t\t\tif tc.getS3Credentials != nil {\n\t\t\t\tgetS3CredentialsFunc = tc.getS3Credentials\n\t\t\t} else {\n\t\t\t\tgetS3CredentialsFunc = GetS3Credentials\n\t\t\t}\n\n\t\t\tactual, err := GetS3ResticEnvVars(tc.config)\n\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Avoid direct comparison of expected and actual to prevent exposing secrets.\n\t\t\t// This may occur if the test doesn't set getS3Credentials func correctly.\n\t\t\tif !reflect.DeepEqual(tc.expected, actual) {\n\t\t\t\tt.Errorf(\"Expected and actual results do not match for test case %q\", tc.name)\n\t\t\t\tfor key, value := range actual {\n\t\t\t\t\tif expVal, err := tc.expected[key]; !err || expVal != value {\n\t\t\t\t\t\tif actualVal, ok := actual[key]; !ok {\n\t\t\t\t\t\t\tt.Errorf(\"Key %q is missing in actual result\", key)\n\t\t\t\t\t\t} else if expVal != actualVal {\n\t\t\t\t\t\t\tt.Errorf(\"Key %q: expected value %q\", key, expVal)\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetS3CredentialsCorrectlyUseProfile(t *testing.T) {\n\ttype args struct {\n\t\tconfig map[string]string\n\t\tsecretFileContents string\n\t}\n\ttests := []struct {\n\t\tname string\n\t\targs args\n\t\twant *aws.Credentials\n\t\twantErr bool\n\t}{\n\t\t{\n\t\t\tname: \"Test GetS3Credentials use profile correctly\",\n\t\t\targs: args{\n\t\t\t\tconfig: map[string]string{\n\t\t\t\t\t\"profile\": \"some-profile\",\n\t\t\t\t},\n\t\t\t\tsecretFileContents: `[default]\n\taws_access_key_id = default-access-key-id\n\taws_secret_access_key = default-secret-access-key\n\t[profile some-profile]\n\taws_access_key_id = some-profile-access-key-id\n\taws_secret_access_key = some-profile-secret-access-key\n\t`,\n\t\t\t},\n\t\t\twant: &aws.Credentials{\n\t\t\t\tAccessKeyID: \"some-profile-access-key-id\",\n\t\t\t\tSecretAccessKey: \"some-profile-secret-access-key\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Test GetS3Credentials default to default profile\",\n\t\t\targs: args{\n\t\t\t\tconfig: map[string]string{},\n\t\t\t\tsecretFileContents: `[default]\n\taws_access_key_id = default-access-key-id\n\taws_secret_access_key = default-secret-access-key\n\t[profile some-profile]\n\taws_access_key_id = some-profile-access-key-id\n\taws_secret_access_key = some-profile-secret-access-key\n\t`,\n\t\t\t},\n\t\t\twant: &aws.Credentials{\n\t\t\t\tAccessKeyID: \"default-access-key-id\",\n\t\t\t\tSecretAccessKey: \"default-secret-access-key\",\n\t\t\t},\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\t// Ensure env variables do not set AWS config entries\n\t\t\tt.Setenv(\"AWS_ACCESS_KEY_ID\", \"\")\n\t\t\tt.Setenv(\"AWS_SECRET_ACCESS_KEY\", \"\")\n\t\t\tt.Setenv(\"AWS_SHARED_CREDENTIALS_FILE\", \"\")\n\n\t\t\ttmpFile, err := os.CreateTemp(t.TempDir(), \"velero-test-aws-credentials\")\n\t\t\tdefer os.Remove(tmpFile.Name())\n\t\t\tif err != nil {\n\t\t\t\tt.Errorf(\"GetS3Credentials() error = %v\", err)\n\t\t\t\treturn\n\t\t\t}\n\t\t\t// write the contents of the secret file to the temp file\n\t\t\t_, err = tmpFile.WriteString(tt.args.secretFileContents)\n\t\t\tif err != nil {\n\t\t\t\tt.Errorf(\"GetS3Credentials() error = %v\", err)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\ttt.args.config[\"credentialsFile\"] = tmpFile.Name()\n\t\t\tgot, err := GetS3Credentials(tt.args.config)\n\t\t\tif (err != nil) != tt.wantErr {\n\t\t\t\tt.Errorf(\"GetS3Credentials() error = %v, wantErr %v\", err, tt.wantErr)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif !reflect.DeepEqual(got.AccessKeyID, tt.want.AccessKeyID) {\n\t\t\t\tt.Errorf(\"GetS3Credentials() want %v\", tt.want.AccessKeyID)\n\t\t\t}\n\t\t\tif !reflect.DeepEqual(got.SecretAccessKey, tt.want.SecretAccessKey) {\n\t\t\t\tt.Errorf(\"GetS3Credentials() want %v\", tt.want.SecretAccessKey)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/repository/config/azure.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage config\n\nimport (\n\t\"github.com/pkg/errors\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/azure\"\n)\n\n// GetAzureResticEnvVars gets the environment variables that restic\n// relies on (AZURE_ACCOUNT_NAME and AZURE_ACCOUNT_KEY) based\n// on info in the provided object storage location config map.\nfunc GetAzureResticEnvVars(config map[string]string) (map[string]string, error) {\n\tstorageAccount := config[azure.BSLConfigStorageAccount]\n\tif storageAccount == \"\" {\n\t\treturn nil, errors.New(\"storageAccount is required in the BSL\")\n\t}\n\n\tcreds, err := azure.LoadCredentials(config)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// restic doesn't support Azure AD, set it as false\n\tconfig[azure.BSLConfigUseAAD] = \"false\"\n\tcredentials, err := azure.GetStorageAccountCredentials(config, creds)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn map[string]string{\n\t\t\"AZURE_ACCOUNT_NAME\": storageAccount,\n\t\t\"AZURE_ACCOUNT_KEY\": credentials[azure.CredentialKeyStorageAccountAccessKey],\n\t}, nil\n}\n" }, { "path": "pkg/repository/config/azure_test.go", "content": "/*\nCopyright the Velero contributors.\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n http://www.apache.org/licenses/LICENSE-2.0\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage config\n\nimport (\n\t\"os\"\n\t\"path/filepath\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/azure\"\n)\n\nfunc TestGetAzureResticEnvVars(t *testing.T) {\n\tconfig := map[string]string{}\n\n\t// no storage account specified\n\t_, err := GetAzureResticEnvVars(config)\n\trequire.Error(t, err)\n\n\t// specify storage account access key\n\tname := filepath.Join(os.TempDir(), \"credential\")\n\tfile, err := os.Create(name)\n\trequire.NoError(t, err)\n\tdefer file.Close()\n\tdefer os.Remove(name)\n\t_, err = file.WriteString(\"AccessKey: accesskey\")\n\trequire.NoError(t, err)\n\n\tconfig[azure.BSLConfigStorageAccount] = \"account01\"\n\tconfig[azure.BSLConfigStorageAccountAccessKeyName] = \"AccessKey\"\n\tconfig[\"credentialsFile\"] = name\n\tenvs, err := GetAzureResticEnvVars(config)\n\trequire.NoError(t, err)\n\n\tassert.Equal(t, \"account01\", envs[\"AZURE_ACCOUNT_NAME\"])\n\tassert.Equal(t, \"accesskey\", envs[\"AZURE_ACCOUNT_KEY\"])\n}\n" }, { "path": "pkg/repository/config/config.go", "content": "/*\nCopyright 2018, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage config\n\nimport (\n\t\"fmt\"\n\t\"path\"\n\t\"strings\"\n\n\t\"github.com/pkg/errors\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/persistence\"\n)\n\ntype BackendType string\n\nconst (\n\tAWSBackend BackendType = \"velero.io/aws\"\n\tAzureBackend BackendType = \"velero.io/azure\"\n\tGCPBackend BackendType = \"velero.io/gcp\"\n\tFSBackend BackendType = \"velero.io/fs\"\n\n\t// CredentialsFileKey is the key within a BSL config that is checked to see if\n\t// the BSL is using its own credentials, rather than those in the environment\n\tCredentialsFileKey = \"credentialsFile\"\n)\n\n// this func is assigned to a package-level variable so it can be\n// replaced when unit-testing\nvar getAWSBucketRegion = GetAWSBucketRegion\n\n// getRepoPrefix returns the prefix of the value of the --repo flag for\n// restic commands, i.e. everything except the \"/\".\nfunc getRepoPrefix(location *velerov1api.BackupStorageLocation) (string, error) {\n\tvar bucket, prefix string\n\n\tif location.Spec.ObjectStorage != nil {\n\t\tlayout := persistence.NewObjectStoreLayout(location.Spec.ObjectStorage.Prefix)\n\n\t\tbucket = location.Spec.ObjectStorage.Bucket\n\t\tprefix = layout.GetResticDir()\n\t}\n\n\tbackendType := GetBackendType(location.Spec.Provider, location.Spec.Config)\n\n\tif repoPrefix := location.Spec.Config[\"resticRepoPrefix\"]; repoPrefix != \"\" {\n\t\treturn repoPrefix, nil\n\t}\n\n\tswitch backendType {\n\tcase AWSBackend:\n\t\tvar url string\n\t\t// non-AWS, S3-compatible object store\n\t\tif s3Url := location.Spec.Config[\"s3Url\"]; s3Url != \"\" {\n\t\t\turl = strings.TrimSuffix(s3Url, \"/\")\n\t\t} else {\n\t\t\tvar err error\n\t\t\tregion := location.Spec.Config[\"region\"]\n\t\t\tif region == \"\" {\n\t\t\t\tregion, err = getAWSBucketRegion(bucket, location.Spec.Config)\n\t\t\t}\n\t\t\tif err != nil {\n\t\t\t\treturn \"\", errors.Wrapf(err, \"failed to detect the region via bucket: %s\", bucket)\n\t\t\t}\n\t\t\turl = fmt.Sprintf(\"s3-%s.amazonaws.com\", region)\n\t\t}\n\n\t\treturn fmt.Sprintf(\"s3:%s/%s\", url, path.Join(bucket, prefix)), nil\n\tcase AzureBackend:\n\t\treturn fmt.Sprintf(\"azure:%s:/%s\", bucket, prefix), nil\n\tcase GCPBackend:\n\t\treturn fmt.Sprintf(\"gs:%s:/%s\", bucket, prefix), nil\n\t}\n\n\treturn \"\", errors.Errorf(\"invalid backend type %s, provider %s\", backendType, location.Spec.Provider)\n}\n\n// GetBackendType returns a backend type that is known by Velero.\n// If the provider doesn't indicate a known backend type, but the endpoint is\n// specified, Velero regards it as a S3 compatible object store and return AWSBackend as the type.\nfunc GetBackendType(provider string, config map[string]string) BackendType {\n\tif !strings.Contains(provider, \"/\") {\n\t\tprovider = \"velero.io/\" + provider\n\t}\n\n\tbt := BackendType(provider)\n\tif IsBackendTypeValid(bt) {\n\t\treturn bt\n\t} else if config != nil && config[\"s3Url\"] != \"\" {\n\t\treturn AWSBackend\n\t} else {\n\t\treturn bt\n\t}\n}\n\nfunc IsBackendTypeValid(backendType BackendType) bool {\n\treturn (backendType == AWSBackend || backendType == AzureBackend || backendType == GCPBackend || backendType == FSBackend)\n}\n\n// GetRepoIdentifier returns the string to be used as the value of the --repo flag in\n// restic commands for the given repository.\nfunc GetRepoIdentifier(location *velerov1api.BackupStorageLocation, name string) (string, error) {\n\tprefix, err := getRepoPrefix(location)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\treturn fmt.Sprintf(\"%s/%s\", strings.TrimSuffix(prefix, \"/\"), name), nil\n}\n" }, { "path": "pkg/repository/config/config_test.go", "content": "/*\nCopyright 2018, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage config\n\nimport (\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\nfunc TestGetRepoIdentifier(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tbsl *velerov1api.BackupStorageLocation\n\t\trepoName string\n\t\tgetAWSBucketRegion func(s string, config map[string]string) (string, error)\n\t\texpected string\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"error is returned if BSL uses unsupported provider and resticRepoPrefix is not set\",\n\t\t\tbsl: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"unsupported-provider\",\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"bucket-2\",\n\t\t\t\t\t\t\tPrefix: \"prefix-2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoName: \"repo-1\",\n\t\t\texpectedErr: \"invalid backend type velero.io/unsupported-provider, provider unsupported-provider\",\n\t\t},\n\t\t{\n\t\t\tname: \"resticRepoPrefix in BSL config is used if set\",\n\t\t\tbsl: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"custom-repo-identifier\",\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"resticRepoPrefix\": \"custom:prefix:/restic\",\n\t\t\t\t\t},\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"bucket\",\n\t\t\t\t\t\t\tPrefix: \"prefix\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoName: \"repo-1\",\n\t\t\texpected: \"custom:prefix:/restic/repo-1\",\n\t\t},\n\t\t{\n\t\t\tname: \"s3Url in BSL config is used\",\n\t\t\tbsl: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"custom-repo-identifier\",\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"s3Url\": \"s3Url\",\n\t\t\t\t\t},\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"bucket\",\n\t\t\t\t\t\t\tPrefix: \"prefix\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoName: \"repo-1\",\n\t\t\texpected: \"s3:s3Url/bucket/prefix/restic/repo-1\",\n\t\t},\n\t\t{\n\t\t\tname: \"s3.amazonaws.com URL format is used if region cannot be determined for AWS BSL\",\n\t\t\tbsl: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"aws\",\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"bucket\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoName: \"repo-1\",\n\t\t\tgetAWSBucketRegion: func(s string, config map[string]string) (string, error) {\n\t\t\t\treturn \"\", errors.New(\"no region found\")\n\t\t\t},\n\t\t\texpected: \"\",\n\t\t\texpectedErr: \"failed to detect the region via bucket: bucket: no region found\",\n\t\t},\n\t\t{\n\t\t\tname: \"s3.s3-.amazonaws.com URL format is used if region can be determined for AWS BSL\",\n\t\t\tbsl: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"aws\",\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"bucket\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoName: \"repo-1\",\n\t\t\tgetAWSBucketRegion: func(string, map[string]string) (string, error) {\n\t\t\t\treturn \"eu-west-1\", nil\n\t\t\t},\n\t\t\texpected: \"s3:s3-eu-west-1.amazonaws.com/bucket/restic/repo-1\",\n\t\t},\n\t\t{\n\t\t\tname: \"prefix is included in repo identifier if set for AWS BSL\",\n\t\t\tbsl: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"aws\",\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"bucket\",\n\t\t\t\t\t\t\tPrefix: \"prefix\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoName: \"repo-1\",\n\t\t\tgetAWSBucketRegion: func(s string, config map[string]string) (string, error) {\n\t\t\t\treturn \"eu-west-1\", nil\n\t\t\t},\n\t\t\texpected: \"s3:s3-eu-west-1.amazonaws.com/bucket/prefix/restic/repo-1\",\n\t\t},\n\t\t{\n\t\t\tname: \"s3Url is used in repo identifier if set for AWS BSL\",\n\t\t\tbsl: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"aws\",\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"s3Url\": \"alternate-url\",\n\t\t\t\t\t},\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"bucket\",\n\t\t\t\t\t\t\tPrefix: \"prefix\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoName: \"repo-1\",\n\t\t\tgetAWSBucketRegion: func(s string, config map[string]string) (string, error) {\n\t\t\t\treturn \"eu-west-1\", nil\n\t\t\t},\n\t\t\texpected: \"s3:alternate-url/bucket/prefix/restic/repo-1\",\n\t\t},\n\t\t{\n\t\t\tname: \"region is used in repo identifier if set for AWS BSL\",\n\t\t\tbsl: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"aws\",\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"region\": \"us-west-1\",\n\t\t\t\t\t},\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"bucket\",\n\t\t\t\t\t\t\tPrefix: \"prefix\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoName: \"aws-repo\",\n\t\t\tgetAWSBucketRegion: func(s string, config map[string]string) (string, error) {\n\t\t\t\treturn \"eu-west-1\", nil\n\t\t\t},\n\t\t\texpected: \"s3:s3-us-west-1.amazonaws.com/bucket/prefix/restic/aws-repo\",\n\t\t},\n\t\t{\n\t\t\tname: \"trailing slash in s3Url is not included in repo identifier for AWS BSL\",\n\t\t\tbsl: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"aws\",\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"s3Url\": \"alternate-url-with-trailing-slash/\",\n\t\t\t\t\t},\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"bucket\",\n\t\t\t\t\t\t\tPrefix: \"prefix\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoName: \"aws-repo\",\n\t\t\tgetAWSBucketRegion: func(s string, config map[string]string) (string, error) {\n\t\t\t\treturn \"eu-west-1\", nil\n\t\t\t},\n\t\t\texpected: \"s3:alternate-url-with-trailing-slash/bucket/prefix/restic/aws-repo\",\n\t\t},\n\t\t{\n\t\t\tname: \"repo identifier includes bucket and prefix for Azure BSL\",\n\t\t\tbsl: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"azure\",\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"azure-bucket\",\n\t\t\t\t\t\t\tPrefix: \"azure-prefix\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoName: \"azure-repo\",\n\t\t\texpected: \"azure:azure-bucket:/azure-prefix/restic/azure-repo\",\n\t\t},\n\t\t{\n\t\t\tname: \"repo identifier includes bucket and prefix for GCP BSL\",\n\t\t\tbsl: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"gcp\",\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"gcp-bucket\",\n\t\t\t\t\t\t\tPrefix: \"gcp-prefix\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoName: \"gcp-repo\",\n\t\t\texpected: \"gs:gcp-bucket:/gcp-prefix/restic/gcp-repo\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tgetAWSBucketRegion = tc.getAWSBucketRegion\n\t\t\tid, err := GetRepoIdentifier(tc.bsl, tc.repoName)\n\t\t\tassert.Equal(t, tc.expected, id)\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\trequire.EqualError(t, err, tc.expectedErr)\n\t\t\t\tassert.Empty(t, id)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/repository/config/gcp.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\n//nolint:gosec // Internal usage. No need to check.\npackage config\n\nimport \"os\"\n\nconst (\n\t// GCP specific environment variable\n\tgcpCredentialsFileEnvVar = \"GOOGLE_APPLICATION_CREDENTIALS\"\n)\n\n// GetGCPResticEnvVars gets the environment variables that restic relies\n// on based on info in the provided object storage location config map.\nfunc GetGCPResticEnvVars(config map[string]string) (map[string]string, error) {\n\tresult := make(map[string]string)\n\n\tif credentialsFile, ok := config[CredentialsFileKey]; ok {\n\t\tresult[gcpCredentialsFileEnvVar] = credentialsFile\n\t}\n\n\treturn result, nil\n}\n\n// GetGCPCredentials gets the credential file required by a GCP bucket connection,\n// if the provided config doean't have the value, get it from system's environment variables\nfunc GetGCPCredentials(config map[string]string) string {\n\tif credentialsFile, ok := config[CredentialsFileKey]; ok {\n\t\treturn credentialsFile\n\t}\n\treturn os.Getenv(gcpCredentialsFileEnvVar)\n}\n" }, { "path": "pkg/repository/config/gcp_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage config\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestGetGCPResticEnvVars(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tconfig map[string]string\n\t\texpected map[string]string\n\t}{\n\t\t{\n\t\t\tname: \"when config is empty, no env vars are returned\",\n\t\t\tconfig: map[string]string{},\n\t\t\texpected: map[string]string{},\n\t\t},\n\t\t{\n\t\t\tname: \"when config contains credentials file key, credentials file env var is set with credentials file value\",\n\t\t\tconfig: map[string]string{\n\t\t\t\t\"credentialsFile\": \"/tmp/credentials/path/to/secret\",\n\t\t\t},\n\t\t\texpected: map[string]string{\n\t\t\t\t\"GOOGLE_APPLICATION_CREDENTIALS\": \"/tmp/credentials/path/to/secret\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactual, err := GetGCPResticEnvVars(tc.config)\n\n\t\t\trequire.NoError(t, err)\n\n\t\t\trequire.Equal(t, tc.expected, actual)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/repository/ensurer.go", "content": "/*\nCopyright 2018, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage repository\n\nimport (\n\t\"context\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n)\n\n// Ensurer ensures that backup repositories are created and ready.\ntype Ensurer struct {\n\tlog logrus.FieldLogger\n\trepoClient client.Client\n\n\t// repoLocksMu synchronizes reads/writes to the repoLocks map itself\n\t// since maps are not threadsafe.\n\trepoLocksMu sync.Mutex\n\trepoLocks map[BackupRepositoryKey]*sync.Mutex\n\tresourceTimeout time.Duration\n}\n\nfunc NewEnsurer(repoClient client.Client, log logrus.FieldLogger, resourceTimeout time.Duration) *Ensurer {\n\treturn &Ensurer{\n\t\tlog: log,\n\t\trepoClient: repoClient,\n\t\trepoLocks: make(map[BackupRepositoryKey]*sync.Mutex),\n\t\tresourceTimeout: resourceTimeout,\n\t}\n}\n\nfunc (r *Ensurer) EnsureRepo(ctx context.Context, namespace, volumeNamespace, backupLocation, repositoryType string) (*velerov1api.BackupRepository, error) {\n\tif volumeNamespace == \"\" || backupLocation == \"\" || repositoryType == \"\" {\n\t\treturn nil, errors.Errorf(\"wrong parameters, namespace %q, backup storage location %q, repository type %q\", volumeNamespace, backupLocation, repositoryType)\n\t}\n\n\tbackupRepoKey := BackupRepositoryKey{volumeNamespace, backupLocation, repositoryType}\n\n\tlog := r.log.WithField(\"volumeNamespace\", volumeNamespace).WithField(\"backupLocation\", backupLocation).WithField(\"repositoryType\", repositoryType)\n\n\t// The BackupRepository is labeled with BackupRepositoryKey.\n\t// This function searches for an existing BackupRepository by BackupRepositoryKey label.\n\t// If it doesn't exist, it creates a new one and wait for its readiness.\n\t//\n\t// The BackupRepository is also named as BackupRepositoryKey.\n\t// It creates a BackupRepository with deterministic name\n\t// so that this function could support multiple thread calling by leveraging API server's optimistic lock mechanism.\n\t// Therefore, the name must be unique for a BackupRepository.\n\t// Don't use name to filter/search BackupRepository, since it may be changed in future, use label instead.\n\tlog.Debug(\"Acquiring lock\")\n\n\trepoMu := r.repoLock(backupRepoKey)\n\trepoMu.Lock()\n\tdefer func() {\n\t\trepoMu.Unlock()\n\t\tlog.Debug(\"Released lock\")\n\t}()\n\n\t_, err := GetBackupRepository(ctx, r.repoClient, namespace, backupRepoKey, false)\n\tif err == nil {\n\t\tlog.Info(\"Founding existing repo\")\n\t\treturn r.waitBackupRepository(ctx, namespace, backupRepoKey)\n\t} else if isBackupRepositoryNotFoundError(err) {\n\t\tlog.Info(\"No repository found, creating one\")\n\n\t\t// no repo found: create one and wait for it to be ready\n\t\treturn r.createBackupRepositoryAndWait(ctx, namespace, backupRepoKey)\n\t} else {\n\t\treturn nil, errors.WithStack(err)\n\t}\n}\n\nfunc (r *Ensurer) repoLock(key BackupRepositoryKey) *sync.Mutex {\n\tr.repoLocksMu.Lock()\n\tdefer r.repoLocksMu.Unlock()\n\n\tif r.repoLocks[key] == nil {\n\t\tr.repoLocks[key] = new(sync.Mutex)\n\t}\n\n\treturn r.repoLocks[key]\n}\n\nfunc (r *Ensurer) createBackupRepositoryAndWait(ctx context.Context, namespace string, backupRepoKey BackupRepositoryKey) (*velerov1api.BackupRepository, error) {\n\ttoCreate := NewBackupRepository(namespace, backupRepoKey)\n\n\tif err := r.repoClient.Create(ctx, toCreate, &client.CreateOptions{}); err != nil && !apierrors.IsAlreadyExists(err) {\n\t\treturn nil, errors.Wrap(err, \"unable to create backup repository resource\")\n\t}\n\n\treturn r.waitBackupRepository(ctx, namespace, backupRepoKey)\n}\n\nfunc (r *Ensurer) waitBackupRepository(ctx context.Context, namespace string, backupRepoKey BackupRepositoryKey) (*velerov1api.BackupRepository, error) {\n\tvar repo *velerov1api.BackupRepository\n\tvar checkErr error\n\tcheckFunc := func(ctx context.Context) (bool, error) {\n\t\tfound, err := GetBackupRepository(ctx, r.repoClient, namespace, backupRepoKey, true)\n\t\tif err == nil {\n\t\t\trepo = found\n\t\t\treturn true, nil\n\t\t} else if isBackupRepositoryNotFoundError(err) || isBackupRepositoryNotProvisionedError(err) {\n\t\t\tcheckErr = err\n\t\t\treturn false, nil\n\t\t} else {\n\t\t\treturn false, err\n\t\t}\n\t}\n\n\terr := wait.PollUntilContextTimeout(ctx, time.Millisecond*500, r.resourceTimeout, true, checkFunc)\n\tif err != nil {\n\t\tif err == context.DeadlineExceeded {\n\t\t\t// if deadline is exceeded, return the error from the last check instead of the wait error\n\t\t\treturn nil, errors.Wrap(checkErr, \"failed to wait BackupRepository, timeout exceeded\")\n\t\t}\n\t\t// if the error is not deadline exceeded, return the error from the wait\n\t\treturn nil, errors.Wrap(err, \"failed to wait BackupRepository, errored early\")\n\t}\n\n\treturn repo, nil\n}\n" }, { "path": "pkg/repository/ensurer_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage repository\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerov1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestEnsureRepo(t *testing.T) {\n\tbkRepoObjReady := NewBackupRepository(velerov1.DefaultNamespace, BackupRepositoryKey{\n\t\tVolumeNamespace: \"fake-ns\",\n\t\tBackupLocation: \"fake-bsl\",\n\t\tRepositoryType: \"fake-repo-type\",\n\t})\n\n\tbkRepoObjReady.Status.Phase = velerov1.BackupRepositoryPhaseReady\n\n\tbkRepoObjNotReady := NewBackupRepository(velerov1.DefaultNamespace, BackupRepositoryKey{\n\t\tVolumeNamespace: \"fake-ns\",\n\t\tBackupLocation: \"fake-bsl\",\n\t\tRepositoryType: \"fake-repo-type\",\n\t})\n\n\tscheme := runtime.NewScheme()\n\tvelerov1.AddToScheme(scheme)\n\n\ttests := []struct {\n\t\tname string\n\t\tnamespace string\n\t\tbsl string\n\t\trepositoryType string\n\t\tkubeClientObj []runtime.Object\n\t\truntimeScheme *runtime.Scheme\n\t\texpectedRepo *velerov1.BackupRepository\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"namespace is empty\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\trepositoryType: \"fake-repo-type\",\n\t\t\terr: \"wrong parameters, namespace \\\"\\\", backup storage location \\\"fake-bsl\\\", repository type \\\"fake-repo-type\\\"\",\n\t\t},\n\t\t{\n\t\t\tname: \"bsl is empty\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\trepositoryType: \"fake-repo-type\",\n\t\t\terr: \"wrong parameters, namespace \\\"fake-ns\\\", backup storage location \\\"\\\", repository type \\\"fake-repo-type\\\"\",\n\t\t},\n\t\t{\n\t\t\tname: \"repositoryType is empty\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\terr: \"wrong parameters, namespace \\\"fake-ns\\\", backup storage location \\\"fake-bsl\\\", repository type \\\"\\\"\",\n\t\t},\n\t\t{\n\t\t\tname: \"get repo fail\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\trepositoryType: \"fake-repo-type\",\n\t\t\terr: \"error getting backup repository list: no kind is registered for the type v1.BackupRepositoryList in scheme\",\n\t\t},\n\t\t{\n\t\t\tname: \"success on existing repo\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\trepositoryType: \"fake-repo-type\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tbkRepoObjReady,\n\t\t\t},\n\t\t\truntimeScheme: scheme,\n\t\t\texpectedRepo: bkRepoObjReady,\n\t\t},\n\t\t{\n\t\t\tname: \"wait existing repo fail\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\trepositoryType: \"fake-repo-type\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tbkRepoObjNotReady,\n\t\t\t},\n\t\t\truntimeScheme: scheme,\n\t\t\terr: \"failed to wait BackupRepository, timeout exceeded: backup repository not provisioned\",\n\t\t},\n\t\t{\n\t\t\tname: \"create fail\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\trepositoryType: \"fake-repo-type\",\n\t\t\truntimeScheme: scheme,\n\t\t\terr: \"failed to wait BackupRepository, timeout exceeded: backup repository not provisioned\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeClientBuilder := fake.NewClientBuilder()\n\t\t\tif test.runtimeScheme != nil {\n\t\t\t\tfakeClientBuilder = fakeClientBuilder.WithScheme(test.runtimeScheme)\n\t\t\t}\n\n\t\t\tfakeClient := fakeClientBuilder.WithRuntimeObjects(test.kubeClientObj...).Build()\n\n\t\t\tensurer := NewEnsurer(fakeClient, velerotest.NewLogger(), time.Millisecond)\n\n\t\t\trepo, err := ensurer.EnsureRepo(t.Context(), velerov1.DefaultNamespace, test.namespace, test.bsl, test.repositoryType)\n\t\t\tif err != nil {\n\t\t\t\trequire.ErrorContains(t, err, test.err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tassert.Equal(t, test.expectedRepo, repo)\n\t\t})\n\t}\n}\n\nfunc TestCreateBackupRepositoryAndWait(t *testing.T) {\n\texistingRepoReady := NewBackupRepository(velerov1.DefaultNamespace, BackupRepositoryKey{\n\t\tVolumeNamespace: \"fake-ns\",\n\t\tBackupLocation: \"fake-bsl\",\n\t\tRepositoryType: \"fake-repo-type\",\n\t})\n\n\texistingRepoReady.Status.Phase = velerov1.BackupRepositoryPhaseReady\n\n\texistingRepoNotReady := NewBackupRepository(velerov1.DefaultNamespace, BackupRepositoryKey{\n\t\tVolumeNamespace: \"fake-ns\",\n\t\tBackupLocation: \"fake-bsl\",\n\t\tRepositoryType: \"fake-repo-type\",\n\t})\n\n\tkey := BackupRepositoryKey{\n\t\tVolumeNamespace: \"fake-ns\",\n\t\tBackupLocation: \"fake-bsl\",\n\t\tRepositoryType: \"fake-repo-type\",\n\t}\n\n\texistingRepoWithUnexpectedName := &velerov1.BackupRepository{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: velerov1.DefaultNamespace,\n\t\t\tName: \"ake-ns-fake-bsl-fake-repo-type-xxx00\",\n\t\t\tLabels: repoLabelsFromKey(key),\n\t\t},\n\t\tSpec: velerov1.BackupRepositorySpec{\n\t\t\tVolumeNamespace: key.VolumeNamespace,\n\t\t\tBackupStorageLocation: key.BackupLocation,\n\t\t\tRepositoryType: key.RepositoryType,\n\t\t},\n\t}\n\n\tscheme := runtime.NewScheme()\n\tvelerov1.AddToScheme(scheme)\n\n\ttests := []struct {\n\t\tname string\n\t\tnamespace string\n\t\tbsl string\n\t\trepositoryType string\n\t\tkubeClientObj []runtime.Object\n\t\truntimeScheme *runtime.Scheme\n\t\texpectedRepo *velerov1.BackupRepository\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"create fail\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\trepositoryType: \"fake-repo-type\",\n\t\t\terr: \"unable to create backup repository resource: no kind is registered for the type v1.BackupRepository in scheme\",\n\t\t},\n\t\t{\n\t\t\tname: \"get repo fail\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\trepositoryType: \"fake-repo-type\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\texistingRepoWithUnexpectedName,\n\t\t\t},\n\t\t\truntimeScheme: scheme,\n\t\t\terr: \"failed to wait BackupRepository, errored early: more than one BackupRepository found for workload namespace \\\"fake-ns\\\", backup storage location \\\"fake-bsl\\\", repository type \\\"fake-repo-type\\\"\",\n\t\t},\n\t\t{\n\t\t\tname: \"wait repo fail\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\trepositoryType: \"fake-repo-type\",\n\t\t\truntimeScheme: scheme,\n\t\t\terr: \"failed to wait BackupRepository, timeout exceeded: backup repository not provisioned\",\n\t\t},\n\t\t{\n\t\t\tname: \"repo already exists and ready\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\trepositoryType: \"fake-repo-type\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\texistingRepoReady,\n\t\t\t},\n\t\t\truntimeScheme: scheme,\n\t\t\texpectedRepo: existingRepoReady,\n\t\t},\n\t\t{\n\t\t\tname: \"repo already exists but not ready\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tbsl: \"fake-bsl\",\n\t\t\trepositoryType: \"fake-repo-type\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\texistingRepoNotReady,\n\t\t\t},\n\t\t\truntimeScheme: scheme,\n\t\t\terr: \"failed to wait BackupRepository, timeout exceeded: backup repository not provisioned\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeClientBuilder := fake.NewClientBuilder()\n\t\t\tif test.runtimeScheme != nil {\n\t\t\t\tfakeClientBuilder = fakeClientBuilder.WithScheme(test.runtimeScheme)\n\t\t\t}\n\n\t\t\tfakeClient := fakeClientBuilder.WithRuntimeObjects(test.kubeClientObj...).Build()\n\n\t\t\tensurer := NewEnsurer(fakeClient, velerotest.NewLogger(), time.Millisecond)\n\n\t\t\trepo, err := ensurer.createBackupRepositoryAndWait(t.Context(), velerov1.DefaultNamespace, BackupRepositoryKey{\n\t\t\t\tVolumeNamespace: test.namespace,\n\t\t\t\tBackupLocation: test.bsl,\n\t\t\t\tRepositoryType: test.repositoryType,\n\t\t\t})\n\t\t\tif err != nil {\n\t\t\t\trequire.ErrorContains(t, err, test.err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tassert.Equal(t, test.expectedRepo, repo)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/repository/keys/keys.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\n//nolint:gosec // Internal call. No need to check.\npackage keys\n\nimport (\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\tcorev1client \"k8s.io/client-go/kubernetes/typed/core/v1\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n)\n\nconst (\n\tcredentialsSecretName = \"velero-repo-credentials\"\n\tcredentialsKey = \"repository-password\"\n\n\tencryptionKey = \"static-passw0rd\"\n)\n\nfunc EnsureCommonRepositoryKey(secretClient corev1client.SecretsGetter, namespace string) error {\n\t_, err := secretClient.Secrets(namespace).Get(context.TODO(), credentialsSecretName, metav1.GetOptions{})\n\tif err != nil && !apierrors.IsNotFound(err) {\n\t\treturn errors.WithStack(err)\n\t}\n\tif err == nil {\n\t\treturn nil\n\t}\n\n\t// if we got here, we got an IsNotFound error, so we need to create the key\n\n\tsecret := &corev1api.Secret{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: namespace,\n\t\t\tName: credentialsSecretName,\n\t\t},\n\t\tType: corev1api.SecretTypeOpaque,\n\t\tData: map[string][]byte{\n\t\t\tcredentialsKey: []byte(encryptionKey),\n\t\t},\n\t}\n\n\tif _, err = secretClient.Secrets(namespace).Create(context.TODO(), secret, metav1.CreateOptions{}); err != nil {\n\t\treturn errors.Wrapf(err, \"error creating %s secret\", credentialsSecretName)\n\t}\n\n\treturn nil\n}\n\n// RepoKeySelector returns the SecretKeySelector which can be used to fetch\n// the backup repository key.\nfunc RepoKeySelector() *corev1api.SecretKeySelector {\n\t// For now, all backup repos share the same key so we don't need the repoName to fetch it.\n\t// When we move to full-backup encryption, we'll likely have a separate key per backup repo\n\t// (all within the Velero server's namespace) so RepoKeySelector will need to select the key\n\t// for that repo.\n\treturn builder.ForSecretKeySelector(credentialsSecretName, credentialsKey).Result()\n}\n" }, { "path": "pkg/repository/keys/keys_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage keys\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestRepoKeySelector(t *testing.T) {\n\tselector := RepoKeySelector()\n\n\trequire.Equal(t, credentialsSecretName, selector.Name)\n\trequire.Equal(t, credentialsKey, selector.Key)\n}\n" }, { "path": "pkg/repository/locker.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage repository\n\nimport \"sync\"\n\n// RepoLocker manages exclusive/non-exclusive locks for\n// operations against backup repositories. The semantics\n// of exclusive/non-exclusive locks are the same as for\n// a sync.RWMutex, where a non-exclusive lock is equivalent\n// to a read lock, and an exclusive lock is equivalent to\n// a write lock.\ntype RepoLocker struct {\n\tmu sync.Mutex\n\tlocks map[string]*sync.RWMutex\n}\n\nfunc NewRepoLocker() *RepoLocker {\n\treturn &RepoLocker{\n\t\tlocks: make(map[string]*sync.RWMutex),\n\t}\n}\n\n// LockExclusive acquires an exclusive lock for the specified\n// repository. This function blocks until no other locks exist\n// for the repo.\nfunc (rl *RepoLocker) LockExclusive(name string) {\n\trl.ensureLock(name).Lock()\n}\n\n// Lock acquires a non-exclusive lock for the specified\n// repository. This function blocks until no exclusive\n// locks exist for the repo.\nfunc (rl *RepoLocker) Lock(name string) {\n\trl.ensureLock(name).RLock()\n}\n\n// UnlockExclusive releases an exclusive lock for the repo.\nfunc (rl *RepoLocker) UnlockExclusive(name string) {\n\trl.ensureLock(name).Unlock()\n}\n\n// Unlock releases a non-exclusive lock for the repo.\nfunc (rl *RepoLocker) Unlock(name string) {\n\trl.ensureLock(name).RUnlock()\n}\n\nfunc (rl *RepoLocker) ensureLock(name string) *sync.RWMutex {\n\trl.mu.Lock()\n\tdefer rl.mu.Unlock()\n\n\tif _, ok := rl.locks[name]; !ok {\n\t\trl.locks[name] = new(sync.RWMutex)\n\t}\n\n\treturn rl.locks[name]\n}\n" }, { "path": "pkg/repository/maintenance/maintenance.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage maintenance\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"math\"\n\t\"sort\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tbatchv1api \"k8s.io/api/batch/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n\tvelerolabel \"github.com/vmware-tanzu/velero/pkg/label\"\n\tvelerotypes \"github.com/vmware-tanzu/velero/pkg/types\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n\tveleroutil \"github.com/vmware-tanzu/velero/pkg/util/velero\"\n)\n\nconst (\n\tRepositoryNameLabel = \"velero.io/repo-name\"\n\tGlobalKeyForRepoMaintenanceJobCM = \"global\"\n\tTerminationLogIndicator = \"Repo maintenance error: \"\n\n\tDefaultKeepLatestMaintenanceJobs = 3\n\tDefaultMaintenanceJobCPURequest = \"0\"\n\tDefaultMaintenanceJobCPULimit = \"0\"\n\tDefaultMaintenanceJobMemRequest = \"0\"\n\tDefaultMaintenanceJobMemLimit = \"0\"\n)\n\nfunc GenerateJobName(repo string) string {\n\tmillisecond := time.Now().UTC().UnixMilli() // millisecond\n\n\tjobName := fmt.Sprintf(\"%s-maintain-job-%d\", repo, millisecond)\n\tif len(jobName) > 63 { // k8s job name length limit\n\t\tjobName = fmt.Sprintf(\"repo-maintain-job-%d\", millisecond)\n\t}\n\n\treturn jobName\n}\n\n// DeleteOldJobs deletes old maintenance jobs and keeps the latest N jobs\nfunc DeleteOldJobs(cli client.Client, repo velerov1api.BackupRepository, keep int, logger logrus.FieldLogger) error {\n\tlogger.Infof(\"Start to delete old maintenance jobs. %d jobs will be kept.\", keep)\n\t// Get the maintenance job list by label\n\tjobList := &batchv1api.JobList{}\n\terr := cli.List(\n\t\tcontext.TODO(),\n\t\tjobList,\n\t\t&client.ListOptions{\n\t\t\tNamespace: repo.Namespace,\n\t\t\tLabelSelector: labels.SelectorFromSet(\n\t\t\t\tmap[string]string{\n\t\t\t\t\tRepositoryNameLabel: velerolabel.ReturnNameOrHash(repo.Name),\n\t\t\t\t},\n\t\t\t),\n\t\t},\n\t)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Delete old maintenance jobs\n\tif len(jobList.Items) > keep {\n\t\tsort.Slice(jobList.Items, func(i, j int) bool {\n\t\t\treturn jobList.Items[i].CreationTimestamp.Before(&jobList.Items[j].CreationTimestamp)\n\t\t})\n\t\tfor i := 0; i < len(jobList.Items)-keep; i++ {\n\t\t\terr = cli.Delete(context.TODO(), &jobList.Items[i], client.PropagationPolicy(metav1.DeletePropagationBackground))\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t}\n\n\treturn nil\n}\n\nvar waitCompletionBackOff = wait.Backoff{\n\tDuration: time.Minute * 20,\n\tSteps: math.MaxInt,\n\tFactor: 2,\n\tCap: time.Hour * 12,\n}\n\n// waitForJobComplete wait for completion of the specified job and update the latest job object\nfunc waitForJobComplete(ctx context.Context, client client.Client, ns string, job string, logger logrus.FieldLogger) (*batchv1api.Job, error) {\n\tvar ret *batchv1api.Job\n\n\tbackOff := waitCompletionBackOff\n\n\tstartTime := time.Now()\n\tnextCheckpoint := startTime.Add(backOff.Step())\n\n\terr := wait.PollUntilContextCancel(ctx, time.Second, true, func(ctx context.Context) (bool, error) {\n\t\tupdated := &batchv1api.Job{}\n\t\terr := client.Get(ctx, types.NamespacedName{Namespace: ns, Name: job}, updated)\n\t\tif err != nil && !apierrors.IsNotFound(err) {\n\t\t\treturn false, err\n\t\t}\n\n\t\tret = updated\n\n\t\tif updated.Status.Succeeded > 0 {\n\t\t\treturn true, nil\n\t\t}\n\n\t\tif updated.Status.Failed > 0 {\n\t\t\treturn true, nil\n\t\t}\n\n\t\tnow := time.Now()\n\t\tif now.After(nextCheckpoint) {\n\t\t\tlogger.Warnf(\"Repo maintenance job %s has lasted %v minutes\", job, now.Sub(startTime).Minutes())\n\t\t\tnextCheckpoint = now.Add(backOff.Step())\n\t\t}\n\n\t\treturn false, nil\n\t})\n\n\treturn ret, err\n}\n\nfunc getResultFromJob(cli client.Client, job *batchv1api.Job) (string, error) {\n\t// Get the maintenance job related pod by label selector\n\tpodList := &corev1api.PodList{}\n\terr := cli.List(context.TODO(), podList, client.InNamespace(job.Namespace), client.MatchingLabels(map[string]string{\"job-name\": job.Name}))\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\tif len(podList.Items) == 0 {\n\t\treturn \"\", errors.Errorf(\"no pod found for job %s\", job.Name)\n\t}\n\n\t// we only have one maintenance pod for the job\n\tpod := podList.Items[0]\n\n\tstatuses := pod.Status.ContainerStatuses\n\tif len(statuses) == 0 {\n\t\treturn \"\", errors.Errorf(\"no container statuses found for job %s\", job.Name)\n\t}\n\n\t// we only have one maintenance container\n\tterminated := statuses[0].State.Terminated\n\tif terminated == nil {\n\t\treturn \"\", errors.Errorf(\"container for job %s is not terminated\", job.Name)\n\t}\n\n\tif terminated.Message == \"\" {\n\t\treturn \"\", nil\n\t}\n\n\tidx := strings.Index(terminated.Message, TerminationLogIndicator)\n\tif idx == -1 {\n\t\treturn \"\", errors.New(\"error to locate repo maintenance error indicator from termination message\")\n\t}\n\n\tif idx+len(TerminationLogIndicator) >= len(terminated.Message) {\n\t\treturn \"\", errors.New(\"nothing after repo maintenance error indicator in termination message\")\n\t}\n\n\treturn terminated.Message[idx+len(TerminationLogIndicator):], nil\n}\n\n// getJobConfig is called to get the Maintenance Job Config for the\n// BackupRepository specified by the repo parameter.\n//\n// Params:\n//\n//\tctx: the Go context used for controller-runtime client.\n//\tclient: the controller-runtime client.\n//\tlogger: the logger.\n//\tveleroNamespace: the Velero-installed namespace. It's used to retrieve the BackupRepository.\n//\trepoMaintenanceJobConfig: the repository maintenance job ConfigMap name.\n//\trepo: the BackupRepository needs to run the maintenance Job.\nfunc getJobConfig(\n\tctx context.Context,\n\tclient client.Client,\n\tlogger logrus.FieldLogger,\n\tveleroNamespace string,\n\trepoMaintenanceJobConfig string,\n\trepo *velerov1api.BackupRepository,\n) (*velerotypes.JobConfigs, error) {\n\tvar cm corev1api.ConfigMap\n\tif err := client.Get(\n\t\tctx,\n\t\ttypes.NamespacedName{\n\t\t\tNamespace: veleroNamespace,\n\t\t\tName: repoMaintenanceJobConfig,\n\t\t},\n\t\t&cm,\n\t); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\treturn nil, nil\n\t\t} else {\n\t\t\treturn nil, errors.Wrapf(\n\t\t\t\terr,\n\t\t\t\t\"fail to get repo maintenance job configs %s\", repoMaintenanceJobConfig)\n\t\t}\n\t}\n\n\tif cm.Data == nil {\n\t\treturn nil, errors.Errorf(\"data is not available in config map %s\", repoMaintenanceJobConfig)\n\t}\n\n\t// Generate the BackupRepository key.\n\t// If using the BackupRepository name as the is more intuitive,\n\t// but the BackupRepository generation is dynamic. We cannot assume\n\t// they are ready when installing Velero.\n\t// Instead we use the volume source namespace, BSL name, and the uploader\n\t// type to represent the BackupRepository. The combination of those three\n\t// keys can identify a unique BackupRepository.\n\trepoJobConfigKey := repo.Spec.VolumeNamespace + \"-\" +\n\t\trepo.Spec.BackupStorageLocation + \"-\" + repo.Spec.RepositoryType\n\n\tvar result *velerotypes.JobConfigs\n\tif _, ok := cm.Data[repoJobConfigKey]; ok {\n\t\tlogger.Debugf(\"Find the repo maintenance config %s for repo %s\", repoJobConfigKey, repo.Name)\n\t\tresult = new(velerotypes.JobConfigs)\n\t\tif err := json.Unmarshal([]byte(cm.Data[repoJobConfigKey]), result); err != nil {\n\t\t\treturn nil, errors.Wrapf(\n\t\t\t\terr,\n\t\t\t\t\"fail to unmarshal configs from %s's key %s\",\n\t\t\t\trepoMaintenanceJobConfig,\n\t\t\t\trepoJobConfigKey)\n\t\t}\n\t}\n\n\tif _, ok := cm.Data[GlobalKeyForRepoMaintenanceJobCM]; ok {\n\t\tlogger.Debugf(\"Find the global repo maintenance config for repo %s\", repo.Name)\n\n\t\tif result == nil {\n\t\t\tresult = new(velerotypes.JobConfigs)\n\t\t}\n\n\t\tglobalResult := new(velerotypes.JobConfigs)\n\n\t\tif err := json.Unmarshal([]byte(cm.Data[GlobalKeyForRepoMaintenanceJobCM]), globalResult); err != nil {\n\t\t\treturn nil, errors.Wrapf(\n\t\t\t\terr,\n\t\t\t\t\"fail to unmarshal configs from %s's key %s\",\n\t\t\t\trepoMaintenanceJobConfig,\n\t\t\t\tGlobalKeyForRepoMaintenanceJobCM)\n\t\t}\n\n\t\tif result.PodResources == nil && globalResult.PodResources != nil {\n\t\t\tresult.PodResources = globalResult.PodResources\n\t\t}\n\n\t\tif len(result.LoadAffinities) == 0 {\n\t\t\tresult.LoadAffinities = globalResult.LoadAffinities\n\t\t}\n\n\t\tif result.KeepLatestMaintenanceJobs == nil && globalResult.KeepLatestMaintenanceJobs != nil {\n\t\t\tresult.KeepLatestMaintenanceJobs = globalResult.KeepLatestMaintenanceJobs\n\t\t}\n\n\t\t// Priority class is only read from global config, not per-repository\n\t\tif globalResult.PriorityClassName != \"\" {\n\t\t\tresult.PriorityClassName = globalResult.PriorityClassName\n\t\t}\n\n\t\t// Pod's labels are only read from global config, not per-repository\n\t\tif len(globalResult.PodLabels) > 0 {\n\t\t\tresult.PodLabels = globalResult.PodLabels\n\t\t}\n\n\t\t// Pod's annotations are only read from global config, not per-repository\n\t\tif len(globalResult.PodAnnotations) > 0 {\n\t\t\tresult.PodAnnotations = globalResult.PodAnnotations\n\t\t}\n\t}\n\n\tlogger.Debugf(\"Configuration content for repository %s is %+v\", repo.Name, result)\n\n\treturn result, nil\n}\n\n// GetKeepLatestMaintenanceJobs returns the configured number of maintenance jobs to keep from the JobConfigs.\n// Because the CLI configured Job kept number is deprecated,\n// if not configured in the ConfigMap, it returns default value to indicate using the fallback value.\nfunc GetKeepLatestMaintenanceJobs(\n\tctx context.Context,\n\tclient client.Client,\n\tlogger logrus.FieldLogger,\n\tveleroNamespace string,\n\trepoMaintenanceJobConfig string,\n\trepo *velerov1api.BackupRepository,\n) (int, error) {\n\tif repoMaintenanceJobConfig == \"\" {\n\t\treturn DefaultKeepLatestMaintenanceJobs, nil\n\t}\n\n\tconfig, err := getJobConfig(ctx, client, logger, veleroNamespace, repoMaintenanceJobConfig, repo)\n\tif err != nil {\n\t\treturn DefaultKeepLatestMaintenanceJobs, err\n\t}\n\n\tif config != nil && config.KeepLatestMaintenanceJobs != nil {\n\t\treturn *config.KeepLatestMaintenanceJobs, nil\n\t}\n\n\treturn DefaultKeepLatestMaintenanceJobs, nil\n}\n\n// WaitJobComplete waits the completion of the specified maintenance job and return the BackupRepositoryMaintenanceStatus\nfunc WaitJobComplete(cli client.Client, ctx context.Context, jobName, ns string, logger logrus.FieldLogger) (velerov1api.BackupRepositoryMaintenanceStatus, error) {\n\tlog := logger.WithField(\"job name\", jobName)\n\n\tmaintenanceJob, err := waitForJobComplete(ctx, cli, ns, jobName, logger)\n\tif err != nil {\n\t\treturn velerov1api.BackupRepositoryMaintenanceStatus{}, errors.Wrap(err, \"error to wait for maintenance job complete\")\n\t}\n\n\tlog.Infof(\"Maintenance repo complete, succeeded %v, failed %v\", maintenanceJob.Status.Succeeded, maintenanceJob.Status.Failed)\n\n\tresult := \"\"\n\tif maintenanceJob.Status.Failed > 0 {\n\t\tif r, err := getResultFromJob(cli, maintenanceJob); err != nil {\n\t\t\tlog.WithError(err).Warn(\"Failed to get maintenance job result\")\n\t\t\tresult = \"Repo maintenance failed but result is not retrieveable\"\n\t\t} else {\n\t\t\tresult = r\n\t\t}\n\t}\n\n\treturn composeStatusFromJob(maintenanceJob, result), nil\n}\n\n// WaitAllJobsComplete checks all the incomplete maintenance jobs of the specified repo and wait for them to complete,\n// and then return the maintenance jobs' status in the range of limit\nfunc WaitAllJobsComplete(ctx context.Context, cli client.Client, repo *velerov1api.BackupRepository, limit int, log logrus.FieldLogger) ([]velerov1api.BackupRepositoryMaintenanceStatus, error) {\n\tjobList := &batchv1api.JobList{}\n\terr := cli.List(\n\t\tcontext.TODO(),\n\t\tjobList,\n\t\t&client.ListOptions{\n\t\t\tNamespace: repo.Namespace,\n\t\t\tLabelSelector: labels.SelectorFromSet(\n\t\t\t\tmap[string]string{\n\t\t\t\t\tRepositoryNameLabel: velerolabel.ReturnNameOrHash(repo.Name),\n\t\t\t\t},\n\t\t\t),\n\t\t},\n\t)\n\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error listing maintenance job for repo %s\", repo.Name)\n\t}\n\n\tif len(jobList.Items) == 0 {\n\t\treturn nil, nil\n\t}\n\n\tsort.Slice(jobList.Items, func(i, j int) bool {\n\t\treturn jobList.Items[i].CreationTimestamp.Time.Before(jobList.Items[j].CreationTimestamp.Time)\n\t})\n\n\thistory := []velerov1api.BackupRepositoryMaintenanceStatus{}\n\n\tstartPos := len(jobList.Items) - limit\n\tif startPos < 0 {\n\t\tstartPos = 0\n\t}\n\n\tfor i := startPos; i < len(jobList.Items); i++ {\n\t\tjob := &jobList.Items[i]\n\n\t\tif job.Status.Succeeded == 0 && job.Status.Failed == 0 {\n\t\t\tlog.Infof(\"Waiting for maintenance job %s to complete\", job.Name)\n\n\t\t\tupdated, err := waitForJobComplete(ctx, cli, job.Namespace, job.Name, log)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, errors.Wrapf(err, \"error waiting maintenance job[%s] complete\", job.Name)\n\t\t\t}\n\n\t\t\tjob = updated\n\t\t}\n\n\t\tmessage := \"\"\n\t\tif job.Status.Failed > 0 {\n\t\t\tif msg, err := getResultFromJob(cli, job); err != nil {\n\t\t\t\tlog.WithError(err).Warnf(\"Failed to get result of maintenance job %s\", job.Name)\n\t\t\t\tmessage = fmt.Sprintf(\"Repo maintenance failed but result is not retrieveable, err: %v\", err)\n\t\t\t} else {\n\t\t\t\tmessage = msg\n\t\t\t}\n\t\t}\n\n\t\thistory = append(history, composeStatusFromJob(job, message))\n\t}\n\n\treturn history, nil\n}\n\n// StartNewJob creates a new maintenance job\nfunc StartNewJob(\n\tcli client.Client,\n\tctx context.Context,\n\trepo *velerov1api.BackupRepository,\n\trepoMaintenanceJobConfig string,\n\tlogLevel logrus.Level,\n\tlogFormat *logging.FormatFlag,\n\tlogger logrus.FieldLogger,\n) (string, error) {\n\tbsl := &velerov1api.BackupStorageLocation{}\n\tif err := cli.Get(ctx, client.ObjectKey{Namespace: repo.Namespace, Name: repo.Spec.BackupStorageLocation}, bsl); err != nil {\n\t\treturn \"\", errors.WithStack(err)\n\t}\n\n\tlog := logger.WithFields(logrus.Fields{\n\t\t\"BSL name\": bsl.Name,\n\t\t\"repo type\": repo.Spec.RepositoryType,\n\t\t\"repo name\": repo.Name,\n\t\t\"repo UID\": repo.UID,\n\t})\n\n\tjobConfig, err := getJobConfig(\n\t\tctx,\n\t\tcli,\n\t\tlog,\n\t\trepo.Namespace,\n\t\trepoMaintenanceJobConfig,\n\t\trepo,\n\t)\n\tif err != nil {\n\t\tlog.Warnf(\"Fail to find the ConfigMap %s to build maintenance job with error: %s. Use default value.\",\n\t\t\trepo.Namespace+\"/\"+repoMaintenanceJobConfig,\n\t\t\terr.Error(),\n\t\t)\n\t}\n\n\tlog.Info(\"Starting maintenance repo\")\n\n\tmaintenanceJob, err := buildJob(cli, ctx, repo, bsl.Name, jobConfig, logLevel, logFormat, log)\n\tif err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error to build maintenance job\")\n\t}\n\n\tlog = log.WithField(\"job\", fmt.Sprintf(\"%s/%s\", maintenanceJob.Namespace, maintenanceJob.Name))\n\n\tif err := cli.Create(ctx, maintenanceJob); err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error to create maintenance job\")\n\t}\n\n\tlog.Info(\"Repo maintenance job started\")\n\n\treturn maintenanceJob.Name, nil\n}\n\n// buildTolerationsForMaintenanceJob builds the tolerations for maintenance jobs.\n// It includes the required Windows toleration for backward compatibility and filters\n// tolerations from the Velero deployment to only include those with keys that are\n// in the ThirdPartyTolerations allowlist, following the same pattern as labels and annotations.\nfunc buildTolerationsForMaintenanceJob(deployment *appsv1api.Deployment) []corev1api.Toleration {\n\t// Start with the Windows toleration for backward compatibility\n\twindowsToleration := corev1api.Toleration{\n\t\tKey: \"os\",\n\t\tOperator: \"Equal\",\n\t\tEffect: \"NoSchedule\",\n\t\tValue: \"windows\",\n\t}\n\tresult := []corev1api.Toleration{windowsToleration}\n\n\t// Filter tolerations from the Velero deployment to only include allowed ones\n\t// Only tolerations that exist on the deployment AND have keys in the allowlist are inherited\n\tdeploymentTolerations := veleroutil.GetTolerationsFromVeleroServer(deployment)\n\tfor _, k := range util.ThirdPartyTolerations {\n\t\tfor _, toleration := range deploymentTolerations {\n\t\t\tif toleration.Key == k {\n\t\t\t\tresult = append(result, toleration)\n\t\t\t\tbreak // Only add the first matching toleration for each allowed key\n\t\t\t}\n\t\t}\n\t}\n\n\treturn result\n}\n\nfunc getPriorityClassName(ctx context.Context, cli client.Client, config *velerotypes.JobConfigs, logger logrus.FieldLogger) string {\n\t// Use the priority class name from the global job configuration if available\n\t// Note: Priority class is only read from global config, not per-repository\n\tif config != nil && config.PriorityClassName != \"\" {\n\t\t// Validate that the priority class exists in the cluster\n\t\tif err := kube.ValidatePriorityClassWithClient(ctx, cli, config.PriorityClassName); err != nil {\n\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\tlogger.Warnf(\"Priority class %q not found in cluster. Job creation may fail if the priority class doesn't exist when jobs are scheduled.\", config.PriorityClassName)\n\t\t\t} else {\n\t\t\t\tlogger.WithError(err).Warnf(\"Failed to validate priority class %q\", config.PriorityClassName)\n\t\t\t}\n\t\t\t// Still return the priority class name to let Kubernetes handle the error\n\t\t\treturn config.PriorityClassName\n\t\t}\n\t\tlogger.Infof(\"Validated priority class %q exists in cluster\", config.PriorityClassName)\n\t\treturn config.PriorityClassName\n\t}\n\treturn \"\"\n}\n\nfunc buildJob(\n\tcli client.Client,\n\tctx context.Context,\n\trepo *velerov1api.BackupRepository,\n\tbslName string,\n\tconfig *velerotypes.JobConfigs,\n\tlogLevel logrus.Level,\n\tlogFormat *logging.FormatFlag,\n\tlogger logrus.FieldLogger,\n) (*batchv1api.Job, error) {\n\t// Get the Velero server deployment\n\tdeployment := &appsv1api.Deployment{}\n\terr := cli.Get(ctx, types.NamespacedName{Name: \"velero\", Namespace: repo.Namespace}, deployment)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Get the environment variables from the Velero server deployment\n\tenvVars := veleroutil.GetEnvVarsFromVeleroServer(deployment)\n\n\t// Get the referenced storage from the Velero server deployment\n\tenvFromSources := veleroutil.GetEnvFromSourcesFromVeleroServer(deployment)\n\n\t// Get the volume mounts from the Velero server deployment\n\tvolumeMounts := veleroutil.GetVolumeMountsFromVeleroServer(deployment)\n\n\t// Get the volumes from the Velero server deployment\n\tvolumes := veleroutil.GetVolumesFromVeleroServer(deployment)\n\n\t// Get the service account from the Velero server deployment\n\tserviceAccount := veleroutil.GetServiceAccountFromVeleroServer(deployment)\n\n\t// Get the security context from the Velero server deployment\n\tsecurityContext := veleroutil.GetContainerSecurityContextsFromVeleroServer(deployment)\n\n\t// Get the pod security context from the Velero server deployment\n\tpodSecurityContext := veleroutil.GetPodSecurityContextsFromVeleroServer(deployment)\n\n\timagePullSecrets := veleroutil.GetImagePullSecretsFromVeleroServer(deployment)\n\n\t// Get image\n\timage := veleroutil.GetVeleroServerImage(deployment)\n\n\t// Set resource limits and requests\n\tcpuRequest := DefaultMaintenanceJobCPURequest\n\tmemRequest := DefaultMaintenanceJobMemRequest\n\tephemeralStorageRequest := constant.DefaultEphemeralStorageRequest\n\tcpuLimit := DefaultMaintenanceJobCPULimit\n\tmemLimit := DefaultMaintenanceJobMemLimit\n\tephemeralStorageLimit := constant.DefaultEphemeralStorageLimit\n\tif config != nil && config.PodResources != nil {\n\t\tcpuRequest = config.PodResources.CPURequest\n\t\tmemRequest = config.PodResources.MemoryRequest\n\t\tcpuLimit = config.PodResources.CPULimit\n\t\tmemLimit = config.PodResources.MemoryLimit\n\t\t// To make the PodResources ConfigMap without ephemeral storage request/limit backward compatible,\n\t\t// need to avoid set value as empty, because empty string will cause parsing error.\n\t\tif config.PodResources.EphemeralStorageRequest != \"\" {\n\t\t\tephemeralStorageRequest = config.PodResources.EphemeralStorageRequest\n\t\t}\n\t\tif config.PodResources.EphemeralStorageLimit != \"\" {\n\t\t\tephemeralStorageLimit = config.PodResources.EphemeralStorageLimit\n\t\t}\n\t}\n\tresources, err := kube.ParseResourceRequirements(\n\t\tcpuRequest,\n\t\tmemRequest,\n\t\tephemeralStorageRequest,\n\t\tcpuLimit,\n\t\tmemLimit,\n\t\tephemeralStorageLimit,\n\t)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to parse resource requirements for maintenance job\")\n\t}\n\n\tpodLabels := map[string]string{\n\t\tRepositoryNameLabel: velerolabel.ReturnNameOrHash(repo.Name),\n\t}\n\tif config != nil && len(config.PodLabels) > 0 {\n\t\tfor k, v := range config.PodLabels {\n\t\t\tpodLabels[k] = v\n\t\t}\n\t} else {\n\t\tfor _, k := range util.ThirdPartyLabels {\n\t\t\tif v := veleroutil.GetVeleroServerLabelValue(deployment, k); v != \"\" {\n\t\t\t\tpodLabels[k] = v\n\t\t\t}\n\t\t}\n\t}\n\n\tpodAnnotations := map[string]string{}\n\tif config != nil && len(config.PodAnnotations) > 0 {\n\t\tfor k, v := range config.PodAnnotations {\n\t\t\tpodAnnotations[k] = v\n\t\t}\n\t} else {\n\t\tfor _, k := range util.ThirdPartyAnnotations {\n\t\t\tif v := veleroutil.GetVeleroServerAnnotationValue(deployment, k); v != \"\" {\n\t\t\t\tpodAnnotations[k] = v\n\t\t\t}\n\t\t}\n\t}\n\n\t// Set arguments\n\targs := []string{\"repo-maintenance\"}\n\targs = append(args, fmt.Sprintf(\"--repo-name=%s\", repo.Spec.VolumeNamespace))\n\targs = append(args, fmt.Sprintf(\"--repo-type=%s\", repo.Spec.RepositoryType))\n\targs = append(args, fmt.Sprintf(\"--backup-storage-location=%s\", bslName))\n\targs = append(args, fmt.Sprintf(\"--log-level=%s\", logLevel.String()))\n\targs = append(args, fmt.Sprintf(\"--log-format=%s\", logFormat.String()))\n\n\t// build the maintenance job\n\tjob := &batchv1api.Job{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: GenerateJobName(repo.Name),\n\t\t\tNamespace: repo.Namespace,\n\t\t\tLabels: map[string]string{\n\t\t\t\tRepositoryNameLabel: velerolabel.ReturnNameOrHash(repo.Name),\n\t\t\t},\n\t\t},\n\t\tSpec: batchv1api.JobSpec{\n\t\t\tBackoffLimit: new(int32), // Never retry\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"velero-repo-maintenance-pod\",\n\t\t\t\t\tLabels: podLabels,\n\t\t\t\t\tAnnotations: podAnnotations,\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"velero-repo-maintenance-container\",\n\t\t\t\t\t\t\tImage: image,\n\t\t\t\t\t\t\tCommand: []string{\n\t\t\t\t\t\t\t\t\"/velero\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tArgs: args,\n\t\t\t\t\t\t\tImagePullPolicy: corev1api.PullIfNotPresent,\n\t\t\t\t\t\t\tEnv: envVars,\n\t\t\t\t\t\t\tEnvFrom: envFromSources,\n\t\t\t\t\t\t\tVolumeMounts: volumeMounts,\n\t\t\t\t\t\t\tResources: resources,\n\t\t\t\t\t\t\tSecurityContext: securityContext,\n\t\t\t\t\t\t\tTerminationMessagePolicy: corev1api.TerminationMessageFallbackToLogsOnError,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tPriorityClassName: getPriorityClassName(ctx, cli, config, logger),\n\t\t\t\t\tRestartPolicy: corev1api.RestartPolicyNever,\n\t\t\t\t\tSecurityContext: podSecurityContext,\n\t\t\t\t\tVolumes: volumes,\n\t\t\t\t\tServiceAccountName: serviceAccount,\n\t\t\t\t\tTolerations: buildTolerationsForMaintenanceJob(deployment),\n\t\t\t\t\tImagePullSecrets: imagePullSecrets,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tif config != nil && len(config.LoadAffinities) > 0 {\n\t\taffinity := kube.ToSystemAffinity(config.LoadAffinities[0], nil)\n\t\tjob.Spec.Template.Spec.Affinity = affinity\n\t}\n\n\treturn job, nil\n}\n\nfunc composeStatusFromJob(job *batchv1api.Job, message string) velerov1api.BackupRepositoryMaintenanceStatus {\n\tresult := velerov1api.BackupRepositoryMaintenanceSucceeded\n\tif job.Status.Failed > 0 {\n\t\tresult = velerov1api.BackupRepositoryMaintenanceFailed\n\t}\n\n\treturn velerov1api.BackupRepositoryMaintenanceStatus{\n\t\tResult: result,\n\t\tStartTimestamp: &job.CreationTimestamp,\n\t\tCompleteTimestamp: job.Status.CompletionTime,\n\t\tMessage: message,\n\t}\n}\n" }, { "path": "pkg/repository/maintenance/maintenance_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage maintenance\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"math\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tbatchv1api \"k8s.io/api/batch/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tschedulingv1 \"k8s.io/api/scheduling/v1\"\n\t\"k8s.io/apimachinery/pkg/api/resource\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tvelerolabel \"github.com/vmware-tanzu/velero/pkg/label\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/provider\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\tvelerotypes \"github.com/vmware-tanzu/velero/pkg/types\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n)\n\nfunc TestGenerateJobName(t *testing.T) {\n\ttestCases := []struct {\n\t\trepo string\n\t\texpectedStart string\n\t}{\n\t\t{\n\t\t\trepo: \"example\",\n\t\t\texpectedStart: \"example-maintain-job-\",\n\t\t},\n\t\t{\n\t\t\trepo: strings.Repeat(\"a\", 60),\n\t\t\texpectedStart: \"repo-maintain-job-\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.repo, func(t *testing.T) {\n\t\t\t// Call the function to test\n\t\t\tjobName := GenerateJobName(tc.repo)\n\n\t\t\t// Check if the generated job name starts with the expected prefix\n\t\t\tif !strings.HasPrefix(jobName, tc.expectedStart) {\n\t\t\t\tt.Errorf(\"generated job name does not start with expected prefix\")\n\t\t\t}\n\n\t\t\t// Check if the length of the generated job name exceeds the Kubernetes limit\n\t\t\tif len(jobName) > 63 {\n\t\t\t\tt.Errorf(\"generated job name exceeds Kubernetes limit\")\n\t\t\t}\n\t\t})\n\t}\n}\nfunc TestDeleteOldJobs(t *testing.T) {\n\t// Set up test repo and keep value\n\trepo := &velerov1api.BackupRepository{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"label with more than 63 characters should be modified\",\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t},\n\t}\n\tkeep := 1\n\n\tjobArray := []client.Object{\n\t\t&batchv1api.Job{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"job-0\",\n\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\tLabels: map[string]string{RepositoryNameLabel: velerolabel.ReturnNameOrHash(repo.Name)},\n\t\t\t},\n\t\t\tSpec: batchv1api.JobSpec{},\n\t\t},\n\t\t&batchv1api.Job{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"job-1\",\n\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\tLabels: map[string]string{RepositoryNameLabel: velerolabel.ReturnNameOrHash(repo.Name)},\n\t\t\t},\n\t\t\tSpec: batchv1api.JobSpec{},\n\t\t},\n\t}\n\n\tnewJob := &batchv1api.Job{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"job-new\",\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\tLabels: map[string]string{RepositoryNameLabel: velerolabel.ReturnNameOrHash(repo.Name)},\n\t\t},\n\t\tSpec: batchv1api.JobSpec{},\n\t}\n\n\t// Create a fake Kubernetes client with 2 jobs.\n\tscheme := runtime.NewScheme()\n\t_ = batchv1api.AddToScheme(scheme)\n\tcli := fake.NewClientBuilder().WithScheme(scheme).WithObjects(jobArray...).Build()\n\n\t// Create a new job\n\trequire.NoError(t, cli.Create(t.Context(), newJob))\n\n\t// Call the function\n\trequire.NoError(t, DeleteOldJobs(cli, *repo, keep, velerotest.NewLogger()))\n\n\t// Get the remaining jobs\n\tjobList := &batchv1api.JobList{}\n\trequire.NoError(t, cli.List(t.Context(), jobList, client.MatchingLabels(map[string]string{RepositoryNameLabel: repo.Name})))\n\n\t// We expect the number of jobs to be equal to 'keep'\n\tassert.Len(t, jobList.Items, keep)\n\n\t// Only the new created job should be left.\n\tassert.Equal(t, jobList.Items[0].Name, newJob.Name)\n}\n\nfunc TestWaitForJobComplete(t *testing.T) {\n\t// Set up test job\n\tjob := &batchv1api.Job{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-job\",\n\t\t\tNamespace: \"default\",\n\t\t},\n\t\tStatus: batchv1api.JobStatus{},\n\t}\n\n\tschemeFail := runtime.NewScheme()\n\n\tscheme := runtime.NewScheme()\n\tbatchv1api.AddToScheme(scheme)\n\n\twaitCompletionBackOff1 := wait.Backoff{\n\t\tDuration: time.Second,\n\t\tSteps: math.MaxInt,\n\t\tFactor: 2,\n\t\tCap: time.Second * 12,\n\t}\n\n\twaitCompletionBackOff2 := wait.Backoff{\n\t\tDuration: time.Second,\n\t\tSteps: math.MaxInt,\n\t\tFactor: 2,\n\t\tCap: time.Second * 2,\n\t}\n\n\t// Define test cases\n\ttests := []struct {\n\t\tdescription string // Test case description\n\t\tkubeClientObj []runtime.Object\n\t\truntimeScheme *runtime.Scheme\n\t\tjobStatus batchv1api.JobStatus // Job status to set for the test\n\t\tlogBackOff wait.Backoff\n\t\tupdateAfter time.Duration\n\t\texpectedLogs int\n\t\texpectError bool // Whether an error is expected\n\t}{\n\t\t{\n\t\t\tdescription: \"wait error\",\n\t\t\truntimeScheme: schemeFail,\n\t\t\texpectError: true,\n\t\t},\n\t\t{\n\t\t\tdescription: \"Job Succeeded\",\n\t\t\truntimeScheme: scheme,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tjob,\n\t\t\t},\n\t\t\tjobStatus: batchv1api.JobStatus{\n\t\t\t\tSucceeded: 1,\n\t\t\t},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tdescription: \"Job Failed\",\n\t\t\truntimeScheme: scheme,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tjob,\n\t\t\t},\n\t\t\tjobStatus: batchv1api.JobStatus{\n\t\t\t\tFailed: 1,\n\t\t\t},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tdescription: \"Log backoff not to cap\",\n\t\t\truntimeScheme: scheme,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tjob,\n\t\t\t},\n\t\t\tlogBackOff: waitCompletionBackOff1,\n\t\t\tupdateAfter: time.Second * 8,\n\t\t\texpectedLogs: 3,\n\t\t},\n\t\t{\n\t\t\tdescription: \"Log backoff to cap\",\n\t\t\truntimeScheme: scheme,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tjob,\n\t\t\t},\n\t\t\tlogBackOff: waitCompletionBackOff2,\n\t\t\tupdateAfter: time.Second * 6,\n\t\t\texpectedLogs: 3,\n\t\t},\n\t}\n\n\t// Run tests\n\tfor _, tc := range tests {\n\t\tt.Run(tc.description, func(t *testing.T) {\n\t\t\t// Set the job status\n\t\t\tjob.Status = tc.jobStatus\n\t\t\t// Create a fake Kubernetes client\n\t\t\tfakeClientBuilder := fake.NewClientBuilder()\n\t\t\tfakeClientBuilder = fakeClientBuilder.WithScheme(tc.runtimeScheme)\n\t\t\tfakeClient := fakeClientBuilder.WithRuntimeObjects(tc.kubeClientObj...).Build()\n\n\t\t\tbuffer := []string{}\n\t\t\tlogger := velerotest.NewMultipleLogger(&buffer)\n\n\t\t\twaitCompletionBackOff = tc.logBackOff\n\n\t\t\tif tc.updateAfter != 0 {\n\t\t\t\tgo func() {\n\t\t\t\t\ttime.Sleep(tc.updateAfter)\n\n\t\t\t\t\toriginal := job.DeepCopy()\n\t\t\t\t\tjob.Status.Succeeded = 1\n\t\t\t\t\terr := fakeClient.Status().Patch(t.Context(), job, client.MergeFrom(original))\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t}()\n\t\t\t}\n\n\t\t\t// Call the function\n\t\t\t_, err := waitForJobComplete(t.Context(), fakeClient, job.Namespace, job.Name, logger)\n\n\t\t\t// Check if the error matches the expectation\n\t\t\tif tc.expectError {\n\t\t\t\trequire.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tassert.LessOrEqual(t, len(buffer), tc.expectedLogs)\n\t\t})\n\t}\n}\n\nfunc TestGetResultFromJob(t *testing.T) {\n\t// Set up test job\n\tjob := &batchv1api.Job{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-job\",\n\t\t\tNamespace: \"default\",\n\t\t},\n\t}\n\n\t// Set up test pod with no status\n\tpod := &corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-pod\",\n\t\t\tNamespace: \"default\",\n\t\t\tLabels: map[string]string{\"job-name\": job.Name},\n\t\t},\n\t}\n\n\t// Create a fake Kubernetes client\n\tcli := fake.NewClientBuilder().Build()\n\n\t// test an error should be returned\n\tresult, err := getResultFromJob(cli, job)\n\trequire.EqualError(t, err, \"no pod found for job test-job\")\n\tassert.Empty(t, result)\n\n\tcli = fake.NewClientBuilder().WithObjects(job, pod).Build()\n\n\t// test an error should be returned\n\tresult, err = getResultFromJob(cli, job)\n\trequire.EqualError(t, err, \"no container statuses found for job test-job\")\n\tassert.Empty(t, result)\n\n\t// Set a non-terminated container status to the pod\n\tpod.Status = corev1api.PodStatus{\n\t\tContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t{\n\t\t\t\tState: corev1api.ContainerState{},\n\t\t\t},\n\t\t},\n\t}\n\n\t// Test an error should be returned\n\tcli = fake.NewClientBuilder().WithObjects(job, pod).Build()\n\tresult, err = getResultFromJob(cli, job)\n\trequire.EqualError(t, err, \"container for job test-job is not terminated\")\n\tassert.Empty(t, result)\n\n\t// Set a terminated container status to the pod\n\tpod.Status = corev1api.PodStatus{\n\t\tContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t{\n\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\tTerminated: &corev1api.ContainerStateTerminated{},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\t// This call should return the termination message with no error\n\tcli = fake.NewClientBuilder().WithObjects(job, pod).Build()\n\tresult, err = getResultFromJob(cli, job)\n\trequire.NoError(t, err)\n\tassert.Empty(t, result)\n\n\t// Set a terminated container status with invalidate message to the pod\n\tpod.Status = corev1api.PodStatus{\n\t\tContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t{\n\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\tTerminated: &corev1api.ContainerStateTerminated{\n\t\t\t\t\t\tMessage: \"fake-message\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tcli = fake.NewClientBuilder().WithObjects(job, pod).Build()\n\tresult, err = getResultFromJob(cli, job)\n\trequire.EqualError(t, err, \"error to locate repo maintenance error indicator from termination message\")\n\tassert.Empty(t, result)\n\n\t// Set a terminated container status with empty maintenance error to the pod\n\tpod.Status = corev1api.PodStatus{\n\t\tContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t{\n\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\tTerminated: &corev1api.ContainerStateTerminated{\n\t\t\t\t\t\tMessage: \"Repo maintenance error: \",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tcli = fake.NewClientBuilder().WithObjects(job, pod).Build()\n\tresult, err = getResultFromJob(cli, job)\n\trequire.EqualError(t, err, \"nothing after repo maintenance error indicator in termination message\")\n\tassert.Empty(t, result)\n\n\t// Set a terminated container status with maintenance error to the pod\n\tpod.Status = corev1api.PodStatus{\n\t\tContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t{\n\t\t\t\tState: corev1api.ContainerState{\n\t\t\t\t\tTerminated: &corev1api.ContainerStateTerminated{\n\t\t\t\t\t\tMessage: \"Repo maintenance error: fake-error\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tcli = fake.NewClientBuilder().WithObjects(job, pod).Build()\n\tresult, err = getResultFromJob(cli, job)\n\trequire.NoError(t, err)\n\tassert.Equal(t, \"fake-error\", result)\n}\n\nfunc TestGetJobConfig(t *testing.T) {\n\tkeepLatestMaintenanceJobs := 1\n\tctx := t.Context()\n\tlogger := logrus.New()\n\tveleroNamespace := \"velero\"\n\trepoMaintenanceJobConfig := \"repo-maintenance-job-config\"\n\trepo := &velerov1api.BackupRepository{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: veleroNamespace,\n\t\t\tName: repoMaintenanceJobConfig,\n\t\t},\n\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\tBackupStorageLocation: \"default\",\n\t\t\tRepositoryType: \"kopia\",\n\t\t\tVolumeNamespace: \"test\",\n\t\t},\n\t}\n\n\ttestCases := []struct {\n\t\tname string\n\t\trepoJobConfig *corev1api.ConfigMap\n\t\texpectedConfig *velerotypes.JobConfigs\n\t\texpectedError error\n\t}{\n\t\t{\n\t\t\tname: \"Config not exist\",\n\t\t\texpectedConfig: nil,\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"Invalid JSON\",\n\t\t\trepoJobConfig: &corev1api.ConfigMap{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: veleroNamespace,\n\t\t\t\t\tName: repoMaintenanceJobConfig,\n\t\t\t\t},\n\t\t\t\tData: map[string]string{\n\t\t\t\t\t\"test-default-kopia\": \"{\\\"cpuRequest:\\\"100m\\\"}\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedConfig: nil,\n\t\t\texpectedError: fmt.Errorf(\"fail to unmarshal configs from %s\", repoMaintenanceJobConfig),\n\t\t},\n\t\t{\n\t\t\tname: \"Find config specific for BackupRepository\",\n\t\t\trepoJobConfig: &corev1api.ConfigMap{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: veleroNamespace,\n\t\t\t\t\tName: repoMaintenanceJobConfig,\n\t\t\t\t},\n\t\t\t\tData: map[string]string{\n\t\t\t\t\t\"test-default-kopia\": \"{\\\"podResources\\\":{\\\"cpuRequest\\\":\\\"100m\\\",\\\"cpuLimit\\\":\\\"200m\\\",\\\"memoryRequest\\\":\\\"100Mi\\\",\\\"memoryLimit\\\":\\\"200Mi\\\"},\\\"loadAffinity\\\":[{\\\"nodeSelector\\\":{\\\"matchExpressions\\\":[{\\\"key\\\":\\\"cloud.google.com/machine-family\\\",\\\"operator\\\":\\\"In\\\",\\\"values\\\":[\\\"e2\\\"]}]}}]}\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedConfig: &velerotypes.JobConfigs{\n\t\t\t\tPodResources: &kube.PodResources{\n\t\t\t\t\tCPURequest: \"100m\",\n\t\t\t\t\tCPULimit: \"200m\",\n\t\t\t\t\tMemoryRequest: \"100Mi\",\n\t\t\t\t\tMemoryLimit: \"200Mi\",\n\t\t\t\t},\n\t\t\t\tLoadAffinities: []*kube.LoadAffinity{\n\t\t\t\t\t{\n\t\t\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tKey: \"cloud.google.com/machine-family\",\n\t\t\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\t\t\t\tValues: []string{\"e2\"},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"Find config specific for global\",\n\t\t\trepoJobConfig: &corev1api.ConfigMap{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: veleroNamespace,\n\t\t\t\t\tName: repoMaintenanceJobConfig,\n\t\t\t\t},\n\t\t\t\tData: map[string]string{\n\t\t\t\t\tGlobalKeyForRepoMaintenanceJobCM: \"{\\\"podResources\\\":{\\\"cpuRequest\\\":\\\"50m\\\",\\\"cpuLimit\\\":\\\"100m\\\",\\\"memoryRequest\\\":\\\"50Mi\\\",\\\"memoryLimit\\\":\\\"100Mi\\\"},\\\"loadAffinity\\\":[{\\\"nodeSelector\\\":{\\\"matchExpressions\\\":[{\\\"key\\\":\\\"cloud.google.com/machine-family\\\",\\\"operator\\\":\\\"In\\\",\\\"values\\\":[\\\"n2\\\"]}]}}]}\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedConfig: &velerotypes.JobConfigs{\n\t\t\t\tPodResources: &kube.PodResources{\n\t\t\t\t\tCPURequest: \"50m\",\n\t\t\t\t\tCPULimit: \"100m\",\n\t\t\t\t\tMemoryRequest: \"50Mi\",\n\t\t\t\t\tMemoryLimit: \"100Mi\",\n\t\t\t\t},\n\t\t\t\tLoadAffinities: []*kube.LoadAffinity{\n\t\t\t\t\t{\n\t\t\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tKey: \"cloud.google.com/machine-family\",\n\t\t\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\t\t\t\tValues: []string{\"n2\"},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"Specific config supersede global config\",\n\t\t\trepoJobConfig: &corev1api.ConfigMap{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: veleroNamespace,\n\t\t\t\t\tName: repoMaintenanceJobConfig,\n\t\t\t\t},\n\t\t\t\tData: map[string]string{\n\t\t\t\t\tGlobalKeyForRepoMaintenanceJobCM: \"{\\\"keepLatestMaintenanceJobs\\\":1,\\\"podResources\\\":{\\\"cpuRequest\\\":\\\"50m\\\",\\\"cpuLimit\\\":\\\"100m\\\",\\\"memoryRequest\\\":\\\"50Mi\\\",\\\"memoryLimit\\\":\\\"100Mi\\\"},\\\"loadAffinity\\\":[{\\\"nodeSelector\\\":{\\\"matchExpressions\\\":[{\\\"key\\\":\\\"cloud.google.com/machine-family\\\",\\\"operator\\\":\\\"In\\\",\\\"values\\\":[\\\"n2\\\"]}]}}]}\",\n\t\t\t\t\t\"test-default-kopia\": \"{\\\"podResources\\\":{\\\"cpuRequest\\\":\\\"100m\\\",\\\"cpuLimit\\\":\\\"200m\\\",\\\"memoryRequest\\\":\\\"100Mi\\\",\\\"memoryLimit\\\":\\\"200Mi\\\"},\\\"loadAffinity\\\":[{\\\"nodeSelector\\\":{\\\"matchExpressions\\\":[{\\\"key\\\":\\\"cloud.google.com/machine-family\\\",\\\"operator\\\":\\\"In\\\",\\\"values\\\":[\\\"e2\\\"]}]}}]}\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedConfig: &velerotypes.JobConfigs{\n\t\t\t\tKeepLatestMaintenanceJobs: &keepLatestMaintenanceJobs,\n\t\t\t\tPodResources: &kube.PodResources{\n\t\t\t\t\tCPURequest: \"100m\",\n\t\t\t\t\tCPULimit: \"200m\",\n\t\t\t\t\tMemoryRequest: \"100Mi\",\n\t\t\t\t\tMemoryLimit: \"200Mi\",\n\t\t\t\t},\n\t\t\t\tLoadAffinities: []*kube.LoadAffinity{\n\t\t\t\t\t{\n\t\t\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tKey: \"cloud.google.com/machine-family\",\n\t\t\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\t\t\t\tValues: []string{\"e2\"},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"Configs only exist in global section should supersede specific config\",\n\t\t\trepoJobConfig: &corev1api.ConfigMap{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: veleroNamespace,\n\t\t\t\t\tName: repoMaintenanceJobConfig,\n\t\t\t\t},\n\t\t\t\tData: map[string]string{\n\t\t\t\t\tGlobalKeyForRepoMaintenanceJobCM: \"{\\\"keepLatestMaintenanceJobs\\\":1,\\\"podResources\\\":{\\\"cpuRequest\\\":\\\"50m\\\",\\\"cpuLimit\\\":\\\"100m\\\",\\\"memoryRequest\\\":\\\"50Mi\\\",\\\"memoryLimit\\\":\\\"100Mi\\\"},\\\"loadAffinity\\\":[{\\\"nodeSelector\\\":{\\\"matchExpressions\\\":[{\\\"key\\\":\\\"cloud.google.com/machine-family\\\",\\\"operator\\\":\\\"In\\\",\\\"values\\\":[\\\"n2\\\"]}]}}],\\\"priorityClassName\\\":\\\"global-priority\\\",\\\"podAnnotations\\\":{\\\"global-key\\\":\\\"global-value\\\"},\\\"podLabels\\\":{\\\"global-key\\\":\\\"global-value\\\"}}\",\n\t\t\t\t\t\"test-default-kopia\": \"{\\\"podResources\\\":{\\\"cpuRequest\\\":\\\"100m\\\",\\\"cpuLimit\\\":\\\"200m\\\",\\\"memoryRequest\\\":\\\"100Mi\\\",\\\"memoryLimit\\\":\\\"200Mi\\\"},\\\"loadAffinity\\\":[{\\\"nodeSelector\\\":{\\\"matchExpressions\\\":[{\\\"key\\\":\\\"cloud.google.com/machine-family\\\",\\\"operator\\\":\\\"In\\\",\\\"values\\\":[\\\"e2\\\"]}]}}],\\\"priorityClassName\\\":\\\"specific-priority\\\",\\\"podAnnotations\\\":{\\\"specific-key\\\":\\\"specific-value\\\"},\\\"podLabels\\\":{\\\"specific-key\\\":\\\"specific-value\\\"}}\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedConfig: &velerotypes.JobConfigs{\n\t\t\t\tKeepLatestMaintenanceJobs: &keepLatestMaintenanceJobs,\n\t\t\t\tPodResources: &kube.PodResources{\n\t\t\t\t\tCPURequest: \"100m\",\n\t\t\t\t\tCPULimit: \"200m\",\n\t\t\t\t\tMemoryRequest: \"100Mi\",\n\t\t\t\t\tMemoryLimit: \"200Mi\",\n\t\t\t\t},\n\t\t\t\tLoadAffinities: []*kube.LoadAffinity{\n\t\t\t\t\t{\n\t\t\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tKey: \"cloud.google.com/machine-family\",\n\t\t\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\t\t\t\tValues: []string{\"e2\"},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tPriorityClassName: \"global-priority\",\n\t\t\t\tPodAnnotations: map[string]string{\"global-key\": \"global-value\"},\n\t\t\t\tPodLabels: map[string]string{\"global-key\": \"global-value\"},\n\t\t\t},\n\t\t\texpectedError: nil,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar fakeClient client.Client\n\t\t\tif tc.repoJobConfig != nil {\n\t\t\t\tfakeClient = velerotest.NewFakeControllerRuntimeClient(t, tc.repoJobConfig)\n\t\t\t} else {\n\t\t\t\tfakeClient = velerotest.NewFakeControllerRuntimeClient(t)\n\t\t\t}\n\n\t\t\tjobConfig, err := getJobConfig(\n\t\t\t\tctx,\n\t\t\t\tfakeClient,\n\t\t\t\tlogger,\n\t\t\t\tveleroNamespace,\n\t\t\t\trepoMaintenanceJobConfig,\n\t\t\t\trepo,\n\t\t\t)\n\n\t\t\tif tc.expectedError != nil {\n\t\t\t\trequire.ErrorContains(t, err, tc.expectedError.Error())\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\t\t\trequire.Equal(t, tc.expectedConfig, jobConfig)\n\t\t})\n\t}\n}\n\nfunc TestWaitAllJobsComplete(t *testing.T) {\n\tctx, cancel := context.WithTimeout(t.Context(), time.Second*2)\n\n\tveleroNamespace := \"velero\"\n\trepo := &velerov1api.BackupRepository{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: veleroNamespace,\n\t\t\tName: \"label with more than 63 characters should be modified\",\n\t\t},\n\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\tBackupStorageLocation: \"default\",\n\t\t\tRepositoryType: \"kopia\",\n\t\t\tVolumeNamespace: \"test\",\n\t\t},\n\t}\n\n\tnow := time.Now().Round(time.Second)\n\n\tjobOtherLabel := &batchv1api.Job{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"job1\",\n\t\t\tNamespace: veleroNamespace,\n\t\t\tLabels: map[string]string{RepositoryNameLabel: \"other-repo\"},\n\t\t\tCreationTimestamp: metav1.Time{Time: now},\n\t\t},\n\t}\n\n\tjobIncomplete := &batchv1api.Job{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"job1\",\n\t\t\tNamespace: veleroNamespace,\n\t\t\tLabels: map[string]string{RepositoryNameLabel: velerolabel.ReturnNameOrHash(repo.Name)},\n\t\t\tCreationTimestamp: metav1.Time{Time: now},\n\t\t},\n\t}\n\n\tjobSucceeded1 := &batchv1api.Job{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"job1\",\n\t\t\tNamespace: veleroNamespace,\n\t\t\tLabels: map[string]string{RepositoryNameLabel: velerolabel.ReturnNameOrHash(repo.Name)},\n\t\t\tCreationTimestamp: metav1.Time{Time: now},\n\t\t},\n\t\tStatus: batchv1api.JobStatus{\n\t\t\tStartTime: &metav1.Time{Time: now},\n\t\t\tCompletionTime: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\tSucceeded: 1,\n\t\t},\n\t}\n\n\tjobPodSucceeded1 := builder.ForPod(veleroNamespace, \"job1\").Labels(map[string]string{\"job-name\": \"job1\"}).ContainerStatuses(&corev1api.ContainerStatus{\n\t\tState: corev1api.ContainerState{\n\t\t\tTerminated: &corev1api.ContainerStateTerminated{},\n\t\t},\n\t}).Result()\n\n\tjobFailed1 := &batchv1api.Job{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"job2\",\n\t\t\tNamespace: veleroNamespace,\n\t\t\tLabels: map[string]string{RepositoryNameLabel: velerolabel.ReturnNameOrHash(repo.Name)},\n\t\t\tCreationTimestamp: metav1.Time{Time: now.Add(time.Hour)},\n\t\t},\n\t\tStatus: batchv1api.JobStatus{\n\t\t\tStartTime: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\tFailed: 1,\n\t\t},\n\t}\n\n\tjobPodFailed1 := builder.ForPod(veleroNamespace, \"job2\").Labels(map[string]string{\"job-name\": \"job2\"}).ContainerStatuses(&corev1api.ContainerStatus{\n\t\tState: corev1api.ContainerState{\n\t\t\tTerminated: &corev1api.ContainerStateTerminated{\n\t\t\t\tMessage: \"Repo maintenance error: fake-message-2\",\n\t\t\t},\n\t\t},\n\t}).Result()\n\n\tjobSucceeded2 := &batchv1api.Job{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"job3\",\n\t\t\tNamespace: veleroNamespace,\n\t\t\tLabels: map[string]string{RepositoryNameLabel: velerolabel.ReturnNameOrHash(repo.Name)},\n\t\t\tCreationTimestamp: metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t},\n\t\tStatus: batchv1api.JobStatus{\n\t\t\tStartTime: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\tCompletionTime: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\tSucceeded: 1,\n\t\t},\n\t}\n\n\tjobPodSucceeded2 := builder.ForPod(veleroNamespace, \"job3\").Labels(map[string]string{\"job-name\": \"job3\"}).ContainerStatuses(&corev1api.ContainerStatus{\n\t\tState: corev1api.ContainerState{\n\t\t\tTerminated: &corev1api.ContainerStateTerminated{},\n\t\t},\n\t}).Result()\n\n\tjobSucceeded3 := &batchv1api.Job{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"job4\",\n\t\t\tNamespace: veleroNamespace,\n\t\t\tLabels: map[string]string{RepositoryNameLabel: velerolabel.ReturnNameOrHash(repo.Name)},\n\t\t\tCreationTimestamp: metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t},\n\t\tStatus: batchv1api.JobStatus{\n\t\t\tStartTime: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\tCompletionTime: &metav1.Time{Time: now.Add(time.Hour * 4)},\n\t\t\tSucceeded: 1,\n\t\t},\n\t}\n\n\tjobPodSucceeded3 := builder.ForPod(veleroNamespace, \"job4\").Labels(map[string]string{\"job-name\": \"job4\"}).ContainerStatuses(&corev1api.ContainerStatus{\n\t\tState: corev1api.ContainerState{\n\t\t\tTerminated: &corev1api.ContainerStateTerminated{},\n\t\t},\n\t}).Result()\n\n\tschemeFail := runtime.NewScheme()\n\n\tscheme := runtime.NewScheme()\n\tbatchv1api.AddToScheme(scheme)\n\tcorev1api.AddToScheme(scheme)\n\n\ttestCases := []struct {\n\t\tname string\n\t\tctx context.Context\n\t\tkubeClientObj []runtime.Object\n\t\truntimeScheme *runtime.Scheme\n\t\texpectedStatus []velerov1api.BackupRepositoryMaintenanceStatus\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"list job error\",\n\t\t\truntimeScheme: schemeFail,\n\t\t\texpectedError: \"error listing maintenance job for repo label with more than 63 characters should be modified: no kind is registered for the type v1.JobList in scheme\",\n\t\t},\n\t\t{\n\t\t\tname: \"job not exist\",\n\t\t\truntimeScheme: scheme,\n\t\t},\n\t\t{\n\t\t\tname: \"no matching job\",\n\t\t\truntimeScheme: scheme,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tjobOtherLabel,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"wait complete error\",\n\t\t\tctx: ctx,\n\t\t\truntimeScheme: scheme,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tjobIncomplete,\n\t\t\t},\n\t\t\texpectedError: \"error waiting maintenance job[job1] complete: context deadline exceeded\",\n\t\t},\n\t\t{\n\t\t\tname: \"get result error on succeeded job\",\n\t\t\tctx: t.Context(),\n\t\t\truntimeScheme: scheme,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tjobSucceeded1,\n\t\t\t},\n\t\t\texpectedStatus: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"get result error on failed job\",\n\t\t\tctx: t.Context(),\n\t\t\truntimeScheme: scheme,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tjobFailed1,\n\t\t\t},\n\t\t\texpectedStatus: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceFailed,\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tMessage: \"Repo maintenance failed but result is not retrieveable, err: no pod found for job job2\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"less than limit\",\n\t\t\tctx: t.Context(),\n\t\t\truntimeScheme: scheme,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tjobFailed1,\n\t\t\t\tjobSucceeded1,\n\t\t\t\tjobPodSucceeded1,\n\t\t\t\tjobPodFailed1,\n\t\t\t},\n\t\t\texpectedStatus: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceFailed,\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tMessage: \"fake-message-2\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"equal to limit\",\n\t\t\tctx: t.Context(),\n\t\t\truntimeScheme: scheme,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tjobSucceeded2,\n\t\t\t\tjobFailed1,\n\t\t\t\tjobSucceeded1,\n\t\t\t\tjobPodSucceeded1,\n\t\t\t\tjobPodFailed1,\n\t\t\t\tjobPodSucceeded2,\n\t\t\t},\n\t\t\texpectedStatus: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceFailed,\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tMessage: \"fake-message-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"more than limit\",\n\t\t\tctx: t.Context(),\n\t\t\truntimeScheme: scheme,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tjobSucceeded3,\n\t\t\t\tjobSucceeded2,\n\t\t\t\tjobFailed1,\n\t\t\t\tjobSucceeded1,\n\t\t\t\tjobPodSucceeded1,\n\t\t\t\tjobPodFailed1,\n\t\t\t\tjobPodSucceeded2,\n\t\t\t\tjobPodSucceeded3,\n\t\t\t},\n\t\t\texpectedStatus: []velerov1api.BackupRepositoryMaintenanceStatus{\n\t\t\t\t{\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceFailed,\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour)},\n\t\t\t\t\tMessage: \"fake-message-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour * 2)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tResult: velerov1api.BackupRepositoryMaintenanceSucceeded,\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: now.Add(time.Hour * 3)},\n\t\t\t\t\tCompleteTimestamp: &metav1.Time{Time: now.Add(time.Hour * 4)},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range testCases {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeClientBuilder := fake.NewClientBuilder()\n\t\t\tfakeClientBuilder = fakeClientBuilder.WithScheme(test.runtimeScheme)\n\n\t\t\tfakeClient := fakeClientBuilder.WithRuntimeObjects(test.kubeClientObj...).Build()\n\n\t\t\thistory, err := WaitAllJobsComplete(test.ctx, fakeClient, repo, 3, velerotest.NewLogger())\n\n\t\t\tif test.expectedError != \"\" {\n\t\t\t\trequire.ErrorContains(t, err, test.expectedError)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tassert.Len(t, history, len(test.expectedStatus))\n\t\t\tfor i := 0; i < len(test.expectedStatus); i++ {\n\t\t\t\tassert.Equal(t, test.expectedStatus[i].Result, history[i].Result)\n\t\t\t\tassert.Equal(t, test.expectedStatus[i].Message, history[i].Message)\n\t\t\t\tassert.Equal(t, test.expectedStatus[i].StartTimestamp.Time, history[i].StartTimestamp.Time)\n\n\t\t\t\tif test.expectedStatus[i].CompleteTimestamp == nil {\n\t\t\t\t\tassert.Nil(t, history[i].CompleteTimestamp)\n\t\t\t\t} else {\n\t\t\t\t\tassert.Equal(t, test.expectedStatus[i].CompleteTimestamp.Time, history[i].CompleteTimestamp.Time)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n\n\tcancel()\n}\n\nfunc TestBuildJob(t *testing.T) {\n\tdeploy := appsv1api.Deployment{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"velero\",\n\t\t\tNamespace: \"velero\",\n\t\t},\n\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tSecurityContext: &corev1api.PodSecurityContext{\n\t\t\t\t\t\tRunAsNonRoot: boolptr.True(),\n\t\t\t\t\t},\n\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"velero-repo-maintenance-container\",\n\t\t\t\t\t\t\tImage: \"velero-image\",\n\t\t\t\t\t\t\tSecurityContext: &corev1api.SecurityContext{\n\t\t\t\t\t\t\t\tRunAsNonRoot: boolptr.True(),\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tEnv: []corev1api.EnvVar{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"test-name\",\n\t\t\t\t\t\t\t\t\tValue: \"test-value\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tEnvFrom: []corev1api.EnvFromSource{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tConfigMapRef: &corev1api.ConfigMapEnvSource{\n\t\t\t\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\t\t\t\tName: \"test-configmap\",\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tSecretRef: &corev1api.SecretEnvSource{\n\t\t\t\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\t\t\t\tName: \"test-secret\",\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tImagePullSecrets: []corev1api.LocalObjectReference{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"imagePullSecret1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tdeploy2 := deploy.DeepCopy()\n\tdeploy2.Spec.Template.Labels = map[string]string{\"azure.workload.identity/use\": \"fake-label-value\"}\n\tdeploy2.Spec.Template.Spec.SecurityContext = nil\n\tdeploy2.Spec.Template.Spec.Containers[0].SecurityContext = nil\n\n\ttestCases := []struct {\n\t\tname string\n\t\tm *velerotypes.JobConfigs\n\t\tdeploy *appsv1api.Deployment\n\t\tlogLevel logrus.Level\n\t\tlogFormat *logging.FormatFlag\n\t\texpectedJobName string\n\t\texpectedError bool\n\t\texpectedEnv []corev1api.EnvVar\n\t\texpectedEnvFrom []corev1api.EnvFromSource\n\t\texpectedPodLabel map[string]string\n\t\texpectedPodAnnotation map[string]string\n\t\texpectedSecurityContext *corev1api.SecurityContext\n\t\texpectedPodSecurityContext *corev1api.PodSecurityContext\n\t\texpectedImagePullSecrets []corev1api.LocalObjectReference\n\t\tbackupRepository *velerov1api.BackupRepository\n\t}{\n\t\t{\n\t\t\tname: \"Valid maintenance job without third party labels\",\n\t\t\tm: &velerotypes.JobConfigs{\n\t\t\t\tPodResources: &kube.PodResources{\n\t\t\t\t\tCPURequest: \"100m\",\n\t\t\t\t\tMemoryRequest: \"128Mi\",\n\t\t\t\t\tCPULimit: \"200m\",\n\t\t\t\t\tMemoryLimit: \"256Mi\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tdeploy: &deploy,\n\t\t\tlogLevel: logrus.InfoLevel,\n\t\t\tlogFormat: logging.NewFormatFlag(),\n\t\t\texpectedJobName: \"test-123-maintain-job\",\n\t\t\texpectedError: false,\n\t\t\texpectedEnv: []corev1api.EnvVar{\n\t\t\t\t{\n\t\t\t\t\tName: \"test-name\",\n\t\t\t\t\tValue: \"test-value\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedEnvFrom: []corev1api.EnvFromSource{\n\t\t\t\t{\n\t\t\t\t\tConfigMapRef: &corev1api.ConfigMapEnvSource{\n\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\tName: \"test-configmap\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tSecretRef: &corev1api.SecretEnvSource{\n\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\tName: \"test-secret\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedPodLabel: map[string]string{\n\t\t\t\tRepositoryNameLabel: \"test-123\",\n\t\t\t},\n\t\t\texpectedSecurityContext: &corev1api.SecurityContext{\n\t\t\t\tRunAsNonRoot: boolptr.True(),\n\t\t\t},\n\t\t\texpectedPodSecurityContext: &corev1api.PodSecurityContext{\n\t\t\t\tRunAsNonRoot: boolptr.True(),\n\t\t\t},\n\t\t\texpectedImagePullSecrets: []corev1api.LocalObjectReference{\n\t\t\t\t{\n\t\t\t\t\tName: \"imagePullSecret1\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Valid maintenance job with third party labels\",\n\t\t\tm: &velerotypes.JobConfigs{\n\t\t\t\tPodResources: &kube.PodResources{\n\t\t\t\t\tCPURequest: \"100m\",\n\t\t\t\t\tMemoryRequest: \"128Mi\",\n\t\t\t\t\tCPULimit: \"200m\",\n\t\t\t\t\tMemoryLimit: \"256Mi\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tdeploy: deploy2,\n\t\t\tlogLevel: logrus.InfoLevel,\n\t\t\tlogFormat: logging.NewFormatFlag(),\n\t\t\texpectedJobName: \"test-123-maintain-job\",\n\t\t\texpectedError: false,\n\t\t\texpectedEnv: []corev1api.EnvVar{\n\t\t\t\t{\n\t\t\t\t\tName: \"test-name\",\n\t\t\t\t\tValue: \"test-value\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedEnvFrom: []corev1api.EnvFromSource{\n\t\t\t\t{\n\t\t\t\t\tConfigMapRef: &corev1api.ConfigMapEnvSource{\n\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\tName: \"test-configmap\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tSecretRef: &corev1api.SecretEnvSource{\n\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\tName: \"test-secret\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedPodLabel: map[string]string{\n\t\t\t\tRepositoryNameLabel: \"test-123\",\n\t\t\t\t\"azure.workload.identity/use\": \"fake-label-value\",\n\t\t\t},\n\t\t\texpectedSecurityContext: nil,\n\t\t\texpectedPodSecurityContext: nil,\n\t\t\texpectedImagePullSecrets: []corev1api.LocalObjectReference{\n\t\t\t\t{\n\t\t\t\t\tName: \"imagePullSecret1\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Error getting Velero server deployment\",\n\t\t\tm: &velerotypes.JobConfigs{\n\t\t\t\tPodResources: &kube.PodResources{\n\t\t\t\t\tCPURequest: \"100m\",\n\t\t\t\t\tMemoryRequest: \"128Mi\",\n\t\t\t\t\tCPULimit: \"200m\",\n\t\t\t\t\tMemoryLimit: \"256Mi\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tlogLevel: logrus.InfoLevel,\n\t\t\tlogFormat: logging.NewFormatFlag(),\n\t\t\texpectedJobName: \"\",\n\t\t\texpectedError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Valid maintenance job customized labels and annotations\",\n\t\t\tm: &velerotypes.JobConfigs{\n\t\t\t\tPodResources: &kube.PodResources{\n\t\t\t\t\tCPURequest: \"100m\",\n\t\t\t\t\tMemoryRequest: \"128Mi\",\n\t\t\t\t\tCPULimit: \"200m\",\n\t\t\t\t\tMemoryLimit: \"256Mi\",\n\t\t\t\t},\n\t\t\t\tPodLabels: map[string]string{\n\t\t\t\t\t\"global-label-1\": \"global-label-value-1\",\n\t\t\t\t\t\"global-label-2\": \"global-label-value-2\",\n\t\t\t\t},\n\t\t\t\tPodAnnotations: map[string]string{\n\t\t\t\t\t\"global-annotation-1\": \"global-annotation-value-1\",\n\t\t\t\t\t\"global-annotation-2\": \"global-annotation-value-2\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tdeploy: deploy2,\n\t\t\tlogLevel: logrus.InfoLevel,\n\t\t\tlogFormat: logging.NewFormatFlag(),\n\t\t\texpectedError: false,\n\t\t\texpectedJobName: \"test-123-maintain-job\",\n\t\t\texpectedEnv: []corev1api.EnvVar{\n\t\t\t\t{\n\t\t\t\t\tName: \"test-name\",\n\t\t\t\t\tValue: \"test-value\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedEnvFrom: []corev1api.EnvFromSource{\n\t\t\t\t{\n\t\t\t\t\tConfigMapRef: &corev1api.ConfigMapEnvSource{\n\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\tName: \"test-configmap\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tSecretRef: &corev1api.SecretEnvSource{\n\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\tName: \"test-secret\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedPodLabel: map[string]string{\n\t\t\t\t\"global-label-1\": \"global-label-value-1\",\n\t\t\t\t\"global-label-2\": \"global-label-value-2\",\n\t\t\t\tRepositoryNameLabel: \"test-123\",\n\t\t\t},\n\t\t\texpectedPodAnnotation: map[string]string{\n\t\t\t\t\"global-annotation-1\": \"global-annotation-value-1\",\n\t\t\t\t\"global-annotation-2\": \"global-annotation-value-2\",\n\t\t\t},\n\t\t\texpectedSecurityContext: nil,\n\t\t\texpectedPodSecurityContext: nil,\n\t\t\texpectedImagePullSecrets: []corev1api.LocalObjectReference{\n\t\t\t\t{\n\t\t\t\t\tName: \"imagePullSecret1\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Valid maintenance job with third party labels and BackupRepository name longer than 63\",\n\t\t\tm: &velerotypes.JobConfigs{\n\t\t\t\tPodResources: &kube.PodResources{\n\t\t\t\t\tCPURequest: \"100m\",\n\t\t\t\t\tMemoryRequest: \"128Mi\",\n\t\t\t\t\tCPULimit: \"200m\",\n\t\t\t\t\tMemoryLimit: \"256Mi\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tdeploy: deploy2,\n\t\t\tlogLevel: logrus.InfoLevel,\n\t\t\tlogFormat: logging.NewFormatFlag(),\n\t\t\texpectedError: false,\n\t\t\texpectedEnv: []corev1api.EnvVar{\n\t\t\t\t{\n\t\t\t\t\tName: \"test-name\",\n\t\t\t\t\tValue: \"test-value\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedEnvFrom: []corev1api.EnvFromSource{\n\t\t\t\t{\n\t\t\t\t\tConfigMapRef: &corev1api.ConfigMapEnvSource{\n\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\tName: \"test-configmap\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tSecretRef: &corev1api.SecretEnvSource{\n\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\tName: \"test-secret\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedPodLabel: map[string]string{\n\t\t\t\tRepositoryNameLabel: velerolabel.ReturnNameOrHash(\"label with more than 63 characters should be modified\"),\n\t\t\t\t\"azure.workload.identity/use\": \"fake-label-value\",\n\t\t\t},\n\t\t\texpectedSecurityContext: nil,\n\t\t\texpectedPodSecurityContext: nil,\n\t\t\texpectedImagePullSecrets: []corev1api.LocalObjectReference{\n\t\t\t\t{\n\t\t\t\t\tName: \"imagePullSecret1\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tbackupRepository: &velerov1api.BackupRepository{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\tName: \"label with more than 63 characters should be modified\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\t\t\tVolumeNamespace: \"test-123\",\n\t\t\t\t\tRepositoryType: \"kopia\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tparam := provider.RepoParam{\n\t\tBackupRepo: &velerov1api.BackupRepository{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: \"velero\",\n\t\t\t\tName: \"test-123\",\n\t\t\t},\n\t\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\t\tVolumeNamespace: \"test-123\",\n\t\t\t\tRepositoryType: \"kopia\",\n\t\t\t},\n\t\t},\n\t\tBackupLocation: &velerov1api.BackupStorageLocation{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tNamespace: \"velero\",\n\t\t\t\tName: \"test-location\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tif tc.backupRepository != nil {\n\t\t\t\tparam.BackupRepo = tc.backupRepository\n\t\t\t}\n\n\t\t\t// Create a fake clientset with resources\n\t\t\tobjs := []runtime.Object{param.BackupLocation, param.BackupRepo}\n\n\t\t\tif tc.deploy != nil {\n\t\t\t\tobjs = append(objs, tc.deploy)\n\t\t\t}\n\t\t\tscheme := runtime.NewScheme()\n\t\t\t_ = appsv1api.AddToScheme(scheme)\n\t\t\t_ = velerov1api.AddToScheme(scheme)\n\t\t\tcli := fake.NewClientBuilder().WithScheme(scheme).WithRuntimeObjects(objs...).Build()\n\n\t\t\t// Call the function to test\n\t\t\tjob, err := buildJob(\n\t\t\t\tcli,\n\t\t\t\tt.Context(),\n\t\t\t\tparam.BackupRepo,\n\t\t\t\tparam.BackupLocation.Name,\n\t\t\t\ttc.m,\n\t\t\t\ttc.logLevel,\n\t\t\t\ttc.logFormat,\n\t\t\t\tlogrus.New(),\n\t\t\t)\n\n\t\t\t// Check the error\n\t\t\tif tc.expectedError {\n\t\t\t\trequire.Error(t, err)\n\t\t\t\tassert.Nil(t, job)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.NotNil(t, job)\n\t\t\t\tassert.Contains(t, job.Name, tc.expectedJobName)\n\t\t\t\tassert.Equal(t, param.BackupRepo.Namespace, job.Namespace)\n\t\t\t\tassert.Equal(t, param.BackupRepo.Name, job.Labels[RepositoryNameLabel])\n\n\t\t\t\tassert.Equal(t, param.BackupRepo.Name, job.Spec.Template.ObjectMeta.Labels[RepositoryNameLabel])\n\n\t\t\t\t// Check container\n\t\t\t\tassert.Len(t, job.Spec.Template.Spec.Containers, 1)\n\t\t\t\tcontainer := job.Spec.Template.Spec.Containers[0]\n\t\t\t\tassert.Equal(t, \"velero-repo-maintenance-container\", container.Name)\n\t\t\t\tassert.Equal(t, \"velero-image\", container.Image)\n\t\t\t\tassert.Equal(t, corev1api.PullIfNotPresent, container.ImagePullPolicy)\n\n\t\t\t\t// Check container env\n\t\t\t\tassert.Equal(t, tc.expectedEnv, container.Env)\n\t\t\t\tassert.Equal(t, tc.expectedEnvFrom, container.EnvFrom)\n\n\t\t\t\t// Check security context\n\t\t\t\tassert.Equal(t, tc.expectedPodSecurityContext, job.Spec.Template.Spec.SecurityContext)\n\t\t\t\tassert.Equal(t, tc.expectedSecurityContext, container.SecurityContext)\n\n\t\t\t\t// Check resources\n\t\t\t\texpectedResources := corev1api.ResourceRequirements{\n\t\t\t\t\tRequests: corev1api.ResourceList{\n\t\t\t\t\t\tcorev1api.ResourceCPU: resource.MustParse(tc.m.PodResources.CPURequest),\n\t\t\t\t\t\tcorev1api.ResourceMemory: resource.MustParse(tc.m.PodResources.MemoryRequest),\n\t\t\t\t\t},\n\t\t\t\t\tLimits: corev1api.ResourceList{\n\t\t\t\t\t\tcorev1api.ResourceCPU: resource.MustParse(tc.m.PodResources.CPULimit),\n\t\t\t\t\t\tcorev1api.ResourceMemory: resource.MustParse(tc.m.PodResources.MemoryLimit),\n\t\t\t\t\t},\n\t\t\t\t}\n\t\t\t\tassert.Equal(t, expectedResources, container.Resources)\n\n\t\t\t\t// Check args\n\t\t\t\texpectedArgs := []string{\n\t\t\t\t\t\"repo-maintenance\",\n\t\t\t\t\tfmt.Sprintf(\"--repo-name=%s\", param.BackupRepo.Spec.VolumeNamespace),\n\t\t\t\t\tfmt.Sprintf(\"--repo-type=%s\", param.BackupRepo.Spec.RepositoryType),\n\t\t\t\t\tfmt.Sprintf(\"--backup-storage-location=%s\", param.BackupLocation.Name),\n\t\t\t\t\tfmt.Sprintf(\"--log-level=%s\", tc.logLevel.String()),\n\t\t\t\t\tfmt.Sprintf(\"--log-format=%s\", tc.logFormat.String()),\n\t\t\t\t}\n\t\t\t\tassert.Equal(t, expectedArgs, container.Args)\n\n\t\t\t\tassert.Equal(t, tc.expectedPodLabel, job.Spec.Template.Labels)\n\n\t\t\t\tassert.Equal(t, tc.expectedImagePullSecrets, job.Spec.Template.Spec.ImagePullSecrets)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetKeepLatestMaintenanceJobs(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\trepoMaintenanceJobConfig string\n\t\tconfigMap *corev1api.ConfigMap\n\t\trepo *velerov1api.BackupRepository\n\t\texpectedValue int\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"no config map name provided\",\n\t\t\trepoMaintenanceJobConfig: \"\",\n\t\t\tconfigMap: nil,\n\t\t\trepo: mockBackupRepo(),\n\t\t\texpectedValue: 3,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"config map not found\",\n\t\t\trepoMaintenanceJobConfig: \"non-existent-config\",\n\t\t\tconfigMap: nil,\n\t\t\trepo: mockBackupRepo(),\n\t\t\texpectedValue: 3,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"config map with global keepLatestMaintenanceJobs\",\n\t\t\trepoMaintenanceJobConfig: \"repo-job-config\",\n\t\t\tconfigMap: &corev1api.ConfigMap{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\tName: \"repo-job-config\",\n\t\t\t\t},\n\t\t\t\tData: map[string]string{\n\t\t\t\t\t\"global\": `{\"keepLatestMaintenanceJobs\": 5}`,\n\t\t\t\t},\n\t\t\t},\n\t\t\trepo: mockBackupRepo(),\n\t\t\texpectedValue: 5,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"config map with specific repo keepLatestMaintenanceJobs overriding global\",\n\t\t\trepoMaintenanceJobConfig: \"repo-job-config\",\n\t\t\tconfigMap: &corev1api.ConfigMap{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\tName: \"repo-job-config\",\n\t\t\t\t},\n\t\t\t\tData: map[string]string{\n\t\t\t\t\t\"global\": `{\"keepLatestMaintenanceJobs\": 5}`,\n\t\t\t\t\t\"test-ns-default-kopia\": `{\"keepLatestMaintenanceJobs\": 10}`,\n\t\t\t\t},\n\t\t\t},\n\t\t\trepo: mockBackupRepo(),\n\t\t\texpectedValue: 10,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"config map with no keepLatestMaintenanceJobs specified\",\n\t\t\trepoMaintenanceJobConfig: \"repo-job-config\",\n\t\t\tconfigMap: &corev1api.ConfigMap{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\tName: \"repo-job-config\",\n\t\t\t\t},\n\t\t\t\tData: map[string]string{\n\t\t\t\t\t\"global\": `{\"podResources\": {\"cpuRequest\": \"100m\"}}`,\n\t\t\t\t},\n\t\t\t},\n\t\t\trepo: mockBackupRepo(),\n\t\t\texpectedValue: 3,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"config map with invalid JSON\",\n\t\t\trepoMaintenanceJobConfig: \"repo-job-config\",\n\t\t\tconfigMap: &corev1api.ConfigMap{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\tName: \"repo-job-config\",\n\t\t\t\t},\n\t\t\t\tData: map[string]string{\n\t\t\t\t\t\"global\": `{\"keepLatestMaintenanceJobs\": invalid}`,\n\t\t\t\t},\n\t\t\t},\n\t\t\trepo: mockBackupRepo(),\n\t\t\texpectedValue: 3,\n\t\t\texpectError: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tscheme := runtime.NewScheme()\n\t\t\tcorev1api.AddToScheme(scheme)\n\n\t\t\tvar objects []runtime.Object\n\t\t\tif test.configMap != nil {\n\t\t\t\tobjects = append(objects, test.configMap)\n\t\t\t}\n\n\t\t\tclient := fake.NewClientBuilder().WithScheme(scheme).WithRuntimeObjects(objects...).Build()\n\t\t\tlogger := velerotest.NewLogger()\n\n\t\t\tresult, err := GetKeepLatestMaintenanceJobs(\n\t\t\t\tt.Context(),\n\t\t\t\tclient,\n\t\t\t\tlogger,\n\t\t\t\t\"velero\",\n\t\t\t\ttest.repoMaintenanceJobConfig,\n\t\t\t\ttest.repo,\n\t\t\t)\n\n\t\t\tif test.expectError {\n\t\t\t\trequire.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, test.expectedValue, result)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc mockBackupRepo() *velerov1api.BackupRepository {\n\treturn &velerov1api.BackupRepository{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"velero\",\n\t\t\tName: \"test-repo\",\n\t\t},\n\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\tVolumeNamespace: \"test-ns\",\n\t\t\tBackupStorageLocation: \"default\",\n\t\t\tRepositoryType: \"kopia\",\n\t\t},\n\t}\n}\n\nfunc TestGetPriorityClassName(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tconfig *velerotypes.JobConfigs\n\t\tpriorityClassExists bool\n\t\texpectedValue string\n\t\texpectedLogContains string\n\t\texpectedLogLevel string\n\t}{\n\t\t{\n\t\t\tname: \"empty priority class name should return empty string\",\n\t\t\tconfig: &velerotypes.JobConfigs{PriorityClassName: \"\"},\n\t\t\texpectedValue: \"\",\n\t\t\texpectedLogContains: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"nil config should return empty string\",\n\t\t\tconfig: nil,\n\t\t\texpectedValue: \"\",\n\t\t\texpectedLogContains: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"existing priority class should log info and return name\",\n\t\t\tconfig: &velerotypes.JobConfigs{PriorityClassName: \"high-priority\"},\n\t\t\tpriorityClassExists: true,\n\t\t\texpectedValue: \"high-priority\",\n\t\t\texpectedLogContains: \"Validated priority class \\\\\\\"high-priority\\\\\\\" exists in cluster\",\n\t\t\texpectedLogLevel: \"info\",\n\t\t},\n\t\t{\n\t\t\tname: \"non-existing priority class should log warning and still return name\",\n\t\t\tconfig: &velerotypes.JobConfigs{PriorityClassName: \"missing-priority\"},\n\t\t\tpriorityClassExists: false,\n\t\t\texpectedValue: \"missing-priority\",\n\t\t\texpectedLogContains: \"Priority class \\\\\\\"missing-priority\\\\\\\" not found in cluster\",\n\t\t\texpectedLogLevel: \"warning\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t// Create a new scheme and add necessary API types\n\t\t\tlocalScheme := runtime.NewScheme()\n\t\t\terr := schedulingv1.AddToScheme(localScheme)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Create fake client builder\n\t\t\tclientBuilder := fake.NewClientBuilder().WithScheme(localScheme)\n\n\t\t\t// Add priority class if it should exist\n\t\t\tif tc.priorityClassExists {\n\t\t\t\tpriorityClass := &schedulingv1.PriorityClass{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: tc.config.PriorityClassName,\n\t\t\t\t\t},\n\t\t\t\t\tValue: 1000,\n\t\t\t\t}\n\t\t\t\tclientBuilder = clientBuilder.WithObjects(priorityClass)\n\t\t\t}\n\n\t\t\tclient := clientBuilder.Build()\n\n\t\t\t// Capture logs\n\t\t\tvar logBuffer strings.Builder\n\t\t\tlogger := logrus.New()\n\t\t\tlogger.SetOutput(&logBuffer)\n\t\t\tlogger.SetLevel(logrus.InfoLevel)\n\n\t\t\t// Call the function\n\t\t\tresult := getPriorityClassName(t.Context(), client, tc.config, logger)\n\n\t\t\t// Verify the result\n\t\t\tassert.Equal(t, tc.expectedValue, result)\n\n\t\t\t// Verify log output\n\t\t\tlogOutput := logBuffer.String()\n\t\t\tif tc.expectedLogContains != \"\" {\n\t\t\t\tassert.Contains(t, logOutput, tc.expectedLogContains)\n\t\t\t}\n\n\t\t\t// Verify log level\n\t\t\tif tc.expectedLogLevel == \"warning\" {\n\t\t\t\tassert.Contains(t, logOutput, \"level=warning\")\n\t\t\t} else if tc.expectedLogLevel == \"info\" {\n\t\t\t\tassert.Contains(t, logOutput, \"level=info\")\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestBuildJobWithPriorityClassName(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tpriorityClassName string\n\t\texpectedValue string\n\t}{\n\t\t{\n\t\t\tname: \"with priority class name\",\n\t\t\tpriorityClassName: \"high-priority\",\n\t\t\texpectedValue: \"high-priority\",\n\t\t},\n\t\t{\n\t\t\tname: \"without priority class name\",\n\t\t\tpriorityClassName: \"\",\n\t\t\texpectedValue: \"\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t// Create a new scheme and add necessary API types\n\t\t\tlocalScheme := runtime.NewScheme()\n\t\t\terr := velerov1api.AddToScheme(localScheme)\n\t\t\trequire.NoError(t, err)\n\t\t\terr = appsv1api.AddToScheme(localScheme)\n\t\t\trequire.NoError(t, err)\n\t\t\terr = batchv1api.AddToScheme(localScheme)\n\t\t\trequire.NoError(t, err)\n\t\t\terr = schedulingv1.AddToScheme(localScheme)\n\t\t\trequire.NoError(t, err)\n\t\t\t// Create a fake client\n\t\t\tclient := fake.NewClientBuilder().WithScheme(localScheme).Build()\n\n\t\t\t// Create a deployment with the specified priority class name\n\t\t\tdeployment := &appsv1api.Deployment{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"velero\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t\t\t\tImage: \"velero/velero:latest\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tPriorityClassName: tc.priorityClassName,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t}\n\n\t\t\t// Create a backup repository\n\t\t\trepo := &velerov1api.BackupRepository{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-repo\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\t\t\tVolumeNamespace: \"velero\",\n\t\t\t\t\tBackupStorageLocation: \"default\",\n\t\t\t\t},\n\t\t\t}\n\n\t\t\t// Create the deployment in the fake client\n\t\t\terr = client.Create(t.Context(), deployment)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Create minimal job configs and resources\n\t\t\tjobConfig := &velerotypes.JobConfigs{\n\t\t\t\tPriorityClassName: tc.priorityClassName,\n\t\t\t}\n\t\t\tlogLevel := logrus.InfoLevel\n\t\t\tlogFormat := logging.NewFormatFlag()\n\t\t\tlogFormat.Set(\"text\")\n\n\t\t\t// Call buildJob\n\t\t\tjob, err := buildJob(client, t.Context(), repo, \"default\", jobConfig, logLevel, logFormat, logrus.New())\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Verify the priority class name is set correctly\n\t\t\tassert.Equal(t, tc.expectedValue, job.Spec.Template.Spec.PriorityClassName)\n\t\t})\n\t}\n}\n\nfunc TestBuildTolerationsForMaintenanceJob(t *testing.T) {\n\twindowsToleration := corev1api.Toleration{\n\t\tKey: \"os\",\n\t\tOperator: \"Equal\",\n\t\tEffect: \"NoSchedule\",\n\t\tValue: \"windows\",\n\t}\n\n\ttestCases := []struct {\n\t\tname string\n\t\tdeploymentTolerations []corev1api.Toleration\n\t\texpectedTolerations []corev1api.Toleration\n\t}{\n\t\t{\n\t\t\tname: \"no tolerations should only include Windows toleration\",\n\t\t\tdeploymentTolerations: nil,\n\t\t\texpectedTolerations: []corev1api.Toleration{\n\t\t\t\twindowsToleration,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"empty tolerations should only include Windows toleration\",\n\t\t\tdeploymentTolerations: []corev1api.Toleration{},\n\t\t\texpectedTolerations: []corev1api.Toleration{\n\t\t\t\twindowsToleration,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"non-allowed toleration should not be inherited\",\n\t\t\tdeploymentTolerations: []corev1api.Toleration{\n\t\t\t\t{\n\t\t\t\t\tKey: \"vng-ondemand\",\n\t\t\t\t\tOperator: \"Equal\",\n\t\t\t\t\tEffect: \"NoSchedule\",\n\t\t\t\t\tValue: \"amd64\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedTolerations: []corev1api.Toleration{\n\t\t\t\twindowsToleration,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"allowed toleration should be inherited\",\n\t\t\tdeploymentTolerations: []corev1api.Toleration{\n\t\t\t\t{\n\t\t\t\t\tKey: \"kubernetes.azure.com/scalesetpriority\",\n\t\t\t\t\tOperator: \"Equal\",\n\t\t\t\t\tEffect: \"NoSchedule\",\n\t\t\t\t\tValue: \"spot\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedTolerations: []corev1api.Toleration{\n\t\t\t\twindowsToleration,\n\t\t\t\t{\n\t\t\t\t\tKey: \"kubernetes.azure.com/scalesetpriority\",\n\t\t\t\t\tOperator: \"Equal\",\n\t\t\t\t\tEffect: \"NoSchedule\",\n\t\t\t\t\tValue: \"spot\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"mixed allowed and non-allowed tolerations should only inherit allowed\",\n\t\t\tdeploymentTolerations: []corev1api.Toleration{\n\t\t\t\t{\n\t\t\t\t\tKey: \"vng-ondemand\", // not in allowlist\n\t\t\t\t\tOperator: \"Equal\",\n\t\t\t\t\tEffect: \"NoSchedule\",\n\t\t\t\t\tValue: \"amd64\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tKey: \"CriticalAddonsOnly\", // in allowlist\n\t\t\t\t\tOperator: \"Exists\",\n\t\t\t\t\tEffect: \"NoSchedule\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tKey: \"custom-key\", // not in allowlist\n\t\t\t\t\tOperator: \"Equal\",\n\t\t\t\t\tEffect: \"NoSchedule\",\n\t\t\t\t\tValue: \"custom-value\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedTolerations: []corev1api.Toleration{\n\t\t\t\twindowsToleration,\n\t\t\t\t{\n\t\t\t\t\tKey: \"CriticalAddonsOnly\",\n\t\t\t\t\tOperator: \"Exists\",\n\t\t\t\t\tEffect: \"NoSchedule\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"multiple allowed tolerations should all be inherited\",\n\t\t\tdeploymentTolerations: []corev1api.Toleration{\n\t\t\t\t{\n\t\t\t\t\tKey: \"kubernetes.azure.com/scalesetpriority\",\n\t\t\t\t\tOperator: \"Equal\",\n\t\t\t\t\tEffect: \"NoSchedule\",\n\t\t\t\t\tValue: \"spot\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tKey: \"CriticalAddonsOnly\",\n\t\t\t\t\tOperator: \"Exists\",\n\t\t\t\t\tEffect: \"NoSchedule\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedTolerations: []corev1api.Toleration{\n\t\t\t\twindowsToleration,\n\t\t\t\t{\n\t\t\t\t\tKey: \"kubernetes.azure.com/scalesetpriority\",\n\t\t\t\t\tOperator: \"Equal\",\n\t\t\t\t\tEffect: \"NoSchedule\",\n\t\t\t\t\tValue: \"spot\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tKey: \"CriticalAddonsOnly\",\n\t\t\t\t\tOperator: \"Exists\",\n\t\t\t\t\tEffect: \"NoSchedule\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t// Create a deployment with the specified tolerations\n\t\t\tdeployment := &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tTolerations: tc.deploymentTolerations,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tresult := buildTolerationsForMaintenanceJob(deployment)\n\t\t\tassert.Equal(t, tc.expectedTolerations, result)\n\t\t})\n\t}\n}\n\nfunc TestBuildJobWithTolerationsInheritance(t *testing.T) {\n\t// Define allowed tolerations that would be set on Velero deployment\n\tallowedTolerations := []corev1api.Toleration{\n\t\t{\n\t\t\tKey: \"kubernetes.azure.com/scalesetpriority\",\n\t\t\tOperator: \"Equal\",\n\t\t\tEffect: \"NoSchedule\",\n\t\t\tValue: \"spot\",\n\t\t},\n\t\t{\n\t\t\tKey: \"CriticalAddonsOnly\",\n\t\t\tOperator: \"Exists\",\n\t\t\tEffect: \"NoSchedule\",\n\t\t},\n\t}\n\n\t// Mixed tolerations (allowed and non-allowed)\n\tmixedTolerations := []corev1api.Toleration{\n\t\t{\n\t\t\tKey: \"vng-ondemand\", // not in allowlist\n\t\t\tOperator: \"Equal\",\n\t\t\tEffect: \"NoSchedule\",\n\t\t\tValue: \"amd64\",\n\t\t},\n\t\t{\n\t\t\tKey: \"CriticalAddonsOnly\", // in allowlist\n\t\t\tOperator: \"Exists\",\n\t\t\tEffect: \"NoSchedule\",\n\t\t},\n\t}\n\n\t// Windows toleration that should always be present\n\twindowsToleration := corev1api.Toleration{\n\t\tKey: \"os\",\n\t\tOperator: \"Equal\",\n\t\tEffect: \"NoSchedule\",\n\t\tValue: \"windows\",\n\t}\n\n\ttestCases := []struct {\n\t\tname string\n\t\tdeploymentTolerations []corev1api.Toleration\n\t\texpectedTolerations []corev1api.Toleration\n\t}{\n\t\t{\n\t\t\tname: \"no tolerations on deployment should only have Windows toleration\",\n\t\t\tdeploymentTolerations: nil,\n\t\t\texpectedTolerations: []corev1api.Toleration{\n\t\t\t\twindowsToleration,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"allowed tolerations should be inherited along with Windows toleration\",\n\t\t\tdeploymentTolerations: allowedTolerations,\n\t\t\texpectedTolerations: []corev1api.Toleration{\n\t\t\t\twindowsToleration,\n\t\t\t\t{\n\t\t\t\t\tKey: \"kubernetes.azure.com/scalesetpriority\",\n\t\t\t\t\tOperator: \"Equal\",\n\t\t\t\t\tEffect: \"NoSchedule\",\n\t\t\t\t\tValue: \"spot\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tKey: \"CriticalAddonsOnly\",\n\t\t\t\t\tOperator: \"Exists\",\n\t\t\t\t\tEffect: \"NoSchedule\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"mixed tolerations should only inherit allowed ones\",\n\t\t\tdeploymentTolerations: mixedTolerations,\n\t\t\texpectedTolerations: []corev1api.Toleration{\n\t\t\t\twindowsToleration,\n\t\t\t\t{\n\t\t\t\t\tKey: \"CriticalAddonsOnly\",\n\t\t\t\t\tOperator: \"Exists\",\n\t\t\t\t\tEffect: \"NoSchedule\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t// Create a new scheme and add necessary API types\n\t\t\tlocalScheme := runtime.NewScheme()\n\t\t\terr := velerov1api.AddToScheme(localScheme)\n\t\t\trequire.NoError(t, err)\n\t\t\terr = appsv1api.AddToScheme(localScheme)\n\t\t\trequire.NoError(t, err)\n\t\t\terr = batchv1api.AddToScheme(localScheme)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Create a deployment with the specified tolerations\n\t\t\tdeployment := &appsv1api.Deployment{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"velero\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"velero\",\n\t\t\t\t\t\t\t\t\tImage: \"velero/velero:latest\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\tTolerations: tc.deploymentTolerations,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t}\n\n\t\t\t// Create a backup repository\n\t\t\trepo := &velerov1api.BackupRepository{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-repo\",\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t\tSpec: velerov1api.BackupRepositorySpec{\n\t\t\t\t\tVolumeNamespace: \"velero\",\n\t\t\t\t\tBackupStorageLocation: \"default\",\n\t\t\t\t},\n\t\t\t}\n\n\t\t\t// Create fake client and add the deployment\n\t\t\tclient := fake.NewClientBuilder().WithScheme(localScheme).WithObjects(deployment).Build()\n\n\t\t\t// Create minimal job configs and resources\n\t\t\tjobConfig := &velerotypes.JobConfigs{}\n\t\t\tlogLevel := logrus.InfoLevel\n\t\t\tlogFormat := logging.NewFormatFlag()\n\t\t\tlogFormat.Set(\"text\")\n\n\t\t\t// Call buildJob\n\t\t\tjob, err := buildJob(client, t.Context(), repo, \"default\", jobConfig, logLevel, logFormat, logrus.New())\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Verify the tolerations are set correctly\n\t\t\tassert.Equal(t, tc.expectedTolerations, job.Spec.Template.Spec.Tolerations)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/repository/manager/manager.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage repository\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/provider\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n)\n\n// Manager manages backup repositories.\ntype Manager interface {\n\t// InitRepo initializes a repo with the specified name and identifier.\n\tInitRepo(repo *velerov1api.BackupRepository) error\n\n\t// ConnectToRepo tries to connect to the specified repo, and returns an error if it fails.\n\t// This is intended to be used to ensure that the repo exists/can be authenticated to.\n\tConnectToRepo(repo *velerov1api.BackupRepository) error\n\n\t// PrepareRepo tries to connect to the specific repo first, if it fails because of the\n\t// repo is not initialized, it turns to initialize the repo\n\tPrepareRepo(repo *velerov1api.BackupRepository) error\n\n\t// PruneRepo deletes unused data from a repo.\n\tPruneRepo(repo *velerov1api.BackupRepository) error\n\n\t// UnlockRepo removes stale locks from a repo.\n\tUnlockRepo(repo *velerov1api.BackupRepository) error\n\n\t// Forget removes a snapshot from the list of\n\t// available snapshots in a repo.\n\tForget(context.Context, *velerov1api.BackupRepository, string) error\n\n\t// BatchForget removes a list of snapshots from the list of\n\t// available snapshots in a repo.\n\tBatchForget(context.Context, *velerov1api.BackupRepository, []string) []error\n\n\t// DefaultMaintenanceFrequency returns the default maintenance frequency from the specific repo\n\tDefaultMaintenanceFrequency(*velerov1api.BackupRepository) (time.Duration, error)\n\n\t// ClientSideCacheLimit returns the max cache size required on client side\n\tClientSideCacheLimit(*velerov1api.BackupRepository) (int64, error)\n}\n\n// ConfigProvider defines the methods to get configurations of a backup repository\ntype ConfigManager interface {\n\t// DefaultMaintenanceFrequency returns the default maintenance frequency from the specific repo\n\tDefaultMaintenanceFrequency(string) (time.Duration, error)\n\n\t// ClientSideCacheLimit returns the max cache size required on client side\n\tClientSideCacheLimit(string, map[string]string) (int64, error)\n}\n\ntype manager struct {\n\tnamespace string\n\tproviders map[string]provider.Provider\n\t// client is the Velero controller manager's client.\n\t// It's limited to resources in the Velero namespace.\n\tclient client.Client\n\trepoLocker *repository.RepoLocker\n\tfileSystem filesystem.Interface\n\tlog logrus.FieldLogger\n}\n\ntype configManager struct {\n\tproviders map[string]provider.ConfigProvider\n\tlog logrus.FieldLogger\n}\n\n// NewManager create a new repository manager.\nfunc NewManager(\n\tnamespace string,\n\tclient client.Client,\n\trepoLocker *repository.RepoLocker,\n\tcredentialFileStore credentials.FileStore,\n\tcredentialSecretStore credentials.SecretStore,\n\tlog logrus.FieldLogger,\n) Manager {\n\tmgr := &manager{\n\t\tnamespace: namespace,\n\t\tclient: client,\n\t\tproviders: map[string]provider.Provider{},\n\t\trepoLocker: repoLocker,\n\t\tfileSystem: filesystem.NewFileSystem(),\n\t\tlog: log,\n\t}\n\n\tmgr.providers[velerov1api.BackupRepositoryTypeRestic] = provider.NewResticRepositoryProvider(credentials.CredentialGetter{\n\t\tFromFile: credentialFileStore,\n\t\tFromSecret: credentialSecretStore,\n\t}, mgr.fileSystem, mgr.log)\n\tmgr.providers[velerov1api.BackupRepositoryTypeKopia] = provider.NewUnifiedRepoProvider(credentials.CredentialGetter{\n\t\tFromFile: credentialFileStore,\n\t\tFromSecret: credentialSecretStore,\n\t}, velerov1api.BackupRepositoryTypeKopia, mgr.log)\n\n\treturn mgr\n}\n\n// NewConfigManager create a new repository config manager.\nfunc NewConfigManager(\n\tlog logrus.FieldLogger,\n) ConfigManager {\n\tmgr := &configManager{\n\t\tproviders: map[string]provider.ConfigProvider{},\n\t\tlog: log,\n\t}\n\n\tmgr.providers[velerov1api.BackupRepositoryTypeKopia] = provider.NewUnifiedRepoConfigProvider(velerov1api.BackupRepositoryTypeKopia, mgr.log)\n\n\treturn mgr\n}\n\nfunc (m *manager) InitRepo(repo *velerov1api.BackupRepository) error {\n\tm.repoLocker.LockExclusive(repo.Name)\n\tdefer m.repoLocker.UnlockExclusive(repo.Name)\n\n\tprd, err := m.getRepositoryProvider(repo)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\tparam, err := m.assembleRepoParam(repo)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\treturn prd.InitRepo(context.Background(), param)\n}\n\nfunc (m *manager) ConnectToRepo(repo *velerov1api.BackupRepository) error {\n\tm.repoLocker.Lock(repo.Name)\n\tdefer m.repoLocker.Unlock(repo.Name)\n\n\tprd, err := m.getRepositoryProvider(repo)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\tparam, err := m.assembleRepoParam(repo)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\treturn prd.ConnectToRepo(context.Background(), param)\n}\n\nfunc (m *manager) PrepareRepo(repo *velerov1api.BackupRepository) error {\n\tm.repoLocker.Lock(repo.Name)\n\tdefer m.repoLocker.Unlock(repo.Name)\n\n\tprd, err := m.getRepositoryProvider(repo)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\tparam, err := m.assembleRepoParam(repo)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\treturn prd.PrepareRepo(context.Background(), param)\n}\n\nfunc (m *manager) PruneRepo(repo *velerov1api.BackupRepository) error {\n\tm.repoLocker.LockExclusive(repo.Name)\n\tdefer m.repoLocker.UnlockExclusive(repo.Name)\n\n\tprd, err := m.getRepositoryProvider(repo)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\tparam, err := m.assembleRepoParam(repo)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\tif err := prd.BoostRepoConnect(context.Background(), param); err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\treturn prd.PruneRepo(context.Background(), param)\n}\n\nfunc (m *manager) UnlockRepo(repo *velerov1api.BackupRepository) error {\n\tm.repoLocker.Lock(repo.Name)\n\tdefer m.repoLocker.Unlock(repo.Name)\n\n\tprd, err := m.getRepositoryProvider(repo)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\tparam, err := m.assembleRepoParam(repo)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\treturn prd.EnsureUnlockRepo(context.Background(), param)\n}\n\nfunc (m *manager) Forget(ctx context.Context, repo *velerov1api.BackupRepository, snapshot string) error {\n\tm.repoLocker.LockExclusive(repo.Name)\n\tdefer m.repoLocker.UnlockExclusive(repo.Name)\n\n\tprd, err := m.getRepositoryProvider(repo)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\tparam, err := m.assembleRepoParam(repo)\n\tif err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\tif err := prd.BoostRepoConnect(context.Background(), param); err != nil {\n\t\treturn errors.WithStack(err)\n\t}\n\n\treturn prd.Forget(context.Background(), snapshot, param)\n}\n\nfunc (m *manager) BatchForget(ctx context.Context, repo *velerov1api.BackupRepository, snapshots []string) []error {\n\tm.repoLocker.LockExclusive(repo.Name)\n\tdefer m.repoLocker.UnlockExclusive(repo.Name)\n\n\tprd, err := m.getRepositoryProvider(repo)\n\tif err != nil {\n\t\treturn []error{errors.WithStack(err)}\n\t}\n\tparam, err := m.assembleRepoParam(repo)\n\tif err != nil {\n\t\treturn []error{errors.WithStack(err)}\n\t}\n\n\tif err := prd.BoostRepoConnect(context.Background(), param); err != nil {\n\t\treturn []error{errors.WithStack(err)}\n\t}\n\n\treturn prd.BatchForget(context.Background(), snapshots, param)\n}\n\nfunc (m *manager) DefaultMaintenanceFrequency(repo *velerov1api.BackupRepository) (time.Duration, error) {\n\tprd, err := m.getRepositoryProvider(repo)\n\tif err != nil {\n\t\treturn 0, errors.WithStack(err)\n\t}\n\n\treturn prd.DefaultMaintenanceFrequency(), nil\n}\n\nfunc (m *manager) ClientSideCacheLimit(repo *velerov1api.BackupRepository) (int64, error) {\n\tprd, err := m.getRepositoryProvider(repo)\n\tif err != nil {\n\t\treturn 0, errors.WithStack(err)\n\t}\n\n\treturn prd.ClientSideCacheLimit(repo.Spec.RepositoryConfig), nil\n}\n\nfunc (m *manager) getRepositoryProvider(repo *velerov1api.BackupRepository) (provider.Provider, error) {\n\tswitch repo.Spec.RepositoryType {\n\tcase \"\", velerov1api.BackupRepositoryTypeRestic:\n\t\treturn m.providers[velerov1api.BackupRepositoryTypeRestic], nil\n\tcase velerov1api.BackupRepositoryTypeKopia:\n\t\treturn m.providers[velerov1api.BackupRepositoryTypeKopia], nil\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"failed to get provider for repository %s\", repo.Spec.RepositoryType)\n\t}\n}\n\nfunc (m *manager) assembleRepoParam(repo *velerov1api.BackupRepository) (provider.RepoParam, error) {\n\tbsl := &velerov1api.BackupStorageLocation{}\n\tif err := m.client.Get(context.Background(), client.ObjectKey{Namespace: m.namespace, Name: repo.Spec.BackupStorageLocation}, bsl); err != nil {\n\t\treturn provider.RepoParam{}, errors.WithStack(err)\n\t}\n\treturn provider.RepoParam{\n\t\tBackupLocation: bsl,\n\t\tBackupRepo: repo,\n\t}, nil\n}\n\nfunc (cm *configManager) DefaultMaintenanceFrequency(repoType string) (time.Duration, error) {\n\tprd, err := cm.getRepositoryProvider(repoType)\n\tif err != nil {\n\t\treturn 0, errors.WithStack(err)\n\t}\n\n\treturn prd.DefaultMaintenanceFrequency(), nil\n}\n\nfunc (cm *configManager) ClientSideCacheLimit(repoType string, repoOption map[string]string) (int64, error) {\n\tprd, err := cm.getRepositoryProvider(repoType)\n\tif err != nil {\n\t\treturn 0, errors.WithStack(err)\n\t}\n\n\treturn prd.ClientSideCacheLimit(repoOption), nil\n}\n\nfunc (cm *configManager) getRepositoryProvider(repoType string) (provider.ConfigProvider, error) {\n\tswitch repoType {\n\tcase velerov1api.BackupRepositoryTypeKopia:\n\t\treturn cm.providers[velerov1api.BackupRepositoryTypeKopia], nil\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"failed to get provider for repository %s\", repoType)\n\t}\n}\n" }, { "path": "pkg/repository/manager/manager_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage repository\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\nfunc TestGetRepositoryProvider(t *testing.T) {\n\tvar fakeClient kbclient.Client\n\tmgr := NewManager(\"\", fakeClient, nil, nil, nil, nil).(*manager)\n\trepo := &velerov1.BackupRepository{}\n\n\t// empty repository type\n\tprovider, err := mgr.getRepositoryProvider(repo)\n\trequire.NoError(t, err)\n\tassert.NotNil(t, provider)\n\n\t// valid repository type\n\trepo.Spec.RepositoryType = velerov1.BackupRepositoryTypeRestic\n\tprovider, err = mgr.getRepositoryProvider(repo)\n\trequire.NoError(t, err)\n\tassert.NotNil(t, provider)\n\n\t// invalid repository type\n\trepo.Spec.RepositoryType = \"unknown\"\n\t_, err = mgr.getRepositoryProvider(repo)\n\trequire.Error(t, err)\n}\n\nfunc TestGetRepositoryConfigProvider(t *testing.T) {\n\tmgr := NewConfigManager(nil).(*configManager)\n\n\t// empty repository type\n\t_, err := mgr.getRepositoryProvider(\"\")\n\trequire.Error(t, err)\n\n\t// valid repository type\n\tprovider, err := mgr.getRepositoryProvider(velerov1.BackupRepositoryTypeKopia)\n\trequire.NoError(t, err)\n\tassert.NotNil(t, provider)\n\n\t// invalid repository type\n\t_, err = mgr.getRepositoryProvider(velerov1.BackupRepositoryTypeRestic)\n\trequire.Error(t, err)\n}\n" }, { "path": "pkg/repository/mocks/ConfigManager.go", "content": "// Code generated by mockery v2.53.2. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tmock \"github.com/stretchr/testify/mock\"\n\n\ttime \"time\"\n)\n\n// ConfigManager is an autogenerated mock type for the ConfigManager type\ntype ConfigManager struct {\n\tmock.Mock\n}\n\n// ClientSideCacheLimit provides a mock function with given fields: repoType, repoOption\nfunc (_m *ConfigManager) ClientSideCacheLimit(repoType string, repoOption map[string]string) (int64, error) {\n\tret := _m.Called(repoType, repoOption)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for ClientSideCacheLimit\")\n\t}\n\n\tvar r0 int64\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string, map[string]string) (int64, error)); ok {\n\t\treturn rf(repoType, repoOption)\n\t}\n\tif rf, ok := ret.Get(0).(func(string, map[string]string) int64); ok {\n\t\tr0 = rf(repoType, repoOption)\n\t} else {\n\t\tr0 = ret.Get(0).(int64)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string, map[string]string) error); ok {\n\t\tr1 = rf(repoType, repoOption)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// DefaultMaintenanceFrequency provides a mock function with given fields: repoType\nfunc (_m *ConfigManager) DefaultMaintenanceFrequency(repoType string) (time.Duration, error) {\n\tret := _m.Called(repoType)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for DefaultMaintenanceFrequency\")\n\t}\n\n\tvar r0 time.Duration\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(string) (time.Duration, error)); ok {\n\t\treturn rf(repoType)\n\t}\n\tif rf, ok := ret.Get(0).(func(string) time.Duration); ok {\n\t\tr0 = rf(repoType)\n\t} else {\n\t\tr0 = ret.Get(0).(time.Duration)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(string) error); ok {\n\t\tr1 = rf(repoType)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// NewConfigManager creates a new instance of ConfigManager. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\n// The first argument is typically a *testing.T value.\nfunc NewConfigManager(t interface {\n\tmock.TestingT\n\tCleanup(func())\n}) *ConfigManager {\n\tmock := &ConfigManager{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/repository/mocks/Manager.go", "content": "// Code generated by mockery v2.53.2. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tcontext \"context\"\n\n\tmock \"github.com/stretchr/testify/mock\"\n\n\ttime \"time\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// Manager is an autogenerated mock type for the Manager type\ntype Manager struct {\n\tmock.Mock\n}\n\n// BatchForget provides a mock function with given fields: _a0, _a1, _a2\nfunc (_m *Manager) BatchForget(_a0 context.Context, _a1 *v1.BackupRepository, _a2 []string) []error {\n\tret := _m.Called(_a0, _a1, _a2)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for BatchForget\")\n\t}\n\n\tvar r0 []error\n\tif rf, ok := ret.Get(0).(func(context.Context, *v1.BackupRepository, []string) []error); ok {\n\t\tr0 = rf(_a0, _a1, _a2)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]error)\n\t\t}\n\t}\n\n\treturn r0\n}\n\n// ClientSideCacheLimit provides a mock function with given fields: _a0\nfunc (_m *Manager) ClientSideCacheLimit(_a0 *v1.BackupRepository) (int64, error) {\n\tret := _m.Called(_a0)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for ClientSideCacheLimit\")\n\t}\n\n\tvar r0 int64\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(*v1.BackupRepository) (int64, error)); ok {\n\t\treturn rf(_a0)\n\t}\n\tif rf, ok := ret.Get(0).(func(*v1.BackupRepository) int64); ok {\n\t\tr0 = rf(_a0)\n\t} else {\n\t\tr0 = ret.Get(0).(int64)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(*v1.BackupRepository) error); ok {\n\t\tr1 = rf(_a0)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// ConnectToRepo provides a mock function with given fields: repo\nfunc (_m *Manager) ConnectToRepo(repo *v1.BackupRepository) error {\n\tret := _m.Called(repo)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for ConnectToRepo\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(*v1.BackupRepository) error); ok {\n\t\tr0 = rf(repo)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// DefaultMaintenanceFrequency provides a mock function with given fields: _a0\nfunc (_m *Manager) DefaultMaintenanceFrequency(_a0 *v1.BackupRepository) (time.Duration, error) {\n\tret := _m.Called(_a0)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for DefaultMaintenanceFrequency\")\n\t}\n\n\tvar r0 time.Duration\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(*v1.BackupRepository) (time.Duration, error)); ok {\n\t\treturn rf(_a0)\n\t}\n\tif rf, ok := ret.Get(0).(func(*v1.BackupRepository) time.Duration); ok {\n\t\tr0 = rf(_a0)\n\t} else {\n\t\tr0 = ret.Get(0).(time.Duration)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(*v1.BackupRepository) error); ok {\n\t\tr1 = rf(_a0)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Forget provides a mock function with given fields: _a0, _a1, _a2\nfunc (_m *Manager) Forget(_a0 context.Context, _a1 *v1.BackupRepository, _a2 string) error {\n\tret := _m.Called(_a0, _a1, _a2)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Forget\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, *v1.BackupRepository, string) error); ok {\n\t\tr0 = rf(_a0, _a1, _a2)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// InitRepo provides a mock function with given fields: repo\nfunc (_m *Manager) InitRepo(repo *v1.BackupRepository) error {\n\tret := _m.Called(repo)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for InitRepo\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(*v1.BackupRepository) error); ok {\n\t\tr0 = rf(repo)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// PrepareRepo provides a mock function with given fields: repo\nfunc (_m *Manager) PrepareRepo(repo *v1.BackupRepository) error {\n\tret := _m.Called(repo)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PrepareRepo\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(*v1.BackupRepository) error); ok {\n\t\tr0 = rf(repo)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// PruneRepo provides a mock function with given fields: repo\nfunc (_m *Manager) PruneRepo(repo *v1.BackupRepository) error {\n\tret := _m.Called(repo)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PruneRepo\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(*v1.BackupRepository) error); ok {\n\t\tr0 = rf(repo)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// UnlockRepo provides a mock function with given fields: repo\nfunc (_m *Manager) UnlockRepo(repo *v1.BackupRepository) error {\n\tret := _m.Called(repo)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for UnlockRepo\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(*v1.BackupRepository) error); ok {\n\t\tr0 = rf(repo)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// NewManager creates a new instance of Manager. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\n// The first argument is typically a *testing.T value.\nfunc NewManager(t interface {\n\tmock.TestingT\n\tCleanup(func())\n}) *Manager {\n\tmock := &Manager{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/repository/mocks/RepositoryWriter.go", "content": "// Code generated by mockery v2.39.1. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tcontext \"context\"\n\n\tindex \"github.com/kopia/kopia/repo/content/index\"\n\tmanifest \"github.com/kopia/kopia/repo/manifest\"\n\n\tmock \"github.com/stretchr/testify/mock\"\n\n\tobject \"github.com/kopia/kopia/repo/object\"\n\n\trepo \"github.com/kopia/kopia/repo\"\n\n\ttime \"time\"\n)\n\n// RepositoryWriter is an autogenerated mock type for the RepositoryWriter type\ntype RepositoryWriter struct {\n\tmock.Mock\n}\n\n// ClientOptions provides a mock function with given fields:\nfunc (_m *RepositoryWriter) ClientOptions() repo.ClientOptions {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for ClientOptions\")\n\t}\n\n\tvar r0 repo.ClientOptions\n\tif rf, ok := ret.Get(0).(func() repo.ClientOptions); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(repo.ClientOptions)\n\t}\n\n\treturn r0\n}\n\n// Close provides a mock function with given fields: ctx\nfunc (_m *RepositoryWriter) Close(ctx context.Context) error {\n\tret := _m.Called(ctx)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Close\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context) error); ok {\n\t\tr0 = rf(ctx)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// ConcatenateObjects provides a mock function with given fields: ctx, objectIDs, opt\nfunc (_m *RepositoryWriter) ConcatenateObjects(ctx context.Context, objectIDs []object.ID, opt repo.ConcatenateOptions) (object.ID, error) {\n\tret := _m.Called(ctx, objectIDs, opt)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for ConcatenateObjects\")\n\t}\n\n\tvar r0 object.ID\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(context.Context, []object.ID, repo.ConcatenateOptions) (object.ID, error)); ok {\n\t\treturn rf(ctx, objectIDs, opt)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, []object.ID, repo.ConcatenateOptions) object.ID); ok {\n\t\tr0 = rf(ctx, objectIDs, opt)\n\t} else {\n\t\tr0 = ret.Get(0).(object.ID)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, []object.ID, repo.ConcatenateOptions) error); ok {\n\t\tr1 = rf(ctx, objectIDs, opt)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// ContentInfo provides a mock function with given fields: ctx, contentID\nfunc (_m *RepositoryWriter) ContentInfo(ctx context.Context, contentID index.ID) (index.Info, error) {\n\tret := _m.Called(ctx, contentID)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for ContentInfo\")\n\t}\n\n\tvar r0 index.Info\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(context.Context, index.ID) (index.Info, error)); ok {\n\t\treturn rf(ctx, contentID)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, index.ID) index.Info); ok {\n\t\tr0 = rf(ctx, contentID)\n\t} else {\n\t\tr0 = ret.Get(0).(index.Info)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, index.ID) error); ok {\n\t\tr1 = rf(ctx, contentID)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// DeleteManifest provides a mock function with given fields: ctx, id\nfunc (_m *RepositoryWriter) DeleteManifest(ctx context.Context, id manifest.ID) error {\n\tret := _m.Called(ctx, id)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for DeleteManifest\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, manifest.ID) error); ok {\n\t\tr0 = rf(ctx, id)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// FindManifests provides a mock function with given fields: ctx, labels\nfunc (_m *RepositoryWriter) FindManifests(ctx context.Context, labels map[string]string) ([]*manifest.EntryMetadata, error) {\n\tret := _m.Called(ctx, labels)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for FindManifests\")\n\t}\n\n\tvar r0 []*manifest.EntryMetadata\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(context.Context, map[string]string) ([]*manifest.EntryMetadata, error)); ok {\n\t\treturn rf(ctx, labels)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, map[string]string) []*manifest.EntryMetadata); ok {\n\t\tr0 = rf(ctx, labels)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]*manifest.EntryMetadata)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, map[string]string) error); ok {\n\t\tr1 = rf(ctx, labels)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Flush provides a mock function with given fields: ctx\nfunc (_m *RepositoryWriter) Flush(ctx context.Context) error {\n\tret := _m.Called(ctx)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Flush\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context) error); ok {\n\t\tr0 = rf(ctx)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// GetManifest provides a mock function with given fields: ctx, id, data\nfunc (_m *RepositoryWriter) GetManifest(ctx context.Context, id manifest.ID, data interface{}) (*manifest.EntryMetadata, error) {\n\tret := _m.Called(ctx, id, data)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetManifest\")\n\t}\n\n\tvar r0 *manifest.EntryMetadata\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(context.Context, manifest.ID, interface{}) (*manifest.EntryMetadata, error)); ok {\n\t\treturn rf(ctx, id, data)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, manifest.ID, interface{}) *manifest.EntryMetadata); ok {\n\t\tr0 = rf(ctx, id, data)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(*manifest.EntryMetadata)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, manifest.ID, interface{}) error); ok {\n\t\tr1 = rf(ctx, id, data)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// NewObjectWriter provides a mock function with given fields: ctx, opt\nfunc (_m *RepositoryWriter) NewObjectWriter(ctx context.Context, opt object.WriterOptions) object.Writer {\n\tret := _m.Called(ctx, opt)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for NewObjectWriter\")\n\t}\n\n\tvar r0 object.Writer\n\tif rf, ok := ret.Get(0).(func(context.Context, object.WriterOptions) object.Writer); ok {\n\t\tr0 = rf(ctx, opt)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(object.Writer)\n\t\t}\n\t}\n\n\treturn r0\n}\n\n// NewWriter provides a mock function with given fields: ctx, opt\nfunc (_m *RepositoryWriter) NewWriter(ctx context.Context, opt repo.WriteSessionOptions) (context.Context, repo.RepositoryWriter, error) {\n\tret := _m.Called(ctx, opt)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for NewWriter\")\n\t}\n\n\tvar r0 context.Context\n\tvar r1 repo.RepositoryWriter\n\tvar r2 error\n\tif rf, ok := ret.Get(0).(func(context.Context, repo.WriteSessionOptions) (context.Context, repo.RepositoryWriter, error)); ok {\n\t\treturn rf(ctx, opt)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, repo.WriteSessionOptions) context.Context); ok {\n\t\tr0 = rf(ctx, opt)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(context.Context)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, repo.WriteSessionOptions) repo.RepositoryWriter); ok {\n\t\tr1 = rf(ctx, opt)\n\t} else {\n\t\tif ret.Get(1) != nil {\n\t\t\tr1 = ret.Get(1).(repo.RepositoryWriter)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(2).(func(context.Context, repo.WriteSessionOptions) error); ok {\n\t\tr2 = rf(ctx, opt)\n\t} else {\n\t\tr2 = ret.Error(2)\n\t}\n\n\treturn r0, r1, r2\n}\n\n// OnSuccessfulFlush provides a mock function with given fields: callback\nfunc (_m *RepositoryWriter) OnSuccessfulFlush(callback repo.RepositoryWriterCallback) {\n\t_m.Called(callback)\n}\n\n// OpenObject provides a mock function with given fields: ctx, id\nfunc (_m *RepositoryWriter) OpenObject(ctx context.Context, id object.ID) (object.Reader, error) {\n\tret := _m.Called(ctx, id)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for OpenObject\")\n\t}\n\n\tvar r0 object.Reader\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(context.Context, object.ID) (object.Reader, error)); ok {\n\t\treturn rf(ctx, id)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, object.ID) object.Reader); ok {\n\t\tr0 = rf(ctx, id)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(object.Reader)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, object.ID) error); ok {\n\t\tr1 = rf(ctx, id)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// PrefetchContents provides a mock function with given fields: ctx, contentIDs, hint\nfunc (_m *RepositoryWriter) PrefetchContents(ctx context.Context, contentIDs []index.ID, hint string) []index.ID {\n\tret := _m.Called(ctx, contentIDs, hint)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PrefetchContents\")\n\t}\n\n\tvar r0 []index.ID\n\tif rf, ok := ret.Get(0).(func(context.Context, []index.ID, string) []index.ID); ok {\n\t\tr0 = rf(ctx, contentIDs, hint)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]index.ID)\n\t\t}\n\t}\n\n\treturn r0\n}\n\n// PrefetchObjects provides a mock function with given fields: ctx, objectIDs, hint\nfunc (_m *RepositoryWriter) PrefetchObjects(ctx context.Context, objectIDs []object.ID, hint string) ([]index.ID, error) {\n\tret := _m.Called(ctx, objectIDs, hint)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PrefetchObjects\")\n\t}\n\n\tvar r0 []index.ID\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(context.Context, []object.ID, string) ([]index.ID, error)); ok {\n\t\treturn rf(ctx, objectIDs, hint)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, []object.ID, string) []index.ID); ok {\n\t\tr0 = rf(ctx, objectIDs, hint)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]index.ID)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, []object.ID, string) error); ok {\n\t\tr1 = rf(ctx, objectIDs, hint)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// PutManifest provides a mock function with given fields: ctx, labels, payload\nfunc (_m *RepositoryWriter) PutManifest(ctx context.Context, labels map[string]string, payload interface{}) (manifest.ID, error) {\n\tret := _m.Called(ctx, labels, payload)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PutManifest\")\n\t}\n\n\tvar r0 manifest.ID\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(context.Context, map[string]string, interface{}) (manifest.ID, error)); ok {\n\t\treturn rf(ctx, labels, payload)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, map[string]string, interface{}) manifest.ID); ok {\n\t\tr0 = rf(ctx, labels, payload)\n\t} else {\n\t\tr0 = ret.Get(0).(manifest.ID)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, map[string]string, interface{}) error); ok {\n\t\tr1 = rf(ctx, labels, payload)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Refresh provides a mock function with given fields: ctx\nfunc (_m *RepositoryWriter) Refresh(ctx context.Context) error {\n\tret := _m.Called(ctx)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Refresh\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context) error); ok {\n\t\tr0 = rf(ctx)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// ReplaceManifests provides a mock function with given fields: ctx, labels, payload\nfunc (_m *RepositoryWriter) ReplaceManifests(ctx context.Context, labels map[string]string, payload interface{}) (manifest.ID, error) {\n\tret := _m.Called(ctx, labels, payload)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for ReplaceManifests\")\n\t}\n\n\tvar r0 manifest.ID\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(context.Context, map[string]string, interface{}) (manifest.ID, error)); ok {\n\t\treturn rf(ctx, labels, payload)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, map[string]string, interface{}) manifest.ID); ok {\n\t\tr0 = rf(ctx, labels, payload)\n\t} else {\n\t\tr0 = ret.Get(0).(manifest.ID)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, map[string]string, interface{}) error); ok {\n\t\tr1 = rf(ctx, labels, payload)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Time provides a mock function with given fields:\nfunc (_m *RepositoryWriter) Time() time.Time {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Time\")\n\t}\n\n\tvar r0 time.Time\n\tif rf, ok := ret.Get(0).(func() time.Time); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(time.Time)\n\t}\n\n\treturn r0\n}\n\n// UpdateDescription provides a mock function with given fields: d\nfunc (_m *RepositoryWriter) UpdateDescription(d string) {\n\t_m.Called(d)\n}\n\n// VerifyObject provides a mock function with given fields: ctx, id\nfunc (_m *RepositoryWriter) VerifyObject(ctx context.Context, id object.ID) ([]index.ID, error) {\n\tret := _m.Called(ctx, id)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for VerifyObject\")\n\t}\n\n\tvar r0 []index.ID\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(context.Context, object.ID) ([]index.ID, error)); ok {\n\t\treturn rf(ctx, id)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, object.ID) []index.ID); ok {\n\t\tr0 = rf(ctx, id)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]index.ID)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, object.ID) error); ok {\n\t\tr1 = rf(ctx, id)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// NewRepositoryWriter creates a new instance of RepositoryWriter. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\n// The first argument is typically a *testing.T value.\nfunc NewRepositoryWriter(t interface {\n\tmock.TestingT\n\tCleanup(func())\n}) *RepositoryWriter {\n\tmock := &RepositoryWriter{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/repository/provider/provider.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage provider\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// RepoParam includes the parameters to manipulate a backup repository\ntype RepoParam struct {\n\tBackupLocation *velerov1api.BackupStorageLocation\n\tBackupRepo *velerov1api.BackupRepository\n\tCacheDir string\n}\n\n// Provider defines the methods to manipulate a backup repository\ntype Provider interface {\n\tConfigProvider\n\n\t// InitRepo is to initialize a repository from a new storage place\n\tInitRepo(ctx context.Context, param RepoParam) error\n\n\t// ConnectToRepo is to establish the connection to a\n\t// storage place that a repository is already initialized\n\tConnectToRepo(ctx context.Context, param RepoParam) error\n\n\t// PrepareRepo is a combination of InitRepo and ConnectToRepo,\n\t// it may do initializing + connecting, connecting only if the repository\n\t// is already initialized, or do nothing if the repository is already connected\n\tPrepareRepo(ctx context.Context, param RepoParam) error\n\n\t// BoostRepoConnect is used to re-ensure the local connection to the repo,\n\t// so that the followed operations could succeed in some environment reset\n\t// scenarios, for example, pod restart\n\tBoostRepoConnect(ctx context.Context, param RepoParam) error\n\n\t// PruneRepo does a full prune/maintenance of the repository\n\tPruneRepo(ctx context.Context, param RepoParam) error\n\n\t// EnsureUnlockRepo esures to remove any stale file locks in the storage\n\tEnsureUnlockRepo(ctx context.Context, param RepoParam) error\n\n\t// Forget is to delete a snapshot from the repository\n\tForget(ctx context.Context, snapshotID string, param RepoParam) error\n\n\t// BatchForget is to delete a list of snapshots from the repository\n\tBatchForget(ctx context.Context, snapshotIDs []string, param RepoParam) []error\n}\n\n// ConfigProvider defines the methods to get configurations of a backup repository\ntype ConfigProvider interface {\n\t// DefaultMaintenanceFrequency returns the default frequency to run maintenance\n\tDefaultMaintenanceFrequency() time.Duration\n\n\t// ClientSideCacheLimit returns the max cache size required on client side\n\tClientSideCacheLimit(repoOption map[string]string) int64\n}\n" }, { "path": "pkg/repository/provider/restic.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage provider\n\nimport (\n\t\"context\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/restic\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n)\n\nfunc NewResticRepositoryProvider(credGetter credentials.CredentialGetter, fs filesystem.Interface, log logrus.FieldLogger) Provider {\n\treturn &resticRepositoryProvider{\n\t\tsvc: restic.NewRepositoryService(credGetter, fs, log),\n\t}\n}\n\ntype resticRepositoryProvider struct {\n\tsvc *restic.RepositoryService\n}\n\nfunc (r *resticRepositoryProvider) InitRepo(ctx context.Context, param RepoParam) error {\n\treturn r.svc.InitRepo(param.BackupLocation, param.BackupRepo)\n}\n\nfunc (r *resticRepositoryProvider) ConnectToRepo(ctx context.Context, param RepoParam) error {\n\treturn r.svc.ConnectToRepo(param.BackupLocation, param.BackupRepo)\n}\n\nfunc (r *resticRepositoryProvider) PrepareRepo(ctx context.Context, param RepoParam) error {\n\tif err := r.ConnectToRepo(ctx, param); err != nil {\n\t\t// If the repository has not yet been initialized, the error message will always include\n\t\t// the following string. This is the only scenario where we should try to initialize it.\n\t\t// Other errors (e.g. \"already locked\") should be returned as-is since the repository\n\t\t// does already exist, but it can't be connected to.\n\t\tif strings.Contains(err.Error(), \"Is there a repository at the following location?\") {\n\t\t\treturn r.InitRepo(ctx, param)\n\t\t}\n\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\nfunc (r *resticRepositoryProvider) BoostRepoConnect(ctx context.Context, param RepoParam) error {\n\treturn nil\n}\n\nfunc (r *resticRepositoryProvider) PruneRepo(ctx context.Context, param RepoParam) error {\n\treturn r.svc.PruneRepo(param.BackupLocation, param.BackupRepo)\n}\n\nfunc (r *resticRepositoryProvider) EnsureUnlockRepo(ctx context.Context, param RepoParam) error {\n\treturn r.svc.UnlockRepo(param.BackupLocation, param.BackupRepo)\n}\n\nfunc (r *resticRepositoryProvider) Forget(ctx context.Context, snapshotID string, param RepoParam) error {\n\treturn r.svc.Forget(param.BackupLocation, param.BackupRepo, snapshotID)\n}\n\nfunc (r *resticRepositoryProvider) BatchForget(ctx context.Context, snapshotIDs []string, param RepoParam) []error {\n\terrs := []error{}\n\tfor _, snapshot := range snapshotIDs {\n\t\terr := r.Forget(ctx, snapshot, param)\n\t\tif err != nil {\n\t\t\terrs = append(errs, err)\n\t\t}\n\t}\n\n\treturn errs\n}\n\nfunc (r *resticRepositoryProvider) DefaultMaintenanceFrequency() time.Duration {\n\treturn r.svc.DefaultMaintenanceFrequency()\n}\n\nfunc (r *resticRepositoryProvider) ClientSideCacheLimit(repoOption map[string]string) int64 {\n\treturn 0\n}\n" }, { "path": "pkg/repository/provider/unified_repo.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage provider\n\nimport (\n\t\"context\"\n\t\"encoding/base64\"\n\t\"fmt\"\n\t\"maps\"\n\t\"net/url\"\n\t\"path\"\n\t\"strconv\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\trepoconfig \"github.com/vmware-tanzu/velero/pkg/repository/config\"\n\trepokey \"github.com/vmware-tanzu/velero/pkg/repository/keys\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n\treposervice \"github.com/vmware-tanzu/velero/pkg/repository/udmrepo/service\"\n)\n\ntype unifiedRepoProvider struct {\n\tcredentialGetter credentials.CredentialGetter\n\tworkPath string\n\trepoService udmrepo.BackupRepoService\n\trepoBackend string\n\tlog logrus.FieldLogger\n}\n\ntype unifiedRepoConfigProvider struct {\n\trepoService udmrepo.BackupRepoService\n\trepoBackend string\n\tlog logrus.FieldLogger\n}\n\n// this func is assigned to a package-level variable so it can be\n// replaced when unit-testing\nvar getS3Credentials = repoconfig.GetS3Credentials\nvar getGCPCredentials = repoconfig.GetGCPCredentials\nvar getS3BucketRegion = repoconfig.GetAWSBucketRegion\n\ntype localFuncTable struct {\n\tgetStorageVariables func(*velerov1api.BackupStorageLocation, string, string, map[string]string, credentials.CredentialGetter) (map[string]string, error)\n\tgetStorageCredentials func(*velerov1api.BackupStorageLocation, credentials.FileStore) (map[string]string, error)\n}\n\nvar funcTable = localFuncTable{\n\tgetStorageVariables: getStorageVariables,\n\tgetStorageCredentials: getStorageCredentials,\n}\n\nconst (\n\trepoOpDescMaintain = \"repo maintenance\"\n\trepoOpDescForget = \"forget\"\n\n\trepoConnectDesc = \"unified repo\"\n)\n\n// NewUnifiedRepoProvider creates the service provider for Unified Repo\nfunc NewUnifiedRepoProvider(\n\tcredentialGetter credentials.CredentialGetter,\n\trepoBackend string,\n\tlog logrus.FieldLogger,\n) Provider {\n\trepo := unifiedRepoProvider{\n\t\tcredentialGetter: credentialGetter,\n\t\trepoBackend: repoBackend,\n\t\tlog: log,\n\t}\n\n\trepo.repoService = createRepoService(repoBackend, log)\n\n\treturn &repo\n}\n\nfunc NewUnifiedRepoConfigProvider(\n\trepoBackend string,\n\tlog logrus.FieldLogger,\n) ConfigProvider {\n\trepo := unifiedRepoConfigProvider{\n\t\trepoBackend: repoBackend,\n\t\tlog: log,\n\t}\n\n\trepo.repoService = createRepoService(repoBackend, log)\n\n\treturn &repo\n}\n\nfunc (urp *unifiedRepoProvider) InitRepo(ctx context.Context, param RepoParam) error {\n\tlog := urp.log.WithFields(logrus.Fields{\n\t\t\"BSL name\": param.BackupLocation.Name,\n\t\t\"repo name\": param.BackupRepo.Name,\n\t\t\"repo UID\": param.BackupRepo.UID,\n\t})\n\n\tlog.Debug(\"Start to init repo\")\n\n\tif param.BackupLocation.Spec.AccessMode == velerov1api.BackupStorageLocationAccessModeReadOnly {\n\t\treturn errors.Errorf(\"cannot create new backup repo for read-only backup storage location %s/%s\", param.BackupLocation.Namespace, param.BackupLocation.Name)\n\t}\n\n\trepoOption, err := udmrepo.NewRepoOptions(\n\t\tudmrepo.WithPassword(urp, param),\n\t\tudmrepo.WithConfigFile(urp.workPath, string(param.BackupRepo.UID)),\n\t\tudmrepo.WithGenOptions(\n\t\t\tmap[string]string{\n\t\t\t\tudmrepo.GenOptionOwnerName: udmrepo.GetRepoUser(),\n\t\t\t\tudmrepo.GenOptionOwnerDomain: udmrepo.GetRepoDomain(),\n\t\t\t},\n\t\t),\n\t\tudmrepo.WithStoreOptions(urp, param),\n\t\tudmrepo.WithDescription(repoConnectDesc),\n\t)\n\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to get repo options\")\n\t}\n\n\terr = urp.repoService.Create(ctx, *repoOption)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to init backup repo\")\n\t}\n\n\tlog.Debug(\"Init repo complete\")\n\n\treturn nil\n}\n\nfunc (urp *unifiedRepoProvider) ConnectToRepo(ctx context.Context, param RepoParam) error {\n\tlog := urp.log.WithFields(logrus.Fields{\n\t\t\"BSL name\": param.BackupLocation.Name,\n\t\t\"repo name\": param.BackupRepo.Name,\n\t\t\"repo UID\": param.BackupRepo.UID,\n\t})\n\n\tlog.Debug(\"Start to connect repo\")\n\n\trepoOption, err := udmrepo.NewRepoOptions(\n\t\tudmrepo.WithPassword(urp, param),\n\t\tudmrepo.WithConfigFile(urp.workPath, string(param.BackupRepo.UID)),\n\t\tudmrepo.WithGenOptions(\n\t\t\tmap[string]string{\n\t\t\t\tudmrepo.GenOptionOwnerName: udmrepo.GetRepoUser(),\n\t\t\t\tudmrepo.GenOptionOwnerDomain: udmrepo.GetRepoDomain(),\n\t\t\t},\n\t\t),\n\t\tudmrepo.WithStoreOptions(urp, param),\n\t\tudmrepo.WithDescription(repoConnectDesc),\n\t)\n\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to get repo options\")\n\t}\n\n\terr = urp.repoService.Connect(ctx, *repoOption)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to connect backup repo\")\n\t}\n\n\tlog.Debug(\"Connect repo complete\")\n\n\treturn nil\n}\n\nfunc (urp *unifiedRepoProvider) PrepareRepo(ctx context.Context, param RepoParam) error {\n\tlog := urp.log.WithFields(logrus.Fields{\n\t\t\"BSL name\": param.BackupLocation.Name,\n\t\t\"repo name\": param.BackupRepo.Name,\n\t\t\"repo UID\": param.BackupRepo.UID,\n\t})\n\n\tlog.Info(\"Start to prepare repo\")\n\n\trepoOption, err := udmrepo.NewRepoOptions(\n\t\tudmrepo.WithPassword(urp, param),\n\t\tudmrepo.WithConfigFile(urp.workPath, string(param.BackupRepo.UID)),\n\t\tudmrepo.WithGenOptions(\n\t\t\tmap[string]string{\n\t\t\t\tudmrepo.GenOptionOwnerName: udmrepo.GetRepoUser(),\n\t\t\t\tudmrepo.GenOptionOwnerDomain: udmrepo.GetRepoDomain(),\n\t\t\t},\n\t\t),\n\t\tudmrepo.WithStoreOptions(urp, param),\n\t\tudmrepo.WithDescription(repoConnectDesc),\n\t)\n\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to get repo options\")\n\t}\n\n\tif created, err := urp.repoService.IsCreated(ctx, *repoOption); err != nil {\n\t\treturn errors.Wrap(err, \"error to check backup repo\")\n\t} else if created {\n\t\tlog.Info(\"Repo has already been initialized\")\n\t\treturn nil\n\t}\n\n\tif param.BackupLocation.Spec.AccessMode == velerov1api.BackupStorageLocationAccessModeReadOnly {\n\t\treturn errors.Errorf(\"cannot create new backup repo for read-only backup storage location %s/%s\", param.BackupLocation.Namespace, param.BackupLocation.Name)\n\t}\n\n\terr = urp.repoService.Create(ctx, *repoOption)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to create backup repo\")\n\t}\n\n\tlog.Info(\"Prepare repo complete\")\n\n\treturn nil\n}\n\nfunc (urp *unifiedRepoProvider) BoostRepoConnect(ctx context.Context, param RepoParam) error {\n\tlog := urp.log.WithFields(logrus.Fields{\n\t\t\"BSL name\": param.BackupLocation.Name,\n\t\t\"repo name\": param.BackupRepo.Name,\n\t\t\"repo UID\": param.BackupRepo.UID,\n\t})\n\n\tlog.Debug(\"Start to boost repo connect\")\n\n\trepoOption, err := udmrepo.NewRepoOptions(\n\t\tudmrepo.WithPassword(urp, param),\n\t\tudmrepo.WithConfigFile(urp.workPath, string(param.BackupRepo.UID)),\n\t\tudmrepo.WithDescription(repoConnectDesc),\n\t)\n\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to get repo options\")\n\t}\n\n\tbkRepo, err := urp.repoService.Open(ctx, *repoOption)\n\tif err == nil {\n\t\tif c := bkRepo.Close(ctx); c != nil {\n\t\t\tlog.WithError(c).Error(\"Failed to close repo\")\n\t\t}\n\n\t\treturn nil\n\t}\n\n\treturn urp.ConnectToRepo(ctx, param)\n}\n\nfunc (urp *unifiedRepoProvider) PruneRepo(ctx context.Context, param RepoParam) error {\n\tlog := urp.log.WithFields(logrus.Fields{\n\t\t\"BSL name\": param.BackupLocation.Name,\n\t\t\"repo name\": param.BackupRepo.Name,\n\t\t\"repo UID\": param.BackupRepo.UID,\n\t})\n\n\tlog.Debug(\"Start to prune repo\")\n\n\trepoOption, err := udmrepo.NewRepoOptions(\n\t\tudmrepo.WithPassword(urp, param),\n\t\tudmrepo.WithConfigFile(urp.workPath, string(param.BackupRepo.UID)),\n\t\tudmrepo.WithDescription(repoOpDescMaintain),\n\t)\n\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to get repo options\")\n\t}\n\n\terr = urp.repoService.Maintain(ctx, *repoOption)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to prune backup repo\")\n\t}\n\n\tlog.Debug(\"Prune repo complete\")\n\n\treturn nil\n}\n\nfunc (urp *unifiedRepoProvider) EnsureUnlockRepo(ctx context.Context, param RepoParam) error {\n\treturn nil\n}\n\nfunc (urp *unifiedRepoProvider) Forget(ctx context.Context, snapshotID string, param RepoParam) error {\n\tlog := urp.log.WithFields(logrus.Fields{\n\t\t\"BSL name\": param.BackupLocation.Name,\n\t\t\"repo name\": param.BackupRepo.Name,\n\t\t\"repo UID\": param.BackupRepo.UID,\n\t\t\"snapshotID\": snapshotID,\n\t})\n\n\tlog.Debug(\"Start to forget snapshot\")\n\n\trepoOption, err := udmrepo.NewRepoOptions(\n\t\tudmrepo.WithPassword(urp, param),\n\t\tudmrepo.WithConfigFile(urp.workPath, string(param.BackupRepo.UID)),\n\t\tudmrepo.WithDescription(repoOpDescForget),\n\t)\n\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to get repo options\")\n\t}\n\n\tbkRepo, err := urp.repoService.Open(ctx, *repoOption)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to open backup repo\")\n\t}\n\n\tdefer func() {\n\t\tc := bkRepo.Close(ctx)\n\t\tif c != nil {\n\t\t\tlog.WithError(c).Error(\"Failed to close repo\")\n\t\t}\n\t}()\n\n\terr = bkRepo.DeleteManifest(ctx, udmrepo.ID(snapshotID))\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to delete manifest\")\n\t}\n\n\terr = bkRepo.Flush(ctx)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to flush repo\")\n\t}\n\n\tlog.Debug(\"Forget snapshot complete\")\n\n\treturn nil\n}\n\nfunc (urp *unifiedRepoProvider) BatchForget(ctx context.Context, snapshotIDs []string, param RepoParam) []error {\n\tlog := urp.log.WithFields(logrus.Fields{\n\t\t\"BSL name\": param.BackupLocation.Name,\n\t\t\"repo name\": param.BackupRepo.Name,\n\t\t\"repo UID\": param.BackupRepo.UID,\n\t\t\"snapshotIDs\": snapshotIDs,\n\t})\n\n\tlog.Debug(\"Start to batch forget snapshot\")\n\n\trepoOption, err := udmrepo.NewRepoOptions(\n\t\tudmrepo.WithPassword(urp, param),\n\t\tudmrepo.WithConfigFile(urp.workPath, string(param.BackupRepo.UID)),\n\t\tudmrepo.WithDescription(repoOpDescForget),\n\t)\n\n\tif err != nil {\n\t\treturn []error{errors.Wrap(err, \"error to get repo options\")}\n\t}\n\n\tbkRepo, err := urp.repoService.Open(ctx, *repoOption)\n\tif err != nil {\n\t\treturn []error{errors.Wrap(err, \"error to open backup repo\")}\n\t}\n\n\tdefer func() {\n\t\tc := bkRepo.Close(ctx)\n\t\tif c != nil {\n\t\t\tlog.WithError(c).Error(\"Failed to close repo\")\n\t\t}\n\t}()\n\n\terrs := []error{}\n\tfor _, snapshotID := range snapshotIDs {\n\t\terr = bkRepo.DeleteManifest(ctx, udmrepo.ID(snapshotID))\n\t\tif err != nil {\n\t\t\terrs = append(errs, errors.Wrapf(err, \"error to delete manifest %s\", snapshotID))\n\t\t}\n\t}\n\n\terr = bkRepo.Flush(ctx)\n\tif err != nil {\n\t\treturn []error{errors.Wrap(err, \"error to flush repo\")}\n\t}\n\n\tlog.Debug(\"Forget snapshot complete\")\n\n\treturn errs\n}\n\nfunc (urp *unifiedRepoProvider) DefaultMaintenanceFrequency() time.Duration {\n\treturn urp.repoService.DefaultMaintenanceFrequency()\n}\n\nfunc (urp *unifiedRepoProvider) ClientSideCacheLimit(repoOption map[string]string) int64 {\n\treturn urp.repoService.ClientSideCacheLimit(repoOption)\n}\n\nfunc (urp *unifiedRepoProvider) GetPassword(param any) (string, error) {\n\t_, ok := param.(RepoParam)\n\tif !ok {\n\t\treturn \"\", errors.Errorf(\"invalid parameter, expect %T, actual %T\", RepoParam{}, param)\n\t}\n\n\trepoPassword, err := getRepoPassword(urp.credentialGetter.FromSecret)\n\tif err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error to get repo password\")\n\t}\n\n\treturn repoPassword, nil\n}\n\nfunc (urp *unifiedRepoProvider) GetStoreType(param any) (string, error) {\n\trepoParam, ok := param.(RepoParam)\n\tif !ok {\n\t\treturn \"\", errors.Errorf(\"invalid parameter, expect %T, actual %T\", RepoParam{}, param)\n\t}\n\n\treturn getStorageType(repoParam.BackupLocation), nil\n}\n\nfunc (urp *unifiedRepoProvider) GetStoreOptions(param any) (map[string]string, error) {\n\trepoParam, ok := param.(RepoParam)\n\tif !ok {\n\t\treturn map[string]string{}, errors.Errorf(\"invalid parameter, expect %T, actual %T\", RepoParam{}, param)\n\t}\n\n\tstoreVar, err := funcTable.getStorageVariables(repoParam.BackupLocation, urp.repoBackend, repoParam.BackupRepo.Spec.VolumeNamespace, repoParam.BackupRepo.Spec.RepositoryConfig, urp.credentialGetter)\n\tif err != nil {\n\t\treturn map[string]string{}, errors.Wrap(err, \"error to get storage variables\")\n\t}\n\n\tstoreCred, err := funcTable.getStorageCredentials(repoParam.BackupLocation, urp.credentialGetter.FromFile)\n\tif err != nil {\n\t\treturn map[string]string{}, errors.Wrap(err, \"error to get repo credentials\")\n\t}\n\n\tstoreOptions := make(map[string]string)\n\tmaps.Copy(storeOptions, storeVar)\n\tmaps.Copy(storeOptions, storeCred)\n\n\tif repoParam.CacheDir != \"\" {\n\t\tstoreOptions[udmrepo.StoreOptionCacheDir] = repoParam.CacheDir\n\t}\n\n\treturn storeOptions, nil\n}\n\nfunc (urcp *unifiedRepoConfigProvider) DefaultMaintenanceFrequency() time.Duration {\n\treturn urcp.repoService.DefaultMaintenanceFrequency()\n}\n\nfunc (urcp *unifiedRepoConfigProvider) ClientSideCacheLimit(repoOption map[string]string) int64 {\n\treturn urcp.repoService.ClientSideCacheLimit(repoOption)\n}\n\nfunc getRepoPassword(secretStore credentials.SecretStore) (string, error) {\n\tif secretStore == nil {\n\t\treturn \"\", errors.New(\"invalid credentials interface\")\n\t}\n\n\trawPass, err := secretStore.Get(repokey.RepoKeySelector())\n\tif err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error to get password\")\n\t}\n\n\treturn strings.TrimSpace(rawPass), nil\n}\n\nfunc getStorageType(backupLocation *velerov1api.BackupStorageLocation) string {\n\tbackendType := repoconfig.GetBackendType(backupLocation.Spec.Provider, backupLocation.Spec.Config)\n\n\tswitch backendType {\n\tcase repoconfig.AWSBackend:\n\t\treturn udmrepo.StorageTypeS3\n\tcase repoconfig.AzureBackend:\n\t\treturn udmrepo.StorageTypeAzure\n\tcase repoconfig.GCPBackend:\n\t\treturn udmrepo.StorageTypeGcs\n\tcase repoconfig.FSBackend:\n\t\treturn udmrepo.StorageTypeFs\n\tdefault:\n\t\treturn \"\"\n\t}\n}\n\nfunc getStorageCredentials(backupLocation *velerov1api.BackupStorageLocation, credentialsFileStore credentials.FileStore) (map[string]string, error) {\n\tresult := make(map[string]string)\n\tvar err error\n\n\tif credentialsFileStore == nil {\n\t\treturn map[string]string{}, errors.New(\"invalid credentials interface\")\n\t}\n\n\tbackendType := repoconfig.GetBackendType(backupLocation.Spec.Provider, backupLocation.Spec.Config)\n\tif !repoconfig.IsBackendTypeValid(backendType) {\n\t\treturn map[string]string{}, errors.New(\"invalid storage provider\")\n\t}\n\n\tconfig := backupLocation.Spec.Config\n\tif config == nil {\n\t\tconfig = map[string]string{}\n\t}\n\n\tif backupLocation.Spec.Credential != nil {\n\t\tconfig[repoconfig.CredentialsFileKey], err = credentialsFileStore.Path(backupLocation.Spec.Credential)\n\t\tif err != nil {\n\t\t\treturn map[string]string{}, errors.Wrap(err, \"error get credential file in bsl\")\n\t\t}\n\t}\n\n\tswitch backendType {\n\tcase repoconfig.AWSBackend:\n\t\tcredValue, err := getS3Credentials(config)\n\t\tif err != nil {\n\t\t\treturn map[string]string{}, errors.Wrap(err, \"error get s3 credentials\")\n\t\t}\n\n\t\tif credValue != nil {\n\t\t\tresult[udmrepo.StoreOptionS3KeyID] = credValue.AccessKeyID\n\t\t\tresult[udmrepo.StoreOptionS3Provider] = credValue.Source\n\t\t\tresult[udmrepo.StoreOptionS3SecretKey] = credValue.SecretAccessKey\n\t\t\tresult[udmrepo.StoreOptionS3Token] = credValue.SessionToken\n\t\t}\n\tcase repoconfig.AzureBackend:\n\t\tif config[repoconfig.CredentialsFileKey] != \"\" {\n\t\t\tresult[repoconfig.CredentialsFileKey] = config[repoconfig.CredentialsFileKey]\n\t\t}\n\tcase repoconfig.GCPBackend:\n\t\tresult[udmrepo.StoreOptionCredentialFile] = getGCPCredentials(config)\n\t}\n\n\treturn result, nil\n}\n\n// Translates user specified options (backupRepoConfig) to internal parameters\n// so we would accept only the options that are well defined in the internal system.\n// Users' inputs should not be treated as safe any time.\n// We remove the unnecessary parameters and keep the modules/logics below safe\nfunc getStorageVariables(backupLocation *velerov1api.BackupStorageLocation, repoBackend string, repoName string, backupRepoConfig map[string]string, credGetter credentials.CredentialGetter) (map[string]string, error) {\n\tresult := make(map[string]string)\n\n\tbackendType := repoconfig.GetBackendType(backupLocation.Spec.Provider, backupLocation.Spec.Config)\n\tif !repoconfig.IsBackendTypeValid(backendType) {\n\t\treturn map[string]string{}, errors.New(\"invalid storage provider\")\n\t}\n\n\tconfig := backupLocation.Spec.Config\n\tif config == nil {\n\t\tconfig = map[string]string{}\n\t}\n\n\tbucket := strings.Trim(config[\"bucket\"], \"/\")\n\tprefix := strings.Trim(config[\"prefix\"], \"/\")\n\tif backupLocation.Spec.ObjectStorage != nil {\n\t\tbucket = strings.Trim(backupLocation.Spec.ObjectStorage.Bucket, \"/\")\n\t\tprefix = strings.Trim(backupLocation.Spec.ObjectStorage.Prefix, \"/\")\n\t}\n\n\tprefix = path.Join(prefix, repoBackend, repoName) + \"/\"\n\n\tregion := config[\"region\"]\n\n\tif backendType == repoconfig.AWSBackend {\n\t\ts3URL := config[\"s3Url\"]\n\t\tdisableTLS := false\n\n\t\tvar err error\n\t\tif s3URL == \"\" {\n\t\t\tif region == \"\" {\n\t\t\t\tregion, err = getS3BucketRegion(bucket, config)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn map[string]string{}, errors.Wrap(err, \"error get s3 bucket region\")\n\t\t\t\t}\n\t\t\t}\n\n\t\t\ts3URL = fmt.Sprintf(\"s3-%s.amazonaws.com\", region)\n\t\t\tdisableTLS = false\n\t\t} else {\n\t\t\turl, err := url.Parse(s3URL)\n\t\t\tif err != nil {\n\t\t\t\treturn map[string]string{}, errors.Wrapf(err, \"error to parse s3Url %s\", s3URL)\n\t\t\t}\n\n\t\t\tif url.Path != \"\" && url.Path != \"/\" {\n\t\t\t\treturn map[string]string{}, errors.Errorf(\"path is not expected in s3Url %s\", s3URL)\n\t\t\t}\n\n\t\t\ts3URL = url.Host\n\t\t\tdisableTLS = url.Scheme == \"http\"\n\t\t}\n\n\t\tresult[udmrepo.StoreOptionS3Endpoint] = strings.Trim(s3URL, \"/\")\n\t\tresult[udmrepo.StoreOptionS3DisableTLSVerify] = config[\"insecureSkipTLSVerify\"]\n\t\tresult[udmrepo.StoreOptionS3DisableTLS] = strconv.FormatBool(disableTLS)\n\t} else if backendType == repoconfig.AzureBackend {\n\t\tfor k, v := range config {\n\t\t\tresult[k] = v\n\t\t}\n\t}\n\n\tresult[udmrepo.StoreOptionOssBucket] = bucket\n\tresult[udmrepo.StoreOptionPrefix] = prefix\n\tif backupLocation.Spec.ObjectStorage != nil {\n\t\tvar caCertData []byte\n\n\t\t// Try CACertRef first (new method), then fall back to CACert (deprecated)\n\t\tif backupLocation.Spec.ObjectStorage.CACertRef != nil {\n\t\t\tcaCertString, err := credGetter.FromSecret.Get(backupLocation.Spec.ObjectStorage.CACertRef)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, errors.Wrap(err, \"error getting CA certificate from secret\")\n\t\t\t}\n\t\t\tcaCertData = []byte(caCertString)\n\t\t} else if backupLocation.Spec.ObjectStorage.CACert != nil {\n\t\t\tcaCertData = backupLocation.Spec.ObjectStorage.CACert\n\t\t}\n\n\t\tif caCertData != nil {\n\t\t\tresult[udmrepo.StoreOptionCACert] = base64.StdEncoding.EncodeToString(caCertData)\n\t\t}\n\t}\n\tresult[udmrepo.StoreOptionOssRegion] = strings.Trim(region, \"/\")\n\tresult[udmrepo.StoreOptionFsPath] = config[\"fspath\"]\n\n\t// We remove the unnecessary parameters and keep the modules/logics below safe\n\tif backupRepoConfig != nil {\n\t\t// range of valid params to keep, everything else will be discarded.\n\t\tvalidParams := []string{\n\t\t\tudmrepo.StoreOptionCacheLimit,\n\t\t\tudmrepo.StoreOptionKeyFullMaintenanceInterval,\n\t\t}\n\t\tfor _, param := range validParams {\n\t\t\tif v, found := backupRepoConfig[param]; found {\n\t\t\t\tresult[param] = v\n\t\t\t}\n\t\t}\n\t}\n\n\treturn result, nil\n}\n\nfunc createRepoService(repoBackend string, log logrus.FieldLogger) udmrepo.BackupRepoService {\n\treturn reposervice.Create(repoBackend, log)\n}\n" }, { "path": "pkg/repository/provider/unified_repo_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage provider\n\nimport (\n\t\"context\"\n\t\"encoding/base64\"\n\t\"errors\"\n\t\"testing\"\n\n\t\"github.com/aws/aws-sdk-go-v2/aws\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tvelerocredentials \"github.com/vmware-tanzu/velero/internal/credentials\"\n\tcredmock \"github.com/vmware-tanzu/velero/internal/credentials/mocks\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n\treposervicenmocks \"github.com/vmware-tanzu/velero/pkg/repository/udmrepo/mocks\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestGetStorageCredentials(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tbackupLocation velerov1api.BackupStorageLocation\n\t\tcredFileStore *credmock.FileStore\n\t\tcredStoreError error\n\t\tcredStorePath string\n\t\tgetS3Credentials func(map[string]string) (*aws.Credentials, error)\n\t\tgetGCPCredentials func(map[string]string) string\n\t\texpected map[string]string\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"invalid credentials file store interface\",\n\t\t\texpected: map[string]string{},\n\t\t\texpectedErr: \"invalid credentials interface\",\n\t\t},\n\t\t{\n\t\t\tname: \"invalid provider\",\n\t\t\tbackupLocation: velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"invalid-provider\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tcredFileStore: new(credmock.FileStore),\n\t\t\texpected: map[string]string{},\n\t\t\texpectedErr: \"invalid storage provider\",\n\t\t},\n\t\t{\n\t\t\tname: \"credential section exists in BSL, file store fail\",\n\t\t\tbackupLocation: velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"aws\",\n\t\t\t\t\tCredential: &corev1api.SecretKeySelector{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tcredFileStore: new(credmock.FileStore),\n\t\t\tcredStoreError: errors.New(\"fake error\"),\n\t\t\texpected: map[string]string{},\n\t\t\texpectedErr: \"error get credential file in bsl: fake error\",\n\t\t},\n\t\t{\n\t\t\tname: \"aws, Credential section not exists in BSL\",\n\t\t\tbackupLocation: velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"velero.io/aws\",\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"credentialsFile\": \"credentials-from-config-map\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tgetS3Credentials: func(config map[string]string) (*aws.Credentials, error) {\n\t\t\t\treturn &aws.Credentials{\n\t\t\t\t\tAccessKeyID: \"from: \" + config[\"credentialsFile\"],\n\t\t\t\t}, nil\n\t\t\t},\n\t\t\tcredFileStore: new(credmock.FileStore),\n\t\t\texpected: map[string]string{\n\t\t\t\t\"accessKeyID\": \"from: credentials-from-config-map\",\n\t\t\t\t\"providerName\": \"\",\n\t\t\t\t\"secretAccessKey\": \"\",\n\t\t\t\t\"sessionToken\": \"\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"aws, Credential section exists in BSL\",\n\t\t\tbackupLocation: velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"velero.io/aws\",\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"credentialsFile\": \"credentials-from-config-map\",\n\t\t\t\t\t},\n\t\t\t\t\tCredential: &corev1api.SecretKeySelector{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tcredFileStore: new(credmock.FileStore),\n\t\t\tcredStorePath: \"credentials-from-credential-key\",\n\t\t\tgetS3Credentials: func(config map[string]string) (*aws.Credentials, error) {\n\t\t\t\treturn &aws.Credentials{\n\t\t\t\t\tAccessKeyID: \"from: \" + config[\"credentialsFile\"],\n\t\t\t\t}, nil\n\t\t\t},\n\n\t\t\texpected: map[string]string{\n\t\t\t\t\"accessKeyID\": \"from: credentials-from-credential-key\",\n\t\t\t\t\"providerName\": \"\",\n\t\t\t\t\"secretAccessKey\": \"\",\n\t\t\t\t\"sessionToken\": \"\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"aws, get credentials fail\",\n\t\t\tbackupLocation: velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"velero.io/aws\",\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"credentialsFile\": \"credentials-from-config-map\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tgetS3Credentials: func(config map[string]string) (*aws.Credentials, error) {\n\t\t\t\treturn nil, errors.New(\"fake error\")\n\t\t\t},\n\t\t\tcredFileStore: new(credmock.FileStore),\n\t\t\texpected: map[string]string{},\n\t\t\texpectedErr: \"error get s3 credentials: fake error\",\n\t\t},\n\t\t{\n\t\t\tname: \"aws, credential file not exist\",\n\t\t\tbackupLocation: velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"velero.io/aws\",\n\t\t\t\t\tConfig: map[string]string{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tgetS3Credentials: func(config map[string]string) (*aws.Credentials, error) {\n\t\t\t\treturn nil, nil\n\t\t\t},\n\t\t\tcredFileStore: new(credmock.FileStore),\n\t\t\texpected: map[string]string{},\n\t\t},\n\t\t{\n\t\t\tname: \"azure\",\n\t\t\tbackupLocation: velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"velero.io/azure\",\n\t\t\t\t\tCredential: &corev1api.SecretKeySelector{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tcredFileStore: new(credmock.FileStore),\n\t\t\texpected: map[string]string{},\n\t\t},\n\t\t{\n\t\t\tname: \"gcp, Credential section not exists in BSL\",\n\t\t\tbackupLocation: velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"velero.io/gcp\",\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"credentialsFile\": \"credentials-from-config-map\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tgetGCPCredentials: func(config map[string]string) string {\n\t\t\t\treturn \"credentials-from-config-map\"\n\t\t\t},\n\t\t\tcredFileStore: new(credmock.FileStore),\n\t\t\texpected: map[string]string{\n\t\t\t\t\"credFile\": \"credentials-from-config-map\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tgetS3Credentials = tc.getS3Credentials\n\t\t\tgetGCPCredentials = tc.getGCPCredentials\n\n\t\t\tvar fileStore velerocredentials.FileStore\n\t\t\tif tc.credFileStore != nil {\n\t\t\t\ttc.credFileStore.On(\"Path\", mock.Anything, mock.Anything).Return(tc.credStorePath, tc.credStoreError)\n\t\t\t\tfileStore = tc.credFileStore\n\t\t\t}\n\n\t\t\tactual, err := getStorageCredentials(&tc.backupLocation, fileStore)\n\n\t\t\trequire.Equal(t, tc.expected, actual)\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetStorageVariables(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tbackupLocation velerov1api.BackupStorageLocation\n\t\tcredFileStore *credmock.FileStore\n\t\trepoName string\n\t\trepoBackend string\n\t\trepoConfig map[string]string\n\t\tgetS3BucketRegion func(bucket string, config map[string]string) (string, error)\n\t\texpected map[string]string\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"invalid provider\",\n\t\t\tbackupLocation: velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"invalid-provider\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: map[string]string{},\n\t\t\texpectedErr: \"invalid storage provider\",\n\t\t},\n\t\t{\n\t\t\tname: \"aws, ObjectStorage section not exists in BSL, s3Url exist, https\",\n\t\t\tbackupLocation: velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"velero.io/aws\",\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"bucket\": \"fake-bucket\",\n\t\t\t\t\t\t\"prefix\": \"fake-prefix\",\n\t\t\t\t\t\t\"region\": \"fake-region/\",\n\t\t\t\t\t\t\"s3Url\": \"https://fake-url/\",\n\t\t\t\t\t\t\"insecureSkipTLSVerify\": \"true\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoBackend: \"fake-repo-type\",\n\t\t\texpected: map[string]string{\n\t\t\t\t\"bucket\": \"fake-bucket\",\n\t\t\t\t\"prefix\": \"fake-prefix/fake-repo-type/\",\n\t\t\t\t\"region\": \"fake-region\",\n\t\t\t\t\"fspath\": \"\",\n\t\t\t\t\"endpoint\": \"fake-url\",\n\t\t\t\t\"doNotUseTLS\": \"false\",\n\t\t\t\t\"skipTLSVerify\": \"true\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"aws, ObjectStorage section not exists in BSL, s3Url exist, invalid\",\n\t\t\tbackupLocation: velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"velero.io/aws\",\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"bucket\": \"fake-bucket\",\n\t\t\t\t\t\t\"prefix\": \"fake-prefix\",\n\t\t\t\t\t\t\"region\": \"fake-region/\",\n\t\t\t\t\t\t\"s3Url\": \"https://fake-url/fake-path\",\n\t\t\t\t\t\t\"insecureSkipTLSVerify\": \"true\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoBackend: \"fake-repo-type\",\n\t\t\texpected: map[string]string{},\n\t\t\texpectedErr: \"path is not expected in s3Url https://fake-url/fake-path\",\n\t\t},\n\t\t{\n\t\t\tname: \"aws, ObjectStorage section not exists in BSL, s3Url not exist\",\n\t\t\tbackupLocation: velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"velero.io/aws\",\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"bucket\": \"fake-bucket\",\n\t\t\t\t\t\t\"prefix\": \"fake-prefix\",\n\t\t\t\t\t\t\"insecureSkipTLSVerify\": \"false\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tgetS3BucketRegion: func(bucket string, config map[string]string) (string, error) {\n\t\t\t\treturn \"region from bucket: \" + bucket, nil\n\t\t\t},\n\t\t\trepoBackend: \"fake-repo-type\",\n\t\t\texpected: map[string]string{\n\t\t\t\t\"bucket\": \"fake-bucket\",\n\t\t\t\t\"prefix\": \"fake-prefix/fake-repo-type/\",\n\t\t\t\t\"region\": \"region from bucket: fake-bucket\",\n\t\t\t\t\"fspath\": \"\",\n\t\t\t\t\"endpoint\": \"s3-region from bucket: fake-bucket.amazonaws.com\",\n\t\t\t\t\"doNotUseTLS\": \"false\",\n\t\t\t\t\"skipTLSVerify\": \"false\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"aws, ObjectStorage section not exists in BSL, s3Url not exist, get region fail\",\n\t\t\tbackupLocation: velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"velero.io/aws\",\n\t\t\t\t\tConfig: map[string]string{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tgetS3BucketRegion: func(bucket string, config map[string]string) (string, error) {\n\t\t\t\treturn \"\", errors.New(\"fake error\")\n\t\t\t},\n\t\t\texpected: map[string]string{},\n\t\t\texpectedErr: \"error get s3 bucket region: fake error\",\n\t\t},\n\t\t{\n\t\t\tname: \"aws, ObjectStorage section exists in BSL, s3Url exist, http\",\n\t\t\tbackupLocation: velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"velero.io/aws\",\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"bucket\": \"fake-bucket-config\",\n\t\t\t\t\t\t\"prefix\": \"fake-prefix-config\",\n\t\t\t\t\t\t\"region\": \"fake-region\",\n\t\t\t\t\t\t\"s3Url\": \"http://fake-url/\",\n\t\t\t\t\t\t\"insecureSkipTLSVerify\": \"false\",\n\t\t\t\t\t},\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"fake-bucket-object-store\",\n\t\t\t\t\t\t\tPrefix: \"fake-prefix-object-store\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tgetS3BucketRegion: func(bucket string, config map[string]string) (string, error) {\n\t\t\t\treturn \"region from bucket: \" + bucket, nil\n\t\t\t},\n\t\t\trepoBackend: \"fake-repo-type\",\n\t\t\texpected: map[string]string{\n\t\t\t\t\"bucket\": \"fake-bucket-object-store\",\n\t\t\t\t\"prefix\": \"fake-prefix-object-store/fake-repo-type/\",\n\t\t\t\t\"region\": \"fake-region\",\n\t\t\t\t\"fspath\": \"\",\n\t\t\t\t\"endpoint\": \"fake-url\",\n\t\t\t\t\"doNotUseTLS\": \"true\",\n\t\t\t\t\"skipTLSVerify\": \"false\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"aws, ObjectStorage section exists in BSL, s3Url exist, https, custom CA exist\",\n\t\t\tbackupLocation: velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"velero.io/aws\",\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"bucket\": \"fake-bucket-config\",\n\t\t\t\t\t\t\"prefix\": \"fake-prefix-config\",\n\t\t\t\t\t\t\"region\": \"fake-region\",\n\t\t\t\t\t\t\"s3Url\": \"https://fake-url/\",\n\t\t\t\t\t\t\"insecureSkipTLSVerify\": \"false\",\n\t\t\t\t\t},\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"fake-bucket-object-store\",\n\t\t\t\t\t\t\tPrefix: \"fake-prefix-object-store\",\n\t\t\t\t\t\t\tCACert: []byte{0x01, 0x02, 0x03, 0x04, 0x05},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tgetS3BucketRegion: func(bucket string, config map[string]string) (string, error) {\n\t\t\t\treturn \"region from bucket: \" + bucket, nil\n\t\t\t},\n\t\t\trepoBackend: \"fake-repo-type\",\n\t\t\texpected: map[string]string{\n\t\t\t\t\"bucket\": \"fake-bucket-object-store\",\n\t\t\t\t\"prefix\": \"fake-prefix-object-store/fake-repo-type/\",\n\t\t\t\t\"region\": \"fake-region\",\n\t\t\t\t\"fspath\": \"\",\n\t\t\t\t\"endpoint\": \"fake-url\",\n\t\t\t\t\"doNotUseTLS\": \"false\",\n\t\t\t\t\"skipTLSVerify\": \"false\",\n\t\t\t\t\"caCert\": base64.StdEncoding.EncodeToString([]byte{0x01, 0x02, 0x03, 0x04, 0x05}),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"azure\",\n\t\t\tbackupLocation: velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"velero.io/azure\",\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"bucket\": \"fake-bucket-config\",\n\t\t\t\t\t\t\"prefix\": \"fake-prefix-config\",\n\t\t\t\t\t\t\"region\": \"fake-region\",\n\t\t\t\t\t\t\"fspath\": \"\",\n\t\t\t\t\t},\n\t\t\t\t\tStorageType: velerov1api.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1api.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"fake-bucket-object-store\",\n\t\t\t\t\t\t\tPrefix: \"fake-prefix-object-store\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tcredFileStore: new(credmock.FileStore),\n\t\t\trepoBackend: \"fake-repo-type\",\n\t\t\texpected: map[string]string{\n\t\t\t\t\"bucket\": \"fake-bucket-object-store\",\n\t\t\t\t\"prefix\": \"fake-prefix-object-store/fake-repo-type/\",\n\t\t\t\t\"region\": \"fake-region\",\n\t\t\t\t\"fspath\": \"\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"fs\",\n\t\t\tbackupLocation: velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"velero.io/fs\",\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"fspath\": \"fake-path\",\n\t\t\t\t\t\t\"prefix\": \"fake-prefix\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoBackend: \"fake-repo-type\",\n\t\t\texpected: map[string]string{\n\t\t\t\t\"fspath\": \"fake-path\",\n\t\t\t\t\"bucket\": \"\",\n\t\t\t\t\"prefix\": \"fake-prefix/fake-repo-type/\",\n\t\t\t\t\"region\": \"\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"fs with repo config\",\n\t\t\tbackupLocation: velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"velero.io/fs\",\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"fspath\": \"fake-path\",\n\t\t\t\t\t\t\"prefix\": \"fake-prefix\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoBackend: \"fake-repo-type\",\n\t\t\trepoConfig: map[string]string{\n\t\t\t\tudmrepo.StoreOptionCacheLimit: \"1000\",\n\t\t\t},\n\t\t\texpected: map[string]string{\n\t\t\t\t\"fspath\": \"fake-path\",\n\t\t\t\t\"bucket\": \"\",\n\t\t\t\t\"prefix\": \"fake-prefix/fake-repo-type/\",\n\t\t\t\t\"region\": \"\",\n\t\t\t\t\"cacheLimitMB\": \"1000\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tgetS3BucketRegion = tc.getS3BucketRegion\n\n\t\t\tactual, err := getStorageVariables(&tc.backupLocation, tc.repoBackend, tc.repoName, tc.repoConfig, velerocredentials.CredentialGetter{})\n\n\t\t\trequire.Equal(t, tc.expected, actual)\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetRepoPassword(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tgetter *credmock.SecretStore\n\t\tcredStoreReturn string\n\t\tcredStoreError error\n\t\tcached string\n\t\texpected string\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"invalid secret interface\",\n\t\t\texpectedErr: \"invalid credentials interface\",\n\t\t},\n\t\t{\n\t\t\tname: \"error from secret interface\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreError: errors.New(\"fake error\"),\n\t\t\texpectedErr: \"error to get password: fake error\",\n\t\t},\n\t\t{\n\t\t\tname: \"secret with whitespace\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \" fake-passwor d \",\n\t\t\texpected: \"fake-passwor d\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar secretStore velerocredentials.SecretStore\n\t\t\tif tc.getter != nil {\n\t\t\t\ttc.getter.On(\"Get\", mock.Anything, mock.Anything).Return(tc.credStoreReturn, tc.credStoreError)\n\t\t\t\tsecretStore = tc.getter\n\t\t\t}\n\n\t\t\turp := unifiedRepoProvider{\n\t\t\t\tcredentialGetter: velerocredentials.CredentialGetter{\n\t\t\t\t\tFromSecret: secretStore,\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tpassword, err := getRepoPassword(urp.credentialGetter.FromSecret)\n\n\t\t\trequire.Equal(t, tc.expected, password)\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetStoreOptions(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tfuncTable localFuncTable\n\t\trepoParam any\n\t\texpected map[string]string\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"wrong param type\",\n\t\t\trepoParam: struct{}{},\n\t\t\texpected: map[string]string{},\n\t\t\texpectedErr: \"invalid parameter, expect provider.RepoParam, actual struct {}\",\n\t\t},\n\t\t{\n\t\t\tname: \"get storage variable fail\",\n\t\t\trepoParam: RepoParam{\n\t\t\t\tBackupLocation: &velerov1api.BackupStorageLocation{},\n\t\t\t\tBackupRepo: &velerov1api.BackupRepository{},\n\t\t\t},\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, errors.New(\"fake-error-2\")\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: map[string]string{},\n\t\t\texpectedErr: \"error to get storage variables: fake-error-2\",\n\t\t},\n\t\t{\n\t\t\tname: \"get storage credentials fail\",\n\t\t\trepoParam: RepoParam{\n\t\t\t\tBackupLocation: &velerov1api.BackupStorageLocation{},\n\t\t\t\tBackupRepo: &velerov1api.BackupRepository{},\n\t\t\t},\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, errors.New(\"fake-error-3\")\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: map[string]string{},\n\t\t\texpectedErr: \"error to get repo credentials: fake-error-3\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfuncTable = tc.funcTable\n\n\t\t\turp := unifiedRepoProvider{}\n\n\t\t\toptions, err := urp.GetStoreOptions(tc.repoParam)\n\n\t\t\trequire.Equal(t, tc.expected, options)\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestPrepareRepo(t *testing.T) {\n\tbsl := velerov1api.BackupStorageLocation{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-bsl\",\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t},\n\t}\n\n\ttestCases := []struct {\n\t\tname string\n\t\tfuncTable localFuncTable\n\t\tgetter *credmock.SecretStore\n\t\trepoService *reposervicenmocks.BackupRepoService\n\t\tretFuncCreate func(context.Context, udmrepo.RepoOptions) error\n\t\tretFuncCheck func(context.Context, udmrepo.RepoOptions) (bool, error)\n\t\tcredStoreReturn string\n\t\tcredStoreError error\n\t\treadOnlyBSL bool\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"get repo option fail\",\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\texpectedErr: \"error to get repo options: error to get repo password: invalid credentials interface\",\n\t\t},\n\t\t{\n\t\t\tname: \"get repo option fail, get password fail\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tcredStoreError: errors.New(\"fake-password-error\"),\n\t\t\texpectedErr: \"error to get repo options: error to get repo password: error to get password: fake-password-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"get repo option fail, get store options fail\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, errors.New(\"fake-store-option-error\")\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedErr: \"error to get repo options: error to get storage variables: fake-store-option-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"check error\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tretFuncCheck: func(ctx context.Context, repoOption udmrepo.RepoOptions) (bool, error) {\n\t\t\t\treturn false, errors.New(\"fake-error\")\n\t\t\t},\n\t\t\texpectedErr: \"error to check backup repo: fake-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"already initialized\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tretFuncCheck: func(ctx context.Context, repoOption udmrepo.RepoOptions) (bool, error) {\n\t\t\t\treturn true, nil\n\t\t\t},\n\t\t\tretFuncCreate: func(ctx context.Context, repoOption udmrepo.RepoOptions) error {\n\t\t\t\treturn errors.New(\"fake-error\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"bsl is readonly\",\n\t\t\treadOnlyBSL: true,\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tretFuncCheck: func(ctx context.Context, repoOption udmrepo.RepoOptions) (bool, error) {\n\t\t\t\treturn false, nil\n\t\t\t},\n\t\t\tretFuncCreate: func(ctx context.Context, repoOption udmrepo.RepoOptions) error {\n\t\t\t\treturn errors.New(\"fake-error-2\")\n\t\t\t},\n\t\t\texpectedErr: \"cannot create new backup repo for read-only backup storage location velero/fake-bsl\",\n\t\t},\n\t\t{\n\t\t\tname: \"create fail\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tretFuncCheck: func(ctx context.Context, repoOption udmrepo.RepoOptions) (bool, error) {\n\t\t\t\treturn false, nil\n\t\t\t},\n\t\t\tretFuncCreate: func(ctx context.Context, repoOption udmrepo.RepoOptions) error {\n\t\t\t\treturn errors.New(\"fake-error-1\")\n\t\t\t},\n\t\t\texpectedErr: \"error to create backup repo: fake-error-1\",\n\t\t},\n\t\t{\n\t\t\tname: \"initialize error\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tretFuncCheck: func(ctx context.Context, repoOption udmrepo.RepoOptions) (bool, error) {\n\t\t\t\treturn false, nil\n\t\t\t},\n\t\t\tretFuncCreate: func(ctx context.Context, repoOption udmrepo.RepoOptions) error {\n\t\t\t\treturn errors.New(\"fake-error-2\")\n\t\t\t},\n\t\t\texpectedErr: \"error to create backup repo: fake-error-2\",\n\t\t},\n\t\t{\n\t\t\tname: \"initialize succeed\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tretFuncCheck: func(ctx context.Context, repoOption udmrepo.RepoOptions) (bool, error) {\n\t\t\t\treturn false, nil\n\t\t\t},\n\t\t\tretFuncCreate: func(ctx context.Context, repoOption udmrepo.RepoOptions) error {\n\t\t\t\treturn nil\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfuncTable = tc.funcTable\n\n\t\t\tvar secretStore velerocredentials.SecretStore\n\t\t\tif tc.getter != nil {\n\t\t\t\ttc.getter.On(\"Get\", mock.Anything, mock.Anything).Return(tc.credStoreReturn, tc.credStoreError)\n\t\t\t\tsecretStore = tc.getter\n\t\t\t}\n\n\t\t\turp := unifiedRepoProvider{\n\t\t\t\tcredentialGetter: velerocredentials.CredentialGetter{\n\t\t\t\t\tFromSecret: secretStore,\n\t\t\t\t},\n\t\t\t\trepoService: tc.repoService,\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\ttc.repoService.On(\"IsCreated\", mock.Anything, mock.Anything).Return(tc.retFuncCheck)\n\t\t\ttc.repoService.On(\"Create\", mock.Anything, mock.Anything, mock.Anything).Return(tc.retFuncCreate)\n\n\t\t\tif tc.readOnlyBSL {\n\t\t\t\tbsl.Spec.AccessMode = velerov1api.BackupStorageLocationAccessModeReadOnly\n\t\t\t} else {\n\t\t\t\tbsl.Spec.AccessMode = velerov1api.BackupStorageLocationAccessModeReadWrite\n\t\t\t}\n\n\t\t\terr := urp.PrepareRepo(t.Context(), RepoParam{\n\t\t\t\tBackupLocation: &bsl,\n\t\t\t\tBackupRepo: &velerov1api.BackupRepository{},\n\t\t\t})\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestForget(t *testing.T) {\n\tvar backupRepo *reposervicenmocks.BackupRepo\n\n\ttestCases := []struct {\n\t\tname string\n\t\tfuncTable localFuncTable\n\t\tgetter *credmock.SecretStore\n\t\trepoService *reposervicenmocks.BackupRepoService\n\t\tbackupRepo *reposervicenmocks.BackupRepo\n\t\tretFuncOpen []any\n\t\tretFuncDelete any\n\t\tretFuncFlush any\n\t\tcredStoreReturn string\n\t\tcredStoreError error\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"get repo option fail\",\n\t\t\texpectedErr: \"error to get repo options: error to get repo password: invalid credentials interface\",\n\t\t},\n\t\t{\n\t\t\tname: \"repo open fail\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tretFuncOpen: []any{\n\t\t\t\tfunc(context.Context, udmrepo.RepoOptions) udmrepo.BackupRepo {\n\t\t\t\t\treturn backupRepo\n\t\t\t\t},\n\n\t\t\t\tfunc(context.Context, udmrepo.RepoOptions) error {\n\t\t\t\t\treturn errors.New(\"fake-error-2\")\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedErr: \"error to open backup repo: fake-error-2\",\n\t\t},\n\t\t{\n\t\t\tname: \"delete fail\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tbackupRepo: new(reposervicenmocks.BackupRepo),\n\t\t\tretFuncOpen: []any{\n\t\t\t\tfunc(context.Context, udmrepo.RepoOptions) udmrepo.BackupRepo {\n\t\t\t\t\treturn backupRepo\n\t\t\t\t},\n\n\t\t\t\tfunc(context.Context, udmrepo.RepoOptions) error {\n\t\t\t\t\treturn nil\n\t\t\t\t},\n\t\t\t},\n\t\t\tretFuncDelete: func(context.Context, udmrepo.ID) error {\n\t\t\t\treturn errors.New(\"fake-error-3\")\n\t\t\t},\n\t\t\texpectedErr: \"error to delete manifest: fake-error-3\",\n\t\t},\n\t\t{\n\t\t\tname: \"flush fail\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tbackupRepo: new(reposervicenmocks.BackupRepo),\n\t\t\tretFuncOpen: []any{\n\t\t\t\tfunc(context.Context, udmrepo.RepoOptions) udmrepo.BackupRepo {\n\t\t\t\t\treturn backupRepo\n\t\t\t\t},\n\n\t\t\t\tfunc(context.Context, udmrepo.RepoOptions) error {\n\t\t\t\t\treturn nil\n\t\t\t\t},\n\t\t\t},\n\t\t\tretFuncDelete: func(context.Context, udmrepo.ID) error {\n\t\t\t\treturn nil\n\t\t\t},\n\t\t\tretFuncFlush: func(context.Context) error {\n\t\t\t\treturn errors.New(\"fake-error-4\")\n\t\t\t},\n\t\t\texpectedErr: \"error to flush repo: fake-error-4\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfuncTable = tc.funcTable\n\n\t\t\tvar secretStore velerocredentials.SecretStore\n\t\t\tif tc.getter != nil {\n\t\t\t\ttc.getter.On(\"Get\", mock.Anything, mock.Anything).Return(tc.credStoreReturn, tc.credStoreError)\n\t\t\t\tsecretStore = tc.getter\n\t\t\t}\n\n\t\t\turp := unifiedRepoProvider{\n\t\t\t\tcredentialGetter: velerocredentials.CredentialGetter{\n\t\t\t\t\tFromSecret: secretStore,\n\t\t\t\t},\n\t\t\t\trepoService: tc.repoService,\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tbackupRepo = tc.backupRepo\n\n\t\t\tif tc.repoService != nil {\n\t\t\t\ttc.repoService.On(\"Open\", mock.Anything, mock.Anything).Return(tc.retFuncOpen[0], tc.retFuncOpen[1])\n\t\t\t}\n\n\t\t\tif tc.backupRepo != nil {\n\t\t\t\tbackupRepo.On(\"DeleteManifest\", mock.Anything, mock.Anything).Return(tc.retFuncDelete)\n\t\t\t\tbackupRepo.On(\"Flush\", mock.Anything).Return(tc.retFuncFlush)\n\t\t\t\tbackupRepo.On(\"Close\", mock.Anything).Return(nil)\n\t\t\t}\n\n\t\t\terr := urp.Forget(t.Context(), \"\", RepoParam{\n\t\t\t\tBackupLocation: &velerov1api.BackupStorageLocation{},\n\t\t\t\tBackupRepo: &velerov1api.BackupRepository{},\n\t\t\t})\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestBatchForget(t *testing.T) {\n\tvar backupRepo *reposervicenmocks.BackupRepo\n\n\ttestCases := []struct {\n\t\tname string\n\t\tfuncTable localFuncTable\n\t\tgetter *credmock.SecretStore\n\t\trepoService *reposervicenmocks.BackupRepoService\n\t\tbackupRepo *reposervicenmocks.BackupRepo\n\t\tretFuncOpen []any\n\t\tretFuncDelete any\n\t\tretFuncFlush any\n\t\tcredStoreReturn string\n\t\tcredStoreError error\n\t\tsnapshots []string\n\t\texpectedErr []string\n\t}{\n\t\t{\n\t\t\tname: \"get repo option fail\",\n\t\t\texpectedErr: []string{\"error to get repo options: error to get repo password: invalid credentials interface\"},\n\t\t},\n\t\t{\n\t\t\tname: \"repo open fail\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tretFuncOpen: []any{\n\t\t\t\tfunc(context.Context, udmrepo.RepoOptions) udmrepo.BackupRepo {\n\t\t\t\t\treturn backupRepo\n\t\t\t\t},\n\n\t\t\t\tfunc(context.Context, udmrepo.RepoOptions) error {\n\t\t\t\t\treturn errors.New(\"fake-error-2\")\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedErr: []string{\"error to open backup repo: fake-error-2\"},\n\t\t},\n\t\t{\n\t\t\tname: \"delete fail\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tbackupRepo: new(reposervicenmocks.BackupRepo),\n\t\t\tretFuncOpen: []any{\n\t\t\t\tfunc(context.Context, udmrepo.RepoOptions) udmrepo.BackupRepo {\n\t\t\t\t\treturn backupRepo\n\t\t\t\t},\n\n\t\t\t\tfunc(context.Context, udmrepo.RepoOptions) error {\n\t\t\t\t\treturn nil\n\t\t\t\t},\n\t\t\t},\n\t\t\tretFuncDelete: func(context.Context, udmrepo.ID) error {\n\t\t\t\treturn errors.New(\"fake-error-3\")\n\t\t\t},\n\t\t\tsnapshots: []string{\"snapshot-1\", \"snapshot-2\"},\n\t\t\texpectedErr: []string{\"error to delete manifest snapshot-1: fake-error-3\", \"error to delete manifest snapshot-2: fake-error-3\"},\n\t\t},\n\t\t{\n\t\t\tname: \"flush fail\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tbackupRepo: new(reposervicenmocks.BackupRepo),\n\t\t\tretFuncOpen: []any{\n\t\t\t\tfunc(context.Context, udmrepo.RepoOptions) udmrepo.BackupRepo {\n\t\t\t\t\treturn backupRepo\n\t\t\t\t},\n\n\t\t\t\tfunc(context.Context, udmrepo.RepoOptions) error {\n\t\t\t\t\treturn nil\n\t\t\t\t},\n\t\t\t},\n\t\t\tretFuncDelete: func(context.Context, udmrepo.ID) error {\n\t\t\t\treturn nil\n\t\t\t},\n\t\t\tretFuncFlush: func(context.Context) error {\n\t\t\t\treturn errors.New(\"fake-error-4\")\n\t\t\t},\n\t\t\texpectedErr: []string{\"error to flush repo: fake-error-4\"},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfuncTable = tc.funcTable\n\n\t\t\tvar secretStore velerocredentials.SecretStore\n\t\t\tif tc.getter != nil {\n\t\t\t\ttc.getter.On(\"Get\", mock.Anything, mock.Anything).Return(tc.credStoreReturn, tc.credStoreError)\n\t\t\t\tsecretStore = tc.getter\n\t\t\t}\n\n\t\t\turp := unifiedRepoProvider{\n\t\t\t\tcredentialGetter: velerocredentials.CredentialGetter{\n\t\t\t\t\tFromSecret: secretStore,\n\t\t\t\t},\n\t\t\t\trepoService: tc.repoService,\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tbackupRepo = tc.backupRepo\n\n\t\t\tif tc.repoService != nil {\n\t\t\t\ttc.repoService.On(\"Open\", mock.Anything, mock.Anything).Return(tc.retFuncOpen[0], tc.retFuncOpen[1])\n\t\t\t}\n\n\t\t\tif tc.backupRepo != nil {\n\t\t\t\tbackupRepo.On(\"DeleteManifest\", mock.Anything, mock.Anything).Return(tc.retFuncDelete)\n\t\t\t\tbackupRepo.On(\"Flush\", mock.Anything).Return(tc.retFuncFlush)\n\t\t\t\tbackupRepo.On(\"Close\", mock.Anything).Return(nil)\n\t\t\t}\n\n\t\t\terrs := urp.BatchForget(t.Context(), tc.snapshots, RepoParam{\n\t\t\t\tBackupLocation: &velerov1api.BackupStorageLocation{},\n\t\t\t\tBackupRepo: &velerov1api.BackupRepository{},\n\t\t\t})\n\n\t\t\tif tc.expectedErr == nil {\n\t\t\t\tassert.Empty(t, errs)\n\t\t\t} else {\n\t\t\t\tassert.Len(t, errs, len(tc.expectedErr))\n\n\t\t\t\tfor i := range tc.expectedErr {\n\t\t\t\t\tassert.EqualError(t, errs[i], tc.expectedErr[i])\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestInitRepo(t *testing.T) {\n\tbsl := velerov1api.BackupStorageLocation{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-bsl\",\n\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t},\n\t}\n\n\ttestCases := []struct {\n\t\tname string\n\t\tfuncTable localFuncTable\n\t\tgetter *credmock.SecretStore\n\t\trepoService *reposervicenmocks.BackupRepoService\n\t\tretFuncInit any\n\t\tcredStoreReturn string\n\t\tcredStoreError error\n\t\treadOnlyBSL bool\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"bsl is readonly\",\n\t\t\treadOnlyBSL: true,\n\t\t\texpectedErr: \"cannot create new backup repo for read-only backup storage location velero/fake-bsl\",\n\t\t},\n\t\t{\n\t\t\tname: \"get repo option fail\",\n\t\t\texpectedErr: \"error to get repo options: error to get repo password: invalid credentials interface\",\n\t\t},\n\t\t{\n\t\t\tname: \"repo init fail\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tretFuncInit: func(context.Context, udmrepo.RepoOptions) error {\n\t\t\t\treturn errors.New(\"fake-error-1\")\n\t\t\t},\n\t\t\texpectedErr: \"error to init backup repo: fake-error-1\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tretFuncInit: func(context.Context, udmrepo.RepoOptions) error {\n\t\t\t\treturn nil\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfuncTable = tc.funcTable\n\n\t\t\tvar secretStore velerocredentials.SecretStore\n\t\t\tif tc.getter != nil {\n\t\t\t\ttc.getter.On(\"Get\", mock.Anything, mock.Anything).Return(tc.credStoreReturn, tc.credStoreError)\n\t\t\t\tsecretStore = tc.getter\n\t\t\t}\n\n\t\t\turp := unifiedRepoProvider{\n\t\t\t\tcredentialGetter: velerocredentials.CredentialGetter{\n\t\t\t\t\tFromSecret: secretStore,\n\t\t\t\t},\n\t\t\t\trepoService: tc.repoService,\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tif tc.repoService != nil {\n\t\t\t\ttc.repoService.On(\"Create\", mock.Anything, mock.Anything).Return(tc.retFuncInit)\n\t\t\t}\n\n\t\t\tif tc.readOnlyBSL {\n\t\t\t\tbsl.Spec.AccessMode = velerov1api.BackupStorageLocationAccessModeReadOnly\n\t\t\t} else {\n\t\t\t\tbsl.Spec.AccessMode = velerov1api.BackupStorageLocationAccessModeReadWrite\n\t\t\t}\n\n\t\t\terr := urp.InitRepo(t.Context(), RepoParam{\n\t\t\t\tBackupLocation: &bsl,\n\t\t\t\tBackupRepo: &velerov1api.BackupRepository{},\n\t\t\t})\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestConnectToRepo(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tfuncTable localFuncTable\n\t\tgetter *credmock.SecretStore\n\t\trepoService *reposervicenmocks.BackupRepoService\n\t\tretFuncInit any\n\t\tcredStoreReturn string\n\t\tcredStoreError error\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"get repo option fail\",\n\t\t\texpectedErr: \"error to get repo options: error to get repo password: invalid credentials interface\",\n\t\t},\n\t\t{\n\t\t\tname: \"repo init fail\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tretFuncInit: func(context.Context, udmrepo.RepoOptions) error {\n\t\t\t\treturn errors.New(\"fake-error-1\")\n\t\t\t},\n\t\t\texpectedErr: \"error to connect backup repo: fake-error-1\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tretFuncInit: func(context.Context, udmrepo.RepoOptions) error {\n\t\t\t\treturn nil\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfuncTable = tc.funcTable\n\n\t\t\tvar secretStore velerocredentials.SecretStore\n\t\t\tif tc.getter != nil {\n\t\t\t\ttc.getter.On(\"Get\", mock.Anything, mock.Anything).Return(tc.credStoreReturn, tc.credStoreError)\n\t\t\t\tsecretStore = tc.getter\n\t\t\t}\n\n\t\t\turp := unifiedRepoProvider{\n\t\t\t\tcredentialGetter: velerocredentials.CredentialGetter{\n\t\t\t\t\tFromSecret: secretStore,\n\t\t\t\t},\n\t\t\t\trepoService: tc.repoService,\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tif tc.repoService != nil {\n\t\t\t\ttc.repoService.On(\"Connect\", mock.Anything, mock.Anything).Return(tc.retFuncInit)\n\t\t\t}\n\n\t\t\terr := urp.ConnectToRepo(t.Context(), RepoParam{\n\t\t\t\tBackupLocation: &velerov1api.BackupStorageLocation{},\n\t\t\t\tBackupRepo: &velerov1api.BackupRepository{},\n\t\t\t})\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestBoostRepoConnect(t *testing.T) {\n\tvar backupRepo *reposervicenmocks.BackupRepo\n\n\ttestCases := []struct {\n\t\tname string\n\t\tfuncTable localFuncTable\n\t\tgetter *credmock.SecretStore\n\t\trepoService *reposervicenmocks.BackupRepoService\n\t\tbackupRepo *reposervicenmocks.BackupRepo\n\t\tretFuncInit any\n\t\tretFuncOpen []any\n\t\tcredStoreReturn string\n\t\tcredStoreError error\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"get repo option fail\",\n\t\t\texpectedErr: \"error to get repo options: error to get repo password: invalid credentials interface\",\n\t\t},\n\t\t{\n\t\t\tname: \"repo not opened and connect fail\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tretFuncOpen: []any{\n\t\t\t\tfunc(context.Context, udmrepo.RepoOptions) udmrepo.BackupRepo {\n\t\t\t\t\treturn backupRepo\n\t\t\t\t},\n\n\t\t\t\tfunc(context.Context, udmrepo.RepoOptions) error {\n\t\t\t\t\treturn errors.New(\"fake-error-1\")\n\t\t\t\t},\n\t\t\t},\n\t\t\tretFuncInit: func(context.Context, udmrepo.RepoOptions) error {\n\t\t\t\treturn errors.New(\"fake-error-2\")\n\t\t\t},\n\t\t\texpectedErr: \"error to connect backup repo: fake-error-2\",\n\t\t},\n\t\t{\n\t\t\tname: \"repo not opened and connect succeed\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tretFuncOpen: []any{\n\t\t\t\tfunc(context.Context, udmrepo.RepoOptions) udmrepo.BackupRepo {\n\t\t\t\t\treturn backupRepo\n\t\t\t\t},\n\n\t\t\t\tfunc(context.Context, udmrepo.RepoOptions) error {\n\t\t\t\t\treturn errors.New(\"fake-error-1\")\n\t\t\t\t},\n\t\t\t},\n\t\t\tretFuncInit: func(context.Context, udmrepo.RepoOptions) error {\n\t\t\t\treturn nil\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"repo is opened\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tbackupRepo: new(reposervicenmocks.BackupRepo),\n\t\t\tretFuncOpen: []any{\n\t\t\t\tfunc(context.Context, udmrepo.RepoOptions) udmrepo.BackupRepo {\n\t\t\t\t\treturn backupRepo\n\t\t\t\t},\n\n\t\t\t\tfunc(context.Context, udmrepo.RepoOptions) error {\n\t\t\t\t\treturn nil\n\t\t\t\t},\n\t\t\t},\n\t\t\tretFuncInit: func(context.Context, udmrepo.RepoOptions) error {\n\t\t\t\treturn nil\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfuncTable = tc.funcTable\n\n\t\t\tvar secretStore velerocredentials.SecretStore\n\t\t\tif tc.getter != nil {\n\t\t\t\ttc.getter.On(\"Get\", mock.Anything, mock.Anything).Return(tc.credStoreReturn, tc.credStoreError)\n\t\t\t\tsecretStore = tc.getter\n\t\t\t}\n\n\t\t\turp := unifiedRepoProvider{\n\t\t\t\tcredentialGetter: velerocredentials.CredentialGetter{\n\t\t\t\t\tFromSecret: secretStore,\n\t\t\t\t},\n\t\t\t\trepoService: tc.repoService,\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tbackupRepo = tc.backupRepo\n\n\t\t\tif tc.repoService != nil {\n\t\t\t\ttc.repoService.On(\"Open\", mock.Anything, mock.Anything).Return(tc.retFuncOpen[0], tc.retFuncOpen[1])\n\t\t\t\ttc.repoService.On(\"Connect\", mock.Anything, mock.Anything).Return(tc.retFuncInit)\n\t\t\t}\n\n\t\t\tif tc.backupRepo != nil {\n\t\t\t\tbackupRepo.On(\"Close\", mock.Anything).Return(nil)\n\t\t\t}\n\n\t\t\terr := urp.BoostRepoConnect(t.Context(), RepoParam{\n\t\t\t\tBackupLocation: &velerov1api.BackupStorageLocation{},\n\t\t\t\tBackupRepo: &velerov1api.BackupRepository{},\n\t\t\t})\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestPruneRepo(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tfuncTable localFuncTable\n\t\tgetter *credmock.SecretStore\n\t\trepoService *reposervicenmocks.BackupRepoService\n\t\tretFuncMaintain any\n\t\tcredStoreReturn string\n\t\tcredStoreError error\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"get repo option fail\",\n\t\t\texpectedErr: \"error to get repo options: error to get repo password: invalid credentials interface\",\n\t\t},\n\t\t{\n\t\t\tname: \"repo maintain fail\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tretFuncMaintain: func(context.Context, udmrepo.RepoOptions) error {\n\t\t\t\treturn errors.New(\"fake-error-1\")\n\t\t\t},\n\t\t\texpectedErr: \"error to prune backup repo: fake-error-1\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tgetter: new(credmock.SecretStore),\n\t\t\tcredStoreReturn: \"fake-password\",\n\t\t\tfuncTable: localFuncTable{\n\t\t\t\tgetStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, map[string]string, velerocredentials.CredentialGetter) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t\tgetStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {\n\t\t\t\t\treturn map[string]string{}, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoService: new(reposervicenmocks.BackupRepoService),\n\t\t\tretFuncMaintain: func(context.Context, udmrepo.RepoOptions) error {\n\t\t\t\treturn nil\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfuncTable = tc.funcTable\n\n\t\t\tvar secretStore velerocredentials.SecretStore\n\t\t\tif tc.getter != nil {\n\t\t\t\ttc.getter.On(\"Get\", mock.Anything, mock.Anything).Return(tc.credStoreReturn, tc.credStoreError)\n\t\t\t\tsecretStore = tc.getter\n\t\t\t}\n\n\t\t\turp := unifiedRepoProvider{\n\t\t\t\tcredentialGetter: velerocredentials.CredentialGetter{\n\t\t\t\t\tFromSecret: secretStore,\n\t\t\t\t},\n\t\t\t\trepoService: tc.repoService,\n\t\t\t\tlog: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tif tc.repoService != nil {\n\t\t\t\ttc.repoService.On(\"Maintain\", mock.Anything, mock.Anything).Return(tc.retFuncMaintain)\n\t\t\t}\n\n\t\t\terr := urp.PruneRepo(t.Context(), RepoParam{\n\t\t\t\tBackupLocation: &velerov1api.BackupStorageLocation{},\n\t\t\t\tBackupRepo: &velerov1api.BackupRepository{},\n\t\t\t})\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetStorageType(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tbackupLocation *velerov1api.BackupStorageLocation\n\t\texpectedRet string\n\t}{\n\t\t{\n\t\t\tname: \"wrong backend type\",\n\t\t\tbackupLocation: &velerov1api.BackupStorageLocation{},\n\t\t},\n\t\t{\n\t\t\tname: \"aws provider\",\n\t\t\tbackupLocation: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"velero.io/aws\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedRet: \"s3\",\n\t\t},\n\t\t{\n\t\t\tname: \"azure provider\",\n\t\t\tbackupLocation: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"velero.io/azure\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedRet: \"azure\",\n\t\t},\n\t\t{\n\t\t\tname: \"gcp provider\",\n\t\t\tbackupLocation: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"velero.io/gcp\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedRet: \"gcs\",\n\t\t},\n\t\t{\n\t\t\tname: \"fs provider\",\n\t\t\tbackupLocation: &velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"velero.io/fs\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedRet: \"filesystem\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tret := getStorageType(tc.backupLocation)\n\t\t\tassert.Equal(t, tc.expectedRet, ret)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/repository/restic/repository.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restic\n\nimport (\n\t\"os\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\trepokey \"github.com/vmware-tanzu/velero/pkg/repository/keys\"\n\t\"github.com/vmware-tanzu/velero/pkg/restic\"\n\tveleroexec \"github.com/vmware-tanzu/velero/pkg/util/exec\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n)\n\nfunc NewRepositoryService(credGetter credentials.CredentialGetter, fs filesystem.Interface, log logrus.FieldLogger) *RepositoryService {\n\treturn &RepositoryService{\n\t\tcredGetter: credGetter,\n\t\tfileSystem: fs,\n\t\tlog: log,\n\t}\n}\n\ntype RepositoryService struct {\n\tcredGetter credentials.CredentialGetter\n\tfileSystem filesystem.Interface\n\tlog logrus.FieldLogger\n}\n\nfunc (r *RepositoryService) InitRepo(bsl *velerov1api.BackupStorageLocation, repo *velerov1api.BackupRepository) error {\n\treturn r.exec(restic.InitCommand(repo.Spec.ResticIdentifier), bsl)\n}\n\nfunc (r *RepositoryService) ConnectToRepo(bsl *velerov1api.BackupStorageLocation, repo *velerov1api.BackupRepository) error {\n\tsnapshotsCmd := restic.SnapshotsCommand(repo.Spec.ResticIdentifier)\n\t// use the '--latest=1' flag to minimize the amount of data fetched since\n\t// we're just validating that the repo exists and can be authenticated\n\t// to.\n\t// \"--last\" is replaced by \"--latest=1\" in restic v0.12.1\n\tsnapshotsCmd.ExtraFlags = append(snapshotsCmd.ExtraFlags, \"--latest=1\")\n\n\treturn r.exec(snapshotsCmd, bsl)\n}\n\nfunc (r *RepositoryService) PruneRepo(bsl *velerov1api.BackupStorageLocation, repo *velerov1api.BackupRepository) error {\n\treturn r.exec(restic.PruneCommand(repo.Spec.ResticIdentifier), bsl)\n}\n\nfunc (r *RepositoryService) UnlockRepo(bsl *velerov1api.BackupStorageLocation, repo *velerov1api.BackupRepository) error {\n\treturn r.exec(restic.UnlockCommand(repo.Spec.ResticIdentifier), bsl)\n}\n\nfunc (r *RepositoryService) Forget(bsl *velerov1api.BackupStorageLocation, repo *velerov1api.BackupRepository, snapshotID string) error {\n\treturn r.exec(restic.ForgetCommand(repo.Spec.ResticIdentifier, snapshotID), bsl)\n}\n\nfunc (r *RepositoryService) DefaultMaintenanceFrequency() time.Duration {\n\treturn restic.DefaultMaintenanceFrequency\n}\n\nfunc (r *RepositoryService) exec(cmd *restic.Command, bsl *velerov1api.BackupStorageLocation) error {\n\tfile, err := r.credGetter.FromFile.Path(repokey.RepoKeySelector())\n\tif err != nil {\n\t\treturn err\n\t}\n\t// ignore error since there's nothing we can do and it's a temp file.\n\tdefer os.Remove(file)\n\n\tcmd.PasswordFile = file\n\n\t// if there's a caCert on the ObjectStorage, write it to disk so that it can be passed to restic\n\tvar caCertFile string\n\tif bsl.Spec.ObjectStorage != nil {\n\t\tvar caCertData []byte\n\n\t\t// Try CACertRef first (new method), then fall back to CACert (deprecated)\n\t\tif bsl.Spec.ObjectStorage.CACertRef != nil {\n\t\t\tcaCertString, err := r.credGetter.FromSecret.Get(bsl.Spec.ObjectStorage.CACertRef)\n\t\t\tif err != nil {\n\t\t\t\treturn errors.Wrap(err, \"error getting CA certificate from secret\")\n\t\t\t}\n\t\t\tcaCertData = []byte(caCertString)\n\t\t} else if bsl.Spec.ObjectStorage.CACert != nil {\n\t\t\tcaCertData = bsl.Spec.ObjectStorage.CACert\n\t\t}\n\n\t\tif caCertData != nil {\n\t\t\tcaCertFile, err = restic.TempCACertFile(caCertData, bsl.Name, r.fileSystem)\n\t\t\tif err != nil {\n\t\t\t\treturn errors.Wrap(err, \"error creating temp cacert file\")\n\t\t\t}\n\t\t\t// ignore error since there's nothing we can do and it's a temp file.\n\t\t\tdefer os.Remove(caCertFile)\n\t\t}\n\t}\n\tcmd.CACertFile = caCertFile\n\n\t// CmdEnv uses credGetter.FromFile (not FromSecret) to get cloud provider credentials.\n\t// FromFile materializes the BSL's Credential secret to a file path that cloud SDKs\n\t// can read (e.g., AWS_SHARED_CREDENTIALS_FILE). This is different from caCertRef above,\n\t// which uses FromSecret to read the CA certificate data directly into memory, then\n\t// writes it to a temp file because restic CLI only accepts file paths (--cacert flag).\n\tenv, err := restic.CmdEnv(bsl, r.credGetter.FromFile)\n\tif err != nil {\n\t\treturn err\n\t}\n\tcmd.Env = env\n\n\t// #4820: restrieve insecureSkipTLSVerify from BSL configuration for\n\t// AWS plugin. If nothing is return, that means insecureSkipTLSVerify\n\t// is not enable for Restic command.\n\tskipTLSRet := restic.GetInsecureSkipTLSVerifyFromBSL(bsl, r.log)\n\tif len(skipTLSRet) > 0 {\n\t\tcmd.ExtraFlags = append(cmd.ExtraFlags, skipTLSRet)\n\t}\n\n\tstdout, stderr, err := veleroexec.RunCommandWithLog(cmd.Cmd(), r.log)\n\tr.log.WithFields(logrus.Fields{\n\t\t\"repository\": cmd.RepoName(),\n\t\t\"command\": cmd.String(),\n\t\t\"stdout\": stdout,\n\t\t\"stderr\": stderr,\n\t}).Debugf(\"Ran restic command\")\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error running command=%s, stdout=%s, stderr=%s\", cmd.String(), stdout, stderr)\n\t}\n\n\treturn nil\n}\n" }, { "path": "pkg/repository/types/snapshotidentifier.go", "content": "package types\n\n// SnapshotIdentifier uniquely identifies a snapshot\n// taken by Velero.\ntype SnapshotIdentifier struct {\n\t// VolumeNamespace is the namespace of the pod/volume that\n\t// the snapshot is for.\n\tVolumeNamespace string `json:\"volumeNamespace\"`\n\n\t// BackupStorageLocation is the backup's storage location\n\t// name.\n\tBackupStorageLocation string `json:\"backupStorageLocation\"`\n\n\t// SnapshotID is the short ID of the snapshot.\n\tSnapshotID string `json:\"snapshotID\"`\n\n\t// RepositoryType is the type of the repository where the\n\t// snapshot is stored\n\tRepositoryType string `json:\"repositoryType\"`\n\n\t// Source is the source of the data saved in the repo by the snapshot\n\tSource string `json:\"source\"`\n\n\t// UploaderType is the type of uploader which saved the snapshot data\n\tUploaderType string `json:\"uploaderType\"`\n\n\t// RepoIdentifier is the identifier of the repository where the\n\t// snapshot is stored\n\tRepoIdentifier string `json:\"repoIdentifier\"`\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/azure/azure_storage_wrapper.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage azure\n\nimport (\n\t\"context\"\n\n\t\"github.com/kopia/kopia/repo/blob\"\n\t\"github.com/kopia/kopia/repo/blob/azure\"\n\t\"github.com/kopia/kopia/repo/blob/throttling\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo/kopialib/backend/logging\"\n\tazureutil \"github.com/vmware-tanzu/velero/pkg/util/azure\"\n)\n\nconst (\n\tstorageType = \"azure\"\n)\n\nfunc init() {\n\tblob.AddSupportedStorage(storageType, Option{}, NewStorage)\n}\n\ntype Option struct {\n\tConfig map[string]string `json:\"config\" kopia:\"sensitive\"`\n\tLimits throttling.Limits\n}\n\ntype Storage struct {\n\tblob.Storage\n\tOption *Option\n}\n\nfunc (s *Storage) ConnectionInfo() blob.ConnectionInfo {\n\treturn blob.ConnectionInfo{\n\t\tType: storageType,\n\t\tConfig: s.Option,\n\t}\n}\n\nfunc NewStorage(ctx context.Context, option *Option, isCreate bool) (blob.Storage, error) {\n\tcfg := option.Config\n\n\t// Get logger from context\n\tlogger := logging.LoggerFromContext(ctx)\n\n\tclient, _, err := azureutil.NewStorageClient(logger, cfg)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\topt := &azure.Options{\n\t\tContainer: cfg[udmrepo.StoreOptionOssBucket],\n\t\tPrefix: cfg[udmrepo.StoreOptionPrefix],\n\t\tLimits: option.Limits,\n\t}\n\tazStorage, err := azure.NewWithClient(ctx, opt, client)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tlogger.Info(\"Successfully created Azure storage backend\")\n\n\treturn &Storage{\n\t\tOption: option,\n\t\tStorage: azStorage,\n\t}, nil\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/azure.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backend\n\nimport (\n\t\"context\"\n\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/kopia/kopia/repo/blob\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo/kopialib/backend/azure\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo/kopialib/backend/logging\"\n)\n\ntype AzureBackend struct {\n\toption azure.Option\n}\n\nfunc (c *AzureBackend) Setup(ctx context.Context, flags map[string]string, logger logrus.FieldLogger) error {\n\tif flags[udmrepo.StoreOptionCACert] != \"\" {\n\t\tflags[\"caCertEncoded\"] = \"true\"\n\t}\n\tc.option = azure.Option{\n\t\tConfig: flags,\n\t\tLimits: setupLimits(ctx, flags),\n\t}\n\treturn nil\n}\n\nfunc (c *AzureBackend) Connect(ctx context.Context, isCreate bool, logger logrus.FieldLogger) (blob.Storage, error) {\n\tctx = logging.WithLogger(ctx, logger)\n\treturn azure.NewStorage(ctx, &c.option, false)\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/azure_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backend\n\nimport (\n\t\"testing\"\n\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\n\t\"github.com/kopia/kopia/repo/blob/throttling\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n)\n\nfunc TestAzureSetup(t *testing.T) {\n\tbackend := AzureBackend{}\n\tlogger := velerotest.NewLogger()\n\n\tflags := map[string]string{\n\t\t\"key\": \"value\",\n\t\tudmrepo.ThrottleOptionReadOps: \"100\",\n\t\tudmrepo.ThrottleOptionUploadBytes: \"200\",\n\t}\n\tlimits := throttling.Limits{\n\t\tReadsPerSecond: 100,\n\t\tUploadBytesPerSecond: 200,\n\t}\n\n\terr := backend.Setup(t.Context(), flags, logger)\n\trequire.NoError(t, err)\n\tassert.Equal(t, flags, backend.option.Config)\n\tassert.Equal(t, limits, backend.option.Limits)\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/backend.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backend\n\nimport (\n\t\"context\"\n\t\"errors\"\n\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/kopia/kopia/repo/blob\"\n)\n\nvar ErrStoreNotExist = errors.New(\"store does not exist\")\n\n// Store defines the methods for Kopia to establish a connection to\n// the backend storage\ntype Store interface {\n\t// Setup setups the variables to a specific backend storage\n\tSetup(ctx context.Context, flags map[string]string, logger logrus.FieldLogger) error\n\n\t// Connect connects to a specific backend storage with the storage variables\n\tConnect(ctx context.Context, isCreate bool, logger logrus.FieldLogger) (blob.Storage, error)\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/common.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backend\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\t\"github.com/kopia/kopia/repo\"\n\t\"github.com/kopia/kopia/repo/blob\"\n\t\"github.com/kopia/kopia/repo/blob/throttling\"\n\t\"github.com/kopia/kopia/repo/content\"\n\t\"github.com/kopia/kopia/repo/encryption\"\n\t\"github.com/kopia/kopia/repo/format\"\n\t\"github.com/kopia/kopia/repo/hashing\"\n\t\"github.com/kopia/kopia/repo/splitter\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n)\n\nconst (\n\tDefaultCacheLimitMB = 5000\n\tmaxCacheDurationSecond = 30\n)\n\nfunc setupLimits(ctx context.Context, flags map[string]string) throttling.Limits {\n\treturn throttling.Limits{\n\t\tDownloadBytesPerSecond: optionalHaveFloat64(ctx, udmrepo.ThrottleOptionDownloadBytes, flags),\n\t\tListsPerSecond: optionalHaveFloat64(ctx, udmrepo.ThrottleOptionListOps, flags),\n\t\tReadsPerSecond: optionalHaveFloat64(ctx, udmrepo.ThrottleOptionReadOps, flags),\n\t\tUploadBytesPerSecond: optionalHaveFloat64(ctx, udmrepo.ThrottleOptionUploadBytes, flags),\n\t\tWritesPerSecond: optionalHaveFloat64(ctx, udmrepo.ThrottleOptionWriteOps, flags),\n\t}\n}\n\n// SetupNewRepositoryOptions setups the options when creating a new Kopia repository\nfunc SetupNewRepositoryOptions(ctx context.Context, flags map[string]string) repo.NewRepositoryOptions {\n\treturn repo.NewRepositoryOptions{\n\t\tBlockFormat: format.ContentFormat{\n\t\t\tHash: optionalHaveStringWithDefault(udmrepo.StoreOptionGenHashAlgo, flags, hashing.DefaultAlgorithm),\n\t\t\tEncryption: optionalHaveStringWithDefault(udmrepo.StoreOptionGenEncryptAlgo, flags, encryption.DefaultAlgorithm),\n\t\t},\n\n\t\tObjectFormat: format.ObjectFormat{\n\t\t\tSplitter: optionalHaveStringWithDefault(udmrepo.StoreOptionGenSplitAlgo, flags, splitter.DefaultAlgorithm),\n\t\t},\n\n\t\tRetentionMode: blob.RetentionMode(optionalHaveString(udmrepo.StoreOptionGenRetentionMode, flags)),\n\t\tRetentionPeriod: optionalHaveDuration(ctx, udmrepo.StoreOptionGenRetentionPeriod, flags),\n\t}\n}\n\n// SetupConnectOptions setups the options when connecting to an existing Kopia repository\nfunc SetupConnectOptions(ctx context.Context, repoOptions udmrepo.RepoOptions) repo.ConnectOptions {\n\tcacheLimit := optionalHaveIntWithDefault(ctx, udmrepo.StoreOptionCacheLimit, repoOptions.StorageOptions, DefaultCacheLimitMB) << 20\n\tcacheDir := optionalHaveString(udmrepo.StoreOptionCacheDir, repoOptions.StorageOptions)\n\n\t// 80% for data cache and 20% for metadata cache and align to KB\n\tdataCacheLimit := (cacheLimit / 5 * 4) >> 10\n\tmetadataCacheLimit := (cacheLimit / 5) >> 10\n\n\treturn repo.ConnectOptions{\n\t\tCachingOptions: content.CachingOptions{\n\t\t\tCacheDirectory: cacheDir,\n\t\t\t// softLimit 80%\n\t\t\tContentCacheSizeBytes: (dataCacheLimit / 5 * 4) << 10,\n\t\t\tMetadataCacheSizeBytes: (metadataCacheLimit / 5 * 4) << 10,\n\t\t\t// hardLimit 100%\n\t\t\tContentCacheSizeLimitBytes: dataCacheLimit << 10,\n\t\t\tMetadataCacheSizeLimitBytes: metadataCacheLimit << 10,\n\t\t\tMaxListCacheDuration: content.DurationSeconds(time.Duration(maxCacheDurationSecond) * time.Second),\n\t\t},\n\t\tClientOptions: repo.ClientOptions{\n\t\t\tHostname: optionalHaveString(udmrepo.GenOptionOwnerDomain, repoOptions.GeneralOptions),\n\t\t\tUsername: optionalHaveString(udmrepo.GenOptionOwnerName, repoOptions.GeneralOptions),\n\t\t\tReadOnly: optionalHaveBool(ctx, udmrepo.StoreOptionGenReadOnly, repoOptions.GeneralOptions),\n\t\t\tDescription: repoOptions.Description,\n\t\t},\n\t}\n}\n\nfunc RepoOwnerFromRepoOptions(repoOptions udmrepo.RepoOptions) string {\n\thostname := optionalHaveStringWithDefault(udmrepo.GenOptionOwnerDomain, repoOptions.GeneralOptions, udmrepo.GetRepoDomain())\n\tusername := optionalHaveStringWithDefault(udmrepo.GenOptionOwnerName, repoOptions.GeneralOptions, udmrepo.GetRepoUser())\n\n\treturn username + \"@\" + hostname\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/common_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backend\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/kopia/kopia/repo\"\n\t\"github.com/kopia/kopia/repo/content\"\n\t\"github.com/kopia/kopia/repo/encryption\"\n\t\"github.com/kopia/kopia/repo/format\"\n\t\"github.com/kopia/kopia/repo/hashing\"\n\t\"github.com/kopia/kopia/repo/splitter\"\n\t\"github.com/stretchr/testify/assert\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n)\n\nfunc TestSetupNewRepositoryOptions(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tflags map[string]string\n\t\texpected repo.NewRepositoryOptions\n\t}{\n\t\t{\n\t\t\tname: \"with hash algo\",\n\t\t\tflags: map[string]string{\n\t\t\t\tudmrepo.StoreOptionGenHashAlgo: \"fake-hash\",\n\t\t\t},\n\t\t\texpected: repo.NewRepositoryOptions{\n\t\t\t\tBlockFormat: format.ContentFormat{\n\t\t\t\t\tHash: \"fake-hash\",\n\t\t\t\t\tEncryption: encryption.DefaultAlgorithm,\n\t\t\t\t},\n\t\t\t\tObjectFormat: format.ObjectFormat{\n\t\t\t\t\tSplitter: splitter.DefaultAlgorithm,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with encrypt algo\",\n\t\t\tflags: map[string]string{\n\t\t\t\tudmrepo.StoreOptionGenEncryptAlgo: \"fake-encrypt\",\n\t\t\t},\n\t\t\texpected: repo.NewRepositoryOptions{\n\t\t\t\tBlockFormat: format.ContentFormat{\n\t\t\t\t\tHash: hashing.DefaultAlgorithm,\n\t\t\t\t\tEncryption: \"fake-encrypt\",\n\t\t\t\t},\n\t\t\t\tObjectFormat: format.ObjectFormat{\n\t\t\t\t\tSplitter: splitter.DefaultAlgorithm,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with splitter algo\",\n\t\t\tflags: map[string]string{\n\t\t\t\tudmrepo.StoreOptionGenSplitAlgo: \"fake-splitter\",\n\t\t\t},\n\t\t\texpected: repo.NewRepositoryOptions{\n\t\t\t\tBlockFormat: format.ContentFormat{\n\t\t\t\t\tHash: hashing.DefaultAlgorithm,\n\t\t\t\t\tEncryption: encryption.DefaultAlgorithm,\n\t\t\t\t},\n\t\t\t\tObjectFormat: format.ObjectFormat{\n\t\t\t\t\tSplitter: \"fake-splitter\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with retention algo\",\n\t\t\tflags: map[string]string{\n\t\t\t\tudmrepo.StoreOptionGenRetentionMode: \"fake-retention-mode\",\n\t\t\t},\n\t\t\texpected: repo.NewRepositoryOptions{\n\t\t\t\tBlockFormat: format.ContentFormat{\n\t\t\t\t\tHash: hashing.DefaultAlgorithm,\n\t\t\t\t\tEncryption: encryption.DefaultAlgorithm,\n\t\t\t\t},\n\t\t\t\tObjectFormat: format.ObjectFormat{\n\t\t\t\t\tSplitter: splitter.DefaultAlgorithm,\n\t\t\t\t},\n\t\t\t\tRetentionMode: \"fake-retention-mode\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tret := SetupNewRepositoryOptions(t.Context(), tc.flags)\n\t\t\tassert.Equal(t, tc.expected, ret)\n\t\t})\n\t}\n}\n\nfunc TestSetupConnectOptions(t *testing.T) {\n\tdefaultCacheOption := content.CachingOptions{\n\t\tContentCacheSizeBytes: 3200 << 20,\n\t\tMetadataCacheSizeBytes: 800 << 20,\n\t\tContentCacheSizeLimitBytes: 4000 << 20,\n\t\tMetadataCacheSizeLimitBytes: 1000 << 20,\n\t\tMaxListCacheDuration: content.DurationSeconds(time.Duration(30) * time.Second),\n\t}\n\n\ttestCases := []struct {\n\t\tname string\n\t\trepoOptions udmrepo.RepoOptions\n\t\texpected repo.ConnectOptions\n\t}{\n\t\t{\n\t\t\tname: \"with domain\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tGeneralOptions: map[string]string{\n\t\t\t\t\tudmrepo.GenOptionOwnerDomain: \"fake-domain\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: repo.ConnectOptions{\n\t\t\t\tCachingOptions: defaultCacheOption,\n\t\t\t\tClientOptions: repo.ClientOptions{\n\t\t\t\t\tHostname: \"fake-domain\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with username\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tGeneralOptions: map[string]string{\n\t\t\t\t\tudmrepo.GenOptionOwnerName: \"fake-user\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: repo.ConnectOptions{\n\t\t\t\tCachingOptions: defaultCacheOption,\n\t\t\t\tClientOptions: repo.ClientOptions{\n\t\t\t\t\tUsername: \"fake-user\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with wrong readonly\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tGeneralOptions: map[string]string{\n\t\t\t\t\tudmrepo.StoreOptionGenReadOnly: \"fake-bool\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: repo.ConnectOptions{\n\t\t\t\tCachingOptions: defaultCacheOption,\n\t\t\t\tClientOptions: repo.ClientOptions{},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with correct readonly\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tGeneralOptions: map[string]string{\n\t\t\t\t\tudmrepo.StoreOptionGenReadOnly: \"true\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: repo.ConnectOptions{\n\t\t\t\tCachingOptions: defaultCacheOption,\n\t\t\t\tClientOptions: repo.ClientOptions{\n\t\t\t\t\tReadOnly: true,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with description\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tDescription: \"fake-description\",\n\t\t\t},\n\t\t\texpected: repo.ConnectOptions{\n\t\t\t\tCachingOptions: defaultCacheOption,\n\t\t\t\tClientOptions: repo.ClientOptions{\n\t\t\t\t\tDescription: \"fake-description\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tret := SetupConnectOptions(t.Context(), tc.repoOptions)\n\t\t\tassert.Equal(t, tc.expected, ret)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/file_system.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backend\n\nimport (\n\t\"context\"\n\t\"os\"\n\t\"path/filepath\"\n\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/kopia/kopia/repo/blob\"\n\t\"github.com/kopia/kopia/repo/blob/filesystem\"\n\t\"github.com/pkg/errors\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo/kopialib/backend/logging\"\n)\n\ntype FsBackend struct {\n\toptions filesystem.Options\n}\n\nconst (\n\tdefaultFileMode = 0o600\n\tdefaultDirMode = 0o700\n)\n\nfunc (c *FsBackend) Setup(ctx context.Context, flags map[string]string, logger logrus.FieldLogger) error {\n\tpath, err := mustHaveString(udmrepo.StoreOptionFsPath, flags)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tprefix := optionalHaveString(udmrepo.StoreOptionPrefix, flags)\n\n\tc.options.Path = filepath.Join(path, prefix)\n\tc.options.FileMode = defaultFileMode\n\tc.options.DirectoryMode = defaultDirMode\n\n\tctx = logging.WithLogger(ctx, logger)\n\n\tc.options.Limits = setupLimits(ctx, flags)\n\n\treturn nil\n}\n\nfunc (c *FsBackend) Connect(ctx context.Context, isCreate bool, logger logrus.FieldLogger) (blob.Storage, error) {\n\tif !filepath.IsAbs(c.options.Path) {\n\t\treturn nil, errors.Errorf(\"filesystem repository path is not absolute, path: %s\", c.options.Path)\n\t}\n\n\tif !isCreate {\n\t\tif _, err := os.Stat(c.options.Path); err != nil {\n\t\t\treturn nil, ErrStoreNotExist\n\t\t}\n\t}\n\n\tctx = logging.WithLogger(ctx, logger)\n\n\treturn filesystem.New(ctx, &c.options, isCreate)\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/file_system_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backend\n\nimport (\n\t\"testing\"\n\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\n\t\"github.com/kopia/kopia/repo/blob/filesystem\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n)\n\nfunc TestFSSetup(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tflags map[string]string\n\t\texpectedOptions filesystem.Options\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"must have fs path\",\n\t\t\tflags: map[string]string{},\n\t\t\texpectedErr: \"key \" + udmrepo.StoreOptionFsPath + \" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"with fs path only\",\n\t\t\tflags: map[string]string{\n\t\t\t\tudmrepo.StoreOptionFsPath: \"fake/path\",\n\t\t\t},\n\t\t\texpectedOptions: filesystem.Options{\n\t\t\t\tPath: \"fake/path\",\n\t\t\t\tFileMode: 0o600,\n\t\t\t\tDirectoryMode: 0o700,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with prefix\",\n\t\t\tflags: map[string]string{\n\t\t\t\tudmrepo.StoreOptionFsPath: \"fake/path\",\n\t\t\t\tudmrepo.StoreOptionPrefix: \"fake-prefix\",\n\t\t\t},\n\t\t\texpectedOptions: filesystem.Options{\n\t\t\t\tPath: \"fake/path/fake-prefix\",\n\t\t\t\tFileMode: 0o600,\n\t\t\t\tDirectoryMode: 0o700,\n\t\t\t},\n\t\t},\n\t}\n\n\tlogger := velerotest.NewLogger()\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfsFlags := FsBackend{}\n\n\t\t\terr := fsFlags.Setup(t.Context(), tc.flags, logger)\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, tc.expectedOptions, fsFlags.options)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/gcs.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backend\n\nimport (\n\t\"context\"\n\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/kopia/kopia/repo/blob\"\n\t\"github.com/kopia/kopia/repo/blob/gcs\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo/kopialib/backend/logging\"\n)\n\ntype GCSBackend struct {\n\toptions gcs.Options\n}\n\nfunc (c *GCSBackend) Setup(ctx context.Context, flags map[string]string, logger logrus.FieldLogger) error {\n\tvar err error\n\tc.options.BucketName, err = mustHaveString(udmrepo.StoreOptionOssBucket, flags)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tc.options.ServiceAccountCredentialsFile, err = mustHaveString(udmrepo.StoreOptionCredentialFile, flags)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tc.options.Prefix = optionalHaveString(udmrepo.StoreOptionPrefix, flags)\n\tc.options.ReadOnly = optionalHaveBool(ctx, udmrepo.StoreOptionGcsReadonly, flags)\n\n\tctx = logging.WithLogger(ctx, logger)\n\n\tc.options.Limits = setupLimits(ctx, flags)\n\n\treturn nil\n}\n\nfunc (c *GCSBackend) Connect(ctx context.Context, isCreate bool, logger logrus.FieldLogger) (blob.Storage, error) {\n\tctx = logging.WithLogger(ctx, logger)\n\treturn gcs.New(ctx, &c.options, false)\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/gcs_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backend\n\nimport (\n\t\"testing\"\n\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\n\t\"github.com/kopia/kopia/repo/blob/gcs\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n)\n\nfunc TestGcsSetup(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tflags map[string]string\n\t\texpectedOptions gcs.Options\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"must have bucket name\",\n\t\t\tflags: map[string]string{},\n\t\t\texpectedErr: \"key \" + udmrepo.StoreOptionOssBucket + \" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"must have credential file\",\n\t\t\tflags: map[string]string{\n\t\t\t\tudmrepo.StoreOptionOssBucket: \"fake-bucket\",\n\t\t\t},\n\t\t\texpectedErr: \"key \" + udmrepo.StoreOptionCredentialFile + \" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"with prefix\",\n\t\t\tflags: map[string]string{\n\t\t\t\tudmrepo.StoreOptionOssBucket: \"fake-bucket\",\n\t\t\t\tudmrepo.StoreOptionCredentialFile: \"fake-credential\",\n\t\t\t\tudmrepo.StoreOptionPrefix: \"fake-prefix\",\n\t\t\t},\n\t\t\texpectedOptions: gcs.Options{\n\t\t\t\tBucketName: \"fake-bucket\",\n\t\t\t\tServiceAccountCredentialsFile: \"fake-credential\",\n\t\t\t\tPrefix: \"fake-prefix\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with wrong readonly\",\n\t\t\tflags: map[string]string{\n\t\t\t\tudmrepo.StoreOptionOssBucket: \"fake-bucket\",\n\t\t\t\tudmrepo.StoreOptionCredentialFile: \"fake-credential\",\n\t\t\t\tudmrepo.StoreOptionGcsReadonly: \"fake-bool\",\n\t\t\t},\n\t\t\texpectedOptions: gcs.Options{\n\t\t\t\tBucketName: \"fake-bucket\",\n\t\t\t\tServiceAccountCredentialsFile: \"fake-credential\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with correct readonly\",\n\t\t\tflags: map[string]string{\n\t\t\t\tudmrepo.StoreOptionOssBucket: \"fake-bucket\",\n\t\t\t\tudmrepo.StoreOptionCredentialFile: \"fake-credential\",\n\t\t\t\tudmrepo.StoreOptionGcsReadonly: \"true\",\n\t\t\t},\n\t\t\texpectedOptions: gcs.Options{\n\t\t\t\tBucketName: \"fake-bucket\",\n\t\t\t\tServiceAccountCredentialsFile: \"fake-credential\",\n\t\t\t\tReadOnly: true,\n\t\t\t},\n\t\t},\n\t}\n\n\tlogger := velerotest.NewLogger()\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tgcsFlags := GCSBackend{}\n\n\t\t\terr := gcsFlags.Setup(t.Context(), tc.flags, logger)\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, tc.expectedOptions, gcsFlags.options)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/logging/context.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage logging\n\nimport (\n\t\"context\"\n\n\t\"github.com/sirupsen/logrus\"\n)\n\ntype ctxKeyLogger struct{}\n\n// WithLogger returns a new context with the provided logger.\nfunc WithLogger(ctx context.Context, logger logrus.FieldLogger) context.Context {\n\treturn context.WithValue(ctx, ctxKeyLogger{}, logger)\n}\n\n// LoggerFromContext retrieves the logger from the context, or returns a default logger if none found.\nfunc LoggerFromContext(ctx context.Context) logrus.FieldLogger {\n\tif logger, ok := ctx.Value(ctxKeyLogger{}).(logrus.FieldLogger); ok && logger != nil {\n\t\treturn logger\n\t}\n\treturn logrus.New()\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/mocks/Logger.go", "content": "// Code generated by mockery v2.22.1. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tmock \"github.com/stretchr/testify/mock\"\n\tzapcore \"go.uber.org/zap/zapcore\"\n)\n\n// Core is an autogenerated mock type for the Core type\ntype Core struct {\n\tmock.Mock\n}\n\n// Check provides a mock function with given fields: _a0, _a1\nfunc (_m *Core) Check(_a0 zapcore.Entry, _a1 *zapcore.CheckedEntry) *zapcore.CheckedEntry {\n\tret := _m.Called(_a0, _a1)\n\n\tvar r0 *zapcore.CheckedEntry\n\tif rf, ok := ret.Get(0).(func(zapcore.Entry, *zapcore.CheckedEntry) *zapcore.CheckedEntry); ok {\n\t\tr0 = rf(_a0, _a1)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(*zapcore.CheckedEntry)\n\t\t}\n\t}\n\n\treturn r0\n}\n\n// Enabled provides a mock function with given fields: _a0\nfunc (_m *Core) Enabled(_a0 zapcore.Level) bool {\n\tret := _m.Called(_a0)\n\n\tvar r0 bool\n\tif rf, ok := ret.Get(0).(func(zapcore.Level) bool); ok {\n\t\tr0 = rf(_a0)\n\t} else {\n\t\tr0 = ret.Get(0).(bool)\n\t}\n\n\treturn r0\n}\n\n// Sync provides a mock function with given fields:\nfunc (_m *Core) Sync() error {\n\tret := _m.Called()\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func() error); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// With provides a mock function with given fields: _a0\nfunc (_m *Core) With(_a0 []zapcore.Field) zapcore.Core {\n\tret := _m.Called(_a0)\n\n\tvar r0 zapcore.Core\n\tif rf, ok := ret.Get(0).(func([]zapcore.Field) zapcore.Core); ok {\n\t\tr0 = rf(_a0)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(zapcore.Core)\n\t\t}\n\t}\n\n\treturn r0\n}\n\n// Write provides a mock function with given fields: _a0, _a1\nfunc (_m *Core) Write(_a0 zapcore.Entry, _a1 []zapcore.Field) error {\n\tret := _m.Called(_a0, _a1)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(zapcore.Entry, []zapcore.Field) error); ok {\n\t\tr0 = rf(_a0, _a1)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\ntype mockConstructorTestingTNewCore interface {\n\tmock.TestingT\n\tCleanup(func())\n}\n\n// NewCore creates a new instance of Core. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\nfunc NewCore(t mockConstructorTestingTNewCore) *Core {\n\tmock := &Core{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/mocks/Reader.go", "content": "// Code generated by mockery v2.22.1. DO NOT EDIT.\n\npackage mocks\n\nimport mock \"github.com/stretchr/testify/mock\"\n\n// Reader is an autogenerated mock type for the Reader type\ntype Reader struct {\n\tmock.Mock\n}\n\n// Close provides a mock function with given fields:\nfunc (_m *Reader) Close() error {\n\tret := _m.Called()\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func() error); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// Length provides a mock function with given fields:\nfunc (_m *Reader) Length() int64 {\n\tret := _m.Called()\n\n\tvar r0 int64\n\tif rf, ok := ret.Get(0).(func() int64); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(int64)\n\t}\n\n\treturn r0\n}\n\n// Read provides a mock function with given fields: p\nfunc (_m *Reader) Read(p []byte) (int, error) {\n\tret := _m.Called(p)\n\n\tvar r0 int\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func([]byte) (int, error)); ok {\n\t\treturn rf(p)\n\t}\n\tif rf, ok := ret.Get(0).(func([]byte) int); ok {\n\t\tr0 = rf(p)\n\t} else {\n\t\tr0 = ret.Get(0).(int)\n\t}\n\n\tif rf, ok := ret.Get(1).(func([]byte) error); ok {\n\t\tr1 = rf(p)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Seek provides a mock function with given fields: offset, whence\nfunc (_m *Reader) Seek(offset int64, whence int) (int64, error) {\n\tret := _m.Called(offset, whence)\n\n\tvar r0 int64\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(int64, int) (int64, error)); ok {\n\t\treturn rf(offset, whence)\n\t}\n\tif rf, ok := ret.Get(0).(func(int64, int) int64); ok {\n\t\tr0 = rf(offset, whence)\n\t} else {\n\t\tr0 = ret.Get(0).(int64)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(int64, int) error); ok {\n\t\tr1 = rf(offset, whence)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\ntype mockConstructorTestingTNewReader interface {\n\tmock.TestingT\n\tCleanup(func())\n}\n\n// NewReader creates a new instance of Reader. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\nfunc NewReader(t mockConstructorTestingTNewReader) *Reader {\n\tmock := &Reader{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/mocks/Storage.go", "content": "// Code generated by mockery v2.22.1. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tcontext \"context\"\n\n\tblob \"github.com/kopia/kopia/repo/blob\"\n\n\tmock \"github.com/stretchr/testify/mock\"\n)\n\n// Storage is an autogenerated mock type for the Storage type\ntype Storage struct {\n\tmock.Mock\n}\n\n// Close provides a mock function with given fields: ctx\nfunc (_m *Storage) Close(ctx context.Context) error {\n\tret := _m.Called(ctx)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context) error); ok {\n\t\tr0 = rf(ctx)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// ConnectionInfo provides a mock function with given fields:\nfunc (_m *Storage) ConnectionInfo() blob.ConnectionInfo {\n\tret := _m.Called()\n\n\tvar r0 blob.ConnectionInfo\n\tif rf, ok := ret.Get(0).(func() blob.ConnectionInfo); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(blob.ConnectionInfo)\n\t}\n\n\treturn r0\n}\n\n// DeleteBlob provides a mock function with given fields: ctx, blobID\nfunc (_m *Storage) DeleteBlob(ctx context.Context, blobID blob.ID) error {\n\tret := _m.Called(ctx, blobID)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, blob.ID) error); ok {\n\t\tr0 = rf(ctx, blobID)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// DisplayName provides a mock function with given fields:\nfunc (_m *Storage) DisplayName() string {\n\tret := _m.Called()\n\n\tvar r0 string\n\tif rf, ok := ret.Get(0).(func() string); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(string)\n\t}\n\n\treturn r0\n}\n\n// ExtendBlobRetention provides a mock function with given fields: ctx, blobID, opts\nfunc (_m *Storage) ExtendBlobRetention(ctx context.Context, blobID blob.ID, opts blob.ExtendOptions) error {\n\tret := _m.Called(ctx, blobID, opts)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, blob.ID, blob.ExtendOptions) error); ok {\n\t\tr0 = rf(ctx, blobID, opts)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// FlushCaches provides a mock function with given fields: ctx\nfunc (_m *Storage) FlushCaches(ctx context.Context) error {\n\tret := _m.Called(ctx)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context) error); ok {\n\t\tr0 = rf(ctx)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// GetBlob provides a mock function with given fields: ctx, blobID, offset, length, output\nfunc (_m *Storage) GetBlob(ctx context.Context, blobID blob.ID, offset int64, length int64, output blob.OutputBuffer) error {\n\tret := _m.Called(ctx, blobID, offset, length, output)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, blob.ID, int64, int64, blob.OutputBuffer) error); ok {\n\t\tr0 = rf(ctx, blobID, offset, length, output)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// GetCapacity provides a mock function with given fields: ctx\nfunc (_m *Storage) GetCapacity(ctx context.Context) (blob.Capacity, error) {\n\tret := _m.Called(ctx)\n\n\tvar r0 blob.Capacity\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(context.Context) (blob.Capacity, error)); ok {\n\t\treturn rf(ctx)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context) blob.Capacity); ok {\n\t\tr0 = rf(ctx)\n\t} else {\n\t\tr0 = ret.Get(0).(blob.Capacity)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context) error); ok {\n\t\tr1 = rf(ctx)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// GetMetadata provides a mock function with given fields: ctx, blobID\nfunc (_m *Storage) GetMetadata(ctx context.Context, blobID blob.ID) (blob.Metadata, error) {\n\tret := _m.Called(ctx, blobID)\n\n\tvar r0 blob.Metadata\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(context.Context, blob.ID) (blob.Metadata, error)); ok {\n\t\treturn rf(ctx, blobID)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, blob.ID) blob.Metadata); ok {\n\t\tr0 = rf(ctx, blobID)\n\t} else {\n\t\tr0 = ret.Get(0).(blob.Metadata)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, blob.ID) error); ok {\n\t\tr1 = rf(ctx, blobID)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// IsReadOnly provides a mock function with given fields:\nfunc (_m *Storage) IsReadOnly() bool {\n\tret := _m.Called()\n\n\tvar r0 bool\n\tif rf, ok := ret.Get(0).(func() bool); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(bool)\n\t}\n\n\treturn r0\n}\n\n// ListBlobs provides a mock function with given fields: ctx, blobIDPrefix, cb\nfunc (_m *Storage) ListBlobs(ctx context.Context, blobIDPrefix blob.ID, cb func(blob.Metadata) error) error {\n\tret := _m.Called(ctx, blobIDPrefix, cb)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, blob.ID, func(blob.Metadata) error) error); ok {\n\t\tr0 = rf(ctx, blobIDPrefix, cb)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// PutBlob provides a mock function with given fields: ctx, blobID, data, opts\nfunc (_m *Storage) PutBlob(ctx context.Context, blobID blob.ID, data blob.Bytes, opts blob.PutOptions) error {\n\tret := _m.Called(ctx, blobID, data, opts)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, blob.ID, blob.Bytes, blob.PutOptions) error); ok {\n\t\tr0 = rf(ctx, blobID, data, opts)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\ntype mockConstructorTestingTNewStorage interface {\n\tmock.TestingT\n\tCleanup(func())\n}\n\n// NewStorage creates a new instance of Storage. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\nfunc NewStorage(t mockConstructorTestingTNewStorage) *Storage {\n\tmock := &Storage{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/mocks/Store.go", "content": "// Code generated by mockery v2.14.0. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tcontext \"context\"\n\t\"github.com/sirupsen/logrus\"\n\n\tblob \"github.com/kopia/kopia/repo/blob\"\n\n\tmock \"github.com/stretchr/testify/mock\"\n)\n\n// Store is an autogenerated mock type for the Store type\ntype Store struct {\n\tmock.Mock\n}\n\n// Connect provides a mock function with given fields: ctx, isCreate\nfunc (_m *Store) Connect(ctx context.Context, isCreate bool, logger logrus.FieldLogger) (blob.Storage, error) {\n\tret := _m.Called(ctx, isCreate)\n\n\tvar r0 blob.Storage\n\tif rf, ok := ret.Get(0).(func(context.Context, bool) blob.Storage); ok {\n\t\tr0 = rf(ctx, isCreate)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(blob.Storage)\n\t\t}\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(context.Context, bool) error); ok {\n\t\tr1 = rf(ctx, isCreate)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Setup provides a mock function with given fields: ctx, flags\nfunc (_m *Store) Setup(ctx context.Context, flags map[string]string, logger logrus.FieldLogger) error {\n\tret := _m.Called(ctx, flags)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, map[string]string) error); ok {\n\t\tr0 = rf(ctx, flags)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\ntype mockConstructorTestingTNewStore interface {\n\tmock.TestingT\n\tCleanup(func())\n}\n\n// NewStore creates a new instance of Store. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\nfunc NewStore(t mockConstructorTestingTNewStore) *Store {\n\tmock := &Store{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/mocks/Writer.go", "content": "// Code generated by mockery v2.22.1. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tobject \"github.com/kopia/kopia/repo/object\"\n\tmock \"github.com/stretchr/testify/mock\"\n)\n\n// Writer is an autogenerated mock type for the Writer type\ntype Writer struct {\n\tmock.Mock\n}\n\n// Checkpoint provides a mock function with given fields:\nfunc (_m *Writer) Checkpoint() (object.ID, error) {\n\tret := _m.Called()\n\n\tvar r0 object.ID\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func() (object.ID, error)); ok {\n\t\treturn rf()\n\t}\n\tif rf, ok := ret.Get(0).(func() object.ID); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(object.ID)\n\t}\n\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Close provides a mock function with given fields:\nfunc (_m *Writer) Close() error {\n\tret := _m.Called()\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func() error); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// Result provides a mock function with given fields:\nfunc (_m *Writer) Result() (object.ID, error) {\n\tret := _m.Called()\n\n\tvar r0 object.ID\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func() (object.ID, error)); ok {\n\t\treturn rf()\n\t}\n\tif rf, ok := ret.Get(0).(func() object.ID); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(object.ID)\n\t}\n\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Write provides a mock function with given fields: p\nfunc (_m *Writer) Write(p []byte) (int, error) {\n\tret := _m.Called(p)\n\n\tvar r0 int\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func([]byte) (int, error)); ok {\n\t\treturn rf(p)\n\t}\n\tif rf, ok := ret.Get(0).(func([]byte) int); ok {\n\t\tr0 = rf(p)\n\t} else {\n\t\tr0 = ret.Get(0).(int)\n\t}\n\n\tif rf, ok := ret.Get(1).(func([]byte) error); ok {\n\t\tr1 = rf(p)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\ntype mockConstructorTestingTNewWriter interface {\n\tmock.TestingT\n\tCleanup(func())\n}\n\n// NewWriter creates a new instance of Writer. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\nfunc NewWriter(t mockConstructorTestingTNewWriter) *Writer {\n\tmock := &Writer{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/mocks/repository.go", "content": "// Code generated by mockery; DO NOT EDIT.\n// github.com/vektra/mockery\n// template: testify\n\npackage mocks\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\t\"github.com/kopia/kopia/repo\"\n\t\"github.com/kopia/kopia/repo/content\"\n\t\"github.com/kopia/kopia/repo/manifest\"\n\t\"github.com/kopia/kopia/repo/object\"\n\tmock \"github.com/stretchr/testify/mock\"\n)\n\n// NewMockRepository creates a new instance of MockRepository. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\n// The first argument is typically a *testing.T value.\nfunc NewMockRepository(t interface {\n\tmock.TestingT\n\tCleanup(func())\n}) *MockRepository {\n\tmock := &MockRepository{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n\n// MockRepository is an autogenerated mock type for the Repository type\ntype MockRepository struct {\n\tmock.Mock\n}\n\ntype MockRepository_Expecter struct {\n\tmock *mock.Mock\n}\n\nfunc (_m *MockRepository) EXPECT() *MockRepository_Expecter {\n\treturn &MockRepository_Expecter{mock: &_m.Mock}\n}\n\n// ClientOptions provides a mock function for the type MockRepository\nfunc (_mock *MockRepository) ClientOptions() repo.ClientOptions {\n\tret := _mock.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for ClientOptions\")\n\t}\n\n\tvar r0 repo.ClientOptions\n\tif returnFunc, ok := ret.Get(0).(func() repo.ClientOptions); ok {\n\t\tr0 = returnFunc()\n\t} else {\n\t\tr0 = ret.Get(0).(repo.ClientOptions)\n\t}\n\treturn r0\n}\n\n// MockRepository_ClientOptions_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'ClientOptions'\ntype MockRepository_ClientOptions_Call struct {\n\t*mock.Call\n}\n\n// ClientOptions is a helper method to define mock.On call\nfunc (_e *MockRepository_Expecter) ClientOptions() *MockRepository_ClientOptions_Call {\n\treturn &MockRepository_ClientOptions_Call{Call: _e.mock.On(\"ClientOptions\")}\n}\n\nfunc (_c *MockRepository_ClientOptions_Call) Run(run func()) *MockRepository_ClientOptions_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\trun()\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepository_ClientOptions_Call) Return(clientOptions repo.ClientOptions) *MockRepository_ClientOptions_Call {\n\t_c.Call.Return(clientOptions)\n\treturn _c\n}\n\nfunc (_c *MockRepository_ClientOptions_Call) RunAndReturn(run func() repo.ClientOptions) *MockRepository_ClientOptions_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// Close provides a mock function for the type MockRepository\nfunc (_mock *MockRepository) Close(ctx context.Context) error {\n\tret := _mock.Called(ctx)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Close\")\n\t}\n\n\tvar r0 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context) error); ok {\n\t\tr0 = returnFunc(ctx)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\treturn r0\n}\n\n// MockRepository_Close_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Close'\ntype MockRepository_Close_Call struct {\n\t*mock.Call\n}\n\n// Close is a helper method to define mock.On call\n// - ctx context.Context\nfunc (_e *MockRepository_Expecter) Close(ctx interface{}) *MockRepository_Close_Call {\n\treturn &MockRepository_Close_Call{Call: _e.mock.On(\"Close\", ctx)}\n}\n\nfunc (_c *MockRepository_Close_Call) Run(run func(ctx context.Context)) *MockRepository_Close_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepository_Close_Call) Return(err error) *MockRepository_Close_Call {\n\t_c.Call.Return(err)\n\treturn _c\n}\n\nfunc (_c *MockRepository_Close_Call) RunAndReturn(run func(ctx context.Context) error) *MockRepository_Close_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// ContentInfo provides a mock function for the type MockRepository\nfunc (_mock *MockRepository) ContentInfo(ctx context.Context, contentID content.ID) (content.Info, error) {\n\tret := _mock.Called(ctx, contentID)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for ContentInfo\")\n\t}\n\n\tvar r0 content.Info\n\tvar r1 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, content.ID) (content.Info, error)); ok {\n\t\treturn returnFunc(ctx, contentID)\n\t}\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, content.ID) content.Info); ok {\n\t\tr0 = returnFunc(ctx, contentID)\n\t} else {\n\t\tr0 = ret.Get(0).(content.Info)\n\t}\n\tif returnFunc, ok := ret.Get(1).(func(context.Context, content.ID) error); ok {\n\t\tr1 = returnFunc(ctx, contentID)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\treturn r0, r1\n}\n\n// MockRepository_ContentInfo_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'ContentInfo'\ntype MockRepository_ContentInfo_Call struct {\n\t*mock.Call\n}\n\n// ContentInfo is a helper method to define mock.On call\n// - ctx context.Context\n// - contentID content.ID\nfunc (_e *MockRepository_Expecter) ContentInfo(ctx interface{}, contentID interface{}) *MockRepository_ContentInfo_Call {\n\treturn &MockRepository_ContentInfo_Call{Call: _e.mock.On(\"ContentInfo\", ctx, contentID)}\n}\n\nfunc (_c *MockRepository_ContentInfo_Call) Run(run func(ctx context.Context, contentID content.ID)) *MockRepository_ContentInfo_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 content.ID\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(content.ID)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepository_ContentInfo_Call) Return(v content.Info, err error) *MockRepository_ContentInfo_Call {\n\t_c.Call.Return(v, err)\n\treturn _c\n}\n\nfunc (_c *MockRepository_ContentInfo_Call) RunAndReturn(run func(ctx context.Context, contentID content.ID) (content.Info, error)) *MockRepository_ContentInfo_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// FindManifests provides a mock function for the type MockRepository\nfunc (_mock *MockRepository) FindManifests(ctx context.Context, labels map[string]string) ([]*manifest.EntryMetadata, error) {\n\tret := _mock.Called(ctx, labels)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for FindManifests\")\n\t}\n\n\tvar r0 []*manifest.EntryMetadata\n\tvar r1 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, map[string]string) ([]*manifest.EntryMetadata, error)); ok {\n\t\treturn returnFunc(ctx, labels)\n\t}\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, map[string]string) []*manifest.EntryMetadata); ok {\n\t\tr0 = returnFunc(ctx, labels)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]*manifest.EntryMetadata)\n\t\t}\n\t}\n\tif returnFunc, ok := ret.Get(1).(func(context.Context, map[string]string) error); ok {\n\t\tr1 = returnFunc(ctx, labels)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\treturn r0, r1\n}\n\n// MockRepository_FindManifests_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'FindManifests'\ntype MockRepository_FindManifests_Call struct {\n\t*mock.Call\n}\n\n// FindManifests is a helper method to define mock.On call\n// - ctx context.Context\n// - labels map[string]string\nfunc (_e *MockRepository_Expecter) FindManifests(ctx interface{}, labels interface{}) *MockRepository_FindManifests_Call {\n\treturn &MockRepository_FindManifests_Call{Call: _e.mock.On(\"FindManifests\", ctx, labels)}\n}\n\nfunc (_c *MockRepository_FindManifests_Call) Run(run func(ctx context.Context, labels map[string]string)) *MockRepository_FindManifests_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 map[string]string\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(map[string]string)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepository_FindManifests_Call) Return(entryMetadatas []*manifest.EntryMetadata, err error) *MockRepository_FindManifests_Call {\n\t_c.Call.Return(entryMetadatas, err)\n\treturn _c\n}\n\nfunc (_c *MockRepository_FindManifests_Call) RunAndReturn(run func(ctx context.Context, labels map[string]string) ([]*manifest.EntryMetadata, error)) *MockRepository_FindManifests_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// GetManifest provides a mock function for the type MockRepository\nfunc (_mock *MockRepository) GetManifest(ctx context.Context, id manifest.ID, data any) (*manifest.EntryMetadata, error) {\n\tret := _mock.Called(ctx, id, data)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetManifest\")\n\t}\n\n\tvar r0 *manifest.EntryMetadata\n\tvar r1 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, manifest.ID, any) (*manifest.EntryMetadata, error)); ok {\n\t\treturn returnFunc(ctx, id, data)\n\t}\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, manifest.ID, any) *manifest.EntryMetadata); ok {\n\t\tr0 = returnFunc(ctx, id, data)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(*manifest.EntryMetadata)\n\t\t}\n\t}\n\tif returnFunc, ok := ret.Get(1).(func(context.Context, manifest.ID, any) error); ok {\n\t\tr1 = returnFunc(ctx, id, data)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\treturn r0, r1\n}\n\n// MockRepository_GetManifest_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'GetManifest'\ntype MockRepository_GetManifest_Call struct {\n\t*mock.Call\n}\n\n// GetManifest is a helper method to define mock.On call\n// - ctx context.Context\n// - id manifest.ID\n// - data any\nfunc (_e *MockRepository_Expecter) GetManifest(ctx interface{}, id interface{}, data interface{}) *MockRepository_GetManifest_Call {\n\treturn &MockRepository_GetManifest_Call{Call: _e.mock.On(\"GetManifest\", ctx, id, data)}\n}\n\nfunc (_c *MockRepository_GetManifest_Call) Run(run func(ctx context.Context, id manifest.ID, data any)) *MockRepository_GetManifest_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 manifest.ID\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(manifest.ID)\n\t\t}\n\t\tvar arg2 any\n\t\tif args[2] != nil {\n\t\t\targ2 = args[2].(any)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t\targ2,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepository_GetManifest_Call) Return(entryMetadata *manifest.EntryMetadata, err error) *MockRepository_GetManifest_Call {\n\t_c.Call.Return(entryMetadata, err)\n\treturn _c\n}\n\nfunc (_c *MockRepository_GetManifest_Call) RunAndReturn(run func(ctx context.Context, id manifest.ID, data any) (*manifest.EntryMetadata, error)) *MockRepository_GetManifest_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// NewWriter provides a mock function for the type MockRepository\nfunc (_mock *MockRepository) NewWriter(ctx context.Context, opt repo.WriteSessionOptions) (context.Context, repo.RepositoryWriter, error) {\n\tret := _mock.Called(ctx, opt)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for NewWriter\")\n\t}\n\n\tvar r0 context.Context\n\tvar r1 repo.RepositoryWriter\n\tvar r2 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, repo.WriteSessionOptions) (context.Context, repo.RepositoryWriter, error)); ok {\n\t\treturn returnFunc(ctx, opt)\n\t}\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, repo.WriteSessionOptions) context.Context); ok {\n\t\tr0 = returnFunc(ctx, opt)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(context.Context)\n\t\t}\n\t}\n\tif returnFunc, ok := ret.Get(1).(func(context.Context, repo.WriteSessionOptions) repo.RepositoryWriter); ok {\n\t\tr1 = returnFunc(ctx, opt)\n\t} else {\n\t\tif ret.Get(1) != nil {\n\t\t\tr1 = ret.Get(1).(repo.RepositoryWriter)\n\t\t}\n\t}\n\tif returnFunc, ok := ret.Get(2).(func(context.Context, repo.WriteSessionOptions) error); ok {\n\t\tr2 = returnFunc(ctx, opt)\n\t} else {\n\t\tr2 = ret.Error(2)\n\t}\n\treturn r0, r1, r2\n}\n\n// MockRepository_NewWriter_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'NewWriter'\ntype MockRepository_NewWriter_Call struct {\n\t*mock.Call\n}\n\n// NewWriter is a helper method to define mock.On call\n// - ctx context.Context\n// - opt repo.WriteSessionOptions\nfunc (_e *MockRepository_Expecter) NewWriter(ctx interface{}, opt interface{}) *MockRepository_NewWriter_Call {\n\treturn &MockRepository_NewWriter_Call{Call: _e.mock.On(\"NewWriter\", ctx, opt)}\n}\n\nfunc (_c *MockRepository_NewWriter_Call) Run(run func(ctx context.Context, opt repo.WriteSessionOptions)) *MockRepository_NewWriter_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 repo.WriteSessionOptions\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(repo.WriteSessionOptions)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepository_NewWriter_Call) Return(context1 context.Context, repositoryWriter repo.RepositoryWriter, err error) *MockRepository_NewWriter_Call {\n\t_c.Call.Return(context1, repositoryWriter, err)\n\treturn _c\n}\n\nfunc (_c *MockRepository_NewWriter_Call) RunAndReturn(run func(ctx context.Context, opt repo.WriteSessionOptions) (context.Context, repo.RepositoryWriter, error)) *MockRepository_NewWriter_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// OpenObject provides a mock function for the type MockRepository\nfunc (_mock *MockRepository) OpenObject(ctx context.Context, id object.ID) (object.Reader, error) {\n\tret := _mock.Called(ctx, id)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for OpenObject\")\n\t}\n\n\tvar r0 object.Reader\n\tvar r1 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, object.ID) (object.Reader, error)); ok {\n\t\treturn returnFunc(ctx, id)\n\t}\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, object.ID) object.Reader); ok {\n\t\tr0 = returnFunc(ctx, id)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(object.Reader)\n\t\t}\n\t}\n\tif returnFunc, ok := ret.Get(1).(func(context.Context, object.ID) error); ok {\n\t\tr1 = returnFunc(ctx, id)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\treturn r0, r1\n}\n\n// MockRepository_OpenObject_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'OpenObject'\ntype MockRepository_OpenObject_Call struct {\n\t*mock.Call\n}\n\n// OpenObject is a helper method to define mock.On call\n// - ctx context.Context\n// - id object.ID\nfunc (_e *MockRepository_Expecter) OpenObject(ctx interface{}, id interface{}) *MockRepository_OpenObject_Call {\n\treturn &MockRepository_OpenObject_Call{Call: _e.mock.On(\"OpenObject\", ctx, id)}\n}\n\nfunc (_c *MockRepository_OpenObject_Call) Run(run func(ctx context.Context, id object.ID)) *MockRepository_OpenObject_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 object.ID\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(object.ID)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepository_OpenObject_Call) Return(reader object.Reader, err error) *MockRepository_OpenObject_Call {\n\t_c.Call.Return(reader, err)\n\treturn _c\n}\n\nfunc (_c *MockRepository_OpenObject_Call) RunAndReturn(run func(ctx context.Context, id object.ID) (object.Reader, error)) *MockRepository_OpenObject_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// PrefetchContents provides a mock function for the type MockRepository\nfunc (_mock *MockRepository) PrefetchContents(ctx context.Context, contentIDs []content.ID, hint string) []content.ID {\n\tret := _mock.Called(ctx, contentIDs, hint)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PrefetchContents\")\n\t}\n\n\tvar r0 []content.ID\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, []content.ID, string) []content.ID); ok {\n\t\tr0 = returnFunc(ctx, contentIDs, hint)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]content.ID)\n\t\t}\n\t}\n\treturn r0\n}\n\n// MockRepository_PrefetchContents_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'PrefetchContents'\ntype MockRepository_PrefetchContents_Call struct {\n\t*mock.Call\n}\n\n// PrefetchContents is a helper method to define mock.On call\n// - ctx context.Context\n// - contentIDs []content.ID\n// - hint string\nfunc (_e *MockRepository_Expecter) PrefetchContents(ctx interface{}, contentIDs interface{}, hint interface{}) *MockRepository_PrefetchContents_Call {\n\treturn &MockRepository_PrefetchContents_Call{Call: _e.mock.On(\"PrefetchContents\", ctx, contentIDs, hint)}\n}\n\nfunc (_c *MockRepository_PrefetchContents_Call) Run(run func(ctx context.Context, contentIDs []content.ID, hint string)) *MockRepository_PrefetchContents_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 []content.ID\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].([]content.ID)\n\t\t}\n\t\tvar arg2 string\n\t\tif args[2] != nil {\n\t\t\targ2 = args[2].(string)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t\targ2,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepository_PrefetchContents_Call) Return(vs []content.ID) *MockRepository_PrefetchContents_Call {\n\t_c.Call.Return(vs)\n\treturn _c\n}\n\nfunc (_c *MockRepository_PrefetchContents_Call) RunAndReturn(run func(ctx context.Context, contentIDs []content.ID, hint string) []content.ID) *MockRepository_PrefetchContents_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// PrefetchObjects provides a mock function for the type MockRepository\nfunc (_mock *MockRepository) PrefetchObjects(ctx context.Context, objectIDs []object.ID, hint string) ([]content.ID, error) {\n\tret := _mock.Called(ctx, objectIDs, hint)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PrefetchObjects\")\n\t}\n\n\tvar r0 []content.ID\n\tvar r1 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, []object.ID, string) ([]content.ID, error)); ok {\n\t\treturn returnFunc(ctx, objectIDs, hint)\n\t}\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, []object.ID, string) []content.ID); ok {\n\t\tr0 = returnFunc(ctx, objectIDs, hint)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]content.ID)\n\t\t}\n\t}\n\tif returnFunc, ok := ret.Get(1).(func(context.Context, []object.ID, string) error); ok {\n\t\tr1 = returnFunc(ctx, objectIDs, hint)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\treturn r0, r1\n}\n\n// MockRepository_PrefetchObjects_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'PrefetchObjects'\ntype MockRepository_PrefetchObjects_Call struct {\n\t*mock.Call\n}\n\n// PrefetchObjects is a helper method to define mock.On call\n// - ctx context.Context\n// - objectIDs []object.ID\n// - hint string\nfunc (_e *MockRepository_Expecter) PrefetchObjects(ctx interface{}, objectIDs interface{}, hint interface{}) *MockRepository_PrefetchObjects_Call {\n\treturn &MockRepository_PrefetchObjects_Call{Call: _e.mock.On(\"PrefetchObjects\", ctx, objectIDs, hint)}\n}\n\nfunc (_c *MockRepository_PrefetchObjects_Call) Run(run func(ctx context.Context, objectIDs []object.ID, hint string)) *MockRepository_PrefetchObjects_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 []object.ID\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].([]object.ID)\n\t\t}\n\t\tvar arg2 string\n\t\tif args[2] != nil {\n\t\t\targ2 = args[2].(string)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t\targ2,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepository_PrefetchObjects_Call) Return(vs []content.ID, err error) *MockRepository_PrefetchObjects_Call {\n\t_c.Call.Return(vs, err)\n\treturn _c\n}\n\nfunc (_c *MockRepository_PrefetchObjects_Call) RunAndReturn(run func(ctx context.Context, objectIDs []object.ID, hint string) ([]content.ID, error)) *MockRepository_PrefetchObjects_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// Refresh provides a mock function for the type MockRepository\nfunc (_mock *MockRepository) Refresh(ctx context.Context) error {\n\tret := _mock.Called(ctx)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Refresh\")\n\t}\n\n\tvar r0 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context) error); ok {\n\t\tr0 = returnFunc(ctx)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\treturn r0\n}\n\n// MockRepository_Refresh_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Refresh'\ntype MockRepository_Refresh_Call struct {\n\t*mock.Call\n}\n\n// Refresh is a helper method to define mock.On call\n// - ctx context.Context\nfunc (_e *MockRepository_Expecter) Refresh(ctx interface{}) *MockRepository_Refresh_Call {\n\treturn &MockRepository_Refresh_Call{Call: _e.mock.On(\"Refresh\", ctx)}\n}\n\nfunc (_c *MockRepository_Refresh_Call) Run(run func(ctx context.Context)) *MockRepository_Refresh_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepository_Refresh_Call) Return(err error) *MockRepository_Refresh_Call {\n\t_c.Call.Return(err)\n\treturn _c\n}\n\nfunc (_c *MockRepository_Refresh_Call) RunAndReturn(run func(ctx context.Context) error) *MockRepository_Refresh_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// Time provides a mock function for the type MockRepository\nfunc (_mock *MockRepository) Time() time.Time {\n\tret := _mock.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Time\")\n\t}\n\n\tvar r0 time.Time\n\tif returnFunc, ok := ret.Get(0).(func() time.Time); ok {\n\t\tr0 = returnFunc()\n\t} else {\n\t\tr0 = ret.Get(0).(time.Time)\n\t}\n\treturn r0\n}\n\n// MockRepository_Time_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Time'\ntype MockRepository_Time_Call struct {\n\t*mock.Call\n}\n\n// Time is a helper method to define mock.On call\nfunc (_e *MockRepository_Expecter) Time() *MockRepository_Time_Call {\n\treturn &MockRepository_Time_Call{Call: _e.mock.On(\"Time\")}\n}\n\nfunc (_c *MockRepository_Time_Call) Run(run func()) *MockRepository_Time_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\trun()\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepository_Time_Call) Return(time1 time.Time) *MockRepository_Time_Call {\n\t_c.Call.Return(time1)\n\treturn _c\n}\n\nfunc (_c *MockRepository_Time_Call) RunAndReturn(run func() time.Time) *MockRepository_Time_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// UpdateDescription provides a mock function for the type MockRepository\nfunc (_mock *MockRepository) UpdateDescription(d string) {\n\t_mock.Called(d)\n\treturn\n}\n\n// MockRepository_UpdateDescription_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'UpdateDescription'\ntype MockRepository_UpdateDescription_Call struct {\n\t*mock.Call\n}\n\n// UpdateDescription is a helper method to define mock.On call\n// - d string\nfunc (_e *MockRepository_Expecter) UpdateDescription(d interface{}) *MockRepository_UpdateDescription_Call {\n\treturn &MockRepository_UpdateDescription_Call{Call: _e.mock.On(\"UpdateDescription\", d)}\n}\n\nfunc (_c *MockRepository_UpdateDescription_Call) Run(run func(d string)) *MockRepository_UpdateDescription_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 string\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(string)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepository_UpdateDescription_Call) Return() *MockRepository_UpdateDescription_Call {\n\t_c.Call.Return()\n\treturn _c\n}\n\nfunc (_c *MockRepository_UpdateDescription_Call) RunAndReturn(run func(d string)) *MockRepository_UpdateDescription_Call {\n\t_c.Run(run)\n\treturn _c\n}\n\n// VerifyObject provides a mock function for the type MockRepository\nfunc (_mock *MockRepository) VerifyObject(ctx context.Context, id object.ID) ([]content.ID, error) {\n\tret := _mock.Called(ctx, id)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for VerifyObject\")\n\t}\n\n\tvar r0 []content.ID\n\tvar r1 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, object.ID) ([]content.ID, error)); ok {\n\t\treturn returnFunc(ctx, id)\n\t}\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, object.ID) []content.ID); ok {\n\t\tr0 = returnFunc(ctx, id)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]content.ID)\n\t\t}\n\t}\n\tif returnFunc, ok := ret.Get(1).(func(context.Context, object.ID) error); ok {\n\t\tr1 = returnFunc(ctx, id)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\treturn r0, r1\n}\n\n// MockRepository_VerifyObject_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'VerifyObject'\ntype MockRepository_VerifyObject_Call struct {\n\t*mock.Call\n}\n\n// VerifyObject is a helper method to define mock.On call\n// - ctx context.Context\n// - id object.ID\nfunc (_e *MockRepository_Expecter) VerifyObject(ctx interface{}, id interface{}) *MockRepository_VerifyObject_Call {\n\treturn &MockRepository_VerifyObject_Call{Call: _e.mock.On(\"VerifyObject\", ctx, id)}\n}\n\nfunc (_c *MockRepository_VerifyObject_Call) Run(run func(ctx context.Context, id object.ID)) *MockRepository_VerifyObject_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 object.ID\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(object.ID)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepository_VerifyObject_Call) Return(vs []content.ID, err error) *MockRepository_VerifyObject_Call {\n\t_c.Call.Return(vs, err)\n\treturn _c\n}\n\nfunc (_c *MockRepository_VerifyObject_Call) RunAndReturn(run func(ctx context.Context, id object.ID) ([]content.ID, error)) *MockRepository_VerifyObject_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/mocks/repository_writer.go", "content": "// Code generated by mockery; DO NOT EDIT.\n// github.com/vektra/mockery\n// template: testify\n\npackage mocks\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\t\"github.com/kopia/kopia/repo\"\n\t\"github.com/kopia/kopia/repo/content\"\n\t\"github.com/kopia/kopia/repo/manifest\"\n\t\"github.com/kopia/kopia/repo/object\"\n\tmock \"github.com/stretchr/testify/mock\"\n)\n\n// NewMockRepositoryWriter creates a new instance of MockRepositoryWriter. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\n// The first argument is typically a *testing.T value.\nfunc NewMockRepositoryWriter(t interface {\n\tmock.TestingT\n\tCleanup(func())\n}) *MockRepositoryWriter {\n\tmock := &MockRepositoryWriter{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n\n// MockRepositoryWriter is an autogenerated mock type for the RepositoryWriter type\ntype MockRepositoryWriter struct {\n\tmock.Mock\n}\n\ntype MockRepositoryWriter_Expecter struct {\n\tmock *mock.Mock\n}\n\nfunc (_m *MockRepositoryWriter) EXPECT() *MockRepositoryWriter_Expecter {\n\treturn &MockRepositoryWriter_Expecter{mock: &_m.Mock}\n}\n\n// ClientOptions provides a mock function for the type MockRepositoryWriter\nfunc (_mock *MockRepositoryWriter) ClientOptions() repo.ClientOptions {\n\tret := _mock.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for ClientOptions\")\n\t}\n\n\tvar r0 repo.ClientOptions\n\tif returnFunc, ok := ret.Get(0).(func() repo.ClientOptions); ok {\n\t\tr0 = returnFunc()\n\t} else {\n\t\tr0 = ret.Get(0).(repo.ClientOptions)\n\t}\n\treturn r0\n}\n\n// MockRepositoryWriter_ClientOptions_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'ClientOptions'\ntype MockRepositoryWriter_ClientOptions_Call struct {\n\t*mock.Call\n}\n\n// ClientOptions is a helper method to define mock.On call\nfunc (_e *MockRepositoryWriter_Expecter) ClientOptions() *MockRepositoryWriter_ClientOptions_Call {\n\treturn &MockRepositoryWriter_ClientOptions_Call{Call: _e.mock.On(\"ClientOptions\")}\n}\n\nfunc (_c *MockRepositoryWriter_ClientOptions_Call) Run(run func()) *MockRepositoryWriter_ClientOptions_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\trun()\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_ClientOptions_Call) Return(clientOptions repo.ClientOptions) *MockRepositoryWriter_ClientOptions_Call {\n\t_c.Call.Return(clientOptions)\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_ClientOptions_Call) RunAndReturn(run func() repo.ClientOptions) *MockRepositoryWriter_ClientOptions_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// Close provides a mock function for the type MockRepositoryWriter\nfunc (_mock *MockRepositoryWriter) Close(ctx context.Context) error {\n\tret := _mock.Called(ctx)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Close\")\n\t}\n\n\tvar r0 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context) error); ok {\n\t\tr0 = returnFunc(ctx)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\treturn r0\n}\n\n// MockRepositoryWriter_Close_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Close'\ntype MockRepositoryWriter_Close_Call struct {\n\t*mock.Call\n}\n\n// Close is a helper method to define mock.On call\n// - ctx context.Context\nfunc (_e *MockRepositoryWriter_Expecter) Close(ctx interface{}) *MockRepositoryWriter_Close_Call {\n\treturn &MockRepositoryWriter_Close_Call{Call: _e.mock.On(\"Close\", ctx)}\n}\n\nfunc (_c *MockRepositoryWriter_Close_Call) Run(run func(ctx context.Context)) *MockRepositoryWriter_Close_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_Close_Call) Return(err error) *MockRepositoryWriter_Close_Call {\n\t_c.Call.Return(err)\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_Close_Call) RunAndReturn(run func(ctx context.Context) error) *MockRepositoryWriter_Close_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// ConcatenateObjects provides a mock function for the type MockRepositoryWriter\nfunc (_mock *MockRepositoryWriter) ConcatenateObjects(ctx context.Context, objectIDs []object.ID, opt repo.ConcatenateOptions) (object.ID, error) {\n\tret := _mock.Called(ctx, objectIDs, opt)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for ConcatenateObjects\")\n\t}\n\n\tvar r0 object.ID\n\tvar r1 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, []object.ID, repo.ConcatenateOptions) (object.ID, error)); ok {\n\t\treturn returnFunc(ctx, objectIDs, opt)\n\t}\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, []object.ID, repo.ConcatenateOptions) object.ID); ok {\n\t\tr0 = returnFunc(ctx, objectIDs, opt)\n\t} else {\n\t\tr0 = ret.Get(0).(object.ID)\n\t}\n\tif returnFunc, ok := ret.Get(1).(func(context.Context, []object.ID, repo.ConcatenateOptions) error); ok {\n\t\tr1 = returnFunc(ctx, objectIDs, opt)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\treturn r0, r1\n}\n\n// MockRepositoryWriter_ConcatenateObjects_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'ConcatenateObjects'\ntype MockRepositoryWriter_ConcatenateObjects_Call struct {\n\t*mock.Call\n}\n\n// ConcatenateObjects is a helper method to define mock.On call\n// - ctx context.Context\n// - objectIDs []object.ID\n// - opt repo.ConcatenateOptions\nfunc (_e *MockRepositoryWriter_Expecter) ConcatenateObjects(ctx interface{}, objectIDs interface{}, opt interface{}) *MockRepositoryWriter_ConcatenateObjects_Call {\n\treturn &MockRepositoryWriter_ConcatenateObjects_Call{Call: _e.mock.On(\"ConcatenateObjects\", ctx, objectIDs, opt)}\n}\n\nfunc (_c *MockRepositoryWriter_ConcatenateObjects_Call) Run(run func(ctx context.Context, objectIDs []object.ID, opt repo.ConcatenateOptions)) *MockRepositoryWriter_ConcatenateObjects_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 []object.ID\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].([]object.ID)\n\t\t}\n\t\tvar arg2 repo.ConcatenateOptions\n\t\tif args[2] != nil {\n\t\t\targ2 = args[2].(repo.ConcatenateOptions)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t\targ2,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_ConcatenateObjects_Call) Return(iD object.ID, err error) *MockRepositoryWriter_ConcatenateObjects_Call {\n\t_c.Call.Return(iD, err)\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_ConcatenateObjects_Call) RunAndReturn(run func(ctx context.Context, objectIDs []object.ID, opt repo.ConcatenateOptions) (object.ID, error)) *MockRepositoryWriter_ConcatenateObjects_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// ContentInfo provides a mock function for the type MockRepositoryWriter\nfunc (_mock *MockRepositoryWriter) ContentInfo(ctx context.Context, contentID content.ID) (content.Info, error) {\n\tret := _mock.Called(ctx, contentID)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for ContentInfo\")\n\t}\n\n\tvar r0 content.Info\n\tvar r1 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, content.ID) (content.Info, error)); ok {\n\t\treturn returnFunc(ctx, contentID)\n\t}\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, content.ID) content.Info); ok {\n\t\tr0 = returnFunc(ctx, contentID)\n\t} else {\n\t\tr0 = ret.Get(0).(content.Info)\n\t}\n\tif returnFunc, ok := ret.Get(1).(func(context.Context, content.ID) error); ok {\n\t\tr1 = returnFunc(ctx, contentID)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\treturn r0, r1\n}\n\n// MockRepositoryWriter_ContentInfo_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'ContentInfo'\ntype MockRepositoryWriter_ContentInfo_Call struct {\n\t*mock.Call\n}\n\n// ContentInfo is a helper method to define mock.On call\n// - ctx context.Context\n// - contentID content.ID\nfunc (_e *MockRepositoryWriter_Expecter) ContentInfo(ctx interface{}, contentID interface{}) *MockRepositoryWriter_ContentInfo_Call {\n\treturn &MockRepositoryWriter_ContentInfo_Call{Call: _e.mock.On(\"ContentInfo\", ctx, contentID)}\n}\n\nfunc (_c *MockRepositoryWriter_ContentInfo_Call) Run(run func(ctx context.Context, contentID content.ID)) *MockRepositoryWriter_ContentInfo_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 content.ID\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(content.ID)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_ContentInfo_Call) Return(v content.Info, err error) *MockRepositoryWriter_ContentInfo_Call {\n\t_c.Call.Return(v, err)\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_ContentInfo_Call) RunAndReturn(run func(ctx context.Context, contentID content.ID) (content.Info, error)) *MockRepositoryWriter_ContentInfo_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// DeleteManifest provides a mock function for the type MockRepositoryWriter\nfunc (_mock *MockRepositoryWriter) DeleteManifest(ctx context.Context, id manifest.ID) error {\n\tret := _mock.Called(ctx, id)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for DeleteManifest\")\n\t}\n\n\tvar r0 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, manifest.ID) error); ok {\n\t\tr0 = returnFunc(ctx, id)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\treturn r0\n}\n\n// MockRepositoryWriter_DeleteManifest_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'DeleteManifest'\ntype MockRepositoryWriter_DeleteManifest_Call struct {\n\t*mock.Call\n}\n\n// DeleteManifest is a helper method to define mock.On call\n// - ctx context.Context\n// - id manifest.ID\nfunc (_e *MockRepositoryWriter_Expecter) DeleteManifest(ctx interface{}, id interface{}) *MockRepositoryWriter_DeleteManifest_Call {\n\treturn &MockRepositoryWriter_DeleteManifest_Call{Call: _e.mock.On(\"DeleteManifest\", ctx, id)}\n}\n\nfunc (_c *MockRepositoryWriter_DeleteManifest_Call) Run(run func(ctx context.Context, id manifest.ID)) *MockRepositoryWriter_DeleteManifest_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 manifest.ID\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(manifest.ID)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_DeleteManifest_Call) Return(err error) *MockRepositoryWriter_DeleteManifest_Call {\n\t_c.Call.Return(err)\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_DeleteManifest_Call) RunAndReturn(run func(ctx context.Context, id manifest.ID) error) *MockRepositoryWriter_DeleteManifest_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// FindManifests provides a mock function for the type MockRepositoryWriter\nfunc (_mock *MockRepositoryWriter) FindManifests(ctx context.Context, labels map[string]string) ([]*manifest.EntryMetadata, error) {\n\tret := _mock.Called(ctx, labels)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for FindManifests\")\n\t}\n\n\tvar r0 []*manifest.EntryMetadata\n\tvar r1 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, map[string]string) ([]*manifest.EntryMetadata, error)); ok {\n\t\treturn returnFunc(ctx, labels)\n\t}\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, map[string]string) []*manifest.EntryMetadata); ok {\n\t\tr0 = returnFunc(ctx, labels)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]*manifest.EntryMetadata)\n\t\t}\n\t}\n\tif returnFunc, ok := ret.Get(1).(func(context.Context, map[string]string) error); ok {\n\t\tr1 = returnFunc(ctx, labels)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\treturn r0, r1\n}\n\n// MockRepositoryWriter_FindManifests_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'FindManifests'\ntype MockRepositoryWriter_FindManifests_Call struct {\n\t*mock.Call\n}\n\n// FindManifests is a helper method to define mock.On call\n// - ctx context.Context\n// - labels map[string]string\nfunc (_e *MockRepositoryWriter_Expecter) FindManifests(ctx interface{}, labels interface{}) *MockRepositoryWriter_FindManifests_Call {\n\treturn &MockRepositoryWriter_FindManifests_Call{Call: _e.mock.On(\"FindManifests\", ctx, labels)}\n}\n\nfunc (_c *MockRepositoryWriter_FindManifests_Call) Run(run func(ctx context.Context, labels map[string]string)) *MockRepositoryWriter_FindManifests_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 map[string]string\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(map[string]string)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_FindManifests_Call) Return(entryMetadatas []*manifest.EntryMetadata, err error) *MockRepositoryWriter_FindManifests_Call {\n\t_c.Call.Return(entryMetadatas, err)\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_FindManifests_Call) RunAndReturn(run func(ctx context.Context, labels map[string]string) ([]*manifest.EntryMetadata, error)) *MockRepositoryWriter_FindManifests_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// Flush provides a mock function for the type MockRepositoryWriter\nfunc (_mock *MockRepositoryWriter) Flush(ctx context.Context) error {\n\tret := _mock.Called(ctx)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Flush\")\n\t}\n\n\tvar r0 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context) error); ok {\n\t\tr0 = returnFunc(ctx)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\treturn r0\n}\n\n// MockRepositoryWriter_Flush_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Flush'\ntype MockRepositoryWriter_Flush_Call struct {\n\t*mock.Call\n}\n\n// Flush is a helper method to define mock.On call\n// - ctx context.Context\nfunc (_e *MockRepositoryWriter_Expecter) Flush(ctx interface{}) *MockRepositoryWriter_Flush_Call {\n\treturn &MockRepositoryWriter_Flush_Call{Call: _e.mock.On(\"Flush\", ctx)}\n}\n\nfunc (_c *MockRepositoryWriter_Flush_Call) Run(run func(ctx context.Context)) *MockRepositoryWriter_Flush_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_Flush_Call) Return(err error) *MockRepositoryWriter_Flush_Call {\n\t_c.Call.Return(err)\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_Flush_Call) RunAndReturn(run func(ctx context.Context) error) *MockRepositoryWriter_Flush_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// GetManifest provides a mock function for the type MockRepositoryWriter\nfunc (_mock *MockRepositoryWriter) GetManifest(ctx context.Context, id manifest.ID, data any) (*manifest.EntryMetadata, error) {\n\tret := _mock.Called(ctx, id, data)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetManifest\")\n\t}\n\n\tvar r0 *manifest.EntryMetadata\n\tvar r1 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, manifest.ID, any) (*manifest.EntryMetadata, error)); ok {\n\t\treturn returnFunc(ctx, id, data)\n\t}\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, manifest.ID, any) *manifest.EntryMetadata); ok {\n\t\tr0 = returnFunc(ctx, id, data)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(*manifest.EntryMetadata)\n\t\t}\n\t}\n\tif returnFunc, ok := ret.Get(1).(func(context.Context, manifest.ID, any) error); ok {\n\t\tr1 = returnFunc(ctx, id, data)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\treturn r0, r1\n}\n\n// MockRepositoryWriter_GetManifest_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'GetManifest'\ntype MockRepositoryWriter_GetManifest_Call struct {\n\t*mock.Call\n}\n\n// GetManifest is a helper method to define mock.On call\n// - ctx context.Context\n// - id manifest.ID\n// - data any\nfunc (_e *MockRepositoryWriter_Expecter) GetManifest(ctx interface{}, id interface{}, data interface{}) *MockRepositoryWriter_GetManifest_Call {\n\treturn &MockRepositoryWriter_GetManifest_Call{Call: _e.mock.On(\"GetManifest\", ctx, id, data)}\n}\n\nfunc (_c *MockRepositoryWriter_GetManifest_Call) Run(run func(ctx context.Context, id manifest.ID, data any)) *MockRepositoryWriter_GetManifest_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 manifest.ID\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(manifest.ID)\n\t\t}\n\t\tvar arg2 any\n\t\tif args[2] != nil {\n\t\t\targ2 = args[2].(any)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t\targ2,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_GetManifest_Call) Return(entryMetadata *manifest.EntryMetadata, err error) *MockRepositoryWriter_GetManifest_Call {\n\t_c.Call.Return(entryMetadata, err)\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_GetManifest_Call) RunAndReturn(run func(ctx context.Context, id manifest.ID, data any) (*manifest.EntryMetadata, error)) *MockRepositoryWriter_GetManifest_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// NewObjectWriter provides a mock function for the type MockRepositoryWriter\nfunc (_mock *MockRepositoryWriter) NewObjectWriter(ctx context.Context, opt object.WriterOptions) object.Writer {\n\tret := _mock.Called(ctx, opt)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for NewObjectWriter\")\n\t}\n\n\tvar r0 object.Writer\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, object.WriterOptions) object.Writer); ok {\n\t\tr0 = returnFunc(ctx, opt)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(object.Writer)\n\t\t}\n\t}\n\treturn r0\n}\n\n// MockRepositoryWriter_NewObjectWriter_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'NewObjectWriter'\ntype MockRepositoryWriter_NewObjectWriter_Call struct {\n\t*mock.Call\n}\n\n// NewObjectWriter is a helper method to define mock.On call\n// - ctx context.Context\n// - opt object.WriterOptions\nfunc (_e *MockRepositoryWriter_Expecter) NewObjectWriter(ctx interface{}, opt interface{}) *MockRepositoryWriter_NewObjectWriter_Call {\n\treturn &MockRepositoryWriter_NewObjectWriter_Call{Call: _e.mock.On(\"NewObjectWriter\", ctx, opt)}\n}\n\nfunc (_c *MockRepositoryWriter_NewObjectWriter_Call) Run(run func(ctx context.Context, opt object.WriterOptions)) *MockRepositoryWriter_NewObjectWriter_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 object.WriterOptions\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(object.WriterOptions)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_NewObjectWriter_Call) Return(writer object.Writer) *MockRepositoryWriter_NewObjectWriter_Call {\n\t_c.Call.Return(writer)\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_NewObjectWriter_Call) RunAndReturn(run func(ctx context.Context, opt object.WriterOptions) object.Writer) *MockRepositoryWriter_NewObjectWriter_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// NewWriter provides a mock function for the type MockRepositoryWriter\nfunc (_mock *MockRepositoryWriter) NewWriter(ctx context.Context, opt repo.WriteSessionOptions) (context.Context, repo.RepositoryWriter, error) {\n\tret := _mock.Called(ctx, opt)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for NewWriter\")\n\t}\n\n\tvar r0 context.Context\n\tvar r1 repo.RepositoryWriter\n\tvar r2 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, repo.WriteSessionOptions) (context.Context, repo.RepositoryWriter, error)); ok {\n\t\treturn returnFunc(ctx, opt)\n\t}\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, repo.WriteSessionOptions) context.Context); ok {\n\t\tr0 = returnFunc(ctx, opt)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(context.Context)\n\t\t}\n\t}\n\tif returnFunc, ok := ret.Get(1).(func(context.Context, repo.WriteSessionOptions) repo.RepositoryWriter); ok {\n\t\tr1 = returnFunc(ctx, opt)\n\t} else {\n\t\tif ret.Get(1) != nil {\n\t\t\tr1 = ret.Get(1).(repo.RepositoryWriter)\n\t\t}\n\t}\n\tif returnFunc, ok := ret.Get(2).(func(context.Context, repo.WriteSessionOptions) error); ok {\n\t\tr2 = returnFunc(ctx, opt)\n\t} else {\n\t\tr2 = ret.Error(2)\n\t}\n\treturn r0, r1, r2\n}\n\n// MockRepositoryWriter_NewWriter_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'NewWriter'\ntype MockRepositoryWriter_NewWriter_Call struct {\n\t*mock.Call\n}\n\n// NewWriter is a helper method to define mock.On call\n// - ctx context.Context\n// - opt repo.WriteSessionOptions\nfunc (_e *MockRepositoryWriter_Expecter) NewWriter(ctx interface{}, opt interface{}) *MockRepositoryWriter_NewWriter_Call {\n\treturn &MockRepositoryWriter_NewWriter_Call{Call: _e.mock.On(\"NewWriter\", ctx, opt)}\n}\n\nfunc (_c *MockRepositoryWriter_NewWriter_Call) Run(run func(ctx context.Context, opt repo.WriteSessionOptions)) *MockRepositoryWriter_NewWriter_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 repo.WriteSessionOptions\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(repo.WriteSessionOptions)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_NewWriter_Call) Return(context1 context.Context, repositoryWriter repo.RepositoryWriter, err error) *MockRepositoryWriter_NewWriter_Call {\n\t_c.Call.Return(context1, repositoryWriter, err)\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_NewWriter_Call) RunAndReturn(run func(ctx context.Context, opt repo.WriteSessionOptions) (context.Context, repo.RepositoryWriter, error)) *MockRepositoryWriter_NewWriter_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// OnSuccessfulFlush provides a mock function for the type MockRepositoryWriter\nfunc (_mock *MockRepositoryWriter) OnSuccessfulFlush(callback repo.RepositoryWriterCallback) {\n\t_mock.Called(callback)\n\treturn\n}\n\n// MockRepositoryWriter_OnSuccessfulFlush_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'OnSuccessfulFlush'\ntype MockRepositoryWriter_OnSuccessfulFlush_Call struct {\n\t*mock.Call\n}\n\n// OnSuccessfulFlush is a helper method to define mock.On call\n// - callback repo.RepositoryWriterCallback\nfunc (_e *MockRepositoryWriter_Expecter) OnSuccessfulFlush(callback interface{}) *MockRepositoryWriter_OnSuccessfulFlush_Call {\n\treturn &MockRepositoryWriter_OnSuccessfulFlush_Call{Call: _e.mock.On(\"OnSuccessfulFlush\", callback)}\n}\n\nfunc (_c *MockRepositoryWriter_OnSuccessfulFlush_Call) Run(run func(callback repo.RepositoryWriterCallback)) *MockRepositoryWriter_OnSuccessfulFlush_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 repo.RepositoryWriterCallback\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(repo.RepositoryWriterCallback)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_OnSuccessfulFlush_Call) Return() *MockRepositoryWriter_OnSuccessfulFlush_Call {\n\t_c.Call.Return()\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_OnSuccessfulFlush_Call) RunAndReturn(run func(callback repo.RepositoryWriterCallback)) *MockRepositoryWriter_OnSuccessfulFlush_Call {\n\t_c.Run(run)\n\treturn _c\n}\n\n// OpenObject provides a mock function for the type MockRepositoryWriter\nfunc (_mock *MockRepositoryWriter) OpenObject(ctx context.Context, id object.ID) (object.Reader, error) {\n\tret := _mock.Called(ctx, id)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for OpenObject\")\n\t}\n\n\tvar r0 object.Reader\n\tvar r1 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, object.ID) (object.Reader, error)); ok {\n\t\treturn returnFunc(ctx, id)\n\t}\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, object.ID) object.Reader); ok {\n\t\tr0 = returnFunc(ctx, id)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(object.Reader)\n\t\t}\n\t}\n\tif returnFunc, ok := ret.Get(1).(func(context.Context, object.ID) error); ok {\n\t\tr1 = returnFunc(ctx, id)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\treturn r0, r1\n}\n\n// MockRepositoryWriter_OpenObject_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'OpenObject'\ntype MockRepositoryWriter_OpenObject_Call struct {\n\t*mock.Call\n}\n\n// OpenObject is a helper method to define mock.On call\n// - ctx context.Context\n// - id object.ID\nfunc (_e *MockRepositoryWriter_Expecter) OpenObject(ctx interface{}, id interface{}) *MockRepositoryWriter_OpenObject_Call {\n\treturn &MockRepositoryWriter_OpenObject_Call{Call: _e.mock.On(\"OpenObject\", ctx, id)}\n}\n\nfunc (_c *MockRepositoryWriter_OpenObject_Call) Run(run func(ctx context.Context, id object.ID)) *MockRepositoryWriter_OpenObject_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 object.ID\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(object.ID)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_OpenObject_Call) Return(reader object.Reader, err error) *MockRepositoryWriter_OpenObject_Call {\n\t_c.Call.Return(reader, err)\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_OpenObject_Call) RunAndReturn(run func(ctx context.Context, id object.ID) (object.Reader, error)) *MockRepositoryWriter_OpenObject_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// PrefetchContents provides a mock function for the type MockRepositoryWriter\nfunc (_mock *MockRepositoryWriter) PrefetchContents(ctx context.Context, contentIDs []content.ID, hint string) []content.ID {\n\tret := _mock.Called(ctx, contentIDs, hint)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PrefetchContents\")\n\t}\n\n\tvar r0 []content.ID\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, []content.ID, string) []content.ID); ok {\n\t\tr0 = returnFunc(ctx, contentIDs, hint)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]content.ID)\n\t\t}\n\t}\n\treturn r0\n}\n\n// MockRepositoryWriter_PrefetchContents_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'PrefetchContents'\ntype MockRepositoryWriter_PrefetchContents_Call struct {\n\t*mock.Call\n}\n\n// PrefetchContents is a helper method to define mock.On call\n// - ctx context.Context\n// - contentIDs []content.ID\n// - hint string\nfunc (_e *MockRepositoryWriter_Expecter) PrefetchContents(ctx interface{}, contentIDs interface{}, hint interface{}) *MockRepositoryWriter_PrefetchContents_Call {\n\treturn &MockRepositoryWriter_PrefetchContents_Call{Call: _e.mock.On(\"PrefetchContents\", ctx, contentIDs, hint)}\n}\n\nfunc (_c *MockRepositoryWriter_PrefetchContents_Call) Run(run func(ctx context.Context, contentIDs []content.ID, hint string)) *MockRepositoryWriter_PrefetchContents_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 []content.ID\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].([]content.ID)\n\t\t}\n\t\tvar arg2 string\n\t\tif args[2] != nil {\n\t\t\targ2 = args[2].(string)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t\targ2,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_PrefetchContents_Call) Return(vs []content.ID) *MockRepositoryWriter_PrefetchContents_Call {\n\t_c.Call.Return(vs)\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_PrefetchContents_Call) RunAndReturn(run func(ctx context.Context, contentIDs []content.ID, hint string) []content.ID) *MockRepositoryWriter_PrefetchContents_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// PrefetchObjects provides a mock function for the type MockRepositoryWriter\nfunc (_mock *MockRepositoryWriter) PrefetchObjects(ctx context.Context, objectIDs []object.ID, hint string) ([]content.ID, error) {\n\tret := _mock.Called(ctx, objectIDs, hint)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PrefetchObjects\")\n\t}\n\n\tvar r0 []content.ID\n\tvar r1 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, []object.ID, string) ([]content.ID, error)); ok {\n\t\treturn returnFunc(ctx, objectIDs, hint)\n\t}\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, []object.ID, string) []content.ID); ok {\n\t\tr0 = returnFunc(ctx, objectIDs, hint)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]content.ID)\n\t\t}\n\t}\n\tif returnFunc, ok := ret.Get(1).(func(context.Context, []object.ID, string) error); ok {\n\t\tr1 = returnFunc(ctx, objectIDs, hint)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\treturn r0, r1\n}\n\n// MockRepositoryWriter_PrefetchObjects_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'PrefetchObjects'\ntype MockRepositoryWriter_PrefetchObjects_Call struct {\n\t*mock.Call\n}\n\n// PrefetchObjects is a helper method to define mock.On call\n// - ctx context.Context\n// - objectIDs []object.ID\n// - hint string\nfunc (_e *MockRepositoryWriter_Expecter) PrefetchObjects(ctx interface{}, objectIDs interface{}, hint interface{}) *MockRepositoryWriter_PrefetchObjects_Call {\n\treturn &MockRepositoryWriter_PrefetchObjects_Call{Call: _e.mock.On(\"PrefetchObjects\", ctx, objectIDs, hint)}\n}\n\nfunc (_c *MockRepositoryWriter_PrefetchObjects_Call) Run(run func(ctx context.Context, objectIDs []object.ID, hint string)) *MockRepositoryWriter_PrefetchObjects_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 []object.ID\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].([]object.ID)\n\t\t}\n\t\tvar arg2 string\n\t\tif args[2] != nil {\n\t\t\targ2 = args[2].(string)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t\targ2,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_PrefetchObjects_Call) Return(vs []content.ID, err error) *MockRepositoryWriter_PrefetchObjects_Call {\n\t_c.Call.Return(vs, err)\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_PrefetchObjects_Call) RunAndReturn(run func(ctx context.Context, objectIDs []object.ID, hint string) ([]content.ID, error)) *MockRepositoryWriter_PrefetchObjects_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// PutManifest provides a mock function for the type MockRepositoryWriter\nfunc (_mock *MockRepositoryWriter) PutManifest(ctx context.Context, labels map[string]string, payload any) (manifest.ID, error) {\n\tret := _mock.Called(ctx, labels, payload)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PutManifest\")\n\t}\n\n\tvar r0 manifest.ID\n\tvar r1 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, map[string]string, any) (manifest.ID, error)); ok {\n\t\treturn returnFunc(ctx, labels, payload)\n\t}\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, map[string]string, any) manifest.ID); ok {\n\t\tr0 = returnFunc(ctx, labels, payload)\n\t} else {\n\t\tr0 = ret.Get(0).(manifest.ID)\n\t}\n\tif returnFunc, ok := ret.Get(1).(func(context.Context, map[string]string, any) error); ok {\n\t\tr1 = returnFunc(ctx, labels, payload)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\treturn r0, r1\n}\n\n// MockRepositoryWriter_PutManifest_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'PutManifest'\ntype MockRepositoryWriter_PutManifest_Call struct {\n\t*mock.Call\n}\n\n// PutManifest is a helper method to define mock.On call\n// - ctx context.Context\n// - labels map[string]string\n// - payload any\nfunc (_e *MockRepositoryWriter_Expecter) PutManifest(ctx interface{}, labels interface{}, payload interface{}) *MockRepositoryWriter_PutManifest_Call {\n\treturn &MockRepositoryWriter_PutManifest_Call{Call: _e.mock.On(\"PutManifest\", ctx, labels, payload)}\n}\n\nfunc (_c *MockRepositoryWriter_PutManifest_Call) Run(run func(ctx context.Context, labels map[string]string, payload any)) *MockRepositoryWriter_PutManifest_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 map[string]string\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(map[string]string)\n\t\t}\n\t\tvar arg2 any\n\t\tif args[2] != nil {\n\t\t\targ2 = args[2].(any)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t\targ2,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_PutManifest_Call) Return(iD manifest.ID, err error) *MockRepositoryWriter_PutManifest_Call {\n\t_c.Call.Return(iD, err)\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_PutManifest_Call) RunAndReturn(run func(ctx context.Context, labels map[string]string, payload any) (manifest.ID, error)) *MockRepositoryWriter_PutManifest_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// Refresh provides a mock function for the type MockRepositoryWriter\nfunc (_mock *MockRepositoryWriter) Refresh(ctx context.Context) error {\n\tret := _mock.Called(ctx)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Refresh\")\n\t}\n\n\tvar r0 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context) error); ok {\n\t\tr0 = returnFunc(ctx)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\treturn r0\n}\n\n// MockRepositoryWriter_Refresh_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Refresh'\ntype MockRepositoryWriter_Refresh_Call struct {\n\t*mock.Call\n}\n\n// Refresh is a helper method to define mock.On call\n// - ctx context.Context\nfunc (_e *MockRepositoryWriter_Expecter) Refresh(ctx interface{}) *MockRepositoryWriter_Refresh_Call {\n\treturn &MockRepositoryWriter_Refresh_Call{Call: _e.mock.On(\"Refresh\", ctx)}\n}\n\nfunc (_c *MockRepositoryWriter_Refresh_Call) Run(run func(ctx context.Context)) *MockRepositoryWriter_Refresh_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_Refresh_Call) Return(err error) *MockRepositoryWriter_Refresh_Call {\n\t_c.Call.Return(err)\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_Refresh_Call) RunAndReturn(run func(ctx context.Context) error) *MockRepositoryWriter_Refresh_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// ReplaceManifests provides a mock function for the type MockRepositoryWriter\nfunc (_mock *MockRepositoryWriter) ReplaceManifests(ctx context.Context, labels map[string]string, payload any) (manifest.ID, error) {\n\tret := _mock.Called(ctx, labels, payload)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for ReplaceManifests\")\n\t}\n\n\tvar r0 manifest.ID\n\tvar r1 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, map[string]string, any) (manifest.ID, error)); ok {\n\t\treturn returnFunc(ctx, labels, payload)\n\t}\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, map[string]string, any) manifest.ID); ok {\n\t\tr0 = returnFunc(ctx, labels, payload)\n\t} else {\n\t\tr0 = ret.Get(0).(manifest.ID)\n\t}\n\tif returnFunc, ok := ret.Get(1).(func(context.Context, map[string]string, any) error); ok {\n\t\tr1 = returnFunc(ctx, labels, payload)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\treturn r0, r1\n}\n\n// MockRepositoryWriter_ReplaceManifests_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'ReplaceManifests'\ntype MockRepositoryWriter_ReplaceManifests_Call struct {\n\t*mock.Call\n}\n\n// ReplaceManifests is a helper method to define mock.On call\n// - ctx context.Context\n// - labels map[string]string\n// - payload any\nfunc (_e *MockRepositoryWriter_Expecter) ReplaceManifests(ctx interface{}, labels interface{}, payload interface{}) *MockRepositoryWriter_ReplaceManifests_Call {\n\treturn &MockRepositoryWriter_ReplaceManifests_Call{Call: _e.mock.On(\"ReplaceManifests\", ctx, labels, payload)}\n}\n\nfunc (_c *MockRepositoryWriter_ReplaceManifests_Call) Run(run func(ctx context.Context, labels map[string]string, payload any)) *MockRepositoryWriter_ReplaceManifests_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 map[string]string\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(map[string]string)\n\t\t}\n\t\tvar arg2 any\n\t\tif args[2] != nil {\n\t\t\targ2 = args[2].(any)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t\targ2,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_ReplaceManifests_Call) Return(iD manifest.ID, err error) *MockRepositoryWriter_ReplaceManifests_Call {\n\t_c.Call.Return(iD, err)\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_ReplaceManifests_Call) RunAndReturn(run func(ctx context.Context, labels map[string]string, payload any) (manifest.ID, error)) *MockRepositoryWriter_ReplaceManifests_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// Time provides a mock function for the type MockRepositoryWriter\nfunc (_mock *MockRepositoryWriter) Time() time.Time {\n\tret := _mock.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Time\")\n\t}\n\n\tvar r0 time.Time\n\tif returnFunc, ok := ret.Get(0).(func() time.Time); ok {\n\t\tr0 = returnFunc()\n\t} else {\n\t\tr0 = ret.Get(0).(time.Time)\n\t}\n\treturn r0\n}\n\n// MockRepositoryWriter_Time_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Time'\ntype MockRepositoryWriter_Time_Call struct {\n\t*mock.Call\n}\n\n// Time is a helper method to define mock.On call\nfunc (_e *MockRepositoryWriter_Expecter) Time() *MockRepositoryWriter_Time_Call {\n\treturn &MockRepositoryWriter_Time_Call{Call: _e.mock.On(\"Time\")}\n}\n\nfunc (_c *MockRepositoryWriter_Time_Call) Run(run func()) *MockRepositoryWriter_Time_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\trun()\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_Time_Call) Return(time1 time.Time) *MockRepositoryWriter_Time_Call {\n\t_c.Call.Return(time1)\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_Time_Call) RunAndReturn(run func() time.Time) *MockRepositoryWriter_Time_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n\n// UpdateDescription provides a mock function for the type MockRepositoryWriter\nfunc (_mock *MockRepositoryWriter) UpdateDescription(d string) {\n\t_mock.Called(d)\n\treturn\n}\n\n// MockRepositoryWriter_UpdateDescription_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'UpdateDescription'\ntype MockRepositoryWriter_UpdateDescription_Call struct {\n\t*mock.Call\n}\n\n// UpdateDescription is a helper method to define mock.On call\n// - d string\nfunc (_e *MockRepositoryWriter_Expecter) UpdateDescription(d interface{}) *MockRepositoryWriter_UpdateDescription_Call {\n\treturn &MockRepositoryWriter_UpdateDescription_Call{Call: _e.mock.On(\"UpdateDescription\", d)}\n}\n\nfunc (_c *MockRepositoryWriter_UpdateDescription_Call) Run(run func(d string)) *MockRepositoryWriter_UpdateDescription_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 string\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(string)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_UpdateDescription_Call) Return() *MockRepositoryWriter_UpdateDescription_Call {\n\t_c.Call.Return()\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_UpdateDescription_Call) RunAndReturn(run func(d string)) *MockRepositoryWriter_UpdateDescription_Call {\n\t_c.Run(run)\n\treturn _c\n}\n\n// VerifyObject provides a mock function for the type MockRepositoryWriter\nfunc (_mock *MockRepositoryWriter) VerifyObject(ctx context.Context, id object.ID) ([]content.ID, error) {\n\tret := _mock.Called(ctx, id)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for VerifyObject\")\n\t}\n\n\tvar r0 []content.ID\n\tvar r1 error\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, object.ID) ([]content.ID, error)); ok {\n\t\treturn returnFunc(ctx, id)\n\t}\n\tif returnFunc, ok := ret.Get(0).(func(context.Context, object.ID) []content.ID); ok {\n\t\tr0 = returnFunc(ctx, id)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]content.ID)\n\t\t}\n\t}\n\tif returnFunc, ok := ret.Get(1).(func(context.Context, object.ID) error); ok {\n\t\tr1 = returnFunc(ctx, id)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\treturn r0, r1\n}\n\n// MockRepositoryWriter_VerifyObject_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'VerifyObject'\ntype MockRepositoryWriter_VerifyObject_Call struct {\n\t*mock.Call\n}\n\n// VerifyObject is a helper method to define mock.On call\n// - ctx context.Context\n// - id object.ID\nfunc (_e *MockRepositoryWriter_Expecter) VerifyObject(ctx interface{}, id interface{}) *MockRepositoryWriter_VerifyObject_Call {\n\treturn &MockRepositoryWriter_VerifyObject_Call{Call: _e.mock.On(\"VerifyObject\", ctx, id)}\n}\n\nfunc (_c *MockRepositoryWriter_VerifyObject_Call) Run(run func(ctx context.Context, id object.ID)) *MockRepositoryWriter_VerifyObject_Call {\n\t_c.Call.Run(func(args mock.Arguments) {\n\t\tvar arg0 context.Context\n\t\tif args[0] != nil {\n\t\t\targ0 = args[0].(context.Context)\n\t\t}\n\t\tvar arg1 object.ID\n\t\tif args[1] != nil {\n\t\t\targ1 = args[1].(object.ID)\n\t\t}\n\t\trun(\n\t\t\targ0,\n\t\t\targ1,\n\t\t)\n\t})\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_VerifyObject_Call) Return(vs []content.ID, err error) *MockRepositoryWriter_VerifyObject_Call {\n\t_c.Call.Return(vs, err)\n\treturn _c\n}\n\nfunc (_c *MockRepositoryWriter_VerifyObject_Call) RunAndReturn(run func(ctx context.Context, id object.ID) ([]content.ID, error)) *MockRepositoryWriter_VerifyObject_Call {\n\t_c.Call.Return(run)\n\treturn _c\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/s3.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backend\n\nimport (\n\t\"context\"\n\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/kopia/kopia/repo/blob\"\n\t\"github.com/kopia/kopia/repo/blob/s3\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo/kopialib/backend/logging\"\n)\n\ntype S3Backend struct {\n\toptions s3.Options\n}\n\nfunc (c *S3Backend) Setup(ctx context.Context, flags map[string]string, logger logrus.FieldLogger) error {\n\tvar err error\n\tc.options.BucketName, err = mustHaveString(udmrepo.StoreOptionOssBucket, flags)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tc.options.AccessKeyID = optionalHaveString(udmrepo.StoreOptionS3KeyID, flags)\n\tc.options.SecretAccessKey = optionalHaveString(udmrepo.StoreOptionS3SecretKey, flags)\n\tc.options.Endpoint = optionalHaveString(udmrepo.StoreOptionS3Endpoint, flags)\n\tc.options.Region = optionalHaveString(udmrepo.StoreOptionOssRegion, flags)\n\tc.options.Prefix = optionalHaveString(udmrepo.StoreOptionPrefix, flags)\n\tc.options.DoNotUseTLS = optionalHaveBool(ctx, udmrepo.StoreOptionS3DisableTLS, flags)\n\tc.options.DoNotVerifyTLS = optionalHaveBool(ctx, udmrepo.StoreOptionS3DisableTLSVerify, flags)\n\tc.options.SessionToken = optionalHaveString(udmrepo.StoreOptionS3Token, flags)\n\tc.options.RootCA = optionalHaveBase64(ctx, udmrepo.StoreOptionCACert, flags)\n\n\tctx = logging.WithLogger(ctx, logger)\n\n\tc.options.Limits = setupLimits(ctx, flags)\n\n\treturn nil\n}\n\nfunc (c *S3Backend) Connect(ctx context.Context, isCreate bool, logger logrus.FieldLogger) (blob.Storage, error) {\n\tctx = logging.WithLogger(ctx, logger)\n\treturn s3.New(ctx, &c.options, false)\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/s3_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backend\n\nimport (\n\t\"testing\"\n\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\n\t\"github.com/kopia/kopia/repo/blob/s3\"\n\t\"github.com/stretchr/testify/assert\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n)\n\nfunc TestS3Setup(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tflags map[string]string\n\t\texpectedOptions s3.Options\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"must have bucket name\",\n\t\t\tflags: map[string]string{},\n\t\t\texpectedErr: \"key \" + udmrepo.StoreOptionOssBucket + \" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"with bucket only\",\n\t\t\tflags: map[string]string{\n\t\t\t\tudmrepo.StoreOptionOssBucket: \"fake-bucket\",\n\t\t\t},\n\t\t\texpectedOptions: s3.Options{\n\t\t\t\tBucketName: \"fake-bucket\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with others\",\n\t\t\tflags: map[string]string{\n\t\t\t\tudmrepo.StoreOptionOssBucket: \"fake-bucket\",\n\t\t\t\tudmrepo.StoreOptionS3KeyID: \"fake-ak\",\n\t\t\t\tudmrepo.StoreOptionS3SecretKey: \"fake-sk\",\n\t\t\t\tudmrepo.StoreOptionS3Endpoint: \"fake-endpoint\",\n\t\t\t\tudmrepo.StoreOptionOssRegion: \"fake-region\",\n\t\t\t\tudmrepo.StoreOptionPrefix: \"fake-prefix\",\n\t\t\t\tudmrepo.StoreOptionS3Token: \"fake-token\",\n\t\t\t},\n\t\t\texpectedOptions: s3.Options{\n\t\t\t\tBucketName: \"fake-bucket\",\n\t\t\t\tAccessKeyID: \"fake-ak\",\n\t\t\t\tSecretAccessKey: \"fake-sk\",\n\t\t\t\tEndpoint: \"fake-endpoint\",\n\t\t\t\tRegion: \"fake-region\",\n\t\t\t\tPrefix: \"fake-prefix\",\n\t\t\t\tSessionToken: \"fake-token\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with wrong tls\",\n\t\t\tflags: map[string]string{\n\t\t\t\tudmrepo.StoreOptionOssBucket: \"fake-bucket\",\n\t\t\t\tudmrepo.StoreOptionS3DisableTLS: \"fake-bool\",\n\t\t\t\tudmrepo.StoreOptionS3DisableTLSVerify: \"fake-bool\",\n\t\t\t},\n\t\t\texpectedOptions: s3.Options{\n\t\t\t\tBucketName: \"fake-bucket\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with correct tls\",\n\t\t\tflags: map[string]string{\n\t\t\t\tudmrepo.StoreOptionOssBucket: \"fake-bucket\",\n\t\t\t\tudmrepo.StoreOptionS3DisableTLS: \"true\",\n\t\t\t\tudmrepo.StoreOptionS3DisableTLSVerify: \"false\",\n\t\t\t},\n\t\t\texpectedOptions: s3.Options{\n\t\t\t\tBucketName: \"fake-bucket\",\n\t\t\t\tDoNotUseTLS: true,\n\t\t\t\tDoNotVerifyTLS: false,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with wrong ca\",\n\t\t\tflags: map[string]string{\n\t\t\t\tudmrepo.StoreOptionOssBucket: \"fake-bucket\",\n\t\t\t\tudmrepo.StoreOptionCACert: \"fake-base-64\",\n\t\t\t},\n\t\t\texpectedOptions: s3.Options{\n\t\t\t\tBucketName: \"fake-bucket\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with correct ca\",\n\t\t\tflags: map[string]string{\n\t\t\t\tudmrepo.StoreOptionOssBucket: \"fake-bucket\",\n\t\t\t\tudmrepo.StoreOptionCACert: \"ZmFrZS1jYQ==\",\n\t\t\t},\n\t\t\texpectedOptions: s3.Options{\n\t\t\t\tBucketName: \"fake-bucket\",\n\t\t\t\tRootCA: []byte{'f', 'a', 'k', 'e', '-', 'c', 'a'},\n\t\t\t},\n\t\t},\n\t}\n\n\tlogger := velerotest.NewLogger()\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\ts3Flags := S3Backend{}\n\n\t\t\terr := s3Flags.Setup(t.Context(), tc.flags, logger)\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/utils.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backend\n\nimport (\n\t\"context\"\n\t\"encoding/base64\"\n\t\"strconv\"\n\t\"time\"\n\n\t\"github.com/kopia/kopia/repo/logging\"\n\t\"github.com/pkg/errors\"\n)\n\nfunc mustHaveString(key string, flags map[string]string) (string, error) {\n\tif value, exist := flags[key]; exist {\n\t\treturn value, nil\n\t}\n\treturn \"\", errors.Errorf(\"key %s not found\", key)\n}\n\nfunc optionalHaveString(key string, flags map[string]string) string {\n\treturn optionalHaveStringWithDefault(key, flags, \"\")\n}\n\nfunc optionalHaveBool(ctx context.Context, key string, flags map[string]string) bool {\n\tif value, exist := flags[key]; exist {\n\t\tif value != \"\" {\n\t\t\tret, err := strconv.ParseBool(value)\n\t\t\tif err == nil {\n\t\t\t\treturn ret\n\t\t\t}\n\n\t\t\tbackendLog()(ctx).Errorf(\"Ignore %s, value [%s] is invalid, err %v\", key, value, err)\n\t\t}\n\t}\n\n\treturn false\n}\n\nfunc optionalHaveFloat64(ctx context.Context, key string, flags map[string]string) float64 {\n\tif value, exist := flags[key]; exist {\n\t\tret, err := strconv.ParseFloat(value, 64)\n\t\tif err == nil {\n\t\t\treturn ret\n\t\t}\n\n\t\tbackendLog()(ctx).Errorf(\"Ignore %s, value [%s] is invalid, err %v\", key, value, err)\n\t}\n\n\treturn 0\n}\n\nfunc optionalHaveStringWithDefault(key string, flags map[string]string, defValue string) string {\n\tif value, exist := flags[key]; exist {\n\t\treturn value\n\t}\n\treturn defValue\n}\n\nfunc optionalHaveDuration(ctx context.Context, key string, flags map[string]string) time.Duration {\n\tif value, exist := flags[key]; exist {\n\t\tret, err := time.ParseDuration(value)\n\t\tif err == nil {\n\t\t\treturn ret\n\t\t}\n\n\t\tbackendLog()(ctx).Errorf(\"Ignore %s, value [%s] is invalid, err %v\", key, value, err)\n\t}\n\n\treturn 0\n}\n\nfunc optionalHaveBase64(ctx context.Context, key string, flags map[string]string) []byte {\n\tif value, exist := flags[key]; exist {\n\t\tret, err := base64.StdEncoding.DecodeString(value)\n\t\tif err == nil {\n\t\t\treturn ret\n\t\t}\n\n\t\tbackendLog()(ctx).Errorf(\"Ignore %s, value [%s] is invalid, err %v\", key, value, err)\n\t}\n\n\treturn nil\n}\n\nfunc optionalHaveIntWithDefault(ctx context.Context, key string, flags map[string]string, defValue int64) int64 {\n\tif value, exist := flags[key]; exist {\n\t\tif value != \"\" {\n\t\t\tret, err := strconv.ParseInt(value, 10, 64)\n\t\t\tif err == nil {\n\t\t\t\treturn ret\n\t\t\t}\n\n\t\t\tbackendLog()(ctx).Errorf(\"Ignore %s, value [%s] is invalid, err %v\", key, value, err)\n\t\t}\n\t}\n\n\treturn defValue\n}\n\nfunc backendLog() func(ctx context.Context) logging.Logger {\n\treturn logging.Module(\"kopialib-bd\")\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/backend/utils_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage backend\n\nimport (\n\t\"testing\"\n\n\t\"github.com/kopia/kopia/repo/logging\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\t\"go.uber.org/zap\"\n\t\"go.uber.org/zap/zapcore\"\n\n\tstoragemocks \"github.com/vmware-tanzu/velero/pkg/repository/udmrepo/kopialib/backend/mocks\"\n)\n\nfunc TestOptionalHaveBool(t *testing.T) {\n\tvar expectMsg string\n\ttestCases := []struct {\n\t\tname string\n\t\tkey string\n\t\tflags map[string]string\n\t\tlogger *storagemocks.Core\n\t\tretFuncCheck func(mock.Arguments)\n\t\texpectMsg string\n\t\tretValue bool\n\t}{\n\t\t{\n\t\t\tname: \"key not exist\",\n\t\t\tkey: \"fake-key\",\n\t\t\tflags: map[string]string{},\n\t\t\tretValue: false,\n\t\t},\n\t\t{\n\t\t\tname: \"value valid\",\n\t\t\tkey: \"fake-key\",\n\t\t\tflags: map[string]string{\n\t\t\t\t\"fake-key\": \"true\",\n\t\t\t},\n\t\t\tretValue: true,\n\t\t},\n\t\t{\n\t\t\tname: \"value invalid\",\n\t\t\tkey: \"fake-key\",\n\t\t\tflags: map[string]string{\n\t\t\t\t\"fake-key\": \"fake-value\",\n\t\t\t},\n\t\t\tlogger: new(storagemocks.Core),\n\t\t\tretFuncCheck: func(args mock.Arguments) {\n\t\t\t\tent := args[0].(zapcore.Entry)\n\t\t\t\tif ent.Level == zapcore.ErrorLevel {\n\t\t\t\t\texpectMsg = ent.Message\n\t\t\t\t}\n\t\t\t},\n\t\t\texpectMsg: \"Ignore fake-key, value [fake-value] is invalid, err strconv.ParseBool: parsing \\\"fake-value\\\": invalid syntax\",\n\t\t\tretValue: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tif tc.logger != nil {\n\t\t\t\ttc.logger.On(\"Enabled\", mock.Anything).Return(true)\n\t\t\t\ttc.logger.On(\"Check\", mock.Anything, mock.Anything).Run(tc.retFuncCheck).Return(&zapcore.CheckedEntry{})\n\t\t\t}\n\n\t\t\tctx := logging.WithLogger(t.Context(), func(module string) logging.Logger {\n\t\t\t\treturn zap.New(tc.logger).Sugar()\n\t\t\t})\n\n\t\t\tretValue := optionalHaveBool(ctx, tc.key, tc.flags)\n\n\t\t\trequire.Equal(t, retValue, tc.retValue)\n\t\t\trequire.Equal(t, tc.expectMsg, expectMsg)\n\t\t})\n\t}\n}\n\nfunc TestOptionalHaveIntWithDefault(t *testing.T) {\n\tvar expectMsg string\n\ttestCases := []struct {\n\t\tname string\n\t\tkey string\n\t\tflags map[string]string\n\t\tdefaultValue int64\n\t\tlogger *storagemocks.Core\n\t\tretFuncCheck func(mock.Arguments)\n\t\texpectMsg string\n\t\tretValue int64\n\t}{\n\t\t{\n\t\t\tname: \"key not exist\",\n\t\t\tkey: \"fake-key\",\n\t\t\tflags: map[string]string{},\n\t\t\tdefaultValue: 2000,\n\t\t\tretValue: 2000,\n\t\t},\n\t\t{\n\t\t\tname: \"value valid\",\n\t\t\tkey: \"fake-key\",\n\t\t\tflags: map[string]string{\n\t\t\t\t\"fake-key\": \"1000\",\n\t\t\t},\n\t\t\tretValue: 1000,\n\t\t},\n\t\t{\n\t\t\tname: \"value invalid\",\n\t\t\tkey: \"fake-key\",\n\t\t\tflags: map[string]string{\n\t\t\t\t\"fake-key\": \"fake-value\",\n\t\t\t},\n\t\t\tlogger: new(storagemocks.Core),\n\t\t\tretFuncCheck: func(args mock.Arguments) {\n\t\t\t\tent := args[0].(zapcore.Entry)\n\t\t\t\tif ent.Level == zapcore.ErrorLevel {\n\t\t\t\t\texpectMsg = ent.Message\n\t\t\t\t}\n\t\t\t},\n\t\t\texpectMsg: \"Ignore fake-key, value [fake-value] is invalid, err strconv.ParseInt: parsing \\\"fake-value\\\": invalid syntax\",\n\t\t\tdefaultValue: 2000,\n\t\t\tretValue: 2000,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tif tc.logger != nil {\n\t\t\t\ttc.logger.On(\"Enabled\", mock.Anything).Return(true)\n\t\t\t\ttc.logger.On(\"Check\", mock.Anything, mock.Anything).Run(tc.retFuncCheck).Return(&zapcore.CheckedEntry{})\n\t\t\t}\n\n\t\t\tctx := logging.WithLogger(t.Context(), func(module string) logging.Logger {\n\t\t\t\treturn zap.New(tc.logger).Sugar()\n\t\t\t})\n\n\t\t\tretValue := optionalHaveIntWithDefault(ctx, tc.key, tc.flags, tc.defaultValue)\n\n\t\t\trequire.Equal(t, retValue, tc.retValue)\n\t\t\trequire.Equal(t, tc.expectMsg, expectMsg)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/lib_repo.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kopialib\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"io\"\n\t\"os\"\n\t\"strings\"\n\t\"sync/atomic\"\n\t\"time\"\n\n\t\"github.com/kopia/kopia/repo\"\n\t\"github.com/kopia/kopia/repo/compression\"\n\t\"github.com/kopia/kopia/repo/content/index\"\n\t\"github.com/kopia/kopia/repo/maintenance\"\n\t\"github.com/kopia/kopia/repo/manifest\"\n\t\"github.com/kopia/kopia/repo/object\"\n\t\"github.com/kopia/kopia/snapshot/snapshotmaintenance\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/kopia\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo/kopialib/backend\"\n)\n\ntype kopiaRepoService struct {\n\tlogger logrus.FieldLogger\n}\n\ntype kopiaRepository struct {\n\trawRepo repo.Repository\n\trawWriter repo.RepositoryWriter\n\tdescription string\n\tuploaded int64\n\topenTime time.Time\n\tthrottle logThrottle\n\tlogger logrus.FieldLogger\n}\n\ntype kopiaMaintenance struct {\n\tmode maintenance.Mode\n\tstartTime time.Time\n\tuploaded int64\n\tthrottle logThrottle\n\tlogger logrus.FieldLogger\n}\n\ntype logThrottle struct {\n\tlastTime int64\n\tinterval time.Duration\n}\n\ntype kopiaObjectReader struct {\n\trawReader object.Reader\n}\n\ntype kopiaObjectWriter struct {\n\trawWriter object.Writer\n}\n\ntype openOptions struct {\n\trepoLogger io.Writer\n}\n\nconst (\n\tdefaultLogInterval = time.Second * 10\n\tdefaultMaintainCheckPeriod = time.Hour\n\toverwriteFullMaintainInterval = time.Duration(0)\n\toverwriteQuickMaintainInterval = time.Duration(0)\n\trepoBackend = \"kopia\"\n)\n\nvar kopiaRepoOpen = repo.Open\n\n// NewKopiaRepoService creates an instance of BackupRepoService implemented by Kopia\nfunc NewKopiaRepoService(logger logrus.FieldLogger) udmrepo.BackupRepoService {\n\tks := &kopiaRepoService{\n\t\tlogger: logger,\n\t}\n\n\treturn ks\n}\n\nfunc (ks *kopiaRepoService) Create(ctx context.Context, repoOption udmrepo.RepoOptions) error {\n\trepoCtx := kopia.SetupKopiaLog(ctx, ks.logger)\n\n\tstatus, err := GetRepositoryStatus(ctx, repoOption, ks.logger)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error getting repo status\")\n\t}\n\n\tif status != RepoStatusSystemNotCreated && status != RepoStatusNotInitialized {\n\t\treturn errors.Errorf(\"unexpected repo status %v\", status)\n\t}\n\n\tif status == RepoStatusSystemNotCreated {\n\t\tif err := CreateBackupRepo(repoCtx, repoOption, ks.logger); err != nil {\n\t\t\treturn errors.Wrap(err, \"error creating backup repo\")\n\t\t}\n\t}\n\n\tif err := InitializeBackupRepo(ctx, repoOption, ks.logger); err != nil {\n\t\treturn errors.Wrap(err, \"error initializing backup repo\")\n\t}\n\n\treturn nil\n}\n\nfunc (ks *kopiaRepoService) Connect(ctx context.Context, repoOption udmrepo.RepoOptions) error {\n\trepoCtx := kopia.SetupKopiaLog(ctx, ks.logger)\n\n\treturn ConnectBackupRepo(repoCtx, repoOption, ks.logger)\n}\n\nfunc (ks *kopiaRepoService) IsCreated(ctx context.Context, repoOption udmrepo.RepoOptions) (bool, error) {\n\trepoCtx := kopia.SetupKopiaLog(ctx, ks.logger)\n\n\tstatus, err := GetRepositoryStatus(repoCtx, repoOption, ks.logger)\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\tif status != RepoStatusCreated {\n\t\tks.logger.Infof(\"Repo is not fully created, status %v\", status)\n\t\treturn false, nil\n\t}\n\n\treturn true, nil\n}\n\nfunc (ks *kopiaRepoService) Open(ctx context.Context, repoOption udmrepo.RepoOptions) (udmrepo.BackupRepo, error) {\n\trepoConfig := repoOption.ConfigFilePath\n\tif repoConfig == \"\" {\n\t\treturn nil, errors.New(\"invalid config file path\")\n\t}\n\n\tif _, err := os.Stat(repoConfig); os.IsNotExist(err) {\n\t\treturn nil, errors.Wrapf(err, \"repo config %s doesn't exist\", repoConfig)\n\t}\n\n\trepoCtx := kopia.SetupKopiaLog(ctx, ks.logger)\n\n\tr, err := openKopiaRepo(repoCtx, repoConfig, repoOption.RepoPassword, &openOptions{repoLogger: kopia.RepositoryLogger(ks.logger)})\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tkr := kopiaRepository{\n\t\trawRepo: r,\n\t\topenTime: time.Now(),\n\t\tdescription: repoOption.Description,\n\t\tthrottle: logThrottle{\n\t\t\tinterval: defaultLogInterval,\n\t\t},\n\t\tlogger: ks.logger,\n\t}\n\n\t_, kr.rawWriter, err = r.NewWriter(repoCtx, repo.WriteSessionOptions{\n\t\tPurpose: repoOption.Description,\n\t\tOnUpload: kr.updateProgress,\n\t})\n\n\tif err != nil {\n\t\tif e := r.Close(repoCtx); e != nil {\n\t\t\tks.logger.WithError(e).Error(\"Failed to close raw repository on error\")\n\t\t}\n\n\t\treturn nil, errors.Wrap(err, \"error to create repo writer\")\n\t}\n\n\treturn &kr, nil\n}\n\nfunc (ks *kopiaRepoService) Maintain(ctx context.Context, repoOption udmrepo.RepoOptions) error {\n\trepoConfig := repoOption.ConfigFilePath\n\tif repoConfig == \"\" {\n\t\treturn errors.New(\"invalid config file path\")\n\t}\n\n\tif _, err := os.Stat(repoConfig); os.IsNotExist(err) {\n\t\treturn errors.Wrapf(err, \"repo config %s doesn't exist\", repoConfig)\n\t}\n\n\trepoCtx := kopia.SetupKopiaLog(ctx, ks.logger)\n\n\tks.logger.Info(\"Start to open repo for maintenance, allow index write on load\")\n\n\tr, err := openKopiaRepo(repoCtx, repoConfig, repoOption.RepoPassword, &openOptions{repoLogger: kopia.RepositoryLogger(ks.logger)})\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tks.logger.Info(\"Succeeded to open repo for maintenance\")\n\n\tdefer func() {\n\t\tc := r.Close(repoCtx)\n\t\tif c != nil {\n\t\t\tks.logger.WithError(c).Error(\"Failed to close repo\")\n\t\t}\n\t}()\n\n\tkm := kopiaMaintenance{\n\t\tmode: maintenance.ModeAuto,\n\t\tstartTime: time.Now(),\n\t\tthrottle: logThrottle{\n\t\t\tinterval: defaultLogInterval,\n\t\t},\n\t\tlogger: ks.logger,\n\t}\n\n\tif mode, exist := repoOption.GeneralOptions[udmrepo.GenOptionMaintainMode]; exist {\n\t\tif strings.EqualFold(mode, udmrepo.GenOptionMaintainFull) {\n\t\t\tkm.mode = maintenance.ModeFull\n\t\t} else if strings.EqualFold(mode, udmrepo.GenOptionMaintainQuick) {\n\t\t\tkm.mode = maintenance.ModeQuick\n\t\t}\n\t}\n\n\terr = repo.DirectWriteSession(repoCtx, r.(repo.DirectRepository), repo.WriteSessionOptions{\n\t\tPurpose: \"UdmRepoMaintenance\",\n\t\tOnUpload: km.maintainProgress,\n\t}, func(ctx context.Context, dw repo.DirectRepositoryWriter) error {\n\t\treturn km.runMaintenance(ctx, dw)\n\t})\n\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to maintain repo\")\n\t}\n\n\treturn nil\n}\n\nfunc (ks *kopiaRepoService) DefaultMaintenanceFrequency() time.Duration {\n\treturn defaultMaintainCheckPeriod\n}\n\nfunc (ks *kopiaRepoService) ClientSideCacheLimit(repoOption map[string]string) int64 {\n\tdefaultLimit := int64(backend.DefaultCacheLimitMB << 20)\n\tif repoOption == nil {\n\t\treturn defaultLimit\n\t}\n\n\tif v, found := repoOption[repoBackend]; found {\n\t\tvar configs map[string]any\n\t\tif err := json.Unmarshal([]byte(v), &configs); err != nil {\n\t\t\tks.logger.WithError(err).Warnf(\"error unmarshalling config data from data %v\", v)\n\t\t\treturn defaultLimit\n\t\t}\n\n\t\tlimit := defaultLimit\n\t\tif v, found := configs[udmrepo.StoreOptionCacheLimit]; found {\n\t\t\tif iv, ok := v.(float64); ok {\n\t\t\t\tlimit = int64(iv) << 20\n\t\t\t} else {\n\t\t\t\tks.logger.Warnf(\"ignore cache limit from data %v\", v)\n\t\t\t}\n\t\t}\n\n\t\treturn limit\n\t}\n\n\treturn defaultLimit\n}\n\nfunc (km *kopiaMaintenance) runMaintenance(ctx context.Context, rep repo.DirectRepositoryWriter) error {\n\terr := snapshotmaintenance.Run(kopia.SetupKopiaLog(ctx, km.logger), rep, km.mode, false, maintenance.SafetyFull)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to run maintenance under mode %s\", km.mode)\n\t}\n\n\treturn nil\n}\n\n// maintainProgress is called when the repository writes a piece of blob data to the storage during the maintenance\nfunc (km *kopiaMaintenance) maintainProgress(uploaded int64) {\n\ttotal := atomic.AddInt64(&km.uploaded, uploaded)\n\n\tif km.throttle.shouldLog() {\n\t\tkm.logger.WithFields(\n\t\t\tlogrus.Fields{\n\t\t\t\t\"Start Time\": km.startTime.Format(time.RFC3339Nano),\n\t\t\t\t\"Current\": time.Now().Format(time.RFC3339Nano),\n\t\t\t},\n\t\t).Debugf(\"Repo maintenance uploaded %d bytes.\", total)\n\t}\n}\n\nfunc (kr *kopiaRepository) OpenObject(ctx context.Context, id udmrepo.ID) (udmrepo.ObjectReader, error) {\n\tif kr.rawRepo == nil {\n\t\treturn nil, errors.New(\"repo is closed or not open\")\n\t}\n\n\tobjID, err := object.ParseID(string(id))\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error to parse object ID from %v\", id)\n\t}\n\n\treader, err := kr.rawRepo.OpenObject(kopia.SetupKopiaLog(ctx, kr.logger), objID)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error to open object\")\n\t}\n\n\treturn &kopiaObjectReader{\n\t\trawReader: reader,\n\t}, nil\n}\n\nfunc (kr *kopiaRepository) GetManifest(ctx context.Context, id udmrepo.ID, mani *udmrepo.RepoManifest) error {\n\tif kr.rawRepo == nil {\n\t\treturn errors.New(\"repo is closed or not open\")\n\t}\n\n\tmetadata, err := kr.rawRepo.GetManifest(kopia.SetupKopiaLog(ctx, kr.logger), manifest.ID(id), mani.Payload)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to get manifest\")\n\t}\n\n\tmani.Metadata = getManifestEntryFromKopia(metadata)\n\n\treturn nil\n}\n\nfunc (kr *kopiaRepository) FindManifests(ctx context.Context, filter udmrepo.ManifestFilter) ([]*udmrepo.ManifestEntryMetadata, error) {\n\tif kr.rawRepo == nil {\n\t\treturn nil, errors.New(\"repo is closed or not open\")\n\t}\n\n\tmetadata, err := kr.rawRepo.FindManifests(kopia.SetupKopiaLog(ctx, kr.logger), filter.Labels)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error to find manifests\")\n\t}\n\n\treturn getManifestEntriesFromKopia(metadata), nil\n}\n\nfunc (kr *kopiaRepository) Time() time.Time {\n\tif kr.rawRepo == nil {\n\t\treturn time.Time{}\n\t}\n\n\treturn kr.rawRepo.Time()\n}\n\nfunc (kr *kopiaRepository) Close(ctx context.Context) error {\n\tif kr.rawWriter != nil {\n\t\terr := kr.rawWriter.Close(kopia.SetupKopiaLog(ctx, kr.logger))\n\t\tif err != nil {\n\t\t\treturn errors.Wrap(err, \"error to close repo writer\")\n\t\t}\n\n\t\tkr.rawWriter = nil\n\t}\n\n\tif kr.rawRepo != nil {\n\t\terr := kr.rawRepo.Close(kopia.SetupKopiaLog(ctx, kr.logger))\n\t\tif err != nil {\n\t\t\treturn errors.Wrap(err, \"error to close repo\")\n\t\t}\n\n\t\tkr.rawRepo = nil\n\t}\n\n\treturn nil\n}\n\nfunc (kr *kopiaRepository) NewObjectWriter(ctx context.Context, opt udmrepo.ObjectWriteOptions) udmrepo.ObjectWriter {\n\tif kr.rawWriter == nil {\n\t\treturn nil\n\t}\n\n\twriter := kr.rawWriter.NewObjectWriter(kopia.SetupKopiaLog(ctx, kr.logger), object.WriterOptions{\n\t\tDescription: opt.Description,\n\t\tPrefix: index.IDPrefix(opt.Prefix),\n\t\tAsyncWrites: opt.AsyncWrites,\n\t\tCompressor: getCompressorForObject(opt),\n\t\tMetadataCompressor: getMetadataCompressor(),\n\t})\n\n\tif writer == nil {\n\t\treturn nil\n\t}\n\n\treturn &kopiaObjectWriter{\n\t\trawWriter: writer,\n\t}\n}\n\nfunc (kr *kopiaRepository) PutManifest(ctx context.Context, manifest udmrepo.RepoManifest) (udmrepo.ID, error) {\n\tif kr.rawWriter == nil {\n\t\treturn \"\", errors.New(\"repo writer is closed or not open\")\n\t}\n\n\tid, err := kr.rawWriter.PutManifest(kopia.SetupKopiaLog(ctx, kr.logger), manifest.Metadata.Labels, manifest.Payload)\n\tif err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error to put manifest\")\n\t}\n\n\treturn udmrepo.ID(id), nil\n}\n\nfunc (kr *kopiaRepository) DeleteManifest(ctx context.Context, id udmrepo.ID) error {\n\tif kr.rawWriter == nil {\n\t\treturn errors.New(\"repo writer is closed or not open\")\n\t}\n\n\terr := kr.rawWriter.DeleteManifest(kopia.SetupKopiaLog(ctx, kr.logger), manifest.ID(id))\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to delete manifest\")\n\t}\n\n\treturn nil\n}\n\nfunc (kr *kopiaRepository) Flush(ctx context.Context) error {\n\tif kr.rawWriter == nil {\n\t\treturn errors.New(\"repo writer is closed or not open\")\n\t}\n\n\terr := kr.rawWriter.Flush(kopia.SetupKopiaLog(ctx, kr.logger))\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to flush repo\")\n\t}\n\n\treturn nil\n}\n\nfunc (kr *kopiaRepository) GetAdvancedFeatures() udmrepo.AdvancedFeatureInfo {\n\treturn udmrepo.AdvancedFeatureInfo{\n\t\tMultiPartBackup: true,\n\t}\n}\n\nfunc (kr *kopiaRepository) ConcatenateObjects(ctx context.Context, objectIDs []udmrepo.ID) (udmrepo.ID, error) {\n\tif kr.rawWriter == nil {\n\t\treturn \"\", errors.New(\"repo writer is closed or not open\")\n\t}\n\n\tif len(objectIDs) == 0 {\n\t\treturn udmrepo.ID(\"\"), errors.New(\"object list is empty\")\n\t}\n\n\trawIDs := []object.ID{}\n\tfor _, id := range objectIDs {\n\t\trawID, err := object.ParseID(string(id))\n\t\tif err != nil {\n\t\t\treturn udmrepo.ID(\"\"), errors.Wrapf(err, \"error to parse object ID from %v\", id)\n\t\t}\n\n\t\trawIDs = append(rawIDs, rawID)\n\t}\n\n\tresult, err := kr.rawWriter.ConcatenateObjects(ctx, rawIDs, repo.ConcatenateOptions{\n\t\tCompressor: getMetadataCompressor(),\n\t})\n\tif err != nil {\n\t\treturn udmrepo.ID(\"\"), errors.Wrap(err, \"error to concatenate objects\")\n\t}\n\n\treturn udmrepo.ID(result.String()), nil\n}\n\n// updateProgress is called when the repository writes a piece of blob data to the storage during data write\nfunc (kr *kopiaRepository) updateProgress(uploaded int64) {\n\ttotal := atomic.AddInt64(&kr.uploaded, uploaded)\n\n\tif kr.throttle.shouldLog() {\n\t\tkr.logger.WithFields(\n\t\t\tlogrus.Fields{\n\t\t\t\t\"Description\": kr.description,\n\t\t\t\t\"Open Time\": kr.openTime.Format(time.RFC3339Nano),\n\t\t\t\t\"Current\": time.Now().Format(time.RFC3339Nano),\n\t\t\t},\n\t\t).Debugf(\"Repo uploaded %d bytes.\", total)\n\t}\n}\n\nfunc (kor *kopiaObjectReader) Read(p []byte) (int, error) {\n\tif kor.rawReader == nil {\n\t\treturn 0, errors.New(\"object reader is closed or not open\")\n\t}\n\n\treturn kor.rawReader.Read(p)\n}\n\nfunc (kor *kopiaObjectReader) Seek(offset int64, whence int) (int64, error) {\n\tif kor.rawReader == nil {\n\t\treturn -1, errors.New(\"object reader is closed or not open\")\n\t}\n\n\treturn kor.rawReader.Seek(offset, whence)\n}\n\nfunc (kor *kopiaObjectReader) Close() error {\n\tif kor.rawReader == nil {\n\t\treturn nil\n\t}\n\n\terr := kor.rawReader.Close()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tkor.rawReader = nil\n\n\treturn nil\n}\n\nfunc (kor *kopiaObjectReader) Length() int64 {\n\tif kor.rawReader == nil {\n\t\treturn -1\n\t}\n\n\treturn kor.rawReader.Length()\n}\n\nfunc (kow *kopiaObjectWriter) Write(p []byte) (int, error) {\n\tif kow.rawWriter == nil {\n\t\treturn 0, errors.New(\"object writer is closed or not open\")\n\t}\n\n\treturn kow.rawWriter.Write(p)\n}\n\nfunc (kow *kopiaObjectWriter) Seek(offset int64, whence int) (int64, error) {\n\treturn -1, errors.New(\"not supported\")\n}\n\nfunc (kow *kopiaObjectWriter) Checkpoint() (udmrepo.ID, error) {\n\tif kow.rawWriter == nil {\n\t\treturn udmrepo.ID(\"\"), errors.New(\"object writer is closed or not open\")\n\t}\n\n\tid, err := kow.rawWriter.Checkpoint()\n\tif err != nil {\n\t\treturn udmrepo.ID(\"\"), errors.Wrap(err, \"error to checkpoint object\")\n\t}\n\n\treturn udmrepo.ID(id.String()), nil\n}\n\nfunc (kow *kopiaObjectWriter) Result() (udmrepo.ID, error) {\n\tif kow.rawWriter == nil {\n\t\treturn udmrepo.ID(\"\"), errors.New(\"object writer is closed or not open\")\n\t}\n\n\tid, err := kow.rawWriter.Result()\n\tif err != nil {\n\t\treturn udmrepo.ID(\"\"), errors.Wrap(err, \"error to wait object\")\n\t}\n\n\treturn udmrepo.ID(id.String()), nil\n}\n\nfunc (kow *kopiaObjectWriter) Close() error {\n\tif kow.rawWriter == nil {\n\t\treturn nil\n\t}\n\n\terr := kow.rawWriter.Close()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tkow.rawWriter = nil\n\n\treturn nil\n}\n\n// getCompressorForObject returns the compressor for an object, at present, we don't support compression\nfunc getCompressorForObject(_ udmrepo.ObjectWriteOptions) compression.Name {\n\treturn \"\"\n}\n\n// getMetadataCompressor returns the compressor for metadata, return kopia's default since we don't support compression\nfunc getMetadataCompressor() compression.Name {\n\treturn \"zstd-fastest\"\n}\n\nfunc getManifestEntryFromKopia(mani *manifest.EntryMetadata) *udmrepo.ManifestEntryMetadata {\n\treturn &udmrepo.ManifestEntryMetadata{\n\t\tID: udmrepo.ID(mani.ID),\n\t\tLabels: mani.Labels,\n\t\tLength: int32(mani.Length),\n\t\tModTime: mani.ModTime,\n\t}\n}\n\nfunc getManifestEntriesFromKopia(mani []*manifest.EntryMetadata) []*udmrepo.ManifestEntryMetadata {\n\tvar ret []*udmrepo.ManifestEntryMetadata\n\n\tfor _, entry := range mani {\n\t\tret = append(ret, &udmrepo.ManifestEntryMetadata{\n\t\t\tID: udmrepo.ID(entry.ID),\n\t\t\tLabels: entry.Labels,\n\t\t\tLength: int32(entry.Length),\n\t\t\tModTime: entry.ModTime,\n\t\t})\n\t}\n\n\treturn ret\n}\n\nfunc (lt *logThrottle) shouldLog() bool {\n\tnextOutputTime := atomic.LoadInt64(<.lastTime)\n\tif nowNano := time.Now().UnixNano(); nowNano > nextOutputTime {\n\t\tif atomic.CompareAndSwapInt64(<.lastTime, nextOutputTime, nowNano+lt.interval.Nanoseconds()) {\n\t\t\treturn true\n\t\t}\n\t}\n\n\treturn false\n}\n\nfunc openKopiaRepo(ctx context.Context, configFile string, password string, options *openOptions) (repo.Repository, error) {\n\tr, err := kopiaRepoOpen(ctx, configFile, password, &repo.Options{\n\t\tContentLogWriter: options.repoLogger,\n\t})\n\tif os.IsNotExist(err) {\n\t\treturn nil, errors.Wrap(err, \"error to open repo, repo doesn't exist\")\n\t}\n\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error to open repo\")\n\t}\n\n\treturn r, nil\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/lib_repo_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kopialib\n\nimport (\n\t\"context\"\n\t\"math\"\n\t\"os\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/kopia/kopia/repo\"\n\t\"github.com/kopia/kopia/repo/manifest\"\n\t\"github.com/kopia/kopia/repo/object\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n\trepomocks \"github.com/vmware-tanzu/velero/pkg/repository/udmrepo/kopialib/backend/mocks\"\n\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestOpen(t *testing.T) {\n\tvar directRpo *repomocks.MockRepository\n\ttestCases := []struct {\n\t\tname string\n\t\trepoOptions udmrepo.RepoOptions\n\t\treturnRepo *repomocks.MockRepository\n\t\trepoOpen func(context.Context, string, string, *repo.Options) (repo.Repository, error)\n\t\tnewWriterError error\n\t\tmockClose bool\n\t\texpectedErr string\n\t\texpected *kopiaRepository\n\t}{\n\t\t{\n\t\t\tname: \"invalid config file\",\n\t\t\texpectedErr: \"invalid config file path\",\n\t\t},\n\t\t{\n\t\t\tname: \"config file doesn't exist\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t},\n\t\t\texpectedErr: \"repo config fake-file doesn't exist: stat fake-file: no such file or directory\",\n\t\t},\n\t\t{\n\t\t\tname: \"repo open fail, repo not exist\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"/tmp\",\n\t\t\t},\n\t\t\trepoOpen: func(context.Context, string, string, *repo.Options) (repo.Repository, error) {\n\t\t\t\treturn nil, os.ErrNotExist\n\t\t\t},\n\t\t\texpectedErr: \"error to open repo, repo doesn't exist: file does not exist\",\n\t\t},\n\t\t{\n\t\t\tname: \"repo open fail, other error\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"/tmp\",\n\t\t\t},\n\t\t\trepoOpen: func(context.Context, string, string, *repo.Options) (repo.Repository, error) {\n\t\t\t\treturn nil, errors.New(\"fake-repo-open-error\")\n\t\t\t},\n\t\t\texpectedErr: \"error to open repo: fake-repo-open-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"create repository writer fail\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"/tmp\",\n\t\t\t},\n\t\t\trepoOpen: func(context.Context, string, string, *repo.Options) (repo.Repository, error) {\n\t\t\t\treturn directRpo, nil\n\t\t\t},\n\t\t\treturnRepo: repomocks.NewMockRepository(t),\n\t\t\tmockClose: true,\n\t\t\tnewWriterError: errors.New(\"fake-new-writer-error\"),\n\t\t\texpectedErr: \"error to create repo writer: fake-new-writer-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"create repository success\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"/tmp\",\n\t\t\t\tDescription: \"fake-description\",\n\t\t\t},\n\t\t\trepoOpen: func(context.Context, string, string, *repo.Options) (repo.Repository, error) {\n\t\t\t\treturn directRpo, nil\n\t\t\t},\n\t\t\treturnRepo: repomocks.NewMockRepository(t),\n\t\t\texpected: &kopiaRepository{\n\t\t\t\tdescription: \"fake-description\",\n\t\t\t\tthrottle: logThrottle{\n\t\t\t\t\tinterval: defaultLogInterval,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlogger := velerotest.NewLogger()\n\n\t\t\tservice := kopiaRepoService{\n\t\t\t\tlogger: logger,\n\t\t\t}\n\n\t\t\tif tc.repoOpen != nil {\n\t\t\t\tkopiaRepoOpen = tc.repoOpen\n\t\t\t}\n\n\t\t\tif tc.returnRepo != nil {\n\t\t\t\tdirectRpo = tc.returnRepo\n\t\t\t}\n\n\t\t\tif tc.returnRepo != nil {\n\t\t\t\ttc.returnRepo.On(\"NewWriter\", mock.Anything, mock.Anything).Return(nil, nil, tc.newWriterError)\n\n\t\t\t\tif tc.mockClose {\n\t\t\t\t\ttc.returnRepo.On(\"Close\", mock.Anything).Return(nil)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\trepo, err := service.Open(t.Context(), tc.repoOptions)\n\n\t\t\tif repo != nil {\n\t\t\t\trequire.Equal(t, tc.expected.description, repo.(*kopiaRepository).description)\n\t\t\t\trequire.Equal(t, tc.expected.throttle.interval, repo.(*kopiaRepository).throttle.interval)\n\t\t\t\trequire.Equal(t, repo.(*kopiaRepository).logger, logger)\n\t\t\t}\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestMaintain(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\trepoOptions udmrepo.RepoOptions\n\t\treturnRepo *repomocks.MockRepository\n\t\treturnRepoWriter *repomocks.MockRepositoryWriter\n\t\trepoOpen func(context.Context, string, string, *repo.Options) (repo.Repository, error)\n\t\tnewRepoWriterError error\n\t\tfindManifestError error\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"invalid config file\",\n\t\t\texpectedErr: \"invalid config file path\",\n\t\t},\n\t\t{\n\t\t\tname: \"config file doesn't exist\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t},\n\t\t\texpectedErr: \"repo config fake-file doesn't exist: stat fake-file: no such file or directory\",\n\t\t},\n\t\t{\n\t\t\tname: \"repo open fail, repo not exist\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"/tmp\",\n\t\t\t\tGeneralOptions: map[string]string{},\n\t\t\t},\n\t\t\trepoOpen: func(context.Context, string, string, *repo.Options) (repo.Repository, error) {\n\t\t\t\treturn nil, os.ErrNotExist\n\t\t\t},\n\t\t\texpectedErr: \"error to open repo, repo doesn't exist: file does not exist\",\n\t\t},\n\t\t{\n\t\t\tname: \"repo open fail, other error\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"/tmp\",\n\t\t\t\tGeneralOptions: map[string]string{},\n\t\t\t},\n\t\t\trepoOpen: func(context.Context, string, string, *repo.Options) (repo.Repository, error) {\n\t\t\t\treturn nil, errors.New(\"fake-repo-open-error\")\n\t\t\t},\n\t\t\texpectedErr: \"error to open repo: fake-repo-open-error\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlogger := velerotest.NewLogger()\n\t\t\tctx := t.Context()\n\n\t\t\tservice := kopiaRepoService{\n\t\t\t\tlogger: logger,\n\t\t\t}\n\n\t\t\tif tc.repoOpen != nil {\n\t\t\t\tkopiaRepoOpen = tc.repoOpen\n\t\t\t}\n\n\t\t\tif tc.returnRepo != nil {\n\t\t\t\ttc.returnRepo.On(\"Close\", mock.Anything).Return(nil)\n\t\t\t}\n\n\t\t\terr := service.Maintain(ctx, tc.repoOptions)\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestShouldLog(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tlastTime int64\n\t\tinterval time.Duration\n\t\tretValue bool\n\t}{\n\t\t{\n\t\t\tname: \"first time\",\n\t\t\tretValue: true,\n\t\t},\n\t\t{\n\t\t\tname: \"not run\",\n\t\t\tlastTime: time.Now().Add(time.Hour).UnixNano(),\n\t\t\tinterval: time.Second * 10,\n\t\t},\n\t\t{\n\t\t\tname: \"not first time, run\",\n\t\t\tlastTime: time.Now().Add(-time.Hour).UnixNano(),\n\t\t\tinterval: time.Second * 10,\n\t\t\tretValue: true,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlt := logThrottle{\n\t\t\t\tlastTime: tc.lastTime,\n\t\t\t\tinterval: tc.interval,\n\t\t\t}\n\n\t\t\tbefore := lt.lastTime\n\n\t\t\tnw := time.Now()\n\n\t\t\ts := lt.shouldLog()\n\n\t\t\trequire.Equal(t, s, tc.retValue)\n\n\t\t\tif s {\n\t\t\t\trequire.GreaterOrEqual(t, lt.lastTime-nw.UnixNano(), lt.interval)\n\t\t\t} else {\n\t\t\t\trequire.Equal(t, lt.lastTime, before)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestOpenObject(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\trawRepo *repomocks.MockRepository\n\t\tobjectID string\n\t\tretErr error\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"raw repo is nil\",\n\t\t\texpectedErr: \"repo is closed or not open\",\n\t\t},\n\t\t{\n\t\t\tname: \"objectID is invalid\",\n\t\t\trawRepo: repomocks.NewMockRepository(t),\n\t\t\tobjectID: \"fake-id\",\n\t\t\texpectedErr: \"error to parse object ID from fake-id: malformed content ID: \\\"fake-id\\\": invalid content prefix\",\n\t\t},\n\t\t{\n\t\t\tname: \"raw open fail\",\n\t\t\trawRepo: repomocks.NewMockRepository(t),\n\t\t\tretErr: errors.New(\"fake-open-error\"),\n\t\t\texpectedErr: \"error to open object: fake-open-error\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tkr := &kopiaRepository{}\n\n\t\t\tif tc.rawRepo != nil {\n\t\t\t\tif tc.retErr != nil {\n\t\t\t\t\ttc.rawRepo.On(\"OpenObject\", mock.Anything, mock.Anything).Return(nil, tc.retErr)\n\t\t\t\t}\n\n\t\t\t\tkr.rawRepo = tc.rawRepo\n\t\t\t}\n\n\t\t\t_, err := kr.OpenObject(t.Context(), udmrepo.ID(tc.objectID))\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetManifest(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\trawRepo *repomocks.MockRepository\n\t\tretErr error\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"raw repo is nil\",\n\t\t\texpectedErr: \"repo is closed or not open\",\n\t\t},\n\t\t{\n\t\t\tname: \"raw get fail\",\n\t\t\trawRepo: repomocks.NewMockRepository(t),\n\t\t\tretErr: errors.New(\"fake-get-error\"),\n\t\t\texpectedErr: \"error to get manifest: fake-get-error\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tkr := &kopiaRepository{}\n\n\t\t\tif tc.rawRepo != nil {\n\t\t\t\tif tc.retErr != nil {\n\t\t\t\t\ttc.rawRepo.On(\"GetManifest\", mock.Anything, mock.Anything, mock.Anything).Return(nil, tc.retErr)\n\t\t\t\t}\n\n\t\t\t\tkr.rawRepo = tc.rawRepo\n\t\t\t}\n\n\t\t\terr := kr.GetManifest(t.Context(), udmrepo.ID(\"\"), &udmrepo.RepoManifest{})\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestFindManifests(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\trawRepo *repomocks.MockRepository\n\t\tretErr error\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"raw repo is nil\",\n\t\t\texpectedErr: \"repo is closed or not open\",\n\t\t},\n\t\t{\n\t\t\tname: \"raw find fail\",\n\t\t\trawRepo: repomocks.NewMockRepository(t),\n\t\t\tretErr: errors.New(\"fake-find-error\"),\n\t\t\texpectedErr: \"error to find manifests: fake-find-error\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tkr := &kopiaRepository{}\n\n\t\t\tif tc.rawRepo != nil {\n\t\t\t\ttc.rawRepo.On(\"FindManifests\", mock.Anything, mock.Anything).Return(nil, tc.retErr)\n\t\t\t\tkr.rawRepo = tc.rawRepo\n\t\t\t}\n\n\t\t\t_, err := kr.FindManifests(t.Context(), udmrepo.ManifestFilter{})\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestClose(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\trawRepo *repomocks.MockRepository\n\t\trawWriter *repomocks.MockRepositoryWriter\n\t\trawRepoRetErr error\n\t\trawWriterRetErr error\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"both nil\",\n\t\t},\n\t\t{\n\t\t\tname: \"writer is not nil\",\n\t\t\trawWriter: repomocks.NewMockRepositoryWriter(t),\n\t\t},\n\t\t{\n\t\t\tname: \"repo is not nil\",\n\t\t\trawRepo: repomocks.NewMockRepository(t),\n\t\t},\n\t\t{\n\t\t\tname: \"writer close error\",\n\t\t\trawWriter: repomocks.NewMockRepositoryWriter(t),\n\t\t\trawWriterRetErr: errors.New(\"fake-writer-close-error\"),\n\t\t\texpectedErr: \"error to close repo writer: fake-writer-close-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"repo is not nil\",\n\t\t\trawRepo: repomocks.NewMockRepository(t),\n\t\t\trawRepoRetErr: errors.New(\"fake-repo-close-error\"),\n\t\t\texpectedErr: \"error to close repo: fake-repo-close-error\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tkr := &kopiaRepository{}\n\n\t\t\tif tc.rawRepo != nil {\n\t\t\t\ttc.rawRepo.On(\"Close\", mock.Anything).Return(tc.rawRepoRetErr)\n\t\t\t\tkr.rawRepo = tc.rawRepo\n\t\t\t}\n\n\t\t\tif tc.rawWriter != nil {\n\t\t\t\ttc.rawWriter.On(\"Close\", mock.Anything).Return(tc.rawWriterRetErr)\n\t\t\t\tkr.rawWriter = tc.rawWriter\n\t\t\t}\n\n\t\t\terr := kr.Close(t.Context())\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestPutManifest(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\trawWriter *repomocks.MockRepositoryWriter\n\t\trawWriterRetErr error\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"raw writer is nil\",\n\t\t\texpectedErr: \"repo writer is closed or not open\",\n\t\t},\n\t\t{\n\t\t\tname: \"raw put fail\",\n\t\t\trawWriter: repomocks.NewMockRepositoryWriter(t),\n\t\t\trawWriterRetErr: errors.New(\"fake-writer-put-error\"),\n\t\t\texpectedErr: \"error to put manifest: fake-writer-put-error\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tkr := &kopiaRepository{}\n\n\t\t\tif tc.rawWriter != nil {\n\t\t\t\ttc.rawWriter.On(\"PutManifest\", mock.Anything, mock.Anything, mock.Anything).Return(manifest.ID(\"\"), tc.rawWriterRetErr)\n\t\t\t\tkr.rawWriter = tc.rawWriter\n\t\t\t}\n\n\t\t\t_, err := kr.PutManifest(t.Context(), udmrepo.RepoManifest{\n\t\t\t\tMetadata: &udmrepo.ManifestEntryMetadata{},\n\t\t\t})\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestDeleteManifest(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\trawWriter *repomocks.MockRepositoryWriter\n\t\trawWriterRetErr error\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"raw writer is nil\",\n\t\t\texpectedErr: \"repo writer is closed or not open\",\n\t\t},\n\t\t{\n\t\t\tname: \"raw delete fail\",\n\t\t\trawWriter: repomocks.NewMockRepositoryWriter(t),\n\t\t\trawWriterRetErr: errors.New(\"fake-writer-delete-error\"),\n\t\t\texpectedErr: \"error to delete manifest: fake-writer-delete-error\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tkr := &kopiaRepository{}\n\n\t\t\tif tc.rawWriter != nil {\n\t\t\t\ttc.rawWriter.On(\"DeleteManifest\", mock.Anything, mock.Anything).Return(tc.rawWriterRetErr)\n\t\t\t\tkr.rawWriter = tc.rawWriter\n\t\t\t}\n\n\t\t\terr := kr.DeleteManifest(t.Context(), udmrepo.ID(\"\"))\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestFlush(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\trawWriter *repomocks.MockRepositoryWriter\n\t\trawWriterRetErr error\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"raw writer is nil\",\n\t\t\texpectedErr: \"repo writer is closed or not open\",\n\t\t},\n\t\t{\n\t\t\tname: \"raw flush fail\",\n\t\t\trawWriter: repomocks.NewMockRepositoryWriter(t),\n\t\t\trawWriterRetErr: errors.New(\"fake-writer-flush-error\"),\n\t\t\texpectedErr: \"error to flush repo: fake-writer-flush-error\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tkr := &kopiaRepository{}\n\n\t\t\tif tc.rawWriter != nil {\n\t\t\t\ttc.rawWriter.On(\"Flush\", mock.Anything).Return(tc.rawWriterRetErr)\n\t\t\t\tkr.rawWriter = tc.rawWriter\n\t\t\t}\n\n\t\t\terr := kr.Flush(t.Context())\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestConcatenateObjects(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tsetWriter bool\n\t\trawWriter *repomocks.MockRepositoryWriter\n\t\trawWriterRetErr error\n\t\tobjectIDs []udmrepo.ID\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"writer is nil\",\n\t\t\texpectedErr: \"repo writer is closed or not open\",\n\t\t},\n\t\t{\n\t\t\tname: \"empty object list\",\n\t\t\tsetWriter: true,\n\t\t\texpectedErr: \"object list is empty\",\n\t\t},\n\t\t{\n\t\t\tname: \"invalid object id\",\n\t\t\tobjectIDs: []udmrepo.ID{\n\t\t\t\t\"I123456\",\n\t\t\t\t\"fake-id\",\n\t\t\t\t\"I678901\",\n\t\t\t},\n\t\t\tsetWriter: true,\n\t\t\texpectedErr: \"error to parse object ID from fake-id: malformed content ID: \\\"fake-id\\\": invalid content prefix\",\n\t\t},\n\t\t{\n\t\t\tname: \"concatenate error\",\n\t\t\trawWriter: repomocks.NewMockRepositoryWriter(t),\n\t\t\trawWriterRetErr: errors.New(\"fake-concatenate-error\"),\n\t\t\tobjectIDs: []udmrepo.ID{\n\t\t\t\t\"I123456\",\n\t\t\t},\n\t\t\tsetWriter: true,\n\t\t\texpectedErr: \"error to concatenate objects: fake-concatenate-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\trawWriter: repomocks.NewMockRepositoryWriter(t),\n\t\t\tobjectIDs: []udmrepo.ID{\n\t\t\t\t\"I123456\",\n\t\t\t},\n\t\t\tsetWriter: true,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tkr := &kopiaRepository{}\n\n\t\t\tif tc.rawWriter != nil {\n\t\t\t\trequire.NotNil(t, tc.rawWriter)\n\t\t\t\ttc.rawWriter.On(\"ConcatenateObjects\", mock.Anything, mock.Anything, mock.Anything).Return(object.ID{}, tc.rawWriterRetErr)\n\t\t\t}\n\n\t\t\tif tc.setWriter {\n\t\t\t\tkr.rawWriter = tc.rawWriter\n\t\t\t}\n\n\t\t\t_, err := kr.ConcatenateObjects(t.Context(), tc.objectIDs)\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestNewObjectWriter(t *testing.T) {\n\trawObjWriter := repomocks.NewWriter(t)\n\ttestCases := []struct {\n\t\tname string\n\t\trawWriter *repomocks.MockRepositoryWriter\n\t\trawWriterRet object.Writer\n\t\texpectedRet udmrepo.ObjectWriter\n\t}{\n\t\t{\n\t\t\tname: \"raw writer is nil\",\n\t\t},\n\t\t{\n\t\t\tname: \"new object writer fail\",\n\t\t\trawWriter: repomocks.NewMockRepositoryWriter(t),\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\trawWriter: repomocks.NewMockRepositoryWriter(t),\n\t\t\trawWriterRet: rawObjWriter,\n\t\t\texpectedRet: &kopiaObjectWriter{rawWriter: rawObjWriter},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tkr := &kopiaRepository{}\n\n\t\t\tif tc.rawWriter != nil {\n\t\t\t\ttc.rawWriter.On(\"NewObjectWriter\", mock.Anything, mock.Anything).Return(tc.rawWriterRet)\n\t\t\t\tkr.rawWriter = tc.rawWriter\n\t\t\t}\n\n\t\t\tret := kr.NewObjectWriter(t.Context(), udmrepo.ObjectWriteOptions{})\n\n\t\t\tassert.Equal(t, tc.expectedRet, ret)\n\t\t})\n\t}\n}\n\nfunc TestUpdateProgress(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tprogress int64\n\t\tuploaded int64\n\t\tthrottle logThrottle\n\t\tlogMessage string\n\t}{\n\t\t{\n\t\t\tname: \"should not output\",\n\t\t\tthrottle: logThrottle{\n\t\t\t\tlastTime: math.MaxInt64,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should output\",\n\t\t\tprogress: 100,\n\t\t\tuploaded: 200,\n\t\t\tlogMessage: \"Repo uploaded 300 bytes.\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlogMessage := \"\"\n\t\t\tkr := &kopiaRepository{\n\t\t\t\tlogger: velerotest.NewSingleLogger(&logMessage),\n\t\t\t\tthrottle: tc.throttle,\n\t\t\t\tuploaded: tc.uploaded,\n\t\t\t}\n\n\t\t\tkr.updateProgress(tc.progress)\n\n\t\t\tif len(tc.logMessage) > 0 {\n\t\t\t\tassert.Contains(t, logMessage, tc.logMessage)\n\t\t\t} else {\n\t\t\t\tassert.Empty(t, logMessage)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestReaderRead(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\trawObjReader *repomocks.Reader\n\t\trawReaderRetErr error\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"raw reader is nil\",\n\t\t\texpectedErr: \"object reader is closed or not open\",\n\t\t},\n\t\t{\n\t\t\tname: \"raw read fail\",\n\t\t\trawObjReader: repomocks.NewReader(t),\n\t\t\trawReaderRetErr: errors.New(\"fake-read-error\"),\n\t\t\texpectedErr: \"fake-read-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\trawObjReader: repomocks.NewReader(t),\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tkr := &kopiaObjectReader{}\n\n\t\t\tif tc.rawObjReader != nil {\n\t\t\t\ttc.rawObjReader.On(\"Read\", mock.Anything).Return(0, tc.rawReaderRetErr)\n\t\t\t\tkr.rawReader = tc.rawObjReader\n\t\t\t}\n\n\t\t\t_, err := kr.Read(nil)\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestReaderSeek(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\trawObjReader *repomocks.Reader\n\t\trawReaderRet int64\n\t\trawReaderRetErr error\n\t\texpectedRet int64\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"raw reader is nil\",\n\t\t\texpectedErr: \"object reader is closed or not open\",\n\t\t},\n\t\t{\n\t\t\tname: \"raw seek fail\",\n\t\t\trawObjReader: repomocks.NewReader(t),\n\t\t\trawReaderRetErr: errors.New(\"fake-seek-error\"),\n\t\t\texpectedErr: \"fake-seek-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\trawObjReader: repomocks.NewReader(t),\n\t\t\trawReaderRet: 100,\n\t\t\texpectedRet: 100,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tkr := &kopiaObjectReader{}\n\n\t\t\tif tc.rawObjReader != nil {\n\t\t\t\ttc.rawObjReader.On(\"Seek\", mock.Anything, mock.Anything).Return(tc.rawReaderRet, tc.rawReaderRetErr)\n\t\t\t\tkr.rawReader = tc.rawObjReader\n\t\t\t}\n\n\t\t\tret, err := kr.Seek(0, 0)\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, tc.expectedRet, ret)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestReaderClose(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\trawObjReader *repomocks.Reader\n\t\trawReaderRetErr error\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"raw reader is nil\",\n\t\t},\n\t\t{\n\t\t\tname: \"raw close fail\",\n\t\t\trawObjReader: repomocks.NewReader(t),\n\t\t\trawReaderRetErr: errors.New(\"fake-close-error\"),\n\t\t\texpectedErr: \"fake-close-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\trawObjReader: repomocks.NewReader(t),\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tkr := &kopiaObjectReader{}\n\n\t\t\tif tc.rawObjReader != nil {\n\t\t\t\ttc.rawObjReader.On(\"Close\").Return(tc.rawReaderRetErr)\n\t\t\t\tkr.rawReader = tc.rawObjReader\n\t\t\t}\n\n\t\t\terr := kr.Close()\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestReaderLength(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\trawObjReader *repomocks.Reader\n\t\trawReaderRet int64\n\t\texpectedRet int64\n\t}{\n\t\t{\n\t\t\tname: \"raw reader is nil\",\n\t\t\texpectedRet: -1,\n\t\t},\n\t\t{\n\t\t\tname: \"raw length fail\",\n\t\t\trawObjReader: repomocks.NewReader(t),\n\t\t\trawReaderRet: 0,\n\t\t\texpectedRet: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\trawObjReader: repomocks.NewReader(t),\n\t\t\trawReaderRet: 200,\n\t\t\texpectedRet: 200,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tkr := &kopiaObjectReader{}\n\n\t\t\tif tc.rawObjReader != nil {\n\t\t\t\ttc.rawObjReader.On(\"Length\").Return(tc.rawReaderRet)\n\t\t\t\tkr.rawReader = tc.rawObjReader\n\t\t\t}\n\n\t\t\tret := kr.Length()\n\n\t\t\tassert.Equal(t, tc.expectedRet, ret)\n\t\t})\n\t}\n}\n\nfunc TestWriterWrite(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\trawObjWriter *repomocks.Writer\n\t\trawWrtierRet int\n\t\trawWriterRetErr error\n\t\texpectedRet int\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"raw writer is nil\",\n\t\t\texpectedErr: \"object writer is closed or not open\",\n\t\t},\n\t\t{\n\t\t\tname: \"raw read fail\",\n\t\t\trawObjWriter: repomocks.NewWriter(t),\n\t\t\trawWriterRetErr: errors.New(\"fake-write-error\"),\n\t\t\texpectedErr: \"fake-write-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\trawObjWriter: repomocks.NewWriter(t),\n\t\t\trawWrtierRet: 200,\n\t\t\texpectedRet: 200,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tkr := &kopiaObjectWriter{}\n\n\t\t\tif tc.rawObjWriter != nil {\n\t\t\t\ttc.rawObjWriter.On(\"Write\", mock.Anything).Return(tc.rawWrtierRet, tc.rawWriterRetErr)\n\t\t\t\tkr.rawWriter = tc.rawObjWriter\n\t\t\t}\n\n\t\t\tret, err := kr.Write(nil)\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, tc.expectedRet, ret)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestWriterCheckpoint(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\trawObjWriter *repomocks.Writer\n\t\trawWrtierRet object.ID\n\t\trawWriterRetErr error\n\t\texpectedRet udmrepo.ID\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"raw writer is nil\",\n\t\t\texpectedErr: \"object writer is closed or not open\",\n\t\t},\n\t\t{\n\t\t\tname: \"raw checkpoint fail\",\n\t\t\trawObjWriter: repomocks.NewWriter(t),\n\t\t\trawWriterRetErr: errors.New(\"fake-checkpoint-error\"),\n\t\t\texpectedErr: \"error to checkpoint object: fake-checkpoint-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\trawObjWriter: repomocks.NewWriter(t),\n\t\t\trawWrtierRet: object.ID{},\n\t\t\texpectedRet: udmrepo.ID(\"\"),\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tkr := &kopiaObjectWriter{}\n\n\t\t\tif tc.rawObjWriter != nil {\n\t\t\t\ttc.rawObjWriter.On(\"Checkpoint\").Return(tc.rawWrtierRet, tc.rawWriterRetErr)\n\t\t\t\tkr.rawWriter = tc.rawObjWriter\n\t\t\t}\n\n\t\t\tret, err := kr.Checkpoint()\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, tc.expectedRet, ret)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestWriterResult(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\trawObjWriter *repomocks.Writer\n\t\trawWrtierRet object.ID\n\t\trawWriterRetErr error\n\t\texpectedRet udmrepo.ID\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"raw writer is nil\",\n\t\t\texpectedErr: \"object writer is closed or not open\",\n\t\t},\n\t\t{\n\t\t\tname: \"raw result fail\",\n\t\t\trawObjWriter: repomocks.NewWriter(t),\n\t\t\trawWriterRetErr: errors.New(\"fake-result-error\"),\n\t\t\texpectedErr: \"error to wait object: fake-result-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\trawObjWriter: repomocks.NewWriter(t),\n\t\t\trawWrtierRet: object.ID{},\n\t\t\texpectedRet: udmrepo.ID(\"\"),\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tkr := &kopiaObjectWriter{}\n\n\t\t\tif tc.rawObjWriter != nil {\n\t\t\t\ttc.rawObjWriter.On(\"Result\").Return(tc.rawWrtierRet, tc.rawWriterRetErr)\n\t\t\t\tkr.rawWriter = tc.rawObjWriter\n\t\t\t}\n\n\t\t\tret, err := kr.Result()\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, tc.expectedRet, ret)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestWriterClose(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\trawObjWriter *repomocks.Writer\n\t\trawWriterRetErr error\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"raw writer is nil\",\n\t\t},\n\t\t{\n\t\t\tname: \"raw close fail\",\n\t\t\trawObjWriter: repomocks.NewWriter(t),\n\t\t\trawWriterRetErr: errors.New(\"fake-close-error\"),\n\t\t\texpectedErr: \"fake-close-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\trawObjWriter: repomocks.NewWriter(t),\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tkr := &kopiaObjectWriter{}\n\n\t\t\tif tc.rawObjWriter != nil {\n\t\t\t\ttc.rawObjWriter.On(\"Close\").Return(tc.rawWriterRetErr)\n\t\t\t\tkr.rawWriter = tc.rawObjWriter\n\t\t\t}\n\n\t\t\terr := kr.Close()\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestMaintainProgress(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tprogress int64\n\t\tuploaded int64\n\t\tthrottle logThrottle\n\t\tlogMessage string\n\t}{\n\t\t{\n\t\t\tname: \"should not output\",\n\t\t\tthrottle: logThrottle{\n\t\t\t\tlastTime: math.MaxInt64,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should output\",\n\t\t\tprogress: 100,\n\t\t\tuploaded: 200,\n\t\t\tlogMessage: \"Repo maintenance uploaded 300 bytes.\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlogMessage := \"\"\n\t\t\tkm := &kopiaMaintenance{\n\t\t\t\tlogger: velerotest.NewSingleLogger(&logMessage),\n\t\t\t\tthrottle: tc.throttle,\n\t\t\t\tuploaded: tc.uploaded,\n\t\t\t}\n\n\t\t\tkm.maintainProgress(tc.progress)\n\n\t\t\tif len(tc.logMessage) > 0 {\n\t\t\t\tassert.Contains(t, logMessage, tc.logMessage)\n\t\t\t} else {\n\t\t\t\tassert.Empty(t, logMessage)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestClientSideCacheLimit(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\trepoOption map[string]string\n\t\texpected int64\n\t}{\n\t\t{\n\t\t\tname: \"nil option\",\n\t\t\texpected: 5000 << 20,\n\t\t},\n\t\t{\n\t\t\tname: \"no option\",\n\t\t\trepoOption: map[string]string{\n\t\t\t\t\"other-repo\": \"\\\"enableCompression\\\": true\",\n\t\t\t},\n\t\t\texpected: 5000 << 20,\n\t\t},\n\t\t{\n\t\t\tname: \"unmarshall fails\",\n\t\t\trepoOption: map[string]string{\n\t\t\t\t\"kopia\": \"wrong-json\",\n\t\t\t},\n\t\t\texpected: 5000 << 20,\n\t\t},\n\t\t{\n\t\t\tname: \"no cache limit\",\n\t\t\trepoOption: map[string]string{\n\t\t\t\t\"kopia\": \"{\\\"enableCompression\\\": true}\",\n\t\t\t},\n\t\t\texpected: 5000 << 20,\n\t\t},\n\t\t{\n\t\t\tname: \"wrong cache value type\",\n\t\t\trepoOption: map[string]string{\n\t\t\t\t\"kopia\": \"{\\\"cacheLimitMB\\\": \\\"abcd\\\",\\\"enableCompression\\\": true}\",\n\t\t\t},\n\t\t\texpected: 5000 << 20,\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\trepoOption: map[string]string{\n\t\t\t\t\"kopia\": \"{\\\"cacheLimitMB\\\": 1,\\\"enableCompression\\\": true}\",\n\t\t\t},\n\t\t\texpected: 1048576,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tks := &kopiaRepoService{\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t}\n\n\t\t\tlimit := ks.ClientSideCacheLimit(tc.repoOption)\n\n\t\t\tassert.Equal(t, tc.expected, limit)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/repo_init.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kopialib\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"io\"\n\t\"slices\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/kopia/kopia/repo\"\n\t\"github.com/kopia/kopia/repo/blob\"\n\t\"github.com/kopia/kopia/repo/format\"\n\t\"github.com/kopia/kopia/repo/maintenance\"\n\t\"github.com/pkg/errors\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/kopia\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo/kopialib/backend\"\n)\n\ntype kopiaBackendStore struct {\n\tname string\n\tdescription string\n\tstore backend.Store\n}\n\n// backendStores lists the supported backend storages at present\nvar backendStores = []kopiaBackendStore{\n\t{udmrepo.StorageTypeAzure, \"an Azure blob storage\", &backend.AzureBackend{}},\n\t{udmrepo.StorageTypeFs, \"a filesystem\", &backend.FsBackend{}},\n\t{udmrepo.StorageTypeGcs, \"a Google Cloud Storage bucket\", &backend.GCSBackend{}},\n\t{udmrepo.StorageTypeS3, \"an S3 bucket\", &backend.S3Backend{}},\n}\n\nconst udmRepoBlobID = \"udmrepo.Repository\"\n\ntype udmRepoMetadata struct {\n\tUniqueID []byte `json:\"uniqueID\"`\n}\n\ntype RepoStatus int\n\nconst (\n\tRepoStatusUnknown = 0\n\tRepoStatusCorrupted = 1\n\tRepoStatusSystemNotCreated = 2\n\tRepoStatusNotInitialized = 3\n\tRepoStatusCreated = 4\n)\n\n// CreateBackupRepo creates a Kopia repository and then connect to it.\n// The storage must be empty, otherwise, it will fail\nfunc CreateBackupRepo(ctx context.Context, repoOption udmrepo.RepoOptions, logger logrus.FieldLogger) error {\n\tbackendStore, err := setupBackendStore(ctx, repoOption.StorageType, repoOption.StorageOptions, logger)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to setup backend storage\")\n\t}\n\n\tst, err := backendStore.store.Connect(ctx, true, logger)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to connect to storage\")\n\t}\n\n\terr = createWithStorage(ctx, st, repoOption)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to create repo with storage\")\n\t}\n\n\treturn nil\n}\n\n// ConnectBackupRepo connects to an existing Kopia repository.\n// If the repository doesn't exist, it will fail\nfunc ConnectBackupRepo(ctx context.Context, repoOption udmrepo.RepoOptions, logger logrus.FieldLogger) error {\n\tif repoOption.ConfigFilePath == \"\" {\n\t\treturn errors.New(\"invalid config file path\")\n\t}\n\n\tst, err := connectStore(ctx, repoOption, logger)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\terr = connectWithStorage(ctx, st, repoOption)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to connect repo with storage\")\n\t}\n\n\treturn nil\n}\n\nfunc GetRepositoryStatus(ctx context.Context, repoOption udmrepo.RepoOptions, logger logrus.FieldLogger) (RepoStatus, error) {\n\tst, err := connectStore(ctx, repoOption, logger)\n\tif errors.Is(err, backend.ErrStoreNotExist) {\n\t\treturn RepoStatusSystemNotCreated, nil\n\t} else if err != nil {\n\t\treturn RepoStatusUnknown, err\n\t}\n\n\tvar formatBytes byteBuffer\n\tif err := st.GetBlob(ctx, format.KopiaRepositoryBlobID, 0, -1, &formatBytes); err != nil {\n\t\tif errors.Is(err, blob.ErrBlobNotFound) {\n\t\t\tlogger.Debug(\"Kopia repository blob is not found\")\n\t\t\treturn RepoStatusSystemNotCreated, nil\n\t\t}\n\n\t\treturn RepoStatusUnknown, errors.Wrap(err, \"error reading format blob\")\n\t}\n\n\trepoFmt, err := format.ParseKopiaRepositoryJSON(formatBytes.buffer)\n\tif err != nil {\n\t\treturn RepoStatusCorrupted, err\n\t}\n\n\tvar initInfoBytes byteBuffer\n\tif err := st.GetBlob(ctx, udmRepoBlobID, 0, -1, &initInfoBytes); err != nil {\n\t\tif errors.Is(err, blob.ErrBlobNotFound) {\n\t\t\tlogger.Debug(\"Udm repo metadata blob is not found\")\n\t\t\treturn RepoStatusNotInitialized, nil\n\t\t}\n\n\t\treturn RepoStatusUnknown, errors.Wrap(err, \"error reading udm repo blob\")\n\t}\n\n\tudmpRepo := &udmRepoMetadata{}\n\tif err := json.Unmarshal(initInfoBytes.buffer, udmpRepo); err != nil {\n\t\treturn RepoStatusCorrupted, errors.Wrap(err, \"invalid udm repo blob\")\n\t}\n\n\tif !slices.Equal(udmpRepo.UniqueID, repoFmt.UniqueID) {\n\t\treturn RepoStatusCorrupted, errors.Errorf(\"unique ID doesn't match: %v(%v)\", udmpRepo.UniqueID, repoFmt.UniqueID)\n\t}\n\n\treturn RepoStatusCreated, nil\n}\n\nfunc InitializeBackupRepo(ctx context.Context, repoOption udmrepo.RepoOptions, logger logrus.FieldLogger) error {\n\tif repoOption.ConfigFilePath == \"\" {\n\t\treturn errors.New(\"invalid config file path\")\n\t}\n\n\tst, err := connectStore(ctx, repoOption, logger)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\terr = connectWithStorage(ctx, st, repoOption)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error connecting repo with storage\")\n\t}\n\n\terr = writeInitParameters(ctx, repoOption, logger)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error writing init parameters\")\n\t}\n\n\terr = writeUdmRepoMetadata(ctx, st)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error writing udm repo metadata\")\n\t}\n\n\treturn nil\n}\n\nfunc writeUdmRepoMetadata(ctx context.Context, st blob.Storage) error {\n\tvar formatBytes byteBuffer\n\tif err := st.GetBlob(ctx, format.KopiaRepositoryBlobID, 0, -1, &formatBytes); err != nil {\n\t\treturn errors.Wrap(err, \"error reading format blob\")\n\t}\n\n\trepoFmt, err := format.ParseKopiaRepositoryJSON(formatBytes.buffer)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tudmpRepo := &udmRepoMetadata{\n\t\tUniqueID: repoFmt.UniqueID,\n\t}\n\n\tbytes, err := json.Marshal(udmpRepo)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error marshaling udm repo metadata\")\n\t}\n\n\terr = st.PutBlob(ctx, udmRepoBlobID, &byteBuffer{bytes}, blob.PutOptions{})\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error writing udm repo metadata\")\n\t}\n\n\treturn nil\n}\n\nfunc connectStore(ctx context.Context, repoOption udmrepo.RepoOptions, logger logrus.FieldLogger) (blob.Storage, error) {\n\tbackendStore, err := setupBackendStore(ctx, repoOption.StorageType, repoOption.StorageOptions, logger)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error to setup backend storage\")\n\t}\n\n\tst, err := backendStore.store.Connect(ctx, false, logger)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error to connect to storage\")\n\t}\n\n\treturn st, nil\n}\n\nfunc findBackendStore(storage string) *kopiaBackendStore {\n\tfor _, options := range backendStores {\n\t\tif strings.EqualFold(options.name, storage) {\n\t\t\treturn &options\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc setupBackendStore(ctx context.Context, storageType string, storageOptions map[string]string, logger logrus.FieldLogger) (*kopiaBackendStore, error) {\n\tbackendStore := findBackendStore(storageType)\n\tif backendStore == nil {\n\t\treturn nil, errors.New(\"error to find storage type\")\n\t}\n\n\terr := backendStore.store.Setup(ctx, storageOptions, logger)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error to setup storage\")\n\t}\n\n\treturn backendStore, nil\n}\n\nfunc createWithStorage(ctx context.Context, st blob.Storage, repoOption udmrepo.RepoOptions) error {\n\terr := ensureEmpty(ctx, st)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to ensure repository storage empty\")\n\t}\n\n\toptions := backend.SetupNewRepositoryOptions(ctx, repoOption.GeneralOptions)\n\n\tif err := repo.Initialize(ctx, st, &options, repoOption.RepoPassword); err != nil {\n\t\treturn errors.Wrap(err, \"error to initialize repository\")\n\t}\n\n\treturn nil\n}\n\nfunc connectWithStorage(ctx context.Context, st blob.Storage, repoOption udmrepo.RepoOptions) error {\n\toptions := backend.SetupConnectOptions(ctx, repoOption)\n\tif err := repo.Connect(ctx, repoOption.ConfigFilePath, st, repoOption.RepoPassword, &options); err != nil {\n\t\treturn errors.Wrap(err, \"error to connect to repository\")\n\t}\n\n\treturn nil\n}\n\nfunc ensureEmpty(ctx context.Context, s blob.Storage) error {\n\thasDataError := errors.Errorf(\"has data\")\n\n\terr := s.ListBlobs(ctx, \"\", func(cb blob.Metadata) error {\n\t\treturn hasDataError\n\t})\n\n\tif errors.Is(err, hasDataError) {\n\t\treturn errors.New(\"found existing data in storage location\")\n\t}\n\n\treturn errors.Wrap(err, \"error to list blobs\")\n}\n\ntype byteBuffer struct {\n\tbuffer []byte\n}\n\ntype byteBufferReader struct {\n\tbuffer []byte\n\tpos int\n}\n\nfunc (b *byteBuffer) Write(p []byte) (int, error) {\n\tb.buffer = append(b.buffer, p...)\n\treturn len(p), nil\n}\n\nfunc (b *byteBuffer) WriteTo(w io.Writer) (int64, error) {\n\tn, err := w.Write(b.buffer)\n\treturn int64(n), err\n}\n\nfunc (b *byteBuffer) Reset() {\n\tb.buffer = nil\n}\n\nfunc (b *byteBuffer) Length() int {\n\treturn len(b.buffer)\n}\n\nfunc (b *byteBuffer) Reader() io.ReadSeekCloser {\n\treturn &byteBufferReader{buffer: b.buffer}\n}\n\nfunc (b *byteBufferReader) Close() error {\n\treturn nil\n}\n\nfunc (b *byteBufferReader) Read(out []byte) (int, error) {\n\tif b.pos == len(b.buffer) {\n\t\treturn 0, io.EOF\n\t}\n\n\tcopied := copy(out, b.buffer[b.pos:])\n\tb.pos += copied\n\n\treturn copied, nil\n}\n\nfunc (b *byteBufferReader) Seek(offset int64, whence int) (int64, error) {\n\tnewOffset := b.pos\n\n\tswitch whence {\n\tcase io.SeekStart:\n\t\tnewOffset = int(offset)\n\tcase io.SeekCurrent:\n\t\tnewOffset += int(offset)\n\tcase io.SeekEnd:\n\t\tnewOffset = len(b.buffer) + int(offset)\n\t}\n\n\tif newOffset < 0 || newOffset > len(b.buffer) {\n\t\treturn -1, errors.New(\"invalid seek\")\n\t}\n\n\tb.pos = newOffset\n\n\treturn int64(newOffset), nil\n}\n\nvar funcGetParam = maintenance.GetParams\n\nfunc writeInitParameters(ctx context.Context, repoOption udmrepo.RepoOptions, logger logrus.FieldLogger) error {\n\tr, err := openKopiaRepo(ctx, repoOption.ConfigFilePath, repoOption.RepoPassword, &openOptions{repoLogger: kopia.RepositoryLogger(logger)})\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdefer func() {\n\t\tc := r.Close(ctx)\n\t\tif c != nil {\n\t\t\tlogger.WithError(c).Error(\"Failed to close repo\")\n\t\t}\n\t}()\n\n\tparams, err := funcGetParam(ctx, r)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error getting existing maintenance params\")\n\t}\n\n\tif params.Owner == backend.RepoOwnerFromRepoOptions(repoOption) {\n\t\tlogger.Warn(\"Init parameters already exists, skip\")\n\t\treturn nil\n\t}\n\n\tif params.Owner != \"\" {\n\t\tlogger.Warnf(\"Overwriting existing init params %v\", params)\n\t}\n\n\terr = repo.WriteSession(ctx, r, repo.WriteSessionOptions{\n\t\tPurpose: \"set init parameters\",\n\t}, func(ctx context.Context, w repo.RepositoryWriter) error {\n\t\tp := maintenance.DefaultParams()\n\n\t\tif overwriteFullMaintainInterval != time.Duration(0) {\n\t\t\tlogger.Infof(\"Full maintenance interval change from %v to %v\", p.FullCycle.Interval, overwriteFullMaintainInterval)\n\t\t\tp.FullCycle.Interval = overwriteFullMaintainInterval\n\t\t}\n\n\t\tif overwriteQuickMaintainInterval != time.Duration(0) {\n\t\t\tlogger.Infof(\"Quick maintenance interval change from %v to %v\", p.QuickCycle.Interval, overwriteQuickMaintainInterval)\n\t\t\tp.QuickCycle.Interval = overwriteQuickMaintainInterval\n\t\t}\n\t\t// the repoOption.StorageOptions are set via\n\t\t// udmrepo.WithStoreOptions -> udmrepo.GetStoreOptions (interface)\n\t\t// -> pkg/repository/provider.GetStoreOptions(param interface{}) -> pkg/repository/provider.getStorageVariables(..., backupRepoConfig)\n\t\t// where backupRepoConfig comes from param.(RepoParam).BackupRepo.Spec.RepositoryConfig map[string]string\n\t\t// where RepositoryConfig comes from pkg/controller/getBackupRepositoryConfig(...)\n\t\t// where it gets a configMap name from pkg/cmd/server/config/Config.BackupRepoConfig\n\t\t// which gets set via velero server flag `backup-repository-configmap` \"The name of ConfigMap containing backup repository configurations.\"\n\t\t// and data stored as json under ConfigMap.Data[repoType] where repoType is BackupRepository.Spec.RepositoryType: either kopia or restic\n\t\t// repoOption.StorageOptions[udmrepo.StoreOptionKeyFullMaintenanceInterval] would for example look like\n\t\t// configMapName.data.kopia: {\"fullMaintenanceInterval\": \"eagerGC\"}\n\t\tfullMaintIntervalOption := udmrepo.FullMaintenanceIntervalOptions(repoOption.StorageOptions[udmrepo.StoreOptionKeyFullMaintenanceInterval])\n\t\tpriorMaintInterval := p.FullCycle.Interval\n\t\tswitch fullMaintIntervalOption {\n\t\tcase udmrepo.FastGC:\n\t\t\tp.FullCycle.Interval = udmrepo.FastGCInterval\n\t\tcase udmrepo.EagerGC:\n\t\t\tp.FullCycle.Interval = udmrepo.EagerGCInterval\n\t\tcase udmrepo.NormalGC:\n\t\t\tp.FullCycle.Interval = udmrepo.NormalGCInterval\n\t\tcase \"\": // do nothing\n\t\tdefault:\n\t\t\treturn errors.Errorf(\"invalid full maintenance interval option %s\", fullMaintIntervalOption)\n\t\t}\n\t\tif priorMaintInterval != p.FullCycle.Interval {\n\t\t\tlogger.Infof(\"Full maintenance interval change from %v to %v\", priorMaintInterval, p.FullCycle.Interval)\n\t\t}\n\n\t\tp.Owner = r.ClientOptions().UsernameAtHost()\n\n\t\tif err := maintenance.SetParams(ctx, w, &p); err != nil {\n\t\t\treturn errors.Wrap(err, \"error to set maintenance params\")\n\t\t}\n\n\t\treturn nil\n\t})\n\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to init write repo parameters\")\n\t}\n\n\treturn nil\n}\n" }, { "path": "pkg/repository/udmrepo/kopialib/repo_init_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kopialib\n\nimport (\n\t\"context\"\n\t\"io\"\n\t\"os\"\n\t\"testing\"\n\n\t\"github.com/kopia/kopia/repo\"\n\t\"github.com/kopia/kopia/repo/blob\"\n\t\"github.com/kopia/kopia/repo/maintenance\"\n\t\"github.com/kopia/kopia/repo/manifest\"\n\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo/kopialib/backend\"\n\trepomocks \"github.com/vmware-tanzu/velero/pkg/repository/udmrepo/kopialib/backend/mocks\"\n\tstoragemocks \"github.com/vmware-tanzu/velero/pkg/repository/udmrepo/kopialib/backend/mocks\"\n\n\t\"github.com/pkg/errors\"\n)\n\ntype comparableError struct {\n\tmessage string\n}\n\nfunc (ce *comparableError) Error() string {\n\treturn ce.message\n}\n\nfunc (ce *comparableError) Is(err error) bool {\n\treturn err.Error() == ce.message\n}\n\nfunc TestCreateBackupRepo(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tbackendStore *storagemocks.Store\n\t\trepoOptions udmrepo.RepoOptions\n\t\tconnectErr error\n\t\tsetupError error\n\t\treturnStore *storagemocks.Storage\n\t\tstoreListErr error\n\t\tgetBlobErr error\n\t\tlistBlobErr error\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"storage setup fail, invalid type\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t},\n\t\t\texpectedErr: \"error to setup backend storage: error to find storage type\",\n\t\t},\n\t\t{\n\t\t\tname: \"storage setup fail, backend store steup fail\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t\tStorageType: udmrepo.StorageTypeAzure,\n\t\t\t},\n\t\t\tbackendStore: new(storagemocks.Store),\n\t\t\tsetupError: errors.New(\"fake-setup-error\"),\n\t\t\texpectedErr: \"error to setup backend storage: error to setup storage: fake-setup-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"storage connect fail\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t\tStorageType: udmrepo.StorageTypeAzure,\n\t\t\t},\n\t\t\tbackendStore: new(storagemocks.Store),\n\t\t\tconnectErr: errors.New(\"fake-connect-error\"),\n\t\t\texpectedErr: \"error to connect to storage: fake-connect-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"create repository error, exist blobs\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t\tStorageType: udmrepo.StorageTypeAzure,\n\t\t\t},\n\t\t\tbackendStore: new(storagemocks.Store),\n\t\t\treturnStore: new(storagemocks.Storage),\n\t\t\tlistBlobErr: &comparableError{\n\t\t\t\tmessage: \"has data\",\n\t\t\t},\n\t\t\texpectedErr: \"error to create repo with storage: error to ensure repository storage empty: found existing data in storage location\",\n\t\t},\n\t\t{\n\t\t\tname: \"create repository error, error list blobs\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t\tStorageType: udmrepo.StorageTypeAzure,\n\t\t\t},\n\t\t\tbackendStore: new(storagemocks.Store),\n\t\t\treturnStore: new(storagemocks.Storage),\n\t\t\tlistBlobErr: errors.New(\"fake-list-blob-error\"),\n\t\t\texpectedErr: \"error to create repo with storage: error to ensure repository storage empty: error to list blobs: fake-list-blob-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"create repository error, initialize error\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t\tStorageType: udmrepo.StorageTypeAzure,\n\t\t\t},\n\t\t\tbackendStore: new(storagemocks.Store),\n\t\t\treturnStore: new(storagemocks.Storage),\n\t\t\tgetBlobErr: errors.New(\"fake-list-blob-error-01\"),\n\t\t\texpectedErr: \"error to create repo with storage: error to initialize repository: unexpected error when checking for format blob: fake-list-blob-error-01\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlogger := velerotest.NewLogger()\n\t\t\tbackendStores = []kopiaBackendStore{\n\t\t\t\t{udmrepo.StorageTypeAzure, \"fake store\", tc.backendStore},\n\t\t\t\t{udmrepo.StorageTypeFs, \"fake store\", tc.backendStore},\n\t\t\t\t{udmrepo.StorageTypeGcs, \"fake store\", tc.backendStore},\n\t\t\t\t{udmrepo.StorageTypeS3, \"fake store\", tc.backendStore},\n\t\t\t}\n\n\t\t\tif tc.backendStore != nil {\n\t\t\t\ttc.backendStore.On(\"Connect\", mock.Anything, mock.Anything, mock.Anything).Return(tc.returnStore, tc.connectErr)\n\t\t\t\ttc.backendStore.On(\"Setup\", mock.Anything, mock.Anything, mock.Anything).Return(tc.setupError)\n\t\t\t}\n\n\t\t\tif tc.returnStore != nil {\n\t\t\t\ttc.returnStore.On(\"ListBlobs\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(tc.listBlobErr)\n\t\t\t\ttc.returnStore.On(\"GetBlob\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(tc.getBlobErr)\n\t\t\t}\n\n\t\t\terr := CreateBackupRepo(t.Context(), tc.repoOptions, logger)\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestConnectBackupRepo(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tbackendStore *storagemocks.Store\n\t\trepoOptions udmrepo.RepoOptions\n\t\tconnectErr error\n\t\tsetupError error\n\t\treturnStore *storagemocks.Storage\n\t\tgetBlobErr error\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"invalid config file\",\n\t\t\texpectedErr: \"invalid config file path\",\n\t\t},\n\t\t{\n\t\t\tname: \"storage setup fail, invalid type\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t},\n\t\t\texpectedErr: \"error to setup backend storage: error to find storage type\",\n\t\t},\n\t\t{\n\t\t\tname: \"storage setup fail, backend store steup fail\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t\tStorageType: udmrepo.StorageTypeAzure,\n\t\t\t},\n\t\t\tbackendStore: new(storagemocks.Store),\n\t\t\tsetupError: errors.New(\"fake-setup-error\"),\n\t\t\texpectedErr: \"error to setup backend storage: error to setup storage: fake-setup-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"storage connect fail\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t\tStorageType: udmrepo.StorageTypeAzure,\n\t\t\t},\n\t\t\tbackendStore: new(storagemocks.Store),\n\t\t\tconnectErr: errors.New(\"fake-connect-error\"),\n\t\t\texpectedErr: \"error to connect to storage: fake-connect-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"connect repository error\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t\tStorageType: udmrepo.StorageTypeAzure,\n\t\t\t},\n\t\t\tbackendStore: new(storagemocks.Store),\n\t\t\treturnStore: new(storagemocks.Storage),\n\t\t\tgetBlobErr: errors.New(\"fake-get-blob-error\"),\n\t\t\texpectedErr: \"error to connect repo with storage: error to connect to repository: unable to read format blob: fake-get-blob-error\",\n\t\t},\n\t}\n\n\tlogger := velerotest.NewLogger()\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tbackendStores = []kopiaBackendStore{\n\t\t\t\t{udmrepo.StorageTypeAzure, \"fake store\", tc.backendStore},\n\t\t\t\t{udmrepo.StorageTypeFs, \"fake store\", tc.backendStore},\n\t\t\t\t{udmrepo.StorageTypeGcs, \"fake store\", tc.backendStore},\n\t\t\t\t{udmrepo.StorageTypeS3, \"fake store\", tc.backendStore},\n\t\t\t}\n\n\t\t\tif tc.backendStore != nil {\n\t\t\t\ttc.backendStore.On(\"Connect\", mock.Anything, mock.Anything, mock.Anything).Return(tc.returnStore, tc.connectErr)\n\t\t\t\ttc.backendStore.On(\"Setup\", mock.Anything, mock.Anything, mock.Anything).Return(tc.setupError)\n\t\t\t}\n\n\t\t\tif tc.returnStore != nil {\n\t\t\t\ttc.returnStore.On(\"GetBlob\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(tc.getBlobErr)\n\t\t\t}\n\n\t\t\terr := ConnectBackupRepo(t.Context(), tc.repoOptions, logger)\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetRepositoryStatus(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tbackendStore *storagemocks.Store\n\t\trepoOptions udmrepo.RepoOptions\n\t\tconnectErr error\n\t\tsetupError error\n\t\treturnStore *storagemocks.Storage\n\t\tretFuncGetBlob func(context.Context, blob.ID, int64, int64, blob.OutputBuffer) error\n\t\texpected RepoStatus\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"storage setup fail, invalid type\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t},\n\t\t\texpected: RepoStatusUnknown,\n\t\t\texpectedErr: \"error to setup backend storage: error to find storage type\",\n\t\t},\n\t\t{\n\t\t\tname: \"storage setup fail, backend store steup fail\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t\tStorageType: udmrepo.StorageTypeAzure,\n\t\t\t},\n\t\t\tbackendStore: new(storagemocks.Store),\n\t\t\tsetupError: errors.New(\"fake-setup-error\"),\n\t\t\texpected: RepoStatusUnknown,\n\t\t\texpectedErr: \"error to setup backend storage: error to setup storage: fake-setup-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"storage connect fail\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t\tStorageType: udmrepo.StorageTypeAzure,\n\t\t\t},\n\t\t\tbackendStore: new(storagemocks.Store),\n\t\t\tconnectErr: errors.New(\"fake-connect-error\"),\n\t\t\texpected: RepoStatusUnknown,\n\t\t\texpectedErr: \"error to connect to storage: fake-connect-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"storage not exist\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t\tStorageType: udmrepo.StorageTypeAzure,\n\t\t\t},\n\t\t\tbackendStore: new(storagemocks.Store),\n\t\t\tconnectErr: backend.ErrStoreNotExist,\n\t\t\texpected: RepoStatusSystemNotCreated,\n\t\t},\n\t\t{\n\t\t\tname: \"get repo blob error\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t\tStorageType: udmrepo.StorageTypeAzure,\n\t\t\t},\n\t\t\tbackendStore: new(storagemocks.Store),\n\t\t\treturnStore: new(storagemocks.Storage),\n\t\t\tretFuncGetBlob: func(context.Context, blob.ID, int64, int64, blob.OutputBuffer) error {\n\t\t\t\treturn errors.New(\"fake-get-blob-error\")\n\t\t\t},\n\t\t\texpected: RepoStatusUnknown,\n\t\t\texpectedErr: \"error reading format blob: fake-get-blob-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"no repo blob\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t\tStorageType: udmrepo.StorageTypeAzure,\n\t\t\t},\n\t\t\tbackendStore: new(storagemocks.Store),\n\t\t\treturnStore: new(storagemocks.Storage),\n\t\t\tretFuncGetBlob: func(context.Context, blob.ID, int64, int64, blob.OutputBuffer) error {\n\t\t\t\treturn blob.ErrBlobNotFound\n\t\t\t},\n\t\t\texpected: RepoStatusSystemNotCreated,\n\t\t},\n\t\t{\n\t\t\tname: \"wrong repo format\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t\tStorageType: udmrepo.StorageTypeAzure,\n\t\t\t},\n\t\t\tbackendStore: new(storagemocks.Store),\n\t\t\treturnStore: new(storagemocks.Storage),\n\t\t\tretFuncGetBlob: func(ctx context.Context, id blob.ID, offset int64, length int64, output blob.OutputBuffer) error {\n\t\t\t\toutput.Write([]byte(\"fake-buffer\"))\n\t\t\t\treturn nil\n\t\t\t},\n\t\t\texpected: RepoStatusCorrupted,\n\t\t\texpectedErr: \"invalid format blob: invalid character 'k' in literal false (expecting 'l')\",\n\t\t},\n\t\t{\n\t\t\tname: \"get udm repo blob error\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t\tStorageType: udmrepo.StorageTypeAzure,\n\t\t\t},\n\t\t\tbackendStore: new(storagemocks.Store),\n\t\t\treturnStore: new(storagemocks.Storage),\n\t\t\tretFuncGetBlob: func(ctx context.Context, blobID blob.ID, offset int64, length int64, output blob.OutputBuffer) error {\n\t\t\t\tif blobID == udmRepoBlobID {\n\t\t\t\t\treturn errors.New(\"fake-get-blob-error\")\n\t\t\t\t} else {\n\t\t\t\t\toutput.Write([]byte(`{\"tool\":\"\",\"buildVersion\":\"\",\"buildInfo\":\"\",\"uniqueID\":[],\"keyAlgo\":\"\",\"encryption\":\"\"}`))\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\t\t\t},\n\t\t\texpected: RepoStatusUnknown,\n\t\t\texpectedErr: \"error reading udm repo blob: fake-get-blob-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"no udm repo blob\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t\tStorageType: udmrepo.StorageTypeAzure,\n\t\t\t},\n\t\t\tbackendStore: new(storagemocks.Store),\n\t\t\treturnStore: new(storagemocks.Storage),\n\t\t\tretFuncGetBlob: func(ctx context.Context, blobID blob.ID, offset int64, length int64, output blob.OutputBuffer) error {\n\t\t\t\tif blobID == udmRepoBlobID {\n\t\t\t\t\treturn blob.ErrBlobNotFound\n\t\t\t\t} else {\n\t\t\t\t\toutput.Write([]byte(`{\"tool\":\"\",\"buildVersion\":\"\",\"buildInfo\":\"\",\"uniqueID\":[],\"keyAlgo\":\"\",\"encryption\":\"\"}`))\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\t\t\t},\n\t\t\texpected: RepoStatusNotInitialized,\n\t\t},\n\t\t{\n\t\t\tname: \"wrong udm repo metadata\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t\tStorageType: udmrepo.StorageTypeAzure,\n\t\t\t},\n\t\t\tbackendStore: new(storagemocks.Store),\n\t\t\treturnStore: new(storagemocks.Storage),\n\t\t\tretFuncGetBlob: func(ctx context.Context, blobID blob.ID, offset int64, length int64, output blob.OutputBuffer) error {\n\t\t\t\tif blobID == udmRepoBlobID {\n\t\t\t\t\toutput.Write([]byte(\"fake-buffer\"))\n\t\t\t\t} else {\n\t\t\t\t\toutput.Write([]byte(`{\"tool\":\"\",\"buildVersion\":\"\",\"buildInfo\":\"\",\"uniqueID\":[],\"keyAlgo\":\"\",\"encryption\":\"\"}`))\n\t\t\t\t}\n\n\t\t\t\treturn nil\n\t\t\t},\n\t\t\texpected: RepoStatusCorrupted,\n\t\t\texpectedErr: \"invalid udm repo blob: invalid character 'k' in literal false (expecting 'l')\",\n\t\t},\n\t\t{\n\t\t\tname: \"wrong unique id\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t\tStorageType: udmrepo.StorageTypeAzure,\n\t\t\t},\n\t\t\tbackendStore: new(storagemocks.Store),\n\t\t\treturnStore: new(storagemocks.Storage),\n\t\t\tretFuncGetBlob: func(ctx context.Context, blobID blob.ID, offset int64, length int64, output blob.OutputBuffer) error {\n\t\t\t\tif blobID == udmRepoBlobID {\n\t\t\t\t\toutput.Write([]byte(`{\"uniqueID\":[4,5,6]}`))\n\t\t\t\t} else {\n\t\t\t\t\toutput.Write([]byte(`{\"tool\":\"\",\"buildVersion\":\"\",\"buildInfo\":\"\",\"uniqueID\":[1,2,3],\"keyAlgo\":\"\",\"encryption\":\"\"}`))\n\t\t\t\t}\n\n\t\t\t\treturn nil\n\t\t\t},\n\t\t\texpected: RepoStatusCorrupted,\n\t\t\texpectedErr: \"unique ID doesn't match: [4 5 6]([1 2 3])\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"fake-file\",\n\t\t\t\tStorageType: udmrepo.StorageTypeAzure,\n\t\t\t},\n\t\t\tbackendStore: new(storagemocks.Store),\n\t\t\treturnStore: new(storagemocks.Storage),\n\t\t\tretFuncGetBlob: func(ctx context.Context, blobID blob.ID, offset int64, length int64, output blob.OutputBuffer) error {\n\t\t\t\tif blobID == udmRepoBlobID {\n\t\t\t\t\toutput.Write([]byte(`{\"uniqueID\":[1,2,3]}`))\n\t\t\t\t} else {\n\t\t\t\t\toutput.Write([]byte(`{\"tool\":\"\",\"buildVersion\":\"\",\"buildInfo\":\"\",\"uniqueID\":[1,2,3],\"keyAlgo\":\"\",\"encryption\":\"\"}`))\n\t\t\t\t}\n\n\t\t\t\treturn nil\n\t\t\t},\n\t\t\texpected: RepoStatusCreated,\n\t\t},\n\t}\n\n\tlogger := velerotest.NewLogger()\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tbackendStores = []kopiaBackendStore{\n\t\t\t\t{udmrepo.StorageTypeAzure, \"fake store\", tc.backendStore},\n\t\t\t\t{udmrepo.StorageTypeFs, \"fake store\", tc.backendStore},\n\t\t\t\t{udmrepo.StorageTypeGcs, \"fake store\", tc.backendStore},\n\t\t\t\t{udmrepo.StorageTypeS3, \"fake store\", tc.backendStore},\n\t\t\t}\n\n\t\t\tif tc.backendStore != nil {\n\t\t\t\ttc.backendStore.On(\"Connect\", mock.Anything, mock.Anything, mock.Anything).Return(tc.returnStore, tc.connectErr)\n\t\t\t\ttc.backendStore.On(\"Setup\", mock.Anything, mock.Anything, mock.Anything).Return(tc.setupError)\n\t\t\t}\n\n\t\t\tif tc.returnStore != nil {\n\t\t\t\ttc.returnStore.On(\"GetBlob\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(tc.retFuncGetBlob)\n\t\t\t}\n\n\t\t\tstatus, err := GetRepositoryStatus(t.Context(), tc.repoOptions, logger)\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t} else {\n\t\t\t\trequire.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\n\t\t\tassert.Equal(t, tc.expected, status)\n\t\t})\n\t}\n}\n\nfunc TestWriteInitParameters(t *testing.T) {\n\tvar directRpo *repomocks.MockRepository\n\tassertFullMaintIntervalEqual := func(expected, actual *maintenance.Params) bool {\n\t\treturn assert.Equal(t, expected.FullCycle.Interval, actual.FullCycle.Interval)\n\t}\n\ttestCases := []struct {\n\t\tname string\n\t\trepoOptions udmrepo.RepoOptions\n\t\treturnRepo *repomocks.MockRepository\n\t\treturnRepoWriter *repomocks.MockRepositoryWriter\n\t\trepoOpen func(context.Context, string, string, *repo.Options) (repo.Repository, error)\n\t\tnewRepoWriterError error\n\t\treplaceManifestError error\n\t\tgetParam func(context.Context, repo.Repository) (*maintenance.Params, error)\n\t\t// expected replacemanifest params to be received by maintenance.SetParams, and therefore writeInitParameters\n\t\texpectedReplaceManifestsParams *maintenance.Params\n\t\t// allows for asserting only certain fields are set as expected\n\t\tassertReplaceManifestsParams func(*maintenance.Params, *maintenance.Params) bool\n\t\tmockNewWriter bool\n\t\tmockClientOptions bool\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"repo open fail, repo not exist\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"/tmp\",\n\t\t\t\tGeneralOptions: map[string]string{},\n\t\t\t},\n\t\t\trepoOpen: func(context.Context, string, string, *repo.Options) (repo.Repository, error) {\n\t\t\t\treturn nil, os.ErrNotExist\n\t\t\t},\n\t\t\texpectedErr: \"error to open repo, repo doesn't exist: file does not exist\",\n\t\t},\n\t\t{\n\t\t\tname: \"repo open fail, other error\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"/tmp\",\n\t\t\t\tGeneralOptions: map[string]string{},\n\t\t\t},\n\t\t\trepoOpen: func(context.Context, string, string, *repo.Options) (repo.Repository, error) {\n\t\t\t\treturn nil, errors.New(\"fake-repo-open-error\")\n\t\t\t},\n\t\t\texpectedErr: \"error to open repo: fake-repo-open-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"get params error\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"/tmp\",\n\t\t\t\tGeneralOptions: map[string]string{},\n\t\t\t},\n\t\t\trepoOpen: func(context.Context, string, string, *repo.Options) (repo.Repository, error) {\n\t\t\t\treturn directRpo, nil\n\t\t\t},\n\t\t\tgetParam: func(context.Context, repo.Repository) (*maintenance.Params, error) {\n\t\t\t\treturn nil, errors.New(\"fake-get-param-error\")\n\t\t\t},\n\t\t\treturnRepo: repomocks.NewMockRepository(t),\n\t\t\texpectedErr: \"error getting existing maintenance params: fake-get-param-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"existing param with identical owner\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"/tmp\",\n\t\t\t\tGeneralOptions: map[string]string{},\n\t\t\t},\n\t\t\trepoOpen: func(context.Context, string, string, *repo.Options) (repo.Repository, error) {\n\t\t\t\treturn directRpo, nil\n\t\t\t},\n\t\t\tgetParam: func(context.Context, repo.Repository) (*maintenance.Params, error) {\n\t\t\t\treturn &maintenance.Params{\n\t\t\t\t\tOwner: \"default@default\",\n\t\t\t\t}, nil\n\t\t\t},\n\t\t\treturnRepo: repomocks.NewMockRepository(t),\n\t\t},\n\t\t{\n\t\t\tname: \"existing param with different owner\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"/tmp\",\n\t\t\t\tGeneralOptions: map[string]string{},\n\t\t\t},\n\t\t\trepoOpen: func(context.Context, string, string, *repo.Options) (repo.Repository, error) {\n\t\t\t\treturn directRpo, nil\n\t\t\t},\n\t\t\tgetParam: func(context.Context, repo.Repository) (*maintenance.Params, error) {\n\t\t\t\treturn &maintenance.Params{\n\t\t\t\t\tOwner: \"fake-owner\",\n\t\t\t\t}, nil\n\t\t\t},\n\t\t\treturnRepo: repomocks.NewMockRepository(t),\n\t\t\treturnRepoWriter: repomocks.NewMockRepositoryWriter(t),\n\t\t\texpectedReplaceManifestsParams: &maintenance.Params{\n\t\t\t\tFullCycle: maintenance.CycleParams{\n\t\t\t\t\tInterval: udmrepo.NormalGCInterval,\n\t\t\t\t},\n\t\t\t},\n\t\t\tmockNewWriter: true,\n\t\t\tmockClientOptions: true,\n\t\t\tassertReplaceManifestsParams: assertFullMaintIntervalEqual,\n\t\t},\n\t\t{\n\t\t\tname: \"write session fail\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"/tmp\",\n\t\t\t\tGeneralOptions: map[string]string{},\n\t\t\t},\n\t\t\trepoOpen: func(context.Context, string, string, *repo.Options) (repo.Repository, error) {\n\t\t\t\treturn directRpo, nil\n\t\t\t},\n\t\t\treturnRepo: repomocks.NewMockRepository(t),\n\t\t\tmockNewWriter: true,\n\t\t\tnewRepoWriterError: errors.New(\"fake-new-writer-error\"),\n\t\t\texpectedErr: \"error to init write repo parameters: unable to create writer: fake-new-writer-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"set repo param fail\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"/tmp\",\n\t\t\t\tGeneralOptions: map[string]string{},\n\t\t\t},\n\t\t\trepoOpen: func(context.Context, string, string, *repo.Options) (repo.Repository, error) {\n\t\t\t\treturn directRpo, nil\n\t\t\t},\n\t\t\treturnRepo: repomocks.NewMockRepository(t),\n\t\t\treturnRepoWriter: repomocks.NewMockRepositoryWriter(t),\n\t\t\tmockNewWriter: true,\n\t\t\tmockClientOptions: true,\n\t\t\treplaceManifestError: errors.New(\"fake-replace-manifest-error\"),\n\t\t\texpectedErr: \"error to init write repo parameters: error to set maintenance params: put manifest: fake-replace-manifest-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"repo with maintenance interval has expected params\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"/tmp\",\n\t\t\t\tStorageOptions: map[string]string{\n\t\t\t\t\tudmrepo.StoreOptionKeyFullMaintenanceInterval: string(udmrepo.FastGC),\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoOpen: func(context.Context, string, string, *repo.Options) (repo.Repository, error) {\n\t\t\t\treturn directRpo, nil\n\t\t\t},\n\t\t\treturnRepo: repomocks.NewMockRepository(t),\n\t\t\treturnRepoWriter: repomocks.NewMockRepositoryWriter(t),\n\t\t\tmockNewWriter: true,\n\t\t\tmockClientOptions: true,\n\t\t\texpectedReplaceManifestsParams: &maintenance.Params{\n\t\t\t\tFullCycle: maintenance.CycleParams{\n\t\t\t\t\tInterval: udmrepo.FastGCInterval,\n\t\t\t\t},\n\t\t\t},\n\t\t\tassertReplaceManifestsParams: assertFullMaintIntervalEqual,\n\t\t},\n\t\t{\n\t\t\tname: \"repo with empty maintenance interval has expected params\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"/tmp\",\n\t\t\t\tStorageOptions: map[string]string{\n\t\t\t\t\tudmrepo.StoreOptionKeyFullMaintenanceInterval: string(\"\"),\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoOpen: func(context.Context, string, string, *repo.Options) (repo.Repository, error) {\n\t\t\t\treturn directRpo, nil\n\t\t\t},\n\t\t\treturnRepo: repomocks.NewMockRepository(t),\n\t\t\treturnRepoWriter: repomocks.NewMockRepositoryWriter(t),\n\t\t\tmockNewWriter: true,\n\t\t\tmockClientOptions: true,\n\t\t\texpectedReplaceManifestsParams: &maintenance.Params{\n\t\t\t\tFullCycle: maintenance.CycleParams{\n\t\t\t\t\tInterval: udmrepo.NormalGCInterval,\n\t\t\t\t},\n\t\t\t},\n\t\t\tassertReplaceManifestsParams: assertFullMaintIntervalEqual,\n\t\t},\n\t\t{\n\t\t\tname: \"repo with invalid maintenance interval has expected errors\",\n\t\t\trepoOptions: udmrepo.RepoOptions{\n\t\t\t\tConfigFilePath: \"/tmp\",\n\t\t\t\tStorageOptions: map[string]string{\n\t\t\t\t\tudmrepo.StoreOptionKeyFullMaintenanceInterval: string(\"foo\"),\n\t\t\t\t},\n\t\t\t},\n\t\t\trepoOpen: func(context.Context, string, string, *repo.Options) (repo.Repository, error) {\n\t\t\t\treturn directRpo, nil\n\t\t\t},\n\t\t\treturnRepo: repomocks.NewMockRepository(t),\n\t\t\treturnRepoWriter: repomocks.NewMockRepositoryWriter(t),\n\t\t\tmockNewWriter: true,\n\t\t\texpectedErr: \"error to init write repo parameters: invalid full maintenance interval option foo\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlogger := velerotest.NewLogger()\n\t\t\tctx := t.Context()\n\n\t\t\tif tc.repoOpen != nil {\n\t\t\t\tkopiaRepoOpen = tc.repoOpen\n\t\t\t}\n\n\t\t\tif tc.returnRepo != nil {\n\t\t\t\tdirectRpo = tc.returnRepo\n\t\t\t}\n\n\t\t\tif tc.returnRepo != nil {\n\t\t\t\tif tc.mockNewWriter {\n\t\t\t\t\ttc.returnRepo.On(\"NewWriter\", mock.Anything, mock.Anything).Return(ctx, tc.returnRepoWriter, tc.newRepoWriterError)\n\t\t\t\t}\n\n\t\t\t\tif tc.mockClientOptions {\n\t\t\t\t\ttc.returnRepo.On(\"ClientOptions\").Return(repo.ClientOptions{})\n\t\t\t\t}\n\n\t\t\t\ttc.returnRepo.On(\"Close\", mock.Anything).Return(nil)\n\t\t\t}\n\n\t\t\tif tc.returnRepoWriter != nil {\n\t\t\t\ttc.returnRepoWriter.On(\"Close\", mock.Anything).Return(nil)\n\t\t\t\tif tc.replaceManifestError != nil {\n\t\t\t\t\ttc.returnRepoWriter.On(\"ReplaceManifests\", mock.Anything, mock.Anything, mock.Anything).Return(manifest.ID(\"\"), tc.replaceManifestError)\n\t\t\t\t}\n\t\t\t\tif tc.expectedReplaceManifestsParams != nil {\n\t\t\t\t\ttc.returnRepoWriter.On(\"ReplaceManifests\", mock.Anything, mock.AnythingOfType(\"map[string]string\"), mock.AnythingOfType(\"*maintenance.Params\")).Return(manifest.ID(\"\"), nil)\n\t\t\t\t\ttc.returnRepoWriter.On(\"Flush\", mock.Anything).Return(nil)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif tc.getParam != nil {\n\t\t\t\tfuncGetParam = tc.getParam\n\t\t\t} else {\n\t\t\t\tfuncGetParam = func(ctx context.Context, rep repo.Repository) (*maintenance.Params, error) {\n\t\t\t\t\treturn &maintenance.Params{}, nil\n\t\t\t\t}\n\t\t\t}\n\n\t\t\terr := writeInitParameters(ctx, tc.repoOptions, logger)\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t} else {\n\t\t\t\trequire.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t\tif tc.expectedReplaceManifestsParams != nil {\n\t\t\t\tactualReplaceManifestsParams, converted := tc.returnRepoWriter.Calls[0].Arguments.Get(2).(*maintenance.Params)\n\t\t\t\tassert.True(t, converted)\n\t\t\t\ttc.assertReplaceManifestsParams(tc.expectedReplaceManifestsParams, actualReplaceManifestsParams)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestWriteUdmRepoMetadata(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tretFuncGetBlob func(context.Context, blob.ID, int64, int64, blob.OutputBuffer) error\n\t\tretFuncPutBlob func(context.Context, blob.ID, blob.Bytes, blob.PutOptions) error\n\t\treplaceMetadata *udmRepoMetadata\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"get repo blob error\",\n\t\t\tretFuncGetBlob: func(context.Context, blob.ID, int64, int64, blob.OutputBuffer) error {\n\t\t\t\treturn errors.New(\"fake-get-blob-error\")\n\t\t\t},\n\t\t\texpectedErr: \"error reading format blob: fake-get-blob-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"wrong repo format\",\n\t\t\tretFuncGetBlob: func(ctx context.Context, id blob.ID, offset int64, length int64, output blob.OutputBuffer) error {\n\t\t\t\toutput.Write([]byte(\"fake-buffer\"))\n\t\t\t\treturn nil\n\t\t\t},\n\t\t\texpectedErr: \"invalid format blob: invalid character 'k' in literal false (expecting 'l')\",\n\t\t},\n\t\t{\n\t\t\tname: \"put udm repo metadata blob error\",\n\t\t\tretFuncGetBlob: func(ctx context.Context, blobID blob.ID, offset int64, length int64, output blob.OutputBuffer) error {\n\t\t\t\toutput.Write([]byte(`{\"tool\":\"\",\"buildVersion\":\"\",\"buildInfo\":\"\",\"uniqueID\":[],\"keyAlgo\":\"\",\"encryption\":\"\"}`))\n\t\t\t\treturn nil\n\t\t\t},\n\t\t\tretFuncPutBlob: func(context.Context, blob.ID, blob.Bytes, blob.PutOptions) error {\n\t\t\t\treturn errors.New(\"fake-put-blob-error\")\n\t\t\t},\n\t\t\texpectedErr: \"error writing udm repo metadata: fake-put-blob-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tretFuncGetBlob: func(ctx context.Context, blobID blob.ID, offset int64, length int64, output blob.OutputBuffer) error {\n\t\t\t\toutput.Write([]byte(`{\"tool\":\"\",\"buildVersion\":\"\",\"buildInfo\":\"\",\"uniqueID\":[],\"keyAlgo\":\"\",\"encryption\":\"\"}`))\n\t\t\t\treturn nil\n\t\t\t},\n\t\t\tretFuncPutBlob: func(context.Context, blob.ID, blob.Bytes, blob.PutOptions) error {\n\t\t\t\treturn nil\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tstorage := new(storagemocks.Storage)\n\t\t\tif tc.retFuncGetBlob != nil {\n\t\t\t\tstorage.On(\"GetBlob\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(tc.retFuncGetBlob)\n\t\t\t}\n\n\t\t\tif tc.retFuncPutBlob != nil {\n\t\t\t\tstorage.On(\"PutBlob\", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(tc.retFuncPutBlob)\n\t\t\t}\n\n\t\t\terr := writeUdmRepoMetadata(t.Context(), storage)\n\n\t\t\tif tc.expectedErr == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t} else {\n\t\t\t\trequire.EqualError(t, err, tc.expectedErr)\n\t\t\t}\n\t\t})\n\t}\n}\n\ntype testRecv struct {\n\tbuffer []byte\n}\n\nfunc (r *testRecv) Write(p []byte) (n int, err error) {\n\tr.buffer = append(r.buffer, p...)\n\treturn len(p), nil\n}\n\nfunc TestByteBuffer(t *testing.T) {\n\tbuffer := &byteBuffer{}\n\twritten, err := buffer.Write([]byte(\"12345\"))\n\trequire.NoError(t, err)\n\trequire.Equal(t, 5, written)\n\n\twritten, err = buffer.Write([]byte(\"67890\"))\n\trequire.NoError(t, err)\n\trequire.Equal(t, 5, written)\n\trequire.Equal(t, 10, buffer.Length())\n\n\trecv := &testRecv{}\n\tcopied, err := buffer.WriteTo(recv)\n\trequire.NoError(t, err)\n\trequire.Equal(t, int64(10), copied)\n\trequire.Equal(t, []byte(\"1234567890\"), recv.buffer)\n\n\tbuffer.Reset()\n\trequire.Zero(t, buffer.Length())\n}\n\nfunc TestByteBufferReader(t *testing.T) {\n\tbuffer := &byteBufferReader{buffer: []byte(\"123456789012345678901234567890\")}\n\toff, err := buffer.Seek(100, io.SeekStart)\n\trequire.Equal(t, int64(-1), off)\n\trequire.EqualError(t, err, \"invalid seek\")\n\trequire.Zero(t, buffer.pos)\n\n\toff, err = buffer.Seek(-100, io.SeekEnd)\n\trequire.Equal(t, int64(-1), off)\n\trequire.EqualError(t, err, \"invalid seek\")\n\trequire.Zero(t, buffer.pos)\n\n\toff, err = buffer.Seek(3, io.SeekCurrent)\n\trequire.Equal(t, int64(3), off)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 3, buffer.pos)\n\n\toutput := make([]byte, 6)\n\tread, err := buffer.Read(output)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 6, read)\n\trequire.Equal(t, 9, buffer.pos)\n\trequire.Equal(t, []byte(\"456789\"), output)\n\n\toff, err = buffer.Seek(21, io.SeekStart)\n\trequire.Equal(t, int64(21), off)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 21, buffer.pos)\n\n\toutput = make([]byte, 6)\n\tread, err = buffer.Read(output)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 6, read)\n\trequire.Equal(t, 27, buffer.pos)\n\trequire.Equal(t, []byte(\"234567\"), output)\n\n\toutput = make([]byte, 6)\n\tread, err = buffer.Read(output)\n\trequire.NoError(t, err)\n\trequire.Equal(t, 3, read)\n\trequire.Equal(t, 30, buffer.pos)\n\trequire.Equal(t, []byte{'8', '9', '0', 0, 0, 0}, output)\n\n\toutput = make([]byte, 6)\n\tread, err = buffer.Read(output)\n\trequire.Zero(t, read)\n\trequire.Equal(t, io.EOF, err)\n\n\terr = buffer.Close()\n\trequire.NoError(t, err)\n}\n" }, { "path": "pkg/repository/udmrepo/mocks/BackupRepo.go", "content": "// Code generated by mockery v2.39.1. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tcontext \"context\"\n\ttime \"time\"\n\n\tmock \"github.com/stretchr/testify/mock\"\n\n\tudmrepo \"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n)\n\n// BackupRepo is an autogenerated mock type for the BackupRepo type\ntype BackupRepo struct {\n\tmock.Mock\n}\n\n// Close provides a mock function with given fields: ctx\nfunc (_m *BackupRepo) Close(ctx context.Context) error {\n\tret := _m.Called(ctx)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Close\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context) error); ok {\n\t\tr0 = rf(ctx)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// ConcatenateObjects provides a mock function with given fields: ctx, objectIDs\nfunc (_m *BackupRepo) ConcatenateObjects(ctx context.Context, objectIDs []udmrepo.ID) (udmrepo.ID, error) {\n\tret := _m.Called(ctx, objectIDs)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for ConcatenateObjects\")\n\t}\n\n\tvar r0 udmrepo.ID\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(context.Context, []udmrepo.ID) (udmrepo.ID, error)); ok {\n\t\treturn rf(ctx, objectIDs)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, []udmrepo.ID) udmrepo.ID); ok {\n\t\tr0 = rf(ctx, objectIDs)\n\t} else {\n\t\tr0 = ret.Get(0).(udmrepo.ID)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, []udmrepo.ID) error); ok {\n\t\tr1 = rf(ctx, objectIDs)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// DeleteManifest provides a mock function with given fields: ctx, id\nfunc (_m *BackupRepo) DeleteManifest(ctx context.Context, id udmrepo.ID) error {\n\tret := _m.Called(ctx, id)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for DeleteManifest\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, udmrepo.ID) error); ok {\n\t\tr0 = rf(ctx, id)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// FindManifests provides a mock function with given fields: ctx, filter\nfunc (_m *BackupRepo) FindManifests(ctx context.Context, filter udmrepo.ManifestFilter) ([]*udmrepo.ManifestEntryMetadata, error) {\n\tret := _m.Called(ctx, filter)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for FindManifests\")\n\t}\n\n\tvar r0 []*udmrepo.ManifestEntryMetadata\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(context.Context, udmrepo.ManifestFilter) ([]*udmrepo.ManifestEntryMetadata, error)); ok {\n\t\treturn rf(ctx, filter)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, udmrepo.ManifestFilter) []*udmrepo.ManifestEntryMetadata); ok {\n\t\tr0 = rf(ctx, filter)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]*udmrepo.ManifestEntryMetadata)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, udmrepo.ManifestFilter) error); ok {\n\t\tr1 = rf(ctx, filter)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Flush provides a mock function with given fields: ctx\nfunc (_m *BackupRepo) Flush(ctx context.Context) error {\n\tret := _m.Called(ctx)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Flush\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context) error); ok {\n\t\tr0 = rf(ctx)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// GetAdvancedFeatures provides a mock function with given fields:\nfunc (_m *BackupRepo) GetAdvancedFeatures() udmrepo.AdvancedFeatureInfo {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetAdvancedFeatures\")\n\t}\n\n\tvar r0 udmrepo.AdvancedFeatureInfo\n\tif rf, ok := ret.Get(0).(func() udmrepo.AdvancedFeatureInfo); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(udmrepo.AdvancedFeatureInfo)\n\t}\n\n\treturn r0\n}\n\n// GetManifest provides a mock function with given fields: ctx, id, mani\nfunc (_m *BackupRepo) GetManifest(ctx context.Context, id udmrepo.ID, mani *udmrepo.RepoManifest) error {\n\tret := _m.Called(ctx, id, mani)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GetManifest\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, udmrepo.ID, *udmrepo.RepoManifest) error); ok {\n\t\tr0 = rf(ctx, id, mani)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// NewObjectWriter provides a mock function with given fields: ctx, opt\nfunc (_m *BackupRepo) NewObjectWriter(ctx context.Context, opt udmrepo.ObjectWriteOptions) udmrepo.ObjectWriter {\n\tret := _m.Called(ctx, opt)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for NewObjectWriter\")\n\t}\n\n\tvar r0 udmrepo.ObjectWriter\n\tif rf, ok := ret.Get(0).(func(context.Context, udmrepo.ObjectWriteOptions) udmrepo.ObjectWriter); ok {\n\t\tr0 = rf(ctx, opt)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(udmrepo.ObjectWriter)\n\t\t}\n\t}\n\n\treturn r0\n}\n\n// OpenObject provides a mock function with given fields: ctx, id\nfunc (_m *BackupRepo) OpenObject(ctx context.Context, id udmrepo.ID) (udmrepo.ObjectReader, error) {\n\tret := _m.Called(ctx, id)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for OpenObject\")\n\t}\n\n\tvar r0 udmrepo.ObjectReader\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(context.Context, udmrepo.ID) (udmrepo.ObjectReader, error)); ok {\n\t\treturn rf(ctx, id)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, udmrepo.ID) udmrepo.ObjectReader); ok {\n\t\tr0 = rf(ctx, id)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(udmrepo.ObjectReader)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, udmrepo.ID) error); ok {\n\t\tr1 = rf(ctx, id)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// PutManifest provides a mock function with given fields: ctx, mani\nfunc (_m *BackupRepo) PutManifest(ctx context.Context, mani udmrepo.RepoManifest) (udmrepo.ID, error) {\n\tret := _m.Called(ctx, mani)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for PutManifest\")\n\t}\n\n\tvar r0 udmrepo.ID\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(context.Context, udmrepo.RepoManifest) (udmrepo.ID, error)); ok {\n\t\treturn rf(ctx, mani)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, udmrepo.RepoManifest) udmrepo.ID); ok {\n\t\tr0 = rf(ctx, mani)\n\t} else {\n\t\tr0 = ret.Get(0).(udmrepo.ID)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, udmrepo.RepoManifest) error); ok {\n\t\tr1 = rf(ctx, mani)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Time provides a mock function with given fields:\nfunc (_m *BackupRepo) Time() time.Time {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Time\")\n\t}\n\n\tvar r0 time.Time\n\tif rf, ok := ret.Get(0).(func() time.Time); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(time.Time)\n\t}\n\n\treturn r0\n}\n\n// NewBackupRepo creates a new instance of BackupRepo. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\n// The first argument is typically a *testing.T value.\nfunc NewBackupRepo(t interface {\n\tmock.TestingT\n\tCleanup(func())\n}) *BackupRepo {\n\tmock := &BackupRepo{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/repository/udmrepo/mocks/BackupRepoService.go", "content": "// Code generated by mockery v2.53.2. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tcontext \"context\"\n\ttime \"time\"\n\n\tmock \"github.com/stretchr/testify/mock\"\n\n\tudmrepo \"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n)\n\n// BackupRepoService is an autogenerated mock type for the BackupRepoService type\ntype BackupRepoService struct {\n\tmock.Mock\n}\n\n// ClientSideCacheLimit provides a mock function with given fields: repoOption\nfunc (_m *BackupRepoService) ClientSideCacheLimit(repoOption map[string]string) int64 {\n\tret := _m.Called(repoOption)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for ClientSideCacheLimit\")\n\t}\n\n\tvar r0 int64\n\tif rf, ok := ret.Get(0).(func(map[string]string) int64); ok {\n\t\tr0 = rf(repoOption)\n\t} else {\n\t\tr0 = ret.Get(0).(int64)\n\t}\n\n\treturn r0\n}\n\n// Connect provides a mock function with given fields: ctx, repoOption\nfunc (_m *BackupRepoService) Connect(ctx context.Context, repoOption udmrepo.RepoOptions) error {\n\tret := _m.Called(ctx, repoOption)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Connect\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, udmrepo.RepoOptions) error); ok {\n\t\tr0 = rf(ctx, repoOption)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// Create provides a mock function with given fields: ctx, repoOption\nfunc (_m *BackupRepoService) Create(ctx context.Context, repoOption udmrepo.RepoOptions) error {\n\tret := _m.Called(ctx, repoOption)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Create\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, udmrepo.RepoOptions) error); ok {\n\t\tr0 = rf(ctx, repoOption)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// DefaultMaintenanceFrequency provides a mock function with no fields\nfunc (_m *BackupRepoService) DefaultMaintenanceFrequency() time.Duration {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for DefaultMaintenanceFrequency\")\n\t}\n\n\tvar r0 time.Duration\n\tif rf, ok := ret.Get(0).(func() time.Duration); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(time.Duration)\n\t}\n\n\treturn r0\n}\n\n// IsCreated provides a mock function with given fields: ctx, repoOption\nfunc (_m *BackupRepoService) IsCreated(ctx context.Context, repoOption udmrepo.RepoOptions) (bool, error) {\n\tret := _m.Called(ctx, repoOption)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for IsCreated\")\n\t}\n\n\tvar r0 bool\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(context.Context, udmrepo.RepoOptions) (bool, error)); ok {\n\t\treturn rf(ctx, repoOption)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, udmrepo.RepoOptions) bool); ok {\n\t\tr0 = rf(ctx, repoOption)\n\t} else {\n\t\tr0 = ret.Get(0).(bool)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, udmrepo.RepoOptions) error); ok {\n\t\tr1 = rf(ctx, repoOption)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Maintain provides a mock function with given fields: ctx, repoOption\nfunc (_m *BackupRepoService) Maintain(ctx context.Context, repoOption udmrepo.RepoOptions) error {\n\tret := _m.Called(ctx, repoOption)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Maintain\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, udmrepo.RepoOptions) error); ok {\n\t\tr0 = rf(ctx, repoOption)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// Open provides a mock function with given fields: ctx, repoOption\nfunc (_m *BackupRepoService) Open(ctx context.Context, repoOption udmrepo.RepoOptions) (udmrepo.BackupRepo, error) {\n\tret := _m.Called(ctx, repoOption)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Open\")\n\t}\n\n\tvar r0 udmrepo.BackupRepo\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(context.Context, udmrepo.RepoOptions) (udmrepo.BackupRepo, error)); ok {\n\t\treturn rf(ctx, repoOption)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, udmrepo.RepoOptions) udmrepo.BackupRepo); ok {\n\t\tr0 = rf(ctx, repoOption)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(udmrepo.BackupRepo)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, udmrepo.RepoOptions) error); ok {\n\t\tr1 = rf(ctx, repoOption)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// NewBackupRepoService creates a new instance of BackupRepoService. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\n// The first argument is typically a *testing.T value.\nfunc NewBackupRepoService(t interface {\n\tmock.TestingT\n\tCleanup(func())\n}) *BackupRepoService {\n\tmock := &BackupRepoService{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/repository/udmrepo/mocks/ObjectReader.go", "content": "// Code generated by mockery v2.39.1. DO NOT EDIT.\n\npackage mocks\n\nimport mock \"github.com/stretchr/testify/mock\"\n\n// ObjectReader is an autogenerated mock type for the ObjectReader type\ntype ObjectReader struct {\n\tmock.Mock\n}\n\n// Close provides a mock function with given fields:\nfunc (_m *ObjectReader) Close() error {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Close\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func() error); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// Length provides a mock function with given fields:\nfunc (_m *ObjectReader) Length() int64 {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Length\")\n\t}\n\n\tvar r0 int64\n\tif rf, ok := ret.Get(0).(func() int64); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(int64)\n\t}\n\n\treturn r0\n}\n\n// Read provides a mock function with given fields: p\nfunc (_m *ObjectReader) Read(p []byte) (int, error) {\n\tret := _m.Called(p)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Read\")\n\t}\n\n\tvar r0 int\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func([]byte) (int, error)); ok {\n\t\treturn rf(p)\n\t}\n\tif rf, ok := ret.Get(0).(func([]byte) int); ok {\n\t\tr0 = rf(p)\n\t} else {\n\t\tr0 = ret.Get(0).(int)\n\t}\n\n\tif rf, ok := ret.Get(1).(func([]byte) error); ok {\n\t\tr1 = rf(p)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Seek provides a mock function with given fields: offset, whence\nfunc (_m *ObjectReader) Seek(offset int64, whence int) (int64, error) {\n\tret := _m.Called(offset, whence)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Seek\")\n\t}\n\n\tvar r0 int64\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(int64, int) (int64, error)); ok {\n\t\treturn rf(offset, whence)\n\t}\n\tif rf, ok := ret.Get(0).(func(int64, int) int64); ok {\n\t\tr0 = rf(offset, whence)\n\t} else {\n\t\tr0 = ret.Get(0).(int64)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(int64, int) error); ok {\n\t\tr1 = rf(offset, whence)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// NewObjectReader creates a new instance of ObjectReader. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\n// The first argument is typically a *testing.T value.\nfunc NewObjectReader(t interface {\n\tmock.TestingT\n\tCleanup(func())\n}) *ObjectReader {\n\tmock := &ObjectReader{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/repository/udmrepo/mocks/ObjectWriter.go", "content": "// Code generated by mockery v2.39.1. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tmock \"github.com/stretchr/testify/mock\"\n\tudmrepo \"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n)\n\n// ObjectWriter is an autogenerated mock type for the ObjectWriter type\ntype ObjectWriter struct {\n\tmock.Mock\n}\n\n// Checkpoint provides a mock function with given fields:\nfunc (_m *ObjectWriter) Checkpoint() (udmrepo.ID, error) {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Checkpoint\")\n\t}\n\n\tvar r0 udmrepo.ID\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func() (udmrepo.ID, error)); ok {\n\t\treturn rf()\n\t}\n\tif rf, ok := ret.Get(0).(func() udmrepo.ID); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(udmrepo.ID)\n\t}\n\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Close provides a mock function with given fields:\nfunc (_m *ObjectWriter) Close() error {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Close\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func() error); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// Result provides a mock function with given fields:\nfunc (_m *ObjectWriter) Result() (udmrepo.ID, error) {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Result\")\n\t}\n\n\tvar r0 udmrepo.ID\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func() (udmrepo.ID, error)); ok {\n\t\treturn rf()\n\t}\n\tif rf, ok := ret.Get(0).(func() udmrepo.ID); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tr0 = ret.Get(0).(udmrepo.ID)\n\t}\n\n\tif rf, ok := ret.Get(1).(func() error); ok {\n\t\tr1 = rf()\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Seek provides a mock function with given fields: offset, whence\nfunc (_m *ObjectWriter) Seek(offset int64, whence int) (int64, error) {\n\tret := _m.Called(offset, whence)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Seek\")\n\t}\n\n\tvar r0 int64\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(int64, int) (int64, error)); ok {\n\t\treturn rf(offset, whence)\n\t}\n\tif rf, ok := ret.Get(0).(func(int64, int) int64); ok {\n\t\tr0 = rf(offset, whence)\n\t} else {\n\t\tr0 = ret.Get(0).(int64)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(int64, int) error); ok {\n\t\tr1 = rf(offset, whence)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// Write provides a mock function with given fields: p\nfunc (_m *ObjectWriter) Write(p []byte) (int, error) {\n\tret := _m.Called(p)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Write\")\n\t}\n\n\tvar r0 int\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func([]byte) (int, error)); ok {\n\t\treturn rf(p)\n\t}\n\tif rf, ok := ret.Get(0).(func([]byte) int); ok {\n\t\tr0 = rf(p)\n\t} else {\n\t\tr0 = ret.Get(0).(int)\n\t}\n\n\tif rf, ok := ret.Get(1).(func([]byte) error); ok {\n\t\tr1 = rf(p)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// NewObjectWriter creates a new instance of ObjectWriter. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\n// The first argument is typically a *testing.T value.\nfunc NewObjectWriter(t interface {\n\tmock.TestingT\n\tCleanup(func())\n}) *ObjectWriter {\n\tmock := &ObjectWriter{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/repository/udmrepo/repo.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage udmrepo\n\nimport (\n\t\"context\"\n\t\"io\"\n\t\"time\"\n)\n\ntype ID string\n\n// ManifestEntryMetadata is the metadata describing one manifest data\ntype ManifestEntryMetadata struct {\n\tID ID // The ID of the manifest data\n\tLength int32 // The data size of the manifest data\n\tLabels map[string]string // Labels saved together with the manifest data\n\tModTime time.Time // Modified time of the manifest data\n}\n\ntype RepoManifest struct {\n\tPayload any // The user data of manifest\n\tMetadata *ManifestEntryMetadata // The metadata data of manifest\n}\n\ntype ManifestFilter struct {\n\tLabels map[string]string\n}\n\nconst (\n\t// Below consts descrbe the data type of one object.\n\t// Metadata: This type describes how the data is organized.\n\t// For a file system backup, the Metadata describes a Dir or File.\n\t// For a block backup, the Metadata describes a Disk and its incremental link.\n\tObjectDataTypeUnknown int = 0\n\tObjectDataTypeMetadata int = 1\n\tObjectDataTypeData int = 2\n\n\t// Below consts defines the access mode when creating an object for write\n\tObjectDataAccessModeUnknown int = 0\n\tObjectDataAccessModeFile int = 1\n\tObjectDataAccessModeBlock int = 2\n\n\tObjectDataBackupModeUnknown int = 0\n\tObjectDataBackupModeFull int = 1\n\tObjectDataBackupModeInc int = 2\n)\n\n// ObjectWriteOptions defines the options when creating an object for write\ntype ObjectWriteOptions struct {\n\tFullPath string // Full logical path of the object\n\tDataType int // OBJECT_DATA_TYPE_*\n\tDescription string // A description of the object, could be empty\n\tPrefix ID // A prefix of the name used to save the object\n\tAccessMode int // OBJECT_DATA_ACCESS_*\n\tBackupMode int // OBJECT_DATA_BACKUP_*\n\tAsyncWrites int // Num of async writes for the object, 0 means no async write\n}\n\ntype AdvancedFeatureInfo struct {\n\tMultiPartBackup bool // if set to true, it means the repo supports multiple-part backup\n}\n\n// BackupRepoService is used to initialize, open or maintain a backup repository\ntype BackupRepoService interface {\n\t// Create creates a new backup repository.\n\t// repoOption: option to the backup repository and the underlying backup storage.\n\tCreate(ctx context.Context, repoOption RepoOptions) error\n\n\t// Connect connects to an existing backup repository.\n\t// repoOption: option to the backup repository and the underlying backup storage.\n\tConnect(ctx context.Context, repoOption RepoOptions) error\n\n\t// IsCreated checks if the backup repository has been created in the underlying backup storage.\n\t// repoOption: option to the underlying backup storage\n\tIsCreated(ctx context.Context, repoOption RepoOptions) (bool, error)\n\n\t// Open opens an backup repository that has been created/connected.\n\t// repoOption: options to open the backup repository and the underlying storage.\n\tOpen(ctx context.Context, repoOption RepoOptions) (BackupRepo, error)\n\n\t// Maintain is periodically called to maintain the backup repository to eliminate redundant data.\n\t// repoOption: options to maintain the backup repository.\n\tMaintain(ctx context.Context, repoOption RepoOptions) error\n\n\t// DefaultMaintenanceFrequency returns the defgault frequency of maintenance, callers refer this\n\t// frequency to maintain the backup repository to get the best maintenance performance\n\tDefaultMaintenanceFrequency() time.Duration\n\n\t// ClientSideCacheLimit returns the max cache size required on client side\n\tClientSideCacheLimit(repoOption map[string]string) int64\n}\n\n// BackupRepo provides the access to the backup repository\ntype BackupRepo interface {\n\t// OpenObject opens an existing object for read.\n\t// id: the object's unified identifier.\n\tOpenObject(ctx context.Context, id ID) (ObjectReader, error)\n\n\t// GetManifest gets a manifest data from the backup repository.\n\tGetManifest(ctx context.Context, id ID, mani *RepoManifest) error\n\n\t// FindManifests gets one or more manifest data that match the given labels\n\tFindManifests(ctx context.Context, filter ManifestFilter) ([]*ManifestEntryMetadata, error)\n\n\t// NewObjectWriter creates a new object and return the object's writer interface.\n\t// return: A unified identifier of the object on success.\n\tNewObjectWriter(ctx context.Context, opt ObjectWriteOptions) ObjectWriter\n\n\t// PutManifest saves a manifest object into the backup repository.\n\tPutManifest(ctx context.Context, mani RepoManifest) (ID, error)\n\n\t// DeleteManifest deletes a manifest object from the backup repository.\n\tDeleteManifest(ctx context.Context, id ID) error\n\n\t// Flush flushes all the backup repository data\n\tFlush(ctx context.Context) error\n\n\t// GetAdvancedFeatures returns the support for advanced features\n\tGetAdvancedFeatures() AdvancedFeatureInfo\n\n\t// ConcatenateObjects is for multiple-part backup, it concatenates multiple objects into one object\n\tConcatenateObjects(ctx context.Context, objectIDs []ID) (ID, error)\n\n\t// Time returns the local time of the backup repository. It may be different from the time of the caller\n\tTime() time.Time\n\n\t// Close closes the backup repository\n\tClose(ctx context.Context) error\n}\n\ntype ObjectReader interface {\n\tio.ReadCloser\n\tio.Seeker\n\n\t// Length returns the logical size of the object\n\tLength() int64\n}\n\ntype ObjectWriter interface {\n\tio.WriteCloser\n\n\t// Seeker is used in the cases that the object is not written sequentially\n\tio.Seeker\n\n\t// Checkpoint is periodically called to preserve the state of data written to the repo so far.\n\t// Checkpoint returns a unified identifier that represent the current state.\n\t// An empty ID could be returned on success if the backup repository doesn't support this.\n\tCheckpoint() (ID, error)\n\n\t// Result waits for the completion of the object write.\n\t// Result returns the object's unified identifier after the write completes.\n\tResult() (ID, error)\n}\n" }, { "path": "pkg/repository/udmrepo/repo_options.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage udmrepo\n\nimport (\n\t\"maps\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"time\"\n)\n\nconst (\n\tStorageTypeS3 = \"s3\"\n\tStorageTypeAzure = \"azure\"\n\tStorageTypeFs = \"filesystem\"\n\tStorageTypeGcs = \"gcs\"\n\n\tGenOptionMaintainMode = \"mode\"\n\tGenOptionMaintainFull = \"full\"\n\tGenOptionMaintainQuick = \"quick\"\n\n\tGenOptionOwnerName = \"username\"\n\tGenOptionOwnerDomain = \"domainname\"\n\n\tStoreOptionS3KeyID = \"accessKeyID\"\n\tStoreOptionS3Provider = \"providerName\"\n\tStoreOptionS3SecretKey = \"secretAccessKey\"\n\tStoreOptionS3Token = \"sessionToken\"\n\tStoreOptionS3Endpoint = \"endpoint\"\n\tStoreOptionS3DisableTLS = \"doNotUseTLS\"\n\tStoreOptionS3DisableTLSVerify = \"skipTLSVerify\"\n\n\tStoreOptionFsPath = \"fspath\"\n\n\tStoreOptionGcsReadonly = \"readonly\"\n\n\tStoreOptionOssBucket = \"bucket\"\n\tStoreOptionOssRegion = \"region\"\n\tStoreOptionCACert = \"caCert\"\n\n\tStoreOptionCredentialFile = \"credFile\"\n\tStoreOptionPrefix = \"prefix\"\n\tStoreOptionPrefixName = \"unified-repo\"\n\n\tStoreOptionGenHashAlgo = \"hashAlgo\"\n\tStoreOptionGenEncryptAlgo = \"encryptAlgo\"\n\tStoreOptionGenSplitAlgo = \"splitAlgo\"\n\n\tStoreOptionGenRetentionMode = \"retentionMode\"\n\tStoreOptionGenRetentionPeriod = \"retentionPeriod\"\n\tStoreOptionGenReadOnly = \"readOnly\"\n\n\tStoreOptionCacheLimit = \"cacheLimitMB\"\n\tStoreOptionCacheDir = \"cacheDir\"\n\n\tThrottleOptionReadOps = \"readOPS\"\n\tThrottleOptionWriteOps = \"writeOPS\"\n\tThrottleOptionListOps = \"listOPS\"\n\tThrottleOptionUploadBytes = \"uploadBytes\"\n\tThrottleOptionDownloadBytes = \"downloadBytes\"\n\t// FullMaintenanceInterval will overwrite kopia maintenance interval\n\t// options are fastGC for 12 hours, eagerGC for 6 hours, normalGC for 24 hours\n\tStoreOptionKeyFullMaintenanceInterval = \"fullMaintenanceInterval\"\n\tFastGC FullMaintenanceIntervalOptions = \"fastGC\"\n\tFastGCInterval time.Duration = 12 * time.Hour\n\tEagerGC FullMaintenanceIntervalOptions = \"eagerGC\"\n\tEagerGCInterval time.Duration = 6 * time.Hour\n\tNormalGC FullMaintenanceIntervalOptions = \"normalGC\"\n\tNormalGCInterval time.Duration = 24 * time.Hour\n)\n\ntype FullMaintenanceIntervalOptions string\n\nconst (\n\tdefaultUsername = \"default\"\n\tdefaultDomain = \"default\"\n)\n\ntype RepoOptions struct {\n\t// StorageType is a repository specific string to identify a backup storage, i.e., \"s3\", \"filesystem\"\n\tStorageType string\n\t// RepoPassword is the backup repository's password, if any\n\tRepoPassword string\n\t// ConfigFilePath is a custom path to save the repository's configuration, if any\n\tConfigFilePath string\n\t// GeneralOptions takes other repository specific options\n\tGeneralOptions map[string]string\n\t// StorageOptions takes storage specific options\n\tStorageOptions map[string]string\n\n\t// Description is a description of the backup repository/backup repository operation.\n\t// It is for logging/debugging purpose only and doesn't control any behavior of the backup repository.\n\tDescription string\n}\n\n// PasswordGetter defines the method to get a repository password.\ntype PasswordGetter interface {\n\tGetPassword(param any) (string, error)\n}\n\n// StoreOptionsGetter defines the methods to get the storage related options.\ntype StoreOptionsGetter interface {\n\tGetStoreType(param any) (string, error)\n\tGetStoreOptions(param any) (map[string]string, error)\n}\n\n// NewRepoOptions creates a new RepoOptions for different purpose\nfunc NewRepoOptions(optionFuncs ...func(*RepoOptions) error) (*RepoOptions, error) {\n\toptions := &RepoOptions{\n\t\tGeneralOptions: make(map[string]string),\n\t\tStorageOptions: make(map[string]string),\n\t}\n\n\tfor _, optionFunc := range optionFuncs {\n\t\terr := optionFunc(options)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\treturn options, nil\n}\n\n// WithPassword sets the RepoPassword to RepoOptions, the password is acquired through\n// the provided interface\nfunc WithPassword(getter PasswordGetter, param any) func(*RepoOptions) error {\n\treturn func(options *RepoOptions) error {\n\t\tpassword, err := getter.GetPassword(param)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\toptions.RepoPassword = password\n\n\t\treturn nil\n\t}\n}\n\n// WithConfigFile sets the ConfigFilePath to RepoOptions\nfunc WithConfigFile(workPath string, repoID string) func(*RepoOptions) error {\n\treturn func(options *RepoOptions) error {\n\t\toptions.ConfigFilePath = getRepoConfigFile(workPath, repoID)\n\t\treturn nil\n\t}\n}\n\n// WithGenOptions sets the GeneralOptions to RepoOptions\nfunc WithGenOptions(genOptions map[string]string) func(*RepoOptions) error {\n\treturn func(options *RepoOptions) error {\n\t\tfor k, v := range genOptions {\n\t\t\toptions.GeneralOptions[k] = v\n\t\t}\n\n\t\treturn nil\n\t}\n}\n\n// WithStoreOptions sets the StorageOptions to RepoOptions, the store options are acquired through\n// the provided interface\nfunc WithStoreOptions(getter StoreOptionsGetter, param any) func(*RepoOptions) error {\n\treturn func(options *RepoOptions) error {\n\t\tstoreType, err := getter.GetStoreType(param)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tstoreOptions, err := getter.GetStoreOptions(param)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\toptions.StorageType = storeType\n\n\t\tmaps.Copy(options.StorageOptions, storeOptions)\n\n\t\treturn nil\n\t}\n}\n\n// WithDescription sets the Description to RepoOptions\nfunc WithDescription(desc string) func(*RepoOptions) error {\n\treturn func(options *RepoOptions) error {\n\t\toptions.Description = desc\n\t\treturn nil\n\t}\n}\n\n// GetRepoUser returns the default username that is used to manipulate the Unified Repo\nfunc GetRepoUser() string {\n\treturn defaultUsername\n}\n\n// GetRepoDomain returns the default user domain that is used to manipulate the Unified Repo\nfunc GetRepoDomain() string {\n\treturn defaultDomain\n}\n\nfunc getRepoConfigFile(workPath string, repoID string) string {\n\tif workPath == \"\" {\n\t\thome := os.Getenv(\"HOME\")\n\t\tif home != \"\" {\n\t\t\tworkPath = filepath.Join(home, \"udmrepo\")\n\t\t} else {\n\t\t\tworkPath = filepath.Join(os.TempDir(), \"udmrepo\")\n\t\t}\n\t}\n\n\tname := \"repo-\" + strings.ToLower(repoID) + \".conf\"\n\n\treturn filepath.Join(workPath, name)\n}\n" }, { "path": "pkg/repository/udmrepo/service/service.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage service\n\nimport (\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo/kopialib\"\n)\n\n// Create creates an instance of BackupRepoService\nfunc Create(repoBackend string, logger logrus.FieldLogger) udmrepo.BackupRepoService {\n\treturn kopialib.NewKopiaRepoService(logger)\n}\n" }, { "path": "pkg/restic/command.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restic\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"strings\"\n)\n\n// Command represents a restic command.\ntype Command struct {\n\tCommand string\n\tRepoIdentifier string\n\tPasswordFile string\n\tCACertFile string\n\tDir string\n\tArgs []string\n\tExtraFlags []string\n\tEnv []string\n}\n\nfunc (c *Command) RepoName() string {\n\tif c.RepoIdentifier == \"\" {\n\t\treturn \"\"\n\t}\n\n\treturn c.RepoIdentifier[strings.LastIndex(c.RepoIdentifier, \"/\")+1:]\n}\n\n// StringSlice returns the command as a slice of strings.\nfunc (c *Command) StringSlice() []string {\n\tres := []string{\"restic\"}\n\n\tres = append(res, c.Command, repoFlag(c.RepoIdentifier))\n\tif c.PasswordFile != \"\" {\n\t\tres = append(res, passwordFlag(c.PasswordFile))\n\t}\n\tif c.CACertFile != \"\" {\n\t\tres = append(res, cacertFlag(c.CACertFile))\n\t}\n\n\t// If VELERO_SCRATCH_DIR is defined, put the restic cache within it. If not,\n\t// allow restic to choose the location. This makes running either in-cluster\n\t// or local (dev) work properly.\n\tif scratch := os.Getenv(\"VELERO_SCRATCH_DIR\"); scratch != \"\" {\n\t\tres = append(res, cacheDirFlag(filepath.Join(scratch, \".cache\", \"restic\")))\n\t}\n\n\tres = append(res, c.Args...)\n\tres = append(res, c.ExtraFlags...)\n\n\treturn res\n}\n\n// String returns the command as a string.\nfunc (c *Command) String() string {\n\treturn strings.Join(c.StringSlice(), \" \")\n}\n\n// Cmd returns an exec.Cmd for the command.\nfunc (c *Command) Cmd() *exec.Cmd {\n\tparts := c.StringSlice()\n\tcmd := exec.Command(parts[0], parts[1:]...) //nolint:gosec,noctx // Internal call. No need to check the parameter. No to add context for deprecated Restic.\n\tcmd.Dir = c.Dir\n\n\tif len(c.Env) > 0 {\n\t\tcmd.Env = c.Env\n\t}\n\n\treturn cmd\n}\n\nfunc repoFlag(repoIdentifier string) string {\n\treturn fmt.Sprintf(\"--repo=%s\", repoIdentifier)\n}\n\nfunc passwordFlag(file string) string {\n\treturn fmt.Sprintf(\"--password-file=%s\", file)\n}\n\nfunc cacheDirFlag(dir string) string {\n\treturn fmt.Sprintf(\"--cache-dir=%s\", dir)\n}\n\nfunc cacertFlag(path string) string {\n\treturn fmt.Sprintf(\"--cacert=%s\", path)\n}\n" }, { "path": "pkg/restic/command_factory.go", "content": "/*\nCopyright 2018, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restic\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n)\n\n// BackupCommand returns a Command for running a restic backup.\nfunc BackupCommand(repoIdentifier, passwordFile, path string, tags map[string]string) *Command {\n\t// --host flag is provided with a generic value because restic uses the host\n\t// to find a parent snapshot, and by default it will be the name of the daemonset pod\n\t// where the `restic backup` command is run. If this pod is recreated, we want to continue\n\t// taking incremental backups rather than triggering a full one due to a new pod name.\n\n\treturn &Command{\n\t\tCommand: \"backup\",\n\t\tRepoIdentifier: repoIdentifier,\n\t\tPasswordFile: passwordFile,\n\t\tDir: path,\n\t\tArgs: []string{\".\"},\n\t\tExtraFlags: append(backupTagFlags(tags), \"--host=velero\", \"--json\"),\n\t}\n}\n\nfunc backupTagFlags(tags map[string]string) []string {\n\tvar flags []string\n\tfor k, v := range tags {\n\t\tflags = append(flags, fmt.Sprintf(\"--tag=%s=%s\", k, v))\n\t}\n\treturn flags\n}\n\n// RestoreCommand returns a Command for running a restic restore.\nfunc RestoreCommand(repoIdentifier, passwordFile, snapshotID, target string) *Command {\n\treturn &Command{\n\t\tCommand: \"restore\",\n\t\tRepoIdentifier: repoIdentifier,\n\t\tPasswordFile: passwordFile,\n\t\tDir: target,\n\t\tArgs: []string{snapshotID},\n\t\tExtraFlags: []string{\"--target=.\"},\n\t}\n}\n\n// GetSnapshotCommand returns a Command for running a restic (get) snapshots.\nfunc GetSnapshotCommand(repoIdentifier, passwordFile string, tags map[string]string) *Command {\n\treturn &Command{\n\t\tCommand: \"snapshots\",\n\t\tRepoIdentifier: repoIdentifier,\n\t\tPasswordFile: passwordFile,\n\t\t// \"--last\" is replaced by \"--latest=1\" in restic v0.12.1\n\t\tExtraFlags: []string{\"--json\", \"--latest=1\", getSnapshotTagFlag(tags)},\n\t}\n}\n\nfunc getSnapshotTagFlag(tags map[string]string) string {\n\tvar tagFilters []string\n\tfor k, v := range tags {\n\t\ttagFilters = append(tagFilters, fmt.Sprintf(\"%s=%s\", k, v))\n\t}\n\n\treturn fmt.Sprintf(\"--tag=%s\", strings.Join(tagFilters, \",\"))\n}\n\nfunc InitCommand(repoIdentifier string) *Command {\n\treturn &Command{\n\t\tCommand: \"init\",\n\t\tRepoIdentifier: repoIdentifier,\n\t}\n}\n\nfunc SnapshotsCommand(repoIdentifier string) *Command {\n\treturn &Command{\n\t\tCommand: \"snapshots\",\n\t\tRepoIdentifier: repoIdentifier,\n\t}\n}\n\nfunc PruneCommand(repoIdentifier string) *Command {\n\treturn &Command{\n\t\tCommand: \"prune\",\n\t\tRepoIdentifier: repoIdentifier,\n\t}\n}\n\nfunc ForgetCommand(repoIdentifier, snapshotID string) *Command {\n\treturn &Command{\n\t\tCommand: \"forget\",\n\t\tRepoIdentifier: repoIdentifier,\n\t\tArgs: []string{snapshotID},\n\t}\n}\n\nfunc UnlockCommand(repoIdentifier string) *Command {\n\treturn &Command{\n\t\tCommand: \"unlock\",\n\t\tRepoIdentifier: repoIdentifier,\n\t}\n}\n\nfunc StatsCommand(repoIdentifier, passwordFile, snapshotID string) *Command {\n\treturn &Command{\n\t\tCommand: \"stats\",\n\t\tRepoIdentifier: repoIdentifier,\n\t\tPasswordFile: passwordFile,\n\t\tArgs: []string{snapshotID},\n\t\tExtraFlags: []string{\"--json\"},\n\t}\n}\n" }, { "path": "pkg/restic/command_factory_test.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage restic\n\nimport (\n\t\"sort\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestBackupCommand(t *testing.T) {\n\tc := BackupCommand(\"repo-id\", \"password-file\", \"path\", map[string]string{\"foo\": \"bar\", \"c\": \"d\"})\n\n\tassert.Equal(t, \"backup\", c.Command)\n\tassert.Equal(t, \"repo-id\", c.RepoIdentifier)\n\tassert.Equal(t, \"password-file\", c.PasswordFile)\n\tassert.Equal(t, \"path\", c.Dir)\n\tassert.Equal(t, []string{\".\"}, c.Args)\n\n\texpected := []string{\"--tag=foo=bar\", \"--tag=c=d\", \"--host=velero\", \"--json\"}\n\tsort.Strings(expected)\n\tsort.Strings(c.ExtraFlags)\n\tassert.Equal(t, expected, c.ExtraFlags)\n}\n\nfunc TestRestoreCommand(t *testing.T) {\n\tc := RestoreCommand(\"repo-id\", \"password-file\", \"snapshot-id\", \"target\")\n\n\tassert.Equal(t, \"restore\", c.Command)\n\tassert.Equal(t, \"repo-id\", c.RepoIdentifier)\n\tassert.Equal(t, \"password-file\", c.PasswordFile)\n\tassert.Equal(t, \"target\", c.Dir)\n\tassert.Equal(t, []string{\"snapshot-id\"}, c.Args)\n\tassert.Equal(t, []string{\"--target=.\"}, c.ExtraFlags)\n}\n\nfunc TestGetSnapshotCommand(t *testing.T) {\n\texpectedTags := map[string]string{\"foo\": \"bar\", \"c\": \"d\"}\n\tc := GetSnapshotCommand(\"repo-id\", \"password-file\", expectedTags)\n\n\tassert.Equal(t, \"snapshots\", c.Command)\n\tassert.Equal(t, \"repo-id\", c.RepoIdentifier)\n\tassert.Equal(t, \"password-file\", c.PasswordFile)\n\n\t// set up expected flag names\n\texpectedFlags := []string{\"--json\", \"--latest=1\", \"--tag\"}\n\t// for tracking actual flag names\n\tactualFlags := []string{}\n\t// for tracking actual --tag values as a map\n\tactualTags := make(map[string]string)\n\n\t// loop through actual flags\n\tfor _, flag := range c.ExtraFlags {\n\t\t// split into 2 parts from the first = sign (if any)\n\t\tparts := strings.SplitN(flag, \"=\", 2)\n\n\t\t// convert --tag data to a map\n\t\tif parts[0] == \"--tag\" {\n\t\t\tactualFlags = append(actualFlags, parts[0])\n\n\t\t\t// split based on ,\n\t\t\ttags := strings.Split(parts[1], \",\")\n\t\t\t// loop through each key-value tag pair\n\t\t\tfor _, tag := range tags {\n\t\t\t\t// split the pair on =\n\t\t\t\tkvs := strings.Split(tag, \"=\")\n\t\t\t\t// record actual key & value\n\t\t\t\tactualTags[kvs[0]] = kvs[1]\n\t\t\t}\n\t\t} else {\n\t\t\tactualFlags = append(actualFlags, flag)\n\t\t}\n\t}\n\n\tassert.Equal(t, expectedFlags, actualFlags)\n\tassert.Equal(t, expectedTags, actualTags)\n}\n\nfunc TestInitCommand(t *testing.T) {\n\tc := InitCommand(\"repo-id\")\n\n\tassert.Equal(t, \"init\", c.Command)\n\tassert.Equal(t, \"repo-id\", c.RepoIdentifier)\n}\n\nfunc TestSnapshotsCommand(t *testing.T) {\n\tc := SnapshotsCommand(\"repo-id\")\n\n\tassert.Equal(t, \"snapshots\", c.Command)\n\tassert.Equal(t, \"repo-id\", c.RepoIdentifier)\n}\n\nfunc TestPruneCommand(t *testing.T) {\n\tc := PruneCommand(\"repo-id\")\n\n\tassert.Equal(t, \"prune\", c.Command)\n\tassert.Equal(t, \"repo-id\", c.RepoIdentifier)\n}\n\nfunc TestForgetCommand(t *testing.T) {\n\tc := ForgetCommand(\"repo-id\", \"snapshot-id\")\n\n\tassert.Equal(t, \"forget\", c.Command)\n\tassert.Equal(t, \"repo-id\", c.RepoIdentifier)\n\tassert.Equal(t, []string{\"snapshot-id\"}, c.Args)\n}\n\nfunc TestStatsCommand(t *testing.T) {\n\tc := StatsCommand(\"repo-id\", \"password-file\", \"snapshot-id\")\n\n\tassert.Equal(t, \"stats\", c.Command)\n\tassert.Equal(t, \"repo-id\", c.RepoIdentifier)\n\tassert.Equal(t, \"password-file\", c.PasswordFile)\n\tassert.Equal(t, []string{\"snapshot-id\"}, c.Args)\n\tassert.Equal(t, []string{\"--json\"}, c.ExtraFlags)\n}\n" }, { "path": "pkg/restic/command_test.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restic\n\nimport (\n\t\"os\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestRepoName(t *testing.T) {\n\tc := &Command{RepoIdentifier: \"\"}\n\tassert.Empty(t, c.RepoName())\n\n\tc.RepoIdentifier = \"s3:s3.amazonaws.com/bucket/prefix/repo\"\n\tassert.Equal(t, \"repo\", c.RepoName())\n\n\tc.RepoIdentifier = \"azure:bucket:/repo\"\n\tassert.Equal(t, \"repo\", c.RepoName())\n\n\tc.RepoIdentifier = \"gs:bucket:/prefix/repo\"\n\tassert.Equal(t, \"repo\", c.RepoName())\n}\n\nfunc TestStringSlice(t *testing.T) {\n\tc := &Command{\n\t\tCommand: \"cmd\",\n\t\tRepoIdentifier: \"repo-id\",\n\t\tPasswordFile: \"/path/to/password-file\",\n\t\tDir: \"/some/pwd\",\n\t\tArgs: []string{\"arg-1\", \"arg-2\"},\n\t\tExtraFlags: []string{\"--foo=bar\"},\n\t}\n\n\trequire.NoError(t, os.Unsetenv(\"VELERO_SCRATCH_DIR\"))\n\tassert.Equal(t, []string{\n\t\t\"restic\",\n\t\t\"cmd\",\n\t\t\"--repo=repo-id\",\n\t\t\"--password-file=/path/to/password-file\",\n\t\t\"arg-1\",\n\t\t\"arg-2\",\n\t\t\"--foo=bar\",\n\t}, c.StringSlice())\n\n\tos.Setenv(\"VELERO_SCRATCH_DIR\", \"/foo\")\n\tassert.Equal(t, []string{\n\t\t\"restic\",\n\t\t\"cmd\",\n\t\t\"--repo=repo-id\",\n\t\t\"--password-file=/path/to/password-file\",\n\t\t\"--cache-dir=/foo/.cache/restic\",\n\t\t\"arg-1\",\n\t\t\"arg-2\",\n\t\t\"--foo=bar\",\n\t}, c.StringSlice())\n\n\trequire.NoError(t, os.Unsetenv(\"VELERO_SCRATCH_DIR\"))\n}\n\nfunc TestString(t *testing.T) {\n\tc := &Command{\n\t\tCommand: \"cmd\",\n\t\tRepoIdentifier: \"repo-id\",\n\t\tPasswordFile: \"/path/to/password-file\",\n\t\tDir: \"/some/pwd\",\n\t\tArgs: []string{\"arg-1\", \"arg-2\"},\n\t\tExtraFlags: []string{\"--foo=bar\"},\n\t}\n\n\trequire.NoError(t, os.Unsetenv(\"VELERO_SCRATCH_DIR\"))\n\tassert.Equal(t, \"restic cmd --repo=repo-id --password-file=/path/to/password-file arg-1 arg-2 --foo=bar\", c.String())\n}\n\nfunc TestCmd(t *testing.T) {\n\tc := &Command{\n\t\tCommand: \"cmd\",\n\t\tRepoIdentifier: \"repo-id\",\n\t\tPasswordFile: \"/path/to/password-file\",\n\t\tDir: \"/some/pwd\",\n\t\tArgs: []string{\"arg-1\", \"arg-2\"},\n\t\tExtraFlags: []string{\"--foo=bar\"},\n\t}\n\n\trequire.NoError(t, os.Unsetenv(\"VELERO_SCRATCH_DIR\"))\n\texecCmd := c.Cmd()\n\n\tassert.Equal(t, c.StringSlice(), execCmd.Args)\n\tassert.Equal(t, c.Dir, execCmd.Dir)\n}\n" }, { "path": "pkg/restic/common.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restic\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"strconv\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\trepoconfig \"github.com/vmware-tanzu/velero/pkg/repository/config\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n)\n\nconst (\n\n\t// DefaultMaintenanceFrequency is the default time interval\n\t// at which restic prune is run.\n\tDefaultMaintenanceFrequency = 7 * 24 * time.Hour\n\n\t// insecureSkipTLSVerifyKey is the flag in BackupStorageLocation's config\n\t// to indicate whether to skip TLS verify to setup insecure HTTPS connection.\n\tinsecureSkipTLSVerifyKey = \"insecureSkipTLSVerify\"\n\n\t// resticInsecureTLSFlag is the flag for Restic command line to indicate\n\t// skip TLS verify on https connection.\n\tresticInsecureTLSFlag = \"--insecure-tls\"\n)\n\n// TempCACertFile creates a temp file containing a CA bundle\n// and returns its path. The caller should generally call os.Remove()\n// to remove the file when done with it.\nfunc TempCACertFile(caCert []byte, bsl string, fs filesystem.Interface) (string, error) {\n\tfile, err := fs.TempFile(\"\", fmt.Sprintf(\"cacert-%s\", bsl))\n\tif err != nil {\n\t\treturn \"\", errors.WithStack(err)\n\t}\n\n\tif _, err := file.Write(caCert); err != nil {\n\t\t// nothing we can do about an error closing the file here, and we're\n\t\t// already returning an error about the write failing.\n\t\tfile.Close()\n\t\treturn \"\", errors.WithStack(err)\n\t}\n\n\tname := file.Name()\n\n\tif err := file.Close(); err != nil {\n\t\treturn \"\", errors.WithStack(err)\n\t}\n\n\treturn name, nil\n}\n\n// environ is a slice of strings representing the environment, in the form \"key=value\".\ntype environ []string\n\n// Unset a single environment variable.\nfunc (e *environ) Unset(key string) {\n\tfor i := range *e {\n\t\tif strings.HasPrefix((*e)[i], key+\"=\") {\n\t\t\t(*e)[i] = (*e)[len(*e)-1]\n\t\t\t*e = (*e)[:len(*e)-1]\n\t\t\tbreak\n\t\t}\n\t}\n}\n\n// CmdEnv returns a list of environment variables (in the format var=val) that\n// should be used when running a restic command for a particular backend provider.\n// This list is the current environment, plus any provider-specific variables restic needs.\nfunc CmdEnv(backupLocation *velerov1api.BackupStorageLocation, credentialFileStore credentials.FileStore) ([]string, error) {\n\tvar env environ\n\tenv = os.Environ()\n\tcustomEnv := map[string]string{}\n\tvar err error\n\n\tconfig := backupLocation.Spec.Config\n\tif config == nil {\n\t\tconfig = map[string]string{}\n\t}\n\n\tif backupLocation.Spec.Credential != nil {\n\t\tcredsFile, err := credentialFileStore.Path(backupLocation.Spec.Credential)\n\t\tif err != nil {\n\t\t\treturn []string{}, errors.WithStack(err)\n\t\t}\n\t\tconfig[repoconfig.CredentialsFileKey] = credsFile\n\t}\n\n\tbackendType := repoconfig.GetBackendType(backupLocation.Spec.Provider, backupLocation.Spec.Config)\n\n\tswitch backendType {\n\tcase repoconfig.AWSBackend:\n\t\tcustomEnv, err = repoconfig.GetS3ResticEnvVars(config)\n\t\tif err != nil {\n\t\t\treturn []string{}, err\n\t\t}\n\tcase repoconfig.AzureBackend:\n\t\tcustomEnv, err = repoconfig.GetAzureResticEnvVars(config)\n\t\tif err != nil {\n\t\t\treturn []string{}, err\n\t\t}\n\tcase repoconfig.GCPBackend:\n\t\tcustomEnv, err = repoconfig.GetGCPResticEnvVars(config)\n\t\tif err != nil {\n\t\t\treturn []string{}, err\n\t\t}\n\t}\n\n\tfor k, v := range customEnv {\n\t\tenv.Unset(k)\n\t\tif v == \"\" {\n\t\t\tcontinue\n\t\t}\n\t\tenv = append(env, fmt.Sprintf(\"%s=%s\", k, v))\n\t}\n\n\treturn env, nil\n}\n\n// GetInsecureSkipTLSVerifyFromBSL get insecureSkipTLSVerify flag from BSL configuration,\n// Then return --insecure-tls flag with boolean value as result.\nfunc GetInsecureSkipTLSVerifyFromBSL(backupLocation *velerov1api.BackupStorageLocation, logger logrus.FieldLogger) string {\n\tresult := \"\"\n\n\tif backupLocation == nil {\n\t\tlogger.Info(\"bsl is nil. return empty.\")\n\t\treturn result\n\t}\n\n\tif insecure, _ := strconv.ParseBool(backupLocation.Spec.Config[insecureSkipTLSVerifyKey]); insecure {\n\t\tlogger.Debugf(\"set --insecure-tls=true for Restic command according to BSL %s config\", backupLocation.Name)\n\t\tresult = resticInsecureTLSFlag + \"=true\"\n\t\treturn result\n\t}\n\n\treturn result\n}\n" }, { "path": "pkg/restic/common_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restic\n\nimport (\n\t\"os\"\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestTempCACertFile(t *testing.T) {\n\tvar (\n\t\tfs = velerotest.NewFakeFileSystem()\n\t\tcaCertData = []byte(\"cacert\")\n\t)\n\n\tfileName, err := TempCACertFile(caCertData, \"default\", fs)\n\trequire.NoError(t, err)\n\n\tcontents, err := fs.ReadFile(fileName)\n\trequire.NoError(t, err)\n\n\tassert.Equal(t, string(caCertData), string(contents))\n\n\tos.Remove(fileName)\n}\n\nfunc TestGetInsecureSkipTLSVerifyFromBSL(t *testing.T) {\n\tlog := logrus.StandardLogger()\n\ttests := []struct {\n\t\tname string\n\t\tbackupLocation *velerov1api.BackupStorageLocation\n\t\tlogger logrus.FieldLogger\n\t\texpected string\n\t}{\n\t\t{\n\t\t\t\"Test with nil BSL. Should return empty string.\",\n\t\t\tnil,\n\t\t\tlog,\n\t\t\t\"\",\n\t\t},\n\t\t{\n\t\t\t\"Test BSL with no configuration. Should return empty string.\",\n\t\t\t&velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"azure\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tlog,\n\t\t\t\"\",\n\t\t},\n\t\t{\n\t\t\t\"Test with AWS BSL's insecureSkipTLSVerify set to false.\",\n\t\t\t&velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"aws\",\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"insecureSkipTLSVerify\": \"false\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tlog,\n\t\t\t\"\",\n\t\t},\n\t\t{\n\t\t\t\"Test with AWS BSL's insecureSkipTLSVerify set to true.\",\n\t\t\t&velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"aws\",\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"insecureSkipTLSVerify\": \"true\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tlog,\n\t\t\t\"--insecure-tls=true\",\n\t\t},\n\t\t{\n\t\t\t\"Test with Azure BSL's insecureSkipTLSVerify set to invalid.\",\n\t\t\t&velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"azure\",\n\t\t\t\t\tConfig: map[string]string{\n\t\t\t\t\t\t\"insecureSkipTLSVerify\": \"invalid\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tlog,\n\t\t\t\"\",\n\t\t},\n\t\t{\n\t\t\t\"Test with GCP without insecureSkipTLSVerify.\",\n\t\t\t&velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"gcp\",\n\t\t\t\t\tConfig: map[string]string{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tlog,\n\t\t\t\"\",\n\t\t},\n\t\t{\n\t\t\t\"Test with AWS without config.\",\n\t\t\t&velerov1api.BackupStorageLocation{\n\t\t\t\tSpec: velerov1api.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"aws\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tlog,\n\t\t\t\"\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tres := GetInsecureSkipTLSVerifyFromBSL(test.backupLocation, test.logger)\n\n\t\t\tassert.Equal(t, test.expected, res)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/restic/exec_commands.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restic\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/exec\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n)\n\nconst restoreProgressCheckInterval = 10 * time.Second\nconst backupProgressCheckInterval = 10 * time.Second\n\nvar fileSystem = filesystem.NewFileSystem()\n\ntype backupStatusLine struct {\n\tMessageType string `json:\"message_type\"`\n\t// seen in status lines\n\tTotalBytes int64 `json:\"total_bytes\"`\n\tBytesDone int64 `json:\"bytes_done\"`\n\t// seen in summary line at the end\n\tTotalBytesProcessed int64 `json:\"total_bytes_processed\"`\n}\n\n// GetSnapshotID runs provided 'restic snapshots' command to get the ID of a snapshot\n// and an error if a unique snapshot cannot be identified.\nfunc GetSnapshotID(snapshotIDCmd *Command) (string, error) {\n\tstdout, stderr, err := exec.RunCommand(snapshotIDCmd.Cmd())\n\tif err != nil {\n\t\treturn \"\", errors.Wrapf(err, \"error running command, stderr=%s\", stderr)\n\t}\n\n\ttype snapshotID struct {\n\t\tShortID string `json:\"short_id\"`\n\t}\n\n\tvar snapshots []snapshotID\n\tif err := json.Unmarshal([]byte(stdout), &snapshots); err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error unmarshaling restic snapshots result\")\n\t}\n\n\tif len(snapshots) != 1 {\n\t\treturn \"\", errors.Errorf(\"expected one matching snapshot by command: %s, got %d\", snapshotIDCmd.String(), len(snapshots))\n\t}\n\n\treturn snapshots[0].ShortID, nil\n}\n\n// RunBackup runs a `restic backup` command and watches the output to provide\n// progress updates to the caller.\nfunc RunBackup(backupCmd *Command, log logrus.FieldLogger, updater uploader.ProgressUpdater) (string, string, error) {\n\t// buffers for copying command stdout/err output into\n\tstdoutBuf := new(bytes.Buffer)\n\tstderrBuf := new(bytes.Buffer)\n\n\t// create a channel to signal when to end the goroutine scanning for progress\n\t// updates\n\tquit := make(chan struct{})\n\n\tcmd := backupCmd.Cmd()\n\tcmd.Stdout = stdoutBuf\n\tcmd.Stderr = stderrBuf\n\n\terr := cmd.Start()\n\tif err != nil {\n\t\texec.LogErrorAsExitCode(err, log)\n\t\treturn stdoutBuf.String(), stderrBuf.String(), err\n\t}\n\n\tgo func() {\n\t\tticker := time.NewTicker(backupProgressCheckInterval)\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-ticker.C:\n\t\t\t\tlastLine := getLastLine(stdoutBuf.Bytes())\n\t\t\t\tif len(lastLine) > 0 {\n\t\t\t\t\tstat, err := decodeBackupStatusLine(lastLine)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\tlog.WithError(err).Errorf(\"error getting restic backup progress\")\n\t\t\t\t\t}\n\n\t\t\t\t\t// if the line contains a non-empty bytes_done field, we can update the\n\t\t\t\t\t// caller with the progress\n\t\t\t\t\tif stat.BytesDone != 0 {\n\t\t\t\t\t\tupdater.UpdateProgress(&uploader.Progress{\n\t\t\t\t\t\t\tTotalBytes: stat.TotalBytes,\n\t\t\t\t\t\t\tBytesDone: stat.BytesDone,\n\t\t\t\t\t\t})\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\tcase <-quit:\n\t\t\t\tticker.Stop()\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}()\n\n\terr = cmd.Wait()\n\tif err != nil {\n\t\texec.LogErrorAsExitCode(err, log)\n\t\treturn stdoutBuf.String(), stderrBuf.String(), err\n\t}\n\tquit <- struct{}{}\n\n\tsummary, err := getSummaryLine(stdoutBuf.Bytes())\n\tif err != nil {\n\t\treturn stdoutBuf.String(), stderrBuf.String(), err\n\t}\n\tstat, err := decodeBackupStatusLine(summary)\n\tif err != nil {\n\t\treturn stdoutBuf.String(), stderrBuf.String(), err\n\t}\n\tif stat.MessageType != \"summary\" {\n\t\treturn stdoutBuf.String(), stderrBuf.String(), errors.WithStack(fmt.Errorf(\"error getting restic backup summary: %s\", string(summary)))\n\t}\n\n\t// update progress to 100%\n\tupdater.UpdateProgress(&uploader.Progress{\n\t\tTotalBytes: stat.TotalBytesProcessed,\n\t\tBytesDone: stat.TotalBytesProcessed,\n\t})\n\n\treturn string(summary), stderrBuf.String(), nil\n}\n\nfunc decodeBackupStatusLine(lastLine []byte) (backupStatusLine, error) {\n\tvar stat backupStatusLine\n\tif err := json.Unmarshal(lastLine, &stat); err != nil {\n\t\treturn stat, errors.Wrapf(err, \"unable to decode backup JSON line: %s\", string(lastLine))\n\t}\n\treturn stat, nil\n}\n\n// getLastLine returns the last line of a byte array. The string is assumed to\n// have a newline at the end of it, so this returns the substring between the\n// last two newlines.\nfunc getLastLine(b []byte) []byte {\n\tif len(b) == 0 {\n\t\treturn []byte(\"\")\n\t}\n\t// subslice the byte array to ignore the newline at the end of the string\n\tlastNewLineIdx := bytes.LastIndex(b[:len(b)-1], []byte(\"\\n\"))\n\treturn b[lastNewLineIdx+1 : len(b)-1]\n}\n\n// getSummaryLine looks for the summary JSON line\n// (`{\"message_type:\"summary\",...`) in the restic backup command output. Due to\n// an issue in Restic, this might not always be the last line\n// (https://github.com/restic/restic/issues/2389). It returns an error if it\n// can't be found.\nfunc getSummaryLine(b []byte) ([]byte, error) {\n\tsummaryLineIdx := bytes.LastIndex(b, []byte(`{\"message_type\":\"summary\"`))\n\tif summaryLineIdx < 0 {\n\t\treturn nil, errors.New(\"unable to find summary in restic backup command output\")\n\t}\n\t// find the end of the summary line\n\tnewLineIdx := bytes.Index(b[summaryLineIdx:], []byte(\"\\n\"))\n\tif newLineIdx < 0 {\n\t\treturn nil, errors.New(\"unable to get summary line from restic backup command output\")\n\t}\n\treturn b[summaryLineIdx : summaryLineIdx+newLineIdx], nil\n}\n\n// RunRestore runs a `restic restore` command and monitors the volume size to\n// provide progress updates to the caller.\nfunc RunRestore(restoreCmd *Command, log logrus.FieldLogger, updater uploader.ProgressUpdater) (string, string, error) {\n\tinsecureTLSFlag := \"\"\n\n\tfor _, extraFlag := range restoreCmd.ExtraFlags {\n\t\tif strings.Contains(extraFlag, resticInsecureTLSFlag) {\n\t\t\tinsecureTLSFlag = extraFlag\n\t\t}\n\t}\n\n\tsnapshotSize, err := getSnapshotSize(restoreCmd.RepoIdentifier, restoreCmd.PasswordFile, restoreCmd.CACertFile, restoreCmd.Args[0], restoreCmd.Env, insecureTLSFlag)\n\tif err != nil {\n\t\treturn \"\", \"\", errors.Wrap(err, \"error getting snapshot size\")\n\t}\n\n\tupdater.UpdateProgress(&uploader.Progress{\n\t\tTotalBytes: snapshotSize,\n\t})\n\n\t// create a channel to signal when to end the goroutine scanning for progress\n\t// updates\n\tquit := make(chan struct{})\n\n\tgo func() {\n\t\tticker := time.NewTicker(restoreProgressCheckInterval)\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-ticker.C:\n\t\t\t\tvolumeSize, err := getVolumeSize(restoreCmd.Dir)\n\t\t\t\tif err != nil {\n\t\t\t\t\tlog.WithError(err).Errorf(\"error getting restic restore progress\")\n\t\t\t\t}\n\n\t\t\t\tif volumeSize != 0 {\n\t\t\t\t\tupdater.UpdateProgress(&uploader.Progress{\n\t\t\t\t\t\tTotalBytes: snapshotSize,\n\t\t\t\t\t\tBytesDone: volumeSize,\n\t\t\t\t\t})\n\t\t\t\t}\n\t\t\tcase <-quit:\n\t\t\t\tticker.Stop()\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}()\n\n\tstdout, stderr, err := exec.RunCommandWithLog(restoreCmd.Cmd(), log)\n\tquit <- struct{}{}\n\n\t// update progress to 100%\n\tupdater.UpdateProgress(&uploader.Progress{\n\t\tTotalBytes: snapshotSize,\n\t\tBytesDone: snapshotSize,\n\t})\n\n\treturn stdout, stderr, err\n}\n\nfunc getSnapshotSize(repoIdentifier, passwordFile, caCertFile, snapshotID string, env []string, insecureTLS string) (int64, error) {\n\tcmd := StatsCommand(repoIdentifier, passwordFile, snapshotID)\n\tcmd.Env = env\n\tcmd.CACertFile = caCertFile\n\n\tif len(insecureTLS) > 0 {\n\t\tcmd.ExtraFlags = append(cmd.ExtraFlags, insecureTLS)\n\t}\n\n\tstdout, stderr, err := exec.RunCommand(cmd.Cmd())\n\tif err != nil {\n\t\treturn 0, errors.Wrapf(err, \"error running command, stderr=%s\", stderr)\n\t}\n\n\tvar snapshotStats struct {\n\t\tTotalSize int64 `json:\"total_size\"`\n\t}\n\n\tif err := json.Unmarshal([]byte(stdout), &snapshotStats); err != nil {\n\t\treturn 0, errors.Wrapf(err, \"error unmarshaling restic stats result, stdout=%s\", stdout)\n\t}\n\n\treturn snapshotStats.TotalSize, nil\n}\n\nfunc getVolumeSize(path string) (int64, error) {\n\tvar size int64\n\n\tfiles, err := fileSystem.ReadDir(path)\n\tif err != nil {\n\t\treturn 0, errors.Wrapf(err, \"error reading directory %s\", path)\n\t}\n\n\tfor _, file := range files {\n\t\tif file.IsDir() {\n\t\t\ts, err := getVolumeSize(fmt.Sprintf(\"%s/%s\", path, file.Name()))\n\t\t\tif err != nil {\n\t\t\t\treturn 0, err\n\t\t\t}\n\t\t\tsize += s\n\t\t} else {\n\t\t\tsize += file.Size()\n\t\t}\n\t}\n\n\treturn size, nil\n}\n" }, { "path": "pkg/restic/exec_commands_test.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restic\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n)\n\nfunc Test_getSummaryLine(t *testing.T) {\n\tsummaryLine := `{\"message_type\":\"summary\",\"files_new\":0,\"files_changed\":0,\"files_unmodified\":3,\"dirs_new\":0,\"dirs_changed\":0,\"dirs_unmodified\":0,\"data_blobs\":0,\"tree_blobs\":0,\"data_added\":0,\"total_files_processed\":3,\"total_bytes_processed\":13238272000,\"total_duration\":0.319265105,\"snapshot_id\":\"38515bb5\"}`\n\ttests := []struct {\n\t\tname string\n\t\toutput string\n\t\twantErr bool\n\t}{\n\t\t{\"no summary\", `{\"message_type\":\"status\",\"percent_done\":0,\"total_files\":1,\"total_bytes\":10485760000}\n{\"message_type\":\"status\",\"percent_done\":0,\"total_files\":3,\"files_done\":1,\"total_bytes\":13238272000}\n`, true},\n\t\t{\"no newline after summary\", `{\"message_type\":\"status\",\"percent_done\":0,\"total_files\":1,\"total_bytes\":10485760000}\n{\"message_type\":\"status\",\"percent_done\":0,\"total_files\":3,\"files_done\":1,\"total_bytes\":13238272000}\n{\"message_type\":\"summary\",\"files_new\":0,\"files_changed\":0,\"files_unmodified\":3,\"dirs_new\":0`, true},\n\t\t{\"summary at end\", `{\"message_type\":\"status\",\"percent_done\":0,\"total_files\":1,\"total_bytes\":10485760000}\n{\"message_type\":\"status\",\"percent_done\":0,\"total_files\":3,\"files_done\":1,\"total_bytes\":13238272000}\n{\"message_type\":\"status\",\"percent_done\":1,\"total_files\":3,\"files_done\":3,\"total_bytes\":13238272000,\"bytes_done\":13238272000}\n{\"message_type\":\"summary\",\"files_new\":0,\"files_changed\":0,\"files_unmodified\":3,\"dirs_new\":0,\"dirs_changed\":0,\"dirs_unmodified\":0,\"data_blobs\":0,\"tree_blobs\":0,\"data_added\":0,\"total_files_processed\":3,\"total_bytes_processed\":13238272000,\"total_duration\":0.319265105,\"snapshot_id\":\"38515bb5\"}\n`, false},\n\t\t{\"summary before status\", `{\"message_type\":\"status\",\"percent_done\":0,\"total_files\":1,\"total_bytes\":10485760000}\n{\"message_type\":\"status\",\"percent_done\":0,\"total_files\":3,\"files_done\":1,\"total_bytes\":13238272000}\n{\"message_type\":\"summary\",\"files_new\":0,\"files_changed\":0,\"files_unmodified\":3,\"dirs_new\":0,\"dirs_changed\":0,\"dirs_unmodified\":0,\"data_blobs\":0,\"tree_blobs\":0,\"data_added\":0,\"total_files_processed\":3,\"total_bytes_processed\":13238272000,\"total_duration\":0.319265105,\"snapshot_id\":\"38515bb5\"}\n{\"message_type\":\"status\",\"percent_done\":1,\"total_files\":3,\"files_done\":3,\"total_bytes\":13238272000,\"bytes_done\":13238272000}\n`, false},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tsummary, err := getSummaryLine([]byte(tt.output))\n\t\t\tif tt.wantErr {\n\t\t\t\tassert.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, summaryLine, string(summary))\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc Test_getLastLine(t *testing.T) {\n\ttests := []struct {\n\t\toutput []byte\n\t\twant string\n\t}{\n\t\t{[]byte(`last line\n`), \"last line\"},\n\t\t{[]byte(`first line\nsecond line\nthird line\n`), \"third line\"},\n\t\t{[]byte(\"\"), \"\"},\n\t\t{nil, \"\"},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.want, func(t *testing.T) {\n\t\t\tassert.Equal(t, []byte(tt.want), getLastLine(tt.output))\n\t\t})\n\t}\n}\n\nfunc Test_getVolumeSize(t *testing.T) {\n\tfiles := map[string][]byte{\n\t\t\"/file1.txt\": []byte(\"file1\"),\n\t\t\"/file2.txt\": []byte(\"file2\"),\n\t\t\"/file3.txt\": []byte(\"file3\"),\n\t\t\"/files/file4.txt\": []byte(\"file4\"),\n\t\t\"/files/nested/file5.txt\": []byte(\"file5\"),\n\t}\n\tfakefs := test.NewFakeFileSystem()\n\n\tvar expectedSize int64\n\tfor path, content := range files {\n\t\tfakefs.WithFile(path, content)\n\t\texpectedSize += int64(len(content))\n\t}\n\n\tfileSystem = fakefs\n\tdefer func() { fileSystem = filesystem.NewFileSystem() }()\n\n\tactualSize, err := getVolumeSize(\"/\")\n\n\trequire.NoError(t, err)\n\tassert.Equal(t, expectedSize, actualSize)\n}\n" }, { "path": "pkg/restore/actions/add_pvc_from_pod_action.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\thttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\ntype AddPVCFromPodAction struct {\n\tlogger logrus.FieldLogger\n}\n\nfunc NewAddPVCFromPodAction(logger logrus.FieldLogger) *AddPVCFromPodAction {\n\treturn &AddPVCFromPodAction{logger: logger}\n}\n\nfunc (a *AddPVCFromPodAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"pods\"},\n\t}, nil\n}\n\nfunc (a *AddPVCFromPodAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\ta.logger.Info(\"Executing AddPVCFromPodAction\")\n\n\tvar pod corev1api.Pod\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(input.Item.UnstructuredContent(), &pod); err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to convert unstructured item to pod\")\n\t}\n\n\tvar additionalItems []velero.ResourceIdentifier\n\n\tfor _, volume := range pod.Spec.Volumes {\n\t\tif volume.PersistentVolumeClaim == nil {\n\t\t\tcontinue\n\t\t}\n\n\t\ta.logger.Infof(\"Adding PVC %s/%s as an additional item to restore\", pod.Namespace, volume.PersistentVolumeClaim.ClaimName)\n\t\tadditionalItems = append(additionalItems, velero.ResourceIdentifier{\n\t\t\tGroupResource: kuberesource.PersistentVolumeClaims,\n\t\t\tNamespace: pod.Namespace,\n\t\t\tName: volume.PersistentVolumeClaim.ClaimName,\n\t\t})\n\t}\n\n\treturn &velero.RestoreItemActionExecuteOutput{\n\t\tUpdatedItem: input.Item,\n\t\tAdditionalItems: additionalItems,\n\t}, nil\n}\n" }, { "path": "pkg/restore/actions/add_pvc_from_pod_action_test.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\t http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestAddPVCFromPodActionExecute(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\titem *corev1api.Pod\n\t\twant []velero.ResourceIdentifier\n\t}{\n\t\t{\n\t\t\tname: \"pod with no volumes returns no additional items\",\n\t\t\titem: &corev1api.Pod{},\n\t\t\twant: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"pod with some PVCs returns them as additional items\",\n\t\t\titem: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\tName: \"foo\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\tEmptyDir: new(corev1api.EmptyDirVolumeSource),\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\t\tClaimName: \"pvc-1\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\t\tClaimName: \"pvc-2\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\tHostPath: new(corev1api.HostPathVolumeSource),\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\t\tClaimName: \"pvc-3\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []velero.ResourceIdentifier{\n\t\t\t\t{GroupResource: kuberesource.PersistentVolumeClaims, Namespace: \"ns-1\", Name: \"pvc-1\"},\n\t\t\t\t{GroupResource: kuberesource.PersistentVolumeClaims, Namespace: \"ns-1\", Name: \"pvc-2\"},\n\t\t\t\t{GroupResource: kuberesource.PersistentVolumeClaims, Namespace: \"ns-1\", Name: \"pvc-3\"},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\titemData, err := runtime.DefaultUnstructuredConverter.ToUnstructured(test.item)\n\t\t\trequire.NoError(t, err)\n\n\t\t\taction := &AddPVCFromPodAction{logger: velerotest.NewLogger()}\n\n\t\t\tinput := &velero.RestoreItemActionExecuteInput{\n\t\t\t\tItem: &unstructured.Unstructured{Object: itemData},\n\t\t\t}\n\n\t\t\tres, err := action.Execute(input)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.Equal(t, test.want, res.AdditionalItems)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/restore/actions/admissionwebhook_config_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// AdmissionWebhookConfigurationAction is a RestoreItemAction plugin applicable to mutatingwebhookconfiguration and\n// validatingwebhookconfiguration to reset the invalid value for \"sideEffects\" of the webhooks.\n// More background please refer to https://github.com/vmware-tanzu/velero/issues/3516\ntype AdmissionWebhookConfigurationAction struct {\n\tlogger logrus.FieldLogger\n}\n\n// NewAdmissionWebhookConfigurationAction creates a new instance of AdmissionWebhookConfigurationAction\nfunc NewAdmissionWebhookConfigurationAction(logger logrus.FieldLogger) *AdmissionWebhookConfigurationAction {\n\treturn &AdmissionWebhookConfigurationAction{logger: logger}\n}\n\n// AppliesTo implements the RestoreItemAction plugin interface method.\nfunc (a *AdmissionWebhookConfigurationAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"mutatingwebhookconfigurations\", \"validatingwebhookconfigurations\"},\n\t}, nil\n}\n\n// Execute will reset the value of \"sideEffects\" attribute of each item in the \"webhooks\" list to \"None\" if they are invalid values for\n// v1, such as \"Unknown\" or \"Some\"\nfunc (a *AdmissionWebhookConfigurationAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\ta.logger.Info(\"Executing ChangeStorageClassAction\")\n\tdefer a.logger.Info(\"Done executing ChangeStorageClassAction\")\n\n\titem := input.Item\n\tapiVersion, _, err := unstructured.NestedString(item.UnstructuredContent(), \"apiVersion\")\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to get the apiVersion from input item\")\n\t}\n\tname, _, _ := unstructured.NestedString(item.UnstructuredContent(), \"metadata\", \"name\")\n\tlogger := a.logger.WithField(\"resource_name\", name)\n\tif apiVersion != \"admissionregistration.k8s.io/v1\" {\n\t\tlogger.Infof(\"unable to handle api version: %s, skip\", apiVersion)\n\t\treturn velero.NewRestoreItemActionExecuteOutput(input.Item), nil\n\t}\n\twebhooks, ok, err := unstructured.NestedSlice(item.UnstructuredContent(), \"webhooks\")\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to get webhooks slice from input item\")\n\t}\n\tif !ok {\n\t\tlogger.Info(\"webhooks is not set, skip\")\n\t\treturn velero.NewRestoreItemActionExecuteOutput(input.Item), nil\n\t}\n\tnewWebhooks := make([]any, 0)\n\tfor i := range webhooks {\n\t\tlogger2 := logger.WithField(\"index\", i)\n\t\tobj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&webhooks[i])\n\t\tif err != nil {\n\t\t\tlogger2.Errorf(\"failed to convert the webhook entry, error: %v, it will be dropped\", err)\n\t\t\tcontinue\n\t\t}\n\t\ts, _, _ := unstructured.NestedString(obj, \"sideEffects\")\n\t\tif s != \"None\" && s != \"NoneOnDryRun\" {\n\t\t\tlogger2.Infof(\"reset the invalid sideEffects value '%s' to 'None'\", s)\n\t\t\tobj[\"sideEffects\"] = \"None\"\n\t\t}\n\t\tnewWebhooks = append(newWebhooks, obj)\n\t}\n\titem.UnstructuredContent()[\"webhooks\"] = newWebhooks\n\treturn velero.NewRestoreItemActionExecuteOutput(item), nil\n}\n" }, { "path": "pkg/restore/actions/admissionwebhook_config_action_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"encoding/json\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestNewAdmissionWebhookConfigurationActionExecute(t *testing.T) {\n\taction := NewAdmissionWebhookConfigurationAction(velerotest.NewLogger())\n\tcases := []struct {\n\t\tname string\n\t\titemJSON string\n\t\twantErr bool\n\t\tNoneSideEffectsIndex []int // the indexes with sideEffects that arereset to None\n\t\tNotNoneSideEffectsIndex []int // the indexes with sideEffects that are not reset to None\n\t}{\n\t\t{\n\t\t\tname: \"v1 mutatingwebhookconfiguration with sideEffects as Unknown\",\n\t\t\titemJSON: `{\n\t\t\t\t \"apiVersion\": \"admissionregistration.k8s.io/v1\",\n\t\t\t\t \"kind\": \"MutatingWebhookConfiguration\",\n\t\t\t\t \"metadata\": {\n\t\t\t\t\t \"name\": \"my-test-mutating\"\n\t\t\t\t },\n\t\t\t\t \"webhooks\": [\n\t\t\t\t\t {\n\t\t\t\t\t\t \"clientConfig\": {\n\t\t\t\t\t\t\t \"url\": \"https://mytest.org\"\n\t\t\t\t\t\t },\n\t\t\t\t\t\t \"rules\": [\n\t\t\t\t\t\t\t {\n\t\t\t\t\t\t\t\t\t\"apiGroups\": [\n\t\t\t\t\t\t\t\t\t\t\"\"\n\t\t\t\t\t\t\t\t\t],\n\t\t\t\t\t\t\t\t\t\"apiVersions\": [\n\t\t\t\t\t\t\t\t\t\t\"v1\"\n\t\t\t\t\t\t\t\t\t],\n\t\t\t\t\t\t\t\t\t\"operations\": [\n\t\t\t\t\t\t\t\t\t\t\"CREATE\"\n\t\t\t\t\t\t\t\t\t],\n\t\t\t\t\t\t\t\t\t\"resources\": [\n\t\t\t\t\t\t\t\t\t\t\"pods\"\n\t\t\t\t\t\t\t\t\t],\n\t\t\t\t\t\t\t\t\t\"scope\": \"Namespaced\"\n\t\t\t\t\t\t\t }\n\t\t\t\t\t\t ],\n\t\t\t\t\t\t \"sideEffects\": \"Unknown\"\n\t\t\t\t\t }\n\t\t\t\t ]\n\t\t\t}`,\n\t\t\twantErr: false,\n\t\t\tNoneSideEffectsIndex: []int{0},\n\t\t},\n\t\t{\n\t\t\tname: \"v1 validatingwebhookconfiguration with sideEffects as Some\",\n\t\t\titemJSON: `{\n\t\t\t\t \"apiVersion\": \"admissionregistration.k8s.io/v1\",\n\t\t\t\t \"kind\": \"ValidatingWebhookConfiguration\",\n\t\t\t\t \"metadata\": {\n\t\t\t\t\t \"name\": \"my-test-validating\"\n\t\t\t\t },\n\t\t\t\t \"webhooks\": [\n\t\t\t\t\t {\n\t\t\t\t\t\t \"clientConfig\": {\n\t\t\t\t\t\t\t \"url\": \"https://mytest.org\"\n\t\t\t\t\t\t },\n\t\t\t\t\t\t \"rules\": [\n\t\t\t\t\t\t\t {\n\t\t\t\t\t\t\t\t\t\"apiGroups\": [\n\t\t\t\t\t\t\t\t\t\t\"\"\n\t\t\t\t\t\t\t\t\t],\n\t\t\t\t\t\t\t\t\t\"apiVersions\": [\n\t\t\t\t\t\t\t\t\t\t\"v1\"\n\t\t\t\t\t\t\t\t\t],\n\t\t\t\t\t\t\t\t\t\"operations\": [\n\t\t\t\t\t\t\t\t\t\t\"CREATE\"\n\t\t\t\t\t\t\t\t\t],\n\t\t\t\t\t\t\t\t\t\"resources\": [\n\t\t\t\t\t\t\t\t\t\t\"pods\"\n\t\t\t\t\t\t\t\t\t],\n\t\t\t\t\t\t\t\t\t\"scope\": \"Namespaced\"\n\t\t\t\t\t\t\t }\n\t\t\t\t\t\t ],\n\t\t\t\t\t\t \"sideEffects\": \"Some\"\n\t\t\t\t\t }\n\t\t\t\t ]\n\t\t\t}`,\n\t\t\twantErr: false,\n\t\t\tNoneSideEffectsIndex: []int{0},\n\t\t},\n\t\t{\n\t\t\tname: \"v1beta1 validatingwebhookconfiguration with sideEffects as Some, nothing should change\",\n\t\t\titemJSON: `{\n\t\t\t\t \"apiVersion\": \"admissionregistration.k8s.io/v1beta1\",\n\t\t\t\t \"kind\": \"ValidatingWebhookConfiguration\",\n\t\t\t\t \"metadata\": {\n\t\t\t\t\t \"name\": \"my-test-validating\"\n\t\t\t\t },\n\t\t\t\t \"webhooks\": [\n\t\t\t\t\t {\n\t\t\t\t\t\t \"clientConfig\": {\n\t\t\t\t\t\t\t \"url\": \"https://mytest.org\"\n\t\t\t\t\t\t },\n\t\t\t\t\t\t \"rules\": [\n\t\t\t\t\t\t\t {\n\t\t\t\t\t\t\t\t\t\"apiGroups\": [\n\t\t\t\t\t\t\t\t\t\t\"\"\n\t\t\t\t\t\t\t\t\t],\n\t\t\t\t\t\t\t\t\t\"apiVersions\": [\n\t\t\t\t\t\t\t\t\t\t\"v1\"\n\t\t\t\t\t\t\t\t\t],\n\t\t\t\t\t\t\t\t\t\"operations\": [\n\t\t\t\t\t\t\t\t\t\t\"CREATE\"\n\t\t\t\t\t\t\t\t\t],\n\t\t\t\t\t\t\t\t\t\"resources\": [\n\t\t\t\t\t\t\t\t\t\t\"pods\"\n\t\t\t\t\t\t\t\t\t],\n\t\t\t\t\t\t\t\t\t\"scope\": \"Namespaced\"\n\t\t\t\t\t\t\t }\n\t\t\t\t\t\t ],\n\t\t\t\t\t\t \"sideEffects\": \"Some\"\n\t\t\t\t\t }\n\t\t\t\t ]\n\t\t\t}`,\n\t\t\twantErr: false,\n\t\t\tNotNoneSideEffectsIndex: []int{0},\n\t\t},\n\t\t{\n\t\t\tname: \"v1 validatingwebhookconfiguration with multiple invalid sideEffects\",\n\t\t\titemJSON: `{\n\t\t\t\t \"apiVersion\": \"admissionregistration.k8s.io/v1\",\n\t\t\t\t \"kind\": \"ValidatingWebhookConfiguration\",\n\t\t\t\t \"metadata\": {\n\t\t\t\t\t \"name\": \"my-test-validating\"\n\t\t\t\t },\n\t\t\t\t \"webhooks\": [\n\t\t\t\t\t {\n\t\t\t\t\t\t \"clientConfig\": {\n\t\t\t\t\t\t\t \"url\": \"https://mytest.org\"\n\t\t\t\t\t\t },\n\t\t\t\t\t\t \"sideEffects\": \"Some\"\n\t\t\t\t\t },\n\t\t\t\t\t {\n\t\t\t\t\t\t \"clientConfig\": {\n\t\t\t\t\t\t\t \"url\": \"https://mytest2.org\"\n\t\t\t\t\t\t },\n\t\t\t\t\t\t \"sideEffects\": \"Some\"\n\t\t\t\t\t }\n\t\t\t\t ]\n\t\t\t}`,\n\t\t\twantErr: false,\n\t\t\tNoneSideEffectsIndex: []int{0, 1},\n\t\t},\n\t}\n\tfor _, tt := range cases {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\to := map[string]any{}\n\t\t\tjson.Unmarshal([]byte(tt.itemJSON), &o)\n\t\t\tinput := &velero.RestoreItemActionExecuteInput{\n\t\t\t\tItem: &unstructured.Unstructured{\n\t\t\t\t\tObject: o,\n\t\t\t\t},\n\t\t\t}\n\t\t\toutput, err := action.Execute(input)\n\t\t\tif tt.wantErr {\n\t\t\t\trequire.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\t\t\tif tt.NoneSideEffectsIndex != nil {\n\t\t\t\twb, _, err := unstructured.NestedSlice(output.UpdatedItem.UnstructuredContent(), \"webhooks\")\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tfor _, i := range tt.NoneSideEffectsIndex {\n\t\t\t\t\tit, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&wb[i])\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\ts := it[\"sideEffects\"].(string)\n\t\t\t\t\tassert.Equal(t, \"None\", s)\n\t\t\t\t}\n\t\t\t}\n\t\t\tif tt.NotNoneSideEffectsIndex != nil {\n\t\t\t\twb, _, err := unstructured.NestedSlice(output.UpdatedItem.UnstructuredContent(), \"webhooks\")\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tfor _, i := range tt.NotNoneSideEffectsIndex {\n\t\t\t\t\tit, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&wb[i])\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t\ts := it[\"sideEffects\"].(string)\n\t\t\t\t\tassert.NotEqual(t, \"None\", s)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/restore/actions/apiservice_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/kube-aggregator/pkg/controllers/autoregister\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\ntype APIServiceAction struct {\n\tlogger logrus.FieldLogger\n}\n\n// NewAPIServiceAction returns an APIServiceAction which is a RestoreItemAction plugin\n// that will skip the restore of any APIServices which are managed by Kubernetes. This\n// is determined by looking for the \"kube-aggregator.kubernetes.io/automanaged\" label on\n// the APIService.\nfunc NewAPIServiceAction(logger logrus.FieldLogger) *APIServiceAction {\n\treturn &APIServiceAction{logger: logger}\n}\n\nfunc (a *APIServiceAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"apiservices\"},\n\t\tLabelSelector: autoregister.AutoRegisterManagedLabel,\n\t}, nil\n}\n\nfunc (a *APIServiceAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\ta.logger.Info(\"Executing APIServiceAction\")\n\tdefer a.logger.Info(\"Done executing APIServiceAction\")\n\n\ta.logger.Infof(\"Skipping restore of APIService as it is managed by Kubernetes\")\n\treturn velero.NewRestoreItemActionExecuteOutput(input.Item).WithoutRestore(), nil\n}\n" }, { "path": "pkg/restore/actions/apiservice_action_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\tapiregistrationv1 \"k8s.io/kube-aggregator/pkg/apis/apiregistration/v1\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestAPIServiceActionExecuteSkipsRestore(t *testing.T) {\n\tobj := apiregistrationv1.APIService{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"v1.test.velero.io\",\n\t\t},\n\t}\n\n\tunstructuredAPIService, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&obj)\n\trequire.NoError(t, err)\n\n\taction := NewAPIServiceAction(velerotest.NewLogger())\n\tres, err := action.Execute(&velero.RestoreItemActionExecuteInput{\n\t\tItem: &unstructured.Unstructured{Object: unstructuredAPIService},\n\t\tItemFromBackup: &unstructured.Unstructured{Object: unstructuredAPIService},\n\t})\n\trequire.NoError(t, err)\n\n\tvar apiService apiregistrationv1.APIService\n\trequire.NoError(t, runtime.DefaultUnstructuredConverter.FromUnstructured(res.UpdatedItem.UnstructuredContent(), &apiService))\n\trequire.Equal(t, obj, apiService)\n\trequire.True(t, res.SkipRestore)\n}\n" }, { "path": "pkg/restore/actions/change_image_name_action.go", "content": "/*\nCopyright 2022 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strings\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\tcorev1client \"k8s.io/client-go/kubernetes/typed/core/v1\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\nconst (\n\tdelimiterValue = \",\"\n)\n\n// ChangeImageNameAction updates a deployment or Pod's image name\n// if a mapping is found in the plugin's config map.\ntype ChangeImageNameAction struct {\n\tlogger logrus.FieldLogger\n\tconfigMapClient corev1client.ConfigMapInterface\n}\n\n// NewChangeImageNameAction is the constructor for ChangeImageNameAction.\nfunc NewChangeImageNameAction(\n\tlogger logrus.FieldLogger,\n\tconfigMapClient corev1client.ConfigMapInterface,\n) *ChangeImageNameAction {\n\treturn &ChangeImageNameAction{\n\t\tlogger: logger,\n\t\tconfigMapClient: configMapClient,\n\t}\n}\n\n// AppliesTo returns the resources that ChangeImageNameAction should\n// be run for.\nfunc (a *ChangeImageNameAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"deployments\", \"statefulsets\", \"daemonsets\", \"replicasets\", \"replicationcontrollers\", \"jobs\", \"cronjobs\", \"pods\"},\n\t}, nil\n}\n\n// Execute updates the item's spec.containers' image if a mapping is found\n// in the config map for the plugin.\nfunc (a *ChangeImageNameAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\ta.logger.Info(\"Executing ChangeImageNameAction\")\n\tdefer a.logger.Info(\"Done executing ChangeImageNameAction\")\n\n\topts := metav1.ListOptions{\n\t\tLabelSelector: fmt.Sprintf(\"velero.io/plugin-config,%s=%s\", \"velero.io/change-image-name\", common.PluginKindRestoreItemAction),\n\t}\n\n\tlist, err := a.configMapClient.List(context.TODO(), opts)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tif len(list.Items) == 0 {\n\t\treturn &velero.RestoreItemActionExecuteOutput{\n\t\t\tUpdatedItem: input.Item,\n\t\t}, nil\n\t}\n\n\tif len(list.Items) > 1 {\n\t\tvar items []string\n\t\tfor _, item := range list.Items {\n\t\t\titems = append(items, item.Name)\n\t\t}\n\t\treturn nil, errors.Errorf(\"found more than one ConfigMap matching label selector %q: %v\", opts.LabelSelector, items)\n\t}\n\n\tconfig := &list.Items[0]\n\tif len(config.Data) == 0 {\n\t\ta.logger.Info(\"No image name mappings found\")\n\t\treturn velero.NewRestoreItemActionExecuteOutput(input.Item), nil\n\t}\n\n\tobj, ok := input.Item.(*unstructured.Unstructured)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"object was of unexpected type %T\", input.Item)\n\t}\n\tif obj.GetKind() == \"Pod\" {\n\t\terr = a.replaceImageName(obj, config, \"spec\", \"containers\")\n\t\tif err != nil {\n\t\t\ta.logger.Infof(\"replace image name meet error: %v\", err)\n\t\t\treturn nil, errors.Wrap(err, \"error getting item's spec.containers\")\n\t\t}\n\n\t\terr = a.replaceImageName(obj, config, \"spec\", \"initContainers\")\n\t\tif err != nil {\n\t\t\ta.logger.Infof(\"replace image name meet error: %v\", err)\n\t\t\treturn nil, errors.Wrap(err, \"error getting item's spec.containers\")\n\t\t}\n\t} else if obj.GetKind() == \"CronJob\" {\n\t\t//handle containers\n\t\terr = a.replaceImageName(obj, config, \"spec\", \"jobTemplate\", \"spec\", \"template\", \"spec\", \"containers\")\n\t\tif err != nil {\n\t\t\ta.logger.Infof(\"replace image name meet error: %v\", err)\n\t\t\treturn nil, errors.Wrap(err, \"error getting item's spec.containers\")\n\t\t}\n\t\t//handle initContainers\n\t\terr = a.replaceImageName(obj, config, \"spec\", \"jobTemplate\", \"spec\", \"template\", \"spec\", \"initContainers\")\n\t\tif err != nil {\n\t\t\ta.logger.Infof(\"replace image name meet error: %v\", err)\n\t\t\treturn nil, errors.Wrap(err, \"error getting item's spec.containers\")\n\t\t}\n\t} else {\n\t\t//handle containers\n\t\terr = a.replaceImageName(obj, config, \"spec\", \"template\", \"spec\", \"containers\")\n\t\tif err != nil {\n\t\t\ta.logger.Infof(\"replace image name meet error: %v\", err)\n\t\t\treturn nil, errors.Wrap(err, \"error getting item's spec.containers\")\n\t\t}\n\n\t\t//handle initContainers\n\t\terr = a.replaceImageName(obj, config, \"spec\", \"template\", \"spec\", \"initContainers\")\n\t\tif err != nil {\n\t\t\ta.logger.Infof(\"replace image name meet error: %v\", err)\n\t\t\treturn nil, errors.Wrap(err, \"error getting item's spec.containers\")\n\t\t}\n\t}\n\treturn velero.NewRestoreItemActionExecuteOutput(obj), nil\n}\n\nfunc (a *ChangeImageNameAction) replaceImageName(obj *unstructured.Unstructured, config *corev1api.ConfigMap, filed ...string) error {\n\tlog := a.logger.WithFields(map[string]any{\n\t\t\"kind\": obj.GetKind(),\n\t\t\"namespace\": obj.GetNamespace(),\n\t\t\"name\": obj.GetName(),\n\t})\n\tneedUpdateObj := false\n\tcontainers, _, err := unstructured.NestedSlice(obj.UnstructuredContent(), filed...)\n\tif err != nil {\n\t\tlog.Infof(\"UnstructuredConverter meet error: %v\", err)\n\t\treturn errors.Wrap(err, \"error getting item's spec.containers\")\n\t}\n\tif len(containers) == 0 {\n\t\treturn nil\n\t}\n\tfor i, container := range containers {\n\t\tlog.Infoln(\"container:\", container)\n\t\tif image, ok := container.(map[string]any)[\"image\"]; ok {\n\t\t\timageName := image.(string)\n\t\t\tif exists, newImageName, err := a.isImageReplaceRuleExist(log, imageName, config); exists && err == nil {\n\t\t\t\tneedUpdateObj = true\n\t\t\t\tlog.Infof(\"Updating item's image from %s to %s\", imageName, newImageName)\n\t\t\t\tcontainer.(map[string]any)[\"image\"] = newImageName\n\t\t\t\tcontainers[i] = container\n\t\t\t}\n\t\t}\n\t}\n\tif needUpdateObj {\n\t\tif err := unstructured.SetNestedField(obj.UnstructuredContent(), containers, filed...); err != nil {\n\t\t\treturn errors.Wrap(err, \"unable to set item's initContainer image\")\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc (a *ChangeImageNameAction) isImageReplaceRuleExist(log *logrus.Entry, oldImageName string, cm *corev1api.ConfigMap) (exists bool, newImageName string, err error) {\n\tif oldImageName == \"\" {\n\t\tlog.Infoln(\"Item has no old image name specified\")\n\t\treturn false, \"\", nil\n\t}\n\tlog.Debug(\"oldImageName: \", oldImageName)\n\n\t//how to use: \"\"\n\t//for current implementation the value can only be \",\"\n\t//e.x: in case your old image name is 1.1.1.1:5000/abc:test\n\t//\"case1\":\"1.1.1.1:5000,2.2.2.2:3000\"\n\t//\"case2\":\"5000,3000\"\n\t//\"case3\":\"abc:test,edf:test\"\n\t//\"case4\":\"1.1.1.1:5000/abc:test,2.2.2.2:3000/edf:test\"\n\tfor _, row := range cm.Data {\n\t\tif !strings.Contains(row, delimiterValue) {\n\t\t\tcontinue\n\t\t}\n\t\tif strings.Contains(oldImageName, strings.TrimSpace(row[0:strings.Index(row, delimiterValue)])) && len(row[strings.Index(row, delimiterValue):]) > len(delimiterValue) {\n\t\t\tlog.Infoln(\"match specific case:\", row)\n\t\t\toldImagePart := strings.TrimSpace(row[0:strings.Index(row, delimiterValue)])\n\t\t\tnewImagePart := strings.TrimSpace(row[strings.Index(row, delimiterValue)+len(delimiterValue):])\n\t\t\tnewImageName = strings.Replace(oldImageName, oldImagePart, newImagePart, -1)\n\t\t\treturn true, newImageName, nil\n\t\t}\n\t}\n\treturn false, \"\", errors.Errorf(\"No mapping rule found for image: %s\", oldImageName)\n}\n" }, { "path": "pkg/restore/actions/change_image_name_action_test.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes/fake\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// TestChangeImageRepositoryActionExecute runs the ChangeImageNameAction's Execute\n// method and validates that the item's image name is modified (or not) as expected.\n// Validation is done by comparing the result of the Execute method to the test case's\n// desired result.\nfunc TestChangeImageRepositoryActionExecute(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpodOrObj any\n\t\tconfigMap *corev1api.ConfigMap\n\t\tfreshedImageName string\n\t\timageNameSlice []string\n\t\twant any\n\t\twantErr error\n\t}{\n\t\t{\n\t\t\tname: \"a valid mapping with spaces for a new image repository is applied correctly\",\n\t\t\tpodOrObj: builder.ForPod(\"default\", \"pod1\").ObjectMeta().\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container1\",\n\t\t\t\t\tImage: \"1.1.1.1:5000/abc:test\",\n\t\t\t\t}).Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-image-name\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-image-name\", \"RestoreItemAction\")).\n\t\t\t\tData(\"case1\", \"1.1.1.1:5000 , 2.2.2.2:3000\").\n\t\t\t\tResult(),\n\t\t\tfreshedImageName: \"2.2.2.2:3000/abc:test\",\n\t\t\twant: \"2.2.2.2:3000/abc:test\",\n\t\t},\n\n\t\t{\n\t\t\tname: \"a valid mapping for a new image repository is applied correctly\",\n\t\t\tpodOrObj: builder.ForPod(\"default\", \"pod1\").ObjectMeta().\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container2\",\n\t\t\t\t\tImage: \"1.1.1.1:5000/abc:test\",\n\t\t\t\t}).Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-image-name\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-image-name\", \"RestoreItemAction\")).\n\t\t\t\tData(\"specific\", \"1.1.1.1:5000,2.2.2.2:3000\").\n\t\t\t\tResult(),\n\t\t\tfreshedImageName: \"2.2.2.2:3000/abc:test\",\n\t\t\twant: \"2.2.2.2:3000/abc:test\",\n\t\t},\n\n\t\t{\n\t\t\tname: \"a valid mapping for a new image name is applied correctly\",\n\t\t\tpodOrObj: builder.ForPod(\"default\", \"pod1\").ObjectMeta().\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container3\",\n\t\t\t\t\tImage: \"1.1.1.1:5000/abc:test\",\n\t\t\t\t}).Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-image-name\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-image-name\", \"RestoreItemAction\")).\n\t\t\t\tData(\"specific\", \"abc:test,myproject:latest\").\n\t\t\t\tResult(),\n\t\t\tfreshedImageName: \"1.1.1.1:5000/myproject:latest\",\n\t\t\twant: \"1.1.1.1:5000/myproject:latest\",\n\t\t},\n\n\t\t{\n\t\t\tname: \"a valid mapping for a new image repository port is applied correctly\",\n\t\t\tpodOrObj: builder.ForPod(\"default\", \"pod1\").ObjectMeta().\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container4\",\n\t\t\t\t\tImage: \"1.1.1.1:5000/abc:test\",\n\t\t\t\t}).Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-image-name\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-image-name\", \"RestoreItemAction\")).\n\t\t\t\tData(\"specific\", \"5000,3333\").\n\t\t\t\tResult(),\n\t\t\tfreshedImageName: \"1.1.1.1:5000/abc:test\",\n\t\t\twant: \"1.1.1.1:3333/abc:test\",\n\t\t},\n\n\t\t{\n\t\t\tname: \"a valid mapping for a new image tag is applied correctly\",\n\t\t\tpodOrObj: builder.ForPod(\"default\", \"pod1\").ObjectMeta().\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container5\",\n\t\t\t\t\tImage: \"1.1.1.1:5000/abc:test\",\n\t\t\t\t}).Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-image-name\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-image-name\", \"RestoreItemAction\")).\n\t\t\t\tData(\"specific\", \"test,latest\").\n\t\t\t\tResult(),\n\t\t\tfreshedImageName: \"1.1.1.1:5000/abc:test\",\n\t\t\twant: \"1.1.1.1:5000/abc:latest\",\n\t\t},\n\n\t\t{\n\t\t\tname: \"image name contains more than one part that matching the replacing words.\",\n\t\t\tpodOrObj: builder.ForPod(\"default\", \"pod1\").ObjectMeta().\n\t\t\t\tContainers(&corev1api.Container{\n\t\t\t\t\tName: \"container6\",\n\t\t\t\t\tImage: \"dev/image1:dev\",\n\t\t\t\t}).Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-image-name\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-image-name\", \"RestoreItemAction\")).\n\t\t\t\tData(\"specific\", \"dev/,test/\").\n\t\t\t\tResult(),\n\t\t\tfreshedImageName: \"dev/image1:dev\",\n\t\t\twant: \"test/image1:dev\",\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tclientset := fake.NewSimpleClientset()\n\t\t\ta := NewChangeImageNameAction(\n\t\t\t\tlogrus.StandardLogger(),\n\t\t\t\tclientset.CoreV1().ConfigMaps(\"velero\"),\n\t\t\t)\n\n\t\t\t// set up test data\n\t\t\tif tc.configMap != nil {\n\t\t\t\t_, err := clientset.CoreV1().ConfigMaps(tc.configMap.Namespace).Create(t.Context(), tc.configMap, metav1.CreateOptions{})\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tunstructuredMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.podOrObj)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tinput := &velero.RestoreItemActionExecuteInput{\n\t\t\t\tItem: &unstructured.Unstructured{\n\t\t\t\t\tObject: unstructuredMap,\n\t\t\t\t},\n\t\t\t}\n\n\t\t\t// execute method under test\n\t\t\tres, err := a.Execute(input)\n\n\t\t\t// validate for both error and non-error cases\n\t\t\tswitch {\n\t\t\tcase tc.wantErr != nil:\n\t\t\t\trequire.EqualError(t, err, tc.wantErr.Error())\n\t\t\tdefault:\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tpod := new(corev1api.Pod)\n\t\t\t\terr = runtime.DefaultUnstructuredConverter.FromUnstructured(res.UpdatedItem.UnstructuredContent(), pod)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, tc.want, pod.Spec.Containers[0].Image)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/restore/actions/change_storageclass_action.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\tcorev1client \"k8s.io/client-go/kubernetes/typed/core/v1\"\n\tstoragev1client \"k8s.io/client-go/kubernetes/typed/storage/v1\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// ChangeStorageClassAction updates a PV or PVC's storage class name\n// if a mapping is found in the plugin's config map.\ntype ChangeStorageClassAction struct {\n\tlogger logrus.FieldLogger\n\tconfigMapClient corev1client.ConfigMapInterface\n\tstorageClassClient storagev1client.StorageClassInterface\n}\n\n// NewChangeStorageClassAction is the constructor for ChangeStorageClassAction.\nfunc NewChangeStorageClassAction(\n\tlogger logrus.FieldLogger,\n\tconfigMapClient corev1client.ConfigMapInterface,\n\tstorageClassClient storagev1client.StorageClassInterface,\n) *ChangeStorageClassAction {\n\treturn &ChangeStorageClassAction{\n\t\tlogger: logger,\n\t\tconfigMapClient: configMapClient,\n\t\tstorageClassClient: storageClassClient,\n\t}\n}\n\n// AppliesTo returns the resources that ChangeStorageClassAction should\n// be run for.\nfunc (a *ChangeStorageClassAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"persistentvolumeclaims\", \"persistentvolumes\", \"statefulsets\"},\n\t}, nil\n}\n\n// Execute updates the item's spec.storageClassName if a mapping is found\n// in the config map for the plugin.\nfunc (a *ChangeStorageClassAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\ta.logger.Info(\"Executing ChangeStorageClassAction\")\n\tdefer a.logger.Info(\"Done executing ChangeStorageClassAction\")\n\n\ta.logger.Debug(\"Getting plugin config\")\n\tconfig, err := common.GetPluginConfig(common.PluginKindRestoreItemAction, \"velero.io/change-storage-class\", a.configMapClient)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif config == nil || len(config.Data) == 0 {\n\t\ta.logger.Debug(\"No storage class mappings found\")\n\t\treturn velero.NewRestoreItemActionExecuteOutput(input.Item), nil\n\t}\n\n\tobj, ok := input.Item.(*unstructured.Unstructured)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"object was of unexpected type %T\", input.Item)\n\t}\n\n\tlog := a.logger.WithFields(map[string]any{\n\t\t\"kind\": obj.GetKind(),\n\t\t\"namespace\": obj.GetNamespace(),\n\t\t\"name\": obj.GetName(),\n\t})\n\n\t// change StatefulSet volumeClaimTemplates storageClassName\n\tif obj.GetKind() == \"StatefulSet\" {\n\t\tsts := new(appsv1api.StatefulSet)\n\t\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(obj.UnstructuredContent(), sts); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tif len(sts.Spec.VolumeClaimTemplates) > 0 {\n\t\t\tfor index, pvc := range sts.Spec.VolumeClaimTemplates {\n\t\t\t\texists, newStorageClass, err := a.isStorageClassExist(log, pvc.Spec.StorageClassName, config)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn nil, err\n\t\t\t\t} else if !exists {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\n\t\t\t\tlog.Infof(\"Updating item's storage class name to %s\", newStorageClass)\n\t\t\t\tsts.Spec.VolumeClaimTemplates[index].Spec.StorageClassName = &newStorageClass\n\t\t\t}\n\n\t\t\tnewObj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(sts)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, errors.Wrap(err, \"convert obj to StatefulSet failed\")\n\t\t\t}\n\t\t\tobj.Object = newObj\n\t\t}\n\t} else {\n\t\t// use the unstructured helpers here since this code is for both PVs and PVCs, and the\n\t\t// field names are the same for both types.\n\t\tstorageClass, _, err := unstructured.NestedString(obj.UnstructuredContent(), \"spec\", \"storageClassName\")\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrap(err, \"error getting item's spec.storageClassName\")\n\t\t}\n\n\t\texists, newStorageClass, err := a.isStorageClassExist(log, &storageClass, config)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t} else if !exists {\n\t\t\treturn velero.NewRestoreItemActionExecuteOutput(input.Item), nil\n\t\t}\n\n\t\tlog.Infof(\"Updating item's storage class name to %s\", newStorageClass)\n\n\t\tif err := unstructured.SetNestedField(obj.UnstructuredContent(), newStorageClass, \"spec\", \"storageClassName\"); err != nil {\n\t\t\treturn nil, errors.Wrap(err, \"unable to set item's spec.storageClassName\")\n\t\t}\n\t}\n\treturn velero.NewRestoreItemActionExecuteOutput(obj), nil\n}\n\nfunc (a *ChangeStorageClassAction) isStorageClassExist(log *logrus.Entry, storageClass *string, cm *corev1api.ConfigMap) (exists bool, newStorageClass string, err error) {\n\tif storageClass == nil || *storageClass == \"\" {\n\t\tlog.Debug(\"Item has no storage class specified\")\n\t\treturn false, \"\", nil\n\t}\n\n\tnewStorageClass, ok := cm.Data[*storageClass]\n\tif !ok {\n\t\tlog.Debugf(\"No mapping found for storage class %s\", *storageClass)\n\t\treturn false, \"\", nil\n\t}\n\n\t// validate that new storage class exists\n\tif _, err := a.storageClassClient.Get(context.TODO(), newStorageClass, metav1.GetOptions{}); err != nil {\n\t\treturn false, \"\", errors.Wrapf(err, \"error getting storage class %s from API\", newStorageClass)\n\t}\n\n\treturn true, newStorageClass, nil\n}\n" }, { "path": "pkg/restore/actions/change_storageclass_action_test.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tstoragev1api \"k8s.io/api/storage/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes/fake\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// TestChangeStorageClassActionExecute runs the ChangeStorageClassAction's Execute\n// method and validates that the item's storage class is modified (or not) as expected.\n// Validation is done by comparing the result of the Execute method to the test case's\n// desired result.\nfunc TestChangeStorageClassActionExecute(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvOrPvcOrSTS any\n\t\tconfigMap *corev1api.ConfigMap\n\t\tstorageClass *storagev1api.StorageClass\n\t\tstorageClassSlice []*storagev1api.StorageClass\n\t\twant any\n\t\twantErr error\n\t}{\n\t\t{\n\t\t\tname: \"a valid mapping for a persistent volume is applied correctly\",\n\t\t\tpvOrPvcOrSTS: builder.ForPersistentVolume(\"pv-1\").StorageClass(\"storageclass-1\").Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-storage-classs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-storage-class\", \"RestoreItemAction\")).\n\t\t\t\tData(\"storageclass-1\", \"storageclass-2\").\n\t\t\t\tResult(),\n\t\t\tstorageClass: builder.ForStorageClass(\"storageclass-2\").Result(),\n\t\t\twant: builder.ForPersistentVolume(\"pv-1\").StorageClass(\"storageclass-2\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"a valid mapping for a persistent volume claim is applied correctly\",\n\t\t\tpvOrPvcOrSTS: builder.ForPersistentVolumeClaim(\"velero\", \"pvc-1\").StorageClass(\"storageclass-1\").Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-storage-classs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-storage-class\", \"RestoreItemAction\")).\n\t\t\t\tData(\"storageclass-1\", \"storageclass-2\").\n\t\t\t\tResult(),\n\t\t\tstorageClass: builder.ForStorageClass(\"storageclass-2\").Result(),\n\t\t\twant: builder.ForPersistentVolumeClaim(\"velero\", \"pvc-1\").StorageClass(\"storageclass-2\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"when no config map exists for the plugin, the item is returned as-is\",\n\t\t\tpvOrPvcOrSTS: builder.ForPersistentVolume(\"pv-1\").StorageClass(\"storageclass-1\").Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-storage-classs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/some-other-plugin\", \"RestoreItemAction\")).\n\t\t\t\tData(\"storageclass-1\", \"storageclass-2\").\n\t\t\t\tResult(),\n\t\t\twant: builder.ForPersistentVolume(\"pv-1\").StorageClass(\"storageclass-1\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"when no storage class mappings exist in the plugin config map, the item is returned as-is\",\n\t\t\tpvOrPvcOrSTS: builder.ForPersistentVolume(\"pv-1\").StorageClass(\"storageclass-1\").Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-storage-classs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-storage-class\", \"RestoreItemAction\")).\n\t\t\t\tResult(),\n\t\t\twant: builder.ForPersistentVolume(\"pv-1\").StorageClass(\"storageclass-1\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"when persistent volume has no storage class, the item is returned as-is\",\n\t\t\tpvOrPvcOrSTS: builder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-storage-classs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-storage-class\", \"RestoreItemAction\")).\n\t\t\t\tData(\"storageclass-1\", \"storageclass-2\").\n\t\t\t\tResult(),\n\t\t\twant: builder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"when persistent volume claim has no storage class, the item is returned as-is\",\n\t\t\tpvOrPvcOrSTS: builder.ForPersistentVolumeClaim(\"velero\", \"pvc-1\").Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-storage-classs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-storage-class\", \"RestoreItemAction\")).\n\t\t\t\tData(\"storageclass-1\", \"storageclass-2\").\n\t\t\t\tResult(),\n\t\t\twant: builder.ForPersistentVolumeClaim(\"velero\", \"pvc-1\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"when persistent volume's storage class has no mapping in the config map, the item is returned as-is\",\n\t\t\tpvOrPvcOrSTS: builder.ForPersistentVolume(\"pv-1\").StorageClass(\"storageclass-1\").Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-storage-classs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-storage-class\", \"RestoreItemAction\")).\n\t\t\t\tData(\"storageclass-3\", \"storageclass-4\").\n\t\t\t\tResult(),\n\t\t\twant: builder.ForPersistentVolume(\"pv-1\").StorageClass(\"storageclass-1\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"when persistent volume claim's storage class has no mapping in the config map, the item is returned as-is\",\n\t\t\tpvOrPvcOrSTS: builder.ForPersistentVolumeClaim(\"velero\", \"pvc-1\").StorageClass(\"storageclass-1\").Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-storage-classs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-storage-class\", \"RestoreItemAction\")).\n\t\t\t\tData(\"storageclass-3\", \"storageclass-4\").\n\t\t\t\tResult(),\n\t\t\twant: builder.ForPersistentVolumeClaim(\"velero\", \"pvc-1\").StorageClass(\"storageclass-1\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"when persistent volume's storage class is mapped to a nonexistent storage class, an error is returned\",\n\t\t\tpvOrPvcOrSTS: builder.ForPersistentVolume(\"pv-1\").StorageClass(\"storageclass-1\").Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-storage-classs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-storage-class\", \"RestoreItemAction\")).\n\t\t\t\tData(\"storageclass-1\", \"nonexistent-storage-class\").\n\t\t\t\tResult(),\n\t\t\twantErr: errors.New(\"error getting storage class nonexistent-storage-class from API: storageclasses.storage.k8s.io \\\"nonexistent-storage-class\\\" not found\"),\n\t\t},\n\t\t{\n\t\t\tname: \"when persistent volume claim's storage class is mapped to a nonexistent storage class, an error is returned\",\n\t\t\tpvOrPvcOrSTS: builder.ForPersistentVolumeClaim(\"velero\", \"pvc-1\").StorageClass(\"storageclass-1\").Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-storage-classs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-storage-class\", \"RestoreItemAction\")).\n\t\t\t\tData(\"storageclass-1\", \"nonexistent-storage-class\").\n\t\t\t\tResult(),\n\t\t\twantErr: errors.New(\"error getting storage class nonexistent-storage-class from API: storageclasses.storage.k8s.io \\\"nonexistent-storage-class\\\" not found\"),\n\t\t},\n\t\t{\n\t\t\tname: \"when statefulset's VolumeClaimTemplates has only one pvc, a valid mapping for a statefulset is applied correctly\",\n\t\t\tpvOrPvcOrSTS: builder.ForStatefulSet(\"velero\", \"sts-1\").StorageClass(\"storageclass-1\").Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-storage-classs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-storage-class\", \"RestoreItemAction\")).\n\t\t\t\tData(\"storageclass-1\", \"storageclass-2\").\n\t\t\t\tResult(),\n\t\t\tstorageClass: builder.ForStorageClass(\"storageclass-2\").Result(),\n\t\t\twant: builder.ForStatefulSet(\"velero\", \"sts-1\").StorageClass(\"storageclass-2\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"when statefulset's VolumeClaimTemplates has more than one same pvc's storageClassName, a valid mapping for a statefulset is applied correctly\",\n\t\t\tpvOrPvcOrSTS: builder.ForStatefulSet(\"velero\", \"sts-1\").StorageClass(\"storageclass-1\", \"storageclass-1\").Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-storage-classs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-storage-class\", \"RestoreItemAction\")).\n\t\t\t\tData(\"storageclass-1\", \"storageclass-2\", \"storageclass-3\", \"storageclass-4\").\n\t\t\t\tResult(),\n\t\t\tstorageClass: builder.ForStorageClass(\"storageclass-2\").Result(),\n\t\t\twant: builder.ForStatefulSet(\"velero\", \"sts-1\").StorageClass(\"storageclass-2\", \"storageclass-2\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"when statefulset's VolumeClaimTemplates has more than one different pvc's storageClassName, a valid mapping for a statefulset is applied correctly\",\n\t\t\tpvOrPvcOrSTS: builder.ForStatefulSet(\"velero\", \"sts-1\").StorageClass(\"storageclass-1\", \"storageclass-2\", \"storageclass-3\").Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-storage-classs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-storage-class\", \"RestoreItemAction\")).\n\t\t\t\tData(\"storageclass-1\", \"storageclass-a\", \"storageclass-2\", \"storageclass-b\", \"storageclass-3\", \"storageclass-c\").\n\t\t\t\tResult(),\n\t\t\tstorageClassSlice: builder.ForStorageClassSlice(\"storageclass-a\", \"storageclass-b\", \"storageclass-c\").SliceResult(),\n\t\t\twant: builder.ForStatefulSet(\"velero\", \"sts-1\").StorageClass(\"storageclass-a\", \"storageclass-b\", \"storageclass-c\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"when no config map exists for the plugin, the statefulset item is returned as-is\",\n\t\t\tpvOrPvcOrSTS: builder.ForStatefulSet(\"velero\", \"sts-1\").StorageClass(\"storageclass-1\").Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-storage-classs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/some-other-plugin\", \"RestoreItemAction\")).\n\t\t\t\tData(\"storageclass-1\", \"storageclass-2\").\n\t\t\t\tResult(),\n\t\t\twant: builder.ForStatefulSet(\"velero\", \"sts-1\").StorageClass(\"storageclass-1\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"when no storage class mappings exist in the plugin config map, the statefulset item is returned as-is\",\n\t\t\tpvOrPvcOrSTS: builder.ForStatefulSet(\"velero\", \"sts-1\").StorageClass(\"storageclass-1\").Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-storage-classs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-storage-class\", \"RestoreItemAction\")).\n\t\t\t\tResult(),\n\t\t\twant: builder.ForStatefulSet(\"velero\", \"sts-1\").StorageClass(\"storageclass-1\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"when persistent volume claim has no storage class, the statefulset item is returned as-is\",\n\t\t\tpvOrPvcOrSTS: builder.ForStatefulSet(\"velero\", \"sts-1\").Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-storage-classs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-storage-class\", \"RestoreItemAction\")).\n\t\t\t\tResult(),\n\t\t\twant: builder.ForStatefulSet(\"velero\", \"sts-1\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"when statefulset's storage class has no mapping in the config map, the item is returned as-is\",\n\t\t\tpvOrPvcOrSTS: builder.ForStatefulSet(\"velero\", \"sts-1\").StorageClass(\"storageclass-1\").Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-storage-classs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-storage-class\", \"RestoreItemAction\")).\n\t\t\t\tData(\"storageclass-3\", \"storageclass-4\").\n\t\t\t\tResult(),\n\t\t\twant: builder.ForStatefulSet(\"velero\", \"sts-1\").StorageClass(\"storageclass-1\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"when statefulset's storage class is mapped to a nonexistent storage class, an error is returned\",\n\t\t\tpvOrPvcOrSTS: builder.ForStatefulSet(\"velero\", \"sts-1\").StorageClass(\"storageclass-1\").Result(),\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"change-storage-classs\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"velero.io/plugin-config\", \"\", \"velero.io/change-storage-class\", \"RestoreItemAction\")).\n\t\t\t\tData(\"storageclass-1\", \"nonexistent-storage-class\").\n\t\t\t\tResult(),\n\t\t\twantErr: errors.New(\"error getting storage class nonexistent-storage-class from API: storageclasses.storage.k8s.io \\\"nonexistent-storage-class\\\" not found\"),\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tclientset := fake.NewSimpleClientset()\n\t\t\ta := NewChangeStorageClassAction(\n\t\t\t\tlogrus.StandardLogger(),\n\t\t\t\tclientset.CoreV1().ConfigMaps(\"velero\"),\n\t\t\t\tclientset.StorageV1().StorageClasses(),\n\t\t\t)\n\n\t\t\t// set up test data\n\t\t\tif tc.configMap != nil {\n\t\t\t\t_, err := clientset.CoreV1().ConfigMaps(tc.configMap.Namespace).Create(t.Context(), tc.configMap, metav1.CreateOptions{})\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif tc.storageClass != nil {\n\t\t\t\t_, err := clientset.StorageV1().StorageClasses().Create(t.Context(), tc.storageClass, metav1.CreateOptions{})\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif tc.storageClassSlice != nil {\n\t\t\t\tfor _, storageClass := range tc.storageClassSlice {\n\t\t\t\t\t_, err := clientset.StorageV1().StorageClasses().Create(t.Context(), storageClass, metav1.CreateOptions{})\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tunstructuredMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.pvOrPvcOrSTS)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tinput := &velero.RestoreItemActionExecuteInput{\n\t\t\t\tItem: &unstructured.Unstructured{\n\t\t\t\t\tObject: unstructuredMap,\n\t\t\t\t},\n\t\t\t}\n\n\t\t\t// execute method under test\n\t\t\tres, err := a.Execute(input)\n\n\t\t\t// validate for both error and non-error cases\n\t\t\tswitch {\n\t\t\tcase tc.wantErr != nil:\n\t\t\t\trequire.EqualError(t, err, tc.wantErr.Error())\n\t\t\tdefault:\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\twantUnstructured, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.want)\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\tassert.Equal(t, &unstructured.Unstructured{Object: wantUnstructured}, res.UpdatedItem)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/restore/actions/clusterrolebinding_action.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\trbacv1 \"k8s.io/api/rbac/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// ClusterRoleBindingAction handle namespace remappings for role bindings\ntype ClusterRoleBindingAction struct {\n\tlogger logrus.FieldLogger\n}\n\nfunc NewClusterRoleBindingAction(logger logrus.FieldLogger) *ClusterRoleBindingAction {\n\treturn &ClusterRoleBindingAction{logger: logger}\n}\n\nfunc (a *ClusterRoleBindingAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"clusterrolebindings\"},\n\t}, nil\n}\n\nfunc (a *ClusterRoleBindingAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\tnamespaceMapping := input.Restore.Spec.NamespaceMapping\n\tif len(namespaceMapping) == 0 {\n\t\treturn velero.NewRestoreItemActionExecuteOutput(&unstructured.Unstructured{Object: input.Item.UnstructuredContent()}), nil\n\t}\n\n\tclusterRoleBinding := new(rbacv1.ClusterRoleBinding)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(input.Item.UnstructuredContent(), clusterRoleBinding); err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tfor i, subject := range clusterRoleBinding.Subjects {\n\t\tif newNamespace, ok := namespaceMapping[subject.Namespace]; ok {\n\t\t\tclusterRoleBinding.Subjects[i].Namespace = newNamespace\n\t\t}\n\t}\n\n\tres, err := runtime.DefaultUnstructuredConverter.ToUnstructured(clusterRoleBinding)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\treturn velero.NewRestoreItemActionExecuteOutput(&unstructured.Unstructured{Object: res}), nil\n}\n" }, { "path": "pkg/restore/actions/clusterrolebinding_action_test.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"sort\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\trbacv1 \"k8s.io/api/rbac/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestClusterRoleBindingActionAppliesTo(t *testing.T) {\n\taction := NewClusterRoleBindingAction(test.NewLogger())\n\tactual, err := action.AppliesTo()\n\trequire.NoError(t, err)\n\tassert.Equal(t, velero.ResourceSelector{IncludedResources: []string{\"clusterrolebindings\"}}, actual)\n}\n\nfunc TestClusterRoleBindingActionExecute(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tnamespaces []string\n\t\tnamespaceMapping map[string]string\n\t\texpected []string\n\t}{\n\t\t{\n\t\t\tname: \"namespace mapping disabled\",\n\t\t\tnamespaces: []string{\"foo\"},\n\t\t\tnamespaceMapping: map[string]string{},\n\t\t\texpected: []string{\"foo\"},\n\t\t},\n\t\t{\n\t\t\tname: \"namespace mapping enabled\",\n\t\t\tnamespaces: []string{\"foo\"},\n\t\t\tnamespaceMapping: map[string]string{\"foo\": \"bar\", \"fizz\": \"buzz\"},\n\t\t\texpected: []string{\"bar\"},\n\t\t},\n\t\t{\n\t\t\tname: \"namespace mapping enabled, not included namespace remains unchanged\",\n\t\t\tnamespaces: []string{\"foo\", \"xyz\"},\n\t\t\tnamespaceMapping: map[string]string{\"foo\": \"bar\", \"fizz\": \"buzz\"},\n\t\t\texpected: []string{\"bar\", \"xyz\"},\n\t\t},\n\t\t{\n\t\t\tname: \"namespace mapping enabled, not included namespace remains unchanged\",\n\t\t\tnamespaces: []string{\"foo\", \"xyz\"},\n\t\t\tnamespaceMapping: map[string]string{\"a\": \"b\", \"c\": \"d\"},\n\t\t\texpected: []string{\"foo\", \"xyz\"},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tsubjects := []rbacv1.Subject{}\n\n\t\t\tfor _, ns := range tc.namespaces {\n\t\t\t\tsubjects = append(subjects, rbacv1.Subject{\n\t\t\t\t\tNamespace: ns,\n\t\t\t\t})\n\t\t\t}\n\n\t\t\tclusterRoleBinding := rbacv1.ClusterRoleBinding{\n\t\t\t\tSubjects: subjects,\n\t\t\t}\n\n\t\t\troleBindingUnstructured, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&clusterRoleBinding)\n\t\t\trequire.NoError(t, err)\n\n\t\t\taction := NewClusterRoleBindingAction(test.NewLogger())\n\t\t\tres, err := action.Execute(&velero.RestoreItemActionExecuteInput{\n\t\t\t\tItem: &unstructured.Unstructured{Object: roleBindingUnstructured},\n\t\t\t\tItemFromBackup: &unstructured.Unstructured{Object: roleBindingUnstructured},\n\t\t\t\tRestore: &api.Restore{\n\t\t\t\t\tSpec: api.RestoreSpec{\n\t\t\t\t\t\tNamespaceMapping: tc.namespaceMapping,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t})\n\t\t\trequire.NoError(t, err)\n\n\t\t\tvar resClusterRoleBinding *rbacv1.ClusterRoleBinding\n\t\t\terr = runtime.DefaultUnstructuredConverter.FromUnstructured(res.UpdatedItem.UnstructuredContent(), &resClusterRoleBinding)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tactual := []string{}\n\t\t\tfor _, subject := range resClusterRoleBinding.Subjects {\n\t\t\t\tactual = append(actual, subject.Namespace)\n\t\t\t}\n\n\t\t\tsort.Strings(tc.expected)\n\t\t\tsort.Strings(actual)\n\n\t\t\tassert.Equal(t, tc.expected, actual)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/restore/actions/crd_v1_preserve_unknown_fields_action.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\t http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"encoding/json\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tapiextv1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// CRDV1PreserveUnknownFieldsAction will take a CRD and inspect it for the API version and the PreserveUnknownFields value.\n// If the API Version is 1 and the PreserveUnknownFields value is True, then the x-preserve-unknown-fields value in the OpenAPIV3 schema will be set to True\n// and PreserveUnknownFields set to False in order to allow Kubernetes 1.16+ servers to accept the object.\ntype CRDV1PreserveUnknownFieldsAction struct {\n\tlogger logrus.FieldLogger\n}\n\nfunc NewCRDV1PreserveUnknownFieldsAction(logger logrus.FieldLogger) *CRDV1PreserveUnknownFieldsAction {\n\treturn &CRDV1PreserveUnknownFieldsAction{logger: logger}\n}\n\nfunc (c *CRDV1PreserveUnknownFieldsAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"customresourcedefinition.apiextensions.k8s.io\"},\n\t}, nil\n}\n\nfunc (c *CRDV1PreserveUnknownFieldsAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\tc.logger.Info(\"Executing CRDV1PreserveUnknownFieldsAction\")\n\n\tname, _, err := unstructured.NestedString(input.Item.UnstructuredContent(), \"name\")\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"could not get CRD name\")\n\t}\n\n\tlog := c.logger.WithField(\"plugin\", \"CRDV1PreserveUnknownFieldsAction\").WithField(\"CRD\", name)\n\n\tversion, _, err := unstructured.NestedString(input.Item.UnstructuredContent(), \"apiVersion\")\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"could not get CRD version\")\n\t}\n\n\t// We don't want to \"fix\" anything in beta CRDs at the moment, just v1 versions with preserveunknownfields = true\n\tif version != \"apiextensions.k8s.io/v1\" {\n\t\treturn &velero.RestoreItemActionExecuteOutput{\n\t\t\tUpdatedItem: input.Item,\n\t\t}, nil\n\t}\n\n\t// Do not use runtime.DefaultUnstructuredConverter.FromUnstructured here because it has a bug when converting integers/whole\n\t// numbers in float fields (https://github.com/kubernetes/kubernetes/issues/87675).\n\t// Using JSON as a go-between avoids this issue, without adding a bunch of type conversion by using unstructured helper functions\n\t// to inspect the fields we want to look at.\n\tcrd, err := fromUnstructured(input.Item.UnstructuredContent())\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to convert CRD from unstructured to structured\")\n\t}\n\n\t// The v1 API doesn't allow the PreserveUnknownFields value to be true, so make sure the schema flag is set instead\n\tif crd.Spec.PreserveUnknownFields {\n\t\t// First, change the top-level value since the Kubernetes API server on 1.16+ will generate errors otherwise.\n\t\tlog.Debug(\"Set PreserveUnknownFields to False\")\n\t\tcrd.Spec.PreserveUnknownFields = false\n\n\t\t// Make sure all versions are set to preserve unknown fields\n\t\tfor _, v := range crd.Spec.Versions {\n\t\t\t// If the schema fields are nil, there are no nested fields to set, so skip over it for this version.\n\t\t\tif v.Schema == nil || v.Schema.OpenAPIV3Schema == nil {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\t// Use the address, since the XPreserveUnknownFields value is nil or\n\t\t\t// a pointer to true (false is not allowed)\n\t\t\tpreserve := true\n\t\t\tv.Schema.OpenAPIV3Schema.XPreserveUnknownFields = &preserve\n\t\t\tlog.Debugf(\"Set x-preserve-unknown-fields in Open API for schema version %s\", v.Name)\n\t\t}\n\t}\n\n\tres, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&crd)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to convert crd to runtime.Unstructured\")\n\t}\n\n\treturn &velero.RestoreItemActionExecuteOutput{\n\t\tUpdatedItem: &unstructured.Unstructured{Object: res},\n\t}, nil\n}\n\nfunc fromUnstructured(unstructured map[string]any) (*apiextv1.CustomResourceDefinition, error) {\n\tvar crd apiextv1.CustomResourceDefinition\n\n\tjs, err := json.Marshal(unstructured)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to convert unstructured item to JSON\")\n\t}\n\n\tif err = json.Unmarshal(js, &crd); err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to convert JSON to CRD Go type\")\n\t}\n\n\treturn &crd, nil\n}\n" }, { "path": "pkg/restore/actions/crd_v1_preserve_unknown_fields_action_test.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\t http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"encoding/json\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestExecuteForACRDWithAnIntOnAFloat64FieldShouldWork(t *testing.T) {\n\t// ref. reopen of https://github.com/vmware-tanzu/velero/issues/2319\n\n\tb := builder.ForV1CustomResourceDefinition(\"test.velero.io\")\n\t// 5 here is just an int value, it could be any other whole number.\n\tschema := builder.ForJSONSchemaPropsBuilder().Maximum(5).Result()\n\tb.Version(builder.ForV1CustomResourceDefinitionVersion(\"v1\").Served(true).Storage(true).Schema(schema).Result())\n\tc := b.Result()\n\n\t// Marshall in and out of JSON because the problem doesn't manifest when we use ToUnstructured directly\n\t// This should simulate the JSON passing over the wire in an HTTP request/response with a dynamic client\n\tjs, err := json.Marshal(c)\n\trequire.NoError(t, err)\n\n\tvar u unstructured.Unstructured\n\terr = json.Unmarshal(js, &u)\n\trequire.NoError(t, err)\n\n\ta := NewCRDV1PreserveUnknownFieldsAction(test.NewLogger())\n\n\t_, err = a.Execute(&velero.RestoreItemActionExecuteInput{Item: &u})\n\trequire.NoError(t, err)\n}\n" }, { "path": "pkg/restore/actions/csi/pvc_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\tutilrand \"k8s.io/apimachinery/pkg/util/rand\"\n\tcrclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\tkuberesource \"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\tplugincommon \"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\triav2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/restoreitemaction/v2\"\n\tuploaderUtil \"github.com/vmware-tanzu/velero/pkg/uploader/util\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n)\n\nconst (\n\tAnnSelectedNode = \"volume.kubernetes.io/selected-node\"\n\tGenerateNameRandomLength = 5\n)\n\n// pvcRestoreItemAction is a restore item action plugin for Velero\ntype pvcRestoreItemAction struct {\n\tlog logrus.FieldLogger\n\tcrClient crclient.Client\n}\n\n// AppliesTo returns information indicating that the\n// PVCRestoreItemAction should be run while restoring PVCs.\nfunc (p *pvcRestoreItemAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"persistentvolumeclaims\"},\n\t\t//TODO: add label selector volumeSnapshotLabel\n\t}, nil\n}\n\n// Execute modifies the PVC's spec to use the VolumeSnapshot object as the\n// data source ensuring that the newly provisioned volume can be pre-populated\n// with data from the VolumeSnapshot.\nfunc (p *pvcRestoreItemAction) Execute(\n\tinput *velero.RestoreItemActionExecuteInput,\n) (*velero.RestoreItemActionExecuteOutput, error) {\n\tvar pvc, pvcFromBackup corev1api.PersistentVolumeClaim\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(\n\t\tinput.Item.UnstructuredContent(), &pvc); err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(\n\t\tinput.ItemFromBackup.UnstructuredContent(), &pvcFromBackup); err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tlogger := p.log.WithFields(logrus.Fields{\n\t\t\"Action\": \"PVCRestoreItemAction\",\n\t\t\"PVC\": pvc.Namespace + \"/\" + pvc.Name,\n\t\t\"Restore\": input.Restore.Namespace + \"/\" + input.Restore.Name,\n\t})\n\tlogger.Info(\"Starting PVCRestoreItemAction for PVC\")\n\n\t// If PVC already exists, returns early.\n\tif p.isResourceExist(pvc, *input.Restore) {\n\t\tlogger.Warnf(\"PVC already exists. Skip restore this PVC.\")\n\t\treturn &velero.RestoreItemActionExecuteOutput{\n\t\t\tUpdatedItem: input.Item,\n\t\t}, nil\n\t}\n\n\t// If cross-namespace restore is configured, change the namespace\n\t// for PVC object to be restored\n\tnewNamespace, ok := input.Restore.Spec.NamespaceMapping[pvc.GetNamespace()]\n\tif !ok {\n\t\t// Use original namespace\n\t\tnewNamespace = pvc.Namespace\n\t}\n\n\toperationID := \"\"\n\n\tadditionalItems := []velero.ResourceIdentifier{}\n\tif boolptr.IsSetToFalse(input.Restore.Spec.RestorePVs) {\n\t\tlogger.Info(\"Restore did not request for PVs to be restored from snapshot\")\n\t\tpvc.Spec.VolumeName = \"\"\n\t\tpvc.Spec.DataSource = nil\n\t\tpvc.Spec.DataSourceRef = nil\n\t} else {\n\t\tbackup := new(velerov1api.Backup)\n\t\terr := p.crClient.Get(\n\t\t\tcontext.TODO(),\n\t\t\tcrclient.ObjectKey{\n\t\t\t\tNamespace: input.Restore.Namespace,\n\t\t\t\tName: input.Restore.Spec.BackupName,\n\t\t\t},\n\t\t\tbackup,\n\t\t)\n\n\t\tif err != nil {\n\t\t\tlogger.Error(\"Fail to get backup for restore.\")\n\t\t\treturn nil, fmt.Errorf(\"fail to get backup for restore: %s\", err.Error())\n\t\t}\n\n\t\tif boolptr.IsSetToTrue(backup.Spec.SnapshotMoveData) {\n\t\t\tlogger.Info(\"Start DataMover restore.\")\n\n\t\t\t// If PVC doesn't have a DataUploadNameLabel, which should be created\n\t\t\t// during backup, then CSI cannot handle the volume during to restore,\n\t\t\t// so return early to let Velero tries to fall back to Velero native snapshot.\n\t\t\tif _, ok := pvcFromBackup.Annotations[velerov1api.DataUploadNameAnnotation]; !ok {\n\t\t\t\tlogger.Warnf(\"PVC doesn't have a DataUpload for data mover. Return.\")\n\t\t\t\treturn &velero.RestoreItemActionExecuteOutput{\n\t\t\t\t\tUpdatedItem: input.Item,\n\t\t\t\t}, nil\n\t\t\t}\n\n\t\t\toperationID = label.GetValidName(\n\t\t\t\tstring(velerov1api.AsyncOperationIDPrefixDataDownload) +\n\t\t\t\t\tstring(input.Restore.UID) + \".\" + string(pvcFromBackup.UID))\n\t\t\tdataDownload, err := restoreFromDataUploadResult(\n\t\t\t\tcontext.Background(), input.Restore, backup, &pvc, newNamespace,\n\t\t\t\toperationID, p.crClient)\n\t\t\tif err != nil {\n\t\t\t\tlogger.Errorf(\"Fail to restore from DataUploadResult: %s\", err.Error())\n\t\t\t\treturn nil, errors.WithStack(err)\n\t\t\t}\n\t\t\tlogger.Infof(\"DataDownload %s/%s is created successfully.\",\n\t\t\t\tdataDownload.Namespace, dataDownload.Name)\n\t\t} else {\n\t\t\t//CSI restore\n\t\t\tvsName, nameOK := pvcFromBackup.Annotations[velerov1api.VolumeSnapshotLabel]\n\t\t\tif !nameOK {\n\t\t\t\tlogger.Info(\"Skipping PVCRestoreItemAction for PVC, PVC does not have a CSI VolumeSnapshot.\")\n\t\t\t\treturn &velero.RestoreItemActionExecuteOutput{\n\t\t\t\t\tUpdatedItem: input.Item,\n\t\t\t\t}, nil\n\t\t\t}\n\n\t\t\t//To avoid confilcs, vs and vsc get a new uniq name based in restore UID\n\t\t\t// and vs name old name\n\t\t\tnewVSName := util.GenerateSha256FromRestoreUIDAndVsName(string(input.Restore.UID), vsName)\n\n\t\t\tp.log.Debugf(\"Setting PVC source to VolumeSnapshot new name: %s\", newVSName)\n\t\t\tresetPVCSourceToVolumeSnapshot(&pvc, newVSName)\n\n\t\t\tadditionalItems = append(additionalItems, velero.ResourceIdentifier{\n\t\t\t\tGroupResource: kuberesource.VolumeSnapshots,\n\t\t\t\tName: vsName,\n\t\t\t\tNamespace: pvc.Namespace,\n\t\t\t})\n\t\t}\n\t}\n\n\tpvcMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&pvc)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\tlogger.Info(\"Returning from PVCRestoreItemAction for PVC\")\n\n\treturn &velero.RestoreItemActionExecuteOutput{\n\t\tUpdatedItem: &unstructured.Unstructured{Object: pvcMap},\n\t\tOperationID: operationID,\n\t\tAdditionalItems: additionalItems,\n\t}, nil\n}\n\nfunc resetPVCSourceToVolumeSnapshot(pvc *corev1api.PersistentVolumeClaim, vsName string) {\n\t// Restore operation for the PVC will use the VolumeSnapshot as the data source.\n\t// So clear out the volume name, which is a ref to the PV\n\tpvc.Spec.VolumeName = \"\"\n\tdataSource := &corev1api.TypedLocalObjectReference{\n\t\tAPIGroup: &snapshotv1api.SchemeGroupVersion.Group,\n\t\tKind: \"VolumeSnapshot\",\n\t\tName: vsName,\n\t}\n\tpvc.Spec.DataSource = dataSource\n\tpvc.Spec.DataSourceRef = nil\n}\n\nfunc (p *pvcRestoreItemAction) Name() string {\n\treturn \"PVCRestoreItemAction\"\n}\n\nfunc (p *pvcRestoreItemAction) Progress(\n\toperationID string,\n\trestore *velerov1api.Restore,\n) (velero.OperationProgress, error) {\n\tprogress := velero.OperationProgress{}\n\n\tif operationID == \"\" {\n\t\treturn progress, riav2.InvalidOperationIDError(operationID)\n\t}\n\tlogger := p.log.WithFields(logrus.Fields{\n\t\t\"Action\": \"PVCRestoreItemAction\",\n\t\t\"OperationID\": operationID,\n\t\t\"Namespace\": restore.Namespace,\n\t})\n\n\tdataDownload, err := getDataDownload(\n\t\tcontext.Background(),\n\t\trestore.Namespace,\n\t\toperationID,\n\t\tp.crClient,\n\t)\n\tif err != nil {\n\t\tlogger.Errorf(\"fail to get DataDownload: %s\", err.Error())\n\t\treturn progress, err\n\t}\n\tif dataDownload.Status.Phase == velerov2alpha1.DataDownloadPhaseNew ||\n\t\tdataDownload.Status.Phase == \"\" {\n\t\tlogger.Debugf(\"DataDownload is still not processed yet. Skip progress update.\")\n\t\treturn progress, nil\n\t}\n\n\tprogress.Description = string(dataDownload.Status.Phase)\n\tprogress.OperationUnits = \"Bytes\"\n\tprogress.NCompleted = dataDownload.Status.Progress.BytesDone\n\tprogress.NTotal = dataDownload.Status.Progress.TotalBytes\n\n\tif dataDownload.Status.StartTimestamp != nil {\n\t\tprogress.Started = dataDownload.Status.StartTimestamp.Time\n\t}\n\n\tif dataDownload.Status.CompletionTimestamp != nil {\n\t\tprogress.Updated = dataDownload.Status.CompletionTimestamp.Time\n\t}\n\n\tif dataDownload.Status.Phase == velerov2alpha1.DataDownloadPhaseCompleted {\n\t\tprogress.Completed = true\n\t} else if dataDownload.Status.Phase == velerov2alpha1.DataDownloadPhaseCanceled {\n\t\tprogress.Completed = true\n\t\tprogress.Err = \"DataDownload is canceled\"\n\t} else if dataDownload.Status.Phase == velerov2alpha1.DataDownloadPhaseFailed {\n\t\tprogress.Completed = true\n\t\tprogress.Err = dataDownload.Status.Message\n\t}\n\n\treturn progress, nil\n}\n\nfunc (p *pvcRestoreItemAction) Cancel(\n\toperationID string, restore *velerov1api.Restore) error {\n\tif operationID == \"\" {\n\t\treturn riav2.InvalidOperationIDError(operationID)\n\t}\n\tlogger := p.log.WithFields(logrus.Fields{\n\t\t\"Action\": \"PVCRestoreItemAction\",\n\t\t\"OperationID\": operationID,\n\t\t\"Namespace\": restore.Namespace,\n\t})\n\n\tdataDownload, err := getDataDownload(\n\t\tcontext.Background(),\n\t\trestore.Namespace,\n\t\toperationID,\n\t\tp.crClient,\n\t)\n\tif err != nil {\n\t\tlogger.Errorf(\"fail to get DataDownload: %s\", err.Error())\n\t\treturn err\n\t}\n\n\terr = cancelDataDownload(context.Background(), p.crClient, dataDownload)\n\tif err != nil {\n\t\tlogger.Errorf(\"fail to cancel DataDownload %s: %s\", dataDownload.Name, err.Error())\n\t}\n\treturn err\n}\n\nfunc (p *pvcRestoreItemAction) AreAdditionalItemsReady(\n\tadditionalItems []velero.ResourceIdentifier,\n\trestore *velerov1api.Restore,\n) (bool, error) {\n\treturn true, nil\n}\n\nfunc getDataUploadResult(\n\tctx context.Context,\n\trestore *velerov1api.Restore,\n\tpvc *corev1api.PersistentVolumeClaim,\n\tcrClient crclient.Client,\n) (*velerov2alpha1.DataUploadResult, error) {\n\tselectorStr := fmt.Sprintf(\"%s=%s,%s=%s,%s=%s\",\n\t\tvelerov1api.PVCNamespaceNameLabel,\n\t\tlabel.GetValidName(pvc.Namespace+\".\"+pvc.Name),\n\t\tvelerov1api.RestoreUIDLabel,\n\t\tlabel.GetValidName(string(restore.UID)),\n\t\tvelerov1api.ResourceUsageLabel,\n\t\tlabel.GetValidName(string(velerov1api.VeleroResourceUsageDataUploadResult)),\n\t)\n\tselector, _ := labels.Parse(selectorStr)\n\n\tcmList := new(corev1api.ConfigMapList)\n\tif err := crClient.List(\n\t\tctx,\n\t\tcmList,\n\t\t&crclient.ListOptions{\n\t\t\tLabelSelector: selector,\n\t\t\tNamespace: restore.Namespace,\n\t\t}); err != nil {\n\t\treturn nil, errors.Wrapf(err,\n\t\t\t\"error to get DataUpload result cm with labels %s\", selectorStr)\n\t}\n\n\tif len(cmList.Items) == 0 {\n\t\treturn nil, errors.Errorf(\n\t\t\t\"no DataUpload result cm found with labels %s\", selectorStr)\n\t}\n\n\tif len(cmList.Items) > 1 {\n\t\treturn nil, errors.Errorf(\n\t\t\t\"multiple DataUpload result cms found with labels %s\", selectorStr)\n\t}\n\n\tjsonBytes, exist := cmList.Items[0].Data[string(restore.UID)]\n\tif !exist {\n\t\treturn nil, errors.Errorf(\n\t\t\t\"no DataUpload result found with restore key %s, restore %s\",\n\t\t\tstring(restore.UID), restore.Name)\n\t}\n\n\tresult := velerov2alpha1.DataUploadResult{}\n\tif err := json.Unmarshal([]byte(jsonBytes), &result); err != nil {\n\t\treturn nil, errors.Errorf(\n\t\t\t\"error to unmarshal DataUploadResult, restore UID %s, restore name %s\",\n\t\t\tstring(restore.UID), restore.Name)\n\t}\n\n\treturn &result, nil\n}\n\nfunc getDataDownload(\n\tctx context.Context,\n\tnamespace string,\n\toperationID string,\n\tcrClient crclient.Client,\n) (*velerov2alpha1.DataDownload, error) {\n\tdataDownloadList := new(velerov2alpha1.DataDownloadList)\n\terr := crClient.List(ctx, dataDownloadList, &crclient.ListOptions{\n\t\tLabelSelector: labels.SelectorFromSet(map[string]string{\n\t\t\tvelerov1api.AsyncOperationIDLabel: operationID,\n\t\t}),\n\t\tNamespace: namespace,\n\t})\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"fail to list DataDownload\")\n\t}\n\n\tif len(dataDownloadList.Items) == 0 {\n\t\treturn nil, errors.Errorf(\"didn't find DataDownload\")\n\t}\n\n\tif len(dataDownloadList.Items) > 1 {\n\t\treturn nil, errors.Errorf(\"find multiple DataDownloads\")\n\t}\n\n\treturn &dataDownloadList.Items[0], nil\n}\n\nfunc cancelDataDownload(ctx context.Context, crClient crclient.Client,\n\tdataDownload *velerov2alpha1.DataDownload) error {\n\tupdatedDataDownload := dataDownload.DeepCopy()\n\tupdatedDataDownload.Spec.Cancel = true\n\n\treturn crClient.Patch(ctx, updatedDataDownload, crclient.MergeFrom(dataDownload))\n}\n\nfunc newDataDownload(\n\trestore *velerov1api.Restore,\n\tbackup *velerov1api.Backup,\n\tdataUploadResult *velerov2alpha1.DataUploadResult,\n\tpvc *corev1api.PersistentVolumeClaim,\n\tnewNamespace, operationID string,\n) *velerov2alpha1.DataDownload {\n\tdataDownload := &velerov2alpha1.DataDownload{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: velerov2alpha1.SchemeGroupVersion.String(),\n\t\t\tKind: \"DataDownload\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: restore.Namespace,\n\t\t\tGenerateName: restore.Name + \"-\",\n\t\t\tOwnerReferences: []metav1.OwnerReference{\n\t\t\t\t{\n\t\t\t\t\tAPIVersion: velerov1api.SchemeGroupVersion.String(),\n\t\t\t\t\tKind: \"Restore\",\n\t\t\t\t\tName: restore.Name,\n\t\t\t\t\tUID: restore.UID,\n\t\t\t\t\tController: boolptr.True(),\n\t\t\t\t},\n\t\t\t},\n\t\t\tLabels: map[string]string{\n\t\t\t\tvelerov1api.RestoreNameLabel: label.GetValidName(restore.Name),\n\t\t\t\tvelerov1api.RestoreUIDLabel: string(restore.UID),\n\t\t\t\tvelerov1api.AsyncOperationIDLabel: operationID,\n\t\t\t},\n\t\t},\n\t\tSpec: velerov2alpha1.DataDownloadSpec{\n\t\t\tTargetVolume: velerov2alpha1.TargetVolumeSpec{\n\t\t\t\tPVC: pvc.Name,\n\t\t\t\tNamespace: newNamespace,\n\t\t\t},\n\t\t\tBackupStorageLocation: dataUploadResult.BackupStorageLocation,\n\t\t\tDataMover: dataUploadResult.DataMover,\n\t\t\tSnapshotID: dataUploadResult.SnapshotID,\n\t\t\tSnapshotSize: dataUploadResult.SnapshotSize,\n\t\t\tSourceNamespace: dataUploadResult.SourceNamespace,\n\t\t\tOperationTimeout: backup.Spec.CSISnapshotTimeout,\n\t\t\tNodeOS: dataUploadResult.NodeOS,\n\t\t},\n\t}\n\tif restore.Spec.UploaderConfig != nil {\n\t\tdataDownload.Spec.DataMoverConfig = uploaderUtil.StoreRestoreConfig(restore.Spec.UploaderConfig)\n\t}\n\treturn dataDownload\n}\n\nfunc restoreFromDataUploadResult(\n\tctx context.Context,\n\trestore *velerov1api.Restore,\n\tbackup *velerov1api.Backup,\n\tpvc *corev1api.PersistentVolumeClaim,\n\tnewNamespace, operationID string,\n\tcrClient crclient.Client,\n) (*velerov2alpha1.DataDownload, error) {\n\tdataUploadResult, err := getDataUploadResult(ctx, restore, pvc, crClient)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"fail get DataUploadResult for restore: %s\",\n\t\t\trestore.Name)\n\t}\n\tpvc.Spec.VolumeName = \"\"\n\tif pvc.Spec.Selector == nil {\n\t\tpvc.Spec.Selector = &metav1.LabelSelector{}\n\t}\n\tif pvc.Spec.Selector.MatchLabels == nil {\n\t\tpvc.Spec.Selector.MatchLabels = make(map[string]string)\n\t}\n\tpvc.Spec.Selector.MatchLabels[velerov1api.DynamicPVRestoreLabel] = label.\n\t\tGetValidName(fmt.Sprintf(\"%s.%s.%s\", newNamespace,\n\t\t\tpvc.Name, utilrand.String(GenerateNameRandomLength)))\n\n\tdataDownload := newDataDownload(\n\t\trestore,\n\t\tbackup,\n\t\tdataUploadResult,\n\t\tpvc,\n\t\tnewNamespace,\n\t\toperationID,\n\t)\n\terr = crClient.Create(ctx, dataDownload)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"fail to create DataDownload\")\n\t}\n\n\treturn dataDownload, nil\n}\n\nfunc (p *pvcRestoreItemAction) isResourceExist(\n\tpvc corev1api.PersistentVolumeClaim,\n\trestore velerov1api.Restore,\n) bool {\n\t// get target namespace to restore into, if different from source namespace\n\ttargetNamespace := pvc.Namespace\n\tif target, ok := restore.Spec.NamespaceMapping[pvc.Namespace]; ok {\n\t\ttargetNamespace = target\n\t}\n\n\ttmpPVC := new(corev1api.PersistentVolumeClaim)\n\tif err := p.crClient.Get(\n\t\tcontext.Background(),\n\t\tcrclient.ObjectKey{\n\t\t\tName: pvc.Name,\n\t\t\tNamespace: targetNamespace,\n\t\t},\n\t\ttmpPVC,\n\t); err == nil {\n\t\treturn true\n\t}\n\treturn false\n}\n\nfunc NewPvcRestoreItemAction(f client.Factory) plugincommon.HandlerInitializer {\n\treturn func(logger logrus.FieldLogger) (any, error) {\n\t\tcrClient, err := f.KubebuilderClient()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\treturn &pvcRestoreItemAction{\n\t\t\tlog: logger,\n\t\t\tcrClient: crClient,\n\t\t}, nil\n\t}\n}\n" }, { "path": "pkg/restore/actions/csi/pvc_action_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\t\"fmt\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/google/go-cmp/cmp\"\n\t\"github.com/google/go-cmp/cmp/cmpopts\"\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/api/resource\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/util/validation\"\n\tcrclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/apis/velero/shared\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n)\n\nfunc TestResetPVCSpec(t *testing.T) {\n\tfileMode := corev1api.PersistentVolumeFilesystem\n\tblockMode := corev1api.PersistentVolumeBlock\n\n\ttestCases := []struct {\n\t\tname string\n\t\tpvc corev1api.PersistentVolumeClaim\n\t\tvsName string\n\t}{\n\t\t{\n\t\t\tname: \"should reset expected fields in pvc using file mode volumes\",\n\t\t\tpvc: corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-pvc\",\n\t\t\t\t\tNamespace: \"test-ns\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\tAccessModes: []corev1api.PersistentVolumeAccessMode{corev1api.ReadOnlyMany, corev1api.ReadWriteMany, corev1api.ReadWriteOnce},\n\t\t\t\t\tSelector: &metav1.LabelSelector{\n\t\t\t\t\t\tMatchLabels: map[string]string{\n\t\t\t\t\t\t\t\"foo\": \"bar\",\n\t\t\t\t\t\t\t\"baz\": \"qux\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tResources: corev1api.VolumeResourceRequirements{\n\t\t\t\t\t\tRequests: corev1api.ResourceList{\n\t\t\t\t\t\t\tcorev1api.ResourceCPU: resource.Quantity{\n\t\t\t\t\t\t\t\tFormat: resource.DecimalExponent,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tVolumeName: \"should-be-removed\",\n\t\t\t\t\tVolumeMode: &fileMode,\n\t\t\t\t},\n\t\t\t},\n\t\t\tvsName: \"test-vs\",\n\t\t},\n\t\t{\n\t\t\tname: \"should reset expected fields in pvc using block mode volumes\",\n\t\t\tpvc: corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-pvc\",\n\t\t\t\t\tNamespace: \"test-ns\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\tAccessModes: []corev1api.PersistentVolumeAccessMode{corev1api.ReadOnlyMany, corev1api.ReadWriteMany, corev1api.ReadWriteOnce},\n\t\t\t\t\tSelector: &metav1.LabelSelector{\n\t\t\t\t\t\tMatchLabels: map[string]string{\n\t\t\t\t\t\t\t\"foo\": \"bar\",\n\t\t\t\t\t\t\t\"baz\": \"qux\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tResources: corev1api.VolumeResourceRequirements{\n\t\t\t\t\t\tRequests: corev1api.ResourceList{\n\t\t\t\t\t\t\tcorev1api.ResourceCPU: resource.Quantity{\n\t\t\t\t\t\t\t\tFormat: resource.DecimalExponent,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tVolumeName: \"should-be-removed\",\n\t\t\t\t\tVolumeMode: &blockMode,\n\t\t\t\t},\n\t\t\t},\n\t\t\tvsName: \"test-vs\",\n\t\t},\n\t\t{\n\t\t\tname: \"should overwrite existing DataSource per reset parameters\",\n\t\t\tpvc: corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-pvc\",\n\t\t\t\t\tNamespace: \"test-ns\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\tAccessModes: []corev1api.PersistentVolumeAccessMode{corev1api.ReadOnlyMany, corev1api.ReadWriteMany, corev1api.ReadWriteOnce},\n\t\t\t\t\tSelector: &metav1.LabelSelector{\n\t\t\t\t\t\tMatchLabels: map[string]string{\n\t\t\t\t\t\t\t\"foo\": \"bar\",\n\t\t\t\t\t\t\t\"baz\": \"qux\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tResources: corev1api.VolumeResourceRequirements{\n\t\t\t\t\t\tRequests: corev1api.ResourceList{\n\t\t\t\t\t\t\tcorev1api.ResourceCPU: resource.Quantity{\n\t\t\t\t\t\t\t\tFormat: resource.DecimalExponent,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tVolumeName: \"should-be-removed\",\n\t\t\t\t\tVolumeMode: &fileMode,\n\t\t\t\t\tDataSource: &corev1api.TypedLocalObjectReference{\n\t\t\t\t\t\tKind: \"something-that-does-not-exist\",\n\t\t\t\t\t\tName: \"not-found\",\n\t\t\t\t\t},\n\t\t\t\t\tDataSourceRef: &corev1api.TypedObjectReference{\n\t\t\t\t\t\tKind: \"something-that-does-not-exist\",\n\t\t\t\t\t\tName: \"not-found\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvsName: \"test-vs\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tbefore := tc.pvc.DeepCopy()\n\t\t\tresetPVCSourceToVolumeSnapshot(&tc.pvc, tc.vsName)\n\n\t\t\tassert.Equalf(t, tc.pvc.Name, before.Name, \"unexpected change to Object.Name, Want: %s; Got %s\", before.Name, tc.pvc.Name)\n\t\t\tassert.Equalf(t, tc.pvc.Namespace, before.Namespace, \"unexpected change to Object.Namespace, Want: %s; Got %s\", before.Namespace, tc.pvc.Namespace)\n\t\t\tassert.Equalf(t, tc.pvc.Spec.AccessModes, before.Spec.AccessModes, \"unexpected Spec.AccessModes, Want: %v; Got: %v\", before.Spec.AccessModes, tc.pvc.Spec.AccessModes)\n\t\t\tassert.Equalf(t, tc.pvc.Spec.Selector, before.Spec.Selector, \"unexpected change to Spec.Selector, Want: %s; Got: %s\", before.Spec.Selector.String(), tc.pvc.Spec.Selector.String())\n\t\t\tassert.Equalf(t, tc.pvc.Spec.Resources, before.Spec.Resources, \"unexpected change to Spec.Resources, Want: %s; Got: %s\", before.Spec.Resources.String(), tc.pvc.Spec.Resources.String())\n\t\t\tassert.Emptyf(t, tc.pvc.Spec.VolumeName, \"expected change to Spec.VolumeName missing, Want: \\\"\\\"; Got: %s\", tc.pvc.Spec.VolumeName)\n\t\t\tassert.Equalf(t, *tc.pvc.Spec.VolumeMode, *before.Spec.VolumeMode, \"expected change to Spec.VolumeName missing, Want: \\\"\\\"; Got: %s\", tc.pvc.Spec.VolumeName)\n\t\t\tassert.NotNil(t, tc.pvc.Spec.DataSource, \"expected change to Spec.DataSource missing\")\n\t\t\tassert.Equalf(t, \"VolumeSnapshot\", tc.pvc.Spec.DataSource.Kind, \"expected change to Spec.DataSource.Kind missing, Want: VolumeSnapshot, Got: %s\", tc.pvc.Spec.DataSource.Kind)\n\t\t\tassert.Equalf(t, tc.pvc.Spec.DataSource.Name, tc.vsName, \"expected change to Spec.DataSource.Name missing, Want: %s, Got: %s\", tc.vsName, tc.pvc.Spec.DataSource.Name)\n\t\t})\n\t}\n}\n\nfunc TestProgress(t *testing.T) {\n\tcurrentTime := time.Now()\n\ttests := []struct {\n\t\tname string\n\t\trestore *velerov1api.Restore\n\t\tdataDownload *velerov2alpha1.DataDownload\n\t\toperationID string\n\t\texpectedErr string\n\t\texpectedProgress velero.OperationProgress\n\t}{\n\t\t{\n\t\t\tname: \"DataDownload cannot be found\",\n\t\t\trestore: builder.ForRestore(\"velero\", \"test\").Result(),\n\t\t\toperationID: \"testing\",\n\t\t\texpectedErr: \"didn't find DataDownload\",\n\t\t},\n\t\t{\n\t\t\tname: \"DataDownload is not in the expected namespace\",\n\t\t\trestore: builder.ForRestore(\"velero\", \"test\").Result(),\n\t\t\tdataDownload: &velerov2alpha1.DataDownload{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"DataUpload\",\n\t\t\t\t\tAPIVersion: velerov2alpha1.SchemeGroupVersion.String(),\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"invalid-namespace\",\n\t\t\t\t\tName: \"testing\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\tvelerov1api.AsyncOperationIDLabel: \"testing\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\toperationID: \"testing\",\n\t\t\texpectedErr: \"didn't find DataDownload\",\n\t\t},\n\t\t{\n\t\t\tname: \"DataUpload is found\",\n\t\t\trestore: builder.ForRestore(\"velero\", \"test\").Result(),\n\t\t\tdataDownload: &velerov2alpha1.DataDownload{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"DataUpload\",\n\t\t\t\t\tAPIVersion: velerov2alpha1.SchemeGroupVersion.String(),\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\tName: \"testing\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\tvelerov1api.AsyncOperationIDLabel: \"testing\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStatus: velerov2alpha1.DataDownloadStatus{\n\t\t\t\t\tPhase: velerov2alpha1.DataDownloadPhaseFailed,\n\t\t\t\t\tProgress: shared.DataMoveOperationProgress{\n\t\t\t\t\t\tBytesDone: 1000,\n\t\t\t\t\t\tTotalBytes: 1000,\n\t\t\t\t\t},\n\t\t\t\t\tStartTimestamp: &metav1.Time{Time: currentTime},\n\t\t\t\t\tCompletionTimestamp: &metav1.Time{Time: currentTime},\n\t\t\t\t\tMessage: \"Testing error\",\n\t\t\t\t},\n\t\t\t},\n\t\t\toperationID: \"testing\",\n\t\t\texpectedProgress: velero.OperationProgress{\n\t\t\t\tCompleted: true,\n\t\t\t\tErr: \"Testing error\",\n\t\t\t\tNCompleted: 1000,\n\t\t\t\tNTotal: 1000,\n\t\t\t\tOperationUnits: \"Bytes\",\n\t\t\t\tDescription: \"Failed\",\n\t\t\t\tStarted: currentTime,\n\t\t\t\tUpdated: currentTime,\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(*testing.T) {\n\t\t\tpvcRIA := pvcRestoreItemAction{\n\t\t\t\tlog: logrus.New(),\n\t\t\t\tcrClient: velerotest.NewFakeControllerRuntimeClient(t),\n\t\t\t}\n\t\t\tif tc.dataDownload != nil {\n\t\t\t\terr := pvcRIA.crClient.Create(t.Context(), tc.dataDownload)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tprogress, err := pvcRIA.Progress(tc.operationID, tc.restore)\n\t\t\tif tc.expectedErr != \"\" {\n\t\t\t\trequire.Equal(t, tc.expectedErr, err.Error())\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\trequire.NoError(t, err)\n\t\t\trequire.True(t, cmp.Equal(tc.expectedProgress, progress, cmpopts.IgnoreFields(velero.OperationProgress{}, \"Started\", \"Updated\")))\n\t\t})\n\t}\n}\n\nfunc TestCancel(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\trestore *velerov1api.Restore\n\t\tdataDownload *velerov2alpha1.DataDownload\n\t\toperationID string\n\t\texpectedErr string\n\t\texpectedDataDownload velerov2alpha1.DataDownload\n\t}{\n\t\t{\n\t\t\tname: \"Cancel DataUpload\",\n\t\t\trestore: builder.ForRestore(\"velero\", \"test\").Result(),\n\t\t\tdataDownload: &velerov2alpha1.DataDownload{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"DataDownload\",\n\t\t\t\t\tAPIVersion: velerov2alpha1.SchemeGroupVersion.String(),\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\tName: \"testing\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\tvelerov1api.AsyncOperationIDLabel: \"testing\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\toperationID: \"testing\",\n\t\t\texpectedErr: \"\",\n\t\t\texpectedDataDownload: velerov2alpha1.DataDownload{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"DataDownload\",\n\t\t\t\t\tAPIVersion: velerov2alpha1.SchemeGroupVersion.String(),\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\tName: \"testing\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\tvelerov1api.AsyncOperationIDLabel: \"testing\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov2alpha1.DataDownloadSpec{\n\t\t\t\t\tCancel: true,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Cannot find DataUpload\",\n\t\t\trestore: builder.ForRestore(\"velero\", \"test\").Result(),\n\t\t\tdataDownload: nil,\n\t\t\toperationID: \"testing\",\n\t\t\texpectedErr: \"didn't find DataDownload\",\n\t\t\texpectedDataDownload: velerov2alpha1.DataDownload{\n\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\tKind: \"DataDownload\",\n\t\t\t\t\tAPIVersion: velerov2alpha1.SchemeGroupVersion.String(),\n\t\t\t\t},\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\tName: \"testing\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\tvelerov1api.AsyncOperationIDLabel: \"testing\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: velerov2alpha1.DataDownloadSpec{\n\t\t\t\t\tCancel: true,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(*testing.T) {\n\t\t\tpvcRIA := pvcRestoreItemAction{\n\t\t\t\tlog: logrus.New(),\n\t\t\t\tcrClient: velerotest.NewFakeControllerRuntimeClient(t),\n\t\t\t}\n\t\t\tif tc.dataDownload != nil {\n\t\t\t\terr := pvcRIA.crClient.Create(t.Context(), tc.dataDownload)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\terr := pvcRIA.Cancel(tc.operationID, tc.restore)\n\t\t\tif tc.expectedErr != \"\" {\n\t\t\t\trequire.Equal(t, tc.expectedErr, err.Error())\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NoError(t, err)\n\n\t\t\tresultDataDownload := new(velerov2alpha1.DataDownload)\n\t\t\terr = pvcRIA.crClient.Get(t.Context(), crclient.ObjectKey{Namespace: tc.dataDownload.Namespace, Name: tc.dataDownload.Name}, resultDataDownload)\n\t\t\trequire.NoError(t, err)\n\n\t\t\trequire.True(t, cmp.Equal(tc.expectedDataDownload, *resultDataDownload, cmpopts.IgnoreFields(velerov2alpha1.DataDownload{}, \"ResourceVersion\", \"Name\")))\n\t\t})\n\t}\n}\n\nfunc TestExecute(t *testing.T) {\n\tvsName := util.GenerateSha256FromRestoreUIDAndVsName(\"restoreUID\", \"vsName\")\n\ttests := []struct {\n\t\tname string\n\t\tbackup *velerov1api.Backup\n\t\trestore *velerov1api.Restore\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\tvs *snapshotv1api.VolumeSnapshot\n\t\tdataUploadResult *corev1api.ConfigMap\n\t\texpectedErr string\n\t\texpectedDataDownload *velerov2alpha1.DataDownload\n\t\texpectedPVC *corev1api.PersistentVolumeClaim\n\t\tpreCreatePVC bool\n\t}{\n\t\t{\n\t\t\tname: \"Don't restore PV\",\n\t\t\trestore: builder.ForRestore(\"velero\", \"testRestore\").Backup(\"testBackup\").RestorePVs(false).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").Result(),\n\t\t\texpectedPVC: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").VolumeName(\"\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"restore's backup cannot be found\",\n\t\t\trestore: builder.ForRestore(\"velero\", \"testRestore\").Backup(\"testBackup\").Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").Result(),\n\t\t\texpectedErr: \"fail to get backup for restore: backups.velero.io \\\"testBackup\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"Restore from VolumeSnapshot\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"testBackup\").Result(),\n\t\t\trestore: builder.ForRestore(\"velero\", \"testRestore\").ObjectMeta(builder.WithUID(\"restoreUID\")).Backup(\"testBackup\").Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").ObjectMeta(builder.WithAnnotations(velerov1api.VolumeSnapshotLabel, \"vsName\")).\n\t\t\t\tRequestResource(map[corev1api.ResourceName]resource.Quantity{corev1api.ResourceStorage: resource.MustParse(\"10Gi\")}).\n\t\t\t\tDataSource(&corev1api.TypedLocalObjectReference{APIGroup: &snapshotv1api.SchemeGroupVersion.Group, Kind: \"VolumeSnapshot\", Name: \"testVS\"}).\n\t\t\t\tDataSourceRef(&corev1api.TypedObjectReference{APIGroup: &snapshotv1api.SchemeGroupVersion.Group, Kind: \"VolumeSnapshot\", Name: \"testVS\"}).\n\t\t\t\tResult(),\n\t\t\tvs: builder.ForVolumeSnapshot(\"velero\", vsName).ObjectMeta(\n\t\t\t\tbuilder.WithAnnotations(velerov1api.VolumeSnapshotRestoreSize, \"10Gi\"),\n\t\t\t).Result(),\n\t\t\texpectedPVC: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").ObjectMeta(builder.WithAnnotations(velerov1api.VolumeSnapshotLabel, \"vsName\")).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Restore from VolumeSnapshot without volume-snapshot-name annotation\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"testBackup\").Result(),\n\t\t\trestore: builder.ForRestore(\"velero\", \"testRestore\").Backup(\"testBackup\").Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").ObjectMeta(builder.WithAnnotations(AnnSelectedNode, \"node1\")).Result(),\n\t\t\tvs: builder.ForVolumeSnapshot(\"velero\", \"testVS\").ObjectMeta(builder.WithAnnotations(velerov1api.VolumeSnapshotRestoreSize, \"10Gi\")).Result(),\n\t\t\texpectedPVC: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").ObjectMeta(builder.WithAnnotations(AnnSelectedNode, \"node1\")).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"DataUploadResult cannot be found\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"testBackup\").SnapshotMoveData(true).Result(),\n\t\t\trestore: builder.ForRestore(\"velero\", \"testRestore\").Backup(\"testBackup\").Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").ObjectMeta(builder.WithAnnotations(velerov1api.VolumeSnapshotRestoreSize, \"10Gi\", velerov1api.DataUploadNameAnnotation, \"velero/\")).Result(),\n\t\t\texpectedPVC: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").Result(),\n\t\t\texpectedErr: \"fail get DataUploadResult for restore: testRestore: no DataUpload result cm found with labels velero.io/pvc-namespace-name=velero.testPVC,velero.io/restore-uid=,velero.io/resource-usage=DataUpload\",\n\t\t},\n\t\t{\n\t\t\tname: \"Restore from DataUploadResult\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"testBackup\").SnapshotMoveData(true).Result(),\n\t\t\trestore: builder.ForRestore(\"velero\", \"testRestore\").Backup(\"testBackup\").ObjectMeta(builder.WithUID(\"uid\")).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").ObjectMeta(builder.WithAnnotations(velerov1api.VolumeSnapshotRestoreSize, \"10Gi\", velerov1api.DataUploadNameAnnotation, \"velero/\")).Result(),\n\t\t\tdataUploadResult: builder.ForConfigMap(\"velero\", \"testCM\").Data(\"uid\", \"{}\").ObjectMeta(builder.WithLabels(velerov1api.RestoreUIDLabel, \"uid\", velerov1api.PVCNamespaceNameLabel, \"velero.testPVC\", velerov1api.ResourceUsageLabel, label.GetValidName(string(velerov1api.VeleroResourceUsageDataUploadResult)))).Result(),\n\t\t\texpectedPVC: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").ObjectMeta(builder.WithAnnotations(\"velero.io/csi-volumesnapshot-restore-size\", \"10Gi\", velerov1api.DataUploadNameAnnotation, \"velero/\")).Result(),\n\t\t\texpectedDataDownload: builder.ForDataDownload(\"velero\", \"name\").TargetVolume(velerov2alpha1.TargetVolumeSpec{PVC: \"testPVC\", Namespace: \"velero\"}).\n\t\t\t\tObjectMeta(builder.WithOwnerReference([]metav1.OwnerReference{{APIVersion: velerov1api.SchemeGroupVersion.String(), Kind: \"Restore\", Name: \"testRestore\", UID: \"uid\", Controller: boolptr.True()}}),\n\t\t\t\t\tbuilder.WithLabelsMap(map[string]string{velerov1api.AsyncOperationIDLabel: \"dd-uid.\", velerov1api.RestoreNameLabel: \"testRestore\", velerov1api.RestoreUIDLabel: \"uid\"}),\n\t\t\t\t\tbuilder.WithGenerateName(\"testRestore-\")).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Restore from DataUploadResult with long source PVC namespace and name\",\n\t\t\tbackup: builder.ForBackup(\"migre209d0da-49c7-45ba-8d5a-3e59fd591ec1\", \"testBackup\").SnapshotMoveData(true).Result(),\n\t\t\trestore: builder.ForRestore(\"migre209d0da-49c7-45ba-8d5a-3e59fd591ec1\", \"testRestore\").Backup(\"testBackup\").ObjectMeta(builder.WithUID(\"uid\")).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"migre209d0da-49c7-45ba-8d5a-3e59fd591ec1\", \"kibishii-data-kibishii-deployment-0\").ObjectMeta(builder.WithAnnotations(velerov1api.VolumeSnapshotRestoreSize, \"10Gi\", velerov1api.DataUploadNameAnnotation, \"velero/\")).Result(),\n\t\t\tdataUploadResult: builder.ForConfigMap(\"migre209d0da-49c7-45ba-8d5a-3e59fd591ec1\", \"testCM\").Data(\"uid\", \"{}\").ObjectMeta(builder.WithLabels(velerov1api.RestoreUIDLabel, \"uid\", velerov1api.PVCNamespaceNameLabel, \"migre209d0da-49c7-45ba-8d5a-3e59fd591ec1.kibishii-data-ki152333\", velerov1api.ResourceUsageLabel, label.GetValidName(string(velerov1api.VeleroResourceUsageDataUploadResult)))).Result(),\n\t\t\texpectedPVC: builder.ForPersistentVolumeClaim(\"migre209d0da-49c7-45ba-8d5a-3e59fd591ec1\", \"kibishii-data-kibishii-deployment-0\").ObjectMeta(builder.WithAnnotations(\"velero.io/csi-volumesnapshot-restore-size\", \"10Gi\", velerov1api.DataUploadNameAnnotation, \"velero/\")).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"PVC had no DataUploadNameLabel annotation\",\n\t\t\tbackup: builder.ForBackup(\"migre209d0da-49c7-45ba-8d5a-3e59fd591ec1\", \"testBackup\").SnapshotMoveData(true).Result(),\n\t\t\trestore: builder.ForRestore(\"migre209d0da-49c7-45ba-8d5a-3e59fd591ec1\", \"testRestore\").Backup(\"testBackup\").ObjectMeta(builder.WithUID(\"uid\")).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"migre209d0da-49c7-45ba-8d5a-3e59fd591ec1\", \"kibishii-data-kibishii-deployment-0\").ObjectMeta(builder.WithAnnotations(velerov1api.VolumeSnapshotRestoreSize, \"10Gi\")).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Restore a PVC that already exists.\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"testBackup\").SnapshotMoveData(true).Result(),\n\t\t\trestore: builder.ForRestore(\"velero\", \"testRestore\").Backup(\"testBackup\").ObjectMeta(builder.WithUID(\"uid\")).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"velero\", \"testPVC\").ObjectMeta(builder.WithAnnotations(velerov1api.VolumeSnapshotRestoreSize, \"10Gi\", velerov1api.DataUploadNameAnnotation, \"velero/\")).Result(),\n\t\t\tpreCreatePVC: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Restore a PVC that already exists in the mapping namespace\",\n\t\t\tbackup: builder.ForBackup(\"velero\", \"testBackup\").SnapshotMoveData(true).Result(),\n\t\t\trestore: builder.ForRestore(\"velero\", \"testRestore\").Backup(\"testBackup\").NamespaceMappings(\"velero\", \"restore\").ObjectMeta(builder.WithUID(\"uid\")).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"restore\", \"testPVC\").ObjectMeta(builder.WithAnnotations(velerov1api.VolumeSnapshotRestoreSize, \"10Gi\", velerov1api.DataUploadNameAnnotation, \"velero/\")).Result(),\n\t\t\tpreCreatePVC: true,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(*testing.T) {\n\t\t\tobject := make([]runtime.Object, 0)\n\t\t\tif tc.backup != nil {\n\t\t\t\tobject = append(object, tc.backup)\n\t\t\t}\n\n\t\t\tif tc.vs != nil {\n\t\t\t\tobject = append(object, tc.vs)\n\t\t\t}\n\n\t\t\tinput := new(velero.RestoreItemActionExecuteInput)\n\n\t\t\tif tc.pvc != nil {\n\t\t\t\tpvcMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.pvc)\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\tinput.Item = &unstructured.Unstructured{Object: pvcMap}\n\t\t\t\tinput.ItemFromBackup = &unstructured.Unstructured{Object: pvcMap}\n\t\t\t\tinput.Restore = tc.restore\n\t\t\t}\n\t\t\tif tc.preCreatePVC {\n\t\t\t\tobject = append(object, tc.pvc)\n\t\t\t}\n\n\t\t\tif tc.dataUploadResult != nil {\n\t\t\t\tobject = append(object, tc.dataUploadResult)\n\t\t\t}\n\n\t\t\tpvcRIA := pvcRestoreItemAction{\n\t\t\t\tlog: logrus.New(),\n\t\t\t\tcrClient: velerotest.NewFakeControllerRuntimeClient(t, object...),\n\t\t\t}\n\n\t\t\toutput, err := pvcRIA.Execute(input)\n\t\t\tif tc.expectedErr != \"\" {\n\t\t\t\trequire.Equal(t, tc.expectedErr, err.Error())\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NoError(t, err)\n\n\t\t\tif tc.expectedPVC != nil {\n\t\t\t\tpvc := new(corev1api.PersistentVolumeClaim)\n\t\t\t\terr := runtime.DefaultUnstructuredConverter.FromUnstructured(output.UpdatedItem.UnstructuredContent(), pvc)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\trequire.Equal(t, tc.expectedPVC.GetObjectMeta(), pvc.GetObjectMeta())\n\t\t\t\tif pvc.Spec.Selector != nil && pvc.Spec.Selector.MatchLabels != nil {\n\t\t\t\t\t// This is used for long name and namespace case.\n\t\t\t\t\tif len(tc.pvc.Namespace+\".\"+tc.pvc.Name) >= validation.DNS1035LabelMaxLength {\n\t\t\t\t\t\trequire.Contains(t, pvc.Spec.Selector.MatchLabels[velerov1api.DynamicPVRestoreLabel], label.GetValidName(tc.pvc.Namespace + \".\" + tc.pvc.Name)[:56])\n\t\t\t\t\t} else {\n\t\t\t\t\t\trequire.Contains(t, pvc.Spec.Selector.MatchLabels[velerov1api.DynamicPVRestoreLabel], tc.pvc.Namespace+\".\"+tc.pvc.Name)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\tif tc.expectedDataDownload != nil {\n\t\t\t\tdataDownloadList := new(velerov2alpha1.DataDownloadList)\n\t\t\t\terr := pvcRIA.crClient.List(t.Context(), dataDownloadList, &crclient.ListOptions{\n\t\t\t\t\tLabelSelector: labels.SelectorFromSet(tc.expectedDataDownload.Labels),\n\t\t\t\t})\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\trequire.True(t, cmp.Equal(tc.expectedDataDownload, &dataDownloadList.Items[0], cmpopts.IgnoreFields(velerov2alpha1.DataDownload{}, \"ResourceVersion\", \"Name\")))\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestPVCAppliesTo(t *testing.T) {\n\tp := pvcRestoreItemAction{\n\t\tlog: logrus.StandardLogger(),\n\t}\n\tselector, err := p.AppliesTo()\n\n\trequire.NoError(t, err)\n\n\trequire.Equal(\n\t\tt,\n\t\tvelero.ResourceSelector{\n\t\t\tIncludedResources: []string{\"persistentvolumeclaims\"},\n\t\t},\n\t\tselector,\n\t)\n}\n\nfunc TestNewPvcRestoreItemAction(t *testing.T) {\n\tlogger := logrus.StandardLogger()\n\tcrClient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\tf := &factorymocks.Factory{}\n\tf.On(\"KubebuilderClient\").Return(nil, fmt.Errorf(\"\"))\n\tplugin := NewPvcRestoreItemAction(f)\n\t_, err := plugin(logger)\n\trequire.Error(t, err)\n\n\tf1 := &factorymocks.Factory{}\n\tf1.On(\"KubebuilderClient\").Return(crClient, nil)\n\tplugin1 := NewPvcRestoreItemAction(f1)\n\t_, err1 := plugin1(logger)\n\trequire.NoError(t, err1)\n}\n" }, { "path": "pkg/restore/actions/csi/volumesnapshot_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\t\"fmt\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\tcrclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\tplugincommon \"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n)\n\n// volumeSnapshotRestoreItemAction is a Velero restore item\n// action plugin for VolumeSnapshots\ntype volumeSnapshotRestoreItemAction struct {\n\tlog logrus.FieldLogger\n\tcrClient crclient.Client\n}\n\n// AppliesTo returns information indicating that\n// VolumeSnapshotRestoreItemAction should be invoked while\n// restoring volumesnapshots.snapshot.storage.k8s.io resources.\nfunc (p *volumeSnapshotRestoreItemAction) AppliesTo() (\n\tvelero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"volumesnapshots.snapshot.storage.k8s.io\"},\n\t}, nil\n}\n\nfunc resetVolumeSnapshotSpecForRestore(vs *snapshotv1api.VolumeSnapshot, vscName *string) {\n\t// Spec of the backed-up object used the PVC as the source\n\t// of the volumeSnapshot. Restore operation will however,\n\t// restore the VolumeSnapshot from the VolumeSnapshotContent\n\tvs.Spec.Source.PersistentVolumeClaimName = nil\n\tvs.Spec.Source.VolumeSnapshotContentName = vscName\n}\n\nfunc resetVolumeSnapshotAnnotation(vs *snapshotv1api.VolumeSnapshot) {\n\tvs.ObjectMeta.Annotations[velerov1api.VSCDeletionPolicyAnnotation] =\n\t\tstring(snapshotv1api.VolumeSnapshotContentRetain)\n}\n\nfunc (p *volumeSnapshotRestoreItemAction) Execute(\n\tinput *velero.RestoreItemActionExecuteInput,\n) (*velero.RestoreItemActionExecuteOutput, error) {\n\tp.log.Info(\"Starting VolumeSnapshotRestoreItemAction\")\n\n\tif boolptr.IsSetToFalse(input.Restore.Spec.RestorePVs) {\n\t\tp.log.Infof(\"Restore %s/%s did not request for PVs to be restored.\",\n\t\t\tinput.Restore.Namespace, input.Restore.Name)\n\t\treturn &velero.RestoreItemActionExecuteOutput{SkipRestore: true}, nil\n\t}\n\n\tvar vs snapshotv1api.VolumeSnapshot\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(\n\t\tinput.Item.UnstructuredContent(), &vs); err != nil {\n\t\treturn &velero.RestoreItemActionExecuteOutput{},\n\t\t\terrors.Wrapf(err, \"failed to convert input.Item from unstructured\")\n\t}\n\n\tvar vsFromBackup snapshotv1api.VolumeSnapshot\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(\n\t\tinput.ItemFromBackup.UnstructuredContent(), &vsFromBackup); err != nil {\n\t\treturn &velero.RestoreItemActionExecuteOutput{},\n\t\t\terrors.Wrapf(err, \"failed to convert input.Item from unstructured\")\n\t}\n\n\tgeneratedName := util.GenerateSha256FromRestoreUIDAndVsName(string(input.Restore.UID), vsFromBackup.Name)\n\n\t// Reset Spec to convert the VolumeSnapshot from using\n\t// the dynamic VolumeSnapshotContent to the static one.\n\tresetVolumeSnapshotSpecForRestore(&vs, &generatedName)\n\t// Also reset the VS name to avoid potential conflict caused by multiple restores of the same backup.\n\t// Both VS and VSC share the same generated name.\n\tvs.Name = generatedName\n\n\t// Reset VolumeSnapshot annotation. By now, only change\n\t// DeletionPolicy to Retain.\n\tresetVolumeSnapshotAnnotation(&vs)\n\n\tif vs.Spec.VolumeSnapshotClassName != nil {\n\t\t// Delete VolumeSnapshotClass from the VolumeSnapshot.\n\t\t// This is necessary to make the restore independent of the VolumeSnapshotClass.\n\t\tvs.Spec.VolumeSnapshotClassName = nil\n\t\tp.log.Debugf(\"Deleted VolumeSnapshotClassName from VolumeSnapshot %s/%s to make restore independent of VolumeSnapshotClass\",\n\t\t\tvs.Namespace, vs.Name)\n\t}\n\n\tvsMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&vs)\n\tif err != nil {\n\t\tp.log.Errorf(\"Fail to convert VS %s to unstructured\", vs.Namespace+\"/\"+vs.Name)\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tif vsFromBackup.Status == nil ||\n\t\tvsFromBackup.Status.BoundVolumeSnapshotContentName == nil {\n\t\tp.log.Errorf(\"VS %s doesn't have bound VSC\", vsFromBackup.Name)\n\t\treturn nil, fmt.Errorf(\"VS %s doesn't have bound VSC\", vsFromBackup.Name)\n\t}\n\n\tvsc := velero.ResourceIdentifier{\n\t\tGroupResource: kuberesource.VolumeSnapshotContents,\n\t\tName: *vsFromBackup.Status.BoundVolumeSnapshotContentName,\n\t}\n\n\tp.log.Infof(`Returning from VolumeSnapshotRestoreItemAction with \n\t\tVolumeSnapshotContent in additionalItems`)\n\n\treturn &velero.RestoreItemActionExecuteOutput{\n\t\tUpdatedItem: &unstructured.Unstructured{Object: vsMap},\n\t\tAdditionalItems: []velero.ResourceIdentifier{vsc},\n\t}, nil\n}\n\nfunc (p *volumeSnapshotRestoreItemAction) Name() string {\n\treturn \"VolumeSnapshotRestoreItemAction\"\n}\n\nfunc (p *volumeSnapshotRestoreItemAction) Progress(\n\toperationID string,\n\trestore *velerov1api.Restore,\n) (velero.OperationProgress, error) {\n\treturn velero.OperationProgress{}, nil\n}\n\nfunc (p *volumeSnapshotRestoreItemAction) Cancel(\n\toperationID string,\n\trestore *velerov1api.Restore,\n) error {\n\t// CSI Specification doesn't support canceling a snapshot creation.\n\treturn nil\n}\n\nfunc (p *volumeSnapshotRestoreItemAction) AreAdditionalItemsReady(\n\tadditionalItems []velero.ResourceIdentifier,\n\trestore *velerov1api.Restore,\n) (bool, error) {\n\treturn true, nil\n}\n\nfunc NewVolumeSnapshotRestoreItemAction(\n\tf client.Factory,\n) plugincommon.HandlerInitializer {\n\treturn func(logger logrus.FieldLogger) (any, error) {\n\t\tcrClient, err := f.KubebuilderClient()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\treturn &volumeSnapshotRestoreItemAction{logger, crClient}, nil\n\t}\n}\n" }, { "path": "pkg/restore/actions/csi/volumesnapshot_action_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\t\"fmt\"\n\t\"testing\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n)\n\nvar (\n\ttestPVC = \"test-pvc\"\n\ttestSnapClass = \"snap-class\"\n\trandText = \"DEADFEED\"\n)\n\nfunc TestResetVolumeSnapshotSpecForRestore(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tvs snapshotv1api.VolumeSnapshot\n\t\tvscName string\n\t}{\n\t\t{\n\t\t\tname: \"should reset spec as expected\",\n\t\t\tvs: snapshotv1api.VolumeSnapshot{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-vs\",\n\t\t\t\t\tNamespace: \"test-ns\",\n\t\t\t\t},\n\t\t\t\tSpec: snapshotv1api.VolumeSnapshotSpec{\n\t\t\t\t\tSource: snapshotv1api.VolumeSnapshotSource{\n\t\t\t\t\t\tPersistentVolumeClaimName: &testPVC,\n\t\t\t\t\t},\n\t\t\t\t\tVolumeSnapshotClassName: &testSnapClass,\n\t\t\t\t},\n\t\t\t},\n\t\t\tvscName: \"test-vsc\",\n\t\t},\n\t\t{\n\t\t\tname: \"should reset spec and overwriting value for Source.VolumeSnapshotContentName\",\n\t\t\tvs: snapshotv1api.VolumeSnapshot{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test-vs\",\n\t\t\t\t\tNamespace: \"test-ns\",\n\t\t\t\t},\n\t\t\t\tSpec: snapshotv1api.VolumeSnapshotSpec{\n\t\t\t\t\tSource: snapshotv1api.VolumeSnapshotSource{\n\t\t\t\t\t\tVolumeSnapshotContentName: &randText,\n\t\t\t\t\t},\n\t\t\t\t\tVolumeSnapshotClassName: &testSnapClass,\n\t\t\t\t},\n\t\t\t},\n\t\t\tvscName: \"test-vsc\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tbefore := tc.vs.DeepCopy()\n\t\t\tresetVolumeSnapshotSpecForRestore(&tc.vs, &tc.vscName)\n\n\t\t\tassert.Equalf(t, tc.vs.Name, before.Name, \"unexpected change to Object.Name, Want: %s; Got %s\", before.Name, tc.vs.Name)\n\t\t\tassert.Equalf(t, tc.vs.Namespace, before.Namespace, \"unexpected change to Object.Namespace, Want: %s; Got %s\", before.Namespace, tc.vs.Namespace)\n\t\t\tassert.NotNil(t, tc.vs.Spec.Source)\n\t\t\tassert.Nil(t, tc.vs.Spec.Source.PersistentVolumeClaimName)\n\t\t\tassert.NotNil(t, tc.vs.Spec.Source.VolumeSnapshotContentName)\n\t\t\tassert.Equal(t, *tc.vs.Spec.Source.VolumeSnapshotContentName, tc.vscName)\n\t\t\tassert.Equalf(t, *tc.vs.Spec.VolumeSnapshotClassName, *before.Spec.VolumeSnapshotClassName, \"unexpected value for Spec.VolumeSnapshotClassName, Want: %s, Got: %s\",\n\t\t\t\t*tc.vs.Spec.VolumeSnapshotClassName, *before.Spec.VolumeSnapshotClassName)\n\t\t\tassert.Nil(t, tc.vs.Status)\n\t\t})\n\t}\n}\n\nfunc TestVSExecute(t *testing.T) {\n\tnewVscName := util.GenerateSha256FromRestoreUIDAndVsName(\"restoreUID\", \"vsName\")\n\ttests := []struct {\n\t\tname string\n\t\titem runtime.Unstructured\n\t\tvs *snapshotv1api.VolumeSnapshot\n\t\trestore *velerov1api.Restore\n\t\texpectErr bool\n\t\tcreateVS bool\n\t\texpectedVS *snapshotv1api.VolumeSnapshot\n\t}{\n\t\t{\n\t\t\tname: \"Restore's RestorePVs is false\",\n\t\t\trestore: builder.ForRestore(\"velero\", \"restore\").RestorePVs(false).Result(),\n\t\t\texpectErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"VS doesn't have VSC in status\",\n\t\t\tvs: builder.ForVolumeSnapshot(\"ns\", \"name\").ObjectMeta(builder.WithAnnotations(\"1\", \"1\")).Status().Result(),\n\t\t\trestore: builder.ForRestore(\"velero\", \"restore\").NamespaceMappings(\"ns\", \"newNS\").Result(),\n\t\t\texpectErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Normal case, VSC should be created\",\n\t\t\tvs: builder.ForVolumeSnapshot(\"ns\", \"vsName\").\n\t\t\t\tObjectMeta(\n\t\t\t\t\tbuilder.WithAnnotationsMap(\n\t\t\t\t\t\tmap[string]string{\n\t\t\t\t\t\t\tvelerov1api.VolumeSnapshotHandleAnnotation: \"vsc\",\n\t\t\t\t\t\t\tvelerov1api.DriverNameAnnotation: \"pd.csi.storage.gke.io\",\n\t\t\t\t\t\t},\n\t\t\t\t\t),\n\t\t\t\t).\n\t\t\t\tSourceVolumeSnapshotContentName(newVscName).\n\t\t\t\tVolumeSnapshotClass(\"vscClass\").\n\t\t\t\tStatus().\n\t\t\t\tBoundVolumeSnapshotContentName(\"vscName\").\n\t\t\t\tResult(),\n\t\t\trestore: builder.ForRestore(\"velero\", \"restore\").ObjectMeta(builder.WithUID(\"restoreUID\")).Result(),\n\t\t\texpectErr: false,\n\t\t\texpectedVS: builder.ForVolumeSnapshot(\"ns\", \"test\").SourceVolumeSnapshotContentName(newVscName).Result(),\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tp := volumeSnapshotRestoreItemAction{\n\t\t\t\tlog: logrus.StandardLogger(),\n\t\t\t\tcrClient: velerotest.NewFakeControllerRuntimeClient(t),\n\t\t\t}\n\n\t\t\tif test.vs != nil {\n\t\t\t\tvsMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(test.vs)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\ttest.item = &unstructured.Unstructured{Object: vsMap}\n\n\t\t\t\tif test.createVS == true {\n\t\t\t\t\tif newNS, ok := test.restore.Spec.NamespaceMapping[test.vs.Namespace]; ok {\n\t\t\t\t\t\ttest.vs.SetNamespace(newNS)\n\t\t\t\t\t}\n\t\t\t\t\trequire.NoError(t, p.crClient.Create(t.Context(), test.vs))\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tresult, err := p.Execute(\n\t\t\t\t&velero.RestoreItemActionExecuteInput{\n\t\t\t\t\tItem: test.item,\n\t\t\t\t\tItemFromBackup: test.item,\n\t\t\t\t\tRestore: test.restore,\n\t\t\t\t},\n\t\t\t)\n\n\t\t\tif test.expectErr == false {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif test.expectedVS != nil {\n\t\t\t\tvar vs snapshotv1api.VolumeSnapshot\n\t\t\t\trequire.NoError(t, runtime.DefaultUnstructuredConverter.FromUnstructured(\n\t\t\t\t\tresult.UpdatedItem.UnstructuredContent(), &vs))\n\t\t\t\trequire.Equal(t, test.expectedVS.Spec, vs.Spec)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestVSAppliesTo(t *testing.T) {\n\tp := volumeSnapshotRestoreItemAction{\n\t\tlog: logrus.StandardLogger(),\n\t}\n\tselector, err := p.AppliesTo()\n\n\trequire.NoError(t, err)\n\n\trequire.Equal(\n\t\tt,\n\t\tvelero.ResourceSelector{\n\t\t\tIncludedResources: []string{\"volumesnapshots.snapshot.storage.k8s.io\"},\n\t\t},\n\t\tselector,\n\t)\n}\n\nfunc TestNewVolumeSnapshotRestoreItemAction(t *testing.T) {\n\tlogger := logrus.StandardLogger()\n\tcrClient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\tf := &factorymocks.Factory{}\n\tf.On(\"KubebuilderClient\").Return(nil, fmt.Errorf(\"\"))\n\tplugin := NewVolumeSnapshotRestoreItemAction(f)\n\t_, err := plugin(logger)\n\trequire.Error(t, err)\n\n\tf1 := &factorymocks.Factory{}\n\tf1.On(\"KubebuilderClient\").Return(crClient, nil)\n\tplugin1 := NewVolumeSnapshotRestoreItemAction(f1)\n\t_, err1 := plugin1(logger)\n\trequire.NoError(t, err1)\n}\n" }, { "path": "pkg/restore/actions/csi/volumesnapshotclass_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/csi\"\n)\n\n// volumeSnapshotClassRestoreItemAction is a Velero restore\n// item action plugin for VolumeSnapshotClass\ntype volumeSnapshotClassRestoreItemAction struct {\n\tlog logrus.FieldLogger\n}\n\n// AppliesTo returns information indicating that VolumeSnapshotClassRestoreItemAction\n// should be invoked while restoring volumesnapshotclass.snapshot.storage.k8s.io resources.\nfunc (p *volumeSnapshotClassRestoreItemAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"volumesnapshotclasses.snapshot.storage.k8s.io\"},\n\t}, nil\n}\n\n// Execute restores VolumeSnapshotClass objects returning any\n// snapshotlister secret as additional items to restore\nfunc (p *volumeSnapshotClassRestoreItemAction) Execute(\n\tinput *velero.RestoreItemActionExecuteInput,\n) (*velero.RestoreItemActionExecuteOutput, error) {\n\tp.log.Info(\"Starting VolumeSnapshotClassRestoreItemAction\")\n\tif boolptr.IsSetToFalse(input.Restore.Spec.RestorePVs) {\n\t\tp.log.Infof(\"Restore did not request for PVs to be restored %s/%s\",\n\t\t\tinput.Restore.Namespace, input.Restore.Name)\n\t\treturn &velero.RestoreItemActionExecuteOutput{SkipRestore: true}, nil\n\t}\n\tvar snapClass snapshotv1api.VolumeSnapshotClass\n\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(\n\t\tinput.Item.UnstructuredContent(), &snapClass); err != nil {\n\t\treturn &velero.RestoreItemActionExecuteOutput{}, errors.Wrapf(err,\n\t\t\t\"failed to convert input.Item from unstructured\")\n\t}\n\n\tadditionalItems := []velero.ResourceIdentifier{}\n\tif csi.IsVolumeSnapshotClassHasListerSecret(&snapClass) {\n\t\tadditionalItems = append(additionalItems, velero.ResourceIdentifier{\n\t\t\tGroupResource: schema.GroupResource{Group: \"\", Resource: \"secrets\"},\n\t\t\tName: snapClass.Annotations[velerov1api.PrefixedListSecretNameAnnotation],\n\t\t\tNamespace: snapClass.Annotations[velerov1api.PrefixedListSecretNamespaceAnnotation],\n\t\t})\n\t}\n\n\tp.log.Infof(\"Returning from VolumeSnapshotClassRestoreItemAction with %d additionalItems\",\n\t\tlen(additionalItems))\n\n\treturn &velero.RestoreItemActionExecuteOutput{\n\t\tUpdatedItem: input.Item,\n\t\tAdditionalItems: additionalItems,\n\t}, nil\n}\n\nfunc (p *volumeSnapshotClassRestoreItemAction) Name() string {\n\treturn \"VolumeSnapshotClassRestoreItemAction\"\n}\n\nfunc (p *volumeSnapshotClassRestoreItemAction) Progress(\n\toperationID string,\n\trestore *velerov1api.Restore,\n) (velero.OperationProgress, error) {\n\treturn velero.OperationProgress{}, nil\n}\n\nfunc (p *volumeSnapshotClassRestoreItemAction) Cancel(\n\toperationID string,\n\trestore *velerov1api.Restore,\n) error {\n\treturn nil\n}\n\nfunc (p *volumeSnapshotClassRestoreItemAction) AreAdditionalItemsReady(\n\tadditionalItems []velero.ResourceIdentifier,\n\trestore *velerov1api.Restore,\n) (bool, error) {\n\treturn true, nil\n}\n\nfunc NewVolumeSnapshotClassRestoreItemAction(\n\tlogger logrus.FieldLogger) (any, error) {\n\treturn &volumeSnapshotClassRestoreItemAction{logger}, nil\n}\n" }, { "path": "pkg/restore/actions/csi/volumesnapshotclass_action_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\t\"testing\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\n\t//\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\nfunc TestVSClassExecute(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\titem runtime.Unstructured\n\t\tvsClass *snapshotv1api.VolumeSnapshotClass\n\t\trestore *velerov1api.Restore\n\t\texpectErr bool\n\t\texpectedItems []velero.ResourceIdentifier\n\t}{\n\t\t{\n\t\t\tname: \"Restore's RestorePVs is false\",\n\t\t\trestore: builder.ForRestore(\"velero\", \"restore\").RestorePVs(false).Result(),\n\t\t\texpectErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"No Secret in the VS Class, no return additional items\",\n\t\t\tvsClass: builder.ForVolumeSnapshotClass(\"test\").Result(),\n\t\t\trestore: builder.ForRestore(\"velero\", \"restore\").Result(),\n\t\t\texpectErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Normal case, additional items should return\",\n\t\t\tvsClass: builder.ForVolumeSnapshotClass(\"test\").ObjectMeta(builder.WithAnnotationsMap(\n\t\t\t\tmap[string]string{\n\t\t\t\t\tvelerov1api.PrefixedListSecretNameAnnotation: \"name\",\n\t\t\t\t\tvelerov1api.PrefixedListSecretNamespaceAnnotation: \"namespace\",\n\t\t\t\t},\n\t\t\t)).Result(),\n\t\t\trestore: builder.ForRestore(\"velero\", \"restore\").Result(),\n\t\t\texpectErr: false,\n\t\t\texpectedItems: []velero.ResourceIdentifier{\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.Secrets,\n\t\t\t\t\tNamespace: \"namespace\",\n\t\t\t\t\tName: \"name\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tp, err := NewVolumeSnapshotClassRestoreItemAction(logrus.StandardLogger())\n\t\t\trequire.NoError(t, err)\n\n\t\t\taction := p.(*volumeSnapshotClassRestoreItemAction)\n\n\t\t\tif test.vsClass != nil {\n\t\t\t\tvsMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(test.vsClass)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\ttest.item = &unstructured.Unstructured{Object: vsMap}\n\t\t\t}\n\n\t\t\toutput, err := action.Execute(\n\t\t\t\t&velero.RestoreItemActionExecuteInput{\n\t\t\t\t\tItem: test.item,\n\t\t\t\t\tRestore: test.restore,\n\t\t\t\t},\n\t\t\t)\n\n\t\t\tif test.expectErr == false {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif len(test.expectedItems) > 0 {\n\t\t\t\trequire.Equal(t, test.expectedItems, output.AdditionalItems)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestVSClassAppliesTo(t *testing.T) {\n\tp := volumeSnapshotClassRestoreItemAction{\n\t\tlog: logrus.StandardLogger(),\n\t}\n\tselector, err := p.AppliesTo()\n\n\trequire.NoError(t, err)\n\n\trequire.Equal(\n\t\tt,\n\t\tvelero.ResourceSelector{\n\t\t\tIncludedResources: []string{\"volumesnapshotclasses.snapshot.storage.k8s.io\"},\n\t\t},\n\t\tselector,\n\t)\n}\n" }, { "path": "pkg/restore/actions/csi/volumesnapshotcontent_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\tcrclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\tplugincommon \"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/csi\"\n)\n\n// volumeSnapshotContentRestoreItemAction is a restore item action\n// plugin for Velero\ntype volumeSnapshotContentRestoreItemAction struct {\n\tlog logrus.FieldLogger\n\tclient crclient.Client\n}\n\n// AppliesTo returns information indicating VolumeSnapshotContentRestoreItemAction\n// action should be invoked while restoring\n// volumesnapshotcontent.snapshot.storage.k8s.io resources\nfunc (p *volumeSnapshotContentRestoreItemAction) AppliesTo() (\n\tvelero.ResourceSelector, error,\n) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"volumesnapshotcontents.snapshot.storage.k8s.io\"},\n\t}, nil\n}\n\n// Execute restores a VolumeSnapshotContent object without modification\n// returning the snapshot lister secret, if any, as additional items to restore.\nfunc (p *volumeSnapshotContentRestoreItemAction) Execute(\n\tinput *velero.RestoreItemActionExecuteInput,\n) (*velero.RestoreItemActionExecuteOutput, error) {\n\tif boolptr.IsSetToFalse(input.Restore.Spec.RestorePVs) {\n\t\tp.log.Infof(\"Restore did not request for PVs to be restored %s/%s\",\n\t\t\tinput.Restore.Namespace, input.Restore.Name)\n\t\treturn &velero.RestoreItemActionExecuteOutput{SkipRestore: true}, nil\n\t}\n\n\tp.log.Info(\"Starting VolumeSnapshotContentRestoreItemAction\")\n\n\tvar vsc snapshotv1api.VolumeSnapshotContent\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(\n\t\tinput.Item.UnstructuredContent(), &vsc); err != nil {\n\t\treturn &velero.RestoreItemActionExecuteOutput{},\n\t\t\terrors.Wrapf(err, \"failed to convert input.Item from unstructured\")\n\t}\n\n\tvar vscFromBackup snapshotv1api.VolumeSnapshotContent\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(\n\t\tinput.ItemFromBackup.UnstructuredContent(), &vscFromBackup); err != nil {\n\t\treturn &velero.RestoreItemActionExecuteOutput{},\n\t\t\terrors.Errorf(err.Error(), \"failed to convert input.ItemFromBackup from unstructured\")\n\t}\n\n\t// If cross-namespace restore is configured, change the namespace\n\t// for VolumeSnapshot object to be restored\n\tnewNamespace, ok := input.Restore.Spec.NamespaceMapping[vsc.Spec.VolumeSnapshotRef.Namespace]\n\tif ok {\n\t\t// Update the referenced VS namespace to the mapping one.\n\t\tvsc.Spec.VolumeSnapshotRef.Namespace = newNamespace\n\t}\n\n\t// Reset VSC name to align with VS.\n\tvsc.Name = util.GenerateSha256FromRestoreUIDAndVsName(\n\t\tstring(input.Restore.UID), vscFromBackup.Spec.VolumeSnapshotRef.Name)\n\t// Also reset the referenced VS name.\n\tvsc.Spec.VolumeSnapshotRef.Name = vsc.Name\n\n\t// Reset the ResourceVersion and UID of referenced VolumeSnapshot.\n\tvsc.Spec.VolumeSnapshotRef.ResourceVersion = \"\"\n\tvsc.Spec.VolumeSnapshotRef.UID = \"\"\n\n\t// Set the DeletionPolicy to Retain to avoid VS deletion will not trigger snapshot deletion\n\tvsc.Spec.DeletionPolicy = snapshotv1api.VolumeSnapshotContentRetain\n\n\tif vscFromBackup.Status != nil && vscFromBackup.Status.SnapshotHandle != nil {\n\t\tvsc.Spec.Source.VolumeHandle = nil\n\t\tvsc.Spec.Source.SnapshotHandle = vscFromBackup.Status.SnapshotHandle\n\t} else {\n\t\tp.log.Errorf(\"fail to get snapshot handle from VSC %s status\", vsc.Name)\n\t\treturn nil, errors.Errorf(\"fail to get snapshot handle from VSC %s status\", vsc.Name)\n\t}\n\n\tif vsc.Spec.VolumeSnapshotClassName != nil {\n\t\t// Delete VolumeSnapshotClass from the VolumeSnapshotContent.\n\t\t// This is necessary to make the restore independent of the VolumeSnapshotClass.\n\t\tvsc.Spec.VolumeSnapshotClassName = nil\n\t\tp.log.Debugf(\"Deleted VolumeSnapshotClassName from VolumeSnapshotContent %s to make restore independent of VolumeSnapshotClass\",\n\t\t\tvsc.Name)\n\t}\n\n\tadditionalItems := []velero.ResourceIdentifier{}\n\tif csi.IsVolumeSnapshotContentHasDeleteSecret(&vsc) {\n\t\tadditionalItems = append(additionalItems,\n\t\t\tvelero.ResourceIdentifier{\n\t\t\t\tGroupResource: schema.GroupResource{Group: \"\", Resource: \"secrets\"},\n\t\t\t\tName: vsc.Annotations[velerov1api.PrefixedSecretNameAnnotation],\n\t\t\t\tNamespace: vsc.Annotations[velerov1api.PrefixedSecretNamespaceAnnotation],\n\t\t\t},\n\t\t)\n\t}\n\n\tvscMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&vsc)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tp.log.Infof(\"Returning from VolumeSnapshotContentRestoreItemAction with %d additionalItems\",\n\t\tlen(additionalItems))\n\treturn &velero.RestoreItemActionExecuteOutput{\n\t\tUpdatedItem: &unstructured.Unstructured{Object: vscMap},\n\t\tAdditionalItems: additionalItems,\n\t}, nil\n}\n\nfunc (p *volumeSnapshotContentRestoreItemAction) Name() string {\n\treturn \"VolumeSnapshotContentRestoreItemAction\"\n}\n\nfunc (p *volumeSnapshotContentRestoreItemAction) Progress(\n\toperationID string,\n\trestore *velerov1api.Restore,\n) (velero.OperationProgress, error) {\n\treturn velero.OperationProgress{}, nil\n}\n\nfunc (p *volumeSnapshotContentRestoreItemAction) Cancel(\n\toperationID string,\n\trestore *velerov1api.Restore,\n) error {\n\treturn nil\n}\n\nfunc (p *volumeSnapshotContentRestoreItemAction) AreAdditionalItemsReady(\n\tadditionalItems []velero.ResourceIdentifier,\n\trestore *velerov1api.Restore,\n) (bool, error) {\n\treturn true, nil\n}\n\nfunc NewVolumeSnapshotContentRestoreItemAction(\n\tf client.Factory,\n) plugincommon.HandlerInitializer {\n\treturn func(logger logrus.FieldLogger) (any, error) {\n\t\tcrClient, err := f.KubebuilderClient()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\treturn &volumeSnapshotContentRestoreItemAction{logger, crClient}, nil\n\t}\n}\n" }, { "path": "pkg/restore/actions/csi/volumesnapshotcontent_action_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\t\"fmt\"\n\t\"testing\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tfactorymocks \"github.com/vmware-tanzu/velero/pkg/client/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n)\n\nfunc TestVSCExecute(t *testing.T) {\n\tsnapshotHandleName := \"testHandle\"\n\tnewVscName := util.GenerateSha256FromRestoreUIDAndVsName(\"restoreUID\", \"vsName\")\n\ttests := []struct {\n\t\tname string\n\t\titem runtime.Unstructured\n\t\tvsc *snapshotv1api.VolumeSnapshotContent\n\t\trestore *velerov1api.Restore\n\t\texpectErr bool\n\t\tcreateVSC bool\n\t\texpectedItems []velero.ResourceIdentifier\n\t\texpectedVSC *snapshotv1api.VolumeSnapshotContent\n\t}{\n\t\t{\n\t\t\tname: \"Restore's RestorePVs is false\",\n\t\t\trestore: builder.ForRestore(\"velero\", \"restore\").RestorePVs(false).Result(),\n\t\t\texpectErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Normal case, additional items should return \",\n\t\t\tvsc: builder.ForVolumeSnapshotContent(\"test\").\n\t\t\t\tObjectMeta(builder.WithAnnotationsMap(\n\t\t\t\t\tmap[string]string{\n\t\t\t\t\t\tvelerov1api.PrefixedSecretNameAnnotation: \"name\",\n\t\t\t\t\t\tvelerov1api.PrefixedSecretNamespaceAnnotation: \"namespace\",\n\t\t\t\t\t},\n\t\t\t\t)).\n\t\t\t\tVolumeSnapshotRef(\"velero\", \"vsName\", \"vsUID\").\n\t\t\t\tVolumeSnapshotClassName(\"vsClass\").\n\t\t\t\tStatus(&snapshotv1api.VolumeSnapshotContentStatus{SnapshotHandle: &snapshotHandleName}).\n\t\t\t\tResult(),\n\t\t\trestore: builder.ForRestore(\"velero\", \"restore\").ObjectMeta(builder.WithUID(\"restoreUID\")).\n\t\t\t\tNamespaceMappings(\"velero\", \"restore\").Result(),\n\t\t\texpectErr: false,\n\t\t\texpectedItems: []velero.ResourceIdentifier{\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.Secrets,\n\t\t\t\t\tNamespace: \"namespace\",\n\t\t\t\t\tName: \"name\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVSC: builder.ForVolumeSnapshotContent(newVscName).\n\t\t\t\tObjectMeta(builder.WithAnnotationsMap(\n\t\t\t\t\tmap[string]string{\n\t\t\t\t\t\tvelerov1api.PrefixedSecretNameAnnotation: \"name\",\n\t\t\t\t\t\tvelerov1api.PrefixedSecretNamespaceAnnotation: \"namespace\",\n\t\t\t\t\t},\n\t\t\t\t)).VolumeSnapshotRef(\"restore\", newVscName, \"\").\n\t\t\t\tSource(snapshotv1api.VolumeSnapshotContentSource{SnapshotHandle: &snapshotHandleName}).\n\t\t\t\tDeletionPolicy(snapshotv1api.VolumeSnapshotContentRetain).\n\t\t\t\tStatus(&snapshotv1api.VolumeSnapshotContentStatus{SnapshotHandle: &snapshotHandleName}).\n\t\t\t\tResult(),\n\t\t},\n\t\t{\n\t\t\tname: \"VSC exists in cluster, same as the normal case\",\n\t\t\tvsc: builder.ForVolumeSnapshotContent(\"test\").ObjectMeta(builder.WithAnnotationsMap(\n\t\t\t\tmap[string]string{\n\t\t\t\t\tvelerov1api.PrefixedSecretNameAnnotation: \"name\",\n\t\t\t\t\tvelerov1api.PrefixedSecretNamespaceAnnotation: \"namespace\",\n\t\t\t\t},\n\t\t\t)).VolumeSnapshotRef(\"velero\", \"vsName\", \"vsUID\").\n\t\t\t\tStatus(&snapshotv1api.VolumeSnapshotContentStatus{SnapshotHandle: &snapshotHandleName}).Result(),\n\t\t\trestore: builder.ForRestore(\"velero\", \"restore\").ObjectMeta(builder.WithUID(\"restoreUID\")).\n\t\t\t\tNamespaceMappings(\"velero\", \"restore\").Result(),\n\t\t\tcreateVSC: true,\n\t\t\texpectErr: false,\n\t\t\texpectedVSC: builder.ForVolumeSnapshotContent(newVscName).ObjectMeta(builder.WithAnnotationsMap(\n\t\t\t\tmap[string]string{\n\t\t\t\t\tvelerov1api.PrefixedSecretNameAnnotation: \"name\",\n\t\t\t\t\tvelerov1api.PrefixedSecretNamespaceAnnotation: \"namespace\",\n\t\t\t\t},\n\t\t\t)).VolumeSnapshotRef(\"restore\", newVscName, \"\").\n\t\t\t\tSource(snapshotv1api.VolumeSnapshotContentSource{SnapshotHandle: &snapshotHandleName}).\n\t\t\t\tDeletionPolicy(snapshotv1api.VolumeSnapshotContentRetain).\n\t\t\t\tStatus(&snapshotv1api.VolumeSnapshotContentStatus{SnapshotHandle: &snapshotHandleName}).Result(),\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\taction := volumeSnapshotContentRestoreItemAction{\n\t\t\t\tlog: logrus.StandardLogger(),\n\t\t\t\tclient: velerotest.NewFakeControllerRuntimeClient(t),\n\t\t\t}\n\n\t\t\tif test.vsc != nil {\n\t\t\t\tvsMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(test.vsc)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\ttest.item = &unstructured.Unstructured{Object: vsMap}\n\n\t\t\t\tif test.createVSC {\n\t\t\t\t\trequire.NoError(t, action.client.Create(t.Context(), test.vsc))\n\t\t\t\t}\n\t\t\t}\n\n\t\t\toutput, err := action.Execute(\n\t\t\t\t&velero.RestoreItemActionExecuteInput{\n\t\t\t\t\tItem: test.item,\n\t\t\t\t\tItemFromBackup: test.item,\n\t\t\t\t\tRestore: test.restore,\n\t\t\t\t},\n\t\t\t)\n\n\t\t\tif test.expectErr == false {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif test.expectedVSC != nil {\n\t\t\t\tvsc := new(snapshotv1api.VolumeSnapshotContent)\n\t\t\t\trequire.NoError(t,\n\t\t\t\t\truntime.DefaultUnstructuredConverter.FromUnstructured(\n\t\t\t\t\t\toutput.UpdatedItem.UnstructuredContent(),\n\t\t\t\t\t\tvsc,\n\t\t\t\t\t),\n\t\t\t\t)\n\n\t\t\t\trequire.Equal(t, test.expectedVSC, vsc)\n\t\t\t}\n\n\t\t\tif len(test.expectedItems) > 0 {\n\t\t\t\trequire.Equal(t, test.expectedItems, output.AdditionalItems)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestVSCAppliesTo(t *testing.T) {\n\tp := volumeSnapshotContentRestoreItemAction{\n\t\tlog: logrus.StandardLogger(),\n\t}\n\tselector, err := p.AppliesTo()\n\n\trequire.NoError(t, err)\n\n\trequire.Equal(\n\t\tt,\n\t\tvelero.ResourceSelector{\n\t\t\tIncludedResources: []string{\"volumesnapshotcontents.snapshot.storage.k8s.io\"},\n\t\t},\n\t\tselector,\n\t)\n}\n\nfunc TestNewVolumeSnapshotContentRestoreItemAction(t *testing.T) {\n\tlogger := logrus.StandardLogger()\n\tcrClient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\tf := &factorymocks.Factory{}\n\tf.On(\"KubebuilderClient\").Return(nil, fmt.Errorf(\"\"))\n\tplugin := NewVolumeSnapshotContentRestoreItemAction(f)\n\t_, err := plugin(logger)\n\trequire.Error(t, err)\n\n\tf1 := &factorymocks.Factory{}\n\tf1.On(\"KubebuilderClient\").Return(crClient, nil)\n\tplugin1 := NewVolumeSnapshotContentRestoreItemAction(f1)\n\t_, err1 := plugin1(logger)\n\trequire.NoError(t, err1)\n}\n" }, { "path": "pkg/restore/actions/dataupload_retrieve_action.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\tveleroclient \"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\ntype DataUploadRetrieveAction struct {\n\tlogger logrus.FieldLogger\n\tclient client.Client\n}\n\nfunc NewDataUploadRetrieveAction(logger logrus.FieldLogger, client client.Client) *DataUploadRetrieveAction {\n\treturn &DataUploadRetrieveAction{\n\t\tlogger: logger,\n\t\tclient: client,\n\t}\n}\n\nfunc (d *DataUploadRetrieveAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"datauploads.velero.io\"},\n\t}, nil\n}\n\nfunc (d *DataUploadRetrieveAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\td.logger.Info(\"Executing DataUploadRetrieveAction\")\n\n\tdataUpload := velerov2alpha1.DataUpload{}\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(input.ItemFromBackup.UnstructuredContent(), &dataUpload); err != nil {\n\t\td.logger.Errorf(\"unable to convert unstructured item to DataUpload: %s\", err.Error())\n\t\treturn nil, errors.Wrap(err, \"unable to convert unstructured item to DataUpload.\")\n\t}\n\n\tbackup := &velerov1api.Backup{}\n\terr := d.client.Get(context.Background(), types.NamespacedName{\n\t\tNamespace: input.Restore.Namespace,\n\t\tName: input.Restore.Spec.BackupName,\n\t}, backup)\n\tif err != nil {\n\t\td.logger.WithError(err).Errorf(\"Fail to get backup for restore %s.\", input.Restore.Name)\n\t\treturn nil, errors.Wrapf(err, \"error to get backup for restore %s\", input.Restore.Name)\n\t}\n\n\tdataUploadResult := velerov2alpha1.DataUploadResult{\n\t\tBackupStorageLocation: backup.Spec.StorageLocation,\n\t\tDataMover: dataUpload.Spec.DataMover,\n\t\tSnapshotID: dataUpload.Status.SnapshotID,\n\t\tSnapshotSize: dataUpload.Status.Progress.TotalBytes,\n\t\tSourceNamespace: dataUpload.Spec.SourceNamespace,\n\t\tDataMoverResult: dataUpload.Status.DataMoverResult,\n\t\tNodeOS: dataUpload.Status.NodeOS,\n\t}\n\n\tjsonBytes, err := json.Marshal(dataUploadResult)\n\tif err != nil {\n\t\td.logger.Errorf(\"fail to convert DataUploadResult to JSON: %s\", err.Error())\n\t\treturn nil, errors.Wrap(err, \"fail to convert DataUploadResult to JSON\")\n\t}\n\n\tcm := corev1api.ConfigMap{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"ConfigMap\",\n\t\t\tAPIVersion: corev1api.SchemeGroupVersion.String(),\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tGenerateName: dataUpload.Name + \"-\",\n\t\t\tNamespace: input.Restore.Namespace,\n\t\t\tLabels: map[string]string{\n\t\t\t\tvelerov1api.RestoreUIDLabel: label.GetValidName(string(input.Restore.UID)),\n\t\t\t\tvelerov1api.PVCNamespaceNameLabel: label.GetValidName(dataUpload.Spec.SourceNamespace + \".\" + dataUpload.Spec.SourcePVC),\n\t\t\t\tvelerov1api.ResourceUsageLabel: label.GetValidName(string(velerov1api.VeleroResourceUsageDataUploadResult)),\n\t\t\t},\n\t\t},\n\t\tData: map[string]string{\n\t\t\tstring(input.Restore.UID): string(jsonBytes),\n\t\t},\n\t}\n\n\terr = veleroclient.CreateRetryGenerateName(d.client, context.Background(), &cm)\n\tif err != nil {\n\t\td.logger.Errorf(\"fail to create DataUploadResult ConfigMap %s/%s: %s\", cm.Namespace, cm.Name, err.Error())\n\t\treturn nil, errors.Wrap(err, \"fail to create DataUploadResult ConfigMap\")\n\t}\n\n\treturn &velero.RestoreItemActionExecuteOutput{\n\t\tSkipRestore: true,\n\t}, nil\n}\n" }, { "path": "pkg/restore/actions/dataupload_retrieve_action_test.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerov1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestDataUploadRetrieveActionExectue(t *testing.T) {\n\tscheme := runtime.NewScheme()\n\tvelerov1.AddToScheme(scheme)\n\tcorev1api.AddToScheme(scheme)\n\n\ttests := []struct {\n\t\tname string\n\t\tdataUpload *velerov2alpha1.DataUpload\n\t\trestore *velerov1.Restore\n\t\texpectedDataUploadResult *corev1api.ConfigMap\n\t\texpectedErr string\n\t\truntimeScheme *runtime.Scheme\n\t\tveleroObjs []runtime.Object\n\t}{\n\t\t{\n\t\t\tname: \"error to find backup\",\n\t\t\tdataUpload: builder.ForDataUpload(\"velero\", \"testDU\").SourceNamespace(\"testNamespace\").SourcePVC(\"testPVC\").Result(),\n\t\t\trestore: builder.ForRestore(\"velero\", \"testRestore\").ObjectMeta(builder.WithUID(\"testingUID\")).Backup(\"testBackup\").Result(),\n\t\t\truntimeScheme: scheme,\n\t\t\texpectedErr: \"error to get backup for restore testRestore: backups.velero.io \\\"testBackup\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"DataUploadRetrieve Action test\",\n\t\t\tdataUpload: builder.ForDataUpload(\"velero\", \"testDU\").SourceNamespace(\"testNamespace\").SourcePVC(\"testPVC\").SnapshotID(\"fake-id\").TotalBytes(1000).Result(),\n\t\t\trestore: builder.ForRestore(\"velero\", \"testRestore\").ObjectMeta(builder.WithUID(\"testingUID\")).Backup(\"testBackup\").Result(),\n\t\t\truntimeScheme: scheme,\n\t\t\tveleroObjs: []runtime.Object{\n\t\t\t\tbuilder.ForBackup(\"velero\", \"testBackup\").StorageLocation(\"testLocation\").Result(),\n\t\t\t},\n\t\t\texpectedDataUploadResult: builder.ForConfigMap(\"velero\", \"\").ObjectMeta(builder.WithGenerateName(\"testDU-\"), builder.WithLabels(velerov1.PVCNamespaceNameLabel, \"testNamespace.testPVC\", velerov1.RestoreUIDLabel, \"testingUID\", velerov1.ResourceUsageLabel, string(velerov1.VeleroResourceUsageDataUploadResult))).Data(\"testingUID\", `{\"backupStorageLocation\":\"testLocation\",\"snapshotID\":\"fake-id\",\"sourceNamespace\":\"testNamespace\",\"snapshotSize\":1000}`).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Long source namespace and PVC name should also work\",\n\t\t\tdataUpload: builder.ForDataUpload(\"velero\", \"testDU\").SourceNamespace(\"migre209d0da-49c7-45ba-8d5a-3e59fd591ec1\").SourcePVC(\"kibishii-data-kibishii-deployment-0\").Result(),\n\t\t\trestore: builder.ForRestore(\"velero\", \"testRestore\").ObjectMeta(builder.WithUID(\"testingUID\")).Backup(\"testBackup\").Result(),\n\t\t\truntimeScheme: scheme,\n\t\t\tveleroObjs: []runtime.Object{\n\t\t\t\tbuilder.ForBackup(\"velero\", \"testBackup\").StorageLocation(\"testLocation\").Result(),\n\t\t\t},\n\t\t\texpectedDataUploadResult: builder.ForConfigMap(\"velero\", \"\").ObjectMeta(builder.WithGenerateName(\"testDU-\"), builder.WithLabels(velerov1.PVCNamespaceNameLabel, \"migre209d0da-49c7-45ba-8d5a-3e59fd591ec1.kibishii-data-ki152333\", velerov1.RestoreUIDLabel, \"testingUID\", velerov1.ResourceUsageLabel, string(velerov1.VeleroResourceUsageDataUploadResult))).Data(\"testingUID\", `{\"backupStorageLocation\":\"testLocation\",\"sourceNamespace\":\"migre209d0da-49c7-45ba-8d5a-3e59fd591ec1\"}`).Result(),\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlogger := velerotest.NewLogger()\n\n\t\t\tfakeClientBuilder := fake.NewClientBuilder()\n\t\t\tif tc.runtimeScheme != nil {\n\t\t\t\tfakeClientBuilder = fakeClientBuilder.WithScheme(tc.runtimeScheme)\n\t\t\t}\n\n\t\t\tfakeClient := fakeClientBuilder.WithRuntimeObjects(tc.veleroObjs...).Build()\n\n\t\t\tvar unstructuredDataUpload map[string]any\n\t\t\tif tc.dataUpload != nil {\n\t\t\t\tvar err error\n\t\t\t\tunstructuredDataUpload, err = runtime.DefaultUnstructuredConverter.ToUnstructured(tc.dataUpload)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\t\t\tinput := velero.RestoreItemActionExecuteInput{\n\t\t\t\tRestore: tc.restore,\n\t\t\t\tItemFromBackup: &unstructured.Unstructured{Object: unstructuredDataUpload},\n\t\t\t}\n\n\t\t\taction := NewDataUploadRetrieveAction(logger, fakeClient)\n\t\t\t_, err := action.Execute(&input)\n\t\t\tif tc.expectedErr != \"\" {\n\t\t\t\trequire.Equal(t, tc.expectedErr, err.Error())\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif tc.expectedDataUploadResult != nil {\n\t\t\t\tvar cmList corev1api.ConfigMapList\n\t\t\t\terr := fakeClient.List(t.Context(), &cmList, &client.ListOptions{\n\t\t\t\t\tLabelSelector: labels.SelectorFromSet(map[string]string{\n\t\t\t\t\t\tvelerov1.RestoreUIDLabel: \"testingUID\",\n\t\t\t\t\t\tvelerov1.PVCNamespaceNameLabel: label.GetValidName(tc.dataUpload.Spec.SourceNamespace + \".\" + tc.dataUpload.Spec.SourcePVC),\n\t\t\t\t\t}),\n\t\t\t\t})\n\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\trequire.Equal(t, tc.expectedDataUploadResult.Labels, cmList.Items[0].Labels)\n\t\t\t\trequire.Equal(t, tc.expectedDataUploadResult.Data, cmList.Items[0].Data)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/restore/actions/init_restorehook_pod_action.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\n\t\"github.com/vmware-tanzu/velero/internal/hook\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// InitRestoreHookPodAction is a RestoreItemAction plugin applicable to pods that runs\n// restore hooks to add init containers to pods prior to them being restored.\ntype InitRestoreHookPodAction struct {\n\tlogger logrus.FieldLogger\n}\n\n// NewInitRestoreHookPodAction returns a new InitRestoreHookPodAction.\nfunc NewInitRestoreHookPodAction(logger logrus.FieldLogger) *InitRestoreHookPodAction {\n\treturn &InitRestoreHookPodAction{logger: logger}\n}\n\n// AppliesTo implements the RestoreItemAction plugin interface method.\nfunc (a *InitRestoreHookPodAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"pods\"},\n\t}, nil\n}\n\n// Execute implements the RestoreItemAction plugin interface method.\nfunc (a *InitRestoreHookPodAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\ta.logger.Infof(\"Executing InitRestoreHookPodAction\")\n\t// handle any init container restore hooks for the pod\n\trestoreHooks, err := hook.GetRestoreHooksFromSpec(&input.Restore.Spec.Hooks)\n\tnsMapping := input.Restore.Spec.NamespaceMapping\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\thookHandler := hook.InitContainerRestoreHookHandler{}\n\tpostHooksItem, err := hookHandler.HandleRestoreHooks(a.logger, kuberesource.Pods, input.Item, restoreHooks, nsMapping)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\ta.logger.Infof(\"Returning from InitRestoreHookPodAction\")\n\n\treturn velero.NewRestoreItemActionExecuteOutput(&unstructured.Unstructured{Object: postHooksItem.UnstructuredContent()}), nil\n}\n" }, { "path": "pkg/restore/actions/init_restorehook_pod_action_test.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"testing\"\n\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestInitContainerRestoreHookPodActionExecute(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tobj *corev1api.Pod\n\t\texpectedErr bool\n\t\texpectedRes *corev1api.Pod\n\t\trestore *velerov1api.Restore\n\t}{\n\t\t{\n\t\t\tname: \"should run restore hooks from pod annotation\",\n\t\t\trestore: &velerov1api.Restore{},\n\t\t\tobj: builder.ForPod(\"default\", \"app1\").\n\t\t\t\tObjectMeta(builder.WithAnnotations(\n\t\t\t\t\t\"init.hook.restore.velero.io/container-image\", \"nginx\",\n\t\t\t\t\t\"init.hook.restore.velero.io/container-name\", \"restore-init-container\",\n\t\t\t\t\t\"init.hook.restore.velero.io/command\", `[\"a\", \"b\", \"c\"]`,\n\t\t\t\t)).\n\t\t\t\tServiceAccount(\"foo\").\n\t\t\t\tVolumes([]*corev1api.Volume{{Name: \"foo\"}}...).\n\t\t\t\tInitContainers([]*corev1api.Container{\n\t\t\t\t\tbuilder.ForContainer(\"init-app-step1\", \"busy-box\").\n\t\t\t\t\t\tCommand([]string{\"init-step1\"}).Result(),\n\t\t\t\t\tbuilder.ForContainer(\"init-app-step2\", \"busy-box\").\n\t\t\t\t\t\tCommand([]string{\"init-step2\"}).Result(),\n\t\t\t\t\tbuilder.ForContainer(\"init-app-step3\", \"busy-box\").\n\t\t\t\t\t\tCommand([]string{\"init-step3\"}).Result()}...).Result(),\n\t\t\texpectedRes: builder.ForPod(\"default\", \"app1\").\n\t\t\t\tObjectMeta(builder.WithAnnotations(\n\t\t\t\t\t\"init.hook.restore.velero.io/container-image\", \"nginx\",\n\t\t\t\t\t\"init.hook.restore.velero.io/container-name\", \"restore-init-container\",\n\t\t\t\t\t\"init.hook.restore.velero.io/command\", `[\"a\", \"b\", \"c\"]`,\n\t\t\t\t)).\n\t\t\t\tServiceAccount(\"foo\").\n\t\t\t\tVolumes([]*corev1api.Volume{{Name: \"foo\"}}...).\n\t\t\t\tInitContainers([]*corev1api.Container{\n\t\t\t\t\tbuilder.ForContainer(\"restore-init-container\", \"nginx\").\n\t\t\t\t\t\tCommand([]string{\"a\", \"b\", \"c\"}).Result(),\n\t\t\t\t\tbuilder.ForContainer(\"init-app-step1\", \"busy-box\").\n\t\t\t\t\t\tCommand([]string{\"init-step1\"}).Result(),\n\t\t\t\t\tbuilder.ForContainer(\"init-app-step2\", \"busy-box\").\n\t\t\t\t\t\tCommand([]string{\"init-step2\"}).Result(),\n\t\t\t\t\tbuilder.ForContainer(\"init-app-step3\", \"busy-box\").\n\t\t\t\t\t\tCommand([]string{\"init-step3\"}).Result()}...).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"should run restore hook from restore spec\",\n\t\t\trestore: &velerov1api.Restore{\n\t\t\t\tSpec: velerov1api.RestoreSpec{\n\t\t\t\t\tHooks: velerov1api.RestoreHooks{\n\t\t\t\t\t\tResources: []velerov1api.RestoreResourceHookSpec{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tName: \"h1\",\n\t\t\t\t\t\t\t\tIncludedNamespaces: []string{\"default\"},\n\t\t\t\t\t\t\t\tIncludedResources: []string{kuberesource.Pods.Resource},\n\t\t\t\t\t\t\t\tPostHooks: []velerov1api.RestoreResourceHook{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tInit: &velerov1api.InitRestoreHook{\n\t\t\t\t\t\t\t\t\t\t\tInitContainers: []runtime.RawExtension{\n\t\t\t\t\t\t\t\t\t\t\t\tbuilder.ForContainer(\"restore-init1\", \"busy-box\").\n\t\t\t\t\t\t\t\t\t\t\t\t\tCommand([]string{\"foobarbaz\"}).ResultRawExtension(),\n\t\t\t\t\t\t\t\t\t\t\t\tbuilder.ForContainer(\"restore-init2\", \"busy-box\").\n\t\t\t\t\t\t\t\t\t\t\t\t\tCommand([]string{\"foobarbaz\"}).ResultRawExtension(),\n\t\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tobj: builder.ForPod(\"default\", \"app1\").\n\t\t\t\tServiceAccount(\"foo\").\n\t\t\t\tVolumes([]*corev1api.Volume{{Name: \"foo\"}}...).Result(),\n\t\t\texpectedRes: builder.ForPod(\"default\", \"app1\").\n\t\t\t\tServiceAccount(\"foo\").\n\t\t\t\tVolumes([]*corev1api.Volume{{Name: \"foo\"}}...).\n\t\t\t\tInitContainers([]*corev1api.Container{\n\t\t\t\t\tbuilder.ForContainer(\"restore-init1\", \"busy-box\").\n\t\t\t\t\t\tCommand([]string{\"foobarbaz\"}).Result(),\n\t\t\t\t\tbuilder.ForContainer(\"restore-init2\", \"busy-box\").\n\t\t\t\t\t\tCommand([]string{\"foobarbaz\"}).Result(),\n\t\t\t\t}...).Result(),\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\taction := NewInitRestoreHookPodAction(velerotest.NewLogger())\n\t\t\tunstructuredPod, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&tc.obj)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tres, err := action.Execute(&velero.RestoreItemActionExecuteInput{\n\t\t\t\tItem: &unstructured.Unstructured{Object: unstructuredPod},\n\t\t\t\tItemFromBackup: &unstructured.Unstructured{Object: unstructuredPod},\n\t\t\t\tRestore: tc.restore,\n\t\t\t})\n\t\t\tif tc.expectedErr {\n\t\t\t\tassert.Error(t, err, \"expected an error\")\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NoError(t, err, \"expected no error, got %v\", err)\n\n\t\t\tvar pod corev1api.Pod\n\t\t\trequire.NoError(t, runtime.DefaultUnstructuredConverter.FromUnstructured(res.UpdatedItem.UnstructuredContent(), &pod))\n\n\t\t\tassert.Equal(t, *tc.expectedRes, pod)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/restore/actions/job_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tbatchv1api \"k8s.io/api/batch/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\nconst (\n\tlegacyControllerUIDLabel = \"controller-uid\" // <=1.27 This still exists in 1.27 for backward compatibility, maybe remove in 1.28?\n\tcontrollerUIDLabel = \"batch.kubernetes.io/controller-uid\" // >=1.27 https://github.com/kubernetes/kubernetes/pull/114930#issuecomment-1384667494\n)\n\ntype JobAction struct {\n\tlogger logrus.FieldLogger\n}\n\nfunc NewJobAction(logger logrus.FieldLogger) *JobAction {\n\treturn &JobAction{logger: logger}\n}\n\nfunc (a *JobAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"jobs\"},\n\t}, nil\n}\n\nfunc (a *JobAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\tjob := new(batchv1api.Job)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(input.Item.UnstructuredContent(), job); err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tif job.Spec.Selector != nil {\n\t\tdelete(job.Spec.Selector.MatchLabels, controllerUIDLabel)\n\t\tdelete(job.Spec.Selector.MatchLabels, legacyControllerUIDLabel)\n\t}\n\tdelete(job.Spec.Template.ObjectMeta.Labels, controllerUIDLabel)\n\tdelete(job.Spec.Template.ObjectMeta.Labels, legacyControllerUIDLabel)\n\n\tres, err := runtime.DefaultUnstructuredConverter.ToUnstructured(job)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\treturn velero.NewRestoreItemActionExecuteOutput(&unstructured.Unstructured{Object: res}), nil\n}\n" }, { "path": "pkg/restore/actions/job_action_test.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tbatchv1api \"k8s.io/api/batch/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestJobActionExecute(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tobj batchv1api.Job\n\t\texpectedErr bool\n\t\texpectedRes batchv1api.Job\n\t}{\n\t\t{\n\t\t\tname: \"missing spec.selector and/or spec.template should not error\",\n\t\t\tobj: batchv1api.Job{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"job-1\"},\n\t\t\t},\n\t\t\texpectedRes: batchv1api.Job{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"job-1\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"missing spec.selector.matchLabels should not error\",\n\t\t\tobj: batchv1api.Job{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"job-1\"},\n\t\t\t\tSpec: batchv1api.JobSpec{\n\t\t\t\t\tSelector: new(metav1.LabelSelector),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedRes: batchv1api.Job{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"job-1\"},\n\t\t\t\tSpec: batchv1api.JobSpec{\n\t\t\t\t\tSelector: new(metav1.LabelSelector),\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"spec.selector.matchLabels[controller-uid] is removed\",\n\t\t\tobj: batchv1api.Job{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"job-1\"},\n\t\t\t\tSpec: batchv1api.JobSpec{\n\t\t\t\t\tSelector: &metav1.LabelSelector{\n\t\t\t\t\t\tMatchLabels: map[string]string{\n\t\t\t\t\t\t\t\"controller-uid\": \"foo\",\n\t\t\t\t\t\t\t\"hello\": \"world\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedRes: batchv1api.Job{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"job-1\"},\n\t\t\t\tSpec: batchv1api.JobSpec{\n\t\t\t\t\tSelector: &metav1.LabelSelector{\n\t\t\t\t\t\tMatchLabels: map[string]string{\n\t\t\t\t\t\t\t\"hello\": \"world\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"missing spec.template.metadata.labels should not error\",\n\t\t\tobj: batchv1api.Job{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"job-1\"},\n\t\t\t\tSpec: batchv1api.JobSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedRes: batchv1api.Job{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"job-1\"},\n\t\t\t\tSpec: batchv1api.JobSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"spec.template.metadata.labels[controller-uid] is removed\",\n\t\t\tobj: batchv1api.Job{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"job-1\"},\n\t\t\t\tSpec: batchv1api.JobSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\t\t\"controller-uid\": \"foo\",\n\t\t\t\t\t\t\t\t\"hello\": \"world\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedRes: batchv1api.Job{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"job-1\"},\n\t\t\t\tSpec: batchv1api.JobSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\t\t\"hello\": \"world\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\taction := NewJobAction(velerotest.NewLogger())\n\n\t\t\tunstructuredJob, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&test.obj)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tres, err := action.Execute(&velero.RestoreItemActionExecuteInput{\n\t\t\t\tItem: &unstructured.Unstructured{Object: unstructuredJob},\n\t\t\t\tItemFromBackup: &unstructured.Unstructured{Object: unstructuredJob},\n\t\t\t\tRestore: nil,\n\t\t\t})\n\n\t\t\tif assert.Equal(t, test.expectedErr, err != nil) {\n\t\t\t\tvar job batchv1api.Job\n\t\t\t\trequire.NoError(t, runtime.DefaultUnstructuredConverter.FromUnstructured(res.UpdatedItem.UnstructuredContent(), &job))\n\n\t\t\t\tassert.Equal(t, test.expectedRes, job)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/restore/actions/pod_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"strings\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\ntype PodAction struct {\n\tlogger logrus.FieldLogger\n}\n\nfunc NewPodAction(logger logrus.FieldLogger) *PodAction {\n\treturn &PodAction{logger: logger}\n}\n\nfunc (a *PodAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"pods\"},\n\t}, nil\n}\n\nfunc (a *PodAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\tpod := new(corev1api.Pod)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(input.Item.UnstructuredContent(), pod); err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tpod.Spec.NodeName = \"\"\n\tpod.Spec.Priority = nil\n\n\tserviceAccountTokenPrefix := pod.Spec.ServiceAccountName + \"-token-\"\n\n\tvar preservedVolumes []corev1api.Volume\n\tfor _, vol := range pod.Spec.Volumes {\n\t\tif !strings.HasPrefix(vol.Name, serviceAccountTokenPrefix) {\n\t\t\tpreservedVolumes = append(preservedVolumes, vol)\n\t\t}\n\t}\n\tpod.Spec.Volumes = preservedVolumes\n\n\tfor i, container := range pod.Spec.Containers {\n\t\tvar preservedVolumeMounts []corev1api.VolumeMount\n\t\tfor _, mount := range container.VolumeMounts {\n\t\t\tif !strings.HasPrefix(mount.Name, serviceAccountTokenPrefix) {\n\t\t\t\tpreservedVolumeMounts = append(preservedVolumeMounts, mount)\n\t\t\t}\n\t\t}\n\t\tpod.Spec.Containers[i].VolumeMounts = preservedVolumeMounts\n\t}\n\n\tfor i, container := range pod.Spec.InitContainers {\n\t\tvar preservedVolumeMounts []corev1api.VolumeMount\n\t\tfor _, mount := range container.VolumeMounts {\n\t\t\tif !strings.HasPrefix(mount.Name, serviceAccountTokenPrefix) {\n\t\t\t\tpreservedVolumeMounts = append(preservedVolumeMounts, mount)\n\t\t\t}\n\t\t}\n\t\tpod.Spec.InitContainers[i].VolumeMounts = preservedVolumeMounts\n\t}\n\n\tres, err := runtime.DefaultUnstructuredConverter.ToUnstructured(pod)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\trestoreExecuteOutput := velero.NewRestoreItemActionExecuteOutput(&unstructured.Unstructured{Object: res})\n\tif pod.Spec.PriorityClassName != \"\" {\n\t\ta.logger.Infof(\"Adding priorityclass %s to AdditionalItems\", pod.Spec.PriorityClassName)\n\t\trestoreExecuteOutput.AdditionalItems = []velero.ResourceIdentifier{\n\t\t\t{GroupResource: kuberesource.PriorityClasses, Name: pod.Spec.PriorityClassName}}\n\t}\n\treturn restoreExecuteOutput, nil\n}\n" }, { "path": "pkg/restore/actions/pod_action_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestPodActionExecute(t *testing.T) {\n\tvar priority int32 = 1\n\n\ttests := []struct {\n\t\tname string\n\t\tobj corev1api.Pod\n\t\texpectedErr bool\n\t\texpectedRes corev1api.Pod\n\t\tadditionalItems []velero.ResourceIdentifier\n\t}{\n\t\t{\n\t\t\tname: \"nodeName (only) should be deleted from spec\",\n\t\t\tobj: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pod-1\"},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tNodeName: \"foo\",\n\t\t\t\t\tServiceAccountName: \"bar\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedRes: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pod-1\"},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tServiceAccountName: \"bar\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"priority (only) should be deleted from spec\",\n\t\t\tobj: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pod-1\"},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tPriority: &priority,\n\t\t\t\t\tServiceAccountName: \"bar\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedRes: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pod-1\"},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tServiceAccountName: \"bar\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"volumes matching prefix -token- should be deleted\",\n\t\t\tobj: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pod-1\"},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tServiceAccountName: \"foo\",\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{Name: \"foo\"},\n\t\t\t\t\t\t{Name: \"foo-token-foo\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedRes: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pod-1\"},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tServiceAccountName: \"foo\",\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{Name: \"foo\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"container volumeMounts matching prefix -token- should be deleted\",\n\t\t\tobj: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pod-1\"},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tServiceAccountName: \"foo\",\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{Name: \"foo\"},\n\t\t\t\t\t\t{Name: \"foo-token-foo\"},\n\t\t\t\t\t},\n\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tVolumeMounts: []corev1api.VolumeMount{\n\t\t\t\t\t\t\t\t{Name: \"foo\"},\n\t\t\t\t\t\t\t\t{Name: \"foo-token-foo\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedRes: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pod-1\"},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tServiceAccountName: \"foo\",\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{Name: \"foo\"},\n\t\t\t\t\t},\n\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tVolumeMounts: []corev1api.VolumeMount{\n\t\t\t\t\t\t\t\t{Name: \"foo\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"initContainer volumeMounts matching prefix -token- should be deleted\",\n\t\t\tobj: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pod-1\"},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tServiceAccountName: \"foo\",\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{Name: \"foo\"},\n\t\t\t\t\t\t{Name: \"foo-token-foo\"},\n\t\t\t\t\t},\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tVolumeMounts: []corev1api.VolumeMount{\n\t\t\t\t\t\t\t\t{Name: \"foo\"},\n\t\t\t\t\t\t\t\t{Name: \"foo-token-foo\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedRes: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pod-1\"},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tServiceAccountName: \"foo\",\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{Name: \"foo\"},\n\t\t\t\t\t},\n\t\t\t\t\tInitContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tVolumeMounts: []corev1api.VolumeMount{\n\t\t\t\t\t\t\t\t{Name: \"foo\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"containers and initContainers with no volume mounts should not error\",\n\t\t\tobj: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pod-1\"},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tServiceAccountName: \"foo\",\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{Name: \"foo\"},\n\t\t\t\t\t\t{Name: \"foo-token-foo\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedRes: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pod-1\"},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tServiceAccountName: \"foo\",\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{Name: \"foo\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"test priority class\",\n\t\t\tobj: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pod-1\"},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tServiceAccountName: \"foo\",\n\t\t\t\t\tPriorityClassName: \"testPriorityClass\",\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{Name: \"foo\"},\n\t\t\t\t\t\t{Name: \"foo-token-foo\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedRes: corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pod-1\"},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tServiceAccountName: \"foo\",\n\t\t\t\t\tPriorityClassName: \"testPriorityClass\",\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{Name: \"foo\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tadditionalItems: []velero.ResourceIdentifier{\n\t\t\t\t{GroupResource: kuberesource.PriorityClasses,\n\t\t\t\t\tName: \"testPriorityClass\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\taction := NewPodAction(velerotest.NewLogger())\n\n\t\t\tunstructuredPod, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&test.obj)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tres, err := action.Execute(&velero.RestoreItemActionExecuteInput{\n\t\t\t\tItem: &unstructured.Unstructured{Object: unstructuredPod},\n\t\t\t\tItemFromBackup: &unstructured.Unstructured{Object: unstructuredPod},\n\t\t\t\tRestore: nil,\n\t\t\t})\n\n\t\t\tif test.expectedErr {\n\t\t\t\tassert.Error(t, err, \"expected an error\")\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NoError(t, err, \"expected no error, got %v\", err)\n\n\t\t\tvar pod corev1api.Pod\n\t\t\trequire.NoError(t, runtime.DefaultUnstructuredConverter.FromUnstructured(res.UpdatedItem.UnstructuredContent(), &pod))\n\n\t\t\tassert.Equal(t, test.expectedRes, pod)\n\t\t\tassert.Equal(t, test.additionalItems, res.AdditionalItems)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/restore/actions/pod_volume_restore_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strings\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\tcorev1client \"k8s.io/client-go/kubernetes/typed/core/v1\"\n\tctrlclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tveleroimage \"github.com/vmware-tanzu/velero/internal/velero\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework/common\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/podvolume\"\n\t\"github.com/vmware-tanzu/velero/pkg/restorehelper\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\tveleroutil \"github.com/vmware-tanzu/velero/pkg/util/velero\"\n)\n\nconst (\n\tdefaultCPURequestLimit = \"100m\"\n\tdefaultMemRequestLimit = \"128Mi\"\n\tdefaultCommand = \"/velero-restore-helper\"\n\trestoreHelperUID = 1000\n)\n\ntype PodVolumeRestoreAction struct {\n\tlogger logrus.FieldLogger\n\tclient corev1client.ConfigMapInterface\n\tcrClient ctrlclient.Client\n\tveleroImage string\n}\n\nfunc NewPodVolumeRestoreAction(logger logrus.FieldLogger, client corev1client.ConfigMapInterface, crClient ctrlclient.Client, namespace string) (*PodVolumeRestoreAction, error) {\n\tdeployment := &appsv1api.Deployment{}\n\tif err := crClient.Get(context.TODO(), types.NamespacedName{Name: \"velero\", Namespace: namespace}, deployment); err != nil {\n\t\treturn nil, err\n\t}\n\timage := veleroutil.GetVeleroServerImage(deployment)\n\treturn &PodVolumeRestoreAction{\n\t\tlogger: logger,\n\t\tclient: client,\n\t\tcrClient: crClient,\n\t\tveleroImage: image,\n\t}, nil\n}\n\nfunc (a *PodVolumeRestoreAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"pods\"},\n\t}, nil\n}\n\nfunc (a *PodVolumeRestoreAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\ta.logger.Info(\"Executing PodVolumeRestoreAction\")\n\tdefer a.logger.Info(\"Done executing PodVolumeRestoreAction\")\n\n\tvar pod corev1api.Pod\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(input.Item.UnstructuredContent(), &pod); err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to convert pod from runtime.Unstructured\")\n\t}\n\n\t// At the point when this function is called, the namespace mapping for the restore\n\t// has not yet been applied to `input.Item` so we can't perform a reverse-lookup in\n\t// the namespace mapping in the restore spec. Instead, use the pod from the backup\n\t// so that if the mapping is applied earlier, we still use the correct namespace.\n\tvar podFromBackup corev1api.Pod\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(input.ItemFromBackup.UnstructuredContent(), &podFromBackup); err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to convert source pod from runtime.Unstructured\")\n\t}\n\n\tlog := a.logger.WithField(\"pod\", kube.NamespaceAndName(&pod))\n\n\topts := &ctrlclient.ListOptions{\n\t\tLabelSelector: label.NewSelectorForBackup(input.Restore.Spec.BackupName),\n\t}\n\tpodVolumeBackupList := new(velerov1api.PodVolumeBackupList)\n\tif err := a.crClient.List(context.TODO(), podVolumeBackupList, opts); err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tvar podVolumeBackups []*velerov1api.PodVolumeBackup\n\tfor i := range podVolumeBackupList.Items {\n\t\tpodVolumeBackups = append(podVolumeBackups, &podVolumeBackupList.Items[i])\n\t}\n\t// Remove all existing restore-wait init containers first to prevent duplicates\n\t// This ensures that even if the pod was previously restored with file system backup\n\t// but now backed up with native datamover or CSI, the unnecessary init container is removed\n\tvar filteredInitContainers []corev1api.Container\n\tremovedCount := 0\n\tfor _, initContainer := range pod.Spec.InitContainers {\n\t\tif initContainer.Name != restorehelper.WaitInitContainer && initContainer.Name != restorehelper.WaitInitContainerLegacy {\n\t\t\tfilteredInitContainers = append(filteredInitContainers, initContainer)\n\t\t} else {\n\t\t\tremovedCount++\n\t\t}\n\t}\n\tpod.Spec.InitContainers = filteredInitContainers\n\tif removedCount > 0 {\n\t\tlog.Infof(\"Removed %d existing restore-wait init container(s)\", removedCount)\n\t}\n\n\tvolumeSnapshots := podvolume.GetVolumeBackupsForPod(podVolumeBackups, &pod, podFromBackup.Namespace)\n\tif len(volumeSnapshots) == 0 {\n\t\tlog.Debug(\"No pod volume backups found for pod\")\n\t\tres, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&pod)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrap(err, \"unable to convert pod to runtime.Unstructured\")\n\t\t}\n\t\treturn velero.NewRestoreItemActionExecuteOutput(&unstructured.Unstructured{Object: res}), nil\n\t}\n\n\tlog.Info(\"Pod volume backups for pod found\")\n\tlog.Debugf(\"Getting plugin config\")\n\tconfig, err := common.GetPluginConfig(common.PluginKindRestoreItemAction, \"velero.io/pod-volume-restore\", a.client)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\timage := getImage(log, config, a.veleroImage)\n\tlog.Infof(\"Using image %q\", image)\n\n\tcpuRequest, memRequest := getResourceRequests(log, config)\n\tcpuLimit, memLimit := getResourceLimits(log, config)\n\tif cpuRequest == \"\" {\n\t\tcpuRequest = defaultCPURequestLimit\n\t}\n\tif cpuLimit == \"\" {\n\t\tcpuLimit = defaultCPURequestLimit\n\t}\n\tif memRequest == \"\" {\n\t\tmemRequest = defaultMemRequestLimit\n\t}\n\tif memLimit == \"\" {\n\t\tmemLimit = defaultMemRequestLimit\n\t}\n\n\tresourceReqs, err := kube.ParseCPUAndMemoryResources(\n\t\tcpuRequest,\n\t\tmemRequest,\n\t\tcpuLimit,\n\t\tmemLimit,\n\t)\n\tif err != nil {\n\t\tlog.Errorf(\"couldn't parse resource requirements: %s.\", err)\n\t\tresourceReqs, _ = kube.ParseCPUAndMemoryResources(\n\t\t\tdefaultCPURequestLimit,\n\t\t\tdefaultMemRequestLimit,\n\t\t\tdefaultCPURequestLimit,\n\t\t\tdefaultMemRequestLimit,\n\t\t)\n\t}\n\n\trunAsUser, runAsGroup, allowPrivilegeEscalation, secCtx := getSecurityContext(log, config)\n\n\tvar securityContext corev1api.SecurityContext\n\tsecurityContextSet := false\n\t// Use securityContext settings from configmap if available\n\tif runAsUser != \"\" || runAsGroup != \"\" || allowPrivilegeEscalation != \"\" || secCtx != \"\" {\n\t\tsecurityContext, err = kube.ParseSecurityContext(runAsUser, runAsGroup, allowPrivilegeEscalation, secCtx)\n\t\tif err != nil {\n\t\t\tlog.Errorf(\"Using default securityContext values, couldn't parse securityContext requirements: %s.\", err)\n\t\t} else {\n\t\t\tsecurityContextSet = true\n\t\t}\n\t}\n\t// if securityContext configmap is unavailable but first container in pod has a SecurityContext set, then copy this security context\n\tif !securityContextSet && len(pod.Spec.Containers) != 0 && pod.Spec.Containers[0].SecurityContext != nil {\n\t\tsecurityContext = *pod.Spec.Containers[0].SecurityContext.DeepCopy()\n\t\tsecurityContextSet = true\n\t}\n\tif !securityContextSet {\n\t\tsecurityContext = defaultSecurityCtx()\n\t}\n\n\tinitContainerBuilder := newRestoreInitContainerBuilder(image, string(input.Restore.UID))\n\tinitContainerBuilder.Resources(&resourceReqs)\n\tinitContainerBuilder.SecurityContext(&securityContext)\n\n\tfor volumeName := range volumeSnapshots {\n\t\tmount := &corev1api.VolumeMount{\n\t\t\tName: volumeName,\n\t\t\tMountPath: \"/restores/\" + volumeName,\n\t\t}\n\t\tinitContainerBuilder.VolumeMounts(mount)\n\t}\n\tinitContainerBuilder.Command(getCommand(log, config))\n\n\tinitContainer := *initContainerBuilder.Result()\n\t// Since we've already removed all restore-wait init containers above,\n\t// we can simply prepend the new init container\n\tpod.Spec.InitContainers = append([]corev1api.Container{initContainer}, pod.Spec.InitContainers...)\n\n\tres, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&pod)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to convert pod to runtime.Unstructured\")\n\t}\n\n\treturn velero.NewRestoreItemActionExecuteOutput(&unstructured.Unstructured{Object: res}), nil\n}\n\nfunc getCommand(log logrus.FieldLogger, config *corev1api.ConfigMap) []string {\n\tif config == nil {\n\t\tlog.Debug(\"No config found for plugin\")\n\t\treturn []string{defaultCommand}\n\t}\n\n\tif config.Data[\"command\"] == \"\" {\n\t\tlog.Debugf(\"No custom command configured\")\n\t\treturn []string{defaultCommand}\n\t}\n\n\tlog.Debugf(\"Using custom command %s\", config.Data[\"command\"])\n\treturn []string{config.Data[\"command\"]}\n}\n\nfunc getImage(log logrus.FieldLogger, config *corev1api.ConfigMap, defaultImage string) string {\n\tif config == nil {\n\t\tlog.Debug(\"No config found for plugin\")\n\t\treturn defaultImage\n\t}\n\n\timage := config.Data[\"image\"]\n\tif image == \"\" {\n\t\tlog.Debugf(\"No custom image configured\")\n\t\treturn defaultImage\n\t}\n\n\tlog = log.WithField(\"image\", image)\n\n\tparts := strings.Split(image, \"/\")\n\n\tif len(parts) == 1 {\n\t\t// Image supplied without registry part\n\t\tlog.Infof(\"Plugin config contains image name without registry name. Using default init container image: %q\", defaultImage)\n\t\treturn defaultImage\n\t}\n\n\tif !(strings.Contains(parts[len(parts)-1], \":\")) {\n\t\ttag := veleroimage.ImageTag()\n\t\t// tag-less image name: add default image tag for this version of Velero\n\t\tlog.Infof(\"Plugin config contains image name without tag. Adding tag: %q\", tag)\n\t\treturn fmt.Sprintf(\"%s:%s\", image, tag)\n\t}\n\t// tagged image name\n\tlog.Debugf(\"Plugin config contains image name with tag\")\n\treturn image\n}\n\n// getResourceRequests extracts the CPU and memory requests from a ConfigMap.\n// The 0 values are valid if the keys are not present\nfunc getResourceRequests(log logrus.FieldLogger, config *corev1api.ConfigMap) (string, string) {\n\tif config == nil {\n\t\tlog.Debug(\"No config found for plugin\")\n\t\treturn \"\", \"\"\n\t}\n\n\treturn config.Data[\"cpuRequest\"], config.Data[\"memRequest\"]\n}\n\n// getResourceLimits extracts the CPU and memory limits from a ConfigMap.\n// The 0 values are valid if the keys are not present\nfunc getResourceLimits(log logrus.FieldLogger, config *corev1api.ConfigMap) (string, string) {\n\tif config == nil {\n\t\tlog.Debug(\"No config found for plugin\")\n\t\treturn \"\", \"\"\n\t}\n\n\treturn config.Data[\"cpuLimit\"], config.Data[\"memLimit\"]\n}\n\n// getSecurityContext extracts securityContext runAsUser, runAsGroup, allowPrivilegeEscalation, and securityContext from a ConfigMap.\nfunc getSecurityContext(log logrus.FieldLogger, config *corev1api.ConfigMap) (string, string, string, string) {\n\tif config == nil {\n\t\tlog.Debug(\"No config found for plugin\")\n\t\treturn \"\", \"\", \"\", \"\"\n\t}\n\n\treturn config.Data[\"secCtxRunAsUser\"],\n\t\tconfig.Data[\"secCtxRunAsGroup\"],\n\t\tconfig.Data[\"secCtxAllowPrivilegeEscalation\"],\n\t\tconfig.Data[\"secCtx\"]\n}\n\nfunc newRestoreInitContainerBuilder(image, restoreUID string) *builder.ContainerBuilder {\n\treturn builder.ForContainer(restorehelper.WaitInitContainer, image).\n\t\tArgs(restoreUID).\n\t\tEnv([]*corev1api.EnvVar{\n\t\t\t{\n\t\t\t\tName: \"POD_NAMESPACE\",\n\t\t\t\tValueFrom: &corev1api.EnvVarSource{\n\t\t\t\t\tFieldRef: &corev1api.ObjectFieldSelector{\n\t\t\t\t\t\tFieldPath: \"metadata.namespace\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\t{\n\t\t\t\tName: \"POD_NAME\",\n\t\t\t\tValueFrom: &corev1api.EnvVarSource{\n\t\t\t\t\tFieldRef: &corev1api.ObjectFieldSelector{\n\t\t\t\t\t\tFieldPath: \"metadata.name\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t}...)\n}\n\n// defaultSecurityCtx returns a default security context for the init container, which has the level \"restricted\" per\n// Pod Security Standards.\nfunc defaultSecurityCtx() corev1api.SecurityContext {\n\tuid := int64(restoreHelperUID)\n\treturn corev1api.SecurityContext{\n\t\tAllowPrivilegeEscalation: boolptr.False(),\n\t\tCapabilities: &corev1api.Capabilities{\n\t\t\tDrop: []corev1api.Capability{\"ALL\"},\n\t\t},\n\t\tSeccompProfile: &corev1api.SeccompProfile{\n\t\t\tType: corev1api.SeccompProfileTypeRuntimeDefault,\n\t\t},\n\t\tRunAsUser: &uid,\n\t\tRunAsNonRoot: boolptr.True(),\n\t}\n}\n" }, { "path": "pkg/restore/actions/pod_volume_restore_action_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"sort\"\n\t\"testing\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes/fake\"\n\tcrfake \"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/buildinfo\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\n\t\"k8s.io/client-go/kubernetes/scheme\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/restorehelper\"\n)\n\nfunc TestGetImage(t *testing.T) {\n\tconfigMapWithData := func(key, val string) *corev1api.ConfigMap {\n\t\treturn &corev1api.ConfigMap{\n\t\t\tData: map[string]string{\n\t\t\t\tkey: val,\n\t\t\t},\n\t\t}\n\t}\n\n\tdefaultImage := \"velero/velero:v1.0\"\n\n\ttests := []struct {\n\t\tname string\n\t\tconfigMap *corev1api.ConfigMap\n\t\tbuildInfoVersion string\n\t\twant string\n\t}{\n\t\t{\n\t\t\tname: \"nil config map returns default image\",\n\t\t\tconfigMap: nil,\n\t\t\twant: defaultImage,\n\t\t},\n\t\t{\n\t\t\tname: \"config map without 'image' key returns default image\",\n\t\t\tconfigMap: configMapWithData(\"non-matching-key\", \"val\"),\n\t\t\twant: defaultImage,\n\t\t},\n\t\t{\n\t\t\tname: \"config map without '/' in image name returns default image\",\n\t\t\tconfigMap: configMapWithData(\"image\", \"my-image\"),\n\t\t\twant: defaultImage,\n\t\t},\n\t\t{\n\t\t\tname: \"config map with untagged image returns image with buildinfo.Version as tag\",\n\t\t\tconfigMap: configMapWithData(\"image\", \"myregistry.io/my-image\"),\n\t\t\tbuildInfoVersion: \"buildinfo-version\",\n\t\t\twant: \"myregistry.io/my-image:buildinfo-version\",\n\t\t},\n\t\t{\n\t\t\tname: \"config map with untagged image and custom registry port with ':' returns image with buildinfo.Version as tag\",\n\t\t\tconfigMap: configMapWithData(\"image\", \"myregistry.io:34567/my-image\"),\n\t\t\tbuildInfoVersion: \"buildinfo-version\",\n\t\t\twant: \"myregistry.io:34567/my-image:buildinfo-version\",\n\t\t},\n\t\t{\n\t\t\tname: \"config map with tagged image returns tagged image\",\n\t\t\tconfigMap: configMapWithData(\"image\", \"myregistry.io/my-image:my-tag\"),\n\t\t\twant: \"myregistry.io/my-image:my-tag\",\n\t\t},\n\t\t{\n\t\t\tname: \"config map with tagged image and custom registry port with ':' returns tagged image\",\n\t\t\tconfigMap: configMapWithData(\"image\", \"myregistry.io:34567/my-image:my-tag\"),\n\t\t\twant: \"myregistry.io:34567/my-image:my-tag\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tif test.buildInfoVersion != \"\" {\n\t\t\t\toriginalVersion := buildinfo.Version\n\t\t\t\tbuildinfo.Version = test.buildInfoVersion\n\t\t\t\tdefer func() {\n\t\t\t\t\tbuildinfo.Version = originalVersion\n\t\t\t\t}()\n\t\t\t}\n\t\t\tassert.Equal(t, test.want, getImage(velerotest.NewLogger(), test.configMap, defaultImage))\n\t\t})\n\t}\n}\n\n// TestPodVolumeRestoreActionExecute tests the pod volume restore item action plugin's Execute method.\nfunc TestPodVolumeRestoreActionExecute(t *testing.T) {\n\tresourceReqs, _ := kube.ParseCPUAndMemoryResources(\n\t\tdefaultCPURequestLimit,\n\t\tdefaultMemRequestLimit,\n\t\tdefaultCPURequestLimit,\n\t\tdefaultMemRequestLimit,\n\t)\n\tid := int64(1000)\n\tsecurityContext := corev1api.SecurityContext{\n\t\tAllowPrivilegeEscalation: boolptr.False(),\n\t\tCapabilities: &corev1api.Capabilities{\n\t\t\tDrop: []corev1api.Capability{\"ALL\"},\n\t\t},\n\t\tSeccompProfile: &corev1api.SeccompProfile{\n\t\t\tType: corev1api.SeccompProfileTypeRuntimeDefault,\n\t\t},\n\t\tRunAsUser: &id,\n\t\tRunAsNonRoot: boolptr.True(),\n\t}\n\tcustomID := int64(44444)\n\tcustomSecurityContext := corev1api.SecurityContext{\n\t\tAllowPrivilegeEscalation: boolptr.False(),\n\t\tCapabilities: &corev1api.Capabilities{\n\t\t\tDrop: []corev1api.Capability{\"ALL\"},\n\t\t},\n\t\tSeccompProfile: &corev1api.SeccompProfile{\n\t\t\tType: corev1api.SeccompProfileTypeRuntimeDefault,\n\t\t},\n\t\tRunAsUser: &customID,\n\t\tRunAsNonRoot: boolptr.True(),\n\t}\n\n\tvar (\n\t\trestoreName = \"my-restore\"\n\t\tbackupName = \"test-backup\"\n\t\tveleroNs = \"velero\"\n\t)\n\n\tdefaultRestoreHelperImage := \"velero/velero:v1.0\"\n\n\ttests := []struct {\n\t\tname string\n\t\tpod *corev1api.Pod\n\t\tpodFromBackup *corev1api.Pod\n\t\tpodVolumeBackups []runtime.Object\n\t\twant *corev1api.Pod\n\t}{\n\t\t{\n\t\t\tname: \"Restoring pod with no other initContainers adds the restore initContainer when volumes need file system restores\",\n\t\t\tpod: builder.ForPod(\"ns-1\", \"my-pod\").\n\t\t\t\tObjectMeta(builder.WithAnnotations(\"snapshot.velero.io/myvol\", \"\")).\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"myvol\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t).\n\t\t\t\tResult(),\n\t\t\tpodVolumeBackups: []runtime.Object{\n\t\t\t\tbuilder.ForPodVolumeBackup(veleroNs, \"pvb-1\").\n\t\t\t\t\tPodName(\"my-pod\").\n\t\t\t\t\tPodNamespace(\"ns-1\").\n\t\t\t\t\tVolume(\"myvol\").\n\t\t\t\t\tObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, backupName)).\n\t\t\t\t\tSnapshotID(\"foo\").\n\t\t\t\t\tResult(),\n\t\t\t},\n\t\t\twant: builder.ForPod(\"ns-1\", \"my-pod\").\n\t\t\t\tObjectMeta(\n\t\t\t\t\tbuilder.WithAnnotations(\"snapshot.velero.io/myvol\", \"\")).\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"myvol\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t).\n\t\t\t\tInitContainers(\n\t\t\t\t\tnewRestoreInitContainerBuilder(defaultRestoreHelperImage, \"\").\n\t\t\t\t\t\tResources(&resourceReqs).\n\t\t\t\t\t\tSecurityContext(&securityContext).\n\t\t\t\t\t\tVolumeMounts(builder.ForVolumeMount(\"myvol\", \"/restores/myvol\").Result()).\n\t\t\t\t\t\tCommand([]string{\"/velero-restore-helper\"}).Result()).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Restoring pod with other initContainers adds the restore initContainer as the first one when volumes need file system restores\",\n\t\t\tpod: builder.ForPod(\"ns-1\", \"my-pod\").\n\t\t\t\tObjectMeta(\n\t\t\t\t\tbuilder.WithAnnotations(\"snapshot.velero.io/myvol\", \"\")).\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"myvol\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t).\n\t\t\t\tInitContainers(builder.ForContainer(\"first-container\", \"\").Result()).\n\t\t\t\tResult(),\n\t\t\tpodVolumeBackups: []runtime.Object{\n\t\t\t\tbuilder.ForPodVolumeBackup(veleroNs, \"pvb-1\").\n\t\t\t\t\tPodName(\"my-pod\").\n\t\t\t\t\tPodNamespace(\"ns-1\").\n\t\t\t\t\tVolume(\"myvol\").\n\t\t\t\t\tObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, backupName)).\n\t\t\t\t\tSnapshotID(\"foo\").\n\t\t\t\t\tResult(),\n\t\t\t},\n\t\t\twant: builder.ForPod(\"ns-1\", \"my-pod\").\n\t\t\t\tObjectMeta(\n\t\t\t\t\tbuilder.WithAnnotations(\"snapshot.velero.io/myvol\", \"\")).\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"myvol\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t).\n\t\t\t\tInitContainers(\n\t\t\t\t\tnewRestoreInitContainerBuilder(defaultRestoreHelperImage, \"\").\n\t\t\t\t\t\tResources(&resourceReqs).\n\t\t\t\t\t\tSecurityContext(&securityContext).\n\t\t\t\t\t\tVolumeMounts(builder.ForVolumeMount(\"myvol\", \"/restores/myvol\").Result()).\n\t\t\t\t\t\tCommand([]string{\"/velero-restore-helper\"}).Result(),\n\t\t\t\t\tbuilder.ForContainer(\"first-container\", \"\").Result()).\n\t\t\t\tResult(),\n\t\t},\n\t\t{\n\t\t\tname: \"Restoring pod with other initContainers adds the restore initContainer as the first one using PVB to identify the volumes and not annotations\",\n\t\t\tpod: builder.ForPod(\"ns-1\", \"my-pod\").\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"vol-1\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForVolume(\"vol-2\").PersistentVolumeClaimSource(\"pvc-2\").Result(),\n\t\t\t\t).\n\t\t\t\tObjectMeta(\n\t\t\t\t\tbuilder.WithAnnotations(\"snapshot.velero.io/not-used\", \"\")).\n\t\t\t\tInitContainers(builder.ForContainer(\"first-container\", \"\").Result()).\n\t\t\t\tResult(),\n\t\t\tpodVolumeBackups: []runtime.Object{\n\t\t\t\tbuilder.ForPodVolumeBackup(veleroNs, \"pvb-1\").\n\t\t\t\t\tPodName(\"my-pod\").\n\t\t\t\t\tPodNamespace(\"ns-1\").\n\t\t\t\t\tVolume(\"vol-1\").\n\t\t\t\t\tObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, backupName)).\n\t\t\t\t\tSnapshotID(\"foo\").\n\t\t\t\t\tResult(),\n\t\t\t\tbuilder.ForPodVolumeBackup(veleroNs, \"pvb-2\").\n\t\t\t\t\tPodName(\"my-pod\").\n\t\t\t\t\tPodNamespace(\"ns-1\").\n\t\t\t\t\tVolume(\"vol-2\").\n\t\t\t\t\tObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, backupName)).\n\t\t\t\t\tSnapshotID(\"foo\").\n\t\t\t\t\tResult(),\n\t\t\t},\n\t\t\twant: builder.ForPod(\"ns-1\", \"my-pod\").\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"vol-1\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForVolume(\"vol-2\").PersistentVolumeClaimSource(\"pvc-2\").Result(),\n\t\t\t\t).\n\t\t\t\tObjectMeta(\n\t\t\t\t\tbuilder.WithAnnotations(\"snapshot.velero.io/not-used\", \"\")).\n\t\t\t\tInitContainers(\n\t\t\t\t\tnewRestoreInitContainerBuilder(defaultRestoreHelperImage, \"\").\n\t\t\t\t\t\tResources(&resourceReqs).\n\t\t\t\t\t\tSecurityContext(&securityContext).\n\t\t\t\t\t\tVolumeMounts(builder.ForVolumeMount(\"vol-1\", \"/restores/vol-1\").Result(), builder.ForVolumeMount(\"vol-2\", \"/restores/vol-2\").Result()).\n\t\t\t\t\t\tCommand([]string{\"/velero-restore-helper\"}).Result(),\n\t\t\t\t\tbuilder.ForContainer(\"first-container\", \"\").Result()).\n\t\t\t\tResult(),\n\t\t},\n\t\t{\n\t\t\tname: \"Restoring pod in another namespace adds the restore initContainer and uses the namespace of the backup pod for matching PVBs\",\n\t\t\tpod: builder.ForPod(\"new-ns\", \"my-pod\").\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"vol-1\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForVolume(\"vol-2\").PersistentVolumeClaimSource(\"pvc-2\").Result(),\n\t\t\t\t).\n\t\t\t\tResult(),\n\t\t\tpodFromBackup: builder.ForPod(\"original-ns\", \"my-pod\").\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"vol-1\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForVolume(\"vol-2\").PersistentVolumeClaimSource(\"pvc-2\").Result(),\n\t\t\t\t).\n\t\t\t\tResult(),\n\t\t\tpodVolumeBackups: []runtime.Object{\n\t\t\t\tbuilder.ForPodVolumeBackup(veleroNs, \"pvb-1\").\n\t\t\t\t\tPodName(\"my-pod\").\n\t\t\t\t\tPodNamespace(\"original-ns\").\n\t\t\t\t\tVolume(\"vol-1\").\n\t\t\t\t\tObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, backupName)).\n\t\t\t\t\tSnapshotID(\"foo\").\n\t\t\t\t\tResult(),\n\t\t\t\tbuilder.ForPodVolumeBackup(veleroNs, \"pvb-2\").\n\t\t\t\t\tPodName(\"my-pod\").\n\t\t\t\t\tPodNamespace(\"original-ns\").\n\t\t\t\t\tVolume(\"vol-2\").\n\t\t\t\t\tObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, backupName)).\n\t\t\t\t\tSnapshotID(\"foo\").\n\t\t\t\t\tResult(),\n\t\t\t},\n\t\t\twant: builder.ForPod(\"new-ns\", \"my-pod\").\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"vol-1\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForVolume(\"vol-2\").PersistentVolumeClaimSource(\"pvc-2\").Result(),\n\t\t\t\t).\n\t\t\t\tInitContainers(\n\t\t\t\t\tnewRestoreInitContainerBuilder(defaultRestoreHelperImage, \"\").\n\t\t\t\t\t\tResources(&resourceReqs).\n\t\t\t\t\t\tSecurityContext(&securityContext).\n\t\t\t\t\t\tVolumeMounts(builder.ForVolumeMount(\"vol-1\", \"/restores/vol-1\").Result(), builder.ForVolumeMount(\"vol-2\", \"/restores/vol-2\").Result()).\n\t\t\t\t\t\tCommand([]string{\"/velero-restore-helper\"}).Result()).\n\t\t\t\tResult(),\n\t\t},\n\t\t{\n\t\t\tname: \"Restoring pod with custom container SecurityContext uses this SecurityContext for the restore initContainer when volumes need file system restores\",\n\t\t\tpod: builder.ForPod(\"ns-1\", \"my-pod\").\n\t\t\t\tObjectMeta(\n\t\t\t\t\tbuilder.WithAnnotations(\"snapshot.velero.io/myvol\", \"\")).\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"myvol\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t).\n\t\t\t\tContainers(\n\t\t\t\t\tbuilder.ForContainer(\"app-container\", \"app-image\").\n\t\t\t\t\t\tSecurityContext(&customSecurityContext).Result()).\n\t\t\t\tResult(),\n\t\t\tpodVolumeBackups: []runtime.Object{\n\t\t\t\tbuilder.ForPodVolumeBackup(veleroNs, \"pvb-1\").\n\t\t\t\t\tPodName(\"my-pod\").\n\t\t\t\t\tPodNamespace(\"ns-1\").\n\t\t\t\t\tVolume(\"myvol\").\n\t\t\t\t\tObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, backupName)).\n\t\t\t\t\tSnapshotID(\"foo\").\n\t\t\t\t\tResult(),\n\t\t\t},\n\t\t\twant: builder.ForPod(\"ns-1\", \"my-pod\").\n\t\t\t\tObjectMeta(\n\t\t\t\t\tbuilder.WithAnnotations(\"snapshot.velero.io/myvol\", \"\")).\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"myvol\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t).\n\t\t\t\tContainers(\n\t\t\t\t\tbuilder.ForContainer(\"app-container\", \"app-image\").\n\t\t\t\t\t\tSecurityContext(&customSecurityContext).Result()).\n\t\t\t\tInitContainers(\n\t\t\t\t\tnewRestoreInitContainerBuilder(defaultRestoreHelperImage, \"\").\n\t\t\t\t\t\tResources(&resourceReqs).\n\t\t\t\t\t\tSecurityContext(&customSecurityContext).\n\t\t\t\t\t\tVolumeMounts(builder.ForVolumeMount(\"myvol\", \"/restores/myvol\").Result()).\n\t\t\t\t\t\tCommand([]string{\"/velero-restore-helper\"}).Result()).Result(),\n\t\t},\n\t}\n\n\tveleroDeployment := &appsv1api.Deployment{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: appsv1api.SchemeGroupVersion.String(),\n\t\t\tKind: \"Deployment\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"velero\",\n\t\t\tName: \"velero\",\n\t\t},\n\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tImage: \"velero/velero:v1.0\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tclientset := fake.NewSimpleClientset()\n\n\t\t\tobjects := []runtime.Object{veleroDeployment}\n\t\t\tobjects = append(objects, tc.podVolumeBackups...)\n\t\t\tcrClient := velerotest.NewFakeControllerRuntimeClient(t, objects...)\n\n\t\t\tunstructuredPod, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.pod)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Default to using the same pod for both Item and ItemFromBackup if podFromBackup not provided\n\t\t\tvar unstructuredPodFromBackup map[string]any\n\t\t\tif tc.podFromBackup != nil {\n\t\t\t\tunstructuredPodFromBackup, err = runtime.DefaultUnstructuredConverter.ToUnstructured(tc.podFromBackup)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tunstructuredPodFromBackup = unstructuredPod\n\t\t\t}\n\n\t\t\tinput := &velero.RestoreItemActionExecuteInput{\n\t\t\t\tItem: &unstructured.Unstructured{\n\t\t\t\t\tObject: unstructuredPod,\n\t\t\t\t},\n\t\t\t\tItemFromBackup: &unstructured.Unstructured{\n\t\t\t\t\tObject: unstructuredPodFromBackup,\n\t\t\t\t},\n\t\t\t\tRestore: builder.ForRestore(veleroNs, restoreName).\n\t\t\t\t\tBackup(backupName).\n\t\t\t\t\tPhase(velerov1api.RestorePhaseInProgress).\n\t\t\t\t\tResult(),\n\t\t\t}\n\n\t\t\ta, err := NewPodVolumeRestoreAction(\n\t\t\t\tlogrus.StandardLogger(),\n\t\t\t\tclientset.CoreV1().ConfigMaps(veleroNs),\n\t\t\t\tcrClient,\n\t\t\t\t\"velero\",\n\t\t\t)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// method under test\n\t\t\tres, err := a.Execute(input)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tupdatedPod := new(corev1api.Pod)\n\t\t\trequire.NoError(t, runtime.DefaultUnstructuredConverter.FromUnstructured(res.UpdatedItem.UnstructuredContent(), updatedPod))\n\n\t\t\tfor _, container := range tc.want.Spec.InitContainers {\n\t\t\t\tsort.Slice(container.VolumeMounts, func(i, j int) bool {\n\t\t\t\t\treturn container.VolumeMounts[i].Name < container.VolumeMounts[j].Name\n\t\t\t\t})\n\t\t\t}\n\t\t\tfor _, container := range updatedPod.Spec.InitContainers {\n\t\t\t\tsort.Slice(container.VolumeMounts, func(i, j int) bool {\n\t\t\t\t\treturn container.VolumeMounts[i].Name < container.VolumeMounts[j].Name\n\t\t\t\t})\n\t\t\t}\n\n\t\t\tassert.Equal(t, tc.want, updatedPod)\n\t\t})\n\t}\n}\n\nfunc TestGetCommand(t *testing.T) {\n\tconfigMapWithData := func(key, val string) *corev1api.ConfigMap {\n\t\treturn &corev1api.ConfigMap{\n\t\t\tData: map[string]string{\n\t\t\t\tkey: val,\n\t\t\t},\n\t\t}\n\t}\n\ttestCases := []struct {\n\t\tname string\n\t\tconfigMap *corev1api.ConfigMap\n\t\texpected []string\n\t}{\n\t\t{\n\t\t\tname: \"should get default command when config key is missing\",\n\t\t\tconfigMap: configMapWithData(\"non-matching-key\", \"val\"),\n\t\t\texpected: []string{defaultCommand},\n\t\t},\n\t\t{\n\t\t\tname: \"should get default command when config key is empty\",\n\t\t\tconfigMap: configMapWithData(\"command\", \"\"),\n\t\t\texpected: []string{defaultCommand},\n\t\t},\n\t\t{\n\t\t\tname: \"should get default command when config is nil\",\n\t\t\tconfigMap: nil,\n\t\t\texpected: []string{defaultCommand},\n\t\t},\n\t\t{\n\t\t\tname: \"should get command from config\",\n\t\t\tconfigMap: configMapWithData(\"command\", \"foobarbz\"),\n\t\t\texpected: []string{\"foobarbz\"},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactual := getCommand(velerotest.NewLogger(), tc.configMap)\n\t\t\tassert.Equal(t, tc.expected, actual)\n\t\t})\n\t}\n}\n\n// This tests that restore-wait is added when file system restore volume exists, nothing added otherwise,\n// and removed if it exists but is not needed.\n// issue: 8870\nfunc TestPodVolumeRestoreActionExecuteWithFileSystemShouldAddWaitInitContainer(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpod *corev1api.Pod\n\t\tpodFromBackup *corev1api.Pod\n\t\tpodVolumeBackups []*velerov1api.PodVolumeBackup\n\t\trestore *velerov1api.Restore\n\t\texpectedInitContainers int\n\t\texpectedError error\n\t}{\n\t\t{\n\t\t\tname: \"no pod volume backups results in no init container\",\n\t\t\tpod: builder.ForPod(\"ns\", \"pod\").\n\t\t\t\tObjectMeta(builder.WithUID(\"pod-uid\")).\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"volume-1\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForVolume(\"volume-2\").PersistentVolumeClaimSource(\"pvc-2\").Result(),\n\t\t\t\t).\n\t\t\t\tResult(),\n\t\t\tpodFromBackup: builder.ForPod(\"ns\", \"pod\").\n\t\t\t\tObjectMeta(builder.WithUID(\"pod-uid\")).\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"volume-1\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForVolume(\"volume-2\").PersistentVolumeClaimSource(\"pvc-2\").Result(),\n\t\t\t\t).\n\t\t\t\tResult(),\n\t\t\tpodVolumeBackups: nil,\n\t\t\trestore: builder.ForRestore(\"velero\", \"restore-1\").Backup(\"test-backup\").Result(),\n\t\t\texpectedInitContainers: 0,\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"pod volume backups that don't match pod's volumes results in no init container\",\n\t\t\tpod: builder.ForPod(\"ns\", \"pod\").\n\t\t\t\tObjectMeta(builder.WithUID(\"pod-uid\")).\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"volume-1\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForVolume(\"volume-2\").PersistentVolumeClaimSource(\"pvc-2\").Result(),\n\t\t\t\t).\n\t\t\t\tResult(),\n\t\t\tpodFromBackup: builder.ForPod(\"ns\", \"pod\").\n\t\t\t\tObjectMeta(builder.WithUID(\"pod-uid\")).\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"volume-1\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForVolume(\"volume-2\").PersistentVolumeClaimSource(\"pvc-2\").Result(),\n\t\t\t\t).\n\t\t\t\tResult(),\n\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-1\").\n\t\t\t\t\tObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, \"test-backup\")).\n\t\t\t\t\tPodName(\"different-pod\").\n\t\t\t\t\tPodNamespace(\"ns\").\n\t\t\t\t\tVolume(\"volume-1\").\n\t\t\t\t\tSnapshotID(\"snapshot-1\").\n\t\t\t\t\tResult(),\n\t\t\t},\n\t\t\trestore: builder.ForRestore(\"velero\", \"restore-1\").Backup(\"test-backup\").Result(),\n\t\t\texpectedInitContainers: 0,\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"matching pod volume backup results in init container being added\",\n\t\t\tpod: builder.ForPod(\"ns\", \"pod\").\n\t\t\t\tObjectMeta(builder.WithUID(\"pod-uid\")).\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"volume-1\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForVolume(\"volume-2\").PersistentVolumeClaimSource(\"pvc-2\").Result(),\n\t\t\t\t).\n\t\t\t\tResult(),\n\t\t\tpodFromBackup: builder.ForPod(\"ns\", \"pod\").\n\t\t\t\tObjectMeta(builder.WithUID(\"pod-uid\")).\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"volume-1\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForVolume(\"volume-2\").PersistentVolumeClaimSource(\"pvc-2\").Result(),\n\t\t\t\t).\n\t\t\t\tResult(),\n\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-1\").\n\t\t\t\t\tObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, \"test-backup\")).\n\t\t\t\t\tPodName(\"pod\").\n\t\t\t\t\tPodNamespace(\"ns\").\n\t\t\t\t\tVolume(\"volume-1\").\n\t\t\t\t\tSnapshotID(\"snapshot-1\").\n\t\t\t\t\tResult(),\n\t\t\t},\n\t\t\trestore: builder.ForRestore(\"velero\", \"restore-1\").Backup(\"test-backup\").Result(),\n\t\t\texpectedInitContainers: 1,\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"matching pod volume backup with matching pod name and namespace results in init container being added\",\n\t\t\tpod: builder.ForPod(\"ns\", \"pod\").\n\t\t\t\tObjectMeta(builder.WithUID(\"pod-uid\")).\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"volume-1\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForVolume(\"volume-2\").PersistentVolumeClaimSource(\"pvc-2\").Result(),\n\t\t\t\t).\n\t\t\t\tResult(),\n\t\t\tpodFromBackup: builder.ForPod(\"ns\", \"pod\").\n\t\t\t\tObjectMeta(builder.WithUID(\"pod-uid\")).\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"volume-1\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForVolume(\"volume-2\").PersistentVolumeClaimSource(\"pvc-2\").Result(),\n\t\t\t\t).\n\t\t\t\tResult(),\n\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-1\").\n\t\t\t\t\tObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, \"test-backup\")).\n\t\t\t\t\tPodName(\"pod\").\n\t\t\t\t\tPodNamespace(\"ns\").\n\t\t\t\t\tVolume(\"volume-1\").\n\t\t\t\t\tSnapshotID(\"snapshot-1\").\n\t\t\t\t\tResult(),\n\t\t\t},\n\t\t\trestore: builder.ForRestore(\"velero\", \"restore-1\").Backup(\"test-backup\").Result(),\n\t\t\texpectedInitContainers: 1,\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"existing init container is removed when no file system restore is needed\",\n\t\t\tpod: builder.ForPod(\"ns\", \"pod\").\n\t\t\t\tObjectMeta(builder.WithUID(\"pod-uid\")).\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"volume-1\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForVolume(\"volume-2\").PersistentVolumeClaimSource(\"pvc-2\").Result(),\n\t\t\t\t).\n\t\t\t\tInitContainers(\n\t\t\t\t\tbuilder.ForContainer(restorehelper.WaitInitContainer, \"velero/velero:latest\").\n\t\t\t\t\t\tCommand([]string{\"/velero-restore-helper\"}).\n\t\t\t\t\t\tArgs(\"restore-1\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t\tbuilder.ForContainer(\"another-init\", \"another-image\").Result(),\n\t\t\t\t).\n\t\t\t\tResult(),\n\t\t\tpodFromBackup: builder.ForPod(\"ns\", \"pod\").\n\t\t\t\tObjectMeta(builder.WithUID(\"pod-uid\")).\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"volume-1\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForVolume(\"volume-2\").PersistentVolumeClaimSource(\"pvc-2\").Result(),\n\t\t\t\t).\n\t\t\t\tResult(),\n\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\t// This PVB doesn't match the pod's name, so needsFileSystemRestore will be false\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-1\").\n\t\t\t\t\tObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, \"test-backup\")).\n\t\t\t\t\tPodName(\"different-pod\").\n\t\t\t\t\tPodNamespace(\"ns\").\n\t\t\t\t\tVolume(\"volume-1\").\n\t\t\t\t\tSnapshotID(\"snapshot-1\").\n\t\t\t\t\tResult(),\n\t\t\t},\n\t\t\trestore: builder.ForRestore(\"velero\", \"restore-1\").Backup(\"test-backup\").Result(),\n\t\t\texpectedInitContainers: 1, // Only the \"another-init\" container should remain\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"existing legacy init container is removed when no file system restore is needed\",\n\t\t\tpod: builder.ForPod(\"ns\", \"pod\").\n\t\t\t\tObjectMeta(builder.WithUID(\"pod-uid\")).\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"volume-1\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForVolume(\"volume-2\").PersistentVolumeClaimSource(\"pvc-2\").Result(),\n\t\t\t\t).\n\t\t\t\tInitContainers(\n\t\t\t\t\tbuilder.ForContainer(restorehelper.WaitInitContainerLegacy, \"velero/velero:latest\").\n\t\t\t\t\t\tCommand([]string{\"/velero-restore-helper\"}).\n\t\t\t\t\t\tArgs(\"restore-1\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t\tbuilder.ForContainer(\"another-init\", \"another-image\").Result(),\n\t\t\t\t).\n\t\t\t\tResult(),\n\t\t\tpodFromBackup: builder.ForPod(\"ns\", \"pod\").\n\t\t\t\tObjectMeta(builder.WithUID(\"pod-uid\")).\n\t\t\t\tVolumes(\n\t\t\t\t\tbuilder.ForVolume(\"volume-1\").PersistentVolumeClaimSource(\"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForVolume(\"volume-2\").PersistentVolumeClaimSource(\"pvc-2\").Result(),\n\t\t\t\t).\n\t\t\t\tResult(),\n\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\t// This PVB doesn't match the pod's name, so needsFileSystemRestore will be false\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-1\").\n\t\t\t\t\tObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, \"test-backup\")).\n\t\t\t\t\tPodName(\"different-pod\").\n\t\t\t\t\tPodNamespace(\"ns\").\n\t\t\t\t\tVolume(\"volume-1\").\n\t\t\t\t\tSnapshotID(\"snapshot-1\").\n\t\t\t\t\tResult(),\n\t\t\t},\n\t\t\trestore: builder.ForRestore(\"velero\", \"restore-1\").Backup(\"test-backup\").Result(),\n\t\t\texpectedInitContainers: 1, // Only the \"another-init\" container should remain\n\t\t\texpectedError: nil,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t// Setup\n\t\t\tvar (\n\t\t\t\tclient = crfake.NewClientBuilder().Build()\n\t\t\t\tcrClient = client\n\t\t\t)\n\n\t\t\t// Register the PodVolumeBackup type with the scheme\n\t\t\trequire.NoError(t, velerov1api.AddToScheme(scheme.Scheme))\n\n\t\t\t// Create the PodVolumeBackups in the fake client\n\t\t\tfor _, pvb := range tc.podVolumeBackups {\n\t\t\t\trequire.NoError(t, crClient.Create(t.Context(), pvb))\n\t\t\t}\n\n\t\t\t// Create a fake clientset\n\t\t\tclientset := fake.NewSimpleClientset()\n\n\t\t\t// Create the action\n\t\t\taction := &PodVolumeRestoreAction{\n\t\t\t\tlogger: logrus.StandardLogger(),\n\t\t\t\tclient: clientset.CoreV1().ConfigMaps(\"velero\"),\n\t\t\t\tcrClient: crClient,\n\t\t\t\tveleroImage: \"velero/velero:latest\",\n\t\t\t}\n\n\t\t\t// Convert the pod to unstructured\n\t\t\tpodMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.pod)\n\t\t\trequire.NoError(t, err)\n\t\t\tpodFromBackupMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.podFromBackup)\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Create the input\n\t\t\tinput := &velero.RestoreItemActionExecuteInput{\n\t\t\t\tItem: &unstructured.Unstructured{Object: podMap},\n\t\t\t\tItemFromBackup: &unstructured.Unstructured{Object: podFromBackupMap},\n\t\t\t\tRestore: tc.restore,\n\t\t\t}\n\n\t\t\t// Execute the action\n\t\t\toutput, err := action.Execute(input)\n\n\t\t\t// Verify the results\n\t\t\tif tc.expectedError != nil {\n\t\t\t\tassert.Equal(t, tc.expectedError, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\t// Convert the output back to a pod\n\t\t\t\toutputPod := new(corev1api.Pod)\n\t\t\t\terr = runtime.DefaultUnstructuredConverter.FromUnstructured(output.UpdatedItem.UnstructuredContent(), outputPod)\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\t// Check if the init container was added or removed as expected\n\t\t\t\tassert.Len(t, outputPod.Spec.InitContainers, tc.expectedInitContainers, \"Unexpected number of init containers\")\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/restore/actions/pvc_action.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\tcorev1client \"k8s.io/client-go/kubernetes/typed/core/v1\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n)\n\nconst (\n\tAnnBindCompleted = \"pv.kubernetes.io/bind-completed\"\n\tAnnBoundByController = \"pv.kubernetes.io/bound-by-controller\"\n\tAnnStorageProvisioner = \"volume.kubernetes.io/storage-provisioner\"\n\tAnnBetaStorageProvisioner = \"volume.beta.kubernetes.io/storage-provisioner\"\n\tAnnSelectedNode = \"volume.kubernetes.io/selected-node\"\n)\n\n// PVCAction updates/reset PVC's node selector\n// if a mapping is found in the plugin's config map.\ntype PVCAction struct {\n\tlogger logrus.FieldLogger\n\tconfigMapClient corev1client.ConfigMapInterface\n\tnodeClient corev1client.NodeInterface\n}\n\n// NewPVCAction is the constructor for PVCAction.\nfunc NewPVCAction(\n\tlogger logrus.FieldLogger,\n\tconfigMapClient corev1client.ConfigMapInterface,\n\tnodeClient corev1client.NodeInterface,\n) *PVCAction {\n\treturn &PVCAction{\n\t\tlogger: logger,\n\t\tconfigMapClient: configMapClient,\n\t\tnodeClient: nodeClient,\n\t}\n}\n\n// AppliesTo returns the resources that PVCAction should be run for\nfunc (p *PVCAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"persistentvolumeclaims\"},\n\t}, nil\n}\n\n// PVC actions for restore:\n// 1. updates the pvc's selected-node annotation:\n// a) if node mapping found in the config map for the plugin\n// b) if node mentioned in annotation doesn't exist\n// 2. removes some additional annotations\n// 3. returns bound PV as an additional item\nfunc (p *PVCAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\tp.logger.Info(\"Executing PVCAction\")\n\tdefer p.logger.Info(\"Done executing PVCAction\")\n\n\tvar pvc, pvcFromBackup corev1api.PersistentVolumeClaim\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(\n\t\tinput.Item.UnstructuredContent(), &pvc); err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(\n\t\tinput.ItemFromBackup.UnstructuredContent(), &pvcFromBackup); err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tlog := p.logger.WithFields(map[string]any{\n\t\t\"kind\": pvc.Kind,\n\t\t\"namespace\": pvc.Namespace,\n\t\t\"name\": pvc.Name,\n\t})\n\n\t// Remove PVC annotations\n\tremovePVCAnnotations(\n\t\t&pvc,\n\t\t[]string{\n\t\t\tAnnBindCompleted,\n\t\t\tAnnBoundByController,\n\t\t\tAnnStorageProvisioner,\n\t\t\tAnnBetaStorageProvisioner,\n\t\t\tAnnSelectedNode,\n\t\t\tvelerov1api.VolumeSnapshotLabel,\n\t\t\tvelerov1api.DataUploadNameAnnotation,\n\t\t},\n\t)\n\n\tpvcMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&pvc)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\toutput := &velero.RestoreItemActionExecuteOutput{\n\t\tUpdatedItem: &unstructured.Unstructured{Object: pvcMap},\n\t}\n\n\t// Add PV as additional item if bound\n\t// use pvcFromBackup because we need to look at status fields, which have been removed from pvc\n\tif pvcFromBackup.Status.Phase != corev1api.ClaimBound || pvcFromBackup.Spec.VolumeName == \"\" {\n\t\tlog.Info(\"PVC is not bound or its volume name is empty\")\n\t} else {\n\t\tlog.Infof(\"Adding PV %s as an additional item to restore\", pvcFromBackup.Spec.VolumeName)\n\t\toutput.AdditionalItems = []velero.ResourceIdentifier{\n\t\t\t{\n\t\t\t\tGroupResource: kuberesource.PersistentVolumes,\n\t\t\t\tName: pvcFromBackup.Spec.VolumeName,\n\t\t\t},\n\t\t}\n\t}\n\treturn output, nil\n}\n\nfunc removePVCAnnotations(pvc *corev1api.PersistentVolumeClaim, remove []string) {\n\tfor k := range pvc.Annotations {\n\t\tif util.Contains(remove, k) {\n\t\t\tdelete(pvc.Annotations, k)\n\t\t}\n\t}\n}\n" }, { "path": "pkg/restore/actions/pvc_action_test.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"fmt\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes/fake\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\n// TestPVCActionExecute runs the PVCAction's Execute\n// method and validates that the item's PVC is modified (or not) as expected.\n// Validation is done by comparing the result of the Execute method to the test case's\n// desired result.\nfunc TestPVCActionExecute(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\twant *corev1api.PersistentVolumeClaim\n\t\twantErr error\n\t}{\n\t\t{\n\t\t\tname: \"a persistent volume claim with no annotation\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"source-ns\", \"pvc-1\").Result(),\n\t\t\twant: builder.ForPersistentVolumeClaim(\"source-ns\", \"pvc-1\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"a persistent volume claim with selected-node annotation\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"source-ns\", \"pvc-1\").\n\t\t\t\tObjectMeta(\n\t\t\t\t\tbuilder.WithAnnotations(\"volume.kubernetes.io/selected-node\", \"source-node\"),\n\t\t\t\t).Result(),\n\t\t\twant: builder.ForPersistentVolumeClaim(\"source-ns\", \"pvc-1\").ObjectMeta(builder.WithAnnotationsMap(map[string]string{})).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"a persistent volume claim with other annotation\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"source-ns\", \"pvc-1\").\n\t\t\t\tObjectMeta(\n\t\t\t\t\tbuilder.WithAnnotations(\"other-anno-1\", \"other-value-1\", \"other-anno-2\", \"other-value-2\"),\n\t\t\t\t).Result(),\n\t\t\twant: builder.ForPersistentVolumeClaim(\"source-ns\", \"pvc-1\").ObjectMeta(\n\t\t\t\tbuilder.WithAnnotations(\"other-anno-1\", \"other-value-1\", \"other-anno-2\", \"other-value-2\"),\n\t\t\t).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"a persistent volume claim with other annotation and selected-node annotation\",\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"source-ns\", \"pvc-1\").\n\t\t\t\tObjectMeta(\n\t\t\t\t\tbuilder.WithAnnotations(\"other-anno\", \"other-value\", \"volume.kubernetes.io/selected-node\", \"source-node\"),\n\t\t\t\t).Result(),\n\t\t\twant: builder.ForPersistentVolumeClaim(\"source-ns\", \"pvc-1\").ObjectMeta(\n\t\t\t\tbuilder.WithAnnotations(\"other-anno\", \"other-value\"),\n\t\t\t).Result(),\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tclientset := fake.NewSimpleClientset()\n\n\t\t\ta := NewPVCAction(\n\t\t\t\tvelerotest.NewLogger(),\n\t\t\t\tclientset.CoreV1().ConfigMaps(\"velero\"),\n\t\t\t\tclientset.CoreV1().Nodes(),\n\t\t\t)\n\n\t\t\t// set up test data\n\t\t\tunstructuredMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.pvc)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tinput := &velero.RestoreItemActionExecuteInput{\n\t\t\t\tItem: &unstructured.Unstructured{\n\t\t\t\t\tObject: unstructuredMap,\n\t\t\t\t},\n\t\t\t\tItemFromBackup: &unstructured.Unstructured{\n\t\t\t\t\tObject: unstructuredMap,\n\t\t\t\t},\n\t\t\t}\n\n\t\t\t// execute method under test\n\t\t\tres, err := a.Execute(input)\n\n\t\t\t// validate for both error and non-error cases\n\t\t\tswitch {\n\t\t\tcase tc.wantErr != nil:\n\t\t\t\trequire.EqualError(t, err, tc.wantErr.Error())\n\t\t\tdefault:\n\t\t\t\tfmt.Printf(\"got +%v\\n\", res.UpdatedItem)\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\twantUnstructured, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.want)\n\t\t\t\tfmt.Printf(\"expected +%v\\n\", wantUnstructured)\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\tassert.Equal(t, &unstructured.Unstructured{Object: wantUnstructured}, res.UpdatedItem)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestAddPVFromPVCActionExecute(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\titemFromBackup *corev1api.PersistentVolumeClaim\n\t\twant []velero.ResourceIdentifier\n\t}{\n\t\t{\n\t\t\tname: \"bound PVC with volume name returns associated PV\",\n\t\t\titemFromBackup: &corev1api.PersistentVolumeClaim{\n\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\tVolumeName: \"bound-pv\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\t\t\tPhase: corev1api.ClaimBound,\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []velero.ResourceIdentifier{\n\t\t\t\t{\n\t\t\t\t\tGroupResource: kuberesource.PersistentVolumes,\n\t\t\t\t\tName: \"bound-pv\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"unbound PVC with volume name does not return any additional items\",\n\t\t\titemFromBackup: &corev1api.PersistentVolumeClaim{\n\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\tVolumeName: \"pending-pv\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\t\t\tPhase: corev1api.ClaimPending,\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"bound PVC without volume name does not return any additional items\",\n\t\t\titemFromBackup: &corev1api.PersistentVolumeClaim{\n\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{},\n\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\t\t\tPhase: corev1api.ClaimBound,\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: nil,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\titemFromBackupData, err := runtime.DefaultUnstructuredConverter.ToUnstructured(test.itemFromBackup)\n\t\t\trequire.NoError(t, err)\n\n\t\t\titemData, err := runtime.DefaultUnstructuredConverter.ToUnstructured(test.itemFromBackup)\n\t\t\trequire.NoError(t, err)\n\t\t\t// item should have no status\n\t\t\tdelete(itemData, \"status\")\n\n\t\t\tclientset := fake.NewSimpleClientset()\n\t\t\taction := NewPVCAction(\n\t\t\t\tvelerotest.NewLogger(),\n\t\t\t\tclientset.CoreV1().ConfigMaps(\"velero\"),\n\t\t\t\tclientset.CoreV1().Nodes(),\n\t\t\t)\n\n\t\t\tinput := &velero.RestoreItemActionExecuteInput{\n\t\t\t\tItem: &unstructured.Unstructured{Object: itemData},\n\t\t\t\tItemFromBackup: &unstructured.Unstructured{Object: itemFromBackupData},\n\t\t\t}\n\n\t\t\tres, err := action.Execute(input)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.Equal(t, test.want, res.AdditionalItems)\n\t\t})\n\t}\n}\n\nfunc TestRemovePVCAnnotations(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tpvc corev1api.PersistentVolumeClaim\n\t\tremoveAnnotations []string\n\t\texpectedAnnotations map[string]string\n\t}{\n\t\t{\n\t\t\tname: \"should preserve all existing annotations\",\n\t\t\tpvc: corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"ann1\": \"ann1-val\",\n\t\t\t\t\t\t\"ann2\": \"ann2-val\",\n\t\t\t\t\t\t\"ann3\": \"ann3-val\",\n\t\t\t\t\t\t\"ann4\": \"ann4-val\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tremoveAnnotations: []string{},\n\t\t\texpectedAnnotations: map[string]string{\n\t\t\t\t\"ann1\": \"ann1-val\",\n\t\t\t\t\"ann2\": \"ann2-val\",\n\t\t\t\t\"ann3\": \"ann3-val\",\n\t\t\t\t\"ann4\": \"ann4-val\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should remove all existing annotations\",\n\t\t\tpvc: corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"ann1\": \"ann1-val\",\n\t\t\t\t\t\t\"ann2\": \"ann2-val\",\n\t\t\t\t\t\t\"ann3\": \"ann3-val\",\n\t\t\t\t\t\t\"ann4\": \"ann4-val\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tremoveAnnotations: []string{\"ann1\", \"ann2\", \"ann3\", \"ann4\"},\n\t\t\texpectedAnnotations: map[string]string{},\n\t\t},\n\t\t{\n\t\t\tname: \"should preserve some existing annotations\",\n\t\t\tpvc: corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"ann1\": \"ann1-val\",\n\t\t\t\t\t\t\"ann2\": \"ann2-val\",\n\t\t\t\t\t\t\"ann3\": \"ann3-val\",\n\t\t\t\t\t\t\"ann4\": \"ann4-val\",\n\t\t\t\t\t\t\"ann5\": \"ann5-val\",\n\t\t\t\t\t\t\"ann6\": \"ann6-val\",\n\t\t\t\t\t\t\"ann7\": \"ann7-val\",\n\t\t\t\t\t\t\"ann8\": \"ann8-val\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tremoveAnnotations: []string{\"ann1\", \"ann2\", \"ann3\", \"ann4\"},\n\t\t\texpectedAnnotations: map[string]string{\n\t\t\t\t\"ann5\": \"ann5-val\",\n\t\t\t\t\"ann6\": \"ann6-val\",\n\t\t\t\t\"ann7\": \"ann7-val\",\n\t\t\t\t\"ann8\": \"ann8-val\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tremovePVCAnnotations(&tc.pvc, tc.removeAnnotations)\n\t\t\tassert.Equal(t, tc.expectedAnnotations, tc.pvc.Annotations)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/restore/actions/rolebinding_action.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\trbacv1 \"k8s.io/api/rbac/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\n// RoleBindingAction handle namespace remappings for role bindings\ntype RoleBindingAction struct {\n\tlogger logrus.FieldLogger\n}\n\nfunc NewRoleBindingAction(logger logrus.FieldLogger) *RoleBindingAction {\n\treturn &RoleBindingAction{logger: logger}\n}\n\nfunc (a *RoleBindingAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"rolebindings\"},\n\t}, nil\n}\n\nfunc (a *RoleBindingAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\tnamespaceMapping := input.Restore.Spec.NamespaceMapping\n\tif len(namespaceMapping) == 0 {\n\t\treturn velero.NewRestoreItemActionExecuteOutput(&unstructured.Unstructured{Object: input.Item.UnstructuredContent()}), nil\n\t}\n\n\troleBinding := new(rbacv1.RoleBinding)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(input.Item.UnstructuredContent(), roleBinding); err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tfor i, subject := range roleBinding.Subjects {\n\t\tif newNamespace, ok := namespaceMapping[subject.Namespace]; ok {\n\t\t\troleBinding.Subjects[i].Namespace = newNamespace\n\t\t}\n\t}\n\n\tres, err := runtime.DefaultUnstructuredConverter.ToUnstructured(roleBinding)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\treturn velero.NewRestoreItemActionExecuteOutput(&unstructured.Unstructured{Object: res}), nil\n}\n" }, { "path": "pkg/restore/actions/rolebinding_action_test.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"sort\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\trbacv1 \"k8s.io/api/rbac/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestRoleBindingActionAppliesTo(t *testing.T) {\n\taction := NewRoleBindingAction(test.NewLogger())\n\tactual, err := action.AppliesTo()\n\trequire.NoError(t, err)\n\tassert.Equal(t, velero.ResourceSelector{IncludedResources: []string{\"rolebindings\"}}, actual)\n}\n\nfunc TestRoleBindingActionExecute(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tnamespaces []string\n\t\tnamespaceMapping map[string]string\n\t\texpected []string\n\t}{\n\t\t{\n\t\t\tname: \"namespace mapping disabled\",\n\t\t\tnamespaces: []string{\"foo\"},\n\t\t\tnamespaceMapping: map[string]string{},\n\t\t\texpected: []string{\"foo\"},\n\t\t},\n\t\t{\n\t\t\tname: \"namespace mapping enabled\",\n\t\t\tnamespaces: []string{\"foo\"},\n\t\t\tnamespaceMapping: map[string]string{\"foo\": \"bar\", \"fizz\": \"buzz\"},\n\t\t\texpected: []string{\"bar\"},\n\t\t},\n\t\t{\n\t\t\tname: \"namespace mapping enabled, not included namespace remains unchanged\",\n\t\t\tnamespaces: []string{\"foo\", \"xyz\"},\n\t\t\tnamespaceMapping: map[string]string{\"foo\": \"bar\", \"fizz\": \"buzz\"},\n\t\t\texpected: []string{\"bar\", \"xyz\"},\n\t\t},\n\t\t{\n\t\t\tname: \"namespace mapping enabled, not included namespace remains unchanged\",\n\t\t\tnamespaces: []string{\"foo\", \"xyz\"},\n\t\t\tnamespaceMapping: map[string]string{\"a\": \"b\", \"c\": \"d\"},\n\t\t\texpected: []string{\"foo\", \"xyz\"},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tsubjects := []rbacv1.Subject{}\n\n\t\t\tfor _, ns := range tc.namespaces {\n\t\t\t\tsubjects = append(subjects, rbacv1.Subject{\n\t\t\t\t\tNamespace: ns,\n\t\t\t\t})\n\t\t\t}\n\n\t\t\troleBinding := rbacv1.RoleBinding{\n\t\t\t\tSubjects: subjects,\n\t\t\t}\n\n\t\t\troleBindingUnstructured, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&roleBinding)\n\t\t\trequire.NoError(t, err)\n\n\t\t\taction := NewRoleBindingAction(test.NewLogger())\n\t\t\tres, err := action.Execute(&velero.RestoreItemActionExecuteInput{\n\t\t\t\tItem: &unstructured.Unstructured{Object: roleBindingUnstructured},\n\t\t\t\tItemFromBackup: &unstructured.Unstructured{Object: roleBindingUnstructured},\n\t\t\t\tRestore: &api.Restore{\n\t\t\t\t\tSpec: api.RestoreSpec{\n\t\t\t\t\t\tNamespaceMapping: tc.namespaceMapping,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t})\n\t\t\trequire.NoError(t, err)\n\n\t\t\tvar resRoleBinding *rbacv1.RoleBinding\n\t\t\terr = runtime.DefaultUnstructuredConverter.FromUnstructured(res.UpdatedItem.UnstructuredContent(), &resRoleBinding)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tactual := []string{}\n\t\t\tfor _, subject := range resRoleBinding.Subjects {\n\t\t\t\tactual = append(actual, subject.Namespace)\n\t\t\t}\n\n\t\t\tsort.Strings(tc.expected)\n\t\t\tsort.Strings(actual)\n\n\t\t\tassert.Equal(t, tc.expected, actual)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/restore/actions/secret_action.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strings\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\n// SecretAction is a restore item action for secrets\ntype SecretAction struct {\n\tlogger logrus.FieldLogger\n\tclient client.Client\n}\n\n// NewSecretAction creates a new SecretAction instance\nfunc NewSecretAction(logger logrus.FieldLogger, client client.Client) *SecretAction {\n\treturn &SecretAction{\n\t\tlogger: logger,\n\t\tclient: client,\n\t}\n}\n\n// AppliesTo indicates which resources this action applies\nfunc (s *SecretAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"secrets\"},\n\t}, nil\n}\n\n// Execute the action\nfunc (s *SecretAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\ts.logger.Info(\"Executing SecretAction\")\n\tdefer s.logger.Info(\"Done executing SecretAction\")\n\n\tvar secret corev1api.Secret\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(input.Item.UnstructuredContent(), &secret); err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to convert secret from runtime.Unstructured\")\n\t}\n\n\tlog := s.logger.WithField(\"secret\", kube.NamespaceAndName(&secret))\n\tif secret.Type != corev1api.SecretTypeServiceAccountToken {\n\t\tlog.Debug(\"No match found - including this secret\")\n\t\treturn &velero.RestoreItemActionExecuteOutput{\n\t\t\tUpdatedItem: input.Item,\n\t\t}, nil\n\t}\n\n\t// The auto created service account token secret will be created by kube controller automatically again(before Kubernetes v1.22), no need to restore.\n\t// This will cause the patch operation of managedFields failed if we restore it as the secret is removed immediately\n\t// after restoration and the patch operation reports not found error.\n\tlist := &corev1api.ServiceAccountList{}\n\tif err := s.client.List(context.Background(), list, &client.ListOptions{Namespace: secret.Namespace}); err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to list the service accounts\")\n\t}\n\tfor _, sa := range list.Items {\n\t\tif strings.HasPrefix(secret.Name, fmt.Sprintf(\"%s-token-\", sa.Name)) {\n\t\t\tlog.Debug(\"auto created service account token secret found - excluding this secret\")\n\t\t\treturn &velero.RestoreItemActionExecuteOutput{\n\t\t\t\tUpdatedItem: input.Item,\n\t\t\t\tSkipRestore: true,\n\t\t\t}, nil\n\t\t}\n\t}\n\n\tlog.Debug(\"service account token secret(not auto created) found - remove some fields from this secret\")\n\t// If the annotation and data are not removed, the secret cannot be restored successfully.\n\t// The kube controller will fill the annotation and data with new value automatically:\n\t// https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets\n\tdelete(secret.Annotations, \"kubernetes.io/service-account.uid\")\n\tdelete(secret.Data, \"token\")\n\tdelete(secret.Data, \"ca.crt\")\n\n\tres, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&secret)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to convert secret to runtime.Unstructured\")\n\t}\n\n\treturn &velero.RestoreItemActionExecuteOutput{\n\t\tUpdatedItem: &unstructured.Unstructured{Object: res},\n\t}, nil\n}\n" }, { "path": "pkg/restore/actions/secret_action_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestSecretActionAppliesTo(t *testing.T) {\n\taction := NewSecretAction(test.NewLogger(), nil)\n\tactual, err := action.AppliesTo()\n\trequire.NoError(t, err)\n\tassert.Equal(t, velero.ResourceSelector{IncludedResources: []string{\"secrets\"}}, actual)\n}\n\nfunc TestSecretActionExecute(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tinput *corev1api.Secret\n\t\tserviceAccount *corev1api.ServiceAccount\n\t\tskipped bool\n\t\toutput *corev1api.Secret\n\t}{\n\t\t{\n\t\t\tname: \"not service account token secret\",\n\t\t\tinput: &corev1api.Secret{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"foo\",\n\t\t\t\t\tName: \"default-token-sfafa\",\n\t\t\t\t},\n\t\t\t\tType: corev1api.SecretTypeOpaque,\n\t\t\t},\n\t\t\tskipped: false,\n\t\t\toutput: &corev1api.Secret{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"foo\",\n\t\t\t\t\tName: \"default-token-sfafa\",\n\t\t\t\t},\n\t\t\t\tType: corev1api.SecretTypeOpaque,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"auto created service account token\",\n\t\t\tinput: &corev1api.Secret{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"foo\",\n\t\t\t\t\tName: \"default-token-sfafa\",\n\t\t\t\t},\n\t\t\t\tType: corev1api.SecretTypeServiceAccountToken,\n\t\t\t},\n\t\t\tserviceAccount: &corev1api.ServiceAccount{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"foo\",\n\t\t\t\t\tName: \"default\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tskipped: true,\n\t\t},\n\t\t{\n\t\t\tname: \"not auto created service account token\",\n\t\t\tinput: &corev1api.Secret{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"foo\",\n\t\t\t\t\tName: \"my-token\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"kubernetes.io/service-account.uid\": \"uid\",\n\t\t\t\t\t\t\"key\": \"value\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tType: corev1api.SecretTypeServiceAccountToken,\n\t\t\t\tData: map[string][]byte{\n\t\t\t\t\t\"token\": []byte(\"token\"),\n\t\t\t\t\t\"ca.crt\": []byte(\"ca\"),\n\t\t\t\t\t\"key\": []byte(\"value\"),\n\t\t\t\t},\n\t\t\t},\n\t\t\tskipped: false,\n\t\t\toutput: &corev1api.Secret{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"foo\",\n\t\t\t\t\tName: \"my-token\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"key\": \"value\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tType: corev1api.SecretTypeServiceAccountToken,\n\t\t\t\tData: map[string][]byte{\n\t\t\t\t\t\"key\": []byte(\"value\"),\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tsecretUnstructured, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.input)\n\t\t\trequire.NoError(t, err)\n\t\t\tvar serviceAccounts []client.Object\n\t\t\tif tc.serviceAccount != nil {\n\t\t\t\tserviceAccounts = append(serviceAccounts, tc.serviceAccount)\n\t\t\t}\n\t\t\tclient := fake.NewClientBuilder().WithObjects(serviceAccounts...).Build()\n\t\t\taction := NewSecretAction(test.NewLogger(), client)\n\t\t\tres, err := action.Execute(&velero.RestoreItemActionExecuteInput{\n\t\t\t\tItem: &unstructured.Unstructured{Object: secretUnstructured},\n\t\t\t})\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Equal(t, tc.skipped, res.SkipRestore)\n\t\t\tif !tc.skipped {\n\t\t\t\tr, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.output)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.EqualValues(t, &unstructured.Unstructured{Object: r}, res.UpdatedItem)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/restore/actions/service_account_action.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"strings\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\ntype ServiceAccountAction struct {\n\tlogger logrus.FieldLogger\n}\n\nfunc NewServiceAccountAction(logger logrus.FieldLogger) *ServiceAccountAction {\n\treturn &ServiceAccountAction{logger: logger}\n}\n\nfunc (a *ServiceAccountAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"serviceaccounts\"},\n\t}, nil\n}\n\nfunc (a *ServiceAccountAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\ta.logger.Info(\"Executing ServiceAccountAction\")\n\tdefer a.logger.Info(\"Done executing ServiceAccountAction\")\n\n\tvar serviceAccount corev1api.ServiceAccount\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(input.Item.UnstructuredContent(), &serviceAccount); err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to convert serviceaccount from runtime.Unstructured\")\n\t}\n\n\tlog := a.logger.WithField(\"serviceaccount\", kube.NamespaceAndName(&serviceAccount))\n\n\tlog.Debug(\"Checking secrets\")\n\tcheck := serviceAccount.Name + \"-token-\"\n\tfor i := len(serviceAccount.Secrets) - 1; i >= 0; i-- {\n\t\tsecret := &serviceAccount.Secrets[i]\n\t\tlog.Debugf(\"Checking if secret %s matches %s\", secret.Name, check)\n\n\t\tif strings.HasPrefix(secret.Name, check) {\n\t\t\t// Copy all secrets *except* -token-\n\t\t\tlog.Debug(\"Match found - excluding this secret\")\n\t\t\tserviceAccount.Secrets = append(serviceAccount.Secrets[:i], serviceAccount.Secrets[i+1:]...)\n\t\t\tbreak\n\t\t} else {\n\t\t\tlog.Debug(\"No match found - including this secret\")\n\t\t}\n\t}\n\n\tres, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&serviceAccount)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to convert serviceaccount to runtime.Unstructured\")\n\t}\n\n\treturn velero.NewRestoreItemActionExecuteOutput(&unstructured.Unstructured{Object: res}), nil\n}\n" }, { "path": "pkg/restore/actions/service_account_action_test.go", "content": "/*\nCopyright 2018, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"sort\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestServiceAccountActionAppliesTo(t *testing.T) {\n\taction := NewServiceAccountAction(test.NewLogger())\n\tactual, err := action.AppliesTo()\n\trequire.NoError(t, err)\n\tassert.Equal(t, velero.ResourceSelector{IncludedResources: []string{\"serviceaccounts\"}}, actual)\n}\n\nfunc TestServiceAccountActionExecute(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tsecrets []string\n\t\texpected []string\n\t}{\n\t\t{\n\t\t\tname: \"no secrets\",\n\t\t\tsecrets: []string{},\n\t\t\texpected: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"no match\",\n\t\t\tsecrets: []string{\"a\", \"bar-TOKN-nomatch\", \"baz\"},\n\t\t\texpected: []string{\"a\", \"bar-TOKN-nomatch\", \"baz\"},\n\t\t},\n\t\t{\n\t\t\tname: \"match - first\",\n\t\t\tsecrets: []string{\"bar-token-a1b2c\", \"a\", \"baz\"},\n\t\t\texpected: []string{\"a\", \"baz\"},\n\t\t},\n\t\t{\n\t\t\tname: \"match - middle\",\n\t\t\tsecrets: []string{\"a\", \"bar-token-a1b2c\", \"baz\"},\n\t\t\texpected: []string{\"a\", \"baz\"},\n\t\t},\n\t\t{\n\t\t\tname: \"match - end\",\n\t\t\tsecrets: []string{\"a\", \"baz\", \"bar-token-a1b2c\"},\n\t\t\texpected: []string{\"a\", \"baz\"},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tsa := corev1api.ServiceAccount{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"foo\",\n\t\t\t\t\tName: \"bar\",\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tfor _, secret := range tc.secrets {\n\t\t\t\tsa.Secrets = append(sa.Secrets, corev1api.ObjectReference{\n\t\t\t\t\tName: secret,\n\t\t\t\t})\n\t\t\t}\n\n\t\t\tsaUnstructured, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&sa)\n\t\t\trequire.NoError(t, err)\n\n\t\t\taction := NewServiceAccountAction(test.NewLogger())\n\t\t\tres, err := action.Execute(&velero.RestoreItemActionExecuteInput{\n\t\t\t\tItem: &unstructured.Unstructured{Object: saUnstructured},\n\t\t\t\tItemFromBackup: &unstructured.Unstructured{Object: saUnstructured},\n\t\t\t\tRestore: nil,\n\t\t\t})\n\t\t\trequire.NoError(t, err)\n\n\t\t\tvar resSA *corev1api.ServiceAccount\n\t\t\terr = runtime.DefaultUnstructuredConverter.FromUnstructured(res.UpdatedItem.UnstructuredContent(), &resSA)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tactual := []string{}\n\t\t\tfor _, secret := range resSA.Secrets {\n\t\t\t\tactual = append(actual, secret.Name)\n\t\t\t}\n\n\t\t\tsort.Strings(tc.expected)\n\t\t\tsort.Strings(actual)\n\n\t\t\tassert.Equal(t, tc.expected, actual)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/restore/actions/service_action.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"strconv\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n)\n\nconst annotationLastAppliedConfig = \"kubectl.kubernetes.io/last-applied-configuration\"\n\ntype ServiceAction struct {\n\tlog logrus.FieldLogger\n}\n\nfunc NewServiceAction(logger logrus.FieldLogger) *ServiceAction {\n\treturn &ServiceAction{log: logger}\n}\n\nfunc (a *ServiceAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn velero.ResourceSelector{\n\t\tIncludedResources: []string{\"services\"},\n\t}, nil\n}\n\nfunc (a *ServiceAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\tservice := new(corev1api.Service)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(input.Item.UnstructuredContent(), service); err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tif service.Spec.ClusterIP != \"None\" {\n\t\tservice.Spec.ClusterIP = \"\"\n\t\tservice.Spec.ClusterIPs = nil\n\t}\n\n\t/* Do not delete NodePorts if restore triggered with \"--preserve-nodeports\" flag */\n\tif boolptr.IsSetToTrue(input.Restore.Spec.PreserveNodePorts) {\n\t\ta.log.Info(\"Restoring Services with original NodePort(s)\")\n\t} else {\n\t\tif err := deleteNodePorts(service); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tif err := deleteHealthCheckNodePort(service); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tres, err := runtime.DefaultUnstructuredConverter.ToUnstructured(service)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\treturn velero.NewRestoreItemActionExecuteOutput(&unstructured.Unstructured{Object: res}), nil\n}\n\nfunc deleteHealthCheckNodePort(service *corev1api.Service) error {\n\t// Check service type and external traffic policy setting,\n\t// if the setting is not applicable for HealthCheckNodePort, return early.\n\tif service.Spec.ExternalTrafficPolicy != corev1api.ServiceExternalTrafficPolicyTypeLocal ||\n\t\tservice.Spec.Type != corev1api.ServiceTypeLoadBalancer {\n\t\treturn nil\n\t}\n\n\t// HealthCheckNodePort is already 0, return.\n\tif service.Spec.HealthCheckNodePort == 0 {\n\t\treturn nil\n\t}\n\n\t// Search HealthCheckNodePort from server's last-applied-configuration\n\t// annotation(HealthCheckNodePort is specified by `kubectl apply` command)\n\tlastAppliedConfig, ok := service.Annotations[annotationLastAppliedConfig]\n\tif ok {\n\t\tappliedServiceUnstructured := new(map[string]any)\n\t\tif err := json.Unmarshal([]byte(lastAppliedConfig), appliedServiceUnstructured); err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\n\t\thealthCheckNodePort, exist, err := unstructured.NestedFloat64(*appliedServiceUnstructured, \"spec\", \"healthCheckNodePort\")\n\t\tif err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\n\t\t// Found healthCheckNodePort in lastAppliedConfig annotation,\n\t\t// and the value is not 0. No need to delete, return.\n\t\tif exist && healthCheckNodePort != 0 {\n\t\t\treturn nil\n\t\t}\n\t}\n\n\t// Search HealthCheckNodePort from ManagedFields(HealthCheckNodePort\n\t// is specified by `kubectl apply --server-side` command).\n\tfor _, entry := range service.GetManagedFields() {\n\t\tif entry.FieldsV1 == nil {\n\t\t\tcontinue\n\t\t}\n\t\tfields := new(map[string]any)\n\t\tif err := json.Unmarshal(entry.FieldsV1.Raw, fields); err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\n\t\t_, exist, err := unstructured.NestedMap(*fields, \"f:spec\", \"f:healthCheckNodePort\")\n\t\tif err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\t\tif !exist {\n\t\t\tcontinue\n\t\t}\n\t\t// Because the format in ManagedFields is `f:healthCheckNodePort: {}`,\n\t\t// cannot get the value, check whether exists is enough.\n\t\t// Found healthCheckNodePort in ManagedFields.\n\t\t// No need to delete. Return.\n\t\treturn nil\n\t}\n\n\t// Cannot find HealthCheckNodePort from Annotation and\n\t// ManagedFields, which means it's auto-generated. Delete it.\n\tservice.Spec.HealthCheckNodePort = 0\n\n\treturn nil\n}\n\nfunc deleteNodePorts(service *corev1api.Service) error {\n\tif service.Spec.Type == corev1api.ServiceTypeExternalName {\n\t\treturn nil\n\t}\n\n\t// find any NodePorts whose values were explicitly specified according\n\t// to the last-applied-config annotation. We'll retain these values, and\n\t// clear out any other (presumably auto-assigned) NodePort values.\n\tlastAppliedConfig, ok := service.Annotations[annotationLastAppliedConfig]\n\tif ok {\n\t\texplicitNodePorts := sets.NewString()\n\t\tunnamedPortInts := sets.NewInt()\n\t\tappliedServiceUnstructured := new(map[string]any)\n\t\tif err := json.Unmarshal([]byte(lastAppliedConfig), appliedServiceUnstructured); err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\n\t\tports, bool, err := unstructured.NestedSlice(*appliedServiceUnstructured, \"spec\", \"ports\")\n\n\t\tif err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\n\t\tif bool {\n\t\t\tfor _, port := range ports {\n\t\t\t\tp, ok := port.(map[string]any)\n\t\t\t\tif !ok {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tnodePort, nodePortBool, err := unstructured.NestedFieldNoCopy(p, \"nodePort\")\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn errors.WithStack(err)\n\t\t\t\t}\n\t\t\t\tif nodePortBool {\n\t\t\t\t\tnodePortInt := 0\n\t\t\t\t\tswitch nodePort := nodePort.(type) {\n\t\t\t\t\tcase int32:\n\t\t\t\t\t\tnodePortInt = int(nodePort)\n\t\t\t\t\tcase float64:\n\t\t\t\t\t\tnodePortInt = int(nodePort)\n\t\t\t\t\tcase string:\n\t\t\t\t\t\tnodePortInt, err = strconv.Atoi(nodePort)\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\treturn errors.WithStack(err)\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t\tif nodePortInt > 0 {\n\t\t\t\t\t\tportName, ok := p[\"name\"]\n\t\t\t\t\t\tif !ok {\n\t\t\t\t\t\t\t// unnamed port\n\t\t\t\t\t\t\tunnamedPortInts.Insert(nodePortInt)\n\t\t\t\t\t\t} else {\n\t\t\t\t\t\t\texplicitNodePorts.Insert(portName.(string))\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\tfor i, port := range service.Spec.Ports {\n\t\t\tif port.Name != \"\" {\n\t\t\t\tif !explicitNodePorts.Has(port.Name) {\n\t\t\t\t\tservice.Spec.Ports[i].NodePort = 0\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif !unnamedPortInts.Has(int(port.NodePort)) {\n\t\t\t\t\tservice.Spec.Ports[i].NodePort = 0\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\treturn nil\n\t}\n\n\texplicitNodePorts := sets.NewString()\n\tfor _, entry := range service.GetManagedFields() {\n\t\tif entry.FieldsV1 == nil {\n\t\t\tcontinue\n\t\t}\n\t\tfields := new(map[string]any)\n\t\tif err := json.Unmarshal(entry.FieldsV1.Raw, fields); err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\n\t\tports, exist, err := unstructured.NestedMap(*fields, \"f:spec\", \"f:ports\")\n\t\tif err != nil {\n\t\t\treturn errors.WithStack(err)\n\t\t}\n\t\tif !exist {\n\t\t\tcontinue\n\t\t}\n\t\tfor key, port := range ports {\n\t\t\tp, ok := port.(map[string]any)\n\t\t\tif !ok {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif _, exist := p[\"f:nodePort\"]; exist {\n\t\t\t\texplicitNodePorts.Insert(key)\n\t\t\t}\n\t\t}\n\t}\n\tfor i, port := range service.Spec.Ports {\n\t\tk := portKey(port)\n\t\tif !explicitNodePorts.Has(k) {\n\t\t\tservice.Spec.Ports[i].NodePort = 0\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc portKey(port corev1api.ServicePort) string {\n\treturn fmt.Sprintf(`k:{\"port\":%d,\"protocol\":\"%s\"}`, port.Port, port.Protocol)\n}\n" }, { "path": "pkg/restore/actions/service_action_test.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actions\n\nimport (\n\t\"encoding/json\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc svcJSON(ports ...corev1api.ServicePort) string {\n\tsvc := corev1api.Service{\n\t\tSpec: corev1api.ServiceSpec{\n\t\t\tHealthCheckNodePort: 8080,\n\t\t\tPorts: ports,\n\t\t},\n\t}\n\n\tdata, err := json.Marshal(svc)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\treturn string(data)\n}\n\nfunc svcJSONFromUnstructured(ports ...map[string]any) string {\n\tsvc := map[string]any{\n\t\t\"spec\": map[string]any{\n\t\t\t\"ports\": ports,\n\t\t},\n\t}\n\n\tdata, err := json.Marshal(svc)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\treturn string(data)\n}\n\nfunc TestServiceActionExecute(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tobj corev1api.Service\n\t\trestore *api.Restore\n\t\texpectedErr bool\n\t\texpectedRes corev1api.Service\n\t}{\n\t\t{\n\t\t\tname: \"clusterIP/clusterIPs should be deleted from spec\",\n\t\t\tobj: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tClusterIP: \"should-be-removed\",\n\t\t\t\t\tClusterIPs: []string{\"should-be-removed\"},\n\t\t\t\t\tLoadBalancerIP: \"should-be-kept\",\n\t\t\t\t},\n\t\t\t},\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").Result(),\n\t\t\texpectedErr: false,\n\t\t\texpectedRes: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tLoadBalancerIP: \"should-be-kept\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"headless clusterIP should not be deleted from spec\",\n\t\t\tobj: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tClusterIP: \"None\",\n\t\t\t\t},\n\t\t\t},\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").Result(),\n\t\t\texpectedRes: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tClusterIP: \"None\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"nodePort (only) should be deleted from all spec.ports\",\n\t\t\tobj: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tPorts: []corev1api.ServicePort{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tPort: 32000,\n\t\t\t\t\t\t\tNodePort: 32000,\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tPort: 32001,\n\t\t\t\t\t\t\tNodePort: 32001,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").Result(),\n\t\t\texpectedRes: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tPorts: []corev1api.ServicePort{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tPort: 32000,\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tPort: 32001,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"unnamed nodePort should be deleted when missing in annotation\",\n\t\t\tobj: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tannotationLastAppliedConfig: svcJSON(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tPorts: []corev1api.ServicePort{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tNodePort: 8080,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").Result(),\n\t\t\texpectedRes: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tannotationLastAppliedConfig: svcJSON(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tPorts: []corev1api.ServicePort{\n\t\t\t\t\t\t{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"unnamed nodePort should be preserved when specified in annotation\",\n\t\t\tobj: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tannotationLastAppliedConfig: svcJSON(corev1api.ServicePort{NodePort: 8080}),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tPorts: []corev1api.ServicePort{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tNodePort: 8080,\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tNodePort: 9090,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").Result(),\n\t\t\texpectedRes: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tannotationLastAppliedConfig: svcJSON(corev1api.ServicePort{NodePort: 8080}),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tPorts: []corev1api.ServicePort{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tNodePort: 8080,\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"unnamed nodePort should be deleted when named nodePort specified in annotation\",\n\t\t\tobj: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tannotationLastAppliedConfig: svcJSON(corev1api.ServicePort{Name: \"http\", NodePort: 8080}),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tPorts: []corev1api.ServicePort{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tNodePort: 8080,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").Result(),\n\t\t\texpectedRes: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tannotationLastAppliedConfig: svcJSON(corev1api.ServicePort{Name: \"http\", NodePort: 8080}),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tPorts: []corev1api.ServicePort{\n\t\t\t\t\t\t{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"unnamed nodePort should be deleted when named a string nodePort specified in annotation\",\n\t\t\tobj: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tannotationLastAppliedConfig: svcJSONFromUnstructured(map[string]any{\"name\": \"http\", \"nodePort\": \"8080\"}),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tPorts: []corev1api.ServicePort{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tNodePort: 8080,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").Result(),\n\t\t\texpectedRes: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tannotationLastAppliedConfig: svcJSONFromUnstructured(map[string]any{\"name\": \"http\", \"nodePort\": \"8080\"}),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tPorts: []corev1api.ServicePort{\n\t\t\t\t\t\t{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"named nodePort should be preserved when specified in annotation\",\n\t\t\tobj: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tannotationLastAppliedConfig: svcJSON(corev1api.ServicePort{Name: \"http\", NodePort: 8080}),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tPorts: []corev1api.ServicePort{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"http\",\n\t\t\t\t\t\t\tNodePort: 8080,\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"admin\",\n\t\t\t\t\t\t\tNodePort: 9090,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").Result(),\n\t\t\texpectedRes: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tannotationLastAppliedConfig: svcJSON(corev1api.ServicePort{Name: \"http\", NodePort: 8080}),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tPorts: []corev1api.ServicePort{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"http\",\n\t\t\t\t\t\t\tNodePort: 8080,\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"admin\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"If PreserveNodePorts is True in restore spec then nodePort always preserved.\",\n\t\t\tobj: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tPorts: []corev1api.ServicePort{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"http\",\n\t\t\t\t\t\t\tPort: 80,\n\t\t\t\t\t\t\tNodePort: 8080,\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"hepsiburada\",\n\t\t\t\t\t\t\tNodePort: 9025,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").PreserveNodePorts(true).Result(),\n\t\t\texpectedRes: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tPorts: []corev1api.ServicePort{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"http\",\n\t\t\t\t\t\t\tPort: 80,\n\t\t\t\t\t\t\tNodePort: 8080,\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"hepsiburada\",\n\t\t\t\t\t\t\tNodePort: 9025,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"nodePort should be delete when not specified in managedFields\",\n\t\t\tobj: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t\tManagedFields: []metav1.ManagedFieldsEntry{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tFieldsV1: &metav1.FieldsV1{\n\t\t\t\t\t\t\t\tRaw: []byte(`{\"f:spec\":{\"f:ports\":{\"k:{\\\"port\\\":443,\\\"protocol\\\":\\\"TCP\\\"}\":{\".\":{},\"f:name\":{},\"f:port\":{}},\"k:{\\\"port\\\":80,\\\"protocol\\\":\\\"TCP\\\"}\":{\".\":{},\"f:name\":{},\"f:port\":{}}},\"f:selector\":{},\"f:type\":{}}}`),\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tPorts: []corev1api.ServicePort{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"http\",\n\t\t\t\t\t\t\tPort: 80,\n\t\t\t\t\t\t\tProtocol: \"TCP\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"https\",\n\t\t\t\t\t\t\tPort: 443,\n\t\t\t\t\t\t\tProtocol: \"TCP\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").Result(),\n\t\t\texpectedRes: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t\tManagedFields: []metav1.ManagedFieldsEntry{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tFieldsV1: &metav1.FieldsV1{\n\t\t\t\t\t\t\t\tRaw: []byte(`{\"f:spec\":{\"f:ports\":{\"k:{\\\"port\\\":443,\\\"protocol\\\":\\\"TCP\\\"}\":{\".\":{},\"f:name\":{},\"f:port\":{}},\"k:{\\\"port\\\":80,\\\"protocol\\\":\\\"TCP\\\"}\":{\".\":{},\"f:name\":{},\"f:port\":{}}},\"f:selector\":{},\"f:type\":{}}}`),\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tPorts: []corev1api.ServicePort{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"http\",\n\t\t\t\t\t\t\tPort: 80,\n\t\t\t\t\t\t\tNodePort: 0,\n\t\t\t\t\t\t\tProtocol: \"TCP\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"https\",\n\t\t\t\t\t\t\tPort: 443,\n\t\t\t\t\t\t\tNodePort: 0,\n\t\t\t\t\t\t\tProtocol: \"TCP\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"nodePort should be preserved when specified in managedFields\",\n\t\t\tobj: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t\tManagedFields: []metav1.ManagedFieldsEntry{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tFieldsV1: &metav1.FieldsV1{\n\t\t\t\t\t\t\t\tRaw: []byte(`{\"f:spec\":{\"f:ports\":{\"k:{\\\"port\\\":443,\\\"protocol\\\":\\\"TCP\\\"}\":{\".\":{},\"f:name\":{},\"f:nodePort\":{},\"f:port\":{}},\"k:{\\\"port\\\":80,\\\"protocol\\\":\\\"TCP\\\"}\":{\".\":{},\"f:name\":{},\"f:nodePort\":{},\"f:port\":{}}},\"f:selector\":{},\"f:type\":{}}}`),\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tPorts: []corev1api.ServicePort{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"http\",\n\t\t\t\t\t\t\tPort: 80,\n\t\t\t\t\t\t\tNodePort: 30000,\n\t\t\t\t\t\t\tProtocol: \"TCP\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"https\",\n\t\t\t\t\t\t\tPort: 443,\n\t\t\t\t\t\t\tNodePort: 30002,\n\t\t\t\t\t\t\tProtocol: \"TCP\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").Result(),\n\t\t\texpectedRes: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t\tManagedFields: []metav1.ManagedFieldsEntry{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tFieldsV1: &metav1.FieldsV1{\n\t\t\t\t\t\t\t\tRaw: []byte(`{\"f:spec\":{\"f:ports\":{\"k:{\\\"port\\\":443,\\\"protocol\\\":\\\"TCP\\\"}\":{\".\":{},\"f:name\":{},\"f:nodePort\":{},\"f:port\":{}},\"k:{\\\"port\\\":80,\\\"protocol\\\":\\\"TCP\\\"}\":{\".\":{},\"f:name\":{},\"f:nodePort\":{},\"f:port\":{}}},\"f:selector\":{},\"f:type\":{}}}`),\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tPorts: []corev1api.ServicePort{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"http\",\n\t\t\t\t\t\t\tPort: 80,\n\t\t\t\t\t\t\tNodePort: 30000,\n\t\t\t\t\t\t\tProtocol: \"TCP\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"https\",\n\t\t\t\t\t\t\tPort: 443,\n\t\t\t\t\t\t\tNodePort: 30002,\n\t\t\t\t\t\t\tProtocol: \"TCP\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"If PreserveNodePorts is True in restore spec then HealthCheckNodePort always preserved.\",\n\t\t\tobj: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tHealthCheckNodePort: 8080,\n\t\t\t\t\tExternalTrafficPolicy: corev1api.ServiceExternalTrafficPolicyTypeLocal,\n\t\t\t\t\tType: corev1api.ServiceTypeLoadBalancer,\n\t\t\t\t\tPorts: []corev1api.ServicePort{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"http\",\n\t\t\t\t\t\t\tPort: 80,\n\t\t\t\t\t\t\tNodePort: 8080,\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"hepsiburada\",\n\t\t\t\t\t\t\tNodePort: 9025,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").PreserveNodePorts(true).Result(),\n\t\t\texpectedRes: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tHealthCheckNodePort: 8080,\n\t\t\t\t\tExternalTrafficPolicy: corev1api.ServiceExternalTrafficPolicyTypeLocal,\n\t\t\t\t\tType: corev1api.ServiceTypeLoadBalancer,\n\t\t\t\t\tPorts: []corev1api.ServicePort{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"http\",\n\t\t\t\t\t\t\tPort: 80,\n\t\t\t\t\t\t\tNodePort: 8080,\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"hepsiburada\",\n\t\t\t\t\t\t\tNodePort: 9025,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"If PreserveNodePorts is False in restore spec then HealthCheckNodePort should be cleaned.\",\n\t\t\tobj: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tHealthCheckNodePort: 8080,\n\t\t\t\t\tExternalTrafficPolicy: corev1api.ServiceExternalTrafficPolicyTypeLocal,\n\t\t\t\t\tType: corev1api.ServiceTypeLoadBalancer,\n\t\t\t\t},\n\t\t\t},\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").PreserveNodePorts(false).Result(),\n\t\t\texpectedRes: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tHealthCheckNodePort: 0,\n\t\t\t\t\tExternalTrafficPolicy: corev1api.ServiceExternalTrafficPolicyTypeLocal,\n\t\t\t\t\tType: corev1api.ServiceTypeLoadBalancer,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"If PreserveNodePorts is false in restore spec, but service is not expected, then HealthCheckNodePort should be kept.\",\n\t\t\tobj: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tHealthCheckNodePort: 8080,\n\t\t\t\t\tExternalTrafficPolicy: corev1api.ServiceExternalTrafficPolicyTypeCluster,\n\t\t\t\t\tType: corev1api.ServiceTypeLoadBalancer,\n\t\t\t\t},\n\t\t\t},\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").PreserveNodePorts(false).Result(),\n\t\t\texpectedRes: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tHealthCheckNodePort: 8080,\n\t\t\t\t\tExternalTrafficPolicy: corev1api.ServiceExternalTrafficPolicyTypeCluster,\n\t\t\t\t\tType: corev1api.ServiceTypeLoadBalancer,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"If PreserveNodePorts is false in restore spec, but HealthCheckNodePort can be found in Annotation, then it should be kept.\",\n\t\t\tobj: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t\tAnnotations: map[string]string{annotationLastAppliedConfig: svcJSON()},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tHealthCheckNodePort: 8080,\n\t\t\t\t\tExternalTrafficPolicy: corev1api.ServiceExternalTrafficPolicyTypeLocal,\n\t\t\t\t\tType: corev1api.ServiceTypeLoadBalancer,\n\t\t\t\t},\n\t\t\t},\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").PreserveNodePorts(false).Result(),\n\t\t\texpectedRes: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t\tAnnotations: map[string]string{annotationLastAppliedConfig: svcJSON()},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tHealthCheckNodePort: 8080,\n\t\t\t\t\tExternalTrafficPolicy: corev1api.ServiceExternalTrafficPolicyTypeLocal,\n\t\t\t\t\tType: corev1api.ServiceTypeLoadBalancer,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"If PreserveNodePorts is false in restore spec, but HealthCheckNodePort can be found in ManagedFields, then it should be kept.\",\n\t\t\tobj: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t\tManagedFields: []metav1.ManagedFieldsEntry{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tFieldsV1: &metav1.FieldsV1{\n\t\t\t\t\t\t\t\tRaw: []byte(`{\"f:spec\":{\"f:healthCheckNodePort\":{}}}`),\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tHealthCheckNodePort: 8080,\n\t\t\t\t\tExternalTrafficPolicy: corev1api.ServiceExternalTrafficPolicyTypeLocal,\n\t\t\t\t\tType: corev1api.ServiceTypeLoadBalancer,\n\t\t\t\t},\n\t\t\t},\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").PreserveNodePorts(false).Result(),\n\t\t\texpectedRes: corev1api.Service{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"svc-1\",\n\t\t\t\t\tManagedFields: []metav1.ManagedFieldsEntry{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tFieldsV1: &metav1.FieldsV1{\n\t\t\t\t\t\t\t\tRaw: []byte(`{\"f:spec\":{\"f:healthCheckNodePort\":{}}}`),\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.ServiceSpec{\n\t\t\t\t\tHealthCheckNodePort: 8080,\n\t\t\t\t\tExternalTrafficPolicy: corev1api.ServiceExternalTrafficPolicyTypeLocal,\n\t\t\t\t\tType: corev1api.ServiceTypeLoadBalancer,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\taction := NewServiceAction(velerotest.NewLogger())\n\n\t\t\tunstructuredSvc, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&test.obj)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tres, err := action.Execute(&velero.RestoreItemActionExecuteInput{\n\t\t\t\tItem: &unstructured.Unstructured{Object: unstructuredSvc},\n\t\t\t\tItemFromBackup: &unstructured.Unstructured{Object: unstructuredSvc},\n\t\t\t\tRestore: test.restore,\n\t\t\t})\n\n\t\t\tif assert.Equal(t, test.expectedErr, err != nil) && !test.expectedErr {\n\t\t\t\tvar svc corev1api.Service\n\t\t\t\trequire.NoError(t, runtime.DefaultUnstructuredConverter.FromUnstructured(res.UpdatedItem.UnstructuredContent(), &svc))\n\n\t\t\t\tassert.Equal(t, test.expectedRes, svc)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/restore/merge_service_account.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"encoding/json\"\n\n\tjsonpatch \"github.com/evanphx/json-patch/v5\"\n\t\"github.com/pkg/errors\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/api/equality\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n)\n\n// mergeServiceAccount takes a backed up serviceaccount and merges attributes into the current in-cluster service account.\n// Labels and Annotations on the backed up version but not on the in-cluster version will be merged. If a key is specified in both, the in-cluster version is retained.\nfunc mergeServiceAccounts(fromCluster, fromBackup *unstructured.Unstructured) (*unstructured.Unstructured, error) {\n\tdesired := new(corev1api.ServiceAccount)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(fromCluster.UnstructuredContent(), desired); err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to convert from-cluster service account from unstructured to serviceaccount\")\n\t}\n\n\tbackupSA := new(corev1api.ServiceAccount)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(fromBackup.UnstructuredContent(), backupSA); err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to convert from backed up service account unstructured to serviceaccount\")\n\t}\n\n\tdesired.Secrets = mergeObjectReferenceSlices(desired.Secrets, backupSA.Secrets)\n\n\tdesired.ImagePullSecrets = mergeLocalObjectReferenceSlices(desired.ImagePullSecrets, backupSA.ImagePullSecrets)\n\n\tdesired.Labels = mergeMaps(desired.Labels, backupSA.Labels)\n\n\tdesired.Annotations = mergeMaps(desired.Annotations, backupSA.Annotations)\n\n\tdesiredUnstructured, err := runtime.DefaultUnstructuredConverter.ToUnstructured(desired)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to convert desired service account to unstructured\")\n\t}\n\t// The DefaultUnstructuredConverter.ToUnstructured function will populate the creation timestamp with the nil value\n\t// However, we remove this on both the backup and cluster objects before comparison, and we don't want it in any patches.\n\tdelete(desiredUnstructured[\"metadata\"].(map[string]any), \"creationTimestamp\")\n\n\treturn &unstructured.Unstructured{Object: desiredUnstructured}, nil\n}\n\nfunc mergeObjectReferenceSlices(first, second []corev1api.ObjectReference) []corev1api.ObjectReference {\n\tfor _, s := range second {\n\t\tvar exists bool\n\t\tfor _, f := range first {\n\t\t\tif s.Name == f.Name {\n\t\t\t\texists = true\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t\tif !exists {\n\t\t\tfirst = append(first, s)\n\t\t}\n\t}\n\n\treturn first\n}\n\nfunc mergeLocalObjectReferenceSlices(first, second []corev1api.LocalObjectReference) []corev1api.LocalObjectReference {\n\tfor _, s := range second {\n\t\tvar exists bool\n\t\tfor _, f := range first {\n\t\t\tif s.Name == f.Name {\n\t\t\t\texists = true\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t\tif !exists {\n\t\t\tfirst = append(first, s)\n\t\t}\n\t}\n\n\treturn first\n}\n\n// mergeMaps takes two map[string]string and merges missing keys from the second into the first.\n// If a key already exists, its value is not overwritten.\nfunc mergeMaps(first, second map[string]string) map[string]string {\n\t// If the first map passed in is empty, just use all of the second map's data\n\tif first == nil {\n\t\tfirst = map[string]string{}\n\t}\n\n\tfor k, v := range second {\n\t\t_, ok := first[k]\n\t\tif !ok {\n\t\t\tfirst[k] = v\n\t\t}\n\t}\n\n\treturn first\n}\n\n// generatePatch will calculate a JSON merge patch for an object's desired state.\n// If the passed in objects are already equal, nil is returned.\nfunc generatePatch(fromCluster, desired *unstructured.Unstructured) ([]byte, error) {\n\t// If the objects are already equal, there's no need to generate a patch.\n\tif equality.Semantic.DeepEqual(fromCluster, desired) {\n\t\treturn nil, nil\n\t}\n\n\tdesiredBytes, err := json.Marshal(desired.Object)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to marshal desired object\")\n\t}\n\n\tfromClusterBytes, err := json.Marshal(fromCluster.Object)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to marshal in-cluster object\")\n\t}\n\n\tpatchBytes, err := jsonpatch.CreateMergePatch(fromClusterBytes, desiredBytes)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"unable to create merge patch\")\n\t}\n\n\treturn patchBytes, nil\n}\n" }, { "path": "pkg/restore/merge_service_account_test.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage restore\n\nimport (\n\t\"strings\"\n\t\"testing\"\n\t\"unicode\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nvar mergedServiceAccountsBenchmarkResult *unstructured.Unstructured\n\nfunc BenchmarkMergeServiceAccountBasic(b *testing.B) {\n\ttests := []struct {\n\t\tname string\n\t\tfromCluster *unstructured.Unstructured\n\t\tfromBackup *unstructured.Unstructured\n\t}{\n\t\t{\n\t\t\tname: \"only default tokens present\",\n\t\t\tfromCluster: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\"\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"default-token-abcde\" }\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\t\t\tfromBackup: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\"\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"default-token-xzy12\" }\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\t\t},\n\t\t{\n\t\t\tname: \"service accounts with multiple secrets\",\n\t\t\tfromCluster: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\"\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"default-token-abcde\" },\n\t\t\t\t\t\t{ \"name\": \"my-secret\" },\n\t\t\t\t\t\t{ \"name\": \"sekrit\" }\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\n\t\t\tfromBackup: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\"\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"default-token-xzy12\" },\n\t\t\t\t\t\t{ \"name\": \"my-old-secret\" },\n\t\t\t\t\t\t{ \"name\": \"secrete\"}\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\t\t},\n\t\t{\n\t\t\tname: \"service accounts with labels and annotations\",\n\t\t\tfromCluster: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\",\n\t\t\t\t\t\t\"labels\": {\n\t\t\t\t\t\t\t\"l1\": \"v1\",\n\t\t\t\t\t\t\t\"l2\": \"v2\",\n\t\t\t\t\t\t\t\"l3\": \"v3\"\n\t\t\t\t\t\t},\n\t\t\t\t\t\t\"annotations\": {\n\t\t\t\t\t\t\t\"a1\": \"v1\",\n\t\t\t\t\t\t\t\"a2\": \"v2\",\n\t\t\t\t\t\t\t\"a3\": \"v3\",\n\t\t\t\t\t\t\t\"a4\": \"v4\"\n\t\t\t\t\t\t}\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"default-token-abcde\" }\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\n\t\t\tfromBackup: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\",\n\t\t\t\t\t\t\"labels\": {\n\t\t\t\t\t\t\t\"l1\": \"v1\",\n\t\t\t\t\t\t\t\"l2\": \"v2\",\n\t\t\t\t\t\t\t\"l3\": \"v3\",\n\t\t\t\t\t\t\t\"l4\": \"v4\",\n\t\t\t\t\t\t\t\"l5\": \"v5\"\n\t\t\t\t\t\t},\n\t\t\t\t\t\t\"annotations\": {\n\t\t\t\t\t\t\t\"a1\": \"v1\",\n\t\t\t\t\t\t\t\"a2\": \"v2\",\n\t\t\t\t\t\t\t\"a3\": \"v3\",\n\t\t\t\t\t\t\t\"a4\": \"v4\",\n\t\t\t\t\t\t\t\"a5\": \"v5\",\n\t\t\t\t\t\t\t\"a6\": \"v6\"\n\t\t\t\t\t\t}\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"default-token-xzy12\" }\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\t\t},\n\t}\n\n\tvar desired *unstructured.Unstructured\n\n\tfor _, test := range tests {\n\t\tb.Run(test.name, func(b *testing.B) {\n\t\t\tfor n := 0; n < b.N; n++ {\n\t\t\t\tdesired, _ = mergeServiceAccounts(test.fromCluster, test.fromBackup)\n\t\t\t}\n\n\t\t\tmergedServiceAccountsBenchmarkResult = desired\n\t\t})\n\t}\n}\n\nfunc TestMergeLocalObjectReferenceSlices(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tfirst []corev1api.LocalObjectReference\n\t\tsecond []corev1api.LocalObjectReference\n\t\texpected []corev1api.LocalObjectReference\n\t}{\n\t\t{\n\t\t\tname: \"two slices without overlapping elements\",\n\t\t\tfirst: []corev1api.LocalObjectReference{\n\t\t\t\t{Name: \"lor1\"},\n\t\t\t\t{Name: \"lor2\"},\n\t\t\t},\n\t\t\tsecond: []corev1api.LocalObjectReference{\n\t\t\t\t{Name: \"lor3\"},\n\t\t\t\t{Name: \"lor4\"},\n\t\t\t},\n\t\t\texpected: []corev1api.LocalObjectReference{\n\t\t\t\t{Name: \"lor1\"},\n\t\t\t\t{Name: \"lor2\"},\n\t\t\t\t{Name: \"lor3\"},\n\t\t\t\t{Name: \"lor4\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"two slices with an overlapping element\",\n\t\t\tfirst: []corev1api.LocalObjectReference{\n\t\t\t\t{Name: \"lor1\"},\n\t\t\t\t{Name: \"lor2\"},\n\t\t\t},\n\t\t\tsecond: []corev1api.LocalObjectReference{\n\t\t\t\t{Name: \"lor3\"},\n\t\t\t\t{Name: \"lor2\"},\n\t\t\t},\n\t\t\texpected: []corev1api.LocalObjectReference{\n\t\t\t\t{Name: \"lor1\"},\n\t\t\t\t{Name: \"lor2\"},\n\t\t\t\t{Name: \"lor3\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"merging always adds elements to the end\",\n\t\t\tfirst: []corev1api.LocalObjectReference{\n\t\t\t\t{Name: \"lor3\"},\n\t\t\t\t{Name: \"lor4\"},\n\t\t\t},\n\t\t\tsecond: []corev1api.LocalObjectReference{\n\t\t\t\t{Name: \"lor1\"},\n\t\t\t\t{Name: \"lor2\"},\n\t\t\t},\n\t\t\texpected: []corev1api.LocalObjectReference{\n\t\t\t\t{Name: \"lor3\"},\n\t\t\t\t{Name: \"lor4\"},\n\t\t\t\t{Name: \"lor1\"},\n\t\t\t\t{Name: \"lor2\"},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tresult := mergeLocalObjectReferenceSlices(test.first, test.second)\n\t\t\tassert.Equal(t, test.expected, result)\n\t\t})\n\t}\n}\n\nfunc TestMergeObjectReferenceSlices(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tfirst []corev1api.ObjectReference\n\t\tsecond []corev1api.ObjectReference\n\t\texpected []corev1api.ObjectReference\n\t}{\n\t\t{\n\t\t\tname: \"two slices without overlapping elements\",\n\t\t\tfirst: []corev1api.ObjectReference{\n\t\t\t\t{Name: \"or1\"},\n\t\t\t\t{Name: \"or2\"},\n\t\t\t},\n\t\t\tsecond: []corev1api.ObjectReference{\n\t\t\t\t{Name: \"or3\"},\n\t\t\t\t{Name: \"or4\"},\n\t\t\t},\n\t\t\texpected: []corev1api.ObjectReference{\n\t\t\t\t{Name: \"or1\"},\n\t\t\t\t{Name: \"or2\"},\n\t\t\t\t{Name: \"or3\"},\n\t\t\t\t{Name: \"or4\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"two slices with an overlapping element\",\n\t\t\tfirst: []corev1api.ObjectReference{\n\t\t\t\t{Name: \"or1\"},\n\t\t\t\t{Name: \"or2\"},\n\t\t\t},\n\t\t\tsecond: []corev1api.ObjectReference{\n\t\t\t\t{Name: \"or3\"},\n\t\t\t\t{Name: \"or2\"},\n\t\t\t},\n\t\t\texpected: []corev1api.ObjectReference{\n\t\t\t\t{Name: \"or1\"},\n\t\t\t\t{Name: \"or2\"},\n\t\t\t\t{Name: \"or3\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"merging always adds elements to the end\",\n\t\t\tfirst: []corev1api.ObjectReference{\n\t\t\t\t{Name: \"or3\"},\n\t\t\t\t{Name: \"or4\"},\n\t\t\t},\n\t\t\tsecond: []corev1api.ObjectReference{\n\t\t\t\t{Name: \"or1\"},\n\t\t\t\t{Name: \"or2\"},\n\t\t\t},\n\t\t\texpected: []corev1api.ObjectReference{\n\t\t\t\t{Name: \"or3\"},\n\t\t\t\t{Name: \"or4\"},\n\t\t\t\t{Name: \"or1\"},\n\t\t\t\t{Name: \"or2\"},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tresult := mergeObjectReferenceSlices(test.first, test.second)\n\t\t\tassert.Equal(t, test.expected, result)\n\t\t})\n\t}\n}\n\n// stripWhitespace removes any Unicode whitespace from a string.\n// Useful for cleaning up formatting on expected JSON strings before comparison\nfunc stripWhitespace(s string) string {\n\treturn strings.Map(func(r rune) rune {\n\t\tif unicode.IsSpace(r) {\n\t\t\treturn -1\n\t\t}\n\t\treturn r\n\t}, s)\n}\n\nfunc TestMergeMaps(t *testing.T) {\n\tvar testCases = []struct {\n\t\tname string\n\t\tsource map[string]string\n\t\tdestination map[string]string\n\t\texpected map[string]string\n\t}{\n\t\t{\n\t\t\tname: \"nil destination should result in source being copied\",\n\t\t\tdestination: nil,\n\t\t\tsource: map[string]string{\n\t\t\t\t\"k1\": \"v1\",\n\t\t\t},\n\t\t\texpected: map[string]string{\n\t\t\t\t\"k1\": \"v1\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"keys missing from destination should be copied from source\",\n\t\t\tdestination: map[string]string{\n\t\t\t\t\"k2\": \"v2\",\n\t\t\t},\n\t\t\tsource: map[string]string{\n\t\t\t\t\"k1\": \"v1\",\n\t\t\t},\n\t\t\texpected: map[string]string{\n\t\t\t\t\"k1\": \"v1\",\n\t\t\t\t\"k2\": \"v2\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"matching key should not have value copied from source\",\n\t\t\tdestination: map[string]string{\n\t\t\t\t\"k1\": \"v1\",\n\t\t\t},\n\t\t\tsource: map[string]string{\n\t\t\t\t\"k1\": \"v2\",\n\t\t\t},\n\t\t\texpected: map[string]string{\n\t\t\t\t\"k1\": \"v1\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tresult := mergeMaps(tc.destination, tc.source)\n\n\t\t\tassert.Equal(t, tc.expected, result)\n\t\t})\n\t}\n}\n\nfunc TestGeneratePatch(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tfromCluster *unstructured.Unstructured\n\t\tdesired *unstructured.Unstructured\n\t\texpectedString string\n\t\texpectedErr bool\n\t}{\n\t\t{\n\t\t\tname: \"objects are equal, no patch needed\",\n\t\t\tfromCluster: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\"\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"default-token-abcde\" }\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\t\t\tdesired: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\"\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"default-token-abcde\" }\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\t\t\texpectedString: \"\",\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"patch is required when labels are present\",\n\t\t\tfromCluster: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\"\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"default-token-abcde\" }\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\t\t\tdesired: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\",\n\t\t\t\t\t\t\"labels\": {\n\t\t\t\t\t\t\t\"label1\": \"value1\",\n\t\t\t\t\t\t\t\"label2\": \"value2\"\n\t\t\t\t\t\t}\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"default-token-abcde\" }\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\t\t\texpectedString: stripWhitespace(\n\t\t\t\t`{\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"labels\": {\n\t\t\t\t\t\t\t\"label1\":\"value1\",\n\t\t\t\t\t\t\t\"label2\":\"value2\"\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}`,\n\t\t\t),\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"patch is required when annotations are present\",\n\t\t\tfromCluster: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\"\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"default-token-abcde\" }\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\t\t\tdesired: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\",\n\t\t\t\t\t\t\"annotations\" :{\n\t\t\t\t\t\t\t\"a1\": \"v1\",\n\t\t\t\t\t\t\t\"a2\": \"v2\"\n\t\t\t\t\t\t}\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"default-token-abcde\" }\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\t\t\texpectedString: stripWhitespace(\n\t\t\t\t`{\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"annotations\": {\n\t\t\t\t\t\t\t\"a1\":\"v1\",\n\t\t\t\t\t\t\t\"a2\":\"v2\"\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}`,\n\t\t\t),\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"patch is required many secrets are present\",\n\t\t\tfromCluster: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\"\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"default-token-abcde\" }\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\t\t\tdesired: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\"\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"default-token-abcde\" },\n\t\t\t\t\t\t{ \"name\": \"sekrit\" },\n\t\t\t\t\t\t{ \"name\": \"secrete\" }\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\t\t\texpectedString: stripWhitespace(\n\t\t\t\t`{\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{\"name\": \"default-token-abcde\"},\n\t\t\t\t\t\t{\"name\": \"sekrit\"},\n\t\t\t\t\t\t{\"name\": \"secrete\"}\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\t\t\texpectedErr: false,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tresult, err := generatePatch(test.fromCluster, test.desired)\n\t\t\tif assert.Equal(t, test.expectedErr, err != nil) {\n\t\t\t\tassert.Equal(t, test.expectedString, string(result))\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestMergeServiceAccountBasic(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tfromCluster *unstructured.Unstructured\n\t\tfromBackup *unstructured.Unstructured\n\t\texpectedRes *unstructured.Unstructured\n\t\texpectedErr bool\n\t}{\n\t\t{\n\t\t\tname: \"only default token\",\n\t\t\tfromCluster: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\"\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"default-token-abcde\" }\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\t\t\t// fromBackup doesn't have the default token because it is expected to already have been removed\n\t\t\t// by the service account action\n\t\t\tfromBackup: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\"\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": []\n\t\t\t\t}`,\n\t\t\t),\n\t\t\texpectedRes: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\"\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"default-token-abcde\" }\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\t\t},\n\t\t{\n\t\t\tname: \"service accounts with multiple secrets\",\n\t\t\tfromCluster: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\"\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"default-token-abcde\" },\n\t\t\t\t\t\t{ \"name\": \"my-secret\" },\n\t\t\t\t\t\t{ \"name\": \"sekrit\" }\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\t\t\t// fromBackup doesn't have the default token because it is expected to already have been removed\n\t\t\t// by the service account action\n\t\t\tfromBackup: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\"\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"my-old-secret\" },\n\t\t\t\t\t\t{ \"name\": \"secrete\"}\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\t\t\texpectedRes: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\"\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"default-token-abcde\" },\n\t\t\t\t\t\t{ \"name\": \"my-secret\" },\n\t\t\t\t\t\t{ \"name\": \"sekrit\" },\n\t\t\t\t\t\t{ \"name\": \"my-old-secret\" },\n\t\t\t\t\t\t{ \"name\": \"secrete\"}\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\t\t},\n\t\t{\n\t\t\tname: \"service accounts with labels and annotations\",\n\t\t\tfromCluster: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\",\n\t\t\t\t\t\t\"labels\": {\n\t\t\t\t\t\t\t\"l1\": \"v1\",\n\t\t\t\t\t\t\t\"l2\": \"v2\",\n\t\t\t\t\t\t\t\"l3\": \"v3\"\n\t\t\t\t\t\t},\n\t\t\t\t\t\t\"annotations\": {\n\t\t\t\t\t\t\t\"a1\": \"v1\",\n\t\t\t\t\t\t\t\"a2\": \"v2\",\n\t\t\t\t\t\t\t\"a3\": \"v3\",\n\t\t\t\t\t\t\t\"a4\": \"v4\"\n\t\t\t\t\t\t}\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"default-token-abcde\" }\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\t\t\t// fromBackup doesn't have the default token because it is expected to already have been removed\n\t\t\t// by the service account action\n\t\t\tfromBackup: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\",\n\t\t\t\t\t\t\"labels\": {\n\t\t\t\t\t\t\t\"l1\": \"v1\",\n\t\t\t\t\t\t\t\"l2\": \"v2\",\n\t\t\t\t\t\t\t\"l3\": \"v3\",\n\t\t\t\t\t\t\t\"l4\": \"v4\",\n\t\t\t\t\t\t\t\"l5\": \"v5\"\n\t\t\t\t\t\t},\n\t\t\t\t\t\t\"annotations\": {\n\t\t\t\t\t\t\t\"a1\": \"v1\",\n\t\t\t\t\t\t\t\"a2\": \"v2\",\n\t\t\t\t\t\t\t\"a3\": \"v3\",\n\t\t\t\t\t\t\t\"a4\": \"v4\",\n\t\t\t\t\t\t\t\"a5\": \"v5\",\n\t\t\t\t\t\t\t\"a6\": \"v6\"\n\t\t\t\t\t\t}\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": []\n\t\t\t\t}`,\n\t\t\t),\n\t\t\texpectedRes: velerotest.UnstructuredOrDie(\n\t\t\t\t`{\n\t\t\t\t\t\"kind\": \"ServiceAccount\",\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"metadata\": {\n\t\t\t\t\t\t\"namespace\": \"ns1\",\n\t\t\t\t\t\t\"name\": \"default\",\n\t\t\t\t\t\t\"labels\": {\n\t\t\t\t\t\t\t\"l1\": \"v1\",\n\t\t\t\t\t\t\t\"l2\": \"v2\",\n\t\t\t\t\t\t\t\"l3\": \"v3\",\n\t\t\t\t\t\t\t\"l4\": \"v4\",\n\t\t\t\t\t\t\t\"l5\": \"v5\"\n\t\t\t\t\t\t},\n\t\t\t\t\t\t\"annotations\": {\n\t\t\t\t\t\t\t\"a1\": \"v1\",\n\t\t\t\t\t\t\t\"a2\": \"v2\",\n\t\t\t\t\t\t\t\"a3\": \"v3\",\n\t\t\t\t\t\t\t\"a4\": \"v4\",\n\t\t\t\t\t\t\t\"a5\": \"v5\",\n\t\t\t\t\t\t\t\"a6\": \"v6\"\n\t\t\t\t\t\t}\n\t\t\t\t\t},\n\t\t\t\t\t\"secrets\": [\n\t\t\t\t\t\t{ \"name\": \"default-token-abcde\" }\n\t\t\t\t\t]\n\t\t\t\t}`,\n\t\t\t),\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tresult, err := mergeServiceAccounts(test.fromCluster, test.fromBackup)\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Equal(t, test.expectedRes, result)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/restore/prioritize_group_version.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"context\"\n\t\"sort\"\n\t\"strings\"\n\n\t\"github.com/pkg/errors\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/version\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/archive\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n)\n\n// ChosenGroupVersion is the API Group version that was selected to restore\n// from potentially multiple backed up version enabled by the feature flag\n// APIGroupVersionsFeatureFlag\ntype ChosenGroupVersion struct {\n\tGroup string\n\tVersion string\n\tDir string\n}\n\n// chooseAPIVersionsToRestore will choose a version to restore based on a user-\n// provided config map prioritization or our version prioritization.\nfunc (ctx *restoreContext) chooseAPIVersionsToRestore() error {\n\tsourceGVs, targetGVs, userGVs, err := ctx.gatherSourceTargetUserGroupVersions()\n\tif err != nil {\n\t\treturn err\n\t}\n\nOUTER:\n\tfor rg, sg := range sourceGVs {\n\t\t// Default to the source preferred version if no other common version\n\t\t// can be found.\n\t\tcgv := ChosenGroupVersion{\n\t\t\tGroup: sg.Name,\n\t\t\tVersion: sg.PreferredVersion.Version,\n\t\t\tDir: sg.PreferredVersion.Version + velerov1api.PreferredVersionDir,\n\t\t}\n\n\t\ttg := findAPIGroup(targetGVs, sg.Name)\n\t\tif len(tg.Versions) == 0 {\n\t\t\tctx.chosenGrpVersToRestore[rg] = cgv\n\t\t\tctx.log.Debugf(\"Chose %s/%s API group version to restore\", cgv.Group, cgv.Version)\n\t\t\tcontinue\n\t\t}\n\n\t\t// Priority 0: User Priority Version\n\t\tif userGVs != nil {\n\t\t\tuv := findSupportedUserVersion(userGVs[rg].Versions, tg.Versions, sg.Versions)\n\t\t\tif uv != \"\" {\n\t\t\t\tcgv.Version = uv\n\t\t\t\tcgv.Dir = uv\n\n\t\t\t\tif uv == sg.PreferredVersion.Version {\n\t\t\t\t\tcgv.Dir += velerov1api.PreferredVersionDir\n\t\t\t\t}\n\n\t\t\t\tctx.chosenGrpVersToRestore[rg] = cgv\n\t\t\t\tctx.log.Debugf(\"APIGroupVersionsFeatureFlag Priority 0: User defined API group version %s chosen for %s\", uv, rg)\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tctx.log.Infof(\"Cannot find user defined version in both the cluster and backup cluster. Ignoring version %s for %s\", uv, rg)\n\t\t}\n\n\t\t// Priority 1: Target Cluster Preferred Version\n\t\tif versionsContain(sg.Versions, tg.PreferredVersion.Version) {\n\t\t\tcgv.Version = tg.PreferredVersion.Version\n\t\t\tcgv.Dir = tg.PreferredVersion.Version\n\n\t\t\tif tg.PreferredVersion.Version == sg.PreferredVersion.Version {\n\t\t\t\tcgv.Dir += velerov1api.PreferredVersionDir\n\t\t\t}\n\n\t\t\tctx.chosenGrpVersToRestore[rg] = cgv\n\t\t\tctx.log.Debugf(\n\t\t\t\t\"APIGroupVersionsFeatureFlag Priority 1: Cluster preferred API group version %s found in backup for %s\",\n\t\t\t\ttg.PreferredVersion.Version,\n\t\t\t\trg,\n\t\t\t)\n\t\t\tcontinue\n\t\t}\n\t\tctx.log.Infof(\"Cannot find cluster preferred API group version in backup. Ignoring version %s for %s\", tg.PreferredVersion.Version, rg)\n\n\t\t// Priority 2: Source Cluster Preferred Version\n\t\tif versionsContain(tg.Versions, sg.PreferredVersion.Version) {\n\t\t\tcgv.Version = sg.PreferredVersion.Version\n\t\t\tcgv.Dir = cgv.Version + velerov1api.PreferredVersionDir\n\n\t\t\tctx.chosenGrpVersToRestore[rg] = cgv\n\t\t\tctx.log.Debugf(\n\t\t\t\t\"APIGroupVersionsFeatureFlag Priority 2: Cluster preferred API group version not found in backup. Using backup preferred version %s for %s\",\n\t\t\t\tsg.PreferredVersion.Version,\n\t\t\t\trg,\n\t\t\t)\n\t\t\tcontinue\n\t\t}\n\t\tctx.log.Infof(\"Cannot find backup preferred API group version in cluster. Ignoring version %s for %s\", sg.PreferredVersion.Version, rg)\n\n\t\t// Priority 3: The Common Supported Version with the Highest Kubernetes Version Priority\n\t\tfor _, tv := range tg.Versions[1:] {\n\t\t\tif versionsContain(sg.Versions[1:], tv.Version) {\n\t\t\t\tcgv.Version = tv.Version\n\t\t\t\tcgv.Dir = tv.Version\n\n\t\t\t\tctx.chosenGrpVersToRestore[rg] = cgv\n\t\t\t\tctx.log.Debugf(\n\t\t\t\t\t\"APIGroupVersionsFeatureFlag Priority 3: Common supported but not preferred API group version %s chosen for %s\",\n\t\t\t\t\ttv.Version,\n\t\t\t\t\trg,\n\t\t\t\t)\n\t\t\t\tcontinue OUTER\n\t\t\t}\n\t\t}\n\t\tctx.log.Infof(\"Cannot find non-preferred a common supported API group version. Using %s (default behavior without feature flag) for %s\", sg.PreferredVersion.Version, rg)\n\n\t\t// Use default group version.\n\t\tctx.chosenGrpVersToRestore[rg] = cgv\n\t\tctx.log.Debugf(\n\t\t\t\"APIGroupVersionsFeatureFlag: Unable to find supported priority API group version. Using backup preferred version %s for %s (default behavior without feature flag).\",\n\t\t\ttg.PreferredVersion.Version,\n\t\t\trg,\n\t\t)\n\t}\n\n\treturn nil\n}\n\n// gatherSourceTargetUserGroupVersions collects the source, target, and user priority versions.\nfunc (ctx *restoreContext) gatherSourceTargetUserGroupVersions() (\n\tmap[string]metav1.APIGroup,\n\t[]metav1.APIGroup,\n\tmap[string]metav1.APIGroup,\n\terror,\n) {\n\tsourceRGVersions, err := archive.NewParser(ctx.log, ctx.fileSystem).ParseGroupVersions(ctx.restoreDir)\n\tif err != nil {\n\t\treturn nil, nil, nil, errors.Wrap(err, \"parsing versions from directory names\")\n\t}\n\n\t// Sort the versions in the APIGroups in sourceRGVersions map values.\n\tfor _, src := range sourceRGVersions {\n\t\tk8sPrioritySort(src.Versions)\n\t}\n\n\ttargetGroupVersions := ctx.discoveryHelper.APIGroups()\n\n\t// Sort the versions in the APIGroups slice in targetGroupVersions.\n\tfor _, target := range targetGroupVersions {\n\t\tk8sPrioritySort(target.Versions)\n\t}\n\n\t// Get the user-provided enableapigroupversion config map.\n\tcm, err := userPriorityConfigMap()\n\tif err != nil {\n\t\treturn nil, nil, nil, errors.Wrap(err, \"retrieving enableapigroupversion config map\")\n\t}\n\n\t// Read user-defined version priorities from config map.\n\tuserRGVPriorities := userResourceGroupVersionPriorities(ctx, cm)\n\n\treturn sourceRGVersions, targetGroupVersions, userRGVPriorities, nil\n}\n\n// k8sPrioritySort sorts slices using Kubernetes' version prioritization.\nfunc k8sPrioritySort(gvs []metav1.GroupVersionForDiscovery) {\n\tsort.SliceStable(gvs, func(i, j int) bool {\n\t\treturn version.CompareKubeAwareVersionStrings(gvs[i].Version, gvs[j].Version) > 0\n\t})\n}\n\n// userResourceGroupVersionPriorities retrieves a user-provided config map and\n// extracts the user priority versions for each resource.\nfunc userResourceGroupVersionPriorities(ctx *restoreContext, cm *corev1api.ConfigMap) map[string]metav1.APIGroup {\n\tif cm == nil {\n\t\tctx.log.Debugf(\"No enableapigroupversion config map found in velero namespace. Using pre-defined priorities.\")\n\t\treturn nil\n\t}\n\n\tpriorities := parseUserPriorities(ctx, cm.Data[\"restoreResourcesVersionPriority\"])\n\tif len(priorities) == 0 {\n\t\tctx.log.Debugf(\"No valid user version priorities found in enableapigroupversion config map. Using pre-defined priorities.\")\n\t\treturn nil\n\t}\n\n\treturn priorities\n}\n\nfunc userPriorityConfigMap() (*corev1api.ConfigMap, error) {\n\tcfg, err := client.LoadConfig()\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"reading client config file\")\n\t}\n\n\tfc := client.NewFactory(\"APIGroupVersionsRestore\", cfg)\n\n\tkc, err := fc.KubeClient()\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"getting Kube client\")\n\t}\n\n\tcm, err := kc.CoreV1().ConfigMaps(fc.Namespace()).Get(\n\t\tcontext.Background(),\n\t\t\"enableapigroupversions\",\n\t\tmetav1.GetOptions{},\n\t)\n\tif err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\treturn nil, nil\n\t\t}\n\n\t\treturn nil, errors.Wrap(err, \"getting enableapigroupversions config map from velero namespace\")\n\t}\n\n\treturn cm, nil\n}\n\nfunc parseUserPriorities(ctx *restoreContext, prioritiesData string) map[string]metav1.APIGroup {\n\tuserPriorities := make(map[string]metav1.APIGroup)\n\n\t// The user priorities will be in a string of the form\n\t// rockbands.music.example.io=v2beta1,v2beta2\\n\n\t// orchestras.music.example.io=v2,v3alpha1\\n\n\t// subscriptions.operators.coreos.com=v2,v1\n\n\tlines := strings.Split(prioritiesData, \"\\n\")\n\tlines = formatUserPriorities(lines)\n\n\tfor _, line := range lines {\n\t\terr := validateUserPriority(line)\n\n\t\tif err == nil {\n\t\t\trgvs := strings.SplitN(line, \"=\", 2)\n\t\t\trg := rgvs[0] // rockbands.music.example.io\n\t\t\tversions := rgvs[1] // v2beta1,v2beta2\n\n\t\t\tvers := strings.Split(versions, \",\")\n\n\t\t\tuserPriorities[rg] = metav1.APIGroup{\n\t\t\t\tVersions: versionsToGroupVersionForDiscovery(vers),\n\t\t\t}\n\t\t} else {\n\t\t\tctx.log.Debugf(\"Unable to validate user priority versions %q due to %v\", line, err)\n\t\t}\n\t}\n\n\treturn userPriorities\n}\n\n// formatUserPriorities removes extra white spaces that cause validation to fail.\nfunc formatUserPriorities(lines []string) []string {\n\ttrimmed := []string{}\n\n\tfor _, line := range lines {\n\t\ttemp := strings.ReplaceAll(line, \" \", \"\")\n\n\t\tif len(temp) > 0 {\n\t\t\ttrimmed = append(trimmed, temp)\n\t\t}\n\t}\n\n\treturn trimmed\n}\n\nfunc validateUserPriority(line string) error {\n\tif strings.Count(line, \"=\") != 1 {\n\t\treturn errors.New(\"line must have one and only one equal sign\")\n\t}\n\n\tpair := strings.Split(line, \"=\")\n\tif len(pair[0]) < 1 || len(pair[1]) < 1 {\n\t\treturn errors.New(\"line must contain at least one character before and after equal sign\")\n\t}\n\n\t// Line must not contain any spaces\n\tif strings.Count(line, \" \") > 0 {\n\t\treturn errors.New(\"line must not contain any spaces\")\n\t}\n\n\treturn nil\n}\n\n// versionsToGroupVersionForDiscovery converts version strings into a Kubernetes format\n// for group versions.\nfunc versionsToGroupVersionForDiscovery(vs []string) []metav1.GroupVersionForDiscovery {\n\tgvs := make([]metav1.GroupVersionForDiscovery, len(vs))\n\n\tfor i, v := range vs {\n\t\tgvs[i] = metav1.GroupVersionForDiscovery{\n\t\t\tVersion: v,\n\t\t}\n\t}\n\n\treturn gvs\n}\n\n// findAPIGroup looks for an API Group by a group name.\nfunc findAPIGroup(groups []metav1.APIGroup, name string) metav1.APIGroup {\n\tfor _, g := range groups {\n\t\tif g.Name == name {\n\t\t\treturn g\n\t\t}\n\t}\n\n\treturn metav1.APIGroup{}\n}\n\n// findSupportedUserVersion finds the first user priority version that both source\n// and target support.\nfunc findSupportedUserVersion(userGVs, targetGVs, sourceGVs []metav1.GroupVersionForDiscovery) string {\n\tfor _, ug := range userGVs {\n\t\tif versionsContain(targetGVs, ug.Version) && versionsContain(sourceGVs, ug.Version) {\n\t\t\treturn ug.Version\n\t\t}\n\t}\n\n\treturn \"\"\n}\n\n// versionsContain will check if a version can be found in a slice of versions.\nfunc versionsContain(list []metav1.GroupVersionForDiscovery, version string) bool {\n\tfor _, v := range list {\n\t\tif v.Version == version {\n\t\t\treturn true\n\t\t}\n\t}\n\n\treturn false\n}\n" }, { "path": "pkg/restore/prioritize_group_version_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestK8sPrioritySort(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\torig []metav1.GroupVersionForDiscovery\n\t\twant []metav1.GroupVersionForDiscovery\n\t}{\n\t\t{\n\t\t\tname: \"sorts Kubernetes API group versions per k8s priority\",\n\t\t\torig: []metav1.GroupVersionForDiscovery{\n\t\t\t\t{Version: \"v2\"},\n\t\t\t\t{Version: \"v11alpha2\"},\n\t\t\t\t{Version: \"foo10\"},\n\t\t\t\t{Version: \"v10\"},\n\t\t\t\t{Version: \"v12alpha1\"},\n\t\t\t\t{Version: \"v3beta1\"},\n\t\t\t\t{Version: \"foo1\"},\n\t\t\t\t{Version: \"v1\"},\n\t\t\t\t{Version: \"v10beta3\"},\n\t\t\t\t{Version: \"v11beta2\"},\n\t\t\t},\n\t\t\twant: []metav1.GroupVersionForDiscovery{\n\t\t\t\t{Version: \"v10\"},\n\t\t\t\t{Version: \"v2\"},\n\t\t\t\t{Version: \"v1\"},\n\t\t\t\t{Version: \"v11beta2\"},\n\t\t\t\t{Version: \"v10beta3\"},\n\t\t\t\t{Version: \"v3beta1\"},\n\t\t\t\t{Version: \"v12alpha1\"},\n\t\t\t\t{Version: \"v11alpha2\"},\n\t\t\t\t{Version: \"foo1\"},\n\t\t\t\t{Version: \"foo10\"},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tk8sPrioritySort(tc.orig)\n\n\t\t\tassert.Equal(t, tc.want, tc.orig)\n\t\t})\n\t}\n}\n\nfunc TestUserResourceGroupVersionPriorities(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tcm *corev1api.ConfigMap\n\t\twant map[string]metav1.APIGroup\n\t\twantErrMsg string\n\t}{\n\t\t{\n\t\t\tname: \"retrieve version priority data from config map\",\n\t\t\tcm: builder.\n\t\t\t\tForConfigMap(\"velero\", \"enableapigroupversions\").\n\t\t\t\tData(\n\t\t\t\t\t\"restoreResourcesVersionPriority\",\n\t\t\t\t\t`rockbands.music.example.io=v2beta1,v2beta2\norchestras.music.example.io=v2,v3alpha1\nsubscriptions.operators.coreos.com=v2,v1`,\n\t\t\t\t).\n\t\t\t\tResult(),\n\t\t\twant: map[string]metav1.APIGroup{\n\t\t\t\t\"rockbands.music.example.io\": {Versions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t\t{Version: \"v2beta1\"},\n\t\t\t\t\t{Version: \"v2beta2\"},\n\t\t\t\t}},\n\t\t\t\t\"orchestras.music.example.io\": {Versions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t\t{Version: \"v2\"},\n\t\t\t\t\t{Version: \"v3alpha1\"},\n\t\t\t\t}},\n\t\t\t\t\"subscriptions.operators.coreos.com\": {Versions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t\t{Version: \"v2\"},\n\t\t\t\t\t{Version: \"v1\"},\n\t\t\t\t}},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"incorrect data format returns an error\",\n\t\t\tcm: builder.\n\t\t\t\tForConfigMap(\"velero\", \"enableapigroupversions\").\n\t\t\t\tData(\n\t\t\t\t\t\"restoreResourcesVersionPriority\",\n\t\t\t\t\t`rockbands.music.example.io=v2beta1,v2beta2\\n orchestras.music.example.io=v2,v3alpha1`,\n\t\t\t\t).\n\t\t\t\tResult(),\n\t\t\twant: nil,\n\t\t\twantErrMsg: \"parsing user priorities: validating user priority: line must have one and only one equal sign\",\n\t\t},\n\t\t{\n\t\t\tname: \"spaces and empty lines are removed before storing user version priorities\",\n\t\t\tcm: builder.\n\t\t\t\tForConfigMap(\"velero\", \"enableapigroupversions\").\n\t\t\t\tData(\n\t\t\t\t\t\"restoreResourcesVersionPriority\",\n\t\t\t\t\t` pods=v2,v1beta2\nhorizontalpodautoscalers.autoscaling = v2beta2\njobs.batch=v3 \n `,\n\t\t\t\t).\n\t\t\t\tResult(),\n\t\t\twant: map[string]metav1.APIGroup{\n\t\t\t\t\"pods\": {Versions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t\t{Version: \"v2\"},\n\t\t\t\t\t{Version: \"v1beta2\"},\n\t\t\t\t}},\n\t\t\t\t\"horizontalpodautoscalers.autoscaling\": {Versions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t\t{Version: \"v2beta2\"},\n\t\t\t\t}},\n\t\t\t\t\"jobs.batch\": {Versions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t\t{Version: \"v3\"},\n\t\t\t\t}},\n\t\t\t},\n\t\t},\n\t}\n\n\tfakeCtx := &restoreContext{\n\t\tlog: test.NewLogger(),\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Log(tc.name)\n\t\tpriorities := userResourceGroupVersionPriorities(fakeCtx, tc.cm)\n\n\t\tassert.Equal(t, tc.want, priorities)\n\t}\n}\n\nfunc TestFindAPIGroup(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\ttargetGrps []metav1.APIGroup\n\t\tgrpName string\n\t\twant metav1.APIGroup\n\t}{\n\t\t{\n\t\t\tname: \"return the API Group in target list matching group string\",\n\t\t\ttargetGrps: []metav1.APIGroup{\n\t\t\t\t{\n\t\t\t\t\tName: \"rbac.authorization.k8s.io\",\n\t\t\t\t\tVersions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t\t\t{Version: \"v2\"},\n\t\t\t\t\t},\n\t\t\t\t\tPreferredVersion: metav1.GroupVersionForDiscovery{Version: \"v2\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"\",\n\t\t\t\t\tVersions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t\t\t{Version: \"v1\"},\n\t\t\t\t\t},\n\t\t\t\t\tPreferredVersion: metav1.GroupVersionForDiscovery{Version: \"v1\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"velero.io\",\n\t\t\t\t\tVersions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t\t\t{Version: \"v2beta1\"},\n\t\t\t\t\t\t{Version: \"v2beta2\"},\n\t\t\t\t\t\t{Version: \"v2\"},\n\t\t\t\t\t},\n\t\t\t\t\tPreferredVersion: metav1.GroupVersionForDiscovery{Version: \"v2\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\tgrpName: \"velero.io\",\n\t\t\twant: metav1.APIGroup{\n\t\t\t\tName: \"velero.io\",\n\t\t\t\tVersions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t\t{Version: \"v2beta1\"},\n\t\t\t\t\t{Version: \"v2beta2\"},\n\t\t\t\t\t{Version: \"v2\"},\n\t\t\t\t},\n\t\t\t\tPreferredVersion: metav1.GroupVersionForDiscovery{Version: \"v2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"return empty API Group if no match in target list\",\n\t\t\ttargetGrps: []metav1.APIGroup{\n\t\t\t\t{\n\t\t\t\t\tName: \"rbac.authorization.k8s.io\",\n\t\t\t\t\tVersions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t\t\t{Version: \"v2\"},\n\t\t\t\t\t},\n\t\t\t\t\tPreferredVersion: metav1.GroupVersionForDiscovery{Version: \"v2\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"\",\n\t\t\t\t\tVersions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t\t\t{Version: \"v1\"},\n\t\t\t\t\t},\n\t\t\t\t\tPreferredVersion: metav1.GroupVersionForDiscovery{Version: \"v1\"},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"velero.io\",\n\t\t\t\t\tVersions: []metav1.GroupVersionForDiscovery{\n\t\t\t\t\t\t{Version: \"v2beta1\"},\n\t\t\t\t\t\t{Version: \"v2beta2\"},\n\t\t\t\t\t\t{Version: \"v2\"},\n\t\t\t\t\t},\n\t\t\t\t\tPreferredVersion: metav1.GroupVersionForDiscovery{Version: \"v2\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\tgrpName: \"autoscaling\",\n\t\t\twant: metav1.APIGroup{},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tgrp := findAPIGroup(tc.targetGrps, tc.grpName)\n\n\t\tassert.Equal(t, tc.want, grp)\n\t}\n}\n\nfunc TestFindSupportedUserVersion(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tuserGVs []metav1.GroupVersionForDiscovery\n\t\ttargetGVs []metav1.GroupVersionForDiscovery\n\t\tsourceGVs []metav1.GroupVersionForDiscovery\n\t\twant string\n\t}{\n\t\t{\n\t\t\tname: \"return the single user group version that has a match in both source and target clusters\",\n\t\t\tuserGVs: []metav1.GroupVersionForDiscovery{\n\t\t\t\t{Version: \"foo\"},\n\t\t\t\t{Version: \"v10alpha2\"},\n\t\t\t\t{Version: \"v3\"},\n\t\t\t},\n\t\t\ttargetGVs: []metav1.GroupVersionForDiscovery{\n\t\t\t\t{Version: \"v9\"},\n\t\t\t\t{Version: \"v10beta1\"},\n\t\t\t\t{Version: \"v10alpha2\"},\n\t\t\t\t{Version: \"v10alpha3\"},\n\t\t\t},\n\t\t\tsourceGVs: []metav1.GroupVersionForDiscovery{\n\t\t\t\t{Version: \"v10alpha2\"},\n\t\t\t\t{Version: \"v9beta1\"},\n\t\t\t},\n\t\t\twant: \"v10alpha2\",\n\t\t},\n\t\t{\n\t\t\tname: \"return the first user group version that has a match in both source and target clusters\",\n\t\t\tuserGVs: []metav1.GroupVersionForDiscovery{\n\t\t\t\t{Version: \"v2beta1\"},\n\t\t\t\t{Version: \"v2beta2\"},\n\t\t\t},\n\t\t\ttargetGVs: []metav1.GroupVersionForDiscovery{\n\t\t\t\t{Version: \"v2beta2\"},\n\t\t\t\t{Version: \"v2beta1\"},\n\t\t\t},\n\t\t\tsourceGVs: []metav1.GroupVersionForDiscovery{\n\t\t\t\t{Version: \"v1\"},\n\t\t\t\t{Version: \"v2beta2\"},\n\t\t\t\t{Version: \"v2beta1\"},\n\t\t\t},\n\t\t\twant: \"v2beta1\",\n\t\t},\n\t\t{\n\t\t\tname: \"return empty string if there's only matches in the source cluster, but not target\",\n\t\t\tuserGVs: []metav1.GroupVersionForDiscovery{\n\t\t\t\t{Version: \"v1\"},\n\t\t\t},\n\t\t\ttargetGVs: []metav1.GroupVersionForDiscovery{\n\t\t\t\t{Version: \"v2\"},\n\t\t\t},\n\t\t\tsourceGVs: []metav1.GroupVersionForDiscovery{\n\t\t\t\t{Version: \"v1\"},\n\t\t\t},\n\t\t\twant: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"return empty string if there's only matches in the target cluster, but not source\",\n\t\t\tuserGVs: []metav1.GroupVersionForDiscovery{\n\t\t\t\t{Version: \"v3\"},\n\t\t\t\t{Version: \"v1\"},\n\t\t\t},\n\t\t\ttargetGVs: []metav1.GroupVersionForDiscovery{\n\t\t\t\t{Version: \"v3\"},\n\t\t\t\t{Version: \"v3beta2\"},\n\t\t\t},\n\t\t\tsourceGVs: []metav1.GroupVersionForDiscovery{\n\t\t\t\t{Version: \"v2\"},\n\t\t\t\t{Version: \"v2beta1\"},\n\t\t\t},\n\t\t\twant: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"return empty string if there is no match with either target and source clusters\",\n\t\t\tuserGVs: []metav1.GroupVersionForDiscovery{\n\t\t\t\t{Version: \"v2beta2\"},\n\t\t\t\t{Version: \"v2beta1\"},\n\t\t\t\t{Version: \"v2beta3\"},\n\t\t\t},\n\t\t\ttargetGVs: []metav1.GroupVersionForDiscovery{\n\t\t\t\t{Version: \"v2\"},\n\t\t\t\t{Version: \"v1\"},\n\t\t\t\t{Version: \"v2alpha1\"},\n\t\t\t},\n\t\t\tsourceGVs: []metav1.GroupVersionForDiscovery{\n\t\t\t\t{Version: \"v1\"},\n\t\t\t\t{Version: \"v2alpha1\"},\n\t\t\t},\n\t\t\twant: \"\",\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tuv := findSupportedUserVersion(tc.userGVs, tc.targetGVs, tc.sourceGVs)\n\n\t\tassert.Equal(t, tc.want, uv)\n\t}\n}\n\nfunc TestVersionsContain(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tGVs []metav1.GroupVersionForDiscovery\n\t\tver string\n\t\twant bool\n\t}{\n\t\t{\n\t\t\tname: \"version is not in list\",\n\t\t\tGVs: []metav1.GroupVersionForDiscovery{\n\t\t\t\t{Version: \"v1\"},\n\t\t\t\t{Version: \"v2alpha1\"},\n\t\t\t\t{Version: \"v2beta1\"},\n\t\t\t},\n\t\t\tver: \"v2\",\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"version is in list\",\n\t\t\tGVs: []metav1.GroupVersionForDiscovery{\n\t\t\t\t{Version: \"v2\"},\n\t\t\t\t{Version: \"v2alpha1\"},\n\t\t\t\t{Version: \"v2beta1\"},\n\t\t\t},\n\t\t\tver: \"v2\",\n\t\t\twant: true,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tassert.Equal(t, tc.want, versionsContain(tc.GVs, tc.ver))\n\t}\n}\n" }, { "path": "pkg/restore/pv_restorer.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n)\n\ntype PVRestorer interface {\n\texecutePVAction(obj *unstructured.Unstructured) (*unstructured.Unstructured, error)\n}\n\ntype pvRestorer struct {\n\tlogger logrus.FieldLogger\n\tbackup *api.Backup\n\trestorePVs *bool\n\tvolumeSnapshots []*volume.Snapshot\n\tvolumeSnapshotterGetter VolumeSnapshotterGetter\n\tkbclient client.Client\n\tcredentialFileStore credentials.FileStore\n\tvolInfoTracker *volume.RestoreVolumeInfoTracker\n}\n\nfunc (r *pvRestorer) executePVAction(obj *unstructured.Unstructured) (*unstructured.Unstructured, error) {\n\tpvName := obj.GetName()\n\tif pvName == \"\" {\n\t\treturn nil, errors.New(\"PersistentVolume is missing its name\")\n\t}\n\n\tif boolptr.IsSetToFalse(r.restorePVs) {\n\t\t// The restore has pv restores disabled, so we can return early\n\t\treturn obj, nil\n\t}\n\n\tlog := r.logger.WithFields(logrus.Fields{\"persistentVolume\": pvName})\n\n\tsnapshotInfo, err := getSnapshotInfo(pvName, r.backup, r.volumeSnapshots, r.kbclient, r.credentialFileStore)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif snapshotInfo == nil {\n\t\tlog.Infof(\"No snapshot found for persistent volume\")\n\t\treturn obj, nil\n\t}\n\n\tvolumeSnapshotter, err := r.volumeSnapshotterGetter.GetVolumeSnapshotter(snapshotInfo.location.Spec.Provider)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tif err := volumeSnapshotter.Init(snapshotInfo.location.Spec.Config); err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tvolumeID, err := volumeSnapshotter.CreateVolumeFromSnapshot(snapshotInfo.providerSnapshotID, snapshotInfo.volumeType, snapshotInfo.volumeAZ, snapshotInfo.volumeIOPS)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\tlog.WithField(\"providerSnapshotID\", snapshotInfo.providerSnapshotID).Info(\"successfully restored persistent volume from snapshot\")\n\n\tupdated1, err := volumeSnapshotter.SetVolumeID(obj, volumeID)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\tupdated2, ok := updated1.(*unstructured.Unstructured)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"unexpected type %T\", updated1)\n\t}\n\tvar iops int64\n\tif snapshotInfo.volumeIOPS != nil {\n\t\tiops = *snapshotInfo.volumeIOPS\n\t}\n\tr.volInfoTracker.TrackNativeSnapshot(updated2.GetName(), snapshotInfo.providerSnapshotID, snapshotInfo.volumeType, snapshotInfo.volumeAZ, iops)\n\treturn updated2, nil\n}\n\ntype snapshotInfo struct {\n\tproviderSnapshotID string\n\tvolumeType string\n\tvolumeAZ string\n\tvolumeIOPS *int64\n\tlocation *api.VolumeSnapshotLocation\n}\n\nfunc getSnapshotInfo(pvName string, backup *api.Backup, volumeSnapshots []*volume.Snapshot, client client.Client, credentialStore credentials.FileStore) (*snapshotInfo, error) {\n\tvar pvSnapshot *volume.Snapshot\n\tfor _, snapshot := range volumeSnapshots {\n\t\tif snapshot.Spec.PersistentVolumeName == pvName {\n\t\t\tpvSnapshot = snapshot\n\t\t\tbreak\n\t\t}\n\t}\n\n\tif pvSnapshot == nil {\n\t\treturn nil, nil\n\t}\n\n\tsnapshotLocation := &api.VolumeSnapshotLocation{}\n\terr := client.Get(\n\t\tcontext.Background(),\n\t\ttypes.NamespacedName{Namespace: backup.Namespace, Name: pvSnapshot.Spec.Location},\n\t\tsnapshotLocation,\n\t)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\t// add credential to config\n\terr = volume.UpdateVolumeSnapshotLocationWithCredentialConfig(snapshotLocation, credentialStore)\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\n\treturn &snapshotInfo{\n\t\tproviderSnapshotID: pvSnapshot.Status.ProviderSnapshotID,\n\t\tvolumeType: pvSnapshot.Spec.VolumeType,\n\t\tvolumeAZ: pvSnapshot.Spec.VolumeAZ,\n\t\tvolumeIOPS: pvSnapshot.Spec.VolumeIOPS,\n\t\tlocation: snapshotLocation,\n\t}, nil\n}\n" }, { "path": "pkg/restore/pv_restorer_test.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tprovidermocks \"github.com/vmware-tanzu/velero/pkg/plugin/velero/mocks/volumesnapshotter/v1\"\n\tvsv1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/volumesnapshotter/v1\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc defaultBackup() *builder.BackupBuilder {\n\treturn builder.ForBackup(api.DefaultNamespace, \"backup-1\")\n}\n\nfunc TestExecutePVAction_NoSnapshotRestores(t *testing.T) {\n\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t)\n\ttests := []struct {\n\t\tname string\n\t\tobj *unstructured.Unstructured\n\t\trestore *api.Restore\n\t\tbackup *api.Backup\n\t\tvolumeSnapshots []*volume.Snapshot\n\t\tlocations []*api.VolumeSnapshotLocation\n\t\texpectedErr bool\n\t\texpectedRes *unstructured.Unstructured\n\t}{\n\t\t{\n\t\t\tname: \"no name should error\",\n\t\t\tobj: newTestUnstructured().WithMetadata().Unstructured,\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").Result(),\n\t\t\texpectedErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"ensure spec.storageClassName is retained\",\n\t\t\tobj: newTestUnstructured().WithName(\"pv-1\").WithAnnotations(\"a\", \"b\").WithSpec(\"storageClassName\", \"someOtherField\").Unstructured,\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").RestorePVs(false).Result(),\n\t\t\tbackup: defaultBackup().Phase(api.BackupPhaseInProgress).Result(),\n\t\t\texpectedRes: newTestUnstructured().WithAnnotations(\"a\", \"b\").WithName(\"pv-1\").WithSpec(\"storageClassName\", \"someOtherField\").Unstructured,\n\t\t},\n\t\t{\n\t\t\tname: \"if backup.spec.snapshotVolumes is false, ignore restore.spec.restorePVs and return early\",\n\t\t\tobj: newTestUnstructured().WithName(\"pv-1\").WithAnnotations(\"a\", \"b\").WithSpec(\"claimRef\", \"storageClassName\", \"someOtherField\").Unstructured,\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").RestorePVs(true).Result(),\n\t\t\tbackup: defaultBackup().Phase(api.BackupPhaseInProgress).SnapshotVolumes(false).Result(),\n\t\t\texpectedRes: newTestUnstructured().WithName(\"pv-1\").WithAnnotations(\"a\", \"b\").WithSpec(\"claimRef\", \"storageClassName\", \"someOtherField\").Unstructured,\n\t\t},\n\t\t{\n\t\t\tname: \"restore.spec.restorePVs=false, return early\",\n\t\t\tobj: newTestUnstructured().WithName(\"pv-1\").WithSpec().Unstructured,\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").RestorePVs(false).Result(),\n\t\t\tbackup: defaultBackup().Phase(api.BackupPhaseInProgress).Result(),\n\t\t\tvolumeSnapshots: []*volume.Snapshot{\n\t\t\t\tnewSnapshot(\"pv-1\", \"loc-1\", \"gp\", \"az-1\", \"snap-1\", 1000),\n\t\t\t},\n\t\t\tlocations: []*api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(api.DefaultNamespace, \"loc-1\").Result(),\n\t\t\t},\n\t\t\texpectedErr: false,\n\t\t\texpectedRes: newTestUnstructured().WithName(\"pv-1\").WithSpec().Unstructured,\n\t\t},\n\t\t{\n\t\t\tname: \"volumeSnapshots is empty: return early\",\n\t\t\tobj: newTestUnstructured().WithName(\"pv-1\").WithSpec().Unstructured,\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").RestorePVs(true).Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tlocations: []*api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(api.DefaultNamespace, \"loc-1\").Result(),\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(api.DefaultNamespace, \"loc-2\").Result(),\n\t\t\t},\n\t\t\tvolumeSnapshots: []*volume.Snapshot{},\n\t\t\texpectedRes: newTestUnstructured().WithName(\"pv-1\").WithSpec().Unstructured,\n\t\t},\n\t\t{\n\t\t\tname: \"volumeSnapshots doesn't have a snapshot for PV: return early\",\n\t\t\tobj: newTestUnstructured().WithName(\"pv-1\").WithSpec().Unstructured,\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").RestorePVs(true).Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tlocations: []*api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(api.DefaultNamespace, \"loc-1\").Result(),\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(api.DefaultNamespace, \"loc-2\").Result(),\n\t\t\t},\n\t\t\tvolumeSnapshots: []*volume.Snapshot{\n\t\t\t\tnewSnapshot(\"non-matching-pv-1\", \"loc-1\", \"type-1\", \"az-1\", \"snap-1\", 1),\n\t\t\t\tnewSnapshot(\"non-matching-pv-2\", \"loc-2\", \"type-2\", \"az-2\", \"snap-2\", 2),\n\t\t\t},\n\t\t\texpectedRes: newTestUnstructured().WithName(\"pv-1\").WithSpec().Unstructured,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tr := &pvRestorer{\n\t\t\t\tlogger: velerotest.NewLogger(),\n\t\t\t\trestorePVs: tc.restore.Spec.RestorePVs,\n\t\t\t\tkbclient: velerotest.NewFakeControllerRuntimeClient(t),\n\t\t\t\tvolInfoTracker: volume.NewRestoreVolInfoTracker(tc.restore, logrus.New(), fakeClient),\n\t\t\t}\n\t\t\tif tc.backup != nil {\n\t\t\t\tr.backup = tc.backup\n\t\t\t}\n\n\t\t\tfor _, loc := range tc.locations {\n\t\t\t\trequire.NoError(t, r.kbclient.Create(t.Context(), loc))\n\t\t\t}\n\n\t\t\tres, err := r.executePVAction(tc.obj)\n\t\t\tswitch tc.expectedErr {\n\t\t\tcase true:\n\t\t\t\tassert.Nil(t, res)\n\t\t\t\trequire.Error(t, err)\n\t\t\tcase false:\n\t\t\t\tassert.Equal(t, tc.expectedRes, res)\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestExecutePVAction_SnapshotRestores(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tobj *unstructured.Unstructured\n\t\trestore *api.Restore\n\t\tbackup *api.Backup\n\t\tvolumeSnapshots []*volume.Snapshot\n\t\tlocations []*api.VolumeSnapshotLocation\n\t\texpectedProvider string\n\t\texpectedSnapshotID string\n\t\texpectedVolumeType string\n\t\texpectedVolumeAZ string\n\t\texpectedVolumeIOPS *int64\n\t\texpectedSnapshot *volume.Snapshot\n\t}{\n\t\t{\n\t\t\tname: \"backup with a matching volume.Snapshot for PV executes restore\",\n\t\t\tobj: newTestUnstructured().WithName(\"pv-1\").WithSpec().Unstructured,\n\t\t\trestore: builder.ForRestore(api.DefaultNamespace, \"\").RestorePVs(true).Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tlocations: []*api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(api.DefaultNamespace, \"loc-1\").Provider(\"provider-1\").Result(),\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(api.DefaultNamespace, \"loc-2\").Provider(\"provider-2\").Result(),\n\t\t\t},\n\t\t\tvolumeSnapshots: []*volume.Snapshot{\n\t\t\t\tnewSnapshot(\"pv-1\", \"loc-1\", \"type-1\", \"az-1\", \"snap-1\", 1),\n\t\t\t\tnewSnapshot(\"pv-2\", \"loc-2\", \"type-2\", \"az-2\", \"snap-2\", 2),\n\t\t\t},\n\t\t\texpectedProvider: \"provider-1\",\n\t\t\texpectedSnapshotID: \"snap-1\",\n\t\t\texpectedVolumeType: \"type-1\",\n\t\t\texpectedVolumeAZ: \"az-1\",\n\t\t\texpectedVolumeIOPS: int64Ptr(1),\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar (\n\t\t\t\tlogger = velerotest.NewLogger()\n\t\t\t\tvolumeSnapshotter = new(providermocks.VolumeSnapshotter)\n\t\t\t\tvolumeSnapshotterGetter = providerToVolumeSnapshotterMap(map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t\ttc.expectedProvider: volumeSnapshotter,\n\t\t\t\t})\n\t\t\t\tfakeClient = velerotest.NewFakeControllerRuntimeClientBuilder(t).Build()\n\t\t\t)\n\n\t\t\tfor _, loc := range tc.locations {\n\t\t\t\trequire.NoError(t, fakeClient.Create(t.Context(), loc))\n\t\t\t}\n\n\t\t\tr := &pvRestorer{\n\t\t\t\tlogger: logger,\n\t\t\t\tbackup: tc.backup,\n\t\t\t\tvolumeSnapshots: tc.volumeSnapshots,\n\t\t\t\tkbclient: fakeClient,\n\t\t\t\tvolumeSnapshotterGetter: volumeSnapshotterGetter,\n\t\t\t\tvolInfoTracker: volume.NewRestoreVolInfoTracker(tc.restore, logger, fakeClient),\n\t\t\t}\n\n\t\t\tvolumeSnapshotter.On(\"Init\", mock.Anything).Return(nil)\n\t\t\tvolumeSnapshotter.On(\"CreateVolumeFromSnapshot\", tc.expectedSnapshotID, tc.expectedVolumeType, tc.expectedVolumeAZ, tc.expectedVolumeIOPS).Return(\"volume-1\", nil)\n\t\t\tvolumeSnapshotter.On(\"SetVolumeID\", tc.obj, \"volume-1\").Return(tc.obj, nil)\n\n\t\t\t_, err := r.executePVAction(tc.obj)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tvolumeSnapshotter.AssertExpectations(t)\n\t\t})\n\t}\n}\n\ntype providerToVolumeSnapshotterMap map[string]vsv1.VolumeSnapshotter\n\nfunc (g providerToVolumeSnapshotterMap) GetVolumeSnapshotter(provider string) (vsv1.VolumeSnapshotter, error) {\n\tbs, ok := g[provider]\n\tif !ok {\n\t\treturn nil, errors.New(\"volume snapshotter not found for provider\")\n\t}\n\treturn bs, nil\n}\n\nfunc newSnapshot(pvName, location, volumeType, volumeAZ, snapshotID string, volumeIOPS int64) *volume.Snapshot {\n\treturn &volume.Snapshot{\n\t\tSpec: volume.SnapshotSpec{\n\t\t\tPersistentVolumeName: pvName,\n\t\t\tLocation: location,\n\t\t\tVolumeType: volumeType,\n\t\t\tVolumeAZ: volumeAZ,\n\t\t\tVolumeIOPS: &volumeIOPS,\n\t\t},\n\t\tStatus: volume.SnapshotStatus{\n\t\t\tProviderSnapshotID: snapshotID,\n\t\t},\n\t}\n}\n\nfunc int64Ptr(val int) *int64 {\n\tr := int64(val)\n\treturn &r\n}\n\ntype testUnstructured struct {\n\t*unstructured.Unstructured\n}\n\nfunc newTestUnstructured() *testUnstructured {\n\tobj := &testUnstructured{\n\t\tUnstructured: &unstructured.Unstructured{\n\t\t\tObject: make(map[string]any),\n\t\t},\n\t}\n\n\treturn obj\n}\n\nfunc (obj *testUnstructured) WithMetadata(fields ...string) *testUnstructured {\n\treturn obj.withMap(\"metadata\", fields...)\n}\n\nfunc (obj *testUnstructured) WithSpec(fields ...string) *testUnstructured {\n\tif _, found := obj.Object[\"spec\"]; found {\n\t\tpanic(\"spec already set - you probably didn't mean to do this twice!\")\n\t}\n\treturn obj.withMap(\"spec\", fields...)\n}\n\nfunc (obj *testUnstructured) WithStatus(fields ...string) *testUnstructured {\n\treturn obj.withMap(\"status\", fields...)\n}\n\nfunc (obj *testUnstructured) WithMetadataField(field string, value any) *testUnstructured {\n\treturn obj.withMapEntry(\"metadata\", field, value)\n}\n\nfunc (obj *testUnstructured) WithSpecField(field string, value any) *testUnstructured {\n\treturn obj.withMapEntry(\"spec\", field, value)\n}\n\nfunc (obj *testUnstructured) WithStatusField(field string, value any) *testUnstructured {\n\treturn obj.withMapEntry(\"status\", field, value)\n}\n\nfunc (obj *testUnstructured) WithAnnotations(fields ...string) *testUnstructured {\n\tvals := map[string]string{}\n\tfor _, field := range fields {\n\t\tvals[field] = \"foo\"\n\t}\n\n\treturn obj.WithAnnotationValues(vals)\n}\n\nfunc (obj *testUnstructured) WithAnnotationValues(fieldVals map[string]string) *testUnstructured {\n\tannotations := make(map[string]any)\n\tfor field, val := range fieldVals {\n\t\tannotations[field] = val\n\t}\n\n\tobj = obj.WithMetadataField(\"annotations\", annotations)\n\n\treturn obj\n}\n\nfunc (obj *testUnstructured) WithName(name string) *testUnstructured {\n\treturn obj.WithMetadataField(\"name\", name)\n}\n\nfunc (obj *testUnstructured) withMap(name string, fields ...string) *testUnstructured {\n\tm := make(map[string]any)\n\tobj.Object[name] = m\n\n\tfor _, field := range fields {\n\t\tm[field] = \"foo\"\n\t}\n\n\treturn obj\n}\n\nfunc (obj *testUnstructured) withMapEntry(mapName, field string, value any) *testUnstructured {\n\tvar m map[string]any\n\n\tif res, ok := obj.Unstructured.Object[mapName]; !ok {\n\t\tm = make(map[string]any)\n\t\tobj.Unstructured.Object[mapName] = m\n\t} else {\n\t\tm = res.(map[string]any)\n\t}\n\n\tm[field] = value\n\n\treturn obj\n}\n" }, { "path": "pkg/restore/request.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"sort\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/internal/resourcemodifiers\"\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\nconst (\n\tItemRestoreResultCreated = \"created\"\n\tItemRestoreResultUpdated = \"updated\"\n\tItemRestoreResultFailed = \"failed\"\n\tItemRestoreResultSkipped = \"skipped\"\n)\n\ntype itemKey struct {\n\tresource string\n\tnamespace string\n\tname string\n}\n\nfunc resourceKey(obj runtime.Object) string {\n\tgvk := obj.GetObjectKind().GroupVersionKind()\n\treturn fmt.Sprintf(\"%s/%s\", gvk.GroupVersion().String(), gvk.Kind)\n}\n\ntype Request struct {\n\t*velerov1api.Restore\n\n\tLog logrus.FieldLogger\n\tBackup *velerov1api.Backup\n\tPodVolumeBackups []*velerov1api.PodVolumeBackup\n\tVolumeSnapshots []*volume.Snapshot\n\tBackupReader io.Reader\n\tRestoredItems map[itemKey]restoredItemStatus\n\titemOperationsList *[]*itemoperation.RestoreOperation\n\tResourceModifiers *resourcemodifiers.ResourceModifiers\n\tDisableInformerCache bool\n\tCSIVolumeSnapshots []*snapshotv1api.VolumeSnapshot\n\tBackupVolumeInfoMap map[string]volume.BackupVolumeInfo\n\tRestoreVolumeInfoTracker *volume.RestoreVolumeInfoTracker\n\tResourceDeletionStatusTracker kube.ResourceDeletionStatusTracker\n}\n\ntype restoredItemStatus struct {\n\taction string\n\titemExists bool\n\tcreatedName string // Actual name assigned by K8s for GenerateName resources\n}\n\n// GetItemOperationsList returns ItemOperationsList, initializing it if necessary\nfunc (r *Request) GetItemOperationsList() *[]*itemoperation.RestoreOperation {\n\tif r.itemOperationsList == nil {\n\t\tlist := []*itemoperation.RestoreOperation{}\n\t\tr.itemOperationsList = &list\n\t}\n\treturn r.itemOperationsList\n}\n\n// RestoredResourceList returns the list of restored resources grouped by the API\n// Version and Kind\nfunc (r *Request) RestoredResourceList() map[string][]string {\n\tresources := map[string][]string{}\n\tfor i, item := range r.RestoredItems {\n\t\t// Use createdName if available (GenerateName case), otherwise itemKey.name\n\t\tname := i.name\n\t\tif item.createdName != \"\" {\n\t\t\tname = item.createdName\n\t\t}\n\n\t\tentry := name\n\t\tif i.namespace != \"\" {\n\t\t\tentry = fmt.Sprintf(\"%s/%s\", i.namespace, name)\n\t\t}\n\t\tentry = fmt.Sprintf(\"%s(%s)\", entry, item.action)\n\t\tresources[i.resource] = append(resources[i.resource], entry)\n\t}\n\n\t// sort namespace/name entries for each GVK\n\tfor _, v := range resources {\n\t\tsort.Strings(v)\n\t}\n\n\treturn resources\n}\n" }, { "path": "pkg/restore/request_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n)\n\nfunc TestResourceKey(t *testing.T) {\n\tnamespace := &corev1api.Namespace{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: \"v1\",\n\t\t\tKind: \"Namespace\",\n\t\t},\n\t}\n\tassert.Equal(t, \"v1/Namespace\", resourceKey(namespace))\n\n\tcr := &corev1api.Namespace{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tAPIVersion: \"customized/v1\",\n\t\t\tKind: \"Cron\",\n\t\t},\n\t}\n\tassert.Equal(t, \"customized/v1/Cron\", resourceKey(cr))\n}\n\nfunc TestRestoredResourceList(t *testing.T) {\n\trequest := &Request{\n\t\tRestoredItems: map[itemKey]restoredItemStatus{\n\t\t\t{\n\t\t\t\tresource: \"v1/Namespace\",\n\t\t\t\tnamespace: \"\",\n\t\t\t\tname: \"default\",\n\t\t\t}: {action: \"created\"},\n\t\t\t{\n\t\t\t\tresource: \"v1/ConfigMap\",\n\t\t\t\tnamespace: \"default\",\n\t\t\t\tname: \"cm\",\n\t\t\t}: {action: \"skipped\"},\n\t\t},\n\t}\n\n\texpected := map[string][]string{\n\t\t\"v1/ConfigMap\": {\"default/cm(skipped)\"},\n\t\t\"v1/Namespace\": {\"default(created)\"},\n\t}\n\n\tassert.Equal(t, expected, request.RestoredResourceList())\n}\n" }, { "path": "pkg/restore/restore.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\tgo_context \"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"os/signal\"\n\t\"path/filepath\"\n\t\"reflect\"\n\t\"slices\"\n\t\"sort\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/google/uuid\"\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/api/equality\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\tkubeerrs \"k8s.io/apimachinery/pkg/util/errors\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"k8s.io/client-go/dynamic/dynamicinformer\"\n\tcorev1 \"k8s.io/client-go/kubernetes/typed/core/v1\"\n\t\"k8s.io/client-go/tools/cache\"\n\t\"k8s.io/client-go/util/retry\"\n\tcrclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\t\"github.com/vmware-tanzu/velero/internal/hook\"\n\t\"github.com/vmware-tanzu/velero/internal/resourcemodifiers\"\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/archive\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/discovery\"\n\t\"github.com/vmware-tanzu/velero/pkg/features\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/framework\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\triav2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/restoreitemaction/v2\"\n\tvsv1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/volumesnapshotter/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/podexec\"\n\t\"github.com/vmware-tanzu/velero/pkg/podvolume\"\n\t\"github.com/vmware-tanzu/velero/pkg/podvolume/configs\"\n\t\"github.com/vmware-tanzu/velero/pkg/types\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/collections\"\n\tcsiutil \"github.com/vmware-tanzu/velero/pkg/util/csi\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/results\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/wildcard\"\n)\n\nconst ObjectStatusRestoreAnnotationKey = \"velero.io/restore-status\"\n\nvar resourceMustHave = []string{\n\t\"datauploads.velero.io\",\n\t\"volumesnapshotcontents.snapshot.storage.k8s.io\",\n}\n\ntype VolumeSnapshotterGetter interface {\n\tGetVolumeSnapshotter(name string) (vsv1.VolumeSnapshotter, error)\n}\n\n// Restorer knows how to restore a backup.\ntype Restorer interface {\n\t// Restore restores the backup data from backupReader, returning warnings and errors.\n\tRestore(req *Request,\n\t\tactions []riav2.RestoreItemAction,\n\t\tvolumeSnapshotterGetter VolumeSnapshotterGetter,\n\t) (results.Result, results.Result)\n\tRestoreWithResolvers(\n\t\treq *Request,\n\t\trestoreItemActionResolver framework.RestoreItemActionResolverV2,\n\t\tvolumeSnapshotterGetter VolumeSnapshotterGetter,\n\t) (results.Result, results.Result)\n}\n\n// kubernetesRestorer implements Restorer for restoring into a Kubernetes cluster.\ntype kubernetesRestorer struct {\n\tdiscoveryHelper discovery.Helper\n\tdynamicFactory client.DynamicFactory\n\tnamespaceClient corev1.NamespaceInterface\n\tpodVolumeRestorerFactory podvolume.RestorerFactory\n\tpodVolumeTimeout time.Duration\n\tpodVolumeContext go_context.Context\n\tresourceTerminatingTimeout time.Duration\n\tresourceTimeout time.Duration\n\tresourcePriorities types.Priorities\n\tfileSystem filesystem.Interface\n\tpvRenamer func(string) (string, error)\n\tlogger logrus.FieldLogger\n\tpodCommandExecutor podexec.PodCommandExecutor\n\tpodGetter cache.Getter\n\tcredentialFileStore credentials.FileStore\n\tkbClient crclient.Client\n\tmultiHookTracker *hook.MultiHookTracker\n\tresourceDeletionStatusTracker kube.ResourceDeletionStatusTracker\n}\n\n// NewKubernetesRestorer creates a new kubernetesRestorer.\nfunc NewKubernetesRestorer(\n\tdiscoveryHelper discovery.Helper,\n\tdynamicFactory client.DynamicFactory,\n\tresourcePriorities types.Priorities,\n\tnamespaceClient corev1.NamespaceInterface,\n\tpodVolumeRestorerFactory podvolume.RestorerFactory,\n\tpodVolumeTimeout time.Duration,\n\tresourceTerminatingTimeout time.Duration,\n\tresourceTimeout time.Duration,\n\tlogger logrus.FieldLogger,\n\tpodCommandExecutor podexec.PodCommandExecutor,\n\tpodGetter cache.Getter,\n\tcredentialStore credentials.FileStore,\n\tkbClient crclient.Client,\n\tmultiHookTracker *hook.MultiHookTracker,\n) (Restorer, error) {\n\treturn &kubernetesRestorer{\n\t\tdiscoveryHelper: discoveryHelper,\n\t\tdynamicFactory: dynamicFactory,\n\t\tnamespaceClient: namespaceClient,\n\t\tpodVolumeRestorerFactory: podVolumeRestorerFactory,\n\t\tpodVolumeTimeout: podVolumeTimeout,\n\t\tresourceTerminatingTimeout: resourceTerminatingTimeout,\n\t\tresourceTimeout: resourceTimeout,\n\t\tresourcePriorities: resourcePriorities,\n\t\tlogger: logger,\n\t\tpvRenamer: func(string) (string, error) {\n\t\t\tveleroCloneUUID, err := uuid.NewRandom()\n\t\t\tif err != nil {\n\t\t\t\treturn \"\", errors.WithStack(err)\n\t\t\t}\n\t\t\tveleroCloneName := \"velero-clone-\" + veleroCloneUUID.String()\n\t\t\treturn veleroCloneName, nil\n\t\t},\n\t\tfileSystem: filesystem.NewFileSystem(),\n\t\tpodCommandExecutor: podCommandExecutor,\n\t\tpodGetter: podGetter,\n\t\tcredentialFileStore: credentialStore,\n\t\tkbClient: kbClient,\n\t\tmultiHookTracker: multiHookTracker,\n\t}, nil\n}\n\n// Restore executes a restore into the target Kubernetes cluster according to the restore spec\n// and using data from the provided backup/backup reader. Returns a warnings and errors RestoreResult,\n// respectively, summarizing info about the restore.\nfunc (kr *kubernetesRestorer) Restore(\n\treq *Request,\n\tactions []riav2.RestoreItemAction,\n\tvolumeSnapshotterGetter VolumeSnapshotterGetter,\n) (results.Result, results.Result) {\n\tresolver := framework.NewRestoreItemActionResolverV2(actions)\n\treturn kr.RestoreWithResolvers(req, resolver, volumeSnapshotterGetter)\n}\n\nfunc (kr *kubernetesRestorer) RestoreWithResolvers(\n\treq *Request,\n\trestoreItemActionResolver framework.RestoreItemActionResolverV2,\n\tvolumeSnapshotterGetter VolumeSnapshotterGetter,\n) (results.Result, results.Result) {\n\t// metav1.LabelSelectorAsSelector converts a nil LabelSelector to a\n\t// Nothing Selector, i.e. a selector that matches nothing. We want\n\t// a selector that matches everything. This can be accomplished by\n\t// passing a non-nil empty LabelSelector.\n\tls := req.Restore.Spec.LabelSelector\n\tif ls == nil {\n\t\tls = &metav1.LabelSelector{}\n\t}\n\n\tvar OrSelectors []labels.Selector\n\tif req.Restore.Spec.OrLabelSelectors != nil {\n\t\tfor _, s := range req.Restore.Spec.OrLabelSelectors {\n\t\t\tlabelAsSelector, err := metav1.LabelSelectorAsSelector(s)\n\t\t\tif err != nil {\n\t\t\t\treturn results.Result{}, results.Result{Velero: []string{err.Error()}}\n\t\t\t}\n\t\t\tOrSelectors = append(OrSelectors, labelAsSelector)\n\t\t}\n\t}\n\n\tselector, err := metav1.LabelSelectorAsSelector(ls)\n\tif err != nil {\n\t\treturn results.Result{}, results.Result{Velero: []string{err.Error()}}\n\t}\n\n\t// Get resource includes-excludes.\n\tresourceIncludesExcludes := collections.GetResourceIncludesExcludes(\n\t\tkr.discoveryHelper,\n\t\treq.Restore.Spec.IncludedResources,\n\t\treq.Restore.Spec.ExcludedResources,\n\t)\n\n\t// Get resource status includes-excludes. Defaults to excluding all resources\n\tvar restoreStatusIncludesExcludes *collections.IncludesExcludes\n\n\tif req.Restore.Spec.RestoreStatus != nil {\n\t\trestoreStatusIncludesExcludes = collections.GetResourceIncludesExcludes(\n\t\t\tkr.discoveryHelper,\n\t\t\treq.Restore.Spec.RestoreStatus.IncludedResources,\n\t\t\treq.Restore.Spec.RestoreStatus.ExcludedResources,\n\t\t)\n\t}\n\n\t// Get namespace includes-excludes.\n\tnamespaceIncludesExcludes := collections.NewIncludesExcludes().\n\t\tIncludes(req.Restore.Spec.IncludedNamespaces...).\n\t\tExcludes(req.Restore.Spec.ExcludedNamespaces...)\n\n\tresolvedActions, err := restoreItemActionResolver.ResolveActions(kr.discoveryHelper, kr.logger)\n\tif err != nil {\n\t\treturn results.Result{}, results.Result{Velero: []string{err.Error()}}\n\t}\n\n\tpodVolumeTimeout := kr.podVolumeTimeout\n\tif val := req.Restore.Annotations[velerov1api.PodVolumeOperationTimeoutAnnotation]; val != \"\" {\n\t\tparsed, err := time.ParseDuration(val)\n\t\tif err != nil {\n\t\t\treq.Log.WithError(errors.WithStack(err)).Errorf(\n\t\t\t\t\"Unable to parse pod volume timeout annotation %s, using server value.\",\n\t\t\t\tval,\n\t\t\t)\n\t\t} else {\n\t\t\tpodVolumeTimeout = parsed\n\t\t}\n\t}\n\n\tvar podVolumeCancelFunc go_context.CancelFunc\n\tkr.podVolumeContext, podVolumeCancelFunc = go_context.WithTimeout(go_context.Background(), podVolumeTimeout)\n\tdefer podVolumeCancelFunc()\n\n\tvar podVolumeRestorer podvolume.Restorer\n\tif kr.podVolumeRestorerFactory != nil {\n\t\tpodVolumeRestorer, err = kr.podVolumeRestorerFactory.NewRestorer(kr.podVolumeContext, req.Restore)\n\t\tif err != nil {\n\t\t\treturn results.Result{}, results.Result{Velero: []string{err.Error()}}\n\t\t}\n\t}\n\n\twaitExecHookHandler := &hook.DefaultWaitExecHookHandler{\n\t\tPodCommandExecutor: kr.podCommandExecutor,\n\t\tListWatchFactory: &hook.DefaultListWatchFactory{\n\t\t\tPodsGetter: kr.podGetter,\n\t\t},\n\t}\n\n\thooksWaitExecutor, err := newHooksWaitExecutor(req.Restore, waitExecHookHandler)\n\tif err != nil {\n\t\treturn results.Result{}, results.Result{Velero: []string{err.Error()}}\n\t}\n\n\tpvRestorer := &pvRestorer{\n\t\tlogger: req.Log,\n\t\tbackup: req.Backup,\n\t\trestorePVs: req.Restore.Spec.RestorePVs,\n\t\tvolumeSnapshots: req.VolumeSnapshots,\n\t\tvolumeSnapshotterGetter: volumeSnapshotterGetter,\n\t\tkbclient: kr.kbClient,\n\t\tcredentialFileStore: kr.credentialFileStore,\n\t\tvolInfoTracker: req.RestoreVolumeInfoTracker,\n\t}\n\n\treq.RestoredItems = make(map[itemKey]restoredItemStatus)\n\n\trestoreCtx := &restoreContext{\n\t\tbackup: req.Backup,\n\t\tbackupReader: req.BackupReader,\n\t\trestore: req.Restore,\n\t\tresourceIncludesExcludes: resourceIncludesExcludes,\n\t\tresourceStatusIncludesExcludes: restoreStatusIncludesExcludes,\n\t\tnamespaceIncludesExcludes: namespaceIncludesExcludes,\n\t\tresourceMustHave: sets.New[string](resourceMustHave...),\n\t\tchosenGrpVersToRestore: make(map[string]ChosenGroupVersion),\n\t\tselector: selector,\n\t\tOrSelectors: OrSelectors,\n\t\tlog: req.Log,\n\t\tdynamicFactory: kr.dynamicFactory,\n\t\tfileSystem: kr.fileSystem,\n\t\tnamespaceClient: kr.namespaceClient,\n\t\trestoreItemActions: resolvedActions,\n\t\tvolumeSnapshotterGetter: volumeSnapshotterGetter,\n\t\tpodVolumeRestorer: podVolumeRestorer,\n\t\tpodVolumeErrs: make(chan error),\n\t\tpvsToProvision: sets.New[string](),\n\t\tpvRestorer: pvRestorer,\n\t\tvolumeSnapshots: req.VolumeSnapshots,\n\t\tcsiVolumeSnapshots: req.CSIVolumeSnapshots,\n\t\tpodVolumeBackups: req.PodVolumeBackups,\n\t\tresourceTerminatingTimeout: kr.resourceTerminatingTimeout,\n\t\tresourceTimeout: kr.resourceTimeout,\n\t\tresourceClients: make(map[resourceClientKey]client.Dynamic),\n\t\trestoredItems: req.RestoredItems,\n\t\trenamedPVs: make(map[string]string),\n\t\tpvRenamer: kr.pvRenamer,\n\t\tdiscoveryHelper: kr.discoveryHelper,\n\t\tresourcePriorities: kr.resourcePriorities,\n\t\tkbClient: kr.kbClient,\n\t\titemOperationsList: req.GetItemOperationsList(),\n\t\tresourceModifiers: req.ResourceModifiers,\n\t\tdisableInformerCache: req.DisableInformerCache,\n\t\tmultiHookTracker: kr.multiHookTracker,\n\t\tbackupVolumeInfoMap: req.BackupVolumeInfoMap,\n\t\trestoreVolumeInfoTracker: req.RestoreVolumeInfoTracker,\n\t\thooksWaitExecutor: hooksWaitExecutor,\n\t\tresourceDeletionStatusTracker: req.ResourceDeletionStatusTracker,\n\t}\n\n\treturn restoreCtx.execute()\n}\n\ntype restoreContext struct {\n\tbackup *velerov1api.Backup\n\tbackupReader io.Reader\n\trestore *velerov1api.Restore\n\trestoreDir string\n\tresourceIncludesExcludes *collections.IncludesExcludes\n\tresourceStatusIncludesExcludes *collections.IncludesExcludes\n\tnamespaceIncludesExcludes *collections.IncludesExcludes\n\tresourceMustHave sets.Set[string]\n\tchosenGrpVersToRestore map[string]ChosenGroupVersion\n\tselector labels.Selector\n\tOrSelectors []labels.Selector\n\tlog logrus.FieldLogger\n\tdynamicFactory client.DynamicFactory\n\tfileSystem filesystem.Interface\n\tnamespaceClient corev1.NamespaceInterface\n\trestoreItemActions []framework.RestoreItemResolvedActionV2\n\tvolumeSnapshotterGetter VolumeSnapshotterGetter\n\tpodVolumeRestorer podvolume.Restorer\n\tpodVolumeWaitGroup sync.WaitGroup\n\tpodVolumeErrs chan error\n\tpvsToProvision sets.Set[string]\n\tpvRestorer PVRestorer\n\tvolumeSnapshots []*volume.Snapshot\n\tcsiVolumeSnapshots []*snapshotv1api.VolumeSnapshot\n\tpodVolumeBackups []*velerov1api.PodVolumeBackup\n\tresourceTerminatingTimeout time.Duration\n\tresourceTimeout time.Duration\n\tresourceClients map[resourceClientKey]client.Dynamic\n\tdynamicInformerFactory *informerFactoryWithContext\n\trestoredItems map[itemKey]restoredItemStatus\n\trenamedPVs map[string]string\n\tpvRenamer func(string) (string, error)\n\tdiscoveryHelper discovery.Helper\n\tresourcePriorities types.Priorities\n\tkbClient crclient.Client\n\titemOperationsList *[]*itemoperation.RestoreOperation\n\tresourceModifiers *resourcemodifiers.ResourceModifiers\n\tdisableInformerCache bool\n\tmultiHookTracker *hook.MultiHookTracker\n\tbackupVolumeInfoMap map[string]volume.BackupVolumeInfo\n\trestoreVolumeInfoTracker *volume.RestoreVolumeInfoTracker\n\thooksWaitExecutor *hooksWaitExecutor\n\tresourceDeletionStatusTracker kube.ResourceDeletionStatusTracker\n}\n\ntype resourceClientKey struct {\n\tresource schema.GroupVersionResource\n\tnamespace string\n}\n\ntype informerFactoryWithContext struct {\n\tfactory dynamicinformer.DynamicSharedInformerFactory\n\tcontext go_context.Context\n\tcancel go_context.CancelFunc\n}\n\n// getOrderedResources returns an ordered list of resource identifiers to restore,\n// based on the provided resource priorities and backup contents. The returned list\n// begins with all of the high prioritized resources (in order), ends with all of\n// the low prioritized resources(in order), and an alphabetized list of resources\n// in the backup(pick out the prioritized resources) is put in the middle.\nfunc getOrderedResources(resourcePriorities types.Priorities, backupResources map[string]*archive.ResourceItems) []string {\n\tpriorities := map[string]struct{}{}\n\tfor _, priority := range resourcePriorities.HighPriorities {\n\t\tpriorities[priority] = struct{}{}\n\t}\n\tfor _, priority := range resourcePriorities.LowPriorities {\n\t\tpriorities[priority] = struct{}{}\n\t}\n\n\t// pick the prioritized resources out\n\tvar orderedBackupResources []string\n\tfor resource := range backupResources {\n\t\tif _, exist := priorities[resource]; exist {\n\t\t\tcontinue\n\t\t}\n\t\torderedBackupResources = append(orderedBackupResources, resource)\n\t}\n\t// alphabetize resources in the backup\n\tsort.Strings(orderedBackupResources)\n\n\tlist := append(resourcePriorities.HighPriorities, orderedBackupResources...)\n\treturn append(list, resourcePriorities.LowPriorities...)\n}\n\ntype progressUpdate struct {\n\ttotalItems, itemsRestored int\n}\n\nfunc (ctx *restoreContext) execute() (results.Result, results.Result) {\n\twarnings, errs := results.Result{}, results.Result{}\n\n\tctx.log.Infof(\"Starting restore of backup %s\", kube.NamespaceAndName(ctx.backup))\n\n\tdir, err := archive.NewExtractor(ctx.log, ctx.fileSystem).UnzipAndExtractBackup(ctx.backupReader)\n\tif err != nil {\n\t\tctx.log.Infof(\"error unzipping and extracting: %v\", err)\n\t\terrs.AddVeleroError(err)\n\t\treturn warnings, errs\n\t}\n\tdefer func() {\n\t\tif err := ctx.fileSystem.RemoveAll(dir); err != nil {\n\t\t\tctx.log.Errorf(\"error removing temporary directory %s: %s\", dir, err.Error())\n\t\t}\n\t}()\n\n\t// Need to stop all informers if enabled\n\tif !ctx.disableInformerCache {\n\t\tcontext, cancel := signal.NotifyContext(go_context.Background(), os.Interrupt)\n\t\tctx.dynamicInformerFactory = &informerFactoryWithContext{\n\t\t\tfactory: ctx.dynamicFactory.DynamicSharedInformerFactory(),\n\t\t\tcontext: context,\n\t\t\tcancel: cancel,\n\t\t}\n\n\t\tdefer func() {\n\t\t\t// Call the cancel func to close the channel for each started informer\n\t\t\tctx.dynamicInformerFactory.cancel()\n\t\t\t// After upgrading to client-go 0.27 or newer, also call Shutdown for each informer factory\n\t\t}()\n\t}\n\n\t// Need to set this for additionalItems to be restored.\n\tctx.restoreDir = dir\n\n\tbackupResources, err := archive.NewParser(ctx.log, ctx.fileSystem).Parse(ctx.restoreDir)\n\t// If ErrNotExist occurs, it implies that the backup to be restored includes zero items.\n\t// Need to add a warning about it and jump out of the function.\n\tif errors.Cause(err) == archive.ErrNotExist {\n\t\twarnings.AddVeleroError(errors.Wrap(err, \"zero items to be restored\"))\n\t\treturn warnings, errs\n\t}\n\tif err != nil {\n\t\terrs.AddVeleroError(errors.Wrap(err, \"error parsing backup contents\"))\n\t\treturn warnings, errs\n\t}\n\n\t// Expand wildcard patterns in namespace includes/excludes if needed\n\tif err := ctx.expandNamespaceWildcards(backupResources); err != nil {\n\t\terrs.AddVeleroError(err)\n\t\treturn warnings, errs\n\t}\n\n\t// TODO: Remove outer feature flag check to make this feature a default in Velero.\n\tif features.IsEnabled(velerov1api.APIGroupVersionsFeatureFlag) {\n\t\tif ctx.backup.Status.FormatVersion >= \"1.1.0\" {\n\t\t\tif err := ctx.chooseAPIVersionsToRestore(); err != nil {\n\t\t\t\terrs.AddVeleroError(errors.Wrap(err, \"choosing API version to restore\"))\n\t\t\t\treturn warnings, errs\n\t\t\t}\n\t\t}\n\t}\n\n\tupdate := make(chan progressUpdate)\n\n\tquit := make(chan struct{})\n\n\tgo func() {\n\t\tticker := time.NewTicker(1 * time.Second)\n\t\tvar lastUpdate *progressUpdate\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-quit:\n\t\t\t\tticker.Stop()\n\t\t\t\treturn\n\t\t\tcase val := <-update:\n\t\t\t\tlastUpdate = &val\n\t\t\tcase <-ticker.C:\n\t\t\t\tif lastUpdate != nil {\n\t\t\t\t\tupdated := ctx.restore.DeepCopy()\n\t\t\t\t\tif updated.Status.Progress == nil {\n\t\t\t\t\t\tupdated.Status.Progress = &velerov1api.RestoreProgress{}\n\t\t\t\t\t}\n\t\t\t\t\tupdated.Status.Progress.TotalItems = lastUpdate.totalItems\n\t\t\t\t\tupdated.Status.Progress.ItemsRestored = lastUpdate.itemsRestored\n\t\t\t\t\terr = kube.PatchResource(ctx.restore, updated, ctx.kbClient)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\tctx.log.WithError(errors.WithStack((err))).\n\t\t\t\t\t\t\tWarn(\"Got error trying to update restore's status.progress\")\n\t\t\t\t\t}\n\t\t\t\t\tlastUpdate = nil\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}()\n\n\t// totalItems: previously discovered items, i: iteration counter.\n\ttotalItems, processedItems, existingNamespaces := 0, 0, sets.New[string]()\n\n\t// First restore CRDs. This is needed so that they are available in the cluster\n\t// when getOrderedResourceCollection is called again on the whole backup and\n\t// needs to validate all resources listed.\n\tcrdResourceCollection, processedResources, w, e := ctx.getOrderedResourceCollection(\n\t\tbackupResources,\n\t\tmake([]restoreableResource, 0),\n\t\tsets.New[string](),\n\t\ttypes.Priorities{HighPriorities: []string{\"customresourcedefinitions\"}},\n\t\tfalse,\n\t)\n\twarnings.Merge(&w)\n\terrs.Merge(&e)\n\n\tfor _, selectedResource := range crdResourceCollection {\n\t\ttotalItems += selectedResource.totalItems\n\t}\n\n\tfor _, selectedResource := range crdResourceCollection {\n\t\tvar w, e results.Result\n\t\t// Restore this resource, the update channel is set to nil, to avoid misleading value of \"totalItems\"\n\t\t// more details see #5990\n\t\tprocessedItems, w, e = ctx.processSelectedResource(\n\t\t\tselectedResource,\n\t\t\ttotalItems,\n\t\t\tprocessedItems,\n\t\t\texistingNamespaces,\n\t\t\tnil,\n\t\t)\n\t\twarnings.Merge(&w)\n\t\terrs.Merge(&e)\n\t}\n\n\tvar createdOrUpdatedCRDs bool\n\tfor _, restoredItem := range ctx.restoredItems {\n\t\tif restoredItem.action == ItemRestoreResultCreated || restoredItem.action == ItemRestoreResultUpdated {\n\t\t\tcreatedOrUpdatedCRDs = true\n\t\t\tbreak\n\t\t}\n\t}\n\t// If we just restored custom resource definitions (CRDs), refresh\n\t// discovery because the restored CRDs may have created or updated new APIs that\n\t// didn't previously exist in the cluster, and we want to be able to\n\t// resolve & restore instances of them in subsequent loop iterations.\n\tif createdOrUpdatedCRDs {\n\t\tif err := ctx.discoveryHelper.Refresh(); err != nil {\n\t\t\twarnings.Add(\"\", errors.Wrap(err, \"refresh discovery after restoring CRDs\"))\n\t\t}\n\t}\n\n\t// Restore everything else\n\tselectedResourceCollection, _, w, e := ctx.getOrderedResourceCollection(\n\t\tbackupResources,\n\t\tcrdResourceCollection,\n\t\tprocessedResources,\n\t\tctx.resourcePriorities,\n\t\ttrue,\n\t)\n\twarnings.Merge(&w)\n\terrs.Merge(&e)\n\n\t// initialize informer caches for selected resources if enabled\n\tif !ctx.disableInformerCache {\n\t\tfor _, informerResource := range selectedResourceCollection {\n\t\t\tif informerResource.totalItems == 0 {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tversion := \"\"\n\t\t\tfor _, items := range informerResource.selectedItemsByNamespace {\n\t\t\t\tif len(items) == 0 {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tversion = items[0].version\n\t\t\t\tbreak\n\t\t\t}\n\t\t\tgvr := schema.ParseGroupResource(informerResource.resource).WithVersion(version)\n\t\t\t_, _, err := ctx.discoveryHelper.ResourceFor(gvr)\n\t\t\tif err != nil {\n\t\t\t\tctx.log.Infof(\"failed to create informer for %s: %v\", gvr, err)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tctx.dynamicInformerFactory.factory.ForResource(gvr)\n\t\t}\n\t\tctx.dynamicInformerFactory.factory.Start(ctx.dynamicInformerFactory.context.Done())\n\t\tctx.log.Info(\"waiting informer cache sync ...\")\n\t\tctx.dynamicInformerFactory.factory.WaitForCacheSync(ctx.dynamicInformerFactory.context.Done())\n\t}\n\n\t// reset processedItems and totalItems before processing full resource list\n\tprocessedItems = 0\n\ttotalItems = 0\n\tfor _, selectedResource := range selectedResourceCollection {\n\t\ttotalItems += selectedResource.totalItems\n\t}\n\n\tfor _, selectedResource := range selectedResourceCollection {\n\t\tvar w, e results.Result\n\t\t// Restore this resource\n\t\tprocessedItems, w, e = ctx.processSelectedResource(\n\t\t\tselectedResource,\n\t\t\ttotalItems,\n\t\t\tprocessedItems,\n\t\t\texistingNamespaces,\n\t\t\tupdate,\n\t\t)\n\t\twarnings.Merge(&w)\n\t\terrs.Merge(&e)\n\t}\n\n\t// Close the progress update channel.\n\tquit <- struct{}{}\n\n\t// Clean the DataUploadResult ConfigMaps\n\tdefer func() {\n\t\topts := []crclient.DeleteAllOfOption{\n\t\t\tcrclient.InNamespace(ctx.restore.Namespace),\n\t\t\tcrclient.MatchingLabels{\n\t\t\t\tvelerov1api.RestoreUIDLabel: string(ctx.restore.UID),\n\t\t\t\tvelerov1api.ResourceUsageLabel: string(velerov1api.VeleroResourceUsageDataUploadResult),\n\t\t\t},\n\t\t}\n\t\terr := ctx.kbClient.DeleteAllOf(go_context.Background(), &corev1api.ConfigMap{}, opts...)\n\t\tif err != nil {\n\t\t\tctx.log.Errorf(\"Fail to batch delete DataUploadResult ConfigMaps for restore %s: %s\", ctx.restore.Name, err.Error())\n\t\t}\n\t}()\n\n\t// Do a final progress update as stopping the ticker might have left last few\n\t// updates from taking place.\n\tupdated := ctx.restore.DeepCopy()\n\tif updated.Status.Progress == nil {\n\t\tupdated.Status.Progress = &velerov1api.RestoreProgress{}\n\t}\n\tupdated.Status.Progress.TotalItems = len(ctx.restoredItems)\n\tupdated.Status.Progress.ItemsRestored = len(ctx.restoredItems)\n\n\t// patch the restore\n\terr = kube.PatchResource(ctx.restore, updated, ctx.kbClient)\n\tif err != nil {\n\t\tctx.log.WithError(errors.WithStack((err))).Warn(\"Updating restore status\")\n\t}\n\n\t// Wait for all of the pod volume restore goroutines to be done, which is\n\t// only possible once all of their errors have been received by the loop\n\t// below, then close the podVolumeErrs channel so the loop terminates.\n\tgo func() {\n\t\tctx.log.Info(\"Waiting for all pod volume restores to complete\")\n\n\t\t// TODO timeout?\n\t\tctx.podVolumeWaitGroup.Wait()\n\t\tclose(ctx.podVolumeErrs)\n\t}()\n\n\t// This loop will only terminate when the ctx.podVolumeErrs channel is closed\n\t// in the above goroutine, *after* all errors from the goroutines have been\n\t// received by this loop.\n\tfor err := range ctx.podVolumeErrs {\n\t\t// TODO: not ideal to be adding these to Velero-level errors\n\t\t// rather than a specific namespace, but don't have a way\n\t\t// to track the namespace right now.\n\t\terrs.Velero = append(errs.Velero, err.Error())\n\t}\n\tctx.log.Info(\"Done waiting for all pod volume restores to complete\")\n\n\treturn warnings, errs\n}\n\n// Process and restore one restoreableResource from the backup and update restore progress\n// metadata. At this point, the resource has already been validated and counted for inclusion\n// in the expected total restore count.\nfunc (ctx *restoreContext) processSelectedResource(\n\tselectedResource restoreableResource,\n\ttotalItems int,\n\tprocessedItems int,\n\texistingNamespaces sets.Set[string],\n\tupdate chan progressUpdate,\n) (int, results.Result, results.Result) {\n\twarnings, errs := results.Result{}, results.Result{}\n\tgroupResource := schema.ParseGroupResource(selectedResource.resource)\n\n\tfor namespace, selectedItems := range selectedResource.selectedItemsByNamespace {\n\t\tfor _, selectedItem := range selectedItems {\n\t\t\ttargetNS := selectedItem.targetNamespace\n\t\t\tif groupResource == kuberesource.Namespaces {\n\t\t\t\t// namespace is a cluster-scoped resource and doesn't have \"targetNamespace\" attribute in the restoreableItem instance\n\t\t\t\tnamespace = selectedItem.name\n\t\t\t\tif n, ok := ctx.restore.Spec.NamespaceMapping[namespace]; ok {\n\t\t\t\t\ttargetNS = n\n\t\t\t\t} else {\n\t\t\t\t\ttargetNS = namespace\n\t\t\t\t}\n\t\t\t}\n\t\t\t// If we don't know whether this namespace exists yet, attempt to create\n\t\t\t// it in order to ensure it exists. Try to get it from the backup tarball\n\t\t\t// (in order to get any backed-up metadata), but if we don't find it there,\n\t\t\t// create a blank one.\n\t\t\tif namespace != \"\" && !existingNamespaces.Has(targetNS) {\n\t\t\t\tlogger := ctx.log.WithField(\"namespace\", namespace)\n\n\t\t\t\tns := getNamespace(\n\t\t\t\t\tlogger,\n\t\t\t\t\tarchive.GetItemFilePath(ctx.restoreDir, \"namespaces\", \"\", namespace),\n\t\t\t\t\ttargetNS,\n\t\t\t\t)\n\t\t\t\t_, nsCreated, err := kube.EnsureNamespaceExistsAndIsReady(\n\t\t\t\t\tns,\n\t\t\t\t\tctx.namespaceClient,\n\t\t\t\t\tctx.resourceTerminatingTimeout,\n\t\t\t\t\tctx.resourceDeletionStatusTracker,\n\t\t\t\t)\n\t\t\t\tif err != nil {\n\t\t\t\t\terrs.AddVeleroError(err)\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\n\t\t\t\t// Add the newly created namespace to the list of restored items.\n\t\t\t\tif nsCreated {\n\t\t\t\t\titemKey := itemKey{\n\t\t\t\t\t\tresource: resourceKey(ns),\n\t\t\t\t\t\tnamespace: ns.Namespace,\n\t\t\t\t\t\tname: ns.Name,\n\t\t\t\t\t}\n\t\t\t\t\tctx.restoredItems[itemKey] = restoredItemStatus{action: ItemRestoreResultCreated, itemExists: true, createdName: ns.Name}\n\t\t\t\t}\n\n\t\t\t\t// Keep track of namespaces that we know exist so we don't\n\t\t\t\t// have to try to create them multiple times.\n\t\t\t\texistingNamespaces.Insert(targetNS)\n\t\t\t}\n\t\t\t// For namespaces resources we don't need to following steps\n\t\t\tif groupResource == kuberesource.Namespaces {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tobj, err := archive.Unmarshal(ctx.fileSystem, selectedItem.path)\n\t\t\tif err != nil {\n\t\t\t\terrs.Add(\n\t\t\t\t\tselectedItem.targetNamespace,\n\t\t\t\t\tfmt.Errorf(\n\t\t\t\t\t\t\"error decoding %q: %v\",\n\t\t\t\t\t\tstrings.Replace(selectedItem.path, ctx.restoreDir+\"/\", \"\", -1),\n\t\t\t\t\t\terr,\n\t\t\t\t\t),\n\t\t\t\t)\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tw, e, _ := ctx.restoreItem(obj, groupResource, targetNS)\n\t\t\twarnings.Merge(&w)\n\t\t\terrs.Merge(&e)\n\t\t\tprocessedItems++\n\n\t\t\t// totalItems keeps the count of items previously known. There\n\t\t\t// may be additional items restored by plugins. We want to include\n\t\t\t// the additional items by looking at restoredItems at the same\n\t\t\t// time, we don't want previously known items counted twice as\n\t\t\t// they are present in both restoredItems and totalItems.\n\t\t\tactualTotalItems := len(ctx.restoredItems) + (totalItems - processedItems)\n\t\t\tif update != nil {\n\t\t\t\tupdate <- progressUpdate{\n\t\t\t\t\ttotalItems: actualTotalItems,\n\t\t\t\t\titemsRestored: len(ctx.restoredItems),\n\t\t\t\t}\n\t\t\t}\n\t\t\tctx.log.WithFields(map[string]any{\n\t\t\t\t\"progress\": \"\",\n\t\t\t\t\"resource\": groupResource.String(),\n\t\t\t\t\"namespace\": selectedItem.targetNamespace,\n\t\t\t\t\"name\": selectedItem.name,\n\t\t\t}).Infof(\"Restored %d items out of an estimated total of %d (estimate will change throughout the restore)\", len(ctx.restoredItems), actualTotalItems)\n\t\t}\n\t}\n\n\treturn processedItems, warnings, errs\n}\n\n// getNamespace returns a namespace API object that we should attempt to\n// create before restoring anything into it. It will come from the backup\n// tarball if it exists, else will be a new one. If from the tarball, it\n// will retain its labels, annotations, and spec.\nfunc getNamespace(logger logrus.FieldLogger, path, remappedName string) *corev1api.Namespace {\n\tvar nsBytes []byte\n\tvar err error\n\n\tif nsBytes, err = os.ReadFile(path); err != nil {\n\t\treturn &corev1api.Namespace{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tKind: \"Namespace\",\n\t\t\t\tAPIVersion: \"v1\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: remappedName,\n\t\t\t},\n\t\t}\n\t}\n\n\tvar backupNS corev1api.Namespace\n\tif err := json.Unmarshal(nsBytes, &backupNS); err != nil {\n\t\tlogger.Warnf(\"Error unmarshaling namespace from backup, creating new one.\")\n\t\treturn &corev1api.Namespace{\n\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\tKind: \"Namespace\",\n\t\t\t\tAPIVersion: \"v1\",\n\t\t\t},\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: remappedName,\n\t\t\t},\n\t\t}\n\t}\n\n\treturn &corev1api.Namespace{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: backupNS.Kind,\n\t\t\tAPIVersion: backupNS.APIVersion,\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: remappedName,\n\t\t\tLabels: backupNS.Labels,\n\t\t\tAnnotations: backupNS.Annotations,\n\t\t},\n\t\tSpec: backupNS.Spec,\n\t}\n}\n\nfunc (ctx *restoreContext) getApplicableActions(groupResource schema.GroupResource, namespace string) []framework.RestoreItemResolvedActionV2 {\n\tvar actions []framework.RestoreItemResolvedActionV2\n\tfor _, action := range ctx.restoreItemActions {\n\t\tif action.ShouldUse(groupResource, namespace, nil, ctx.log) {\n\t\t\tactions = append(actions, action)\n\t\t}\n\t}\n\treturn actions\n}\n\nfunc (ctx *restoreContext) shouldRestore(name string, pvClient client.Dynamic) (bool, error) {\n\tpvLogger := ctx.log.WithField(\"pvName\", name)\n\n\tvar shouldRestore bool\n\terr := wait.PollUntilContextTimeout(go_context.Background(), time.Second, ctx.resourceTerminatingTimeout, true, func(go_context.Context) (bool, error) {\n\t\tunstructuredPV, err := pvClient.Get(name, metav1.GetOptions{})\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tpvLogger.Debug(\"PV not found, safe to restore\")\n\t\t\t// PV not found, can safely exit loop and proceed with restore.\n\t\t\tshouldRestore = true\n\t\t\treturn true, nil\n\t\t}\n\t\tif err != nil {\n\t\t\treturn false, errors.Wrapf(err, \"could not retrieve in-cluster copy of PV %s\", name)\n\t\t}\n\n\t\tclusterPV := new(corev1api.PersistentVolume)\n\t\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(unstructuredPV.Object, clusterPV); err != nil {\n\t\t\treturn false, errors.Wrap(err, \"error converting PV from unstructured\")\n\t\t}\n\n\t\tif clusterPV.Status.Phase == corev1api.VolumeReleased || clusterPV.DeletionTimestamp != nil {\n\t\t\t// PV was found and marked for deletion, or it was released; wait for it to go away.\n\t\t\tpvLogger.Debugf(\"PV found, but marked for deletion, waiting\")\n\t\t\treturn false, nil\n\t\t}\n\n\t\t// Check for the namespace and PVC to see if anything that's referencing the PV is deleting.\n\t\t// If either the namespace or PVC is in a deleting/terminating state, wait for them to finish before\n\t\t// trying to restore the PV\n\t\t// Not doing so may result in the underlying PV disappearing but not restoring due to timing issues,\n\t\t// then the PVC getting restored and showing as lost.\n\t\tif clusterPV.Spec.ClaimRef == nil {\n\t\t\tpvLogger.Debugf(\"PV is not marked for deletion and is not claimed by a PVC\")\n\t\t\treturn true, nil\n\t\t}\n\n\t\tnamespace := clusterPV.Spec.ClaimRef.Namespace\n\t\tpvcName := clusterPV.Spec.ClaimRef.Name\n\n\t\t// Have to create the PVC client here because we don't know what namespace we're using til we get to this point.\n\t\t// Using a dynamic client since it's easier to mock for testing\n\t\tpvcResource := metav1.APIResource{Name: \"persistentvolumeclaims\", Namespaced: true}\n\t\tpvcClient, err := ctx.dynamicFactory.ClientForGroupVersionResource(schema.GroupVersion{Group: \"\", Version: \"v1\"}, pvcResource, namespace)\n\t\tif err != nil {\n\t\t\treturn false, errors.Wrapf(err, \"error getting pvc client\")\n\t\t}\n\n\t\tpvc, err := pvcClient.Get(pvcName, metav1.GetOptions{})\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tpvLogger.Debugf(\"PVC %s for PV not found, waiting\", pvcName)\n\t\t\t// PVC wasn't found, but the PV still exists, so continue to wait.\n\t\t\treturn false, nil\n\t\t}\n\t\tif err != nil {\n\t\t\treturn false, errors.Wrapf(err, \"error getting claim %s for persistent volume\", pvcName)\n\t\t}\n\n\t\tif pvc != nil && pvc.GetDeletionTimestamp() != nil {\n\t\t\tpvLogger.Debugf(\"PVC for PV marked for deletion, waiting\")\n\t\t\t// PVC is still deleting, continue to wait.\n\t\t\treturn false, nil\n\t\t}\n\n\t\t// Check the namespace associated with the claimRef to see if it's\n\t\t// deleting/terminating before proceeding.\n\t\tns, err := ctx.namespaceClient.Get(go_context.TODO(), namespace, metav1.GetOptions{})\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tpvLogger.Debugf(\"namespace %s for PV not found, waiting\", namespace)\n\t\t\t// Namespace not found but the PV still exists, so continue to wait.\n\t\t\treturn false, nil\n\t\t}\n\t\tif err != nil {\n\t\t\treturn false, errors.Wrapf(err, \"error getting namespace %s associated with PV %s\", namespace, name)\n\t\t}\n\n\t\tif ns != nil && (ns.GetDeletionTimestamp() != nil || ns.Status.Phase == corev1api.NamespaceTerminating) {\n\t\t\tpvLogger.Debugf(\"namespace %s associated with PV is deleting, waiting\", namespace)\n\t\t\t// Namespace is in the process of deleting, keep looping.\n\t\t\treturn false, nil\n\t\t}\n\n\t\t// None of the PV, PVC, or NS are marked for deletion, break the loop.\n\t\tpvLogger.Debug(\"PV, associated PVC and namespace are not marked for deletion\")\n\t\treturn true, nil\n\t})\n\n\tif wait.Interrupted(err) {\n\t\tpvLogger.Warn(\"timeout reached waiting for persistent volume to delete\")\n\t}\n\n\treturn shouldRestore, err\n}\n\n// crdAvailable waits for a CRD to be available for use before letting the\n// restore continue.\nfunc (ctx *restoreContext) crdAvailable(name string, crdClient client.Dynamic) (bool, error) {\n\tcrdLogger := ctx.log.WithField(\"crdName\", name)\n\n\tvar available bool\n\n\terr := wait.PollUntilContextTimeout(go_context.Background(), time.Second, ctx.resourceTimeout, true, func(ctx go_context.Context) (bool, error) {\n\t\tunstructuredCRD, err := crdClient.Get(name, metav1.GetOptions{})\n\t\tif err != nil {\n\t\t\treturn true, err\n\t\t}\n\t\tavailable, err = kube.IsCRDReady(unstructuredCRD)\n\t\tif err != nil {\n\t\t\treturn true, err\n\t\t}\n\n\t\tif !available {\n\t\t\tcrdLogger.Debug(\"CRD not yet ready for use\")\n\t\t}\n\n\t\t// If the CRD is not available, keep polling (false, nil).\n\t\t// If the CRD is available, break the poll and return to caller (true, nil).\n\t\treturn available, nil\n\t})\n\n\tif wait.Interrupted(err) {\n\t\tcrdLogger.Debug(\"timeout reached waiting for custom resource definition to be ready\")\n\t}\n\n\treturn available, err\n}\n\n// itemsAvailable waits for the passed-in additional items to be available for use before letting the restore continue.\nfunc (ctx *restoreContext) itemsAvailable(action framework.RestoreItemResolvedActionV2, restoreItemOut *velero.RestoreItemActionExecuteOutput) (bool, error) {\n\t// if RestoreItemAction doesn't define set WaitForAdditionalItems, then return true\n\tif !restoreItemOut.WaitForAdditionalItems {\n\t\treturn true, nil\n\t}\n\tvar available bool\n\ttimeout := ctx.resourceTimeout\n\tif restoreItemOut.AdditionalItemsReadyTimeout != 0 {\n\t\ttimeout = restoreItemOut.AdditionalItemsReadyTimeout\n\t}\n\n\terr := wait.PollUntilContextTimeout(go_context.Background(), time.Second, timeout, true, func(go_context.Context) (bool, error) {\n\t\tvar err error\n\t\tavailable, err = action.AreAdditionalItemsReady(restoreItemOut.AdditionalItems, ctx.restore)\n\n\t\tif err != nil {\n\t\t\treturn true, err\n\t\t}\n\n\t\tif !available {\n\t\t\tctx.log.Debug(\"AdditionalItems not yet ready for use\")\n\t\t}\n\n\t\t// If the AdditionalItems are not available, keep polling (false, nil)\n\t\t// If the AdditionalItems are available, break the poll and return back to caller (true, nil)\n\t\treturn available, nil\n\t})\n\n\tif wait.Interrupted(err) {\n\t\tctx.log.Debug(\"timeout reached waiting for AdditionalItems to be ready\")\n\t}\n\n\treturn available, err\n}\n\nfunc getResourceClientKey(groupResource schema.GroupResource, version, namespace string) resourceClientKey {\n\treturn resourceClientKey{\n\t\tresource: groupResource.WithVersion(version),\n\t\tnamespace: namespace,\n\t}\n}\nfunc (ctx *restoreContext) getResourceClient(groupResource schema.GroupResource, obj *unstructured.Unstructured, namespace string) (client.Dynamic, error) {\n\tkey := getResourceClientKey(groupResource, obj.GroupVersionKind().Version, namespace)\n\n\tif client, ok := ctx.resourceClients[key]; ok {\n\t\treturn client, nil\n\t}\n\n\t// Initialize client for this resource. We need metadata from an object to\n\t// do this.\n\tctx.log.Infof(\"Getting client for %v\", obj.GroupVersionKind())\n\n\tresource := metav1.APIResource{\n\t\tNamespaced: len(namespace) > 0,\n\t\tName: groupResource.Resource,\n\t}\n\n\tclient, err := ctx.dynamicFactory.ClientForGroupVersionResource(obj.GroupVersionKind().GroupVersion(), resource, namespace)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tctx.resourceClients[key] = client\n\treturn client, nil\n}\n\nfunc (ctx *restoreContext) getResourceLister(groupResource schema.GroupResource, obj *unstructured.Unstructured, namespace string) (cache.GenericNamespaceLister, error) {\n\t_, _, err := ctx.discoveryHelper.KindFor(obj.GroupVersionKind())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tinformer := ctx.dynamicInformerFactory.factory.ForResource(groupResource.WithVersion(obj.GroupVersionKind().Version))\n\t// if the restore contains CRDs or the RIA returns new resources, need to make sure the corresponding informers are synced\n\tif !informer.Informer().HasSynced() {\n\t\tctx.dynamicInformerFactory.factory.Start(ctx.dynamicInformerFactory.context.Done())\n\t\tctx.log.Infof(\"waiting informer cache sync for %s, %s/%s ...\", groupResource, namespace, obj.GetName())\n\t\tctx.dynamicInformerFactory.factory.WaitForCacheSync(ctx.dynamicInformerFactory.context.Done())\n\t}\n\n\tif namespace == \"\" {\n\t\treturn informer.Lister(), nil\n\t}\n\treturn informer.Lister().ByNamespace(namespace), nil\n}\n\nfunc getResourceID(groupResource schema.GroupResource, namespace, name string) string {\n\tif namespace == \"\" {\n\t\treturn fmt.Sprintf(\"%s/%s\", groupResource.String(), name)\n\t}\n\n\treturn fmt.Sprintf(\"%s/%s/%s\", groupResource.String(), namespace, name)\n}\n\nfunc (ctx *restoreContext) getResource(groupResource schema.GroupResource, obj *unstructured.Unstructured, namespace string) (*unstructured.Unstructured, error) {\n\tlister, err := ctx.getResourceLister(groupResource, obj, namespace)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"Error getting lister for %s\", getResourceID(groupResource, namespace, obj.GetName()))\n\t}\n\tclusterObj, err := lister.Get(obj.GetName())\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error getting resource from lister for %s, %s/%s\", groupResource, namespace, obj.GetName())\n\t}\n\tu, ok := clusterObj.(*unstructured.Unstructured)\n\tif !ok {\n\t\tctx.log.WithError(errors.WithStack(fmt.Errorf(\"expected *unstructured.Unstructured but got %T\", u))).Error(\"unable to understand entry returned from client\")\n\t\treturn nil, fmt.Errorf(\"expected *unstructured.Unstructured but got %T\", u)\n\t}\n\tctx.log.Debugf(\"get %s, %s/%s from informer cache\", groupResource, namespace, obj.GetName())\n\treturn u, nil\n}\n\nfunc (ctx *restoreContext) restoreItem(obj *unstructured.Unstructured, groupResource schema.GroupResource, namespace string) (results.Result, results.Result, bool) {\n\twarnings, errs := results.Result{}, results.Result{}\n\t// itemExists bool is used to determine whether to include this item in the \"wait for additional items\" list\n\titemExists := false\n\tresourceID := getResourceID(groupResource, namespace, obj.GetName())\n\tresourceKind := obj.GetKind()\n\tbackupResourceName := obj.GetName()\n\n\trestoreLogger := ctx.log.WithFields(logrus.Fields{\n\t\t\"namespace\": obj.GetNamespace(),\n\t\t\"original name\": backupResourceName,\n\t\t\"groupResource\": groupResource.String(),\n\t})\n\n\t// Check if group/resource should be restored. We need to do this here since\n\t// this method may be getting called for an additional item which is a group/resource\n\t// that's excluded.\n\tif !ctx.resourceIncludesExcludes.ShouldInclude(groupResource.String()) && !ctx.resourceMustHave.Has(groupResource.String()) {\n\t\trestoreLogger.Info(\"Not restoring item because resource is excluded\")\n\t\treturn warnings, errs, itemExists\n\t}\n\n\t// Check if namespace/cluster-scoped resource should be restored. We need\n\t// to do this here since this method may be getting called for an additional\n\t// item which is in a namespace that's excluded, or which is cluster-scoped\n\t// and should be excluded. Note that we're checking the object's namespace (\n\t// via obj.GetNamespace()) instead of the namespace parameter, because we want\n\t// to check the *original* namespace, not the remapped one if it's been remapped.\n\tif namespace != \"\" {\n\t\tif !ctx.namespaceIncludesExcludes.ShouldInclude(obj.GetNamespace()) && !ctx.resourceMustHave.Has(groupResource.String()) {\n\t\t\trestoreLogger.Info(\"Not restoring item because namespace is excluded\")\n\t\t\treturn warnings, errs, itemExists\n\t\t}\n\n\t\t// If the namespace scoped resource should be restored, ensure that the\n\t\t// namespace into which the resource is being restored into exists.\n\t\t// This is the *remapped* namespace that we are ensuring exists.\n\t\tnsToEnsure := getNamespace(restoreLogger, archive.GetItemFilePath(ctx.restoreDir, \"namespaces\", \"\", obj.GetNamespace()), namespace)\n\t\t_, nsCreated, err := kube.EnsureNamespaceExistsAndIsReady(nsToEnsure, ctx.namespaceClient, ctx.resourceTerminatingTimeout, ctx.resourceDeletionStatusTracker)\n\t\tif err != nil {\n\t\t\terrs.AddVeleroError(err)\n\t\t\treturn warnings, errs, itemExists\n\t\t}\n\t\t// Add the newly created namespace to the list of restored items.\n\t\tif nsCreated {\n\t\t\titemKey := itemKey{\n\t\t\t\tresource: resourceKey(nsToEnsure),\n\t\t\t\tnamespace: nsToEnsure.Namespace,\n\t\t\t\tname: nsToEnsure.Name,\n\t\t\t}\n\t\t\tctx.restoredItems[itemKey] = restoredItemStatus{action: ItemRestoreResultCreated, itemExists: true, createdName: nsToEnsure.Name}\n\t\t}\n\t} else {\n\t\tif boolptr.IsSetToFalse(ctx.restore.Spec.IncludeClusterResources) {\n\t\t\trestoreLogger.Info(\"Not restoring item because it's cluster-scoped\")\n\t\t\treturn warnings, errs, itemExists\n\t\t}\n\t}\n\n\t// Make a copy of object retrieved from backup to make it available unchanged\n\t//inside restore actions.\n\titemFromBackup := obj.DeepCopy()\n\n\tcomplete, err := isCompleted(obj, groupResource)\n\tif err != nil {\n\t\terrs.Add(namespace, fmt.Errorf(\"error checking completion of %q: %v\", resourceID, err))\n\t\treturn warnings, errs, itemExists\n\t}\n\tif complete {\n\t\trestoreLogger.Infof(\"%s is complete - skipping\", kube.NamespaceAndName(obj))\n\t\treturn warnings, errs, itemExists\n\t}\n\n\t// Check if we've already restored this itemKey.\n\titemKey := itemKey{\n\t\tresource: resourceKey(obj),\n\t\tnamespace: namespace,\n\t\tname: backupResourceName,\n\t}\n\tif prevRestoredItemStatus, exists := ctx.restoredItems[itemKey]; exists {\n\t\trestoreLogger.Infof(\"Skipping %s because it's already been restored.\", resourceID)\n\t\titemExists = prevRestoredItemStatus.itemExists\n\t\treturn warnings, errs, itemExists\n\t}\n\tctx.restoredItems[itemKey] = restoredItemStatus{itemExists: itemExists}\n\tdefer func() {\n\t\titemStatus := ctx.restoredItems[itemKey]\n\t\t// the action field is set explicitly\n\t\tif len(itemStatus.action) > 0 {\n\t\t\treturn\n\t\t}\n\t\t// no action specified, and no warnings and errors\n\t\tif errs.IsEmpty() && warnings.IsEmpty() {\n\t\t\titemStatus.action = ItemRestoreResultSkipped\n\t\t\tctx.restoredItems[itemKey] = itemStatus\n\t\t\treturn\n\t\t}\n\t\t// others are all failed\n\t\titemStatus.action = ItemRestoreResultFailed\n\t\tctx.restoredItems[itemKey] = itemStatus\n\t}()\n\n\t// TODO: move to restore item action if/when we add a ShouldRestore() method\n\t// to the interface.\n\tif groupResource == kuberesource.Pods && obj.GetAnnotations()[corev1api.MirrorPodAnnotationKey] != \"\" {\n\t\trestoreLogger.Infof(\"Not restoring pod because it's a mirror pod\")\n\t\treturn warnings, errs, itemExists\n\t}\n\n\tif groupResource == kuberesource.PersistentVolumes {\n\t\tresourceClient, err := ctx.getResourceClient(groupResource, obj, namespace)\n\t\tif err != nil {\n\t\t\terrs.AddVeleroError(fmt.Errorf(\"error getting resource client for namespace %q, resource %q: %v\", namespace, &groupResource, err))\n\t\t\treturn warnings, errs, itemExists\n\t\t}\n\n\t\tif volumeInfo, ok := ctx.backupVolumeInfoMap[obj.GetName()]; ok {\n\t\t\trestoreLogger.Infof(\"Find BackupVolumeInfo for PV %s.\", obj.GetName())\n\n\t\t\tswitch volumeInfo.BackupMethod {\n\t\t\tcase volume.NativeSnapshot:\n\t\t\t\tobj, err = ctx.handlePVHasNativeSnapshot(obj, resourceClient)\n\t\t\t\tif err != nil {\n\t\t\t\t\terrs.Add(namespace, err)\n\t\t\t\t\treturn warnings, errs, itemExists\n\t\t\t\t}\n\n\t\t\tcase volume.PodVolumeBackup:\n\t\t\t\trestoreLogger.Infof(\"Dynamically re-provisioning persistent volume because it has a pod volume backup to be restored.\")\n\t\t\t\tctx.pvsToProvision.Insert(backupResourceName)\n\n\t\t\t\t// Return early because we don't want to restore the PV itself, we\n\t\t\t\t// want to dynamically re-provision it.\n\t\t\t\treturn warnings, errs, itemExists\n\n\t\t\tcase volume.CSISnapshot:\n\t\t\t\trestoreLogger.Infof(\"Dynamically re-provisioning persistent volume because it has a CSI VolumeSnapshot or a related snapshot DataUpload.\")\n\t\t\t\tctx.pvsToProvision.Insert(backupResourceName)\n\n\t\t\t\t// Return early because we don't want to restore the PV itself, we\n\t\t\t\t// want to dynamically re-provision it.\n\t\t\t\treturn warnings, errs, itemExists\n\n\t\t\t// When the PV data is skipped from backup, it's BackupVolumeInfo BackupMethod\n\t\t\t// is not set, and it will fall into the default case.\n\t\t\tdefault:\n\t\t\t\tif hasDeleteReclaimPolicy(obj.Object) {\n\t\t\t\t\trestoreLogger.Infof(\"Dynamically re-provisioning persistent volume because it doesn't have a snapshot and its reclaim policy is Delete.\")\n\t\t\t\t\tctx.pvsToProvision.Insert(backupResourceName)\n\n\t\t\t\t\t// Return early because we don't want to restore the PV itself, we\n\t\t\t\t\t// want to dynamically re-provision it.\n\t\t\t\t\treturn warnings, errs, itemExists\n\t\t\t\t} else {\n\t\t\t\t\tobj, err = ctx.handleSkippedPVHasRetainPolicy(obj, restoreLogger)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\terrs.Add(namespace, err)\n\t\t\t\t\t\treturn warnings, errs, itemExists\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t} else {\n\t\t\t// TODO: BackupVolumeInfo is adopted and old logic is deprecated in v1.13.\n\t\t\t// Remove the old logic in v1.15.\n\t\t\trestoreLogger.Infof(\"Cannot find BackupVolumeInfo for PV %s.\", obj.GetName())\n\n\t\t\tswitch {\n\t\t\tcase hasSnapshot(backupResourceName, ctx.volumeSnapshots):\n\t\t\t\tobj, err = ctx.handlePVHasNativeSnapshot(obj, resourceClient)\n\t\t\t\tif err != nil {\n\t\t\t\t\terrs.Add(namespace, err)\n\t\t\t\t\treturn warnings, errs, itemExists\n\t\t\t\t}\n\n\t\t\tcase hasPodVolumeBackup(obj, ctx):\n\t\t\t\trestoreLogger.Infof(\"Dynamically re-provisioning persistent volume because it has a pod volume backup to be restored.\")\n\t\t\t\tctx.pvsToProvision.Insert(backupResourceName)\n\n\t\t\t\t// Return early because we don't want to restore the PV itself, we\n\t\t\t\t// want to dynamically re-provision it.\n\t\t\t\treturn warnings, errs, itemExists\n\n\t\t\tcase hasCSIVolumeSnapshot(ctx, obj):\n\t\t\t\tfallthrough\n\t\t\tcase hasSnapshotDataUpload(ctx, obj):\n\t\t\t\trestoreLogger.Infof(\"Dynamically re-provisioning persistent volume because it has a CSI VolumeSnapshot or a related snapshot DataUpload.\")\n\t\t\t\tctx.pvsToProvision.Insert(backupResourceName)\n\n\t\t\t\t// Return early because we don't want to restore the PV itself, we\n\t\t\t\t// want to dynamically re-provision it.\n\t\t\t\treturn warnings, errs, itemExists\n\n\t\t\tcase hasDeleteReclaimPolicy(obj.Object):\n\t\t\t\trestoreLogger.Infof(\"Dynamically re-provisioning persistent volume because it doesn't have a snapshot and its reclaim policy is Delete.\")\n\t\t\t\tctx.pvsToProvision.Insert(backupResourceName)\n\n\t\t\t\t// Return early because we don't want to restore the PV itself, we\n\t\t\t\t// want to dynamically re-provision it.\n\t\t\t\treturn warnings, errs, itemExists\n\n\t\t\tdefault:\n\t\t\t\tobj, err = ctx.handleSkippedPVHasRetainPolicy(obj, restoreLogger)\n\t\t\t\tif err != nil {\n\t\t\t\t\terrs.Add(namespace, err)\n\t\t\t\t\treturn warnings, errs, itemExists\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\tobjStatus, statusFieldExists, statusFieldErr := unstructured.NestedFieldCopy(obj.Object, \"status\")\n\t// Clear out non-core metadata fields and status.\n\tif obj, err = resetMetadataAndStatus(obj); err != nil {\n\t\terrs.Add(namespace, err)\n\t\treturn warnings, errs, itemExists\n\t}\n\n\trestoreLogger.Infof(\"restore status includes excludes: %+v\", ctx.resourceStatusIncludesExcludes)\n\n\tfor _, action := range ctx.getApplicableActions(groupResource, namespace) {\n\t\tif !action.Selector.Matches(labels.Set(obj.GetLabels())) {\n\t\t\tcontinue\n\t\t}\n\n\t\t// If the EnableCSI feature is not enabled, but the executing action is from CSI plugin, skip the action.\n\t\tif csiutil.ShouldSkipAction(action.Name()) {\n\t\t\trestoreLogger.Infof(\"Skip action %s for resource %s:%s/%s, because the CSI feature is not enabled. Feature setting is %s.\",\n\t\t\t\taction.Name(), groupResource.String(), obj.GetNamespace(), obj.GetName(), features.Serialize())\n\t\t\tcontinue\n\t\t}\n\n\t\trestoreLogger.Infof(\"Executing item action for %v\", &groupResource)\n\t\texecuteOutput, err := action.RestoreItemAction.Execute(&velero.RestoreItemActionExecuteInput{\n\t\t\tItem: obj,\n\t\t\tItemFromBackup: itemFromBackup,\n\t\t\tRestore: ctx.restore,\n\t\t})\n\t\tif err != nil {\n\t\t\terrs.Add(namespace, fmt.Errorf(\"error preparing %s: %v\", resourceID, err))\n\t\t\treturn warnings, errs, itemExists\n\t\t}\n\n\t\t// If async plugin started async operation, add it to the ItemOperations list\n\t\tif executeOutput.OperationID != \"\" {\n\t\t\tresourceIdentifier := velero.ResourceIdentifier{\n\t\t\t\tGroupResource: groupResource,\n\t\t\t\tNamespace: namespace,\n\t\t\t\tName: backupResourceName,\n\t\t\t}\n\t\t\tnow := metav1.Now()\n\t\t\tnewOperation := itemoperation.RestoreOperation{\n\t\t\t\tSpec: itemoperation.RestoreOperationSpec{\n\t\t\t\t\tRestoreName: ctx.restore.Name,\n\t\t\t\t\tRestoreUID: string(ctx.restore.UID),\n\t\t\t\t\tRestoreItemAction: action.RestoreItemAction.Name(),\n\t\t\t\t\tResourceIdentifier: resourceIdentifier,\n\t\t\t\t\tOperationID: executeOutput.OperationID,\n\t\t\t\t},\n\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\tPhase: itemoperation.OperationPhaseNew,\n\t\t\t\t\tCreated: &now,\n\t\t\t\t},\n\t\t\t}\n\t\t\titemOperList := ctx.itemOperationsList\n\t\t\t*itemOperList = append(*itemOperList, &newOperation)\n\t\t}\n\t\tif executeOutput.SkipRestore {\n\t\t\trestoreLogger.Infof(\"Skipping restore because a registered plugin discarded it\")\n\t\t\treturn warnings, errs, itemExists\n\t\t}\n\t\tunstructuredObj, ok := executeOutput.UpdatedItem.(*unstructured.Unstructured)\n\t\tif !ok {\n\t\t\terrs.Add(namespace, fmt.Errorf(\"%s: unexpected type %T\", resourceID, executeOutput.UpdatedItem))\n\t\t\treturn warnings, errs, itemExists\n\t\t}\n\n\t\tobj = unstructuredObj\n\n\t\tvar filteredAdditionalItems []velero.ResourceIdentifier\n\t\tfor _, additionalItem := range executeOutput.AdditionalItems {\n\t\t\titemPath := archive.GetItemFilePath(ctx.restoreDir, additionalItem.GroupResource.String(), additionalItem.Namespace, additionalItem.Name)\n\n\t\t\tif _, err := ctx.fileSystem.Stat(itemPath); err != nil {\n\t\t\t\trestoreLogger.WithError(err).WithFields(logrus.Fields{\n\t\t\t\t\t\"additionalResource\": additionalItem.GroupResource.String(),\n\t\t\t\t\t\"additionalResourceNamespace\": additionalItem.Namespace,\n\t\t\t\t\t\"additionalResourceName\": additionalItem.Name,\n\t\t\t\t}).Warn(\"unable to restore additional item\")\n\t\t\t\twarnings.Add(additionalItem.Namespace, err)\n\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tadditionalResourceID := getResourceID(additionalItem.GroupResource, additionalItem.Namespace, additionalItem.Name)\n\t\t\tadditionalObj, err := archive.Unmarshal(ctx.fileSystem, itemPath)\n\t\t\tif err != nil {\n\t\t\t\terrs.Add(namespace, errors.Wrapf(err, \"error restoring additional item %s\", additionalResourceID))\n\t\t\t}\n\n\t\t\tadditionalItemNamespace := additionalItem.Namespace\n\t\t\tif additionalItemNamespace != \"\" {\n\t\t\t\tif remapped, ok := ctx.restore.Spec.NamespaceMapping[additionalItemNamespace]; ok {\n\t\t\t\t\tadditionalItemNamespace = remapped\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tw, e, additionalItemExists := ctx.restoreItem(additionalObj, additionalItem.GroupResource, additionalItemNamespace)\n\t\t\tif additionalItemExists {\n\t\t\t\tfilteredAdditionalItems = append(filteredAdditionalItems, additionalItem)\n\t\t\t}\n\n\t\t\twarnings.Merge(&w)\n\t\t\terrs.Merge(&e)\n\t\t}\n\t\texecuteOutput.AdditionalItems = filteredAdditionalItems\n\t\tavailable, err := ctx.itemsAvailable(action, executeOutput)\n\t\tif err != nil {\n\t\t\terrs.Add(namespace, errors.Wrapf(err, \"error verifying additional items are ready to use\"))\n\t\t} else if !available {\n\t\t\terrs.Add(namespace, fmt.Errorf(\"additional items for %s are not ready to use\", resourceID))\n\t\t}\n\t}\n\n\t// This comes after running item actions because we have built-in actions that restore\n\t// a PVC's associated PV (if applicable). As part of the PV being restored, the 'pvsToProvision'\n\t// set may be inserted into, and this needs to happen *before* running the following block of logic.\n\t//\n\t// The side effect of this is that it's impossible for a user to write a restore item action that\n\t// adjusts this behavior (i.e. of resetting the PVC for dynamic provisioning if it claims a PV with\n\t// a reclaim policy of Delete and no snapshot). If/when that becomes an issue for users, we can\n\t// revisit. This would be easier with a multi-pass restore process.\n\tif groupResource == kuberesource.PersistentVolumeClaims {\n\t\tpvc := new(corev1api.PersistentVolumeClaim)\n\t\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(obj.UnstructuredContent(), pvc); err != nil {\n\t\t\terrs.Add(namespace, err)\n\t\t\treturn warnings, errs, itemExists\n\t\t}\n\n\t\tif pvc.Spec.VolumeName != \"\" {\n\t\t\t// This used to only happen with PVB volumes, but now always remove this binding metadata\n\t\t\tobj = resetVolumeBindingInfo(obj)\n\n\t\t\t// This is the case for PVB volumes, where we need to actually have an empty volume created instead of restoring one.\n\t\t\t// The assumption is that any PV in pvsToProvision doesn't have an associated snapshot.\n\t\t\tif ctx.pvsToProvision.Has(pvc.Spec.VolumeName) {\n\t\t\t\trestoreLogger.Infof(\"Resetting PersistentVolumeClaim for dynamic provisioning\")\n\t\t\t\tunstructured.RemoveNestedField(obj.Object, \"spec\", \"volumeName\")\n\t\t\t}\n\t\t}\n\n\t\tif newName, ok := ctx.renamedPVs[pvc.Spec.VolumeName]; ok {\n\t\t\trestoreLogger.Infof(\"Updating persistent volume claim %s/%s to reference renamed persistent volume (%s -> %s)\", namespace, obj.GetName(), pvc.Spec.VolumeName, newName)\n\t\t\tif err := unstructured.SetNestedField(obj.Object, newName, \"spec\", \"volumeName\"); err != nil {\n\t\t\t\terrs.Add(namespace, err)\n\t\t\t\treturn warnings, errs, itemExists\n\t\t\t}\n\t\t}\n\t}\n\n\tif ctx.resourceModifiers != nil {\n\t\tif errList := ctx.resourceModifiers.ApplyResourceModifierRules(obj, groupResource.String(), ctx.kbClient.Scheme(), restoreLogger); errList != nil {\n\t\t\tfor _, err := range errList {\n\t\t\t\terrs.Add(namespace, err)\n\t\t\t}\n\t\t}\n\t}\n\n\t// Necessary because we may have remapped the namespace if the namespace is\n\t// blank, don't create the key.\n\toriginalNamespace := obj.GetNamespace()\n\tif namespace != \"\" {\n\t\tobj.SetNamespace(namespace)\n\t}\n\n\t// Label the resource with the restore's name and the restored backup's name\n\t// for easy identification of all cluster resources created by this restore\n\t// and which backup they came from.\n\taddRestoreLabels(obj, ctx.restore.Name, ctx.restore.Spec.BackupName)\n\n\t// The object apiVersion might get modified by a RestorePlugin so we need to\n\t// get a new client to reflect updated resource path.\n\tnewGR := schema.GroupResource{Group: obj.GroupVersionKind().Group, Resource: groupResource.Resource}\n\t// obj kind might change within a special RIA which is used to convert objects,\n\t// like from Openshift DeploymentConfig to native Deployment.\n\t// we should re-get the newGR.Resource again in such a case\n\tif obj.GetKind() != resourceKind {\n\t\trestoreLogger.Infof(\"Resource kind changed from %s to %s\", resourceKind, obj.GetKind())\n\t\tgvr, _, err := ctx.discoveryHelper.KindFor(obj.GroupVersionKind())\n\t\tif err != nil {\n\t\t\terrs.Add(namespace, fmt.Errorf(\"error getting GVR for %s: %v\", obj.GroupVersionKind(), err))\n\t\t\treturn warnings, errs, itemExists\n\t\t}\n\t\tnewGR.Resource = gvr.Resource\n\t}\n\tif !reflect.DeepEqual(newGR, groupResource) {\n\t\trestoreLogger.Infof(\"Resource to be restored changed from %v to %v\", groupResource, newGR)\n\t}\n\tresourceClient, err := ctx.getResourceClient(newGR, obj, obj.GetNamespace())\n\tif err != nil {\n\t\twarnings.Add(namespace, fmt.Errorf(\"error getting updated resource client for namespace %q, resource %q: %v\", namespace, &newGR, err))\n\t\treturn warnings, errs, itemExists\n\t}\n\n\trestoreLogger.Infof(\"Attempting to restore %s: %s.\", obj.GroupVersionKind().Kind, obj.GetName())\n\n\t// check if we want to treat the error as a warning, in some cases the creation call might not get executed due to object API validations\n\t// and Velero might not get the already exists error type but in reality the object already exists\n\tvar fromCluster, createdObj *unstructured.Unstructured\n\tvar restoreErr error\n\n\t// only attempt Get before Create if using informer cache, otherwise this will slow down restore into\n\t// new namespace\n\tif !ctx.disableInformerCache {\n\t\trestoreLogger.Debugf(\"Checking for existence %s\", obj.GetName())\n\t\tfromCluster, err = ctx.getResource(newGR, obj, namespace)\n\t}\n\tif err != nil || fromCluster == nil {\n\t\t// couldn't find the resource, attempt to create\n\t\trestoreLogger.Debugf(\"Creating %s\", obj.GetName())\n\t\tcreatedObj, restoreErr = resourceClient.Create(obj)\n\t\tif restoreErr == nil {\n\t\t\titemExists = true\n\t\t\tctx.restoredItems[itemKey] = restoredItemStatus{\n\t\t\t\taction: ItemRestoreResultCreated,\n\t\t\t\titemExists: itemExists,\n\t\t\t\tcreatedName: createdObj.GetName(),\n\t\t\t}\n\t\t}\n\t}\n\n\tisAlreadyExistsError, err := isAlreadyExistsError(ctx, obj, restoreErr, resourceClient)\n\tif err != nil {\n\t\terrs.Add(namespace, err)\n\t\treturn warnings, errs, itemExists\n\t}\n\n\tif restoreErr != nil {\n\t\t// check for the existence of the object that failed creation due to alreadyExist in cluster, if no error then it implies that object exists.\n\t\t// and if err then itemExists remains false as we were not able to confirm the existence of the object via Get call or creation call.\n\t\t// We return the get error as a warning to notify the user that the object could exist in cluster and we were not able to confirm it.\n\t\tif !ctx.disableInformerCache {\n\t\t\tfromCluster, err = ctx.getResource(newGR, obj, namespace)\n\t\t} else {\n\t\t\tfromCluster, err = resourceClient.Get(obj.GetName(), metav1.GetOptions{})\n\t\t}\n\t\tif err != nil && isAlreadyExistsError {\n\t\t\trestoreLogger.Warnf(\"Unable to retrieve in-cluster version of %s: %s, object won't be restored by velero or have restore labels, and existing resource policy is not applied\", kube.NamespaceAndName(obj), err.Error())\n\t\t\twarnings.Add(namespace, err)\n\t\t\treturn warnings, errs, itemExists\n\t\t}\n\t}\n\n\tif fromCluster != nil {\n\t\titemExists = true\n\t\titemStatus := ctx.restoredItems[itemKey]\n\t\titemStatus.itemExists = itemExists\n\t\tctx.restoredItems[itemKey] = itemStatus\n\t\t// Remove insubstantial metadata.\n\t\tfromCluster, err = resetMetadataAndStatus(fromCluster)\n\t\tif err != nil {\n\t\t\trestoreLogger.Infof(\"Error trying to reset metadata for %s: %s\", kube.NamespaceAndName(obj), err.Error())\n\t\t\twarnings.Add(namespace, err)\n\t\t\treturn warnings, errs, itemExists\n\t\t}\n\n\t\t// We know the object from the cluster won't have the backup/restore name\n\t\t// labels, so copy them from the object we attempted to restore.\n\t\tlabels := obj.GetLabels()\n\t\taddRestoreLabels(fromCluster, labels[velerov1api.RestoreNameLabel], labels[velerov1api.BackupNameLabel])\n\t\tfromClusterWithLabels := fromCluster.DeepCopy() // saving the in-cluster object so that we can create label patch if overall patch fails\n\n\t\tif !equality.Semantic.DeepEqual(fromCluster, obj) {\n\t\t\tswitch newGR {\n\t\t\tcase kuberesource.ServiceAccounts:\n\t\t\t\tdesired, err := mergeServiceAccounts(fromCluster, obj)\n\t\t\t\tif err != nil {\n\t\t\t\t\trestoreLogger.Infof(\"error merging secrets for ServiceAccount %s: %s\", kube.NamespaceAndName(obj), err.Error())\n\t\t\t\t\twarnings.Add(namespace, err)\n\t\t\t\t\treturn warnings, errs, itemExists\n\t\t\t\t}\n\n\t\t\t\tpatchBytes, err := generatePatch(fromCluster, desired)\n\t\t\t\tif err != nil {\n\t\t\t\t\trestoreLogger.Infof(\"error generating patch for ServiceAccount %s: %s\", kube.NamespaceAndName(obj), err.Error())\n\t\t\t\t\twarnings.Add(namespace, err)\n\t\t\t\t\treturn warnings, errs, itemExists\n\t\t\t\t}\n\n\t\t\t\tif patchBytes == nil {\n\t\t\t\t\t// In-cluster and desired state are the same, so move on to\n\t\t\t\t\t// the next item.\n\t\t\t\t\treturn warnings, errs, itemExists\n\t\t\t\t}\n\n\t\t\t\t_, err = resourceClient.Patch(obj.GetName(), patchBytes)\n\t\t\t\tif err != nil {\n\t\t\t\t\twarnings.Add(namespace, err)\n\t\t\t\t\t// check if there is existingResourcePolicy and if it is set to update policy\n\t\t\t\t\tif len(ctx.restore.Spec.ExistingResourcePolicy) > 0 && ctx.restore.Spec.ExistingResourcePolicy == velerov1api.PolicyTypeUpdate {\n\t\t\t\t\t\t// remove restore labels so that we apply the latest backup/restore names on the object via patch\n\t\t\t\t\t\tremoveRestoreLabels(fromCluster)\n\t\t\t\t\t\t//try patching just the backup/restore labels\n\t\t\t\t\t\twarningsFromUpdate, errsFromUpdate := ctx.updateBackupRestoreLabels(fromCluster, fromClusterWithLabels, namespace, resourceClient)\n\t\t\t\t\t\twarnings.Merge(&warningsFromUpdate)\n\t\t\t\t\t\terrs.Merge(&errsFromUpdate)\n\t\t\t\t\t}\n\t\t\t\t} else {\n\t\t\t\t\titemStatus.action = ItemRestoreResultUpdated\n\t\t\t\t\tctx.restoredItems[itemKey] = itemStatus\n\t\t\t\t\trestoreLogger.Infof(\"ServiceAccount %s successfully updated\", kube.NamespaceAndName(obj))\n\t\t\t\t}\n\t\t\tdefault:\n\t\t\t\t// check for the presence of existingResourcePolicy\n\t\t\t\tif len(ctx.restore.Spec.ExistingResourcePolicy) > 0 {\n\t\t\t\t\tresourcePolicy := ctx.restore.Spec.ExistingResourcePolicy\n\t\t\t\t\trestoreLogger.Infof(\"restore API has resource policy defined %s, executing restore workflow accordingly for changed resource %s %s\", resourcePolicy, fromCluster.GroupVersionKind().Kind, kube.NamespaceAndName(fromCluster))\n\n\t\t\t\t\t// existingResourcePolicy is set as none, add warning\n\t\t\t\t\tif resourcePolicy == velerov1api.PolicyTypeNone {\n\t\t\t\t\t\te := errors.Errorf(\"could not restore, %s %q already exists. Warning: the in-cluster version is different than the backed-up version\",\n\t\t\t\t\t\t\tobj.GetKind(), obj.GetName())\n\t\t\t\t\t\twarnings.Add(namespace, e)\n\t\t\t\t\t\t// existingResourcePolicy is set as update, attempt patch on the resource and add warning if it fails\n\t\t\t\t\t} else if resourcePolicy == velerov1api.PolicyTypeUpdate {\n\t\t\t\t\t\t// processing update as existingResourcePolicy\n\t\t\t\t\t\twarningsFromUpdateRP, errsFromUpdateRP := ctx.processUpdateResourcePolicy(fromCluster, fromClusterWithLabels, obj, namespace, resourceClient)\n\t\t\t\t\t\tif warningsFromUpdateRP.IsEmpty() && errsFromUpdateRP.IsEmpty() {\n\t\t\t\t\t\t\titemStatus.action = ItemRestoreResultUpdated\n\t\t\t\t\t\t\tctx.restoredItems[itemKey] = itemStatus\n\t\t\t\t\t\t}\n\t\t\t\t\t\twarnings.Merge(&warningsFromUpdateRP)\n\t\t\t\t\t\terrs.Merge(&errsFromUpdateRP)\n\t\t\t\t\t}\n\t\t\t\t} else {\n\t\t\t\t\t// Preserved Velero behavior when existingResourcePolicy is not specified by the user\n\t\t\t\t\te := errors.Errorf(\"could not restore, %s:%s already exists. Warning: the in-cluster version is different than the backed-up version\",\n\t\t\t\t\t\tobj.GetKind(), obj.GetName())\n\t\t\t\t\twarnings.Add(namespace, e)\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn warnings, errs, itemExists\n\t\t}\n\n\t\t//update backup/restore labels on the unchanged resources if existingResourcePolicy is set as update\n\t\tif ctx.restore.Spec.ExistingResourcePolicy == velerov1api.PolicyTypeUpdate {\n\t\t\tresourcePolicy := ctx.restore.Spec.ExistingResourcePolicy\n\t\t\trestoreLogger.Infof(\"restore API has resource policy defined %s, executing restore workflow accordingly for unchanged resource %s %s \", resourcePolicy, obj.GroupVersionKind().Kind, kube.NamespaceAndName(fromCluster))\n\t\t\t// remove restore labels so that we apply the latest backup/restore names on the object via patch\n\t\t\tremoveRestoreLabels(fromCluster)\n\t\t\t// try updating the backup/restore labels for the in-cluster object\n\t\t\twarningsFromUpdate, errsFromUpdate := ctx.updateBackupRestoreLabels(fromCluster, obj, namespace, resourceClient)\n\t\t\twarnings.Merge(&warningsFromUpdate)\n\t\t\terrs.Merge(&errsFromUpdate)\n\t\t}\n\n\t\trestoreLogger.Infof(\"Restore of %s skipped: it already exists in the cluster and is the same as the backed up version\", obj.GetName())\n\t\treturn warnings, errs, itemExists\n\t}\n\n\t// Error was something other than an AlreadyExists.\n\tif restoreErr != nil {\n\t\trestoreLogger.Errorf(\"error restoring %s: %s\", obj.GetName(), restoreErr.Error())\n\t\terrs.Add(namespace, fmt.Errorf(\"error restoring %s: %v\", resourceID, restoreErr))\n\t\treturn warnings, errs, itemExists\n\t}\n\n\t// determine whether to restore status according to original GR\n\tshouldRestoreStatus := determineRestoreStatus(obj, ctx.resourceStatusIncludesExcludes, groupResource.String(), restoreLogger)\n\n\tif shouldRestoreStatus && statusFieldErr != nil {\n\t\terr := fmt.Errorf(\"could not get status to be restored %s: %v\", kube.NamespaceAndName(obj), statusFieldErr)\n\t\trestoreLogger.Error(err.Error())\n\t\terrs.Add(namespace, err)\n\t\treturn warnings, errs, itemExists\n\t}\n\n\t// Proceed with status restoration if decided\n\tif statusFieldExists && shouldRestoreStatus {\n\t\tif err := unstructured.SetNestedField(obj.Object, objStatus, \"status\"); err != nil {\n\t\t\trestoreLogger.Errorf(\"could not set status field %s: %s\", kube.NamespaceAndName(obj), err.Error())\n\t\t\terrs.Add(namespace, err)\n\t\t\treturn warnings, errs, itemExists\n\t\t}\n\n\t\tresourceVersion := createdObj.GetResourceVersion()\n\t\tif err := retry.RetryOnConflict(retry.DefaultRetry, func() error {\n\t\t\tobj.SetResourceVersion(resourceVersion)\n\t\t\tupdated, err := resourceClient.UpdateStatus(obj, metav1.UpdateOptions{})\n\t\t\tif err != nil {\n\t\t\t\tif apierrors.IsConflict(err) {\n\t\t\t\t\tres, err := resourceClient.Get(obj.GetName(), metav1.GetOptions{})\n\t\t\t\t\tif err == nil {\n\t\t\t\t\t\tresourceVersion = res.GetResourceVersion()\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\tcreatedObj = updated\n\t\t\treturn nil\n\t\t}); err != nil {\n\t\t\trestoreLogger.Infof(\"status field update failed %s: %s\", kube.NamespaceAndName(obj), err.Error())\n\t\t\twarnings.Add(namespace, err)\n\t\t}\n\t}\n\n\t// restore the managedFields\n\twithoutManagedFields := createdObj.DeepCopy()\n\tcreatedObj.SetManagedFields(obj.GetManagedFields())\n\tpatchBytes, err := generatePatch(withoutManagedFields, createdObj)\n\tif err != nil {\n\t\trestoreLogger.Errorf(\"error generating patch for managed fields %s: %s\", kube.NamespaceAndName(createdObj), err.Error())\n\t\terrs.Add(namespace, err)\n\t\treturn warnings, errs, itemExists\n\t}\n\tif patchBytes != nil {\n\t\tif _, err = resourceClient.Patch(createdObj.GetName(), patchBytes); err != nil {\n\t\t\tif !apierrors.IsNotFound(err) {\n\t\t\t\trestoreLogger.Errorf(\"error patch for managed fields %s: %s\", kube.NamespaceAndName(createdObj), err.Error())\n\t\t\t\terrs.Add(namespace, err)\n\t\t\t\treturn warnings, errs, itemExists\n\t\t\t}\n\t\t\trestoreLogger.Warnf(\"item not found when patching managed fields %s: %s\", kube.NamespaceAndName(createdObj), err.Error())\n\t\t\twarnings.Add(namespace, err)\n\t\t} else {\n\t\t\trestoreLogger.Infof(\"the managed fields for %s is patched\", kube.NamespaceAndName(createdObj))\n\t\t}\n\t}\n\n\tif newGR == kuberesource.Pods {\n\t\tpod := new(corev1api.Pod)\n\t\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(obj.UnstructuredContent(), pod); err != nil {\n\t\t\terrs.Add(namespace, err)\n\t\t\treturn warnings, errs, itemExists\n\t\t}\n\n\t\t// Do not create podvolumerestore when current restore excludes pv/pvc\n\t\tif ctx.resourceIncludesExcludes.ShouldInclude(kuberesource.PersistentVolumeClaims.String()) &&\n\t\t\tctx.resourceIncludesExcludes.ShouldInclude(kuberesource.PersistentVolumes.String()) &&\n\t\t\tlen(podvolume.GetVolumeBackupsForPod(ctx.podVolumeBackups, pod, originalNamespace)) > 0 {\n\t\t\trestorePodVolumeBackups(ctx, createdObj, originalNamespace)\n\t\t}\n\t}\n\n\t// Asynchronously executes restore exec hooks if any\n\t// Velero will wait for all the asynchronous hook operations to finish in finalizing phase, using hook tracker to track the execution progress.\n\tif newGR == kuberesource.Pods {\n\t\tpod := new(corev1api.Pod)\n\t\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(createdObj.UnstructuredContent(), &pod); err != nil {\n\t\t\trestoreLogger.Errorf(\"error converting pod %s: %s\", kube.NamespaceAndName(obj), err.Error())\n\t\t\terrs.Add(namespace, err)\n\t\t\treturn warnings, errs, itemExists\n\t\t}\n\n\t\texecHooksByContainer, err := ctx.hooksWaitExecutor.groupHooks(ctx.restore.Name, pod, ctx.multiHookTracker)\n\t\tif err != nil {\n\t\t\trestoreLogger.Errorf(\"error grouping hooks from pod %s: %s\", kube.NamespaceAndName(obj), err.Error())\n\t\t\terrs.Add(namespace, err)\n\t\t\treturn warnings, errs, itemExists\n\t\t}\n\n\t\tctx.hooksWaitExecutor.exec(execHooksByContainer, pod, ctx.multiHookTracker, ctx.restore.Name)\n\t}\n\n\t// Wait for a CRD to be available for instantiating resources\n\t// before continuing.\n\tif newGR == kuberesource.CustomResourceDefinitions {\n\t\tavailable, err := ctx.crdAvailable(obj.GetName(), resourceClient)\n\t\tif err != nil {\n\t\t\terrs.Add(namespace, errors.Wrapf(err, \"error verifying the CRD %s is ready to use\", obj.GetName()))\n\t\t} else if !available {\n\t\t\terrs.Add(namespace, fmt.Errorf(\"the CRD %s is not available to use for custom resources\", obj.GetName()))\n\t\t}\n\t}\n\n\treturn warnings, errs, itemExists\n}\n\nfunc isAlreadyExistsError(ctx *restoreContext, obj *unstructured.Unstructured, err error, client client.Dynamic) (bool, error) {\n\tif err == nil {\n\t\treturn false, nil\n\t}\n\tif apierrors.IsAlreadyExists(err) {\n\t\treturn true, nil\n\t}\n\t// The \"invalid value error\" or \"internal error\" rather than \"already exists\" error returns when restoring nodePort service in the following two cases:\n\t// 1. For NodePort service, the service has nodePort preservation while the same nodePort service already exists. - Get invalid value error\n\t// 2. For LoadBalancer service, the \"healthCheckNodePort\" already exists. - Get internal error\n\t// If this is the case, the function returns true to avoid reporting error.\n\t// Refer to https://github.com/vmware-tanzu/velero/issues/2308 for more details\n\tif obj.GetKind() != \"Service\" {\n\t\treturn false, nil\n\t}\n\tstatusErr, ok := err.(*apierrors.StatusError)\n\tif !ok || statusErr.Status().Details == nil || len(statusErr.Status().Details.Causes) == 0 {\n\t\treturn false, nil\n\t}\n\t// make sure all the causes are \"port allocated\" error\n\tfor _, cause := range statusErr.Status().Details.Causes {\n\t\tif !strings.Contains(cause.Message, \"provided port is already allocated\") {\n\t\t\treturn false, nil\n\t\t}\n\t}\n\n\t// the \"already allocated\" error may be caused by other services, check whether the expected service exists or not\n\tif _, err = client.Get(obj.GetName(), metav1.GetOptions{}); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tctx.log.Debugf(\"Service %s not found\", kube.NamespaceAndName(obj))\n\t\t\treturn false, nil\n\t\t}\n\t\treturn false, errors.Wrapf(err, \"Unable to get the service %s while checking the NodePort is already allocated error\", kube.NamespaceAndName(obj))\n\t}\n\n\tctx.log.Infof(\"Service %s exists, ignore the provided port is already allocated error\", kube.NamespaceAndName(obj))\n\treturn true, nil\n}\n\n// shouldRenamePV returns a boolean indicating whether a persistent volume should\n// be given a new name before being restored, or an error if this cannot be determined.\n// A persistent volume will be given a new name if and only if (a) a PV with the\n// original name already exists in-cluster, and (b) in the backup, the PV is claimed\n// by a PVC in a namespace that's being remapped during the restore.\nfunc shouldRenamePV(ctx *restoreContext, obj *unstructured.Unstructured, client client.Dynamic) (bool, error) {\n\tif len(ctx.restore.Spec.NamespaceMapping) == 0 {\n\t\tctx.log.Debugf(\"Persistent volume does not need to be renamed because restore is not remapping any namespaces\")\n\t\treturn false, nil\n\t}\n\n\tpv := new(corev1api.PersistentVolume)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(obj.Object, pv); err != nil {\n\t\treturn false, errors.Wrapf(err, \"error converting persistent volume to structured\")\n\t}\n\n\tif pv.Spec.ClaimRef == nil {\n\t\tctx.log.Debugf(\"Persistent volume does not need to be renamed because it's not claimed\")\n\t\treturn false, nil\n\t}\n\n\tif _, ok := ctx.restore.Spec.NamespaceMapping[pv.Spec.ClaimRef.Namespace]; !ok {\n\t\tctx.log.Debugf(\"Persistent volume does not need to be renamed because it's not claimed by a PVC in a namespace that's being remapped\")\n\t\treturn false, nil\n\t}\n\n\t_, err := client.Get(pv.Name, metav1.GetOptions{})\n\tswitch {\n\tcase apierrors.IsNotFound(err):\n\t\tctx.log.Debugf(\"Persistent volume does not need to be renamed because it does not exist in the cluster\")\n\t\treturn false, nil\n\tcase err != nil:\n\t\treturn false, errors.Wrapf(err, \"error checking if persistent volume exists in the cluster\")\n\t}\n\n\t// No error returned: the PV was found in-cluster, so we need to rename it.\n\treturn true, nil\n}\n\n// remapClaimRefNS remaps a PersistentVolume's claimRef.Namespace based on a\n// restore's NamespaceMappings, if necessary. Returns true if the namespace was\n// remapped, false if it was not required.\nfunc remapClaimRefNS(ctx *restoreContext, obj *unstructured.Unstructured) (bool, error) { //nolint:unparam // ignore the result 0 (bool) is never used warning.\n\tif len(ctx.restore.Spec.NamespaceMapping) == 0 {\n\t\tctx.log.Debug(\"Persistent volume does not need to have the claimRef.namespace remapped because restore is not remapping any namespaces\")\n\t\treturn false, nil\n\t}\n\n\t// Conversion to the real type here is more readable than all the error checking\n\t// involved with reading each field individually.\n\tpv := new(corev1api.PersistentVolume)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(obj.Object, pv); err != nil {\n\t\treturn false, errors.Wrapf(err, \"error converting persistent volume to structured\")\n\t}\n\n\tif pv.Spec.ClaimRef == nil {\n\t\tctx.log.Debugf(\"Persistent volume does not need to have the claimRef.namespace remapped because it's not claimed\")\n\t\treturn false, nil\n\t}\n\n\ttargetNS, ok := ctx.restore.Spec.NamespaceMapping[pv.Spec.ClaimRef.Namespace]\n\n\tif !ok {\n\t\tctx.log.Debugf(\"Persistent volume does not need to have the claimRef.namespace remapped because it's not claimed by a PVC in a namespace that's being remapped\")\n\t\treturn false, nil\n\t}\n\n\terr := unstructured.SetNestedField(obj.Object, targetNS, \"spec\", \"claimRef\", \"namespace\")\n\tif err != nil {\n\t\treturn false, err\n\t}\n\tctx.log.Debug(\"Persistent volume's namespace was updated\")\n\treturn true, nil\n}\n\n// restorePodVolumeBackups restores the PodVolumeBackups for the given restored pod\nfunc restorePodVolumeBackups(ctx *restoreContext, createdObj *unstructured.Unstructured, originalNamespace string) {\n\tif ctx.podVolumeRestorer == nil {\n\t\tctx.log.Warn(\"No pod volume restorer, not restoring pod's volumes\")\n\t} else {\n\t\tctx.podVolumeWaitGroup.Add(1)\n\t\tgo func() {\n\t\t\t// Done() will only be called after all errors have been successfully\n\t\t\t// sent on the ctx.podVolumeErrs channel\n\t\t\tdefer ctx.podVolumeWaitGroup.Done()\n\n\t\t\tpod := new(corev1api.Pod)\n\t\t\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(createdObj.UnstructuredContent(), &pod); err != nil {\n\t\t\t\tctx.log.WithError(err).Error(\"error converting unstructured pod\")\n\t\t\t\tctx.podVolumeErrs <- err\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tdata := podvolume.RestoreData{\n\t\t\t\tRestore: ctx.restore,\n\t\t\t\tPod: pod,\n\t\t\t\tPodVolumeBackups: ctx.podVolumeBackups,\n\t\t\t\tSourceNamespace: originalNamespace,\n\t\t\t\tBackupLocation: ctx.backup.Spec.StorageLocation,\n\t\t\t}\n\t\t\tif errs := ctx.podVolumeRestorer.RestorePodVolumes(data, ctx.restoreVolumeInfoTracker); errs != nil {\n\t\t\t\tctx.log.WithError(kubeerrs.NewAggregate(errs)).Error(\"unable to successfully complete pod volume restores of pod's volumes\")\n\n\t\t\t\tfor _, err := range errs {\n\t\t\t\t\tctx.podVolumeErrs <- err\n\t\t\t\t}\n\t\t\t}\n\t\t}()\n\t}\n}\n\n// hooksWaitExecutor is used to collect necessary fields that are required to asynchronously execute restore exec hooks\n// note that fields are shared across different pods within a specific restore\n// and separate hooksWaitExecutors instance will be created for different restores without interfering with each other.\ntype hooksWaitExecutor struct {\n\tlog logrus.FieldLogger\n\thooksContext go_context.Context\n\thooksCancelFunc go_context.CancelFunc\n\tresourceRestoreHooks []hook.ResourceRestoreHook\n\twaitExecHookHandler hook.WaitExecHookHandler\n}\n\nfunc newHooksWaitExecutor(restore *velerov1api.Restore, waitExecHookHandler hook.WaitExecHookHandler) (*hooksWaitExecutor, error) {\n\tresourceRestoreHooks, err := hook.GetRestoreHooksFromSpec(&restore.Spec.Hooks)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\thooksCtx, hooksCancelFunc := go_context.WithCancel(go_context.Background())\n\thwe := &hooksWaitExecutor{\n\t\tlog: logrus.WithField(\"restore\", restore.Name),\n\t\thooksContext: hooksCtx,\n\t\thooksCancelFunc: hooksCancelFunc,\n\t\tresourceRestoreHooks: resourceRestoreHooks,\n\t\twaitExecHookHandler: waitExecHookHandler,\n\t}\n\treturn hwe, nil\n}\n\n// groupHooks returns a list of hooks to be executed in a pod grouped bycontainer name.\nfunc (hwe *hooksWaitExecutor) groupHooks(restoreName string, pod *corev1api.Pod, multiHookTracker *hook.MultiHookTracker) (map[string][]hook.PodExecRestoreHook, error) {\n\texecHooksByContainer, err := hook.GroupRestoreExecHooks(restoreName, hwe.resourceRestoreHooks, pod, hwe.log, multiHookTracker)\n\treturn execHooksByContainer, err\n}\n\n// exec asynchronously executes hooks in a restored pod's containers when they become ready.\n// Goroutine within this function will continue running until the hook executions are complete.\n// Velero will wait for goroutine to finish in finalizing phase, using hook tracker to track the progress.\n// To optimize memory usage, ensure that the variables used in this function are kept to a minimum to prevent unnecessary retention in memory.\nfunc (hwe *hooksWaitExecutor) exec(execHooksByContainer map[string][]hook.PodExecRestoreHook, pod *corev1api.Pod, multiHookTracker *hook.MultiHookTracker, restoreName string) {\n\tgo func() {\n\t\tif errs := hwe.waitExecHookHandler.HandleHooks(hwe.hooksContext, hwe.log, pod, execHooksByContainer, multiHookTracker, restoreName); len(errs) > 0 {\n\t\t\thwe.log.WithError(kubeerrs.NewAggregate(errs)).Error(\"unable to successfully execute post-restore hooks\")\n\t\t\thwe.hooksCancelFunc()\n\t\t}\n\t}()\n}\n\nfunc hasSnapshot(pvName string, snapshots []*volume.Snapshot) bool {\n\tfor _, snapshot := range snapshots {\n\t\tif snapshot.Spec.PersistentVolumeName == pvName {\n\t\t\treturn true\n\t\t}\n\t}\n\n\treturn false\n}\n\nfunc hasCSIVolumeSnapshot(ctx *restoreContext, unstructuredPV *unstructured.Unstructured) bool {\n\tpv := new(corev1api.PersistentVolume)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(unstructuredPV.Object, pv); err != nil {\n\t\tctx.log.WithError(err).Warnf(\"Unable to convert PV from unstructured to structured\")\n\t\treturn false\n\t}\n\t// ignoring static PV cases where there is no claimRef\n\tif pv.Spec.ClaimRef == nil {\n\t\treturn false\n\t}\n\n\tfor _, vs := range ctx.csiVolumeSnapshots {\n\t\t// In some error cases, the VSs' source PVC could be nil. Skip them.\n\t\tif vs.Spec.Source.PersistentVolumeClaimName == nil {\n\t\t\tcontinue\n\t\t}\n\n\t\tif pv.Spec.ClaimRef.Name == *vs.Spec.Source.PersistentVolumeClaimName &&\n\t\t\tpv.Spec.ClaimRef.Namespace == vs.Namespace {\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n\nfunc hasSnapshotDataUpload(ctx *restoreContext, unstructuredPV *unstructured.Unstructured) bool {\n\tpv := new(corev1api.PersistentVolume)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(unstructuredPV.Object, pv); err != nil {\n\t\tctx.log.WithError(err).Warnf(\"Unable to convert PV from unstructured to structured\")\n\t\treturn false\n\t}\n\n\tif pv.Spec.ClaimRef == nil {\n\t\treturn false\n\t}\n\n\tdataUploadResultList := new(corev1api.ConfigMapList)\n\terr := ctx.kbClient.List(go_context.TODO(), dataUploadResultList, &crclient.ListOptions{\n\t\tLabelSelector: labels.SelectorFromSet(map[string]string{\n\t\t\tvelerov1api.RestoreUIDLabel: label.GetValidName(string(ctx.restore.GetUID())),\n\t\t\tvelerov1api.PVCNamespaceNameLabel: label.GetValidName(pv.Spec.ClaimRef.Namespace + \".\" + pv.Spec.ClaimRef.Name),\n\t\t\tvelerov1api.ResourceUsageLabel: label.GetValidName(string(velerov1api.VeleroResourceUsageDataUploadResult)),\n\t\t}),\n\t})\n\tif err != nil {\n\t\tctx.log.WithError(err).Warnf(\"Fail to list DataUpload result CM.\")\n\t\treturn false\n\t}\n\n\tif len(dataUploadResultList.Items) != 1 {\n\t\tctx.log.WithError(fmt.Errorf(\"dataupload result number is not expected\")).\n\t\t\tWarnf(\"Got %d DataUpload result. Expect one.\", len(dataUploadResultList.Items))\n\t\treturn false\n\t}\n\n\treturn true\n}\n\nfunc hasPodVolumeBackup(unstructuredPV *unstructured.Unstructured, ctx *restoreContext) bool {\n\tif len(ctx.podVolumeBackups) == 0 {\n\t\treturn false\n\t}\n\n\tpv := new(corev1api.PersistentVolume)\n\tif err := runtime.DefaultUnstructuredConverter.FromUnstructured(unstructuredPV.Object, pv); err != nil {\n\t\tctx.log.WithError(err).Warnf(\"Unable to convert PV from unstructured to structured\")\n\t\treturn false\n\t}\n\n\tif pv.Spec.ClaimRef == nil {\n\t\treturn false\n\t}\n\n\tvar found bool\n\tfor _, pvb := range ctx.podVolumeBackups {\n\t\tif pvb.Spec.Pod.Namespace == pv.Spec.ClaimRef.Namespace && pvb.GetAnnotations()[configs.PVCNameAnnotation] == pv.Spec.ClaimRef.Name {\n\t\t\tfound = true\n\t\t\tbreak\n\t\t}\n\t}\n\n\treturn found\n}\n\nfunc hasDeleteReclaimPolicy(obj map[string]any) bool {\n\tpolicy, _, _ := unstructured.NestedString(obj, \"spec\", \"persistentVolumeReclaimPolicy\")\n\treturn policy == string(corev1api.PersistentVolumeReclaimDelete)\n}\n\n// resetVolumeBindingInfo clears any necessary metadata out of a PersistentVolume\n// or PersistentVolumeClaim that would make it ineligible to be re-bound by Velero.\nfunc resetVolumeBindingInfo(obj *unstructured.Unstructured) *unstructured.Unstructured {\n\t// Clean out ClaimRef UID and resourceVersion, since this information is\n\t// highly unique.\n\tunstructured.RemoveNestedField(obj.Object, \"spec\", \"claimRef\", \"uid\")\n\tunstructured.RemoveNestedField(obj.Object, \"spec\", \"claimRef\", \"resourceVersion\")\n\n\t// Clear out any annotations used by the Kubernetes PV controllers to track\n\t// bindings.\n\tannotations := obj.GetAnnotations()\n\n\t// Upon restore, this new PV will look like a statically provisioned, manually-\n\t// bound volume rather than one bound by the controller, so remove the annotation\n\t// that signals that a controller bound it.\n\tdelete(annotations, kube.KubeAnnBindCompleted)\n\t// Remove the annotation that signals that the PV is already bound; we want\n\t// the PV(C) controller to take the two objects and bind them again.\n\tdelete(annotations, kube.KubeAnnBoundByController)\n\n\t// GetAnnotations returns a copy, so we have to set them again.\n\tobj.SetAnnotations(annotations)\n\n\treturn obj\n}\n\nfunc resetMetadata(obj *unstructured.Unstructured) (*unstructured.Unstructured, error) {\n\tres, ok := obj.Object[\"metadata\"]\n\tif !ok {\n\t\treturn nil, errors.New(\"metadata not found\")\n\t}\n\tmetadata, ok := res.(map[string]any)\n\tif !ok {\n\t\treturn nil, errors.Errorf(\"metadata was of type %T, expected map[string]any\", res)\n\t}\n\n\tfor k := range metadata {\n\t\tswitch k {\n\t\tcase \"generateName\", \"selfLink\", \"uid\", \"resourceVersion\", \"generation\", \"creationTimestamp\", \"deletionTimestamp\",\n\t\t\t\"deletionGracePeriodSeconds\", \"ownerReferences\":\n\t\t\tdelete(metadata, k)\n\t\t}\n\t}\n\n\treturn obj, nil\n}\n\nfunc resetStatus(obj *unstructured.Unstructured) {\n\tunstructured.RemoveNestedField(obj.UnstructuredContent(), \"status\")\n}\n\nfunc resetMetadataAndStatus(obj *unstructured.Unstructured) (*unstructured.Unstructured, error) {\n\t_, err := resetMetadata(obj)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tresetStatus(obj)\n\treturn obj, nil\n}\n\n// addRestoreLabels labels the provided object with the restore name and the\n// restored backup's name.\nfunc addRestoreLabels(obj metav1.Object, restoreName, backupName string) {\n\tlabels := obj.GetLabels()\n\n\tif labels == nil {\n\t\tlabels = make(map[string]string)\n\t}\n\n\tlabels[velerov1api.BackupNameLabel] = label.GetValidName(backupName)\n\tlabels[velerov1api.RestoreNameLabel] = label.GetValidName(restoreName)\n\n\tobj.SetLabels(labels)\n}\n\n// isCompleted returns whether or not an object is considered completed. Used to\n// identify whether or not an object should be restored. Only Jobs or Pods are\n// considered.\nfunc isCompleted(obj *unstructured.Unstructured, groupResource schema.GroupResource) (bool, error) {\n\tswitch groupResource {\n\tcase kuberesource.Pods:\n\t\tphase, _, err := unstructured.NestedString(obj.UnstructuredContent(), \"status\", \"phase\")\n\t\tif err != nil {\n\t\t\treturn false, errors.WithStack(err)\n\t\t}\n\t\tif phase == string(corev1api.PodFailed) || phase == string(corev1api.PodSucceeded) {\n\t\t\treturn true, nil\n\t\t}\n\n\tcase kuberesource.Jobs:\n\t\tct, found, err := unstructured.NestedString(obj.UnstructuredContent(), \"status\", \"completionTime\")\n\t\tif err != nil {\n\t\t\treturn false, errors.WithStack(err)\n\t\t}\n\t\tif found && ct != \"\" {\n\t\t\treturn true, nil\n\t\t}\n\t}\n\t// Assume any other resource isn't complete and can be restored.\n\treturn false, nil\n}\n\n// restoreableResource represents map of individual items of each resource\n// identifier grouped by their original namespaces.\ntype restoreableResource struct {\n\tresource string\n\tselectedItemsByNamespace map[string][]restoreableItem\n\ttotalItems int\n}\n\n// restoreableItem represents an item by its target namespace contains enough\n// information required to restore the item.\ntype restoreableItem struct {\n\tpath string\n\ttargetNamespace string\n\tname string\n\tversion string // used for initializing informer cache\n}\n\n// getOrderedResourceCollection iterates over list of ordered resource\n// identifiers, applies resource include/exclude criteria, and Kubernetes\n// selectors to make a list of resources to be actually restored preserving the\n// original order.\nfunc (ctx *restoreContext) getOrderedResourceCollection(\n\tbackupResources map[string]*archive.ResourceItems,\n\trestoreResourceCollection []restoreableResource,\n\tprocessedResources sets.Set[string],\n\tresourcePriorities types.Priorities,\n\tincludeAllResources bool,\n) ([]restoreableResource, sets.Set[string], results.Result, results.Result) {\n\tvar warnings, errs results.Result\n\t// Iterate through an ordered list of resources to restore, checking each\n\t// one to see if it should be restored. Note that resources *may* be in this\n\t// list twice, i.e. once due to being a prioritized resource, and once due\n\t// to being in the backup tarball. We can't de-dupe this upfront, because\n\t// it's possible that items in the prioritized resources list may not be\n\t// fully resolved group-resource strings (e.g. may be specified as \"po\"\n\t// instead of \"pods\"), and we don't want to fully resolve them via discovery\n\t// until we reach them in the loop, because it is possible that the\n\t// resource/API itself is being restored via a custom resource definition,\n\t// meaning it's not available via discovery prior to beginning the restore.\n\t//\n\t// Since we keep track of the fully-resolved group-resources that we *have*\n\t// restored, we won't try to restore a resource twice even if it's in the\n\t// ordered list twice.\n\tvar resourceList []string\n\tif includeAllResources {\n\t\tresourceList = getOrderedResources(resourcePriorities, backupResources)\n\t} else {\n\t\tresourceList = resourcePriorities.HighPriorities\n\t}\n\tfor _, resource := range resourceList {\n\t\tgroupResource := schema.ParseGroupResource(resource)\n\t\t// try to resolve the resource via discovery to a complete group/version/resource\n\t\tgvr, _, err := ctx.discoveryHelper.ResourceFor(groupResource.WithVersion(\"\"))\n\t\tif err != nil {\n\t\t\t// don't skip if we can't resolve the resource via discovery, log it\n\t\t\t// the gv of this resource may be changed in a RIA, we can try to get it after that\n\t\t\tctx.log.WithField(\"resource\", resource).Infof(\"resource cannot be resolved via discovery\")\n\t\t} else {\n\t\t\tgroupResource = gvr.GroupResource()\n\t\t}\n\n\t\t// Check if we've already restored this resource (this would happen if\n\t\t// the resource we're currently looking at was already restored because\n\t\t// it was a prioritized resource, and now we're looking at it as part of\n\t\t// the backup contents).\n\t\tif processedResources.Has(groupResource.String()) {\n\t\t\tctx.log.WithField(\"resource\", groupResource.String()).Debugf(\"Skipping restore of resource because it's already been processed\")\n\t\t\tcontinue\n\t\t}\n\n\t\t// Check if the resource should be restored according to the resource\n\t\t// includes/excludes.\n\t\tif !ctx.resourceIncludesExcludes.ShouldInclude(groupResource.String()) && !ctx.resourceMustHave.Has(groupResource.String()) {\n\t\t\tctx.log.WithField(\"resource\", groupResource.String()).Infof(\"Skipping restore of resource because the restore spec excludes it\")\n\t\t\tcontinue\n\t\t}\n\n\t\t// Check if the resource is present in the backup\n\t\tresourceList := backupResources[groupResource.String()]\n\t\tif resourceList == nil {\n\t\t\tctx.log.WithField(\"resource\", groupResource.String()).Debugf(\"Skipping restore of resource because it's not present in the backup tarball\")\n\t\t\tcontinue\n\t\t}\n\n\t\t// Iterate through each namespace that contains instances of the\n\t\t// resource and append to the list of to-be restored resources.\n\t\tfor namespace, items := range resourceList.ItemsByNamespace {\n\t\t\tif namespace != \"\" && !ctx.namespaceIncludesExcludes.ShouldInclude(namespace) && !ctx.resourceMustHave.Has(groupResource.String()) {\n\t\t\t\tctx.log.Infof(\"Skipping namespace %s\", namespace)\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tif namespace == \"\" && boolptr.IsSetToFalse(ctx.restore.Spec.IncludeClusterResources) {\n\t\t\t\tctx.log.Infof(\"Skipping resource %s because it's cluster-scoped\", resource)\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tif namespace == \"\" && !boolptr.IsSetToTrue(ctx.restore.Spec.IncludeClusterResources) && !ctx.namespaceIncludesExcludes.IncludeEverything() {\n\t\t\t\tctx.log.Infof(\"Skipping resource %s because it's cluster-scoped and only specific namespaces are included in the restore\", resource)\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tres, w, e := ctx.getSelectedRestoreableItems(groupResource.String(), namespace, items)\n\t\t\twarnings.Merge(&w)\n\t\t\terrs.Merge(&e)\n\n\t\t\trestoreResourceCollection = append(restoreResourceCollection, res)\n\t\t}\n\n\t\t// record that we've restored the resource\n\t\tprocessedResources.Insert(groupResource.String())\n\t}\n\treturn restoreResourceCollection, processedResources, warnings, errs\n}\n\n// getSelectedRestoreableItems applies Kubernetes selectors on individual items\n// of each resource type to create a list of items which will be actually\n// restored.\nfunc (ctx *restoreContext) getSelectedRestoreableItems(resource string, originalNamespace string, items []string) (restoreableResource, results.Result, results.Result) { //nolint:unparam // Ignore the warnings is always nil warning.\n\twarnings, errs := results.Result{}, results.Result{}\n\n\trestorable := restoreableResource{\n\t\tresource: resource,\n\t}\n\n\tif restorable.selectedItemsByNamespace == nil {\n\t\trestorable.selectedItemsByNamespace = make(map[string][]restoreableItem)\n\t}\n\n\ttargetNamespace := originalNamespace\n\tif target, ok := ctx.restore.Spec.NamespaceMapping[originalNamespace]; ok {\n\t\ttargetNamespace = target\n\t}\n\n\tif targetNamespace != \"\" {\n\t\tctx.log.Infof(\"Resource '%s' will be restored into namespace '%s'\", resource, targetNamespace)\n\t} else {\n\t\tctx.log.Infof(\"Resource '%s' will be restored at cluster scope\", resource)\n\t}\n\n\tresourceForPath := resource\n\t// If the APIGroupVersionsFeatureFlag is enabled, the item path will be\n\t// updated to include the API group version that was chosen for restore. For\n\t// example, for \"horizontalpodautoscalers.autoscaling\", if v2beta1 is chosen\n\t// to be restored, then \"horizontalpodautoscalers.autoscaling/v2beta1\" will\n\t// be part of item path. Different versions would only have been stored\n\t// if the APIGroupVersionsFeatureFlag was enabled during backup. The\n\t// chosenGrpVersToRestore map would only be populated if\n\t// APIGroupVersionsFeatureFlag was enabled for restore and the minimum\n\t// required backup format version has been met.\n\tcgv, ok := ctx.chosenGrpVersToRestore[resource]\n\tif ok {\n\t\tresourceForPath = filepath.Join(resource, cgv.Dir)\n\t}\n\n\tfor _, item := range items {\n\t\titemPath := archive.GetItemFilePath(ctx.restoreDir, resourceForPath, originalNamespace, item)\n\n\t\tobj, err := archive.Unmarshal(ctx.fileSystem, itemPath)\n\t\tif err != nil {\n\t\t\terrs.Add(\n\t\t\t\ttargetNamespace,\n\t\t\t\tfmt.Errorf(\n\t\t\t\t\t\"error decoding %q: %v\",\n\t\t\t\t\tstrings.Replace(itemPath, ctx.restoreDir+\"/\", \"\", -1),\n\t\t\t\t\terr,\n\t\t\t\t),\n\t\t\t)\n\t\t\tcontinue\n\t\t}\n\n\t\tif !ctx.resourceMustHave.Has(resource) {\n\t\t\tif !ctx.selector.Matches(labels.Set(obj.GetLabels())) {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\t// Processing OrLabelSelectors when specified in the restore request. LabelSelectors as well as OrLabelSelectors\n\t\t\t// cannot co-exist, only one of them can be specified\n\t\t\tvar skipItem = false\n\t\t\tvar skip = 0\n\t\t\tctx.log.Debugf(\"orSelectors specified: %s for item: %s\", ctx.OrSelectors, item)\n\t\t\tfor _, s := range ctx.OrSelectors {\n\t\t\t\tif !s.Matches(labels.Set(obj.GetLabels())) {\n\t\t\t\t\tskip++\n\t\t\t\t}\n\n\t\t\t\tif len(ctx.OrSelectors) == skip && skip > 0 {\n\t\t\t\t\tctx.log.Infof(\"setting skip flag to true for item: %s\", item)\n\t\t\t\t\tskipItem = true\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif skipItem {\n\t\t\t\tctx.log.Infof(\"restore orSelector labels did not match, skipping restore of item: %s\", skipItem, item)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t}\n\n\t\tselectedItem := restoreableItem{\n\t\t\tpath: itemPath,\n\t\t\tname: item,\n\t\t\ttargetNamespace: targetNamespace,\n\t\t\tversion: obj.GroupVersionKind().Version,\n\t\t}\n\t\trestorable.selectedItemsByNamespace[originalNamespace] =\n\t\t\tappend(restorable.selectedItemsByNamespace[originalNamespace], selectedItem)\n\t\trestorable.totalItems++\n\t}\n\treturn restorable, warnings, errs\n}\n\n// extractNamespacesFromBackup extracts all available namespaces from backup resources\nfunc extractNamespacesFromBackup(backupResources map[string]*archive.ResourceItems) []string {\n\tnamespaceSet := make(map[string]struct{})\n\tfor _, resource := range backupResources {\n\t\tfor namespace := range resource.ItemsByNamespace {\n\t\t\tif namespace != \"\" { // Skip cluster-scoped resources (empty namespace)\n\t\t\t\tnamespaceSet[namespace] = struct{}{}\n\t\t\t}\n\t\t}\n\t}\n\n\tnamespaces := make([]string, 0, len(namespaceSet))\n\tfor ns := range namespaceSet {\n\t\tnamespaces = append(namespaces, ns)\n\t}\n\treturn namespaces\n}\n\n// expandNamespaceWildcards expands wildcard patterns in namespace includes/excludes\n// and updates the restore context with the expanded patterns and status\nfunc (ctx *restoreContext) expandNamespaceWildcards(backupResources map[string]*archive.ResourceItems) error {\n\tif !wildcard.ShouldExpandWildcards(ctx.restore.Spec.IncludedNamespaces, ctx.restore.Spec.ExcludedNamespaces) {\n\t\treturn nil\n\t}\n\n\t// If `*` is mentioned in restore excludes, something is wrong\n\tif slices.Contains(ctx.restore.Spec.ExcludedNamespaces, \"*\") {\n\t\treturn errors.New(\"wildcard '*' is not allowed in restore excludes\")\n\t}\n\n\tavailableNamespaces := extractNamespacesFromBackup(backupResources)\n\texpandedIncludes, expandedExcludes, err := wildcard.ExpandWildcards(\n\t\tavailableNamespaces,\n\t\tctx.restore.Spec.IncludedNamespaces,\n\t\tctx.restore.Spec.ExcludedNamespaces,\n\t)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error expanding wildcard patterns in namespace includes/excludes\")\n\t}\n\n\t// Update namespace includes/excludes with expanded patterns\n\tctx.namespaceIncludesExcludes = collections.NewIncludesExcludes().\n\t\tIncludes(expandedIncludes...).\n\t\tExcludes(expandedExcludes...)\n\n\tselectedNamespaces := wildcard.GetWildcardResult(expandedIncludes, expandedExcludes)\n\n\tctx.log.Infof(\"Expanded namespace wildcards - includes: %v, excludes: %v, final: %v\",\n\t\texpandedIncludes, expandedExcludes, selectedNamespaces)\n\n\treturn nil\n}\n\n// removeRestoreLabels removes the restore name and the\n// restored backup's name.\nfunc removeRestoreLabels(obj metav1.Object) {\n\tlabels := obj.GetLabels()\n\n\tif labels == nil {\n\t\tlabels = make(map[string]string)\n\t}\n\n\tlabels[velerov1api.BackupNameLabel] = \"\"\n\tlabels[velerov1api.RestoreNameLabel] = \"\"\n\n\tobj.SetLabels(labels)\n}\n\n// updates the backup/restore labels\nfunc (ctx *restoreContext) updateBackupRestoreLabels(fromCluster, fromClusterWithLabels *unstructured.Unstructured, namespace string, resourceClient client.Dynamic) (warnings, errs results.Result) { //nolint:unparam // Ignore the warnings is nil warning.\n\tpatchBytes, err := generatePatch(fromCluster, fromClusterWithLabels)\n\tif err != nil {\n\t\tctx.log.Errorf(\"error generating patch for %s %s: %v\", fromCluster.GroupVersionKind().Kind, kube.NamespaceAndName(fromCluster), err)\n\t\terrs.Add(namespace, err)\n\t\treturn warnings, errs\n\t}\n\n\tif patchBytes == nil {\n\t\t// In-cluster and desired state are the same, so move on to\n\t\t// the next items\n\t\tctx.log.Errorf(\"skipped updating backup/restore labels for %s %s: in-cluster and desired state are the same along-with the labels\", fromCluster.GroupVersionKind().Kind, kube.NamespaceAndName(fromCluster))\n\t\treturn warnings, errs\n\t}\n\n\t// try patching the in-cluster resource (with only latest backup/restore labels)\n\t_, err = resourceClient.Patch(fromCluster.GetName(), patchBytes)\n\tif err != nil {\n\t\tctx.log.Errorf(\"backup/restore label patch attempt failed for %s %s: %v\", fromCluster.GroupVersionKind(), kube.NamespaceAndName(fromCluster), err)\n\t\terrs.Add(namespace, err)\n\t} else {\n\t\tctx.log.Infof(\"backup/restore labels successfully updated for %s %s\", fromCluster.GroupVersionKind().Kind, kube.NamespaceAndName(fromCluster))\n\t}\n\treturn warnings, errs\n}\n\n// function to process existingResourcePolicy as update, tries to patch the diff between in-cluster and restore obj first\n// if the patch fails then tries to update the backup/restore labels for the in-cluster version\nfunc (ctx *restoreContext) processUpdateResourcePolicy(fromCluster, fromClusterWithLabels, obj *unstructured.Unstructured, namespace string, resourceClient client.Dynamic) (warnings, errs results.Result) {\n\tctx.log.Infof(\"restore API has existingResourcePolicy defined as update , executing restore workflow accordingly for changed resource %s %s \", obj.GroupVersionKind().Kind, kube.NamespaceAndName(fromCluster))\n\tctx.log.Infof(\"attempting patch on %s %q\", fromCluster.GetKind(), fromCluster.GetName())\n\t// remove restore labels so that we apply the latest backup/restore names on the object via patch\n\tremoveRestoreLabels(fromCluster)\n\tpatchBytes, err := generatePatch(fromCluster, obj)\n\tif err != nil {\n\t\tctx.log.Errorf(\"error generating patch for %s %s: %v\", obj.GroupVersionKind().Kind, kube.NamespaceAndName(obj), err)\n\t\terrs.Add(namespace, err)\n\t\treturn warnings, errs\n\t}\n\n\tif patchBytes == nil {\n\t\t// In-cluster and desired state are the same, so move on to\n\t\t// the next items\n\t\tctx.log.Errorf(\"skipped updating %s %s: in-cluster and desired state are the same\", fromCluster.GroupVersionKind().Kind, kube.NamespaceAndName(fromCluster))\n\t\treturn warnings, errs\n\t}\n\n\t// try patching the in-cluster resource (resource diff plus latest backup/restore labels)\n\t_, err = resourceClient.Patch(obj.GetName(), patchBytes)\n\tif err != nil {\n\t\tctx.log.Warnf(\"patch attempt failed for %s %s: %v\", fromCluster.GroupVersionKind(), kube.NamespaceAndName(fromCluster), err)\n\t\twarnings.Add(namespace, err)\n\t\t// try just patching the labels\n\t\twarningsFromUpdate, errsFromUpdate := ctx.updateBackupRestoreLabels(fromCluster, fromClusterWithLabels, namespace, resourceClient)\n\t\twarnings.Merge(&warningsFromUpdate)\n\t\terrs.Merge(&errsFromUpdate)\n\t} else {\n\t\tctx.log.Infof(\"%s %s successfully updated\", obj.GroupVersionKind().Kind, kube.NamespaceAndName(obj))\n\t}\n\treturn warnings, errs\n}\n\nfunc (ctx *restoreContext) handlePVHasNativeSnapshot(obj *unstructured.Unstructured, resourceClient client.Dynamic) (*unstructured.Unstructured, error) {\n\tretObj := obj.DeepCopy()\n\toldName := obj.GetName()\n\tshouldRenamePV, err := shouldRenamePV(ctx, retObj, resourceClient)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\t// Check to see if the claimRef.namespace field needs to be remapped,\n\t// and do so if necessary.\n\t_, err = remapClaimRefNS(ctx, retObj)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tvar shouldRestoreSnapshot bool\n\tif !shouldRenamePV {\n\t\t// Check if the PV exists in the cluster before attempting to create\n\t\t// a volume from the snapshot, in order to avoid orphaned volumes (GH #609)\n\t\tshouldRestoreSnapshot, err = ctx.shouldRestore(oldName, resourceClient)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrapf(err, \"error waiting on in-cluster persistentvolume %s\", oldName)\n\t\t}\n\t} else {\n\t\t// If we're renaming the PV, we're going to give it a new random name,\n\t\t// so we can assume it doesn't already exist in the cluster and therefore\n\t\t// we should proceed with restoring from snapshot.\n\t\tshouldRestoreSnapshot = true\n\t}\n\n\tif shouldRestoreSnapshot {\n\t\t// Reset the PV's binding status so that Kubernetes can properly\n\t\t// associate it with the restored PVC.\n\t\tretObj = resetVolumeBindingInfo(retObj)\n\n\t\t// Even if we're renaming the PV, obj still has the old name here, because the pvRestorer\n\t\t// uses the original name to look up metadata about the snapshot.\n\t\tctx.log.Infof(\"Restoring persistent volume from snapshot.\")\n\t\tretObj, err = ctx.pvRestorer.executePVAction(retObj)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"error executing PVAction for %s: %v\", getResourceID(kuberesource.PersistentVolumes, \"\", oldName), err)\n\t\t}\n\n\t\t// VolumeSnapshotter has modified the PV name, we should rename the PV.\n\t\tif oldName != retObj.GetName() {\n\t\t\tshouldRenamePV = true\n\t\t}\n\t}\n\n\tif shouldRenamePV {\n\t\tvar pvName string\n\t\tif oldName == retObj.GetName() {\n\t\t\t// pvRestorer hasn't modified the PV name, we need to rename the PV.\n\t\t\tpvName, err = ctx.pvRenamer(oldName)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, errors.Wrapf(err, \"error renaming PV\")\n\t\t\t}\n\t\t} else {\n\t\t\t// VolumeSnapshotter could have modified the PV name through\n\t\t\t// function `SetVolumeID`,\n\t\t\tpvName = retObj.GetName()\n\t\t}\n\n\t\tctx.renamedPVs[oldName] = pvName\n\t\tretObj.SetName(pvName)\n\t\tctx.restoreVolumeInfoTracker.RenamePVForNativeSnapshot(oldName, pvName)\n\t\t// Add the original PV name as an annotation.\n\t\tannotations := retObj.GetAnnotations()\n\t\tif annotations == nil {\n\t\t\tannotations = map[string]string{}\n\t\t}\n\t\tannotations[\"velero.io/original-pv-name\"] = oldName\n\t\tretObj.SetAnnotations(annotations)\n\t}\n\n\treturn retObj, nil\n}\n\nfunc (ctx *restoreContext) handleSkippedPVHasRetainPolicy(\n\tobj *unstructured.Unstructured,\n\tlogger logrus.FieldLogger,\n) (*unstructured.Unstructured, error) {\n\tlogger.Infof(\"Restoring persistent volume as-is because it doesn't have a snapshot and its reclaim policy is not Delete.\")\n\n\t// Check to see if the claimRef.namespace field needs to be remapped, and do so if necessary.\n\tif _, err := remapClaimRefNS(ctx, obj); err != nil {\n\t\treturn nil, err\n\t}\n\n\tobj = resetVolumeBindingInfo(obj)\n\treturn obj, nil\n}\n\nfunc determineRestoreStatus(\n\tobj *unstructured.Unstructured,\n\tresourceIncludesExcludes *collections.IncludesExcludes,\n\tgroupResource string,\n\tlog logrus.FieldLogger,\n) bool {\n\tvar shouldRestoreStatus bool\n\n\t// Determine restore spec behavior\n\tif resourceIncludesExcludes != nil {\n\t\tshouldRestoreStatus = resourceIncludesExcludes.ShouldInclude(groupResource)\n\t}\n\n\t// Retrieve annotations\n\tannotations := obj.GetAnnotations()\n\n\tif annotations == nil {\n\t\tlog.Warnf(\"No annotations found for %s, using restore spec setting: %v\",\n\t\t\tkube.NamespaceAndName(obj), shouldRestoreStatus)\n\t\treturn shouldRestoreStatus\n\t}\n\n\t// Check for object-level annotation\n\tobjectAnnotation, annotationExists := annotations[ObjectStatusRestoreAnnotationKey]\n\n\tif !annotationExists {\n\t\tlog.Debugf(\"No restore status-specific annotation found for %s, using restore spec setting: %v\",\n\t\t\tkube.NamespaceAndName(obj), shouldRestoreStatus)\n\t\treturn shouldRestoreStatus\n\t}\n\n\tnormalizedValue := strings.ToLower(strings.TrimSpace(objectAnnotation))\n\tswitch normalizedValue {\n\tcase \"true\":\n\t\tshouldRestoreStatus = true\n\tcase \"false\":\n\t\tshouldRestoreStatus = false\n\tdefault:\n\t\tlog.Warnf(\"Invalid annotation value '%s' on %s, using restore spec setting: %v\",\n\t\t\tobjectAnnotation, kube.NamespaceAndName(obj), shouldRestoreStatus)\n\t}\n\n\tlog.Infof(\"Final status restore decision for %s: %v (annotation: %v, restore spec: %v)\",\n\t\tkube.NamespaceAndName(obj), shouldRestoreStatus, annotationExists, shouldRestoreStatus)\n\n\treturn shouldRestoreStatus\n}\n" }, { "path": "pkg/restore/restore_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"sort\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/collections\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\t\"k8s.io/apimachinery/pkg/api/meta\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\t\"k8s.io/client-go/dynamic\"\n\tkubetesting \"k8s.io/client-go/testing\"\n\n\t\"github.com/vmware-tanzu/velero/internal/volume\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/archive\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n\t\"github.com/vmware-tanzu/velero/pkg/discovery\"\n\t\"github.com/vmware-tanzu/velero/pkg/features\"\n\t\"github.com/vmware-tanzu/velero/pkg/itemoperation\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n\triav2 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/restoreitemaction/v2\"\n\tvsv1 \"github.com/vmware-tanzu/velero/pkg/plugin/velero/volumesnapshotter/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/podvolume\"\n\tuploadermocks \"github.com/vmware-tanzu/velero/pkg/podvolume/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/types\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\tkubeutil \"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\t. \"github.com/vmware-tanzu/velero/pkg/util/results\"\n)\n\nfunc TestRestorePVWithVolumeInfo(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\trestore *velerov1api.Restore\n\t\tbackup *velerov1api.Backup\n\t\tapiResources []*test.APIResource\n\t\ttarball io.Reader\n\t\twant map[*test.APIResource][]string\n\t\tvolumeInfoMap map[string]volume.BackupVolumeInfo\n\t}{\n\t\t{\n\t\t\tname: \"Restore PV with native snapshot\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).Result(),\n\t\t\t\t).Done(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\tvolumeInfoMap: map[string]volume.BackupVolumeInfo{\n\t\t\t\t\"pv-1\": {\n\t\t\t\t\tBackupMethod: volume.NativeSnapshot,\n\t\t\t\t\tPVName: \"pv-1\",\n\t\t\t\t\tNativeSnapshotInfo: &volume.NativeSnapshotInfo{\n\t\t\t\t\t\tSnapshotHandle: \"testSnapshotHandle\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.PVs(): {\"/pv-1\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Restore PV with PVB\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).Result(),\n\t\t\t\t).Done(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\tvolumeInfoMap: map[string]volume.BackupVolumeInfo{\n\t\t\t\t\"pv-1\": {\n\t\t\t\t\tBackupMethod: volume.PodVolumeBackup,\n\t\t\t\t\tPVName: \"pv-1\",\n\t\t\t\t\tPVBInfo: &volume.PodVolumeInfo{\n\t\t\t\t\t\tSnapshotHandle: \"testSnapshotHandle\",\n\t\t\t\t\t\tSize: 100,\n\t\t\t\t\t\tNodeName: \"testNode\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.PVs(): {},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Restore PV with CSI VolumeSnapshot\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).Result(),\n\t\t\t\t).Done(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\tvolumeInfoMap: map[string]volume.BackupVolumeInfo{\n\t\t\t\t\"pv-1\": {\n\t\t\t\t\tBackupMethod: volume.CSISnapshot,\n\t\t\t\t\tSnapshotDataMoved: false,\n\t\t\t\t\tPVName: \"pv-1\",\n\t\t\t\t\tCSISnapshotInfo: &volume.CSISnapshotInfo{\n\t\t\t\t\t\tDriver: \"pd.csi.storage.gke.io\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.PVs(): {},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Restore PV with DataUpload\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).Result(),\n\t\t\t\t).Done(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\tvolumeInfoMap: map[string]volume.BackupVolumeInfo{\n\t\t\t\t\"pv-1\": {\n\t\t\t\t\tBackupMethod: volume.CSISnapshot,\n\t\t\t\t\tSnapshotDataMoved: true,\n\t\t\t\t\tPVName: \"pv-1\",\n\t\t\t\t\tCSISnapshotInfo: &volume.CSISnapshotInfo{\n\t\t\t\t\t\tDriver: \"pd.csi.storage.gke.io\",\n\t\t\t\t\t},\n\t\t\t\t\tSnapshotDataMovementInfo: &volume.SnapshotDataMovementInfo{\n\t\t\t\t\t\tDataMover: \"velero\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.PVs(): {},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Restore PV with ClaimPolicy as Delete\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ReclaimPolicy(corev1api.PersistentVolumeReclaimDelete).Result(),\n\t\t\t\t).Done(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\tvolumeInfoMap: map[string]volume.BackupVolumeInfo{\n\t\t\t\t\"pv-1\": {\n\t\t\t\t\tPVName: \"pv-1\",\n\t\t\t\t\tSkipped: true,\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.PVs(): {},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Restore PV with ClaimPolicy as Retain\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).Result(),\n\t\t\t\t).Done(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\tvolumeInfoMap: map[string]volume.BackupVolumeInfo{\n\t\t\t\t\"pv-1\": {\n\t\t\t\t\tPVName: \"pv-1\",\n\t\t\t\t\tSkipped: true,\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.PVs(): {\"/pv-1\"},\n\t\t\t},\n\t\t},\n\t}\n\n\tfeatures.Enable(\"EnableCSI\")\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\th := newHarness(t)\n\n\t\t\tfor _, r := range tc.apiResources {\n\t\t\t\th.DiscoveryClient.WithAPIResource(r)\n\t\t\t}\n\t\t\trequire.NoError(t, h.restorer.discoveryHelper.Refresh())\n\n\t\t\tdata := &Request{\n\t\t\t\tLog: h.log,\n\t\t\t\tRestore: tc.restore,\n\t\t\t\tBackup: tc.backup,\n\t\t\t\tPodVolumeBackups: nil,\n\t\t\t\tVolumeSnapshots: nil,\n\t\t\t\tBackupReader: tc.tarball,\n\t\t\t\tBackupVolumeInfoMap: tc.volumeInfoMap,\n\t\t\t}\n\t\t\twarnings, errs := h.restorer.Restore(\n\t\t\t\tdata,\n\t\t\t\tnil, // restoreItemActions\n\t\t\t\tnil, // volume snapshotter getter\n\t\t\t)\n\n\t\t\tassertEmptyResults(t, warnings, errs)\n\t\t\tassertAPIContents(t, h, tc.want)\n\t\t})\n\t}\n}\n\n// TestRestoreResourceFiltering runs restores with different combinations\n// of resource filters (included/excluded resources, included/excluded\n// namespaces, label selectors, \"include cluster resources\" flag), and\n// verifies that the set of items created in the API are correct.\n// Validation is done by looking at the namespaces/names of the items in\n// the API; contents are not checked.\nfunc TestRestoreResourceFiltering(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\trestore *velerov1api.Restore\n\t\tbackup *velerov1api.Backup\n\t\tapiResources []*test.APIResource\n\t\ttarball io.Reader\n\t\twant map[*test.APIResource][]string\n\t}{\n\t\t{\n\t\t\tname: \"no filters restores everything\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\", \"ns-2/pod-2\"},\n\t\t\t\ttest.PVs(): {\"/pv-1\", \"/pv-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"included resources filter only restores resources of those types\",\n\t\t\trestore: defaultRestore().IncludedResources(\"pods\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\", \"ns-2/pod-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"excluded resources filter only restores resources not of those types\",\n\t\t\trestore: defaultRestore().ExcludedResources(\"pvs\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\", \"ns-2/pod-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"included namespaces filter only restores resources in those namespaces\",\n\t\t\trestore: defaultRestore().IncludedNamespaces(\"ns-1\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"deployments.apps\",\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"ns-2\", \"deploy-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t\ttest.Deployments(),\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\"},\n\t\t\t\ttest.Deployments(): {\"ns-1/deploy-1\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"excluded namespaces filter only restores resources not in those namespaces\",\n\t\t\trestore: defaultRestore().ExcludedNamespaces(\"ns-2\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"deployments.apps\",\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"ns-2\", \"deploy-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t\ttest.Deployments(),\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\"},\n\t\t\t\ttest.Deployments(): {\"ns-1/deploy-1\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"IncludeClusterResources=false only restores namespaced resources\",\n\t\t\trestore: defaultRestore().IncludeClusterResources(false).Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"deployments.apps\",\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"ns-2\", \"deploy-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t\ttest.Deployments(),\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\", \"ns-2/pod-2\"},\n\t\t\t\ttest.Deployments(): {\"ns-1/deploy-1\", \"ns-2/deploy-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"label selector only restores matching resources\",\n\t\t\trestore: defaultRestore().LabelSelector(&metav1.LabelSelector{MatchLabels: map[string]string{\"a\": \"b\"}}).Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").ObjectMeta(builder.WithLabels(\"a\", \"b\")).Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"deployments.apps\",\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"ns-2\", \"deploy-2\").ObjectMeta(builder.WithLabels(\"a\", \"b\")).Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ObjectMeta(builder.WithLabels(\"a\", \"b\")).Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").ObjectMeta(builder.WithLabels(\"a\", \"c\")).Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t\ttest.Deployments(),\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\"},\n\t\t\t\ttest.Deployments(): {\"ns-2/deploy-2\"},\n\t\t\t\ttest.PVs(): {\"/pv-1\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"OrLabelSelectors only restores matching resources\",\n\t\t\trestore: defaultRestore().OrLabelSelector([]*metav1.LabelSelector{{MatchLabels: map[string]string{\"a1\": \"b1\"}}, {MatchLabels: map[string]string{\"a2\": \"b2\"}},\n\t\t\t\t{MatchLabels: map[string]string{\"a3\": \"b3\"}}, {MatchLabels: map[string]string{\"a4\": \"b4\"}}}).Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").ObjectMeta(builder.WithLabels(\"a1\", \"b1\")).Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"deployments.apps\",\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"ns-2\", \"deploy-2\").ObjectMeta(builder.WithLabels(\"a3\", \"b3\")).Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ObjectMeta(builder.WithLabels(\"a5\", \"b5\")).Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").ObjectMeta(builder.WithLabels(\"a4\", \"b4\")).Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t\ttest.Deployments(),\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\"},\n\t\t\t\ttest.Deployments(): {\"ns-2/deploy-2\"},\n\t\t\t\ttest.PVs(): {\"/pv-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should include cluster-scoped resources if restoring subset of namespaces and IncludeClusterResources=true\",\n\t\t\trestore: defaultRestore().IncludedNamespaces(\"ns-1\").IncludeClusterResources(true).Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"deployments.apps\",\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"ns-2\", \"deploy-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t\ttest.Deployments(),\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\"},\n\t\t\t\ttest.Deployments(): {\"ns-1/deploy-1\"},\n\t\t\t\ttest.PVs(): {\"/pv-1\", \"/pv-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should not include cluster-scoped resources if restoring subset of namespaces and IncludeClusterResources=false\",\n\t\t\trestore: defaultRestore().IncludedNamespaces(\"ns-1\").IncludeClusterResources(false).Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"deployments.apps\",\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"ns-2\", \"deploy-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t\ttest.Deployments(),\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\"},\n\t\t\t\ttest.Deployments(): {\"ns-1/deploy-1\"},\n\t\t\t\ttest.PVs(): {},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should not include cluster-scoped resources if restoring subset of namespaces and IncludeClusterResources=nil\",\n\t\t\trestore: defaultRestore().IncludedNamespaces(\"ns-1\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"deployments.apps\",\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"ns-2\", \"deploy-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t\ttest.Deployments(),\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\"},\n\t\t\t\ttest.Deployments(): {\"ns-1/deploy-1\"},\n\t\t\t\ttest.PVs(): {},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should include cluster-scoped resources if restoring all namespaces and IncludeClusterResources=true\",\n\t\t\trestore: defaultRestore().IncludeClusterResources(true).Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"deployments.apps\",\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"ns-2\", \"deploy-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t\ttest.Deployments(),\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\", \"ns-2/pod-2\"},\n\t\t\t\ttest.Deployments(): {\"ns-1/deploy-1\", \"ns-2/deploy-2\"},\n\t\t\t\ttest.PVs(): {\"/pv-1\", \"/pv-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should not include cluster-scoped resources if restoring all namespaces and IncludeClusterResources=false\",\n\t\t\trestore: defaultRestore().IncludeClusterResources(false).Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"deployments.apps\",\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"ns-2\", \"deploy-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t\ttest.Deployments(),\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\", \"ns-2/pod-2\"},\n\t\t\t\ttest.Deployments(): {\"ns-1/deploy-1\", \"ns-2/deploy-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when a wildcard and a specific resource are included, the wildcard takes precedence\",\n\t\t\trestore: defaultRestore().IncludedResources(\"*\", \"pods\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"deployments.apps\",\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"ns-2\", \"deploy-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t\ttest.Deployments(),\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\", \"ns-2/pod-2\"},\n\t\t\t\ttest.Deployments(): {\"ns-1/deploy-1\", \"ns-2/deploy-2\"},\n\t\t\t\ttest.PVs(): {\"/pv-1\", \"/pv-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard excludes are ignored\",\n\t\t\trestore: defaultRestore().ExcludedResources(\"*\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"deployments.apps\",\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"ns-2\", \"deploy-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t\ttest.Deployments(),\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\", \"ns-2/pod-2\"},\n\t\t\t\ttest.Deployments(): {\"ns-1/deploy-1\", \"ns-2/deploy-2\"},\n\t\t\t\ttest.PVs(): {\"/pv-1\", \"/pv-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"unresolvable included resources are ignored\",\n\t\t\trestore: defaultRestore().IncludedResources(\"pods\", \"unresolvable\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"deployments.apps\",\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"ns-2\", \"deploy-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t\ttest.Deployments(),\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\", \"ns-2/pod-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"unresolvable excluded resources are ignored\",\n\t\t\trestore: defaultRestore().ExcludedResources(\"deployments\", \"unresolvable\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"deployments.apps\",\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"ns-2\", \"deploy-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t\ttest.Deployments(),\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\", \"ns-2/pod-2\"},\n\t\t\t\ttest.PVs(): {\"/pv-1\", \"/pv-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"mirror pods are not restored\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).AddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").ObjectMeta(builder.WithAnnotations(corev1api.MirrorPodAnnotationKey, \"foo\")).Result()).Done(),\n\t\t\tapiResources: []*test.APIResource{test.Pods()},\n\t\t\twant: map[*test.APIResource][]string{test.Pods(): {}},\n\t\t},\n\t\t{\n\t\t\tname: \"service accounts are restored\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).AddItems(\"serviceaccounts\", builder.ForServiceAccount(\"ns-1\", \"sa-1\").Result()).Done(),\n\t\t\tapiResources: []*test.APIResource{test.ServiceAccounts()},\n\t\t\twant: map[*test.APIResource][]string{test.ServiceAccounts(): {\"ns-1/sa-1\"}},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\th := newHarness(t)\n\n\t\t\tfor _, r := range tc.apiResources {\n\t\t\t\th.DiscoveryClient.WithAPIResource(r)\n\t\t\t}\n\t\t\trequire.NoError(t, h.restorer.discoveryHelper.Refresh())\n\n\t\t\tdata := &Request{\n\t\t\t\tLog: h.log,\n\t\t\t\tRestore: tc.restore,\n\t\t\t\tBackup: tc.backup,\n\t\t\t\tPodVolumeBackups: nil,\n\t\t\t\tVolumeSnapshots: nil,\n\t\t\t\tBackupReader: tc.tarball,\n\t\t\t}\n\t\t\twarnings, errs := h.restorer.Restore(\n\t\t\t\tdata,\n\t\t\t\tnil, // restoreItemActions\n\t\t\t\tnil, // volume snapshotter getter\n\t\t\t)\n\n\t\t\tassertEmptyResults(t, warnings, errs)\n\t\t\tassertAPIContents(t, h, tc.want)\n\t\t})\n\t}\n}\n\n// TestRestoreNamespaceMapping runs restores with namespace mappings specified,\n// and verifies that the set of items created in the API are in the correct\n// namespaces. Validation is done by looking at the namespaces/names of the items\n// in the API; contents are not checked.\nfunc TestRestoreNamespaceMapping(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\trestore *velerov1api.Restore\n\t\tbackup *velerov1api.Backup\n\t\tapiResources []*test.APIResource\n\t\ttarball io.Reader\n\t\twant map[*test.APIResource][]string\n\t}{\n\t\t{\n\t\t\tname: \"namespace mappings are applied\",\n\t\t\trestore: defaultRestore().NamespaceMappings(\"ns-1\", \"mapped-ns-1\", \"ns-2\", \"mapped-ns-2\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-3\", \"pod-3\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"mapped-ns-1/pod-1\", \"mapped-ns-2/pod-2\", \"ns-3/pod-3\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"namespace mappings are applied when IncludedNamespaces are specified\",\n\t\t\trestore: defaultRestore().IncludedNamespaces(\"ns-1\", \"ns-2\").NamespaceMappings(\"ns-1\", \"mapped-ns-1\", \"ns-2\", \"mapped-ns-2\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-3\", \"pod-3\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"mapped-ns-1/pod-1\", \"mapped-ns-2/pod-2\"},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\th := newHarness(t)\n\n\t\t\tfor _, r := range tc.apiResources {\n\t\t\t\th.DiscoveryClient.WithAPIResource(r)\n\t\t\t}\n\t\t\trequire.NoError(t, h.restorer.discoveryHelper.Refresh())\n\n\t\t\tdata := &Request{\n\t\t\t\tLog: h.log,\n\t\t\t\tRestore: tc.restore,\n\t\t\t\tBackup: tc.backup,\n\t\t\t\tPodVolumeBackups: nil,\n\t\t\t\tVolumeSnapshots: nil,\n\t\t\t\tBackupReader: tc.tarball,\n\t\t\t}\n\t\t\twarnings, errs := h.restorer.Restore(\n\t\t\t\tdata,\n\t\t\t\tnil, // restoreItemActions\n\t\t\t\tnil, // volume snapshotter getter\n\t\t\t)\n\n\t\t\tassertEmptyResults(t, warnings, errs)\n\t\t\tassertAPIContents(t, h, tc.want)\n\t\t})\n\t}\n}\n\n// TestRestoreResourcePriorities runs restores with resource priorities specified,\n// and verifies that the set of items created in the API are created in the expected\n// order. Validation is done by adding a Reactor to the fake dynamic client that records\n// resource identifiers as they're created, and comparing that to the expected order.\nfunc TestRestoreResourcePriorities(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\trestore *velerov1api.Restore\n\t\tbackup *velerov1api.Backup\n\t\tapiResources []*test.APIResource\n\t\ttarball io.Reader\n\t\tresourcePriorities types.Priorities\n\t}{\n\t\t{\n\t\t\tname: \"resources are restored according to the specified resource priorities\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"deployments.apps\",\n\t\t\t\t\tbuilder.ForDeployment(\"ns-1\", \"deploy-1\").Result(),\n\t\t\t\t\tbuilder.ForDeployment(\"ns-2\", \"deploy-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"serviceaccounts\",\n\t\t\t\t\tbuilder.ForServiceAccount(\"ns-1\", \"sa-1\").Result(),\n\t\t\t\t\tbuilder.ForServiceAccount(\"ns-2\", \"sa-2\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"persistentvolumeclaims\",\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns-2\", \"pvc-2\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t\ttest.PVs(),\n\t\t\t\ttest.Deployments(),\n\t\t\t\ttest.ServiceAccounts(),\n\t\t\t},\n\t\t\tresourcePriorities: types.Priorities{\n\t\t\t\tHighPriorities: []string{\"persistentvolumes\", \"persistentvolumeclaims\", \"serviceaccounts\"},\n\t\t\t\tLowPriorities: []string{\"deployments.apps\"},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\th := newHarness(t)\n\t\th.restorer.resourcePriorities = tc.resourcePriorities\n\n\t\trecorder := &createRecorder{t: t}\n\t\th.DynamicClient.PrependReactor(\"create\", \"*\", recorder.reactor())\n\n\t\tfor _, r := range tc.apiResources {\n\t\t\th.DiscoveryClient.WithAPIResource(r)\n\t\t}\n\t\trequire.NoError(t, h.restorer.discoveryHelper.Refresh())\n\n\t\tdata := &Request{\n\t\t\tLog: h.log,\n\t\t\tRestore: tc.restore,\n\t\t\tBackup: tc.backup,\n\t\t\tPodVolumeBackups: nil,\n\t\t\tVolumeSnapshots: nil,\n\t\t\tBackupReader: tc.tarball,\n\t\t}\n\t\twarnings, errs := h.restorer.Restore(\n\t\t\tdata,\n\t\t\tnil, // restoreItemActions\n\t\t\tnil, // volume snapshotter getter\n\t\t)\n\n\t\tassertEmptyResults(t, warnings, errs)\n\t\tassertResourceCreationOrder(t, []string{\"persistentvolumes\", \"persistentvolumeclaims\", \"serviceaccounts\", \"pods\", \"deployments.apps\"}, recorder.resources)\n\t}\n}\n\n// TestInvalidTarballContents runs restores for tarballs that are invalid in some way, and\n// verifies that the set of items created in the API and the errors returned are correct.\n// Validation is done by looking at the namespaces/names of the items in the API and the\n// Result objects returned from the restorer.\nfunc TestInvalidTarballContents(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\trestore *velerov1api.Restore\n\t\tbackup *velerov1api.Backup\n\t\tapiResources []*test.APIResource\n\t\ttarball io.Reader\n\t\twant map[*test.APIResource][]string\n\t\twantErrs Result\n\t\twantWarnings Result\n\t}{\n\t\t{\n\t\t\tname: \"empty tarball returns a warning\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tDone(),\n\t\t\twantWarnings: Result{\n\t\t\t\tVelero: []string{archive.ErrNotExist.Error()},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"invalid JSON is reported as an error and restore continues\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAdd(\"resources/pods/namespaces/ns-1/pod-1.json\", []byte(\"invalid JSON\")).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-2\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-2\"},\n\t\t\t},\n\t\t\twantErrs: Result{\n\t\t\t\tNamespaces: map[string][]string{\n\t\t\t\t\t\"ns-1\": {\"error decoding\"},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\th := newHarness(t)\n\n\t\t\tfor _, r := range tc.apiResources {\n\t\t\t\th.DiscoveryClient.WithAPIResource(r)\n\t\t\t}\n\t\t\trequire.NoError(t, h.restorer.discoveryHelper.Refresh())\n\n\t\t\tdata := &Request{\n\t\t\t\tLog: h.log,\n\t\t\t\tRestore: tc.restore,\n\t\t\t\tBackup: tc.backup,\n\t\t\t\tPodVolumeBackups: nil,\n\t\t\t\tVolumeSnapshots: nil,\n\t\t\t\tBackupReader: tc.tarball,\n\t\t\t}\n\t\t\twarnings, errs := h.restorer.Restore(\n\t\t\t\tdata,\n\t\t\t\tnil, // restoreItemActions\n\t\t\t\tnil, // volume snapshotter getter\n\t\t\t)\n\t\t\tassertWantErrsOrWarnings(t, tc.wantWarnings, warnings)\n\t\t\tassertWantErrsOrWarnings(t, tc.wantErrs, errs)\n\t\t\tassertAPIContents(t, h, tc.want)\n\t\t})\n\t}\n}\n\nfunc assertWantErrsOrWarnings(t *testing.T, wantRes Result, res Result) {\n\tt.Helper()\n\tif wantRes.Velero != nil {\n\t\tassert.Len(t, res.Velero, len(wantRes.Velero))\n\t\tfor i := range res.Velero {\n\t\t\tassert.Contains(t, res.Velero[i], wantRes.Velero[i])\n\t\t}\n\t}\n\tif wantRes.Namespaces != nil {\n\t\tassert.Len(t, res.Namespaces, len(wantRes.Namespaces))\n\t\tfor ns := range res.Namespaces {\n\t\t\tassert.Len(t, res.Namespaces[ns], len(wantRes.Namespaces[ns]))\n\t\t\tfor i := range res.Namespaces[ns] {\n\t\t\t\tassert.Contains(t, res.Namespaces[ns][i], wantRes.Namespaces[ns][i])\n\t\t\t}\n\t\t}\n\t}\n\tif wantRes.Cluster != nil {\n\t\tassert.Equal(t, wantRes.Cluster, res.Cluster)\n\t}\n}\n\n// TestRestoreItems runs restores of specific items and validates that they are created\n// with the expected metadata/spec/status in the API.\nfunc TestRestoreItems(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\trestore *velerov1api.Restore\n\t\tbackup *velerov1api.Backup\n\t\tapiResources []*test.APIResource\n\t\ttarball io.Reader\n\t\twant []*test.APIResource\n\t\texpectedRestoreItems map[itemKey]restoredItemStatus\n\t\tdisableInformer bool\n\t}{\n\t\t{\n\t\t\tname: \"metadata uid/resourceVersion/etc. gets removed\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithLabels(\"key-1\", \"val-1\"),\n\t\t\t\t\t\t\tbuilder.WithAnnotations(\"key-1\", \"val-1\"),\n\t\t\t\t\t\t\tbuilder.WithFinalizers(\"finalizer-1\"),\n\t\t\t\t\t\t\tbuilder.WithUID(\"uid\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tResult(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithLabels(\"key-1\", \"val-1\", \"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\"),\n\t\t\t\t\t\t\tbuilder.WithAnnotations(\"key-1\", \"val-1\"),\n\t\t\t\t\t\t\tbuilder.WithFinalizers(\"finalizer-1\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t},\n\t\t\texpectedRestoreItems: map[itemKey]restoredItemStatus{\n\t\t\t\t{resource: \"v1/Namespace\", namespace: \"\", name: \"ns-1\"}: {action: \"created\", itemExists: true, createdName: \"ns-1\"},\n\t\t\t\t{resource: \"v1/Pod\", namespace: \"ns-1\", name: \"pod-1\"}: {action: \"created\", itemExists: true, createdName: \"pod-1\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"status gets removed\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\t&corev1api.Pod{\n\t\t\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\t\t\tAPIVersion: \"v1\",\n\t\t\t\t\t\t\tKind: \"Pod\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pod-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\t\t\tMessage: \"a non-empty status\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").ObjectMeta(builder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\")).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"object gets labeled with full backup and restore names when they're both shorter than 63 characters\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.Pods(builder.ForPod(\"ns-1\", \"pod-1\").ObjectMeta(builder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\")).Result()),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"object gets labeled with full backup and restore names when they're both equal to 63 characters\",\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"the-really-long-kube-service-name-that-is-exactly-63-characters\").\n\t\t\t\tBackup(\"the-really-long-kube-service-name-that-is-exactly-63-characters\").\n\t\t\t\tResult(),\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"the-really-long-kube-service-name-that-is-exactly-63-characters\").Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithLabels(\n\t\t\t\t\t\t\t\t\"velero.io/backup-name\", \"the-really-long-kube-service-name-that-is-exactly-63-characters\",\n\t\t\t\t\t\t\t\t\"velero.io/restore-name\", \"the-really-long-kube-service-name-that-is-exactly-63-characters\",\n\t\t\t\t\t\t\t),\n\t\t\t\t\t\t).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"object gets labeled with shortened backup and restore names when they're both longer than 63 characters\",\n\t\t\trestore: builder.ForRestore(velerov1api.DefaultNamespace, \"the-really-long-kube-service-name-that-is-much-greater-than-63-characters\").\n\t\t\t\tBackup(\"the-really-long-kube-service-name-that-is-much-greater-than-63-characters\").\n\t\t\t\tResult(),\n\t\t\tbackup: builder.ForBackup(velerov1api.DefaultNamespace, \"the-really-long-kube-service-name-that-is-much-greater-than-63-characters\").Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.Pods(builder.ForPod(\"ns-1\", \"pod-1\").\n\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\tbuilder.WithLabels(\n\t\t\t\t\t\t\t\"velero.io/backup-name\", \"the-really-long-kube-service-name-that-is-much-greater-th8a11b3\",\n\t\t\t\t\t\t\t\"velero.io/restore-name\", \"the-really-long-kube-service-name-that-is-much-greater-th8a11b3\",\n\t\t\t\t\t\t),\n\t\t\t\t\t).\n\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"no error when service account already exists in cluster and is identical to the backed up one\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"serviceaccounts\", builder.ForServiceAccount(\"ns-1\", \"sa-1\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.ServiceAccounts(builder.ForServiceAccount(\"ns-1\", \"sa-1\").Result()),\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.ServiceAccounts(builder.ForServiceAccount(\"ns-1\", \"sa-1\").Result()),\n\t\t\t},\n\t\t\texpectedRestoreItems: map[itemKey]restoredItemStatus{\n\t\t\t\t{resource: \"v1/Namespace\", namespace: \"\", name: \"ns-1\"}: {action: \"created\", itemExists: true, createdName: \"ns-1\"},\n\t\t\t\t{resource: \"v1/ServiceAccount\", namespace: \"ns-1\", name: \"sa-1\"}: {action: \"skipped\", itemExists: true},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"update secret data and labels when secret exists in cluster and is not identical to the backed up one, existing resource policy is update\",\n\t\t\trestore: defaultRestore().ExistingResourcePolicy(\"update\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"secrets\", builder.ForSecret(\"ns-1\", \"sa-1\").Data(map[string][]byte{\"key-1\": []byte(\"value-1\")}).Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Secrets(builder.ForSecret(\"ns-1\", \"sa-1\").Data(map[string][]byte{\"foo\": []byte(\"bar\")}).Result()),\n\t\t\t},\n\t\t\tdisableInformer: true,\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.Secrets(builder.ForSecret(\"ns-1\", \"sa-1\").ObjectMeta(builder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\")).Data(map[string][]byte{\"key-1\": []byte(\"value-1\")}).Result()),\n\t\t\t},\n\t\t\texpectedRestoreItems: map[itemKey]restoredItemStatus{\n\t\t\t\t{resource: \"v1/Namespace\", namespace: \"\", name: \"ns-1\"}: {action: \"created\", itemExists: true, createdName: \"ns-1\"},\n\t\t\t\t{resource: \"v1/Secret\", namespace: \"ns-1\", name: \"sa-1\"}: {action: \"updated\", itemExists: true},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"update secret data and labels when secret exists in cluster and is not identical to the backed up one, existing resource policy is update, using informer cache\",\n\t\t\trestore: defaultRestore().ExistingResourcePolicy(\"update\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"secrets\", builder.ForSecret(\"ns-1\", \"sa-1\").Data(map[string][]byte{\"key-1\": []byte(\"value-1\")}).Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Secrets(builder.ForSecret(\"ns-1\", \"sa-1\").Data(map[string][]byte{\"foo\": []byte(\"bar\")}).Result()),\n\t\t\t},\n\t\t\tdisableInformer: false,\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.Secrets(builder.ForSecret(\"ns-1\", \"sa-1\").ObjectMeta(builder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\")).Data(map[string][]byte{\"key-1\": []byte(\"value-1\")}).Result()),\n\t\t\t},\n\t\t\texpectedRestoreItems: map[itemKey]restoredItemStatus{\n\t\t\t\t{resource: \"v1/Namespace\", namespace: \"\", name: \"ns-1\"}: {action: \"created\", itemExists: true, createdName: \"ns-1\"},\n\t\t\t\t{resource: \"v1/Secret\", namespace: \"ns-1\", name: \"sa-1\"}: {action: \"updated\", itemExists: true},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"update service account labels when service account exists in cluster and is identical to the backed up one, existing resource policy is update\",\n\t\t\trestore: defaultRestore().ExistingResourcePolicy(\"update\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"serviceaccounts\", builder.ForServiceAccount(\"ns-1\", \"sa-1\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.ServiceAccounts(builder.ForServiceAccount(\"ns-1\", \"sa-1\").ObjectMeta(builder.WithLabels(\"velero.io/backup-name\", \"foo\", \"velero.io/restore-name\", \"bar\")).Result()),\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.ServiceAccounts(builder.ForServiceAccount(\"ns-1\", \"sa-1\").ObjectMeta(builder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\")).Result()),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"update pod labels when pod exists in cluster and is identical to the backed up one, existing resource policy is update\",\n\t\t\trestore: defaultRestore().ExistingResourcePolicy(\"update\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"sa-1\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(builder.ForPod(\"ns-1\", \"sa-1\").ObjectMeta(builder.WithLabels(\"velero.io/backup-name\", \"foo\", \"velero.io/restore-name\", \"bar\")).Result()),\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.Pods(builder.ForPod(\"ns-1\", \"sa-1\").ObjectMeta(builder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\")).Result()),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"do not update pod labels when pod exists in cluster and is identical to the backed up one, existing resource policy is none\",\n\t\t\trestore: defaultRestore().ExistingResourcePolicy(\"none\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"sa-1\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(builder.ForPod(\"ns-1\", \"sa-1\").ObjectMeta(builder.WithLabels(\"velero.io/backup-name\", \"foo\", \"velero.io/restore-name\", \"bar\")).Result()),\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.Pods(builder.ForPod(\"ns-1\", \"sa-1\").ObjectMeta(builder.WithLabels(\"velero.io/backup-name\", \"foo\", \"velero.io/restore-name\", \"bar\")).Result()),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"do not update pod labels when pod exists in cluster and is identical to the backed up one, existing resource policy is not specified, velero behavior is preserved\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"sa-1\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(builder.ForPod(\"ns-1\", \"sa-1\").ObjectMeta(builder.WithLabels(\"velero.io/backup-name\", \"foo\", \"velero.io/restore-name\", \"bar\")).Result()),\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.Pods(builder.ForPod(\"ns-1\", \"sa-1\").ObjectMeta(builder.WithLabels(\"velero.io/backup-name\", \"foo\", \"velero.io/restore-name\", \"bar\")).Result()),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"service account secrets and image pull secrets are restored when service account already exists in cluster\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"serviceaccounts\", &corev1api.ServiceAccount{\n\t\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\t\tAPIVersion: \"v1\",\n\t\t\t\t\t\tKind: \"ServiceAccount\",\n\t\t\t\t\t},\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\tName: \"sa-1\",\n\t\t\t\t\t},\n\t\t\t\t\tSecrets: []corev1api.ObjectReference{{Name: \"secret-1\"}},\n\t\t\t\t\tImagePullSecrets: []corev1api.LocalObjectReference{{Name: \"pull-secret-1\"}},\n\t\t\t\t}).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.ServiceAccounts(builder.ForServiceAccount(\"ns-1\", \"sa-1\").Result()),\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.ServiceAccounts(&corev1api.ServiceAccount{\n\t\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\t\tAPIVersion: \"v1\",\n\t\t\t\t\t\tKind: \"ServiceAccount\",\n\t\t\t\t\t},\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\tName: \"sa-1\",\n\t\t\t\t\t},\n\t\t\t\t\tSecrets: []corev1api.ObjectReference{{Name: \"secret-1\"}},\n\t\t\t\t\tImagePullSecrets: []corev1api.LocalObjectReference{{Name: \"pull-secret-1\"}},\n\t\t\t\t}),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"metadata managedFields gets restored\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithManagedFields([]metav1.ManagedFieldsEntry{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tManager: \"kubectl\",\n\t\t\t\t\t\t\t\t\tOperation: \"Apply\",\n\t\t\t\t\t\t\t\t\tAPIVersion: \"v1\",\n\t\t\t\t\t\t\t\t\tFieldsType: \"FieldsV1\",\n\t\t\t\t\t\t\t\t\tFieldsV1: &metav1.FieldsV1{\n\t\t\t\t\t\t\t\t\t\tRaw: []byte(`{\"f:data\": {\"f:key\":{}}}`),\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t}),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tResult(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\"),\n\t\t\t\t\t\t\tbuilder.WithManagedFields([]metav1.ManagedFieldsEntry{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tManager: \"kubectl\",\n\t\t\t\t\t\t\t\t\tOperation: \"Apply\",\n\t\t\t\t\t\t\t\t\tAPIVersion: \"v1\",\n\t\t\t\t\t\t\t\t\tFieldsType: \"FieldsV1\",\n\t\t\t\t\t\t\t\t\tFieldsV1: &metav1.FieldsV1{\n\t\t\t\t\t\t\t\t\t\tRaw: []byte(`{\"f:data\": {\"f:key\":{}}}`),\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t}),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\th := newHarness(t)\n\n\t\t\tfor _, r := range tc.apiResources {\n\t\t\t\th.AddItems(t, r)\n\t\t\t}\n\n\t\t\tdata := &Request{\n\t\t\t\tLog: h.log,\n\t\t\t\tRestore: tc.restore,\n\t\t\t\tBackup: tc.backup,\n\t\t\t\tPodVolumeBackups: nil,\n\t\t\t\tVolumeSnapshots: nil,\n\t\t\t\tBackupReader: tc.tarball,\n\t\t\t\tRestoredItems: map[itemKey]restoredItemStatus{},\n\t\t\t\tDisableInformerCache: tc.disableInformer,\n\t\t\t}\n\t\t\twarnings, errs := h.restorer.Restore(\n\t\t\t\tdata,\n\t\t\t\tnil, // restoreItemActions\n\t\t\t\tnil, // volume snapshotter getter\n\t\t\t)\n\n\t\t\tassertEmptyResults(t, warnings, errs)\n\t\t\tassertRestoredItems(t, h, tc.want)\n\t\t\tif len(tc.expectedRestoreItems) > 0 {\n\t\t\t\tassert.Equal(t, tc.expectedRestoreItems, data.RestoredItems)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// recordResourcesAction is a restore item action that can be configured\n// to run for specific resources/namespaces and simply records the items\n// that it is executed for.\ntype recordResourcesAction struct {\n\tselector velero.ResourceSelector\n\tids []string\n\tadditionalItems []velero.ResourceIdentifier\n\toperationID string\n\twaitForAdditionalItems bool\n\tadditionalItemsReadyTimeout time.Duration\n}\n\nfunc (a *recordResourcesAction) Name() string {\n\treturn \"\"\n}\n\nfunc (a *recordResourcesAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn a.selector, nil\n}\n\nfunc (a *recordResourcesAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\tmetadata, err := meta.Accessor(input.Item)\n\tif err != nil {\n\t\treturn &velero.RestoreItemActionExecuteOutput{\n\t\t\tUpdatedItem: input.Item,\n\t\t\tAdditionalItems: a.additionalItems,\n\t\t\tOperationID: a.operationID,\n\t\t\tWaitForAdditionalItems: a.waitForAdditionalItems,\n\t\t\tAdditionalItemsReadyTimeout: a.additionalItemsReadyTimeout,\n\t\t}, err\n\t}\n\ta.ids = append(a.ids, kubeutil.NamespaceAndName(metadata))\n\n\treturn &velero.RestoreItemActionExecuteOutput{\n\t\tUpdatedItem: input.Item,\n\t\tAdditionalItems: a.additionalItems,\n\t\tOperationID: a.operationID,\n\t\tWaitForAdditionalItems: a.waitForAdditionalItems,\n\t\tAdditionalItemsReadyTimeout: a.additionalItemsReadyTimeout,\n\t}, nil\n}\n\nfunc (a *recordResourcesAction) Progress(operationID string, restore *velerov1api.Restore) (velero.OperationProgress, error) {\n\treturn velero.OperationProgress{}, nil\n}\n\nfunc (a *recordResourcesAction) Cancel(operationID string, restore *velerov1api.Restore) error {\n\treturn nil\n}\n\nfunc (a *recordResourcesAction) AreAdditionalItemsReady(additionalItems []velero.ResourceIdentifier, restore *velerov1api.Restore) (bool, error) {\n\treturn true, nil\n}\n\nfunc (a *recordResourcesAction) ForResource(resource string) *recordResourcesAction {\n\ta.selector.IncludedResources = append(a.selector.IncludedResources, resource)\n\treturn a\n}\n\nfunc (a *recordResourcesAction) ForNamespace(namespace string) *recordResourcesAction {\n\ta.selector.IncludedNamespaces = append(a.selector.IncludedNamespaces, namespace)\n\treturn a\n}\n\nfunc (a *recordResourcesAction) ForLabelSelector(selector string) *recordResourcesAction {\n\ta.selector.LabelSelector = selector\n\treturn a\n}\n\nfunc (a *recordResourcesAction) WithAdditionalItems(items []velero.ResourceIdentifier) *recordResourcesAction {\n\ta.additionalItems = items\n\treturn a\n}\n\n// TestRestoreActionsRunForCorrectItems runs restores with restore item actions, and\n// verifies that each restore item action is run for the correct set of resources based on its\n// AppliesTo() resource selector. Verification is done by using the recordResourcesAction struct,\n// which records which resources it's executed for.\nfunc TestRestoreActionsRunForCorrectItems(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\trestore *velerov1api.Restore\n\t\tbackup *velerov1api.Backup\n\t\tapiResources []*test.APIResource\n\t\ttarball io.Reader\n\t\tactions map[*recordResourcesAction][]string\n\t}{\n\t\t{\n\t\t\tname: \"single action with no selector runs for all items\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result(), builder.ForPod(\"ns-2\", \"pod-2\").Result()).\n\t\t\t\tAddItems(\"persistentvolumes\", builder.ForPersistentVolume(\"pv-1\").Result(), builder.ForPersistentVolume(\"pv-2\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{test.Pods(), test.PVs()},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction): {\"ns-1/pod-1\", \"ns-2/pod-2\", \"pv-1\", \"pv-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"single action with a resource selector for namespaced resources runs only for matching resources\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result(), builder.ForPod(\"ns-2\", \"pod-2\").Result()).\n\t\t\t\tAddItems(\"persistentvolumes\", builder.ForPersistentVolume(\"pv-1\").Result(), builder.ForPersistentVolume(\"pv-2\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{test.Pods(), test.PVs()},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForResource(\"pods\"): {\"ns-1/pod-1\", \"ns-2/pod-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"single action with a resource selector for cluster-scoped resources runs only for matching resources\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result(), builder.ForPod(\"ns-2\", \"pod-2\").Result()).\n\t\t\t\tAddItems(\"persistentvolumes\", builder.ForPersistentVolume(\"pv-1\").Result(), builder.ForPersistentVolume(\"pv-2\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{test.Pods(), test.PVs()},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForResource(\"persistentvolumes\"): {\"pv-1\", \"pv-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"single action with a namespace selector runs only for resources in that namespace\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result(), builder.ForPod(\"ns-2\", \"pod-2\").Result()).\n\t\t\t\tAddItems(\"persistentvolumeclaims\", builder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").Result(), builder.ForPersistentVolumeClaim(\"ns-2\", \"pvc-2\").Result()).\n\t\t\t\tAddItems(\"persistentvolumes\", builder.ForPersistentVolume(\"pv-1\").Result(), builder.ForPersistentVolume(\"pv-2\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{test.Pods(), test.PVCs(), test.PVs()},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForNamespace(\"ns-1\"): {\"ns-1/pod-1\", \"ns-1/pvc-1\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"single action with a resource and namespace selector runs only for matching resources in that namespace\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result(), builder.ForPod(\"ns-2\", \"pod-2\").Result()).\n\t\t\t\tAddItems(\"persistentvolumeclaims\", builder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").Result(), builder.ForPersistentVolumeClaim(\"ns-2\", \"pvc-2\").Result()).\n\t\t\t\tAddItems(\"persistentvolumes\", builder.ForPersistentVolume(\"pv-1\").Result(), builder.ForPersistentVolume(\"pv-2\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{test.Pods(), test.PVCs(), test.PVs()},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForNamespace(\"ns-1\").ForResource(\"pods\"): {\"ns-1/pod-1\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"single action with a resource and label selector runs only for resources matching that label\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\",\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").ObjectMeta(builder.WithLabels(\"restore-resource\", \"true\")).Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-2\").ObjectMeta(builder.WithLabels(\"do-not-restore-resource\", \"true\")).Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-1\").Result(),\n\t\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-2\").ObjectMeta(builder.WithLabels(\"restore-resource\")).Result(),\n\t\t\t\t).Done(),\n\t\t\tapiResources: []*test.APIResource{test.Pods()},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForResource(\"pods\").ForLabelSelector(\"restore-resource\"): {\"ns-1/pod-1\", \"ns-2/pod-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"multiple actions, each with a different resource selector using short name, run for matching resources\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result(), builder.ForPod(\"ns-2\", \"pod-2\").Result()).\n\t\t\t\tAddItems(\"persistentvolumeclaims\", builder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").Result(), builder.ForPersistentVolumeClaim(\"ns-2\", \"pvc-2\").Result()).\n\t\t\t\tAddItems(\"persistentvolumes\", builder.ForPersistentVolume(\"pv-1\").Result(), builder.ForPersistentVolume(\"pv-2\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{test.Pods(), test.PVCs(), test.PVs()},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForResource(\"po\"): {\"ns-1/pod-1\", \"ns-2/pod-2\"},\n\t\t\t\tnew(recordResourcesAction).ForResource(\"pv\"): {\"pv-1\", \"pv-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"actions with selectors that don't match anything don't run for any resources\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result()).\n\t\t\t\tAddItems(\"persistentvolumeclaims\", builder.ForPersistentVolumeClaim(\"ns-2\", \"pvc-2\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{test.Pods(), test.PVCs(), test.PVs()},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForNamespace(\"ns-1\").ForResource(\"persistentvolumeclaims\"): nil,\n\t\t\t\tnew(recordResourcesAction).ForNamespace(\"ns-2\").ForResource(\"pods\"): nil,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"actions run for datauploads resource\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"datauploads.velero.io\", builder.ForDataUpload(\"velero\", \"du\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{test.DataUploads()},\n\t\t\tactions: map[*recordResourcesAction][]string{\n\t\t\t\tnew(recordResourcesAction).ForNamespace(\"velero\").ForResource(\"datauploads.velero.io\"): {\"velero/du\"},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\th := newHarness(t)\n\n\t\t\tfor _, r := range tc.apiResources {\n\t\t\t\th.AddItems(t, r)\n\t\t\t}\n\n\t\t\tactions := []riav2.RestoreItemAction{}\n\t\t\tfor action := range tc.actions {\n\t\t\t\tactions = append(actions, action)\n\t\t\t}\n\n\t\t\tdata := &Request{\n\t\t\t\tLog: h.log,\n\t\t\t\tRestore: tc.restore,\n\t\t\t\tBackup: tc.backup,\n\t\t\t\tPodVolumeBackups: nil,\n\t\t\t\tVolumeSnapshots: nil,\n\t\t\t\tBackupReader: tc.tarball,\n\t\t\t}\n\t\t\twarnings, errs := h.restorer.Restore(\n\t\t\t\tdata,\n\t\t\t\tactions,\n\t\t\t\tnil, // volume snapshotter getter\n\t\t\t)\n\n\t\t\tassertEmptyResults(t, warnings, errs)\n\n\t\t\tfor action, want := range tc.actions {\n\t\t\t\tsort.Strings(want)\n\t\t\t\tsort.Strings(action.ids)\n\t\t\t\tassert.Equal(t, want, action.ids)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// pluggableAction is a restore item action that can be plugged with an Execute\n// function body at runtime.\ntype pluggableAction struct {\n\tselector velero.ResourceSelector\n\texecuteFunc func(*velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error)\n\tprogressFunc func(string, *velerov1api.Restore) (velero.OperationProgress, error)\n}\n\nfunc (a *pluggableAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\tif a.executeFunc == nil {\n\t\treturn &velero.RestoreItemActionExecuteOutput{\n\t\t\tUpdatedItem: input.Item,\n\t\t}, nil\n\t}\n\n\treturn a.executeFunc(input)\n}\n\nfunc (a *pluggableAction) Name() string {\n\treturn \"\"\n}\n\nfunc (a *pluggableAction) AppliesTo() (velero.ResourceSelector, error) {\n\treturn a.selector, nil\n}\n\nfunc (a *pluggableAction) Progress(operationID string, restore *velerov1api.Restore) (velero.OperationProgress, error) {\n\treturn velero.OperationProgress{}, nil\n}\n\nfunc (a *pluggableAction) Cancel(operationID string, restore *velerov1api.Restore) error {\n\treturn nil\n}\n\nfunc (a *pluggableAction) addSelector(selector velero.ResourceSelector) *pluggableAction {\n\ta.selector = selector\n\treturn a\n}\n\nfunc (a *pluggableAction) AreAdditionalItemsReady(additionalItems []velero.ResourceIdentifier, restore *velerov1api.Restore) (bool, error) {\n\treturn true, nil\n}\n\n// TestRestoreActionModifications runs restores with restore item actions that modify resources, and\n// verifies that the modified item is correctly created in the API. Verification is done by looking\n// at the full object in the API.\nfunc TestRestoreActionModifications(t *testing.T) {\n\t// modifyingActionGetter is a helper function that returns a *pluggableAction, whose Execute(...)\n\t// method modifies the item being passed in by calling the 'modify' function on it.\n\tmodifyingActionGetter := func(modify func(*unstructured.Unstructured)) *pluggableAction {\n\t\treturn &pluggableAction{\n\t\t\texecuteFunc: func(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\t\t\t\tobj, ok := input.Item.(*unstructured.Unstructured)\n\t\t\t\tif !ok {\n\t\t\t\t\treturn nil, errors.Errorf(\"unexpected type %T\", input.Item)\n\t\t\t\t}\n\n\t\t\t\tres := obj.DeepCopy()\n\t\t\t\tmodify(res)\n\n\t\t\t\treturn &velero.RestoreItemActionExecuteOutput{\n\t\t\t\t\tUpdatedItem: res,\n\t\t\t\t}, nil\n\t\t\t},\n\t\t}\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\trestore *velerov1api.Restore\n\t\tbackup *velerov1api.Backup\n\t\tapiResources []*test.APIResource\n\t\ttarball io.Reader\n\t\tactions []riav2.RestoreItemAction\n\t\twant []*test.APIResource\n\t}{\n\t\t{\n\t\t\tname: \"action that adds a label to item gets restored\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).AddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result()).Done(),\n\t\t\tapiResources: []*test.APIResource{test.Pods()},\n\t\t\tactions: []riav2.RestoreItemAction{\n\t\t\t\tmodifyingActionGetter(func(item *unstructured.Unstructured) {\n\t\t\t\t\titem.SetLabels(map[string]string{\"updated\": \"true\"})\n\t\t\t\t}),\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.Pods(\n\t\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-1\").ObjectMeta(builder.WithLabels(\"updated\", \"true\")).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"action that removes a label to item gets restored\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).AddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").ObjectMeta(builder.WithLabels(\"should-be-removed\", \"true\")).Result()).Done(),\n\t\t\tapiResources: []*test.APIResource{test.Pods()},\n\t\t\tactions: []riav2.RestoreItemAction{\n\t\t\t\tmodifyingActionGetter(func(item *unstructured.Unstructured) {\n\t\t\t\t\titem.SetLabels(nil)\n\t\t\t\t}),\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.Pods(builder.ForPod(\"ns-1\", \"pod-1\").Result()),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"action with non-matching label selector doesn't prevent restore\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).AddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result()).Done(),\n\t\t\tapiResources: []*test.APIResource{test.Pods()},\n\t\t\tactions: []riav2.RestoreItemAction{\n\t\t\t\tmodifyingActionGetter(func(item *unstructured.Unstructured) {\n\t\t\t\t\titem.SetLabels(map[string]string{\"updated\": \"true\"})\n\t\t\t\t}).addSelector(velero.ResourceSelector{\n\t\t\t\t\tIncludedResources: []string{\n\t\t\t\t\t\t\"Pod\",\n\t\t\t\t\t},\n\t\t\t\t\tLabelSelector: \"nonmatching=label\",\n\t\t\t\t}),\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.Pods(builder.ForPod(\"ns-1\", \"pod-1\").Result()),\n\t\t\t},\n\t\t},\n\t\t// TODO action that modifies namespace/name - what's the expected behavior?\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\th := newHarness(t)\n\n\t\t\tfor _, r := range tc.apiResources {\n\t\t\t\th.AddItems(t, r)\n\t\t\t}\n\n\t\t\t// every restored item should have the restore and backup name labels, set\n\t\t\t// them here so we don't have to do it in every test case definition above.\n\t\t\tfor _, resource := range tc.want {\n\t\t\t\tfor _, item := range resource.Items {\n\t\t\t\t\tlabels := item.GetLabels()\n\t\t\t\t\tif labels == nil {\n\t\t\t\t\t\tlabels = make(map[string]string)\n\t\t\t\t\t}\n\n\t\t\t\t\tlabels[\"velero.io/restore-name\"] = tc.restore.Name\n\t\t\t\t\tlabels[\"velero.io/backup-name\"] = tc.restore.Spec.BackupName\n\n\t\t\t\t\titem.SetLabels(labels)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tdata := &Request{\n\t\t\t\tLog: h.log,\n\t\t\t\tRestore: tc.restore,\n\t\t\t\tBackup: tc.backup,\n\t\t\t\tPodVolumeBackups: nil,\n\t\t\t\tVolumeSnapshots: nil,\n\t\t\t\tBackupReader: tc.tarball,\n\t\t\t}\n\t\t\twarnings, errs := h.restorer.Restore(\n\t\t\t\tdata,\n\t\t\t\ttc.actions,\n\t\t\t\tnil, // volume snapshotter getter\n\t\t\t)\n\n\t\t\tassertEmptyResults(t, warnings, errs)\n\t\t\tassertRestoredItems(t, h, tc.want)\n\t\t})\n\t}\n}\n\n// TestRestoreWithAsyncOperations runs restores which return operationIDs and\n// verifies that the itemoperations are tracked as appropriate. Verification is done by\n// looking at the restore request's itemOperationsList field.\nfunc TestRestoreWithAsyncOperations(t *testing.T) {\n\t// completedOperationAction is a *pluggableAction, whose Execute(...)\n\t// method returns an operationID which will always be done when calling Progress.\n\tcompletedOperationAction := &pluggableAction{\n\t\texecuteFunc: func(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\t\t\tobj, ok := input.Item.(*unstructured.Unstructured)\n\t\t\tif !ok {\n\t\t\t\treturn nil, errors.Errorf(\"unexpected type %T\", input.Item)\n\t\t\t}\n\t\t\treturn &velero.RestoreItemActionExecuteOutput{\n\t\t\t\tUpdatedItem: obj,\n\t\t\t\tOperationID: obj.GetName() + \"-1\",\n\t\t\t}, nil\n\t\t},\n\t\tprogressFunc: func(operationID string, restore *velerov1api.Restore) (velero.OperationProgress, error) {\n\t\t\treturn velero.OperationProgress{\n\t\t\t\tCompleted: true,\n\t\t\t\tDescription: \"Done!\",\n\t\t\t}, nil\n\t\t},\n\t}\n\n\t// incompleteOperationAction is a *pluggableAction, whose Execute(...)\n\t// method returns an operationID which will never be done when calling Progress.\n\tincompleteOperationAction := &pluggableAction{\n\t\texecuteFunc: func(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\t\t\tobj, ok := input.Item.(*unstructured.Unstructured)\n\t\t\tif !ok {\n\t\t\t\treturn nil, errors.Errorf(\"unexpected type %T\", input.Item)\n\t\t\t}\n\t\t\treturn &velero.RestoreItemActionExecuteOutput{\n\t\t\t\tUpdatedItem: obj,\n\t\t\t\tOperationID: obj.GetName() + \"-1\",\n\t\t\t}, nil\n\t\t},\n\t\tprogressFunc: func(operationID string, restore *velerov1api.Restore) (velero.OperationProgress, error) {\n\t\t\treturn velero.OperationProgress{\n\t\t\t\tCompleted: false,\n\t\t\t\tDescription: \"Working...\",\n\t\t\t}, nil\n\t\t},\n\t}\n\n\t// noOperationAction is a *pluggableAction, whose Execute(...)\n\t// method does not return an operationID.\n\tnoOperationAction := &pluggableAction{\n\t\texecuteFunc: func(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\t\t\tobj, ok := input.Item.(*unstructured.Unstructured)\n\t\t\tif !ok {\n\t\t\t\treturn nil, errors.Errorf(\"unexpected type %T\", input.Item)\n\t\t\t}\n\t\t\treturn &velero.RestoreItemActionExecuteOutput{\n\t\t\t\tUpdatedItem: obj,\n\t\t\t}, nil\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\trestore *velerov1api.Restore\n\t\tbackup *velerov1api.Backup\n\t\tapiResources []*test.APIResource\n\t\ttarball io.Reader\n\t\tactions []riav2.RestoreItemAction\n\t\twant []*itemoperation.RestoreOperation\n\t}{\n\t\t{\n\t\t\tname: \"action that starts a short-running process records operation\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{test.Pods()},\n\t\t\ttarball: test.NewTarWriter(t).AddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result()).Done(),\n\t\t\tactions: []riav2.RestoreItemAction{\n\t\t\t\tcompletedOperationAction,\n\t\t\t},\n\t\t\twant: []*itemoperation.RestoreOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.RestoreOperationSpec{\n\t\t\t\t\t\tRestoreName: \"restore-1\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pod-1\"},\n\t\t\t\t\t\tOperationID: \"pod-1-1\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: \"New\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"action that starts a long-running process records operation\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{test.Pods()},\n\t\t\ttarball: test.NewTarWriter(t).AddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-2\").Result()).Done(),\n\t\t\tactions: []riav2.RestoreItemAction{\n\t\t\t\tincompleteOperationAction,\n\t\t\t},\n\t\t\twant: []*itemoperation.RestoreOperation{\n\t\t\t\t{\n\t\t\t\t\tSpec: itemoperation.RestoreOperationSpec{\n\t\t\t\t\t\tRestoreName: \"restore-1\",\n\t\t\t\t\t\tResourceIdentifier: velero.ResourceIdentifier{\n\t\t\t\t\t\t\tGroupResource: kuberesource.Pods,\n\t\t\t\t\t\t\tNamespace: \"ns-1\",\n\t\t\t\t\t\t\tName: \"pod-2\"},\n\t\t\t\t\t\tOperationID: \"pod-2-1\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: itemoperation.OperationStatus{\n\t\t\t\t\t\tPhase: \"New\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"action that has no operation doesn't record one\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{test.Pods()},\n\t\t\ttarball: test.NewTarWriter(t).AddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-3\").Result()).Done(),\n\t\t\tactions: []riav2.RestoreItemAction{\n\t\t\t\tnoOperationAction,\n\t\t\t},\n\t\t\twant: []*itemoperation.RestoreOperation{},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\th := newHarness(t)\n\n\t\t\tfor _, r := range tc.apiResources {\n\t\t\t\th.AddItems(t, r)\n\t\t\t}\n\n\t\t\tdata := &Request{\n\t\t\t\tLog: h.log,\n\t\t\t\tRestore: tc.restore,\n\t\t\t\tBackup: tc.backup,\n\t\t\t\tPodVolumeBackups: nil,\n\t\t\t\tVolumeSnapshots: nil,\n\t\t\t\tBackupReader: tc.tarball,\n\t\t\t}\n\t\t\twarnings, errs := h.restorer.Restore(\n\t\t\t\tdata,\n\t\t\t\ttc.actions,\n\t\t\t\tnil, // volume snapshotter getter\n\t\t\t)\n\n\t\t\tassertEmptyResults(t, warnings, errs)\n\t\t\tresultOper := *data.GetItemOperationsList()\n\t\t\t// set want Created times so it won't fail the assert.Equal test\n\t\t\tfor i, wantOper := range tc.want {\n\t\t\t\twantOper.Status.Created = resultOper[i].Status.Created\n\t\t\t}\n\t\t\tassert.Equal(t, tc.want, *data.GetItemOperationsList())\n\t\t})\n\t}\n}\n\n// TestRestoreActionAdditionalItems runs restores with restore item actions that return additional items\n// to be restored, and verifies that the correct set of items is created in the API. Verification is\n// done by looking at the namespaces/names of the items in the API; contents are not checked.\nfunc TestRestoreActionAdditionalItems(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\trestore *velerov1api.Restore\n\t\tbackup *velerov1api.Backup\n\t\ttarball io.Reader\n\t\tapiResources []*test.APIResource\n\t\tactions []riav2.RestoreItemAction\n\t\twant map[*test.APIResource][]string\n\t}{\n\t\t{\n\t\t\tname: \"additional items that are already being restored are not restored twice\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).AddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result(), builder.ForPod(\"ns-2\", \"pod-2\").Result()).Done(),\n\t\t\tapiResources: []*test.APIResource{test.Pods()},\n\t\t\tactions: []riav2.RestoreItemAction{\n\t\t\t\t&pluggableAction{\n\t\t\t\t\tselector: velero.ResourceSelector{IncludedNamespaces: []string{\"ns-1\"}},\n\t\t\t\t\texecuteFunc: func(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\t\t\t\t\t\treturn &velero.RestoreItemActionExecuteOutput{\n\t\t\t\t\t\t\tUpdatedItem: input.Item,\n\t\t\t\t\t\t\tAdditionalItems: []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"ns-2\", Name: \"pod-2\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\", \"ns-2/pod-2\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when using a restore namespace filter, additional items that are in a non-included namespace are not restored\",\n\t\t\trestore: defaultRestore().IncludedNamespaces(\"ns-1\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).AddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result(), builder.ForPod(\"ns-2\", \"pod-2\").Result()).Done(),\n\t\t\tapiResources: []*test.APIResource{test.Pods()},\n\t\t\tactions: []riav2.RestoreItemAction{\n\t\t\t\t&pluggableAction{\n\t\t\t\t\texecuteFunc: func(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\t\t\t\t\t\treturn &velero.RestoreItemActionExecuteOutput{\n\t\t\t\t\t\t\tUpdatedItem: input.Item,\n\t\t\t\t\t\t\tAdditionalItems: []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t\t{GroupResource: kuberesource.Pods, Namespace: \"ns-2\", Name: \"pod-2\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when using a restore namespace filter, additional items that are cluster-scoped are restored when IncludeClusterResources=nil\",\n\t\t\trestore: defaultRestore().IncludedNamespaces(\"ns-1\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result()).\n\t\t\t\tAddItems(\"persistentvolumes\", builder.ForPersistentVolume(\"pv-1\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{test.Pods(), test.PVs()},\n\t\t\tactions: []riav2.RestoreItemAction{\n\t\t\t\t&pluggableAction{\n\t\t\t\t\texecuteFunc: func(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\t\t\t\t\t\treturn &velero.RestoreItemActionExecuteOutput{\n\t\t\t\t\t\t\tUpdatedItem: input.Item,\n\t\t\t\t\t\t\tAdditionalItems: []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"pv-1\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\"},\n\t\t\t\ttest.PVs(): {\"/pv-1\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"additional items that are cluster-scoped are not restored when IncludeClusterResources=false\",\n\t\t\trestore: defaultRestore().IncludeClusterResources(false).Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result()).\n\t\t\t\tAddItems(\"persistentvolumes\", builder.ForPersistentVolume(\"pv-1\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{test.Pods(), test.PVs()},\n\t\t\tactions: []riav2.RestoreItemAction{\n\t\t\t\t&pluggableAction{\n\t\t\t\t\texecuteFunc: func(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\t\t\t\t\t\treturn &velero.RestoreItemActionExecuteOutput{\n\t\t\t\t\t\t\tUpdatedItem: input.Item,\n\t\t\t\t\t\t\tAdditionalItems: []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"pv-1\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\"},\n\t\t\t\ttest.PVs(): nil,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when using a restore resource filter, additional items that are non-included resources are not restored\",\n\t\t\trestore: defaultRestore().IncludedResources(\"pods\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"pods\", builder.ForPod(\"ns-1\", \"pod-1\").Result()).\n\t\t\t\tAddItems(\"persistentvolumes\", builder.ForPersistentVolume(\"pv-1\").Result()).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{test.Pods(), test.PVs()},\n\t\t\tactions: []riav2.RestoreItemAction{\n\t\t\t\t&pluggableAction{\n\t\t\t\t\texecuteFunc: func(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {\n\t\t\t\t\t\treturn &velero.RestoreItemActionExecuteOutput{\n\t\t\t\t\t\t\tUpdatedItem: input.Item,\n\t\t\t\t\t\t\tAdditionalItems: []velero.ResourceIdentifier{\n\t\t\t\t\t\t\t\t{GroupResource: kuberesource.PersistentVolumes, Name: \"pv-1\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t}, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-1\"},\n\t\t\t\ttest.PVs(): nil,\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\th := newHarness(t)\n\n\t\t\tfor _, r := range tc.apiResources {\n\t\t\t\th.AddItems(t, r)\n\t\t\t}\n\n\t\t\tdata := &Request{\n\t\t\t\tLog: h.log,\n\t\t\t\tRestore: tc.restore,\n\t\t\t\tBackup: tc.backup,\n\t\t\t\tPodVolumeBackups: nil,\n\t\t\t\tVolumeSnapshots: nil,\n\t\t\t\tBackupReader: tc.tarball,\n\t\t\t}\n\t\t\twarnings, errs := h.restorer.Restore(\n\t\t\t\tdata,\n\t\t\t\ttc.actions,\n\t\t\t\tnil, // volume snapshotter getter\n\t\t\t)\n\n\t\t\tassertEmptyResults(t, warnings, errs)\n\t\t\tassertAPIContents(t, h, tc.want)\n\t\t})\n\t}\n}\n\n// TestShouldRestore runs the ShouldRestore function for various permutations of\n// existing/nonexisting/being-deleted PVs, PVCs, and namespaces, and verifies the\n// result/error matches expectations.\nfunc TestShouldRestore(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvName string\n\t\tapiResources []*test.APIResource\n\t\tnamespaces []*corev1api.Namespace\n\t\twant bool\n\t\twantErr error\n\t}{\n\t\t{\n\t\t\tname: \"when PV is not found, result is true\",\n\t\t\tpvName: \"pv-1\",\n\t\t\twant: true,\n\t\t},\n\t\t{\n\t\t\tname: \"when PV is found and has Phase=Released, result is false\",\n\t\t\tpvName: \"pv-1\",\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(&corev1api.PersistentVolume{\n\t\t\t\t\tTypeMeta: metav1.TypeMeta{\n\t\t\t\t\t\tAPIVersion: \"v1\",\n\t\t\t\t\t\tKind: \"PersistentVolume\",\n\t\t\t\t\t},\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"pv-1\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: corev1api.PersistentVolumeStatus{\n\t\t\t\t\t\tPhase: corev1api.VolumeReleased,\n\t\t\t\t\t},\n\t\t\t\t}),\n\t\t\t},\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"when PV is found and has associated PVC and namespace that aren't deleting, result is false\",\n\t\t\tpvName: \"pv-1\",\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ClaimRef(\"ns-1\", \"pvc-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(builder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").Result()),\n\t\t\t},\n\t\t\tnamespaces: []*corev1api.Namespace{builder.ForNamespace(\"ns-1\").Result()},\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"when PV is found and has associated PVC that is deleting, result is false + timeout error\",\n\t\t\tpvName: \"pv-1\",\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ClaimRef(\"ns-1\", \"pvc-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").ObjectMeta(builder.WithDeletionTimestamp(time.Now())).Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: false,\n\t\t\twantErr: errors.New(\"context deadline exceeded\"),\n\t\t},\n\t\t{\n\t\t\tname: \"when PV is found, has associated PVC that's not deleting, has associated NS that is terminating, result is false + timeout error\",\n\t\t\tpvName: \"pv-1\",\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ClaimRef(\"ns-1\", \"pvc-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(builder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").Result()),\n\t\t\t},\n\t\t\tnamespaces: []*corev1api.Namespace{\n\t\t\t\tbuilder.ForNamespace(\"ns-1\").Phase(corev1api.NamespaceTerminating).Result(),\n\t\t\t},\n\t\t\twant: false,\n\t\t\twantErr: errors.New(\"context deadline exceeded\"),\n\t\t},\n\t\t{\n\t\t\tname: \"when PV is found, has associated PVC that's not deleting, has associated NS that has deletion timestamp, result is false + timeout error\",\n\t\t\tpvName: \"pv-1\",\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ClaimRef(\"ns-1\", \"pvc-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(builder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").Result()),\n\t\t\t},\n\t\t\tnamespaces: []*corev1api.Namespace{\n\t\t\t\tbuilder.ForNamespace(\"ns-1\").ObjectMeta(builder.WithDeletionTimestamp(time.Now())).Result(),\n\t\t\t},\n\t\t\twant: false,\n\t\t\twantErr: errors.New(\"context deadline exceeded\"),\n\t\t},\n\t\t{\n\t\t\tname: \"when PV is found, associated PVC is not found, result is false + timeout error\",\n\t\t\tpvName: \"pv-1\",\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ClaimRef(\"ns-1\", \"pvc-1\").Result(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twant: false,\n\t\t\twantErr: errors.New(\"context deadline exceeded\"),\n\t\t},\n\t\t{\n\t\t\tname: \"when PV is found, has associated PVC, associated namespace not found, result is false + timeout error\",\n\t\t\tpvName: \"pv-1\",\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ClaimRef(\"ns-1\", \"pvc-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(builder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").Result()),\n\t\t\t},\n\t\t\twant: false,\n\t\t\twantErr: errors.New(\"context deadline exceeded\"),\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\th := newHarness(t)\n\n\t\t\tctx := &restoreContext{\n\t\t\t\tlog: h.log,\n\t\t\t\tdynamicFactory: client.NewDynamicFactory(h.DynamicClient),\n\t\t\t\tnamespaceClient: h.KubeClient.CoreV1().Namespaces(),\n\t\t\t\tresourceTerminatingTimeout: time.Millisecond,\n\t\t\t\tresourceDeletionStatusTracker: kube.NewResourceDeletionStatusTracker(),\n\t\t\t}\n\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.AddItems(t, resource)\n\t\t\t}\n\n\t\t\tfor _, ns := range tc.namespaces {\n\t\t\t\t_, err := ctx.namespaceClient.Create(t.Context(), ns, metav1.CreateOptions{})\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tpvClient, err := ctx.dynamicFactory.ClientForGroupVersionResource(\n\t\t\t\tschema.GroupVersion{Group: \"\", Version: \"v1\"},\n\t\t\t\tmetav1.APIResource{Name: \"persistentvolumes\"},\n\t\t\t\t\"\",\n\t\t\t)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tres, err := ctx.shouldRestore(tc.pvName, pvClient)\n\t\t\tassert.Equal(t, tc.want, res)\n\t\t\tif tc.wantErr != nil {\n\t\t\t\tif assert.Error(t, err, \"expected a non-nil error\") {\n\t\t\t\t\tassert.EqualError(t, err, tc.wantErr.Error())\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc assertRestoredItems(t *testing.T, h *harness, want []*test.APIResource) {\n\tt.Helper()\n\n\tfor _, resource := range want {\n\t\tresourceClient := h.DynamicClient.Resource(resource.GVR())\n\t\tfor _, item := range resource.Items {\n\t\t\tvar client dynamic.ResourceInterface\n\t\t\tif item.GetNamespace() != \"\" {\n\t\t\t\tclient = resourceClient.Namespace(item.GetNamespace())\n\t\t\t} else {\n\t\t\t\tclient = resourceClient\n\t\t\t}\n\n\t\t\tres, err := client.Get(t.Context(), item.GetName(), metav1.GetOptions{})\n\t\t\tif !assert.NoError(t, err) { //nolint:testifylint // require is inappropriate\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\titemJSON, err := json.Marshal(item)\n\t\t\tif !assert.NoError(t, err) { //nolint:testifylint // require is inappropriate\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tt.Logf(\"%v\", string(itemJSON))\n\n\t\t\tu := make(map[string]any)\n\t\t\tif !assert.NoError(t, json.Unmarshal(itemJSON, &u)) { //nolint:testifylint // require is inappropriate\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\twant := &unstructured.Unstructured{Object: u}\n\n\t\t\t// These fields get non-nil zero values in the unstructured objects if they're\n\t\t\t// empty in the structured objects. Remove them to make comparison easier.\n\t\t\tunstructured.RemoveNestedField(want.Object, \"metadata\", \"creationTimestamp\")\n\t\t\tunstructured.RemoveNestedField(want.Object, \"status\")\n\t\t\tunstructured.RemoveNestedField(res.Object, \"status\")\n\n\t\t\tassert.Equal(t, want, res)\n\t\t}\n\t}\n}\n\n// volumeSnapshotterGetter is a simple implementation of the VolumeSnapshotterGetter\n// interface that returns vsv1.VolumeSnapshotters from a map if they exist.\ntype volumeSnapshotterGetter map[string]vsv1.VolumeSnapshotter\n\nfunc (vsg volumeSnapshotterGetter) GetVolumeSnapshotter(name string) (vsv1.VolumeSnapshotter, error) {\n\tsnapshotter, ok := vsg[name]\n\tif !ok {\n\t\treturn nil, errors.New(\"volume snapshotter not found\")\n\t}\n\n\treturn snapshotter, nil\n}\n\n// volumeSnapshotter is a test fake for the vsv1.VolumeSnapshotter interface\ntype volumeSnapshotter struct {\n\t// a map from snapshotID to volumeID\n\tsnapshotVolumes map[string]string\n\n\t// a map from volumeID to new pv name\n\tpvName map[string]string\n}\n\n// Init is a no-op.\nfunc (vs *volumeSnapshotter) Init(config map[string]string) error {\n\treturn nil\n}\n\n// CreateVolumeFromSnapshot looks up the specified snapshotID in the snapshotVolumes\n// map and returns the corresponding volumeID if it exists, or an error otherwise.\nfunc (vs *volumeSnapshotter) CreateVolumeFromSnapshot(snapshotID, volumeType, volumeAZ string, iops *int64) (volumeID string, err error) {\n\tvolumeID, ok := vs.snapshotVolumes[snapshotID]\n\tif !ok {\n\t\treturn \"\", errors.New(\"snapshot not found\")\n\t}\n\n\treturn volumeID, nil\n}\n\n// SetVolumeID sets the persistent volume's spec.awsElasticBlockStore.volumeID field\n// with the provided volumeID.\nfunc (vs *volumeSnapshotter) SetVolumeID(pv runtime.Unstructured, volumeID string) (runtime.Unstructured, error) {\n\tunstructured.SetNestedField(pv.UnstructuredContent(), volumeID, \"spec\", \"awsElasticBlockStore\", \"volumeID\")\n\n\tnewPVName, ok := vs.pvName[volumeID]\n\tif !ok {\n\t\treturn pv, nil\n\t}\n\n\tunstructured.SetNestedField(pv.UnstructuredContent(), newPVName, \"metadata\", \"name\")\n\treturn pv, nil\n}\n\n// GetVolumeID panics because it's not expected to be used for restores.\nfunc (*volumeSnapshotter) GetVolumeID(pv runtime.Unstructured) (string, error) {\n\tpanic(\"GetVolumeID should not be used for restores\")\n}\n\n// CreateSnapshot panics because it's not expected to be used for restores.\nfunc (*volumeSnapshotter) CreateSnapshot(volumeID, volumeAZ string, tags map[string]string) (snapshotID string, err error) {\n\tpanic(\"CreateSnapshot should not be used for restores\")\n}\n\n// GetVolumeInfo panics because it's not expected to be used for restores.\nfunc (*volumeSnapshotter) GetVolumeInfo(volumeID, volumeAZ string) (string, *int64, error) {\n\tpanic(\"GetVolumeInfo should not be used for restores\")\n}\n\n// DeleteSnapshot panics because it's not expected to be used for restores.\nfunc (*volumeSnapshotter) DeleteSnapshot(snapshotID string) error {\n\tpanic(\"DeleteSnapshot should not be used for backups\")\n}\n\n// TestRestorePersistentVolumes runs restores for persistent volumes and verifies that\n// they are restored as expected, including restoring volumes from snapshots when expected.\n// Verification is done by looking at the contents of the API and the metadata/spec/status of\n// the items in the API.\nfunc TestRestorePersistentVolumes(t *testing.T) {\n\ttestPVCName := \"testPVC\"\n\ttests := []struct {\n\t\tname string\n\t\trestore *velerov1api.Restore\n\t\tbackup *velerov1api.Backup\n\t\ttarball io.Reader\n\t\tapiResources []*test.APIResource\n\t\tvolumeSnapshots []*volume.Snapshot\n\t\tvolumeSnapshotLocations []*velerov1api.VolumeSnapshotLocation\n\t\tvolumeSnapshotterGetter volumeSnapshotterGetter\n\t\tcsiVolumeSnapshots []*snapshotv1api.VolumeSnapshot\n\t\tdataUploadResult *corev1api.ConfigMap\n\t\twant []*test.APIResource\n\t\twantError bool\n\t\twantWarning bool\n\t\tcsiFeatureVerifierErr string\n\t}{\n\t\t{\n\t\t\tname: \"when a PV with a reclaim policy of delete has no snapshot and does not exist in-cluster, it does not get restored, and its PVC gets reset for dynamic provisioning\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ReclaimPolicy(corev1api.PersistentVolumeReclaimDelete).ClaimRef(\"ns-1\", \"pvc-1\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\"persistentvolumeclaims\",\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").\n\t\t\t\t\t\tVolumeName(\"pv-1\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithAnnotations(\"pv.kubernetes.io/bind-completed\", \"true\", \"pv.kubernetes.io/bound-by-controller\", \"true\", \"foo\", \"bar\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tResult(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t\ttest.PVCs(),\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"ns-1\", \"pvc-1\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithAnnotations(\"foo\", \"bar\"),\n\t\t\t\t\t\t\tbuilder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when a PV with a reclaim policy of retain has no snapshot and does not exist in-cluster, it gets restored, with its claim ref\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).ClaimRef(\"ns-1\", \"pvc-1\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t\ttest.PVCs(),\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").\n\t\t\t\t\t\tReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tClaimRef(\"ns-1\", \"pvc-1\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when a PV with a reclaim policy of delete has a snapshot and does not exist in-cluster, the snapshot and PV are restored\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").ReclaimPolicy(corev1api.PersistentVolumeReclaimDelete).AWSEBSVolumeID(\"old-volume\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t\ttest.PVCs(),\n\t\t\t},\n\t\t\tvolumeSnapshots: []*volume.Snapshot{\n\t\t\t\t{\n\t\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\t\tBackupName: \"backup-1\",\n\t\t\t\t\t\tLocation: \"default\",\n\t\t\t\t\t\tPersistentVolumeName: \"pv-1\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: volume.SnapshotStatus{\n\t\t\t\t\t\tPhase: volume.SnapshotPhaseCompleted,\n\t\t\t\t\t\tProviderSnapshotID: \"snapshot-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshotLocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"default\").Provider(\"provider-1\").Result(),\n\t\t\t},\n\t\t\tvolumeSnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t\"provider-1\": &volumeSnapshotter{\n\t\t\t\t\tsnapshotVolumes: map[string]string{\"snapshot-1\": \"new-volume\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").\n\t\t\t\t\t\tReclaimPolicy(corev1api.PersistentVolumeReclaimDelete).\n\t\t\t\t\t\tAWSEBSVolumeID(\"new-volume\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when a PV with a reclaim policy of retain has a snapshot and does not exist in-cluster, the snapshot and PV are restored\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").\n\t\t\t\t\t\tReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).\n\t\t\t\t\t\tAWSEBSVolumeID(\"old-volume\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t\ttest.PVCs(),\n\t\t\t},\n\t\t\tvolumeSnapshots: []*volume.Snapshot{\n\t\t\t\t{\n\t\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\t\tBackupName: \"backup-1\",\n\t\t\t\t\t\tLocation: \"default\",\n\t\t\t\t\t\tPersistentVolumeName: \"pv-1\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: volume.SnapshotStatus{\n\t\t\t\t\t\tPhase: volume.SnapshotPhaseCompleted,\n\t\t\t\t\t\tProviderSnapshotID: \"snapshot-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshotLocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"default\").Provider(\"provider-1\").Result(),\n\t\t\t},\n\t\t\tvolumeSnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t\"provider-1\": &volumeSnapshotter{\n\t\t\t\t\tsnapshotVolumes: map[string]string{\"snapshot-1\": \"new-volume\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").\n\t\t\t\t\t\tReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).\n\t\t\t\t\t\tAWSEBSVolumeID(\"new-volume\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when a PV with a reclaim policy of delete has a snapshot and exists in-cluster, neither the snapshot nor the PV are restored\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").\n\t\t\t\t\t\tReclaimPolicy(corev1api.PersistentVolumeReclaimDelete).\n\t\t\t\t\t\tAWSEBSVolumeID(\"old-volume\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").\n\t\t\t\t\t\tReclaimPolicy(corev1api.PersistentVolumeReclaimDelete).\n\t\t\t\t\t\tAWSEBSVolumeID(\"old-volume\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(),\n\t\t\t},\n\t\t\tvolumeSnapshots: []*volume.Snapshot{\n\t\t\t\t{\n\t\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\t\tBackupName: \"backup-1\",\n\t\t\t\t\t\tLocation: \"default\",\n\t\t\t\t\t\tPersistentVolumeName: \"pv-1\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: volume.SnapshotStatus{\n\t\t\t\t\t\tPhase: volume.SnapshotPhaseCompleted,\n\t\t\t\t\t\tProviderSnapshotID: \"snapshot-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshotLocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"default\").Provider(\"provider-1\").Result(),\n\t\t\t},\n\t\t\tvolumeSnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t// the volume snapshotter fake is not configured with any snapshotID -> volumeID\n\t\t\t\t// mappings as a way to verify that the snapshot is not restored, since if it were\n\t\t\t\t// restored, we'd get an error of \"snapshot not found\".\n\t\t\t\t\"provider-1\": &volumeSnapshotter{},\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").\n\t\t\t\t\t\tReclaimPolicy(corev1api.PersistentVolumeReclaimDelete).\n\t\t\t\t\t\tAWSEBSVolumeID(\"old-volume\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when a PV with a reclaim policy of retain has a snapshot and exists in-cluster, neither the snapshot nor the PV are restored\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").\n\t\t\t\t\t\tReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).\n\t\t\t\t\t\tAWSEBSVolumeID(\"old-volume\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").\n\t\t\t\t\t\tReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).\n\t\t\t\t\t\tAWSEBSVolumeID(\"old-volume\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(),\n\t\t\t},\n\t\t\tvolumeSnapshots: []*volume.Snapshot{\n\t\t\t\t{\n\t\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\t\tBackupName: \"backup-1\",\n\t\t\t\t\t\tLocation: \"default\",\n\t\t\t\t\t\tPersistentVolumeName: \"pv-1\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: volume.SnapshotStatus{\n\t\t\t\t\t\tPhase: volume.SnapshotPhaseCompleted,\n\t\t\t\t\t\tProviderSnapshotID: \"snapshot-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshotLocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"default\").Provider(\"provider-1\").Result(),\n\t\t\t},\n\t\t\tvolumeSnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t// the volume snapshotter fake is not configured with any snapshotID -> volumeID\n\t\t\t\t// mappings as a way to verify that the snapshot is not restored, since if it were\n\t\t\t\t// restored, we'd get an error of \"snapshot not found\".\n\t\t\t\t\"provider-1\": &volumeSnapshotter{},\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").\n\t\t\t\t\t\tReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).\n\t\t\t\t\t\tAWSEBSVolumeID(\"old-volume\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when a PV with a snapshot is used by a PVC in a namespace that's being remapped, and the original PV exists in-cluster, the PV is renamed\",\n\t\t\trestore: defaultRestore().NamespaceMappings(\"source-ns\", \"target-ns\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\n\t\t\t\t\t\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"source-pv\").AWSEBSVolumeID(\"source-volume\").ClaimRef(\"source-ns\", \"pvc-1\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\n\t\t\t\t\t\"persistentvolumeclaims\",\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"source-ns\", \"pvc-1\").VolumeName(\"source-pv\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"source-pv\").AWSEBSVolumeID(\"source-volume\").ClaimRef(\"source-ns\", \"pvc-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(),\n\t\t\t},\n\t\t\tvolumeSnapshots: []*volume.Snapshot{\n\t\t\t\t{\n\t\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\t\tBackupName: \"backup-1\",\n\t\t\t\t\t\tLocation: \"default\",\n\t\t\t\t\t\tPersistentVolumeName: \"source-pv\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: volume.SnapshotStatus{\n\t\t\t\t\t\tPhase: volume.SnapshotPhaseCompleted,\n\t\t\t\t\t\tProviderSnapshotID: \"snapshot-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshotLocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"default\").Provider(\"provider-1\").Result(),\n\t\t\t},\n\t\t\tvolumeSnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t\"provider-1\": &volumeSnapshotter{\n\t\t\t\t\tsnapshotVolumes: map[string]string{\"snapshot-1\": \"new-volume\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"source-pv\").AWSEBSVolumeID(\"source-volume\").ClaimRef(\"source-ns\", \"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"renamed-source-pv\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithAnnotations(\"velero.io/original-pv-name\", \"source-pv\"),\n\t\t\t\t\t\t\tbuilder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\"),\n\t\t\t\t\t\t\t// the namespace for this PV's claimRef should be the one that the PVC was remapped into.\n\t\t\t\t\t\t).ClaimRef(\"target-ns\", \"pvc-1\").\n\t\t\t\t\t\tAWSEBSVolumeID(\"new-volume\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"target-ns\", \"pvc-1\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tVolumeName(\"renamed-source-pv\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when a PV with a snapshot is used by a PVC in a namespace that's being remapped, and the original PV does not exist in-cluster, the PV is not renamed\",\n\t\t\trestore: defaultRestore().NamespaceMappings(\"source-ns\", \"target-ns\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\n\t\t\t\t\t\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"source-pv\").AWSEBSVolumeID(\"source-volume\").ClaimRef(\"source-ns\", \"pvc-1\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\n\t\t\t\t\t\"persistentvolumeclaims\",\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"source-ns\", \"pvc-1\").VolumeName(\"source-pv\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t\ttest.PVCs(),\n\t\t\t},\n\t\t\tvolumeSnapshots: []*volume.Snapshot{\n\t\t\t\t{\n\t\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\t\tBackupName: \"backup-1\",\n\t\t\t\t\t\tLocation: \"default\",\n\t\t\t\t\t\tPersistentVolumeName: \"source-pv\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: volume.SnapshotStatus{\n\t\t\t\t\t\tPhase: volume.SnapshotPhaseCompleted,\n\t\t\t\t\t\tProviderSnapshotID: \"snapshot-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshotLocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"default\").Provider(\"provider-1\").Result(),\n\t\t\t},\n\t\t\tvolumeSnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t\"provider-1\": &volumeSnapshotter{\n\t\t\t\t\tsnapshotVolumes: map[string]string{\"snapshot-1\": \"new-volume\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"source-pv\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tClaimRef(\"target-ns\", \"pvc-1\").\n\t\t\t\t\t\tAWSEBSVolumeID(\"new-volume\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"target-ns\", \"pvc-1\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tVolumeName(\"source-pv\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when a PV without a snapshot is used by a PVC in a namespace that's being remapped, and the original PV exists in-cluster, the PV is not replaced and there is a restore warning\",\n\t\t\trestore: defaultRestore().NamespaceMappings(\"source-ns\", \"target-ns\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\n\t\t\t\t\t\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"source-pv\").\n\t\t\t\t\t\t//ReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).\n\t\t\t\t\t\tAWSEBSVolumeID(\"source-volume\").\n\t\t\t\t\t\tClaimRef(\"source-ns\", \"pvc-1\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t).\n\t\t\t\tAddItems(\n\t\t\t\t\t\"persistentvolumeclaims\",\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"source-ns\", \"pvc-1\").VolumeName(\"source-pv\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"source-pv\").\n\t\t\t\t\t\t//ReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).\n\t\t\t\t\t\tAWSEBSVolumeID(\"source-volume\").\n\t\t\t\t\t\tClaimRef(\"source-ns\", \"pvc-1\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(),\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"source-pv\").\n\t\t\t\t\t\tAWSEBSVolumeID(\"source-volume\").\n\t\t\t\t\t\tClaimRef(\"source-ns\", \"pvc-1\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"target-ns\", \"pvc-1\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tVolumeName(\"source-pv\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t},\n\t\t\twantWarning: true,\n\t\t},\n\t\t{\n\t\t\tname: \"when a PV without a snapshot is used by a PVC in a namespace that's being remapped, and the original PV does not exist in-cluster, the PV is not renamed\",\n\t\t\trestore: defaultRestore().NamespaceMappings(\"source-ns\", \"target-ns\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\n\t\t\t\t\t\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"source-pv\").\n\t\t\t\t\t\tAWSEBSVolumeID(\"source-volume\").\n\t\t\t\t\t\tClaimRef(\"source-ns\", \"pvc-1\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t).\n\t\t\t\tAddItems(\n\t\t\t\t\t\"persistentvolumeclaims\",\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"source-ns\", \"pvc-1\").VolumeName(\"source-pv\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t\ttest.PVCs(),\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"source-pv\").\n\t\t\t\t\t\t//ReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\t// the namespace for this PV's claimRef should be the one that the PVC was remapped into.\n\t\t\t\t\t\tClaimRef(\"target-ns\", \"pvc-1\").\n\t\t\t\t\t\tAWSEBSVolumeID(\"source-volume\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"target-ns\", \"pvc-1\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tVolumeName(\"source-pv\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when a PV is renamed and the original PV does not exist in-cluster, the PV should be renamed\",\n\t\t\trestore: defaultRestore().NamespaceMappings(\"source-ns\", \"target-ns\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\n\t\t\t\t\t\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"source-pv\").AWSEBSVolumeID(\"source-volume\").ClaimRef(\"source-ns\", \"pvc-1\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\n\t\t\t\t\t\"persistentvolumeclaims\",\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"source-ns\", \"pvc-1\").VolumeName(\"source-pv\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t\ttest.PVCs(),\n\t\t\t},\n\t\t\tvolumeSnapshots: []*volume.Snapshot{\n\t\t\t\t{\n\t\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\t\tBackupName: \"backup-1\",\n\t\t\t\t\t\tLocation: \"default\",\n\t\t\t\t\t\tPersistentVolumeName: \"source-pv\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: volume.SnapshotStatus{\n\t\t\t\t\t\tPhase: volume.SnapshotPhaseCompleted,\n\t\t\t\t\t\tProviderSnapshotID: \"snapshot-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshotLocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"default\").Provider(\"provider-1\").Result(),\n\t\t\t},\n\t\t\tvolumeSnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t\"provider-1\": &volumeSnapshotter{\n\t\t\t\t\tsnapshotVolumes: map[string]string{\"snapshot-1\": \"new-pvname\"},\n\t\t\t\t\tpvName: map[string]string{\"new-pvname\": \"new-pvname\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"new-pvname\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\"),\n\t\t\t\t\t\t\tbuilder.WithAnnotations(\"velero.io/original-pv-name\", \"source-pv\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tClaimRef(\"target-ns\", \"pvc-1\").\n\t\t\t\t\t\tAWSEBSVolumeID(\"new-pvname\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"target-ns\", \"pvc-1\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tVolumeName(\"new-pvname\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when a PV with a reclaim policy of retain has a snapshot and exists in-cluster, neither the snapshot nor the PV are restored\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").\n\t\t\t\t\t\tReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).\n\t\t\t\t\t\tAWSEBSVolumeID(\"old-volume\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").\n\t\t\t\t\t\tReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).\n\t\t\t\t\t\tAWSEBSVolumeID(\"old-volume\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(),\n\t\t\t},\n\t\t\tvolumeSnapshots: []*volume.Snapshot{\n\t\t\t\t{\n\t\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\t\tBackupName: \"backup-1\",\n\t\t\t\t\t\tLocation: \"default\",\n\t\t\t\t\t\tPersistentVolumeName: \"pv-1\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: volume.SnapshotStatus{\n\t\t\t\t\t\tPhase: volume.SnapshotPhaseCompleted,\n\t\t\t\t\t\tProviderSnapshotID: \"snapshot-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshotLocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tNamespace: velerov1api.DefaultNamespace,\n\t\t\t\t\t\tName: \"default\",\n\t\t\t\t\t},\n\t\t\t\t\tSpec: velerov1api.VolumeSnapshotLocationSpec{\n\t\t\t\t\t\tProvider: \"provider-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t// the volume snapshotter fake is not configured with any snapshotID -> volumeID\n\t\t\t\t// mappings as a way to verify that the snapshot is not restored, since if it were\n\t\t\t\t// restored, we'd get an error of \"snapshot not found\".\n\t\t\t\t\"provider-1\": &volumeSnapshotter{},\n\t\t\t},\n\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").\n\t\t\t\t\t\tReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).\n\t\t\t\t\t\tAWSEBSVolumeID(\"old-volume\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when a PV with a snapshot is used by a PVC in a namespace that's being remapped, and the original PV exists in-cluster, the PV is renamed by volumesnapshotter\",\n\t\t\trestore: defaultRestore().NamespaceMappings(\"source-ns\", \"target-ns\").Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\n\t\t\t\t\t\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"source-pv\").AWSEBSVolumeID(\"source-volume\").ClaimRef(\"source-ns\", \"pvc-1\").Result(),\n\t\t\t\t).\n\t\t\t\tAddItems(\n\t\t\t\t\t\"persistentvolumeclaims\",\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"source-ns\", \"pvc-1\").VolumeName(\"source-pv\").Result(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"source-pv\").AWSEBSVolumeID(\"source-volume\").ClaimRef(\"source-ns\", \"pvc-1\").Result(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(),\n\t\t\t},\n\t\t\tvolumeSnapshots: []*volume.Snapshot{\n\t\t\t\t{\n\t\t\t\t\tSpec: volume.SnapshotSpec{\n\t\t\t\t\t\tBackupName: \"backup-1\",\n\t\t\t\t\t\tLocation: \"default\",\n\t\t\t\t\t\tPersistentVolumeName: \"source-pv\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: volume.SnapshotStatus{\n\t\t\t\t\t\tPhase: volume.SnapshotPhaseCompleted,\n\t\t\t\t\t\tProviderSnapshotID: \"snapshot-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshotLocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"default\").Provider(\"provider-1\").Result(),\n\t\t\t},\n\t\t\tvolumeSnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t\"provider-1\": &volumeSnapshotter{\n\t\t\t\t\tsnapshotVolumes: map[string]string{\"snapshot-1\": \"new-volume\"},\n\t\t\t\t\tpvName: map[string]string{\"new-volume\": \"volumesnapshotter-renamed-source-pv\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []*test.APIResource{\n\t\t\t\ttest.PVs(\n\t\t\t\t\tbuilder.ForPersistentVolume(\"source-pv\").AWSEBSVolumeID(\"source-volume\").ClaimRef(\"source-ns\", \"pvc-1\").Result(),\n\t\t\t\t\tbuilder.ForPersistentVolume(\"volumesnapshotter-renamed-source-pv\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithAnnotations(\"velero.io/original-pv-name\", \"source-pv\"),\n\t\t\t\t\t\t\tbuilder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tClaimRef(\"target-ns\", \"pvc-1\").\n\t\t\t\t\t\tAWSEBSVolumeID(\"new-volume\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t\ttest.PVCs(\n\t\t\t\t\tbuilder.ForPersistentVolumeClaim(\"target-ns\", \"pvc-1\").\n\t\t\t\t\t\tObjectMeta(\n\t\t\t\t\t\t\tbuilder.WithLabels(\"velero.io/backup-name\", \"backup-1\", \"velero.io/restore-name\", \"restore-1\"),\n\t\t\t\t\t\t).\n\t\t\t\t\t\tVolumeName(\"volumesnapshotter-renamed-source-pv\").\n\t\t\t\t\t\tResult(),\n\t\t\t\t),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when a PV with a reclaim policy of retain has a CSI VolumeSnapshot and does not exist in-cluster, the PV is not restored\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").\n\t\t\t\t\t\tReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).\n\t\t\t\t\t\tClaimRef(\"velero\", testPVCName).\n\t\t\t\t\t\tResult(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t\ttest.PVCs(),\n\t\t\t},\n\t\t\tcsiVolumeSnapshots: []*snapshotv1api.VolumeSnapshot{\n\t\t\t\t{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t\t\tName: \"test\",\n\t\t\t\t\t},\n\t\t\t\t\tSpec: snapshotv1api.VolumeSnapshotSpec{\n\t\t\t\t\t\tSource: snapshotv1api.VolumeSnapshotSource{\n\t\t\t\t\t\t\tPersistentVolumeClaimName: &testPVCName,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeSnapshotLocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"default\").Provider(\"provider-1\").Result(),\n\t\t\t},\n\t\t\tvolumeSnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t\"provider-1\": &volumeSnapshotter{\n\t\t\t\t\tsnapshotVolumes: map[string]string{\"snapshot-1\": \"new-volume\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []*test.APIResource{},\n\t\t},\n\t\t{\n\t\t\tname: \"when a PV with a reclaim policy of retain has a DataUpload result CM and does not exist in-cluster, the PV is not restored\",\n\t\t\trestore: defaultRestore().ObjectMeta(builder.WithUID(\"fakeUID\")).Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\ttarball: test.NewTarWriter(t).\n\t\t\t\tAddItems(\"persistentvolumes\",\n\t\t\t\t\tbuilder.ForPersistentVolume(\"pv-1\").\n\t\t\t\t\t\tReclaimPolicy(corev1api.PersistentVolumeReclaimRetain).\n\t\t\t\t\t\tClaimRef(\"velero\", testPVCName).\n\t\t\t\t\t\tResult(),\n\t\t\t\t).\n\t\t\t\tDone(),\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t\ttest.PVCs(),\n\t\t\t\ttest.ConfigMaps(),\n\t\t\t},\n\t\t\tvolumeSnapshotLocations: []*velerov1api.VolumeSnapshotLocation{\n\t\t\t\tbuilder.ForVolumeSnapshotLocation(velerov1api.DefaultNamespace, \"default\").Provider(\"provider-1\").Result(),\n\t\t\t},\n\t\t\tvolumeSnapshotterGetter: map[string]vsv1.VolumeSnapshotter{\n\t\t\t\t\"provider-1\": &volumeSnapshotter{\n\t\t\t\t\tsnapshotVolumes: map[string]string{\"snapshot-1\": \"new-volume\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\tdataUploadResult: builder.ForConfigMap(\"velero\", \"test\").ObjectMeta(builder.WithLabelsMap(map[string]string{\n\t\t\t\tvelerov1api.RestoreUIDLabel: \"fakeUID\",\n\t\t\t\tvelerov1api.PVCNamespaceNameLabel: \"velero.testPVC\",\n\t\t\t\tvelerov1api.ResourceUsageLabel: string(velerov1api.VeleroResourceUsageDataUploadResult),\n\t\t\t})).Result(),\n\t\t\twant: []*test.APIResource{},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\th := newHarness(t)\n\t\t\th.restorer.resourcePriorities = types.Priorities{HighPriorities: []string{\"persistentvolumes\", \"persistentvolumeclaims\"}}\n\t\t\th.restorer.pvRenamer = func(oldName string) (string, error) {\n\t\t\t\trenamed := \"renamed-\" + oldName\n\t\t\t\treturn renamed, nil\n\t\t\t}\n\n\t\t\t// set up the VolumeSnapshotLocation client and add test data to it\n\t\t\tfor _, vsl := range tc.volumeSnapshotLocations {\n\t\t\t\trequire.NoError(t, h.restorer.kbClient.Create(t.Context(), vsl))\n\t\t\t}\n\n\t\t\tif tc.dataUploadResult != nil {\n\t\t\t\trequire.NoError(t, h.restorer.kbClient.Create(t.Context(), tc.dataUploadResult))\n\t\t\t}\n\n\t\t\tfor _, r := range tc.apiResources {\n\t\t\t\th.AddItems(t, r)\n\t\t\t}\n\n\t\t\t// Collect the IDs of all of the wanted resources so we can ensure the\n\t\t\t// exact set exists in the API after restore.\n\t\t\twantIDs := make(map[*test.APIResource][]string)\n\t\t\tfor i, resource := range tc.want {\n\t\t\t\twantIDs[tc.want[i]] = []string{}\n\n\t\t\t\tfor _, item := range resource.Items {\n\t\t\t\t\twantIDs[tc.want[i]] = append(wantIDs[tc.want[i]], fmt.Sprintf(\"%s/%s\", item.GetNamespace(), item.GetName()))\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tdata := &Request{\n\t\t\t\tLog: h.log,\n\t\t\t\tRestore: tc.restore,\n\t\t\t\tBackup: tc.backup,\n\t\t\t\tVolumeSnapshots: tc.volumeSnapshots,\n\t\t\t\tBackupReader: tc.tarball,\n\t\t\t\tCSIVolumeSnapshots: tc.csiVolumeSnapshots,\n\t\t\t\tRestoreVolumeInfoTracker: volume.NewRestoreVolInfoTracker(tc.restore, h.log, test.NewFakeControllerRuntimeClient(t)),\n\t\t\t}\n\t\t\twarnings, errs := h.restorer.Restore(\n\t\t\t\tdata,\n\t\t\t\tnil, // restoreItemActions\n\t\t\t\ttc.volumeSnapshotterGetter,\n\t\t\t)\n\n\t\t\tif tc.wantWarning {\n\t\t\t\tassertNonEmptyResults(t, \"warning\", warnings)\n\t\t\t} else {\n\t\t\t\tassertEmptyResults(t, warnings)\n\t\t\t}\n\t\t\tif tc.wantError {\n\t\t\t\tassertNonEmptyResults(t, \"error\", errs)\n\t\t\t} else {\n\t\t\t\tassertEmptyResults(t, errs)\n\t\t\t}\n\t\t\tassertAPIContents(t, h, wantIDs)\n\t\t\tassertRestoredItems(t, h, tc.want)\n\t\t})\n\t}\n}\n\ntype fakePodVolumeRestorerFactory struct {\n\trestorer *uploadermocks.Restorer\n}\n\nfunc (f *fakePodVolumeRestorerFactory) NewRestorer(context.Context, *velerov1api.Restore) (podvolume.Restorer, error) {\n\treturn f.restorer, nil\n}\n\n// TestRestoreWithPodVolume verifies that a call to RestorePodVolumes was made as and when\n// expected for the given pods by using a mock for the pod volume restorer.\nfunc TestRestoreWithPodVolume(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\trestore *velerov1api.Restore\n\t\tbackup *velerov1api.Backup\n\t\tapiResources []*test.APIResource\n\t\tpodVolumeBackups []*velerov1api.PodVolumeBackup\n\t\tpodWithPVBs, podWithoutPVBs []*corev1api.Pod\n\t\twant map[*test.APIResource][]string\n\t}{\n\t\t{\n\t\t\tname: \"a pod that exists in given backup and contains associated PVBs should have RestorePodVolumes called\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{test.Pods()},\n\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-1\").PodName(\"pod-1\").SnapshotID(\"foo\").Result(),\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-2\").PodName(\"pod-2\").PodNamespace(\"ns-1\").SnapshotID(\"foo\").Result(),\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-3\").PodName(\"pod-4\").PodNamespace(\"ns-2\").SnapshotID(\"foo\").Result(),\n\t\t\t},\n\t\t\tpodWithPVBs: []*corev1api.Pod{\n\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-2\").\n\t\t\t\t\tResult(),\n\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-4\").\n\t\t\t\t\tResult(),\n\t\t\t},\n\t\t\tpodWithoutPVBs: []*corev1api.Pod{\n\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-3\").\n\t\t\t\t\tResult(),\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-2\", \"ns-2/pod-3\", \"ns-2/pod-4\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"a pod that exists in given backup but does not contain associated PVBs should not have should have RestorePodVolumes called\",\n\t\t\trestore: defaultRestore().Result(),\n\t\t\tbackup: defaultBackup().Result(),\n\t\t\tapiResources: []*test.APIResource{test.Pods()},\n\t\t\tpodVolumeBackups: []*velerov1api.PodVolumeBackup{\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-1\").PodName(\"pod-1\").Result(),\n\t\t\t\tbuilder.ForPodVolumeBackup(\"velero\", \"pvb-2\").PodName(\"pod-2\").Result(),\n\t\t\t},\n\t\t\tpodWithPVBs: []*corev1api.Pod{},\n\t\t\tpodWithoutPVBs: []*corev1api.Pod{\n\t\t\t\tbuilder.ForPod(\"ns-1\", \"pod-3\").\n\t\t\t\t\tResult(),\n\t\t\t\tbuilder.ForPod(\"ns-2\", \"pod-4\").\n\t\t\t\t\tResult(),\n\t\t\t},\n\t\t\twant: map[*test.APIResource][]string{\n\t\t\t\ttest.Pods(): {\"ns-1/pod-3\", \"ns-2/pod-4\"},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\th := newHarness(t)\n\t\t\trestorer := new(uploadermocks.Restorer)\n\t\t\tdefer restorer.AssertExpectations(t)\n\t\t\th.restorer.podVolumeRestorerFactory = &fakePodVolumeRestorerFactory{\n\t\t\t\trestorer: restorer,\n\t\t\t}\n\n\t\t\t// needed only to indicate resource types that can be restored, in this case, pods\n\t\t\tfor _, resource := range tc.apiResources {\n\t\t\t\th.AddItems(t, resource)\n\t\t\t}\n\n\t\t\ttarball := test.NewTarWriter(t)\n\n\t\t\t// these backed up pods don't have any PVBs associated with them, so a call to RestorePodVolumes is not expected to be made for them\n\t\t\tfor _, pod := range tc.podWithoutPVBs {\n\t\t\t\ttarball.AddItems(\"pods\", pod)\n\t\t\t}\n\n\t\t\t// these backed up pods have PVBs associated with them, so a call to RestorePodVolumes will be made for each of them\n\t\t\tfor _, pod := range tc.podWithPVBs {\n\t\t\t\ttarball.AddItems(\"pods\", pod)\n\n\t\t\t\t// the restore process adds these labels before restoring, so we must add them here too otherwise they won't match\n\t\t\t\tpod.Labels = map[string]string{\"velero.io/backup-name\": tc.backup.Name, \"velero.io/restore-name\": tc.restore.Name}\n\t\t\t\texpectedArgs := podvolume.RestoreData{\n\t\t\t\t\tRestore: tc.restore,\n\t\t\t\t\tPod: pod,\n\t\t\t\t\tPodVolumeBackups: tc.podVolumeBackups,\n\t\t\t\t\tSourceNamespace: pod.Namespace,\n\t\t\t\t\tBackupLocation: \"\",\n\t\t\t\t}\n\t\t\t\trestorer.\n\t\t\t\t\tOn(\"RestorePodVolumes\", expectedArgs, mock.Anything).\n\t\t\t\t\tReturn(nil)\n\t\t\t}\n\n\t\t\tdata := &Request{\n\t\t\t\tLog: h.log,\n\t\t\t\tRestore: tc.restore,\n\t\t\t\tBackup: tc.backup,\n\t\t\t\tPodVolumeBackups: tc.podVolumeBackups,\n\t\t\t\tBackupReader: tarball.Done(),\n\t\t\t}\n\n\t\t\twarnings, errs := h.restorer.Restore(\n\t\t\t\tdata,\n\t\t\t\tnil, // restoreItemActions\n\t\t\t\tnil, // volume snapshotter getter\n\t\t\t)\n\n\t\t\tassertEmptyResults(t, warnings, errs)\n\t\t\tassertAPIContents(t, h, tc.want)\n\t\t})\n\t}\n}\n\nfunc TestResetMetadata(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tobj *unstructured.Unstructured\n\t\texpectedErr bool\n\t\texpectedRes *unstructured.Unstructured\n\t}{\n\t\t{\n\t\t\tname: \"no metadata causes error\",\n\t\t\tobj: &unstructured.Unstructured{},\n\t\t\texpectedErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"keep name, namespace, labels, annotations, managedFields, finalizers\",\n\t\t\tobj: newTestUnstructured().WithMetadata(\"name\", \"namespace\", \"labels\", \"annotations\", \"managedFields\", \"finalizers\").Unstructured,\n\t\t\texpectedErr: false,\n\t\t\texpectedRes: newTestUnstructured().WithMetadata(\"name\", \"namespace\", \"labels\", \"annotations\", \"managedFields\", \"finalizers\").Unstructured,\n\t\t},\n\t\t{\n\t\t\tname: \"remove uid, ownerReferences\",\n\t\t\tobj: newTestUnstructured().WithMetadata(\"name\", \"namespace\", \"uid\", \"ownerReferences\").Unstructured,\n\t\t\texpectedErr: false,\n\t\t\texpectedRes: newTestUnstructured().WithMetadata(\"name\", \"namespace\").Unstructured,\n\t\t},\n\t\t{\n\t\t\tname: \"keep status\",\n\t\t\tobj: newTestUnstructured().WithMetadata().WithStatus().Unstructured,\n\t\t\texpectedErr: false,\n\t\t\texpectedRes: newTestUnstructured().WithMetadata().WithStatus().Unstructured,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tres, err := resetMetadata(test.obj)\n\n\t\t\tif assert.Equal(t, test.expectedErr, err != nil) {\n\t\t\t\tassert.Equal(t, test.expectedRes, res)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestResetStatus(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tobj *unstructured.Unstructured\n\t\texpectedRes *unstructured.Unstructured\n\t}{\n\t\t{\n\t\t\tname: \"no status don't cause error\",\n\t\t\tobj: &unstructured.Unstructured{},\n\t\t\texpectedRes: &unstructured.Unstructured{},\n\t\t},\n\t\t{\n\t\t\tname: \"remove status\",\n\t\t\tobj: newTestUnstructured().WithMetadata().WithStatus().Unstructured,\n\t\t\texpectedRes: newTestUnstructured().WithMetadata().Unstructured,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tresetStatus(test.obj)\n\t\t\tassert.Equal(t, test.expectedRes, test.obj)\n\t\t})\n\t}\n}\n\nfunc TestIsCompleted(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\texpected bool\n\t\tcontent string\n\t\tgroupResource schema.GroupResource\n\t\texpectedErr bool\n\t}{\n\t\t{\n\t\t\tname: \"Failed pods are complete\",\n\t\t\texpected: true,\n\t\t\tcontent: `{\"apiVersion\":\"v1\",\"kind\":\"Pod\",\"metadata\":{\"namespace\":\"ns\",\"name\":\"pod1\"}, \"status\": {\"phase\": \"Failed\"}}`,\n\t\t\tgroupResource: schema.GroupResource{Group: \"\", Resource: \"pods\"},\n\t\t},\n\t\t{\n\t\t\tname: \"Succeeded pods are complete\",\n\t\t\texpected: true,\n\t\t\tcontent: `{\"apiVersion\":\"v1\",\"kind\":\"Pod\",\"metadata\":{\"namespace\":\"ns\",\"name\":\"pod1\"}, \"status\": {\"phase\": \"Succeeded\"}}`,\n\t\t\tgroupResource: schema.GroupResource{Group: \"\", Resource: \"pods\"},\n\t\t},\n\t\t{\n\t\t\tname: \"Pending pods aren't complete\",\n\t\t\texpected: false,\n\t\t\tcontent: `{\"apiVersion\":\"v1\",\"kind\":\"Pod\",\"metadata\":{\"namespace\":\"ns\",\"name\":\"pod1\"}, \"status\": {\"phase\": \"Pending\"}}`,\n\t\t\tgroupResource: schema.GroupResource{Group: \"\", Resource: \"pods\"},\n\t\t},\n\t\t{\n\t\t\tname: \"Running pods aren't complete\",\n\t\t\texpected: false,\n\t\t\tcontent: `{\"apiVersion\":\"v1\",\"kind\":\"Pod\",\"metadata\":{\"namespace\":\"ns\",\"name\":\"pod1\"}, \"status\": {\"phase\": \"Running\"}}`,\n\t\t\tgroupResource: schema.GroupResource{Group: \"\", Resource: \"pods\"},\n\t\t},\n\t\t{\n\t\t\tname: \"Jobs without a completion time aren't complete\",\n\t\t\texpected: false,\n\t\t\tcontent: `{\"apiVersion\":\"v1\",\"kind\":\"Pod\",\"metadata\":{\"namespace\":\"ns\",\"name\":\"pod1\"}}`,\n\t\t\tgroupResource: schema.GroupResource{Group: \"batch\", Resource: \"jobs\"},\n\t\t},\n\t\t{\n\t\t\tname: \"Jobs with a completion time are completed\",\n\t\t\texpected: true,\n\t\t\tcontent: `{\"apiVersion\":\"v1\",\"kind\":\"Pod\",\"metadata\":{\"namespace\":\"ns\",\"name\":\"pod1\"}, \"status\": {\"completionTime\": \"bar\"}}`,\n\t\t\tgroupResource: schema.GroupResource{Group: \"batch\", Resource: \"jobs\"},\n\t\t},\n\t\t{\n\t\t\tname: \"Jobs with an empty completion time are not completed\",\n\t\t\texpected: false,\n\t\t\tcontent: `{\"apiVersion\":\"v1\",\"kind\":\"Pod\",\"metadata\":{\"namespace\":\"ns\",\"name\":\"pod1\"}, \"status\": {\"completionTime\": \"\"}}`,\n\t\t\tgroupResource: schema.GroupResource{Group: \"batch\", Resource: \"jobs\"},\n\t\t},\n\t\t{\n\t\t\tname: \"Something not a pod or a job may actually be complete, but we're not concerned with that\",\n\t\t\texpected: false,\n\t\t\tcontent: `{\"apiVersion\": \"v1\", \"kind\": \"Namespace\", \"metadata\": {\"name\": \"ns\"}, \"status\": {\"completionTime\": \"bar\", \"phase\":\"Completed\"}}`,\n\t\t\tgroupResource: schema.GroupResource{Group: \"\", Resource: \"namespaces\"},\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tu := test.UnstructuredOrDie(tt.content)\n\t\t\tbackup, err := isCompleted(u, tt.groupResource)\n\n\t\t\tif assert.Equal(t, tt.expectedErr, err != nil) {\n\t\t\t\tassert.Equal(t, tt.expected, backup)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc Test_getOrderedResources(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tresourcePriorities types.Priorities\n\t\tbackupResources map[string]*archive.ResourceItems\n\t\twant []string\n\t}{\n\t\t{\n\t\t\tname: \"when only priorities are specified, they're returned in order\",\n\t\t\tresourcePriorities: types.Priorities{HighPriorities: []string{\"prio-3\", \"prio-2\", \"prio-1\"}},\n\t\t\tbackupResources: nil,\n\t\t\twant: []string{\"prio-3\", \"prio-2\", \"prio-1\"},\n\t\t},\n\t\t{\n\t\t\tname: \"when only backup resources are specified, they're returned in alphabetical order\",\n\t\t\tresourcePriorities: types.Priorities{},\n\t\t\tbackupResources: map[string]*archive.ResourceItems{\n\t\t\t\t\"backup-resource-3\": nil,\n\t\t\t\t\"backup-resource-2\": nil,\n\t\t\t\t\"backup-resource-1\": nil,\n\t\t\t},\n\t\t\twant: []string{\"backup-resource-1\", \"backup-resource-2\", \"backup-resource-3\"},\n\t\t},\n\t\t{\n\t\t\tname: \"when priorities and backup resources are specified, they're returned in the correct order\",\n\t\t\tresourcePriorities: types.Priorities{HighPriorities: []string{\"prio-3\", \"prio-2\", \"prio-1\"}},\n\t\t\tbackupResources: map[string]*archive.ResourceItems{\n\t\t\t\t\"prio-3\": nil,\n\t\t\t\t\"backup-resource-3\": nil,\n\t\t\t\t\"backup-resource-2\": nil,\n\t\t\t\t\"backup-resource-1\": nil,\n\t\t\t},\n\t\t\twant: []string{\"prio-3\", \"prio-2\", \"prio-1\", \"backup-resource-1\", \"backup-resource-2\", \"backup-resource-3\"},\n\t\t},\n\t\t{\n\t\t\tname: \"when priorities and backup resources are specified, they're returned in the correct order\",\n\t\t\tresourcePriorities: types.Priorities{HighPriorities: []string{\"prio-3\", \"prio-2\", \"prio-1\"}, LowPriorities: []string{\"prio-0\"}},\n\t\t\tbackupResources: map[string]*archive.ResourceItems{\n\t\t\t\t\"prio-3\": nil,\n\t\t\t\t\"prio-0\": nil,\n\t\t\t\t\"backup-resource-3\": nil,\n\t\t\t\t\"backup-resource-2\": nil,\n\t\t\t\t\"backup-resource-1\": nil,\n\t\t\t},\n\t\t\twant: []string{\"prio-3\", \"prio-2\", \"prio-1\", \"backup-resource-1\", \"backup-resource-2\", \"backup-resource-3\", \"prio-0\"},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tassert.Equal(t, tc.want, getOrderedResources(tc.resourcePriorities, tc.backupResources))\n\t\t})\n\t}\n}\n\n// assertResourceCreationOrder ensures that resources were created in the expected\n// order. Any resources *not* in resourcePriorities are required to come *after* all\n// resources in any order.\nfunc assertResourceCreationOrder(t *testing.T, resourcePriorities []string, createdResources []resourceID) {\n\tt.Helper()\n\t// lastSeen tracks the index in 'resourcePriorities' of the last resource type\n\t// we saw created. Once we've seen a resource in 'resourcePriorities', we should\n\t// never see another instance of a prior resource.\n\tlastSeen := 0\n\n\t// Find the index in 'resourcePriorities' of the resource type for\n\t// the current item, if it exists. This index ('current') *must*\n\t// be greater than or equal to 'lastSeen', which was the last resource\n\t// we saw, since otherwise the current resource would be out of order. By\n\t// initializing current to len(ordered), we're saying that if the resource\n\t// is not explicitly in orderedResources, then it must come *after*\n\t// all orderedResources.\n\tfor _, r := range createdResources {\n\t\tcurrent := len(resourcePriorities)\n\t\tfor i, item := range resourcePriorities {\n\t\t\tif item == r.groupResource {\n\t\t\t\tcurrent = i\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\n\t\t// the index of the current resource must be the same as or greater than the index of\n\t\t// the last resource we saw for the restored order to be correct.\n\t\tassert.GreaterOrEqual(t, current, lastSeen, \"%s was restored out of order\", r.groupResource)\n\t\tlastSeen = current\n\t}\n}\n\ntype resourceID struct {\n\tgroupResource string\n\tnsAndName string\n}\n\n// createRecorder provides a Reactor that can be used to capture\n// resources created in a fake client.\ntype createRecorder struct {\n\tt *testing.T\n\tresources []resourceID\n}\n\nfunc (cr *createRecorder) reactor() func(kubetesting.Action) (bool, runtime.Object, error) {\n\treturn func(action kubetesting.Action) (bool, runtime.Object, error) {\n\t\tcreateAction, ok := action.(kubetesting.CreateAction)\n\t\tif !ok {\n\t\t\treturn false, nil, nil\n\t\t}\n\n\t\taccessor, err := meta.Accessor(createAction.GetObject())\n\t\trequire.NoError(cr.t, err)\n\n\t\tcr.resources = append(cr.resources, resourceID{\n\t\t\tgroupResource: action.GetResource().GroupResource().String(),\n\t\t\tnsAndName: fmt.Sprintf(\"%s/%s\", action.GetNamespace(), accessor.GetName()),\n\t\t})\n\n\t\treturn false, nil, nil\n\t}\n}\n\nfunc defaultRestore() *builder.RestoreBuilder {\n\treturn builder.ForRestore(velerov1api.DefaultNamespace, \"restore-1\").Backup(\"backup-1\")\n}\n\n// assertAPIContents asserts that the dynamic client on the provided harness contains\n// all of the items specified in 'want' (a map from an APIResource definition to a slice\n// of resource identifiers, formatted as /).\nfunc assertAPIContents(t *testing.T, h *harness, want map[*test.APIResource][]string) {\n\tt.Helper()\n\n\tfor r, want := range want {\n\t\tres, err := h.DynamicClient.Resource(r.GVR()).List(t.Context(), metav1.ListOptions{})\n\t\trequire.NoError(t, err)\n\t\tif err != nil {\n\t\t\tcontinue\n\t\t}\n\n\t\tgot := sets.NewString()\n\t\tfor _, item := range res.Items {\n\t\t\tgot.Insert(fmt.Sprintf(\"%s/%s\", item.GetNamespace(), item.GetName()))\n\t\t}\n\n\t\tassert.Equal(t, sets.NewString(want...), got)\n\t}\n}\n\nfunc assertEmptyResults(t *testing.T, res ...Result) {\n\tt.Helper()\n\n\tfor _, r := range res {\n\t\tassert.Empty(t, r.Cluster)\n\t\tassert.Empty(t, r.Namespaces)\n\t\tassert.Empty(t, r.Velero)\n\t}\n}\n\nfunc assertNonEmptyResults(t *testing.T, typeMsg string, res ...Result) {\n\tt.Helper()\n\ttotal := 0\n\tfor _, r := range res {\n\t\ttotal += len(r.Cluster)\n\t\ttotal += len(r.Namespaces)\n\t\ttotal += len(r.Velero)\n\t}\n\tassert.Positive(t, total, \"Expected at least one \"+typeMsg)\n}\n\ntype harness struct {\n\t*test.APIServer\n\n\trestorer *kubernetesRestorer\n\tlog logrus.FieldLogger\n}\n\nfunc newHarness(t *testing.T) *harness {\n\tt.Helper()\n\n\tapiServer := test.NewAPIServer(t)\n\tlog := logrus.StandardLogger()\n\tkbClient := test.NewFakeControllerRuntimeClient(t)\n\n\tdiscoveryHelper, err := discovery.NewHelper(apiServer.DiscoveryClient, log)\n\trequire.NoError(t, err)\n\n\treturn &harness{\n\t\tAPIServer: apiServer,\n\t\trestorer: &kubernetesRestorer{\n\t\t\tdiscoveryHelper: discoveryHelper,\n\t\t\tdynamicFactory: client.NewDynamicFactory(apiServer.DynamicClient),\n\t\t\tnamespaceClient: apiServer.KubeClient.CoreV1().Namespaces(),\n\t\t\tresourceTerminatingTimeout: time.Minute,\n\t\t\tlogger: log,\n\t\t\tfileSystem: test.NewFakeFileSystem(),\n\n\t\t\t// unsupported\n\t\t\tpodVolumeRestorerFactory: nil,\n\t\t\tpodVolumeTimeout: 0,\n\t\t\tkbClient: kbClient,\n\t\t\tresourceDeletionStatusTracker: kube.NewResourceDeletionStatusTracker(),\n\t\t},\n\t\tlog: log,\n\t}\n}\n\nfunc (h *harness) AddItems(t *testing.T, resource *test.APIResource) {\n\tt.Helper()\n\n\th.DiscoveryClient.WithAPIResource(resource)\n\trequire.NoError(t, h.restorer.discoveryHelper.Refresh())\n\n\tfor _, item := range resource.Items {\n\t\tobj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(item)\n\t\trequire.NoError(t, err)\n\n\t\tunstructuredObj := &unstructured.Unstructured{Object: obj}\n\n\t\t// These fields have non-nil zero values in the unstructured objects. We remove\n\t\t// them to make comparison easier in our tests.\n\t\tunstructured.RemoveNestedField(unstructuredObj.Object, \"metadata\", \"creationTimestamp\")\n\t\tunstructured.RemoveNestedField(unstructuredObj.Object, \"status\")\n\n\t\tif resource.Namespaced {\n\t\t\t_, err = h.DynamicClient.Resource(resource.GVR()).Namespace(item.GetNamespace()).Create(t.Context(), unstructuredObj, metav1.CreateOptions{})\n\t\t} else {\n\t\t\t_, err = h.DynamicClient.Resource(resource.GVR()).Create(t.Context(), unstructuredObj, metav1.CreateOptions{})\n\t\t}\n\t\trequire.NoError(t, err)\n\t}\n}\n\nfunc Test_resetVolumeBindingInfo(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tobj *unstructured.Unstructured\n\t\texpected *unstructured.Unstructured\n\t}{\n\t\t{\n\t\t\tname: \"PVs that are bound have their binding and dynamic provisioning annotations removed\",\n\t\t\tobj: newTestUnstructured().WithMetadataField(\"kind\", \"persistentVolume\").\n\t\t\t\tWithName(\"pv-1\").WithAnnotations(\n\t\t\t\tkubeutil.KubeAnnBindCompleted,\n\t\t\t\tkubeutil.KubeAnnBoundByController,\n\t\t\t\tkubeutil.KubeAnnDynamicallyProvisioned,\n\t\t\t).WithSpecField(\"claimRef\", map[string]any{\n\t\t\t\t\"namespace\": \"ns-1\",\n\t\t\t\t\"name\": \"pvc-1\",\n\t\t\t\t\"uid\": \"abc\",\n\t\t\t\t\"resourceVersion\": \"1\"}).Unstructured,\n\t\t\texpected: newTestUnstructured().WithMetadataField(\"kind\", \"persistentVolume\").\n\t\t\t\tWithName(\"pv-1\").\n\t\t\t\tWithAnnotations(kubeutil.KubeAnnDynamicallyProvisioned).\n\t\t\t\tWithSpecField(\"claimRef\", map[string]any{\n\t\t\t\t\t\"namespace\": \"ns-1\", \"name\": \"pvc-1\"}).Unstructured,\n\t\t},\n\t\t{\n\t\t\tname: \"PVCs that are bound have their binding annotations removed, but the volume name stays\",\n\t\t\tobj: newTestUnstructured().WithMetadataField(\"kind\", \"persistentVolumeClaim\").\n\t\t\t\tWithName(\"pvc-1\").WithAnnotations(\n\t\t\t\tkubeutil.KubeAnnBindCompleted,\n\t\t\t\tkubeutil.KubeAnnBoundByController,\n\t\t\t).WithSpecField(\"volumeName\", \"pv-1\").Unstructured,\n\t\t\texpected: newTestUnstructured().WithMetadataField(\"kind\", \"persistentVolumeClaim\").\n\t\t\t\tWithName(\"pvc-1\").WithAnnotations().\n\t\t\t\tWithSpecField(\"volumeName\", \"pv-1\").Unstructured,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactual := resetVolumeBindingInfo(tc.obj)\n\t\t\tassert.Equal(t, tc.expected, actual)\n\t\t})\n\t}\n}\n\nfunc TestIsAlreadyExistsError(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tapiResource *test.APIResource\n\t\tobj *unstructured.Unstructured\n\t\terr error\n\t\texpected bool\n\t}{\n\t\t{\n\t\t\tname: \"The input error is IsAlreadyExists error\",\n\t\t\terr: apierrors.NewAlreadyExists(schema.GroupResource{}, \"\"),\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"The input obj isn't service\",\n\t\t\tobj: &unstructured.Unstructured{\n\t\t\t\tObject: map[string]any{\n\t\t\t\t\t\"kind\": \"Pod\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"The StatusError contains no causes\",\n\t\t\tobj: &unstructured.Unstructured{\n\t\t\t\tObject: map[string]any{\n\t\t\t\t\t\"kind\": \"Service\",\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: &apierrors.StatusError{\n\t\t\t\tErrStatus: metav1.Status{\n\t\t\t\t\tReason: metav1.StatusReasonInvalid,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"The causes contains not only port already allocated error\",\n\t\t\tobj: &unstructured.Unstructured{\n\t\t\t\tObject: map[string]any{\n\t\t\t\t\t\"kind\": \"Service\",\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: &apierrors.StatusError{\n\t\t\t\tErrStatus: metav1.Status{\n\t\t\t\t\tReason: metav1.StatusReasonInvalid,\n\t\t\t\t\tDetails: &metav1.StatusDetails{\n\t\t\t\t\t\tCauses: []metav1.StatusCause{\n\t\t\t\t\t\t\t{Message: \"provided port is already allocated\"},\n\t\t\t\t\t\t\t{Message: \"other error\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Get already allocated error but the service doesn't exist\",\n\t\t\tobj: &unstructured.Unstructured{\n\t\t\t\tObject: map[string]any{\n\t\t\t\t\t\"kind\": \"Service\",\n\t\t\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\t\t\"namespace\": \"default\",\n\t\t\t\t\t\t\"name\": \"test\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: &apierrors.StatusError{\n\t\t\t\tErrStatus: metav1.Status{\n\t\t\t\t\tReason: metav1.StatusReasonInvalid,\n\t\t\t\t\tDetails: &metav1.StatusDetails{\n\t\t\t\t\t\tCauses: []metav1.StatusCause{\n\t\t\t\t\t\t\t{Message: \"provided port is already allocated\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Get already allocated error and the service exists\",\n\t\t\tapiResource: test.Services(\n\t\t\t\tbuilder.ForService(\"default\", \"test\").Result(),\n\t\t\t),\n\t\t\tobj: &unstructured.Unstructured{\n\t\t\t\tObject: map[string]any{\n\t\t\t\t\t\"kind\": \"Service\",\n\t\t\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\t\t\"namespace\": \"default\",\n\t\t\t\t\t\t\"name\": \"test\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: &apierrors.StatusError{\n\t\t\t\tErrStatus: metav1.Status{\n\t\t\t\t\tReason: metav1.StatusReasonInvalid,\n\t\t\t\t\tDetails: &metav1.StatusDetails{\n\t\t\t\t\t\tCauses: []metav1.StatusCause{\n\t\t\t\t\t\t\t{Message: \"provided port is already allocated\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: true,\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\th := newHarness(t)\n\n\t\tctx := &restoreContext{\n\t\t\tlog: h.log,\n\t\t\tdynamicFactory: client.NewDynamicFactory(h.DynamicClient),\n\t\t\tnamespaceClient: h.KubeClient.CoreV1().Namespaces(),\n\t\t\tresourceDeletionStatusTracker: kube.NewResourceDeletionStatusTracker(),\n\t\t}\n\n\t\tif test.apiResource != nil {\n\t\t\th.AddItems(t, test.apiResource)\n\t\t}\n\n\t\tclient, err := ctx.dynamicFactory.ClientForGroupVersionResource(\n\t\t\tschema.GroupVersion{Group: \"\", Version: \"v1\"},\n\t\t\tmetav1.APIResource{Name: \"services\"},\n\t\t\t\"default\",\n\t\t)\n\t\trequire.NoError(t, err)\n\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tresult, err := isAlreadyExistsError(ctx, test.obj, test.err, client)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tassert.Equal(t, test.expected, result)\n\t\t})\n\t}\n}\n\nfunc TestHasCSIVolumeSnapshot(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tvs *snapshotv1api.VolumeSnapshot\n\t\tobj *unstructured.Unstructured\n\t\texpectedResult bool\n\t}{\n\t\t{\n\t\t\tname: \"Invalid PV, expect false.\",\n\t\t\tobj: &unstructured.Unstructured{\n\t\t\t\tObject: map[string]any{\n\t\t\t\t\t\"kind\": 1,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedResult: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Cannot find VS, expect false\",\n\t\t\tobj: &unstructured.Unstructured{\n\t\t\t\tObject: map[string]any{\n\t\t\t\t\t\"kind\": \"PersistentVolume\",\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\t\t\"namespace\": \"default\",\n\t\t\t\t\t\t\"name\": \"test\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedResult: false,\n\t\t},\n\t\t{\n\t\t\tname: \"VS's source PVC is nil, expect false\",\n\t\t\tobj: &unstructured.Unstructured{\n\t\t\t\tObject: map[string]any{\n\t\t\t\t\t\"kind\": \"PersistentVolume\",\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\t\t\"namespace\": \"default\",\n\t\t\t\t\t\t\"name\": \"test\",\n\t\t\t\t\t},\n\t\t\t\t\t\"spec\": map[string]any{\n\t\t\t\t\t\t\"claimRef\": map[string]any{\n\t\t\t\t\t\t\t\"namespace\": \"velero\",\n\t\t\t\t\t\t\t\"name\": \"test\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvs: builder.ForVolumeSnapshot(\"velero\", \"test\").Result(),\n\t\t\texpectedResult: false,\n\t\t},\n\t\t{\n\t\t\tname: \"PVs claimref is nil, expect false.\",\n\t\t\tobj: &unstructured.Unstructured{\n\t\t\t\tObject: map[string]any{\n\t\t\t\t\t\"kind\": \"PersistentVolume\",\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\t\t\"namespace\": \"velero\",\n\t\t\t\t\t\t\"name\": \"test\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvs: builder.ForVolumeSnapshot(\"velero\", \"test\").SourcePVC(\"test\").Result(),\n\t\t\texpectedResult: false,\n\t\t},\n\n\t\t{\n\t\t\tname: \"Find VS, expect true.\",\n\t\t\tobj: &unstructured.Unstructured{\n\t\t\t\tObject: map[string]any{\n\t\t\t\t\t\"kind\": \"PersistentVolume\",\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\t\t\"namespace\": \"velero\",\n\t\t\t\t\t\t\"name\": \"test\",\n\t\t\t\t\t},\n\t\t\t\t\t\"spec\": map[string]any{\n\t\t\t\t\t\t\"claimRef\": map[string]any{\n\t\t\t\t\t\t\t\"namespace\": \"velero\",\n\t\t\t\t\t\t\t\"name\": \"test\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvs: builder.ForVolumeSnapshot(\"velero\", \"test\").SourcePVC(\"test\").Result(),\n\t\t\texpectedResult: true,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\th := newHarness(t)\n\n\t\tctx := &restoreContext{\n\t\t\tlog: h.log,\n\t\t\tresourceDeletionStatusTracker: kube.NewResourceDeletionStatusTracker(),\n\t\t}\n\n\t\tif tc.vs != nil {\n\t\t\tctx.csiVolumeSnapshots = []*snapshotv1api.VolumeSnapshot{tc.vs}\n\t\t}\n\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\trequire.Equal(t, tc.expectedResult, hasCSIVolumeSnapshot(ctx, tc.obj))\n\t\t})\n\t}\n}\n\nfunc TestHasSnapshotDataUpload(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tduResult *corev1api.ConfigMap\n\t\tobj *unstructured.Unstructured\n\t\texpectedResult bool\n\t\trestore *velerov1api.Restore\n\t}{\n\t\t{\n\t\t\tname: \"Invalid PV, expect false.\",\n\t\t\tobj: &unstructured.Unstructured{\n\t\t\t\tObject: map[string]any{\n\t\t\t\t\t\"kind\": 1,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedResult: false,\n\t\t},\n\t\t{\n\t\t\tname: \"PV without ClaimRef, expect false\",\n\t\t\tobj: &unstructured.Unstructured{\n\t\t\t\tObject: map[string]any{\n\t\t\t\t\t\"kind\": \"PersistentVolume\",\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\t\t\"namespace\": \"default\",\n\t\t\t\t\t\t\"name\": \"test\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tduResult: builder.ForConfigMap(\"velero\", \"test\").Result(),\n\t\t\trestore: builder.ForRestore(\"velero\", \"test\").ObjectMeta(builder.WithUID(\"fakeUID\")).Result(),\n\t\t\texpectedResult: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Cannot find DataUploadResult CM, expect false\",\n\t\t\tobj: &unstructured.Unstructured{\n\t\t\t\tObject: map[string]any{\n\t\t\t\t\t\"kind\": \"PersistentVolume\",\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\t\t\"namespace\": \"default\",\n\t\t\t\t\t\t\"name\": \"test\",\n\t\t\t\t\t},\n\t\t\t\t\t\"spec\": map[string]any{\n\t\t\t\t\t\t\"claimRef\": map[string]any{\n\t\t\t\t\t\t\t\"namespace\": \"velero\",\n\t\t\t\t\t\t\t\"name\": \"testPVC\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tduResult: builder.ForConfigMap(\"velero\", \"test\").Result(),\n\t\t\trestore: builder.ForRestore(\"velero\", \"test\").ObjectMeta(builder.WithUID(\"fakeUID\")).Result(),\n\t\t\texpectedResult: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Find DataUploadResult CM, expect true\",\n\t\t\tobj: &unstructured.Unstructured{\n\t\t\t\tObject: map[string]any{\n\t\t\t\t\t\"kind\": \"PersistentVolume\",\n\t\t\t\t\t\"apiVersion\": \"v1\",\n\t\t\t\t\t\"metadata\": map[string]any{\n\t\t\t\t\t\t\"namespace\": \"default\",\n\t\t\t\t\t\t\"name\": \"test\",\n\t\t\t\t\t},\n\t\t\t\t\t\"spec\": map[string]any{\n\t\t\t\t\t\t\"claimRef\": map[string]any{\n\t\t\t\t\t\t\t\"namespace\": \"velero\",\n\t\t\t\t\t\t\t\"name\": \"testPVC\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tduResult: builder.ForConfigMap(\"velero\", \"test\").ObjectMeta(builder.WithLabelsMap(map[string]string{\n\t\t\t\tvelerov1api.RestoreUIDLabel: \"fakeUID\",\n\t\t\t\tvelerov1api.PVCNamespaceNameLabel: \"velero/testPVC\",\n\t\t\t\tvelerov1api.ResourceUsageLabel: string(velerov1api.VeleroResourceUsageDataUploadResult),\n\t\t\t})).Result(),\n\t\t\trestore: builder.ForRestore(\"velero\", \"test\").ObjectMeta(builder.WithUID(\"fakeUID\")).Result(),\n\t\t\texpectedResult: false,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\th := newHarness(t)\n\n\t\tctx := &restoreContext{\n\t\t\tlog: h.log,\n\t\t\tkbClient: h.restorer.kbClient,\n\t\t\trestore: tc.restore,\n\t\t\tresourceDeletionStatusTracker: kube.NewResourceDeletionStatusTracker(),\n\t\t}\n\n\t\tif tc.duResult != nil {\n\t\t\trequire.NoError(t, ctx.kbClient.Create(t.Context(), tc.duResult))\n\t\t}\n\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\trequire.Equal(t, tc.expectedResult, hasSnapshotDataUpload(ctx, tc.obj))\n\t\t})\n\t}\n}\n\nfunc TestDetermineRestoreStatus(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tannotations map[string]string\n\t\trestoreSpecIncludes *bool\n\t\texpectedDecision bool\n\t}{\n\t\t{\n\t\t\tname: \"No annotation, fallback to restore spec\",\n\t\t\tannotations: nil,\n\t\t\trestoreSpecIncludes: boolptr.True(),\n\t\t\texpectedDecision: true,\n\t\t},\n\t\t{\n\t\t\tname: \"No annotation, restore spec excludes\",\n\t\t\tannotations: nil,\n\t\t\trestoreSpecIncludes: boolptr.False(),\n\t\t\texpectedDecision: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Annotation explicitly set to true, restore spec is false\",\n\t\t\tannotations: map[string]string{ObjectStatusRestoreAnnotationKey: \"true\"},\n\t\t\trestoreSpecIncludes: boolptr.False(),\n\t\t\texpectedDecision: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Annotation explicitly set to false, restore spec is true\",\n\t\t\tannotations: map[string]string{ObjectStatusRestoreAnnotationKey: \"false\"},\n\t\t\trestoreSpecIncludes: boolptr.True(),\n\t\t\texpectedDecision: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Invalid annotation value, fallback to restore spec\",\n\t\t\tannotations: map[string]string{ObjectStatusRestoreAnnotationKey: \"foo\"},\n\t\t\trestoreSpecIncludes: boolptr.True(),\n\t\t\texpectedDecision: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Empty annotation value, fallback to restore spec\",\n\t\t\tannotations: map[string]string{ObjectStatusRestoreAnnotationKey: \"\"},\n\t\t\trestoreSpecIncludes: boolptr.False(),\n\t\t\texpectedDecision: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Mixed-case annotation value 'True' should be treated as true\",\n\t\t\tannotations: map[string]string{ObjectStatusRestoreAnnotationKey: \"True\"},\n\t\t\trestoreSpecIncludes: boolptr.True(),\n\t\t\texpectedDecision: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Mixed-case annotation value 'FALSE' should be treated as false\",\n\t\t\tannotations: map[string]string{ObjectStatusRestoreAnnotationKey: \"FALSE\"},\n\t\t\trestoreSpecIncludes: boolptr.True(),\n\t\t\texpectedDecision: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Nil IncludesExcludes, but annotation is 'true'\",\n\t\t\tannotations: map[string]string{ObjectStatusRestoreAnnotationKey: \"true\"},\n\t\t\trestoreSpecIncludes: nil,\n\t\t\texpectedDecision: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Nil IncludesExcludes, but annotation is 'false'\",\n\t\t\tannotations: map[string]string{ObjectStatusRestoreAnnotationKey: \"false\"},\n\t\t\trestoreSpecIncludes: nil,\n\t\t\texpectedDecision: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Nil IncludesExcludes, no annotation (default to false)\",\n\t\t\tannotations: nil,\n\t\t\trestoreSpecIncludes: nil,\n\t\t\texpectedDecision: false,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tobj := &unstructured.Unstructured{}\n\t\t\tobj.SetAnnotations(test.annotations)\n\n\t\t\tvar includesExcludes *collections.IncludesExcludes\n\t\t\tif test.restoreSpecIncludes != nil {\n\t\t\t\tincludesExcludes = collections.NewIncludesExcludes()\n\t\t\t\tif *test.restoreSpecIncludes {\n\t\t\t\t\tincludesExcludes.Includes(\"*\")\n\t\t\t\t} else {\n\t\t\t\t\tincludesExcludes.Excludes(\"*\")\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tlog := logrus.New()\n\t\t\tresult := determineRestoreStatus(obj, includesExcludes, \"testGroupResource\", log)\n\n\t\t\tassert.Equal(t, test.expectedDecision, result)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/restore/restore_wildcard_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restore\n\nimport (\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/archive\"\n)\n\nfunc TestExpandNamespaceWildcards(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tincludeNamespaces []string\n\t\texcludeNamespaces []string\n\t\tbackupResources map[string]*archive.ResourceItems\n\t\texpectedIncludeMatches []string\n\t\texpectedExcludeMatches []string\n\t\texpectedWildcardResult []string\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"No wildcards - should not expand\",\n\t\t\tincludeNamespaces: []string{\"ns1\", \"ns2\"},\n\t\t\texcludeNamespaces: []string{\"ns3\"},\n\t\t\tbackupResources: map[string]*archive.ResourceItems{\n\t\t\t\t\"namespaces\": {ItemsByNamespace: map[string][]string{\"ns1\": {}, \"ns2\": {}, \"ns3\": {}}},\n\t\t\t},\n\t\t\texpectedIncludeMatches: nil,\n\t\t\texpectedExcludeMatches: nil,\n\t\t\texpectedWildcardResult: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"Simple wildcard include pattern\",\n\t\t\tincludeNamespaces: []string{\"test*\"},\n\t\t\texcludeNamespaces: []string{},\n\t\t\tbackupResources: map[string]*archive.ResourceItems{\n\t\t\t\t\"namespaces\": {ItemsByNamespace: map[string][]string{\"test1\": {}, \"test2\": {}, \"prod1\": {}}},\n\t\t\t},\n\t\t\texpectedIncludeMatches: []string{\"test1\", \"test2\"},\n\t\t\texpectedExcludeMatches: []string{},\n\t\t\texpectedWildcardResult: []string{\"test1\", \"test2\"},\n\t\t},\n\t\t{\n\t\t\tname: \"Multiple wildcard patterns\",\n\t\t\tincludeNamespaces: []string{\"test*\", \"dev*\"},\n\t\t\texcludeNamespaces: []string{},\n\t\t\tbackupResources: map[string]*archive.ResourceItems{\n\t\t\t\t\"namespaces\": {ItemsByNamespace: map[string][]string{\"test1\": {}, \"test2\": {}, \"dev1\": {}, \"prod1\": {}}},\n\t\t\t},\n\t\t\texpectedIncludeMatches: []string{\"dev1\", \"test1\", \"test2\"},\n\t\t\texpectedExcludeMatches: []string{},\n\t\t\texpectedWildcardResult: []string{\"dev1\", \"test1\", \"test2\"},\n\t\t},\n\t\t{\n\t\t\tname: \"Wildcard include with wildcard exclude\",\n\t\t\tincludeNamespaces: []string{\"test*\"},\n\t\t\texcludeNamespaces: []string{\"*-temp\"},\n\t\t\tbackupResources: map[string]*archive.ResourceItems{\n\t\t\t\t\"namespaces\": {ItemsByNamespace: map[string][]string{\"test1\": {}, \"test2-temp\": {}, \"test3\": {}}},\n\t\t\t},\n\t\t\texpectedIncludeMatches: []string{\"test1\", \"test2-temp\", \"test3\"},\n\t\t\texpectedExcludeMatches: []string{\"test2-temp\"},\n\t\t\texpectedWildcardResult: []string{\"test1\", \"test3\"},\n\t\t},\n\t\t{\n\t\t\tname: \"Wildcard include with literal exclude\",\n\t\t\tincludeNamespaces: []string{\"app-*\"},\n\t\t\texcludeNamespaces: []string{\"app-test\"},\n\t\t\tbackupResources: map[string]*archive.ResourceItems{\n\t\t\t\t\"namespaces\": {ItemsByNamespace: map[string][]string{\"app-prod\": {}, \"app-test\": {}, \"app-dev\": {}}},\n\t\t\t},\n\t\t\texpectedIncludeMatches: []string{\"app-dev\", \"app-prod\", \"app-test\"},\n\t\t\texpectedExcludeMatches: []string{\"app-test\"},\n\t\t\texpectedWildcardResult: []string{\"app-dev\", \"app-prod\"},\n\t\t},\n\t\t{\n\t\t\tname: \"Error: wildcard * in excludes\",\n\t\t\tincludeNamespaces: []string{\"test*\"},\n\t\t\texcludeNamespaces: []string{\"*\"},\n\t\t\tbackupResources: map[string]*archive.ResourceItems{\n\t\t\t\t\"namespaces\": {ItemsByNamespace: map[string][]string{\"test1\": {}}},\n\t\t\t},\n\t\t\texpectedError: \"wildcard '*' is not allowed in restore excludes\",\n\t\t},\n\t\t{\n\t\t\tname: \"Empty backup - no matches\",\n\t\t\tincludeNamespaces: []string{\"test*\"},\n\t\t\texcludeNamespaces: []string{},\n\t\t\tbackupResources: map[string]*archive.ResourceItems{},\n\t\t\texpectedIncludeMatches: []string{},\n\t\t\texpectedExcludeMatches: []string{},\n\t\t\texpectedWildcardResult: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"Wildcard with no matches\",\n\t\t\tincludeNamespaces: []string{\"nonexistent*\"},\n\t\t\texcludeNamespaces: []string{},\n\t\t\tbackupResources: map[string]*archive.ResourceItems{\n\t\t\t\t\"namespaces\": {ItemsByNamespace: map[string][]string{\"test1\": {}, \"test2\": {}}},\n\t\t\t},\n\t\t\texpectedIncludeMatches: []string{},\n\t\t\texpectedExcludeMatches: []string{},\n\t\t\texpectedWildcardResult: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"Complex pattern with prefix and suffix\",\n\t\t\tincludeNamespaces: []string{\"app-*-prod\"},\n\t\t\texcludeNamespaces: []string{},\n\t\t\tbackupResources: map[string]*archive.ResourceItems{\n\t\t\t\t\"namespaces\": {ItemsByNamespace: map[string][]string{\"app-frontend-prod\": {}, \"app-backend-prod\": {}, \"app-frontend-dev\": {}}},\n\t\t\t},\n\t\t\texpectedIncludeMatches: []string{\"app-backend-prod\", \"app-frontend-prod\"},\n\t\t\texpectedExcludeMatches: []string{},\n\t\t\texpectedWildcardResult: []string{\"app-backend-prod\", \"app-frontend-prod\"},\n\t\t},\n\t\t{\n\t\t\tname: \"Backup with cluster resources\",\n\t\t\tincludeNamespaces: []string{\"test*\"},\n\t\t\texcludeNamespaces: []string{},\n\t\t\tbackupResources: map[string]*archive.ResourceItems{\n\t\t\t\t\"namespaces\": {ItemsByNamespace: map[string][]string{\"test1\": {}, \"test2\": {}}},\n\t\t\t\t\"persistentvolumes\": {ItemsByNamespace: map[string][]string{\"\": {}}}, // cluster-scoped\n\t\t\t\t\"pods.v1\": {ItemsByNamespace: map[string][]string{\"test1\": {\"pod1\"}}},\n\t\t\t},\n\t\t\texpectedIncludeMatches: []string{\"test1\", \"test2\"},\n\t\t\texpectedExcludeMatches: []string{},\n\t\t\texpectedWildcardResult: []string{\"test1\", \"test2\"},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\trestore := &velerov1api.Restore{\n\t\t\t\tSpec: velerov1api.RestoreSpec{\n\t\t\t\t\tIncludedNamespaces: tc.includeNamespaces,\n\t\t\t\t\tExcludedNamespaces: tc.excludeNamespaces,\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tctx := &restoreContext{\n\t\t\t\trestore: restore,\n\t\t\t\tlog: logrus.StandardLogger(),\n\t\t\t}\n\n\t\t\terr := ctx.expandNamespaceWildcards(tc.backupResources)\n\n\t\t\tif tc.expectedError != \"\" {\n\t\t\t\trequire.Error(t, err)\n\t\t\t\tassert.Contains(t, err.Error(), tc.expectedError)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\trequire.NoError(t, err)\n\t\t})\n\t}\n}\n\nfunc TestExtractNamespacesFromBackup(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackupResources map[string]*archive.ResourceItems\n\t\texpected []string\n\t}{\n\t\t{\n\t\t\tname: \"Multiple namespaces in backup\",\n\t\t\tbackupResources: map[string]*archive.ResourceItems{\n\t\t\t\t\"namespaces\": {ItemsByNamespace: map[string][]string{\"ns1\": {}, \"ns2\": {}, \"ns3\": {}}},\n\t\t\t},\n\t\t\texpected: []string{\"ns1\", \"ns2\", \"ns3\"},\n\t\t},\n\t\t{\n\t\t\tname: \"Namespaces with resources\",\n\t\t\tbackupResources: map[string]*archive.ResourceItems{\n\t\t\t\t\"namespaces\": {ItemsByNamespace: map[string][]string{\"app1\": {}, \"app2\": {}}},\n\t\t\t\t\"pods.v1\": {ItemsByNamespace: map[string][]string{\"app1\": {\"pod1\"}}},\n\t\t\t\t\"services.v1\": {ItemsByNamespace: map[string][]string{\"app2\": {\"svc1\"}}},\n\t\t\t},\n\t\t\texpected: []string{\"app1\", \"app2\"},\n\t\t},\n\t\t{\n\t\t\tname: \"Mixed cluster and namespaced resources\",\n\t\t\tbackupResources: map[string]*archive.ResourceItems{\n\t\t\t\t\"namespaces\": {ItemsByNamespace: map[string][]string{\"test\": {}}},\n\t\t\t\t\"persistentvolumes\": {ItemsByNamespace: map[string][]string{\"\": {\"pv1\"}}},\n\t\t\t\t\"clusterroles\": {ItemsByNamespace: map[string][]string{\"\": {\"cr1\"}}},\n\t\t\t\t\"pods.v1\": {ItemsByNamespace: map[string][]string{\"test\": {\"pod1\"}}},\n\t\t\t},\n\t\t\texpected: []string{\"test\"},\n\t\t},\n\t\t{\n\t\t\tname: \"Empty backup\",\n\t\t\tbackupResources: map[string]*archive.ResourceItems{},\n\t\t\texpected: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"Only cluster resources\",\n\t\t\tbackupResources: map[string]*archive.ResourceItems{\n\t\t\t\t\"persistentvolumes\": {ItemsByNamespace: map[string][]string{\"\": {\"pv1\"}}},\n\t\t\t\t\"clusterroles\": {ItemsByNamespace: map[string][]string{\"\": {\"cr1\"}}},\n\t\t\t\t\"storageclasses\": {ItemsByNamespace: map[string][]string{\"\": {\"sc1\"}}},\n\t\t\t},\n\t\t\texpected: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"Duplicate namespace entries\",\n\t\t\tbackupResources: map[string]*archive.ResourceItems{\n\t\t\t\t\"namespaces\": {ItemsByNamespace: map[string][]string{\"app\": {}}},\n\t\t\t\t\"pods.v1\": {ItemsByNamespace: map[string][]string{\"app\": {\"pod1\"}}},\n\t\t\t\t\"configmaps.v1\": {ItemsByNamespace: map[string][]string{\"app\": {\"cm1\"}}},\n\t\t\t},\n\t\t\texpected: []string{\"app\"},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tresult := extractNamespacesFromBackup(tc.backupResources)\n\t\t\tassert.ElementsMatch(t, tc.expected, result)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/restorehelper/util.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage restorehelper\n\nconst (\n\t// WaitInitContainer is the name of the init container added\n\t// to workload pods to help with restores.\n\t// If Velero needs to further process the volume data after PVC is\n\t// provisioned, this init container is used to block Pod from running\n\t// until the volume data is ready\n\tWaitInitContainer = \"restore-wait\"\n\n\t// This is the name of the init container added by pre-v1.10 for the same\n\t// purpose with WaitInitContainer.\n\t// For compatibility, we need to check it when restoring backups created by\n\t// old releases. The pods backed up by old releases may contain this init container\n\t// since the init container is not deleted after pod is restored.\n\tWaitInitContainerLegacy = \"restic-wait\"\n)\n" }, { "path": "pkg/test/api_server.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage test\n\nimport (\n\t\"testing\"\n\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\tdiscoveryfake \"k8s.io/client-go/discovery/fake\"\n\tdynamicfake \"k8s.io/client-go/dynamic/fake\"\n\tkubefake \"k8s.io/client-go/kubernetes/fake\"\n)\n\n// APIServer contains in-memory fakes for all of the relevant\n// Kubernetes API server clients.\ntype APIServer struct {\n\tKubeClient *kubefake.Clientset\n\tDynamicClient *dynamicfake.FakeDynamicClient\n\tDiscoveryClient *DiscoveryClient\n}\n\n// NewAPIServer constructs an APIServer with all of its clients\n// initialized.\nfunc NewAPIServer(t *testing.T) *APIServer {\n\tt.Helper()\n\n\tvar (\n\t\tkubeClient = kubefake.NewSimpleClientset()\n\t\tdynamicClient = dynamicfake.NewSimpleDynamicClientWithCustomListKinds(runtime.NewScheme(),\n\t\t\tmap[schema.GroupVersionResource]string{\n\t\t\t\t{Group: \"\", Version: \"v1\", Resource: \"namespaces\"}: \"NamespacesList\",\n\t\t\t\t{Group: \"\", Version: \"v1\", Resource: \"pods\"}: \"PodsList\",\n\t\t\t\t{Group: \"\", Version: \"v1\", Resource: \"persistentvolumes\"}: \"PVList\",\n\t\t\t\t{Group: \"\", Version: \"v1\", Resource: \"persistentvolumeclaims\"}: \"PVCList\",\n\t\t\t\t{Group: \"\", Version: \"v1\", Resource: \"secrets\"}: \"SecretsList\",\n\t\t\t\t{Group: \"\", Version: \"v1\", Resource: \"serviceaccounts\"}: \"ServiceAccountsList\",\n\t\t\t\t{Group: \"apps\", Version: \"v1\", Resource: \"deployments\"}: \"DeploymentsList\",\n\t\t\t\t{Group: \"apiextensions.k8s.io\", Version: \"v1beta1\", Resource: \"customresourcedefinitions\"}: \"CRDList\",\n\t\t\t\t{Group: \"velero.io\", Version: \"v1\", Resource: \"volumesnapshotlocations\"}: \"VSLList\",\n\t\t\t\t{Group: \"velero.io\", Version: \"v1\", Resource: \"backups\"}: \"BackupList\",\n\t\t\t\t{Group: \"extensions\", Version: \"v1\", Resource: \"deployments\"}: \"ExtDeploymentsList\",\n\t\t\t\t{Group: \"velero.io\", Version: \"v1\", Resource: \"deployments\"}: \"VeleroDeploymentsList\",\n\t\t\t\t{Group: \"velero.io\", Version: \"v2alpha1\", Resource: \"datauploads\"}: \"DataUploadsList\",\n\t\t\t})\n\t\tdiscoveryClient = &DiscoveryClient{FakeDiscovery: kubeClient.Discovery().(*discoveryfake.FakeDiscovery)}\n\t)\n\n\treturn &APIServer{\n\t\tKubeClient: kubeClient,\n\t\tDynamicClient: dynamicClient,\n\t\tDiscoveryClient: discoveryClient,\n\t}\n}\n" }, { "path": "pkg/test/comparisons.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage test\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"reflect\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/google/go-cmp/cmp\"\n\t\"github.com/google/go-cmp/cmp/cmpopts\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/api/equality\"\n\t\"k8s.io/apimachinery/pkg/util/diff\"\n\tcore \"k8s.io/client-go/testing\"\n)\n\n// CompareActions checks slices of actual and expected Actions\n// for equality (ignoring order). It checks that the lengths of\n// the slices are the same, that each actual Action has a\n// corresponding expected Action, and that each expected Action\n// has a corresponding actual Action.\nfunc CompareActions(t *testing.T, expected, actual []core.Action) {\n\tt.Helper()\n\tassert.Len(t, actual, len(expected))\n\n\tfor _, e := range expected {\n\t\tfound := false\n\t\tfor _, a := range actual {\n\t\t\tif reflect.DeepEqual(e, a) {\n\t\t\t\tfound = true\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t\tif !found {\n\t\t\tt.Errorf(\"missing expected action %#v\", e)\n\t\t}\n\t}\n\n\tfor _, a := range actual {\n\t\tfound := false\n\t\tfor _, e := range expected {\n\t\t\tif reflect.DeepEqual(e, a) {\n\t\t\t\tfound = true\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t\tif !found {\n\t\t\tt.Errorf(\"unexpected action %#v\", a)\n\t\t}\n\t}\n}\n\n// ValidatePatch tests the validity of an action. It checks\n// that the action is a PatchAction, that the patch decodes from JSON\n// with the provided decode func and has no extraneous fields, and that\n// the decoded patch matches the expected.\nfunc ValidatePatch(t *testing.T, action core.Action, expected any, decodeFunc func(*json.Decoder) (any, error)) {\n\tt.Helper()\n\tpatchAction, ok := action.(core.PatchAction)\n\trequire.True(t, ok, \"action is not a PatchAction\")\n\n\tdecoder := json.NewDecoder(bytes.NewReader(patchAction.GetPatch()))\n\tdecoder.DisallowUnknownFields()\n\n\tactual, err := decodeFunc(decoder)\n\trequire.NoError(t, err)\n\n\tAssertDeepEqual(t, expected, actual)\n}\n\n// TimesAreEqual compares two times for equality.\n// This function is used by equality.Semantic.DeepEqual to compare two time objects\n// without having to call a method.\nfunc TimesAreEqual(t1, t2 time.Time) bool {\n\treturn t1.Equal(t2)\n}\n\n// AssertDeepEqual asserts the semantic equality of objects.\n// This function exists in order to make sure time.Time and metav1.Time objects\n// can be compared correctly. See https://github.com/stretchr/testify/issues/502.\nfunc AssertDeepEqual(t *testing.T, expected, actual any) bool {\n\tt.Helper()\n\t// By default, the equality.Semantic object doesn't have a function for comparing time.Times\n\terr := equality.Semantic.AddFunc(TimesAreEqual)\n\tif err != nil {\n\t\t// Programmer error, the test should die.\n\t\tt.Fatalf(\"Could not register equality function: %s\", err)\n\t}\n\n\tif !equality.Semantic.DeepEqual(expected, actual) {\n\t\ts := diff.ObjectDiff(expected, actual)\n\t\treturn assert.Fail(t, fmt.Sprintf(\"Objects not equal:\\n\\n%s\", s))\n\t}\n\n\treturn true\n}\n\n// AssertErrorMatches asserts that if expected is the empty string, actual\n// is nil, otherwise, that actual's error string matches expected.\nfunc AssertErrorMatches(t *testing.T, expected string, actual error) bool {\n\tt.Helper()\n\tif expected != \"\" {\n\t\treturn assert.EqualError(t, actual, expected)\n\t}\n\n\treturn assert.NoError(t, actual)\n}\n\nfunc CompareSlice(x, y []string) bool {\n\tless := func(a, b string) bool { return a < b }\n\treturn cmp.Diff(x, y, cmpopts.SortSlices(less)) == \"\"\n}\n" }, { "path": "pkg/test/discovery_client.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage test\n\nimport (\n\t\"strings\"\n\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/client-go/discovery\"\n\tdiscoveryfake \"k8s.io/client-go/discovery/fake\"\n)\n\n// DiscoveryClient is a wrapper for the client-go FakeDiscovery struct. It\n// adds some extra functionality that's necessary/useful for Velero tests.\ntype DiscoveryClient struct {\n\t*discoveryfake.FakeDiscovery\n}\n\nfunc (c *DiscoveryClient) ServerPreferredResources() ([]*metav1.APIResourceList, error) {\n\treturn discovery.ServerPreferredResources(c)\n}\n\n//\n// TEST HELPERS\n//\n\n// WithAPIResource adds the API resource to the discovery client.\nfunc (c *DiscoveryClient) WithAPIResource(resource *APIResource) *DiscoveryClient {\n\tgv := metav1.GroupVersion{\n\t\tGroup: resource.Group,\n\t\tVersion: resource.Version,\n\t}\n\n\tvar resourceList *metav1.APIResourceList\n\tfor _, itm := range c.Resources {\n\t\tif itm.GroupVersion == gv.String() {\n\t\t\tresourceList = itm\n\t\t\tbreak\n\t\t}\n\t}\n\n\tif resourceList == nil {\n\t\tresourceList = &metav1.APIResourceList{\n\t\t\tGroupVersion: gv.String(),\n\t\t}\n\n\t\tc.Resources = append(c.Resources, resourceList)\n\t}\n\n\tfor _, itm := range resourceList.APIResources {\n\t\tif itm.Name == resource.Name {\n\t\t\treturn c\n\t\t}\n\t}\n\n\tresourceList.APIResources = append(resourceList.APIResources, metav1.APIResource{\n\t\tName: resource.Name,\n\t\tSingularName: strings.TrimSuffix(resource.Name, \"s\"),\n\t\tNamespaced: resource.Namespaced,\n\t\tGroup: resource.Group,\n\t\tVersion: resource.Version,\n\t\tKind: resource.Kind,\n\t\tVerbs: metav1.Verbs([]string{\"list\", \"create\", \"get\", \"delete\"}),\n\t\tShortNames: []string{resource.ShortName},\n\t})\n\n\treturn c\n}\n\nfunc (c *DiscoveryClient) GroupsAndMaybeResources() (*metav1.APIGroupList, map[schema.GroupVersion]*metav1.APIResourceList, map[schema.GroupVersion]error, error) {\n\tapiGroupList, err := c.ServerGroups()\n\tif err != nil {\n\t\treturn nil, nil, nil, err\n\t}\n\treturn apiGroupList, nil, nil, nil\n}\n" }, { "path": "pkg/test/fake_controller_runtime_client.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage test\n\nimport (\n\t\"testing\"\n\n\tvolumegroupsnapshotv1beta1 \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumegroupsnapshot/v1beta1\"\n\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\t\"github.com/stretchr/testify/require\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tbatchv1api \"k8s.io/api/batch/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tstoragev1api \"k8s.io/api/storage/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\tk8sfake \"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerov2alpha1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n)\n\nfunc NewFakeControllerRuntimeClientBuilder(t *testing.T) *k8sfake.ClientBuilder {\n\tt.Helper()\n\tscheme := runtime.NewScheme()\n\n\trequire.NoError(t, velerov1api.AddToScheme(scheme))\n\trequire.NoError(t, velerov2alpha1api.AddToScheme(scheme))\n\trequire.NoError(t, corev1api.AddToScheme(scheme))\n\trequire.NoError(t, appsv1api.AddToScheme(scheme))\n\trequire.NoError(t, snapshotv1api.AddToScheme(scheme))\n\trequire.NoError(t, storagev1api.AddToScheme(scheme))\n\n\treturn k8sfake.NewClientBuilder().WithScheme(scheme)\n}\n\nfunc NewFakeControllerRuntimeClient(t *testing.T, initObjs ...runtime.Object) client.Client {\n\tt.Helper()\n\tscheme := runtime.NewScheme()\n\n\trequire.NoError(t, velerov1api.AddToScheme(scheme))\n\trequire.NoError(t, velerov2alpha1api.AddToScheme(scheme))\n\trequire.NoError(t, corev1api.AddToScheme(scheme))\n\trequire.NoError(t, appsv1api.AddToScheme(scheme))\n\trequire.NoError(t, snapshotv1api.AddToScheme(scheme))\n\trequire.NoError(t, storagev1api.AddToScheme(scheme))\n\trequire.NoError(t, batchv1api.AddToScheme(scheme))\n\trequire.NoError(t, volumegroupsnapshotv1beta1.AddToScheme(scheme))\n\n\treturn k8sfake.NewClientBuilder().WithScheme(scheme).WithRuntimeObjects(initObjs...).Build()\n}\n\nfunc NewFakeControllerRuntimeWatchClient(\n\tt *testing.T,\n\tinitObjs ...runtime.Object,\n) client.WithWatch {\n\tt.Helper()\n\treturn NewFakeControllerRuntimeClientBuilder(t).WithRuntimeObjects(initObjs...).Build()\n}\n" }, { "path": "pkg/test/fake_credential_file_store.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage test\n\nimport (\n\tcorev1api \"k8s.io/api/core/v1\"\n)\n\n// FileStore defines operations for interacting with credentials\n// that are stored on a file system.\ntype FileStore interface {\n\t// Path returns a path on disk where the secret key defined by\n\t// the given selector is serialized.\n\tPath(selector *corev1api.SecretKeySelector) (string, error)\n}\n\ntype fakeCredentialsFileStore struct {\n\tpath string\n\terr error\n}\n\n// Path returns a path on disk where the secret key defined by\n// the given selector is serialized.\nfunc (f *fakeCredentialsFileStore) Path(*corev1api.SecretKeySelector) (string, error) {\n\treturn f.path, f.err\n}\n\n// NewFakeCredentialsFileStore creates a FileStore which will return the given path\n// and error when Path is called.\nfunc NewFakeCredentialsFileStore(path string, err error) FileStore {\n\treturn &fakeCredentialsFileStore{\n\t\tpath: path,\n\t\terr: err,\n\t}\n}\n\n// SecretStore defines operations for interacting with credentials\n// that are stored in Secret.\ntype SecretStore interface {\n\t// Get returns the secret key defined by the given selector\n\tGet(selector *corev1api.SecretKeySelector) (string, error)\n}\n\ntype fakeCredentialsSecretStore struct {\n\tdata string\n\terr error\n}\n\n// Get returns the secret data.\nfunc (f *fakeCredentialsSecretStore) Get(*corev1api.SecretKeySelector) (string, error) {\n\treturn f.data, f.err\n}\n\n// NewFakeCredentialsSecretStore creates a SecretStore which will return the given data\n// and error when Get is called.\n// data is the secret value to return (e.g., certificate content).\n// err is the error to return, if any.\nfunc NewFakeCredentialsSecretStore(data string, err error) SecretStore {\n\treturn &fakeCredentialsSecretStore{\n\t\tdata: data,\n\t\terr: err,\n\t}\n}\n" }, { "path": "pkg/test/fake_discovery_helper.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage test\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\n\t\"k8s.io/apimachinery/pkg/api/meta\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/apimachinery/pkg/version\"\n\t\"k8s.io/client-go/discovery\"\n)\n\ntype FakeDiscoveryHelper struct {\n\tResourceList []*metav1.APIResourceList\n\tMapper meta.RESTMapper\n\tAutoReturnResource bool\n\tAPIGroupsList []metav1.APIGroup\n\tServerVersionData *version.Info\n}\n\nfunc (dh *FakeDiscoveryHelper) KindFor(input schema.GroupVersionKind) (schema.GroupVersionResource, metav1.APIResource, error) {\n\tpanic(\"implement me\")\n}\n\nfunc NewFakeDiscoveryHelper(autoReturnResource bool, resources map[schema.GroupVersionResource]schema.GroupVersionResource) *FakeDiscoveryHelper {\n\thelper := &FakeDiscoveryHelper{\n\t\tAutoReturnResource: autoReturnResource,\n\t\tMapper: &FakeMapper{\n\t\t\tResources: resources,\n\t\t},\n\t}\n\n\tif resources == nil {\n\t\treturn helper\n\t}\n\n\tapiResourceMap := make(map[string][]metav1.APIResource)\n\n\tfor _, gvr := range resources {\n\t\tvar gvString string\n\t\tif gvr.Version != \"\" && gvr.Group != \"\" {\n\t\t\tgvString = fmt.Sprintf(\"%s/%s\", gvr.Group, gvr.Version)\n\t\t} else {\n\t\t\tgvString = fmt.Sprintf(\"%s%s\", gvr.Group, gvr.Version)\n\t\t}\n\n\t\tapiResourceMap[gvString] = append(apiResourceMap[gvString], metav1.APIResource{Name: gvr.Resource})\n\t\thelper.APIGroupsList = append(helper.APIGroupsList,\n\t\t\tmetav1.APIGroup{\n\t\t\t\tName: gvr.Group,\n\t\t\t\tPreferredVersion: metav1.GroupVersionForDiscovery{\n\t\t\t\t\tGroupVersion: gvString,\n\t\t\t\t\tVersion: gvr.Version,\n\t\t\t\t},\n\t\t\t})\n\t}\n\n\tfor group, resources := range apiResourceMap {\n\t\thelper.ResourceList = append(helper.ResourceList, &metav1.APIResourceList{GroupVersion: group, APIResources: resources})\n\t}\n\n\t// FakeTest of version.Info\n\n\tserverVersion := &version.Info{\n\t\tMajor: \"1\",\n\t\tMinor: \"16\",\n\t\tGitVersion: \"v1.16.4\",\n\t\tGitCommit: \"FakeTest\",\n\t\tGitTreeState: \"\",\n\t\tBuildDate: \"\",\n\t\tGoVersion: \"\",\n\t\tCompiler: \"\",\n\t\tPlatform: \"\",\n\t}\n\n\thelper.ServerVersionData = serverVersion\n\n\treturn helper\n}\n\nfunc (dh *FakeDiscoveryHelper) Resources() []*metav1.APIResourceList {\n\treturn dh.ResourceList\n}\n\nfunc (dh *FakeDiscoveryHelper) Refresh() error {\n\treturn nil\n}\n\nfunc (dh *FakeDiscoveryHelper) ResourceFor(input schema.GroupVersionResource) (schema.GroupVersionResource, metav1.APIResource, error) {\n\tif dh.AutoReturnResource {\n\t\treturn schema.GroupVersionResource{\n\t\t\t\tGroup: input.Group,\n\t\t\t\tVersion: input.Version,\n\t\t\t\tResource: input.Resource,\n\t\t\t},\n\t\t\tmetav1.APIResource{\n\t\t\t\tName: input.Resource,\n\t\t\t},\n\t\t\tnil\n\t}\n\n\tgvr, err := dh.Mapper.ResourceFor(input)\n\tif err != nil {\n\t\treturn schema.GroupVersionResource{}, metav1.APIResource{}, err\n\t}\n\n\tvar gvString string\n\tif gvr.Version != \"\" && gvr.Group != \"\" {\n\t\tgvString = fmt.Sprintf(\"%s/%s\", gvr.Group, gvr.Version)\n\t} else {\n\t\tgvString = gvr.Version + gvr.Group\n\t}\n\n\tfor _, gr := range dh.ResourceList {\n\t\tif gr.GroupVersion != gvString {\n\t\t\tcontinue\n\t\t}\n\n\t\tfor _, resource := range gr.APIResources {\n\t\t\tif resource.Name == gvr.Resource {\n\t\t\t\treturn gvr, resource, nil\n\t\t\t}\n\t\t}\n\t}\n\n\treturn schema.GroupVersionResource{}, metav1.APIResource{}, errors.New(\"APIResource not found\")\n}\n\nfunc (dh *FakeDiscoveryHelper) APIGroups() []metav1.APIGroup {\n\treturn dh.APIGroupsList\n}\n\ntype FakeServerResourcesInterface struct {\n\tResourceList []*metav1.APIResourceList\n\tAPIGroup []*metav1.APIGroup\n\tFailedGroups map[schema.GroupVersion]error\n\tReturnError error\n}\n\nfunc (di *FakeServerResourcesInterface) ServerPreferredResources() ([]*metav1.APIResourceList, error) {\n\tif di.ReturnError != nil {\n\t\treturn di.ResourceList, di.ReturnError\n\t}\n\tif len(di.FailedGroups) == 0 {\n\t\treturn di.ResourceList, nil\n\t}\n\treturn di.ResourceList, &discovery.ErrGroupDiscoveryFailed{Groups: di.FailedGroups}\n}\n\nfunc (di *FakeServerResourcesInterface) ServerGroupsAndResources() ([]*metav1.APIGroup, []*metav1.APIResourceList, error) {\n\tif di.ReturnError != nil {\n\t\treturn di.APIGroup, di.ResourceList, di.ReturnError\n\t}\n\tif len(di.FailedGroups) == 0 {\n\t\treturn di.APIGroup, di.ResourceList, nil\n\t}\n\treturn di.APIGroup, di.ResourceList, &discovery.ErrGroupDiscoveryFailed{Groups: di.FailedGroups}\n}\n\nfunc NewFakeServerResourcesInterface(resourceList []*metav1.APIResourceList, apiGroup []*metav1.APIGroup, failedGroups map[schema.GroupVersion]error, returnError error) *FakeServerResourcesInterface {\n\thelper := &FakeServerResourcesInterface{\n\t\tResourceList: resourceList,\n\t\tAPIGroup: apiGroup,\n\t\tFailedGroups: failedGroups,\n\t\tReturnError: returnError,\n\t}\n\treturn helper\n}\n\nfunc (dh *FakeDiscoveryHelper) ServerVersion() *version.Info {\n\treturn dh.ServerVersionData\n}\n" }, { "path": "pkg/test/fake_dynamic.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage test\n\nimport (\n\t\"github.com/stretchr/testify/mock\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/apimachinery/pkg/watch\"\n\t\"k8s.io/client-go/dynamic/dynamicinformer\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/client\"\n)\n\ntype FakeDynamicFactory struct {\n\tmock.Mock\n}\n\nvar _ client.DynamicFactory = &FakeDynamicFactory{}\n\nfunc (df *FakeDynamicFactory) ClientForGroupVersionResource(gv schema.GroupVersion, resource metav1.APIResource, namespace string) (client.Dynamic, error) {\n\targs := df.Called(gv, resource, namespace)\n\treturn args.Get(0).(client.Dynamic), args.Error(1)\n}\n\nfunc (df *FakeDynamicFactory) DynamicSharedInformerFactory() dynamicinformer.DynamicSharedInformerFactory {\n\targs := df.Called()\n\treturn args.Get(0).(dynamicinformer.DynamicSharedInformerFactory)\n}\n\ntype FakeDynamicClient struct {\n\tmock.Mock\n}\n\nvar _ client.Dynamic = &FakeDynamicClient{}\n\nfunc (c *FakeDynamicClient) List(options metav1.ListOptions) (*unstructured.UnstructuredList, error) {\n\targs := c.Called(options)\n\treturn args.Get(0).(*unstructured.UnstructuredList), args.Error(1)\n}\n\nfunc (c *FakeDynamicClient) Create(obj *unstructured.Unstructured) (*unstructured.Unstructured, error) {\n\targs := c.Called(obj)\n\treturn args.Get(0).(*unstructured.Unstructured), args.Error(1)\n}\n\nfunc (c *FakeDynamicClient) Watch(options metav1.ListOptions) (watch.Interface, error) {\n\targs := c.Called(options)\n\treturn args.Get(0).(watch.Interface), args.Error(1)\n}\n\nfunc (c *FakeDynamicClient) Get(name string, opts metav1.GetOptions) (*unstructured.Unstructured, error) {\n\targs := c.Called(name, opts)\n\treturn args.Get(0).(*unstructured.Unstructured), args.Error(1)\n}\n\nfunc (c *FakeDynamicClient) Patch(name string, data []byte) (*unstructured.Unstructured, error) {\n\targs := c.Called(name, data)\n\treturn args.Get(0).(*unstructured.Unstructured), args.Error(1)\n}\n\nfunc (c *FakeDynamicClient) Delete(name string, opts metav1.DeleteOptions) error {\n\targs := c.Called(name, opts)\n\treturn args.Error(1)\n}\n\nfunc (c *FakeDynamicClient) UpdateStatus(obj *unstructured.Unstructured, opts metav1.UpdateOptions) (*unstructured.Unstructured, error) {\n\targs := c.Called(obj, opts)\n\treturn args.Get(0).(*unstructured.Unstructured), args.Error(1)\n}\n\nfunc (c *FakeDynamicClient) Apply(name string, obj *unstructured.Unstructured, opts metav1.ApplyOptions) (*unstructured.Unstructured, error) {\n\targs := c.Called(name, obj, opts)\n\treturn args.Get(0).(*unstructured.Unstructured), args.Error(1)\n}\n" }, { "path": "pkg/test/fake_file_system.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage test\n\nimport (\n\t\"io\"\n\t\"io/fs\"\n\t\"os\"\n\n\t\"github.com/spf13/afero\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n)\n\ntype FakeFileSystem struct {\n\tfs afero.Fs\n\n\tReadDirCalls []string\n}\n\nfunc NewFakeFileSystem() *FakeFileSystem {\n\treturn &FakeFileSystem{\n\t\tfs: afero.NewMemMapFs(),\n\t}\n}\n\nfunc (fs *FakeFileSystem) Glob(path string) ([]string, error) {\n\treturn afero.Glob(fs.fs, path)\n}\n\nfunc (fs *FakeFileSystem) TempDir(dir, prefix string) (string, error) {\n\treturn afero.TempDir(fs.fs, dir, prefix)\n}\n\nfunc (fs *FakeFileSystem) MkdirAll(path string, perm os.FileMode) error {\n\treturn fs.fs.MkdirAll(path, perm)\n}\n\nfunc (fs *FakeFileSystem) Create(name string) (io.WriteCloser, error) {\n\treturn fs.fs.Create(name)\n}\n\nfunc (fs *FakeFileSystem) OpenFile(name string, flag int, perm os.FileMode) (io.WriteCloser, error) {\n\treturn fs.fs.OpenFile(name, flag, perm)\n}\n\nfunc (fs *FakeFileSystem) RemoveAll(path string) error {\n\treturn fs.fs.RemoveAll(path)\n}\n\nfunc (fs *FakeFileSystem) ReadDir(dirname string) ([]fs.FileInfo, error) {\n\tfs.ReadDirCalls = append(fs.ReadDirCalls, dirname)\n\treturn afero.ReadDir(fs.fs, dirname)\n}\n\nfunc (fs *FakeFileSystem) ReadFile(filename string) ([]byte, error) {\n\treturn afero.ReadFile(fs.fs, filename)\n}\n\nfunc (fs *FakeFileSystem) DirExists(path string) (bool, error) {\n\treturn afero.DirExists(fs.fs, path)\n}\n\nfunc (fs *FakeFileSystem) Stat(path string) (os.FileInfo, error) {\n\treturn fs.fs.Stat(path)\n}\n\nfunc (fs *FakeFileSystem) WithFile(path string, data []byte) *FakeFileSystem {\n\tfile, _ := fs.fs.Create(path)\n\t_, _ = file.Write(data)\n\tfile.Close()\n\n\treturn fs\n}\n\nfunc (fs *FakeFileSystem) WithFileAndMode(path string, data []byte, mode os.FileMode) *FakeFileSystem {\n\tfile, _ := fs.fs.OpenFile(path, os.O_CREATE|os.O_RDWR, mode)\n\t_, _ = file.Write(data)\n\tfile.Close()\n\n\treturn fs\n}\n\nfunc (fs *FakeFileSystem) WithDirectory(path string) *FakeFileSystem {\n\t_ = fs.fs.MkdirAll(path, 0755)\n\treturn fs\n}\n\nfunc (fs *FakeFileSystem) WithDirectories(path ...string) *FakeFileSystem {\n\tfor _, dir := range path {\n\t\tfs = fs.WithDirectory(dir)\n\t}\n\n\treturn fs\n}\n\nfunc (fs *FakeFileSystem) TempFile(dir, prefix string) (filesystem.NameWriteCloser, error) {\n\treturn afero.TempFile(fs.fs, dir, prefix)\n}\n" }, { "path": "pkg/test/fake_mapper.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage test\n\nimport (\n\t\"github.com/pkg/errors\"\n\t\"k8s.io/apimachinery/pkg/api/meta\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n)\n\ntype FakeMapper struct {\n\tmeta.RESTMapper\n\tAutoReturnResource bool\n\tResources map[schema.GroupVersionResource]schema.GroupVersionResource\n\tKindToPluralResource map[schema.GroupVersionKind]schema.GroupVersionResource\n}\n\nfunc (m *FakeMapper) ResourceFor(input schema.GroupVersionResource) (schema.GroupVersionResource, error) {\n\tif m.AutoReturnResource {\n\t\treturn schema.GroupVersionResource{\n\t\t\tGroup: input.Group,\n\t\t\tVersion: input.Version,\n\t\t\tResource: input.Resource,\n\t\t}, nil\n\t}\n\tif m.Resources == nil {\n\t\treturn schema.GroupVersionResource{}, errors.Errorf(\"invalid resource %q\", input.String())\n\t}\n\n\tif gr, found := m.Resources[input]; found {\n\t\treturn gr, nil\n\t}\n\tif input.Version == \"\" {\n\t\tinput.Version = \"v1\"\n\t\tif gr, found := m.Resources[input]; found {\n\t\t\treturn gr, nil\n\t\t}\n\t\tinput.Version = \"v1beta1\"\n\t\tif gr, found := m.Resources[input]; found {\n\t\t\treturn gr, nil\n\t\t}\n\t}\n\n\treturn schema.GroupVersionResource{}, errors.Errorf(\"invalid resource %q\", input.String())\n}\n\nfunc (m *FakeMapper) RESTMapping(gk schema.GroupKind, versions ...string) (*meta.RESTMapping, error) {\n\tpotentialGVK := make([]schema.GroupVersionKind, 0)\n\t// Pick an appropriate version\n\tfor _, version := range versions {\n\t\tif len(version) == 0 || version == runtime.APIVersionInternal {\n\t\t\tcontinue\n\t\t}\n\t\tcurrGVK := gk.WithVersion(version)\n\t\tif _, ok := m.KindToPluralResource[currGVK]; ok {\n\t\t\tpotentialGVK = append(potentialGVK, currGVK)\n\t\t\tbreak\n\t\t}\n\t}\n\tif len(potentialGVK) == 0 {\n\t\treturn nil, &meta.NoKindMatchError{GroupKind: gk, SearchedVersions: versions}\n\t}\n\n\tfor _, gvk := range potentialGVK {\n\t\t//Ensure we have a REST mapping\n\t\tres, ok := m.KindToPluralResource[gvk]\n\t\tif !ok {\n\t\t\tcontinue\n\t\t}\n\n\t\treturn &meta.RESTMapping{\n\t\t\tResource: res,\n\t\t\tGroupVersionKind: gvk,\n\t\t\tScope: meta.RESTScopeNamespace,\n\t\t}, nil\n\t}\n\treturn nil, &meta.NoResourceMatchError{PartialResource: schema.GroupVersionResource{Group: gk.Group, Resource: gk.Kind}}\n}\n" }, { "path": "pkg/test/fake_namespace.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage test\n\nimport (\n\t\"context\"\n\n\t\"github.com/stretchr/testify/mock\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/watch\"\n\tv1 \"k8s.io/client-go/applyconfigurations/core/v1\"\n\tcorev1 \"k8s.io/client-go/kubernetes/typed/core/v1\"\n)\n\ntype FakeNamespaceClient struct {\n\tmock.Mock\n}\n\nvar _ corev1.NamespaceInterface = &FakeNamespaceClient{}\n\nfunc (c *FakeNamespaceClient) List(ctx context.Context, options metav1.ListOptions) (*corev1api.NamespaceList, error) {\n\targs := c.Called(options)\n\treturn args.Get(0).(*corev1api.NamespaceList), args.Error(1)\n}\n\nfunc (c *FakeNamespaceClient) Create(ctx context.Context, obj *corev1api.Namespace, options metav1.CreateOptions) (*corev1api.Namespace, error) {\n\targs := c.Called(obj)\n\treturn args.Get(0).(*corev1api.Namespace), args.Error(1)\n}\n\nfunc (c *FakeNamespaceClient) Watch(ctx context.Context, options metav1.ListOptions) (watch.Interface, error) {\n\targs := c.Called(options)\n\treturn args.Get(0).(watch.Interface), args.Error(1)\n}\n\nfunc (c *FakeNamespaceClient) Get(ctx context.Context, name string, opts metav1.GetOptions) (*corev1api.Namespace, error) {\n\targs := c.Called(name, opts)\n\treturn args.Get(0).(*corev1api.Namespace), args.Error(1)\n}\n\nfunc (c *FakeNamespaceClient) Patch(ctx context.Context, name string, pt types.PatchType, data []byte, opts metav1.PatchOptions, subresources ...string) (*corev1api.Namespace, error) {\n\targs := c.Called(name, pt, data, subresources)\n\treturn args.Get(0).(*corev1api.Namespace), args.Error(1)\n}\n\nfunc (c *FakeNamespaceClient) Delete(ctx context.Context, name string, opts metav1.DeleteOptions) error {\n\targs := c.Called(name, opts)\n\treturn args.Error(1)\n}\n\nfunc (c *FakeNamespaceClient) Finalize(ctx context.Context, item *corev1api.Namespace, options metav1.UpdateOptions) (*corev1api.Namespace, error) {\n\targs := c.Called(item)\n\treturn args.Get(0).(*corev1api.Namespace), args.Error(1)\n}\n\nfunc (c *FakeNamespaceClient) Update(ctx context.Context, namespace *corev1api.Namespace, options metav1.UpdateOptions) (*corev1api.Namespace, error) {\n\targs := c.Called(namespace)\n\treturn args.Get(0).(*corev1api.Namespace), args.Error(1)\n}\n\nfunc (c *FakeNamespaceClient) UpdateStatus(ctx context.Context, namespace *corev1api.Namespace, options metav1.UpdateOptions) (*corev1api.Namespace, error) {\n\targs := c.Called(namespace)\n\treturn args.Get(0).(*corev1api.Namespace), args.Error(1)\n}\n\nfunc (c *FakeNamespaceClient) Apply(ctx context.Context, namespace *v1.NamespaceApplyConfiguration, opts metav1.ApplyOptions) (result *corev1api.Namespace, err error) {\n\targs := c.Called(namespace)\n\treturn args.Get(0).(*corev1api.Namespace), args.Error(1)\n}\n\nfunc (c *FakeNamespaceClient) ApplyStatus(ctx context.Context, namespace *v1.NamespaceApplyConfiguration, opts metav1.ApplyOptions) (result *corev1api.Namespace, err error) {\n\targs := c.Called(namespace)\n\treturn args.Get(0).(*corev1api.Namespace), args.Error(1)\n}\n" }, { "path": "pkg/test/fake_volume_snapshotter.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage test\n\nimport (\n\t\"errors\"\n\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n)\n\ntype VolumeBackupInfo struct {\n\tSnapshotID string\n\tType string\n\tIops *int64\n\tAvailabilityZone string\n}\n\ntype FakeVolumeSnapshotter struct {\n\t// SnapshotID->VolumeID\n\tSnapshotsTaken sets.String\n\n\t// VolumeID -> (SnapshotID, Type, Iops)\n\tSnapshottableVolumes map[string]VolumeBackupInfo\n\n\t// VolumeBackupInfo -> VolumeID\n\tRestorableVolumes map[VolumeBackupInfo]string\n\n\tVolumeID string\n\tVolumeIDSet string\n\n\tError error\n}\n\nfunc (bs *FakeVolumeSnapshotter) Init(config map[string]string) error {\n\treturn nil\n}\n\nfunc (bs *FakeVolumeSnapshotter) CreateSnapshot(volumeID, volumeAZ string, tags map[string]string) (string, error) {\n\tif bs.Error != nil {\n\t\treturn \"\", bs.Error\n\t}\n\n\tif _, exists := bs.SnapshottableVolumes[volumeID]; !exists {\n\t\treturn \"\", errors.New(\"snapshottable volume not found\")\n\t}\n\n\tif bs.SnapshotsTaken == nil {\n\t\tbs.SnapshotsTaken = sets.NewString()\n\t}\n\tbs.SnapshotsTaken.Insert(bs.SnapshottableVolumes[volumeID].SnapshotID)\n\n\treturn bs.SnapshottableVolumes[volumeID].SnapshotID, nil\n}\n\nfunc (bs *FakeVolumeSnapshotter) CreateVolumeFromSnapshot(snapshotID, volumeType, volumeAZ string, iops *int64) (string, error) {\n\tif bs.Error != nil {\n\t\treturn \"\", bs.Error\n\t}\n\n\tkey := VolumeBackupInfo{\n\t\tSnapshotID: snapshotID,\n\t\tType: volumeType,\n\t\tIops: iops,\n\t\tAvailabilityZone: volumeAZ,\n\t}\n\n\treturn bs.RestorableVolumes[key], nil\n}\n\nfunc (bs *FakeVolumeSnapshotter) DeleteSnapshot(snapshotID string) error {\n\tif bs.Error != nil {\n\t\treturn bs.Error\n\t}\n\n\tif !bs.SnapshotsTaken.Has(snapshotID) {\n\t\treturn errors.New(\"snapshot not found\")\n\t}\n\n\tbs.SnapshotsTaken.Delete(snapshotID)\n\n\treturn nil\n}\n\nfunc (bs *FakeVolumeSnapshotter) GetVolumeInfo(volumeID, volumeAZ string) (string, *int64, error) {\n\tif bs.Error != nil {\n\t\treturn \"\", nil, bs.Error\n\t}\n\n\tvolumeInfo, exists := bs.SnapshottableVolumes[volumeID]\n\tif !exists {\n\t\treturn \"\", nil, errors.New(\"VolumeID not found\")\n\t}\n\treturn volumeInfo.Type, volumeInfo.Iops, nil\n}\n\nfunc (bs *FakeVolumeSnapshotter) GetVolumeID(pv runtime.Unstructured) (string, error) {\n\treturn bs.VolumeID, nil\n}\n\nfunc (bs *FakeVolumeSnapshotter) SetVolumeID(pv runtime.Unstructured, volumeID string) (runtime.Unstructured, error) {\n\tbs.VolumeIDSet = volumeID\n\treturn pv, bs.Error\n}\n" }, { "path": "pkg/test/helpers.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage test\n\nimport (\n\t\"encoding/json\"\n\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n)\n\nfunc UnstructuredOrDie(data string) *unstructured.Unstructured {\n\to, _, err := unstructured.UnstructuredJSONScheme.Decode([]byte(data), nil, nil)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn o.(*unstructured.Unstructured)\n}\n\nfunc GetAsMap(j string) (map[string]any, error) {\n\tm := make(map[string]any)\n\terr := json.Unmarshal([]byte(j), &m)\n\treturn m, err\n}\n" }, { "path": "pkg/test/mock_pod_command_executor.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage test\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/mock\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\ntype MockPodCommandExecutor struct {\n\tmock.Mock\n\t// hook execution order\n\tHookExecutionLog []HookExecutionEntry\n}\n\ntype HookExecutionEntry struct {\n\tNamespace, Name, HookName string\n\tHookCommand []string\n}\n\nfunc (h HookExecutionEntry) String() string {\n\treturn fmt.Sprintf(\"%s.%s.%s.%s\", h.Namespace, h.Name, h.HookName, strings.Join(h.HookCommand, \",\"))\n}\n\nfunc (e *MockPodCommandExecutor) ExecutePodCommand(log logrus.FieldLogger, item map[string]any, namespace, name, hookName string, hook *v1.ExecHook) error {\n\te.HookExecutionLog = append(e.HookExecutionLog, HookExecutionEntry{\n\t\tNamespace: namespace,\n\t\tName: name,\n\t\tHookName: hookName,\n\t\tHookCommand: hook.Command,\n\t})\n\targs := e.Called(log, item, namespace, name, hookName, hook)\n\treturn args.Error(0)\n}\n" }, { "path": "pkg/test/mocks/VolumeSnapshotLister.go", "content": "// Code generated by mockery v2.35.4. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tmock \"github.com/stretchr/testify/mock\"\n\tlabels \"k8s.io/apimachinery/pkg/labels\"\n\n\tv1 \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\n\tvolumesnapshotv1 \"github.com/kubernetes-csi/external-snapshotter/client/v8/listers/volumesnapshot/v1\"\n)\n\n// VolumeSnapshotLister is an autogenerated mock type for the VolumeSnapshotLister type\ntype VolumeSnapshotLister struct {\n\tmock.Mock\n}\n\n// List provides a mock function with given fields: selector\nfunc (_m *VolumeSnapshotLister) List(selector labels.Selector) ([]*v1.VolumeSnapshot, error) {\n\tret := _m.Called(selector)\n\n\tvar r0 []*v1.VolumeSnapshot\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(labels.Selector) ([]*v1.VolumeSnapshot, error)); ok {\n\t\treturn rf(selector)\n\t}\n\tif rf, ok := ret.Get(0).(func(labels.Selector) []*v1.VolumeSnapshot); ok {\n\t\tr0 = rf(selector)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]*v1.VolumeSnapshot)\n\t\t}\n\t}\n\n\tif rf, ok := ret.Get(1).(func(labels.Selector) error); ok {\n\t\tr1 = rf(selector)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// VolumeSnapshots provides a mock function with given fields: namespace\nfunc (_m *VolumeSnapshotLister) VolumeSnapshots(namespace string) volumesnapshotv1.VolumeSnapshotNamespaceLister {\n\tret := _m.Called(namespace)\n\n\tvar r0 volumesnapshotv1.VolumeSnapshotNamespaceLister\n\tif rf, ok := ret.Get(0).(func(string) volumesnapshotv1.VolumeSnapshotNamespaceLister); ok {\n\t\tr0 = rf(namespace)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(volumesnapshotv1.VolumeSnapshotNamespaceLister)\n\t\t}\n\t}\n\n\treturn r0\n}\n\n// NewVolumeSnapshotLister creates a new instance of VolumeSnapshotLister. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\n// The first argument is typically a *testing.T value.\nfunc NewVolumeSnapshotLister(t interface {\n\tmock.TestingT\n\tCleanup(func())\n}) *VolumeSnapshotLister {\n\tmock := &VolumeSnapshotLister{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/test/mocks.go", "content": "package test\n\nimport (\n\tsnapshotv1 \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\tsnapshotv1listers \"github.com/kubernetes-csi/external-snapshotter/client/v8/listers/volumesnapshot/v1\"\n\t\"k8s.io/apimachinery/pkg/labels\"\n)\n\n// VolumeSnapshotLister helps list VolumeSnapshots.\n// All objects returned here must be treated as read-only.\n//\n//go:generate mockery --name VolumeSnapshotLister\ntype VolumeSnapshotLister interface {\n\t// List lists all VolumeSnapshots in the indexer.\n\t// Objects returned here must be treated as read-only.\n\tList(selector labels.Selector) (ret []*snapshotv1.VolumeSnapshot, err error)\n\t// VolumeSnapshots returns an object that can list and get VolumeSnapshots.\n\tVolumeSnapshots(namespace string) snapshotv1listers.VolumeSnapshotNamespaceLister\n\tsnapshotv1listers.VolumeSnapshotListerExpansion\n}\n" }, { "path": "pkg/test/resources.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage test\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n)\n\n// APIResource stores information about a specific Kubernetes API\n// resource.\ntype APIResource struct {\n\tGroup string\n\tVersion string\n\tName string\n\tKind string\n\tShortName string\n\tNamespaced bool\n\tItems []metav1.Object\n}\n\n// GVR returns a GroupVersionResource representing the resource.\nfunc (r *APIResource) GVR() schema.GroupVersionResource {\n\treturn schema.GroupVersionResource{\n\t\tGroup: r.Group,\n\t\tVersion: r.Version,\n\t\tResource: r.Name,\n\t}\n}\n\n// Pods returns an APIResource describing core/v1's Pods.\nfunc Pods(items ...metav1.Object) *APIResource {\n\treturn &APIResource{\n\t\tGroup: \"\",\n\t\tVersion: \"v1\",\n\t\tName: \"pods\",\n\t\tShortName: \"po\",\n\t\tNamespaced: true,\n\t\tItems: items,\n\t\tKind: \"Pod\",\n\t}\n}\n\nfunc PVCs(items ...metav1.Object) *APIResource {\n\treturn &APIResource{\n\t\tGroup: \"\",\n\t\tVersion: \"v1\",\n\t\tName: \"persistentvolumeclaims\",\n\t\tShortName: \"pvc\",\n\t\tKind: \"PersistentVolumeClaim\",\n\t\tNamespaced: true,\n\t\tItems: items,\n\t}\n}\n\nfunc PVs(items ...metav1.Object) *APIResource {\n\treturn &APIResource{\n\t\tGroup: \"\",\n\t\tVersion: \"v1\",\n\t\tName: \"persistentvolumes\",\n\t\tShortName: \"pv\",\n\t\tKind: \"PersistentVolume\",\n\t\tNamespaced: false,\n\t\tItems: items,\n\t}\n}\n\nfunc Secrets(items ...metav1.Object) *APIResource {\n\treturn &APIResource{\n\t\tGroup: \"\",\n\t\tVersion: \"v1\",\n\t\tName: \"secrets\",\n\t\tShortName: \"secrets\",\n\t\tKind: \"Secret\",\n\t\tNamespaced: true,\n\t\tItems: items,\n\t}\n}\n\nfunc Deployments(items ...metav1.Object) *APIResource {\n\treturn &APIResource{\n\t\tGroup: \"apps\",\n\t\tVersion: \"v1\",\n\t\tName: \"deployments\",\n\t\tShortName: \"deploy\",\n\t\tKind: \"Deployment\",\n\t\tNamespaced: true,\n\t\tItems: items,\n\t}\n}\n\nfunc ExtensionsDeployments(items ...metav1.Object) *APIResource {\n\treturn &APIResource{\n\t\tGroup: \"extensions\",\n\t\tVersion: \"v1\",\n\t\tName: \"deployments\",\n\t\tShortName: \"deploy\",\n\t\tKind: \"Deployment\",\n\t\tNamespaced: true,\n\t\tItems: items,\n\t}\n}\n\n// test CRD\nfunc VeleroDeployments(items ...metav1.Object) *APIResource {\n\treturn &APIResource{\n\t\tGroup: \"velero.io\",\n\t\tVersion: \"v1\",\n\t\tName: \"deployments\",\n\t\tShortName: \"deploy\",\n\t\tKind: \"Deployment\",\n\t\tNamespaced: true,\n\t\tItems: items,\n\t}\n}\n\nfunc Namespaces(items ...metav1.Object) *APIResource {\n\treturn &APIResource{\n\t\tGroup: \"\",\n\t\tVersion: \"v1\",\n\t\tName: \"namespaces\",\n\t\tShortName: \"ns\",\n\t\tKind: \"Namespace\",\n\t\tNamespaced: false,\n\t\tItems: items,\n\t}\n}\n\nfunc ServiceAccounts(items ...metav1.Object) *APIResource {\n\treturn &APIResource{\n\t\tGroup: \"\",\n\t\tVersion: \"v1\",\n\t\tName: \"serviceaccounts\",\n\t\tShortName: \"sa\",\n\t\tKind: \"ServiceAccount\",\n\t\tNamespaced: true,\n\t\tItems: items,\n\t}\n}\n\nfunc ConfigMaps(items ...metav1.Object) *APIResource {\n\treturn &APIResource{\n\t\tGroup: \"\",\n\t\tVersion: \"v1\",\n\t\tName: \"configmaps\",\n\t\tShortName: \"cm\",\n\t\tKind: \"ConfigMap\",\n\t\tNamespaced: true,\n\t\tItems: items,\n\t}\n}\n\nfunc CRDs(items ...metav1.Object) *APIResource {\n\treturn &APIResource{\n\t\tGroup: \"apiextensions.k8s.io\",\n\t\tVersion: \"v1beta1\",\n\t\tName: \"customresourcedefinitions\",\n\t\tShortName: \"crd\",\n\t\tKind: \"CustomResourceDefinition\",\n\t\tNamespaced: false,\n\t\tItems: items,\n\t}\n}\n\nfunc VSLs(items ...metav1.Object) *APIResource {\n\treturn &APIResource{\n\t\tGroup: \"velero.io\",\n\t\tVersion: \"v1\",\n\t\tName: \"volumesnapshotlocations\",\n\t\tKind: \"VolumeSnapshotLocation\",\n\t\tNamespaced: true,\n\t\tItems: items,\n\t}\n}\n\nfunc Backups(items ...metav1.Object) *APIResource {\n\treturn &APIResource{\n\t\tGroup: \"velero.io\",\n\t\tVersion: \"v1\",\n\t\tName: \"backups\",\n\t\tKind: \"Backup\",\n\t\tNamespaced: true,\n\t\tItems: items,\n\t}\n}\n\nfunc Services(items ...metav1.Object) *APIResource {\n\treturn &APIResource{\n\t\tGroup: \"\",\n\t\tVersion: \"v1\",\n\t\tName: \"services\",\n\t\tShortName: \"svc\",\n\t\tKind: \"Service\",\n\t\tNamespaced: true,\n\t\tItems: items,\n\t}\n}\n\nfunc DataUploads(items ...metav1.Object) *APIResource {\n\treturn &APIResource{\n\t\tGroup: \"velero.io\",\n\t\tVersion: \"v2alpha1\",\n\t\tName: \"datauploads\",\n\t\tKind: \"DataUpload\",\n\t\tNamespaced: true,\n\t\tItems: items,\n\t}\n}\n" }, { "path": "pkg/test/tar_writer.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage test\n\nimport (\n\t\"archive/tar\"\n\t\"bytes\"\n\t\"compress/gzip\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/encode\"\n)\n\ntype TarWriter struct {\n\tt *testing.T\n\tbuf *bytes.Buffer\n\tgzw *gzip.Writer\n\ttw *tar.Writer\n}\n\nfunc NewTarWriter(t *testing.T) *TarWriter {\n\tt.Helper()\n\ttw := new(TarWriter)\n\ttw.t = t\n\ttw.buf = new(bytes.Buffer)\n\ttw.gzw = gzip.NewWriter(tw.buf)\n\ttw.tw = tar.NewWriter(tw.gzw)\n\n\treturn tw\n}\n\nfunc (tw *TarWriter) AddItems(groupResource string, items ...metav1.Object) *TarWriter {\n\ttw.t.Helper()\n\n\tfor _, obj := range items {\n\t\tvar path string\n\t\tif obj.GetNamespace() == \"\" {\n\t\t\tpath = fmt.Sprintf(\"resources/%s/cluster/%s.json\", groupResource, obj.GetName())\n\t\t} else {\n\t\t\tpath = fmt.Sprintf(\"resources/%s/namespaces/%s/%s.json\", groupResource, obj.GetNamespace(), obj.GetName())\n\t\t}\n\n\t\ttw.Add(path, obj)\n\t}\n\n\treturn tw\n}\n\nfunc (tw *TarWriter) Add(name string, obj any) *TarWriter {\n\ttw.t.Helper()\n\n\tvar data []byte\n\tvar err error\n\n\tswitch objType := obj.(type) {\n\tcase runtime.Object:\n\t\tdata, err = encode.Encode(objType, \"json\")\n\tcase []byte:\n\t\tdata = objType\n\tdefault:\n\t\tdata, err = json.Marshal(obj)\n\t}\n\trequire.NoError(tw.t, err)\n\n\trequire.NoError(tw.t, tw.tw.WriteHeader(&tar.Header{\n\t\tName: name,\n\t\tSize: int64(len(data)),\n\t\tTypeflag: tar.TypeReg,\n\t\tMode: 0755,\n\t\tModTime: time.Now(),\n\t}))\n\n\t_, err = tw.tw.Write(data)\n\trequire.NoError(tw.t, err)\n\n\treturn tw\n}\n\nfunc (tw *TarWriter) Done() *bytes.Buffer {\n\trequire.NoError(tw.t, tw.tw.Close())\n\trequire.NoError(tw.t, tw.gzw.Close())\n\n\treturn tw.buf\n}\n" }, { "path": "pkg/test/test_logger.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage test\n\nimport (\n\t\"io\"\n\n\t\"github.com/sirupsen/logrus\"\n)\n\nfunc NewLogger() logrus.FieldLogger {\n\tlogger := logrus.New()\n\tlogger.Out = io.Discard\n\treturn logrus.NewEntry(logger)\n}\n\nfunc NewLoggerWithLevel(level logrus.Level) logrus.FieldLogger {\n\tlogger := logrus.New()\n\tlogger.Out = io.Discard\n\tlogger.Level = level\n\treturn logrus.NewEntry(logger)\n}\n\ntype singleLogRecorder struct {\n\tbuffer *string\n}\n\nfunc (s *singleLogRecorder) Write(p []byte) (n int, err error) {\n\t*s.buffer = *s.buffer + string(p[:])\n\treturn len(p), nil\n}\n\nfunc NewSingleLogger(buffer *string) logrus.FieldLogger {\n\tlogger := logrus.New()\n\tlogger.Out = &singleLogRecorder{buffer: buffer}\n\tlogger.Level = logrus.TraceLevel\n\treturn logrus.NewEntry(logger)\n}\n\nfunc NewSingleLoggerWithHooks(buffer *string, hooks []logrus.Hook) logrus.FieldLogger {\n\tlogger := logrus.New()\n\tlogger.Out = &singleLogRecorder{buffer: buffer}\n\tlogger.Level = logrus.TraceLevel\n\n\tfor _, hook := range hooks {\n\t\tlogger.Hooks.Add(hook)\n\t}\n\n\treturn logrus.NewEntry(logger)\n}\n\ntype multipleLogRecorder struct {\n\tbuffer *[]string\n}\n\nfunc (m *multipleLogRecorder) Write(p []byte) (n int, err error) {\n\t*m.buffer = append(*m.buffer, string(p[:]))\n\treturn len(p), nil\n}\n\nfunc NewMultipleLogger(buffer *[]string) logrus.FieldLogger {\n\tlogger := logrus.New()\n\tlogger.Out = &multipleLogRecorder{buffer}\n\tlogger.Level = logrus.TraceLevel\n\treturn logrus.NewEntry(logger)\n}\n" }, { "path": "pkg/types/node_agent.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage types\n\nimport (\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/kube\"\n)\n\ntype LoadConcurrency struct {\n\t// GlobalConfig specifies the concurrency number to all nodes for which per-node config is not specified\n\tGlobalConfig int `json:\"globalConfig,omitempty\"`\n\n\t// PerNodeConfig specifies the concurrency number to nodes matched by rules\n\tPerNodeConfig []RuledConfigs `json:\"perNodeConfig,omitempty\"`\n\n\t// PrepareQueueLength specifies the max number of loads that are under expose\n\tPrepareQueueLength int `json:\"prepareQueueLength,omitempty\"`\n}\n\ntype LoadAffinity struct {\n\t// NodeSelector specifies the label selector to match nodes\n\tNodeSelector metav1.LabelSelector `json:\"nodeSelector\"`\n}\n\ntype RuledConfigs struct {\n\t// NodeSelector specifies the label selector to match nodes\n\tNodeSelector metav1.LabelSelector `json:\"nodeSelector\"`\n\n\t// Number specifies the number value associated to the matched nodes\n\tNumber int `json:\"number\"`\n}\n\ntype BackupPVC struct {\n\t// StorageClass is the name of storage class to be used by the backupPVC\n\tStorageClass string `json:\"storageClass,omitempty\"`\n\n\t// ReadOnly sets the backupPVC's access mode as read only\n\tReadOnly bool `json:\"readOnly,omitempty\"`\n\n\t// SPCNoRelabeling sets Spec.SecurityContext.SELinux.Type to \"spc_t\" for the pod mounting the backupPVC\n\t// ignored if ReadOnly is false\n\tSPCNoRelabeling bool `json:\"spcNoRelabeling,omitempty\"`\n\n\t// Annotations permits setting annotations for the backupPVC\n\tAnnotations map[string]string `json:\"annotations,omitempty\"`\n}\n\ntype RestorePVC struct {\n\t// IgnoreDelayBinding indicates to ignore delay binding the restorePVC when it is in WaitForFirstConsumer mode\n\tIgnoreDelayBinding bool `json:\"ignoreDelayBinding,omitempty\"`\n}\n\ntype CachePVC struct {\n\t// StorageClass specifies the storage class for cache PVC\n\tStorageClass string `json:\"storageClass,omitempty\"`\n\n\t// ResidentThresholdInMB specifies the minimum size of the backup data to create cache PVC\n\tResidentThresholdInMB int64 `json:\"residentThresholdInMB,omitempty\"`\n}\n\ntype NodeAgentConfigs struct {\n\t// LoadConcurrency is the config for data path load concurrency per node.\n\tLoadConcurrency *LoadConcurrency `json:\"loadConcurrency,omitempty\"`\n\n\t// LoadAffinity is the config for data path load affinity.\n\tLoadAffinity []*kube.LoadAffinity `json:\"loadAffinity,omitempty\"`\n\n\t// BackupPVCConfig is the config for backupPVC (intermediate PVC) of snapshot data movement\n\tBackupPVCConfig map[string]BackupPVC `json:\"backupPVC,omitempty\"`\n\n\t// RestoreVCConfig is the config for restorePVC (intermediate PVC) of generic restore\n\tRestorePVCConfig *RestorePVC `json:\"restorePVC,omitempty\"`\n\n\t// PodResources is the resource config for various types of pods launched by node-agent, i.e., data mover pods.\n\tPodResources *kube.PodResources `json:\"podResources,omitempty\"`\n\n\t// PriorityClassName is the priority class name for data mover pods created by the node agent\n\tPriorityClassName string `json:\"priorityClassName,omitempty\"`\n\n\t// PrivilegedFsBackup determines whether to create fs-backup pods as privileged pods\n\tPrivilegedFsBackup bool `json:\"privilegedFsBackup,omitempty\"`\n\n\t// CachePVCConfig is the config for cachePVC\n\tCachePVCConfig *CachePVC `json:\"cachePVC,omitempty\"`\n\n\t// PodAnnotations are annotations to be added to pods created by node-agent, i.e., data mover pods.\n\tPodAnnotations map[string]string `json:\"podAnnotations,omitempty\"`\n\n\t// PodLabels are labels to be added to pods created by node-agent, i.e., data mover pods.\n\tPodLabels map[string]string `json:\"podLabels,omitempty\"`\n}\n" }, { "path": "pkg/types/priority.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage types\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n)\n\nconst (\n\tprioritySeparator = \"-\"\n)\n\n// Priorities defines the desired order of resource operations:\n// Resources in the HighPriorities list will be handled first\n// Resources in the LowPriorities list will be handled last\n// Other resources will be handled alphabetically after the high prioritized resources and before the low prioritized resources\ntype Priorities struct {\n\tHighPriorities []string\n\tLowPriorities []string\n}\n\n// String returns a string representation of Priority.\nfunc (p *Priorities) String() string {\n\tpriorities := p.HighPriorities\n\tif len(p.LowPriorities) > 0 {\n\t\tpriorities = append(priorities, prioritySeparator)\n\t\tpriorities = append(priorities, p.LowPriorities...)\n\t}\n\treturn strings.Join(priorities, \",\")\n}\n\n// Set parses the provided string to the priority object\nfunc (p *Priorities) Set(s string) error {\n\tif len(s) == 0 {\n\t\treturn nil\n\t}\n\tstrs := strings.Split(s, \",\")\n\tseparatorIndex := -1\n\tfor i, str := range strs {\n\t\tif str == prioritySeparator {\n\t\t\tif separatorIndex > -1 {\n\t\t\t\treturn fmt.Errorf(\"multiple priority separator %q found\", prioritySeparator)\n\t\t\t}\n\t\t\tseparatorIndex = i\n\t\t}\n\t}\n\t// has no separator\n\tif separatorIndex == -1 {\n\t\tp.HighPriorities = strs\n\t\treturn nil\n\t}\n\t// start with separator\n\tif separatorIndex == 0 {\n\t\t// contain only separator\n\t\tif len(strs) == 1 {\n\t\t\treturn nil\n\t\t}\n\t\tp.LowPriorities = strs[1:]\n\t\treturn nil\n\t}\n\t// end with separator\n\tif separatorIndex == len(strs)-1 {\n\t\tp.HighPriorities = strs[:len(strs)-1]\n\t\treturn nil\n\t}\n\n\t// separator in the middle\n\tp.HighPriorities = strs[:separatorIndex]\n\tp.LowPriorities = strs[separatorIndex+1:]\n\n\treturn nil\n}\n\n// Type specifies the flag type\nfunc (p *Priorities) Type() string {\n\treturn \"stringArray\"\n}\n" }, { "path": "pkg/types/priority_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage types\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestStringOfPriorities(t *testing.T) {\n\tpriority := Priorities{\n\t\tHighPriorities: []string{\"high\"},\n\t}\n\tassert.Equal(t, \"high\", priority.String())\n\n\tpriority = Priorities{\n\t\tHighPriorities: []string{\"high\"},\n\t\tLowPriorities: []string{\"low\"},\n\t}\n\tassert.Equal(t, \"high,-,low\", priority.String())\n}\n\nfunc TestSetOfPriority(t *testing.T) {\n\tcases := []struct {\n\t\tname string\n\t\tinput string\n\t\tpriorities Priorities\n\t\thasErr bool\n\t}{\n\t\t{\n\t\t\tname: \"empty input\",\n\t\t\tinput: \"\",\n\t\t\tpriorities: Priorities{},\n\t\t\thasErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"only high priorities\",\n\t\t\tinput: \"p0\",\n\t\t\tpriorities: Priorities{\n\t\t\t\tHighPriorities: []string{\"p0\"},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"only low priorities\",\n\t\t\tinput: \"-,p9\",\n\t\t\tpriorities: Priorities{\n\t\t\t\tLowPriorities: []string{\"p9\"},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"only separator\",\n\t\t\tinput: \"-\",\n\t\t\tpriorities: Priorities{},\n\t\t\thasErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"multiple separators\",\n\t\t\tinput: \"-,-\",\n\t\t\tpriorities: Priorities{},\n\t\t\thasErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"contain both high and low priorities\",\n\t\t\tinput: \"p0,p1,p2,-,p9\",\n\t\t\tpriorities: Priorities{\n\t\t\t\tHighPriorities: []string{\"p0\", \"p1\", \"p2\"},\n\t\t\t\tLowPriorities: []string{\"p9\"},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"end with separator\",\n\t\t\tinput: \"p0,-\",\n\t\t\tpriorities: Priorities{\n\t\t\t\tHighPriorities: []string{\"p0\"},\n\t\t\t},\n\t\t\thasErr: false,\n\t\t},\n\t}\n\tfor _, c := range cases {\n\t\tt.Run(c.name, func(t *testing.T) {\n\t\t\tp := Priorities{}\n\t\t\terr := p.Set(c.input)\n\t\t\tif c.hasErr {\n\t\t\t\trequire.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\t\t\tassert.Equal(t, c.priorities, p)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/types/repo_maintenance.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage types\n\nimport \"github.com/vmware-tanzu/velero/pkg/util/kube\"\n\ntype JobConfigs struct {\n\t// LoadAffinities is the config for repository maintenance job load affinity.\n\tLoadAffinities []*kube.LoadAffinity `json:\"loadAffinity,omitempty\"`\n\n\t// PodResources is the config for the CPU and memory resources setting.\n\tPodResources *kube.PodResources `json:\"podResources,omitempty\"`\n\n\t// KeepLatestMaintenanceJobs is the number of latest maintenance jobs to keep for the repository.\n\tKeepLatestMaintenanceJobs *int `json:\"keepLatestMaintenanceJobs,omitempty\"`\n\n\t// PriorityClassName is the priority class name for the maintenance job pod\n\t// Note: This is only read from the global configuration, not per-repository\n\tPriorityClassName string `json:\"priorityClassName,omitempty\"`\n\n\t// PodAnnotations are annotations to be added to maintenance job pods.\n\t// Note: This is only read from the global configuration, not per-repository\n\tPodAnnotations map[string]string `json:\"podAnnotations,omitempty\"`\n\n\t// PodLabels are labels to be added to maintenance job pods.\n\t// Note: This is only read from the global configuration, not per-repository\n\tPodLabels map[string]string `json:\"podLabels,omitempty\"`\n}\n" }, { "path": "pkg/uploader/kopia/block_backup.go", "content": "//go:build !windows\n// +build !windows\n\n/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kopia\n\nimport (\n\t\"os\"\n\t\"syscall\"\n\n\t\"github.com/kopia/kopia/fs\"\n\t\"github.com/kopia/kopia/fs/virtualfs\"\n\t\"github.com/pkg/errors\"\n)\n\nconst ErrNotPermitted = \"operation not permitted\"\n\nfunc getLocalBlockEntry(sourcePath string) (fs.Entry, error) {\n\tsource, err := resolveSymlink(sourcePath)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"resolveSymlink\")\n\t}\n\n\tfileInfo, err := os.Lstat(source)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"unable to get the source device information %s\", source)\n\t}\n\n\tif (fileInfo.Sys().(*syscall.Stat_t).Mode & syscall.S_IFMT) != syscall.S_IFBLK {\n\t\treturn nil, errors.Errorf(\"source path %s is not a block device\", source)\n\t}\n\n\tdevice, err := os.Open(source)\n\tif err != nil {\n\t\tif os.IsPermission(err) || err.Error() == ErrNotPermitted {\n\t\t\treturn nil, errors.Wrapf(err, \"no permission to open the source device %s, make sure that node agent is running in privileged mode\", source)\n\t\t}\n\t\treturn nil, errors.Wrapf(err, \"unable to open the source device %s\", source)\n\t}\n\n\tsf := virtualfs.StreamingFileFromReader(source, device)\n\treturn virtualfs.NewStaticDirectory(source, []fs.Entry{sf}), nil\n}\n" }, { "path": "pkg/uploader/kopia/block_backup_windows.go", "content": "//go:build windows\n// +build windows\n\n/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kopia\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/kopia/kopia/fs\"\n)\n\nfunc getLocalBlockEntry(sourcePath string) (fs.Entry, error) {\n\treturn nil, fmt.Errorf(\"block mode is not supported for Windows\")\n}\n" }, { "path": "pkg/uploader/kopia/block_restore.go", "content": "//go:build !windows\n// +build !windows\n\n/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kopia\n\nimport (\n\t\"context\"\n\t\"io\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"syscall\"\n\n\t\"github.com/kopia/kopia/fs\"\n\t\"github.com/kopia/kopia/snapshot/restore\"\n\t\"github.com/pkg/errors\"\n)\n\ntype BlockOutput struct {\n\t*restore.FilesystemOutput\n\n\ttargetFileName string\n\ttargetFile *os.File\n}\n\nvar _ restore.Output = &BlockOutput{}\n\nconst bufferSize = 128 * 1024\n\nfunc (o *BlockOutput) WriteFile(ctx context.Context, relativePath string, remoteFile fs.File, progressCb restore.FileWriteProgress) error {\n\tremoteReader, err := remoteFile.Open(ctx)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"failed to open remote file %s\", remoteFile.Name())\n\t}\n\tdefer remoteReader.Close()\n\n\ttargetFile, err := os.Create(o.targetFileName)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"failed to open file %s\", o.targetFileName)\n\t}\n\to.targetFile = targetFile\n\n\tbuffer := make([]byte, bufferSize)\n\n\treadData := true\n\tfor readData {\n\t\tbytesToWrite, err := remoteReader.Read(buffer)\n\t\tif err != nil {\n\t\t\tif err != io.EOF {\n\t\t\t\treturn errors.Wrapf(err, \"failed to read data from remote file %s\", o.targetFileName)\n\t\t\t}\n\t\t\treadData = false\n\t\t}\n\n\t\tif bytesToWrite > 0 {\n\t\t\toffset := 0\n\t\t\tfor bytesToWrite > 0 {\n\t\t\t\tif bytesWritten, err := targetFile.Write(buffer[offset:bytesToWrite]); err == nil {\n\t\t\t\t\tprogressCb(int64(bytesWritten))\n\t\t\t\t\tbytesToWrite -= bytesWritten\n\t\t\t\t\toffset += bytesWritten\n\t\t\t\t} else {\n\t\t\t\t\treturn errors.Wrapf(err, \"failed to write data to file %s\", o.targetFileName)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (o *BlockOutput) BeginDirectory(ctx context.Context, relativePath string, e fs.Directory) error {\n\tvar err error\n\to.targetFileName, err = filepath.EvalSymlinks(o.TargetPath)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"unable to evaluate symlinks for %s\", o.targetFileName)\n\t}\n\n\tfileInfo, err := os.Lstat(o.targetFileName)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"unable to get the target device information for %s\", o.TargetPath)\n\t}\n\n\tif (fileInfo.Sys().(*syscall.Stat_t).Mode & syscall.S_IFMT) != syscall.S_IFBLK {\n\t\treturn errors.Errorf(\"target file %s is not a block device\", o.TargetPath)\n\t}\n\n\treturn nil\n}\n\nfunc (o *BlockOutput) Flush() error {\n\tif o.targetFile != nil {\n\t\tif err := o.targetFile.Sync(); err != nil {\n\t\t\treturn errors.Wrapf(err, \"error syncing block dev %v\", o.targetFileName)\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (o *BlockOutput) Terminate() error {\n\tif o.targetFile != nil {\n\t\tif err := o.targetFile.Close(); err != nil {\n\t\t\treturn errors.Wrapf(err, \"error closing block dev %v\", o.targetFileName)\n\t\t}\n\t}\n\n\treturn nil\n}\n" }, { "path": "pkg/uploader/kopia/block_restore_windows.go", "content": "//go:build windows\n// +build windows\n\n/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kopia\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\t\"github.com/kopia/kopia/fs\"\n\t\"github.com/kopia/kopia/snapshot/restore\"\n)\n\ntype BlockOutput struct {\n\t*restore.FilesystemOutput\n\n\ttargetFileName string\n}\n\nfunc (o *BlockOutput) WriteFile(ctx context.Context, relativePath string, remoteFile fs.File, progressCb restore.FileWriteProgress) error {\n\treturn fmt.Errorf(\"block mode is not supported for Windows\")\n}\n\nfunc (o *BlockOutput) BeginDirectory(ctx context.Context, relativePath string, e fs.Directory) error {\n\treturn fmt.Errorf(\"block mode is not supported for Windows\")\n}\n\nfunc (o *BlockOutput) Flush() error {\n\treturn flushVolume(o.targetFileName)\n}\n\nfunc (o *BlockOutput) Terminate() error {\n\treturn nil\n}\n" }, { "path": "pkg/uploader/kopia/flush_volume_linux.go", "content": "//go:build linux\n// +build linux\n\n/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kopia\n\nimport (\n\t\"os\"\n\n\t\"github.com/pkg/errors\"\n\t\"golang.org/x/sys/unix\"\n)\n\nfunc flushVolume(dirPath string) error {\n\tdir, err := os.Open(dirPath)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error opening dir %v\", dirPath)\n\t}\n\n\traw, err := dir.SyscallConn()\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error getting handle of dir %v\", dirPath)\n\t}\n\n\tvar syncErr error\n\tif err := raw.Control(func(fd uintptr) {\n\t\tif e := unix.Syncfs(int(fd)); e != nil {\n\t\t\tsyncErr = e\n\t\t}\n\t}); err != nil {\n\t\treturn errors.Wrapf(err, \"error calling fs sync from %v\", dirPath)\n\t}\n\n\treturn errors.Wrapf(syncErr, \"error syncing fs from %v\", dirPath)\n}\n" }, { "path": "pkg/uploader/kopia/flush_volume_other.go", "content": "//go:build !linux\n// +build !linux\n\n/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kopia\n\nfunc flushVolume(_ string) error {\n\treturn errFlushUnsupported\n}\n" }, { "path": "pkg/uploader/kopia/progress.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kopia\n\nimport (\n\t\"sync/atomic\"\n\t\"time\"\n\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\n\t\"github.com/kopia/kopia/snapshot/upload\"\n)\n\n// Throttle throttles controlle the interval of output result\ntype Throttle struct {\n\tthrottle int64\n\tinterval time.Duration\n}\n\nfunc (t *Throttle) ShouldOutput() bool {\n\tnextOutputTimeUnixNano := atomic.LoadInt64(&t.throttle)\n\tif nowNano := time.Now().UnixNano(); nowNano > nextOutputTimeUnixNano {\n\t\tif atomic.CompareAndSwapInt64(&t.throttle, nextOutputTimeUnixNano, nowNano+t.interval.Nanoseconds()) {\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n\n// Progress represents a backup or restore counters.\ntype Progress struct {\n\t// all int64 must precede all int32 due to alignment requirements on ARM\n\t// +checkatomic\n\tuploadedBytes int64 //the total bytes has uploaded\n\tcachedBytes int64 //the total bytes has cached\n\thashededBytes int64 //the total bytes has hashed\n\t// +checkatomic\n\tuploadedFiles int32 //the total files has ignored\n\t// +checkatomic\n\tignoredErrorCount int32 //the total errors has ignored\n\t// +checkatomic\n\tfatalErrorCount int32 //the total errors has occurred\n\testimatedFileCount int64 // +checklocksignore the total count of files to be processed\n\testimatedTotalBytes int64 // +checklocksignore\tthe total size of files to be processed\n\t// +checkatomic\n\tprocessedBytes int64 // which statistic all bytes has been processed currently\n\toutputThrottle Throttle // which control the frequency of update progress\n\tupdater uploader.ProgressUpdater //which kopia progress will call the UpdateProgress interface, the third party will implement the interface to do the progress update\n\tlog logrus.FieldLogger // output info into log when backup\n\testimationParam upload.EstimationParameters\n}\n\nfunc NewProgress(updater uploader.ProgressUpdater, interval time.Duration, log logrus.FieldLogger) *Progress {\n\treturn &Progress{\n\t\toutputThrottle: Throttle{\n\t\t\tthrottle: 0,\n\t\t\tinterval: interval,\n\t\t},\n\t\tupdater: updater,\n\t\testimationParam: upload.EstimationParameters{\n\t\t\tType: upload.EstimationTypeClassic,\n\t\t\tAdaptiveThreshold: 300000,\n\t\t},\n\t\tlog: log,\n\t}\n}\n\n// UploadedBytes the total bytes has uploaded currently\nfunc (p *Progress) UploadedBytes(numBytes int64) {\n\tatomic.AddInt64(&p.uploadedBytes, numBytes)\n\tatomic.AddInt32(&p.uploadedFiles, 1)\n\n\tp.UpdateProgress()\n}\n\n// Error statistic the total Error has occurred\nfunc (p *Progress) Error(path string, err error, isIgnored bool) {\n\tif isIgnored {\n\t\tatomic.AddInt32(&p.ignoredErrorCount, 1)\n\t\tp.log.Warnf(\"Ignored error when processing %v: %v\", path, err)\n\t} else {\n\t\tatomic.AddInt32(&p.fatalErrorCount, 1)\n\t\tp.log.Errorf(\"Error when processing %v: %v\", path, err)\n\t}\n}\n\n// EstimatedDataSize statistic the total size of files to be processed and total files to be processed\nfunc (p *Progress) EstimatedDataSize(fileCount int64, totalBytes int64) {\n\tatomic.StoreInt64(&p.estimatedTotalBytes, totalBytes)\n\tatomic.StoreInt64(&p.estimatedFileCount, fileCount)\n\n\tp.UpdateProgress()\n}\n\n// UpdateProgress which calls Updater UpdateProgress interface, update progress by third-party implementation\nfunc (p *Progress) UpdateProgress() {\n\tif p.outputThrottle.ShouldOutput() {\n\t\tp.updater.UpdateProgress(&uploader.Progress{TotalBytes: p.estimatedTotalBytes, BytesDone: p.processedBytes})\n\t}\n}\n\n// UploadStarted statistic the total Error has occurred\nfunc (p *Progress) UploadStarted() {}\n\n// CachedFile statistic the total bytes been cached currently\nfunc (p *Progress) CachedFile(fname string, numBytes int64) {\n\tatomic.AddInt64(&p.cachedBytes, numBytes)\n\tatomic.AddInt64(&p.processedBytes, numBytes)\n\tp.UpdateProgress()\n}\n\n// HashedBytes statistic the total bytes been hashed currently\nfunc (p *Progress) HashedBytes(numBytes int64) {\n\tatomic.AddInt64(&p.processedBytes, numBytes)\n\tatomic.AddInt64(&p.hashededBytes, numBytes)\n\tp.UpdateProgress()\n}\n\n// HashingFile statistic the file been hashed currently\nfunc (p *Progress) HashingFile(fname string) {}\n\n// ExcludedFile statistic the file been excluded currently\nfunc (p *Progress) ExcludedFile(fname string, numBytes int64) {}\n\n// ExcludedDir statistic the dir been excluded currently\nfunc (p *Progress) ExcludedDir(dirname string) {\n\tp.log.Infof(\"Excluded dir %s\", dirname)\n}\n\n// FinishedHashingFile which will called when specific file finished hash\nfunc (p *Progress) FinishedHashingFile(fname string, numBytes int64) {\n\tp.UpdateProgress()\n}\n\n// StartedDirectory called when begin to upload one directory\nfunc (p *Progress) StartedDirectory(dirname string) {}\n\n// FinishedDirectory called when finish to upload one directory\nfunc (p *Progress) FinishedDirectory(dirname string) {\n\tp.UpdateProgress()\n}\n\n// UploadFinished which report the files flushed after the Upload has completed.\nfunc (p *Progress) UploadFinished() {\n\tp.UpdateProgress()\n}\n\n// ProgressBytes which statistic all bytes has been processed currently\nfunc (p *Progress) ProgressBytes(processedBytes int64, totalBytes int64) {\n\tatomic.StoreInt64(&p.processedBytes, processedBytes)\n\tatomic.StoreInt64(&p.estimatedTotalBytes, totalBytes)\n\tp.UpdateProgress()\n}\n\nfunc (p *Progress) FinishedFile(fname string, err error) {}\n\nfunc (p *Progress) EstimationParameters() upload.EstimationParameters {\n\treturn p.estimationParam\n}\n\nfunc (p *Progress) Enabled() bool {\n\treturn true\n}\n\nfunc (p *Progress) GetIncrementalSize() int64 {\n\treturn p.estimatedTotalBytes - p.cachedBytes\n}\n" }, { "path": "pkg/uploader/kopia/progress_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kopia\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n)\n\ntype fakeProgressUpdater struct{}\n\nfunc (f *fakeProgressUpdater) UpdateProgress(p *uploader.Progress) {}\n\nfunc TestThrottle_ShouldOutput(t *testing.T) {\n\ttestCases := []struct {\n\t\tinterval time.Duration\n\t\tthrottle int64\n\t\texpectedOutput bool\n\t}{\n\t\t{interval: time.Second, expectedOutput: true},\n\t\t{interval: time.Second, throttle: time.Now().UnixNano() + int64(time.Nanosecond*100000000), expectedOutput: false},\n\t}\n\n\tfor _, tc := range testCases {\n\t\t// Setup\n\t\tp := &Progress{\n\t\t\toutputThrottle: Throttle{\n\t\t\t\tinterval: tc.interval,\n\t\t\t\tthrottle: tc.throttle,\n\t\t\t},\n\t\t}\n\n\t\t// Perform the test\n\n\t\toutput := p.outputThrottle.ShouldOutput()\n\n\t\t// Verify the result\n\t\tif output != tc.expectedOutput {\n\t\t\tt.Errorf(\"Expected ShouldOutput to return %v, but got %v\", tc.expectedOutput, output)\n\t\t}\n\t}\n}\n\nfunc TestProgress(t *testing.T) {\n\tfileName := \"test-filename\"\n\tvar numBytes int64 = 1\n\ttestCases := []struct {\n\t\tinterval time.Duration\n\t\tthrottle int64\n\t}{\n\t\t{interval: time.Second},\n\t\t{interval: time.Second, throttle: time.Now().UnixNano() + int64(time.Nanosecond*10000)},\n\t}\n\n\tfor _, tc := range testCases {\n\t\t// Setup\n\t\tp := &Progress{\n\t\t\toutputThrottle: Throttle{\n\t\t\t\tinterval: tc.interval,\n\t\t\t\tthrottle: tc.throttle,\n\t\t\t},\n\t\t\tupdater: &fakeProgressUpdater{},\n\t\t\tlog: logrus.New(),\n\t\t}\n\n\t\t// All below calls put together for the implementation are empty or just very simple and just want to cover testing\n\t\t// If wanting to write unit tests for some functions could remove it and with writing new function alone\n\t\tp.UpdateProgress()\n\t\tp.UploadedBytes(numBytes)\n\t\tp.Error(\"test-path\", nil, true)\n\t\tp.Error(\"test-path\", errors.New(\"processing error\"), false)\n\t\tp.UploadStarted()\n\t\tp.EstimatedDataSize(1, numBytes)\n\t\tp.CachedFile(fileName, numBytes)\n\t\tp.HashedBytes(numBytes)\n\t\tp.HashingFile(fileName)\n\t\tp.ExcludedFile(fileName, numBytes)\n\t\tp.ExcludedDir(fileName)\n\t\tp.FinishedHashingFile(fileName, numBytes)\n\t\tp.StartedDirectory(fileName)\n\t\tp.FinishedDirectory(fileName)\n\t\tp.UploadFinished()\n\t\tp.ProgressBytes(numBytes, numBytes)\n\t\tp.FinishedFile(fileName, nil)\n\t}\n}\n" }, { "path": "pkg/uploader/kopia/restore_output.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kopia\n\nimport (\n\t\"github.com/kopia/kopia/snapshot/restore\"\n\t\"github.com/pkg/errors\"\n)\n\nvar errFlushUnsupported = errors.New(\"flush is not supported\")\n\ntype RestoreOutput interface {\n\trestore.Output\n\tFlush() error\n\tTerminate() error\n}\n" }, { "path": "pkg/uploader/kopia/shim.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kopia\n\nimport (\n\t\"context\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n\n\t\"github.com/kopia/kopia/repo\"\n\t\"github.com/kopia/kopia/repo/content\"\n\t\"github.com/kopia/kopia/repo/content/index\"\n\t\"github.com/kopia/kopia/repo/manifest\"\n\t\"github.com/kopia/kopia/repo/object\"\n)\n\n// shimRepository which is one adapter for unified repo and kopia.\n// it implement kopia RepositoryWriter interfaces\ntype shimRepository struct {\n\tudmRepo udmrepo.BackupRepo\n}\n\n// shimObjectWriter object writer for unifited repo\ntype shimObjectWriter struct {\n\trepoWriter udmrepo.ObjectWriter\n}\n\n// shimObjectReader object reader for unifited repo\ntype shimObjectReader struct {\n\trepoReader udmrepo.ObjectReader\n}\n\nfunc NewShimRepo(repo udmrepo.BackupRepo) repo.RepositoryWriter {\n\treturn &shimRepository{\n\t\tudmRepo: repo,\n\t}\n}\n\n// OpenObject open specific object\nfunc (sr *shimRepository) OpenObject(ctx context.Context, id object.ID) (object.Reader, error) {\n\treader, err := sr.udmRepo.OpenObject(ctx, udmrepo.ID(id.String()))\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"failed to open object with id %v\", id)\n\t}\n\tif reader == nil {\n\t\treturn nil, err\n\t}\n\n\treturn &shimObjectReader{\n\t\trepoReader: reader,\n\t}, err\n}\n\n// VerifyObject not supported\nfunc (sr *shimRepository) VerifyObject(ctx context.Context, id object.ID) ([]content.ID, error) {\n\treturn nil, errors.New(\"VerifyObject is not supported\")\n}\n\n// Get one or more manifest data that match the specific manifest id\nfunc (sr *shimRepository) GetManifest(ctx context.Context, id manifest.ID, payload any) (*manifest.EntryMetadata, error) {\n\trepoMani := udmrepo.RepoManifest{\n\t\tPayload: payload,\n\t}\n\n\tif err := sr.udmRepo.GetManifest(ctx, udmrepo.ID(id), &repoMani); err != nil {\n\t\treturn nil, errors.Wrapf(err, \"failed to get manifest with id %v\", id)\n\t}\n\treturn GetKopiaManifestEntry(repoMani.Metadata), nil\n}\n\n// Get one or more manifest data that match the given labels\nfunc (sr *shimRepository) FindManifests(ctx context.Context, labels map[string]string) ([]*manifest.EntryMetadata, error) {\n\tmetadata, err := sr.udmRepo.FindManifests(ctx, udmrepo.ManifestFilter{Labels: labels})\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"failed to get manifests with labels %v\", labels)\n\t}\n\treturn GetKopiaManifestEntries(metadata), nil\n}\n\n// GetKopiaManifestEntry get metadata from specific ManifestEntryMetadata\nfunc GetKopiaManifestEntry(uMani *udmrepo.ManifestEntryMetadata) *manifest.EntryMetadata {\n\tvar ret manifest.EntryMetadata\n\n\tret.ID = manifest.ID(uMani.ID)\n\tret.Labels = uMani.Labels\n\tret.Length = int(uMani.Length)\n\tret.ModTime = uMani.ModTime\n\n\treturn &ret\n}\n\n// GetKopiaManifestEntries get metadata list from specific ManifestEntryMetadata\nfunc GetKopiaManifestEntries(uMani []*udmrepo.ManifestEntryMetadata) []*manifest.EntryMetadata {\n\tvar ret []*manifest.EntryMetadata\n\n\tfor _, entry := range uMani {\n\t\tvar e manifest.EntryMetadata\n\t\te.ID = manifest.ID(entry.ID)\n\t\te.Labels = entry.Labels\n\t\te.Length = int(entry.Length)\n\t\te.ModTime = entry.ModTime\n\n\t\tret = append(ret, &e)\n\t}\n\n\treturn ret\n}\n\n// Time Get the local time of the unified repo\nfunc (sr *shimRepository) Time() time.Time {\n\treturn sr.udmRepo.Time()\n}\n\n// ClientOptions is not supported by unified repo\nfunc (sr *shimRepository) ClientOptions() repo.ClientOptions {\n\treturn repo.ClientOptions{}\n}\n\n// Refresh not supported\nfunc (sr *shimRepository) Refresh(ctx context.Context) error {\n\treturn errors.New(\"Refresh is not supported\")\n}\n\n// ContentInfo not supported\nfunc (sr *shimRepository) ContentInfo(ctx context.Context, contentID content.ID) (content.Info, error) {\n\treturn index.Info{}, errors.New(\"ContentInfo is not supported\")\n}\n\n// PrefetchContents is not supported by unified repo\nfunc (sr *shimRepository) PrefetchContents(ctx context.Context, contentIDs []content.ID, hint string) []content.ID {\n\treturn nil\n}\n\n// PrefetchObjects is not supported by unified repo\nfunc (sr *shimRepository) PrefetchObjects(ctx context.Context, objectIDs []object.ID, hint string) ([]content.ID, error) {\n\treturn nil, errors.New(\"PrefetchObjects is not supported\")\n}\n\n// UpdateDescription is not supported by unified repo\nfunc (sr *shimRepository) UpdateDescription(d string) {\n}\n\n// NewWriter is not supported by unified repo\nfunc (sr *shimRepository) NewWriter(ctx context.Context, option repo.WriteSessionOptions) (context.Context, repo.RepositoryWriter, error) {\n\treturn nil, nil, errors.New(\"NewWriter is not supported\")\n}\n\n// Close will close unified repo\nfunc (sr *shimRepository) Close(ctx context.Context) error {\n\treturn sr.udmRepo.Close(ctx)\n}\n\n// NewObjectWriter creates an object writer\nfunc (sr *shimRepository) NewObjectWriter(ctx context.Context, option object.WriterOptions) object.Writer {\n\tvar opt udmrepo.ObjectWriteOptions\n\topt.Description = option.Description\n\topt.Prefix = udmrepo.ID(option.Prefix)\n\topt.FullPath = \"\"\n\topt.AccessMode = udmrepo.ObjectDataAccessModeFile\n\topt.AsyncWrites = option.AsyncWrites\n\n\tif strings.HasPrefix(option.Description, \"DIR:\") {\n\t\topt.DataType = udmrepo.ObjectDataTypeMetadata\n\t} else {\n\t\topt.DataType = udmrepo.ObjectDataTypeData\n\t}\n\n\twriter := sr.udmRepo.NewObjectWriter(ctx, opt)\n\tif writer == nil {\n\t\treturn nil\n\t}\n\n\treturn &shimObjectWriter{\n\t\trepoWriter: writer,\n\t}\n}\n\n// PutManifest saves the given manifest payload with a set of labels.\nfunc (sr *shimRepository) PutManifest(ctx context.Context, labels map[string]string, payload any) (manifest.ID, error) {\n\tid, err := sr.udmRepo.PutManifest(ctx, udmrepo.RepoManifest{\n\t\tPayload: payload,\n\t\tMetadata: &udmrepo.ManifestEntryMetadata{\n\t\t\tLabels: labels,\n\t\t},\n\t})\n\n\treturn manifest.ID(id), err\n}\n\n// DeleteManifest deletes the manifest with a given ID.\nfunc (sr *shimRepository) DeleteManifest(ctx context.Context, id manifest.ID) error {\n\treturn sr.udmRepo.DeleteManifest(ctx, udmrepo.ID(id))\n}\n\nfunc (sr *shimRepository) ReplaceManifests(ctx context.Context, labels map[string]string, payload any) (manifest.ID, error) {\n\tconst minReplaceManifestTimeDelta = 100 * time.Millisecond\n\n\tmd, err := sr.FindManifests(ctx, labels)\n\tif err != nil {\n\t\treturn \"\", errors.Wrap(err, \"unable to load manifests\")\n\t}\n\n\tfor _, em := range md {\n\t\tage := sr.Time().Sub(em.ModTime)\n\t\tif age < minReplaceManifestTimeDelta {\n\t\t\ttime.Sleep(minReplaceManifestTimeDelta)\n\t\t}\n\n\t\tif err := sr.DeleteManifest(ctx, em.ID); err != nil {\n\t\t\treturn \"\", errors.Wrapf(err, \"unable to delete previous manifest %v\", em.ID)\n\t\t}\n\t}\n\n\treturn sr.PutManifest(ctx, labels, payload)\n}\n\n// Flush all the unifited repository data\nfunc (sr *shimRepository) Flush(ctx context.Context) error {\n\treturn sr.udmRepo.Flush(ctx)\n}\n\nfunc (sr *shimRepository) ConcatenateObjects(ctx context.Context, objectIDs []object.ID, opt repo.ConcatenateOptions) (object.ID, error) {\n\tif len(objectIDs) == 0 {\n\t\treturn object.EmptyID, errors.New(\"object list is empty\")\n\t}\n\n\tids := []udmrepo.ID{}\n\tfor _, id := range objectIDs {\n\t\tids = append(ids, udmrepo.ID(id.String()))\n\t}\n\n\tid, err := sr.udmRepo.ConcatenateObjects(ctx, ids)\n\tif err != nil {\n\t\treturn object.EmptyID, err\n\t}\n\n\treturn object.ParseID(string(id))\n}\n\nfunc (sr *shimRepository) OnSuccessfulFlush(callback repo.RepositoryWriterCallback) {\n}\n\n// Flush all the unifited repository data\nfunc (sr *shimObjectReader) Read(p []byte) (n int, err error) {\n\treturn sr.repoReader.Read(p)\n}\n\nfunc (sr *shimObjectReader) Seek(offset int64, whence int) (int64, error) {\n\treturn sr.repoReader.Seek(offset, whence)\n}\n\n// Close current io for ObjectReader\nfunc (sr *shimObjectReader) Close() error {\n\treturn sr.repoReader.Close()\n}\n\n// Length returns the logical size of the object\nfunc (sr *shimObjectReader) Length() int64 {\n\treturn sr.repoReader.Length()\n}\n\n// Write data\nfunc (sr *shimObjectWriter) Write(p []byte) (n int, err error) {\n\treturn sr.repoWriter.Write(p)\n}\n\n// Periodically called to preserve the state of data written to the repo so far.\nfunc (sr *shimObjectWriter) Checkpoint() (object.ID, error) {\n\tid, err := sr.repoWriter.Checkpoint()\n\tif err != nil {\n\t\treturn object.ID{}, err\n\t}\n\n\tobjID, err := object.ParseID(string(id))\n\tif err != nil {\n\t\treturn object.ID{}, errors.Wrapf(err, \"error to parse object ID from %v\", id)\n\t}\n\n\treturn objID, err\n}\n\n// Result returns the object's unified identifier after the write completes.\nfunc (sr *shimObjectWriter) Result() (object.ID, error) {\n\tid, err := sr.repoWriter.Result()\n\tif err != nil {\n\t\treturn object.ID{}, err\n\t}\n\n\tobjID, err := object.ParseID(string(id))\n\tif err != nil {\n\t\treturn object.ID{}, errors.Wrapf(err, \"error to parse object ID from %v\", id)\n\t}\n\n\treturn objID, err\n}\n\n// Close closes the repository and releases all resources.\nfunc (sr *shimObjectWriter) Close() error {\n\treturn sr.repoWriter.Close()\n}\n" }, { "path": "pkg/uploader/kopia/shim_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kopia\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/kopia/kopia/repo\"\n\t\"github.com/kopia/kopia/repo/content\"\n\t\"github.com/kopia/kopia/repo/manifest\"\n\t\"github.com/kopia/kopia/repo/object\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo/mocks\"\n)\n\nfunc TestShimRepo(t *testing.T) {\n\tctx := t.Context()\n\tbackupRepo := &mocks.BackupRepo{}\n\tbackupRepo.On(\"Time\").Return(time.Time{})\n\tshim := NewShimRepo(backupRepo)\n\t// All below calls put together for the implementation are empty or just very simple, and just want to cover testing\n\t// If wanting to write unit tests for some functions could remove it and with writing new function alone\n\tshim.VerifyObject(ctx, object.ID{})\n\tshim.Time()\n\tshim.ClientOptions()\n\tshim.Refresh(ctx)\n\tshim.ContentInfo(ctx, content.ID{})\n\tshim.PrefetchContents(ctx, []content.ID{}, \"hint\")\n\tshim.PrefetchObjects(ctx, []object.ID{}, \"hint\")\n\tshim.UpdateDescription(\"desc\")\n\tshim.NewWriter(ctx, repo.WriteSessionOptions{})\n\tshim.OnSuccessfulFlush(func(ctx context.Context, w repo.RepositoryWriter) error { return nil })\n\n\tbackupRepo.On(\"Close\", mock.Anything).Return(nil)\n\tNewShimRepo(backupRepo).Close(ctx)\n\n\tvar id udmrepo.ID\n\tbackupRepo.On(\"PutManifest\", mock.Anything, mock.Anything).Return(id, nil)\n\tNewShimRepo(backupRepo).PutManifest(ctx, map[string]string{}, nil)\n\n\tvar mf manifest.ID\n\tbackupRepo.On(\"DeleteManifest\", mock.Anything, mock.Anything).Return(nil)\n\tNewShimRepo(backupRepo).DeleteManifest(ctx, mf)\n\n\tbackupRepo.On(\"Flush\", mock.Anything).Return(nil)\n\tNewShimRepo(backupRepo).Flush(ctx)\n\n\tbackupRepo.On(\"NewObjectWriter\", mock.Anything, mock.Anything).Return(nil)\n\tNewShimRepo(backupRepo).NewObjectWriter(ctx, object.WriterOptions{})\n}\n\nfunc TestOpenObject(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackupRepo *mocks.BackupRepo\n\t\tisOpenObjectError bool\n\t\tisReaderNil bool\n\t}{\n\t\t{\n\t\t\tname: \"Success\",\n\t\t\tbackupRepo: func() *mocks.BackupRepo {\n\t\t\t\tbackupRepo := &mocks.BackupRepo{}\n\t\t\t\tbackupRepo.On(\"OpenObject\", mock.Anything, mock.Anything).Return(&shimObjectReader{}, nil)\n\t\t\t\treturn backupRepo\n\t\t\t}(),\n\t\t},\n\t\t{\n\t\t\tname: \"Open object error\",\n\t\t\tbackupRepo: func() *mocks.BackupRepo {\n\t\t\t\tbackupRepo := &mocks.BackupRepo{}\n\t\t\t\tbackupRepo.On(\"OpenObject\", mock.Anything, mock.Anything).Return(&shimObjectReader{}, errors.New(\"Error open object\"))\n\t\t\t\treturn backupRepo\n\t\t\t}(),\n\t\t\tisOpenObjectError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Get nil reader\",\n\t\t\tbackupRepo: func() *mocks.BackupRepo {\n\t\t\t\tbackupRepo := &mocks.BackupRepo{}\n\t\t\t\tbackupRepo.On(\"OpenObject\", mock.Anything, mock.Anything).Return(nil, nil)\n\t\t\t\treturn backupRepo\n\t\t\t}(),\n\t\t\tisReaderNil: true,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\t\t\treader, err := NewShimRepo(tc.backupRepo).OpenObject(ctx, object.ID{})\n\t\t\tif tc.isOpenObjectError {\n\t\t\t\trequire.ErrorContains(t, err, \"failed to open object\")\n\t\t\t} else if tc.isReaderNil {\n\t\t\t\tassert.Nil(t, reader)\n\t\t\t} else {\n\t\t\t\tassert.NotNil(t, reader)\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestFindManifests(t *testing.T) {\n\tmeta := []*udmrepo.ManifestEntryMetadata{}\n\ttests := []struct {\n\t\tname string\n\t\tbackupRepo *mocks.BackupRepo\n\t\tisGetManifestError bool\n\t}{\n\t\t{\n\t\t\tname: \"Success\",\n\t\t\tbackupRepo: func() *mocks.BackupRepo {\n\t\t\t\tbackupRepo := &mocks.BackupRepo{}\n\t\t\t\tbackupRepo.On(\"FindManifests\", mock.Anything, mock.Anything).Return(meta, nil)\n\t\t\t\treturn backupRepo\n\t\t\t}(),\n\t\t},\n\t\t{\n\t\t\tname: \"Failed to find manifest\",\n\t\t\tisGetManifestError: true,\n\t\t\tbackupRepo: func() *mocks.BackupRepo {\n\t\t\t\tbackupRepo := &mocks.BackupRepo{}\n\t\t\t\tbackupRepo.On(\"FindManifests\", mock.Anything, mock.Anything).Return(meta,\n\t\t\t\t\terrors.New(\"failed to find manifest\"))\n\t\t\t\treturn backupRepo\n\t\t\t}(),\n\t\t},\n\t}\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\t\t\t_, err := NewShimRepo(tc.backupRepo).FindManifests(ctx, map[string]string{})\n\t\t\tif tc.isGetManifestError {\n\t\t\t\trequire.ErrorContains(t, err, \"failed\")\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestShimObjReader(t *testing.T) {\n\treader := new(shimObjectReader)\n\tobjReader := &mocks.ObjectReader{}\n\treader.repoReader = objReader\n\t// All below calls put together for the implementation are empty or just very simple, and just want to cover testing\n\t// If wanting to write unit tests for some functions could remove it and with writing new function alone\n\tobjReader.On(\"Seek\", mock.Anything, mock.Anything).Return(int64(0), nil)\n\treader.Seek(int64(0), 0)\n\n\tobjReader.On(\"Read\", mock.Anything).Return(0, nil)\n\treader.Read(nil)\n\n\tobjReader.On(\"Close\").Return(nil)\n\treader.Close()\n\n\tobjReader.On(\"Length\").Return(int64(0))\n\treader.Length()\n}\n\nfunc TestShimObjWriter(t *testing.T) {\n\twriter := new(shimObjectWriter)\n\tobjWriter := &mocks.ObjectWriter{}\n\twriter.repoWriter = objWriter\n\t// All below calls put together for the implementation are empty or just very simple, and just want to cover testing\n\t// If wanting to write unit tests for some functions could remove it and with writing new function alone\n\tvar id udmrepo.ID\n\tobjWriter.On(\"Checkpoint\").Return(id, nil)\n\twriter.Checkpoint()\n\n\tobjWriter.On(\"Result\").Return(id, nil)\n\twriter.Result()\n\n\tobjWriter.On(\"Write\", mock.Anything).Return(0, nil)\n\twriter.Write(nil)\n\n\tobjWriter.On(\"Close\").Return(nil)\n\twriter.Close()\n}\n\nfunc TestReplaceManifests(t *testing.T) {\n\tmeta1 := udmrepo.ManifestEntryMetadata{\n\t\tID: \"mani-1\",\n\t}\n\n\tmeta2 := udmrepo.ManifestEntryMetadata{\n\t\tID: \"mani-2\",\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tbackupRepo *mocks.BackupRepo\n\t\tisGetManifestError bool\n\t\texpectedError string\n\t\texpectedID manifest.ID\n\t}{\n\t\t{\n\t\t\tname: \"Failed to find manifest\",\n\t\t\tisGetManifestError: true,\n\t\t\tbackupRepo: func() *mocks.BackupRepo {\n\t\t\t\tbackupRepo := &mocks.BackupRepo{}\n\t\t\t\tbackupRepo.On(\"FindManifests\", mock.Anything, mock.Anything).Return([]*udmrepo.ManifestEntryMetadata{},\n\t\t\t\t\terrors.New(\"fake-find-error\"))\n\t\t\t\treturn backupRepo\n\t\t\t}(),\n\t\t\texpectedError: \"unable to load manifests: failed to get manifests with labels map[]: fake-find-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"Failed to delete manifest\",\n\t\t\tisGetManifestError: true,\n\t\t\tbackupRepo: func() *mocks.BackupRepo {\n\t\t\t\tbackupRepo := &mocks.BackupRepo{}\n\t\t\t\tbackupRepo.On(\"FindManifests\", mock.Anything, mock.Anything).Return([]*udmrepo.ManifestEntryMetadata{\n\t\t\t\t\t&meta1,\n\t\t\t\t\t&meta2,\n\t\t\t\t}, nil)\n\t\t\t\tbackupRepo.On(\"Time\").Return(time.Now())\n\t\t\t\tbackupRepo.On(\"DeleteManifest\", mock.Anything, mock.Anything).Return(errors.New(\"fake-delete-error\"))\n\t\t\t\treturn backupRepo\n\t\t\t}(),\n\t\t\texpectedError: \"unable to delete previous manifest mani-1: fake-delete-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"Failed to put manifest\",\n\t\t\tbackupRepo: func() *mocks.BackupRepo {\n\t\t\t\tbackupRepo := &mocks.BackupRepo{}\n\t\t\t\tbackupRepo.On(\"FindManifests\", mock.Anything, mock.Anything).Return([]*udmrepo.ManifestEntryMetadata{\n\t\t\t\t\t&meta1,\n\t\t\t\t\t&meta2,\n\t\t\t\t}, nil)\n\t\t\t\tbackupRepo.On(\"Time\").Return(time.Now())\n\t\t\t\tbackupRepo.On(\"DeleteManifest\", mock.Anything, mock.Anything).Return(nil)\n\t\t\t\tbackupRepo.On(\"PutManifest\", mock.Anything, mock.Anything).Return(udmrepo.ID(\"\"), errors.New(\"fake-put-error\"))\n\t\t\t\treturn backupRepo\n\t\t\t}(),\n\t\t\texpectedError: \"fake-put-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"Success\",\n\t\t\tbackupRepo: func() *mocks.BackupRepo {\n\t\t\t\tbackupRepo := &mocks.BackupRepo{}\n\t\t\t\tbackupRepo.On(\"FindManifests\", mock.Anything, mock.Anything).Return([]*udmrepo.ManifestEntryMetadata{\n\t\t\t\t\t&meta1,\n\t\t\t\t\t&meta2,\n\t\t\t\t}, nil)\n\t\t\t\tbackupRepo.On(\"Time\").Return(time.Now())\n\t\t\t\tbackupRepo.On(\"DeleteManifest\", mock.Anything, mock.Anything).Return(nil)\n\t\t\t\tbackupRepo.On(\"PutManifest\", mock.Anything, mock.Anything).Return(udmrepo.ID(\"fake-id\"), nil)\n\t\t\t\treturn backupRepo\n\t\t\t}(),\n\t\t\texpectedID: manifest.ID(\"fake-id\"),\n\t\t},\n\t}\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\t\t\tid, err := NewShimRepo(tc.backupRepo).ReplaceManifests(ctx, map[string]string{}, nil)\n\n\t\t\tif tc.expectedError != \"\" {\n\t\t\t\trequire.EqualError(t, err, tc.expectedError)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tassert.Equal(t, tc.expectedID, id)\n\t\t})\n\t}\n}\n\nfunc TestConcatenateObjects(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackupRepo *mocks.BackupRepo\n\t\tobjectIDs []object.ID\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"empty object list\",\n\t\t\texpectedError: \"object list is empty\",\n\t\t},\n\t\t{\n\t\t\tname: \"concatenate error\",\n\t\t\tbackupRepo: func() *mocks.BackupRepo {\n\t\t\t\tbackupRepo := &mocks.BackupRepo{}\n\t\t\t\tbackupRepo.On(\"ConcatenateObjects\", mock.Anything, mock.Anything).Return(udmrepo.ID(\"\"), errors.New(\"fake-concatenate-error\"))\n\t\t\t\treturn backupRepo\n\t\t\t}(),\n\t\t\tobjectIDs: []object.ID{\n\t\t\t\t{},\n\t\t\t},\n\t\t\texpectedError: \"fake-concatenate-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"parse error\",\n\t\t\tbackupRepo: func() *mocks.BackupRepo {\n\t\t\t\tbackupRepo := &mocks.BackupRepo{}\n\t\t\t\tbackupRepo.On(\"ConcatenateObjects\", mock.Anything, mock.Anything).Return(udmrepo.ID(\"fake-id\"), nil)\n\t\t\t\treturn backupRepo\n\t\t\t}(),\n\t\t\tobjectIDs: []object.ID{\n\t\t\t\t{},\n\t\t\t},\n\t\t\texpectedError: \"malformed content ID: \\\"fake-id\\\": invalid content prefix\",\n\t\t},\n\t\t{\n\t\t\tname: \"success\",\n\t\t\tbackupRepo: func() *mocks.BackupRepo {\n\t\t\t\tbackupRepo := &mocks.BackupRepo{}\n\t\t\t\tbackupRepo.On(\"ConcatenateObjects\", mock.Anything, mock.Anything).Return(udmrepo.ID(\"I123456\"), nil)\n\t\t\t\treturn backupRepo\n\t\t\t}(),\n\t\t\tobjectIDs: []object.ID{\n\t\t\t\t{},\n\t\t\t},\n\t\t},\n\t}\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tctx := t.Context()\n\t\t\t_, err := NewShimRepo(tc.backupRepo).ConcatenateObjects(ctx, tc.objectIDs, repo.ConcatenateOptions{})\n\n\t\t\tif tc.expectedError != \"\" {\n\t\t\t\tassert.EqualError(t, err, tc.expectedError)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/uploader/kopia/snapshot.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kopia\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"math\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"runtime\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/kopia/kopia/fs\"\n\t\"github.com/kopia/kopia/fs/localfs\"\n\t\"github.com/kopia/kopia/repo\"\n\t\"github.com/kopia/kopia/repo/manifest\"\n\t\"github.com/kopia/kopia/snapshot\"\n\t\"github.com/kopia/kopia/snapshot/policy\"\n\t\"github.com/kopia/kopia/snapshot/restore\"\n\t\"github.com/kopia/kopia/snapshot/snapshotfs\"\n\t\"github.com/pkg/errors\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/kopia\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\tuploaderutil \"github.com/vmware-tanzu/velero/pkg/uploader/util\"\n)\n\n// All function mainly used to make testing more convenient\nvar applyRetentionPolicyFunc = policy.ApplyRetentionPolicy\nvar treeForSourceFunc = policy.TreeForSource\nvar setPolicyFunc = policy.SetPolicy\nvar saveSnapshotFunc = snapshot.SaveSnapshot\nvar loadSnapshotFunc = snapshot.LoadSnapshot\nvar listSnapshotsFunc = snapshot.ListSnapshots\nvar filesystemEntryFunc = snapshotfs.FilesystemEntryFromIDWithPath\nvar restoreEntryFunc = restore.Entry\nvar flushVolumeFunc = flushVolume\n\nconst UploaderConfigMultipartKey = \"uploader-multipart\"\nconst MaxErrorReported = 10\n\n// SnapshotUploader which mainly used for UT test that could overwrite Upload interface\ntype SnapshotUploader interface {\n\tUpload(\n\t\tctx context.Context,\n\t\tsource fs.Entry,\n\t\tpolicyTree *policy.Tree,\n\t\tsourceInfo snapshot.SourceInfo,\n\t\tpreviousManifests ...*snapshot.Manifest,\n\t) (*snapshot.Manifest, error)\n}\n\nfunc newOptionalInt(b int) *policy.OptionalInt {\n\tob := policy.OptionalInt(b)\n\treturn &ob\n}\n\nfunc newOptionalInt64(b int64) *policy.OptionalInt64 {\n\tob := policy.OptionalInt64(b)\n\treturn &ob\n}\n\nfunc newOptionalBool(b bool) *policy.OptionalBool {\n\tob := policy.OptionalBool(b)\n\treturn &ob\n}\n\nfunc getDefaultPolicy() *policy.Policy {\n\treturn &policy.Policy{\n\t\tRetentionPolicy: policy.RetentionPolicy{\n\t\t\tKeepLatest: newOptionalInt(math.MaxInt32),\n\t\t\tKeepAnnual: newOptionalInt(math.MaxInt32),\n\t\t\tKeepDaily: newOptionalInt(math.MaxInt32),\n\t\t\tKeepHourly: newOptionalInt(math.MaxInt32),\n\t\t\tKeepMonthly: newOptionalInt(math.MaxInt32),\n\t\t\tKeepWeekly: newOptionalInt(math.MaxInt32),\n\t\t},\n\t\tCompressionPolicy: policy.CompressionPolicy{\n\t\t\tCompressorName: \"none\",\n\t\t},\n\t\tUploadPolicy: policy.UploadPolicy{\n\t\t\tMaxParallelFileReads: newOptionalInt(runtime.NumCPU()),\n\t\t\tParallelUploadAboveSize: newOptionalInt64(math.MaxInt64),\n\t\t},\n\t\tSchedulingPolicy: policy.SchedulingPolicy{\n\t\t\tManual: true,\n\t\t},\n\t\tErrorHandlingPolicy: policy.ErrorHandlingPolicy{\n\t\t\tIgnoreUnknownTypes: newOptionalBool(true),\n\t\t},\n\t}\n}\n\nfunc setupPolicy(ctx context.Context, rep repo.RepositoryWriter, sourceInfo snapshot.SourceInfo, uploaderCfg map[string]string) (*policy.Tree, error) {\n\t// some internal operations from Kopia code retrieves policies from repo directly, so we need to persist the policy to repo\n\tcurPolicy := getDefaultPolicy()\n\n\tif len(uploaderCfg) > 0 {\n\t\tparallelUpload, err := uploaderutil.GetParallelFilesUpload(uploaderCfg)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrap(err, \"failed to get uploader config\")\n\t\t}\n\t\tif parallelUpload > 0 {\n\t\t\tcurPolicy.UploadPolicy.MaxParallelFileReads = newOptionalInt(parallelUpload)\n\t\t}\n\t}\n\n\tif _, ok := uploaderCfg[UploaderConfigMultipartKey]; ok {\n\t\tcurPolicy.UploadPolicy.ParallelUploadAboveSize = newOptionalInt64(2 << 30)\n\t}\n\n\tif runtime.GOOS == \"windows\" {\n\t\tcurPolicy.FilesPolicy.IgnoreRules = []string{\"/System Volume Information/\", \"/$Recycle.Bin/\"}\n\t}\n\n\terr := setPolicyFunc(ctx, rep, sourceInfo, curPolicy)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error to set policy\")\n\t}\n\n\terr = rep.Flush(ctx)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error to flush repo\")\n\t}\n\n\t// retrieve policy from repo\n\tpolicyTree, err := treeForSourceFunc(ctx, rep, sourceInfo)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error to retrieve policy\")\n\t}\n\n\treturn policyTree, nil\n}\n\n// Backup backup specific sourcePath and update progress\nfunc Backup(ctx context.Context, fsUploader SnapshotUploader, repoWriter repo.RepositoryWriter, sourcePath string, realSource string,\n\tforceFull bool, parentSnapshot string, volMode uploader.PersistentVolumeMode, uploaderCfg map[string]string, tags map[string]string, log logrus.FieldLogger) (*uploader.SnapshotInfo, bool, error) {\n\tif fsUploader == nil {\n\t\treturn nil, false, errors.New(\"get empty kopia uploader\")\n\t}\n\tsource, err := filepath.Abs(sourcePath)\n\tif err != nil {\n\t\treturn nil, false, errors.Wrapf(err, \"Invalid source path '%s'\", sourcePath)\n\t}\n\n\tsource = filepath.Clean(source)\n\n\tsourceInfo := snapshot.SourceInfo{\n\t\tUserName: udmrepo.GetRepoUser(),\n\t\tHost: udmrepo.GetRepoDomain(),\n\t\tPath: filepath.Clean(realSource),\n\t}\n\tif realSource == \"\" {\n\t\tsourceInfo.Path = source\n\t}\n\n\tvar sourceEntry fs.Entry\n\n\tif volMode == uploader.PersistentVolumeBlock {\n\t\tsourceEntry, err = getLocalBlockEntry(source)\n\t\tif err != nil {\n\t\t\treturn nil, false, errors.Wrap(err, \"unable to get local block device entry\")\n\t\t}\n\t} else {\n\t\tsourceEntry, err = getLocalFSEntry(source)\n\t\tif err != nil {\n\t\t\treturn nil, false, errors.Wrap(err, \"unable to get local filesystem entry\")\n\t\t}\n\t}\n\n\tkopiaCtx := kopia.SetupKopiaLog(ctx, log)\n\n\tsnapID, snapshotSize, err := SnapshotSource(kopiaCtx, repoWriter, fsUploader, sourceInfo, sourceEntry, forceFull, parentSnapshot, tags, uploaderCfg, log, \"Kopia Uploader\")\n\tsnapshotInfo := &uploader.SnapshotInfo{\n\t\tID: snapID,\n\t\tSize: snapshotSize,\n\t}\n\n\treturn snapshotInfo, false, err\n}\n\nfunc getLocalFSEntry(path0 string) (fs.Entry, error) {\n\tpath, err := resolveSymlink(path0)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"resolveSymlink\")\n\t}\n\n\te, err := localfs.NewEntry(path)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"can't get local fs entry\")\n\t}\n\n\treturn e, nil\n}\n\n// resolveSymlink returns the path name after the evaluation of any symbolic links\nfunc resolveSymlink(path string) (string, error) {\n\tst, err := os.Lstat(path)\n\tif err != nil {\n\t\treturn \"\", errors.Wrap(err, \"stat\")\n\t}\n\n\tif (st.Mode() & os.ModeSymlink) == 0 {\n\t\treturn path, nil\n\t}\n\n\treturn filepath.EvalSymlinks(path)\n}\n\n// SnapshotSource which setup policy for snapshot, upload snapshot, update progress\nfunc SnapshotSource(\n\tctx context.Context,\n\trep repo.RepositoryWriter,\n\tu SnapshotUploader,\n\tsourceInfo snapshot.SourceInfo,\n\trootDir fs.Entry,\n\tforceFull bool,\n\tparentSnapshot string,\n\tsnapshotTags map[string]string,\n\tuploaderCfg map[string]string,\n\tlog logrus.FieldLogger,\n\tdescription string,\n) (string, int64, error) {\n\tlog.Info(\"Start to snapshot...\")\n\tsnapshotStartTime := time.Now()\n\n\tvar previous []*snapshot.Manifest\n\tif !forceFull {\n\t\tif parentSnapshot != \"\" {\n\t\t\tlog.Infof(\"Using provided parent snapshot %s\", parentSnapshot)\n\n\t\t\tmani, err := loadSnapshotFunc(ctx, rep, manifest.ID(parentSnapshot))\n\t\t\tif err != nil {\n\t\t\t\tlog.WithError(err).Warnf(\"Failed to load previous snapshot %v from kopia, fallback to full backup\", parentSnapshot)\n\t\t\t} else {\n\t\t\t\tprevious = append(previous, mani)\n\t\t\t}\n\t\t} else {\n\t\t\tlog.Infof(\"Searching for parent snapshot\")\n\n\t\t\tpre, err := findPreviousSnapshotManifest(ctx, rep, sourceInfo, snapshotTags, nil, log)\n\t\t\tif err != nil {\n\t\t\t\treturn \"\", 0, errors.Wrapf(err, \"Failed to find previous kopia snapshot manifests for si %v\", sourceInfo)\n\t\t\t}\n\n\t\t\tprevious = pre\n\t\t}\n\t} else {\n\t\tlog.Info(\"Forcing full snapshot\")\n\t}\n\n\tfor i := range previous {\n\t\tlog.Infof(\"Using parent snapshot %s, start time %v, end time %v, description %s\", previous[i].ID, previous[i].StartTime.ToTime(), previous[i].EndTime.ToTime(), previous[i].Description)\n\t}\n\n\tpolicyTree, err := setupPolicy(ctx, rep, sourceInfo, uploaderCfg)\n\tif err != nil {\n\t\treturn \"\", 0, errors.Wrapf(err, \"unable to set policy for si %v\", sourceInfo)\n\t}\n\n\tmanifest, err := u.Upload(ctx, rootDir, policyTree, sourceInfo, previous...)\n\tif err != nil {\n\t\treturn \"\", 0, errors.Wrapf(err, \"Failed to upload the kopia snapshot for si %v\", sourceInfo)\n\t}\n\n\tmanifest.Tags = snapshotTags\n\n\tmanifest.Description = description\n\tmanifest.Pins = []string{\"velero-pin\"}\n\n\tif _, err = saveSnapshotFunc(ctx, rep, manifest); err != nil {\n\t\treturn \"\", 0, errors.Wrapf(err, \"Failed to save kopia manifest %v\", manifest.ID)\n\t}\n\n\t_, err = applyRetentionPolicyFunc(ctx, rep, sourceInfo, true)\n\tif err != nil {\n\t\treturn \"\", 0, errors.Wrapf(err, \"Failed to apply kopia retention policy for si %v\", sourceInfo)\n\t}\n\n\tif err = rep.Flush(ctx); err != nil {\n\t\treturn \"\", 0, errors.Wrapf(err, \"Failed to flush kopia repository\")\n\t}\n\tlog.Infof(\"Created snapshot with root %v and ID %v in %v\", manifest.RootObjectID(), manifest.ID, time.Since(snapshotStartTime).Truncate(time.Second))\n\treturn reportSnapshotStatus(manifest, policyTree)\n}\n\nfunc reportSnapshotStatus(manifest *snapshot.Manifest, policyTree *policy.Tree) (string, int64, error) {\n\tmanifestID := manifest.ID\n\tsnapSize := manifest.Stats.TotalFileSize\n\n\tvar errs []string\n\tif ds := manifest.RootEntry.DirSummary; ds != nil {\n\t\tfor _, ent := range ds.FailedEntries {\n\t\t\tif len(errs) > MaxErrorReported {\n\t\t\t\terrs = append(errs, \"too many errors, ignored...\")\n\t\t\t\tbreak\n\t\t\t}\n\t\t\tpolicy := policyTree.EffectivePolicy()\n\t\t\tif !(policy != nil && bool(*policy.ErrorHandlingPolicy.IgnoreUnknownTypes) && strings.Contains(ent.Error, fs.ErrUnknown.Error())) {\n\t\t\t\terrs = append(errs, fmt.Sprintf(\"Error when processing %v: %v\", ent.EntryPath, ent.Error))\n\t\t\t}\n\t\t}\n\t}\n\n\tif len(errs) != 0 {\n\t\treturn string(manifestID), snapSize, errors.New(strings.Join(errs, \"\\n\"))\n\t}\n\n\treturn string(manifestID), snapSize, nil\n}\n\n// findPreviousSnapshotManifest returns the list of previous snapshots for a given source, including\n// last complete snapshot following it.\nfunc findPreviousSnapshotManifest(ctx context.Context, rep repo.Repository, sourceInfo snapshot.SourceInfo, snapshotTags map[string]string, noLaterThan *fs.UTCTimestamp, log logrus.FieldLogger) ([]*snapshot.Manifest, error) {\n\tman, err := listSnapshotsFunc(ctx, rep, sourceInfo)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tvar previousComplete *snapshot.Manifest\n\tvar result []*snapshot.Manifest\n\n\tfor _, p := range man {\n\t\tlog.Debugf(\"Found one snapshot %s, start time %v, incomplete %s, tags %v\", p.ID, p.StartTime.ToTime(), p.IncompleteReason, p.Tags)\n\n\t\trequester, found := p.Tags[uploader.SnapshotRequesterTag]\n\t\tif !found {\n\t\t\tcontinue\n\t\t}\n\n\t\tif requester != snapshotTags[uploader.SnapshotRequesterTag] {\n\t\t\tcontinue\n\t\t}\n\n\t\tuploaderName, found := p.Tags[uploader.SnapshotUploaderTag]\n\t\tif !found {\n\t\t\tcontinue\n\t\t}\n\n\t\tif uploaderName != snapshotTags[uploader.SnapshotUploaderTag] {\n\t\t\tcontinue\n\t\t}\n\n\t\tif noLaterThan != nil && p.StartTime.After(*noLaterThan) {\n\t\t\tcontinue\n\t\t}\n\n\t\tif p.IncompleteReason == \"\" && (previousComplete == nil || p.StartTime.After(previousComplete.StartTime)) {\n\t\t\tpreviousComplete = p\n\t\t}\n\t}\n\n\tif previousComplete != nil {\n\t\tresult = append(result, previousComplete)\n\t}\n\n\treturn result, nil\n}\n\ntype fileSystemRestoreOutput struct {\n\t*restore.FilesystemOutput\n}\n\nfunc (o *fileSystemRestoreOutput) Flush() error {\n\treturn flushVolumeFunc(o.TargetPath)\n}\n\nfunc (o *fileSystemRestoreOutput) Terminate() error {\n\treturn nil\n}\n\n// Restore restore specific sourcePath with given snapshotID and update progress\nfunc Restore(ctx context.Context, rep repo.RepositoryWriter, progress *Progress, snapshotID, dest string, volMode uploader.PersistentVolumeMode, uploaderCfg map[string]string,\n\tlog logrus.FieldLogger, cancleCh chan struct{}) (int64, int32, error) {\n\tlog.Info(\"Start to restore...\")\n\n\tkopiaCtx := kopia.SetupKopiaLog(ctx, log)\n\n\tsnapshot, err := snapshot.LoadSnapshot(kopiaCtx, rep, manifest.ID(snapshotID))\n\tif err != nil {\n\t\treturn 0, 0, errors.Wrapf(err, \"Unable to load snapshot %v\", snapshotID)\n\t}\n\n\tlog.Infof(\"Restore from snapshot %s, description %s, created time %v, tags %v\", snapshotID, snapshot.Description, snapshot.EndTime.ToTime(), snapshot.Tags)\n\n\trootEntry, err := filesystemEntryFunc(kopiaCtx, rep, snapshotID, false)\n\tif err != nil {\n\t\treturn 0, 0, errors.Wrapf(err, \"Unable to get filesystem entry for snapshot %v\", snapshotID)\n\t}\n\n\tpath, err := filepath.Abs(dest)\n\tif err != nil {\n\t\treturn 0, 0, errors.Wrapf(err, \"Unable to resolve path %v\", dest)\n\t}\n\n\tfsOutput := &restore.FilesystemOutput{\n\t\tTargetPath: path,\n\t\tOverwriteDirectories: true,\n\t\tOverwriteFiles: true,\n\t\tOverwriteSymlinks: true,\n\t\tIgnorePermissionErrors: true,\n\t}\n\n\trestoreConcurrency := runtime.NumCPU()\n\n\tif len(uploaderCfg) > 0 {\n\t\twriteSparseFiles, err := uploaderutil.GetWriteSparseFiles(uploaderCfg)\n\t\tif err != nil {\n\t\t\treturn 0, 0, errors.Wrap(err, \"failed to get uploader config\")\n\t\t}\n\t\tif writeSparseFiles {\n\t\t\tfsOutput.WriteSparseFiles = true\n\t\t}\n\n\t\tconcurrency, err := uploaderutil.GetRestoreConcurrency(uploaderCfg)\n\t\tif err != nil {\n\t\t\treturn 0, 0, errors.Wrap(err, \"failed to get parallel restore uploader config\")\n\t\t}\n\t\tif concurrency > 0 {\n\t\t\trestoreConcurrency = concurrency\n\t\t}\n\t}\n\n\tlog.Debugf(\"Restore filesystem output %v, concurrency %d\", fsOutput, restoreConcurrency)\n\n\terr = fsOutput.Init(ctx)\n\tif err != nil {\n\t\treturn 0, 0, errors.Wrap(err, \"error to init output\")\n\t}\n\n\tvar output RestoreOutput\n\tif volMode == uploader.PersistentVolumeBlock {\n\t\toutput = &BlockOutput{\n\t\t\tFilesystemOutput: fsOutput,\n\t\t}\n\t} else {\n\t\toutput = &fileSystemRestoreOutput{\n\t\t\tFilesystemOutput: fsOutput,\n\t\t}\n\t}\n\n\tdefer func() {\n\t\tif err := output.Terminate(); err != nil {\n\t\t\tlog.Warnf(\"error terminating restore output for %v\", path)\n\t\t}\n\t}()\n\n\tstat, err := restoreEntryFunc(kopiaCtx, rep, output, rootEntry, restore.Options{\n\t\tParallel: restoreConcurrency,\n\t\tRestoreDirEntryAtDepth: math.MaxInt32,\n\t\tCancel: cancleCh,\n\t\tProgressCallback: func(ctx context.Context, stats restore.Stats) {\n\t\t\tprogress.ProgressBytes(stats.RestoredTotalFileSize, stats.EnqueuedTotalFileSize)\n\t\t},\n\t})\n\n\tif err != nil {\n\t\treturn 0, 0, errors.Wrapf(err, \"Failed to copy snapshot data to the target\")\n\t}\n\n\tif err := output.Flush(); err != nil {\n\t\tif err == errFlushUnsupported {\n\t\t\tlog.Warnf(\"Skip flushing data for %v under the current OS %v\", path, runtime.GOOS)\n\t\t} else {\n\t\t\treturn 0, 0, errors.Wrapf(err, \"Failed to flush data to target\")\n\t\t}\n\t} else {\n\t\tlog.Infof(\"Flush done for volume dir %v\", path)\n\t}\n\n\treturn stat.RestoredTotalFileSize, stat.RestoredFileCount, nil\n}\n" }, { "path": "pkg/uploader/kopia/snapshot_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kopia\n\nimport (\n\t\"context\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/kopia/kopia/fs\"\n\t\"github.com/kopia/kopia/fs/virtualfs\"\n\t\"github.com/kopia/kopia/repo\"\n\t\"github.com/kopia/kopia/repo/manifest\"\n\t\"github.com/kopia/kopia/snapshot\"\n\t\"github.com/kopia/kopia/snapshot/policy\"\n\t\"github.com/kopia/kopia/snapshot/restore\"\n\t\"github.com/kopia/kopia/snapshot/snapshotfs\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\n\trepomocks \"github.com/vmware-tanzu/velero/pkg/repository/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\tuploadermocks \"github.com/vmware-tanzu/velero/pkg/uploader/mocks\"\n)\n\ntype snapshotMockes struct {\n\tpolicyMock *uploadermocks.Policy\n\tsnapshotMock *uploadermocks.Snapshot\n\tuploderMock *uploadermocks.Uploader\n\trepoWriterMock *repomocks.RepositoryWriter\n}\n\ntype mockArgs struct {\n\tmethodName string\n\treturns []any\n}\n\nfunc injectSnapshotFuncs() *snapshotMockes {\n\ts := &snapshotMockes{\n\t\tpolicyMock: &uploadermocks.Policy{},\n\t\tsnapshotMock: &uploadermocks.Snapshot{},\n\t\tuploderMock: &uploadermocks.Uploader{},\n\t\trepoWriterMock: &repomocks.RepositoryWriter{},\n\t}\n\n\tapplyRetentionPolicyFunc = s.policyMock.ApplyRetentionPolicy\n\tsetPolicyFunc = s.policyMock.SetPolicy\n\ttreeForSourceFunc = s.policyMock.TreeForSource\n\tloadSnapshotFunc = s.snapshotMock.LoadSnapshot\n\tsaveSnapshotFunc = s.snapshotMock.SaveSnapshot\n\treturn s\n}\n\nfunc MockFuncs(s *snapshotMockes, args []mockArgs) {\n\ts.snapshotMock.On(\"LoadSnapshot\", mock.Anything, mock.Anything, mock.Anything).Return(args[0].returns...)\n\ts.snapshotMock.On(\"SaveSnapshot\", mock.Anything, mock.Anything, mock.Anything).Return(args[1].returns...)\n\ts.policyMock.On(\"TreeForSource\", mock.Anything, mock.Anything, mock.Anything).Return(args[2].returns...)\n\ts.policyMock.On(\"ApplyRetentionPolicy\", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(args[3].returns...)\n\ts.policyMock.On(\"SetPolicy\", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(args[4].returns...)\n\ts.uploderMock.On(\"Upload\", mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(args[5].returns...)\n\ts.repoWriterMock.On(\"Flush\", mock.Anything).Return(args[6].returns...)\n}\n\nfunc TestSnapshotSource(t *testing.T) {\n\tctx := t.Context()\n\tsourceInfo := snapshot.SourceInfo{\n\t\tUserName: \"testUserName\",\n\t\tHost: \"testHost\",\n\t\tPath: \"/var\",\n\t}\n\trootDir, err := getLocalFSEntry(sourceInfo.Path)\n\trequire.NoError(t, err)\n\tlog := logrus.New()\n\tmanifest := &snapshot.Manifest{\n\t\tID: \"test\",\n\t\tRootEntry: &snapshot.DirEntry{},\n\t}\n\n\ttestCases := []struct {\n\t\tname string\n\t\targs []mockArgs\n\t\tuploaderCfg map[string]string\n\t\tnotError bool\n\t}{\n\t\t{\n\t\t\tname: \"regular test\",\n\t\t\targs: []mockArgs{\n\t\t\t\t{methodName: \"LoadSnapshot\", returns: []any{manifest, nil}},\n\t\t\t\t{methodName: \"SaveSnapshot\", returns: []any{manifest.ID, nil}},\n\t\t\t\t{methodName: \"TreeForSource\", returns: []any{nil, nil}},\n\t\t\t\t{methodName: \"ApplyRetentionPolicy\", returns: []any{nil, nil}},\n\t\t\t\t{methodName: \"SetPolicy\", returns: []any{nil}},\n\t\t\t\t{methodName: \"Upload\", returns: []any{manifest, nil}},\n\t\t\t\t{methodName: \"Flush\", returns: []any{nil}},\n\t\t\t},\n\t\t\tnotError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"failed to load snapshot, should fallback to full backup and not error\",\n\t\t\targs: []mockArgs{\n\t\t\t\t{methodName: \"LoadSnapshot\", returns: []any{manifest, errors.New(\"failed to load snapshot\")}},\n\t\t\t\t{methodName: \"SaveSnapshot\", returns: []any{manifest.ID, nil}},\n\t\t\t\t{methodName: \"TreeForSource\", returns: []any{nil, nil}},\n\t\t\t\t{methodName: \"ApplyRetentionPolicy\", returns: []any{nil, nil}},\n\t\t\t\t{methodName: \"SetPolicy\", returns: []any{nil}},\n\t\t\t\t{methodName: \"Upload\", returns: []any{manifest, nil}},\n\t\t\t\t{methodName: \"Flush\", returns: []any{nil}},\n\t\t\t},\n\t\t\tnotError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"failed to save snapshot\",\n\t\t\targs: []mockArgs{\n\t\t\t\t{methodName: \"LoadSnapshot\", returns: []any{manifest, nil}},\n\t\t\t\t{methodName: \"SaveSnapshot\", returns: []any{manifest.ID, errors.New(\"failed to save snapshot\")}},\n\t\t\t\t{methodName: \"TreeForSource\", returns: []any{nil, nil}},\n\t\t\t\t{methodName: \"ApplyRetentionPolicy\", returns: []any{nil, nil}},\n\t\t\t\t{methodName: \"SetPolicy\", returns: []any{nil}},\n\t\t\t\t{methodName: \"Upload\", returns: []any{manifest, nil}},\n\t\t\t\t{methodName: \"Flush\", returns: []any{nil}},\n\t\t\t},\n\t\t\tnotError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"failed to set policy\",\n\t\t\targs: []mockArgs{\n\t\t\t\t{methodName: \"LoadSnapshot\", returns: []any{manifest, nil}},\n\t\t\t\t{methodName: \"SaveSnapshot\", returns: []any{manifest.ID, nil}},\n\t\t\t\t{methodName: \"TreeForSource\", returns: []any{nil, nil}},\n\t\t\t\t{methodName: \"ApplyRetentionPolicy\", returns: []any{nil, nil}},\n\t\t\t\t{methodName: \"SetPolicy\", returns: []any{errors.New(\"failed to set policy\")}},\n\t\t\t\t{methodName: \"Upload\", returns: []any{manifest, nil}},\n\t\t\t\t{methodName: \"Flush\", returns: []any{nil}},\n\t\t\t},\n\t\t\tnotError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"set policy with parallel files upload\",\n\t\t\targs: []mockArgs{\n\t\t\t\t{methodName: \"LoadSnapshot\", returns: []any{manifest, nil}},\n\t\t\t\t{methodName: \"SaveSnapshot\", returns: []any{manifest.ID, nil}},\n\t\t\t\t{methodName: \"TreeForSource\", returns: []any{nil, nil}},\n\t\t\t\t{methodName: \"ApplyRetentionPolicy\", returns: []any{nil, nil}},\n\t\t\t\t{methodName: \"SetPolicy\", returns: []any{nil}},\n\t\t\t\t{methodName: \"Upload\", returns: []any{manifest, nil}},\n\t\t\t\t{methodName: \"Flush\", returns: []any{nil}},\n\t\t\t},\n\t\t\tuploaderCfg: map[string]string{\n\t\t\t\t\"ParallelFilesUpload\": \"10\",\n\t\t\t},\n\t\t\tnotError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"failed to upload snapshot\",\n\t\t\targs: []mockArgs{\n\t\t\t\t{methodName: \"LoadSnapshot\", returns: []any{manifest, nil}},\n\t\t\t\t{methodName: \"SaveSnapshot\", returns: []any{manifest.ID, nil}},\n\t\t\t\t{methodName: \"TreeForSource\", returns: []any{nil, nil}},\n\t\t\t\t{methodName: \"ApplyRetentionPolicy\", returns: []any{nil, nil}},\n\t\t\t\t{methodName: \"SetPolicy\", returns: []any{nil}},\n\t\t\t\t{methodName: \"Upload\", returns: []any{manifest, errors.New(\"failed to upload snapshot\")}},\n\t\t\t\t{methodName: \"Flush\", returns: []any{nil}},\n\t\t\t},\n\t\t\tnotError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"failed to flush repo\",\n\t\t\targs: []mockArgs{\n\t\t\t\t{methodName: \"LoadSnapshot\", returns: []any{manifest, nil}},\n\t\t\t\t{methodName: \"SaveSnapshot\", returns: []any{manifest.ID, errors.New(\"failed to save snapshot\")}},\n\t\t\t\t{methodName: \"TreeForSource\", returns: []any{nil, nil}},\n\t\t\t\t{methodName: \"ApplyRetentionPolicy\", returns: []any{nil, nil}},\n\t\t\t\t{methodName: \"SetPolicy\", returns: []any{nil}},\n\t\t\t\t{methodName: \"Upload\", returns: []any{manifest, nil}},\n\t\t\t\t{methodName: \"Flush\", returns: []any{errors.New(\"failed to flush repo\")}},\n\t\t\t},\n\t\t\tnotError: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\ts := injectSnapshotFuncs()\n\t\t\tMockFuncs(s, tc.args)\n\t\t\t_, _, err = SnapshotSource(ctx, s.repoWriterMock, s.uploderMock, sourceInfo, rootDir, false, \"/\", nil, tc.uploaderCfg, log, \"TestSnapshotSource\")\n\t\t\tif tc.notError {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.Error(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestReportSnapshotStatus(t *testing.T) {\n\ttestCases := []struct {\n\t\tshouldError bool\n\t\texpectedResult string\n\t\texpectedSize int64\n\t\tdirectorySummary *fs.DirectorySummary\n\t\texpectedErrors []string\n\t}{\n\t\t{\n\t\t\tshouldError: false,\n\t\t\texpectedResult: \"sample-manifest-id\",\n\t\t\texpectedSize: 1024,\n\t\t\tdirectorySummary: &fs.DirectorySummary{\n\t\t\t\tTotalFileSize: 1024,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tshouldError: true,\n\t\t\texpectedResult: \"sample-manifest-id\",\n\t\t\texpectedSize: 1024,\n\t\t\tdirectorySummary: &fs.DirectorySummary{\n\t\t\t\tFailedEntries: []*fs.EntryWithError{\n\t\t\t\t\t{\n\t\t\t\t\t\tEntryPath: \"/path/to/file.txt\",\n\t\t\t\t\t\tError: \"Unknown file error\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedErrors: []string{\"Error when processing /path/to/file.txt: Unknown file error\"},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tmanifest := &snapshot.Manifest{\n\t\t\tID: manifest.ID(\"sample-manifest-id\"),\n\t\t\tStats: snapshot.Stats{\n\t\t\t\tTotalFileSize: 1024,\n\t\t\t},\n\t\t\tRootEntry: &snapshot.DirEntry{\n\t\t\t\tDirSummary: tc.directorySummary,\n\t\t\t},\n\t\t}\n\n\t\tresult, size, err := reportSnapshotStatus(manifest, policy.BuildTree(nil, getDefaultPolicy()))\n\n\t\tswitch {\n\t\tcase tc.shouldError && err == nil:\n\t\t\tt.Errorf(\"expected error, but got nil\")\n\t\tcase !tc.shouldError && err != nil:\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\tcase tc.shouldError && err != nil:\n\t\t\texpectedErr := strings.Join(tc.expectedErrors, \"\\n\")\n\t\t\tif err.Error() != expectedErr {\n\t\t\t\tt.Errorf(\"unexpected error: got %v, want %v\", err, expectedErr)\n\t\t\t}\n\t\t}\n\n\t\tif result != tc.expectedResult {\n\t\t\tt.Errorf(\"unexpected result: got %v, want %v\", result, tc.expectedResult)\n\t\t}\n\n\t\tif size != tc.expectedSize {\n\t\t\tt.Errorf(\"unexpected size: got %v, want %v\", size, tc.expectedSize)\n\t\t}\n\t}\n}\n\nfunc TestFindPreviousSnapshotManifest(t *testing.T) {\n\t// Prepare test data\n\tsourceInfo := snapshot.SourceInfo{\n\t\tUserName: \"user1\",\n\t\tHost: \"host1\",\n\t\tPath: \"/path/to/dir1\",\n\t}\n\tsnapshotTags := map[string]string{\n\t\tuploader.SnapshotRequesterTag: \"user1\",\n\t\tuploader.SnapshotUploaderTag: \"uploader1\",\n\t}\n\tnoLaterThan := fs.UTCTimestampFromTime(time.Now())\n\n\ttestCases := []struct {\n\t\tname string\n\t\tlistSnapshotsFunc func(ctx context.Context, rep repo.Repository, si snapshot.SourceInfo) ([]*snapshot.Manifest, error)\n\t\texpectedSnapshots []*snapshot.Manifest\n\t\texpectedError error\n\t}{\n\t\t// No matching snapshots\n\t\t{\n\t\t\tname: \"No matching snapshots\",\n\t\t\tlistSnapshotsFunc: func(ctx context.Context, rep repo.Repository, si snapshot.SourceInfo) ([]*snapshot.Manifest, error) {\n\t\t\t\treturn []*snapshot.Manifest{}, nil\n\t\t\t},\n\t\t\texpectedSnapshots: []*snapshot.Manifest{},\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"Error getting manifest\",\n\t\t\tlistSnapshotsFunc: func(ctx context.Context, rep repo.Repository, si snapshot.SourceInfo) ([]*snapshot.Manifest, error) {\n\t\t\t\treturn []*snapshot.Manifest{}, errors.New(\"Error getting manifest\")\n\t\t\t},\n\t\t\texpectedSnapshots: []*snapshot.Manifest{},\n\t\t\texpectedError: errors.New(\"Error getting manifest\"),\n\t\t},\n\t\t// Only one matching snapshot\n\t\t{\n\t\t\tname: \"One matching snapshot\",\n\t\t\tlistSnapshotsFunc: func(ctx context.Context, rep repo.Repository, si snapshot.SourceInfo) ([]*snapshot.Manifest, error) {\n\t\t\t\treturn []*snapshot.Manifest{\n\t\t\t\t\t{\n\t\t\t\t\t\tTags: map[string]string{\n\t\t\t\t\t\t\tuploader.SnapshotRequesterTag: \"user1\",\n\t\t\t\t\t\t\tuploader.SnapshotUploaderTag: \"uploader1\",\n\t\t\t\t\t\t\t\"otherTag\": \"value\",\n\t\t\t\t\t\t\t\"anotherCustomTag\": \"123\",\n\t\t\t\t\t\t\t\"snapshotRequestor\": \"user1\",\n\t\t\t\t\t\t\t\"snapshotUploader\": \"uploader1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t}, nil\n\t\t\t},\n\t\t\texpectedSnapshots: []*snapshot.Manifest{\n\t\t\t\t{\n\t\t\t\t\tTags: map[string]string{\n\t\t\t\t\t\tuploader.SnapshotRequesterTag: \"user1\",\n\t\t\t\t\t\tuploader.SnapshotUploaderTag: \"uploader1\",\n\t\t\t\t\t\t\"otherTag\": \"value\",\n\t\t\t\t\t\t\"anotherCustomTag\": \"123\",\n\t\t\t\t\t\t\"snapshotRequestor\": \"user1\",\n\t\t\t\t\t\t\"snapshotUploader\": \"uploader1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedError: nil,\n\t\t},\n\t\t// Multiple matching snapshots\n\t\t{\n\t\t\tname: \"Multiple matching snapshots\",\n\t\t\tlistSnapshotsFunc: func(ctx context.Context, rep repo.Repository, si snapshot.SourceInfo) ([]*snapshot.Manifest, error) {\n\t\t\t\treturn []*snapshot.Manifest{\n\t\t\t\t\t{\n\t\t\t\t\t\tTags: map[string]string{\n\t\t\t\t\t\t\tuploader.SnapshotRequesterTag: \"user1\",\n\t\t\t\t\t\t\tuploader.SnapshotUploaderTag: \"uploader1\",\n\t\t\t\t\t\t\t\"otherTag\": \"value1\",\n\t\t\t\t\t\t\t\"snapshotRequestor\": \"user1\",\n\t\t\t\t\t\t\t\"snapshotUploader\": \"uploader1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tTags: map[string]string{\n\t\t\t\t\t\t\tuploader.SnapshotRequesterTag: \"user1\",\n\t\t\t\t\t\t\tuploader.SnapshotUploaderTag: \"uploader1\",\n\t\t\t\t\t\t\t\"otherTag\": \"value2\",\n\t\t\t\t\t\t\t\"snapshotRequestor\": \"user1\",\n\t\t\t\t\t\t\t\"snapshotUploader\": \"uploader1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t}, nil\n\t\t\t},\n\t\t\texpectedSnapshots: []*snapshot.Manifest{\n\t\t\t\t{\n\t\t\t\t\tTags: map[string]string{\n\t\t\t\t\t\tuploader.SnapshotRequesterTag: \"user1\",\n\t\t\t\t\t\tuploader.SnapshotUploaderTag: \"uploader1\",\n\t\t\t\t\t\t\"otherTag\": \"value1\",\n\t\t\t\t\t\t\"snapshotRequestor\": \"user1\",\n\t\t\t\t\t\t\"snapshotUploader\": \"uploader1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedError: nil,\n\t\t},\n\t\t// Snapshot with different requester\n\t\t{\n\t\t\tname: \"Snapshot with different requester\",\n\t\t\tlistSnapshotsFunc: func(ctx context.Context, rep repo.Repository, si snapshot.SourceInfo) ([]*snapshot.Manifest, error) {\n\t\t\t\treturn []*snapshot.Manifest{\n\t\t\t\t\t{\n\t\t\t\t\t\tTags: map[string]string{\n\t\t\t\t\t\t\tuploader.SnapshotRequesterTag: \"user2\",\n\t\t\t\t\t\t\tuploader.SnapshotUploaderTag: \"uploader1\",\n\t\t\t\t\t\t\t\"otherTag\": \"value\",\n\t\t\t\t\t\t\t\"snapshotRequestor\": \"user2\",\n\t\t\t\t\t\t\t\"snapshotUploader\": \"uploader1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t}, nil\n\t\t\t},\n\t\t\texpectedSnapshots: []*snapshot.Manifest{},\n\t\t\texpectedError: nil,\n\t\t},\n\t\t// Snapshot with different uploader\n\t\t{\n\t\t\tname: \"Snapshot with different uploader\",\n\t\t\tlistSnapshotsFunc: func(ctx context.Context, rep repo.Repository, si snapshot.SourceInfo) ([]*snapshot.Manifest, error) {\n\t\t\t\treturn []*snapshot.Manifest{\n\t\t\t\t\t{\n\t\t\t\t\t\tTags: map[string]string{\n\t\t\t\t\t\t\tuploader.SnapshotRequesterTag: \"user1\",\n\t\t\t\t\t\t\tuploader.SnapshotUploaderTag: \"uploader2\",\n\t\t\t\t\t\t\t\"otherTag\": \"value\",\n\t\t\t\t\t\t\t\"snapshotRequestor\": \"user1\",\n\t\t\t\t\t\t\t\"snapshotUploader\": \"uploader2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t}, nil\n\t\t\t},\n\t\t\texpectedSnapshots: []*snapshot.Manifest{},\n\t\t\texpectedError: nil,\n\t\t},\n\t\t// Snapshot with a later start time\n\t\t{\n\t\t\tname: \"Snapshot with a later start time\",\n\t\t\tlistSnapshotsFunc: func(ctx context.Context, rep repo.Repository, si snapshot.SourceInfo) ([]*snapshot.Manifest, error) {\n\t\t\t\treturn []*snapshot.Manifest{\n\t\t\t\t\t{\n\t\t\t\t\t\tTags: map[string]string{\n\t\t\t\t\t\t\tuploader.SnapshotRequesterTag: \"user1\",\n\t\t\t\t\t\t\tuploader.SnapshotUploaderTag: \"uploader1\",\n\t\t\t\t\t\t\t\"otherTag\": \"value\",\n\t\t\t\t\t\t\t\"snapshotRequestor\": \"user1\",\n\t\t\t\t\t\t\t\"snapshotUploader\": \"uploader1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tStartTime: fs.UTCTimestampFromTime(time.Now().Add(time.Hour)),\n\t\t\t\t\t},\n\t\t\t\t}, nil\n\t\t\t},\n\t\t\texpectedSnapshots: []*snapshot.Manifest{},\n\t\t\texpectedError: nil,\n\t\t},\n\t\t// Snapshot with incomplete reason\n\t\t{\n\t\t\tname: \"Snapshot with incomplete reason\",\n\t\t\tlistSnapshotsFunc: func(ctx context.Context, rep repo.Repository, si snapshot.SourceInfo) ([]*snapshot.Manifest, error) {\n\t\t\t\treturn []*snapshot.Manifest{\n\t\t\t\t\t{\n\t\t\t\t\t\tTags: map[string]string{\n\t\t\t\t\t\t\tuploader.SnapshotRequesterTag: \"user1\",\n\t\t\t\t\t\t\tuploader.SnapshotUploaderTag: \"uploader1\",\n\t\t\t\t\t\t\t\"otherTag\": \"value\",\n\t\t\t\t\t\t\t\"snapshotRequestor\": \"user1\",\n\t\t\t\t\t\t\t\"snapshotUploader\": \"uploader1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tIncompleteReason: \"reason\",\n\t\t\t\t\t},\n\t\t\t\t}, nil\n\t\t\t},\n\t\t\texpectedSnapshots: []*snapshot.Manifest{},\n\t\t\texpectedError: nil,\n\t\t},\n\t\t// Multiple snapshots with some matching conditions\n\t\t{\n\t\t\tname: \"Multiple snapshots with matching conditions\",\n\t\t\tlistSnapshotsFunc: func(ctx context.Context, rep repo.Repository, si snapshot.SourceInfo) ([]*snapshot.Manifest, error) {\n\t\t\t\treturn []*snapshot.Manifest{\n\t\t\t\t\t{\n\t\t\t\t\t\tTags: map[string]string{\n\t\t\t\t\t\t\tuploader.SnapshotRequesterTag: \"user1\",\n\t\t\t\t\t\t\tuploader.SnapshotUploaderTag: \"uploader1\",\n\t\t\t\t\t\t\t\"otherTag\": \"value1\",\n\t\t\t\t\t\t\t\"snapshotRequestor\": \"user1\",\n\t\t\t\t\t\t\t\"snapshotUploader\": \"uploader1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tTags: map[string]string{\n\t\t\t\t\t\t\tuploader.SnapshotRequesterTag: \"user1\",\n\t\t\t\t\t\t\tuploader.SnapshotUploaderTag: \"uploader1\",\n\t\t\t\t\t\t\t\"otherTag\": \"value2\",\n\t\t\t\t\t\t\t\"snapshotRequestor\": \"user1\",\n\t\t\t\t\t\t\t\"snapshotUploader\": \"uploader1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tStartTime: fs.UTCTimestampFromTime(time.Now().Add(-time.Hour)),\n\t\t\t\t\t\tIncompleteReason: \"reason\",\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tTags: map[string]string{\n\t\t\t\t\t\t\tuploader.SnapshotRequesterTag: \"user1\",\n\t\t\t\t\t\t\tuploader.SnapshotUploaderTag: \"uploader1\",\n\t\t\t\t\t\t\t\"otherTag\": \"value3\",\n\t\t\t\t\t\t\t\"snapshotRequestor\": \"user1\",\n\t\t\t\t\t\t\t\"snapshotUploader\": \"uploader1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tStartTime: fs.UTCTimestampFromTime(time.Now().Add(-time.Hour)),\n\t\t\t\t\t},\n\t\t\t\t}, nil\n\t\t\t},\n\t\t\texpectedSnapshots: []*snapshot.Manifest{\n\t\t\t\t{\n\t\t\t\t\tTags: map[string]string{\n\t\t\t\t\t\tuploader.SnapshotRequesterTag: \"user1\",\n\t\t\t\t\t\tuploader.SnapshotUploaderTag: \"uploader1\",\n\t\t\t\t\t\t\"otherTag\": \"value3\",\n\t\t\t\t\t\t\"snapshotRequestor\": \"user1\",\n\t\t\t\t\t\t\"snapshotUploader\": \"uploader1\",\n\t\t\t\t\t},\n\t\t\t\t\tStartTime: fs.UTCTimestampFromTime(time.Now().Add(-time.Hour)),\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedError: nil,\n\t\t},\n\t\t// Snapshot with manifest SnapshotRequesterTag not found\n\t\t{\n\t\t\tname: \"Snapshot with manifest SnapshotRequesterTag not found\",\n\t\t\tlistSnapshotsFunc: func(ctx context.Context, rep repo.Repository, si snapshot.SourceInfo) ([]*snapshot.Manifest, error) {\n\t\t\t\treturn []*snapshot.Manifest{\n\t\t\t\t\t{\n\t\t\t\t\t\tTags: map[string]string{\n\t\t\t\t\t\t\t\"requester\": \"user1\",\n\t\t\t\t\t\t\tuploader.SnapshotUploaderTag: \"uploader1\",\n\t\t\t\t\t\t\t\"otherTag\": \"value\",\n\t\t\t\t\t\t\t\"snapshotRequestor\": \"user1\",\n\t\t\t\t\t\t\t\"snapshotUploader\": \"uploader1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tIncompleteReason: \"reason\",\n\t\t\t\t\t},\n\t\t\t\t}, nil\n\t\t\t},\n\t\t\texpectedSnapshots: []*snapshot.Manifest{},\n\t\t\texpectedError: nil,\n\t\t},\n\t\t// Snapshot with manifest SnapshotRequesterTag not found\n\t\t{\n\t\t\tname: \"Snapshot with manifest SnapshotUploaderTag not found\",\n\t\t\tlistSnapshotsFunc: func(ctx context.Context, rep repo.Repository, si snapshot.SourceInfo) ([]*snapshot.Manifest, error) {\n\t\t\t\treturn []*snapshot.Manifest{\n\t\t\t\t\t{\n\t\t\t\t\t\tTags: map[string]string{\n\t\t\t\t\t\t\tuploader.SnapshotRequesterTag: \"user1\",\n\t\t\t\t\t\t\t\"uploader\": \"uploader1\",\n\t\t\t\t\t\t\t\"otherTag\": \"value\",\n\t\t\t\t\t\t\t\"snapshotRequestor\": \"user1\",\n\t\t\t\t\t\t\t\"snapshotUploader\": \"uploader1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tIncompleteReason: \"reason\",\n\t\t\t\t\t},\n\t\t\t\t}, nil\n\t\t\t},\n\t\t\texpectedSnapshots: []*snapshot.Manifest{},\n\t\t\texpectedError: nil,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar repo repo.Repository\n\t\t\tlistSnapshotsFunc = tc.listSnapshotsFunc\n\t\t\tsnapshots, err := findPreviousSnapshotManifest(t.Context(), repo, sourceInfo, snapshotTags, &noLaterThan, logrus.New())\n\n\t\t\t// Check if the returned error matches the expected error\n\t\t\tif tc.expectedError != nil {\n\t\t\t\trequire.ErrorContains(t, err, tc.expectedError.Error())\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\t// Check the number of returned snapshots\n\t\t\tif len(snapshots) != len(tc.expectedSnapshots) {\n\t\t\t\tt.Errorf(\"Expected %d snapshots, got %d\", len(tc.expectedSnapshots), len(snapshots))\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestBackup(t *testing.T) {\n\ttype testCase struct {\n\t\tname string\n\t\tsourcePath string\n\t\tforceFull bool\n\t\tparentSnapshot string\n\t\ttags map[string]string\n\t\tisEmptyUploader bool\n\t\tisSnapshotSourceError bool\n\t\texpectedError error\n\t\texpectedEmpty bool\n\t\tvolMode uploader.PersistentVolumeMode\n\t}\n\tmanifest := &snapshot.Manifest{\n\t\tID: \"test\",\n\t\tRootEntry: &snapshot.DirEntry{},\n\t}\n\t// Define test cases\n\ttestCases := []testCase{\n\t\t{\n\t\t\tname: \"Successful backup\",\n\t\t\tsourcePath: \"/\",\n\t\t\ttags: map[string]string{},\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"Empty fsUploader\",\n\t\t\tisEmptyUploader: true,\n\t\t\tsourcePath: \"/\",\n\t\t\ttags: nil,\n\t\t\texpectedError: errors.New(\"get empty kopia uploader\"),\n\t\t},\n\t\t{\n\t\t\tname: \"Unable to read directory\",\n\t\t\tsourcePath: \"/invalid/path\",\n\t\t\ttags: nil,\n\t\t\texpectedError: errors.New(\"no such file or directory\"),\n\t\t},\n\t\t{\n\t\t\tname: \"Source path is not a block device\",\n\t\t\tsourcePath: \"/\",\n\t\t\ttags: nil,\n\t\t\tvolMode: uploader.PersistentVolumeBlock,\n\t\t\texpectedError: errors.New(\"source path / is not a block device\"),\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tif tc.volMode == \"\" {\n\t\t\t\ttc.volMode = uploader.PersistentVolumeFilesystem\n\t\t\t}\n\t\t\ts := injectSnapshotFuncs()\n\t\t\targs := []mockArgs{\n\t\t\t\t{methodName: \"LoadSnapshot\", returns: []any{manifest, nil}},\n\t\t\t\t{methodName: \"SaveSnapshot\", returns: []any{manifest.ID, nil}},\n\t\t\t\t{methodName: \"TreeForSource\", returns: []any{nil, nil}},\n\t\t\t\t{methodName: \"ApplyRetentionPolicy\", returns: []any{nil, nil}},\n\t\t\t\t{methodName: \"SetPolicy\", returns: []any{nil}},\n\t\t\t\t{methodName: \"Upload\", returns: []any{manifest, nil}},\n\t\t\t\t{methodName: \"Flush\", returns: []any{nil}},\n\t\t\t}\n\t\t\tMockFuncs(s, args)\n\t\t\tif tc.isSnapshotSourceError {\n\t\t\t\ts.repoWriterMock.On(\"FindManifests\", mock.Anything, mock.Anything).Return(nil, errors.New(\"Failed to get manifests\"))\n\t\t\t\ts.repoWriterMock.On(\"Flush\", mock.Anything).Return(errors.New(\"Failed to get manifests\"))\n\t\t\t} else {\n\t\t\t\ts.repoWriterMock.On(\"FindManifests\", mock.Anything, mock.Anything).Return(nil, nil)\n\t\t\t}\n\n\t\t\tvar isSnapshotEmpty bool\n\t\t\tvar snapshotInfo *uploader.SnapshotInfo\n\t\t\tvar err error\n\t\t\tif tc.isEmptyUploader {\n\t\t\t\tsnapshotInfo, isSnapshotEmpty, err = Backup(t.Context(), nil, s.repoWriterMock, tc.sourcePath, \"\", tc.forceFull, tc.parentSnapshot, tc.volMode, map[string]string{}, tc.tags, &logrus.Logger{})\n\t\t\t} else {\n\t\t\t\tsnapshotInfo, isSnapshotEmpty, err = Backup(t.Context(), s.uploderMock, s.repoWriterMock, tc.sourcePath, \"\", tc.forceFull, tc.parentSnapshot, tc.volMode, map[string]string{}, tc.tags, &logrus.Logger{})\n\t\t\t}\n\t\t\t// Check if the returned error matches the expected error\n\t\t\tif tc.expectedError != nil {\n\t\t\t\trequire.ErrorContains(t, err, tc.expectedError.Error())\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tassert.Equal(t, tc.expectedEmpty, isSnapshotEmpty)\n\n\t\t\tif err == nil {\n\t\t\t\tassert.NotNil(t, snapshotInfo)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestRestore(t *testing.T) {\n\ttype testCase struct {\n\t\tname string\n\t\tsnapshotID string\n\t\tinvalidManifestType bool\n\t\tfilesystemEntryFunc func(ctx context.Context, rep repo.Repository, rootID string, consistentAttributes bool) (fs.Entry, error)\n\t\trestoreEntryFunc func(ctx context.Context, rep repo.Repository, output restore.Output, rootEntry fs.Entry, options restore.Options) (restore.Stats, error)\n\t\tflushVolumeFunc func(string) error\n\t\tdest string\n\t\texpectedBytes int64\n\t\texpectedCount int32\n\t\texpectedError error\n\t\tvolMode uploader.PersistentVolumeMode\n\t}\n\n\t// Define test cases\n\ttestCases := []testCase{\n\t\t{\n\t\t\tname: \"manifest is not a snapshot\",\n\t\t\tinvalidManifestType: true,\n\t\t\tdest: \"/path/to/destination\",\n\t\t\texpectedError: errors.New(\"Unable to load snapshot\"),\n\t\t},\n\t\t{\n\t\t\tname: \"Failed to get filesystem entry\",\n\t\t\tsnapshotID: \"snapshot-123\",\n\t\t\texpectedError: errors.New(\"Unable to get filesystem entry\"),\n\t\t},\n\t\t{\n\t\t\tname: \"Failed to restore with filesystem entry\",\n\t\t\tfilesystemEntryFunc: func(ctx context.Context, rep repo.Repository, rootID string, consistentAttributes bool) (fs.Entry, error) {\n\t\t\t\treturn snapshotfs.EntryFromDirEntry(rep, &snapshot.DirEntry{Type: snapshot.EntryTypeFile}), nil\n\t\t\t},\n\t\t\trestoreEntryFunc: func(ctx context.Context, rep repo.Repository, output restore.Output, rootEntry fs.Entry, options restore.Options) (restore.Stats, error) {\n\t\t\t\treturn restore.Stats{}, errors.New(\"Unable to get filesystem entry\")\n\t\t\t},\n\t\t\tsnapshotID: \"snapshot-123\",\n\t\t\texpectedError: errors.New(\"Unable to get filesystem entry\"),\n\t\t},\n\t\t{\n\t\t\tname: \"Expect successful\",\n\t\t\tfilesystemEntryFunc: func(ctx context.Context, rep repo.Repository, rootID string, consistentAttributes bool) (fs.Entry, error) {\n\t\t\t\treturn snapshotfs.EntryFromDirEntry(rep, &snapshot.DirEntry{Type: snapshot.EntryTypeFile}), nil\n\t\t\t},\n\t\t\trestoreEntryFunc: func(ctx context.Context, rep repo.Repository, output restore.Output, rootEntry fs.Entry, options restore.Options) (restore.Stats, error) {\n\t\t\t\treturn restore.Stats{}, nil\n\t\t\t},\n\t\t\tsnapshotID: \"snapshot-123\",\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"Expect block volume successful\",\n\t\t\tfilesystemEntryFunc: func(ctx context.Context, rep repo.Repository, rootID string, consistentAttributes bool) (fs.Entry, error) {\n\t\t\t\treturn snapshotfs.EntryFromDirEntry(rep, &snapshot.DirEntry{Type: snapshot.EntryTypeFile}), nil\n\t\t\t},\n\t\t\trestoreEntryFunc: func(ctx context.Context, rep repo.Repository, output restore.Output, rootEntry fs.Entry, options restore.Options) (restore.Stats, error) {\n\t\t\t\treturn restore.Stats{}, nil\n\t\t\t},\n\t\t\tsnapshotID: \"snapshot-123\",\n\t\t\texpectedError: nil,\n\t\t\tvolMode: uploader.PersistentVolumeBlock,\n\t\t},\n\t\t{\n\t\t\tname: \"Unable to evaluate symlinks for block volume\",\n\t\t\tfilesystemEntryFunc: func(ctx context.Context, rep repo.Repository, rootID string, consistentAttributes bool) (fs.Entry, error) {\n\t\t\t\treturn snapshotfs.EntryFromDirEntry(rep, &snapshot.DirEntry{Type: snapshot.EntryTypeFile}), nil\n\t\t\t},\n\t\t\trestoreEntryFunc: func(ctx context.Context, rep repo.Repository, output restore.Output, rootEntry fs.Entry, options restore.Options) (restore.Stats, error) {\n\t\t\t\terr := output.BeginDirectory(ctx, \"fake-dir\", virtualfs.NewStaticDirectory(\"fake-dir\", nil))\n\t\t\t\treturn restore.Stats{}, err\n\t\t\t},\n\t\t\tsnapshotID: \"snapshot-123\",\n\t\t\texpectedError: errors.New(\"unable to evaluate symlinks for\"),\n\t\t\tvolMode: uploader.PersistentVolumeBlock,\n\t\t\tdest: \"/wrong-dest\",\n\t\t},\n\t\t{\n\t\t\tname: \"Target file is not a block device\",\n\t\t\tfilesystemEntryFunc: func(ctx context.Context, rep repo.Repository, rootID string, consistentAttributes bool) (fs.Entry, error) {\n\t\t\t\treturn snapshotfs.EntryFromDirEntry(rep, &snapshot.DirEntry{Type: snapshot.EntryTypeFile}), nil\n\t\t\t},\n\t\t\trestoreEntryFunc: func(ctx context.Context, rep repo.Repository, output restore.Output, rootEntry fs.Entry, options restore.Options) (restore.Stats, error) {\n\t\t\t\terr := output.BeginDirectory(ctx, \"fake-dir\", virtualfs.NewStaticDirectory(\"fake-dir\", nil))\n\t\t\t\treturn restore.Stats{}, err\n\t\t\t},\n\t\t\tsnapshotID: \"snapshot-123\",\n\t\t\texpectedError: errors.New(\"target file /tmp is not a block device\"),\n\t\t\tvolMode: uploader.PersistentVolumeBlock,\n\t\t\tdest: \"/tmp\",\n\t\t},\n\t\t{\n\t\t\tname: \"Flush is not supported\",\n\t\t\tfilesystemEntryFunc: func(ctx context.Context, rep repo.Repository, rootID string, consistentAttributes bool) (fs.Entry, error) {\n\t\t\t\treturn snapshotfs.EntryFromDirEntry(rep, &snapshot.DirEntry{Type: snapshot.EntryTypeFile}), nil\n\t\t\t},\n\t\t\trestoreEntryFunc: func(ctx context.Context, rep repo.Repository, output restore.Output, rootEntry fs.Entry, options restore.Options) (restore.Stats, error) {\n\t\t\t\treturn restore.Stats{}, nil\n\t\t\t},\n\t\t\tflushVolumeFunc: func(string) error { return errFlushUnsupported },\n\t\t\tsnapshotID: \"snapshot-123\",\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"Flush fails\",\n\t\t\tfilesystemEntryFunc: func(ctx context.Context, rep repo.Repository, rootID string, consistentAttributes bool) (fs.Entry, error) {\n\t\t\t\treturn snapshotfs.EntryFromDirEntry(rep, &snapshot.DirEntry{Type: snapshot.EntryTypeFile}), nil\n\t\t\t},\n\t\t\trestoreEntryFunc: func(ctx context.Context, rep repo.Repository, output restore.Output, rootEntry fs.Entry, options restore.Options) (restore.Stats, error) {\n\t\t\t\treturn restore.Stats{}, nil\n\t\t\t},\n\t\t\tflushVolumeFunc: func(string) error { return errors.New(\"fake-flush-error\") },\n\t\t\tsnapshotID: \"snapshot-123\",\n\t\t\texpectedError: errors.New(\"fake-flush-error\"),\n\t\t},\n\t}\n\n\tem := &manifest.EntryMetadata{\n\t\tID: \"test\",\n\t\tLabels: map[string]string{},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tif tc.volMode == \"\" {\n\t\t\t\ttc.volMode = uploader.PersistentVolumeFilesystem\n\t\t\t}\n\n\t\t\tif tc.invalidManifestType {\n\t\t\t\tem.Labels[manifest.TypeLabelKey] = \"\"\n\t\t\t} else {\n\t\t\t\tem.Labels[manifest.TypeLabelKey] = snapshot.ManifestType\n\t\t\t}\n\n\t\t\tif tc.filesystemEntryFunc != nil {\n\t\t\t\tfilesystemEntryFunc = tc.filesystemEntryFunc\n\t\t\t}\n\n\t\t\tif tc.restoreEntryFunc != nil {\n\t\t\t\trestoreEntryFunc = tc.restoreEntryFunc\n\t\t\t}\n\n\t\t\tif tc.flushVolumeFunc != nil {\n\t\t\t\tflushVolumeFunc = tc.flushVolumeFunc\n\t\t\t}\n\n\t\t\trepoWriterMock := &repomocks.RepositoryWriter{}\n\t\t\trepoWriterMock.On(\"GetManifest\", mock.Anything, mock.Anything, mock.Anything).Return(em, nil)\n\t\t\trepoWriterMock.On(\"OpenObject\", mock.Anything, mock.Anything).Return(em, nil)\n\n\t\t\tprogress := new(Progress)\n\t\t\tbytesRestored, fileCount, err := Restore(t.Context(), repoWriterMock, progress, tc.snapshotID, tc.dest, tc.volMode, map[string]string{}, logrus.New(), nil)\n\n\t\t\t// Check if the returned error matches the expected error\n\t\t\tif tc.expectedError != nil {\n\t\t\t\trequire.ErrorContains(t, err, tc.expectedError.Error())\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\t// Check the number of bytes restored\n\t\t\tassert.Equal(t, tc.expectedBytes, bytesRestored)\n\n\t\t\t// Check the number of files restored\n\t\t\tassert.Equal(t, tc.expectedCount, fileCount)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/uploader/mocks/policy.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage mocks\n\nimport (\n\t\"context\"\n\n\t\"github.com/kopia/kopia/repo/manifest\"\n\t\"github.com/kopia/kopia/snapshot/policy\"\n\t\"github.com/stretchr/testify/mock\"\n\n\t\"github.com/kopia/kopia/repo\"\n\n\t\"github.com/kopia/kopia/snapshot\"\n)\n\n// policy is an autogenerated mock type for the TreeForSource type\ntype Policy struct {\n\tmock.Mock\n}\n\n// Execute provides a mock function with given fields: ctx, rep, si\nfunc (_m *Policy) TreeForSource(ctx context.Context, rep repo.Repository, si snapshot.SourceInfo) (*policy.Tree, error) {\n\tret := _m.Called(ctx, rep, si)\n\n\tvar r0 *policy.Tree\n\tif rf, ok := ret.Get(0).(func(context.Context, repo.Repository, snapshot.SourceInfo) *policy.Tree); ok {\n\t\tr0 = rf(ctx, rep, si)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(*policy.Tree)\n\t\t}\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(context.Context, repo.Repository, snapshot.SourceInfo) error); ok {\n\t\tr1 = rf(ctx, rep, si)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// ApplyRetentionPolicy provides a mock function with given fields: ctx, rep, sourceInfo, reallyDelete\nfunc (_m *Policy) ApplyRetentionPolicy(ctx context.Context, rep repo.RepositoryWriter, sourceInfo snapshot.SourceInfo, reallyDelete bool) ([]manifest.ID, error) {\n\tret := _m.Called(ctx, rep, sourceInfo, reallyDelete)\n\n\tvar r0 []manifest.ID\n\tif rf, ok := ret.Get(0).(func(context.Context, repo.RepositoryWriter, snapshot.SourceInfo, bool) []manifest.ID); ok {\n\t\tr0 = rf(ctx, rep, sourceInfo, reallyDelete)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).([]manifest.ID)\n\t\t}\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(context.Context, repo.RepositoryWriter, snapshot.SourceInfo, bool) error); ok {\n\t\tr1 = rf(ctx, rep, sourceInfo, reallyDelete)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\nfunc (_m *Policy) SetPolicy(ctx context.Context, rep repo.RepositoryWriter, si snapshot.SourceInfo, pol *policy.Policy) error {\n\tret := _m.Called(ctx, rep, si, pol)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, repo.RepositoryWriter, snapshot.SourceInfo, *policy.Policy) error); ok {\n\t\tr0 = rf(ctx, rep, si, pol)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n" }, { "path": "pkg/uploader/mocks/shim.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage mocks\n\nimport (\n\tcontext \"context\"\n\n\tmock \"github.com/stretchr/testify/mock\"\n)\n\n// shimRepository is an autogenerated mock type for the shimRepository type\ntype ShimRepository struct {\n\tmock.Mock\n}\n\n// Flush provides a mock function with given fields: ctx\nfunc (_m *ShimRepository) Flush(ctx context.Context) error {\n\tret := _m.Called(ctx)\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context) error); ok {\n\t\tr0 = rf(ctx)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n" }, { "path": "pkg/uploader/mocks/snapshot.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage mocks\n\nimport (\n\t\"context\"\n\n\t\"github.com/kopia/kopia/repo/manifest\"\n\t\"github.com/kopia/kopia/snapshot\"\n\t\"github.com/stretchr/testify/mock\"\n\n\t\"github.com/kopia/kopia/repo\"\n)\n\n// snapshot is an autogenerated mock type for the snapshot type\ntype Snapshot struct {\n\tmock.Mock\n}\n\n// LoadSnapshot provides a mock function with given fields: ctx, rep, manifestID\nfunc (_m *Snapshot) LoadSnapshot(ctx context.Context, rep repo.Repository, manifestID manifest.ID) (*snapshot.Manifest, error) {\n\tret := _m.Called(ctx, rep, manifestID)\n\n\tvar r0 *snapshot.Manifest\n\tif rf, ok := ret.Get(0).(func(context.Context, repo.Repository, manifest.ID) *snapshot.Manifest); ok {\n\t\tr0 = rf(ctx, rep, manifestID)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(*snapshot.Manifest)\n\t\t}\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(context.Context, repo.Repository, manifest.ID) error); ok {\n\t\tr1 = rf(ctx, rep, manifestID)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// SaveSnapshot provides a mock function with given fields: ctx, rep, man\nfunc (_m *Snapshot) SaveSnapshot(ctx context.Context, rep repo.RepositoryWriter, man *snapshot.Manifest) (manifest.ID, error) {\n\tret := _m.Called(ctx, rep, man)\n\n\tvar r0 manifest.ID\n\tif rf, ok := ret.Get(0).(func(context.Context, repo.RepositoryWriter, *snapshot.Manifest) manifest.ID); ok {\n\t\tr0 = rf(ctx, rep, man)\n\t} else {\n\t\tr0 = ret.Get(0).(manifest.ID)\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(context.Context, repo.RepositoryWriter, *snapshot.Manifest) error); ok {\n\t\tr1 = rf(ctx, rep, man)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n" }, { "path": "pkg/uploader/mocks/uploader.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage mocks\n\nimport (\n\t\"context\"\n\n\t\"github.com/kopia/kopia/fs\"\n\t\"github.com/stretchr/testify/mock\"\n\n\t\"github.com/kopia/kopia/snapshot/policy\"\n\n\t\"github.com/kopia/kopia/snapshot\"\n)\n\n// Upload is an autogenerated mock type for the Upload type\ntype Uploader struct {\n\tmock.Mock\n}\n\n// Execute provides a mock function with given fields: ctx, source, policyTree, sourceInfo, previousManifests\nfunc (_m *Uploader) Upload(ctx context.Context, source fs.Entry, policyTree *policy.Tree, sourceInfo snapshot.SourceInfo, previousManifests ...*snapshot.Manifest) (*snapshot.Manifest, error) {\n\t_va := make([]any, len(previousManifests))\n\tfor _i := range previousManifests {\n\t\t_va[_i] = previousManifests[_i]\n\t}\n\tvar _ca []any\n\t_ca = append(_ca, ctx, source, policyTree, sourceInfo)\n\t_ca = append(_ca, _va...)\n\tret := _m.Called(_ca...)\n\n\tvar r0 *snapshot.Manifest\n\tif rf, ok := ret.Get(0).(func(context.Context, fs.Entry, *policy.Tree, snapshot.SourceInfo, ...*snapshot.Manifest) *snapshot.Manifest); ok {\n\t\tr0 = rf(ctx, source, policyTree, sourceInfo, previousManifests...)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(*snapshot.Manifest)\n\t\t}\n\t}\n\n\tvar r1 error\n\tif rf, ok := ret.Get(1).(func(context.Context, fs.Entry, *policy.Tree, snapshot.SourceInfo, ...*snapshot.Manifest) error); ok {\n\t\tr1 = rf(ctx, source, policyTree, sourceInfo, previousManifests...)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n" }, { "path": "pkg/uploader/provider/kopia.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage provider\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strings\"\n\t\"sync/atomic\"\n\n\t\"github.com/kopia/kopia/snapshot/upload\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader/kopia\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\trepokeys \"github.com/vmware-tanzu/velero/pkg/repository/keys\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository/udmrepo/service\"\n)\n\n// BackupFunc mainly used to make testing more convenient\nvar BackupFunc = kopia.Backup\nvar RestoreFunc = kopia.Restore\nvar BackupRepoServiceCreateFunc = service.Create\n\n// kopiaProvider recorded info related with kopiaProvider\ntype kopiaProvider struct {\n\trequestorType string\n\tbkRepo udmrepo.BackupRepo\n\tcredGetter *credentials.CredentialGetter\n\tlog logrus.FieldLogger\n\tcanceling int32\n}\n\n// NewKopiaUploaderProvider initialized with open or create a repository\nfunc NewKopiaUploaderProvider(\n\trequestorType string,\n\tctx context.Context,\n\tcredGetter *credentials.CredentialGetter,\n\tbackupRepo *velerov1api.BackupRepository,\n\tlog logrus.FieldLogger,\n) (Provider, error) {\n\tkp := &kopiaProvider{\n\t\trequestorType: requestorType,\n\t\tlog: log,\n\t\tcredGetter: credGetter,\n\t}\n\t//repoUID which is used to generate kopia repository config with unique directory path\n\trepoUID := string(backupRepo.GetUID())\n\trepoOpt, err := udmrepo.NewRepoOptions(\n\t\tudmrepo.WithPassword(kp, \"\"),\n\t\tudmrepo.WithConfigFile(\"\", repoUID),\n\t\tudmrepo.WithDescription(\"Initial kopia uploader provider\"),\n\t)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error to get repo options\")\n\t}\n\n\trepoSvc := BackupRepoServiceCreateFunc(backupRepo.Spec.RepositoryType, log)\n\tlog.WithField(\"repoUID\", repoUID).Info(\"Opening backup repo\")\n\n\tkp.bkRepo, err = repoSvc.Open(ctx, *repoOpt)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"Failed to find kopia repository\")\n\t}\n\treturn kp, nil\n}\n\n// CheckContext check context status check if context is timeout or cancel and backup restore once finished it will quit and return\nfunc (kp *kopiaProvider) CheckContext(ctx context.Context, finishChan chan struct{}, restoreChan chan struct{}, uploader *upload.Uploader) {\n\tselect {\n\tcase <-finishChan:\n\t\tkp.log.Infof(\"Action finished\")\n\t\treturn\n\tcase <-ctx.Done():\n\t\tatomic.StoreInt32(&kp.canceling, 1)\n\n\t\tif uploader != nil {\n\t\t\tuploader.Cancel()\n\t\t\tkp.log.Infof(\"Backup is been canceled\")\n\t\t}\n\t\tif restoreChan != nil {\n\t\t\tclose(restoreChan)\n\t\t\tkp.log.Infof(\"Restore is been canceled\")\n\t\t}\n\t\treturn\n\t}\n}\n\nfunc (kp *kopiaProvider) Close(ctx context.Context) error {\n\treturn kp.bkRepo.Close(ctx)\n}\n\n// RunBackup which will backup specific path and update backup progress\n// return snapshotID, isEmptySnapshot, error\nfunc (kp *kopiaProvider) RunBackup(\n\tctx context.Context,\n\tpath string,\n\trealSource string,\n\ttags map[string]string,\n\tforceFull bool,\n\tparentSnapshot string,\n\tvolMode uploader.PersistentVolumeMode,\n\tuploaderCfg map[string]string,\n\tupdater uploader.ProgressUpdater) (string, bool, int64, int64, error) {\n\tif updater == nil {\n\t\treturn \"\", false, 0, 0, errors.New(\"Need to initial backup progress updater first\")\n\t}\n\n\tif path == \"\" {\n\t\treturn \"\", false, 0, 0, errors.New(\"path is empty\")\n\t}\n\n\tlog := kp.log.WithFields(logrus.Fields{\n\t\t\"path\": path,\n\t\t\"realSource\": realSource,\n\t\t\"parentSnapshot\": parentSnapshot,\n\t})\n\trepoWriter := kopia.NewShimRepo(kp.bkRepo)\n\tkpUploader := upload.NewUploader(repoWriter)\n\tprogress := kopia.NewProgress(updater, backupProgressCheckInterval, log)\n\tkpUploader.Progress = progress\n\tkpUploader.FailFast = true\n\tquit := make(chan struct{})\n\tlog.Info(\"Starting backup\")\n\tgo kp.CheckContext(ctx, quit, nil, kpUploader)\n\n\tdefer func() {\n\t\tclose(quit)\n\t}()\n\n\tif tags == nil {\n\t\ttags = make(map[string]string)\n\t}\n\ttags[uploader.SnapshotRequesterTag] = kp.requestorType\n\ttags[uploader.SnapshotUploaderTag] = uploader.KopiaType\n\n\tif realSource != \"\" {\n\t\trealSource = fmt.Sprintf(\"%s/%s/%s\", kp.requestorType, uploader.KopiaType, realSource)\n\t}\n\n\tif kp.bkRepo.GetAdvancedFeatures().MultiPartBackup {\n\t\tif uploaderCfg == nil {\n\t\t\tuploaderCfg = make(map[string]string)\n\t\t}\n\n\t\tuploaderCfg[kopia.UploaderConfigMultipartKey] = \"true\"\n\t}\n\n\tsnapshotInfo, _, err := BackupFunc(ctx, kpUploader, repoWriter, path, realSource, forceFull, parentSnapshot, volMode, uploaderCfg, tags, log)\n\tif err != nil {\n\t\tsnapshotID := \"\"\n\t\tif snapshotInfo != nil {\n\t\t\tsnapshotID = snapshotInfo.ID\n\t\t} else {\n\t\t\tlog.Infof(\"Kopia backup failed with %v and get empty snapshot ID\", err)\n\t\t}\n\n\t\tif kpUploader.IsCanceled() {\n\t\t\tlog.Warn(\"Kopia backup is canceled\")\n\t\t\treturn snapshotID, false, 0, 0, ErrorCanceled\n\t\t}\n\t\treturn snapshotID, false, 0, 0, errors.Wrapf(err, \"Failed to run kopia backup\")\n\t}\n\n\t// which ensure that the statistic data of TotalBytes equal to BytesDone when finished\n\tupdater.UpdateProgress(\n\t\t&uploader.Progress{\n\t\t\tTotalBytes: snapshotInfo.Size,\n\t\t\tBytesDone: snapshotInfo.Size,\n\t\t},\n\t)\n\n\tlog.Debugf(\"Kopia backup finished, snapshot ID %s, backup size %d\", snapshotInfo.ID, snapshotInfo.Size)\n\treturn snapshotInfo.ID, false, snapshotInfo.Size, progress.GetIncrementalSize(), nil\n}\n\nfunc (kp *kopiaProvider) GetPassword(param any) (string, error) {\n\tif kp.credGetter.FromSecret == nil {\n\t\treturn \"\", errors.New(\"invalid credentials interface\")\n\t}\n\trawPass, err := kp.credGetter.FromSecret.Get(repokeys.RepoKeySelector())\n\tif err != nil {\n\t\treturn \"\", errors.Wrap(err, \"error to get password\")\n\t}\n\n\treturn strings.TrimSpace(rawPass), nil\n}\n\n// RunRestore which will restore specific path and update restore progress\nfunc (kp *kopiaProvider) RunRestore(\n\tctx context.Context,\n\tsnapshotID string,\n\tvolumePath string,\n\tvolMode uploader.PersistentVolumeMode,\n\tuploaderCfg map[string]string,\n\tupdater uploader.ProgressUpdater) (int64, error) {\n\tlog := kp.log.WithFields(logrus.Fields{\n\t\t\"snapshotID\": snapshotID,\n\t\t\"volumePath\": volumePath,\n\t})\n\n\trepoWriter := kopia.NewShimRepo(kp.bkRepo)\n\tprogress := kopia.NewProgress(updater, restoreProgressCheckInterval, log)\n\trestoreCancel := make(chan struct{})\n\tquit := make(chan struct{})\n\n\tlog.Info(\"Starting restore\")\n\tdefer func() {\n\t\tclose(quit)\n\t}()\n\n\tgo kp.CheckContext(ctx, quit, restoreCancel, nil)\n\n\t// We use the cancel channel to control the restore cancel, so don't pass a context with cancel to Kopia restore.\n\t// Otherwise, Kopia restore will not response to the cancel control but return an arbitrary error.\n\t// Kopia restore cancel is not designed as well as Kopia backup which uses the context to control backup cancel all the way.\n\tsize, fileCount, err := RestoreFunc(context.Background(), repoWriter, progress, snapshotID, volumePath, volMode, uploaderCfg, log, restoreCancel)\n\n\tif err != nil {\n\t\treturn 0, errors.Wrapf(err, \"Failed to run kopia restore\")\n\t}\n\n\tif atomic.LoadInt32(&kp.canceling) == 1 {\n\t\tlog.Error(\"Kopia restore is canceled\")\n\t\treturn 0, ErrorCanceled\n\t}\n\n\t// which ensure that the statistic data of TotalBytes equal to BytesDone when finished\n\tupdater.UpdateProgress(&uploader.Progress{\n\t\tTotalBytes: size,\n\t\tBytesDone: size,\n\t})\n\n\toutput := fmt.Sprintf(\"Kopia restore finished, restore size %d, file count %d\", size, fileCount)\n\n\tlog.Info(output)\n\n\treturn size, nil\n}\n" }, { "path": "pkg/uploader/provider/kopia_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage provider\n\nimport (\n\t\"context\"\n\t\"sync\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/kopia/kopia/repo\"\n\t\"github.com/kopia/kopia/snapshot/upload\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\t\"github.com/vmware-tanzu/velero/internal/credentials/mocks\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/repository\"\n\tudmrepo \"github.com/vmware-tanzu/velero/pkg/repository/udmrepo\"\n\tudmrepomocks \"github.com/vmware-tanzu/velero/pkg/repository/udmrepo/mocks\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader/kopia\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n)\n\ntype FakeBackupProgressUpdater struct {\n\tPodVolumeBackup *velerov1api.PodVolumeBackup\n\tLog logrus.FieldLogger\n\tCtx context.Context\n\tCli client.Client\n}\n\nfunc (f *FakeBackupProgressUpdater) UpdateProgress(p *uploader.Progress) {}\n\ntype FakeRestoreProgressUpdater struct {\n\tPodVolumeRestore *velerov1api.PodVolumeRestore\n\tLog logrus.FieldLogger\n\tCtx context.Context\n\tCli client.Client\n}\n\nfunc (f *FakeRestoreProgressUpdater) UpdateProgress(p *uploader.Progress) {}\n\nfunc TestRunBackup(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\thookBackupFunc func(ctx context.Context, fsUploader kopia.SnapshotUploader, repoWriter repo.RepositoryWriter, sourcePath string, realSource string, forceFull bool, parentSnapshot string, volMode uploader.PersistentVolumeMode, uploaderCfg map[string]string, tags map[string]string, log logrus.FieldLogger) (*uploader.SnapshotInfo, bool, error)\n\t\tvolMode uploader.PersistentVolumeMode\n\t\tnotError bool\n\t}{\n\t\t{\n\t\t\tname: \"success to backup\",\n\t\t\thookBackupFunc: func(ctx context.Context, fsUploader kopia.SnapshotUploader, repoWriter repo.RepositoryWriter, sourcePath string, realSource string, forceFull bool, parentSnapshot string, volMode uploader.PersistentVolumeMode, uploaderCfg map[string]string, tags map[string]string, log logrus.FieldLogger) (*uploader.SnapshotInfo, bool, error) {\n\t\t\t\treturn &uploader.SnapshotInfo{}, false, nil\n\t\t\t},\n\t\t\tnotError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"get error to backup\",\n\t\t\thookBackupFunc: func(ctx context.Context, fsUploader kopia.SnapshotUploader, repoWriter repo.RepositoryWriter, sourcePath string, realSource string, forceFull bool, parentSnapshot string, volMode uploader.PersistentVolumeMode, uploaderCfg map[string]string, tags map[string]string, log logrus.FieldLogger) (*uploader.SnapshotInfo, bool, error) {\n\t\t\t\treturn &uploader.SnapshotInfo{}, false, errors.New(\"failed to backup\")\n\t\t\t},\n\t\t\tnotError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"success to backup block mode volume\",\n\t\t\thookBackupFunc: func(ctx context.Context, fsUploader kopia.SnapshotUploader, repoWriter repo.RepositoryWriter, sourcePath string, realSource string, forceFull bool, parentSnapshot string, volMode uploader.PersistentVolumeMode, uploaderCfg map[string]string, tags map[string]string, log logrus.FieldLogger) (*uploader.SnapshotInfo, bool, error) {\n\t\t\t\treturn &uploader.SnapshotInfo{}, false, nil\n\t\t\t},\n\t\t\tvolMode: uploader.PersistentVolumeBlock,\n\t\t\tnotError: true,\n\t\t},\n\t}\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tmockBRepo := udmrepomocks.NewBackupRepo(t)\n\t\t\tmockBRepo.On(\"GetAdvancedFeatures\").Return(udmrepo.AdvancedFeatureInfo{})\n\n\t\t\tvar kp kopiaProvider\n\t\t\tkp.log = logrus.New()\n\t\t\tkp.bkRepo = mockBRepo\n\t\t\tupdater := FakeBackupProgressUpdater{PodVolumeBackup: &velerov1api.PodVolumeBackup{}, Log: kp.log, Ctx: t.Context(), Cli: fake.NewClientBuilder().WithScheme(util.VeleroScheme).Build()}\n\n\t\t\tif tc.volMode == \"\" {\n\t\t\t\ttc.volMode = uploader.PersistentVolumeFilesystem\n\t\t\t}\n\t\t\tBackupFunc = tc.hookBackupFunc\n\t\t\t_, _, _, _, err := kp.RunBackup(t.Context(), \"var\", \"\", nil, false, \"\", tc.volMode, map[string]string{}, &updater)\n\t\t\tif tc.notError {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.Error(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestRunRestore(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\thookRestoreFunc func(ctx context.Context, rep repo.RepositoryWriter, progress *kopia.Progress, snapshotID, dest string, volMode uploader.PersistentVolumeMode, uploaderCfg map[string]string, log logrus.FieldLogger, cancleCh chan struct{}) (int64, int32, error)\n\t\tnotError bool\n\t\tvolMode uploader.PersistentVolumeMode\n\t}{\n\t\t{\n\t\t\tname: \"normal restore\",\n\t\t\thookRestoreFunc: func(ctx context.Context, rep repo.RepositoryWriter, progress *kopia.Progress, snapshotID, dest string, volMode uploader.PersistentVolumeMode, uploaderCfg map[string]string, log logrus.FieldLogger, cancleCh chan struct{}) (int64, int32, error) {\n\t\t\t\treturn 0, 0, nil\n\t\t\t},\n\t\t\tnotError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"normal block mode restore\",\n\t\t\thookRestoreFunc: func(ctx context.Context, rep repo.RepositoryWriter, progress *kopia.Progress, snapshotID, dest string, volMode uploader.PersistentVolumeMode, uploaderCfg map[string]string, log logrus.FieldLogger, cancleCh chan struct{}) (int64, int32, error) {\n\t\t\t\treturn 0, 0, nil\n\t\t\t},\n\t\t\tvolMode: uploader.PersistentVolumeBlock,\n\t\t\tnotError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"failed to restore\",\n\t\t\thookRestoreFunc: func(ctx context.Context, rep repo.RepositoryWriter, progress *kopia.Progress, snapshotID, dest string, volMode uploader.PersistentVolumeMode, uploaderCfg map[string]string, log logrus.FieldLogger, cancleCh chan struct{}) (int64, int32, error) {\n\t\t\t\treturn 0, 0, errors.New(\"failed to restore\")\n\t\t\t},\n\t\t\tnotError: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar kp kopiaProvider\n\t\t\tkp.log = logrus.New()\n\t\t\tupdater := FakeRestoreProgressUpdater{PodVolumeRestore: &velerov1api.PodVolumeRestore{}, Log: kp.log, Ctx: t.Context(), Cli: fake.NewClientBuilder().WithScheme(util.VeleroScheme).Build()}\n\n\t\t\tif tc.volMode == \"\" {\n\t\t\t\ttc.volMode = uploader.PersistentVolumeFilesystem\n\t\t\t}\n\t\t\tRestoreFunc = tc.hookRestoreFunc\n\t\t\t_, err := kp.RunRestore(t.Context(), \"\", \"/var\", tc.volMode, map[string]string{}, &updater)\n\t\t\tif tc.notError {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\tassert.Error(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestCheckContext(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tfinishChan chan struct{}\n\t\trestoreChan chan struct{}\n\t\tuploader *upload.Uploader\n\t\texpectCancel bool\n\t\texpectBackup bool\n\t\texpectRestore bool\n\t}{\n\t\t{\n\t\t\tname: \"FinishChan\",\n\t\t\tfinishChan: make(chan struct{}),\n\t\t\trestoreChan: make(chan struct{}),\n\t\t\tuploader: &upload.Uploader{},\n\t\t\texpectCancel: false,\n\t\t\texpectBackup: false,\n\t\t\texpectRestore: false,\n\t\t},\n\t\t{\n\t\t\tname: \"nil uploader\",\n\t\t\tfinishChan: make(chan struct{}),\n\t\t\trestoreChan: make(chan struct{}),\n\t\t\tuploader: nil,\n\t\t\texpectCancel: true,\n\t\t\texpectBackup: false,\n\t\t\texpectRestore: true,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar wg sync.WaitGroup\n\t\t\twg.Add(1)\n\n\t\t\tif tc.expectBackup {\n\t\t\t\tgo func() {\n\t\t\t\t\twg.Wait()\n\t\t\t\t\ttc.restoreChan <- struct{}{}\n\t\t\t\t}()\n\t\t\t}\n\n\t\t\tctx, cancel := context.WithCancel(t.Context())\n\t\t\tdefer cancel()\n\n\t\t\tgo func() {\n\t\t\t\ttime.Sleep(100 * time.Millisecond)\n\t\t\t\tcancel()\n\t\t\t\twg.Done()\n\t\t\t}()\n\n\t\t\tkp := &kopiaProvider{log: logrus.New()}\n\t\t\tkp.CheckContext(ctx, tc.finishChan, tc.restoreChan, tc.uploader)\n\n\t\t\tif tc.expectCancel && tc.uploader != nil {\n\t\t\t\tt.Error(\"Expected the uploader to be canceled\")\n\t\t\t}\n\n\t\t\tif tc.expectBackup && tc.uploader == nil && len(tc.restoreChan) > 0 {\n\t\t\t\tt.Error(\"Expected the restore channel to be closed\")\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetPassword(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tempytSecret bool\n\t\tcredGetterFunc func(*mocks.SecretStore, *corev1api.SecretKeySelector)\n\t\texpectError bool\n\t\texpectedPass string\n\t}{\n\t\t{\n\t\t\tname: \"valid credentials interface\",\n\t\t\tcredGetterFunc: func(ss *mocks.SecretStore, selector *corev1api.SecretKeySelector) {\n\t\t\t\tss.On(\"Get\", selector).Return(\"test\", nil)\n\t\t\t},\n\t\t\texpectError: false,\n\t\t\texpectedPass: \"test\",\n\t\t},\n\t\t{\n\t\t\tname: \"empty from secret\",\n\t\t\tempytSecret: true,\n\t\t\texpectError: true,\n\t\t\texpectedPass: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"ErrorGettingPassword\",\n\t\t\tcredGetterFunc: func(ss *mocks.SecretStore, selector *corev1api.SecretKeySelector) {\n\t\t\t\tss.On(\"Get\", selector).Return(\"\", errors.New(\"error getting password\"))\n\t\t\t},\n\t\t\texpectError: true,\n\t\t\texpectedPass: \"\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\t// Mock CredentialGetter\n\t\t\tcredGetter := &credentials.CredentialGetter{}\n\t\t\tmockCredGetter := &mocks.SecretStore{}\n\t\t\tif !tc.empytSecret {\n\t\t\t\tcredGetter.FromSecret = mockCredGetter\n\t\t\t}\n\t\t\trepoKeySelector := &corev1api.SecretKeySelector{LocalObjectReference: corev1api.LocalObjectReference{Name: \"velero-repo-credentials\"}, Key: \"repository-password\"}\n\n\t\t\tif tc.credGetterFunc != nil {\n\t\t\t\ttc.credGetterFunc(mockCredGetter, repoKeySelector)\n\t\t\t}\n\n\t\t\tkp := &kopiaProvider{\n\t\t\t\tcredGetter: credGetter,\n\t\t\t}\n\n\t\t\tpassword, err := kp.GetPassword(nil)\n\t\t\tif tc.expectError {\n\t\t\t\trequire.Error(t, err, \"Expected an error\")\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err, \"Expected no error\")\n\t\t\t}\n\n\t\t\tassert.Equal(t, tc.expectedPass, password, \"Expected password to match\")\n\t\t})\n\t}\n}\n\nfunc (m *MockCredentialGetter) GetCredentials() (string, error) {\n\targs := m.Called()\n\treturn args.String(0), args.Error(1)\n}\n\n// MockRepoSvc is a mock implementation of the RepoService interface.\ntype MockRepoSvc struct {\n\tmock.Mock\n}\n\nfunc (m *MockRepoSvc) Open(ctx context.Context, opts udmrepo.RepoOptions) (udmrepo.BackupRepo, error) {\n\targs := m.Called(ctx, opts)\n\treturn args.Get(0).(udmrepo.BackupRepo), args.Error(1)\n}\n\nfunc TestNewKopiaUploaderProvider(t *testing.T) {\n\trequestorType := \"testRequestor\"\n\tctx := t.Context()\n\tbackupRepo := repository.NewBackupRepository(velerov1api.DefaultNamespace, repository.BackupRepositoryKey{VolumeNamespace: \"fake-volume-ns-02\", BackupLocation: \"fake-bsl-02\", RepositoryType: \"fake-repository-type-02\"})\n\tmockLog := logrus.New()\n\n\t// Define test cases\n\ttestCases := []struct {\n\t\tname string\n\t\tmockCredGetter *mocks.SecretStore\n\t\tmockBackupRepoService udmrepo.BackupRepoService\n\t\texpectedError string\n\t}{\n\t\t{\n\t\t\tname: \"Success\",\n\t\t\tmockCredGetter: func() *mocks.SecretStore {\n\t\t\t\tmockCredGetter := &mocks.SecretStore{}\n\t\t\t\tmockCredGetter.On(\"Get\", mock.Anything).Return(\"test\", nil)\n\t\t\t\treturn mockCredGetter\n\t\t\t}(),\n\t\t\tmockBackupRepoService: func() udmrepo.BackupRepoService {\n\t\t\t\tbackupRepoService := &udmrepomocks.BackupRepoService{}\n\t\t\t\tvar backupRepo udmrepo.BackupRepo\n\t\t\t\tbackupRepoService.On(\"Open\", t.Context(), mock.Anything).Return(backupRepo, nil)\n\t\t\t\treturn backupRepoService\n\t\t\t}(),\n\t\t\texpectedError: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"Error to get repo options\",\n\t\t\tmockCredGetter: func() *mocks.SecretStore {\n\t\t\t\tmockCredGetter := &mocks.SecretStore{}\n\t\t\t\tmockCredGetter.On(\"Get\", mock.Anything).Return(\"test\", errors.New(\"failed to get password\"))\n\t\t\t\treturn mockCredGetter\n\t\t\t}(),\n\t\t\tmockBackupRepoService: func() udmrepo.BackupRepoService {\n\t\t\t\tbackupRepoService := &udmrepomocks.BackupRepoService{}\n\t\t\t\tvar backupRepo udmrepo.BackupRepo\n\t\t\t\tbackupRepoService.On(\"Open\", t.Context(), mock.Anything).Return(backupRepo, nil)\n\t\t\t\treturn backupRepoService\n\t\t\t}(),\n\t\t\texpectedError: \"error to get repo options\",\n\t\t},\n\t\t{\n\t\t\tname: \"Error open repository service\",\n\t\t\tmockCredGetter: func() *mocks.SecretStore {\n\t\t\t\tmockCredGetter := &mocks.SecretStore{}\n\t\t\t\tmockCredGetter.On(\"Get\", mock.Anything).Return(\"test\", nil)\n\t\t\t\treturn mockCredGetter\n\t\t\t}(),\n\t\t\tmockBackupRepoService: func() udmrepo.BackupRepoService {\n\t\t\t\tbackupRepoService := &udmrepomocks.BackupRepoService{}\n\t\t\t\tvar backupRepo udmrepo.BackupRepo\n\t\t\t\tbackupRepoService.On(\"Open\", t.Context(), mock.Anything).Return(backupRepo, errors.New(\"failed to init repository\"))\n\t\t\t\treturn backupRepoService\n\t\t\t}(),\n\t\t\texpectedError: \"Failed to find kopia repository\",\n\t\t},\n\t}\n\n\t// Iterate through test cases\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tcredGetter := &credentials.CredentialGetter{FromSecret: tc.mockCredGetter}\n\t\t\tBackupRepoServiceCreateFunc = func(string, logrus.FieldLogger) udmrepo.BackupRepoService {\n\t\t\t\treturn tc.mockBackupRepoService\n\t\t\t}\n\t\t\t// Call the function being tested.\n\t\t\t_, err := NewKopiaUploaderProvider(requestorType, ctx, credGetter, backupRepo, mockLog)\n\n\t\t\t// Assertions\n\t\t\tif tc.expectedError != \"\" {\n\t\t\t\trequire.ErrorContains(t, err, tc.expectedError)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\t// Verify that the expected methods were called on the mocks.\n\t\t\ttc.mockCredGetter.AssertExpectations(t)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/uploader/provider/mocks/Provider.go", "content": "// Code generated by mockery v2.53.5. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tcontext \"context\"\n\n\tmock \"github.com/stretchr/testify/mock\"\n\n\tuploader \"github.com/vmware-tanzu/velero/pkg/uploader\"\n)\n\n// Provider is an autogenerated mock type for the Provider type\ntype Provider struct {\n\tmock.Mock\n}\n\n// Close provides a mock function with given fields: ctx\nfunc (_m *Provider) Close(ctx context.Context) error {\n\tret := _m.Called(ctx)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Close\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context) error); ok {\n\t\tr0 = rf(ctx)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// RunBackup provides a mock function with given fields: ctx, path, realSource, tags, forceFull, parentSnapshot, volMode, uploaderCfg, updater\nfunc (_m *Provider) RunBackup(ctx context.Context, path string, realSource string, tags map[string]string, forceFull bool, parentSnapshot string, volMode uploader.PersistentVolumeMode, uploaderCfg map[string]string, updater uploader.ProgressUpdater) (string, bool, int64, int64, error) {\n\tret := _m.Called(ctx, path, realSource, tags, forceFull, parentSnapshot, volMode, uploaderCfg, updater)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for RunBackup\")\n\t}\n\n\tvar r0 string\n\tvar r1 bool\n\tvar r2 int64\n\tvar r3 int64\n\tvar r4 error\n\tif rf, ok := ret.Get(0).(func(context.Context, string, string, map[string]string, bool, string, uploader.PersistentVolumeMode, map[string]string, uploader.ProgressUpdater) (string, bool, int64, int64, error)); ok {\n\t\treturn rf(ctx, path, realSource, tags, forceFull, parentSnapshot, volMode, uploaderCfg, updater)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, string, string, map[string]string, bool, string, uploader.PersistentVolumeMode, map[string]string, uploader.ProgressUpdater) string); ok {\n\t\tr0 = rf(ctx, path, realSource, tags, forceFull, parentSnapshot, volMode, uploaderCfg, updater)\n\t} else {\n\t\tr0 = ret.Get(0).(string)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, string, string, map[string]string, bool, string, uploader.PersistentVolumeMode, map[string]string, uploader.ProgressUpdater) bool); ok {\n\t\tr1 = rf(ctx, path, realSource, tags, forceFull, parentSnapshot, volMode, uploaderCfg, updater)\n\t} else {\n\t\tr1 = ret.Get(1).(bool)\n\t}\n\n\tif rf, ok := ret.Get(2).(func(context.Context, string, string, map[string]string, bool, string, uploader.PersistentVolumeMode, map[string]string, uploader.ProgressUpdater) int64); ok {\n\t\tr2 = rf(ctx, path, realSource, tags, forceFull, parentSnapshot, volMode, uploaderCfg, updater)\n\t} else {\n\t\tr2 = ret.Get(2).(int64)\n\t}\n\n\tif rf, ok := ret.Get(3).(func(context.Context, string, string, map[string]string, bool, string, uploader.PersistentVolumeMode, map[string]string, uploader.ProgressUpdater) int64); ok {\n\t\tr3 = rf(ctx, path, realSource, tags, forceFull, parentSnapshot, volMode, uploaderCfg, updater)\n\t} else {\n\t\tr3 = ret.Get(3).(int64)\n\t}\n\n\tif rf, ok := ret.Get(4).(func(context.Context, string, string, map[string]string, bool, string, uploader.PersistentVolumeMode, map[string]string, uploader.ProgressUpdater) error); ok {\n\t\tr4 = rf(ctx, path, realSource, tags, forceFull, parentSnapshot, volMode, uploaderCfg, updater)\n\t} else {\n\t\tr4 = ret.Error(4)\n\t}\n\n\treturn r0, r1, r2, r3, r4\n}\n\n// RunRestore provides a mock function with given fields: ctx, snapshotID, volumePath, volMode, uploaderConfig, updater\nfunc (_m *Provider) RunRestore(ctx context.Context, snapshotID string, volumePath string, volMode uploader.PersistentVolumeMode, uploaderConfig map[string]string, updater uploader.ProgressUpdater) (int64, error) {\n\tret := _m.Called(ctx, snapshotID, volumePath, volMode, uploaderConfig, updater)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for RunRestore\")\n\t}\n\n\tvar r0 int64\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(context.Context, string, string, uploader.PersistentVolumeMode, map[string]string, uploader.ProgressUpdater) (int64, error)); ok {\n\t\treturn rf(ctx, snapshotID, volumePath, volMode, uploaderConfig, updater)\n\t}\n\tif rf, ok := ret.Get(0).(func(context.Context, string, string, uploader.PersistentVolumeMode, map[string]string, uploader.ProgressUpdater) int64); ok {\n\t\tr0 = rf(ctx, snapshotID, volumePath, volMode, uploaderConfig, updater)\n\t} else {\n\t\tr0 = ret.Get(0).(int64)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(context.Context, string, string, uploader.PersistentVolumeMode, map[string]string, uploader.ProgressUpdater) error); ok {\n\t\tr1 = rf(ctx, snapshotID, volumePath, volMode, uploaderConfig, updater)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// NewProvider creates a new instance of Provider. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\n// The first argument is typically a *testing.T value.\nfunc NewProvider(t interface {\n\tmock.TestingT\n\tCleanup(func())\n}) *Provider {\n\tmock := &Provider{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/uploader/provider/provider.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage provider\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n)\n\nconst restoreProgressCheckInterval = 10 * time.Second\nconst backupProgressCheckInterval = 10 * time.Second\n\nvar ErrorCanceled error = errors.New(\"uploader is canceled\")\n\n// Provider which is designed for one pod volume to do the backup or restore\ntype Provider interface {\n\t// RunBackup which will do backup for one specific volume and return snapshotID, isSnapshotEmpty, error\n\t// updater is used for updating backup progress which implement by third-party\n\tRunBackup(\n\t\tctx context.Context,\n\t\tpath string,\n\t\trealSource string,\n\t\ttags map[string]string,\n\t\tforceFull bool,\n\t\tparentSnapshot string,\n\t\tvolMode uploader.PersistentVolumeMode,\n\t\tuploaderCfg map[string]string,\n\t\tupdater uploader.ProgressUpdater) (string, bool, int64, int64, error)\n\t// RunRestore which will do restore for one specific volume with given snapshot id and return error\n\t// updater is used for updating backup progress which implement by third-party\n\tRunRestore(\n\t\tctx context.Context,\n\t\tsnapshotID string,\n\t\tvolumePath string,\n\t\tvolMode uploader.PersistentVolumeMode,\n\t\tuploaderConfig map[string]string,\n\t\tupdater uploader.ProgressUpdater) (int64, error)\n\t// Close which will close related repository\n\tClose(ctx context.Context) error\n}\n\n// NewUploaderProvider initialize provider with specific uploaderType\nfunc NewUploaderProvider(\n\tctx context.Context,\n\tclient client.Client,\n\tuploaderType string,\n\trequesterType string,\n\trepoIdentifier string,\n\tbsl *velerov1api.BackupStorageLocation,\n\tbackupRepo *velerov1api.BackupRepository,\n\tcredGetter *credentials.CredentialGetter,\n\trepoKeySelector *corev1api.SecretKeySelector,\n\tlog logrus.FieldLogger,\n) (Provider, error) {\n\tif requesterType == \"\" {\n\t\treturn nil, errors.New(\"requester type is empty\")\n\t}\n\n\tif credGetter.FromFile == nil {\n\t\treturn nil, errors.New(\"uninitialized FileStore credential is not supported\")\n\t}\n\tif uploaderType == uploader.KopiaType {\n\t\treturn NewKopiaUploaderProvider(requesterType, ctx, credGetter, backupRepo, log)\n\t} else {\n\t\treturn NewResticUploaderProvider(repoIdentifier, bsl, credGetter, repoKeySelector, log)\n\t}\n}\n" }, { "path": "pkg/uploader/provider/provider_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage provider\n\nimport (\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\t\"github.com/vmware-tanzu/velero/internal/credentials/mocks\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n)\n\ntype NewUploaderProviderTestCase struct {\n\tDescription string\n\tUploaderType string\n\tRequestorType string\n\tExpectedError string\n\tneedFromFile bool\n}\n\nfunc TestNewUploaderProvider(t *testing.T) {\n\t// Mock objects or dependencies\n\tctx := t.Context()\n\tclient := fake.NewClientBuilder().WithScheme(util.VeleroScheme).Build()\n\trepoIdentifier := \"repoIdentifier\"\n\tbsl := &velerov1api.BackupStorageLocation{}\n\tbackupRepo := &velerov1api.BackupRepository{}\n\tcredGetter := &credentials.CredentialGetter{}\n\trepoKeySelector := &corev1api.SecretKeySelector{}\n\tlog := logrus.New()\n\n\ttestCases := []NewUploaderProviderTestCase{\n\t\t{\n\t\t\tDescription: \"When requestorType is empty, it should return an error\",\n\t\t\tUploaderType: \"kopia\",\n\t\t\tRequestorType: \"\",\n\t\t\tExpectedError: \"requester type is empty\",\n\t\t},\n\t\t{\n\t\t\tDescription: \"When FileStore credential is uninitialized, it should return an error\",\n\t\t\tUploaderType: \"kopia\",\n\t\t\tRequestorType: \"requester\",\n\t\t\tExpectedError: \"uninitialized FileStore credential\",\n\t\t},\n\t\t{\n\t\t\tDescription: \"When uploaderType is kopia, it should return a KopiaUploaderProvider\",\n\t\t\tUploaderType: \"kopia\",\n\t\t\tRequestorType: \"requester\",\n\t\t\tneedFromFile: true,\n\t\t\tExpectedError: \"invalid credentials interface\",\n\t\t},\n\t\t{\n\t\t\tDescription: \"When uploaderType is not kopia, it should return a ResticUploaderProvider\",\n\t\t\tUploaderType: \"restic\",\n\t\t\tRequestorType: \"requester\",\n\t\t\tneedFromFile: true,\n\t\t\tExpectedError: \"\",\n\t\t},\n\t}\n\n\tfor _, testCase := range testCases {\n\t\tt.Run(testCase.Description, func(t *testing.T) {\n\t\t\tif testCase.needFromFile {\n\t\t\t\tmockFileGetter := &mocks.FileStore{}\n\t\t\t\tmockFileGetter.On(\"Path\", &corev1api.SecretKeySelector{}).Return(\"\", nil)\n\t\t\t\tcredGetter.FromFile = mockFileGetter\n\t\t\t}\n\t\t\t_, err := NewUploaderProvider(ctx, client, testCase.UploaderType, testCase.RequestorType, repoIdentifier, bsl, backupRepo, credGetter, repoKeySelector, log)\n\t\t\tif testCase.ExpectedError == \"\" {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t} else {\n\t\t\t\trequire.ErrorContains(t, err, testCase.ExpectedError)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/uploader/provider/restic.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage provider\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"os\"\n\t\"strings\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/restic\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\tuploaderutil \"github.com/vmware-tanzu/velero/pkg/uploader/util\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n)\n\n// resticBackupCMDFunc and resticRestoreCMDFunc are mainly used to make testing more convenient\nvar resticBackupCMDFunc = restic.BackupCommand\nvar resticBackupFunc = restic.RunBackup\nvar resticGetSnapshotFunc = restic.GetSnapshotCommand\nvar resticGetSnapshotIDFunc = restic.GetSnapshotID\nvar resticRestoreCMDFunc = restic.RestoreCommand\nvar resticTempCACertFileFunc = restic.TempCACertFile\nvar resticCmdEnvFunc = restic.CmdEnv\n\ntype resticProvider struct {\n\trepoIdentifier string\n\tcredentialsFile string\n\tcaCertFile string\n\tcmdEnv []string\n\textraFlags []string\n\tbsl *velerov1api.BackupStorageLocation\n\tlog logrus.FieldLogger\n}\n\nfunc NewResticUploaderProvider(\n\trepoIdentifier string,\n\tbsl *velerov1api.BackupStorageLocation,\n\tcredGetter *credentials.CredentialGetter,\n\trepoKeySelector *corev1api.SecretKeySelector,\n\tlog logrus.FieldLogger,\n) (Provider, error) {\n\tprovider := resticProvider{\n\t\trepoIdentifier: repoIdentifier,\n\t\tbsl: bsl,\n\t\tlog: log,\n\t}\n\n\tvar err error\n\tprovider.credentialsFile, err = credGetter.FromFile.Path(repoKeySelector)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error creating temp restic credentials file\")\n\t}\n\n\t// if there's a caCert on the ObjectStorage, write it to disk so that it can be passed to restic\n\tif bsl.Spec.ObjectStorage != nil {\n\t\tvar caCertData []byte\n\n\t\t// Try CACertRef first (new method), then fall back to CACert (deprecated)\n\t\tif bsl.Spec.ObjectStorage.CACertRef != nil {\n\t\t\tcaCertString, err := credGetter.FromSecret.Get(bsl.Spec.ObjectStorage.CACertRef)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, errors.Wrap(err, \"error getting CA certificate from secret\")\n\t\t\t}\n\t\t\tcaCertData = []byte(caCertString)\n\t\t} else if bsl.Spec.ObjectStorage.CACert != nil {\n\t\t\tcaCertData = bsl.Spec.ObjectStorage.CACert\n\t\t}\n\n\t\tif caCertData != nil {\n\t\t\tprovider.caCertFile, err = resticTempCACertFileFunc(caCertData, bsl.Name, filesystem.NewFileSystem())\n\t\t\tif err != nil {\n\t\t\t\treturn nil, errors.Wrap(err, \"error create temp cert file\")\n\t\t\t}\n\t\t}\n\t}\n\n\tprovider.cmdEnv, err = resticCmdEnvFunc(bsl, credGetter.FromFile)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error generating repository cmnd env\")\n\t}\n\n\t// #4820: restrieve insecureSkipTLSVerify from BSL configuration for\n\t// AWS plugin. If nothing is return, that means insecureSkipTLSVerify\n\t// is not enable for Restic command.\n\tskipTLSRet := restic.GetInsecureSkipTLSVerifyFromBSL(bsl, log)\n\tif len(skipTLSRet) > 0 {\n\t\tprovider.extraFlags = append(provider.extraFlags, skipTLSRet)\n\t}\n\n\treturn &provider, nil\n}\n\nfunc (rp *resticProvider) Close(ctx context.Context) error {\n\t_, err := os.Stat(rp.credentialsFile)\n\tif err == nil {\n\t\treturn os.Remove(rp.credentialsFile)\n\t} else if !os.IsNotExist(err) {\n\t\treturn errors.Errorf(\"failed to get file %s info with error %v\", rp.credentialsFile, err)\n\t}\n\n\t_, err = os.Stat(rp.caCertFile)\n\tif err == nil {\n\t\treturn os.Remove(rp.caCertFile)\n\t} else if !os.IsNotExist(err) {\n\t\treturn errors.Errorf(\"failed to get file %s info with error %v\", rp.caCertFile, err)\n\t}\n\treturn nil\n}\n\n// RunBackup runs a `backup` command and watches the output to provide\n// progress updates to the caller and return snapshotID, isEmptySnapshot, error\nfunc (rp *resticProvider) RunBackup(\n\tctx context.Context,\n\tpath string,\n\trealSource string,\n\ttags map[string]string,\n\tforceFull bool,\n\tparentSnapshot string,\n\tvolMode uploader.PersistentVolumeMode,\n\tuploaderCfg map[string]string,\n\tupdater uploader.ProgressUpdater) (string, bool, int64, int64, error) {\n\tif updater == nil {\n\t\treturn \"\", false, 0, 0, errors.New(\"Need to initial backup progress updater first\")\n\t}\n\n\tif path == \"\" {\n\t\treturn \"\", false, 0, 0, errors.New(\"path is empty\")\n\t}\n\n\tif realSource != \"\" {\n\t\treturn \"\", false, 0, 0, errors.New(\"real source is not empty, this is not supported by restic uploader\")\n\t}\n\n\tif volMode == uploader.PersistentVolumeBlock {\n\t\treturn \"\", false, 0, 0, errors.New(\"unable to support block mode\")\n\t}\n\n\tlog := rp.log.WithFields(logrus.Fields{\n\t\t\"path\": path,\n\t\t\"parentSnapshot\": parentSnapshot,\n\t})\n\n\tif len(uploaderCfg) > 0 {\n\t\tparallelFilesUpload, err := uploaderutil.GetParallelFilesUpload(uploaderCfg)\n\t\tif err != nil {\n\t\t\treturn \"\", false, 0, 0, errors.Wrap(err, \"failed to get uploader config\")\n\t\t}\n\t\tif parallelFilesUpload > 0 {\n\t\t\tlog.Warnf(\"ParallelFilesUpload is set to %d, but restic does not support parallel file uploads. Ignoring.\", parallelFilesUpload)\n\t\t}\n\t}\n\n\tbackupCmd := resticBackupCMDFunc(rp.repoIdentifier, rp.credentialsFile, path, tags)\n\tbackupCmd.Env = rp.cmdEnv\n\tbackupCmd.CACertFile = rp.caCertFile\n\tif len(rp.extraFlags) != 0 {\n\t\tbackupCmd.ExtraFlags = append(backupCmd.ExtraFlags, rp.extraFlags...)\n\t}\n\n\tif parentSnapshot != \"\" {\n\t\tbackupCmd.ExtraFlags = append(backupCmd.ExtraFlags, fmt.Sprintf(\"--parent=%s\", parentSnapshot))\n\t}\n\n\tsummary, stderrBuf, err := resticBackupFunc(backupCmd, log, updater)\n\tif err != nil {\n\t\tif strings.Contains(stderrBuf, \"snapshot is empty\") {\n\t\t\tlog.Debugf(\"Restic backup got empty dir with %s path\", path)\n\t\t\treturn \"\", true, 0, 0, nil\n\t\t}\n\t\treturn \"\", false, 0, 0, errors.WithStack(fmt.Errorf(\"error running restic backup command %s with error: %v stderr: %v\", backupCmd.String(), err, stderrBuf))\n\t}\n\t// GetSnapshotID\n\tsnapshotIDCmd := resticGetSnapshotFunc(rp.repoIdentifier, rp.credentialsFile, tags)\n\tsnapshotIDCmd.Env = rp.cmdEnv\n\tsnapshotIDCmd.CACertFile = rp.caCertFile\n\tif len(rp.extraFlags) != 0 {\n\t\tsnapshotIDCmd.ExtraFlags = append(snapshotIDCmd.ExtraFlags, rp.extraFlags...)\n\t}\n\tsnapshotID, err := resticGetSnapshotIDFunc(snapshotIDCmd)\n\tif err != nil {\n\t\treturn \"\", false, 0, 0, errors.WithStack(fmt.Errorf(\"error getting snapshot id with error: %v\", err))\n\t}\n\tlog.Infof(\"Run command=%s, stdout=%s, stderr=%s\", backupCmd.String(), summary, stderrBuf)\n\treturn snapshotID, false, 0, 0, nil\n}\n\n// RunRestore runs a `restore` command and monitors the volume size to\n// provide progress updates to the caller.\nfunc (rp *resticProvider) RunRestore(\n\tctx context.Context,\n\tsnapshotID string,\n\tvolumePath string,\n\tvolMode uploader.PersistentVolumeMode,\n\tuploaderCfg map[string]string,\n\tupdater uploader.ProgressUpdater) (int64, error) {\n\tif updater == nil {\n\t\treturn 0, errors.New(\"Need to initial backup progress updater first\")\n\t}\n\tlog := rp.log.WithFields(logrus.Fields{\n\t\t\"snapshotID\": snapshotID,\n\t\t\"volumePath\": volumePath,\n\t})\n\n\tif volMode == uploader.PersistentVolumeBlock {\n\t\treturn 0, errors.New(\"unable to support block mode\")\n\t}\n\n\trestoreCmd := resticRestoreCMDFunc(rp.repoIdentifier, rp.credentialsFile, snapshotID, volumePath)\n\trestoreCmd.Env = rp.cmdEnv\n\trestoreCmd.CACertFile = rp.caCertFile\n\tif len(rp.extraFlags) != 0 {\n\t\trestoreCmd.ExtraFlags = append(restoreCmd.ExtraFlags, rp.extraFlags...)\n\t}\n\n\textraFlags, err := rp.parseRestoreExtraFlags(uploaderCfg)\n\tif err != nil {\n\t\treturn 0, errors.Wrap(err, \"failed to parse uploader config\")\n\t} else if len(extraFlags) != 0 {\n\t\trestoreCmd.ExtraFlags = append(restoreCmd.ExtraFlags, extraFlags...)\n\t}\n\n\tstdout, stderr, err := restic.RunRestore(restoreCmd, log, updater)\n\n\tlog.Infof(\"Run command=%v, stdout=%s, stderr=%s\", restoreCmd, stdout, stderr)\n\treturn 0, err\n}\n\nfunc (rp *resticProvider) parseRestoreExtraFlags(uploaderCfg map[string]string) ([]string, error) {\n\textraFlags := []string{}\n\tif len(uploaderCfg) == 0 {\n\t\treturn extraFlags, nil\n\t}\n\n\twriteSparseFiles, err := uploaderutil.GetWriteSparseFiles(uploaderCfg)\n\tif err != nil {\n\t\treturn extraFlags, errors.Wrap(err, \"failed to get uploader config\")\n\t}\n\n\tif writeSparseFiles {\n\t\textraFlags = append(extraFlags, \"--sparse\")\n\t}\n\n\tif restoreConcurrency, err := uploaderutil.GetRestoreConcurrency(uploaderCfg); err == nil && restoreConcurrency > 0 {\n\t\treturn extraFlags, errors.New(\"restic does not support parallel restore\")\n\t}\n\n\treturn extraFlags, nil\n}\n" }, { "path": "pkg/uploader/provider/restic_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage provider\n\nimport (\n\t\"errors\"\n\t\"os\"\n\t\"reflect\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/mock\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\t\"github.com/vmware-tanzu/velero/internal/credentials\"\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/restic\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n)\n\nfunc TestResticRunBackup(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tnilUpdater bool\n\t\tparentSnapshot string\n\t\trp *resticProvider\n\t\tvolMode uploader.PersistentVolumeMode\n\t\thookBackupFunc func(string, string, string, map[string]string) *restic.Command\n\t\thookResticBackupFunc func(*restic.Command, logrus.FieldLogger, uploader.ProgressUpdater) (string, string, error)\n\t\thookResticGetSnapshotFunc func(string, string, map[string]string) *restic.Command\n\t\thookResticGetSnapshotIDFunc func(*restic.Command) (string, error)\n\t\terrorHandleFunc func(err error) bool\n\t}{\n\t\t{\n\t\t\tname: \"nil uploader\",\n\t\t\trp: &resticProvider{log: logrus.New()},\n\t\t\tnilUpdater: true,\n\t\t\thookBackupFunc: func(repoIdentifier string, passwordFile string, path string, tags map[string]string) *restic.Command {\n\t\t\t\treturn &restic.Command{Command: \"date\"}\n\t\t\t},\n\t\t\terrorHandleFunc: func(err error) bool {\n\t\t\t\treturn strings.Contains(err.Error(), \"Need to initial backup progress updater first\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"wrong restic execute command\",\n\t\t\trp: &resticProvider{log: logrus.New()},\n\t\t\thookBackupFunc: func(repoIdentifier string, passwordFile string, path string, tags map[string]string) *restic.Command {\n\t\t\t\treturn &restic.Command{Command: \"date\"}\n\t\t\t},\n\t\t\terrorHandleFunc: func(err error) bool {\n\t\t\t\treturn strings.Contains(err.Error(), \"error running\")\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"has parent snapshot\",\n\t\t\trp: &resticProvider{log: logrus.New()},\n\t\t\tparentSnapshot: \"parentSnapshot\",\n\t\t\thookBackupFunc: func(repoIdentifier string, passwordFile string, path string, tags map[string]string) *restic.Command {\n\t\t\t\treturn &restic.Command{Command: \"date\"}\n\t\t\t},\n\t\t\thookResticBackupFunc: func(*restic.Command, logrus.FieldLogger, uploader.ProgressUpdater) (string, string, error) {\n\t\t\t\treturn \"\", \"\", nil\n\t\t\t},\n\n\t\t\thookResticGetSnapshotIDFunc: func(*restic.Command) (string, error) { return \"test-snapshot-id\", nil },\n\t\t\terrorHandleFunc: func(err error) bool {\n\t\t\t\treturn err == nil\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"has extra flags\",\n\t\t\trp: &resticProvider{log: logrus.New(), extraFlags: []string{\"testFlags\"}},\n\t\t\thookBackupFunc: func(string, string, string, map[string]string) *restic.Command {\n\t\t\t\treturn &restic.Command{Command: \"date\"}\n\t\t\t},\n\t\t\thookResticBackupFunc: func(*restic.Command, logrus.FieldLogger, uploader.ProgressUpdater) (string, string, error) {\n\t\t\t\treturn \"\", \"\", nil\n\t\t\t},\n\t\t\thookResticGetSnapshotIDFunc: func(*restic.Command) (string, error) { return \"test-snapshot-id\", nil },\n\t\t\terrorHandleFunc: func(err error) bool {\n\t\t\t\treturn err == nil\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"failed to get snapshot id\",\n\t\t\trp: &resticProvider{log: logrus.New(), extraFlags: []string{\"testFlags\"}},\n\t\t\thookBackupFunc: func(string, string, string, map[string]string) *restic.Command {\n\t\t\t\treturn &restic.Command{Command: \"date\"}\n\t\t\t},\n\t\t\thookResticBackupFunc: func(*restic.Command, logrus.FieldLogger, uploader.ProgressUpdater) (string, string, error) {\n\t\t\t\treturn \"\", \"\", nil\n\t\t\t},\n\t\t\thookResticGetSnapshotIDFunc: func(*restic.Command) (string, error) {\n\t\t\t\treturn \"test-snapshot-id\", errors.New(\"failed to get snapshot id\")\n\t\t\t},\n\t\t\terrorHandleFunc: func(err error) bool {\n\t\t\t\treturn strings.Contains(err.Error(), \"failed to get snapshot id\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"failed to use block mode\",\n\t\t\trp: &resticProvider{log: logrus.New(), extraFlags: []string{\"testFlags\"}},\n\t\t\tvolMode: uploader.PersistentVolumeBlock,\n\t\t\terrorHandleFunc: func(err error) bool {\n\t\t\t\treturn strings.Contains(err.Error(), \"unable to support block mode\")\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar err error\n\t\t\tparentSnapshot := tc.parentSnapshot\n\t\t\tif tc.hookBackupFunc != nil {\n\t\t\t\tresticBackupCMDFunc = tc.hookBackupFunc\n\t\t\t}\n\t\t\tif tc.hookResticBackupFunc != nil {\n\t\t\t\tresticBackupFunc = tc.hookResticBackupFunc\n\t\t\t}\n\t\t\tif tc.hookResticGetSnapshotFunc != nil {\n\t\t\t\tresticGetSnapshotFunc = tc.hookResticGetSnapshotFunc\n\t\t\t}\n\t\t\tif tc.hookResticGetSnapshotIDFunc != nil {\n\t\t\t\tresticGetSnapshotIDFunc = tc.hookResticGetSnapshotIDFunc\n\t\t\t}\n\t\t\tif tc.volMode == \"\" {\n\t\t\t\ttc.volMode = uploader.PersistentVolumeFilesystem\n\t\t\t}\n\t\t\tif !tc.nilUpdater {\n\t\t\t\tupdater := FakeBackupProgressUpdater{PodVolumeBackup: &velerov1api.PodVolumeBackup{}, Log: tc.rp.log, Ctx: t.Context(), Cli: fake.NewClientBuilder().WithScheme(util.VeleroScheme).Build()}\n\t\t\t\t_, _, _, _, err = tc.rp.RunBackup(t.Context(), \"var\", \"\", map[string]string{}, false, parentSnapshot, tc.volMode, map[string]string{}, &updater)\n\t\t\t} else {\n\t\t\t\t_, _, _, _, err = tc.rp.RunBackup(t.Context(), \"var\", \"\", map[string]string{}, false, parentSnapshot, tc.volMode, map[string]string{}, nil)\n\t\t\t}\n\n\t\t\ttc.rp.log.Infof(\"test name %v error %v\", tc.name, err)\n\t\t\trequire.True(t, tc.errorHandleFunc(err))\n\t\t})\n\t}\n}\n\nfunc TestResticRunRestore(t *testing.T) {\n\tresticRestoreCMDFunc = func(repoIdentifier, passwordFile, snapshotID, target string) *restic.Command {\n\t\treturn &restic.Command{Args: []string{\"\"}}\n\t}\n\ttestCases := []struct {\n\t\tname string\n\t\trp *resticProvider\n\t\tnilUpdater bool\n\t\thookResticRestoreFunc func(repoIdentifier, passwordFile, snapshotID, target string) *restic.Command\n\t\terrorHandleFunc func(err error) bool\n\t\tvolMode uploader.PersistentVolumeMode\n\t}{\n\t\t{\n\t\t\tname: \"wrong restic execute command\",\n\t\t\trp: &resticProvider{log: logrus.New()},\n\t\t\tnilUpdater: true,\n\t\t\terrorHandleFunc: func(err error) bool {\n\t\t\t\treturn strings.Contains(err.Error(), \"Need to initial backup progress updater first\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"has extral flags\",\n\t\t\trp: &resticProvider{log: logrus.New(), extraFlags: []string{\"test-extra-flags\"}},\n\t\t\thookResticRestoreFunc: func(repoIdentifier, passwordFile, snapshotID, target string) *restic.Command {\n\t\t\t\treturn &restic.Command{Args: []string{\"date\"}}\n\t\t\t},\n\t\t\terrorHandleFunc: func(err error) bool {\n\t\t\t\treturn strings.Contains(err.Error(), \"error running command\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"wrong restic execute command\",\n\t\t\trp: &resticProvider{log: logrus.New()},\n\t\t\thookResticRestoreFunc: func(repoIdentifier, passwordFile, snapshotID, target string) *restic.Command {\n\t\t\t\treturn &restic.Command{Args: []string{\"date\"}}\n\t\t\t},\n\t\t\terrorHandleFunc: func(err error) bool {\n\t\t\t\treturn strings.Contains(err.Error(), \"error running command\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"error block volume mode\",\n\t\t\trp: &resticProvider{log: logrus.New()},\n\t\t\terrorHandleFunc: func(err error) bool {\n\t\t\t\treturn strings.Contains(err.Error(), \"unable to support block mode\")\n\t\t\t},\n\t\t\tvolMode: uploader.PersistentVolumeBlock,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tif tc.volMode == \"\" {\n\t\t\t\ttc.volMode = uploader.PersistentVolumeFilesystem\n\t\t\t}\n\t\t\tresticRestoreCMDFunc = tc.hookResticRestoreFunc\n\t\t\tif tc.volMode == \"\" {\n\t\t\t\ttc.volMode = uploader.PersistentVolumeFilesystem\n\t\t\t}\n\t\t\tvar err error\n\t\t\tif !tc.nilUpdater {\n\t\t\t\tupdater := FakeBackupProgressUpdater{PodVolumeBackup: &velerov1api.PodVolumeBackup{}, Log: tc.rp.log, Ctx: t.Context(), Cli: fake.NewClientBuilder().WithScheme(util.VeleroScheme).Build()}\n\t\t\t\t_, err = tc.rp.RunRestore(t.Context(), \"\", \"var\", tc.volMode, map[string]string{}, &updater)\n\t\t\t} else {\n\t\t\t\t_, err = tc.rp.RunRestore(t.Context(), \"\", \"var\", tc.volMode, map[string]string{}, nil)\n\t\t\t}\n\n\t\t\ttc.rp.log.Infof(\"test name %v error %v\", tc.name, err)\n\t\t\trequire.True(t, tc.errorHandleFunc(err))\n\t\t})\n\t}\n}\n\nfunc TestClose(t *testing.T) {\n\tt.Run(\"Delete existing credentials file\", func(t *testing.T) {\n\t\t// Create temporary files for the credentials and caCert\n\t\tcredentialsFile, err := os.CreateTemp(t.TempDir(), \"credentialsFile\")\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"failed to create temp file: %v\", err)\n\t\t}\n\t\tdefer os.Remove(credentialsFile.Name())\n\n\t\tcaCertFile, err := os.CreateTemp(t.TempDir(), \"caCertFile\")\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"failed to create temp file: %v\", err)\n\t\t}\n\t\tdefer os.Remove(caCertFile.Name())\n\t\trp := &resticProvider{\n\t\t\tcredentialsFile: credentialsFile.Name(),\n\t\t\tcaCertFile: caCertFile.Name(),\n\t\t}\n\t\t// Test deleting an existing credentials file\n\t\terr = rp.Close(t.Context())\n\t\tif err != nil {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\n\t\t_, err = os.Stat(rp.credentialsFile)\n\t\tif !os.IsNotExist(err) {\n\t\t\tt.Errorf(\"expected credentials file to be deleted, got error: %v\", err)\n\t\t}\n\t})\n\n\tt.Run(\"Delete existing caCert file\", func(t *testing.T) {\n\t\t// Create temporary files for the credentials and caCert\n\t\tcaCertFile, err := os.CreateTemp(t.TempDir(), \"caCertFile\")\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"failed to create temp file: %v\", err)\n\t\t}\n\t\tdefer os.Remove(caCertFile.Name())\n\t\trp := &resticProvider{\n\t\t\tcredentialsFile: \"\",\n\t\t\tcaCertFile: \"\",\n\t\t}\n\t\terr = rp.Close(t.Context())\n\t\t// Test deleting an existing caCert file\n\t\tif err != nil {\n\t\t\tt.Errorf(\"unexpected error: %v\", err)\n\t\t}\n\n\t\t_, err = os.Stat(rp.caCertFile)\n\t\tif !os.IsNotExist(err) {\n\t\t\tt.Errorf(\"expected caCert file to be deleted, got error: %v\", err)\n\t\t}\n\t})\n}\n\ntype MockCredentialGetter struct {\n\tmock.Mock\n}\n\nfunc (m *MockCredentialGetter) Path(selector *corev1api.SecretKeySelector) (string, error) {\n\targs := m.Called(selector)\n\treturn args.Get(0).(string), args.Error(1)\n}\n\nfunc TestNewResticUploaderProvider(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\temptyBSL bool\n\t\tmockCredFunc func(*MockCredentialGetter, *corev1api.SecretKeySelector)\n\t\tresticCmdEnvFunc func(backupLocation *velerov1api.BackupStorageLocation, credentialFileStore credentials.FileStore) ([]string, error)\n\t\tresticTempCACertFileFunc func(caCert []byte, bsl string, fs filesystem.Interface) (string, error)\n\t\tcheckFunc func(t *testing.T, provider Provider, err error)\n\t}{\n\t\t{\n\t\t\tname: \"No error in creating temp credentials file\",\n\t\t\tmockCredFunc: func(credGetter *MockCredentialGetter, repoKeySelector *corev1api.SecretKeySelector) {\n\t\t\t\tcredGetter.On(\"Path\", repoKeySelector).Return(\"temp-credentials\", nil)\n\t\t\t},\n\t\t\tcheckFunc: func(t *testing.T, provider Provider, err error) {\n\t\t\t\tt.Helper()\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.NotNil(t, provider)\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"Error in creating temp credentials file\",\n\t\t\tmockCredFunc: func(credGetter *MockCredentialGetter, repoKeySelector *corev1api.SecretKeySelector) {\n\t\t\t\tcredGetter.On(\"Path\", repoKeySelector).Return(\"\", errors.New(\"error creating temp credentials file\"))\n\t\t\t},\n\t\t\tcheckFunc: func(t *testing.T, provider Provider, err error) {\n\t\t\t\tt.Helper()\n\t\t\t\trequire.Error(t, err)\n\t\t\t\tassert.Nil(t, provider)\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"ObjectStorage with CACert present and creating CACert file failed\",\n\t\t\tmockCredFunc: func(credGetter *MockCredentialGetter, repoKeySelector *corev1api.SecretKeySelector) {\n\t\t\t\tcredGetter.On(\"Path\", repoKeySelector).Return(\"temp-credentials\", nil)\n\t\t\t},\n\t\t\tresticTempCACertFileFunc: func(caCert []byte, bsl string, fs filesystem.Interface) (string, error) {\n\t\t\t\treturn \"\", errors.New(\"error writing CACert file\")\n\t\t\t},\n\t\t\tcheckFunc: func(t *testing.T, provider Provider, err error) {\n\t\t\t\tt.Helper()\n\t\t\t\trequire.Error(t, err)\n\t\t\t\tassert.Nil(t, provider)\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"Generating repository cmd failed\",\n\t\t\tmockCredFunc: func(credGetter *MockCredentialGetter, repoKeySelector *corev1api.SecretKeySelector) {\n\t\t\t\tcredGetter.On(\"Path\", repoKeySelector).Return(\"temp-credentials\", nil)\n\t\t\t},\n\t\t\tresticTempCACertFileFunc: func(caCert []byte, bsl string, fs filesystem.Interface) (string, error) {\n\t\t\t\treturn \"test-ca\", nil\n\t\t\t},\n\t\t\tresticCmdEnvFunc: func(backupLocation *velerov1api.BackupStorageLocation, credentialFileStore credentials.FileStore) ([]string, error) {\n\t\t\t\treturn nil, errors.New(\"error generating repository cmnd env\")\n\t\t\t},\n\t\t\tcheckFunc: func(t *testing.T, provider Provider, err error) {\n\t\t\t\tt.Helper()\n\t\t\t\trequire.Error(t, err)\n\t\t\t\tassert.Nil(t, provider)\n\t\t\t},\n\t\t}, {\n\t\t\tname: \"New provider with not nil bsl\",\n\t\t\tmockCredFunc: func(credGetter *MockCredentialGetter, repoKeySelector *corev1api.SecretKeySelector) {\n\t\t\t\tcredGetter.On(\"Path\", repoKeySelector).Return(\"temp-credentials\", nil)\n\t\t\t},\n\t\t\tresticTempCACertFileFunc: func(caCert []byte, bsl string, fs filesystem.Interface) (string, error) {\n\t\t\t\treturn \"test-ca\", nil\n\t\t\t},\n\t\t\tresticCmdEnvFunc: func(backupLocation *velerov1api.BackupStorageLocation, credentialFileStore credentials.FileStore) ([]string, error) {\n\t\t\t\treturn nil, nil\n\t\t\t},\n\t\t\tcheckFunc: func(t *testing.T, provider Provider, err error) {\n\t\t\t\tt.Helper()\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.NotNil(t, provider)\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"New provider with nil bsl\",\n\t\t\temptyBSL: true,\n\t\t\tmockCredFunc: func(credGetter *MockCredentialGetter, repoKeySelector *corev1api.SecretKeySelector) {\n\t\t\t\tcredGetter.On(\"Path\", repoKeySelector).Return(\"temp-credentials\", nil)\n\t\t\t},\n\t\t\tresticTempCACertFileFunc: func(caCert []byte, bsl string, fs filesystem.Interface) (string, error) {\n\t\t\t\treturn \"test-ca\", nil\n\t\t\t},\n\t\t\tresticCmdEnvFunc: func(backupLocation *velerov1api.BackupStorageLocation, credentialFileStore credentials.FileStore) ([]string, error) {\n\t\t\t\treturn nil, nil\n\t\t\t},\n\t\t\tcheckFunc: func(t *testing.T, provider Provider, err error) {\n\t\t\t\tt.Helper()\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.NotNil(t, provider)\n\t\t\t},\n\t\t},\n\t}\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\trepoIdentifier := \"my-repo\"\n\t\t\tbsl := &velerov1api.BackupStorageLocation{}\n\t\t\tif !tc.emptyBSL {\n\t\t\t\tbsl = builder.ForBackupStorageLocation(\"test-ns\", \"test-name\").CACert([]byte(\"my-cert\")).Result()\n\t\t\t}\n\t\t\tcredGetter := &credentials.CredentialGetter{}\n\t\t\trepoKeySelector := &corev1api.SecretKeySelector{}\n\t\t\tlog := logrus.New()\n\n\t\t\t// Mock CredentialGetter\n\t\t\tmockCredGetter := &MockCredentialGetter{}\n\t\t\tcredGetter.FromFile = mockCredGetter\n\t\t\ttc.mockCredFunc(mockCredGetter, repoKeySelector)\n\t\t\tif tc.resticCmdEnvFunc != nil {\n\t\t\t\tresticCmdEnvFunc = tc.resticCmdEnvFunc\n\t\t\t}\n\t\t\tif tc.resticTempCACertFileFunc != nil {\n\t\t\t\tresticTempCACertFileFunc = tc.resticTempCACertFileFunc\n\t\t\t}\n\t\t\tprovider, err := NewResticUploaderProvider(repoIdentifier, bsl, credGetter, repoKeySelector, log)\n\t\t\ttc.checkFunc(t, provider, err)\n\t\t})\n\t}\n}\n\nfunc TestParseUploaderConfig(t *testing.T) {\n\trp := &resticProvider{}\n\n\ttestCases := []struct {\n\t\tname string\n\t\tuploaderConfig map[string]string\n\t\texpectedFlags []string\n\t}{\n\t\t{\n\t\t\tname: \"SparseFilesEnabled\",\n\t\t\tuploaderConfig: map[string]string{\n\t\t\t\t\"WriteSparseFiles\": \"true\",\n\t\t\t},\n\t\t\texpectedFlags: []string{\"--sparse\"},\n\t\t},\n\t\t{\n\t\t\tname: \"SparseFilesDisabled\",\n\t\t\tuploaderConfig: map[string]string{\n\t\t\t\t\"writeSparseFiles\": \"false\",\n\t\t\t},\n\t\t\texpectedFlags: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"RestoreConcorrency\",\n\t\t\tuploaderConfig: map[string]string{\n\t\t\t\t\"Parallel\": \"5\",\n\t\t\t},\n\t\t\texpectedFlags: []string{},\n\t\t},\n\t}\n\n\tfor _, testCase := range testCases {\n\t\tt.Run(testCase.name, func(t *testing.T) {\n\t\t\tresult, err := rp.parseRestoreExtraFlags(testCase.uploaderConfig)\n\t\t\tif err != nil {\n\t\t\t\tt.Errorf(\"Test case %s failed with error: %v\", testCase.name, err)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tif !reflect.DeepEqual(result, testCase.expectedFlags) {\n\t\t\t\tt.Errorf(\"Test case %s failed. Expected: %v, Got: %v\", testCase.name, testCase.expectedFlags, result)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/uploader/types.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage uploader\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n)\n\nconst (\n\tResticType = \"restic\"\n\tKopiaType = \"kopia\"\n\tSnapshotRequesterTag = \"snapshot-requester\"\n\tSnapshotUploaderTag = \"snapshot-uploader\"\n)\n\ntype PersistentVolumeMode string\n\nconst (\n\t// PersistentVolumeBlock means the volume will not be formatted with a filesystem and will remain a raw block device.\n\tPersistentVolumeBlock PersistentVolumeMode = \"Block\"\n\t// PersistentVolumeFilesystem means the volume will be or is formatted with a filesystem.\n\tPersistentVolumeFilesystem PersistentVolumeMode = \"Filesystem\"\n)\n\n// ValidateUploaderType validates if the input param is a valid uploader type.\n// It will return an error if it's invalid.\nfunc ValidateUploaderType(t string) (string, error) {\n\tt = strings.TrimSpace(t)\n\tif t != KopiaType {\n\t\treturn \"\", fmt.Errorf(\"invalid uploader type '%s', valid type: '%s'\", t, KopiaType)\n\t}\n\n\treturn \"\", nil\n}\n\ntype SnapshotInfo struct {\n\tID string `json:\"id\"`\n\tSize int64 `json:\"Size\"`\n}\n\n// Progress which defined two variables to record progress\ntype Progress struct {\n\tTotalBytes int64 `json:\"totalBytes,omitempty\"`\n\tBytesDone int64 `json:\"doneBytes,omitempty\"`\n}\n\n// UploaderProgress which defined generic interface to update progress\ntype ProgressUpdater interface {\n\tUpdateProgress(p *Progress)\n}\n" }, { "path": "pkg/uploader/types_test.go", "content": "package uploader\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestValidateUploaderType(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tinput string\n\t\twantErr string\n\t\twantMsg string\n\t}{\n\t\t{\n\t\t\t\"' kopia ' is a valid type (space will be trimmed)\",\n\t\t\t\" kopia \",\n\t\t\t\"\",\n\t\t\t\"\",\n\t\t},\n\t\t{\n\t\t\t\"'anything_else' is invalid\",\n\t\t\t\"anything_else\",\n\t\t\t\"invalid uploader type 'anything_else', valid type: 'kopia'\",\n\t\t\t\"\",\n\t\t},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tmsg, err := ValidateUploaderType(tt.input)\n\t\t\tif tt.wantErr != \"\" {\n\t\t\t\trequire.EqualError(t, err, tt.wantErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tassert.Equal(t, tt.wantMsg, msg)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/uploader/util/uploader_config.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\thttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage util\n\nimport (\n\t\"strconv\"\n\n\t\"github.com/pkg/errors\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\nconst (\n\tParallelFilesUpload = \"ParallelFilesUpload\"\n\tWriteSparseFiles = \"WriteSparseFiles\"\n\tRestoreConcurrency = \"ParallelFilesDownload\"\n)\n\nfunc StoreBackupConfig(config *velerov1api.UploaderConfigForBackup) map[string]string {\n\tdata := make(map[string]string)\n\tdata[ParallelFilesUpload] = strconv.Itoa(config.ParallelFilesUpload)\n\treturn data\n}\n\nfunc StoreRestoreConfig(config *velerov1api.UploaderConfigForRestore) map[string]string {\n\tdata := make(map[string]string)\n\tif config.WriteSparseFiles != nil {\n\t\tdata[WriteSparseFiles] = strconv.FormatBool(*config.WriteSparseFiles)\n\t} else {\n\t\tdata[WriteSparseFiles] = strconv.FormatBool(false)\n\t}\n\n\tif config.ParallelFilesDownload > 0 {\n\t\tdata[RestoreConcurrency] = strconv.Itoa(config.ParallelFilesDownload)\n\t}\n\treturn data\n}\n\nfunc GetParallelFilesUpload(uploaderCfg map[string]string) (int, error) {\n\tparallelFilesUpload, ok := uploaderCfg[ParallelFilesUpload]\n\tif ok {\n\t\tparallelFilesUploadInt, err := strconv.Atoi(parallelFilesUpload)\n\t\tif err != nil {\n\t\t\treturn 0, errors.Wrap(err, \"failed to parse ParallelFilesUpload config\")\n\t\t}\n\t\treturn parallelFilesUploadInt, nil\n\t}\n\treturn 0, nil\n}\n\nfunc GetWriteSparseFiles(uploaderCfg map[string]string) (bool, error) {\n\twriteSparseFiles, ok := uploaderCfg[WriteSparseFiles]\n\tif ok {\n\t\twriteSparseFilesBool, err := strconv.ParseBool(writeSparseFiles)\n\t\tif err != nil {\n\t\t\treturn false, errors.Wrap(err, \"failed to parse WriteSparseFiles config\")\n\t\t}\n\t\treturn writeSparseFilesBool, nil\n\t}\n\treturn false, nil\n}\n\nfunc GetRestoreConcurrency(uploaderCfg map[string]string) (int, error) {\n\trestoreConcurrency, ok := uploaderCfg[RestoreConcurrency]\n\tif ok {\n\t\trestoreConcurrencyInt, err := strconv.Atoi(restoreConcurrency)\n\t\tif err != nil {\n\t\t\treturn 0, errors.Wrap(err, \"failed to parse RestoreConcurrency config\")\n\t\t}\n\t\treturn restoreConcurrencyInt, nil\n\t}\n\treturn 0, nil\n}\n" }, { "path": "pkg/uploader/util/uploader_config_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage util\n\nimport (\n\t\"reflect\"\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\nfunc TestStoreBackupConfig(t *testing.T) {\n\tconfig := &velerov1api.UploaderConfigForBackup{\n\t\tParallelFilesUpload: 3,\n\t}\n\n\texpectedData := map[string]string{\n\t\tParallelFilesUpload: \"3\",\n\t}\n\n\tresult := StoreBackupConfig(config)\n\n\tif !reflect.DeepEqual(result, expectedData) {\n\t\tt.Errorf(\"Expected: %v, but got: %v\", expectedData, result)\n\t}\n}\n\nfunc TestStoreRestoreConfig(t *testing.T) {\n\tvar (\n\t\tboolTrue = true\n\t\tboolFalse = false\n\t)\n\ttestCases := []struct {\n\t\tname string\n\t\tconfig *velerov1api.UploaderConfigForRestore\n\t\texpectedData map[string]string\n\t}{\n\t\t{\n\t\t\tname: \"WriteSparseFiles is true\",\n\t\t\tconfig: &velerov1api.UploaderConfigForRestore{\n\t\t\t\tWriteSparseFiles: &boolTrue,\n\t\t\t},\n\t\t\texpectedData: map[string]string{\n\t\t\t\tWriteSparseFiles: \"true\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"WriteSparseFiles is false\",\n\t\t\tconfig: &velerov1api.UploaderConfigForRestore{\n\t\t\t\tWriteSparseFiles: &boolFalse,\n\t\t\t},\n\t\t\texpectedData: map[string]string{\n\t\t\t\tWriteSparseFiles: \"false\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"WriteSparseFiles is nil\",\n\t\t\tconfig: &velerov1api.UploaderConfigForRestore{\n\t\t\t\tWriteSparseFiles: nil,\n\t\t\t},\n\t\t\texpectedData: map[string]string{\n\t\t\t\tWriteSparseFiles: \"false\", // Assuming default value is false for nil case\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Parallel is set\",\n\t\t\tconfig: &velerov1api.UploaderConfigForRestore{\n\t\t\t\tParallelFilesDownload: 5,\n\t\t\t},\n\t\t\texpectedData: map[string]string{\n\t\t\t\tRestoreConcurrency: \"5\",\n\t\t\t\tWriteSparseFiles: \"false\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tresult := StoreRestoreConfig(tc.config)\n\n\t\t\tif !reflect.DeepEqual(result, tc.expectedData) {\n\t\t\t\tt.Errorf(\"Expected: %v, but got: %v\", tc.expectedData, result)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetParallelFilesUpload(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tuploaderCfg map[string]string\n\t\texpectedResult int\n\t\texpectedError error\n\t}{\n\t\t{\n\t\t\tname: \"Valid ParallelFilesUpload\",\n\t\t\tuploaderCfg: map[string]string{ParallelFilesUpload: \"5\"},\n\t\t\texpectedResult: 5,\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"Missing ParallelFilesUpload\",\n\t\t\tuploaderCfg: map[string]string{},\n\t\t\texpectedResult: 0,\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"Invalid ParallelFilesUpload (not a number)\",\n\t\t\tuploaderCfg: map[string]string{ParallelFilesUpload: \"invalid\"},\n\t\t\texpectedResult: 0,\n\t\t\texpectedError: errors.Wrap(errors.New(\"strconv.Atoi: parsing \\\"invalid\\\": invalid syntax\"), \"failed to parse ParallelFilesUpload config\"),\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tresult, err := GetParallelFilesUpload(test.uploaderCfg)\n\n\t\t\tif result != test.expectedResult {\n\t\t\t\tt.Errorf(\"Expected result %d, but got %d\", test.expectedResult, result)\n\t\t\t}\n\n\t\t\tif (err == nil && test.expectedError != nil) || (err != nil && test.expectedError == nil) || (err != nil && test.expectedError != nil && err.Error() != test.expectedError.Error()) {\n\t\t\t\tt.Errorf(\"Expected error '%v', but got '%v'\", test.expectedError, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetWriteSparseFiles(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tuploaderCfg map[string]string\n\t\texpectedResult bool\n\t\texpectedError error\n\t}{\n\t\t{\n\t\t\tname: \"Valid WriteSparseFiles (true)\",\n\t\t\tuploaderCfg: map[string]string{WriteSparseFiles: \"true\"},\n\t\t\texpectedResult: true,\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"Valid WriteSparseFiles (false)\",\n\t\t\tuploaderCfg: map[string]string{WriteSparseFiles: \"false\"},\n\t\t\texpectedResult: false,\n\t\t\texpectedError: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"Invalid WriteSparseFiles (not a boolean)\",\n\t\t\tuploaderCfg: map[string]string{WriteSparseFiles: \"invalid\"},\n\t\t\texpectedResult: false,\n\t\t\texpectedError: errors.Wrap(errors.New(\"strconv.ParseBool: parsing \\\"invalid\\\": invalid syntax\"), \"failed to parse WriteSparseFiles config\"),\n\t\t},\n\t\t{\n\t\t\tname: \"Missing WriteSparseFiles\",\n\t\t\tuploaderCfg: map[string]string{},\n\t\t\texpectedResult: false,\n\t\t\texpectedError: nil,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tresult, err := GetWriteSparseFiles(test.uploaderCfg)\n\n\t\t\tif result != test.expectedResult {\n\t\t\t\tt.Errorf(\"Expected result %t, but got %t\", test.expectedResult, result)\n\t\t\t}\n\n\t\t\tif (err == nil && test.expectedError != nil) || (err != nil && test.expectedError == nil) || (err != nil && test.expectedError != nil && err.Error() != test.expectedError.Error()) {\n\t\t\t\tt.Errorf(\"Expected error '%v', but got '%v'\", test.expectedError, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetRestoreConcurrency(t *testing.T) {\n\ttestCases := []struct {\n\t\tName string\n\t\tUploaderCfg map[string]string\n\t\tExpectedResult int\n\t\tExpectedError bool\n\t\tExpectedErrorMsg string\n\t}{\n\t\t{\n\t\t\tName: \"Valid Configuration\",\n\t\t\tUploaderCfg: map[string]string{RestoreConcurrency: \"10\"},\n\t\t\tExpectedResult: 10,\n\t\t\tExpectedError: false,\n\t\t},\n\t\t{\n\t\t\tName: \"Missing Configuration\",\n\t\t\tUploaderCfg: map[string]string{},\n\t\t\tExpectedResult: 0,\n\t\t\tExpectedError: false,\n\t\t},\n\t\t{\n\t\t\tName: \"Invalid Configuration\",\n\t\t\tUploaderCfg: map[string]string{RestoreConcurrency: \"not_an_integer\"},\n\t\t\tExpectedResult: 0,\n\t\t\tExpectedError: true,\n\t\t\tExpectedErrorMsg: \"failed to parse RestoreConcurrency config: strconv.Atoi: parsing \\\"not_an_integer\\\": invalid syntax\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.Name, func(t *testing.T) {\n\t\t\tresult, err := GetRestoreConcurrency(tc.UploaderCfg)\n\n\t\t\tif tc.ExpectedError {\n\t\t\t\tif err.Error() != tc.ExpectedErrorMsg {\n\t\t\t\t\tt.Errorf(\"Expected error message %s, but got %s\", tc.ExpectedErrorMsg, err.Error())\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Errorf(\"Expected no error, but got %v\", err)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif result != tc.ExpectedResult {\n\t\t\t\tt.Errorf(\"Expected result %d, but got %d\", tc.ExpectedResult, result)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/util/actionhelpers/pod_helper.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actionhelpers\n\nimport (\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\nfunc RelatedItemsForPod(pod *corev1api.Pod, log logrus.FieldLogger) []velero.ResourceIdentifier {\n\tvar additionalItems []velero.ResourceIdentifier\n\tif pod.Spec.PriorityClassName != \"\" {\n\t\tlog.Infof(\"Adding priorityclass %s to additionalItems\", pod.Spec.PriorityClassName)\n\t\tadditionalItems = append(additionalItems, velero.ResourceIdentifier{\n\t\t\tGroupResource: kuberesource.PriorityClasses,\n\t\t\tName: pod.Spec.PriorityClassName,\n\t\t})\n\t}\n\n\tif len(pod.Spec.Volumes) == 0 {\n\t\tlog.Info(\"pod has no volumes\")\n\t}\n\n\tfor _, volume := range pod.Spec.Volumes {\n\t\tif volume.PersistentVolumeClaim != nil && volume.PersistentVolumeClaim.ClaimName != \"\" {\n\t\t\tlog.Infof(\"Adding pvc %s to additionalItems\", volume.PersistentVolumeClaim.ClaimName)\n\n\t\t\tadditionalItems = append(additionalItems, velero.ResourceIdentifier{\n\t\t\t\tGroupResource: kuberesource.PersistentVolumeClaims,\n\t\t\t\tNamespace: pod.Namespace,\n\t\t\t\tName: volume.PersistentVolumeClaim.ClaimName,\n\t\t\t})\n\t\t}\n\t}\n\treturn additionalItems\n}\n" }, { "path": "pkg/util/actionhelpers/pvc_helper.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actionhelpers\n\nimport (\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\nfunc RelatedItemsForPVC(pvc *corev1api.PersistentVolumeClaim, log logrus.FieldLogger) []velero.ResourceIdentifier {\n\treturn []velero.ResourceIdentifier{\n\t\t{\n\t\t\tGroupResource: kuberesource.PersistentVolumes,\n\t\t\tName: pvc.Spec.VolumeName,\n\t\t},\n\t}\n}\n" }, { "path": "pkg/util/actionhelpers/rbac.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actionhelpers\n\nimport (\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\trbacv1 \"k8s.io/api/rbac/v1\"\n\trbacbeta \"k8s.io/api/rbac/v1beta1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/client-go/kubernetes\"\n\trbacclient \"k8s.io/client-go/kubernetes/typed/rbac/v1\"\n\trbacbetaclient \"k8s.io/client-go/kubernetes/typed/rbac/v1beta1\"\n)\n\n// ClusterRoleBindingLister allows for listing ClusterRoleBindings in a version-independent way.\ntype ClusterRoleBindingLister interface {\n\t// List returns a slice of ClusterRoleBindings which can represent either v1 or v1beta1 ClusterRoleBindings.\n\tList() ([]ClusterRoleBinding, error)\n}\n\n// noopClusterRoleBindingLister exists to handle clusters where RBAC is disabled.\ntype NoopClusterRoleBindingLister struct {\n}\n\nfunc (noop NoopClusterRoleBindingLister) List() ([]ClusterRoleBinding, error) {\n\treturn []ClusterRoleBinding{}, nil\n}\n\ntype V1ClusterRoleBindingLister struct {\n\tclient rbacclient.ClusterRoleBindingInterface\n}\n\nfunc (v1 V1ClusterRoleBindingLister) List() ([]ClusterRoleBinding, error) {\n\tcrbList, err := v1.client.List(context.TODO(), metav1.ListOptions{})\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\tvar crbs []ClusterRoleBinding\n\tfor _, crb := range crbList.Items {\n\t\tcrbs = append(crbs, V1ClusterRoleBinding{Crb: crb})\n\t}\n\n\treturn crbs, nil\n}\n\ntype V1beta1ClusterRoleBindingLister struct {\n\tclient rbacbetaclient.ClusterRoleBindingInterface\n}\n\nfunc (v1beta1 V1beta1ClusterRoleBindingLister) List() ([]ClusterRoleBinding, error) {\n\tcrbList, err := v1beta1.client.List(context.TODO(), metav1.ListOptions{})\n\tif err != nil {\n\t\treturn nil, errors.WithStack(err)\n\t}\n\tvar crbs []ClusterRoleBinding\n\tfor _, crb := range crbList.Items {\n\t\tcrbs = append(crbs, V1beta1ClusterRoleBinding{Crb: crb})\n\t}\n\n\treturn crbs, nil\n}\n\n// NewClusterRoleBindingListerMap creates a map of RBAC version strings to their associated\n// ClusterRoleBindingLister structs.\n// Necessary so that callers to the ClusterRoleBindingLister interfaces don't need the kubernetes.Interface.\nfunc NewClusterRoleBindingListerMap(clientset kubernetes.Interface) map[string]ClusterRoleBindingLister {\n\treturn map[string]ClusterRoleBindingLister{\n\t\trbacv1.SchemeGroupVersion.Version: V1ClusterRoleBindingLister{client: clientset.RbacV1().ClusterRoleBindings()},\n\t\trbacbeta.SchemeGroupVersion.Version: V1beta1ClusterRoleBindingLister{client: clientset.RbacV1beta1().ClusterRoleBindings()},\n\t\t\"\": NoopClusterRoleBindingLister{},\n\t}\n}\n\n// ClusterRoleBinding abstracts access to ClusterRoleBindings whether they're v1 or v1beta1.\ntype ClusterRoleBinding interface {\n\t// Name returns the name of a ClusterRoleBinding.\n\tName() string\n\t// ServiceAccountSubjects returns the names of subjects that are service accounts in the given namespace.\n\tServiceAccountSubjects(namespace string) []string\n\t// RoleRefName returns the name of a ClusterRoleBinding's RoleRef.\n\tRoleRefName() string\n}\n\ntype V1ClusterRoleBinding struct {\n\tCrb rbacv1.ClusterRoleBinding\n}\n\nfunc (c V1ClusterRoleBinding) Name() string {\n\treturn c.Crb.Name\n}\n\nfunc (c V1ClusterRoleBinding) RoleRefName() string {\n\treturn c.Crb.RoleRef.Name\n}\n\nfunc (c V1ClusterRoleBinding) ServiceAccountSubjects(namespace string) []string {\n\tvar saSubjects []string\n\tfor _, s := range c.Crb.Subjects {\n\t\tif s.Kind == rbacv1.ServiceAccountKind && s.Namespace == namespace {\n\t\t\tsaSubjects = append(saSubjects, s.Name)\n\t\t}\n\t}\n\treturn saSubjects\n}\n\ntype V1beta1ClusterRoleBinding struct {\n\tCrb rbacbeta.ClusterRoleBinding\n}\n\nfunc (c V1beta1ClusterRoleBinding) Name() string {\n\treturn c.Crb.Name\n}\n\nfunc (c V1beta1ClusterRoleBinding) RoleRefName() string {\n\treturn c.Crb.RoleRef.Name\n}\n\nfunc (c V1beta1ClusterRoleBinding) ServiceAccountSubjects(namespace string) []string {\n\tvar saSubjects []string\n\tfor _, s := range c.Crb.Subjects {\n\t\tif s.Kind == rbacv1.ServiceAccountKind && s.Namespace == namespace {\n\t\t\tsaSubjects = append(saSubjects, s.Name)\n\t\t}\n\t}\n\treturn saSubjects\n}\n" }, { "path": "pkg/util/actionhelpers/service_account_helper.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage actionhelpers\n\nimport (\n\t\"github.com/sirupsen/logrus\"\n\trbacv1 \"k8s.io/api/rbac/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\n\tvelerodiscovery \"github.com/vmware-tanzu/velero/pkg/discovery\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/plugin/velero\"\n)\n\nfunc ClusterRoleBindingsForAction(clusterRoleBindingListers map[string]ClusterRoleBindingLister, discoveryHelper velerodiscovery.Helper) ([]ClusterRoleBinding, error) {\n\t// Look up the supported RBAC version\n\tvar supportedAPI metav1.GroupVersionForDiscovery\n\tfor _, ag := range discoveryHelper.APIGroups() {\n\t\tif ag.Name == rbacv1.GroupName {\n\t\t\tsupportedAPI = ag.PreferredVersion\n\t\t\tbreak\n\t\t}\n\t}\n\n\tcrbLister := clusterRoleBindingListers[supportedAPI.Version]\n\n\t// This should be safe because the List call will return a 0-item slice\n\t// if there's no matching API version.\n\treturn crbLister.List()\n}\n\nfunc RelatedItemsForServiceAccount(objectMeta metav1.Object, clusterRoleBindings []ClusterRoleBinding, log logrus.FieldLogger) []velero.ResourceIdentifier {\n\tvar (\n\t\tnamespace = objectMeta.GetNamespace()\n\t\tname = objectMeta.GetName()\n\t\tbindings = sets.NewString()\n\t\troles = sets.NewString()\n\t)\n\n\tfor _, crb := range clusterRoleBindings {\n\t\tfor _, s := range crb.ServiceAccountSubjects(namespace) {\n\t\t\tif s == name {\n\t\t\t\tlog.Infof(\"Adding clusterrole %s and clusterrolebinding %s to relatedItems since serviceaccount %s/%s is a subject\",\n\t\t\t\t\tcrb.RoleRefName(), crb.Name(), namespace, name)\n\n\t\t\t\tbindings.Insert(crb.Name())\n\t\t\t\troles.Insert(crb.RoleRefName())\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t}\n\n\tvar relatedItems []velero.ResourceIdentifier\n\tfor binding := range bindings {\n\t\trelatedItems = append(relatedItems, velero.ResourceIdentifier{\n\t\t\tGroupResource: kuberesource.ClusterRoleBindings,\n\t\t\tName: binding,\n\t\t})\n\t}\n\n\tfor role := range roles {\n\t\trelatedItems = append(relatedItems, velero.ResourceIdentifier{\n\t\t\tGroupResource: kuberesource.ClusterRoles,\n\t\t\tName: role,\n\t\t})\n\t}\n\n\treturn relatedItems\n}\n" }, { "path": "pkg/util/azure/credential.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage azure\n\nimport (\n\t\"os\"\n\t\"strings\"\n\n\t\"github.com/Azure/azure-sdk-for-go/sdk/azcore\"\n\t\"github.com/Azure/azure-sdk-for-go/sdk/azcore/policy\"\n\t\"github.com/Azure/azure-sdk-for-go/sdk/azidentity\"\n\t\"github.com/pkg/errors\"\n)\n\n// NewCredential constructs a Credential that tries the config credential, workload identity credential\n// and managed identity credential according to the provided creds.\nfunc NewCredential(creds map[string]string, options policy.ClientOptions) (azcore.TokenCredential, error) {\n\tadditionalTenants := []string{}\n\tif tenants := creds[CredentialKeyAdditionallyAllowedTenants]; tenants != \"\" {\n\t\tadditionalTenants = strings.Split(tenants, \";\")\n\t}\n\n\t// config credential\n\tif len(creds[CredentialKeyClientSecret]) > 0 ||\n\t\tlen(creds[CredentialKeyClientCertificate]) > 0 ||\n\t\tlen(creds[CredentialKeyClientCertificatePath]) > 0 ||\n\t\tlen(creds[CredentialKeyUsername]) > 0 {\n\t\treturn newConfigCredential(creds, configCredentialOptions{\n\t\t\tClientOptions: options,\n\t\t\tAdditionallyAllowedTenants: additionalTenants,\n\t\t})\n\t}\n\n\t// workload identity credential\n\tif len(os.Getenv(\"AZURE_FEDERATED_TOKEN_FILE\")) > 0 {\n\t\treturn azidentity.NewWorkloadIdentityCredential(&azidentity.WorkloadIdentityCredentialOptions{\n\t\t\tAdditionallyAllowedTenants: additionalTenants,\n\t\t\tClientOptions: options,\n\t\t})\n\t}\n\n\t// managed identity credential\n\to := &azidentity.ManagedIdentityCredentialOptions{ClientOptions: options, ID: azidentity.ClientID(creds[CredentialKeyClientID])}\n\treturn azidentity.NewManagedIdentityCredential(o)\n}\n\ntype configCredentialOptions struct {\n\tazcore.ClientOptions\n\tAdditionallyAllowedTenants []string\n}\n\n// newConfigCredential works similar as the azidentity.EnvironmentCredential but reads the credentials from a map\n// rather than environment variables. This is required for Velero to run B/R concurrently\n// https://github.com/Azure/azure-sdk-for-go/blob/sdk/azidentity/v1.3.0/sdk/azidentity/environment_credential.go#L80\nfunc newConfigCredential(creds map[string]string, options configCredentialOptions) (azcore.TokenCredential, error) {\n\ttenantID := creds[CredentialKeyTenantID]\n\tif tenantID == \"\" {\n\t\treturn nil, errors.Errorf(\"%s is required\", CredentialKeyTenantID)\n\t}\n\tclientID := creds[CredentialKeyClientID]\n\tif clientID == \"\" {\n\t\treturn nil, errors.Errorf(\"%s is required\", CredentialKeyClientID)\n\t}\n\n\t// client secret\n\tif clientSecret := creds[CredentialKeyClientSecret]; clientSecret != \"\" {\n\t\treturn azidentity.NewClientSecretCredential(tenantID, clientID, clientSecret, &azidentity.ClientSecretCredentialOptions{\n\t\t\tAdditionallyAllowedTenants: options.AdditionallyAllowedTenants,\n\t\t\tClientOptions: options.ClientOptions,\n\t\t})\n\t}\n\n\t// raw certificate or certificate file\n\tif rawCerts, certsPath := []byte(creds[CredentialKeyClientCertificate]), creds[CredentialKeyClientCertificatePath]; len(rawCerts) > 0 || len(certsPath) > 0 {\n\t\tvar err error\n\t\t// raw certificate isn't specified while certificate path is specified\n\t\tif len(rawCerts) == 0 {\n\t\t\trawCerts, err = os.ReadFile(certsPath)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, errors.Wrapf(err, \"failed to read certificate file %s\", certsPath)\n\t\t\t}\n\t\t}\n\n\t\tvar password []byte\n\t\tif v := creds[CredentialKeyClientCertificatePassword]; v != \"\" {\n\t\t\tpassword = []byte(v)\n\t\t}\n\t\tcerts, key, err := azidentity.ParseCertificates(rawCerts, password)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrap(err, \"failed to parse certificate\")\n\t\t}\n\t\to := &azidentity.ClientCertificateCredentialOptions{\n\t\t\tAdditionallyAllowedTenants: options.AdditionallyAllowedTenants,\n\t\t\tClientOptions: options.ClientOptions,\n\t\t}\n\t\tif v, ok := creds[CredentialKeySendCertChain]; ok {\n\t\t\to.SendCertificateChain = v == \"1\" || strings.ToLower(v) == \"true\"\n\t\t}\n\t\treturn azidentity.NewClientCertificateCredential(tenantID, clientID, certs, key, o)\n\t}\n\n\treturn nil, errors.New(\"incomplete credential configuration. Only AZURE_TENANT_ID and AZURE_CLIENT_ID are set\")\n}\n" }, { "path": "pkg/util/azure/credential_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage azure\n\nimport (\n\t\"os\"\n\t\"testing\"\n\n\t\"github.com/Azure/azure-sdk-for-go/sdk/azcore/policy\"\n\t\"github.com/Azure/azure-sdk-for-go/sdk/azidentity\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestNewCredential(t *testing.T) {\n\toptions := policy.ClientOptions{}\n\n\t// invalid client secret credential (missing tenant ID)\n\tcreds := map[string]string{\n\t\tCredentialKeyClientID: \"clientid\",\n\t\tCredentialKeyClientSecret: \"secret\",\n\t}\n\t_, err := NewCredential(creds, options)\n\trequire.Error(t, err)\n\n\t// valid client secret credential\n\tcreds = map[string]string{\n\t\tCredentialKeyTenantID: \"tenantid\",\n\t\tCredentialKeyClientID: \"clientid\",\n\t\tCredentialKeyClientSecret: \"secret\",\n\t}\n\ttokenCredential, err := NewCredential(creds, options)\n\trequire.NoError(t, err)\n\tassert.IsType(t, &azidentity.ClientSecretCredential{}, tokenCredential)\n\n\t// client certificate credential\n\tcertData, err := readCertData()\n\trequire.NoError(t, err)\n\tcreds = map[string]string{\n\t\tCredentialKeyTenantID: \"tenantid\",\n\t\tCredentialKeyClientID: \"clientid\",\n\t\tCredentialKeyClientCertificate: certData,\n\t}\n\ttokenCredential, err = NewCredential(creds, options)\n\trequire.NoError(t, err)\n\tassert.IsType(t, &azidentity.ClientCertificateCredential{}, tokenCredential)\n\n\t// workload identity credential\n\tos.Setenv(CredentialKeyTenantID, \"tenantid\")\n\tos.Setenv(CredentialKeyClientID, \"clientid\")\n\tos.Setenv(\"AZURE_FEDERATED_TOKEN_FILE\", \"/tmp/token\")\n\tcreds = map[string]string{}\n\ttokenCredential, err = NewCredential(creds, options)\n\trequire.NoError(t, err)\n\tassert.IsType(t, &azidentity.WorkloadIdentityCredential{}, tokenCredential)\n\tos.Clearenv()\n\n\t// managed identity credential\n\tcreds = map[string]string{CredentialKeyClientID: \"clientid\"}\n\ttokenCredential, err = NewCredential(creds, options)\n\trequire.NoError(t, err)\n\tassert.IsType(t, &azidentity.ManagedIdentityCredential{}, tokenCredential)\n}\n\nfunc Test_newConfigCredential(t *testing.T) {\n\toptions := configCredentialOptions{}\n\n\t// tenantID not specified\n\tcreds := map[string]string{}\n\t_, err := newConfigCredential(creds, options)\n\trequire.Error(t, err)\n\n\t// clientID not specified\n\tcreds = map[string]string{\n\t\tCredentialKeyTenantID: \"clientid\",\n\t}\n\t_, err = newConfigCredential(creds, options)\n\trequire.Error(t, err)\n\n\t// client secret\n\tcreds = map[string]string{\n\t\tCredentialKeyTenantID: \"clientid\",\n\t\tCredentialKeyClientID: \"clientid\",\n\t\tCredentialKeyClientSecret: \"secret\",\n\t}\n\tcredential, err := newConfigCredential(creds, options)\n\trequire.NoError(t, err)\n\trequire.NotNil(t, credential)\n\t_, ok := credential.(*azidentity.ClientSecretCredential)\n\trequire.True(t, ok)\n\n\t// client certificate\n\tcertData, err := readCertData()\n\trequire.NoError(t, err)\n\tcreds = map[string]string{\n\t\tCredentialKeyTenantID: \"clientid\",\n\t\tCredentialKeyClientID: \"clientid\",\n\t\tCredentialKeyClientCertificate: certData,\n\t}\n\tcredential, err = newConfigCredential(creds, options)\n\trequire.NoError(t, err)\n\trequire.NotNil(t, credential)\n\t_, ok = credential.(*azidentity.ClientCertificateCredential)\n\trequire.True(t, ok)\n}\n\nfunc readCertData() (string, error) {\n\tdata, err := os.ReadFile(\"testdata/certificate.pem\")\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\treturn string(data), nil\n}\n" }, { "path": "pkg/util/azure/storage.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage azure\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strconv\"\n\t\"strings\"\n\n\t\"github.com/Azure/azure-sdk-for-go/sdk/azcore/arm\"\n\t_ \"github.com/Azure/azure-sdk-for-go/sdk/azcore/arm/runtime\"\n\t\"github.com/Azure/azure-sdk-for-go/sdk/azcore/cloud\"\n\t\"github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storage/armstorage\"\n\t\"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n)\n\nconst (\n\t// the keys of Azure BSL config:\n\t// https://github.com/vmware-tanzu/velero-plugin-for-microsoft-azure/blob/main/backupstoragelocation.md\n\tBSLConfigResourceGroup = \"resourceGroup\"\n\tBSLConfigStorageAccount = \"storageAccount\"\n\tBSLConfigStorageAccountAccessKeyName = \"storageAccountKeyEnvVar\"\n\tBSLConfigSubscriptionID = \"subscriptionId\"\n\tBSLConfigStorageAccountURI = \"storageAccountURI\"\n\tBSLConfigUseAAD = \"useAAD\"\n\tBSLConfigActiveDirectoryAuthorityURI = \"activeDirectoryAuthorityURI\"\n\n\tserviceNameBlob cloud.ServiceName = \"blob\"\n)\n\nfunc init() {\n\tcloud.AzureChina.Services[serviceNameBlob] = cloud.ServiceConfiguration{\n\t\tEndpoint: \"blob.core.chinacloudapi.cn\",\n\t}\n\tcloud.AzureGovernment.Services[serviceNameBlob] = cloud.ServiceConfiguration{\n\t\tEndpoint: \"blob.core.usgovcloudapi.net\",\n\t}\n\tcloud.AzurePublic.Services[serviceNameBlob] = cloud.ServiceConfiguration{\n\t\tEndpoint: \"blob.core.windows.net\",\n\t}\n}\n\n// NewStorageClient creates a blob storage client(data plane) with the provided config which contains BSL config and the credential file name.\n// The returned azblob.SharedKeyCredential is needed for Azure plugin to generate the SAS URL when auth with storage\n// account access key\nfunc NewStorageClient(log logrus.FieldLogger, config map[string]string) (*azblob.Client, *azblob.SharedKeyCredential, error) {\n\t// rename to bslCfg for easy understanding\n\tbslCfg := config\n\n\t// storage account is required\n\tstorageAccount := bslCfg[BSLConfigStorageAccount]\n\tif storageAccount == \"\" {\n\t\treturn nil, nil, errors.Errorf(\"%s is required in BSL\", BSLConfigStorageAccount)\n\t}\n\n\t// read the credentials provided by users\n\tcreds, err := LoadCredentials(config)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\t// exchange the storage account access key if needed\n\tcreds, err = GetStorageAccountCredentials(bslCfg, creds)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\t// get the storage account URI\n\turi, err := getStorageAccountURI(log, bslCfg, creds)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\tclientOptions, err := GetClientOptions(bslCfg, creds)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\tblobClientOptions := &azblob.ClientOptions{\n\t\tClientOptions: clientOptions,\n\t}\n\n\t// auth with storage account access key\n\taccessKey := creds[CredentialKeyStorageAccountAccessKey]\n\tif accessKey != \"\" {\n\t\tlog.Info(\"auth with the storage account access key\")\n\t\tcred, err := azblob.NewSharedKeyCredential(storageAccount, accessKey)\n\t\tif err != nil {\n\t\t\treturn nil, nil, errors.Wrap(err, \"failed to create storage account access key credential\")\n\t\t}\n\t\tclient, err := azblob.NewClientWithSharedKeyCredential(uri, cred, blobClientOptions)\n\t\tif err != nil {\n\t\t\treturn nil, nil, errors.Wrap(err, \"failed to create blob client with the storage account access key\")\n\t\t}\n\t\treturn client, cred, nil\n\t}\n\n\t// auth with Azure AD\n\tlog.Info(\"auth with Azure AD\")\n\tcred, err := NewCredential(creds, clientOptions)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\tclient, err := azblob.NewClient(uri, cred, blobClientOptions)\n\tif err != nil {\n\t\treturn nil, nil, errors.Wrap(err, \"failed to create blob client with the Azure AD credential\")\n\t}\n\treturn client, nil, nil\n}\n\n// GetStorageAccountCredentials returns the credentials to interactive with storage account according to the config of BSL\n// and credential file by the following order:\n// 1. Return the storage account access key directly if it is provided\n// 2. Return the content of the credential file directly if \"userAAD\" is set as true in BSL config\n// 3. Call Azure API to exchange the storage account access key\nfunc GetStorageAccountCredentials(bslCfg map[string]string, creds map[string]string) (map[string]string, error) {\n\t// use storage account access key if specified\n\tif name := bslCfg[BSLConfigStorageAccountAccessKeyName]; name != \"\" {\n\t\taccessKey := creds[name]\n\t\tif accessKey == \"\" {\n\t\t\treturn nil, errors.Errorf(\"no storage account access key with key %s found in credential\", name)\n\t\t}\n\t\tcreds[CredentialKeyStorageAccountAccessKey] = accessKey\n\t\treturn creds, nil\n\t}\n\n\t// use AAD\n\tif bslCfg[BSLConfigUseAAD] != \"\" {\n\t\tuseAAD, err := strconv.ParseBool(bslCfg[BSLConfigUseAAD])\n\t\tif err != nil {\n\t\t\treturn nil, errors.Errorf(\"failed to parse bool from useAAD string: %s\", bslCfg[BSLConfigUseAAD])\n\t\t}\n\n\t\tif useAAD {\n\t\t\treturn creds, nil\n\t\t}\n\t}\n\n\t// exchange the storage account access key\n\taccessKey, err := exchangeStorageAccountAccessKey(bslCfg, creds)\n\tif err != nil {\n\t\treturn nil, errors.WithMessage(err, \"failed to get storage account access key\")\n\t}\n\tcreds[CredentialKeyStorageAccountAccessKey] = accessKey\n\treturn creds, nil\n}\n\n// getStorageAccountURI returns the storage account URI by the following order:\n// 1. Return the storage account URI directly if it is specified in BSL config\n// 2. Try to call Azure API to get the storage account URI if possible(Background: https://github.com/vmware-tanzu/velero/issues/6163)\n// 3. Fall back to return the default URI\nfunc getStorageAccountURI(log logrus.FieldLogger, bslCfg map[string]string, creds map[string]string) (string, error) {\n\t// if the URI is specified in the BSL, return it directly\n\turi := bslCfg[BSLConfigStorageAccountURI]\n\tif uri != \"\" {\n\t\tlog.Infof(\"the storage account URI %q is specified in the BSL, use it directly\", uri)\n\t\treturn uri, nil\n\t}\n\n\tstorageAccount := bslCfg[BSLConfigStorageAccount]\n\n\tcloudCfg, err := getCloudConfiguration(bslCfg, creds)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\t// the default URI\n\turi = fmt.Sprintf(\"https://%s.%s\", storageAccount, cloudCfg.Services[serviceNameBlob].Endpoint)\n\n\t// the storage account access key cannot be used to get the storage account properties,\n\t// so fallback to the default URI\n\tif name := bslCfg[BSLConfigStorageAccountAccessKeyName]; name != \"\" && creds[name] != \"\" {\n\t\tlog.Infof(\"auth with the storage account access key, cannot retrieve the storage account properties, fallback to use the default URI %q\", uri)\n\t\treturn uri, nil\n\t}\n\n\tclient, err := newStorageAccountManagemenClient(bslCfg, creds)\n\tif err != nil {\n\t\tlog.Infof(\"failed to create the storage account management client: %v, fallback to use the default URI %q\", err, uri)\n\t\treturn uri, nil\n\t}\n\n\tresourceGroup := GetFromLocationConfigOrCredential(bslCfg, creds, BSLConfigResourceGroup, CredentialKeyResourceGroup)\n\t// we cannot get the storage account properties without the resource group, so fallback to the default URI\n\tif resourceGroup == \"\" {\n\t\tlog.Infof(\"resource group isn't set which is required to retrieve the storage account properties, fallback to use the default URI %q\", uri)\n\t\treturn uri, nil\n\t}\n\n\tproperties, err := client.GetProperties(context.Background(), resourceGroup, storageAccount, nil)\n\t// get error, fallback to the default URI\n\tif err != nil {\n\t\tlog.Infof(\"failed to retrieve the storage account properties: %v, fallback to use the default URI %q\", err, uri)\n\t\treturn uri, nil\n\t}\n\n\turi = *properties.Account.Properties.PrimaryEndpoints.Blob\n\tlog.Infof(\"use the storage account URI retrieved from the storage account properties %q\", uri)\n\treturn uri, nil\n}\n\n// try to exchange the storage account access key with the provided credentials\nfunc exchangeStorageAccountAccessKey(bslCfg, creds map[string]string) (string, error) {\n\tclient, err := newStorageAccountManagemenClient(bslCfg, creds)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\tresourceGroup := GetFromLocationConfigOrCredential(bslCfg, creds, BSLConfigResourceGroup, CredentialKeyResourceGroup)\n\tif resourceGroup == \"\" {\n\t\treturn \"\", errors.New(\"resource group is required in BSL or credential to exchange the storage account access key\")\n\t}\n\tstorageAccount := bslCfg[BSLConfigStorageAccount]\n\tif storageAccount == \"\" {\n\t\treturn \"\", errors.Errorf(\"%s is required in the BSL to exchange the storage account access key\", BSLConfigStorageAccount)\n\t}\n\n\texpand := \"kerb\"\n\tresp, err := client.ListKeys(context.Background(), resourceGroup, storageAccount, &armstorage.AccountsClientListKeysOptions{\n\t\tExpand: &expand,\n\t})\n\tif err != nil {\n\t\treturn \"\", errors.Wrap(err, \"failed to list storage account access keys\")\n\t}\n\tfor _, key := range resp.Keys {\n\t\tif key == nil || key.Permissions == nil {\n\t\t\tcontinue\n\t\t}\n\t\tif strings.EqualFold(string(*key.Permissions), string(armstorage.KeyPermissionFull)) {\n\t\t\treturn *key.Value, nil\n\t\t}\n\t}\n\treturn \"\", errors.New(\"no storage key with Full permissions found\")\n}\n\n// new a management client for the storage account\nfunc newStorageAccountManagemenClient(bslCfg map[string]string, creds map[string]string) (*armstorage.AccountsClient, error) {\n\tclientOptions, err := GetClientOptions(bslCfg, creds)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcred, err := NewCredential(creds, clientOptions)\n\tif err != nil {\n\t\treturn nil, errors.WithMessage(err, \"failed to create Azure AD credential\")\n\t}\n\n\tsubID := GetFromLocationConfigOrCredential(bslCfg, creds, BSLConfigSubscriptionID, CredentialKeySubscriptionID)\n\tif subID == \"\" {\n\t\treturn nil, errors.New(\"subscription ID is required in BSL or credential to create the storage account client\")\n\t}\n\n\tclient, err := armstorage.NewAccountsClient(subID, cred, &arm.ClientOptions{\n\t\tClientOptions: clientOptions,\n\t})\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"failed to create the storage account client\")\n\t}\n\n\treturn client, nil\n}\n" }, { "path": "pkg/util/azure/storage_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage azure\n\nimport (\n\t\"os\"\n\t\"path/filepath\"\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestNewStorageClient(t *testing.T) {\n\tlog := logrus.New()\n\tconfig := map[string]string{}\n\n\tname := filepath.Join(os.TempDir(), \"credential\")\n\tfile, err := os.Create(name)\n\trequire.NoError(t, err)\n\tdefer file.Close()\n\tdefer os.Remove(name)\n\t_, err = file.WriteString(\"AccessKey: YWNjZXNza2V5\\nAZURE_TENANT_ID: tenantid\\nAZURE_CLIENT_ID: clientid\\nAZURE_CLIENT_SECRET: secret\")\n\trequire.NoError(t, err)\n\n\t// storage account isn't specified\n\t_, _, err = NewStorageClient(log, config)\n\trequire.Error(t, err)\n\n\t// auth with storage account access key\n\tconfig = map[string]string{\n\t\tBSLConfigStorageAccount: \"storage-account\",\n\t\t\"credentialsFile\": name,\n\t\tBSLConfigStorageAccountAccessKeyName: \"AccessKey\",\n\t}\n\tclient, credential, err := NewStorageClient(log, config)\n\trequire.NoError(t, err)\n\tassert.NotNil(t, client)\n\tassert.NotNil(t, credential)\n\n\t// auth with Azure AD\n\tconfig = map[string]string{\n\t\tBSLConfigStorageAccount: \"storage-account\",\n\t\t\"credentialsFile\": name,\n\t\t\"useAAD\": \"true\",\n\t}\n\tclient, credential, err = NewStorageClient(log, config)\n\trequire.NoError(t, err)\n\tassert.NotNil(t, client)\n\tassert.Nil(t, credential)\n}\n\nfunc TestGetStorageAccountCredentials(t *testing.T) {\n\t// use access secret but no secret specified\n\tcfg := map[string]string{\n\t\tBSLConfigStorageAccountAccessKeyName: \"KEY\",\n\t}\n\tcreds := map[string]string{}\n\t_, err := GetStorageAccountCredentials(cfg, creds)\n\trequire.Error(t, err)\n\n\t// use access secret\n\tcfg = map[string]string{\n\t\tBSLConfigStorageAccountAccessKeyName: \"KEY\",\n\t}\n\tcreds = map[string]string{\n\t\t\"KEY\": \"key\",\n\t}\n\tm, err := GetStorageAccountCredentials(cfg, creds)\n\trequire.NoError(t, err)\n\tassert.Equal(t, \"key\", m[CredentialKeyStorageAccountAccessKey])\n\n\t// use AAD, but useAAD invalid\n\tcfg = map[string]string{\n\t\t\"useAAD\": \"invalid\",\n\t}\n\tcreds = map[string]string{}\n\t_, err = GetStorageAccountCredentials(cfg, creds)\n\trequire.Error(t, err)\n\n\t// use AAD\n\tcfg = map[string]string{\n\t\t\"useAAD\": \"true\",\n\t}\n\tcreds = map[string]string{\n\t\t\"KEY\": \"key\",\n\t}\n\tm, err = GetStorageAccountCredentials(cfg, creds)\n\trequire.NoError(t, err)\n\tassert.Equal(t, creds, m)\n}\n\nfunc Test_getStorageAccountURI(t *testing.T) {\n\tlog := logrus.New()\n\n\t// URI specified\n\tbslCfg := map[string]string{\n\t\tBSLConfigStorageAccountURI: \"uri\",\n\t}\n\tcreds := map[string]string{}\n\turi, err := getStorageAccountURI(log, bslCfg, creds)\n\trequire.NoError(t, err)\n\tassert.Equal(t, \"uri\", uri)\n\n\t// no URI specified, and auth with access key\n\tbslCfg = map[string]string{\n\t\tBSLConfigStorageAccountAccessKeyName: \"KEY\",\n\t}\n\tcreds = map[string]string{\n\t\t\"KEY\": \"value\",\n\t}\n\turi, err = getStorageAccountURI(log, bslCfg, creds)\n\trequire.NoError(t, err)\n\tassert.Equal(t, \"https://.blob.core.windows.net\", uri)\n\n\t// no URI specified, auth with AAD, resource group isn't specified\n\tbslCfg = map[string]string{\n\t\tBSLConfigSubscriptionID: \"subscriptionid\",\n\t}\n\tcreds = map[string]string{\n\t\t\"AZURE_TENANT_ID\": \"tenantid\",\n\t\t\"AZURE_CLIENT_ID\": \"clientid\",\n\t\t\"AZURE_CLIENT_SECRET\": \"secret\",\n\t}\n\turi, err = getStorageAccountURI(log, bslCfg, creds)\n\trequire.NoError(t, err)\n\tassert.Equal(t, \"https://.blob.core.windows.net\", uri)\n\n\t// no URI specified, auth with AAD, resource group specified\n\tbslCfg = map[string]string{\n\t\tBSLConfigSubscriptionID: \"subscriptionid\",\n\t\tBSLConfigResourceGroup: \"resourcegroup\",\n\t\tBSLConfigStorageAccount: \"account\",\n\t}\n\tcreds = map[string]string{\n\t\t\"AZURE_TENANT_ID\": \"tenantid\",\n\t\t\"AZURE_CLIENT_ID\": \"clientid\",\n\t\t\"AZURE_CLIENT_SECRET\": \"secret\",\n\t}\n\turi, err = getStorageAccountURI(log, bslCfg, creds)\n\trequire.NoError(t, err)\n\tassert.Equal(t, \"https://account.blob.core.windows.net\", uri)\n}\n\nfunc Test_exchangeStorageAccountAccessKey(t *testing.T) {\n\t// resource group isn't specified\n\tbslCfg := map[string]string{\n\t\tBSLConfigSubscriptionID: \"subscriptionid\",\n\t}\n\tcreds := map[string]string{\n\t\t\"AZURE_TENANT_ID\": \"tenantid\",\n\t\t\"AZURE_CLIENT_ID\": \"clientid\",\n\t\t\"AZURE_CLIENT_SECRET\": \"secret\",\n\t}\n\t_, err := exchangeStorageAccountAccessKey(bslCfg, creds)\n\trequire.Error(t, err)\n\n\t// storage account isn't specified\n\tbslCfg = map[string]string{\n\t\tBSLConfigSubscriptionID: \"subscriptionid\",\n\t\tBSLConfigResourceGroup: \"resourcegroup\",\n\t}\n\tcreds = map[string]string{\n\t\t\"AZURE_TENANT_ID\": \"tenantid\",\n\t\t\"AZURE_CLIENT_ID\": \"clientid\",\n\t\t\"AZURE_CLIENT_SECRET\": \"secret\",\n\t}\n\t_, err = exchangeStorageAccountAccessKey(bslCfg, creds)\n\trequire.Error(t, err)\n\n\t// storage account specified\n\tbslCfg = map[string]string{\n\t\tBSLConfigSubscriptionID: \"subscriptionid\",\n\t\tBSLConfigResourceGroup: \"resourcegroup\",\n\t\tBSLConfigStorageAccount: \"account\",\n\t}\n\tcreds = map[string]string{\n\t\t\"AZURE_TENANT_ID\": \"tenantid\",\n\t\t\"AZURE_CLIENT_ID\": \"clientid\",\n\t\t\"AZURE_CLIENT_SECRET\": \"secret\",\n\t}\n\t_, err = exchangeStorageAccountAccessKey(bslCfg, creds)\n\trequire.Error(t, err)\n}\n\nfunc Test_newStorageAccountManagemenClient(t *testing.T) {\n\t// subscription ID isn't specified\n\tbslCfg := map[string]string{}\n\tcreds := map[string]string{\n\t\t\"AZURE_TENANT_ID\": \"tenantid\",\n\t\t\"AZURE_CLIENT_ID\": \"clientid\",\n\t\t\"AZURE_CLIENT_SECRET\": \"secret\",\n\t}\n\t_, err := newStorageAccountManagemenClient(bslCfg, creds)\n\trequire.Error(t, err)\n\n\t// subscription ID isn't specified\n\tbslCfg = map[string]string{\n\t\tBSLConfigSubscriptionID: \"subscriptionid\",\n\t}\n\tcreds = map[string]string{\n\t\t\"AZURE_TENANT_ID\": \"tenantid\",\n\t\t\"AZURE_CLIENT_ID\": \"clientid\",\n\t\t\"AZURE_CLIENT_SECRET\": \"secret\",\n\t}\n\t_, err = newStorageAccountManagemenClient(bslCfg, creds)\n\trequire.NoError(t, err)\n}\n" }, { "path": "pkg/util/azure/testdata/certificate.pem", "content": "-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDL1hG+JYCfIPp3\ntlZ05J4pYIJ3Ckfs432bE3rYuWlR2w9KqdjWkKxuAxpjJ+T+uoqVaT3BFMfi4ZRY\nOCI69s4+lP3DwR8uBCp9xyVkF8thXfS3iui0liGDviVBoBJJWvjDFU8a/Hseg+Qf\noxAb6tx0kEc7V3ozBLWoIDJjfwJ3NdsLZGVtAC34qCWeEIvS97CDA4g3Kc6hYJIr\nAa7pxHzo/Nd0U3e7z+DlBcJV7dY6TZUyjBVTpzppWe+XQEOfKsjkDNykHEC1C1bC\nlG0u7unS7QOBMd6bOGkeL+Bc+n22slTzs5amsbDLNuobSaUsFt9vgD5jRD6FwhpX\nwj/Ek0F7AgMBAAECggEAblU3UWdXUcs2CCqIbcl52wfEVs8X05/n01MeAcWKvqYG\nhvGcz7eLvhir5dQoXcF3VhybMrIe6C4WcBIiZSxGwxU+rwEP8YaLwX1UPfOrQM7s\nsZTdFTLWfUslO3p7q300fdRA92iG9COMDZvkElh0cBvQksxs9sSr149l9vk+ymtC\nuBhZtHG6Ki0BIMBNC9jGUqDuOatXl/dkK4tNjXrNJT7tVwzPaqnNALIWl6B+k9oQ\nm1oNhSH2rvs9tw2ITXfIoIk9KdOMjQVUD43wKOaz0hNZhUsb1OFuls7UtRzaFcZH\nrMd/M8DtA104QTTlHK+XS7r+nqdv7+ZyB+suTdM+oQKBgQDxCrJZU3hJ0eJ4VYhK\nxGDfVGNpYxNkQ4CDB9fwRNbFr/Ck3kgzfE9QxTx1pJOolVmfuFmk9B86in4UNy91\nKdaqT79AU5RdOBXNN6tuMbLC0AVqe8sZq+1vWVVwbCstffxEMmyW1Ju/FLYPl2Zp\ne5P96dBh5B3mXrQtpDJ0RkxxaQKBgQDYfE6tQQnQSs2ewD6ae8Mu6j8ueDlVoZ37\nvze1QdBasR26xu2H8XBt3u41zc524BwQsB1GE1tnC8ZylrqwVEayK4FesSQRCO6o\nyK8QSdb06I5J4TaN+TppCDPLzstOh0Dmxp+iFUGoErb7AEOLAJ/VebhF9kBZObL/\nHYy4Es+bQwKBgHW/4vYuB3IQXNCp/+V+X1BZ+iJOaves3gekekF+b2itFSKFD8JO\n9LQhVfKmTheptdmHhgtF0keXxhV8C+vxX1Ndl7EF41FSh5vzmQRAtPHkCvFEviex\nTFD70/gSb1lO1UA/Xbqk69yBcprVPAtFejss0EYx2MVj+CLftmIEwW0ZAoGBAIMG\nEVQ45eikLXjkn78+Iq7VZbIJX6IdNBH29I+GqsUJJ5Yw6fh6P3KwF3qG+mvmTfYn\nsUAFXS+r58rYwVsRVsxlGmKmUc7hmhibhaEVH72QtvWuEiexbRG+viKfIVuA7t39\n3wXpWZiQ4yBdU4Pgt9wrVEU7ukyGaHiReOa7s90jAoGAJc0K7smn98YutQQ+g2ur\nybfnsl0YdsksaP2S2zvZUmNevKPrgnaIDDabOlhYYga+AK1G3FQ7/nefUgiIg1Nd\nkr+T6Q4osS3xHB6Az9p/jaF4R2KaWN2nNVCn7ecsmPxDdM7k1vLxaT26vwO9OP5f\nYU/5CeIzrfA5nQyPZkOXZBk=\n-----END PRIVATE KEY-----\n-----BEGIN CERTIFICATE-----\nMIIDazCCAlOgAwIBAgIUF2VIP4+AnEtb52KTCHbo4+fESfswDQYJKoZIhvcNAQEL\nBQAwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM\nGEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0xOTEwMzAyMjQ2MjBaFw0yMjA4\nMTkyMjQ2MjBaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw\nHwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB\nAQUAA4IBDwAwggEKAoIBAQDL1hG+JYCfIPp3tlZ05J4pYIJ3Ckfs432bE3rYuWlR\n2w9KqdjWkKxuAxpjJ+T+uoqVaT3BFMfi4ZRYOCI69s4+lP3DwR8uBCp9xyVkF8th\nXfS3iui0liGDviVBoBJJWvjDFU8a/Hseg+QfoxAb6tx0kEc7V3ozBLWoIDJjfwJ3\nNdsLZGVtAC34qCWeEIvS97CDA4g3Kc6hYJIrAa7pxHzo/Nd0U3e7z+DlBcJV7dY6\nTZUyjBVTpzppWe+XQEOfKsjkDNykHEC1C1bClG0u7unS7QOBMd6bOGkeL+Bc+n22\nslTzs5amsbDLNuobSaUsFt9vgD5jRD6FwhpXwj/Ek0F7AgMBAAGjUzBRMB0GA1Ud\nDgQWBBT6Mf9uXFB67bY2PeW3GCTKfkO7vDAfBgNVHSMEGDAWgBT6Mf9uXFB67bY2\nPeW3GCTKfkO7vDAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQCZ\n1+kTISX85v9/ag7glavaPFUYsOSOOofl8gSzov7L01YL+srq7tXdvZmWrjQ/dnOY\nh18rp9rb24vwIYxNioNG/M2cW1jBJwEGsDPOwdPV1VPcRmmUJW9kY130gRHBCd/N\nqB7dIkcQnpNsxPIIWI+sRQp73U0ijhOByDnCNHLHon6vbfFTwkO1XggmV5BdZ3uQ\nJNJyckILyNzlhmf6zhonMp4lVzkgxWsAm2vgdawd6dmBa+7Avb2QK9s+IdUSutFh\nDgW2L12Obgh12Y4sf1iKQXA0RbZ2k+XQIz8EKZa7vJQY0ciYXSgB/BV3a96xX3cx\nLIPL8Vam8Ytkopi3gsGA\n-----END CERTIFICATE-----" }, { "path": "pkg/util/azure/util.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage azure\n\nimport (\n\t\"crypto/tls\"\n\t\"crypto/x509\"\n\t\"encoding/base64\"\n\t\"fmt\"\n\t\"net\"\n\t\"net/http\"\n\t\"os\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/Azure/azure-sdk-for-go/sdk/azcore/cloud\"\n\t\"github.com/Azure/azure-sdk-for-go/sdk/azcore/policy\"\n\t\"github.com/joho/godotenv\"\n\t\"github.com/pkg/errors\"\n)\n\nconst (\n\t// the keys of Azure variables in credential\n\tCredentialKeySubscriptionID = \"AZURE_SUBSCRIPTION_ID\" // #nosec\n\tCredentialKeyResourceGroup = \"AZURE_RESOURCE_GROUP\" // #nosec\n\tCredentialKeyCloudName = \"AZURE_CLOUD_NAME\" // #nosec\n\tCredentialKeyStorageAccountAccessKey = \"AZURE_STORAGE_KEY\" // #nosec\n\tCredentialKeyAdditionallyAllowedTenants = \"AZURE_ADDITIONALLY_ALLOWED_TENANTS\" // #nosec\n\tCredentialKeyTenantID = \"AZURE_TENANT_ID\" // #nosec\n\tCredentialKeyClientID = \"AZURE_CLIENT_ID\" // #nosec\n\tCredentialKeyClientSecret = \"AZURE_CLIENT_SECRET\" // #nosec\n\tCredentialKeyClientCertificate = \"AZURE_CLIENT_CERTIFICATE\" // #nosec\n\tCredentialKeyClientCertificatePath = \"AZURE_CLIENT_CERTIFICATE_PATH\" // #nosec\n\tCredentialKeyClientCertificatePassword = \"AZURE_CLIENT_CERTIFICATE_PASSWORD\" // #nosec\n\tCredentialKeySendCertChain = \"AZURE_CLIENT_SEND_CERTIFICATE_CHAIN\" // #nosec\n\tCredentialKeyUsername = \"AZURE_USERNAME\" // #nosec\n\tCredentialKeyPassword = \"AZURE_PASSWORD\" // #nosec\n\n\tcredentialFile = \"credentialsFile\"\n)\n\n// LoadCredentials gets the credential file from config and loads it into a map\nfunc LoadCredentials(config map[string]string) (map[string]string, error) {\n\t// the default credential file\n\tcredFile := os.Getenv(\"AZURE_CREDENTIALS_FILE\")\n\n\t// use the credential file specified in the BSL spec if provided\n\tif config != nil && config[credentialFile] != \"\" {\n\t\tcredFile = config[credentialFile]\n\t}\n\n\tif len(credFile) == 0 {\n\t\treturn map[string]string{}, nil\n\t}\n\n\t// put the credential file content into a map\n\tcreds, err := godotenv.Read(credFile)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"failed to read credentials from file %s\", credFile)\n\t}\n\treturn creds, nil\n}\n\n// GetClientOptions returns the client options based on the BSL/VSL config and credentials\nfunc GetClientOptions(locationCfg, creds map[string]string) (policy.ClientOptions, error) {\n\toptions := policy.ClientOptions{}\n\n\tcloudCfg, err := getCloudConfiguration(locationCfg, creds)\n\tif err != nil {\n\t\treturn options, err\n\t}\n\toptions.Cloud = cloudCfg\n\n\tif locationCfg[\"caCert\"] != \"\" {\n\t\tcertPool, _ := x509.SystemCertPool()\n\t\tif certPool == nil {\n\t\t\tcertPool = x509.NewCertPool()\n\t\t}\n\t\tvar caCert []byte\n\t\t// As this function is used in both repository and plugin, the caCert isn't encoded\n\t\t// when passing to the plugin while is encoded when works with repository, use one\n\t\t// config item to distinguish these two cases\n\t\tif locationCfg[\"caCertEncoded\"] != \"\" {\n\t\t\tcaCert, err = base64.StdEncoding.DecodeString(locationCfg[\"caCert\"])\n\t\t\tif err != nil {\n\t\t\t\treturn options, err\n\t\t\t}\n\t\t} else {\n\t\t\tcaCert = []byte(locationCfg[\"caCert\"])\n\t\t}\n\n\t\tcertPool.AppendCertsFromPEM(caCert)\n\n\t\t// https://github.com/Azure/azure-sdk-for-go/blob/sdk/azcore/v1.6.1/sdk/azcore/runtime/transport_default_http_client.go#L19\n\t\ttransport := &http.Transport{\n\t\t\tProxy: http.ProxyFromEnvironment,\n\t\t\tDialContext: (&net.Dialer{\n\t\t\t\tTimeout: 30 * time.Second,\n\t\t\t\tKeepAlive: 30 * time.Second,\n\t\t\t}).DialContext,\n\t\t\tForceAttemptHTTP2: true,\n\t\t\tMaxIdleConns: 100,\n\t\t\tIdleConnTimeout: 90 * time.Second,\n\t\t\tTLSHandshakeTimeout: 10 * time.Second,\n\t\t\tExpectContinueTimeout: 1 * time.Second,\n\t\t\tTLSClientConfig: &tls.Config{\n\t\t\t\tMinVersion: tls.VersionTLS12,\n\t\t\t\tRootCAs: certPool,\n\t\t\t},\n\t\t}\n\t\toptions.Transport = &http.Client{\n\t\t\tTransport: transport,\n\t\t}\n\t}\n\n\treturn options, nil\n}\n\n// getCloudConfiguration based on the BSL/VSL config and credentials\nfunc getCloudConfiguration(locationCfg, creds map[string]string) (cloud.Configuration, error) {\n\tname := creds[CredentialKeyCloudName]\n\tactiveDirectoryAuthorityURI := locationCfg[BSLConfigActiveDirectoryAuthorityURI]\n\n\tvar cfg cloud.Configuration\n\tswitch strings.ToUpper(name) {\n\tcase \"\", \"AZURECLOUD\", \"AZUREPUBLICCLOUD\":\n\t\tcfg = cloud.AzurePublic\n\tcase \"AZURECHINACLOUD\":\n\t\tcfg = cloud.AzureChina\n\tcase \"AZUREUSGOVERNMENT\", \"AZUREUSGOVERNMENTCLOUD\":\n\t\tcfg = cloud.AzureGovernment\n\tdefault:\n\t\treturn cloud.Configuration{}, errors.New(fmt.Sprintf(\"unknown cloud: %s\", name))\n\t}\n\tif activeDirectoryAuthorityURI != \"\" {\n\t\tcfg.ActiveDirectoryAuthorityHost = activeDirectoryAuthorityURI\n\t}\n\treturn cfg, nil\n}\n\n// GetFromLocationConfigOrCredential returns the value of the specified key from BSL/VSL config or credentials\n// as some common configuration items can be set in BSL/VSL config or credential file(such as the subscription ID or resource group)\n// Reading from BSL/VSL config takes first.\nfunc GetFromLocationConfigOrCredential(cfg, creds map[string]string, cfgKey, credKey string) string {\n\tvalue := cfg[cfgKey]\n\tif value != \"\" {\n\t\treturn value\n\t}\n\treturn creds[credKey]\n}\n" }, { "path": "pkg/util/azure/util_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage azure\n\nimport (\n\t\"os\"\n\t\"path/filepath\"\n\t\"testing\"\n\n\t\"github.com/Azure/azure-sdk-for-go/sdk/azcore/cloud\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestLoadCredentials(t *testing.T) {\n\t// no credential file\n\tcredentials, err := LoadCredentials(nil)\n\trequire.NoError(t, err)\n\tassert.NotNil(t, credentials)\n\n\t// specified credential file in the config\n\tname := filepath.Join(os.TempDir(), \"credential\")\n\tfile, err := os.Create(name)\n\trequire.NoError(t, err)\n\tdefer file.Close()\n\tdefer os.Remove(name)\n\t_, err = file.WriteString(\"key: value\")\n\trequire.NoError(t, err)\n\n\tconfig := map[string]string{\n\t\t\"credentialsFile\": name,\n\t}\n\tcredentials, err = LoadCredentials(config)\n\trequire.NoError(t, err)\n\tassert.Equal(t, \"value\", credentials[\"key\"])\n\n\t// use the default path defined via env variable\n\tconfig = nil\n\tos.Setenv(\"AZURE_CREDENTIALS_FILE\", name)\n\tcredentials, err = LoadCredentials(config)\n\trequire.NoError(t, err)\n\tassert.Equal(t, \"value\", credentials[\"key\"])\n}\n\nfunc TestGetClientOptions(t *testing.T) {\n\t// invalid cloud name\n\tbslCfg := map[string]string{}\n\tcreds := map[string]string{\n\t\tCredentialKeyCloudName: \"invalid\",\n\t}\n\t_, err := GetClientOptions(bslCfg, creds)\n\trequire.Error(t, err)\n\n\t// specify caCert\n\tbslCfg = map[string]string{\n\t\tCredentialKeyCloudName: \"\",\n\t\t\"caCert\": \"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUZTakNDQXpLZ0F3SUJBZ0lVWmcxbzRpWld2bVh5ekJrQ0J6SGdiODZGemtFd0RRWUpLb1pJaHZjTkFRRUwKQlFBd1VqRUxNQWtHQTFVRUJoTUNRMDR4RERBS0JnTlZCQWdNQTFCRlN6RVJNQThHQTFVRUJ3d0lRbVZwSUVwcApibWN4RHpBTkJnTlZCQW9NQmxaTmQyRnlaVEVSTUE4R0ExVUVBd3dJU0dGeVltOXlRMEV3SGhjTk1qTXdPVEEyCk1ESXpOakUyV2hjTk1qUXdPVEExTURJek5qRTJXakJYTVFzd0NRWURWUVFHRXdKRFRqRU1NQW9HQTFVRUNBd0QKVUVWTE1SRXdEd1lEVlFRSERBaENaV2tnU21sdVp6RVBNQTBHQTFVRUNnd0dWazEzWVhKbE1SWXdGQVlEVlFRRApEQTFJWVhKaWIzSk5ZVzVoWjJWeU1JSUNJakFOQmdrcWhraUc5dzBCQVFFRkFBT0NBZzhBTUlJQ0NnS0NBZ0VBCnIrK1FHaHYvUnBDUTFIcncrMnYyQWNoaGhUVTVQL3hCd2RIWkZHWWJzMmxGbGtiL3oycEs2Y05ycFZmNUtmdjIKVUNpZEovMjFhZHc2SWNGZWxkSnFudU4rSlJaWXh5S0w0bzdRRGNVSk1sUTZJZk5kbEI0NUNwcGFBZVA4blVVTgo0YUwyV244b094L1pROTd2YmRXeERIR1FqZGR4N3p0Q09PaVZ0SEk4NS9Ka3kydTJnNmVhMklndmh1ZEVPZ3JtCjJzNU8zZlVtdHhSTEhwNnpDbURYZGZFUWg4ZFpndCs1d0RlazRWR2t4Zk81VG1tUHJ0LzBPTnVGYjJMWGVWV0sKeXkzVDFFTGNOSWhPSzQ1amhEejNnb2JhQzAwK0JTdzJMejVocXRxdUQ2RGxmME53TWtBQkt6d1dMZkROOXNrRApVazVYTmZNa2c0L3JhblRIWHlCNUNKSlk1akhORUtBQWlnM2NFSFNvejVjeGJqTE14VDhoMEk3MitEWldmUzFTCjU3Rm1SN2ZTNXk0QUcrU3Z1U3kvbktCUktJS2dQZ0t2Rjl6NktuL2ZPMTNsbk5LbHQwWU5mWFBFV2hZQytmMUoKTWpTOWc5eHBpYkZhM2Q0aHpOeWZhMWJHcUxtbkUwNVNpbWZNMVI1Z21Tamw1Z1FKQlltTHA3dWRLdjFDSUNRSwptQng2WG4vcnJEZHFiMndCRUNRSjBMbUo2SW5SaFZtT0s0WUdFeFRqZ1FRMldSWHYxMnhVK05GYWlZS3cxZkp0CkdaemFQeENxaG5JZXM5cGNPY0FjdmFHVngwSjlFYnRod0ltekdoTjBBREdCOVZaL1dFdHYvN0gwQ2xjOVlyT0gKNnRMb212b2pjQUZnN0xFbXZxeFFEOFFSTzlZZVdTTkgvV1REY1hVb3R5a0NBd0VBQWFNVE1CRXdEd1lEVlIwUgpCQWd3Qm9jRUNycFFxakFOQmdrcWhraUc5dzBCQVFzRkFBT0NBZ0VBZnRVdmR1UFMvajhSaWx4ME5aelhSeEY3Ck9HZW9qU2JaQ1ZvamNPdnRNTVlMaFkxdDE2Y2laY1VWMGF4Z2FUWkkvak9WMGJPTEl3dmEvcVB2Z1RmSWZqL0gKVzhiMlNTRVdIUzZPSFFaR1BYNy9zVFVwQzB6QVcva2haN1FWR1BoWEcyK0V1NjFaNE95ejZ5dTRPdi9MYjlMUQpmMU9zTXhwandkbmhxazFKaERxUkpZbGIvZ05TRGZnVlN0YmhHVzVhb3paUlBBMUtqVXVaT3QzR2xQR09Wd3ZLCnpUcFFMdGVTUHNibTJMcUl2ZEg4dlgzK1kwcHIzdEdtdnExbWtIWUhYQTlBZWtYRkVsRHc4dGtZVHdLaEFqblUKZEFjWTFkTis5ODNiMDI0L0JQUXZKQlRTVjd4blEyUnlrUmMrVGxIL3B5RlM1cEtVbUF0aU9qTElxL2ZEMmJVagorTzlxT1hjK0c1b0xEaXlXWDRXSG9XdkZZdTdva1gwT1dGcHFETXFOcHlLUkRzQ1FENXViMEVQaVlVS0hnWEhiCnV3UXVtK0pRRUREdzRXL1kzZktnMW9TWW1XOHJndFNPZmtRQlQ0UnlaTUg2SzN6cFp5dVVsbmJUV0NWeEcyYVoKWVo0T2JpbUFGbVlveGRYdktWdFU0YUdlTjRoaXBvb2dzaXVXKzZYQ3Bqa2pWZlZuUEY4elZVNlZ3anRQVkkzKwpxdWxRNWJLS3lKYng3bk9NNXFob2svSmk2N1pyZDhob3ZwclhhRUdvakNDTVI3MllPWGVuMlB3bVlZZWNkQ2pyCnErSDdHNUV3ZXBoRWxrN3RWRWY4RVV4OEc1Mk9SVEtZMkF1dlRGVlliUC8yaTROS1FlMWdEWWZrWnNzUk1MajEKK0JCQVVJcnFVMnRuUHhwZW4vMD0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=\",\n\t}\n\tcreds = map[string]string{}\n\toptions, err := GetClientOptions(bslCfg, creds)\n\trequire.NoError(t, err)\n\tassert.Equal(t, options.Cloud, cloud.AzurePublic)\n\tassert.NotNil(t, options.Transport)\n\n\t// doesn't specify caCert\n\tbslCfg = map[string]string{\n\t\tCredentialKeyCloudName: \"\",\n\t}\n\tcreds = map[string]string{}\n\toptions, err = GetClientOptions(bslCfg, creds)\n\trequire.NoError(t, err)\n\tassert.Equal(t, options.Cloud, cloud.AzurePublic)\n\tassert.Nil(t, options.Transport)\n}\n\nfunc Test_getCloudConfiguration(t *testing.T) {\n\tpublicCloudWithADURI := cloud.AzurePublic\n\tpublicCloudWithADURI.ActiveDirectoryAuthorityHost = \"https://example.com\"\n\tcases := []struct {\n\t\tname string\n\t\tbslCfg map[string]string\n\t\tcreds map[string]string\n\t\terr bool\n\t\texpected cloud.Configuration\n\t}{\n\t\t{\n\t\t\tname: \"invalid cloud name\",\n\t\t\tbslCfg: map[string]string{},\n\t\t\tcreds: map[string]string{\n\t\t\t\tCredentialKeyCloudName: \"invalid\",\n\t\t\t},\n\t\t\terr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"null cloud name\",\n\t\t\tbslCfg: map[string]string{},\n\t\t\tcreds: map[string]string{\n\t\t\t\tCredentialKeyCloudName: \"\",\n\t\t\t},\n\t\t\terr: false,\n\t\t\texpected: cloud.AzurePublic,\n\t\t},\n\t\t{\n\t\t\tname: \"azure public cloud\",\n\t\t\tbslCfg: map[string]string{},\n\t\t\tcreds: map[string]string{\n\t\t\t\tCredentialKeyCloudName: \"AZURECLOUD\",\n\t\t\t},\n\t\t\terr: false,\n\t\t\texpected: cloud.AzurePublic,\n\t\t},\n\t\t{\n\t\t\tname: \"azure public cloud\",\n\t\t\tbslCfg: map[string]string{},\n\t\t\tcreds: map[string]string{\n\t\t\t\tCredentialKeyCloudName: \"AZUREPUBLICCLOUD\",\n\t\t\t},\n\t\t\terr: false,\n\t\t\texpected: cloud.AzurePublic,\n\t\t},\n\t\t{\n\t\t\tname: \"azure public cloud\",\n\t\t\tbslCfg: map[string]string{},\n\t\t\tcreds: map[string]string{\n\t\t\t\tCredentialKeyCloudName: \"azurecloud\",\n\t\t\t},\n\t\t\terr: false,\n\t\t\texpected: cloud.AzurePublic,\n\t\t},\n\t\t{\n\t\t\tname: \"azure China cloud\",\n\t\t\tbslCfg: map[string]string{},\n\t\t\tcreds: map[string]string{\n\t\t\t\tCredentialKeyCloudName: \"AZURECHINACLOUD\",\n\t\t\t},\n\t\t\terr: false,\n\t\t\texpected: cloud.AzureChina,\n\t\t},\n\t\t{\n\t\t\tname: \"azure US government cloud\",\n\t\t\tbslCfg: map[string]string{},\n\t\t\tcreds: map[string]string{\n\t\t\t\tCredentialKeyCloudName: \"AZUREUSGOVERNMENT\",\n\t\t\t},\n\t\t\terr: false,\n\t\t\texpected: cloud.AzureGovernment,\n\t\t},\n\t\t{\n\t\t\tname: \"azure US government cloud\",\n\t\t\tbslCfg: map[string]string{},\n\t\t\tcreds: map[string]string{\n\t\t\t\tCredentialKeyCloudName: \"AZUREUSGOVERNMENTCLOUD\",\n\t\t\t},\n\t\t\terr: false,\n\t\t\texpected: cloud.AzureGovernment,\n\t\t},\n\t\t{\n\t\t\tname: \"AD authority URI provided\",\n\t\t\tbslCfg: map[string]string{\n\t\t\t\tBSLConfigActiveDirectoryAuthorityURI: \"https://example.com\",\n\t\t\t},\n\t\t\tcreds: map[string]string{\n\t\t\t\tCredentialKeyCloudName: \"\",\n\t\t\t},\n\t\t\terr: false,\n\t\t\texpected: publicCloudWithADURI,\n\t\t},\n\t}\n\n\tfor _, c := range cases {\n\t\tt.Run(c.name, func(t *testing.T) {\n\t\t\tcfg, err := getCloudConfiguration(c.bslCfg, c.creds)\n\t\t\trequire.Equal(t, c.err, err != nil)\n\t\t\tif !c.err {\n\t\t\t\tassert.Equal(t, c.expected, cfg)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetFromLocationConfigOrCredential(t *testing.T) {\n\t// from cfg\n\tcfg := map[string]string{\n\t\t\"cfgkey\": \"value\",\n\t}\n\tcreds := map[string]string{}\n\tcfgKey, credKey := \"cfgkey\", \"credkey\"\n\tstr := GetFromLocationConfigOrCredential(cfg, creds, cfgKey, credKey)\n\tassert.Equal(t, \"value\", str)\n\n\t// from cred\n\tcfg = map[string]string{}\n\tcreds = map[string]string{\n\t\t\"credkey\": \"value\",\n\t}\n\tstr = GetFromLocationConfigOrCredential(cfg, creds, cfgKey, credKey)\n\tassert.Equal(t, \"value\", str)\n}\n" }, { "path": "pkg/util/boolptr/boolptr.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage boolptr\n\n// IsSetToTrue returns true if and only if the bool pointer is non-nil and set to true.\nfunc IsSetToTrue(b *bool) bool {\n\treturn b != nil && *b\n}\n\n// IsSetToFalse returns true if and only if the bool pointer is non-nil and set to false.\nfunc IsSetToFalse(b *bool) bool {\n\treturn b != nil && !*b\n}\n\n// True returns a *bool whose underlying value is true.\nfunc True() *bool {\n\tt := true\n\treturn &t\n}\n\n// False returns a *bool whose underlying value is false.\nfunc False() *bool {\n\tt := false\n\treturn &t\n}\n" }, { "path": "pkg/util/collections/includes_excludes.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage collections\n\nimport (\n\t\"strings\"\n\n\t\"github.com/vmware-tanzu/velero/internal/resourcepolicies\"\n\n\t\"github.com/gobwas/glob\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/api/validation\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/discovery\"\n\t\"github.com/vmware-tanzu/velero/pkg/kuberesource\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/wildcard\"\n)\n\ntype globStringSet struct {\n\tsets.String\n}\n\nfunc newGlobStringSet() globStringSet {\n\treturn globStringSet{sets.NewString()}\n}\n\nfunc (gss globStringSet) match(match string) bool {\n\tfor _, item := range gss.List() {\n\t\tg, err := glob.Compile(item)\n\t\tif err != nil {\n\t\t\treturn false\n\t\t}\n\t\tif g.Match(match) {\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n\n// NamespaceIncludesExcludes adds some features to IncludesExcludes\n// to handle namespace-specific functionality. In particular, it\n// provides a way to list all namespaces included in order to determine\n// overlap between backups, and it will be expanded in the future to\n// handle namespace wildcard values\ntype NamespaceIncludesExcludes struct {\n\tactiveNamespaces []string\n\tincludesExcludes *IncludesExcludes\n\twildcardExpanded bool\n\twildcardResult []string\n}\n\nfunc NewNamespaceIncludesExcludes() *NamespaceIncludesExcludes {\n\treturn &NamespaceIncludesExcludes{\n\t\tactiveNamespaces: []string{},\n\t\tincludesExcludes: NewIncludesExcludes(),\n\t}\n}\n\nfunc (nie *NamespaceIncludesExcludes) ActiveNamespaces(activeNamespaces []string) *NamespaceIncludesExcludes {\n\tnie.activeNamespaces = activeNamespaces\n\treturn nie\n}\n\nfunc (nie *NamespaceIncludesExcludes) IsWildcardExpanded() bool {\n\treturn nie.wildcardExpanded\n}\n\n// Includes adds items to the includes list. '*' is a wildcard\n// value meaning \"include everything\".\nfunc (nie *NamespaceIncludesExcludes) Includes(includes ...string) *NamespaceIncludesExcludes {\n\tnie.includesExcludes.Includes(includes...)\n\treturn nie\n}\n\n// GetIncludes returns the items in the includes list\nfunc (nie *NamespaceIncludesExcludes) GetIncludes() []string {\n\treturn nie.includesExcludes.GetIncludes()\n}\n\nfunc (nie *NamespaceIncludesExcludes) GetExcludes() []string {\n\treturn nie.includesExcludes.GetExcludes()\n}\n\n// SetIncludes sets the includes list to the given list\nfunc (nie *NamespaceIncludesExcludes) SetIncludes(includes []string) *NamespaceIncludesExcludes {\n\tnie.includesExcludes.includes = newGlobStringSet()\n\tnie.includesExcludes.includes.Insert(includes...)\n\treturn nie\n}\n\n// SetExcludes sets the excludes list to the given list\nfunc (nie *NamespaceIncludesExcludes) SetExcludes(excludes []string) *NamespaceIncludesExcludes {\n\tnie.includesExcludes.excludes = newGlobStringSet()\n\tnie.includesExcludes.excludes.Insert(excludes...)\n\treturn nie\n}\n\n// IncludesString returns a string containing all of the includes, separated by commas, or * if the\n// list is empty.\nfunc (nie *NamespaceIncludesExcludes) IncludesString() string {\n\treturn nie.includesExcludes.IncludesString()\n}\n\n// Excludes adds items to the includes list. '*' is a wildcard\n// value meaning \"include everything\".\nfunc (nie *NamespaceIncludesExcludes) Excludes(excludes ...string) *NamespaceIncludesExcludes {\n\tnie.includesExcludes.Excludes(excludes...)\n\treturn nie\n}\n\n// IncludesString returns a string containing all of the excludes, separated by commas, or * if the\n// list is empty.\nfunc (nie *NamespaceIncludesExcludes) ExcludesString() string {\n\treturn nie.includesExcludes.ExcludesString()\n}\n\n// ShouldInclude returns whether the specified item should be\n// included or not. Everything in the includes list except those\n// items in the excludes list should be included.\nfunc (nie *NamespaceIncludesExcludes) ShouldInclude(s string) bool {\n\t// Special case: if wildcard expansion occurred and resulted in an empty includes list,\n\t// it means the wildcard pattern matched nothing, so we should include nothing.\n\t// This differs from the default behavior where an empty includes list means \"include everything\".\n\tif nie.wildcardExpanded && nie.includesExcludes.includes.Len() == 0 {\n\t\treturn false\n\t}\n\treturn nie.includesExcludes.ShouldInclude(s)\n}\n\n// IncludeEverything returns true if the includes list is empty or '*'\n// and the excludes list is empty, or false otherwise.\nfunc (nie *NamespaceIncludesExcludes) IncludeEverything() bool {\n\treturn nie.includesExcludes.IncludeEverything()\n}\n\n// Attempts to expand wildcard patterns, if any, in the includes and excludes lists.\nfunc (nie *NamespaceIncludesExcludes) ExpandIncludesExcludes() error {\n\tincludes := nie.GetIncludes()\n\texcludes := nie.GetExcludes()\n\n\tif wildcard.ShouldExpandWildcards(includes, excludes) {\n\t\texpandedIncludes, expandedExcludes, err := wildcard.ExpandWildcards(\n\t\t\tnie.activeNamespaces, includes, excludes)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tnie.SetIncludes(expandedIncludes)\n\t\tnie.SetExcludes(expandedExcludes)\n\t\tnie.wildcardExpanded = true\n\t}\n\n\treturn nil\n}\n\n// ResolveNamespaceList returns a list of all namespaces which will be backed up.\n// The second return value indicates whether wildcard expansion was performed.\nfunc (nie *NamespaceIncludesExcludes) ResolveNamespaceList() ([]string, error) {\n\t// Check if this is being called by non-backup processing e.g. backup queue controller\n\tif !nie.wildcardExpanded {\n\t\terr := nie.ExpandIncludesExcludes()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\toutNamespaces := []string{}\n\tfor _, ns := range nie.activeNamespaces {\n\t\tif nie.ShouldInclude(ns) {\n\t\t\toutNamespaces = append(outNamespaces, ns)\n\t\t}\n\t}\n\tnie.wildcardResult = outNamespaces\n\treturn nie.wildcardResult, nil\n}\n\n// IncludesExcludes is a type that manages lists of included\n// and excluded items. The logic implemented is that everything\n// in the included list except those items in the excluded list\n// should be included. '*' in the includes list means \"include\n// everything\", but it is not valid in the exclude list.\ntype IncludesExcludes struct {\n\tincludes globStringSet\n\texcludes globStringSet\n}\n\nfunc NewIncludesExcludes() *IncludesExcludes {\n\treturn &IncludesExcludes{\n\t\tincludes: newGlobStringSet(),\n\t\texcludes: newGlobStringSet(),\n\t}\n}\n\n// Includes adds items to the includes list. '*' is a wildcard\n// value meaning \"include everything\".\nfunc (ie *IncludesExcludes) Includes(includes ...string) *IncludesExcludes {\n\tie.includes.Insert(includes...)\n\treturn ie\n}\n\n// GetIncludes returns the items in the includes list\nfunc (ie *IncludesExcludes) GetIncludes() []string {\n\treturn ie.includes.List()\n}\n\n// Excludes adds items to the excludes list\nfunc (ie *IncludesExcludes) Excludes(excludes ...string) *IncludesExcludes {\n\tie.excludes.Insert(excludes...)\n\treturn ie\n}\n\n// GetExcludes returns the items in the excludes list\nfunc (ie *IncludesExcludes) GetExcludes() []string {\n\treturn ie.excludes.List()\n}\n\n// ShouldInclude returns whether the specified item should be\n// included or not. Everything in the includes list except those\n// items in the excludes list should be included.\nfunc (ie *IncludesExcludes) ShouldInclude(s string) bool {\n\tif ie.excludes.match(s) {\n\t\treturn false\n\t}\n\n\t// len=0 means include everything\n\treturn ie.includes.Len() == 0 || ie.includes.Has(\"*\") || ie.includes.match(s)\n}\n\n// IncludesString returns a string containing all of the includes, separated by commas, or * if the\n// list is empty.\nfunc (ie *IncludesExcludes) IncludesString() string {\n\treturn asString(ie.GetIncludes(), \"*\")\n}\n\n// ExcludesString returns a string containing all of the excludes, separated by commas, or if the\n// list is empty.\nfunc (ie *IncludesExcludes) ExcludesString() string {\n\treturn asString(ie.GetExcludes(), \"\")\n}\n\n// IncludeEverything returns true if the includes list is empty or '*'\n// and the excludes list is empty, or false otherwise.\nfunc (ie *IncludesExcludes) IncludeEverything() bool {\n\treturn ie.excludes.Len() == 0 && (ie.includes.Len() == 0 || (ie.includes.Len() == 1 && ie.includes.Has(\"*\")))\n}\n\n// GetResourceIncludesExcludes takes the lists of resources to include and exclude, uses the\n// discovery helper to resolve them to fully-qualified group-resource names, and returns an\n// IncludesExcludes list.\nfunc GetResourceIncludesExcludes(helper discovery.Helper, includes, excludes []string) *IncludesExcludes {\n\tresources := generateIncludesExcludes(\n\t\tincludes,\n\t\texcludes,\n\t\tfunc(item string) string {\n\t\t\tgvr, _, err := helper.ResourceFor(schema.ParseGroupResource(item).WithVersion(\"\"))\n\t\t\tif err != nil {\n\t\t\t\t// If we can't resolve it, return it as-is. This prevents the generated\n\t\t\t\t// includes-excludes list from including *everything*, if none of the includes\n\t\t\t\t// can be resolved. ref. https://github.com/vmware-tanzu/velero/issues/2461\n\t\t\t\treturn item\n\t\t\t}\n\n\t\t\tgr := gvr.GroupResource()\n\t\t\treturn gr.String()\n\t\t},\n\t)\n\n\treturn resources\n}\n\nfunc asString(in []string, empty string) string {\n\tif len(in) == 0 {\n\t\treturn empty\n\t}\n\treturn strings.Join(in, \", \")\n}\n\n// IncludesExcludesInterface is used as polymorphic IncludesExcludes for Global and scope\n// resources Include/Exclude.\ntype IncludesExcludesInterface interface {\n\t// ShouldInclude checks whether the type name passed in by parameter should be included.\n\t// typeName should be k8s.io/apimachinery/pkg/runtime/schema GroupResource's String() result.\n\tShouldInclude(typeName string) bool\n\n\t// ShouldExclude checks whether the type name passed in by parameter should be excluded.\n\t// typeName should be k8s.io/apimachinery/pkg/runtime/schema GroupResource's String() result.\n\tShouldExclude(typeName string) bool\n}\n\ntype GlobalIncludesExcludes struct {\n\tresourceFilter IncludesExcludes\n\tincludeClusterResources *bool\n\tnamespaceFilter NamespaceIncludesExcludes\n\n\thelper discovery.Helper\n\tlogger logrus.FieldLogger\n}\n\n// ShouldInclude returns whether the specified item should be\n// included or not. Everything in the includes list except those\n// items in the excludes list should be included.\n// It has some exceptional cases. When IncludeClusterResources is set to false,\n// no need to check the filter, all cluster resources are excluded.\nfunc (ie *GlobalIncludesExcludes) ShouldInclude(typeName string) bool {\n\t_, resource, err := ie.helper.ResourceFor(schema.ParseGroupResource(typeName).WithVersion(\"\"))\n\tif err != nil {\n\t\tie.logger.Errorf(\"fail to get resource %s. %s\", typeName, err.Error())\n\t\treturn false\n\t}\n\n\tif !resource.Namespaced && boolptr.IsSetToFalse(ie.includeClusterResources) {\n\t\tie.logger.Info(\"Skipping resource %s, because it's cluster-scoped, and IncludeClusterResources is set to false.\", typeName)\n\t\treturn false\n\t}\n\n\t// when IncludeClusterResources == nil (auto), only directly\n\t// back up cluster-scoped resources if we're doing a full-cluster\n\t// (all namespaces and all namespace scope types) backup. Note that in the case of a subset of\n\t// namespaces being backed up, some related cluster-scoped resources\n\t// may still be backed up if triggered by a custom action (e.g. PVC->PV).\n\t// If we're processing namespaces themselves, we will not skip here, they may be\n\t// filtered out later.\n\tif typeName != kuberesource.Namespaces.String() && !resource.Namespaced &&\n\t\tie.includeClusterResources == nil && !ie.namespaceFilter.IncludeEverything() {\n\t\tie.logger.Infof(\"Skipping resource %s, because it's cluster-scoped and only specific namespaces or namespace scope types are included in the backup.\", typeName)\n\t\treturn false\n\t}\n\n\treturn ie.resourceFilter.ShouldInclude(typeName)\n}\n\n// ShouldExclude returns whether the resource type should be excluded or not.\nfunc (ie *GlobalIncludesExcludes) ShouldExclude(typeName string) bool {\n\t// if the type name is specified in excluded list, it's excluded.\n\tif ie.resourceFilter.excludes.match(typeName) {\n\t\treturn true\n\t}\n\n\t_, resource, err := ie.helper.ResourceFor(schema.ParseGroupResource(typeName).WithVersion(\"\"))\n\tif err != nil {\n\t\tie.logger.Errorf(\"fail to get resource %s. %s\", typeName, err.Error())\n\t\treturn true\n\t}\n\n\t// the resource type is cluster scope\n\tif !resource.Namespaced {\n\t\t// if includeClusterResources is set to false, cluster resource should be excluded.\n\t\tif boolptr.IsSetToFalse(ie.includeClusterResources) {\n\t\t\treturn true\n\t\t}\n\t\t// if includeClusterResources is set to nil, check whether it's included by resource\n\t\t// filter.\n\t\tif ie.includeClusterResources == nil && !ie.resourceFilter.ShouldInclude(typeName) {\n\t\t\treturn true\n\t\t}\n\t}\n\n\treturn false\n}\n\nfunc GetGlobalResourceIncludesExcludes(helper discovery.Helper, logger logrus.FieldLogger, includes, excludes []string, includeClusterResources *bool, nsIncludesExcludes NamespaceIncludesExcludes) *GlobalIncludesExcludes {\n\tret := &GlobalIncludesExcludes{\n\t\tresourceFilter: *GetResourceIncludesExcludes(helper, includes, excludes),\n\t\tincludeClusterResources: includeClusterResources,\n\t\tnamespaceFilter: nsIncludesExcludes,\n\t\thelper: helper,\n\t\tlogger: logger,\n\t}\n\n\tlogger.Infof(\"Including resources: %s\", ret.resourceFilter.IncludesString())\n\tlogger.Infof(\"Excluding resources: %s\", ret.resourceFilter.ExcludesString())\n\treturn ret\n}\n\ntype ScopeIncludesExcludes struct {\n\tnamespaceScopedResourceFilter IncludesExcludes // namespace-scoped resource filter\n\tclusterScopedResourceFilter IncludesExcludes // cluster-scoped resource filter\n\tnamespaceFilter NamespaceIncludesExcludes // namespace filter\n\n\thelper discovery.Helper\n\tlogger logrus.FieldLogger\n}\n\n// ShouldInclude returns whether the specified resource should be included or not.\n// The function will check whether the resource is namespace-scoped resource first.\n// For namespace-scoped resource, except resources listed in excludes, other things should be included.\n// For cluster-scoped resource, except resources listed in excludes, only include the resource specified by the included.\n// It also has some exceptional checks. For namespace, as long as it's not excluded, it is involved.\n// If all namespace-scoped resources are included, all cluster-scoped resource are returned to get a full backup.\nfunc (ie *ScopeIncludesExcludes) ShouldInclude(typeName string) bool {\n\t_, resource, err := ie.helper.ResourceFor(schema.ParseGroupResource(typeName).WithVersion(\"\"))\n\tif err != nil {\n\t\tie.logger.Errorf(\"fail to get resource %s. %s\", typeName, err.Error())\n\t\treturn false\n\t}\n\n\tif resource.Namespaced {\n\t\tif ie.namespaceScopedResourceFilter.excludes.Has(\"*\") || ie.namespaceScopedResourceFilter.excludes.match(typeName) {\n\t\t\treturn false\n\t\t}\n\n\t\t// len=0 means include everything\n\t\treturn ie.namespaceScopedResourceFilter.includes.Len() == 0 || ie.namespaceScopedResourceFilter.includes.Has(\"*\") || ie.namespaceScopedResourceFilter.includes.match(typeName)\n\t}\n\n\tif ie.clusterScopedResourceFilter.excludes.Has(\"*\") || ie.clusterScopedResourceFilter.excludes.match(typeName) {\n\t\treturn false\n\t}\n\n\t// when IncludedClusterScopedResources and ExcludedClusterScopedResources are not specified,\n\t// only directly back up cluster-scoped resources if we're doing a full-cluster\n\t// (all namespaces and all namespace-scoped types) backup.\n\tif len(ie.clusterScopedResourceFilter.includes.List()) == 0 &&\n\t\tlen(ie.clusterScopedResourceFilter.excludes.List()) == 0 &&\n\t\tie.namespaceFilter.IncludeEverything() &&\n\t\tie.namespaceScopedResourceFilter.IncludeEverything() {\n\t\treturn true\n\t}\n\n\t// Also include namespace resource by default.\n\treturn ie.clusterScopedResourceFilter.includes.Has(\"*\") || ie.clusterScopedResourceFilter.includes.match(typeName) || typeName == kuberesource.Namespaces.String()\n}\n\n// ShouldExclude returns whether the resource type should be excluded or not.\n// For ScopeIncludesExcludes, if the resource type is specified in the exclude\n// list, it should be excluded.\nfunc (ie *ScopeIncludesExcludes) ShouldExclude(typeName string) bool {\n\t_, resource, err := ie.helper.ResourceFor(schema.ParseGroupResource(typeName).WithVersion(\"\"))\n\tif err != nil {\n\t\tie.logger.Errorf(\"fail to get resource %s. %s\", typeName, err.Error())\n\t\treturn true\n\t}\n\n\tif resource.Namespaced {\n\t\tif ie.namespaceScopedResourceFilter.excludes.match(typeName) {\n\t\t\treturn true\n\t\t}\n\t} else {\n\t\tif ie.clusterScopedResourceFilter.excludes.match(typeName) {\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n\nfunc (ie *ScopeIncludesExcludes) CombineWithPolicy(policy *resourcepolicies.IncludeExcludePolicy) {\n\tif policy == nil {\n\t\treturn\n\t}\n\tmapFunc := scopeResourceMapFunc(ie.helper)\n\tfor _, item := range policy.ExcludedNamespaceScopedResources {\n\t\tresolvedItem := mapFunc(item, true)\n\t\tif resolvedItem == \"\" {\n\t\t\tcontinue\n\t\t}\n\t\t// The existing includeExcludes in the struct has higher priority, therefore, we should only add the item to the filter\n\t\t// when the struct does not include this item and this item is not yet in the excludes filter.\n\t\tif !ie.namespaceScopedResourceFilter.includes.match(resolvedItem) &&\n\t\t\t!ie.namespaceScopedResourceFilter.excludes.match(resolvedItem) {\n\t\t\tie.namespaceScopedResourceFilter.Excludes(resolvedItem)\n\t\t}\n\t}\n\tfor _, item := range policy.IncludedNamespaceScopedResources {\n\t\tresolvedItem := mapFunc(item, true)\n\t\tif resolvedItem == \"\" {\n\t\t\tcontinue\n\t\t}\n\t\t// The existing includeExcludes in the struct has higher priority, therefore, we should only add the item to the filter\n\t\t// when the struct does not exclude this item and this item is not yet in the includes filter.\n\t\tif !ie.namespaceScopedResourceFilter.includes.match(resolvedItem) &&\n\t\t\t!ie.namespaceScopedResourceFilter.excludes.match(resolvedItem) {\n\t\t\tie.namespaceScopedResourceFilter.Includes(resolvedItem)\n\t\t}\n\t}\n\tfor _, item := range policy.ExcludedClusterScopedResources {\n\t\tresolvedItem := mapFunc(item, false)\n\t\tif resolvedItem == \"\" {\n\t\t\tcontinue\n\t\t}\n\t\tif !ie.clusterScopedResourceFilter.includes.match(resolvedItem) &&\n\t\t\t!ie.clusterScopedResourceFilter.excludes.match(resolvedItem) {\n\t\t\t// The existing includeExcludes in the struct has higher priority, therefore, we should only add the item to the filter\n\t\t\t// when the struct does not exclude this item and this item is not yet in the includes filter.\n\t\t\tie.clusterScopedResourceFilter.Excludes(resolvedItem)\n\t\t}\n\t}\n\tfor _, item := range policy.IncludedClusterScopedResources {\n\t\tresolvedItem := mapFunc(item, false)\n\t\tif resolvedItem == \"\" {\n\t\t\tcontinue\n\t\t}\n\t\tif !ie.clusterScopedResourceFilter.includes.match(resolvedItem) &&\n\t\t\t!ie.clusterScopedResourceFilter.excludes.match(resolvedItem) {\n\t\t\t// The existing includeExcludes in the struct has higher priority, therefore, we should only add the item to the filter\n\t\t\t// when the struct does not exclude this item and this item is not yet in the includes filter.\n\t\t\tie.clusterScopedResourceFilter.Includes(resolvedItem)\n\t\t}\n\t}\n\tie.logger.Infof(\"Scoped resource includes/excludes after combining with resource policy\")\n\tie.logger.Infof(\"Including namespace-scoped resources: %s\", ie.namespaceScopedResourceFilter.IncludesString())\n\tie.logger.Infof(\"Excluding namespace-scoped resources: %s\", ie.namespaceScopedResourceFilter.ExcludesString())\n\tie.logger.Infof(\"Including cluster-scoped resources: %s\", ie.clusterScopedResourceFilter.GetIncludes())\n\tie.logger.Infof(\"Excluding cluster-scoped resources: %s\", ie.clusterScopedResourceFilter.ExcludesString())\n}\n\nfunc newScopeIncludesExcludes(nsIncludesExcludes NamespaceIncludesExcludes, helper discovery.Helper, logger logrus.FieldLogger) *ScopeIncludesExcludes {\n\tret := &ScopeIncludesExcludes{\n\t\tnamespaceScopedResourceFilter: IncludesExcludes{\n\t\t\tincludes: newGlobStringSet(),\n\t\t\texcludes: newGlobStringSet(),\n\t\t},\n\t\tclusterScopedResourceFilter: IncludesExcludes{\n\t\t\tincludes: newGlobStringSet(),\n\t\t\texcludes: newGlobStringSet(),\n\t\t},\n\t\tnamespaceFilter: nsIncludesExcludes,\n\t\thelper: helper,\n\t\tlogger: logger,\n\t}\n\n\treturn ret\n}\n\n// GetScopeResourceIncludesExcludes function is similar with GetResourceIncludesExcludes,\n// but it's used for scoped Includes/Excludes, and can handle both cluster-scoped and namespace-scoped resources.\nfunc GetScopeResourceIncludesExcludes(helper discovery.Helper, logger logrus.FieldLogger, namespaceIncludes, namespaceExcludes, clusterIncludes, clusterExcludes []string, nsIncludesExcludes NamespaceIncludesExcludes) *ScopeIncludesExcludes {\n\tret := generateScopedIncludesExcludes(\n\t\tnamespaceIncludes,\n\t\tnamespaceExcludes,\n\t\tclusterIncludes,\n\t\tclusterExcludes,\n\t\tscopeResourceMapFunc(helper),\n\t\tnsIncludesExcludes,\n\t\thelper,\n\t\tlogger,\n\t)\n\tlogger.Infof(\"Scoped resource includes/excludes after initialization\")\n\tlogger.Infof(\"Including namespace-scoped resources: %s\", ret.namespaceScopedResourceFilter.IncludesString())\n\tlogger.Infof(\"Excluding namespace-scoped resources: %s\", ret.namespaceScopedResourceFilter.ExcludesString())\n\tlogger.Infof(\"Including cluster-scoped resources: %s\", ret.clusterScopedResourceFilter.GetIncludes())\n\tlogger.Infof(\"Excluding cluster-scoped resources: %s\", ret.clusterScopedResourceFilter.ExcludesString())\n\n\treturn ret\n}\n\nfunc scopeResourceMapFunc(helper discovery.Helper) func(string, bool) string {\n\treturn func(item string, namespaced bool) string {\n\t\tgvr, resource, err := helper.ResourceFor(schema.ParseGroupResource(item).WithVersion(\"\"))\n\t\tif err != nil {\n\t\t\treturn item\n\t\t}\n\t\tif resource.Namespaced != namespaced {\n\t\t\treturn \"\"\n\t\t}\n\n\t\tgr := gvr.GroupResource()\n\t\treturn gr.String()\n\t}\n}\n\n// ValidateIncludesExcludes checks provided lists of included and excluded\n// items to ensure they are a valid set of IncludesExcludes data.\nfunc ValidateIncludesExcludes(includesList, excludesList []string) []error {\n\t// TODO we should not allow an IncludesExcludes object to be created that\n\t// does not meet these criteria. Do a more significant refactoring to embed\n\t// this logic in object creation/modification.\n\n\tvar errs []error\n\n\tincludes := sets.NewString(includesList...)\n\texcludes := sets.NewString(excludesList...)\n\n\tif includes.Len() > 1 && includes.Has(\"*\") {\n\t\terrs = append(errs, errors.New(\"includes list must either contain '*' only, or a non-empty list of items\"))\n\t}\n\n\tif excludes.Has(\"*\") {\n\t\terrs = append(errs, errors.New(\"excludes list cannot contain '*'\"))\n\t}\n\n\tfor _, itm := range excludes.List() {\n\t\tif includes.Has(itm) {\n\t\t\terrs = append(errs, errors.Errorf(\"excludes list cannot contain an item in the includes list: %v\", itm))\n\t\t}\n\t}\n\n\treturn errs\n}\n\n// ValidateNamespaceIncludesExcludes checks provided lists of included and\n// excluded namespaces to ensure they are a valid set of IncludesExcludes data.\nfunc ValidateNamespaceIncludesExcludes(includesList, excludesList []string) []error {\n\terrs := ValidateIncludesExcludes(includesList, excludesList)\n\n\tincludes := sets.NewString(includesList...)\n\texcludes := sets.NewString(excludesList...)\n\n\tfor _, itm := range includes.List() {\n\t\tif nsErrs := validateNamespaceName(itm); nsErrs != nil {\n\t\t\terrs = append(errs, nsErrs...)\n\t\t}\n\t}\n\tfor _, itm := range excludes.List() {\n\t\tif nsErrs := validateNamespaceName(itm); nsErrs != nil {\n\t\t\terrs = append(errs, nsErrs...)\n\t\t}\n\t}\n\n\treturn errs\n}\n\n// ValidateScopedIncludesExcludes checks provided lists of namespace-scoped or cluster-scoped\n// included and excluded items to ensure they are a valid set of IncludesExcludes data.\nfunc ValidateScopedIncludesExcludes(includesList, excludesList []string) []error {\n\tvar errs []error\n\n\tincludes := sets.NewString(includesList...)\n\texcludes := sets.NewString(excludesList...)\n\n\tif includes.Len() > 1 && includes.Has(\"*\") {\n\t\terrs = append(errs, errors.New(\"includes list must either contain '*' only, or a non-empty list of items\"))\n\t}\n\n\tif excludes.Len() > 1 && excludes.Has(\"*\") {\n\t\terrs = append(errs, errors.New(\"excludes list must either contain '*' only, or a non-empty list of items\"))\n\t}\n\n\tif includes.Len() > 0 && excludes.Has(\"*\") {\n\t\terrs = append(errs, errors.New(\"when exclude is '*', include cannot have value\"))\n\t}\n\n\tfor _, itm := range excludes.List() {\n\t\tif includes.Has(itm) {\n\t\t\terrs = append(errs, errors.Errorf(\"excludes list cannot contain an item in the includes list: %v\", itm))\n\t\t}\n\t}\n\n\treturn errs\n}\n\nfunc validateNamespaceName(ns string) []error {\n\tvar errs []error\n\n\t// Velero interprets empty string as \"no namespace\", so allow it even though\n\t// it is not a valid Kubernetes name.\n\tif ns == \"\" {\n\t\treturn nil\n\t}\n\n\t// Validate the namespace name to ensure it is a valid wildcard pattern\n\tif err := wildcard.ValidateNamespaceName(ns); err != nil {\n\t\treturn []error{err}\n\t}\n\n\t// Kubernetes does not allow wildcard characters in namespaces but Velero uses them\n\t// for glob patterns. Replace wildcard characters with valid characters to pass\n\t// Kubernetes validation.\n\ttmpNamespace := ns\n\n\t// Replace glob wildcard characters with valid alphanumeric characters\n\t// Note: Validation of wildcard patterns is handled by the wildcard package.\n\ttmpNamespace = strings.ReplaceAll(tmpNamespace, \"*\", \"x\") // matches any sequence\n\ttmpNamespace = strings.ReplaceAll(tmpNamespace, \"?\", \"x\") // matches single character\n\ttmpNamespace = strings.ReplaceAll(tmpNamespace, \"[\", \"x\") // character class start\n\ttmpNamespace = strings.ReplaceAll(tmpNamespace, \"]\", \"x\") // character class end\n\n\tif errMsgs := validation.ValidateNamespaceName(tmpNamespace, false); errMsgs != nil {\n\t\tfor _, msg := range errMsgs {\n\t\t\terrs = append(errs, errors.Errorf(\"invalid namespace %q: %s\", ns, msg))\n\t\t}\n\t}\n\n\treturn errs\n}\n\n// generateIncludesExcludes constructs an IncludesExcludes struct by taking the provided\n// include/exclude slices, applying the specified mapping function to each item in them,\n// and adding the output of the function to the new struct. If the mapping function returns\n// an empty string for an item, it is omitted from the result.\nfunc generateIncludesExcludes(includes, excludes []string, mapFunc func(string) string) *IncludesExcludes {\n\tres := NewIncludesExcludes()\n\n\tfor _, item := range includes {\n\t\tif item == \"*\" {\n\t\t\tres.Includes(item)\n\t\t\tcontinue\n\t\t}\n\n\t\tkey := mapFunc(item)\n\t\tif key == \"\" {\n\t\t\tcontinue\n\t\t}\n\t\tres.Includes(key)\n\t}\n\n\tfor _, item := range excludes {\n\t\t// wildcards are invalid for excludes,\n\t\t// so ignore them.\n\t\tif item == \"*\" {\n\t\t\tcontinue\n\t\t}\n\n\t\tkey := mapFunc(item)\n\t\tif key == \"\" {\n\t\t\tcontinue\n\t\t}\n\t\tres.Excludes(key)\n\t}\n\n\treturn res\n}\n\n// generateScopedIncludesExcludes function is similar with generateIncludesExcludes,\n// but it's used for scoped Includes/Excludes.\nfunc generateScopedIncludesExcludes(namespacedIncludes, namespacedExcludes, clusterIncludes, clusterExcludes []string, mapFunc func(string, bool) string, nsIncludesExcludes NamespaceIncludesExcludes, helper discovery.Helper, logger logrus.FieldLogger) *ScopeIncludesExcludes {\n\tres := newScopeIncludesExcludes(nsIncludesExcludes, helper, logger)\n\n\tgenerateFilter(res.namespaceScopedResourceFilter.includes, namespacedIncludes, mapFunc, true)\n\tgenerateFilter(res.namespaceScopedResourceFilter.excludes, namespacedExcludes, mapFunc, true)\n\tgenerateFilter(res.clusterScopedResourceFilter.includes, clusterIncludes, mapFunc, false)\n\tgenerateFilter(res.clusterScopedResourceFilter.excludes, clusterExcludes, mapFunc, false)\n\n\treturn res\n}\n\nfunc generateFilter(filter globStringSet, resources []string, mapFunc func(string, bool) string, namespaced bool) {\n\tfor _, item := range resources {\n\t\tif item == \"*\" {\n\t\t\tfilter.Insert(item)\n\t\t\tcontinue\n\t\t}\n\n\t\tkey := mapFunc(item, namespaced)\n\t\tif key == \"\" {\n\t\t\tcontinue\n\t\t}\n\t\tfilter.Insert(key)\n\t}\n}\n\n// UseOldResourceFilters checks whether to use old resource filters (IncludeClusterResources,\n// IncludedResources and ExcludedResources), depending the backup's filters setting.\n// New filters are IncludedClusterScopedResources, ExcludedClusterScopedResources,\n// IncludedNamespaceScopedResources and ExcludedNamespaceScopedResources.\n// If all resource filters are none, it is treated as using new parameter filters.\nfunc UseOldResourceFilters(backupSpec velerov1api.BackupSpec) bool {\n\tif backupSpec.IncludeClusterResources != nil ||\n\t\tlen(backupSpec.IncludedResources) > 0 ||\n\t\tlen(backupSpec.ExcludedResources) > 0 {\n\t\treturn true\n\t}\n\treturn false\n}\n" }, { "path": "pkg/util/collections/includes_excludes_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage collections\n\nimport (\n\t\"testing\"\n\n\t\"github.com/vmware-tanzu/velero/internal/resourcepolicies\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestShouldInclude(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tincludes []string\n\t\texcludes []string\n\t\titem string\n\t\twant bool\n\t}{\n\t\t{\n\t\t\tname: \"empty string should include every item\",\n\t\t\titem: \"foo\",\n\t\t\twant: true,\n\t\t},\n\t\t{\n\t\t\tname: \"include * should include every item\",\n\t\t\tincludes: []string{\"*\"},\n\t\t\titem: \"foo\",\n\t\t\twant: true,\n\t\t},\n\t\t{\n\t\t\tname: \"item in includes list should include item\",\n\t\t\tincludes: []string{\"foo\", \"bar\", \"baz\"},\n\t\t\titem: \"foo\",\n\t\t\twant: true,\n\t\t},\n\t\t{\n\t\t\tname: \"item not in includes list should not include item\",\n\t\t\tincludes: []string{\"foo\", \"baz\"},\n\t\t\titem: \"bar\",\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"include *, excluded item should not include item\",\n\t\t\tincludes: []string{\"*\"},\n\t\t\texcludes: []string{\"foo\"},\n\t\t\titem: \"foo\",\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"include *, exclude foo, bar should be included\",\n\t\t\tincludes: []string{\"*\"},\n\t\t\texcludes: []string{\"foo\"},\n\t\t\titem: \"bar\",\n\t\t\twant: true,\n\t\t},\n\t\t{\n\t\t\tname: \"an item both included and excluded should not be included\",\n\t\t\tincludes: []string{\"foo\"},\n\t\t\texcludes: []string{\"foo\"},\n\t\t\titem: \"foo\",\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard should include item\",\n\t\t\tincludes: []string{\"*.bar\"},\n\t\t\titem: \"foo.bar\",\n\t\t\twant: true,\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard mismatch should not include item\",\n\t\t\tincludes: []string{\"*.bar\"},\n\t\t\titem: \"bar.foo\",\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard exclude should not include item\",\n\t\t\tincludes: []string{\"*\"},\n\t\t\texcludes: []string{\"*.bar\"},\n\t\t\titem: \"foo.bar\",\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard mismatch should include item\",\n\t\t\tincludes: []string{\"*\"},\n\t\t\texcludes: []string{\"*.bar\"},\n\t\t\titem: \"bar.foo\",\n\t\t\twant: true,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tincludesExcludes := NewIncludesExcludes().Includes(tc.includes...).Excludes(tc.excludes...)\n\n\t\t\tif got := includesExcludes.ShouldInclude((tc.item)); got != tc.want {\n\t\t\t\tt.Errorf(\"want %t, got %t\", tc.want, got)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestValidateIncludesExcludes(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tincludes []string\n\t\texcludes []string\n\t\twant []error\n\t}{\n\t\t{\n\t\t\tname: \"empty includes (everything) is allowed\",\n\t\t\tincludes: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"include everything\",\n\t\t\tincludes: []string{\"*\"},\n\t\t},\n\t\t{\n\t\t\tname: \"include everything not allowed with other includes\",\n\t\t\tincludes: []string{\"*\", \"foo\"},\n\t\t\twant: []error{errors.New(\"includes list must either contain '*' only, or a non-empty list of items\")},\n\t\t},\n\t\t{\n\t\t\tname: \"exclude everything not allowed\",\n\t\t\tincludes: []string{\"foo\"},\n\t\t\texcludes: []string{\"*\"},\n\t\t\twant: []error{errors.New(\"excludes list cannot contain '*'\")},\n\t\t},\n\t\t{\n\t\t\tname: \"excludes cannot contain items in includes\",\n\t\t\tincludes: []string{\"foo\", \"bar\"},\n\t\t\texcludes: []string{\"bar\"},\n\t\t\twant: []error{errors.New(\"excludes list cannot contain an item in the includes list: bar\")},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\terrs := ValidateIncludesExcludes(tc.includes, tc.excludes)\n\n\t\t\trequire.Len(t, errs, len(tc.want))\n\n\t\t\tfor i := 0; i < len(tc.want); i++ {\n\t\t\t\tassert.Equal(t, tc.want[i].Error(), errs[i].Error())\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestIncludeExcludeString(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tincludes []string\n\t\texcludes []string\n\t\twantIncludes string\n\t\twantExcludes string\n\t}{\n\t\t{\n\t\t\tname: \"unspecified includes/excludes should return '*'/''\",\n\t\t\tincludes: nil,\n\t\t\texcludes: nil,\n\t\t\twantIncludes: \"*\",\n\t\t\twantExcludes: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"specific resources should result in sorted joined string\",\n\t\t\tincludes: []string{\"foo\", \"bar\"},\n\t\t\texcludes: []string{\"baz\", \"xyz\"},\n\t\t\twantIncludes: \"bar, foo\",\n\t\t\twantExcludes: \"baz, xyz\",\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tincludesExcludes := NewIncludesExcludes().Includes(tc.includes...).Excludes(tc.excludes...)\n\t\t\tassert.Equal(t, tc.wantIncludes, includesExcludes.IncludesString())\n\t\t\tassert.Equal(t, tc.wantExcludes, includesExcludes.ExcludesString())\n\t\t})\n\t}\n}\n\nfunc TestValidateNamespaceIncludesExcludes(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tincludes []string\n\t\texcludes []string\n\t\twantErr bool\n\t}{\n\t\t{\n\t\t\tname: \"empty slice doesn't return error\",\n\t\t\tincludes: []string{},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"asterisk by itself is valid\",\n\t\t\tincludes: []string{\"*\"},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"alphanumeric names with optional dash inside are valid\",\n\t\t\tincludes: []string{\"foobar\", \"bar-321\", \"foo123bar\"},\n\t\t\texcludes: []string{\"123bar\", \"barfoo\", \"foo-321\", \"bar123foo\"},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"not starting or ending with an alphanumeric character is invalid\",\n\t\t\tincludes: []string{\"-123foo\"},\n\t\t\texcludes: []string{\"foo321-\", \"foo321-\"},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"special characters in name is invalid\",\n\t\t\tincludes: []string{\"foo?\", \"foo.bar\", \"bar_321\"},\n\t\t\texcludes: []string{\"$foo\", \"foo>bar\", \"bar=321\"},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"empty includes (everything) is valid\",\n\t\t\tincludes: []string{},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"empty string includes is valid (includes nothing)\",\n\t\t\tincludes: []string{\"\"},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"empty string excludes is valid (excludes nothing)\",\n\t\t\texcludes: []string{\"\"},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"include everything using asterisk is valid\",\n\t\t\tincludes: []string{\"*\"},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"excludes can contain wildcard\",\n\t\t\tincludes: []string{\"foo\", \"bar\"},\n\t\t\texcludes: []string{\"nginx-ingress-*\", \"*-bar\", \"*-ingress-*\"},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"includes can contain wildcard\",\n\t\t\tincludes: []string{\"*-foo\", \"kube-*\", \"*kube*\"},\n\t\t\texcludes: []string{\"bar\"},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"include everything not allowed with other includes\",\n\t\t\tincludes: []string{\"*\", \"foo\"},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"exclude everything not allowed\",\n\t\t\tincludes: []string{\"foo\"},\n\t\t\texcludes: []string{\"*\"},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"excludes cannot contain items in includes\",\n\t\t\tincludes: []string{\"foo\", \"bar\"},\n\t\t\texcludes: []string{\"bar\"},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"glob characters in includes should not error\",\n\t\t\tincludes: []string{\"kube-*\", \"test-?\", \"ns-[0-9]\"},\n\t\t\texcludes: []string{},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"glob characters in excludes should not error\",\n\t\t\tincludes: []string{\"default\"},\n\t\t\texcludes: []string{\"test-*\", \"app-?\", \"ns-[1-5]\"},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"character class in includes should not error\",\n\t\t\tincludes: []string{\"ns-[abc]\", \"test-[0-9]\"},\n\t\t\texcludes: []string{},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"mixed glob patterns should not error\",\n\t\t\tincludes: []string{\"kube-*\", \"test-?\"},\n\t\t\texcludes: []string{\"*-test\", \"debug-[0-9]\"},\n\t\t\twantErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"pipe character in includes should error\",\n\t\t\tincludes: []string{\"namespace|other\"},\n\t\t\texcludes: []string{},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"parentheses in includes should error\",\n\t\t\tincludes: []string{\"namespace(prod)\", \"test-(dev)\"},\n\t\t\texcludes: []string{},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"exclamation mark in includes should error\",\n\t\t\tincludes: []string{\"!namespace\", \"test!\"},\n\t\t\texcludes: []string{},\n\t\t\twantErr: true,\n\t\t},\n\t\t{\n\t\t\tname: \"unsupported characters in excludes should error\",\n\t\t\tincludes: []string{\"default\"},\n\t\t\texcludes: []string{\"test|prod\", \"app(staging)\"},\n\t\t\twantErr: true,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\terrs := ValidateNamespaceIncludesExcludes(tc.includes, tc.excludes)\n\n\t\t\tif tc.wantErr && len(errs) == 0 {\n\t\t\t\tt.Errorf(\"%s: wanted errors but got none\", tc.name)\n\t\t\t}\n\n\t\t\tif !tc.wantErr && len(errs) != 0 {\n\t\t\t\tt.Errorf(\"%s: wanted no errors but got: %v\", tc.name, errs)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestValidateScopedIncludesExcludes(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tincludes []string\n\t\texcludes []string\n\t\twantErr []error\n\t}{\n\t\t// includes testing\n\t\t{\n\t\t\tname: \"empty includes is valid\",\n\t\t\tincludes: []string{},\n\t\t\twantErr: []error{},\n\t\t},\n\t\t{\n\t\t\tname: \"asterisk includes is valid\",\n\t\t\tincludes: []string{\"*\"},\n\t\t\twantErr: []error{},\n\t\t},\n\t\t{\n\t\t\tname: \"include everything not allowed with other includes\",\n\t\t\tincludes: []string{\"*\", \"foo\"},\n\t\t\twantErr: []error{errors.New(\"includes list must either contain '*' only, or a non-empty list of items\")},\n\t\t},\n\t\t// excludes testing\n\t\t{\n\t\t\tname: \"empty excludes is valid\",\n\t\t\texcludes: []string{},\n\t\t\twantErr: []error{},\n\t\t},\n\t\t{\n\t\t\tname: \"asterisk excludes is valid\",\n\t\t\texcludes: []string{\"*\"},\n\t\t\twantErr: []error{},\n\t\t},\n\t\t{\n\t\t\tname: \"exclude everything not allowed with other excludes\",\n\t\t\texcludes: []string{\"*\", \"foo\"},\n\t\t\twantErr: []error{errors.New(\"excludes list must either contain '*' only, or a non-empty list of items\")},\n\t\t},\n\t\t// includes and excludes combination testing\n\t\t{\n\t\t\tname: \"asterisk excludes doesn't work with non-empty includes\",\n\t\t\tincludes: []string{\"foo\"},\n\t\t\texcludes: []string{\"*\"},\n\t\t\twantErr: []error{errors.New(\"when exclude is '*', include cannot have value\")},\n\t\t},\n\t\t{\n\t\t\tname: \"excludes cannot contain items in includes\",\n\t\t\tincludes: []string{\"foo\", \"bar\"},\n\t\t\texcludes: []string{\"bar\"},\n\t\t\twantErr: []error{errors.New(\"excludes list cannot contain an item in the includes list: bar\")},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\terrs := ValidateScopedIncludesExcludes(tc.includes, tc.excludes)\n\n\t\t\trequire.Len(t, errs, len(tc.wantErr))\n\n\t\t\tfor i := 0; i < len(tc.wantErr); i++ {\n\t\t\t\tassert.Equal(t, tc.wantErr[i].Error(), errs[i].Error())\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestNamespaceScopedShouldInclude(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tnamespaceScopedIncludes []string\n\t\tnamespaceScopedExcludes []string\n\t\titem string\n\t\twant bool\n\t\tapiResources []*test.APIResource\n\t}{\n\t\t{\n\t\t\tname: \"empty string should include every item\",\n\t\t\titem: \"pods\",\n\t\t\twant: true,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"include * should include every item\",\n\t\t\tnamespaceScopedIncludes: []string{\"*\"},\n\t\t\titem: \"pods\",\n\t\t\twant: true,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"item in includes list should include item\",\n\t\t\tnamespaceScopedIncludes: []string{\"foo\", \"bar\", \"pods\"},\n\t\t\titem: \"pods\",\n\t\t\twant: true,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"item not in includes list should not include item\",\n\t\t\tnamespaceScopedIncludes: []string{\"foo\", \"baz\"},\n\t\t\titem: \"pods\",\n\t\t\twant: false,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"include *, excluded item should not include item\",\n\t\t\tnamespaceScopedIncludes: []string{\"*\"},\n\t\t\tnamespaceScopedExcludes: []string{\"pods\"},\n\t\t\titem: \"pods\",\n\t\t\twant: false,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"include *, exclude foo, bar should be included\",\n\t\t\tnamespaceScopedIncludes: []string{\"*\"},\n\t\t\tnamespaceScopedExcludes: []string{\"foo\"},\n\t\t\titem: \"pods\",\n\t\t\twant: true,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"an item both included and excluded should not be included\",\n\t\t\tnamespaceScopedIncludes: []string{\"pods\"},\n\t\t\tnamespaceScopedExcludes: []string{\"pods\"},\n\t\t\titem: \"pods\",\n\t\t\twant: false,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard should include item\",\n\t\t\tnamespaceScopedIncludes: []string{\"*s\"},\n\t\t\titem: \"pods\",\n\t\t\twant: true,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard mismatch should not include item\",\n\t\t\tnamespaceScopedIncludes: []string{\"*.bar\"},\n\t\t\titem: \"pods\",\n\t\t\twant: false,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"exclude * should include nothing\",\n\t\t\tnamespaceScopedExcludes: []string{\"*\"},\n\t\t\titem: \"pods\",\n\t\t\twant: false,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard exclude should not include item\",\n\t\t\tnamespaceScopedIncludes: []string{\"*\"},\n\t\t\tnamespaceScopedExcludes: []string{\"*s\"},\n\t\t\titem: \"pods\",\n\t\t\twant: false,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard exclude mismatch should include item\",\n\t\t\tnamespaceScopedExcludes: []string{\"*.bar\"},\n\t\t\titem: \"pods\",\n\t\t\twant: true,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"resource cannot be found by discovery client should not be include\",\n\t\t\titem: \"pods\",\n\t\t\twant: false,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tdiscoveryHelper := setupDiscoveryClientWithResources(tc.apiResources)\n\t\t\tlogger := logrus.StandardLogger()\n\t\t\tscopeIncludesExcludes := GetScopeResourceIncludesExcludes(discoveryHelper, logger, tc.namespaceScopedIncludes, tc.namespaceScopedExcludes, []string{}, []string{}, *NewNamespaceIncludesExcludes())\n\n\t\t\tif got := scopeIncludesExcludes.ShouldInclude((tc.item)); got != tc.want {\n\t\t\t\tt.Errorf(\"want %t, got %t\", tc.want, got)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestClusterScopedShouldInclude(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tclusterScopedIncludes []string\n\t\tclusterScopedExcludes []string\n\t\tnsIncludes []string\n\t\titem string\n\t\twant bool\n\t\tapiResources []*test.APIResource\n\t}{\n\t\t{\n\t\t\tname: \"empty string should include nothing\",\n\t\t\tnsIncludes: []string{\"default\"},\n\t\t\titem: \"persistentvolumes\",\n\t\t\twant: false,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"include * should include every item\",\n\t\t\tclusterScopedIncludes: []string{\"*\"},\n\t\t\titem: \"persistentvolumes\",\n\t\t\twant: true,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"item in includes list should include item\",\n\t\t\tclusterScopedIncludes: []string{\"namespaces\", \"bar\", \"baz\"},\n\t\t\titem: \"namespaces\",\n\t\t\twant: true,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Namespaces(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"item not in includes list should not include item\",\n\t\t\tclusterScopedIncludes: []string{\"foo\", \"baz\"},\n\t\t\tnsIncludes: []string{\"default\"},\n\t\t\titem: \"persistentvolumes\",\n\t\t\twant: false,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"include *, excluded item should not include item\",\n\t\t\tclusterScopedIncludes: []string{\"*\"},\n\t\t\tclusterScopedExcludes: []string{\"namespaces\"},\n\t\t\titem: \"namespaces\",\n\t\t\twant: false,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Namespaces(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"include *, exclude foo, bar should be included\",\n\t\t\tclusterScopedIncludes: []string{\"*\"},\n\t\t\tclusterScopedExcludes: []string{\"foo\"},\n\t\t\titem: \"namespaces\",\n\t\t\twant: true,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Namespaces(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"an item both included and excluded should not be included\",\n\t\t\tclusterScopedIncludes: []string{\"namespaces\"},\n\t\t\tclusterScopedExcludes: []string{\"namespaces\"},\n\t\t\titem: \"namespaces\",\n\t\t\twant: false,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Namespaces(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard should include item\",\n\t\t\tclusterScopedIncludes: []string{\"*spaces\"},\n\t\t\titem: \"namespaces\",\n\t\t\twant: true,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Namespaces(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard mismatch should not include item\",\n\t\t\tclusterScopedIncludes: []string{\"*.bar\"},\n\t\t\tnsIncludes: []string{\"default\"},\n\t\t\titem: \"persistentvolumes\",\n\t\t\twant: false,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"exclude * should include nothing\",\n\t\t\tclusterScopedExcludes: []string{\"*\"},\n\t\t\titem: \"namespaces\",\n\t\t\twant: false,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Namespaces(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard exclude should not include item\",\n\t\t\tclusterScopedIncludes: []string{\"*\"},\n\t\t\tclusterScopedExcludes: []string{\"*spaces\"},\n\t\t\titem: \"namespaces\",\n\t\t\twant: false,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Namespaces(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard exclude mismatch should not include item\",\n\t\t\tclusterScopedExcludes: []string{\"*spaces\"},\n\t\t\titem: \"namespaces\",\n\t\t\twant: false,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Namespaces(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"resource cannot be found by discovery client should not be include\",\n\t\t\titem: \"namespaces\",\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"even namespaces is not in the include list, it should also be involved.\",\n\t\t\tclusterScopedIncludes: []string{\"foo\", \"baz\"},\n\t\t\titem: \"namespaces\",\n\t\t\twant: true,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Namespaces(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"When all namespaces and namespace scope resources are included, cluster resource should be included.\",\n\t\t\tclusterScopedIncludes: []string{},\n\t\t\tnsIncludes: []string{\"*\"},\n\t\t\titem: \"persistentvolumes\",\n\t\t\twant: true,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"When all namespaces and namespace scope resources are included, but cluster resource is excluded.\",\n\t\t\tclusterScopedIncludes: []string{},\n\t\t\tclusterScopedExcludes: []string{\"persistentvolumes\"},\n\t\t\tnsIncludes: []string{\"*\"},\n\t\t\titem: \"persistentvolumes\",\n\t\t\twant: false,\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t},\n\t}\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tdiscoveryHelper := setupDiscoveryClientWithResources(tc.apiResources)\n\t\t\tlogger := logrus.StandardLogger()\n\t\t\tnsIncludeExclude := NewNamespaceIncludesExcludes().Includes(tc.nsIncludes...)\n\t\t\tscopeIncludesExcludes := GetScopeResourceIncludesExcludes(discoveryHelper, logger, []string{}, []string{}, tc.clusterScopedIncludes, tc.clusterScopedExcludes, *nsIncludeExclude)\n\n\t\t\tif got := scopeIncludesExcludes.ShouldInclude((tc.item)); got != tc.want {\n\t\t\t\tt.Errorf(\"want %t, got %t\", tc.want, got)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetScopedResourceIncludesExcludes(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tnamespaceScopedIncludes []string\n\t\tnamespaceScopedExcludes []string\n\t\tclusterScopedIncludes []string\n\t\tclusterScopedExcludes []string\n\t\texpectedNamespaceScopedIncludes []string\n\t\texpectedNamespaceScopedExcludes []string\n\t\texpectedClusterScopedIncludes []string\n\t\texpectedClusterScopedExcludes []string\n\t\tapiResources []*test.APIResource\n\t}{\n\t\t{\n\t\t\tname: \"only include namespace-scoped resources in IncludesExcludes\",\n\t\t\tnamespaceScopedIncludes: []string{\"deployments.apps\", \"persistentvolumes\"},\n\t\t\tnamespaceScopedExcludes: []string{\"pods\", \"persistentvolumes\"},\n\t\t\texpectedNamespaceScopedIncludes: []string{\"deployments.apps\"},\n\t\t\texpectedNamespaceScopedExcludes: []string{\"pods\"},\n\t\t\texpectedClusterScopedIncludes: []string{},\n\t\t\texpectedClusterScopedExcludes: []string{},\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Deployments(),\n\t\t\t\ttest.PVs(),\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"only include cluster-scoped resources in IncludesExcludes\",\n\t\t\tclusterScopedIncludes: []string{\"deployments.apps\", \"persistentvolumes\"},\n\t\t\tclusterScopedExcludes: []string{\"pods\", \"persistentvolumes\"},\n\t\t\texpectedNamespaceScopedIncludes: []string{},\n\t\t\texpectedNamespaceScopedExcludes: []string{},\n\t\t\texpectedClusterScopedIncludes: []string{\"persistentvolumes\"},\n\t\t\texpectedClusterScopedExcludes: []string{\"persistentvolumes\"},\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.Deployments(),\n\t\t\t\ttest.PVs(),\n\t\t\t\ttest.Pods(),\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlogger := logrus.StandardLogger()\n\t\t\tnsIncludeExclude := NewNamespaceIncludesExcludes()\n\t\t\tresources := GetScopeResourceIncludesExcludes(setupDiscoveryClientWithResources(tc.apiResources), logger, tc.namespaceScopedIncludes, tc.namespaceScopedExcludes, tc.clusterScopedIncludes, tc.clusterScopedExcludes, *nsIncludeExclude)\n\n\t\t\tassert.Equal(t, tc.expectedNamespaceScopedIncludes, resources.namespaceScopedResourceFilter.includes.List())\n\t\t\tassert.Equal(t, tc.expectedNamespaceScopedExcludes, resources.namespaceScopedResourceFilter.excludes.List())\n\t\t\tassert.Equal(t, tc.expectedClusterScopedIncludes, resources.clusterScopedResourceFilter.includes.List())\n\t\t\tassert.Equal(t, tc.expectedClusterScopedExcludes, resources.clusterScopedResourceFilter.excludes.List())\n\t\t})\n\t}\n}\n\nfunc TestScopeIncludesExcludes_CombineWithPolicy(t *testing.T) {\n\tapiResources := []*test.APIResource{test.Deployments(), test.Pods(), test.ConfigMaps(), test.Secrets(), test.PVs(), test.CRDs(), test.ServiceAccounts()}\n\ttests := []struct {\n\t\tname string\n\t\tnamespaceScopedIncludes []string\n\t\tnamespaceScopedExcludes []string\n\t\tclusterScopedIncludes []string\n\t\tclusterScopedExcludes []string\n\t\tpolicy *resourcepolicies.IncludeExcludePolicy\n\t\tverify func(sie ScopeIncludesExcludes) bool\n\t}{\n\t\t{\n\t\t\tname: \"When policy is nil, the original includes excludes filters should not change\",\n\t\t\tnamespaceScopedIncludes: []string{\"deployments\", \"pods\"},\n\t\t\tnamespaceScopedExcludes: []string{\"configmaps\"},\n\t\t\tclusterScopedIncludes: []string{\"persistentvolumes\"},\n\t\t\tclusterScopedExcludes: []string{\"crds\"},\n\t\t\tpolicy: nil,\n\t\t\tverify: func(sie ScopeIncludesExcludes) bool {\n\t\t\t\treturn sie.clusterScopedResourceFilter.ShouldInclude(\"persistentvolumes\") &&\n\t\t\t\t\t!sie.clusterScopedResourceFilter.ShouldInclude(\"crds\") &&\n\t\t\t\t\tsie.namespaceScopedResourceFilter.ShouldInclude(\"deployments\") &&\n\t\t\t\t\t!sie.namespaceScopedResourceFilter.ShouldInclude(\"configmaps\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"policy includes excludes should be merged to the original includes excludes when there's no conflict\",\n\t\t\tnamespaceScopedIncludes: []string{\"pods\"},\n\t\t\tnamespaceScopedExcludes: []string{\"configmaps\"},\n\t\t\tclusterScopedIncludes: []string{},\n\t\t\tclusterScopedExcludes: []string{\"crds\"},\n\t\t\tpolicy: &resourcepolicies.IncludeExcludePolicy{\n\t\t\t\tIncludedNamespaceScopedResources: []string{\"deployments\"},\n\t\t\t\tExcludedNamespaceScopedResources: []string{\"secrets\"},\n\t\t\t\tIncludedClusterScopedResources: []string{\"persistentvolumes\"},\n\t\t\t\tExcludedClusterScopedResources: []string{},\n\t\t\t},\n\t\t\tverify: func(sie ScopeIncludesExcludes) bool {\n\t\t\t\treturn sie.clusterScopedResourceFilter.ShouldInclude(\"persistentvolumes\") &&\n\t\t\t\t\t!sie.clusterScopedResourceFilter.ShouldInclude(\"crds\") &&\n\t\t\t\t\tsie.namespaceScopedResourceFilter.ShouldInclude(\"deployments\") &&\n\t\t\t\t\t!sie.namespaceScopedResourceFilter.ShouldInclude(\"configmaps\") &&\n\t\t\t\t\t!sie.namespaceScopedResourceFilter.ShouldInclude(\"secrets\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when there are conflicts, the existing includes excludes filters have higher priorities\",\n\t\t\tnamespaceScopedIncludes: []string{\"pods\", \"deployments\"},\n\t\t\tnamespaceScopedExcludes: []string{\"configmaps\"},\n\t\t\tclusterScopedIncludes: []string{\"crds\"},\n\t\t\tclusterScopedExcludes: []string{\"persistentvolumes\"},\n\t\t\tpolicy: &resourcepolicies.IncludeExcludePolicy{\n\t\t\t\tIncludedNamespaceScopedResources: []string{\"configmaps\"},\n\t\t\t\tExcludedNamespaceScopedResources: []string{\"pods\", \"secrets\"},\n\t\t\t\tIncludedClusterScopedResources: []string{\"persistentvolumes\"},\n\t\t\t\tExcludedClusterScopedResources: []string{\"crds\"},\n\t\t\t},\n\t\t\tverify: func(sie ScopeIncludesExcludes) bool {\n\t\t\t\treturn sie.clusterScopedResourceFilter.ShouldInclude(\"crds\") &&\n\t\t\t\t\t!sie.clusterScopedResourceFilter.ShouldInclude(\"persistentvolumes\") &&\n\t\t\t\t\tsie.namespaceScopedResourceFilter.ShouldInclude(\"pods\") &&\n\t\t\t\t\t!sie.namespaceScopedResourceFilter.ShouldInclude(\"configmaps\") &&\n\t\t\t\t\t!sie.namespaceScopedResourceFilter.ShouldInclude(\"secrets\")\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"verify the case when there's '*' in the original include filter\",\n\t\t\tnamespaceScopedIncludes: []string{\"*\"},\n\t\t\tnamespaceScopedExcludes: []string{},\n\t\t\tclusterScopedIncludes: []string{},\n\t\t\tclusterScopedExcludes: []string{},\n\t\t\tpolicy: &resourcepolicies.IncludeExcludePolicy{\n\t\t\t\tIncludedNamespaceScopedResources: []string{\"deployments\", \"pods\"},\n\t\t\t\tExcludedNamespaceScopedResources: []string{\"configmaps\", \"secrets\"},\n\t\t\t\tIncludedClusterScopedResources: []string{},\n\t\t\t\tExcludedClusterScopedResources: []string{},\n\t\t\t},\n\t\t\tverify: func(sie ScopeIncludesExcludes) bool {\n\t\t\t\treturn sie.namespaceScopedResourceFilter.ShouldInclude(\"configmaps\") &&\n\t\t\t\t\tsie.namespaceScopedResourceFilter.ShouldInclude(\"secrets\")\n\t\t\t},\n\t\t},\n\t}\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlogger := logrus.StandardLogger()\n\t\t\tdiscoveryHelper := setupDiscoveryClientWithResources(apiResources)\n\t\t\tsie := GetScopeResourceIncludesExcludes(discoveryHelper, logger, tc.namespaceScopedIncludes, tc.namespaceScopedExcludes, tc.clusterScopedIncludes, tc.clusterScopedExcludes, *NewNamespaceIncludesExcludes())\n\t\t\tsie.CombineWithPolicy(tc.policy)\n\t\t\tassert.True(t, tc.verify(*sie))\n\t\t})\n\t}\n}\n\nfunc TestUseOldResourceFilters(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tbackup velerov1api.Backup\n\t\tuseOldResourceFilters bool\n\t}{\n\t\t{\n\t\t\tname: \"backup with no filters should use new filters\",\n\t\t\tbackup: *defaultBackup().Result(),\n\t\t\tuseOldResourceFilters: false,\n\t\t},\n\t\t{\n\t\t\tname: \"backup with only old filters should use old filters\",\n\t\t\tbackup: *defaultBackup().IncludeClusterResources(true).Result(),\n\t\t\tuseOldResourceFilters: true,\n\t\t},\n\t\t{\n\t\t\tname: \"backup with only new filters should use new filters\",\n\t\t\tbackup: *defaultBackup().IncludedClusterScopedResources(\"StorageClass\").Result(),\n\t\t\tuseOldResourceFilters: false,\n\t\t},\n\t\t{\n\t\t\t// This case should not happen in Velero workflow, because filter validation not old and new\n\t\t\t// filters used together. So this is only used for UT checking, and I assume old filters\n\t\t\t// have higher priority, because old parameter should be the default one.\n\t\t\tname: \"backup with both old and new filters should use old filters\",\n\t\t\tbackup: *defaultBackup().IncludeClusterResources(true).IncludedClusterScopedResources(\"StorageClass\").Result(),\n\t\t\tuseOldResourceFilters: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tassert.Equal(t, test.useOldResourceFilters, UseOldResourceFilters(test.backup.Spec))\n\t\t})\n\t}\n}\n\nfunc defaultBackup() *builder.BackupBuilder {\n\treturn builder.ForBackup(velerov1api.DefaultNamespace, \"backup-1\").DefaultVolumesToFsBackup(false)\n}\n\nfunc TestShouldExcluded(t *testing.T) {\n\tfalseBoolean := false\n\ttrueBoolean := true\n\ttests := []struct {\n\t\tname string\n\t\tclusterIncludes []string\n\t\tclusterExcludes []string\n\t\tincludeClusterResources *bool\n\t\tfilterType string\n\t\tresourceName string\n\t\tapiResources []*test.APIResource\n\t\tresourceIsExcluded bool\n\t}{\n\t\t{\n\t\t\tname: \"GlobalResourceIncludesExcludes: filters are all default\",\n\t\t\tclusterIncludes: []string{},\n\t\t\tclusterExcludes: []string{},\n\t\t\tincludeClusterResources: nil,\n\t\t\tfilterType: \"global\",\n\t\t\tresourceName: \"persistentvolumes\",\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\tresourceIsExcluded: false,\n\t\t},\n\t\t{\n\t\t\tname: \"GlobalResourceIncludesExcludes: IncludeClusterResources is set to true\",\n\t\t\tclusterIncludes: []string{},\n\t\t\tclusterExcludes: []string{},\n\t\t\tincludeClusterResources: &trueBoolean,\n\t\t\tfilterType: \"global\",\n\t\t\tresourceName: \"persistentvolumes\",\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\tresourceIsExcluded: false,\n\t\t},\n\t\t{\n\t\t\tname: \"GlobalResourceIncludesExcludes: IncludeClusterResources is set to false\",\n\t\t\tclusterIncludes: []string{\"persistentvolumes\"},\n\t\t\tclusterExcludes: []string{},\n\t\t\tincludeClusterResources: &falseBoolean,\n\t\t\tfilterType: \"global\",\n\t\t\tresourceName: \"persistentvolumes\",\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\tresourceIsExcluded: true,\n\t\t},\n\t\t{\n\t\t\tname: \"GlobalResourceIncludesExcludes: resource is in the include list\",\n\t\t\tclusterIncludes: []string{\"persistentvolumes\"},\n\t\t\tclusterExcludes: []string{},\n\t\t\tincludeClusterResources: nil,\n\t\t\tfilterType: \"global\",\n\t\t\tresourceName: \"persistentvolumes\",\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\tresourceIsExcluded: false,\n\t\t},\n\t\t{\n\t\t\tname: \"ScopeResourceIncludesExcludes: resource is in the include list\",\n\t\t\tclusterIncludes: []string{\"persistentvolumes\"},\n\t\t\tclusterExcludes: []string{},\n\t\t\tfilterType: \"scope\",\n\t\t\tresourceName: \"persistentvolumes\",\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\tresourceIsExcluded: false,\n\t\t},\n\t\t{\n\t\t\tname: \"ScopeResourceIncludesExcludes: filters are all default\",\n\t\t\tclusterIncludes: []string{},\n\t\t\tclusterExcludes: []string{},\n\t\t\tfilterType: \"scope\",\n\t\t\tresourceName: \"persistentvolumes\",\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\tresourceIsExcluded: false,\n\t\t},\n\t\t{\n\t\t\tname: \"ScopeResourceIncludesExcludes: resource is not in the exclude list\",\n\t\t\tclusterIncludes: []string{},\n\t\t\tclusterExcludes: []string{\"namespaces\"},\n\t\t\tfilterType: \"scope\",\n\t\t\tresourceName: \"persistentvolumes\",\n\t\t\tapiResources: []*test.APIResource{\n\t\t\t\ttest.PVs(),\n\t\t\t},\n\t\t\tresourceIsExcluded: false,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tlogger := logrus.StandardLogger()\n\n\t\t\tvar ie IncludesExcludesInterface\n\t\t\tif tc.filterType == \"global\" {\n\t\t\t\tie = GetGlobalResourceIncludesExcludes(setupDiscoveryClientWithResources(tc.apiResources), logger, tc.clusterIncludes, tc.clusterExcludes, tc.includeClusterResources, *NewNamespaceIncludesExcludes())\n\t\t\t} else if tc.filterType == \"scope\" {\n\t\t\t\tie = GetScopeResourceIncludesExcludes(setupDiscoveryClientWithResources(tc.apiResources), logger, []string{}, []string{}, tc.clusterIncludes, tc.clusterExcludes, *NewNamespaceIncludesExcludes())\n\t\t\t}\n\t\t\tassert.Equal(t, tc.resourceIsExcluded, ie.ShouldExclude(tc.resourceName))\n\t\t})\n\t}\n}\n\nfunc TestExpandIncludesExcludes(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tincludes []string\n\t\texcludes []string\n\t\tactiveNamespaces []string\n\t\texpectedIncludes []string\n\t\texpectedExcludes []string\n\t\texpectedWildcardExpanded bool\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"no wildcards - should not expand\",\n\t\t\tincludes: []string{\"default\", \"kube-system\"},\n\t\t\texcludes: []string{\"kube-public\"},\n\t\t\tactiveNamespaces: []string{\"default\", \"kube-system\", \"kube-public\", \"test\"},\n\t\t\texpectedIncludes: []string{\"default\", \"kube-system\"},\n\t\t\texpectedExcludes: []string{\"kube-public\"},\n\t\t\texpectedWildcardExpanded: false,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"asterisk alone - should not expand\",\n\t\t\tincludes: []string{\"*\"},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{\"default\", \"kube-system\", \"test\"},\n\t\t\texpectedIncludes: []string{\"*\"},\n\t\t\texpectedExcludes: []string{},\n\t\t\texpectedWildcardExpanded: false,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard in includes - should expand\",\n\t\t\tincludes: []string{\"kube-*\"},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{\"default\", \"kube-system\", \"kube-public\", \"test\"},\n\t\t\texpectedIncludes: []string{\"kube-system\", \"kube-public\"},\n\t\t\texpectedExcludes: []string{},\n\t\t\texpectedWildcardExpanded: true,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard in excludes - should expand\",\n\t\t\tincludes: []string{\"default\"},\n\t\t\texcludes: []string{\"*-test\"},\n\t\t\tactiveNamespaces: []string{\"default\", \"kube-test\", \"app-test\", \"prod\"},\n\t\t\texpectedIncludes: []string{\"default\"},\n\t\t\texpectedExcludes: []string{\"kube-test\", \"app-test\"},\n\t\t\texpectedWildcardExpanded: true,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"wildcards in both includes and excludes\",\n\t\t\tincludes: []string{\"kube-*\", \"app-*\"},\n\t\t\texcludes: []string{\"*-test\"},\n\t\t\tactiveNamespaces: []string{\"kube-system\", \"kube-test\", \"app-prod\", \"app-test\", \"default\"},\n\t\t\texpectedIncludes: []string{\"kube-system\", \"kube-test\", \"app-prod\", \"app-test\"},\n\t\t\texpectedExcludes: []string{\"kube-test\", \"app-test\"},\n\t\t\texpectedWildcardExpanded: true,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard pattern matches nothing\",\n\t\t\tincludes: []string{\"nonexistent-*\"},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{\"default\", \"kube-system\"},\n\t\t\texpectedIncludes: []string{},\n\t\t\texpectedExcludes: []string{},\n\t\t\texpectedWildcardExpanded: true,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"mix of wildcards and non-wildcards in includes\",\n\t\t\tincludes: []string{\"default\", \"kube-*\"},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{\"default\", \"kube-system\", \"kube-public\", \"test\"},\n\t\t\texpectedIncludes: []string{\"default\", \"kube-system\", \"kube-public\"},\n\t\t\texpectedExcludes: []string{},\n\t\t\texpectedWildcardExpanded: true,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"question mark wildcard\",\n\t\t\tincludes: []string{\"test-?\"},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{\"test-1\", \"test-2\", \"test-10\", \"default\"},\n\t\t\texpectedIncludes: []string{\"test-1\", \"test-2\"},\n\t\t\texpectedExcludes: []string{},\n\t\t\texpectedWildcardExpanded: true,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"empty activeNamespaces with wildcards\",\n\t\t\tincludes: []string{\"kube-*\"},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{},\n\t\t\texpectedIncludes: []string{},\n\t\t\texpectedExcludes: []string{},\n\t\t\texpectedWildcardExpanded: true,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"invalid wildcard pattern - consecutive asterisks\",\n\t\t\tincludes: []string{\"kube-**\"},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{\"default\"},\n\t\t\texpectedIncludes: []string{\"kube-**\"},\n\t\t\texpectedExcludes: []string{},\n\t\t\texpectedWildcardExpanded: false,\n\t\t\texpectError: true,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tnie := NewNamespaceIncludesExcludes().\n\t\t\t\tActiveNamespaces(tc.activeNamespaces).\n\t\t\t\tIncludes(tc.includes...).\n\t\t\t\tExcludes(tc.excludes...)\n\n\t\t\terr := nie.ExpandIncludesExcludes()\n\n\t\t\tif tc.expectError {\n\t\t\t\tassert.Error(t, err)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Equal(t, tc.expectedWildcardExpanded, nie.IsWildcardExpanded())\n\n\t\t\t// Check includes - convert to sets for order-independent comparison\n\t\t\tactualIncludes := sets.NewString(nie.GetIncludes()...)\n\t\t\texpectedIncludes := sets.NewString(tc.expectedIncludes...)\n\t\t\tassert.True(t, actualIncludes.Equal(expectedIncludes),\n\t\t\t\t\"includes mismatch: expected %v, got %v\", tc.expectedIncludes, nie.GetIncludes())\n\n\t\t\t// Check excludes\n\t\t\tactualExcludes := sets.NewString(nie.GetExcludes()...)\n\t\t\texpectedExcludes := sets.NewString(tc.expectedExcludes...)\n\t\t\tassert.True(t, actualExcludes.Equal(expectedExcludes),\n\t\t\t\t\"excludes mismatch: expected %v, got %v\", tc.expectedExcludes, nie.GetExcludes())\n\t\t})\n\t}\n}\n\nfunc TestResolveNamespaceList(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tincludes []string\n\t\texcludes []string\n\t\tactiveNamespaces []string\n\t\texpectedNamespaces []string\n\t\tpreExpandWildcards bool\n\t}{\n\t\t{\n\t\t\tname: \"no includes/excludes - all active namespaces\",\n\t\t\tincludes: []string{},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{\"default\", \"kube-system\", \"test\"},\n\t\t\texpectedNamespaces: []string{\"default\", \"kube-system\", \"test\"},\n\t\t},\n\t\t{\n\t\t\tname: \"asterisk includes - all active namespaces\",\n\t\t\tincludes: []string{\"*\"},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{\"default\", \"kube-system\", \"test\"},\n\t\t\texpectedNamespaces: []string{\"default\", \"kube-system\", \"test\"},\n\t\t},\n\t\t{\n\t\t\tname: \"specific includes - only those namespaces\",\n\t\t\tincludes: []string{\"default\", \"test\"},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{\"default\", \"kube-system\", \"test\"},\n\t\t\texpectedNamespaces: []string{\"default\", \"test\"},\n\t\t},\n\t\t{\n\t\t\tname: \"includes with excludes\",\n\t\t\tincludes: []string{\"*\"},\n\t\t\texcludes: []string{\"kube-system\"},\n\t\t\tactiveNamespaces: []string{\"default\", \"kube-system\", \"test\"},\n\t\t\texpectedNamespaces: []string{\"default\", \"test\"},\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard includes - expands and filters\",\n\t\t\tincludes: []string{\"kube-*\"},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{\"default\", \"kube-system\", \"kube-public\", \"test\"},\n\t\t\texpectedNamespaces: []string{\"kube-system\", \"kube-public\"},\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard includes with wildcard excludes\",\n\t\t\tincludes: []string{\"app-*\"},\n\t\t\texcludes: []string{\"*-test\"},\n\t\t\tactiveNamespaces: []string{\"app-prod\", \"app-dev\", \"app-test\", \"default\"},\n\t\t\texpectedNamespaces: []string{\"app-prod\", \"app-dev\"},\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard matches nothing - empty result\",\n\t\t\tincludes: []string{\"nonexistent-*\"},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{\"default\", \"kube-system\"},\n\t\t\texpectedNamespaces: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"empty active namespaces\",\n\t\t\tincludes: []string{\"*\"},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{},\n\t\t\texpectedNamespaces: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"includes namespace not in active namespaces\",\n\t\t\tincludes: []string{\"default\", \"nonexistent\"},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{\"default\", \"test\"},\n\t\t\texpectedNamespaces: []string{\"default\"},\n\t\t},\n\t\t{\n\t\t\tname: \"excludes all namespaces from includes\",\n\t\t\tincludes: []string{\"default\", \"test\"},\n\t\t\texcludes: []string{\"default\", \"test\"},\n\t\t\tactiveNamespaces: []string{\"default\", \"test\", \"prod\"},\n\t\t\texpectedNamespaces: []string{},\n\t\t},\n\t\t{\n\t\t\tname: \"pre-expanded wildcards - should not expand again\",\n\t\t\tincludes: []string{\"kube-*\"},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{\"default\", \"kube-system\", \"kube-public\"},\n\t\t\texpectedNamespaces: []string{\"kube-system\", \"kube-public\"},\n\t\t\tpreExpandWildcards: true,\n\t\t},\n\t\t{\n\t\t\tname: \"question mark wildcard pattern\",\n\t\t\tincludes: []string{\"ns-?\"},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{\"ns-1\", \"ns-2\", \"ns-10\", \"default\"},\n\t\t\texpectedNamespaces: []string{\"ns-1\", \"ns-2\"},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tnie := NewNamespaceIncludesExcludes().\n\t\t\t\tActiveNamespaces(tc.activeNamespaces).\n\t\t\t\tIncludes(tc.includes...).\n\t\t\t\tExcludes(tc.excludes...)\n\n\t\t\t// Pre-expand wildcards if requested\n\t\t\tif tc.preExpandWildcards {\n\t\t\t\terr := nie.ExpandIncludesExcludes()\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tnamespaces, err := nie.ResolveNamespaceList()\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Convert to sets for order-independent comparison\n\t\t\tactualNs := sets.NewString(namespaces...)\n\t\t\texpectedNs := sets.NewString(tc.expectedNamespaces...)\n\t\t\tassert.True(t, actualNs.Equal(expectedNs),\n\t\t\t\t\"namespaces mismatch: expected %v, got %v\", tc.expectedNamespaces, namespaces)\n\t\t})\n\t}\n}\n\nfunc TestResolveNamespaceListError(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tincludes []string\n\t\texcludes []string\n\t\tactiveNamespaces []string\n\t}{\n\t\t{\n\t\t\tname: \"invalid wildcard pattern in includes\",\n\t\t\tincludes: []string{\"kube-**\"},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{\"default\"},\n\t\t},\n\t\t{\n\t\t\tname: \"invalid wildcard pattern in excludes\",\n\t\t\tincludes: []string{\"default\"},\n\t\t\texcludes: []string{\"test-**\"},\n\t\t\tactiveNamespaces: []string{\"default\"},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tnie := NewNamespaceIncludesExcludes().\n\t\t\t\tActiveNamespaces(tc.activeNamespaces).\n\t\t\t\tIncludes(tc.includes...).\n\t\t\t\tExcludes(tc.excludes...)\n\n\t\t\t_, err := nie.ResolveNamespaceList()\n\t\t\tassert.Error(t, err)\n\t\t})\n\t}\n}\n\nfunc TestNamespaceIncludesExcludesShouldIncludeAfterWildcardExpansion(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tincludes []string\n\t\texcludes []string\n\t\tactiveNamespaces []string\n\t\ttestNamespace string\n\t\texpectedResult bool\n\t}{\n\t\t{\n\t\t\tname: \"wildcard expanded to empty includes - should not include anything\",\n\t\t\tincludes: []string{\"nonexistent-*\"},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{\"default\", \"kube-system\"},\n\t\t\ttestNamespace: \"default\",\n\t\t\texpectedResult: false,\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard expanded with matches - should include matched namespace\",\n\t\t\tincludes: []string{\"kube-*\"},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{\"default\", \"kube-system\", \"kube-public\"},\n\t\t\ttestNamespace: \"kube-system\",\n\t\t\texpectedResult: true,\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard expanded with matches - should not include unmatched namespace\",\n\t\t\tincludes: []string{\"kube-*\"},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{\"default\", \"kube-system\", \"kube-public\"},\n\t\t\ttestNamespace: \"default\",\n\t\t\texpectedResult: false,\n\t\t},\n\t\t{\n\t\t\tname: \"no wildcard expansion - empty includes means include all\",\n\t\t\tincludes: []string{},\n\t\t\texcludes: []string{},\n\t\t\tactiveNamespaces: []string{\"default\", \"kube-system\"},\n\t\t\ttestNamespace: \"default\",\n\t\t\texpectedResult: true,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tnie := NewNamespaceIncludesExcludes().\n\t\t\t\tActiveNamespaces(tc.activeNamespaces).\n\t\t\t\tIncludes(tc.includes...).\n\t\t\t\tExcludes(tc.excludes...)\n\n\t\t\terr := nie.ExpandIncludesExcludes()\n\t\t\trequire.NoError(t, err)\n\n\t\t\tresult := nie.ShouldInclude(tc.testNamespace)\n\t\t\tassert.Equal(t, tc.expectedResult, result)\n\t\t})\n\t}\n}\n\nfunc setupDiscoveryClientWithResources(APIResources []*test.APIResource) *test.FakeDiscoveryHelper {\n\tresourcesMap := make(map[schema.GroupVersionResource]schema.GroupVersionResource)\n\tresourceList := make([]*metav1.APIResourceList, 0)\n\n\tfor _, resource := range APIResources {\n\t\tgvr := schema.GroupVersionResource{\n\t\t\tGroup: resource.Group,\n\t\t\tVersion: resource.Version,\n\t\t\tResource: resource.Name,\n\t\t}\n\t\tresourcesMap[gvr] = gvr\n\n\t\tresourceList = append(resourceList,\n\t\t\t&metav1.APIResourceList{\n\t\t\t\tGroupVersion: gvr.GroupVersion().String(),\n\t\t\t\tAPIResources: []metav1.APIResource{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: resource.Name,\n\t\t\t\t\t\tKind: resource.Name,\n\t\t\t\t\t\tNamespaced: resource.Namespaced,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t)\n\t}\n\n\tdiscoveryHelper := test.NewFakeDiscoveryHelper(false, resourcesMap)\n\tdiscoveryHelper.ResourceList = resourceList\n\treturn discoveryHelper\n}\n" }, { "path": "pkg/util/csi/util.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\t\"strings\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/features\"\n)\n\nconst (\n\tcsiPluginNamePrefix = \"velero.io/csi-\"\n)\n\nfunc ShouldSkipAction(actionName string) bool {\n\treturn !features.IsEnabled(velerov1api.CSIFeatureFlag) && strings.Contains(actionName, csiPluginNamePrefix)\n}\n" }, { "path": "pkg/util/csi/util_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/features\"\n)\n\nfunc TestCSIFeatureNotEnabledAndPluginIsFromCSI(t *testing.T) {\n\tfeatures.NewFeatureFlagSet(\"EnableCSI\")\n\trequire.False(t, ShouldSkipAction(\"abc\"))\n\trequire.False(t, ShouldSkipAction(\"velero.io/csi-pvc-backupper\"))\n\n\tfeatures.NewFeatureFlagSet(\"\")\n\trequire.True(t, ShouldSkipAction(\"velero.io/csi-pvc-backupper\"))\n\trequire.False(t, ShouldSkipAction(\"abc\"))\n}\n" }, { "path": "pkg/util/csi/volume_snapshot.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"strings\"\n\t\"time\"\n\n\tjsonpatch \"github.com/evanphx/json-patch/v5\"\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\tsnapshotter \"github.com/kubernetes-csi/external-snapshotter/client/v8/clientset/versioned/typed/volumesnapshot/v1\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\tcrclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/stringptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/stringslice\"\n)\n\nconst (\n\twaitInternal = 2 * time.Second\n\tvolumeSnapshotContentProtectFinalizer = \"velero.io/volume-snapshot-content-protect-finalizer\"\n)\n\n// WaitVolumeSnapshotReady waits a VS to become ready to use until the timeout reaches\nfunc WaitVolumeSnapshotReady(\n\tctx context.Context,\n\tsnapshotClient snapshotter.SnapshotV1Interface,\n\tvolumeSnapshot string,\n\tvolumeSnapshotNS string,\n\ttimeout time.Duration,\n\tlog logrus.FieldLogger,\n) (*snapshotv1api.VolumeSnapshot, error) {\n\tvar updated *snapshotv1api.VolumeSnapshot\n\terrMessage := sets.NewString()\n\n\terr := wait.PollUntilContextTimeout(\n\t\tctx,\n\t\twaitInternal,\n\t\ttimeout,\n\t\ttrue,\n\t\tfunc(ctx context.Context) (bool, error) {\n\t\t\ttmpVS, err := snapshotClient.VolumeSnapshots(volumeSnapshotNS).Get(\n\t\t\t\tctx, volumeSnapshot, metav1.GetOptions{})\n\t\t\tif err != nil {\n\t\t\t\treturn false, errors.Wrapf(\n\t\t\t\t\terr,\n\t\t\t\t\t\"error to get VolumeSnapshot %s/%s\",\n\t\t\t\t\tvolumeSnapshotNS, volumeSnapshot,\n\t\t\t\t)\n\t\t\t}\n\n\t\t\tif tmpVS.Status == nil {\n\t\t\t\treturn false, nil\n\t\t\t}\n\n\t\t\tif tmpVS.Status.Error != nil {\n\t\t\t\terrMessage.Insert(stringptr.GetString(tmpVS.Status.Error.Message))\n\t\t\t}\n\n\t\t\tif !boolptr.IsSetToTrue(tmpVS.Status.ReadyToUse) {\n\t\t\t\treturn false, nil\n\t\t\t}\n\n\t\t\tupdated = tmpVS\n\t\t\treturn true, nil\n\t\t},\n\t)\n\n\tif wait.Interrupted(err) {\n\t\terr = errors.Errorf(\n\t\t\t\"volume snapshot is not ready until timeout, errors: %v\",\n\t\t\terrMessage.List(),\n\t\t)\n\t}\n\n\tif errMessage.Len() > 0 {\n\t\tlog.Warnf(\"Some errors happened during waiting for ready snapshot, errors: %v\",\n\t\t\terrMessage.List())\n\t}\n\n\treturn updated, err\n}\n\n// GetVolumeSnapshotContentForVolumeSnapshot returns the VolumeSnapshotContent\n// object associated with the VolumeSnapshot.\nfunc GetVolumeSnapshotContentForVolumeSnapshot(\n\tvolSnap *snapshotv1api.VolumeSnapshot,\n\tsnapshotClient snapshotter.SnapshotV1Interface,\n) (*snapshotv1api.VolumeSnapshotContent, error) {\n\tif volSnap.Status == nil || volSnap.Status.BoundVolumeSnapshotContentName == nil {\n\t\treturn nil, errors.Errorf(\"invalid snapshot info in volume snapshot %s\", volSnap.Name)\n\t}\n\n\tvsc, err := snapshotClient.VolumeSnapshotContents().Get(\n\t\tcontext.TODO(),\n\t\t*volSnap.Status.BoundVolumeSnapshotContentName,\n\t\tmetav1.GetOptions{},\n\t)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error getting volume snapshot content from API\")\n\t}\n\n\treturn vsc, nil\n}\n\n// RetainVSC updates the VSC's deletion policy to Retain and then return the update VSC\nfunc RetainVSC(ctx context.Context, snapshotClient snapshotter.SnapshotV1Interface,\n\tvsc *snapshotv1api.VolumeSnapshotContent) (*snapshotv1api.VolumeSnapshotContent, error) {\n\tif vsc.Spec.DeletionPolicy == snapshotv1api.VolumeSnapshotContentRetain {\n\t\treturn vsc, nil\n\t}\n\n\treturn patchVSC(ctx, snapshotClient, vsc, func(updated *snapshotv1api.VolumeSnapshotContent) {\n\t\tupdated.Spec.DeletionPolicy = snapshotv1api.VolumeSnapshotContentRetain\n\t})\n}\n\n// DeleteVolumeSnapshotContentIfAny deletes a VSC by name if it exists,\n// and log an error when the deletion fails.\nfunc DeleteVolumeSnapshotContentIfAny(\n\tctx context.Context,\n\tsnapshotClient snapshotter.SnapshotV1Interface,\n\tvscName string, log logrus.FieldLogger,\n) {\n\terr := snapshotClient.VolumeSnapshotContents().Delete(ctx, vscName, metav1.DeleteOptions{})\n\tif err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.WithError(err).Debugf(\"Abort deleting VSC, it doesn't exist %s\", vscName)\n\t\t} else {\n\t\t\tlog.WithError(err).Errorf(\"Failed to delete volume snapshot content %s\", vscName)\n\t\t}\n\t}\n}\n\n// EnsureDeleteVS asserts the existence of a VS by name, deletes it and waits for its\n// disappearance and returns errors on any failure.\nfunc EnsureDeleteVS(ctx context.Context, snapshotClient snapshotter.SnapshotV1Interface,\n\tvsName string, vsNamespace string, timeout time.Duration) error {\n\terr := snapshotClient.VolumeSnapshots(vsNamespace).Delete(ctx, vsName, metav1.DeleteOptions{})\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"error to delete volume snapshot\")\n\t}\n\n\tvar updated *snapshotv1api.VolumeSnapshot\n\terr = wait.PollUntilContextTimeout(ctx, waitInternal, timeout, true, func(ctx context.Context) (bool, error) {\n\t\tvs, err := snapshotClient.VolumeSnapshots(vsNamespace).Get(ctx, vsName, metav1.GetOptions{})\n\t\tif err != nil {\n\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\treturn true, nil\n\t\t\t}\n\n\t\t\treturn false, errors.Wrapf(err, \"error to get VolumeSnapshot %s\", vsName)\n\t\t}\n\n\t\tupdated = vs\n\t\treturn false, nil\n\t})\n\n\tif err != nil {\n\t\tif errors.Is(err, context.DeadlineExceeded) {\n\t\t\treturn errors.Errorf(\"timeout to assure VolumeSnapshot %s is deleted, finalizers in VS %v\", vsName, updated.Finalizers)\n\t\t} else {\n\t\t\treturn errors.Wrapf(err, \"error to assure VolumeSnapshot is deleted, %s\", vsName)\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc RemoveVSCProtect(ctx context.Context, snapshotClient snapshotter.SnapshotV1Interface, vscName string, timeout time.Duration) error {\n\terr := wait.PollUntilContextTimeout(ctx, waitInternal, timeout, true, func(ctx context.Context) (bool, error) {\n\t\tvsc, err := snapshotClient.VolumeSnapshotContents().Get(ctx, vscName, metav1.GetOptions{})\n\t\tif err != nil {\n\t\t\treturn false, errors.Wrapf(err, \"error to get VolumeSnapshotContent %s\", vscName)\n\t\t}\n\n\t\tvsc.Finalizers = stringslice.Except(vsc.Finalizers, volumeSnapshotContentProtectFinalizer)\n\n\t\t_, err = snapshotClient.VolumeSnapshotContents().Update(ctx, vsc, metav1.UpdateOptions{})\n\t\tif err == nil {\n\t\t\treturn true, nil\n\t\t}\n\n\t\tif !apierrors.IsConflict(err) {\n\t\t\treturn false, errors.Wrapf(err, \"error to update VolumeSnapshotContent %s\", vscName)\n\t\t}\n\n\t\treturn false, nil\n\t})\n\n\treturn err\n}\n\n// EnsureDeleteVSC asserts the existence of a VSC by name, deletes it and waits for its\n// disappearance and returns errors on any failure.\nfunc EnsureDeleteVSC(ctx context.Context, snapshotClient snapshotter.SnapshotV1Interface,\n\tvscName string, timeout time.Duration) error {\n\terr := snapshotClient.VolumeSnapshotContents().Delete(ctx, vscName, metav1.DeleteOptions{})\n\tif err != nil && !apierrors.IsNotFound(err) {\n\t\treturn errors.Wrap(err, \"error to delete volume snapshot content\")\n\t}\n\n\tvar updated *snapshotv1api.VolumeSnapshotContent\n\terr = wait.PollUntilContextTimeout(ctx, waitInternal, timeout, true, func(ctx context.Context) (bool, error) {\n\t\tvsc, err := snapshotClient.VolumeSnapshotContents().Get(ctx, vscName, metav1.GetOptions{})\n\t\tif err != nil {\n\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\treturn true, nil\n\t\t\t}\n\n\t\t\treturn false, errors.Wrapf(err, \"error to get VolumeSnapshotContent %s\", vscName)\n\t\t}\n\n\t\tupdated = vsc\n\t\treturn false, nil\n\t})\n\n\tif err != nil {\n\t\tif errors.Is(err, context.DeadlineExceeded) {\n\t\t\treturn errors.Errorf(\"timeout to assure VolumeSnapshotContent %s is deleted, finalizers in VSC %v\", vscName, updated.Finalizers)\n\t\t} else {\n\t\t\treturn errors.Wrapf(err, \"error to assure VolumeSnapshotContent is deleted, %s\", vscName)\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// DeleteVolumeSnapshotIfAny deletes a VS by name if it exists,\n// and log an error when the deletion fails\nfunc DeleteVolumeSnapshotIfAny(\n\tctx context.Context,\n\tsnapshotClient snapshotter.SnapshotV1Interface,\n\tvsName string,\n\tvsNamespace string,\n\tlog logrus.FieldLogger,\n) {\n\terr := snapshotClient.VolumeSnapshots(vsNamespace).Delete(ctx, vsName, metav1.DeleteOptions{})\n\tif err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.WithError(err).Debugf(\n\t\t\t\t\"Abort deleting volume snapshot, it doesn't exist %s/%s\",\n\t\t\t\tvsNamespace, vsName)\n\t\t} else {\n\t\t\tlog.WithError(err).Errorf(\n\t\t\t\t\"Failed to delete volume snapshot %s/%s\", vsNamespace, vsName)\n\t\t}\n\t}\n}\n\nfunc patchVSC(\n\tctx context.Context,\n\tsnapshotClient snapshotter.SnapshotV1Interface,\n\tvsc *snapshotv1api.VolumeSnapshotContent,\n\tupdateFunc func(*snapshotv1api.VolumeSnapshotContent),\n) (*snapshotv1api.VolumeSnapshotContent, error) {\n\torigBytes, err := json.Marshal(vsc)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error marshaling original VSC\")\n\t}\n\n\tupdated := vsc.DeepCopy()\n\tupdateFunc(updated)\n\n\tupdatedBytes, err := json.Marshal(updated)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error marshaling updated VSC\")\n\t}\n\n\tpatchBytes, err := jsonpatch.CreateMergePatch(origBytes, updatedBytes)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error creating json merge patch for VSC\")\n\t}\n\n\tpatched, err := snapshotClient.VolumeSnapshotContents().Patch(ctx, vsc.Name, types.MergePatchType, patchBytes, metav1.PatchOptions{})\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error patching VSC\")\n\t}\n\n\treturn patched, nil\n}\n\nfunc GetVolumeSnapshotClass(\n\tprovisioner string,\n\tbackup *velerov1api.Backup,\n\tpvc *corev1api.PersistentVolumeClaim,\n\tlog logrus.FieldLogger,\n\tcrClient crclient.Client,\n) (*snapshotv1api.VolumeSnapshotClass, error) {\n\tsnapshotClasses := new(snapshotv1api.VolumeSnapshotClassList)\n\terr := crClient.List(context.TODO(), snapshotClasses)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error listing VolumeSnapshotClass\")\n\t}\n\t// If a snapshot class is set for provider in PVC annotations, use that\n\tsnapshotClass, err := GetVolumeSnapshotClassFromPVCAnnotationsForDriver(\n\t\tpvc, provisioner, snapshotClasses,\n\t)\n\tif err != nil {\n\t\tlog.Debugf(\"Didn't find VolumeSnapshotClass from PVC annotations: %v\", err)\n\t}\n\tif snapshotClass != nil {\n\t\treturn snapshotClass, nil\n\t}\n\n\t// If there is no annotation in PVC, attempt to fetch it from backup annotations\n\tsnapshotClass, err = GetVolumeSnapshotClassFromBackupAnnotationsForDriver(\n\t\tbackup, provisioner, snapshotClasses)\n\tif err != nil {\n\t\tlog.Debugf(\"Didn't find VolumeSnapshotClass from Backup annotations: %v\", err)\n\t}\n\tif snapshotClass != nil {\n\t\treturn snapshotClass, nil\n\t}\n\n\t// fallback to default behavior of fetching snapshot class based on label\n\tsnapshotClass, err = GetVolumeSnapshotClassForStorageClass(\n\t\tprovisioner, snapshotClasses)\n\tif err != nil || snapshotClass == nil {\n\t\treturn nil, errors.Wrap(err, \"error getting VolumeSnapshotClass\")\n\t}\n\n\treturn snapshotClass, nil\n}\n\nfunc GetVolumeSnapshotClassFromPVCAnnotationsForDriver(\n\tpvc *corev1api.PersistentVolumeClaim,\n\tprovisioner string,\n\tsnapshotClasses *snapshotv1api.VolumeSnapshotClassList,\n) (*snapshotv1api.VolumeSnapshotClass, error) {\n\tannotationKey := velerov1api.VolumeSnapshotClassDriverPVCAnnotation\n\tsnapshotClassName, ok := pvc.ObjectMeta.Annotations[annotationKey]\n\tif !ok {\n\t\treturn nil, nil\n\t}\n\tfor _, sc := range snapshotClasses.Items {\n\t\tif strings.EqualFold(snapshotClassName, sc.ObjectMeta.Name) {\n\t\t\tif !strings.EqualFold(sc.Driver, provisioner) {\n\t\t\t\treturn nil, errors.Errorf(\n\t\t\t\t\t\"Incorrect VolumeSnapshotClass %s is not for driver %s\",\n\t\t\t\t\tsc.ObjectMeta.Name, provisioner,\n\t\t\t\t)\n\t\t\t}\n\t\t\treturn &sc, nil\n\t\t}\n\t}\n\treturn nil, errors.Errorf(\n\t\t\"No CSI VolumeSnapshotClass found with name %s for provisioner %s for PVC %s\",\n\t\tsnapshotClassName, provisioner, pvc.Name,\n\t)\n}\n\n// GetVolumeSnapshotClassFromAnnotationsForDriver returns a\n// VolumeSnapshotClass for the supplied volume provisioner/driver\n// name from the annotation of the backup.\nfunc GetVolumeSnapshotClassFromBackupAnnotationsForDriver(\n\tbackup *velerov1api.Backup,\n\tprovisioner string,\n\tsnapshotClasses *snapshotv1api.VolumeSnapshotClassList,\n) (*snapshotv1api.VolumeSnapshotClass, error) {\n\tannotationKey := fmt.Sprintf(\n\t\t\"%s_%s\",\n\t\tvelerov1api.VolumeSnapshotClassDriverBackupAnnotationPrefix,\n\t\tstrings.ToLower(provisioner),\n\t)\n\tsnapshotClassName, ok := backup.ObjectMeta.Annotations[annotationKey]\n\tif !ok {\n\t\treturn nil, nil\n\t}\n\tfor _, sc := range snapshotClasses.Items {\n\t\tif strings.EqualFold(snapshotClassName, sc.ObjectMeta.Name) {\n\t\t\tif !strings.EqualFold(sc.Driver, provisioner) {\n\t\t\t\treturn nil, errors.Errorf(\n\t\t\t\t\t\"Incorrect VolumeSnapshotClass %s is not for driver %s for backup %s\",\n\t\t\t\t\tsc.ObjectMeta.Name, provisioner, backup.Name,\n\t\t\t\t)\n\t\t\t}\n\t\t\treturn &sc, nil\n\t\t}\n\t}\n\treturn nil, errors.Errorf(\n\t\t\"No CSI VolumeSnapshotClass found with name %s for driver %s for backup %s\",\n\t\tsnapshotClassName, provisioner, backup.Name,\n\t)\n}\n\n// GetVolumeSnapshotClassForStorageClass returns a VolumeSnapshotClass\n// for the supplied volume provisioner/ driver name.\nfunc GetVolumeSnapshotClassForStorageClass(\n\tprovisioner string,\n\tsnapshotClasses *snapshotv1api.VolumeSnapshotClassList,\n) (*snapshotv1api.VolumeSnapshotClass, error) {\n\tn := 0\n\tvar vsClass snapshotv1api.VolumeSnapshotClass\n\t// We pick the VolumeSnapshotClass that matches the CSI driver name\n\t// and has a 'velero.io/csi-volumesnapshot-class' label. This allows\n\t// multiple VolumeSnapshotClasses for the same driver with different\n\t// values for the other fields in the spec.\n\tfor _, sc := range snapshotClasses.Items {\n\t\t_, hasLabelSelector := sc.Labels[velerov1api.VolumeSnapshotClassSelectorLabel]\n\t\tif sc.Driver == provisioner {\n\t\t\tn++\n\t\t\tvsClass = sc\n\t\t\tif hasLabelSelector {\n\t\t\t\treturn &sc, nil\n\t\t\t}\n\t\t}\n\t}\n\t// not found by label, pick by annotation\n\tfor _, sc := range snapshotClasses.Items {\n\t\t_, hasDefaultAnnotation := sc.Annotations[velerov1api.VolumeSnapshotClassKubernetesAnnotation]\n\t\tif sc.Driver == provisioner {\n\t\t\tvsClass = sc\n\t\t\tif hasDefaultAnnotation {\n\t\t\t\treturn &sc, nil\n\t\t\t}\n\t\t}\n\t}\n\t// If there's only one volumesnapshotclass for the driver, return it.\n\tif n == 1 {\n\t\treturn &vsClass, nil\n\t}\n\treturn nil, fmt.Errorf(\n\t\t\"failed to get VolumeSnapshotClass for provisioner %s: \"+\n\t\t\t\"ensure that the desired VolumeSnapshotClass has the %s label or %s annotation, \"+\n\t\t\t\"and that its driver matches the StorageClass provisioner\",\n\t\tprovisioner,\n\t\tvelerov1api.VolumeSnapshotClassSelectorLabel,\n\t\tvelerov1api.VolumeSnapshotClassKubernetesAnnotation,\n\t)\n}\n\n// IsVolumeSnapshotClassHasListerSecret returns whether a volumesnapshotclass has a snapshotlister secret\nfunc IsVolumeSnapshotClassHasListerSecret(vc *snapshotv1api.VolumeSnapshotClass) bool {\n\t// https://github.com/kubernetes-csi/external-snapshotter/blob/master/pkg/utils/util.go#L59-L60\n\t// There is no release w/ these constants exported. Using the strings for now.\n\t_, nameExists := vc.Annotations[velerov1api.PrefixedListSecretNameAnnotation]\n\t_, nsExists := vc.Annotations[velerov1api.PrefixedListSecretNamespaceAnnotation]\n\treturn nameExists && nsExists\n}\n\n// IsVolumeSnapshotContentHasDeleteSecret returns whether a volumesnapshotcontent has a deletesnapshot secret\nfunc IsVolumeSnapshotContentHasDeleteSecret(vsc *snapshotv1api.VolumeSnapshotContent) bool {\n\t// https://github.com/kubernetes-csi/external-snapshotter/blob/master/pkg/utils/util.go#L56-L57\n\t// use exported constants in the next release\n\t_, nameExists := vsc.Annotations[velerov1api.PrefixedSecretNameAnnotation]\n\t_, nsExists := vsc.Annotations[velerov1api.PrefixedSecretNamespaceAnnotation]\n\treturn nameExists && nsExists\n}\n\n// IsVolumeSnapshotExists returns whether a specific volumesnapshot object exists.\nfunc IsVolumeSnapshotExists(\n\tns,\n\tname string,\n\tcrClient crclient.Client,\n) bool {\n\tvs := new(snapshotv1api.VolumeSnapshot)\n\terr := crClient.Get(\n\t\tcontext.TODO(),\n\t\tcrclient.ObjectKey{Namespace: ns, Name: name},\n\t\tvs,\n\t)\n\n\treturn err == nil\n}\n\nfunc SetVolumeSnapshotContentDeletionPolicy(\n\tvscName string,\n\tcrClient crclient.Client,\n\tpolicy snapshotv1api.DeletionPolicy,\n) (*snapshotv1api.VolumeSnapshotContent, error) {\n\tvsc := new(snapshotv1api.VolumeSnapshotContent)\n\tif err := crClient.Get(context.TODO(), crclient.ObjectKey{Name: vscName}, vsc); err != nil {\n\t\treturn nil, err\n\t}\n\n\toriginVSC := vsc.DeepCopy()\n\tvsc.Spec.DeletionPolicy = policy\n\n\treturn vsc, crClient.Patch(context.TODO(), vsc, crclient.MergeFrom(originVSC))\n}\n\n// CleanupVolumeSnapshot deletes the VolumeSnapshot and the associated VolumeSnapshotContent. It will make sure the\n// physical snapshot is also deleted.\nfunc CleanupVolumeSnapshot(\n\tvolSnap *snapshotv1api.VolumeSnapshot,\n\tcrClient crclient.Client,\n\tlog logrus.FieldLogger,\n) {\n\tlog.Infof(\"Deleting Volumesnapshot %s/%s\", volSnap.Namespace, volSnap.Name)\n\tvs := new(snapshotv1api.VolumeSnapshot)\n\terr := crClient.Get(\n\t\tcontext.TODO(),\n\t\tcrclient.ObjectKey{Name: volSnap.Name, Namespace: volSnap.Namespace},\n\t\tvs,\n\t)\n\tif err != nil {\n\t\tlog.Debugf(\"Failed to get volumesnapshot %s/%s\", volSnap.Namespace, volSnap.Name)\n\t\treturn\n\t}\n\n\tif vs.Status != nil && vs.Status.BoundVolumeSnapshotContentName != nil {\n\t\t// we patch the DeletionPolicy of the VolumeSnapshotContent to set it to Delete.\n\t\t// This ensures that the volume snapshot in the storage provider is also deleted.\n\t\t_, err := SetVolumeSnapshotContentDeletionPolicy(\n\t\t\t*vs.Status.BoundVolumeSnapshotContentName,\n\t\t\tcrClient,\n\t\t\tsnapshotv1api.VolumeSnapshotContentDelete,\n\t\t)\n\t\tif err != nil {\n\t\t\tlog.Debugf(\"Failed to patch DeletionPolicy of volume snapshot %s/%s\",\n\t\t\t\tvs.Namespace, vs.Name)\n\t\t}\n\t}\n\terr = crClient.Delete(context.TODO(), vs)\n\tif err != nil {\n\t\tlog.Debugf(\"Failed to delete volumesnapshot %s/%s: %v\", vs.Namespace, vs.Name, err)\n\t} else {\n\t\tlog.Infof(\"Deleted volumesnapshot with volumesnapshotContent %s/%s\",\n\t\t\tvs.Namespace, vs.Name)\n\t}\n}\n\nfunc DeleteReadyVolumeSnapshot(\n\tvs snapshotv1api.VolumeSnapshot,\n\tclient crclient.Client,\n\tlogger logrus.FieldLogger,\n) {\n\tlogger.Infof(\"Deleting Volumesnapshot %s/%s\", vs.Namespace, vs.Name)\n\tif vs.Status == nil ||\n\t\tvs.Status.BoundVolumeSnapshotContentName == nil ||\n\t\tlen(*vs.Status.BoundVolumeSnapshotContentName) <= 0 {\n\t\tlogger.Errorf(\"VolumeSnapshot %s/%s is not ready. This is not expected.\",\n\t\t\tvs.Namespace, vs.Name)\n\t\treturn\n\t}\n\n\tvar vsc *snapshotv1api.VolumeSnapshotContent\n\n\tif vs.Status != nil && vs.Status.BoundVolumeSnapshotContentName != nil {\n\t\tvar err error\n\n\t\t// Patch the DeletionPolicy of the VolumeSnapshotContent to set it to Retain.\n\t\t// This ensures that the volume snapshot in the storage provider is kept.\n\t\tif vsc, err = SetVolumeSnapshotContentDeletionPolicy(\n\t\t\t*vs.Status.BoundVolumeSnapshotContentName,\n\t\t\tclient,\n\t\t\tsnapshotv1api.VolumeSnapshotContentRetain,\n\t\t); err != nil {\n\t\t\tlogger.Warnf(\"Failed to patch DeletionPolicy of VolumeSnapshot %s/%s\",\n\t\t\t\tvs.Namespace, vs.Name)\n\t\t\treturn\n\t\t}\n\n\t\tif err := client.Delete(context.TODO(), vsc); err != nil {\n\t\t\tlogger.WithError(err).Warnf(\"Failed to delete the VolumeSnapshotContent %s\", vsc.Name)\n\t\t}\n\t}\n\tif err := client.Delete(context.TODO(), &vs); err != nil {\n\t\tlogger.WithError(err).Warnf(\"Failed to delete VolumeSnapshot %s\", vs.Namespace+\"/\"+vs.Name)\n\t} else {\n\t\tlogger.Infof(\"Deleted VolumeSnapshot %s and VolumeSnapshotContent %s\",\n\t\t\tvs.Namespace+\"/\"+vs.Name, vsc.Name)\n\t}\n}\n\n// WaitUntilVSCHandleIsReady returns the VolumeSnapshotContent\n// object associated with the volumesnapshot\nfunc WaitUntilVSCHandleIsReady(\n\tvolSnap *snapshotv1api.VolumeSnapshot,\n\tcrClient crclient.Client,\n\tlog logrus.FieldLogger,\n\tcsiSnapshotTimeout time.Duration,\n) (*snapshotv1api.VolumeSnapshotContent, error) {\n\t// We'll wait 10m for the VSC to be reconciled polling\n\t// every 5s unless backup's csiSnapshotTimeout is set\n\tinterval := 5 * time.Second\n\tvsc := new(snapshotv1api.VolumeSnapshotContent)\n\n\terr := wait.PollUntilContextTimeout(\n\t\tcontext.Background(),\n\t\tinterval,\n\t\tcsiSnapshotTimeout,\n\t\ttrue,\n\t\tfunc(ctx context.Context) (bool, error) {\n\t\t\tvs := new(snapshotv1api.VolumeSnapshot)\n\t\t\tif err := crClient.Get(\n\t\t\t\tctx,\n\t\t\t\tcrclient.ObjectKeyFromObject(volSnap),\n\t\t\t\tvs,\n\t\t\t); err != nil {\n\t\t\t\treturn false,\n\t\t\t\t\terrors.Wrapf(\n\t\t\t\t\t\terr,\n\t\t\t\t\t\t\"failed to get volumesnapshot %s/%s\",\n\t\t\t\t\t\tvolSnap.Namespace, volSnap.Name,\n\t\t\t\t\t)\n\t\t\t}\n\n\t\t\tif vs.Status == nil || vs.Status.BoundVolumeSnapshotContentName == nil {\n\t\t\t\tlog.Infof(\"Waiting for CSI driver to reconcile volumesnapshot %s/%s. Retrying in %ds\",\n\t\t\t\t\tvolSnap.Namespace, volSnap.Name, interval/time.Second)\n\t\t\t\treturn false, nil\n\t\t\t}\n\n\t\t\tif err := crClient.Get(\n\t\t\t\tctx,\n\t\t\t\tcrclient.ObjectKey{\n\t\t\t\t\tName: *vs.Status.BoundVolumeSnapshotContentName,\n\t\t\t\t},\n\t\t\t\tvsc,\n\t\t\t); err != nil {\n\t\t\t\treturn false,\n\t\t\t\t\terrors.Wrapf(\n\t\t\t\t\t\terr,\n\t\t\t\t\t\t\"failed to get VolumeSnapshotContent %s for VolumeSnapshot %s/%s\",\n\t\t\t\t\t\t*vs.Status.BoundVolumeSnapshotContentName, vs.Namespace, vs.Name,\n\t\t\t\t\t)\n\t\t\t}\n\n\t\t\t// we need to wait for the VolumeSnapshotContent\n\t\t\t// to have a snapshot handle because during restore,\n\t\t\t// we'll use that snapshot handle as the source for\n\t\t\t// the VolumeSnapshotContent so it's statically\n\t\t\t// bound to the existing snapshot.\n\t\t\tif vsc.Status == nil ||\n\t\t\t\tvsc.Status.SnapshotHandle == nil {\n\t\t\t\tlog.Infof(\n\t\t\t\t\t\"Waiting for VolumeSnapshotContents %s to have snapshot handle. Retrying in %ds\",\n\t\t\t\t\tvsc.Name, interval/time.Second)\n\t\t\t\tif vsc.Status != nil &&\n\t\t\t\t\tvsc.Status.Error != nil {\n\t\t\t\t\tlog.Warnf(\"VolumeSnapshotContent %s has error: %v\",\n\t\t\t\t\t\tvsc.Name, *vsc.Status.Error.Message)\n\t\t\t\t}\n\t\t\t\treturn false, nil\n\t\t\t}\n\n\t\t\treturn true, nil\n\t\t},\n\t)\n\n\tif err != nil {\n\t\tif wait.Interrupted(err) {\n\t\t\tif vsc != nil &&\n\t\t\t\tvsc.Status != nil &&\n\t\t\t\tvsc.Status.Error != nil {\n\t\t\t\tlog.Errorf(\n\t\t\t\t\t\"Timed out awaiting reconciliation of VolumeSnapshot, VolumeSnapshotContent %s has error: %v\",\n\t\t\t\t\tvsc.Name, *vsc.Status.Error.Message)\n\t\t\t\treturn nil,\n\t\t\t\t\terrors.Errorf(\"CSI got timed out with error: %v\",\n\t\t\t\t\t\t*vsc.Status.Error.Message)\n\t\t\t} else {\n\t\t\t\tlog.Errorf(\n\t\t\t\t\t\"Timed out awaiting reconciliation of volumesnapshot %s/%s\",\n\t\t\t\t\tvolSnap.Namespace, volSnap.Name)\n\t\t\t}\n\t\t}\n\t\treturn nil, err\n\t}\n\n\treturn vsc, nil\n}\n\nfunc DiagnoseVS(vs *snapshotv1api.VolumeSnapshot, events *corev1api.EventList) string {\n\tvscName := \"\"\n\treadyToUse := false\n\terrMessage := \"\"\n\n\tif vs.Status != nil {\n\t\tif vs.Status.BoundVolumeSnapshotContentName != nil {\n\t\t\tvscName = *vs.Status.BoundVolumeSnapshotContentName\n\t\t}\n\n\t\tif vs.Status.ReadyToUse != nil {\n\t\t\treadyToUse = *vs.Status.ReadyToUse\n\t\t}\n\n\t\tif vs.Status.Error != nil && vs.Status.Error.Message != nil {\n\t\t\terrMessage = *vs.Status.Error.Message\n\t\t}\n\t}\n\n\tdiag := fmt.Sprintf(\"VS %s/%s, bind to %s, readyToUse %v, errMessage %s\\n\", vs.Namespace, vs.Name, vscName, readyToUse, errMessage)\n\n\tif events != nil {\n\t\tfor _, e := range events.Items {\n\t\t\tif e.InvolvedObject.UID == vs.UID && e.Type == corev1api.EventTypeWarning {\n\t\t\t\tdiag += fmt.Sprintf(\"VS event reason %s, message %s\\n\", e.Reason, e.Message)\n\t\t\t}\n\t\t}\n\t}\n\n\treturn diag\n}\n\nfunc DiagnoseVSC(vsc *snapshotv1api.VolumeSnapshotContent) string {\n\thandle := \"\"\n\treadyToUse := false\n\terrMessage := \"\"\n\n\tif vsc.Status != nil {\n\t\tif vsc.Status.SnapshotHandle != nil {\n\t\t\thandle = *vsc.Status.SnapshotHandle\n\t\t}\n\n\t\tif vsc.Status.ReadyToUse != nil {\n\t\t\treadyToUse = *vsc.Status.ReadyToUse\n\t\t}\n\n\t\tif vsc.Status.Error != nil && vsc.Status.Error.Message != nil {\n\t\t\terrMessage = *vsc.Status.Error.Message\n\t\t}\n\t}\n\n\tdiag := fmt.Sprintf(\"VSC %s, readyToUse %v, errMessage %s, handle %s\\n\", vsc.Name, readyToUse, errMessage, handle)\n\n\treturn diag\n}\n\n// GetVSCForVS returns the VolumeSnapshotContent object associated with the VolumeSnapshot.\nfunc GetVSCForVS(\n\tctx context.Context,\n\tvs *snapshotv1api.VolumeSnapshot,\n\tclient crclient.Client,\n) (*snapshotv1api.VolumeSnapshotContent, error) {\n\tif vs.Status == nil || vs.Status.BoundVolumeSnapshotContentName == nil {\n\t\treturn nil, errors.Errorf(\"invalid snapshot info in volume snapshot %s\", vs.Name)\n\t}\n\n\tvsc := new(snapshotv1api.VolumeSnapshotContent)\n\n\tif err := client.Get(\n\t\tctx,\n\t\tcrclient.ObjectKey{\n\t\t\tName: *vs.Status.BoundVolumeSnapshotContentName,\n\t\t},\n\t\tvsc,\n\t); err != nil {\n\t\treturn nil, errors.Wrap(err, \"error getting volume snapshot content from API\")\n\t}\n\n\treturn vsc, nil\n}\n" }, { "path": "pkg/util/csi/volume_snapshot_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage csi\n\nimport (\n\t\"errors\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/google/go-cmp/cmp\"\n\t\"github.com/google/go-cmp/cmp/cmpopts\"\n\tsnapshotv1api \"github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1\"\n\tsnapshotFake \"github.com/kubernetes-csi/external-snapshotter/client/v8/clientset/versioned/fake\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\t\"k8s.io/apimachinery/pkg/api/resource\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\tclientTesting \"k8s.io/client-go/testing\"\n\tcrclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/test\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/logging\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/stringptr\"\n)\n\ntype reactor struct {\n\tverb string\n\tresource string\n\treactorFunc clientTesting.ReactionFunc\n}\n\n// expected: &v1.VolumeSnapshot{TypeMeta:v1.TypeMeta{Kind:\"\", APIVersion:\"\"}, ObjectMeta:v1.ObjectMeta{Name:\"fake-vs\", GenerateName:\"\", Namespace:\"fake-ns\", SelfLink:\"\", UID:\"\", ResourceVersion:\"999\", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.VolumeSnapshotSpec{Source:v1.VolumeSnapshotSource{PersistentVolumeClaimName:(*string)(nil), VolumeSnapshotContentName:(*string)(nil)}, VolumeSnapshotClassName:(*string)(nil)}, Status:(*v1.VolumeSnapshotStatus)(0x140000af8c0)}\n// actual : &v1.VolumeSnapshot{TypeMeta:v1.TypeMeta{Kind:\"\", APIVersion:\"\"}, ObjectMeta:v1.ObjectMeta{Name:\"fake-vs\", GenerateName:\"\", Namespace:\"fake-ns\", SelfLink:\"\", UID:\"\", ResourceVersion:\"999\", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.VolumeSnapshotSpec{Source:v1.VolumeSnapshotSource{PersistentVolumeClaimName:(*string)(nil), VolumeSnapshotContentName:(*string)(nil)}, VolumeSnapshotClassName:(*string)(nil)}, Status:(*v1.VolumeSnapshotStatus)(0x1400024ed20)}\n\nfunc TestWaitVolumeSnapshotReady(t *testing.T) {\n\tvscName := \"fake-vsc\"\n\tquantity := resource.MustParse(\"0\")\n\tvsObj := &snapshotv1api.VolumeSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-vs\",\n\t\t\tNamespace: \"fake-ns\",\n\t\t},\n\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\tBoundVolumeSnapshotContentName: &vscName,\n\t\t\tReadyToUse: boolptr.True(),\n\t\t\tRestoreSize: &quantity,\n\t\t},\n\t}\n\n\terrMessage := \"fake-snapshot-creation-error\"\n\n\ttests := []struct {\n\t\tname string\n\t\tclientObj []runtime.Object\n\t\tvsName string\n\t\tnamespace string\n\t\terr string\n\t\texpect *snapshotv1api.VolumeSnapshot\n\t}{\n\t\t{\n\t\t\tname: \"get vs error\",\n\t\t\tvsName: \"fake-vs-1\",\n\t\t\tnamespace: \"fake-ns-1\",\n\t\t\terr: \"error to get VolumeSnapshot fake-ns-1/fake-vs-1: volumesnapshots.snapshot.storage.k8s.io \\\"fake-vs-1\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"vs status is nil\",\n\t\t\tvsName: \"fake-vs\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tclientObj: []runtime.Object{\n\t\t\t\t&snapshotv1api.VolumeSnapshot{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"fake-vs\",\n\t\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"volume snapshot is not ready until timeout, errors: []\",\n\t\t},\n\t\t{\n\t\t\tname: \"vsc is nil in status\",\n\t\t\tvsName: \"fake-vs\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tclientObj: []runtime.Object{\n\t\t\t\t&snapshotv1api.VolumeSnapshot{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"fake-vs\",\n\t\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"volume snapshot is not ready until timeout, errors: []\",\n\t\t},\n\t\t{\n\t\t\tname: \"ready to use is nil in status\",\n\t\t\tvsName: \"fake-vs\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tclientObj: []runtime.Object{\n\t\t\t\t&snapshotv1api.VolumeSnapshot{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"fake-vs\",\n\t\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\t\t\t\tBoundVolumeSnapshotContentName: &vscName,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"volume snapshot is not ready until timeout, errors: []\",\n\t\t},\n\t\t{\n\t\t\tname: \"ready to use is false\",\n\t\t\tvsName: \"fake-vs\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tclientObj: []runtime.Object{\n\t\t\t\t&snapshotv1api.VolumeSnapshot{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"fake-vs\",\n\t\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\t\t\t\tBoundVolumeSnapshotContentName: &vscName,\n\t\t\t\t\t\tReadyToUse: boolptr.False(),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"volume snapshot is not ready until timeout, errors: []\",\n\t\t},\n\t\t{\n\t\t\tname: \"snapshot creation error with message\",\n\t\t\tvsName: \"fake-vs\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tclientObj: []runtime.Object{\n\t\t\t\t&snapshotv1api.VolumeSnapshot{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"fake-vs\",\n\t\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\t\t\t\tError: &snapshotv1api.VolumeSnapshotError{\n\t\t\t\t\t\t\tMessage: &errMessage,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"volume snapshot is not ready until timeout, errors: [fake-snapshot-creation-error]\",\n\t\t},\n\t\t{\n\t\t\tname: \"snapshot creation error without message\",\n\t\t\tvsName: \"fake-vs\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tclientObj: []runtime.Object{\n\t\t\t\t&snapshotv1api.VolumeSnapshot{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"fake-vs\",\n\t\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\t},\n\t\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\t\t\t\tError: &snapshotv1api.VolumeSnapshotError{},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"volume snapshot is not ready until timeout, errors: [\" + stringptr.NilString + \"]\",\n\t\t},\n\t\t{\n\t\t\tname: \"success\",\n\t\t\tvsName: \"fake-vs\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tclientObj: []runtime.Object{\n\t\t\t\tvsObj,\n\t\t\t},\n\t\t\texpect: vsObj,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeClient := snapshotFake.NewSimpleClientset(test.clientObj...)\n\n\t\t\tvs, err := WaitVolumeSnapshotReady(t.Context(), fakeClient.SnapshotV1(), test.vsName, test.namespace, time.Millisecond, velerotest.NewLogger())\n\t\t\tif err != nil {\n\t\t\t\trequire.EqualError(t, err, test.err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tassert.Equal(t, test.expect, vs)\n\t\t})\n\t}\n}\n\nfunc TestGetVolumeSnapshotContentForVolumeSnapshot(t *testing.T) {\n\tvscName := \"fake-vsc\"\n\tvsObj := &snapshotv1api.VolumeSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-vs\",\n\t\t\tNamespace: \"fake-ns\",\n\t\t},\n\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\tBoundVolumeSnapshotContentName: &vscName,\n\t\t\tReadyToUse: boolptr.True(),\n\t\t\tRestoreSize: &resource.Quantity{},\n\t\t},\n\t}\n\n\tvscObj := &snapshotv1api.VolumeSnapshotContent{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-vsc\",\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tsnapshotObj *snapshotv1api.VolumeSnapshot\n\t\tclientObj []runtime.Object\n\t\tvsName string\n\t\tnamespace string\n\t\terr string\n\t\texpect *snapshotv1api.VolumeSnapshotContent\n\t}{\n\t\t{\n\t\t\tname: \"vs status is nil\",\n\t\t\tvsName: \"fake-vs\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tsnapshotObj: &snapshotv1api.VolumeSnapshot{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vs\",\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"invalid snapshot info in volume snapshot fake-vs\",\n\t\t},\n\t\t{\n\t\t\tname: \"vsc is nil in status\",\n\t\t\tvsName: \"fake-vs\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tsnapshotObj: &snapshotv1api.VolumeSnapshot{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vs\",\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"invalid snapshot info in volume snapshot fake-vs\",\n\t\t},\n\t\t{\n\t\t\tname: \"get vsc fail\",\n\t\t\tvsName: \"fake-vs\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tsnapshotObj: vsObj,\n\t\t\terr: \"error getting volume snapshot content from API: volumesnapshotcontents.snapshot.storage.k8s.io \\\"fake-vsc\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"success\",\n\t\t\tvsName: \"fake-vs\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tsnapshotObj: vsObj,\n\t\t\tclientObj: []runtime.Object{vscObj},\n\t\t\texpect: vscObj,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeClient := snapshotFake.NewSimpleClientset(test.clientObj...)\n\n\t\t\tvs, err := GetVolumeSnapshotContentForVolumeSnapshot(test.snapshotObj, fakeClient.SnapshotV1())\n\t\t\tif err != nil {\n\t\t\t\trequire.EqualError(t, err, test.err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tassert.Equal(t, test.expect, vs)\n\t\t})\n\t}\n}\n\nfunc TestEnsureDeleteVS(t *testing.T) {\n\tvsObj := &snapshotv1api.VolumeSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-vs\",\n\t\t\tNamespace: \"fake-ns\",\n\t\t},\n\t}\n\n\tvsObjWithFinalizer := &snapshotv1api.VolumeSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-vs\",\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tFinalizers: []string{\"fake-finalizer-1\", \"fake-finalizer-2\"},\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tclientObj []runtime.Object\n\t\tvsName string\n\t\tnamespace string\n\t\treactors []reactor\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"delete fail\",\n\t\t\tvsName: \"fake-vs\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\terr: \"error to delete volume snapshot: volumesnapshots.snapshot.storage.k8s.io \\\"fake-vs\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"wait fail\",\n\t\t\tvsName: \"fake-vs\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tclientObj: []runtime.Object{vsObj},\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"get\",\n\t\t\t\t\tresource: \"volumesnapshots\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-get-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to assure VolumeSnapshot is deleted, fake-vs: error to get VolumeSnapshot fake-vs: fake-get-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"wait timeout\",\n\t\t\tvsName: \"fake-vs\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tclientObj: []runtime.Object{vsObjWithFinalizer},\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"volumesnapshots\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"timeout to assure VolumeSnapshot fake-vs is deleted, finalizers in VS [fake-finalizer-1 fake-finalizer-2]\",\n\t\t},\n\t\t{\n\t\t\tname: \"wait timeout, no finalizer\",\n\t\t\tvsName: \"fake-vs\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tclientObj: []runtime.Object{vsObj},\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"volumesnapshots\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"timeout to assure VolumeSnapshot fake-vs is deleted, finalizers in VS []\",\n\t\t},\n\t\t{\n\t\t\tname: \"success\",\n\t\t\tvsName: \"fake-vs\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tclientObj: []runtime.Object{vsObj},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeSnapshotClient := snapshotFake.NewSimpleClientset(test.clientObj...)\n\n\t\t\tfor _, reactor := range test.reactors {\n\t\t\t\tfakeSnapshotClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\terr := EnsureDeleteVS(t.Context(), fakeSnapshotClient.SnapshotV1(), test.vsName, test.namespace, time.Millisecond)\n\t\t\tif err != nil {\n\t\t\t\tassert.EqualError(t, err, test.err)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestEnsureDeleteVSC(t *testing.T) {\n\tvscObj := &snapshotv1api.VolumeSnapshotContent{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-vsc\",\n\t\t},\n\t}\n\n\tvscObjWithFinalizer := &snapshotv1api.VolumeSnapshotContent{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-vsc\",\n\t\t\tFinalizers: []string{\"fake-finalizer-1\", \"fake-finalizer-2\"},\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tclientObj []runtime.Object\n\t\treactors []reactor\n\t\tvscName string\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"delete fail on VSC not found\",\n\t\t\tvscName: \"fake-vsc\",\n\t\t},\n\t\t{\n\t\t\tname: \"delete fail on others\",\n\t\t\tvscName: \"fake-vsc\",\n\t\t\tclientObj: []runtime.Object{vscObj},\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"volumesnapshotcontents\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-delete-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to delete volume snapshot content: fake-delete-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"wait fail\",\n\t\t\tvscName: \"fake-vsc\",\n\t\t\tclientObj: []runtime.Object{vscObj},\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"get\",\n\t\t\t\t\tresource: \"volumesnapshotcontents\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-get-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to assure VolumeSnapshotContent is deleted, fake-vsc: error to get VolumeSnapshotContent fake-vsc: fake-get-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"wait timeout\",\n\t\t\tvscName: \"fake-vsc\",\n\t\t\tclientObj: []runtime.Object{vscObjWithFinalizer},\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"volumesnapshotcontents\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"timeout to assure VolumeSnapshotContent fake-vsc is deleted, finalizers in VSC [fake-finalizer-1 fake-finalizer-2]\",\n\t\t},\n\t\t{\n\t\t\tname: \"wait timeout, no finalizer\",\n\t\t\tvscName: \"fake-vsc\",\n\t\t\tclientObj: []runtime.Object{vscObj},\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"volumesnapshotcontents\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"timeout to assure VolumeSnapshotContent fake-vsc is deleted, finalizers in VSC []\",\n\t\t},\n\t\t{\n\t\t\tname: \"success\",\n\t\t\tvscName: \"fake-vsc\",\n\t\t\tclientObj: []runtime.Object{vscObj},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeSnapshotClient := snapshotFake.NewSimpleClientset(test.clientObj...)\n\n\t\t\tfor _, reactor := range test.reactors {\n\t\t\t\tfakeSnapshotClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\terr := EnsureDeleteVSC(t.Context(), fakeSnapshotClient.SnapshotV1(), test.vscName, time.Millisecond)\n\t\t\tif test.err != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.err)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestDeleteVolumeSnapshotContentIfAny(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tclientObj []runtime.Object\n\t\treactors []reactor\n\t\tvscName string\n\t\tlogMessage string\n\t\tlogLevel string\n\t\tlogError string\n\t}{\n\t\t{\n\t\t\tname: \"vsc not exist\",\n\t\t\tvscName: \"fake-vsc\",\n\t\t\tlogMessage: \"Abort deleting VSC, it doesn't exist fake-vsc\",\n\t\t\tlogLevel: \"level=debug\",\n\t\t},\n\t\t{\n\t\t\tname: \"deleete fail\",\n\t\t\tvscName: \"fake-vsc\",\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"volumesnapshotcontents\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-delete-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tlogMessage: \"Failed to delete volume snapshot content fake-vsc\",\n\t\t\tlogLevel: \"level=error\",\n\t\t\tlogError: \"error=fake-delete-error\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeSnapshotClient := snapshotFake.NewSimpleClientset(test.clientObj...)\n\n\t\t\tfor _, reactor := range test.reactors {\n\t\t\t\tfakeSnapshotClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\tlogMessage := \"\"\n\n\t\t\tDeleteVolumeSnapshotContentIfAny(t.Context(), fakeSnapshotClient.SnapshotV1(), test.vscName, velerotest.NewSingleLogger(&logMessage))\n\n\t\t\tif len(test.logMessage) > 0 {\n\t\t\t\tassert.Contains(t, logMessage, test.logMessage)\n\t\t\t}\n\n\t\t\tif len(test.logLevel) > 0 {\n\t\t\t\tassert.Contains(t, logMessage, test.logLevel)\n\t\t\t}\n\n\t\t\tif len(test.logError) > 0 {\n\t\t\t\tassert.Contains(t, logMessage, test.logError)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestDeleteVolumeSnapshotIfAny(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tclientObj []runtime.Object\n\t\treactors []reactor\n\t\tvsName string\n\t\tvsNamespace string\n\t\tlogMessage string\n\t\tlogLevel string\n\t\tlogError string\n\t}{\n\t\t{\n\t\t\tname: \"vs not exist\",\n\t\t\tvsName: \"fake-vs\",\n\t\t\tvsNamespace: \"fake-ns\",\n\t\t\tlogMessage: \"Abort deleting volume snapshot, it doesn't exist fake-ns/fake-vs\",\n\t\t\tlogLevel: \"level=debug\",\n\t\t},\n\t\t{\n\t\t\tname: \"delete fail\",\n\t\t\tvsName: \"fake-vs\",\n\t\t\tvsNamespace: \"fake-ns\",\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"volumesnapshots\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-delete-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tlogMessage: \"Failed to delete volume snapshot fake-ns/fake-vs\",\n\t\t\tlogLevel: \"level=error\",\n\t\t\tlogError: \"error=fake-delete-error\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeSnapshotClient := snapshotFake.NewSimpleClientset(test.clientObj...)\n\n\t\t\tfor _, reactor := range test.reactors {\n\t\t\t\tfakeSnapshotClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\tlogMessage := \"\"\n\n\t\t\tDeleteVolumeSnapshotIfAny(t.Context(), fakeSnapshotClient.SnapshotV1(), test.vsName, test.vsNamespace, velerotest.NewSingleLogger(&logMessage))\n\n\t\t\tif len(test.logMessage) > 0 {\n\t\t\t\tassert.Contains(t, logMessage, test.logMessage)\n\t\t\t}\n\n\t\t\tif len(test.logLevel) > 0 {\n\t\t\t\tassert.Contains(t, logMessage, test.logLevel)\n\t\t\t}\n\n\t\t\tif len(test.logError) > 0 {\n\t\t\t\tassert.Contains(t, logMessage, test.logError)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestRetainVSC(t *testing.T) {\n\tvscObj := &snapshotv1api.VolumeSnapshotContent{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-vsc\",\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tclientObj []runtime.Object\n\t\treactors []reactor\n\t\tvsc *snapshotv1api.VolumeSnapshotContent\n\t\tupdated *snapshotv1api.VolumeSnapshotContent\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"already retained\",\n\t\t\tvsc: &snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vsc\",\n\t\t\t\t},\n\t\t\t\tSpec: snapshotv1api.VolumeSnapshotContentSpec{\n\t\t\t\t\tDeletionPolicy: snapshotv1api.VolumeSnapshotContentRetain,\n\t\t\t\t},\n\t\t\t},\n\t\t\tupdated: &snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vsc\",\n\t\t\t\t},\n\t\t\t\tSpec: snapshotv1api.VolumeSnapshotContentSpec{\n\t\t\t\t\tDeletionPolicy: snapshotv1api.VolumeSnapshotContentRetain,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"path vsc fail\",\n\t\t\tvsc: &snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vsc\",\n\t\t\t\t},\n\t\t\t\tSpec: snapshotv1api.VolumeSnapshotContentSpec{\n\t\t\t\t\tDeletionPolicy: snapshotv1api.VolumeSnapshotContentDelete,\n\t\t\t\t},\n\t\t\t},\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"patch\",\n\t\t\t\t\tresource: \"volumesnapshotcontents\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-patch-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error patching VSC: fake-patch-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"success\",\n\t\t\tvsc: &snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vsc\",\n\t\t\t\t},\n\t\t\t\tSpec: snapshotv1api.VolumeSnapshotContentSpec{\n\t\t\t\t\tDeletionPolicy: snapshotv1api.VolumeSnapshotContentDelete,\n\t\t\t\t},\n\t\t\t},\n\t\t\tclientObj: []runtime.Object{vscObj},\n\t\t\tupdated: &snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vsc\",\n\t\t\t\t},\n\t\t\t\tSpec: snapshotv1api.VolumeSnapshotContentSpec{\n\t\t\t\t\tDeletionPolicy: snapshotv1api.VolumeSnapshotContentRetain,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeSnapshotClient := snapshotFake.NewSimpleClientset(test.clientObj...)\n\n\t\t\tfor _, reactor := range test.reactors {\n\t\t\t\tfakeSnapshotClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\treturned, err := RetainVSC(t.Context(), fakeSnapshotClient.SnapshotV1(), test.vsc)\n\n\t\t\tif len(test.err) == 0 {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t} else {\n\t\t\t\trequire.EqualError(t, err, test.err)\n\t\t\t}\n\n\t\t\tif test.updated != nil {\n\t\t\t\tassert.Equal(t, *test.updated, *returned)\n\t\t\t} else {\n\t\t\t\tassert.Nil(t, returned)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestRemoveVSCProtect(t *testing.T) {\n\tvscObj := &snapshotv1api.VolumeSnapshotContent{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-vsc\",\n\t\t\tFinalizers: []string{volumeSnapshotContentProtectFinalizer},\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tclientObj []runtime.Object\n\t\treactors []reactor\n\t\tvsc string\n\t\tupdated *snapshotv1api.VolumeSnapshotContent\n\t\ttimeout time.Duration\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"get vsc error\",\n\t\t\tvsc: \"fake-vsc\",\n\t\t\terr: \"error to get VolumeSnapshotContent fake-vsc: volumesnapshotcontents.snapshot.storage.k8s.io \\\"fake-vsc\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"update vsc fail\",\n\t\t\tvsc: \"fake-vsc\",\n\t\t\tclientObj: []runtime.Object{vscObj},\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"update\",\n\t\t\t\t\tresource: \"volumesnapshotcontents\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-update-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to update VolumeSnapshotContent fake-vsc: fake-update-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"update vsc timeout\",\n\t\t\tvsc: \"fake-vsc\",\n\t\t\tclientObj: []runtime.Object{vscObj},\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"update\",\n\t\t\t\t\tresource: \"volumesnapshotcontents\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, &apierrors.StatusError{ErrStatus: metav1.Status{\n\t\t\t\t\t\t\tReason: metav1.StatusReasonConflict,\n\t\t\t\t\t\t}}\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\ttimeout: time.Second,\n\t\t\terr: \"context deadline exceeded\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tvsc: \"fake-vsc\",\n\t\t\tclientObj: []runtime.Object{vscObj},\n\t\t\ttimeout: time.Second,\n\t\t\tupdated: &snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vsc\",\n\t\t\t\t\tFinalizers: []string{},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeSnapshotClient := snapshotFake.NewSimpleClientset(test.clientObj...)\n\n\t\t\tfor _, reactor := range test.reactors {\n\t\t\t\tfakeSnapshotClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\terr := RemoveVSCProtect(t.Context(), fakeSnapshotClient.SnapshotV1(), test.vsc, test.timeout)\n\n\t\t\tif len(test.err) == 0 {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t} else {\n\t\t\t\trequire.EqualError(t, err, test.err)\n\t\t\t}\n\n\t\t\tif test.updated != nil {\n\t\t\t\tupdated, err := fakeSnapshotClient.SnapshotV1().VolumeSnapshotContents().Get(t.Context(), test.vsc, metav1.GetOptions{})\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\tassert.Equal(t, test.updated.Finalizers, updated.Finalizers)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetVolumeSnapshotClass(t *testing.T) {\n\t// backups\n\tbackupFoo := &velerov1api.Backup{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"foo\",\n\t\t\tAnnotations: map[string]string{\n\t\t\t\t\"velero.io/csi-volumesnapshot-class_foo.csi.k8s.io\": \"foowithoutlabel\",\n\t\t\t},\n\t\t},\n\t\tSpec: velerov1api.BackupSpec{\n\t\t\tIncludedNamespaces: []string{\"ns1\", \"ns2\"},\n\t\t},\n\t}\n\tbackupFoo2 := &velerov1api.Backup{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"foo2\",\n\t\t\tAnnotations: map[string]string{\n\t\t\t\t\"velero.io/csi-volumesnapshot-class_foo.csi.k8s.io\": \"foo2\",\n\t\t\t},\n\t\t},\n\t\tSpec: velerov1api.BackupSpec{\n\t\t\tIncludedNamespaces: []string{\"ns1\", \"ns2\"},\n\t\t},\n\t}\n\n\tbackupBar2 := &velerov1api.Backup{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"bar\",\n\t\t\tAnnotations: map[string]string{\n\t\t\t\t\"velero.io/csi-volumesnapshot-class_bar.csi.k8s.io\": \"bar2\",\n\t\t\t},\n\t\t},\n\t\tSpec: velerov1api.BackupSpec{\n\t\t\tIncludedNamespaces: []string{\"ns1\", \"ns2\"},\n\t\t},\n\t}\n\n\tbackupNone := &velerov1api.Backup{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"none\",\n\t\t},\n\t\tSpec: velerov1api.BackupSpec{\n\t\t\tIncludedNamespaces: []string{\"ns1\", \"ns2\"},\n\t\t},\n\t}\n\n\t// pvcs\n\tpvcFoo := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"foo\",\n\t\t\tAnnotations: map[string]string{\n\t\t\t\t\"velero.io/csi-volumesnapshot-class\": \"foowithoutlabel\",\n\t\t\t},\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{},\n\t}\n\tpvcFoo2 := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"foo\",\n\t\t\tAnnotations: map[string]string{\n\t\t\t\t\"velero.io/csi-volumesnapshot-class\": \"foo2\",\n\t\t\t},\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{},\n\t}\n\tpvcNone := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"none\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{},\n\t}\n\n\t// vsclasses\n\thostpathClass := &snapshotv1api.VolumeSnapshotClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"hostpath\",\n\t\t\tLabels: map[string]string{\n\t\t\t\tvelerov1api.VolumeSnapshotClassSelectorLabel: \"foo\",\n\t\t\t},\n\t\t},\n\t\tDriver: \"hostpath.csi.k8s.io\",\n\t}\n\n\tfooClass := &snapshotv1api.VolumeSnapshotClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"foo\",\n\t\t\tLabels: map[string]string{\n\t\t\t\tvelerov1api.VolumeSnapshotClassSelectorLabel: \"foo\",\n\t\t\t},\n\t\t},\n\t\tDriver: \"foo.csi.k8s.io\",\n\t}\n\tfooClassWithoutLabel := &snapshotv1api.VolumeSnapshotClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"foowithoutlabel\",\n\t\t},\n\t\tDriver: \"foo.csi.k8s.io\",\n\t}\n\n\tbarClass := &snapshotv1api.VolumeSnapshotClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"bar\",\n\t\t\tLabels: map[string]string{\n\t\t\t\tvelerov1api.VolumeSnapshotClassSelectorLabel: \"true\",\n\t\t\t},\n\t\t},\n\t\tDriver: \"bar.csi.k8s.io\",\n\t}\n\n\tbarClass2 := &snapshotv1api.VolumeSnapshotClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"bar2\",\n\t\t\tLabels: map[string]string{\n\t\t\t\tvelerov1api.VolumeSnapshotClassSelectorLabel: \"true\",\n\t\t\t},\n\t\t},\n\t\tDriver: \"bar.csi.k8s.io\",\n\t}\n\n\tobjs := []runtime.Object{hostpathClass, fooClass, barClass, fooClassWithoutLabel, barClass2}\n\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, objs...)\n\n\ttestCases := []struct {\n\t\tname string\n\t\tdriverName string\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\tbackup *velerov1api.Backup\n\t\texpectedVSC *snapshotv1api.VolumeSnapshotClass\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"no annotations on pvc and backup, should find hostpath volumesnapshotclass using default behavior of labels\",\n\t\t\tdriverName: \"hostpath.csi.k8s.io\",\n\t\t\tpvc: pvcNone,\n\t\t\tbackup: backupNone,\n\t\t\texpectedVSC: hostpathClass,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"foowithoutlabel VSC annotations on pvc\",\n\t\t\tdriverName: \"foo.csi.k8s.io\",\n\t\t\tpvc: pvcFoo,\n\t\t\tbackup: backupNone,\n\t\t\texpectedVSC: fooClassWithoutLabel,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"foowithoutlabel VSC annotations on pvc, but csi driver does not match, no annotation on backup so fallback to default behavior of labels\",\n\t\t\tdriverName: \"bar.csi.k8s.io\",\n\t\t\tpvc: pvcFoo,\n\t\t\tbackup: backupNone,\n\t\t\texpectedVSC: barClass,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"foowithoutlabel VSC annotations on pvc, but csi driver does not match so fallback to fetch from backupAnnotations \",\n\t\t\tdriverName: \"bar.csi.k8s.io\",\n\t\t\tpvc: pvcFoo,\n\t\t\tbackup: backupBar2,\n\t\t\texpectedVSC: barClass2,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"foowithoutlabel VSC annotations on backup for foo.csi.k8s.io\",\n\t\t\tdriverName: \"foo.csi.k8s.io\",\n\t\t\tpvc: pvcNone,\n\t\t\tbackup: backupFoo,\n\t\t\texpectedVSC: fooClassWithoutLabel,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"foowithoutlabel VSC annotations on backup for bar.csi.k8s.io, no annotation corresponding to foo.csi.k8s.io, so fallback to default behavior of labels\",\n\t\t\tdriverName: \"bar.csi.k8s.io\",\n\t\t\tpvc: pvcNone,\n\t\t\tbackup: backupFoo,\n\t\t\texpectedVSC: barClass,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"no snapshotClass for given driver\",\n\t\t\tdriverName: \"blah.csi.k8s.io\",\n\t\t\tpvc: pvcNone,\n\t\t\tbackup: backupNone,\n\t\t\texpectedVSC: nil,\n\t\t\texpectError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"foo2 VSC annotations on pvc, but doesn't exist in cluster, fallback to default behavior of labels\",\n\t\t\tdriverName: \"foo.csi.k8s.io\",\n\t\t\tpvc: pvcFoo2,\n\t\t\tbackup: backupFoo2,\n\t\t\texpectedVSC: fooClass,\n\t\t\texpectError: false,\n\t\t},\n\t}\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactualSnapshotClass, actualError := GetVolumeSnapshotClass(\n\t\t\t\ttc.driverName, tc.backup, tc.pvc, logrus.New(), fakeClient)\n\t\t\tif tc.expectError {\n\t\t\t\trequire.Error(t, actualError)\n\t\t\t\tassert.Nil(t, actualSnapshotClass)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tassert.Equal(t, tc.expectedVSC, actualSnapshotClass)\n\t\t})\n\t}\n}\n\nfunc TestGetVolumeSnapshotClassForStorageClass(t *testing.T) {\n\thostpathClass := &snapshotv1api.VolumeSnapshotClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"hostpath\",\n\t\t\tLabels: map[string]string{\n\t\t\t\tvelerov1api.VolumeSnapshotClassSelectorLabel: \"foo\",\n\t\t\t},\n\t\t},\n\t\tDriver: \"hostpath.csi.k8s.io\",\n\t}\n\n\tfooClass := &snapshotv1api.VolumeSnapshotClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"foo\",\n\t\t\tLabels: map[string]string{\n\t\t\t\tvelerov1api.VolumeSnapshotClassSelectorLabel: \"foo\",\n\t\t\t},\n\t\t},\n\t\tDriver: \"foo.csi.k8s.io\",\n\t}\n\n\tbarClass := &snapshotv1api.VolumeSnapshotClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"bar\",\n\t\t\tLabels: map[string]string{\n\t\t\t\tvelerov1api.VolumeSnapshotClassSelectorLabel: \"foo\",\n\t\t\t},\n\t\t},\n\t\tDriver: \"bar.csi.k8s.io\",\n\t}\n\n\tbazClass := &snapshotv1api.VolumeSnapshotClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"baz\",\n\t\t},\n\t\tDriver: \"baz.csi.k8s.io\",\n\t}\n\n\tambClass1 := &snapshotv1api.VolumeSnapshotClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"amb1\",\n\t\t},\n\t\tDriver: \"amb.csi.k8s.io\",\n\t}\n\n\tambClass2 := &snapshotv1api.VolumeSnapshotClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"amb2\",\n\t\t},\n\t\tDriver: \"amb.csi.k8s.io\",\n\t}\n\n\tsnapshotClasses := &snapshotv1api.VolumeSnapshotClassList{\n\t\tItems: []snapshotv1api.VolumeSnapshotClass{\n\t\t\t*hostpathClass, *fooClass, *barClass, *bazClass, *ambClass1, *ambClass2},\n\t}\n\n\ttestCases := []struct {\n\t\tname string\n\t\tdriverName string\n\t\texpectedVSC *snapshotv1api.VolumeSnapshotClass\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"should find hostpath volumesnapshotclass\",\n\t\t\tdriverName: \"hostpath.csi.k8s.io\",\n\t\t\texpectedVSC: hostpathClass,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should find foo volumesnapshotclass\",\n\t\t\tdriverName: \"foo.csi.k8s.io\",\n\t\t\texpectedVSC: fooClass,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should find bar volumesnapshotclass\",\n\t\t\tdriverName: \"bar.csi.k8s.io\",\n\t\t\texpectedVSC: barClass,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should find baz volumesnapshotclass without \\\"velero.io/csi-volumesnapshot-class\\\" label, b/c there's only one vsclass matching the driver name\",\n\t\t\tdriverName: \"baz.csi.k8s.io\",\n\t\t\texpectedVSC: bazClass,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should not find amb volumesnapshotclass without \\\"velero.io/csi-volumesnapshot-class\\\" label, b/c there're more than one vsclass matching the driver name\",\n\t\t\tdriverName: \"amb.csi.k8s.io\",\n\t\t\texpectedVSC: nil,\n\t\t\texpectError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"should not find does-not-exist volumesnapshotclass\",\n\t\t\tdriverName: \"not-found.csi.k8s.io\",\n\t\t\texpectedVSC: nil,\n\t\t\texpectError: true,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactualVSC, actualError := GetVolumeSnapshotClassForStorageClass(tc.driverName, snapshotClasses)\n\n\t\t\tif tc.expectError {\n\t\t\t\trequire.Error(t, actualError)\n\t\t\t\tassert.Nil(t, actualVSC)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tassert.Equalf(t, tc.expectedVSC.Name, actualVSC.Name, \"unexpected volumesnapshotclass name returned. Want: %s; Got:%s\", tc.expectedVSC.Name, actualVSC.Name)\n\t\t\tassert.Equalf(t, tc.expectedVSC.Driver, actualVSC.Driver, \"unexpected driver name returned. Want: %s; Got:%s\", tc.expectedVSC.Driver, actualVSC.Driver)\n\t\t})\n\t}\n}\n\nfunc TestIsVolumeSnapshotClassHasListerSecret(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tsnapClass snapshotv1api.VolumeSnapshotClass\n\t\texpected bool\n\t}{\n\t\t{\n\t\t\tname: \"should find both annotations\",\n\t\t\tsnapClass: snapshotv1api.VolumeSnapshotClass{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"class-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tvelerov1api.PrefixedListSecretNameAnnotation: \"snapListSecret\",\n\t\t\t\t\t\tvelerov1api.PrefixedListSecretNamespaceAnnotation: \"awesome-ns\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"should not find both annotations name is missing\",\n\t\t\tsnapClass: snapshotv1api.VolumeSnapshotClass{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"class-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"foo\": \"snapListSecret\",\n\t\t\t\t\t\tvelerov1api.PrefixedListSecretNamespaceAnnotation: \"awesome-ns\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should not find both annotations namespace is missing\",\n\t\t\tsnapClass: snapshotv1api.VolumeSnapshotClass{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"class-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tvelerov1api.PrefixedListSecretNameAnnotation: \"snapListSecret\",\n\t\t\t\t\t\t\"foo\": \"awesome-ns\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should not find expected annotation non-empty annotation\",\n\t\t\tsnapClass: snapshotv1api.VolumeSnapshotClass{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"class-2\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"foo\": \"snapListSecret\",\n\t\t\t\t\t\t\"bar\": \"awesome-ns\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should not find expected annotation nil annotation\",\n\t\t\tsnapClass: snapshotv1api.VolumeSnapshotClass{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"class-3\",\n\t\t\t\t\tAnnotations: nil,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should not find expected annotation empty annotation\",\n\t\t\tsnapClass: snapshotv1api.VolumeSnapshotClass{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"class-3\",\n\t\t\t\t\tAnnotations: map[string]string{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactual := IsVolumeSnapshotClassHasListerSecret(&tc.snapClass)\n\t\t\tassert.Equal(t, tc.expected, actual)\n\t\t})\n\t}\n}\n\nfunc TestIsVolumeSnapshotContentHasDeleteSecret(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tvsc snapshotv1api.VolumeSnapshotContent\n\t\texpected bool\n\t}{\n\t\t{\n\t\t\tname: \"should find both annotations\",\n\t\t\tvsc: snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"vsc-1\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tvelerov1api.PrefixedSecretNameAnnotation: \"delSnapSecret\",\n\t\t\t\t\t\tvelerov1api.PrefixedSecretNamespaceAnnotation: \"awesome-ns\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"should not find both annotations name is missing\",\n\t\t\tvsc: snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"vsc-2\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"foo\": \"delSnapSecret\",\n\t\t\t\t\t\tvelerov1api.PrefixedSecretNamespaceAnnotation: \"awesome-ns\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should not find both annotations namespace is missing\",\n\t\t\tvsc: snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"vsc-3\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tvelerov1api.PrefixedSecretNameAnnotation: \"delSnapSecret\",\n\t\t\t\t\t\t\"foo\": \"awesome-ns\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should not find expected annotation non-empty annotation\",\n\t\t\tvsc: snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"vsc-4\",\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\"foo\": \"delSnapSecret\",\n\t\t\t\t\t\t\"bar\": \"awesome-ns\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should not find expected annotation empty annotation\",\n\t\t\tvsc: snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"vsc-5\",\n\t\t\t\t\tAnnotations: map[string]string{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should not find expected annotation nil annotation\",\n\t\t\tvsc: snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"vsc-6\",\n\t\t\t\t\tAnnotations: nil,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactual := IsVolumeSnapshotContentHasDeleteSecret(&tc.vsc)\n\t\t\tassert.Equal(t, tc.expected, actual)\n\t\t})\n\t}\n}\n\nfunc TestIsVolumeSnapshotExists(t *testing.T) {\n\tvsExists := &snapshotv1api.VolumeSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"vs-exists\",\n\t\t\tNamespace: \"default\",\n\t\t},\n\t}\n\tvsNotExists := &snapshotv1api.VolumeSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"vs-does-not-exists\",\n\t\t\tNamespace: \"default\",\n\t\t},\n\t}\n\n\tobjs := []runtime.Object{vsExists}\n\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, objs...)\n\ttestCases := []struct {\n\t\tname string\n\t\texpected bool\n\t\tvs *snapshotv1api.VolumeSnapshot\n\t}{\n\t\t{\n\t\t\tname: \"should find existing VolumeSnapshot object\",\n\t\t\texpected: true,\n\t\t\tvs: vsExists,\n\t\t},\n\t\t{\n\t\t\tname: \"should not find non-existing VolumeSnapshot object\",\n\t\t\texpected: false,\n\t\t\tvs: vsNotExists,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactual := IsVolumeSnapshotExists(tc.vs.Namespace, tc.vs.Name, fakeClient)\n\t\t\tassert.Equal(t, tc.expected, actual)\n\t\t})\n\t}\n}\n\nfunc TestSetVolumeSnapshotContentDeletionPolicy(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tinputVSCName string\n\t\tpolicy snapshotv1api.DeletionPolicy\n\t\tobjs []runtime.Object\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"should update DeletionPolicy of a VSC from retain to delete\",\n\t\t\tinputVSCName: \"retainVSC\",\n\t\t\tpolicy: snapshotv1api.VolumeSnapshotContentDelete,\n\t\t\tobjs: []runtime.Object{\n\t\t\t\t&snapshotv1api.VolumeSnapshotContent{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"retainVSC\",\n\t\t\t\t\t},\n\t\t\t\t\tSpec: snapshotv1api.VolumeSnapshotContentSpec{\n\t\t\t\t\t\tDeletionPolicy: snapshotv1api.VolumeSnapshotContentRetain,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should be a no-op updating if DeletionPolicy of a VSC is already Delete\",\n\t\t\tinputVSCName: \"deleteVSC\",\n\t\t\tpolicy: snapshotv1api.VolumeSnapshotContentDelete,\n\t\t\tobjs: []runtime.Object{\n\t\t\t\t&snapshotv1api.VolumeSnapshotContent{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"deleteVSC\",\n\t\t\t\t\t},\n\t\t\t\t\tSpec: snapshotv1api.VolumeSnapshotContentSpec{\n\t\t\t\t\t\tDeletionPolicy: snapshotv1api.VolumeSnapshotContentDelete,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should update DeletionPolicy of a VSC with no DeletionPolicy\",\n\t\t\tinputVSCName: \"nothingVSC\",\n\t\t\tpolicy: snapshotv1api.VolumeSnapshotContentDelete,\n\t\t\tobjs: []runtime.Object{\n\t\t\t\t&snapshotv1api.VolumeSnapshotContent{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"nothingVSC\",\n\t\t\t\t\t},\n\t\t\t\t\tSpec: snapshotv1api.VolumeSnapshotContentSpec{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should return not found error if supplied VSC does not exist\",\n\t\t\tinputVSCName: \"does-not-exist\",\n\t\t\tobjs: []runtime.Object{},\n\t\t\texpectError: true,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, tc.objs...)\n\t\t\t_, err := SetVolumeSnapshotContentDeletionPolicy(tc.inputVSCName, fakeClient, tc.policy)\n\t\t\tif tc.expectError {\n\t\t\t\tassert.Error(t, err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tactual := new(snapshotv1api.VolumeSnapshotContent)\n\t\t\t\terr := fakeClient.Get(\n\t\t\t\t\tt.Context(),\n\t\t\t\t\tcrclient.ObjectKey{Name: tc.inputVSCName},\n\t\t\t\t\tactual,\n\t\t\t\t)\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(\n\t\t\t\t\tt,\n\t\t\t\t\ttc.policy,\n\t\t\t\t\tactual.Spec.DeletionPolicy,\n\t\t\t\t)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestDeleteVolumeSnapshots(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tvs snapshotv1api.VolumeSnapshot\n\t\tvsc snapshotv1api.VolumeSnapshotContent\n\t\tkeepVSAndVSC bool\n\t}{\n\t\t{\n\t\t\tname: \"VS is ReadyToUse, and VS has corresponding VSC. VS should be deleted.\",\n\t\t\tvs: *builder.ForVolumeSnapshot(\"velero\", \"vs1\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"testing-vs\", \"vs1\")).\n\t\t\t\tStatus().BoundVolumeSnapshotContentName(\"vsc1\").Result(),\n\t\t\tvsc: *builder.ForVolumeSnapshotContent(\"vsc1\").\n\t\t\t\tDeletionPolicy(snapshotv1api.VolumeSnapshotContentDelete).\n\t\t\t\tStatus(&snapshotv1api.VolumeSnapshotContentStatus{}).Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"VS status is nil. VSC should not be modified.\",\n\t\t\tvs: *builder.ForVolumeSnapshot(\"velero\", \"vs1\").\n\t\t\t\tObjectMeta(builder.WithLabels(\"testing-vs\", \"vs1\")).Result(),\n\t\t\tvsc: *builder.ForVolumeSnapshotContent(\"vsc1\").\n\t\t\t\tDeletionPolicy(snapshotv1api.VolumeSnapshotContentDelete).\n\t\t\t\tStatus(&snapshotv1api.VolumeSnapshotContentStatus{}).Result(),\n\t\t\tkeepVSAndVSC: true,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tclient := velerotest.NewFakeControllerRuntimeClient(\n\t\t\t\tt,\n\t\t\t\t[]runtime.Object{&tc.vs, &tc.vsc}...,\n\t\t\t)\n\t\t\tlogger := logging.DefaultLogger(logrus.DebugLevel, logging.FormatText)\n\n\t\t\tDeleteReadyVolumeSnapshot(tc.vs, client, logger)\n\n\t\t\tvsList := new(snapshotv1api.VolumeSnapshotList)\n\t\t\terr := client.List(\n\t\t\t\tt.Context(),\n\t\t\t\tvsList,\n\t\t\t\t&crclient.ListOptions{\n\t\t\t\t\tNamespace: \"velero\",\n\t\t\t\t},\n\t\t\t)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tvscList := new(snapshotv1api.VolumeSnapshotContentList)\n\t\t\terr = client.List(\n\t\t\t\tt.Context(),\n\t\t\t\tvscList,\n\t\t\t)\n\t\t\trequire.NoError(t, err)\n\n\t\t\tif tc.keepVSAndVSC {\n\t\t\t\trequire.Equal(t, crclient.ObjectKeyFromObject(&tc.vs), crclient.ObjectKeyFromObject(&vsList.Items[0]))\n\t\t\t\trequire.Equal(t, crclient.ObjectKeyFromObject(&tc.vsc), crclient.ObjectKeyFromObject(&vscList.Items[0]))\n\t\t\t} else {\n\t\t\t\trequire.Empty(t, vsList.Items)\n\t\t\t\trequire.Empty(t, vscList.Items)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestWaitUntilVSCHandleIsReady(t *testing.T) {\n\tvscName := \"snapcontent-7d1bdbd1-d10d-439c-8d8e-e1c2565ddc53\"\n\tsnapshotHandle := \"snapshot-handle\"\n\tvscObj := &snapshotv1api.VolumeSnapshotContent{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: vscName,\n\t\t},\n\t\tSpec: snapshotv1api.VolumeSnapshotContentSpec{\n\t\t\tVolumeSnapshotRef: corev1api.ObjectReference{\n\t\t\t\tName: \"vol-snap-1\",\n\t\t\t\tAPIVersion: snapshotv1api.SchemeGroupVersion.String(),\n\t\t\t},\n\t\t},\n\t\tStatus: &snapshotv1api.VolumeSnapshotContentStatus{\n\t\t\tSnapshotHandle: &snapshotHandle,\n\t\t},\n\t}\n\tvalidVS := &snapshotv1api.VolumeSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"vs\",\n\t\t\tNamespace: \"default\",\n\t\t},\n\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\tBoundVolumeSnapshotContentName: &vscName,\n\t\t},\n\t}\n\n\tnotFound := \"does-not-exist\"\n\tvsWithVSCNotFound := &snapshotv1api.VolumeSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: notFound,\n\t\t\tNamespace: \"default\",\n\t\t},\n\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\tBoundVolumeSnapshotContentName: ¬Found,\n\t\t},\n\t}\n\n\tvsWithNilStatus := &snapshotv1api.VolumeSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"nil-status-vs\",\n\t\t\tNamespace: \"default\",\n\t\t},\n\t\tStatus: nil,\n\t}\n\tvsWithNilStatusField := &snapshotv1api.VolumeSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"nil-status-field-vs\",\n\t\t\tNamespace: \"default\",\n\t\t},\n\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\tBoundVolumeSnapshotContentName: nil,\n\t\t},\n\t}\n\n\tnilStatusVsc := \"nil-status-vsc\"\n\tvscWithNilStatus := &snapshotv1api.VolumeSnapshotContent{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: nilStatusVsc,\n\t\t},\n\t\tSpec: snapshotv1api.VolumeSnapshotContentSpec{\n\t\t\tVolumeSnapshotRef: corev1api.ObjectReference{\n\t\t\t\tName: \"vol-snap-1\",\n\t\t\t\tAPIVersion: snapshotv1api.SchemeGroupVersion.String(),\n\t\t\t},\n\t\t},\n\t\tStatus: nil,\n\t}\n\tvsForNilStatusVsc := &snapshotv1api.VolumeSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"vs-for-nil-status-vsc\",\n\t\t\tNamespace: \"default\",\n\t\t},\n\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\tBoundVolumeSnapshotContentName: &nilStatusVsc,\n\t\t},\n\t}\n\n\tnilStatusFieldVsc := \"nil-status-field-vsc\"\n\tvscWithNilStatusField := &snapshotv1api.VolumeSnapshotContent{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: nilStatusFieldVsc,\n\t\t},\n\t\tSpec: snapshotv1api.VolumeSnapshotContentSpec{\n\t\t\tVolumeSnapshotRef: corev1api.ObjectReference{\n\t\t\t\tName: \"vol-snap-1\",\n\t\t\t\tAPIVersion: snapshotv1api.SchemeGroupVersion.String(),\n\t\t\t},\n\t\t},\n\t\tStatus: &snapshotv1api.VolumeSnapshotContentStatus{\n\t\t\tSnapshotHandle: nil,\n\t\t},\n\t}\n\tvsForNilStatusFieldVsc := &snapshotv1api.VolumeSnapshot{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"vs-for-nil-status-field\",\n\t\t\tNamespace: \"default\",\n\t\t},\n\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\tBoundVolumeSnapshotContentName: &nilStatusFieldVsc,\n\t\t},\n\t}\n\n\tobjs := []runtime.Object{\n\t\tvscObj,\n\t\tvalidVS,\n\t\tvsWithVSCNotFound,\n\t\tvsWithNilStatus,\n\t\tvsWithNilStatusField,\n\t\tvscWithNilStatus,\n\t\tvsForNilStatusVsc,\n\t\tvscWithNilStatusField,\n\t\tvsForNilStatusFieldVsc,\n\t}\n\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, objs...)\n\ttestCases := []struct {\n\t\tname string\n\t\tvolSnap *snapshotv1api.VolumeSnapshot\n\t\texepctedVSC *snapshotv1api.VolumeSnapshotContent\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"waitEnabled should find volumesnapshotcontent for volumesnapshot\",\n\t\t\tvolSnap: validVS,\n\t\t\texepctedVSC: vscObj,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"waitEnabled should not find volumesnapshotcontent for volumesnapshot with non-existing snapshotcontent name in status.BoundVolumeSnapshotContentName\",\n\t\t\tvolSnap: vsWithVSCNotFound,\n\t\t\texepctedVSC: nil,\n\t\t\texpectError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"waitEnabled should not find volumesnapshotcontent for a non-existent volumesnapshot\",\n\t\t\texepctedVSC: nil,\n\t\t\texpectError: true,\n\t\t\tvolSnap: &snapshotv1api.VolumeSnapshot{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"not-found\",\n\t\t\t\t\tNamespace: \"default\",\n\t\t\t\t},\n\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\t\t\tBoundVolumeSnapshotContentName: &nilStatusVsc,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactualVSC, actualError := WaitUntilVSCHandleIsReady(tc.volSnap, fakeClient, logrus.New().WithField(\"fake\", \"test\"), 0)\n\t\t\tif tc.expectError && actualError == nil {\n\t\t\t\trequire.Error(t, actualError)\n\t\t\t\tassert.Nil(t, actualVSC)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tassert.Equal(t, tc.exepctedVSC, actualVSC)\n\t\t})\n\t}\n}\n\nfunc TestDiagnoseVS(t *testing.T) {\n\tvscName := \"fake-vsc\"\n\treadyToUse := true\n\tmessage := \"fake-message\"\n\n\ttestCases := []struct {\n\t\tname string\n\t\tvs *snapshotv1api.VolumeSnapshot\n\t\tevents *corev1api.EventList\n\t\texpected string\n\t}{\n\t\t{\n\t\t\tname: \"VS with no status\",\n\t\t\tvs: &snapshotv1api.VolumeSnapshot{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vs\",\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: \"VS fake-ns/fake-vs, bind to , readyToUse false, errMessage \\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"VS with empty status\",\n\t\t\tvs: &snapshotv1api.VolumeSnapshot{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vs\",\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t},\n\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{},\n\t\t\t},\n\t\t\texpected: \"VS fake-ns/fake-vs, bind to , readyToUse false, errMessage \\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"VS with VSC name\",\n\t\t\tvs: &snapshotv1api.VolumeSnapshot{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vs\",\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t},\n\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\t\t\tBoundVolumeSnapshotContentName: &vscName,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: \"VS fake-ns/fake-vs, bind to fake-vsc, readyToUse false, errMessage \\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"VS with VSC name+ready\",\n\t\t\tvs: &snapshotv1api.VolumeSnapshot{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vs\",\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t},\n\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\t\t\tBoundVolumeSnapshotContentName: &vscName,\n\t\t\t\t\tReadyToUse: &readyToUse,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: \"VS fake-ns/fake-vs, bind to fake-vsc, readyToUse true, errMessage \\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"VS with VSC name+ready+empty error\",\n\t\t\tvs: &snapshotv1api.VolumeSnapshot{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vs\",\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t},\n\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\t\t\tBoundVolumeSnapshotContentName: &vscName,\n\t\t\t\t\tReadyToUse: &readyToUse,\n\t\t\t\t\tError: &snapshotv1api.VolumeSnapshotError{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: \"VS fake-ns/fake-vs, bind to fake-vsc, readyToUse true, errMessage \\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"VS with VSC name+ready+error\",\n\t\t\tvs: &snapshotv1api.VolumeSnapshot{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vs\",\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t},\n\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\t\t\tBoundVolumeSnapshotContentName: &vscName,\n\t\t\t\t\tReadyToUse: &readyToUse,\n\t\t\t\t\tError: &snapshotv1api.VolumeSnapshotError{\n\t\t\t\t\t\tMessage: &message,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: \"VS fake-ns/fake-vs, bind to fake-vsc, readyToUse true, errMessage fake-message\\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"VS with VSC and empty event\",\n\t\t\tvs: &snapshotv1api.VolumeSnapshot{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vs\",\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t},\n\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\t\t\tBoundVolumeSnapshotContentName: &vscName,\n\t\t\t\t\tReadyToUse: &readyToUse,\n\t\t\t\t\tError: &snapshotv1api.VolumeSnapshotError{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tevents: &corev1api.EventList{},\n\t\t\texpected: \"VS fake-ns/fake-vs, bind to fake-vsc, readyToUse true, errMessage \\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"VS with VSC and events\",\n\t\t\tvs: &snapshotv1api.VolumeSnapshot{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vs\",\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\tUID: \"fake-vs-uid\",\n\t\t\t\t},\n\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotStatus{\n\t\t\t\t\tBoundVolumeSnapshotContentName: &vscName,\n\t\t\t\t\tReadyToUse: &readyToUse,\n\t\t\t\t\tError: &snapshotv1api.VolumeSnapshotError{},\n\t\t\t\t},\n\t\t\t},\n\t\t\tevents: &corev1api.EventList{Items: []corev1api.Event{\n\t\t\t\t{\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-uid-1\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tReason: \"reason-1\",\n\t\t\t\t\tMessage: \"message-1\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-uid-2\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tReason: \"reason-2\",\n\t\t\t\t\tMessage: \"message-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-vs-uid\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tReason: \"reason-3\",\n\t\t\t\t\tMessage: \"message-3\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-vs-uid\"},\n\t\t\t\t\tType: corev1api.EventTypeNormal,\n\t\t\t\t\tReason: \"reason-4\",\n\t\t\t\t\tMessage: \"message-4\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-vs-uid\"},\n\t\t\t\t\tType: corev1api.EventTypeNormal,\n\t\t\t\t\tReason: \"reason-5\",\n\t\t\t\t\tMessage: \"message-5\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-vs-uid\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tReason: \"reason-6\",\n\t\t\t\t\tMessage: \"message-6\",\n\t\t\t\t},\n\t\t\t}},\n\t\t\texpected: \"VS fake-ns/fake-vs, bind to fake-vsc, readyToUse true, errMessage \\nVS event reason reason-3, message message-3\\nVS event reason reason-6, message message-6\\n\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tdiag := DiagnoseVS(tc.vs, tc.events)\n\t\t\tassert.Equal(t, tc.expected, diag)\n\t\t})\n\t}\n}\n\nfunc TestDiagnoseVSC(t *testing.T) {\n\treadyToUse := true\n\tmessage := \"fake-message\"\n\thandle := \"fake-handle\"\n\n\ttestCases := []struct {\n\t\tname string\n\t\tvsc *snapshotv1api.VolumeSnapshotContent\n\t\texpected string\n\t}{\n\t\t{\n\t\t\tname: \"VS with no status\",\n\t\t\tvsc: &snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vsc\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: \"VSC fake-vsc, readyToUse false, errMessage , handle \\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"VSC with empty status\",\n\t\t\tvsc: &snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vsc\",\n\t\t\t\t},\n\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotContentStatus{},\n\t\t\t},\n\t\t\texpected: \"VSC fake-vsc, readyToUse false, errMessage , handle \\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"VSC with ready\",\n\t\t\tvsc: &snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vsc\",\n\t\t\t\t},\n\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotContentStatus{\n\t\t\t\t\tReadyToUse: &readyToUse,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: \"VSC fake-vsc, readyToUse true, errMessage , handle \\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"VSC with ready+handle\",\n\t\t\tvsc: &snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vsc\",\n\t\t\t\t},\n\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotContentStatus{\n\t\t\t\t\tReadyToUse: &readyToUse,\n\t\t\t\t\tSnapshotHandle: &handle,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: \"VSC fake-vsc, readyToUse true, errMessage , handle fake-handle\\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"VSC with ready+handle+empty error\",\n\t\t\tvsc: &snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vsc\",\n\t\t\t\t},\n\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotContentStatus{\n\t\t\t\t\tReadyToUse: &readyToUse,\n\t\t\t\t\tSnapshotHandle: &handle,\n\t\t\t\t\tError: &snapshotv1api.VolumeSnapshotError{},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: \"VSC fake-vsc, readyToUse true, errMessage , handle fake-handle\\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"VSC with ready+handle+error\",\n\t\t\tvsc: &snapshotv1api.VolumeSnapshotContent{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-vsc\",\n\t\t\t\t},\n\t\t\t\tStatus: &snapshotv1api.VolumeSnapshotContentStatus{\n\t\t\t\t\tReadyToUse: &readyToUse,\n\t\t\t\t\tSnapshotHandle: &handle,\n\t\t\t\t\tError: &snapshotv1api.VolumeSnapshotError{\n\t\t\t\t\t\tMessage: &message,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: \"VSC fake-vsc, readyToUse true, errMessage fake-message, handle fake-handle\\n\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tdiag := DiagnoseVSC(tc.vsc)\n\t\t\tassert.Equal(t, tc.expected, diag)\n\t\t})\n\t}\n}\n\nfunc TestGetVSCForVS(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tvs *snapshotv1api.VolumeSnapshot\n\t\tvsc *snapshotv1api.VolumeSnapshotContent\n\t\texpectedErr string\n\t\texpectedVSC *snapshotv1api.VolumeSnapshotContent\n\t}{\n\t\t{\n\t\t\tname: \"vs has no status\",\n\t\t\tvs: builder.ForVolumeSnapshot(\"ns1\", \"vs1\").Result(),\n\t\t\texpectedErr: \"invalid snapshot info in volume snapshot vs1\",\n\t\t},\n\t\t{\n\t\t\tname: \"vs has no bound vsc\",\n\t\t\tvs: builder.ForVolumeSnapshot(\"ns1\", \"vs1\").Status().Result(),\n\t\t\texpectedErr: \"invalid snapshot info in volume snapshot vs1\",\n\t\t},\n\t\t{\n\t\t\tname: \"vs bound vsc cannot be found\",\n\t\t\tvs: builder.ForVolumeSnapshot(\"ns1\", \"vs1\").Status().BoundVolumeSnapshotContentName(\"vsc1\").Result(),\n\t\t\texpectedErr: \"error getting volume snapshot content from API: volumesnapshotcontents.snapshot.storage.k8s.io \\\"vsc1\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"normal case\",\n\t\t\tvs: builder.ForVolumeSnapshot(\"ns1\", \"vs1\").Status().BoundVolumeSnapshotContentName(\"vsc1\").Result(),\n\t\t\tvsc: builder.ForVolumeSnapshotContent(\"vsc1\").Result(),\n\t\t\texpectedVSC: builder.ForVolumeSnapshotContent(\"vsc1\").Result(),\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tobjs := []runtime.Object{tc.vs}\n\t\t\tif tc.vsc != nil {\n\t\t\t\tobjs = append(objs, tc.vsc)\n\t\t\t}\n\n\t\t\tclient := test.NewFakeControllerRuntimeClient(t, objs...)\n\t\t\tvsc, err := GetVSCForVS(t.Context(), tc.vs, client)\n\n\t\t\tif tc.expectedErr != \"\" {\n\t\t\t\trequire.EqualError(t, err, tc.expectedErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tif tc.expectedVSC != nil {\n\t\t\t\trequire.True(t, cmp.Equal(tc.expectedVSC, vsc, cmpopts.IgnoreFields(snapshotv1api.VolumeSnapshotContent{}, \"ResourceVersion\")))\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/util/encode/encode.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage encode\n\nimport (\n\t\"bytes\"\n\t\"compress/gzip\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\n\t\"github.com/pkg/errors\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/serializer\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n)\n\n// Encode converts the provided object to the specified format\n// and returns a byte slice of the encoded data.\nfunc Encode(obj runtime.Object, format string) ([]byte, error) {\n\tbuf := new(bytes.Buffer)\n\n\tif err := To(obj, format, buf); err != nil {\n\t\treturn nil, err\n\t}\n\treturn buf.Bytes(), nil\n}\n\n// To converts the provided object to the specified format and\n// writes the encoded data to the provided io.Writer.\nfunc To(obj runtime.Object, format string, w io.Writer) error {\n\tencoder, err := EncoderFor(format, obj)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn errors.WithStack(encoder.Encode(obj, w))\n}\n\n// EncoderFor gets the appropriate encoder for the specified format.\n// Only objects registered in the velero scheme, or objects with their TypeMeta set will have valid encoders.\nfunc EncoderFor(format string, obj runtime.Object) (runtime.Encoder, error) {\n\tvar encoder runtime.Encoder\n\n\tcodecFactory := serializer.NewCodecFactory(util.VeleroScheme)\n\n\tdesiredMediaType := fmt.Sprintf(\"application/%s\", format)\n\tserializerInfo, found := runtime.SerializerInfoForMediaType(codecFactory.SupportedMediaTypes(), desiredMediaType)\n\tif !found {\n\t\treturn nil, errors.Errorf(\"unable to locate an encoder for %q\", desiredMediaType)\n\t}\n\tif serializerInfo.PrettySerializer != nil {\n\t\tencoder = serializerInfo.PrettySerializer\n\t} else {\n\t\tencoder = serializerInfo.Serializer\n\t}\n\tif !obj.GetObjectKind().GroupVersionKind().Empty() {\n\t\treturn encoder, nil\n\t}\n\tencoder = codecFactory.EncoderForVersion(encoder, v1.SchemeGroupVersion)\n\treturn encoder, nil\n}\n\n// ToJSONGzip takes arbitrary Go data and encodes it to GZip compressed JSON in a buffer, as well as a description of the data to put into an error should encoding fail.\nfunc ToJSONGzip(data any, desc string) (*bytes.Buffer, []error) {\n\tbuf := new(bytes.Buffer)\n\tgzw := gzip.NewWriter(buf)\n\n\t// Since both encoding and closing the gzip writer could fail separately and both errors are useful,\n\t// collect both errors to report back.\n\terrs := []error{}\n\n\tif err := json.NewEncoder(gzw).Encode(data); err != nil {\n\t\terrs = append(errs, errors.Wrapf(err, \"error encoding %s\", desc))\n\t}\n\tif err := gzw.Close(); err != nil {\n\t\terrs = append(errs, errors.Wrapf(err, \"error closing gzip writer for %s\", desc))\n\t}\n\n\tif len(errs) > 0 {\n\t\treturn nil, errs\n\t}\n\n\treturn buf, nil\n}\n" }, { "path": "pkg/util/exec/exec.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage exec\n\nimport (\n\t\"bytes\"\n\t\"io\"\n\t\"os/exec\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n)\n\n// RunCommand runs a command and returns its stdout, stderr, and its returned\n// error (if any). If there are errors reading stdout or stderr, their return\n// value(s) will contain the error as a string.\nfunc RunCommand(cmd *exec.Cmd) (string, string, error) {\n\tstdoutBuf := new(bytes.Buffer)\n\tstderrBuf := new(bytes.Buffer)\n\n\tcmd.Stdout = stdoutBuf\n\tcmd.Stderr = stderrBuf\n\n\trunErr := cmd.Run()\n\n\tvar stdout, stderr string\n\n\tif res, readErr := io.ReadAll(stdoutBuf); readErr != nil {\n\t\tstdout = errors.Wrap(readErr, \"error reading command's stdout\").Error()\n\t} else {\n\t\tstdout = string(res)\n\t}\n\n\tif res, readErr := io.ReadAll(stderrBuf); readErr != nil {\n\t\tstderr = errors.Wrap(readErr, \"error reading command's stderr\").Error()\n\t} else {\n\t\tstderr = string(res)\n\t}\n\n\treturn stdout, stderr, runErr\n}\n\nfunc RunCommandWithLog(cmd *exec.Cmd, log logrus.FieldLogger) (string, string, error) {\n\tstdout, stderr, err := RunCommand(cmd)\n\tLogErrorAsExitCode(err, log)\n\treturn stdout, stderr, err\n}\n\nfunc LogErrorAsExitCode(err error, log logrus.FieldLogger) {\n\tif err != nil {\n\t\tif exitError, ok := err.(*exec.ExitError); ok {\n\t\t\tlog.Errorf(\"Restic command fail with ExitCode: %d. Process ID is %d, Exit error is: %s\", exitError.ExitCode(), exitError.Pid(), exitError.String())\n\t\t\t// Golang's os.exec -1 ExitCode means signal kill. Usually this is caused\n\t\t\t// by CGroup's OOM. Log a warning to notice user.\n\t\t\t// https://github.com/golang/go/blob/master/src/os/exec_posix.go#L128-L136\n\t\t\tif exitError.ExitCode() == -1 {\n\t\t\t\tlog.Warnf(\"The ExitCode is -1, which means the process is terminated by signal. Usually this is caused by CGroup kill due to out of memory. Please check whether there is such information in the work nodes' dmesg log.\")\n\t\t\t}\n\t\t} else {\n\t\t\tlog.WithError(err).Info(\"Error cannot be convert to ExitError format.\")\n\t\t}\n\t}\n}\n" }, { "path": "pkg/util/filesystem/file_system.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage filesystem\n\nimport (\n\t\"io\"\n\t\"os\"\n\t\"path/filepath\"\n)\n\n// Interface defines methods for interacting with an\n// underlying file system.\ntype Interface interface {\n\tTempDir(dir, prefix string) (string, error)\n\tMkdirAll(path string, perm os.FileMode) error\n\tCreate(name string) (io.WriteCloser, error)\n\tOpenFile(name string, flag int, perm os.FileMode) (io.WriteCloser, error)\n\tRemoveAll(path string) error\n\tReadDir(dirname string) ([]os.FileInfo, error)\n\tReadFile(filename string) ([]byte, error)\n\tDirExists(path string) (bool, error)\n\tTempFile(dir, prefix string) (NameWriteCloser, error)\n\tStat(path string) (os.FileInfo, error)\n\tGlob(path string) ([]string, error)\n}\n\ntype NameWriteCloser interface {\n\tio.WriteCloser\n\n\tName() string\n}\n\nfunc NewFileSystem() Interface {\n\treturn &osFileSystem{}\n}\n\ntype osFileSystem struct{}\n\nfunc (fs *osFileSystem) Glob(path string) ([]string, error) {\n\treturn filepath.Glob(path)\n}\n\nfunc (fs *osFileSystem) TempDir(dir, prefix string) (string, error) {\n\treturn os.MkdirTemp(dir, prefix)\n}\n\nfunc (fs *osFileSystem) MkdirAll(path string, perm os.FileMode) error {\n\treturn os.MkdirAll(path, perm)\n}\n\nfunc (fs *osFileSystem) Create(name string) (io.WriteCloser, error) {\n\treturn os.Create(name)\n}\n\nfunc (fs *osFileSystem) OpenFile(name string, flag int, perm os.FileMode) (io.WriteCloser, error) {\n\treturn os.OpenFile(name, flag, perm)\n}\n\nfunc (fs *osFileSystem) RemoveAll(path string) error {\n\treturn os.RemoveAll(path)\n}\n\nfunc (fs *osFileSystem) ReadDir(dirname string) ([]os.FileInfo, error) {\n\tvar fileInfos []os.FileInfo\n\tdirInfos, ise := os.ReadDir(dirname)\n\tif ise != nil {\n\t\treturn fileInfos, ise\n\t}\n\tfor _, dirInfo := range dirInfos {\n\t\tfileInfo, ise := dirInfo.Info()\n\t\tif ise == nil {\n\t\t\tfileInfos = append(fileInfos, fileInfo)\n\t\t}\n\t}\n\treturn fileInfos, nil\n}\n\nfunc (fs *osFileSystem) ReadFile(filename string) ([]byte, error) {\n\treturn os.ReadFile(filename)\n}\n\nfunc (fs *osFileSystem) DirExists(path string) (bool, error) {\n\t_, err := os.Stat(path)\n\tif err == nil {\n\t\treturn true, nil\n\t}\n\tif os.IsNotExist(err) {\n\t\treturn false, nil\n\t}\n\treturn false, err\n}\n\nfunc (fs *osFileSystem) TempFile(dir, prefix string) (NameWriteCloser, error) {\n\treturn os.CreateTemp(dir, prefix)\n}\n\nfunc (fs *osFileSystem) Stat(path string) (os.FileInfo, error) {\n\treturn os.Stat(path)\n}\n" }, { "path": "pkg/util/kube/client.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"context\"\n\t\"time\"\n\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tveleroPkgClient \"github.com/vmware-tanzu/velero/pkg/client\"\n)\n\nfunc PatchResource(original, updated client.Object, kbClient client.Client) error {\n\terr := kbClient.Patch(context.Background(), updated, client.MergeFrom(original))\n\n\treturn err\n}\n\n// PatchResourceWithRetries patches the original resource with the updated resource, retrying when the provided retriable function returns true.\nfunc PatchResourceWithRetries(maxDuration time.Duration, original, updated client.Object, kbClient client.Client, retriable func(error) bool) error {\n\treturn veleroPkgClient.RetryOnRetriableMaxBackOff(maxDuration, func() error { return PatchResource(original, updated, kbClient) }, retriable)\n}\n\n// PatchResourceWithRetriesOnErrors patches the original resource with the updated resource, retrying when the operation returns an error.\nfunc PatchResourceWithRetriesOnErrors(maxDuration time.Duration, original, updated client.Object, kbClient client.Client) error {\n\treturn PatchResourceWithRetries(maxDuration, original, updated, kbClient, func(err error) bool {\n\t\t// retry using DefaultBackoff to resolve connection refused error that may occur when the server is under heavy load\n\t\t// TODO: consider using a more specific error type to retry, for now, we retry on all errors\n\t\t// specific errors:\n\t\t// - connection refused: https://pkg.go.dev/syscall#:~:text=Errno(0x67)-,ECONNREFUSED,-%3D%20Errno(0x6f\n\t\treturn err != nil\n\t})\n}\n" }, { "path": "pkg/util/kube/event.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage kube\n\nimport (\n\t\"context\"\n\t\"math\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/google/uuid\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes\"\n\ttypedcorev1 \"k8s.io/client-go/kubernetes/typed/core/v1\"\n\t\"k8s.io/client-go/tools/record\"\n)\n\ntype EventRecorder interface {\n\tEvent(object runtime.Object, warning bool, reason string, message string, a ...any)\n\tEndingEvent(object runtime.Object, warning bool, reason string, message string, a ...any)\n\tShutdown()\n}\n\ntype eventRecorder struct {\n\tbroadcaster record.EventBroadcaster\n\trecorder record.EventRecorder\n\tlock sync.Mutex\n\tendingSentinel *eventElement\n\tlog logrus.FieldLogger\n}\n\ntype eventElement struct {\n\tt string\n\tr string\n\tm string\n\tsinked chan struct{}\n}\n\ntype eventSink struct {\n\trecorder *eventRecorder\n\tsink typedcorev1.EventInterface\n}\n\nfunc NewEventRecorder(kubeClient kubernetes.Interface, scheme *runtime.Scheme, eventSource string, eventNode string, log logrus.FieldLogger) EventRecorder {\n\tres := eventRecorder{\n\t\tlog: log,\n\t}\n\n\tres.broadcaster = record.NewBroadcasterWithCorrelatorOptions(record.CorrelatorOptions{\n\t\t// Bypass the built-in EventCorrelator's rate filtering, otherwise, the event will be abandoned if the rate exceeds.\n\t\t// The callers (i.e., data mover pods) have controlled the rate and total number outside. E.g., the progress is designed to be updated every 10 seconds and is changeable.\n\t\tBurstSize: math.MaxInt32,\n\t\tMaxEvents: 1,\n\t\tMessageFunc: func(event *corev1api.Event) string {\n\t\t\treturn event.Message\n\t\t},\n\t})\n\n\tres.broadcaster.StartRecordingToSink(&eventSink{\n\t\trecorder: &res,\n\t\tsink: kubeClient.CoreV1().Events(\"\"),\n\t})\n\n\tres.recorder = res.broadcaster.NewRecorder(scheme, corev1api.EventSource{\n\t\tComponent: eventSource,\n\t\tHost: eventNode,\n\t})\n\n\treturn &res\n}\n\nfunc (er *eventRecorder) Event(object runtime.Object, warning bool, reason string, message string, a ...any) {\n\tif er.broadcaster == nil {\n\t\treturn\n\t}\n\n\teventType := corev1api.EventTypeNormal\n\tif warning {\n\t\teventType = corev1api.EventTypeWarning\n\t}\n\n\tif len(a) > 0 {\n\t\ter.recorder.Eventf(object, eventType, reason, message, a...)\n\t} else {\n\t\ter.recorder.Event(object, eventType, reason, message)\n\t}\n}\n\nfunc (er *eventRecorder) EndingEvent(object runtime.Object, warning bool, reason string, message string, a ...any) {\n\tif er.broadcaster == nil {\n\t\treturn\n\t}\n\n\ter.Event(object, warning, reason, message, a...)\n\n\tvar sentinelEvent string\n\n\ter.lock.Lock()\n\tif er.endingSentinel == nil {\n\t\tsentinelEvent = uuid.NewString()\n\t\ter.endingSentinel = &eventElement{\n\t\t\tt: corev1api.EventTypeNormal,\n\t\t\tr: sentinelEvent,\n\t\t\tm: sentinelEvent,\n\t\t\tsinked: make(chan struct{}),\n\t\t}\n\t}\n\ter.lock.Unlock()\n\n\tif sentinelEvent != \"\" {\n\t\ter.Event(object, false, sentinelEvent, sentinelEvent)\n\t} else {\n\t\ter.log.Warn(\"More than one ending events, ignore\")\n\t}\n}\n\nvar shutdownTimeout = time.Minute\n\nfunc (er *eventRecorder) Shutdown() {\n\tvar wait chan struct{}\n\ter.lock.Lock()\n\tif er.endingSentinel != nil {\n\t\twait = er.endingSentinel.sinked\n\t}\n\ter.lock.Unlock()\n\n\tif wait != nil {\n\t\ter.log.Info(\"Waiting sentinel before shutdown\")\n\n\twaitloop:\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-wait:\n\t\t\t\tbreak waitloop\n\t\t\tcase <-time.After(shutdownTimeout):\n\t\t\t\ter.log.Warn(\"Timeout waiting for assured events processed\")\n\t\t\t\tbreak waitloop\n\t\t\t}\n\t\t}\n\t}\n\n\ter.broadcaster.Shutdown()\n\ter.broadcaster = nil\n\n\ter.lock.Lock()\n\ter.endingSentinel = nil\n\ter.lock.Unlock()\n}\n\nfunc (er *eventRecorder) sentinelWatch(event *corev1api.Event) bool {\n\ter.lock.Lock()\n\tdefer er.lock.Unlock()\n\n\tif er.endingSentinel == nil {\n\t\treturn false\n\t}\n\n\tif er.endingSentinel.m == event.Message && er.endingSentinel.r == event.Reason && er.endingSentinel.t == event.Type {\n\t\tclose(er.endingSentinel.sinked)\n\t\treturn true\n\t}\n\n\treturn false\n}\n\nfunc (es *eventSink) Create(event *corev1api.Event) (*corev1api.Event, error) {\n\tif es.recorder.sentinelWatch(event) {\n\t\treturn event, nil\n\t}\n\n\treturn es.sink.CreateWithEventNamespaceWithContext(context.Background(), event)\n}\n\nfunc (es *eventSink) Update(event *corev1api.Event) (*corev1api.Event, error) {\n\treturn es.sink.UpdateWithEventNamespaceWithContext(context.Background(), event)\n}\n\nfunc (es *eventSink) Patch(event *corev1api.Event, data []byte) (*corev1api.Event, error) {\n\treturn es.sink.PatchWithEventNamespaceWithContext(context.Background(), event, data)\n}\n" }, { "path": "pkg/util/kube/event_handler.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"context\"\n\n\t\"k8s.io/client-go/util/workqueue\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/event\"\n\t\"sigs.k8s.io/controller-runtime/pkg/handler\"\n\t\"sigs.k8s.io/controller-runtime/pkg/reconcile\"\n)\n\n// EnqueueRequestsFromMapUpdateFunc has the same purpose with handler.EnqueueRequestsFromMapFunc.\n// It's simpler on Update event because mapAndEnqueue is called once with the new object. EnqueueRequestsFromMapFunc is called twice with the old and new object.\nfunc EnqueueRequestsFromMapUpdateFunc(fn handler.MapFunc) handler.EventHandler {\n\treturn TypedEnqueueRequestsFromMapUpdateFunc(fn)\n}\n\nfunc TypedEnqueueRequestsFromMapUpdateFunc[object any, request comparable](fn handler.TypedMapFunc[object, request]) handler.TypedEventHandler[object, request] {\n\treturn &enqueueRequestsFromMapFunc[object, request]{\n\t\ttoRequests: fn,\n\t}\n}\n\nvar _ handler.EventHandler = &enqueueRequestsFromMapFunc[client.Object, reconcile.Request]{}\n\ntype enqueueRequestsFromMapFunc[object any, request comparable] struct {\n\ttoRequests handler.TypedMapFunc[object, request]\n}\n\n// Create implements EventHandler.\nfunc (e *enqueueRequestsFromMapFunc[object, request]) Create(ctx context.Context, evt event.TypedCreateEvent[object], q workqueue.TypedRateLimitingInterface[request]) {\n\te.mapAndEnqueue(ctx, q, evt.Object)\n}\n\n// Update implements EventHandler.\nfunc (e *enqueueRequestsFromMapFunc[object, request]) Update(ctx context.Context, evt event.TypedUpdateEvent[object], q workqueue.TypedRateLimitingInterface[request]) {\n\te.mapAndEnqueue(ctx, q, evt.ObjectNew)\n}\n\n// Delete implements EventHandler.\nfunc (e *enqueueRequestsFromMapFunc[object, request]) Delete(ctx context.Context, evt event.TypedDeleteEvent[object], q workqueue.TypedRateLimitingInterface[request]) {\n\te.mapAndEnqueue(ctx, q, evt.Object)\n}\n\n// Generic implements EventHandler.\nfunc (e *enqueueRequestsFromMapFunc[object, request]) Generic(ctx context.Context, evt event.TypedGenericEvent[object], q workqueue.TypedRateLimitingInterface[request]) {\n\te.mapAndEnqueue(ctx, q, evt.Object)\n}\n\nfunc (e *enqueueRequestsFromMapFunc[object, request]) mapAndEnqueue(ctx context.Context, q workqueue.TypedRateLimitingInterface[request], obj object) {\n\treqs := map[request]struct{}{}\n\n\tfor _, req := range e.toRequests(ctx, obj) {\n\t\t_, ok := reqs[req]\n\t\tif !ok {\n\t\t\tq.Add(req)\n\t\t\treqs[req] = struct{}{}\n\t\t}\n\t}\n}\n" }, { "path": "pkg/util/kube/event_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"fmt\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/client-go/kubernetes/fake\"\n\n\tcorev1api \"k8s.io/api/core/v1\"\n\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestEvent(t *testing.T) {\n\ttype testEvent struct {\n\t\twarning bool\n\t\treason string\n\t\tmessage string\n\t\tending bool\n\t}\n\n\tcases := []struct {\n\t\tname string\n\t\tevents []testEvent\n\t\tgenerateDiff int\n\t\tgenerateSame int\n\t\tgenerateEnding bool\n\t\texpected int\n\t}{\n\t\t{\n\t\t\tname: \"update events, different message\",\n\t\t\tevents: []testEvent{\n\t\t\t\t{\n\t\t\t\t\twarning: false,\n\t\t\t\t\treason: \"Progress\",\n\t\t\t\t\tmessage: \"progress-1\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\twarning: false,\n\t\t\t\t\treason: \"Progress\",\n\t\t\t\t\tmessage: \"progress-2\",\n\t\t\t\t\tending: true,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"create events, different reason\",\n\t\t\tevents: []testEvent{\n\t\t\t\t{\n\t\t\t\t\twarning: false,\n\t\t\t\t\treason: \"action-1-1\",\n\t\t\t\t\tmessage: \"fake-message\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\twarning: false,\n\t\t\t\t\treason: \"action-1-2\",\n\t\t\t\t\tmessage: \"fake-message\",\n\t\t\t\t\tending: true,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: 2,\n\t\t},\n\t\t{\n\t\t\tname: \"create events, different warning\",\n\t\t\tevents: []testEvent{\n\t\t\t\t{\n\t\t\t\t\twarning: false,\n\t\t\t\t\treason: \"action-2-1\",\n\t\t\t\t\tmessage: \"fake-message\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\twarning: true,\n\t\t\t\t\treason: \"action-2-1\",\n\t\t\t\t\tmessage: \"fake-message\",\n\t\t\t\t\tending: true,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: 2,\n\t\t},\n\t\t{\n\t\t\tname: \"endingEvent, double entrance\",\n\t\t\tevents: []testEvent{\n\t\t\t\t{\n\t\t\t\t\twarning: false,\n\t\t\t\t\treason: \"action-2-1\",\n\t\t\t\t\tmessage: \"fake-message\",\n\t\t\t\t\tending: true,\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\twarning: true,\n\t\t\t\t\treason: \"action-2-1\",\n\t\t\t\t\tmessage: \"fake-message\",\n\t\t\t\t\tending: true,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: -1,\n\t\t},\n\t\t{\n\t\t\tname: \"auto generate 200\",\n\t\t\tgenerateDiff: 200,\n\t\t\tgenerateEnding: true,\n\t\t\texpected: 201,\n\t\t},\n\t\t{\n\t\t\tname: \"auto generate 200, update\",\n\t\t\tgenerateSame: 200,\n\t\t\tgenerateEnding: true,\n\t\t\texpected: 2,\n\t\t},\n\t}\n\n\tshutdownTimeout = time.Second * 5\n\n\tfor _, tc := range cases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tclient := fake.NewSimpleClientset()\n\t\t\tscheme := runtime.NewScheme()\n\t\t\terr := corev1api.AddToScheme(scheme)\n\t\t\trequire.NoError(t, err)\n\n\t\t\trecorder := NewEventRecorder(client, scheme, \"source-1\", \"fake-node\", velerotest.NewLogger())\n\n\t\t\tpod := &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\tName: \"fake-pod\",\n\t\t\t\t\tUID: types.UID(\"fake-uid\"),\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tNodeName: \"fake-node\",\n\t\t\t\t},\n\t\t\t}\n\n\t\t\t_, err = client.CoreV1().Pods(\"fake-ns\").Create(t.Context(), pod, metav1.CreateOptions{})\n\t\t\trequire.NoError(t, err)\n\n\t\t\tfor i := 0; i < tc.generateDiff; i++ {\n\t\t\t\ttc.events = append(tc.events, testEvent{\n\t\t\t\t\treason: fmt.Sprintf(\"fake-reason-%v\", i),\n\t\t\t\t\tmessage: fmt.Sprintf(\"fake-message-%v\", i),\n\t\t\t\t})\n\t\t\t}\n\n\t\t\tfor i := 0; i < tc.generateSame; i++ {\n\t\t\t\ttc.events = append(tc.events, testEvent{\n\t\t\t\t\treason: \"fake-reason\",\n\t\t\t\t\tmessage: fmt.Sprintf(\"fake-message-%v\", i),\n\t\t\t\t})\n\t\t\t}\n\n\t\t\tif tc.generateEnding {\n\t\t\t\ttc.events = append(tc.events, testEvent{\n\t\t\t\t\treason: \"fake-ending-reason\",\n\t\t\t\t\tmessage: \"fake-ending-message\",\n\t\t\t\t\tending: true,\n\t\t\t\t})\n\t\t\t}\n\n\t\t\tfor _, e := range tc.events {\n\t\t\t\tif e.ending {\n\t\t\t\t\trecorder.EndingEvent(pod, e.warning, e.reason, e.message)\n\t\t\t\t} else {\n\t\t\t\t\trecorder.Event(pod, e.warning, e.reason, e.message)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\trecorder.Shutdown()\n\n\t\t\titems, err := client.CoreV1().Events(\"fake-ns\").List(t.Context(), metav1.ListOptions{})\n\t\t\trequire.NoError(t, err)\n\n\t\t\tif tc.expected != len(items.Items) {\n\t\t\t\tfor _, i := range items.Items {\n\t\t\t\t\tfmt.Printf(\"event (%s, %s, %s)\\n\", i.Type, i.Message, i.Reason)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif tc.expected >= 0 {\n\t\t\t\tassert.Len(t, items.Items, tc.expected)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/util/kube/list_watch.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"context\"\n\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/watch\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n)\n\ntype InternalLW struct {\n\tClient kbclient.WithWatch\n\tNamespace string\n\tObjectList kbclient.ObjectList\n}\n\nfunc (lw *InternalLW) Watch(options metav1.ListOptions) (watch.Interface, error) {\n\treturn lw.Client.Watch(context.Background(), lw.ObjectList, &kbclient.ListOptions{Raw: &options, Namespace: lw.Namespace})\n}\n\nfunc (lw *InternalLW) List(options metav1.ListOptions) (runtime.Object, error) {\n\terr := lw.Client.List(context.Background(), lw.ObjectList, &kbclient.ListOptions{Raw: &options, Namespace: lw.Namespace})\n\treturn lw.ObjectList, err\n}\n" }, { "path": "pkg/util/kube/list_watch_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/client-go/tools/cache\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestInternalLW(t *testing.T) {\n\tstop := make(chan struct{})\n\tclient := velerotest.NewFakeControllerRuntimeClient(t).(kbclient.WithWatch)\n\tlw := InternalLW{\n\t\tClient: client,\n\t\tNamespace: \"velero\",\n\t\tObjectList: new(velerov1api.BackupList),\n\t}\n\n\trestoreInformer := cache.NewSharedInformer(&lw, &velerov1api.BackupList{}, time.Second)\n\tgo restoreInformer.Run(stop)\n\n\ttime.Sleep(1 * time.Second)\n\tclose(stop)\n\n\tbackupList := new(velerov1api.BackupList)\n\terr := client.List(t.Context(), backupList)\n\trequire.NoError(t, err)\n\n\t_, err = client.Watch(t.Context(), backupList)\n\trequire.NoError(t, err)\n}\n" }, { "path": "pkg/util/kube/mocks/Client.go", "content": "// Code generated by mockery v2.42.1. DO NOT EDIT.\n\npackage mocks\n\nimport (\n\tcontext \"context\"\n\n\tclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tmeta \"k8s.io/apimachinery/pkg/api/meta\"\n\n\tmock \"github.com/stretchr/testify/mock\"\n\n\truntime \"k8s.io/apimachinery/pkg/runtime\"\n\n\tschema \"k8s.io/apimachinery/pkg/runtime/schema\"\n\n\ttypes \"k8s.io/apimachinery/pkg/types\"\n)\n\n// Client is an autogenerated mock type for the Client type\ntype Client struct {\n\tmock.Mock\n}\n\n// Create provides a mock function with given fields: ctx, obj, opts\nfunc (_m *Client) Create(ctx context.Context, obj client.Object, opts ...client.CreateOption) error {\n\t_va := make([]interface{}, len(opts))\n\tfor _i := range opts {\n\t\t_va[_i] = opts[_i]\n\t}\n\tvar _ca []interface{}\n\t_ca = append(_ca, ctx, obj)\n\t_ca = append(_ca, _va...)\n\tret := _m.Called(_ca...)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Create\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, client.Object, ...client.CreateOption) error); ok {\n\t\tr0 = rf(ctx, obj, opts...)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// Delete provides a mock function with given fields: ctx, obj, opts\nfunc (_m *Client) Delete(ctx context.Context, obj client.Object, opts ...client.DeleteOption) error {\n\t_va := make([]interface{}, len(opts))\n\tfor _i := range opts {\n\t\t_va[_i] = opts[_i]\n\t}\n\tvar _ca []interface{}\n\t_ca = append(_ca, ctx, obj)\n\t_ca = append(_ca, _va...)\n\tret := _m.Called(_ca...)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Delete\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, client.Object, ...client.DeleteOption) error); ok {\n\t\tr0 = rf(ctx, obj, opts...)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// DeleteAllOf provides a mock function with given fields: ctx, obj, opts\nfunc (_m *Client) DeleteAllOf(ctx context.Context, obj client.Object, opts ...client.DeleteAllOfOption) error {\n\t_va := make([]interface{}, len(opts))\n\tfor _i := range opts {\n\t\t_va[_i] = opts[_i]\n\t}\n\tvar _ca []interface{}\n\t_ca = append(_ca, ctx, obj)\n\t_ca = append(_ca, _va...)\n\tret := _m.Called(_ca...)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for DeleteAllOf\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, client.Object, ...client.DeleteAllOfOption) error); ok {\n\t\tr0 = rf(ctx, obj, opts...)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// Get provides a mock function with given fields: ctx, key, obj, opts\nfunc (_m *Client) Get(ctx context.Context, key types.NamespacedName, obj client.Object, opts ...client.GetOption) error {\n\t_va := make([]interface{}, len(opts))\n\tfor _i := range opts {\n\t\t_va[_i] = opts[_i]\n\t}\n\tvar _ca []interface{}\n\t_ca = append(_ca, ctx, key, obj)\n\t_ca = append(_ca, _va...)\n\tret := _m.Called(_ca...)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Get\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, types.NamespacedName, client.Object, ...client.GetOption) error); ok {\n\t\tr0 = rf(ctx, key, obj, opts...)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// GroupVersionKindFor provides a mock function with given fields: obj\nfunc (_m *Client) GroupVersionKindFor(obj runtime.Object) (schema.GroupVersionKind, error) {\n\tret := _m.Called(obj)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for GroupVersionKindFor\")\n\t}\n\n\tvar r0 schema.GroupVersionKind\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(runtime.Object) (schema.GroupVersionKind, error)); ok {\n\t\treturn rf(obj)\n\t}\n\tif rf, ok := ret.Get(0).(func(runtime.Object) schema.GroupVersionKind); ok {\n\t\tr0 = rf(obj)\n\t} else {\n\t\tr0 = ret.Get(0).(schema.GroupVersionKind)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(runtime.Object) error); ok {\n\t\tr1 = rf(obj)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// IsObjectNamespaced provides a mock function with given fields: obj\nfunc (_m *Client) IsObjectNamespaced(obj runtime.Object) (bool, error) {\n\tret := _m.Called(obj)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for IsObjectNamespaced\")\n\t}\n\n\tvar r0 bool\n\tvar r1 error\n\tif rf, ok := ret.Get(0).(func(runtime.Object) (bool, error)); ok {\n\t\treturn rf(obj)\n\t}\n\tif rf, ok := ret.Get(0).(func(runtime.Object) bool); ok {\n\t\tr0 = rf(obj)\n\t} else {\n\t\tr0 = ret.Get(0).(bool)\n\t}\n\n\tif rf, ok := ret.Get(1).(func(runtime.Object) error); ok {\n\t\tr1 = rf(obj)\n\t} else {\n\t\tr1 = ret.Error(1)\n\t}\n\n\treturn r0, r1\n}\n\n// List provides a mock function with given fields: ctx, list, opts\nfunc (_m *Client) List(ctx context.Context, list client.ObjectList, opts ...client.ListOption) error {\n\t_va := make([]interface{}, len(opts))\n\tfor _i := range opts {\n\t\t_va[_i] = opts[_i]\n\t}\n\tvar _ca []interface{}\n\t_ca = append(_ca, ctx, list)\n\t_ca = append(_ca, _va...)\n\tret := _m.Called(_ca...)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for List\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, client.ObjectList, ...client.ListOption) error); ok {\n\t\tr0 = rf(ctx, list, opts...)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// Patch provides a mock function with given fields: ctx, obj, patch, opts\nfunc (_m *Client) Patch(ctx context.Context, obj client.Object, patch client.Patch, opts ...client.PatchOption) error {\n\t_va := make([]interface{}, len(opts))\n\tfor _i := range opts {\n\t\t_va[_i] = opts[_i]\n\t}\n\tvar _ca []interface{}\n\t_ca = append(_ca, ctx, obj, patch)\n\t_ca = append(_ca, _va...)\n\tret := _m.Called(_ca...)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Patch\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, client.Object, client.Patch, ...client.PatchOption) error); ok {\n\t\tr0 = rf(ctx, obj, patch, opts...)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// RESTMapper provides a mock function with given fields:\nfunc (_m *Client) RESTMapper() meta.RESTMapper {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for RESTMapper\")\n\t}\n\n\tvar r0 meta.RESTMapper\n\tif rf, ok := ret.Get(0).(func() meta.RESTMapper); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(meta.RESTMapper)\n\t\t}\n\t}\n\n\treturn r0\n}\n\n// Scheme provides a mock function with given fields:\nfunc (_m *Client) Scheme() *runtime.Scheme {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Scheme\")\n\t}\n\n\tvar r0 *runtime.Scheme\n\tif rf, ok := ret.Get(0).(func() *runtime.Scheme); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(*runtime.Scheme)\n\t\t}\n\t}\n\n\treturn r0\n}\n\n// Status provides a mock function with given fields:\nfunc (_m *Client) Status() client.SubResourceWriter {\n\tret := _m.Called()\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Status\")\n\t}\n\n\tvar r0 client.SubResourceWriter\n\tif rf, ok := ret.Get(0).(func() client.SubResourceWriter); ok {\n\t\tr0 = rf()\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(client.SubResourceWriter)\n\t\t}\n\t}\n\n\treturn r0\n}\n\n// SubResource provides a mock function with given fields: subResource\nfunc (_m *Client) SubResource(subResource string) client.SubResourceClient {\n\tret := _m.Called(subResource)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for SubResource\")\n\t}\n\n\tvar r0 client.SubResourceClient\n\tif rf, ok := ret.Get(0).(func(string) client.SubResourceClient); ok {\n\t\tr0 = rf(subResource)\n\t} else {\n\t\tif ret.Get(0) != nil {\n\t\t\tr0 = ret.Get(0).(client.SubResourceClient)\n\t\t}\n\t}\n\n\treturn r0\n}\n\n// Update provides a mock function with given fields: ctx, obj, opts\nfunc (_m *Client) Update(ctx context.Context, obj client.Object, opts ...client.UpdateOption) error {\n\t_va := make([]interface{}, len(opts))\n\tfor _i := range opts {\n\t\t_va[_i] = opts[_i]\n\t}\n\tvar _ca []interface{}\n\t_ca = append(_ca, ctx, obj)\n\t_ca = append(_ca, _va...)\n\tret := _m.Called(_ca...)\n\n\tif len(ret) == 0 {\n\t\tpanic(\"no return value specified for Update\")\n\t}\n\n\tvar r0 error\n\tif rf, ok := ret.Get(0).(func(context.Context, client.Object, ...client.UpdateOption) error); ok {\n\t\tr0 = rf(ctx, obj, opts...)\n\t} else {\n\t\tr0 = ret.Error(0)\n\t}\n\n\treturn r0\n}\n\n// NewClient creates a new instance of Client. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.\n// The first argument is typically a *testing.T value.\nfunc NewClient(t interface {\n\tmock.TestingT\n\tCleanup(func())\n}) *Client {\n\tmock := &Client{}\n\tmock.Mock.Test(t)\n\n\tt.Cleanup(func() { mock.AssertExpectations(t) })\n\n\treturn mock\n}\n" }, { "path": "pkg/util/kube/mocks.go", "content": "package kube\n\nimport (\n\t\"k8s.io/apimachinery/pkg/api/meta\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n)\n\n// Client knows how to perform CRUD operations on Kubernetes objects.\n//\n//go:generate mockery --name=Client\ntype Client interface {\n\tclient.Reader\n\tclient.Writer\n\tclient.StatusClient\n\tclient.SubResourceClientConstructor\n\n\t// Scheme returns the scheme this client is using.\n\tScheme() *runtime.Scheme\n\t// RESTMapper returns the rest this client is using.\n\tRESTMapper() meta.RESTMapper\n\t// GroupVersionKindFor returns the GroupVersionKind for the given object.\n\tGroupVersionKindFor(obj runtime.Object) (schema.GroupVersionKind, error)\n\t// IsObjectNamespaced returns true if the GroupVersionKind of the object is namespaced.\n\tIsObjectNamespaced(obj runtime.Object) (bool, error)\n}\n" }, { "path": "pkg/util/kube/node.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage kube\n\nimport (\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\tcorev1client \"k8s.io/client-go/kubernetes/typed/core/v1\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n)\n\nconst (\n\tNodeOSLinux = \"linux\"\n\tNodeOSWindows = \"windows\"\n\tNodeOSLabel = \"kubernetes.io/os\"\n)\n\nvar realNodeOSMap = map[string]string{\n\t\"linux\": NodeOSLinux,\n\t\"windows\": NodeOSWindows,\n}\n\nfunc IsLinuxNode(ctx context.Context, nodeName string, client client.Client) error {\n\tnode := &corev1api.Node{}\n\tif err := client.Get(ctx, types.NamespacedName{Name: nodeName}, node); err != nil {\n\t\treturn errors.Wrapf(err, \"error getting node %s\", nodeName)\n\t}\n\n\tos, found := node.Labels[NodeOSLabel]\n\tif !found {\n\t\treturn errors.Errorf(\"no os type label for node %s\", nodeName)\n\t}\n\n\tif getRealOS(os) != NodeOSLinux {\n\t\treturn errors.Errorf(\"os type %s for node %s is not linux\", os, nodeName)\n\t}\n\n\treturn nil\n}\n\nfunc WithLinuxNode(ctx context.Context, client client.Client, log logrus.FieldLogger) bool {\n\treturn withOSNode(ctx, client, NodeOSLinux, log)\n}\n\nfunc WithWindowsNode(ctx context.Context, client client.Client, log logrus.FieldLogger) bool {\n\treturn withOSNode(ctx, client, NodeOSWindows, log)\n}\n\nfunc withOSNode(ctx context.Context, client client.Client, osType string, log logrus.FieldLogger) bool {\n\tnodeList := new(corev1api.NodeList)\n\tif err := client.List(ctx, nodeList); err != nil {\n\t\tlog.Warnf(\"Failed to list nodes, cannot decide existence of nodes of OS %s\", osType)\n\t\treturn false\n\t}\n\n\tallNodeLabeled := true\n\tfor _, node := range nodeList.Items {\n\t\tos, found := node.Labels[NodeOSLabel]\n\n\t\tif getRealOS(os) == osType {\n\t\t\treturn true\n\t\t}\n\n\t\tif !found {\n\t\t\tallNodeLabeled = false\n\t\t}\n\t}\n\n\tif !allNodeLabeled {\n\t\tlog.Warnf(\"Not all nodes have os type label, cannot decide existence of nodes of OS %s\", osType)\n\t}\n\n\treturn false\n}\n\nfunc GetNodeOS(ctx context.Context, nodeName string, nodeClient corev1client.CoreV1Interface) (string, error) {\n\tnode, err := nodeClient.Nodes().Get(context.Background(), nodeName, metav1.GetOptions{})\n\tif err != nil {\n\t\treturn \"\", errors.Wrapf(err, \"error getting node %s\", nodeName)\n\t}\n\n\tif node.Labels == nil {\n\t\treturn \"\", nil\n\t}\n\n\treturn getRealOS(node.Labels[NodeOSLabel]), nil\n}\n\nfunc HasNodeWithOS(ctx context.Context, os string, nodeClient corev1client.CoreV1Interface) error {\n\tif os == \"\" {\n\t\treturn errors.New(\"invalid node OS\")\n\t}\n\n\tnodes, err := nodeClient.Nodes().List(ctx, metav1.ListOptions{})\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error listing nodes with OS %s\", os)\n\t}\n\n\tfor _, node := range nodes.Items {\n\t\tosLabel, found := node.Labels[NodeOSLabel]\n\t\tif !found {\n\t\t\tcontinue\n\t\t}\n\n\t\tif getRealOS(osLabel) == os {\n\t\t\treturn nil\n\t\t}\n\t}\n\n\treturn errors.Errorf(\"node with OS %s doesn't exist\", os)\n}\n\nfunc getRealOS(osLabel string) string {\n\tif os, found := realNodeOSMap[osLabel]; !found {\n\t\treturn NodeOSLinux\n\t} else {\n\t\treturn os\n\t}\n}\n" }, { "path": "pkg/util/kube/node_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\n\tkubeClientFake \"k8s.io/client-go/kubernetes/fake\"\n\tclientTesting \"k8s.io/client-go/testing\"\n\tclientFake \"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestIsLinuxNode(t *testing.T) {\n\tnodeNoOSLabel := builder.ForNode(\"fake-node\").Result()\n\tnodeWindows := builder.ForNode(\"fake-node\").Labels(map[string]string{\"kubernetes.io/os\": \"windows\"}).Result()\n\tnodeLinux := builder.ForNode(\"fake-node\").Labels(map[string]string{\"kubernetes.io/os\": \"linux\"}).Result()\n\n\tscheme := runtime.NewScheme()\n\tcorev1api.AddToScheme(scheme)\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"error getting node\",\n\t\t\terr: \"error getting node fake-node: nodes \\\"fake-node\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"no os label\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tnodeNoOSLabel,\n\t\t\t},\n\t\t\terr: \"no os type label for node fake-node\",\n\t\t},\n\t\t{\n\t\t\tname: \"os label does not match\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tnodeWindows,\n\t\t\t},\n\t\t\terr: \"os type windows for node fake-node is not linux\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tnodeLinux,\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeClientBuilder := clientFake.NewClientBuilder()\n\t\t\tfakeClientBuilder = fakeClientBuilder.WithScheme(scheme)\n\n\t\t\tfakeClient := fakeClientBuilder.WithRuntimeObjects(test.kubeClientObj...).Build()\n\n\t\t\terr := IsLinuxNode(t.Context(), \"fake-node\", fakeClient)\n\t\t\tif err != nil {\n\t\t\t\tassert.EqualError(t, err, test.err)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestWithLinuxNode(t *testing.T) {\n\tnodeWindows := builder.ForNode(\"fake-node-1\").Labels(map[string]string{\"kubernetes.io/os\": \"windows\"}).Result()\n\tnodeLinux := builder.ForNode(\"fake-node-2\").Labels(map[string]string{\"kubernetes.io/os\": \"linux\"}).Result()\n\n\tscheme := runtime.NewScheme()\n\tcorev1api.AddToScheme(scheme)\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\tresult bool\n\t}{\n\t\t{\n\t\t\tname: \"error listing node\",\n\t\t},\n\t\t{\n\t\t\tname: \"with node of other type\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tnodeWindows,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with node of the same type\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tnodeWindows,\n\t\t\t\tnodeLinux,\n\t\t\t},\n\t\t\tresult: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeClientBuilder := clientFake.NewClientBuilder()\n\t\t\tfakeClientBuilder = fakeClientBuilder.WithScheme(scheme)\n\n\t\t\tfakeClient := fakeClientBuilder.WithRuntimeObjects(test.kubeClientObj...).Build()\n\n\t\t\tresult := withOSNode(t.Context(), fakeClient, \"linux\", velerotest.NewLogger())\n\t\t\tassert.Equal(t, test.result, result)\n\t\t})\n\t}\n}\n\nfunc TestGetNodeOSType(t *testing.T) {\n\tnodeNoOSLabel := builder.ForNode(\"fake-node\").Result()\n\tnodeWindows := builder.ForNode(\"fake-node\").Labels(map[string]string{\"kubernetes.io/os\": \"windows\"}).Result()\n\tnodeLinux := builder.ForNode(\"fake-node\").Labels(map[string]string{\"kubernetes.io/os\": \"linux\"}).Result()\n\tscheme := runtime.NewScheme()\n\tcorev1api.AddToScheme(scheme)\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\terr string\n\t\texpectedOSType string\n\t}{\n\t\t{\n\t\t\tname: \"error getting node\",\n\t\t\terr: \"error getting node fake-node: nodes \\\"fake-node\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"no os label\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tnodeNoOSLabel,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"windows node\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tnodeWindows,\n\t\t\t},\n\t\t\texpectedOSType: \"windows\",\n\t\t},\n\t\t{\n\t\t\tname: \"linux node\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tnodeLinux,\n\t\t\t},\n\t\t\texpectedOSType: \"linux\",\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := kubeClientFake.NewSimpleClientset(test.kubeClientObj...)\n\t\t\tosType, err := GetNodeOS(t.Context(), \"fake-node\", fakeKubeClient.CoreV1())\n\t\t\tif err != nil {\n\t\t\t\tassert.EqualError(t, err, test.err)\n\t\t\t} else {\n\t\t\t\tassert.Equal(t, test.expectedOSType, osType)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestHasNodeWithOS(t *testing.T) {\n\tnodeNoOSLabel := builder.ForNode(\"fake-node-1\").Result()\n\tnodeWindows := builder.ForNode(\"fake-node-2\").Labels(map[string]string{\"kubernetes.io/os\": \"windows\"}).Result()\n\tnodeLinux := builder.ForNode(\"fake-node-3\").Labels(map[string]string{\"kubernetes.io/os\": \"linux\"}).Result()\n\n\tscheme := runtime.NewScheme()\n\tcorev1api.AddToScheme(scheme)\n\n\ttests := []struct {\n\t\tname string\n\t\tkubeClientObj []runtime.Object\n\t\tkubeReactors []reactor\n\t\tos string\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"os is empty\",\n\t\t\terr: \"invalid node OS\",\n\t\t},\n\t\t{\n\t\t\tname: \"error to list node\",\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"list\",\n\t\t\t\t\tresource: \"nodes\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-list-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tos: \"linux\",\n\t\t\terr: \"error listing nodes with OS linux: fake-list-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"no expected node - no node\",\n\t\t\tos: \"linux\",\n\t\t\terr: \"node with OS linux doesn't exist\",\n\t\t},\n\t\t{\n\t\t\tname: \"no expected node - no node with label\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tnodeNoOSLabel,\n\t\t\t\tnodeWindows,\n\t\t\t},\n\t\t\tos: \"linux\",\n\t\t\terr: \"node with OS linux doesn't exist\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tnodeNoOSLabel,\n\t\t\t\tnodeWindows,\n\t\t\t\tnodeLinux,\n\t\t\t},\n\t\t\tos: \"windows\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := kubeClientFake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tfor _, reactor := range test.kubeReactors {\n\t\t\t\tfakeKubeClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\terr := HasNodeWithOS(t.Context(), test.os, fakeKubeClient.CoreV1())\n\t\t\tif test.err != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.err)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/util/kube/periodical_enqueue_source.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"reflect\"\n\t\"time\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"k8s.io/apimachinery/pkg/api/meta\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"k8s.io/client-go/util/workqueue\"\n\tctrl \"sigs.k8s.io/controller-runtime\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/event\"\n\t\"sigs.k8s.io/controller-runtime/pkg/predicate\"\n\t\"sigs.k8s.io/controller-runtime/pkg/reconcile\"\n)\n\nfunc NewPeriodicalEnqueueSource(\n\tlogger logrus.FieldLogger,\n\tclient client.Client,\n\tobjList client.ObjectList,\n\tperiod time.Duration,\n\toption PeriodicalEnqueueSourceOption) *PeriodicalEnqueueSource {\n\treturn &PeriodicalEnqueueSource{\n\t\tlogger: logger.WithField(\"resource\", reflect.TypeOf(objList).String()),\n\t\tClient: client,\n\t\tobjList: objList,\n\t\tperiod: period,\n\t\toption: option,\n\t}\n}\n\n// PeriodicalEnqueueSource is an implementation of interface sigs.k8s.io/controller-runtime/pkg/source/Source\n// It reads the specific resources from Kubernetes/cache and enqueues them into the queue to trigger\n// the reconcile logic periodically\ntype PeriodicalEnqueueSource struct {\n\tclient.Client\n\tlogger logrus.FieldLogger\n\tobjList client.ObjectList\n\tperiod time.Duration\n\toption PeriodicalEnqueueSourceOption\n}\n\ntype PeriodicalEnqueueSourceOption struct {\n\tOrderFunc func(objList client.ObjectList) client.ObjectList\n\tPredicates []predicate.Predicate // the predicates only apply to the GenericEvent\n}\n\n// Start enqueue items periodically\nfunc (p *PeriodicalEnqueueSource) Start(ctx context.Context, q workqueue.TypedRateLimitingInterface[reconcile.Request]) error {\n\tgo wait.Until(func() {\n\t\tp.logger.Debug(\"enqueueing resources ...\")\n\t\t// empty the list otherwise the result of the new list call will be appended\n\t\tif err := meta.SetList(p.objList, nil); err != nil {\n\t\t\tp.logger.WithError(err).Error(\"error reset resource list\")\n\t\t\treturn\n\t\t}\n\t\tif err := p.List(ctx, p.objList); err != nil {\n\t\t\tp.logger.WithError(err).Error(\"error listing resources\")\n\t\t\treturn\n\t\t}\n\t\tif meta.LenList(p.objList) == 0 {\n\t\t\tp.logger.Debug(\"no resources, skip\")\n\t\t\treturn\n\t\t}\n\t\tif p.option.OrderFunc != nil {\n\t\t\tp.objList = p.option.OrderFunc(p.objList)\n\t\t}\n\t\tif err := meta.EachListItem(p.objList, func(object runtime.Object) error {\n\t\t\tobj, ok := object.(client.Object)\n\t\t\tif !ok {\n\t\t\t\tp.logger.Error(\"%s's type isn't metav1.Object\", object.GetObjectKind().GroupVersionKind().String())\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\tevent := event.GenericEvent{Object: obj}\n\t\t\tfor _, predicate := range p.option.Predicates {\n\t\t\t\tif !predicate.Generic(event) {\n\t\t\t\t\tp.logger.Debugf(\"skip enqueue object %s/%s due to the predicate.\", obj.GetNamespace(), obj.GetName())\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tq.Add(ctrl.Request{\n\t\t\t\tNamespacedName: types.NamespacedName{\n\t\t\t\t\tNamespace: obj.GetNamespace(),\n\t\t\t\t\tName: obj.GetName(),\n\t\t\t\t},\n\t\t\t})\n\t\t\tp.logger.Debugf(\"resource %s/%s enqueued\", obj.GetNamespace(), obj.GetName())\n\t\t\treturn nil\n\t\t}); err != nil {\n\t\t\tp.logger.WithError(err).Error(\"error enqueueing resources\")\n\t\t\treturn\n\t\t}\n\t}, p.period, ctx.Done())\n\n\treturn nil\n}\n\nfunc (p *PeriodicalEnqueueSource) String() string {\n\tif p.objList != nil {\n\t\treturn fmt.Sprintf(\"periodical enqueue source: %T\", p.objList)\n\t}\n\treturn \"periodical enqueue source: unknown type\"\n}\n" }, { "path": "pkg/util/kube/periodical_enqueue_source_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t\"context\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/require\"\n\t\"k8s.io/apimachinery/pkg/api/meta\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes/scheme\"\n\t\"k8s.io/client-go/util/workqueue\"\n\tcrclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client/fake\"\n\t\"sigs.k8s.io/controller-runtime/pkg/predicate\"\n\t\"sigs.k8s.io/controller-runtime/pkg/reconcile\"\n\n\t\"github.com/vmware-tanzu/velero/internal/storage\"\n\tvelerov1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\nfunc TestStart(t *testing.T) {\n\trequire.NoError(t, velerov1.AddToScheme(scheme.Scheme))\n\n\tctx, cancelFunc := context.WithCancel(t.Context())\n\tclient := (&fake.ClientBuilder{}).Build()\n\tqueue := workqueue.NewTypedRateLimitingQueue(workqueue.DefaultTypedItemBasedRateLimiter[reconcile.Request]())\n\tsource := NewPeriodicalEnqueueSource(logrus.WithContext(ctx).WithField(\"controller\", \"PES_TEST\"), client, &velerov1.ScheduleList{}, 1*time.Second, PeriodicalEnqueueSourceOption{})\n\n\trequire.NoError(t, source.Start(ctx, queue))\n\n\t// no resources\n\ttime.Sleep(1 * time.Second)\n\trequire.Equal(t, 0, queue.Len())\n\n\t// contain one resource\n\trequire.NoError(t, client.Create(ctx, &velerov1.Schedule{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"schedule\",\n\t\t},\n\t}))\n\ttime.Sleep(2 * time.Second)\n\trequire.Equal(t, 1, queue.Len())\n\n\t// context canceled, the enqueue source shouldn't run anymore\n\titem, _ := queue.Get()\n\tqueue.Forget(item)\n\trequire.Equal(t, 0, queue.Len())\n\tcancelFunc()\n\ttime.Sleep(2 * time.Second)\n\trequire.Equal(t, 0, queue.Len())\n}\n\nfunc TestPredicate(t *testing.T) {\n\trequire.NoError(t, velerov1.AddToScheme(scheme.Scheme))\n\n\tctx, cancelFunc := context.WithCancel(t.Context())\n\tclient := (&fake.ClientBuilder{}).Build()\n\tqueue := workqueue.NewTypedRateLimitingQueue(workqueue.DefaultTypedItemBasedRateLimiter[reconcile.Request]())\n\n\tpred := NewGenericEventPredicate(func(object crclient.Object) bool {\n\t\tlocation := object.(*velerov1.BackupStorageLocation)\n\t\treturn storage.IsReadyToValidate(location.Spec.ValidationFrequency, location.Status.LastValidationTime, 1*time.Minute, logrus.WithContext(ctx).WithField(\"BackupStorageLocation\", location.Name))\n\t})\n\tsource := NewPeriodicalEnqueueSource(\n\t\tlogrus.WithContext(ctx).WithField(\"controller\", \"PES_TEST\"),\n\t\tclient,\n\t\t&velerov1.BackupStorageLocationList{},\n\t\t1*time.Second,\n\t\tPeriodicalEnqueueSourceOption{\n\t\t\tPredicates: []predicate.Predicate{pred},\n\t\t},\n\t)\n\n\trequire.NoError(t, source.Start(ctx, queue))\n\n\t// Should not patch a backup storage location object status phase\n\t// if the location's validation frequency is specifically set to zero\n\trequire.NoError(t, client.Create(ctx, &velerov1.BackupStorageLocation{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"location1\",\n\t\t\tNamespace: \"default\",\n\t\t},\n\t\tSpec: velerov1.BackupStorageLocationSpec{\n\t\t\tValidationFrequency: &metav1.Duration{Duration: 0},\n\t\t},\n\t\tStatus: velerov1.BackupStorageLocationStatus{\n\t\t\tLastValidationTime: &metav1.Time{Time: time.Now()},\n\t\t},\n\t}))\n\ttime.Sleep(2 * time.Second)\n\n\trequire.Equal(t, 0, queue.Len())\n\n\tcancelFunc()\n}\n\nfunc TestOrder(t *testing.T) {\n\trequire.NoError(t, velerov1.AddToScheme(scheme.Scheme))\n\n\tctx, cancelFunc := context.WithCancel(t.Context())\n\tclient := (&fake.ClientBuilder{}).Build()\n\tqueue := workqueue.NewTypedRateLimitingQueue(workqueue.DefaultTypedItemBasedRateLimiter[reconcile.Request]())\n\tsource := NewPeriodicalEnqueueSource(\n\t\tlogrus.WithContext(ctx).WithField(\"controller\", \"PES_TEST\"),\n\t\tclient,\n\t\t&velerov1.BackupStorageLocationList{},\n\t\t1*time.Second,\n\t\tPeriodicalEnqueueSourceOption{\n\t\t\tOrderFunc: func(objList crclient.ObjectList) crclient.ObjectList {\n\t\t\t\tlocationList := &velerov1.BackupStorageLocationList{}\n\t\t\t\tobjArray := make([]runtime.Object, 0)\n\n\t\t\t\t// Generate BSL array.\n\t\t\t\tlocations, _ := meta.ExtractList(objList)\n\t\t\t\t// Move default BSL to tail of array.\n\t\t\t\tobjArray = append(objArray, locations[1])\n\t\t\t\tobjArray = append(objArray, locations[0])\n\n\t\t\t\tmeta.SetList(locationList, objArray)\n\n\t\t\t\treturn locationList\n\t\t\t},\n\t\t},\n\t)\n\n\trequire.NoError(t, source.Start(ctx, queue))\n\n\t// Should not patch a backup storage location object status phase\n\t// if the location's validation frequency is specifically set to zero\n\trequire.NoError(t, client.Create(ctx, &velerov1.BackupStorageLocation{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"location1\",\n\t\t\tNamespace: \"default\",\n\t\t},\n\t\tSpec: velerov1.BackupStorageLocationSpec{\n\t\t\tValidationFrequency: &metav1.Duration{Duration: 0},\n\t\t},\n\t\tStatus: velerov1.BackupStorageLocationStatus{\n\t\t\tLastValidationTime: &metav1.Time{Time: time.Now()},\n\t\t},\n\t}))\n\trequire.NoError(t, client.Create(ctx, &velerov1.BackupStorageLocation{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"location2\",\n\t\t\tNamespace: \"default\",\n\t\t},\n\t\tSpec: velerov1.BackupStorageLocationSpec{\n\t\t\tValidationFrequency: &metav1.Duration{Duration: 0},\n\t\t\tDefault: true,\n\t\t},\n\t\tStatus: velerov1.BackupStorageLocationStatus{\n\t\t\tLastValidationTime: &metav1.Time{Time: time.Now()},\n\t\t},\n\t}))\n\ttime.Sleep(2 * time.Second)\n\n\tfirst, _ := queue.Get()\n\tbsl := &velerov1.BackupStorageLocation{}\n\trequire.Equal(t, \"location2\", first.Name)\n\trequire.NoError(t, client.Get(ctx, first.NamespacedName, bsl))\n\trequire.True(t, bsl.Spec.Default)\n\n\tcancelFunc()\n}\n" }, { "path": "pkg/util/kube/pod.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage kube\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\tcorev1client \"k8s.io/client-go/kubernetes/typed/core/v1\"\n)\n\ntype LoadAffinity struct {\n\t// NodeSelector specifies the label selector to match nodes\n\tNodeSelector metav1.LabelSelector `json:\"nodeSelector\"`\n\n\t// StorageClass specifies the VGDPs the LoadAffinity applied to. If the StorageClass doesn't have value, it applies to all. If not, it applies to only the VGDPs that use this StorageClass.\n\tStorageClass string `json:\"storageClass\"`\n}\n\ntype PodResources struct {\n\tCPURequest string `json:\"cpuRequest,omitempty\"`\n\tCPULimit string `json:\"cpuLimit,omitempty\"`\n\tMemoryRequest string `json:\"memoryRequest,omitempty\"`\n\tMemoryLimit string `json:\"memoryLimit,omitempty\"`\n\tEphemeralStorageRequest string `json:\"ephemeralStorageRequest,omitempty\"`\n\tEphemeralStorageLimit string `json:\"ephemeralStorageLimit,omitempty\"`\n}\n\n// IsPodRunning does a well-rounded check to make sure the specified pod is running stably.\n// If not, return the error found\nfunc IsPodRunning(pod *corev1api.Pod) error {\n\treturn isPodScheduledInStatus(pod, func(pod *corev1api.Pod) error {\n\t\tif pod.Status.Phase != corev1api.PodRunning {\n\t\t\treturn errors.New(\"pod is not running\")\n\t\t}\n\n\t\treturn nil\n\t})\n}\n\n// IsPodScheduled does a well-rounded check to make sure the specified pod has been scheduled into a node and in a stable status.\n// If not, return the error found\nfunc IsPodScheduled(pod *corev1api.Pod) error {\n\treturn isPodScheduledInStatus(pod, func(pod *corev1api.Pod) error {\n\t\tif pod.Status.Phase != corev1api.PodRunning && pod.Status.Phase != corev1api.PodPending {\n\t\t\treturn errors.New(\"pod is not running or pending\")\n\t\t}\n\n\t\treturn nil\n\t})\n}\n\nfunc isPodScheduledInStatus(pod *corev1api.Pod, statusCheckFunc func(*corev1api.Pod) error) error {\n\tif pod == nil {\n\t\treturn errors.New(\"invalid input pod\")\n\t}\n\n\tif pod.Spec.NodeName == \"\" {\n\t\treturn errors.Errorf(\"pod is not scheduled, name=%s, namespace=%s, phase=%s\", pod.Name, pod.Namespace, pod.Status.Phase)\n\t}\n\n\tif err := statusCheckFunc(pod); err != nil {\n\t\treturn errors.Wrapf(err, \"pod is not in the expected status, name=%s, namespace=%s, phase=%s\", pod.Name, pod.Namespace, pod.Status.Phase)\n\t}\n\n\tif pod.DeletionTimestamp != nil {\n\t\treturn errors.Errorf(\"pod is being terminated, name=%s, namespace=%s, phase=%s\", pod.Name, pod.Namespace, pod.Status.Phase)\n\t}\n\n\treturn nil\n}\n\n// DeletePodIfAny deletes a pod by name if it exists, and log an error when the deletion fails\nfunc DeletePodIfAny(ctx context.Context, podGetter corev1client.CoreV1Interface, podName string, podNamespace string, log logrus.FieldLogger) {\n\terr := podGetter.Pods(podNamespace).Delete(ctx, podName, metav1.DeleteOptions{})\n\tif err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.WithError(err).Debugf(\"Abort deleting pod, it doesn't exist %s/%s\", podNamespace, podName)\n\t\t} else {\n\t\t\tlog.WithError(err).Errorf(\"Failed to delete pod %s/%s\", podNamespace, podName)\n\t\t}\n\t}\n}\n\n// EnsureDeletePod asserts the existence of a pod by name, deletes it and waits for its disappearance and returns errors on any failure\nfunc EnsureDeletePod(ctx context.Context, podGetter corev1client.CoreV1Interface, pod string, namespace string, timeout time.Duration) error {\n\terr := podGetter.Pods(namespace).Delete(ctx, pod, metav1.DeleteOptions{})\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to delete pod %s\", pod)\n\t}\n\n\tvar updated *corev1api.Pod\n\terr = wait.PollUntilContextTimeout(ctx, waitInternal, timeout, true, func(ctx context.Context) (bool, error) {\n\t\tpo, err := podGetter.Pods(namespace).Get(ctx, pod, metav1.GetOptions{})\n\t\tif err != nil {\n\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\treturn true, nil\n\t\t\t}\n\n\t\t\treturn false, errors.Wrapf(err, \"error to get pod %s\", pod)\n\t\t}\n\n\t\tupdated = po\n\t\treturn false, nil\n\t})\n\n\tif err != nil {\n\t\tif errors.Is(err, context.DeadlineExceeded) {\n\t\t\treturn errors.Errorf(\"timeout to assure pod %s is deleted, finalizers in pod %v\", pod, updated.Finalizers)\n\t\t} else {\n\t\t\treturn errors.Wrapf(err, \"error to assure pod is deleted, %s\", pod)\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// IsPodUnrecoverable checks if the pod is in an abnormal state and could not be recovered\n// It could not cover all the cases but we could add more cases in the future\nfunc IsPodUnrecoverable(pod *corev1api.Pod, log logrus.FieldLogger) (bool, string) {\n\t// Check the Phase field\n\tif pod.Status.Phase == corev1api.PodFailed || pod.Status.Phase == corev1api.PodUnknown {\n\t\tmessage := \"\"\n\t\tif pod.Status.Message != \"\" {\n\t\t\tmessage += pod.Status.Message + \"/\"\n\t\t}\n\n\t\tmessage += GetPodTerminateMessage(pod)\n\n\t\tlog.Warnf(\"Pod is in abnormal state %s, message [%s]\", pod.Status.Phase, message)\n\t\treturn true, fmt.Sprintf(\"Pod is in abnormal state [%s], message [%s]\", pod.Status.Phase, message)\n\t}\n\n\t// removed \"Unschedulable\" check since unschedulable condition isn't always permanent\n\n\t// Check the Status field\n\tfor _, containerStatus := range pod.Status.ContainerStatuses {\n\t\t// If the container's image state is ImagePullBackOff, it indicates an image pull failure\n\t\tif containerStatus.State.Waiting != nil && (containerStatus.State.Waiting.Reason == \"ImagePullBackOff\" || containerStatus.State.Waiting.Reason == \"ErrImageNeverPull\") {\n\t\t\tlog.Warnf(\"Container %s in Pod %s/%s is in pull image failed with reason %s\", containerStatus.Name, pod.Namespace, pod.Name, containerStatus.State.Waiting.Reason)\n\t\t\treturn true, fmt.Sprintf(\"Container %s in Pod %s/%s is in pull image failed with reason %s\", containerStatus.Name, pod.Namespace, pod.Name, containerStatus.State.Waiting.Reason)\n\t\t}\n\t}\n\treturn false, \"\"\n}\n\n// GetPodContainerTerminateMessage returns the terminate message for a specific container of a pod\nfunc GetPodContainerTerminateMessage(pod *corev1api.Pod, container string) string {\n\tmessage := \"\"\n\tfor _, containerStatus := range pod.Status.ContainerStatuses {\n\t\tif containerStatus.Name == container {\n\t\t\tif containerStatus.State.Terminated != nil {\n\t\t\t\tmessage = containerStatus.State.Terminated.Message\n\t\t\t}\n\t\t\tbreak\n\t\t}\n\t}\n\n\treturn message\n}\n\n// GetPodTerminateMessage returns the terminate message for all containers of a pod\nfunc GetPodTerminateMessage(pod *corev1api.Pod) string {\n\tmessage := \"\"\n\tfor _, containerStatus := range pod.Status.ContainerStatuses {\n\t\tif containerStatus.State.Terminated != nil {\n\t\t\tif containerStatus.State.Terminated.Message != \"\" {\n\t\t\t\tmessage += containerStatus.State.Terminated.Message + \"/\"\n\t\t\t}\n\t\t}\n\t}\n\n\treturn message\n}\n\nfunc getPodLogReader(ctx context.Context, podGetter corev1client.CoreV1Interface, pod string, namespace string, logOptions *corev1api.PodLogOptions) (io.ReadCloser, error) {\n\trequest := podGetter.Pods(namespace).GetLogs(pod, logOptions)\n\treturn request.Stream(ctx)\n}\n\nvar podLogReaderGetter = getPodLogReader\n\n// CollectPodLogs collects logs of the specified container of a pod and write to the output\nfunc CollectPodLogs(ctx context.Context, podGetter corev1client.CoreV1Interface, pod string, namespace string, container string, output io.Writer) error {\n\tlogIndicator := fmt.Sprintf(\"***************************begin pod logs[%s/%s]***************************\\n\", pod, container)\n\n\tif _, err := output.Write([]byte(logIndicator)); err != nil {\n\t\treturn errors.Wrap(err, \"error to write begin pod log indicator\")\n\t}\n\n\tlogOptions := &corev1api.PodLogOptions{\n\t\tContainer: container,\n\t}\n\n\tif input, err := podLogReaderGetter(ctx, podGetter, pod, namespace, logOptions); err != nil {\n\t\tlogIndicator = fmt.Sprintf(\"No present log retrieved, err: %v\\n\", err)\n\t} else {\n\t\tif _, err := io.Copy(output, input); err != nil {\n\t\t\treturn errors.Wrap(err, \"error to copy input\")\n\t\t}\n\n\t\tlogIndicator = \"\"\n\t}\n\n\tlogIndicator += fmt.Sprintf(\"***************************end pod logs[%s/%s]***************************\\n\", pod, container)\n\tif _, err := output.Write([]byte(logIndicator)); err != nil {\n\t\treturn errors.Wrap(err, \"error to write end pod log indicator\")\n\t}\n\n\treturn nil\n}\n\nfunc ToSystemAffinity(loadAffinity *LoadAffinity, volumeTopology *corev1api.NodeSelector) *corev1api.Affinity {\n\trequirements := []corev1api.NodeSelectorRequirement{}\n\tif loadAffinity != nil {\n\t\tfor k, v := range loadAffinity.NodeSelector.MatchLabels {\n\t\t\trequirements = append(requirements, corev1api.NodeSelectorRequirement{\n\t\t\t\tKey: k,\n\t\t\t\tValues: []string{v},\n\t\t\t\tOperator: corev1api.NodeSelectorOpIn,\n\t\t\t})\n\t\t}\n\n\t\tfor _, exp := range loadAffinity.NodeSelector.MatchExpressions {\n\t\t\trequirements = append(requirements, corev1api.NodeSelectorRequirement{\n\t\t\t\tKey: exp.Key,\n\t\t\t\tValues: exp.Values,\n\t\t\t\tOperator: corev1api.NodeSelectorOperator(exp.Operator),\n\t\t\t})\n\t\t}\n\t}\n\n\tresult := new(corev1api.Affinity)\n\tresult.NodeAffinity = new(corev1api.NodeAffinity)\n\tresult.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution = new(corev1api.NodeSelector)\n\n\tif volumeTopology != nil {\n\t\tresult.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution.NodeSelectorTerms = append(result.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution.NodeSelectorTerms, volumeTopology.NodeSelectorTerms...)\n\t} else if len(requirements) > 0 {\n\t\tresult.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution.NodeSelectorTerms = make([]corev1api.NodeSelectorTerm, 1)\n\t} else {\n\t\treturn nil\n\t}\n\n\tfor i := range result.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution.NodeSelectorTerms {\n\t\tresult.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution.NodeSelectorTerms[i].MatchExpressions = append(result.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution.NodeSelectorTerms[i].MatchExpressions, requirements...)\n\t}\n\n\treturn result\n}\n\nfunc DiagnosePod(pod *corev1api.Pod, events *corev1api.EventList) string {\n\tdiag := fmt.Sprintf(\"Pod %s/%s, phase %s, node name %s, message %s\\n\", pod.Namespace, pod.Name, pod.Status.Phase, pod.Spec.NodeName, pod.Status.Message)\n\n\tfor _, condition := range pod.Status.Conditions {\n\t\tdiag += fmt.Sprintf(\"Pod condition %s, status %s, reason %s, message %s\\n\", condition.Type, condition.Status, condition.Reason, condition.Message)\n\t}\n\n\tif events != nil {\n\t\tfor _, e := range events.Items {\n\t\t\tif e.InvolvedObject.UID == pod.UID && e.Type == corev1api.EventTypeWarning {\n\t\t\t\tdiag += fmt.Sprintf(\"Pod event reason %s, message %s\\n\", e.Reason, e.Message)\n\t\t\t}\n\t\t}\n\t}\n\n\treturn diag\n}\n\nvar funcExit = os.Exit\nvar funcCreateFile = os.Create\n\nfunc ExitPodWithMessage(logger logrus.FieldLogger, succeed bool, message string, a ...any) {\n\texitCode := 0\n\tif !succeed {\n\t\texitCode = 1\n\t}\n\n\ttoWrite := fmt.Sprintf(message, a...)\n\n\tpodFile, err := funcCreateFile(\"/dev/termination-log\")\n\tif err != nil {\n\t\tlogger.WithError(err).Error(\"Failed to create termination log file\")\n\t\texitCode = 1\n\t} else {\n\t\tif _, err := podFile.WriteString(toWrite); err != nil {\n\t\t\tlogger.WithError(err).Error(\"Failed to write error to termination log file\")\n\t\t\texitCode = 1\n\t\t}\n\n\t\tpodFile.Close()\n\t}\n\n\tfuncExit(exitCode)\n}\n\n// GetLoadAffinityByStorageClass retrieves the LoadAffinity from the parameter affinityList.\n// The function first try to find by the scName. If there is no such LoadAffinity,\n// it will try to get the LoadAffinity whose StorageClass has no value.\nfunc GetLoadAffinityByStorageClass(\n\taffinityList []*LoadAffinity,\n\tscName string,\n\tlogger logrus.FieldLogger,\n) *LoadAffinity {\n\tvar globalAffinity *LoadAffinity\n\n\tfor _, affinity := range affinityList {\n\t\tif affinity.StorageClass == scName {\n\t\t\tlogger.WithField(\"StorageClass\", scName).Info(\"Found pod's affinity setting per StorageClass.\")\n\t\t\treturn affinity\n\t\t}\n\n\t\tif affinity.StorageClass == \"\" && globalAffinity == nil {\n\t\t\tglobalAffinity = affinity\n\t\t}\n\t}\n\n\tif globalAffinity != nil {\n\t\tlogger.Info(\"Use the Global affinity for pod.\")\n\t} else {\n\t\tlogger.Info(\"No Affinity is found for pod.\")\n\t}\n\n\treturn globalAffinity\n}\n" }, { "path": "pkg/util/kube/pod_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"reflect\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/google/uuid\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"k8s.io/client-go/kubernetes/fake\"\n\n\tclientTesting \"k8s.io/client-go/testing\"\n\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\n\tcorev1client \"k8s.io/client-go/kubernetes/typed/core/v1\"\n)\n\nfunc TestEnsureDeletePod(t *testing.T) {\n\tpodObject := &corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"fake-pod\",\n\t\t},\n\t}\n\n\tpodObjectWithFinalizer := &corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"fake-pod\",\n\t\t\tFinalizers: []string{\"fake-finalizer-1\", \"fake-finalizer-2\"},\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tclientObj []runtime.Object\n\t\tpodName string\n\t\tnamespace string\n\t\treactors []reactor\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"delete fail\",\n\t\t\tpodName: \"fake-pod\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\terr: \"error to delete pod fake-pod: pods \\\"fake-pod\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"wait timeout\",\n\t\t\tpodName: \"fake-pod\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tclientObj: []runtime.Object{podObjectWithFinalizer},\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"pods\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"timeout to assure pod fake-pod is deleted, finalizers in pod [fake-finalizer-1 fake-finalizer-2]\",\n\t\t},\n\t\t{\n\t\t\tname: \"wait timeout, no finalizer\",\n\t\t\tpodName: \"fake-pod\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tclientObj: []runtime.Object{podObject},\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"pods\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"timeout to assure pod fake-pod is deleted, finalizers in pod []\",\n\t\t},\n\t\t{\n\t\t\tname: \"wait fail\",\n\t\t\tpodName: \"fake-pod\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tclientObj: []runtime.Object{podObject},\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"get\",\n\t\t\t\t\tresource: \"pods\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-get-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to assure pod is deleted, fake-pod: error to get pod fake-pod: fake-get-error\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.clientObj...)\n\n\t\t\tfor _, reactor := range test.reactors {\n\t\t\t\tfakeKubeClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\tvar kubeClient kubernetes.Interface = fakeKubeClient\n\n\t\t\terr := EnsureDeletePod(t.Context(), kubeClient.CoreV1(), test.podName, test.namespace, time.Millisecond)\n\t\t\tif err != nil {\n\t\t\t\tassert.EqualError(t, err, test.err)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestIsPodRunning(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpod *corev1api.Pod\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"pod is nil\",\n\t\t\terr: \"invalid input pod\",\n\t\t},\n\t\t{\n\t\t\tname: \"pod is not scheduled\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\tName: \"fake-pod\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tPhase: \"fake-phase\",\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"pod is not scheduled, name=fake-pod, namespace=fake-ns, phase=fake-phase\",\n\t\t},\n\t\t{\n\t\t\tname: \"pod is not running\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\tName: \"fake-pod\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tNodeName: \"fake-node\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tPhase: \"fake-phase\",\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"pod is not in the expected status, name=fake-pod, namespace=fake-ns, phase=fake-phase: pod is not running\",\n\t\t},\n\t\t{\n\t\t\tname: \"pod is being deleted\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\tName: \"fake-pod\",\n\t\t\t\t\tDeletionTimestamp: &metav1.Time{Time: time.Now()},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tNodeName: \"fake-node\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tPhase: corev1api.PodRunning,\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"pod is being terminated, name=fake-pod, namespace=fake-ns, phase=Running\",\n\t\t},\n\t\t{\n\t\t\tname: \"success\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\tName: \"fake-pod\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tNodeName: \"fake-node\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tPhase: corev1api.PodRunning,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\terr := IsPodRunning(test.pod)\n\t\t\tif err != nil {\n\t\t\t\tassert.EqualError(t, err, test.err)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestIsPodScheduled(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpod *corev1api.Pod\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"pod is nil\",\n\t\t\terr: \"invalid input pod\",\n\t\t},\n\t\t{\n\t\t\tname: \"pod is not scheduled\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\tName: \"fake-pod\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tPhase: \"fake-phase\",\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"pod is not scheduled, name=fake-pod, namespace=fake-ns, phase=fake-phase\",\n\t\t},\n\t\t{\n\t\t\tname: \"pod is not running or pending\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\tName: \"fake-pod\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tNodeName: \"fake-node\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tPhase: \"fake-phase\",\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"pod is not in the expected status, name=fake-pod, namespace=fake-ns, phase=fake-phase: pod is not running or pending\",\n\t\t},\n\t\t{\n\t\t\tname: \"pod is being deleted\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\tName: \"fake-pod\",\n\t\t\t\t\tDeletionTimestamp: &metav1.Time{Time: time.Now()},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tNodeName: \"fake-node\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tPhase: corev1api.PodRunning,\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"pod is being terminated, name=fake-pod, namespace=fake-ns, phase=Running\",\n\t\t},\n\t\t{\n\t\t\tname: \"success on running\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\tName: \"fake-pod\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tNodeName: \"fake-node\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tPhase: corev1api.PodRunning,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"success on pending\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\tName: \"fake-pod\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tNodeName: \"fake-node\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tPhase: corev1api.PodPending,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\terr := IsPodScheduled(test.pod)\n\t\t\tif err != nil {\n\t\t\t\tassert.EqualError(t, err, test.err)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestDeletePodIfAny(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpodName string\n\t\tpodNamespace string\n\t\tkubeClientObj []runtime.Object\n\t\tkubeReactors []reactor\n\t\tlogMessage string\n\t\tlogLevel string\n\t\tlogError string\n\t}{\n\t\t{\n\t\t\tname: \"get fail\",\n\t\t\tpodName: \"fake-pod\",\n\t\t\tpodNamespace: \"fake-namespace\",\n\t\t\tlogMessage: \"Abort deleting pod, it doesn't exist fake-namespace/fake-pod\",\n\t\t\tlogLevel: \"level=debug\",\n\t\t},\n\t\t{\n\t\t\tname: \"delete fail\",\n\t\t\tpodName: \"fake-pod\",\n\t\t\tpodNamespace: \"fake-namespace\",\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"pods\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-delete-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tlogMessage: \"Failed to delete pod fake-namespace/fake-pod\",\n\t\t\tlogLevel: \"level=error\",\n\t\t\tlogError: \"error=fake-delete-error\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tfor _, reactor := range test.kubeReactors {\n\t\t\t\tfakeKubeClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\tvar kubeClient kubernetes.Interface = fakeKubeClient\n\n\t\t\tlogMessage := \"\"\n\t\t\tDeletePodIfAny(t.Context(), kubeClient.CoreV1(), test.podName, test.podNamespace, velerotest.NewSingleLogger(&logMessage))\n\n\t\t\tif len(test.logMessage) > 0 {\n\t\t\t\tassert.Contains(t, logMessage, test.logMessage)\n\t\t\t}\n\n\t\t\tif len(test.logLevel) > 0 {\n\t\t\t\tassert.Contains(t, logMessage, test.logLevel)\n\t\t\t}\n\n\t\t\tif len(test.logError) > 0 {\n\t\t\t\tassert.Contains(t, logMessage, test.logError)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestIsPodUnrecoverable(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpod *corev1api.Pod\n\t\twant bool\n\t}{\n\t\t{\n\t\t\tname: \"pod is in failed state\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tPhase: corev1api.PodFailed,\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: true,\n\t\t},\n\t\t{\n\t\t\tname: \"pod is in unknown state\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tPhase: corev1api.PodUnknown,\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: true,\n\t\t},\n\t\t{\n\t\t\tname: \"container image pull failure\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t\t\t\t{State: corev1api.ContainerState{Waiting: &corev1api.ContainerStateWaiting{Reason: \"ImagePullBackOff\"}}},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: true,\n\t\t},\n\t\t{\n\t\t\tname: \"container image pull failure with different reason\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t\t\t\t{State: corev1api.ContainerState{Waiting: &corev1api.ContainerStateWaiting{Reason: \"ErrImageNeverPull\"}}},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: true,\n\t\t},\n\t\t{\n\t\t\tname: \"container image pull failure with different reason\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t\t\t\t{State: corev1api.ContainerState{Waiting: &corev1api.ContainerStateWaiting{Reason: \"OtherReason\"}}},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"pod is normal\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tPhase: corev1api.PodRunning,\n\t\t\t\t\tContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t\t\t\t{Ready: true, State: corev1api.ContainerState{Running: &corev1api.ContainerStateRunning{}}},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: false,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tgot, _ := IsPodUnrecoverable(test.pod, velerotest.NewLogger())\n\t\t\tassert.Equal(t, test.want, got)\n\t\t})\n\t}\n}\n\nfunc TestGetPodTerminateMessage(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpod *corev1api.Pod\n\t\tmessage string\n\t}{\n\t\t{\n\t\t\tname: \"empty message when no container status\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tPhase: corev1api.PodFailed,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"empty message when no termination status\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t\t\t\t{Name: \"container-1\", State: corev1api.ContainerState{Waiting: &corev1api.ContainerStateWaiting{Reason: \"ImagePullBackOff\"}}},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"empty message when no termination message\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t\t\t\t{Name: \"container-1\", State: corev1api.ContainerState{Terminated: &corev1api.ContainerStateTerminated{Reason: \"fake-reason\"}}},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with termination message\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t\t\t\t{Name: \"container-1\", State: corev1api.ContainerState{Terminated: &corev1api.ContainerStateTerminated{Message: \"message-1\"}}},\n\t\t\t\t\t\t{Name: \"container-2\", State: corev1api.ContainerState{Terminated: &corev1api.ContainerStateTerminated{Message: \"message-2\"}}},\n\t\t\t\t\t\t{Name: \"container-3\", State: corev1api.ContainerState{Terminated: &corev1api.ContainerStateTerminated{Message: \"message-3\"}}},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tmessage: \"message-1/message-2/message-3/\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tmessage := GetPodTerminateMessage(test.pod)\n\t\t\tassert.Equal(t, test.message, message)\n\t\t})\n\t}\n}\n\nfunc TestGetPodContainerTerminateMessage(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpod *corev1api.Pod\n\t\tcontainer string\n\t\tmessage string\n\t}{\n\t\t{\n\t\t\tname: \"empty message when no container status\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tPhase: corev1api.PodFailed,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"empty message when no termination status\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t\t\t\t{Name: \"container-1\", State: corev1api.ContainerState{Waiting: &corev1api.ContainerStateWaiting{Reason: \"ImagePullBackOff\"}}},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tcontainer: \"container-1\",\n\t\t},\n\t\t{\n\t\t\tname: \"empty message when no termination message\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t\t\t\t{Name: \"container-1\", State: corev1api.ContainerState{Terminated: &corev1api.ContainerStateTerminated{Reason: \"fake-reason\"}}},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tcontainer: \"container-1\",\n\t\t},\n\t\t{\n\t\t\tname: \"not matched container name\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t\t\t\t{Name: \"container-1\", State: corev1api.ContainerState{Terminated: &corev1api.ContainerStateTerminated{Message: \"message-1\"}}},\n\t\t\t\t\t\t{Name: \"container-2\", State: corev1api.ContainerState{Terminated: &corev1api.ContainerStateTerminated{Message: \"message-2\"}}},\n\t\t\t\t\t\t{Name: \"container-3\", State: corev1api.ContainerState{Terminated: &corev1api.ContainerStateTerminated{Message: \"message-3\"}}},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tcontainer: \"container-0\",\n\t\t},\n\t\t{\n\t\t\tname: \"with termination message\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tContainerStatuses: []corev1api.ContainerStatus{\n\t\t\t\t\t\t{Name: \"container-1\", State: corev1api.ContainerState{Terminated: &corev1api.ContainerStateTerminated{Message: \"message-1\"}}},\n\t\t\t\t\t\t{Name: \"container-2\", State: corev1api.ContainerState{Terminated: &corev1api.ContainerStateTerminated{Message: \"message-2\"}}},\n\t\t\t\t\t\t{Name: \"container-3\", State: corev1api.ContainerState{Terminated: &corev1api.ContainerStateTerminated{Message: \"message-3\"}}},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tcontainer: \"container-2\",\n\t\t\tmessage: \"message-2\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tmessage := GetPodContainerTerminateMessage(test.pod, test.container)\n\t\t\tassert.Equal(t, test.message, message)\n\t\t})\n\t}\n}\n\ntype fakePodLog struct {\n\tgetError error\n\treadError error\n\tbeginWriteError error\n\tendWriteError error\n\twriteError error\n\tlogMessage string\n\toutputMessage string\n\treadPos int\n}\n\nfunc (fp *fakePodLog) GetPodLogReader(ctx context.Context, podGetter corev1client.CoreV1Interface, pod string, namespace string, logOptions *corev1api.PodLogOptions) (io.ReadCloser, error) {\n\tif fp.getError != nil {\n\t\treturn nil, fp.getError\n\t}\n\n\treturn fp, nil\n}\n\nfunc (fp *fakePodLog) Read(p []byte) (n int, err error) {\n\tif fp.readError != nil {\n\t\treturn -1, fp.readError\n\t}\n\n\tif fp.readPos == len(fp.logMessage) {\n\t\treturn 0, io.EOF\n\t}\n\n\tcopy(p, []byte(fp.logMessage))\n\tfp.readPos += len(fp.logMessage)\n\n\treturn len(fp.logMessage), nil\n}\n\nfunc (fp *fakePodLog) Close() error {\n\treturn nil\n}\n\nfunc (fp *fakePodLog) Write(p []byte) (n int, err error) {\n\tmessage := string(p)\n\tif strings.Contains(message, \"begin pod logs\") {\n\t\tif fp.beginWriteError != nil {\n\t\t\treturn -1, fp.beginWriteError\n\t\t}\n\t} else if strings.Contains(message, \"end pod logs\") {\n\t\tif fp.endWriteError != nil {\n\t\t\treturn -1, fp.endWriteError\n\t\t}\n\t} else {\n\t\tif fp.writeError != nil {\n\t\t\treturn -1, fp.writeError\n\t\t}\n\t}\n\n\tfp.outputMessage += message\n\n\treturn len(message), nil\n}\n\nfunc TestCollectPodLogs(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpod string\n\t\tcontainer string\n\t\tgetError error\n\t\treadError error\n\t\tbeginWriteError error\n\t\tendWriteError error\n\t\twriteError error\n\t\treadMessage string\n\t\tmessage string\n\t\texpectErr string\n\t}{\n\t\t{\n\t\t\tname: \"error to write begin indicator\",\n\t\t\tbeginWriteError: errors.New(\"fake-write-error-01\"),\n\t\t\texpectErr: \"error to write begin pod log indicator: fake-write-error-01\",\n\t\t},\n\t\t{\n\t\t\tname: \"error to get log\",\n\t\t\tpod: \"fake-pod\",\n\t\t\tcontainer: \"fake-container\",\n\t\t\tgetError: errors.New(\"fake-get-error\"),\n\t\t\tmessage: \"***************************begin pod logs[fake-pod/fake-container]***************************\\nNo present log retrieved, err: fake-get-error\\n***************************end pod logs[fake-pod/fake-container]***************************\\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"error to read pod log\",\n\t\t\tpod: \"fake-pod\",\n\t\t\tcontainer: \"fake-container\",\n\t\t\treadError: errors.New(\"fake-read-error\"),\n\t\t\texpectErr: \"error to copy input: fake-read-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"error to write pod log\",\n\t\t\tpod: \"fake-pod\",\n\t\t\tcontainer: \"fake-container\",\n\t\t\twriteError: errors.New(\"fake-write-error-03\"),\n\t\t\treadMessage: \"fake pod message 01\\n fake pod message 02\\n fake pod message 03\\n\",\n\t\t\texpectErr: \"error to copy input: fake-write-error-03\",\n\t\t},\n\t\t{\n\t\t\tname: \"error to write end indicator\",\n\t\t\tpod: \"fake-pod\",\n\t\t\tcontainer: \"fake-container\",\n\t\t\tendWriteError: errors.New(\"fake-write-error-02\"),\n\t\t\treadMessage: \"fake pod message 01\\n fake pod message 02\\n fake pod message 03\\n\",\n\t\t\texpectErr: \"error to write end pod log indicator: fake-write-error-02\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tpod: \"fake-pod\",\n\t\t\tcontainer: \"fake-container\",\n\t\t\treadMessage: \"fake pod message 01\\n fake pod message 02\\n fake pod message 03\\n\",\n\t\t\tmessage: \"***************************begin pod logs[fake-pod/fake-container]***************************\\nfake pod message 01\\n fake pod message 02\\n fake pod message 03\\n***************************end pod logs[fake-pod/fake-container]***************************\\n\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfp := &fakePodLog{\n\t\t\t\tgetError: test.getError,\n\t\t\t\treadError: test.readError,\n\t\t\t\tbeginWriteError: test.beginWriteError,\n\t\t\t\tendWriteError: test.endWriteError,\n\t\t\t\twriteError: test.writeError,\n\t\t\t\tlogMessage: test.readMessage,\n\t\t\t}\n\t\t\tpodLogReaderGetter = fp.GetPodLogReader\n\n\t\t\terr := CollectPodLogs(t.Context(), nil, test.pod, \"\", test.container, fp)\n\t\t\tif test.expectErr != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.expectErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\tassert.Equal(t, fp.outputMessage, test.message)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestToSystemAffinity(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tloadAffinity *LoadAffinity\n\t\tvolumeTopology *corev1api.NodeSelector\n\t\texpected *corev1api.Affinity\n\t}{\n\t\t{\n\t\t\tname: \"loadAffinity is nil\",\n\t\t},\n\t\t{\n\t\t\tname: \"loadAffinity is empty\",\n\t\t\tloadAffinity: &LoadAffinity{},\n\t\t},\n\t\t{\n\t\t\tname: \"with match label\",\n\t\t\tloadAffinity: &LoadAffinity{\n\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\tMatchLabels: map[string]string{\n\t\t\t\t\t\t\"key-1\": \"value-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: &corev1api.Affinity{\n\t\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-1\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-1\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpIn,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with match expression\",\n\t\t\tloadAffinity: &LoadAffinity{\n\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\tMatchLabels: map[string]string{\n\t\t\t\t\t\t\"key-2\": \"value-2\",\n\t\t\t\t\t},\n\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKey: \"key-3\",\n\t\t\t\t\t\t\tValues: []string{\"value-3-1\", \"value-3-2\"},\n\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpNotIn,\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKey: \"key-4\",\n\t\t\t\t\t\t\tValues: []string{\"value-4-1\", \"value-4-2\", \"value-4-3\"},\n\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpDoesNotExist,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: &corev1api.Affinity{\n\t\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-2\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-2\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpIn,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-3\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-3-1\", \"value-3-2\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-4\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-4-1\", \"value-4-2\", \"value-4-3\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpDoesNotExist,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with olume topology\",\n\t\t\tvolumeTopology: &corev1api.NodeSelector{\n\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t{\n\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"key-5\",\n\t\t\t\t\t\t\t\tValues: []string{\"value-5-1\", \"value-5-2\", \"value-5-3\"},\n\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"key-6\",\n\t\t\t\t\t\t\t\tValues: []string{\"value-5-1\", \"value-5-2\", \"value-5-3\"},\n\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"key-7\",\n\t\t\t\t\t\t\t\tValues: []string{\"value-7-1\", \"value-7-2\", \"value-7-3\"},\n\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"key-8\",\n\t\t\t\t\t\t\t\tValues: []string{\"value-8-1\", \"value-8-2\", \"value-8-3\"},\n\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tMatchFields: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"key-9\",\n\t\t\t\t\t\t\t\tValues: []string{\"value-9-1\", \"value-9-2\", \"value-9-3\"},\n\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"key-a\",\n\t\t\t\t\t\t\t\tValues: []string{\"value-a-1\", \"value-a-2\", \"value-a-3\"},\n\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: &corev1api.Affinity{\n\t\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-5\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-5-1\", \"value-5-2\", \"value-5-3\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-6\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-5-1\", \"value-5-2\", \"value-5-3\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-7\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-7-1\", \"value-7-2\", \"value-7-3\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-8\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-8-1\", \"value-8-2\", \"value-8-3\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchFields: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-9\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-9-1\", \"value-9-2\", \"value-9-3\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-a\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-a-1\", \"value-a-2\", \"value-a-3\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"with match expression and volume topology\",\n\t\t\tloadAffinity: &LoadAffinity{\n\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\tMatchLabels: map[string]string{\n\t\t\t\t\t\t\"key-2\": \"value-2\",\n\t\t\t\t\t},\n\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKey: \"key-3\",\n\t\t\t\t\t\t\tValues: []string{\"value-3-1\", \"value-3-2\"},\n\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpNotIn,\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKey: \"key-4\",\n\t\t\t\t\t\t\tValues: []string{\"value-4-1\", \"value-4-2\", \"value-4-3\"},\n\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpDoesNotExist,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolumeTopology: &corev1api.NodeSelector{\n\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t{\n\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"key-5\",\n\t\t\t\t\t\t\t\tValues: []string{\"value-5-1\", \"value-5-2\", \"value-5-3\"},\n\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"key-6\",\n\t\t\t\t\t\t\t\tValues: []string{\"value-5-1\", \"value-5-2\", \"value-5-3\"},\n\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"key-7\",\n\t\t\t\t\t\t\t\tValues: []string{\"value-7-1\", \"value-7-2\", \"value-7-3\"},\n\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"key-8\",\n\t\t\t\t\t\t\t\tValues: []string{\"value-8-1\", \"value-8-2\", \"value-8-3\"},\n\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tMatchFields: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"key-9\",\n\t\t\t\t\t\t\t\tValues: []string{\"value-9-1\", \"value-9-2\", \"value-9-3\"},\n\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"key-a\",\n\t\t\t\t\t\t\t\tValues: []string{\"value-a-1\", \"value-a-2\", \"value-a-3\"},\n\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: &corev1api.Affinity{\n\t\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-5\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-5-1\", \"value-5-2\", \"value-5-3\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-6\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-5-1\", \"value-5-2\", \"value-5-3\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-2\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-2\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpIn,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-3\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-3-1\", \"value-3-2\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-4\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-4-1\", \"value-4-2\", \"value-4-3\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpDoesNotExist,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-7\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-7-1\", \"value-7-2\", \"value-7-3\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-8\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-8-1\", \"value-8-2\", \"value-8-3\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-2\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-2\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpIn,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-3\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-3-1\", \"value-3-2\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-4\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-4-1\", \"value-4-2\", \"value-4-3\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpDoesNotExist,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-2\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-2\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpIn,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-3\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-3-1\", \"value-3-2\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpNotIn,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-4\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-4-1\", \"value-4-2\", \"value-4-3\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpDoesNotExist,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\tMatchFields: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-9\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-9-1\", \"value-9-2\", \"value-9-3\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"key-a\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"value-a-1\", \"value-a-2\", \"value-a-3\"},\n\t\t\t\t\t\t\t\t\t\tOperator: corev1api.NodeSelectorOpGt,\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\taffinity := ToSystemAffinity(test.loadAffinity, test.volumeTopology)\n\t\t\tassert.True(t, reflect.DeepEqual(affinity, test.expected))\n\t\t})\n\t}\n}\n\nfunc TestDiagnosePod(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tpod *corev1api.Pod\n\t\tevents *corev1api.EventList\n\t\texpected string\n\t}{\n\t\t{\n\t\t\tname: \"pod with all info but event\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-pod\",\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tNodeName: \"fake-node\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tPhase: corev1api.PodPending,\n\t\t\t\t\tConditions: []corev1api.PodCondition{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tType: corev1api.PodInitialized,\n\t\t\t\t\t\t\tStatus: corev1api.ConditionTrue,\n\t\t\t\t\t\t\tReason: \"fake-reason-1\",\n\t\t\t\t\t\t\tMessage: \"fake-message-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tType: corev1api.PodScheduled,\n\t\t\t\t\t\t\tStatus: corev1api.ConditionFalse,\n\t\t\t\t\t\t\tReason: \"fake-reason-2\",\n\t\t\t\t\t\t\tMessage: \"fake-message-2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tMessage: \"fake-message-3\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: \"Pod fake-ns/fake-pod, phase Pending, node name fake-node, message fake-message-3\\nPod condition Initialized, status True, reason fake-reason-1, message fake-message-1\\nPod condition PodScheduled, status False, reason fake-reason-2, message fake-message-2\\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"pod with all info and empty event list\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-pod\",\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tNodeName: \"fake-node\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tPhase: corev1api.PodPending,\n\t\t\t\t\tConditions: []corev1api.PodCondition{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tType: corev1api.PodInitialized,\n\t\t\t\t\t\t\tStatus: corev1api.ConditionTrue,\n\t\t\t\t\t\t\tReason: \"fake-reason-1\",\n\t\t\t\t\t\t\tMessage: \"fake-message-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tType: corev1api.PodScheduled,\n\t\t\t\t\t\t\tStatus: corev1api.ConditionFalse,\n\t\t\t\t\t\t\tReason: \"fake-reason-2\",\n\t\t\t\t\t\t\tMessage: \"fake-message-2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tMessage: \"fake-message-3\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tevents: &corev1api.EventList{},\n\t\t\texpected: \"Pod fake-ns/fake-pod, phase Pending, node name fake-node, message fake-message-3\\nPod condition Initialized, status True, reason fake-reason-1, message fake-message-1\\nPod condition PodScheduled, status False, reason fake-reason-2, message fake-message-2\\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"pod with all info and events\",\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-pod\",\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\tUID: \"fake-pod-uid\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tNodeName: \"fake-node\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PodStatus{\n\t\t\t\t\tPhase: corev1api.PodPending,\n\t\t\t\t\tConditions: []corev1api.PodCondition{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tType: corev1api.PodInitialized,\n\t\t\t\t\t\t\tStatus: corev1api.ConditionTrue,\n\t\t\t\t\t\t\tReason: \"fake-reason-1\",\n\t\t\t\t\t\t\tMessage: \"fake-message-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tType: corev1api.PodScheduled,\n\t\t\t\t\t\t\tStatus: corev1api.ConditionFalse,\n\t\t\t\t\t\t\tReason: \"fake-reason-2\",\n\t\t\t\t\t\t\tMessage: \"fake-message-2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tMessage: \"fake-message-3\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tevents: &corev1api.EventList{Items: []corev1api.Event{\n\t\t\t\t{\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-uid-1\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tReason: \"reason-1\",\n\t\t\t\t\tMessage: \"message-1\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-uid-2\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tReason: \"reason-2\",\n\t\t\t\t\tMessage: \"message-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-pod-uid\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tReason: \"reason-3\",\n\t\t\t\t\tMessage: \"message-3\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-pod-uid\"},\n\t\t\t\t\tType: corev1api.EventTypeNormal,\n\t\t\t\t\tReason: \"reason-4\",\n\t\t\t\t\tMessage: \"message-4\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-pod-uid\"},\n\t\t\t\t\tType: corev1api.EventTypeNormal,\n\t\t\t\t\tReason: \"reason-5\",\n\t\t\t\t\tMessage: \"message-5\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-pod-uid\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tReason: \"reason-6\",\n\t\t\t\t\tMessage: \"message-6\",\n\t\t\t\t},\n\t\t\t}},\n\t\t\texpected: \"Pod fake-ns/fake-pod, phase Pending, node name fake-node, message fake-message-3\\nPod condition Initialized, status True, reason fake-reason-1, message fake-message-1\\nPod condition PodScheduled, status False, reason fake-reason-2, message fake-message-2\\nPod event reason reason-3, message message-3\\nPod event reason reason-6, message message-6\\n\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tdiag := DiagnosePod(tc.pod, tc.events)\n\t\t\tassert.Equal(t, tc.expected, diag)\n\t\t})\n\t}\n}\n\ntype exitWithMessageMock struct {\n\tcreateErr error\n\twriteFail bool\n\tfilePath string\n\texitCode int\n}\n\nfunc (em *exitWithMessageMock) Exit(code int) {\n\tem.exitCode = code\n}\n\nfunc (em *exitWithMessageMock) CreateFile(name string) (*os.File, error) {\n\tif em.createErr != nil {\n\t\treturn nil, em.createErr\n\t}\n\n\tif em.writeFail {\n\t\treturn os.OpenFile(em.filePath, os.O_CREATE|os.O_RDONLY, 0500)\n\t} else {\n\t\treturn os.Create(em.filePath)\n\t}\n}\n\nfunc TestExitPodWithMessage(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tmessage string\n\t\tsucceed bool\n\t\targs []any\n\t\tcreateErr error\n\t\twriteFail bool\n\t\texpectedExitCode int\n\t\texpectedMessage string\n\t}{\n\t\t{\n\t\t\tname: \"create pod file failed\",\n\t\t\tcreateErr: errors.New(\"fake-create-file-error\"),\n\t\t\tsucceed: true,\n\t\t\texpectedExitCode: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"write pod file failed\",\n\t\t\twriteFail: true,\n\t\t\tsucceed: true,\n\t\t\texpectedExitCode: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"not succeed\",\n\t\t\tmessage: \"fake-message-1, arg-1 %s, arg-2 %v, arg-3 %v\",\n\t\t\targs: []any{\n\t\t\t\t\"arg-1-1\",\n\t\t\t\t10,\n\t\t\t\tfalse,\n\t\t\t},\n\t\t\texpectedExitCode: 1,\n\t\t\texpectedMessage: fmt.Sprintf(\"fake-message-1, arg-1 %s, arg-2 %v, arg-3 %v\", \"arg-1-1\", 10, false),\n\t\t},\n\t\t{\n\t\t\tname: \"not succeed\",\n\t\t\tmessage: \"fake-message-2, arg-1 %s, arg-2 %v, arg-3 %v\",\n\t\t\targs: []any{\n\t\t\t\t\"arg-1-2\",\n\t\t\t\t20,\n\t\t\t\ttrue,\n\t\t\t},\n\t\t\tsucceed: true,\n\t\t\texpectedMessage: fmt.Sprintf(\"fake-message-2, arg-1 %s, arg-2 %v, arg-3 %v\", \"arg-1-2\", 20, true),\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tpodFile := filepath.Join(os.TempDir(), uuid.NewString())\n\n\t\t\tem := exitWithMessageMock{\n\t\t\t\tcreateErr: test.createErr,\n\t\t\t\twriteFail: test.writeFail,\n\t\t\t\tfilePath: podFile,\n\t\t\t}\n\n\t\t\tfuncExit = em.Exit\n\t\t\tfuncCreateFile = em.CreateFile\n\n\t\t\tExitPodWithMessage(velerotest.NewLogger(), test.succeed, test.message, test.args...)\n\n\t\t\tassert.Equal(t, test.expectedExitCode, em.exitCode)\n\n\t\t\tif test.createErr == nil && !test.writeFail {\n\t\t\t\treader, err := os.Open(podFile)\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\tmessage, err := io.ReadAll(reader)\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\treader.Close()\n\n\t\t\t\tassert.Equal(t, test.expectedMessage, string(message))\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetLoadAffinityByStorageClass(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\taffinityList []*LoadAffinity\n\t\tscName string\n\t\texpectedAffinity *LoadAffinity\n\t}{\n\t\t{\n\t\t\tname: \"get global affinity\",\n\t\t\taffinityList: []*LoadAffinity{\n\t\t\t\t{\n\t\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"kubernetes.io/arch\",\n\t\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\t\t\tValues: []string{\"amd64\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tStorageClass: \"storage-class-01\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\t\t\tValues: []string{\"Linux\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tscName: \"\",\n\t\t\texpectedAffinity: &LoadAffinity{\n\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\t\tValues: []string{\"Linux\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"get affinity for StorageClass but only global affinity exists\",\n\t\t\taffinityList: []*LoadAffinity{\n\t\t\t\t{\n\t\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\t\t\tValues: []string{\"Linux\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"kubernetes.io/arch\",\n\t\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\t\t\tValues: []string{\"amd64\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\t\t\tValues: []string{\"Windows\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tscName: \"storage-class-01\",\n\t\t\texpectedAffinity: &LoadAffinity{\n\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\t\tValues: []string{\"Linux\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"get affinity for StorageClass\",\n\t\t\taffinityList: []*LoadAffinity{\n\t\t\t\t{\n\t\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"kubernetes.io/control-plane=\",\n\t\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\t\t\tValues: []string{\"\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\t\t\tValues: []string{\"Linux\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tStorageClass: \"storage-class-01\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"kubernetes.io/arch\",\n\t\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\t\t\tValues: []string{\"amd64\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tStorageClass: \"invalid-storage-class\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tscName: \"storage-class-01\",\n\t\t\texpectedAffinity: &LoadAffinity{\n\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tKey: \"kubernetes.io/os\",\n\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\t\tValues: []string{\"Linux\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStorageClass: \"storage-class-01\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"Cannot find a match Affinity\",\n\t\t\taffinityList: []*LoadAffinity{\n\t\t\t\t{\n\t\t\t\t\tNodeSelector: metav1.LabelSelector{\n\t\t\t\t\t\tMatchExpressions: []metav1.LabelSelectorRequirement{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"kubernetes.io/arch\",\n\t\t\t\t\t\t\t\tOperator: metav1.LabelSelectorOpIn,\n\t\t\t\t\t\t\t\tValues: []string{\"amd64\"},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tStorageClass: \"invalid-storage-class\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tscName: \"storage-class-01\",\n\t\t\texpectedAffinity: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"affinityList is nil\",\n\t\t\taffinityList: nil,\n\t\t\tscName: \"storage-class-01\",\n\t\t\texpectedAffinity: nil,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tresult := GetLoadAffinityByStorageClass(test.affinityList, test.scName, velerotest.NewLogger())\n\n\t\t\tassert.Equal(t, test.expectedAffinity, result)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/util/kube/predicate.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"reflect\"\n\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/event\"\n\t\"sigs.k8s.io/controller-runtime/pkg/predicate\"\n)\n\n// SpecChangePredicate implements a default update predicate function on Spec change\n// As Velero doesn't enable subresource in CRDs, we cannot use the object's metadata.generation field to check the spec change\n// More details about the generation field refer to https://github.com/kubernetes-sigs/controller-runtime/blob/v0.12.2/pkg/predicate/predicate.go#L156\ntype SpecChangePredicate struct {\n\tpredicate.Funcs\n}\n\nfunc (SpecChangePredicate) Update(e event.UpdateEvent) bool {\n\tif e.ObjectOld == nil {\n\t\treturn false\n\t}\n\tif e.ObjectNew == nil {\n\t\treturn false\n\t}\n\toldSpec := reflect.ValueOf(e.ObjectOld).Elem().FieldByName(\"Spec\")\n\t// contains no field named \"Spec\", return false directly\n\tif oldSpec.IsZero() {\n\t\treturn false\n\t}\n\tnewSpec := reflect.ValueOf(e.ObjectNew).Elem().FieldByName(\"Spec\")\n\treturn !reflect.DeepEqual(oldSpec.Interface(), newSpec.Interface())\n}\n\n// NewAllEventPredicate creates a new Predicate that checks all the events with the provided func\nfunc NewAllEventPredicate(f func(object client.Object) bool) predicate.Predicate {\n\treturn predicate.Funcs{\n\t\tCreateFunc: func(event event.CreateEvent) bool {\n\t\t\treturn f(event.Object)\n\t\t},\n\t\tDeleteFunc: func(event event.DeleteEvent) bool {\n\t\t\treturn f(event.Object)\n\t\t},\n\t\tUpdateFunc: func(event event.UpdateEvent) bool {\n\t\t\treturn f(event.ObjectNew)\n\t\t},\n\t\tGenericFunc: func(event event.GenericEvent) bool {\n\t\t\treturn f(event.Object)\n\t\t},\n\t}\n}\n\n// FalsePredicate always returns false for all kinds of events\ntype FalsePredicate struct{}\n\n// Create always returns false\nfunc (f FalsePredicate) Create(event.CreateEvent) bool {\n\treturn false\n}\n\n// Delete always returns false\nfunc (f FalsePredicate) Delete(event.DeleteEvent) bool {\n\treturn false\n}\n\n// Update always returns false\nfunc (f FalsePredicate) Update(event.UpdateEvent) bool {\n\treturn false\n}\n\n// Generic always returns false\nfunc (f FalsePredicate) Generic(event.GenericEvent) bool {\n\treturn false\n}\n\n// NewGenericEventPredicate creates a new Predicate that checks the Generic event with the provided func\nfunc NewGenericEventPredicate(f func(object client.Object) bool) predicate.Predicate {\n\treturn predicate.Funcs{\n\t\tGenericFunc: func(event event.GenericEvent) bool {\n\t\t\treturn f(event.Object)\n\t\t},\n\t}\n}\n\n// NewUpdateEventPredicate creates a new Predicate that checks the Update event with the provided func\nfunc NewUpdateEventPredicate(\n\tf func(objectOld client.Object, objectNew client.Object) bool,\n) predicate.Predicate {\n\treturn predicate.Funcs{\n\t\tUpdateFunc: func(event event.UpdateEvent) bool {\n\t\t\treturn f(event.ObjectOld, event.ObjectNew)\n\t\t},\n\t}\n}\n\nfunc NewCreateEventPredicate(\n\tf func(object client.Object) bool,\n) predicate.Predicate {\n\treturn predicate.Funcs{\n\t\tCreateFunc: func(event event.CreateEvent) bool {\n\t\t\treturn f(event.Object)\n\t\t},\n\t}\n}\n" }, { "path": "pkg/util/kube/predicate_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/testify/assert\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\t\"sigs.k8s.io/controller-runtime/pkg/event\"\n\n\tvelerov1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\nfunc TestSpecChangePredicate(t *testing.T) {\n\tcases := []struct {\n\t\tname string\n\t\toldObj client.Object\n\t\tnewObj client.Object\n\t\tchanged bool\n\t}{\n\t\t{\n\t\t\tname: \"Contains no spec field\",\n\t\t\toldObj: &velerov1.BackupStorageLocation{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"bsl01\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tnewObj: &velerov1.BackupStorageLocation{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"bsl01\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tchanged: false,\n\t\t},\n\t\t{\n\t\t\tname: \"ObjectMetas are different, Specs are same\",\n\t\t\toldObj: &velerov1.BackupStorageLocation{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"bsl01\",\n\t\t\t\t\tAnnotations: map[string]string{\"key1\": \"value1\"},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"azure\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tnewObj: &velerov1.BackupStorageLocation{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"bsl01\",\n\t\t\t\t\tAnnotations: map[string]string{\"key2\": \"value2\"},\n\t\t\t\t},\n\t\t\t\tSpec: velerov1.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"azure\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tchanged: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Statuses are different, Specs are same\",\n\t\t\toldObj: &velerov1.BackupStorageLocation{\n\t\t\t\tSpec: velerov1.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"azure\",\n\t\t\t\t},\n\t\t\t\tStatus: velerov1.BackupStorageLocationStatus{\n\t\t\t\t\tPhase: velerov1.BackupStorageLocationPhaseAvailable,\n\t\t\t\t},\n\t\t\t},\n\t\t\tnewObj: &velerov1.BackupStorageLocation{\n\t\t\t\tSpec: velerov1.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"azure\",\n\t\t\t\t},\n\t\t\t\tStatus: velerov1.BackupStorageLocationStatus{\n\t\t\t\t\tPhase: velerov1.BackupStorageLocationPhaseUnavailable,\n\t\t\t\t},\n\t\t\t},\n\t\t\tchanged: false,\n\t\t},\n\t\t{\n\t\t\tname: \"Specs are different\",\n\t\t\toldObj: &velerov1.BackupStorageLocation{\n\t\t\t\tSpec: velerov1.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"azure\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tnewObj: &velerov1.BackupStorageLocation{\n\t\t\t\tSpec: velerov1.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"aws\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tchanged: true,\n\t\t},\n\t\t{\n\t\t\tname: \"Specs are same\",\n\t\t\toldObj: &velerov1.BackupStorageLocation{\n\t\t\t\tSpec: velerov1.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"azure\",\n\t\t\t\t\tConfig: map[string]string{\"key\": \"value\"},\n\t\t\t\t\tCredential: &corev1api.SecretKeySelector{\n\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\tName: \"secret\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tKey: \"credential\",\n\t\t\t\t\t},\n\t\t\t\t\tStorageType: velerov1.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"bucket1\",\n\t\t\t\t\t\t\tPrefix: \"prefix\",\n\t\t\t\t\t\t\tCACert: []byte{'a'},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tDefault: true,\n\t\t\t\t\tAccessMode: velerov1.BackupStorageLocationAccessModeReadWrite,\n\t\t\t\t\tBackupSyncPeriod: &metav1.Duration{\n\t\t\t\t\t\tDuration: 1 * time.Minute,\n\t\t\t\t\t},\n\t\t\t\t\tValidationFrequency: &metav1.Duration{\n\t\t\t\t\t\tDuration: 1 * time.Minute,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tnewObj: &velerov1.BackupStorageLocation{\n\t\t\t\tSpec: velerov1.BackupStorageLocationSpec{\n\t\t\t\t\tProvider: \"azure\",\n\t\t\t\t\tConfig: map[string]string{\"key\": \"value\"},\n\t\t\t\t\tCredential: &corev1api.SecretKeySelector{\n\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\tName: \"secret\",\n\t\t\t\t\t\t},\n\t\t\t\t\t\tKey: \"credential\",\n\t\t\t\t\t},\n\t\t\t\t\tStorageType: velerov1.StorageType{\n\t\t\t\t\t\tObjectStorage: &velerov1.ObjectStorageLocation{\n\t\t\t\t\t\t\tBucket: \"bucket1\",\n\t\t\t\t\t\t\tPrefix: \"prefix\",\n\t\t\t\t\t\t\tCACert: []byte{'a'},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tDefault: true,\n\t\t\t\t\tAccessMode: velerov1.BackupStorageLocationAccessModeReadWrite,\n\t\t\t\t\tBackupSyncPeriod: &metav1.Duration{\n\t\t\t\t\t\tDuration: 1 * time.Minute,\n\t\t\t\t\t},\n\t\t\t\t\tValidationFrequency: &metav1.Duration{\n\t\t\t\t\t\tDuration: 1 * time.Minute,\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tchanged: false,\n\t\t},\n\t}\n\n\tpredicate := SpecChangePredicate{}\n\n\tfor _, c := range cases {\n\t\tt.Run(c.name, func(t *testing.T) {\n\t\t\tchanged := predicate.Update(event.UpdateEvent{\n\t\t\t\tObjectOld: c.oldObj,\n\t\t\t\tObjectNew: c.newObj,\n\t\t\t})\n\t\t\tassert.Equal(t, c.changed, changed)\n\t\t})\n\t}\n}\n\nfunc TestNewAllEventPredicate(t *testing.T) {\n\tpredicate := NewAllEventPredicate(func(object client.Object) bool {\n\t\treturn false\n\t})\n\n\tassert.False(t, predicate.Create(event.CreateEvent{}))\n\tassert.False(t, predicate.Update(event.UpdateEvent{}))\n\tassert.False(t, predicate.Delete(event.DeleteEvent{}))\n\tassert.False(t, predicate.Generic(event.GenericEvent{}))\n}\n\nfunc TestNewGenericEventPredicate(t *testing.T) {\n\tpredicate := NewGenericEventPredicate(func(object client.Object) bool {\n\t\treturn false\n\t})\n\n\tassert.False(t, predicate.Generic(event.GenericEvent{}))\n\tassert.True(t, predicate.Update(event.UpdateEvent{}))\n\tassert.True(t, predicate.Create(event.CreateEvent{}))\n\tassert.True(t, predicate.Delete(event.DeleteEvent{}))\n}\n\nfunc TestNewUpdateEventPredicate(t *testing.T) {\n\tpredicate := NewUpdateEventPredicate(\n\t\tfunc(client.Object, client.Object) bool {\n\t\t\treturn false\n\t\t},\n\t)\n\n\tassert.False(t, predicate.Update(event.UpdateEvent{}))\n\tassert.True(t, predicate.Create(event.CreateEvent{}))\n\tassert.True(t, predicate.Delete(event.DeleteEvent{}))\n\tassert.True(t, predicate.Generic(event.GenericEvent{}))\n}\n\nfunc TestNewCreateEventPredicate(t *testing.T) {\n\tpredicate := NewCreateEventPredicate(\n\t\tfunc(client.Object) bool {\n\t\t\treturn false\n\t\t},\n\t)\n\n\tassert.False(t, predicate.Create(event.CreateEvent{}))\n\tassert.True(t, predicate.Update(event.UpdateEvent{}))\n\tassert.True(t, predicate.Generic(event.GenericEvent{}))\n\tassert.True(t, predicate.Delete(event.DeleteEvent{}))\n}\n" }, { "path": "pkg/util/kube/priority_class.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"context\"\n\n\t\"github.com/sirupsen/logrus\"\n\tschedulingv1 \"k8s.io/api/scheduling/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n)\n\n// ValidatePriorityClass checks if the specified priority class exists in the cluster\n// Returns true if the priority class exists or if priorityClassName is empty\n// Returns false if the priority class doesn't exist or validation fails\n// Logs warnings when the priority class doesn't exist\nfunc ValidatePriorityClass(ctx context.Context, kubeClient kubernetes.Interface, priorityClassName string, logger logrus.FieldLogger) bool {\n\tif priorityClassName == \"\" {\n\t\treturn true\n\t}\n\n\t_, err := kubeClient.SchedulingV1().PriorityClasses().Get(ctx, priorityClassName, metav1.GetOptions{})\n\tif err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlogger.Warnf(\"Priority class %q not found in cluster. Pod creation may fail if the priority class doesn't exist when pods are scheduled.\", priorityClassName)\n\t\t} else {\n\t\t\tlogger.WithError(err).Warnf(\"Failed to validate priority class %q\", priorityClassName)\n\t\t}\n\t\treturn false\n\t}\n\tlogger.Infof(\"Validated priority class %q exists in cluster\", priorityClassName)\n\treturn true\n}\n\n// ValidatePriorityClassWithClient checks if the specified priority class exists in the cluster using controller-runtime client\n// Returns nil if the priority class exists or if priorityClassName is empty\n// Returns error if the priority class doesn't exist or validation fails\nfunc ValidatePriorityClassWithClient(ctx context.Context, cli client.Client, priorityClassName string) error {\n\tif priorityClassName == \"\" {\n\t\treturn nil\n\t}\n\n\tpriorityClass := &schedulingv1.PriorityClass{}\n\terr := cli.Get(ctx, types.NamespacedName{Name: priorityClassName}, priorityClass)\n\treturn err\n}\n" }, { "path": "pkg/util/kube/priority_class_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"fmt\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\tschedulingv1 \"k8s.io/api/scheduling/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes/fake\"\n\tk8stesting \"k8s.io/client-go/testing\"\n\n\tvelerotesting \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestValidatePriorityClass(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpriorityClassName string\n\t\texistingPCs []runtime.Object\n\t\tclientReactors []k8stesting.ReactionFunc\n\t\texpectedLogs []string\n\t\texpectedLogLevel string\n\t\texpectedResult bool\n\t}{\n\t\t{\n\t\t\tname: \"empty priority class name should return without logging\",\n\t\t\tpriorityClassName: \"\",\n\t\t\texistingPCs: nil,\n\t\t\texpectedLogs: nil,\n\t\t\texpectedResult: true,\n\t\t},\n\t\t{\n\t\t\tname: \"existing priority class should log info message\",\n\t\t\tpriorityClassName: \"high-priority\",\n\t\t\texistingPCs: []runtime.Object{\n\t\t\t\t&schedulingv1.PriorityClass{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"high-priority\",\n\t\t\t\t\t},\n\t\t\t\t\tValue: 100,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedLogs: []string{\"Validated priority class \\\\\\\"high-priority\\\\\\\" exists in cluster\"},\n\t\t\texpectedLogLevel: \"info\",\n\t\t\texpectedResult: true,\n\t\t},\n\t\t{\n\t\t\tname: \"non-existing priority class should log warning\",\n\t\t\tpriorityClassName: \"does-not-exist\",\n\t\t\texistingPCs: nil,\n\t\t\texpectedLogs: []string{\"Priority class \\\\\\\"does-not-exist\\\\\\\" not found in cluster. Pod creation may fail if the priority class doesn't exist when pods are scheduled.\"},\n\t\t\texpectedLogLevel: \"warning\",\n\t\t\texpectedResult: false,\n\t\t},\n\t\t{\n\t\t\tname: \"API error should log warning with error\",\n\t\t\tpriorityClassName: \"test-priority\",\n\t\t\texistingPCs: nil,\n\t\t\tclientReactors: []k8stesting.ReactionFunc{\n\t\t\t\tfunc(action k8stesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\tif action.GetVerb() == \"get\" && action.GetResource().Resource == \"priorityclasses\" {\n\t\t\t\t\t\treturn true, nil, fmt.Errorf(\"API server error\")\n\t\t\t\t\t}\n\t\t\t\t\treturn false, nil, nil\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedLogs: []string{\"Failed to validate priority class \\\\\\\"test-priority\\\\\\\"\"},\n\t\t\texpectedLogLevel: \"warning\",\n\t\t\texpectedResult: false,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\t// Create fake client with existing priority classes\n\t\t\tkubeClient := fake.NewSimpleClientset(test.existingPCs...)\n\n\t\t\t// Add any custom reactors\n\t\t\tfor _, reactor := range test.clientReactors {\n\t\t\t\tkubeClient.PrependReactor(\"*\", \"*\", reactor)\n\t\t\t}\n\n\t\t\t// Create test logger with buffer\n\t\t\tbuffer := []string{}\n\t\t\tlogger := velerotesting.NewMultipleLogger(&buffer)\n\n\t\t\t// Call the function\n\t\t\tresult := ValidatePriorityClass(t.Context(), kubeClient, test.priorityClassName, logger)\n\n\t\t\t// Check result\n\t\t\tassert.Equal(t, test.expectedResult, result, \"ValidatePriorityClass returned unexpected result\")\n\n\t\t\t// Check logs\n\t\t\tif test.expectedLogs == nil {\n\t\t\t\tassert.Empty(t, buffer)\n\t\t\t} else {\n\t\t\t\tassert.Len(t, buffer, len(test.expectedLogs))\n\n\t\t\t\tfor i, expectedLog := range test.expectedLogs {\n\t\t\t\t\tassert.Contains(t, buffer[i], expectedLog)\n\t\t\t\t\tif test.expectedLogLevel == \"info\" {\n\t\t\t\t\t\tassert.Contains(t, buffer[i], \"level=info\")\n\t\t\t\t\t} else if test.expectedLogLevel == \"warning\" {\n\t\t\t\t\t\tassert.Contains(t, buffer[i], \"level=warning\")\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/util/kube/pvc_pv.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"strings\"\n\t\"time\"\n\n\tjsonpatch \"github.com/evanphx/json-patch/v5\"\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/types\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\tcorev1client \"k8s.io/client-go/kubernetes/typed/core/v1\"\n\tcrclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tstoragev1api \"k8s.io/api/storage/v1\"\n\tstoragev1 \"k8s.io/client-go/kubernetes/typed/storage/v1\"\n)\n\nconst (\n\twaitInternal = 2 * time.Second\n)\n\n// DeletePVAndPVCIfAny deletes PVC and delete the bound PV if it exists and log an error when the deletion fails.\n// It first sets the reclaim policy of the PV to Delete, then PV will be deleted automatically when PVC is deleted.\n// If ensureTimeout is not 0, it waits until the PVC is deleted or timeout.\nfunc DeletePVAndPVCIfAny(ctx context.Context, client corev1client.CoreV1Interface, pvcName, pvcNamespace string, ensureTimeout time.Duration, log logrus.FieldLogger) {\n\tpvcObj, err := client.PersistentVolumeClaims(pvcNamespace).Get(ctx, pvcName, metav1.GetOptions{})\n\tif err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.WithError(err).Debugf(\"Abort deleting PV and PVC, for related PVC doesn't exist, %s/%s\", pvcNamespace, pvcName)\n\t\t\treturn\n\t\t} else {\n\t\t\tlog.Warnf(\"failed to get pvc %s/%s with err %v\", pvcNamespace, pvcName, err)\n\t\t\treturn\n\t\t}\n\t}\n\n\tif pvcObj.Spec.VolumeName == \"\" {\n\t\tlog.Warnf(\"failed to delete PV, for related PVC %s/%s has no bind volume name\", pvcNamespace, pvcName)\n\t} else {\n\t\tpvObj, err := client.PersistentVolumes().Get(ctx, pvcObj.Spec.VolumeName, metav1.GetOptions{})\n\t\tif err != nil {\n\t\t\tlog.Warnf(\"failed to delete PV %s with err %v\", pvcObj.Spec.VolumeName, err)\n\t\t} else {\n\t\t\t_, err = SetPVReclaimPolicy(ctx, client, pvObj, corev1api.PersistentVolumeReclaimDelete)\n\t\t\tif err != nil {\n\t\t\t\tlog.Warnf(\"failed to set reclaim policy of PV %s to delete with err %v\", pvObj.Name, err)\n\t\t\t}\n\t\t}\n\t}\n\n\tif err := EnsureDeletePVC(ctx, client, pvcName, pvcNamespace, ensureTimeout); err != nil {\n\t\tlog.Warnf(\"failed to delete pvc %s/%s with err %v\", pvcNamespace, pvcName, err)\n\t}\n\n\tif err := EnsurePVDeleted(ctx, client, pvcObj.Spec.VolumeName, ensureTimeout); err != nil {\n\t\tlog.Warnf(\"pv %s was not removed with err %v\", pvcObj.Spec.VolumeName, err)\n\t}\n}\n\n// WaitPVCBound wait for binding of a PVC specified by name and returns the bound PV object\nfunc WaitPVCBound(ctx context.Context, pvcGetter corev1client.CoreV1Interface,\n\tpvGetter corev1client.CoreV1Interface, pvc string, namespace string, timeout time.Duration) (*corev1api.PersistentVolume, error) {\n\tvar updated *corev1api.PersistentVolumeClaim\n\terr := wait.PollUntilContextTimeout(ctx, waitInternal, timeout, true, func(ctx context.Context) (bool, error) {\n\t\ttmpPVC, err := pvcGetter.PersistentVolumeClaims(namespace).Get(ctx, pvc, metav1.GetOptions{})\n\t\tif err != nil {\n\t\t\treturn false, errors.Wrapf(err, \"error to get pvc %s/%s\", namespace, pvc)\n\t\t}\n\n\t\tif tmpPVC.Spec.VolumeName == \"\" {\n\t\t\treturn false, nil\n\t\t}\n\n\t\tupdated = tmpPVC\n\n\t\treturn true, nil\n\t})\n\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error to wait for rediness of PVC\")\n\t}\n\n\tpv, err := pvGetter.PersistentVolumes().Get(ctx, updated.Spec.VolumeName, metav1.GetOptions{})\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error to get PV\")\n\t}\n\n\treturn pv, err\n}\n\n// DeletePVIfAny deletes a PV by name if it exists, and log an error when the deletion fails\nfunc DeletePVIfAny(ctx context.Context, pvGetter corev1client.CoreV1Interface, pvName string, log logrus.FieldLogger) {\n\terr := pvGetter.PersistentVolumes().Delete(ctx, pvName, metav1.DeleteOptions{})\n\tif err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\tlog.WithError(err).Debugf(\"Abort deleting PV, it doesn't exist, %s\", pvName)\n\t\t} else {\n\t\t\tlog.WithError(err).Errorf(\"Failed to delete PV %s\", pvName)\n\t\t}\n\t}\n}\n\n// EnsureDeletePVC asserts the existence of a PVC by name, deletes it and waits for its disappearance and returns errors on any failure\n// If timeout is 0, it doesn't wait and return nil\nfunc EnsureDeletePVC(ctx context.Context, pvcGetter corev1client.CoreV1Interface, pvcName string, namespace string, timeout time.Duration) error {\n\terr := pvcGetter.PersistentVolumeClaims(namespace).Delete(ctx, pvcName, metav1.DeleteOptions{})\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"error to delete pvc %s\", pvcName)\n\t}\n\n\tif timeout == 0 {\n\t\treturn nil\n\t}\n\n\tvar updated *corev1api.PersistentVolumeClaim\n\terr = wait.PollUntilContextTimeout(ctx, waitInternal, timeout, true, func(ctx context.Context) (bool, error) {\n\t\tpvc, err := pvcGetter.PersistentVolumeClaims(namespace).Get(ctx, pvcName, metav1.GetOptions{})\n\t\tif err != nil {\n\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\treturn true, nil\n\t\t\t}\n\n\t\t\treturn false, errors.Wrapf(err, \"error to get pvc %s\", pvcName)\n\t\t}\n\n\t\tupdated = pvc\n\t\treturn false, nil\n\t})\n\n\tif err != nil {\n\t\tif errors.Is(err, context.DeadlineExceeded) {\n\t\t\treturn errors.Errorf(\"timeout to assure pvc %s is deleted, finalizers in pvc %v\", pvcName, updated.Finalizers)\n\t\t} else {\n\t\t\treturn errors.Wrapf(err, \"error to ensure pvc deleted for %s\", pvcName)\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// EnsurePVDeleted ensures a PV has been deleted. This function is supposed to be called after EnsureDeletePVC\n// If timeout is 0, it doesn't wait and return nil\nfunc EnsurePVDeleted(ctx context.Context, pvGetter corev1client.CoreV1Interface, pvName string, timeout time.Duration) error {\n\tif timeout == 0 {\n\t\treturn nil\n\t}\n\n\terr := wait.PollUntilContextTimeout(ctx, waitInternal, timeout, true, func(ctx context.Context) (bool, error) {\n\t\t_, err := pvGetter.PersistentVolumes().Get(ctx, pvName, metav1.GetOptions{})\n\t\tif err != nil {\n\t\t\tif apierrors.IsNotFound(err) {\n\t\t\t\treturn true, nil\n\t\t\t}\n\n\t\t\treturn false, errors.Wrapf(err, \"error to get pv %s\", pvName)\n\t\t}\n\n\t\treturn false, nil\n\t})\n\n\tif err != nil {\n\t\tif errors.Is(err, context.DeadlineExceeded) {\n\t\t\treturn errors.Errorf(\"timeout to assure pv %s is deleted\", pvName)\n\t\t} else {\n\t\t\treturn errors.Wrapf(err, \"error to ensure pv is deleted for %s\", pvName)\n\t\t}\n\t}\n\n\treturn nil\n}\n\n// RebindPVC rebinds a PVC by modifying its VolumeName to the specific PV\nfunc RebindPVC(ctx context.Context, pvcGetter corev1client.CoreV1Interface,\n\tpvc *corev1api.PersistentVolumeClaim, pv string) (*corev1api.PersistentVolumeClaim, error) {\n\torigBytes, err := json.Marshal(pvc)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error marshaling original PVC\")\n\t}\n\n\tupdated := pvc.DeepCopy()\n\tupdated.Spec.VolumeName = pv\n\tdelete(updated.Annotations, KubeAnnBindCompleted)\n\tdelete(updated.Annotations, KubeAnnBoundByController)\n\n\tupdatedBytes, err := json.Marshal(updated)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error marshaling updated PV\")\n\t}\n\n\tpatchBytes, err := jsonpatch.CreateMergePatch(origBytes, updatedBytes)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error creating json merge patch for PV\")\n\t}\n\n\tupdated, err = pvcGetter.PersistentVolumeClaims(pvc.Namespace).Patch(ctx, pvc.Name, types.MergePatchType, patchBytes, metav1.PatchOptions{})\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error patching PVC\")\n\t}\n\n\treturn updated, nil\n}\n\n// ResetPVBinding resets the binding info of a PV and adds the required labels so as to make it ready for binding\nfunc ResetPVBinding(ctx context.Context, pvGetter corev1client.CoreV1Interface, pv *corev1api.PersistentVolume,\n\tlabels map[string]string, pvc *corev1api.PersistentVolumeClaim) (*corev1api.PersistentVolume, error) {\n\torigBytes, err := json.Marshal(pv)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error marshaling original PV\")\n\t}\n\n\tupdated := pv.DeepCopy()\n\tupdated.Spec.ClaimRef = &corev1api.ObjectReference{\n\t\tKind: pvc.Kind,\n\t\tNamespace: pvc.Namespace,\n\t\tName: pvc.Name,\n\t}\n\tdelete(updated.Annotations, KubeAnnBoundByController)\n\n\tif labels != nil {\n\t\tif updated.Labels == nil {\n\t\t\tupdated.Labels = make(map[string]string)\n\t\t}\n\n\t\tfor k, v := range labels {\n\t\t\tif _, ok := updated.Labels[k]; !ok {\n\t\t\t\tupdated.Labels[k] = v\n\t\t\t}\n\t\t}\n\t}\n\n\tupdatedBytes, err := json.Marshal(updated)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error marshaling updated PV\")\n\t}\n\n\tpatchBytes, err := jsonpatch.CreateMergePatch(origBytes, updatedBytes)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error creating json merge patch for PV\")\n\t}\n\n\tupdated, err = pvGetter.PersistentVolumes().Patch(ctx, pv.Name, types.MergePatchType, patchBytes, metav1.PatchOptions{})\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error patching PV\")\n\t}\n\n\treturn updated, nil\n}\n\n// SetPVReclaimPolicy sets the specified reclaim policy to a PV\nfunc SetPVReclaimPolicy(ctx context.Context, pvGetter corev1client.CoreV1Interface, pv *corev1api.PersistentVolume,\n\tpolicy corev1api.PersistentVolumeReclaimPolicy) (*corev1api.PersistentVolume, error) {\n\tif pv.Spec.PersistentVolumeReclaimPolicy == policy {\n\t\treturn nil, nil\n\t}\n\n\torigBytes, err := json.Marshal(pv)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error marshaling original PV\")\n\t}\n\n\tupdated := pv.DeepCopy()\n\tupdated.Spec.PersistentVolumeReclaimPolicy = policy\n\n\tupdatedBytes, err := json.Marshal(updated)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error marshaling updated PV\")\n\t}\n\n\tpatchBytes, err := jsonpatch.CreateMergePatch(origBytes, updatedBytes)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error creating json merge patch for PV\")\n\t}\n\n\tupdated, err = pvGetter.PersistentVolumes().Patch(ctx, pv.Name, types.MergePatchType, patchBytes, metav1.PatchOptions{})\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error patching PV\")\n\t}\n\n\treturn updated, nil\n}\n\n// WaitPVCConsumed waits for a PVC to be consumed by a pod so that the selected node is set by the pod scheduling; or does\n// nothing if the consuming doesn't affect the PV provision.\n// The latest PVC and the selected node will be returned.\nfunc WaitPVCConsumed(\n\tctx context.Context,\n\tpvcGetter corev1client.CoreV1Interface,\n\tpvc string, namespace string,\n\tstorageClient storagev1.StorageV1Interface,\n\ttimeout time.Duration,\n\tignoreConsume bool,\n) (string, *corev1api.PersistentVolumeClaim, error) {\n\tselectedNode := \"\"\n\tvar updated *corev1api.PersistentVolumeClaim\n\tvar storageClass *storagev1api.StorageClass\n\n\terr := wait.PollUntilContextTimeout(ctx, waitInternal, timeout, true, func(ctx context.Context) (bool, error) {\n\t\ttmpPVC, err := pvcGetter.PersistentVolumeClaims(namespace).Get(ctx, pvc, metav1.GetOptions{})\n\t\tif err != nil {\n\t\t\treturn false, errors.Wrapf(err, \"error to get pvc %s/%s\", namespace, pvc)\n\t\t}\n\n\t\tif !ignoreConsume {\n\t\t\tif tmpPVC.Spec.StorageClassName != nil && storageClass == nil {\n\t\t\t\tstorageClass, err = storageClient.StorageClasses().Get(ctx, *tmpPVC.Spec.StorageClassName, metav1.GetOptions{})\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn false, errors.Wrapf(err, \"error to get storage class %s\", *tmpPVC.Spec.StorageClassName)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tif storageClass != nil {\n\t\t\t\tif storageClass.VolumeBindingMode != nil && *storageClass.VolumeBindingMode == storagev1api.VolumeBindingWaitForFirstConsumer {\n\t\t\t\t\tselectedNode = tmpPVC.Annotations[KubeAnnSelectedNode]\n\t\t\t\t\tif selectedNode == \"\" {\n\t\t\t\t\t\treturn false, nil\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\tupdated = tmpPVC\n\n\t\treturn true, nil\n\t})\n\n\tif err != nil {\n\t\treturn \"\", nil, errors.Wrap(err, \"error to wait for PVC\")\n\t}\n\n\treturn selectedNode, updated, err\n}\n\n// WaitPVBound wait for binding of a PV specified by name and returns the bound PV object\nfunc WaitPVBound(ctx context.Context, pvGetter corev1client.CoreV1Interface, pvName string, pvcName string, pvcNamespace string, timeout time.Duration) (*corev1api.PersistentVolume, error) {\n\tvar updated *corev1api.PersistentVolume\n\terr := wait.PollUntilContextTimeout(ctx, waitInternal, timeout, true, func(ctx context.Context) (bool, error) {\n\t\ttmpPV, err := pvGetter.PersistentVolumes().Get(ctx, pvName, metav1.GetOptions{})\n\t\tif err != nil {\n\t\t\treturn false, errors.Wrapf(err, \"failed to get pv %s\", pvName)\n\t\t}\n\n\t\tif tmpPV.Spec.ClaimRef == nil {\n\t\t\treturn false, nil\n\t\t}\n\n\t\tif tmpPV.Status.Phase != corev1api.VolumeBound {\n\t\t\treturn false, nil\n\t\t}\n\n\t\tif tmpPV.Spec.ClaimRef.Name != pvcName {\n\t\t\treturn false, errors.Errorf(\"pv has been bound by unexpected pvc %s/%s\", tmpPV.Spec.ClaimRef.Namespace, tmpPV.Spec.ClaimRef.Name)\n\t\t}\n\n\t\tif tmpPV.Spec.ClaimRef.Namespace != pvcNamespace {\n\t\t\treturn false, errors.Errorf(\"pv has been bound by unexpected pvc %s/%s\", tmpPV.Spec.ClaimRef.Namespace, tmpPV.Spec.ClaimRef.Name)\n\t\t}\n\n\t\tupdated = tmpPV\n\n\t\treturn true, nil\n\t})\n\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error to wait for bound of PV\")\n\t} else {\n\t\treturn updated, nil\n\t}\n}\n\n// IsPVCBound returns true if the specified PVC has been bound\nfunc IsPVCBound(pvc *corev1api.PersistentVolumeClaim) bool {\n\treturn pvc.Spec.VolumeName != \"\"\n}\n\n// MakePodPVCAttachment returns the volume mounts and devices for a pod needed to attach a PVC\nfunc MakePodPVCAttachment(volumeName string, volumeMode *corev1api.PersistentVolumeMode, readOnly bool) ([]corev1api.VolumeMount, []corev1api.VolumeDevice, string) {\n\tvar volumeMounts []corev1api.VolumeMount\n\tvar volumeDevices []corev1api.VolumeDevice\n\tvolumePath := \"/\" + volumeName\n\n\tif volumeMode != nil && *volumeMode == corev1api.PersistentVolumeBlock {\n\t\tvolumeDevices = []corev1api.VolumeDevice{{\n\t\t\tName: volumeName,\n\t\t\tDevicePath: volumePath,\n\t\t}}\n\t} else {\n\t\tvolumeMounts = []corev1api.VolumeMount{{\n\t\t\tName: volumeName,\n\t\t\tMountPath: volumePath,\n\t\t\tReadOnly: readOnly,\n\t\t}}\n\t}\n\n\treturn volumeMounts, volumeDevices, volumePath\n}\n\n// GetPVForPVC returns the PersistentVolume backing a PVC\n// returns PV, error.\n// PV will be nil on error\nfunc GetPVForPVC(\n\tpvc *corev1api.PersistentVolumeClaim,\n\tcrClient crclient.Client,\n) (*corev1api.PersistentVolume, error) {\n\tif pvc.Spec.VolumeName == \"\" {\n\t\treturn nil, errors.Errorf(\"PVC %s/%s has no volume backing this claim\", pvc.Namespace, pvc.Name)\n\t}\n\tif pvc.Status.Phase != corev1api.ClaimBound {\n\t\treturn nil, errors.Errorf(\"PVC %s/%s is in phase %v and is not bound to a volume\",\n\t\t\tpvc.Namespace, pvc.Name, pvc.Status.Phase)\n\t}\n\n\tpv := &corev1api.PersistentVolume{}\n\terr := crClient.Get(\n\t\tcontext.TODO(),\n\t\tcrclient.ObjectKey{Name: pvc.Spec.VolumeName},\n\t\tpv,\n\t)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"failed to get PV %s for PVC %s/%s\",\n\t\t\tpvc.Spec.VolumeName, pvc.Namespace, pvc.Name)\n\t}\n\treturn pv, nil\n}\n\nfunc GetPVCForPodVolume(vol *corev1api.Volume, pod *corev1api.Pod, crClient crclient.Client) (*corev1api.PersistentVolumeClaim, error) {\n\tif vol.PersistentVolumeClaim == nil {\n\t\treturn nil, errors.Errorf(\"volume %s/%s has no PVC associated with it\", pod.Namespace, vol.Name)\n\t}\n\tpvc := &corev1api.PersistentVolumeClaim{}\n\terr := crClient.Get(\n\t\tcontext.TODO(),\n\t\tcrclient.ObjectKey{Name: vol.PersistentVolumeClaim.ClaimName, Namespace: pod.Namespace},\n\t\tpvc,\n\t)\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"failed to get PVC %s for Volume %s/%s\",\n\t\t\tvol.PersistentVolumeClaim.ClaimName, pod.Namespace, vol.Name)\n\t}\n\n\treturn pvc, nil\n}\n\nfunc DiagnosePVC(pvc *corev1api.PersistentVolumeClaim, events *corev1api.EventList) string {\n\tdiag := fmt.Sprintf(\"PVC %s/%s, phase %s, binding to %s\\n\", pvc.Namespace, pvc.Name, pvc.Status.Phase, pvc.Spec.VolumeName)\n\n\tif events != nil {\n\t\tfor _, e := range events.Items {\n\t\t\tif e.InvolvedObject.UID == pvc.UID && e.Type == corev1api.EventTypeWarning {\n\t\t\t\tdiag += fmt.Sprintf(\"PVC event reason %s, message %s\\n\", e.Reason, e.Message)\n\t\t\t}\n\t\t}\n\t}\n\n\treturn diag\n}\n\nfunc DiagnosePV(pv *corev1api.PersistentVolume) string {\n\tdiag := fmt.Sprintf(\"PV %s, phase %s, reason %s, message %s\\n\", pv.Name, pv.Status.Phase, pv.Status.Reason, pv.Status.Message)\n\treturn diag\n}\n\nfunc GetPVCAttachingNodeOS(pvc *corev1api.PersistentVolumeClaim, nodeClient corev1client.CoreV1Interface,\n\tstorageClient storagev1.StorageV1Interface, log logrus.FieldLogger) string {\n\tvar nodeOS string\n\tvar scFsType string\n\n\tif pvc.Spec.VolumeMode != nil && *pvc.Spec.VolumeMode == corev1api.PersistentVolumeBlock {\n\t\tlog.Infof(\"Use linux node for block mode PVC %s/%s\", pvc.Namespace, pvc.Name)\n\t\treturn NodeOSLinux\n\t}\n\n\tif pvc.Spec.VolumeName == \"\" {\n\t\tlog.Warnf(\"PVC %s/%s is not bound to a PV\", pvc.Namespace, pvc.Name)\n\t}\n\n\tif pvc.Spec.StorageClassName == nil {\n\t\tlog.Warnf(\"PVC %s/%s is not with storage class\", pvc.Namespace, pvc.Name)\n\t}\n\n\tnodeName := \"\"\n\tif pvc.Spec.VolumeName != \"\" {\n\t\tif n, err := GetPVAttachedNode(context.Background(), pvc.Spec.VolumeName, storageClient); err != nil {\n\t\t\tlog.WithError(err).Warnf(\"Failed to get attached node for PVC %s/%s\", pvc.Namespace, pvc.Name)\n\t\t} else {\n\t\t\tnodeName = n\n\t\t}\n\t}\n\n\tif nodeName == \"\" {\n\t\tif value := pvc.Annotations[KubeAnnSelectedNode]; value != \"\" {\n\t\t\tnodeName = value\n\t\t}\n\t}\n\n\tif nodeName != \"\" {\n\t\tif os, err := GetNodeOS(context.Background(), nodeName, nodeClient); err != nil {\n\t\t\tlog.WithError(err).Warnf(\"Failed to get os from node %s for PVC %s/%s\", nodeName, pvc.Namespace, pvc.Name)\n\t\t} else {\n\t\t\tnodeOS = os\n\t\t}\n\t}\n\n\tif pvc.Spec.StorageClassName != nil {\n\t\tif sc, err := storageClient.StorageClasses().Get(context.Background(), *pvc.Spec.StorageClassName, metav1.GetOptions{}); err != nil {\n\t\t\tlog.WithError(err).Warnf(\"Failed to get storage class %s for PVC %s/%s\", *pvc.Spec.StorageClassName, pvc.Namespace, pvc.Name)\n\t\t} else if sc.Parameters != nil {\n\t\t\tscFsType = strings.ToLower(sc.Parameters[\"csi.storage.k8s.io/fstype\"])\n\t\t}\n\t}\n\n\tif nodeOS != \"\" {\n\t\tlog.Infof(\"Deduced node os %s from selected/attached node for PVC %s/%s (fsType %s)\", nodeOS, pvc.Namespace, pvc.Name, scFsType)\n\t\treturn nodeOS\n\t}\n\n\tif scFsType == \"ntfs\" {\n\t\tlog.Infof(\"Deduced Windows node os from fsType %s for PVC %s/%s\", scFsType, pvc.Namespace, pvc.Name)\n\t\treturn NodeOSWindows\n\t}\n\n\tif scFsType != \"\" {\n\t\tlog.Infof(\"Deduced linux node os from fsType %s for PVC %s/%s\", scFsType, pvc.Namespace, pvc.Name)\n\t\treturn NodeOSLinux\n\t}\n\n\tlog.Warnf(\"Cannot deduce node os for PVC %s/%s, default to linux\", pvc.Namespace, pvc.Name)\n\treturn NodeOSLinux\n}\n\nfunc GetPVAttachedNode(ctx context.Context, pv string, storageClient storagev1.StorageV1Interface) (string, error) {\n\tvaList, err := storageClient.VolumeAttachments().List(ctx, metav1.ListOptions{})\n\tif err != nil {\n\t\treturn \"\", errors.Wrapf(err, \"error listing volumeattachment\")\n\t}\n\n\tfor _, va := range vaList.Items {\n\t\tif va.Spec.Source.PersistentVolumeName != nil && *va.Spec.Source.PersistentVolumeName == pv {\n\t\t\treturn va.Spec.NodeName, nil\n\t\t}\n\t}\n\n\treturn \"\", nil\n}\n\nfunc GetPVAttachedNodes(ctx context.Context, pv string, storageClient storagev1.StorageV1Interface) ([]string, error) {\n\tvaList, err := storageClient.VolumeAttachments().List(ctx, metav1.ListOptions{})\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error listing volumeattachment\")\n\t}\n\n\tnodes := []string{}\n\tfor _, va := range vaList.Items {\n\t\tif va.Spec.Source.PersistentVolumeName != nil && *va.Spec.Source.PersistentVolumeName == pv {\n\t\t\tnodes = append(nodes, va.Spec.NodeName)\n\t\t}\n\t}\n\n\treturn nodes, nil\n}\n\nfunc GetVolumeTopology(ctx context.Context, volumeClient corev1client.CoreV1Interface, storageClient storagev1.StorageV1Interface, pvName string, scName string) (*corev1api.NodeSelector, error) {\n\tif pvName == \"\" || scName == \"\" {\n\t\treturn nil, errors.Errorf(\"invalid parameter, pv %s, sc %s\", pvName, scName)\n\t}\n\n\tsc, err := storageClient.StorageClasses().Get(ctx, scName, metav1.GetOptions{})\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error getting storage class %s\", scName)\n\t}\n\n\tif sc.VolumeBindingMode == nil || *sc.VolumeBindingMode != storagev1api.VolumeBindingWaitForFirstConsumer {\n\t\treturn nil, nil\n\t}\n\n\tpv, err := volumeClient.PersistentVolumes().Get(ctx, pvName, metav1.GetOptions{})\n\tif err != nil {\n\t\treturn nil, errors.Wrapf(err, \"error getting PV %s\", pvName)\n\t}\n\n\tif pv.Spec.NodeAffinity == nil {\n\t\treturn nil, nil\n\t}\n\n\treturn pv.Spec.NodeAffinity.Required, nil\n}\n" }, { "path": "pkg/util/kube/pvc_pv_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/client-go/kubernetes\"\n\t\"k8s.io/client-go/kubernetes/fake\"\n\n\tcorev1api \"k8s.io/api/core/v1\"\n\tstoragev1api \"k8s.io/api/storage/v1\"\n\n\tclientTesting \"k8s.io/client-go/testing\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\ntype reactor struct {\n\tverb string\n\tresource string\n\treactorFunc clientTesting.ReactionFunc\n}\n\nfunc TestWaitPVCBound(t *testing.T) {\n\tpvcObject := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-namespace\",\n\t\t\tName: \"fake-pvc\",\n\t\t},\n\t}\n\n\tpvcObjectBound := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-namespace\",\n\t\t\tName: \"fake-pvc\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tVolumeName: \"fake-pv\",\n\t\t},\n\t}\n\n\tpvObj := &corev1api.PersistentVolume{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-pv\",\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tpvcName string\n\t\tpvcNamespace string\n\t\tkubeClientObj []runtime.Object\n\t\tkubeReactors []reactor\n\t\texpected *corev1api.PersistentVolume\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"wait pvc error\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\terr: \"error to wait for rediness of PVC: error to get pvc fake-namespace/fake-pvc: persistentvolumeclaims \\\"fake-pvc\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"wait pvc timeout\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpvcObject,\n\t\t\t},\n\t\t\terr: \"error to wait for rediness of PVC: context deadline exceeded\",\n\t\t},\n\t\t{\n\t\t\tname: \"get pv fail\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpvcObjectBound,\n\t\t\t},\n\t\t\terr: \"error to get PV: persistentvolumes \\\"fake-pv\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"success\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpvcObjectBound,\n\t\t\t\tpvObj,\n\t\t\t},\n\t\t\texpected: pvObj,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tfor _, reactor := range test.kubeReactors {\n\t\t\t\tfakeKubeClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\tvar kubeClient kubernetes.Interface = fakeKubeClient\n\n\t\t\tpv, err := WaitPVCBound(t.Context(), kubeClient.CoreV1(), kubeClient.CoreV1(), test.pvcName, test.pvcNamespace, time.Millisecond)\n\n\t\t\tif err != nil {\n\t\t\t\trequire.EqualError(t, err, test.err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tassert.Equal(t, test.expected, pv)\n\t\t})\n\t}\n}\n\nfunc TestWaitPVCConsumed(t *testing.T) {\n\tstorageClass := \"fake-storage-class\"\n\tbindModeImmediate := storagev1api.VolumeBindingImmediate\n\tbindModeWait := storagev1api.VolumeBindingWaitForFirstConsumer\n\n\tpvcObject := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-namespace\",\n\t\t\tName: \"fake-pvc-1\",\n\t\t},\n\t}\n\n\tpvcObjectWithSC := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-namespace\",\n\t\t\tName: \"fake-pvc-2\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tStorageClassName: &storageClass,\n\t\t},\n\t}\n\n\tscObjWithoutBindMode := &storagev1api.StorageClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-storage-class\",\n\t\t},\n\t}\n\n\tscObjWaitBind := &storagev1api.StorageClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-storage-class\",\n\t\t},\n\t\tVolumeBindingMode: &bindModeWait,\n\t}\n\n\tscObjWithImmidateBinding := &storagev1api.StorageClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-storage-class\",\n\t\t},\n\t\tVolumeBindingMode: &bindModeImmediate,\n\t}\n\n\tpvcObjectWithSCAndAnno := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-namespace\",\n\t\t\tName: \"fake-pvc-3\",\n\t\t\tAnnotations: map[string]string{\"volume.kubernetes.io/selected-node\": \"fake-node-1\"},\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tStorageClassName: &storageClass,\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tpvcName string\n\t\tpvcNamespace string\n\t\tkubeClientObj []runtime.Object\n\t\tkubeReactors []reactor\n\t\texpectedPVC *corev1api.PersistentVolumeClaim\n\t\tselectedNode string\n\t\tignoreWaitForFirstConsumer bool\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"get pvc error\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\terr: \"error to wait for PVC: error to get pvc fake-namespace/fake-pvc: persistentvolumeclaims \\\"fake-pvc\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"success when no sc\",\n\t\t\tpvcName: \"fake-pvc-1\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpvcObject,\n\t\t\t},\n\t\t\texpectedPVC: pvcObject,\n\t\t},\n\t\t{\n\t\t\tname: \"success when ignore wait for first consumer\",\n\t\t\tpvcName: \"fake-pvc-2\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\tignoreWaitForFirstConsumer: true,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpvcObjectWithSC,\n\t\t\t},\n\t\t\texpectedPVC: pvcObjectWithSC,\n\t\t},\n\t\t{\n\t\t\tname: \"get sc fail\",\n\t\t\tpvcName: \"fake-pvc-2\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpvcObjectWithSC,\n\t\t\t},\n\t\t\terr: \"error to wait for PVC: error to get storage class fake-storage-class: storageclasses.storage.k8s.io \\\"fake-storage-class\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"success on sc without binding mode\",\n\t\t\tpvcName: \"fake-pvc-2\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpvcObjectWithSC,\n\t\t\t\tscObjWithoutBindMode,\n\t\t\t},\n\t\t\texpectedPVC: pvcObjectWithSC,\n\t\t},\n\t\t{\n\t\t\tname: \"success on sc without immediate binding mode\",\n\t\t\tpvcName: \"fake-pvc-2\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpvcObjectWithSC,\n\t\t\t\tscObjWithImmidateBinding,\n\t\t\t},\n\t\t\texpectedPVC: pvcObjectWithSC,\n\t\t},\n\t\t{\n\t\t\tname: \"pvc annotation miss\",\n\t\t\tpvcName: \"fake-pvc-2\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpvcObjectWithSC,\n\t\t\t\tscObjWaitBind,\n\t\t\t},\n\t\t\terr: \"error to wait for PVC: context deadline exceeded\",\n\t\t},\n\t\t{\n\t\t\tname: \"success on sc without wait binding mode\",\n\t\t\tpvcName: \"fake-pvc-3\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpvcObjectWithSCAndAnno,\n\t\t\t\tscObjWaitBind,\n\t\t\t},\n\t\t\texpectedPVC: pvcObjectWithSCAndAnno,\n\t\t\tselectedNode: \"fake-node-1\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tfor _, reactor := range test.kubeReactors {\n\t\t\t\tfakeKubeClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\tvar kubeClient kubernetes.Interface = fakeKubeClient\n\n\t\t\tselectedNode, pvc, err := WaitPVCConsumed(t.Context(), kubeClient.CoreV1(), test.pvcName, test.pvcNamespace, kubeClient.StorageV1(), time.Millisecond, test.ignoreWaitForFirstConsumer)\n\n\t\t\tif err != nil {\n\t\t\t\trequire.EqualError(t, err, test.err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tassert.Equal(t, test.expectedPVC, pvc)\n\t\t\tassert.Equal(t, test.selectedNode, selectedNode)\n\t\t})\n\t}\n}\n\nfunc TestDeletePVCIfAny(t *testing.T) {\n\tpvObject := &corev1api.PersistentVolume{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-pv\",\n\t\t\tAnnotations: map[string]string{\n\t\t\t\tKubeAnnBoundByController: \"true\",\n\t\t\t},\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\tClaimRef: &corev1api.ObjectReference{\n\t\t\t\tKind: \"fake-kind\",\n\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\tName: \"fake-pvc\",\n\t\t\t},\n\t\t},\n\t}\n\n\tpvcObject := &corev1api.PersistentVolumeClaim{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"fake-kind-1\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-namespace\",\n\t\t\tName: \"fake-pvc\",\n\t\t\tAnnotations: map[string]string{\n\t\t\t\tKubeAnnBindCompleted: \"true\",\n\t\t\t\tKubeAnnBoundByController: \"true\",\n\t\t\t},\n\t\t},\n\t}\n\n\tpvcWithVolume := pvcObject.DeepCopy()\n\tpvcWithVolume.Spec.VolumeName = \"fake-pv\"\n\n\ttests := []struct {\n\t\tname string\n\t\tpvcName string\n\t\tpvcNamespace string\n\t\tpvName string\n\t\tkubeClientObj []runtime.Object\n\t\tkubeReactors []reactor\n\t\tlogMessage string\n\t\tlogLevel string\n\t\tlogError string\n\t\tensureTimeout time.Duration\n\t}{\n\t\t{\n\t\t\tname: \"pvc not found\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\tlogMessage: \"Abort deleting PV and PVC, for related PVC doesn't exist, fake-namespace/fake-pvc\",\n\t\t\tlogLevel: \"level=debug\",\n\t\t},\n\t\t{\n\t\t\tname: \"failed to get pvc\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"get\",\n\t\t\t\t\tresource: \"persistentvolumeclaims\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-get-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tlogMessage: \"failed to get pvc fake-namespace/fake-pvc with err fake-get-error\",\n\t\t\tlogLevel: \"level=warning\",\n\t\t},\n\t\t{\n\t\t\tname: \"pvc has no volume name\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpvcObject,\n\t\t\t\tpvObject,\n\t\t\t},\n\t\t\tlogMessage: \"failed to delete PV, for related PVC fake-namespace/fake-pvc has no bind volume name\",\n\t\t\tlogLevel: \"level=warning\",\n\t\t},\n\t\t{\n\t\t\tname: \"failed to delete pvc\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"persistentvolumeclaims\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-delete-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpvcWithVolume,\n\t\t\t\tpvObject,\n\t\t\t},\n\t\t\tlogMessage: \"failed to delete pvc fake-namespace/fake-pvc with err error to delete pvc fake-pvc: fake-delete-error\",\n\t\t\tlogLevel: \"level=warning\",\n\t\t},\n\t\t{\n\t\t\tname: \"failed to get pv\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"get\",\n\t\t\t\t\tresource: \"persistentvolumes\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-get-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpvcWithVolume,\n\t\t\t\tpvObject,\n\t\t\t},\n\t\t\tlogMessage: \"failed to delete PV fake-pv with err fake-get-error\",\n\t\t\tlogLevel: \"level=warning\",\n\t\t},\n\t\t{\n\t\t\tname: \"set reclaim policy fail\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpvcWithVolume,\n\t\t\t\tpvObject,\n\t\t\t},\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"patch\",\n\t\t\t\t\tresource: \"persistentvolumes\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, pvObject, errors.New(\"fake-patch-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tlogMessage: \"failed to set reclaim policy of PV fake-pv to delete with err error patching PV: fake-patch-error\",\n\t\t\tlogLevel: \"level=warning\",\n\t\t\tlogError: \"fake-patch-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"delete pv pvc success\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpvcWithVolume,\n\t\t\t\tpvObject,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"delete pv pvc success but wait fail\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpvcWithVolume,\n\t\t\t\tpvObject,\n\t\t\t},\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"persistentvolumeclaims\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, pvcWithVolume, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tensureTimeout: time.Second,\n\t\t\tlogMessage: \"failed to delete pvc fake-namespace/fake-pvc with err timeout to assure pvc fake-pvc is deleted, finalizers in pvc []\",\n\t\t\tlogLevel: \"level=warning\",\n\t\t},\n\t\t{\n\t\t\tname: \"delete pv pvc success, wait won't succeed but ensureTimeout is 0\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tpvcNamespace: \"fake-namespace\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tpvcWithVolume,\n\t\t\t\tpvObject,\n\t\t\t},\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"persistentvolumeclaims\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, pvcWithVolume, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tfor _, reactor := range test.kubeReactors {\n\t\t\t\tfakeKubeClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\tvar kubeClient kubernetes.Interface = fakeKubeClient\n\n\t\t\tlogMessage := \"\"\n\t\t\tDeletePVAndPVCIfAny(t.Context(), kubeClient.CoreV1(), test.pvcName, test.pvcNamespace, test.ensureTimeout, velerotest.NewSingleLogger(&logMessage))\n\n\t\t\tif len(test.logMessage) > 0 {\n\t\t\t\tassert.Contains(t, logMessage, test.logMessage)\n\t\t\t}\n\n\t\t\tif len(test.logLevel) > 0 {\n\t\t\t\tassert.Contains(t, logMessage, test.logLevel)\n\t\t\t}\n\n\t\t\tif len(test.logError) > 0 {\n\t\t\t\tassert.Contains(t, logMessage, test.logError)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestDeletePVIfAny(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvName string\n\t\tkubeClientObj []runtime.Object\n\t\tkubeReactors []reactor\n\t\tlogMessage string\n\t\tlogLevel string\n\t\tlogError string\n\t}{\n\t\t{\n\t\t\tname: \"get fail\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tlogMessage: \"Abort deleting PV, it doesn't exist, fake-pv\",\n\t\t\tlogLevel: \"level=debug\",\n\t\t},\n\t\t{\n\t\t\tname: \"delete fail\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tkubeReactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"persistentvolumes\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-delete-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tlogMessage: \"Failed to delete PV fake-pv\",\n\t\t\tlogLevel: \"level=error\",\n\t\t\tlogError: \"error=fake-delete-error\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tfor _, reactor := range test.kubeReactors {\n\t\t\t\tfakeKubeClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\tvar kubeClient kubernetes.Interface = fakeKubeClient\n\n\t\t\tlogMessage := \"\"\n\t\t\tDeletePVIfAny(t.Context(), kubeClient.CoreV1(), test.pvName, velerotest.NewSingleLogger(&logMessage))\n\n\t\t\tif len(test.logMessage) > 0 {\n\t\t\t\tassert.Contains(t, logMessage, test.logMessage)\n\t\t\t}\n\n\t\t\tif len(test.logLevel) > 0 {\n\t\t\t\tassert.Contains(t, logMessage, test.logLevel)\n\t\t\t}\n\n\t\t\tif len(test.logError) > 0 {\n\t\t\t\tassert.Contains(t, logMessage, test.logError)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestEnsureDeletePVC(t *testing.T) {\n\tpvcObject := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"fake-pvc\",\n\t\t},\n\t}\n\n\tpvcObjectWithFinalizer := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"fake-pvc\",\n\t\t\tFinalizers: []string{\"fake-finalizer-1\", \"fake-finalizer-2\"},\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tclientObj []runtime.Object\n\t\tpvcName string\n\t\tnamespace string\n\t\treactors []reactor\n\t\ttimeout time.Duration\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"delete fail\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\terr: \"error to delete pvc fake-pvc: persistentvolumeclaims \\\"fake-pvc\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"0 timeout\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tclientObj: []runtime.Object{pvcObject},\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"persistentvolumeclaims\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, pvcObject, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"wait fail\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tclientObj: []runtime.Object{pvcObject},\n\t\t\ttimeout: time.Millisecond,\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"get\",\n\t\t\t\t\tresource: \"persistentvolumeclaims\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-get-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to ensure pvc deleted for fake-pvc: error to get pvc fake-pvc: fake-get-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"wait timeout\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tclientObj: []runtime.Object{pvcObjectWithFinalizer},\n\t\t\ttimeout: time.Millisecond,\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"persistentvolumeclaims\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, pvcObject, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"timeout to assure pvc fake-pvc is deleted, finalizers in pvc [fake-finalizer-1 fake-finalizer-2]\",\n\t\t},\n\t\t{\n\t\t\tname: \"wait timeout, no finalizer\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tnamespace: \"fake-ns\",\n\t\t\tclientObj: []runtime.Object{pvcObject},\n\t\t\ttimeout: time.Millisecond,\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"delete\",\n\t\t\t\t\tresource: \"persistentvolumeclaims\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, pvcObject, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"timeout to assure pvc fake-pvc is deleted, finalizers in pvc []\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.clientObj...)\n\n\t\t\tfor _, reactor := range test.reactors {\n\t\t\t\tfakeKubeClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\tvar kubeClient kubernetes.Interface = fakeKubeClient\n\n\t\t\terr := EnsureDeletePVC(t.Context(), kubeClient.CoreV1(), test.pvcName, test.namespace, test.timeout)\n\t\t\tif err != nil {\n\t\t\t\tassert.EqualError(t, err, test.err)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestEnsureDeletePV(t *testing.T) {\n\tpvObject := &corev1api.PersistentVolume{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-pv\",\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tclientObj []runtime.Object\n\t\tpvName string\n\t\treactors []reactor\n\t\ttimeout time.Duration\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"get fail\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\terr: \"error to get pv fake-pv: persistentvolumes \\\"fake-pv\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"0 timeout\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tclientObj: []runtime.Object{pvObject},\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"get\",\n\t\t\t\t\tresource: \"persistentvolumes\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, pvObject, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"wait fail\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tclientObj: []runtime.Object{pvObject},\n\t\t\ttimeout: time.Millisecond,\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"get\",\n\t\t\t\t\tresource: \"persistentvolumes\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-get-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to ensure pv is deleted for fake-pv: error to get pv fake-pv: fake-get-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"wait timeout\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tclientObj: []runtime.Object{pvObject},\n\t\t\ttimeout: time.Millisecond,\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"get\",\n\t\t\t\t\tresource: \"persistentvolumes\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, pvObject, nil\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"timeout to assure pv fake-pv is deleted\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.clientObj...)\n\n\t\t\tfor _, reactor := range test.reactors {\n\t\t\t\tfakeKubeClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\tvar kubeClient kubernetes.Interface = fakeKubeClient\n\n\t\t\terr := EnsurePVDeleted(t.Context(), kubeClient.CoreV1(), test.pvName, test.timeout)\n\t\t\tif err != nil {\n\t\t\t\tassert.EqualError(t, err, test.err)\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestRebindPVC(t *testing.T) {\n\tpvcObject := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns\",\n\t\t\tName: \"fake-pvc\",\n\t\t\tAnnotations: map[string]string{\n\t\t\t\tKubeAnnBindCompleted: \"true\",\n\t\t\t\tKubeAnnBoundByController: \"true\",\n\t\t\t},\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tclientObj []runtime.Object\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\tpv string\n\t\treactors []reactor\n\t\tresult *corev1api.PersistentVolumeClaim\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"path fail\",\n\t\t\tpvc: pvcObject,\n\t\t\tpv: \"fake-pv\",\n\t\t\tclientObj: []runtime.Object{pvcObject},\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"patch\",\n\t\t\t\t\tresource: \"persistentvolumeclaims\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-patch-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error patching PVC: fake-patch-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tpvc: pvcObject,\n\t\t\tpv: \"fake-pv\",\n\t\t\tclientObj: []runtime.Object{pvcObject},\n\t\t\tresult: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\tName: \"fake-pvc\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\tVolumeName: \"fake-pv\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.clientObj...)\n\n\t\t\tfor _, reactor := range test.reactors {\n\t\t\t\tfakeKubeClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\tvar kubeClient kubernetes.Interface = fakeKubeClient\n\n\t\t\tresult, err := RebindPVC(t.Context(), kubeClient.CoreV1(), test.pvc, test.pv)\n\t\t\tif err != nil {\n\t\t\t\trequire.EqualError(t, err, test.err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tassert.Equal(t, test.result, result)\n\t\t})\n\t}\n}\n\nfunc TestResetPVBinding(t *testing.T) {\n\tpvObject := &corev1api.PersistentVolume{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-pv\",\n\t\t\tAnnotations: map[string]string{\n\t\t\t\tKubeAnnBoundByController: \"true\",\n\t\t\t},\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\tClaimRef: &corev1api.ObjectReference{\n\t\t\t\tKind: \"fake-kind\",\n\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\tName: \"fake-pvc\",\n\t\t\t},\n\t\t},\n\t}\n\n\tpvcObject := &corev1api.PersistentVolumeClaim{\n\t\tTypeMeta: metav1.TypeMeta{\n\t\t\tKind: \"fake-kind-1\",\n\t\t},\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-ns-1\",\n\t\t\tName: \"fake-pvc-1\",\n\t\t\tAnnotations: map[string]string{\n\t\t\t\tKubeAnnBindCompleted: \"true\",\n\t\t\t\tKubeAnnBoundByController: \"true\",\n\t\t\t},\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tclientObj []runtime.Object\n\t\tpv *corev1api.PersistentVolume\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\tlabels map[string]string\n\t\treactors []reactor\n\t\tresult *corev1api.PersistentVolume\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"path fail\",\n\t\t\tpv: pvObject,\n\t\t\tpvc: pvcObject,\n\t\t\tclientObj: []runtime.Object{pvObject},\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"patch\",\n\t\t\t\t\tresource: \"persistentvolumes\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-patch-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error patching PV: fake-patch-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tpv: pvObject,\n\t\t\tpvc: pvcObject,\n\t\t\tlabels: map[string]string{\n\t\t\t\t\"fake-label-1\": \"fake-value-1\",\n\t\t\t\t\"fake-label-2\": \"fake-value-2\",\n\t\t\t},\n\t\t\tclientObj: []runtime.Object{pvObject},\n\t\t\tresult: &corev1api.PersistentVolume{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-pv\",\n\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\"fake-label-1\": \"fake-value-1\",\n\t\t\t\t\t\t\"fake-label-2\": \"fake-value-2\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tClaimRef: &corev1api.ObjectReference{\n\t\t\t\t\t\tKind: \"fake-kind-1\",\n\t\t\t\t\t\tNamespace: \"fake-ns-1\",\n\t\t\t\t\t\tName: \"fake-pvc-1\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.clientObj...)\n\n\t\t\tfor _, reactor := range test.reactors {\n\t\t\t\tfakeKubeClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\tvar kubeClient kubernetes.Interface = fakeKubeClient\n\n\t\t\tresult, err := ResetPVBinding(t.Context(), kubeClient.CoreV1(), test.pv, test.labels, test.pvc)\n\t\t\tif err != nil {\n\t\t\t\trequire.EqualError(t, err, test.err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tassert.Equal(t, test.result, result)\n\t\t})\n\t}\n}\n\nfunc TestSetPVReclaimPolicy(t *testing.T) {\n\tpvObject := &corev1api.PersistentVolume{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-pv\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\tPersistentVolumeReclaimPolicy: corev1api.PersistentVolumeReclaimRetain,\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tclientObj []runtime.Object\n\t\tpv *corev1api.PersistentVolume\n\t\tpolicy corev1api.PersistentVolumeReclaimPolicy\n\t\treactors []reactor\n\t\tresult *corev1api.PersistentVolume\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"policy not changed\",\n\t\t\tpv: pvObject,\n\t\t\tpolicy: corev1api.PersistentVolumeReclaimRetain,\n\t\t},\n\t\t{\n\t\t\tname: \"path fail\",\n\t\t\tpv: pvObject,\n\t\t\tpolicy: corev1api.PersistentVolumeReclaimDelete,\n\t\t\tclientObj: []runtime.Object{pvObject},\n\t\t\treactors: []reactor{\n\t\t\t\t{\n\t\t\t\t\tverb: \"patch\",\n\t\t\t\t\tresource: \"persistentvolumes\",\n\t\t\t\t\treactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {\n\t\t\t\t\t\treturn true, nil, errors.New(\"fake-patch-error\")\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error patching PV: fake-patch-error\",\n\t\t},\n\t\t{\n\t\t\tname: \"succeed\",\n\t\t\tpv: pvObject,\n\t\t\tpolicy: corev1api.PersistentVolumeReclaimDelete,\n\t\t\tclientObj: []runtime.Object{pvObject},\n\t\t\tresult: &corev1api.PersistentVolume{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-pv\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tPersistentVolumeReclaimPolicy: corev1api.PersistentVolumeReclaimDelete,\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.clientObj...)\n\n\t\t\tfor _, reactor := range test.reactors {\n\t\t\t\tfakeKubeClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\tvar kubeClient kubernetes.Interface = fakeKubeClient\n\n\t\t\tresult, err := SetPVReclaimPolicy(t.Context(), kubeClient.CoreV1(), test.pv, test.policy)\n\t\t\tif err != nil {\n\t\t\t\trequire.EqualError(t, err, test.err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tassert.Equal(t, test.result, result)\n\t\t})\n\t}\n}\n\nfunc TestWaitPVBound(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvName string\n\t\tpvcName string\n\t\tpvcNamespace string\n\t\tkubeClientObj []runtime.Object\n\t\tkubeReactors []reactor\n\t\texpectedPV *corev1api.PersistentVolume\n\t\terr string\n\t}{\n\t\t{\n\t\t\tname: \"get pv error\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\terr: \"error to wait for bound of PV: failed to get pv fake-pv: persistentvolumes \\\"fake-pv\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"pvc claimRef miss\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&corev1api.PersistentVolume{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"fake-pv\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to wait for bound of PV: context deadline exceeded\",\n\t\t},\n\t\t{\n\t\t\tname: \"pvc status not bound\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&corev1api.PersistentVolume{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"fake-pv\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to wait for bound of PV: context deadline exceeded\",\n\t\t},\n\t\t{\n\t\t\tname: \"pvc claimRef pvc name mismatch\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tpvcNamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&corev1api.PersistentVolume{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"fake-pv\",\n\t\t\t\t\t},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\tClaimRef: &corev1api.ObjectReference{\n\t\t\t\t\t\t\tKind: \"fake-kind\",\n\t\t\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\t\t\tName: \"fake-pvc-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tStatus: corev1api.PersistentVolumeStatus{\n\t\t\t\t\t\tPhase: \"Bound\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to wait for bound of PV: pv has been bound by unexpected pvc fake-ns/fake-pvc-1\",\n\t\t},\n\t\t{\n\t\t\tname: \"pvc claimRef pvc namespace mismatch\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tpvcNamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&corev1api.PersistentVolume{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"fake-pv\",\n\t\t\t\t\t},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\tClaimRef: &corev1api.ObjectReference{\n\t\t\t\t\t\t\tKind: \"fake-kind\",\n\t\t\t\t\t\t\tNamespace: \"fake-ns-1\",\n\t\t\t\t\t\t\tName: \"fake-pvc\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tStatus: corev1api.PersistentVolumeStatus{\n\t\t\t\t\t\tPhase: \"Bound\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\terr: \"error to wait for bound of PV: pv has been bound by unexpected pvc fake-ns-1/fake-pvc\",\n\t\t},\n\t\t{\n\t\t\tname: \"success\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tpvcName: \"fake-pvc\",\n\t\t\tpvcNamespace: \"fake-ns\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\t&corev1api.PersistentVolume{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\tName: \"fake-pv\",\n\t\t\t\t\t},\n\t\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\t\tClaimRef: &corev1api.ObjectReference{\n\t\t\t\t\t\t\tKind: \"fake-kind\",\n\t\t\t\t\t\t\tName: \"fake-pvc\",\n\t\t\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\tStatus: corev1api.PersistentVolumeStatus{\n\t\t\t\t\t\tPhase: \"Bound\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedPV: &corev1api.PersistentVolume{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-pv\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\t\t\tClaimRef: &corev1api.ObjectReference{\n\t\t\t\t\t\tKind: \"fake-kind\",\n\t\t\t\t\t\tName: \"fake-pvc\",\n\t\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PersistentVolumeStatus{\n\t\t\t\t\tPhase: \"Bound\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tfor _, reactor := range test.kubeReactors {\n\t\t\t\tfakeKubeClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)\n\t\t\t}\n\n\t\t\tvar kubeClient kubernetes.Interface = fakeKubeClient\n\n\t\t\tpv, err := WaitPVBound(t.Context(), kubeClient.CoreV1(), test.pvName, test.pvcName, test.pvcNamespace, time.Millisecond)\n\n\t\t\tif err != nil {\n\t\t\t\trequire.EqualError(t, err, test.err)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\n\t\t\tassert.Equal(t, test.expectedPV, pv)\n\t\t})\n\t}\n}\n\nfunc TestIsPVCBound(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\texpect bool\n\t}{\n\t\t{\n\t\t\tname: \"expect bound\",\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\tName: \"fake-pvc\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\tVolumeName: \"fake-volume\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpect: true,\n\t\t},\n\t\t{\n\t\t\tname: \"expect not bound\",\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\tName: \"fake-pvc\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpect: false,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tresult := IsPVCBound(test.pvc)\n\n\t\t\tassert.Equal(t, test.expect, result)\n\t\t})\n\t}\n}\n\nvar (\n\tcsiStorageClass = \"csi-hostpath-sc\"\n)\n\nfunc TestGetPVForPVC(t *testing.T) {\n\tboundPVC := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-csi-pvc\",\n\t\t\tNamespace: \"default\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tAccessModes: []corev1api.PersistentVolumeAccessMode{corev1api.ReadWriteOnce},\n\t\t\tResources: corev1api.VolumeResourceRequirements{\n\t\t\t\tRequests: corev1api.ResourceList{},\n\t\t\t},\n\t\t\tStorageClassName: &csiStorageClass,\n\t\t\tVolumeName: \"test-csi-7d28e566-ade7-4ed6-9e15-2e44d2fbcc08\",\n\t\t},\n\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\tPhase: corev1api.ClaimBound,\n\t\t\tAccessModes: []corev1api.PersistentVolumeAccessMode{corev1api.ReadWriteOnce},\n\t\t\tCapacity: corev1api.ResourceList{},\n\t\t},\n\t}\n\tmatchingPV := &corev1api.PersistentVolume{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-csi-7d28e566-ade7-4ed6-9e15-2e44d2fbcc08\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\tAccessModes: []corev1api.PersistentVolumeAccessMode{corev1api.ReadWriteOnce},\n\t\t\tCapacity: corev1api.ResourceList{},\n\t\t\tClaimRef: &corev1api.ObjectReference{\n\t\t\t\tKind: \"PersistentVolumeClaim\",\n\t\t\t\tName: \"test-csi-pvc\",\n\t\t\t\tNamespace: \"default\",\n\t\t\t\tResourceVersion: \"1027\",\n\t\t\t\tUID: \"7d28e566-ade7-4ed6-9e15-2e44d2fbcc08\",\n\t\t\t},\n\t\t\tPersistentVolumeSource: corev1api.PersistentVolumeSource{\n\t\t\t\tCSI: &corev1api.CSIPersistentVolumeSource{\n\t\t\t\t\tDriver: \"hostpath.csi.k8s.io\",\n\t\t\t\t\tFSType: \"ext4\",\n\t\t\t\t\tVolumeAttributes: map[string]string{\n\t\t\t\t\t\t\"storage.kubernetes.io/csiProvisionerIdentity\": \"1582049697841-8081-hostpath.csi.k8s.io\",\n\t\t\t\t\t},\n\t\t\t\t\tVolumeHandle: \"e61f2b48-527a-11ea-b54f-cab6317018f1\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tPersistentVolumeReclaimPolicy: corev1api.PersistentVolumeReclaimDelete,\n\t\t\tStorageClassName: csiStorageClass,\n\t\t},\n\t\tStatus: corev1api.PersistentVolumeStatus{\n\t\t\tPhase: corev1api.VolumeBound,\n\t\t},\n\t}\n\n\tpvcWithNoVolumeName := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"no-vol-pvc\",\n\t\t\tNamespace: \"default\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tAccessModes: []corev1api.PersistentVolumeAccessMode{corev1api.ReadWriteOnce},\n\t\t\tResources: corev1api.VolumeResourceRequirements{\n\t\t\t\tRequests: corev1api.ResourceList{},\n\t\t\t},\n\t\t\tStorageClassName: &csiStorageClass,\n\t\t},\n\t\tStatus: corev1api.PersistentVolumeClaimStatus{},\n\t}\n\n\tunboundPVC := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"unbound-pvc\",\n\t\t\tNamespace: \"default\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tAccessModes: []corev1api.PersistentVolumeAccessMode{corev1api.ReadWriteOnce},\n\t\t\tResources: corev1api.VolumeResourceRequirements{\n\t\t\t\tRequests: corev1api.ResourceList{},\n\t\t\t},\n\t\t\tStorageClassName: &csiStorageClass,\n\t\t\tVolumeName: \"test-csi-7d28e566-ade7-4ed6-9e15-2e44d2fbcc08\",\n\t\t},\n\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\tPhase: corev1api.ClaimPending,\n\t\t\tAccessModes: []corev1api.PersistentVolumeAccessMode{corev1api.ReadWriteOnce},\n\t\t\tCapacity: corev1api.ResourceList{},\n\t\t},\n\t}\n\n\ttestCases := []struct {\n\t\tname string\n\t\tinPVC *corev1api.PersistentVolumeClaim\n\t\texpectError bool\n\t\texpectedPV *corev1api.PersistentVolume\n\t}{\n\t\t{\n\t\t\tname: \"should find PV matching the PVC\",\n\t\t\tinPVC: boundPVC,\n\t\t\texpectError: false,\n\t\t\texpectedPV: matchingPV,\n\t\t},\n\t\t{\n\t\t\tname: \"should fail to find PV for PVC with no volumeName\",\n\t\t\tinPVC: pvcWithNoVolumeName,\n\t\t\texpectError: true,\n\t\t\texpectedPV: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"should fail to find PV for PVC not in bound phase\",\n\t\t\tinPVC: unboundPVC,\n\t\t\texpectError: true,\n\t\t\texpectedPV: nil,\n\t\t},\n\t}\n\n\tobjs := []runtime.Object{boundPVC, matchingPV, pvcWithNoVolumeName, unboundPVC}\n\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, objs...)\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactualPV, actualError := GetPVForPVC(tc.inPVC, fakeClient)\n\n\t\t\tif tc.expectError {\n\t\t\t\trequire.Error(t, actualError, \"Want error; Got nil error\")\n\t\t\t\tassert.Nilf(t, actualPV, \"Want PV: nil; Got PV: %q\", actualPV)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\trequire.NoErrorf(t, actualError, \"Want: nil error; Got: %v\", actualError)\n\t\t\tassert.Equalf(t, actualPV.Name, tc.expectedPV.Name, \"Want PV with name %q; Got PV with name %q\", tc.expectedPV.Name, actualPV.Name)\n\t\t})\n\t}\n}\n\nfunc TestGetPVCForPodVolume(t *testing.T) {\n\tsampleVol := &corev1api.Volume{\n\t\tName: \"sample-volume\",\n\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\tClaimName: \"sample-pvc\",\n\t\t\t},\n\t\t},\n\t}\n\n\tsampleVol2 := &corev1api.Volume{\n\t\tName: \"sample-volume\",\n\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\tClaimName: \"sample-pvc-1\",\n\t\t\t},\n\t\t},\n\t}\n\n\tsampleVol3 := &corev1api.Volume{\n\t\tName: \"sample-volume\",\n\t\tVolumeSource: corev1api.VolumeSource{},\n\t}\n\n\tsamplePod := &corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"sample-pod\",\n\t\t\tNamespace: \"sample-ns\",\n\t\t},\n\n\t\tSpec: corev1api.PodSpec{\n\t\t\tContainers: []corev1api.Container{\n\t\t\t\t{\n\t\t\t\t\tName: \"sample-container\",\n\t\t\t\t\tImage: \"sample-image\",\n\t\t\t\t\tVolumeMounts: []corev1api.VolumeMount{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"sample-vm\",\n\t\t\t\t\t\t\tMountPath: \"/etc/pod-info\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t{\n\t\t\t\t\tName: \"sample-volume\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\tClaimName: \"sample-pvc\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tmatchingPVC := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"sample-pvc\",\n\t\t\tNamespace: \"sample-ns\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tAccessModes: []corev1api.PersistentVolumeAccessMode{corev1api.ReadWriteOnce},\n\t\t\tResources: corev1api.VolumeResourceRequirements{\n\t\t\t\tRequests: corev1api.ResourceList{},\n\t\t\t},\n\t\t\tStorageClassName: &csiStorageClass,\n\t\t\tVolumeName: \"test-csi-7d28e566-ade7-4ed6-9e15-2e44d2fbcc08\",\n\t\t},\n\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\tPhase: corev1api.ClaimBound,\n\t\t\tAccessModes: []corev1api.PersistentVolumeAccessMode{corev1api.ReadWriteOnce},\n\t\t\tCapacity: corev1api.ResourceList{},\n\t\t},\n\t}\n\n\ttestCases := []struct {\n\t\tname string\n\t\tvol *corev1api.Volume\n\t\tpod *corev1api.Pod\n\t\texpectedPVC *corev1api.PersistentVolumeClaim\n\t\texpectedError bool\n\t}{\n\t\t{\n\t\t\tname: \"should find PVC for volume\",\n\t\t\tvol: sampleVol,\n\t\t\tpod: samplePod,\n\t\t\texpectedPVC: matchingPVC,\n\t\t\texpectedError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should not find PVC for volume not found error case\",\n\t\t\tvol: sampleVol2,\n\t\t\tpod: samplePod,\n\t\t\texpectedPVC: nil,\n\t\t\texpectedError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"should not find PVC vol has no PVC, error case\",\n\t\t\tvol: sampleVol3,\n\t\t\tpod: samplePod,\n\t\t\texpectedPVC: nil,\n\t\t\texpectedError: true,\n\t\t},\n\t}\n\n\tobjs := []runtime.Object{matchingPVC}\n\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, objs...)\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactualPVC, actualError := GetPVCForPodVolume(tc.vol, samplePod, fakeClient)\n\t\t\tif tc.expectedError {\n\t\t\t\trequire.Error(t, actualError, \"Want error; Got nil error\")\n\t\t\t\tassert.Nilf(t, actualPVC, \"Want PV: nil; Got PV: %q\", actualPVC)\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NoErrorf(t, actualError, \"Want: nil error; Got: %v\", actualError)\n\t\t\tassert.Equalf(t, actualPVC.Name, tc.expectedPVC.Name, \"Want PVC with name %q; Got PVC with name %q\", tc.expectedPVC.Name, actualPVC)\n\t\t})\n\t}\n}\n\nfunc TestMakePodPVCAttachment(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tvolumeName string\n\t\tvolumeMode corev1api.PersistentVolumeMode\n\t\treadOnly bool\n\t\texpectedVolumeMount []corev1api.VolumeMount\n\t\texpectedVolumeDevice []corev1api.VolumeDevice\n\t\texpectedVolumePath string\n\t}{\n\t\t{\n\t\t\tname: \"no volume mode specified\",\n\t\t\tvolumeName: \"volume-1\",\n\t\t\treadOnly: true,\n\t\t\texpectedVolumeMount: []corev1api.VolumeMount{\n\t\t\t\t{\n\t\t\t\t\tName: \"volume-1\",\n\t\t\t\t\tMountPath: \"/volume-1\",\n\t\t\t\t\tReadOnly: true,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumePath: \"/volume-1\",\n\t\t},\n\t\t{\n\t\t\tname: \"fs mode specified\",\n\t\t\tvolumeName: \"volume-2\",\n\t\t\tvolumeMode: corev1api.PersistentVolumeFilesystem,\n\t\t\treadOnly: true,\n\t\t\texpectedVolumeMount: []corev1api.VolumeMount{\n\t\t\t\t{\n\t\t\t\t\tName: \"volume-2\",\n\t\t\t\t\tMountPath: \"/volume-2\",\n\t\t\t\t\tReadOnly: true,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumePath: \"/volume-2\",\n\t\t},\n\t\t{\n\t\t\tname: \"block volume mode specified\",\n\t\t\tvolumeName: \"volume-3\",\n\t\t\tvolumeMode: corev1api.PersistentVolumeBlock,\n\t\t\texpectedVolumeDevice: []corev1api.VolumeDevice{\n\t\t\t\t{\n\t\t\t\t\tName: \"volume-3\",\n\t\t\t\t\tDevicePath: \"/volume-3\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumePath: \"/volume-3\",\n\t\t},\n\t\t{\n\t\t\tname: \"fs mode specified with readOnly as false\",\n\t\t\tvolumeName: \"volume-4\",\n\t\t\treadOnly: false,\n\t\t\tvolumeMode: corev1api.PersistentVolumeFilesystem,\n\t\t\texpectedVolumeMount: []corev1api.VolumeMount{\n\t\t\t\t{\n\t\t\t\t\tName: \"volume-4\",\n\t\t\t\t\tMountPath: \"/volume-4\",\n\t\t\t\t\tReadOnly: false,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedVolumePath: \"/volume-4\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar volMode *corev1api.PersistentVolumeMode\n\t\t\tif tc.volumeMode != \"\" {\n\t\t\t\tvolMode = &tc.volumeMode\n\t\t\t}\n\n\t\t\tmount, device, path := MakePodPVCAttachment(tc.volumeName, volMode, tc.readOnly)\n\n\t\t\tassert.Equal(t, tc.expectedVolumeMount, mount)\n\t\t\tassert.Equal(t, tc.expectedVolumeDevice, device)\n\t\t\tassert.Equal(t, tc.expectedVolumePath, path)\n\t\t\tif tc.expectedVolumeMount != nil {\n\t\t\t\tassert.Equal(t, tc.expectedVolumeMount[0].ReadOnly, tc.readOnly)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestDiagnosePVC(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\tevents *corev1api.EventList\n\t\texpected string\n\t}{\n\t\t{\n\t\t\tname: \"pvc with all info but events\",\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-pvc\",\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\tVolumeName: \"fake-pv\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\t\t\tPhase: corev1api.ClaimPending,\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: \"PVC fake-ns/fake-pvc, phase Pending, binding to fake-pv\\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"pvc with all info and empty events\",\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-pvc\",\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\tVolumeName: \"fake-pv\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\t\t\tPhase: corev1api.ClaimPending,\n\t\t\t\t},\n\t\t\t},\n\t\t\tevents: &corev1api.EventList{},\n\t\t\texpected: \"PVC fake-ns/fake-pvc, phase Pending, binding to fake-pv\\n\",\n\t\t},\n\t\t{\n\t\t\tname: \"pvc with all info and events\",\n\t\t\tpvc: &corev1api.PersistentVolumeClaim{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-pvc\",\n\t\t\t\t\tNamespace: \"fake-ns\",\n\t\t\t\t\tUID: \"fake-pvc-uid\",\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\t\t\tVolumeName: \"fake-pv\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PersistentVolumeClaimStatus{\n\t\t\t\t\tPhase: corev1api.ClaimPending,\n\t\t\t\t},\n\t\t\t},\n\t\t\tevents: &corev1api.EventList{Items: []corev1api.Event{\n\t\t\t\t{\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-uid-1\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tReason: \"reason-1\",\n\t\t\t\t\tMessage: \"message-1\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-uid-2\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tReason: \"reason-2\",\n\t\t\t\t\tMessage: \"message-2\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-pvc-uid\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tReason: \"reason-3\",\n\t\t\t\t\tMessage: \"message-3\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-pvc-uid\"},\n\t\t\t\t\tType: corev1api.EventTypeNormal,\n\t\t\t\t\tReason: \"reason-4\",\n\t\t\t\t\tMessage: \"message-4\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-pvc-uid\"},\n\t\t\t\t\tType: corev1api.EventTypeNormal,\n\t\t\t\t\tReason: \"reason-5\",\n\t\t\t\t\tMessage: \"message-5\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tInvolvedObject: corev1api.ObjectReference{UID: \"fake-pvc-uid\"},\n\t\t\t\t\tType: corev1api.EventTypeWarning,\n\t\t\t\t\tReason: \"reason-6\",\n\t\t\t\t\tMessage: \"message-6\",\n\t\t\t\t},\n\t\t\t}},\n\t\t\texpected: \"PVC fake-ns/fake-pvc, phase Pending, binding to fake-pv\\nPVC event reason reason-3, message message-3\\nPVC event reason reason-6, message message-6\\n\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tdiag := DiagnosePVC(tc.pvc, tc.events)\n\t\t\tassert.Equal(t, tc.expected, diag)\n\t\t})\n\t}\n}\n\nfunc TestDiagnosePV(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tpv *corev1api.PersistentVolume\n\t\texpected string\n\t}{\n\t\t{\n\t\t\tname: \"pv with all info\",\n\t\t\tpv: &corev1api.PersistentVolume{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"fake-pv\",\n\t\t\t\t},\n\t\t\t\tStatus: corev1api.PersistentVolumeStatus{\n\t\t\t\t\tPhase: corev1api.VolumePending,\n\t\t\t\t\tMessage: \"fake-message\",\n\t\t\t\t\tReason: \"fake-reason\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: \"PV fake-pv, phase Pending, reason fake-reason, message fake-message\\n\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tdiag := DiagnosePV(tc.pv)\n\t\t\tassert.Equal(t, tc.expected, diag)\n\t\t})\n\t}\n}\n\nfunc TestGetPVCAttachingNodeOS(t *testing.T) {\n\tstorageClass := \"fake-storage-class\"\n\tnodeNoOSLabel := builder.ForNode(\"fake-node\").Result()\n\tnodeWindows := builder.ForNode(\"fake-node\").Labels(map[string]string{\"kubernetes.io/os\": \"windows\"}).Result()\n\n\tpvcObj := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-namespace\",\n\t\t\tName: \"fake-pvc\",\n\t\t},\n\t}\n\n\tblockMode := corev1api.PersistentVolumeBlock\n\tpvcObjBlockMode := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-namespace\",\n\t\t\tName: \"fake-pvc\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tVolumeMode: &blockMode,\n\t\t},\n\t}\n\n\tpvName := \"fake-volume-name\"\n\tpvcObjWithAll := &corev1api.PersistentVolumeClaim{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tNamespace: \"fake-namespace\",\n\t\t\tName: \"fake-pvc\",\n\t\t\tAnnotations: map[string]string{KubeAnnSelectedNode: \"fake-node\"},\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeClaimSpec{\n\t\t\tVolumeName: pvName,\n\t\t\tStorageClassName: &storageClass,\n\t\t},\n\t}\n\n\tscObjWithoutFSType := &storagev1api.StorageClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-storage-class\",\n\t\t},\n\t}\n\n\tscObjWithFSType := &storagev1api.StorageClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-storage-class\",\n\t\t},\n\t\tParameters: map[string]string{\"csi.storage.k8s.io/fstype\": \"ntfs\"},\n\t}\n\n\tscObjWithFSTypeExt := &storagev1api.StorageClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-storage-class\",\n\t\t},\n\t\tParameters: map[string]string{\"csi.storage.k8s.io/fstype\": \"ext4\"},\n\t}\n\n\tvolAttachEmpty := &storagev1api.VolumeAttachment{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-volume-attach-1\",\n\t\t},\n\t}\n\n\tvolAttachWithVolume := &storagev1api.VolumeAttachment{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-volume-attach-2\",\n\t\t},\n\t\tSpec: storagev1api.VolumeAttachmentSpec{\n\t\t\tSource: storagev1api.VolumeAttachmentSource{\n\t\t\t\tPersistentVolumeName: &pvName,\n\t\t\t},\n\t\t\tNodeName: \"fake-node\",\n\t\t},\n\t}\n\n\totherPVName := \"other-volume-name\"\n\tvolAttachWithOtherVolume := &storagev1api.VolumeAttachment{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-volume-attach-3\",\n\t\t},\n\t\tSpec: storagev1api.VolumeAttachmentSpec{\n\t\t\tSource: storagev1api.VolumeAttachmentSource{\n\t\t\t\tPersistentVolumeName: &otherPVName,\n\t\t\t},\n\t\t},\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\tkubeClientObj []runtime.Object\n\t\texpectedNodeOS string\n\t}{\n\t\t{\n\t\t\tname: \"no selected node, volume name and storage class\",\n\t\t\tpvc: pvcObj,\n\t\t\texpectedNodeOS: NodeOSLinux,\n\t\t},\n\t\t{\n\t\t\tname: \"fallback\",\n\t\t\tpvc: pvcObjWithAll,\n\t\t\texpectedNodeOS: NodeOSLinux,\n\t\t},\n\t\t{\n\t\t\tname: \"with selected node, but node without label\",\n\t\t\tpvc: pvcObjWithAll,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tnodeNoOSLabel,\n\t\t\t},\n\t\t\texpectedNodeOS: NodeOSLinux,\n\t\t},\n\t\t{\n\t\t\tname: \"volume attachment exist, but get node os fails\",\n\t\t\tpvc: pvcObjWithAll,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tscObjWithFSType,\n\t\t\t\tvolAttachWithVolume,\n\t\t\t},\n\t\t\texpectedNodeOS: NodeOSWindows,\n\t\t},\n\t\t{\n\t\t\tname: \"volume attachment exist, node without label\",\n\t\t\tpvc: pvcObjWithAll,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tnodeNoOSLabel,\n\t\t\t\tscObjWithFSType,\n\t\t\t\tvolAttachWithVolume,\n\t\t\t},\n\t\t\texpectedNodeOS: NodeOSWindows,\n\t\t},\n\t\t{\n\t\t\tname: \"sc without fsType\",\n\t\t\tpvc: pvcObjWithAll,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tscObjWithoutFSType,\n\t\t\t},\n\t\t\texpectedNodeOS: NodeOSLinux,\n\t\t},\n\t\t{\n\t\t\tname: \"deduce from node os by selected node\",\n\t\t\tpvc: pvcObjWithAll,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tnodeWindows,\n\t\t\t\tscObjWithFSTypeExt,\n\t\t\t},\n\t\t\texpectedNodeOS: NodeOSWindows,\n\t\t},\n\t\t{\n\t\t\tname: \"deduce from sc\",\n\t\t\tpvc: pvcObjWithAll,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tnodeNoOSLabel,\n\t\t\t\tscObjWithFSType,\n\t\t\t},\n\t\t\texpectedNodeOS: NodeOSWindows,\n\t\t},\n\t\t{\n\t\t\tname: \"deduce from attached node os\",\n\t\t\tpvc: pvcObjWithAll,\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tnodeWindows,\n\t\t\t\tscObjWithFSTypeExt,\n\t\t\t\tvolAttachEmpty,\n\t\t\t\tvolAttachWithVolume,\n\t\t\t\tvolAttachWithOtherVolume,\n\t\t\t},\n\t\t\texpectedNodeOS: NodeOSWindows,\n\t\t},\n\t\t{\n\t\t\tname: \"block access\",\n\t\t\tpvc: pvcObjBlockMode,\n\t\t\texpectedNodeOS: NodeOSLinux,\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tvar kubeClient kubernetes.Interface = fakeKubeClient\n\n\t\t\tnodeOS := GetPVCAttachingNodeOS(test.pvc, kubeClient.CoreV1(), kubeClient.StorageV1(), velerotest.NewLogger())\n\n\t\t\tassert.Equal(t, test.expectedNodeOS, nodeOS)\n\t\t})\n\t}\n}\n\nfunc TestGetVolumeTopology(t *testing.T) {\n\tpvWithoutNodeAffinity := &corev1api.PersistentVolume{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-pv\",\n\t\t},\n\t}\n\n\tpvWithNodeAffinity := &corev1api.PersistentVolume{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-pv\",\n\t\t},\n\t\tSpec: corev1api.PersistentVolumeSpec{\n\t\t\tNodeAffinity: &corev1api.VolumeNodeAffinity{\n\t\t\t\tRequired: &corev1api.NodeSelector{\n\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tKey: \"fake-key\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tscObjWithoutVolumeBind := &storagev1api.StorageClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-storage-class\",\n\t\t},\n\t}\n\n\tvolumeBindImmediate := storagev1api.VolumeBindingImmediate\n\tscObjWithImeediateBind := &storagev1api.StorageClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-storage-class\",\n\t\t},\n\t\tVolumeBindingMode: &volumeBindImmediate,\n\t}\n\n\tvolumeBindWffc := storagev1api.VolumeBindingWaitForFirstConsumer\n\tscObjWithWffcBind := &storagev1api.StorageClass{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"fake-storage-class\",\n\t\t},\n\t\tVolumeBindingMode: &volumeBindWffc,\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\tpvName string\n\t\tscName string\n\t\tkubeClientObj []runtime.Object\n\t\texpectedErr string\n\t\texpected *corev1api.NodeSelector\n\t}{\n\t\t{\n\t\t\tname: \"invalid pvName\",\n\t\t\tscName: \"fake-storage-class\",\n\t\t\texpectedErr: \"invalid parameter, pv , sc fake-storage-class\",\n\t\t},\n\t\t{\n\t\t\tname: \"invalid scName\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\texpectedErr: \"invalid parameter, pv fake-pv, sc \",\n\t\t},\n\t\t{\n\t\t\tname: \"no sc\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tscName: \"fake-storage-class\",\n\t\t\texpectedErr: \"error getting storage class fake-storage-class: storageclasses.storage.k8s.io \\\"fake-storage-class\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"sc without binding mode\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tscName: \"fake-storage-class\",\n\t\t\tkubeClientObj: []runtime.Object{scObjWithoutVolumeBind},\n\t\t},\n\t\t{\n\t\t\tname: \"sc without immediate binding mode\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tscName: \"fake-storage-class\",\n\t\t\tkubeClientObj: []runtime.Object{scObjWithImeediateBind},\n\t\t},\n\t\t{\n\t\t\tname: \"get pv fail\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tscName: \"fake-storage-class\",\n\t\t\tkubeClientObj: []runtime.Object{scObjWithWffcBind},\n\t\t\texpectedErr: \"error getting PV fake-pv: persistentvolumes \\\"fake-pv\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"pv with no affinity\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tscName: \"fake-storage-class\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tscObjWithWffcBind,\n\t\t\t\tpvWithoutNodeAffinity,\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"pv with affinity\",\n\t\t\tpvName: \"fake-pv\",\n\t\t\tscName: \"fake-storage-class\",\n\t\t\tkubeClientObj: []runtime.Object{\n\t\t\t\tscObjWithWffcBind,\n\t\t\t\tpvWithNodeAffinity,\n\t\t\t},\n\t\t\texpected: &corev1api.NodeSelector{\n\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t{\n\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tKey: \"fake-key\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tfakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)\n\n\t\t\tvar kubeClient kubernetes.Interface = fakeKubeClient\n\n\t\t\taffinity, err := GetVolumeTopology(t.Context(), kubeClient.CoreV1(), kubeClient.StorageV1(), test.pvName, test.scName)\n\n\t\t\tif test.expectedErr != \"\" {\n\t\t\t\tassert.EqualError(t, err, test.expectedErr)\n\t\t\t} else {\n\t\t\t\tassert.Equal(t, test.expected, affinity)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/util/kube/resource_deletionstatus_tracker.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"fmt\"\n\t\"sync\"\n\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n)\n\n// resourceDeletionStatusTracker keeps track of items pending deletion.\ntype ResourceDeletionStatusTracker interface {\n\t// Add informs the tracker that a polling is in progress to check namespace deletion status.\n\tAdd(kind, ns, name string)\n\t// Delete informs the tracker that a namespace deletion is completed.\n\tDelete(kind, ns, name string)\n\t// Contains returns true if the tracker is tracking the namespace deletion progress.\n\tContains(kind, ns, name string) bool\n}\n\ntype resourceDeletionStatusTracker struct {\n\tlock sync.RWMutex\n\tisNameSpacePresentInPollingSet sets.Set[string]\n}\n\n// NewResourceDeletionStatusTracker returns a new ResourceDeletionStatusTracker.\nfunc NewResourceDeletionStatusTracker() ResourceDeletionStatusTracker {\n\treturn &resourceDeletionStatusTracker{\n\t\tisNameSpacePresentInPollingSet: sets.New[string](),\n\t}\n}\n\nfunc (bt *resourceDeletionStatusTracker) Add(kind, ns, name string) {\n\tbt.lock.Lock()\n\tdefer bt.lock.Unlock()\n\n\tbt.isNameSpacePresentInPollingSet.Insert(resourceDeletionStatusTrackerKey(kind, ns, name))\n}\n\nfunc (bt *resourceDeletionStatusTracker) Delete(kind, ns, name string) {\n\tbt.lock.Lock()\n\tdefer bt.lock.Unlock()\n\n\tbt.isNameSpacePresentInPollingSet.Delete(resourceDeletionStatusTrackerKey(kind, ns, name))\n}\n\nfunc (bt *resourceDeletionStatusTracker) Contains(kind, ns, name string) bool {\n\tbt.lock.RLock()\n\tdefer bt.lock.RUnlock()\n\n\treturn bt.isNameSpacePresentInPollingSet.Has(resourceDeletionStatusTrackerKey(kind, ns, name))\n}\n\nfunc resourceDeletionStatusTrackerKey(kind, ns, name string) string {\n\treturn fmt.Sprintf(\"%s/%s/%s\", kind, ns, name)\n}\n" }, { "path": "pkg/util/kube/resource_requirements.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"github.com/pkg/errors\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/api/resource\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/constant\"\n)\n\n// ParseCPUAndMemoryResources is a helper function that parses CPU and memory requests and limits,\n// using default values for ephemeral storage.\nfunc ParseCPUAndMemoryResources(cpuRequest, memRequest, cpuLimit, memLimit string) (corev1api.ResourceRequirements, error) {\n\treturn ParseResourceRequirements(\n\t\tcpuRequest,\n\t\tmemRequest,\n\t\tconstant.DefaultEphemeralStorageRequest,\n\t\tcpuLimit,\n\t\tmemLimit,\n\t\tconstant.DefaultEphemeralStorageLimit,\n\t)\n}\n\n// ParseResourceRequirements takes a set of CPU, memory, ephemeral storage requests and limit string\n// values and returns a ResourceRequirements struct to be used in a Container.\n// An error is returned if we cannot parse the request/limit.\nfunc ParseResourceRequirements(\n\tcpuRequest,\n\tmemRequest,\n\tephemeralStorageRequest,\n\tcpuLimit,\n\tmemLimit,\n\tephemeralStorageLimit string,\n) (corev1api.ResourceRequirements, error) {\n\tresources := corev1api.ResourceRequirements{\n\t\tRequests: corev1api.ResourceList{},\n\t\tLimits: corev1api.ResourceList{},\n\t}\n\n\tparsedCPURequest, err := resource.ParseQuantity(cpuRequest)\n\tif err != nil {\n\t\treturn resources, errors.Wrapf(err, `couldn't parse CPU request \"%s\"`, cpuRequest)\n\t}\n\n\tparsedMemRequest, err := resource.ParseQuantity(memRequest)\n\tif err != nil {\n\t\treturn resources, errors.Wrapf(err, `couldn't parse memory request \"%s\"`, memRequest)\n\t}\n\n\tparsedEphemeralStorageRequest, err := resource.ParseQuantity(ephemeralStorageRequest)\n\tif err != nil {\n\t\treturn resources, errors.Wrapf(err, `couldn't parse ephemeral storage request \"%s\"`, ephemeralStorageRequest)\n\t}\n\n\tparsedCPULimit, err := resource.ParseQuantity(cpuLimit)\n\tif err != nil {\n\t\treturn resources, errors.Wrapf(err, `couldn't parse CPU limit \"%s\"`, cpuLimit)\n\t}\n\n\tparsedMemLimit, err := resource.ParseQuantity(memLimit)\n\tif err != nil {\n\t\treturn resources, errors.Wrapf(err, `couldn't parse memory limit \"%s\"`, memLimit)\n\t}\n\n\tparsedEphemeralStorageLimit, err := resource.ParseQuantity(ephemeralStorageLimit)\n\tif err != nil {\n\t\treturn resources, errors.Wrapf(err, `couldn't parse ephemeral storage limit \"%s\"`, ephemeralStorageLimit)\n\t}\n\n\t// A quantity of 0 is treated as unbounded\n\tunbounded := resource.MustParse(\"0\")\n\n\tif parsedCPULimit != unbounded && parsedCPURequest.Cmp(parsedCPULimit) > 0 {\n\t\treturn resources, errors.WithStack(errors.Errorf(`CPU request \"%s\" must be less than or equal to CPU limit \"%s\"`, cpuRequest, cpuLimit))\n\t}\n\n\tif parsedMemLimit != unbounded && parsedMemRequest.Cmp(parsedMemLimit) > 0 {\n\t\treturn resources, errors.WithStack(errors.Errorf(`Memory request \"%s\" must be less than or equal to Memory limit \"%s\"`, memRequest, memLimit))\n\t}\n\n\tif parsedEphemeralStorageLimit != unbounded && parsedEphemeralStorageRequest.Cmp(parsedEphemeralStorageLimit) > 0 {\n\t\treturn resources, errors.WithStack(errors.Errorf(`Ephemeral storage request \"%s\" must be less than or equal to Ephemeral storage limit \"%s\"`, ephemeralStorageRequest, ephemeralStorageLimit))\n\t}\n\n\t// Only set resources if they are not unbounded\n\tif parsedCPURequest != unbounded {\n\t\tresources.Requests[corev1api.ResourceCPU] = parsedCPURequest\n\t}\n\tif parsedMemRequest != unbounded {\n\t\tresources.Requests[corev1api.ResourceMemory] = parsedMemRequest\n\t}\n\tif parsedEphemeralStorageRequest != unbounded {\n\t\tresources.Requests[corev1api.ResourceEphemeralStorage] = parsedEphemeralStorageRequest\n\t}\n\tif parsedCPULimit != unbounded {\n\t\tresources.Limits[corev1api.ResourceCPU] = parsedCPULimit\n\t}\n\tif parsedMemLimit != unbounded {\n\t\tresources.Limits[corev1api.ResourceMemory] = parsedMemLimit\n\t}\n\tif parsedEphemeralStorageLimit != unbounded {\n\t\tresources.Limits[corev1api.ResourceEphemeralStorage] = parsedEphemeralStorageLimit\n\t}\n\n\treturn resources, nil\n}\n" }, { "path": "pkg/util/kube/resource_requirements_test.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"k8s.io/apimachinery/pkg/api/resource\"\n)\n\nfunc TestParseResourceRequirements(t *testing.T) {\n\ttype args struct {\n\t\tcpuRequest string\n\t\tmemRequest string\n\t\tephemeralStorageRequest string\n\t\tcpuLimit string\n\t\tmemLimit string\n\t\tephemeralStorageLimit string\n\t}\n\ttests := []struct {\n\t\tname string\n\t\targs args\n\t\twantErr bool\n\t\texpected *corev1api.ResourceRequirements\n\t}{\n\t\t{\"unbounded quantities\", args{\"0\", \"0\", \"0\", \"0\", \"0\", \"0\"}, false, &corev1api.ResourceRequirements{\n\t\t\tRequests: corev1api.ResourceList{},\n\t\t\tLimits: corev1api.ResourceList{},\n\t\t}},\n\t\t{\"valid quantities\", args{\"100m\", \"128Mi\", \"5Gi\", \"200m\", \"256Mi\", \"10Gi\"}, false, nil},\n\t\t{\"CPU request with unbounded limit\", args{\"100m\", \"128Mi\", \"5Gi\", \"0\", \"256Mi\", \"10Gi\"}, false, &corev1api.ResourceRequirements{\n\t\t\tRequests: corev1api.ResourceList{\n\t\t\t\tcorev1api.ResourceCPU: resource.MustParse(\"100m\"),\n\t\t\t\tcorev1api.ResourceMemory: resource.MustParse(\"128Mi\"),\n\t\t\t\tcorev1api.ResourceEphemeralStorage: resource.MustParse(\"5Gi\"),\n\t\t\t},\n\t\t\tLimits: corev1api.ResourceList{\n\t\t\t\tcorev1api.ResourceMemory: resource.MustParse(\"256Mi\"),\n\t\t\t\tcorev1api.ResourceEphemeralStorage: resource.MustParse(\"10Gi\"),\n\t\t\t},\n\t\t}},\n\t\t{\"Mem request with unbounded limit\", args{\"100m\", \"128Mi\", \"5Gi\", \"200m\", \"0\", \"10Gi\"}, false, &corev1api.ResourceRequirements{\n\t\t\tRequests: corev1api.ResourceList{\n\t\t\t\tcorev1api.ResourceCPU: resource.MustParse(\"100m\"),\n\t\t\t\tcorev1api.ResourceMemory: resource.MustParse(\"128Mi\"),\n\t\t\t\tcorev1api.ResourceEphemeralStorage: resource.MustParse(\"5Gi\"),\n\t\t\t},\n\t\t\tLimits: corev1api.ResourceList{\n\t\t\t\tcorev1api.ResourceCPU: resource.MustParse(\"200m\"),\n\t\t\t\tcorev1api.ResourceEphemeralStorage: resource.MustParse(\"10Gi\"),\n\t\t\t},\n\t\t}},\n\t\t{\"Ephemeral storage request with unbounded limit\", args{\"100m\", \"128Mi\", \"5Gi\", \"200m\", \"256Mi\", \"0\"}, false, &corev1api.ResourceRequirements{\n\t\t\tRequests: corev1api.ResourceList{\n\t\t\t\tcorev1api.ResourceCPU: resource.MustParse(\"100m\"),\n\t\t\t\tcorev1api.ResourceMemory: resource.MustParse(\"128Mi\"),\n\t\t\t\tcorev1api.ResourceEphemeralStorage: resource.MustParse(\"5Gi\"),\n\t\t\t},\n\t\t\tLimits: corev1api.ResourceList{\n\t\t\t\tcorev1api.ResourceCPU: resource.MustParse(\"200m\"),\n\t\t\t\tcorev1api.ResourceMemory: resource.MustParse(\"256Mi\"),\n\t\t\t},\n\t\t}},\n\n\t\t{\"CPU/Mem/EphemeralStorage requests with unbounded limits\", args{\"100m\", \"128Mi\", \"5Gi\", \"0\", \"0\", \"0\"}, false, &corev1api.ResourceRequirements{\n\t\t\tRequests: corev1api.ResourceList{\n\t\t\t\tcorev1api.ResourceCPU: resource.MustParse(\"100m\"),\n\t\t\t\tcorev1api.ResourceMemory: resource.MustParse(\"128Mi\"),\n\t\t\t\tcorev1api.ResourceEphemeralStorage: resource.MustParse(\"5Gi\"),\n\t\t\t},\n\t\t\tLimits: corev1api.ResourceList{},\n\t\t}},\n\t\t{\"invalid quantity\", args{\"100m\", \"invalid\", \"1Gi\", \"200m\", \"256Mi\", \"valid\"}, true, nil},\n\t\t{\"CPU request greater than limit\", args{\"300m\", \"128Mi\", \"5Gi\", \"200m\", \"256Mi\", \"10Gi\"}, true, nil},\n\t\t{\"memory request greater than limit\", args{\"100m\", \"512Mi\", \"5Gi\", \"200m\", \"256Mi\", \"10Gi\"}, true, nil},\n\t\t{\"ephemeral storage request greater than limit\", args{\"100m\", \"128Mi\", \"10Gi\", \"200m\", \"256Mi\", \"5Gi\"}, true, nil},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tgot, err := ParseResourceRequirements(tt.args.cpuRequest, tt.args.memRequest, tt.args.ephemeralStorageRequest, tt.args.cpuLimit, tt.args.memLimit, tt.args.ephemeralStorageLimit)\n\t\t\tif tt.wantErr {\n\t\t\t\tassert.Error(t, err)\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NoError(t, err)\n\n\t\t\tvar expected corev1api.ResourceRequirements\n\t\t\tif tt.expected == nil {\n\t\t\t\texpected = corev1api.ResourceRequirements{\n\t\t\t\t\tRequests: corev1api.ResourceList{\n\t\t\t\t\t\tcorev1api.ResourceCPU: resource.MustParse(tt.args.cpuRequest),\n\t\t\t\t\t\tcorev1api.ResourceMemory: resource.MustParse(tt.args.memRequest),\n\t\t\t\t\t\tcorev1api.ResourceEphemeralStorage: resource.MustParse(tt.args.ephemeralStorageRequest),\n\t\t\t\t\t},\n\t\t\t\t\tLimits: corev1api.ResourceList{\n\t\t\t\t\t\tcorev1api.ResourceCPU: resource.MustParse(tt.args.cpuLimit),\n\t\t\t\t\t\tcorev1api.ResourceMemory: resource.MustParse(tt.args.memLimit),\n\t\t\t\t\t\tcorev1api.ResourceEphemeralStorage: resource.MustParse(tt.args.ephemeralStorageLimit),\n\t\t\t\t\t},\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\texpected = *tt.expected\n\t\t\t}\n\n\t\t\tassert.Equal(t, expected, got)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/util/kube/secrets.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"context\"\n\n\t\"github.com/pkg/errors\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tkbclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n)\n\nfunc GetSecret(client kbclient.Client, namespace, name string) (*corev1api.Secret, error) {\n\tsecret := &corev1api.Secret{}\n\tif err := client.Get(context.TODO(), kbclient.ObjectKey{\n\t\tNamespace: namespace,\n\t\tName: name,\n\t}, secret); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn secret, nil\n}\n\nfunc GetSecretKey(client kbclient.Client, namespace string, selector *corev1api.SecretKeySelector) ([]byte, error) {\n\tsecret, err := GetSecret(client, namespace, selector.Name)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tkey, found := secret.Data[selector.Key]\n\tif !found {\n\t\treturn nil, errors.Errorf(\"%q secret is missing data for key %q\", selector.Name, selector.Key)\n\t}\n\n\treturn key, nil\n}\n" }, { "path": "pkg/util/kube/secrets_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestGetSecretKey(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tsecrets []*corev1api.Secret\n\t\tnamespace string\n\t\tselector *corev1api.SecretKeySelector\n\t\texpectedData string\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"error is returned when secret doesn't exist\",\n\t\t\tsecrets: []*corev1api.Secret{\n\t\t\t\tbuilder.ForSecret(\"ns-1\", \"secret1\").Data(map[string][]byte{\n\t\t\t\t\t\"key1\": []byte(\"ns1-secretdata1\"),\n\t\t\t\t}).Result(),\n\t\t\t},\n\t\t\tnamespace: \"ns-1\",\n\t\t\tselector: builder.ForSecretKeySelector(\"non-existent-secret\", \"key2\").Result(),\n\t\t\texpectedErr: \"secrets \\\"non-existent-secret\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"error is returned when key is missing from secret\",\n\t\t\tsecrets: []*corev1api.Secret{\n\t\t\t\tbuilder.ForSecret(\"ns-1\", \"secret1\").Data(map[string][]byte{\n\t\t\t\t\t\"key1\": []byte(\"ns1-secretdata1\"),\n\t\t\t\t}).Result(),\n\t\t\t},\n\t\t\tnamespace: \"ns-1\",\n\t\t\tselector: builder.ForSecretKeySelector(\"secret1\", \"non-existent-key\").Result(),\n\t\t\texpectedErr: \"\\\"secret1\\\" secret is missing data for key \\\"non-existent-key\\\"\",\n\t\t},\n\t\t{\n\t\t\tname: \"key specified in selector is returned\",\n\t\t\tsecrets: []*corev1api.Secret{\n\t\t\t\tbuilder.ForSecret(\"ns-1\", \"secret1\").Data(map[string][]byte{\n\t\t\t\t\t\"key1\": []byte(\"ns1-secretdata1\"),\n\t\t\t\t\t\"key2\": []byte(\"ns1-secretdata2\"),\n\t\t\t\t}).Result(),\n\t\t\t\tbuilder.ForSecret(\"ns-2\", \"secret1\").Data(map[string][]byte{\n\t\t\t\t\t\"key1\": []byte(\"ns2-secretdata1\"),\n\t\t\t\t\t\"key2\": []byte(\"ns2-secretdata2\"),\n\t\t\t\t}).Result(),\n\t\t\t},\n\t\t\tnamespace: \"ns-2\",\n\t\t\tselector: builder.ForSecretKeySelector(\"secret1\", \"key2\").Result(),\n\t\t\texpectedData: \"ns2-secretdata2\",\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t)\n\n\t\t\tfor _, secret := range tc.secrets {\n\t\t\t\trequire.NoError(t, fakeClient.Create(t.Context(), secret))\n\t\t\t}\n\n\t\t\tdata, err := GetSecretKey(fakeClient, tc.namespace, tc.selector)\n\t\t\tif tc.expectedErr != \"\" {\n\t\t\t\trequire.Nil(t, data)\n\t\t\t\trequire.EqualError(t, err, tc.expectedErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t\trequire.Equal(t, tc.expectedData, string(data))\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/util/kube/security_context.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"strconv\"\n\n\t\"github.com/pkg/errors\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\t\"sigs.k8s.io/yaml\"\n)\n\nfunc ParseSecurityContext(runAsUser string, runAsGroup string, allowPrivilegeEscalation string, secCtx string) (corev1api.SecurityContext, error) {\n\tsecurityContext := corev1api.SecurityContext{}\n\n\tif runAsUser != \"\" {\n\t\tparsedRunAsUser, err := strconv.ParseInt(runAsUser, 10, 64)\n\t\tif err != nil {\n\t\t\treturn securityContext, errors.WithStack(errors.Errorf(`Security context runAsUser \"%s\" is not a number`, runAsUser))\n\t\t}\n\n\t\tsecurityContext.RunAsUser = &parsedRunAsUser\n\t}\n\n\tif runAsGroup != \"\" {\n\t\tparsedRunAsGroup, err := strconv.ParseInt(runAsGroup, 10, 64)\n\t\tif err != nil {\n\t\t\treturn securityContext, errors.WithStack(errors.Errorf(`Security context runAsGroup \"%s\" is not a number`, runAsGroup))\n\t\t}\n\n\t\tsecurityContext.RunAsGroup = &parsedRunAsGroup\n\t}\n\n\tif allowPrivilegeEscalation != \"\" {\n\t\tparsedAllowPrivilegeEscalation, err := strconv.ParseBool(allowPrivilegeEscalation)\n\t\tif err != nil {\n\t\t\treturn securityContext, errors.WithStack(errors.Errorf(`Security context allowPrivilegeEscalation \"%s\" is not a boolean`, allowPrivilegeEscalation))\n\t\t}\n\n\t\tsecurityContext.AllowPrivilegeEscalation = &parsedAllowPrivilegeEscalation\n\t}\n\n\tif secCtx != \"\" {\n\t\terr := yaml.UnmarshalStrict([]byte(secCtx), &securityContext)\n\t\tif err != nil {\n\t\t\treturn securityContext, errors.WithStack(errors.Errorf(`Security context secCtx error: \"%s\"`, err))\n\t\t}\n\t}\n\n\treturn securityContext, nil\n}\n" }, { "path": "pkg/util/kube/security_context_test.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n)\n\nfunc TestParseSecurityContext(t *testing.T) {\n\ttype args struct {\n\t\trunAsUser string\n\t\trunAsGroup string\n\t\tallowPrivilegeEscalation string\n\t\tsecCtx string\n\t}\n\ttests := []struct {\n\t\tname string\n\t\targs args\n\t\twantErr bool\n\t\texpected *corev1api.SecurityContext\n\t}{\n\t\t{\n\t\t\t\"valid security context\",\n\t\t\targs{\"1001\", \"999\", \"true\", ``},\n\t\t\tfalse,\n\t\t\t&corev1api.SecurityContext{\n\t\t\t\tRunAsUser: pointInt64(1001),\n\t\t\t\tRunAsGroup: pointInt64(999),\n\t\t\t\tAllowPrivilegeEscalation: boolptr.True(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\t\"valid security context with override runAsUser\",\n\t\t\targs{\"1001\", \"999\", \"true\", `runAsUser: 2000`},\n\t\t\tfalse,\n\t\t\t&corev1api.SecurityContext{\n\t\t\t\tRunAsUser: pointInt64(2000),\n\t\t\t\tRunAsGroup: pointInt64(999),\n\t\t\t\tAllowPrivilegeEscalation: boolptr.True(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\t\"valid securityContext with comments only secCtx key\",\n\t\t\targs{\"\", \"\", \"\", `\ncapabilities:\n drop:\n - ALL\n add:\n - cap1\n - cap2\nseLinuxOptions:\n user: userLabel\n role: roleLabel\n type: typeLabel\n level: levelLabel\n# user www-data\nrunAsUser: 3333\n# group www-data\nrunAsGroup: 3333\nrunAsNonRoot: true\nreadOnlyRootFilesystem: true\nallowPrivilegeEscalation: false`},\n\t\t\tfalse,\n\t\t\t&corev1api.SecurityContext{\n\t\t\t\tRunAsUser: pointInt64(3333),\n\t\t\t\tRunAsGroup: pointInt64(3333),\n\t\t\t\tCapabilities: &corev1api.Capabilities{\n\t\t\t\t\tDrop: []corev1api.Capability{\"ALL\"},\n\t\t\t\t\tAdd: []corev1api.Capability{\"cap1\", \"cap2\"},\n\t\t\t\t},\n\t\t\t\tSELinuxOptions: &corev1api.SELinuxOptions{\n\t\t\t\t\tUser: \"userLabel\",\n\t\t\t\t\tRole: \"roleLabel\",\n\t\t\t\t\tType: \"typeLabel\",\n\t\t\t\t\tLevel: \"levelLabel\",\n\t\t\t\t},\n\t\t\t\tRunAsNonRoot: boolptr.True(),\n\t\t\t\tReadOnlyRootFilesystem: boolptr.True(),\n\t\t\t\tAllowPrivilegeEscalation: boolptr.False(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\t\"valid securityContext with comments only secCtx key check seLinuxOptions is correctly parsed\",\n\t\t\targs{\"\", \"\", \"\", `\ncapabilities:\n drop:\n - ALL\n add:\n - cap1\n - cap2\nseLinuxOptions:\n user: userLabelFail\n role: roleLabel\n type: typeLabel\n level: levelLabel\n# user www-data\nrunAsUser: 3333\n# group www-data\nrunAsGroup: 3333\nrunAsNonRoot: true\nreadOnlyRootFilesystem: true\nallowPrivilegeEscalation: false`},\n\t\t\ttrue,\n\t\t\t&corev1api.SecurityContext{\n\t\t\t\tRunAsUser: pointInt64(3333),\n\t\t\t\tRunAsGroup: pointInt64(3333),\n\t\t\t\tCapabilities: &corev1api.Capabilities{\n\t\t\t\t\tDrop: []corev1api.Capability{\"ALL\"},\n\t\t\t\t\tAdd: []corev1api.Capability{\"cap1\", \"cap2\"},\n\t\t\t\t},\n\t\t\t\tSELinuxOptions: &corev1api.SELinuxOptions{\n\t\t\t\t\tUser: \"userLabel\",\n\t\t\t\t\tRole: \"roleLabel\",\n\t\t\t\t\tType: \"typeLabel\",\n\t\t\t\t\tLevel: \"levelLabel\",\n\t\t\t\t},\n\t\t\t\tRunAsNonRoot: boolptr.True(),\n\t\t\t\tReadOnlyRootFilesystem: boolptr.True(),\n\t\t\t\tAllowPrivilegeEscalation: boolptr.False(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\t\"valid securityContext with secCtx key override runAsUser runAsGroup and allowPrivilegeEscalation\",\n\t\t\targs{\"1001\", \"999\", \"true\", `\ncapabilities:\n drop:\n - ALL\n add:\n - cap1\n - cap2\nseLinuxOptions:\n user: userLabel\n role: roleLabel\n type: typeLabel\n level: levelLabel\n# user www-data\nrunAsUser: 3333\n# group www-data\nrunAsGroup: 3333\nrunAsNonRoot: true\nreadOnlyRootFilesystem: true\nallowPrivilegeEscalation: false`},\n\t\t\tfalse,\n\t\t\t&corev1api.SecurityContext{\n\t\t\t\tRunAsUser: pointInt64(3333),\n\t\t\t\tRunAsGroup: pointInt64(3333),\n\t\t\t\tCapabilities: &corev1api.Capabilities{\n\t\t\t\t\tDrop: []corev1api.Capability{\"ALL\"},\n\t\t\t\t\tAdd: []corev1api.Capability{\"cap1\", \"cap2\"},\n\t\t\t\t},\n\t\t\t\tSELinuxOptions: &corev1api.SELinuxOptions{\n\t\t\t\t\tUser: \"userLabel\",\n\t\t\t\t\tRole: \"roleLabel\",\n\t\t\t\t\tType: \"typeLabel\",\n\t\t\t\t\tLevel: \"levelLabel\",\n\t\t\t\t},\n\t\t\t\tRunAsNonRoot: boolptr.True(),\n\t\t\t\tReadOnlyRootFilesystem: boolptr.True(),\n\t\t\t\tAllowPrivilegeEscalation: boolptr.False(),\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\t\"another valid security context\",\n\t\t\targs{\"1001\", \"999\", \"false\", \"\"},\n\t\t\tfalse,\n\t\t\t&corev1api.SecurityContext{\n\t\t\t\tRunAsUser: pointInt64(1001),\n\t\t\t\tRunAsGroup: pointInt64(999),\n\t\t\t\tAllowPrivilegeEscalation: boolptr.False(),\n\t\t\t},\n\t\t},\n\t\t{\"security context without runAsGroup\", args{\"1001\", \"\", \"\", \"\"}, false, &corev1api.SecurityContext{\n\t\t\tRunAsUser: pointInt64(1001),\n\t\t}},\n\t\t{\"security context without runAsUser\", args{\"\", \"999\", \"\", \"\"}, false, &corev1api.SecurityContext{\n\t\t\tRunAsGroup: pointInt64(999),\n\t\t}},\n\t\t{\"empty context without runAsUser\", args{\"\", \"\", \"\", \"\"}, false, &corev1api.SecurityContext{}},\n\t\t{\n\t\t\t\"invalid securityContext secCtx unknown key\",\n\t\t\targs{\"\", \"\", \"\", `\ncapabilitiesUnknownkey:\n drop:\n - ALL\n add:\n - cap1\n - cap2\n# user www-data\nrunAsUser: 3333\n# group www-data\nrunAsGroup: 3333\nrunAsNonRoot: true\nreadOnlyRootFilesystem: true\nallowPrivilegeEscalation: false`},\n\t\t\ttrue, nil,\n\t\t},\n\t\t{\n\t\t\t\"invalid securityContext secCtx wrong value type string instead of bool\",\n\t\t\targs{\"\", \"\", \"\", `\ncapabilitiesUnknownkey:\n drop:\n - ALL\n add:\n - cap1\n - cap2\n# user www-data\nrunAsUser: 3333\n# group www-data\nrunAsGroup: 3333\nrunAsNonRoot: plop\nreadOnlyRootFilesystem: true\nallowPrivilegeEscalation: false`},\n\t\t\ttrue, nil,\n\t\t},\n\t\t{\n\t\t\t\"invalid securityContext secCtx wrong value type string instead of int\",\n\t\t\targs{\"\", \"\", \"\", `\ncapabilitiesUnknownkey:\n drop:\n - ALL\n add:\n - cap1\n - cap2\n# user www-data\nrunAsUser: plop\n# group www-data\nrunAsGroup: 3333\nrunAsNonRoot: true\nreadOnlyRootFilesystem: true\nallowPrivilegeEscalation: false`},\n\t\t\ttrue, nil,\n\t\t},\n\t\t{\"invalid security context runAsUser\", args{\"not a number\", \"\", \"\", \"\"}, true, nil},\n\t\t{\"invalid security context runAsGroup\", args{\"\", \"not a number\", \"\", \"\"}, true, nil},\n\t\t{\"invalid security context allowPrivilegeEscalation\", args{\"\", \"\", \"not a bool\", \"\"}, true, nil},\n\t}\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tgot, err := ParseSecurityContext(tt.args.runAsUser, tt.args.runAsGroup, tt.args.allowPrivilegeEscalation, tt.args.secCtx)\n\t\t\tif err != nil && tt.wantErr {\n\t\t\t\tassert.Error(t, err)\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NoError(t, err)\n\n\t\t\tif tt.expected == nil {\n\t\t\t\ttt.expected = &corev1api.SecurityContext{}\n\t\t\t}\n\n\t\t\tif tt.wantErr {\n\t\t\t\tassert.NotEqual(t, *tt.expected, got)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tassert.Equal(t, *tt.expected, got)\n\t\t})\n\t}\n}\n\nfunc pointInt64(i int64) *int64 {\n\treturn &i\n}\n" }, { "path": "pkg/util/kube/utils.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tapiextv1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1\"\n\tapiextv1beta1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/util/wait\"\n\t\"k8s.io/client-go/kubernetes\"\n\tcorev1client \"k8s.io/client-go/kubernetes/typed/core/v1\"\n\t\"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/label\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/filesystem\"\n)\n\n// These annotations are taken from the Kubernetes persistent volume/persistent volume claim controller.\n// They cannot be directly importing because they are part of the kubernetes/kubernetes package, and importing that package is unsupported.\n// Their values are well-known and slow changing. They're duplicated here as constants to provide compile-time checking.\n// Originals can be found in kubernetes/kubernetes/pkg/controller/volume/persistentvolume/util/util.go.\nconst (\n\tKubeAnnBindCompleted = \"pv.kubernetes.io/bind-completed\"\n\tKubeAnnBoundByController = \"pv.kubernetes.io/bound-by-controller\"\n\tKubeAnnDynamicallyProvisioned = \"pv.kubernetes.io/provisioned-by\"\n\tKubeAnnMigratedTo = \"pv.kubernetes.io/migrated-to\"\n\tKubeAnnSelectedNode = \"volume.kubernetes.io/selected-node\"\n)\n\n// VolumeSnapshotContentManagedByLabel is applied by the snapshot controller\n// to the VolumeSnapshotContent object in case distributed snapshotting is enabled.\n// The value contains the name of the node that handles the snapshot for the volume local to that node.\nconst VolumeSnapshotContentManagedByLabel = \"snapshot.storage.kubernetes.io/managed-by\"\n\nvar ErrorPodVolumeIsNotPVC = errors.New(\"pod volume is not a PVC\")\n\n// NamespaceAndName returns a string in the format /\nfunc NamespaceAndName(objMeta metav1.Object) string {\n\tif objMeta.GetNamespace() == \"\" {\n\t\treturn objMeta.GetName()\n\t}\n\treturn fmt.Sprintf(\"%s/%s\", objMeta.GetNamespace(), objMeta.GetName())\n}\n\n// EnsureNamespaceExistsAndIsReady attempts to create the provided Kubernetes namespace.\n// It returns three values:\n// - a bool indicating whether or not the namespace is ready,\n// - a bool indicating whether or not the namespace was created\n// - an error if one occurred.\n//\n// examples:\n//\n//\tnamespace already exists and is not ready, this function will return (false, false, nil).\n//\tIf the namespace exists and is marked for deletion, this function will wait up to the timeout for it to fully delete.\nfunc EnsureNamespaceExistsAndIsReady(namespace *corev1api.Namespace, client corev1client.NamespaceInterface, timeout time.Duration, resourceDeletionStatusTracker ResourceDeletionStatusTracker) (ready bool, nsCreated bool, err error) {\n\t// nsCreated tells whether the namespace was created by this method\n\t// required for keeping track of number of restored items\n\t// if namespace is marked for deletion, and we timed out, report an error\n\tvar terminatingNamespace bool\n\n\tvar namespaceAlreadyInDeletionTracker bool\n\n\terr = wait.PollUntilContextTimeout(context.Background(), time.Second, timeout, true, func(ctx context.Context) (bool, error) {\n\t\tclusterNS, err := client.Get(ctx, namespace.Name, metav1.GetOptions{})\n\t\t// if namespace is marked for deletion, and we timed out, report an error\n\n\t\tif apierrors.IsNotFound(err) {\n\t\t\t// Namespace isn't in cluster, we're good to create.\n\t\t\treturn true, nil\n\t\t}\n\n\t\tif err != nil {\n\t\t\t// Return the err and exit the loop.\n\t\t\treturn true, err\n\t\t}\n\t\tif clusterNS != nil && (clusterNS.GetDeletionTimestamp() != nil || clusterNS.Status.Phase == corev1api.NamespaceTerminating) {\n\t\t\tif resourceDeletionStatusTracker.Contains(clusterNS.Kind, clusterNS.Name, clusterNS.Name) {\n\t\t\t\tnamespaceAlreadyInDeletionTracker = true\n\t\t\t\treturn true, errors.Errorf(\"namespace %s is already present in the polling set, skipping execution\", namespace.Name)\n\t\t\t}\n\n\t\t\t// Marked for deletion, keep waiting\n\t\t\tterminatingNamespace = true\n\t\t\treturn false, nil\n\t\t}\n\n\t\t// clusterNS found, is not nil, and not marked for deletion, therefore we shouldn't create it.\n\t\tready = true\n\t\treturn true, nil\n\t})\n\n\t// err will be set if we timed out or encountered issues retrieving the namespace,\n\tif err != nil {\n\t\tif terminatingNamespace {\n\t\t\t// If the namespace is marked for deletion, and we timed out, adding it in tracker\n\t\t\tresourceDeletionStatusTracker.Add(namespace.Kind, namespace.Name, namespace.Name)\n\t\t\treturn false, nsCreated, errors.Wrapf(err, \"timed out waiting for terminating namespace %s to disappear before restoring\", namespace.Name)\n\t\t} else if namespaceAlreadyInDeletionTracker {\n\t\t\t// If the namespace is already in the tracker, return an error.\n\t\t\treturn false, nsCreated, errors.Wrapf(err, \"skipping polling for terminating namespace %s\", namespace.Name)\n\t\t}\n\t\treturn false, nsCreated, errors.Wrapf(err, \"error getting namespace %s\", namespace.Name)\n\t}\n\n\t// In the case the namespace already exists and isn't marked for deletion, assume it's ready for use.\n\tif ready {\n\t\treturn true, nsCreated, nil\n\t}\n\n\tclusterNS, err := client.Create(context.TODO(), namespace, metav1.CreateOptions{})\n\tif apierrors.IsAlreadyExists(err) {\n\t\tif clusterNS != nil && (clusterNS.GetDeletionTimestamp() != nil || clusterNS.Status.Phase == corev1api.NamespaceTerminating) {\n\t\t\t// Somehow created after all our polling and marked for deletion, return an error\n\t\t\treturn false, nsCreated, errors.Errorf(\"namespace %s created and marked for termination after timeout\", namespace.Name)\n\t\t}\n\t} else if err != nil {\n\t\treturn false, nsCreated, errors.Wrapf(err, \"error creating namespace %s\", namespace.Name)\n\t} else {\n\t\tnsCreated = true\n\t}\n\n\t// The namespace created successfully\n\treturn true, nsCreated, nil\n}\n\n// GetVolumeDirectory gets the name of the directory on the host, under /var/lib/kubelet/pods//volumes/,\n// where the specified volume lives.\n// For volumes with a CSIVolumeSource, append \"/mount\" to the directory name.\nfunc GetVolumeDirectory(ctx context.Context, log logrus.FieldLogger, pod *corev1api.Pod, volumeName string, kubeClient kubernetes.Interface) (string, error) {\n\tpvc, pv, volume, err := GetPodPVCVolume(ctx, log, pod, volumeName, kubeClient)\n\tif err != nil {\n\t\t// This case implies the administrator created the PV and attached it directly, without PVC.\n\t\t// Note that only one VolumeSource can be populated per Volume on a pod\n\t\tif err == ErrorPodVolumeIsNotPVC {\n\t\t\tif volume.VolumeSource.CSI != nil {\n\t\t\t\treturn volume.Name + \"/mount\", nil\n\t\t\t}\n\t\t\treturn volume.Name, nil\n\t\t}\n\t\treturn \"\", errors.WithStack(err)\n\t}\n\n\t// Most common case is that we have a PVC VolumeSource, and we need to check the PV it points to for a CSI source.\n\t// PV's been created with a CSI source.\n\tisProvisionedByCSI, err := isProvisionedByCSI(log, pv, kubeClient)\n\tif err != nil {\n\t\treturn \"\", errors.WithStack(err)\n\t}\n\tif isProvisionedByCSI {\n\t\tif pv.Spec.VolumeMode != nil && *pv.Spec.VolumeMode == corev1api.PersistentVolumeBlock {\n\t\t\treturn pvc.Spec.VolumeName, nil\n\t\t}\n\t\treturn pvc.Spec.VolumeName + \"/mount\", nil\n\t}\n\n\treturn pvc.Spec.VolumeName, nil\n}\n\n// GetVolumeMode gets the uploader.PersistentVolumeMode of the volume.\nfunc GetVolumeMode(ctx context.Context, log logrus.FieldLogger, pod *corev1api.Pod, volumeName string, kubeClient kubernetes.Interface) (\n\tuploader.PersistentVolumeMode, error) {\n\t_, pv, _, err := GetPodPVCVolume(ctx, log, pod, volumeName, kubeClient)\n\n\tif err != nil {\n\t\tif err == ErrorPodVolumeIsNotPVC {\n\t\t\treturn uploader.PersistentVolumeFilesystem, nil\n\t\t}\n\t\treturn \"\", errors.WithStack(err)\n\t}\n\n\tif pv.Spec.VolumeMode != nil && *pv.Spec.VolumeMode == corev1api.PersistentVolumeBlock {\n\t\treturn uploader.PersistentVolumeBlock, nil\n\t}\n\treturn uploader.PersistentVolumeFilesystem, nil\n}\n\n// GetPodPVCVolume gets the PVC, PV and volume for a pod volume name.\n// Returns pod volume in case of ErrorPodVolumeIsNotPVC error\nfunc GetPodPVCVolume(ctx context.Context, log logrus.FieldLogger, pod *corev1api.Pod, volumeName string, kubeClient kubernetes.Interface) (\n\t*corev1api.PersistentVolumeClaim, *corev1api.PersistentVolume, *corev1api.Volume, error) {\n\tvar volume *corev1api.Volume\n\n\tfor i := range pod.Spec.Volumes {\n\t\tif pod.Spec.Volumes[i].Name == volumeName {\n\t\t\tvolume = &pod.Spec.Volumes[i]\n\t\t\tbreak\n\t\t}\n\t}\n\n\tif volume == nil {\n\t\treturn nil, nil, nil, errors.New(\"volume not found in pod\")\n\t}\n\n\tif volume.VolumeSource.PersistentVolumeClaim == nil {\n\t\treturn nil, nil, volume, ErrorPodVolumeIsNotPVC // There is a pod volume but it is not a PVC\n\t}\n\n\tpvc, err := kubeClient.CoreV1().PersistentVolumeClaims(pod.Namespace).Get(ctx, volume.VolumeSource.PersistentVolumeClaim.ClaimName, metav1.GetOptions{})\n\tif err != nil {\n\t\treturn nil, nil, nil, errors.WithStack(err)\n\t}\n\n\tpv, err := kubeClient.CoreV1().PersistentVolumes().Get(ctx, pvc.Spec.VolumeName, metav1.GetOptions{})\n\tif err != nil {\n\t\treturn nil, nil, nil, errors.WithStack(err)\n\t}\n\n\treturn pvc, pv, volume, nil\n}\n\n// isProvisionedByCSI function checks whether this is a CSI PV by annotation.\n// Either \"pv.kubernetes.io/provisioned-by\" or \"pv.kubernetes.io/migrated-to\" indicates\n// PV is provisioned by CSI.\nfunc isProvisionedByCSI(log logrus.FieldLogger, pv *corev1api.PersistentVolume, kubeClient kubernetes.Interface) (bool, error) {\n\tif pv.Spec.CSI != nil {\n\t\treturn true, nil\n\t}\n\t// Although the pv.Spec.CSI is nil, the volume could be provisioned by a CSI driver when enabling the CSI migration\n\t// Refer to https://github.com/vmware-tanzu/velero/issues/4496 for more details\n\tif pv.Annotations != nil {\n\t\tdriverName := pv.Annotations[KubeAnnDynamicallyProvisioned]\n\t\tmigratedDriver := pv.Annotations[KubeAnnMigratedTo]\n\t\tif len(driverName) > 0 || len(migratedDriver) > 0 {\n\t\t\tlist, err := kubeClient.StorageV1().CSIDrivers().List(context.TODO(), metav1.ListOptions{})\n\t\t\tif err != nil {\n\t\t\t\treturn false, err\n\t\t\t}\n\n\t\t\tfor _, driver := range list.Items {\n\t\t\t\tif driverName == driver.Name || migratedDriver == driver.Name {\n\t\t\t\t\tlog.Debugf(\"the annotation %s or %s equals to %s indicates the volume is provisioned by a CSI driver\", KubeAnnDynamicallyProvisioned, KubeAnnMigratedTo, driver.Name)\n\t\t\t\t\treturn true, nil\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\treturn false, nil\n}\n\n// SinglePathMatch checks whether pass-in volume path is valid.\n// Check whether there is only one match by the path's pattern.\nfunc SinglePathMatch(path string, fs filesystem.Interface, log logrus.FieldLogger) (string, error) {\n\tmatches, err := fs.Glob(path)\n\tif err != nil {\n\t\treturn \"\", errors.WithStack(err)\n\t}\n\tif len(matches) != 1 {\n\t\treturn \"\", errors.Errorf(\"expected one matching path: %s, got %d\", path, len(matches))\n\t}\n\n\tlog.Debugf(\"This is a valid volume path: %s.\", matches[0])\n\treturn matches[0], nil\n}\n\n// IsV1CRDReady checks a v1 CRD to see if it's ready, with both the Established and NamesAccepted conditions.\nfunc IsV1CRDReady(crd *apiextv1.CustomResourceDefinition) bool {\n\tvar isEstablished, namesAccepted bool\n\tfor _, cond := range crd.Status.Conditions {\n\t\tif cond.Type == apiextv1.Established && cond.Status == apiextv1.ConditionTrue {\n\t\t\tisEstablished = true\n\t\t}\n\t\tif cond.Type == apiextv1.NamesAccepted && cond.Status == apiextv1.ConditionTrue {\n\t\t\tnamesAccepted = true\n\t\t}\n\t}\n\n\treturn (isEstablished && namesAccepted)\n}\n\n// IsV1Beta1CRDReady checks a v1beta1 CRD to see if it's ready, with both the Established and NamesAccepted conditions.\nfunc IsV1Beta1CRDReady(crd *apiextv1beta1.CustomResourceDefinition) bool {\n\tvar isEstablished, namesAccepted bool\n\tfor _, cond := range crd.Status.Conditions {\n\t\tif cond.Type == apiextv1beta1.Established && cond.Status == apiextv1beta1.ConditionTrue {\n\t\t\tisEstablished = true\n\t\t}\n\t\tif cond.Type == apiextv1beta1.NamesAccepted && cond.Status == apiextv1beta1.ConditionTrue {\n\t\t\tnamesAccepted = true\n\t\t}\n\t}\n\n\treturn (isEstablished && namesAccepted)\n}\n\n// IsCRDReady triggers IsV1Beta1CRDReady/IsV1CRDReady according to the version of the input param\nfunc IsCRDReady(crd *unstructured.Unstructured) (bool, error) {\n\tver := crd.GroupVersionKind().Version\n\tswitch ver {\n\tcase \"v1beta1\":\n\t\tv1beta1crd := &apiextv1beta1.CustomResourceDefinition{}\n\t\terr := runtime.DefaultUnstructuredConverter.FromUnstructured(crd.Object, v1beta1crd)\n\t\tif err != nil {\n\t\t\treturn false, err\n\t\t}\n\t\treturn IsV1Beta1CRDReady(v1beta1crd), nil\n\tcase \"v1\":\n\t\tv1crd := &apiextv1.CustomResourceDefinition{}\n\t\terr := runtime.DefaultUnstructuredConverter.FromUnstructured(crd.Object, v1crd)\n\t\tif err != nil {\n\t\t\treturn false, err\n\t\t}\n\t\treturn IsV1CRDReady(v1crd), nil\n\tdefault:\n\t\treturn false, fmt.Errorf(\"unable to handle CRD with version %s\", ver)\n\t}\n}\n\n// AddAnnotations adds the supplied key-values to the annotations on the object\nfunc AddAnnotations(o *metav1.ObjectMeta, vals map[string]string) {\n\tif o.Annotations == nil {\n\t\to.Annotations = make(map[string]string)\n\t}\n\tfor k, v := range vals {\n\t\to.Annotations[k] = v\n\t}\n}\n\n// AddLabels adds the supplied key-values to the labels on the object\nfunc AddLabels(o *metav1.ObjectMeta, vals map[string]string) {\n\tif o.Labels == nil {\n\t\to.Labels = make(map[string]string)\n\t}\n\tfor k, v := range vals {\n\t\to.Labels[k] = label.GetValidName(v)\n\t}\n}\n\nfunc HasBackupLabel(o *metav1.ObjectMeta, backupName string) bool {\n\tif o.Labels == nil || len(strings.TrimSpace(backupName)) == 0 {\n\t\treturn false\n\t}\n\treturn o.Labels[velerov1api.BackupNameLabel] == label.GetValidName(backupName)\n}\n\nfunc VerifyJSONConfigs(ctx context.Context, namespace string, crClient client.Client, configName string, configType any) error {\n\tcm := new(corev1api.ConfigMap)\n\terr := crClient.Get(ctx, client.ObjectKey{Namespace: namespace, Name: configName}, cm)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"fail to find ConfigMap %s\", configName)\n\t}\n\n\tif cm.Data == nil {\n\t\treturn errors.Errorf(\"data is not available in ConfigMap %s\", configName)\n\t}\n\n\t// Verify all the keys in ConfigMap's data.\n\tjsonString := \"\"\n\tfor _, v := range cm.Data {\n\t\tjsonString = v\n\n\t\tconfigs := configType\n\t\terr = json.Unmarshal([]byte(jsonString), configs)\n\t\tif err != nil {\n\t\t\treturn errors.Wrapf(err, \"error to unmarshall data from ConfigMap %s\", configName)\n\t\t}\n\t}\n\n\treturn nil\n}\n" }, { "path": "pkg/util/kube/utils_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage kube\n\nimport (\n\t\"encoding/json\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tstoragev1api \"k8s.io/api/storage/v1\"\n\tapiextv1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1\"\n\tapiextv1beta1 \"k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1\"\n\tapierrors \"k8s.io/apimachinery/pkg/api/errors\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\t\"k8s.io/apimachinery/pkg/runtime/schema\"\n\t\"k8s.io/client-go/kubernetes/fake\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n\t\"github.com/vmware-tanzu/velero/pkg/uploader\"\n)\n\nfunc TestNamespaceAndName(t *testing.T) {\n\t//TODO\n}\n\nfunc TestEnsureNamespaceExistsAndIsReady(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\texpectNSFound bool\n\t\tnsPhase corev1api.NamespacePhase\n\t\tnsDeleting bool\n\t\texpectCreate bool\n\t\talreadyExists bool\n\t\texpectedResult bool\n\t\texpectedCreatedResult bool\n\t\tnsAlreadyInTerminationTracker bool\n\t\tResourceDeletionStatusTracker ResourceDeletionStatusTracker\n\t}{\n\t\t{\n\t\t\tname: \"namespace found, not deleting\",\n\t\t\texpectNSFound: true,\n\t\t\texpectedResult: true,\n\t\t\texpectedCreatedResult: false,\n\t\t},\n\t\t{\n\t\t\tname: \"namespace found, terminating phase\",\n\t\t\texpectNSFound: true,\n\t\t\tnsPhase: corev1api.NamespaceTerminating,\n\t\t\texpectedResult: false,\n\t\t\texpectedCreatedResult: false,\n\t\t},\n\t\t{\n\t\t\tname: \"namespace found, deletiontimestamp set\",\n\t\t\texpectNSFound: true,\n\t\t\tnsDeleting: true,\n\t\t\texpectedResult: false,\n\t\t\texpectedCreatedResult: false,\n\t\t},\n\t\t{\n\t\t\tname: \"namespace not found, successfully created\",\n\t\t\texpectCreate: true,\n\t\t\texpectedResult: true,\n\t\t\texpectedCreatedResult: true,\n\t\t},\n\t\t{\n\t\t\tname: \"namespace not found initially, create returns already exists error, returned namespace is ready\",\n\t\t\talreadyExists: true,\n\t\t\texpectedResult: true,\n\t\t\texpectedCreatedResult: false,\n\t\t},\n\t\t{\n\t\t\tname: \"namespace not found initially, create returns already exists error, returned namespace is terminating\",\n\t\t\talreadyExists: true,\n\t\t\tnsPhase: corev1api.NamespaceTerminating,\n\t\t\texpectedResult: false,\n\t\t\texpectedCreatedResult: false,\n\t\t},\n\t\t{\n\t\t\tname: \"same namespace found earlier, terminating phase already tracked\",\n\t\t\texpectNSFound: true,\n\t\t\tnsPhase: corev1api.NamespaceTerminating,\n\t\t\texpectedResult: false,\n\t\t\texpectedCreatedResult: false,\n\t\t\tnsAlreadyInTerminationTracker: true,\n\t\t},\n\t}\n\n\tresourceDeletionStatusTracker := NewResourceDeletionStatusTracker()\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tnamespace := &corev1api.Namespace{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tName: \"test\",\n\t\t\t\t},\n\t\t\t}\n\n\t\t\tif test.nsPhase != \"\" {\n\t\t\t\tnamespace.Status.Phase = test.nsPhase\n\t\t\t}\n\n\t\t\tif test.nsDeleting {\n\t\t\t\tnamespace.SetDeletionTimestamp(&metav1.Time{Time: time.Now()})\n\t\t\t}\n\n\t\t\ttimeout := time.Millisecond\n\n\t\t\tnsClient := &velerotest.FakeNamespaceClient{}\n\t\t\tdefer nsClient.AssertExpectations(t)\n\n\t\t\tif test.expectNSFound {\n\t\t\t\tnsClient.On(\"Get\", \"test\", metav1.GetOptions{}).Return(namespace, nil)\n\t\t\t} else {\n\t\t\t\tnsClient.On(\"Get\", \"test\", metav1.GetOptions{}).Return(&corev1api.Namespace{}, apierrors.NewNotFound(schema.GroupResource{Resource: \"namespaces\"}, \"test\"))\n\t\t\t}\n\n\t\t\tif test.alreadyExists {\n\t\t\t\tnsClient.On(\"Create\", namespace).Return(namespace, apierrors.NewAlreadyExists(schema.GroupResource{Resource: \"namespaces\"}, \"test\"))\n\t\t\t}\n\n\t\t\tif test.expectCreate {\n\t\t\t\tnsClient.On(\"Create\", namespace).Return(namespace, nil)\n\t\t\t}\n\n\t\t\tif test.nsAlreadyInTerminationTracker {\n\t\t\t\tresourceDeletionStatusTracker.Add(namespace.Kind, \"test\", \"test\")\n\t\t\t}\n\n\t\t\tresult, nsCreated, _ := EnsureNamespaceExistsAndIsReady(namespace, nsClient, timeout, resourceDeletionStatusTracker)\n\n\t\t\tassert.Equal(t, test.expectedResult, result)\n\t\t\tassert.Equal(t, test.expectedCreatedResult, nsCreated)\n\t\t})\n\t}\n}\n\n// TestGetVolumeDirectorySuccess tests that the GetVolumeDirectory function\n// returns a volume's name or a volume's name plus '/mount' when a PVC is present.\nfunc TestGetVolumeDirectorySuccess(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpod *corev1api.Pod\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\tpv *corev1api.PersistentVolume\n\t\twant string\n\t}{\n\t\t{\n\t\t\tname: \"Non-CSI volume with a PVC/PV returns the volume's name\",\n\t\t\tpod: builder.ForPod(\"ns-1\", \"my-pod\").Volumes(builder.ForVolume(\"my-vol\").PersistentVolumeClaimSource(\"my-pvc\").Result()).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns-1\", \"my-pvc\").VolumeName(\"a-pv\").Result(),\n\t\t\tpv: builder.ForPersistentVolume(\"a-pv\").Result(),\n\t\t\twant: \"a-pv\",\n\t\t},\n\t\t{\n\t\t\tname: \"CSI volume with a PVC/PV appends '/mount' to the volume name\",\n\t\t\tpod: builder.ForPod(\"ns-1\", \"my-pod\").Volumes(builder.ForVolume(\"my-vol\").PersistentVolumeClaimSource(\"my-pvc\").Result()).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns-1\", \"my-pvc\").VolumeName(\"a-pv\").Result(),\n\t\t\tpv: builder.ForPersistentVolume(\"a-pv\").CSI(\"csi.test.com\", \"provider-volume-id\").Result(),\n\t\t\twant: \"a-pv/mount\",\n\t\t},\n\t\t{\n\t\t\tname: \"Block CSI volume with a PVC/PV does not append '/mount' to the volume name\",\n\t\t\tpod: builder.ForPod(\"ns-1\", \"my-pod\").Volumes(builder.ForVolume(\"my-vol\").PersistentVolumeClaimSource(\"my-pvc\").Result()).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns-1\", \"my-pvc\").VolumeName(\"a-pv\").Result(),\n\t\t\tpv: builder.ForPersistentVolume(\"a-pv\").CSI(\"csi.test.com\", \"provider-volume-id\").VolumeMode(corev1api.PersistentVolumeBlock).Result(),\n\t\t\twant: \"a-pv\",\n\t\t},\n\t\t{\n\t\t\tname: \"CSI volume mounted without a PVC appends '/mount' to the volume name\",\n\t\t\tpod: builder.ForPod(\"ns-1\", \"my-pod\").Volumes(builder.ForVolume(\"my-vol\").CSISource(\"csi.test.com\").Result()).Result(),\n\t\t\twant: \"my-vol/mount\",\n\t\t},\n\t\t{\n\t\t\tname: \"Non-CSI volume without a PVC returns the volume name\",\n\t\t\tpod: builder.ForPod(\"ns-1\", \"my-pod\").Volumes(builder.ForVolume(\"my-vol\").Result()).Result(),\n\t\t\twant: \"my-vol\",\n\t\t},\n\t\t{\n\t\t\tname: \"Volume with CSI annotation appends '/mount' to the volume name\",\n\t\t\tpod: builder.ForPod(\"ns-1\", \"my-pod\").Volumes(builder.ForVolume(\"my-vol\").PersistentVolumeClaimSource(\"my-pvc\").Result()).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns-1\", \"my-pvc\").VolumeName(\"a-pv\").Result(),\n\t\t\tpv: builder.ForPersistentVolume(\"a-pv\").ObjectMeta(builder.WithAnnotations(KubeAnnDynamicallyProvisioned, \"csi.test.com\")).Result(),\n\t\t\twant: \"a-pv/mount\",\n\t\t},\n\t\t{\n\t\t\tname: \"Volume with CSI annotation 'pv.kubernetes.io/migrated-to' appends '/mount' to the volume name\",\n\t\t\tpod: builder.ForPod(\"ns-1\", \"my-pod\").Volumes(builder.ForVolume(\"my-vol\").PersistentVolumeClaimSource(\"my-pvc\").Result()).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns-1\", \"my-pvc\").VolumeName(\"a-pv\").Result(),\n\t\t\tpv: builder.ForPersistentVolume(\"a-pv\").ObjectMeta(builder.WithAnnotations(KubeAnnMigratedTo, \"csi.test.com\")).Result(),\n\t\t\twant: \"a-pv/mount\",\n\t\t},\n\t}\n\n\tcsiDriver := storagev1api.CSIDriver{\n\t\tObjectMeta: metav1.ObjectMeta{Name: \"csi.test.com\"},\n\t}\n\tfor _, tc := range tests {\n\t\tobjs := []runtime.Object{&csiDriver}\n\t\tif tc.pvc != nil {\n\t\t\tobjs = append(objs, tc.pvc)\n\t\t}\n\t\tif tc.pv != nil {\n\t\t\tobjs = append(objs, tc.pv)\n\t\t}\n\n\t\tfakeKubeClient := fake.NewSimpleClientset(objs...)\n\n\t\t// Function under test\n\t\tdir, err := GetVolumeDirectory(t.Context(), logrus.StandardLogger(), tc.pod, tc.pod.Spec.Volumes[0].Name, fakeKubeClient)\n\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, tc.want, dir)\n\t}\n}\n\n// TestGetVolumeModeSuccess tests the GetVolumeMode function\nfunc TestGetVolumeModeSuccess(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpod *corev1api.Pod\n\t\tpvc *corev1api.PersistentVolumeClaim\n\t\tpv *corev1api.PersistentVolume\n\t\twant uploader.PersistentVolumeMode\n\t}{\n\t\t{\n\t\t\tname: \"Filesystem PVC volume\",\n\t\t\tpod: builder.ForPod(\"ns-1\", \"my-pod\").Volumes(builder.ForVolume(\"my-vol\").PersistentVolumeClaimSource(\"my-pvc\").Result()).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns-1\", \"my-pvc\").VolumeName(\"a-pv\").Result(),\n\t\t\tpv: builder.ForPersistentVolume(\"a-pv\").VolumeMode(corev1api.PersistentVolumeFilesystem).Result(),\n\t\t\twant: uploader.PersistentVolumeFilesystem,\n\t\t},\n\t\t{\n\t\t\tname: \"Block PVC volume\",\n\t\t\tpod: builder.ForPod(\"ns-1\", \"my-pod\").Volumes(builder.ForVolume(\"my-vol\").PersistentVolumeClaimSource(\"my-pvc\").Result()).Result(),\n\t\t\tpvc: builder.ForPersistentVolumeClaim(\"ns-1\", \"my-pvc\").VolumeName(\"a-pv\").Result(),\n\t\t\tpv: builder.ForPersistentVolume(\"a-pv\").VolumeMode(corev1api.PersistentVolumeBlock).Result(),\n\t\t\twant: uploader.PersistentVolumeBlock,\n\t\t},\n\t\t{\n\t\t\tname: \"Pod volume without a PVC\",\n\t\t\tpod: builder.ForPod(\"ns-1\", \"my-pod\").Volumes(builder.ForVolume(\"my-vol\").Result()).Result(),\n\t\t\twant: uploader.PersistentVolumeFilesystem,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tobjs := []runtime.Object{}\n\t\tif tc.pvc != nil {\n\t\t\tobjs = append(objs, tc.pvc)\n\t\t}\n\t\tif tc.pv != nil {\n\t\t\tobjs = append(objs, tc.pv)\n\t\t}\n\n\t\tfakeKubeClient := fake.NewSimpleClientset(objs...)\n\n\t\t// Function under test\n\t\tmode, err := GetVolumeMode(t.Context(), logrus.StandardLogger(), tc.pod, tc.pod.Spec.Volumes[0].Name, fakeKubeClient)\n\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, tc.want, mode)\n\t}\n}\n\nfunc TestIsV1Beta1CRDReady(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tcrd *apiextv1beta1.CustomResourceDefinition\n\t\twant bool\n\t}{\n\t\t{\n\t\t\tname: \"CRD is not established & not accepting names - not ready\",\n\t\t\tcrd: builder.ForCustomResourceDefinitionV1Beta1(\"MyCRD\").Result(),\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"CRD is established & not accepting names - not ready\",\n\t\t\tcrd: builder.ForCustomResourceDefinitionV1Beta1(\"MyCRD\").\n\t\t\t\tCondition(builder.ForCustomResourceDefinitionV1Beta1Condition().Type(apiextv1beta1.Established).Status(apiextv1beta1.ConditionTrue).Result()).Result(),\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"CRD is not established & accepting names - not ready\",\n\t\t\tcrd: builder.ForCustomResourceDefinitionV1Beta1(\"MyCRD\").\n\t\t\t\tCondition(builder.ForCustomResourceDefinitionV1Beta1Condition().Type(apiextv1beta1.NamesAccepted).Status(apiextv1beta1.ConditionTrue).Result()).Result(),\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"CRD is established & accepting names - ready\",\n\t\t\tcrd: builder.ForCustomResourceDefinitionV1Beta1(\"MyCRD\").\n\t\t\t\tCondition(builder.ForCustomResourceDefinitionV1Beta1Condition().Type(apiextv1beta1.Established).Status(apiextv1beta1.ConditionTrue).Result()).\n\t\t\t\tCondition(builder.ForCustomResourceDefinitionV1Beta1Condition().Type(apiextv1beta1.NamesAccepted).Status(apiextv1beta1.ConditionTrue).Result()).\n\t\t\t\tResult(),\n\t\t\twant: true,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tresult := IsV1Beta1CRDReady(tc.crd)\n\t\tassert.Equal(t, tc.want, result)\n\t}\n}\n\nfunc TestIsV1CRDReady(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tcrd *apiextv1.CustomResourceDefinition\n\t\twant bool\n\t}{\n\t\t{\n\t\t\tname: \"CRD is not established & not accepting names - not ready\",\n\t\t\tcrd: builder.ForV1CustomResourceDefinition(\"MyCRD\").Result(),\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"CRD is established & not accepting names - not ready\",\n\t\t\tcrd: builder.ForV1CustomResourceDefinition(\"MyCRD\").\n\t\t\t\tCondition(builder.ForV1CustomResourceDefinitionCondition().Type(apiextv1.Established).Status(apiextv1.ConditionTrue).Result()).Result(),\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"CRD is not established & accepting names - not ready\",\n\t\t\tcrd: builder.ForV1CustomResourceDefinition(\"MyCRD\").\n\t\t\t\tCondition(builder.ForV1CustomResourceDefinitionCondition().Type(apiextv1.NamesAccepted).Status(apiextv1.ConditionTrue).Result()).Result(),\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"CRD is established & accepting names - ready\",\n\t\t\tcrd: builder.ForV1CustomResourceDefinition(\"MyCRD\").\n\t\t\t\tCondition(builder.ForV1CustomResourceDefinitionCondition().Type(apiextv1.Established).Status(apiextv1.ConditionTrue).Result()).\n\t\t\t\tCondition(builder.ForV1CustomResourceDefinitionCondition().Type(apiextv1.NamesAccepted).Status(apiextv1.ConditionTrue).Result()).\n\t\t\t\tResult(),\n\t\t\twant: true,\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tresult := IsV1CRDReady(tc.crd)\n\t\tassert.Equal(t, tc.want, result)\n\t}\n}\n\nfunc TestIsCRDReady(t *testing.T) {\n\tv1beta1tests := []struct {\n\t\tname string\n\t\tcrd *apiextv1beta1.CustomResourceDefinition\n\t\twant bool\n\t}{\n\t\t{\n\t\t\tname: \"v1beta1CRD is not established & not accepting names - not ready\",\n\t\t\tcrd: builder.ForCustomResourceDefinitionV1Beta1(\"MyCRD\").Result(),\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"v1beta1CRD is established & not accepting names - not ready\",\n\t\t\tcrd: builder.ForCustomResourceDefinitionV1Beta1(\"MyCRD\").\n\t\t\t\tCondition(builder.ForCustomResourceDefinitionV1Beta1Condition().Type(apiextv1beta1.Established).Status(apiextv1beta1.ConditionTrue).Result()).Result(),\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"v1beta1CRD is not established & accepting names - not ready\",\n\t\t\tcrd: builder.ForCustomResourceDefinitionV1Beta1(\"MyCRD\").\n\t\t\t\tCondition(builder.ForCustomResourceDefinitionV1Beta1Condition().Type(apiextv1beta1.NamesAccepted).Status(apiextv1beta1.ConditionTrue).Result()).Result(),\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"v1beta1CRD is established & accepting names - ready\",\n\t\t\tcrd: builder.ForCustomResourceDefinitionV1Beta1(\"MyCRD\").\n\t\t\t\tCondition(builder.ForCustomResourceDefinitionV1Beta1Condition().Type(apiextv1beta1.Established).Status(apiextv1beta1.ConditionTrue).Result()).\n\t\t\t\tCondition(builder.ForCustomResourceDefinitionV1Beta1Condition().Type(apiextv1beta1.NamesAccepted).Status(apiextv1beta1.ConditionTrue).Result()).\n\t\t\t\tResult(),\n\t\t\twant: true,\n\t\t},\n\t}\n\n\tfor _, tc := range v1beta1tests {\n\t\tm, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.crd)\n\t\trequire.NoError(t, err)\n\t\tresult, err := IsCRDReady(&unstructured.Unstructured{Object: m})\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, tc.want, result)\n\t}\n\n\tv1tests := []struct {\n\t\tname string\n\t\tcrd *apiextv1.CustomResourceDefinition\n\t\twant bool\n\t}{\n\t\t{\n\t\t\tname: \"v1CRD is not established & not accepting names - not ready\",\n\t\t\tcrd: builder.ForV1CustomResourceDefinition(\"MyCRD\").Result(),\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"v1CRD is established & not accepting names - not ready\",\n\t\t\tcrd: builder.ForV1CustomResourceDefinition(\"MyCRD\").\n\t\t\t\tCondition(builder.ForV1CustomResourceDefinitionCondition().Type(apiextv1.Established).Status(apiextv1.ConditionTrue).Result()).Result(),\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"v1CRD is not established & accepting names - not ready\",\n\t\t\tcrd: builder.ForV1CustomResourceDefinition(\"MyCRD\").\n\t\t\t\tCondition(builder.ForV1CustomResourceDefinitionCondition().Type(apiextv1.NamesAccepted).Status(apiextv1.ConditionTrue).Result()).Result(),\n\t\t\twant: false,\n\t\t},\n\t\t{\n\t\t\tname: \"v1CRD is established & accepting names - ready\",\n\t\t\tcrd: builder.ForV1CustomResourceDefinition(\"MyCRD\").\n\t\t\t\tCondition(builder.ForV1CustomResourceDefinitionCondition().Type(apiextv1.Established).Status(apiextv1.ConditionTrue).Result()).\n\t\t\t\tCondition(builder.ForV1CustomResourceDefinitionCondition().Type(apiextv1.NamesAccepted).Status(apiextv1.ConditionTrue).Result()).\n\t\t\t\tResult(),\n\t\t\twant: true,\n\t\t},\n\t}\n\n\tfor _, tc := range v1tests {\n\t\tm, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.crd)\n\t\trequire.NoError(t, err)\n\t\tresult, err := IsCRDReady(&unstructured.Unstructured{Object: m})\n\t\trequire.NoError(t, err)\n\t\tassert.Equal(t, tc.want, result)\n\t}\n\n\t// input param is unrecognized\n\tresBytes := []byte(`\n{\n\t\"apiVersion\": \"apiextensions.k8s.io/v9\",\n\t\"kind\": \"CustomResourceDefinition\",\n\t\"metadata\": {\n\t\t\"name\": \"foos.example.foo.com\"\n\t},\n\t\"spec\": {\n\t\t\"group\": \"example.foo.com\",\n\t\t\"version\": \"v1alpha1\",\n\t\t\"scope\": \"Namespaced\",\n\t\t\"names\": {\n\t\t\t\"plural\": \"foos\",\n\t\t\t\"singular\": \"foo\",\n\t\t\t\"kind\": \"Foo\"\n\t\t},\n\t\t\"validation\": {\n\t\t\t\"openAPIV3Schema\": {\n\t\t\t\t\"required\": [\n\t\t\t\t\t\"spec\"\n\t\t\t\t],\n\t\t\t\t\"properties\": {\n\t\t\t\t\t\"spec\": {\n\t\t\t\t\t\t\"required\": [\n\t\t\t\t\t\t\t\"bar\"\n\t\t\t\t\t\t],\n\t\t\t\t\t\t\"properties\": {\n\t\t\t\t\t\t\t\"bar\": {\n\t\t\t\t\t\t\t\t\"type\": \"integer\",\n\t\t\t\t\t\t\t\t\"minimum\": 1\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n}\n`)\n\tobj := &unstructured.Unstructured{}\n\terr := json.Unmarshal(resBytes, obj)\n\trequire.NoError(t, err)\n\t_, err = IsCRDReady(obj)\n\tassert.Error(t, err)\n}\n\nfunc TestSinglePathMatch(t *testing.T) {\n\tfakeFS := velerotest.NewFakeFileSystem()\n\tfakeFS.MkdirAll(\"testDir1/subpath\", 0755)\n\tfakeFS.MkdirAll(\"testDir2/subpath\", 0755)\n\n\t_, err := SinglePathMatch(\"./*/subpath\", fakeFS, logrus.StandardLogger())\n\trequire.ErrorContains(t, err, \"expected one matching path\")\n}\n\nfunc TestAddAnnotations(t *testing.T) {\n\tannotationValues := map[string]string{\n\t\t\"k1\": \"v1\",\n\t\t\"k2\": \"v2\",\n\t\t\"k3\": \"v3\",\n\t\t\"k4\": \"v4\",\n\t\t\"k5\": \"v5\",\n\t}\n\ttestCases := []struct {\n\t\tname string\n\t\to metav1.ObjectMeta\n\t\ttoAdd map[string]string\n\t}{\n\t\t{\n\t\t\tname: \"should create a new annotation map when annotation is nil\",\n\t\t\to: metav1.ObjectMeta{\n\t\t\t\tAnnotations: nil,\n\t\t\t},\n\t\t\ttoAdd: annotationValues,\n\t\t},\n\t\t{\n\t\t\tname: \"should add all supplied annotations into empty annotation\",\n\t\t\to: metav1.ObjectMeta{\n\t\t\t\tAnnotations: map[string]string{},\n\t\t\t},\n\t\t\ttoAdd: annotationValues,\n\t\t},\n\t\t{\n\t\t\tname: \"should add all supplied annotations to existing annotation\",\n\t\t\to: metav1.ObjectMeta{\n\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\"k100\": \"v100\",\n\t\t\t\t\t\"k200\": \"v200\",\n\t\t\t\t\t\"k300\": \"v300\",\n\t\t\t\t},\n\t\t\t},\n\t\t\ttoAdd: annotationValues,\n\t\t},\n\t\t{\n\t\t\tname: \"should overwrite some existing annotations\",\n\t\t\to: metav1.ObjectMeta{\n\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\"k100\": \"v100\",\n\t\t\t\t\t\"k2\": \"v200\",\n\t\t\t\t\t\"k300\": \"v300\",\n\t\t\t\t},\n\t\t\t},\n\t\t\ttoAdd: annotationValues,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tAddAnnotations(&tc.o, tc.toAdd)\n\t\t\tfor k, v := range tc.toAdd {\n\t\t\t\tactual, exists := tc.o.Annotations[k]\n\t\t\t\tassert.True(t, exists)\n\t\t\t\tassert.Equal(t, v, actual)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestAddLabels(t *testing.T) {\n\tlabelValues := map[string]string{\n\t\t\"l1\": \"v1\",\n\t\t\"l2\": \"v2\",\n\t\t\"l3\": \"v3\",\n\t\t\"l4\": \"v4\",\n\t\t\"l5\": \"v5\",\n\t}\n\ttestCases := []struct {\n\t\tname string\n\t\to metav1.ObjectMeta\n\t\ttoAdd map[string]string\n\t}{\n\t\t{\n\t\t\tname: \"should create a new labels map when labels is nil\",\n\t\t\to: metav1.ObjectMeta{\n\t\t\t\tLabels: nil,\n\t\t\t},\n\t\t\ttoAdd: labelValues,\n\t\t},\n\t\t{\n\t\t\tname: \"should add all supplied labels into empty labels\",\n\t\t\to: metav1.ObjectMeta{\n\t\t\t\tLabels: map[string]string{},\n\t\t\t},\n\t\t\ttoAdd: labelValues,\n\t\t},\n\t\t{\n\t\t\tname: \"should add all supplied labels to existing labels\",\n\t\t\to: metav1.ObjectMeta{\n\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\"l100\": \"v100\",\n\t\t\t\t\t\"l200\": \"v200\",\n\t\t\t\t\t\"l300\": \"v300\",\n\t\t\t\t},\n\t\t\t},\n\t\t\ttoAdd: labelValues,\n\t\t},\n\t\t{\n\t\t\tname: \"should overwrite some existing labels\",\n\t\t\to: metav1.ObjectMeta{\n\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\"l100\": \"v100\",\n\t\t\t\t\t\"l2\": \"v200\",\n\t\t\t\t\t\"l300\": \"v300\",\n\t\t\t\t},\n\t\t\t},\n\t\t\ttoAdd: labelValues,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tAddLabels(&tc.o, tc.toAdd)\n\t\t\tfor k, v := range tc.toAdd {\n\t\t\t\tactual, exists := tc.o.Labels[k]\n\t\t\t\tassert.True(t, exists)\n\t\t\t\tassert.Equal(t, v, actual)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestHasBackupLabel(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\to metav1.ObjectMeta\n\t\tbackupName string\n\t\texpected bool\n\t}{\n\t\t{\n\t\t\tname: \"object has no labels\",\n\t\t\to: metav1.ObjectMeta{},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"object has no velero backup label\",\n\t\t\tbackupName: \"csi-b1\",\n\t\t\to: metav1.ObjectMeta{\n\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\"l100\": \"v100\",\n\t\t\t\t\t\"l2\": \"v200\",\n\t\t\t\t\t\"l300\": \"v300\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"object has velero backup label but value not equal to backup name\",\n\t\t\tbackupName: \"csi-b1\",\n\t\t\to: metav1.ObjectMeta{\n\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\"velero.io/backup-name\": \"does-not-match\",\n\t\t\t\t\t\"l100\": \"v100\",\n\t\t\t\t\t\"l2\": \"v200\",\n\t\t\t\t\t\"l300\": \"v300\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"object has backup label with matching backup name value\",\n\t\t\tbackupName: \"does-match\",\n\t\t\to: metav1.ObjectMeta{\n\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\"velero.io/backup-name\": \"does-match\",\n\t\t\t\t\t\"l100\": \"v100\",\n\t\t\t\t\t\"l2\": \"v200\",\n\t\t\t\t\t\"l300\": \"v300\",\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: true,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tactual := HasBackupLabel(&tc.o, tc.backupName)\n\t\tassert.Equal(t, tc.expected, actual)\n\t}\n}\n\nfunc TestVerifyJsonConfigs(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tconfigMapName string\n\t\tconfigMap *corev1api.ConfigMap\n\t\tconfigType any\n\t\texpectedErr string\n\t}{\n\t\t{\n\t\t\tname: \"ConfigMap not exist\",\n\t\t\tconfigMapName: \"non-exist\",\n\t\t\texpectedErr: \"fail to find ConfigMap non-exist: configmaps \\\"non-exist\\\" not found\",\n\t\t},\n\t\t{\n\t\t\tname: \"ConfigMap doesn't have data\",\n\t\t\tconfigMapName: \"no-data\",\n\t\t\texpectedErr: \"data is not available in ConfigMap no-data\",\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"no-data\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"ConfigMap data is invalid\",\n\t\t\tconfigMapName: \"invalid\",\n\t\t\texpectedErr: \"error to unmarshall data from ConfigMap invalid: unexpected end of JSON input\",\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"invalid\").Data(\"global\", \"{\\\"podResources\\\": {\\\"cpuRequest\\\": \\\"100m\\\", \\\"cpuLimit\\\": \\\"200m\\\", \\\"memoryRequest\\\": \\\"100Mi\\\", \\\"memoryLimit\\\": \\\"200Mi\\\"}, \\\"keepLatestMaintenanceJobs\\\": 1}\", \"other\", \"{\\\"podResources\\\": {\\\"cpuRequest\\\": \\\"100m\\\", \\\"cpuLimit\\\": \\\"200m\\\", \\\"memoryRequest\\\": \\\"100Mi\\\", \\\"memoryLimit\\\": \\\"200Mi\\\"}, \\\"keepLatestMaintenanceJobs: 1}\").Result(),\n\t\t},\n\t\t{\n\t\t\tname: \"Normal case\",\n\t\t\tconfigMapName: \"normal\",\n\t\t\tconfigMap: builder.ForConfigMap(\"velero\", \"normal\").Data(\"global\", \"{\\\"podResources\\\": {\\\"cpuRequest\\\": \\\"100m\\\", \\\"cpuLimit\\\": \\\"200m\\\", \\\"memoryRequest\\\": \\\"100Mi\\\", \\\"memoryLimit\\\": \\\"200Mi\\\"}, \\\"keepLatestMaintenanceJobs\\\": 1}\", \"other\", \"{\\\"podResources\\\": {\\\"cpuRequest\\\": \\\"100m\\\", \\\"cpuLimit\\\": \\\"200m\\\", \\\"memoryRequest\\\": \\\"100Mi\\\", \\\"memoryLimit\\\": \\\"200Mi\\\"}, \\\"keepLatestMaintenanceJobs\\\": 1}\").Result(),\n\t\t\tconfigType: make(map[string]any),\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tobjects := make([]runtime.Object, 0)\n\t\t\tif tc.configMap != nil {\n\t\t\t\tobjects = append(objects, tc.configMap)\n\t\t\t}\n\t\t\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, objects...)\n\t\t\terr := VerifyJSONConfigs(t.Context(), \"velero\", fakeClient, tc.configMapName, tc.configMap)\n\t\t\tif len(tc.expectedErr) > 0 {\n\t\t\t\trequire.EqualError(t, err, tc.expectedErr)\n\t\t\t} else {\n\t\t\t\trequire.NoError(t, err)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/util/logging/default_logger.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage logging\n\nimport (\n\t\"os\"\n\n\t\"github.com/sirupsen/logrus\"\n)\n\n// DefaultHooks returns a slice of the default\n// logrus hooks to be used by a logger.\nfunc DefaultHooks(merge bool) []logrus.Hook {\n\thooks := []logrus.Hook{\n\t\t&LogLocationHook{},\n\t\t&ErrorLocationHook{},\n\t}\n\n\tif merge {\n\t\thooks = append(hooks, &MergeHook{})\n\t}\n\n\treturn hooks\n}\n\n// DefaultLogger returns a Logger with the default properties\n// and hooks. The desired output format is passed as a LogFormat Enum.\nfunc DefaultLogger(level logrus.Level, format Format) *logrus.Logger {\n\treturn createLogger(level, format, false)\n}\n\n// DefaultLogger returns a Logger with the default properties\n// and hooks, and also a hook to support log merge.\n// The desired output format is passed as a LogFormat Enum.\nfunc DefaultMergeLogger(level logrus.Level, format Format) *logrus.Logger {\n\treturn createLogger(level, format, true)\n}\n\nfunc createLogger(level logrus.Level, format Format, merge bool) *logrus.Logger {\n\tlogger := logrus.New()\n\n\tif format == FormatJSON {\n\t\tlogger.Formatter = new(logrus.JSONFormatter)\n\t\t// Error hooks inject nested fields under \"error.*\" with the error\n\t\t// string message at \"error\".\n\t\t//\n\t\t// This structure is incompatible with recent Elasticsearch versions\n\t\t// where dots in field names are automatically expanded to objects;\n\t\t// field \"error\" cannot be both a string and an object at the same\n\t\t// time.\n\t\t//\n\t\t// ELK being a popular choice for log ingestion and a common reason\n\t\t// for enabling JSON logging in the first place, we avoid this mapping\n\t\t// problem by nesting the error's message at \"error.message\".\n\t\t//\n\t\t// This also follows the Elastic Common Schema (ECS) recommendation.\n\t\t// https://www.elastic.co/guide/en/ecs/current/ecs-error.html\n\t\tlogrus.ErrorKey = \"error.message\"\n\t} else {\n\t\tlogrus.ErrorKey = \"error\"\n\t}\n\n\t// Make sure the output is set to stdout so log messages don't show up as errors in cloud log dashboards.\n\tlogger.Out = os.Stdout\n\n\tlogger.Level = level\n\n\tfor _, hook := range DefaultHooks(merge) {\n\t\tlogger.Hooks.Add(hook)\n\t}\n\n\treturn logger\n}\n" }, { "path": "pkg/util/logging/default_logger_test.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage logging\n\nimport (\n\t\"os\"\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestDefaultLogger(t *testing.T) {\n\tformatFlag := NewFormatFlag()\n\n\tfor _, testFormat := range formatFlag.AllowedValues() {\n\t\tformatFlag.Set(testFormat)\n\t\tlogger := DefaultLogger(logrus.InfoLevel, formatFlag.Parse())\n\t\tassert.Equal(t, logrus.InfoLevel, logger.Level)\n\t\tassert.Equal(t, os.Stdout, logger.Out)\n\n\t\tfor _, level := range logrus.AllLevels {\n\t\t\tassert.Equal(t, DefaultHooks(false), logger.Hooks[level])\n\t\t}\n\t}\n}\n\nfunc TestDefaultMergeLogger(t *testing.T) {\n\tformatFlag := NewFormatFlag()\n\n\tfor _, testFormat := range formatFlag.AllowedValues() {\n\t\tformatFlag.Set(testFormat)\n\t\tlogger := DefaultMergeLogger(logrus.InfoLevel, formatFlag.Parse())\n\t\tassert.Equal(t, logrus.InfoLevel, logger.Level)\n\t\tassert.Equal(t, os.Stdout, logger.Out)\n\n\t\tassert.Equal(t, DefaultHooks(true), logger.Hooks[ListeningLevel])\n\t}\n}\n" }, { "path": "pkg/util/logging/dual_mode_logger.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage logging\n\nimport (\n\t\"compress/gzip\"\n\t\"io\"\n\t\"os\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n)\n\n// DualModeLogger is a thread safe logger interface to write logs to dual targets, one of which\n// is a persist file, so that the log could be further transferred.\ntype DualModeLogger interface {\n\tlogrus.FieldLogger\n\t// DoneForPersist stops outputting logs to the persist file\n\tDoneForPersist(log logrus.FieldLogger)\n\t// GetPersistFile moves the persist file pointer to beginning and returns it\n\tGetPersistFile() (*os.File, error)\n\t// Dispose closes the temp file pointer and removes the file\n\tDispose(log logrus.FieldLogger)\n}\n\ntype tempFileLogger struct {\n\tlogrus.FieldLogger\n\tlogger *logrus.Logger\n\tfile *os.File\n\tw *gzip.Writer\n}\n\n// NewTempFileLogger creates a DualModeLogger instance that writes logs to both Stdout and a file in the temp folder.\nfunc NewTempFileLogger(logLevel logrus.Level, logFormat Format, hook *LogHook, fields logrus.Fields) (DualModeLogger, error) {\n\tfile, err := os.CreateTemp(\"\", \"\")\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"error creating temp file\")\n\t}\n\n\tw := gzip.NewWriter(file)\n\n\tlogger := DefaultLogger(logLevel, logFormat)\n\tlogger.Out = io.MultiWriter(os.Stdout, w)\n\n\tif hook != nil {\n\t\tlogger.Hooks.Add(hook)\n\t}\n\n\treturn &tempFileLogger{\n\t\tFieldLogger: logger.WithFields(fields),\n\t\tlogger: logger,\n\t\tfile: file,\n\t\tw: w,\n\t}, nil\n}\n\nfunc (p *tempFileLogger) DoneForPersist(log logrus.FieldLogger) {\n\tp.logger.SetOutput(os.Stdout)\n\n\tif err := p.w.Close(); err != nil {\n\t\tlog.WithError(err).Warn(\"error closing gzip writer\")\n\t}\n}\n\nfunc (p *tempFileLogger) GetPersistFile() (*os.File, error) {\n\tif _, err := p.file.Seek(0, 0); err != nil {\n\t\treturn nil, errors.Wrap(err, \"error resetting log file offset to 0\")\n\t}\n\n\treturn p.file, nil\n}\n\nfunc (p *tempFileLogger) Dispose(log logrus.FieldLogger) {\n\tp.w.Close()\n\tcloseAndRemoveFile(p.file, log)\n}\n\nfunc closeAndRemoveFile(file *os.File, log logrus.FieldLogger) {\n\tif file == nil {\n\t\tlog.Debug(\"Skipping removal of temp log file due to nil file pointer\")\n\t\treturn\n\t}\n\tif err := file.Close(); err != nil {\n\t\tlog.WithError(err).WithField(\"file\", file.Name()).Warn(\"error closing temp log file\")\n\t}\n\tif err := os.Remove(file.Name()); err != nil {\n\t\tlog.WithError(err).WithField(\"file\", file.Name()).Warn(\"error removing temp log file\")\n\t}\n}\n" }, { "path": "pkg/util/logging/dual_mode_logger_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage logging\n\nimport (\n\t\"compress/gzip\"\n\t\"io\"\n\t\"os\"\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestDualModeLogger(t *testing.T) {\n\tlogMsgExpect := \"Expected message in log\"\n\tlogMsgUnexpect := \"Unexpected message in log\"\n\n\tlogger, err := NewTempFileLogger(logrus.DebugLevel, FormatText, nil, logrus.Fields{})\n\trequire.NoError(t, err)\n\n\tlogger.Info(logMsgExpect)\n\n\tlogger.DoneForPersist(velerotest.NewLogger())\n\n\tlogger.Info(logMsgUnexpect)\n\n\tlogFile, err := logger.GetPersistFile()\n\trequire.NoError(t, err)\n\n\tlogStr, err := readLogString(logFile)\n\trequire.NoError(t, err)\n\n\tassert.Contains(t, logStr, logMsgExpect)\n\tassert.NotContains(t, logStr, logMsgUnexpect)\n\n\tlogger.Dispose(velerotest.NewLogger())\n\n\t_, err = os.Stat(logFile.Name())\n\n\tassert.True(t, os.IsNotExist(err))\n}\n\nfunc readLogString(file *os.File) (string, error) {\n\tgzr, err := gzip.NewReader(file)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\n\tbuffer := make([]byte, 1024)\n\t_, err = gzr.Read(buffer)\n\tif err != io.EOF {\n\t\treturn \"\", err\n\t}\n\n\treturn string(buffer[:]), nil\n}\n" }, { "path": "pkg/util/logging/error_location_hook.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage logging\n\nimport (\n\t\"fmt\"\n\t\"strconv\"\n\t\"strings\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n)\n\nconst (\n\terrorFileField = \"error.file\"\n\terrorFunctionField = \"error.function\"\n)\n\n// ErrorLocationHook is a logrus hook that attaches error location information\n// to log entries if an error is being logged and it has stack-trace information\n// (i.e. if it originates from or is wrapped by github.com/pkg/errors, or if it\n// implements the errorLocationer interface, like errors returned from plugins\n// typically do).\ntype ErrorLocationHook struct{}\n\nfunc (h *ErrorLocationHook) Levels() []logrus.Level {\n\treturn logrus.AllLevels\n}\n\nfunc (h *ErrorLocationHook) Fire(entry *logrus.Entry) error {\n\terrObj, ok := entry.Data[logrus.ErrorKey]\n\tif !ok {\n\t\treturn nil\n\t}\n\n\tif _, ok := entry.Data[errorFileField]; ok {\n\t\t// If there is already an error file field, preserve it instead of overwriting it. This field will already exist if\n\t\t// the log message occurred in the server half of a plugin.\n\t\treturn nil\n\t}\n\tif _, ok := entry.Data[errorFunctionField]; ok {\n\t\t// If there is already an error function field, preserve it instead of overwriting it. This field will already exist if\n\t\t// the log message occurred in the server half of a plugin.\n\t\treturn nil\n\t}\n\n\terr, ok := errObj.(error)\n\tif !ok {\n\t\t// if the value isn't an error type, skip trying to get location info,\n\t\t// and just let it be logged as whatever it was\n\t\treturn nil\n\t}\n\n\tif errorLocationer, ok := err.(errorLocationer); ok {\n\t\tentry.Data[errorFileField] = fmt.Sprintf(\"%s:%d\", errorLocationer.File(), errorLocationer.Line())\n\t\tentry.Data[errorFunctionField] = errorLocationer.Function()\n\t\treturn nil\n\t}\n\n\tif stackErr := getInnermostTrace(err); stackErr != nil {\n\t\tlocation := GetFrameLocationInfo(stackErr.StackTrace()[0])\n\n\t\tentry.Data[errorFileField] = fmt.Sprintf(\"%s:%d\", location.File, location.Line)\n\t\tentry.Data[errorFunctionField] = location.Function\n\t}\n\n\treturn nil\n}\n\n// LocationInfo specifies the location of a line\n// of code.\ntype LocationInfo struct {\n\tFile string\n\tFunction string\n\tLine int\n}\n\n// GetFrameLocationInfo returns the location of a frame.\nfunc GetFrameLocationInfo(frame errors.Frame) LocationInfo {\n\t// see https://godoc.org/github.com/pkg/errors#Frame.Format for\n\t// details on formatting verbs\n\tfunctionNameAndFileAndLine := fmt.Sprintf(\"%+v\", frame)\n\n\tnewLineIndex := strings.Index(functionNameAndFileAndLine, \"\\n\")\n\tfunctionName := functionNameAndFileAndLine[0:newLineIndex]\n\n\ttabIndex := strings.LastIndex(functionNameAndFileAndLine, \"\\t\")\n\tfileAndLine := strings.Split(functionNameAndFileAndLine[tabIndex+1:], \":\")\n\n\tline, err := strconv.Atoi(fileAndLine[1])\n\tif err != nil {\n\t\tline = -1\n\t}\n\n\treturn LocationInfo{\n\t\tFile: fileAndLine[0],\n\t\tFunction: functionName,\n\t\tLine: line,\n\t}\n}\n\ntype errorLocationer interface {\n\tFile() string\n\tLine() int32\n\tFunction() string\n}\n\ntype stackTracer interface {\n\terror\n\tStackTrace() errors.StackTrace\n}\n\ntype causer interface {\n\tCause() error\n}\n\n// getInnermostTrace returns the innermost error that\n// has a stack trace attached\nfunc getInnermostTrace(err error) stackTracer {\n\tvar tracer stackTracer\n\n\tfor {\n\t\tt, isTracer := err.(stackTracer)\n\t\tif isTracer {\n\t\t\ttracer = t\n\t\t}\n\n\t\tc, isCauser := err.(causer)\n\t\tif isCauser {\n\t\t\terr = c.Cause()\n\t\t} else {\n\t\t\treturn tracer\n\t\t}\n\t}\n}\n" }, { "path": "pkg/util/logging/error_location_hook_test.go", "content": "/*\nCopyright 2017, 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage logging\n\nimport (\n\t\"errors\"\n\t\"testing\"\n\n\tpkgerrs \"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestFire(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpreEntryFields map[string]any\n\t\texpectedEntryFields map[string]any\n\t\texpectedErr bool\n\t}{\n\t\t{\n\t\t\tname: \"no error\",\n\t\t\tpreEntryFields: map[string]any{\"foo\": \"bar\"},\n\t\t\texpectedEntryFields: map[string]any{\"foo\": \"bar\"},\n\t\t},\n\t\t{\n\t\t\tname: \"basic (non-pkg/errors) error\",\n\t\t\tpreEntryFields: map[string]any{logrus.ErrorKey: errors.New(\"a normal error\")},\n\t\t\texpectedEntryFields: map[string]any{logrus.ErrorKey: errors.New(\"a normal error\")},\n\t\t},\n\t\t{\n\t\t\tname: \"non-error logged in error field\",\n\t\t\tpreEntryFields: map[string]any{logrus.ErrorKey: \"not an error\"},\n\t\t\texpectedEntryFields: map[string]any{logrus.ErrorKey: \"not an error\"},\n\t\t\texpectedErr: false,\n\t\t},\n\t\t{\n\t\t\tname: \"pkg/errors error\",\n\t\t\tpreEntryFields: map[string]any{logrus.ErrorKey: pkgerrs.New(\"a pkg/errors error\")},\n\t\t\texpectedEntryFields: map[string]any{\n\t\t\t\tlogrus.ErrorKey: pkgerrs.New(\"a pkg/errors error\"),\n\t\t\t\terrorFileField: \"\",\n\t\t\t\terrorFunctionField: \"github.com/vmware-tanzu/velero/pkg/util/logging.TestFire\",\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"already have error file and function fields\",\n\t\t\tpreEntryFields: map[string]any{\n\t\t\t\tlogrus.ErrorKey: pkgerrs.New(\"a pkg/errors error\"),\n\t\t\t\terrorFileField: \"some_file.go:123\",\n\t\t\t\terrorFunctionField: \"SomeFunction\",\n\t\t\t},\n\t\t\texpectedEntryFields: map[string]any{\n\t\t\t\tlogrus.ErrorKey: pkgerrs.New(\"a pkg/errors error\"),\n\t\t\t\terrorFileField: \"some_file.go:123\",\n\t\t\t\terrorFunctionField: \"SomeFunction\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\thook := &ErrorLocationHook{}\n\n\t\t\tentry := &logrus.Entry{\n\t\t\t\tData: logrus.Fields(test.preEntryFields),\n\t\t\t}\n\n\t\t\t// method under test\n\t\t\terr := hook.Fire(entry)\n\n\t\t\trequire.Equal(t, test.expectedErr, err != nil)\n\t\t\trequire.Len(t, entry.Data, len(test.expectedEntryFields))\n\n\t\t\tfor key, expectedValue := range test.expectedEntryFields {\n\t\t\t\tactualValue, found := entry.Data[key]\n\t\t\t\tassert.True(t, found, \"expected key not found: %s\", key)\n\n\t\t\t\tswitch key {\n\t\t\t\t// test existence of this field only since testing the value\n\t\t\t\t// is fragile\n\t\t\t\tcase errorFileField:\n\t\t\t\tcase logrus.ErrorKey:\n\t\t\t\t\tif err, ok := expectedValue.(error); ok {\n\t\t\t\t\t\tassert.Equal(t, err.Error(), actualValue.(error).Error())\n\t\t\t\t\t} else {\n\t\t\t\t\t\tassert.Equal(t, expectedValue, actualValue)\n\t\t\t\t\t}\n\t\t\t\tdefault:\n\t\t\t\t\tassert.Equal(t, expectedValue, actualValue)\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetInnermostTrace(t *testing.T) {\n\tnewError := func() error {\n\t\treturn errors.New(\"a normal error\")\n\t}\n\n\ttests := []struct {\n\t\tname string\n\t\terr error\n\t\texpectedRes error\n\t}{\n\t\t{\n\t\t\tname: \"normal error\",\n\t\t\terr: newError(),\n\t\t\texpectedRes: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"pkg/errs error\",\n\t\t\terr: pkgerrs.New(\"a pkg/errs error\"),\n\t\t\texpectedRes: pkgerrs.New(\"a pkg/errs error\"),\n\t\t},\n\t\t{\n\t\t\tname: \"one level of stack-ing a normal error\",\n\t\t\terr: pkgerrs.WithStack(newError()),\n\t\t\texpectedRes: pkgerrs.WithStack(newError()),\n\t\t},\n\t\t{\n\t\t\tname: \"two levels of stack-ing a normal error\",\n\t\t\terr: pkgerrs.WithStack(pkgerrs.WithStack(newError())),\n\t\t\texpectedRes: pkgerrs.WithStack(newError()),\n\t\t},\n\t\t{\n\t\t\tname: \"one level of stack-ing a pkg/errors error\",\n\t\t\terr: pkgerrs.WithStack(pkgerrs.New(\"a pkg/errs error\")),\n\t\t\texpectedRes: pkgerrs.New(\"a pkg/errs error\"),\n\t\t},\n\t\t{\n\t\t\tname: \"two levels of stack-ing a pkg/errors error\",\n\t\t\terr: pkgerrs.WithStack(pkgerrs.WithStack(pkgerrs.New(\"a pkg/errs error\"))),\n\t\t\texpectedRes: pkgerrs.New(\"a pkg/errs error\"),\n\t\t},\n\t\t{\n\t\t\tname: \"two levels of wrapping a normal error\",\n\t\t\terr: pkgerrs.Wrap(pkgerrs.Wrap(newError(), \"wrap 1\"), \"wrap 2\"),\n\t\t\texpectedRes: pkgerrs.Wrap(newError(), \"wrap 1\"),\n\t\t},\n\t\t{\n\t\t\tname: \"two levels of wrapping a pkg/errors error\",\n\t\t\terr: pkgerrs.Wrap(pkgerrs.Wrap(pkgerrs.New(\"a pkg/errs error\"), \"wrap 1\"), \"wrap 2\"),\n\t\t\texpectedRes: pkgerrs.New(\"a pkg/errs error\"),\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tres := getInnermostTrace(test.err)\n\n\t\t\tif test.expectedRes == nil {\n\t\t\t\trequire.NoError(t, res)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tassert.Equal(t, test.expectedRes.Error(), res.Error())\n\t\t})\n\t}\n}\n" }, { "path": "pkg/util/logging/format_flag.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage logging\n\nimport \"github.com/vmware-tanzu/velero/pkg/cmd/util/flag\"\n\n// Format is a string representation of the desired output format for logs\ntype Format string\n\nconst (\n\tFormatText Format = \"text\"\n\tFormatJSON Format = \"json\"\n\tdefaultValue Format = FormatText\n)\n\n// FormatFlag is a command-line flag for setting the logrus\n// log format.\ntype FormatFlag struct {\n\t*flag.Enum\n\tdefaultValue Format\n}\n\n// NewFormatFlag constructs a new log level flag.\nfunc NewFormatFlag() *FormatFlag {\n\treturn &FormatFlag{\n\t\tEnum: flag.NewEnum(\n\t\t\tstring(defaultValue),\n\t\t\tstring(FormatText),\n\t\t\tstring(FormatJSON),\n\t\t),\n\t\tdefaultValue: defaultValue,\n\t}\n}\n\n// Parse returns the flag's value as a Format.\nfunc (f *FormatFlag) Parse() Format {\n\treturn Format(f.String())\n}\n" }, { "path": "pkg/util/logging/hclog_level_hook.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage logging\n\nimport (\n\t\"github.com/sirupsen/logrus\"\n)\n\n// HcLogLevelHook adds an hclog-compatible field (\"@level\") containing\n// the log level. Note that if you use this, you SHOULD NOT use\n// logrus.JSONFormatter's FieldMap to set the level key to \"@level\" because\n// that will result in the hclog-compatible info written here being\n// overwritten.\ntype HcLogLevelHook struct{}\n\nfunc (h *HcLogLevelHook) Levels() []logrus.Level {\n\treturn logrus.AllLevels\n}\n\nfunc (h *HcLogLevelHook) Fire(entry *logrus.Entry) error {\n\tswitch entry.Level {\n\t// logrus uses \"warning\" to represent WarnLevel,\n\t// which is not compatible with hclog's \"warn\".\n\tcase logrus.WarnLevel:\n\t\tentry.Data[\"@level\"] = \"warn\"\n\tdefault:\n\t\tentry.Data[\"@level\"] = entry.Level.String()\n\t}\n\n\treturn nil\n}\n" }, { "path": "pkg/util/logging/log_counter_hook.go", "content": "/*\nCopyright 2019 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage logging\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"sync\"\n\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/results\"\n)\n\n// LogHook is a logrus hook that counts the number of log\n// statements that have been written at each logrus level. It also\n// maintains log entries at each logrus level in result structure.\ntype LogHook struct {\n\tmu sync.RWMutex\n\tcounts map[logrus.Level]int\n\tentries map[logrus.Level]*results.Result\n}\n\n// NewLogHook returns a pointer to an initialized LogHook.\nfunc NewLogHook() *LogHook {\n\treturn &LogHook{\n\t\tcounts: make(map[logrus.Level]int),\n\t\tentries: make(map[logrus.Level]*results.Result),\n\t}\n}\n\n// Levels returns the logrus levels that the hook should be fired for.\nfunc (h *LogHook) Levels() []logrus.Level {\n\treturn logrus.AllLevels\n}\n\n// Fire executes the hook's logic.\nfunc (h *LogHook) Fire(entry *logrus.Entry) error {\n\th.mu.Lock()\n\tdefer h.mu.Unlock()\n\n\th.counts[entry.Level]++\n\tif h.entries[entry.Level] == nil {\n\t\th.entries[entry.Level] = &results.Result{}\n\t}\n\n\tnamespace, isNamespacePresent := entry.Data[\"namespace\"]\n\terrorField, isErrorFieldPresent := entry.Data[\"error\"]\n\t// When JSON logging format is enabled, error message is placed at \"error.message\" instead of \"error\"\n\terrorMsgField, isErrorMsgFieldPresent := entry.Data[\"error.message\"]\n\tresourceField, isResourceFieldPresent := entry.Data[\"resource\"]\n\tnameField, isNameFieldPresent := entry.Data[\"name\"]\n\tmsgField, isMsgFieldPresent := entry.Message, true\n\n\tentryMessage := \"\"\n\tif isResourceFieldPresent {\n\t\tentryMessage = fmt.Sprintf(\"%s resource: /%s\", entryMessage, resourceField.(string))\n\t}\n\tif isNameFieldPresent {\n\t\tentryMessage = fmt.Sprintf(\"%s name: /%s\", entryMessage, nameField.(string))\n\t}\n\tif isMsgFieldPresent {\n\t\tentryMessage = fmt.Sprintf(\"%s message: /%v\", entryMessage, msgField)\n\t}\n\tif isErrorFieldPresent {\n\t\tentryMessage = fmt.Sprintf(\"%s error: /%v\", entryMessage, errorField)\n\t}\n\tif isErrorMsgFieldPresent {\n\t\tentryMessage = fmt.Sprintf(\"%s error: /%v\", entryMessage, errorMsgField)\n\t}\n\n\tif isNamespacePresent {\n\t\th.entries[entry.Level].Add(namespace.(string), errors.New(entryMessage))\n\t} else {\n\t\th.entries[entry.Level].AddVeleroError(errors.New(entryMessage))\n\t}\n\treturn nil\n}\n\n// GetCount returns the number of log statements that have been\n// written at the specific level provided.\nfunc (h *LogHook) GetCount(level logrus.Level) int {\n\th.mu.RLock()\n\tdefer h.mu.RUnlock()\n\n\treturn h.counts[level]\n}\n\n// GetEntries returns the log statements that have been\n// written at the specific level provided.\nfunc (h *LogHook) GetEntries(level logrus.Level) results.Result {\n\th.mu.RLock()\n\tdefer h.mu.RUnlock()\n\tresponse, isPresent := h.entries[level]\n\tif isPresent {\n\t\treturn *response\n\t}\n\treturn results.Result{}\n}\n" }, { "path": "pkg/util/logging/log_counter_hook_test.go", "content": "package logging\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"testing\"\n\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/util/results\"\n)\n\nfunc TestLogHook_Fire(t *testing.T) {\n\thook := NewLogHook()\n\n\tentry := &logrus.Entry{\n\t\tLevel: logrus.ErrorLevel,\n\t\tData: logrus.Fields{\n\t\t\t\"namespace\": \"test-namespace\",\n\t\t\t\"error\": errors.New(\"test-error\"),\n\t\t\t\"resource\": \"test-resource\",\n\t\t\t\"name\": \"test-name\",\n\t\t},\n\t\tMessage: \"test-message\",\n\t}\n\n\terr := hook.Fire(entry)\n\trequire.NoError(t, err)\n\n\t// Verify the counts\n\tassert.Equal(t, 1, hook.counts[logrus.ErrorLevel])\n\n\t// Verify the entries\n\texpectedResult := &results.Result{}\n\texpectedResult.Add(\"test-namespace\", fmt.Errorf(\" resource: /test-resource name: /test-name message: /test-message error: /%v\", entry.Data[\"error\"]))\n\tassert.Equal(t, expectedResult, hook.entries[logrus.ErrorLevel])\n\n\tentry1 := &logrus.Entry{\n\t\tLevel: logrus.ErrorLevel,\n\t\tData: logrus.Fields{\n\t\t\t\"error.message\": errors.New(\"test-error\"),\n\t\t\t\"resource\": \"test-resource\",\n\t\t\t\"name\": \"test-name\",\n\t\t},\n\t\tMessage: \"test-message\",\n\t}\n\n\terr = hook.Fire(entry1)\n\trequire.NoError(t, err)\n\n\t// Verify the counts\n\tassert.Equal(t, 2, hook.counts[logrus.ErrorLevel])\n\n\t// Verify the entries\n\texpectedResult = &results.Result{}\n\texpectedResult.Add(\"test-namespace\", fmt.Errorf(\" resource: /test-resource name: /test-name message: /test-message error: /%v\", entry.Data[\"error\"]))\n\texpectedResult.AddVeleroError(fmt.Errorf(\" resource: /test-resource name: /test-name message: /test-message error: /%v\", entry1.Data[\"error.message\"]))\n\tassert.Equal(t, expectedResult, hook.entries[logrus.ErrorLevel])\n}\n\nfunc TestLogHook_Levels(t *testing.T) {\n\thook := NewLogHook()\n\n\tlevels := hook.Levels()\n\n\texpectedLevels := []logrus.Level{\n\t\tlogrus.PanicLevel,\n\t\tlogrus.FatalLevel,\n\t\tlogrus.ErrorLevel,\n\t\tlogrus.WarnLevel,\n\t\tlogrus.InfoLevel,\n\t\tlogrus.DebugLevel,\n\t\tlogrus.TraceLevel,\n\t}\n\n\tassert.Equal(t, expectedLevels, levels)\n}\n\nfunc TestLogHook_GetCount(t *testing.T) {\n\thook := NewLogHook()\n\n\t// Set up test data\n\thook.counts[logrus.ErrorLevel] = 5\n\thook.counts[logrus.WarnLevel] = 10\n\n\t// Test GetCount for ErrorLevel\n\tcount := hook.GetCount(logrus.ErrorLevel)\n\tassert.Equal(t, 5, count)\n\n\t// Test GetCount for WarnLevel\n\tcount = hook.GetCount(logrus.WarnLevel)\n\tassert.Equal(t, 10, count)\n\n\t// Test GetCount for other levels\n\tcount = hook.GetCount(logrus.InfoLevel)\n\tassert.Equal(t, 0, count)\n\n\tcount = hook.GetCount(logrus.DebugLevel)\n\tassert.Equal(t, 0, count)\n}\n\nfunc TestLogHook_GetEntries(t *testing.T) {\n\thook := NewLogHook()\n\n\t// Set up test data\n\tentry := &logrus.Entry{\n\t\tLevel: logrus.ErrorLevel,\n\t\tData: logrus.Fields{\n\t\t\t\"namespace\": \"test-namespace\",\n\t\t\t\"error\": errors.New(\"test-error\"),\n\t\t\t\"resource\": \"test-resource\",\n\t\t\t\"name\": \"test-name\",\n\t\t},\n\t\tMessage: \"test-message\",\n\t}\n\texpectedResult := &results.Result{}\n\texpectedResult.Add(\"test-namespace\", fmt.Errorf(\" resource: /test-resource name: /test-name message: /test-message error: /%v\", entry.Data[\"error\"]))\n\thook.entries[logrus.ErrorLevel] = expectedResult\n\n\t// Test GetEntries for ErrorLevel\n\tresult := hook.GetEntries(logrus.ErrorLevel)\n\tassert.Equal(t, *expectedResult, result)\n\n\t// Test GetEntries for WarnLevel\n\tresult = hook.GetEntries(logrus.WarnLevel)\n\tassert.Equal(t, results.Result{}, result)\n\n\t// Test GetEntries for other levels\n\tresult = hook.GetEntries(logrus.InfoLevel)\n\tassert.Equal(t, results.Result{}, result)\n\n\tresult = hook.GetEntries(logrus.DebugLevel)\n\tassert.Equal(t, results.Result{}, result)\n}\n" }, { "path": "pkg/util/logging/log_level_flag.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\npackage logging\n\nimport (\n\t\"sort\"\n\t\"strings\"\n\n\t\"github.com/sirupsen/logrus\"\n\n\t\"github.com/vmware-tanzu/velero/pkg/cmd/util/flag\"\n)\n\nvar sortedLogLevels = sortLogLevels()\n\n// LevelFlag is a command-line flag for setting the logrus\n// log level.\ntype LevelFlag struct {\n\t*flag.Enum\n\tdefaultValue logrus.Level\n}\n\n// LogLevelFlag constructs a new log level flag.\nfunc LogLevelFlag(defaultValue logrus.Level) *LevelFlag {\n\treturn &LevelFlag{\n\t\tEnum: flag.NewEnum(defaultValue.String(), sortedLogLevels...),\n\t\tdefaultValue: defaultValue,\n\t}\n}\n\n// Parse returns the flag's value as a logrus.Level.\nfunc (f *LevelFlag) Parse() logrus.Level {\n\tif parsed, err := logrus.ParseLevel(f.String()); err == nil {\n\t\treturn parsed\n\t}\n\n\t// This should theoretically never happen assuming the enum flag\n\t// is constructed correctly because the enum flag will not allow\n\t// an invalid value to be set.\n\tlogrus.Errorf(\"log-level flag has invalid value %s\", strings.ToUpper(f.String()))\n\treturn f.defaultValue\n}\n\n// sortLogLevels returns a string slice containing all of the valid logrus\n// log levels (based on logrus.AllLevels), sorted in ascending order of severity.\nfunc sortLogLevels() []string {\n\tvar (\n\t\tsortedLogLevels = make([]logrus.Level, len(logrus.AllLevels))\n\t\tlogLevelsStrings []string\n\t)\n\n\tcopy(sortedLogLevels, logrus.AllLevels)\n\n\t// logrus.Panic has the lowest value, so the compare function uses \">\"\n\tsort.Slice(sortedLogLevels, func(i, j int) bool { return sortedLogLevels[i] > sortedLogLevels[j] })\n\n\tfor _, level := range sortedLogLevels {\n\t\tlogLevelsStrings = append(logLevelsStrings, level.String())\n\t}\n\n\treturn logLevelsStrings\n}\n" }, { "path": "pkg/util/logging/log_location_hook.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage logging\n\nimport (\n\t\"fmt\"\n\t\"runtime\"\n\t\"strings\"\n\n\t\"github.com/sirupsen/logrus\"\n)\n\nconst (\n\tlogSourceField = \"logSource\"\n\tlogSourceSetMarkerField = \"@logSourceSetBy\"\n\tlogrusPackage = \"github.com/sirupsen/logrus\"\n\tveleroPackage = \"github.com/vmware-tanzu/velero/\"\n\tveleroPackageLen = len(veleroPackage)\n)\n\n// LogLocationHook is a logrus hook that attaches location information\n// to log entries, i.e. the file and line number of the logrus log call.\n// This hook is designed for use in both the Velero server and Velero plugin\n// implementations. When triggered within a plugin, a marker field will\n// be set on the log entry indicating that the location came from a plugin.\n// The Velero server instance will not overwrite location information if\n// it sees this marker.\ntype LogLocationHook struct {\n\tloggerName string\n}\n\n// WithLoggerName gives the hook a name to use when setting the marker field\n// on a log entry indicating the location has been recorded by a plugin. This\n// should only be used when setting up a hook for a logger used in a plugin.\nfunc (h *LogLocationHook) WithLoggerName(name string) *LogLocationHook {\n\th.loggerName = name\n\treturn h\n}\n\nfunc (h *LogLocationHook) Levels() []logrus.Level {\n\treturn logrus.AllLevels\n}\n\nfunc (h *LogLocationHook) Fire(entry *logrus.Entry) error {\n\tpcs := make([]uintptr, 64)\n\n\t// skip 2 frames:\n\t// runtime.Callers\n\t// github.com/vmware-tanzu/velero/pkg/util/logging/(*LogLocationHook).Fire\n\tn := runtime.Callers(2, pcs)\n\n\t// re-slice pcs based on the number of entries written\n\tframes := runtime.CallersFrames(pcs[:n])\n\n\t// now traverse up the call stack looking for the first non-logrus\n\t// func, which will be the logrus invoker\n\tvar (\n\t\tframe runtime.Frame\n\t\tmore = true\n\t)\n\n\tfor more {\n\t\tframe, more = frames.Next()\n\n\t\tif strings.Contains(frame.File, logrusPackage) {\n\t\t\tcontinue\n\t\t}\n\n\t\t// set the marker field if we're within a plugin indicating that\n\t\t// the location comes from the plugin.\n\t\tif h.loggerName != \"\" {\n\t\t\tentry.Data[logSourceSetMarkerField] = h.loggerName\n\t\t}\n\n\t\t// record the log statement location if we're within a plugin OR if\n\t\t// we're in Velero server and not logging something that has the marker\n\t\t// set (which would indicate the log statement is coming from a plugin).\n\t\tif h.loggerName != \"\" || getLogSourceSetMarker(entry) == \"\" {\n\t\t\tfile := removeVeleroPackagePrefix(frame.File)\n\n\t\t\tentry.Data[logSourceField] = fmt.Sprintf(\"%s:%d\", file, frame.Line)\n\t\t}\n\n\t\t// if we're in the Velero server, remove the marker field since we don't\n\t\t// want to record it in the actual log.\n\t\tif h.loggerName == \"\" {\n\t\t\tdelete(entry.Data, logSourceSetMarkerField)\n\t\t}\n\n\t\tbreak\n\t}\n\n\treturn nil\n}\n\nfunc getLogSourceSetMarker(entry *logrus.Entry) string {\n\tnameVal, found := entry.Data[logSourceSetMarkerField]\n\tif !found {\n\t\treturn \"\"\n\t}\n\n\tif name, ok := nameVal.(string); ok {\n\t\treturn name\n\t}\n\n\treturn fmt.Sprintf(\"%s\", nameVal)\n}\n\nfunc removeVeleroPackagePrefix(file string) string {\n\tif index := strings.Index(file, veleroPackage); index != -1 {\n\t\t// strip off .../github.com/vmware-tanzu/velero/ so we just have pkg/...\n\t\treturn file[index+veleroPackageLen:]\n\t}\n\n\treturn file\n}\n" }, { "path": "pkg/util/logging/log_location_hook_test.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage logging\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestRemoveVeleroPackagePrefix(t *testing.T) {\n\tassert.Equal(t, \"pkg/foo.go\", removeVeleroPackagePrefix(\"github.com/vmware-tanzu/velero/pkg/foo.go\"))\n\tassert.Equal(t, \"github.com/vmware-tanzu/velero-plugin-example/foo.go\", removeVeleroPackagePrefix(\"github.com/vmware-tanzu/velero-plugin-example/foo.go\"))\n}\n" }, { "path": "pkg/util/logging/log_merge_hook.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage logging\n\nimport (\n\t\"bytes\"\n\t\"io\"\n\t\"os\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n)\n\nconst (\n\tListeningLevel = logrus.ErrorLevel\n\tListeningMessage = \"merge-log-57847fd0-0c7c-48e3-b5f7-984b293d8376\"\n\tLogSourceKey = \"log-source\"\n)\n\n// MergeHook is used to redirect a batch of logs to another logger atomically.\n// It hooks a log with ListeningMessage message, once the message is hit it replaces\n// the logger's output to HookWriter so that HookWriter retrieves the logs from a file indicated\n// by LogSourceKey field.\ntype MergeHook struct {\n}\n\ntype hookWriter struct {\n\torgWriter io.Writer\n\tsource string\n\tlogger *logrus.Logger\n}\n\nfunc newHookWriter(orgWriter io.Writer, source string, logger *logrus.Logger) io.Writer {\n\treturn &hookWriter{\n\t\torgWriter: orgWriter,\n\t\tsource: source,\n\t\tlogger: logger,\n\t}\n}\n\nfunc (h *MergeHook) Levels() []logrus.Level {\n\treturn []logrus.Level{ListeningLevel}\n}\n\nfunc (h *MergeHook) Fire(entry *logrus.Entry) error {\n\tif entry.Message != ListeningMessage {\n\t\treturn nil\n\t}\n\n\tsource, exist := entry.Data[LogSourceKey]\n\tif !exist {\n\t\treturn nil\n\t}\n\n\tentry.Logger.SetOutput(newHookWriter(entry.Logger.Out, source.(string), entry.Logger))\n\n\treturn nil\n}\n\nfunc (w *hookWriter) Write(p []byte) (n int, err error) {\n\tif !bytes.Contains(p, []byte(ListeningMessage)) {\n\t\treturn w.orgWriter.Write(p)\n\t}\n\n\tdefer func() {\n\t\tw.logger.Out = w.orgWriter\n\t}()\n\n\tsourceFile, err := os.OpenFile(w.source, os.O_RDONLY, 0400)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\tdefer sourceFile.Close()\n\n\ttotal := 0\n\n\tbuffer := make([]byte, 2048)\n\tfor {\n\t\tread, err := sourceFile.Read(buffer)\n\t\tif err == io.EOF {\n\t\t\treturn total, nil\n\t\t}\n\n\t\tif err != nil {\n\t\t\treturn total, errors.Wrapf(err, \"error to read source file %s at pos %v\", w.source, total)\n\t\t}\n\n\t\twritten, err := w.orgWriter.Write(buffer[0:read])\n\t\tif err != nil {\n\t\t\treturn total, errors.Wrapf(err, \"error to write log at pos %v\", total)\n\t\t}\n\n\t\tif written != read {\n\t\t\treturn total, errors.Errorf(\"error to write log at pos %v, read %v but written %v\", total, read, written)\n\t\t}\n\n\t\ttotal += read\n\t}\n}\n" }, { "path": "pkg/util/logging/log_merge_hook_test.go", "content": "/*\nCopyright The Velero Contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage logging\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/sirupsen/logrus\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestMergeHook_Fire(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tentry logrus.Entry\n\t\texpectHook bool\n\t}{\n\t\t{\n\t\t\tname: \"normal message\",\n\t\t\tentry: logrus.Entry{\n\t\t\t\tLevel: logrus.ErrorLevel,\n\t\t\t\tMessage: \"fake-message\",\n\t\t\t},\n\t\t\texpectHook: false,\n\t\t},\n\t\t{\n\t\t\tname: \"normal source\",\n\t\t\tentry: logrus.Entry{\n\t\t\t\tLevel: logrus.ErrorLevel,\n\t\t\t\tMessage: ListeningMessage,\n\t\t\t\tData: logrus.Fields{\"fake-key\": \"fake-value\"},\n\t\t\t},\n\t\t\texpectHook: false,\n\t\t},\n\t\t{\n\t\t\tname: \"hook hit\",\n\t\t\tentry: logrus.Entry{\n\t\t\t\tLevel: logrus.ErrorLevel,\n\t\t\t\tMessage: ListeningMessage,\n\t\t\t\tData: logrus.Fields{LogSourceKey: \"any-value\"},\n\t\t\t\tLogger: &logrus.Logger{},\n\t\t\t},\n\t\t\texpectHook: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\thook := &MergeHook{}\n\t\t\t// method under test\n\t\t\terr := hook.Fire(&test.entry)\n\n\t\t\trequire.NoError(t, err)\n\n\t\t\tif test.expectHook {\n\t\t\t\tassert.NotNil(t, test.entry.Logger.Out.(*hookWriter))\n\t\t\t}\n\t\t})\n\t}\n}\n\ntype fakeWriter struct {\n\tp []byte\n\twriteError error\n\twrittenLen int\n}\n\nfunc (fw *fakeWriter) Write(p []byte) (n int, err error) {\n\tif fw.writeError != nil || fw.writtenLen != -1 {\n\t\treturn fw.writtenLen, fw.writeError\n\t}\n\n\tfw.p = append(fw.p, p...)\n\n\treturn len(p), nil\n}\n\nfunc TestMergeHook_Write(t *testing.T) {\n\tsourceFile, err := os.CreateTemp(t.TempDir(), \"\")\n\trequire.NoError(t, err)\n\n\tlogMessage := \"fake-message-1\\nfake-message-2\"\n\t_, err = sourceFile.WriteString(logMessage)\n\trequire.NoError(t, err)\n\n\ttests := []struct {\n\t\tname string\n\t\tcontent []byte\n\t\tsource string\n\t\twriteErr error\n\t\twrittenLen int\n\t\texpectError string\n\t\tneedRollBackHook bool\n\t}{\n\t\t{\n\t\t\tname: \"normal message\",\n\t\t\tcontent: []byte(\"fake-message\"),\n\t\t\twrittenLen: -1,\n\t\t},\n\t\t{\n\t\t\tname: \"failed to open source file\",\n\t\t\tcontent: []byte(ListeningMessage),\n\t\t\tsource: \"non-exist\",\n\t\t\tneedRollBackHook: true,\n\t\t\texpectError: \"open non-exist: no such file or directory\",\n\t\t},\n\t\t{\n\t\t\tname: \"write error\",\n\t\t\tcontent: []byte(ListeningMessage),\n\t\t\tsource: sourceFile.Name(),\n\t\t\twriteErr: errors.New(\"fake-error\"),\n\t\t\texpectError: \"error to write log at pos 0: fake-error\",\n\t\t\tneedRollBackHook: true,\n\t\t},\n\t\t{\n\t\t\tname: \"write len mismatch\",\n\t\t\tcontent: []byte(ListeningMessage),\n\t\t\tsource: sourceFile.Name(),\n\t\t\twrittenLen: 100,\n\t\t\texpectError: fmt.Sprintf(\"error to write log at pos 0, read %v but written 100\", len(logMessage)),\n\t\t\tneedRollBackHook: true,\n\t\t},\n\t\t{\n\t\t\tname: \"success\",\n\t\t\tcontent: []byte(ListeningMessage),\n\t\t\tsource: sourceFile.Name(),\n\t\t\twrittenLen: -1,\n\t\t\tneedRollBackHook: true,\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\twriter := hookWriter{\n\t\t\t\torgWriter: &fakeWriter{\n\t\t\t\t\twriteError: test.writeErr,\n\t\t\t\t\twrittenLen: test.writtenLen,\n\t\t\t\t},\n\t\t\t\tsource: test.source,\n\t\t\t\tlogger: &logrus.Logger{},\n\t\t\t}\n\n\t\t\tn, err := writer.Write(test.content)\n\n\t\t\tif test.expectError == \"\" {\n\t\t\t\trequire.NoError(t, err)\n\n\t\t\t\texpectStr := string(test.content)\n\t\t\t\tif expectStr == ListeningMessage {\n\t\t\t\t\texpectStr = logMessage\n\t\t\t\t}\n\n\t\t\t\tassert.Len(t, expectStr, n)\n\n\t\t\t\tfakeWriter := writer.orgWriter.(*fakeWriter)\n\t\t\t\twrittenStr := string(fakeWriter.p)\n\t\t\t\tassert.Equal(t, writtenStr, expectStr)\n\t\t\t} else {\n\t\t\t\trequire.EqualError(t, err, test.expectError)\n\t\t\t}\n\n\t\t\tif test.needRollBackHook {\n\t\t\t\tassert.Equal(t, writer.logger.Out, writer.orgWriter)\n\t\t\t}\n\t\t})\n\t}\n}\n" }, { "path": "pkg/util/podvolume/pod_volume.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"context\"\n\t\"strings\"\n\t\"sync\"\n\n\t\"github.com/pkg/errors\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\tcrclient \"sigs.k8s.io/controller-runtime/pkg/client\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/util\"\n)\n\n// PVCPodCache provides a cached mapping from PVC to the pods that use it.\n// This cache is built once per backup to avoid repeated pod listings which\n// cause O(N*M) performance issues when there are many PVCs and pods.\ntype PVCPodCache struct {\n\tmu sync.RWMutex\n\t// cache maps namespace -> pvcName -> []Pod\n\tcache map[string]map[string][]corev1api.Pod\n\t// built indicates whether the cache has been populated\n\tbuilt bool\n}\n\n// NewPVCPodCache creates a new empty PVC to Pod cache.\nfunc NewPVCPodCache() *PVCPodCache {\n\treturn &PVCPodCache{\n\t\tcache: make(map[string]map[string][]corev1api.Pod),\n\t\tbuilt: false,\n\t}\n}\n\n// BuildCacheForNamespaces builds the cache by listing pods once per namespace.\n// This is much more efficient than listing pods for each PVC lookup.\nfunc (c *PVCPodCache) BuildCacheForNamespaces(\n\tctx context.Context,\n\tnamespaces []string,\n\tcrClient crclient.Client,\n) error {\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\n\tfor _, ns := range namespaces {\n\t\tpodList := new(corev1api.PodList)\n\t\tif err := crClient.List(\n\t\t\tctx,\n\t\t\tpodList,\n\t\t\t&crclient.ListOptions{Namespace: ns},\n\t\t); err != nil {\n\t\t\treturn errors.Wrapf(err, \"failed to list pods in namespace %s\", ns)\n\t\t}\n\n\t\tif c.cache[ns] == nil {\n\t\t\tc.cache[ns] = make(map[string][]corev1api.Pod)\n\t\t}\n\n\t\t// Build mapping from PVC name to pods\n\t\tfor i := range podList.Items {\n\t\t\tpod := podList.Items[i]\n\t\t\tfor _, v := range pod.Spec.Volumes {\n\t\t\t\tif v.PersistentVolumeClaim != nil {\n\t\t\t\t\tpvcName := v.PersistentVolumeClaim.ClaimName\n\t\t\t\t\tc.cache[ns][pvcName] = append(c.cache[ns][pvcName], pod)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\tc.built = true\n\treturn nil\n}\n\n// GetPodsUsingPVC retrieves pods using a specific PVC from the cache.\n// Returns nil slice if the PVC is not found in the cache.\nfunc (c *PVCPodCache) GetPodsUsingPVC(namespace, pvcName string) []corev1api.Pod {\n\tc.mu.RLock()\n\tdefer c.mu.RUnlock()\n\n\tif nsPods, ok := c.cache[namespace]; ok {\n\t\tif pods, ok := nsPods[pvcName]; ok {\n\t\t\t// Return a copy to avoid race conditions\n\t\t\tresult := make([]corev1api.Pod, len(pods))\n\t\t\tcopy(result, pods)\n\t\t\treturn result\n\t\t}\n\t}\n\treturn nil\n}\n\n// IsBuilt returns true if the cache has been built.\nfunc (c *PVCPodCache) IsBuilt() bool {\n\tc.mu.RLock()\n\tdefer c.mu.RUnlock()\n\treturn c.built\n}\n\n// IsNamespaceBuilt returns true if the cache has been built for the given namespace.\nfunc (c *PVCPodCache) IsNamespaceBuilt(namespace string) bool {\n\tc.mu.RLock()\n\tdefer c.mu.RUnlock()\n\t_, ok := c.cache[namespace]\n\treturn ok\n}\n\n// BuildCacheForNamespace builds the cache for a single namespace lazily.\n// This is used by plugins where namespaces are encountered one at a time.\n// If the namespace is already cached, this is a no-op.\nfunc (c *PVCPodCache) BuildCacheForNamespace(\n\tctx context.Context,\n\tnamespace string,\n\tcrClient crclient.Client,\n) error {\n\t// Check if already built (read lock first for performance)\n\tc.mu.RLock()\n\tif _, ok := c.cache[namespace]; ok {\n\t\tc.mu.RUnlock()\n\t\treturn nil\n\t}\n\tc.mu.RUnlock()\n\n\t// Need to build - acquire write lock\n\tc.mu.Lock()\n\tdefer c.mu.Unlock()\n\n\t// Double-check after acquiring write lock\n\tif _, ok := c.cache[namespace]; ok {\n\t\treturn nil\n\t}\n\n\tpodList := new(corev1api.PodList)\n\tif err := crClient.List(\n\t\tctx,\n\t\tpodList,\n\t\t&crclient.ListOptions{Namespace: namespace},\n\t); err != nil {\n\t\treturn errors.Wrapf(err, \"failed to list pods in namespace %s\", namespace)\n\t}\n\n\tc.cache[namespace] = make(map[string][]corev1api.Pod)\n\n\t// Build mapping from PVC name to pods\n\tfor i := range podList.Items {\n\t\tpod := podList.Items[i]\n\t\tfor _, v := range pod.Spec.Volumes {\n\t\t\tif v.PersistentVolumeClaim != nil {\n\t\t\t\tpvcName := v.PersistentVolumeClaim.ClaimName\n\t\t\t\tc.cache[namespace][pvcName] = append(c.cache[namespace][pvcName], pod)\n\t\t\t}\n\t\t}\n\t}\n\n\t// Mark as built for GetPodsUsingPVCWithCache fallback logic\n\tc.built = true\n\treturn nil\n}\n\n// GetVolumesByPod returns a list of volume names to backup for the provided pod.\nfunc GetVolumesByPod(pod *corev1api.Pod, defaultVolumesToFsBackup, backupExcludePVC bool, volsToProcessByLegacyApproach []string) ([]string, []string) {\n\t// tracks the volumes that have been explicitly opted out of backup via the annotation in the pod\n\toptedOutVolumes := make([]string, 0)\n\n\tif !defaultVolumesToFsBackup {\n\t\treturn GetVolumesToBackup(pod), optedOutVolumes\n\t}\n\n\tvolsToExclude := GetVolumesToExclude(pod)\n\tpodVolumes := []string{}\n\t// Identify volume to process\n\t// For normal case all the pod volume will be processed\n\t// For case when volsToProcessByLegacyApproach is non-empty then only those volume will be processed\n\tvolsToProcess := GetVolumesToProcess(pod.Spec.Volumes, volsToProcessByLegacyApproach)\n\tfor _, pv := range volsToProcess {\n\t\t// cannot backup hostpath volumes as they are not mounted into /var/lib/kubelet/pods\n\t\t// and therefore not accessible to the node agent daemon set.\n\t\tif pv.HostPath != nil {\n\t\t\tcontinue\n\t\t}\n\t\t// don't backup volumes mounting secrets. Secrets will be backed up separately.\n\t\tif pv.Secret != nil {\n\t\t\tcontinue\n\t\t}\n\t\t// don't backup volumes mounting ConfigMaps. ConfigMaps will be backed up separately.\n\t\tif pv.ConfigMap != nil {\n\t\t\tcontinue\n\t\t}\n\t\t// don't backup volumes mounted as projected volumes, all data in those come from kube state.\n\t\tif pv.Projected != nil {\n\t\t\tcontinue\n\t\t}\n\t\t// don't backup DownwardAPI volumes, all data in those come from kube state.\n\t\tif pv.DownwardAPI != nil {\n\t\t\tcontinue\n\t\t}\n\t\tif pv.PersistentVolumeClaim != nil && backupExcludePVC {\n\t\t\tcontinue\n\t\t}\n\t\t// don't backup volumes that are included in the exclude list.\n\t\tif util.Contains(volsToExclude, pv.Name) {\n\t\t\toptedOutVolumes = append(optedOutVolumes, pv.Name)\n\t\t\tcontinue\n\t\t}\n\t\t// don't include volumes that mount the default service account token.\n\t\tif strings.HasPrefix(pv.Name, \"default-token\") {\n\t\t\tcontinue\n\t\t}\n\t\tpodVolumes = append(podVolumes, pv.Name)\n\t}\n\treturn podVolumes, optedOutVolumes\n}\n\n// GetVolumesToBackup returns a list of volume names to backup for\n// the provided pod.\n// Deprecated: Use GetVolumesByPod instead.\nfunc GetVolumesToBackup(obj metav1.Object) []string {\n\tannotations := obj.GetAnnotations()\n\tif annotations == nil {\n\t\treturn nil\n\t}\n\n\tbackupsValue := annotations[velerov1api.VolumesToBackupAnnotation]\n\tif backupsValue == \"\" {\n\t\treturn nil\n\t}\n\n\treturn strings.Split(backupsValue, \",\")\n}\n\nfunc GetVolumesToExclude(obj metav1.Object) []string {\n\tannotations := obj.GetAnnotations()\n\tif annotations == nil {\n\t\treturn nil\n\t}\n\n\treturn strings.Split(annotations[velerov1api.VolumesToExcludeAnnotation], \",\")\n}\n\n// IsPVCDefaultToFSBackupWithCache checks if a PVC should default to fs-backup based on pod annotations.\n// If cache is nil or not built, it falls back to listing pods directly.\n// Note: In the main backup path, the cache is always built (via NewVolumeHelperImplWithNamespaces),\n// so the fallback is only used by plugins that don't need cache optimization.\nfunc IsPVCDefaultToFSBackupWithCache(\n\tpvcNamespace, pvcName string,\n\tcrClient crclient.Client,\n\tdefaultVolumesToFsBackup bool,\n\tcache *PVCPodCache,\n) (bool, error) {\n\tvar pods []corev1api.Pod\n\tvar err error\n\n\t// Use cache if available, otherwise fall back to direct lookup\n\tif cache != nil && cache.IsBuilt() {\n\t\tpods = cache.GetPodsUsingPVC(pvcNamespace, pvcName)\n\t} else {\n\t\tpods, err = getPodsUsingPVCDirect(pvcNamespace, pvcName, crClient)\n\t\tif err != nil {\n\t\t\treturn false, errors.WithStack(err)\n\t\t}\n\t}\n\n\treturn checkPodsForFSBackup(pods, pvcName, defaultVolumesToFsBackup)\n}\n\n// checkPodsForFSBackup is a helper function that checks if any pod using the PVC\n// has the volume selected for fs-backup.\nfunc checkPodsForFSBackup(pods []corev1api.Pod, pvcName string, defaultVolumesToFsBackup bool) (bool, error) {\n\tfor index := range pods {\n\t\tvols, _ := GetVolumesByPod(&pods[index], defaultVolumesToFsBackup, false, []string{})\n\t\tif len(vols) > 0 {\n\t\t\tvolName, err := getPodVolumeNameForPVC(pods[index], pvcName)\n\t\t\tif err != nil {\n\t\t\t\treturn false, err\n\t\t\t}\n\t\t\tif util.Contains(vols, volName) {\n\t\t\t\treturn true, nil\n\t\t\t}\n\t\t}\n\t}\n\n\treturn false, nil\n}\n\nfunc getPodVolumeNameForPVC(pod corev1api.Pod, pvcName string) (string, error) {\n\tfor _, v := range pod.Spec.Volumes {\n\t\tif v.PersistentVolumeClaim != nil && v.PersistentVolumeClaim.ClaimName == pvcName {\n\t\t\treturn v.Name, nil\n\t\t}\n\t}\n\treturn \"\", errors.Errorf(\"Pod %s/%s does not use PVC %s/%s\", pod.Namespace, pod.Name, pod.Namespace, pvcName)\n}\n\n// GetPodsUsingPVCWithCache returns all pods that use the specified PVC.\n// If cache is available and built, it uses the cache for O(1) lookup.\n// Otherwise, it falls back to listing pods directly.\n// Note: In the main backup path, the cache is always built (via NewVolumeHelperImplWithNamespaces),\n// so the fallback is only used by plugins that don't need cache optimization.\nfunc GetPodsUsingPVCWithCache(\n\tpvcNamespace, pvcName string,\n\tcrClient crclient.Client,\n\tcache *PVCPodCache,\n) ([]corev1api.Pod, error) {\n\t// Use cache if available\n\tif cache != nil && cache.IsBuilt() {\n\t\tpods := cache.GetPodsUsingPVC(pvcNamespace, pvcName)\n\t\tif pods == nil {\n\t\t\treturn []corev1api.Pod{}, nil\n\t\t}\n\t\treturn pods, nil\n\t}\n\n\t// Fall back to direct lookup (for plugins without cache)\n\treturn getPodsUsingPVCDirect(pvcNamespace, pvcName, crClient)\n}\n\n// getPodsUsingPVCDirect returns all pods in the given namespace that use the specified PVC.\n// This is an internal function that lists all pods in the namespace and filters them.\nfunc getPodsUsingPVCDirect(\n\tpvcNamespace, pvcName string,\n\tcrClient crclient.Client,\n) ([]corev1api.Pod, error) {\n\tpodsUsingPVC := []corev1api.Pod{}\n\tpodList := new(corev1api.PodList)\n\tif err := crClient.List(\n\t\tcontext.TODO(),\n\t\tpodList,\n\t\t&crclient.ListOptions{Namespace: pvcNamespace},\n\t); err != nil {\n\t\treturn nil, err\n\t}\n\n\tfor _, p := range podList.Items {\n\t\tfor _, v := range p.Spec.Volumes {\n\t\t\tif v.PersistentVolumeClaim != nil && v.PersistentVolumeClaim.ClaimName == pvcName {\n\t\t\t\tpodsUsingPVC = append(podsUsingPVC, p)\n\t\t\t}\n\t\t}\n\t}\n\n\treturn podsUsingPVC, nil\n}\n\nfunc GetVolumesToProcess(volumes []corev1api.Volume, volsToProcessByLegacyApproach []string) []corev1api.Volume {\n\tvolsToProcess := make([]corev1api.Volume, 0)\n\n\t// return empty list when no volumes associated with pod\n\tif len(volumes) == 0 {\n\t\treturn volsToProcess\n\t}\n\n\t// legacy approach as a fallback option case\n\tif len(volsToProcessByLegacyApproach) > 0 {\n\t\tfor _, vol := range volumes {\n\t\t\t// don't process volumes that are already matched for supported action in volume policy approach\n\t\t\tif !util.Contains(volsToProcessByLegacyApproach, vol.Name) {\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\t// add volume that is not processed in volume policy approach\n\t\t\tvolsToProcess = append(volsToProcess, vol)\n\t\t}\n\n\t\treturn volsToProcess\n\t}\n\t// normal case return the list as in\n\treturn volumes\n}\n" }, { "path": "pkg/util/podvolume/pod_volume_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage podvolume\n\nimport (\n\t\"sort\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\tvelerotest \"github.com/vmware-tanzu/velero/pkg/test\"\n)\n\nfunc TestGetVolumesToBackup(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tannotations map[string]string\n\t\texpected []string\n\t}{\n\t\t{\n\t\t\tname: \"nil annotations\",\n\t\t\tannotations: nil,\n\t\t\texpected: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"no volumes to backup\",\n\t\t\tannotations: map[string]string{\"foo\": \"bar\"},\n\t\t\texpected: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"one volume to backup\",\n\t\t\tannotations: map[string]string{\"foo\": \"bar\", velerov1api.VolumesToBackupAnnotation: \"volume-1\"},\n\t\t\texpected: []string{\"volume-1\"},\n\t\t},\n\t\t{\n\t\t\tname: \"multiple volumes to backup\",\n\t\t\tannotations: map[string]string{\"foo\": \"bar\", velerov1api.VolumesToBackupAnnotation: \"volume-1,volume-2,volume-3\"},\n\t\t\texpected: []string{\"volume-1\", \"volume-2\", \"volume-3\"},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tpod := &corev1api.Pod{}\n\t\t\tpod.Annotations = test.annotations\n\n\t\t\tres := GetVolumesToBackup(pod)\n\n\t\t\t// sort to ensure good compare of slices\n\t\t\tsort.Strings(test.expected)\n\t\t\tsort.Strings(res)\n\n\t\t\tassert.Equal(t, test.expected, res)\n\t\t})\n\t}\n}\n\nfunc TestGetVolumesByPod(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tpod *corev1api.Pod\n\t\texpected struct {\n\t\t\tincluded []string\n\t\t\toptedOut []string\n\t\t}\n\t\tdefaultVolumesToFsBackup bool\n\t\tbackupExcludePVC bool\n\t}{\n\t\t{\n\t\t\tname: \"should get PVs from VolumesToBackupAnnotation when defaultVolumesToFsBackup is false\",\n\t\t\tdefaultVolumesToFsBackup: false,\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tvelerov1api.VolumesToBackupAnnotation: \"pvbPV1,pvbPV2,pvbPV3\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: struct {\n\t\t\t\tincluded []string\n\t\t\t\toptedOut []string\n\t\t\t}{\n\t\t\t\tincluded: []string{\"pvbPV1\", \"pvbPV2\", \"pvbPV3\"},\n\t\t\t\toptedOut: []string{},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should get all pod volumes when defaultVolumesToFsBackup is true and no PVs are excluded\",\n\t\t\tdefaultVolumesToFsBackup: true,\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t// PVB Volumes\n\t\t\t\t\t\t{Name: \"pvbPV1\"}, {Name: \"pvbPV2\"}, {Name: \"pvbPV3\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: struct {\n\t\t\t\tincluded []string\n\t\t\t\toptedOut []string\n\t\t\t}{\n\t\t\t\tincluded: []string{\"pvbPV1\", \"pvbPV2\", \"pvbPV3\"},\n\t\t\t\toptedOut: []string{},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should get all pod volumes except ones excluded when defaultVolumesToFsBackup is true\",\n\t\t\tdefaultVolumesToFsBackup: true,\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tvelerov1api.VolumesToExcludeAnnotation: \"nonPvbPV1,nonPvbPV2,nonPvbPV3\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t// PVB Volumes\n\t\t\t\t\t\t{Name: \"pvbPV1\"}, {Name: \"pvbPV2\"}, {Name: \"pvbPV3\"},\n\t\t\t\t\t\t/// Excluded from PVB through annotation\n\t\t\t\t\t\t{Name: \"nonPvbPV1\"}, {Name: \"nonPvbPV2\"}, {Name: \"nonPvbPV3\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: struct {\n\t\t\t\tincluded []string\n\t\t\t\toptedOut []string\n\t\t\t}{\n\t\t\t\tincluded: []string{\"pvbPV1\", \"pvbPV2\", \"pvbPV3\"},\n\t\t\t\toptedOut: []string{\"nonPvbPV1\", \"nonPvbPV2\", \"nonPvbPV3\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should exclude default service account token from pod volume backup\",\n\t\t\tdefaultVolumesToFsBackup: true,\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t// PVB Volumes\n\t\t\t\t\t\t{Name: \"pvbPV1\"}, {Name: \"pvbPV2\"}, {Name: \"pvbPV3\"},\n\t\t\t\t\t\t/// Excluded from PVB because volume mounting default service account token\n\t\t\t\t\t\t{Name: \"default-token-5xq45\"},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: struct {\n\t\t\t\tincluded []string\n\t\t\t\toptedOut []string\n\t\t\t}{\n\t\t\t\tincluded: []string{\"pvbPV1\", \"pvbPV2\", \"pvbPV3\"},\n\t\t\t\toptedOut: []string{},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should exclude host path volumes from pod volume backups\",\n\t\t\tdefaultVolumesToFsBackup: true,\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tvelerov1api.VolumesToExcludeAnnotation: \"nonPvbPV1,nonPvbPV2,nonPvbPV3\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t// PVB Volumes\n\t\t\t\t\t\t{Name: \"pvbPV1\"}, {Name: \"pvbPV2\"}, {Name: \"pvbPV3\"},\n\t\t\t\t\t\t/// Excluded from pod volume backup through annotation\n\t\t\t\t\t\t{Name: \"nonPvbPV1\"}, {Name: \"nonPvbPV2\"}, {Name: \"nonPvbPV3\"},\n\t\t\t\t\t\t// Excluded from pod volume backup because hostpath\n\t\t\t\t\t\t{Name: \"hostPath1\", VolumeSource: corev1api.VolumeSource{HostPath: &corev1api.HostPathVolumeSource{Path: \"/hostpathVol\"}}},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: struct {\n\t\t\t\tincluded []string\n\t\t\t\toptedOut []string\n\t\t\t}{\n\t\t\t\tincluded: []string{\"pvbPV1\", \"pvbPV2\", \"pvbPV3\"},\n\t\t\t\toptedOut: []string{\"nonPvbPV1\", \"nonPvbPV2\", \"nonPvbPV3\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should exclude volumes mounting secrets\",\n\t\t\tdefaultVolumesToFsBackup: true,\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tvelerov1api.VolumesToExcludeAnnotation: \"nonPvbPV1,nonPvbPV2,nonPvbPV3\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t// PVB Volumes\n\t\t\t\t\t\t{Name: \"pvbPV1\"}, {Name: \"pvbPV2\"}, {Name: \"pvbPV3\"},\n\t\t\t\t\t\t/// Excluded from pod volume backup through annotation\n\t\t\t\t\t\t{Name: \"nonPvbPV1\"}, {Name: \"nonPvbPV2\"}, {Name: \"nonPvbPV3\"},\n\t\t\t\t\t\t// Excluded from pod volume backup because hostpath\n\t\t\t\t\t\t{Name: \"superSecret\", VolumeSource: corev1api.VolumeSource{Secret: &corev1api.SecretVolumeSource{SecretName: \"super-secret\"}}},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: struct {\n\t\t\t\tincluded []string\n\t\t\t\toptedOut []string\n\t\t\t}{\n\t\t\t\tincluded: []string{\"pvbPV1\", \"pvbPV2\", \"pvbPV3\"},\n\t\t\t\toptedOut: []string{\"nonPvbPV1\", \"nonPvbPV2\", \"nonPvbPV3\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should exclude volumes mounting ConfigMaps\",\n\t\t\tdefaultVolumesToFsBackup: true,\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tvelerov1api.VolumesToExcludeAnnotation: \"nonPvbPV1,nonPvbPV2,nonPvbPV3\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t// PVB Volumes\n\t\t\t\t\t\t{Name: \"pvbPV1\"}, {Name: \"pvbPV2\"}, {Name: \"pvbPV3\"},\n\t\t\t\t\t\t/// Excluded from pod volume backup through annotation\n\t\t\t\t\t\t{Name: \"nonPvbPV1\"}, {Name: \"nonPvbPV2\"}, {Name: \"nonPvbPV3\"},\n\t\t\t\t\t\t// Excluded from pod volume backup because hostpath\n\t\t\t\t\t\t{Name: \"appCOnfig\", VolumeSource: corev1api.VolumeSource{ConfigMap: &corev1api.ConfigMapVolumeSource{LocalObjectReference: corev1api.LocalObjectReference{Name: \"app-config\"}}}},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: struct {\n\t\t\t\tincluded []string\n\t\t\t\toptedOut []string\n\t\t\t}{\n\t\t\t\tincluded: []string{\"pvbPV1\", \"pvbPV2\", \"pvbPV3\"},\n\t\t\t\toptedOut: []string{\"nonPvbPV1\", \"nonPvbPV2\", \"nonPvbPV3\"},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should exclude projected volumes\",\n\t\t\tdefaultVolumesToFsBackup: true,\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tvelerov1api.VolumesToExcludeAnnotation: \"nonPvbPV1,nonPvbPV2,nonPvbPV3\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{Name: \"pvbPV1\"}, {Name: \"pvbPV2\"}, {Name: \"pvbPV3\"},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"projected\",\n\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\tProjected: &corev1api.ProjectedVolumeSource{\n\t\t\t\t\t\t\t\t\tSources: []corev1api.VolumeProjection{{\n\t\t\t\t\t\t\t\t\t\tSecret: &corev1api.SecretProjection{\n\t\t\t\t\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{},\n\t\t\t\t\t\t\t\t\t\t\tItems: nil,\n\t\t\t\t\t\t\t\t\t\t\tOptional: nil,\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t\tDownwardAPI: nil,\n\t\t\t\t\t\t\t\t\t\tConfigMap: nil,\n\t\t\t\t\t\t\t\t\t\tServiceAccountToken: nil,\n\t\t\t\t\t\t\t\t\t}},\n\t\t\t\t\t\t\t\t\tDefaultMode: nil,\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: struct {\n\t\t\t\tincluded []string\n\t\t\t\toptedOut []string\n\t\t\t}{\n\t\t\t\tincluded: []string{\"pvbPV1\", \"pvbPV2\", \"pvbPV3\"},\n\t\t\t\toptedOut: []string{},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should exclude DownwardAPI volumes\",\n\t\t\tdefaultVolumesToFsBackup: true,\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tvelerov1api.VolumesToExcludeAnnotation: \"nonPvbPV1,nonPvbPV2,nonPvbPV3\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{Name: \"pvbPV1\"}, {Name: \"pvbPV2\"}, {Name: \"pvbPV3\"},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"downwardAPI\",\n\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\tDownwardAPI: &corev1api.DownwardAPIVolumeSource{\n\t\t\t\t\t\t\t\t\tItems: []corev1api.DownwardAPIVolumeFile{\n\t\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\t\tPath: \"labels\",\n\t\t\t\t\t\t\t\t\t\t\tFieldRef: &corev1api.ObjectFieldSelector{\n\t\t\t\t\t\t\t\t\t\t\t\tAPIVersion: \"v1\",\n\t\t\t\t\t\t\t\t\t\t\t\tFieldPath: \"metadata.labels\",\n\t\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: struct {\n\t\t\t\tincluded []string\n\t\t\t\toptedOut []string\n\t\t\t}{\n\t\t\t\tincluded: []string{\"pvbPV1\", \"pvbPV2\", \"pvbPV3\"},\n\t\t\t\toptedOut: []string{},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"should exclude PVC volume when backup excludes PVC resource\",\n\t\t\tdefaultVolumesToFsBackup: true,\n\t\t\tbackupExcludePVC: true,\n\t\t\tpod: &corev1api.Pod{\n\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\tvelerov1api.VolumesToExcludeAnnotation: \"nonPvbPV1,nonPvbPV2,nonPvbPV3\",\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{Name: \"pvbPV1\"}, {Name: \"pvbPV2\"}, {Name: \"pvbPV3\"},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"downwardAPI\",\n\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\t\tClaimName: \"testPVC\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: struct {\n\t\t\t\tincluded []string\n\t\t\t\toptedOut []string\n\t\t\t}{\n\t\t\t\tincluded: []string{\"pvbPV1\", \"pvbPV2\", \"pvbPV3\"},\n\t\t\t\toptedOut: []string{},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactualIncluded, actualOptedOut := GetVolumesByPod(tc.pod, tc.defaultVolumesToFsBackup, tc.backupExcludePVC, []string{})\n\n\t\t\tsort.Strings(tc.expected.included)\n\t\t\tsort.Strings(actualIncluded)\n\t\t\tassert.Equal(t, tc.expected.included, actualIncluded)\n\n\t\t\tsort.Strings(tc.expected.optedOut)\n\t\t\tsort.Strings(actualOptedOut)\n\t\t\tassert.Equal(t, tc.expected.optedOut, actualOptedOut)\n\t\t})\n\t}\n}\n\nfunc TestGetPodVolumeNameForPVC(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tpod corev1api.Pod\n\t\tpvcName string\n\t\texpectError bool\n\t\texpectedVolumeName string\n\t}{\n\t\t{\n\t\t\tname: \"should get volume name for pod with multiple PVCs\",\n\t\t\tpod: corev1api.Pod{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"csi-vol1\",\n\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\t\tClaimName: \"csi-pvc1\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"csi-vol2\",\n\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\t\tClaimName: \"csi-pvc2\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"csi-vol3\",\n\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\t\tClaimName: \"csi-pvc3\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvcName: \"csi-pvc2\",\n\t\t\texpectedVolumeName: \"csi-vol2\",\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should get volume name from pod using exactly one PVC\",\n\t\t\tpod: corev1api.Pod{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"csi-vol1\",\n\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\t\tClaimName: \"csi-pvc1\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvcName: \"csi-pvc1\",\n\t\t\texpectedVolumeName: \"csi-vol1\",\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should return error for pod with no PVCs\",\n\t\t\tpod: corev1api.Pod{\n\t\t\t\tSpec: corev1api.PodSpec{},\n\t\t\t},\n\t\t\tpvcName: \"csi-pvc2\",\n\t\t\texpectError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"should return error for pod with no matching PVC\",\n\t\t\tpod: corev1api.Pod{\n\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: \"csi-vol1\",\n\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\t\tClaimName: \"csi-pvc1\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tpvcName: \"mismatch-pvc\",\n\t\t\texpectError: true,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactualVolumeName, err := getPodVolumeNameForPVC(tc.pod, tc.pvcName)\n\t\t\tif tc.expectError && err == nil {\n\t\t\t\tassert.Error(t, err, \"Want error; Got nil error\")\n\t\t\t\treturn\n\t\t\t}\n\t\t\tassert.Equalf(t, tc.expectedVolumeName, actualVolumeName, \"unexpected podVolumename returned. Want %s; Got %s\", tc.expectedVolumeName, actualVolumeName)\n\t\t})\n\t}\n}\n\nfunc TestGetVolumesToProcess(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tvolumes []corev1api.Volume\n\t\tvolsToProcessByLegacyApproach []string\n\t\texpectedVolumes []corev1api.Volume\n\t}{\n\t\t{\n\t\t\tname: \"pod has 2 volumes empty volsToProcessByLegacyApproach list return 2 volumes\",\n\t\t\tvolumes: []corev1api.Volume{\n\t\t\t\t{\n\t\t\t\t\tName: \"sample-volume-1\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\tClaimName: \"sample-pvc-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"sample-volume-2\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\tClaimName: \"sample-pvc-2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolsToProcessByLegacyApproach: []string{},\n\t\t\texpectedVolumes: []corev1api.Volume{\n\t\t\t\t{\n\t\t\t\t\tName: \"sample-volume-1\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\tClaimName: \"sample-pvc-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"sample-volume-2\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\tClaimName: \"sample-pvc-2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"pod has 2 volumes non-empty volsToProcessByLegacyApproach list returns 1 volumes\",\n\t\t\tvolumes: []corev1api.Volume{\n\t\t\t\t{\n\t\t\t\t\tName: \"sample-volume-1\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\tClaimName: \"sample-pvc-1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"sample-volume-2\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\tClaimName: \"sample-pvc-2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\tvolsToProcessByLegacyApproach: []string{\"sample-volume-2\"},\n\t\t\texpectedVolumes: []corev1api.Volume{\n\t\t\t\t{\n\t\t\t\t\tName: \"sample-volume-2\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\tClaimName: \"sample-pvc-2\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"empty case, return empty list\",\n\t\t\tvolumes: []corev1api.Volume{},\n\t\t\tvolsToProcessByLegacyApproach: []string{},\n\t\t\texpectedVolumes: []corev1api.Volume{},\n\t\t},\n\t}\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactualVolumes := GetVolumesToProcess(tc.volumes, tc.volsToProcessByLegacyApproach)\n\t\t\tassert.Equal(t, tc.expectedVolumes, actualVolumes, \"Want Volumes List %v; Got Volumes List %v\", tc.expectedVolumes, actualVolumes)\n\t\t})\n\t}\n}\n\nfunc TestPVCPodCache_BuildAndGet(t *testing.T) {\n\tobjs := []runtime.Object{\n\t\t&corev1api.Pod{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"pod1\",\n\t\t\t\tNamespace: \"default\",\n\t\t\t},\n\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"vol1\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t&corev1api.Pod{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"pod2\",\n\t\t\t\tNamespace: \"default\",\n\t\t\t},\n\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"vol1\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"vol2\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc2\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t&corev1api.Pod{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"pod3\",\n\t\t\t\tNamespace: \"default\",\n\t\t\t},\n\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"vol1\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tEmptyDir: &corev1api.EmptyDirVolumeSource{},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t&corev1api.Pod{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"pod4\",\n\t\t\t\tNamespace: \"other-ns\",\n\t\t\t},\n\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"vol1\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, objs...)\n\n\ttestCases := []struct {\n\t\tname string\n\t\tnamespaces []string\n\t\tpvcNamespace string\n\t\tpvcName string\n\t\texpectedPodCount int\n\t}{\n\t\t{\n\t\t\tname: \"should find 2 pods using pvc1 in default namespace\",\n\t\t\tnamespaces: []string{\"default\", \"other-ns\"},\n\t\t\tpvcNamespace: \"default\",\n\t\t\tpvcName: \"pvc1\",\n\t\t\texpectedPodCount: 2,\n\t\t},\n\t\t{\n\t\t\tname: \"should find 1 pod using pvc2 in default namespace\",\n\t\t\tnamespaces: []string{\"default\", \"other-ns\"},\n\t\t\tpvcNamespace: \"default\",\n\t\t\tpvcName: \"pvc2\",\n\t\t\texpectedPodCount: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"should find 1 pod using pvc1 in other-ns\",\n\t\t\tnamespaces: []string{\"default\", \"other-ns\"},\n\t\t\tpvcNamespace: \"other-ns\",\n\t\t\tpvcName: \"pvc1\",\n\t\t\texpectedPodCount: 1,\n\t\t},\n\t\t{\n\t\t\tname: \"should find 0 pods for non-existent PVC\",\n\t\t\tnamespaces: []string{\"default\", \"other-ns\"},\n\t\t\tpvcNamespace: \"default\",\n\t\t\tpvcName: \"non-existent\",\n\t\t\texpectedPodCount: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"should find 0 pods for non-existent namespace\",\n\t\t\tnamespaces: []string{\"default\", \"other-ns\"},\n\t\t\tpvcNamespace: \"non-existent-ns\",\n\t\t\tpvcName: \"pvc1\",\n\t\t\texpectedPodCount: 0,\n\t\t},\n\t\t{\n\t\t\tname: \"should find 0 pods when namespace not in cache\",\n\t\t\tnamespaces: []string{\"default\"},\n\t\t\tpvcNamespace: \"other-ns\",\n\t\t\tpvcName: \"pvc1\",\n\t\t\texpectedPodCount: 0,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tcache := NewPVCPodCache()\n\t\t\terr := cache.BuildCacheForNamespaces(t.Context(), tc.namespaces, fakeClient)\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.True(t, cache.IsBuilt())\n\n\t\t\tpods := cache.GetPodsUsingPVC(tc.pvcNamespace, tc.pvcName)\n\t\t\tassert.Len(t, pods, tc.expectedPodCount, \"unexpected number of pods\")\n\t\t})\n\t}\n}\n\nfunc TestGetPodsUsingPVCWithCache(t *testing.T) {\n\tobjs := []runtime.Object{\n\t\t&corev1api.Pod{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"pod1\",\n\t\t\t\tNamespace: \"default\",\n\t\t\t},\n\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"vol1\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t&corev1api.Pod{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"pod2\",\n\t\t\t\tNamespace: \"default\",\n\t\t\t},\n\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"vol1\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, objs...)\n\n\ttestCases := []struct {\n\t\tname string\n\t\tpvcNamespace string\n\t\tpvcName string\n\t\tbuildCache bool\n\t\tuseNilCache bool\n\t\texpectedPodCount int\n\t}{\n\t\t{\n\t\t\tname: \"returns cached results when cache is available\",\n\t\t\tpvcNamespace: \"default\",\n\t\t\tpvcName: \"pvc1\",\n\t\t\tbuildCache: true,\n\t\t\tuseNilCache: false,\n\t\t\texpectedPodCount: 2,\n\t\t},\n\t\t{\n\t\t\tname: \"falls back to direct lookup when cache is nil\",\n\t\t\tpvcNamespace: \"default\",\n\t\t\tpvcName: \"pvc1\",\n\t\t\tbuildCache: false,\n\t\t\tuseNilCache: true,\n\t\t\texpectedPodCount: 2,\n\t\t},\n\t\t{\n\t\t\tname: \"falls back to direct lookup when cache is not built\",\n\t\t\tpvcNamespace: \"default\",\n\t\t\tpvcName: \"pvc1\",\n\t\t\tbuildCache: false,\n\t\t\tuseNilCache: false,\n\t\t\texpectedPodCount: 2,\n\t\t},\n\t\t{\n\t\t\tname: \"returns empty slice for non-existent PVC with cache\",\n\t\t\tpvcNamespace: \"default\",\n\t\t\tpvcName: \"non-existent\",\n\t\t\tbuildCache: true,\n\t\t\tuseNilCache: false,\n\t\t\texpectedPodCount: 0,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar cache *PVCPodCache\n\t\t\tif !tc.useNilCache {\n\t\t\t\tcache = NewPVCPodCache()\n\t\t\t\tif tc.buildCache {\n\t\t\t\t\terr := cache.BuildCacheForNamespaces(t.Context(), []string{\"default\"}, fakeClient)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tpods, err := GetPodsUsingPVCWithCache(tc.pvcNamespace, tc.pvcName, fakeClient, cache)\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Len(t, pods, tc.expectedPodCount, \"unexpected number of pods\")\n\t\t})\n\t}\n}\n\nfunc TestIsPVCDefaultToFSBackupWithCache(t *testing.T) {\n\tobjs := []runtime.Object{\n\t\t&corev1api.Pod{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"pod1\",\n\t\t\t\tNamespace: \"default\",\n\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\"backup.velero.io/backup-volumes\": \"vol1\",\n\t\t\t\t},\n\t\t\t},\n\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"vol1\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc1\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t&corev1api.Pod{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: \"pod2\",\n\t\t\t\tNamespace: \"default\",\n\t\t\t},\n\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t{\n\t\t\t\t\t\tName: \"vol1\",\n\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\tClaimName: \"pvc2\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, objs...)\n\n\ttestCases := []struct {\n\t\tname string\n\t\tpvcNamespace string\n\t\tpvcName string\n\t\tdefaultVolumesToFsBackup bool\n\t\tbuildCache bool\n\t\tuseNilCache bool\n\t\texpectedResult bool\n\t}{\n\t\t{\n\t\t\tname: \"returns true for PVC with opt-in annotation using cache\",\n\t\t\tpvcNamespace: \"default\",\n\t\t\tpvcName: \"pvc1\",\n\t\t\tdefaultVolumesToFsBackup: false,\n\t\t\tbuildCache: true,\n\t\t\tuseNilCache: false,\n\t\t\texpectedResult: true,\n\t\t},\n\t\t{\n\t\t\tname: \"returns false for PVC without annotation using cache\",\n\t\t\tpvcNamespace: \"default\",\n\t\t\tpvcName: \"pvc2\",\n\t\t\tdefaultVolumesToFsBackup: false,\n\t\t\tbuildCache: true,\n\t\t\tuseNilCache: false,\n\t\t\texpectedResult: false,\n\t\t},\n\t\t{\n\t\t\tname: \"returns true for any PVC with defaultVolumesToFsBackup using cache\",\n\t\t\tpvcNamespace: \"default\",\n\t\t\tpvcName: \"pvc2\",\n\t\t\tdefaultVolumesToFsBackup: true,\n\t\t\tbuildCache: true,\n\t\t\tuseNilCache: false,\n\t\t\texpectedResult: true,\n\t\t},\n\t\t{\n\t\t\tname: \"falls back to direct lookup when cache is nil\",\n\t\t\tpvcNamespace: \"default\",\n\t\t\tpvcName: \"pvc1\",\n\t\t\tdefaultVolumesToFsBackup: false,\n\t\t\tbuildCache: false,\n\t\t\tuseNilCache: true,\n\t\t\texpectedResult: true,\n\t\t},\n\t\t{\n\t\t\tname: \"returns false for non-existent PVC\",\n\t\t\tpvcNamespace: \"default\",\n\t\t\tpvcName: \"non-existent\",\n\t\t\tdefaultVolumesToFsBackup: false,\n\t\t\tbuildCache: true,\n\t\t\tuseNilCache: false,\n\t\t\texpectedResult: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tvar cache *PVCPodCache\n\t\t\tif !tc.useNilCache {\n\t\t\t\tcache = NewPVCPodCache()\n\t\t\t\tif tc.buildCache {\n\t\t\t\t\terr := cache.BuildCacheForNamespaces(t.Context(), []string{\"default\"}, fakeClient)\n\t\t\t\t\trequire.NoError(t, err)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tresult, err := IsPVCDefaultToFSBackupWithCache(tc.pvcNamespace, tc.pvcName, fakeClient, tc.defaultVolumesToFsBackup, cache)\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.Equal(t, tc.expectedResult, result)\n\t\t})\n\t}\n}\n\n// TestIsNamespaceBuilt tests the IsNamespaceBuilt method for lazy per-namespace caching.\nfunc TestIsNamespaceBuilt(t *testing.T) {\n\tcache := NewPVCPodCache()\n\n\t// Initially no namespace should be built\n\tassert.False(t, cache.IsNamespaceBuilt(\"ns1\"), \"namespace should not be built initially\")\n\tassert.False(t, cache.IsNamespaceBuilt(\"ns2\"), \"namespace should not be built initially\")\n\n\t// Create a fake client with a pod in ns1\n\tpod := &corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\tName: \"test-pod\",\n\t\t\tNamespace: \"ns1\",\n\t\t},\n\t\tSpec: corev1api.PodSpec{\n\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t{\n\t\t\t\t\tName: \"vol1\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\tClaimName: \"pvc1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, pod)\n\n\t// Build cache for ns1\n\terr := cache.BuildCacheForNamespace(t.Context(), \"ns1\", fakeClient)\n\trequire.NoError(t, err)\n\n\t// ns1 should be built, ns2 should not\n\tassert.True(t, cache.IsNamespaceBuilt(\"ns1\"), \"namespace ns1 should be built\")\n\tassert.False(t, cache.IsNamespaceBuilt(\"ns2\"), \"namespace ns2 should not be built\")\n\n\t// Build cache for ns2 (empty namespace)\n\terr = cache.BuildCacheForNamespace(t.Context(), \"ns2\", fakeClient)\n\trequire.NoError(t, err)\n\n\t// Both should now be built\n\tassert.True(t, cache.IsNamespaceBuilt(\"ns1\"), \"namespace ns1 should still be built\")\n\tassert.True(t, cache.IsNamespaceBuilt(\"ns2\"), \"namespace ns2 should now be built\")\n}\n\n// TestBuildCacheForNamespace tests the lazy per-namespace cache building.\nfunc TestBuildCacheForNamespace(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpods []runtime.Object\n\t\tnamespace string\n\t\texpectedPVCs map[string]int // pvcName -> expected pod count\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"build cache for namespace with pods using PVCs\",\n\t\t\tnamespace: \"ns1\",\n\t\t\tpods: []runtime.Object{\n\t\t\t\t&corev1api.Pod{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pod1\", Namespace: \"ns1\"},\n\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tName: \"vol1\",\n\t\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\t\t\tClaimName: \"pvc1\",\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t&corev1api.Pod{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pod2\", Namespace: \"ns1\"},\n\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tName: \"vol1\",\n\t\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\t\t\tClaimName: \"pvc1\",\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedPVCs: map[string]int{\"pvc1\": 2},\n\t\t},\n\t\t{\n\t\t\tname: \"build cache for empty namespace\",\n\t\t\tnamespace: \"empty-ns\",\n\t\t\tpods: []runtime.Object{},\n\t\t\texpectedPVCs: map[string]int{},\n\t\t},\n\t\t{\n\t\t\tname: \"build cache ignores pods without PVCs\",\n\t\t\tnamespace: \"ns1\",\n\t\t\tpods: []runtime.Object{\n\t\t\t\t&corev1api.Pod{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pod1\", Namespace: \"ns1\"},\n\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tName: \"config-vol\",\n\t\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\t\tConfigMap: &corev1api.ConfigMapVolumeSource{\n\t\t\t\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\t\t\t\tName: \"my-config\",\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedPVCs: map[string]int{},\n\t\t},\n\t\t{\n\t\t\tname: \"build cache only for specified namespace\",\n\t\t\tnamespace: \"ns1\",\n\t\t\tpods: []runtime.Object{\n\t\t\t\t&corev1api.Pod{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pod1\", Namespace: \"ns1\"},\n\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tName: \"vol1\",\n\t\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\t\t\tClaimName: \"pvc1\",\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t&corev1api.Pod{\n\t\t\t\t\tObjectMeta: metav1.ObjectMeta{Name: \"pod2\", Namespace: \"ns2\"},\n\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tName: \"vol1\",\n\t\t\t\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\t\t\t\tClaimName: \"pvc2\",\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpectedPVCs: map[string]int{\"pvc1\": 1},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, tc.pods...)\n\t\t\tcache := NewPVCPodCache()\n\n\t\t\t// Build cache for the namespace\n\t\t\terr := cache.BuildCacheForNamespace(t.Context(), tc.namespace, fakeClient)\n\t\t\tif tc.expectError {\n\t\t\t\trequire.Error(t, err)\n\t\t\t\treturn\n\t\t\t}\n\t\t\trequire.NoError(t, err)\n\n\t\t\t// Verify namespace is marked as built\n\t\t\tassert.True(t, cache.IsNamespaceBuilt(tc.namespace))\n\n\t\t\t// Verify PVC to pod mappings\n\t\t\tfor pvcName, expectedCount := range tc.expectedPVCs {\n\t\t\t\tpods := cache.GetPodsUsingPVC(tc.namespace, pvcName)\n\t\t\t\tassert.Len(t, pods, expectedCount, \"unexpected pod count for PVC %s\", pvcName)\n\t\t\t}\n\n\t\t\t// Calling BuildCacheForNamespace again should be a no-op\n\t\t\terr = cache.BuildCacheForNamespace(t.Context(), tc.namespace, fakeClient)\n\t\t\trequire.NoError(t, err)\n\t\t})\n\t}\n}\n\n// TestBuildCacheForNamespaceIdempotent verifies that building cache multiple times is safe.\nfunc TestBuildCacheForNamespaceIdempotent(t *testing.T) {\n\tpod := &corev1api.Pod{\n\t\tObjectMeta: metav1.ObjectMeta{Name: \"pod1\", Namespace: \"ns1\"},\n\t\tSpec: corev1api.PodSpec{\n\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t{\n\t\t\t\t\tName: \"vol1\",\n\t\t\t\t\tVolumeSource: corev1api.VolumeSource{\n\t\t\t\t\t\tPersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{\n\t\t\t\t\t\t\tClaimName: \"pvc1\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\tfakeClient := velerotest.NewFakeControllerRuntimeClient(t, pod)\n\tcache := NewPVCPodCache()\n\n\t// Build cache multiple times - should be idempotent\n\tfor i := 0; i < 3; i++ {\n\t\terr := cache.BuildCacheForNamespace(t.Context(), \"ns1\", fakeClient)\n\t\trequire.NoError(t, err)\n\t\tassert.True(t, cache.IsNamespaceBuilt(\"ns1\"))\n\n\t\tpods := cache.GetPodsUsingPVC(\"ns1\", \"pvc1\")\n\t\tassert.Len(t, pods, 1, \"should have exactly 1 pod using pvc1\")\n\t}\n}\n" }, { "path": "pkg/util/results/result.go", "content": "/*\nCopyright 2019, 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage results\n\n// Result is a collection of messages that were generated during\n// execution of a backup or restore. This will typically store either\n// warning or error messages.\ntype Result struct {\n\t// Velero is a slice of messages related to the operation of Velero\n\t// itself (for example, messages related to connecting to the\n\t// cloud, reading a backup file, etc.)\n\tVelero []string `json:\"velero,omitempty\"`\n\n\t// Cluster is a slice of messages related to backup or restore of\n\t// cluster-scoped resources.\n\tCluster []string `json:\"cluster,omitempty\"`\n\n\t// Namespaces is a map of namespace name to slice of messages\n\t// related to backup or restore namespace-scoped resources.\n\tNamespaces map[string][]string `json:\"namespaces,omitempty\"`\n}\n\n// Merge combines two Result objects into one\n// by appending the corresponding lists to one another.\nfunc (r *Result) Merge(other *Result) {\n\tr.Cluster = append(r.Cluster, other.Cluster...)\n\tr.Velero = append(r.Velero, other.Velero...)\n\tfor k, v := range other.Namespaces {\n\t\tif r.Namespaces == nil {\n\t\t\tr.Namespaces = make(map[string][]string)\n\t\t}\n\t\tr.Namespaces[k] = append(r.Namespaces[k], v...)\n\t}\n}\n\n// AddVeleroError appends an error to the provided Result's Velero list.\nfunc (r *Result) AddVeleroError(err error) {\n\tr.Velero = append(r.Velero, err.Error())\n}\n\n// Add appends an error to the provided Result, either within\n// the cluster-scoped list (if ns == \"\") or within the provided namespace's\n// entry.\nfunc (r *Result) Add(ns string, e error) {\n\tif ns == \"\" {\n\t\tr.Cluster = append(r.Cluster, e.Error())\n\t} else {\n\t\tif r.Namespaces == nil {\n\t\t\tr.Namespaces = make(map[string][]string)\n\t\t}\n\t\tr.Namespaces[ns] = append(r.Namespaces[ns], e.Error())\n\t}\n}\n\n// IsEmpty returns true if all collections of messages are empty\nfunc (r *Result) IsEmpty() bool {\n\treturn len(r.Velero) == 0 && len(r.Cluster) == 0 && len(r.Namespaces) == 0\n}\n" }, { "path": "pkg/util/results/result_test.go", "content": "/*\nCopyright 2020 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage results\n\nimport (\n\t\"testing\"\n\n\t\"github.com/pkg/errors\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestMerge(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tresult *Result\n\t\tother *Result\n\t\twant *Result\n\t}{\n\t\t{\n\t\t\tname: \"when an empty result is merged into a non-empty result, the result does not change\",\n\t\t\tresult: &Result{\n\t\t\t\tCluster: []string{\"foo\"},\n\t\t\t\tNamespaces: map[string][]string{\n\t\t\t\t\t\"ns-1\": {\"bar\"},\n\t\t\t\t\t\"ns-2\": {\"baz\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\tother: &Result{},\n\t\t\twant: &Result{\n\t\t\t\tCluster: []string{\"foo\"},\n\t\t\t\tNamespaces: map[string][]string{\n\t\t\t\t\t\"ns-1\": {\"bar\"},\n\t\t\t\t\t\"ns-2\": {\"baz\"},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when a non-empty result is merged into an result, the result looks like the non-empty result\",\n\t\t\tresult: &Result{},\n\t\t\tother: &Result{\n\t\t\t\tCluster: []string{\"foo\"},\n\t\t\t\tNamespaces: map[string][]string{\n\t\t\t\t\t\"ns-1\": {\"bar\"},\n\t\t\t\t\t\"ns-2\": {\"baz\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: &Result{\n\t\t\t\tCluster: []string{\"foo\"},\n\t\t\t\tNamespaces: map[string][]string{\n\t\t\t\t\t\"ns-1\": {\"bar\"},\n\t\t\t\t\t\"ns-2\": {\"baz\"},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when two non-empty results are merged, the result is the union of the two\",\n\t\t\tresult: &Result{\n\t\t\t\tCluster: []string{\"cluster-err-1\"},\n\t\t\t\tNamespaces: map[string][]string{\n\t\t\t\t\t\"ns-1\": {\"ns-1-err-1\"},\n\t\t\t\t\t\"ns-2\": {\"ns-2-err-1\"},\n\t\t\t\t\t\"ns-3\": {\"ns-3-err-1\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\tother: &Result{\n\t\t\t\tCluster: []string{\"cluster-err-2\"},\n\t\t\t\tNamespaces: map[string][]string{\n\t\t\t\t\t\"ns-1\": {\"ns-1-err-2\"},\n\t\t\t\t\t\"ns-2\": {\"ns-2-err-2\"},\n\t\t\t\t\t\"ns-4\": {\"ns-4-err-1\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: &Result{\n\t\t\t\tCluster: []string{\"cluster-err-1\", \"cluster-err-2\"},\n\t\t\t\tNamespaces: map[string][]string{\n\t\t\t\t\t\"ns-1\": {\"ns-1-err-1\", \"ns-1-err-2\"},\n\t\t\t\t\t\"ns-2\": {\"ns-2-err-1\", \"ns-2-err-2\"},\n\t\t\t\t\t\"ns-3\": {\"ns-3-err-1\"},\n\t\t\t\t\t\"ns-4\": {\"ns-4-err-1\"},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\ttc.result.Merge(tc.other)\n\t\t\tassert.Equal(t, tc.want, tc.result)\n\t\t})\n\t}\n}\n\nfunc TestAddVeleroError(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tresult *Result\n\t\terr error\n\t\twant *Result\n\t}{\n\t\t{\n\t\t\tname: \"when AddVeleroError is called for a result with no velero errors, the result has the new error added properly\",\n\t\t\tresult: &Result{},\n\t\t\terr: errors.New(\"foo\"),\n\t\t\twant: &Result{Velero: []string{\"foo\"}},\n\t\t},\n\n\t\t{\n\t\t\tname: \"when AddVeleroError is called for a result with existing velero errors, the result has the new error appended properly\",\n\t\t\tresult: &Result{Velero: []string{\"bar\"}},\n\t\t\terr: errors.New(\"foo\"),\n\t\t\twant: &Result{Velero: []string{\"bar\", \"foo\"}},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\ttc.result.AddVeleroError(tc.err)\n\t\t\tassert.Equal(t, tc.want, tc.result)\n\t\t})\n\t}\n}\n\nfunc TestAdd(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tresult *Result\n\t\tns string\n\t\terr error\n\t\twant *Result\n\t}{\n\t\t{\n\t\t\tname: \"when Add is called for a result with no existing errors and an empty namespace, the error is added to the cluster-scoped list\",\n\t\t\tresult: &Result{},\n\t\t\tns: \"\",\n\t\t\terr: errors.New(\"foo\"),\n\t\t\twant: &Result{Cluster: []string{\"foo\"}},\n\t\t},\n\t\t{\n\t\t\tname: \"when Add is called for a result with some existing errors and an empty namespace, the error is added to the cluster-scoped list\",\n\t\t\tresult: &Result{Cluster: []string{\"bar\"}},\n\t\t\tns: \"\",\n\t\t\terr: errors.New(\"foo\"),\n\t\t\twant: &Result{Cluster: []string{\"bar\", \"foo\"}},\n\t\t},\n\n\t\t{\n\t\t\tname: \"when Add is called for a result with no existing errors and a non-empty namespace, the error is added to the namespace list\",\n\t\t\tresult: &Result{},\n\t\t\tns: \"ns-1\",\n\t\t\terr: errors.New(\"foo\"),\n\t\t\twant: &Result{\n\t\t\t\tNamespaces: map[string][]string{\n\t\t\t\t\t\"ns-1\": {\"foo\"},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"when Add is called for a result with some existing errors and a non-empty namespace, the error is added to the namespace list\",\n\t\t\tresult: &Result{\n\t\t\t\tNamespaces: map[string][]string{\n\t\t\t\t\t\"ns-1\": {\"bar\"},\n\t\t\t\t\t\"ns-2\": {\"baz\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\tns: \"ns-1\",\n\t\t\terr: errors.New(\"foo\"),\n\t\t\twant: &Result{\n\t\t\t\tNamespaces: map[string][]string{\n\t\t\t\t\t\"ns-1\": {\"bar\", \"foo\"},\n\t\t\t\t\t\"ns-2\": {\"baz\"},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, tc := range tests {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\ttc.result.Add(tc.ns, tc.err)\n\t\t\tassert.Equal(t, tc.want, tc.result)\n\t\t})\n\t}\n}\n\nfunc TestIsEmpty(t *testing.T) {\n\tresult := &Result{\n\t\tVelero: nil,\n\t\tCluster: nil,\n\t\tNamespaces: nil,\n\t}\n\tassert.True(t, result.IsEmpty())\n\n\tresult = &Result{\n\t\tVelero: []string{\"error\"},\n\t\tCluster: nil,\n\t\tNamespaces: nil,\n\t}\n\tassert.False(t, result.IsEmpty())\n}\n" }, { "path": "pkg/util/scheme.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage util\n\nimport (\n\t\"k8s.io/apimachinery/pkg/runtime\"\n\tutilruntime \"k8s.io/apimachinery/pkg/util/runtime\"\n\n\tv1 \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1\"\n)\n\nvar VeleroScheme = runtime.NewScheme()\n\nfunc init() {\n\tlocalSchemeBuilder := runtime.SchemeBuilder{\n\t\tv1.AddToScheme,\n\t\tv2alpha1.AddToScheme,\n\t}\n\tutilruntime.Must(localSchemeBuilder.AddToScheme(VeleroScheme))\n}\n" }, { "path": "pkg/util/stringptr/stringptr.go", "content": "/*\nCopyright 2017 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage stringptr\n\nconst NilString = \"\"\n\nfunc GetString(str *string) string {\n\tif str == nil {\n\t\treturn NilString\n\t} else {\n\t\treturn *str\n\t}\n}\n" }, { "path": "pkg/util/stringslice/stringslice.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage stringslice\n\n// Has returns true if the `items` slice contains the\n// value `val`, or false otherwise.\nfunc Has(items []string, val string) bool {\n\tfor _, itm := range items {\n\t\tif itm == val {\n\t\t\treturn true\n\t\t}\n\t}\n\n\treturn false\n}\n\n// Except returns a new string slice that contains all of the entries\n// from `items` except `val`.\nfunc Except(items []string, val string) []string {\n\t// Default the capacity the len(items) instead of len(items)-1 in case items does not contain val.\n\tnewItems := make([]string, 0, len(items))\n\n\tfor _, itm := range items {\n\t\tif itm != val {\n\t\t\tnewItems = append(newItems, itm)\n\t\t}\n\t}\n\n\treturn newItems\n}\n" }, { "path": "pkg/util/stringslice/stringslice_test.go", "content": "/*\nCopyright 2018 the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage stringslice\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestHas(t *testing.T) {\n\titems := []string{}\n\tassert.False(t, Has(items, \"a\"))\n\n\titems = []string{\"a\", \"b\", \"c\"}\n\tassert.True(t, Has(items, \"a\"))\n\tassert.True(t, Has(items, \"b\"))\n\tassert.True(t, Has(items, \"c\"))\n\tassert.False(t, Has(items, \"d\"))\n}\n\nfunc TestExcept(t *testing.T) {\n\titems := []string{}\n\texcept := Except(items, \"asdf\")\n\tassert.Empty(t, except)\n\n\titems = []string{\"a\", \"b\", \"c\"}\n\tassert.Equal(t, []string{\"b\", \"c\"}, Except(items, \"a\"))\n\tassert.Equal(t, []string{\"a\", \"c\"}, Except(items, \"b\"))\n\tassert.Equal(t, []string{\"a\", \"b\"}, Except(items, \"c\"))\n\tassert.Equal(t, []string{\"a\", \"b\", \"c\"}, Except(items, \"d\"))\n}\n" }, { "path": "pkg/util/third_party.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage util\n\nvar ThirdPartyLabels = []string{\n\t\"azure.workload.identity/use\",\n}\n\nvar ThirdPartyAnnotations = []string{\n\t\"iam.amazonaws.com/role\",\n}\n\nvar ThirdPartyTolerations = []string{\n\t\"kubernetes.azure.com/scalesetpriority\",\n\t\"CriticalAddonsOnly\",\n}\n\nconst (\n\tVSphereCNSFastCloneAnno = \"csi.vsphere.volume/fast-provisioning\"\n)\n" }, { "path": "pkg/util/util.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage util\n\nimport (\n\t\"crypto/sha256\"\n\t\"encoding/hex\"\n)\n\nfunc Contains(slice []string, key string) bool {\n\tfor _, i := range slice {\n\t\tif i == key {\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n\n// GenerateSha256FromRestoreUIDAndVsName Use the restore UID and the VS Name to generate\n// the new VSC and VS name. By this way, VS and VSC RIA action can get the same VSC name.\nfunc GenerateSha256FromRestoreUIDAndVsName(restoreUID string, vsName string) string {\n\tsha256Bytes := sha256.Sum256([]byte(restoreUID + \"/\" + vsName))\n\treturn hex.EncodeToString(sha256Bytes[:])\n}\n" }, { "path": "pkg/util/util_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage util\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc TestContains(t *testing.T) {\n\ttestCases := []struct {\n\t\tname string\n\t\tinSlice []string\n\t\tinKey string\n\t\texpectedResult bool\n\t}{\n\t\t{\n\t\t\tname: \"should find the key\",\n\t\t\tinSlice: []string{\"key1\", \"key2\", \"key3\", \"key4\", \"key5\"},\n\t\t\tinKey: \"key3\",\n\t\t\texpectedResult: true,\n\t\t},\n\t\t{\n\t\t\tname: \"should not find the key in non-empty slice\",\n\t\t\tinSlice: []string{\"key1\", \"key2\", \"key3\", \"key4\", \"key5\"},\n\t\t\tinKey: \"key300\",\n\t\t\texpectedResult: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should not find key in empty slice\",\n\t\t\tinSlice: []string{},\n\t\t\tinKey: \"key300\",\n\t\t\texpectedResult: false,\n\t\t},\n\t\t{\n\t\t\tname: \"should not find key in nil slice\",\n\t\t\tinSlice: nil,\n\t\t\tinKey: \"key300\",\n\t\t\texpectedResult: false,\n\t\t},\n\t}\n\n\tfor _, tc := range testCases {\n\t\tt.Run(tc.name, func(t *testing.T) {\n\t\t\tactualResult := Contains(tc.inSlice, tc.inKey)\n\t\t\tassert.Equal(t, tc.expectedResult, actualResult)\n\t\t})\n\t}\n}\n" }, { "path": "pkg/util/velero/restore/util.go", "content": "package restore\n\nimport (\n\tapi \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\nfunc IsResourcePolicyValid(resourcePolicy string) bool {\n\tif resourcePolicy == string(api.PolicyTypeNone) || resourcePolicy == string(api.PolicyTypeUpdate) {\n\t\treturn true\n\t}\n\treturn false\n}\n" }, { "path": "pkg/util/velero/restore/util_test.go", "content": "package restore\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\nfunc TestIsResourcePolicyValid(t *testing.T) {\n\trequire.True(t, IsResourcePolicyValid(string(velerov1api.PolicyTypeNone)))\n\trequire.True(t, IsResourcePolicyValid(string(velerov1api.PolicyTypeUpdate)))\n\trequire.False(t, IsResourcePolicyValid(\"\"))\n}\n" }, { "path": "pkg/util/velero/velero.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage velero\n\nimport (\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n)\n\n// GetNodeSelectorFromVeleroServer get the node selector from the Velero server deployment\nfunc GetNodeSelectorFromVeleroServer(deployment *appsv1api.Deployment) map[string]string {\n\treturn deployment.Spec.Template.Spec.NodeSelector\n}\n\n// GetTolerationsFromVeleroServer get the tolerations from the Velero server deployment\nfunc GetTolerationsFromVeleroServer(deployment *appsv1api.Deployment) []corev1api.Toleration {\n\treturn deployment.Spec.Template.Spec.Tolerations\n}\n\n// GetAffinityFromVeleroServer get the affinity from the Velero server deployment\nfunc GetAffinityFromVeleroServer(deployment *appsv1api.Deployment) *corev1api.Affinity {\n\treturn deployment.Spec.Template.Spec.Affinity\n}\n\n// GetEnvVarsFromVeleroServer get the environment variables from the Velero server deployment\nfunc GetEnvVarsFromVeleroServer(deployment *appsv1api.Deployment) []corev1api.EnvVar {\n\tfor _, container := range deployment.Spec.Template.Spec.Containers {\n\t\t// We only have one container in the Velero server deployment\n\t\treturn container.Env\n\t}\n\treturn nil\n}\n\n// GetEnvFromSourcesFromVeleroServer get the environment sources from the Velero server deployment\nfunc GetEnvFromSourcesFromVeleroServer(deployment *appsv1api.Deployment) []corev1api.EnvFromSource {\n\tfor _, container := range deployment.Spec.Template.Spec.Containers {\n\t\t// We only have one container in the Velero server deployment\n\t\treturn container.EnvFrom\n\t}\n\treturn nil\n}\n\n// GetVolumeMountsFromVeleroServer get the volume mounts from the Velero server deployment\nfunc GetVolumeMountsFromVeleroServer(deployment *appsv1api.Deployment) []corev1api.VolumeMount {\n\tfor _, container := range deployment.Spec.Template.Spec.Containers {\n\t\t// We only have one container in the Velero server deployment\n\t\treturn container.VolumeMounts\n\t}\n\treturn nil\n}\n\n// GetPodSecurityContextsFromVeleroServer get the pod security context from the Velero server deployment\nfunc GetPodSecurityContextsFromVeleroServer(deployment *appsv1api.Deployment) *corev1api.PodSecurityContext {\n\treturn deployment.Spec.Template.Spec.SecurityContext\n}\n\n// GetContainerSecurityContextsFromVeleroServer get the security context from the Velero server deployment\nfunc GetContainerSecurityContextsFromVeleroServer(deployment *appsv1api.Deployment) *corev1api.SecurityContext {\n\tfor _, container := range deployment.Spec.Template.Spec.Containers {\n\t\t// We only have one container in the Velero server deployment\n\t\treturn container.SecurityContext\n\t}\n\treturn nil\n}\n\n// GetVolumesFromVeleroServer get the volumes from the Velero server deployment\nfunc GetVolumesFromVeleroServer(deployment *appsv1api.Deployment) []corev1api.Volume {\n\treturn deployment.Spec.Template.Spec.Volumes\n}\n\n// GetServiceAccountFromVeleroServer get the service account from the Velero server deployment\nfunc GetServiceAccountFromVeleroServer(deployment *appsv1api.Deployment) string {\n\treturn deployment.Spec.Template.Spec.ServiceAccountName\n}\n\n// GetImagePullSecretsFromVeleroServer get the image pull secrets from the Velero server deployment\nfunc GetImagePullSecretsFromVeleroServer(deployment *appsv1api.Deployment) []corev1api.LocalObjectReference {\n\treturn deployment.Spec.Template.Spec.ImagePullSecrets\n}\n\n// getVeleroServerImage get the image of the Velero server deployment\nfunc GetVeleroServerImage(deployment *appsv1api.Deployment) string {\n\treturn deployment.Spec.Template.Spec.Containers[0].Image\n}\n\n// GetVeleroServerLables get the labels of the Velero server deployment\nfunc GetVeleroServerLables(deployment *appsv1api.Deployment) map[string]string {\n\treturn deployment.Spec.Template.Labels\n}\n\n// GetVeleroServerAnnotations get the annotations of the Velero server deployment\nfunc GetVeleroServerAnnotations(deployment *appsv1api.Deployment) map[string]string {\n\treturn deployment.Spec.Template.Annotations\n}\n\n// GetVeleroServerLabelValue returns the value of specified label of Velero server deployment\nfunc GetVeleroServerLabelValue(deployment *appsv1api.Deployment, key string) string {\n\tif deployment.Spec.Template.Labels == nil {\n\t\treturn \"\"\n\t}\n\n\treturn deployment.Spec.Template.Labels[key]\n}\n\n// GetVeleroServerAnnotationValue returns the value of specified annotation of Velero server deployment\nfunc GetVeleroServerAnnotationValue(deployment *appsv1api.Deployment, key string) string {\n\tif deployment.Spec.Template.Annotations == nil {\n\t\treturn \"\"\n\t}\n\n\treturn deployment.Spec.Template.Annotations[key]\n}\n\nfunc BSLIsAvailable(bsl velerov1api.BackupStorageLocation) bool {\n\treturn bsl.Status.Phase == velerov1api.BackupStorageLocationPhaseAvailable\n}\n" }, { "path": "pkg/util/velero/velero_test.go", "content": "/*\nCopyright the Velero contributors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n*/\n\npackage velero\n\nimport (\n\t\"reflect\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\tappsv1api \"k8s.io/api/apps/v1\"\n\tcorev1api \"k8s.io/api/core/v1\"\n\tmetav1 \"k8s.io/apimachinery/pkg/apis/meta/v1\"\n\n\tvelerov1api \"github.com/vmware-tanzu/velero/pkg/apis/velero/v1\"\n\t\"github.com/vmware-tanzu/velero/pkg/builder\"\n\t\"github.com/vmware-tanzu/velero/pkg/util/boolptr\"\n)\n\nfunc TestGetNodeSelectorFromVeleroServer(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdeploy *appsv1api.Deployment\n\t\twant map[string]string\n\t}{\n\t\t{\n\t\t\tname: \"no node selector\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tNodeSelector: map[string]string{},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: map[string]string{},\n\t\t},\n\t\t{\n\t\t\tname: \"node selector\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tNodeSelector: map[string]string{\n\t\t\t\t\t\t\t\t\"foo\": \"bar\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: map[string]string{\n\t\t\t\t\"foo\": \"bar\",\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tgot := GetNodeSelectorFromVeleroServer(test.deploy)\n\t\t\tif len(got) != len(test.want) {\n\t\t\t\tt.Errorf(\"expected node selector to have %d elements, got %d\", len(test.want), len(got))\n\t\t\t}\n\t\t\tfor k, v := range test.want {\n\t\t\t\tif got[k] != v {\n\t\t\t\t\tt.Errorf(\"expected node selector to have key %s with value %s, got %s\", k, v, got[k])\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetTolerationsFromVeleroServer(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdeploy *appsv1api.Deployment\n\t\twant []corev1api.Toleration\n\t}{\n\t\t{\n\t\t\tname: \"no tolerations\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tTolerations: []corev1api.Toleration{},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []corev1api.Toleration{},\n\t\t},\n\t\t{\n\t\t\tname: \"tolerations\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tTolerations: []corev1api.Toleration{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tKey: \"foo\",\n\t\t\t\t\t\t\t\t\tOperator: \"Exists\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []corev1api.Toleration{\n\t\t\t\t{\n\t\t\t\t\tKey: \"foo\",\n\t\t\t\t\tOperator: \"Exists\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tgot := GetTolerationsFromVeleroServer(test.deploy)\n\t\t\tif len(got) != len(test.want) {\n\t\t\t\tt.Errorf(\"expected tolerations to have %d elements, got %d\", len(test.want), len(got))\n\t\t\t}\n\t\t\tfor i, want := range test.want {\n\t\t\t\tif got[i] != want {\n\t\t\t\t\tt.Errorf(\"expected toleration at index %d to be %v, got %v\", i, want, got[i])\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetAffinityFromVeleroServer(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdeploy *appsv1api.Deployment\n\t\twant *corev1api.Affinity\n\t}{\n\t\t{\n\t\t\tname: \"no affinity\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tAffinity: nil,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"affinity\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tAffinity: &corev1api.Affinity{\n\t\t\t\t\t\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\t\t\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\t\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\t\t\t\t\tKey: \"foo\",\n\t\t\t\t\t\t\t\t\t\t\t\t\t\tOperator: \"In\",\n\t\t\t\t\t\t\t\t\t\t\t\t\t\tValues: []string{\"bar\"},\n\t\t\t\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: &corev1api.Affinity{\n\t\t\t\tNodeAffinity: &corev1api.NodeAffinity{\n\t\t\t\t\tRequiredDuringSchedulingIgnoredDuringExecution: &corev1api.NodeSelector{\n\t\t\t\t\t\tNodeSelectorTerms: []corev1api.NodeSelectorTerm{\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tMatchExpressions: []corev1api.NodeSelectorRequirement{\n\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\tKey: \"foo\",\n\t\t\t\t\t\t\t\t\t\tOperator: \"In\",\n\t\t\t\t\t\t\t\t\t\tValues: []string{\"bar\"},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tgot := GetAffinityFromVeleroServer(test.deploy)\n\t\t\tif test.want != nil {\n\t\t\t\trequire.NotNilf(t, got, \"expected affinity to be %v, got nil\", test.want)\n\t\t\t\tif test.want.NodeAffinity != nil {\n\t\t\t\t\trequire.NotNilf(t, got.NodeAffinity, \"expected node affinity to be %v, got nil\", test.want.NodeAffinity)\n\t\t\t\t\tif test.want.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution != nil {\n\t\t\t\t\t\trequire.NotNilf(t, got.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution, \"expected required during scheduling ignored during execution to be %v, got nil\", test.want.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution)\n\t\t\t\t\t\tassert.Truef(t, reflect.DeepEqual(got.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution, test.want.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution), \"expected required during scheduling ignored during execution to be %v, got %v\", test.want.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution, got.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution)\n\t\t\t\t\t} else {\n\t\t\t\t\t\tassert.Nilf(t, got.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution, \"expected required during scheduling ignored during execution to be nil, got %v\", got.NodeAffinity.RequiredDuringSchedulingIgnoredDuringExecution)\n\t\t\t\t\t}\n\t\t\t\t} else {\n\t\t\t\t\tassert.Nilf(t, got.NodeAffinity, \"expected node affinity to be nil, got %v\", got.NodeAffinity)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tassert.Nilf(t, got, \"expected affinity to be nil, got %v\", got)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetEnvVarsFromVeleroServer(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdeploy *appsv1api.Deployment\n\t\twant []corev1api.EnvVar\n\t}{\n\t\t{\n\t\t\tname: \"no env vars\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tEnv: []corev1api.EnvVar{},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []corev1api.EnvVar{},\n\t\t},\n\t\t{\n\t\t\tname: \"env vars\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tEnv: []corev1api.EnvVar{\n\t\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\t\tName: \"foo\",\n\t\t\t\t\t\t\t\t\t\t\tValue: \"bar\",\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []corev1api.EnvVar{\n\t\t\t\t{\n\t\t\t\t\tName: \"foo\",\n\t\t\t\t\tValue: \"bar\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tgot := GetEnvVarsFromVeleroServer(test.deploy)\n\t\t\tif len(got) != len(test.want) {\n\t\t\t\tt.Errorf(\"expected env vars to have %d elements, got %d\", len(test.want), len(got))\n\t\t\t}\n\t\t\tfor i, want := range test.want {\n\t\t\t\tif got[i] != want {\n\t\t\t\t\tt.Errorf(\"expected env var at index %d to be %v, got %v\", i, want, got[i])\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetEnvFromSourcesFromVeleroServer(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdeploy *appsv1api.Deployment\n\t\texpected []corev1api.EnvFromSource\n\t}{\n\t\t{\n\t\t\tname: \"no env vars\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tEnvFrom: []corev1api.EnvFromSource{},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: []corev1api.EnvFromSource{},\n\t\t},\n\t\t{\n\t\t\tname: \"configmap\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tEnvFrom: []corev1api.EnvFromSource{\n\t\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\t\tConfigMapRef: &corev1api.ConfigMapEnvSource{\n\t\t\t\t\t\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\t\t\t\t\t\tName: \"foo\",\n\t\t\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: []corev1api.EnvFromSource{\n\t\t\t\t{\n\t\t\t\t\tConfigMapRef: &corev1api.ConfigMapEnvSource{\n\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\tName: \"foo\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t{\n\t\t\tname: \"secret\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tEnvFrom: []corev1api.EnvFromSource{\n\t\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\t\tSecretRef: &corev1api.SecretEnvSource{\n\t\t\t\t\t\t\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\t\t\t\t\t\tName: \"foo\",\n\t\t\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: []corev1api.EnvFromSource{\n\t\t\t\t{\n\t\t\t\t\tSecretRef: &corev1api.SecretEnvSource{\n\t\t\t\t\t\tLocalObjectReference: corev1api.LocalObjectReference{\n\t\t\t\t\t\t\tName: \"foo\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tresult := GetEnvFromSourcesFromVeleroServer(test.deploy)\n\t\t\tassert.Equal(t, test.expected, result)\n\t\t})\n\t}\n}\n\nfunc TestGetVolumeMountsFromVeleroServer(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdeploy *appsv1api.Deployment\n\t\twant []corev1api.VolumeMount\n\t}{\n\t\t{\n\t\t\tname: \"no volume mounts\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tVolumeMounts: []corev1api.VolumeMount{},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []corev1api.VolumeMount{},\n\t\t},\n\t\t{\n\t\t\tname: \"volume mounts\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tVolumeMounts: []corev1api.VolumeMount{\n\t\t\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\t\t\tName: \"foo\",\n\t\t\t\t\t\t\t\t\t\t\tMountPath: \"/bar\",\n\t\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []corev1api.VolumeMount{\n\t\t\t\t{\n\t\t\t\t\tName: \"foo\",\n\t\t\t\t\tMountPath: \"/bar\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tgot := GetVolumeMountsFromVeleroServer(test.deploy)\n\t\t\tif len(got) != len(test.want) {\n\t\t\t\tt.Errorf(\"expected volume mounts to have %d elements, got %d\", len(test.want), len(got))\n\t\t\t}\n\t\t\tfor i, want := range test.want {\n\t\t\t\tif got[i] != want {\n\t\t\t\t\tt.Errorf(\"expected volume mount at index %d to be %v, got %v\", i, want, got[i])\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetVolumesFromVeleroServer(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdeploy *appsv1api.Deployment\n\t\twant []corev1api.Volume\n\t}{\n\t\t{\n\t\t\tname: \"no volumes\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tVolumes: []corev1api.Volume{},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []corev1api.Volume{},\n\t\t},\n\t\t{\n\t\t\tname: \"volumes\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tVolumes: []corev1api.Volume{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"foo\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []corev1api.Volume{\n\t\t\t\t{\n\t\t\t\t\tName: \"foo\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tgot := GetVolumesFromVeleroServer(test.deploy)\n\t\t\tif len(got) != len(test.want) {\n\t\t\t\tt.Errorf(\"expected volumes to have %d elements, got %d\", len(test.want), len(got))\n\t\t\t}\n\t\t\tfor i, want := range test.want {\n\t\t\t\tif got[i] != want {\n\t\t\t\t\tt.Errorf(\"expected volume at index %d to be %v, got %v\", i, want, got[i])\n\t\t\t\t}\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetPodSecurityContextsFromVeleroServer(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdeploy *appsv1api.Deployment\n\t\twant *corev1api.PodSecurityContext\n\t}{\n\t\t{\n\t\t\tname: \"no security context\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tSecurityContext: nil,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"security context\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tSecurityContext: &corev1api.PodSecurityContext{\n\t\t\t\t\t\t\t\tRunAsNonRoot: boolptr.True(),\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: &corev1api.PodSecurityContext{\n\t\t\t\tRunAsNonRoot: boolptr.True(),\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tgot := GetPodSecurityContextsFromVeleroServer(test.deploy)\n\t\t\tassert.Equal(t, test.want, got)\n\t\t})\n\t}\n}\n\nfunc TestGetContainerSecurityContextsFromVeleroServer(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdeploy *appsv1api.Deployment\n\t\twant *corev1api.SecurityContext\n\t}{\n\t\t{\n\t\t\tname: \"no container\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tContainers: []corev1api.Container{},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"no security context\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tSecurityContext: nil,\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"security context\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tSecurityContext: &corev1api.SecurityContext{\n\t\t\t\t\t\t\t\t\t\tRunAsNonRoot: boolptr.True(),\n\t\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: &corev1api.SecurityContext{\n\t\t\t\tRunAsNonRoot: boolptr.True(),\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tgot := GetContainerSecurityContextsFromVeleroServer(test.deploy)\n\t\t\tassert.Equal(t, test.want, got)\n\t\t})\n\t}\n}\n\nfunc TestGetServiceAccountFromVeleroServer(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdeploy *appsv1api.Deployment\n\t\twant string\n\t}{\n\t\t{\n\t\t\tname: \"no service account\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tServiceAccountName: \"\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"service account\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tServiceAccountName: \"foo\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: \"foo\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tgot := GetServiceAccountFromVeleroServer(test.deploy)\n\t\t\tif got != test.want {\n\t\t\t\tt.Errorf(\"expected service account to be %s, got %s\", test.want, got)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetImagePullSecretsFromVeleroServer(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdeploy *appsv1api.Deployment\n\t\twant []corev1api.LocalObjectReference\n\t}{\n\t\t{\n\t\t\tname: \"no image pull secrets\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tServiceAccountName: \"\",\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: nil,\n\t\t},\n\t\t{\n\t\t\tname: \"image pull secrets\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tImagePullSecrets: []corev1api.LocalObjectReference{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"imagePullSecret1\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tName: \"imagePullSecret2\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: []corev1api.LocalObjectReference{\n\t\t\t\t{\n\t\t\t\t\tName: \"imagePullSecret1\",\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\tName: \"imagePullSecret2\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tgot := GetImagePullSecretsFromVeleroServer(test.deploy)\n\n\t\t\trequire.Equal(t, test.want, got)\n\t\t})\n\t}\n}\n\nfunc TestGetVeleroServerImage(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdeploy *appsv1api.Deployment\n\t\twant string\n\t}{\n\t\t{\n\t\t\tname: \"velero server image\",\n\t\t\tdeploy: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tSpec: corev1api.PodSpec{\n\t\t\t\t\t\t\tContainers: []corev1api.Container{\n\t\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\t\tImage: \"velero/velero:latest\",\n\t\t\t\t\t\t\t\t},\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twant: \"velero/velero:latest\",\n\t\t},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(test.name, func(t *testing.T) {\n\t\t\tgot := GetVeleroServerImage(test.deploy)\n\t\t\tif got != test.want {\n\t\t\t\tt.Errorf(\"expected velero server image to be %s, got %s\", test.want, got)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestGetVeleroServerLables(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdeployment *appsv1api.Deployment\n\t\texpected map[string]string\n\t}{\n\t\t{\n\t\t\tname: \"Empty Labels\",\n\t\t\tdeployment: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tLabels: map[string]string{},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: map[string]string{},\n\t\t},\n\t\t{\n\t\t\tname: \"Non-empty Labels\",\n\t\t\tdeployment: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tLabels: map[string]string{\n\t\t\t\t\t\t\t\t\"app\": \"velero\",\n\t\t\t\t\t\t\t\t\"component\": \"server\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: map[string]string{\n\t\t\t\t\"app\": \"velero\",\n\t\t\t\t\"component\": \"server\",\n\t\t\t},\n\t\t},\n\t}\n\n\t// Run tests\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tresult := GetVeleroServerLables(tt.deployment)\n\t\t\tassert.Equal(t, tt.expected, result)\n\t\t})\n\t}\n}\n\nfunc TestGetVeleroServerAnnotations(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdeployment *appsv1api.Deployment\n\t\texpected map[string]string\n\t}{\n\t\t{\n\t\t\tname: \"Empty Labels\",\n\t\t\tdeployment: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tAnnotations: map[string]string{},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: map[string]string{},\n\t\t},\n\t\t{\n\t\t\tname: \"Non-empty Labels\",\n\t\t\tdeployment: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tAnnotations: map[string]string{\n\t\t\t\t\t\t\t\t\"app\": \"velero\",\n\t\t\t\t\t\t\t\t\"component\": \"server\",\n\t\t\t\t\t\t\t},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: map[string]string{\n\t\t\t\t\"app\": \"velero\",\n\t\t\t\t\"component\": \"server\",\n\t\t\t},\n\t\t},\n\t}\n\n\t// Run tests\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tresult := GetVeleroServerAnnotations(tt.deployment)\n\t\t\tassert.Equal(t, tt.expected, result)\n\t\t})\n\t}\n}\n\nfunc TestGetVeleroServerLabelValue(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tdeployment *appsv1api.Deployment\n\t\texpected string\n\t}{\n\t\t{\n\t\t\tname: \"nil Labels\",\n\t\t\tdeployment: &appsv1api.Deployment{},\n\t\t\texpected: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"no label key\",\n\t\t\tdeployment: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tLabels: map[string]string{},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: \"\",\n\t\t},\n\t\t{\n\t\t\tname: \"with label key\",\n\t\t\tdeployment: &appsv1api.Deployment{\n\t\t\t\tSpec: appsv1api.DeploymentSpec{\n\t\t\t\t\tTemplate: corev1api.PodTemplateSpec{\n\t\t\t\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\t\t\t\tLabels: map[string]string{\"fake-key\": \"fake-value\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpected: \"fake-value\",\n\t\t},\n\t}\n\n\t// Run tests\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tresult := GetVeleroServerLabelValue(tt.deployment, \"fake-key\")\n\t\t\tassert.Equal(t, tt.expected, result)\n\t\t})\n\t}\n}\n\nfunc TestBSLIsAvailable(t *testing.T) {\n\tavailableBSL := builder.ForBackupStorageLocation(\"velero\", \"available\").Phase(velerov1api.BackupStorageLocationPhaseAvailable).Result()\n\tunavailableBSL := builder.ForBackupStorageLocation(\"velero\", \"unavailable\").Phase(velerov1api.BackupStorageLocationPhaseUnavailable).Result()\n\n\tassert.True(t, BSLIsAvailable(*availableBSL))\n\tassert.False(t, BSLIsAvailable(*unavailableBSL))\n}\n" }, { "path": "pkg/util/wildcard/expand.go", "content": "package wildcard\n\nimport (\n\t\"errors\"\n\t\"strings\"\n\n\t\"github.com/gobwas/glob\"\n\t\"k8s.io/apimachinery/pkg/util/sets\"\n)\n\nfunc ShouldExpandWildcards(includes []string, excludes []string) bool {\n\twildcardFound := false\n\tfor _, include := range includes {\n\t\t// Special case: \"*\" alone means \"match all\" - don't expand\n\t\tif include == \"*\" {\n\t\t\treturn false\n\t\t}\n\n\t\tif containsWildcardPattern(include) {\n\t\t\twildcardFound = true\n\t\t}\n\t}\n\n\tfor _, exclude := range excludes {\n\t\tif containsWildcardPattern(exclude) {\n\t\t\twildcardFound = true\n\t\t}\n\t}\n\n\treturn wildcardFound\n}\n\n// containsWildcardPattern checks if a pattern contains any wildcard symbols\n// Supported patterns: *, ?, [abc]\n// Note: . and + are treated as literal characters (not wildcards)\n// Note: ** and consecutive asterisks are NOT supported (will cause validation error)\nfunc containsWildcardPattern(pattern string) bool {\n\treturn strings.ContainsAny(pattern, \"*?[\")\n}\n\nfunc validateWildcardPatterns(patterns []string) error {\n\tfor _, pattern := range patterns {\n\t\tif err := ValidateNamespaceName(pattern); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc ValidateNamespaceName(pattern string) error {\n\t// Check for invalid characters that are not supported in glob patterns\n\tif strings.ContainsAny(pattern, \"|()!{},\") {\n\t\treturn errors.New(\"wildcard pattern contains unsupported characters: |, (, ), !, {, }, ,\")\n\t}\n\n\t// Check for consecutive asterisks (2 or more)\n\tif strings.Contains(pattern, \"**\") {\n\t\treturn errors.New(\"wildcard pattern contains consecutive asterisks (only single * allowed)\")\n\t}\n\n\t// Check for malformed brace patterns\n\tif err := validateBracePatterns(pattern); err != nil {\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\n// validateBracePatterns checks for malformed brace patterns like unclosed braces or empty braces\n// Also validates bracket patterns [] for character classes\nfunc validateBracePatterns(pattern string) error {\n\tbracketDepth := 0\n\n\tfor i := 0; i < len(pattern); i++ {\n\t\tif pattern[i] == '[' {\n\t\t\tbracketStart := i\n\t\t\tbracketDepth++\n\n\t\t\t// Scan ahead to find the matching closing bracket and validate content\n\t\t\tfor j := i + 1; j < len(pattern) && bracketDepth > 0; j++ {\n\t\t\t\tif pattern[j] == ']' {\n\t\t\t\t\tbracketDepth--\n\t\t\t\t\tif bracketDepth == 0 {\n\t\t\t\t\t\t// Found matching closing bracket - validate content\n\t\t\t\t\t\tcontent := pattern[bracketStart+1 : j]\n\t\t\t\t\t\tif content == \"\" {\n\t\t\t\t\t\t\treturn errors.New(\"wildcard pattern contains empty bracket pattern '[]'\")\n\t\t\t\t\t\t}\n\t\t\t\t\t\t// Skip to the closing bracket\n\t\t\t\t\t\ti = j\n\t\t\t\t\t\tbreak\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\n\t\t\t// If we exited the loop without finding a match (bracketDepth > 0), bracket is unclosed\n\t\t\tif bracketDepth > 0 {\n\t\t\t\treturn errors.New(\"wildcard pattern contains unclosed bracket '['\")\n\t\t\t}\n\n\t\t\t// i is now positioned at the closing bracket; the outer loop will increment it\n\t\t} else if pattern[i] == ']' {\n\t\t\t// Found a closing bracket without a matching opening bracket\n\t\t\treturn errors.New(\"wildcard pattern contains unmatched closing bracket ']'\")\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc ExpandWildcards(activeNamespaces []string, includes []string, excludes []string) ([]string, []string, error) {\n\texpandedIncludes, err := expandWildcards(includes, activeNamespaces)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\texpandedExcludes, err := expandWildcards(excludes, activeNamespaces)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\n\treturn expandedIncludes, expandedExcludes, nil\n}\n\n// expands wildcard patterns into a list of namespaces, while normally passing non-wildcard patterns\nfunc expandWildcards(patterns []string, activeNamespaces []string) ([]string, error) {\n\tif len(patterns) == 0 {\n\t\treturn nil, nil\n\t}\n\n\t// Validate patterns before processing\n\tif err := validateWildcardPatterns(patterns); err != nil {\n\t\treturn nil, err\n\t}\n\n\tmatchedSet := make(map[string]struct{})\n\n\tfor _, pattern := range patterns {\n\t\t// If the pattern is a non-wildcard pattern, we can just add it to the result\n\t\tif !containsWildcardPattern(pattern) {\n\t\t\tmatchedSet[pattern] = struct{}{}\n\t\t\tcontinue\n\t\t}\n\n\t\t// Compile glob pattern\n\t\tg, err := glob.Compile(pattern)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\t// Match against all namespaces\n\t\tfor _, ns := range activeNamespaces {\n\t\t\tif g.Match(ns) {\n\t\t\t\tmatchedSet[ns] = struct{}{}\n\t\t\t}\n\t\t}\n\t}\n\n\t// Convert set to slice\n\tresult := make([]string, 0, len(matchedSet))\n\tfor ns := range matchedSet {\n\t\tresult = append(result, ns)\n\t}\n\n\treturn result, nil\n}\n\n// GetWildcardResult returns the final list of namespaces after applying wildcard include/exclude logic\nfunc GetWildcardResult(expandedIncludes []string, expandedExcludes []string) []string {\n\t// Set check: set of expandedIncludes - set of expandedExcludes\n\texpandedIncludesSet := sets.New(expandedIncludes...)\n\texpandedExcludesSet := sets.New(expandedExcludes...)\n\tselectedNamespacesSet := expandedIncludesSet.Difference(expandedExcludesSet)\n\n\t// Convert the set to a slice\n\tselectedNamespaces := make([]string, 0, selectedNamespacesSet.Len())\n\tfor ns := range selectedNamespacesSet {\n\t\tselectedNamespaces = append(selectedNamespaces, ns)\n\t}\n\n\treturn selectedNamespaces\n}\n" }, { "path": "pkg/util/wildcard/expand_test.go", "content": "package wildcard\n\nimport (\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestShouldExpandWildcards(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tincludes []string\n\t\texcludes []string\n\t\texpected bool\n\t}{\n\t\t{\n\t\t\tname: \"no wildcards\",\n\t\t\tincludes: []string{\"ns1\", \"ns2\"},\n\t\t\texcludes: []string{\"ns3\", \"ns4\"},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"includes has star - should not expand\",\n\t\t\tincludes: []string{\"*\"},\n\t\t\texcludes: []string{\"ns1\"},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"includes has star after a wildcard pattern - should not expand\",\n\t\t\tincludes: []string{\"ns*\", \"*\"},\n\t\t\texcludes: []string{\"ns1\"},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"includes has wildcard pattern\",\n\t\t\tincludes: []string{\"ns*\"},\n\t\t\texcludes: []string{\"ns1\"},\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"excludes has wildcard pattern\",\n\t\t\tincludes: []string{\"ns1\"},\n\t\t\texcludes: []string{\"ns*\"},\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"both have wildcard patterns\",\n\t\t\tincludes: []string{\"app-*\"},\n\t\t\texcludes: []string{\"test-*\"},\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"includes has star and wildcard - star takes precedence\",\n\t\t\tincludes: []string{\"*\", \"ns*\"},\n\t\t\texcludes: []string{},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"double asterisk should be detected as wildcard\",\n\t\t\tincludes: []string{\"**\"},\n\t\t\texcludes: []string{},\n\t\t\texpected: true, // ** is a wildcard pattern (but will error during validation)\n\t\t},\n\t\t{\n\t\t\tname: \"empty slices\",\n\t\t\tincludes: []string{},\n\t\t\texcludes: []string{},\n\t\t\texpected: false,\n\t\t},\n\t\t{\n\t\t\tname: \"complex wildcard patterns\",\n\t\t\tincludes: []string{\"*-prod\"},\n\t\t\texcludes: []string{\"test-*-staging\"},\n\t\t\texpected: true,\n\t\t},\n\t\t{\n\t\t\tname: \"question mark wildcard\",\n\t\t\tincludes: []string{\"ns?\"},\n\t\t\texcludes: []string{},\n\t\t\texpected: true, // question mark is now considered a wildcard\n\t\t},\n\t\t{\n\t\t\tname: \"character class wildcard\",\n\t\t\tincludes: []string{\"ns[abc]\"},\n\t\t\texcludes: []string{},\n\t\t\texpected: true, // character class is considered wildcard\n\t\t},\n\t\t{\n\t\t\tname: \"brace alternatives wildcard\",\n\t\t\tincludes: []string{\"ns{prod,staging}\"},\n\t\t\texcludes: []string{},\n\t\t\texpected: false, // brace alternatives are not supported\n\t\t},\n\t\t{\n\t\t\tname: \"dot is literal - not wildcard\",\n\t\t\tincludes: []string{\"app.prod\"},\n\t\t\texcludes: []string{},\n\t\t\texpected: false, // dot is literal, not wildcard\n\t\t},\n\t\t{\n\t\t\tname: \"plus is literal - not wildcard\",\n\t\t\tincludes: []string{\"app+\"},\n\t\t\texcludes: []string{},\n\t\t\texpected: false, // plus is literal, not wildcard\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tresult := ShouldExpandWildcards(tt.includes, tt.excludes)\n\t\t\tassert.Equal(t, tt.expected, result)\n\t\t})\n\t}\n}\n\nfunc TestExpandWildcards(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tactiveNamespaces []string\n\t\tincludes []string\n\t\texcludes []string\n\t\texpectedIncludes []string\n\t\texpectedExcludes []string\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"no wildcards\",\n\t\t\tactiveNamespaces: []string{\"ns1\", \"ns2\", \"ns3\"},\n\t\t\tincludes: []string{\"ns1\", \"ns4\"},\n\t\t\texcludes: []string{\"ns2\"},\n\t\t\texpectedIncludes: []string{\"ns1\", \"ns4\"},\n\t\t\texpectedExcludes: []string{\"ns2\"},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard in includes\",\n\t\t\tactiveNamespaces: []string{\"app-prod\", \"app-staging\", \"db-prod\", \"test-ns\"},\n\t\t\tincludes: []string{\"app-*\"},\n\t\t\texcludes: []string{\"test-ns\"},\n\t\t\texpectedIncludes: []string{\"app-prod\", \"app-staging\"},\n\t\t\texpectedExcludes: []string{\"test-ns\"},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard in excludes\",\n\t\t\tactiveNamespaces: []string{\"app-prod\", \"app-staging\", \"db-prod\", \"test-ns\"},\n\t\t\tincludes: []string{\"app-prod\"},\n\t\t\texcludes: []string{\"*-staging\"},\n\t\t\texpectedIncludes: []string{\"app-prod\"},\n\t\t\texpectedExcludes: []string{\"app-staging\"},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"wildcards in both\",\n\t\t\tactiveNamespaces: []string{\"app-prod\", \"app-staging\", \"db-prod\", \"db-staging\", \"test-ns\"},\n\t\t\tincludes: []string{\"*-prod\"},\n\t\t\texcludes: []string{\"*-staging\"},\n\t\t\texpectedIncludes: []string{\"app-prod\", \"db-prod\"},\n\t\t\texpectedExcludes: []string{\"app-staging\", \"db-staging\"},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"star pattern in includes\",\n\t\t\tactiveNamespaces: []string{\"ns1\", \"ns2\", \"ns3\"},\n\t\t\tincludes: []string{\"*\"},\n\t\t\texcludes: []string{},\n\t\t\texpectedIncludes: []string{\"ns1\", \"ns2\", \"ns3\"},\n\t\t\texpectedExcludes: nil,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"empty active namespaces\",\n\t\t\tactiveNamespaces: []string{},\n\t\t\tincludes: []string{\"app-*\"},\n\t\t\texcludes: []string{\"test-*\"},\n\t\t\texpectedIncludes: nil,\n\t\t\texpectedExcludes: nil,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"empty includes and excludes\",\n\t\t\tactiveNamespaces: []string{\"ns1\", \"ns2\"},\n\t\t\tincludes: []string{},\n\t\t\texcludes: []string{},\n\t\t\texpectedIncludes: nil,\n\t\t\texpectedExcludes: nil,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"complex patterns\",\n\t\t\tactiveNamespaces: []string{\"my-app-prod\", \"my-app-staging\", \"your-app-prod\", \"system-ns\"},\n\t\t\tincludes: []string{\"*-app-*\"},\n\t\t\texcludes: []string{\"*-staging\"},\n\t\t\texpectedIncludes: []string{\"my-app-prod\", \"my-app-staging\", \"your-app-prod\"},\n\t\t\texpectedExcludes: []string{\"my-app-staging\"},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"double asterisk should error\",\n\t\t\tactiveNamespaces: []string{\"ns1\", \"ns2\", \"ns3\"},\n\t\t\tincludes: []string{\"**\"},\n\t\t\texcludes: []string{},\n\t\t\texpectedIncludes: nil,\n\t\t\texpectedExcludes: nil,\n\t\t\texpectError: true, // ** is invalid\n\t\t},\n\t\t{\n\t\t\tname: \"double asterisk in pattern should error\",\n\t\t\tactiveNamespaces: []string{\"ns1\", \"ns2\", \"ns3\"},\n\t\t\tincludes: []string{\"app-**\"},\n\t\t\texcludes: []string{},\n\t\t\texpectedIncludes: nil,\n\t\t\texpectedExcludes: nil,\n\t\t\texpectError: true, // app-** contains ** which is invalid\n\t\t},\n\t\t{\n\t\t\tname: \"question mark patterns\",\n\t\t\tactiveNamespaces: []string{\"ns1\", \"ns2\", \"ns10\", \"test\"},\n\t\t\tincludes: []string{\"ns?\"},\n\t\t\texcludes: []string{},\n\t\t\texpectedIncludes: []string{\"ns1\", \"ns2\"}, // ? matches single character\n\t\t\texpectedExcludes: nil,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"character class patterns\",\n\t\t\tactiveNamespaces: []string{\"nsa\", \"nsb\", \"nsc\", \"nsx\", \"ns1\"},\n\t\t\tincludes: []string{\"ns[abc]\"},\n\t\t\texcludes: []string{},\n\t\t\texpectedIncludes: []string{\"nsa\", \"nsb\", \"nsc\"}, // [abc] matches a, b, or c\n\t\t\texpectedExcludes: nil,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"brace alternative patterns\",\n\t\t\tactiveNamespaces: []string{\"app-prod\", \"app-staging\", \"app-dev\", \"db-prod\"},\n\t\t\tincludes: []string{\"app-{prod,staging}\"},\n\t\t\texcludes: []string{},\n\t\t\texpectedIncludes: nil,\n\t\t\texpectedExcludes: nil,\n\t\t\texpectError: true,\n\t\t},\n\t\t{\n\t\t\tname: \"literal dot and plus patterns\",\n\t\t\tactiveNamespaces: []string{\"app.prod\", \"app-prod\", \"app+\", \"app\"},\n\t\t\tincludes: []string{\"app.prod\", \"app+\"},\n\t\t\texcludes: []string{},\n\t\t\texpectedIncludes: []string{\"app.prod\", \"app+\"}, // . and + are literal\n\t\t\texpectedExcludes: nil,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"unsupported regex patterns should error\",\n\t\t\tactiveNamespaces: []string{\"ns1\", \"ns2\"},\n\t\t\tincludes: []string{\"ns(1|2)\"},\n\t\t\texcludes: []string{},\n\t\t\texpectedIncludes: nil,\n\t\t\texpectedExcludes: nil,\n\t\t\texpectError: true, // |, (, ) are not supported\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tincludes, excludes, err := ExpandWildcards(tt.activeNamespaces, tt.includes, tt.excludes)\n\n\t\t\tif tt.expectError {\n\t\t\t\tassert.Error(t, err)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\trequire.NoError(t, err)\n\t\t\tassert.ElementsMatch(t, tt.expectedIncludes, includes)\n\t\t\tassert.ElementsMatch(t, tt.expectedExcludes, excludes)\n\t\t})\n\t}\n}\n\nfunc TestExpandWildcardsPrivate(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpatterns []string\n\t\tactiveNamespaces []string\n\t\texpected []string\n\t\texpectError bool\n\t}{\n\t\t{\n\t\t\tname: \"empty patterns\",\n\t\t\tpatterns: []string{},\n\t\t\tactiveNamespaces: []string{\"ns1\", \"ns2\"},\n\t\t\texpected: nil,\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"non-wildcard patterns\",\n\t\t\tpatterns: []string{\"ns1\", \"ns3\"},\n\t\t\tactiveNamespaces: []string{\"ns1\", \"ns2\"},\n\t\t\texpected: []string{\"ns1\", \"ns3\"}, // includes ns3 even if not in active\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"star pattern\",\n\t\t\tpatterns: []string{\"*\"},\n\t\t\tactiveNamespaces: []string{\"ns1\", \"ns2\", \"ns3\"},\n\t\t\texpected: []string{\"ns1\", \"ns2\", \"ns3\"},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"simple wildcard\",\n\t\t\tpatterns: []string{\"app-*\"},\n\t\t\tactiveNamespaces: []string{\"app-prod\", \"app-staging\", \"db-prod\"},\n\t\t\texpected: []string{\"app-prod\", \"app-staging\"},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"multiple patterns\",\n\t\t\tpatterns: []string{\"app-*\", \"db-prod\", \"*-test\"},\n\t\t\tactiveNamespaces: []string{\"app-prod\", \"app-staging\", \"db-prod\", \"service-test\", \"other\"},\n\t\t\texpected: []string{\"app-prod\", \"app-staging\", \"db-prod\", \"service-test\"},\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"wildcard with no matches\",\n\t\t\tpatterns: []string{\"missing-*\"},\n\t\t\tactiveNamespaces: []string{\"app-prod\", \"db-staging\"},\n\t\t\texpected: []string{}, // returns empty slice, not nil\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"duplicate matches from multiple patterns\",\n\t\t\tpatterns: []string{\"app-*\", \"*-prod\"},\n\t\t\tactiveNamespaces: []string{\"app-prod\", \"app-staging\", \"db-prod\"},\n\t\t\texpected: []string{\"app-prod\", \"app-staging\", \"db-prod\"}, // no duplicates\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"question mark pattern - glob wildcard\",\n\t\t\tpatterns: []string{\"ns?\"},\n\t\t\tactiveNamespaces: []string{\"ns1\", \"ns2\", \"ns10\"},\n\t\t\texpected: []string{\"ns1\", \"ns2\"}, // ? is a glob pattern for single character\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"character class patterns\",\n\t\t\tpatterns: []string{\"ns[12]\"},\n\t\t\tactiveNamespaces: []string{\"ns1\", \"ns2\", \"ns3\", \"nsa\"},\n\t\t\texpected: []string{\"ns1\", \"ns2\"}, // [12] matches 1 or 2\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"character range patterns\",\n\t\t\tpatterns: []string{\"ns[a-c]\"},\n\t\t\tactiveNamespaces: []string{\"nsa\", \"nsb\", \"nsc\", \"nsd\", \"ns1\"},\n\t\t\texpected: []string{\"nsa\", \"nsb\", \"nsc\"}, // [a-c] matches a to c\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"double asterisk should error\",\n\t\t\tpatterns: []string{\"**\"},\n\t\t\tactiveNamespaces: []string{\"app-prod\", \"app.staging\", \"db/prod\"},\n\t\t\texpected: nil,\n\t\t\texpectError: true, // ** is not allowed\n\t\t},\n\t\t{\n\t\t\tname: \"unsupported regex symbols should error\",\n\t\t\tpatterns: []string{\"ns(1|2)\"},\n\t\t\tactiveNamespaces: []string{\"ns1\", \"ns2\"},\n\t\t\texpected: nil,\n\t\t\texpectError: true, // |, (, ) not supported\n\t\t},\n\t\t{\n\t\t\tname: \"double asterisk should error\",\n\t\t\tpatterns: []string{\"**\"},\n\t\t\tactiveNamespaces: []string{\"ns1\", \"ns2\"},\n\t\t\texpected: nil,\n\t\t\texpectError: true, // ** not allowed\n\t\t},\n\t\t{\n\t\t\tname: \"double asterisk in pattern should error\",\n\t\t\tpatterns: []string{\"app-**-prod\"},\n\t\t\tactiveNamespaces: []string{\"app-prod\"},\n\t\t\texpected: nil,\n\t\t\texpectError: true, // ** not allowed anywhere\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\tresult, err := expandWildcards(tt.patterns, tt.activeNamespaces)\n\n\t\t\tif tt.expectError {\n\t\t\t\tassert.Error(t, err)\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\trequire.NoError(t, err)\n\t\t\tif tt.expected == nil {\n\t\t\t\tassert.Nil(t, result)\n\t\t\t} else if len(tt.expected) == 0 {\n\t\t\t\tassert.Empty(t, result)\n\t\t\t} else {\n\t\t\t\tassert.ElementsMatch(t, tt.expected, result)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestValidateBracePatterns(t *testing.T) {\n\ttests := []struct {\n\t\tname string\n\t\tpattern string\n\t\texpectError bool\n\t\terrorMsg string\n\t}{\n\t\t// Valid square bracket patterns\n\t\t{\n\t\t\tname: \"valid square bracket pattern\",\n\t\t\tpattern: \"ns[abc]\",\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"valid square bracket pattern with range\",\n\t\t\tpattern: \"ns[a-z]\",\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"valid square bracket pattern with numbers\",\n\t\t\tpattern: \"ns[0-9]\",\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"valid square bracket pattern with mixed\",\n\t\t\tpattern: \"ns[a-z0-9]\",\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"valid square bracket pattern with single character\",\n\t\t\tpattern: \"ns[a]\",\n\t\t\texpectError: false,\n\t\t},\n\t\t{\n\t\t\tname: \"valid square bracket pattern with text before and after\",\n\t\t\tpattern: \"prefix-[abc]-suffix\",\n\t\t\texpectError: false,\n\t\t},\n\t\t// Unclosed opening brackets\n\t\t{\n\t\t\tname: \"unclosed opening bracket at end\",\n\t\t\tpattern: \"ns[abc\",\n\t\t\texpectError: true,\n\t\t\terrorMsg: \"unclosed bracket\",\n\t\t},\n\t\t{\n\t\t\tname: \"unclosed opening bracket at start\",\n\t\t\tpattern: \"[abc\",\n\t\t\texpectError: true,\n\t\t\terrorMsg: \"unclosed bracket\",\n\t\t},\n\t\t{\n\t\t\tname: \"unclosed opening bracket in middle\",\n\t\t\tpattern: \"ns[abc-test\",\n\t\t\texpectError: true,\n\t\t\terrorMsg: \"unclosed bracket\",\n\t\t},\n\n\t\t// Unmatched closing brackets\n\t\t{\n\t\t\tname: \"unmatched closing bracket at end\",\n\t\t\tpattern: \"ns-abc]\",\n\t\t\texpectError: true,\n\t\t\terrorMsg: \"unmatched closing bracket\",\n\t\t},\n\t\t{\n\t\t\tname: \"unmatched closing bracket at start\",\n\t\t\tpattern: \"]ns-abc\",\n\t\t\texpectError: true,\n\t\t\terrorMsg: \"unmatched closing bracket\",\n\t\t},\n\t\t{\n\t\t\tname: \"unmatched closing bracket in middle\",\n\t\t\tpattern: \"ns-]abc\",\n\t\t\texpectError: true,\n\t\t\terrorMsg: \"unmatched closing bracket\",\n\t\t},\n\t\t{\n\t\t\tname: \"extra closing bracket after valid pair\",\n\t\t\tpattern: \"ns[abc]]\",\n\t\t\texpectError: true,\n\t\t\terrorMsg: \"unmatched closing bracket\",\n\t\t},\n\n\t\t// Empty bracket patterns\n\t\t{\n\t\t\tname: \"completely empty brackets\",\n\t\t\tpattern: \"ns[]\",\n\t\t\texpectError: true,\n\t\t\terrorMsg: \"empty bracket pattern\",\n\t\t},\n\t\t{\n\t\t\tname: \"empty brackets at start\",\n\t\t\tpattern: \"[]ns\",\n\t\t\texpectError: true,\n\t\t\terrorMsg: \"empty bracket pattern\",\n\t\t},\n\t\t{\n\t\t\tname: \"empty brackets standalone\",\n\t\t\tpattern: \"[]\",\n\t\t\texpectError: true,\n\t\t\terrorMsg: \"empty bracket pattern\",\n\t\t},\n\n\t\t// Edge cases\n\t\t{\n\t\t\tname: \"empty pattern\",\n\t\t\tpattern: \"\",\n\t\t\texpectError: false,\n\t\t},\n\t}\n\n\tfor _, tt := range tests {\n\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\terr := validateBracePatterns(tt.pattern)\n\n\t\t\tif tt.expectError {\n\t\t\t\trequire.Error(t, err, \"Expected error for pattern: %s\", tt.pattern)\n\t\t\t\tif tt.errorMsg != \"\" {\n\t\t\t\t\tassert.Contains(t, err.Error(), tt.errorMsg, \"Error message should contain: %s\", tt.errorMsg)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tassert.NoError(t, err, \"Expected no error for pattern: %s\", tt.pattern)\n\t\t\t}\n\t\t})\n\t}\n}\n\n// Edge case tests\nfunc TestExpandWildcardsEdgeCases(t *testing.T) {\n\tt.Run(\"nil inputs\", func(t *testing.T) {\n\t\tincludes, excludes, err := ExpandWildcards(nil, nil, nil)\n\t\trequire.NoError(t, err)\n\t\tassert.Nil(t, includes)\n\t\tassert.Nil(t, excludes)\n\t})\n\n\tt.Run(\"empty string patterns\", func(t *testing.T) {\n\t\tactiveNamespaces := []string{\"ns1\", \"ns2\"}\n\t\tresult, err := expandWildcards([]string{\"\"}, activeNamespaces)\n\t\trequire.NoError(t, err)\n\t\tassert.ElementsMatch(t, []string{\"\"}, result) // empty string is treated as literal\n\t})\n\n\tt.Run(\"whitespace patterns\", func(t *testing.T) {\n\t\tactiveNamespaces := []string{\"ns1\", \" \", \"ns2\"}\n\t\tresult, err := expandWildcards([]string{\" \"}, activeNamespaces)\n\t\trequire.NoError(t, err)\n\t\tassert.ElementsMatch(t, []string{\" \"}, result)\n\t})\n\n\tt.Run(\"special characters in namespace names\", func(t *testing.T) {\n\t\tactiveNamespaces := []string{\"ns-1\", \"ns_2\", \"ns.3\", \"ns@4\"}\n\t\tresult, err := expandWildcards([]string{\"ns*\"}, activeNamespaces)\n\t\trequire.NoError(t, err)\n\t\tassert.ElementsMatch(t, []string{\"ns-1\", \"ns_2\", \"ns.3\", \"ns@4\"}, result)\n\t})\n\n\tt.Run(\"mixed literal and wildcard patterns\", func(t *testing.T) {\n\t\tactiveNamespaces := []string{\"app.prod\", \"app-prod\", \"app_prod\", \"test.ns\"}\n\t\tresult, err := expandWildcards([]string{\"app.prod\", \"app?prod\"}, activeNamespaces)\n\t\trequire.NoError(t, err)\n\t\tassert.ElementsMatch(t, []string{\"app.prod\", \"app-prod\", \"app_prod\"}, result)\n\t})\n\n\tt.Run(\"conservative asterisk validation\", func(t *testing.T) {\n\t\ttests := []struct {\n\t\t\tname string\n\t\t\tpattern string\n\t\t\tshouldError bool\n\t\t}{\n\t\t\t{\"single asterisk\", \"*\", false},\n\t\t\t{\"double asterisk\", \"**\", true},\n\t\t\t{\"triple asterisk\", \"***\", true},\n\t\t\t{\"quadruple asterisk\", \"****\", true},\n\t\t\t{\"mixed with double\", \"app-**\", true},\n\t\t\t{\"double in middle\", \"app-**-prod\", true},\n\t\t}\n\n\t\tfor _, tt := range tests {\n\t\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\t\t_, err := expandWildcards([]string{tt.pattern}, []string{\"test\"})\n\t\t\t\tif tt.shouldError {\n\t\t\t\t\tassert.Error(t, err)\n\t\t\t\t} else {\n\t\t\t\t\tassert.NoError(t, err)\n\t\t\t\t}\n\t\t\t})\n\t\t}\n\t})\n\n\tt.Run(\"malformed pattern validation\", func(t *testing.T) {\n\t\ttests := []struct {\n\t\t\tname string\n\t\t\tpattern string\n\t\t\tshouldError bool\n\t\t}{\n\t\t\t{\"unclosed bracket\", \"ns[abc\", true},\n\t\t\t{\"valid bracket\", \"ns[abc]\", false},\n\t\t\t{\"empty bracket\", \"ns[]\", true}, // empty brackets are invalid\n\t\t}\n\n\t\tfor _, tt := range tests {\n\t\t\tt.Run(tt.name, func(t *testing.T) {\n\t\t\t\t_, err := expandWildcards([]string{tt.pattern}, []string{\"test\"})\n\t\t\t\tif tt.shouldError {\n\t\t\t\t\tassert.Error(t, err, \"Expected error for pattern: %s\", tt.pattern)\n\t\t\t\t} else {\n\t\t\t\t\tassert.NoError(t, err, \"Expected no error for pattern: %s\", tt.pattern)\n\t\t\t\t}\n\t\t\t})\n\t\t}\n\t})\n}\n" }, { "path": "restore-hooks_product-requirements.md", "content": "Velero Restore Hooks - PRD (Product Requirements Document) \n\nMVP Feature Set\n\nRelates to: [https://github.com/vmware-tanzu/velero/issues/2116](https://github.com/vmware-tanzu/velero/issues/2116)\n\n\n# **Change tracking**\n\n_This is a live document, you can reach me on the following channels for more information or for any questions:_\n\n\n\n* _Email: [bstephanie@vmware.com](mailto:bstephanie@vmware.com)_\n* _Kubernetes slack: _\n * _Channel: #velero_\n * _@Stephanie Bauman_\n* _VMware internal slack:_\n * _@bstephanie_\n\n\n# **Relates to Git Issues:**\n\n\n\n* [https://app.zenhub.com/workspaces/velero-5c59c15e39d47b774b5864e3/issues/vmware-tanzu/velero/2465](https://app.zenhub.com/workspaces/velero-5c59c15e39d47b774b5864e3/issues/vmware-tanzu/velero/2465)\n * [https://github.com/vmware-tanzu/velero/pull/2465/files?short_path=140e0c6#diff-140e0c6b370f250ee97f6ecafc8dbb7a](https://github.com/vmware-tanzu/velero/pull/2465/files?short_path=140e0c6#diff-140e0c6b370f250ee97f6ecafc8dbb7a)\n* [https://github.com/vmware-tanzu/velero/issues/1150](https://github.com/vmware-tanzu/velero/issues/1150)\n\nBackground\n\nVelero supports restore operations but there are gaps in the process. Gaps in the restore process require users to manually carry out steps to start, clean up, and end the restore process. Other gaps in the restore process can cause issues with application performance for applications running in a pod when a restore operation is carried out. \n\nOn a restore, Velero currently does not include hooks to execute a pre- or -post restore script. As a result, users are required to perform additional actions following a velero restore operation. Some gaps that currently exist in the Velero restore process are:\n\n\n\n* Users can create a restore operation but has no option to customize or automate commands during the start of the restore operation\n* Users can perform post restore operations but have no option to customize or automate commands during the end of the restore operation\n\nStrategic Fit\n\nAdding a restore hook action today would allow Velero to unpack the data that was backed up in an automated way by enabling Velero to execute commands in containers during a restore. This will also improve the restore operations on a container and mitigate against any negative performance impacts of apps running in the container during restore.\n\nPurpose / Goal\n\nThe purpose of this feature is to improve the extensibility and user experience of pre and post restore operations for Velero users.\n\nGoals for this feature include:\n\n\n\n* Enhance application performance during and following restore events \n* Provide pre-restore hooks for customizing start restore operations\n* Provide actions for things like retry actions, event logging, etc... during restore operations \n* Provide observability/status of restore commands run in restored pods\n* Extend restore logs to include status, error codes and necessary metadata for restore commands run during restore operations for enhanced troubleshooting capabilities \n* Provide post-restore hooks for customizing end of restore operations\n* Include pre-populated data that has been serialized by a Velero backup hook into a container or external service prior to allowing a restore to be marked as completed. \n\nNon-goals\n\nFeature Description\n\nThis feature will automate the restore operations/processes in Velero, and will provide restore hook actions in Velero that allows users to execute restore commands on a container. Restore hooks include pre-hook actions and post-hook actions.\n\nThis feature will mirror the actions of a Velero backup by allowing Velero to check for any restore hooks specified for a pod. \n\nAssumptions\n\n\n\n* Restore operations will be run at the pod level instead of at the volume level. Some databases require the pod to be running and in some cases a user cannot manipulate a volume without the pod running. We need to support more than one type of database for this feature and so we need to ensure that this works broadly as opposed to providing support only for specific dbs. \n* Velero will be responsible for invoking the restore hook. \n\n \n\nMVP Use Cases \n\nThe following use cases must be included as part of the Velero restore hooks MVP (minimum viable product). \n\n**Note: **Processing of concurrent vs sequential workloads is slated later in the Velero roadmap (see [https://github.com/vmware-tanzu/velero/pull/2548/files](https://github.com/vmware-tanzu/velero/pull/2548/files)). The MVP for this feature set will align with restore of single workloads vs concurrent workload restores. A second epic will be created to address the concurrent restore operations and will be added to the backlog for priority visibility. \n\n**Note: **Please refer to the Requirements section of this document for more details on what items are P0 (must have in the MVP), P1 (should not ship without for the MVP), P2 (nice to haves).\n\n**USE CASE 1**\n\n\n **Title:** Run restore hook before pod restart.\n\n\n **Description: **As a user, I would like to run a restore hook before the applications in my pod are restarted.\n\n**______________________________________________________________**\n\n**USE CASE 2**\n\n\n **Title: **Allow restore hook to run on non-kubernetes databases\n\n\n **Description: **As a user, I would like to run restore hook operations even on databases that are external to Kubernetes (such as postgres, elastic, etc…).\n\n**______________________________________________________________**\n\n**USE CASE 3**\n\n\n **Title: **Run restore at pod level.\n\n\n **Description: **As a user, I would like to make sure that I can run the restore hook at the pod level. And, I would like the option to run this with an annotation flag in line or using an init container.** **The restore pre-hook should allow the user to run the command on the container where the pre-hook should be executed. Similar to the backup hooks, this hook should run to default to run on the first container in the pod. \n\n**______________________________________________________________**\n\n**USE CASE 4**\n\n\n **Title:** Specify container in which to run pre-hook\n\n\n **Description: **As a user, if I do not want to run the pre-hook command on the first container in the pod (default container), I would like the option to annotate the specific container that i would like the hook to run in.\n\n**______________________________________________________________**\n\n**USE CASE 5**\n\n\n **Title: **Check for latest snapshot \n\n\n **Description: **As a user, I would like Velero to run a check for the latest snapshot in object storage prior to starting restore operations on a pod. \n\n**______________________________________________________________**\n\n**USE CASE 6**\n\n\n **Title: **Display/surface output from restore hooks/restore status \n\n\n **Description:** As a user, I would like to see the output of the restore hooks/status of my restore surfaced from the pod volume restore status. Including statuses: Pending, Running/In Progress, Succeeded, Failed, Unknown.\n\n**______________________________________________________________**\n\n**USE CASE 7**\n\n\n **Title: **Restore metadata\n\n\n **Description: **As a user, I would like to have the metadata of the contents of what was restored using Velero.\n\n\n _Note: Kubernetes patterns may result in some snapshot metadata being overwritten during restore operations._\n\n**______________________________________________________________**\n\n**USE CASE 8*\n\n **Title: **Increase default restore and retry limits. \n\n\n **Description: **As a user, I would like to increase the default restore retry and timeout limits from the default values to some other value I would like to specify. \n\n\n _Note: See use case 11 for the default value specifications. _\n\n**______________________________________________________________**\n\nUser Experience\n\nThe following is representative of what the user experience could look like for Velero restore pre-hooks and post-hooks. \n\n**_Note: These examples are representative and are not to be considered for use in pre- and post- restore hook operations until the technical design is complete._**\n\n**Restore Pre-Hooks**\n\nContainer Command\n\n\n```\npre.hook.restore.velero.io/container\n\nkubectl patch backupstoragelocation \\\n --namespace velero \\\n --type merge \\\n --patch '{\"spec\":{\"accessMode\":\"ReadOnly\"}}'\n```\n\n\nCommand Execute\n\n\n```\npre.hook.restore.velero.io/command\n```\n\n\nIncludes commands for:\n\n\n\n* Create\n * Create from most recent backup\n * Create from specific backup - allow user to list backups\n* Set backup storage location to read only \n\n ```\n kubectl patch backupstoragelocation \\\n --namespace velero \\\n --type merge \\\n --patch '{\"spec\":{\"accessMode\":\"ReadOnly\"}}\n ```\n\n\n* Set backup storage location to read-write\n\n ```\n kubectl patch backupstoragelocation \\\n --namespace velero \\\n --type merge \\\n --patch '{\"spec\":{\"accessMode\":\"ReadWrite\"}}'\n\n ```\n\n\nError handling \n\n\n```\npre.hook.restore.velero.io/on-error\n```\n\n\nTimeout \n\n\n```\npre.hook.restore.velero.io/retry\n```\n\n\nRequirements \n\n**_P0_** = must not ship without \n\n\n (absolute requirement for MVP, engineering requirements for long term viability usually fall in here for ex., and incompletion nearing or by deadline means delaying code freeze and GA date)\n\n**_P1_** = should not ship without \n\n\n (required for feature to achieve general adoption, should be scoped into feature but can be pushed back to later iterations if needed)\n\n**_P2_** = nice to have \n\n\n (opportunistic, should not be scoped into the overall feature if it has dependencies and could cause delay)\n\n**P0 Requirements**\nP0. Use Case 1 - Run restore hook before pod restart.\n\nP0. Use Case 3- Run restore at pod level.\n\nP0. Use Case 5 - Check for latest snapshot\n\nP0. Use Case 9 - ** **Display/surface restore status \n\nP0. Use Case 10 - Restore metadata \n\nP0. Use Case 11 - Retry restore upon restore failure/error/timeout\n\nP0. Use Case 12 - Increase default restore and retry limits. \n\n**P1 Requirements**\nP1. Use Case 2 - Allow restore hook to run on non-kubernetes databases\n\nP1. Use Case 4 - Specify container in which to run pre-hook\n\nP1. Use Case 6 - Specify backup snapshot to use for restore\n\nP1. Use Case 7 - ** **Include or exclude namespaces from restore\n\n**P2 Requirements**\nP2. \n\n**Out of scope**\n\nThe following requirements are out of scope for the Velero Restore Hooks MVP:\n\n\n1. Verifying the integrity of a backup, resource, or other artifact will not be included in the scope of this effort. \n2. Verifying the integrity of a snapshot using Kubernetes hash checks.\n3. Running concurrent restore operations (for the MVP) a secondary epic will be opened to align better with the concurrent workload operations currently set on the Velero roadmap for Q4 timeframe. \n\n**Questions** \n\n\n\n1. For USE CASE 1: Init vs app containers - if multiple containers are specified for a pod kubelet will run each init container sequentially - does this have an impact on things like concurrent workload processing? \n2. Can velero allow a user to specify a specific backup if the most recent backup is not desired in a restore?\n3. If a backup specified for a restore operation fails, can velero retry and pick up the next most recent backup in the restore?\n4. Can velero provide a delta between the two backups if a different backup needs to be picked up (other than the most recent because the most recent backup cannot be accessed?)\n5. What types of errors can velero surface about backups, namespaces, pods, resources, if a backup has an issue with it preventing a restore from being done? \n\n \n\nFor questions, please contact michaelmi@vmware.com, [bstephanie@vmware.com](mailto:bstephanie@vmware.com) \n" }, { "path": "site/Dockerfile", "content": "FROM klakegg/hugo:0.73.0-ext-ubuntu\n\nWORKDIR /srv/hugo\n\nEXPOSE 1313\n" }, { "path": "site/README-HUGO.md", "content": "# Running in Docker\n\nTo run this site in a Docker container, you can use `make serve-docs` from the root directory.\n\n# Dependencies for MacOS\n\nInstall the following for an easy to use dev environment:\n\n* `brew install hugo`\n\n# Dependencies for Linux\nIf you are running a build on Ubuntu you will need the following packages:\n* hugo\n\n\n# Local Development\n1. Clone down your own fork, or clone the main repo `git clone https://github.com/vmware-tanzu/velero` and add your own remote.\n1. `cd velero/site`\n1. Serve the site and watch for markup/sass changes `hugo serve`.\n1. View your website at http://127.0.0.1:1313/\n1. Commit any changes and push everything to your fork.\n1. Once you're ready, submit a PR of your changes. Netlify will automatically generate a preview of your changes.\n\n# Jetbrains IDE setup (IntelliJ, Goland, etc)\n1. Install the `Hugo Integration` plugin: https://plugins.jetbrains.com/plugin/13215-hugo-integration\n - Under `Preferences...` -> `Plugins`\n1. Create a new configuration:\n - Click `Edit Configurations...`\n - Click the `+` button to create a new configuration and select `Hugo`\n - Select `hugo serve` and make sure it is running under the `site` directory\n - Save and run the new Configuration\n - View your website at http://127.0.0.1:1313/\n - Any changes in `site` will reload the website automatically\n\n# Adding a New Docs Version\n\nTo add a new set of versioned docs to go with a new Velero release:\n\n1. In the root of the repository, run:\n\n ```bash\n # set to the appropriate version numbers\n NEW_DOCS_VERSION=vX.Y VELERO_VERSION=vX.Y.Z make gen-docs\n ```\n\n1. [Pre-release only] In `site/config.yaml`, revert the change to the `latest` field, so the pre-release docs do not become the default.\n" }, { "path": "site/assets/_scss/_styles.scss", "content": "@import \"./site/settings/variables\";\n\n// 3rd party\n@import \"./bootstrap-4.1.3/bootstrap.scss\";\n\n// Common\n@import \"./site/common/fonts\";\n@import \"./site/common/core\";\n@import \"./site/common/type\";\n\n// Layouts\n@import \"./site/layouts/container\";\n@import \"./site/layouts/documentation\";\n@import \"./site/layouts/docsearch\";\n\n// Objects\n@import \"./site/objects/header\";\n@import \"./site/objects/footer\";\n@import \"./site/objects/section\";\n@import \"./site/objects/home-hero\";\n@import \"./site/objects/card\";\n@import \"./site/objects/alternating-cards\";\n@import \"./site/objects/button\";\n@import \"./site/objects/thumbnail-grid\";\n@import \"./site/objects/post\";\n\n// Utilities\n@import \"./site/utilities/image\";\n@import \"./site/utilities/type\";\n" }, { "path": "site/assets/_scss/bootstrap-4.1.3/_alert.scss", "content": "//\n// Base styles\n//\n\n.alert {\n position: relative;\n padding: $alert-padding-y $alert-padding-x;\n margin-bottom: $alert-margin-bottom;\n border: $alert-border-width solid transparent;\n @include border-radius($alert-border-radius);\n}\n\n// Headings for larger alerts\n.alert-heading {\n // Specified to prevent conflicts of changing $headings-color\n color: inherit;\n}\n\n// Provide class for links that match alerts\n.alert-link {\n font-weight: $alert-link-font-weight;\n}\n\n\n// Dismissible alerts\n//\n// Expand the right padding and account for the close button's positioning.\n\n.alert-dismissible {\n padding-right: ($close-font-size + $alert-padding-x * 2);\n\n // Adjust close link position\n .close {\n position: absolute;\n top: 0;\n right: 0;\n padding: $alert-padding-y $alert-padding-x;\n color: inherit;\n }\n}\n\n\n// Alternate styles\n//\n// Generate contextual modifier classes for colorizing the alert.\n\n@each $color, $value in $theme-colors {\n .alert-#{$color} {\n @include alert-variant(theme-color-level($color, $alert-bg-level), theme-color-level($color, $alert-border-level), theme-color-level($color, $alert-color-level));\n }\n}\n" }, { "path": "site/assets/_scss/bootstrap-4.1.3/_badge.scss", "content": "// Base class\n//\n// Requires one of the contextual, color modifier classes for `color` and\n// `background-color`.\n\n.badge {\n display: inline-block;\n padding: $badge-padding-y $badge-padding-x;\n font-size: $badge-font-size;\n font-weight: $badge-font-weight;\n line-height: 1;\n text-align: center;\n white-space: nowrap;\n vertical-align: baseline;\n @include border-radius($badge-border-radius);\n\n // Empty badges collapse automatically\n &:empty {\n display: none;\n }\n}\n\n// Quick fix for badges in buttons\n.btn .badge {\n position: relative;\n top: -1px;\n}\n\n// Pill badges\n//\n// Make them extra rounded with a modifier to replace v3's badges.\n\n.badge-pill {\n padding-right: $badge-pill-padding-x;\n padding-left: $badge-pill-padding-x;\n @include border-radius($badge-pill-border-radius);\n}\n\n// Colors\n//\n// Contextual variations (linked badges get darker on :hover).\n\n@each $color, $value in $theme-colors {\n .badge-#{$color} {\n @include badge-variant($value);\n }\n}\n" }, { "path": "site/assets/_scss/bootstrap-4.1.3/_breadcrumb.scss", "content": ".breadcrumb {\n display: flex;\n flex-wrap: wrap;\n padding: $breadcrumb-padding-y $breadcrumb-padding-x;\n margin-bottom: $breadcrumb-margin-bottom;\n list-style: none;\n background-color: $breadcrumb-bg;\n @include border-radius($breadcrumb-border-radius);\n}\n\n.breadcrumb-item {\n // The separator between breadcrumbs (by default, a forward-slash: \"/\")\n + .breadcrumb-item {\n padding-left: $breadcrumb-item-padding;\n\n &::before {\n display: inline-block; // Suppress underlining of the separator in modern browsers\n padding-right: $breadcrumb-item-padding;\n color: $breadcrumb-divider-color;\n content: $breadcrumb-divider;\n }\n }\n\n // IE9-11 hack to properly handle hyperlink underlines for breadcrumbs built\n // without `